2024-04-24T23:58:29.479 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T23:58:29.479 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsstress}} --owner scheduled_teuthology@teuthology --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672239 --verbose -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672239/orig.config.yaml 2024-04-24T23:58:29.539 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T23:58:29.635 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672239 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsstress}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672239' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi107.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA0mxxTq8WG9Rcr14taJrxZkJCNH4BKjKd2tIDcFMaa67Ckiw7fnWC+bsaxAuAZLeB36PARVD13ZM/rrmNBpoCA= smithi170.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFEMdl9brMdRjWgUhHKHr7I0EMJ1Lwvc9X/qUJ1I4ngPdn1fR3AkarY+IQJl/HiXebbVGx6J8Oq8yXpNV9oVNpg= smithi195.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHYZG8Q2djo6KhFDdJ2UdCOfYD+N+IB64KxSiMem2SiJnd3SIfGo0BL2332zPSpW7GM7/UkldTyhXJ+lTNI793I= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/fsstress.sh timeout: 6h teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-24T23:58:29.635 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-24T23:58:29.636 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-24T23:58:29.636 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T23:58:29.638 INFO:teuthology.task.internal:Checking packages... 2024-04-24T23:58:29.659 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-24T23:58:29.659 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T23:58:29.659 INFO:teuthology.packaging:ref: None 2024-04-24T23:58:29.659 INFO:teuthology.packaging:tag: None 2024-04-24T23:58:29.659 INFO:teuthology.packaging:branch: squid 2024-04-24T23:58:29.659 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:58:29.659 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-24T23:58:29.961 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445.gd2c45d97 2024-04-24T23:58:29.964 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T23:58:29.971 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T23:58:29.971 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T23:58:29.978 INFO:teuthology.task.internal:Saving configuration 2024-04-24T23:58:30.001 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T23:58:30.008 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T23:58:30.026 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672239', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:53:19.921543', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA0mxxTq8WG9Rcr14taJrxZkJCNH4BKjKd2tIDcFMaa67Ckiw7fnWC+bsaxAuAZLeB36PARVD13ZM/rrmNBpoCA='} 2024-04-24T23:58:30.045 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672239', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:53:19.919708', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFEMdl9brMdRjWgUhHKHr7I0EMJ1Lwvc9X/qUJ1I4ngPdn1fR3AkarY+IQJl/HiXebbVGx6J8Oq8yXpNV9oVNpg='} 2024-04-24T23:58:30.065 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672239', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:53:19.923228', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHYZG8Q2djo6KhFDdJ2UdCOfYD+N+IB64KxSiMem2SiJnd3SIfGo0BL2332zPSpW7GM7/UkldTyhXJ+lTNI793I='} 2024-04-24T23:58:30.065 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T23:58:30.072 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-24T23:58:30.072 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-24T23:58:30.072 INFO:teuthology.task.internal:roles: ubuntu@smithi195.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-24T23:58:30.072 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T23:58:30.169 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa6b7d7a160>, signals=[15]) 2024-04-24T23:58:30.170 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T23:58:30.175 INFO:teuthology.task.internal:Opening connections... 2024-04-24T23:58:30.176 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2024-04-24T23:58:30.177 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:58:30.254 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi170.front.sepia.ceph.com 2024-04-24T23:58:30.255 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:58:30.325 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi195.front.sepia.ceph.com 2024-04-24T23:58:30.325 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:58:30.398 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T23:58:30.405 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2024-04-24T23:58:30.422 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2024-04-24T23:58:30.422 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2024-04-24T23:58:30.479 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Stream" 2024-04-24T23:58:30.480 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="9" 2024-04-24T23:58:30.480 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2024-04-24T23:58:30.480 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2024-04-24T23:58:30.480 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="9" 2024-04-24T23:58:30.480 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el9" 2024-04-24T23:58:30.480 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T23:58:30.480 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2024-04-24T23:58:30.480 INFO:teuthology.orchestra.run.smithi107.stdout:LOGO="fedora-logo-icon" 2024-04-24T23:58:30.480 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T23:58:30.480 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://centos.org/" 2024-04-24T23:58:30.480 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T23:58:30.480 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T23:58:30.480 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T23:58:30.481 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2024-04-24T23:58:30.511 DEBUG:teuthology.orchestra.run.smithi170:> uname -m 2024-04-24T23:58:30.528 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2024-04-24T23:58:30.529 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2024-04-24T23:58:30.585 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="CentOS Stream" 2024-04-24T23:58:30.585 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="9" 2024-04-24T23:58:30.585 INFO:teuthology.orchestra.run.smithi170.stdout:ID="centos" 2024-04-24T23:58:30.585 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="rhel fedora" 2024-04-24T23:58:30.585 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="9" 2024-04-24T23:58:30.586 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el9" 2024-04-24T23:58:30.586 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T23:58:30.586 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31" 2024-04-24T23:58:30.586 INFO:teuthology.orchestra.run.smithi170.stdout:LOGO="fedora-logo-icon" 2024-04-24T23:58:30.586 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T23:58:30.586 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://centos.org/" 2024-04-24T23:58:30.586 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T23:58:30.586 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T23:58:30.586 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T23:58:30.586 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2024-04-24T23:58:30.610 DEBUG:teuthology.orchestra.run.smithi195:> uname -m 2024-04-24T23:58:30.628 INFO:teuthology.orchestra.run.smithi195.stdout:x86_64 2024-04-24T23:58:30.628 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release 2024-04-24T23:58:30.685 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="CentOS Stream" 2024-04-24T23:58:30.685 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="9" 2024-04-24T23:58:30.685 INFO:teuthology.orchestra.run.smithi195.stdout:ID="centos" 2024-04-24T23:58:30.685 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE="rhel fedora" 2024-04-24T23:58:30.686 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="9" 2024-04-24T23:58:30.686 INFO:teuthology.orchestra.run.smithi195.stdout:PLATFORM_ID="platform:el9" 2024-04-24T23:58:30.686 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T23:58:30.686 INFO:teuthology.orchestra.run.smithi195.stdout:ANSI_COLOR="0;31" 2024-04-24T23:58:30.686 INFO:teuthology.orchestra.run.smithi195.stdout:LOGO="fedora-logo-icon" 2024-04-24T23:58:30.686 INFO:teuthology.orchestra.run.smithi195.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T23:58:30.686 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://centos.org/" 2024-04-24T23:58:30.686 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T23:58:30.686 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T23:58:30.686 INFO:teuthology.orchestra.run.smithi195.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T23:58:30.686 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server 2024-04-24T23:58:30.710 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T23:58:30.720 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T23:58:30.726 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T23:58:30.726 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2024-04-24T23:58:30.728 DEBUG:teuthology.orchestra.run.smithi170:> test '!' -e /home/ubuntu/cephtest 2024-04-24T23:58:30.731 DEBUG:teuthology.orchestra.run.smithi195:> test '!' -e /home/ubuntu/cephtest 2024-04-24T23:58:30.747 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T23:58:30.753 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T23:58:30.753 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2024-04-24T23:58:30.786 DEBUG:teuthology.orchestra.run.smithi170:> test -z $(ls -A /var/lib/ceph) 2024-04-24T23:58:30.788 DEBUG:teuthology.orchestra.run.smithi195:> test -z $(ls -A /var/lib/ceph) 2024-04-24T23:58:30.882 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T23:58:30.948 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T23:58:30.962 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-04-24T23:58:30.962 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-24T23:58:30.962 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-24T23:58:30.963 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T23:58:30.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:58:30.979 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-04-24T23:58:31.037 INFO:teuthology.orchestra.run.smithi107.stdout:5.14.0-437.el9.x86_64 2024-04-24T23:58:31.037 INFO:teuthology.task.kernel:Running kernel on smithi107: 5.14.0-437.el9.x86_64 2024-04-24T23:58:31.037 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum install -y kernel 2024-04-24T23:58:37.906 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 1.5 MB/s | 8.0 MB 00:05 2024-04-24T23:58:39.582 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 30 MB/s | 19 MB 00:00 2024-04-24T23:58:43.783 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.2 MB 00:00 2024-04-24T23:58:45.675 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-04-24T23:58:46.818 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-24T23:58:51.851 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-24T23:58:53.765 INFO:teuthology.orchestra.run.smithi107.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-24T23:58:53.765 INFO:teuthology.orchestra.run.smithi107.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-24T23:58:53.854 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-24T23:58:53.855 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-04-24T23:58:53.855 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-24T23:58:53.970 DEBUG:teuthology.orchestra.run.smithi107:> echo no | sudo yum reinstall kernel || true 2024-04-24T23:58:54.469 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 11:58:51 PM UTC. 2024-04-24T23:58:54.600 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-24T23:58:54.607 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-24T23:58:54.607 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-04-24T23:58:54.607 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-24T23:58:54.607 INFO:teuthology.orchestra.run.smithi107.stdout:Reinstalling: 2024-04-24T23:58:54.607 INFO:teuthology.orchestra.run.smithi107.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T23:58:54.607 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-24T23:58:54.607 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-24T23:58:54.607 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-24T23:58:54.607 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-24T23:58:54.607 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 255 k 2024-04-24T23:58:54.607 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 0 2024-04-24T23:58:54.608 INFO:teuthology.orchestra.run.smithi107.stderr:Operation aborted. 2024-04-24T23:58:54.671 INFO:teuthology.orchestra.run.smithi107.stdout:Is this ok [y/N]: 2024-04-24T23:58:54.671 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum reinstall -y kernel || true 2024-04-24T23:58:55.160 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:04 ago on Wed 24 Apr 2024 11:58:51 PM UTC. 2024-04-24T23:58:55.275 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-24T23:58:55.276 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-24T23:58:55.276 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-04-24T23:58:55.276 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-24T23:58:55.276 INFO:teuthology.orchestra.run.smithi107.stdout:Reinstalling: 2024-04-24T23:58:55.276 INFO:teuthology.orchestra.run.smithi107.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-24T23:58:55.276 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-24T23:58:55.276 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-24T23:58:55.276 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-24T23:58:55.276 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-24T23:58:55.276 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 255 k 2024-04-24T23:58:55.276 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 0 2024-04-24T23:58:55.276 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-04-24T23:58:55.519 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.4 MB/s | 255 kB 00:00 2024-04-24T23:58:55.519 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-24T23:58:55.519 INFO:teuthology.orchestra.run.smithi107.stdout:Total 1.0 MB/s | 255 kB 00:00 2024-04-24T23:58:55.540 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-04-24T23:58:55.730 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-04-24T23:58:55.730 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-04-24T23:58:55.735 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-04-24T23:58:55.735 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-04-24T23:58:55.867 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-04-24T23:58:55.937 INFO:teuthology.orchestra.run.smithi107.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T23:58:55.937 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:58:56.669 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:58:56.669 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-24T23:58:57.377 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-24T23:58:57.377 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-24T23:58:57.377 INFO:teuthology.orchestra.run.smithi107.stdout:Reinstalled: 2024-04-24T23:58:57.377 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-24T23:58:57.377 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-24T23:58:57.377 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-24T23:58:57.572 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T23:58:57.617 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-24T23:58:57.618 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-24T23:58:57.618 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-24T23:58:57.618 DEBUG:teuthology.parallel:result is None 2024-04-24T23:58:57.618 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T23:58:57.629 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T23:58:57.629 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T23:58:57.632 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T23:58:57.634 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T23:58:57.653 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T23:58:57.660 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T23:58:57.667 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T23:58:57.667 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T23:58:57.686 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T23:58:57.692 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T23:58:57.805 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T23:58:57.813 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T23:58:57.813 DEBUG:teuthology.orchestra.run.smithi170:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T23:58:57.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:58:57.829 DEBUG:teuthology.orchestra.run.smithi195:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T23:58:57.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:58:57.846 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T23:58:57.848 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T23:58:57.873 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T23:58:57.878 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:58:57.893 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:58:57.902 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:58:57.928 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:58:57.936 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:58:57.946 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:58:57.948 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T23:58:57.955 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T23:58:57.955 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T23:58:57.958 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T23:58:57.981 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T23:58:58.014 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T23:58:58.024 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T23:58:58.024 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T23:58:58.027 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T23:58:58.046 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T23:58:58.071 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:58:58.124 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:58:58.179 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:58:58.262 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:58:58.299 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-24T23:58:58.299 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T23:58:58.364 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:58:58.408 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:58:58.455 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:58:58.539 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:58:58.581 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-04-24T23:58:58.581 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T23:58:58.646 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:58:58.691 DEBUG:teuthology.orchestra.run.smithi195:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:58:58.747 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:58:58.829 DEBUG:teuthology.orchestra.run.smithi195:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:58:58.875 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-04-24T23:58:58.875 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T23:58:58.943 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2024-04-24T23:58:58.946 DEBUG:teuthology.orchestra.run.smithi170:> sudo service rsyslog restart 2024-04-24T23:58:58.948 DEBUG:teuthology.orchestra.run.smithi195:> sudo service rsyslog restart 2024-04-24T23:58:58.980 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:58:58.984 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:58:59.022 INFO:teuthology.orchestra.run.smithi195.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:58:59.472 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T23:58:59.480 INFO:teuthology.task.internal:Starting timer... 2024-04-24T23:58:59.480 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T23:58:59.590 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T23:58:59.600 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-24T23:58:59.600 DEBUG:teuthology.orchestra.run.smithi107:> sudo service auditd rotate 2024-04-24T23:58:59.658 INFO:teuthology.orchestra.run.smithi107.stdout:Rotating logs: 2024-04-24T23:58:59.660 DEBUG:teuthology.orchestra.run.smithi170:> sudo service auditd rotate 2024-04-24T23:58:59.715 INFO:teuthology.orchestra.run.smithi170.stdout:Rotating logs: 2024-04-24T23:58:59.717 DEBUG:teuthology.orchestra.run.smithi195:> sudo service auditd rotate 2024-04-24T23:58:59.790 INFO:teuthology.orchestra.run.smithi195.stdout:Rotating logs: 2024-04-24T23:58:59.792 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T23:58:59.792 DEBUG:teuthology.orchestra.run.smithi107:> /usr/sbin/getenforce 2024-04-24T23:58:59.812 INFO:teuthology.orchestra.run.smithi107.stdout:Permissive 2024-04-24T23:58:59.812 DEBUG:teuthology.orchestra.run.smithi170:> /usr/sbin/getenforce 2024-04-24T23:58:59.837 INFO:teuthology.orchestra.run.smithi170.stdout:Permissive 2024-04-24T23:58:59.838 DEBUG:teuthology.orchestra.run.smithi195:> /usr/sbin/getenforce 2024-04-24T23:58:59.882 INFO:teuthology.orchestra.run.smithi195.stdout:Permissive 2024-04-24T23:58:59.883 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi107.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi170.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi195.front.sepia.ceph.com': 'permissive'} 2024-04-24T23:58:59.883 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:58:59.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:58:59.907 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:58:59.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:58:59.935 DEBUG:teuthology.orchestra.run.smithi195:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:58:59.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:58:59.962 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T23:58:59.962 DEBUG:teuthology.orchestra.run.smithi107:> sudo /usr/sbin/setenforce permissive 2024-04-24T23:58:59.986 DEBUG:teuthology.orchestra.run.smithi170:> sudo /usr/sbin/setenforce permissive 2024-04-24T23:59:00.015 DEBUG:teuthology.orchestra.run.smithi195:> sudo /usr/sbin/setenforce permissive 2024-04-24T23:59:00.040 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T23:59:00.050 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T23:59:00.070 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T23:59:00.072 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi107.front.sepia.ceph.com,smithi170.front.sepia.ceph.com,smithi195.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T00:03:15.043 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi107.front.sepia.ceph.com'), Remote(name='ubuntu@smithi170.front.sepia.ceph.com'), Remote(name='ubuntu@smithi195.front.sepia.ceph.com')] 2024-04-25T00:03:15.045 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-04-25T00:03:15.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:03:15.123 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-04-25T00:03:15.200 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-04-25T00:03:15.201 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-04-25T00:03:15.201 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:03:15.274 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-04-25T00:03:15.356 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-04-25T00:03:15.357 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-04-25T00:03:15.357 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:03:15.429 DEBUG:teuthology.orchestra.run.smithi195:> true 2024-04-25T00:03:15.505 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-04-25T00:03:15.505 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T00:03:15.546 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T00:03:15.546 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T00:03:15.547 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:03:15.549 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T00:03:15.549 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:03:15.552 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T00:03:15.552 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:03:15.574 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T00:03:15.577 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T00:03:15.580 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T00:03:15.587 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T00:03:15.590 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T00:03:15.593 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T00:03:15.609 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: ntpd: command not found 2024-04-25T00:03:15.613 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: ntpd: command not found 2024-04-25T00:03:15.615 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: ntpd: command not found 2024-04-25T00:03:15.619 INFO:teuthology.orchestra.run.smithi107.stdout:506 Cannot talk to daemon 2024-04-25T00:03:15.623 INFO:teuthology.orchestra.run.smithi170.stdout:506 Cannot talk to daemon 2024-04-25T00:03:15.626 INFO:teuthology.orchestra.run.smithi195.stdout:506 Cannot talk to daemon 2024-04-25T00:03:15.633 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T00:03:15.636 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T00:03:15.639 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T00:03:15.645 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T00:03:15.649 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T00:03:15.651 INFO:teuthology.orchestra.run.smithi195.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T00:03:15.700 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:03:15.703 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:03:15.703 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-04-25T00:03:15.703 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:03:15.703 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:03:15.703 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:03:15.703 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:03:15.705 INFO:teuthology.orchestra.run.smithi170.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:03:15.706 INFO:teuthology.orchestra.run.smithi195.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:03:15.709 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:03:15.709 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2024-04-25T00:03:15.709 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:03:15.709 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:03:15.709 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:03:15.709 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:03:15.711 INFO:teuthology.orchestra.run.smithi195.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:03:15.711 INFO:teuthology.orchestra.run.smithi195.stdout:=============================================================================== 2024-04-25T00:03:15.711 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:03:15.711 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:03:15.711 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:03:15.711 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:03:15.711 INFO:teuthology.run_tasks:Running task install... 2024-04-25T00:03:15.730 DEBUG:teuthology.task.install:project ceph 2024-04-25T00:03:15.730 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-25T00:03:15.730 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-25T00:03:15.730 INFO:teuthology.task.install:Using flavor: default 2024-04-25T00:03:15.736 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T00:03:15.736 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T00:03:15.737 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-25T00:03:15.737 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:03:15.742 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-25T00:03:15.742 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:03:15.745 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-25T00:03:15.745 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:03:15.900 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-25T00:03:15.901 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-25T00:03:15.960 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-25T00:03:15.960 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-25T00:03:15.971 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-25T00:03:15.971 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-25T00:03:16.071 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T00:03:16.071 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-04-25T00:03:16.071 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T00:03:16.096 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T00:03:16.096 DEBUG:teuthology.orchestra.run.smithi195:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T00:03:16.139 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T00:03:16.139 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T00:03:16.139 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T00:03:16.142 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T00:03:16.142 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-04-25T00:03:16.142 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T00:03:16.164 DEBUG:teuthology.orchestra.run.smithi195:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T00:03:16.167 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T00:03:16.167 DEBUG:teuthology.orchestra.run.smithi107:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T00:03:16.168 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T00:03:16.169 DEBUG:teuthology.orchestra.run.smithi170:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T00:03:16.231 DEBUG:teuthology.orchestra.run.smithi107:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T00:03:16.233 DEBUG:teuthology.orchestra.run.smithi170:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T00:03:16.236 DEBUG:teuthology.orchestra.run.smithi195:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T00:03:16.300 INFO:teuthology.orchestra.run.smithi195.stdout:check_obsoletes = 1 2024-04-25T00:03:16.301 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum clean all 2024-04-25T00:03:16.304 DEBUG:teuthology.orchestra.run.smithi107:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T00:03:16.305 DEBUG:teuthology.orchestra.run.smithi170:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T00:03:16.366 INFO:teuthology.orchestra.run.smithi107.stdout:check_obsoletes = 1 2024-04-25T00:03:16.368 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum clean all 2024-04-25T00:03:16.369 INFO:teuthology.orchestra.run.smithi170.stdout:check_obsoletes = 1 2024-04-25T00:03:16.369 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum clean all 2024-04-25T00:03:16.526 INFO:teuthology.orchestra.run.smithi195.stdout:48 files removed 2024-04-25T00:03:16.553 DEBUG:teuthology.orchestra.run.smithi195:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T00:03:16.591 INFO:teuthology.orchestra.run.smithi107.stdout:48 files removed 2024-04-25T00:03:16.594 INFO:teuthology.orchestra.run.smithi170.stdout:48 files removed 2024-04-25T00:03:16.618 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T00:03:16.622 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T00:03:17.193 INFO:teuthology.orchestra.run.smithi107.stdout:ceph packages for x86_64 263 kB/s | 83 kB 00:00 2024-04-25T00:03:17.227 INFO:teuthology.orchestra.run.smithi195.stdout:ceph packages for x86_64 201 kB/s | 83 kB 00:00 2024-04-25T00:03:17.292 INFO:teuthology.orchestra.run.smithi170.stdout:ceph packages for x86_64 203 kB/s | 83 kB 00:00 2024-04-25T00:03:17.670 INFO:teuthology.orchestra.run.smithi107.stdout:ceph noarch packages 27 kB/s | 11 kB 00:00 2024-04-25T00:03:17.778 INFO:teuthology.orchestra.run.smithi195.stdout:ceph noarch packages 23 kB/s | 11 kB 00:00 2024-04-25T00:03:17.843 INFO:teuthology.orchestra.run.smithi170.stdout:ceph noarch packages 23 kB/s | 11 kB 00:00 2024-04-25T00:03:18.336 INFO:teuthology.orchestra.run.smithi107.stdout:ceph source packages 3.0 kB/s | 1.9 kB 00:00 2024-04-25T00:03:18.446 INFO:teuthology.orchestra.run.smithi195.stdout:ceph source packages 3.0 kB/s | 1.9 kB 00:00 2024-04-25T00:03:18.477 INFO:teuthology.orchestra.run.smithi170.stdout:ceph source packages 3.2 kB/s | 1.9 kB 00:00 2024-04-25T00:03:18.943 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.0 MB 00:00 2024-04-25T00:03:20.686 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-04-25T00:03:21.069 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 8.0 MB 00:02 2024-04-25T00:03:22.504 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - BaseOS 2.0 MB/s | 8.0 MB 00:04 2024-04-25T00:03:23.045 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-25T00:03:24.155 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - AppStream 33 MB/s | 19 MB 00:00 2024-04-25T00:03:27.270 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.2 MB 00:02 2024-04-25T00:03:28.237 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.2 MB 00:01 2024-04-25T00:03:28.239 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-25T00:03:28.888 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-25T00:03:29.790 INFO:teuthology.orchestra.run.smithi195.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 16 kB 00:00 2024-04-25T00:03:30.342 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-25T00:03:33.874 INFO:teuthology.orchestra.run.smithi195.stdout:Extra Packages for Enterprise Linux 5.3 MB/s | 21 MB 00:03 2024-04-25T00:03:34.587 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 3.1 kB/s | 16 kB 00:05 2024-04-25T00:03:35.448 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-25T00:03:35.547 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-04-25T00:03:37.557 INFO:teuthology.orchestra.run.smithi170.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:03:37.558 INFO:teuthology.orchestra.run.smithi170.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:03:37.569 INFO:teuthology.orchestra.run.smithi170.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T00:03:37.570 INFO:teuthology.orchestra.run.smithi170.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T00:03:37.570 INFO:teuthology.orchestra.run.smithi170.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T00:03:37.571 INFO:teuthology.orchestra.run.smithi170.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T00:03:37.573 INFO:teuthology.orchestra.run.smithi170.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T00:03:37.575 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T00:03:37.576 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T00:03:37.638 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-04-25T00:03:37.658 INFO:teuthology.orchestra.run.smithi170.stdout:======================================================================================= 2024-04-25T00:03:37.658 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2024-04-25T00:03:37.658 INFO:teuthology.orchestra.run.smithi170.stdout:======================================================================================= 2024-04-25T00:03:37.658 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-04-25T00:03:37.658 INFO:teuthology.orchestra.run.smithi170.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T00:03:37.658 INFO:teuthology.orchestra.run.smithi170.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-25T00:03:37.658 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-25T00:03:37.658 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-25T00:03:37.658 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-25T00:03:37.658 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-25T00:03:37.658 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-25T00:03:37.659 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-25T00:03:37.659 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-25T00:03:37.659 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-25T00:03:37.659 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-25T00:03:37.659 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-25T00:03:37.659 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-25T00:03:37.659 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-25T00:03:37.659 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-25T00:03:37.659 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-25T00:03:37.659 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-25T00:03:37.659 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T00:03:37.659 INFO:teuthology.orchestra.run.smithi170.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T00:03:37.659 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T00:03:37.660 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-25T00:03:37.660 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-25T00:03:37.660 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-25T00:03:37.660 INFO:teuthology.orchestra.run.smithi170.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T00:03:37.660 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T00:03:37.660 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T00:03:37.660 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T00:03:37.660 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T00:03:37.660 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T00:03:37.660 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-25T00:03:37.660 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-25T00:03:37.660 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-25T00:03:37.661 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-25T00:03:37.661 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-25T00:03:37.661 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:03:37.661 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-25T00:03:37.661 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T00:03:37.661 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrading: 2024-04-25T00:03:37.661 INFO:teuthology.orchestra.run.smithi170.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-25T00:03:37.661 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:03:37.661 INFO:teuthology.orchestra.run.smithi170.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T00:03:37.661 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T00:03:37.661 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2024-04-25T00:03:37.661 INFO:teuthology.orchestra.run.smithi170.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T00:03:37.661 INFO:teuthology.orchestra.run.smithi170.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T00:03:37.662 INFO:teuthology.orchestra.run.smithi170.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T00:03:37.662 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-25T00:03:37.662 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-25T00:03:37.662 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-25T00:03:37.662 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-25T00:03:37.662 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-25T00:03:37.662 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-25T00:03:37.662 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-25T00:03:37.662 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-25T00:03:37.662 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T00:03:37.662 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T00:03:37.662 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T00:03:37.662 INFO:teuthology.orchestra.run.smithi170.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T00:03:37.663 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T00:03:37.663 INFO:teuthology.orchestra.run.smithi170.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T00:03:37.663 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T00:03:37.663 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T00:03:37.663 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-25T00:03:37.663 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T00:03:37.663 INFO:teuthology.orchestra.run.smithi170.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T00:03:37.663 INFO:teuthology.orchestra.run.smithi170.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T00:03:37.663 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T00:03:37.663 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T00:03:37.663 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-25T00:03:37.663 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T00:03:37.663 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-25T00:03:37.664 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T00:03:37.664 INFO:teuthology.orchestra.run.smithi170.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T00:03:37.664 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T00:03:37.664 INFO:teuthology.orchestra.run.smithi170.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T00:03:37.664 INFO:teuthology.orchestra.run.smithi170.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T00:03:37.664 INFO:teuthology.orchestra.run.smithi170.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T00:03:37.664 INFO:teuthology.orchestra.run.smithi170.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T00:03:37.664 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T00:03:37.664 INFO:teuthology.orchestra.run.smithi170.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T00:03:37.664 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T00:03:37.664 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T00:03:37.664 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T00:03:37.664 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T00:03:37.665 INFO:teuthology.orchestra.run.smithi170.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T00:03:37.665 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T00:03:37.665 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T00:03:37.665 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T00:03:37.665 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-25T00:03:37.665 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-25T00:03:37.665 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T00:03:37.665 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T00:03:37.665 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T00:03:37.665 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T00:03:37.665 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T00:03:37.665 INFO:teuthology.orchestra.run.smithi170.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T00:03:37.665 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T00:03:37.665 INFO:teuthology.orchestra.run.smithi170.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T00:03:37.666 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T00:03:37.666 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T00:03:37.666 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T00:03:37.666 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T00:03:37.666 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T00:03:37.666 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T00:03:37.666 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T00:03:37.666 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T00:03:37.666 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T00:03:37.666 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T00:03:37.666 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T00:03:37.666 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T00:03:37.666 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T00:03:37.667 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T00:03:37.667 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T00:03:37.667 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T00:03:37.667 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T00:03:37.667 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T00:03:37.667 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T00:03:37.667 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T00:03:37.667 INFO:teuthology.orchestra.run.smithi170.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T00:03:37.667 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T00:03:37.667 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T00:03:37.667 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T00:03:37.667 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T00:03:37.667 INFO:teuthology.orchestra.run.smithi170.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T00:03:37.667 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T00:03:37.668 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T00:03:37.668 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T00:03:37.668 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T00:03:37.668 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T00:03:37.668 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T00:03:37.668 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T00:03:37.668 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T00:03:37.668 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T00:03:37.668 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T00:03:37.668 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T00:03:37.668 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T00:03:37.668 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T00:03:37.668 INFO:teuthology.orchestra.run.smithi170.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T00:03:37.668 INFO:teuthology.orchestra.run.smithi170.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T00:03:37.669 INFO:teuthology.orchestra.run.smithi170.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T00:03:37.669 INFO:teuthology.orchestra.run.smithi170.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T00:03:37.669 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T00:03:37.669 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T00:03:37.669 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T00:03:37.669 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T00:03:37.669 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T00:03:37.669 INFO:teuthology.orchestra.run.smithi170.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T00:03:37.669 INFO:teuthology.orchestra.run.smithi170.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T00:03:37.669 INFO:teuthology.orchestra.run.smithi170.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T00:03:37.669 INFO:teuthology.orchestra.run.smithi170.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T00:03:37.669 INFO:teuthology.orchestra.run.smithi170.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T00:03:37.669 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T00:03:37.669 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T00:03:37.670 INFO:teuthology.orchestra.run.smithi170.stdout:Installing weak dependencies: 2024-04-25T00:03:37.670 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T00:03:37.670 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-25T00:03:37.670 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-04-25T00:03:37.670 INFO:teuthology.orchestra.run.smithi170.stdout:======================================================================================= 2024-04-25T00:03:37.670 INFO:teuthology.orchestra.run.smithi170.stdout:Install 148 Packages 2024-04-25T00:03:37.670 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrade 4 Packages 2024-04-25T00:03:37.670 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-25T00:03:37.671 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 220 M 2024-04-25T00:03:37.671 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-04-25T00:03:38.374 INFO:teuthology.orchestra.run.smithi170.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 36 kB/s | 6.6 kB 00:00 2024-04-25T00:03:38.607 INFO:teuthology.orchestra.run.smithi170.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 3.7 MB/s | 890 kB 00:00 2024-04-25T00:03:38.674 INFO:teuthology.orchestra.run.smithi170.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 2.1 MB/s | 145 kB 00:00 2024-04-25T00:03:38.758 INFO:teuthology.orchestra.run.smithi170.stdout:(4/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 9.5 MB/s | 5.4 MB 00:00 2024-04-25T00:03:38.891 INFO:teuthology.orchestra.run.smithi170.stdout:(5/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 11 MB/s | 1.5 MB 00:00 2024-04-25T00:03:38.967 INFO:teuthology.orchestra.run.smithi170.stdout:(6/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 8.1 MB/s | 2.4 MB 00:00 2024-04-25T00:03:39.099 INFO:teuthology.orchestra.run.smithi195.stdout:lab-extras 7.3 kB/s | 1.7 kB 00:00 2024-04-25T00:03:39.184 INFO:teuthology.orchestra.run.smithi170.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 16 MB/s | 4.7 MB 00:00 2024-04-25T00:03:39.867 INFO:teuthology.orchestra.run.smithi170.stdout:(8/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el9 15 MB/s | 10 MB 00:00 2024-04-25T00:03:39.917 INFO:teuthology.orchestra.run.smithi170.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 504 kB/s | 25 kB 00:00 2024-04-25T00:03:40.613 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-25T00:03:41.001 INFO:teuthology.orchestra.run.smithi170.stdout:(10/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 8.3 MB/s | 17 MB 00:02 2024-04-25T00:03:41.051 INFO:teuthology.orchestra.run.smithi170.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 4.4 MB/s | 222 kB 00:00 2024-04-25T00:03:41.126 INFO:teuthology.orchestra.run.smithi170.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 362 kB/s | 27 kB 00:00 2024-04-25T00:03:41.193 INFO:teuthology.orchestra.run.smithi170.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 475 kB/s | 32 kB 00:00 2024-04-25T00:03:41.208 INFO:teuthology.orchestra.run.smithi195.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:03:41.209 INFO:teuthology.orchestra.run.smithi195.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:03:41.219 INFO:teuthology.orchestra.run.smithi195.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T00:03:41.220 INFO:teuthology.orchestra.run.smithi195.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T00:03:41.221 INFO:teuthology.orchestra.run.smithi195.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T00:03:41.221 INFO:teuthology.orchestra.run.smithi195.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T00:03:41.223 INFO:teuthology.orchestra.run.smithi195.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T00:03:41.225 INFO:teuthology.orchestra.run.smithi195.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T00:03:41.226 INFO:teuthology.orchestra.run.smithi195.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T00:03:41.285 INFO:teuthology.orchestra.run.smithi195.stdout:Dependencies resolved. 2024-04-25T00:03:41.308 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================= 2024-04-25T00:03:41.308 INFO:teuthology.orchestra.run.smithi195.stdout: Package Arch Version Repository Size 2024-04-25T00:03:41.308 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================= 2024-04-25T00:03:41.309 INFO:teuthology.orchestra.run.smithi195.stdout:Installing: 2024-04-25T00:03:41.309 INFO:teuthology.orchestra.run.smithi195.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T00:03:41.309 INFO:teuthology.orchestra.run.smithi195.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-25T00:03:41.309 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-25T00:03:41.309 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-25T00:03:41.309 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-25T00:03:41.309 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-25T00:03:41.309 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-25T00:03:41.309 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-25T00:03:41.310 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-25T00:03:41.310 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-25T00:03:41.310 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-25T00:03:41.310 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-25T00:03:41.310 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-25T00:03:41.310 INFO:teuthology.orchestra.run.smithi195.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-25T00:03:41.310 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-25T00:03:41.311 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-25T00:03:41.311 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-25T00:03:41.311 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T00:03:41.311 INFO:teuthology.orchestra.run.smithi195.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T00:03:41.311 INFO:teuthology.orchestra.run.smithi195.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T00:03:41.311 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-25T00:03:41.311 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-25T00:03:41.311 INFO:teuthology.orchestra.run.smithi195.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-25T00:03:41.311 INFO:teuthology.orchestra.run.smithi195.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T00:03:41.312 INFO:teuthology.orchestra.run.smithi195.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T00:03:41.312 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T00:03:41.312 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T00:03:41.312 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T00:03:41.312 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T00:03:41.312 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-25T00:03:41.312 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-25T00:03:41.312 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-25T00:03:41.312 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-25T00:03:41.313 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-25T00:03:41.313 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:03:41.313 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-25T00:03:41.313 INFO:teuthology.orchestra.run.smithi195.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T00:03:41.313 INFO:teuthology.orchestra.run.smithi195.stdout:Upgrading: 2024-04-25T00:03:41.313 INFO:teuthology.orchestra.run.smithi195.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-25T00:03:41.313 INFO:teuthology.orchestra.run.smithi195.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:03:41.313 INFO:teuthology.orchestra.run.smithi195.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T00:03:41.313 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T00:03:41.314 INFO:teuthology.orchestra.run.smithi195.stdout:Installing dependencies: 2024-04-25T00:03:41.314 INFO:teuthology.orchestra.run.smithi195.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T00:03:41.314 INFO:teuthology.orchestra.run.smithi195.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T00:03:41.314 INFO:teuthology.orchestra.run.smithi195.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T00:03:41.314 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-25T00:03:41.314 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-25T00:03:41.314 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-25T00:03:41.315 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-25T00:03:41.315 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-25T00:03:41.315 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-25T00:03:41.315 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-25T00:03:41.315 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-25T00:03:41.315 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T00:03:41.315 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T00:03:41.315 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T00:03:41.315 INFO:teuthology.orchestra.run.smithi195.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T00:03:41.315 INFO:teuthology.orchestra.run.smithi195.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T00:03:41.316 INFO:teuthology.orchestra.run.smithi195.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T00:03:41.316 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T00:03:41.316 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T00:03:41.316 INFO:teuthology.orchestra.run.smithi195.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-25T00:03:41.316 INFO:teuthology.orchestra.run.smithi195.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T00:03:41.316 INFO:teuthology.orchestra.run.smithi195.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T00:03:41.316 INFO:teuthology.orchestra.run.smithi195.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T00:03:41.316 INFO:teuthology.orchestra.run.smithi195.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T00:03:41.316 INFO:teuthology.orchestra.run.smithi195.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T00:03:41.316 INFO:teuthology.orchestra.run.smithi195.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-25T00:03:41.317 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T00:03:41.317 INFO:teuthology.orchestra.run.smithi195.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-25T00:03:41.317 INFO:teuthology.orchestra.run.smithi195.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T00:03:41.317 INFO:teuthology.orchestra.run.smithi195.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T00:03:41.317 INFO:teuthology.orchestra.run.smithi195.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T00:03:41.317 INFO:teuthology.orchestra.run.smithi195.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T00:03:41.317 INFO:teuthology.orchestra.run.smithi195.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T00:03:41.317 INFO:teuthology.orchestra.run.smithi195.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T00:03:41.317 INFO:teuthology.orchestra.run.smithi195.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T00:03:41.317 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T00:03:41.317 INFO:teuthology.orchestra.run.smithi195.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T00:03:41.318 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T00:03:41.318 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T00:03:41.318 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T00:03:41.318 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T00:03:41.318 INFO:teuthology.orchestra.run.smithi195.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T00:03:41.318 INFO:teuthology.orchestra.run.smithi195.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T00:03:41.318 INFO:teuthology.orchestra.run.smithi195.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T00:03:41.318 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T00:03:41.318 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-25T00:03:41.318 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-25T00:03:41.319 INFO:teuthology.orchestra.run.smithi195.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T00:03:41.319 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T00:03:41.319 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T00:03:41.319 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T00:03:41.319 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T00:03:41.319 INFO:teuthology.orchestra.run.smithi195.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T00:03:41.319 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T00:03:41.319 INFO:teuthology.orchestra.run.smithi195.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T00:03:41.319 INFO:teuthology.orchestra.run.smithi195.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T00:03:41.320 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T00:03:41.320 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T00:03:41.320 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T00:03:41.320 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T00:03:41.320 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T00:03:41.320 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T00:03:41.320 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T00:03:41.320 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T00:03:41.320 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T00:03:41.320 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T00:03:41.321 INFO:teuthology.orchestra.run.smithi195.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T00:03:41.321 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T00:03:41.321 INFO:teuthology.orchestra.run.smithi195.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T00:03:41.321 INFO:teuthology.orchestra.run.smithi195.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T00:03:41.321 INFO:teuthology.orchestra.run.smithi195.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T00:03:41.321 INFO:teuthology.orchestra.run.smithi195.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T00:03:41.321 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T00:03:41.321 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T00:03:41.321 INFO:teuthology.orchestra.run.smithi195.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T00:03:41.322 INFO:teuthology.orchestra.run.smithi195.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T00:03:41.322 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T00:03:41.322 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T00:03:41.322 INFO:teuthology.orchestra.run.smithi195.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T00:03:41.322 INFO:teuthology.orchestra.run.smithi195.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T00:03:41.322 INFO:teuthology.orchestra.run.smithi195.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T00:03:41.322 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T00:03:41.322 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T00:03:41.322 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T00:03:41.323 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T00:03:41.323 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T00:03:41.323 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T00:03:41.323 INFO:teuthology.orchestra.run.smithi195.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T00:03:41.323 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T00:03:41.323 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T00:03:41.323 INFO:teuthology.orchestra.run.smithi195.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T00:03:41.323 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T00:03:41.323 INFO:teuthology.orchestra.run.smithi195.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T00:03:41.323 INFO:teuthology.orchestra.run.smithi195.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T00:03:41.324 INFO:teuthology.orchestra.run.smithi195.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T00:03:41.324 INFO:teuthology.orchestra.run.smithi195.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T00:03:41.324 INFO:teuthology.orchestra.run.smithi195.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T00:03:41.324 INFO:teuthology.orchestra.run.smithi195.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T00:03:41.324 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T00:03:41.324 INFO:teuthology.orchestra.run.smithi195.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T00:03:41.324 INFO:teuthology.orchestra.run.smithi195.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T00:03:41.324 INFO:teuthology.orchestra.run.smithi195.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T00:03:41.324 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T00:03:41.324 INFO:teuthology.orchestra.run.smithi195.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T00:03:41.324 INFO:teuthology.orchestra.run.smithi195.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T00:03:41.325 INFO:teuthology.orchestra.run.smithi195.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T00:03:41.325 INFO:teuthology.orchestra.run.smithi195.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T00:03:41.325 INFO:teuthology.orchestra.run.smithi195.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T00:03:41.325 INFO:teuthology.orchestra.run.smithi195.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T00:03:41.325 INFO:teuthology.orchestra.run.smithi195.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T00:03:41.325 INFO:teuthology.orchestra.run.smithi195.stdout:Installing weak dependencies: 2024-04-25T00:03:41.325 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T00:03:41.325 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-04-25T00:03:41.325 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction Summary 2024-04-25T00:03:41.325 INFO:teuthology.orchestra.run.smithi195.stdout:======================================================================================= 2024-04-25T00:03:41.326 INFO:teuthology.orchestra.run.smithi195.stdout:Install 148 Packages 2024-04-25T00:03:41.326 INFO:teuthology.orchestra.run.smithi195.stdout:Upgrade 4 Packages 2024-04-25T00:03:41.326 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-04-25T00:03:41.327 INFO:teuthology.orchestra.run.smithi195.stdout:Total download size: 220 M 2024-04-25T00:03:41.327 INFO:teuthology.orchestra.run.smithi195.stdout:Downloading Packages: 2024-04-25T00:03:41.335 INFO:teuthology.orchestra.run.smithi170.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 5.1 MB/s | 745 kB 00:00 2024-04-25T00:03:41.410 INFO:teuthology.orchestra.run.smithi170.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 2.1 MB/s | 163 kB 00:00 2024-04-25T00:03:41.486 INFO:teuthology.orchestra.run.smithi170.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.6 MB/s | 127 kB 00:00 2024-04-25T00:03:41.586 INFO:teuthology.orchestra.run.smithi170.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 4.9 MB/s | 501 kB 00:00 2024-04-25T00:03:42.329 INFO:teuthology.orchestra.run.smithi170.stdout:(18/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 7.2 MB/s | 5.3 MB 00:00 2024-04-25T00:03:42.412 INFO:teuthology.orchestra.run.smithi170.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 542 kB/s | 45 kB 00:00 2024-04-25T00:03:42.822 INFO:teuthology.orchestra.run.smithi107.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:03:42.824 INFO:teuthology.orchestra.run.smithi107.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:03:42.829 INFO:teuthology.orchestra.run.smithi170.stdout:(20/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 17 MB/s | 48 MB 00:02 2024-04-25T00:03:42.834 INFO:teuthology.orchestra.run.smithi107.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T00:03:42.834 INFO:teuthology.orchestra.run.smithi107.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T00:03:42.835 INFO:teuthology.orchestra.run.smithi107.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T00:03:42.836 INFO:teuthology.orchestra.run.smithi107.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T00:03:42.838 INFO:teuthology.orchestra.run.smithi107.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T00:03:42.840 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T00:03:42.841 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T00:03:42.855 INFO:teuthology.orchestra.run.smithi170.stdout:(21/152): python3-ceph-common-19.0.0-2445.gd2c4 301 kB/s | 133 kB 00:00 2024-04-25T00:03:42.901 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-25T00:03:42.910 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-04-25T00:03:42.910 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-04-25T00:03:42.910 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-04-25T00:03:42.910 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-04-25T00:03:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T00:03:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-25T00:03:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-25T00:03:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-25T00:03:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-25T00:03:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-25T00:03:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-25T00:03:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-25T00:03:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-25T00:03:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-25T00:03:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-25T00:03:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-25T00:03:42.911 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-25T00:03:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-25T00:03:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-25T00:03:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-25T00:03:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-25T00:03:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T00:03:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T00:03:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T00:03:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-25T00:03:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-25T00:03:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-25T00:03:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T00:03:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T00:03:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T00:03:42.912 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T00:03:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T00:03:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T00:03:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-25T00:03:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-25T00:03:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-25T00:03:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-25T00:03:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-25T00:03:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:03:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-25T00:03:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T00:03:42.913 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2024-04-25T00:03:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-25T00:03:42.913 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:03:42.914 INFO:teuthology.orchestra.run.smithi107.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T00:03:42.914 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T00:03:42.914 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-04-25T00:03:42.914 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T00:03:42.914 INFO:teuthology.orchestra.run.smithi107.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T00:03:42.914 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T00:03:42.914 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-25T00:03:42.914 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-25T00:03:42.914 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-25T00:03:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-25T00:03:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-25T00:03:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-25T00:03:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-25T00:03:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-25T00:03:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T00:03:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T00:03:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T00:03:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T00:03:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T00:03:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T00:03:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T00:03:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T00:03:42.915 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-25T00:03:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T00:03:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T00:03:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T00:03:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T00:03:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T00:03:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-25T00:03:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T00:03:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-25T00:03:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T00:03:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T00:03:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T00:03:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T00:03:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T00:03:42.916 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T00:03:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T00:03:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T00:03:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T00:03:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T00:03:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T00:03:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T00:03:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T00:03:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T00:03:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T00:03:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T00:03:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T00:03:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-25T00:03:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-25T00:03:42.917 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T00:03:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T00:03:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T00:03:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T00:03:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T00:03:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T00:03:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T00:03:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T00:03:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T00:03:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T00:03:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T00:03:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T00:03:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T00:03:42.918 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T00:03:42.919 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T00:03:42.919 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T00:03:42.919 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T00:03:42.919 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T00:03:42.919 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T00:03:42.919 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T00:03:42.919 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T00:03:42.919 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T00:03:42.919 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T00:03:42.919 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T00:03:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T00:03:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T00:03:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T00:03:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T00:03:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T00:03:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T00:03:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T00:03:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T00:03:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T00:03:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T00:03:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T00:03:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T00:03:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T00:03:42.920 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T00:03:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T00:03:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T00:03:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T00:03:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T00:03:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T00:03:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T00:03:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T00:03:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T00:03:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T00:03:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T00:03:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T00:03:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T00:03:42.921 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T00:03:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T00:03:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T00:03:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T00:03:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T00:03:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T00:03:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T00:03:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T00:03:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T00:03:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T00:03:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T00:03:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T00:03:42.922 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T00:03:42.922 INFO:teuthology.orchestra.run.smithi107.stdout:Installing weak dependencies: 2024-04-25T00:03:42.923 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T00:03:42.923 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:03:42.923 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-25T00:03:42.923 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-04-25T00:03:42.923 INFO:teuthology.orchestra.run.smithi107.stdout:Install 148 Packages 2024-04-25T00:03:42.923 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 4 Packages 2024-04-25T00:03:42.923 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:03:42.924 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 220 M 2024-04-25T00:03:42.924 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-04-25T00:03:42.930 INFO:teuthology.orchestra.run.smithi170.stdout:(22/152): python3-cephfs-19.0.0-2445.gd2c45d97. 1.6 MB/s | 162 kB 00:00 2024-04-25T00:03:42.989 INFO:teuthology.orchestra.run.smithi170.stdout:(23/152): python3-rados-19.0.0-2445.gd2c45d97.e 2.3 MB/s | 321 kB 00:00 2024-04-25T00:03:43.014 INFO:teuthology.orchestra.run.smithi170.stdout:(24/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 3.5 MB/s | 299 kB 00:00 2024-04-25T00:03:43.039 INFO:teuthology.orchestra.run.smithi170.stdout:(25/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.9 MB/s | 99 kB 00:00 2024-04-25T00:03:43.098 INFO:teuthology.orchestra.run.smithi170.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-25T00:03:43.292 INFO:teuthology.orchestra.run.smithi195.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-04-25T00:03:43.805 INFO:teuthology.orchestra.run.smithi107.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-25T00:03:43.906 INFO:teuthology.orchestra.run.smithi170.stdout:(27/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 3.6 MB/s | 3.1 MB 00:00 2024-04-25T00:03:43.940 INFO:teuthology.orchestra.run.smithi170.stdout:(28/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 203 kB/s | 171 kB 00:00 2024-04-25T00:03:44.007 INFO:teuthology.orchestra.run.smithi170.stdout:(29/152): ceph-grafana-dashboards-19.0.0-2445.g 317 kB/s | 32 kB 00:00 2024-04-25T00:03:44.049 INFO:teuthology.orchestra.run.smithi170.stdout:(30/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.3 MB/s | 141 kB 00:00 2024-04-25T00:03:44.334 INFO:teuthology.orchestra.run.smithi195.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 854 kB/s | 890 kB 00:01 2024-04-25T00:03:44.484 INFO:teuthology.orchestra.run.smithi195.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 967 kB/s | 145 kB 00:00 2024-04-25T00:03:45.172 INFO:teuthology.orchestra.run.smithi107.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 651 kB/s | 890 kB 00:01 2024-04-25T00:03:45.531 INFO:teuthology.orchestra.run.smithi107.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 404 kB/s | 145 kB 00:00 2024-04-25T00:03:45.749 INFO:teuthology.orchestra.run.smithi170.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 4.3 MB/s | 7.4 MB 00:01 2024-04-25T00:03:45.874 INFO:teuthology.orchestra.run.smithi170.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 1.9 MB/s | 249 kB 00:00 2024-04-25T00:03:45.950 INFO:teuthology.orchestra.run.smithi170.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 658 kB/s | 49 kB 00:00 2024-04-25T00:03:46.017 INFO:teuthology.orchestra.run.smithi170.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 230 kB/s | 15 kB 00:00 2024-04-25T00:03:46.022 INFO:teuthology.orchestra.run.smithi107.stdout:(4/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 2.0 MB/s | 5.4 MB 00:02 2024-04-25T00:03:46.084 INFO:teuthology.orchestra.run.smithi195.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 1.5 MB/s | 2.4 MB 00:01 2024-04-25T00:03:46.092 INFO:teuthology.orchestra.run.smithi170.stdout:(35/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 1.7 MB/s | 3.5 MB 00:02 2024-04-25T00:03:46.184 INFO:teuthology.orchestra.run.smithi170.stdout:(36/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 1.5 MB/s | 264 kB 00:00 2024-04-25T00:03:46.268 INFO:teuthology.orchestra.run.smithi170.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 219 kB/s | 18 kB 00:00 2024-04-25T00:03:46.551 INFO:teuthology.orchestra.run.smithi170.stdout:(38/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 1.6 MB/s | 762 kB 00:00 2024-04-25T00:03:46.685 INFO:teuthology.orchestra.run.smithi170.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 193 kB/s | 80 kB 00:00 2024-04-25T00:03:46.706 INFO:teuthology.orchestra.run.smithi107.stdout:(5/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 2.2 MB/s | 1.5 MB 00:00 2024-04-25T00:03:46.901 INFO:teuthology.orchestra.run.smithi195.stdout:(5/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 1.8 MB/s | 1.5 MB 00:00 2024-04-25T00:03:46.952 INFO:teuthology.orchestra.run.smithi170.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 732 kB/s | 195 kB 00:00 2024-04-25T00:03:47.143 INFO:teuthology.orchestra.run.smithi170.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-04-25T00:03:47.352 INFO:teuthology.orchestra.run.smithi170.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-04-25T00:03:47.469 INFO:teuthology.orchestra.run.smithi170.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 912 kB/s | 106 kB 00:00 2024-04-25T00:03:47.662 INFO:teuthology.orchestra.run.smithi170.stdout:(44/152): ceph-common-19.0.0-2445.gd2c45d97.el9 2.2 MB/s | 21 MB 00:09 2024-04-25T00:03:47.687 INFO:teuthology.orchestra.run.smithi170.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 623 kB/s | 135 kB 00:00 2024-04-25T00:03:47.721 INFO:teuthology.orchestra.run.smithi170.stdout:(46/152): python3-cryptography-36.0.1-4.el9.x86 2.1 MB/s | 1.2 MB 00:00 2024-04-25T00:03:47.804 INFO:teuthology.orchestra.run.smithi170.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-04-25T00:03:48.046 INFO:teuthology.orchestra.run.smithi170.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-04-25T00:03:48.088 INFO:teuthology.orchestra.run.smithi170.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 689 kB 00:00 2024-04-25T00:03:48.138 INFO:teuthology.orchestra.run.smithi170.stdout:(50/152): python3-requests-2.25.1-8.el9.noarch. 262 kB/s | 125 kB 00:00 2024-04-25T00:03:48.230 INFO:teuthology.orchestra.run.smithi170.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 582 kB/s | 107 kB 00:00 2024-04-25T00:03:48.255 INFO:teuthology.orchestra.run.smithi170.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 151 kB/s | 21 kB 00:00 2024-04-25T00:03:48.339 INFO:teuthology.orchestra.run.smithi170.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 1.7 MB/s | 313 kB 00:00 2024-04-25T00:03:48.364 INFO:teuthology.orchestra.run.smithi170.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 246 kB/s | 33 kB 00:00 2024-04-25T00:03:48.398 INFO:teuthology.orchestra.run.smithi107.stdout:(6/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 844 kB/s | 2.4 MB 00:02 2024-04-25T00:03:48.464 INFO:teuthology.orchestra.run.smithi170.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 147 kB/s | 18 kB 00:00 2024-04-25T00:03:48.490 INFO:teuthology.orchestra.run.smithi170.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 110 kB/s | 11 kB 00:00 2024-04-25T00:03:48.565 INFO:teuthology.orchestra.run.smithi107.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 2.5 MB/s | 4.7 MB 00:01 2024-04-25T00:03:48.598 INFO:teuthology.orchestra.run.smithi170.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.4 MB/s | 3.0 MB 00:00 2024-04-25T00:03:48.657 INFO:teuthology.orchestra.run.smithi170.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 317 kB/s | 45 kB 00:00 2024-04-25T00:03:48.682 INFO:teuthology.orchestra.run.smithi170.stdout:(59/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 738 kB/s | 160 kB 00:00 2024-04-25T00:03:48.774 INFO:teuthology.orchestra.run.smithi170.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 662 kB 00:00 2024-04-25T00:03:48.824 INFO:teuthology.orchestra.run.smithi170.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 215 kB/s | 25 kB 00:00 2024-04-25T00:03:48.866 INFO:teuthology.orchestra.run.smithi170.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 2.7 MB/s | 582 kB 00:00 2024-04-25T00:03:48.909 INFO:teuthology.orchestra.run.smithi170.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 357 kB/s | 47 kB 00:00 2024-04-25T00:03:48.992 INFO:teuthology.orchestra.run.smithi170.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-04-25T00:03:49.042 INFO:teuthology.orchestra.run.smithi170.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 325 kB/s | 35 kB 00:00 2024-04-25T00:03:49.076 INFO:teuthology.orchestra.run.smithi170.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-04-25T00:03:49.218 INFO:teuthology.orchestra.run.smithi195.stdout:(6/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 2.0 MB/s | 4.7 MB 00:02 2024-04-25T00:03:49.268 INFO:teuthology.orchestra.run.smithi170.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-04-25T00:03:49.335 INFO:teuthology.orchestra.run.smithi170.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 14 MB/s | 4.7 MB 00:00 2024-04-25T00:03:49.460 INFO:teuthology.orchestra.run.smithi170.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-25T00:03:49.544 INFO:teuthology.orchestra.run.smithi170.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 3.2 MB/s | 911 kB 00:00 2024-04-25T00:03:49.571 INFO:teuthology.orchestra.run.smithi170.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 633 kB/s | 133 kB 00:00 2024-04-25T00:03:49.696 INFO:teuthology.orchestra.run.smithi170.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 868 kB/s | 87 kB 00:00 2024-04-25T00:03:49.771 INFO:teuthology.orchestra.run.smithi170.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.4 MB/s | 1.0 MB 00:00 2024-04-25T00:03:49.831 INFO:teuthology.orchestra.run.smithi107.stdout:(8/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 3.3 MB/s | 21 MB 00:06 2024-04-25T00:03:49.915 INFO:teuthology.orchestra.run.smithi107.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 302 kB/s | 25 kB 00:00 2024-04-25T00:03:49.930 INFO:teuthology.orchestra.run.smithi170.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 12 MB/s | 5.8 MB 00:00 2024-04-25T00:03:50.014 INFO:teuthology.orchestra.run.smithi170.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.0 MB/s | 244 kB 00:00 2024-04-25T00:03:50.081 INFO:teuthology.orchestra.run.smithi170.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-04-25T00:03:50.139 INFO:teuthology.orchestra.run.smithi170.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 205 kB/s | 26 kB 00:00 2024-04-25T00:03:50.223 INFO:teuthology.orchestra.run.smithi170.stdout:(78/152): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-25T00:03:50.273 INFO:teuthology.orchestra.run.smithi170.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 100 kB/s | 19 kB 00:00 2024-04-25T00:03:50.357 INFO:teuthology.orchestra.run.smithi170.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 797 kB/s | 172 kB 00:00 2024-04-25T00:03:50.382 INFO:teuthology.orchestra.run.smithi170.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 259 kB/s | 35 kB 00:00 2024-04-25T00:03:50.533 INFO:teuthology.orchestra.run.smithi170.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 250 kB 00:00 2024-04-25T00:03:50.591 INFO:teuthology.orchestra.run.smithi170.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 222 kB 00:00 2024-04-25T00:03:50.650 INFO:teuthology.orchestra.run.smithi170.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 662 kB/s | 77 kB 00:00 2024-04-25T00:03:50.708 INFO:teuthology.orchestra.run.smithi170.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 364 kB/s | 42 kB 00:00 2024-04-25T00:03:50.833 INFO:teuthology.orchestra.run.smithi170.stdout:(86/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.5 MB/s | 5.1 MB 00:00 2024-04-25T00:03:50.867 INFO:teuthology.orchestra.run.smithi170.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 1.2 MB/s | 269 kB 00:00 2024-04-25T00:03:50.917 INFO:teuthology.orchestra.run.smithi170.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 763 kB/s | 159 kB 00:00 2024-04-25T00:03:51.001 INFO:teuthology.orchestra.run.smithi170.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 279 kB 00:00 2024-04-25T00:03:51.018 INFO:teuthology.orchestra.run.smithi170.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 405 kB/s | 51 kB 00:00 2024-04-25T00:03:51.060 INFO:teuthology.orchestra.run.smithi170.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 457 kB/s | 54 kB 00:00 2024-04-25T00:03:51.160 INFO:teuthology.orchestra.run.smithi170.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 353 kB/s | 42 kB 00:00 2024-04-25T00:03:51.177 INFO:teuthology.orchestra.run.smithi170.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 482 kB/s | 44 kB 00:00 2024-04-25T00:03:51.294 INFO:teuthology.orchestra.run.smithi170.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 389 kB/s | 36 kB 00:00 2024-04-25T00:03:51.319 INFO:teuthology.orchestra.run.smithi170.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-04-25T00:03:51.436 INFO:teuthology.orchestra.run.smithi170.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 335 kB/s | 47 kB 00:00 2024-04-25T00:03:51.512 INFO:teuthology.orchestra.run.smithi170.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.8 MB/s | 305 kB 00:00 2024-04-25T00:03:51.579 INFO:teuthology.orchestra.run.smithi170.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 408 kB/s | 58 kB 00:00 2024-04-25T00:03:51.684 INFO:teuthology.orchestra.run.smithi195.stdout:(7/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 631 kB/s | 5.4 MB 00:08 2024-04-25T00:03:51.779 INFO:teuthology.orchestra.run.smithi170.stdout:(99/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-04-25T00:03:51.821 INFO:teuthology.orchestra.run.smithi170.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 304 kB/s | 94 kB 00:00 2024-04-25T00:03:51.871 INFO:teuthology.orchestra.run.smithi170.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 218 kB/s | 64 kB 00:00 2024-04-25T00:03:51.922 INFO:teuthology.orchestra.run.smithi170.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 297 kB/s | 22 kB 00:00 2024-04-25T00:03:51.947 INFO:teuthology.orchestra.run.smithi170.stdout:(103/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 270 kB/s | 45 kB 00:00 2024-04-25T00:03:52.097 INFO:teuthology.orchestra.run.smithi170.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 3.8 MB/s | 863 kB 00:00 2024-04-25T00:03:52.290 INFO:teuthology.orchestra.run.smithi107.stdout:(10/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 2.7 MB/s | 10 MB 00:03 2024-04-25T00:03:52.398 INFO:teuthology.orchestra.run.smithi170.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 65 kB/s | 19 kB 00:00 2024-04-25T00:03:52.431 INFO:teuthology.orchestra.run.smithi170.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 638 kB/s | 309 kB 00:00 2024-04-25T00:03:52.432 INFO:teuthology.orchestra.run.smithi107.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 1.5 MB/s | 222 kB 00:00 2024-04-25T00:03:52.473 INFO:teuthology.orchestra.run.smithi170.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 995 kB/s | 548 kB 00:00 2024-04-25T00:03:52.515 INFO:teuthology.orchestra.run.smithi107.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 325 kB/s | 27 kB 00:00 2024-04-25T00:03:52.540 INFO:teuthology.orchestra.run.smithi170.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 239 kB/s | 26 kB 00:00 2024-04-25T00:03:52.565 INFO:teuthology.orchestra.run.smithi170.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 529 kB/s | 49 kB 00:00 2024-04-25T00:03:52.591 INFO:teuthology.orchestra.run.smithi107.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 424 kB/s | 32 kB 00:00 2024-04-25T00:03:52.649 INFO:teuthology.orchestra.run.smithi170.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 619 kB/s | 67 kB 00:00 2024-04-25T00:03:52.674 INFO:teuthology.orchestra.run.smithi170.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-25T00:03:52.775 INFO:teuthology.orchestra.run.smithi170.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 5.4 MB/s | 548 kB 00:00 2024-04-25T00:03:52.824 INFO:teuthology.orchestra.run.smithi107.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 3.1 MB/s | 745 kB 00:00 2024-04-25T00:03:52.867 INFO:teuthology.orchestra.run.smithi170.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 474 kB/s | 43 kB 00:00 2024-04-25T00:03:52.874 INFO:teuthology.orchestra.run.smithi107.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 3.2 MB/s | 163 kB 00:00 2024-04-25T00:03:52.908 INFO:teuthology.orchestra.run.smithi170.stdout:(114/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.2 MB/s | 838 kB 00:00 2024-04-25T00:03:52.941 INFO:teuthology.orchestra.run.smithi107.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.9 MB/s | 127 kB 00:00 2024-04-25T00:03:52.959 INFO:teuthology.orchestra.run.smithi170.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 351 kB/s | 32 kB 00:00 2024-04-25T00:03:53.034 INFO:teuthology.orchestra.run.smithi170.stdout:(116/152): libarrow-9.0.0-10.el9.x86_64.rpm 6.9 MB/s | 4.4 MB 00:00 2024-04-25T00:03:53.059 INFO:teuthology.orchestra.run.smithi170.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 99 kB/s | 15 kB 00:00 2024-04-25T00:03:53.085 INFO:teuthology.orchestra.run.smithi170.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-25T00:03:53.100 INFO:teuthology.orchestra.run.smithi107.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 3.1 MB/s | 501 kB 00:00 2024-04-25T00:03:53.143 INFO:teuthology.orchestra.run.smithi170.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-25T00:03:53.169 INFO:teuthology.orchestra.run.smithi170.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-25T00:03:53.194 INFO:teuthology.orchestra.run.smithi170.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 324 kB/s | 35 kB 00:00 2024-04-25T00:03:53.244 INFO:teuthology.orchestra.run.smithi170.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-04-25T00:03:53.286 INFO:teuthology.orchestra.run.smithi170.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-25T00:03:53.336 INFO:teuthology.orchestra.run.smithi170.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-04-25T00:03:53.379 INFO:teuthology.orchestra.run.smithi170.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-04-25T00:03:53.429 INFO:teuthology.orchestra.run.smithi170.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 252 kB/s | 23 kB 00:00 2024-04-25T00:03:53.471 INFO:teuthology.orchestra.run.smithi170.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-04-25T00:03:53.521 INFO:teuthology.orchestra.run.smithi170.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-04-25T00:03:53.580 INFO:teuthology.orchestra.run.smithi170.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-04-25T00:03:53.605 INFO:teuthology.orchestra.run.smithi170.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 554 kB/s | 46 kB 00:00 2024-04-25T00:03:53.672 INFO:teuthology.orchestra.run.smithi170.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 858 kB/s | 79 kB 00:00 2024-04-25T00:03:53.689 INFO:teuthology.orchestra.run.smithi170.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 689 kB/s | 58 kB 00:00 2024-04-25T00:03:53.764 INFO:teuthology.orchestra.run.smithi170.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-25T00:03:53.781 INFO:teuthology.orchestra.run.smithi170.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-04-25T00:03:53.857 INFO:teuthology.orchestra.run.smithi170.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 979 kB/s | 90 kB 00:00 2024-04-25T00:03:53.882 INFO:teuthology.orchestra.run.smithi170.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-04-25T00:03:53.949 INFO:teuthology.orchestra.run.smithi170.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-04-25T00:03:53.982 INFO:teuthology.orchestra.run.smithi170.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-25T00:03:54.041 INFO:teuthology.orchestra.run.smithi170.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 643 kB/s | 59 kB 00:00 2024-04-25T00:03:54.083 INFO:teuthology.orchestra.run.smithi170.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 356 kB/s | 36 kB 00:00 2024-04-25T00:03:54.125 INFO:teuthology.orchestra.run.smithi170.stdout:(141/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.1 MB/s | 2.0 MB 00:00 2024-04-25T00:03:54.150 INFO:teuthology.orchestra.run.smithi170.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 468 kB/s | 51 kB 00:00 2024-04-25T00:03:54.192 INFO:teuthology.orchestra.run.smithi170.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-25T00:03:54.226 INFO:teuthology.orchestra.run.smithi170.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 893 kB/s | 90 kB 00:00 2024-04-25T00:03:54.251 INFO:teuthology.orchestra.run.smithi170.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-04-25T00:03:54.301 INFO:teuthology.orchestra.run.smithi170.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 183 kB/s | 20 kB 00:00 2024-04-25T00:03:54.327 INFO:teuthology.orchestra.run.smithi170.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-25T00:03:54.892 INFO:teuthology.orchestra.run.smithi107.stdout:(18/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 3.0 MB/s | 5.3 MB 00:01 2024-04-25T00:03:54.984 INFO:teuthology.orchestra.run.smithi107.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 493 kB/s | 45 kB 00:00 2024-04-25T00:03:55.043 INFO:teuthology.orchestra.run.smithi170.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-04-25T00:03:55.075 INFO:teuthology.orchestra.run.smithi107.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.4 MB/s | 133 kB 00:00 2024-04-25T00:03:55.167 INFO:teuthology.orchestra.run.smithi107.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 1.7 MB/s | 162 kB 00:00 2024-04-25T00:03:55.318 INFO:teuthology.orchestra.run.smithi107.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 2.1 MB/s | 321 kB 00:00 2024-04-25T00:03:55.451 INFO:teuthology.orchestra.run.smithi107.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 2.2 MB/s | 299 kB 00:00 2024-04-25T00:03:55.501 INFO:teuthology.orchestra.run.smithi195.stdout:(8/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x86 2.7 MB/s | 17 MB 00:06 2024-04-25T00:03:55.543 INFO:teuthology.orchestra.run.smithi107.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.1 MB/s | 99 kB 00:00 2024-04-25T00:03:55.601 INFO:teuthology.orchestra.run.smithi195.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 253 kB/s | 25 kB 00:00 2024-04-25T00:03:55.627 INFO:teuthology.orchestra.run.smithi107.stdout:(25/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-25T00:03:55.869 INFO:teuthology.orchestra.run.smithi170.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 1.6 MB/s | 1.3 MB 00:00 2024-04-25T00:03:56.110 INFO:teuthology.orchestra.run.smithi195.stdout:(10/152): ceph-common-19.0.0-2445.gd2c45d97.el9 1.6 MB/s | 21 MB 00:13 2024-04-25T00:03:56.169 INFO:teuthology.orchestra.run.smithi170.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 8.0 MB/s | 2.4 MB 00:00 2024-04-25T00:03:56.260 INFO:teuthology.orchestra.run.smithi195.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 1.4 MB/s | 222 kB 00:00 2024-04-25T00:03:56.351 INFO:teuthology.orchestra.run.smithi195.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 296 kB/s | 27 kB 00:00 2024-04-25T00:03:56.385 INFO:teuthology.orchestra.run.smithi107.stdout:(26/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 4.1 MB/s | 3.1 MB 00:00 2024-04-25T00:03:56.435 INFO:teuthology.orchestra.run.smithi195.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 381 kB/s | 32 kB 00:00 2024-04-25T00:03:56.460 INFO:teuthology.orchestra.run.smithi107.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 2.2 MB/s | 171 kB 00:00 2024-04-25T00:03:56.536 INFO:teuthology.orchestra.run.smithi107.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 421 kB/s | 32 kB 00:00 2024-04-25T00:03:56.620 INFO:teuthology.orchestra.run.smithi107.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.7 MB/s | 141 kB 00:00 2024-04-25T00:03:56.627 INFO:teuthology.orchestra.run.smithi170.stdout:(151/152): librados2-19.0.0-2445.gd2c45d97.el9. 1.5 MB/s | 3.4 MB 00:02 2024-04-25T00:03:56.760 INFO:teuthology.orchestra.run.smithi195.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 2.2 MB/s | 745 kB 00:00 2024-04-25T00:03:56.869 INFO:teuthology.orchestra.run.smithi195.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 1.5 MB/s | 163 kB 00:00 2024-04-25T00:03:56.969 INFO:teuthology.orchestra.run.smithi195.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.2 MB/s | 127 kB 00:00 2024-04-25T00:03:57.211 INFO:teuthology.orchestra.run.smithi170.stdout:(152/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 1.1 MB/s | 3.1 MB 00:02 2024-04-25T00:03:57.214 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-04-25T00:03:57.214 INFO:teuthology.orchestra.run.smithi170.stdout:Total 11 MB/s | 220 MB 00:19 2024-04-25T00:03:57.236 INFO:teuthology.orchestra.run.smithi195.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 1.8 MB/s | 501 kB 00:00 2024-04-25T00:03:57.612 INFO:teuthology.orchestra.run.smithi107.stdout:(30/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 1.8 MB/s | 17 MB 00:09 2024-04-25T00:03:57.670 INFO:teuthology.orchestra.run.smithi107.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 3.3 MB/s | 3.5 MB 00:01 2024-04-25T00:03:57.787 INFO:teuthology.orchestra.run.smithi107.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 2.1 MB/s | 249 kB 00:00 2024-04-25T00:03:57.812 INFO:teuthology.orchestra.run.smithi195.stdout:(18/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 1.7 MB/s | 10 MB 00:06 2024-04-25T00:03:57.854 INFO:teuthology.orchestra.run.smithi107.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 740 kB/s | 49 kB 00:00 2024-04-25T00:03:57.912 INFO:teuthology.orchestra.run.smithi195.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 453 kB/s | 45 kB 00:00 2024-04-25T00:03:57.929 INFO:teuthology.orchestra.run.smithi107.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 204 kB/s | 15 kB 00:00 2024-04-25T00:03:58.013 INFO:teuthology.orchestra.run.smithi195.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.3 MB/s | 133 kB 00:00 2024-04-25T00:03:58.079 INFO:teuthology.orchestra.run.smithi107.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 1.7 MB/s | 264 kB 00:00 2024-04-25T00:03:58.112 INFO:teuthology.orchestra.run.smithi195.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 1.6 MB/s | 162 kB 00:00 2024-04-25T00:03:58.289 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-04-25T00:03:58.296 INFO:teuthology.orchestra.run.smithi195.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 1.7 MB/s | 321 kB 00:00 2024-04-25T00:03:58.305 INFO:teuthology.orchestra.run.smithi107.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 3.3 MB/s | 762 kB 00:00 2024-04-25T00:03:58.380 INFO:teuthology.orchestra.run.smithi107.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 243 kB/s | 18 kB 00:00 2024-04-25T00:03:58.433 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-04-25T00:03:58.434 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-04-25T00:03:58.455 INFO:teuthology.orchestra.run.smithi195.stdout:(23/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 1.8 MB/s | 299 kB 00:00 2024-04-25T00:03:58.506 INFO:teuthology.orchestra.run.smithi107.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 643 kB/s | 80 kB 00:00 2024-04-25T00:03:58.555 INFO:teuthology.orchestra.run.smithi195.stdout:(24/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 993 kB/s | 99 kB 00:00 2024-04-25T00:03:58.631 INFO:teuthology.orchestra.run.smithi107.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-04-25T00:03:58.664 INFO:teuthology.orchestra.run.smithi195.stdout:(25/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 788 kB/s | 85 kB 00:00 2024-04-25T00:03:58.681 INFO:teuthology.orchestra.run.smithi107.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-04-25T00:03:58.731 INFO:teuthology.orchestra.run.smithi107.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-25T00:03:58.815 INFO:teuthology.orchestra.run.smithi107.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-04-25T00:03:58.865 INFO:teuthology.orchestra.run.smithi107.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-25T00:03:58.916 INFO:teuthology.orchestra.run.smithi107.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-25T00:03:59.024 INFO:teuthology.orchestra.run.smithi107.stdout:(45/152): ceph-mgr-diskprediction-local-19.0.0- 5.2 MB/s | 7.4 MB 00:01 2024-04-25T00:03:59.049 INFO:teuthology.orchestra.run.smithi107.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 936 kB/s | 125 kB 00:00 2024-04-25T00:03:59.141 INFO:teuthology.orchestra.run.smithi107.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 7.3 MB/s | 689 kB 00:00 2024-04-25T00:03:59.189 INFO:teuthology.orchestra.run.smithi195.stdout:(26/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 2.7 MB/s | 5.3 MB 00:01 2024-04-25T00:03:59.191 INFO:teuthology.orchestra.run.smithi107.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.3 MB/s | 215 kB 00:00 2024-04-25T00:03:59.250 INFO:teuthology.orchestra.run.smithi107.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-25T00:03:59.256 INFO:teuthology.orchestra.run.smithi195.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 2.5 MB/s | 171 kB 00:00 2024-04-25T00:03:59.284 INFO:teuthology.orchestra.run.smithi107.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 6.5 MB/s | 943 kB 00:00 2024-04-25T00:03:59.326 INFO:teuthology.orchestra.run.smithi107.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 286 kB/s | 21 kB 00:00 2024-04-25T00:03:59.333 INFO:teuthology.orchestra.run.smithi195.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 425 kB/s | 32 kB 00:00 2024-04-25T00:03:59.376 INFO:teuthology.orchestra.run.smithi107.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 656 kB/s | 33 kB 00:00 2024-04-25T00:03:59.401 INFO:teuthology.orchestra.run.smithi107.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-04-25T00:03:59.406 INFO:teuthology.orchestra.run.smithi195.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.8 MB/s | 141 kB 00:00 2024-04-25T00:03:59.452 INFO:teuthology.orchestra.run.smithi107.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-04-25T00:03:59.494 INFO:teuthology.orchestra.run.smithi107.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-04-25T00:03:59.577 INFO:teuthology.orchestra.run.smithi107.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-04-25T00:03:59.619 INFO:teuthology.orchestra.run.smithi107.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-04-25T00:03:59.636 INFO:teuthology.orchestra.run.smithi107.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 773 kB/s | 45 kB 00:00 2024-04-25T00:03:59.690 INFO:teuthology.orchestra.run.smithi195.stdout:(30/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 3.0 MB/s | 3.1 MB 00:01 2024-04-25T00:03:59.728 INFO:teuthology.orchestra.run.smithi107.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-04-25T00:03:59.753 INFO:teuthology.orchestra.run.smithi107.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 4.9 MB/s | 582 kB 00:00 2024-04-25T00:03:59.787 INFO:teuthology.orchestra.run.smithi107.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 436 kB/s | 25 kB 00:00 2024-04-25T00:03:59.804 INFO:teuthology.orchestra.run.smithi107.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-04-25T00:03:59.871 INFO:teuthology.orchestra.run.smithi107.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-25T00:03:59.896 INFO:teuthology.orchestra.run.smithi107.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-04-25T00:03:59.938 INFO:teuthology.orchestra.run.smithi107.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 525 kB/s | 35 kB 00:00 2024-04-25T00:04:00.130 INFO:teuthology.orchestra.run.smithi107.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-04-25T00:04:00.206 INFO:teuthology.orchestra.run.smithi107.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-25T00:04:00.224 INFO:teuthology.orchestra.run.smithi195.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 4.3 MB/s | 3.5 MB 00:00 2024-04-25T00:04:00.274 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-04-25T00:04:00.274 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-04-25T00:04:00.314 INFO:teuthology.orchestra.run.smithi107.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 8.2 MB/s | 911 kB 00:00 2024-04-25T00:04:00.358 INFO:teuthology.orchestra.run.smithi195.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 1.8 MB/s | 249 kB 00:00 2024-04-25T00:04:00.359 INFO:teuthology.orchestra.run.smithi107.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 7.0 MB/s | 1.6 MB 00:00 2024-04-25T00:04:00.406 INFO:teuthology.orchestra.run.smithi107.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-25T00:04:00.433 INFO:teuthology.orchestra.run.smithi195.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 658 kB/s | 49 kB 00:00 2024-04-25T00:04:00.491 INFO:teuthology.orchestra.run.smithi195.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 263 kB/s | 15 kB 00:00 2024-04-25T00:04:00.523 INFO:teuthology.orchestra.run.smithi107.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.6 MB/s | 1.0 MB 00:00 2024-04-25T00:04:00.590 INFO:teuthology.orchestra.run.smithi107.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-25T00:04:00.633 INFO:teuthology.orchestra.run.smithi195.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 1.8 MB/s | 264 kB 00:00 2024-04-25T00:04:00.666 INFO:teuthology.orchestra.run.smithi107.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-04-25T00:04:00.741 INFO:teuthology.orchestra.run.smithi107.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-04-25T00:04:00.816 INFO:teuthology.orchestra.run.smithi107.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-04-25T00:04:00.834 INFO:teuthology.orchestra.run.smithi195.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 3.7 MB/s | 762 kB 00:00 2024-04-25T00:04:00.874 INFO:teuthology.orchestra.run.smithi107.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 441 kB/s | 26 kB 00:00 2024-04-25T00:04:00.917 INFO:teuthology.orchestra.run.smithi195.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 219 kB/s | 18 kB 00:00 2024-04-25T00:04:00.967 INFO:teuthology.orchestra.run.smithi107.stdout:(77/152): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-04-25T00:04:00.984 INFO:teuthology.orchestra.run.smithi195.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 80 kB 00:00 2024-04-25T00:04:00.992 INFO:teuthology.orchestra.run.smithi107.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 164 kB/s | 19 kB 00:00 2024-04-25T00:04:01.076 INFO:teuthology.orchestra.run.smithi195.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 8.6 MB/s | 803 kB 00:00 2024-04-25T00:04:01.143 INFO:teuthology.orchestra.run.smithi195.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-04-25T00:04:01.218 INFO:teuthology.orchestra.run.smithi195.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-04-25T00:04:01.327 INFO:teuthology.orchestra.run.smithi195.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-04-25T00:04:01.395 INFO:teuthology.orchestra.run.smithi195.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-25T00:04:01.434 INFO:teuthology.orchestra.run.smithi107.stdout:(79/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 4.2 MB/s | 48 MB 00:11 2024-04-25T00:04:01.461 INFO:teuthology.orchestra.run.smithi195.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-25T00:04:01.476 INFO:teuthology.orchestra.run.smithi107.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 72 kB/s | 35 kB 00:00 2024-04-25T00:04:01.501 INFO:teuthology.orchestra.run.smithi107.stdout:(81/152): python3-mako-1.1.4-6.el9.noarch.rpm 322 kB/s | 172 kB 00:00 2024-04-25T00:04:01.528 INFO:teuthology.orchestra.run.smithi195.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-25T00:04:01.568 INFO:teuthology.orchestra.run.smithi107.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-04-25T00:04:01.602 INFO:teuthology.orchestra.run.smithi107.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-04-25T00:04:01.603 INFO:teuthology.orchestra.run.smithi195.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-04-25T00:04:01.660 INFO:teuthology.orchestra.run.smithi107.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 841 kB/s | 77 kB 00:00 2024-04-25T00:04:01.685 INFO:teuthology.orchestra.run.smithi107.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 507 kB/s | 42 kB 00:00 2024-04-25T00:04:01.745 INFO:teuthology.orchestra.run.smithi195.stdout:(47/152): ceph-mgr-diskprediction-local-19.0.0- 3.6 MB/s | 7.4 MB 00:02 2024-04-25T00:04:01.769 INFO:teuthology.orchestra.run.smithi107.stdout:(86/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-04-25T00:04:01.811 INFO:teuthology.orchestra.run.smithi107.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-04-25T00:04:01.836 INFO:teuthology.orchestra.run.smithi107.stdout:(88/152): python3-protobuf-3.14.0-13.el9.noarch 1.5 MB/s | 269 kB 00:00 2024-04-25T00:04:01.845 INFO:teuthology.orchestra.run.smithi195.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.8 MB/s | 689 kB 00:00 2024-04-25T00:04:01.878 INFO:teuthology.orchestra.run.smithi107.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-04-25T00:04:01.879 INFO:teuthology.orchestra.run.smithi195.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 107 kB 00:00 2024-04-25T00:04:01.903 INFO:teuthology.orchestra.run.smithi107.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 553 kB/s | 51 kB 00:00 2024-04-25T00:04:01.921 INFO:teuthology.orchestra.run.smithi195.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 5.3 MB/s | 943 kB 00:00 2024-04-25T00:04:01.928 INFO:teuthology.orchestra.run.smithi107.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 582 kB/s | 54 kB 00:00 2024-04-25T00:04:01.946 INFO:teuthology.orchestra.run.smithi195.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 322 kB/s | 21 kB 00:00 2024-04-25T00:04:01.972 INFO:teuthology.orchestra.run.smithi195.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T00:04:01.979 INFO:teuthology.orchestra.run.smithi107.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 554 kB/s | 42 kB 00:00 2024-04-25T00:04:01.997 INFO:teuthology.orchestra.run.smithi195.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 649 kB/s | 33 kB 00:00 2024-04-25T00:04:02.005 INFO:teuthology.orchestra.run.smithi107.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 587 kB/s | 44 kB 00:00 2024-04-25T00:04:02.039 INFO:teuthology.orchestra.run.smithi195.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-04-25T00:04:02.063 INFO:teuthology.orchestra.run.smithi107.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 614 kB/s | 36 kB 00:00 2024-04-25T00:04:02.088 INFO:teuthology.orchestra.run.smithi107.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-04-25T00:04:02.106 INFO:teuthology.orchestra.run.smithi195.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-04-25T00:04:02.131 INFO:teuthology.orchestra.run.smithi195.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 121 kB/s | 11 kB 00:00 2024-04-25T00:04:02.138 INFO:teuthology.orchestra.run.smithi107.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 632 kB/s | 47 kB 00:00 2024-04-25T00:04:02.157 INFO:teuthology.orchestra.run.smithi195.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-25T00:04:02.182 INFO:teuthology.orchestra.run.smithi195.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-04-25T00:04:02.189 INFO:teuthology.orchestra.run.smithi107.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-04-25T00:04:02.214 INFO:teuthology.orchestra.run.smithi107.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 771 kB/s | 58 kB 00:00 2024-04-25T00:04:02.224 INFO:teuthology.orchestra.run.smithi195.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-04-25T00:04:02.266 INFO:teuthology.orchestra.run.smithi195.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 582 kB 00:00 2024-04-25T00:04:02.273 INFO:teuthology.orchestra.run.smithi107.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-04-25T00:04:02.291 INFO:teuthology.orchestra.run.smithi195.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 380 kB/s | 25 kB 00:00 2024-04-25T00:04:02.298 INFO:teuthology.orchestra.run.smithi107.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 763 kB/s | 64 kB 00:00 2024-04-25T00:04:02.316 INFO:teuthology.orchestra.run.smithi195.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 949 kB/s | 47 kB 00:00 2024-04-25T00:04:02.342 INFO:teuthology.orchestra.run.smithi195.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T00:04:02.357 INFO:teuthology.orchestra.run.smithi107.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 536 kB/s | 45 kB 00:00 2024-04-25T00:04:02.367 INFO:teuthology.orchestra.run.smithi195.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T00:04:02.392 INFO:teuthology.orchestra.run.smithi195.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 702 kB/s | 35 kB 00:00 2024-04-25T00:04:02.484 INFO:teuthology.orchestra.run.smithi195.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 41 MB/s | 4.7 MB 00:00 2024-04-25T00:04:02.548 INFO:teuthology.orchestra.run.smithi107.stdout:(102/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-04-25T00:04:02.551 INFO:teuthology.orchestra.run.smithi195.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-04-25T00:04:02.593 INFO:teuthology.orchestra.run.smithi195.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-25T00:04:02.627 INFO:teuthology.orchestra.run.smithi195.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-25T00:04:02.652 INFO:teuthology.orchestra.run.smithi195.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-25T00:04:02.702 INFO:teuthology.orchestra.run.smithi195.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-04-25T00:04:02.777 INFO:teuthology.orchestra.run.smithi195.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 38 MB/s | 5.8 MB 00:00 2024-04-25T00:04:02.782 INFO:teuthology.orchestra.run.smithi107.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 46 kB/s | 22 kB 00:00 2024-04-25T00:04:02.803 INFO:teuthology.orchestra.run.smithi195.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 875 kB/s | 87 kB 00:00 2024-04-25T00:04:02.845 INFO:teuthology.orchestra.run.smithi195.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 5.8 MB/s | 244 kB 00:00 2024-04-25T00:04:02.920 INFO:teuthology.orchestra.run.smithi195.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 42 MB/s | 6.0 MB 00:00 2024-04-25T00:04:02.945 INFO:teuthology.orchestra.run.smithi195.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-04-25T00:04:02.970 INFO:teuthology.orchestra.run.smithi195.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-04-25T00:04:02.996 INFO:teuthology.orchestra.run.smithi195.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-04-25T00:04:03.022 INFO:teuthology.orchestra.run.smithi195.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-25T00:04:03.046 INFO:teuthology.orchestra.run.smithi195.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-04-25T00:04:03.090 INFO:teuthology.orchestra.run.smithi195.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.6 MB/s | 250 kB 00:00 2024-04-25T00:04:03.166 INFO:teuthology.orchestra.run.smithi195.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 35 MB/s | 5.1 MB 00:00 2024-04-25T00:04:03.191 INFO:teuthology.orchestra.run.smithi195.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-04-25T00:04:03.216 INFO:teuthology.orchestra.run.smithi195.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T00:04:03.242 INFO:teuthology.orchestra.run.smithi195.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-04-25T00:04:03.267 INFO:teuthology.orchestra.run.smithi195.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-25T00:04:03.292 INFO:teuthology.orchestra.run.smithi195.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T00:04:03.308 INFO:teuthology.orchestra.run.smithi107.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 723 kB/s | 548 kB 00:00 2024-04-25T00:04:03.317 INFO:teuthology.orchestra.run.smithi195.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-04-25T00:04:03.359 INFO:teuthology.orchestra.run.smithi195.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 762 kB/s | 51 kB 00:00 2024-04-25T00:04:03.374 INFO:teuthology.orchestra.run.smithi107.stdout:(105/152): protobuf-compiler-3.14.0-13.el9.x86_ 849 kB/s | 863 kB 00:01 2024-04-25T00:04:03.385 INFO:teuthology.orchestra.run.smithi195.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 800 kB/s | 54 kB 00:00 2024-04-25T00:04:03.399 INFO:teuthology.orchestra.run.smithi107.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 212 kB/s | 19 kB 00:00 2024-04-25T00:04:03.427 INFO:teuthology.orchestra.run.smithi195.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-04-25T00:04:03.434 INFO:teuthology.orchestra.run.smithi107.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 474 kB/s | 309 kB 00:00 2024-04-25T00:04:03.460 INFO:teuthology.orchestra.run.smithi195.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-04-25T00:04:03.492 INFO:teuthology.orchestra.run.smithi107.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 282 kB/s | 26 kB 00:00 2024-04-25T00:04:03.502 INFO:teuthology.orchestra.run.smithi195.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-04-25T00:04:03.536 INFO:teuthology.orchestra.run.smithi195.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 1.1 MB/s | 36 kB 00:00 2024-04-25T00:04:03.550 INFO:teuthology.orchestra.run.smithi107.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 416 kB/s | 49 kB 00:00 2024-04-25T00:04:03.592 INFO:teuthology.orchestra.run.smithi107.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 672 kB/s | 67 kB 00:00 2024-04-25T00:04:03.667 INFO:teuthology.orchestra.run.smithi107.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-25T00:04:03.695 INFO:teuthology.orchestra.run.smithi195.stdout:(95/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 49 MB/s | 16 MB 00:00 2024-04-25T00:04:03.720 INFO:teuthology.orchestra.run.smithi195.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 258 kB/s | 47 kB 00:00 2024-04-25T00:04:03.745 INFO:teuthology.orchestra.run.smithi195.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-04-25T00:04:03.770 INFO:teuthology.orchestra.run.smithi195.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T00:04:03.796 INFO:teuthology.orchestra.run.smithi195.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-25T00:04:03.809 INFO:teuthology.orchestra.run.smithi107.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.8 MB/s | 838 kB 00:00 2024-04-25T00:04:03.821 INFO:teuthology.orchestra.run.smithi195.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T00:04:03.886 INFO:teuthology.orchestra.run.smithi195.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 496 kB/s | 45 kB 00:00 2024-04-25T00:04:03.901 INFO:teuthology.orchestra.run.smithi107.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 2.3 MB/s | 548 kB 00:00 2024-04-25T00:04:03.927 INFO:teuthology.orchestra.run.smithi107.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 371 kB/s | 43 kB 00:00 2024-04-25T00:04:04.010 INFO:teuthology.orchestra.run.smithi107.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 297 kB/s | 32 kB 00:00 2024-04-25T00:04:04.027 INFO:teuthology.orchestra.run.smithi107.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2024-04-25T00:04:04.111 INFO:teuthology.orchestra.run.smithi107.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-25T00:04:04.145 INFO:teuthology.orchestra.run.smithi107.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-04-25T00:04:04.155 INFO:teuthology.orchestra.run.smithi195.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 67 kB/s | 22 kB 00:00 2024-04-25T00:04:04.219 INFO:teuthology.orchestra.run.smithi107.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-25T00:04:04.236 INFO:teuthology.orchestra.run.smithi107.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 385 kB/s | 35 kB 00:00 2024-04-25T00:04:04.312 INFO:teuthology.orchestra.run.smithi107.stdout:(121/152): libarrow-9.0.0-10.el9.x86_64.rpm 4.7 MB/s | 4.4 MB 00:00 2024-04-25T00:04:04.337 INFO:teuthology.orchestra.run.smithi107.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-04-25T00:04:04.404 INFO:teuthology.orchestra.run.smithi107.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-25T00:04:04.431 INFO:teuthology.orchestra.run.smithi107.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 114 kB/s | 11 kB 00:00 2024-04-25T00:04:04.482 INFO:teuthology.orchestra.run.smithi107.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 8.3 MB/s | 2.0 MB 00:00 2024-04-25T00:04:04.507 INFO:teuthology.orchestra.run.smithi107.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 172 kB/s | 18 kB 00:00 2024-04-25T00:04:04.532 INFO:teuthology.orchestra.run.smithi107.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-04-25T00:04:04.599 INFO:teuthology.orchestra.run.smithi107.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-04-25T00:04:04.624 INFO:teuthology.orchestra.run.smithi107.stdout:(129/152): python3-jaraco-functools-3.5.0-2.el9 137 kB/s | 19 kB 00:00 2024-04-25T00:04:04.675 INFO:teuthology.orchestra.run.smithi107.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-04-25T00:04:04.688 INFO:teuthology.orchestra.run.smithi195.stdout:(103/152): ceph-test-19.0.0-2445.gd2c45d97.el9. 5.3 MB/s | 48 MB 00:09 2024-04-25T00:04:04.700 INFO:teuthology.orchestra.run.smithi107.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 461 kB/s | 46 kB 00:00 2024-04-25T00:04:04.725 INFO:teuthology.orchestra.run.smithi107.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 784 kB/s | 79 kB 00:00 2024-04-25T00:04:04.767 INFO:teuthology.orchestra.run.smithi107.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 626 kB/s | 58 kB 00:00 2024-04-25T00:04:04.792 INFO:teuthology.orchestra.run.smithi107.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-25T00:04:04.818 INFO:teuthology.orchestra.run.smithi107.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 178 kB/s | 16 kB 00:00 2024-04-25T00:04:04.868 INFO:teuthology.orchestra.run.smithi107.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 895 kB/s | 90 kB 00:00 2024-04-25T00:04:04.893 INFO:teuthology.orchestra.run.smithi107.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 281 kB/s | 28 kB 00:00 2024-04-25T00:04:04.919 INFO:teuthology.orchestra.run.smithi107.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 306 kB/s | 31 kB 00:00 2024-04-25T00:04:04.969 INFO:teuthology.orchestra.run.smithi107.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-25T00:04:04.994 INFO:teuthology.orchestra.run.smithi107.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 588 kB/s | 59 kB 00:00 2024-04-25T00:04:05.019 INFO:teuthology.orchestra.run.smithi107.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 356 kB/s | 36 kB 00:00 2024-04-25T00:04:05.038 INFO:teuthology.orchestra.run.smithi195.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 750 kB/s | 863 kB 00:01 2024-04-25T00:04:05.061 INFO:teuthology.orchestra.run.smithi107.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 553 kB/s | 51 kB 00:00 2024-04-25T00:04:05.095 INFO:teuthology.orchestra.run.smithi107.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-25T00:04:05.120 INFO:teuthology.orchestra.run.smithi107.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 891 kB/s | 90 kB 00:00 2024-04-25T00:04:05.139 INFO:teuthology.orchestra.run.smithi195.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 686 kB/s | 309 kB 00:00 2024-04-25T00:04:05.170 INFO:teuthology.orchestra.run.smithi107.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-04-25T00:04:05.196 INFO:teuthology.orchestra.run.smithi107.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-04-25T00:04:05.221 INFO:teuthology.orchestra.run.smithi107.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-25T00:04:05.222 INFO:teuthology.orchestra.run.smithi195.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 513 kB/s | 548 kB 00:01 2024-04-25T00:04:05.306 INFO:teuthology.orchestra.run.smithi195.stdout:(107/152): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-04-25T00:04:05.331 INFO:teuthology.orchestra.run.smithi195.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 240 kB/s | 26 kB 00:00 2024-04-25T00:04:05.338 INFO:teuthology.orchestra.run.smithi107.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-04-25T00:04:05.431 INFO:teuthology.orchestra.run.smithi195.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 675 kB/s | 67 kB 00:00 2024-04-25T00:04:05.474 INFO:teuthology.orchestra.run.smithi195.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 290 kB/s | 49 kB 00:00 2024-04-25T00:04:05.546 INFO:teuthology.orchestra.run.smithi107.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 6.5 MB/s | 1.3 MB 00:00 2024-04-25T00:04:05.574 INFO:teuthology.orchestra.run.smithi195.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-04-25T00:04:05.608 INFO:teuthology.orchestra.run.smithi195.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 859 kB/s | 151 kB 00:00 2024-04-25T00:04:05.663 INFO:teuthology.orchestra.run.smithi107.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 21 MB/s | 2.4 MB 00:00 2024-04-25T00:04:05.700 INFO:teuthology.orchestra.run.smithi195.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-04-25T00:04:05.725 INFO:teuthology.orchestra.run.smithi195.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 372 kB/s | 43 kB 00:00 2024-04-25T00:04:05.792 INFO:teuthology.orchestra.run.smithi195.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 350 kB/s | 32 kB 00:00 2024-04-25T00:04:05.817 INFO:teuthology.orchestra.run.smithi195.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-04-25T00:04:05.892 INFO:teuthology.orchestra.run.smithi195.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-25T00:04:05.967 INFO:teuthology.orchestra.run.smithi195.stdout:(118/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.7 MB/s | 838 kB 00:00 2024-04-25T00:04:06.001 INFO:teuthology.orchestra.run.smithi195.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-25T00:04:06.060 INFO:teuthology.orchestra.run.smithi195.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 384 kB/s | 35 kB 00:00 2024-04-25T00:04:06.102 INFO:teuthology.orchestra.run.smithi195.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-04-25T00:04:06.143 INFO:teuthology.orchestra.run.smithi195.stdout:(122/152): python3-cherrypy-18.6.1-2.el9.noarch 1.1 MB/s | 358 kB 00:00 2024-04-25T00:04:06.180 INFO:teuthology.orchestra.run.smithi107.stdout:(151/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 3.3 MB/s | 3.1 MB 00:00 2024-04-25T00:04:06.194 INFO:teuthology.orchestra.run.smithi195.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-25T00:04:06.230 INFO:teuthology.orchestra.run.smithi107.stdout:(152/152): librados2-19.0.0-2445.gd2c45d97.el9. 3.3 MB/s | 3.4 MB 00:01 2024-04-25T00:04:06.233 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-25T00:04:06.233 INFO:teuthology.orchestra.run.smithi107.stdout:Total 9.4 MB/s | 220 MB 00:23 2024-04-25T00:04:06.235 INFO:teuthology.orchestra.run.smithi195.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-04-25T00:04:06.286 INFO:teuthology.orchestra.run.smithi195.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-04-25T00:04:06.312 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-04-25T00:04:06.328 INFO:teuthology.orchestra.run.smithi195.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 252 kB/s | 23 kB 00:00 2024-04-25T00:04:06.378 INFO:teuthology.orchestra.run.smithi195.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-04-25T00:04:06.398 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T00:04:06.420 INFO:teuthology.orchestra.run.smithi195.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-04-25T00:04:06.444 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T00:04:06.495 INFO:teuthology.orchestra.run.smithi195.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-04-25T00:04:06.497 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T00:04:06.521 INFO:teuthology.orchestra.run.smithi195.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 462 kB/s | 46 kB 00:00 2024-04-25T00:04:06.549 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T00:04:06.583 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T00:04:06.587 INFO:teuthology.orchestra.run.smithi195.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 859 kB/s | 79 kB 00:00 2024-04-25T00:04:06.613 INFO:teuthology.orchestra.run.smithi195.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 628 kB/s | 58 kB 00:00 2024-04-25T00:04:06.633 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T00:04:06.688 INFO:teuthology.orchestra.run.smithi195.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-04-25T00:04:06.705 INFO:teuthology.orchestra.run.smithi195.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-04-25T00:04:06.780 INFO:teuthology.orchestra.run.smithi195.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 980 kB/s | 90 kB 00:00 2024-04-25T00:04:06.797 INFO:teuthology.orchestra.run.smithi195.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 307 kB/s | 28 kB 00:00 2024-04-25T00:04:06.839 INFO:teuthology.orchestra.run.smithi195.stdout:(137/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.6 MB/s | 2.0 MB 00:00 2024-04-25T00:04:06.864 INFO:teuthology.orchestra.run.smithi195.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-04-25T00:04:06.934 INFO:teuthology.orchestra.run.smithi195.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 643 kB/s | 59 kB 00:00 2024-04-25T00:04:06.957 INFO:teuthology.orchestra.run.smithi195.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 388 kB/s | 36 kB 00:00 2024-04-25T00:04:06.964 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T00:04:06.982 INFO:teuthology.orchestra.run.smithi195.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 1.0 MB/s | 188 kB 00:00 2024-04-25T00:04:07.024 INFO:teuthology.orchestra.run.smithi195.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 552 kB/s | 51 kB 00:00 2024-04-25T00:04:07.058 INFO:teuthology.orchestra.run.smithi195.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-25T00:04:07.075 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:04:07.100 INFO:teuthology.orchestra.run.smithi195.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 765 kB/s | 90 kB 00:00 2024-04-25T00:04:07.150 INFO:teuthology.orchestra.run.smithi195.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-04-25T00:04:07.197 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:04:07.200 INFO:teuthology.orchestra.run.smithi195.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.4 MB/s | 427 kB 00:00 2024-04-25T00:04:07.227 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:04:07.276 INFO:teuthology.orchestra.run.smithi195.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-04-25T00:04:07.307 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-04-25T00:04:07.318 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:04:07.356 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:04:07.451 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-04-25T00:04:07.451 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-04-25T00:04:07.509 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T00:04:07.561 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T00:04:07.567 INFO:teuthology.orchestra.run.smithi195.stdout:(148/152): librados2-19.0.0-2445.gd2c45d97.el9. 9.2 MB/s | 3.4 MB 00:00 2024-04-25T00:04:07.589 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:04:07.684 INFO:teuthology.orchestra.run.smithi195.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 12 MB/s | 1.3 MB 00:00 2024-04-25T00:04:07.693 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:04:07.715 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:04:07.778 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:04:07.800 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:04:07.823 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:04:07.835 INFO:teuthology.orchestra.run.smithi195.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-04-25T00:04:07.967 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:04:07.999 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:04:08.010 INFO:teuthology.orchestra.run.smithi195.stdout:(151/152): thrift-0.15.0-2.el9.x86_64.rpm 1.8 MB/s | 1.6 MB 00:00 2024-04-25T00:04:08.106 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T00:04:08.170 INFO:teuthology.orchestra.run.smithi195.stdout:(152/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 3.5 MB/s | 3.1 MB 00:00 2024-04-25T00:04:08.171 INFO:teuthology.orchestra.run.smithi195.stdout:-------------------------------------------------------------------------------- 2024-04-25T00:04:08.171 INFO:teuthology.orchestra.run.smithi195.stdout:Total 8.2 MB/s | 220 MB 00:26 2024-04-25T00:04:08.193 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T00:04:08.224 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T00:04:08.267 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T00:04:08.349 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T00:04:08.392 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T00:04:08.436 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T00:04:08.476 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T00:04:08.504 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T00:04:08.574 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T00:04:08.635 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T00:04:08.666 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T00:04:08.710 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T00:04:08.795 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T00:04:08.828 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-25T00:04:08.872 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-25T00:04:08.900 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-25T00:04:08.970 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T00:04:09.015 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T00:04:09.089 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T00:04:09.218 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-04-25T00:04:09.219 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-04-25T00:04:09.223 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction check 2024-04-25T00:04:09.224 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T00:04:09.284 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T00:04:09.367 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction check succeeded. 2024-04-25T00:04:09.367 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction test 2024-04-25T00:04:09.806 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T00:04:09.888 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T00:04:09.917 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T00:04:09.960 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T00:04:09.998 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-25T00:04:10.063 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T00:04:10.217 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T00:04:10.276 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T00:04:10.330 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T00:04:10.403 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T00:04:10.435 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T00:04:10.472 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T00:04:10.519 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T00:04:10.562 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T00:04:10.606 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T00:04:10.648 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T00:04:10.782 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T00:04:10.828 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T00:04:10.859 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T00:04:10.901 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T00:04:10.994 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T00:04:11.051 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T00:04:11.091 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T00:04:11.121 INFO:teuthology.orchestra.run.smithi195.stdout:Transaction test succeeded. 2024-04-25T00:04:11.122 INFO:teuthology.orchestra.run.smithi195.stdout:Running transaction 2024-04-25T00:04:11.124 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T00:04:11.149 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T00:04:11.223 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T00:04:11.358 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T00:04:11.390 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T00:04:11.427 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T00:04:11.461 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T00:04:11.500 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T00:04:11.532 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T00:04:11.561 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T00:04:11.596 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T00:04:11.638 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T00:04:11.688 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T00:04:11.722 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T00:04:11.892 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T00:04:12.003 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T00:04:12.034 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T00:04:12.081 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T00:04:12.186 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T00:04:12.244 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T00:04:12.960 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T00:04:12.999 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T00:04:13.033 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T00:04:13.130 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T00:04:13.182 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T00:04:13.263 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T00:04:13.311 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T00:04:13.487 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T00:04:13.517 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T00:04:13.594 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T00:04:14.232 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T00:04:14.347 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T00:04:15.280 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-04-25T00:04:15.359 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T00:04:15.397 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T00:04:15.450 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T00:04:15.485 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T00:04:15.510 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T00:04:15.553 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T00:04:15.562 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T00:04:15.654 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T00:04:15.683 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T00:04:15.716 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T00:04:15.875 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T00:04:15.978 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:04:16.026 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T00:04:16.079 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:04:16.099 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:04:16.138 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:04:16.161 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:04:16.192 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-25T00:04:16.228 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T00:04:16.238 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:04:16.267 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:04:16.415 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T00:04:16.464 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T00:04:16.492 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:04:16.594 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:04:16.618 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:04:16.680 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:04:16.702 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:04:16.705 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T00:04:16.720 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:04:16.740 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:04:16.811 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:04:16.842 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-25T00:04:16.919 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:04:16.952 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:04:17.059 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T00:04:17.082 INFO:teuthology.orchestra.run.smithi195.stdout: Preparing : 1/1 2024-04-25T00:04:17.154 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T00:04:17.180 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T00:04:17.201 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T00:04:17.226 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T00:04:17.235 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T00:04:17.280 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T00:04:17.300 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T00:04:17.314 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T00:04:17.334 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T00:04:17.340 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T00:04:17.382 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T00:04:17.386 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T00:04:17.419 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T00:04:17.446 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T00:04:17.515 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T00:04:17.568 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T00:04:17.599 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T00:04:17.635 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T00:04:17.711 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T00:04:17.712 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T00:04:17.746 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-25T00:04:17.781 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-25T00:04:17.810 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-25T00:04:17.816 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:04:17.878 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T00:04:17.924 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T00:04:17.937 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:04:17.967 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:04:17.998 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T00:04:18.050 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:04:18.088 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:04:18.130 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T00:04:18.184 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T00:04:18.241 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T00:04:18.369 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T00:04:18.396 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:04:18.491 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:04:18.514 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:04:18.560 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:04:18.582 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:04:18.607 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:04:18.711 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T00:04:18.789 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T00:04:18.814 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:04:18.826 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T00:04:18.848 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:04:18.901 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T00:04:18.907 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-25T00:04:18.949 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T00:04:18.956 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T00:04:19.052 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T00:04:19.089 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T00:04:19.098 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T00:04:19.121 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:04:19.132 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T00:04:19.143 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T00:04:19.155 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:04:19.197 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T00:04:19.198 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T00:04:19.231 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T00:04:19.270 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T00:04:19.282 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T00:04:19.302 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T00:04:19.315 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T00:04:19.330 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T00:04:19.343 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T00:04:19.377 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T00:04:19.413 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T00:04:19.413 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T00:04:19.456 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T00:04:19.466 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T00:04:19.497 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T00:04:19.498 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T00:04:19.531 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T00:04:19.608 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T00:04:19.634 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T00:04:19.642 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-25T00:04:19.677 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-25T00:04:19.687 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T00:04:19.695 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:04:19.706 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-25T00:04:19.717 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T00:04:19.742 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:04:19.758 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T00:04:19.778 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T00:04:19.821 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T00:04:19.838 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:04:19.838 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T00:04:19.838 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T00:04:19.838 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-25T00:04:19.852 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T00:04:19.876 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:04:19.892 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T00:04:19.895 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T00:04:19.932 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T00:04:19.958 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T00:04:19.983 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T00:04:20.031 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T00:04:20.058 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T00:04:20.082 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T00:04:20.209 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T00:04:20.240 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T00:04:20.268 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T00:04:20.302 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T00:04:20.341 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T00:04:20.373 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T00:04:20.402 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T00:04:20.437 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T00:04:20.470 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T00:04:20.512 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T00:04:20.546 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T00:04:20.611 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T00:04:20.635 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T00:04:20.677 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T00:04:20.694 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T00:04:20.708 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T00:04:20.723 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T00:04:20.747 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T00:04:20.766 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T00:04:20.804 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-25T00:04:20.845 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T00:04:20.846 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T00:04:20.893 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T00:04:21.008 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T00:04:21.065 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T00:04:21.120 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T00:04:21.193 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T00:04:21.224 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T00:04:21.269 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T00:04:21.316 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T00:04:21.351 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T00:04:21.394 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T00:04:21.437 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T00:04:21.573 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T00:04:21.612 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T00:04:21.626 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T00:04:21.656 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T00:04:21.665 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T00:04:21.697 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T00:04:21.698 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T00:04:21.792 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T00:04:21.796 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T00:04:21.830 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T00:04:21.847 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T00:04:21.879 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T00:04:21.904 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T00:04:21.929 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T00:04:21.935 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T00:04:21.984 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T00:04:22.005 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T00:04:22.139 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T00:04:22.161 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T00:04:22.179 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T00:04:22.198 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T00:04:22.206 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T00:04:22.240 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T00:04:22.276 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T00:04:22.280 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T00:04:22.311 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T00:04:22.341 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T00:04:22.376 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T00:04:22.409 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T00:04:22.442 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T00:04:22.476 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T00:04:22.564 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T00:04:22.607 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T00:04:22.638 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T00:04:22.686 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T00:04:22.785 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T00:04:22.848 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T00:04:22.909 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T00:04:23.022 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T00:04:23.581 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T00:04:23.620 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T00:04:23.654 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T00:04:23.756 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T00:04:23.806 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T00:04:24.042 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T00:04:24.122 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T00:04:24.229 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T00:04:24.300 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T00:04:24.310 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T00:04:24.330 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T00:04:24.340 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T00:04:24.371 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T00:04:24.406 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T00:04:24.682 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T00:04:24.718 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:04:24.807 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:04:24.839 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-25T00:04:24.875 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T00:04:25.062 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T00:04:25.186 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T00:04:25.344 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T00:04:25.371 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:04:25.425 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:04:25.456 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-25T00:04:26.448 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T00:04:26.541 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T00:04:26.568 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T00:04:26.602 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T00:04:26.914 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T00:04:26.956 INFO:teuthology.orchestra.run.smithi195.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:04:27.072 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:04:27.102 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-25T00:04:27.138 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T00:04:27.606 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T00:04:27.634 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:04:27.705 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:04:27.713 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:04:27.727 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-25T00:04:27.752 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:04:28.291 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:04:28.370 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:04:28.729 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:04:28.730 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T00:04:28.730 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T00:04:28.730 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:04:28.756 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:04:29.987 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:04:30.024 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:04:30.558 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:04:30.611 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:04:30.702 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:04:30.703 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T00:04:30.703 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T00:04:30.703 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-04-25T00:04:30.728 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:04:31.258 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:04:31.258 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /sys 2024-04-25T00:04:31.258 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /proc 2024-04-25T00:04:31.258 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /mnt 2024-04-25T00:04:31.259 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /var/tmp 2024-04-25T00:04:31.259 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /home 2024-04-25T00:04:31.259 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /root 2024-04-25T00:04:31.259 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /tmp 2024-04-25T00:04:31.259 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-25T00:04:31.431 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:04:31.453 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:04:31.453 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:31.453 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T00:04:31.453 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-25T00:04:32.926 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:04:32.951 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:04:32.951 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:32.952 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T00:04:32.952 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:04:32.952 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:04:32.952 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-25T00:04:33.177 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:04:33.199 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:04:33.199 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:33.199 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T00:04:33.199 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:04:33.199 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:04:33.199 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-25T00:04:33.644 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T00:04:33.667 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T00:04:33.667 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:33.667 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T00:04:33.667 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T00:04:33.667 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T00:04:33.667 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-25T00:04:33.774 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T00:04:33.823 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T00:04:33.878 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T00:04:34.062 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T00:04:34.111 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T00:04:34.197 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T00:04:34.285 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T00:04:34.425 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T00:04:34.462 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T00:04:34.494 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T00:04:34.530 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T00:04:34.987 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T00:04:35.012 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T00:04:35.044 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-25T00:04:35.105 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-25T00:04:35.144 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T00:04:36.173 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T00:04:36.212 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T00:04:37.172 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T00:04:37.204 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T00:04:37.306 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T00:04:37.467 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-25T00:04:37.490 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T00:04:37.512 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T00:04:37.512 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:37.513 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T00:04:37.513 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T00:04:37.513 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T00:04:37.513 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-25T00:04:37.557 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T00:04:37.568 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T00:04:37.665 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-25T00:04:37.714 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-25T00:04:38.615 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T00:04:38.651 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T00:04:38.673 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T00:04:38.674 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:38.674 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T00:04:38.674 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:04:38.674 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:04:38.674 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-25T00:04:38.718 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T00:04:38.740 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T00:04:38.741 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:38.741 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T00:04:38.741 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-25T00:04:38.786 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T00:04:38.808 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T00:04:38.808 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:38.808 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T00:04:38.809 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:04:38.809 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:04:38.809 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-25T00:04:39.120 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T00:04:39.143 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T00:04:39.143 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:39.143 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T00:04:39.143 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T00:04:39.143 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T00:04:39.144 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-25T00:04:40.163 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:04:40.163 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /sys 2024-04-25T00:04:40.163 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /proc 2024-04-25T00:04:40.164 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /mnt 2024-04-25T00:04:40.164 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /var/tmp 2024-04-25T00:04:40.164 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /home 2024-04-25T00:04:40.164 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /root 2024-04-25T00:04:40.164 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /tmp 2024-04-25T00:04:40.164 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:04:40.337 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:04:40.350 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:04:40.350 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:40.350 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T00:04:40.350 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:04:41.813 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:04:41.838 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:04:41.838 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:41.838 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T00:04:41.839 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:04:41.839 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:04:41.839 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:04:42.063 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:04:42.085 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:04:42.086 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:42.086 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T00:04:42.086 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:04:42.086 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:04:42.086 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:04:42.303 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:04:42.303 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /sys 2024-04-25T00:04:42.304 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /proc 2024-04-25T00:04:42.304 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /mnt 2024-04-25T00:04:42.304 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /var/tmp 2024-04-25T00:04:42.304 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /home 2024-04-25T00:04:42.304 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /root 2024-04-25T00:04:42.304 INFO:teuthology.orchestra.run.smithi195.stdout:skipping the directory /tmp 2024-04-25T00:04:42.304 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-04-25T00:04:42.474 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:04:42.489 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:04:42.489 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:42.489 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T00:04:42.489 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-04-25T00:04:42.531 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T00:04:42.552 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T00:04:42.552 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:42.552 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T00:04:42.553 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T00:04:42.553 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T00:04:42.553 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:04:42.661 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T00:04:42.718 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T00:04:42.773 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T00:04:42.958 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T00:04:43.015 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T00:04:43.093 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T00:04:43.190 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T00:04:43.329 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T00:04:43.365 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T00:04:43.396 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T00:04:43.424 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T00:04:43.641 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-25T00:04:43.682 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-25T00:04:43.731 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-25T00:04:43.771 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T00:04:43.798 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T00:04:43.848 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T00:04:43.873 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T00:04:43.937 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-25T00:04:43.952 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:04:43.977 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:04:43.977 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:43.978 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T00:04:43.978 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:04:43.978 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:04:43.978 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-04-25T00:04:43.979 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-25T00:04:43.999 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-25T00:04:44.020 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T00:04:44.038 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T00:04:44.050 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-25T00:04:44.211 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:04:44.233 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:04:44.233 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:44.233 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T00:04:44.233 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:04:44.233 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:04:44.233 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-04-25T00:04:44.570 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:04:44.606 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:04:44.626 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:04:44.670 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T00:04:44.692 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T00:04:44.692 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:44.692 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T00:04:44.692 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T00:04:44.693 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T00:04:44.693 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-04-25T00:04:44.800 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T00:04:44.857 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T00:04:44.913 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T00:04:45.070 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T00:04:45.098 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T00:04:45.114 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T00:04:45.146 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T00:04:45.223 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T00:04:45.312 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T00:04:45.453 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T00:04:45.455 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-25T00:04:45.504 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T00:04:45.536 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T00:04:45.553 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T00:04:45.563 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T00:04:45.594 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T00:04:45.675 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T00:04:45.707 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T00:04:45.707 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T00:04:45.791 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T00:04:45.824 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T00:04:45.824 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T00:04:45.925 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T00:04:45.989 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:04:46.029 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T00:04:46.054 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T00:04:46.080 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T00:04:46.086 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-25T00:04:46.106 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T00:04:46.148 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-25T00:04:46.194 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T00:04:46.207 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T00:04:46.368 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-25T00:04:46.408 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T00:04:46.431 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T00:04:46.431 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:46.431 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T00:04:46.431 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T00:04:46.431 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T00:04:46.432 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:04:46.476 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T00:04:46.486 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T00:04:46.585 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-25T00:04:46.633 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-25T00:04:47.230 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T00:04:47.262 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T00:04:47.522 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T00:04:47.570 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T00:04:47.592 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T00:04:47.592 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:47.592 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T00:04:47.592 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:04:47.593 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:04:47.593 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:04:47.637 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T00:04:47.659 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T00:04:47.659 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:47.659 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T00:04:47.659 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:04:47.704 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T00:04:47.726 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T00:04:47.726 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:47.726 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T00:04:47.726 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:04:47.726 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:04:47.726 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:04:48.030 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T00:04:48.052 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T00:04:48.052 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:48.052 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T00:04:48.052 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T00:04:48.052 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T00:04:48.052 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:04:48.229 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T00:04:48.254 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T00:04:48.356 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T00:04:48.527 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-25T00:04:48.565 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T00:04:48.587 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T00:04:48.587 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:48.587 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T00:04:48.587 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T00:04:48.587 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T00:04:48.588 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-04-25T00:04:48.632 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T00:04:48.643 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T00:04:48.740 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-25T00:04:48.789 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-25T00:04:49.686 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T00:04:49.718 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T00:04:49.741 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T00:04:49.741 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:49.741 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T00:04:49.742 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:04:49.742 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:04:49.742 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-04-25T00:04:49.785 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T00:04:49.807 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T00:04:49.807 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:49.807 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T00:04:49.807 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-04-25T00:04:49.852 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T00:04:49.874 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T00:04:49.874 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:49.875 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T00:04:49.875 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:04:49.875 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:04:49.875 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-04-25T00:04:50.178 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T00:04:50.200 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T00:04:50.200 INFO:teuthology.orchestra.run.smithi195.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:04:50.200 INFO:teuthology.orchestra.run.smithi195.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T00:04:50.201 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T00:04:50.201 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T00:04:50.201 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-04-25T00:04:51.914 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:04:51.914 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-25T00:04:51.914 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-25T00:04:51.914 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-25T00:04:51.914 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-25T00:04:51.914 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-25T00:04:51.914 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-25T00:04:51.914 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-25T00:04:51.914 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:04:51.914 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:04:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:04:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-25T00:04:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-25T00:04:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:04:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-25T00:04:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-25T00:04:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-25T00:04:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-25T00:04:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-25T00:04:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-25T00:04:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-25T00:04:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-25T00:04:51.915 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-25T00:04:51.916 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-25T00:04:51.916 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-25T00:04:51.916 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-25T00:04:51.916 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-25T00:04:51.916 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-25T00:04:51.916 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-25T00:04:51.916 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-25T00:04:51.916 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-25T00:04:51.916 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-25T00:04:51.929 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-25T00:04:51.929 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-25T00:04:51.929 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-25T00:04:51.929 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-25T00:04:51.929 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-25T00:04:51.929 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-25T00:04:51.929 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-25T00:04:51.929 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-25T00:04:51.929 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T00:04:51.929 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T00:04:51.929 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T00:04:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T00:04:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T00:04:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T00:04:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T00:04:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T00:04:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T00:04:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T00:04:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T00:04:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T00:04:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T00:04:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T00:04:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T00:04:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T00:04:51.930 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T00:04:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T00:04:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T00:04:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T00:04:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T00:04:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T00:04:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T00:04:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T00:04:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T00:04:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T00:04:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T00:04:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T00:04:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T00:04:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T00:04:51.931 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T00:04:51.932 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T00:04:51.932 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T00:04:51.932 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T00:04:51.932 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T00:04:51.932 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T00:04:51.932 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T00:04:51.932 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T00:04:51.932 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T00:04:51.932 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T00:04:51.932 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T00:04:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T00:04:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T00:04:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T00:04:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T00:04:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T00:04:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T00:04:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T00:04:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T00:04:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T00:04:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T00:04:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T00:04:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T00:04:51.933 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T00:04:51.934 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T00:04:51.934 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T00:04:51.934 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T00:04:51.934 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T00:04:51.934 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T00:04:51.934 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T00:04:51.934 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T00:04:51.934 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T00:04:51.934 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T00:04:51.934 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T00:04:51.934 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T00:04:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T00:04:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T00:04:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T00:04:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T00:04:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T00:04:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T00:04:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T00:04:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T00:04:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T00:04:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T00:04:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T00:04:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T00:04:51.935 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T00:04:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T00:04:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T00:04:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T00:04:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T00:04:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T00:04:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T00:04:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T00:04:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T00:04:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T00:04:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T00:04:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T00:04:51.936 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T00:04:51.937 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T00:04:51.937 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T00:04:51.937 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T00:04:51.937 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T00:04:51.937 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T00:04:51.937 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T00:04:51.937 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T00:04:51.937 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T00:04:51.937 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T00:04:51.937 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T00:04:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T00:04:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T00:04:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T00:04:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T00:04:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T00:04:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T00:04:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T00:04:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T00:04:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T00:04:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-25T00:04:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T00:04:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-25T00:04:51.938 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T00:04:51.939 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T00:04:51.939 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T00:04:51.939 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T00:04:52.542 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-25T00:04:52.583 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-25T00:04:52.623 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-25T00:04:52.656 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T00:04:52.683 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T00:04:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-25T00:04:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T00:04:52.934 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-25T00:04:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:04:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:04:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:04:54.037 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:04:54.037 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-25T00:04:54.037 INFO:teuthology.orchestra.run.smithi170.stdout:Upgraded: 2024-04-25T00:04:54.037 INFO:teuthology.orchestra.run.smithi170.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.037 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.037 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T00:04:54.037 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T00:04:54.037 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-04-25T00:04:54.037 INFO:teuthology.orchestra.run.smithi170.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T00:04:54.037 INFO:teuthology.orchestra.run.smithi170.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T00:04:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T00:04:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T00:04:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:04:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:04:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:04:54.038 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:04:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:04:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:04:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:04:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:04:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:04:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.039 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:04:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T00:04:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T00:04:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T00:04:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T00:04:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T00:04:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T00:04:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T00:04:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T00:04:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T00:04:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T00:04:54.040 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T00:04:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T00:04:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T00:04:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T00:04:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T00:04:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T00:04:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.041 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T00:04:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T00:04:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T00:04:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T00:04:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T00:04:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T00:04:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T00:04:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T00:04:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T00:04:54.042 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T00:04:54.043 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T00:04:54.043 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T00:04:54.043 INFO:teuthology.orchestra.run.smithi170.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T00:04:54.043 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T00:04:54.043 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T00:04:54.043 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T00:04:54.043 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T00:04:54.043 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T00:04:54.043 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T00:04:54.043 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T00:04:54.043 INFO:teuthology.orchestra.run.smithi170.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T00:04:54.044 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T00:04:54.044 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T00:04:54.044 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T00:04:54.044 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.044 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.044 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.044 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T00:04:54.044 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T00:04:54.044 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T00:04:54.044 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T00:04:54.044 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T00:04:54.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T00:04:54.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T00:04:54.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T00:04:54.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T00:04:54.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T00:04:54.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T00:04:54.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T00:04:54.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T00:04:54.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T00:04:54.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T00:04:54.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T00:04:54.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T00:04:54.045 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T00:04:54.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T00:04:54.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T00:04:54.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T00:04:54.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T00:04:54.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T00:04:54.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T00:04:54.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T00:04:54.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T00:04:54.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T00:04:54.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T00:04:54.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T00:04:54.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T00:04:54.046 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T00:04:54.047 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T00:04:54.047 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T00:04:54.047 INFO:teuthology.orchestra.run.smithi170.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T00:04:54.047 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T00:04:54.047 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T00:04:54.047 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T00:04:54.047 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T00:04:54.047 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T00:04:54.047 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T00:04:54.047 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.048 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.048 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T00:04:54.048 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T00:04:54.048 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T00:04:54.048 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.048 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T00:04:54.048 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T00:04:54.048 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T00:04:54.048 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T00:04:54.048 INFO:teuthology.orchestra.run.smithi170.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T00:04:54.048 INFO:teuthology.orchestra.run.smithi170.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T00:04:54.048 INFO:teuthology.orchestra.run.smithi170.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T00:04:54.048 INFO:teuthology.orchestra.run.smithi170.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T00:04:54.048 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T00:04:54.049 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T00:04:54.049 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T00:04:54.049 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T00:04:54.049 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T00:04:54.049 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T00:04:54.049 INFO:teuthology.orchestra.run.smithi170.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T00:04:54.049 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.049 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.049 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:04:54.049 INFO:teuthology.orchestra.run.smithi170.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T00:04:54.049 INFO:teuthology.orchestra.run.smithi170.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T00:04:54.049 INFO:teuthology.orchestra.run.smithi170.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T00:04:54.049 INFO:teuthology.orchestra.run.smithi170.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T00:04:54.049 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T00:04:54.050 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T00:04:54.050 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T00:04:54.050 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-04-25T00:04:54.050 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-04-25T00:04:54.428 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-25T00:04:54.445 DEBUG:teuthology.parallel:result is None 2024-04-25T00:04:54.520 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T00:04:54.560 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T00:04:54.641 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T00:04:54.674 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T00:04:54.674 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T00:04:54.740 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-25T00:04:54.758 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T00:04:54.781 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-25T00:04:54.799 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T00:04:54.800 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T00:04:54.822 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-25T00:04:54.855 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T00:04:54.882 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T00:04:54.916 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T00:04:54.981 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:04:55.068 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-25T00:04:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T00:04:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-25T00:04:55.791 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:04:55.830 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:04:55.850 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:04:56.672 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-25T00:04:56.744 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T00:04:56.785 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T00:04:56.868 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T00:04:56.906 INFO:teuthology.orchestra.run.smithi195.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T00:04:56.906 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T00:04:56.990 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T00:04:57.023 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T00:04:57.023 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T00:04:57.094 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T00:04:57.164 INFO:teuthology.orchestra.run.smithi195.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:05:00.785 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:05:00.785 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-25T00:05:00.785 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-25T00:05:00.785 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-25T00:05:00.785 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-25T00:05:00.785 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-25T00:05:00.785 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-25T00:05:00.785 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-25T00:05:00.785 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:05:00.786 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:05:00.786 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:05:00.786 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-25T00:05:00.786 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-25T00:05:00.786 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:05:00.786 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-25T00:05:00.786 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-25T00:05:00.786 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-25T00:05:00.786 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-25T00:05:00.786 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-25T00:05:00.786 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-25T00:05:00.786 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-25T00:05:00.787 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-25T00:05:00.787 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-25T00:05:00.787 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-25T00:05:00.787 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-25T00:05:00.787 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-25T00:05:00.787 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-25T00:05:00.787 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-25T00:05:00.787 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-25T00:05:00.799 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-25T00:05:00.799 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-25T00:05:00.799 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-25T00:05:00.800 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-25T00:05:00.800 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-25T00:05:00.800 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-25T00:05:00.800 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-25T00:05:00.800 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-25T00:05:00.800 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-25T00:05:00.800 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-25T00:05:00.800 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-25T00:05:00.800 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T00:05:00.800 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T00:05:00.800 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T00:05:00.800 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T00:05:00.800 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T00:05:00.801 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T00:05:00.801 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T00:05:00.801 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T00:05:00.801 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T00:05:00.801 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T00:05:00.801 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T00:05:00.801 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T00:05:00.801 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T00:05:00.801 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T00:05:00.801 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T00:05:00.801 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T00:05:00.801 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T00:05:00.801 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T00:05:00.802 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T00:05:00.802 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T00:05:00.802 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T00:05:00.802 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T00:05:00.802 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T00:05:00.802 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T00:05:00.802 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T00:05:00.802 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T00:05:00.802 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T00:05:00.802 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T00:05:00.802 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T00:05:00.802 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T00:05:00.802 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T00:05:00.803 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T00:05:00.803 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T00:05:00.803 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T00:05:00.803 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T00:05:00.803 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T00:05:00.803 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T00:05:00.803 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T00:05:00.803 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T00:05:00.803 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T00:05:00.803 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T00:05:00.803 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T00:05:00.803 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T00:05:00.803 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T00:05:00.803 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T00:05:00.804 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T00:05:00.804 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T00:05:00.804 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T00:05:00.804 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T00:05:00.804 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T00:05:00.804 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T00:05:00.804 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T00:05:00.804 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T00:05:00.804 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T00:05:00.804 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T00:05:00.804 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T00:05:00.804 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T00:05:00.804 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T00:05:00.804 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T00:05:00.805 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T00:05:00.805 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T00:05:00.805 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T00:05:00.805 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T00:05:00.805 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T00:05:00.805 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T00:05:00.805 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T00:05:00.805 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T00:05:00.805 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T00:05:00.805 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T00:05:00.805 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T00:05:00.805 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T00:05:00.805 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T00:05:00.805 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T00:05:00.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T00:05:00.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T00:05:00.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T00:05:00.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T00:05:00.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T00:05:00.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T00:05:00.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T00:05:00.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T00:05:00.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T00:05:00.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T00:05:00.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T00:05:00.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T00:05:00.806 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T00:05:00.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T00:05:00.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T00:05:00.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T00:05:00.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T00:05:00.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T00:05:00.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T00:05:00.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T00:05:00.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T00:05:00.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T00:05:00.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T00:05:00.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T00:05:00.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T00:05:00.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T00:05:00.807 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T00:05:00.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T00:05:00.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T00:05:00.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T00:05:00.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T00:05:00.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T00:05:00.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T00:05:00.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T00:05:00.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T00:05:00.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T00:05:00.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-25T00:05:00.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T00:05:00.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-25T00:05:00.808 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T00:05:00.809 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T00:05:00.809 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T00:05:00.809 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T00:05:02.843 INFO:teuthology.orchestra.run.smithi195.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:05:02.843 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-25T00:05:02.843 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-25T00:05:02.844 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-25T00:05:02.844 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-25T00:05:02.844 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-25T00:05:02.844 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-25T00:05:02.844 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-25T00:05:02.844 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:05:02.844 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:05:02.844 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:05:02.844 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-25T00:05:02.844 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-25T00:05:02.844 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:05:02.844 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-25T00:05:02.844 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-25T00:05:02.844 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-25T00:05:02.845 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-25T00:05:02.845 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-25T00:05:02.845 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-25T00:05:02.845 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-25T00:05:02.845 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-25T00:05:02.845 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-25T00:05:02.845 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-25T00:05:02.845 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-25T00:05:02.845 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-25T00:05:02.845 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-25T00:05:02.845 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-25T00:05:02.858 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-25T00:05:02.858 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-25T00:05:02.858 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-25T00:05:02.858 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-25T00:05:02.858 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-25T00:05:02.858 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-25T00:05:02.858 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-25T00:05:02.859 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-25T00:05:02.859 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-25T00:05:02.859 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-25T00:05:02.859 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-25T00:05:02.859 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-25T00:05:02.859 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T00:05:02.859 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T00:05:02.859 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T00:05:02.859 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T00:05:02.859 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T00:05:02.859 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T00:05:02.859 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T00:05:02.859 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T00:05:02.860 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T00:05:02.860 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T00:05:02.860 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T00:05:02.860 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T00:05:02.860 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T00:05:02.860 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T00:05:02.860 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T00:05:02.860 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T00:05:02.860 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T00:05:02.860 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T00:05:02.860 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T00:05:02.860 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T00:05:02.860 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T00:05:02.861 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T00:05:02.861 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T00:05:02.861 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T00:05:02.861 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T00:05:02.861 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T00:05:02.861 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T00:05:02.861 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T00:05:02.861 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T00:05:02.861 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T00:05:02.861 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T00:05:02.861 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T00:05:02.861 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T00:05:02.861 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T00:05:02.862 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T00:05:02.862 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T00:05:02.862 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T00:05:02.862 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T00:05:02.862 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T00:05:02.862 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T00:05:02.862 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T00:05:02.862 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T00:05:02.862 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T00:05:02.862 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T00:05:02.862 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T00:05:02.862 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T00:05:02.862 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T00:05:02.863 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T00:05:02.863 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T00:05:02.863 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T00:05:02.863 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T00:05:02.863 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T00:05:02.863 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T00:05:02.863 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T00:05:02.863 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T00:05:02.863 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T00:05:02.863 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T00:05:02.863 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T00:05:02.864 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T00:05:02.864 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T00:05:02.864 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T00:05:02.864 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T00:05:02.864 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T00:05:02.864 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T00:05:02.864 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T00:05:02.864 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T00:05:02.864 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T00:05:02.864 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T00:05:02.864 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T00:05:02.864 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T00:05:02.864 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T00:05:02.865 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T00:05:02.865 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T00:05:02.865 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T00:05:02.865 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T00:05:02.865 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T00:05:02.865 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T00:05:02.865 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T00:05:02.865 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T00:05:02.865 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T00:05:02.865 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T00:05:02.865 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T00:05:02.865 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T00:05:02.865 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T00:05:02.866 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T00:05:02.866 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T00:05:02.866 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T00:05:02.866 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T00:05:02.866 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T00:05:02.866 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T00:05:02.866 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T00:05:02.866 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T00:05:02.866 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T00:05:02.866 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T00:05:02.866 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T00:05:02.866 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T00:05:02.867 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T00:05:02.867 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T00:05:02.867 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T00:05:02.867 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T00:05:02.867 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T00:05:02.867 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T00:05:02.867 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T00:05:02.867 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T00:05:02.867 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T00:05:02.867 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T00:05:02.867 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T00:05:02.867 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T00:05:02.867 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T00:05:02.867 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-25T00:05:02.868 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T00:05:02.868 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-25T00:05:02.868 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T00:05:02.868 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T00:05:02.868 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T00:05:02.868 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T00:05:02.877 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:05:02.877 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:05:02.877 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2024-04-25T00:05:02.877 INFO:teuthology.orchestra.run.smithi107.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.877 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.877 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T00:05:02.877 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T00:05:02.878 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-04-25T00:05:02.878 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T00:05:02.878 INFO:teuthology.orchestra.run.smithi107.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T00:05:02.878 INFO:teuthology.orchestra.run.smithi107.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T00:05:02.878 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T00:05:02.878 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.878 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.878 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.878 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.878 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:02.878 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.878 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.878 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.879 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:02.879 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:02.879 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:02.879 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:02.879 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:02.879 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.879 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.879 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:02.879 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.879 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.879 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.879 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:02.879 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:02.879 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.880 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.880 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:02.880 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T00:05:02.880 INFO:teuthology.orchestra.run.smithi107.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T00:05:02.880 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T00:05:02.880 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T00:05:02.880 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T00:05:02.880 INFO:teuthology.orchestra.run.smithi107.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T00:05:02.880 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T00:05:02.880 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T00:05:02.880 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T00:05:02.880 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T00:05:02.880 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T00:05:02.880 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.881 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.881 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.881 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T00:05:02.881 INFO:teuthology.orchestra.run.smithi107.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T00:05:02.881 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T00:05:02.881 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T00:05:02.881 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T00:05:02.881 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.881 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.881 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T00:05:02.881 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.881 INFO:teuthology.orchestra.run.smithi107.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T00:05:02.881 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T00:05:02.881 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T00:05:02.882 INFO:teuthology.orchestra.run.smithi107.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T00:05:02.882 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T00:05:02.882 INFO:teuthology.orchestra.run.smithi107.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T00:05:02.882 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T00:05:02.882 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T00:05:02.882 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T00:05:02.882 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T00:05:02.882 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T00:05:02.882 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T00:05:02.882 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T00:05:02.882 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T00:05:02.882 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T00:05:02.882 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T00:05:02.882 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T00:05:02.883 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T00:05:02.883 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T00:05:02.883 INFO:teuthology.orchestra.run.smithi107.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T00:05:02.883 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T00:05:02.883 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T00:05:02.883 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T00:05:02.883 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.883 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.883 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.883 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T00:05:02.883 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T00:05:02.883 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T00:05:02.883 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T00:05:02.883 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T00:05:02.884 INFO:teuthology.orchestra.run.smithi107.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T00:05:02.884 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T00:05:02.884 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T00:05:02.884 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T00:05:02.884 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T00:05:02.884 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T00:05:02.884 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T00:05:02.884 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T00:05:02.884 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T00:05:02.884 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T00:05:02.884 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T00:05:02.884 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T00:05:02.884 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T00:05:02.884 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T00:05:02.885 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T00:05:02.885 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T00:05:02.885 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T00:05:02.885 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T00:05:02.885 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T00:05:02.885 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T00:05:02.885 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T00:05:02.885 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T00:05:02.885 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T00:05:02.885 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T00:05:02.885 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T00:05:02.885 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T00:05:02.885 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T00:05:02.885 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T00:05:02.886 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T00:05:02.886 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T00:05:02.886 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T00:05:02.886 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T00:05:02.886 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T00:05:02.886 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T00:05:02.886 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T00:05:02.886 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.886 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.886 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T00:05:02.886 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T00:05:02.886 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T00:05:02.886 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.886 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T00:05:02.887 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T00:05:02.887 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T00:05:02.887 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T00:05:02.887 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T00:05:02.887 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T00:05:02.887 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T00:05:02.887 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T00:05:02.887 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T00:05:02.887 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T00:05:02.887 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T00:05:02.887 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T00:05:02.887 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T00:05:02.887 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T00:05:02.887 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T00:05:02.888 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.888 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.888 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:02.888 INFO:teuthology.orchestra.run.smithi107.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T00:05:02.888 INFO:teuthology.orchestra.run.smithi107.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T00:05:02.888 INFO:teuthology.orchestra.run.smithi107.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T00:05:02.888 INFO:teuthology.orchestra.run.smithi107.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T00:05:02.888 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T00:05:02.888 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T00:05:02.889 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T00:05:02.889 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:05:02.889 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-25T00:05:03.275 DEBUG:teuthology.parallel:result is None 2024-04-25T00:05:04.974 INFO:teuthology.orchestra.run.smithi195.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:05:04.975 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-04-25T00:05:04.975 INFO:teuthology.orchestra.run.smithi195.stdout:Upgraded: 2024-04-25T00:05:04.975 INFO:teuthology.orchestra.run.smithi195.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.975 INFO:teuthology.orchestra.run.smithi195.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.975 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T00:05:04.975 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T00:05:04.975 INFO:teuthology.orchestra.run.smithi195.stdout:Installed: 2024-04-25T00:05:04.975 INFO:teuthology.orchestra.run.smithi195.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T00:05:04.975 INFO:teuthology.orchestra.run.smithi195.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T00:05:04.975 INFO:teuthology.orchestra.run.smithi195.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T00:05:04.975 INFO:teuthology.orchestra.run.smithi195.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T00:05:04.975 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.975 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.976 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.976 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.976 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:04.976 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.976 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.976 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.976 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:04.976 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:04.976 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:04.976 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:04.976 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:04.976 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.976 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.977 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:04.977 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.977 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.977 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.977 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:04.977 INFO:teuthology.orchestra.run.smithi195.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:04.977 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.977 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.977 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:04.977 INFO:teuthology.orchestra.run.smithi195.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T00:05:04.977 INFO:teuthology.orchestra.run.smithi195.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T00:05:04.977 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T00:05:04.977 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T00:05:04.978 INFO:teuthology.orchestra.run.smithi195.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T00:05:04.978 INFO:teuthology.orchestra.run.smithi195.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T00:05:04.978 INFO:teuthology.orchestra.run.smithi195.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T00:05:04.978 INFO:teuthology.orchestra.run.smithi195.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T00:05:04.978 INFO:teuthology.orchestra.run.smithi195.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T00:05:04.978 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T00:05:04.978 INFO:teuthology.orchestra.run.smithi195.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T00:05:04.978 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.978 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.978 INFO:teuthology.orchestra.run.smithi195.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.978 INFO:teuthology.orchestra.run.smithi195.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T00:05:04.978 INFO:teuthology.orchestra.run.smithi195.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T00:05:04.978 INFO:teuthology.orchestra.run.smithi195.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T00:05:04.979 INFO:teuthology.orchestra.run.smithi195.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T00:05:04.979 INFO:teuthology.orchestra.run.smithi195.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T00:05:04.979 INFO:teuthology.orchestra.run.smithi195.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.979 INFO:teuthology.orchestra.run.smithi195.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.979 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T00:05:04.979 INFO:teuthology.orchestra.run.smithi195.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.979 INFO:teuthology.orchestra.run.smithi195.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T00:05:04.979 INFO:teuthology.orchestra.run.smithi195.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T00:05:04.979 INFO:teuthology.orchestra.run.smithi195.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T00:05:04.979 INFO:teuthology.orchestra.run.smithi195.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T00:05:04.979 INFO:teuthology.orchestra.run.smithi195.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T00:05:04.979 INFO:teuthology.orchestra.run.smithi195.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T00:05:04.979 INFO:teuthology.orchestra.run.smithi195.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T00:05:04.979 INFO:teuthology.orchestra.run.smithi195.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T00:05:04.980 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T00:05:04.980 INFO:teuthology.orchestra.run.smithi195.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T00:05:04.980 INFO:teuthology.orchestra.run.smithi195.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T00:05:04.980 INFO:teuthology.orchestra.run.smithi195.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T00:05:04.980 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T00:05:04.980 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T00:05:04.980 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T00:05:04.980 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T00:05:04.980 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T00:05:04.980 INFO:teuthology.orchestra.run.smithi195.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T00:05:04.980 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T00:05:04.980 INFO:teuthology.orchestra.run.smithi195.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T00:05:04.980 INFO:teuthology.orchestra.run.smithi195.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T00:05:04.980 INFO:teuthology.orchestra.run.smithi195.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T00:05:04.981 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T00:05:04.981 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.981 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.981 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.981 INFO:teuthology.orchestra.run.smithi195.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T00:05:04.981 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T00:05:04.981 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T00:05:04.981 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T00:05:04.981 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T00:05:04.981 INFO:teuthology.orchestra.run.smithi195.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T00:05:04.981 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T00:05:04.981 INFO:teuthology.orchestra.run.smithi195.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T00:05:04.981 INFO:teuthology.orchestra.run.smithi195.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T00:05:04.981 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T00:05:04.982 INFO:teuthology.orchestra.run.smithi195.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T00:05:04.982 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T00:05:04.982 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T00:05:04.982 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T00:05:04.982 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T00:05:04.982 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T00:05:04.982 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T00:05:04.982 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T00:05:04.982 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T00:05:04.982 INFO:teuthology.orchestra.run.smithi195.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T00:05:04.982 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T00:05:04.982 INFO:teuthology.orchestra.run.smithi195.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T00:05:04.982 INFO:teuthology.orchestra.run.smithi195.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T00:05:04.982 INFO:teuthology.orchestra.run.smithi195.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T00:05:04.983 INFO:teuthology.orchestra.run.smithi195.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T00:05:04.983 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T00:05:04.983 INFO:teuthology.orchestra.run.smithi195.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T00:05:04.983 INFO:teuthology.orchestra.run.smithi195.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T00:05:04.983 INFO:teuthology.orchestra.run.smithi195.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T00:05:04.983 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T00:05:04.983 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T00:05:04.983 INFO:teuthology.orchestra.run.smithi195.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T00:05:04.983 INFO:teuthology.orchestra.run.smithi195.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T00:05:04.983 INFO:teuthology.orchestra.run.smithi195.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T00:05:04.983 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T00:05:04.983 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T00:05:04.983 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T00:05:04.983 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T00:05:04.984 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T00:05:04.984 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T00:05:04.984 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.984 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.984 INFO:teuthology.orchestra.run.smithi195.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T00:05:04.984 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T00:05:04.984 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T00:05:04.984 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.984 INFO:teuthology.orchestra.run.smithi195.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T00:05:04.984 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T00:05:04.984 INFO:teuthology.orchestra.run.smithi195.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T00:05:04.984 INFO:teuthology.orchestra.run.smithi195.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T00:05:04.984 INFO:teuthology.orchestra.run.smithi195.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T00:05:04.984 INFO:teuthology.orchestra.run.smithi195.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T00:05:04.985 INFO:teuthology.orchestra.run.smithi195.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T00:05:04.985 INFO:teuthology.orchestra.run.smithi195.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T00:05:04.985 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T00:05:04.985 INFO:teuthology.orchestra.run.smithi195.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T00:05:04.985 INFO:teuthology.orchestra.run.smithi195.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T00:05:04.985 INFO:teuthology.orchestra.run.smithi195.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T00:05:04.985 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T00:05:04.985 INFO:teuthology.orchestra.run.smithi195.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T00:05:04.985 INFO:teuthology.orchestra.run.smithi195.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T00:05:04.985 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.985 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.985 INFO:teuthology.orchestra.run.smithi195.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:04.985 INFO:teuthology.orchestra.run.smithi195.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T00:05:04.986 INFO:teuthology.orchestra.run.smithi195.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T00:05:04.986 INFO:teuthology.orchestra.run.smithi195.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T00:05:04.986 INFO:teuthology.orchestra.run.smithi195.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T00:05:04.986 INFO:teuthology.orchestra.run.smithi195.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T00:05:04.986 INFO:teuthology.orchestra.run.smithi195.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T00:05:04.986 INFO:teuthology.orchestra.run.smithi195.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T00:05:04.987 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-04-25T00:05:04.987 INFO:teuthology.orchestra.run.smithi195.stdout:Complete! 2024-04-25T00:05:05.347 DEBUG:teuthology.parallel:result is None 2024-04-25T00:05:05.348 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:05:05.576 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T00:05:05.599 INFO:teuthology.orchestra.run.smithi107.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-25T00:05:05.600 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-25T00:05:05.600 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-25T00:05:05.604 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:05:05.832 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T00:05:05.858 INFO:teuthology.orchestra.run.smithi170.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-25T00:05:05.858 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-25T00:05:05.858 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-25T00:05:05.860 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:05:06.088 DEBUG:teuthology.orchestra.run.smithi195:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T00:05:06.110 INFO:teuthology.orchestra.run.smithi195.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-25T00:05:06.110 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-25T00:05:06.110 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-25T00:05:06.112 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T00:05:06.112 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T00:05:06.112 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T00:05:06.140 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-04-25T00:05:06.140 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T00:05:06.171 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-04-25T00:05:06.171 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T00:05:06.197 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T00:05:06.197 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T00:05:06.197 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T00:05:06.223 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T00:05:06.287 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-04-25T00:05:06.287 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T00:05:06.315 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T00:05:06.381 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-04-25T00:05:06.382 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T00:05:06.406 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T00:05:06.471 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T00:05:06.471 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T00:05:06.472 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T00:05:06.496 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T00:05:06.560 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-04-25T00:05:06.560 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T00:05:06.588 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T00:05:06.654 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-04-25T00:05:06.654 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T00:05:06.678 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T00:05:06.742 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T00:05:06.742 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T00:05:06.742 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T00:05:06.766 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T00:05:06.830 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-04-25T00:05:06.830 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T00:05:06.857 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T00:05:06.923 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-04-25T00:05:06.923 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T00:05:06.946 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T00:05:07.010 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T00:05:07.143 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'subvols': {'create': 2}} 2024-04-25T00:05:07.143 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:05:07.143 INFO:tasks.cephadm:Cluster fsid is 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:05:07.144 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T00:05:07.144 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.107', 'mon.b': '172.21.15.170', 'mon.c': '172.21.15.195'} 2024-04-25T00:05:07.144 INFO:tasks.cephadm:First mon is mon.a on smithi107 2024-04-25T00:05:07.144 INFO:tasks.cephadm:First mgr is x 2024-04-25T00:05:07.144 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T00:05:07.144 DEBUG:teuthology.orchestra.run.smithi107:> sudo hostname $(hostname -s) 2024-04-25T00:05:07.169 DEBUG:teuthology.orchestra.run.smithi170:> sudo hostname $(hostname -s) 2024-04-25T00:05:07.196 DEBUG:teuthology.orchestra.run.smithi195:> sudo hostname $(hostname -s) 2024-04-25T00:05:07.220 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T00:05:07.220 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:05:07.364 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54987/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2445-gd2c45d97', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2445.gd2c45d97'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 22:37:46.422067', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54981/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2445-gd2c45d97', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2445.gd2c45d97'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 17:00:00.872117', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-25T00:05:07.594 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 d2c45d977142e024a59b68a8f1dce8f0b9840551 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:05:07.596 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm 2024-04-25T00:05:07.597 DEBUG:teuthology.orchestra.run.smithi107:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T00:05:08.454 INFO:teuthology.orchestra.run.smithi107.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 25 00:05 /home/ubuntu/cephtest/cephadm 2024-04-25T00:05:08.454 DEBUG:teuthology.orchestra.run.smithi170:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T00:05:09.413 INFO:teuthology.orchestra.run.smithi170.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 25 00:05 /home/ubuntu/cephtest/cephadm 2024-04-25T00:05:09.413 DEBUG:teuthology.orchestra.run.smithi195:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T00:05:10.249 INFO:teuthology.orchestra.run.smithi195.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 25 00:05 /home/ubuntu/cephtest/cephadm 2024-04-25T00:05:10.249 DEBUG:teuthology.orchestra.run.smithi107:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T00:05:10.268 DEBUG:teuthology.orchestra.run.smithi170:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T00:05:10.287 DEBUG:teuthology.orchestra.run.smithi195:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T00:05:10.328 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 on all hosts... 2024-04-25T00:05:10.328 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-25T00:05:10.331 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-25T00:05:10.333 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-25T00:05:11.925 INFO:teuthology.orchestra.run.smithi107.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T00:05:12.004 INFO:teuthology.orchestra.run.smithi170.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T00:05:12.061 INFO:teuthology.orchestra.run.smithi195.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T00:05:39.671 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-04-25T00:05:39.671 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:05:39.671 INFO:teuthology.orchestra.run.smithi170.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-25T00:05:39.671 INFO:teuthology.orchestra.run.smithi170.stdout: "repo_digests": [ 2024-04-25T00:05:39.672 INFO:teuthology.orchestra.run.smithi170.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-25T00:05:39.672 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-04-25T00:05:39.672 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-04-25T00:05:45.884 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:05:45.884 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:05:45.884 INFO:teuthology.orchestra.run.smithi107.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-25T00:05:45.885 INFO:teuthology.orchestra.run.smithi107.stdout: "repo_digests": [ 2024-04-25T00:05:45.885 INFO:teuthology.orchestra.run.smithi107.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-25T00:05:45.885 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-04-25T00:05:45.885 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:05:45.999 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-04-25T00:05:45.999 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:05:45.999 INFO:teuthology.orchestra.run.smithi195.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-25T00:05:45.999 INFO:teuthology.orchestra.run.smithi195.stdout: "repo_digests": [ 2024-04-25T00:05:45.999 INFO:teuthology.orchestra.run.smithi195.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-25T00:05:45.999 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-04-25T00:05:45.999 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-04-25T00:05:46.025 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph 2024-04-25T00:05:46.051 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /etc/ceph 2024-04-25T00:05:46.083 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph 2024-04-25T00:05:46.111 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /etc/ceph 2024-04-25T00:05:46.134 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 777 /etc/ceph 2024-04-25T00:05:46.161 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 777 /etc/ceph 2024-04-25T00:05:46.186 INFO:tasks.cephadm:Writing seed config... 2024-04-25T00:05:46.187 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T00:05:46.187 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T00:05:46.187 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T00:05:46.187 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T00:05:46.187 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T00:05:46.187 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T00:05:46.187 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T00:05:46.187 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T00:05:46.188 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T00:05:46.188 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T00:05:46.188 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T00:05:46.188 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T00:05:46.188 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T00:05:46.188 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T00:05:46.188 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T00:05:46.188 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T00:05:46.188 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T00:05:46.188 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-25T00:05:46.188 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T00:05:46.188 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T00:05:46.188 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T00:05:46.188 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T00:05:46.188 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-25T00:05:46.189 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T00:05:46.189 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T00:05:46.189 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T00:05:46.189 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T00:05:46.189 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T00:05:46.189 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T00:05:46.189 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T00:05:46.189 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T00:05:46.189 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T00:05:46.189 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T00:05:46.190 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-25T00:05:46.190 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-25T00:05:46.190 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T00:05:46.190 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T00:05:46.190 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-25T00:05:46.190 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T00:05:46.190 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T00:05:46.190 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T00:05:46.190 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T00:05:46.190 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T00:05:46.190 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T00:05:46.190 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T00:05:46.190 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-04-25T00:05:46.191 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T00:05:46.191 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T00:05:46.206 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 79602f58-0297-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T00:05:46.207 DEBUG:teuthology.orchestra.run.smithi107:mon.a> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T00:05:46.250 DEBUG:teuthology.orchestra.run.smithi107:mgr.x> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T00:05:46.294 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T00:05:46.294 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 -v bootstrap --fsid 79602f58-0297-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.107 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:05:46.482 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-25T00:05:46.482 INFO:teuthology.orchestra.run.smithi107.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551', '-v', 'bootstrap', '--fsid', '79602f58-0297-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.107', '--skip-admin-label'] 2024-04-25T00:05:46.482 INFO:teuthology.orchestra.run.smithi107.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T00:05:46.482 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying podman|docker is present... 2024-04-25T00:05:46.508 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T00:05:46.508 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying lvm2 is present... 2024-04-25T00:05:46.508 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying time synchronization is in place... 2024-04-25T00:05:46.515 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T00:05:46.515 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T00:05:46.521 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T00:05:46.521 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-04-25T00:05:46.527 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout enabled 2024-04-25T00:05:46.533 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout active 2024-04-25T00:05:46.533 INFO:teuthology.orchestra.run.smithi107.stdout:Unit chronyd.service is enabled and running 2024-04-25T00:05:46.533 INFO:teuthology.orchestra.run.smithi107.stdout:Repeating the final host check... 2024-04-25T00:05:46.557 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T00:05:46.557 INFO:teuthology.orchestra.run.smithi107.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T00:05:46.557 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl is present 2024-04-25T00:05:46.557 INFO:teuthology.orchestra.run.smithi107.stdout:lvcreate is present 2024-04-25T00:05:46.563 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T00:05:46.563 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T00:05:46.569 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T00:05:46.569 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-04-25T00:05:46.575 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout enabled 2024-04-25T00:05:46.581 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout active 2024-04-25T00:05:46.581 INFO:teuthology.orchestra.run.smithi107.stdout:Unit chronyd.service is enabled and running 2024-04-25T00:05:46.581 INFO:teuthology.orchestra.run.smithi107.stdout:Host looks OK 2024-04-25T00:05:46.581 INFO:teuthology.orchestra.run.smithi107.stdout:Cluster fsid: 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:05:46.581 INFO:teuthology.orchestra.run.smithi107.stdout:Acquiring lock 140391079891872 on /run/cephadm/79602f58-0297-11ef-bc93-c7b262605968.lock 2024-04-25T00:05:46.581 INFO:teuthology.orchestra.run.smithi107.stdout:Lock 140391079891872 acquired on /run/cephadm/79602f58-0297-11ef-bc93-c7b262605968.lock 2024-04-25T00:05:46.582 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying IP 172.21.15.107 port 3300 ... 2024-04-25T00:05:46.582 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying IP 172.21.15.107 port 6789 ... 2024-04-25T00:05:46.582 INFO:teuthology.orchestra.run.smithi107.stdout:Base mon IP(s) is [172.21.15.107:3300, 172.21.15.107:6789], mon addrv is [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2024-04-25T00:05:46.585 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.107 metric 100 2024-04-25T00:05:46.585 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.107 metric 100 2024-04-25T00:05:46.588 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T00:05:46.588 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-25T00:05:46.588 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-25T00:05:46.590 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T00:05:46.590 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T00:05:46.591 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T00:05:46.591 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-25T00:05:46.591 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout inet6 fe80::d21f:963:7e10:29d9/64 scope link noprefixroute 2024-04-25T00:05:46.591 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T00:05:46.591 INFO:teuthology.orchestra.run.smithi107.stdout:Mon IP `172.21.15.107` is in CIDR network `172.21.0.0/20` 2024-04-25T00:05:46.591 INFO:teuthology.orchestra.run.smithi107.stdout:Mon IP `172.21.15.107` is in CIDR network `172.21.0.0/20` 2024-04-25T00:05:46.592 INFO:teuthology.orchestra.run.smithi107.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T00:05:46.592 INFO:teuthology.orchestra.run.smithi107.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T00:05:46.592 INFO:teuthology.orchestra.run.smithi107.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T00:05:47.054 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e 2024-04-25T00:05:47.054 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T00:05:47.054 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T00:05:47.054 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying blob sha256:58a99251f41e8a467bafa0ec849428709568906ca732f10972304f37056be6c5 2024-04-25T00:05:47.054 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-25T00:05:47.054 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying config sha256:f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e 2024-04-25T00:05:47.054 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T00:05:47.967 INFO:teuthology.orchestra.run.smithi107.stdout:ceph: stdout ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev) 2024-04-25T00:05:47.967 INFO:teuthology.orchestra.run.smithi107.stdout:Ceph version: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev) 2024-04-25T00:05:47.967 INFO:teuthology.orchestra.run.smithi107.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T00:05:48.948 INFO:teuthology.orchestra.run.smithi107.stdout:stat: stdout 167 167 2024-04-25T00:05:48.948 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial keys... 2024-04-25T00:05:49.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQBdnilmrDJOEBAAWmMKYODpm3tw16XONuz6BA== 2024-04-25T00:05:50.918 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQBenilmhnS2ExAAJXmKGnQJ7a7kLScQakTY0w== 2024-04-25T00:05:51.822 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQBfnilmUoAiDxAAPdUhqkeOFnDooE9f0mg+sw== 2024-04-25T00:05:51.823 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial monmap... 2024-04-25T00:05:52.756 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T00:05:52.756 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T00:05:52.756 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:05:52.756 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T00:05:52.756 INFO:teuthology.orchestra.run.smithi107.stdout:monmaptool for a [v2:172.21.15.107:3300,v1:172.21.15.107:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T00:05:52.756 INFO:teuthology.orchestra.run.smithi107.stdout:setting min_mon_release = quincy 2024-04-25T00:05:52.756 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: set fsid to 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:05:52.756 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T00:05:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:05:52.757 INFO:teuthology.orchestra.run.smithi107.stdout:Creating mon... 2024-04-25T00:05:53.662 INFO:teuthology.orchestra.run.smithi107.stdout:create mon.a on 2024-04-25T00:05:53.878 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T00:05:54.015 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T00:05:54.204 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-79602f58-0297-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-79602f58-0297-11ef-bc93-c7b262605968.target. 2024-04-25T00:05:54.204 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-79602f58-0297-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-79602f58-0297-11ef-bc93-c7b262605968.target. 2024-04-25T00:05:54.438 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-79602f58-0297-11ef-bc93-c7b262605968@mon.a 2024-04-25T00:05:54.438 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to reset failed state of unit ceph-79602f58-0297-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-79602f58-0297-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T00:05:54.599 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-79602f58-0297-11ef-bc93-c7b262605968.target.wants/ceph-79602f58-0297-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-79602f58-0297-11ef-bc93-c7b262605968@.service. 2024-04-25T00:05:55.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:05:54 smithi107 podman[28750]: 2024-04-25 00:05:54.72962049 +0000 UTC m=+0.017760669 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:05:56.375 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:05:56 smithi107 podman[28750]: 2024-04-25 00:05:56.000065343 +0000 UTC m=+1.288205522 container init 0eec6764212d496fbd8eba7f7c557b0721d98d13317a55d0431b89f75dc0cedc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T00:05:56.375 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:05:56 smithi107 podman[28750]: 2024-04-25 00:05:56.003595936 +0000 UTC m=+1.291736103 container start 0eec6764212d496fbd8eba7f7c557b0721d98d13317a55d0431b89f75dc0cedc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T00:05:56.375 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:05:56 smithi107 ceph-mon[28779]: mkfs 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:05:56.375 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:05:56 smithi107 ceph-mon[28779]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T00:05:56.926 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T00:05:56.926 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-04-25T00:05:56.935 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T00:05:56.935 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-04-25T00:05:56.936 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-04-25T00:05:56.936 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T00:05:56.936 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mon to start... 2024-04-25T00:05:56.936 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mon... 2024-04-25T00:05:57.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:05:56 smithi107 bash[28750]: 0eec6764212d496fbd8eba7f7c557b0721d98d13317a55d0431b89f75dc0cedc 2024-04-25T00:05:57.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:05:56 smithi107 systemd[1]: Started Ceph mon.a for 79602f58-0297-11ef-bc93-c7b262605968. 2024-04-25T00:05:57.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:05:57 smithi107 ceph-mon[28779]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T00:05:57.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:05:57 smithi107 ceph-mon[28779]: monmap epoch 1 2024-04-25T00:05:57.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:05:57 smithi107 ceph-mon[28779]: fsid 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:05:57.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:05:57 smithi107 ceph-mon[28779]: last_changed 2024-04-25T00:05:52.163469+0000 2024-04-25T00:05:57.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:05:57 smithi107 ceph-mon[28779]: created 2024-04-25T00:05:52.163469+0000 2024-04-25T00:05:57.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:05:57 smithi107 ceph-mon[28779]: min_mon_release 19 (squid) 2024-04-25T00:05:57.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:05:57 smithi107 ceph-mon[28779]: election_strategy: 1 2024-04-25T00:05:57.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:05:57 smithi107 ceph-mon[28779]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-04-25T00:05:57.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:05:57 smithi107 ceph-mon[28779]: fsmap 2024-04-25T00:05:57.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:05:57 smithi107 ceph-mon[28779]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T00:05:57.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:05:57 smithi107 ceph-mon[28779]: mgrmap e1: no daemons active 2024-04-25T00:06:00.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:00 smithi107 ceph-mon[28779]: from='client.? 172.21.15.107:0/1298336655' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T00:06:06.671 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T00:06:06.671 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout id: 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:06:06.671 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T00:06:06.671 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-25T00:06:06.671 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout services: 2024-04-25T00:06:06.671 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 4s) 2024-04-25T00:06:06.671 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T00:06:06.671 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T00:06:06.671 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-25T00:06:06.671 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout data: 2024-04-25T00:06:06.672 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T00:06:06.672 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T00:06:06.672 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T00:06:06.672 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T00:06:06.672 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-25T00:06:06.672 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.125+0000 7f244b0e6700 1 Processor -- start 2024-04-25T00:06:06.672 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.125+0000 7f244b0e6700 1 -- start start 2024-04-25T00:06:06.672 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.125+0000 7f244b0e6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2444105f30 0x7f2444106350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:06.672 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.125+0000 7f244b0e6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2444106920 con 0x7f2444105f30 2024-04-25T00:06:06.672 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.126+0000 7f2448e82700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2444105f30 0x7f2444106350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:06.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.126+0000 7f2448e82700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2444105f30 0x7f2444106350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35658/0 (socket says 172.21.15.107:35658) 2024-04-25T00:06:06.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.126+0000 7f2448e82700 1 -- 172.21.15.107:0/2683978342 learned_addr learned my addr 172.21.15.107:0/2683978342 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:06.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.126+0000 7f2448e82700 1 -- 172.21.15.107:0/2683978342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2444108ec0 con 0x7f2444105f30 2024-04-25T00:06:06.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.126+0000 7f2448e82700 1 --2- 172.21.15.107:0/2683978342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2444105f30 0x7f2444106350 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f24380098a0 tx=0x7f2438009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716627108d8678d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:06.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.126+0000 7f24437fe700 1 -- 172.21.15.107:0/2683978342 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2438004160 con 0x7f2444105f30 2024-04-25T00:06:06.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.126+0000 7f24437fe700 1 -- 172.21.15.107:0/2683978342 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2438004300 con 0x7f2444105f30 2024-04-25T00:06:06.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.126+0000 7f24437fe700 1 -- 172.21.15.107:0/2683978342 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24380036a0 con 0x7f2444105f30 2024-04-25T00:06:06.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.127+0000 7f244b0e6700 1 -- 172.21.15.107:0/2683978342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2444105f30 msgr2=0x7f2444106350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:06.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.127+0000 7f244b0e6700 1 --2- 172.21.15.107:0/2683978342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2444105f30 0x7f2444106350 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f24380098a0 tx=0x7f2438009bb0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.127+0000 7f244b0e6700 1 -- 172.21.15.107:0/2683978342 shutdown_connections 2024-04-25T00:06:06.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.127+0000 7f244b0e6700 1 --2- 172.21.15.107:0/2683978342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2444105f30 0x7f2444106350 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.127+0000 7f244b0e6700 1 -- 172.21.15.107:0/2683978342 >> 172.21.15.107:0/2683978342 conn(0x7f24441018a0 msgr2=0x7f2444103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:06.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.127+0000 7f244b0e6700 1 -- 172.21.15.107:0/2683978342 shutdown_connections 2024-04-25T00:06:06.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.127+0000 7f244b0e6700 1 -- 172.21.15.107:0/2683978342 wait complete. 2024-04-25T00:06:06.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.127+0000 7f244b0e6700 1 Processor -- start 2024-04-25T00:06:06.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.127+0000 7f244b0e6700 1 -- start start 2024-04-25T00:06:06.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.128+0000 7f244b0e6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2444105f30 0x7f244419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:06.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.128+0000 7f244b0e6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f24441093f0 con 0x7f2444105f30 2024-04-25T00:06:06.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.128+0000 7f2448e82700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2444105f30 0x7f244419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:06.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.128+0000 7f2448e82700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2444105f30 0x7f244419b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35666/0 (socket says 172.21.15.107:35666) 2024-04-25T00:06:06.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.128+0000 7f2448e82700 1 -- 172.21.15.107:0/1298336655 learned_addr learned my addr 172.21.15.107:0/1298336655 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:06.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.128+0000 7f2448e82700 1 -- 172.21.15.107:0/1298336655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f244419b8f0 con 0x7f2444105f30 2024-04-25T00:06:06.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.128+0000 7f2448e82700 1 --2- 172.21.15.107:0/1298336655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2444105f30 0x7f244419b3b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2444106c10 tx=0x7f2438004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:06.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.128+0000 7f2441ffb700 1 -- 172.21.15.107:0/1298336655 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2438003db0 con 0x7f2444105f30 2024-04-25T00:06:06.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.128+0000 7f2441ffb700 1 -- 172.21.15.107:0/1298336655 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2438011630 con 0x7f2444105f30 2024-04-25T00:06:06.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.128+0000 7f244b0e6700 1 -- 172.21.15.107:0/1298336655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f244419bb80 con 0x7f2444105f30 2024-04-25T00:06:06.675 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.128+0000 7f2441ffb700 1 -- 172.21.15.107:0/1298336655 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2438011930 con 0x7f2444105f30 2024-04-25T00:06:06.675 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.129+0000 7f244b0e6700 1 -- 172.21.15.107:0/1298336655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f244419e870 con 0x7f2444105f30 2024-04-25T00:06:06.675 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.129+0000 7f2441ffb700 1 -- 172.21.15.107:0/1298336655 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2438024480 con 0x7f2444105f30 2024-04-25T00:06:06.675 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.129+0000 7f2441ffb700 1 -- 172.21.15.107:0/1298336655 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f243801a7a0 con 0x7f2444105f30 2024-04-25T00:06:06.675 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.129+0000 7f244b0e6700 1 -- 172.21.15.107:0/1298336655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2444063890 con 0x7f2444105f30 2024-04-25T00:06:06.675 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.132+0000 7f2441ffb700 1 -- 172.21.15.107:0/1298336655 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f243801aa40 con 0x7f2444105f30 2024-04-25T00:06:06.675 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.202+0000 7f244b0e6700 1 -- 172.21.15.107:0/1298336655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f244402dd20 con 0x7f2444105f30 2024-04-25T00:06:06.675 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.202+0000 7f2441ffb700 1 -- 172.21.15.107:0/1298336655 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+310 (secure 0 0 0) 0x7f243801ac20 con 0x7f2444105f30 2024-04-25T00:06:06.675 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.203+0000 7f244b0e6700 1 -- 172.21.15.107:0/1298336655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2444105f30 msgr2=0x7f244419b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:06.675 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.203+0000 7f244b0e6700 1 --2- 172.21.15.107:0/1298336655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2444105f30 0x7f244419b3b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2444106c10 tx=0x7f2438004750 comp rx=0 tx=0).stop 2024-04-25T00:06:06.675 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.203+0000 7f244b0e6700 1 -- 172.21.15.107:0/1298336655 shutdown_connections 2024-04-25T00:06:06.675 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.203+0000 7f244b0e6700 1 --2- 172.21.15.107:0/1298336655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2444105f30 0x7f244419b3b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:06.675 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.203+0000 7f244b0e6700 1 -- 172.21.15.107:0/1298336655 >> 172.21.15.107:0/1298336655 conn(0x7f24441018a0 msgr2=0x7f2444191fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:06.676 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.203+0000 7f244b0e6700 1 -- 172.21.15.107:0/1298336655 shutdown_connections 2024-04-25T00:06:06.676 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:00.203+0000 7f244b0e6700 1 -- 172.21.15.107:0/1298336655 wait complete. 2024-04-25T00:06:06.676 INFO:teuthology.orchestra.run.smithi107.stdout:mon is available 2024-04-25T00:06:06.676 INFO:teuthology.orchestra.run.smithi107.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T00:06:08.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:08 smithi107 ceph-mon[28779]: from='client.? 172.21.15.107:0/1625671996' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T00:06:08.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:08 smithi107 ceph-mon[28779]: from='client.? 172.21.15.107:0/1625671996' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T00:06:08.833 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-25T00:06:08.833 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T00:06:08.833 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout fsid = 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:06:08.833 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2024-04-25T00:06:08.833 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T00:06:08.833 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T00:06:08.833 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T00:06:08.833 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T00:06:08.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-25T00:06:08.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T00:06:08.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T00:06:08.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-25T00:06:08.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T00:06:08.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T00:06:08.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T00:06:08.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T00:06:08.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T00:06:08.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T00:06:08.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.140+0000 7f2ffe433700 1 Processor -- start 2024-04-25T00:06:08.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.140+0000 7f2ffe433700 1 -- start start 2024-04-25T00:06:08.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.140+0000 7f2ffe433700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff8109260 0x7f2ff8109680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:08.834 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.140+0000 7f2ffe433700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff8109c50 con 0x7f2ff8109260 2024-04-25T00:06:08.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.141+0000 7f2ff7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff8109260 0x7f2ff8109680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:08.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.141+0000 7f2ff7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff8109260 0x7f2ff8109680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35676/0 (socket says 172.21.15.107:35676) 2024-04-25T00:06:08.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.141+0000 7f2ff7fff700 1 -- 172.21.15.107:0/3638200733 learned_addr learned my addr 172.21.15.107:0/3638200733 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:08.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.141+0000 7f2ff7fff700 1 -- 172.21.15.107:0/3638200733 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff8109dd0 con 0x7f2ff8109260 2024-04-25T00:06:08.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.141+0000 7f2ff7fff700 1 --2- 172.21.15.107:0/3638200733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff8109260 0x7f2ff8109680 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80098a0 tx=0x7f2fe8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5004d3d59da7de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:08.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.141+0000 7f2ff6ffd700 1 -- 172.21.15.107:0/3638200733 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fe8004160 con 0x7f2ff8109260 2024-04-25T00:06:08.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.141+0000 7f2ff6ffd700 1 -- 172.21.15.107:0/3638200733 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe8004300 con 0x7f2ff8109260 2024-04-25T00:06:08.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.141+0000 7f2ff6ffd700 1 -- 172.21.15.107:0/3638200733 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fe80036a0 con 0x7f2ff8109260 2024-04-25T00:06:08.835 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.142+0000 7f2ffe433700 1 -- 172.21.15.107:0/3638200733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff8109260 msgr2=0x7f2ff8109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:08.836 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.142+0000 7f2ffe433700 1 --2- 172.21.15.107:0/3638200733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff8109260 0x7f2ff8109680 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80098a0 tx=0x7f2fe8009bb0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.836 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.142+0000 7f2ffe433700 1 -- 172.21.15.107:0/3638200733 shutdown_connections 2024-04-25T00:06:08.836 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.142+0000 7f2ffe433700 1 --2- 172.21.15.107:0/3638200733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff8109260 0x7f2ff8109680 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.836 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.142+0000 7f2ffe433700 1 -- 172.21.15.107:0/3638200733 >> 172.21.15.107:0/3638200733 conn(0x7f2ff81018a0 msgr2=0x7f2ff8103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:08.836 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.142+0000 7f2ffe433700 1 -- 172.21.15.107:0/3638200733 shutdown_connections 2024-04-25T00:06:08.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.142+0000 7f2ffe433700 1 -- 172.21.15.107:0/3638200733 wait complete. 2024-04-25T00:06:08.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.142+0000 7f2ffe433700 1 Processor -- start 2024-04-25T00:06:08.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.142+0000 7f2ffe433700 1 -- start start 2024-04-25T00:06:08.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.143+0000 7f2ffe433700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff8109260 0x7f2ff819b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:08.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.143+0000 7f2ffe433700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff8105fe0 con 0x7f2ff8109260 2024-04-25T00:06:08.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.143+0000 7f2ff7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff8109260 0x7f2ff819b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:08.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.143+0000 7f2ff7fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff8109260 0x7f2ff819b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35688/0 (socket says 172.21.15.107:35688) 2024-04-25T00:06:08.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.143+0000 7f2ff7fff700 1 -- 172.21.15.107:0/1625671996 learned_addr learned my addr 172.21.15.107:0/1625671996 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:08.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.143+0000 7f2ff7fff700 1 -- 172.21.15.107:0/1625671996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff819b960 con 0x7f2ff8109260 2024-04-25T00:06:08.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.143+0000 7f2ff7fff700 1 --2- 172.21.15.107:0/1625671996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff8109260 0x7f2ff819b420 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8000c00 tx=0x7f2fe8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:08.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.143+0000 7f2ff57fa700 1 -- 172.21.15.107:0/1625671996 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fe8003db0 con 0x7f2ff8109260 2024-04-25T00:06:08.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.143+0000 7f2ff57fa700 1 -- 172.21.15.107:0/1625671996 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe8011630 con 0x7f2ff8109260 2024-04-25T00:06:08.837 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.143+0000 7f2ffe433700 1 -- 172.21.15.107:0/1625671996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff819bc50 con 0x7f2ff8109260 2024-04-25T00:06:08.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.143+0000 7f2ff57fa700 1 -- 172.21.15.107:0/1625671996 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fe8011950 con 0x7f2ff8109260 2024-04-25T00:06:08.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.143+0000 7f2ffe433700 1 -- 172.21.15.107:0/1625671996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff819e890 con 0x7f2ff8109260 2024-04-25T00:06:08.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.144+0000 7f2ff57fa700 1 -- 172.21.15.107:0/1625671996 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2fe801a460 con 0x7f2ff8109260 2024-04-25T00:06:08.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.144+0000 7f2ff57fa700 1 -- 172.21.15.107:0/1625671996 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2fe801b780 con 0x7f2ff8109260 2024-04-25T00:06:08.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.144+0000 7f2ffe433700 1 -- 172.21.15.107:0/1625671996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ff8063890 con 0x7f2ff8109260 2024-04-25T00:06:08.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.147+0000 7f2ff57fa700 1 -- 172.21.15.107:0/1625671996 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2fe8031020 con 0x7f2ff8109260 2024-04-25T00:06:08.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.213+0000 7f2ffe433700 1 -- 172.21.15.107:0/1625671996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f2ff802dd40 con 0x7f2ff8109260 2024-04-25T00:06:08.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.216+0000 7f2ff57fa700 1 -- 172.21.15.107:0/1625671996 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+431 (secure 0 0 0) 0x7f2fe801b9d0 con 0x7f2ff8109260 2024-04-25T00:06:08.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.217+0000 7f2ff57fa700 1 -- 172.21.15.107:0/1625671996 <== mon.0 v2:172.21.15.107:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2fe801ada0 con 0x7f2ff8109260 2024-04-25T00:06:08.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.218+0000 7f2ffe433700 1 -- 172.21.15.107:0/1625671996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff8109260 msgr2=0x7f2ff819b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:08.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.218+0000 7f2ffe433700 1 --2- 172.21.15.107:0/1625671996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff8109260 0x7f2ff819b420 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8000c00 tx=0x7f2fe8004750 comp rx=0 tx=0).stop 2024-04-25T00:06:08.838 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.218+0000 7f2ffe433700 1 -- 172.21.15.107:0/1625671996 shutdown_connections 2024-04-25T00:06:08.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.218+0000 7f2ffe433700 1 --2- 172.21.15.107:0/1625671996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff8109260 0x7f2ff819b420 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:08.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.218+0000 7f2ffe433700 1 -- 172.21.15.107:0/1625671996 >> 172.21.15.107:0/1625671996 conn(0x7f2ff81018a0 msgr2=0x7f2ff8191fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:08.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.218+0000 7f2ffe433700 1 -- 172.21.15.107:0/1625671996 shutdown_connections 2024-04-25T00:06:08.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:08.218+0000 7f2ffe433700 1 -- 172.21.15.107:0/1625671996 wait complete. 2024-04-25T00:06:08.839 INFO:teuthology.orchestra.run.smithi107.stdout:Generating new minimal ceph.conf... 2024-04-25T00:06:09.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:09 smithi107 ceph-mon[28779]: from='client.? 172.21.15.107:0/2740380446' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:06:09.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.232+0000 7efc390c0700 1 Processor -- start 2024-04-25T00:06:09.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.232+0000 7efc390c0700 1 -- start start 2024-04-25T00:06:09.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.232+0000 7efc390c0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc3407dea0 0x7efc3407c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:09.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.232+0000 7efc390c0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efc3407c8d0 con 0x7efc3407dea0 2024-04-25T00:06:09.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.232+0000 7efc32d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc3407dea0 0x7efc3407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:09.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.232+0000 7efc32d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc3407dea0 0x7efc3407c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35696/0 (socket says 172.21.15.107:35696) 2024-04-25T00:06:09.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.232+0000 7efc32d9d700 1 -- 172.21.15.107:0/3522558491 learned_addr learned my addr 172.21.15.107:0/3522558491 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:09.839 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.232+0000 7efc32d9d700 1 -- 172.21.15.107:0/3522558491 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3407ca50 con 0x7efc3407dea0 2024-04-25T00:06:09.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.232+0000 7efc32d9d700 1 --2- 172.21.15.107:0/3522558491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc3407dea0 0x7efc3407c300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efc240098a0 tx=0x7efc24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427aa0b977b2ace9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:09.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.233+0000 7efc31d9b700 1 -- 172.21.15.107:0/3522558491 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc24004160 con 0x7efc3407dea0 2024-04-25T00:06:09.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.233+0000 7efc31d9b700 1 -- 172.21.15.107:0/3522558491 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efc24004300 con 0x7efc3407dea0 2024-04-25T00:06:09.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.233+0000 7efc31d9b700 1 -- 172.21.15.107:0/3522558491 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc24011b50 con 0x7efc3407dea0 2024-04-25T00:06:09.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.233+0000 7efc390c0700 1 -- 172.21.15.107:0/3522558491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc3407dea0 msgr2=0x7efc3407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:09.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.233+0000 7efc390c0700 1 --2- 172.21.15.107:0/3522558491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc3407dea0 0x7efc3407c300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efc240098a0 tx=0x7efc24009bb0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.233+0000 7efc390c0700 1 -- 172.21.15.107:0/3522558491 shutdown_connections 2024-04-25T00:06:09.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.233+0000 7efc390c0700 1 --2- 172.21.15.107:0/3522558491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc3407dea0 0x7efc3407c300 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.233+0000 7efc390c0700 1 -- 172.21.15.107:0/3522558491 >> 172.21.15.107:0/3522558491 conn(0x7efc341031e0 msgr2=0x7efc34105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:09.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.233+0000 7efc390c0700 1 -- 172.21.15.107:0/3522558491 shutdown_connections 2024-04-25T00:06:09.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.233+0000 7efc390c0700 1 -- 172.21.15.107:0/3522558491 wait complete. 2024-04-25T00:06:09.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.234+0000 7efc390c0700 1 Processor -- start 2024-04-25T00:06:09.840 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.234+0000 7efc390c0700 1 -- start start 2024-04-25T00:06:09.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.234+0000 7efc390c0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc3407dea0 0x7efc3419de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:09.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.234+0000 7efc390c0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efc3410e250 con 0x7efc3407dea0 2024-04-25T00:06:09.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.234+0000 7efc32d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc3407dea0 0x7efc3419de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:09.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.234+0000 7efc32d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc3407dea0 0x7efc3419de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35706/0 (socket says 172.21.15.107:35706) 2024-04-25T00:06:09.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.234+0000 7efc32d9d700 1 -- 172.21.15.107:0/2740380446 learned_addr learned my addr 172.21.15.107:0/2740380446 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:09.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.235+0000 7efc32d9d700 1 -- 172.21.15.107:0/2740380446 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3419e380 con 0x7efc3407dea0 2024-04-25T00:06:09.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.235+0000 7efc32d9d700 1 --2- 172.21.15.107:0/2740380446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc3407dea0 0x7efc3419de40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efc24011850 tx=0x7efc24011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:09.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.235+0000 7efc1bfff700 1 -- 172.21.15.107:0/2740380446 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc24003af0 con 0x7efc3407dea0 2024-04-25T00:06:09.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.235+0000 7efc390c0700 1 -- 172.21.15.107:0/2740380446 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc3419e610 con 0x7efc3407dea0 2024-04-25T00:06:09.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.235+0000 7efc1bfff700 1 -- 172.21.15.107:0/2740380446 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efc24020070 con 0x7efc3407dea0 2024-04-25T00:06:09.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.235+0000 7efc1bfff700 1 -- 172.21.15.107:0/2740380446 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc240195a0 con 0x7efc3407dea0 2024-04-25T00:06:09.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.235+0000 7efc390c0700 1 -- 172.21.15.107:0/2740380446 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc341a1300 con 0x7efc3407dea0 2024-04-25T00:06:09.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.235+0000 7efc1bfff700 1 -- 172.21.15.107:0/2740380446 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efc24019c30 con 0x7efc3407dea0 2024-04-25T00:06:09.841 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.235+0000 7efc1bfff700 1 -- 172.21.15.107:0/2740380446 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efc24011e00 con 0x7efc3407dea0 2024-04-25T00:06:09.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.236+0000 7efc390c0700 1 -- 172.21.15.107:0/2740380446 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc34063890 con 0x7efc3407dea0 2024-04-25T00:06:09.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.238+0000 7efc1bfff700 1 -- 172.21.15.107:0/2740380446 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7efc24003c90 con 0x7efc3407dea0 2024-04-25T00:06:09.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.306+0000 7efc390c0700 1 -- 172.21.15.107:0/2740380446 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7efc341a14d0 con 0x7efc3407dea0 2024-04-25T00:06:09.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.306+0000 7efc1bfff700 1 -- 172.21.15.107:0/2740380446 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+177 (secure 0 0 0) 0x7efc2401a430 con 0x7efc3407dea0 2024-04-25T00:06:09.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.307+0000 7efc390c0700 1 -- 172.21.15.107:0/2740380446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc3407dea0 msgr2=0x7efc3419de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:09.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.307+0000 7efc390c0700 1 --2- 172.21.15.107:0/2740380446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc3407dea0 0x7efc3419de40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efc24011850 tx=0x7efc24011c10 comp rx=0 tx=0).stop 2024-04-25T00:06:09.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.307+0000 7efc390c0700 1 -- 172.21.15.107:0/2740380446 shutdown_connections 2024-04-25T00:06:09.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.307+0000 7efc390c0700 1 --2- 172.21.15.107:0/2740380446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc3407dea0 0x7efc3419de40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:09.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.307+0000 7efc390c0700 1 -- 172.21.15.107:0/2740380446 >> 172.21.15.107:0/2740380446 conn(0x7efc341031e0 msgr2=0x7efc34103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:09.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.307+0000 7efc390c0700 1 -- 172.21.15.107:0/2740380446 shutdown_connections 2024-04-25T00:06:09.842 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:09.308+0000 7efc390c0700 1 -- 172.21.15.107:0/2740380446 wait complete. 2024-04-25T00:06:09.842 INFO:teuthology.orchestra.run.smithi107.stdout:Restarting the monitor... 2024-04-25T00:06:10.102 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:09 smithi107 systemd[1]: Stopping Ceph mon.a for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:06:10.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:10 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-a[28764]: 2024-04-25T00:06:10.101+0000 7fadac435700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:06:10.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:10 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-a[28764]: 2024-04-25T00:06:10.101+0000 7fadac435700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T00:06:10.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:10 smithi107 podman[29026]: 2024-04-25 00:06:10.327113317 +0000 UTC m=+0.327311025 container died 0eec6764212d496fbd8eba7f7c557b0721d98d13317a55d0431b89f75dc0cedc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:06:10.819 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:10 smithi107 podman[29026]: 2024-04-25 00:06:10.593467591 +0000 UTC m=+0.593665299 container cleanup 0eec6764212d496fbd8eba7f7c557b0721d98d13317a55d0431b89f75dc0cedc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-a, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:06:10.819 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:10 smithi107 bash[29026]: ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-a 2024-04-25T00:06:11.186 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:10 smithi107 podman[29040]: 2024-04-25 00:06:10.819240211 +0000 UTC m=+0.487329598 container remove 0eec6764212d496fbd8eba7f7c557b0721d98d13317a55d0431b89f75dc0cedc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:06:11.186 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:10 smithi107 systemd[1]: ceph-79602f58-0297-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T00:06:11.186 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:10 smithi107 systemd[1]: Stopped Ceph mon.a for 79602f58-0297-11ef-bc93-c7b262605968. 2024-04-25T00:06:11.186 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:10 smithi107 systemd[1]: Starting Ceph mon.a for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:06:11.464 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 podman[29135]: 2024-04-25 00:06:11.089898565 +0000 UTC m=+0.017901549 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:06:11.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 podman[29135]: 2024-04-25 00:06:11.195476383 +0000 UTC m=+0.123479360 container create 8662034e7721df30bfaf633987d37d05c01485dfd9cce32ba3dbfc5ca1c54ed2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:06:11.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 podman[29135]: 2024-04-25 00:06:11.389649414 +0000 UTC m=+0.317652389 container init 8662034e7721df30bfaf633987d37d05c01485dfd9cce32ba3dbfc5ca1c54ed2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-04-25T00:06:11.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 podman[29135]: 2024-04-25 00:06:11.392455329 +0000 UTC m=+0.320458307 container start 8662034e7721df30bfaf633987d37d05c01485dfd9cce32ba3dbfc5ca1c54ed2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9) 2024-04-25T00:06:11.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T00:06:11.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-25T00:06:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: pidfile_write: ignore empty --pid-file 2024-04-25T00:06:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: load: jerasure load: lrc 2024-04-25T00:06:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: RocksDB version: 7.9.2 2024-04-25T00:06:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Git sha 0 2024-04-25T00:06:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-25T00:06:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: DB SUMMARY 2024-04-25T00:06:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: DB Session ID: QDVIHU4G94EEV7XP5YVP 2024-04-25T00:06:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: CURRENT file: CURRENT 2024-04-25T00:06:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T00:06:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T00:06:11.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T00:06:11.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105073 ; 2024-04-25T00:06:11.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.error_if_exists: 0 2024-04-25T00:06:11.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.create_if_missing: 0 2024-04-25T00:06:11.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.paranoid_checks: 1 2024-04-25T00:06:11.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T00:06:11.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T00:06:11.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T00:06:11.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.env: 0x555a8cf92c60 2024-04-25T00:06:11.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T00:06:11.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.info_log: 0x555a8dd35aa0 2024-04-25T00:06:11.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T00:06:11.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.statistics: (nil) 2024-04-25T00:06:11.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.use_fsync: 0 2024-04-25T00:06:11.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_log_file_size: 0 2024-04-25T00:06:11.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T00:06:11.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T00:06:11.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T00:06:11.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T00:06:11.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.allow_fallocate: 1 2024-04-25T00:06:11.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T00:06:11.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T00:06:11.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.use_direct_reads: 0 2024-04-25T00:06:11.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T00:06:11.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T00:06:11.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.db_log_dir: 2024-04-25T00:06:11.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.wal_dir: 2024-04-25T00:06:11.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T00:06:11.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T00:06:11.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T00:06:11.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T00:06:11.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T00:06:11.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T00:06:11.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T00:06:11.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T00:06:11.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.write_buffer_manager: 0x555a8dd081e0 2024-04-25T00:06:11.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T00:06:11.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T00:06:11.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T00:06:11.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T00:06:11.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T00:06:11.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T00:06:11.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T00:06:11.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T00:06:11.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.unordered_write: 0 2024-04-25T00:06:11.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T00:06:11.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T00:06:11.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T00:06:11.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T00:06:11.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.row_cache: None 2024-04-25T00:06:11.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.wal_filter: None 2024-04-25T00:06:11.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T00:06:11.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T00:06:11.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.two_write_queues: 0 2024-04-25T00:06:11.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T00:06:11.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.wal_compression: 0 2024-04-25T00:06:11.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.atomic_flush: 0 2024-04-25T00:06:11.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T00:06:11.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T00:06:11.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T00:06:11.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.log_readahead_size: 0 2024-04-25T00:06:11.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T00:06:11.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T00:06:11.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T00:06:11.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T00:06:11.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T00:06:11.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T00:06:11.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T00:06:11.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_background_jobs: 2 2024-04-25T00:06:11.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_background_compactions: -1 2024-04-25T00:06:11.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_subcompactions: 1 2024-04-25T00:06:11.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T00:06:11.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T00:06:11.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T00:06:11.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T00:06:11.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T00:06:11.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T00:06:11.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T00:06:11.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T00:06:11.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_open_files: -1 2024-04-25T00:06:11.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T00:06:11.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T00:06:11.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T00:06:11.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T00:06:11.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_background_flushes: -1 2024-04-25T00:06:11.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Compression algorithms supported: 2024-04-25T00:06:11.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T00:06:11.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: kZSTD supported: 0 2024-04-25T00:06:11.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: kXpressCompression supported: 0 2024-04-25T00:06:11.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T00:06:11.474 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: kZlibCompression supported: 1 2024-04-25T00:06:11.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: kSnappyCompression supported: 1 2024-04-25T00:06:11.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: kLZ4Compression supported: 1 2024-04-25T00:06:11.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: kBZip2Compression supported: 0 2024-04-25T00:06:11.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T00:06:11.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T00:06:11.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T00:06:11.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T00:06:11.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T00:06:11.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.merge_operator: 2024-04-25T00:06:11.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.compaction_filter: None 2024-04-25T00:06:11.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T00:06:11.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T00:06:11.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T00:06:11.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T00:06:11.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555a8dd35ba0) 2024-04-25T00:06:11.476 INFO:journalctl@ceph.mon.a.smithi107.stdout: cache_index_and_filter_blocks: 1 2024-04-25T00:06:11.476 INFO:journalctl@ceph.mon.a.smithi107.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T00:06:11.476 INFO:journalctl@ceph.mon.a.smithi107.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T00:06:11.476 INFO:journalctl@ceph.mon.a.smithi107.stdout: pin_top_level_index_and_filter: 1 2024-04-25T00:06:11.476 INFO:journalctl@ceph.mon.a.smithi107.stdout: index_type: 0 2024-04-25T00:06:11.476 INFO:journalctl@ceph.mon.a.smithi107.stdout: data_block_index_type: 0 2024-04-25T00:06:11.476 INFO:journalctl@ceph.mon.a.smithi107.stdout: index_shortening: 1 2024-04-25T00:06:11.476 INFO:journalctl@ceph.mon.a.smithi107.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T00:06:11.476 INFO:journalctl@ceph.mon.a.smithi107.stdout: checksum: 4 2024-04-25T00:06:11.476 INFO:journalctl@ceph.mon.a.smithi107.stdout: no_block_cache: 0 2024-04-25T00:06:11.476 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache: 0x555a8dd39090 2024-04-25T00:06:11.476 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache_name: BinnedLRUCache 2024-04-25T00:06:11.476 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache_options: 2024-04-25T00:06:11.476 INFO:journalctl@ceph.mon.a.smithi107.stdout: capacity : 536870912 2024-04-25T00:06:11.476 INFO:journalctl@ceph.mon.a.smithi107.stdout: num_shard_bits : 4 2024-04-25T00:06:11.477 INFO:journalctl@ceph.mon.a.smithi107.stdout: strict_capacity_limit : 0 2024-04-25T00:06:11.477 INFO:journalctl@ceph.mon.a.smithi107.stdout: high_pri_pool_ratio: 0.000 2024-04-25T00:06:11.477 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache_compressed: (nil) 2024-04-25T00:06:11.477 INFO:journalctl@ceph.mon.a.smithi107.stdout: persistent_cache: (nil) 2024-04-25T00:06:11.477 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_size: 4096 2024-04-25T00:06:11.477 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_size_deviation: 10 2024-04-25T00:06:11.477 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_restart_interval: 16 2024-04-25T00:06:11.477 INFO:journalctl@ceph.mon.a.smithi107.stdout: index_block_restart_interval: 1 2024-04-25T00:06:11.477 INFO:journalctl@ceph.mon.a.smithi107.stdout: metadata_block_size: 4096 2024-04-25T00:06:11.477 INFO:journalctl@ceph.mon.a.smithi107.stdout: partition_filters: 0 2024-04-25T00:06:11.477 INFO:journalctl@ceph.mon.a.smithi107.stdout: use_delta_encoding: 1 2024-04-25T00:06:11.477 INFO:journalctl@ceph.mon.a.smithi107.stdout: filter_policy: bloomfilter 2024-04-25T00:06:11.477 INFO:journalctl@ceph.mon.a.smithi107.stdout: whole_key_filtering: 1 2024-04-25T00:06:11.477 INFO:journalctl@ceph.mon.a.smithi107.stdout: verify_compression: 0 2024-04-25T00:06:11.478 INFO:journalctl@ceph.mon.a.smithi107.stdout: read_amp_bytes_per_bit: 0 2024-04-25T00:06:11.478 INFO:journalctl@ceph.mon.a.smithi107.stdout: format_version: 5 2024-04-25T00:06:11.478 INFO:journalctl@ceph.mon.a.smithi107.stdout: enable_index_compression: 1 2024-04-25T00:06:11.478 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_align: 0 2024-04-25T00:06:11.478 INFO:journalctl@ceph.mon.a.smithi107.stdout: max_auto_readahead_size: 262144 2024-04-25T00:06:11.478 INFO:journalctl@ceph.mon.a.smithi107.stdout: prepopulate_block_cache: 0 2024-04-25T00:06:11.478 INFO:journalctl@ceph.mon.a.smithi107.stdout: initial_auto_readahead_size: 8192 2024-04-25T00:06:11.478 INFO:journalctl@ceph.mon.a.smithi107.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T00:06:11.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T00:06:11.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T00:06:11.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.compression: NoCompression 2024-04-25T00:06:11.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T00:06:11.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T00:06:11.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T00:06:11.478 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.num_levels: 7 2024-04-25T00:06:11.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T00:06:11.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T00:06:11.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T00:06:11.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T00:06:11.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T00:06:11.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T00:06:11.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T00:06:11.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T00:06:11.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T00:06:11.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T00:06:11.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T00:06:11.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T00:06:11.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T00:06:11.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T00:06:11.479 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T00:06:11.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T00:06:11.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T00:06:11.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T00:06:11.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T00:06:11.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T00:06:11.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T00:06:11.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T00:06:11.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T00:06:11.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T00:06:11.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T00:06:11.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T00:06:11.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T00:06:11.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T00:06:11.480 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T00:06:11.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T00:06:11.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T00:06:11.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T00:06:11.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T00:06:11.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T00:06:11.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T00:06:11.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T00:06:11.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T00:06:11.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T00:06:11.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T00:06:11.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T00:06:11.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T00:06:11.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T00:06:11.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T00:06:11.481 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T00:06:11.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T00:06:11.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T00:06:11.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T00:06:11.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T00:06:11.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T00:06:11.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T00:06:11.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T00:06:11.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T00:06:11.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T00:06:11.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.table_properties_collectors: 2024-04-25T00:06:11.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.inplace_update_support: 0 2024-04-25T00:06:11.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T00:06:11.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T00:06:11.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T00:06:11.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T00:06:11.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.bloom_locality: 0 2024-04-25T00:06:11.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.max_successive_merges: 0 2024-04-25T00:06:11.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T00:06:11.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T00:06:11.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T00:06:11.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T00:06:11.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.ttl: 2592000 2024-04-25T00:06:11.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T00:06:11.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T00:06:11.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T00:06:11.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.enable_blob_files: false 2024-04-25T00:06:11.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.min_blob_size: 0 2024-04-25T00:06:11.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T00:06:11.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T00:06:11.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T00:06:11.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T00:06:11.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T00:06:11.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T00:06:11.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T00:06:11.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T00:06:11.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T00:06:11.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T00:06:11.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8cf210bb-e815-4eba-8c60-72f8d5d69563 2024-04-25T00:06:11.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714003571421309, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T00:06:11.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T00:06:11.485 INFO:teuthology.orchestra.run.smithi107.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T00:06:11.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714003571422283, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100044, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 332, "table_properties": {"data_size": 97954, "index_size": 308, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14205, "raw_average_key_size": 49, "raw_value_size": 90229, "raw_average_value_size": 314, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714003571, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8cf210bb-e815-4eba-8c60-72f8d5d69563", "db_session_id": "QDVIHU4G94EEV7XP5YVP", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T00:06:11.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714003571422363, "job": 1, "event": "recovery_finished"} 2024-04-25T00:06:11.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T00:06:11.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T00:06:11.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x555a8de2a000 2024-04-25T00:06:11.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: DB pointer 0x555a8de14000 2024-04-25T00:06:11.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T00:06:11.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T00:06:11.486 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** DB Stats ** 2024-04-25T00:06:11.486 INFO:journalctl@ceph.mon.a.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T00:06:11.486 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T00:06:11.486 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T00:06:11.486 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T00:06:11.486 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T00:06:11.486 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T00:06:11.539 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T00:06:11.540 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-04-25T00:06:11.540 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** Compaction Stats [default] ** 2024-04-25T00:06:11.540 INFO:journalctl@ceph.mon.a.smithi107.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T00:06:11.540 INFO:journalctl@ceph.mon.a.smithi107.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T00:06:11.540 INFO:journalctl@ceph.mon.a.smithi107.stdout: L0 2/0 99.53 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:06:11.540 INFO:journalctl@ceph.mon.a.smithi107.stdout: Sum 2/0 99.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:06:11.540 INFO:journalctl@ceph.mon.a.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 156.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:06:11.540 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-04-25T00:06:11.540 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** Compaction Stats [default] ** 2024-04-25T00:06:11.540 INFO:journalctl@ceph.mon.a.smithi107.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T00:06:11.540 INFO:journalctl@ceph.mon.a.smithi107.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T00:06:11.540 INFO:journalctl@ceph.mon.a.smithi107.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 156.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:06:11.540 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-04-25T00:06:11.541 INFO:journalctl@ceph.mon.a.smithi107.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T00:06:11.541 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-04-25T00:06:11.541 INFO:journalctl@ceph.mon.a.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T00:06:11.541 INFO:journalctl@ceph.mon.a.smithi107.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T00:06:11.541 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T00:06:11.541 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T00:06:11.541 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T00:06:11.541 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T00:06:11.541 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative compaction: 0.00 GB write, 33.08 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T00:06:11.541 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval compaction: 0.00 GB write, 33.08 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T00:06:11.541 INFO:journalctl@ceph.mon.a.smithi107.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T00:06:11.541 INFO:journalctl@ceph.mon.a.smithi107.stdout: Block cache BinnedLRUCache@0x555a8dd39090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T00:06:11.541 INFO:journalctl@ceph.mon.a.smithi107.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T00:06:11.541 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-04-25T00:06:11.542 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T00:06:11.542 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: starting mon.a rank 0 at public addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] at bind addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:06:11.542 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: mon.a@-1(???) e1 preinit fsid 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:06:11.542 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: mon.a@-1(???).mds e1 new map 2024-04-25T00:06:11.542 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: mon.a@-1(???).mds e1 print_map 2024-04-25T00:06:11.542 INFO:journalctl@ceph.mon.a.smithi107.stdout: e1 2024-04-25T00:06:11.542 INFO:journalctl@ceph.mon.a.smithi107.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T00:06:11.542 INFO:journalctl@ceph.mon.a.smithi107.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T00:06:11.543 INFO:journalctl@ceph.mon.a.smithi107.stdout: legacy client fscid: -1 2024-04-25T00:06:11.543 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-04-25T00:06:11.543 INFO:journalctl@ceph.mon.a.smithi107.stdout: No filesystems configured 2024-04-25T00:06:11.543 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T00:06:11.543 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:06:11.543 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:06:11.543 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:06:11.543 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T00:06:11.543 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 bash[29135]: 8662034e7721df30bfaf633987d37d05c01485dfd9cce32ba3dbfc5ca1c54ed2 2024-04-25T00:06:11.543 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 systemd[1]: Started Ceph mon.a for 79602f58-0297-11ef-bc93-c7b262605968. 2024-04-25T00:06:11.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T00:06:11.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: monmap epoch 1 2024-04-25T00:06:11.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: fsid 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:06:11.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: last_changed 2024-04-25T00:05:52.163469+0000 2024-04-25T00:06:11.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: created 2024-04-25T00:05:52.163469+0000 2024-04-25T00:06:11.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: min_mon_release 19 (squid) 2024-04-25T00:06:11.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: election_strategy: 1 2024-04-25T00:06:11.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-04-25T00:06:11.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: fsmap 2024-04-25T00:06:11.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T00:06:11.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:11 smithi107 ceph-mon[29153]: mgrmap e1: no daemons active 2024-04-25T00:06:12.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.842+0000 7f6ffae19700 1 Processor -- start 2024-04-25T00:06:12.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.842+0000 7f6ffae19700 1 -- start start 2024-04-25T00:06:12.428 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.842+0000 7f6ffae19700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff4107040 0x7f6ff4107460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:12.429 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.842+0000 7f6ffae19700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff4107a30 con 0x7f6ff4107040 2024-04-25T00:06:12.429 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.842+0000 7f6ff8bb5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff4107040 0x7f6ff4107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:12.429 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.842+0000 7f6ff8bb5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff4107040 0x7f6ff4107460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56142/0 (socket says 172.21.15.107:56142) 2024-04-25T00:06:12.429 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.842+0000 7f6ff8bb5700 1 -- 172.21.15.107:0/3414508862 learned_addr learned my addr 172.21.15.107:0/3414508862 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:12.429 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.843+0000 7f6ff8bb5700 1 -- 172.21.15.107:0/3414508862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff4107d60 con 0x7f6ff4107040 2024-04-25T00:06:12.429 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.843+0000 7f6ff8bb5700 1 --2- 172.21.15.107:0/3414508862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff4107040 0x7f6ff4107460 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6fe80098a0 tx=0x7f6fe8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f06c56a8f3431d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:12.429 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.843+0000 7f6ff37fe700 1 -- 172.21.15.107:0/3414508862 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fe8004160 con 0x7f6ff4107040 2024-04-25T00:06:12.429 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.843+0000 7f6ff37fe700 1 -- 172.21.15.107:0/3414508862 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6fe8004300 con 0x7f6ff4107040 2024-04-25T00:06:12.429 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.843+0000 7f6ff37fe700 1 -- 172.21.15.107:0/3414508862 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fe8011b50 con 0x7f6ff4107040 2024-04-25T00:06:12.429 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.844+0000 7f6ffae19700 1 -- 172.21.15.107:0/3414508862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff4107040 msgr2=0x7f6ff4107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:12.429 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.844+0000 7f6ffae19700 1 --2- 172.21.15.107:0/3414508862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff4107040 0x7f6ff4107460 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6fe80098a0 tx=0x7f6fe8009bb0 comp rx=0 tx=0).stop 2024-04-25T00:06:12.429 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.844+0000 7f6ffae19700 1 -- 172.21.15.107:0/3414508862 shutdown_connections 2024-04-25T00:06:12.429 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.844+0000 7f6ffae19700 1 --2- 172.21.15.107:0/3414508862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff4107040 0x7f6ff4107460 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:12.429 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.844+0000 7f6ffae19700 1 -- 172.21.15.107:0/3414508862 >> 172.21.15.107:0/3414508862 conn(0x7f6ff41018a0 msgr2=0x7f6ff4103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:12.430 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.844+0000 7f6ffae19700 1 -- 172.21.15.107:0/3414508862 shutdown_connections 2024-04-25T00:06:12.430 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.844+0000 7f6ffae19700 1 -- 172.21.15.107:0/3414508862 wait complete. 2024-04-25T00:06:12.430 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.844+0000 7f6ffae19700 1 Processor -- start 2024-04-25T00:06:12.430 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.844+0000 7f6ffae19700 1 -- start start 2024-04-25T00:06:12.430 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.845+0000 7f6ffae19700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff4107040 0x7f6ff419bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:12.430 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.845+0000 7f6ffae19700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff410b110 con 0x7f6ff4107040 2024-04-25T00:06:12.430 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.845+0000 7f6ff8bb5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff4107040 0x7f6ff419bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:12.430 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.845+0000 7f6ff8bb5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff4107040 0x7f6ff419bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56146/0 (socket says 172.21.15.107:56146) 2024-04-25T00:06:12.430 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.845+0000 7f6ff8bb5700 1 -- 172.21.15.107:0/3312472664 learned_addr learned my addr 172.21.15.107:0/3312472664 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:12.430 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.845+0000 7f6ff8bb5700 1 -- 172.21.15.107:0/3312472664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff419c1a0 con 0x7f6ff4107040 2024-04-25T00:06:12.430 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.845+0000 7f6ff8bb5700 1 --2- 172.21.15.107:0/3312472664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff4107040 0x7f6ff419bc60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009fb0 tx=0x7f6fe8011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:12.430 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.845+0000 7f6ff1ffb700 1 -- 172.21.15.107:0/3312472664 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fe80038b0 con 0x7f6ff4107040 2024-04-25T00:06:12.430 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.845+0000 7f6ff1ffb700 1 -- 172.21.15.107:0/3312472664 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6fe8011e50 con 0x7f6ff4107040 2024-04-25T00:06:12.431 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.845+0000 7f6ffae19700 1 -- 172.21.15.107:0/3312472664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff419c430 con 0x7f6ff4107040 2024-04-25T00:06:12.431 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.845+0000 7f6ff1ffb700 1 -- 172.21.15.107:0/3312472664 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fe8019420 con 0x7f6ff4107040 2024-04-25T00:06:12.431 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.845+0000 7f6ffae19700 1 -- 172.21.15.107:0/3312472664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff419f120 con 0x7f6ff4107040 2024-04-25T00:06:12.431 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.846+0000 7f6ff1ffb700 1 -- 172.21.15.107:0/3312472664 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6fe8019a80 con 0x7f6ff4107040 2024-04-25T00:06:12.431 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.846+0000 7f6ff1ffb700 1 -- 172.21.15.107:0/3312472664 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6fe8024730 con 0x7f6ff4107040 2024-04-25T00:06:12.431 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.846+0000 7f6ffae19700 1 -- 172.21.15.107:0/3312472664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ff4063890 con 0x7f6ff4107040 2024-04-25T00:06:12.431 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.849+0000 7f6ff1ffb700 1 -- 172.21.15.107:0/3312472664 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f6fe8043dd0 con 0x7f6ff4107040 2024-04-25T00:06:12.431 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.915+0000 7f6ffae19700 1 -- 172.21.15.107:0/3312472664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f6ff402ddf0 con 0x7f6ff4107040 2024-04-25T00:06:12.431 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.919+0000 7f6ff1ffb700 1 -- 172.21.15.107:0/3312472664 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f6fe8032030 con 0x7f6ff4107040 2024-04-25T00:06:12.431 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.920+0000 7f6ffae19700 1 -- 172.21.15.107:0/3312472664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff4107040 msgr2=0x7f6ff419bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:12.431 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.920+0000 7f6ffae19700 1 --2- 172.21.15.107:0/3312472664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff4107040 0x7f6ff419bc60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009fb0 tx=0x7f6fe8011bf0 comp rx=0 tx=0).stop 2024-04-25T00:06:12.431 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.920+0000 7f6ffae19700 1 -- 172.21.15.107:0/3312472664 shutdown_connections 2024-04-25T00:06:12.431 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.920+0000 7f6ffae19700 1 --2- 172.21.15.107:0/3312472664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff4107040 0x7f6ff419bc60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:12.431 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.920+0000 7f6ffae19700 1 -- 172.21.15.107:0/3312472664 >> 172.21.15.107:0/3312472664 conn(0x7f6ff41018a0 msgr2=0x7f6ff4102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:12.431 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.920+0000 7f6ffae19700 1 -- 172.21.15.107:0/3312472664 shutdown_connections 2024-04-25T00:06:12.432 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:11.920+0000 7f6ffae19700 1 -- 172.21.15.107:0/3312472664 wait complete. 2024-04-25T00:06:12.432 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T00:06:12.445 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:06:12.445 INFO:teuthology.orchestra.run.smithi107.stdout:Creating mgr... 2024-04-25T00:06:12.446 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T00:06:12.446 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T00:06:12.689 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-79602f58-0297-11ef-bc93-c7b262605968@mgr.x 2024-04-25T00:06:12.689 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to reset failed state of unit ceph-79602f58-0297-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-79602f58-0297-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T00:06:12.836 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-79602f58-0297-11ef-bc93-c7b262605968.target.wants/ceph-79602f58-0297-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-79602f58-0297-11ef-bc93-c7b262605968@.service. 2024-04-25T00:06:12.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:12 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3312472664' entity='client.admin' 2024-04-25T00:06:13.484 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:13 smithi107 podman[29367]: 2024-04-25 00:06:12.971222178 +0000 UTC m=+0.017879318 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:06:14.592 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:14 smithi107 podman[29367]: 2024-04-25 00:06:14.239165611 +0000 UTC m=+1.285822752 container create 081648f9ff72297e7f440f7ab3000caaefe90cb202b9a4d6ae28841084bc278c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T00:06:14.592 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:14 smithi107 podman[29367]: 2024-04-25 00:06:14.483902462 +0000 UTC m=+1.530559604 container init 081648f9ff72297e7f440f7ab3000caaefe90cb202b9a4d6ae28841084bc278c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T00:06:14.592 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:14 smithi107 podman[29367]: 2024-04-25 00:06:14.48705688 +0000 UTC m=+1.533714025 container start 081648f9ff72297e7f440f7ab3000caaefe90cb202b9a4d6ae28841084bc278c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-25T00:06:14.607 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T00:06:14.607 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-04-25T00:06:14.614 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T00:06:14.614 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-04-25T00:06:14.614 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-04-25T00:06:14.614 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T00:06:14.622 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T00:06:14.623 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-04-25T00:06:14.630 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T00:06:14.630 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-04-25T00:06:14.630 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-04-25T00:06:14.630 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T00:06:14.630 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr to start... 2024-04-25T00:06:14.630 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr... 2024-04-25T00:06:14.845 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:14 smithi107 bash[29367]: 081648f9ff72297e7f440f7ab3000caaefe90cb202b9a4d6ae28841084bc278c 2024-04-25T00:06:14.845 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:14 smithi107 systemd[1]: Started Ceph mgr.x for 79602f58-0297-11ef-bc93-c7b262605968. 2024-04-25T00:06:14.845 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:14 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:14.642+0000 7f867c8f2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:06:15.144 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:14 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:14.845+0000 7f867c8f2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:06:15.144 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:14 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:14.952+0000 7f867c8f2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:06:15.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:15 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2955319075' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T00:06:15.484 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:15 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:15.280+0000 7f867c8f2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:06:15.484 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:15 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:15.366+0000 7f867c8f2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:06:15.755 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-25T00:06:15.755 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:06:15.755 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "79602f58-0297-11ef-bc93-c7b262605968", 2024-04-25T00:06:15.755 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T00:06:15.755 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T00:06:15.755 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T00:06:15.755 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T00:06:15.756 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:15.756 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T00:06:15.756 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T00:06:15.756 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-04-25T00:06:15.756 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:06:15.756 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T00:06:15.756 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T00:06:15.756 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:06:15.756 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-25T00:06:15.756 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T00:06:15.756 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:06:15.756 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T00:06:15.756 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T00:06:15.756 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:15.756 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T00:06:15.757 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:06:15.757 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T00:06:15.757 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T00:06:15.757 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T00:06:15.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T00:06:15.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T00:06:15.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T00:06:15.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:15.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T00:06:15.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T00:06:15.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T00:06:15.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T00:06:15.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T00:06:15.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T00:06:15.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T00:06:15.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T00:06:15.766 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T00:06:15.767 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:15.767 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T00:06:15.767 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:06:15.767 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T00:06:15.767 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T00:06:15.767 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:15.767 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T00:06:15.767 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T00:06:15.767 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T00:06:15.767 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T00:06:15.767 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T00:06:15.767 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T00:06:15.767 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T00:06:15.768 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:06:15.768 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:06:15.768 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:15.768 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T00:06:15.768 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:06:15.768 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T00:05:56.053812+0000", 2024-04-25T00:06:15.768 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:06:15.768 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:15.768 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T00:06:15.768 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:06:15.768 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.063+0000 7f030847e700 1 Processor -- start 2024-04-25T00:06:15.768 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.064+0000 7f030847e700 1 -- start start 2024-04-25T00:06:15.768 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.064+0000 7f030847e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0300107010 0x7f0300107430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:15.768 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.064+0000 7f030847e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0300107a00 con 0x7f0300107010 2024-04-25T00:06:15.769 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.064+0000 7f030621a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0300107010 0x7f0300107430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:15.769 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.064+0000 7f030621a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0300107010 0x7f0300107430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56186/0 (socket says 172.21.15.107:56186) 2024-04-25T00:06:15.769 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.064+0000 7f030621a700 1 -- 172.21.15.107:0/1256956231 learned_addr learned my addr 172.21.15.107:0/1256956231 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:15.769 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.065+0000 7f030621a700 1 -- 172.21.15.107:0/1256956231 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0300107d30 con 0x7f0300107010 2024-04-25T00:06:15.769 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.065+0000 7f030621a700 1 --2- 172.21.15.107:0/1256956231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0300107010 0x7f0300107430 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0098a0 tx=0x7f02fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9568be8dca7292 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:15.769 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.065+0000 7f0305218700 1 -- 172.21.15.107:0/1256956231 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02fc004160 con 0x7f0300107010 2024-04-25T00:06:15.769 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.065+0000 7f0305218700 1 -- 172.21.15.107:0/1256956231 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f02fc004300 con 0x7f0300107010 2024-04-25T00:06:15.769 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.065+0000 7f0305218700 1 -- 172.21.15.107:0/1256956231 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02fc011b50 con 0x7f0300107010 2024-04-25T00:06:15.769 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.066+0000 7f030847e700 1 -- 172.21.15.107:0/1256956231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0300107010 msgr2=0x7f0300107430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:15.769 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.066+0000 7f030847e700 1 --2- 172.21.15.107:0/1256956231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0300107010 0x7f0300107430 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0098a0 tx=0x7f02fc009bb0 comp rx=0 tx=0).stop 2024-04-25T00:06:15.769 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.066+0000 7f030847e700 1 -- 172.21.15.107:0/1256956231 shutdown_connections 2024-04-25T00:06:15.769 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.066+0000 7f030847e700 1 --2- 172.21.15.107:0/1256956231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0300107010 0x7f0300107430 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:15.769 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.066+0000 7f030847e700 1 -- 172.21.15.107:0/1256956231 >> 172.21.15.107:0/1256956231 conn(0x7f0300101810 msgr2=0x7f0300103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:15.770 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.066+0000 7f030847e700 1 -- 172.21.15.107:0/1256956231 shutdown_connections 2024-04-25T00:06:15.770 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.066+0000 7f030847e700 1 -- 172.21.15.107:0/1256956231 wait complete. 2024-04-25T00:06:15.770 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.066+0000 7f030847e700 1 Processor -- start 2024-04-25T00:06:15.770 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.066+0000 7f030847e700 1 -- start start 2024-04-25T00:06:15.770 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.067+0000 7f030847e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0300107010 0x7f030019bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:15.770 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.067+0000 7f030847e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f030010b0e0 con 0x7f0300107010 2024-04-25T00:06:15.770 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.067+0000 7f030621a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0300107010 0x7f030019bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:15.770 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.067+0000 7f030621a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0300107010 0x7f030019bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56194/0 (socket says 172.21.15.107:56194) 2024-04-25T00:06:15.770 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.067+0000 7f030621a700 1 -- 172.21.15.107:0/2955319075 learned_addr learned my addr 172.21.15.107:0/2955319075 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:15.770 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.067+0000 7f030621a700 1 -- 172.21.15.107:0/2955319075 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030019c140 con 0x7f0300107010 2024-04-25T00:06:15.770 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.067+0000 7f030621a700 1 --2- 172.21.15.107:0/2955319075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0300107010 0x7f030019bc00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f02fc011670 tx=0x7f02fc011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:15.770 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.067+0000 7f02f77fe700 1 -- 172.21.15.107:0/2955319075 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02fc0039c0 con 0x7f0300107010 2024-04-25T00:06:15.770 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.067+0000 7f02f77fe700 1 -- 172.21.15.107:0/2955319075 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f02fc011e50 con 0x7f0300107010 2024-04-25T00:06:15.771 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.067+0000 7f030847e700 1 -- 172.21.15.107:0/2955319075 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f030019c3d0 con 0x7f0300107010 2024-04-25T00:06:15.771 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.068+0000 7f030847e700 1 -- 172.21.15.107:0/2955319075 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f030019f0c0 con 0x7f0300107010 2024-04-25T00:06:15.771 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.067+0000 7f02f77fe700 1 -- 172.21.15.107:0/2955319075 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02fc01a5c0 con 0x7f0300107010 2024-04-25T00:06:15.771 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.068+0000 7f02f77fe700 1 -- 172.21.15.107:0/2955319075 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f02fc01ac00 con 0x7f0300107010 2024-04-25T00:06:15.771 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.068+0000 7f02f77fe700 1 -- 172.21.15.107:0/2955319075 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f02fc019700 con 0x7f0300107010 2024-04-25T00:06:15.771 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.069+0000 7f030847e700 1 -- 172.21.15.107:0/2955319075 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0300063890 con 0x7f0300107010 2024-04-25T00:06:15.771 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.071+0000 7f02f77fe700 1 -- 172.21.15.107:0/2955319075 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f02fc003b60 con 0x7f0300107010 2024-04-25T00:06:15.771 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.144+0000 7f030847e700 1 -- 172.21.15.107:0/2955319075 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f03000008d0 con 0x7f0300107010 2024-04-25T00:06:15.771 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.144+0000 7f02f77fe700 1 -- 172.21.15.107:0/2955319075 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f02fc0199a0 con 0x7f0300107010 2024-04-25T00:06:15.771 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.145+0000 7f030847e700 1 -- 172.21.15.107:0/2955319075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0300107010 msgr2=0x7f030019bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:15.771 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.145+0000 7f030847e700 1 --2- 172.21.15.107:0/2955319075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0300107010 0x7f030019bc00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f02fc011670 tx=0x7f02fc011b60 comp rx=0 tx=0).stop 2024-04-25T00:06:15.771 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.146+0000 7f030847e700 1 -- 172.21.15.107:0/2955319075 shutdown_connections 2024-04-25T00:06:15.771 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.146+0000 7f030847e700 1 --2- 172.21.15.107:0/2955319075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0300107010 0x7f030019bc00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:15.771 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.146+0000 7f030847e700 1 -- 172.21.15.107:0/2955319075 >> 172.21.15.107:0/2955319075 conn(0x7f0300101810 msgr2=0x7f0300102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:15.772 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.146+0000 7f030847e700 1 -- 172.21.15.107:0/2955319075 shutdown_connections 2024-04-25T00:06:15.772 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:15.146+0000 7f030847e700 1 -- 172.21.15.107:0/2955319075 wait complete. 2024-04-25T00:06:15.772 INFO:teuthology.orchestra.run.smithi107.stdout:mgr not available, waiting (1/15)... 2024-04-25T00:06:16.901 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:16 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:16.567+0000 7f867c8f2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:06:16.901 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:16 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:16.732+0000 7f867c8f2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:06:16.902 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:16 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:16.901+0000 7f867c8f2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:06:17.478 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:17 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:17.185+0000 7f867c8f2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:06:17.478 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:17 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:17.276+0000 7f867c8f2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:06:17.478 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:17 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:17.477+0000 7f867c8f2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:06:17.734 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:17 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:17.667+0000 7f867c8f2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:06:18.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:18 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3678864819' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T00:06:18.484 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:18 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:18.298+0000 7f867c8f2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:06:18.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-25T00:06:18.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:06:18.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "79602f58-0297-11ef-bc93-c7b262605968", 2024-04-25T00:06:18.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T00:06:18.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T00:06:18.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T00:06:18.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T00:06:18.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:18.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T00:06:18.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T00:06:18.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-04-25T00:06:18.877 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:06:18.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T00:06:18.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T00:06:18.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:06:18.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-04-25T00:06:18.878 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T00:06:18.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:06:18.879 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T00:06:18.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T00:06:18.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:18.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T00:06:18.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:06:18.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T00:06:18.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T00:06:18.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T00:06:18.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T00:06:18.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T00:06:18.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T00:06:18.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:18.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T00:06:18.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T00:06:18.880 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T00:06:18.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T00:06:18.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T00:06:18.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T00:06:18.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T00:06:18.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T00:06:18.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T00:06:18.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:18.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T00:06:18.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:06:18.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T00:06:18.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T00:06:18.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:18.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T00:06:18.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T00:06:18.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T00:06:18.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T00:06:18.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T00:06:18.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T00:06:18.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T00:06:18.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:06:18.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:06:18.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:18.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T00:06:18.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:06:18.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T00:05:56.053812+0000", 2024-04-25T00:06:18.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:06:18.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:18.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T00:06:18.882 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:06:18.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.178+0000 7fba1ef5d700 1 Processor -- start 2024-04-25T00:06:18.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.178+0000 7fba1ef5d700 1 -- start start 2024-04-25T00:06:18.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.179+0000 7fba1ef5d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba18107c70 0x7fba18108090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:18.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.179+0000 7fba1ef5d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fba18108660 con 0x7fba18107c70 2024-04-25T00:06:18.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.179+0000 7fba1ccf9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba18107c70 0x7fba18108090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:18.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.179+0000 7fba1ccf9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba18107c70 0x7fba18108090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56208/0 (socket says 172.21.15.107:56208) 2024-04-25T00:06:18.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.179+0000 7fba1ccf9700 1 -- 172.21.15.107:0/351923890 learned_addr learned my addr 172.21.15.107:0/351923890 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:18.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.179+0000 7fba1ccf9700 1 -- 172.21.15.107:0/351923890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba18108e70 con 0x7fba18107c70 2024-04-25T00:06:18.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.179+0000 7fba1ccf9700 1 --2- 172.21.15.107:0/351923890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba18107c70 0x7fba18108090 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0098a0 tx=0x7fba0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59cc329bf2edd7cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:18.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.179+0000 7fba177fe700 1 -- 172.21.15.107:0/351923890 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba0c004160 con 0x7fba18107c70 2024-04-25T00:06:18.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.179+0000 7fba177fe700 1 -- 172.21.15.107:0/351923890 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fba0c004300 con 0x7fba18107c70 2024-04-25T00:06:18.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.179+0000 7fba177fe700 1 -- 172.21.15.107:0/351923890 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba0c011b50 con 0x7fba18107c70 2024-04-25T00:06:18.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.180+0000 7fba1ef5d700 1 -- 172.21.15.107:0/351923890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba18107c70 msgr2=0x7fba18108090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:18.883 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.180+0000 7fba1ef5d700 1 --2- 172.21.15.107:0/351923890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba18107c70 0x7fba18108090 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0098a0 tx=0x7fba0c009bb0 comp rx=0 tx=0).stop 2024-04-25T00:06:18.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.180+0000 7fba1ef5d700 1 -- 172.21.15.107:0/351923890 shutdown_connections 2024-04-25T00:06:18.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.180+0000 7fba1ef5d700 1 --2- 172.21.15.107:0/351923890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba18107c70 0x7fba18108090 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:18.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.180+0000 7fba1ef5d700 1 -- 172.21.15.107:0/351923890 >> 172.21.15.107:0/351923890 conn(0x7fba18103210 msgr2=0x7fba18105650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:18.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.180+0000 7fba1ef5d700 1 -- 172.21.15.107:0/351923890 shutdown_connections 2024-04-25T00:06:18.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.180+0000 7fba1ef5d700 1 -- 172.21.15.107:0/351923890 wait complete. 2024-04-25T00:06:18.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.181+0000 7fba1ef5d700 1 Processor -- start 2024-04-25T00:06:18.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.181+0000 7fba1ef5d700 1 -- start start 2024-04-25T00:06:18.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.181+0000 7fba1ef5d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba18107c70 0x7fba1807db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:18.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.181+0000 7fba1ef5d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fba1810c220 con 0x7fba18107c70 2024-04-25T00:06:18.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.181+0000 7fba1ccf9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba18107c70 0x7fba1807db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:18.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.181+0000 7fba1ccf9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba18107c70 0x7fba1807db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56212/0 (socket says 172.21.15.107:56212) 2024-04-25T00:06:18.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.181+0000 7fba1ccf9700 1 -- 172.21.15.107:0/3678864819 learned_addr learned my addr 172.21.15.107:0/3678864819 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:18.884 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.181+0000 7fba1ccf9700 1 -- 172.21.15.107:0/3678864819 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1807e0b0 con 0x7fba18107c70 2024-04-25T00:06:18.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.181+0000 7fba1ccf9700 1 --2- 172.21.15.107:0/3678864819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba18107c70 0x7fba1807db70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fba0c011670 tx=0x7fba0c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:18.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.182+0000 7fba15ffb700 1 -- 172.21.15.107:0/3678864819 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba0c0039c0 con 0x7fba18107c70 2024-04-25T00:06:18.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.182+0000 7fba1ef5d700 1 -- 172.21.15.107:0/3678864819 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba1807c2a0 con 0x7fba18107c70 2024-04-25T00:06:18.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.182+0000 7fba15ffb700 1 -- 172.21.15.107:0/3678864819 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fba0c011e50 con 0x7fba18107c70 2024-04-25T00:06:18.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.182+0000 7fba15ffb700 1 -- 172.21.15.107:0/3678864819 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba0c01a5c0 con 0x7fba18107c70 2024-04-25T00:06:18.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.182+0000 7fba1ef5d700 1 -- 172.21.15.107:0/3678864819 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba1807c7e0 con 0x7fba18107c70 2024-04-25T00:06:18.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.182+0000 7fba15ffb700 1 -- 172.21.15.107:0/3678864819 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fba0c01ac00 con 0x7fba18107c70 2024-04-25T00:06:18.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.182+0000 7fba15ffb700 1 -- 172.21.15.107:0/3678864819 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fba0c019700 con 0x7fba18107c70 2024-04-25T00:06:18.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.183+0000 7fba1ef5d700 1 -- 172.21.15.107:0/3678864819 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba18063890 con 0x7fba18107c70 2024-04-25T00:06:18.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.186+0000 7fba15ffb700 1 -- 172.21.15.107:0/3678864819 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fba0c003b60 con 0x7fba18107c70 2024-04-25T00:06:18.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.260+0000 7fba1ef5d700 1 -- 172.21.15.107:0/3678864819 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fba1807cd10 con 0x7fba18107c70 2024-04-25T00:06:18.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.260+0000 7fba15ffb700 1 -- 172.21.15.107:0/3678864819 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fba0c0199a0 con 0x7fba18107c70 2024-04-25T00:06:18.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.262+0000 7fba1ef5d700 1 -- 172.21.15.107:0/3678864819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba18107c70 msgr2=0x7fba1807db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:18.885 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.262+0000 7fba1ef5d700 1 --2- 172.21.15.107:0/3678864819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba18107c70 0x7fba1807db70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fba0c011670 tx=0x7fba0c011b60 comp rx=0 tx=0).stop 2024-04-25T00:06:18.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.262+0000 7fba1ef5d700 1 -- 172.21.15.107:0/3678864819 shutdown_connections 2024-04-25T00:06:18.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.262+0000 7fba1ef5d700 1 --2- 172.21.15.107:0/3678864819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba18107c70 0x7fba1807db70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:18.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.262+0000 7fba1ef5d700 1 -- 172.21.15.107:0/3678864819 >> 172.21.15.107:0/3678864819 conn(0x7fba18103210 msgr2=0x7fba18103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:18.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.262+0000 7fba1ef5d700 1 -- 172.21.15.107:0/3678864819 shutdown_connections 2024-04-25T00:06:18.886 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:18.262+0000 7fba1ef5d700 1 -- 172.21.15.107:0/3678864819 wait complete. 2024-04-25T00:06:18.886 INFO:teuthology.orchestra.run.smithi107.stdout:mgr not available, waiting (2/15)... 2024-04-25T00:06:18.984 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:18 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:18.738+0000 7f867c8f2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:06:18.984 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:18 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:18.835+0000 7f867c8f2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:06:19.484 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:19 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:19.035+0000 7f867c8f2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:06:19.886 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:19 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:19.556+0000 7f867c8f2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:06:19.886 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:19 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:19.646+0000 7f867c8f2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:06:20.228 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:19 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:19.885+0000 7f867c8f2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:06:20.228 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:20 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:20.018+0000 7f867c8f2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:06:20.228 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:20 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:20.112+0000 7f867c8f2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:06:20.228 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:20 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:20.227+0000 7f867c8f2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:06:20.484 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:20 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:20.317+0000 7f867c8f2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:06:21.241 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:21 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:21.148+0000 7f867c8f2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:06:21.661 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:21 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:21.377+0000 7f867c8f2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:06:21.661 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:21 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1908649717' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T00:06:21.662 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:21 smithi107 ceph-mon[29153]: Activating manager daemon x 2024-04-25T00:06:21.662 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:21 smithi107 ceph-mon[29153]: mgrmap e2: x(active, starting, since 0.00313956s) 2024-04-25T00:06:21.662 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:21 smithi107 ceph-mon[29153]: from='mgr.14100 172.21.15.107:0/3901897199' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T00:06:21.662 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:21 smithi107 ceph-mon[29153]: from='mgr.14100 172.21.15.107:0/3901897199' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T00:06:21.662 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:21 smithi107 ceph-mon[29153]: from='mgr.14100 172.21.15.107:0/3901897199' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T00:06:21.662 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:21 smithi107 ceph-mon[29153]: from='mgr.14100 172.21.15.107:0/3901897199' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:06:21.662 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:21 smithi107 ceph-mon[29153]: from='mgr.14100 172.21.15.107:0/3901897199' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T00:06:21.662 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:21 smithi107 ceph-mon[29153]: Manager daemon x is now available 2024-04-25T00:06:21.970 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-25T00:06:21.970 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:06:21.970 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "79602f58-0297-11ef-bc93-c7b262605968", 2024-04-25T00:06:21.970 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T00:06:21.970 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T00:06:21.970 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T00:06:21.970 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T00:06:21.970 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:21.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T00:06:21.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T00:06:21.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-04-25T00:06:21.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:06:21.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T00:06:21.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T00:06:21.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:06:21.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-04-25T00:06:21.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T00:06:21.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:06:21.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T00:06:21.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T00:06:21.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:21.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T00:06:21.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:06:21.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T00:06:21.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T00:06:21.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T00:06:21.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T00:06:21.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T00:06:21.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T00:06:21.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:21.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T00:06:21.973 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T00:06:21.973 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T00:06:21.973 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T00:06:21.973 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T00:06:21.973 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T00:06:21.973 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T00:06:21.973 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T00:06:21.973 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T00:06:21.973 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:21.973 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T00:06:21.973 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:06:21.973 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T00:06:21.974 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T00:06:21.974 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:21.974 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T00:06:21.974 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T00:06:21.974 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T00:06:21.974 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T00:06:21.974 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T00:06:21.974 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T00:06:21.974 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T00:06:21.974 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:06:21.975 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:06:21.975 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:21.975 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T00:06:21.975 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:06:21.975 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T00:05:56.053812+0000", 2024-04-25T00:06:21.975 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:06:21.975 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:21.975 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T00:06:21.975 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:06:21.976 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.263+0000 7fe1f5a78700 1 Processor -- start 2024-04-25T00:06:21.976 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.263+0000 7fe1f5a78700 1 -- start start 2024-04-25T00:06:21.976 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.263+0000 7fe1f5a78700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1f01086e0 0x7fe1f010aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:21.976 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.263+0000 7fe1f5a78700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f007beb0 con 0x7fe1f01086e0 2024-04-25T00:06:21.976 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.263+0000 7fe1ef7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1f01086e0 0x7fe1f010aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:21.976 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.263+0000 7fe1ef7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1f01086e0 0x7fe1f010aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46182/0 (socket says 172.21.15.107:46182) 2024-04-25T00:06:21.976 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.263+0000 7fe1ef7fe700 1 -- 172.21.15.107:0/2702580177 learned_addr learned my addr 172.21.15.107:0/2702580177 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:21.976 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.264+0000 7fe1ef7fe700 1 -- 172.21.15.107:0/2702580177 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f010b010 con 0x7fe1f01086e0 2024-04-25T00:06:21.976 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.264+0000 7fe1ef7fe700 1 --2- 172.21.15.107:0/2702580177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1f01086e0 0x7fe1f010aad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe1e00098a0 tx=0x7fe1e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7a5b00b7f2f1e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:21.976 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.264+0000 7fe1ee7fc700 1 -- 172.21.15.107:0/2702580177 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1e0004160 con 0x7fe1f01086e0 2024-04-25T00:06:21.976 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.264+0000 7fe1ee7fc700 1 -- 172.21.15.107:0/2702580177 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe1e0004300 con 0x7fe1f01086e0 2024-04-25T00:06:21.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.264+0000 7fe1ee7fc700 1 -- 172.21.15.107:0/2702580177 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1e0011b50 con 0x7fe1f01086e0 2024-04-25T00:06:21.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.265+0000 7fe1f5a78700 1 -- 172.21.15.107:0/2702580177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1f01086e0 msgr2=0x7fe1f010aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:21.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.265+0000 7fe1f5a78700 1 --2- 172.21.15.107:0/2702580177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1f01086e0 0x7fe1f010aad0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe1e00098a0 tx=0x7fe1e0009bb0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.265+0000 7fe1f5a78700 1 -- 172.21.15.107:0/2702580177 shutdown_connections 2024-04-25T00:06:21.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.265+0000 7fe1f5a78700 1 --2- 172.21.15.107:0/2702580177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1f01086e0 0x7fe1f010aad0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.265+0000 7fe1f5a78700 1 -- 172.21.15.107:0/2702580177 >> 172.21.15.107:0/2702580177 conn(0x7fe1f0102130 msgr2=0x7fe1f0104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:21.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.265+0000 7fe1f5a78700 1 -- 172.21.15.107:0/2702580177 shutdown_connections 2024-04-25T00:06:21.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.265+0000 7fe1f5a78700 1 -- 172.21.15.107:0/2702580177 wait complete. 2024-04-25T00:06:21.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.265+0000 7fe1f5a78700 1 Processor -- start 2024-04-25T00:06:21.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.265+0000 7fe1f5a78700 1 -- start start 2024-04-25T00:06:21.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.266+0000 7fe1f5a78700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1f01086e0 0x7fe1f019de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:21.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.266+0000 7fe1f5a78700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f010c040 con 0x7fe1f01086e0 2024-04-25T00:06:21.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.266+0000 7fe1ef7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1f01086e0 0x7fe1f019de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:21.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.266+0000 7fe1ef7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1f01086e0 0x7fe1f019de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46186/0 (socket says 172.21.15.107:46186) 2024-04-25T00:06:21.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.266+0000 7fe1ef7fe700 1 -- 172.21.15.107:0/1908649717 learned_addr learned my addr 172.21.15.107:0/1908649717 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:21.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.266+0000 7fe1ef7fe700 1 -- 172.21.15.107:0/1908649717 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f019e3d0 con 0x7fe1f01086e0 2024-04-25T00:06:21.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.266+0000 7fe1ef7fe700 1 --2- 172.21.15.107:0/1908649717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1f01086e0 0x7fe1f019de90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0011b60 tx=0x7fe1e0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:21.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.266+0000 7fe1ecff9700 1 -- 172.21.15.107:0/1908649717 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1e0003970 con 0x7fe1f01086e0 2024-04-25T00:06:21.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.266+0000 7fe1ecff9700 1 -- 172.21.15.107:0/1908649717 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe1e0003b10 con 0x7fe1f01086e0 2024-04-25T00:06:21.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.266+0000 7fe1f5a78700 1 -- 172.21.15.107:0/1908649717 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f019e660 con 0x7fe1f01086e0 2024-04-25T00:06:21.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.266+0000 7fe1ecff9700 1 -- 172.21.15.107:0/1908649717 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1e0024610 con 0x7fe1f01086e0 2024-04-25T00:06:21.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.266+0000 7fe1f5a78700 1 -- 172.21.15.107:0/1908649717 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f01a1350 con 0x7fe1f01086e0 2024-04-25T00:06:21.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.267+0000 7fe1ecff9700 1 -- 172.21.15.107:0/1908649717 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe1e0024ca0 con 0x7fe1f01086e0 2024-04-25T00:06:21.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.267+0000 7fe1ecff9700 1 -- 172.21.15.107:0/1908649717 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe1e0011e00 con 0x7fe1f01086e0 2024-04-25T00:06:21.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.267+0000 7fe1f5a78700 1 -- 172.21.15.107:0/1908649717 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1f019eb40 con 0x7fe1f01086e0 2024-04-25T00:06:21.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.269+0000 7fe1ecff9700 1 -- 172.21.15.107:0/1908649717 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe1f019eb40 con 0x7fe1f01086e0 2024-04-25T00:06:21.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.341+0000 7fe1f5a78700 1 -- 172.21.15.107:0/1908649717 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe1f01a1520 con 0x7fe1f01086e0 2024-04-25T00:06:21.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.342+0000 7fe1ecff9700 1 -- 172.21.15.107:0/1908649717 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fe1e001a9f0 con 0x7fe1f01086e0 2024-04-25T00:06:21.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.343+0000 7fe1f5a78700 1 -- 172.21.15.107:0/1908649717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1f01086e0 msgr2=0x7fe1f019de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:21.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.343+0000 7fe1f5a78700 1 --2- 172.21.15.107:0/1908649717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1f01086e0 0x7fe1f019de90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0011b60 tx=0x7fe1e0011c40 comp rx=0 tx=0).stop 2024-04-25T00:06:21.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.343+0000 7fe1f5a78700 1 -- 172.21.15.107:0/1908649717 shutdown_connections 2024-04-25T00:06:21.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.343+0000 7fe1f5a78700 1 --2- 172.21.15.107:0/1908649717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1f01086e0 0x7fe1f019de90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:21.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.343+0000 7fe1f5a78700 1 -- 172.21.15.107:0/1908649717 >> 172.21.15.107:0/1908649717 conn(0x7fe1f0102130 msgr2=0x7fe1f0102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:21.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.343+0000 7fe1f5a78700 1 -- 172.21.15.107:0/1908649717 shutdown_connections 2024-04-25T00:06:21.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:21.343+0000 7fe1f5a78700 1 -- 172.21.15.107:0/1908649717 wait complete. 2024-04-25T00:06:21.980 INFO:teuthology.orchestra.run.smithi107.stdout:mgr not available, waiting (3/15)... 2024-04-25T00:06:22.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:22 smithi107 ceph-mon[29153]: from='mgr.14100 172.21.15.107:0/3901897199' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T00:06:22.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:22 smithi107 ceph-mon[29153]: from='mgr.14100 172.21.15.107:0/3901897199' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T00:06:22.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:22 smithi107 ceph-mon[29153]: from='mgr.14100 172.21.15.107:0/3901897199' entity='mgr.x' 2024-04-25T00:06:22.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:22 smithi107 ceph-mon[29153]: from='mgr.14100 172.21.15.107:0/3901897199' entity='mgr.x' 2024-04-25T00:06:22.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:22 smithi107 ceph-mon[29153]: from='mgr.14100 172.21.15.107:0/3901897199' entity='mgr.x' 2024-04-25T00:06:22.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:22 smithi107 ceph-mon[29153]: mgrmap e3: x(active, since 1.00746s) 2024-04-25T00:06:24.670 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:24 smithi107 ceph-mon[29153]: mgrmap e4: x(active, since 2s) 2024-04-25T00:06:25.293 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-25T00:06:25.293 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:06:25.293 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "79602f58-0297-11ef-bc93-c7b262605968", 2024-04-25T00:06:25.293 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T00:06:25.293 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T00:06:25.293 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T00:06:25.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T00:06:25.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:25.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T00:06:25.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T00:06:25.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-04-25T00:06:25.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:06:25.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T00:06:25.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T00:06:25.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:06:25.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-04-25T00:06:25.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T00:06:25.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:06:25.294 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T00:06:25.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T00:06:25.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:25.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T00:06:25.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:06:25.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T00:06:25.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T00:06:25.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T00:06:25.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T00:06:25.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T00:06:25.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T00:06:25.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:25.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T00:06:25.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T00:06:25.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T00:06:25.295 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T00:06:25.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T00:06:25.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T00:06:25.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T00:06:25.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T00:06:25.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T00:06:25.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:25.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T00:06:25.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:06:25.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T00:06:25.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T00:06:25.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:25.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T00:06:25.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T00:06:25.296 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T00:06:25.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T00:06:25.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T00:06:25.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T00:06:25.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T00:06:25.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:06:25.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:06:25.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:25.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T00:06:25.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:06:25.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T00:05:56.053812+0000", 2024-04-25T00:06:25.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:06:25.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:06:25.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T00:06:25.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:06:25.297 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.382+0000 7f808cfe8700 1 Processor -- start 2024-04-25T00:06:25.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.382+0000 7f808cfe8700 1 -- start start 2024-04-25T00:06:25.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.383+0000 7f808cfe8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8088108120 0x7f8088108540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.383+0000 7f808cfe8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8088108b10 con 0x7f8088108120 2024-04-25T00:06:25.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.383+0000 7f808659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8088108120 0x7f8088108540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.383+0000 7f808659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8088108120 0x7f8088108540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46276/0 (socket says 172.21.15.107:46276) 2024-04-25T00:06:25.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.383+0000 7f808659c700 1 -- 172.21.15.107:0/961166783 learned_addr learned my addr 172.21.15.107:0/961166783 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:25.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.383+0000 7f808659c700 1 -- 172.21.15.107:0/961166783 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8088106d30 con 0x7f8088108120 2024-04-25T00:06:25.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.383+0000 7f808659c700 1 --2- 172.21.15.107:0/961166783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8088108120 0x7f8088108540 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f80780098a0 tx=0x7f8078009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d808cd1f7385d7b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:25.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.384+0000 7f808559a700 1 -- 172.21.15.107:0/961166783 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8078004160 con 0x7f8088108120 2024-04-25T00:06:25.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.384+0000 7f808559a700 1 -- 172.21.15.107:0/961166783 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8078004300 con 0x7f8088108120 2024-04-25T00:06:25.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.384+0000 7f808559a700 1 -- 172.21.15.107:0/961166783 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8078011b50 con 0x7f8088108120 2024-04-25T00:06:25.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.384+0000 7f808cfe8700 1 -- 172.21.15.107:0/961166783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8088108120 msgr2=0x7f8088108540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:25.298 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.384+0000 7f808cfe8700 1 --2- 172.21.15.107:0/961166783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8088108120 0x7f8088108540 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f80780098a0 tx=0x7f8078009bb0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.384+0000 7f808cfe8700 1 -- 172.21.15.107:0/961166783 shutdown_connections 2024-04-25T00:06:25.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.384+0000 7f808cfe8700 1 --2- 172.21.15.107:0/961166783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8088108120 0x7f8088108540 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.384+0000 7f808cfe8700 1 -- 172.21.15.107:0/961166783 >> 172.21.15.107:0/961166783 conn(0x7f80881018a0 msgr2=0x7f8088103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:25.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.384+0000 7f808cfe8700 1 -- 172.21.15.107:0/961166783 shutdown_connections 2024-04-25T00:06:25.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.384+0000 7f808cfe8700 1 -- 172.21.15.107:0/961166783 wait complete. 2024-04-25T00:06:25.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.385+0000 7f808cfe8700 1 Processor -- start 2024-04-25T00:06:25.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.385+0000 7f808cfe8700 1 -- start start 2024-04-25T00:06:25.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.385+0000 7f808cfe8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8088108120 0x7f808819bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.385+0000 7f808cfe8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f808810b0f0 con 0x7f8088108120 2024-04-25T00:06:25.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.385+0000 7f808659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8088108120 0x7f808819bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.385+0000 7f808659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8088108120 0x7f808819bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46290/0 (socket says 172.21.15.107:46290) 2024-04-25T00:06:25.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.385+0000 7f808659c700 1 -- 172.21.15.107:0/420531771 learned_addr learned my addr 172.21.15.107:0/420531771 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:25.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.386+0000 7f808659c700 1 -- 172.21.15.107:0/420531771 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808819c200 con 0x7f8088108120 2024-04-25T00:06:25.299 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.386+0000 7f808659c700 1 --2- 172.21.15.107:0/420531771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8088108120 0x7f808819bcc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8078011850 tx=0x7f8078011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:25.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.386+0000 7f80777fe700 1 -- 172.21.15.107:0/420531771 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8078003af0 con 0x7f8088108120 2024-04-25T00:06:25.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.386+0000 7f80777fe700 1 -- 172.21.15.107:0/420531771 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8078020070 con 0x7f8088108120 2024-04-25T00:06:25.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.386+0000 7f808cfe8700 1 -- 172.21.15.107:0/420531771 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808819c490 con 0x7f8088108120 2024-04-25T00:06:25.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.386+0000 7f80777fe700 1 -- 172.21.15.107:0/420531771 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80780195a0 con 0x7f8088108120 2024-04-25T00:06:25.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.386+0000 7f808cfe8700 1 -- 172.21.15.107:0/420531771 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f808819f180 con 0x7f8088108120 2024-04-25T00:06:25.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.387+0000 7f80777fe700 1 -- 172.21.15.107:0/420531771 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f80780198a0 con 0x7f8088108120 2024-04-25T00:06:25.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.387+0000 7f808cfe8700 1 -- 172.21.15.107:0/420531771 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8088063890 con 0x7f8088108120 2024-04-25T00:06:25.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.387+0000 7f80777fe700 1 --2- 172.21.15.107:0/420531771 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f8070039000 0x7f807003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:25.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.387+0000 7f80777fe700 1 -- 172.21.15.107:0/420531771 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8078011e00 con 0x7f8088108120 2024-04-25T00:06:25.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.387+0000 7f8085d9b700 1 --2- 172.21.15.107:0/420531771 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f8070039000 0x7f807003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:25.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.387+0000 7f8085d9b700 1 --2- 172.21.15.107:0/420531771 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f8070039000 0x7f807003b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f807c006fd0 tx=0x7f807c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:25.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.392+0000 7f80777fe700 1 -- 172.21.15.107:0/420531771 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8078003c90 con 0x7f8088108120 2024-04-25T00:06:25.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.669+0000 7f808cfe8700 1 -- 172.21.15.107:0/420531771 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f80880008d0 con 0x7f8088108120 2024-04-25T00:06:25.300 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.669+0000 7f80777fe700 1 -- 172.21.15.107:0/420531771 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f8078023530 con 0x7f8088108120 2024-04-25T00:06:25.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.671+0000 7f808cfe8700 1 -- 172.21.15.107:0/420531771 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f8070039000 msgr2=0x7f807003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:25.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.671+0000 7f808cfe8700 1 --2- 172.21.15.107:0/420531771 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f8070039000 0x7f807003b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f807c006fd0 tx=0x7f807c006e40 comp rx=0 tx=0).stop 2024-04-25T00:06:25.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.671+0000 7f808cfe8700 1 -- 172.21.15.107:0/420531771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8088108120 msgr2=0x7f808819bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:25.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.671+0000 7f808cfe8700 1 --2- 172.21.15.107:0/420531771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8088108120 0x7f808819bcc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8078011850 tx=0x7f8078011c10 comp rx=0 tx=0).stop 2024-04-25T00:06:25.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.671+0000 7f808cfe8700 1 -- 172.21.15.107:0/420531771 shutdown_connections 2024-04-25T00:06:25.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.671+0000 7f808cfe8700 1 --2- 172.21.15.107:0/420531771 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f8070039000 0x7f807003b4c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.671+0000 7f808cfe8700 1 --2- 172.21.15.107:0/420531771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8088108120 0x7f808819bcc0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:25.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.671+0000 7f808cfe8700 1 -- 172.21.15.107:0/420531771 >> 172.21.15.107:0/420531771 conn(0x7f80881018a0 msgr2=0x7f80881027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:25.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.671+0000 7f808cfe8700 1 -- 172.21.15.107:0/420531771 shutdown_connections 2024-04-25T00:06:25.301 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:24.671+0000 7f808cfe8700 1 -- 172.21.15.107:0/420531771 wait complete. 2024-04-25T00:06:25.301 INFO:teuthology.orchestra.run.smithi107.stdout:mgr is available 2024-04-25T00:06:25.542 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:25 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/420531771' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T00:06:26.535 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-25T00:06:26.536 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T00:06:26.536 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout fsid = 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:06:26.536 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T00:06:26.536 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T00:06:26.536 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T00:06:26.536 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T00:06:26.536 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-25T00:06:26.536 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T00:06:26.536 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T00:06:26.536 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-04-25T00:06:26.536 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T00:06:26.536 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T00:06:26.536 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T00:06:26.536 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T00:06:26.536 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T00:06:26.537 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T00:06:26.537 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.762+0000 7f0b5bba9700 1 Processor -- start 2024-04-25T00:06:26.537 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.762+0000 7f0b5bba9700 1 -- start start 2024-04-25T00:06:26.537 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.762+0000 7f0b5bba9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b54105f00 0x7f0b54106320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:26.537 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.762+0000 7f0b5bba9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0b541068f0 con 0x7f0b54105f00 2024-04-25T00:06:26.537 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.763+0000 7f0b59945700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b54105f00 0x7f0b54106320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:26.537 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.763+0000 7f0b59945700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b54105f00 0x7f0b54106320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46294/0 (socket says 172.21.15.107:46294) 2024-04-25T00:06:26.537 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.763+0000 7f0b59945700 1 -- 172.21.15.107:0/2188866481 learned_addr learned my addr 172.21.15.107:0/2188866481 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:26.537 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.763+0000 7f0b59945700 1 -- 172.21.15.107:0/2188866481 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b54108e90 con 0x7f0b54105f00 2024-04-25T00:06:26.537 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.763+0000 7f0b59945700 1 --2- 172.21.15.107:0/2188866481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b54105f00 0x7f0b54106320 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009e80 tx=0x7f0b5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653e728f38adfa6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:26.537 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.763+0000 7f0b58943700 1 -- 172.21.15.107:0/2188866481 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b50017070 con 0x7f0b54105f00 2024-04-25T00:06:26.537 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.763+0000 7f0b58943700 1 -- 172.21.15.107:0/2188866481 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0b5000b8c0 con 0x7f0b54105f00 2024-04-25T00:06:26.537 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.763+0000 7f0b58943700 1 -- 172.21.15.107:0/2188866481 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b500045c0 con 0x7f0b54105f00 2024-04-25T00:06:26.538 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.764+0000 7f0b5bba9700 1 -- 172.21.15.107:0/2188866481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b54105f00 msgr2=0x7f0b54106320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:26.538 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.764+0000 7f0b5bba9700 1 --2- 172.21.15.107:0/2188866481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b54105f00 0x7f0b54106320 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0b50009e80 tx=0x7f0b5000b330 comp rx=0 tx=0).stop 2024-04-25T00:06:26.538 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.764+0000 7f0b5bba9700 1 -- 172.21.15.107:0/2188866481 shutdown_connections 2024-04-25T00:06:26.538 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.764+0000 7f0b5bba9700 1 --2- 172.21.15.107:0/2188866481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b54105f00 0x7f0b54106320 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:26.538 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.764+0000 7f0b5bba9700 1 -- 172.21.15.107:0/2188866481 >> 172.21.15.107:0/2188866481 conn(0x7f0b54101810 msgr2=0x7f0b54103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:26.538 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.764+0000 7f0b5bba9700 1 -- 172.21.15.107:0/2188866481 shutdown_connections 2024-04-25T00:06:26.538 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.764+0000 7f0b5bba9700 1 -- 172.21.15.107:0/2188866481 wait complete. 2024-04-25T00:06:26.538 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.764+0000 7f0b5bba9700 1 Processor -- start 2024-04-25T00:06:26.538 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.764+0000 7f0b5bba9700 1 -- start start 2024-04-25T00:06:26.538 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.765+0000 7f0b5bba9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b54105f00 0x7f0b5419bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:26.538 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.765+0000 7f0b5bba9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5410c240 con 0x7f0b54105f00 2024-04-25T00:06:26.538 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.765+0000 7f0b59945700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b54105f00 0x7f0b5419bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:26.538 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.765+0000 7f0b59945700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b54105f00 0x7f0b5419bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46308/0 (socket says 172.21.15.107:46308) 2024-04-25T00:06:26.538 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.765+0000 7f0b59945700 1 -- 172.21.15.107:0/2232178409 learned_addr learned my addr 172.21.15.107:0/2232178409 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:26.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.765+0000 7f0b59945700 1 -- 172.21.15.107:0/2232178409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b5419c100 con 0x7f0b54105f00 2024-04-25T00:06:26.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.765+0000 7f0b59945700 1 --2- 172.21.15.107:0/2232178409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b54105f00 0x7f0b5419bbc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0b500072a0 tx=0x7f0b5000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:26.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.765+0000 7f0b4affd700 1 -- 172.21.15.107:0/2232178409 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b50020020 con 0x7f0b54105f00 2024-04-25T00:06:26.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.765+0000 7f0b4affd700 1 -- 172.21.15.107:0/2232178409 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0b50003b50 con 0x7f0b54105f00 2024-04-25T00:06:26.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.765+0000 7f0b5bba9700 1 -- 172.21.15.107:0/2232178409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b5419c390 con 0x7f0b54105f00 2024-04-25T00:06:26.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.765+0000 7f0b4affd700 1 -- 172.21.15.107:0/2232178409 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b50017040 con 0x7f0b54105f00 2024-04-25T00:06:26.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.765+0000 7f0b5bba9700 1 -- 172.21.15.107:0/2232178409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b5419f080 con 0x7f0b54105f00 2024-04-25T00:06:26.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.766+0000 7f0b4affd700 1 -- 172.21.15.107:0/2232178409 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f0b50023020 con 0x7f0b54105f00 2024-04-25T00:06:26.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.766+0000 7f0b4affd700 1 --2- 172.21.15.107:0/2232178409 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f0b40038fb0 0x7f0b4003b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:26.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.766+0000 7f0b5bba9700 1 -- 172.21.15.107:0/2232178409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b54063890 con 0x7f0b54105f00 2024-04-25T00:06:26.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.766+0000 7f0b4affd700 1 -- 172.21.15.107:0/2232178409 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0b5000f050 con 0x7f0b54105f00 2024-04-25T00:06:26.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.767+0000 7f0b59144700 1 --2- 172.21.15.107:0/2232178409 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f0b40038fb0 0x7f0b4003b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:26.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.767+0000 7f0b59144700 1 --2- 172.21.15.107:0/2232178409 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f0b40038fb0 0x7f0b4003b470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0b44006fd0 tx=0x7f0b44006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:26.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.771+0000 7f0b4affd700 1 -- 172.21.15.107:0/2232178409 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0b5002a020 con 0x7f0b54105f00 2024-04-25T00:06:26.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.963+0000 7f0b5bba9700 1 -- 172.21.15.107:0/2232178409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f0b541a08c0 con 0x7f0b54105f00 2024-04-25T00:06:26.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.964+0000 7f0b4affd700 1 -- 172.21.15.107:0/2232178409 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f0b5000ade0 con 0x7f0b54105f00 2024-04-25T00:06:26.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.967+0000 7f0b5bba9700 1 -- 172.21.15.107:0/2232178409 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f0b40038fb0 msgr2=0x7f0b4003b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:26.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.967+0000 7f0b5bba9700 1 --2- 172.21.15.107:0/2232178409 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f0b40038fb0 0x7f0b4003b470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0b44006fd0 tx=0x7f0b44006e40 comp rx=0 tx=0).stop 2024-04-25T00:06:26.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.967+0000 7f0b5bba9700 1 -- 172.21.15.107:0/2232178409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b54105f00 msgr2=0x7f0b5419bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:26.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.967+0000 7f0b5bba9700 1 --2- 172.21.15.107:0/2232178409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b54105f00 0x7f0b5419bbc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0b500072a0 tx=0x7f0b5000bd50 comp rx=0 tx=0).stop 2024-04-25T00:06:26.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.967+0000 7f0b5bba9700 1 -- 172.21.15.107:0/2232178409 shutdown_connections 2024-04-25T00:06:26.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.967+0000 7f0b5bba9700 1 --2- 172.21.15.107:0/2232178409 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f0b40038fb0 0x7f0b4003b470 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:26.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.967+0000 7f0b5bba9700 1 --2- 172.21.15.107:0/2232178409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b54105f00 0x7f0b5419bbc0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:26.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.967+0000 7f0b5bba9700 1 -- 172.21.15.107:0/2232178409 >> 172.21.15.107:0/2232178409 conn(0x7f0b54101810 msgr2=0x7f0b54102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:26.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.967+0000 7f0b5bba9700 1 -- 172.21.15.107:0/2232178409 shutdown_connections 2024-04-25T00:06:26.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:25.967+0000 7f0b5bba9700 1 -- 172.21.15.107:0/2232178409 wait complete. 2024-04-25T00:06:26.540 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling cephadm module... 2024-04-25T00:06:26.654 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:26 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2232178409' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T00:06:27.564 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:27 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2221471518' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T00:06:27.564 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:27 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: ignoring --setuser ceph since I am not root 2024-04-25T00:06:27.564 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:27 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: ignoring --setgroup ceph since I am not root 2024-04-25T00:06:27.564 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:27 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:27.562+0000 7f8e35e9d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:06:27.849 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:27 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:27.746+0000 7f8e35e9d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:06:27.850 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:27 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:27.848+0000 7f8e35e9d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:06:27.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.990+0000 7f85169c6700 1 Processor -- start 2024-04-25T00:06:27.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.990+0000 7f85169c6700 1 -- start start 2024-04-25T00:06:27.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.990+0000 7f85169c6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8510106b20 0x7f8510106f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:27.965 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.990+0000 7f85169c6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8510107510 con 0x7f8510106b20 2024-04-25T00:06:27.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.991+0000 7f850ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8510106b20 0x7f8510106f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:27.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.991+0000 7f850ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8510106b20 0x7f8510106f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46314/0 (socket says 172.21.15.107:46314) 2024-04-25T00:06:27.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.991+0000 7f850ffff700 1 -- 172.21.15.107:0/511867469 learned_addr learned my addr 172.21.15.107:0/511867469 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:27.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.991+0000 7f850ffff700 1 -- 172.21.15.107:0/511867469 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8510107d20 con 0x7f8510106b20 2024-04-25T00:06:27.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.991+0000 7f850ffff700 1 --2- 172.21.15.107:0/511867469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8510106b20 0x7f8510106f40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f85000098a0 tx=0x7f8500009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa93e9dcff95ddc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:27.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.991+0000 7f850effd700 1 -- 172.21.15.107:0/511867469 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8500004160 con 0x7f8510106b20 2024-04-25T00:06:27.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.991+0000 7f850effd700 1 -- 172.21.15.107:0/511867469 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8500004300 con 0x7f8510106b20 2024-04-25T00:06:27.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.991+0000 7f850effd700 1 -- 172.21.15.107:0/511867469 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8500011b50 con 0x7f8510106b20 2024-04-25T00:06:27.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.992+0000 7f85169c6700 1 -- 172.21.15.107:0/511867469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8510106b20 msgr2=0x7f8510106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:27.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.992+0000 7f85169c6700 1 --2- 172.21.15.107:0/511867469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8510106b20 0x7f8510106f40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f85000098a0 tx=0x7f8500009bb0 comp rx=0 tx=0).stop 2024-04-25T00:06:27.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.992+0000 7f85169c6700 1 -- 172.21.15.107:0/511867469 shutdown_connections 2024-04-25T00:06:27.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.992+0000 7f85169c6700 1 --2- 172.21.15.107:0/511867469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8510106b20 0x7f8510106f40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:27.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.992+0000 7f85169c6700 1 -- 172.21.15.107:0/511867469 >> 172.21.15.107:0/511867469 conn(0x7f85101020a0 msgr2=0x7f8510104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:27.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.992+0000 7f85169c6700 1 -- 172.21.15.107:0/511867469 shutdown_connections 2024-04-25T00:06:27.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.992+0000 7f85169c6700 1 -- 172.21.15.107:0/511867469 wait complete. 2024-04-25T00:06:27.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.992+0000 7f85169c6700 1 Processor -- start 2024-04-25T00:06:27.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.992+0000 7f85169c6700 1 -- start start 2024-04-25T00:06:27.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.993+0000 7f85169c6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8510106b20 0x7f851007bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:27.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.993+0000 7f85169c6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f851010b0d0 con 0x7f8510106b20 2024-04-25T00:06:27.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.993+0000 7f850ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8510106b20 0x7f851007bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:27.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.993+0000 7f850ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8510106b20 0x7f851007bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46316/0 (socket says 172.21.15.107:46316) 2024-04-25T00:06:27.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.993+0000 7f850ffff700 1 -- 172.21.15.107:0/2221471518 learned_addr learned my addr 172.21.15.107:0/2221471518 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:27.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.993+0000 7f850ffff700 1 -- 172.21.15.107:0/2221471518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851019e920 con 0x7f8510106b20 2024-04-25T00:06:27.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.993+0000 7f850ffff700 1 --2- 172.21.15.107:0/2221471518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8510106b20 0x7f851007bab0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8500011670 tx=0x7f8500011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:27.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.993+0000 7f850d7fa700 1 -- 172.21.15.107:0/2221471518 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85000039c0 con 0x7f8510106b20 2024-04-25T00:06:27.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.993+0000 7f850d7fa700 1 -- 172.21.15.107:0/2221471518 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8500011e50 con 0x7f8510106b20 2024-04-25T00:06:27.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.994+0000 7f85169c6700 1 -- 172.21.15.107:0/2221471518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f851019eaf0 con 0x7f8510106b20 2024-04-25T00:06:27.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.994+0000 7f850d7fa700 1 -- 172.21.15.107:0/2221471518 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f850001a5c0 con 0x7f8510106b20 2024-04-25T00:06:27.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.994+0000 7f85169c6700 1 -- 172.21.15.107:0/2221471518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f851019ef10 con 0x7f8510106b20 2024-04-25T00:06:27.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.994+0000 7f850d7fa700 1 -- 172.21.15.107:0/2221471518 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f850001a8c0 con 0x7f8510106b20 2024-04-25T00:06:27.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.994+0000 7f850d7fa700 1 --2- 172.21.15.107:0/2221471518 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f84f8038fb0 0x7f84f803b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:27.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.994+0000 7f85169c6700 1 -- 172.21.15.107:0/2221471518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8510063890 con 0x7f8510106b20 2024-04-25T00:06:27.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.995+0000 7f850d7fa700 1 -- 172.21.15.107:0/2221471518 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f850004d5d0 con 0x7f8510106b20 2024-04-25T00:06:27.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.995+0000 7f850f7fe700 1 --2- 172.21.15.107:0/2221471518 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f84f8038fb0 0x7f84f803b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:27.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:26.995+0000 7f850f7fe700 1 --2- 172.21.15.107:0/2221471518 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f84f8038fb0 0x7f84f803b470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8504006fd0 tx=0x7f8504006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:27.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:27.000+0000 7f850d7fa700 1 -- 172.21.15.107:0/2221471518 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8500019c70 con 0x7f8510106b20 2024-04-25T00:06:27.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:27.248+0000 7f85169c6700 1 -- 172.21.15.107:0/2221471518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f851019f750 con 0x7f8510106b20 2024-04-25T00:06:27.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:27.399+0000 7f850d7fa700 1 -- 172.21.15.107:0/2221471518 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f850002cc00 con 0x7f8510106b20 2024-04-25T00:06:27.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:27.401+0000 7f850d7fa700 1 -- 172.21.15.107:0/2221471518 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f850004c090 con 0x7f8510106b20 2024-04-25T00:06:27.969 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:27.402+0000 7f85169c6700 1 -- 172.21.15.107:0/2221471518 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f84f8038fb0 msgr2=0x7f84f803b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:27.969 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:27.402+0000 7f85169c6700 1 --2- 172.21.15.107:0/2221471518 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f84f8038fb0 0x7f84f803b470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8504006fd0 tx=0x7f8504006e40 comp rx=0 tx=0).stop 2024-04-25T00:06:27.969 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:27.402+0000 7f85169c6700 1 -- 172.21.15.107:0/2221471518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8510106b20 msgr2=0x7f851007bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:27.969 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:27.402+0000 7f85169c6700 1 --2- 172.21.15.107:0/2221471518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8510106b20 0x7f851007bab0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8500011670 tx=0x7f8500011b60 comp rx=0 tx=0).stop 2024-04-25T00:06:27.969 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:27.402+0000 7f85169c6700 1 -- 172.21.15.107:0/2221471518 shutdown_connections 2024-04-25T00:06:27.969 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:27.402+0000 7f85169c6700 1 --2- 172.21.15.107:0/2221471518 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f84f8038fb0 0x7f84f803b470 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:27.969 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:27.402+0000 7f85169c6700 1 --2- 172.21.15.107:0/2221471518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8510106b20 0x7f851007bab0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:27.969 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:27.402+0000 7f85169c6700 1 -- 172.21.15.107:0/2221471518 >> 172.21.15.107:0/2221471518 conn(0x7f85101020a0 msgr2=0x7f8510102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:27.969 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:27.403+0000 7f85169c6700 1 -- 172.21.15.107:0/2221471518 shutdown_connections 2024-04-25T00:06:27.969 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:27.403+0000 7f85169c6700 1 -- 172.21.15.107:0/2221471518 wait complete. 2024-04-25T00:06:28.402 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:28 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:28.161+0000 7f8e35e9d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:06:28.402 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:28 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:28.247+0000 7f8e35e9d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:06:28.668 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:28 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2221471518' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T00:06:28.669 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:28 smithi107 ceph-mon[29153]: mgrmap e5: x(active, since 6s) 2024-04-25T00:06:29.188 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:06:29.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T00:06:29.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T00:06:29.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T00:06:29.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T00:06:29.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:06:29.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.374+0000 7f3df75f9700 1 Processor -- start 2024-04-25T00:06:29.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.374+0000 7f3df75f9700 1 -- start start 2024-04-25T00:06:29.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.374+0000 7f3df75f9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3df01086c0 0x7f3df010aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:29.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.374+0000 7f3df75f9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3df007beb0 con 0x7f3df01086c0 2024-04-25T00:06:29.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.375+0000 7f3df5395700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3df01086c0 0x7f3df010aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:29.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.375+0000 7f3df5395700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3df01086c0 0x7f3df010aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46358/0 (socket says 172.21.15.107:46358) 2024-04-25T00:06:29.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.375+0000 7f3df5395700 1 -- 172.21.15.107:0/1179343715 learned_addr learned my addr 172.21.15.107:0/1179343715 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:29.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.375+0000 7f3df5395700 1 -- 172.21.15.107:0/1179343715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df010aff0 con 0x7f3df01086c0 2024-04-25T00:06:29.189 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.375+0000 7f3df5395700 1 --2- 172.21.15.107:0/1179343715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3df01086c0 0x7f3df010aab0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3dec0098a0 tx=0x7f3dec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da77b59c68a1cae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:29.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.375+0000 7f3de7fff700 1 -- 172.21.15.107:0/1179343715 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3dec004160 con 0x7f3df01086c0 2024-04-25T00:06:29.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.375+0000 7f3de7fff700 1 -- 172.21.15.107:0/1179343715 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3dec004300 con 0x7f3df01086c0 2024-04-25T00:06:29.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.375+0000 7f3de7fff700 1 -- 172.21.15.107:0/1179343715 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3dec011b50 con 0x7f3df01086c0 2024-04-25T00:06:29.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.376+0000 7f3df75f9700 1 -- 172.21.15.107:0/1179343715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3df01086c0 msgr2=0x7f3df010aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:29.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.376+0000 7f3df75f9700 1 --2- 172.21.15.107:0/1179343715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3df01086c0 0x7f3df010aab0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3dec0098a0 tx=0x7f3dec009bb0 comp rx=0 tx=0).stop 2024-04-25T00:06:29.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.376+0000 7f3df75f9700 1 -- 172.21.15.107:0/1179343715 shutdown_connections 2024-04-25T00:06:29.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.376+0000 7f3df75f9700 1 --2- 172.21.15.107:0/1179343715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3df01086c0 0x7f3df010aab0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:29.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.376+0000 7f3df75f9700 1 -- 172.21.15.107:0/1179343715 >> 172.21.15.107:0/1179343715 conn(0x7f3df01020b0 msgr2=0x7f3df0104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:29.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.376+0000 7f3df75f9700 1 -- 172.21.15.107:0/1179343715 shutdown_connections 2024-04-25T00:06:29.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.376+0000 7f3df75f9700 1 -- 172.21.15.107:0/1179343715 wait complete. 2024-04-25T00:06:29.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.377+0000 7f3df75f9700 1 Processor -- start 2024-04-25T00:06:29.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.377+0000 7f3df75f9700 1 -- start start 2024-04-25T00:06:29.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.377+0000 7f3df75f9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3df01086c0 0x7f3df0107f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:29.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.377+0000 7f3df75f9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3df010c020 con 0x7f3df01086c0 2024-04-25T00:06:29.190 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.377+0000 7f3df5395700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3df01086c0 0x7f3df0107f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:29.191 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.377+0000 7f3df5395700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3df01086c0 0x7f3df0107f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46374/0 (socket says 172.21.15.107:46374) 2024-04-25T00:06:29.191 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.377+0000 7f3df5395700 1 -- 172.21.15.107:0/2518542905 learned_addr learned my addr 172.21.15.107:0/2518542905 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:29.191 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.377+0000 7f3df5395700 1 -- 172.21.15.107:0/2518542905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df0108480 con 0x7f3df01086c0 2024-04-25T00:06:29.191 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.377+0000 7f3df5395700 1 --2- 172.21.15.107:0/2518542905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3df01086c0 0x7f3df0107f40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3dec011670 tx=0x7f3dec011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:29.191 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.378+0000 7f3de67fc700 1 -- 172.21.15.107:0/2518542905 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3dec0039c0 con 0x7f3df01086c0 2024-04-25T00:06:29.191 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.378+0000 7f3de67fc700 1 -- 172.21.15.107:0/2518542905 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f3dec011e50 con 0x7f3df01086c0 2024-04-25T00:06:29.191 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.378+0000 7f3df75f9700 1 -- 172.21.15.107:0/2518542905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df0106670 con 0x7f3df01086c0 2024-04-25T00:06:29.191 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.378+0000 7f3de67fc700 1 -- 172.21.15.107:0/2518542905 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3dec01a5c0 con 0x7f3df01086c0 2024-04-25T00:06:29.191 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.378+0000 7f3df75f9700 1 -- 172.21.15.107:0/2518542905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df0106bb0 con 0x7f3df01086c0 2024-04-25T00:06:29.191 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.378+0000 7f3de67fc700 1 -- 172.21.15.107:0/2518542905 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f3dec01a8c0 con 0x7f3df01086c0 2024-04-25T00:06:29.191 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.379+0000 7f3de67fc700 1 --2- 172.21.15.107:0/2518542905 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f3ddc039000 0x7f3ddc03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:29.191 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.379+0000 7f3de67fc700 1 -- 172.21.15.107:0/2518542905 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3dec04e830 con 0x7f3df01086c0 2024-04-25T00:06:29.191 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.379+0000 7f3df4b94700 1 -- 172.21.15.107:0/2518542905 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f3ddc039000 msgr2=0x7f3ddc03b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/120616774 2024-04-25T00:06:29.191 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.379+0000 7f3df4b94700 1 --2- 172.21.15.107:0/2518542905 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f3ddc039000 0x7f3ddc03b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:06:29.191 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.379+0000 7f3df75f9700 1 -- 172.21.15.107:0/2518542905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3df0063890 con 0x7f3df01086c0 2024-04-25T00:06:29.192 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.384+0000 7f3de67fc700 1 -- 172.21.15.107:0/2518542905 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3dec00b3e0 con 0x7f3df01086c0 2024-04-25T00:06:29.192 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.580+0000 7f3df4b94700 1 -- 172.21.15.107:0/2518542905 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f3ddc039000 msgr2=0x7f3ddc03b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/120616774 2024-04-25T00:06:29.192 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.580+0000 7f3df4b94700 1 --2- 172.21.15.107:0/2518542905 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f3ddc039000 0x7f3ddc03b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T00:06:29.192 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.667+0000 7f3df75f9700 1 -- 172.21.15.107:0/2518542905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f3df0107250 con 0x7f3df01086c0 2024-04-25T00:06:29.192 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.668+0000 7f3de67fc700 1 -- 172.21.15.107:0/2518542905 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f3dec01ade0 con 0x7f3df01086c0 2024-04-25T00:06:29.192 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.671+0000 7f3df75f9700 1 -- 172.21.15.107:0/2518542905 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f3ddc039000 msgr2=0x7f3ddc03b4c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:06:29.192 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.671+0000 7f3df75f9700 1 --2- 172.21.15.107:0/2518542905 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f3ddc039000 0x7f3ddc03b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:29.192 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.671+0000 7f3df75f9700 1 -- 172.21.15.107:0/2518542905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3df01086c0 msgr2=0x7f3df0107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:29.192 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.671+0000 7f3df75f9700 1 --2- 172.21.15.107:0/2518542905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3df01086c0 0x7f3df0107f40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3dec011670 tx=0x7f3dec011b60 comp rx=0 tx=0).stop 2024-04-25T00:06:29.192 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.671+0000 7f3df75f9700 1 -- 172.21.15.107:0/2518542905 shutdown_connections 2024-04-25T00:06:29.192 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.671+0000 7f3df75f9700 1 --2- 172.21.15.107:0/2518542905 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f3ddc039000 0x7f3ddc03b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:29.192 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.671+0000 7f3df75f9700 1 --2- 172.21.15.107:0/2518542905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3df01086c0 0x7f3df0107f40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:29.192 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.671+0000 7f3df75f9700 1 -- 172.21.15.107:0/2518542905 >> 172.21.15.107:0/2518542905 conn(0x7f3df01020b0 msgr2=0x7f3df0102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:29.192 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.672+0000 7f3df75f9700 1 -- 172.21.15.107:0/2518542905 shutdown_connections 2024-04-25T00:06:29.192 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:28.672+0000 7f3df75f9700 1 -- 172.21.15.107:0/2518542905 wait complete. 2024-04-25T00:06:29.193 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for the mgr to restart... 2024-04-25T00:06:29.193 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr epoch 5... 2024-04-25T00:06:29.448 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:29 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2518542905' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T00:06:29.734 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:29 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:29.446+0000 7f8e35e9d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:06:29.734 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:29 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:29.621+0000 7f8e35e9d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:06:30.081 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:29 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:29.793+0000 7f8e35e9d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:06:30.081 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:30 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:30.080+0000 7f8e35e9d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:06:30.375 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:30 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:30.172+0000 7f8e35e9d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:06:30.375 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:30 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:30.373+0000 7f8e35e9d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:06:30.734 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:30 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:30.563+0000 7f8e35e9d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:06:31.484 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:31 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:31.148+0000 7f8e35e9d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:06:31.851 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:31 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:31.561+0000 7f8e35e9d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:06:31.851 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:31 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:31.656+0000 7f8e35e9d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:06:31.851 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:31 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:31.851+0000 7f8e35e9d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:06:32.642 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:32 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:32.361+0000 7f8e35e9d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:06:32.642 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:32 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:32.451+0000 7f8e35e9d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:06:32.912 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:32 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:32.688+0000 7f8e35e9d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:06:32.913 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:32 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:32.819+0000 7f8e35e9d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:06:32.913 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:32 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:32.912+0000 7f8e35e9d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:06:33.234 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:33 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:33.026+0000 7f8e35e9d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:06:33.234 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:33 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:33.116+0000 7f8e35e9d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:06:34.176 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:33 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:33.910+0000 7f8e35e9d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:06:34.176 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:34 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:34.122+0000 7f8e35e9d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:06:34.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:34 smithi107 ceph-mon[29153]: Active manager daemon x restarted 2024-04-25T00:06:34.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:34 smithi107 ceph-mon[29153]: Activating manager daemon x 2024-04-25T00:06:34.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:34 smithi107 ceph-mon[29153]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T00:06:34.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:34 smithi107 ceph-mon[29153]: mgrmap e6: x(active, starting, since 0.00346399s) 2024-04-25T00:06:34.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:34 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:06:34.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:34 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T00:06:34.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:34 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T00:06:34.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:34 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T00:06:34.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:34 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T00:06:34.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:34 smithi107 ceph-mon[29153]: Manager daemon x is now available 2024-04-25T00:06:34.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:34 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' 2024-04-25T00:06:34.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:34 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' 2024-04-25T00:06:34.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:34 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:06:34.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:34 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:06:34.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:34 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T00:06:34.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:34 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T00:06:35.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:35 smithi107 ceph-mon[29153]: Found migration_current of "None". Setting to last migration. 2024-04-25T00:06:35.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:35 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' 2024-04-25T00:06:35.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:35 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' 2024-04-25T00:06:35.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:35 smithi107 ceph-mon[29153]: mgrmap e7: x(active, since 1.00576s) 2024-04-25T00:06:35.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:06:35.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T00:06:35.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T00:06:35.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:06:35.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.631+0000 7f4682c86700 1 Processor -- start 2024-04-25T00:06:35.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.631+0000 7f4682c86700 1 -- start start 2024-04-25T00:06:35.741 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.631+0000 7f4682c86700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f467c108690 0x7f467c10aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:35.741 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.631+0000 7f4682c86700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f467c07beb0 con 0x7f467c108690 2024-04-25T00:06:35.741 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.632+0000 7f4680a22700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f467c108690 0x7f467c10aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:35.741 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.632+0000 7f4680a22700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f467c108690 0x7f467c10aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55266/0 (socket says 172.21.15.107:55266) 2024-04-25T00:06:35.741 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.632+0000 7f4680a22700 1 -- 172.21.15.107:0/2400235557 learned_addr learned my addr 172.21.15.107:0/2400235557 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:35.741 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.632+0000 7f4680a22700 1 -- 172.21.15.107:0/2400235557 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467c10afc0 con 0x7f467c108690 2024-04-25T00:06:35.741 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.632+0000 7f4680a22700 1 --2- 172.21.15.107:0/2400235557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f467c108690 0x7f467c10aa80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f46700098a0 tx=0x7f4670009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84e4827d5926cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:35.741 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.632+0000 7f467b7fe700 1 -- 172.21.15.107:0/2400235557 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4670004160 con 0x7f467c108690 2024-04-25T00:06:35.741 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.632+0000 7f467b7fe700 1 -- 172.21.15.107:0/2400235557 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4670004300 con 0x7f467c108690 2024-04-25T00:06:35.741 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.632+0000 7f467b7fe700 1 -- 172.21.15.107:0/2400235557 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4670011b50 con 0x7f467c108690 2024-04-25T00:06:35.741 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.633+0000 7f4682c86700 1 -- 172.21.15.107:0/2400235557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f467c108690 msgr2=0x7f467c10aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:35.742 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.633+0000 7f4682c86700 1 --2- 172.21.15.107:0/2400235557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f467c108690 0x7f467c10aa80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f46700098a0 tx=0x7f4670009bb0 comp rx=0 tx=0).stop 2024-04-25T00:06:35.742 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.633+0000 7f4682c86700 1 -- 172.21.15.107:0/2400235557 shutdown_connections 2024-04-25T00:06:35.742 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.633+0000 7f4682c86700 1 --2- 172.21.15.107:0/2400235557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f467c108690 0x7f467c10aa80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:35.742 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.633+0000 7f4682c86700 1 -- 172.21.15.107:0/2400235557 >> 172.21.15.107:0/2400235557 conn(0x7f467c1020a0 msgr2=0x7f467c104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:35.742 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.633+0000 7f4682c86700 1 -- 172.21.15.107:0/2400235557 shutdown_connections 2024-04-25T00:06:35.742 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.633+0000 7f4682c86700 1 -- 172.21.15.107:0/2400235557 wait complete. 2024-04-25T00:06:35.742 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.634+0000 7f4682c86700 1 Processor -- start 2024-04-25T00:06:35.742 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.634+0000 7f4682c86700 1 -- start start 2024-04-25T00:06:35.742 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.634+0000 7f4682c86700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f467c108690 0x7f467c19bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:35.742 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.634+0000 7f4682c86700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f467c10bff0 con 0x7f467c108690 2024-04-25T00:06:35.742 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.634+0000 7f4680a22700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f467c108690 0x7f467c19bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:35.743 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.634+0000 7f4680a22700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f467c108690 0x7f467c19bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55276/0 (socket says 172.21.15.107:55276) 2024-04-25T00:06:35.743 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.634+0000 7f4680a22700 1 -- 172.21.15.107:0/1744799399 learned_addr learned my addr 172.21.15.107:0/1744799399 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:35.743 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.634+0000 7f4680a22700 1 -- 172.21.15.107:0/1744799399 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467c19c110 con 0x7f467c108690 2024-04-25T00:06:35.743 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.634+0000 7f4680a22700 1 --2- 172.21.15.107:0/1744799399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f467c108690 0x7f467c19bbd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4670011670 tx=0x7f4670011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:35.743 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.635+0000 7f4679ffb700 1 -- 172.21.15.107:0/1744799399 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46700039c0 con 0x7f467c108690 2024-04-25T00:06:35.743 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.635+0000 7f4679ffb700 1 -- 172.21.15.107:0/1744799399 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4670011e50 con 0x7f467c108690 2024-04-25T00:06:35.743 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.635+0000 7f4682c86700 1 -- 172.21.15.107:0/1744799399 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f467c19c3a0 con 0x7f467c108690 2024-04-25T00:06:35.743 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.635+0000 7f4679ffb700 1 -- 172.21.15.107:0/1744799399 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f467001a5c0 con 0x7f467c108690 2024-04-25T00:06:35.743 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.635+0000 7f4682c86700 1 -- 172.21.15.107:0/1744799399 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f467c19f090 con 0x7f467c108690 2024-04-25T00:06:35.743 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.636+0000 7f4679ffb700 1 -- 172.21.15.107:0/1744799399 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f467001a8c0 con 0x7f467c108690 2024-04-25T00:06:35.743 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.636+0000 7f4679ffb700 1 --2- 172.21.15.107:0/1744799399 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f466c039000 0x7f466c03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:35.743 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.636+0000 7f4682c86700 1 -- 172.21.15.107:0/1744799399 --> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f467c063890 con 0x7f466c039000 2024-04-25T00:06:35.743 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.636+0000 7f4679ffb700 1 -- 172.21.15.107:0/1744799399 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f467004e440 con 0x7f467c108690 2024-04-25T00:06:35.743 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.636+0000 7f467bfff700 1 -- 172.21.15.107:0/1744799399 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f466c039000 msgr2=0x7f466c03b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/120616774 2024-04-25T00:06:35.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.636+0000 7f467bfff700 1 --2- 172.21.15.107:0/1744799399 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f466c039000 0x7f466c03b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:06:35.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.837+0000 7f467bfff700 1 -- 172.21.15.107:0/1744799399 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f466c039000 msgr2=0x7f466c03b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/120616774 2024-04-25T00:06:35.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:29.837+0000 7f467bfff700 1 --2- 172.21.15.107:0/1744799399 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f466c039000 0x7f466c03b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T00:06:35.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:30.238+0000 7f467bfff700 1 -- 172.21.15.107:0/1744799399 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f466c039000 msgr2=0x7f466c03b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/120616774 2024-04-25T00:06:35.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:30.238+0000 7f467bfff700 1 --2- 172.21.15.107:0/1744799399 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f466c039000 0x7f466c03b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T00:06:35.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:31.039+0000 7f467bfff700 1 -- 172.21.15.107:0/1744799399 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f466c039000 msgr2=0x7f466c03b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/120616774 2024-04-25T00:06:35.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:31.039+0000 7f467bfff700 1 --2- 172.21.15.107:0/1744799399 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f466c039000 0x7f466c03b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T00:06:35.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:32.641+0000 7f467bfff700 1 -- 172.21.15.107:0/1744799399 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f466c039000 msgr2=0x7f466c03b4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/120616774 2024-04-25T00:06:35.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:32.641+0000 7f467bfff700 1 --2- 172.21.15.107:0/1744799399 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f466c039000 0x7f466c03b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T00:06:35.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:34.126+0000 7f4679ffb700 1 -- 172.21.15.107:0/1744799399 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f467002d910 con 0x7f467c108690 2024-04-25T00:06:35.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:34.126+0000 7f4679ffb700 1 -- 172.21.15.107:0/1744799399 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f466c039000 msgr2=0x7f466c03b4c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:06:35.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:34.126+0000 7f4679ffb700 1 --2- 172.21.15.107:0/1744799399 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f466c039000 0x7f466c03b4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:35.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:35.127+0000 7f4679ffb700 1 -- 172.21.15.107:0/1744799399 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f467000bce0 con 0x7f467c108690 2024-04-25T00:06:35.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:35.128+0000 7f4679ffb700 1 --2- 172.21.15.107:0/1744799399 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f466c03c740 0x7f466c03eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:35.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:35.128+0000 7f4679ffb700 1 -- 172.21.15.107:0/1744799399 --> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4670047570 con 0x7f466c03c740 2024-04-25T00:06:35.744 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:35.128+0000 7f467bfff700 1 --2- 172.21.15.107:0/1744799399 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f466c03c740 0x7f466c03eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:35.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:35.129+0000 7f467bfff700 1 --2- 172.21.15.107:0/1744799399 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f466c03c740 0x7f466c03eb30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4668003a10 tx=0x7f46680092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:35.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:35.129+0000 7f4679ffb700 1 -- 172.21.15.107:0/1744799399 <== mgr.14120 v2:172.21.15.107:6800/153169302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f4670047570 con 0x7f466c03c740 2024-04-25T00:06:35.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:35.136+0000 7f4682c86700 1 -- 172.21.15.107:0/1744799399 --> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f467c103310 con 0x7f466c03c740 2024-04-25T00:06:35.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:35.136+0000 7f4679ffb700 1 -- 172.21.15.107:0/1744799399 <== mgr.14120 v2:172.21.15.107:6800/153169302 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f467c103310 con 0x7f466c03c740 2024-04-25T00:06:35.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:35.136+0000 7f4682c86700 1 -- 172.21.15.107:0/1744799399 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f466c03c740 msgr2=0x7f466c03eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:35.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:35.136+0000 7f4682c86700 1 --2- 172.21.15.107:0/1744799399 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f466c03c740 0x7f466c03eb30 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4668003a10 tx=0x7f46680092b0 comp rx=0 tx=0).stop 2024-04-25T00:06:35.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:35.136+0000 7f4682c86700 1 -- 172.21.15.107:0/1744799399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f467c108690 msgr2=0x7f467c19bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:35.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:35.136+0000 7f4682c86700 1 --2- 172.21.15.107:0/1744799399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f467c108690 0x7f467c19bbd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4670011670 tx=0x7f4670011b60 comp rx=0 tx=0).stop 2024-04-25T00:06:35.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:35.136+0000 7f4682c86700 1 -- 172.21.15.107:0/1744799399 shutdown_connections 2024-04-25T00:06:35.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:35.136+0000 7f4682c86700 1 --2- 172.21.15.107:0/1744799399 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f466c03c740 0x7f466c03eb30 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:35.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:35.136+0000 7f4682c86700 1 --2- 172.21.15.107:0/1744799399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f467c108690 0x7f467c19bbd0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:35.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:35.136+0000 7f4682c86700 1 --2- 172.21.15.107:0/1744799399 >> [v2:172.21.15.107:6800/120616774,v1:172.21.15.107:6801/120616774] conn(0x7f466c039000 0x7f466c03b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:35.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:35.136+0000 7f4682c86700 1 -- 172.21.15.107:0/1744799399 >> 172.21.15.107:0/1744799399 conn(0x7f467c1020a0 msgr2=0x7f467c102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:35.745 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:35.136+0000 7f4682c86700 1 -- 172.21.15.107:0/1744799399 shutdown_connections 2024-04-25T00:06:35.746 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:35.136+0000 7f4682c86700 1 -- 172.21.15.107:0/1744799399 wait complete. 2024-04-25T00:06:35.746 INFO:teuthology.orchestra.run.smithi107.stdout:mgr epoch 5 is available 2024-04-25T00:06:35.746 INFO:teuthology.orchestra.run.smithi107.stdout:Setting orchestrator backend to cephadm... 2024-04-25T00:06:36.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:36 smithi107 ceph-mon[29153]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T00:06:36.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:36 smithi107 ceph-mon[29153]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T00:06:36.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:36 smithi107 ceph-mon[29153]: [25/Apr/2024:00:06:35] ENGINE Bus STARTING 2024-04-25T00:06:36.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:36 smithi107 ceph-mon[29153]: [25/Apr/2024:00:06:35] ENGINE Serving on http://172.21.15.107:8765 2024-04-25T00:06:36.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:36 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:06:36.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.122+0000 7f2d90fdd700 1 Processor -- start 2024-04-25T00:06:36.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.122+0000 7f2d90fdd700 1 -- start start 2024-04-25T00:06:36.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.122+0000 7f2d90fdd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d8c106550 0x7f2d8c106970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:36.977 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.122+0000 7f2d90fdd700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c07be30 con 0x7f2d8c106550 2024-04-25T00:06:36.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.122+0000 7f2d8a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d8c106550 0x7f2d8c106970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:36.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.123+0000 7f2d8a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d8c106550 0x7f2d8c106970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55346/0 (socket says 172.21.15.107:55346) 2024-04-25T00:06:36.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.123+0000 7f2d8a59c700 1 -- 172.21.15.107:0/2120454204 learned_addr learned my addr 172.21.15.107:0/2120454204 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:36.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.123+0000 7f2d8a59c700 1 -- 172.21.15.107:0/2120454204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8c106eb0 con 0x7f2d8c106550 2024-04-25T00:06:36.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.123+0000 7f2d8a59c700 1 --2- 172.21.15.107:0/2120454204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d8c106550 0x7f2d8c106970 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c0098a0 tx=0x7f2d7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e675ad5ad5d2b4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:36.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.123+0000 7f2d89d9b700 1 -- 172.21.15.107:0/2120454204 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d7c004160 con 0x7f2d8c106550 2024-04-25T00:06:36.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.123+0000 7f2d89d9b700 1 -- 172.21.15.107:0/2120454204 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2d7c004300 con 0x7f2d8c106550 2024-04-25T00:06:36.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.123+0000 7f2d89d9b700 1 -- 172.21.15.107:0/2120454204 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d7c011b50 con 0x7f2d8c106550 2024-04-25T00:06:36.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.124+0000 7f2d90fdd700 1 -- 172.21.15.107:0/2120454204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d8c106550 msgr2=0x7f2d8c106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:36.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.124+0000 7f2d90fdd700 1 --2- 172.21.15.107:0/2120454204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d8c106550 0x7f2d8c106970 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c0098a0 tx=0x7f2d7c009bb0 comp rx=0 tx=0).stop 2024-04-25T00:06:36.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.124+0000 7f2d90fdd700 1 -- 172.21.15.107:0/2120454204 shutdown_connections 2024-04-25T00:06:36.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.124+0000 7f2d90fdd700 1 --2- 172.21.15.107:0/2120454204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d8c106550 0x7f2d8c106970 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:36.978 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.124+0000 7f2d90fdd700 1 -- 172.21.15.107:0/2120454204 >> 172.21.15.107:0/2120454204 conn(0x7f2d8c102130 msgr2=0x7f2d8c104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:36.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.124+0000 7f2d90fdd700 1 -- 172.21.15.107:0/2120454204 shutdown_connections 2024-04-25T00:06:36.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.124+0000 7f2d90fdd700 1 -- 172.21.15.107:0/2120454204 wait complete. 2024-04-25T00:06:36.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.125+0000 7f2d90fdd700 1 Processor -- start 2024-04-25T00:06:36.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.125+0000 7f2d90fdd700 1 -- start start 2024-04-25T00:06:36.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.125+0000 7f2d90fdd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d8c106550 0x7f2d8c19de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:36.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.125+0000 7f2d90fdd700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8c107f40 con 0x7f2d8c106550 2024-04-25T00:06:36.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.125+0000 7f2d8a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d8c106550 0x7f2d8c19de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:36.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.125+0000 7f2d8a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d8c106550 0x7f2d8c19de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55358/0 (socket says 172.21.15.107:55358) 2024-04-25T00:06:36.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.125+0000 7f2d8a59c700 1 -- 172.21.15.107:0/3177418489 learned_addr learned my addr 172.21.15.107:0/3177418489 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:36.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.125+0000 7f2d8a59c700 1 -- 172.21.15.107:0/3177418489 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8c19e370 con 0x7f2d8c106550 2024-04-25T00:06:36.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.125+0000 7f2d8a59c700 1 --2- 172.21.15.107:0/3177418489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d8c106550 0x7f2d8c19de30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009fb0 tx=0x7f2d7c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:36.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.126+0000 7f2d83fff700 1 -- 172.21.15.107:0/3177418489 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d7c0038b0 con 0x7f2d8c106550 2024-04-25T00:06:36.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.126+0000 7f2d90fdd700 1 -- 172.21.15.107:0/3177418489 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d8c19e600 con 0x7f2d8c106550 2024-04-25T00:06:36.979 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.126+0000 7f2d83fff700 1 -- 172.21.15.107:0/3177418489 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2d7c011e50 con 0x7f2d8c106550 2024-04-25T00:06:36.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.126+0000 7f2d83fff700 1 -- 172.21.15.107:0/3177418489 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d7c019420 con 0x7f2d8c106550 2024-04-25T00:06:36.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.126+0000 7f2d90fdd700 1 -- 172.21.15.107:0/3177418489 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d8c1a12f0 con 0x7f2d8c106550 2024-04-25T00:06:36.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.127+0000 7f2d83fff700 1 -- 172.21.15.107:0/3177418489 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f2d7c019720 con 0x7f2d8c106550 2024-04-25T00:06:36.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.127+0000 7f2d90fdd700 1 -- 172.21.15.107:0/3177418489 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d8c063890 con 0x7f2d8c106550 2024-04-25T00:06:36.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.127+0000 7f2d83fff700 1 --2- 172.21.15.107:0/3177418489 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2d78038ef0 0x7f2d7803b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:36.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.127+0000 7f2d83fff700 1 -- 172.21.15.107:0/3177418489 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2d7c04d140 con 0x7f2d8c106550 2024-04-25T00:06:36.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.127+0000 7f2d81d9b700 1 --2- 172.21.15.107:0/3177418489 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2d78038ef0 0x7f2d7803b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:36.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.127+0000 7f2d81d9b700 1 --2- 172.21.15.107:0/3177418489 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2d78038ef0 0x7f2d7803b3b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2d74006fd0 tx=0x7f2d74006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:36.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.132+0000 7f2d83fff700 1 -- 172.21.15.107:0/3177418489 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2d7c00bc70 con 0x7f2d8c106550 2024-04-25T00:06:36.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.179+0000 7f2d83fff700 1 -- 172.21.15.107:0/3177418489 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f2d7c003a50 con 0x7f2d8c106550 2024-04-25T00:06:36.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.348+0000 7f2d90fdd700 1 -- 172.21.15.107:0/3177418489 --> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f2d8c0008d0 con 0x7f2d78038ef0 2024-04-25T00:06:36.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.355+0000 7f2d83fff700 1 -- 172.21.15.107:0/3177418489 <== mgr.14120 v2:172.21.15.107:6800/153169302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2d8c0008d0 con 0x7f2d78038ef0 2024-04-25T00:06:36.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.358+0000 7f2d90fdd700 1 -- 172.21.15.107:0/3177418489 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2d78038ef0 msgr2=0x7f2d7803b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:36.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.358+0000 7f2d90fdd700 1 --2- 172.21.15.107:0/3177418489 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2d78038ef0 0x7f2d7803b3b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2d74006fd0 tx=0x7f2d74006e40 comp rx=0 tx=0).stop 2024-04-25T00:06:36.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.358+0000 7f2d90fdd700 1 -- 172.21.15.107:0/3177418489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d8c106550 msgr2=0x7f2d8c19de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:36.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.358+0000 7f2d90fdd700 1 --2- 172.21.15.107:0/3177418489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d8c106550 0x7f2d8c19de30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009fb0 tx=0x7f2d7c011bf0 comp rx=0 tx=0).stop 2024-04-25T00:06:36.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.358+0000 7f2d90fdd700 1 -- 172.21.15.107:0/3177418489 shutdown_connections 2024-04-25T00:06:36.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.358+0000 7f2d90fdd700 1 --2- 172.21.15.107:0/3177418489 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2d78038ef0 0x7f2d7803b3b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:36.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.358+0000 7f2d90fdd700 1 --2- 172.21.15.107:0/3177418489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d8c106550 0x7f2d8c19de30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:36.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.358+0000 7f2d90fdd700 1 -- 172.21.15.107:0/3177418489 >> 172.21.15.107:0/3177418489 conn(0x7f2d8c102130 msgr2=0x7f2d8c194a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:36.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.358+0000 7f2d90fdd700 1 -- 172.21.15.107:0/3177418489 shutdown_connections 2024-04-25T00:06:36.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:36.358+0000 7f2d90fdd700 1 -- 172.21.15.107:0/3177418489 wait complete. 2024-04-25T00:06:37.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:37 smithi107 ceph-mon[29153]: [25/Apr/2024:00:06:35] ENGINE Serving on https://172.21.15.107:7150 2024-04-25T00:06:37.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:37 smithi107 ceph-mon[29153]: [25/Apr/2024:00:06:35] ENGINE Bus STARTED 2024-04-25T00:06:37.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:37 smithi107 ceph-mon[29153]: mgrmap e8: x(active, since 2s) 2024-04-25T00:06:37.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:37 smithi107 ceph-mon[29153]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:06:37.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:37 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' 2024-04-25T00:06:37.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:37 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:06:38.242 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T00:06:38.242 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.388+0000 7f2d8f731700 1 Processor -- start 2024-04-25T00:06:38.242 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.388+0000 7f2d8f731700 1 -- start start 2024-04-25T00:06:38.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.388+0000 7f2d8f731700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d881062c0 0x7f2d881066e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:38.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.389+0000 7f2d8f731700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2d88106cb0 con 0x7f2d881062c0 2024-04-25T00:06:38.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.389+0000 7f2d8d4cd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d881062c0 0x7f2d881066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:38.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.389+0000 7f2d8d4cd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d881062c0 0x7f2d881066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55362/0 (socket says 172.21.15.107:55362) 2024-04-25T00:06:38.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.389+0000 7f2d8d4cd700 1 -- 172.21.15.107:0/3817591068 learned_addr learned my addr 172.21.15.107:0/3817591068 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:38.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.389+0000 7f2d8d4cd700 1 -- 172.21.15.107:0/3817591068 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d881074c0 con 0x7f2d881062c0 2024-04-25T00:06:38.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.389+0000 7f2d8d4cd700 1 --2- 172.21.15.107:0/3817591068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d881062c0 0x7f2d881066e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400d180 tx=0x7f2d8400d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bcc81c36f92a36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:38.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.389+0000 7f2d7ffff700 1 -- 172.21.15.107:0/3817591068 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d8401b070 con 0x7f2d881062c0 2024-04-25T00:06:38.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.389+0000 7f2d7ffff700 1 -- 172.21.15.107:0/3817591068 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2d8400dc10 con 0x7f2d881062c0 2024-04-25T00:06:38.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.389+0000 7f2d7ffff700 1 -- 172.21.15.107:0/3817591068 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d840048e0 con 0x7f2d881062c0 2024-04-25T00:06:38.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.390+0000 7f2d8f731700 1 -- 172.21.15.107:0/3817591068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d881062c0 msgr2=0x7f2d881066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:38.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.390+0000 7f2d8f731700 1 --2- 172.21.15.107:0/3817591068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d881062c0 0x7f2d881066e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400d180 tx=0x7f2d8400d490 comp rx=0 tx=0).stop 2024-04-25T00:06:38.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.390+0000 7f2d8f731700 1 -- 172.21.15.107:0/3817591068 shutdown_connections 2024-04-25T00:06:38.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.390+0000 7f2d8f731700 1 --2- 172.21.15.107:0/3817591068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d881062c0 0x7f2d881066e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:38.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.390+0000 7f2d8f731700 1 -- 172.21.15.107:0/3817591068 >> 172.21.15.107:0/3817591068 conn(0x7f2d881018a0 msgr2=0x7f2d88103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:38.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.390+0000 7f2d8f731700 1 -- 172.21.15.107:0/3817591068 shutdown_connections 2024-04-25T00:06:38.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.390+0000 7f2d8f731700 1 -- 172.21.15.107:0/3817591068 wait complete. 2024-04-25T00:06:38.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.391+0000 7f2d8f731700 1 Processor -- start 2024-04-25T00:06:38.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.391+0000 7f2d8f731700 1 -- start start 2024-04-25T00:06:38.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.391+0000 7f2d8f731700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d881062c0 0x7f2d8819dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:38.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.391+0000 7f2d8f731700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2d88107c40 con 0x7f2d881062c0 2024-04-25T00:06:38.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.391+0000 7f2d8d4cd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d881062c0 0x7f2d8819dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:38.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.391+0000 7f2d8d4cd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d881062c0 0x7f2d8819dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55374/0 (socket says 172.21.15.107:55374) 2024-04-25T00:06:38.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.391+0000 7f2d8d4cd700 1 -- 172.21.15.107:0/3555768522 learned_addr learned my addr 172.21.15.107:0/3555768522 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:38.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.391+0000 7f2d8d4cd700 1 -- 172.21.15.107:0/3555768522 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8819e2a0 con 0x7f2d881062c0 2024-04-25T00:06:38.244 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.391+0000 7f2d8d4cd700 1 --2- 172.21.15.107:0/3555768522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d881062c0 0x7f2d8819dd60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400f010 tx=0x7f2d84003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:38.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.391+0000 7f2d7e7fc700 1 -- 172.21.15.107:0/3555768522 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d8401b040 con 0x7f2d881062c0 2024-04-25T00:06:38.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.391+0000 7f2d7e7fc700 1 -- 172.21.15.107:0/3555768522 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2d84024070 con 0x7f2d881062c0 2024-04-25T00:06:38.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.391+0000 7f2d8f731700 1 -- 172.21.15.107:0/3555768522 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d8819e530 con 0x7f2d881062c0 2024-04-25T00:06:38.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.391+0000 7f2d7e7fc700 1 -- 172.21.15.107:0/3555768522 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d84016dd0 con 0x7f2d881062c0 2024-04-25T00:06:38.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.391+0000 7f2d8f731700 1 -- 172.21.15.107:0/3555768522 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d881a1220 con 0x7f2d881062c0 2024-04-25T00:06:38.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.392+0000 7f2d7e7fc700 1 -- 172.21.15.107:0/3555768522 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f2d84029460 con 0x7f2d881062c0 2024-04-25T00:06:38.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.392+0000 7f2d8f731700 1 -- 172.21.15.107:0/3555768522 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d88063890 con 0x7f2d881062c0 2024-04-25T00:06:38.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.392+0000 7f2d7e7fc700 1 --2- 172.21.15.107:0/3555768522 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2d74039000 0x7f2d7403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:38.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.392+0000 7f2d7e7fc700 1 -- 172.21.15.107:0/3555768522 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2d84011030 con 0x7f2d881062c0 2024-04-25T00:06:38.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.393+0000 7f2d8cccc700 1 --2- 172.21.15.107:0/3555768522 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2d74039000 0x7f2d7403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:38.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.393+0000 7f2d8cccc700 1 --2- 172.21.15.107:0/3555768522 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2d74039000 0x7f2d7403b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2d78006fd0 tx=0x7f2d78006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:38.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.398+0000 7f2d7e7fc700 1 -- 172.21.15.107:0/3555768522 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2d8400bce0 con 0x7f2d881062c0 2024-04-25T00:06:38.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.596+0000 7f2d8f731700 1 -- 172.21.15.107:0/3555768522 --> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f2d880008d0 con 0x7f2d74039000 2024-04-25T00:06:38.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.596+0000 7f2d7e7fc700 1 -- 172.21.15.107:0/3555768522 <== mgr.14120 v2:172.21.15.107:6800/153169302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f2d880008d0 con 0x7f2d74039000 2024-04-25T00:06:38.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.599+0000 7f2d8f731700 1 -- 172.21.15.107:0/3555768522 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2d74039000 msgr2=0x7f2d7403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:38.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.599+0000 7f2d8f731700 1 --2- 172.21.15.107:0/3555768522 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2d74039000 0x7f2d7403b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2d78006fd0 tx=0x7f2d78006e40 comp rx=0 tx=0).stop 2024-04-25T00:06:38.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.599+0000 7f2d8f731700 1 -- 172.21.15.107:0/3555768522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d881062c0 msgr2=0x7f2d8819dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:38.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.599+0000 7f2d8f731700 1 --2- 172.21.15.107:0/3555768522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d881062c0 0x7f2d8819dd60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2d8400f010 tx=0x7f2d84003d00 comp rx=0 tx=0).stop 2024-04-25T00:06:38.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.599+0000 7f2d8f731700 1 -- 172.21.15.107:0/3555768522 shutdown_connections 2024-04-25T00:06:38.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.599+0000 7f2d8f731700 1 --2- 172.21.15.107:0/3555768522 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2d74039000 0x7f2d7403b4c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:38.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.599+0000 7f2d8f731700 1 --2- 172.21.15.107:0/3555768522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d881062c0 0x7f2d8819dd60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:38.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.599+0000 7f2d8f731700 1 -- 172.21.15.107:0/3555768522 >> 172.21.15.107:0/3555768522 conn(0x7f2d881018a0 msgr2=0x7f2d88190570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:38.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.599+0000 7f2d8f731700 1 -- 172.21.15.107:0/3555768522 shutdown_connections 2024-04-25T00:06:38.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:37.599+0000 7f2d8f731700 1 -- 172.21.15.107:0/3555768522 wait complete. 2024-04-25T00:06:38.246 INFO:teuthology.orchestra.run.smithi107.stdout:Generating ssh key... 2024-04-25T00:06:38.850 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:38 smithi107 ceph-mon[29153]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:06:39.234 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:38 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: Generating public/private rsa key pair. 2024-04-25T00:06:39.234 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:38 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: Your identification has been saved in /tmp/tmp3hvducv7/key. 2024-04-25T00:06:39.234 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:38 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: Your public key has been saved in /tmp/tmp3hvducv7/key.pub. 2024-04-25T00:06:39.235 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:38 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: The key fingerprint is: 2024-04-25T00:06:39.235 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:38 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: SHA256:aT3+QUc6qS1qFrlZTlhPVKSA4pYiOAk3F8GDXmQ8OBo ceph-79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:06:39.235 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:38 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: The key's randomart image is: 2024-04-25T00:06:39.235 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:38 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: +---[RSA 3072]----+ 2024-04-25T00:06:39.235 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:38 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: | *=o .. oo | 2024-04-25T00:06:39.235 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:38 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: |E *.B . . ... | 2024-04-25T00:06:39.235 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:38 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: |.B = + o .. . | 2024-04-25T00:06:39.235 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:38 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: |= o . + o. .+ | 2024-04-25T00:06:39.235 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:38 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: | . . o S+oo= . | 2024-04-25T00:06:39.235 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:38 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: | .+.o=.o | 2024-04-25T00:06:39.235 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:38 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: | B+ o | 2024-04-25T00:06:39.235 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:38 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: | =..o . | 2024-04-25T00:06:39.235 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:38 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: | o. . | 2024-04-25T00:06:39.236 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:38 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: +----[SHA256]-----+ 2024-04-25T00:06:39.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.646+0000 7f21034e7700 1 Processor -- start 2024-04-25T00:06:39.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.647+0000 7f21034e7700 1 -- start start 2024-04-25T00:06:39.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.647+0000 7f21034e7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20fc07dec0 0x7f20fc07c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:39.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.647+0000 7f21034e7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f20fc07c8f0 con 0x7f20fc07dec0 2024-04-25T00:06:39.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.647+0000 7f2101283700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20fc07dec0 0x7f20fc07c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:39.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.647+0000 7f2101283700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20fc07dec0 0x7f20fc07c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55380/0 (socket says 172.21.15.107:55380) 2024-04-25T00:06:39.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.647+0000 7f2101283700 1 -- 172.21.15.107:0/2431245572 learned_addr learned my addr 172.21.15.107:0/2431245572 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:39.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.647+0000 7f2101283700 1 -- 172.21.15.107:0/2431245572 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20fc07ca70 con 0x7f20fc07dec0 2024-04-25T00:06:39.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.647+0000 7f2101283700 1 --2- 172.21.15.107:0/2431245572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20fc07dec0 0x7f20fc07c320 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0098a0 tx=0x7f20ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55103232e05ff998 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:39.550 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.647+0000 7f20f3fff700 1 -- 172.21.15.107:0/2431245572 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20ec004160 con 0x7f20fc07dec0 2024-04-25T00:06:39.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.648+0000 7f20f3fff700 1 -- 172.21.15.107:0/2431245572 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f20ec004300 con 0x7f20fc07dec0 2024-04-25T00:06:39.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.648+0000 7f20f3fff700 1 -- 172.21.15.107:0/2431245572 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20ec011b50 con 0x7f20fc07dec0 2024-04-25T00:06:39.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.648+0000 7f21034e7700 1 -- 172.21.15.107:0/2431245572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20fc07dec0 msgr2=0x7f20fc07c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:39.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.648+0000 7f21034e7700 1 --2- 172.21.15.107:0/2431245572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20fc07dec0 0x7f20fc07c320 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0098a0 tx=0x7f20ec009bb0 comp rx=0 tx=0).stop 2024-04-25T00:06:39.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.648+0000 7f21034e7700 1 -- 172.21.15.107:0/2431245572 shutdown_connections 2024-04-25T00:06:39.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.648+0000 7f21034e7700 1 --2- 172.21.15.107:0/2431245572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20fc07dec0 0x7f20fc07c320 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:39.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.648+0000 7f21034e7700 1 -- 172.21.15.107:0/2431245572 >> 172.21.15.107:0/2431245572 conn(0x7f20fc1031e0 msgr2=0x7f20fc105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:39.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.648+0000 7f21034e7700 1 -- 172.21.15.107:0/2431245572 shutdown_connections 2024-04-25T00:06:39.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.648+0000 7f21034e7700 1 -- 172.21.15.107:0/2431245572 wait complete. 2024-04-25T00:06:39.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.649+0000 7f21034e7700 1 Processor -- start 2024-04-25T00:06:39.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.649+0000 7f21034e7700 1 -- start start 2024-04-25T00:06:39.551 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.649+0000 7f21034e7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20fc07dec0 0x7f20fc19de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:39.552 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.649+0000 7f21034e7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f20fc10e230 con 0x7f20fc07dec0 2024-04-25T00:06:39.552 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.649+0000 7f2101283700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20fc07dec0 0x7f20fc19de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:39.552 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.649+0000 7f2101283700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20fc07dec0 0x7f20fc19de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55386/0 (socket says 172.21.15.107:55386) 2024-04-25T00:06:39.552 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.649+0000 7f2101283700 1 -- 172.21.15.107:0/2550852687 learned_addr learned my addr 172.21.15.107:0/2550852687 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:39.552 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.649+0000 7f2101283700 1 -- 172.21.15.107:0/2550852687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20fc19e370 con 0x7f20fc07dec0 2024-04-25T00:06:39.552 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.649+0000 7f2101283700 1 --2- 172.21.15.107:0/2550852687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20fc07dec0 0x7f20fc19de30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009fb0 tx=0x7f20ec011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:39.552 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.650+0000 7f20f27fc700 1 -- 172.21.15.107:0/2550852687 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20ec0038b0 con 0x7f20fc07dec0 2024-04-25T00:06:39.552 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.650+0000 7f21034e7700 1 -- 172.21.15.107:0/2550852687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20fc19e600 con 0x7f20fc07dec0 2024-04-25T00:06:39.552 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.650+0000 7f20f27fc700 1 -- 172.21.15.107:0/2550852687 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f20ec011e50 con 0x7f20fc07dec0 2024-04-25T00:06:39.552 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.650+0000 7f20f27fc700 1 -- 172.21.15.107:0/2550852687 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20ec019550 con 0x7f20fc07dec0 2024-04-25T00:06:39.552 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.650+0000 7f21034e7700 1 -- 172.21.15.107:0/2550852687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20fc1a12f0 con 0x7f20fc07dec0 2024-04-25T00:06:39.552 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.651+0000 7f20f27fc700 1 -- 172.21.15.107:0/2550852687 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f20ec01a430 con 0x7f20fc07dec0 2024-04-25T00:06:39.553 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.651+0000 7f21034e7700 1 -- 172.21.15.107:0/2550852687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20fc063890 con 0x7f20fc07dec0 2024-04-25T00:06:39.553 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.651+0000 7f20f27fc700 1 --2- 172.21.15.107:0/2550852687 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f20e8039000 0x7f20e803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:39.553 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.651+0000 7f20f27fc700 1 -- 172.21.15.107:0/2550852687 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f20ec04d2c0 con 0x7f20fc07dec0 2024-04-25T00:06:39.553 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.651+0000 7f2100a82700 1 --2- 172.21.15.107:0/2550852687 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f20e8039000 0x7f20e803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:39.553 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.651+0000 7f2100a82700 1 --2- 172.21.15.107:0/2550852687 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f20e8039000 0x7f20e803b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f20f8006fd0 tx=0x7f20f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:39.553 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.656+0000 7f20f27fc700 1 -- 172.21.15.107:0/2550852687 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f20ec016df0 con 0x7f20fc07dec0 2024-04-25T00:06:39.553 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:38.850+0000 7f21034e7700 1 -- 172.21.15.107:0/2550852687 --> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f20fc1044b0 con 0x7f20e8039000 2024-04-25T00:06:39.553 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.002+0000 7f20f27fc700 1 -- 172.21.15.107:0/2550852687 <== mgr.14120 v2:172.21.15.107:6800/153169302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f20fc1044b0 con 0x7f20e8039000 2024-04-25T00:06:39.553 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.005+0000 7f21034e7700 1 -- 172.21.15.107:0/2550852687 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f20e8039000 msgr2=0x7f20e803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:39.553 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.005+0000 7f21034e7700 1 --2- 172.21.15.107:0/2550852687 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f20e8039000 0x7f20e803b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f20f8006fd0 tx=0x7f20f8006e40 comp rx=0 tx=0).stop 2024-04-25T00:06:39.553 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.005+0000 7f21034e7700 1 -- 172.21.15.107:0/2550852687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20fc07dec0 msgr2=0x7f20fc19de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:39.553 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.005+0000 7f21034e7700 1 --2- 172.21.15.107:0/2550852687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20fc07dec0 0x7f20fc19de30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009fb0 tx=0x7f20ec011bf0 comp rx=0 tx=0).stop 2024-04-25T00:06:39.554 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.005+0000 7f21034e7700 1 -- 172.21.15.107:0/2550852687 shutdown_connections 2024-04-25T00:06:39.554 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.005+0000 7f21034e7700 1 --2- 172.21.15.107:0/2550852687 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f20e8039000 0x7f20e803b4c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:39.554 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.005+0000 7f21034e7700 1 --2- 172.21.15.107:0/2550852687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20fc07dec0 0x7f20fc19de30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:39.554 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.005+0000 7f21034e7700 1 -- 172.21.15.107:0/2550852687 >> 172.21.15.107:0/2550852687 conn(0x7f20fc1031e0 msgr2=0x7f20fc103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:39.554 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.005+0000 7f21034e7700 1 -- 172.21.15.107:0/2550852687 shutdown_connections 2024-04-25T00:06:39.554 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.005+0000 7f21034e7700 1 -- 172.21.15.107:0/2550852687 wait complete. 2024-04-25T00:06:40.172 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:39 smithi107 ceph-mon[29153]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:06:40.172 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:39 smithi107 ceph-mon[29153]: Generating ssh key... 2024-04-25T00:06:40.173 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:39 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' 2024-04-25T00:06:40.173 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:40 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' 2024-04-25T00:06:40.722 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtn9okv5SpIZDqnTSq6enC6t7ieT4CePD91sxTABXCeULXaUSZ99+za+yoo0m/bLoN7jOi29Rup10KWcmXZxHFWp0ZCYKW8NktYeAW/SBz5TKY8l5/X4RgGd60BWjY120VXqX0rx4GH/h19SkKdtROJsbzXwQWliL7GSLzQv89nUekIDpkX8jVJXsrIfwfIpObs0tIQJaVQTOZ7LvULInhQ11UCy1KF6uuPxQpHavHjGvb00GqFuXKew3GQ+HeYL6EEZc3b4uUFpZlxK3bIRcueniMl4tVhkOssNHgPTcLMnnoW6mDyipQ1oNJIAsxLQNYcB9ZSnPxx/AOiAW17FgIIemk99K4ayyHbnan495mqDSElj1BM1P4fqzlYePquKavXjDoyFLIdIhAvwEnrMrXgZYOcD1xHWvPbPRIEddyvp41ljqG23jQqVgNosWMgU8WEW6XuIatxbZatKYruqhVP4ylbFUiG/kE9XcGW+mClhFj8nq3NdCM7tmbtub/PeM= ceph-79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:06:40.723 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.965+0000 7f13ef729700 1 Processor -- start 2024-04-25T00:06:40.723 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.965+0000 7f13ef729700 1 -- start start 2024-04-25T00:06:40.723 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.965+0000 7f13ef729700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e807de80 0x7f13e807c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:40.723 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.965+0000 7f13ef729700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f13e807c8b0 con 0x7f13e807de80 2024-04-25T00:06:40.723 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.965+0000 7f13ed4c5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e807de80 0x7f13e807c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:40.723 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.965+0000 7f13ed4c5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e807de80 0x7f13e807c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38246/0 (socket says 172.21.15.107:38246) 2024-04-25T00:06:40.723 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.965+0000 7f13ed4c5700 1 -- 172.21.15.107:0/2675623871 learned_addr learned my addr 172.21.15.107:0/2675623871 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:40.723 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.966+0000 7f13ed4c5700 1 -- 172.21.15.107:0/2675623871 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e807ca30 con 0x7f13e807de80 2024-04-25T00:06:40.723 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.966+0000 7f13ed4c5700 1 --2- 172.21.15.107:0/2675623871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e807de80 0x7f13e807c2e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009e80 tx=0x7f13e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712ddf5a308f5a5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:40.723 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.966+0000 7f13dffff700 1 -- 172.21.15.107:0/2675623871 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13e0017070 con 0x7f13e807de80 2024-04-25T00:06:40.723 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.966+0000 7f13dffff700 1 -- 172.21.15.107:0/2675623871 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f13e000b8c0 con 0x7f13e807de80 2024-04-25T00:06:40.723 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.966+0000 7f13dffff700 1 -- 172.21.15.107:0/2675623871 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13e00045c0 con 0x7f13e807de80 2024-04-25T00:06:40.723 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.967+0000 7f13ef729700 1 -- 172.21.15.107:0/2675623871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e807de80 msgr2=0x7f13e807c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:40.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.967+0000 7f13ef729700 1 --2- 172.21.15.107:0/2675623871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e807de80 0x7f13e807c2e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009e80 tx=0x7f13e000b330 comp rx=0 tx=0).stop 2024-04-25T00:06:40.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.967+0000 7f13ef729700 1 -- 172.21.15.107:0/2675623871 shutdown_connections 2024-04-25T00:06:40.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.967+0000 7f13ef729700 1 --2- 172.21.15.107:0/2675623871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e807de80 0x7f13e807c2e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:40.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.967+0000 7f13ef729700 1 -- 172.21.15.107:0/2675623871 >> 172.21.15.107:0/2675623871 conn(0x7f13e81031e0 msgr2=0x7f13e8105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:40.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.967+0000 7f13ef729700 1 -- 172.21.15.107:0/2675623871 shutdown_connections 2024-04-25T00:06:40.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.967+0000 7f13ef729700 1 -- 172.21.15.107:0/2675623871 wait complete. 2024-04-25T00:06:40.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.967+0000 7f13ef729700 1 Processor -- start 2024-04-25T00:06:40.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.967+0000 7f13ef729700 1 -- start start 2024-04-25T00:06:40.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.967+0000 7f13ef729700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e807de80 0x7f13e819ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:40.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.967+0000 7f13ef729700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f13e810e270 con 0x7f13e807de80 2024-04-25T00:06:40.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.968+0000 7f13ed4c5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e807de80 0x7f13e819ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:40.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.968+0000 7f13ed4c5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e807de80 0x7f13e819ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38248/0 (socket says 172.21.15.107:38248) 2024-04-25T00:06:40.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.968+0000 7f13ed4c5700 1 -- 172.21.15.107:0/3882690115 learned_addr learned my addr 172.21.15.107:0/3882690115 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:40.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.968+0000 7f13ed4c5700 1 -- 172.21.15.107:0/3882690115 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e819e310 con 0x7f13e807de80 2024-04-25T00:06:40.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.968+0000 7f13ed4c5700 1 --2- 172.21.15.107:0/3882690115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e807de80 0x7f13e819ddd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009530 tx=0x7f13e000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:40.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.968+0000 7f13de7fc700 1 -- 172.21.15.107:0/3882690115 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13e0020020 con 0x7f13e807de80 2024-04-25T00:06:40.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.968+0000 7f13de7fc700 1 -- 172.21.15.107:0/3882690115 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f13e0003b50 con 0x7f13e807de80 2024-04-25T00:06:40.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.968+0000 7f13ef729700 1 -- 172.21.15.107:0/3882690115 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e819e5a0 con 0x7f13e807de80 2024-04-25T00:06:40.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.968+0000 7f13de7fc700 1 -- 172.21.15.107:0/3882690115 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13e0017040 con 0x7f13e807de80 2024-04-25T00:06:40.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.968+0000 7f13ef729700 1 -- 172.21.15.107:0/3882690115 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e81a1290 con 0x7f13e807de80 2024-04-25T00:06:40.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.969+0000 7f13de7fc700 1 -- 172.21.15.107:0/3882690115 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f13e0023020 con 0x7f13e807de80 2024-04-25T00:06:40.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.969+0000 7f13de7fc700 1 --2- 172.21.15.107:0/3882690115 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f13d40390a0 0x7f13d403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:40.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.969+0000 7f13de7fc700 1 -- 172.21.15.107:0/3882690115 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f13e000f050 con 0x7f13e807de80 2024-04-25T00:06:40.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.969+0000 7f13ef729700 1 -- 172.21.15.107:0/3882690115 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e8063890 con 0x7f13e807de80 2024-04-25T00:06:40.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.969+0000 7f13eccc4700 1 --2- 172.21.15.107:0/3882690115 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f13d40390a0 0x7f13d403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:40.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.970+0000 7f13eccc4700 1 --2- 172.21.15.107:0/3882690115 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f13d40390a0 0x7f13d403b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f13d8006fd0 tx=0x7f13d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:40.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:39.974+0000 7f13de7fc700 1 -- 172.21.15.107:0/3882690115 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f13e002a020 con 0x7f13e807de80 2024-04-25T00:06:40.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:40.171+0000 7f13ef729700 1 -- 172.21.15.107:0/3882690115 --> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f13e8104410 con 0x7f13d40390a0 2024-04-25T00:06:40.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:40.172+0000 7f13de7fc700 1 -- 172.21.15.107:0/3882690115 <== mgr.14120 v2:172.21.15.107:6800/153169302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f13e8104410 con 0x7f13d40390a0 2024-04-25T00:06:40.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:40.175+0000 7f13ef729700 1 -- 172.21.15.107:0/3882690115 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f13d40390a0 msgr2=0x7f13d403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:40.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:40.175+0000 7f13ef729700 1 --2- 172.21.15.107:0/3882690115 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f13d40390a0 0x7f13d403b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f13d8006fd0 tx=0x7f13d8006e40 comp rx=0 tx=0).stop 2024-04-25T00:06:40.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:40.175+0000 7f13ef729700 1 -- 172.21.15.107:0/3882690115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e807de80 msgr2=0x7f13e819ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:40.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:40.175+0000 7f13ef729700 1 --2- 172.21.15.107:0/3882690115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e807de80 0x7f13e819ddd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009530 tx=0x7f13e000bd50 comp rx=0 tx=0).stop 2024-04-25T00:06:40.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:40.175+0000 7f13ef729700 1 -- 172.21.15.107:0/3882690115 shutdown_connections 2024-04-25T00:06:40.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:40.175+0000 7f13ef729700 1 --2- 172.21.15.107:0/3882690115 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f13d40390a0 0x7f13d403b560 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:40.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:40.175+0000 7f13ef729700 1 --2- 172.21.15.107:0/3882690115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13e807de80 0x7f13e819ddd0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:40.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:40.175+0000 7f13ef729700 1 -- 172.21.15.107:0/3882690115 >> 172.21.15.107:0/3882690115 conn(0x7f13e81031e0 msgr2=0x7f13e8103d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:40.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:40.175+0000 7f13ef729700 1 -- 172.21.15.107:0/3882690115 shutdown_connections 2024-04-25T00:06:40.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:40.175+0000 7f13ef729700 1 -- 172.21.15.107:0/3882690115 wait complete. 2024-04-25T00:06:40.727 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T00:06:40.727 INFO:teuthology.orchestra.run.smithi107.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T00:06:40.727 INFO:teuthology.orchestra.run.smithi107.stdout:Adding host smithi107... 2024-04-25T00:06:41.084 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:41 smithi107 ceph-mon[29153]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:06:42.180 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:42 smithi107 ceph-mon[29153]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi107", "addr": "172.21.15.107", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:06:43.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:43 smithi107 ceph-mon[29153]: Deploying cephadm binary to smithi107 2024-04-25T00:06:46.032 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Added host 'smithi107' with addr '172.21.15.107' 2024-04-25T00:06:46.032 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.126+0000 7f97829a1700 1 Processor -- start 2024-04-25T00:06:46.032 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.127+0000 7f97829a1700 1 -- start start 2024-04-25T00:06:46.032 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.127+0000 7f97829a1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f977c107a40 0x7f977c107e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:46.032 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.127+0000 7f97829a1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f977c108430 con 0x7f977c107a40 2024-04-25T00:06:46.032 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.127+0000 7f978199f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f977c107a40 0x7f977c107e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:46.032 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.127+0000 7f978199f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f977c107a40 0x7f977c107e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38254/0 (socket says 172.21.15.107:38254) 2024-04-25T00:06:46.032 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.127+0000 7f978199f700 1 -- 172.21.15.107:0/3622735476 learned_addr learned my addr 172.21.15.107:0/3622735476 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:46.032 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.127+0000 7f978199f700 1 -- 172.21.15.107:0/3622735476 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977c108c40 con 0x7f977c107a40 2024-04-25T00:06:46.032 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.127+0000 7f978199f700 1 --2- 172.21.15.107:0/3622735476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f977c107a40 0x7f977c107e60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f976c0098a0 tx=0x7f976c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71349e16a61cf25e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:46.033 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.128+0000 7f978099d700 1 -- 172.21.15.107:0/3622735476 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f976c004160 con 0x7f977c107a40 2024-04-25T00:06:46.033 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.128+0000 7f978099d700 1 -- 172.21.15.107:0/3622735476 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f976c004300 con 0x7f977c107a40 2024-04-25T00:06:46.033 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.128+0000 7f978099d700 1 -- 172.21.15.107:0/3622735476 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f976c011b50 con 0x7f977c107a40 2024-04-25T00:06:46.033 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.128+0000 7f97829a1700 1 -- 172.21.15.107:0/3622735476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f977c107a40 msgr2=0x7f977c107e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:46.033 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.128+0000 7f97829a1700 1 --2- 172.21.15.107:0/3622735476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f977c107a40 0x7f977c107e60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f976c0098a0 tx=0x7f976c009bb0 comp rx=0 tx=0).stop 2024-04-25T00:06:46.033 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.128+0000 7f97829a1700 1 -- 172.21.15.107:0/3622735476 shutdown_connections 2024-04-25T00:06:46.033 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.128+0000 7f97829a1700 1 --2- 172.21.15.107:0/3622735476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f977c107a40 0x7f977c107e60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:46.033 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.128+0000 7f97829a1700 1 -- 172.21.15.107:0/3622735476 >> 172.21.15.107:0/3622735476 conn(0x7f977c103080 msgr2=0x7f977c1054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:46.033 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.128+0000 7f97829a1700 1 -- 172.21.15.107:0/3622735476 shutdown_connections 2024-04-25T00:06:46.034 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.128+0000 7f97829a1700 1 -- 172.21.15.107:0/3622735476 wait complete. 2024-04-25T00:06:46.034 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.129+0000 7f97829a1700 1 Processor -- start 2024-04-25T00:06:46.034 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.129+0000 7f97829a1700 1 -- start start 2024-04-25T00:06:46.034 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.129+0000 7f97829a1700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f977c107a40 0x7f977c19ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:46.034 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.129+0000 7f97829a1700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f977c10bff0 con 0x7f977c107a40 2024-04-25T00:06:46.034 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.129+0000 7f978199f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f977c107a40 0x7f977c19ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:46.034 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.129+0000 7f978199f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f977c107a40 0x7f977c19ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38258/0 (socket says 172.21.15.107:38258) 2024-04-25T00:06:46.034 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.129+0000 7f978199f700 1 -- 172.21.15.107:0/603700200 learned_addr learned my addr 172.21.15.107:0/603700200 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:46.034 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.129+0000 7f978199f700 1 -- 172.21.15.107:0/603700200 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977c19bfc0 con 0x7f977c107a40 2024-04-25T00:06:46.034 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.129+0000 7f978199f700 1 --2- 172.21.15.107:0/603700200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f977c107a40 0x7f977c19ba80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f976c009fb0 tx=0x7f976c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:46.034 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.130+0000 7f9772ffd700 1 -- 172.21.15.107:0/603700200 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f976c0038b0 con 0x7f977c107a40 2024-04-25T00:06:46.034 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.130+0000 7f9772ffd700 1 -- 172.21.15.107:0/603700200 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f976c011e50 con 0x7f977c107a40 2024-04-25T00:06:46.034 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.130+0000 7f97829a1700 1 -- 172.21.15.107:0/603700200 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977c19c250 con 0x7f977c107a40 2024-04-25T00:06:46.035 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.130+0000 7f9772ffd700 1 -- 172.21.15.107:0/603700200 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f976c019550 con 0x7f977c107a40 2024-04-25T00:06:46.035 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.130+0000 7f97829a1700 1 -- 172.21.15.107:0/603700200 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f977c19ef40 con 0x7f977c107a40 2024-04-25T00:06:46.035 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.130+0000 7f9772ffd700 1 -- 172.21.15.107:0/603700200 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f976c019800 con 0x7f977c107a40 2024-04-25T00:06:46.035 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.131+0000 7f97829a1700 1 -- 172.21.15.107:0/603700200 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f977c063890 con 0x7f977c107a40 2024-04-25T00:06:46.035 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.131+0000 7f9772ffd700 1 --2- 172.21.15.107:0/603700200 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f9768039050 0x7f976803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:46.035 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.131+0000 7f9772ffd700 1 -- 172.21.15.107:0/603700200 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f976c04d250 con 0x7f977c107a40 2024-04-25T00:06:46.035 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.131+0000 7f978119e700 1 --2- 172.21.15.107:0/603700200 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f9768039050 0x7f976803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:46.035 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.131+0000 7f978119e700 1 --2- 172.21.15.107:0/603700200 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f9768039050 0x7f976803b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9778006fd0 tx=0x7f9778006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:46.035 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.136+0000 7f9772ffd700 1 -- 172.21.15.107:0/603700200 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f976c00fda0 con 0x7f977c107a40 2024-04-25T00:06:46.035 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:41.334+0000 7f97829a1700 1 -- 172.21.15.107:0/603700200 --> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi107", "addr": "172.21.15.107", "target": ["mon-mgr", ""]}) -- 0x7f977c104390 con 0x7f9768039050 2024-04-25T00:06:46.035 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:45.394+0000 7f9772ffd700 1 -- 172.21.15.107:0/603700200 <== mgr.14120 v2:172.21.15.107:6800/153169302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f977c104390 con 0x7f9768039050 2024-04-25T00:06:46.035 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:45.397+0000 7f97829a1700 1 -- 172.21.15.107:0/603700200 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f9768039050 msgr2=0x7f976803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:46.035 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:45.397+0000 7f97829a1700 1 --2- 172.21.15.107:0/603700200 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f9768039050 0x7f976803b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9778006fd0 tx=0x7f9778006e40 comp rx=0 tx=0).stop 2024-04-25T00:06:46.035 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:45.397+0000 7f97829a1700 1 -- 172.21.15.107:0/603700200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f977c107a40 msgr2=0x7f977c19ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:46.036 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:45.397+0000 7f97829a1700 1 --2- 172.21.15.107:0/603700200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f977c107a40 0x7f977c19ba80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f976c009fb0 tx=0x7f976c011bf0 comp rx=0 tx=0).stop 2024-04-25T00:06:46.036 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:45.397+0000 7f97829a1700 1 -- 172.21.15.107:0/603700200 shutdown_connections 2024-04-25T00:06:46.036 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:45.397+0000 7f97829a1700 1 --2- 172.21.15.107:0/603700200 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f9768039050 0x7f976803b510 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:46.036 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:45.397+0000 7f97829a1700 1 --2- 172.21.15.107:0/603700200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f977c107a40 0x7f977c19ba80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:46.036 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:45.397+0000 7f97829a1700 1 -- 172.21.15.107:0/603700200 >> 172.21.15.107:0/603700200 conn(0x7f977c103080 msgr2=0x7f977c103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:46.036 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:45.397+0000 7f97829a1700 1 -- 172.21.15.107:0/603700200 shutdown_connections 2024-04-25T00:06:46.036 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:45.397+0000 7f97829a1700 1 -- 172.21.15.107:0/603700200 wait complete. 2024-04-25T00:06:46.036 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying unmanaged mon service... 2024-04-25T00:06:46.623 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:46 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' 2024-04-25T00:06:46.624 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:46 smithi107 ceph-mon[29153]: Added host smithi107 2024-04-25T00:06:46.624 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:46 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:06:47.398 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T00:06:47.398 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.445+0000 7f8f9e4d5700 1 Processor -- start 2024-04-25T00:06:47.398 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.445+0000 7f8f9e4d5700 1 -- start start 2024-04-25T00:06:47.398 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.445+0000 7f8f9e4d5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f98107c60 0x7f8f98108080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:47.398 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.445+0000 7f8f9e4d5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8f981085c0 con 0x7f8f98107c60 2024-04-25T00:06:47.398 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.445+0000 7f8f97fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f98107c60 0x7f8f98108080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:47.398 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.445+0000 7f8f97fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f98107c60 0x7f8f98108080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38274/0 (socket says 172.21.15.107:38274) 2024-04-25T00:06:47.398 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.445+0000 7f8f97fff700 1 -- 172.21.15.107:0/3932398332 learned_addr learned my addr 172.21.15.107:0/3932398332 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:47.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.445+0000 7f8f97fff700 1 -- 172.21.15.107:0/3932398332 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f98108740 con 0x7f8f98107c60 2024-04-25T00:06:47.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.445+0000 7f8f97fff700 1 --2- 172.21.15.107:0/3932398332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f98107c60 0x7f8f98108080 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8f880098a0 tx=0x7f8f88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d774e94e1b31c80a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:47.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.446+0000 7f8f96ffd700 1 -- 172.21.15.107:0/3932398332 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f88004160 con 0x7f8f98107c60 2024-04-25T00:06:47.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.446+0000 7f8f96ffd700 1 -- 172.21.15.107:0/3932398332 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8f88004300 con 0x7f8f98107c60 2024-04-25T00:06:47.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.446+0000 7f8f96ffd700 1 -- 172.21.15.107:0/3932398332 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f88011b50 con 0x7f8f98107c60 2024-04-25T00:06:47.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.446+0000 7f8f9e4d5700 1 -- 172.21.15.107:0/3932398332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f98107c60 msgr2=0x7f8f98108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:47.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.446+0000 7f8f9e4d5700 1 --2- 172.21.15.107:0/3932398332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f98107c60 0x7f8f98108080 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8f880098a0 tx=0x7f8f88009bb0 comp rx=0 tx=0).stop 2024-04-25T00:06:47.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.446+0000 7f8f9e4d5700 1 -- 172.21.15.107:0/3932398332 shutdown_connections 2024-04-25T00:06:47.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.446+0000 7f8f9e4d5700 1 --2- 172.21.15.107:0/3932398332 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f98107c60 0x7f8f98108080 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:47.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.446+0000 7f8f9e4d5700 1 -- 172.21.15.107:0/3932398332 >> 172.21.15.107:0/3932398332 conn(0x7f8f981032a0 msgr2=0x7f8f981056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:47.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.446+0000 7f8f9e4d5700 1 -- 172.21.15.107:0/3932398332 shutdown_connections 2024-04-25T00:06:47.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.446+0000 7f8f9e4d5700 1 -- 172.21.15.107:0/3932398332 wait complete. 2024-04-25T00:06:47.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.447+0000 7f8f9e4d5700 1 Processor -- start 2024-04-25T00:06:47.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.447+0000 7f8f9e4d5700 1 -- start start 2024-04-25T00:06:47.399 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.447+0000 7f8f9e4d5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f98107c60 0x7f8f9819bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:47.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.447+0000 7f8f9e4d5700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9810c0d0 con 0x7f8f98107c60 2024-04-25T00:06:47.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.447+0000 7f8f97fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f98107c60 0x7f8f9819bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:47.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.448+0000 7f8f97fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f98107c60 0x7f8f9819bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38278/0 (socket says 172.21.15.107:38278) 2024-04-25T00:06:47.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.448+0000 7f8f97fff700 1 -- 172.21.15.107:0/823034442 learned_addr learned my addr 172.21.15.107:0/823034442 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:47.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.448+0000 7f8f97fff700 1 -- 172.21.15.107:0/823034442 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9819c0f0 con 0x7f8f98107c60 2024-04-25T00:06:47.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.448+0000 7f8f97fff700 1 --2- 172.21.15.107:0/823034442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f98107c60 0x7f8f9819bbb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8f88011670 tx=0x7f8f88011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:47.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.448+0000 7f8f957fa700 1 -- 172.21.15.107:0/823034442 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f880039c0 con 0x7f8f98107c60 2024-04-25T00:06:47.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.448+0000 7f8f9e4d5700 1 -- 172.21.15.107:0/823034442 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f9819c380 con 0x7f8f98107c60 2024-04-25T00:06:47.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.448+0000 7f8f957fa700 1 -- 172.21.15.107:0/823034442 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8f88011e50 con 0x7f8f98107c60 2024-04-25T00:06:47.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.448+0000 7f8f957fa700 1 -- 172.21.15.107:0/823034442 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f8801a5c0 con 0x7f8f98107c60 2024-04-25T00:06:47.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.448+0000 7f8f9e4d5700 1 -- 172.21.15.107:0/823034442 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f9819f070 con 0x7f8f98107c60 2024-04-25T00:06:47.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.449+0000 7f8f957fa700 1 -- 172.21.15.107:0/823034442 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8f8801a870 con 0x7f8f98107c60 2024-04-25T00:06:47.400 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.449+0000 7f8f9e4d5700 1 -- 172.21.15.107:0/823034442 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f98063890 con 0x7f8f98107c60 2024-04-25T00:06:47.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.449+0000 7f8f957fa700 1 --2- 172.21.15.107:0/823034442 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f8f80039050 0x7f8f8003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:47.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.449+0000 7f8f957fa700 1 -- 172.21.15.107:0/823034442 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8f8804d390 con 0x7f8f98107c60 2024-04-25T00:06:47.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.449+0000 7f8f977fe700 1 --2- 172.21.15.107:0/823034442 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f8f80039050 0x7f8f8003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:47.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.450+0000 7f8f977fe700 1 --2- 172.21.15.107:0/823034442 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f8f80039050 0x7f8f8003b510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c006fd0 tx=0x7f8f8c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:47.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.454+0000 7f8f957fa700 1 -- 172.21.15.107:0/823034442 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8f88019c70 con 0x7f8f98107c60 2024-04-25T00:06:47.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.669+0000 7f8f9e4d5700 1 -- 172.21.15.107:0/823034442 --> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f8f98104480 con 0x7f8f80039050 2024-04-25T00:06:47.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.673+0000 7f8f957fa700 1 -- 172.21.15.107:0/823034442 <== mgr.14120 v2:172.21.15.107:6800/153169302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8f98104480 con 0x7f8f80039050 2024-04-25T00:06:47.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.675+0000 7f8f9e4d5700 1 -- 172.21.15.107:0/823034442 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f8f80039050 msgr2=0x7f8f8003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:47.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.675+0000 7f8f9e4d5700 1 --2- 172.21.15.107:0/823034442 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f8f80039050 0x7f8f8003b510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c006fd0 tx=0x7f8f8c006e40 comp rx=0 tx=0).stop 2024-04-25T00:06:47.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.675+0000 7f8f9e4d5700 1 -- 172.21.15.107:0/823034442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f98107c60 msgr2=0x7f8f9819bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:47.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.676+0000 7f8f9e4d5700 1 --2- 172.21.15.107:0/823034442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f98107c60 0x7f8f9819bbb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8f88011670 tx=0x7f8f88011b60 comp rx=0 tx=0).stop 2024-04-25T00:06:47.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.676+0000 7f8f9e4d5700 1 -- 172.21.15.107:0/823034442 shutdown_connections 2024-04-25T00:06:47.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.676+0000 7f8f9e4d5700 1 --2- 172.21.15.107:0/823034442 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f8f80039050 0x7f8f8003b510 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:47.401 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.676+0000 7f8f9e4d5700 1 --2- 172.21.15.107:0/823034442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f98107c60 0x7f8f9819bbb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:47.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.676+0000 7f8f9e4d5700 1 -- 172.21.15.107:0/823034442 >> 172.21.15.107:0/823034442 conn(0x7f8f981032a0 msgr2=0x7f8f98103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:47.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.676+0000 7f8f9e4d5700 1 -- 172.21.15.107:0/823034442 shutdown_connections 2024-04-25T00:06:47.402 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:46.676+0000 7f8f9e4d5700 1 -- 172.21.15.107:0/823034442 wait complete. 2024-04-25T00:06:47.402 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying unmanaged mgr service... 2024-04-25T00:06:47.952 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:47 smithi107 ceph-mon[29153]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:06:47.952 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:47 smithi107 ceph-mon[29153]: Saving service mon spec with placement count:5 2024-04-25T00:06:47.952 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:47 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' 2024-04-25T00:06:49.428 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:49 smithi107 ceph-mon[29153]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:06:49.428 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:49 smithi107 ceph-mon[29153]: Saving service mgr spec with placement count:2 2024-04-25T00:06:49.428 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:49 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' 2024-04-25T00:06:49.428 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:49 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' 2024-04-25T00:06:49.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T00:06:49.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.853+0000 7f106f2b7700 1 Processor -- start 2024-04-25T00:06:49.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.853+0000 7f106f2b7700 1 -- start start 2024-04-25T00:06:49.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.853+0000 7f106f2b7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1068105f00 0x7f1068106320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:49.436 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.853+0000 7f106f2b7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f10681068f0 con 0x7f1068105f00 2024-04-25T00:06:49.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.854+0000 7f106d053700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1068105f00 0x7f1068106320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:49.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.854+0000 7f106d053700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1068105f00 0x7f1068106320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38288/0 (socket says 172.21.15.107:38288) 2024-04-25T00:06:49.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.854+0000 7f106d053700 1 -- 172.21.15.107:0/1199268890 learned_addr learned my addr 172.21.15.107:0/1199268890 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:49.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.854+0000 7f106d053700 1 -- 172.21.15.107:0/1199268890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1068108e90 con 0x7f1068105f00 2024-04-25T00:06:49.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.854+0000 7f106d053700 1 --2- 172.21.15.107:0/1199268890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1068105f00 0x7f1068106320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f105c0098a0 tx=0x7f105c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef43f0dfedea0850 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:49.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.854+0000 7f105bfff700 1 -- 172.21.15.107:0/1199268890 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f105c004160 con 0x7f1068105f00 2024-04-25T00:06:49.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.854+0000 7f105bfff700 1 -- 172.21.15.107:0/1199268890 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f105c004300 con 0x7f1068105f00 2024-04-25T00:06:49.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.854+0000 7f105bfff700 1 -- 172.21.15.107:0/1199268890 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f105c011b50 con 0x7f1068105f00 2024-04-25T00:06:49.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.855+0000 7f106f2b7700 1 -- 172.21.15.107:0/1199268890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1068105f00 msgr2=0x7f1068106320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:49.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.855+0000 7f106f2b7700 1 --2- 172.21.15.107:0/1199268890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1068105f00 0x7f1068106320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f105c0098a0 tx=0x7f105c009bb0 comp rx=0 tx=0).stop 2024-04-25T00:06:49.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.855+0000 7f106f2b7700 1 -- 172.21.15.107:0/1199268890 shutdown_connections 2024-04-25T00:06:49.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.855+0000 7f106f2b7700 1 --2- 172.21.15.107:0/1199268890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1068105f00 0x7f1068106320 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:49.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.855+0000 7f106f2b7700 1 -- 172.21.15.107:0/1199268890 >> 172.21.15.107:0/1199268890 conn(0x7f1068101810 msgr2=0x7f1068103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:49.437 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.855+0000 7f106f2b7700 1 -- 172.21.15.107:0/1199268890 shutdown_connections 2024-04-25T00:06:49.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.855+0000 7f106f2b7700 1 -- 172.21.15.107:0/1199268890 wait complete. 2024-04-25T00:06:49.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.855+0000 7f106f2b7700 1 Processor -- start 2024-04-25T00:06:49.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.856+0000 7f106f2b7700 1 -- start start 2024-04-25T00:06:49.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.856+0000 7f106f2b7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1068105f00 0x7f106819bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:49.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.856+0000 7f106f2b7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f106810c240 con 0x7f1068105f00 2024-04-25T00:06:49.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.856+0000 7f106d053700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1068105f00 0x7f106819bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:49.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.856+0000 7f106d053700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1068105f00 0x7f106819bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38290/0 (socket says 172.21.15.107:38290) 2024-04-25T00:06:49.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.856+0000 7f106d053700 1 -- 172.21.15.107:0/2633849967 learned_addr learned my addr 172.21.15.107:0/2633849967 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:49.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.856+0000 7f106d053700 1 -- 172.21.15.107:0/2633849967 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106819c140 con 0x7f1068105f00 2024-04-25T00:06:49.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.856+0000 7f106d053700 1 --2- 172.21.15.107:0/2633849967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1068105f00 0x7f106819bc00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f105c011670 tx=0x7f105c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:49.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.856+0000 7f105a7fc700 1 -- 172.21.15.107:0/2633849967 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f105c0039c0 con 0x7f1068105f00 2024-04-25T00:06:49.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.856+0000 7f106f2b7700 1 -- 172.21.15.107:0/2633849967 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f106819c3d0 con 0x7f1068105f00 2024-04-25T00:06:49.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.856+0000 7f105a7fc700 1 -- 172.21.15.107:0/2633849967 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f105c011e50 con 0x7f1068105f00 2024-04-25T00:06:49.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.856+0000 7f105a7fc700 1 -- 172.21.15.107:0/2633849967 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f105c01a5c0 con 0x7f1068105f00 2024-04-25T00:06:49.438 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.856+0000 7f106f2b7700 1 -- 172.21.15.107:0/2633849967 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f106819f0c0 con 0x7f1068105f00 2024-04-25T00:06:49.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.857+0000 7f105a7fc700 1 -- 172.21.15.107:0/2633849967 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f105c024430 con 0x7f1068105f00 2024-04-25T00:06:49.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.857+0000 7f106f2b7700 1 -- 172.21.15.107:0/2633849967 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1068063890 con 0x7f1068105f00 2024-04-25T00:06:49.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.857+0000 7f105a7fc700 1 --2- 172.21.15.107:0/2633849967 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f1054039000 0x7f105403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:49.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.857+0000 7f105a7fc700 1 -- 172.21.15.107:0/2633849967 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f105c04d520 con 0x7f1068105f00 2024-04-25T00:06:49.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.858+0000 7f106c852700 1 --2- 172.21.15.107:0/2633849967 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f1054039000 0x7f105403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:49.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.858+0000 7f106c852700 1 --2- 172.21.15.107:0/2633849967 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f1054039000 0x7f105403b4c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1064006fd0 tx=0x7f1064006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:49.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:47.862+0000 7f105a7fc700 1 -- 172.21.15.107:0/2633849967 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f105c003b60 con 0x7f1068105f00 2024-04-25T00:06:49.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:48.073+0000 7f106f2b7700 1 -- 172.21.15.107:0/2633849967 --> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f1068102ae0 con 0x7f1054039000 2024-04-25T00:06:49.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:48.078+0000 7f105a7fc700 1 -- 172.21.15.107:0/2633849967 <== mgr.14120 v2:172.21.15.107:6800/153169302 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f1068102ae0 con 0x7f1054039000 2024-04-25T00:06:49.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:48.080+0000 7f106f2b7700 1 -- 172.21.15.107:0/2633849967 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f1054039000 msgr2=0x7f105403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:49.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:48.080+0000 7f106f2b7700 1 --2- 172.21.15.107:0/2633849967 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f1054039000 0x7f105403b4c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1064006fd0 tx=0x7f1064006e40 comp rx=0 tx=0).stop 2024-04-25T00:06:49.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:48.080+0000 7f106f2b7700 1 -- 172.21.15.107:0/2633849967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1068105f00 msgr2=0x7f106819bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:49.439 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:48.080+0000 7f106f2b7700 1 --2- 172.21.15.107:0/2633849967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1068105f00 0x7f106819bc00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f105c011670 tx=0x7f105c011b60 comp rx=0 tx=0).stop 2024-04-25T00:06:49.440 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:48.081+0000 7f106f2b7700 1 -- 172.21.15.107:0/2633849967 shutdown_connections 2024-04-25T00:06:49.440 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:48.081+0000 7f106f2b7700 1 --2- 172.21.15.107:0/2633849967 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f1054039000 0x7f105403b4c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:49.440 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:48.081+0000 7f106f2b7700 1 --2- 172.21.15.107:0/2633849967 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1068105f00 0x7f106819bc00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:49.440 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:48.081+0000 7f106f2b7700 1 -- 172.21.15.107:0/2633849967 >> 172.21.15.107:0/2633849967 conn(0x7f1068101810 msgr2=0x7f10681023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:49.440 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:48.081+0000 7f106f2b7700 1 -- 172.21.15.107:0/2633849967 shutdown_connections 2024-04-25T00:06:49.440 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:48.081+0000 7f106f2b7700 1 -- 172.21.15.107:0/2633849967 wait complete. 2024-04-25T00:06:50.759 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.887+0000 7fdb8aa06700 1 Processor -- start 2024-04-25T00:06:50.759 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.887+0000 7fdb8aa06700 1 -- start start 2024-04-25T00:06:50.759 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.887+0000 7fdb8aa06700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb84108770 0x7fdb8410ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:50.760 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.887+0000 7fdb8aa06700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8407bdd0 con 0x7fdb84108770 2024-04-25T00:06:50.760 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.887+0000 7fdb83fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb84108770 0x7fdb8410ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:50.760 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.887+0000 7fdb83fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb84108770 0x7fdb8410ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37726/0 (socket says 172.21.15.107:37726) 2024-04-25T00:06:50.760 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.887+0000 7fdb83fff700 1 -- 172.21.15.107:0/1887062628 learned_addr learned my addr 172.21.15.107:0/1887062628 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:50.760 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.887+0000 7fdb83fff700 1 -- 172.21.15.107:0/1887062628 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8410b0a0 con 0x7fdb84108770 2024-04-25T00:06:50.760 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.887+0000 7fdb83fff700 1 --2- 172.21.15.107:0/1887062628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb84108770 0x7fdb8410ab60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdb740098a0 tx=0x7fdb74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe6cb2baab1b0c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:50.760 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.888+0000 7fdb82ffd700 1 -- 172.21.15.107:0/1887062628 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb74004160 con 0x7fdb84108770 2024-04-25T00:06:50.760 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.888+0000 7fdb82ffd700 1 -- 172.21.15.107:0/1887062628 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb74004300 con 0x7fdb84108770 2024-04-25T00:06:50.760 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.888+0000 7fdb82ffd700 1 -- 172.21.15.107:0/1887062628 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb74011b20 con 0x7fdb84108770 2024-04-25T00:06:50.760 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.888+0000 7fdb8aa06700 1 -- 172.21.15.107:0/1887062628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb84108770 msgr2=0x7fdb8410ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:50.760 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.888+0000 7fdb8aa06700 1 --2- 172.21.15.107:0/1887062628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb84108770 0x7fdb8410ab60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdb740098a0 tx=0x7fdb74009bb0 comp rx=0 tx=0).stop 2024-04-25T00:06:50.760 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.888+0000 7fdb8aa06700 1 -- 172.21.15.107:0/1887062628 shutdown_connections 2024-04-25T00:06:50.760 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.888+0000 7fdb8aa06700 1 --2- 172.21.15.107:0/1887062628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb84108770 0x7fdb8410ab60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:50.760 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.888+0000 7fdb8aa06700 1 -- 172.21.15.107:0/1887062628 >> 172.21.15.107:0/1887062628 conn(0x7fdb841020a0 msgr2=0x7fdb84104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:50.761 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.888+0000 7fdb8aa06700 1 -- 172.21.15.107:0/1887062628 shutdown_connections 2024-04-25T00:06:50.761 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.888+0000 7fdb8aa06700 1 -- 172.21.15.107:0/1887062628 wait complete. 2024-04-25T00:06:50.761 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.889+0000 7fdb8aa06700 1 Processor -- start 2024-04-25T00:06:50.761 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.889+0000 7fdb8aa06700 1 -- start start 2024-04-25T00:06:50.761 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.889+0000 7fdb8aa06700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb84108770 0x7fdb8419bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:50.761 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.889+0000 7fdb8aa06700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8410c040 con 0x7fdb84108770 2024-04-25T00:06:50.761 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.889+0000 7fdb83fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb84108770 0x7fdb8419bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:50.761 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.889+0000 7fdb83fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb84108770 0x7fdb8419bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37742/0 (socket says 172.21.15.107:37742) 2024-04-25T00:06:50.761 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.889+0000 7fdb83fff700 1 -- 172.21.15.107:0/3694718958 learned_addr learned my addr 172.21.15.107:0/3694718958 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:50.761 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.890+0000 7fdb83fff700 1 -- 172.21.15.107:0/3694718958 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8419c0d0 con 0x7fdb84108770 2024-04-25T00:06:50.762 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.890+0000 7fdb83fff700 1 --2- 172.21.15.107:0/3694718958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb84108770 0x7fdb8419bb90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdb740117e0 tx=0x7fdb74011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:50.762 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.890+0000 7fdb817fa700 1 -- 172.21.15.107:0/3694718958 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb74003ba0 con 0x7fdb84108770 2024-04-25T00:06:50.762 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.890+0000 7fdb8aa06700 1 -- 172.21.15.107:0/3694718958 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb8419c360 con 0x7fdb84108770 2024-04-25T00:06:50.762 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.890+0000 7fdb817fa700 1 -- 172.21.15.107:0/3694718958 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb7401a910 con 0x7fdb84108770 2024-04-25T00:06:50.762 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.890+0000 7fdb817fa700 1 -- 172.21.15.107:0/3694718958 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb740248f0 con 0x7fdb84108770 2024-04-25T00:06:50.762 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.890+0000 7fdb8aa06700 1 -- 172.21.15.107:0/3694718958 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb8419c7c0 con 0x7fdb84108770 2024-04-25T00:06:50.762 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.891+0000 7fdb817fa700 1 -- 172.21.15.107:0/3694718958 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fdb74022020 con 0x7fdb84108770 2024-04-25T00:06:50.762 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.891+0000 7fdb8aa06700 1 -- 172.21.15.107:0/3694718958 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb84063890 con 0x7fdb84108770 2024-04-25T00:06:50.762 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.891+0000 7fdb817fa700 1 --2- 172.21.15.107:0/3694718958 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7fdb6c039050 0x7fdb6c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:50.762 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.891+0000 7fdb817fa700 1 -- 172.21.15.107:0/3694718958 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdb7404d8f0 con 0x7fdb84108770 2024-04-25T00:06:50.762 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.892+0000 7fdb837fe700 1 --2- 172.21.15.107:0/3694718958 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7fdb6c039050 0x7fdb6c03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:50.762 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.892+0000 7fdb837fe700 1 --2- 172.21.15.107:0/3694718958 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7fdb6c039050 0x7fdb6c03b510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdb78006fd0 tx=0x7fdb78006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:50.762 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:49.896+0000 7fdb817fa700 1 -- 172.21.15.107:0/3694718958 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdb74020c10 con 0x7fdb84108770 2024-04-25T00:06:50.762 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:50.099+0000 7fdb8aa06700 1 -- 172.21.15.107:0/3694718958 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fdb8419f110 con 0x7fdb84108770 2024-04-25T00:06:50.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:50.102+0000 7fdb817fa700 1 -- 172.21.15.107:0/3694718958 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fdb7400b5a0 con 0x7fdb84108770 2024-04-25T00:06:50.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:50.104+0000 7fdb8aa06700 1 -- 172.21.15.107:0/3694718958 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7fdb6c039050 msgr2=0x7fdb6c03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:50.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:50.104+0000 7fdb8aa06700 1 --2- 172.21.15.107:0/3694718958 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7fdb6c039050 0x7fdb6c03b510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdb78006fd0 tx=0x7fdb78006e40 comp rx=0 tx=0).stop 2024-04-25T00:06:50.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:50.104+0000 7fdb8aa06700 1 -- 172.21.15.107:0/3694718958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb84108770 msgr2=0x7fdb8419bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:50.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:50.104+0000 7fdb8aa06700 1 --2- 172.21.15.107:0/3694718958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb84108770 0x7fdb8419bb90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdb740117e0 tx=0x7fdb74011ba0 comp rx=0 tx=0).stop 2024-04-25T00:06:50.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:50.104+0000 7fdb8aa06700 1 -- 172.21.15.107:0/3694718958 shutdown_connections 2024-04-25T00:06:50.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:50.104+0000 7fdb8aa06700 1 --2- 172.21.15.107:0/3694718958 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7fdb6c039050 0x7fdb6c03b510 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:50.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:50.104+0000 7fdb8aa06700 1 --2- 172.21.15.107:0/3694718958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb84108770 0x7fdb8419bb90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:50.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:50.104+0000 7fdb8aa06700 1 -- 172.21.15.107:0/3694718958 >> 172.21.15.107:0/3694718958 conn(0x7fdb841020a0 msgr2=0x7fdb84102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:50.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:50.104+0000 7fdb8aa06700 1 -- 172.21.15.107:0/3694718958 shutdown_connections 2024-04-25T00:06:50.763 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:50.104+0000 7fdb8aa06700 1 -- 172.21.15.107:0/3694718958 wait complete. 2024-04-25T00:06:51.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:50 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' 2024-04-25T00:06:51.006 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:50 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3694718958' entity='client.admin' 2024-04-25T00:06:52.173 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.206+0000 7ff3384fa700 1 Processor -- start 2024-04-25T00:06:52.173 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.206+0000 7ff3384fa700 1 -- start start 2024-04-25T00:06:52.173 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.206+0000 7ff3384fa700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff330105ed0 0x7ff3301062f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:52.173 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.206+0000 7ff3384fa700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff3301068c0 con 0x7ff330105ed0 2024-04-25T00:06:52.174 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.206+0000 7ff336296700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff330105ed0 0x7ff3301062f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:52.174 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.207+0000 7ff336296700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff330105ed0 0x7ff3301062f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37748/0 (socket says 172.21.15.107:37748) 2024-04-25T00:06:52.174 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.207+0000 7ff336296700 1 -- 172.21.15.107:0/3784263540 learned_addr learned my addr 172.21.15.107:0/3784263540 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:52.174 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.207+0000 7ff336296700 1 -- 172.21.15.107:0/3784263540 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff330106a40 con 0x7ff330105ed0 2024-04-25T00:06:52.174 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.207+0000 7ff336296700 1 --2- 172.21.15.107:0/3784263540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff330105ed0 0x7ff3301062f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0098a0 tx=0x7ff32c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a56ffc8c20e944 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:52.174 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.207+0000 7ff335294700 1 -- 172.21.15.107:0/3784263540 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff32c004160 con 0x7ff330105ed0 2024-04-25T00:06:52.174 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.207+0000 7ff335294700 1 -- 172.21.15.107:0/3784263540 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff32c004300 con 0x7ff330105ed0 2024-04-25T00:06:52.174 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.207+0000 7ff335294700 1 -- 172.21.15.107:0/3784263540 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff32c011b20 con 0x7ff330105ed0 2024-04-25T00:06:52.174 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.208+0000 7ff3384fa700 1 -- 172.21.15.107:0/3784263540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff330105ed0 msgr2=0x7ff3301062f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:52.174 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.208+0000 7ff3384fa700 1 --2- 172.21.15.107:0/3784263540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff330105ed0 0x7ff3301062f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0098a0 tx=0x7ff32c009bb0 comp rx=0 tx=0).stop 2024-04-25T00:06:52.174 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.208+0000 7ff3384fa700 1 -- 172.21.15.107:0/3784263540 shutdown_connections 2024-04-25T00:06:52.174 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.208+0000 7ff3384fa700 1 --2- 172.21.15.107:0/3784263540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff330105ed0 0x7ff3301062f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:52.174 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.208+0000 7ff3384fa700 1 -- 172.21.15.107:0/3784263540 >> 172.21.15.107:0/3784263540 conn(0x7ff330101810 msgr2=0x7ff330103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:52.175 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.208+0000 7ff3384fa700 1 -- 172.21.15.107:0/3784263540 shutdown_connections 2024-04-25T00:06:52.175 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.208+0000 7ff3384fa700 1 -- 172.21.15.107:0/3784263540 wait complete. 2024-04-25T00:06:52.175 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.208+0000 7ff3384fa700 1 Processor -- start 2024-04-25T00:06:52.175 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.208+0000 7ff3384fa700 1 -- start start 2024-04-25T00:06:52.175 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.209+0000 7ff3384fa700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff330105ed0 0x7ff33019bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:52.175 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.209+0000 7ff3384fa700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff33010c180 con 0x7ff330105ed0 2024-04-25T00:06:52.175 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.209+0000 7ff336296700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff330105ed0 0x7ff33019bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:52.175 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.209+0000 7ff336296700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff330105ed0 0x7ff33019bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37754/0 (socket says 172.21.15.107:37754) 2024-04-25T00:06:52.175 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.209+0000 7ff336296700 1 -- 172.21.15.107:0/2316140966 learned_addr learned my addr 172.21.15.107:0/2316140966 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:52.175 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.209+0000 7ff336296700 1 -- 172.21.15.107:0/2316140966 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33019c180 con 0x7ff330105ed0 2024-04-25T00:06:52.175 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.209+0000 7ff336296700 1 --2- 172.21.15.107:0/2316140966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff330105ed0 0x7ff33019bc40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0117e0 tx=0x7ff32c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:52.175 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.209+0000 7ff3277fe700 1 -- 172.21.15.107:0/2316140966 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff32c003bf0 con 0x7ff330105ed0 2024-04-25T00:06:52.175 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.209+0000 7ff3277fe700 1 -- 172.21.15.107:0/2316140966 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff32c01a910 con 0x7ff330105ed0 2024-04-25T00:06:52.175 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.209+0000 7ff3277fe700 1 -- 172.21.15.107:0/2316140966 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff32c0249a0 con 0x7ff330105ed0 2024-04-25T00:06:52.175 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.209+0000 7ff3384fa700 1 -- 172.21.15.107:0/2316140966 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff33019c410 con 0x7ff330105ed0 2024-04-25T00:06:52.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.209+0000 7ff3384fa700 1 -- 172.21.15.107:0/2316140966 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff33019c870 con 0x7ff330105ed0 2024-04-25T00:06:52.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.210+0000 7ff3277fe700 1 -- 172.21.15.107:0/2316140966 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff32c022020 con 0x7ff330105ed0 2024-04-25T00:06:52.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.210+0000 7ff3384fa700 1 -- 172.21.15.107:0/2316140966 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff330063890 con 0x7ff330105ed0 2024-04-25T00:06:52.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.210+0000 7ff3277fe700 1 --2- 172.21.15.107:0/2316140966 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7ff31c039050 0x7ff31c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:52.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.210+0000 7ff3277fe700 1 -- 172.21.15.107:0/2316140966 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff32c04d3d0 con 0x7ff330105ed0 2024-04-25T00:06:52.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.210+0000 7ff335a95700 1 --2- 172.21.15.107:0/2316140966 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7ff31c039050 0x7ff31c03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:52.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.211+0000 7ff335a95700 1 --2- 172.21.15.107:0/2316140966 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7ff31c039050 0x7ff31c03b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff320006fd0 tx=0x7ff320006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:52.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.215+0000 7ff3277fe700 1 -- 172.21.15.107:0/2316140966 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff32c011e20 con 0x7ff330105ed0 2024-04-25T00:06:52.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.433+0000 7ff3384fa700 1 -- 172.21.15.107:0/2316140966 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7ff33019f100 con 0x7ff330105ed0 2024-04-25T00:06:52.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.435+0000 7ff3277fe700 1 -- 172.21.15.107:0/2316140966 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7ff32c02c350 con 0x7ff330105ed0 2024-04-25T00:06:52.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.437+0000 7ff3384fa700 1 -- 172.21.15.107:0/2316140966 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7ff31c039050 msgr2=0x7ff31c03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:52.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.437+0000 7ff3384fa700 1 --2- 172.21.15.107:0/2316140966 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7ff31c039050 0x7ff31c03b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff320006fd0 tx=0x7ff320006e40 comp rx=0 tx=0).stop 2024-04-25T00:06:52.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.437+0000 7ff3384fa700 1 -- 172.21.15.107:0/2316140966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff330105ed0 msgr2=0x7ff33019bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:52.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.437+0000 7ff3384fa700 1 --2- 172.21.15.107:0/2316140966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff330105ed0 0x7ff33019bc40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff32c0117e0 tx=0x7ff32c011c10 comp rx=0 tx=0).stop 2024-04-25T00:06:52.177 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.437+0000 7ff3384fa700 1 -- 172.21.15.107:0/2316140966 shutdown_connections 2024-04-25T00:06:52.177 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.437+0000 7ff3384fa700 1 --2- 172.21.15.107:0/2316140966 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7ff31c039050 0x7ff31c03b510 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:52.177 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.437+0000 7ff3384fa700 1 --2- 172.21.15.107:0/2316140966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff330105ed0 0x7ff33019bc40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:52.177 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.437+0000 7ff3384fa700 1 -- 172.21.15.107:0/2316140966 >> 172.21.15.107:0/2316140966 conn(0x7ff330101810 msgr2=0x7ff330102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:52.177 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.438+0000 7ff3384fa700 1 -- 172.21.15.107:0/2316140966 shutdown_connections 2024-04-25T00:06:52.177 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:51.438+0000 7ff3384fa700 1 -- 172.21.15.107:0/2316140966 wait complete. 2024-04-25T00:06:52.177 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling the dashboard module... 2024-04-25T00:06:52.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:52 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2316140966' entity='client.admin' 2024-04-25T00:06:54.202 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:53 smithi107 ceph-mon[29153]: from='mgr.14120 172.21.15.107:0/3931718172' entity='mgr.x' 2024-04-25T00:06:54.202 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:53 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2272494924' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T00:06:54.202 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:53 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: ignoring --setuser ceph since I am not root 2024-04-25T00:06:54.202 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:53 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: ignoring --setgroup ceph since I am not root 2024-04-25T00:06:54.202 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:54 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:54.119+0000 7eff189bb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:06:54.484 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:54 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:54.307+0000 7eff189bb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:06:54.484 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:54 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:54.409+0000 7eff189bb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:06:54.511 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.842+0000 7f1a99a62700 1 Processor -- start 2024-04-25T00:06:54.511 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.842+0000 7f1a99a62700 1 -- start start 2024-04-25T00:06:54.511 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.842+0000 7f1a99a62700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a94108770 0x7f1a9410ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:54.511 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.842+0000 7f1a99a62700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9407bdd0 con 0x7f1a94108770 2024-04-25T00:06:54.511 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.843+0000 7f1a937fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a94108770 0x7f1a9410ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:54.511 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.843+0000 7f1a937fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a94108770 0x7f1a9410ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37760/0 (socket says 172.21.15.107:37760) 2024-04-25T00:06:54.511 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.843+0000 7f1a937fe700 1 -- 172.21.15.107:0/763317962 learned_addr learned my addr 172.21.15.107:0/763317962 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:54.511 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.843+0000 7f1a937fe700 1 -- 172.21.15.107:0/763317962 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a9410b0a0 con 0x7f1a94108770 2024-04-25T00:06:54.511 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.843+0000 7f1a937fe700 1 --2- 172.21.15.107:0/763317962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a94108770 0x7f1a9410ab60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1a840098a0 tx=0x7f1a84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9075e4a71617ecff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:54.511 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.843+0000 7f1a927fc700 1 -- 172.21.15.107:0/763317962 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a84004160 con 0x7f1a94108770 2024-04-25T00:06:54.511 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.843+0000 7f1a927fc700 1 -- 172.21.15.107:0/763317962 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a84004300 con 0x7f1a94108770 2024-04-25T00:06:54.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.843+0000 7f1a927fc700 1 -- 172.21.15.107:0/763317962 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a84011b20 con 0x7f1a94108770 2024-04-25T00:06:54.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.844+0000 7f1a99a62700 1 -- 172.21.15.107:0/763317962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a94108770 msgr2=0x7f1a9410ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:54.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.844+0000 7f1a99a62700 1 --2- 172.21.15.107:0/763317962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a94108770 0x7f1a9410ab60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1a840098a0 tx=0x7f1a84009bb0 comp rx=0 tx=0).stop 2024-04-25T00:06:54.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.844+0000 7f1a99a62700 1 -- 172.21.15.107:0/763317962 shutdown_connections 2024-04-25T00:06:54.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.844+0000 7f1a99a62700 1 --2- 172.21.15.107:0/763317962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a94108770 0x7f1a9410ab60 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:54.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.844+0000 7f1a99a62700 1 -- 172.21.15.107:0/763317962 >> 172.21.15.107:0/763317962 conn(0x7f1a941020a0 msgr2=0x7f1a94104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:54.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.844+0000 7f1a99a62700 1 -- 172.21.15.107:0/763317962 shutdown_connections 2024-04-25T00:06:54.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.844+0000 7f1a99a62700 1 -- 172.21.15.107:0/763317962 wait complete. 2024-04-25T00:06:54.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.845+0000 7f1a99a62700 1 Processor -- start 2024-04-25T00:06:54.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.845+0000 7f1a99a62700 1 -- start start 2024-04-25T00:06:54.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.845+0000 7f1a99a62700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a94108770 0x7f1a9419bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:54.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.845+0000 7f1a99a62700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9410c040 con 0x7f1a94108770 2024-04-25T00:06:54.512 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.845+0000 7f1a937fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a94108770 0x7f1a9419bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:54.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.845+0000 7f1a937fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a94108770 0x7f1a9419bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37772/0 (socket says 172.21.15.107:37772) 2024-04-25T00:06:54.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.845+0000 7f1a937fe700 1 -- 172.21.15.107:0/2272494924 learned_addr learned my addr 172.21.15.107:0/2272494924 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:54.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.845+0000 7f1a937fe700 1 -- 172.21.15.107:0/2272494924 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a9419c0d0 con 0x7f1a94108770 2024-04-25T00:06:54.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.845+0000 7f1a937fe700 1 --2- 172.21.15.107:0/2272494924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a94108770 0x7f1a9419bb90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1a840117e0 tx=0x7f1a84011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:54.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.845+0000 7f1a90ff9700 1 -- 172.21.15.107:0/2272494924 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a84003ba0 con 0x7f1a94108770 2024-04-25T00:06:54.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.846+0000 7f1a99a62700 1 -- 172.21.15.107:0/2272494924 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a9419c360 con 0x7f1a94108770 2024-04-25T00:06:54.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.846+0000 7f1a90ff9700 1 -- 172.21.15.107:0/2272494924 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a8401a910 con 0x7f1a94108770 2024-04-25T00:06:54.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.846+0000 7f1a90ff9700 1 -- 172.21.15.107:0/2272494924 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a840248f0 con 0x7f1a94108770 2024-04-25T00:06:54.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.846+0000 7f1a99a62700 1 -- 172.21.15.107:0/2272494924 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a9419c7c0 con 0x7f1a94108770 2024-04-25T00:06:54.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.846+0000 7f1a90ff9700 1 -- 172.21.15.107:0/2272494924 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1a84022020 con 0x7f1a94108770 2024-04-25T00:06:54.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.847+0000 7f1a99a62700 1 -- 172.21.15.107:0/2272494924 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a94063890 con 0x7f1a94108770 2024-04-25T00:06:54.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.847+0000 7f1a90ff9700 1 --2- 172.21.15.107:0/2272494924 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f1a800390a0 0x7f1a8003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:54.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.847+0000 7f1a90ff9700 1 -- 172.21.15.107:0/2272494924 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1a8401aac0 con 0x7f1a94108770 2024-04-25T00:06:54.513 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.847+0000 7f1a92ffd700 1 --2- 172.21.15.107:0/2272494924 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f1a800390a0 0x7f1a8003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:54.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.847+0000 7f1a92ffd700 1 --2- 172.21.15.107:0/2272494924 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f1a800390a0 0x7f1a8003b560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c006fd0 tx=0x7f1a7c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:54.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:52.852+0000 7f1a90ff9700 1 -- 172.21.15.107:0/2272494924 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1a84020c10 con 0x7f1a94108770 2024-04-25T00:06:54.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:53.119+0000 7f1a99a62700 1 -- 172.21.15.107:0/2272494924 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f1a9419f190 con 0x7f1a94108770 2024-04-25T00:06:54.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:53.940+0000 7f1a90ff9700 1 -- 172.21.15.107:0/2272494924 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f1a8400b5a0 con 0x7f1a94108770 2024-04-25T00:06:54.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:53.942+0000 7f1a90ff9700 1 -- 172.21.15.107:0/2272494924 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f1a8402dc90 con 0x7f1a94108770 2024-04-25T00:06:54.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:53.943+0000 7f1a99a62700 1 -- 172.21.15.107:0/2272494924 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f1a800390a0 msgr2=0x7f1a8003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:54.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:53.943+0000 7f1a99a62700 1 --2- 172.21.15.107:0/2272494924 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f1a800390a0 0x7f1a8003b560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c006fd0 tx=0x7f1a7c006e40 comp rx=0 tx=0).stop 2024-04-25T00:06:54.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:53.943+0000 7f1a99a62700 1 -- 172.21.15.107:0/2272494924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a94108770 msgr2=0x7f1a9419bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:54.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:53.943+0000 7f1a99a62700 1 --2- 172.21.15.107:0/2272494924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a94108770 0x7f1a9419bb90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1a840117e0 tx=0x7f1a84011ba0 comp rx=0 tx=0).stop 2024-04-25T00:06:54.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:53.943+0000 7f1a99a62700 1 -- 172.21.15.107:0/2272494924 shutdown_connections 2024-04-25T00:06:54.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:53.944+0000 7f1a99a62700 1 --2- 172.21.15.107:0/2272494924 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f1a800390a0 0x7f1a8003b560 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:54.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:53.944+0000 7f1a99a62700 1 --2- 172.21.15.107:0/2272494924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a94108770 0x7f1a9419bb90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:54.514 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:53.944+0000 7f1a99a62700 1 -- 172.21.15.107:0/2272494924 >> 172.21.15.107:0/2272494924 conn(0x7f1a941020a0 msgr2=0x7f1a94102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:54.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:53.944+0000 7f1a99a62700 1 -- 172.21.15.107:0/2272494924 shutdown_connections 2024-04-25T00:06:54.515 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:53.944+0000 7f1a99a62700 1 -- 172.21.15.107:0/2272494924 wait complete. 2024-04-25T00:06:54.815 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:54 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:54.728+0000 7eff189bb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:06:54.815 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:54 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:54.815+0000 7eff189bb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:06:55.149 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:54 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2272494924' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T00:06:55.149 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:54 smithi107 ceph-mon[29153]: mgrmap e9: x(active, since 19s) 2024-04-25T00:06:55.811 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:06:55.811 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T00:06:55.811 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T00:06:55.811 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T00:06:55.811 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T00:06:55.811 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:06:55.811 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.944+0000 7fadf553f700 1 Processor -- start 2024-04-25T00:06:55.811 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.944+0000 7fadf553f700 1 -- start start 2024-04-25T00:06:55.811 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.944+0000 7fadf553f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadf0105f30 0x7fadf0106350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:55.811 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.944+0000 7fadf553f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fadf0106920 con 0x7fadf0105f30 2024-04-25T00:06:55.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.944+0000 7fadeeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadf0105f30 0x7fadf0106350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:55.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.944+0000 7fadeeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadf0105f30 0x7fadf0106350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37794/0 (socket says 172.21.15.107:37794) 2024-04-25T00:06:55.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.944+0000 7fadeeffd700 1 -- 172.21.15.107:0/3000245639 learned_addr learned my addr 172.21.15.107:0/3000245639 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:55.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.944+0000 7fadeeffd700 1 -- 172.21.15.107:0/3000245639 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf0108f60 con 0x7fadf0105f30 2024-04-25T00:06:55.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.944+0000 7fadeeffd700 1 --2- 172.21.15.107:0/3000245639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadf0105f30 0x7fadf0106350 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fade00098a0 tx=0x7fade0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132efd549737cd41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:55.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.945+0000 7fadedffb700 1 -- 172.21.15.107:0/3000245639 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fade0004160 con 0x7fadf0105f30 2024-04-25T00:06:55.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.945+0000 7fadedffb700 1 -- 172.21.15.107:0/3000245639 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fade0004300 con 0x7fadf0105f30 2024-04-25T00:06:55.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.945+0000 7fadedffb700 1 -- 172.21.15.107:0/3000245639 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fade0011b20 con 0x7fadf0105f30 2024-04-25T00:06:55.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.945+0000 7fadf553f700 1 -- 172.21.15.107:0/3000245639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadf0105f30 msgr2=0x7fadf0106350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:55.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.945+0000 7fadf553f700 1 --2- 172.21.15.107:0/3000245639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadf0105f30 0x7fadf0106350 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fade00098a0 tx=0x7fade0009bb0 comp rx=0 tx=0).stop 2024-04-25T00:06:55.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.945+0000 7fadf553f700 1 -- 172.21.15.107:0/3000245639 shutdown_connections 2024-04-25T00:06:55.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.945+0000 7fadf553f700 1 --2- 172.21.15.107:0/3000245639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadf0105f30 0x7fadf0106350 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:55.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.945+0000 7fadf553f700 1 -- 172.21.15.107:0/3000245639 >> 172.21.15.107:0/3000245639 conn(0x7fadf01018a0 msgr2=0x7fadf0103cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:55.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.945+0000 7fadf553f700 1 -- 172.21.15.107:0/3000245639 shutdown_connections 2024-04-25T00:06:55.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.945+0000 7fadf553f700 1 -- 172.21.15.107:0/3000245639 wait complete. 2024-04-25T00:06:55.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.946+0000 7fadf553f700 1 Processor -- start 2024-04-25T00:06:55.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.946+0000 7fadf553f700 1 -- start start 2024-04-25T00:06:55.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.946+0000 7fadf553f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadf0105f30 0x7fadf019bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:55.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.946+0000 7fadf553f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fadf010c190 con 0x7fadf0105f30 2024-04-25T00:06:55.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.946+0000 7fadeeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadf0105f30 0x7fadf019bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:06:55.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.946+0000 7fadeeffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadf0105f30 0x7fadf019bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37806/0 (socket says 172.21.15.107:37806) 2024-04-25T00:06:55.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.946+0000 7fadeeffd700 1 -- 172.21.15.107:0/1499839215 learned_addr learned my addr 172.21.15.107:0/1499839215 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:06:55.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.947+0000 7fadeeffd700 1 -- 172.21.15.107:0/1499839215 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf019c120 con 0x7fadf0105f30 2024-04-25T00:06:55.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.947+0000 7fadeeffd700 1 --2- 172.21.15.107:0/1499839215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadf0105f30 0x7fadf019bbe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fade00116a0 tx=0x7fade0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:06:55.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.947+0000 7fadd7fff700 1 -- 172.21.15.107:0/1499839215 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fade0003a40 con 0x7fadf0105f30 2024-04-25T00:06:55.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.947+0000 7fadd7fff700 1 -- 172.21.15.107:0/1499839215 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fade0003be0 con 0x7fadf0105f30 2024-04-25T00:06:55.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.947+0000 7fadf553f700 1 -- 172.21.15.107:0/1499839215 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf019c3b0 con 0x7fadf0105f30 2024-04-25T00:06:55.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.947+0000 7fadd7fff700 1 -- 172.21.15.107:0/1499839215 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fade001a720 con 0x7fadf0105f30 2024-04-25T00:06:55.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.947+0000 7fadf553f700 1 -- 172.21.15.107:0/1499839215 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf019c810 con 0x7fadf0105f30 2024-04-25T00:06:55.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.948+0000 7fadf553f700 1 -- 172.21.15.107:0/1499839215 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadf0063890 con 0x7fadf0105f30 2024-04-25T00:06:55.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.948+0000 7fadd7fff700 1 -- 172.21.15.107:0/1499839215 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fade001a9d0 con 0x7fadf0105f30 2024-04-25T00:06:55.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.948+0000 7fadd7fff700 1 --2- 172.21.15.107:0/1499839215 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7fadd80390b0 0x7fadd803b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:06:55.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.948+0000 7fadd7fff700 1 -- 172.21.15.107:0/1499839215 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fade004d620 con 0x7fadf0105f30 2024-04-25T00:06:55.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.948+0000 7fadee7fc700 1 -- 172.21.15.107:0/1499839215 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7fadd80390b0 msgr2=0x7fadd803b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/153169302 2024-04-25T00:06:55.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.948+0000 7fadee7fc700 1 --2- 172.21.15.107:0/1499839215 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7fadd80390b0 0x7fadd803b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:06:55.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:54.953+0000 7fadd7fff700 1 -- 172.21.15.107:0/1499839215 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fade0011e20 con 0x7fadf0105f30 2024-04-25T00:06:55.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:55.149+0000 7fadee7fc700 1 -- 172.21.15.107:0/1499839215 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7fadd80390b0 msgr2=0x7fadd803b570 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/153169302 2024-04-25T00:06:55.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:55.149+0000 7fadee7fc700 1 --2- 172.21.15.107:0/1499839215 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7fadd80390b0 0x7fadd803b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T00:06:55.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:55.238+0000 7fadf553f700 1 -- 172.21.15.107:0/1499839215 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fadf019f630 con 0x7fadf0105f30 2024-04-25T00:06:55.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:55.239+0000 7fadd7fff700 1 -- 172.21.15.107:0/1499839215 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fade001adc0 con 0x7fadf0105f30 2024-04-25T00:06:55.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:55.242+0000 7fadf553f700 1 -- 172.21.15.107:0/1499839215 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7fadd80390b0 msgr2=0x7fadd803b570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:06:55.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:55.242+0000 7fadf553f700 1 --2- 172.21.15.107:0/1499839215 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7fadd80390b0 0x7fadd803b570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:55.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:55.242+0000 7fadf553f700 1 -- 172.21.15.107:0/1499839215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadf0105f30 msgr2=0x7fadf019bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:06:55.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:55.242+0000 7fadf553f700 1 --2- 172.21.15.107:0/1499839215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadf0105f30 0x7fadf019bbe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fade00116a0 tx=0x7fade0011ba0 comp rx=0 tx=0).stop 2024-04-25T00:06:55.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:55.242+0000 7fadf553f700 1 -- 172.21.15.107:0/1499839215 shutdown_connections 2024-04-25T00:06:55.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:55.242+0000 7fadf553f700 1 --2- 172.21.15.107:0/1499839215 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7fadd80390b0 0x7fadd803b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:55.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:55.242+0000 7fadf553f700 1 --2- 172.21.15.107:0/1499839215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadf0105f30 0x7fadf019bbe0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:06:55.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:55.242+0000 7fadf553f700 1 -- 172.21.15.107:0/1499839215 >> 172.21.15.107:0/1499839215 conn(0x7fadf01018a0 msgr2=0x7fadf0102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:06:55.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:55.242+0000 7fadf553f700 1 -- 172.21.15.107:0/1499839215 shutdown_connections 2024-04-25T00:06:55.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:55.242+0000 7fadf553f700 1 -- 172.21.15.107:0/1499839215 wait complete. 2024-04-25T00:06:55.815 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for the mgr to restart... 2024-04-25T00:06:55.815 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr epoch 9... 2024-04-25T00:06:56.196 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:06:55 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1499839215' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T00:06:56.196 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:56 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:56.023+0000 7eff189bb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:06:56.484 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:56 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:56.204+0000 7eff189bb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:06:56.484 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:56 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:56.375+0000 7eff189bb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:06:56.963 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:56 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:56.665+0000 7eff189bb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:06:56.964 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:56 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:56.761+0000 7eff189bb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:06:56.964 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:56 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:56.963+0000 7eff189bb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:06:57.233 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:57 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:57.155+0000 7eff189bb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:06:57.984 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:57 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:57.752+0000 7eff189bb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:06:58.477 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:58 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:58.174+0000 7eff189bb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:06:58.477 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:58 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:58.271+0000 7eff189bb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:06:58.477 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:58 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:58.476+0000 7eff189bb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:06:59.338 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:59 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:59.002+0000 7eff189bb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:06:59.338 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:59 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:59.093+0000 7eff189bb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:06:59.685 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:59 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:59.338+0000 7eff189bb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:06:59.685 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:59 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:59.472+0000 7eff189bb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:06:59.685 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:59 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:59.568+0000 7eff189bb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:06:59.685 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:59 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:59.685+0000 7eff189bb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:06:59.984 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:06:59 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:06:59.779+0000 7eff189bb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:07:00.889 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:07:00 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:07:00.615+0000 7eff189bb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:07:00.890 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:07:00 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:07:00.836+0000 7eff189bb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:07:01.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:00 smithi107 ceph-mon[29153]: Active manager daemon x restarted 2024-04-25T00:07:01.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:00 smithi107 ceph-mon[29153]: Activating manager daemon x 2024-04-25T00:07:01.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:00 smithi107 ceph-mon[29153]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T00:07:01.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:00 smithi107 ceph-mon[29153]: mgrmap e10: x(active, starting, since 0.00346578s) 2024-04-25T00:07:01.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:00 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:07:01.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:00 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T00:07:01.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:00 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T00:07:01.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:00 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T00:07:01.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:00 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T00:07:01.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:00 smithi107 ceph-mon[29153]: Manager daemon x is now available 2024-04-25T00:07:01.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:00 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:07:01.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:00 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T00:07:01.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:00 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T00:07:02.723 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:07:02.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T00:07:02.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T00:07:02.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:07:02.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.221+0000 7f2ffee3c700 1 Processor -- start 2024-04-25T00:07:02.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.221+0000 7f2ffee3c700 1 -- start start 2024-04-25T00:07:02.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.221+0000 7f2ffee3c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff81080c0 0x7f2ff81084e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:02.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.221+0000 7f2ffee3c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff8108ab0 con 0x7f2ff81080c0 2024-04-25T00:07:02.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.221+0000 7f2ffcbd8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff81080c0 0x7f2ff81084e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:02.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.221+0000 7f2ffcbd8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff81080c0 0x7f2ff81084e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37818/0 (socket says 172.21.15.107:37818) 2024-04-25T00:07:02.724 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.221+0000 7f2ffcbd8700 1 -- 172.21.15.107:0/2986839899 learned_addr learned my addr 172.21.15.107:0/2986839899 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:07:02.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.221+0000 7f2ffcbd8700 1 -- 172.21.15.107:0/2986839899 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff8106d20 con 0x7f2ff81080c0 2024-04-25T00:07:02.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.221+0000 7f2ffcbd8700 1 --2- 172.21.15.107:0/2986839899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff81080c0 0x7f2ff81084e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00bb40 tx=0x7f2fec00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38fba84d401735b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:02.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.222+0000 7f2ff77fe700 1 -- 172.21.15.107:0/2986839899 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fec00dbe0 con 0x7f2ff81080c0 2024-04-25T00:07:02.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.222+0000 7f2ff77fe700 1 -- 172.21.15.107:0/2986839899 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fec004510 con 0x7f2ff81080c0 2024-04-25T00:07:02.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.222+0000 7f2ff77fe700 1 -- 172.21.15.107:0/2986839899 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fec010470 con 0x7f2ff81080c0 2024-04-25T00:07:02.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.222+0000 7f2ffee3c700 1 -- 172.21.15.107:0/2986839899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff81080c0 msgr2=0x7f2ff81084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:02.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.222+0000 7f2ffee3c700 1 --2- 172.21.15.107:0/2986839899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff81080c0 0x7f2ff81084e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00bb40 tx=0x7f2fec00be50 comp rx=0 tx=0).stop 2024-04-25T00:07:02.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.222+0000 7f2ffee3c700 1 -- 172.21.15.107:0/2986839899 shutdown_connections 2024-04-25T00:07:02.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.222+0000 7f2ffee3c700 1 --2- 172.21.15.107:0/2986839899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff81080c0 0x7f2ff81084e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:02.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.222+0000 7f2ffee3c700 1 -- 172.21.15.107:0/2986839899 >> 172.21.15.107:0/2986839899 conn(0x7f2ff8101810 msgr2=0x7f2ff8103c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:02.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.222+0000 7f2ffee3c700 1 -- 172.21.15.107:0/2986839899 shutdown_connections 2024-04-25T00:07:02.725 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.223+0000 7f2ffee3c700 1 -- 172.21.15.107:0/2986839899 wait complete. 2024-04-25T00:07:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.223+0000 7f2ffee3c700 1 Processor -- start 2024-04-25T00:07:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.223+0000 7f2ffee3c700 1 -- start start 2024-04-25T00:07:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.223+0000 7f2ffee3c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff81080c0 0x7f2ff81999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.223+0000 7f2ffee3c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff8105c10 con 0x7f2ff81080c0 2024-04-25T00:07:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.223+0000 7f2ffcbd8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff81080c0 0x7f2ff81999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.223+0000 7f2ffcbd8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff81080c0 0x7f2ff81999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37832/0 (socket says 172.21.15.107:37832) 2024-04-25T00:07:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.223+0000 7f2ffcbd8700 1 -- 172.21.15.107:0/3640507858 learned_addr learned my addr 172.21.15.107:0/3640507858 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:07:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.224+0000 7f2ffcbd8700 1 -- 172.21.15.107:0/3640507858 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff8199f10 con 0x7f2ff81080c0 2024-04-25T00:07:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.224+0000 7f2ffcbd8700 1 --2- 172.21.15.107:0/3640507858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff81080c0 0x7f2ff81999d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00b340 tx=0x7f2fec003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.224+0000 7f2ff5ffb700 1 -- 172.21.15.107:0/3640507858 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fec027070 con 0x7f2ff81080c0 2024-04-25T00:07:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.224+0000 7f2ffee3c700 1 -- 172.21.15.107:0/3640507858 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff819a1a0 con 0x7f2ff81080c0 2024-04-25T00:07:02.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.224+0000 7f2ff5ffb700 1 -- 172.21.15.107:0/3640507858 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fec010c50 con 0x7f2ff81080c0 2024-04-25T00:07:02.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.224+0000 7f2ff5ffb700 1 -- 172.21.15.107:0/3640507858 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fec022c10 con 0x7f2ff81080c0 2024-04-25T00:07:02.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.224+0000 7f2ffee3c700 1 -- 172.21.15.107:0/3640507858 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff819ce90 con 0x7f2ff81080c0 2024-04-25T00:07:02.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.225+0000 7f2ff5ffb700 1 -- 172.21.15.107:0/3640507858 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f2fec02a070 con 0x7f2ff81080c0 2024-04-25T00:07:02.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.225+0000 7f2ff5ffb700 1 --2- 172.21.15.107:0/3640507858 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2fe8039050 0x7f2fe803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:02.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.225+0000 7f2ff5ffb700 1 -- 172.21.15.107:0/3640507858 --> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2fe803bc20 con 0x7f2fe8039050 2024-04-25T00:07:02.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.225+0000 7f2ff5ffb700 1 -- 172.21.15.107:0/3640507858 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2fec01f070 con 0x7f2ff81080c0 2024-04-25T00:07:02.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.225+0000 7f2ff7fff700 1 -- 172.21.15.107:0/3640507858 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2fe8039050 msgr2=0x7f2fe803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/153169302 2024-04-25T00:07:02.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.225+0000 7f2ff7fff700 1 --2- 172.21.15.107:0/3640507858 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2fe8039050 0x7f2fe803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:07:02.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.426+0000 7f2ff7fff700 1 -- 172.21.15.107:0/3640507858 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2fe8039050 msgr2=0x7f2fe803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/153169302 2024-04-25T00:07:02.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.426+0000 7f2ff7fff700 1 --2- 172.21.15.107:0/3640507858 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2fe8039050 0x7f2fe803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T00:07:02.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.827+0000 7f2ff7fff700 1 -- 172.21.15.107:0/3640507858 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2fe8039050 msgr2=0x7f2fe803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/153169302 2024-04-25T00:07:02.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:56.827+0000 7f2ff7fff700 1 --2- 172.21.15.107:0/3640507858 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2fe8039050 0x7f2fe803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T00:07:02.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:57.628+0000 7f2ff7fff700 1 -- 172.21.15.107:0/3640507858 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2fe8039050 msgr2=0x7f2fe803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/153169302 2024-04-25T00:07:02.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:57.628+0000 7f2ff7fff700 1 --2- 172.21.15.107:0/3640507858 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2fe8039050 0x7f2fe803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T00:07:02.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:59.229+0000 7f2ff7fff700 1 -- 172.21.15.107:0/3640507858 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2fe8039050 msgr2=0x7f2fe803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/153169302 2024-04-25T00:07:02.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:06:59.229+0000 7f2ff7fff700 1 --2- 172.21.15.107:0/3640507858 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2fe8039050 0x7f2fe803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T00:07:02.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:00.839+0000 7f2ff5ffb700 1 -- 172.21.15.107:0/3640507858 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f2fec02cce0 con 0x7f2ff81080c0 2024-04-25T00:07:02.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:00.839+0000 7f2ff5ffb700 1 -- 172.21.15.107:0/3640507858 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2fe8039050 msgr2=0x7f2fe803b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:07:02.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:00.839+0000 7f2ff5ffb700 1 --2- 172.21.15.107:0/3640507858 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2fe8039050 0x7f2fe803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:02.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:01.841+0000 7f2ff5ffb700 1 -- 172.21.15.107:0/3640507858 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f2fec02c440 con 0x7f2ff81080c0 2024-04-25T00:07:02.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:01.841+0000 7f2ff5ffb700 1 --2- 172.21.15.107:0/3640507858 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f2fe803cb10 0x7f2fe803ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:02.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:01.841+0000 7f2ff5ffb700 1 -- 172.21.15.107:0/3640507858 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2fec00b7d0 con 0x7f2fe803cb10 2024-04-25T00:07:02.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:01.841+0000 7f2ff7fff700 1 --2- 172.21.15.107:0/3640507858 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f2fe803cb10 0x7f2fe803ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:02.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:01.842+0000 7f2ff7fff700 1 --2- 172.21.15.107:0/3640507858 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f2fe803cb10 0x7f2fe803ef00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4003a10 tx=0x7f2fe40092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:02.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:01.843+0000 7f2ff5ffb700 1 -- 172.21.15.107:0/3640507858 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f2fec00b7d0 con 0x7f2fe803cb10 2024-04-25T00:07:02.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:01.849+0000 7f2ffee3c700 1 -- 172.21.15.107:0/3640507858 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f2ff8063890 con 0x7f2fe803cb10 2024-04-25T00:07:02.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:01.849+0000 7f2ff5ffb700 1 -- 172.21.15.107:0/3640507858 <== mgr.14152 v2:172.21.15.107:6800/1644942185 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f2ff8063890 con 0x7f2fe803cb10 2024-04-25T00:07:02.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:01.850+0000 7f2ffee3c700 1 -- 172.21.15.107:0/3640507858 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f2fe803cb10 msgr2=0x7f2fe803ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:02.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:01.850+0000 7f2ffee3c700 1 --2- 172.21.15.107:0/3640507858 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f2fe803cb10 0x7f2fe803ef00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4003a10 tx=0x7f2fe40092b0 comp rx=0 tx=0).stop 2024-04-25T00:07:02.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:01.850+0000 7f2ffee3c700 1 -- 172.21.15.107:0/3640507858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff81080c0 msgr2=0x7f2ff81999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:02.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:01.850+0000 7f2ffee3c700 1 --2- 172.21.15.107:0/3640507858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff81080c0 0x7f2ff81999d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00b340 tx=0x7f2fec003e80 comp rx=0 tx=0).stop 2024-04-25T00:07:02.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:01.850+0000 7f2ffee3c700 1 -- 172.21.15.107:0/3640507858 shutdown_connections 2024-04-25T00:07:02.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:01.850+0000 7f2ffee3c700 1 --2- 172.21.15.107:0/3640507858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ff81080c0 0x7f2ff81999d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:02.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:01.850+0000 7f2ffee3c700 1 --2- 172.21.15.107:0/3640507858 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f2fe803cb10 0x7f2fe803ef00 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:02.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:01.850+0000 7f2ffee3c700 1 --2- 172.21.15.107:0/3640507858 >> [v2:172.21.15.107:6800/153169302,v1:172.21.15.107:6801/153169302] conn(0x7f2fe8039050 0x7f2fe803b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:02.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:01.850+0000 7f2ffee3c700 1 -- 172.21.15.107:0/3640507858 >> 172.21.15.107:0/3640507858 conn(0x7f2ff8101810 msgr2=0x7f2ff8190010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:02.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:01.850+0000 7f2ffee3c700 1 -- 172.21.15.107:0/3640507858 shutdown_connections 2024-04-25T00:07:02.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:01.850+0000 7f2ffee3c700 1 -- 172.21.15.107:0/3640507858 wait complete. 2024-04-25T00:07:02.730 INFO:teuthology.orchestra.run.smithi107.stdout:mgr epoch 9 is available 2024-04-25T00:07:02.730 INFO:teuthology.orchestra.run.smithi107.stdout:Generating a dashboard self-signed certificate... 2024-04-25T00:07:03.148 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:02 smithi107 ceph-mon[29153]: [25/Apr/2024:00:07:01] ENGINE Bus STARTING 2024-04-25T00:07:03.148 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:02 smithi107 ceph-mon[29153]: [25/Apr/2024:00:07:01] ENGINE Serving on http://172.21.15.107:8765 2024-04-25T00:07:03.148 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:02 smithi107 ceph-mon[29153]: [25/Apr/2024:00:07:01] ENGINE Serving on https://172.21.15.107:7150 2024-04-25T00:07:03.148 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:02 smithi107 ceph-mon[29153]: [25/Apr/2024:00:07:01] ENGINE Bus STARTED 2024-04-25T00:07:03.148 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:02 smithi107 ceph-mon[29153]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T00:07:03.148 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:02 smithi107 ceph-mon[29153]: mgrmap e11: x(active, since 1.00598s) 2024-04-25T00:07:03.148 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:02 smithi107 ceph-mon[29153]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T00:07:03.148 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:04.093 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T00:07:04.093 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.182+0000 7f0a588c7700 1 Processor -- start 2024-04-25T00:07:04.093 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.182+0000 7f0a588c7700 1 -- start start 2024-04-25T00:07:04.093 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.183+0000 7f0a588c7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a54108750 0x7f0a5410ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:04.093 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.183+0000 7f0a588c7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5407bdf0 con 0x7f0a54108750 2024-04-25T00:07:04.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.183+0000 7f0a5259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a54108750 0x7f0a5410ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:04.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.183+0000 7f0a5259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a54108750 0x7f0a5410ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52026/0 (socket says 172.21.15.107:52026) 2024-04-25T00:07:04.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.183+0000 7f0a5259c700 1 -- 172.21.15.107:0/3175479556 learned_addr learned my addr 172.21.15.107:0/3175479556 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:07:04.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.183+0000 7f0a5259c700 1 -- 172.21.15.107:0/3175479556 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5410b080 con 0x7f0a54108750 2024-04-25T00:07:04.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.183+0000 7f0a5259c700 1 --2- 172.21.15.107:0/3175479556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a54108750 0x7f0a5410ab40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0a440098a0 tx=0x7f0a44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f123e74713bdf902 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:04.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.183+0000 7f0a5159a700 1 -- 172.21.15.107:0/3175479556 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a44004160 con 0x7f0a54108750 2024-04-25T00:07:04.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.183+0000 7f0a5159a700 1 -- 172.21.15.107:0/3175479556 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a44004300 con 0x7f0a54108750 2024-04-25T00:07:04.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.183+0000 7f0a5159a700 1 -- 172.21.15.107:0/3175479556 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a44011b20 con 0x7f0a54108750 2024-04-25T00:07:04.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.184+0000 7f0a588c7700 1 -- 172.21.15.107:0/3175479556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a54108750 msgr2=0x7f0a5410ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:04.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.184+0000 7f0a588c7700 1 --2- 172.21.15.107:0/3175479556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a54108750 0x7f0a5410ab40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0a440098a0 tx=0x7f0a44009bb0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.184+0000 7f0a588c7700 1 -- 172.21.15.107:0/3175479556 shutdown_connections 2024-04-25T00:07:04.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.184+0000 7f0a588c7700 1 --2- 172.21.15.107:0/3175479556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a54108750 0x7f0a5410ab40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.094 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.184+0000 7f0a588c7700 1 -- 172.21.15.107:0/3175479556 >> 172.21.15.107:0/3175479556 conn(0x7f0a541020a0 msgr2=0x7f0a54104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:04.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.184+0000 7f0a588c7700 1 -- 172.21.15.107:0/3175479556 shutdown_connections 2024-04-25T00:07:04.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.184+0000 7f0a588c7700 1 -- 172.21.15.107:0/3175479556 wait complete. 2024-04-25T00:07:04.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.185+0000 7f0a588c7700 1 Processor -- start 2024-04-25T00:07:04.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.185+0000 7f0a588c7700 1 -- start start 2024-04-25T00:07:04.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.185+0000 7f0a588c7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a54108750 0x7f0a5419bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:04.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.185+0000 7f0a588c7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5410c030 con 0x7f0a54108750 2024-04-25T00:07:04.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.185+0000 7f0a5259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a54108750 0x7f0a5419bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:04.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.185+0000 7f0a5259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a54108750 0x7f0a5419bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52040/0 (socket says 172.21.15.107:52040) 2024-04-25T00:07:04.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.185+0000 7f0a5259c700 1 -- 172.21.15.107:0/3652958221 learned_addr learned my addr 172.21.15.107:0/3652958221 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:07:04.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.186+0000 7f0a5259c700 1 -- 172.21.15.107:0/3652958221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5419c090 con 0x7f0a54108750 2024-04-25T00:07:04.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.186+0000 7f0a5259c700 1 --2- 172.21.15.107:0/3652958221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a54108750 0x7f0a5419bb50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0a440117e0 tx=0x7f0a44011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:04.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.186+0000 7f0a437fe700 1 -- 172.21.15.107:0/3652958221 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a44003bf0 con 0x7f0a54108750 2024-04-25T00:07:04.095 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.186+0000 7f0a588c7700 1 -- 172.21.15.107:0/3652958221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a5419c320 con 0x7f0a54108750 2024-04-25T00:07:04.096 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.186+0000 7f0a437fe700 1 -- 172.21.15.107:0/3652958221 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a4401a910 con 0x7f0a54108750 2024-04-25T00:07:04.096 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.186+0000 7f0a437fe700 1 -- 172.21.15.107:0/3652958221 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a44024970 con 0x7f0a54108750 2024-04-25T00:07:04.096 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.186+0000 7f0a588c7700 1 -- 172.21.15.107:0/3652958221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a5419c780 con 0x7f0a54108750 2024-04-25T00:07:04.096 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.187+0000 7f0a437fe700 1 -- 172.21.15.107:0/3652958221 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f0a44022020 con 0x7f0a54108750 2024-04-25T00:07:04.096 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.187+0000 7f0a437fe700 1 --2- 172.21.15.107:0/3652958221 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f0a3c03d350 0x7f0a3c03f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:04.096 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.187+0000 7f0a588c7700 1 -- 172.21.15.107:0/3652958221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a54063890 con 0x7f0a54108750 2024-04-25T00:07:04.096 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.187+0000 7f0a437fe700 1 -- 172.21.15.107:0/3652958221 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0a4404d5e0 con 0x7f0a54108750 2024-04-25T00:07:04.096 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.187+0000 7f0a51d9b700 1 --2- 172.21.15.107:0/3652958221 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f0a3c03d350 0x7f0a3c03f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:04.096 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.188+0000 7f0a51d9b700 1 --2- 172.21.15.107:0/3652958221 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f0a3c03d350 0x7f0a3c03f810 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0a48006fd0 tx=0x7f0a48006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:04.096 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.192+0000 7f0a437fe700 1 -- 172.21.15.107:0/3652958221 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0a44011e20 con 0x7f0a54108750 2024-04-25T00:07:04.096 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.404+0000 7f0a588c7700 1 -- 172.21.15.107:0/3652958221 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f0a54103280 con 0x7f0a3c03d350 2024-04-25T00:07:04.096 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.444+0000 7f0a437fe700 1 -- 172.21.15.107:0/3652958221 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f0a4402d910 con 0x7f0a54108750 2024-04-25T00:07:04.096 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.522+0000 7f0a437fe700 1 -- 172.21.15.107:0/3652958221 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f0a54103280 con 0x7f0a3c03d350 2024-04-25T00:07:04.096 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.524+0000 7f0a588c7700 1 -- 172.21.15.107:0/3652958221 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f0a3c03d350 msgr2=0x7f0a3c03f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:04.097 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.524+0000 7f0a588c7700 1 --2- 172.21.15.107:0/3652958221 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f0a3c03d350 0x7f0a3c03f810 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0a48006fd0 tx=0x7f0a48006e40 comp rx=0 tx=0).stop 2024-04-25T00:07:04.097 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.524+0000 7f0a588c7700 1 -- 172.21.15.107:0/3652958221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a54108750 msgr2=0x7f0a5419bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:04.097 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.524+0000 7f0a588c7700 1 --2- 172.21.15.107:0/3652958221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a54108750 0x7f0a5419bb50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0a440117e0 tx=0x7f0a44011c10 comp rx=0 tx=0).stop 2024-04-25T00:07:04.097 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.524+0000 7f0a588c7700 1 -- 172.21.15.107:0/3652958221 shutdown_connections 2024-04-25T00:07:04.097 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.524+0000 7f0a588c7700 1 --2- 172.21.15.107:0/3652958221 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f0a3c03d350 0x7f0a3c03f810 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.097 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.524+0000 7f0a588c7700 1 --2- 172.21.15.107:0/3652958221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a54108750 0x7f0a5419bb50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:04.097 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.524+0000 7f0a588c7700 1 -- 172.21.15.107:0/3652958221 >> 172.21.15.107:0/3652958221 conn(0x7f0a541020a0 msgr2=0x7f0a54102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:04.097 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.525+0000 7f0a588c7700 1 -- 172.21.15.107:0/3652958221 shutdown_connections 2024-04-25T00:07:04.097 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:03.525+0000 7f0a588c7700 1 -- 172.21.15.107:0/3652958221 wait complete. 2024-04-25T00:07:04.097 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial admin user... 2024-04-25T00:07:04.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:04 smithi107 ceph-mon[29153]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:07:04.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:04 smithi107 ceph-mon[29153]: mgrmap e12: x(active, since 2s) 2024-04-25T00:07:04.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:04 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:04.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:04 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:05.487 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$qmJahFIGOrRm8KlnN1Hfz.N.wj/vWee7DUPCnx9a1q83G69vTYq1m", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714003624, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T00:07:05.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.465+0000 7f4a23d3e700 1 Processor -- start 2024-04-25T00:07:05.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.465+0000 7f4a23d3e700 1 -- start start 2024-04-25T00:07:05.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.465+0000 7f4a23d3e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a1c107c40 0x7f4a1c108060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:05.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.465+0000 7f4a23d3e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c108630 con 0x7f4a1c107c40 2024-04-25T00:07:05.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.466+0000 7f4a21ada700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a1c107c40 0x7f4a1c108060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:05.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.466+0000 7f4a21ada700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a1c107c40 0x7f4a1c108060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52042/0 (socket says 172.21.15.107:52042) 2024-04-25T00:07:05.488 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.466+0000 7f4a21ada700 1 -- 172.21.15.107:0/739542511 learned_addr learned my addr 172.21.15.107:0/739542511 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:07:05.489 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.466+0000 7f4a21ada700 1 -- 172.21.15.107:0/739542511 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1c108e40 con 0x7f4a1c107c40 2024-04-25T00:07:05.489 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.466+0000 7f4a21ada700 1 --2- 172.21.15.107:0/739542511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a1c107c40 0x7f4a1c108060 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4a180098a0 tx=0x7f4a18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2342e517704a93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:05.489 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.466+0000 7f4a20ad8700 1 -- 172.21.15.107:0/739542511 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a18004160 con 0x7f4a1c107c40 2024-04-25T00:07:05.489 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.466+0000 7f4a20ad8700 1 -- 172.21.15.107:0/739542511 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a18004300 con 0x7f4a1c107c40 2024-04-25T00:07:05.489 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.466+0000 7f4a20ad8700 1 -- 172.21.15.107:0/739542511 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a18011b20 con 0x7f4a1c107c40 2024-04-25T00:07:05.489 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.467+0000 7f4a23d3e700 1 -- 172.21.15.107:0/739542511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a1c107c40 msgr2=0x7f4a1c108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:05.490 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.467+0000 7f4a23d3e700 1 --2- 172.21.15.107:0/739542511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a1c107c40 0x7f4a1c108060 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4a180098a0 tx=0x7f4a18009bb0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.490 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.467+0000 7f4a23d3e700 1 -- 172.21.15.107:0/739542511 shutdown_connections 2024-04-25T00:07:05.490 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.467+0000 7f4a23d3e700 1 --2- 172.21.15.107:0/739542511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a1c107c40 0x7f4a1c108060 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.490 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.467+0000 7f4a23d3e700 1 -- 172.21.15.107:0/739542511 >> 172.21.15.107:0/739542511 conn(0x7f4a1c1031e0 msgr2=0x7f4a1c105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:05.490 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.467+0000 7f4a23d3e700 1 -- 172.21.15.107:0/739542511 shutdown_connections 2024-04-25T00:07:05.490 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.467+0000 7f4a23d3e700 1 -- 172.21.15.107:0/739542511 wait complete. 2024-04-25T00:07:05.490 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.468+0000 7f4a23d3e700 1 Processor -- start 2024-04-25T00:07:05.491 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.468+0000 7f4a23d3e700 1 -- start start 2024-04-25T00:07:05.491 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.468+0000 7f4a23d3e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a1c107c40 0x7f4a1c19bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:05.491 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.468+0000 7f4a23d3e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c10c1f0 con 0x7f4a1c107c40 2024-04-25T00:07:05.491 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.468+0000 7f4a21ada700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a1c107c40 0x7f4a1c19bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:05.491 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.468+0000 7f4a21ada700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a1c107c40 0x7f4a1c19bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52058/0 (socket says 172.21.15.107:52058) 2024-04-25T00:07:05.491 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.468+0000 7f4a21ada700 1 -- 172.21.15.107:0/1075096907 learned_addr learned my addr 172.21.15.107:0/1075096907 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:07:05.492 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.468+0000 7f4a21ada700 1 -- 172.21.15.107:0/1075096907 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1c19c0f0 con 0x7f4a1c107c40 2024-04-25T00:07:05.492 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.468+0000 7f4a21ada700 1 --2- 172.21.15.107:0/1075096907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a1c107c40 0x7f4a1c19bbb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4a180116a0 tx=0x7f4a18011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:05.492 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.468+0000 7f4a12ffd700 1 -- 172.21.15.107:0/1075096907 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a18003a40 con 0x7f4a1c107c40 2024-04-25T00:07:05.492 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.469+0000 7f4a12ffd700 1 -- 172.21.15.107:0/1075096907 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a18003be0 con 0x7f4a1c107c40 2024-04-25T00:07:05.492 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.469+0000 7f4a23d3e700 1 -- 172.21.15.107:0/1075096907 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a1c19c380 con 0x7f4a1c107c40 2024-04-25T00:07:05.492 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.469+0000 7f4a12ffd700 1 -- 172.21.15.107:0/1075096907 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a1801a720 con 0x7f4a1c107c40 2024-04-25T00:07:05.492 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.469+0000 7f4a23d3e700 1 -- 172.21.15.107:0/1075096907 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a1c19c7e0 con 0x7f4a1c107c40 2024-04-25T00:07:05.493 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.469+0000 7f4a12ffd700 1 -- 172.21.15.107:0/1075096907 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f4a1801a9f0 con 0x7f4a1c107c40 2024-04-25T00:07:05.493 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.469+0000 7f4a23d3e700 1 -- 172.21.15.107:0/1075096907 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a1c063890 con 0x7f4a1c107c40 2024-04-25T00:07:05.493 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.470+0000 7f4a12ffd700 1 --2- 172.21.15.107:0/1075096907 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f4a080390b0 0x7f4a0803b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:05.493 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.470+0000 7f4a12ffd700 1 -- 172.21.15.107:0/1075096907 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4a1804d6f0 con 0x7f4a1c107c40 2024-04-25T00:07:05.493 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.470+0000 7f4a212d9700 1 --2- 172.21.15.107:0/1075096907 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f4a080390b0 0x7f4a0803b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:05.493 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.470+0000 7f4a212d9700 1 --2- 172.21.15.107:0/1075096907 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f4a080390b0 0x7f4a0803b570 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c006fd0 tx=0x7f4a0c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:05.494 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.475+0000 7f4a12ffd700 1 -- 172.21.15.107:0/1075096907 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4a18011e20 con 0x7f4a1c107c40 2024-04-25T00:07:05.494 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.696+0000 7f4a23d3e700 1 -- 172.21.15.107:0/1075096907 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f4a1c19f1f0 con 0x7f4a080390b0 2024-04-25T00:07:05.494 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.919+0000 7f4a12ffd700 1 -- 172.21.15.107:0/1075096907 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f4a1c19f1f0 con 0x7f4a080390b0 2024-04-25T00:07:05.494 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.921+0000 7f4a23d3e700 1 -- 172.21.15.107:0/1075096907 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f4a080390b0 msgr2=0x7f4a0803b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:05.494 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.921+0000 7f4a23d3e700 1 --2- 172.21.15.107:0/1075096907 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f4a080390b0 0x7f4a0803b570 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c006fd0 tx=0x7f4a0c006e40 comp rx=0 tx=0).stop 2024-04-25T00:07:05.494 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.921+0000 7f4a23d3e700 1 -- 172.21.15.107:0/1075096907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a1c107c40 msgr2=0x7f4a1c19bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:05.495 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.921+0000 7f4a23d3e700 1 --2- 172.21.15.107:0/1075096907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a1c107c40 0x7f4a1c19bbb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4a180116a0 tx=0x7f4a18011ba0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.495 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.921+0000 7f4a23d3e700 1 -- 172.21.15.107:0/1075096907 shutdown_connections 2024-04-25T00:07:05.495 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.921+0000 7f4a23d3e700 1 --2- 172.21.15.107:0/1075096907 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f4a080390b0 0x7f4a0803b570 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.495 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.921+0000 7f4a23d3e700 1 --2- 172.21.15.107:0/1075096907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a1c107c40 0x7f4a1c19bbb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:05.495 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.921+0000 7f4a23d3e700 1 -- 172.21.15.107:0/1075096907 >> 172.21.15.107:0/1075096907 conn(0x7f4a1c1031e0 msgr2=0x7f4a1c103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:05.495 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.921+0000 7f4a23d3e700 1 -- 172.21.15.107:0/1075096907 shutdown_connections 2024-04-25T00:07:05.495 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:04.921+0000 7f4a23d3e700 1 -- 172.21.15.107:0/1075096907 wait complete. 2024-04-25T00:07:05.496 INFO:teuthology.orchestra.run.smithi107.stdout:Fetching dashboard port number... 2024-04-25T00:07:05.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:05 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:05.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:05 smithi107 ceph-mon[29153]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:07:05.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:05 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:06.851 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:06 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/4169452868' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T00:07:06.854 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T00:07:06.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.932+0000 7f79c8b5a700 1 Processor -- start 2024-04-25T00:07:06.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.932+0000 7f79c8b5a700 1 -- start start 2024-04-25T00:07:06.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.932+0000 7f79c8b5a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c4107c40 0x7f79c4108060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:06.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.933+0000 7f79c8b5a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f79c4108630 con 0x7f79c4107c40 2024-04-25T00:07:06.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.933+0000 7f79c259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c4107c40 0x7f79c4108060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:06.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.933+0000 7f79c259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c4107c40 0x7f79c4108060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52062/0 (socket says 172.21.15.107:52062) 2024-04-25T00:07:06.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.933+0000 7f79c259c700 1 -- 172.21.15.107:0/3525798040 learned_addr learned my addr 172.21.15.107:0/3525798040 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:07:06.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.933+0000 7f79c259c700 1 -- 172.21.15.107:0/3525798040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c4108e40 con 0x7f79c4107c40 2024-04-25T00:07:06.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.933+0000 7f79c259c700 1 --2- 172.21.15.107:0/3525798040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c4107c40 0x7f79c4108060 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f79b80098a0 tx=0x7f79b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84078a3205e7db6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:06.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.933+0000 7f79c159a700 1 -- 172.21.15.107:0/3525798040 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79b8004160 con 0x7f79c4107c40 2024-04-25T00:07:06.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.933+0000 7f79c159a700 1 -- 172.21.15.107:0/3525798040 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79b8004300 con 0x7f79c4107c40 2024-04-25T00:07:06.860 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.933+0000 7f79c159a700 1 -- 172.21.15.107:0/3525798040 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79b8011b20 con 0x7f79c4107c40 2024-04-25T00:07:06.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.934+0000 7f79c8b5a700 1 -- 172.21.15.107:0/3525798040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c4107c40 msgr2=0x7f79c4108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:06.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.934+0000 7f79c8b5a700 1 --2- 172.21.15.107:0/3525798040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c4107c40 0x7f79c4108060 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f79b80098a0 tx=0x7f79b8009bb0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.934+0000 7f79c8b5a700 1 -- 172.21.15.107:0/3525798040 shutdown_connections 2024-04-25T00:07:06.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.934+0000 7f79c8b5a700 1 --2- 172.21.15.107:0/3525798040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c4107c40 0x7f79c4108060 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.934+0000 7f79c8b5a700 1 -- 172.21.15.107:0/3525798040 >> 172.21.15.107:0/3525798040 conn(0x7f79c41031e0 msgr2=0x7f79c4105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:06.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.934+0000 7f79c8b5a700 1 -- 172.21.15.107:0/3525798040 shutdown_connections 2024-04-25T00:07:06.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.934+0000 7f79c8b5a700 1 -- 172.21.15.107:0/3525798040 wait complete. 2024-04-25T00:07:06.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.935+0000 7f79c8b5a700 1 Processor -- start 2024-04-25T00:07:06.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.935+0000 7f79c8b5a700 1 -- start start 2024-04-25T00:07:06.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.935+0000 7f79c8b5a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c4107c40 0x7f79c407db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:06.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.935+0000 7f79c8b5a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f79c410c1f0 con 0x7f79c4107c40 2024-04-25T00:07:06.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.935+0000 7f79c259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c4107c40 0x7f79c407db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:06.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.935+0000 7f79c259c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c4107c40 0x7f79c407db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52074/0 (socket says 172.21.15.107:52074) 2024-04-25T00:07:06.861 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.935+0000 7f79c259c700 1 -- 172.21.15.107:0/4169452868 learned_addr learned my addr 172.21.15.107:0/4169452868 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:07:06.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.935+0000 7f79c259c700 1 -- 172.21.15.107:0/4169452868 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c407e0b0 con 0x7f79c4107c40 2024-04-25T00:07:06.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.936+0000 7f79c259c700 1 --2- 172.21.15.107:0/4169452868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c4107c40 0x7f79c407db70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f79b80116a0 tx=0x7f79b8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:06.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.936+0000 7f79b37fe700 1 -- 172.21.15.107:0/4169452868 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79b8003a40 con 0x7f79c4107c40 2024-04-25T00:07:06.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.936+0000 7f79c8b5a700 1 -- 172.21.15.107:0/4169452868 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c407c2a0 con 0x7f79c4107c40 2024-04-25T00:07:06.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.936+0000 7f79b37fe700 1 -- 172.21.15.107:0/4169452868 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79b8003be0 con 0x7f79c4107c40 2024-04-25T00:07:06.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.936+0000 7f79c8b5a700 1 -- 172.21.15.107:0/4169452868 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c407c760 con 0x7f79c4107c40 2024-04-25T00:07:06.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.936+0000 7f79b37fe700 1 -- 172.21.15.107:0/4169452868 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79b801a720 con 0x7f79c4107c40 2024-04-25T00:07:06.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.937+0000 7f79b37fe700 1 -- 172.21.15.107:0/4169452868 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f79b801aa50 con 0x7f79c4107c40 2024-04-25T00:07:06.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.937+0000 7f79c8b5a700 1 -- 172.21.15.107:0/4169452868 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79c4063890 con 0x7f79c4107c40 2024-04-25T00:07:06.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.937+0000 7f79b37fe700 1 --2- 172.21.15.107:0/4169452868 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f79ac0390b0 0x7f79ac03b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:06.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.937+0000 7f79b37fe700 1 -- 172.21.15.107:0/4169452868 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f79b804d6b0 con 0x7f79c4107c40 2024-04-25T00:07:06.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.937+0000 7f79c1d9b700 1 --2- 172.21.15.107:0/4169452868 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f79ac0390b0 0x7f79ac03b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:06.862 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.938+0000 7f79c1d9b700 1 --2- 172.21.15.107:0/4169452868 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f79ac0390b0 0x7f79ac03b570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f79b4006fd0 tx=0x7f79b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:06.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:05.942+0000 7f79b37fe700 1 -- 172.21.15.107:0/4169452868 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f79b8011e20 con 0x7f79c4107c40 2024-04-25T00:07:06.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:06.159+0000 7f79c8b5a700 1 -- 172.21.15.107:0/4169452868 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f79c407caa0 con 0x7f79c4107c40 2024-04-25T00:07:06.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:06.159+0000 7f79b37fe700 1 -- 172.21.15.107:0/4169452868 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f79c407caa0 con 0x7f79c4107c40 2024-04-25T00:07:06.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:06.161+0000 7f79c8b5a700 1 -- 172.21.15.107:0/4169452868 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f79ac0390b0 msgr2=0x7f79ac03b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:06.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:06.161+0000 7f79c8b5a700 1 --2- 172.21.15.107:0/4169452868 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f79ac0390b0 0x7f79ac03b570 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f79b4006fd0 tx=0x7f79b4006e40 comp rx=0 tx=0).stop 2024-04-25T00:07:06.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:06.161+0000 7f79c8b5a700 1 -- 172.21.15.107:0/4169452868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c4107c40 msgr2=0x7f79c407db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:06.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:06.161+0000 7f79c8b5a700 1 --2- 172.21.15.107:0/4169452868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c4107c40 0x7f79c407db70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f79b80116a0 tx=0x7f79b8011ba0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:06.161+0000 7f79c8b5a700 1 -- 172.21.15.107:0/4169452868 shutdown_connections 2024-04-25T00:07:06.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:06.161+0000 7f79c8b5a700 1 --2- 172.21.15.107:0/4169452868 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f79ac0390b0 0x7f79ac03b570 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:06.161+0000 7f79c8b5a700 1 --2- 172.21.15.107:0/4169452868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79c4107c40 0x7f79c407db70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:06.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:06.161+0000 7f79c8b5a700 1 -- 172.21.15.107:0/4169452868 >> 172.21.15.107:0/4169452868 conn(0x7f79c41031e0 msgr2=0x7f79c4103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:06.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:06.162+0000 7f79c8b5a700 1 -- 172.21.15.107:0/4169452868 shutdown_connections 2024-04-25T00:07:06.863 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:06.162+0000 7f79c8b5a700 1 -- 172.21.15.107:0/4169452868 wait complete. 2024-04-25T00:07:06.866 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T00:07:06.866 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-04-25T00:07:06.874 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T00:07:06.874 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-04-25T00:07:06.874 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-04-25T00:07:06.874 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T00:07:06.875 INFO:teuthology.orchestra.run.smithi107.stdout:Ceph Dashboard is now available at: 2024-04-25T00:07:06.875 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:07:06.875 INFO:teuthology.orchestra.run.smithi107.stdout: URL: https://smithi107:8443/ 2024-04-25T00:07:06.875 INFO:teuthology.orchestra.run.smithi107.stdout: User: admin 2024-04-25T00:07:06.875 INFO:teuthology.orchestra.run.smithi107.stdout: Password: 37p8tey8ba 2024-04-25T00:07:06.875 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:07:06.875 INFO:teuthology.orchestra.run.smithi107.stdout:Saving cluster configuration to /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config directory 2024-04-25T00:07:06.876 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling autotune for osd_memory_target 2024-04-25T00:07:08.804 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:08 smithi107 ceph-mon[29153]: mgrmap e13: x(active, since 6s) 2024-04-25T00:07:08.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.043+0000 7f7da172e700 1 Processor -- start 2024-04-25T00:07:08.812 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.043+0000 7f7da172e700 1 -- start start 2024-04-25T00:07:08.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.043+0000 7f7da172e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d9c108760 0x7f7d9c10ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.043+0000 7f7da172e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c07be30 con 0x7f7d9c108760 2024-04-25T00:07:08.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.043+0000 7f7d9affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d9c108760 0x7f7d9c10ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.043+0000 7f7d9affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d9c108760 0x7f7d9c10ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52084/0 (socket says 172.21.15.107:52084) 2024-04-25T00:07:08.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.043+0000 7f7d9affd700 1 -- 172.21.15.107:0/1796075495 learned_addr learned my addr 172.21.15.107:0/1796075495 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:07:08.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.044+0000 7f7d9affd700 1 -- 172.21.15.107:0/1796075495 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9c10b090 con 0x7f7d9c108760 2024-04-25T00:07:08.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.044+0000 7f7d9affd700 1 --2- 172.21.15.107:0/1796075495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d9c108760 0x7f7d9c10ab50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0098a0 tx=0x7f7d8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18861f0bb8a87847 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:08.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.044+0000 7f7d99ffb700 1 -- 172.21.15.107:0/1796075495 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d8c004160 con 0x7f7d9c108760 2024-04-25T00:07:08.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.044+0000 7f7d99ffb700 1 -- 172.21.15.107:0/1796075495 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d8c004300 con 0x7f7d9c108760 2024-04-25T00:07:08.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.044+0000 7f7d99ffb700 1 -- 172.21.15.107:0/1796075495 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d8c011b20 con 0x7f7d9c108760 2024-04-25T00:07:08.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.044+0000 7f7da172e700 1 -- 172.21.15.107:0/1796075495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d9c108760 msgr2=0x7f7d9c10ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:08.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.044+0000 7f7da172e700 1 --2- 172.21.15.107:0/1796075495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d9c108760 0x7f7d9c10ab50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0098a0 tx=0x7f7d8c009bb0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.813 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.044+0000 7f7da172e700 1 -- 172.21.15.107:0/1796075495 shutdown_connections 2024-04-25T00:07:08.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.044+0000 7f7da172e700 1 --2- 172.21.15.107:0/1796075495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d9c108760 0x7f7d9c10ab50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.045+0000 7f7da172e700 1 -- 172.21.15.107:0/1796075495 >> 172.21.15.107:0/1796075495 conn(0x7f7d9c102130 msgr2=0x7f7d9c104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:08.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.045+0000 7f7da172e700 1 -- 172.21.15.107:0/1796075495 shutdown_connections 2024-04-25T00:07:08.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.045+0000 7f7da172e700 1 -- 172.21.15.107:0/1796075495 wait complete. 2024-04-25T00:07:08.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.045+0000 7f7da172e700 1 Processor -- start 2024-04-25T00:07:08.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.045+0000 7f7da172e700 1 -- start start 2024-04-25T00:07:08.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.045+0000 7f7da172e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d9c108760 0x7f7d9c19bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.045+0000 7f7da172e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9c10c090 con 0x7f7d9c108760 2024-04-25T00:07:08.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.046+0000 7f7d9affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d9c108760 0x7f7d9c19bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.046+0000 7f7d9affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d9c108760 0x7f7d9c19bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52088/0 (socket says 172.21.15.107:52088) 2024-04-25T00:07:08.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.046+0000 7f7d9affd700 1 -- 172.21.15.107:0/3333854246 learned_addr learned my addr 172.21.15.107:0/3333854246 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:07:08.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.046+0000 7f7d9affd700 1 -- 172.21.15.107:0/3333854246 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d9c19c100 con 0x7f7d9c108760 2024-04-25T00:07:08.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.046+0000 7f7d9affd700 1 --2- 172.21.15.107:0/3333854246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d9c108760 0x7f7d9c19bbc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c011b60 tx=0x7f7d8c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:08.814 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.046+0000 7f7d83fff700 1 -- 172.21.15.107:0/3333854246 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d8c003b60 con 0x7f7d9c108760 2024-04-25T00:07:08.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.046+0000 7f7d83fff700 1 -- 172.21.15.107:0/3333854246 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d8c01a910 con 0x7f7d9c108760 2024-04-25T00:07:08.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.046+0000 7f7da172e700 1 -- 172.21.15.107:0/3333854246 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d9c19c390 con 0x7f7d9c108760 2024-04-25T00:07:08.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.046+0000 7f7d83fff700 1 -- 172.21.15.107:0/3333854246 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d8c024930 con 0x7f7d9c108760 2024-04-25T00:07:08.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.046+0000 7f7da172e700 1 -- 172.21.15.107:0/3333854246 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d9c19c7f0 con 0x7f7d9c108760 2024-04-25T00:07:08.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.047+0000 7f7d83fff700 1 -- 172.21.15.107:0/3333854246 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f7d8c022020 con 0x7f7d9c108760 2024-04-25T00:07:08.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.047+0000 7f7d83fff700 1 --2- 172.21.15.107:0/3333854246 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7d84039140 0x7f7d8403b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:08.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.047+0000 7f7da172e700 1 -- 172.21.15.107:0/3333854246 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d9c063890 con 0x7f7d9c108760 2024-04-25T00:07:08.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.047+0000 7f7d83fff700 1 -- 172.21.15.107:0/3333854246 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7d8c04d520 con 0x7f7d9c108760 2024-04-25T00:07:08.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.047+0000 7f7d9a7fc700 1 --2- 172.21.15.107:0/3333854246 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7d84039140 0x7f7d8403b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:08.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.048+0000 7f7d9a7fc700 1 --2- 172.21.15.107:0/3333854246 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7d84039140 0x7f7d8403b600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7d90006fd0 tx=0x7f7d90006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:08.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.052+0000 7f7d83fff700 1 -- 172.21.15.107:0/3333854246 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7d8c02d3f0 con 0x7f7d9c108760 2024-04-25T00:07:08.815 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.245+0000 7f7da172e700 1 -- 172.21.15.107:0/3333854246 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f7d9c19f1c0 con 0x7f7d9c108760 2024-04-25T00:07:08.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.246+0000 7f7d83fff700 1 -- 172.21.15.107:0/3333854246 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f7d8c01a430 con 0x7f7d9c108760 2024-04-25T00:07:08.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.248+0000 7f7da172e700 1 -- 172.21.15.107:0/3333854246 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7d84039140 msgr2=0x7f7d8403b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:08.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.248+0000 7f7da172e700 1 --2- 172.21.15.107:0/3333854246 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7d84039140 0x7f7d8403b600 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7d90006fd0 tx=0x7f7d90006e40 comp rx=0 tx=0).stop 2024-04-25T00:07:08.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.248+0000 7f7da172e700 1 -- 172.21.15.107:0/3333854246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d9c108760 msgr2=0x7f7d9c19bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:08.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.248+0000 7f7da172e700 1 --2- 172.21.15.107:0/3333854246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d9c108760 0x7f7d9c19bbc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c011b60 tx=0x7f7d8c011c40 comp rx=0 tx=0).stop 2024-04-25T00:07:08.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.248+0000 7f7da172e700 1 -- 172.21.15.107:0/3333854246 shutdown_connections 2024-04-25T00:07:08.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.248+0000 7f7da172e700 1 --2- 172.21.15.107:0/3333854246 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7d84039140 0x7f7d8403b600 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.248+0000 7f7da172e700 1 --2- 172.21.15.107:0/3333854246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d9c108760 0x7f7d9c19bbc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:08.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.248+0000 7f7da172e700 1 -- 172.21.15.107:0/3333854246 >> 172.21.15.107:0/3333854246 conn(0x7f7d9c102130 msgr2=0x7f7d9c102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:08.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.248+0000 7f7da172e700 1 -- 172.21.15.107:0/3333854246 shutdown_connections 2024-04-25T00:07:08.816 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:08.248+0000 7f7da172e700 1 -- 172.21.15.107:0/3333854246 wait complete. 2024-04-25T00:07:10.108 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.232+0000 7f57bc059700 1 Processor -- start 2024-04-25T00:07:10.108 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.232+0000 7f57bc059700 1 -- start start 2024-04-25T00:07:10.108 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.232+0000 7f57bc059700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57b4106550 0x7f57b4106970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.108 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.232+0000 7f57bc059700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f57b407be30 con 0x7f57b4106550 2024-04-25T00:07:10.108 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.232+0000 7f57b9df5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57b4106550 0x7f57b4106970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.109 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.232+0000 7f57b9df5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57b4106550 0x7f57b4106970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52104/0 (socket says 172.21.15.107:52104) 2024-04-25T00:07:10.109 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.232+0000 7f57b9df5700 1 -- 172.21.15.107:0/2933014064 learned_addr learned my addr 172.21.15.107:0/2933014064 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:07:10.109 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.233+0000 7f57b9df5700 1 -- 172.21.15.107:0/2933014064 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b4106eb0 con 0x7f57b4106550 2024-04-25T00:07:10.109 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.233+0000 7f57b9df5700 1 --2- 172.21.15.107:0/2933014064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57b4106550 0x7f57b4106970 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f57b00098a0 tx=0x7f57b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133f984a9b507a6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:10.109 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.233+0000 7f57b8df3700 1 -- 172.21.15.107:0/2933014064 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57b0004160 con 0x7f57b4106550 2024-04-25T00:07:10.109 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.233+0000 7f57b8df3700 1 -- 172.21.15.107:0/2933014064 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57b0004300 con 0x7f57b4106550 2024-04-25T00:07:10.109 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.233+0000 7f57b8df3700 1 -- 172.21.15.107:0/2933014064 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57b0011b20 con 0x7f57b4106550 2024-04-25T00:07:10.109 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.233+0000 7f57bc059700 1 -- 172.21.15.107:0/2933014064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57b4106550 msgr2=0x7f57b4106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:10.109 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.233+0000 7f57bc059700 1 --2- 172.21.15.107:0/2933014064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57b4106550 0x7f57b4106970 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f57b00098a0 tx=0x7f57b0009bb0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.109 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.234+0000 7f57bc059700 1 -- 172.21.15.107:0/2933014064 shutdown_connections 2024-04-25T00:07:10.109 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.234+0000 7f57bc059700 1 --2- 172.21.15.107:0/2933014064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57b4106550 0x7f57b4106970 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.109 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.234+0000 7f57bc059700 1 -- 172.21.15.107:0/2933014064 >> 172.21.15.107:0/2933014064 conn(0x7f57b4102130 msgr2=0x7f57b4104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:10.110 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.234+0000 7f57bc059700 1 -- 172.21.15.107:0/2933014064 shutdown_connections 2024-04-25T00:07:10.110 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.234+0000 7f57bc059700 1 -- 172.21.15.107:0/2933014064 wait complete. 2024-04-25T00:07:10.110 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.234+0000 7f57bc059700 1 Processor -- start 2024-04-25T00:07:10.110 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.234+0000 7f57bc059700 1 -- start start 2024-04-25T00:07:10.110 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.234+0000 7f57bc059700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57b4106550 0x7f57b419ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.110 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.234+0000 7f57bc059700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f57b4107f80 con 0x7f57b4106550 2024-04-25T00:07:10.110 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.234+0000 7f57b9df5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57b4106550 0x7f57b419ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.110 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.234+0000 7f57b9df5700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57b4106550 0x7f57b419ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52112/0 (socket says 172.21.15.107:52112) 2024-04-25T00:07:10.110 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.234+0000 7f57b9df5700 1 -- 172.21.15.107:0/591053193 learned_addr learned my addr 172.21.15.107:0/591053193 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:07:10.110 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.235+0000 7f57b9df5700 1 -- 172.21.15.107:0/591053193 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b419e300 con 0x7f57b4106550 2024-04-25T00:07:10.110 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.235+0000 7f57b9df5700 1 --2- 172.21.15.107:0/591053193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57b4106550 0x7f57b419ddc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f57b00116a0 tx=0x7f57b0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:10.110 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.235+0000 7f57aaffd700 1 -- 172.21.15.107:0/591053193 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57b0003a40 con 0x7f57b4106550 2024-04-25T00:07:10.110 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.235+0000 7f57aaffd700 1 -- 172.21.15.107:0/591053193 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57b0003be0 con 0x7f57b4106550 2024-04-25T00:07:10.111 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.235+0000 7f57aaffd700 1 -- 172.21.15.107:0/591053193 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57b001a750 con 0x7f57b4106550 2024-04-25T00:07:10.111 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.235+0000 7f57bc059700 1 -- 172.21.15.107:0/591053193 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b419e590 con 0x7f57b4106550 2024-04-25T00:07:10.111 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.235+0000 7f57bc059700 1 -- 172.21.15.107:0/591053193 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b419e9f0 con 0x7f57b4106550 2024-04-25T00:07:10.111 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.236+0000 7f57aaffd700 1 -- 172.21.15.107:0/591053193 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f57b0024430 con 0x7f57b4106550 2024-04-25T00:07:10.111 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.236+0000 7f57bc059700 1 -- 172.21.15.107:0/591053193 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57b4063890 con 0x7f57b4106550 2024-04-25T00:07:10.111 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.236+0000 7f57aaffd700 1 --2- 172.21.15.107:0/591053193 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f57a0039150 0x7f57a003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:10.111 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.236+0000 7f57aaffd700 1 -- 172.21.15.107:0/591053193 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f57b004d820 con 0x7f57b4106550 2024-04-25T00:07:10.111 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.236+0000 7f57b95f4700 1 --2- 172.21.15.107:0/591053193 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f57a0039150 0x7f57a003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:10.111 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.236+0000 7f57b95f4700 1 --2- 172.21.15.107:0/591053193 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f57a0039150 0x7f57a003b610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f57a4006fd0 tx=0x7f57a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:10.111 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.241+0000 7f57aaffd700 1 -- 172.21.15.107:0/591053193 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f57b0011e20 con 0x7f57b4106550 2024-04-25T00:07:10.111 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.543+0000 7f57bc059700 1 -- 172.21.15.107:0/591053193 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f57b4062e40 con 0x7f57b4106550 2024-04-25T00:07:10.111 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.545+0000 7f57aaffd700 1 -- 172.21.15.107:0/591053193 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f57b0023390 con 0x7f57b4106550 2024-04-25T00:07:10.112 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.547+0000 7f57bc059700 1 -- 172.21.15.107:0/591053193 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f57a0039150 msgr2=0x7f57a003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:10.112 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.547+0000 7f57bc059700 1 --2- 172.21.15.107:0/591053193 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f57a0039150 0x7f57a003b610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f57a4006fd0 tx=0x7f57a4006e40 comp rx=0 tx=0).stop 2024-04-25T00:07:10.112 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.547+0000 7f57bc059700 1 -- 172.21.15.107:0/591053193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57b4106550 msgr2=0x7f57b419ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:10.112 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.547+0000 7f57bc059700 1 --2- 172.21.15.107:0/591053193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57b4106550 0x7f57b419ddc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f57b00116a0 tx=0x7f57b0011ba0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.112 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.547+0000 7f57bc059700 1 -- 172.21.15.107:0/591053193 shutdown_connections 2024-04-25T00:07:10.112 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.547+0000 7f57bc059700 1 --2- 172.21.15.107:0/591053193 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f57a0039150 0x7f57a003b610 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.112 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.547+0000 7f57bc059700 1 --2- 172.21.15.107:0/591053193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57b4106550 0x7f57b419ddc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:10.112 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.547+0000 7f57bc059700 1 -- 172.21.15.107:0/591053193 >> 172.21.15.107:0/591053193 conn(0x7f57b4102130 msgr2=0x7f57b4194a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:10.112 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.547+0000 7f57bc059700 1 -- 172.21.15.107:0/591053193 shutdown_connections 2024-04-25T00:07:10.112 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:09.547+0000 7f57bc059700 1 -- 172.21.15.107:0/591053193 wait complete. 2024-04-25T00:07:10.112 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T00:07:10.112 INFO:teuthology.orchestra.run.smithi107.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T00:07:10.112 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:07:10.112 INFO:teuthology.orchestra.run.smithi107.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:07:10.113 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:07:10.113 INFO:teuthology.orchestra.run.smithi107.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T00:07:10.113 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:07:10.113 INFO:teuthology.orchestra.run.smithi107.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T00:07:10.113 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:07:10.113 INFO:teuthology.orchestra.run.smithi107.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T00:07:10.113 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:07:10.113 INFO:teuthology.orchestra.run.smithi107.stdout: ceph telemetry on 2024-04-25T00:07:10.113 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:07:10.113 INFO:teuthology.orchestra.run.smithi107.stdout:For more information see: 2024-04-25T00:07:10.113 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:07:10.113 INFO:teuthology.orchestra.run.smithi107.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T00:07:10.113 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:07:10.113 INFO:teuthology.orchestra.run.smithi107.stdout:Bootstrap complete. 2024-04-25T00:07:10.139 INFO:tasks.cephadm:Fetching config... 2024-04-25T00:07:10.140 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T00:07:10.140 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T00:07:10.155 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T00:07:10.155 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T00:07:10.155 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T00:07:10.210 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T00:07:10.210 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T00:07:10.210 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T00:07:10.274 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T00:07:10.274 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T00:07:10.274 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T00:07:10.330 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T00:07:10.331 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtn9okv5SpIZDqnTSq6enC6t7ieT4CePD91sxTABXCeULXaUSZ99+za+yoo0m/bLoN7jOi29Rup10KWcmXZxHFWp0ZCYKW8NktYeAW/SBz5TKY8l5/X4RgGd60BWjY120VXqX0rx4GH/h19SkKdtROJsbzXwQWliL7GSLzQv89nUekIDpkX8jVJXsrIfwfIpObs0tIQJaVQTOZ7LvULInhQ11UCy1KF6uuPxQpHavHjGvb00GqFuXKew3GQ+HeYL6EEZc3b4uUFpZlxK3bIRcueniMl4tVhkOssNHgPTcLMnnoW6mDyipQ1oNJIAsxLQNYcB9ZSnPxx/AOiAW17FgIIemk99K4ayyHbnan495mqDSElj1BM1P4fqzlYePquKavXjDoyFLIdIhAvwEnrMrXgZYOcD1xHWvPbPRIEddyvp41ljqG23jQqVgNosWMgU8WEW6XuIatxbZatKYruqhVP4ylbFUiG/kE9XcGW+mClhFj8nq3NdCM7tmbtub/PeM= ceph-79602f58-0297-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T00:07:10.406 INFO:teuthology.orchestra.run.smithi107.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtn9okv5SpIZDqnTSq6enC6t7ieT4CePD91sxTABXCeULXaUSZ99+za+yoo0m/bLoN7jOi29Rup10KWcmXZxHFWp0ZCYKW8NktYeAW/SBz5TKY8l5/X4RgGd60BWjY120VXqX0rx4GH/h19SkKdtROJsbzXwQWliL7GSLzQv89nUekIDpkX8jVJXsrIfwfIpObs0tIQJaVQTOZ7LvULInhQ11UCy1KF6uuPxQpHavHjGvb00GqFuXKew3GQ+HeYL6EEZc3b4uUFpZlxK3bIRcueniMl4tVhkOssNHgPTcLMnnoW6mDyipQ1oNJIAsxLQNYcB9ZSnPxx/AOiAW17FgIIemk99K4ayyHbnan495mqDSElj1BM1P4fqzlYePquKavXjDoyFLIdIhAvwEnrMrXgZYOcD1xHWvPbPRIEddyvp41ljqG23jQqVgNosWMgU8WEW6XuIatxbZatKYruqhVP4ylbFUiG/kE9XcGW+mClhFj8nq3NdCM7tmbtub/PeM= ceph-79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:07:10.414 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtn9okv5SpIZDqnTSq6enC6t7ieT4CePD91sxTABXCeULXaUSZ99+za+yoo0m/bLoN7jOi29Rup10KWcmXZxHFWp0ZCYKW8NktYeAW/SBz5TKY8l5/X4RgGd60BWjY120VXqX0rx4GH/h19SkKdtROJsbzXwQWliL7GSLzQv89nUekIDpkX8jVJXsrIfwfIpObs0tIQJaVQTOZ7LvULInhQ11UCy1KF6uuPxQpHavHjGvb00GqFuXKew3GQ+HeYL6EEZc3b4uUFpZlxK3bIRcueniMl4tVhkOssNHgPTcLMnnoW6mDyipQ1oNJIAsxLQNYcB9ZSnPxx/AOiAW17FgIIemk99K4ayyHbnan495mqDSElj1BM1P4fqzlYePquKavXjDoyFLIdIhAvwEnrMrXgZYOcD1xHWvPbPRIEddyvp41ljqG23jQqVgNosWMgU8WEW6XuIatxbZatKYruqhVP4ylbFUiG/kE9XcGW+mClhFj8nq3NdCM7tmbtub/PeM= ceph-79602f58-0297-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T00:07:10.455 INFO:teuthology.orchestra.run.smithi170.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtn9okv5SpIZDqnTSq6enC6t7ieT4CePD91sxTABXCeULXaUSZ99+za+yoo0m/bLoN7jOi29Rup10KWcmXZxHFWp0ZCYKW8NktYeAW/SBz5TKY8l5/X4RgGd60BWjY120VXqX0rx4GH/h19SkKdtROJsbzXwQWliL7GSLzQv89nUekIDpkX8jVJXsrIfwfIpObs0tIQJaVQTOZ7LvULInhQ11UCy1KF6uuPxQpHavHjGvb00GqFuXKew3GQ+HeYL6EEZc3b4uUFpZlxK3bIRcueniMl4tVhkOssNHgPTcLMnnoW6mDyipQ1oNJIAsxLQNYcB9ZSnPxx/AOiAW17FgIIemk99K4ayyHbnan495mqDSElj1BM1P4fqzlYePquKavXjDoyFLIdIhAvwEnrMrXgZYOcD1xHWvPbPRIEddyvp41ljqG23jQqVgNosWMgU8WEW6XuIatxbZatKYruqhVP4ylbFUiG/kE9XcGW+mClhFj8nq3NdCM7tmbtub/PeM= ceph-79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:07:10.467 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtn9okv5SpIZDqnTSq6enC6t7ieT4CePD91sxTABXCeULXaUSZ99+za+yoo0m/bLoN7jOi29Rup10KWcmXZxHFWp0ZCYKW8NktYeAW/SBz5TKY8l5/X4RgGd60BWjY120VXqX0rx4GH/h19SkKdtROJsbzXwQWliL7GSLzQv89nUekIDpkX8jVJXsrIfwfIpObs0tIQJaVQTOZ7LvULInhQ11UCy1KF6uuPxQpHavHjGvb00GqFuXKew3GQ+HeYL6EEZc3b4uUFpZlxK3bIRcueniMl4tVhkOssNHgPTcLMnnoW6mDyipQ1oNJIAsxLQNYcB9ZSnPxx/AOiAW17FgIIemk99K4ayyHbnan495mqDSElj1BM1P4fqzlYePquKavXjDoyFLIdIhAvwEnrMrXgZYOcD1xHWvPbPRIEddyvp41ljqG23jQqVgNosWMgU8WEW6XuIatxbZatKYruqhVP4ylbFUiG/kE9XcGW+mClhFj8nq3NdCM7tmbtub/PeM= ceph-79602f58-0297-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T00:07:10.508 INFO:teuthology.orchestra.run.smithi195.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCtn9okv5SpIZDqnTSq6enC6t7ieT4CePD91sxTABXCeULXaUSZ99+za+yoo0m/bLoN7jOi29Rup10KWcmXZxHFWp0ZCYKW8NktYeAW/SBz5TKY8l5/X4RgGd60BWjY120VXqX0rx4GH/h19SkKdtROJsbzXwQWliL7GSLzQv89nUekIDpkX8jVJXsrIfwfIpObs0tIQJaVQTOZ7LvULInhQ11UCy1KF6uuPxQpHavHjGvb00GqFuXKew3GQ+HeYL6EEZc3b4uUFpZlxK3bIRcueniMl4tVhkOssNHgPTcLMnnoW6mDyipQ1oNJIAsxLQNYcB9ZSnPxx/AOiAW17FgIIemk99K4ayyHbnan495mqDSElj1BM1P4fqzlYePquKavXjDoyFLIdIhAvwEnrMrXgZYOcD1xHWvPbPRIEddyvp41ljqG23jQqVgNosWMgU8WEW6XuIatxbZatKYruqhVP4ylbFUiG/kE9XcGW+mClhFj8nq3NdCM7tmbtub/PeM= ceph-79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:07:10.520 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T00:07:10.760 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:07:10.882 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:10 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/591053193' entity='client.admin' 2024-04-25T00:07:12.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.590+0000 7f669c7c9700 1 -- 172.21.15.107:0/553874192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6694101c80 msgr2=0x7f66941020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:12.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.590+0000 7f669c7c9700 1 --2- 172.21.15.107:0/553874192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6694101c80 0x7f66941020a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6690009910 tx=0x7f6690009c20 comp rx=0 tx=0).stop 2024-04-25T00:07:12.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.590+0000 7f669c7c9700 1 -- 172.21.15.107:0/553874192 shutdown_connections 2024-04-25T00:07:12.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.590+0000 7f669c7c9700 1 --2- 172.21.15.107:0/553874192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6694101c80 0x7f66941020a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:12.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.590+0000 7f669c7c9700 1 -- 172.21.15.107:0/553874192 >> 172.21.15.107:0/553874192 conn(0x7f66940fd220 msgr2=0x7f66940ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:12.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.590+0000 7f669c7c9700 1 -- 172.21.15.107:0/553874192 shutdown_connections 2024-04-25T00:07:12.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.590+0000 7f669c7c9700 1 -- 172.21.15.107:0/553874192 wait complete. 2024-04-25T00:07:12.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.591+0000 7f669c7c9700 1 Processor -- start 2024-04-25T00:07:12.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.591+0000 7f669c7c9700 1 -- start start 2024-04-25T00:07:12.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.591+0000 7f669c7c9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6694101c80 0x7f6694197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:12.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.591+0000 7f669c7c9700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6694107d10 con 0x7f6694101c80 2024-04-25T00:07:12.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.591+0000 7f669a565700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6694101c80 0x7f6694197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:12.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.591+0000 7f669a565700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6694101c80 0x7f6694197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43898/0 (socket says 172.21.15.107:43898) 2024-04-25T00:07:12.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.591+0000 7f669a565700 1 -- 172.21.15.107:0/1175609189 learned_addr learned my addr 172.21.15.107:0/1175609189 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:07:12.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.591+0000 7f669a565700 1 -- 172.21.15.107:0/1175609189 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66941978c0 con 0x7f6694101c80 2024-04-25T00:07:12.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.591+0000 7f669a565700 1 --2- 172.21.15.107:0/1175609189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6694101c80 0x7f6694197380 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6690006010 tx=0x7f6690005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:12.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.592+0000 7f668b7fe700 1 -- 172.21.15.107:0/1175609189 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6690027070 con 0x7f6694101c80 2024-04-25T00:07:12.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.592+0000 7f668b7fe700 1 -- 172.21.15.107:0/1175609189 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f669000be40 con 0x7f6694101c80 2024-04-25T00:07:12.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.592+0000 7f669c7c9700 1 -- 172.21.15.107:0/1175609189 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6694197b50 con 0x7f6694101c80 2024-04-25T00:07:12.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.592+0000 7f668b7fe700 1 -- 172.21.15.107:0/1175609189 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f669000f460 con 0x7f6694101c80 2024-04-25T00:07:12.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.592+0000 7f669c7c9700 1 -- 172.21.15.107:0/1175609189 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f669419a840 con 0x7f6694101c80 2024-04-25T00:07:12.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.592+0000 7f668b7fe700 1 -- 172.21.15.107:0/1175609189 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f669002a070 con 0x7f6694101c80 2024-04-25T00:07:12.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.593+0000 7f669c7c9700 1 -- 172.21.15.107:0/1175609189 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6694108490 con 0x7f6694101c80 2024-04-25T00:07:12.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.593+0000 7f668b7fe700 1 --2- 172.21.15.107:0/1175609189 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f66800390f0 0x7f668003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:12.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.593+0000 7f668b7fe700 1 -- 172.21.15.107:0/1175609189 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f669001e030 con 0x7f6694101c80 2024-04-25T00:07:12.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.593+0000 7f6699d64700 1 --2- 172.21.15.107:0/1175609189 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f66800390f0 0x7f668003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:12.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.593+0000 7f6699d64700 1 --2- 172.21.15.107:0/1175609189 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f66800390f0 0x7f668003b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6684006fd0 tx=0x7f6684006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:12.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.598+0000 7f668b7fe700 1 -- 172.21.15.107:0/1175609189 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6690013070 con 0x7f6694101c80 2024-04-25T00:07:12.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.800+0000 7f669c7c9700 1 -- 172.21.15.107:0/1175609189 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f669402dc40 con 0x7f6694101c80 2024-04-25T00:07:12.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.803+0000 7f668b7fe700 1 -- 172.21.15.107:0/1175609189 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f6690031070 con 0x7f6694101c80 2024-04-25T00:07:12.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.805+0000 7f669c7c9700 1 -- 172.21.15.107:0/1175609189 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f66800390f0 msgr2=0x7f668003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:12.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.805+0000 7f669c7c9700 1 --2- 172.21.15.107:0/1175609189 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f66800390f0 0x7f668003b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6684006fd0 tx=0x7f6684006e40 comp rx=0 tx=0).stop 2024-04-25T00:07:12.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.805+0000 7f669c7c9700 1 -- 172.21.15.107:0/1175609189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6694101c80 msgr2=0x7f6694197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:12.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.805+0000 7f669c7c9700 1 --2- 172.21.15.107:0/1175609189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6694101c80 0x7f6694197380 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6690006010 tx=0x7f6690005050 comp rx=0 tx=0).stop 2024-04-25T00:07:12.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.805+0000 7f669c7c9700 1 -- 172.21.15.107:0/1175609189 shutdown_connections 2024-04-25T00:07:12.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.805+0000 7f669c7c9700 1 --2- 172.21.15.107:0/1175609189 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f66800390f0 0x7f668003b5b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:12.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.805+0000 7f669c7c9700 1 --2- 172.21.15.107:0/1175609189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6694101c80 0x7f6694197380 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:12.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.805+0000 7f669c7c9700 1 -- 172.21.15.107:0/1175609189 >> 172.21.15.107:0/1175609189 conn(0x7f66940fd220 msgr2=0x7f66940fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:12.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.805+0000 7f669c7c9700 1 -- 172.21.15.107:0/1175609189 shutdown_connections 2024-04-25T00:07:12.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:12.805+0000 7f669c7c9700 1 -- 172.21.15.107:0/1175609189 wait complete. 2024-04-25T00:07:13.586 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T00:07:13.586 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T00:07:13.819 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:07:13.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:13 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1175609189' entity='client.admin' 2024-04-25T00:07:13.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:13 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:13.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:13 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:13.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:13 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:07:13.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:13 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:13.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:13 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:07:13.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:13 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:15.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.274+0000 7f8a63472700 1 -- 172.21.15.107:0/2605588328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a5c1027c0 msgr2=0x7f8a5c104bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:15.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.274+0000 7f8a63472700 1 --2- 172.21.15.107:0/2605588328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a5c1027c0 0x7f8a5c104bb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009910 tx=0x7f8a58009c20 comp rx=0 tx=0).stop 2024-04-25T00:07:15.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.274+0000 7f8a63472700 1 -- 172.21.15.107:0/2605588328 shutdown_connections 2024-04-25T00:07:15.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.274+0000 7f8a63472700 1 --2- 172.21.15.107:0/2605588328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a5c1027c0 0x7f8a5c104bb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.274+0000 7f8a63472700 1 -- 172.21.15.107:0/2605588328 >> 172.21.15.107:0/2605588328 conn(0x7f8a5c0fc130 msgr2=0x7f8a5c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:15.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.274+0000 7f8a63472700 1 -- 172.21.15.107:0/2605588328 shutdown_connections 2024-04-25T00:07:15.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.274+0000 7f8a63472700 1 -- 172.21.15.107:0/2605588328 wait complete. 2024-04-25T00:07:15.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.274+0000 7f8a63472700 1 Processor -- start 2024-04-25T00:07:15.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.274+0000 7f8a63472700 1 -- start start 2024-04-25T00:07:15.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.275+0000 7f8a63472700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a5c1027c0 0x7f8a5c102040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.275+0000 7f8a63472700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c107d40 con 0x7f8a5c1027c0 2024-04-25T00:07:15.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.275+0000 7f8a6120e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a5c1027c0 0x7f8a5c102040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.275+0000 7f8a6120e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a5c1027c0 0x7f8a5c102040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43904/0 (socket says 172.21.15.107:43904) 2024-04-25T00:07:15.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.275+0000 7f8a6120e700 1 -- 172.21.15.107:0/1315510667 learned_addr learned my addr 172.21.15.107:0/1315510667 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:07:15.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.275+0000 7f8a6120e700 1 -- 172.21.15.107:0/1315510667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5c102580 con 0x7f8a5c1027c0 2024-04-25T00:07:15.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.275+0000 7f8a6120e700 1 --2- 172.21.15.107:0/1315510667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a5c1027c0 0x7f8a5c102040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8a58005b40 tx=0x7f8a58005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:15.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.275+0000 7f8a527fc700 1 -- 172.21.15.107:0/1315510667 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a58027070 con 0x7f8a5c1027c0 2024-04-25T00:07:15.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.275+0000 7f8a527fc700 1 -- 172.21.15.107:0/1315510667 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a5800be40 con 0x7f8a5c1027c0 2024-04-25T00:07:15.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.275+0000 7f8a63472700 1 -- 172.21.15.107:0/1315510667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5c100770 con 0x7f8a5c1027c0 2024-04-25T00:07:15.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.275+0000 7f8a527fc700 1 -- 172.21.15.107:0/1315510667 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a5800f460 con 0x7f8a5c1027c0 2024-04-25T00:07:15.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.275+0000 7f8a63472700 1 -- 172.21.15.107:0/1315510667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a5c100cb0 con 0x7f8a5c1027c0 2024-04-25T00:07:15.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.276+0000 7f8a527fc700 1 -- 172.21.15.107:0/1315510667 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f8a5802a070 con 0x7f8a5c1027c0 2024-04-25T00:07:15.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.276+0000 7f8a63472700 1 -- 172.21.15.107:0/1315510667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a5c1084c0 con 0x7f8a5c1027c0 2024-04-25T00:07:15.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.276+0000 7f8a527fc700 1 --2- 172.21.15.107:0/1315510667 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8a480390f0 0x7f8a4803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:15.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.276+0000 7f8a527fc700 1 -- 172.21.15.107:0/1315510667 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8a5801e030 con 0x7f8a5c1027c0 2024-04-25T00:07:15.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.276+0000 7f8a60a0d700 1 --2- 172.21.15.107:0/1315510667 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8a480390f0 0x7f8a4803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:15.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.277+0000 7f8a60a0d700 1 --2- 172.21.15.107:0/1315510667 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8a480390f0 0x7f8a4803b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c006fd0 tx=0x7f8a4c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:15.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.281+0000 7f8a527fc700 1 -- 172.21.15.107:0/1315510667 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8a58013070 con 0x7f8a5c1027c0 2024-04-25T00:07:15.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.480+0000 7f8a63472700 1 -- 172.21.15.107:0/1315510667 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f8a5c190850 con 0x7f8a480390f0 2024-04-25T00:07:15.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.484+0000 7f8a527fc700 1 -- 172.21.15.107:0/1315510667 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8a5c190850 con 0x7f8a480390f0 2024-04-25T00:07:15.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.486+0000 7f8a63472700 1 -- 172.21.15.107:0/1315510667 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8a480390f0 msgr2=0x7f8a4803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:15.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.486+0000 7f8a63472700 1 --2- 172.21.15.107:0/1315510667 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8a480390f0 0x7f8a4803b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c006fd0 tx=0x7f8a4c006e40 comp rx=0 tx=0).stop 2024-04-25T00:07:15.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.486+0000 7f8a63472700 1 -- 172.21.15.107:0/1315510667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a5c1027c0 msgr2=0x7f8a5c102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:15.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.486+0000 7f8a63472700 1 --2- 172.21.15.107:0/1315510667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a5c1027c0 0x7f8a5c102040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8a58005b40 tx=0x7f8a58005030 comp rx=0 tx=0).stop 2024-04-25T00:07:15.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.486+0000 7f8a63472700 1 -- 172.21.15.107:0/1315510667 shutdown_connections 2024-04-25T00:07:15.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.486+0000 7f8a63472700 1 --2- 172.21.15.107:0/1315510667 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8a480390f0 0x7f8a4803b5b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.486+0000 7f8a63472700 1 --2- 172.21.15.107:0/1315510667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a5c1027c0 0x7f8a5c102040 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:15.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.486+0000 7f8a63472700 1 -- 172.21.15.107:0/1315510667 >> 172.21.15.107:0/1315510667 conn(0x7f8a5c0fc130 msgr2=0x7f8a5c0fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:15.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.486+0000 7f8a63472700 1 -- 172.21.15.107:0/1315510667 shutdown_connections 2024-04-25T00:07:15.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:15.486+0000 7f8a63472700 1 -- 172.21.15.107:0/1315510667 wait complete. 2024-04-25T00:07:16.262 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi170 2024-04-25T00:07:16.262 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-04-25T00:07:16.262 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/etc/ceph/ceph.conf 2024-04-25T00:07:16.281 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-04-25T00:07:16.281 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:07:16.338 INFO:tasks.cephadm:Adding host smithi170 to orchestrator... 2024-04-25T00:07:16.339 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph orch host add smithi170 2024-04-25T00:07:16.575 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:07:16.675 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:16 smithi107 ceph-mon[29153]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:07:16.675 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:16 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:16.675 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:16 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:07:16.675 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:16 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:07:16.675 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:16 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:07:16.675 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:16 smithi107 ceph-mon[29153]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-25T00:07:16.675 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:16 smithi107 ceph-mon[29153]: Updating smithi107:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:07:16.675 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:16 smithi107 ceph-mon[29153]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:07:16.675 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:16 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:16.676 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:16 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:16.676 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:16 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:17.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:17 smithi107 ceph-mon[29153]: Updating smithi107:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:07:18.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.018+0000 7f8743bc7700 1 -- 172.21.15.107:0/2185739359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c102dd0 msgr2=0x7f873c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:18.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.018+0000 7f8743bc7700 1 --2- 172.21.15.107:0/2185739359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c102dd0 0x7f873c1031b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8738009910 tx=0x7f8738009c20 comp rx=0 tx=0).stop 2024-04-25T00:07:18.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.018+0000 7f8743bc7700 1 -- 172.21.15.107:0/2185739359 shutdown_connections 2024-04-25T00:07:18.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.018+0000 7f8743bc7700 1 --2- 172.21.15.107:0/2185739359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c102dd0 0x7f873c1031b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:18.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.018+0000 7f8743bc7700 1 -- 172.21.15.107:0/2185739359 >> 172.21.15.107:0/2185739359 conn(0x7f873c077800 msgr2=0x7f873c0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:18.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.018+0000 7f8743bc7700 1 -- 172.21.15.107:0/2185739359 shutdown_connections 2024-04-25T00:07:18.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.018+0000 7f8743bc7700 1 -- 172.21.15.107:0/2185739359 wait complete. 2024-04-25T00:07:18.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.019+0000 7f8743bc7700 1 Processor -- start 2024-04-25T00:07:18.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.019+0000 7f8743bc7700 1 -- start start 2024-04-25T00:07:18.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.019+0000 7f8743bc7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c102dd0 0x7f873c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:18.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.019+0000 7f8743bc7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f873c10c2c0 con 0x7f873c102dd0 2024-04-25T00:07:18.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.019+0000 7f8741963700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c102dd0 0x7f873c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:18.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.019+0000 7f8741963700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c102dd0 0x7f873c19d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43922/0 (socket says 172.21.15.107:43922) 2024-04-25T00:07:18.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.019+0000 7f8741963700 1 -- 172.21.15.107:0/697383367 learned_addr learned my addr 172.21.15.107:0/697383367 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:07:18.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.019+0000 7f8741963700 1 -- 172.21.15.107:0/697383367 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f873c19d9a0 con 0x7f873c102dd0 2024-04-25T00:07:18.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.020+0000 7f8741963700 1 --2- 172.21.15.107:0/697383367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c102dd0 0x7f873c19d460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8738006010 tx=0x7f8738005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:18.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.020+0000 7f8732ffd700 1 -- 172.21.15.107:0/697383367 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8738026070 con 0x7f873c102dd0 2024-04-25T00:07:18.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.020+0000 7f8732ffd700 1 -- 172.21.15.107:0/697383367 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f873800bc30 con 0x7f873c102dd0 2024-04-25T00:07:18.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.020+0000 7f8732ffd700 1 -- 172.21.15.107:0/697383367 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87380039a0 con 0x7f873c102dd0 2024-04-25T00:07:18.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.020+0000 7f8743bc7700 1 -- 172.21.15.107:0/697383367 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f873c19dc30 con 0x7f873c102dd0 2024-04-25T00:07:18.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.020+0000 7f8743bc7700 1 -- 172.21.15.107:0/697383367 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f873c197620 con 0x7f873c102dd0 2024-04-25T00:07:18.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.021+0000 7f8732ffd700 1 -- 172.21.15.107:0/697383367 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f8738029070 con 0x7f873c102dd0 2024-04-25T00:07:18.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.021+0000 7f8743bc7700 1 -- 172.21.15.107:0/697383367 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f873c069f40 con 0x7f873c102dd0 2024-04-25T00:07:18.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.021+0000 7f8732ffd700 1 --2- 172.21.15.107:0/697383367 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f87280390f0 0x7f872803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:18.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.021+0000 7f8732ffd700 1 -- 172.21.15.107:0/697383367 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f873801e070 con 0x7f873c102dd0 2024-04-25T00:07:18.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.021+0000 7f8741162700 1 --2- 172.21.15.107:0/697383367 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f87280390f0 0x7f872803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:18.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.021+0000 7f8741162700 1 --2- 172.21.15.107:0/697383367 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f87280390f0 0x7f872803b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f872c006fd0 tx=0x7f872c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:18.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.026+0000 7f8732ffd700 1 -- 172.21.15.107:0/697383367 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8738013070 con 0x7f873c102dd0 2024-04-25T00:07:18.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:18.225+0000 7f8743bc7700 1 -- 172.21.15.107:0/697383367 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}) -- 0x7f873c198b50 con 0x7f87280390f0 2024-04-25T00:07:19.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:19 smithi107 ceph-mon[29153]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:07:19.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:19 smithi107 ceph-mon[29153]: Deploying cephadm binary to smithi170 2024-04-25T00:07:22.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:21 smithi107 ceph-mon[29153]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:22.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:22.427+0000 7f8732ffd700 1 -- 172.21.15.107:0/697383367 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f873c198b50 con 0x7f87280390f0 2024-04-25T00:07:22.429 INFO:teuthology.orchestra.run.smithi107.stdout:Added host 'smithi170' with addr '172.21.15.170' 2024-04-25T00:07:22.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:22.430+0000 7f8743bc7700 1 -- 172.21.15.107:0/697383367 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f87280390f0 msgr2=0x7f872803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:22.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:22.430+0000 7f8743bc7700 1 --2- 172.21.15.107:0/697383367 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f87280390f0 0x7f872803b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f872c006fd0 tx=0x7f872c006e40 comp rx=0 tx=0).stop 2024-04-25T00:07:22.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:22.430+0000 7f8743bc7700 1 -- 172.21.15.107:0/697383367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c102dd0 msgr2=0x7f873c19d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:22.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:22.430+0000 7f8743bc7700 1 --2- 172.21.15.107:0/697383367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c102dd0 0x7f873c19d460 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8738006010 tx=0x7f8738005030 comp rx=0 tx=0).stop 2024-04-25T00:07:22.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:22.430+0000 7f8743bc7700 1 -- 172.21.15.107:0/697383367 shutdown_connections 2024-04-25T00:07:22.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:22.430+0000 7f8743bc7700 1 --2- 172.21.15.107:0/697383367 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f87280390f0 0x7f872803b5b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:22.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:22.430+0000 7f8743bc7700 1 --2- 172.21.15.107:0/697383367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f873c102dd0 0x7f873c19d460 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:22.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:22.430+0000 7f8743bc7700 1 -- 172.21.15.107:0/697383367 >> 172.21.15.107:0/697383367 conn(0x7f873c077800 msgr2=0x7f873c103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:22.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:22.430+0000 7f8743bc7700 1 -- 172.21.15.107:0/697383367 shutdown_connections 2024-04-25T00:07:22.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:22.430+0000 7f8743bc7700 1 -- 172.21.15.107:0/697383367 wait complete. 2024-04-25T00:07:23.007 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T00:07:23.239 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:07:23.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:23 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:23.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:23 smithi107 ceph-mon[29153]: Added host smithi170 2024-04-25T00:07:23.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:23 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:07:23.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:23 smithi107 ceph-mon[29153]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:23.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:23 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:24.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:24 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:24.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.779+0000 7f3392f73700 1 -- 172.21.15.107:0/3064969484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f338c101ea0 msgr2=0x7f338c102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:24.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.779+0000 7f3392f73700 1 --2- 172.21.15.107:0/3064969484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f338c101ea0 0x7f338c102280 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3380009910 tx=0x7f3380009c20 comp rx=0 tx=0).stop 2024-04-25T00:07:24.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.779+0000 7f3392f73700 1 -- 172.21.15.107:0/3064969484 shutdown_connections 2024-04-25T00:07:24.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.779+0000 7f3392f73700 1 --2- 172.21.15.107:0/3064969484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f338c101ea0 0x7f338c102280 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.779+0000 7f3392f73700 1 -- 172.21.15.107:0/3064969484 >> 172.21.15.107:0/3064969484 conn(0x7f338c0fd750 msgr2=0x7f338c0ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:24.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.779+0000 7f3392f73700 1 -- 172.21.15.107:0/3064969484 shutdown_connections 2024-04-25T00:07:24.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.779+0000 7f3392f73700 1 -- 172.21.15.107:0/3064969484 wait complete. 2024-04-25T00:07:24.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.780+0000 7f3392f73700 1 Processor -- start 2024-04-25T00:07:24.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.780+0000 7f3392f73700 1 -- start start 2024-04-25T00:07:24.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.780+0000 7f3392f73700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f338c101ea0 0x7f338c19d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.780+0000 7f3392f73700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f338c10c270 con 0x7f338c101ea0 2024-04-25T00:07:24.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.780+0000 7f3390d0f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f338c101ea0 0x7f338c19d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:24.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.780+0000 7f3390d0f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f338c101ea0 0x7f338c19d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54602/0 (socket says 172.21.15.107:54602) 2024-04-25T00:07:24.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.780+0000 7f3390d0f700 1 -- 172.21.15.107:0/304266029 learned_addr learned my addr 172.21.15.107:0/304266029 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:07:24.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.780+0000 7f3390d0f700 1 -- 172.21.15.107:0/304266029 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338c19d910 con 0x7f338c101ea0 2024-04-25T00:07:24.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.780+0000 7f3390d0f700 1 --2- 172.21.15.107:0/304266029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f338c101ea0 0x7f338c19d3d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3380006010 tx=0x7f3380005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:24.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.781+0000 7f3389ffb700 1 -- 172.21.15.107:0/304266029 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3380026070 con 0x7f338c101ea0 2024-04-25T00:07:24.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.781+0000 7f3392f73700 1 -- 172.21.15.107:0/304266029 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f338c19dc00 con 0x7f338c101ea0 2024-04-25T00:07:24.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.781+0000 7f3389ffb700 1 -- 172.21.15.107:0/304266029 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f338000bbd0 con 0x7f338c101ea0 2024-04-25T00:07:24.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.781+0000 7f3389ffb700 1 -- 172.21.15.107:0/304266029 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3380003910 con 0x7f338c101ea0 2024-04-25T00:07:24.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.781+0000 7f3392f73700 1 -- 172.21.15.107:0/304266029 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f338c197590 con 0x7f338c101ea0 2024-04-25T00:07:24.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.782+0000 7f3389ffb700 1 -- 172.21.15.107:0/304266029 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f3380029070 con 0x7f338c101ea0 2024-04-25T00:07:24.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.782+0000 7f3392f73700 1 -- 172.21.15.107:0/304266029 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f338c103380 con 0x7f338c101ea0 2024-04-25T00:07:24.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.782+0000 7f3389ffb700 1 --2- 172.21.15.107:0/304266029 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f337c0390f0 0x7f337c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.782+0000 7f3389ffb700 1 -- 172.21.15.107:0/304266029 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f338001e070 con 0x7f338c101ea0 2024-04-25T00:07:24.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.782+0000 7f338bfff700 1 --2- 172.21.15.107:0/304266029 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f337c0390f0 0x7f337c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:24.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.782+0000 7f338bfff700 1 --2- 172.21.15.107:0/304266029 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f337c0390f0 0x7f337c03b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3378006fd0 tx=0x7f3378006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:24.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.787+0000 7f3389ffb700 1 -- 172.21.15.107:0/304266029 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3380013070 con 0x7f338c101ea0 2024-04-25T00:07:24.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.989+0000 7f3392f73700 1 -- 172.21.15.107:0/304266029 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f338c198ac0 con 0x7f337c0390f0 2024-04-25T00:07:24.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.990+0000 7f3389ffb700 1 -- 172.21.15.107:0/304266029 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+161 (secure 0 0 0) 0x7f338c198ac0 con 0x7f337c0390f0 2024-04-25T00:07:24.992 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:07:24.992 INFO:teuthology.orchestra.run.smithi107.stdout:[{"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}, {"addr": "172.21.15.170", "hostname": "smithi170", "labels": [], "status": ""}] 2024-04-25T00:07:24.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.992+0000 7f3392f73700 1 -- 172.21.15.107:0/304266029 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f337c0390f0 msgr2=0x7f337c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:24.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.992+0000 7f3392f73700 1 --2- 172.21.15.107:0/304266029 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f337c0390f0 0x7f337c03b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3378006fd0 tx=0x7f3378006e40 comp rx=0 tx=0).stop 2024-04-25T00:07:24.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.992+0000 7f3392f73700 1 -- 172.21.15.107:0/304266029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f338c101ea0 msgr2=0x7f338c19d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:24.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.992+0000 7f3392f73700 1 --2- 172.21.15.107:0/304266029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f338c101ea0 0x7f338c19d3d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3380006010 tx=0x7f3380005030 comp rx=0 tx=0).stop 2024-04-25T00:07:24.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.992+0000 7f3392f73700 1 -- 172.21.15.107:0/304266029 shutdown_connections 2024-04-25T00:07:24.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.992+0000 7f3392f73700 1 --2- 172.21.15.107:0/304266029 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f337c0390f0 0x7f337c03b5b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.992+0000 7f3392f73700 1 --2- 172.21.15.107:0/304266029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f338c101ea0 0x7f338c19d3d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.992+0000 7f3392f73700 1 -- 172.21.15.107:0/304266029 >> 172.21.15.107:0/304266029 conn(0x7f338c0fd750 msgr2=0x7f338c104c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:24.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.992+0000 7f3392f73700 1 -- 172.21.15.107:0/304266029 shutdown_connections 2024-04-25T00:07:24.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:24.992+0000 7f3392f73700 1 -- 172.21.15.107:0/304266029 wait complete. 2024-04-25T00:07:25.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:25 smithi107 ceph-mon[29153]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:25.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:25 smithi107 ceph-mon[29153]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:07:25.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:25 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:25.701 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi195 2024-04-25T00:07:25.701 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-04-25T00:07:25.701 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/etc/ceph/ceph.conf 2024-04-25T00:07:25.719 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-04-25T00:07:25.720 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:07:25.777 INFO:tasks.cephadm:Adding host smithi195 to orchestrator... 2024-04-25T00:07:25.777 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph orch host add smithi195 2024-04-25T00:07:26.009 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:07:27.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.418+0000 7f4165e65700 1 -- 172.21.15.107:0/814435831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4160106be0 msgr2=0x7f4160106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.418+0000 7f4165e65700 1 --2- 172.21.15.107:0/814435831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4160106be0 0x7f4160106fc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4150009910 tx=0x7f4150009c20 comp rx=0 tx=0).stop 2024-04-25T00:07:27.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.418+0000 7f4165e65700 1 -- 172.21.15.107:0/814435831 shutdown_connections 2024-04-25T00:07:27.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.418+0000 7f4165e65700 1 --2- 172.21.15.107:0/814435831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4160106be0 0x7f4160106fc0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.418+0000 7f4165e65700 1 -- 172.21.15.107:0/814435831 >> 172.21.15.107:0/814435831 conn(0x7f4160077800 msgr2=0x7f41600feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:27.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.418+0000 7f4165e65700 1 -- 172.21.15.107:0/814435831 shutdown_connections 2024-04-25T00:07:27.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.418+0000 7f4165e65700 1 -- 172.21.15.107:0/814435831 wait complete. 2024-04-25T00:07:27.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.418+0000 7f4165e65700 1 Processor -- start 2024-04-25T00:07:27.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.418+0000 7f4165e65700 1 -- start start 2024-04-25T00:07:27.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.419+0000 7f4165e65700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4160106be0 0x7f41601965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.419+0000 7f4165e65700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f416010b1a0 con 0x7f4160106be0 2024-04-25T00:07:27.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.419+0000 7f415f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4160106be0 0x7f41601965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.419+0000 7f415f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4160106be0 0x7f41601965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54618/0 (socket says 172.21.15.107:54618) 2024-04-25T00:07:27.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.419+0000 7f415f7fe700 1 -- 172.21.15.107:0/1432512143 learned_addr learned my addr 172.21.15.107:0/1432512143 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:07:27.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.419+0000 7f415f7fe700 1 -- 172.21.15.107:0/1432512143 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4160196b20 con 0x7f4160106be0 2024-04-25T00:07:27.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.419+0000 7f415f7fe700 1 --2- 172.21.15.107:0/1432512143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4160106be0 0x7f41601965e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4150006010 tx=0x7f4150005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:27.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.419+0000 7f415d7fa700 1 -- 172.21.15.107:0/1432512143 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4150026070 con 0x7f4160106be0 2024-04-25T00:07:27.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.419+0000 7f4165e65700 1 -- 172.21.15.107:0/1432512143 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4160196db0 con 0x7f4160106be0 2024-04-25T00:07:27.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.419+0000 7f415d7fa700 1 -- 172.21.15.107:0/1432512143 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f415000bc30 con 0x7f4160106be0 2024-04-25T00:07:27.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.419+0000 7f415d7fa700 1 -- 172.21.15.107:0/1432512143 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41500039d0 con 0x7f4160106be0 2024-04-25T00:07:27.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.420+0000 7f4165e65700 1 -- 172.21.15.107:0/1432512143 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416019aeb0 con 0x7f4160106be0 2024-04-25T00:07:27.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.421+0000 7f415d7fa700 1 -- 172.21.15.107:0/1432512143 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f4150029070 con 0x7f4160106be0 2024-04-25T00:07:27.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.421+0000 7f4165e65700 1 -- 172.21.15.107:0/1432512143 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41601021e0 con 0x7f4160106be0 2024-04-25T00:07:27.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.421+0000 7f415d7fa700 1 --2- 172.21.15.107:0/1432512143 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f414c0390f0 0x7f414c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.421+0000 7f415d7fa700 1 -- 172.21.15.107:0/1432512143 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f415001e070 con 0x7f4160106be0 2024-04-25T00:07:27.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.421+0000 7f4156ffd700 1 --2- 172.21.15.107:0/1432512143 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f414c0390f0 0x7f414c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.421+0000 7f4156ffd700 1 --2- 172.21.15.107:0/1432512143 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f414c0390f0 0x7f414c03b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4148006fd0 tx=0x7f4148006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:27.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.426+0000 7f415d7fa700 1 -- 172.21.15.107:0/1432512143 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4150013070 con 0x7f4160106be0 2024-04-25T00:07:27.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:27.626+0000 7f4165e65700 1 -- 172.21.15.107:0/1432512143 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi195", "target": ["mon-mgr", ""]}) -- 0x7f4160197940 con 0x7f414c0390f0 2024-04-25T00:07:28.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:27 smithi107 ceph-mon[29153]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:29.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:28 smithi107 ceph-mon[29153]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi195", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:07:29.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:28 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:29.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:28 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:29.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:28 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:29.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:28 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:29.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:28 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:07:29.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:28 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:07:29.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:28 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:07:30.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:29 smithi107 ceph-mon[29153]: Updating smithi170:/etc/ceph/ceph.conf 2024-04-25T00:07:30.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:29 smithi107 ceph-mon[29153]: Deploying cephadm binary to smithi195 2024-04-25T00:07:30.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:29 smithi107 ceph-mon[29153]: Updating smithi170:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:07:30.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:29 smithi107 ceph-mon[29153]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:07:30.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:29 smithi107 ceph-mon[29153]: Updating smithi170:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:07:30.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:29 smithi107 ceph-mon[29153]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:31.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:31.819+0000 7f415d7fa700 1 -- 172.21.15.107:0/1432512143 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f4160197940 con 0x7f414c0390f0 2024-04-25T00:07:31.819 INFO:teuthology.orchestra.run.smithi107.stdout:Added host 'smithi195' with addr '172.21.15.195' 2024-04-25T00:07:31.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:31.821+0000 7f4165e65700 1 -- 172.21.15.107:0/1432512143 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f414c0390f0 msgr2=0x7f414c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:31.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:31.821+0000 7f4165e65700 1 --2- 172.21.15.107:0/1432512143 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f414c0390f0 0x7f414c03b5b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4148006fd0 tx=0x7f4148006e40 comp rx=0 tx=0).stop 2024-04-25T00:07:31.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:31.821+0000 7f4165e65700 1 -- 172.21.15.107:0/1432512143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4160106be0 msgr2=0x7f41601965e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:31.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:31.821+0000 7f4165e65700 1 --2- 172.21.15.107:0/1432512143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4160106be0 0x7f41601965e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4150006010 tx=0x7f4150005030 comp rx=0 tx=0).stop 2024-04-25T00:07:31.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:31.821+0000 7f4165e65700 1 -- 172.21.15.107:0/1432512143 shutdown_connections 2024-04-25T00:07:31.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:31.821+0000 7f4165e65700 1 --2- 172.21.15.107:0/1432512143 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f414c0390f0 0x7f414c03b5b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:31.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:31.821+0000 7f4165e65700 1 --2- 172.21.15.107:0/1432512143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4160106be0 0x7f41601965e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:31.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:31.821+0000 7f4165e65700 1 -- 172.21.15.107:0/1432512143 >> 172.21.15.107:0/1432512143 conn(0x7f4160077800 msgr2=0x7f41600fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:31.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:31.821+0000 7f4165e65700 1 -- 172.21.15.107:0/1432512143 shutdown_connections 2024-04-25T00:07:31.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:31.821+0000 7f4165e65700 1 -- 172.21.15.107:0/1432512143 wait complete. 2024-04-25T00:07:32.125 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:31 smithi107 ceph-mon[29153]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:32.125 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:31 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:32.125 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:31 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:32.125 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:31 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:32.125 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:31 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:32.125 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:31 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:07:32.421 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T00:07:32.653 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:07:33.095 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:32 smithi107 ceph-mon[29153]: Added host smithi195 2024-04-25T00:07:33.095 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:32 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:33.095 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:32 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:33.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:33 smithi107 ceph-mon[29153]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:34.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.168+0000 7fe02ce12700 1 -- 172.21.15.107:0/3672279587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe028106c00 msgr2=0x7fe028106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:34.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.168+0000 7fe02ce12700 1 --2- 172.21.15.107:0/3672279587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe028106c00 0x7fe028106fe0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe018009910 tx=0x7fe018009c20 comp rx=0 tx=0).stop 2024-04-25T00:07:34.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.169+0000 7fe02ce12700 1 -- 172.21.15.107:0/3672279587 shutdown_connections 2024-04-25T00:07:34.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.169+0000 7fe02ce12700 1 --2- 172.21.15.107:0/3672279587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe028106c00 0x7fe028106fe0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:34.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.169+0000 7fe02ce12700 1 -- 172.21.15.107:0/3672279587 >> 172.21.15.107:0/3672279587 conn(0x7fe0280fc680 msgr2=0x7fe0280feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:34.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.169+0000 7fe02ce12700 1 -- 172.21.15.107:0/3672279587 shutdown_connections 2024-04-25T00:07:34.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.169+0000 7fe02ce12700 1 -- 172.21.15.107:0/3672279587 wait complete. 2024-04-25T00:07:34.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.169+0000 7fe02ce12700 1 Processor -- start 2024-04-25T00:07:34.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.169+0000 7fe02ce12700 1 -- start start 2024-04-25T00:07:34.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.170+0000 7fe02ce12700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe028106c00 0x7fe0281965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:34.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.170+0000 7fe02ce12700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe02810b1c0 con 0x7fe028106c00 2024-04-25T00:07:34.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.170+0000 7fe02659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe028106c00 0x7fe0281965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:34.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.170+0000 7fe02659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe028106c00 0x7fe0281965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41874/0 (socket says 172.21.15.107:41874) 2024-04-25T00:07:34.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.170+0000 7fe02659c700 1 -- 172.21.15.107:0/3392129041 learned_addr learned my addr 172.21.15.107:0/3392129041 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:07:34.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.170+0000 7fe02659c700 1 -- 172.21.15.107:0/3392129041 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe028196b20 con 0x7fe028106c00 2024-04-25T00:07:34.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.170+0000 7fe02659c700 1 --2- 172.21.15.107:0/3392129041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe028106c00 0x7fe0281965e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe018006010 tx=0x7fe018005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:34.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.170+0000 7fe0177fe700 1 -- 172.21.15.107:0/3392129041 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe018026070 con 0x7fe028106c00 2024-04-25T00:07:34.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.170+0000 7fe0177fe700 1 -- 172.21.15.107:0/3392129041 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe01800bc30 con 0x7fe028106c00 2024-04-25T00:07:34.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.170+0000 7fe02ce12700 1 -- 172.21.15.107:0/3392129041 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe028196db0 con 0x7fe028106c00 2024-04-25T00:07:34.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.170+0000 7fe0177fe700 1 -- 172.21.15.107:0/3392129041 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0180039d0 con 0x7fe028106c00 2024-04-25T00:07:34.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.170+0000 7fe02ce12700 1 -- 172.21.15.107:0/3392129041 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe02819aeb0 con 0x7fe028106c00 2024-04-25T00:07:34.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.171+0000 7fe0177fe700 1 -- 172.21.15.107:0/3392129041 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fe018029070 con 0x7fe028106c00 2024-04-25T00:07:34.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.171+0000 7fe0177fe700 1 --2- 172.21.15.107:0/3392129041 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe01003b360 0x7fe01003d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:34.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.171+0000 7fe0177fe700 1 -- 172.21.15.107:0/3392129041 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe01801e070 con 0x7fe028106c00 2024-04-25T00:07:34.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.171+0000 7fe02ce12700 1 -- 172.21.15.107:0/3392129041 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe028102200 con 0x7fe028106c00 2024-04-25T00:07:34.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.171+0000 7fe025d9b700 1 --2- 172.21.15.107:0/3392129041 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe01003b360 0x7fe01003d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:34.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.172+0000 7fe025d9b700 1 --2- 172.21.15.107:0/3392129041 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe01003b360 0x7fe01003d820 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe01c006fd0 tx=0x7fe01c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:34.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.176+0000 7fe0177fe700 1 -- 172.21.15.107:0/3392129041 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe018030070 con 0x7fe028106c00 2024-04-25T00:07:34.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.376+0000 7fe02ce12700 1 -- 172.21.15.107:0/3392129041 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe028197940 con 0x7fe01003b360 2024-04-25T00:07:34.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.377+0000 7fe0177fe700 1 -- 172.21.15.107:0/3392129041 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+241 (secure 0 0 0) 0x7fe028197940 con 0x7fe01003b360 2024-04-25T00:07:34.378 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:07:34.378 INFO:teuthology.orchestra.run.smithi107.stdout:[{"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}, {"addr": "172.21.15.170", "hostname": "smithi170", "labels": [], "status": ""}, {"addr": "172.21.15.195", "hostname": "smithi195", "labels": [], "status": ""}] 2024-04-25T00:07:34.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.379+0000 7fe02ce12700 1 -- 172.21.15.107:0/3392129041 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe01003b360 msgr2=0x7fe01003d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:34.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.379+0000 7fe02ce12700 1 --2- 172.21.15.107:0/3392129041 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe01003b360 0x7fe01003d820 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe01c006fd0 tx=0x7fe01c006e40 comp rx=0 tx=0).stop 2024-04-25T00:07:34.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.379+0000 7fe02ce12700 1 -- 172.21.15.107:0/3392129041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe028106c00 msgr2=0x7fe0281965e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:34.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.379+0000 7fe02ce12700 1 --2- 172.21.15.107:0/3392129041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe028106c00 0x7fe0281965e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe018006010 tx=0x7fe018005030 comp rx=0 tx=0).stop 2024-04-25T00:07:34.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.379+0000 7fe02ce12700 1 -- 172.21.15.107:0/3392129041 shutdown_connections 2024-04-25T00:07:34.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.379+0000 7fe02ce12700 1 --2- 172.21.15.107:0/3392129041 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe01003b360 0x7fe01003d820 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:34.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.379+0000 7fe02ce12700 1 --2- 172.21.15.107:0/3392129041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe028106c00 0x7fe0281965e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:34.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.379+0000 7fe02ce12700 1 -- 172.21.15.107:0/3392129041 >> 172.21.15.107:0/3392129041 conn(0x7fe0280fc680 msgr2=0x7fe0280fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:34.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.380+0000 7fe02ce12700 1 -- 172.21.15.107:0/3392129041 shutdown_connections 2024-04-25T00:07:34.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:34.380+0000 7fe02ce12700 1 -- 172.21.15.107:0/3392129041 wait complete. 2024-04-25T00:07:34.914 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T00:07:34.915 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T00:07:35.145 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:07:35.565 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:35 smithi107 ceph-mon[29153]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:07:35.565 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:35 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:35.566 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:35 smithi107 ceph-mon[29153]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:36.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.706+0000 7f9f0a4dd700 1 -- 172.21.15.107:0/3395898392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f04106be0 msgr2=0x7f9f04106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:36.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.706+0000 7f9f0a4dd700 1 --2- 172.21.15.107:0/3395898392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f04106be0 0x7f9f04106fc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8009910 tx=0x7f9ef8009c20 comp rx=0 tx=0).stop 2024-04-25T00:07:36.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.706+0000 7f9f0a4dd700 1 -- 172.21.15.107:0/3395898392 shutdown_connections 2024-04-25T00:07:36.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.706+0000 7f9f0a4dd700 1 --2- 172.21.15.107:0/3395898392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f04106be0 0x7f9f04106fc0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:36.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.706+0000 7f9f0a4dd700 1 -- 172.21.15.107:0/3395898392 >> 172.21.15.107:0/3395898392 conn(0x7f9f040fc600 msgr2=0x7f9f040fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:36.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.706+0000 7f9f0a4dd700 1 -- 172.21.15.107:0/3395898392 shutdown_connections 2024-04-25T00:07:36.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.706+0000 7f9f0a4dd700 1 -- 172.21.15.107:0/3395898392 wait complete. 2024-04-25T00:07:36.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.707+0000 7f9f0a4dd700 1 Processor -- start 2024-04-25T00:07:36.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.707+0000 7f9f0a4dd700 1 -- start start 2024-04-25T00:07:36.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.707+0000 7f9f0a4dd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f04106be0 0x7f9f040fcab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:36.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.707+0000 7f9f0a4dd700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0410b1a0 con 0x7f9f04106be0 2024-04-25T00:07:36.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.707+0000 7f9f03fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f04106be0 0x7f9f040fcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:36.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.707+0000 7f9f03fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f04106be0 0x7f9f040fcab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41882/0 (socket says 172.21.15.107:41882) 2024-04-25T00:07:36.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.707+0000 7f9f03fff700 1 -- 172.21.15.107:0/204275765 learned_addr learned my addr 172.21.15.107:0/204275765 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:07:36.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.707+0000 7f9f03fff700 1 -- 172.21.15.107:0/204275765 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f040fe9a0 con 0x7f9f04106be0 2024-04-25T00:07:36.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.708+0000 7f9f03fff700 1 --2- 172.21.15.107:0/204275765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f04106be0 0x7f9f040fcab0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8005850 tx=0x7f9ef8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:36.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.708+0000 7f9f017fa700 1 -- 172.21.15.107:0/204275765 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ef8026070 con 0x7f9f04106be0 2024-04-25T00:07:36.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.708+0000 7f9f017fa700 1 -- 172.21.15.107:0/204275765 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ef800bc30 con 0x7f9f04106be0 2024-04-25T00:07:36.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.708+0000 7f9f017fa700 1 -- 172.21.15.107:0/204275765 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ef8003a20 con 0x7f9f04106be0 2024-04-25T00:07:36.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.708+0000 7f9f0a4dd700 1 -- 172.21.15.107:0/204275765 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f040fcff0 con 0x7f9f04106be0 2024-04-25T00:07:36.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.708+0000 7f9f0a4dd700 1 -- 172.21.15.107:0/204275765 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f040fd4b0 con 0x7f9f04106be0 2024-04-25T00:07:36.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.709+0000 7f9f017fa700 1 -- 172.21.15.107:0/204275765 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f9ef8029070 con 0x7f9f04106be0 2024-04-25T00:07:36.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.709+0000 7f9f0a4dd700 1 -- 172.21.15.107:0/204275765 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f041021e0 con 0x7f9f04106be0 2024-04-25T00:07:36.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.709+0000 7f9f017fa700 1 --2- 172.21.15.107:0/204275765 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f9eec0390f0 0x7f9eec03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:36.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.709+0000 7f9f017fa700 1 -- 172.21.15.107:0/204275765 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9ef801e070 con 0x7f9f04106be0 2024-04-25T00:07:36.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.709+0000 7f9f037fe700 1 --2- 172.21.15.107:0/204275765 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f9eec0390f0 0x7f9eec03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:36.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.709+0000 7f9f037fe700 1 --2- 172.21.15.107:0/204275765 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f9eec0390f0 0x7f9eec03b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4006fd0 tx=0x7f9ef4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:36.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.714+0000 7f9f017fa700 1 -- 172.21.15.107:0/204275765 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9ef8013070 con 0x7f9f04106be0 2024-04-25T00:07:36.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:36.914+0000 7f9f0a4dd700 1 -- 172.21.15.107:0/204275765 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f9f040fdfe0 con 0x7f9f04106be0 2024-04-25T00:07:37.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:37.900+0000 7f9f017fa700 1 -- 172.21.15.107:0/204275765 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f9ef8030090 con 0x7f9f04106be0 2024-04-25T00:07:37.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:37.903+0000 7f9f0a4dd700 1 -- 172.21.15.107:0/204275765 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f9eec0390f0 msgr2=0x7f9eec03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:37.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:37.903+0000 7f9f0a4dd700 1 --2- 172.21.15.107:0/204275765 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f9eec0390f0 0x7f9eec03b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4006fd0 tx=0x7f9ef4006e40 comp rx=0 tx=0).stop 2024-04-25T00:07:37.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:37.903+0000 7f9f0a4dd700 1 -- 172.21.15.107:0/204275765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f04106be0 msgr2=0x7f9f040fcab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:37.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:37.903+0000 7f9f0a4dd700 1 --2- 172.21.15.107:0/204275765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f04106be0 0x7f9f040fcab0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8005850 tx=0x7f9ef8005030 comp rx=0 tx=0).stop 2024-04-25T00:07:37.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:37.903+0000 7f9f0a4dd700 1 -- 172.21.15.107:0/204275765 shutdown_connections 2024-04-25T00:07:37.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:37.903+0000 7f9f0a4dd700 1 --2- 172.21.15.107:0/204275765 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f9eec0390f0 0x7f9eec03b5b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:37.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:37.903+0000 7f9f0a4dd700 1 --2- 172.21.15.107:0/204275765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f04106be0 0x7f9f040fcab0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:37.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:37.904+0000 7f9f0a4dd700 1 -- 172.21.15.107:0/204275765 >> 172.21.15.107:0/204275765 conn(0x7f9f040fc600 msgr2=0x7f9f0406a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:37.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:37.904+0000 7f9f0a4dd700 1 -- 172.21.15.107:0/204275765 shutdown_connections 2024-04-25T00:07:37.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:07:37.904+0000 7f9f0a4dd700 1 -- 172.21.15.107:0/204275765 wait complete. 2024-04-25T00:07:37.907 INFO:teuthology.orchestra.run.smithi107.stderr:adjusted tunables profile to default 2024-04-25T00:07:38.208 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:37 smithi107 ceph-mon[29153]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:38.208 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:37 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/204275765' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T00:07:38.208 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:37 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:38.208 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:37 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:38.208 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:37 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:38.208 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:37 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:38.208 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:37 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi195", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:07:38.208 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:37 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:07:38.208 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:37 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:07:38.552 INFO:tasks.cephadm:Adding mon.a on smithi107 2024-04-25T00:07:38.552 INFO:tasks.cephadm:Adding mon.b on smithi170 2024-04-25T00:07:38.552 INFO:tasks.cephadm:Adding mon.c on smithi195 2024-04-25T00:07:38.552 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi107:172.21.15.107=a;smithi170:172.21.15.170=b;smithi195:172.21.15.195=c' 2024-04-25T00:07:38.775 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:07:39.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:38 smithi107 ceph-mon[29153]: Updating smithi195:/etc/ceph/ceph.conf 2024-04-25T00:07:39.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:38 smithi107 ceph-mon[29153]: Updating smithi195:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:07:39.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:38 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/204275765' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T00:07:39.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:38 smithi107 ceph-mon[29153]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:07:39.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:38 smithi107 ceph-mon[29153]: Updating smithi195:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:07:39.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:38 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:39.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:38 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:39.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:38 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:40.226 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.225+0000 7f6f56078700 1 -- 172.21.15.195:0/2499504664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f50100da0 msgr2=0x7f6f50101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:40.226 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.225+0000 7f6f56078700 1 --2- 172.21.15.195:0/2499504664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f50100da0 0x7f6f50101180 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6f40009910 tx=0x7f6f40009c20 comp rx=0 tx=0).stop 2024-04-25T00:07:40.226 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.225+0000 7f6f56078700 1 -- 172.21.15.195:0/2499504664 shutdown_connections 2024-04-25T00:07:40.226 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.225+0000 7f6f56078700 1 --2- 172.21.15.195:0/2499504664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f50100da0 0x7f6f50101180 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:40.226 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.225+0000 7f6f56078700 1 -- 172.21.15.195:0/2499504664 >> 172.21.15.195:0/2499504664 conn(0x7f6f500fc670 msgr2=0x7f6f500fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:40.226 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.225+0000 7f6f56078700 1 -- 172.21.15.195:0/2499504664 shutdown_connections 2024-04-25T00:07:40.226 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.225+0000 7f6f56078700 1 -- 172.21.15.195:0/2499504664 wait complete. 2024-04-25T00:07:40.229 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.226+0000 7f6f56078700 1 Processor -- start 2024-04-25T00:07:40.229 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.226+0000 7f6f56078700 1 -- start start 2024-04-25T00:07:40.229 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.226+0000 7f6f56078700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f50100da0 0x7f6f50196680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:40.229 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.226+0000 7f6f56078700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6f5010b170 con 0x7f6f50100da0 2024-04-25T00:07:40.229 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.227+0000 7f6f4f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f50100da0 0x7f6f50196680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:40.229 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.227+0000 7f6f4f7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f50100da0 0x7f6f50196680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.195:48524/0 (socket says 172.21.15.195:48524) 2024-04-25T00:07:40.229 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.227+0000 7f6f4f7fe700 1 -- 172.21.15.195:0/3608784024 learned_addr learned my addr 172.21.15.195:0/3608784024 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-04-25T00:07:40.229 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.227+0000 7f6f4f7fe700 1 -- 172.21.15.195:0/3608784024 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f50196bc0 con 0x7f6f50100da0 2024-04-25T00:07:40.230 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.227+0000 7f6f4f7fe700 1 --2- 172.21.15.195:0/3608784024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f50100da0 0x7f6f50196680 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6f40005850 tx=0x7f6f40005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:40.230 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.227+0000 7f6f4cff9700 1 -- 172.21.15.195:0/3608784024 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f40026070 con 0x7f6f50100da0 2024-04-25T00:07:40.230 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.227+0000 7f6f56078700 1 -- 172.21.15.195:0/3608784024 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f50196eb0 con 0x7f6f50100da0 2024-04-25T00:07:40.230 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.227+0000 7f6f4cff9700 1 -- 172.21.15.195:0/3608784024 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f4000bbd0 con 0x7f6f50100da0 2024-04-25T00:07:40.230 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.227+0000 7f6f4cff9700 1 -- 172.21.15.195:0/3608784024 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f40003970 con 0x7f6f50100da0 2024-04-25T00:07:40.230 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.227+0000 7f6f56078700 1 -- 172.21.15.195:0/3608784024 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f5019af50 con 0x7f6f50100da0 2024-04-25T00:07:40.230 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.228+0000 7f6f4cff9700 1 -- 172.21.15.195:0/3608784024 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f6f40029070 con 0x7f6f50100da0 2024-04-25T00:07:40.230 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.228+0000 7f6f56078700 1 -- 172.21.15.195:0/3608784024 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f50102280 con 0x7f6f50100da0 2024-04-25T00:07:40.230 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.228+0000 7f6f4cff9700 1 --2- 172.21.15.195:0/3608784024 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f6f380390f0 0x7f6f3803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:40.230 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.229+0000 7f6f4cff9700 1 -- 172.21.15.195:0/3608784024 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6f4001e070 con 0x7f6f50100da0 2024-04-25T00:07:40.230 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.229+0000 7f6f4effd700 1 --2- 172.21.15.195:0/3608784024 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f6f380390f0 0x7f6f3803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:40.231 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.229+0000 7f6f4effd700 1 --2- 172.21.15.195:0/3608784024 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f6f380390f0 0x7f6f3803b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6f44006fd0 tx=0x7f6f44006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:40.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:39 smithi107 ceph-mon[29153]: Updating smithi195:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:07:40.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:39 smithi107 ceph-mon[29153]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:40.234 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.234+0000 7f6f4cff9700 1 -- 172.21.15.195:0/3608784024 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6f40013070 con 0x7f6f50100da0 2024-04-25T00:07:40.441 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.441+0000 7f6f56078700 1 -- 172.21.15.195:0/3608784024 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi170:172.21.15.170=b;smithi195:172.21.15.195=c", "target": ["mon-mgr", ""]}) -- 0x7f6f501979e0 con 0x7f6f380390f0 2024-04-25T00:07:40.448 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.447+0000 7f6f4cff9700 1 -- 172.21.15.195:0/3608784024 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6f501979e0 con 0x7f6f380390f0 2024-04-25T00:07:40.448 INFO:teuthology.orchestra.run.smithi195.stdout:Scheduled mon update... 2024-04-25T00:07:40.450 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.449+0000 7f6f56078700 1 -- 172.21.15.195:0/3608784024 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f6f380390f0 msgr2=0x7f6f3803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:40.450 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.449+0000 7f6f56078700 1 --2- 172.21.15.195:0/3608784024 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f6f380390f0 0x7f6f3803b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6f44006fd0 tx=0x7f6f44006e40 comp rx=0 tx=0).stop 2024-04-25T00:07:40.450 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.449+0000 7f6f56078700 1 -- 172.21.15.195:0/3608784024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f50100da0 msgr2=0x7f6f50196680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:40.450 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.449+0000 7f6f56078700 1 --2- 172.21.15.195:0/3608784024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f50100da0 0x7f6f50196680 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6f40005850 tx=0x7f6f40005030 comp rx=0 tx=0).stop 2024-04-25T00:07:40.450 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.449+0000 7f6f56078700 1 -- 172.21.15.195:0/3608784024 shutdown_connections 2024-04-25T00:07:40.450 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.449+0000 7f6f56078700 1 --2- 172.21.15.195:0/3608784024 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f6f380390f0 0x7f6f3803b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:40.450 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.449+0000 7f6f56078700 1 --2- 172.21.15.195:0/3608784024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f50100da0 0x7f6f50196680 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:40.450 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.449+0000 7f6f56078700 1 -- 172.21.15.195:0/3608784024 >> 172.21.15.195:0/3608784024 conn(0x7f6f500fc670 msgr2=0x7f6f50102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:40.451 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.450+0000 7f6f56078700 1 -- 172.21.15.195:0/3608784024 shutdown_connections 2024-04-25T00:07:40.451 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:40.450+0000 7f6f56078700 1 -- 172.21.15.195:0/3608784024 wait complete. 2024-04-25T00:07:41.080 DEBUG:teuthology.orchestra.run.smithi170:mon.b> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T00:07:41.082 DEBUG:teuthology.orchestra.run.smithi195:mon.c> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T00:07:41.085 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T00:07:41.085 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T00:07:41.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:41 smithi107 ceph-mon[29153]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi170:172.21.15.170=b;smithi195:172.21.15.195=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:07:41.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:41 smithi107 ceph-mon[29153]: Saving service mon spec with placement smithi107:172.21.15.107=a;smithi170:172.21.15.170=b;smithi195:172.21.15.195=c;count:3 2024-04-25T00:07:41.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:41 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:41.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:41 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:07:41.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:41 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:07:41.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:41 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:07:41.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:41 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:41.736 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:41 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:07:41.736 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:41 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:07:41.736 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:41 smithi107 ceph-mon[29153]: Deploying daemon mon.c on smithi195 2024-04-25T00:07:41.736 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:41 smithi107 ceph-mon[29153]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:43.070 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:07:44.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:43 smithi107 ceph-mon[29153]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:44.925 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.923+0000 7ff0a3e0a700 1 -- 172.21.15.195:0/2534123243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff09c107df0 msgr2=0x7ff09c1081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:44.926 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.923+0000 7ff0a3e0a700 1 --2- 172.21.15.195:0/2534123243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff09c107df0 0x7ff09c1081d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009910 tx=0x7ff08c009c20 comp rx=0 tx=0).stop 2024-04-25T00:07:44.926 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.923+0000 7ff0a3e0a700 1 -- 172.21.15.195:0/2534123243 shutdown_connections 2024-04-25T00:07:44.926 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.923+0000 7ff0a3e0a700 1 --2- 172.21.15.195:0/2534123243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff09c107df0 0x7ff09c1081d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:44.926 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.923+0000 7ff0a3e0a700 1 -- 172.21.15.195:0/2534123243 >> 172.21.15.195:0/2534123243 conn(0x7ff09c075f90 msgr2=0x7ff09c0763a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:44.926 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.923+0000 7ff0a3e0a700 1 -- 172.21.15.195:0/2534123243 shutdown_connections 2024-04-25T00:07:44.926 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.924+0000 7ff0a3e0a700 1 -- 172.21.15.195:0/2534123243 wait complete. 2024-04-25T00:07:44.927 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.924+0000 7ff0a3e0a700 1 Processor -- start 2024-04-25T00:07:44.927 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.924+0000 7ff0a3e0a700 1 -- start start 2024-04-25T00:07:44.927 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.924+0000 7ff0a3e0a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff09c107df0 0x7ff09c101650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:44.927 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.924+0000 7ff0a3e0a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c10c3b0 con 0x7ff09c107df0 2024-04-25T00:07:44.927 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.924+0000 7ff0a1ba6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff09c107df0 0x7ff09c101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:44.927 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.924+0000 7ff0a1ba6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff09c107df0 0x7ff09c101650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.195:57110/0 (socket says 172.21.15.195:57110) 2024-04-25T00:07:44.927 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.925+0000 7ff0a1ba6700 1 -- 172.21.15.195:0/1998154021 learned_addr learned my addr 172.21.15.195:0/1998154021 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-04-25T00:07:44.927 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.925+0000 7ff0a1ba6700 1 -- 172.21.15.195:0/1998154021 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09c103540 con 0x7ff09c107df0 2024-04-25T00:07:44.927 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.925+0000 7ff0a1ba6700 1 --2- 172.21.15.195:0/1998154021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff09c107df0 0x7ff09c101650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff08c005850 tx=0x7ff08c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:44.927 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.925+0000 7ff092ffd700 1 -- 172.21.15.195:0/1998154021 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff08c026070 con 0x7ff09c107df0 2024-04-25T00:07:44.927 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.925+0000 7ff0a3e0a700 1 -- 172.21.15.195:0/1998154021 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff09c101b90 con 0x7ff09c107df0 2024-04-25T00:07:44.927 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.925+0000 7ff092ffd700 1 -- 172.21.15.195:0/1998154021 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff08c00bc30 con 0x7ff09c107df0 2024-04-25T00:07:44.927 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.925+0000 7ff092ffd700 1 -- 172.21.15.195:0/1998154021 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff08c003a20 con 0x7ff09c107df0 2024-04-25T00:07:44.928 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.925+0000 7ff0a3e0a700 1 -- 172.21.15.195:0/1998154021 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff09c102050 con 0x7ff09c107df0 2024-04-25T00:07:44.928 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.926+0000 7ff0a3e0a700 1 -- 172.21.15.195:0/1998154021 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff09c06a150 con 0x7ff09c107df0 2024-04-25T00:07:44.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.926+0000 7ff092ffd700 1 -- 172.21.15.195:0/1998154021 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7ff08c029070 con 0x7ff09c107df0 2024-04-25T00:07:44.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.926+0000 7ff092ffd700 1 --2- 172.21.15.195:0/1998154021 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff0880390f0 0x7ff08803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:44.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.926+0000 7ff092ffd700 1 -- 172.21.15.195:0/1998154021 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff08c01e070 con 0x7ff09c107df0 2024-04-25T00:07:44.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.927+0000 7ff0a13a5700 1 --2- 172.21.15.195:0/1998154021 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff0880390f0 0x7ff08803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:44.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.927+0000 7ff0a13a5700 1 --2- 172.21.15.195:0/1998154021 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff0880390f0 0x7ff08803b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff098006fd0 tx=0x7ff098006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:44.932 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:44.932+0000 7ff092ffd700 1 -- 172.21.15.195:0/1998154021 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff08c013070 con 0x7ff09c107df0 2024-04-25T00:07:45.222 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:45.220+0000 7ff0a3e0a700 1 -- 172.21.15.195:0/1998154021 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff09c102c20 con 0x7ff09c107df0 2024-04-25T00:07:45.222 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:45.221+0000 7ff092ffd700 1 -- 172.21.15.195:0/1998154021 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+749 (secure 0 0 0) 0x7ff08c030020 con 0x7ff09c107df0 2024-04-25T00:07:45.222 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-04-25T00:07:45.222 INFO:teuthology.orchestra.run.smithi195.stdout:{"epoch":1,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","modified":"2024-04-25T00:05:52.163469Z","created":"2024-04-25T00:05:52.163469Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T00:07:45.224 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:45.223+0000 7ff0a3e0a700 1 -- 172.21.15.195:0/1998154021 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff0880390f0 msgr2=0x7ff08803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:45.224 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:45.223+0000 7ff0a3e0a700 1 --2- 172.21.15.195:0/1998154021 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff0880390f0 0x7ff08803b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff098006fd0 tx=0x7ff098006e40 comp rx=0 tx=0).stop 2024-04-25T00:07:45.224 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:45.223+0000 7ff0a3e0a700 1 -- 172.21.15.195:0/1998154021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff09c107df0 msgr2=0x7ff09c101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:45.224 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:45.223+0000 7ff0a3e0a700 1 --2- 172.21.15.195:0/1998154021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff09c107df0 0x7ff09c101650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff08c005850 tx=0x7ff08c005030 comp rx=0 tx=0).stop 2024-04-25T00:07:45.224 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:45.223+0000 7ff0a3e0a700 1 -- 172.21.15.195:0/1998154021 shutdown_connections 2024-04-25T00:07:45.224 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:45.223+0000 7ff0a3e0a700 1 --2- 172.21.15.195:0/1998154021 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff0880390f0 0x7ff08803b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:45.224 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:45.223+0000 7ff0a3e0a700 1 --2- 172.21.15.195:0/1998154021 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff09c107df0 0x7ff09c101650 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:45.224 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:45.223+0000 7ff0a3e0a700 1 -- 172.21.15.195:0/1998154021 >> 172.21.15.195:0/1998154021 conn(0x7ff09c075f90 msgr2=0x7ff09c0fd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:45.224 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:45.223+0000 7ff0a3e0a700 1 -- 172.21.15.195:0/1998154021 shutdown_connections 2024-04-25T00:07:45.224 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:45.223+0000 7ff0a3e0a700 1 -- 172.21.15.195:0/1998154021 wait complete. 2024-04-25T00:07:45.225 INFO:teuthology.orchestra.run.smithi195.stderr:dumped monmap epoch 1 2024-04-25T00:07:45.270 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:45 smithi195 ceph-mon[32833]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T00:07:45.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:45 smithi195 bash[32793]: a6bea1777566720a83b270ead10994456067e5c708e376c45c3def7866f39070 2024-04-25T00:07:45.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:45 smithi195 systemd[1]: Started Ceph mon.c for 79602f58-0297-11ef-bc93-c7b262605968. 2024-04-25T00:07:46.959 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T00:07:46.960 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T00:07:47.206 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:07:50.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:07:50.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: mon.a calling monitor election 2024-04-25T00:07:50.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:07:50.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:07:50.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:50.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:07:50.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: mon.c calling monitor election 2024-04-25T00:07:50.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:07:50.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:50.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:07:50.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:07:50.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T00:07:50.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: monmap epoch 2 2024-04-25T00:07:50.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: fsid 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:07:50.591 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: last_changed 2024-04-25T00:07:45.253386+0000 2024-04-25T00:07:50.591 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: created 2024-04-25T00:05:52.163469+0000 2024-04-25T00:07:50.591 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: min_mon_release 19 (squid) 2024-04-25T00:07:50.591 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: election_strategy: 1 2024-04-25T00:07:50.591 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-04-25T00:07:50.591 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-04-25T00:07:50.591 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: fsmap 2024-04-25T00:07:50.591 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:07:50.591 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: mgrmap e13: x(active, since 49s) 2024-04-25T00:07:50.591 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: overall HEALTH_OK 2024-04-25T00:07:50.591 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:50.591 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:50.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:07:50.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: mon.a calling monitor election 2024-04-25T00:07:50.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:07:50.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:07:50.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:50.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:07:50.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: mon.c calling monitor election 2024-04-25T00:07:50.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:07:50.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:50.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:07:50.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:07:50.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T00:07:50.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: monmap epoch 2 2024-04-25T00:07:50.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: fsid 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:07:50.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: last_changed 2024-04-25T00:07:45.253386+0000 2024-04-25T00:07:50.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: created 2024-04-25T00:05:52.163469+0000 2024-04-25T00:07:50.736 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: min_mon_release 19 (squid) 2024-04-25T00:07:50.736 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: election_strategy: 1 2024-04-25T00:07:50.736 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-04-25T00:07:50.736 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-04-25T00:07:50.736 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: fsmap 2024-04-25T00:07:50.736 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:07:50.736 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: mgrmap e13: x(active, since 49s) 2024-04-25T00:07:50.737 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: overall HEALTH_OK 2024-04-25T00:07:50.737 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:50.737 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:51.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:51 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:51.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:51 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:07:51.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:51 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:07:51.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:51 smithi195 ceph-mon[32833]: Deploying daemon mon.b on smithi170 2024-04-25T00:07:51.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:51 smithi195 ceph-mon[32833]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:51.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:51 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:07:51.599 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.593+0000 7fd2b1268700 1 -- 172.21.15.195:0/2273375514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2940035c0 msgr2=0x7fd294005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:51.599 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.593+0000 7fd2b1268700 1 --2- 172.21.15.195:0/2273375514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2940035c0 0x7fd294005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac100ba0 tx=0x7fd29c00feb0 comp rx=0 tx=0).stop 2024-04-25T00:07:51.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.593+0000 7fd2b1268700 1 -- 172.21.15.195:0/2273375514 shutdown_connections 2024-04-25T00:07:51.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.593+0000 7fd2b1268700 1 --2- 172.21.15.195:0/2273375514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2940035c0 0x7fd294005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:51.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.593+0000 7fd2b1268700 1 -- 172.21.15.195:0/2273375514 >> 172.21.15.195:0/2273375514 conn(0x7fd2ac0fc780 msgr2=0x7fd2ac0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:51.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.593+0000 7fd2b1268700 1 -- 172.21.15.195:0/2273375514 shutdown_connections 2024-04-25T00:07:51.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.593+0000 7fd2b1268700 1 -- 172.21.15.195:0/2273375514 wait complete. 2024-04-25T00:07:51.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.594+0000 7fd2b1268700 1 Processor -- start 2024-04-25T00:07:51.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.595+0000 7fd2b1268700 1 -- start start 2024-04-25T00:07:51.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.595+0000 7fd2b1268700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2940035c0 0x7fd2ac19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:51.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.595+0000 7fd2b1268700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2ac19db90 0x7fd2ac197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:51.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.595+0000 7fd2b1268700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac108d00 con 0x7fd2940035c0 2024-04-25T00:07:51.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.595+0000 7fd2b1268700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd2ac108b80 con 0x7fd2ac19db90 2024-04-25T00:07:51.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.595+0000 7fd2aa7fc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2ac19db90 0x7fd2ac197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:51.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.595+0000 7fd2aa7fc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2ac19db90 0x7fd2ac197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:54942/0 (socket says 172.21.15.195:54942) 2024-04-25T00:07:51.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.595+0000 7fd2aa7fc700 1 -- 172.21.15.195:0/461834893 learned_addr learned my addr 172.21.15.195:0/461834893 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-04-25T00:07:51.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.595+0000 7fd2aaffd700 1 --2- 172.21.15.195:0/461834893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2940035c0 0x7fd2ac19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:51.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.595+0000 7fd2aa7fc700 1 -- 172.21.15.195:0/461834893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2ac19db90 msgr2=0x7fd2ac197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T00:07:51.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.595+0000 7fd2aa7fc700 1 -- 172.21.15.195:0/461834893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2ac19db90 msgr2=0x7fd2ac197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T00:07:51.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.595+0000 7fd2aa7fc700 1 --2- 172.21.15.195:0/461834893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2ac19db90 0x7fd2ac197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T00:07:51.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.596+0000 7fd2aa7fc700 1 --2- 172.21.15.195:0/461834893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2ac19db90 0x7fd2ac197770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:07:51.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.596+0000 7fd2aaffd700 1 -- 172.21.15.195:0/461834893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2ac19db90 msgr2=0x7fd2ac197770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:07:51.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.596+0000 7fd2aaffd700 1 --2- 172.21.15.195:0/461834893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2ac19db90 0x7fd2ac197770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:51.602 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.596+0000 7fd2aaffd700 1 -- 172.21.15.195:0/461834893 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2ac19e110 con 0x7fd2940035c0 2024-04-25T00:07:51.602 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.596+0000 7fd2aaffd700 1 --2- 172.21.15.195:0/461834893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2940035c0 0x7fd2ac19d650 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0059d0 tx=0x7fd29c0057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:51.602 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.596+0000 7fd293fff700 1 -- 172.21.15.195:0/461834893 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd29c004b20 con 0x7fd2940035c0 2024-04-25T00:07:51.602 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.596+0000 7fd293fff700 1 -- 172.21.15.195:0/461834893 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd29c013e40 con 0x7fd2940035c0 2024-04-25T00:07:51.603 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.596+0000 7fd2b1268700 1 -- 172.21.15.195:0/461834893 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2ac197d10 con 0x7fd2940035c0 2024-04-25T00:07:51.603 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.596+0000 7fd293fff700 1 -- 172.21.15.195:0/461834893 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd29c011ca0 con 0x7fd2940035c0 2024-04-25T00:07:51.603 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.597+0000 7fd2b1268700 1 -- 172.21.15.195:0/461834893 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2ac1982a0 con 0x7fd2940035c0 2024-04-25T00:07:51.603 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.597+0000 7fd293fff700 1 -- 172.21.15.195:0/461834893 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7fd29c023070 con 0x7fd2940035c0 2024-04-25T00:07:51.603 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.598+0000 7fd2b1268700 1 -- 172.21.15.195:0/461834893 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2ac1023b0 con 0x7fd2940035c0 2024-04-25T00:07:51.603 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.598+0000 7fd293fff700 1 --2- 172.21.15.195:0/461834893 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd298038e90 0x7fd29803b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:51.603 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.598+0000 7fd293fff700 1 -- 172.21.15.195:0/461834893 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd29c020050 con 0x7fd2940035c0 2024-04-25T00:07:51.603 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.598+0000 7fd2aa7fc700 1 --2- 172.21.15.195:0/461834893 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd298038e90 0x7fd29803b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:51.603 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.598+0000 7fd2aa7fc700 1 --2- 172.21.15.195:0/461834893 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd298038e90 0x7fd29803b350 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009180 tx=0x7fd2a0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:51.604 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.603+0000 7fd293fff700 1 -- 172.21.15.195:0/461834893 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd29c031020 con 0x7fd2940035c0 2024-04-25T00:07:51.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:51 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:51.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:51 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:07:51.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:51 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:07:51.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:51 smithi107 ceph-mon[29153]: Deploying daemon mon.b on smithi170 2024-04-25T00:07:51.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:51 smithi107 ceph-mon[29153]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:51.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:51 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:07:51.888 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.887+0000 7fd2b1268700 1 -- 172.21.15.195:0/461834893 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd2ac1985b0 con 0x7fd2940035c0 2024-04-25T00:07:51.888 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.888+0000 7fd293fff700 1 -- 172.21.15.195:0/461834893 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1019 (secure 0 0 0) 0x7fd29c028030 con 0x7fd2940035c0 2024-04-25T00:07:51.889 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-04-25T00:07:51.889 INFO:teuthology.orchestra.run.smithi195.stdout:{"epoch":2,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","modified":"2024-04-25T00:07:45.253386Z","created":"2024-04-25T00:05:52.163469Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T00:07:51.891 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.890+0000 7fd2b1268700 1 -- 172.21.15.195:0/461834893 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd298038e90 msgr2=0x7fd29803b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:51.891 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.890+0000 7fd2b1268700 1 --2- 172.21.15.195:0/461834893 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd298038e90 0x7fd29803b350 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009180 tx=0x7fd2a0008040 comp rx=0 tx=0).stop 2024-04-25T00:07:51.891 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.890+0000 7fd2b1268700 1 -- 172.21.15.195:0/461834893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2940035c0 msgr2=0x7fd2ac19d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:51.891 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.890+0000 7fd2b1268700 1 --2- 172.21.15.195:0/461834893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2940035c0 0x7fd2ac19d650 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0059d0 tx=0x7fd29c0057d0 comp rx=0 tx=0).stop 2024-04-25T00:07:51.891 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.890+0000 7fd2b1268700 1 -- 172.21.15.195:0/461834893 shutdown_connections 2024-04-25T00:07:51.891 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.890+0000 7fd2b1268700 1 --2- 172.21.15.195:0/461834893 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd298038e90 0x7fd29803b350 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:51.891 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.890+0000 7fd2b1268700 1 --2- 172.21.15.195:0/461834893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2940035c0 0x7fd2ac19d650 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:51.891 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.890+0000 7fd2b1268700 1 --2- 172.21.15.195:0/461834893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2ac19db90 0x7fd2ac197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:51.892 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.890+0000 7fd2b1268700 1 -- 172.21.15.195:0/461834893 >> 172.21.15.195:0/461834893 conn(0x7fd2ac0fc780 msgr2=0x7fd2ac0fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:51.892 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.891+0000 7fd2b1268700 1 -- 172.21.15.195:0/461834893 shutdown_connections 2024-04-25T00:07:51.892 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:51.891+0000 7fd2b1268700 1 -- 172.21.15.195:0/461834893 wait complete. 2024-04-25T00:07:51.892 INFO:teuthology.orchestra.run.smithi195.stderr:dumped monmap epoch 2 2024-04-25T00:07:52.465 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:52 smithi195 ceph-mon[32833]: from='client.? 172.21.15.195:0/461834893' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:07:52.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:52 smithi107 ceph-mon[29153]: from='client.? 172.21.15.195:0/461834893' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:07:53.492 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T00:07:53.492 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T00:07:53.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:53 smithi195 ceph-mon[32833]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:53.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:53 smithi107 ceph-mon[29153]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:53.738 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:07:54.026 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:53 smithi170 podman[32789]: 2024-04-25 00:07:53.938386438 +0000 UTC m=+0.119992540 container create 21037d33514913d2b774353f386998aa34b2df3702c6492f51e3d224e71a9ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-b, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:07:54.316 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 podman[32789]: 2024-04-25 00:07:54.107584457 +0000 UTC m=+0.289190574 container init 21037d33514913d2b774353f386998aa34b2df3702c6492f51e3d224e71a9ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T00:07:54.316 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 podman[32789]: 2024-04-25 00:07:54.110564404 +0000 UTC m=+0.292170517 container start 21037d33514913d2b774353f386998aa34b2df3702c6492f51e3d224e71a9ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T00:07:54.316 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T00:07:54.316 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-25T00:07:54.316 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: pidfile_write: ignore empty --pid-file 2024-04-25T00:07:54.316 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: load: jerasure load: lrc 2024-04-25T00:07:54.316 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: RocksDB version: 7.9.2 2024-04-25T00:07:54.316 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Git sha 0 2024-04-25T00:07:54.317 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-25T00:07:54.317 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: DB SUMMARY 2024-04-25T00:07:54.317 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: DB Session ID: R7IKYNU5TPI0C89LKTHJ 2024-04-25T00:07:54.317 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: CURRENT file: CURRENT 2024-04-25T00:07:54.317 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T00:07:54.317 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T00:07:54.317 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-25T00:07:54.317 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-25T00:07:54.317 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.error_if_exists: 0 2024-04-25T00:07:54.317 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.create_if_missing: 0 2024-04-25T00:07:54.317 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.paranoid_checks: 1 2024-04-25T00:07:54.317 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T00:07:54.317 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T00:07:54.318 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T00:07:54.318 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.env: 0x55d38d1d5c60 2024-04-25T00:07:54.318 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T00:07:54.318 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.info_log: 0x55d38f11daa0 2024-04-25T00:07:54.318 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T00:07:54.318 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.statistics: (nil) 2024-04-25T00:07:54.318 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.use_fsync: 0 2024-04-25T00:07:54.318 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_log_file_size: 0 2024-04-25T00:07:54.318 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T00:07:54.318 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T00:07:54.318 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T00:07:54.318 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T00:07:54.319 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.allow_fallocate: 1 2024-04-25T00:07:54.319 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T00:07:54.319 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T00:07:54.319 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.use_direct_reads: 0 2024-04-25T00:07:54.319 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T00:07:54.319 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T00:07:54.319 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.db_log_dir: 2024-04-25T00:07:54.319 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.wal_dir: 2024-04-25T00:07:54.319 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T00:07:54.319 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T00:07:54.319 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T00:07:54.319 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T00:07:54.319 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T00:07:54.320 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T00:07:54.320 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T00:07:54.320 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T00:07:54.320 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.write_buffer_manager: 0x55d38f0f01e0 2024-04-25T00:07:54.320 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T00:07:54.320 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T00:07:54.320 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T00:07:54.320 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T00:07:54.320 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T00:07:54.320 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T00:07:54.320 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T00:07:54.320 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T00:07:54.320 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.unordered_write: 0 2024-04-25T00:07:54.321 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T00:07:54.321 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T00:07:54.321 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T00:07:54.321 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T00:07:54.321 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.row_cache: None 2024-04-25T00:07:54.321 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.wal_filter: None 2024-04-25T00:07:54.323 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T00:07:54.323 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T00:07:54.323 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.two_write_queues: 0 2024-04-25T00:07:54.323 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T00:07:54.323 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.wal_compression: 0 2024-04-25T00:07:54.323 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.atomic_flush: 0 2024-04-25T00:07:54.323 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T00:07:54.324 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T00:07:54.324 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T00:07:54.324 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.log_readahead_size: 0 2024-04-25T00:07:54.324 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T00:07:54.324 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T00:07:54.324 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T00:07:54.324 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T00:07:54.324 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T00:07:54.324 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T00:07:54.324 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T00:07:54.324 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_background_jobs: 2 2024-04-25T00:07:54.324 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_background_compactions: -1 2024-04-25T00:07:54.325 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_subcompactions: 1 2024-04-25T00:07:54.325 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T00:07:54.325 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T00:07:54.325 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T00:07:54.325 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T00:07:54.325 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T00:07:54.325 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T00:07:54.325 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T00:07:54.325 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T00:07:54.325 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_open_files: -1 2024-04-25T00:07:54.325 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T00:07:54.326 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T00:07:54.326 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T00:07:54.326 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T00:07:54.326 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_background_flushes: -1 2024-04-25T00:07:54.326 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Compression algorithms supported: 2024-04-25T00:07:54.326 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T00:07:54.326 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: kZSTD supported: 0 2024-04-25T00:07:54.326 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: kXpressCompression supported: 0 2024-04-25T00:07:54.326 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T00:07:54.326 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: kZlibCompression supported: 1 2024-04-25T00:07:54.326 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: kSnappyCompression supported: 1 2024-04-25T00:07:54.326 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: kLZ4Compression supported: 1 2024-04-25T00:07:54.326 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: kBZip2Compression supported: 0 2024-04-25T00:07:54.326 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T00:07:54.327 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T00:07:54.327 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-25T00:07:54.327 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T00:07:54.327 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T00:07:54.327 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.merge_operator: 2024-04-25T00:07:54.327 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.compaction_filter: None 2024-04-25T00:07:54.327 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T00:07:54.327 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T00:07:54.327 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T00:07:54.327 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T00:07:54.327 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d38f11dba0) 2024-04-25T00:07:54.327 INFO:journalctl@ceph.mon.b.smithi170.stdout: cache_index_and_filter_blocks: 1 2024-04-25T00:07:54.327 INFO:journalctl@ceph.mon.b.smithi170.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T00:07:54.327 INFO:journalctl@ceph.mon.b.smithi170.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T00:07:54.328 INFO:journalctl@ceph.mon.b.smithi170.stdout: pin_top_level_index_and_filter: 1 2024-04-25T00:07:54.328 INFO:journalctl@ceph.mon.b.smithi170.stdout: index_type: 0 2024-04-25T00:07:54.328 INFO:journalctl@ceph.mon.b.smithi170.stdout: data_block_index_type: 0 2024-04-25T00:07:54.328 INFO:journalctl@ceph.mon.b.smithi170.stdout: index_shortening: 1 2024-04-25T00:07:54.328 INFO:journalctl@ceph.mon.b.smithi170.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T00:07:54.328 INFO:journalctl@ceph.mon.b.smithi170.stdout: checksum: 4 2024-04-25T00:07:54.328 INFO:journalctl@ceph.mon.b.smithi170.stdout: no_block_cache: 0 2024-04-25T00:07:54.328 INFO:journalctl@ceph.mon.b.smithi170.stdout: block_cache: 0x55d38f121090 2024-04-25T00:07:54.328 INFO:journalctl@ceph.mon.b.smithi170.stdout: block_cache_name: BinnedLRUCache 2024-04-25T00:07:54.328 INFO:journalctl@ceph.mon.b.smithi170.stdout: block_cache_options: 2024-04-25T00:07:54.328 INFO:journalctl@ceph.mon.b.smithi170.stdout: capacity : 536870912 2024-04-25T00:07:54.328 INFO:journalctl@ceph.mon.b.smithi170.stdout: num_shard_bits : 4 2024-04-25T00:07:54.328 INFO:journalctl@ceph.mon.b.smithi170.stdout: strict_capacity_limit : 0 2024-04-25T00:07:54.329 INFO:journalctl@ceph.mon.b.smithi170.stdout: high_pri_pool_ratio: 0.000 2024-04-25T00:07:54.329 INFO:journalctl@ceph.mon.b.smithi170.stdout: block_cache_compressed: (nil) 2024-04-25T00:07:54.329 INFO:journalctl@ceph.mon.b.smithi170.stdout: persistent_cache: (nil) 2024-04-25T00:07:54.329 INFO:journalctl@ceph.mon.b.smithi170.stdout: block_size: 4096 2024-04-25T00:07:54.329 INFO:journalctl@ceph.mon.b.smithi170.stdout: block_size_deviation: 10 2024-04-25T00:07:54.329 INFO:journalctl@ceph.mon.b.smithi170.stdout: block_restart_interval: 16 2024-04-25T00:07:54.329 INFO:journalctl@ceph.mon.b.smithi170.stdout: index_block_restart_interval: 1 2024-04-25T00:07:54.329 INFO:journalctl@ceph.mon.b.smithi170.stdout: metadata_block_size: 4096 2024-04-25T00:07:54.329 INFO:journalctl@ceph.mon.b.smithi170.stdout: partition_filters: 0 2024-04-25T00:07:54.329 INFO:journalctl@ceph.mon.b.smithi170.stdout: use_delta_encoding: 1 2024-04-25T00:07:54.329 INFO:journalctl@ceph.mon.b.smithi170.stdout: filter_policy: bloomfilter 2024-04-25T00:07:54.329 INFO:journalctl@ceph.mon.b.smithi170.stdout: whole_key_filtering: 1 2024-04-25T00:07:54.329 INFO:journalctl@ceph.mon.b.smithi170.stdout: verify_compression: 0 2024-04-25T00:07:54.329 INFO:journalctl@ceph.mon.b.smithi170.stdout: read_amp_bytes_per_bit: 0 2024-04-25T00:07:54.329 INFO:journalctl@ceph.mon.b.smithi170.stdout: format_version: 5 2024-04-25T00:07:54.330 INFO:journalctl@ceph.mon.b.smithi170.stdout: enable_index_compression: 1 2024-04-25T00:07:54.330 INFO:journalctl@ceph.mon.b.smithi170.stdout: block_align: 0 2024-04-25T00:07:54.330 INFO:journalctl@ceph.mon.b.smithi170.stdout: max_auto_readahead_size: 262144 2024-04-25T00:07:54.330 INFO:journalctl@ceph.mon.b.smithi170.stdout: prepopulate_block_cache: 0 2024-04-25T00:07:54.330 INFO:journalctl@ceph.mon.b.smithi170.stdout: initial_auto_readahead_size: 8192 2024-04-25T00:07:54.330 INFO:journalctl@ceph.mon.b.smithi170.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T00:07:54.330 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T00:07:54.330 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T00:07:54.330 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.compression: NoCompression 2024-04-25T00:07:54.330 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T00:07:54.330 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T00:07:54.330 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T00:07:54.330 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.num_levels: 7 2024-04-25T00:07:54.331 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T00:07:54.331 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T00:07:54.331 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T00:07:54.331 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T00:07:54.331 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T00:07:54.331 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T00:07:54.331 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T00:07:54.331 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T00:07:54.331 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T00:07:54.331 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T00:07:54.331 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T00:07:54.331 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T00:07:54.331 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T00:07:54.331 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T00:07:54.331 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T00:07:54.332 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T00:07:54.332 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T00:07:54.332 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T00:07:54.332 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T00:07:54.332 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T00:07:54.332 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T00:07:54.332 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T00:07:54.332 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T00:07:54.332 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T00:07:54.332 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T00:07:54.332 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T00:07:54.332 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T00:07:54.332 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T00:07:54.332 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T00:07:54.332 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T00:07:54.333 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T00:07:54.333 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T00:07:54.333 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T00:07:54.333 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T00:07:54.333 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T00:07:54.333 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T00:07:54.333 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T00:07:54.333 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T00:07:54.333 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T00:07:54.333 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T00:07:54.333 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T00:07:54.333 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T00:07:54.333 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T00:07:54.333 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T00:07:54.333 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T00:07:54.333 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T00:07:54.334 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T00:07:54.334 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T00:07:54.334 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T00:07:54.334 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T00:07:54.334 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T00:07:54.334 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T00:07:54.334 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T00:07:54.334 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.table_properties_collectors: 2024-04-25T00:07:54.334 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.inplace_update_support: 0 2024-04-25T00:07:54.334 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T00:07:54.334 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T00:07:54.334 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T00:07:54.334 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T00:07:54.334 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.bloom_locality: 0 2024-04-25T00:07:54.334 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.max_successive_merges: 0 2024-04-25T00:07:54.335 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T00:07:54.335 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T00:07:54.335 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T00:07:54.335 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T00:07:54.335 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.ttl: 2592000 2024-04-25T00:07:54.335 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T00:07:54.335 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T00:07:54.335 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T00:07:54.335 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.enable_blob_files: false 2024-04-25T00:07:54.335 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.min_blob_size: 0 2024-04-25T00:07:54.335 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T00:07:54.335 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T00:07:54.335 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T00:07:54.335 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T00:07:54.335 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T00:07:54.335 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T00:07:54.336 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T00:07:54.336 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T00:07:54.336 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T00:07:54.336 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T00:07:54.336 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6669bc22-ffb5-484d-8a29-6f0be89b7f19 2024-04-25T00:07:54.336 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714003674140710, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T00:07:54.336 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T00:07:54.336 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714003674141254, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714003674, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6669bc22-ffb5-484d-8a29-6f0be89b7f19", "db_session_id": "R7IKYNU5TPI0C89LKTHJ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T00:07:54.336 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714003674141355, "job": 1, "event": "recovery_finished"} 2024-04-25T00:07:54.336 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T00:07:54.336 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T00:07:54.336 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d38f212000 2024-04-25T00:07:54.336 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: DB pointer 0x55d38f1fc000 2024-04-25T00:07:54.336 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T00:07:54.336 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T00:07:54.337 INFO:journalctl@ceph.mon.b.smithi170.stdout: ** DB Stats ** 2024-04-25T00:07:54.337 INFO:journalctl@ceph.mon.b.smithi170.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T00:07:54.337 INFO:journalctl@ceph.mon.b.smithi170.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T00:07:54.337 INFO:journalctl@ceph.mon.b.smithi170.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T00:07:54.337 INFO:journalctl@ceph.mon.b.smithi170.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T00:07:54.337 INFO:journalctl@ceph.mon.b.smithi170.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T00:07:54.337 INFO:journalctl@ceph.mon.b.smithi170.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T00:07:54.337 INFO:journalctl@ceph.mon.b.smithi170.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T00:07:54.337 INFO:journalctl@ceph.mon.b.smithi170.stdout: 2024-04-25T00:07:54.337 INFO:journalctl@ceph.mon.b.smithi170.stdout: ** Compaction Stats [default] ** 2024-04-25T00:07:54.337 INFO:journalctl@ceph.mon.b.smithi170.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T00:07:54.337 INFO:journalctl@ceph.mon.b.smithi170.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T00:07:54.337 INFO:journalctl@ceph.mon.b.smithi170.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:07:54.337 INFO:journalctl@ceph.mon.b.smithi170.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:07:54.337 INFO:journalctl@ceph.mon.b.smithi170.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:07:54.338 INFO:journalctl@ceph.mon.b.smithi170.stdout: 2024-04-25T00:07:54.338 INFO:journalctl@ceph.mon.b.smithi170.stdout: ** Compaction Stats [default] ** 2024-04-25T00:07:54.338 INFO:journalctl@ceph.mon.b.smithi170.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T00:07:54.338 INFO:journalctl@ceph.mon.b.smithi170.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T00:07:54.338 INFO:journalctl@ceph.mon.b.smithi170.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:07:54.338 INFO:journalctl@ceph.mon.b.smithi170.stdout: 2024-04-25T00:07:54.338 INFO:journalctl@ceph.mon.b.smithi170.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T00:07:54.338 INFO:journalctl@ceph.mon.b.smithi170.stdout: 2024-04-25T00:07:54.338 INFO:journalctl@ceph.mon.b.smithi170.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T00:07:54.338 INFO:journalctl@ceph.mon.b.smithi170.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T00:07:54.338 INFO:journalctl@ceph.mon.b.smithi170.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T00:07:54.338 INFO:journalctl@ceph.mon.b.smithi170.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T00:07:54.338 INFO:journalctl@ceph.mon.b.smithi170.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T00:07:54.338 INFO:journalctl@ceph.mon.b.smithi170.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T00:07:54.339 INFO:journalctl@ceph.mon.b.smithi170.stdout: Cumulative compaction: 0.00 GB write, 0.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T00:07:54.339 INFO:journalctl@ceph.mon.b.smithi170.stdout: Interval compaction: 0.00 GB write, 0.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T00:07:54.339 INFO:journalctl@ceph.mon.b.smithi170.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T00:07:54.339 INFO:journalctl@ceph.mon.b.smithi170.stdout: Block cache BinnedLRUCache@0x55d38f121090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-25T00:07:54.339 INFO:journalctl@ceph.mon.b.smithi170.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T00:07:54.339 INFO:journalctl@ceph.mon.b.smithi170.stdout: 2024-04-25T00:07:54.339 INFO:journalctl@ceph.mon.b.smithi170.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T00:07:54.339 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-25T00:07:54.339 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: using public_addr v2:172.21.15.170:0/0 -> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] 2024-04-25T00:07:54.339 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: starting mon.b rank -1 at public addrs [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] at bind addrs [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:07:54.339 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: mon.b@-1(???) e0 preinit fsid 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:07:54.339 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: mon.b@-1(synchronizing).mds e1 new map 2024-04-25T00:07:54.339 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-25T00:07:54.339 INFO:journalctl@ceph.mon.b.smithi170.stdout: e1 2024-04-25T00:07:54.339 INFO:journalctl@ceph.mon.b.smithi170.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T00:07:54.339 INFO:journalctl@ceph.mon.b.smithi170.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T00:07:54.340 INFO:journalctl@ceph.mon.b.smithi170.stdout: legacy client fscid: -1 2024-04-25T00:07:54.340 INFO:journalctl@ceph.mon.b.smithi170.stdout: 2024-04-25T00:07:54.340 INFO:journalctl@ceph.mon.b.smithi170.stdout: No filesystems configured 2024-04-25T00:07:54.340 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T00:07:54.340 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T00:07:54.340 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T00:07:54.340 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T00:07:54.340 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T00:07:54.340 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T00:07:54.340 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T00:07:54.340 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:07:54.340 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:07:54.340 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:07:54.340 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi195", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:07:54.340 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:54.340 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:54.341 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:54.341 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:54.341 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:07:54.341 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:07:54.341 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:07:54.341 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: Updating smithi170:/etc/ceph/ceph.conf 2024-04-25T00:07:54.341 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: Deploying cephadm binary to smithi195 2024-04-25T00:07:54.341 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: Updating smithi170:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:07:54.341 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:07:54.341 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: Updating smithi170:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:07:54.341 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:54.341 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:54.341 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:54.341 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:54.341 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:54.341 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:54.342 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:07:54.342 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: Added host smithi195 2024-04-25T00:07:54.342 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:54.342 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:54.342 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:54.342 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:07:54.342 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:54.342 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:54.342 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:54.342 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/204275765' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T00:07:54.342 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:54.342 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:54.342 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:54.342 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:54.343 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi195", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:07:54.343 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:07:54.343 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:07:54.343 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: Updating smithi195:/etc/ceph/ceph.conf 2024-04-25T00:07:54.343 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: Updating smithi195:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:07:54.343 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/204275765' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T00:07:54.343 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:07:54.343 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: Updating smithi195:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:07:54.343 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:54.343 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:54.343 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:54.343 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: Updating smithi195:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:07:54.343 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:54.343 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi170:172.21.15.170=b;smithi195:172.21.15.195=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:07:54.343 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: Saving service mon spec with placement smithi107:172.21.15.107=a;smithi170:172.21.15.170=b;smithi195:172.21.15.195=c;count:3 2024-04-25T00:07:54.344 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:54.344 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:07:54.344 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:07:54.344 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:07:54.344 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:54.344 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:07:54.344 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:07:54.344 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: Deploying daemon mon.c on smithi195 2024-04-25T00:07:54.344 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:54.344 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:54.344 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:07:54.344 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: mon.a calling monitor election 2024-04-25T00:07:54.344 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:07:54.344 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:07:54.345 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:54.345 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:07:54.345 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: mon.c calling monitor election 2024-04-25T00:07:54.345 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:07:54.345 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:54.345 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:07:54.345 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:07:54.345 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T00:07:54.345 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: monmap epoch 2 2024-04-25T00:07:54.345 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: fsid 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:07:54.345 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: last_changed 2024-04-25T00:07:45.253386+0000 2024-04-25T00:07:54.345 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: created 2024-04-25T00:05:52.163469+0000 2024-04-25T00:07:54.345 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: min_mon_release 19 (squid) 2024-04-25T00:07:54.345 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: election_strategy: 1 2024-04-25T00:07:54.345 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-04-25T00:07:54.346 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-04-25T00:07:54.346 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: fsmap 2024-04-25T00:07:54.346 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:07:54.346 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: mgrmap e13: x(active, since 49s) 2024-04-25T00:07:54.346 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: overall HEALTH_OK 2024-04-25T00:07:54.346 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:54.346 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:54.346 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:54.346 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:07:54.346 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:07:54.346 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: Deploying daemon mon.b on smithi170 2024-04-25T00:07:54.346 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:54.346 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:07:54.346 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: from='client.? 172.21.15.195:0/461834893' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:07:54.347 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:54.347 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 ceph-mon[32807]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T00:07:54.347 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 bash[32789]: 21037d33514913d2b774353f386998aa34b2df3702c6492f51e3d224e71a9ca1 2024-04-25T00:07:54.347 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:07:54 smithi170 systemd[1]: Started Ceph mon.b for 79602f58-0297-11ef-bc93-c7b262605968. 2024-04-25T00:07:59.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:07:59.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: mon.c calling monitor election 2024-04-25T00:07:59.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: mon.a calling monitor election 2024-04-25T00:07:59.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:07:59.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:07:59.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:59.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:07:59.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:07:59.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:59.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:07:59.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:07:59.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:59.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:07:59.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T00:07:59.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: monmap epoch 3 2024-04-25T00:07:59.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: fsid 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:07:59.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: last_changed 2024-04-25T00:07:54.164889+0000 2024-04-25T00:07:59.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: created 2024-04-25T00:05:52.163469+0000 2024-04-25T00:07:59.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: min_mon_release 19 (squid) 2024-04-25T00:07:59.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: election_strategy: 1 2024-04-25T00:07:59.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-04-25T00:07:59.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-04-25T00:07:59.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: 2: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.b 2024-04-25T00:07:59.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: fsmap 2024-04-25T00:07:59.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:07:59.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: mgrmap e13: x(active, since 58s) 2024-04-25T00:07:59.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: overall HEALTH_OK 2024-04-25T00:07:59.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:59.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:59.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:59.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:07:59 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:59.547 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.546+0000 7f82f9fc6700 1 -- 172.21.15.195:0/4130210071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82dc0035c0 msgr2=0x7f82dc005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:59.547 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.546+0000 7f82f9fc6700 1 --2- 172.21.15.195:0/4130210071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82dc0035c0 0x7f82dc005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f82f4078470 tx=0x7f82e8010e70 comp rx=0 tx=0).stop 2024-04-25T00:07:59.547 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.546+0000 7f82f9fc6700 1 -- 172.21.15.195:0/4130210071 shutdown_connections 2024-04-25T00:07:59.547 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.546+0000 7f82f9fc6700 1 --2- 172.21.15.195:0/4130210071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82dc0035c0 0x7f82dc005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.547 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.546+0000 7f82f9fc6700 1 -- 172.21.15.195:0/4130210071 >> 172.21.15.195:0/4130210071 conn(0x7f82f4077110 msgr2=0x7f82f4077520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:59.547 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.546+0000 7f82f9fc6700 1 -- 172.21.15.195:0/4130210071 shutdown_connections 2024-04-25T00:07:59.548 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.546+0000 7f82f9fc6700 1 -- 172.21.15.195:0/4130210071 wait complete. 2024-04-25T00:07:59.548 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:07:59.548 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: mon.c calling monitor election 2024-04-25T00:07:59.548 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: mon.a calling monitor election 2024-04-25T00:07:59.548 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:07:59.548 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:07:59.548 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:59.548 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:07:59.548 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:07:59.549 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:59.549 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:07:59.549 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:07:59.549 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:07:59.549 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:07:59.549 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T00:07:59.549 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: monmap epoch 3 2024-04-25T00:07:59.549 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: fsid 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:07:59.549 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: last_changed 2024-04-25T00:07:54.164889+0000 2024-04-25T00:07:59.549 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: created 2024-04-25T00:05:52.163469+0000 2024-04-25T00:07:59.549 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: min_mon_release 19 (squid) 2024-04-25T00:07:59.549 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: election_strategy: 1 2024-04-25T00:07:59.549 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-04-25T00:07:59.549 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-04-25T00:07:59.550 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: 2: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.b 2024-04-25T00:07:59.550 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: fsmap 2024-04-25T00:07:59.550 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:07:59.550 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: mgrmap e13: x(active, since 58s) 2024-04-25T00:07:59.550 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: overall HEALTH_OK 2024-04-25T00:07:59.550 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:59.550 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:59.550 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:59.550 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:07:59 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:07:59.553 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.547+0000 7f82f9fc6700 1 Processor -- start 2024-04-25T00:07:59.553 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.547+0000 7f82f9fc6700 1 -- start start 2024-04-25T00:07:59.553 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.547+0000 7f82f9fc6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82dc0035c0 0x7f82f419d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:59.553 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.547+0000 7f82f9fc6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82f419dc20 0x7f82f41978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:59.553 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.547+0000 7f82f9fc6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f4197de0 0x7f82f4198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:59.553 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.547+0000 7f82f9fc6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f82f4107b80 con 0x7f82f4197de0 2024-04-25T00:07:59.553 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.547+0000 7f82f9fc6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f82f4107a00 con 0x7f82dc0035c0 2024-04-25T00:07:59.553 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.547+0000 7f82f9fc6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f82f4107d00 con 0x7f82f419dc20 2024-04-25T00:07:59.553 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.548+0000 7f82f2ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82f419dc20 0x7f82f41978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:59.553 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.548+0000 7f82f2ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82f419dc20 0x7f82f41978a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:54946/0 (socket says 172.21.15.195:54946) 2024-04-25T00:07:59.553 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.548+0000 7f82f2ffd700 1 -- 172.21.15.195:0/2569359810 learned_addr learned my addr 172.21.15.195:0/2569359810 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-04-25T00:07:59.554 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.548+0000 7f82f37fe700 1 --2- 172.21.15.195:0/2569359810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82dc0035c0 0x7f82f419d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:59.554 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.548+0000 7f82f2ffd700 1 -- 172.21.15.195:0/2569359810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82f419dc20 msgr2=0x7f82f41978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T00:07:59.554 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.548+0000 7f82f2ffd700 1 -- 172.21.15.195:0/2569359810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82f419dc20 msgr2=0x7f82f41978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T00:07:59.554 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.548+0000 7f82f2ffd700 1 --2- 172.21.15.195:0/2569359810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82f419dc20 0x7f82f41978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T00:07:59.554 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.548+0000 7f82f2ffd700 1 --2- 172.21.15.195:0/2569359810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82f419dc20 0x7f82f41978a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:07:59.554 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.548+0000 7f82f37fe700 1 -- 172.21.15.195:0/2569359810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82dc0035c0 msgr2=0x7f82f419d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T00:07:59.554 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.548+0000 7f82f37fe700 1 -- 172.21.15.195:0/2569359810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82dc0035c0 msgr2=0x7f82f419d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T00:07:59.554 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.548+0000 7f82f37fe700 1 --2- 172.21.15.195:0/2569359810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82dc0035c0 0x7f82f419d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T00:07:59.554 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.548+0000 7f82f37fe700 1 --2- 172.21.15.195:0/2569359810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82dc0035c0 0x7f82f419d6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:07:59.554 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.549+0000 7f82f3fff700 1 --2- 172.21.15.195:0/2569359810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f4197de0 0x7f82f4198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:59.554 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.549+0000 7f82f3fff700 1 -- 172.21.15.195:0/2569359810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82f419dc20 msgr2=0x7f82f41978a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:07:59.554 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.549+0000 7f82f3fff700 1 --2- 172.21.15.195:0/2569359810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82f419dc20 0x7f82f41978a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.555 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.549+0000 7f82f3fff700 1 -- 172.21.15.195:0/2569359810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82dc0035c0 msgr2=0x7f82f419d6e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:07:59.555 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.549+0000 7f82f3fff700 1 --2- 172.21.15.195:0/2569359810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82dc0035c0 0x7f82f419d6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.555 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.549+0000 7f82f3fff700 1 -- 172.21.15.195:0/2569359810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f41a45b0 con 0x7f82f4197de0 2024-04-25T00:07:59.555 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.550+0000 7f82f3fff700 1 --2- 172.21.15.195:0/2569359810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f4197de0 0x7f82f4198290 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00bd50 tx=0x7f82ec00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:59.555 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.550+0000 7f82f0ff9700 1 -- 172.21.15.195:0/2569359810 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec019070 con 0x7f82f4197de0 2024-04-25T00:07:59.555 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.550+0000 7f82f0ff9700 1 -- 172.21.15.195:0/2569359810 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82ec00cd10 con 0x7f82f4197de0 2024-04-25T00:07:59.555 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.550+0000 7f82f9fc6700 1 -- 172.21.15.195:0/2569359810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f41a47e0 con 0x7f82f4197de0 2024-04-25T00:07:59.555 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.550+0000 7f82f0ff9700 1 -- 172.21.15.195:0/2569359810 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec0127e0 con 0x7f82f4197de0 2024-04-25T00:07:59.555 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.550+0000 7f82f9fc6700 1 -- 172.21.15.195:0/2569359810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f41a4d70 con 0x7f82f4197de0 2024-04-25T00:07:59.555 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.551+0000 7f82f0ff9700 1 -- 172.21.15.195:0/2569359810 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f82ec01c070 con 0x7f82f4197de0 2024-04-25T00:07:59.555 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.551+0000 7f82f0ff9700 1 --2- 172.21.15.195:0/2569359810 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f82e0039580 0x7f82e003ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:59.555 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.551+0000 7f82f0ff9700 1 -- 172.21.15.195:0/2569359810 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f82ec04cb60 con 0x7f82f4197de0 2024-04-25T00:07:59.555 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.551+0000 7f82f37fe700 1 --2- 172.21.15.195:0/2569359810 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f82e0039580 0x7f82e003ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:59.555 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.551+0000 7f82f9fc6700 1 -- 172.21.15.195:0/2569359810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f4063890 con 0x7f82f4197de0 2024-04-25T00:07:59.556 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.552+0000 7f82f37fe700 1 --2- 172.21.15.195:0/2569359810 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f82e0039580 0x7f82e003ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f82e8010a20 tx=0x7f82e8001e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:59.557 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.557+0000 7f82f0ff9700 1 -- 172.21.15.195:0/2569359810 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f82ec013c30 con 0x7f82f4197de0 2024-04-25T00:07:59.843 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.842+0000 7f82f9fc6700 1 -- 172.21.15.195:0/2569359810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f82f41a4fd0 con 0x7f82f4197de0 2024-04-25T00:07:59.843 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.843+0000 7f82f0ff9700 1 -- 172.21.15.195:0/2569359810 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1287 (secure 0 0 0) 0x7f82ec01d730 con 0x7f82f4197de0 2024-04-25T00:07:59.844 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-04-25T00:07:59.844 INFO:teuthology.orchestra.run.smithi195.stdout:{"epoch":3,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","modified":"2024-04-25T00:07:54.164889Z","created":"2024-04-25T00:05:52.163469Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:3300","nonce":0},{"type":"v1","addr":"172.21.15.195:6789","nonce":0}]},"addr":"172.21.15.195:6789/0","public_addr":"172.21.15.195:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:3300","nonce":0},{"type":"v1","addr":"172.21.15.170:6789","nonce":0}]},"addr":"172.21.15.170:6789/0","public_addr":"172.21.15.170:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T00:07:59.845 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.845+0000 7f82f9fc6700 1 -- 172.21.15.195:0/2569359810 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f82e0039580 msgr2=0x7f82e003ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:59.846 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.845+0000 7f82f9fc6700 1 --2- 172.21.15.195:0/2569359810 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f82e0039580 0x7f82e003ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f82e8010a20 tx=0x7f82e8001e50 comp rx=0 tx=0).stop 2024-04-25T00:07:59.846 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.845+0000 7f82f9fc6700 1 -- 172.21.15.195:0/2569359810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f4197de0 msgr2=0x7f82f4198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:59.846 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.845+0000 7f82f9fc6700 1 --2- 172.21.15.195:0/2569359810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f4197de0 0x7f82f4198290 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f82ec00bd50 tx=0x7f82ec00be30 comp rx=0 tx=0).stop 2024-04-25T00:07:59.846 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.845+0000 7f82f9fc6700 1 -- 172.21.15.195:0/2569359810 shutdown_connections 2024-04-25T00:07:59.846 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.845+0000 7f82f9fc6700 1 --2- 172.21.15.195:0/2569359810 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f82e0039580 0x7f82e003ba40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.846 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.845+0000 7f82f9fc6700 1 --2- 172.21.15.195:0/2569359810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82f4197de0 0x7f82f4198290 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.846 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.845+0000 7f82f9fc6700 1 --2- 172.21.15.195:0/2569359810 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82dc0035c0 0x7f82f419d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.846 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.845+0000 7f82f9fc6700 1 --2- 172.21.15.195:0/2569359810 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82f419dc20 0x7f82f41978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:59.846 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.845+0000 7f82f9fc6700 1 -- 172.21.15.195:0/2569359810 >> 172.21.15.195:0/2569359810 conn(0x7f82f4077110 msgr2=0x7f82f40fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:59.846 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.845+0000 7f82f9fc6700 1 -- 172.21.15.195:0/2569359810 shutdown_connections 2024-04-25T00:07:59.846 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:07:59.845+0000 7f82f9fc6700 1 -- 172.21.15.195:0/2569359810 wait complete. 2024-04-25T00:07:59.847 INFO:teuthology.orchestra.run.smithi195.stderr:dumped monmap epoch 3 2024-04-25T00:08:00.427 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:00 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:00.427 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:00 smithi195 ceph-mon[32833]: from='client.? 172.21.15.195:0/2569359810' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:08:00.427 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:00 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:08:00.453 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T00:08:00.453 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T00:08:00.482 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:00 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:00.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:00 smithi107 ceph-mon[29153]: from='client.? 172.21.15.195:0/2569359810' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:08:00.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:00 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:08:00.701 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:08:01.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:08:01.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: mon.c calling monitor election 2024-04-25T00:08:01.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: mon.a calling monitor election 2024-04-25T00:08:01.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:08:01.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:08:01.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:01.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:08:01.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:08:01.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:01.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:08:01.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:08:01.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:01.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:08:01.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T00:08:01.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: monmap epoch 3 2024-04-25T00:08:01.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: fsid 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:08:01.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: last_changed 2024-04-25T00:07:54.164889+0000 2024-04-25T00:08:01.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: created 2024-04-25T00:05:52.163469+0000 2024-04-25T00:08:01.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: min_mon_release 19 (squid) 2024-04-25T00:08:01.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: election_strategy: 1 2024-04-25T00:08:01.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-04-25T00:08:01.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-04-25T00:08:01.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: 2: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.b 2024-04-25T00:08:01.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: fsmap 2024-04-25T00:08:01.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:08:01.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: mgrmap e13: x(active, since 58s) 2024-04-25T00:08:01.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: overall HEALTH_OK 2024-04-25T00:08:01.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:01.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:01.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:01.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:01.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:01.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='client.? 172.21.15.195:0/2569359810' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:08:01.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:08:02.167 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: mon.b calling monitor election 2024-04-25T00:08:02.167 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: mon.b calling monitor election 2024-04-25T00:08:02.167 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: mon.a calling monitor election 2024-04-25T00:08:02.167 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: mon.c calling monitor election 2024-04-25T00:08:02.167 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T00:08:02.167 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: monmap epoch 3 2024-04-25T00:08:02.167 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: fsid 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:08:02.167 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: last_changed 2024-04-25T00:07:54.164889+0000 2024-04-25T00:08:02.167 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: created 2024-04-25T00:05:52.163469+0000 2024-04-25T00:08:02.168 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: min_mon_release 19 (squid) 2024-04-25T00:08:02.168 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: election_strategy: 1 2024-04-25T00:08:02.168 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-04-25T00:08:02.168 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-04-25T00:08:02.168 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: 2: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.b 2024-04-25T00:08:02.168 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: fsmap 2024-04-25T00:08:02.168 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:08:02.168 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: mgrmap e13: x(active, since 60s) 2024-04-25T00:08:02.168 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: overall HEALTH_OK 2024-04-25T00:08:02.168 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.168 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.168 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:02.168 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:02.168 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.168 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.169 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.169 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.169 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.169 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.169 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: mon.b calling monitor election 2024-04-25T00:08:02.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: mon.b calling monitor election 2024-04-25T00:08:02.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: mon.a calling monitor election 2024-04-25T00:08:02.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: mon.c calling monitor election 2024-04-25T00:08:02.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T00:08:02.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: monmap epoch 3 2024-04-25T00:08:02.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: fsid 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:08:02.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: last_changed 2024-04-25T00:07:54.164889+0000 2024-04-25T00:08:02.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: created 2024-04-25T00:05:52.163469+0000 2024-04-25T00:08:02.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: min_mon_release 19 (squid) 2024-04-25T00:08:02.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: election_strategy: 1 2024-04-25T00:08:02.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-04-25T00:08:02.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-04-25T00:08:02.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: 2: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.b 2024-04-25T00:08:02.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: fsmap 2024-04-25T00:08:02.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:08:02.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: mgrmap e13: x(active, since 60s) 2024-04-25T00:08:02.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: overall HEALTH_OK 2024-04-25T00:08:02.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:02.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:02.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: mon.b calling monitor election 2024-04-25T00:08:02.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: mon.b calling monitor election 2024-04-25T00:08:02.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: mon.a calling monitor election 2024-04-25T00:08:02.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: mon.c calling monitor election 2024-04-25T00:08:02.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T00:08:02.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: monmap epoch 3 2024-04-25T00:08:02.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: fsid 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:08:02.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: last_changed 2024-04-25T00:07:54.164889+0000 2024-04-25T00:08:02.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: created 2024-04-25T00:05:52.163469+0000 2024-04-25T00:08:02.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: min_mon_release 19 (squid) 2024-04-25T00:08:02.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: election_strategy: 1 2024-04-25T00:08:02.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-04-25T00:08:02.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: 1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.c 2024-04-25T00:08:02.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: 2: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.b 2024-04-25T00:08:02.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: fsmap 2024-04-25T00:08:02.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:08:02.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: mgrmap e13: x(active, since 60s) 2024-04-25T00:08:02.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: overall HEALTH_OK 2024-04-25T00:08:02.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:02.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:02.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:02.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.410+0000 7f46ba066700 1 -- 172.21.15.107:0/3281481474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46b4106be0 msgr2=0x7f46b4106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:02.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.410+0000 7f46ba066700 1 --2- 172.21.15.107:0/3281481474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46b4106be0 0x7f46b4106fc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009910 tx=0x7f46a4009c20 comp rx=0 tx=0).stop 2024-04-25T00:08:02.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.410+0000 7f46ba066700 1 -- 172.21.15.107:0/3281481474 shutdown_connections 2024-04-25T00:08:02.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.410+0000 7f46ba066700 1 --2- 172.21.15.107:0/3281481474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46b4106be0 0x7f46b4106fc0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.410+0000 7f46ba066700 1 -- 172.21.15.107:0/3281481474 >> 172.21.15.107:0/3281481474 conn(0x7f46b4077800 msgr2=0x7f46b40feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:02.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.410+0000 7f46ba066700 1 -- 172.21.15.107:0/3281481474 shutdown_connections 2024-04-25T00:08:02.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.410+0000 7f46ba066700 1 -- 172.21.15.107:0/3281481474 wait complete. 2024-04-25T00:08:02.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.410+0000 7f46ba066700 1 Processor -- start 2024-04-25T00:08:02.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.411+0000 7f46ba066700 1 -- start start 2024-04-25T00:08:02.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.411+0000 7f46ba066700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46b4106be0 0x7f46b419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:02.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.411+0000 7f46ba066700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46b419db60 0x7f46b41977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:02.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.411+0000 7f46ba066700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46b4197ce0 0x7f46b4198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:02.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.411+0000 7f46ba066700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f46b410b150 con 0x7f46b419db60 2024-04-25T00:08:02.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.411+0000 7f46ba066700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f46b410afd0 con 0x7f46b4197ce0 2024-04-25T00:08:02.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.411+0000 7f46ba066700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f46b410b2d0 con 0x7f46b4106be0 2024-04-25T00:08:02.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.411+0000 7f46b2ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46b419db60 0x7f46b41977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:02.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.411+0000 7f46b2ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46b419db60 0x7f46b41977a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39506/0 (socket says 172.21.15.107:39506) 2024-04-25T00:08:02.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.411+0000 7f46b2ffd700 1 -- 172.21.15.107:0/273039675 learned_addr learned my addr 172.21.15.107:0/273039675 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:08:02.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.411+0000 7f46b3fff700 1 --2- 172.21.15.107:0/273039675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46b4197ce0 0x7f46b4198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:02.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.411+0000 7f46b37fe700 1 --2- 172.21.15.107:0/273039675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46b4106be0 0x7f46b419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:02.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.411+0000 7f46b2ffd700 1 -- 172.21.15.107:0/273039675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46b4106be0 msgr2=0x7f46b419d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:02.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.411+0000 7f46b2ffd700 1 --2- 172.21.15.107:0/273039675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46b4106be0 0x7f46b419d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.411+0000 7f46b2ffd700 1 -- 172.21.15.107:0/273039675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46b4197ce0 msgr2=0x7f46b4198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:02.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.411+0000 7f46b2ffd700 1 --2- 172.21.15.107:0/273039675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46b4197ce0 0x7f46b4198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.411+0000 7f46b2ffd700 1 -- 172.21.15.107:0/273039675 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b41a4480 con 0x7f46b419db60 2024-04-25T00:08:02.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.411+0000 7f46b2ffd700 1 --2- 172.21.15.107:0/273039675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46b419db60 0x7f46b41977a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f46a800b780 tx=0x7f46a800ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:02.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.412+0000 7f46b0ff9700 1 -- 172.21.15.107:0/273039675 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a8004230 con 0x7f46b419db60 2024-04-25T00:08:02.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.412+0000 7f46b0ff9700 1 -- 172.21.15.107:0/273039675 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46a80043d0 con 0x7f46b419db60 2024-04-25T00:08:02.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.412+0000 7f46b0ff9700 1 -- 172.21.15.107:0/273039675 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a8011c90 con 0x7f46b419db60 2024-04-25T00:08:02.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.413+0000 7f46ba066700 1 -- 172.21.15.107:0/273039675 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b41a4710 con 0x7f46b419db60 2024-04-25T00:08:02.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.413+0000 7f46ba066700 1 -- 172.21.15.107:0/273039675 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b41a4c50 con 0x7f46b419db60 2024-04-25T00:08:02.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.413+0000 7f46b0ff9700 1 -- 172.21.15.107:0/273039675 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f46a80036a0 con 0x7f46b419db60 2024-04-25T00:08:02.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.413+0000 7f46ba066700 1 -- 172.21.15.107:0/273039675 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b41021e0 con 0x7f46b419db60 2024-04-25T00:08:02.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.414+0000 7f46b0ff9700 1 --2- 172.21.15.107:0/273039675 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f469c039580 0x7f469c03ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:02.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.414+0000 7f46b0ff9700 1 -- 172.21.15.107:0/273039675 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f46a804bba0 con 0x7f46b419db60 2024-04-25T00:08:02.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.414+0000 7f46b37fe700 1 --2- 172.21.15.107:0/273039675 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f469c039580 0x7f469c03ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:02.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.414+0000 7f46b37fe700 1 --2- 172.21.15.107:0/273039675 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f469c039580 0x7f469c03ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009750 tx=0x7f46a400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:02.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.419+0000 7f46b0ff9700 1 -- 172.21.15.107:0/273039675 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f46a80151c0 con 0x7f46b419db60 2024-04-25T00:08:02.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.628+0000 7f46ba066700 1 -- 172.21.15.107:0/273039675 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f46b4063890 con 0x7f46b419db60 2024-04-25T00:08:02.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.628+0000 7f46b0ff9700 1 -- 172.21.15.107:0/273039675 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+277 (secure 0 0 0) 0x7f46a8020de0 con 0x7f46b419db60 2024-04-25T00:08:02.630 INFO:teuthology.orchestra.run.smithi107.stdout:# minimal ceph.conf for 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:08:02.630 INFO:teuthology.orchestra.run.smithi107.stdout:[global] 2024-04-25T00:08:02.630 INFO:teuthology.orchestra.run.smithi107.stdout: fsid = 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:08:02.630 INFO:teuthology.orchestra.run.smithi107.stdout: mon_host = [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] 2024-04-25T00:08:02.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.631+0000 7f46ba066700 1 -- 172.21.15.107:0/273039675 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f469c039580 msgr2=0x7f469c03ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:02.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.631+0000 7f46ba066700 1 --2- 172.21.15.107:0/273039675 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f469c039580 0x7f469c03ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009750 tx=0x7f46a400b540 comp rx=0 tx=0).stop 2024-04-25T00:08:02.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.631+0000 7f46ba066700 1 -- 172.21.15.107:0/273039675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46b419db60 msgr2=0x7f46b41977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:02.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.631+0000 7f46ba066700 1 --2- 172.21.15.107:0/273039675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46b419db60 0x7f46b41977a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f46a800b780 tx=0x7f46a800ba90 comp rx=0 tx=0).stop 2024-04-25T00:08:02.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.631+0000 7f46ba066700 1 -- 172.21.15.107:0/273039675 shutdown_connections 2024-04-25T00:08:02.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.631+0000 7f46ba066700 1 --2- 172.21.15.107:0/273039675 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f469c039580 0x7f469c03ba40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.631+0000 7f46ba066700 1 --2- 172.21.15.107:0/273039675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46b4197ce0 0x7f46b4198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.631+0000 7f46ba066700 1 --2- 172.21.15.107:0/273039675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46b4106be0 0x7f46b419d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.631+0000 7f46ba066700 1 --2- 172.21.15.107:0/273039675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46b419db60 0x7f46b41977a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:02.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.631+0000 7f46ba066700 1 -- 172.21.15.107:0/273039675 >> 172.21.15.107:0/273039675 conn(0x7f46b4077800 msgr2=0x7f46b41046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:02.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.631+0000 7f46ba066700 1 -- 172.21.15.107:0/273039675 shutdown_connections 2024-04-25T00:08:02.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:02.631+0000 7f46ba066700 1 -- 172.21.15.107:0/273039675 wait complete. 2024-04-25T00:08:03.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:02 smithi170 ceph-mon[32807]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-25T00:08:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:02 smithi170 ceph-mon[32807]: Updating smithi170:/etc/ceph/ceph.conf 2024-04-25T00:08:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:02 smithi170 ceph-mon[32807]: Updating smithi195:/etc/ceph/ceph.conf 2024-04-25T00:08:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:02 smithi170 ceph-mon[32807]: Updating smithi107:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:08:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:02 smithi170 ceph-mon[32807]: Updating smithi170:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:08:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:02 smithi170 ceph-mon[32807]: Updating smithi195:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:08:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:02 smithi170 ceph-mon[32807]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T00:08:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:08:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:08:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:03.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:02 smithi170 ceph-mon[32807]: Reconfiguring daemon mon.a on smithi107 2024-04-25T00:08:03.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:08:03.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:02 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/273039675' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:03.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:02 smithi107 ceph-mon[29153]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-25T00:08:03.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:02 smithi107 ceph-mon[29153]: Updating smithi170:/etc/ceph/ceph.conf 2024-04-25T00:08:03.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:02 smithi107 ceph-mon[29153]: Updating smithi195:/etc/ceph/ceph.conf 2024-04-25T00:08:03.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:02 smithi107 ceph-mon[29153]: Updating smithi107:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:08:03.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:02 smithi107 ceph-mon[29153]: Updating smithi170:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:08:03.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:02 smithi107 ceph-mon[29153]: Updating smithi195:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:08:03.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:03.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:03.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:03.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:03.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:02 smithi107 ceph-mon[29153]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T00:08:03.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:08:03.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:08:03.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:03.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:02 smithi107 ceph-mon[29153]: Reconfiguring daemon mon.a on smithi107 2024-04-25T00:08:03.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:08:03.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:02 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/273039675' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:03.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:02 smithi195 ceph-mon[32833]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-25T00:08:03.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:02 smithi195 ceph-mon[32833]: Updating smithi170:/etc/ceph/ceph.conf 2024-04-25T00:08:03.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:02 smithi195 ceph-mon[32833]: Updating smithi195:/etc/ceph/ceph.conf 2024-04-25T00:08:03.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:02 smithi195 ceph-mon[32833]: Updating smithi107:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:08:03.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:02 smithi195 ceph-mon[32833]: Updating smithi170:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:08:03.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:02 smithi195 ceph-mon[32833]: Updating smithi195:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:08:03.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:03.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:03.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:03.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:03.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:02 smithi195 ceph-mon[32833]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T00:08:03.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:08:03.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:08:03.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:03.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:02 smithi195 ceph-mon[32833]: Reconfiguring daemon mon.a on smithi107 2024-04-25T00:08:03.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:08:03.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:02 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/273039675' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:04.008 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T00:08:04.009 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T00:08:04.009 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T00:08:04.077 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T00:08:04.077 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:08:04.143 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-04-25T00:08:04.143 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T00:08:04.166 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:03 smithi170 ceph-mon[32807]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:04.166 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:03 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:04.166 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:03 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:04.166 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:03 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:08:04.166 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:03 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:08:04.166 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:03 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:04.171 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-04-25T00:08:04.171 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:08:04.234 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-04-25T00:08:04.235 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T00:08:04.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:03 smithi107 ceph-mon[29153]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:04.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:03 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:04.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:03 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:04.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:03 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:08:04.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:03 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:08:04.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:03 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:04.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:03 smithi195 ceph-mon[32833]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:04.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:03 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:04.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:03 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:04.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:03 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:08:04.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:03 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:08:04.261 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:03 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:04.267 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-04-25T00:08:04.267 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:08:04.335 INFO:tasks.cephadm:Adding mgr.x on smithi107 2024-04-25T00:08:04.335 INFO:tasks.cephadm:Adding mgr.y on smithi170 2024-04-25T00:08:04.336 INFO:tasks.cephadm:Adding mgr.z on smithi195 2024-04-25T00:08:04.336 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi107=x;smithi170=y;smithi195=z' 2024-04-25T00:08:04.618 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:08:05.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:04 smithi195 ceph-mon[32833]: Reconfiguring mon.b (monmap changed)... 2024-04-25T00:08:05.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:04 smithi195 ceph-mon[32833]: Reconfiguring daemon mon.b on smithi170 2024-04-25T00:08:05.183 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:04 smithi170 ceph-mon[32807]: Reconfiguring mon.b (monmap changed)... 2024-04-25T00:08:05.183 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:04 smithi170 ceph-mon[32807]: Reconfiguring daemon mon.b on smithi170 2024-04-25T00:08:05.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:04 smithi107 ceph-mon[29153]: Reconfiguring mon.b (monmap changed)... 2024-04-25T00:08:05.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:04 smithi107 ceph-mon[29153]: Reconfiguring daemon mon.b on smithi170 2024-04-25T00:08:06.069 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.066+0000 7f6b7098c700 1 -- 172.21.15.195:0/2029266196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b6c101dc0 msgr2=0x7f6b6c1021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:06.069 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.066+0000 7f6b7098c700 1 --2- 172.21.15.195:0/2029266196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b6c101dc0 0x7f6b6c1021e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009910 tx=0x7f6b5c009c20 comp rx=0 tx=0).stop 2024-04-25T00:08:06.069 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.066+0000 7f6b7098c700 1 -- 172.21.15.195:0/2029266196 shutdown_connections 2024-04-25T00:08:06.069 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.066+0000 7f6b7098c700 1 --2- 172.21.15.195:0/2029266196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b6c101dc0 0x7f6b6c1021e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.069 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.066+0000 7f6b7098c700 1 -- 172.21.15.195:0/2029266196 >> 172.21.15.195:0/2029266196 conn(0x7f6b6c0fd360 msgr2=0x7f6b6c0ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:06.069 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.066+0000 7f6b7098c700 1 -- 172.21.15.195:0/2029266196 shutdown_connections 2024-04-25T00:08:06.069 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.066+0000 7f6b7098c700 1 -- 172.21.15.195:0/2029266196 wait complete. 2024-04-25T00:08:06.070 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.067+0000 7f6b7098c700 1 Processor -- start 2024-04-25T00:08:06.070 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.067+0000 7f6b7098c700 1 -- start start 2024-04-25T00:08:06.070 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.067+0000 7f6b7098c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b6c101dc0 0x7f6b6c197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:06.070 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.067+0000 7f6b7098c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b6c197b80 0x7f6b6c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:06.070 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.067+0000 7f6b7098c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b6c198000 0x7f6b6c19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:06.070 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.067+0000 7f6b7098c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c107e00 con 0x7f6b6c198000 2024-04-25T00:08:06.070 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.067+0000 7f6b7098c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c107c80 con 0x7f6b6c101dc0 2024-04-25T00:08:06.070 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.067+0000 7f6b7098c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c107f80 con 0x7f6b6c197b80 2024-04-25T00:08:06.070 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.068+0000 7f6b69d9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b6c197b80 0x7f6b6c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:06.070 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.068+0000 7f6b69d9b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b6c197b80 0x7f6b6c19cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:48710/0 (socket says 172.21.15.195:48710) 2024-04-25T00:08:06.070 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.068+0000 7f6b69d9b700 1 -- 172.21.15.195:0/2053582195 learned_addr learned my addr 172.21.15.195:0/2053582195 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-04-25T00:08:06.070 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.068+0000 7f6b6ad9d700 1 --2- 172.21.15.195:0/2053582195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b6c198000 0x7f6b6c19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:06.070 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.068+0000 7f6b6a59c700 1 --2- 172.21.15.195:0/2053582195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b6c101dc0 0x7f6b6c197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:06.071 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.068+0000 7f6b69d9b700 1 -- 172.21.15.195:0/2053582195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b6c101dc0 msgr2=0x7f6b6c197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:06.071 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.068+0000 7f6b69d9b700 1 --2- 172.21.15.195:0/2053582195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b6c101dc0 0x7f6b6c197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.071 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.068+0000 7f6b69d9b700 1 -- 172.21.15.195:0/2053582195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b6c198000 msgr2=0x7f6b6c19f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:06.071 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.068+0000 7f6b69d9b700 1 --2- 172.21.15.195:0/2053582195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b6c198000 0x7f6b6c19f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.071 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.068+0000 7f6b69d9b700 1 -- 172.21.15.195:0/2053582195 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6c19fa00 con 0x7f6b6c197b80 2024-04-25T00:08:06.071 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.068+0000 7f6b6ad9d700 1 --2- 172.21.15.195:0/2053582195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b6c198000 0x7f6b6c19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:08:06.071 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.068+0000 7f6b69d9b700 1 --2- 172.21.15.195:0/2053582195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b6c197b80 0x7f6b6c19cba0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6b6000b780 tx=0x7f6b6000ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:06.071 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.068+0000 7f6b5b7fe700 1 -- 172.21.15.195:0/2053582195 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b60016070 con 0x7f6b6c197b80 2024-04-25T00:08:06.071 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.068+0000 7f6b5b7fe700 1 -- 172.21.15.195:0/2053582195 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b600036a0 con 0x7f6b6c197b80 2024-04-25T00:08:06.071 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.068+0000 7f6b7098c700 1 -- 172.21.15.195:0/2053582195 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b6c19fcf0 con 0x7f6b6c197b80 2024-04-25T00:08:06.071 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.068+0000 7f6b5b7fe700 1 -- 172.21.15.195:0/2053582195 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b60011af0 con 0x7f6b6c197b80 2024-04-25T00:08:06.071 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.068+0000 7f6b7098c700 1 -- 172.21.15.195:0/2053582195 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b6c1a0250 con 0x7f6b6c197b80 2024-04-25T00:08:06.072 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.069+0000 7f6b5b7fe700 1 -- 172.21.15.195:0/2053582195 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f6b60019070 con 0x7f6b6c197b80 2024-04-25T00:08:06.072 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.069+0000 7f6b7098c700 1 -- 172.21.15.195:0/2053582195 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b6c063890 con 0x7f6b6c197b80 2024-04-25T00:08:06.072 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.069+0000 7f6b5b7fe700 1 --2- 172.21.15.195:0/2053582195 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f6b54039580 0x7f6b5403ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:06.072 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.069+0000 7f6b5b7fe700 1 -- 172.21.15.195:0/2053582195 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6b6004ad50 con 0x7f6b6c197b80 2024-04-25T00:08:06.073 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.070+0000 7f6b6a59c700 1 --2- 172.21.15.195:0/2053582195 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f6b54039580 0x7f6b5403ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:06.073 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.070+0000 7f6b6a59c700 1 --2- 172.21.15.195:0/2053582195 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f6b54039580 0x7f6b5403ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c005200 tx=0x7f6b5c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:06.075 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.075+0000 7f6b5b7fe700 1 -- 172.21.15.195:0/2053582195 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6b6004e030 con 0x7f6b6c197b80 2024-04-25T00:08:06.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:05 smithi170 ceph-mon[32807]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:06.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:05 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:06.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:05 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:06.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:05 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:08:06.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:05 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:08:06.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:05 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:06.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:05 smithi107 ceph-mon[29153]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:06.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:05 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:06.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:05 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:06.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:05 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:08:06.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:05 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:08:06.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:05 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:06.237 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:05 smithi195 ceph-mon[32833]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:06.237 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:05 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:06.238 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:05 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:06.238 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:05 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:08:06.238 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:05 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:08:06.238 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:05 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:06.288 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.288+0000 7f6b7098c700 1 -- 172.21.15.195:0/2053582195 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi107=x;smithi170=y;smithi195=z", "target": ["mon-mgr", ""]}) -- 0x7f6b6c062700 con 0x7f6b54039580 2024-04-25T00:08:06.294 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.294+0000 7f6b5b7fe700 1 -- 172.21.15.195:0/2053582195 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6b6c062700 con 0x7f6b54039580 2024-04-25T00:08:06.295 INFO:teuthology.orchestra.run.smithi195.stdout:Scheduled mgr update... 2024-04-25T00:08:06.297 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.296+0000 7f6b7098c700 1 -- 172.21.15.195:0/2053582195 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f6b54039580 msgr2=0x7f6b5403ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:06.297 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.296+0000 7f6b7098c700 1 --2- 172.21.15.195:0/2053582195 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f6b54039580 0x7f6b5403ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c005200 tx=0x7f6b5c01a040 comp rx=0 tx=0).stop 2024-04-25T00:08:06.297 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.296+0000 7f6b7098c700 1 -- 172.21.15.195:0/2053582195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b6c197b80 msgr2=0x7f6b6c19cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:06.297 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.296+0000 7f6b7098c700 1 --2- 172.21.15.195:0/2053582195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b6c197b80 0x7f6b6c19cba0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6b6000b780 tx=0x7f6b6000ba90 comp rx=0 tx=0).stop 2024-04-25T00:08:06.297 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.296+0000 7f6b7098c700 1 -- 172.21.15.195:0/2053582195 shutdown_connections 2024-04-25T00:08:06.297 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.296+0000 7f6b7098c700 1 --2- 172.21.15.195:0/2053582195 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f6b54039580 0x7f6b5403ba40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.297 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.296+0000 7f6b7098c700 1 --2- 172.21.15.195:0/2053582195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b6c198000 0x7f6b6c19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.297 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.296+0000 7f6b7098c700 1 --2- 172.21.15.195:0/2053582195 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b6c101dc0 0x7f6b6c197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.297 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.296+0000 7f6b7098c700 1 --2- 172.21.15.195:0/2053582195 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6b6c197b80 0x7f6b6c19cba0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.298 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.296+0000 7f6b7098c700 1 -- 172.21.15.195:0/2053582195 >> 172.21.15.195:0/2053582195 conn(0x7f6b6c0fd360 msgr2=0x7f6b6c0fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:06.298 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.296+0000 7f6b7098c700 1 -- 172.21.15.195:0/2053582195 shutdown_connections 2024-04-25T00:08:06.298 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:08:06.296+0000 7f6b7098c700 1 -- 172.21.15.195:0/2053582195 wait complete. 2024-04-25T00:08:06.925 DEBUG:teuthology.orchestra.run.smithi170:mgr.y> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T00:08:06.928 DEBUG:teuthology.orchestra.run.smithi195:mgr.z> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T00:08:06.930 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T00:08:06.931 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T00:08:06.931 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T00:08:06.950 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T00:08:06.950 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_1 2024-04-25T00:08:07.007 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T00:08:07.007 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:08:07.007 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T00:08:07.007 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:08:07.007 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:08:07.007 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-04-25 00:07:11.077166901 +0000 2024-04-25T00:08:07.007 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-04-25 00:02:21.259530599 +0000 2024-04-25T00:08:07.007 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-04-25 00:02:21.259530599 +0000 2024-04-25T00:08:07.007 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-04-25 00:02:21.259530599 +0000 2024-04-25T00:08:07.008 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T00:08:07.029 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:06 smithi195 ceph-mon[32833]: Reconfiguring mon.c (monmap changed)... 2024-04-25T00:08:07.029 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:06 smithi195 ceph-mon[32833]: Reconfiguring daemon mon.c on smithi195 2024-04-25T00:08:07.029 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:06 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:07.072 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-04-25T00:08:07.073 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-04-25T00:08:07.073 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000231046 s, 2.2 MB/s 2024-04-25T00:08:07.074 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T00:08:07.131 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_2 2024-04-25T00:08:07.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:06 smithi170 ceph-mon[32807]: Reconfiguring mon.c (monmap changed)... 2024-04-25T00:08:07.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:06 smithi170 ceph-mon[32807]: Reconfiguring daemon mon.c on smithi195 2024-04-25T00:08:07.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:06 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:07.188 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T00:08:07.188 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:08:07.188 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T00:08:07.188 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:08:07.188 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:08:07.188 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-04-25 00:07:11.106166421 +0000 2024-04-25T00:08:07.188 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-04-25 00:02:21.601524846 +0000 2024-04-25T00:08:07.188 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-04-25 00:02:21.601524846 +0000 2024-04-25T00:08:07.188 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-04-25 00:02:21.601524846 +0000 2024-04-25T00:08:07.189 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T00:08:07.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:06 smithi107 ceph-mon[29153]: Reconfiguring mon.c (monmap changed)... 2024-04-25T00:08:07.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:06 smithi107 ceph-mon[29153]: Reconfiguring daemon mon.c on smithi195 2024-04-25T00:08:07.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:06 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:07.252 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-04-25T00:08:07.252 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-04-25T00:08:07.252 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000216703 s, 2.4 MB/s 2024-04-25T00:08:07.253 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T00:08:07.309 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_3 2024-04-25T00:08:07.366 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T00:08:07.366 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:08:07.366 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T00:08:07.366 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:08:07.366 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:08:07.366 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-04-25 00:07:11.132165990 +0000 2024-04-25T00:08:07.366 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-04-25 00:02:21.944519076 +0000 2024-04-25T00:08:07.366 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-04-25 00:02:21.944519076 +0000 2024-04-25T00:08:07.366 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-04-25 00:02:21.944519076 +0000 2024-04-25T00:08:07.367 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T00:08:07.429 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-04-25T00:08:07.430 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-04-25T00:08:07.430 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000236165 s, 2.2 MB/s 2024-04-25T00:08:07.431 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T00:08:07.487 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_4 2024-04-25T00:08:07.542 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T00:08:07.542 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:08:07.542 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T00:08:07.542 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:08:07.542 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:08:07.542 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-04-25 00:07:11.157165575 +0000 2024-04-25T00:08:07.542 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-04-25 00:02:22.292513221 +0000 2024-04-25T00:08:07.542 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-04-25 00:02:22.292513221 +0000 2024-04-25T00:08:07.543 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-04-25 00:02:22.292513221 +0000 2024-04-25T00:08:07.543 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T00:08:07.604 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-04-25T00:08:07.604 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-04-25T00:08:07.604 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000171725 s, 3.0 MB/s 2024-04-25T00:08:07.605 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T00:08:07.661 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-04-25T00:08:07.661 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T00:08:07.680 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T00:08:07.680 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_1 2024-04-25T00:08:07.738 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T00:08:07.738 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:08:07.739 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T00:08:07.739 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:08:07.739 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:08:07.739 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-04-25 00:07:53.633484675 +0000 2024-04-25T00:08:07.739 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-04-25 00:02:56.749470377 +0000 2024-04-25T00:08:07.739 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-04-25 00:02:56.749470377 +0000 2024-04-25T00:08:07.739 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-04-25 00:02:56.749470377 +0000 2024-04-25T00:08:07.739 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T00:08:07.805 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-04-25T00:08:07.805 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-04-25T00:08:07.805 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000129044 s, 4.0 MB/s 2024-04-25T00:08:07.806 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T00:08:07.864 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_2 2024-04-25T00:08:07.920 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T00:08:07.921 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:08:07.921 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T00:08:07.921 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:08:07.921 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:08:07.921 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-04-25 00:07:53.633484675 +0000 2024-04-25T00:08:07.921 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-04-25 00:02:57.066463111 +0000 2024-04-25T00:08:07.921 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-04-25 00:02:57.066463111 +0000 2024-04-25T00:08:07.921 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-04-25 00:02:57.066463111 +0000 2024-04-25T00:08:07.921 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T00:08:07.985 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-04-25T00:08:07.985 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-04-25T00:08:07.985 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000140756 s, 3.6 MB/s 2024-04-25T00:08:07.987 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T00:08:08.044 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_3 2024-04-25T00:08:08.100 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T00:08:08.100 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:08:08.101 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T00:08:08.101 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:08:08.101 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:08:08.101 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-04-25 00:07:53.633484675 +0000 2024-04-25T00:08:08.101 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-04-25 00:02:57.402455409 +0000 2024-04-25T00:08:08.101 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-04-25 00:02:57.402455409 +0000 2024-04-25T00:08:08.101 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-04-25 00:02:57.402455409 +0000 2024-04-25T00:08:08.101 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T00:08:08.161 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:07 smithi170 ceph-mon[32807]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi107=x;smithi170=y;smithi195=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:08:08.161 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:07 smithi170 ceph-mon[32807]: Saving service mgr spec with placement smithi107=x;smithi170=y;smithi195=z;count:3 2024-04-25T00:08:08.161 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:07 smithi170 ceph-mon[32807]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:08.162 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:07 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:08.162 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:07 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:08.162 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:07 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:08.163 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-04-25T00:08:08.163 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-04-25T00:08:08.163 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000118431 s, 4.3 MB/s 2024-04-25T00:08:08.165 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T00:08:08.222 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_4 2024-04-25T00:08:08.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:07 smithi107 ceph-mon[29153]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi107=x;smithi170=y;smithi195=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:08:08.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:07 smithi107 ceph-mon[29153]: Saving service mgr spec with placement smithi107=x;smithi170=y;smithi195=z;count:3 2024-04-25T00:08:08.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:07 smithi107 ceph-mon[29153]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:08.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:07 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:08.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:07 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:08.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:07 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:08.278 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T00:08:08.278 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:08:08.278 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T00:08:08.278 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:08:08.279 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:08:08.279 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-04-25 00:07:53.634484652 +0000 2024-04-25T00:08:08.279 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-04-25 00:02:57.738447707 +0000 2024-04-25T00:08:08.279 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-04-25 00:02:57.738447707 +0000 2024-04-25T00:08:08.279 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-04-25 00:02:57.738447707 +0000 2024-04-25T00:08:08.279 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T00:08:08.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:07 smithi195 ceph-mon[32833]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi107=x;smithi170=y;smithi195=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:08:08.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:07 smithi195 ceph-mon[32833]: Saving service mgr spec with placement smithi107=x;smithi170=y;smithi195=z;count:3 2024-04-25T00:08:08.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:07 smithi195 ceph-mon[32833]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:08.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:07 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:08.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:07 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:08.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:07 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:08.341 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-04-25T00:08:08.341 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-04-25T00:08:08.341 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 8.7247e-05 s, 5.9 MB/s 2024-04-25T00:08:08.342 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T00:08:08.398 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-04-25T00:08:08.398 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T00:08:08.414 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T00:08:08.414 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_1 2024-04-25T00:08:08.471 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T00:08:08.471 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:08:08.471 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T00:08:08.471 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:08:08.471 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:08:08.471 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-04-25 00:07:44.055640865 +0000 2024-04-25T00:08:08.471 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-04-25 00:02:48.705775717 +0000 2024-04-25T00:08:08.472 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-04-25 00:02:48.705775717 +0000 2024-04-25T00:08:08.472 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-04-25 00:02:48.705775717 +0000 2024-04-25T00:08:08.472 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T00:08:08.534 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-04-25T00:08:08.534 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-04-25T00:08:08.534 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000110854 s, 4.6 MB/s 2024-04-25T00:08:08.535 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T00:08:08.590 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_2 2024-04-25T00:08:08.646 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T00:08:08.647 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:08:08.647 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T00:08:08.647 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:08:08.647 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:08:08.647 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-04-25 00:07:44.055640865 +0000 2024-04-25T00:08:08.648 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-04-25 00:02:49.038769449 +0000 2024-04-25T00:08:08.648 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-04-25 00:02:49.038769449 +0000 2024-04-25T00:08:08.648 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-04-25 00:02:49.038769449 +0000 2024-04-25T00:08:08.648 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T00:08:08.709 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-04-25T00:08:08.709 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-04-25T00:08:08.709 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000112177 s, 4.6 MB/s 2024-04-25T00:08:08.711 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T00:08:08.766 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_3 2024-04-25T00:08:08.821 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T00:08:08.821 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:08:08.821 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T00:08:08.821 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:08:08.821 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:08:08.822 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-04-25 00:07:44.056640847 +0000 2024-04-25T00:08:08.822 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-04-25 00:02:49.384762935 +0000 2024-04-25T00:08:08.822 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-04-25 00:02:49.384762935 +0000 2024-04-25T00:08:08.822 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-04-25 00:02:49.384762935 +0000 2024-04-25T00:08:08.822 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T00:08:08.885 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-04-25T00:08:08.885 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-04-25T00:08:08.885 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 8.8052e-05 s, 5.8 MB/s 2024-04-25T00:08:08.886 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T00:08:08.941 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_4 2024-04-25T00:08:08.996 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T00:08:08.996 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:08:08.996 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T00:08:08.996 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:08:08.996 INFO:teuthology.orchestra.run.smithi195.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:08:08.996 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-04-25 00:07:44.056640847 +0000 2024-04-25T00:08:08.996 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-04-25 00:02:49.720756610 +0000 2024-04-25T00:08:08.996 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-04-25 00:02:49.720756610 +0000 2024-04-25T00:08:08.996 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: 2024-04-25 00:02:49.720756610 +0000 2024-04-25T00:08:08.996 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T00:08:09.057 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-04-25T00:08:09.058 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-04-25T00:08:09.058 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000110082 s, 4.7 MB/s 2024-04-25T00:08:09.059 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T00:08:09.115 INFO:tasks.cephadm:Deploying osd.0 on smithi107 with /dev/vg_nvme/lv_4... 2024-04-25T00:08:09.115 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T00:08:09.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:08 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:09.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:08 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:09.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:08 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:09.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:08 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:08:09.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:08 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T00:08:09.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:08 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:08:09.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:08 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:09.201 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:08 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:09.201 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:08 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:09.201 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:08 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:09.201 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:08 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:08:09.202 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:08 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T00:08:09.202 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:08 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:08:09.202 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:08 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:09.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:08 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:09.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:08 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:09.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:08 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:09.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:08 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:08:09.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:08 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T00:08:09.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:08 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:08:09.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:08 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:09.356 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:08:10.173 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:09 smithi195 ceph-mon[32833]: Deploying daemon mgr.z on smithi195 2024-04-25T00:08:10.173 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:09 smithi195 ceph-mon[32833]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:10.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:09 smithi170 ceph-mon[32807]: Deploying daemon mgr.z on smithi195 2024-04-25T00:08:10.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:09 smithi170 ceph-mon[32807]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:10.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:09 smithi107 ceph-mon[29153]: Deploying daemon mgr.z on smithi195 2024-04-25T00:08:10.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:09 smithi107 ceph-mon[29153]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:10.557 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:10 smithi195 systemd[1]: Starting Ceph mgr.z for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:08:10.834 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:10 smithi195 podman[34686]: 2024-04-25 00:08:10.557563397 +0000 UTC m=+0.111864895 container create 52ec1f7c266ae7c2707d8caafa86c2e8efd88dd45998dda67259f6c5ac40c96e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424) 2024-04-25T00:08:10.835 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:10 smithi195 podman[34686]: 2024-04-25 00:08:10.463458922 +0000 UTC m=+0.017760394 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:08:10.835 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:10 smithi195 podman[34686]: 2024-04-25 00:08:10.776441386 +0000 UTC m=+0.330742841 container init 52ec1f7c266ae7c2707d8caafa86c2e8efd88dd45998dda67259f6c5ac40c96e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T00:08:10.835 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:10 smithi195 podman[34686]: 2024-04-25 00:08:10.779039893 +0000 UTC m=+0.333341350 container start 52ec1f7c266ae7c2707d8caafa86c2e8efd88dd45998dda67259f6c5ac40c96e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T00:08:10.835 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:10 smithi195 ceph-mgr[34706]: -- 172.21.15.195:0/1551509486 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ad7049a5a0 con 0x55ad7046e800 2024-04-25T00:08:10.835 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:10 smithi195 bash[34686]: 52ec1f7c266ae7c2707d8caafa86c2e8efd88dd45998dda67259f6c5ac40c96e 2024-04-25T00:08:11.089 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:10 smithi195 systemd[1]: Started Ceph mgr.z for 79602f58-0297-11ef-bc93-c7b262605968. 2024-04-25T00:08:11.089 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:10 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:10.934+0000 7fc772474200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:08:11.196 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T00:08:11.197 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T00:08:11.197 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-04-25T00:08:11.197 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-04-25T00:08:11.197 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132666 s, 790 MB/s 2024-04-25T00:08:11.197 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-04-25T00:08:11.525 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:11 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:11.114+0000 7fc772474200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:08:11.525 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:11 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:11.216+0000 7fc772474200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:08:11.526 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:11 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:11.525+0000 7fc772474200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:08:11.764 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_4 2024-04-25T00:08:11.839 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:11 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:11.609+0000 7fc772474200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:08:12.008 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:08:12.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:11 smithi107 ceph-mon[29153]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:12.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:11 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:12.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:11 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:12.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:11 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:12.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:11 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:08:12.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:11 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T00:08:12.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:11 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:08:12.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:11 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:12.114 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:11 smithi107 ceph-mon[29153]: Deploying daemon mgr.y on smithi170 2024-04-25T00:08:12.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:11 smithi170 ceph-mon[32807]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:12.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:11 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:12.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:11 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:12.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:11 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:12.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:11 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:08:12.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:11 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T00:08:12.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:11 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:08:12.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:11 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:12.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:11 smithi170 ceph-mon[32807]: Deploying daemon mgr.y on smithi170 2024-04-25T00:08:12.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:11 smithi195 ceph-mon[32833]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:12.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:11 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:12.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:11 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:12.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:11 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:12.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:11 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:08:12.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:11 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T00:08:12.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:11 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:08:12.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:11 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:12.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:11 smithi195 ceph-mon[32833]: Deploying daemon mgr.y on smithi170 2024-04-25T00:08:12.935 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:12 smithi170 systemd[1]: Starting Ceph mgr.y for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:08:13.089 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:12 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:12.801+0000 7fc772474200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:08:13.089 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:12 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:12.974+0000 7fc772474200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:08:13.312 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:13 smithi170 podman[34177]: 2024-04-25 00:08:12.95136095 +0000 UTC m=+0.018191398 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:08:13.313 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:13 smithi170 podman[34177]: 2024-04-25 00:08:13.051892199 +0000 UTC m=+0.118722619 container create 655ed70eabc80bf95f6219a8040b8130c30956292630472dd34950ae477b220c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:08:13.313 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:13 smithi170 podman[34177]: 2024-04-25 00:08:13.246200808 +0000 UTC m=+0.313031229 container init 655ed70eabc80bf95f6219a8040b8130c30956292630472dd34950ae477b220c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T00:08:13.313 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:13 smithi170 podman[34177]: 2024-04-25 00:08:13.249714538 +0000 UTC m=+0.316544959 container start 655ed70eabc80bf95f6219a8040b8130c30956292630472dd34950ae477b220c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T00:08:13.313 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:13 smithi170 ceph-mgr[34198]: -- 172.21.15.170:0/3187359224 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560824ad05a0 con 0x560824aa4800 2024-04-25T00:08:13.313 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:13 smithi170 bash[34177]: 655ed70eabc80bf95f6219a8040b8130c30956292630472dd34950ae477b220c 2024-04-25T00:08:13.453 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:13 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:13.149+0000 7fc772474200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:08:13.453 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:13 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:13.450+0000 7fc772474200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:08:13.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.544+0000 7f56f4961700 1 -- 172.21.15.107:0/4112058935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56f01034b0 msgr2=0x7f56f0103910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:13.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.544+0000 7f56f4961700 1 --2- 172.21.15.107:0/4112058935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56f01034b0 0x7f56f0103910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f56e40099c0 tx=0x7f56e4009cd0 comp rx=0 tx=0).stop 2024-04-25T00:08:13.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.545+0000 7f56f4961700 1 -- 172.21.15.107:0/4112058935 shutdown_connections 2024-04-25T00:08:13.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.545+0000 7f56f4961700 1 --2- 172.21.15.107:0/4112058935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56f0103e50 0x7f56f010a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:13.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.545+0000 7f56f4961700 1 --2- 172.21.15.107:0/4112058935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56f01022b0 0x7f56f01026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:13.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.545+0000 7f56f4961700 1 --2- 172.21.15.107:0/4112058935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56f01034b0 0x7f56f0103910 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:13.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.545+0000 7f56f4961700 1 -- 172.21.15.107:0/4112058935 >> 172.21.15.107:0/4112058935 conn(0x7f56f00fd850 msgr2=0x7f56f00ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:13.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.545+0000 7f56f4961700 1 -- 172.21.15.107:0/4112058935 shutdown_connections 2024-04-25T00:08:13.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.545+0000 7f56f4961700 1 -- 172.21.15.107:0/4112058935 wait complete. 2024-04-25T00:08:13.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.545+0000 7f56f4961700 1 Processor -- start 2024-04-25T00:08:13.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.545+0000 7f56f4961700 1 -- start start 2024-04-25T00:08:13.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.545+0000 7f56f4961700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56f01022b0 0x7f56f0198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:13.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.545+0000 7f56f4961700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56f01034b0 0x7f56f0198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:13.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.546+0000 7f56f4961700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56f0103e50 0x7f56f019fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:13.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.546+0000 7f56f4961700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f56f010de90 con 0x7f56f01022b0 2024-04-25T00:08:13.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.546+0000 7f56f4961700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f56f010dd10 con 0x7f56f01034b0 2024-04-25T00:08:13.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.546+0000 7f56f4961700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f56f010e010 con 0x7f56f0103e50 2024-04-25T00:08:13.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.546+0000 7f56ee59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56f01022b0 0x7f56f0198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:13.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.546+0000 7f56ee59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56f01022b0 0x7f56f0198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38866/0 (socket says 172.21.15.107:38866) 2024-04-25T00:08:13.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.546+0000 7f56ee59c700 1 -- 172.21.15.107:0/330673089 learned_addr learned my addr 172.21.15.107:0/330673089 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:08:13.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.546+0000 7f56edd9b700 1 --2- 172.21.15.107:0/330673089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56f01034b0 0x7f56f0198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:13.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.546+0000 7f56eed9d700 1 --2- 172.21.15.107:0/330673089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56f0103e50 0x7f56f019fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:13.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.546+0000 7f56ee59c700 1 -- 172.21.15.107:0/330673089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56f0103e50 msgr2=0x7f56f019fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:13.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.546+0000 7f56ee59c700 1 --2- 172.21.15.107:0/330673089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56f0103e50 0x7f56f019fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:13.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.546+0000 7f56ee59c700 1 -- 172.21.15.107:0/330673089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56f01034b0 msgr2=0x7f56f0198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:13.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.546+0000 7f56ee59c700 1 --2- 172.21.15.107:0/330673089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56f01034b0 0x7f56f0198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:13.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.546+0000 7f56ee59c700 1 -- 172.21.15.107:0/330673089 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56f01a00b0 con 0x7f56f01022b0 2024-04-25T00:08:13.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.546+0000 7f56ee59c700 1 --2- 172.21.15.107:0/330673089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56f01022b0 0x7f56f0198450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f56e000d9b0 tx=0x7f56e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:13.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.546+0000 7f56df7fe700 1 -- 172.21.15.107:0/330673089 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e0017030 con 0x7f56f01022b0 2024-04-25T00:08:13.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.546+0000 7f56df7fe700 1 -- 172.21.15.107:0/330673089 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56e0004500 con 0x7f56f01022b0 2024-04-25T00:08:13.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.546+0000 7f56f4961700 1 -- 172.21.15.107:0/330673089 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56f01a03a0 con 0x7f56f01022b0 2024-04-25T00:08:13.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.546+0000 7f56df7fe700 1 -- 172.21.15.107:0/330673089 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e0003970 con 0x7f56f01022b0 2024-04-25T00:08:13.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.546+0000 7f56f4961700 1 -- 172.21.15.107:0/330673089 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56f01a0930 con 0x7f56f01022b0 2024-04-25T00:08:13.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.547+0000 7f56df7fe700 1 -- 172.21.15.107:0/330673089 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 46344+0+0 (secure 0 0 0) 0x7f56e001a070 con 0x7f56f01022b0 2024-04-25T00:08:13.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.547+0000 7f56f4961700 1 -- 172.21.15.107:0/330673089 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56f0068f20 con 0x7f56f01022b0 2024-04-25T00:08:13.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.547+0000 7f56df7fe700 1 --2- 172.21.15.107:0/330673089 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f56d8039330 0x7f56d803b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:13.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.547+0000 7f56df7fe700 1 -- 172.21.15.107:0/330673089 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f56e004bbb0 con 0x7f56f01022b0 2024-04-25T00:08:13.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.547+0000 7f56edd9b700 1 --2- 172.21.15.107:0/330673089 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f56d8039330 0x7f56d803b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:13.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.548+0000 7f56edd9b700 1 --2- 172.21.15.107:0/330673089 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f56d8039330 0x7f56d803b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f56e4000c00 tx=0x7f56e4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:13.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.552+0000 7f56df7fe700 1 -- 172.21.15.107:0/330673089 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f56e001abf0 con 0x7f56f01022b0 2024-04-25T00:08:13.579 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:13 smithi170 systemd[1]: Started Ceph mgr.y for 79602f58-0297-11ef-bc93-c7b262605968. 2024-04-25T00:08:13.579 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:13 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:13.425+0000 7f554bf75200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:08:13.759 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:13 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:13.545+0000 7fc772474200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:08:13.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:13.762+0000 7f56f4961700 1 -- 172.21.15.107:0/330673089 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f56f0062670 con 0x7f56d8039330 2024-04-25T00:08:13.906 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:13 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:13.619+0000 7f554bf75200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:08:13.906 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:13 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:13.731+0000 7f554bf75200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:08:14.089 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:13 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:13.758+0000 7fc772474200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:08:14.089 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:13 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:13.981+0000 7fc772474200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:08:14.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:13 smithi195 ceph-mon[32833]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:14.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:13 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:14.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:13 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:14.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:13 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:14.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:13 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:14.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:13 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:14.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:13 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:08:14.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:13 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:08:14.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:13 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:14.162 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:14 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:14.062+0000 7f554bf75200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:08:14.162 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:14 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:14.162+0000 7f554bf75200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:08:14.163 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:13 smithi170 ceph-mon[32807]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:14.163 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:13 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:14.163 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:13 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:14.163 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:13 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:14.163 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:13 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:14.163 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:13 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:14.163 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:13 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:08:14.163 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:13 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:08:14.163 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:13 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:14.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:13 smithi107 ceph-mon[29153]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:14.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:13 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:14.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:13 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:14.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:13 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:14.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:13 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:14.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:13 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:14.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:13 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:08:14.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:13 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:08:14.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:13 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:14.839 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:14 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:14.627+0000 7fc772474200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:08:15.162 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:15 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:15.064+0000 7fc772474200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:08:15.163 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:15 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:15.162+0000 7fc772474200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:08:15.163 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:14 smithi195 ceph-mon[32833]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:08:15.163 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:14 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:15.163 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:14 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:15.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:14 smithi170 ceph-mon[32807]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:08:15.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:14 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:15.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:14 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:15.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:14 smithi107 ceph-mon[29153]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:08:15.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:14 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:15.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:14 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:15.589 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:15 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:15.361+0000 7fc772474200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:08:15.684 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:15 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:15.397+0000 7f554bf75200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:08:15.684 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:15 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:15.564+0000 7f554bf75200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:08:16.023 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:15 smithi170 ceph-mon[32807]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:16.023 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:15 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:16.023 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:15 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:16.023 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:15 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:16.023 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:15 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:16.023 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:15 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:16.023 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:15 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:08:16.023 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:15 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:08:16.023 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:15 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:16.023 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:15 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:16.024 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:15 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:15.740+0000 7f554bf75200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:08:16.024 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:16 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:16.022+0000 7f554bf75200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:08:16.220 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:15 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:15.885+0000 7fc772474200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:08:16.220 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:15 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:15.976+0000 7fc772474200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:08:16.220 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:15 smithi195 ceph-mon[32833]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:16.220 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:15 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:16.220 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:15 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:16.220 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:15 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:16.220 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:15 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:16.221 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:15 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:16.221 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:15 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:08:16.221 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:15 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:08:16.221 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:15 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:16.221 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:15 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:16.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:15 smithi107 ceph-mon[29153]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:16.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:15 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:16.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:15 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:16.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:15 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:16.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:15 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:16.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:15 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:16.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:15 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:08:16.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:15 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:08:16.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:15 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:16.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:15 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:16.316 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:16 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:16.112+0000 7f554bf75200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:08:16.316 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:16 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:16.315+0000 7f554bf75200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:08:16.567 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:16 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:16.219+0000 7fc772474200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:08:16.567 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:16 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:16.353+0000 7fc772474200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:08:16.567 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:16 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:16.449+0000 7fc772474200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:08:16.567 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:16 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:16.566+0000 7fc772474200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:08:16.684 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:16 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:16.504+0000 7f554bf75200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:08:16.839 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:16 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:16.659+0000 7fc772474200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:08:17.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:16 smithi107 ceph-mon[29153]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T00:08:17.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:16 smithi107 ceph-mon[29153]: Reconfiguring daemon mgr.x on smithi107 2024-04-25T00:08:17.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:16 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3580890323' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a74e9b9-fc11-4472-913b-ea0084ac1492"}]: dispatch 2024-04-25T00:08:17.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:16 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3580890323' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a74e9b9-fc11-4472-913b-ea0084ac1492"}]': finished 2024-04-25T00:08:17.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:16 smithi107 ceph-mon[29153]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T00:08:17.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:16 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:08:17.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:16 smithi195 ceph-mon[32833]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T00:08:17.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:16 smithi195 ceph-mon[32833]: Reconfiguring daemon mgr.x on smithi107 2024-04-25T00:08:17.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:16 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3580890323' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a74e9b9-fc11-4472-913b-ea0084ac1492"}]: dispatch 2024-04-25T00:08:17.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:16 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3580890323' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a74e9b9-fc11-4472-913b-ea0084ac1492"}]': finished 2024-04-25T00:08:17.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:16 smithi195 ceph-mon[32833]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T00:08:17.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:16 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:08:17.434 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:17 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:17.101+0000 7f554bf75200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:08:17.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:16 smithi170 ceph-mon[32807]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T00:08:17.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:16 smithi170 ceph-mon[32807]: Reconfiguring daemon mgr.x on smithi107 2024-04-25T00:08:17.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:16 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3580890323' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5a74e9b9-fc11-4472-913b-ea0084ac1492"}]: dispatch 2024-04-25T00:08:17.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:16 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3580890323' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5a74e9b9-fc11-4472-913b-ea0084ac1492"}]': finished 2024-04-25T00:08:17.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:16 smithi170 ceph-mon[32807]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T00:08:17.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:16 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:08:17.809 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:17 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:17.516+0000 7f554bf75200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:08:17.809 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:17 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:17.611+0000 7f554bf75200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:08:17.809 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:17 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:17.808+0000 7f554bf75200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:08:17.839 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:17 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:17.477+0000 7fc772474200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:08:17.839 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:08:17 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:08:17.694+0000 7fc772474200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:08:18.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:17 smithi170 ceph-mon[32807]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:18.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:17 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3186920871' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:08:18.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:17 smithi170 ceph-mon[32807]: Standby manager daemon z started 2024-04-25T00:08:18.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:17 smithi170 ceph-mon[32807]: from='mgr.? 172.21.15.195:0/3845852079' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T00:08:18.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:17 smithi170 ceph-mon[32807]: from='mgr.? 172.21.15.195:0/3845852079' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:08:18.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:17 smithi170 ceph-mon[32807]: from='mgr.? 172.21.15.195:0/3845852079' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T00:08:18.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:17 smithi170 ceph-mon[32807]: from='mgr.? 172.21.15.195:0/3845852079' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:08:18.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:17 smithi107 ceph-mon[29153]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:18.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:17 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3186920871' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:08:18.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:17 smithi107 ceph-mon[29153]: Standby manager daemon z started 2024-04-25T00:08:18.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:17 smithi107 ceph-mon[29153]: from='mgr.? 172.21.15.195:0/3845852079' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T00:08:18.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:17 smithi107 ceph-mon[29153]: from='mgr.? 172.21.15.195:0/3845852079' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:08:18.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:17 smithi107 ceph-mon[29153]: from='mgr.? 172.21.15.195:0/3845852079' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T00:08:18.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:17 smithi107 ceph-mon[29153]: from='mgr.? 172.21.15.195:0/3845852079' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:08:18.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:17 smithi195 ceph-mon[32833]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:18.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:17 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3186920871' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:08:18.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:17 smithi195 ceph-mon[32833]: Standby manager daemon z started 2024-04-25T00:08:18.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:17 smithi195 ceph-mon[32833]: from='mgr.? 172.21.15.195:0/3845852079' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T00:08:18.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:17 smithi195 ceph-mon[32833]: from='mgr.? 172.21.15.195:0/3845852079' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:08:18.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:17 smithi195 ceph-mon[32833]: from='mgr.? 172.21.15.195:0/3845852079' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T00:08:18.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:17 smithi195 ceph-mon[32833]: from='mgr.? 172.21.15.195:0/3845852079' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:08:18.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:18.399+0000 7f56df7fe700 1 -- 172.21.15.107:0/330673089 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 14) ==== 92061+0+0 (secure 0 0 0) 0x7f56e00109d0 con 0x7f56f01022b0 2024-04-25T00:08:18.665 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:18 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:18.335+0000 7f554bf75200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:08:18.665 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:18 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:18.426+0000 7f554bf75200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:08:18.934 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:18 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:18.664+0000 7f554bf75200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:08:18.935 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:18 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:18.794+0000 7f554bf75200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:08:18.935 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:18 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:18.888+0000 7f554bf75200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:08:19.400 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:19 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:19.002+0000 7f554bf75200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:08:19.400 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:19 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:19.093+0000 7f554bf75200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:08:19.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:19 smithi170 ceph-mon[32807]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T00:08:19.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:19 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T00:08:19.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:19 smithi170 ceph-mon[32807]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:19.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:19 smithi107 ceph-mon[29153]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T00:08:19.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:19 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T00:08:19.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:19 smithi107 ceph-mon[29153]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:19.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:19 smithi195 ceph-mon[32833]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-25T00:08:19.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:19 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T00:08:19.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:19 smithi195 ceph-mon[32833]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:20.185 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:19 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:19.903+0000 7f554bf75200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:08:20.185 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:08:20 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:08:20.115+0000 7f554bf75200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:08:20.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:20.411+0000 7f56df7fe700 1 -- 172.21.15.107:0/330673089 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f56e000e350 con 0x7f56f01022b0 2024-04-25T00:08:20.677 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:20 smithi107 ceph-mon[29153]: Standby manager daemon y started 2024-04-25T00:08:20.678 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:20 smithi107 ceph-mon[29153]: from='mgr.? 172.21.15.170:0/670328283' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T00:08:20.678 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:20 smithi107 ceph-mon[29153]: from='mgr.? 172.21.15.170:0/670328283' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:08:20.678 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:20 smithi107 ceph-mon[29153]: from='mgr.? 172.21.15.170:0/670328283' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T00:08:20.678 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:20 smithi107 ceph-mon[29153]: from='mgr.? 172.21.15.170:0/670328283' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:08:20.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:20 smithi170 ceph-mon[32807]: Standby manager daemon y started 2024-04-25T00:08:20.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:20 smithi170 ceph-mon[32807]: from='mgr.? 172.21.15.170:0/670328283' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T00:08:20.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:20 smithi170 ceph-mon[32807]: from='mgr.? 172.21.15.170:0/670328283' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:08:20.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:20 smithi170 ceph-mon[32807]: from='mgr.? 172.21.15.170:0/670328283' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T00:08:20.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:20 smithi170 ceph-mon[32807]: from='mgr.? 172.21.15.170:0/670328283' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:08:20.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:20 smithi195 ceph-mon[32833]: Standby manager daemon y started 2024-04-25T00:08:20.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:20 smithi195 ceph-mon[32833]: from='mgr.? 172.21.15.170:0/670328283' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T00:08:20.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:20 smithi195 ceph-mon[32833]: from='mgr.? 172.21.15.170:0/670328283' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:08:20.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:20 smithi195 ceph-mon[32833]: from='mgr.? 172.21.15.170:0/670328283' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T00:08:20.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:20 smithi195 ceph-mon[32833]: from='mgr.? 172.21.15.170:0/670328283' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:08:21.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:21 smithi170 ceph-mon[32807]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-25T00:08:21.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:21 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T00:08:21.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:21 smithi170 ceph-mon[32807]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:21.708 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:21 smithi107 ceph-mon[29153]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-25T00:08:21.708 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:21 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T00:08:21.708 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:21 smithi107 ceph-mon[29153]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:21.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:21 smithi195 ceph-mon[32833]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-25T00:08:21.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:21 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T00:08:21.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:21 smithi195 ceph-mon[32833]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:22.881 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:22 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:22.881 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:22 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:22.882 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:22 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:22.882 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:22 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:22.882 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:22 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:22.882 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:22 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:23.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:22 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:23.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:22 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:23.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:22 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:23.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:22 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:23.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:22 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:23.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:22 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:23.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:22 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:23.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:22 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:23.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:22 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:23.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:22 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:23.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:22 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:23.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:22 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:23.870 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:23 smithi107 ceph-mon[29153]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:24.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:23 smithi195 ceph-mon[32833]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:24.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:23 smithi170 ceph-mon[32807]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:24.877 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:24 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T00:08:24.877 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:24 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:25.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:24 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T00:08:25.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:24 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:25.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:24 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T00:08:25.334 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:24 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:25.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:25 smithi107 ceph-mon[29153]: Deploying daemon osd.0 on smithi107 2024-04-25T00:08:25.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:25 smithi107 ceph-mon[29153]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:26.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:25 smithi195 ceph-mon[32833]: Deploying daemon osd.0 on smithi107 2024-04-25T00:08:26.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:25 smithi195 ceph-mon[32833]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:26.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:25 smithi170 ceph-mon[32807]: Deploying daemon osd.0 on smithi107 2024-04-25T00:08:26.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:25 smithi170 ceph-mon[32807]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:28.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:27 smithi170 ceph-mon[32807]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:28.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:27 smithi107 ceph-mon[29153]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:28.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:27 smithi195 ceph-mon[32833]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:30.170 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:29 smithi107 ceph-mon[29153]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:30.170 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:29 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:30.170 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:29 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:30.171 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:29 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:30.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:29 smithi170 ceph-mon[32807]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:30.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:29 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:30.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:29 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:30.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:29 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:30.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:29 smithi195 ceph-mon[32833]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:30.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:29 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:30.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:29 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:30.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:29 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:32.108 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:31 smithi107 ceph-mon[29153]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:32.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:31 smithi170 ceph-mon[32807]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:32.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:31 smithi195 ceph-mon[32833]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:33.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:33.317+0000 7f56df7fe700 1 -- 172.21.15.107:0/330673089 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f56f0062670 con 0x7f56d8039330 2024-04-25T00:08:33.318 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 0 on host 'smithi107' 2024-04-25T00:08:33.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:33.319+0000 7f56f4961700 1 -- 172.21.15.107:0/330673089 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f56d8039330 msgr2=0x7f56d803b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:33.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:33.319+0000 7f56f4961700 1 --2- 172.21.15.107:0/330673089 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f56d8039330 0x7f56d803b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f56e4000c00 tx=0x7f56e4005fb0 comp rx=0 tx=0).stop 2024-04-25T00:08:33.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:33.319+0000 7f56f4961700 1 -- 172.21.15.107:0/330673089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56f01022b0 msgr2=0x7f56f0198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:33.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:33.319+0000 7f56f4961700 1 --2- 172.21.15.107:0/330673089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56f01022b0 0x7f56f0198450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f56e000d9b0 tx=0x7f56e000dcc0 comp rx=0 tx=0).stop 2024-04-25T00:08:33.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:33.319+0000 7f56f4961700 1 -- 172.21.15.107:0/330673089 shutdown_connections 2024-04-25T00:08:33.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:33.319+0000 7f56f4961700 1 --2- 172.21.15.107:0/330673089 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f56d8039330 0x7f56d803b7f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:33.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:33.319+0000 7f56f4961700 1 --2- 172.21.15.107:0/330673089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56f0103e50 0x7f56f019fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:33.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:33.319+0000 7f56f4961700 1 --2- 172.21.15.107:0/330673089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56f01022b0 0x7f56f0198450 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:33.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:33.319+0000 7f56f4961700 1 --2- 172.21.15.107:0/330673089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56f01034b0 0x7f56f0198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:33.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:33.319+0000 7f56f4961700 1 -- 172.21.15.107:0/330673089 >> 172.21.15.107:0/330673089 conn(0x7f56f00fd850 msgr2=0x7f56f01067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:33.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:33.319+0000 7f56f4961700 1 -- 172.21.15.107:0/330673089 shutdown_connections 2024-04-25T00:08:33.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:08:33.319+0000 7f56f4961700 1 -- 172.21.15.107:0/330673089 wait complete. 2024-04-25T00:08:33.629 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:33 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:33.629 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:33 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:33.629 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:33 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:33.629 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:33 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:33.630 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:33 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:33.630 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:33 smithi107 ceph-mon[29153]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:33.630 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:33 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:33.630 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:33 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:33.630 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:33 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:33.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:33 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:33.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:33 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:33.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:33 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:33.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:33 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:33.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:33 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:33.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:33 smithi170 ceph-mon[32807]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:33.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:33 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:33.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:33 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:33.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:33 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:33.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:33 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:33.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:33 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:33.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:33 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:33.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:33 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:33.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:33 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:33.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:33 smithi195 ceph-mon[32833]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:33.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:33 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:33.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:33 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:33.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:33 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:33.972 DEBUG:teuthology.orchestra.run.smithi107:osd.0> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T00:08:33.975 INFO:tasks.cephadm:Deploying osd.1 on smithi170 with /dev/vg_nvme/lv_4... 2024-04-25T00:08:33.975 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T00:08:34.219 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:08:34.435 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:34 smithi107 ceph-mon[29153]: from='osd.0 [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T00:08:34.628 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:34 smithi170 ceph-mon[32807]: from='osd.0 [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T00:08:34.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:34 smithi195 ceph-mon[32833]: from='osd.0 [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T00:08:35.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:35 smithi107 ceph-mon[29153]: from='osd.0 [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T00:08:35.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:35 smithi107 ceph-mon[29153]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T00:08:35.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:35 smithi107 ceph-mon[29153]: from='osd.0 [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-25T00:08:35.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:35 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:08:35.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:35 smithi107 ceph-mon[29153]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:35.552 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:35 smithi170 ceph-mon[32807]: from='osd.0 [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T00:08:35.552 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:35 smithi170 ceph-mon[32807]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T00:08:35.552 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:35 smithi170 ceph-mon[32807]: from='osd.0 [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-25T00:08:35.552 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:35 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:08:35.553 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:35 smithi170 ceph-mon[32807]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:35.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:35 smithi195 ceph-mon[32833]: from='osd.0 [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T00:08:35.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:35 smithi195 ceph-mon[32833]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T00:08:35.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:35 smithi195 ceph-mon[32833]: from='osd.0 [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-25T00:08:35.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:35 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:08:35.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:35 smithi195 ceph-mon[32833]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:36.086 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T00:08:36.087 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T00:08:36.087 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-04-25T00:08:36.087 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-04-25T00:08:36.087 INFO:teuthology.orchestra.run.smithi170.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00833787 s, 1.3 GB/s 2024-04-25T00:08:36.087 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-04-25T00:08:36.343 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 25 00:08:36 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-0[41253]: 2024-04-25T00:08:36.082+0000 7f4ce478a700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T00:08:36.343 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 25 00:08:36 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-0[41253]: 2024-04-25T00:08:36.087+0000 7f4cdc575700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:08:36.627 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:36 smithi107 ceph-mon[29153]: from='osd.0 [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-25T00:08:36.627 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:36 smithi107 ceph-mon[29153]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T00:08:36.627 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:36 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:08:36.627 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:36 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:08:36.627 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:36 smithi107 ceph-mon[29153]: from='osd.0 [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636]' entity='osd.0' 2024-04-25T00:08:36.672 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:36 smithi170 ceph-mon[32807]: from='osd.0 [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-25T00:08:36.672 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:36 smithi170 ceph-mon[32807]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T00:08:36.672 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:36 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:08:36.672 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:36 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:08:36.672 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:36 smithi170 ceph-mon[32807]: from='osd.0 [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636]' entity='osd.0' 2024-04-25T00:08:36.700 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi170:vg_nvme/lv_4 2024-04-25T00:08:36.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:36 smithi195 ceph-mon[32833]: from='osd.0 [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-25T00:08:36.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:36 smithi195 ceph-mon[32833]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T00:08:36.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:36 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:08:36.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:36 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:08:36.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:36 smithi195 ceph-mon[32833]: from='osd.0 [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636]' entity='osd.0' 2024-04-25T00:08:36.946 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:08:37.662 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:37 smithi170 ceph-mon[32807]: purged_snaps scrub starts 2024-04-25T00:08:37.662 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:37 smithi170 ceph-mon[32807]: purged_snaps scrub ok 2024-04-25T00:08:37.662 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:37 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:08:37.662 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:37 smithi170 ceph-mon[32807]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:37.662 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:37 smithi170 ceph-mon[32807]: Detected new or changed devices on smithi107 2024-04-25T00:08:37.662 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:37 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:37.662 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:37 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:37.663 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:37 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:08:37.663 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:37 smithi170 ceph-mon[32807]: Adjusting osd_memory_target on smithi107 to 17088M 2024-04-25T00:08:37.663 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:37 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:37.663 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:37 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:37.663 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:37 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:37.663 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:37 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:37.663 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:37 smithi170 ceph-mon[32807]: osd.0 [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636] boot 2024-04-25T00:08:37.663 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:37 smithi170 ceph-mon[32807]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T00:08:37.663 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:37 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:08:37.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:37 smithi107 ceph-mon[29153]: purged_snaps scrub starts 2024-04-25T00:08:37.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:37 smithi107 ceph-mon[29153]: purged_snaps scrub ok 2024-04-25T00:08:37.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:37 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:08:37.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:37 smithi107 ceph-mon[29153]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:37.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:37 smithi107 ceph-mon[29153]: Detected new or changed devices on smithi107 2024-04-25T00:08:37.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:37 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:37.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:37 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:37.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:37 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:08:37.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:37 smithi107 ceph-mon[29153]: Adjusting osd_memory_target on smithi107 to 17088M 2024-04-25T00:08:37.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:37 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:37.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:37 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:37.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:37 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:37.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:37 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:37.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:37 smithi107 ceph-mon[29153]: osd.0 [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636] boot 2024-04-25T00:08:37.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:37 smithi107 ceph-mon[29153]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T00:08:37.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:37 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:08:37.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:37 smithi195 ceph-mon[32833]: purged_snaps scrub starts 2024-04-25T00:08:37.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:37 smithi195 ceph-mon[32833]: purged_snaps scrub ok 2024-04-25T00:08:37.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:37 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:08:37.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:37 smithi195 ceph-mon[32833]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:37.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:37 smithi195 ceph-mon[32833]: Detected new or changed devices on smithi107 2024-04-25T00:08:37.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:37 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:37.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:37 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:37.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:37 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:08:37.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:37 smithi195 ceph-mon[32833]: Adjusting osd_memory_target on smithi107 to 17088M 2024-04-25T00:08:37.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:37 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:37.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:37 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:37.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:37 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:37.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:37 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:37.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:37 smithi195 ceph-mon[32833]: osd.0 [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636] boot 2024-04-25T00:08:37.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:37 smithi195 ceph-mon[32833]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T00:08:37.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:37 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:08:38.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.401+0000 7f35872a8700 1 -- 172.21.15.170:0/1432233120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3580103490 msgr2=0x7f35801038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:38.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.401+0000 7f35872a8700 1 --2- 172.21.15.170:0/1432233120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3580103490 0x7f35801038f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f35700099c0 tx=0x7f3570009cd0 comp rx=0 tx=0).stop 2024-04-25T00:08:38.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.401+0000 7f35872a8700 1 -- 172.21.15.170:0/1432233120 shutdown_connections 2024-04-25T00:08:38.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.401+0000 7f35872a8700 1 --2- 172.21.15.170:0/1432233120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3580103e30 0x7f358010a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:38.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.401+0000 7f35872a8700 1 --2- 172.21.15.170:0/1432233120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3580102290 0x7f35801026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:38.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.401+0000 7f35872a8700 1 --2- 172.21.15.170:0/1432233120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3580103490 0x7f35801038f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:38.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.401+0000 7f35872a8700 1 -- 172.21.15.170:0/1432233120 >> 172.21.15.170:0/1432233120 conn(0x7f35800fd830 msgr2=0x7f35800ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:38.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.401+0000 7f35872a8700 1 -- 172.21.15.170:0/1432233120 shutdown_connections 2024-04-25T00:08:38.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.401+0000 7f35872a8700 1 -- 172.21.15.170:0/1432233120 wait complete. 2024-04-25T00:08:38.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.402+0000 7f35872a8700 1 Processor -- start 2024-04-25T00:08:38.404 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.402+0000 7f35872a8700 1 -- start start 2024-04-25T00:08:38.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.402+0000 7f35872a8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3580102290 0x7f3580075fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:38.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.402+0000 7f35872a8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3580103490 0x7f3580074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:38.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.402+0000 7f35872a8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3580103e30 0x7f3580074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:38.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.402+0000 7f35872a8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f358010de50 con 0x7f3580102290 2024-04-25T00:08:38.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.402+0000 7f35872a8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f358010dcd0 con 0x7f3580103490 2024-04-25T00:08:38.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.402+0000 7f35872a8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f358010dfd0 con 0x7f3580103e30 2024-04-25T00:08:38.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.402+0000 7f3584843700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3580103490 0x7f3580074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:38.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.402+0000 7f3584843700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3580103490 0x7f3580074620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:53806/0 (socket says 172.21.15.170:53806) 2024-04-25T00:08:38.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.402+0000 7f3584843700 1 -- 172.21.15.170:0/3783689795 learned_addr learned my addr 172.21.15.170:0/3783689795 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-04-25T00:08:38.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.402+0000 7f3585845700 1 --2- 172.21.15.170:0/3783689795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3580103e30 0x7f3580074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:38.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.402+0000 7f3585044700 1 --2- 172.21.15.170:0/3783689795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3580102290 0x7f3580075fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:38.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.402+0000 7f3584843700 1 -- 172.21.15.170:0/3783689795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3580103e30 msgr2=0x7f3580074b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:38.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.402+0000 7f3584843700 1 --2- 172.21.15.170:0/3783689795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3580103e30 0x7f3580074b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:38.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.402+0000 7f3584843700 1 -- 172.21.15.170:0/3783689795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3580102290 msgr2=0x7f3580075fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:38.405 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.402+0000 7f3584843700 1 --2- 172.21.15.170:0/3783689795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3580102290 0x7f3580075fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:38.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.402+0000 7f3584843700 1 -- 172.21.15.170:0/3783689795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3580075410 con 0x7f3580103490 2024-04-25T00:08:38.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.403+0000 7f3584843700 1 --2- 172.21.15.170:0/3783689795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3580103490 0x7f3580074620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3570005910 tx=0x7f3570009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:38.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.403+0000 7f35767fc700 1 -- 172.21.15.170:0/3783689795 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570027070 con 0x7f3580103490 2024-04-25T00:08:38.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.403+0000 7f35767fc700 1 -- 172.21.15.170:0/3783689795 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3570004150 con 0x7f3580103490 2024-04-25T00:08:38.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.403+0000 7f35872a8700 1 -- 172.21.15.170:0/3783689795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35801a25c0 con 0x7f3580103490 2024-04-25T00:08:38.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.403+0000 7f35767fc700 1 -- 172.21.15.170:0/3783689795 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3570022e60 con 0x7f3580103490 2024-04-25T00:08:38.406 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.403+0000 7f35872a8700 1 -- 172.21.15.170:0/3783689795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35801a2a10 con 0x7f3580103490 2024-04-25T00:08:38.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.404+0000 7f35872a8700 1 -- 172.21.15.170:0/3783689795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3580068f20 con 0x7f3580103490 2024-04-25T00:08:38.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.405+0000 7f35767fc700 1 -- 172.21.15.170:0/3783689795 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f357002a070 con 0x7f3580103490 2024-04-25T00:08:38.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.406+0000 7f35767fc700 1 --2- 172.21.15.170:0/3783689795 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f356c0a2b90 0x7f356c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:38.407 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.406+0000 7f35767fc700 1 -- 172.21.15.170:0/3783689795 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f35700ce5b0 con 0x7f3580103490 2024-04-25T00:08:38.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.406+0000 7f3585044700 1 --2- 172.21.15.170:0/3783689795 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f356c0a2b90 0x7f356c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:38.408 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.407+0000 7f3585044700 1 --2- 172.21.15.170:0/3783689795 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f356c0a2b90 0x7f356c0a5050 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f357c005fd0 tx=0x7f357c005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:38.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.410+0000 7f35767fc700 1 -- 172.21.15.170:0/3783689795 <== mon.2 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3570013070 con 0x7f3580103490 2024-04-25T00:08:38.615 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:38.615+0000 7f35872a8700 1 -- 172.21.15.170:0/3783689795 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3580062730 con 0x7f356c0a2b90 2024-04-25T00:08:39.347 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:39 smithi170 ceph-mon[32807]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T00:08:39.347 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:39 smithi170 ceph-mon[32807]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:08:39.347 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:39 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:08:39.347 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:39 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:08:39.347 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:39 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:39.347 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:39 smithi170 ceph-mon[32807]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:39.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:39 smithi107 ceph-mon[29153]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T00:08:39.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:39 smithi107 ceph-mon[29153]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:08:39.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:39 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:08:39.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:39 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:08:39.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:39 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:39.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:39 smithi107 ceph-mon[29153]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:39.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:39 smithi195 ceph-mon[32833]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T00:08:39.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:39 smithi195 ceph-mon[32833]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:08:39.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:39 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:08:39.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:39 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:08:39.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:39 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:39.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:39 smithi195 ceph-mon[32833]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:42.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:41 smithi170 ceph-mon[32807]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:42.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:41 smithi170 ceph-mon[32807]: from='client.? 172.21.15.170:0/1475669233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d009a8aa-9bf8-44c8-b873-ca2ab7c0587c"}]: dispatch 2024-04-25T00:08:42.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:41 smithi170 ceph-mon[32807]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d009a8aa-9bf8-44c8-b873-ca2ab7c0587c"}]: dispatch 2024-04-25T00:08:42.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:41 smithi170 ceph-mon[32807]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d009a8aa-9bf8-44c8-b873-ca2ab7c0587c"}]': finished 2024-04-25T00:08:42.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:41 smithi170 ceph-mon[32807]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T00:08:42.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:41 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:08:42.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:41 smithi170 ceph-mon[32807]: from='client.? 172.21.15.170:0/1754720521' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:08:42.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:41 smithi107 ceph-mon[29153]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:42.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:41 smithi107 ceph-mon[29153]: from='client.? 172.21.15.170:0/1475669233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d009a8aa-9bf8-44c8-b873-ca2ab7c0587c"}]: dispatch 2024-04-25T00:08:42.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:41 smithi107 ceph-mon[29153]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d009a8aa-9bf8-44c8-b873-ca2ab7c0587c"}]: dispatch 2024-04-25T00:08:42.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:41 smithi107 ceph-mon[29153]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d009a8aa-9bf8-44c8-b873-ca2ab7c0587c"}]': finished 2024-04-25T00:08:42.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:41 smithi107 ceph-mon[29153]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T00:08:42.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:41 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:08:42.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:41 smithi107 ceph-mon[29153]: from='client.? 172.21.15.170:0/1754720521' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:08:42.293 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:41 smithi195 ceph-mon[32833]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:42.293 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:41 smithi195 ceph-mon[32833]: from='client.? 172.21.15.170:0/1475669233' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d009a8aa-9bf8-44c8-b873-ca2ab7c0587c"}]: dispatch 2024-04-25T00:08:42.293 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:41 smithi195 ceph-mon[32833]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d009a8aa-9bf8-44c8-b873-ca2ab7c0587c"}]: dispatch 2024-04-25T00:08:42.293 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:41 smithi195 ceph-mon[32833]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d009a8aa-9bf8-44c8-b873-ca2ab7c0587c"}]': finished 2024-04-25T00:08:42.293 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:41 smithi195 ceph-mon[32833]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T00:08:42.293 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:41 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:08:42.293 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:41 smithi195 ceph-mon[32833]: from='client.? 172.21.15.170:0/1754720521' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:08:44.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:43 smithi170 ceph-mon[32807]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:44.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:43 smithi107 ceph-mon[29153]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:44.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:43 smithi195 ceph-mon[32833]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:46.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:45 smithi170 ceph-mon[32807]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:46.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:45 smithi107 ceph-mon[29153]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:46.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:45 smithi195 ceph-mon[32833]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:48.085 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:47 smithi170 ceph-mon[32807]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:48.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:47 smithi107 ceph-mon[29153]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:48.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:47 smithi195 ceph-mon[32833]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:48.995 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:48 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T00:08:48.996 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:48 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:49.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:48 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T00:08:49.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:48 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:49.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:48 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T00:08:49.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:48 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:50.176 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:49 smithi170 ceph-mon[32807]: Deploying daemon osd.1 on smithi170 2024-04-25T00:08:50.177 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:49 smithi170 ceph-mon[32807]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:50.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:49 smithi107 ceph-mon[29153]: Deploying daemon osd.1 on smithi170 2024-04-25T00:08:50.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:49 smithi107 ceph-mon[29153]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:50.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:49 smithi195 ceph-mon[32833]: Deploying daemon osd.1 on smithi170 2024-04-25T00:08:50.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:49 smithi195 ceph-mon[32833]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:52.097 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:51 smithi170 ceph-mon[32807]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:52.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:51 smithi107 ceph-mon[29153]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:52.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:51 smithi195 ceph-mon[32833]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:54.028 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:53 smithi170 ceph-mon[32807]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:54.028 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:53 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:54.028 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:53 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:54.028 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:53 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:54.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:53 smithi107 ceph-mon[29153]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:54.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:53 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:54.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:53 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:54.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:53 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:54.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:53 smithi195 ceph-mon[32833]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:54.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:53 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:54.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:53 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:54.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:53 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:56.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:55 smithi170 ceph-mon[32807]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:56.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:55 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:56.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:55 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:56.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:55 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:56.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:55 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:56.126 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:55 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:56.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:55 smithi107 ceph-mon[29153]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:56.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:55 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:56.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:55 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:56.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:55 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:56.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:55 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:56.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:55 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:56.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:55 smithi195 ceph-mon[32833]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:56.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:55 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:56.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:55 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:56.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:55 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:56.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:55 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:56.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:55 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:56.779 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:56.779+0000 7f35767fc700 1 -- 172.21.15.170:0/3783689795 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3580062730 con 0x7f356c0a2b90 2024-04-25T00:08:56.779 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 1 on host 'smithi170' 2024-04-25T00:08:56.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:56.781+0000 7f35872a8700 1 -- 172.21.15.170:0/3783689795 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f356c0a2b90 msgr2=0x7f356c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:56.781+0000 7f35872a8700 1 --2- 172.21.15.170:0/3783689795 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f356c0a2b90 0x7f356c0a5050 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f357c005fd0 tx=0x7f357c005dd0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:56.781+0000 7f35872a8700 1 -- 172.21.15.170:0/3783689795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3580103490 msgr2=0x7f3580074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:56.781+0000 7f35872a8700 1 --2- 172.21.15.170:0/3783689795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3580103490 0x7f3580074620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3570005910 tx=0x7f3570009580 comp rx=0 tx=0).stop 2024-04-25T00:08:56.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:56.781+0000 7f35872a8700 1 -- 172.21.15.170:0/3783689795 shutdown_connections 2024-04-25T00:08:56.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:56.781+0000 7f35872a8700 1 --2- 172.21.15.170:0/3783689795 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f356c0a2b90 0x7f356c0a5050 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:56.781+0000 7f35872a8700 1 --2- 172.21.15.170:0/3783689795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3580103e30 0x7f3580074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:56.781+0000 7f35872a8700 1 --2- 172.21.15.170:0/3783689795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3580102290 0x7f3580075fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:56.781+0000 7f35872a8700 1 --2- 172.21.15.170:0/3783689795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3580103490 0x7f3580074620 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.781 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:56.781+0000 7f35872a8700 1 -- 172.21.15.170:0/3783689795 >> 172.21.15.170:0/3783689795 conn(0x7f35800fd830 msgr2=0x7f3580106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:56.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:56.781+0000 7f35872a8700 1 -- 172.21.15.170:0/3783689795 shutdown_connections 2024-04-25T00:08:56.782 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:08:56.781+0000 7f35872a8700 1 -- 172.21.15.170:0/3783689795 wait complete. 2024-04-25T00:08:57.412 DEBUG:teuthology.orchestra.run.smithi170:osd.1> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T00:08:57.414 INFO:tasks.cephadm:Deploying osd.2 on smithi195 with /dev/vg_nvme/lv_4... 2024-04-25T00:08:57.414 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T00:08:57.657 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:08:58.015 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:57 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:58.015 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:57 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:58.015 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:57 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:58.015 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:57 smithi195 ceph-mon[32833]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:58.016 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:57 smithi195 ceph-mon[32833]: from='osd.1 [v2:172.21.15.170:6800/453760841,v1:172.21.15.170:6801/453760841]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T00:08:58.016 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:57 smithi195 ceph-mon[32833]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T00:08:58.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:57 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:58.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:57 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:58.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:57 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:58.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:57 smithi170 ceph-mon[32807]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:58.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:57 smithi170 ceph-mon[32807]: from='osd.1 [v2:172.21.15.170:6800/453760841,v1:172.21.15.170:6801/453760841]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T00:08:58.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:57 smithi170 ceph-mon[32807]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T00:08:58.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:57 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:58.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:57 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:58.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:57 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:08:58.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:57 smithi107 ceph-mon[29153]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:08:58.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:57 smithi107 ceph-mon[29153]: from='osd.1 [v2:172.21.15.170:6800/453760841,v1:172.21.15.170:6801/453760841]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T00:08:58.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:57 smithi107 ceph-mon[29153]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T00:08:58.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:58 smithi170 ceph-mon[32807]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T00:08:58.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:58 smithi170 ceph-mon[32807]: from='osd.1 [v2:172.21.15.170:6800/453760841,v1:172.21.15.170:6801/453760841]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-25T00:08:58.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:58 smithi170 ceph-mon[32807]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T00:08:58.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:58 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:08:58.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:58 smithi170 ceph-mon[32807]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-25T00:08:58.935 INFO:journalctl@ceph.osd.1.smithi170.stdout:Apr 25 00:08:58 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-1[37418]: 2024-04-25T00:08:58.779+0000 7f09e6a1d700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T00:08:58.935 INFO:journalctl@ceph.osd.1.smithi170.stdout:Apr 25 00:08:58 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-1[37418]: 2024-04-25T00:08:58.783+0000 7f09e2810700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:08:59.076 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:58 smithi195 ceph-mon[32833]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T00:08:59.076 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:58 smithi195 ceph-mon[32833]: from='osd.1 [v2:172.21.15.170:6800/453760841,v1:172.21.15.170:6801/453760841]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-25T00:08:59.076 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:58 smithi195 ceph-mon[32833]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T00:08:59.076 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:58 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:08:59.076 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:58 smithi195 ceph-mon[32833]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-25T00:08:59.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:58 smithi107 ceph-mon[29153]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T00:08:59.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:58 smithi107 ceph-mon[29153]: from='osd.1 [v2:172.21.15.170:6800/453760841,v1:172.21.15.170:6801/453760841]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-25T00:08:59.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:58 smithi107 ceph-mon[29153]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T00:08:59.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:58 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:08:59.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:58 smithi107 ceph-mon[29153]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-25T00:08:59.513 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T00:08:59.513 INFO:teuthology.orchestra.run.smithi195.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T00:08:59.513 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10+0 records in 2024-04-25T00:08:59.513 INFO:teuthology.orchestra.run.smithi195.stderr:10+0 records out 2024-04-25T00:08:59.513 INFO:teuthology.orchestra.run.smithi195.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00850093 s, 1.2 GB/s 2024-04-25T00:08:59.513 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping successful for: 2024-04-25T00:09:00.063 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:59 smithi195 ceph-mon[32833]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-04-25T00:09:00.063 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:59 smithi195 ceph-mon[32833]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T00:09:00.063 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:59 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:09:00.063 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:59 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:09:00.063 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:08:59 smithi195 ceph-mon[32833]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:09:00.092 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi195:vg_nvme/lv_4 2024-04-25T00:09:00.099 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:59 smithi170 ceph-mon[32807]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-04-25T00:09:00.100 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:59 smithi170 ceph-mon[32807]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T00:09:00.100 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:59 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:09:00.100 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:59 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:09:00.100 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:08:59 smithi170 ceph-mon[32807]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:09:00.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:59 smithi107 ceph-mon[29153]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-04-25T00:09:00.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:59 smithi107 ceph-mon[29153]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T00:09:00.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:59 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:09:00.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:59 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:09:00.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:08:59 smithi107 ceph-mon[29153]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:09:00.344 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:09:01.027 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:00 smithi195 ceph-mon[32833]: purged_snaps scrub starts 2024-04-25T00:09:01.027 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:00 smithi195 ceph-mon[32833]: purged_snaps scrub ok 2024-04-25T00:09:01.027 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:00 smithi195 ceph-mon[32833]: osd.1 [v2:172.21.15.170:6800/453760841,v1:172.21.15.170:6801/453760841] boot 2024-04-25T00:09:01.027 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:00 smithi195 ceph-mon[32833]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T00:09:01.028 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:00 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:09:01.028 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:00 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:01.028 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:00 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:01.028 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:00 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:01.028 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:00 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:01.028 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:00 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:01.028 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:00 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:01.028 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:00 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:01.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:00 smithi170 ceph-mon[32807]: purged_snaps scrub starts 2024-04-25T00:09:01.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:00 smithi170 ceph-mon[32807]: purged_snaps scrub ok 2024-04-25T00:09:01.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:00 smithi170 ceph-mon[32807]: osd.1 [v2:172.21.15.170:6800/453760841,v1:172.21.15.170:6801/453760841] boot 2024-04-25T00:09:01.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:00 smithi170 ceph-mon[32807]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T00:09:01.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:00 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:09:01.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:00 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:01.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:00 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:01.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:00 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:01.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:00 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:01.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:00 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:01.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:00 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:01.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:00 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:01.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:00 smithi107 ceph-mon[29153]: purged_snaps scrub starts 2024-04-25T00:09:01.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:00 smithi107 ceph-mon[29153]: purged_snaps scrub ok 2024-04-25T00:09:01.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:00 smithi107 ceph-mon[29153]: osd.1 [v2:172.21.15.170:6800/453760841,v1:172.21.15.170:6801/453760841] boot 2024-04-25T00:09:01.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:00 smithi107 ceph-mon[29153]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T00:09:01.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:00 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:09:01.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:00 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:01.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:00 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:01.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:00 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:01.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:00 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:01.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:00 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:01.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:00 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:01.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:00 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:01.729 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.725+0000 7f211bc19700 1 -- 172.21.15.195:0/3029576239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2114101430 msgr2=0x7f21141018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:01.729 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.725+0000 7f211bc19700 1 --2- 172.21.15.195:0/3029576239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2114101430 0x7f21141018b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2104009990 tx=0x7f2104009ca0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.729 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.725+0000 7f211bc19700 1 -- 172.21.15.195:0/3029576239 shutdown_connections 2024-04-25T00:09:01.729 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.725+0000 7f211bc19700 1 --2- 172.21.15.195:0/3029576239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2114101df0 0x7f2114109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.730 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.725+0000 7f211bc19700 1 --2- 172.21.15.195:0/3029576239 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21141027d0 0x7f2114102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.730 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.725+0000 7f211bc19700 1 --2- 172.21.15.195:0/3029576239 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2114101430 0x7f21141018b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.730 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.725+0000 7f211bc19700 1 -- 172.21.15.195:0/3029576239 >> 172.21.15.195:0/3029576239 conn(0x7f21140fbee0 msgr2=0x7f21140fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:01.730 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.726+0000 7f211bc19700 1 -- 172.21.15.195:0/3029576239 shutdown_connections 2024-04-25T00:09:01.730 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.726+0000 7f211bc19700 1 -- 172.21.15.195:0/3029576239 wait complete. 2024-04-25T00:09:01.730 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.726+0000 7f211bc19700 1 Processor -- start 2024-04-25T00:09:01.730 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.726+0000 7f211bc19700 1 -- start start 2024-04-25T00:09:01.730 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.726+0000 7f211bc19700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2114101430 0x7f2114100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:01.730 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.726+0000 7f211bc19700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2114101df0 0x7f211419d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:01.730 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.726+0000 7f211bc19700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21141027d0 0x7f211419f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:01.730 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.726+0000 7f211bc19700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f211410cdf0 con 0x7f2114101430 2024-04-25T00:09:01.730 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.726+0000 7f211bc19700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f211410cc70 con 0x7f21141027d0 2024-04-25T00:09:01.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.726+0000 7f211bc19700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f211410cf70 con 0x7f2114101df0 2024-04-25T00:09:01.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.726+0000 7f21191b4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2114101df0 0x7f211419d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:01.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.726+0000 7f21191b4700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2114101df0 0x7f211419d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:46990/0 (socket says 172.21.15.195:46990) 2024-04-25T00:09:01.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.726+0000 7f21191b4700 1 -- 172.21.15.195:0/2263248801 learned_addr learned my addr 172.21.15.195:0/2263248801 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-04-25T00:09:01.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.727+0000 7f21199b5700 1 --2- 172.21.15.195:0/2263248801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2114101430 0x7f2114100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:01.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.727+0000 7f21191b4700 1 -- 172.21.15.195:0/2263248801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21141027d0 msgr2=0x7f211419f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:01.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.727+0000 7f21191b4700 1 --2- 172.21.15.195:0/2263248801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21141027d0 0x7f211419f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.727+0000 7f21191b4700 1 -- 172.21.15.195:0/2263248801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2114101430 msgr2=0x7f2114100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:01.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.727+0000 7f21191b4700 1 --2- 172.21.15.195:0/2263248801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2114101430 0x7f2114100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:01.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.727+0000 7f21191b4700 1 -- 172.21.15.195:0/2263248801 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21141a0120 con 0x7f2114101df0 2024-04-25T00:09:01.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.727+0000 7f21191b4700 1 --2- 172.21.15.195:0/2263248801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2114101df0 0x7f211419d210 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2104005910 tx=0x7f2104009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:01.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.727+0000 7f210affd700 1 -- 172.21.15.195:0/2263248801 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2104027070 con 0x7f2114101df0 2024-04-25T00:09:01.731 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.727+0000 7f211bc19700 1 -- 172.21.15.195:0/2263248801 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21141a03b0 con 0x7f2114101df0 2024-04-25T00:09:01.732 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.727+0000 7f210affd700 1 -- 172.21.15.195:0/2263248801 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2104004210 con 0x7f2114101df0 2024-04-25T00:09:01.732 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.727+0000 7f210affd700 1 -- 172.21.15.195:0/2263248801 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210400f460 con 0x7f2114101df0 2024-04-25T00:09:01.732 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.727+0000 7f211bc19700 1 -- 172.21.15.195:0/2263248801 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21141a08e0 con 0x7f2114101df0 2024-04-25T00:09:01.732 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.728+0000 7f211bc19700 1 -- 172.21.15.195:0/2263248801 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2114068f20 con 0x7f2114101df0 2024-04-25T00:09:01.732 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.729+0000 7f210affd700 1 -- 172.21.15.195:0/2263248801 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7f210402a070 con 0x7f2114101df0 2024-04-25T00:09:01.733 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.729+0000 7f210affd700 1 --2- 172.21.15.195:0/2263248801 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f21000a2b90 0x7f21000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:01.733 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.730+0000 7f210affd700 1 -- 172.21.15.195:0/2263248801 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f21040ceea0 con 0x7f2114101df0 2024-04-25T00:09:01.733 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.730+0000 7f21199b5700 1 --2- 172.21.15.195:0/2263248801 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f21000a2b90 0x7f21000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:01.733 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.730+0000 7f21199b5700 1 --2- 172.21.15.195:0/2263248801 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f21000a2b90 0x7f21000a5050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2110005950 tx=0x7f21100058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:01.733 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.733+0000 7f210affd700 1 -- 172.21.15.195:0/2263248801 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2104031070 con 0x7f2114101df0 2024-04-25T00:09:01.937 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:01.937+0000 7f211bc19700 1 -- 172.21.15.195:0/2263248801 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2114062670 con 0x7f21000a2b90 2024-04-25T00:09:01.937 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:01 smithi195 ceph-mon[32833]: Detected new or changed devices on smithi170 2024-04-25T00:09:01.938 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:01 smithi195 ceph-mon[32833]: Adjusting osd_memory_target on smithi170 to 17088M 2024-04-25T00:09:01.938 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:01 smithi195 ceph-mon[32833]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:01.938 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:01 smithi195 ceph-mon[32833]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T00:09:02.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:01 smithi170 ceph-mon[32807]: Detected new or changed devices on smithi170 2024-04-25T00:09:02.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:01 smithi170 ceph-mon[32807]: Adjusting osd_memory_target on smithi170 to 17088M 2024-04-25T00:09:02.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:01 smithi170 ceph-mon[32807]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:02.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:01 smithi170 ceph-mon[32807]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T00:09:02.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:01 smithi107 ceph-mon[29153]: Detected new or changed devices on smithi170 2024-04-25T00:09:02.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:01 smithi107 ceph-mon[29153]: Adjusting osd_memory_target on smithi170 to 17088M 2024-04-25T00:09:02.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:01 smithi107 ceph-mon[29153]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:02.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:01 smithi107 ceph-mon[29153]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T00:09:03.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:02 smithi195 ceph-mon[32833]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:09:03.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:09:03.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:09:03.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:03.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:02 smithi170 ceph-mon[32807]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:09:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:09:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:09:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:03.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:02 smithi107 ceph-mon[29153]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:09:03.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:09:03.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:09:03.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:04.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:03 smithi195 ceph-mon[32833]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:04.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:03 smithi170 ceph-mon[32807]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:04.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:03 smithi107 ceph-mon[29153]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:05.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:04 smithi195 ceph-mon[32833]: from='client.? 172.21.15.195:0/1219434352' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4df1456c-7657-4efd-8295-bf8248fcbcaf"}]: dispatch 2024-04-25T00:09:05.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:04 smithi195 ceph-mon[32833]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4df1456c-7657-4efd-8295-bf8248fcbcaf"}]: dispatch 2024-04-25T00:09:05.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:04 smithi195 ceph-mon[32833]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4df1456c-7657-4efd-8295-bf8248fcbcaf"}]': finished 2024-04-25T00:09:05.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:04 smithi195 ceph-mon[32833]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T00:09:05.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:04 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:09:05.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:04 smithi170 ceph-mon[32807]: from='client.? 172.21.15.195:0/1219434352' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4df1456c-7657-4efd-8295-bf8248fcbcaf"}]: dispatch 2024-04-25T00:09:05.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:04 smithi170 ceph-mon[32807]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4df1456c-7657-4efd-8295-bf8248fcbcaf"}]: dispatch 2024-04-25T00:09:05.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:04 smithi170 ceph-mon[32807]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4df1456c-7657-4efd-8295-bf8248fcbcaf"}]': finished 2024-04-25T00:09:05.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:04 smithi170 ceph-mon[32807]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T00:09:05.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:04 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:09:05.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:04 smithi107 ceph-mon[29153]: from='client.? 172.21.15.195:0/1219434352' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4df1456c-7657-4efd-8295-bf8248fcbcaf"}]: dispatch 2024-04-25T00:09:05.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:04 smithi107 ceph-mon[29153]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4df1456c-7657-4efd-8295-bf8248fcbcaf"}]: dispatch 2024-04-25T00:09:05.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:04 smithi107 ceph-mon[29153]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4df1456c-7657-4efd-8295-bf8248fcbcaf"}]': finished 2024-04-25T00:09:05.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:04 smithi107 ceph-mon[29153]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T00:09:05.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:04 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:09:06.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:05 smithi195 ceph-mon[32833]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:06.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:05 smithi195 ceph-mon[32833]: from='client.? 172.21.15.195:0/882267995' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:09:06.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:05 smithi170 ceph-mon[32807]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:06.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:05 smithi170 ceph-mon[32807]: from='client.? 172.21.15.195:0/882267995' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:09:06.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:05 smithi107 ceph-mon[29153]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:06.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:05 smithi107 ceph-mon[29153]: from='client.? 172.21.15.195:0/882267995' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:09:08.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:07 smithi170 ceph-mon[32807]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:08.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:07 smithi107 ceph-mon[29153]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:08.269 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:07 smithi195 ceph-mon[32833]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:10.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:09 smithi195 ceph-mon[32833]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:10.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:09 smithi170 ceph-mon[32807]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:10.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:09 smithi107 ceph-mon[29153]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:12.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:11 smithi170 ceph-mon[32807]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:12.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:11 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T00:09:12.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:11 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:12.185 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:11 smithi195 ceph-mon[32833]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:12.185 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:11 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T00:09:12.185 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:11 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:12.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:11 smithi107 ceph-mon[29153]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:12.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:11 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T00:09:12.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:11 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:13.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:12 smithi195 ceph-mon[32833]: Deploying daemon osd.2 on smithi195 2024-04-25T00:09:13.179 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:12 smithi170 ceph-mon[32807]: Deploying daemon osd.2 on smithi195 2024-04-25T00:09:13.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:12 smithi107 ceph-mon[29153]: Deploying daemon osd.2 on smithi195 2024-04-25T00:09:14.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:13 smithi170 ceph-mon[32807]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:14.193 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:13 smithi195 ceph-mon[32833]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:14.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:13 smithi107 ceph-mon[29153]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:16.019 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:15 smithi195 ceph-mon[32833]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:16.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:15 smithi170 ceph-mon[32807]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:16.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:15 smithi107 ceph-mon[29153]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:17.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:17 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:17.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:17 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:17.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:17 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:17.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:17 smithi195 ceph-mon[32833]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:17.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:17 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:17.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:17 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:17.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:17 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:17.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:17 smithi170 ceph-mon[32807]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:17.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:17 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:17.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:17 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:17.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:17 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:17.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:17 smithi107 ceph-mon[29153]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:19.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:19 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:19.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:19 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:19.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:19 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:19.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:19 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:19.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:19 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:19.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:19 smithi107 ceph-mon[29153]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:19.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:19 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:19.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:19 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:19.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:19 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:19.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:19 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:19.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:19 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:19.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:19 smithi195 ceph-mon[32833]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:19.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:19 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:19.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:19 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:19.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:19 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:19.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:19 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:19.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:19 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:19.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:19 smithi170 ceph-mon[32807]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:20.199 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:20.198+0000 7f210affd700 1 -- 172.21.15.195:0/2263248801 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2114062670 con 0x7f21000a2b90 2024-04-25T00:09:20.199 INFO:teuthology.orchestra.run.smithi195.stdout:Created osd(s) 2 on host 'smithi195' 2024-04-25T00:09:20.201 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:20.200+0000 7f211bc19700 1 -- 172.21.15.195:0/2263248801 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f21000a2b90 msgr2=0x7f21000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.201 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:20.200+0000 7f211bc19700 1 --2- 172.21.15.195:0/2263248801 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f21000a2b90 0x7f21000a5050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2110005950 tx=0x7f21100058e0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.201 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:20.200+0000 7f211bc19700 1 -- 172.21.15.195:0/2263248801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2114101df0 msgr2=0x7f211419d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:20.201 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:20.200+0000 7f211bc19700 1 --2- 172.21.15.195:0/2263248801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2114101df0 0x7f211419d210 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2104005910 tx=0x7f2104009580 comp rx=0 tx=0).stop 2024-04-25T00:09:20.201 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:20.201+0000 7f211bc19700 1 -- 172.21.15.195:0/2263248801 shutdown_connections 2024-04-25T00:09:20.201 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:20.201+0000 7f211bc19700 1 --2- 172.21.15.195:0/2263248801 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f21000a2b90 0x7f21000a5050 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.201 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:20.201+0000 7f211bc19700 1 --2- 172.21.15.195:0/2263248801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21141027d0 0x7f211419f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.202 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:20.201+0000 7f211bc19700 1 --2- 172.21.15.195:0/2263248801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2114101430 0x7f2114100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.202 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:20.201+0000 7f211bc19700 1 --2- 172.21.15.195:0/2263248801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2114101df0 0x7f211419d210 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:20.202 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:20.201+0000 7f211bc19700 1 -- 172.21.15.195:0/2263248801 >> 172.21.15.195:0/2263248801 conn(0x7f21140fbee0 msgr2=0x7f21141056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:20.202 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:20.201+0000 7f211bc19700 1 -- 172.21.15.195:0/2263248801 shutdown_connections 2024-04-25T00:09:20.202 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:09:20.201+0000 7f211bc19700 1 -- 172.21.15.195:0/2263248801 wait complete. 2024-04-25T00:09:20.857 DEBUG:teuthology.orchestra.run.smithi195:osd.2> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T00:09:20.859 INFO:tasks.cephadm:Deploying osd.3 on smithi107 with /dev/vg_nvme/lv_3... 2024-04-25T00:09:20.860 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T00:09:21.105 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:09:21.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:21 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:21.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:21 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:21.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:21 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:21.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:21 smithi170 ceph-mon[32807]: from='osd.2 [v2:172.21.15.195:6800/90978480,v1:172.21.15.195:6801/90978480]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T00:09:21.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:21 smithi170 ceph-mon[32807]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T00:09:21.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:21 smithi170 ceph-mon[32807]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:21.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:21 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:21.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:21 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:21.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:21 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:21.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:21 smithi107 ceph-mon[29153]: from='osd.2 [v2:172.21.15.195:6800/90978480,v1:172.21.15.195:6801/90978480]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T00:09:21.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:21 smithi107 ceph-mon[29153]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T00:09:21.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:21 smithi107 ceph-mon[29153]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:21.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:21 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:21.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:21 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:21.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:21 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:21.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:21 smithi195 ceph-mon[32833]: from='osd.2 [v2:172.21.15.195:6800/90978480,v1:172.21.15.195:6801/90978480]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T00:09:21.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:21 smithi195 ceph-mon[32833]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T00:09:21.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:21 smithi195 ceph-mon[32833]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:22.269 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:22 smithi195 ceph-mon[32833]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T00:09:22.269 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:22 smithi195 ceph-mon[32833]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T00:09:22.269 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:22 smithi195 ceph-mon[32833]: from='osd.2 [v2:172.21.15.195:6800/90978480,v1:172.21.15.195:6801/90978480]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-04-25T00:09:22.269 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:22 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:09:22.269 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:22 smithi195 ceph-mon[32833]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-04-25T00:09:22.269 INFO:journalctl@ceph.osd.2.smithi195.stdout:Apr 25 00:09:22 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-2[38010]: 2024-04-25T00:09:22.193+0000 7f462a19a700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T00:09:22.269 INFO:journalctl@ceph.osd.2.smithi195.stdout:Apr 25 00:09:22 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-2[38010]: 2024-04-25T00:09:22.197+0000 7f4625f8d700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:09:22.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:22 smithi107 ceph-mon[29153]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T00:09:22.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:22 smithi107 ceph-mon[29153]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T00:09:22.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:22 smithi107 ceph-mon[29153]: from='osd.2 [v2:172.21.15.195:6800/90978480,v1:172.21.15.195:6801/90978480]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-04-25T00:09:22.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:22 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:09:22.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:22 smithi107 ceph-mon[29153]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-04-25T00:09:22.682 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:22 smithi170 ceph-mon[32807]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T00:09:22.686 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:22 smithi170 ceph-mon[32807]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T00:09:22.686 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:22 smithi170 ceph-mon[32807]: from='osd.2 [v2:172.21.15.195:6800/90978480,v1:172.21.15.195:6801/90978480]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-04-25T00:09:22.686 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:22 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:09:22.686 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:22 smithi170 ceph-mon[32807]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-04-25T00:09:23.092 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T00:09:23.092 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T00:09:23.092 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-04-25T00:09:23.092 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-04-25T00:09:23.093 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114228 s, 918 MB/s 2024-04-25T00:09:23.093 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-04-25T00:09:23.394 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:23 smithi107 ceph-mon[29153]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-04-25T00:09:23.394 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:23 smithi107 ceph-mon[29153]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T00:09:23.394 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:23 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:09:23.394 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:23 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:09:23.394 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:23 smithi107 ceph-mon[29153]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:23.447 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:23 smithi195 ceph-mon[32833]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-04-25T00:09:23.447 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:23 smithi195 ceph-mon[32833]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T00:09:23.447 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:23 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:09:23.447 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:23 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:09:23.447 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:23 smithi195 ceph-mon[32833]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:23.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:23 smithi170 ceph-mon[32807]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-04-25T00:09:23.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:23 smithi170 ceph-mon[32807]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T00:09:23.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:23 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:09:23.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:23 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:09:23.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:23 smithi170 ceph-mon[32807]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:09:23.710 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_3 2024-04-25T00:09:23.959 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:09:24.378 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:24 smithi107 ceph-mon[29153]: purged_snaps scrub starts 2024-04-25T00:09:24.378 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:24 smithi107 ceph-mon[29153]: purged_snaps scrub ok 2024-04-25T00:09:24.378 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:24 smithi107 ceph-mon[29153]: osd.2 [v2:172.21.15.195:6800/90978480,v1:172.21.15.195:6801/90978480] boot 2024-04-25T00:09:24.378 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:24 smithi107 ceph-mon[29153]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T00:09:24.378 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:24 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:09:24.378 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:24 smithi107 ceph-mon[29153]: Detected new or changed devices on smithi195 2024-04-25T00:09:24.378 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:24 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:24.378 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:24 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:24.379 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:24 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:24.379 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:24 smithi107 ceph-mon[29153]: Adjusting osd_memory_target on smithi195 to 17088M 2024-04-25T00:09:24.379 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:24 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:24.379 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:24 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:24.379 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:24 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:24.379 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:24 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:24.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:24 smithi195 ceph-mon[32833]: purged_snaps scrub starts 2024-04-25T00:09:24.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:24 smithi195 ceph-mon[32833]: purged_snaps scrub ok 2024-04-25T00:09:24.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:24 smithi195 ceph-mon[32833]: osd.2 [v2:172.21.15.195:6800/90978480,v1:172.21.15.195:6801/90978480] boot 2024-04-25T00:09:24.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:24 smithi195 ceph-mon[32833]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T00:09:24.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:24 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:09:24.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:24 smithi195 ceph-mon[32833]: Detected new or changed devices on smithi195 2024-04-25T00:09:24.591 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:24 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:24.591 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:24 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:24.591 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:24 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:24.591 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:24 smithi195 ceph-mon[32833]: Adjusting osd_memory_target on smithi195 to 17088M 2024-04-25T00:09:24.591 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:24 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:24.591 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:24 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:24.591 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:24 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:24.592 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:24 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:24.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:24 smithi170 ceph-mon[32807]: purged_snaps scrub starts 2024-04-25T00:09:24.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:24 smithi170 ceph-mon[32807]: purged_snaps scrub ok 2024-04-25T00:09:24.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:24 smithi170 ceph-mon[32807]: osd.2 [v2:172.21.15.195:6800/90978480,v1:172.21.15.195:6801/90978480] boot 2024-04-25T00:09:24.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:24 smithi170 ceph-mon[32807]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T00:09:24.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:24 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:09:24.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:24 smithi170 ceph-mon[32807]: Detected new or changed devices on smithi195 2024-04-25T00:09:24.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:24 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:24.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:24 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:24.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:24 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:24.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:24 smithi170 ceph-mon[32807]: Adjusting osd_memory_target on smithi195 to 17088M 2024-04-25T00:09:24.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:24 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:24.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:24 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:24.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:24 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:24.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:24 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:25.304 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:25 smithi107 ceph-mon[29153]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T00:09:25.304 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:25 smithi107 ceph-mon[29153]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:25.304 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:25 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:09:25.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.511+0000 7fd7a751d700 1 -- 172.21.15.107:0/2815509945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7a0108000 msgr2=0x7fd7a010a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:25.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.511+0000 7fd7a751d700 1 --2- 172.21.15.107:0/2815509945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7a0108000 0x7fd7a010a410 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd79800b0a0 tx=0x7fd79800b3b0 comp rx=0 tx=0).stop 2024-04-25T00:09:25.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.511+0000 7fd7a751d700 1 -- 172.21.15.107:0/2815509945 shutdown_connections 2024-04-25T00:09:25.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.511+0000 7fd7a751d700 1 --2- 172.21.15.107:0/2815509945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7a0108000 0x7fd7a010a410 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:25.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.511+0000 7fd7a751d700 1 --2- 172.21.15.107:0/2815509945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7a0102da0 0x7fd7a0105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:25.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.511+0000 7fd7a751d700 1 --2- 172.21.15.107:0/2815509945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7a01056d0 0x7fd7a0107ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:25.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.511+0000 7fd7a751d700 1 -- 172.21.15.107:0/2815509945 >> 172.21.15.107:0/2815509945 conn(0x7fd7a00fc790 msgr2=0x7fd7a00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:25.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.511+0000 7fd7a751d700 1 -- 172.21.15.107:0/2815509945 shutdown_connections 2024-04-25T00:09:25.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.511+0000 7fd7a751d700 1 -- 172.21.15.107:0/2815509945 wait complete. 2024-04-25T00:09:25.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.512+0000 7fd7a751d700 1 Processor -- start 2024-04-25T00:09:25.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.512+0000 7fd7a751d700 1 -- start start 2024-04-25T00:09:25.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.512+0000 7fd7a751d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7a0102da0 0x7fd7a0198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:25.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.512+0000 7fd7a751d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7a01056d0 0x7fd7a0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:25.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.512+0000 7fd7a751d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7a0108000 0x7fd7a019fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:25.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.512+0000 7fd7a751d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a010de90 con 0x7fd7a0102da0 2024-04-25T00:09:25.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.512+0000 7fd7a751d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a010dd10 con 0x7fd7a01056d0 2024-04-25T00:09:25.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.512+0000 7fd7a751d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a010e010 con 0x7fd7a0108000 2024-04-25T00:09:25.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.512+0000 7fd7a52b9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7a0102da0 0x7fd7a0198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:25.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.512+0000 7fd7a52b9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7a0102da0 0x7fd7a0198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52748/0 (socket says 172.21.15.107:52748) 2024-04-25T00:09:25.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.512+0000 7fd7a52b9700 1 -- 172.21.15.107:0/2341192492 learned_addr learned my addr 172.21.15.107:0/2341192492 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:09:25.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.512+0000 7fd7a4ab8700 1 --2- 172.21.15.107:0/2341192492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7a01056d0 0x7fd7a0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:25.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.512+0000 7fd7a5aba700 1 --2- 172.21.15.107:0/2341192492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7a0108000 0x7fd7a019fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:25.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.512+0000 7fd7a52b9700 1 -- 172.21.15.107:0/2341192492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7a0108000 msgr2=0x7fd7a019fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:25.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.512+0000 7fd7a52b9700 1 --2- 172.21.15.107:0/2341192492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7a0108000 0x7fd7a019fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:25.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.512+0000 7fd7a52b9700 1 -- 172.21.15.107:0/2341192492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7a01056d0 msgr2=0x7fd7a0198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:25.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.512+0000 7fd7a52b9700 1 --2- 172.21.15.107:0/2341192492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7a01056d0 0x7fd7a0198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:25.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.512+0000 7fd7a52b9700 1 -- 172.21.15.107:0/2341192492 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a01a00a0 con 0x7fd7a0102da0 2024-04-25T00:09:25.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.512+0000 7fd7a52b9700 1 --2- 172.21.15.107:0/2341192492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7a0102da0 0x7fd7a0198440 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00e9d0 tx=0x7fd79c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:25.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.512+0000 7fd7967fc700 1 -- 172.21.15.107:0/2341192492 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c00cbd0 con 0x7fd7a0102da0 2024-04-25T00:09:25.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.513+0000 7fd7a751d700 1 -- 172.21.15.107:0/2341192492 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a01a0390 con 0x7fd7a0102da0 2024-04-25T00:09:25.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.513+0000 7fd7967fc700 1 -- 172.21.15.107:0/2341192492 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd79c00cd70 con 0x7fd7a0102da0 2024-04-25T00:09:25.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.513+0000 7fd7967fc700 1 -- 172.21.15.107:0/2341192492 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c018680 con 0x7fd7a0102da0 2024-04-25T00:09:25.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.513+0000 7fd7a751d700 1 -- 172.21.15.107:0/2341192492 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a01a08d0 con 0x7fd7a0102da0 2024-04-25T00:09:25.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.513+0000 7fd7a751d700 1 -- 172.21.15.107:0/2341192492 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7a0068f20 con 0x7fd7a0102da0 2024-04-25T00:09:25.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.515+0000 7fd7967fc700 1 -- 172.21.15.107:0/2341192492 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) ==== 137778+0+0 (secure 0 0 0) 0x7fd79c0188e0 con 0x7fd7a0102da0 2024-04-25T00:09:25.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.515+0000 7fd7967fc700 1 --2- 172.21.15.107:0/2341192492 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd78c0a2b90 0x7fd78c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:25.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.515+0000 7fd7a4ab8700 1 --2- 172.21.15.107:0/2341192492 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd78c0a2b90 0x7fd78c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:25.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.515+0000 7fd7967fc700 1 -- 172.21.15.107:0/2341192492 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3226+0+0 (secure 0 0 0) 0x7fd79c0ce2b0 con 0x7fd7a0102da0 2024-04-25T00:09:25.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.516+0000 7fd7a4ab8700 1 --2- 172.21.15.107:0/2341192492 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd78c0a2b90 0x7fd78c0a5050 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd790009c60 tx=0x7fd790009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:25.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.519+0000 7fd7967fc700 1 -- 172.21.15.107:0/2341192492 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd79c09c6d0 con 0x7fd7a0102da0 2024-04-25T00:09:25.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:25 smithi195 ceph-mon[32833]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T00:09:25.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:25 smithi195 ceph-mon[32833]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:25.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:25 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:09:25.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:25 smithi170 ceph-mon[32807]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T00:09:25.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:25 smithi170 ceph-mon[32807]: pgmap v81: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:25.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:25 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:09:25.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:25.721+0000 7fd7a751d700 1 -- 172.21.15.107:0/2341192492 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd7a0062670 con 0x7fd78c0a2b90 2024-04-25T00:09:26.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:26 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T00:09:26.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:26 smithi107 ceph-mon[29153]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T00:09:26.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:26 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:09:26.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:26 smithi107 ceph-mon[29153]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:09:26.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:26 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:09:26.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:26 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:09:26.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:26 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:26.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:26 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T00:09:26.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:26 smithi195 ceph-mon[32833]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T00:09:26.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:26 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:09:26.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:26 smithi195 ceph-mon[32833]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:09:26.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:26 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:09:26.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:26 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:09:26.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:26 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:26.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:26 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T00:09:26.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:26 smithi170 ceph-mon[32807]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T00:09:26.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:26 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:09:26.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:26 smithi170 ceph-mon[32807]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:09:26.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:26 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:09:26.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:26 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:09:26.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:26 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:26.814 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 25 00:09:26 smithi107 sudo[45648]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:09:26.814 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 25 00:09:26 smithi107 sudo[45648]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:09:26.814 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 25 00:09:26 smithi107 sudo[45648]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:09:26.814 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 25 00:09:26 smithi107 sudo[45648]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:09:27.217 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 25 00:09:26 smithi107 sudo[45664]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:09:27.217 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 25 00:09:26 smithi107 sudo[45664]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:09:27.218 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 25 00:09:26 smithi107 sudo[45664]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:09:27.218 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 25 00:09:26 smithi107 sudo[45664]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:09:27.334 INFO:journalctl@ceph.osd.1.smithi170.stdout:Apr 25 00:09:27 smithi170 sudo[41101]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:09:27.376 INFO:journalctl@ceph.osd.1.smithi170.stdout:Apr 25 00:09:27 smithi170 sudo[41101]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:09:27.376 INFO:journalctl@ceph.osd.1.smithi170.stdout:Apr 25 00:09:27 smithi170 sudo[41101]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:09:27.376 INFO:journalctl@ceph.osd.1.smithi170.stdout:Apr 25 00:09:27 smithi170 sudo[41101]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:09:27.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:27 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:09:27.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:27 smithi170 ceph-mon[32807]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T00:09:27.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:27 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:27.377 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:27 smithi170 ceph-mon[32807]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:27.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:27 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:09:27.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:27 smithi107 ceph-mon[29153]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T00:09:27.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:27 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:27.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:27 smithi107 ceph-mon[29153]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:27.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:27 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:09:27.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:27 smithi195 ceph-mon[32833]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T00:09:27.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:27 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:27.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:27 smithi195 ceph-mon[32833]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:27.684 INFO:journalctl@ceph.osd.1.smithi170.stdout:Apr 25 00:09:27 smithi170 sudo[41104]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:09:27.685 INFO:journalctl@ceph.osd.1.smithi170.stdout:Apr 25 00:09:27 smithi170 sudo[41104]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:09:27.685 INFO:journalctl@ceph.osd.1.smithi170.stdout:Apr 25 00:09:27 smithi170 sudo[41104]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:09:27.685 INFO:journalctl@ceph.osd.1.smithi170.stdout:Apr 25 00:09:27 smithi170 sudo[41104]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:09:27.850 INFO:journalctl@ceph.osd.2.smithi195.stdout:Apr 25 00:09:27 smithi195 sudo[41930]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:09:27.850 INFO:journalctl@ceph.osd.2.smithi195.stdout:Apr 25 00:09:27 smithi195 sudo[41930]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:09:27.850 INFO:journalctl@ceph.osd.2.smithi195.stdout:Apr 25 00:09:27 smithi195 sudo[41930]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:09:27.850 INFO:journalctl@ceph.osd.2.smithi195.stdout:Apr 25 00:09:27 smithi195 sudo[41930]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:09:27.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:27.850+0000 7fd7967fc700 1 -- 172.21.15.107:0/2341192492 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fd79c098640 con 0x7fd7a0102da0 2024-04-25T00:09:28.339 INFO:journalctl@ceph.osd.2.smithi195.stdout:Apr 25 00:09:27 smithi195 sudo[41933]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:09:28.339 INFO:journalctl@ceph.osd.2.smithi195.stdout:Apr 25 00:09:27 smithi195 sudo[41933]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:09:28.339 INFO:journalctl@ceph.osd.2.smithi195.stdout:Apr 25 00:09:27 smithi195 sudo[41933]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:09:28.339 INFO:journalctl@ceph.osd.2.smithi195.stdout:Apr 25 00:09:27 smithi195 sudo[41933]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:09:28.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:28 smithi107 sudo[45712]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:09:28.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:28 smithi107 sudo[45712]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:09:28.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:28 smithi107 sudo[45712]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:09:28.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:28 smithi107 sudo[45712]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:09:28.644 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:28 smithi195 ceph-mon[32833]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T00:09:28.644 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:28 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:28.644 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:28 smithi195 ceph-mon[32833]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T00:09:28.644 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:28 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:28.644 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:28 smithi195 ceph-mon[32833]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:09:28.644 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:28 smithi195 sudo[41936]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:09:28.644 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:28 smithi195 sudo[41936]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:09:28.644 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:28 smithi195 sudo[41936]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:09:28.644 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:28 smithi195 sudo[41936]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:09:28.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:28 smithi170 ceph-mon[32807]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T00:09:28.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:28 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:28.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:28 smithi170 ceph-mon[32807]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T00:09:28.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:28 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:28.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:28 smithi170 ceph-mon[32807]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:09:28.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:28 smithi107 sudo[45742]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:09:28.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:28 smithi107 sudo[45742]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:09:28.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:28 smithi107 sudo[45742]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:09:28.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:28 smithi107 sudo[45742]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:09:28.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:28 smithi107 ceph-mon[29153]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T00:09:28.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:28 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:28.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:28 smithi107 ceph-mon[29153]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T00:09:28.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:28 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:28.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:28 smithi107 ceph-mon[29153]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:09:29.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:28 smithi195 sudo[41939]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:09:29.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:28 smithi195 sudo[41939]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:09:29.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:28 smithi195 sudo[41939]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:09:29.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:28 smithi195 sudo[41939]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:09:29.375 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 sudo[41107]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:09:29.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 sudo[41107]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:09:29.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 sudo[41107]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:09:29.376 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 sudo[41107]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:09:29.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 sudo[41110]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:09:29.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 sudo[41110]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:09:29.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 sudo[41110]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:09:29.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 sudo[41110]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:09:29.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 ceph-mon[32807]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:09:29.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:09:29.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:29.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:29.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 ceph-mon[32807]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:09:29.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:09:29.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:29.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:29.686 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/4213491180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c75ecce-0fee-4e65-8442-71125e8e5ad3"}]: dispatch 2024-04-25T00:09:29.686 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 ceph-mon[32807]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:29.686 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 ceph-mon[32807]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:09:29.686 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/4213491180' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c75ecce-0fee-4e65-8442-71125e8e5ad3"}]': finished 2024-04-25T00:09:29.686 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 ceph-mon[32807]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:09:29.686 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 ceph-mon[32807]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T00:09:29.686 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:09:29.686 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:09:29.686 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:29.686 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:29.686 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:29 smithi170 ceph-mon[32807]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:09:29.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:29 smithi107 ceph-mon[29153]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:09:29.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:29 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:09:29.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:29 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:29.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:29 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:29.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:29 smithi107 ceph-mon[29153]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:09:29.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:29 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:09:29.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:29 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:29.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:29 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:29.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:29 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/4213491180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c75ecce-0fee-4e65-8442-71125e8e5ad3"}]: dispatch 2024-04-25T00:09:29.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:29 smithi107 ceph-mon[29153]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:29.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:29 smithi107 ceph-mon[29153]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:09:29.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:29 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/4213491180' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c75ecce-0fee-4e65-8442-71125e8e5ad3"}]': finished 2024-04-25T00:09:29.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:29 smithi107 ceph-mon[29153]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:09:29.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:29 smithi107 ceph-mon[29153]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T00:09:29.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:29 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:09:29.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:29 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:09:29.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:29 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:29.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:29 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:29.736 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:29 smithi107 ceph-mon[29153]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:09:29.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:29 smithi195 ceph-mon[32833]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:09:29.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:29 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:09:29.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:29 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:29.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:29 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:29.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:29 smithi195 ceph-mon[32833]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:09:29.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:29 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:09:29.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:29 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:29.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:29 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:29.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:29 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/4213491180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c75ecce-0fee-4e65-8442-71125e8e5ad3"}]: dispatch 2024-04-25T00:09:29.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:29 smithi195 ceph-mon[32833]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:29.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:29 smithi195 ceph-mon[32833]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:09:29.841 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:29 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/4213491180' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c75ecce-0fee-4e65-8442-71125e8e5ad3"}]': finished 2024-04-25T00:09:29.841 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:29 smithi195 ceph-mon[32833]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:09:29.841 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:29 smithi195 ceph-mon[32833]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T00:09:29.841 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:29 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:09:29.841 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:29 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:09:29.841 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:29 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:29.841 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:29 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:29.841 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:29 smithi195 ceph-mon[32833]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:09:30.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:30 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/4263280993' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:09:30.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:30 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/4263280993' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:09:30.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:30 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/4263280993' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:09:31.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:31 smithi170 ceph-mon[32807]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:31.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:31 smithi107 ceph-mon[29153]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:31.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:31 smithi195 ceph-mon[32833]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:34.155 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:33 smithi107 ceph-mon[29153]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:34.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:33 smithi170 ceph-mon[32807]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:34.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:33 smithi195 ceph-mon[32833]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:35.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:35 smithi107 ceph-mon[29153]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:36.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:35 smithi170 ceph-mon[32807]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:36.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:35 smithi195 ceph-mon[32833]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:37.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:36 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T00:09:37.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:36 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:37.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:36 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T00:09:37.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:36 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:37.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:36 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T00:09:37.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:36 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:38.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:37 smithi170 ceph-mon[32807]: Deploying daemon osd.3 on smithi107 2024-04-25T00:09:38.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:37 smithi170 ceph-mon[32807]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:38.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:37 smithi107 ceph-mon[29153]: Deploying daemon osd.3 on smithi107 2024-04-25T00:09:38.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:37 smithi107 ceph-mon[29153]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:38.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:37 smithi195 ceph-mon[32833]: Deploying daemon osd.3 on smithi107 2024-04-25T00:09:38.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:37 smithi195 ceph-mon[32833]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:39.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:39 smithi107 ceph-mon[29153]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:40.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:39 smithi170 ceph-mon[32807]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:40.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:39 smithi195 ceph-mon[32833]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:42.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:41 smithi170 ceph-mon[32807]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:42.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:41 smithi107 ceph-mon[29153]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:42.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:41 smithi195 ceph-mon[32833]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:43.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:42 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:43.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:42 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:43.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:42 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:43.335 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:42 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:43.336 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:42 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:43.336 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:42 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:43.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:42 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:43.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:42 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:43.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:42 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:44.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:43 smithi107 ceph-mon[29153]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:44.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:43 smithi195 ceph-mon[32833]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:44.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:43 smithi170 ceph-mon[32807]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:45.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:45.426+0000 7fd7967fc700 1 -- 172.21.15.107:0/2341192492 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd7a0062670 con 0x7fd78c0a2b90 2024-04-25T00:09:45.427 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 3 on host 'smithi107' 2024-04-25T00:09:45.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:45.430+0000 7fd7a751d700 1 -- 172.21.15.107:0/2341192492 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd78c0a2b90 msgr2=0x7fd78c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:45.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:45.430+0000 7fd7a751d700 1 --2- 172.21.15.107:0/2341192492 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd78c0a2b90 0x7fd78c0a5050 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd790009c60 tx=0x7fd790009260 comp rx=0 tx=0).stop 2024-04-25T00:09:45.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:45.430+0000 7fd7a751d700 1 -- 172.21.15.107:0/2341192492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7a0102da0 msgr2=0x7fd7a0198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:45.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:45.430+0000 7fd7a751d700 1 --2- 172.21.15.107:0/2341192492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7a0102da0 0x7fd7a0198440 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00e9d0 tx=0x7fd79c00ece0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:45.430+0000 7fd7a751d700 1 -- 172.21.15.107:0/2341192492 shutdown_connections 2024-04-25T00:09:45.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:45.430+0000 7fd7a751d700 1 --2- 172.21.15.107:0/2341192492 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd78c0a2b90 0x7fd78c0a5050 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:45.430+0000 7fd7a751d700 1 --2- 172.21.15.107:0/2341192492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd7a0108000 0x7fd7a019fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:45.430+0000 7fd7a751d700 1 --2- 172.21.15.107:0/2341192492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7a0102da0 0x7fd7a0198440 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:45.430+0000 7fd7a751d700 1 --2- 172.21.15.107:0/2341192492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd7a01056d0 0x7fd7a0198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:45.430+0000 7fd7a751d700 1 -- 172.21.15.107:0/2341192492 >> 172.21.15.107:0/2341192492 conn(0x7fd7a00fc790 msgr2=0x7fd7a01036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:45.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:45.431+0000 7fd7a751d700 1 -- 172.21.15.107:0/2341192492 shutdown_connections 2024-04-25T00:09:45.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:09:45.431+0000 7fd7a751d700 1 -- 172.21.15.107:0/2341192492 wait complete. 2024-04-25T00:09:45.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:45 smithi107 ceph-mon[29153]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:45.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:45 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:45.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:45 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:45.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:45 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:45.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:45 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:45.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:45 smithi107 ceph-mon[29153]: from='osd.3 [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T00:09:46.100 DEBUG:teuthology.orchestra.run.smithi107:osd.3> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T00:09:46.102 INFO:tasks.cephadm:Deploying osd.4 on smithi170 with /dev/vg_nvme/lv_3... 2024-04-25T00:09:46.103 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T00:09:46.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:45 smithi195 ceph-mon[32833]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:46.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:45 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:46.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:45 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:46.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:45 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:46.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:45 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:46.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:45 smithi195 ceph-mon[32833]: from='osd.3 [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T00:09:46.345 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:09:46.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:45 smithi170 ceph-mon[32807]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:46.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:45 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:46.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:45 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:46.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:45 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:46.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:45 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:46.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:45 smithi170 ceph-mon[32807]: from='osd.3 [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T00:09:47.615 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:47 smithi107 ceph-mon[29153]: from='osd.3 [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T00:09:47.615 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:47 smithi107 ceph-mon[29153]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T00:09:47.616 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:47 smithi107 ceph-mon[29153]: from='osd.3 [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-25T00:09:47.616 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:09:47.616 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:47 smithi107 ceph-mon[29153]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:47.749 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:47 smithi170 ceph-mon[32807]: from='osd.3 [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T00:09:47.749 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:47 smithi170 ceph-mon[32807]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T00:09:47.749 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:47 smithi170 ceph-mon[32807]: from='osd.3 [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-25T00:09:47.749 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:09:47.749 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:47 smithi170 ceph-mon[32807]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:47.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:47 smithi195 ceph-mon[32833]: from='osd.3 [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T00:09:47.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:47 smithi195 ceph-mon[32833]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T00:09:47.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:47 smithi195 ceph-mon[32833]: from='osd.3 [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-25T00:09:47.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:09:47.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:47 smithi195 ceph-mon[32833]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:09:48.298 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T00:09:48.299 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T00:09:48.299 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-04-25T00:09:48.299 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-04-25T00:09:48.299 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00869652 s, 1.2 GB/s 2024-04-25T00:09:48.299 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-04-25T00:09:48.495 INFO:journalctl@ceph.osd.3.smithi107.stdout:Apr 25 00:09:48 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-3[47784]: 2024-04-25T00:09:48.232+0000 7fcca9c21700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T00:09:48.496 INFO:journalctl@ceph.osd.3.smithi107.stdout:Apr 25 00:09:48 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-3[47784]: 2024-04-25T00:09:48.238+0000 7fcca1a0c700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:09:48.838 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:48 smithi107 ceph-mon[29153]: from='osd.3 [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-25T00:09:48.838 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:48 smithi107 ceph-mon[29153]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T00:09:48.838 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:48 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:09:48.838 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:48 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:09:48.838 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:48 smithi107 ceph-mon[29153]: from='osd.3 [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444]' entity='osd.3' 2024-04-25T00:09:48.838 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:48 smithi107 ceph-mon[29153]: osd.3 [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444] boot 2024-04-25T00:09:48.838 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:48 smithi107 ceph-mon[29153]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T00:09:48.838 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:48 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:09:48.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:48 smithi195 ceph-mon[32833]: from='osd.3 [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-25T00:09:48.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:48 smithi195 ceph-mon[32833]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T00:09:48.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:48 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:09:48.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:48 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:09:48.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:48 smithi195 ceph-mon[32833]: from='osd.3 [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444]' entity='osd.3' 2024-04-25T00:09:48.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:48 smithi195 ceph-mon[32833]: osd.3 [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444] boot 2024-04-25T00:09:48.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:48 smithi195 ceph-mon[32833]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T00:09:48.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:48 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:09:48.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:48 smithi170 ceph-mon[32807]: from='osd.3 [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-25T00:09:48.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:48 smithi170 ceph-mon[32807]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T00:09:48.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:48 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:09:48.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:48 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:09:48.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:48 smithi170 ceph-mon[32807]: from='osd.3 [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444]' entity='osd.3' 2024-04-25T00:09:48.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:48 smithi170 ceph-mon[32807]: osd.3 [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444] boot 2024-04-25T00:09:48.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:48 smithi170 ceph-mon[32807]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T00:09:48.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:48 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:09:49.004 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi170:vg_nvme/lv_3 2024-04-25T00:09:49.244 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:09:49.623 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:49 smithi170 ceph-mon[32807]: purged_snaps scrub starts 2024-04-25T00:09:49.623 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:49 smithi170 ceph-mon[32807]: purged_snaps scrub ok 2024-04-25T00:09:49.623 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:49 smithi170 ceph-mon[32807]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 512 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:09:49.623 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:49 smithi170 ceph-mon[32807]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T00:09:49.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:49 smithi107 ceph-mon[29153]: purged_snaps scrub starts 2024-04-25T00:09:49.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:49 smithi107 ceph-mon[29153]: purged_snaps scrub ok 2024-04-25T00:09:49.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:49 smithi107 ceph-mon[29153]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 512 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:09:49.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:49 smithi107 ceph-mon[29153]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T00:09:49.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:49 smithi195 ceph-mon[32833]: purged_snaps scrub starts 2024-04-25T00:09:49.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:49 smithi195 ceph-mon[32833]: purged_snaps scrub ok 2024-04-25T00:09:49.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:49 smithi195 ceph-mon[32833]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 512 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:09:49.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:49 smithi195 ceph-mon[32833]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T00:09:50.747 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:50 smithi170 ceph-mon[32807]: Detected new or changed devices on smithi107 2024-04-25T00:09:50.747 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:50 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:50.747 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:50 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:50.748 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:50 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:50.748 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:50 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:50.748 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:50 smithi170 ceph-mon[32807]: Adjusting osd_memory_target on smithi107 to 8544M 2024-04-25T00:09:50.748 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:50 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:50.748 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:50 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:50.748 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:50 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:50.748 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:50 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:50.748 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:50 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:50.748 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:50 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:50.748 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:50 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:50.748 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:50 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:50.748 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:50 smithi170 ceph-mon[32807]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T00:09:50.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:50 smithi195 ceph-mon[32833]: Detected new or changed devices on smithi107 2024-04-25T00:09:50.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:50.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:50.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:50.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:50.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:50 smithi195 ceph-mon[32833]: Adjusting osd_memory_target on smithi107 to 8544M 2024-04-25T00:09:50.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:50.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:50.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:50.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:50.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:50.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:50.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:50.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:50.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:50 smithi195 ceph-mon[32833]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T00:09:50.851 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.851+0000 7fa2a9709700 1 -- 172.21.15.170:0/1624361984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a4105710 msgr2=0x7fa2a4107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:50.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.851+0000 7fa2a9709700 1 --2- 172.21.15.170:0/1624361984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a4105710 0x7fa2a4107b00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa294009990 tx=0x7fa294009ca0 comp rx=0 tx=0).stop 2024-04-25T00:09:50.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.851+0000 7fa2a9709700 1 -- 172.21.15.170:0/1624361984 shutdown_connections 2024-04-25T00:09:50.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.851+0000 7fa2a9709700 1 --2- 172.21.15.170:0/1624361984 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2a4108040 0x7fa2a410a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:50.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.851+0000 7fa2a9709700 1 --2- 172.21.15.170:0/1624361984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a4102de0 0x7fa2a41051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:50.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.851+0000 7fa2a9709700 1 --2- 172.21.15.170:0/1624361984 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a4105710 0x7fa2a4107b00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:50.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.851+0000 7fa2a9709700 1 -- 172.21.15.170:0/1624361984 >> 172.21.15.170:0/1624361984 conn(0x7fa2a40fc750 msgr2=0x7fa2a40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:50.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.851+0000 7fa2a9709700 1 -- 172.21.15.170:0/1624361984 shutdown_connections 2024-04-25T00:09:50.852 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.851+0000 7fa2a9709700 1 -- 172.21.15.170:0/1624361984 wait complete. 2024-04-25T00:09:50.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.852+0000 7fa2a9709700 1 Processor -- start 2024-04-25T00:09:50.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.852+0000 7fa2a9709700 1 -- start start 2024-04-25T00:09:50.854 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.852+0000 7fa2a9709700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a4102de0 0x7fa2a4198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:50.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.852+0000 7fa2a9709700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2a4105710 0x7fa2a41989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:50.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.852+0000 7fa2a9709700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a4108040 0x7fa2a419fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:50.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.852+0000 7fa2a9709700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a410de40 con 0x7fa2a4102de0 2024-04-25T00:09:50.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.852+0000 7fa2a9709700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a410dcc0 con 0x7fa2a4108040 2024-04-25T00:09:50.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.852+0000 7fa2a9709700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a410dfc0 con 0x7fa2a4105710 2024-04-25T00:09:50.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.852+0000 7fa2a37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a4108040 0x7fa2a419fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:50.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.852+0000 7fa2a37fe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a4108040 0x7fa2a419fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:60012/0 (socket says 172.21.15.170:60012) 2024-04-25T00:09:50.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.852+0000 7fa2a37fe700 1 -- 172.21.15.170:0/1316816526 learned_addr learned my addr 172.21.15.170:0/1316816526 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-04-25T00:09:50.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.852+0000 7fa2a2ffd700 1 --2- 172.21.15.170:0/1316816526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a4102de0 0x7fa2a4198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:50.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.852+0000 7fa2a27fc700 1 --2- 172.21.15.170:0/1316816526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2a4105710 0x7fa2a41989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:50.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.853+0000 7fa2a37fe700 1 -- 172.21.15.170:0/1316816526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2a4105710 msgr2=0x7fa2a41989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:50.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.853+0000 7fa2a37fe700 1 --2- 172.21.15.170:0/1316816526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2a4105710 0x7fa2a41989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:50.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.853+0000 7fa2a37fe700 1 -- 172.21.15.170:0/1316816526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a4102de0 msgr2=0x7fa2a4198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:50.855 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.853+0000 7fa2a37fe700 1 --2- 172.21.15.170:0/1316816526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a4102de0 0x7fa2a4198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:50.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.853+0000 7fa2a37fe700 1 -- 172.21.15.170:0/1316816526 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a41a00f0 con 0x7fa2a4108040 2024-04-25T00:09:50.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.853+0000 7fa2a37fe700 1 --2- 172.21.15.170:0/1316816526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a4108040 0x7fa2a419fa50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00ada0 tx=0x7fa29c00c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:50.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.853+0000 7fa28bfff700 1 -- 172.21.15.170:0/1316816526 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c017020 con 0x7fa2a4108040 2024-04-25T00:09:50.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.853+0000 7fa2a9709700 1 -- 172.21.15.170:0/1316816526 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a41a03e0 con 0x7fa2a4108040 2024-04-25T00:09:50.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.853+0000 7fa28bfff700 1 -- 172.21.15.170:0/1316816526 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa29c00a0d0 con 0x7fa2a4108040 2024-04-25T00:09:50.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.853+0000 7fa28bfff700 1 -- 172.21.15.170:0/1316816526 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c005480 con 0x7fa2a4108040 2024-04-25T00:09:50.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.853+0000 7fa2a9709700 1 -- 172.21.15.170:0/1316816526 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a41a08d0 con 0x7fa2a4108040 2024-04-25T00:09:50.856 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.854+0000 7fa2a9709700 1 -- 172.21.15.170:0/1316816526 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a4068f20 con 0x7fa2a4108040 2024-04-25T00:09:50.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.855+0000 7fa28bfff700 1 -- 172.21.15.170:0/1316816526 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa29c005620 con 0x7fa2a4108040 2024-04-25T00:09:50.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.856+0000 7fa28bfff700 1 --2- 172.21.15.170:0/1316816526 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa28c0a2c20 0x7fa28c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:50.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.856+0000 7fa28bfff700 1 -- 172.21.15.170:0/1316816526 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3698+0+0 (secure 0 0 0) 0x7fa29c0cdba0 con 0x7fa2a4108040 2024-04-25T00:09:50.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.856+0000 7fa2a2ffd700 1 --2- 172.21.15.170:0/1316816526 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa28c0a2c20 0x7fa28c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:50.857 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.856+0000 7fa2a2ffd700 1 --2- 172.21.15.170:0/1316816526 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa28c0a2c20 0x7fa28c0a50e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa298005950 tx=0x7fa29800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:50.860 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:50.859+0000 7fa28bfff700 1 -- 172.21.15.170:0/1316816526 <== mon.2 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa29c021020 con 0x7fa2a4108040 2024-04-25T00:09:50.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:50 smithi107 ceph-mon[29153]: Detected new or changed devices on smithi107 2024-04-25T00:09:50.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:50.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:50.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:50.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:50.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:50 smithi107 ceph-mon[29153]: Adjusting osd_memory_target on smithi107 to 8544M 2024-04-25T00:09:50.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:50.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:50.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:50.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:50.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:50.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:50.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:50.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:09:50.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:50 smithi107 ceph-mon[29153]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T00:09:51.064 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:09:51.064+0000 7fa2a9709700 1 -- 172.21.15.170:0/1316816526 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa2a4062670 con 0x7fa28c0a2c20 2024-04-25T00:09:51.775 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:51 smithi170 ceph-mon[32807]: pgmap v103: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 512 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:09:51.775 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:51 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:09:51.775 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:51 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:09:51.775 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:51 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:51.775 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:51 smithi170 ceph-mon[32807]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T00:09:51.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:51 smithi195 ceph-mon[32833]: pgmap v103: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 512 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:09:51.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:51 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:09:51.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:51 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:09:51.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:51 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:51.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:51 smithi195 ceph-mon[32833]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T00:09:51.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:51 smithi107 ceph-mon[29153]: pgmap v103: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 512 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:09:51.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:51 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:09:51.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:51 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:09:51.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:51 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:51.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:51 smithi107 ceph-mon[29153]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T00:09:52.797 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:52 smithi170 ceph-mon[32807]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:09:52.797 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:52 smithi170 ceph-mon[32807]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T00:09:52.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:52 smithi195 ceph-mon[32833]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:09:52.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:52 smithi195 ceph-mon[32833]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T00:09:52.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:52 smithi107 ceph-mon[29153]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:09:52.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:52 smithi107 ceph-mon[29153]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T00:09:53.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:53 smithi195 ceph-mon[32833]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 512 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:09:53.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:53 smithi170 ceph-mon[32807]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 512 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:09:53.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:53 smithi107 ceph-mon[29153]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 512 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:09:54.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:54 smithi195 ceph-mon[32833]: from='client.? 172.21.15.170:0/1393254540' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f61a3aa-62cc-43aa-b4b0-8931c9837b57"}]: dispatch 2024-04-25T00:09:54.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:54 smithi195 ceph-mon[32833]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f61a3aa-62cc-43aa-b4b0-8931c9837b57"}]: dispatch 2024-04-25T00:09:54.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:54 smithi195 ceph-mon[32833]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6f61a3aa-62cc-43aa-b4b0-8931c9837b57"}]': finished 2024-04-25T00:09:54.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:54 smithi195 ceph-mon[32833]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T00:09:54.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:54 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:09:54.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:54 smithi195 ceph-mon[32833]: from='client.? 172.21.15.170:0/3094081105' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:09:54.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:54 smithi170 ceph-mon[32807]: from='client.? 172.21.15.170:0/1393254540' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f61a3aa-62cc-43aa-b4b0-8931c9837b57"}]: dispatch 2024-04-25T00:09:54.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:54 smithi170 ceph-mon[32807]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f61a3aa-62cc-43aa-b4b0-8931c9837b57"}]: dispatch 2024-04-25T00:09:54.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:54 smithi170 ceph-mon[32807]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6f61a3aa-62cc-43aa-b4b0-8931c9837b57"}]': finished 2024-04-25T00:09:54.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:54 smithi170 ceph-mon[32807]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T00:09:54.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:09:54.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:54 smithi170 ceph-mon[32807]: from='client.? 172.21.15.170:0/3094081105' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:09:54.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:54 smithi107 ceph-mon[29153]: from='client.? 172.21.15.170:0/1393254540' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f61a3aa-62cc-43aa-b4b0-8931c9837b57"}]: dispatch 2024-04-25T00:09:54.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:54 smithi107 ceph-mon[29153]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6f61a3aa-62cc-43aa-b4b0-8931c9837b57"}]: dispatch 2024-04-25T00:09:54.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:54 smithi107 ceph-mon[29153]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6f61a3aa-62cc-43aa-b4b0-8931c9837b57"}]': finished 2024-04-25T00:09:54.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:54 smithi107 ceph-mon[29153]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T00:09:54.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:54 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:09:54.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:54 smithi107 ceph-mon[29153]: from='client.? 172.21.15.170:0/3094081105' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:09:55.784 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:55 smithi107 ceph-mon[29153]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-04-25T00:09:55.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:55 smithi195 ceph-mon[32833]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-04-25T00:09:55.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:55 smithi170 ceph-mon[32807]: pgmap v108: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 132 KiB/s, 0 objects/s recovering 2024-04-25T00:09:58.054 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:57 smithi170 ceph-mon[32807]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:09:58.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:57 smithi107 ceph-mon[29153]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:09:58.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:57 smithi195 ceph-mon[32833]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:00.032 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:09:59 smithi170 ceph-mon[32807]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:00.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:09:59 smithi107 ceph-mon[29153]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:00.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:09:59 smithi195 ceph-mon[32833]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:01.011 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:00 smithi170 ceph-mon[32807]: overall HEALTH_OK 2024-04-25T00:10:01.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:00 smithi107 ceph-mon[29153]: overall HEALTH_OK 2024-04-25T00:10:01.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:00 smithi195 ceph-mon[32833]: overall HEALTH_OK 2024-04-25T00:10:02.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:01 smithi170 ceph-mon[32807]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:02.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T00:10:02.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:02.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:01 smithi107 ceph-mon[29153]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:02.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T00:10:02.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:02.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:01 smithi195 ceph-mon[32833]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:02.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T00:10:02.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:02 smithi170 ceph-mon[32807]: Deploying daemon osd.4 on smithi170 2024-04-25T00:10:03.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:02 smithi107 ceph-mon[29153]: Deploying daemon osd.4 on smithi170 2024-04-25T00:10:03.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:02 smithi195 ceph-mon[32833]: Deploying daemon osd.4 on smithi170 2024-04-25T00:10:04.073 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:03 smithi170 ceph-mon[32807]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:04.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:03 smithi107 ceph-mon[29153]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:04.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:03 smithi195 ceph-mon[32833]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:06.025 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:05 smithi170 ceph-mon[32807]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:06.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:05 smithi107 ceph-mon[29153]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:06.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:05 smithi195 ceph-mon[32833]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:07.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:07 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:07.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:07 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:07.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:07 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:07.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:07 smithi170 ceph-mon[32807]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:07.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:07 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:07.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:07 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:07.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:07 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:07.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:07 smithi107 ceph-mon[29153]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:08.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:07 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:08.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:07 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:08.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:07 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:08.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:07 smithi195 ceph-mon[32833]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:10.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:09 smithi170 ceph-mon[32807]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:10.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:09 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:10.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:09 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:10.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:09 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:10.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:09 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:10.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:09 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:10.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:09 smithi107 ceph-mon[29153]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:10.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:09 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:10.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:09 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:10.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:09 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:10.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:09 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:10.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:09 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:10.235 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:10:10.233+0000 7fa28bfff700 1 -- 172.21.15.170:0/1316816526 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa2a4062670 con 0x7fa28c0a2c20 2024-04-25T00:10:10.235 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 4 on host 'smithi170' 2024-04-25T00:10:10.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:10:10.235+0000 7fa2a9709700 1 -- 172.21.15.170:0/1316816526 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa28c0a2c20 msgr2=0x7fa28c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:10.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:10:10.235+0000 7fa2a9709700 1 --2- 172.21.15.170:0/1316816526 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa28c0a2c20 0x7fa28c0a50e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa298005950 tx=0x7fa29800a300 comp rx=0 tx=0).stop 2024-04-25T00:10:10.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:10:10.235+0000 7fa2a9709700 1 -- 172.21.15.170:0/1316816526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a4108040 msgr2=0x7fa2a419fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:10.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:10:10.235+0000 7fa2a9709700 1 --2- 172.21.15.170:0/1316816526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a4108040 0x7fa2a419fa50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00ada0 tx=0x7fa29c00c330 comp rx=0 tx=0).stop 2024-04-25T00:10:10.236 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:10:10.235+0000 7fa2a9709700 1 -- 172.21.15.170:0/1316816526 shutdown_connections 2024-04-25T00:10:10.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:10:10.235+0000 7fa2a9709700 1 --2- 172.21.15.170:0/1316816526 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa28c0a2c20 0x7fa28c0a50e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:10.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:10:10.235+0000 7fa2a9709700 1 --2- 172.21.15.170:0/1316816526 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a4108040 0x7fa2a419fa50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:10.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:10:10.235+0000 7fa2a9709700 1 --2- 172.21.15.170:0/1316816526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a4102de0 0x7fa2a4198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:10.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:10:10.235+0000 7fa2a9709700 1 --2- 172.21.15.170:0/1316816526 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2a4105710 0x7fa2a41989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:10.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:10:10.235+0000 7fa2a9709700 1 -- 172.21.15.170:0/1316816526 >> 172.21.15.170:0/1316816526 conn(0x7fa2a40fc750 msgr2=0x7fa2a4103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:10.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:10:10.236+0000 7fa2a9709700 1 -- 172.21.15.170:0/1316816526 shutdown_connections 2024-04-25T00:10:10.237 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:10:10.236+0000 7fa2a9709700 1 -- 172.21.15.170:0/1316816526 wait complete. 2024-04-25T00:10:10.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:09 smithi195 ceph-mon[32833]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:10.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:09 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:10.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:09 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:10.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:09 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:10.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:09 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:10.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:09 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:10.883 DEBUG:teuthology.orchestra.run.smithi170:osd.4> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T00:10:10.886 INFO:tasks.cephadm:Deploying osd.5 on smithi195 with /dev/vg_nvme/lv_3... 2024-04-25T00:10:10.886 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T00:10:11.131 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:10:11.429 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:11 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:11.429 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:11 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:11.429 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:11 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:11.429 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:11 smithi170 ceph-mon[32807]: from='osd.4 [v2:172.21.15.170:6808/3780266808,v1:172.21.15.170:6809/3780266808]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T00:10:11.429 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:11 smithi170 ceph-mon[32807]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T00:10:11.429 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:11 smithi170 ceph-mon[32807]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:11.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:11 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:11.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:11 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:11.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:11 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:11.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:11 smithi107 ceph-mon[29153]: from='osd.4 [v2:172.21.15.170:6808/3780266808,v1:172.21.15.170:6809/3780266808]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T00:10:11.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:11 smithi107 ceph-mon[29153]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T00:10:11.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:11 smithi107 ceph-mon[29153]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:11.539 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:11 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:11.539 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:11 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:11.539 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:11 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:11.539 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:11 smithi195 ceph-mon[32833]: from='osd.4 [v2:172.21.15.170:6808/3780266808,v1:172.21.15.170:6809/3780266808]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T00:10:11.539 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:11 smithi195 ceph-mon[32833]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T00:10:11.540 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:11 smithi195 ceph-mon[32833]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:12.366 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:12 smithi170 ceph-mon[32807]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T00:10:12.366 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:12 smithi170 ceph-mon[32807]: from='osd.4 [v2:172.21.15.170:6808/3780266808,v1:172.21.15.170:6809/3780266808]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-25T00:10:12.366 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:12 smithi170 ceph-mon[32807]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T00:10:12.366 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:12 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:10:12.366 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:12 smithi170 ceph-mon[32807]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-25T00:10:12.544 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:12 smithi195 ceph-mon[32833]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T00:10:12.544 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:12 smithi195 ceph-mon[32833]: from='osd.4 [v2:172.21.15.170:6808/3780266808,v1:172.21.15.170:6809/3780266808]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-25T00:10:12.544 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:12 smithi195 ceph-mon[32833]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T00:10:12.544 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:12 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:10:12.545 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:12 smithi195 ceph-mon[32833]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-25T00:10:12.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:12 smithi107 ceph-mon[29153]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T00:10:12.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:12 smithi107 ceph-mon[29153]: from='osd.4 [v2:172.21.15.170:6808/3780266808,v1:172.21.15.170:6809/3780266808]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-25T00:10:12.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:12 smithi107 ceph-mon[29153]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T00:10:12.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:12 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:10:12.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:12 smithi107 ceph-mon[29153]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-25T00:10:12.935 INFO:journalctl@ceph.osd.4.smithi170.stdout:Apr 25 00:10:12 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-4[43915]: 2024-04-25T00:10:12.878+0000 7fbaf7c8f700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T00:10:12.935 INFO:journalctl@ceph.osd.4.smithi170.stdout:Apr 25 00:10:12 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-4[43915]: 2024-04-25T00:10:12.886+0000 7fbaefa7a700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:10:13.092 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T00:10:13.092 INFO:teuthology.orchestra.run.smithi195.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T00:10:13.092 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10+0 records in 2024-04-25T00:10:13.092 INFO:teuthology.orchestra.run.smithi195.stderr:10+0 records out 2024-04-25T00:10:13.092 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0084784 s, 1.2 GB/s 2024-04-25T00:10:13.092 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping successful for: 2024-04-25T00:10:13.397 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:13 smithi195 ceph-mon[32833]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-04-25T00:10:13.397 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:13 smithi195 ceph-mon[32833]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T00:10:13.397 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:13 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:10:13.397 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:13 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:10:13.397 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:13 smithi195 ceph-mon[32833]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:13.398 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:13 smithi195 ceph-mon[32833]: from='osd.4 ' entity='osd.4' 2024-04-25T00:10:13.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:13 smithi170 ceph-mon[32807]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-04-25T00:10:13.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:13 smithi170 ceph-mon[32807]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T00:10:13.646 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:13 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:10:13.647 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:13 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:10:13.647 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:13 smithi170 ceph-mon[32807]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:13.647 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:13 smithi170 ceph-mon[32807]: from='osd.4 ' entity='osd.4' 2024-04-25T00:10:13.710 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi195:vg_nvme/lv_3 2024-04-25T00:10:13.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:13 smithi107 ceph-mon[29153]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-04-25T00:10:13.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:13 smithi107 ceph-mon[29153]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T00:10:13.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:13 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:10:13.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:13 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:10:13.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:13 smithi107 ceph-mon[29153]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-25T00:10:13.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:13 smithi107 ceph-mon[29153]: from='osd.4 ' entity='osd.4' 2024-04-25T00:10:13.955 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:10:14.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:14 smithi195 ceph-mon[32833]: purged_snaps scrub starts 2024-04-25T00:10:14.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:14 smithi195 ceph-mon[32833]: purged_snaps scrub ok 2024-04-25T00:10:14.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:14 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:10:14.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:14 smithi195 ceph-mon[32833]: osd.4 [v2:172.21.15.170:6808/3780266808,v1:172.21.15.170:6809/3780266808] boot 2024-04-25T00:10:14.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:14 smithi195 ceph-mon[32833]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T00:10:14.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:14 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:10:14.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:14 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:14.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:14 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:14.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:14 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:10:14.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:14 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:10:14.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:14 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:14.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:14 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:14.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:14 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:14.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:14 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:14.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:14 smithi170 ceph-mon[32807]: purged_snaps scrub starts 2024-04-25T00:10:14.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:14 smithi170 ceph-mon[32807]: purged_snaps scrub ok 2024-04-25T00:10:14.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:14 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:10:14.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:14 smithi170 ceph-mon[32807]: osd.4 [v2:172.21.15.170:6808/3780266808,v1:172.21.15.170:6809/3780266808] boot 2024-04-25T00:10:14.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:14 smithi170 ceph-mon[32807]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T00:10:14.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:14 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:10:14.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:14 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:14.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:14 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:14.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:14 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:10:14.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:14 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:10:14.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:14 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:14.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:14 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:14.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:14 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:14.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:14 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:14.635 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:14 smithi107 ceph-mon[29153]: purged_snaps scrub starts 2024-04-25T00:10:14.635 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:14 smithi107 ceph-mon[29153]: purged_snaps scrub ok 2024-04-25T00:10:14.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:14 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:10:14.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:14 smithi107 ceph-mon[29153]: osd.4 [v2:172.21.15.170:6808/3780266808,v1:172.21.15.170:6809/3780266808] boot 2024-04-25T00:10:14.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:14 smithi107 ceph-mon[29153]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T00:10:14.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:14 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:10:14.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:14 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:14.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:14 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:14.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:14 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:10:14.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:14 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:10:14.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:14 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:14.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:14 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:14.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:14 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:14.636 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:14 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:15.393 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:15 smithi195 ceph-mon[32833]: Detected new or changed devices on smithi170 2024-04-25T00:10:15.394 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:15 smithi195 ceph-mon[32833]: Adjusting osd_memory_target on smithi170 to 8544M 2024-04-25T00:10:15.394 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:15 smithi195 ceph-mon[32833]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:15.596 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.596+0000 7f7050b8f700 1 -- 172.21.15.195:0/253333654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c103e30 msgr2=0x7f704c10a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:15.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.596+0000 7f7050b8f700 1 --2- 172.21.15.195:0/253333654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c103e30 0x7f704c10a6e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f704400b0d0 tx=0x7f704400b3e0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.596+0000 7f7050b8f700 1 -- 172.21.15.195:0/253333654 shutdown_connections 2024-04-25T00:10:15.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.596+0000 7f7050b8f700 1 --2- 172.21.15.195:0/253333654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c103e30 0x7f704c10a6e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.596+0000 7f7050b8f700 1 --2- 172.21.15.195:0/253333654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f704c102290 0x7f704c1026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.596+0000 7f7050b8f700 1 --2- 172.21.15.195:0/253333654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f704c103490 0x7f704c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.596+0000 7f7050b8f700 1 -- 172.21.15.195:0/253333654 >> 172.21.15.195:0/253333654 conn(0x7f704c0fd830 msgr2=0x7f704c0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:15.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.596+0000 7f7050b8f700 1 -- 172.21.15.195:0/253333654 shutdown_connections 2024-04-25T00:10:15.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.596+0000 7f7050b8f700 1 -- 172.21.15.195:0/253333654 wait complete. 2024-04-25T00:10:15.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.597+0000 7f7050b8f700 1 Processor -- start 2024-04-25T00:10:15.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.597+0000 7f7050b8f700 1 -- start start 2024-04-25T00:10:15.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.597+0000 7f7050b8f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f704c102290 0x7f704c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:15.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.597+0000 7f7050b8f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f704c103490 0x7f704c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:15.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.597+0000 7f7050b8f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c103e30 0x7f704c19f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:15.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.597+0000 7f7050b8f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f704c10de70 con 0x7f704c103e30 2024-04-25T00:10:15.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.597+0000 7f7050b8f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f704c10dcf0 con 0x7f704c103490 2024-04-25T00:10:15.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.597+0000 7f7050b8f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f704c10dff0 con 0x7f704c102290 2024-04-25T00:10:15.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.597+0000 7f704a59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f704c102290 0x7f704c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:15.600 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.597+0000 7f704a59c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f704c102290 0x7f704c1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:42990/0 (socket says 172.21.15.195:42990) 2024-04-25T00:10:15.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.597+0000 7f704a59c700 1 -- 172.21.15.195:0/1594086626 learned_addr learned my addr 172.21.15.195:0/1594086626 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-04-25T00:10:15.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.597+0000 7f704ad9d700 1 --2- 172.21.15.195:0/1594086626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c103e30 0x7f704c19f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:15.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.597+0000 7f7049d9b700 1 --2- 172.21.15.195:0/1594086626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f704c103490 0x7f704c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:15.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.597+0000 7f704a59c700 1 -- 172.21.15.195:0/1594086626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f704c103490 msgr2=0x7f704c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:15.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.597+0000 7f704a59c700 1 --2- 172.21.15.195:0/1594086626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f704c103490 0x7f704c198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.597+0000 7f704a59c700 1 -- 172.21.15.195:0/1594086626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c103e30 msgr2=0x7f704c19f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:15.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.597+0000 7f704a59c700 1 --2- 172.21.15.195:0/1594086626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c103e30 0x7f704c19f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:15.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.597+0000 7f704a59c700 1 -- 172.21.15.195:0/1594086626 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704c1a0050 con 0x7f704c102290 2024-04-25T00:10:15.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.598+0000 7f704a59c700 1 --2- 172.21.15.195:0/1594086626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f704c102290 0x7f704c1983f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f703c00d9b0 tx=0x7f703c00dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:15.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.598+0000 7f703b7fe700 1 -- 172.21.15.195:0/1594086626 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c016070 con 0x7f704c102290 2024-04-25T00:10:15.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.598+0000 7f703b7fe700 1 -- 172.21.15.195:0/1594086626 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f703c00be30 con 0x7f704c102290 2024-04-25T00:10:15.601 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.598+0000 7f7050b8f700 1 -- 172.21.15.195:0/1594086626 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f704c1a0340 con 0x7f704c102290 2024-04-25T00:10:15.602 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.598+0000 7f703b7fe700 1 -- 172.21.15.195:0/1594086626 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c00f3c0 con 0x7f704c102290 2024-04-25T00:10:15.602 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.598+0000 7f7050b8f700 1 -- 172.21.15.195:0/1594086626 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f704c1a0880 con 0x7f704c102290 2024-04-25T00:10:15.602 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.599+0000 7f7050b8f700 1 -- 172.21.15.195:0/1594086626 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f704c068f20 con 0x7f704c102290 2024-04-25T00:10:15.602 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.600+0000 7f703b7fe700 1 -- 172.21.15.195:0/1594086626 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f703c019070 con 0x7f704c102290 2024-04-25T00:10:15.603 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.600+0000 7f703b7fe700 1 --2- 172.21.15.195:0/1594086626 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f70340a2c20 0x7f70340a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:15.603 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.601+0000 7f703b7fe700 1 -- 172.21.15.195:0/1594086626 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f703c0ce3b0 con 0x7f704c102290 2024-04-25T00:10:15.603 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.601+0000 7f7049d9b700 1 --2- 172.21.15.195:0/1594086626 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f70340a2c20 0x7f70340a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:15.603 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.601+0000 7f7049d9b700 1 --2- 172.21.15.195:0/1594086626 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f70340a2c20 0x7f70340a50e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7040009b10 tx=0x7f7040009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:15.605 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.604+0000 7f703b7fe700 1 -- 172.21.15.195:0/1594086626 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f703c020020 con 0x7f704c102290 2024-04-25T00:10:15.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:15 smithi170 ceph-mon[32807]: Detected new or changed devices on smithi170 2024-04-25T00:10:15.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:15 smithi170 ceph-mon[32807]: Adjusting osd_memory_target on smithi170 to 8544M 2024-04-25T00:10:15.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:15 smithi170 ceph-mon[32807]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:15.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:15 smithi107 ceph-mon[29153]: Detected new or changed devices on smithi170 2024-04-25T00:10:15.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:15 smithi107 ceph-mon[29153]: Adjusting osd_memory_target on smithi170 to 8544M 2024-04-25T00:10:15.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:15 smithi107 ceph-mon[29153]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:15.809 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:15.809+0000 7f7050b8f700 1 -- 172.21.15.195:0/1594086626 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f704c062670 con 0x7f70340a2c20 2024-04-25T00:10:16.508 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:16 smithi195 ceph-mon[32833]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T00:10:16.508 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:16 smithi195 ceph-mon[32833]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:10:16.508 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:16 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:10:16.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:16 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:10:16.509 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:16 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:16.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:16 smithi170 ceph-mon[32807]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T00:10:16.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:16 smithi170 ceph-mon[32807]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:10:16.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:16 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:10:16.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:16 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:10:16.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:16 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:16.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:16 smithi107 ceph-mon[29153]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T00:10:16.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:16 smithi107 ceph-mon[29153]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:10:16.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:16 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:10:16.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:16 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:10:16.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:16 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:17.529 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:17 smithi195 ceph-mon[32833]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:17.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:17 smithi170 ceph-mon[32807]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:17.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:17 smithi107 ceph-mon[29153]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:18.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:18 smithi195 ceph-mon[32833]: from='client.? 172.21.15.195:0/3303295377' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "104da40f-019b-4dc3-8c62-090920ebe24f"}]: dispatch 2024-04-25T00:10:18.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:18 smithi195 ceph-mon[32833]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "104da40f-019b-4dc3-8c62-090920ebe24f"}]: dispatch 2024-04-25T00:10:18.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:18 smithi195 ceph-mon[32833]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "104da40f-019b-4dc3-8c62-090920ebe24f"}]': finished 2024-04-25T00:10:18.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:18 smithi195 ceph-mon[32833]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T00:10:18.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:18 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:10:18.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:18 smithi170 ceph-mon[32807]: from='client.? 172.21.15.195:0/3303295377' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "104da40f-019b-4dc3-8c62-090920ebe24f"}]: dispatch 2024-04-25T00:10:18.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:18 smithi170 ceph-mon[32807]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "104da40f-019b-4dc3-8c62-090920ebe24f"}]: dispatch 2024-04-25T00:10:18.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:18 smithi170 ceph-mon[32807]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "104da40f-019b-4dc3-8c62-090920ebe24f"}]': finished 2024-04-25T00:10:18.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:18 smithi170 ceph-mon[32807]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T00:10:18.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:18 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:10:18.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:18 smithi107 ceph-mon[29153]: from='client.? 172.21.15.195:0/3303295377' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "104da40f-019b-4dc3-8c62-090920ebe24f"}]: dispatch 2024-04-25T00:10:18.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:18 smithi107 ceph-mon[29153]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "104da40f-019b-4dc3-8c62-090920ebe24f"}]: dispatch 2024-04-25T00:10:18.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:18 smithi107 ceph-mon[29153]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "104da40f-019b-4dc3-8c62-090920ebe24f"}]': finished 2024-04-25T00:10:18.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:18 smithi107 ceph-mon[29153]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T00:10:18.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:18 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:10:19.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:19 smithi195 ceph-mon[32833]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:19.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:19 smithi195 ceph-mon[32833]: from='client.? 172.21.15.195:0/2983647658' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:10:19.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:19 smithi170 ceph-mon[32807]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:19.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:19 smithi170 ceph-mon[32807]: from='client.? 172.21.15.195:0/2983647658' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:10:19.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:19 smithi107 ceph-mon[29153]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:19.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:19 smithi107 ceph-mon[29153]: from='client.? 172.21.15.195:0/2983647658' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:10:22.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:21 smithi170 ceph-mon[32807]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:22.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:21 smithi107 ceph-mon[29153]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:22.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:21 smithi195 ceph-mon[32833]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:24.050 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:23 smithi195 ceph-mon[32833]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:24.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:23 smithi170 ceph-mon[32807]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:24.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:23 smithi107 ceph-mon[29153]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:26.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:25 smithi170 ceph-mon[32807]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:26.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:25 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T00:10:26.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:25 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:26.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:25 smithi107 ceph-mon[29153]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:26.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:25 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T00:10:26.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:25 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:26.271 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:25 smithi195 ceph-mon[32833]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:26.271 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:25 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T00:10:26.271 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:25 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:27.214 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:26 smithi195 ceph-mon[32833]: Deploying daemon osd.5 on smithi195 2024-04-25T00:10:27.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:26 smithi107 ceph-mon[29153]: Deploying daemon osd.5 on smithi195 2024-04-25T00:10:27.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:26 smithi170 ceph-mon[32807]: Deploying daemon osd.5 on smithi195 2024-04-25T00:10:28.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:27 smithi107 ceph-mon[29153]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:28.245 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:27 smithi195 ceph-mon[32833]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:28.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:27 smithi170 ceph-mon[32807]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:30.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:29 smithi107 ceph-mon[29153]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:30.246 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:29 smithi195 ceph-mon[32833]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:30.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:29 smithi170 ceph-mon[32807]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:32.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:32 smithi195 ceph-mon[32833]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:32.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:32 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:32.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:32 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:32.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:32 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:32.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:32 smithi170 ceph-mon[32807]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:32.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:32 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:32.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:32 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:32.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:32 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:32.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:32 smithi107 ceph-mon[29153]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:32.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:32 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:32.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:32 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:32.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:32 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:33.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:33 smithi195 ceph-mon[32833]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:33.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:33 smithi170 ceph-mon[32807]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:33.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:33 smithi107 ceph-mon[29153]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:34.476 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:34.475+0000 7f703b7fe700 1 -- 172.21.15.195:0/1594086626 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f704c062670 con 0x7f70340a2c20 2024-04-25T00:10:34.476 INFO:teuthology.orchestra.run.smithi195.stdout:Created osd(s) 5 on host 'smithi195' 2024-04-25T00:10:34.478 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:34.477+0000 7f7050b8f700 1 -- 172.21.15.195:0/1594086626 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f70340a2c20 msgr2=0x7f70340a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:34.478 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:34.477+0000 7f7050b8f700 1 --2- 172.21.15.195:0/1594086626 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f70340a2c20 0x7f70340a50e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7040009b10 tx=0x7f7040009260 comp rx=0 tx=0).stop 2024-04-25T00:10:34.478 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:34.477+0000 7f7050b8f700 1 -- 172.21.15.195:0/1594086626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f704c102290 msgr2=0x7f704c1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:34.479 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:34.477+0000 7f7050b8f700 1 --2- 172.21.15.195:0/1594086626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f704c102290 0x7f704c1983f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f703c00d9b0 tx=0x7f703c00dcc0 comp rx=0 tx=0).stop 2024-04-25T00:10:34.479 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:34.477+0000 7f7050b8f700 1 -- 172.21.15.195:0/1594086626 shutdown_connections 2024-04-25T00:10:34.479 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:34.477+0000 7f7050b8f700 1 --2- 172.21.15.195:0/1594086626 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f70340a2c20 0x7f70340a50e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:34.479 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:34.477+0000 7f7050b8f700 1 --2- 172.21.15.195:0/1594086626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f704c103e30 0x7f704c19f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:34.479 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:34.478+0000 7f7050b8f700 1 --2- 172.21.15.195:0/1594086626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f704c102290 0x7f704c1983f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:34.479 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:34.478+0000 7f7050b8f700 1 --2- 172.21.15.195:0/1594086626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f704c103490 0x7f704c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:34.479 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:34.478+0000 7f7050b8f700 1 -- 172.21.15.195:0/1594086626 >> 172.21.15.195:0/1594086626 conn(0x7f704c0fd830 msgr2=0x7f704c106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:34.479 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:34.478+0000 7f7050b8f700 1 -- 172.21.15.195:0/1594086626 shutdown_connections 2024-04-25T00:10:34.479 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:10:34.478+0000 7f7050b8f700 1 -- 172.21.15.195:0/1594086626 wait complete. 2024-04-25T00:10:35.141 DEBUG:teuthology.orchestra.run.smithi195:osd.5> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T00:10:35.143 INFO:tasks.cephadm:Deploying osd.6 on smithi107 with /dev/vg_nvme/lv_2... 2024-04-25T00:10:35.144 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T00:10:35.390 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:10:35.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:35 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:35.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:35 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:35.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:35 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:35.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:35 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:35.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:35 smithi107 ceph-mon[29153]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:35.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:35 smithi107 ceph-mon[29153]: from='osd.5 [v2:172.21.15.195:6808/2322632437,v1:172.21.15.195:6809/2322632437]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T00:10:35.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:35 smithi107 ceph-mon[29153]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T00:10:35.750 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:35 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:35.750 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:35 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:35.750 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:35 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:35.750 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:35 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:35.750 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:35 smithi195 ceph-mon[32833]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:35.750 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:35 smithi195 ceph-mon[32833]: from='osd.5 [v2:172.21.15.195:6808/2322632437,v1:172.21.15.195:6809/2322632437]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T00:10:35.750 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:35 smithi195 ceph-mon[32833]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T00:10:35.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:35 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:35.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:35 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:35.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:35 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:35.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:35 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:35.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:35 smithi170 ceph-mon[32807]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:35.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:35 smithi170 ceph-mon[32807]: from='osd.5 [v2:172.21.15.195:6808/2322632437,v1:172.21.15.195:6809/2322632437]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T00:10:35.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:35 smithi170 ceph-mon[32807]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T00:10:37.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:36 smithi195 ceph-mon[32833]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T00:10:37.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:36 smithi195 ceph-mon[32833]: from='osd.5 [v2:172.21.15.195:6808/2322632437,v1:172.21.15.195:6809/2322632437]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-04-25T00:10:37.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:36 smithi195 ceph-mon[32833]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T00:10:37.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:36 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:10:37.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:36 smithi195 ceph-mon[32833]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-04-25T00:10:37.090 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:10:36 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:10:36.832+0000 7f7e00027700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T00:10:37.090 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:10:36 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:10:36.840+0000 7f7dfbe1a700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:10:37.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:36 smithi107 ceph-mon[29153]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T00:10:37.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:36 smithi107 ceph-mon[29153]: from='osd.5 [v2:172.21.15.195:6808/2322632437,v1:172.21.15.195:6809/2322632437]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-04-25T00:10:37.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:36 smithi107 ceph-mon[29153]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T00:10:37.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:36 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:10:37.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:36 smithi107 ceph-mon[29153]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-04-25T00:10:37.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:36 smithi170 ceph-mon[32807]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T00:10:37.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:36 smithi170 ceph-mon[32807]: from='osd.5 [v2:172.21.15.195:6808/2322632437,v1:172.21.15.195:6809/2322632437]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-04-25T00:10:37.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:36 smithi170 ceph-mon[32807]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T00:10:37.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:36 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:10:37.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:36 smithi170 ceph-mon[32807]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-04-25T00:10:37.497 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T00:10:37.497 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T00:10:37.497 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-04-25T00:10:37.497 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-04-25T00:10:37.497 INFO:teuthology.orchestra.run.smithi107.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0134582 s, 779 MB/s 2024-04-25T00:10:37.497 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-04-25T00:10:38.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:37 smithi195 ceph-mon[32833]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-04-25T00:10:38.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:37 smithi195 ceph-mon[32833]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T00:10:38.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:37 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:10:38.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:37 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:10:38.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:37 smithi195 ceph-mon[32833]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:38.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:37 smithi195 ceph-mon[32833]: osd.5 [v2:172.21.15.195:6808/2322632437,v1:172.21.15.195:6809/2322632437] boot 2024-04-25T00:10:38.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:37 smithi195 ceph-mon[32833]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T00:10:38.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:37 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:10:38.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:37 smithi170 ceph-mon[32807]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-04-25T00:10:38.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:37 smithi170 ceph-mon[32807]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T00:10:38.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:37 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:10:38.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:37 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:10:38.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:37 smithi170 ceph-mon[32807]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:38.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:37 smithi170 ceph-mon[32807]: osd.5 [v2:172.21.15.195:6808/2322632437,v1:172.21.15.195:6809/2322632437] boot 2024-04-25T00:10:38.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:37 smithi170 ceph-mon[32807]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T00:10:38.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:37 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:10:38.191 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:37 smithi107 ceph-mon[29153]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-04-25T00:10:38.191 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:37 smithi107 ceph-mon[29153]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T00:10:38.191 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:37 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:10:38.191 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:37 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:10:38.191 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:37 smithi107 ceph-mon[29153]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:10:38.192 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:37 smithi107 ceph-mon[29153]: osd.5 [v2:172.21.15.195:6808/2322632437,v1:172.21.15.195:6809/2322632437] boot 2024-04-25T00:10:38.192 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:37 smithi107 ceph-mon[29153]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T00:10:38.192 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:37 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:10:38.219 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_2 2024-04-25T00:10:38.468 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:10:39.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:38 smithi170 ceph-mon[32807]: purged_snaps scrub starts 2024-04-25T00:10:39.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:38 smithi170 ceph-mon[32807]: purged_snaps scrub ok 2024-04-25T00:10:39.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:38 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:39.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:38 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:39.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:38 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:10:39.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:38 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:10:39.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:38 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:39.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:38 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:39.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:38 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:39.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:38 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:39.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:38 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:39.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:38 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:39.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:38 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:39.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:38 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:39.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:38 smithi170 ceph-mon[32807]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T00:10:39.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:38 smithi107 ceph-mon[29153]: purged_snaps scrub starts 2024-04-25T00:10:39.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:38 smithi107 ceph-mon[29153]: purged_snaps scrub ok 2024-04-25T00:10:39.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:38 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:39.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:38 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:39.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:38 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:10:39.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:38 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:10:39.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:38 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:39.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:38 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:39.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:38 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:39.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:38 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:39.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:38 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:39.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:38 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:39.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:38 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:39.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:38 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:39.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:38 smithi107 ceph-mon[29153]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T00:10:39.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:38 smithi195 ceph-mon[32833]: purged_snaps scrub starts 2024-04-25T00:10:39.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:38 smithi195 ceph-mon[32833]: purged_snaps scrub ok 2024-04-25T00:10:39.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:38 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:39.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:38 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:39.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:38 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:10:39.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:38 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:10:39.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:38 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:39.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:38 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:39.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:38 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:39.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:38 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:39.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:38 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:39.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:38 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:39.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:38 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:39.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:38 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:39.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:38 smithi195 ceph-mon[32833]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T00:10:40.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.064+0000 7fddd62ae700 1 -- 172.21.15.107:0/3330878997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddd010e8a0 msgr2=0x7fddd0110c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:40.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.064+0000 7fddd62ae700 1 --2- 172.21.15.107:0/3330878997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddd010e8a0 0x7fddd0110c90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fddc800b0a0 tx=0x7fddc800b3b0 comp rx=0 tx=0).stop 2024-04-25T00:10:40.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.064+0000 7fddd62ae700 1 -- 172.21.15.107:0/3330878997 shutdown_connections 2024-04-25T00:10:40.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.064+0000 7fddd62ae700 1 --2- 172.21.15.107:0/3330878997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddd010e8a0 0x7fddd0110c90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:40.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.064+0000 7fddd62ae700 1 --2- 172.21.15.107:0/3330878997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fddd0076260 0x7fddd00746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:40.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.064+0000 7fddd62ae700 1 --2- 172.21.15.107:0/3330878997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddd0074c90 0x7fddd0075110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:40.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.064+0000 7fddd62ae700 1 -- 172.21.15.107:0/3330878997 >> 172.21.15.107:0/3330878997 conn(0x7fddd00fd880 msgr2=0x7fddd00ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:40.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.064+0000 7fddd62ae700 1 -- 172.21.15.107:0/3330878997 shutdown_connections 2024-04-25T00:10:40.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.064+0000 7fddd62ae700 1 -- 172.21.15.107:0/3330878997 wait complete. 2024-04-25T00:10:40.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.064+0000 7fddd62ae700 1 Processor -- start 2024-04-25T00:10:40.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.064+0000 7fddd62ae700 1 -- start start 2024-04-25T00:10:40.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.065+0000 7fddd62ae700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddd0074c90 0x7fddd019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:40.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.065+0000 7fddd62ae700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fddd0076260 0x7fddd019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:40.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.065+0000 7fddd62ae700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddd010e8a0 0x7fddd01a5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:40.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.065+0000 7fddd62ae700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fddd01144a0 con 0x7fddd0074c90 2024-04-25T00:10:40.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.065+0000 7fddd62ae700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fddd0114320 con 0x7fddd010e8a0 2024-04-25T00:10:40.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.065+0000 7fddd62ae700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fddd0114620 con 0x7fddd0076260 2024-04-25T00:10:40.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.065+0000 7fddcffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddd0074c90 0x7fddd019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:40.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.065+0000 7fddcffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddd0074c90 0x7fddd019e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46068/0 (socket says 172.21.15.107:46068) 2024-04-25T00:10:40.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.065+0000 7fddcffff700 1 -- 172.21.15.107:0/2501390871 learned_addr learned my addr 172.21.15.107:0/2501390871 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:10:40.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.065+0000 7fddcf7fe700 1 --2- 172.21.15.107:0/2501390871 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fddd0076260 0x7fddd019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:40.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.065+0000 7fddd484b700 1 --2- 172.21.15.107:0/2501390871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddd010e8a0 0x7fddd01a5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:40.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.065+0000 7fddcffff700 1 -- 172.21.15.107:0/2501390871 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fddd0076260 msgr2=0x7fddd019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:40.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.065+0000 7fddcffff700 1 --2- 172.21.15.107:0/2501390871 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fddd0076260 0x7fddd019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:40.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.065+0000 7fddcffff700 1 -- 172.21.15.107:0/2501390871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddd010e8a0 msgr2=0x7fddd01a5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:40.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.065+0000 7fddcffff700 1 --2- 172.21.15.107:0/2501390871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddd010e8a0 0x7fddd01a5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:40.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.065+0000 7fddcffff700 1 -- 172.21.15.107:0/2501390871 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd01a65f0 con 0x7fddd0074c90 2024-04-25T00:10:40.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.065+0000 7fddd484b700 1 --2- 172.21.15.107:0/2501390871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddd010e8a0 0x7fddd01a5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:40.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.065+0000 7fddcffff700 1 --2- 172.21.15.107:0/2501390871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddd0074c90 0x7fddd019e990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fddc000ca10 tx=0x7fddc000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:40.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.065+0000 7fddcd7fa700 1 -- 172.21.15.107:0/2501390871 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0007a60 con 0x7fddd0074c90 2024-04-25T00:10:40.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.065+0000 7fddd62ae700 1 -- 172.21.15.107:0/2501390871 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd01a68e0 con 0x7fddd0074c90 2024-04-25T00:10:40.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.065+0000 7fddcd7fa700 1 -- 172.21.15.107:0/2501390871 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddc0007c00 con 0x7fddd0074c90 2024-04-25T00:10:40.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.065+0000 7fddcd7fa700 1 -- 172.21.15.107:0/2501390871 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0018660 con 0x7fddd0074c90 2024-04-25T00:10:40.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.065+0000 7fddd62ae700 1 -- 172.21.15.107:0/2501390871 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd01a6e20 con 0x7fddd0074c90 2024-04-25T00:10:40.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.066+0000 7fddd62ae700 1 -- 172.21.15.107:0/2501390871 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd0068f20 con 0x7fddd0074c90 2024-04-25T00:10:40.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.068+0000 7fddcd7fa700 1 -- 172.21.15.107:0/2501390871 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fddc000f450 con 0x7fddd0074c90 2024-04-25T00:10:40.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.068+0000 7fddcd7fa700 1 --2- 172.21.15.107:0/2501390871 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fddb80a2c20 0x7fddb80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:40.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.068+0000 7fddcf7fe700 1 --2- 172.21.15.107:0/2501390871 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fddb80a2c20 0x7fddb80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:40.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.068+0000 7fddcd7fa700 1 -- 172.21.15.107:0/2501390871 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7fddc00ce400 con 0x7fddd0074c90 2024-04-25T00:10:40.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.069+0000 7fddcf7fe700 1 --2- 172.21.15.107:0/2501390871 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fddb80a2c20 0x7fddb80a50e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fddd0075f90 tx=0x7fddc400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:40.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.072+0000 7fddcd7fa700 1 -- 172.21.15.107:0/2501390871 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fddc0014030 con 0x7fddd0074c90 2024-04-25T00:10:40.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:39 smithi170 ceph-mon[32807]: Detected new or changed devices on smithi195 2024-04-25T00:10:40.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:39 smithi170 ceph-mon[32807]: Adjusting osd_memory_target on smithi195 to 8544M 2024-04-25T00:10:40.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:39 smithi170 ceph-mon[32807]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:10:40.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:39 smithi170 ceph-mon[32807]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T00:10:40.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:39 smithi107 ceph-mon[29153]: Detected new or changed devices on smithi195 2024-04-25T00:10:40.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:39 smithi107 ceph-mon[29153]: Adjusting osd_memory_target on smithi195 to 8544M 2024-04-25T00:10:40.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:39 smithi107 ceph-mon[29153]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:10:40.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:39 smithi107 ceph-mon[29153]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T00:10:40.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:10:40.279+0000 7fddd62ae700 1 -- 172.21.15.107:0/2501390871 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fddd0062670 con 0x7fddb80a2c20 2024-04-25T00:10:40.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:39 smithi195 ceph-mon[32833]: Detected new or changed devices on smithi195 2024-04-25T00:10:40.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:39 smithi195 ceph-mon[32833]: Adjusting osd_memory_target on smithi195 to 8544M 2024-04-25T00:10:40.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:39 smithi195 ceph-mon[32833]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:10:40.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:39 smithi195 ceph-mon[32833]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T00:10:41.103 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:40 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:10:41.104 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:40 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:10:41.104 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:40 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:41.130 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:40 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:10:41.131 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:40 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:10:41.131 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:40 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:41.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:40 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:10:41.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:40 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:10:41.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:40 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:42.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:41 smithi170 ceph-mon[32807]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:10:42.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:41 smithi170 ceph-mon[32807]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T00:10:42.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:41 smithi170 ceph-mon[32807]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:10:42.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:41 smithi107 ceph-mon[29153]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:10:42.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:41 smithi107 ceph-mon[29153]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T00:10:42.222 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:41 smithi107 ceph-mon[29153]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:10:42.240 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:41 smithi195 ceph-mon[32833]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:10:42.240 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:41 smithi195 ceph-mon[32833]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T00:10:42.240 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:41 smithi195 ceph-mon[32833]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:10:43.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:42 smithi170 ceph-mon[32807]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T00:10:43.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:42 smithi107 ceph-mon[29153]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T00:10:43.251 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:42 smithi195 ceph-mon[32833]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T00:10:44.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:43 smithi170 ceph-mon[32807]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-04-25T00:10:44.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:43 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/723335135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "543e0a0a-23fc-402f-8a71-3bcd276999db"}]: dispatch 2024-04-25T00:10:44.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:43 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/723335135' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "543e0a0a-23fc-402f-8a71-3bcd276999db"}]': finished 2024-04-25T00:10:44.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:43 smithi170 ceph-mon[32807]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T00:10:44.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:43 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:10:44.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:43 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1928283006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:10:44.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:43 smithi107 ceph-mon[29153]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-04-25T00:10:44.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:43 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/723335135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "543e0a0a-23fc-402f-8a71-3bcd276999db"}]: dispatch 2024-04-25T00:10:44.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:43 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/723335135' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "543e0a0a-23fc-402f-8a71-3bcd276999db"}]': finished 2024-04-25T00:10:44.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:43 smithi107 ceph-mon[29153]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T00:10:44.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:43 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:10:44.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:43 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1928283006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:10:44.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:43 smithi195 ceph-mon[32833]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-04-25T00:10:44.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:43 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/723335135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "543e0a0a-23fc-402f-8a71-3bcd276999db"}]: dispatch 2024-04-25T00:10:44.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:43 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/723335135' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "543e0a0a-23fc-402f-8a71-3bcd276999db"}]': finished 2024-04-25T00:10:44.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:43 smithi195 ceph-mon[32833]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T00:10:44.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:43 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:10:44.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:43 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1928283006' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:10:46.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:45 smithi170 ceph-mon[32807]: pgmap v146: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-04-25T00:10:46.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:45 smithi107 ceph-mon[29153]: pgmap v146: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-04-25T00:10:46.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:45 smithi195 ceph-mon[32833]: pgmap v146: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail; 114 KiB/s, 0 objects/s recovering 2024-04-25T00:10:48.205 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:47 smithi107 ceph-mon[29153]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:10:48.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:47 smithi195 ceph-mon[32833]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:10:48.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:47 smithi170 ceph-mon[32807]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:10:50.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:49 smithi107 ceph-mon[29153]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:10:50.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:49 smithi195 ceph-mon[32833]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:10:50.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:49 smithi170 ceph-mon[32807]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:10:51.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T00:10:51.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:51.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T00:10:51.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:51.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:50 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T00:10:51.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:50 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:52.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:51 smithi170 ceph-mon[32807]: Deploying daemon osd.6 on smithi107 2024-04-25T00:10:52.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:51 smithi170 ceph-mon[32807]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:10:52.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:51 smithi107 ceph-mon[29153]: Deploying daemon osd.6 on smithi107 2024-04-25T00:10:52.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:51 smithi107 ceph-mon[29153]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:10:52.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:51 smithi195 ceph-mon[32833]: Deploying daemon osd.6 on smithi107 2024-04-25T00:10:52.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:51 smithi195 ceph-mon[32833]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:10:54.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:53 smithi107 ceph-mon[29153]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:10:54.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:53 smithi195 ceph-mon[32833]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:10:54.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:53 smithi170 ceph-mon[32807]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:10:56.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:55 smithi107 ceph-mon[29153]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:10:56.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:55 smithi195 ceph-mon[32833]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:10:56.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:55 smithi170 ceph-mon[32807]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:10:57.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:57 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:57.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:57 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:57.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:57 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:57.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:57 smithi195 ceph-mon[32833]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:10:57.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:57 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:57.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:57 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:57.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:57 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:57.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:57 smithi170 ceph-mon[32807]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:10:57.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:57 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:57.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:57 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:10:57.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:57 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:57.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:57 smithi107 ceph-mon[29153]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:10:59.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:10:59 smithi107 ceph-mon[29153]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:11:00.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:00.039+0000 7fddcd7fa700 1 -- 172.21.15.107:0/2501390871 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fddd0062670 con 0x7fddb80a2c20 2024-04-25T00:11:00.039 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 6 on host 'smithi107' 2024-04-25T00:11:00.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:00.042+0000 7fddd62ae700 1 -- 172.21.15.107:0/2501390871 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fddb80a2c20 msgr2=0x7fddb80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:00.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:00.042+0000 7fddd62ae700 1 --2- 172.21.15.107:0/2501390871 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fddb80a2c20 0x7fddb80a50e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fddd0075f90 tx=0x7fddc400a380 comp rx=0 tx=0).stop 2024-04-25T00:11:00.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:00.042+0000 7fddd62ae700 1 -- 172.21.15.107:0/2501390871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddd0074c90 msgr2=0x7fddd019e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:00.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:00.042+0000 7fddd62ae700 1 --2- 172.21.15.107:0/2501390871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddd0074c90 0x7fddd019e990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fddc000ca10 tx=0x7fddc000cd20 comp rx=0 tx=0).stop 2024-04-25T00:11:00.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:00.042+0000 7fddd62ae700 1 -- 172.21.15.107:0/2501390871 shutdown_connections 2024-04-25T00:11:00.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:00.042+0000 7fddd62ae700 1 --2- 172.21.15.107:0/2501390871 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fddb80a2c20 0x7fddb80a50e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:00.042+0000 7fddd62ae700 1 --2- 172.21.15.107:0/2501390871 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fddd010e8a0 0x7fddd01a5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:00.042+0000 7fddd62ae700 1 --2- 172.21.15.107:0/2501390871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddd0074c90 0x7fddd019e990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:00.042+0000 7fddd62ae700 1 --2- 172.21.15.107:0/2501390871 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fddd0076260 0x7fddd019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:00.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:00.042+0000 7fddd62ae700 1 -- 172.21.15.107:0/2501390871 >> 172.21.15.107:0/2501390871 conn(0x7fddd00fd880 msgr2=0x7fddd0101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:00.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:00.042+0000 7fddd62ae700 1 -- 172.21.15.107:0/2501390871 shutdown_connections 2024-04-25T00:11:00.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:00.043+0000 7fddd62ae700 1 -- 172.21.15.107:0/2501390871 wait complete. 2024-04-25T00:11:00.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:10:59 smithi195 ceph-mon[32833]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:11:00.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:10:59 smithi170 ceph-mon[32807]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:11:00.777 DEBUG:teuthology.orchestra.run.smithi107:osd.6> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T00:11:00.781 INFO:tasks.cephadm:Deploying osd.7 on smithi170 with /dev/vg_nvme/lv_2... 2024-04-25T00:11:00.782 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T00:11:01.027 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:11:01.150 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:01.150 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:01.151 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:01 smithi170 ceph-mon[32807]: from='osd.6 [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T00:11:01.151 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:01.151 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:01.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:01.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:01.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:01 smithi195 ceph-mon[32833]: from='osd.6 [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T00:11:01.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:01.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:01.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:01.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:01.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:01 smithi107 ceph-mon[29153]: from='osd.6 [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T00:11:01.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:01.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:02.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:02 smithi107 ceph-mon[29153]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:11:02.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:02 smithi107 ceph-mon[29153]: from='osd.6 [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T00:11:02.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:02 smithi107 ceph-mon[29153]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T00:11:02.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:02 smithi107 ceph-mon[29153]: from='osd.6 [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-25T00:11:02.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:11:02.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:02 smithi195 ceph-mon[32833]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:11:02.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:02 smithi195 ceph-mon[32833]: from='osd.6 [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T00:11:02.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:02 smithi195 ceph-mon[32833]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T00:11:02.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:02 smithi195 ceph-mon[32833]: from='osd.6 [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-25T00:11:02.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:11:02.380 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:02 smithi170 ceph-mon[32807]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:11:02.380 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:02 smithi170 ceph-mon[32807]: from='osd.6 [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T00:11:02.380 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:02 smithi170 ceph-mon[32807]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T00:11:02.381 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:02 smithi170 ceph-mon[32807]: from='osd.6 [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-25T00:11:02.381 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:11:02.877 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:11:02 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:11:02.757+0000 7f8ceb48c700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T00:11:02.878 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:11:02 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:11:02.766+0000 7f8ce3277700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:11:03.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:03 smithi107 ceph-mon[29153]: from='osd.6 [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-25T00:11:03.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:03 smithi107 ceph-mon[29153]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T00:11:03.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:03 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:11:03.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:03 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:11:03.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:03 smithi107 ceph-mon[29153]: from='osd.6 [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415]' entity='osd.6' 2024-04-25T00:11:03.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:03 smithi107 ceph-mon[29153]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:11:03.273 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:03 smithi170 ceph-mon[32807]: from='osd.6 [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-25T00:11:03.273 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:03 smithi170 ceph-mon[32807]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T00:11:03.273 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:03 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:11:03.273 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:03 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:11:03.273 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:03 smithi170 ceph-mon[32807]: from='osd.6 [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415]' entity='osd.6' 2024-04-25T00:11:03.273 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:03 smithi170 ceph-mon[32807]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:11:03.274 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T00:11:03.274 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T00:11:03.274 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-04-25T00:11:03.274 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-04-25T00:11:03.274 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00852939 s, 1.2 GB/s 2024-04-25T00:11:03.274 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-04-25T00:11:03.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:03 smithi195 ceph-mon[32833]: from='osd.6 [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-25T00:11:03.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:03 smithi195 ceph-mon[32833]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T00:11:03.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:03 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:11:03.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:03 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:11:03.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:03 smithi195 ceph-mon[32833]: from='osd.6 [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415]' entity='osd.6' 2024-04-25T00:11:03.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:03 smithi195 ceph-mon[32833]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:11:04.024 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi170:vg_nvme/lv_2 2024-04-25T00:11:04.267 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:11:04.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:04 smithi195 ceph-mon[32833]: purged_snaps scrub starts 2024-04-25T00:11:04.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:04 smithi195 ceph-mon[32833]: purged_snaps scrub ok 2024-04-25T00:11:04.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:04 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:11:04.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:04 smithi195 ceph-mon[32833]: osd.6 [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415] boot 2024-04-25T00:11:04.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:04 smithi195 ceph-mon[32833]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T00:11:04.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:04 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:11:04.388 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:04 smithi170 ceph-mon[32807]: purged_snaps scrub starts 2024-04-25T00:11:04.389 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:04 smithi170 ceph-mon[32807]: purged_snaps scrub ok 2024-04-25T00:11:04.389 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:04 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:11:04.389 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:04 smithi170 ceph-mon[32807]: osd.6 [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415] boot 2024-04-25T00:11:04.389 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:04 smithi170 ceph-mon[32807]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T00:11:04.389 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:04 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:11:04.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:04 smithi107 ceph-mon[29153]: purged_snaps scrub starts 2024-04-25T00:11:04.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:04 smithi107 ceph-mon[29153]: purged_snaps scrub ok 2024-04-25T00:11:04.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:04 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:11:04.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:04 smithi107 ceph-mon[29153]: osd.6 [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415] boot 2024-04-25T00:11:04.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:04 smithi107 ceph-mon[29153]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T00:11:04.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:04 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:11:05.903 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.901+0000 7f38847bb700 1 -- 172.21.15.170:0/4278328146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f387c101180 msgr2=0x7f387c1015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.901+0000 7f38847bb700 1 --2- 172.21.15.170:0/4278328146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f387c101180 0x7f387c1015a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f386c0099c0 tx=0x7f386c009cd0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.901+0000 7f38847bb700 1 -- 172.21.15.170:0/4278328146 shutdown_connections 2024-04-25T00:11:05.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.901+0000 7f38847bb700 1 --2- 172.21.15.170:0/4278328146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f387c102d20 0x7f387c1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.901+0000 7f38847bb700 1 --2- 172.21.15.170:0/4278328146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f387c101180 0x7f387c1015a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.901+0000 7f38847bb700 1 --2- 172.21.15.170:0/4278328146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f387c102380 0x7f387c1027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.901+0000 7f38847bb700 1 -- 172.21.15.170:0/4278328146 >> 172.21.15.170:0/4278328146 conn(0x7f387c0fc720 msgr2=0x7f387c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:05.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.901+0000 7f38847bb700 1 -- 172.21.15.170:0/4278328146 shutdown_connections 2024-04-25T00:11:05.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.901+0000 7f38847bb700 1 -- 172.21.15.170:0/4278328146 wait complete. 2024-04-25T00:11:05.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.902+0000 7f38847bb700 1 Processor -- start 2024-04-25T00:11:05.904 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.902+0000 7f38847bb700 1 -- start start 2024-04-25T00:11:05.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.902+0000 7f38847bb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f387c101180 0x7f387c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.902+0000 7f38847bb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f387c102380 0x7f387c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.902+0000 7f38847bb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f387c102d20 0x7f387c19fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.902+0000 7f38847bb700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f387c10cd40 con 0x7f387c101180 2024-04-25T00:11:05.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.902+0000 7f38847bb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f387c10cbc0 con 0x7f387c102d20 2024-04-25T00:11:05.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.902+0000 7f38847bb700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f387c10cec0 con 0x7f387c102380 2024-04-25T00:11:05.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.902+0000 7f3882d58700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f387c102d20 0x7f387c19fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.902+0000 7f3882d58700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f387c102d20 0x7f387c19fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:50648/0 (socket says 172.21.15.170:50648) 2024-04-25T00:11:05.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.902+0000 7f3882d58700 1 -- 172.21.15.170:0/4134275180 learned_addr learned my addr 172.21.15.170:0/4134275180 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-04-25T00:11:05.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.902+0000 7f3881d56700 1 --2- 172.21.15.170:0/4134275180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f387c102380 0x7f387c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.902+0000 7f3882d58700 1 -- 172.21.15.170:0/4134275180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f387c102380 msgr2=0x7f387c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.902+0000 7f3882d58700 1 --2- 172.21.15.170:0/4134275180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f387c102380 0x7f387c1989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.902+0000 7f3882d58700 1 -- 172.21.15.170:0/4134275180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f387c101180 msgr2=0x7f387c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:05.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.902+0000 7f3882d58700 1 --2- 172.21.15.170:0/4134275180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f387c101180 0x7f387c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:05.905 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.902+0000 7f3882d58700 1 -- 172.21.15.170:0/4134275180 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f387c1a00c0 con 0x7f387c102d20 2024-04-25T00:11:05.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.903+0000 7f3882d58700 1 --2- 172.21.15.170:0/4134275180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f387c102d20 0x7f387c19fa20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3874007ee0 tx=0x7f387400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:05.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.903+0000 7f38737fe700 1 -- 172.21.15.170:0/4134275180 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387400dbd0 con 0x7f387c102d20 2024-04-25T00:11:05.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.903+0000 7f38737fe700 1 -- 172.21.15.170:0/4134275180 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f387400dd70 con 0x7f387c102d20 2024-04-25T00:11:05.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.903+0000 7f38847bb700 1 -- 172.21.15.170:0/4134275180 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f387c1a03b0 con 0x7f387c102d20 2024-04-25T00:11:05.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.903+0000 7f38737fe700 1 -- 172.21.15.170:0/4134275180 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38740133f0 con 0x7f387c102d20 2024-04-25T00:11:05.906 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.903+0000 7f38847bb700 1 -- 172.21.15.170:0/4134275180 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f387c1a08f0 con 0x7f387c102d20 2024-04-25T00:11:05.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.904+0000 7f38847bb700 1 -- 172.21.15.170:0/4134275180 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f387c068f20 con 0x7f387c102d20 2024-04-25T00:11:05.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.905+0000 7f38737fe700 1 -- 172.21.15.170:0/4134275180 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f387401a020 con 0x7f387c102d20 2024-04-25T00:11:05.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.906+0000 7f38737fe700 1 --2- 172.21.15.170:0/4134275180 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f38680a2b00 0x7f38680a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:05.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.906+0000 7f38737fe700 1 -- 172.21.15.170:0/4134275180 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f38740cd330 con 0x7f387c102d20 2024-04-25T00:11:05.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.906+0000 7f3882557700 1 --2- 172.21.15.170:0/4134275180 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f38680a2b00 0x7f38680a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:05.907 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.906+0000 7f3882557700 1 --2- 172.21.15.170:0/4134275180 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f38680a2b00 0x7f38680a4fc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f386c0098e0 tx=0x7f386c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:05.911 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:05.910+0000 7f38737fe700 1 -- 172.21.15.170:0/4134275180 <== mon.2 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3874010070 con 0x7f387c102d20 2024-04-25T00:11:05.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:05 smithi170 ceph-mon[32807]: Detected new or changed devices on smithi107 2024-04-25T00:11:05.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:05 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:05.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:05 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:05.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:05 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:05.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:05 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:05.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:05 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:05.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:05 smithi170 ceph-mon[32807]: Adjusting osd_memory_target on smithi107 to 5696M 2024-04-25T00:11:05.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:05 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:05.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:05 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:05.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:05 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:05.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:05 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:05.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:05 smithi170 ceph-mon[32807]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:05.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:05 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:05.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:05 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:05.936 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:05 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:05.936 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:05 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:05.936 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:05 smithi170 ceph-mon[32807]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T00:11:05.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:05 smithi107 ceph-mon[29153]: Detected new or changed devices on smithi107 2024-04-25T00:11:05.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:05 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:05.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:05 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:05.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:05 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:05.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:05 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:05.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:05 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:05.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:05 smithi107 ceph-mon[29153]: Adjusting osd_memory_target on smithi107 to 5696M 2024-04-25T00:11:05.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:05 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:05.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:05 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:05.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:05 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:05.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:05 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:05.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:05 smithi107 ceph-mon[29153]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:05.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:05 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:05.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:05 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:05.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:05 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:05.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:05 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:05.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:05 smithi107 ceph-mon[29153]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T00:11:06.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:05 smithi195 ceph-mon[32833]: Detected new or changed devices on smithi107 2024-04-25T00:11:06.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:05 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:06.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:05 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:06.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:05 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:06.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:05 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:06.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:05 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:06.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:05 smithi195 ceph-mon[32833]: Adjusting osd_memory_target on smithi107 to 5696M 2024-04-25T00:11:06.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:05 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:06.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:05 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:06.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:05 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:06.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:05 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:06.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:05 smithi195 ceph-mon[32833]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:06.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:05 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:06.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:05 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:06.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:05 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:06.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:05 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:06.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:05 smithi195 ceph-mon[32833]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T00:11:06.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:06.112+0000 7f38847bb700 1 -- 172.21.15.170:0/4134275180 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f387c0fe8e0 con 0x7f38680a2b00 2024-04-25T00:11:06.960 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:06 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:11:06.960 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:06 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:11:06.961 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:06 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:06.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:06 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:11:06.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:06 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:11:06.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:06 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:07.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:06 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:11:07.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:06 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:11:07.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:06 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:07.965 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:07 smithi170 ceph-mon[32807]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:11:07.966 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:07 smithi170 ceph-mon[32807]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:07.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:07 smithi107 ceph-mon[29153]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:11:07.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:07 smithi107 ceph-mon[29153]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:08.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:07 smithi195 ceph-mon[32833]: from='client.24284 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:11:08.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:07 smithi195 ceph-mon[32833]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:09.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:08 smithi107 ceph-mon[29153]: from='client.? 172.21.15.170:0/525786890' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa97be8a-73cd-4452-b143-825370290232"}]: dispatch 2024-04-25T00:11:09.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:08 smithi107 ceph-mon[29153]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa97be8a-73cd-4452-b143-825370290232"}]: dispatch 2024-04-25T00:11:09.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:08 smithi107 ceph-mon[29153]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa97be8a-73cd-4452-b143-825370290232"}]': finished 2024-04-25T00:11:09.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:08 smithi107 ceph-mon[29153]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T00:11:09.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:08 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:11:09.257 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:08 smithi170 ceph-mon[32807]: from='client.? 172.21.15.170:0/525786890' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa97be8a-73cd-4452-b143-825370290232"}]: dispatch 2024-04-25T00:11:09.257 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:08 smithi170 ceph-mon[32807]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa97be8a-73cd-4452-b143-825370290232"}]: dispatch 2024-04-25T00:11:09.257 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:08 smithi170 ceph-mon[32807]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa97be8a-73cd-4452-b143-825370290232"}]': finished 2024-04-25T00:11:09.257 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:08 smithi170 ceph-mon[32807]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T00:11:09.257 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:08 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:11:09.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:08 smithi195 ceph-mon[32833]: from='client.? 172.21.15.170:0/525786890' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa97be8a-73cd-4452-b143-825370290232"}]: dispatch 2024-04-25T00:11:09.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:08 smithi195 ceph-mon[32833]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa97be8a-73cd-4452-b143-825370290232"}]: dispatch 2024-04-25T00:11:09.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:08 smithi195 ceph-mon[32833]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa97be8a-73cd-4452-b143-825370290232"}]': finished 2024-04-25T00:11:09.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:08 smithi195 ceph-mon[32833]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T00:11:09.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:08 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:11:10.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:09 smithi107 ceph-mon[29153]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:10.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:09 smithi107 ceph-mon[29153]: from='client.? 172.21.15.170:0/4078863399' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:11:10.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:09 smithi195 ceph-mon[32833]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:10.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:09 smithi195 ceph-mon[32833]: from='client.? 172.21.15.170:0/4078863399' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:11:10.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:09 smithi170 ceph-mon[32807]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:10.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:09 smithi170 ceph-mon[32807]: from='client.? 172.21.15.170:0/4078863399' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:11:12.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:11 smithi107 ceph-mon[29153]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:12.237 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:11 smithi170 ceph-mon[32807]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:12.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:11 smithi195 ceph-mon[32833]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:14.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:13 smithi170 ceph-mon[32807]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:14.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:13 smithi107 ceph-mon[29153]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:14.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:13 smithi195 ceph-mon[32833]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:16.164 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:15 smithi170 ceph-mon[32807]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:16.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:15 smithi107 ceph-mon[29153]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:16.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:15 smithi195 ceph-mon[32833]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:17.134 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:16 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T00:11:17.134 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:16 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:17.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:16 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T00:11:17.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:16 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:17.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:16 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T00:11:17.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:16 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:18.091 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:17 smithi170 ceph-mon[32807]: Deploying daemon osd.7 on smithi170 2024-04-25T00:11:18.092 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:17 smithi170 ceph-mon[32807]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:18.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:17 smithi107 ceph-mon[29153]: Deploying daemon osd.7 on smithi170 2024-04-25T00:11:18.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:17 smithi107 ceph-mon[29153]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:18.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:17 smithi195 ceph-mon[32833]: Deploying daemon osd.7 on smithi170 2024-04-25T00:11:18.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:17 smithi195 ceph-mon[32833]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:20.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:19 smithi107 ceph-mon[29153]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:20.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:19 smithi195 ceph-mon[32833]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:20.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:19 smithi170 ceph-mon[32807]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:22.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:21 smithi107 ceph-mon[29153]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:22.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:21 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:22.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:21 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:22.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:21 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:22.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:21 smithi195 ceph-mon[32833]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:22.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:21 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:22.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:21 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:22.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:21 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:22.421 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:21 smithi170 ceph-mon[32807]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:22.422 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:21 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:22.422 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:21 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:22.422 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:21 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:24.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:23 smithi170 ceph-mon[32807]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:24.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:23 smithi107 ceph-mon[29153]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:24.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:23 smithi195 ceph-mon[32833]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:25.678 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 7 on host 'smithi170' 2024-04-25T00:11:25.678 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:25.677+0000 7f38737fe700 1 -- 172.21.15.170:0/4134275180 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f387c0fe8e0 con 0x7f38680a2b00 2024-04-25T00:11:25.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:25.679+0000 7f38847bb700 1 -- 172.21.15.170:0/4134275180 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f38680a2b00 msgr2=0x7f38680a4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:25.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:25.679+0000 7f38847bb700 1 --2- 172.21.15.170:0/4134275180 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f38680a2b00 0x7f38680a4fc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f386c0098e0 tx=0x7f386c000bc0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:25.679+0000 7f38847bb700 1 -- 172.21.15.170:0/4134275180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f387c102d20 msgr2=0x7f387c19fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:25.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:25.679+0000 7f38847bb700 1 --2- 172.21.15.170:0/4134275180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f387c102d20 0x7f387c19fa20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3874007ee0 tx=0x7f387400d3b0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:25.679+0000 7f38847bb700 1 -- 172.21.15.170:0/4134275180 shutdown_connections 2024-04-25T00:11:25.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:25.679+0000 7f38847bb700 1 --2- 172.21.15.170:0/4134275180 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f38680a2b00 0x7f38680a4fc0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:25.679+0000 7f38847bb700 1 --2- 172.21.15.170:0/4134275180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f387c102d20 0x7f387c19fa20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:25.679+0000 7f38847bb700 1 --2- 172.21.15.170:0/4134275180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f387c101180 0x7f387c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.680 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:25.679+0000 7f38847bb700 1 --2- 172.21.15.170:0/4134275180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f387c102380 0x7f387c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:25.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:25.679+0000 7f38847bb700 1 -- 172.21.15.170:0/4134275180 >> 172.21.15.170:0/4134275180 conn(0x7f387c0fc720 msgr2=0x7f387c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:25.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:25.679+0000 7f38847bb700 1 -- 172.21.15.170:0/4134275180 shutdown_connections 2024-04-25T00:11:25.681 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:11:25.679+0000 7f38847bb700 1 -- 172.21.15.170:0/4134275180 wait complete. 2024-04-25T00:11:25.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:25 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:25.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:25 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:25.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:25 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:25.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:25 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:25.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:25 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:25.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:25 smithi170 ceph-mon[32807]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:25.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:25 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:25.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:25 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:25.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:25 smithi170 ceph-mon[32807]: from='osd.7 [v2:172.21.15.170:6816/3231152590,v1:172.21.15.170:6817/3231152590]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T00:11:25.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:25 smithi170 ceph-mon[32807]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T00:11:25.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:25 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:26.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:25 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:26.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:25 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:26.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:25 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:26.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:25 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:26.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:25 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:26.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:25 smithi195 ceph-mon[32833]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:26.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:25 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:26.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:25 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:26.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:25 smithi195 ceph-mon[32833]: from='osd.7 [v2:172.21.15.170:6816/3231152590,v1:172.21.15.170:6817/3231152590]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T00:11:26.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:25 smithi195 ceph-mon[32833]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T00:11:26.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:25 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:26.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:25 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:26.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:25 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:26.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:25 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:26.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:25 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:26.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:25 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:26.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:25 smithi107 ceph-mon[29153]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:26.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:25 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:26.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:25 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:26.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:25 smithi107 ceph-mon[29153]: from='osd.7 [v2:172.21.15.170:6816/3231152590,v1:172.21.15.170:6817/3231152590]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T00:11:26.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:25 smithi107 ceph-mon[29153]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T00:11:26.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:25 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:26.399 DEBUG:teuthology.orchestra.run.smithi170:osd.7> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T00:11:26.402 INFO:tasks.cephadm:Deploying osd.8 on smithi195 with /dev/vg_nvme/lv_2... 2024-04-25T00:11:26.402 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T00:11:26.648 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:11:27.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:27 smithi170 ceph-mon[32807]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T00:11:27.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:27 smithi170 ceph-mon[32807]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T00:11:27.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:27 smithi170 ceph-mon[32807]: from='osd.7 [v2:172.21.15.170:6816/3231152590,v1:172.21.15.170:6817/3231152590]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-25T00:11:27.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:27 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:11:27.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:27 smithi170 ceph-mon[32807]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-25T00:11:27.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:27 smithi170 ceph-mon[32807]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:27.936 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:11:27 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:11:27.654+0000 7f7a70fe4700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T00:11:27.936 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:11:27 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:11:27.664+0000 7f7a6cdd7700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:11:27.976 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:27 smithi195 ceph-mon[32833]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T00:11:27.977 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:27 smithi195 ceph-mon[32833]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T00:11:27.977 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:27 smithi195 ceph-mon[32833]: from='osd.7 [v2:172.21.15.170:6816/3231152590,v1:172.21.15.170:6817/3231152590]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-25T00:11:27.977 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:27 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:11:27.977 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:27 smithi195 ceph-mon[32833]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-25T00:11:27.977 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:27 smithi195 ceph-mon[32833]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:27.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:27 smithi107 ceph-mon[29153]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T00:11:27.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:27 smithi107 ceph-mon[29153]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T00:11:27.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:27 smithi107 ceph-mon[29153]: from='osd.7 [v2:172.21.15.170:6816/3231152590,v1:172.21.15.170:6817/3231152590]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-25T00:11:27.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:27 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:11:27.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:27 smithi107 ceph-mon[29153]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-25T00:11:27.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:27 smithi107 ceph-mon[29153]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:11:28.490 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T00:11:28.490 INFO:teuthology.orchestra.run.smithi195.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T00:11:28.490 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10+0 records in 2024-04-25T00:11:28.490 INFO:teuthology.orchestra.run.smithi195.stderr:10+0 records out 2024-04-25T00:11:28.490 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00853002 s, 1.2 GB/s 2024-04-25T00:11:28.490 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping successful for: 2024-04-25T00:11:28.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:28 smithi170 ceph-mon[32807]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-04-25T00:11:28.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:28 smithi170 ceph-mon[32807]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T00:11:28.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:28 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:11:28.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:28 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:11:28.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:28 smithi107 ceph-mon[29153]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-04-25T00:11:28.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:28 smithi107 ceph-mon[29153]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T00:11:28.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:28 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:11:28.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:28 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:11:29.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:28 smithi195 ceph-mon[32833]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-04-25T00:11:29.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:28 smithi195 ceph-mon[32833]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T00:11:29.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:28 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:11:29.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:28 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:11:29.135 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi195:vg_nvme/lv_2 2024-04-25T00:11:29.377 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:11:29.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:29 smithi170 ceph-mon[32807]: purged_snaps scrub starts 2024-04-25T00:11:29.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:29 smithi170 ceph-mon[32807]: purged_snaps scrub ok 2024-04-25T00:11:29.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:29 smithi170 ceph-mon[32807]: osd.7 [v2:172.21.15.170:6816/3231152590,v1:172.21.15.170:6817/3231152590] boot 2024-04-25T00:11:29.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:29 smithi170 ceph-mon[32807]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T00:11:29.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:29 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:11:29.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:29 smithi170 ceph-mon[32807]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:29.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:29 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:29.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:29 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:29.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:29 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:29.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:29 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:29.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:29 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:29.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:29 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:29.936 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:29 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:29.936 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:29 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:29.936 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:29 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:29.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:29 smithi107 ceph-mon[29153]: purged_snaps scrub starts 2024-04-25T00:11:29.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:29 smithi107 ceph-mon[29153]: purged_snaps scrub ok 2024-04-25T00:11:29.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:29 smithi107 ceph-mon[29153]: osd.7 [v2:172.21.15.170:6816/3231152590,v1:172.21.15.170:6817/3231152590] boot 2024-04-25T00:11:29.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:29 smithi107 ceph-mon[29153]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T00:11:29.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:29 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:11:29.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:29 smithi107 ceph-mon[29153]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:29.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:29 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:29.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:29 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:29.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:29 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:29.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:29 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:29.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:29 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:29.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:29 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:29.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:29 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:29.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:29 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:29.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:29 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:30.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:29 smithi195 ceph-mon[32833]: purged_snaps scrub starts 2024-04-25T00:11:30.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:29 smithi195 ceph-mon[32833]: purged_snaps scrub ok 2024-04-25T00:11:30.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:29 smithi195 ceph-mon[32833]: osd.7 [v2:172.21.15.170:6816/3231152590,v1:172.21.15.170:6817/3231152590] boot 2024-04-25T00:11:30.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:29 smithi195 ceph-mon[32833]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T00:11:30.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:29 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:11:30.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:29 smithi195 ceph-mon[32833]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:30.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:29 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:30.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:29 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:30.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:29 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:30.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:29 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:30.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:29 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:30.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:29 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:30.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:29 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:30.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:29 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:30.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:29 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:30.758 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:30 smithi195 ceph-mon[32833]: Detected new or changed devices on smithi170 2024-04-25T00:11:30.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:30 smithi195 ceph-mon[32833]: Adjusting osd_memory_target on smithi170 to 5696M 2024-04-25T00:11:30.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:30 smithi195 ceph-mon[32833]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T00:11:30.759 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:30 smithi195 ceph-mon[32833]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T00:11:30.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:30 smithi170 ceph-mon[32807]: Detected new or changed devices on smithi170 2024-04-25T00:11:30.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:30 smithi170 ceph-mon[32807]: Adjusting osd_memory_target on smithi170 to 5696M 2024-04-25T00:11:30.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:30 smithi170 ceph-mon[32807]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T00:11:30.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:30 smithi170 ceph-mon[32807]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T00:11:30.965 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.962+0000 7f042033f700 1 -- 172.21.15.195:0/1216180674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0418102d90 msgr2=0x7f0418105180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:30.965 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.962+0000 7f042033f700 1 --2- 172.21.15.195:0/1216180674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0418102d90 0x7f0418105180 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0414009990 tx=0x7f0414009ca0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.965 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.963+0000 7f042033f700 1 -- 172.21.15.195:0/1216180674 shutdown_connections 2024-04-25T00:11:30.965 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.963+0000 7f042033f700 1 --2- 172.21.15.195:0/1216180674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0418107ff0 0x7f041810a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.965 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.963+0000 7f042033f700 1 --2- 172.21.15.195:0/1216180674 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0418102d90 0x7f0418105180 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.965 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.963+0000 7f042033f700 1 --2- 172.21.15.195:0/1216180674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f04181056c0 0x7f0418107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.965 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.963+0000 7f042033f700 1 -- 172.21.15.195:0/1216180674 >> 172.21.15.195:0/1216180674 conn(0x7f04180fc760 msgr2=0x7f04180feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:30.965 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.963+0000 7f042033f700 1 -- 172.21.15.195:0/1216180674 shutdown_connections 2024-04-25T00:11:30.965 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.963+0000 7f042033f700 1 -- 172.21.15.195:0/1216180674 wait complete. 2024-04-25T00:11:30.965 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.963+0000 7f042033f700 1 Processor -- start 2024-04-25T00:11:30.965 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.963+0000 7f042033f700 1 -- start start 2024-04-25T00:11:30.965 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.963+0000 7f042033f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0418102d90 0x7f0418198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:30.966 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.963+0000 7f042033f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04181056c0 0x7f0418198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:30.966 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.963+0000 7f042033f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0418107ff0 0x7f041819fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:30.966 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.963+0000 7f042033f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f041810def0 con 0x7f04181056c0 2024-04-25T00:11:30.966 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.963+0000 7f042033f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f041810dd70 con 0x7f0418102d90 2024-04-25T00:11:30.966 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.963+0000 7f042033f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f041810e070 con 0x7f0418107ff0 2024-04-25T00:11:30.966 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.964+0000 7f041e8dc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0418107ff0 0x7f041819fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:30.966 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.964+0000 7f041e8dc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0418107ff0 0x7f041819fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:52268/0 (socket says 172.21.15.195:52268) 2024-04-25T00:11:30.966 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.964+0000 7f041e8dc700 1 -- 172.21.15.195:0/3106696229 learned_addr learned my addr 172.21.15.195:0/3106696229 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-04-25T00:11:30.966 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.964+0000 7f041d8da700 1 --2- 172.21.15.195:0/3106696229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04181056c0 0x7f0418198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:30.966 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.964+0000 7f041e0db700 1 --2- 172.21.15.195:0/3106696229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0418102d90 0x7f0418198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:30.966 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.964+0000 7f041e8dc700 1 -- 172.21.15.195:0/3106696229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0418102d90 msgr2=0x7f0418198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:30.966 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.964+0000 7f041e8dc700 1 --2- 172.21.15.195:0/3106696229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0418102d90 0x7f0418198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.966 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.964+0000 7f041e8dc700 1 -- 172.21.15.195:0/3106696229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04181056c0 msgr2=0x7f0418198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:30.966 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.964+0000 7f041e8dc700 1 --2- 172.21.15.195:0/3106696229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04181056c0 0x7f0418198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:30.967 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.964+0000 7f041e8dc700 1 -- 172.21.15.195:0/3106696229 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04181a01c0 con 0x7f0418107ff0 2024-04-25T00:11:30.967 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.964+0000 7f041e0db700 1 --2- 172.21.15.195:0/3106696229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0418102d90 0x7f0418198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:30.967 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.964+0000 7f041e8dc700 1 --2- 172.21.15.195:0/3106696229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0418107ff0 0x7f041819fb20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0410007ee0 tx=0x7f041000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:30.967 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.964+0000 7f040f7fe700 1 -- 172.21.15.195:0/3106696229 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410004db0 con 0x7f0418107ff0 2024-04-25T00:11:30.967 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.964+0000 7f042033f700 1 -- 172.21.15.195:0/3106696229 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04181a04b0 con 0x7f0418107ff0 2024-04-25T00:11:30.967 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.964+0000 7f040f7fe700 1 -- 172.21.15.195:0/3106696229 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f041000dbd0 con 0x7f0418107ff0 2024-04-25T00:11:30.967 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.964+0000 7f040f7fe700 1 -- 172.21.15.195:0/3106696229 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410015660 con 0x7f0418107ff0 2024-04-25T00:11:30.968 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.964+0000 7f042033f700 1 -- 172.21.15.195:0/3106696229 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04181a0a20 con 0x7f0418107ff0 2024-04-25T00:11:30.968 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.965+0000 7f042033f700 1 -- 172.21.15.195:0/3106696229 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0418068f20 con 0x7f0418107ff0 2024-04-25T00:11:30.968 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.966+0000 7f040f7fe700 1 -- 172.21.15.195:0/3106696229 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f0410005000 con 0x7f0418107ff0 2024-04-25T00:11:30.968 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.967+0000 7f040f7fe700 1 --2- 172.21.15.195:0/3106696229 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f04040a2c20 0x7f04040a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:30.968 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.967+0000 7f040f7fe700 1 -- 172.21.15.195:0/3106696229 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7f04100ce870 con 0x7f0418107ff0 2024-04-25T00:11:30.968 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.967+0000 7f041e0db700 1 --2- 172.21.15.195:0/3106696229 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f04040a2c20 0x7f04040a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:30.969 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.968+0000 7f041e0db700 1 --2- 172.21.15.195:0/3106696229 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f04040a2c20 0x7f04040a50e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f04140039c0 tx=0x7f0414005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:30.971 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:30.971+0000 7f040f7fe700 1 -- 172.21.15.195:0/3106696229 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0410010070 con 0x7f0418107ff0 2024-04-25T00:11:30.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:30 smithi107 ceph-mon[29153]: Detected new or changed devices on smithi170 2024-04-25T00:11:30.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:30 smithi107 ceph-mon[29153]: Adjusting osd_memory_target on smithi170 to 5696M 2024-04-25T00:11:30.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:30 smithi107 ceph-mon[29153]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T00:11:30.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:30 smithi107 ceph-mon[29153]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T00:11:31.175 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:31.174+0000 7f042033f700 1 -- 172.21.15.195:0/3106696229 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f0418062670 con 0x7f04040a2c20 2024-04-25T00:11:31.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:31 smithi170 ceph-mon[32807]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:31.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:31 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:11:31.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:31 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:11:31.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:31 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:31.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:31 smithi170 ceph-mon[32807]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T00:11:31.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:31 smithi107 ceph-mon[29153]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:31.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:31 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:11:31.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:31 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:11:31.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:31 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:31.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:31 smithi107 ceph-mon[29153]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T00:11:32.004 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:31 smithi195 ceph-mon[32833]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:32.004 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:31 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:11:32.004 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:31 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:11:32.005 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:31 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:32.005 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:31 smithi195 ceph-mon[32833]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T00:11:32.750 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:32 smithi195 ceph-mon[32833]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:11:32.751 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:32 smithi195 ceph-mon[32833]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T00:11:32.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:32 smithi170 ceph-mon[32807]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:11:32.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:32 smithi170 ceph-mon[32807]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T00:11:32.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:32 smithi107 ceph-mon[29153]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:11:32.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:32 smithi107 ceph-mon[29153]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T00:11:33.936 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:33 smithi170 ceph-mon[32807]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:33.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:33 smithi107 ceph-mon[29153]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:34.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:33 smithi195 ceph-mon[32833]: pgmap v182: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:34.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:34 smithi107 ceph-mon[29153]: from='client.? 172.21.15.195:0/3949411346' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebddecde-4dac-4b5e-af58-f01f285aa7c5"}]: dispatch 2024-04-25T00:11:34.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:34 smithi107 ceph-mon[29153]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebddecde-4dac-4b5e-af58-f01f285aa7c5"}]: dispatch 2024-04-25T00:11:34.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:34 smithi107 ceph-mon[29153]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ebddecde-4dac-4b5e-af58-f01f285aa7c5"}]': finished 2024-04-25T00:11:34.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:34 smithi107 ceph-mon[29153]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T00:11:34.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:34 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:11:34.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:34 smithi107 ceph-mon[29153]: from='client.? 172.21.15.195:0/3403018782' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:11:35.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:34 smithi195 ceph-mon[32833]: from='client.? 172.21.15.195:0/3949411346' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebddecde-4dac-4b5e-af58-f01f285aa7c5"}]: dispatch 2024-04-25T00:11:35.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:34 smithi195 ceph-mon[32833]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebddecde-4dac-4b5e-af58-f01f285aa7c5"}]: dispatch 2024-04-25T00:11:35.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:34 smithi195 ceph-mon[32833]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ebddecde-4dac-4b5e-af58-f01f285aa7c5"}]': finished 2024-04-25T00:11:35.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:34 smithi195 ceph-mon[32833]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T00:11:35.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:34 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:11:35.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:34 smithi195 ceph-mon[32833]: from='client.? 172.21.15.195:0/3403018782' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:11:35.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:34 smithi170 ceph-mon[32807]: from='client.? 172.21.15.195:0/3949411346' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebddecde-4dac-4b5e-af58-f01f285aa7c5"}]: dispatch 2024-04-25T00:11:35.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:34 smithi170 ceph-mon[32807]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ebddecde-4dac-4b5e-af58-f01f285aa7c5"}]: dispatch 2024-04-25T00:11:35.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:34 smithi170 ceph-mon[32807]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ebddecde-4dac-4b5e-af58-f01f285aa7c5"}]': finished 2024-04-25T00:11:35.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:34 smithi170 ceph-mon[32807]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T00:11:35.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:34 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:11:35.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:34 smithi170 ceph-mon[32807]: from='client.? 172.21.15.195:0/3403018782' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:11:35.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:35 smithi107 ceph-mon[29153]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:36.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:35 smithi195 ceph-mon[32833]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:36.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:35 smithi170 ceph-mon[32807]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:38.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:37 smithi107 ceph-mon[29153]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:38.234 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:37 smithi195 ceph-mon[32833]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:38.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:37 smithi170 ceph-mon[32807]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:40.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:39 smithi195 ceph-mon[32833]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:40.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:39 smithi107 ceph-mon[29153]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:40.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:39 smithi170 ceph-mon[32807]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:42.028 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:41 smithi195 ceph-mon[32833]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:42.028 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:41 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T00:11:42.028 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:41 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:42.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:41 smithi107 ceph-mon[29153]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:42.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:41 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T00:11:42.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:41 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:42.237 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:41 smithi170 ceph-mon[32807]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:42.237 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:41 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T00:11:42.237 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:41 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:43.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:42 smithi107 ceph-mon[29153]: Deploying daemon osd.8 on smithi195 2024-04-25T00:11:43.244 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:42 smithi195 ceph-mon[32833]: Deploying daemon osd.8 on smithi195 2024-04-25T00:11:43.431 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:42 smithi170 ceph-mon[32807]: Deploying daemon osd.8 on smithi195 2024-04-25T00:11:44.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:43 smithi107 ceph-mon[29153]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:44.264 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:43 smithi195 ceph-mon[32833]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:44.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:43 smithi170 ceph-mon[32807]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:46.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:45 smithi107 ceph-mon[29153]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:46.305 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:45 smithi195 ceph-mon[32833]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:46.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:45 smithi170 ceph-mon[32807]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:47.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:47.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:47.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:47.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:47 smithi195 ceph-mon[32833]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:47.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:47.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:47.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:47.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:47 smithi170 ceph-mon[32807]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:47.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:47.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:47.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:47.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:47 smithi107 ceph-mon[29153]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:50.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:49 smithi195 ceph-mon[32833]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:50.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:49 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:50.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:49 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:50.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:49 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:50.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:49 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:50.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:49 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:50.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:49 smithi107 ceph-mon[29153]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:50.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:49 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:50.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:49 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:50.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:49 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:50.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:49 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:50.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:49 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:50.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:49 smithi170 ceph-mon[32807]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:50.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:49 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:50.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:49 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:50.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:49 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:50.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:49 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:50.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:49 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:50.495 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:50.494+0000 7f040f7fe700 1 -- 172.21.15.195:0/3106696229 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0418062670 con 0x7f04040a2c20 2024-04-25T00:11:50.495 INFO:teuthology.orchestra.run.smithi195.stdout:Created osd(s) 8 on host 'smithi195' 2024-04-25T00:11:50.496 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:50.496+0000 7f042033f700 1 -- 172.21.15.195:0/3106696229 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f04040a2c20 msgr2=0x7f04040a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:50.497 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:50.496+0000 7f042033f700 1 --2- 172.21.15.195:0/3106696229 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f04040a2c20 0x7f04040a50e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f04140039c0 tx=0x7f0414005fb0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.497 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:50.496+0000 7f042033f700 1 -- 172.21.15.195:0/3106696229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0418107ff0 msgr2=0x7f041819fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:50.497 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:50.496+0000 7f042033f700 1 --2- 172.21.15.195:0/3106696229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0418107ff0 0x7f041819fb20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0410007ee0 tx=0x7f041000d3b0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.497 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:50.496+0000 7f042033f700 1 -- 172.21.15.195:0/3106696229 shutdown_connections 2024-04-25T00:11:50.497 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:50.496+0000 7f042033f700 1 --2- 172.21.15.195:0/3106696229 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f04040a2c20 0x7f04040a50e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.497 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:50.496+0000 7f042033f700 1 --2- 172.21.15.195:0/3106696229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0418107ff0 0x7f041819fb20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.497 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:50.496+0000 7f042033f700 1 --2- 172.21.15.195:0/3106696229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0418102d90 0x7f0418198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.497 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:50.496+0000 7f042033f700 1 --2- 172.21.15.195:0/3106696229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04181056c0 0x7f0418198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:50.497 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:50.496+0000 7f042033f700 1 -- 172.21.15.195:0/3106696229 >> 172.21.15.195:0/3106696229 conn(0x7f04180fc760 msgr2=0x7f0418103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:50.497 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:50.496+0000 7f042033f700 1 -- 172.21.15.195:0/3106696229 shutdown_connections 2024-04-25T00:11:50.497 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:11:50.496+0000 7f042033f700 1 -- 172.21.15.195:0/3106696229 wait complete. 2024-04-25T00:11:51.127 DEBUG:teuthology.orchestra.run.smithi195:osd.8> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T00:11:51.130 INFO:tasks.cephadm:Deploying osd.9 on smithi107 with /dev/vg_nvme/lv_1... 2024-04-25T00:11:51.130 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T00:11:51.375 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:11:51.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:51 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:51.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:51 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:51.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:51 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:51.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:51 smithi107 ceph-mon[29153]: from='osd.8 [v2:172.21.15.195:6816/2391866428,v1:172.21.15.195:6817/2391866428]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T00:11:51.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:51 smithi107 ceph-mon[29153]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T00:11:51.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:51 smithi107 ceph-mon[29153]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:51.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:51 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:51.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:51 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:51.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:51 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:51.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:51 smithi195 ceph-mon[32833]: from='osd.8 [v2:172.21.15.195:6816/2391866428,v1:172.21.15.195:6817/2391866428]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T00:11:51.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:51 smithi195 ceph-mon[32833]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T00:11:51.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:51 smithi195 ceph-mon[32833]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:51.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:51 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:51.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:51 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:51.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:51 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:51.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:51 smithi170 ceph-mon[32807]: from='osd.8 [v2:172.21.15.195:6816/2391866428,v1:172.21.15.195:6817/2391866428]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T00:11:51.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:51 smithi170 ceph-mon[32807]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T00:11:51.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:51 smithi170 ceph-mon[32807]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:52.553 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:52 smithi195 ceph-mon[32833]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T00:11:52.553 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:52 smithi195 ceph-mon[32833]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T00:11:52.553 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:52 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:11:52.553 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:52 smithi195 ceph-mon[32833]: from='osd.8 [v2:172.21.15.195:6816/2391866428,v1:172.21.15.195:6817/2391866428]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-04-25T00:11:52.553 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:52 smithi195 ceph-mon[32833]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-04-25T00:11:52.553 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:11:52 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:11:52.461+0000 7fbd4a688700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T00:11:52.553 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:11:52 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:11:52.471+0000 7fbd4647b700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:11:52.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:52 smithi107 ceph-mon[29153]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T00:11:52.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:52 smithi107 ceph-mon[29153]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T00:11:52.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:52 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:11:52.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:52 smithi107 ceph-mon[29153]: from='osd.8 [v2:172.21.15.195:6816/2391866428,v1:172.21.15.195:6817/2391866428]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-04-25T00:11:52.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:52 smithi107 ceph-mon[29153]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-04-25T00:11:52.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:52 smithi170 ceph-mon[32807]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T00:11:52.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:52 smithi170 ceph-mon[32807]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T00:11:52.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:52 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:11:52.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:52 smithi170 ceph-mon[32807]: from='osd.8 [v2:172.21.15.195:6816/2391866428,v1:172.21.15.195:6817/2391866428]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-04-25T00:11:52.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:52 smithi170 ceph-mon[32807]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-04-25T00:11:53.506 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T00:11:53.506 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T00:11:53.506 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-04-25T00:11:53.507 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-04-25T00:11:53.507 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114454 s, 916 MB/s 2024-04-25T00:11:53.507 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-04-25T00:11:53.690 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:53 smithi195 ceph-mon[32833]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-04-25T00:11:53.690 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:53 smithi195 ceph-mon[32833]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T00:11:53.690 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:53 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:11:53.690 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:53 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:11:53.690 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:53 smithi195 ceph-mon[32833]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:53.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:53 smithi107 ceph-mon[29153]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-04-25T00:11:53.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:53 smithi107 ceph-mon[29153]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T00:11:53.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:53 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:11:53.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:53 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:11:53.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:53 smithi107 ceph-mon[29153]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:53.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:53 smithi170 ceph-mon[32807]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-04-25T00:11:53.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:53 smithi170 ceph-mon[32807]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T00:11:53.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:53 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:11:53.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:53 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:11:53.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:53 smithi170 ceph-mon[32807]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:11:54.326 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_1 2024-04-25T00:11:54.565 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:11:54.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:54 smithi195 ceph-mon[32833]: purged_snaps scrub starts 2024-04-25T00:11:54.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:54 smithi195 ceph-mon[32833]: purged_snaps scrub ok 2024-04-25T00:11:54.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:54 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:11:54.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:54 smithi195 ceph-mon[32833]: osd.8 [v2:172.21.15.195:6816/2391866428,v1:172.21.15.195:6817/2391866428] boot 2024-04-25T00:11:54.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:54 smithi195 ceph-mon[32833]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T00:11:54.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:54 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:11:54.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:54 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:54.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:54 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:54.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:54 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:54.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:54 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:54.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:54 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:54.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:54 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:54.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:54 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:54.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:54 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:54.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:54 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:54.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:54 smithi107 ceph-mon[29153]: purged_snaps scrub starts 2024-04-25T00:11:54.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:54 smithi107 ceph-mon[29153]: purged_snaps scrub ok 2024-04-25T00:11:54.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:54 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:11:54.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:54 smithi107 ceph-mon[29153]: osd.8 [v2:172.21.15.195:6816/2391866428,v1:172.21.15.195:6817/2391866428] boot 2024-04-25T00:11:54.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:54 smithi107 ceph-mon[29153]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T00:11:54.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:54 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:11:54.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:54 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:54.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:54 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:54.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:54 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:54.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:54 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:54.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:54 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:54.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:54 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:54.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:54 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:54.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:54 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:54.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:54 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:54.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:54 smithi170 ceph-mon[32807]: purged_snaps scrub starts 2024-04-25T00:11:54.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:54 smithi170 ceph-mon[32807]: purged_snaps scrub ok 2024-04-25T00:11:54.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:11:54.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:54 smithi170 ceph-mon[32807]: osd.8 [v2:172.21.15.195:6816/2391866428,v1:172.21.15.195:6817/2391866428] boot 2024-04-25T00:11:54.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:54 smithi170 ceph-mon[32807]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T00:11:54.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:11:54.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:54.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:54.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:54.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:54.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:54.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:54.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:54.936 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:54.936 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:54 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:11:55.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:55 smithi107 ceph-mon[29153]: Detected new or changed devices on smithi195 2024-04-25T00:11:55.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:55 smithi107 ceph-mon[29153]: Adjusting osd_memory_target on smithi195 to 5696M 2024-04-25T00:11:55.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:55 smithi107 ceph-mon[29153]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T00:11:55.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:55 smithi107 ceph-mon[29153]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:11:55.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:55 smithi195 ceph-mon[32833]: Detected new or changed devices on smithi195 2024-04-25T00:11:55.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:55 smithi195 ceph-mon[32833]: Adjusting osd_memory_target on smithi195 to 5696M 2024-04-25T00:11:55.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:55 smithi195 ceph-mon[32833]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T00:11:55.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:55 smithi195 ceph-mon[32833]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:11:55.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:55 smithi170 ceph-mon[32807]: Detected new or changed devices on smithi195 2024-04-25T00:11:55.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:55 smithi170 ceph-mon[32807]: Adjusting osd_memory_target on smithi195 to 5696M 2024-04-25T00:11:55.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:55 smithi170 ceph-mon[32807]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T00:11:55.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:55 smithi170 ceph-mon[32807]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:11:56.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.316+0000 7f46da1ce700 1 -- 172.21.15.107:0/942660974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d4101df0 msgr2=0x7f46d4109690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:56.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.316+0000 7f46da1ce700 1 --2- 172.21.15.107:0/942660974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d4101df0 0x7f46d4109690 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00b0a0 tx=0x7f46cc00b3b0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.316+0000 7f46da1ce700 1 -- 172.21.15.107:0/942660974 shutdown_connections 2024-04-25T00:11:56.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.316+0000 7f46da1ce700 1 --2- 172.21.15.107:0/942660974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d4101df0 0x7f46d4109690 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.316+0000 7f46da1ce700 1 --2- 172.21.15.107:0/942660974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46d4102860 0x7f46d4102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.316+0000 7f46da1ce700 1 --2- 172.21.15.107:0/942660974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46d4101430 0x7f46d41018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.316+0000 7f46da1ce700 1 -- 172.21.15.107:0/942660974 >> 172.21.15.107:0/942660974 conn(0x7f46d40fbf00 msgr2=0x7f46d40fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:56.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.316+0000 7f46da1ce700 1 -- 172.21.15.107:0/942660974 shutdown_connections 2024-04-25T00:11:56.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.316+0000 7f46da1ce700 1 -- 172.21.15.107:0/942660974 wait complete. 2024-04-25T00:11:56.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.317+0000 7f46da1ce700 1 Processor -- start 2024-04-25T00:11:56.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.317+0000 7f46da1ce700 1 -- start start 2024-04-25T00:11:56.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.317+0000 7f46da1ce700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46d4101430 0x7f46d41051d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:56.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.317+0000 7f46da1ce700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d4101df0 0x7f46d4103820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:56.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.317+0000 7f46da1ce700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46d4102860 0x7f46d4103d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:56.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.317+0000 7f46da1ce700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f46d410ce20 con 0x7f46d4101df0 2024-04-25T00:11:56.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.317+0000 7f46da1ce700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f46d410cca0 con 0x7f46d4102860 2024-04-25T00:11:56.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.317+0000 7f46da1ce700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f46d410cfa0 con 0x7f46d4101430 2024-04-25T00:11:56.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.317+0000 7f46d2ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d4101df0 0x7f46d4103820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:56.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.317+0000 7f46d2ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d4101df0 0x7f46d4103820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52978/0 (socket says 172.21.15.107:52978) 2024-04-25T00:11:56.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.317+0000 7f46d2ffd700 1 -- 172.21.15.107:0/3678935583 learned_addr learned my addr 172.21.15.107:0/3678935583 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:11:56.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.317+0000 7f46d2ffd700 1 -- 172.21.15.107:0/3678935583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46d4101430 msgr2=0x7f46d41051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:56.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.317+0000 7f46d2ffd700 1 --2- 172.21.15.107:0/3678935583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46d4101430 0x7f46d41051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.317+0000 7f46d3fff700 1 --2- 172.21.15.107:0/3678935583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46d4102860 0x7f46d4103d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:56.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.317+0000 7f46d2ffd700 1 -- 172.21.15.107:0/3678935583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46d4102860 msgr2=0x7f46d4103d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:56.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.317+0000 7f46d2ffd700 1 --2- 172.21.15.107:0/3678935583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46d4102860 0x7f46d4103d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:56.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.318+0000 7f46d2ffd700 1 -- 172.21.15.107:0/3678935583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d4104610 con 0x7f46d4101df0 2024-04-25T00:11:56.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.318+0000 7f46d3fff700 1 --2- 172.21.15.107:0/3678935583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46d4102860 0x7f46d4103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:56.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.318+0000 7f46d2ffd700 1 --2- 172.21.15.107:0/3678935583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d4101df0 0x7f46d4103820 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f46c400b530 tx=0x7f46c400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:56.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.318+0000 7f46d0ff9700 1 -- 172.21.15.107:0/3678935583 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c4015020 con 0x7f46d4101df0 2024-04-25T00:11:56.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.318+0000 7f46d0ff9700 1 -- 172.21.15.107:0/3678935583 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46c4011d10 con 0x7f46d4101df0 2024-04-25T00:11:56.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.318+0000 7f46da1ce700 1 -- 172.21.15.107:0/3678935583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d41a25e0 con 0x7f46d4101df0 2024-04-25T00:11:56.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.318+0000 7f46d0ff9700 1 -- 172.21.15.107:0/3678935583 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c4010e50 con 0x7f46d4101df0 2024-04-25T00:11:56.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.318+0000 7f46da1ce700 1 -- 172.21.15.107:0/3678935583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d41a2af0 con 0x7f46d4101df0 2024-04-25T00:11:56.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.319+0000 7f46da1ce700 1 -- 172.21.15.107:0/3678935583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d4068f20 con 0x7f46d4101df0 2024-04-25T00:11:56.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.320+0000 7f46d0ff9700 1 -- 172.21.15.107:0/3678935583 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f46c4018020 con 0x7f46d4101df0 2024-04-25T00:11:56.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.320+0000 7f46d0ff9700 1 --2- 172.21.15.107:0/3678935583 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f46c00a2dc0 0x7f46c00a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:56.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.320+0000 7f46d37fe700 1 --2- 172.21.15.107:0/3678935583 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f46c00a2dc0 0x7f46c00a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:56.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.320+0000 7f46d0ff9700 1 -- 172.21.15.107:0/3678935583 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5858+0+0 (secure 0 0 0) 0x7f46c40cdf40 con 0x7f46d4101df0 2024-04-25T00:11:56.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.321+0000 7f46d37fe700 1 --2- 172.21.15.107:0/3678935583 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f46c00a2dc0 0x7f46c00a5280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f46bc005fd0 tx=0x7f46bc005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:56.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.325+0000 7f46d0ff9700 1 -- 172.21.15.107:0/3678935583 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f46c401f020 con 0x7f46d4101df0 2024-04-25T00:11:56.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:11:56.535+0000 7f46da1ce700 1 -- 172.21.15.107:0/3678935583 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f46d41a2db0 con 0x7f46c00a2dc0 2024-04-25T00:11:56.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:56 smithi107 ceph-mon[29153]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T00:11:56.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:56 smithi195 ceph-mon[32833]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T00:11:56.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:56 smithi170 ceph-mon[32807]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T00:11:57.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:57 smithi107 ceph-mon[29153]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T00:11:57.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:57 smithi107 ceph-mon[29153]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:11:57.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:57 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:11:57.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:57 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:11:57.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:57 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:57.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:57 smithi107 ceph-mon[29153]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:11:57.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:57 smithi195 ceph-mon[32833]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T00:11:57.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:57 smithi195 ceph-mon[32833]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:11:57.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:57 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:11:57.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:57 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:11:57.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:57 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:57.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:57 smithi195 ceph-mon[32833]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:11:57.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:57 smithi170 ceph-mon[32807]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T00:11:57.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:57 smithi170 ceph-mon[32807]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:11:57.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:57 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:11:57.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:57 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:11:57.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:57 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:57.936 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:57 smithi170 ceph-mon[32807]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:11:58.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:58 smithi107 ceph-mon[29153]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T00:11:58.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:58 smithi195 ceph-mon[32833]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T00:11:58.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:58 smithi170 ceph-mon[32807]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T00:11:59.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:59 smithi195 ceph-mon[32833]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-04-25T00:11:59.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:59 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/568470510' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f85eab7a-7430-4520-9b2d-932398199b4c"}]: dispatch 2024-04-25T00:11:59.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:59 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/568470510' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f85eab7a-7430-4520-9b2d-932398199b4c"}]': finished 2024-04-25T00:11:59.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:59 smithi195 ceph-mon[32833]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T00:11:59.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:11:59 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:11:59.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:59 smithi170 ceph-mon[32807]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-04-25T00:11:59.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:59 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/568470510' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f85eab7a-7430-4520-9b2d-932398199b4c"}]: dispatch 2024-04-25T00:11:59.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:59 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/568470510' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f85eab7a-7430-4520-9b2d-932398199b4c"}]': finished 2024-04-25T00:11:59.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:59 smithi170 ceph-mon[32807]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T00:11:59.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:11:59 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:11:59.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:59 smithi107 ceph-mon[29153]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 130 KiB/s, 0 objects/s recovering 2024-04-25T00:11:59.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:59 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/568470510' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f85eab7a-7430-4520-9b2d-932398199b4c"}]: dispatch 2024-04-25T00:11:59.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:59 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/568470510' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f85eab7a-7430-4520-9b2d-932398199b4c"}]': finished 2024-04-25T00:11:59.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:59 smithi107 ceph-mon[29153]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T00:11:59.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:11:59 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:12:00.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:00 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/4123759792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:12:00.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:00 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/4123759792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:12:00.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:00 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/4123759792' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:12:01.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:01 smithi195 ceph-mon[32833]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-25T00:12:01.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:01 smithi170 ceph-mon[32807]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-25T00:12:01.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:01 smithi107 ceph-mon[29153]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-04-25T00:12:04.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:03 smithi107 ceph-mon[29153]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-25T00:12:04.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:03 smithi195 ceph-mon[32833]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-25T00:12:04.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:03 smithi170 ceph-mon[32807]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-04-25T00:12:06.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:05 smithi107 ceph-mon[29153]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T00:12:06.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:05 smithi195 ceph-mon[32833]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T00:12:06.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:05 smithi170 ceph-mon[32807]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T00:12:07.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:06 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T00:12:07.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:06 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:07.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:06 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T00:12:07.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:06 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:07.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:06 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T00:12:07.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:06 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:08.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:07 smithi107 ceph-mon[29153]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-25T00:12:08.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:07 smithi107 ceph-mon[29153]: Deploying daemon osd.9 on smithi107 2024-04-25T00:12:08.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:07 smithi195 ceph-mon[32833]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-25T00:12:08.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:07 smithi195 ceph-mon[32833]: Deploying daemon osd.9 on smithi107 2024-04-25T00:12:08.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:07 smithi170 ceph-mon[32807]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-25T00:12:08.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:07 smithi170 ceph-mon[32807]: Deploying daemon osd.9 on smithi107 2024-04-25T00:12:10.139 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:09 smithi107 ceph-mon[29153]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:12:10.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:09 smithi195 ceph-mon[32833]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:12:10.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:09 smithi170 ceph-mon[32807]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:12:12.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:11 smithi107 ceph-mon[29153]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:12:12.236 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:11 smithi170 ceph-mon[32807]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:12:12.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:11 smithi195 ceph-mon[32833]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:12:13.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:12 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:13.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:12 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:13.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:12 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:13.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:12 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:13.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:12 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:13.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:12 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:13.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:12 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:13.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:12 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:13.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:12 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:14.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:13 smithi107 ceph-mon[29153]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:12:14.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:13 smithi195 ceph-mon[32833]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:12:14.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:13 smithi170 ceph-mon[32807]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:12:16.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:15 smithi107 ceph-mon[29153]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:12:16.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:15 smithi195 ceph-mon[32833]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:12:16.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:15 smithi170 ceph-mon[32807]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:12:17.142 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 9 on host 'smithi107' 2024-04-25T00:12:17.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:12:17.142+0000 7f46d0ff9700 1 -- 172.21.15.107:0/3678935583 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f46d41a2db0 con 0x7f46c00a2dc0 2024-04-25T00:12:17.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:12:17.144+0000 7f46da1ce700 1 -- 172.21.15.107:0/3678935583 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f46c00a2dc0 msgr2=0x7f46c00a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:17.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:12:17.144+0000 7f46da1ce700 1 --2- 172.21.15.107:0/3678935583 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f46c00a2dc0 0x7f46c00a5280 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f46bc005fd0 tx=0x7f46bc005d00 comp rx=0 tx=0).stop 2024-04-25T00:12:17.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:12:17.144+0000 7f46da1ce700 1 -- 172.21.15.107:0/3678935583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d4101df0 msgr2=0x7f46d4103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:17.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:12:17.144+0000 7f46da1ce700 1 --2- 172.21.15.107:0/3678935583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d4101df0 0x7f46d4103820 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f46c400b530 tx=0x7f46c400b8f0 comp rx=0 tx=0).stop 2024-04-25T00:12:17.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:12:17.144+0000 7f46da1ce700 1 -- 172.21.15.107:0/3678935583 shutdown_connections 2024-04-25T00:12:17.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:12:17.144+0000 7f46da1ce700 1 --2- 172.21.15.107:0/3678935583 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f46c00a2dc0 0x7f46c00a5280 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:17.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:12:17.144+0000 7f46da1ce700 1 --2- 172.21.15.107:0/3678935583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46d4102860 0x7f46d4103d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:17.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:12:17.144+0000 7f46da1ce700 1 --2- 172.21.15.107:0/3678935583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46d4101430 0x7f46d41051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:17.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:12:17.144+0000 7f46da1ce700 1 --2- 172.21.15.107:0/3678935583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d4101df0 0x7f46d4103820 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:17.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:12:17.145+0000 7f46da1ce700 1 -- 172.21.15.107:0/3678935583 >> 172.21.15.107:0/3678935583 conn(0x7f46d40fbf00 msgr2=0x7f46d40fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:17.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:12:17.145+0000 7f46da1ce700 1 -- 172.21.15.107:0/3678935583 shutdown_connections 2024-04-25T00:12:17.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:12:17.145+0000 7f46da1ce700 1 -- 172.21.15.107:0/3678935583 wait complete. 2024-04-25T00:12:17.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:17 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:17.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:17 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:17.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:17 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:17.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:17 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:17.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:17 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:17.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:17 smithi107 ceph-mon[29153]: from='osd.9 [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T00:12:17.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:17 smithi107 ceph-mon[29153]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:12:17.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:17 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:17.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:17 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:17.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:17 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:17.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:17 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:17.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:17 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:17.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:17 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:17.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:17 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:17.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:17 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:17.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:17 smithi195 ceph-mon[32833]: from='osd.9 [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T00:12:17.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:17 smithi195 ceph-mon[32833]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:12:17.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:17 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:17.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:17 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:17.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:17 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:17.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:17 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:17.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:17 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:17.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:17 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:17.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:17 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:17.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:17 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:17.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:17 smithi170 ceph-mon[32807]: from='osd.9 [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T00:12:17.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:17 smithi170 ceph-mon[32807]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:12:17.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:17 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:17.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:17 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:17.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:17 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:17.831 DEBUG:teuthology.orchestra.run.smithi107:osd.9> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T00:12:17.833 INFO:tasks.cephadm:Deploying osd.10 on smithi170 with /dev/vg_nvme/lv_1... 2024-04-25T00:12:17.834 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T00:12:18.079 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:12:18.452 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:18 smithi170 ceph-mon[32807]: from='osd.9 [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T00:12:18.452 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:18 smithi170 ceph-mon[32807]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T00:12:18.452 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:18 smithi170 ceph-mon[32807]: from='osd.9 [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-25T00:12:18.452 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:18 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:12:18.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:18 smithi195 ceph-mon[32833]: from='osd.9 [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T00:12:18.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:18 smithi195 ceph-mon[32833]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T00:12:18.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:18 smithi195 ceph-mon[32833]: from='osd.9 [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-25T00:12:18.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:18 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:12:18.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:18 smithi107 ceph-mon[29153]: from='osd.9 [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T00:12:18.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:18 smithi107 ceph-mon[29153]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T00:12:18.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:18 smithi107 ceph-mon[29153]: from='osd.9 [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-04-25T00:12:18.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:18 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:12:19.329 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:12:19 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:12:19.021+0000 7fc8a2d47700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T00:12:19.329 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:12:19 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:12:19.032+0000 7fc89ab32700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:12:19.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:19 smithi170 ceph-mon[32807]: from='osd.9 [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-25T00:12:19.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:19 smithi170 ceph-mon[32807]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T00:12:19.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:19 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:12:19.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:19 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:12:19.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:19 smithi170 ceph-mon[32807]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:12:19.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:19 smithi170 ceph-mon[32807]: from='osd.9 [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257]' entity='osd.9' 2024-04-25T00:12:19.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:19 smithi195 ceph-mon[32833]: from='osd.9 [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-25T00:12:19.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:19 smithi195 ceph-mon[32833]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T00:12:19.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:19 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:12:19.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:19 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:12:19.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:19 smithi195 ceph-mon[32833]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:12:19.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:19 smithi195 ceph-mon[32833]: from='osd.9 [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257]' entity='osd.9' 2024-04-25T00:12:19.667 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:19 smithi107 ceph-mon[29153]: from='osd.9 [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-04-25T00:12:19.667 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:19 smithi107 ceph-mon[29153]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T00:12:19.667 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:19 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:12:19.667 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:19 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:12:19.667 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:19 smithi107 ceph-mon[29153]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 265 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:12:19.667 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:19 smithi107 ceph-mon[29153]: from='osd.9 [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257]' entity='osd.9' 2024-04-25T00:12:20.009 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T00:12:20.009 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T00:12:20.009 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-04-25T00:12:20.009 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-04-25T00:12:20.010 INFO:teuthology.orchestra.run.smithi170.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00841702 s, 1.2 GB/s 2024-04-25T00:12:20.010 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-04-25T00:12:20.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:20 smithi195 ceph-mon[32833]: purged_snaps scrub starts 2024-04-25T00:12:20.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:20 smithi195 ceph-mon[32833]: purged_snaps scrub ok 2024-04-25T00:12:20.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:20 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:12:20.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:20 smithi195 ceph-mon[32833]: osd.9 [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257] boot 2024-04-25T00:12:20.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:20 smithi195 ceph-mon[32833]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T00:12:20.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:20 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:12:20.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:20 smithi170 ceph-mon[32807]: purged_snaps scrub starts 2024-04-25T00:12:20.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:20 smithi170 ceph-mon[32807]: purged_snaps scrub ok 2024-04-25T00:12:20.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:20 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:12:20.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:20 smithi170 ceph-mon[32807]: osd.9 [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257] boot 2024-04-25T00:12:20.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:20 smithi170 ceph-mon[32807]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T00:12:20.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:20 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:12:20.728 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:20 smithi107 ceph-mon[29153]: purged_snaps scrub starts 2024-04-25T00:12:20.728 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:20 smithi107 ceph-mon[29153]: purged_snaps scrub ok 2024-04-25T00:12:20.728 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:20 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:12:20.728 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:20 smithi107 ceph-mon[29153]: osd.9 [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257] boot 2024-04-25T00:12:20.729 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:20 smithi107 ceph-mon[29153]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T00:12:20.729 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:20 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:12:20.730 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi170:vg_nvme/lv_1 2024-04-25T00:12:20.979 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:12:21.398 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:21 smithi170 ceph-mon[32807]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:21.398 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:21 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:21.399 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:21 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:21.399 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:21 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:21.399 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:21 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:21.399 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:21 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:21.399 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:21 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:21.399 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:21 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:21.399 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:21 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:21.399 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:21 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:21.399 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:21 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:21.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:21 smithi195 ceph-mon[32833]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:21.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:21 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:21.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:21 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:21.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:21 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:21.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:21 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:21.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:21 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:21.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:21 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:21.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:21 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:21.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:21 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:21.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:21 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:21.591 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:21 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:21.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:21 smithi107 ceph-mon[29153]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:21.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:21 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:21.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:21 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:21.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:21 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:21.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:21 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:21.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:21 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:21.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:21 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:21.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:21 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:21.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:21 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:21.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:21 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:21.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:21 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:22.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.566+0000 7fd431736700 1 -- 172.21.15.170:0/2530756043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd42c102380 msgr2=0x7fd42c1027e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:22.570 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.566+0000 7fd431736700 1 --2- 172.21.15.170:0/2530756043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd42c102380 0x7fd42c1027e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd4200099c0 tx=0x7fd420009cd0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.566+0000 7fd431736700 1 -- 172.21.15.170:0/2530756043 shutdown_connections 2024-04-25T00:12:22.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.566+0000 7fd431736700 1 --2- 172.21.15.170:0/2530756043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd42c102d20 0x7fd42c1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.566+0000 7fd431736700 1 --2- 172.21.15.170:0/2530756043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd42c101180 0x7fd42c1015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.566+0000 7fd431736700 1 --2- 172.21.15.170:0/2530756043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd42c102380 0x7fd42c1027e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.566+0000 7fd431736700 1 -- 172.21.15.170:0/2530756043 >> 172.21.15.170:0/2530756043 conn(0x7fd42c0fc720 msgr2=0x7fd42c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:22.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.567+0000 7fd431736700 1 -- 172.21.15.170:0/2530756043 shutdown_connections 2024-04-25T00:12:22.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.567+0000 7fd431736700 1 -- 172.21.15.170:0/2530756043 wait complete. 2024-04-25T00:12:22.571 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.567+0000 7fd431736700 1 Processor -- start 2024-04-25T00:12:22.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.567+0000 7fd431736700 1 -- start start 2024-04-25T00:12:22.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.567+0000 7fd431736700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd42c101180 0x7fd42c197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:22.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.567+0000 7fd431736700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd42c102380 0x7fd42c1978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:22.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.567+0000 7fd431736700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd42c102d20 0x7fd42c19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:22.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.567+0000 7fd431736700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c10cdd0 con 0x7fd42c101180 2024-04-25T00:12:22.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.567+0000 7fd431736700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c10cc50 con 0x7fd42c102380 2024-04-25T00:12:22.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.568+0000 7fd431736700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd42c10cf50 con 0x7fd42c102d20 2024-04-25T00:12:22.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.568+0000 7fd42a7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd42c102380 0x7fd42c1978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:22.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.568+0000 7fd42a7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd42c102380 0x7fd42c1978a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46268/0 (socket says 172.21.15.170:46268) 2024-04-25T00:12:22.572 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.568+0000 7fd42a7fc700 1 -- 172.21.15.170:0/1528985042 learned_addr learned my addr 172.21.15.170:0/1528985042 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-04-25T00:12:22.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.568+0000 7fd42b7fe700 1 --2- 172.21.15.170:0/1528985042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd42c102d20 0x7fd42c19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:22.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.568+0000 7fd42affd700 1 --2- 172.21.15.170:0/1528985042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd42c101180 0x7fd42c197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:22.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.568+0000 7fd42a7fc700 1 -- 172.21.15.170:0/1528985042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd42c102d20 msgr2=0x7fd42c19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:22.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.568+0000 7fd42a7fc700 1 --2- 172.21.15.170:0/1528985042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd42c102d20 0x7fd42c19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.568+0000 7fd42a7fc700 1 -- 172.21.15.170:0/1528985042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd42c101180 msgr2=0x7fd42c197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:22.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.568+0000 7fd42a7fc700 1 --2- 172.21.15.170:0/1528985042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd42c101180 0x7fd42c197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:22.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.568+0000 7fd42a7fc700 1 -- 172.21.15.170:0/1528985042 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd42c19efc0 con 0x7fd42c102380 2024-04-25T00:12:22.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.568+0000 7fd42affd700 1 --2- 172.21.15.170:0/1528985042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd42c101180 0x7fd42c197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:12:22.573 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.568+0000 7fd42a7fc700 1 --2- 172.21.15.170:0/1528985042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd42c102380 0x7fd42c1978a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd420003890 tx=0x7fd4200047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:22.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.568+0000 7fd413fff700 1 -- 172.21.15.170:0/1528985042 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd420027070 con 0x7fd42c102380 2024-04-25T00:12:22.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.568+0000 7fd431736700 1 -- 172.21.15.170:0/1528985042 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd42c19f250 con 0x7fd42c102380 2024-04-25T00:12:22.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.568+0000 7fd413fff700 1 -- 172.21.15.170:0/1528985042 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4200041d0 con 0x7fd42c102380 2024-04-25T00:12:22.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.568+0000 7fd413fff700 1 -- 172.21.15.170:0/1528985042 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42000f460 con 0x7fd42c102380 2024-04-25T00:12:22.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.568+0000 7fd431736700 1 -- 172.21.15.170:0/1528985042 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd42c19f720 con 0x7fd42c102380 2024-04-25T00:12:22.574 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.569+0000 7fd431736700 1 -- 172.21.15.170:0/1528985042 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd42c068f20 con 0x7fd42c102380 2024-04-25T00:12:22.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.571+0000 7fd413fff700 1 -- 172.21.15.170:0/1528985042 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fd42002a070 con 0x7fd42c102380 2024-04-25T00:12:22.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.571+0000 7fd413fff700 1 --2- 172.21.15.170:0/1528985042 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd4140a2c20 0x7fd4140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:22.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.572+0000 7fd413fff700 1 -- 172.21.15.170:0/1528985042 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7fd4200cf920 con 0x7fd42c102380 2024-04-25T00:12:22.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.572+0000 7fd42affd700 1 --2- 172.21.15.170:0/1528985042 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd4140a2c20 0x7fd4140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:22.575 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.572+0000 7fd42affd700 1 --2- 172.21.15.170:0/1528985042 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd4140a2c20 0x7fd4140a50e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd42c1021e0 tx=0x7fd41c00a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:22.576 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.575+0000 7fd413fff700 1 -- 172.21.15.170:0/1528985042 <== mon.2 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd420013070 con 0x7fd42c102380 2024-04-25T00:12:22.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:22 smithi195 ceph-mon[32833]: Detected new or changed devices on smithi107 2024-04-25T00:12:22.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:22 smithi195 ceph-mon[32833]: Adjusting osd_memory_target on smithi107 to 4272M 2024-04-25T00:12:22.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:22 smithi195 ceph-mon[32833]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T00:12:22.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:22 smithi170 ceph-mon[32807]: Detected new or changed devices on smithi107 2024-04-25T00:12:22.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:22 smithi170 ceph-mon[32807]: Adjusting osd_memory_target on smithi107 to 4272M 2024-04-25T00:12:22.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:22 smithi170 ceph-mon[32807]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T00:12:22.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:22 smithi107 ceph-mon[29153]: Detected new or changed devices on smithi107 2024-04-25T00:12:22.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:22 smithi107 ceph-mon[29153]: Adjusting osd_memory_target on smithi107 to 4272M 2024-04-25T00:12:22.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:22 smithi107 ceph-mon[29153]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T00:12:22.784 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:22.784+0000 7fd431736700 1 -- 172.21.15.170:0/1528985042 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd42c062670 con 0x7fd4140a2c20 2024-04-25T00:12:23.592 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:23 smithi170 ceph-mon[32807]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:12:23.593 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:23 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:12:23.593 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:23 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:12:23.593 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:23 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:23.593 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:23 smithi170 ceph-mon[32807]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:23.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:23 smithi107 ceph-mon[29153]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:12:23.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:23 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:12:23.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:23 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:12:23.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:23 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:23.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:23 smithi107 ceph-mon[29153]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:23.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:23 smithi195 ceph-mon[32833]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:12:24.025 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:23 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:12:24.025 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:23 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:12:24.025 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:23 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:24.025 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:23 smithi195 ceph-mon[32833]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:26.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:25 smithi170 ceph-mon[32807]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:26.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:25 smithi170 ceph-mon[32807]: from='client.? 172.21.15.170:0/1461659242' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4acd7b2f-39b0-44d8-8d3a-69d58cbd2226"}]: dispatch 2024-04-25T00:12:26.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:25 smithi170 ceph-mon[32807]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4acd7b2f-39b0-44d8-8d3a-69d58cbd2226"}]: dispatch 2024-04-25T00:12:26.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:25 smithi170 ceph-mon[32807]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4acd7b2f-39b0-44d8-8d3a-69d58cbd2226"}]': finished 2024-04-25T00:12:26.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:25 smithi170 ceph-mon[32807]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T00:12:26.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:25 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:12:26.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:25 smithi107 ceph-mon[29153]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:26.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:25 smithi107 ceph-mon[29153]: from='client.? 172.21.15.170:0/1461659242' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4acd7b2f-39b0-44d8-8d3a-69d58cbd2226"}]: dispatch 2024-04-25T00:12:26.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:25 smithi107 ceph-mon[29153]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4acd7b2f-39b0-44d8-8d3a-69d58cbd2226"}]: dispatch 2024-04-25T00:12:26.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:25 smithi107 ceph-mon[29153]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4acd7b2f-39b0-44d8-8d3a-69d58cbd2226"}]': finished 2024-04-25T00:12:26.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:25 smithi107 ceph-mon[29153]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T00:12:26.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:25 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:12:26.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:25 smithi195 ceph-mon[32833]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:26.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:25 smithi195 ceph-mon[32833]: from='client.? 172.21.15.170:0/1461659242' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4acd7b2f-39b0-44d8-8d3a-69d58cbd2226"}]: dispatch 2024-04-25T00:12:26.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:25 smithi195 ceph-mon[32833]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4acd7b2f-39b0-44d8-8d3a-69d58cbd2226"}]: dispatch 2024-04-25T00:12:26.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:25 smithi195 ceph-mon[32833]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4acd7b2f-39b0-44d8-8d3a-69d58cbd2226"}]': finished 2024-04-25T00:12:26.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:25 smithi195 ceph-mon[32833]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T00:12:26.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:25 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:12:27.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:26 smithi107 ceph-mon[29153]: from='client.? 172.21.15.170:0/1075322700' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:12:27.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:26 smithi195 ceph-mon[32833]: from='client.? 172.21.15.170:0/1075322700' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:12:27.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:26 smithi170 ceph-mon[32807]: from='client.? 172.21.15.170:0/1075322700' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:12:28.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:27 smithi107 ceph-mon[29153]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:28.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:27 smithi195 ceph-mon[32833]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:28.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:27 smithi170 ceph-mon[32807]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:30.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:29 smithi107 ceph-mon[29153]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:30.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:29 smithi195 ceph-mon[32833]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:30.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:29 smithi170 ceph-mon[32807]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:32.193 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:31 smithi170 ceph-mon[32807]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:32.240 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:31 smithi107 ceph-mon[29153]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:32.348 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:31 smithi195 ceph-mon[32833]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:33.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:32 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T00:12:33.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:32 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:33.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:32 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T00:12:33.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:32 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:33.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:32 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T00:12:33.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:32 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:34.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:33 smithi170 ceph-mon[32807]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:34.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:33 smithi170 ceph-mon[32807]: Deploying daemon osd.10 on smithi170 2024-04-25T00:12:34.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:33 smithi107 ceph-mon[29153]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:34.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:33 smithi107 ceph-mon[29153]: Deploying daemon osd.10 on smithi170 2024-04-25T00:12:34.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:33 smithi195 ceph-mon[32833]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:34.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:33 smithi195 ceph-mon[32833]: Deploying daemon osd.10 on smithi170 2024-04-25T00:12:36.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:35 smithi107 ceph-mon[29153]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:36.292 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:35 smithi170 ceph-mon[32807]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:36.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:35 smithi195 ceph-mon[32833]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:38.032 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:37 smithi170 ceph-mon[32807]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:38.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:37 smithi107 ceph-mon[29153]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:38.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:37 smithi195 ceph-mon[32833]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:39.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:39 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:39.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:39 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:39.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:39 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:39.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:39 smithi170 ceph-mon[32807]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:39.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:39 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:39.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:39 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:39.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:39 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:39.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:39 smithi107 ceph-mon[29153]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:40.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:39 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:40.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:39 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:40.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:39 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:40.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:39 smithi195 ceph-mon[32833]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:42.090 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:41 smithi170 ceph-mon[32807]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:42.149 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 10 on host 'smithi170' 2024-04-25T00:12:42.150 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:42.148+0000 7fd413fff700 1 -- 172.21.15.170:0/1528985042 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd42c062670 con 0x7fd4140a2c20 2024-04-25T00:12:42.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:42.151+0000 7fd431736700 1 -- 172.21.15.170:0/1528985042 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd4140a2c20 msgr2=0x7fd4140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:42.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:42.151+0000 7fd431736700 1 --2- 172.21.15.170:0/1528985042 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd4140a2c20 0x7fd4140a50e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd42c1021e0 tx=0x7fd41c00a680 comp rx=0 tx=0).stop 2024-04-25T00:12:42.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:42.151+0000 7fd431736700 1 -- 172.21.15.170:0/1528985042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd42c102380 msgr2=0x7fd42c1978a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:42.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:42.151+0000 7fd431736700 1 --2- 172.21.15.170:0/1528985042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd42c102380 0x7fd42c1978a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd420003890 tx=0x7fd4200047b0 comp rx=0 tx=0).stop 2024-04-25T00:12:42.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:42.151+0000 7fd431736700 1 -- 172.21.15.170:0/1528985042 shutdown_connections 2024-04-25T00:12:42.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:42.151+0000 7fd431736700 1 --2- 172.21.15.170:0/1528985042 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd4140a2c20 0x7fd4140a50e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:42.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:42.151+0000 7fd431736700 1 --2- 172.21.15.170:0/1528985042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd42c102d20 0x7fd42c19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:42.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:42.151+0000 7fd431736700 1 --2- 172.21.15.170:0/1528985042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd42c101180 0x7fd42c197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:42.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:42.151+0000 7fd431736700 1 --2- 172.21.15.170:0/1528985042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd42c102380 0x7fd42c1978a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:42.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:42.151+0000 7fd431736700 1 -- 172.21.15.170:0/1528985042 >> 172.21.15.170:0/1528985042 conn(0x7fd42c0fc720 msgr2=0x7fd42c105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:42.152 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:42.151+0000 7fd431736700 1 -- 172.21.15.170:0/1528985042 shutdown_connections 2024-04-25T00:12:42.153 INFO:teuthology.orchestra.run.smithi170.stderr:2024-04-25T00:12:42.151+0000 7fd431736700 1 -- 172.21.15.170:0/1528985042 wait complete. 2024-04-25T00:12:42.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:41 smithi107 ceph-mon[29153]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:42.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:41 smithi195 ceph-mon[32833]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:42.856 DEBUG:teuthology.orchestra.run.smithi170:osd.10> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T00:12:42.859 INFO:tasks.cephadm:Deploying osd.11 on smithi195 with /dev/vg_nvme/lv_1... 2024-04-25T00:12:42.859 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T00:12:43.123 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:12:43.156 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:43 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:43.156 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:43 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:43.156 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:43 smithi195 ceph-mon[32833]: from='osd.10 [v2:172.21.15.170:6824/1847892477,v1:172.21.15.170:6825/1847892477]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T00:12:43.156 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:43 smithi195 ceph-mon[32833]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T00:12:43.156 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:43 smithi195 ceph-mon[32833]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:43.156 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:43 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:43.156 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:43 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:43.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:43 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:43.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:43 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:43.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:43 smithi170 ceph-mon[32807]: from='osd.10 [v2:172.21.15.170:6824/1847892477,v1:172.21.15.170:6825/1847892477]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T00:12:43.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:43 smithi170 ceph-mon[32807]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T00:12:43.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:43 smithi170 ceph-mon[32807]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:43.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:43 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:43.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:43 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:43.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:43 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:43.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:43 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:43.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:43 smithi107 ceph-mon[29153]: from='osd.10 [v2:172.21.15.170:6824/1847892477,v1:172.21.15.170:6825/1847892477]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T00:12:43.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:43 smithi107 ceph-mon[29153]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T00:12:43.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:43 smithi107 ceph-mon[29153]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:43.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:43 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:43.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:43 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:44.246 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:44 smithi195 ceph-mon[32833]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T00:12:44.247 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:44 smithi195 ceph-mon[32833]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T00:12:44.247 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:44 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:12:44.247 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:44 smithi195 ceph-mon[32833]: from='osd.10 [v2:172.21.15.170:6824/1847892477,v1:172.21.15.170:6825/1847892477]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-25T00:12:44.247 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:44 smithi195 ceph-mon[32833]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-25T00:12:44.399 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:44 smithi107 ceph-mon[29153]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T00:12:44.400 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:44 smithi107 ceph-mon[29153]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T00:12:44.400 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:44 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:12:44.400 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:44 smithi107 ceph-mon[29153]: from='osd.10 [v2:172.21.15.170:6824/1847892477,v1:172.21.15.170:6825/1847892477]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-25T00:12:44.400 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:44 smithi107 ceph-mon[29153]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-25T00:12:44.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:44 smithi170 ceph-mon[32807]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T00:12:44.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:44 smithi170 ceph-mon[32807]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T00:12:44.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:44 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:12:44.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:44 smithi170 ceph-mon[32807]: from='osd.10 [v2:172.21.15.170:6824/1847892477,v1:172.21.15.170:6825/1847892477]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-25T00:12:44.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:44 smithi170 ceph-mon[32807]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-04-25T00:12:44.935 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:12:44 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:12:44.706+0000 7fd9f3092700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T00:12:44.935 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:12:44 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:12:44.718+0000 7fd9eae7d700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:12:45.074 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T00:12:45.074 INFO:teuthology.orchestra.run.smithi195.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T00:12:45.074 INFO:teuthology.orchestra.run.smithi195.stderr: stderr: 10+0 records in 2024-04-25T00:12:45.075 INFO:teuthology.orchestra.run.smithi195.stderr:10+0 records out 2024-04-25T00:12:45.075 INFO:teuthology.orchestra.run.smithi195.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00854971 s, 1.2 GB/s 2024-04-25T00:12:45.075 INFO:teuthology.orchestra.run.smithi195.stderr:--> Zapping successful for: 2024-04-25T00:12:45.404 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:45 smithi195 ceph-mon[32833]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-04-25T00:12:45.405 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:45 smithi195 ceph-mon[32833]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T00:12:45.405 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:45 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:12:45.405 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:45 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:12:45.405 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:45 smithi195 ceph-mon[32833]: from='osd.10 ' entity='osd.10' 2024-04-25T00:12:45.405 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:45 smithi195 ceph-mon[32833]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:45.405 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:45 smithi195 ceph-mon[32833]: osd.10 [v2:172.21.15.170:6824/1847892477,v1:172.21.15.170:6825/1847892477] boot 2024-04-25T00:12:45.405 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:45 smithi195 ceph-mon[32833]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T00:12:45.405 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:45 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:12:45.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:45 smithi170 ceph-mon[32807]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-04-25T00:12:45.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:45 smithi170 ceph-mon[32807]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T00:12:45.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:45 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:12:45.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:45 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:12:45.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:45 smithi170 ceph-mon[32807]: from='osd.10 ' entity='osd.10' 2024-04-25T00:12:45.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:45 smithi170 ceph-mon[32807]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:45.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:45 smithi170 ceph-mon[32807]: osd.10 [v2:172.21.15.170:6824/1847892477,v1:172.21.15.170:6825/1847892477] boot 2024-04-25T00:12:45.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:45 smithi170 ceph-mon[32807]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T00:12:45.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:45 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:12:45.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:45 smithi107 ceph-mon[29153]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-04-25T00:12:45.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:45 smithi107 ceph-mon[29153]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T00:12:45.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:45 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:12:45.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:45 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:12:45.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:45 smithi107 ceph-mon[29153]: from='osd.10 ' entity='osd.10' 2024-04-25T00:12:45.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:45 smithi107 ceph-mon[29153]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:12:45.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:45 smithi107 ceph-mon[29153]: osd.10 [v2:172.21.15.170:6824/1847892477,v1:172.21.15.170:6825/1847892477] boot 2024-04-25T00:12:45.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:45 smithi107 ceph-mon[29153]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T00:12:45.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:45 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:12:45.898 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi195:vg_nvme/lv_1 2024-04-25T00:12:46.137 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:12:46.404 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:46 smithi170 ceph-mon[32807]: purged_snaps scrub starts 2024-04-25T00:12:46.405 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:46 smithi170 ceph-mon[32807]: purged_snaps scrub ok 2024-04-25T00:12:46.484 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:46 smithi195 ceph-mon[32833]: purged_snaps scrub starts 2024-04-25T00:12:46.484 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:46 smithi195 ceph-mon[32833]: purged_snaps scrub ok 2024-04-25T00:12:46.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:46 smithi107 ceph-mon[29153]: purged_snaps scrub starts 2024-04-25T00:12:46.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:46 smithi107 ceph-mon[29153]: purged_snaps scrub ok 2024-04-25T00:12:47.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:47 smithi170 ceph-mon[32807]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T00:12:47.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:47 smithi170 ceph-mon[32807]: Detected new or changed devices on smithi170 2024-04-25T00:12:47.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:47.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:47.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:47.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:47.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:47.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:47.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:47 smithi170 ceph-mon[32807]: Adjusting osd_memory_target on smithi170 to 4272M 2024-04-25T00:12:47.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:47.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:47.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:47.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:47.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:47.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:47.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:47.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:47.441 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:47 smithi170 ceph-mon[32807]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 731 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:12:47.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:47 smithi107 ceph-mon[29153]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T00:12:47.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:47 smithi107 ceph-mon[29153]: Detected new or changed devices on smithi170 2024-04-25T00:12:47.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:47.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:47.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:47.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:47.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:47.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:47.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:47 smithi107 ceph-mon[29153]: Adjusting osd_memory_target on smithi170 to 4272M 2024-04-25T00:12:47.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:47.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:47.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:47.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:47.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:47.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:47.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:47.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:47.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:47 smithi107 ceph-mon[29153]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 731 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:12:47.494 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:47 smithi195 ceph-mon[32833]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T00:12:47.494 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:47 smithi195 ceph-mon[32833]: Detected new or changed devices on smithi170 2024-04-25T00:12:47.494 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:47.494 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:47.494 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:47.494 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:47.494 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:47.495 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:47.495 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:47 smithi195 ceph-mon[32833]: Adjusting osd_memory_target on smithi170 to 4272M 2024-04-25T00:12:47.495 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:47.495 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:47.495 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:47.495 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:47.495 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:47.495 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:47.495 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:47.495 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:12:47.496 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:47 smithi195 ceph-mon[32833]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 731 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:12:47.872 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.870+0000 7f2f95669700 1 -- 172.21.15.195:0/977799120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f90102290 msgr2=0x7f2f901026b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:47.872 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.870+0000 7f2f95669700 1 --2- 172.21.15.195:0/977799120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f90102290 0x7f2f901026b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009990 tx=0x7f2f78009ca0 comp rx=0 tx=0).stop 2024-04-25T00:12:47.872 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.870+0000 7f2f95669700 1 -- 172.21.15.195:0/977799120 shutdown_connections 2024-04-25T00:12:47.872 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.870+0000 7f2f95669700 1 --2- 172.21.15.195:0/977799120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f90103e30 0x7f2f9010a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:47.873 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.870+0000 7f2f95669700 1 --2- 172.21.15.195:0/977799120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f90102290 0x7f2f901026b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:47.873 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.870+0000 7f2f95669700 1 --2- 172.21.15.195:0/977799120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f90103490 0x7f2f901038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:47.873 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.870+0000 7f2f95669700 1 -- 172.21.15.195:0/977799120 >> 172.21.15.195:0/977799120 conn(0x7f2f900fd830 msgr2=0x7f2f900ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:47.873 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.870+0000 7f2f95669700 1 -- 172.21.15.195:0/977799120 shutdown_connections 2024-04-25T00:12:47.873 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.870+0000 7f2f95669700 1 -- 172.21.15.195:0/977799120 wait complete. 2024-04-25T00:12:47.873 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.871+0000 7f2f95669700 1 Processor -- start 2024-04-25T00:12:47.873 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.871+0000 7f2f95669700 1 -- start start 2024-04-25T00:12:47.875 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.871+0000 7f2f95669700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f90102290 0x7f2f90198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:47.875 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.871+0000 7f2f95669700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f90103490 0x7f2f90198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:47.875 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.871+0000 7f2f95669700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f90103e30 0x7f2f9019faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:47.876 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.871+0000 7f2f95669700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9010def0 con 0x7f2f90103490 2024-04-25T00:12:47.876 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.871+0000 7f2f95669700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9010dd70 con 0x7f2f90102290 2024-04-25T00:12:47.876 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.871+0000 7f2f95669700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9010e070 con 0x7f2f90103e30 2024-04-25T00:12:47.876 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.871+0000 7f2f8f7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f90103e30 0x7f2f9019faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:47.876 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.871+0000 7f2f8f7fe700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f90103e30 0x7f2f9019faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:40272/0 (socket says 172.21.15.195:40272) 2024-04-25T00:12:47.876 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.871+0000 7f2f8f7fe700 1 -- 172.21.15.195:0/3288109825 learned_addr learned my addr 172.21.15.195:0/3288109825 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-04-25T00:12:47.876 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.871+0000 7f2f8effd700 1 --2- 172.21.15.195:0/3288109825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f90102290 0x7f2f90198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:47.876 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.872+0000 7f2f8e7fc700 1 --2- 172.21.15.195:0/3288109825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f90103490 0x7f2f90198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:47.876 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.872+0000 7f2f8f7fe700 1 -- 172.21.15.195:0/3288109825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f90102290 msgr2=0x7f2f90198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:47.876 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.872+0000 7f2f8f7fe700 1 --2- 172.21.15.195:0/3288109825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f90102290 0x7f2f90198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:47.876 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.872+0000 7f2f8f7fe700 1 -- 172.21.15.195:0/3288109825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f90103490 msgr2=0x7f2f90198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:47.876 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.872+0000 7f2f8f7fe700 1 --2- 172.21.15.195:0/3288109825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f90103490 0x7f2f90198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:47.876 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.872+0000 7f2f8f7fe700 1 -- 172.21.15.195:0/3288109825 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f901a0190 con 0x7f2f90103e30 2024-04-25T00:12:47.877 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.872+0000 7f2f8f7fe700 1 --2- 172.21.15.195:0/3288109825 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f90103e30 0x7f2f9019faf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2f88007f10 tx=0x7f2f8800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:47.877 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.872+0000 7f2f87fff700 1 -- 172.21.15.195:0/3288109825 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f88004bb0 con 0x7f2f90103e30 2024-04-25T00:12:47.877 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.872+0000 7f2f87fff700 1 -- 172.21.15.195:0/3288109825 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f88004d50 con 0x7f2f90103e30 2024-04-25T00:12:47.877 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.872+0000 7f2f95669700 1 -- 172.21.15.195:0/3288109825 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f901a0480 con 0x7f2f90103e30 2024-04-25T00:12:47.877 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.872+0000 7f2f87fff700 1 -- 172.21.15.195:0/3288109825 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f880059f0 con 0x7f2f90103e30 2024-04-25T00:12:47.877 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.872+0000 7f2f95669700 1 -- 172.21.15.195:0/3288109825 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f901a0a40 con 0x7f2f90103e30 2024-04-25T00:12:47.877 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.873+0000 7f2f95669700 1 -- 172.21.15.195:0/3288109825 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f90068f20 con 0x7f2f90103e30 2024-04-25T00:12:47.877 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.874+0000 7f2f87fff700 1 -- 172.21.15.195:0/3288109825 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f2f8801a020 con 0x7f2f90103e30 2024-04-25T00:12:47.878 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.875+0000 7f2f87fff700 1 --2- 172.21.15.195:0/3288109825 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f2f7c0a2c20 0x7f2f7c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:47.878 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.875+0000 7f2f87fff700 1 -- 172.21.15.195:0/3288109825 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6723+0+0 (secure 0 0 0) 0x7f2f880ce7b0 con 0x7f2f90103e30 2024-04-25T00:12:47.878 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.875+0000 7f2f8effd700 1 --2- 172.21.15.195:0/3288109825 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f2f7c0a2c20 0x7f2f7c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:47.878 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.875+0000 7f2f8effd700 1 --2- 172.21.15.195:0/3288109825 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f2f7c0a2c20 0x7f2f7c0a50e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2f78003890 tx=0x7f2f78005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:47.879 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:47.878+0000 7f2f87fff700 1 -- 172.21.15.195:0/3288109825 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2f88010070 con 0x7f2f90103e30 2024-04-25T00:12:48.081 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:12:48.080+0000 7f2f95669700 1 -- 172.21.15.195:0/3288109825 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2f90062670 con 0x7f2f7c0a2c20 2024-04-25T00:12:48.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:48 smithi195 ceph-mon[32833]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T00:12:48.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:48 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:12:48.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:48 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:12:48.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:48 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:48.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:48 smithi170 ceph-mon[32807]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T00:12:48.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:48 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:12:48.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:48 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:12:48.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:48 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:48.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:48 smithi107 ceph-mon[29153]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T00:12:48.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:48 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:12:48.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:48 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:12:48.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:48 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:49.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:49 smithi195 ceph-mon[32833]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:12:49.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:49 smithi195 ceph-mon[32833]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T00:12:49.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:49 smithi195 ceph-mon[32833]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 732 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:12:49.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:49 smithi170 ceph-mon[32807]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:12:49.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:49 smithi170 ceph-mon[32807]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T00:12:49.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:49 smithi170 ceph-mon[32807]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 732 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:12:49.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:49 smithi107 ceph-mon[29153]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi195:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:12:49.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:49 smithi107 ceph-mon[29153]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T00:12:49.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:49 smithi107 ceph-mon[29153]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 732 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:12:50.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:50 smithi195 ceph-mon[32833]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T00:12:50.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:50 smithi170 ceph-mon[32807]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T00:12:50.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:50 smithi107 ceph-mon[29153]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T00:12:51.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:51 smithi170 ceph-mon[32807]: from='client.? 172.21.15.195:0/987642984' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61c096c8-f55a-442c-9fa8-00231fdbd258"}]: dispatch 2024-04-25T00:12:51.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:51 smithi170 ceph-mon[32807]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61c096c8-f55a-442c-9fa8-00231fdbd258"}]: dispatch 2024-04-25T00:12:51.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:51 smithi170 ceph-mon[32807]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61c096c8-f55a-442c-9fa8-00231fdbd258"}]': finished 2024-04-25T00:12:51.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:51 smithi170 ceph-mon[32807]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T00:12:51.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:51 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:12:51.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:51 smithi170 ceph-mon[32807]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:12:51.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:51 smithi107 ceph-mon[29153]: from='client.? 172.21.15.195:0/987642984' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61c096c8-f55a-442c-9fa8-00231fdbd258"}]: dispatch 2024-04-25T00:12:51.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:51 smithi107 ceph-mon[29153]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61c096c8-f55a-442c-9fa8-00231fdbd258"}]: dispatch 2024-04-25T00:12:51.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:51 smithi107 ceph-mon[29153]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61c096c8-f55a-442c-9fa8-00231fdbd258"}]': finished 2024-04-25T00:12:51.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:51 smithi107 ceph-mon[29153]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T00:12:51.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:51 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:12:51.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:51 smithi107 ceph-mon[29153]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:12:51.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:51 smithi195 ceph-mon[32833]: from='client.? 172.21.15.195:0/987642984' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61c096c8-f55a-442c-9fa8-00231fdbd258"}]: dispatch 2024-04-25T00:12:51.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:51 smithi195 ceph-mon[32833]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61c096c8-f55a-442c-9fa8-00231fdbd258"}]: dispatch 2024-04-25T00:12:51.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:51 smithi195 ceph-mon[32833]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61c096c8-f55a-442c-9fa8-00231fdbd258"}]': finished 2024-04-25T00:12:51.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:51 smithi195 ceph-mon[32833]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T00:12:51.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:51 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:12:51.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:51 smithi195 ceph-mon[32833]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T00:12:52.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:52 smithi170 ceph-mon[32807]: from='client.? 172.21.15.195:0/2199080466' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:12:52.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:52 smithi107 ceph-mon[29153]: from='client.? 172.21.15.195:0/2199080466' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:12:52.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:52 smithi195 ceph-mon[32833]: from='client.? 172.21.15.195:0/2199080466' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:12:53.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:53 smithi170 ceph-mon[32807]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-25T00:12:53.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:53 smithi107 ceph-mon[29153]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-25T00:12:53.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:53 smithi195 ceph-mon[32833]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-04-25T00:12:56.087 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:55 smithi195 ceph-mon[32833]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-25T00:12:56.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:55 smithi107 ceph-mon[29153]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-25T00:12:56.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:55 smithi170 ceph-mon[32807]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-04-25T00:12:58.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:57 smithi107 ceph-mon[29153]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:12:58.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:57 smithi195 ceph-mon[32833]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:12:58.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:57 smithi170 ceph-mon[32807]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:12:59.077 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:58 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T00:12:59.078 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:58 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:59.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:58 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T00:12:59.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:58 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:59.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:58 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T00:12:59.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:58 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:00.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:59 smithi107 ceph-mon[29153]: Deploying daemon osd.11 on smithi195 2024-04-25T00:13:00.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:12:59 smithi107 ceph-mon[29153]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:13:00.235 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:59 smithi195 ceph-mon[32833]: Deploying daemon osd.11 on smithi195 2024-04-25T00:13:00.235 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:12:59 smithi195 ceph-mon[32833]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:13:00.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:59 smithi170 ceph-mon[32807]: Deploying daemon osd.11 on smithi195 2024-04-25T00:13:00.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:12:59 smithi170 ceph-mon[32807]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:13:02.020 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:01 smithi195 ceph-mon[32833]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:13:02.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:01 smithi107 ceph-mon[29153]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:13:02.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:01 smithi170 ceph-mon[32807]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:13:04.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:03 smithi107 ceph-mon[29153]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:13:04.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:03 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:04.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:03 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:04.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:03 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:04.255 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:03 smithi195 ceph-mon[32833]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:13:04.256 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:03 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:04.256 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:03 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:04.256 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:03 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:04.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:03 smithi170 ceph-mon[32807]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:13:04.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:03 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:04.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:03 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:04.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:03 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:06.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:05 smithi107 ceph-mon[29153]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:13:06.264 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:05 smithi195 ceph-mon[32833]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:13:06.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:05 smithi170 ceph-mon[32807]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:13:07.503 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:07.501+0000 7f2f87fff700 1 -- 172.21.15.195:0/3288109825 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f2f90062670 con 0x7f2f7c0a2c20 2024-04-25T00:13:07.503 INFO:teuthology.orchestra.run.smithi195.stdout:Created osd(s) 11 on host 'smithi195' 2024-04-25T00:13:07.504 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:07.504+0000 7f2f95669700 1 -- 172.21.15.195:0/3288109825 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f2f7c0a2c20 msgr2=0x7f2f7c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.504 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:07.504+0000 7f2f95669700 1 --2- 172.21.15.195:0/3288109825 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f2f7c0a2c20 0x7f2f7c0a50e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2f78003890 tx=0x7f2f78005fd0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.505 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:07.504+0000 7f2f95669700 1 -- 172.21.15.195:0/3288109825 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f90103e30 msgr2=0x7f2f9019faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:07.505 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:07.504+0000 7f2f95669700 1 --2- 172.21.15.195:0/3288109825 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f90103e30 0x7f2f9019faf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2f88007f10 tx=0x7f2f8800d3b0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.505 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:07.504+0000 7f2f95669700 1 -- 172.21.15.195:0/3288109825 shutdown_connections 2024-04-25T00:13:07.505 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:07.504+0000 7f2f95669700 1 --2- 172.21.15.195:0/3288109825 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f2f7c0a2c20 0x7f2f7c0a50e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.505 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:07.504+0000 7f2f95669700 1 --2- 172.21.15.195:0/3288109825 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f90103e30 0x7f2f9019faf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.505 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:07.504+0000 7f2f95669700 1 --2- 172.21.15.195:0/3288109825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f90102290 0x7f2f90198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.505 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:07.504+0000 7f2f95669700 1 --2- 172.21.15.195:0/3288109825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f90103490 0x7f2f90198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:07.505 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:07.504+0000 7f2f95669700 1 -- 172.21.15.195:0/3288109825 >> 172.21.15.195:0/3288109825 conn(0x7f2f900fd830 msgr2=0x7f2f901067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:07.505 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:07.504+0000 7f2f95669700 1 -- 172.21.15.195:0/3288109825 shutdown_connections 2024-04-25T00:13:07.505 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:07.504+0000 7f2f95669700 1 -- 172.21.15.195:0/3288109825 wait complete. 2024-04-25T00:13:07.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:07 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:07.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:07 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:07.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:07 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:07.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:07 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:07.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:07 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:07.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:07 smithi195 ceph-mon[32833]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:13:07.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:07 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:07.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:07 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:07.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:07 smithi195 ceph-mon[32833]: from='osd.11 [v2:172.21.15.195:6824/645474884,v1:172.21.15.195:6825/645474884]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T00:13:07.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:07 smithi195 ceph-mon[32833]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T00:13:07.813 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:07 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:07.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:07 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:07.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:07 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:07.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:07 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:07.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:07 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:07.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:07 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:07.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:07 smithi170 ceph-mon[32807]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:13:07.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:07 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:07.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:07 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:07.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:07 smithi170 ceph-mon[32807]: from='osd.11 [v2:172.21.15.195:6824/645474884,v1:172.21.15.195:6825/645474884]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T00:13:07.936 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:07 smithi170 ceph-mon[32807]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T00:13:07.936 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:07 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:07.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:07 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:07.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:07 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:07.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:07 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:07.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:07 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:07.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:07 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:07.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:07 smithi107 ceph-mon[29153]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:13:07.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:07 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:07.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:07 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:07.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:07 smithi107 ceph-mon[29153]: from='osd.11 [v2:172.21.15.195:6824/645474884,v1:172.21.15.195:6825/645474884]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T00:13:07.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:07 smithi107 ceph-mon[29153]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T00:13:07.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:07 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:08.140 DEBUG:teuthology.orchestra.run.smithi195:osd.11> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T00:13:08.142 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T00:13:08.143 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T00:13:08.387 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:13:09.677 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:09 smithi195 ceph-mon[32833]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T00:13:09.677 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:09 smithi195 ceph-mon[32833]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T00:13:09.677 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:09 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:13:09.677 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:09 smithi195 ceph-mon[32833]: from='osd.11 [v2:172.21.15.195:6824/645474884,v1:172.21.15.195:6825/645474884]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-04-25T00:13:09.677 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:09 smithi195 ceph-mon[32833]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-04-25T00:13:09.677 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:09 smithi195 ceph-mon[32833]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:13:09.677 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:13:09 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:13:09.505+0000 7fc69f65f700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T00:13:09.678 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:13:09 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:13:09.519+0000 7fc69b452700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:13:09.804 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:09 smithi107 ceph-mon[29153]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T00:13:09.804 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:09 smithi107 ceph-mon[29153]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T00:13:09.804 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:09 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:13:09.804 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:09 smithi107 ceph-mon[29153]: from='osd.11 [v2:172.21.15.195:6824/645474884,v1:172.21.15.195:6825/645474884]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-04-25T00:13:09.804 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:09 smithi107 ceph-mon[29153]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-04-25T00:13:09.805 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:09 smithi107 ceph-mon[29153]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:13:09.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:09 smithi170 ceph-mon[32807]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T00:13:09.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:09 smithi170 ceph-mon[32807]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T00:13:09.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:09 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:13:09.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:09 smithi170 ceph-mon[32807]: from='osd.11 [v2:172.21.15.195:6824/645474884,v1:172.21.15.195:6825/645474884]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-04-25T00:13:09.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:09 smithi170 ceph-mon[32807]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]: dispatch 2024-04-25T00:13:09.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:09 smithi170 ceph-mon[32807]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:13:10.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.183+0000 7f91ef721700 1 -- 172.21.15.107:0/3014660974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e8103490 msgr2=0x7f91e81038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:10.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.183+0000 7f91ef721700 1 --2- 172.21.15.107:0/3014660974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e8103490 0x7f91e81038f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f91d80099c0 tx=0x7f91d8009cd0 comp rx=0 tx=0).stop 2024-04-25T00:13:10.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.183+0000 7f91ef721700 1 -- 172.21.15.107:0/3014660974 shutdown_connections 2024-04-25T00:13:10.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.183+0000 7f91ef721700 1 --2- 172.21.15.107:0/3014660974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91e8103e30 0x7f91e810a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:10.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.184+0000 7f91ef721700 1 --2- 172.21.15.107:0/3014660974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91e8102290 0x7f91e81026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:10.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.184+0000 7f91ef721700 1 --2- 172.21.15.107:0/3014660974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e8103490 0x7f91e81038f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:10.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.184+0000 7f91ef721700 1 -- 172.21.15.107:0/3014660974 >> 172.21.15.107:0/3014660974 conn(0x7f91e80fd830 msgr2=0x7f91e80ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:10.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.184+0000 7f91ef721700 1 -- 172.21.15.107:0/3014660974 shutdown_connections 2024-04-25T00:13:10.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.184+0000 7f91ef721700 1 -- 172.21.15.107:0/3014660974 wait complete. 2024-04-25T00:13:10.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.184+0000 7f91ef721700 1 Processor -- start 2024-04-25T00:13:10.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.184+0000 7f91ef721700 1 -- start start 2024-04-25T00:13:10.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.185+0000 7f91ef721700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91e8102290 0x7f91e81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:10.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.185+0000 7f91ef721700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e8103490 0x7f91e81989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:10.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.185+0000 7f91ef721700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91e8103e30 0x7f91e819fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:10.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.185+0000 7f91ef721700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f91e810de70 con 0x7f91e8103490 2024-04-25T00:13:10.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.185+0000 7f91ef721700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f91e810dcf0 con 0x7f91e8102290 2024-04-25T00:13:10.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.185+0000 7f91ef721700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f91e810dff0 con 0x7f91e8103e30 2024-04-25T00:13:10.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.185+0000 7f91eccbc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e8103490 0x7f91e81989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:10.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.185+0000 7f91eccbc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e8103490 0x7f91e81989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44802/0 (socket says 172.21.15.107:44802) 2024-04-25T00:13:10.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.185+0000 7f91eccbc700 1 -- 172.21.15.107:0/646922406 learned_addr learned my addr 172.21.15.107:0/646922406 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:10.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.185+0000 7f91edcbe700 1 --2- 172.21.15.107:0/646922406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91e8103e30 0x7f91e819fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:10.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.185+0000 7f91ed4bd700 1 --2- 172.21.15.107:0/646922406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91e8102290 0x7f91e81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:10.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.185+0000 7f91eccbc700 1 -- 172.21.15.107:0/646922406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91e8103e30 msgr2=0x7f91e819fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:10.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.185+0000 7f91eccbc700 1 --2- 172.21.15.107:0/646922406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91e8103e30 0x7f91e819fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:10.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.185+0000 7f91eccbc700 1 -- 172.21.15.107:0/646922406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91e8102290 msgr2=0x7f91e81984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:10.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.185+0000 7f91eccbc700 1 --2- 172.21.15.107:0/646922406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91e8102290 0x7f91e81984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:10.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.185+0000 7f91eccbc700 1 -- 172.21.15.107:0/646922406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e81a0110 con 0x7f91e8103490 2024-04-25T00:13:10.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.185+0000 7f91eccbc700 1 --2- 172.21.15.107:0/646922406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e8103490 0x7f91e81989f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f91d800f710 tx=0x7f91d800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:10.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.185+0000 7f91de7fc700 1 -- 172.21.15.107:0/646922406 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8027070 con 0x7f91e8103490 2024-04-25T00:13:10.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.185+0000 7f91ef721700 1 -- 172.21.15.107:0/646922406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e81a03a0 con 0x7f91e8103490 2024-04-25T00:13:10.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.185+0000 7f91de7fc700 1 -- 172.21.15.107:0/646922406 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91d800fe40 con 0x7f91e8103490 2024-04-25T00:13:10.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.185+0000 7f91de7fc700 1 -- 172.21.15.107:0/646922406 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d80173f0 con 0x7f91e8103490 2024-04-25T00:13:10.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.186+0000 7f91ef721700 1 -- 172.21.15.107:0/646922406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e81a0880 con 0x7f91e8103490 2024-04-25T00:13:10.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.186+0000 7f91ef721700 1 -- 172.21.15.107:0/646922406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91e8068f20 con 0x7f91e8103490 2024-04-25T00:13:10.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.187+0000 7f91de7fc700 1 -- 172.21.15.107:0/646922406 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f91d802a070 con 0x7f91e8103490 2024-04-25T00:13:10.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.188+0000 7f91de7fc700 1 --2- 172.21.15.107:0/646922406 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f91d40a2c20 0x7f91d40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:10.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.188+0000 7f91ed4bd700 1 --2- 172.21.15.107:0/646922406 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f91d40a2c20 0x7f91d40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:10.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.189+0000 7f91de7fc700 1 -- 172.21.15.107:0/646922406 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6847+0+0 (secure 0 0 0) 0x7f91d80cf6b0 con 0x7f91e8103490 2024-04-25T00:13:10.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.189+0000 7f91ed4bd700 1 --2- 172.21.15.107:0/646922406 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f91d40a2c20 0x7f91d40a50e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f91e81032f0 tx=0x7f91e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:10.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.193+0000 7f91de7fc700 1 -- 172.21.15.107:0/646922406 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f91d805e740 con 0x7f91e8103490 2024-04-25T00:13:10.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.390+0000 7f91ef721700 1 -- 172.21.15.107:0/646922406 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f91e8062e40 con 0x7f91e8103490 2024-04-25T00:13:10.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.391+0000 7f91de7fc700 1 -- 172.21.15.107:0/646922406 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f91d8013070 con 0x7f91e8103490 2024-04-25T00:13:10.391 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:13:10.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.393+0000 7f91ef721700 1 -- 172.21.15.107:0/646922406 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f91d40a2c20 msgr2=0x7f91d40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:10.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.393+0000 7f91ef721700 1 --2- 172.21.15.107:0/646922406 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f91d40a2c20 0x7f91d40a50e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f91e81032f0 tx=0x7f91e4009260 comp rx=0 tx=0).stop 2024-04-25T00:13:10.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.393+0000 7f91ef721700 1 -- 172.21.15.107:0/646922406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e8103490 msgr2=0x7f91e81989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:10.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.393+0000 7f91ef721700 1 --2- 172.21.15.107:0/646922406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e8103490 0x7f91e81989f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f91d800f710 tx=0x7f91d800f740 comp rx=0 tx=0).stop 2024-04-25T00:13:10.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.393+0000 7f91ef721700 1 -- 172.21.15.107:0/646922406 shutdown_connections 2024-04-25T00:13:10.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.393+0000 7f91ef721700 1 --2- 172.21.15.107:0/646922406 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f91d40a2c20 0x7f91d40a50e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:10.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.393+0000 7f91ef721700 1 --2- 172.21.15.107:0/646922406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f91e8103e30 0x7f91e819fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:10.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.393+0000 7f91ef721700 1 --2- 172.21.15.107:0/646922406 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91e8102290 0x7f91e81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:10.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.393+0000 7f91ef721700 1 --2- 172.21.15.107:0/646922406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91e8103490 0x7f91e81989f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:10.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.393+0000 7f91ef721700 1 -- 172.21.15.107:0/646922406 >> 172.21.15.107:0/646922406 conn(0x7f91e80fd830 msgr2=0x7f91e8106810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:10.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.393+0000 7f91ef721700 1 -- 172.21.15.107:0/646922406 shutdown_connections 2024-04-25T00:13:10.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:10.393+0000 7f91ef721700 1 -- 172.21.15.107:0/646922406 wait complete. 2024-04-25T00:13:10.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:10 smithi107 ceph-mon[29153]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-04-25T00:13:10.688 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:10 smithi107 ceph-mon[29153]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T00:13:10.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:10 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:13:10.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:10 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:13:10.689 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:10 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/646922406' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T00:13:10.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:10 smithi195 ceph-mon[32833]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-04-25T00:13:10.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:10 smithi195 ceph-mon[32833]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T00:13:10.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:10 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:13:10.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:10 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:13:10.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:10 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/646922406' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T00:13:10.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:10 smithi170 ceph-mon[32807]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi195", "root=default"]}]': finished 2024-04-25T00:13:10.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:10 smithi170 ceph-mon[32807]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T00:13:10.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:10 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:13:10.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:10 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:13:10.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:10 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/646922406' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T00:13:11.106 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":80,"num_osds":12,"num_up_osds":11,"osd_up_since":1714003965,"num_in_osds":12,"osd_in_since":1714003970,"num_remapped_pgs":0} 2024-04-25T00:13:11.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:11 smithi195 ceph-mon[32833]: purged_snaps scrub starts 2024-04-25T00:13:11.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:11 smithi195 ceph-mon[32833]: purged_snaps scrub ok 2024-04-25T00:13:11.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:11 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:13:11.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:11 smithi195 ceph-mon[32833]: osd.11 [v2:172.21.15.195:6824/645474884,v1:172.21.15.195:6825/645474884] boot 2024-04-25T00:13:11.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:11 smithi195 ceph-mon[32833]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T00:13:11.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:11 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:13:11.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:11 smithi195 ceph-mon[32833]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:13:11.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:11 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:11.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:11 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:11.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:11 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:11.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:11 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:11.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:11 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:11.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:11 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:11.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:11 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:11.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:11 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:11.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:11 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:11.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:11 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:11.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:11 smithi170 ceph-mon[32807]: purged_snaps scrub starts 2024-04-25T00:13:11.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:11 smithi170 ceph-mon[32807]: purged_snaps scrub ok 2024-04-25T00:13:11.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:11 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:13:11.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:11 smithi170 ceph-mon[32807]: osd.11 [v2:172.21.15.195:6824/645474884,v1:172.21.15.195:6825/645474884] boot 2024-04-25T00:13:11.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:11 smithi170 ceph-mon[32807]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T00:13:11.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:11 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:13:11.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:11 smithi170 ceph-mon[32807]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:13:11.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:11 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:11.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:11 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:11.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:11 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:11.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:11 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:11.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:11 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:11.936 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:11 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:11.936 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:11 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:11.936 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:11 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:11.936 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:11 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:11.936 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:11 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:11.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:11 smithi107 ceph-mon[29153]: purged_snaps scrub starts 2024-04-25T00:13:11.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:11 smithi107 ceph-mon[29153]: purged_snaps scrub ok 2024-04-25T00:13:11.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:11 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:13:11.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:11 smithi107 ceph-mon[29153]: osd.11 [v2:172.21.15.195:6824/645474884,v1:172.21.15.195:6825/645474884] boot 2024-04-25T00:13:11.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:11 smithi107 ceph-mon[29153]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T00:13:11.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:11 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:13:11.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:11 smithi107 ceph-mon[29153]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 337 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:13:11.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:11 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:11.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:11 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:11.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:11 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:11.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:11 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:11.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:11 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:11.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:11 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:11.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:11 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:11.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:11 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:11.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:11 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:11.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:11 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:12.107 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T00:13:12.346 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:13:12.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:12 smithi107 ceph-mon[29153]: Detected new or changed devices on smithi195 2024-04-25T00:13:12.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:12 smithi107 ceph-mon[29153]: Adjusting osd_memory_target on smithi195 to 4272M 2024-04-25T00:13:12.731 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:12 smithi107 ceph-mon[29153]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T00:13:12.836 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:12 smithi170 ceph-mon[32807]: Detected new or changed devices on smithi195 2024-04-25T00:13:12.836 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:12 smithi170 ceph-mon[32807]: Adjusting osd_memory_target on smithi195 to 4272M 2024-04-25T00:13:12.836 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:12 smithi170 ceph-mon[32807]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T00:13:12.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:12 smithi195 ceph-mon[32833]: Detected new or changed devices on smithi195 2024-04-25T00:13:12.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:12 smithi195 ceph-mon[32833]: Adjusting osd_memory_target on smithi195 to 4272M 2024-04-25T00:13:12.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:12 smithi195 ceph-mon[32833]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T00:13:13.768 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:13 smithi107 ceph-mon[29153]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:13.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:13 smithi195 ceph-mon[32833]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:13.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:13 smithi170 ceph-mon[32807]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:14.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.182+0000 7f747561e700 1 -- 172.21.15.107:0/260478901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74701034c0 msgr2=0x7f7470103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:14.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.182+0000 7f747561e700 1 --2- 172.21.15.107:0/260478901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74701034c0 0x7f7470103920 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f74640099c0 tx=0x7f7464009cd0 comp rx=0 tx=0).stop 2024-04-25T00:13:14.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.182+0000 7f747561e700 1 -- 172.21.15.107:0/260478901 shutdown_connections 2024-04-25T00:13:14.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.182+0000 7f747561e700 1 --2- 172.21.15.107:0/260478901 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7470103e60 0x7f747010a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:14.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.182+0000 7f747561e700 1 --2- 172.21.15.107:0/260478901 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74701022c0 0x7f74701026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:14.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.182+0000 7f747561e700 1 --2- 172.21.15.107:0/260478901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74701034c0 0x7f7470103920 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:14.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.182+0000 7f747561e700 1 -- 172.21.15.107:0/260478901 >> 172.21.15.107:0/260478901 conn(0x7f74700fd840 msgr2=0x7f74700ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:14.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.182+0000 7f747561e700 1 -- 172.21.15.107:0/260478901 shutdown_connections 2024-04-25T00:13:14.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.183+0000 7f747561e700 1 -- 172.21.15.107:0/260478901 wait complete. 2024-04-25T00:13:14.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.183+0000 7f747561e700 1 Processor -- start 2024-04-25T00:13:14.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.183+0000 7f747561e700 1 -- start start 2024-04-25T00:13:14.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.183+0000 7f747561e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74701022c0 0x7f7470075fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:14.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.183+0000 7f747561e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74701034c0 0x7f7470074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:14.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.183+0000 7f747561e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7470103e60 0x7f7470074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:14.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.183+0000 7f747561e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f747010dfa0 con 0x7f74701022c0 2024-04-25T00:13:14.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.183+0000 7f747561e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f747010de20 con 0x7f74701034c0 2024-04-25T00:13:14.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.183+0000 7f747561e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f747010e120 con 0x7f7470103e60 2024-04-25T00:13:14.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.184+0000 7f746effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74701022c0 0x7f7470075fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:14.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.184+0000 7f746effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74701022c0 0x7f7470075fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44820/0 (socket says 172.21.15.107:44820) 2024-04-25T00:13:14.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.184+0000 7f746effd700 1 -- 172.21.15.107:0/342540485 learned_addr learned my addr 172.21.15.107:0/342540485 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:14.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.184+0000 7f746f7fe700 1 --2- 172.21.15.107:0/342540485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7470103e60 0x7f7470074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:14.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.184+0000 7f746e7fc700 1 --2- 172.21.15.107:0/342540485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74701034c0 0x7f7470074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:14.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.184+0000 7f746effd700 1 -- 172.21.15.107:0/342540485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7470103e60 msgr2=0x7f7470074b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:14.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.184+0000 7f746effd700 1 --2- 172.21.15.107:0/342540485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7470103e60 0x7f7470074b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:14.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.184+0000 7f746effd700 1 -- 172.21.15.107:0/342540485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74701034c0 msgr2=0x7f7470074620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:14.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.184+0000 7f746effd700 1 --2- 172.21.15.107:0/342540485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74701034c0 0x7f7470074620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:14.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.184+0000 7f746effd700 1 -- 172.21.15.107:0/342540485 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7470075410 con 0x7f74701022c0 2024-04-25T00:13:14.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.184+0000 7f746f7fe700 1 --2- 172.21.15.107:0/342540485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7470103e60 0x7f7470074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:14.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.184+0000 7f746effd700 1 --2- 172.21.15.107:0/342540485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74701022c0 0x7f7470075fd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f746000b500 tx=0x7f746000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:14.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.184+0000 7f7457fff700 1 -- 172.21.15.107:0/342540485 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460015020 con 0x7f74701022c0 2024-04-25T00:13:14.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.184+0000 7f7457fff700 1 -- 172.21.15.107:0/342540485 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7460011d10 con 0x7f74701022c0 2024-04-25T00:13:14.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.184+0000 7f747561e700 1 -- 172.21.15.107:0/342540485 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74701a26e0 con 0x7f74701022c0 2024-04-25T00:13:14.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.184+0000 7f7457fff700 1 -- 172.21.15.107:0/342540485 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7460010e50 con 0x7f74701022c0 2024-04-25T00:13:14.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.184+0000 7f747561e700 1 -- 172.21.15.107:0/342540485 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74701a2ca0 con 0x7f74701022c0 2024-04-25T00:13:14.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.185+0000 7f747561e700 1 -- 172.21.15.107:0/342540485 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7470068f20 con 0x7f74701022c0 2024-04-25T00:13:14.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.186+0000 7f7457fff700 1 -- 172.21.15.107:0/342540485 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f7460018020 con 0x7f74701022c0 2024-04-25T00:13:14.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.187+0000 7f7457fff700 1 --2- 172.21.15.107:0/342540485 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f74580a2e40 0x7f74580a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:14.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.187+0000 7f746e7fc700 1 --2- 172.21.15.107:0/342540485 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f74580a2e40 0x7f74580a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:14.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.187+0000 7f7457fff700 1 -- 172.21.15.107:0/342540485 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f74600ce2e0 con 0x7f74701022c0 2024-04-25T00:13:14.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.187+0000 7f746e7fc700 1 --2- 172.21.15.107:0/342540485 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f74580a2e40 0x7f74580a5300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7464000c00 tx=0x7f7464005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:14.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.192+0000 7f7457fff700 1 -- 172.21.15.107:0/342540485 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f746001f020 con 0x7f74701022c0 2024-04-25T00:13:14.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.390+0000 7f747561e700 1 -- 172.21.15.107:0/342540485 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f747002db30 con 0x7f74701022c0 2024-04-25T00:13:14.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.390+0000 7f7457fff700 1 -- 172.21.15.107:0/342540485 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7f746009b6e0 con 0x7f74701022c0 2024-04-25T00:13:14.391 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:13:14.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.393+0000 7f747561e700 1 -- 172.21.15.107:0/342540485 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f74580a2e40 msgr2=0x7f74580a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:14.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.393+0000 7f747561e700 1 --2- 172.21.15.107:0/342540485 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f74580a2e40 0x7f74580a5300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7464000c00 tx=0x7f7464005fb0 comp rx=0 tx=0).stop 2024-04-25T00:13:14.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.393+0000 7f747561e700 1 -- 172.21.15.107:0/342540485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74701022c0 msgr2=0x7f7470075fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:14.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.393+0000 7f747561e700 1 --2- 172.21.15.107:0/342540485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74701022c0 0x7f7470075fd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f746000b500 tx=0x7f746000b810 comp rx=0 tx=0).stop 2024-04-25T00:13:14.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.393+0000 7f747561e700 1 -- 172.21.15.107:0/342540485 shutdown_connections 2024-04-25T00:13:14.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.393+0000 7f747561e700 1 --2- 172.21.15.107:0/342540485 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f74580a2e40 0x7f74580a5300 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:14.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.393+0000 7f747561e700 1 --2- 172.21.15.107:0/342540485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7470103e60 0x7f7470074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:14.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.393+0000 7f747561e700 1 --2- 172.21.15.107:0/342540485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74701022c0 0x7f7470075fd0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:14.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.393+0000 7f747561e700 1 --2- 172.21.15.107:0/342540485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74701034c0 0x7f7470074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:14.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.393+0000 7f747561e700 1 -- 172.21.15.107:0/342540485 >> 172.21.15.107:0/342540485 conn(0x7f74700fd840 msgr2=0x7f7470106760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:14.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.393+0000 7f747561e700 1 -- 172.21.15.107:0/342540485 shutdown_connections 2024-04-25T00:13:14.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:14.393+0000 7f747561e700 1 -- 172.21.15.107:0/342540485 wait complete. 2024-04-25T00:13:14.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:14 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/342540485' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T00:13:14.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:14 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/342540485' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T00:13:14.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:14 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/342540485' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T00:13:15.211 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1714003990,"num_in_osds":12,"osd_in_since":1714003970,"num_remapped_pgs":0} 2024-04-25T00:13:15.211 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T00:13:15.450 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:13:15.809 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:15 smithi107 ceph-mon[29153]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:15.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:15 smithi195 ceph-mon[32833]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:15.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:15 smithi170 ceph-mon[32807]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:17.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.137+0000 7f977a52c700 1 -- 172.21.15.107:0/1722928420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9774100bf0 msgr2=0x7f9774101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:17.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.137+0000 7f977a52c700 1 --2- 172.21.15.107:0/1722928420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9774100bf0 0x7f9774101070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9768009990 tx=0x7f9768009ca0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.137+0000 7f977a52c700 1 -- 172.21.15.107:0/1722928420 shutdown_connections 2024-04-25T00:13:17.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.137+0000 7f977a52c700 1 --2- 172.21.15.107:0/1722928420 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97741015b0 0x7f977410d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.137+0000 7f977a52c700 1 --2- 172.21.15.107:0/1722928420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97741069a0 0x7f9774106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.137+0000 7f977a52c700 1 --2- 172.21.15.107:0/1722928420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9774100bf0 0x7f9774101070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.137+0000 7f977a52c700 1 -- 172.21.15.107:0/1722928420 >> 172.21.15.107:0/1722928420 conn(0x7f9774076450 msgr2=0x7f9774076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:17.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.137+0000 7f977a52c700 1 -- 172.21.15.107:0/1722928420 shutdown_connections 2024-04-25T00:13:17.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.137+0000 7f977a52c700 1 -- 172.21.15.107:0/1722928420 wait complete. 2024-04-25T00:13:17.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.138+0000 7f977a52c700 1 Processor -- start 2024-04-25T00:13:17.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.138+0000 7f977a52c700 1 -- start start 2024-04-25T00:13:17.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.138+0000 7f977a52c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9774100bf0 0x7f97741024c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:17.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.138+0000 7f977a52c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97741015b0 0x7f9774102a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:17.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.138+0000 7f977a52c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97741069a0 0x7f9774102f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:17.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.138+0000 7f977a52c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f97741114f0 con 0x7f97741015b0 2024-04-25T00:13:17.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.138+0000 7f977a52c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9774111370 con 0x7f97741069a0 2024-04-25T00:13:17.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.138+0000 7f977a52c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9774111670 con 0x7f9774100bf0 2024-04-25T00:13:17.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.139+0000 7f97737fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97741015b0 0x7f9774102a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:17.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.139+0000 7f97737fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97741015b0 0x7f9774102a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44824/0 (socket says 172.21.15.107:44824) 2024-04-25T00:13:17.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.139+0000 7f97737fe700 1 -- 172.21.15.107:0/1241807595 learned_addr learned my addr 172.21.15.107:0/1241807595 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:17.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.139+0000 7f9773fff700 1 --2- 172.21.15.107:0/1241807595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9774100bf0 0x7f97741024c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:17.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.139+0000 7f9778ac9700 1 --2- 172.21.15.107:0/1241807595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97741069a0 0x7f9774102f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:17.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.139+0000 7f97737fe700 1 -- 172.21.15.107:0/1241807595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9774100bf0 msgr2=0x7f97741024c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:17.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.139+0000 7f97737fe700 1 --2- 172.21.15.107:0/1241807595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9774100bf0 0x7f97741024c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.139+0000 7f97737fe700 1 -- 172.21.15.107:0/1241807595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97741069a0 msgr2=0x7f9774102f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:17.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.139+0000 7f97737fe700 1 --2- 172.21.15.107:0/1241807595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97741069a0 0x7f9774102f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.139+0000 7f97737fe700 1 -- 172.21.15.107:0/1241807595 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97741a6e60 con 0x7f97741015b0 2024-04-25T00:13:17.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.139+0000 7f9773fff700 1 --2- 172.21.15.107:0/1241807595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9774100bf0 0x7f97741024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:17.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.139+0000 7f9778ac9700 1 --2- 172.21.15.107:0/1241807595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97741069a0 0x7f9774102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:17.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.139+0000 7f97737fe700 1 --2- 172.21.15.107:0/1241807595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97741015b0 0x7f9774102a00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f97680038b0 tx=0x7f9768005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:17.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.139+0000 7f97717fa700 1 -- 172.21.15.107:0/1241807595 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9768027070 con 0x7f97741015b0 2024-04-25T00:13:17.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.139+0000 7f97717fa700 1 -- 172.21.15.107:0/1241807595 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97680040d0 con 0x7f97741015b0 2024-04-25T00:13:17.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.139+0000 7f97717fa700 1 -- 172.21.15.107:0/1241807595 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9768022dd0 con 0x7f97741015b0 2024-04-25T00:13:17.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.139+0000 7f977a52c700 1 -- 172.21.15.107:0/1241807595 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97741a70f0 con 0x7f97741015b0 2024-04-25T00:13:17.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.139+0000 7f977a52c700 1 -- 172.21.15.107:0/1241807595 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97741a7580 con 0x7f97741015b0 2024-04-25T00:13:17.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.140+0000 7f977a52c700 1 -- 172.21.15.107:0/1241807595 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9774068f20 con 0x7f97741015b0 2024-04-25T00:13:17.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.141+0000 7f97717fa700 1 -- 172.21.15.107:0/1241807595 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f976802a070 con 0x7f97741015b0 2024-04-25T00:13:17.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.142+0000 7f97717fa700 1 --2- 172.21.15.107:0/1241807595 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f975c0a2ca0 0x7f975c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:17.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.142+0000 7f9773fff700 1 --2- 172.21.15.107:0/1241807595 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f975c0a2ca0 0x7f975c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:17.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.142+0000 7f97717fa700 1 -- 172.21.15.107:0/1241807595 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f97680cfed0 con 0x7f97741015b0 2024-04-25T00:13:17.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.142+0000 7f9773fff700 1 --2- 172.21.15.107:0/1241807595 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f975c0a2ca0 0x7f975c0a5160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9764005e10 tx=0x7f9764005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:17.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.145+0000 7f97717fa700 1 -- 172.21.15.107:0/1241807595 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9768031020 con 0x7f97741015b0 2024-04-25T00:13:17.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.346+0000 7f977a52c700 1 -- 172.21.15.107:0/1241807595 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f977402dcf0 con 0x7f97741015b0 2024-04-25T00:13:17.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.347+0000 7f97717fa700 1 -- 172.21.15.107:0/1241807595 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18462 (secure 0 0 0) 0x7f9768013070 con 0x7f97741015b0 2024-04-25T00:13:17.348 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:13:17.348 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":82,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:13:12.278840+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:13:17.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.349+0000 7f977a52c700 1 -- 172.21.15.107:0/1241807595 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f975c0a2ca0 msgr2=0x7f975c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:17.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.349+0000 7f977a52c700 1 --2- 172.21.15.107:0/1241807595 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f975c0a2ca0 0x7f975c0a5160 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9764005e10 tx=0x7f9764005d60 comp rx=0 tx=0).stop 2024-04-25T00:13:17.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.349+0000 7f977a52c700 1 -- 172.21.15.107:0/1241807595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97741015b0 msgr2=0x7f9774102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:17.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.349+0000 7f977a52c700 1 --2- 172.21.15.107:0/1241807595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97741015b0 0x7f9774102a00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f97680038b0 tx=0x7f9768005590 comp rx=0 tx=0).stop 2024-04-25T00:13:17.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.349+0000 7f977a52c700 1 -- 172.21.15.107:0/1241807595 shutdown_connections 2024-04-25T00:13:17.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.349+0000 7f977a52c700 1 --2- 172.21.15.107:0/1241807595 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f975c0a2ca0 0x7f975c0a5160 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.349+0000 7f977a52c700 1 --2- 172.21.15.107:0/1241807595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97741069a0 0x7f9774102f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.349+0000 7f977a52c700 1 --2- 172.21.15.107:0/1241807595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9774100bf0 0x7f97741024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.349+0000 7f977a52c700 1 --2- 172.21.15.107:0/1241807595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97741015b0 0x7f9774102a00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:17.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.349+0000 7f977a52c700 1 -- 172.21.15.107:0/1241807595 >> 172.21.15.107:0/1241807595 conn(0x7f9774076450 msgr2=0x7f977410e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:17.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.349+0000 7f977a52c700 1 -- 172.21.15.107:0/1241807595 shutdown_connections 2024-04-25T00:13:17.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:17.349+0000 7f977a52c700 1 -- 172.21.15.107:0/1241807595 wait complete. 2024-04-25T00:13:17.959 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:09:25.160329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:13:17.960 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T00:13:18.196 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:13:18.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:17 smithi107 ceph-mon[29153]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:18.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:17 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1241807595' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:13:18.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:17 smithi195 ceph-mon[32833]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:18.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:17 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1241807595' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:13:18.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:17 smithi170 ceph-mon[32807]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:18.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:17 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1241807595' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:13:19.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.933+0000 7f96004a8700 1 -- 172.21.15.107:0/1957156032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f806ac00 msgr2=0x7f95f806afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:19.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.933+0000 7f96004a8700 1 --2- 172.21.15.107:0/1957156032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f806ac00 0x7f95f806afe0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f95f40099c0 tx=0x7f95f4009cd0 comp rx=0 tx=0).stop 2024-04-25T00:13:19.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.934+0000 7f96004a8700 1 -- 172.21.15.107:0/1957156032 shutdown_connections 2024-04-25T00:13:19.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.934+0000 7f96004a8700 1 --2- 172.21.15.107:0/1957156032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95f8107570 0x7f95f810f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:19.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.934+0000 7f96004a8700 1 --2- 172.21.15.107:0/1957156032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f806ac00 0x7f95f806afe0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:19.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.934+0000 7f96004a8700 1 --2- 172.21.15.107:0/1957156032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95f806b5b0 0x7f95f8106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:19.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.934+0000 7f96004a8700 1 -- 172.21.15.107:0/1957156032 >> 172.21.15.107:0/1957156032 conn(0x7f95f8077180 msgr2=0x7f95f8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:19.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.934+0000 7f96004a8700 1 -- 172.21.15.107:0/1957156032 shutdown_connections 2024-04-25T00:13:19.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.934+0000 7f96004a8700 1 -- 172.21.15.107:0/1957156032 wait complete. 2024-04-25T00:13:19.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.934+0000 7f96004a8700 1 Processor -- start 2024-04-25T00:13:19.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.934+0000 7f96004a8700 1 -- start start 2024-04-25T00:13:19.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.934+0000 7f96004a8700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95f806ac00 0x7f95f8101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:19.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.934+0000 7f96004a8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95f806b5b0 0x7f95f81017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:19.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.934+0000 7f96004a8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f8107570 0x7f95f8101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:19.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.934+0000 7f96004a8700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f95f8112550 con 0x7f95f8107570 2024-04-25T00:13:19.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.935+0000 7f96004a8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f95f81123d0 con 0x7f95f806b5b0 2024-04-25T00:13:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.935+0000 7f96004a8700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f95f81126d0 con 0x7f95f806ac00 2024-04-25T00:13:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.935+0000 7f95fea45700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f8107570 0x7f95f8101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.935+0000 7f95fea45700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f8107570 0x7f95f8101d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46512/0 (socket says 172.21.15.107:46512) 2024-04-25T00:13:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.935+0000 7f95fea45700 1 -- 172.21.15.107:0/2202310384 learned_addr learned my addr 172.21.15.107:0/2202310384 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.935+0000 7f95fe244700 1 --2- 172.21.15.107:0/2202310384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95f806ac00 0x7f95f8101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.935+0000 7f95fda43700 1 --2- 172.21.15.107:0/2202310384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95f806b5b0 0x7f95f81017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.935+0000 7f95fea45700 1 -- 172.21.15.107:0/2202310384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95f806ac00 msgr2=0x7f95f8101290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.935+0000 7f95fea45700 1 --2- 172.21.15.107:0/2202310384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95f806ac00 0x7f95f8101290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.935+0000 7f95fea45700 1 -- 172.21.15.107:0/2202310384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95f806b5b0 msgr2=0x7f95f81017d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.935+0000 7f95fea45700 1 --2- 172.21.15.107:0/2202310384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95f806b5b0 0x7f95f81017d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.935+0000 7f95fea45700 1 -- 172.21.15.107:0/2202310384 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f81a6e90 con 0x7f95f8107570 2024-04-25T00:13:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.935+0000 7f95fea45700 1 --2- 172.21.15.107:0/2202310384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f8107570 0x7f95f8101d10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f95f0007d70 tx=0x7f95f000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:19.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.935+0000 7f95ef7fe700 1 -- 172.21.15.107:0/2202310384 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f0016020 con 0x7f95f8107570 2024-04-25T00:13:19.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.935+0000 7f96004a8700 1 -- 172.21.15.107:0/2202310384 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f81a7180 con 0x7f95f8107570 2024-04-25T00:13:19.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.935+0000 7f95ef7fe700 1 -- 172.21.15.107:0/2202310384 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95f000ad40 con 0x7f95f8107570 2024-04-25T00:13:19.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.935+0000 7f95ef7fe700 1 -- 172.21.15.107:0/2202310384 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f0011600 con 0x7f95f8107570 2024-04-25T00:13:19.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.935+0000 7f96004a8700 1 -- 172.21.15.107:0/2202310384 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f81a76c0 con 0x7f95f8107570 2024-04-25T00:13:19.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.937+0000 7f96004a8700 1 -- 172.21.15.107:0/2202310384 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f8068f20 con 0x7f95f8107570 2024-04-25T00:13:19.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.937+0000 7f95ef7fe700 1 -- 172.21.15.107:0/2202310384 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f95f0019020 con 0x7f95f8107570 2024-04-25T00:13:19.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.938+0000 7f95ef7fe700 1 --2- 172.21.15.107:0/2202310384 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f95e40a2c20 0x7f95e40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:19.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.938+0000 7f95fe244700 1 --2- 172.21.15.107:0/2202310384 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f95e40a2c20 0x7f95e40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:19.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.938+0000 7f95ef7fe700 1 -- 172.21.15.107:0/2202310384 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f95f00ceae0 con 0x7f95f8107570 2024-04-25T00:13:19.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.939+0000 7f95fe244700 1 --2- 172.21.15.107:0/2202310384 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f95e40a2c20 0x7f95e40a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009fd0 tx=0x7f95f4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:19.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:19.942+0000 7f95ef7fe700 1 -- 172.21.15.107:0/2202310384 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f95f0020020 con 0x7f95f8107570 2024-04-25T00:13:20.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:20.138+0000 7f96004a8700 1 -- 172.21.15.107:0/2202310384 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f95f81027c0 con 0x7f95f8107570 2024-04-25T00:13:20.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:20.139+0000 7f95ef7fe700 1 -- 172.21.15.107:0/2202310384 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f95f009bee0 con 0x7f95f8107570 2024-04-25T00:13:20.139 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-04-25T00:13:20.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:20.141+0000 7f96004a8700 1 -- 172.21.15.107:0/2202310384 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f95e40a2c20 msgr2=0x7f95e40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:20.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:20.141+0000 7f96004a8700 1 --2- 172.21.15.107:0/2202310384 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f95e40a2c20 0x7f95e40a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009fd0 tx=0x7f95f4005fb0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:20.141+0000 7f96004a8700 1 -- 172.21.15.107:0/2202310384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f8107570 msgr2=0x7f95f8101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:20.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:20.141+0000 7f96004a8700 1 --2- 172.21.15.107:0/2202310384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f8107570 0x7f95f8101d10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f95f0007d70 tx=0x7f95f000a340 comp rx=0 tx=0).stop 2024-04-25T00:13:20.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:20.141+0000 7f96004a8700 1 -- 172.21.15.107:0/2202310384 shutdown_connections 2024-04-25T00:13:20.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:20.141+0000 7f96004a8700 1 --2- 172.21.15.107:0/2202310384 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f95e40a2c20 0x7f95e40a50e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:20.141+0000 7f96004a8700 1 --2- 172.21.15.107:0/2202310384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95f8107570 0x7f95f8101d10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:20.141+0000 7f96004a8700 1 --2- 172.21.15.107:0/2202310384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95f806ac00 0x7f95f8101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:20.141+0000 7f96004a8700 1 --2- 172.21.15.107:0/2202310384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95f806b5b0 0x7f95f81017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:20.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:20.141+0000 7f96004a8700 1 -- 172.21.15.107:0/2202310384 >> 172.21.15.107:0/2202310384 conn(0x7f95f8077180 msgr2=0x7f95f80ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:20.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:20.141+0000 7f96004a8700 1 -- 172.21.15.107:0/2202310384 shutdown_connections 2024-04-25T00:13:20.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:20.141+0000 7f96004a8700 1 -- 172.21.15.107:0/2202310384 wait complete. 2024-04-25T00:13:20.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:19 smithi107 ceph-mon[29153]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:20.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:19 smithi195 ceph-mon[32833]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:20.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:19 smithi170 ceph-mon[32807]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:20.849 INFO:tasks.cephadm:Adding mds.a on smithi107 2024-04-25T00:13:20.849 INFO:tasks.cephadm:Adding mds.d on smithi107 2024-04-25T00:13:20.849 INFO:tasks.cephadm:Adding mds.g on smithi107 2024-04-25T00:13:20.849 INFO:tasks.cephadm:Adding mds.j on smithi107 2024-04-25T00:13:20.850 INFO:tasks.cephadm:Adding mds.b on smithi170 2024-04-25T00:13:20.850 INFO:tasks.cephadm:Adding mds.e on smithi170 2024-04-25T00:13:20.850 INFO:tasks.cephadm:Adding mds.h on smithi170 2024-04-25T00:13:20.850 INFO:tasks.cephadm:Adding mds.k on smithi170 2024-04-25T00:13:20.850 INFO:tasks.cephadm:Adding mds.c on smithi195 2024-04-25T00:13:20.850 INFO:tasks.cephadm:Adding mds.f on smithi195 2024-04-25T00:13:20.850 INFO:tasks.cephadm:Adding mds.i on smithi195 2024-04-25T00:13:20.850 INFO:tasks.cephadm:Adding mds.l on smithi195 2024-04-25T00:13:20.850 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi170=b;smithi170=e;smithi170=h;smithi170=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l' 2024-04-25T00:13:21.102 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:13:21.224 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:20 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2202310384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:13:21.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:20 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2202310384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:13:21.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:20 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2202310384' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:13:22.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:21 smithi107 ceph-mon[29153]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:22.234 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:21 smithi195 ceph-mon[32833]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:22.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:21 smithi170 ceph-mon[32807]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:22.887 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.886+0000 7f5e114d0700 1 -- 172.21.15.195:0/756625407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e0c102390 msgr2=0x7f5e0c1027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:22.887 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.886+0000 7f5e114d0700 1 --2- 172.21.15.195:0/756625407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e0c102390 0x7f5e0c1027f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009990 tx=0x7f5e00009ca0 comp rx=0 tx=0).stop 2024-04-25T00:13:22.887 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.886+0000 7f5e114d0700 1 -- 172.21.15.195:0/756625407 shutdown_connections 2024-04-25T00:13:22.887 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.886+0000 7f5e114d0700 1 --2- 172.21.15.195:0/756625407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e0c102d30 0x7f5e0c1095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:22.887 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.886+0000 7f5e114d0700 1 --2- 172.21.15.195:0/756625407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e0c101190 0x7f5e0c1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:22.887 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.886+0000 7f5e114d0700 1 --2- 172.21.15.195:0/756625407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e0c102390 0x7f5e0c1027f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:22.887 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.886+0000 7f5e114d0700 1 -- 172.21.15.195:0/756625407 >> 172.21.15.195:0/756625407 conn(0x7f5e0c0fc730 msgr2=0x7f5e0c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:22.888 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.886+0000 7f5e114d0700 1 -- 172.21.15.195:0/756625407 shutdown_connections 2024-04-25T00:13:22.888 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.886+0000 7f5e114d0700 1 -- 172.21.15.195:0/756625407 wait complete. 2024-04-25T00:13:22.889 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.887+0000 7f5e114d0700 1 Processor -- start 2024-04-25T00:13:22.889 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.887+0000 7f5e114d0700 1 -- start start 2024-04-25T00:13:22.889 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.887+0000 7f5e114d0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e0c101190 0x7f5e0c198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:22.890 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.887+0000 7f5e114d0700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e0c102390 0x7f5e0c198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:22.890 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.887+0000 7f5e114d0700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e0c102d30 0x7f5e0c19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:22.890 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.887+0000 7f5e114d0700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c10ce70 con 0x7f5e0c102d30 2024-04-25T00:13:22.890 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.887+0000 7f5e114d0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c10ccf0 con 0x7f5e0c101190 2024-04-25T00:13:22.890 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.887+0000 7f5e114d0700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c10cff0 con 0x7f5e0c102390 2024-04-25T00:13:22.890 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.887+0000 7f5e0a7fc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e0c102390 0x7f5e0c198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:22.890 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.888+0000 7f5e0a7fc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e0c102390 0x7f5e0c198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:44334/0 (socket says 172.21.15.195:44334) 2024-04-25T00:13:22.890 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.888+0000 7f5e0a7fc700 1 -- 172.21.15.195:0/2465195040 learned_addr learned my addr 172.21.15.195:0/2465195040 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-04-25T00:13:22.890 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.888+0000 7f5e0a7fc700 1 -- 172.21.15.195:0/2465195040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e0c101190 msgr2=0x7f5e0c198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:22.890 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.888+0000 7f5e0affd700 1 --2- 172.21.15.195:0/2465195040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e0c101190 0x7f5e0c198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:22.890 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.888+0000 7f5e0a7fc700 1 --2- 172.21.15.195:0/2465195040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e0c101190 0x7f5e0c198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:22.890 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.888+0000 7f5e0b7fe700 1 --2- 172.21.15.195:0/2465195040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e0c102d30 0x7f5e0c19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:22.890 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.888+0000 7f5e0a7fc700 1 -- 172.21.15.195:0/2465195040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e0c102d30 msgr2=0x7f5e0c19fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:22.890 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.888+0000 7f5e0a7fc700 1 --2- 172.21.15.195:0/2465195040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e0c102d30 0x7f5e0c19fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:22.891 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.888+0000 7f5e0a7fc700 1 -- 172.21.15.195:0/2465195040 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c1a0290 con 0x7f5e0c102390 2024-04-25T00:13:22.891 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.888+0000 7f5e0affd700 1 --2- 172.21.15.195:0/2465195040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e0c101190 0x7f5e0c198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:22.891 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.888+0000 7f5e0b7fe700 1 --2- 172.21.15.195:0/2465195040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e0c102d30 0x7f5e0c19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:22.891 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.888+0000 7f5e0a7fc700 1 --2- 172.21.15.195:0/2465195040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e0c102390 0x7f5e0c198bc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009fc0 tx=0x7f5e00004580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:22.891 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.888+0000 7f5df3fff700 1 -- 172.21.15.195:0/2465195040 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e00027070 con 0x7f5e0c102390 2024-04-25T00:13:22.891 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.888+0000 7f5e114d0700 1 -- 172.21.15.195:0/2465195040 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0c1a0520 con 0x7f5e0c102390 2024-04-25T00:13:22.891 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.888+0000 7f5df3fff700 1 -- 172.21.15.195:0/2465195040 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e00022470 con 0x7f5e0c102390 2024-04-25T00:13:22.891 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.888+0000 7f5df3fff700 1 -- 172.21.15.195:0/2465195040 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0000f6c0 con 0x7f5e0c102390 2024-04-25T00:13:22.891 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.888+0000 7f5e114d0700 1 -- 172.21.15.195:0/2465195040 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e0c1a09f0 con 0x7f5e0c102390 2024-04-25T00:13:22.891 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.889+0000 7f5e114d0700 1 -- 172.21.15.195:0/2465195040 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e0c068f20 con 0x7f5e0c102390 2024-04-25T00:13:22.892 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.890+0000 7f5df3fff700 1 -- 172.21.15.195:0/2465195040 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5e0002a070 con 0x7f5e0c102390 2024-04-25T00:13:22.892 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.891+0000 7f5df3fff700 1 --2- 172.21.15.195:0/2465195040 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5df40a2c20 0x7f5df40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:22.892 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.891+0000 7f5df3fff700 1 -- 172.21.15.195:0/2465195040 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5e000d0140 con 0x7f5e0c102390 2024-04-25T00:13:22.892 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.891+0000 7f5e0affd700 1 --2- 172.21.15.195:0/2465195040 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5df40a2c20 0x7f5df40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:22.892 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.892+0000 7f5e0affd700 1 --2- 172.21.15.195:0/2465195040 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5df40a2c20 0x7f5df40a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c1021f0 tx=0x7f5dfc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:22.895 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:22.895+0000 7f5df3fff700 1 -- 172.21.15.195:0/2465195040 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5e00031070 con 0x7f5e0c102390 2024-04-25T00:13:23.100 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:23.099+0000 7f5e114d0700 1 -- 172.21.15.195:0/2465195040 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi170=b;smithi170=e;smithi170=h;smithi170=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l", "target": ["mon-mgr", ""]}) -- 0x7f5e0c0fea80 con 0x7f5df40a2c20 2024-04-25T00:13:23.107 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:23.107+0000 7f5df3fff700 1 -- 172.21.15.195:0/2465195040 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f5e0c0fea80 con 0x7f5df40a2c20 2024-04-25T00:13:23.107 INFO:teuthology.orchestra.run.smithi195.stdout:Scheduled mds.all update... 2024-04-25T00:13:23.110 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:23.109+0000 7f5e114d0700 1 -- 172.21.15.195:0/2465195040 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5df40a2c20 msgr2=0x7f5df40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:23.110 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:23.109+0000 7f5e114d0700 1 --2- 172.21.15.195:0/2465195040 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5df40a2c20 0x7f5df40a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c1021f0 tx=0x7f5dfc008040 comp rx=0 tx=0).stop 2024-04-25T00:13:23.110 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:23.109+0000 7f5e114d0700 1 -- 172.21.15.195:0/2465195040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e0c102390 msgr2=0x7f5e0c198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:23.110 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:23.109+0000 7f5e114d0700 1 --2- 172.21.15.195:0/2465195040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e0c102390 0x7f5e0c198bc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009fc0 tx=0x7f5e00004580 comp rx=0 tx=0).stop 2024-04-25T00:13:23.110 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:23.109+0000 7f5e114d0700 1 -- 172.21.15.195:0/2465195040 shutdown_connections 2024-04-25T00:13:23.110 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:23.109+0000 7f5e114d0700 1 --2- 172.21.15.195:0/2465195040 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5df40a2c20 0x7f5df40a50e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.110 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:23.109+0000 7f5e114d0700 1 --2- 172.21.15.195:0/2465195040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e0c102d30 0x7f5e0c19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.110 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:23.109+0000 7f5e114d0700 1 --2- 172.21.15.195:0/2465195040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5e0c101190 0x7f5e0c198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.110 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:23.109+0000 7f5e114d0700 1 --2- 172.21.15.195:0/2465195040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e0c102390 0x7f5e0c198bc0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:23.110 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:23.109+0000 7f5e114d0700 1 -- 172.21.15.195:0/2465195040 >> 172.21.15.195:0/2465195040 conn(0x7f5e0c0fc730 msgr2=0x7f5e0c0fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:23.110 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:23.109+0000 7f5e114d0700 1 -- 172.21.15.195:0/2465195040 shutdown_connections 2024-04-25T00:13:23.110 INFO:teuthology.orchestra.run.smithi195.stderr:2024-04-25T00:13:23.109+0000 7f5e114d0700 1 -- 172.21.15.195:0/2465195040 wait complete. 2024-04-25T00:13:23.778 DEBUG:teuthology.orchestra.run.smithi107:mds.a> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T00:13:23.780 DEBUG:teuthology.orchestra.run.smithi107:mds.d> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T00:13:23.823 DEBUG:teuthology.orchestra.run.smithi107:mds.g> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T00:13:23.866 DEBUG:teuthology.orchestra.run.smithi107:mds.j> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T00:13:23.909 DEBUG:teuthology.orchestra.run.smithi170:mds.b> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T00:13:23.912 DEBUG:teuthology.orchestra.run.smithi170:mds.e> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T00:13:23.955 DEBUG:teuthology.orchestra.run.smithi170:mds.h> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T00:13:23.999 DEBUG:teuthology.orchestra.run.smithi170:mds.k> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T00:13:24.042 DEBUG:teuthology.orchestra.run.smithi195:mds.c> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T00:13:24.044 DEBUG:teuthology.orchestra.run.smithi195:mds.f> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T00:13:24.063 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:23 smithi195 ceph-mon[32833]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:24.064 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:23 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:24.064 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:23 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:24.064 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:23 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:24.064 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:23 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:24.064 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:23 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:24.064 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:23 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:24.064 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:23 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:24.064 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:23 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:24.065 DEBUG:teuthology.orchestra.run.smithi195:mds.i> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T00:13:24.107 DEBUG:teuthology.orchestra.run.smithi195:mds.l> sudo journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T00:13:24.149 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T00:13:24.149 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:13:24.171 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:23 smithi107 ceph-mon[29153]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:24.171 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:23 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:24.171 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:23 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:24.171 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:23 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:24.172 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:23 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:24.172 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:23 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:24.172 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:23 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:24.172 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:23 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:24.172 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:23 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:24.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:23 smithi170 ceph-mon[32807]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:24.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:23 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:24.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:23 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:24.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:23 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:24.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:23 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:24.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:23 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:24.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:23 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:24.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:23 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:24.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:23 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:24.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.275+0000 7fe8f7e61640 1 -- 172.21.15.107:0/2460402148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8f0104b40 msgr2=0x7fe8f0106f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.275+0000 7fe8f7e61640 1 --2- 172.21.15.107:0/2460402148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8f0104b40 0x7fe8f0106f30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009900 tx=0x7fe8e002a410 comp rx=0 tx=0).stop 2024-04-25T00:13:24.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.275+0000 7fe8f7e61640 1 -- 172.21.15.107:0/2460402148 shutdown_connections 2024-04-25T00:13:24.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.275+0000 7fe8f7e61640 1 --2- 172.21.15.107:0/2460402148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8f0107470 0x7fe8f01098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.275+0000 7fe8f7e61640 1 --2- 172.21.15.107:0/2460402148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8f0104b40 0x7fe8f0106f30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.275+0000 7fe8f7e61640 1 --2- 172.21.15.107:0/2460402148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f0102210 0x7fe8f0104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.275+0000 7fe8f7e61640 1 -- 172.21.15.107:0/2460402148 >> 172.21.15.107:0/2460402148 conn(0x7fe8f00fbc20 msgr2=0x7fe8f00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:24.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.275+0000 7fe8f7e61640 1 -- 172.21.15.107:0/2460402148 shutdown_connections 2024-04-25T00:13:24.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.275+0000 7fe8f7e61640 1 -- 172.21.15.107:0/2460402148 wait complete. 2024-04-25T00:13:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.276+0000 7fe8f7e61640 1 Processor -- start 2024-04-25T00:13:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.276+0000 7fe8f7e61640 1 -- start start 2024-04-25T00:13:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.276+0000 7fe8f7e61640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8f0102210 0x7fe8f0197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.276+0000 7fe8f7e61640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8f0104b40 0x7fe8f0198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.276+0000 7fe8f7e61640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f0107470 0x7fe8f019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.276+0000 7fe8f7e61640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f010d400 con 0x7fe8f0104b40 2024-04-25T00:13:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.276+0000 7fe8f7e61640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f010d280 con 0x7fe8f0107470 2024-04-25T00:13:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.276+0000 7fe8f7e61640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f010d580 con 0x7fe8f0102210 2024-04-25T00:13:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.276+0000 7fe8f53fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8f0104b40 0x7fe8f0198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.276+0000 7fe8f53fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8f0104b40 0x7fe8f0198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46526/0 (socket says 172.21.15.107:46526) 2024-04-25T00:13:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.276+0000 7fe8f53fc640 1 -- 172.21.15.107:0/2652700428 learned_addr learned my addr 172.21.15.107:0/2652700428 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.276+0000 7fe8f5bfd640 1 --2- 172.21.15.107:0/2652700428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8f0102210 0x7fe8f0197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.276+0000 7fe8f63fe640 1 --2- 172.21.15.107:0/2652700428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f0107470 0x7fe8f019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.276+0000 7fe8f53fc640 1 -- 172.21.15.107:0/2652700428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8f0102210 msgr2=0x7fe8f0197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.276+0000 7fe8f53fc640 1 --2- 172.21.15.107:0/2652700428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8f0102210 0x7fe8f0197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.276+0000 7fe8f53fc640 1 -- 172.21.15.107:0/2652700428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f0107470 msgr2=0x7fe8f019f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.277+0000 7fe8f53fc640 1 --2- 172.21.15.107:0/2652700428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f0107470 0x7fe8f019f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.277+0000 7fe8f53fc640 1 -- 172.21.15.107:0/2652700428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f019f6f0 con 0x7fe8f0104b40 2024-04-25T00:13:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.277+0000 7fe8f53fc640 1 --2- 172.21.15.107:0/2652700428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8f0104b40 0x7fe8f0198010 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0031610 tx=0x7fe8e0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.277+0000 7fe8e6ffd640 1 -- 172.21.15.107:0/2652700428 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e003f030 con 0x7fe8f0104b40 2024-04-25T00:13:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.277+0000 7fe8e6ffd640 1 -- 172.21.15.107:0/2652700428 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e0003ad0 con 0x7fe8f0104b40 2024-04-25T00:13:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.277+0000 7fe8e6ffd640 1 -- 172.21.15.107:0/2652700428 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0035070 con 0x7fe8f0104b40 2024-04-25T00:13:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.277+0000 7fe8f7e61640 1 -- 172.21.15.107:0/2652700428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f019f980 con 0x7fe8f0104b40 2024-04-25T00:13:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.277+0000 7fe8f7e61640 1 -- 172.21.15.107:0/2652700428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f019fe70 con 0x7fe8f0104b40 2024-04-25T00:13:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.278+0000 7fe8f7e61640 1 -- 172.21.15.107:0/2652700428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8f00fd710 con 0x7fe8f0104b40 2024-04-25T00:13:24.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.279+0000 7fe8e6ffd640 1 -- 172.21.15.107:0/2652700428 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe8e0042030 con 0x7fe8f0104b40 2024-04-25T00:13:24.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.279+0000 7fe8e6ffd640 1 --2- 172.21.15.107:0/2652700428 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe8dc0a2b90 0x7fe8dc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:24.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.279+0000 7fe8f5bfd640 1 --2- 172.21.15.107:0/2652700428 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe8dc0a2b90 0x7fe8dc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:24.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.280+0000 7fe8e6ffd640 1 -- 172.21.15.107:0/2652700428 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe8e00ef150 con 0x7fe8f0104b40 2024-04-25T00:13:24.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.280+0000 7fe8f5bfd640 1 --2- 172.21.15.107:0/2652700428 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe8dc0a2b90 0x7fe8dc0a5050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009bd0 tx=0x7fe8ec005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:24.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.284+0000 7fe8e6ffd640 1 -- 172.21.15.107:0/2652700428 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe8e00bbf00 con 0x7fe8f0104b40 2024-04-25T00:13:24.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.463+0000 7fe8f7e61640 1 -- 172.21.15.107:0/2652700428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe8f002d3c0 con 0x7fe8f0104b40 2024-04-25T00:13:24.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.464+0000 7fe8e6ffd640 1 -- 172.21.15.107:0/2652700428 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18462 (secure 0 0 0) 0x7fe8e00c0200 con 0x7fe8f0104b40 2024-04-25T00:13:24.466 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:13:24.466 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":82,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:13:12.278840+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:13:24.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.466+0000 7fe8f7e61640 1 -- 172.21.15.107:0/2652700428 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe8dc0a2b90 msgr2=0x7fe8dc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.466+0000 7fe8f7e61640 1 --2- 172.21.15.107:0/2652700428 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe8dc0a2b90 0x7fe8dc0a5050 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009bd0 tx=0x7fe8ec005f90 comp rx=0 tx=0).stop 2024-04-25T00:13:24.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.466+0000 7fe8f7e61640 1 -- 172.21.15.107:0/2652700428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8f0104b40 msgr2=0x7fe8f0198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.466+0000 7fe8f7e61640 1 --2- 172.21.15.107:0/2652700428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8f0104b40 0x7fe8f0198010 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0031610 tx=0x7fe8e0004270 comp rx=0 tx=0).stop 2024-04-25T00:13:24.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.466+0000 7fe8f7e61640 1 -- 172.21.15.107:0/2652700428 shutdown_connections 2024-04-25T00:13:24.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.466+0000 7fe8f7e61640 1 --2- 172.21.15.107:0/2652700428 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe8dc0a2b90 0x7fe8dc0a5050 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.466+0000 7fe8f7e61640 1 --2- 172.21.15.107:0/2652700428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8f0107470 0x7fe8f019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.466+0000 7fe8f7e61640 1 --2- 172.21.15.107:0/2652700428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8f0104b40 0x7fe8f0198010 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.466+0000 7fe8f7e61640 1 --2- 172.21.15.107:0/2652700428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8f0102210 0x7fe8f0197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.466+0000 7fe8f7e61640 1 -- 172.21.15.107:0/2652700428 >> 172.21.15.107:0/2652700428 conn(0x7fe8f00fbc20 msgr2=0x7fe8f0102cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:24.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.466+0000 7fe8f7e61640 1 -- 172.21.15.107:0/2652700428 shutdown_connections 2024-04-25T00:13:24.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.467+0000 7fe8f7e61640 1 -- 172.21.15.107:0/2652700428 wait complete. 2024-04-25T00:13:24.480 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:09:25.160329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:13:24.480 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:13:24.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.620+0000 7f693dd9e640 1 -- 172.21.15.107:0/3332150739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6938100840 msgr2=0x7f6938100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.620+0000 7f693dd9e640 1 --2- 172.21.15.107:0/3332150739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6938100840 0x7f6938100cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6924009900 tx=0x7f692402a410 comp rx=0 tx=0).stop 2024-04-25T00:13:24.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.620+0000 7f693dd9e640 1 -- 172.21.15.107:0/3332150739 shutdown_connections 2024-04-25T00:13:24.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.620+0000 7f693dd9e640 1 --2- 172.21.15.107:0/3332150739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6938101200 0x7f6938108aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.620+0000 7f693dd9e640 1 --2- 172.21.15.107:0/3332150739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6938100840 0x7f6938100cc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.620+0000 7f693dd9e640 1 --2- 172.21.15.107:0/3332150739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6938101b10 0x7f6938101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.620+0000 7f693dd9e640 1 -- 172.21.15.107:0/3332150739 >> 172.21.15.107:0/3332150739 conn(0x7f69380fb290 msgr2=0x7f69380fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:24.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.620+0000 7f693dd9e640 1 -- 172.21.15.107:0/3332150739 shutdown_connections 2024-04-25T00:13:24.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.620+0000 7f693dd9e640 1 -- 172.21.15.107:0/3332150739 wait complete. 2024-04-25T00:13:24.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.620+0000 7f693dd9e640 1 Processor -- start 2024-04-25T00:13:24.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.621+0000 7f693dd9e640 1 -- start start 2024-04-25T00:13:24.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.621+0000 7f693dd9e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6938100840 0x7f69381002c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:24.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.621+0000 7f693dd9e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6938101200 0x7f693819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:24.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.621+0000 7f693dd9e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6938101b10 0x7f693819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:24.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.621+0000 7f693dd9e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f693810c220 con 0x7f6938100840 2024-04-25T00:13:24.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.621+0000 7f693dd9e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f693810c0a0 con 0x7f6938101200 2024-04-25T00:13:24.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.621+0000 7f693dd9e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f693810c3a0 con 0x7f6938101b10 2024-04-25T00:13:24.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.621+0000 7f693cd9c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6938100840 0x7f69381002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:24.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.621+0000 7f693cd9c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6938100840 0x7f69381002c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46552/0 (socket says 172.21.15.107:46552) 2024-04-25T00:13:24.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.621+0000 7f693cd9c640 1 -- 172.21.15.107:0/3698163950 learned_addr learned my addr 172.21.15.107:0/3698163950 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:24.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.621+0000 7f693d59d640 1 --2- 172.21.15.107:0/3698163950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6938101b10 0x7f693819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:24.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.621+0000 7f6937fff640 1 --2- 172.21.15.107:0/3698163950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6938101200 0x7f693819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:24.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.621+0000 7f693cd9c640 1 -- 172.21.15.107:0/3698163950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6938101b10 msgr2=0x7f693819ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.621+0000 7f693cd9c640 1 --2- 172.21.15.107:0/3698163950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6938101b10 0x7f693819ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.621+0000 7f693cd9c640 1 -- 172.21.15.107:0/3698163950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6938101200 msgr2=0x7f693819c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.621+0000 7f693cd9c640 1 --2- 172.21.15.107:0/3698163950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6938101200 0x7f693819c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.621+0000 7f693cd9c640 1 -- 172.21.15.107:0/3698163950 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693819f5b0 con 0x7f6938100840 2024-04-25T00:13:24.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.621+0000 7f6937fff640 1 --2- 172.21.15.107:0/3698163950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6938101200 0x7f693819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:24.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.621+0000 7f693cd9c640 1 --2- 172.21.15.107:0/3698163950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6938100840 0x7f69381002c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f692c00d770 tx=0x7f692c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:24.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.621+0000 7f6935ffb640 1 -- 172.21.15.107:0/3698163950 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c017070 con 0x7f6938100840 2024-04-25T00:13:24.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.621+0000 7f693dd9e640 1 -- 172.21.15.107:0/3698163950 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f693819f8a0 con 0x7f6938100840 2024-04-25T00:13:24.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.621+0000 7f6935ffb640 1 -- 172.21.15.107:0/3698163950 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f692c009c30 con 0x7f6938100840 2024-04-25T00:13:24.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.621+0000 7f6935ffb640 1 -- 172.21.15.107:0/3698163950 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c003dd0 con 0x7f6938100840 2024-04-25T00:13:24.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.622+0000 7f693dd9e640 1 -- 172.21.15.107:0/3698163950 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f693819fdf0 con 0x7f6938100840 2024-04-25T00:13:24.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.623+0000 7f693dd9e640 1 -- 172.21.15.107:0/3698163950 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6938107980 con 0x7f6938100840 2024-04-25T00:13:24.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.624+0000 7f6935ffb640 1 -- 172.21.15.107:0/3698163950 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f692c003fb0 con 0x7f6938100840 2024-04-25T00:13:24.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.624+0000 7f6935ffb640 1 --2- 172.21.15.107:0/3698163950 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f69280a2b90 0x7f69280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:24.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.624+0000 7f6937fff640 1 --2- 172.21.15.107:0/3698163950 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f69280a2b90 0x7f69280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:24.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.624+0000 7f6935ffb640 1 -- 172.21.15.107:0/3698163950 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f692c0ce530 con 0x7f6938100840 2024-04-25T00:13:24.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.625+0000 7f6937fff640 1 --2- 172.21.15.107:0/3698163950 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f69280a2b90 0x7f69280a5050 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6924000c00 tx=0x7f6924031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:24.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.628+0000 7f6935ffb640 1 -- 172.21.15.107:0/3698163950 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f692c09b930 con 0x7f6938100840 2024-04-25T00:13:24.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.805+0000 7f693dd9e640 1 -- 172.21.15.107:0/3698163950 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f693802d360 con 0x7f6938100840 2024-04-25T00:13:24.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.805+0000 7f6935ffb640 1 -- 172.21.15.107:0/3698163950 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f692c009de0 con 0x7f6938100840 2024-04-25T00:13:24.807 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-04-25T00:13:24.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.807+0000 7f693dd9e640 1 -- 172.21.15.107:0/3698163950 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f69280a2b90 msgr2=0x7f69280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.807+0000 7f693dd9e640 1 --2- 172.21.15.107:0/3698163950 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f69280a2b90 0x7f69280a5050 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6924000c00 tx=0x7f6924031400 comp rx=0 tx=0).stop 2024-04-25T00:13:24.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.807+0000 7f693dd9e640 1 -- 172.21.15.107:0/3698163950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6938100840 msgr2=0x7f69381002c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.807+0000 7f693dd9e640 1 --2- 172.21.15.107:0/3698163950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6938100840 0x7f69381002c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f692c00d770 tx=0x7f692c00dc30 comp rx=0 tx=0).stop 2024-04-25T00:13:24.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.807+0000 7f693dd9e640 1 -- 172.21.15.107:0/3698163950 shutdown_connections 2024-04-25T00:13:24.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.807+0000 7f693dd9e640 1 --2- 172.21.15.107:0/3698163950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6938101b10 0x7f693819ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.807+0000 7f693dd9e640 1 --2- 172.21.15.107:0/3698163950 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f69280a2b90 0x7f69280a5050 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.807+0000 7f693dd9e640 1 --2- 172.21.15.107:0/3698163950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6938101200 0x7f693819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.807+0000 7f693dd9e640 1 --2- 172.21.15.107:0/3698163950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6938100840 0x7f69381002c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.807+0000 7f693dd9e640 1 -- 172.21.15.107:0/3698163950 >> 172.21.15.107:0/3698163950 conn(0x7f69380fb290 msgr2=0x7f6938104b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:24.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.807+0000 7f693dd9e640 1 -- 172.21.15.107:0/3698163950 shutdown_connections 2024-04-25T00:13:24.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.807+0000 7f693dd9e640 1 -- 172.21.15.107:0/3698163950 wait complete. 2024-04-25T00:13:24.822 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:24.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.918+0000 7f09265dc640 1 -- 172.21.15.107:0/545100347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0920102250 msgr2=0x7f0920108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.918+0000 7f09265dc640 1 --2- 172.21.15.107:0/545100347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0920102250 0x7f0920108ae0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0918009ca0 tx=0x7f091802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.918+0000 7f09265dc640 1 -- 172.21.15.107:0/545100347 shutdown_connections 2024-04-25T00:13:24.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.918+0000 7f09265dc640 1 --2- 172.21.15.107:0/545100347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0920102250 0x7f0920108ae0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.918+0000 7f09265dc640 1 --2- 172.21.15.107:0/545100347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0920101890 0x7f0920101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.918+0000 7f09265dc640 1 --2- 172.21.15.107:0/545100347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09201006a0 0x7f0920100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.918+0000 7f09265dc640 1 -- 172.21.15.107:0/545100347 >> 172.21.15.107:0/545100347 conn(0x7f09200fbc30 msgr2=0x7f09200fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:24.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.918+0000 7f09265dc640 1 -- 172.21.15.107:0/545100347 shutdown_connections 2024-04-25T00:13:24.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.918+0000 7f09265dc640 1 -- 172.21.15.107:0/545100347 wait complete. 2024-04-25T00:13:24.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.919+0000 7f09265dc640 1 Processor -- start 2024-04-25T00:13:24.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.919+0000 7f09265dc640 1 -- start start 2024-04-25T00:13:24.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.919+0000 7f09265dc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09201006a0 0x7f0920196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:24.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.919+0000 7f09265dc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0920101890 0x7f0920196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:24.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.919+0000 7f09265dc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0920102250 0x7f092019df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:24.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.919+0000 7f09265dc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f092010c370 con 0x7f09201006a0 2024-04-25T00:13:24.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.919+0000 7f09265dc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f092010c1f0 con 0x7f0920101890 2024-04-25T00:13:24.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.919+0000 7f09265dc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f092010c4f0 con 0x7f0920102250 2024-04-25T00:13:24.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.919+0000 7f091ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09201006a0 0x7f0920196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:24.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.919+0000 7f091ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09201006a0 0x7f0920196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46560/0 (socket says 172.21.15.107:46560) 2024-04-25T00:13:24.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.919+0000 7f091ffff640 1 -- 172.21.15.107:0/3270857073 learned_addr learned my addr 172.21.15.107:0/3270857073 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:24.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.920+0000 7f0924b79640 1 --2- 172.21.15.107:0/3270857073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0920102250 0x7f092019df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:24.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.920+0000 7f091f7fe640 1 --2- 172.21.15.107:0/3270857073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0920101890 0x7f0920196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:24.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.920+0000 7f091ffff640 1 -- 172.21.15.107:0/3270857073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0920102250 msgr2=0x7f092019df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.920+0000 7f091ffff640 1 --2- 172.21.15.107:0/3270857073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0920102250 0x7f092019df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.920+0000 7f091ffff640 1 -- 172.21.15.107:0/3270857073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0920101890 msgr2=0x7f0920196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:24.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.920+0000 7f091ffff640 1 --2- 172.21.15.107:0/3270857073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0920101890 0x7f0920196e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:24.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.920+0000 7f091ffff640 1 -- 172.21.15.107:0/3270857073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f092019e570 con 0x7f09201006a0 2024-04-25T00:13:24.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.920+0000 7f091ffff640 1 --2- 172.21.15.107:0/3270857073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09201006a0 0x7f0920196950 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f091000d950 tx=0x7f091000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:24.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.920+0000 7f091d7fa640 1 -- 172.21.15.107:0/3270857073 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0910017070 con 0x7f09201006a0 2024-04-25T00:13:24.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.920+0000 7f091d7fa640 1 -- 172.21.15.107:0/3270857073 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0910009c30 con 0x7f09201006a0 2024-04-25T00:13:24.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.920+0000 7f09265dc640 1 -- 172.21.15.107:0/3270857073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f092019e860 con 0x7f09201006a0 2024-04-25T00:13:24.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.920+0000 7f091d7fa640 1 -- 172.21.15.107:0/3270857073 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091000f600 con 0x7f09201006a0 2024-04-25T00:13:24.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.920+0000 7f09265dc640 1 -- 172.21.15.107:0/3270857073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f092019ed60 con 0x7f09201006a0 2024-04-25T00:13:24.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.921+0000 7f09265dc640 1 -- 172.21.15.107:0/3270857073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0920107960 con 0x7f09201006a0 2024-04-25T00:13:24.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.922+0000 7f091d7fa640 1 -- 172.21.15.107:0/3270857073 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f091001a070 con 0x7f09201006a0 2024-04-25T00:13:24.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.923+0000 7f091d7fa640 1 --2- 172.21.15.107:0/3270857073 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f09080a2b90 0x7f09080a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:24.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.923+0000 7f091f7fe640 1 --2- 172.21.15.107:0/3270857073 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f09080a2b90 0x7f09080a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:24.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.923+0000 7f091d7fa640 1 -- 172.21.15.107:0/3270857073 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f09100ce560 con 0x7f09201006a0 2024-04-25T00:13:24.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.923+0000 7f091f7fe640 1 --2- 172.21.15.107:0/3270857073 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f09080a2b90 0x7f09080a5050 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0920197e60 tx=0x7f091400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:24.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:24.926+0000 7f091d7fa640 1 -- 172.21.15.107:0/3270857073 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0910009de0 con 0x7f09201006a0 2024-04-25T00:13:25.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:25.139+0000 7f09265dc640 1 -- 172.21.15.107:0/3270857073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f092002d170 con 0x7f09201006a0 2024-04-25T00:13:25.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:25.139+0000 7f091d7fa640 1 -- 172.21.15.107:0/3270857073 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f091009b310 con 0x7f09201006a0 2024-04-25T00:13:25.141 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:13:25.141 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T00:13:25.141 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 1 2024-04-25T00:13:25.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:25.141+0000 7f09265dc640 1 -- 172.21.15.107:0/3270857073 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f09080a2b90 msgr2=0x7f09080a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:25.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:25.141+0000 7f09265dc640 1 --2- 172.21.15.107:0/3270857073 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f09080a2b90 0x7f09080a5050 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0920197e60 tx=0x7f091400b390 comp rx=0 tx=0).stop 2024-04-25T00:13:25.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:25.141+0000 7f09265dc640 1 -- 172.21.15.107:0/3270857073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09201006a0 msgr2=0x7f0920196950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:25.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:25.141+0000 7f09265dc640 1 --2- 172.21.15.107:0/3270857073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09201006a0 0x7f0920196950 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f091000d950 tx=0x7f091000de10 comp rx=0 tx=0).stop 2024-04-25T00:13:25.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:25.141+0000 7f09265dc640 1 -- 172.21.15.107:0/3270857073 shutdown_connections 2024-04-25T00:13:25.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:25.141+0000 7f09265dc640 1 --2- 172.21.15.107:0/3270857073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0920102250 0x7f092019df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:25.141+0000 7f09265dc640 1 --2- 172.21.15.107:0/3270857073 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f09080a2b90 0x7f09080a5050 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:25.141+0000 7f09265dc640 1 --2- 172.21.15.107:0/3270857073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0920101890 0x7f0920196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:25.141+0000 7f09265dc640 1 --2- 172.21.15.107:0/3270857073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09201006a0 0x7f0920196950 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:25.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:25.141+0000 7f09265dc640 1 -- 172.21.15.107:0/3270857073 >> 172.21.15.107:0/3270857073 conn(0x7f09200fbc30 msgr2=0x7f0920104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:25.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:25.141+0000 7f09265dc640 1 -- 172.21.15.107:0/3270857073 shutdown_connections 2024-04-25T00:13:25.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:25.142+0000 7f09265dc640 1 -- 172.21.15.107:0/3270857073 wait complete. 2024-04-25T00:13:25.153 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:24 smithi107 ceph-mon[29153]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi170=b;smithi170=e;smithi170=h;smithi170=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:13:25.153 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:24 smithi107 ceph-mon[29153]: Saving service mds.all spec with placement smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi170=b;smithi170=e;smithi170=h;smithi170=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l;count:12 2024-04-25T00:13:25.153 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:24 smithi107 ceph-mon[29153]: Deploying daemon mds.l on smithi195 2024-04-25T00:13:25.153 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:24 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2652700428' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:13:25.153 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:24 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3698163950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:13:25.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:24 smithi195 ceph-mon[32833]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi170=b;smithi170=e;smithi170=h;smithi170=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:13:25.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:24 smithi195 ceph-mon[32833]: Saving service mds.all spec with placement smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi170=b;smithi170=e;smithi170=h;smithi170=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l;count:12 2024-04-25T00:13:25.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:24 smithi195 ceph-mon[32833]: Deploying daemon mds.l on smithi195 2024-04-25T00:13:25.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:24 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2652700428' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:13:25.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:24 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3698163950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:13:25.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:24 smithi170 ceph-mon[32807]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi170=b;smithi170=e;smithi170=h;smithi170=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:13:25.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:24 smithi170 ceph-mon[32807]: Saving service mds.all spec with placement smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi170=b;smithi170=e;smithi170=h;smithi170=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l;count:12 2024-04-25T00:13:25.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:24 smithi170 ceph-mon[32807]: Deploying daemon mds.l on smithi195 2024-04-25T00:13:25.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:24 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2652700428' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:13:25.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:24 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3698163950' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:13:25.979 INFO:journalctl@ceph.mds.l.smithi195.stdout:Apr 25 00:13:25 smithi195 systemd[1]: Starting Ceph mds.l for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:13:26.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:25 smithi107 ceph-mon[29153]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:26.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:25 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3270857073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:26.302 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:25 smithi195 ceph-mon[32833]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:26.303 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:25 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3270857073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:26.303 INFO:journalctl@ceph.mds.l.smithi195.stdout:Apr 25 00:13:26 smithi195 podman[63156]: 2024-04-25 00:13:25.945282557 +0000 UTC m=+0.018021387 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:13:26.303 INFO:journalctl@ceph.mds.l.smithi195.stdout:Apr 25 00:13:26 smithi195 podman[63156]: 2024-04-25 00:13:26.058806662 +0000 UTC m=+0.131545478 container create 261749b799aff3759687abf7b03c21d8cd033e51f57b03adb5afa28b9bb9a6df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:13:26.303 INFO:journalctl@ceph.mds.l.smithi195.stdout:Apr 25 00:13:26 smithi195 podman[63156]: 2024-04-25 00:13:26.303415484 +0000 UTC m=+0.376154301 container init 261749b799aff3759687abf7b03c21d8cd033e51f57b03adb5afa28b9bb9a6df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-l, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T00:13:26.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:25 smithi170 ceph-mon[32807]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:26.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:25 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3270857073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:26.589 INFO:journalctl@ceph.mds.l.smithi195.stdout:Apr 25 00:13:26 smithi195 podman[63156]: 2024-04-25 00:13:26.3062068 +0000 UTC m=+0.378945630 container start 261749b799aff3759687abf7b03c21d8cd033e51f57b03adb5afa28b9bb9a6df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T00:13:26.589 INFO:journalctl@ceph.mds.l.smithi195.stdout:Apr 25 00:13:26 smithi195 ceph-mds[63175]: -- 172.21.15.195:0/2267934411 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c157715a40 con 0x55c1576e8c00 2024-04-25T00:13:26.589 INFO:journalctl@ceph.mds.l.smithi195.stdout:Apr 25 00:13:26 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-l[63171]: starting mds.l at 2024-04-25T00:13:26.589 INFO:journalctl@ceph.mds.l.smithi195.stdout:Apr 25 00:13:26 smithi195 bash[63156]: 261749b799aff3759687abf7b03c21d8cd033e51f57b03adb5afa28b9bb9a6df 2024-04-25T00:13:26.589 INFO:journalctl@ceph.mds.l.smithi195.stdout:Apr 25 00:13:26 smithi195 systemd[1]: Started Ceph mds.l for 79602f58-0297-11ef-bc93-c7b262605968. 2024-04-25T00:13:27.155 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:27.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.263+0000 7fcbc167d640 1 -- 172.21.15.107:0/1909360997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbbc0ffa80 msgr2=0x7fcbbc0ffea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:27.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.263+0000 7fcbc167d640 1 --2- 172.21.15.107:0/1909360997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbbc0ffa80 0x7fcbbc0ffea0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009930 tx=0x7fcbac02a420 comp rx=0 tx=0).stop 2024-04-25T00:13:27.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.263+0000 7fcbc167d640 1 -- 172.21.15.107:0/1909360997 shutdown_connections 2024-04-25T00:13:27.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.263+0000 7fcbc167d640 1 --2- 172.21.15.107:0/1909360997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbbc1034b0 0x7fcbbc109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:27.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.263+0000 7fcbc167d640 1 --2- 172.21.15.107:0/1909360997 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbbc102af0 0x7fcbbc102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:27.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.263+0000 7fcbc167d640 1 --2- 172.21.15.107:0/1909360997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbbc0ffa80 0x7fcbbc0ffea0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:27.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.263+0000 7fcbc167d640 1 -- 172.21.15.107:0/1909360997 >> 172.21.15.107:0/1909360997 conn(0x7fcbbc0fb3b0 msgr2=0x7fcbbc0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:27.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.263+0000 7fcbc167d640 1 -- 172.21.15.107:0/1909360997 shutdown_connections 2024-04-25T00:13:27.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.263+0000 7fcbc167d640 1 -- 172.21.15.107:0/1909360997 wait complete. 2024-04-25T00:13:27.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.264+0000 7fcbc167d640 1 Processor -- start 2024-04-25T00:13:27.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.264+0000 7fcbc167d640 1 -- start start 2024-04-25T00:13:27.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.264+0000 7fcbc167d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbbc0ffa80 0x7fcbbc197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:27.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.264+0000 7fcbc167d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbbc102af0 0x7fcbbc198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:27.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.264+0000 7fcbc167d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbbc1034b0 0x7fcbbc19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:27.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.264+0000 7fcbc167d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc10d430 con 0x7fcbbc0ffa80 2024-04-25T00:13:27.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.264+0000 7fcbc167d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc10d2b0 con 0x7fcbbc102af0 2024-04-25T00:13:27.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.264+0000 7fcbc167d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcbbc10d5b0 con 0x7fcbbc1034b0 2024-04-25T00:13:27.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.264+0000 7fcbbaffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbbc0ffa80 0x7fcbbc197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:27.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.264+0000 7fcbbaffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbbc0ffa80 0x7fcbbc197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46578/0 (socket says 172.21.15.107:46578) 2024-04-25T00:13:27.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.264+0000 7fcbbaffd640 1 -- 172.21.15.107:0/2127587308 learned_addr learned my addr 172.21.15.107:0/2127587308 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:27.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.265+0000 7fcbba7fc640 1 --2- 172.21.15.107:0/2127587308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbbc102af0 0x7fcbbc198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:27.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.265+0000 7fcbbb7fe640 1 --2- 172.21.15.107:0/2127587308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbbc1034b0 0x7fcbbc19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:27.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.265+0000 7fcbbaffd640 1 -- 172.21.15.107:0/2127587308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbbc1034b0 msgr2=0x7fcbbc19f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:27.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.265+0000 7fcbbaffd640 1 --2- 172.21.15.107:0/2127587308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbbc1034b0 0x7fcbbc19f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:27.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.265+0000 7fcbbaffd640 1 -- 172.21.15.107:0/2127587308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbbc102af0 msgr2=0x7fcbbc198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:27.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.265+0000 7fcbbaffd640 1 --2- 172.21.15.107:0/2127587308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbbc102af0 0x7fcbbc198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:27.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.265+0000 7fcbbaffd640 1 -- 172.21.15.107:0/2127587308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbbc19f730 con 0x7fcbbc0ffa80 2024-04-25T00:13:27.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.265+0000 7fcbba7fc640 1 --2- 172.21.15.107:0/2127587308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbbc102af0 0x7fcbbc198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:27.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.265+0000 7fcbbaffd640 1 --2- 172.21.15.107:0/2127587308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbbc0ffa80 0x7fcbbc197b10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcbac031440 tx=0x7fcbac0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:27.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.265+0000 7fcba3fff640 1 -- 172.21.15.107:0/2127587308 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac03f030 con 0x7fcbbc0ffa80 2024-04-25T00:13:27.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.265+0000 7fcbc167d640 1 -- 172.21.15.107:0/2127587308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbbc19f9c0 con 0x7fcbbc0ffa80 2024-04-25T00:13:27.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.265+0000 7fcba3fff640 1 -- 172.21.15.107:0/2127587308 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbac003ce0 con 0x7fcbbc0ffa80 2024-04-25T00:13:27.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.265+0000 7fcba3fff640 1 -- 172.21.15.107:0/2127587308 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac035070 con 0x7fcbbc0ffa80 2024-04-25T00:13:27.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.265+0000 7fcbc167d640 1 -- 172.21.15.107:0/2127587308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbbc19feb0 con 0x7fcbbc0ffa80 2024-04-25T00:13:27.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.266+0000 7fcbc167d640 1 -- 172.21.15.107:0/2127587308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbbc108c40 con 0x7fcbbc0ffa80 2024-04-25T00:13:27.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.267+0000 7fcba3fff640 1 -- 172.21.15.107:0/2127587308 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fcbac042030 con 0x7fcbbc0ffa80 2024-04-25T00:13:27.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.268+0000 7fcba3fff640 1 --2- 172.21.15.107:0/2127587308 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcba40a2b90 0x7fcba40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:27.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.268+0000 7fcbba7fc640 1 --2- 172.21.15.107:0/2127587308 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcba40a2b90 0x7fcba40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:27.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.268+0000 7fcba3fff640 1 -- 172.21.15.107:0/2127587308 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fcbac0ef210 con 0x7fcbbc0ffa80 2024-04-25T00:13:27.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.268+0000 7fcbba7fc640 1 --2- 172.21.15.107:0/2127587308 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcba40a2b90 0x7fcba40a5050 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00058c0 tx=0x7fcbb0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:27.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.272+0000 7fcba3fff640 1 -- 172.21.15.107:0/2127587308 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcbac0bc610 con 0x7fcbbc0ffa80 2024-04-25T00:13:27.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:27 smithi170 ceph-mon[32807]: mds.? [v2:172.21.15.195:6832/2162039457,v1:172.21.15.195:6833/2162039457] up:boot 2024-04-25T00:13:27.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:27 smithi170 ceph-mon[32807]: fsmap 1 up:standby 2024-04-25T00:13:27.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:27 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T00:13:27.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:27 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:27.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:27 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:27.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:27 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:27.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:27 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:27.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:27 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:27.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:27 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:27.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:27 smithi170 ceph-mon[32807]: Deploying daemon mds.k on smithi170 2024-04-25T00:13:27.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:27 smithi170 ceph-mon[32807]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:27.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:27 smithi107 ceph-mon[29153]: mds.? [v2:172.21.15.195:6832/2162039457,v1:172.21.15.195:6833/2162039457] up:boot 2024-04-25T00:13:27.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:27 smithi107 ceph-mon[29153]: fsmap 1 up:standby 2024-04-25T00:13:27.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:27 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T00:13:27.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:27 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:27.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:27 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:27.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:27 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:27.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:27 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:27.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:27 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:27.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:27 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:27.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:27 smithi107 ceph-mon[29153]: Deploying daemon mds.k on smithi170 2024-04-25T00:13:27.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:27 smithi107 ceph-mon[29153]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:27.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.486+0000 7fcbc167d640 1 -- 172.21.15.107:0/2127587308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbbc02d170 con 0x7fcbbc0ffa80 2024-04-25T00:13:27.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.486+0000 7fcba3fff640 1 -- 172.21.15.107:0/2127587308 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fcbac0bbf10 con 0x7fcbbc0ffa80 2024-04-25T00:13:27.488 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:13:27.488 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T00:13:27.488 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 2 2024-04-25T00:13:27.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.488+0000 7fcbc167d640 1 -- 172.21.15.107:0/2127587308 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcba40a2b90 msgr2=0x7fcba40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:27.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.488+0000 7fcbc167d640 1 --2- 172.21.15.107:0/2127587308 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcba40a2b90 0x7fcba40a5050 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00058c0 tx=0x7fcbb0005850 comp rx=0 tx=0).stop 2024-04-25T00:13:27.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.488+0000 7fcbc167d640 1 -- 172.21.15.107:0/2127587308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbbc0ffa80 msgr2=0x7fcbbc197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:27.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.488+0000 7fcbc167d640 1 --2- 172.21.15.107:0/2127587308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbbc0ffa80 0x7fcbbc197b10 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcbac031440 tx=0x7fcbac0043b0 comp rx=0 tx=0).stop 2024-04-25T00:13:27.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.488+0000 7fcbc167d640 1 -- 172.21.15.107:0/2127587308 shutdown_connections 2024-04-25T00:13:27.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.488+0000 7fcbc167d640 1 --2- 172.21.15.107:0/2127587308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcbbc1034b0 0x7fcbbc19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:27.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.488+0000 7fcbc167d640 1 --2- 172.21.15.107:0/2127587308 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcba40a2b90 0x7fcba40a5050 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:27.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.488+0000 7fcbc167d640 1 --2- 172.21.15.107:0/2127587308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcbbc102af0 0x7fcbbc198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:27.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.488+0000 7fcbc167d640 1 --2- 172.21.15.107:0/2127587308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbbc0ffa80 0x7fcbbc197b10 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:27.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.488+0000 7fcbc167d640 1 -- 172.21.15.107:0/2127587308 >> 172.21.15.107:0/2127587308 conn(0x7fcbbc0fb3b0 msgr2=0x7fcbbc105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:27.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.488+0000 7fcbc167d640 1 -- 172.21.15.107:0/2127587308 shutdown_connections 2024-04-25T00:13:27.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:27.488+0000 7fcbc167d640 1 -- 172.21.15.107:0/2127587308 wait complete. 2024-04-25T00:13:27.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:27 smithi195 ceph-mon[32833]: mds.? [v2:172.21.15.195:6832/2162039457,v1:172.21.15.195:6833/2162039457] up:boot 2024-04-25T00:13:27.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:27 smithi195 ceph-mon[32833]: fsmap 1 up:standby 2024-04-25T00:13:27.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:27 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T00:13:27.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:27 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:27.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:27 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:27.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:27 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:27.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:27 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:27.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:27 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:27.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:27 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:27.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:27 smithi195 ceph-mon[32833]: Deploying daemon mds.k on smithi170 2024-04-25T00:13:27.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:27 smithi195 ceph-mon[32833]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:28.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:28 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2127587308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:28.669 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:28 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2127587308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:28.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:28 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2127587308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:28.935 INFO:journalctl@ceph.mds.k.smithi170.stdout:Apr 25 00:13:28 smithi170 systemd[1]: Starting Ceph mds.k for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:13:29.322 INFO:journalctl@ceph.mds.k.smithi170.stdout:Apr 25 00:13:29 smithi170 podman[61692]: 2024-04-25 00:13:28.972164388 +0000 UTC m=+0.018015538 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:13:29.322 INFO:journalctl@ceph.mds.k.smithi170.stdout:Apr 25 00:13:29 smithi170 podman[61692]: 2024-04-25 00:13:29.086993019 +0000 UTC m=+0.132844214 container create 94fc407f53443b7ff85e6b4f7b2f32b4af8ac5b32352477f13977d6165646f79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T00:13:29.323 INFO:journalctl@ceph.mds.k.smithi170.stdout:Apr 25 00:13:29 smithi170 podman[61692]: 2024-04-25 00:13:29.322959742 +0000 UTC m=+0.368810891 container init 94fc407f53443b7ff85e6b4f7b2f32b4af8ac5b32352477f13977d6165646f79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:13:29.501 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:29.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.622+0000 7fd073472640 1 -- 172.21.15.107:0/1471380380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd06c100ff0 msgr2=0x7fd06c101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:29.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.622+0000 7fd073472640 1 --2- 172.21.15.107:0/1471380380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd06c100ff0 0x7fd06c101470 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd05c009900 tx=0x7fd05c02a410 comp rx=0 tx=0).stop 2024-04-25T00:13:29.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.622+0000 7fd073472640 1 -- 172.21.15.107:0/1471380380 shutdown_connections 2024-04-25T00:13:29.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.622+0000 7fd073472640 1 --2- 172.21.15.107:0/1471380380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd06c1019b0 0x7fd06c108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.622+0000 7fd073472640 1 --2- 172.21.15.107:0/1471380380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd06c100ff0 0x7fd06c101470 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.622+0000 7fd073472640 1 --2- 172.21.15.107:0/1471380380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd06c0ffe00 0x7fd06c100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.622+0000 7fd073472640 1 -- 172.21.15.107:0/1471380380 >> 172.21.15.107:0/1471380380 conn(0x7fd06c0fb370 msgr2=0x7fd06c0fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:29.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.622+0000 7fd073472640 1 -- 172.21.15.107:0/1471380380 shutdown_connections 2024-04-25T00:13:29.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.622+0000 7fd073472640 1 -- 172.21.15.107:0/1471380380 wait complete. 2024-04-25T00:13:29.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.623+0000 7fd073472640 1 Processor -- start 2024-04-25T00:13:29.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.623+0000 7fd073472640 1 -- start start 2024-04-25T00:13:29.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.623+0000 7fd073472640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd06c0ffe00 0x7fd06c196a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.623+0000 7fd073472640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd06c100ff0 0x7fd06c196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.623+0000 7fd073472640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd06c1019b0 0x7fd06c19dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.623+0000 7fd073472640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c10c380 con 0x7fd06c1019b0 2024-04-25T00:13:29.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.623+0000 7fd073472640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c10c200 con 0x7fd06c100ff0 2024-04-25T00:13:29.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.623+0000 7fd073472640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c10c500 con 0x7fd06c0ffe00 2024-04-25T00:13:29.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.623+0000 7fd071a0f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd06c1019b0 0x7fd06c19dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.623+0000 7fd071a0f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd06c1019b0 0x7fd06c19dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43638/0 (socket says 172.21.15.107:43638) 2024-04-25T00:13:29.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.623+0000 7fd071a0f640 1 -- 172.21.15.107:0/3414799180 learned_addr learned my addr 172.21.15.107:0/3414799180 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:29.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.623+0000 7fd070a0d640 1 --2- 172.21.15.107:0/3414799180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd06c100ff0 0x7fd06c196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.623+0000 7fd07120e640 1 --2- 172.21.15.107:0/3414799180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd06c0ffe00 0x7fd06c196a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.624+0000 7fd071a0f640 1 -- 172.21.15.107:0/3414799180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd06c0ffe00 msgr2=0x7fd06c196a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:29.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.624+0000 7fd071a0f640 1 --2- 172.21.15.107:0/3414799180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd06c0ffe00 0x7fd06c196a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.624+0000 7fd071a0f640 1 -- 172.21.15.107:0/3414799180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd06c100ff0 msgr2=0x7fd06c196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:29.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.624+0000 7fd071a0f640 1 --2- 172.21.15.107:0/3414799180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd06c100ff0 0x7fd06c196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.624+0000 7fd071a0f640 1 -- 172.21.15.107:0/3414799180 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06c19e620 con 0x7fd06c1019b0 2024-04-25T00:13:29.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.624+0000 7fd070a0d640 1 --2- 172.21.15.107:0/3414799180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd06c100ff0 0x7fd06c196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:29.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.624+0000 7fd071a0f640 1 --2- 172.21.15.107:0/3414799180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd06c1019b0 0x7fd06c19dfb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd06400ebb0 tx=0x7fd064007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:29.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.624+0000 7fd0627fc640 1 -- 172.21.15.107:0/3414799180 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd064007ca0 con 0x7fd06c1019b0 2024-04-25T00:13:29.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.624+0000 7fd0627fc640 1 -- 172.21.15.107:0/3414799180 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd064007e40 con 0x7fd06c1019b0 2024-04-25T00:13:29.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.624+0000 7fd073472640 1 -- 172.21.15.107:0/3414799180 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd06c108780 con 0x7fd06c1019b0 2024-04-25T00:13:29.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.624+0000 7fd0627fc640 1 -- 172.21.15.107:0/3414799180 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0640058a0 con 0x7fd06c1019b0 2024-04-25T00:13:29.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.624+0000 7fd073472640 1 -- 172.21.15.107:0/3414799180 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd06c108d00 con 0x7fd06c1019b0 2024-04-25T00:13:29.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.625+0000 7fd073472640 1 -- 172.21.15.107:0/3414799180 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd06c067cd0 con 0x7fd06c1019b0 2024-04-25T00:13:29.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.626+0000 7fd0627fc640 1 -- 172.21.15.107:0/3414799180 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fd064004a40 con 0x7fd06c1019b0 2024-04-25T00:13:29.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.626+0000 7fd0627fc640 1 --2- 172.21.15.107:0/3414799180 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd0580a2b90 0x7fd0580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:29.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.627+0000 7fd07120e640 1 --2- 172.21.15.107:0/3414799180 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd0580a2b90 0x7fd0580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:29.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.627+0000 7fd0627fc640 1 -- 172.21.15.107:0/3414799180 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd0640d01f0 con 0x7fd06c1019b0 2024-04-25T00:13:29.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.627+0000 7fd07120e640 1 --2- 172.21.15.107:0/3414799180 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd0580a2b90 0x7fd0580a5050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd06c100e50 tx=0x7fd068008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:29.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.631+0000 7fd0627fc640 1 -- 172.21.15.107:0/3414799180 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd06409d5f0 con 0x7fd06c1019b0 2024-04-25T00:13:29.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:29 smithi170 ceph-mon[32807]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:29.685 INFO:journalctl@ceph.mds.k.smithi170.stdout:Apr 25 00:13:29 smithi170 podman[61692]: 2024-04-25 00:13:29.325553124 +0000 UTC m=+0.371404284 container start 94fc407f53443b7ff85e6b4f7b2f32b4af8ac5b32352477f13977d6165646f79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-k, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T00:13:29.685 INFO:journalctl@ceph.mds.k.smithi170.stdout:Apr 25 00:13:29 smithi170 ceph-mds[61711]: -- 172.21.15.170:0/1995510875 <== mon.2 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c63994ba40 con 0x55c63991f000 2024-04-25T00:13:29.685 INFO:journalctl@ceph.mds.k.smithi170.stdout:Apr 25 00:13:29 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-k[61707]: starting mds.k at 2024-04-25T00:13:29.685 INFO:journalctl@ceph.mds.k.smithi170.stdout:Apr 25 00:13:29 smithi170 bash[61692]: 94fc407f53443b7ff85e6b4f7b2f32b4af8ac5b32352477f13977d6165646f79 2024-04-25T00:13:29.685 INFO:journalctl@ceph.mds.k.smithi170.stdout:Apr 25 00:13:29 smithi170 systemd[1]: Started Ceph mds.k for 79602f58-0297-11ef-bc93-c7b262605968. 2024-04-25T00:13:29.733 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:29 smithi107 ceph-mon[29153]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:29.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:29 smithi195 ceph-mon[32833]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:29.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.856+0000 7fd073472640 1 -- 172.21.15.107:0/3414799180 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd06c106d40 con 0x7fd06c1019b0 2024-04-25T00:13:29.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.857+0000 7fd0627fc640 1 -- 172.21.15.107:0/3414799180 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fd06409cef0 con 0x7fd06c1019b0 2024-04-25T00:13:29.858 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:13:29.858 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T00:13:29.859 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 2 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.859+0000 7fd073472640 1 -- 172.21.15.107:0/3414799180 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd0580a2b90 msgr2=0x7fd0580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.859+0000 7fd073472640 1 --2- 172.21.15.107:0/3414799180 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd0580a2b90 0x7fd0580a5050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd06c100e50 tx=0x7fd068008040 comp rx=0 tx=0).stop 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.859+0000 7fd073472640 1 -- 172.21.15.107:0/3414799180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd06c1019b0 msgr2=0x7fd06c19dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.859+0000 7fd073472640 1 --2- 172.21.15.107:0/3414799180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd06c1019b0 0x7fd06c19dfb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd06400ebb0 tx=0x7fd064007590 comp rx=0 tx=0).stop 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.859+0000 7fd073472640 1 -- 172.21.15.107:0/3414799180 shutdown_connections 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.859+0000 7fd073472640 1 --2- 172.21.15.107:0/3414799180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd06c1019b0 0x7fd06c19dfb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.859+0000 7fd073472640 1 --2- 172.21.15.107:0/3414799180 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd0580a2b90 0x7fd0580a5050 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.859+0000 7fd073472640 1 --2- 172.21.15.107:0/3414799180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd06c100ff0 0x7fd06c196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.859+0000 7fd073472640 1 --2- 172.21.15.107:0/3414799180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd06c0ffe00 0x7fd06c196a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.859+0000 7fd073472640 1 -- 172.21.15.107:0/3414799180 >> 172.21.15.107:0/3414799180 conn(0x7fd06c0fb370 msgr2=0x7fd06c1042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.859+0000 7fd073472640 1 -- 172.21.15.107:0/3414799180 shutdown_connections 2024-04-25T00:13:29.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:29.859+0000 7fd073472640 1 -- 172.21.15.107:0/3414799180 wait complete. 2024-04-25T00:13:30.750 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:30 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:30.751 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:30 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:30.751 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:30 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:30.751 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:30 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:30.751 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:30 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:30.751 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:30 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:30.751 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:30 smithi170 ceph-mon[32807]: Deploying daemon mds.j on smithi107 2024-04-25T00:13:30.751 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:30 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3414799180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:30.751 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:30 smithi170 ceph-mon[32807]: mds.? [v2:172.21.15.170:6832/674283763,v1:172.21.15.170:6833/674283763] up:boot 2024-04-25T00:13:30.751 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:30 smithi170 ceph-mon[32807]: fsmap 2 up:standby 2024-04-25T00:13:30.751 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:30 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T00:13:30.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:30 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:30.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:30 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:30.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:30 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:30.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:30 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:30.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:30 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:30.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:30 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:30.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:30 smithi195 ceph-mon[32833]: Deploying daemon mds.j on smithi107 2024-04-25T00:13:30.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:30 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3414799180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:30.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:30 smithi195 ceph-mon[32833]: mds.? [v2:172.21.15.170:6832/674283763,v1:172.21.15.170:6833/674283763] up:boot 2024-04-25T00:13:30.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:30 smithi195 ceph-mon[32833]: fsmap 2 up:standby 2024-04-25T00:13:30.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:30 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T00:13:30.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:30 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:30.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:30 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:30.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:30 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:30.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:30 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:30.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:30 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:30.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:30 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:30.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:30 smithi107 ceph-mon[29153]: Deploying daemon mds.j on smithi107 2024-04-25T00:13:30.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:30 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3414799180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:30.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:30 smithi107 ceph-mon[29153]: mds.? [v2:172.21.15.170:6832/674283763,v1:172.21.15.170:6833/674283763] up:boot 2024-04-25T00:13:30.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:30 smithi107 ceph-mon[29153]: fsmap 2 up:standby 2024-04-25T00:13:30.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:30 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T00:13:31.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:31 smithi195 ceph-mon[32833]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:31.873 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:31.894 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:31 smithi107 ceph-mon[29153]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:31.894 INFO:journalctl@ceph.mds.j.smithi107.stdout:Apr 25 00:13:31 smithi107 systemd[1]: Starting Ceph mds.j for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:13:31.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:31 smithi170 ceph-mon[32807]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:31.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.976+0000 7f672dba5640 1 -- 172.21.15.107:0/3327547361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6728102250 msgr2=0x7f6728108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:31.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.976+0000 7f672dba5640 1 --2- 172.21.15.107:0/3327547361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6728102250 0x7f6728108ae0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6720009ca0 tx=0x7f672002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:13:31.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.976+0000 7f672dba5640 1 -- 172.21.15.107:0/3327547361 shutdown_connections 2024-04-25T00:13:31.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.976+0000 7f672dba5640 1 --2- 172.21.15.107:0/3327547361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6728102250 0x7f6728108ae0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:31.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.976+0000 7f672dba5640 1 --2- 172.21.15.107:0/3327547361 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6728101890 0x7f6728101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:31.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.976+0000 7f672dba5640 1 --2- 172.21.15.107:0/3327547361 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67281006a0 0x7f6728100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:31.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.976+0000 7f672dba5640 1 -- 172.21.15.107:0/3327547361 >> 172.21.15.107:0/3327547361 conn(0x7f67280fbc30 msgr2=0x7f67280fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:31.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.976+0000 7f672dba5640 1 -- 172.21.15.107:0/3327547361 shutdown_connections 2024-04-25T00:13:31.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.976+0000 7f672dba5640 1 -- 172.21.15.107:0/3327547361 wait complete. 2024-04-25T00:13:31.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.976+0000 7f672dba5640 1 Processor -- start 2024-04-25T00:13:31.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.976+0000 7f672dba5640 1 -- start start 2024-04-25T00:13:31.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.977+0000 7f672dba5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67281006a0 0x7f672806a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:31.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.977+0000 7f672dba5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6728101890 0x7f672819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:31.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.977+0000 7f672dba5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6728102250 0x7f672819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:31.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.977+0000 7f672dba5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f672810c300 con 0x7f67281006a0 2024-04-25T00:13:31.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.977+0000 7f672dba5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f672810c180 con 0x7f6728102250 2024-04-25T00:13:31.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.977+0000 7f672dba5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f672810c480 con 0x7f6728101890 2024-04-25T00:13:31.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.977+0000 7f67277fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67281006a0 0x7f672806a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:31.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.977+0000 7f67277fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67281006a0 0x7f672806a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43650/0 (socket says 172.21.15.107:43650) 2024-04-25T00:13:31.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.977+0000 7f67277fe640 1 -- 172.21.15.107:0/3255540119 learned_addr learned my addr 172.21.15.107:0/3255540119 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:31.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.977+0000 7f6726ffd640 1 --2- 172.21.15.107:0/3255540119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6728101890 0x7f672819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:31.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.977+0000 7f67277fe640 1 -- 172.21.15.107:0/3255540119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6728101890 msgr2=0x7f672819c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:31.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.977+0000 7f67277fe640 1 --2- 172.21.15.107:0/3255540119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6728101890 0x7f672819c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:31.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.977+0000 7f67277fe640 1 -- 172.21.15.107:0/3255540119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6728102250 msgr2=0x7f672819ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:31.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.977+0000 7f6727fff640 1 --2- 172.21.15.107:0/3255540119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6728102250 0x7f672819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:31.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.977+0000 7f67277fe640 1 --2- 172.21.15.107:0/3255540119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6728102250 0x7f672819ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:31.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.977+0000 7f67277fe640 1 -- 172.21.15.107:0/3255540119 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672819f6a0 con 0x7f67281006a0 2024-04-25T00:13:31.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.977+0000 7f6726ffd640 1 --2- 172.21.15.107:0/3255540119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6728101890 0x7f672819c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:31.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.977+0000 7f67277fe640 1 --2- 172.21.15.107:0/3255540119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67281006a0 0x7f672806a930 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f671800d950 tx=0x7f671800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:31.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.977+0000 7f6724ff9640 1 -- 172.21.15.107:0/3255540119 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718017020 con 0x7f67281006a0 2024-04-25T00:13:31.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.977+0000 7f6724ff9640 1 -- 172.21.15.107:0/3255540119 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6718009c30 con 0x7f67281006a0 2024-04-25T00:13:31.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.977+0000 7f672dba5640 1 -- 172.21.15.107:0/3255540119 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f672819f990 con 0x7f67281006a0 2024-04-25T00:13:31.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.977+0000 7f6724ff9640 1 -- 172.21.15.107:0/3255540119 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671800f740 con 0x7f67281006a0 2024-04-25T00:13:31.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.977+0000 7f672dba5640 1 -- 172.21.15.107:0/3255540119 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f672819fee0 con 0x7f67281006a0 2024-04-25T00:13:31.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.978+0000 7f672dba5640 1 -- 172.21.15.107:0/3255540119 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67280689b0 con 0x7f67281006a0 2024-04-25T00:13:31.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.979+0000 7f6724ff9640 1 -- 172.21.15.107:0/3255540119 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f671801a020 con 0x7f67281006a0 2024-04-25T00:13:31.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.980+0000 7f6724ff9640 1 --2- 172.21.15.107:0/3255540119 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f67100a2b90 0x7f67100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:31.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.980+0000 7f6726ffd640 1 --2- 172.21.15.107:0/3255540119 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f67100a2b90 0x7f67100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:31.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.980+0000 7f6724ff9640 1 -- 172.21.15.107:0/3255540119 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f67180ce650 con 0x7f67281006a0 2024-04-25T00:13:31.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.980+0000 7f6726ffd640 1 --2- 172.21.15.107:0/3255540119 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f67100a2b90 0x7f67100a5050 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f672806a6a0 tx=0x7f671c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:31.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:31.984+0000 7f6724ff9640 1 -- 172.21.15.107:0/3255540119 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6718021070 con 0x7f67281006a0 2024-04-25T00:13:32.162 INFO:journalctl@ceph.mds.j.smithi107.stdout:Apr 25 00:13:31 smithi107 podman[66599]: 2024-04-25 00:13:31.797854256 +0000 UTC m=+0.018120257 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:13:32.162 INFO:journalctl@ceph.mds.j.smithi107.stdout:Apr 25 00:13:31 smithi107 podman[66599]: 2024-04-25 00:13:31.91369415 +0000 UTC m=+0.133960141 container create 260019258da807624c706215e501e7c6c75f508e9836cc3ce41b229ab983b647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9) 2024-04-25T00:13:32.162 INFO:journalctl@ceph.mds.j.smithi107.stdout:Apr 25 00:13:32 smithi107 podman[66599]: 2024-04-25 00:13:32.134010149 +0000 UTC m=+0.354276149 container init 260019258da807624c706215e501e7c6c75f508e9836cc3ce41b229ab983b647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-j, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9) 2024-04-25T00:13:32.162 INFO:journalctl@ceph.mds.j.smithi107.stdout:Apr 25 00:13:32 smithi107 podman[66599]: 2024-04-25 00:13:32.137418988 +0000 UTC m=+0.357684986 container start 260019258da807624c706215e501e7c6c75f508e9836cc3ce41b229ab983b647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:13:32.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:32.216+0000 7f672dba5640 1 -- 172.21.15.107:0/3255540119 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6728107a90 con 0x7f67281006a0 2024-04-25T00:13:32.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:32.216+0000 7f6724ff9640 1 -- 172.21.15.107:0/3255540119 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f6718009de0 con 0x7f67281006a0 2024-04-25T00:13:32.218 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:13:32.218 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T00:13:32.218 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 3 2024-04-25T00:13:32.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:32.220+0000 7f672dba5640 1 -- 172.21.15.107:0/3255540119 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f67100a2b90 msgr2=0x7f67100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:32.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:32.220+0000 7f672dba5640 1 --2- 172.21.15.107:0/3255540119 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f67100a2b90 0x7f67100a5050 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f672806a6a0 tx=0x7f671c005f90 comp rx=0 tx=0).stop 2024-04-25T00:13:32.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:32.220+0000 7f672dba5640 1 -- 172.21.15.107:0/3255540119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67281006a0 msgr2=0x7f672806a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:32.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:32.220+0000 7f672dba5640 1 --2- 172.21.15.107:0/3255540119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67281006a0 0x7f672806a930 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f671800d950 tx=0x7f671800de10 comp rx=0 tx=0).stop 2024-04-25T00:13:32.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:32.220+0000 7f672dba5640 1 -- 172.21.15.107:0/3255540119 shutdown_connections 2024-04-25T00:13:32.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:32.220+0000 7f672dba5640 1 --2- 172.21.15.107:0/3255540119 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f67100a2b90 0x7f67100a5050 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:32.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:32.220+0000 7f672dba5640 1 --2- 172.21.15.107:0/3255540119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6728102250 0x7f672819ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:32.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:32.220+0000 7f672dba5640 1 --2- 172.21.15.107:0/3255540119 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6728101890 0x7f672819c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:32.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:32.220+0000 7f672dba5640 1 --2- 172.21.15.107:0/3255540119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67281006a0 0x7f672806a930 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:32.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:32.220+0000 7f672dba5640 1 -- 172.21.15.107:0/3255540119 >> 172.21.15.107:0/3255540119 conn(0x7f67280fbc30 msgr2=0x7f6728104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:32.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:32.220+0000 7f672dba5640 1 -- 172.21.15.107:0/3255540119 shutdown_connections 2024-04-25T00:13:32.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:32.220+0000 7f672dba5640 1 -- 172.21.15.107:0/3255540119 wait complete. 2024-04-25T00:13:32.484 INFO:journalctl@ceph.mds.j.smithi107.stdout:Apr 25 00:13:32 smithi107 ceph-mds[66657]: -- 172.21.15.107:0/3677657774 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e1881cba40 con 0x55e18819ec00 2024-04-25T00:13:32.484 INFO:journalctl@ceph.mds.j.smithi107.stdout:Apr 25 00:13:32 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-j[66653]: starting mds.j at 2024-04-25T00:13:32.484 INFO:journalctl@ceph.mds.j.smithi107.stdout:Apr 25 00:13:32 smithi107 bash[66599]: 260019258da807624c706215e501e7c6c75f508e9836cc3ce41b229ab983b647 2024-04-25T00:13:32.484 INFO:journalctl@ceph.mds.j.smithi107.stdout:Apr 25 00:13:32 smithi107 systemd[1]: Started Ceph mds.j for 79602f58-0297-11ef-bc93-c7b262605968. 2024-04-25T00:13:32.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:32 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3255540119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:32.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:32 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:32.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:32 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:32.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:32 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:32.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:32 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:32.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:32 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:32.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:32 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:32.883 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:32 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3255540119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:32.884 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:32 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:32.884 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:32 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:32.884 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:32 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:32.884 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:32 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:32.884 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:32 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:32.884 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:32 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:32.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:32 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3255540119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:32.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:32 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:32.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:32 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:32.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:32 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:32.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:32 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:32.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:32 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:32.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:32 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:33.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:33 smithi195 ceph-mon[32833]: Deploying daemon mds.d on smithi107 2024-04-25T00:13:33.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:33 smithi195 ceph-mon[32833]: mds.? [v2:172.21.15.107:6834/2628342623,v1:172.21.15.107:6835/2628342623] up:boot 2024-04-25T00:13:33.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:33 smithi195 ceph-mon[32833]: fsmap 3 up:standby 2024-04-25T00:13:33.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:33 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T00:13:33.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:33 smithi195 ceph-mon[32833]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:33.888 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:33 smithi107 ceph-mon[29153]: Deploying daemon mds.d on smithi107 2024-04-25T00:13:33.888 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:33 smithi107 ceph-mon[29153]: mds.? [v2:172.21.15.107:6834/2628342623,v1:172.21.15.107:6835/2628342623] up:boot 2024-04-25T00:13:33.888 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:33 smithi107 ceph-mon[29153]: fsmap 3 up:standby 2024-04-25T00:13:33.888 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:33 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T00:13:33.889 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:33 smithi107 ceph-mon[29153]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:33.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:33 smithi170 ceph-mon[32807]: Deploying daemon mds.d on smithi107 2024-04-25T00:13:33.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:33 smithi170 ceph-mon[32807]: mds.? [v2:172.21.15.107:6834/2628342623,v1:172.21.15.107:6835/2628342623] up:boot 2024-04-25T00:13:33.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:33 smithi170 ceph-mon[32807]: fsmap 3 up:standby 2024-04-25T00:13:33.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:33 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T00:13:33.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:33 smithi170 ceph-mon[32807]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:34.234 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:34.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.355+0000 7fdd81d9e640 1 -- 172.21.15.107:0/1045305762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd7c102070 msgr2=0x7fdd7c108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.355+0000 7fdd81d9e640 1 --2- 172.21.15.107:0/1045305762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd7c102070 0x7fdd7c108900 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdd74009ca0 tx=0x7fdd7402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.355+0000 7fdd81d9e640 1 -- 172.21.15.107:0/1045305762 shutdown_connections 2024-04-25T00:13:34.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.355+0000 7fdd81d9e640 1 --2- 172.21.15.107:0/1045305762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd7c102070 0x7fdd7c108900 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.355+0000 7fdd81d9e640 1 --2- 172.21.15.107:0/1045305762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd7c1016b0 0x7fdd7c101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.355+0000 7fdd81d9e640 1 --2- 172.21.15.107:0/1045305762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd7c1004c0 0x7fdd7c1008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.355+0000 7fdd81d9e640 1 -- 172.21.15.107:0/1045305762 >> 172.21.15.107:0/1045305762 conn(0x7fdd7c0fba10 msgr2=0x7fdd7c0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:34.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.355+0000 7fdd81d9e640 1 -- 172.21.15.107:0/1045305762 shutdown_connections 2024-04-25T00:13:34.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.355+0000 7fdd81d9e640 1 -- 172.21.15.107:0/1045305762 wait complete. 2024-04-25T00:13:34.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.355+0000 7fdd81d9e640 1 Processor -- start 2024-04-25T00:13:34.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.355+0000 7fdd81d9e640 1 -- start start 2024-04-25T00:13:34.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.355+0000 7fdd81d9e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd7c1004c0 0x7fdd7c069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.355+0000 7fdd81d9e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd7c1016b0 0x7fdd7c19e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.355+0000 7fdd81d9e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd7c102070 0x7fdd7c1a0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.355+0000 7fdd81d9e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7c10c150 con 0x7fdd7c1016b0 2024-04-25T00:13:34.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.355+0000 7fdd81d9e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7c10bfd0 con 0x7fdd7c1004c0 2024-04-25T00:13:34.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.355+0000 7fdd81d9e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7c10c2d0 con 0x7fdd7c102070 2024-04-25T00:13:34.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.356+0000 7fdd7bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd7c1016b0 0x7fdd7c19e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.356+0000 7fdd7bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd7c1016b0 0x7fdd7c19e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43680/0 (socket says 172.21.15.107:43680) 2024-04-25T00:13:34.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.356+0000 7fdd7bfff640 1 -- 172.21.15.107:0/3290549471 learned_addr learned my addr 172.21.15.107:0/3290549471 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:34.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.356+0000 7fdd8159d640 1 --2- 172.21.15.107:0/3290549471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd7c102070 0x7fdd7c1a0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.356+0000 7fdd80d9c640 1 --2- 172.21.15.107:0/3290549471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd7c1004c0 0x7fdd7c069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.356+0000 7fdd7bfff640 1 -- 172.21.15.107:0/3290549471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd7c102070 msgr2=0x7fdd7c1a0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.356+0000 7fdd7bfff640 1 --2- 172.21.15.107:0/3290549471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd7c102070 0x7fdd7c1a0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.356+0000 7fdd7bfff640 1 -- 172.21.15.107:0/3290549471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd7c1004c0 msgr2=0x7fdd7c069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.356+0000 7fdd7bfff640 1 --2- 172.21.15.107:0/3290549471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd7c1004c0 0x7fdd7c069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.356+0000 7fdd7bfff640 1 -- 172.21.15.107:0/3290549471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7c1a1710 con 0x7fdd7c1016b0 2024-04-25T00:13:34.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.356+0000 7fdd80d9c640 1 --2- 172.21.15.107:0/3290549471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd7c1004c0 0x7fdd7c069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:34.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.356+0000 7fdd7bfff640 1 --2- 172.21.15.107:0/3290549471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd7c1016b0 0x7fdd7c19e9d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800d770 tx=0x7fdd6800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:34.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.356+0000 7fdd79ffb640 1 -- 172.21.15.107:0/3290549471 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68017070 con 0x7fdd7c1016b0 2024-04-25T00:13:34.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.356+0000 7fdd81d9e640 1 -- 172.21.15.107:0/3290549471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd7c1a19a0 con 0x7fdd7c1016b0 2024-04-25T00:13:34.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.356+0000 7fdd79ffb640 1 -- 172.21.15.107:0/3290549471 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd68005500 con 0x7fdd7c1016b0 2024-04-25T00:13:34.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.356+0000 7fdd79ffb640 1 -- 172.21.15.107:0/3290549471 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68004e00 con 0x7fdd7c1016b0 2024-04-25T00:13:34.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.357+0000 7fdd81d9e640 1 -- 172.21.15.107:0/3290549471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd7c1a1ef0 con 0x7fdd7c1016b0 2024-04-25T00:13:34.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.357+0000 7fdd81d9e640 1 -- 172.21.15.107:0/3290549471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd7c1077e0 con 0x7fdd7c1016b0 2024-04-25T00:13:34.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.359+0000 7fdd79ffb640 1 -- 172.21.15.107:0/3290549471 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fdd68004fa0 con 0x7fdd7c1016b0 2024-04-25T00:13:34.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.360+0000 7fdd79ffb640 1 --2- 172.21.15.107:0/3290549471 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fdd6c0a2bc0 0x7fdd6c0a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:34.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.360+0000 7fdd80d9c640 1 --2- 172.21.15.107:0/3290549471 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fdd6c0a2bc0 0x7fdd6c0a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:34.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.360+0000 7fdd79ffb640 1 -- 172.21.15.107:0/3290549471 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fdd680ce570 con 0x7fdd7c1016b0 2024-04-25T00:13:34.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.360+0000 7fdd80d9c640 1 --2- 172.21.15.107:0/3290549471 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fdd6c0a2bc0 0x7fdd6c0a5080 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdd70005d00 tx=0x7fdd70005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:34.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.368+0000 7fdd79ffb640 1 -- 172.21.15.107:0/3290549471 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdd6809b970 con 0x7fdd7c1016b0 2024-04-25T00:13:34.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.599+0000 7fdd81d9e640 1 -- 172.21.15.107:0/3290549471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd7c02d370 con 0x7fdd7c1016b0 2024-04-25T00:13:34.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.600+0000 7fdd79ffb640 1 -- 172.21.15.107:0/3290549471 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3049 (secure 0 0 0) 0x7fdd6809b270 con 0x7fdd7c1016b0 2024-04-25T00:13:34.602 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:13:34.602 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T00:13:34.603 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 4 2024-04-25T00:13:34.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.602+0000 7fdd81d9e640 1 -- 172.21.15.107:0/3290549471 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fdd6c0a2bc0 msgr2=0x7fdd6c0a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.602+0000 7fdd81d9e640 1 --2- 172.21.15.107:0/3290549471 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fdd6c0a2bc0 0x7fdd6c0a5080 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdd70005d00 tx=0x7fdd70005c90 comp rx=0 tx=0).stop 2024-04-25T00:13:34.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.602+0000 7fdd81d9e640 1 -- 172.21.15.107:0/3290549471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd7c1016b0 msgr2=0x7fdd7c19e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:34.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.602+0000 7fdd81d9e640 1 --2- 172.21.15.107:0/3290549471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd7c1016b0 0x7fdd7c19e9d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800d770 tx=0x7fdd6800dc30 comp rx=0 tx=0).stop 2024-04-25T00:13:34.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.602+0000 7fdd81d9e640 1 -- 172.21.15.107:0/3290549471 shutdown_connections 2024-04-25T00:13:34.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.602+0000 7fdd81d9e640 1 --2- 172.21.15.107:0/3290549471 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd7c102070 0x7fdd7c1a0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.602+0000 7fdd81d9e640 1 --2- 172.21.15.107:0/3290549471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd7c1016b0 0x7fdd7c19e9d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.602+0000 7fdd81d9e640 1 --2- 172.21.15.107:0/3290549471 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fdd6c0a2bc0 0x7fdd6c0a5080 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.602+0000 7fdd81d9e640 1 --2- 172.21.15.107:0/3290549471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd7c1004c0 0x7fdd7c069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:34.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.602+0000 7fdd81d9e640 1 -- 172.21.15.107:0/3290549471 >> 172.21.15.107:0/3290549471 conn(0x7fdd7c0fba10 msgr2=0x7fdd7c1049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:34.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.602+0000 7fdd81d9e640 1 -- 172.21.15.107:0/3290549471 shutdown_connections 2024-04-25T00:13:34.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:34.602+0000 7fdd81d9e640 1 -- 172.21.15.107:0/3290549471 wait complete. 2024-04-25T00:13:34.614 INFO:journalctl@ceph.mds.d.smithi107.stdout:Apr 25 00:13:34 smithi107 systemd[1]: Starting Ceph mds.d for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:13:34.932 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:34 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3290549471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:34.932 INFO:journalctl@ceph.mds.d.smithi107.stdout:Apr 25 00:13:34 smithi107 podman[66985]: 2024-04-25 00:13:34.584064654 +0000 UTC m=+0.018349885 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:13:34.932 INFO:journalctl@ceph.mds.d.smithi107.stdout:Apr 25 00:13:34 smithi107 podman[66985]: 2024-04-25 00:13:34.688138244 +0000 UTC m=+0.122423448 container create df3dd6c9d0de6bed60777bd070d7f1053733ca867888c8c46818917ace4f1425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T00:13:34.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:34 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3290549471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:35.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:34 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3290549471' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:35.235 INFO:journalctl@ceph.mds.d.smithi107.stdout:Apr 25 00:13:34 smithi107 podman[66985]: 2024-04-25 00:13:34.932362724 +0000 UTC m=+0.366647930 container init df3dd6c9d0de6bed60777bd070d7f1053733ca867888c8c46818917ace4f1425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-25T00:13:35.235 INFO:journalctl@ceph.mds.d.smithi107.stdout:Apr 25 00:13:34 smithi107 podman[66985]: 2024-04-25 00:13:34.935271131 +0000 UTC m=+0.369556329 container start df3dd6c9d0de6bed60777bd070d7f1053733ca867888c8c46818917ace4f1425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T00:13:35.235 INFO:journalctl@ceph.mds.d.smithi107.stdout:Apr 25 00:13:34 smithi107 ceph-mds[67006]: -- 172.21.15.107:0/3305465309 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557d0548ba40 con 0x557d0545e800 2024-04-25T00:13:35.235 INFO:journalctl@ceph.mds.d.smithi107.stdout:Apr 25 00:13:34 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-d[67002]: starting mds.d at 2024-04-25T00:13:35.235 INFO:journalctl@ceph.mds.d.smithi107.stdout:Apr 25 00:13:35 smithi107 bash[66985]: df3dd6c9d0de6bed60777bd070d7f1053733ca867888c8c46818917ace4f1425 2024-04-25T00:13:35.235 INFO:journalctl@ceph.mds.d.smithi107.stdout:Apr 25 00:13:35 smithi107 systemd[1]: Started Ceph mds.d for 79602f58-0297-11ef-bc93-c7b262605968. 2024-04-25T00:13:35.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:35 smithi170 ceph-mon[32807]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:35.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:35 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:35.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:35 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:35.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:35 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:35.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:35 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:35.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:35 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:35.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:35 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:35.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:35 smithi107 ceph-mon[29153]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:35.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:35 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:35.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:35 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:35.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:35 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:35.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:35 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:35.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:35 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:35.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:35 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:36.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:35 smithi195 ceph-mon[32833]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:36.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:35 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:36.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:35 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:36.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:35 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:36.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:35 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:36.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:35 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:36.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:35 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:36.616 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:36.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.730+0000 7f89f6f4f640 1 -- 172.21.15.107:0/1954510371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89f0101870 msgr2=0x7f89f0101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:36.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.730+0000 7f89f6f4f640 1 --2- 172.21.15.107:0/1954510371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89f0101870 0x7f89f0101cf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f89dc009900 tx=0x7f89dc02a410 comp rx=0 tx=0).stop 2024-04-25T00:13:36.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.730+0000 7f89f6f4f640 1 -- 172.21.15.107:0/1954510371 shutdown_connections 2024-04-25T00:13:36.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.730+0000 7f89f6f4f640 1 --2- 172.21.15.107:0/1954510371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89f0102230 0x7f89f0108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.730+0000 7f89f6f4f640 1 --2- 172.21.15.107:0/1954510371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89f0101870 0x7f89f0101cf0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.730+0000 7f89f6f4f640 1 --2- 172.21.15.107:0/1954510371 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89f0100680 0x7f89f0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.730+0000 7f89f6f4f640 1 -- 172.21.15.107:0/1954510371 >> 172.21.15.107:0/1954510371 conn(0x7f89f00fbbf0 msgr2=0x7f89f00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:36.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.730+0000 7f89f6f4f640 1 -- 172.21.15.107:0/1954510371 shutdown_connections 2024-04-25T00:13:36.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.730+0000 7f89f6f4f640 1 -- 172.21.15.107:0/1954510371 wait complete. 2024-04-25T00:13:36.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.731+0000 7f89f6f4f640 1 Processor -- start 2024-04-25T00:13:36.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.731+0000 7f89f6f4f640 1 -- start start 2024-04-25T00:13:36.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.731+0000 7f89f6f4f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89f0100680 0x7f89f01969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:36.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.731+0000 7f89f6f4f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89f0101870 0x7f89f0196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:36.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.731+0000 7f89f6f4f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89f0102230 0x7f89f019df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:36.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.731+0000 7f89f6f4f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f89f010c300 con 0x7f89f0102230 2024-04-25T00:13:36.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.731+0000 7f89f6f4f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f89f010c180 con 0x7f89f0101870 2024-04-25T00:13:36.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.731+0000 7f89f6f4f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f89f010c480 con 0x7f89f0100680 2024-04-25T00:13:36.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.731+0000 7f89f54ec640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89f0102230 0x7f89f019df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:36.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.731+0000 7f89f54ec640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89f0102230 0x7f89f019df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43724/0 (socket says 172.21.15.107:43724) 2024-04-25T00:13:36.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.731+0000 7f89f54ec640 1 -- 172.21.15.107:0/246756606 learned_addr learned my addr 172.21.15.107:0/246756606 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:36.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.731+0000 7f89effff640 1 --2- 172.21.15.107:0/246756606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89f0101870 0x7f89f0196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:36.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.731+0000 7f89f4ceb640 1 --2- 172.21.15.107:0/246756606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89f0100680 0x7f89f01969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:36.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.731+0000 7f89f54ec640 1 -- 172.21.15.107:0/246756606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89f0100680 msgr2=0x7f89f01969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:36.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.731+0000 7f89f54ec640 1 --2- 172.21.15.107:0/246756606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89f0100680 0x7f89f01969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.731+0000 7f89f54ec640 1 -- 172.21.15.107:0/246756606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89f0101870 msgr2=0x7f89f0196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:36.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.731+0000 7f89f54ec640 1 --2- 172.21.15.107:0/246756606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89f0101870 0x7f89f0196ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.731+0000 7f89f54ec640 1 -- 172.21.15.107:0/246756606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f019e5c0 con 0x7f89f0102230 2024-04-25T00:13:36.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.731+0000 7f89effff640 1 --2- 172.21.15.107:0/246756606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89f0101870 0x7f89f0196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:36.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.731+0000 7f89f54ec640 1 --2- 172.21.15.107:0/246756606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89f0102230 0x7f89f019df50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f89e800eaa0 tx=0x7f89e800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:36.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.732+0000 7f89edffb640 1 -- 172.21.15.107:0/246756606 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e800cdb0 con 0x7f89f0102230 2024-04-25T00:13:36.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.732+0000 7f89edffb640 1 -- 172.21.15.107:0/246756606 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89e8005020 con 0x7f89f0102230 2024-04-25T00:13:36.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.732+0000 7f89f6f4f640 1 -- 172.21.15.107:0/246756606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89f019e8b0 con 0x7f89f0102230 2024-04-25T00:13:36.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.732+0000 7f89edffb640 1 -- 172.21.15.107:0/246756606 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e80116b0 con 0x7f89f0102230 2024-04-25T00:13:36.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.732+0000 7f89f6f4f640 1 -- 172.21.15.107:0/246756606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89f019ed80 con 0x7f89f0102230 2024-04-25T00:13:36.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.733+0000 7f89f6f4f640 1 -- 172.21.15.107:0/246756606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89f0107a20 con 0x7f89f0102230 2024-04-25T00:13:36.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.734+0000 7f89edffb640 1 -- 172.21.15.107:0/246756606 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f89e8005560 con 0x7f89f0102230 2024-04-25T00:13:36.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.735+0000 7f89edffb640 1 --2- 172.21.15.107:0/246756606 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f89e00a2b90 0x7f89e00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:36.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.735+0000 7f89f4ceb640 1 --2- 172.21.15.107:0/246756606 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f89e00a2b90 0x7f89e00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:36.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.735+0000 7f89edffb640 1 -- 172.21.15.107:0/246756606 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f89e80ceed0 con 0x7f89f0102230 2024-04-25T00:13:36.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.735+0000 7f89f4ceb640 1 --2- 172.21.15.107:0/246756606 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f89e00a2b90 0x7f89e00a5050 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f89e4005fd0 tx=0x7f89e4005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:36.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.739+0000 7f89edffb640 1 -- 172.21.15.107:0/246756606 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f89e8016030 con 0x7f89f0102230 2024-04-25T00:13:36.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:36 smithi170 ceph-mon[32807]: Deploying daemon mds.e on smithi170 2024-04-25T00:13:36.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:36 smithi170 ceph-mon[32807]: mds.? [v2:172.21.15.107:6836/2695944193,v1:172.21.15.107:6837/2695944193] up:boot 2024-04-25T00:13:36.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:36 smithi170 ceph-mon[32807]: fsmap 4 up:standby 2024-04-25T00:13:36.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:36 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T00:13:36.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.951+0000 7f89f6f4f640 1 -- 172.21.15.107:0/246756606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89f002d170 con 0x7f89f0102230 2024-04-25T00:13:36.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.951+0000 7f89edffb640 1 -- 172.21.15.107:0/246756606 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3877 (secure 0 0 0) 0x7f89e809c2d0 con 0x7f89f0102230 2024-04-25T00:13:36.952 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:13:36.952 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T00:13:36.953 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 5 2024-04-25T00:13:36.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.953+0000 7f89f6f4f640 1 -- 172.21.15.107:0/246756606 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f89e00a2b90 msgr2=0x7f89e00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:36.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.953+0000 7f89f6f4f640 1 --2- 172.21.15.107:0/246756606 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f89e00a2b90 0x7f89e00a5050 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f89e4005fd0 tx=0x7f89e4005850 comp rx=0 tx=0).stop 2024-04-25T00:13:36.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.953+0000 7f89f6f4f640 1 -- 172.21.15.107:0/246756606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89f0102230 msgr2=0x7f89f019df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:36.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.953+0000 7f89f6f4f640 1 --2- 172.21.15.107:0/246756606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89f0102230 0x7f89f019df50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f89e800eaa0 tx=0x7f89e800ef60 comp rx=0 tx=0).stop 2024-04-25T00:13:36.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.953+0000 7f89f6f4f640 1 -- 172.21.15.107:0/246756606 shutdown_connections 2024-04-25T00:13:36.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.953+0000 7f89f6f4f640 1 --2- 172.21.15.107:0/246756606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89f0102230 0x7f89f019df50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.953+0000 7f89f6f4f640 1 --2- 172.21.15.107:0/246756606 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f89e00a2b90 0x7f89e00a5050 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.953+0000 7f89f6f4f640 1 --2- 172.21.15.107:0/246756606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f89f0101870 0x7f89f0196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.953+0000 7f89f6f4f640 1 --2- 172.21.15.107:0/246756606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89f0100680 0x7f89f01969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:36.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.953+0000 7f89f6f4f640 1 -- 172.21.15.107:0/246756606 >> 172.21.15.107:0/246756606 conn(0x7f89f00fbbf0 msgr2=0x7f89f0104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:36.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.954+0000 7f89f6f4f640 1 -- 172.21.15.107:0/246756606 shutdown_connections 2024-04-25T00:13:36.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:36.954+0000 7f89f6f4f640 1 -- 172.21.15.107:0/246756606 wait complete. 2024-04-25T00:13:36.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:36 smithi107 ceph-mon[29153]: Deploying daemon mds.e on smithi170 2024-04-25T00:13:36.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:36 smithi107 ceph-mon[29153]: mds.? [v2:172.21.15.107:6836/2695944193,v1:172.21.15.107:6837/2695944193] up:boot 2024-04-25T00:13:36.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:36 smithi107 ceph-mon[29153]: fsmap 4 up:standby 2024-04-25T00:13:36.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:36 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T00:13:37.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:36 smithi195 ceph-mon[32833]: Deploying daemon mds.e on smithi170 2024-04-25T00:13:37.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:36 smithi195 ceph-mon[32833]: mds.? [v2:172.21.15.107:6836/2695944193,v1:172.21.15.107:6837/2695944193] up:boot 2024-04-25T00:13:37.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:36 smithi195 ceph-mon[32833]: fsmap 4 up:standby 2024-04-25T00:13:37.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:36 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T00:13:37.422 INFO:journalctl@ceph.mds.e.smithi170.stdout:Apr 25 00:13:37 smithi170 systemd[1]: Starting Ceph mds.e for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:13:37.682 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:37 smithi170 ceph-mon[32807]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:37.682 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:37 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/246756606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:37.683 INFO:journalctl@ceph.mds.e.smithi170.stdout:Apr 25 00:13:37 smithi170 podman[62009]: 2024-04-25 00:13:37.32524424 +0000 UTC m=+0.018470594 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:13:37.683 INFO:journalctl@ceph.mds.e.smithi170.stdout:Apr 25 00:13:37 smithi170 podman[62009]: 2024-04-25 00:13:37.443640218 +0000 UTC m=+0.136866589 container create 2bd248e19770260215a5b5acb5f4e893f71dd94577dadb0b0b585059d5f9b57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T00:13:37.683 INFO:journalctl@ceph.mds.e.smithi170.stdout:Apr 25 00:13:37 smithi170 podman[62009]: 2024-04-25 00:13:37.679738064 +0000 UTC m=+0.372964381 container init 2bd248e19770260215a5b5acb5f4e893f71dd94577dadb0b0b585059d5f9b57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-25T00:13:37.935 INFO:journalctl@ceph.mds.e.smithi170.stdout:Apr 25 00:13:37 smithi170 podman[62009]: 2024-04-25 00:13:37.68250794 +0000 UTC m=+0.375734254 container start 2bd248e19770260215a5b5acb5f4e893f71dd94577dadb0b0b585059d5f9b57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T00:13:37.935 INFO:journalctl@ceph.mds.e.smithi170.stdout:Apr 25 00:13:37 smithi170 ceph-mds[62028]: -- 172.21.15.170:0/659677221 <== mon.2 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b69a171a40 con 0x55b69a144c00 2024-04-25T00:13:37.935 INFO:journalctl@ceph.mds.e.smithi170.stdout:Apr 25 00:13:37 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-e[62024]: starting mds.e at 2024-04-25T00:13:37.935 INFO:journalctl@ceph.mds.e.smithi170.stdout:Apr 25 00:13:37 smithi170 bash[62009]: 2bd248e19770260215a5b5acb5f4e893f71dd94577dadb0b0b585059d5f9b57e 2024-04-25T00:13:37.935 INFO:journalctl@ceph.mds.e.smithi170.stdout:Apr 25 00:13:37 smithi170 systemd[1]: Started Ceph mds.e for 79602f58-0297-11ef-bc93-c7b262605968. 2024-04-25T00:13:37.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:37 smithi107 ceph-mon[29153]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:37.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:37 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/246756606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:38.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:37 smithi195 ceph-mon[32833]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:38.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:37 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/246756606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:38.967 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:39.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.077+0000 7fcf96761640 1 -- 172.21.15.107:0/2933495738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf90101890 msgr2=0x7fcf90101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:39.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.077+0000 7fcf96761640 1 --2- 172.21.15.107:0/2933495738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf90101890 0x7fcf90101d10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcf80009900 tx=0x7fcf8002a410 comp rx=0 tx=0).stop 2024-04-25T00:13:39.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.077+0000 7fcf96761640 1 -- 172.21.15.107:0/2933495738 shutdown_connections 2024-04-25T00:13:39.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.077+0000 7fcf96761640 1 --2- 172.21.15.107:0/2933495738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf90102250 0x7fcf90108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.077+0000 7fcf96761640 1 --2- 172.21.15.107:0/2933495738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf90101890 0x7fcf90101d10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.077+0000 7fcf96761640 1 --2- 172.21.15.107:0/2933495738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf901006a0 0x7fcf90100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.077+0000 7fcf96761640 1 -- 172.21.15.107:0/2933495738 >> 172.21.15.107:0/2933495738 conn(0x7fcf900fbc30 msgr2=0x7fcf900fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:39.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.077+0000 7fcf96761640 1 -- 172.21.15.107:0/2933495738 shutdown_connections 2024-04-25T00:13:39.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.077+0000 7fcf96761640 1 -- 172.21.15.107:0/2933495738 wait complete. 2024-04-25T00:13:39.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.077+0000 7fcf96761640 1 Processor -- start 2024-04-25T00:13:39.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.077+0000 7fcf96761640 1 -- start start 2024-04-25T00:13:39.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.078+0000 7fcf96761640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf901006a0 0x7fcf9006a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:39.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.078+0000 7fcf96761640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf90101890 0x7fcf9019c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:39.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.078+0000 7fcf96761640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf90102250 0x7fcf9019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:39.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.078+0000 7fcf96761640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9010c320 con 0x7fcf90102250 2024-04-25T00:13:39.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.078+0000 7fcf96761640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9010c1a0 con 0x7fcf901006a0 2024-04-25T00:13:39.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.078+0000 7fcf96761640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9010c4a0 con 0x7fcf90101890 2024-04-25T00:13:39.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.078+0000 7fcf94cfe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf90102250 0x7fcf9019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:39.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.078+0000 7fcf94cfe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf90102250 0x7fcf9019ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43742/0 (socket says 172.21.15.107:43742) 2024-04-25T00:13:39.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.078+0000 7fcf94cfe640 1 -- 172.21.15.107:0/3727204509 learned_addr learned my addr 172.21.15.107:0/3727204509 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:39.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.078+0000 7fcf8ffff640 1 --2- 172.21.15.107:0/3727204509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf901006a0 0x7fcf9006a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:39.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.078+0000 7fcf8f7fe640 1 --2- 172.21.15.107:0/3727204509 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf90101890 0x7fcf9019c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:39.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.078+0000 7fcf94cfe640 1 -- 172.21.15.107:0/3727204509 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf90101890 msgr2=0x7fcf9019c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:39.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.078+0000 7fcf94cfe640 1 --2- 172.21.15.107:0/3727204509 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf90101890 0x7fcf9019c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.078+0000 7fcf94cfe640 1 -- 172.21.15.107:0/3727204509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf901006a0 msgr2=0x7fcf9006a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:39.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.078+0000 7fcf94cfe640 1 --2- 172.21.15.107:0/3727204509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf901006a0 0x7fcf9006a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.078+0000 7fcf94cfe640 1 -- 172.21.15.107:0/3727204509 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9019f770 con 0x7fcf90102250 2024-04-25T00:13:39.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.078+0000 7fcf94cfe640 1 --2- 172.21.15.107:0/3727204509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf90102250 0x7fcf9019ee60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009fd0 tx=0x7fcf8800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:39.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.078+0000 7fcf8d7fa640 1 -- 172.21.15.107:0/3727204509 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf88017070 con 0x7fcf90102250 2024-04-25T00:13:39.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.078+0000 7fcf96761640 1 -- 172.21.15.107:0/3727204509 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf9019fa00 con 0x7fcf90102250 2024-04-25T00:13:39.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.078+0000 7fcf8d7fa640 1 -- 172.21.15.107:0/3727204509 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf88003e50 con 0x7fcf90102250 2024-04-25T00:13:39.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.078+0000 7fcf8d7fa640 1 -- 172.21.15.107:0/3727204509 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf880106c0 con 0x7fcf90102250 2024-04-25T00:13:39.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.079+0000 7fcf96761640 1 -- 172.21.15.107:0/3727204509 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf9019fed0 con 0x7fcf90102250 2024-04-25T00:13:39.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.079+0000 7fcf96761640 1 -- 172.21.15.107:0/3727204509 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf900689b0 con 0x7fcf90102250 2024-04-25T00:13:39.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.081+0000 7fcf8d7fa640 1 -- 172.21.15.107:0/3727204509 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fcf88003970 con 0x7fcf90102250 2024-04-25T00:13:39.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.081+0000 7fcf8d7fa640 1 --2- 172.21.15.107:0/3727204509 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcf780a2b90 0x7fcf780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:39.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.081+0000 7fcf8ffff640 1 --2- 172.21.15.107:0/3727204509 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcf780a2b90 0x7fcf780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:39.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.082+0000 7fcf8d7fa640 1 -- 172.21.15.107:0/3727204509 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fcf880ce890 con 0x7fcf90102250 2024-04-25T00:13:39.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.082+0000 7fcf8ffff640 1 --2- 172.21.15.107:0/3727204509 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcf780a2b90 0x7fcf780a5050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcf84005fd0 tx=0x7fcf8400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:39.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.086+0000 7fcf8d7fa640 1 -- 172.21.15.107:0/3727204509 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcf8809b640 con 0x7fcf90102250 2024-04-25T00:13:39.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:38 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:39.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:38 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:39.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:38 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:39.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:38 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:39.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:38 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:39.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:38 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:39.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:38 smithi195 ceph-mon[32833]: Deploying daemon mds.f on smithi195 2024-04-25T00:13:39.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:38 smithi195 ceph-mon[32833]: mds.? [v2:172.21.15.170:6834/2926641872,v1:172.21.15.170:6835/2926641872] up:boot 2024-04-25T00:13:39.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:38 smithi195 ceph-mon[32833]: fsmap 5 up:standby 2024-04-25T00:13:39.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:38 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:13:39.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:38 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:39.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:38 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:39.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:38 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:39.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:38 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:39.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:38 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:39.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:38 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:39.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:38 smithi170 ceph-mon[32807]: Deploying daemon mds.f on smithi195 2024-04-25T00:13:39.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:38 smithi170 ceph-mon[32807]: mds.? [v2:172.21.15.170:6834/2926641872,v1:172.21.15.170:6835/2926641872] up:boot 2024-04-25T00:13:39.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:38 smithi170 ceph-mon[32807]: fsmap 5 up:standby 2024-04-25T00:13:39.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:38 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:13:39.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:38 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:39.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:38 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:39.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:38 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:39.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:38 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:39.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:38 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:39.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:38 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:39.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:38 smithi107 ceph-mon[29153]: Deploying daemon mds.f on smithi195 2024-04-25T00:13:39.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:38 smithi107 ceph-mon[29153]: mds.? [v2:172.21.15.170:6834/2926641872,v1:172.21.15.170:6835/2926641872] up:boot 2024-04-25T00:13:39.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:38 smithi107 ceph-mon[29153]: fsmap 5 up:standby 2024-04-25T00:13:39.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:38 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:13:39.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.299+0000 7fcf96761640 1 -- 172.21.15.107:0/3727204509 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf9002d1f0 con 0x7fcf90102250 2024-04-25T00:13:39.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.300+0000 7fcf8d7fa640 1 -- 172.21.15.107:0/3727204509 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4705 (secure 0 0 0) 0x7fcf88097020 con 0x7fcf90102250 2024-04-25T00:13:39.300 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:13:39.301 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T00:13:39.301 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 6 2024-04-25T00:13:39.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.302+0000 7fcf96761640 1 -- 172.21.15.107:0/3727204509 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcf780a2b90 msgr2=0x7fcf780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:39.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.302+0000 7fcf96761640 1 --2- 172.21.15.107:0/3727204509 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcf780a2b90 0x7fcf780a5050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcf84005fd0 tx=0x7fcf8400b390 comp rx=0 tx=0).stop 2024-04-25T00:13:39.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.302+0000 7fcf96761640 1 -- 172.21.15.107:0/3727204509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf90102250 msgr2=0x7fcf9019ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:39.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.302+0000 7fcf96761640 1 --2- 172.21.15.107:0/3727204509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf90102250 0x7fcf9019ee60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009fd0 tx=0x7fcf8800b260 comp rx=0 tx=0).stop 2024-04-25T00:13:39.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.302+0000 7fcf96761640 1 -- 172.21.15.107:0/3727204509 shutdown_connections 2024-04-25T00:13:39.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.302+0000 7fcf96761640 1 --2- 172.21.15.107:0/3727204509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf90102250 0x7fcf9019ee60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.302+0000 7fcf96761640 1 --2- 172.21.15.107:0/3727204509 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf90101890 0x7fcf9019c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.302+0000 7fcf96761640 1 --2- 172.21.15.107:0/3727204509 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcf780a2b90 0x7fcf780a5050 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.302+0000 7fcf96761640 1 --2- 172.21.15.107:0/3727204509 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf901006a0 0x7fcf9006a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:39.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.302+0000 7fcf96761640 1 -- 172.21.15.107:0/3727204509 >> 172.21.15.107:0/3727204509 conn(0x7fcf900fbc30 msgr2=0x7fcf90104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:39.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.302+0000 7fcf96761640 1 -- 172.21.15.107:0/3727204509 shutdown_connections 2024-04-25T00:13:39.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:39.302+0000 7fcf96761640 1 -- 172.21.15.107:0/3727204509 wait complete. 2024-04-25T00:13:40.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:39 smithi195 ceph-mon[32833]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:40.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:39 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3727204509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:40.090 INFO:journalctl@ceph.mds.f.smithi195.stdout:Apr 25 00:13:39 smithi195 systemd[1]: Starting Ceph mds.f for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:13:40.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:39 smithi170 ceph-mon[32807]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:40.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:39 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3727204509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:40.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:39 smithi107 ceph-mon[29153]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:40.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:39 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3727204509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:40.458 INFO:journalctl@ceph.mds.f.smithi195.stdout:Apr 25 00:13:40 smithi195 podman[63468]: 2024-04-25 00:13:40.097779569 +0000 UTC m=+0.018102982 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:13:40.458 INFO:journalctl@ceph.mds.f.smithi195.stdout:Apr 25 00:13:40 smithi195 podman[63468]: 2024-04-25 00:13:40.213384714 +0000 UTC m=+0.133708112 container create ad2dd935d23d8f8ad1d28e84ab2886a99dd941d73148e548ebe233ed3dc35893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T00:13:40.458 INFO:journalctl@ceph.mds.f.smithi195.stdout:Apr 25 00:13:40 smithi195 podman[63468]: 2024-04-25 00:13:40.432618128 +0000 UTC m=+0.352941528 container init ad2dd935d23d8f8ad1d28e84ab2886a99dd941d73148e548ebe233ed3dc35893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T00:13:40.458 INFO:journalctl@ceph.mds.f.smithi195.stdout:Apr 25 00:13:40 smithi195 podman[63468]: 2024-04-25 00:13:40.435281272 +0000 UTC m=+0.355604672 container start ad2dd935d23d8f8ad1d28e84ab2886a99dd941d73148e548ebe233ed3dc35893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-25T00:13:40.839 INFO:journalctl@ceph.mds.f.smithi195.stdout:Apr 25 00:13:40 smithi195 ceph-mds[63487]: -- 172.21.15.195:0/3479242036 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561f1d763a40 con 0x561f1d736c00 2024-04-25T00:13:40.839 INFO:journalctl@ceph.mds.f.smithi195.stdout:Apr 25 00:13:40 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-f[63483]: starting mds.f at 2024-04-25T00:13:40.839 INFO:journalctl@ceph.mds.f.smithi195.stdout:Apr 25 00:13:40 smithi195 bash[63468]: ad2dd935d23d8f8ad1d28e84ab2886a99dd941d73148e548ebe233ed3dc35893 2024-04-25T00:13:40.839 INFO:journalctl@ceph.mds.f.smithi195.stdout:Apr 25 00:13:40 smithi195 systemd[1]: Started Ceph mds.f for 79602f58-0297-11ef-bc93-c7b262605968. 2024-04-25T00:13:41.318 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:41.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.428+0000 7f262b3c8640 1 -- 172.21.15.107:0/152355838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2624102980 msgr2=0x7f2624102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:41.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.428+0000 7f262b3c8640 1 --2- 172.21.15.107:0/152355838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2624102980 0x7f2624102e00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2614009900 tx=0x7f261402a410 comp rx=0 tx=0).stop 2024-04-25T00:13:41.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.428+0000 7f262b3c8640 1 -- 172.21.15.107:0/152355838 shutdown_connections 2024-04-25T00:13:41.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.428+0000 7f262b3c8640 1 --2- 172.21.15.107:0/152355838 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2624103340 0x7f2624109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:41.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.428+0000 7f262b3c8640 1 --2- 172.21.15.107:0/152355838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2624102980 0x7f2624102e00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:41.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.428+0000 7f262b3c8640 1 --2- 172.21.15.107:0/152355838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2624101790 0x7f2624101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:41.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.428+0000 7f262b3c8640 1 -- 172.21.15.107:0/152355838 >> 172.21.15.107:0/152355838 conn(0x7f26240fcd20 msgr2=0x7f26240ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:41.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.428+0000 7f262b3c8640 1 -- 172.21.15.107:0/152355838 shutdown_connections 2024-04-25T00:13:41.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.428+0000 7f262b3c8640 1 -- 172.21.15.107:0/152355838 wait complete. 2024-04-25T00:13:41.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.429+0000 7f262b3c8640 1 Processor -- start 2024-04-25T00:13:41.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.429+0000 7f262b3c8640 1 -- start start 2024-04-25T00:13:41.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.429+0000 7f262b3c8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2624101790 0x7f2624197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:41.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.429+0000 7f262b3c8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2624102980 0x7f2624197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:41.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.429+0000 7f262b3c8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2624103340 0x7f262419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:41.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.429+0000 7f262b3c8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f262410d410 con 0x7f2624102980 2024-04-25T00:13:41.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.429+0000 7f262b3c8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f262410d290 con 0x7f2624101790 2024-04-25T00:13:41.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.429+0000 7f262b3c8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f262410d590 con 0x7f2624103340 2024-04-25T00:13:41.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.429+0000 7f2628963640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2624102980 0x7f2624197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:41.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.429+0000 7f2628963640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2624102980 0x7f2624197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53132/0 (socket says 172.21.15.107:53132) 2024-04-25T00:13:41.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.429+0000 7f2628963640 1 -- 172.21.15.107:0/257635453 learned_addr learned my addr 172.21.15.107:0/257635453 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:41.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.429+0000 7f2629164640 1 --2- 172.21.15.107:0/257635453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2624101790 0x7f2624197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:41.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.430+0000 7f2629965640 1 --2- 172.21.15.107:0/257635453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2624103340 0x7f262419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:41.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.430+0000 7f2628963640 1 -- 172.21.15.107:0/257635453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2624103340 msgr2=0x7f262419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:41.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.430+0000 7f2628963640 1 --2- 172.21.15.107:0/257635453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2624103340 0x7f262419f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:41.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.430+0000 7f2628963640 1 -- 172.21.15.107:0/257635453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2624101790 msgr2=0x7f2624197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:41.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.430+0000 7f2628963640 1 --2- 172.21.15.107:0/257635453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2624101790 0x7f2624197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:41.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.430+0000 7f2628963640 1 -- 172.21.15.107:0/257635453 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262419f680 con 0x7f2624102980 2024-04-25T00:13:41.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.430+0000 7f2628963640 1 --2- 172.21.15.107:0/257635453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2624102980 0x7f2624197fa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2614009230 tx=0x7f2614031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:41.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.430+0000 7f261a7fc640 1 -- 172.21.15.107:0/257635453 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261403f030 con 0x7f2624102980 2024-04-25T00:13:41.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.430+0000 7f261a7fc640 1 -- 172.21.15.107:0/257635453 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2614003ad0 con 0x7f2624102980 2024-04-25T00:13:41.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.430+0000 7f262b3c8640 1 -- 172.21.15.107:0/257635453 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f262419f910 con 0x7f2624102980 2024-04-25T00:13:41.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.430+0000 7f261a7fc640 1 -- 172.21.15.107:0/257635453 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614035070 con 0x7f2624102980 2024-04-25T00:13:41.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.430+0000 7f262b3c8640 1 -- 172.21.15.107:0/257635453 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f262419fe00 con 0x7f2624102980 2024-04-25T00:13:41.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.431+0000 7f262b3c8640 1 -- 172.21.15.107:0/257635453 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2624108b30 con 0x7f2624102980 2024-04-25T00:13:41.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.432+0000 7f261a7fc640 1 -- 172.21.15.107:0/257635453 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f2614042030 con 0x7f2624102980 2024-04-25T00:13:41.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.433+0000 7f261a7fc640 1 --2- 172.21.15.107:0/257635453 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f26100a2b90 0x7f26100a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:41.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.433+0000 7f2629164640 1 --2- 172.21.15.107:0/257635453 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f26100a2b90 0x7f26100a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:41.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.433+0000 7f261a7fc640 1 -- 172.21.15.107:0/257635453 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f26140ef580 con 0x7f2624102980 2024-04-25T00:13:41.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.433+0000 7f2629164640 1 --2- 172.21.15.107:0/257635453 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f26100a2b90 0x7f26100a5050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f26241027e0 tx=0x7f262000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:41.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.436+0000 7f261a7fc640 1 -- 172.21.15.107:0/257635453 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f26140bc980 con 0x7f2624102980 2024-04-25T00:13:41.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.648+0000 7f262b3c8640 1 -- 172.21.15.107:0/257635453 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f262402d170 con 0x7f2624102980 2024-04-25T00:13:41.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.649+0000 7f261a7fc640 1 -- 172.21.15.107:0/257635453 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5533 (secure 0 0 0) 0x7f26140bc280 con 0x7f2624102980 2024-04-25T00:13:41.650 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:13:41.650 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T00:13:41.650 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 7 2024-04-25T00:13:41.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.651+0000 7f262b3c8640 1 -- 172.21.15.107:0/257635453 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f26100a2b90 msgr2=0x7f26100a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:41.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.651+0000 7f262b3c8640 1 --2- 172.21.15.107:0/257635453 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f26100a2b90 0x7f26100a5050 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f26241027e0 tx=0x7f262000b390 comp rx=0 tx=0).stop 2024-04-25T00:13:41.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.651+0000 7f262b3c8640 1 -- 172.21.15.107:0/257635453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2624102980 msgr2=0x7f2624197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:41.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.651+0000 7f262b3c8640 1 --2- 172.21.15.107:0/257635453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2624102980 0x7f2624197fa0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2614009230 tx=0x7f2614031b60 comp rx=0 tx=0).stop 2024-04-25T00:13:41.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.651+0000 7f262b3c8640 1 -- 172.21.15.107:0/257635453 shutdown_connections 2024-04-25T00:13:41.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.651+0000 7f262b3c8640 1 --2- 172.21.15.107:0/257635453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2624103340 0x7f262419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:41.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.651+0000 7f262b3c8640 1 --2- 172.21.15.107:0/257635453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2624102980 0x7f2624197fa0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:41.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.651+0000 7f262b3c8640 1 --2- 172.21.15.107:0/257635453 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f26100a2b90 0x7f26100a5050 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:41.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.651+0000 7f262b3c8640 1 --2- 172.21.15.107:0/257635453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2624101790 0x7f2624197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:41.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.651+0000 7f262b3c8640 1 -- 172.21.15.107:0/257635453 >> 172.21.15.107:0/257635453 conn(0x7f26240fcd20 msgr2=0x7f2624105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:41.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.651+0000 7f262b3c8640 1 -- 172.21.15.107:0/257635453 shutdown_connections 2024-04-25T00:13:41.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:41.651+0000 7f262b3c8640 1 -- 172.21.15.107:0/257635453 wait complete. 2024-04-25T00:13:41.661 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:41 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:41.661 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:41 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:41.661 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:41 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:41.661 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:41 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:41.662 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:41 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:41.662 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:41 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:41.662 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:41 smithi107 ceph-mon[29153]: Deploying daemon mds.c on smithi195 2024-04-25T00:13:41.662 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:41 smithi107 ceph-mon[29153]: mds.? [v2:172.21.15.195:6834/2828335061,v1:172.21.15.195:6835/2828335061] up:boot 2024-04-25T00:13:41.662 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:41 smithi107 ceph-mon[29153]: fsmap 6 up:standby 2024-04-25T00:13:41.662 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:41 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:13:41.662 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:41 smithi107 ceph-mon[29153]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:41.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:41 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:41.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:41 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:41.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:41 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:41.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:41 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:41.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:41 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:41.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:41 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:41.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:41 smithi195 ceph-mon[32833]: Deploying daemon mds.c on smithi195 2024-04-25T00:13:41.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:41 smithi195 ceph-mon[32833]: mds.? [v2:172.21.15.195:6834/2828335061,v1:172.21.15.195:6835/2828335061] up:boot 2024-04-25T00:13:41.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:41 smithi195 ceph-mon[32833]: fsmap 6 up:standby 2024-04-25T00:13:41.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:41 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:13:41.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:41 smithi195 ceph-mon[32833]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:41.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:41 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:41.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:41 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:41.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:41 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:41.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:41 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:41.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:41 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:41.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:41 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:41.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:41 smithi170 ceph-mon[32807]: Deploying daemon mds.c on smithi195 2024-04-25T00:13:41.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:41 smithi170 ceph-mon[32807]: mds.? [v2:172.21.15.195:6834/2828335061,v1:172.21.15.195:6835/2828335061] up:boot 2024-04-25T00:13:41.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:41 smithi170 ceph-mon[32807]: fsmap 6 up:standby 2024-04-25T00:13:41.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:41 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:13:41.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:41 smithi170 ceph-mon[32807]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:42.837 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:42 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/257635453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:42.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:42 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/257635453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:42.839 INFO:journalctl@ceph.mds.c.smithi195.stdout:Apr 25 00:13:42 smithi195 systemd[1]: Starting Ceph mds.c for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:13:42.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:42 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/257635453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:43.197 INFO:journalctl@ceph.mds.c.smithi195.stdout:Apr 25 00:13:42 smithi195 podman[63769]: 2024-04-25 00:13:42.819116895 +0000 UTC m=+0.018239434 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:13:43.197 INFO:journalctl@ceph.mds.c.smithi195.stdout:Apr 25 00:13:42 smithi195 podman[63769]: 2024-04-25 00:13:42.952998723 +0000 UTC m=+0.152121241 container create 2eabc1c069458c00056194044b6b66ade50f14a35d6db4cca1a2eeec7867387e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-25T00:13:43.197 INFO:journalctl@ceph.mds.c.smithi195.stdout:Apr 25 00:13:43 smithi195 podman[63769]: 2024-04-25 00:13:43.197299532 +0000 UTC m=+0.396422051 container init 2eabc1c069458c00056194044b6b66ade50f14a35d6db4cca1a2eeec7867387e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:13:43.573 INFO:journalctl@ceph.mds.c.smithi195.stdout:Apr 25 00:13:43 smithi195 podman[63769]: 2024-04-25 00:13:43.200595904 +0000 UTC m=+0.399718424 container start 2eabc1c069458c00056194044b6b66ade50f14a35d6db4cca1a2eeec7867387e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T00:13:43.573 INFO:journalctl@ceph.mds.c.smithi195.stdout:Apr 25 00:13:43 smithi195 ceph-mds[63788]: -- 172.21.15.195:0/3565930099 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55af61343a40 con 0x55af61316c00 2024-04-25T00:13:43.573 INFO:journalctl@ceph.mds.c.smithi195.stdout:Apr 25 00:13:43 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-c[63784]: starting mds.c at 2024-04-25T00:13:43.573 INFO:journalctl@ceph.mds.c.smithi195.stdout:Apr 25 00:13:43 smithi195 bash[63769]: 2eabc1c069458c00056194044b6b66ade50f14a35d6db4cca1a2eeec7867387e 2024-04-25T00:13:43.574 INFO:journalctl@ceph.mds.c.smithi195.stdout:Apr 25 00:13:43 smithi195 systemd[1]: Started Ceph mds.c for 79602f58-0297-11ef-bc93-c7b262605968. 2024-04-25T00:13:43.663 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:43.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.773+0000 7ff213963640 1 -- 172.21.15.107:0/3543753396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff20c100680 msgr2=0x7ff20c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:43.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.773+0000 7ff213963640 1 --2- 172.21.15.107:0/3543753396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff20c100680 0x7ff20c100aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009900 tx=0x7ff1fc02a410 comp rx=0 tx=0).stop 2024-04-25T00:13:43.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.773+0000 7ff213963640 1 -- 172.21.15.107:0/3543753396 shutdown_connections 2024-04-25T00:13:43.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.773+0000 7ff213963640 1 --2- 172.21.15.107:0/3543753396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff20c102230 0x7ff20c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:43.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.773+0000 7ff213963640 1 --2- 172.21.15.107:0/3543753396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff20c101870 0x7ff20c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:43.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.773+0000 7ff213963640 1 --2- 172.21.15.107:0/3543753396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff20c100680 0x7ff20c100aa0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:43.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.773+0000 7ff213963640 1 -- 172.21.15.107:0/3543753396 >> 172.21.15.107:0/3543753396 conn(0x7ff20c0fbbf0 msgr2=0x7ff20c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:43.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.773+0000 7ff213963640 1 -- 172.21.15.107:0/3543753396 shutdown_connections 2024-04-25T00:13:43.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.773+0000 7ff213963640 1 -- 172.21.15.107:0/3543753396 wait complete. 2024-04-25T00:13:43.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.773+0000 7ff213963640 1 Processor -- start 2024-04-25T00:13:43.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.773+0000 7ff213963640 1 -- start start 2024-04-25T00:13:43.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.773+0000 7ff213963640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff20c100680 0x7ff20c069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:43.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.773+0000 7ff213963640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff20c101870 0x7ff20c19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:43.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.773+0000 7ff213963640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff20c102230 0x7ff20c19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:43.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.773+0000 7ff213963640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c10c300 con 0x7ff20c101870 2024-04-25T00:13:43.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.773+0000 7ff213963640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c10c180 con 0x7ff20c102230 2024-04-25T00:13:43.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.773+0000 7ff213963640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c10c480 con 0x7ff20c100680 2024-04-25T00:13:43.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.774+0000 7ff210efe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff20c101870 0x7ff20c19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:43.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.774+0000 7ff210efe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff20c101870 0x7ff20c19c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53162/0 (socket says 172.21.15.107:53162) 2024-04-25T00:13:43.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.774+0000 7ff210efe640 1 -- 172.21.15.107:0/3166761035 learned_addr learned my addr 172.21.15.107:0/3166761035 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:43.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.774+0000 7ff211f00640 1 --2- 172.21.15.107:0/3166761035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff20c102230 0x7ff20c19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:43.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.774+0000 7ff2116ff640 1 --2- 172.21.15.107:0/3166761035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff20c100680 0x7ff20c069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:43.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.774+0000 7ff210efe640 1 -- 172.21.15.107:0/3166761035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff20c100680 msgr2=0x7ff20c069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:43.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.774+0000 7ff210efe640 1 --2- 172.21.15.107:0/3166761035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff20c100680 0x7ff20c069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:43.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.774+0000 7ff210efe640 1 -- 172.21.15.107:0/3166761035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff20c102230 msgr2=0x7ff20c19ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:43.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.774+0000 7ff210efe640 1 --2- 172.21.15.107:0/3166761035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff20c102230 0x7ff20c19ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:43.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.774+0000 7ff210efe640 1 -- 172.21.15.107:0/3166761035 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff20c19f750 con 0x7ff20c101870 2024-04-25T00:13:43.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.774+0000 7ff210efe640 1 --2- 172.21.15.107:0/3166761035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff20c101870 0x7ff20c19c820 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff20800e9f0 tx=0x7ff20800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:43.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.774+0000 7ff2027fc640 1 -- 172.21.15.107:0/3166761035 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20800cd80 con 0x7ff20c101870 2024-04-25T00:13:43.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.774+0000 7ff213963640 1 -- 172.21.15.107:0/3166761035 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff20c19f9e0 con 0x7ff20c101870 2024-04-25T00:13:43.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.774+0000 7ff2027fc640 1 -- 172.21.15.107:0/3166761035 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff208005500 con 0x7ff20c101870 2024-04-25T00:13:43.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.774+0000 7ff2027fc640 1 -- 172.21.15.107:0/3166761035 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208010690 con 0x7ff20c101870 2024-04-25T00:13:43.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.774+0000 7ff213963640 1 -- 172.21.15.107:0/3166761035 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff20c19ff30 con 0x7ff20c101870 2024-04-25T00:13:43.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.775+0000 7ff213963640 1 -- 172.21.15.107:0/3166761035 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff20c1079f0 con 0x7ff20c101870 2024-04-25T00:13:43.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.776+0000 7ff2027fc640 1 -- 172.21.15.107:0/3166761035 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff2080046c0 con 0x7ff20c101870 2024-04-25T00:13:43.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.777+0000 7ff2027fc640 1 --2- 172.21.15.107:0/3166761035 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff1f80a2b90 0x7ff1f80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:43.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.777+0000 7ff2116ff640 1 --2- 172.21.15.107:0/3166761035 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff1f80a2b90 0x7ff1f80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:43.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.777+0000 7ff2027fc640 1 -- 172.21.15.107:0/3166761035 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff2080cefc0 con 0x7ff20c101870 2024-04-25T00:13:43.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.778+0000 7ff2116ff640 1 --2- 172.21.15.107:0/3166761035 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff1f80a2b90 0x7ff1f80a5050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc005ab0 tx=0x7ff1fc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:43.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.782+0000 7ff2027fc640 1 -- 172.21.15.107:0/3166761035 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff20809c3c0 con 0x7ff20c101870 2024-04-25T00:13:43.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:43 smithi195 ceph-mon[32833]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:43.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:43 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:43.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:43 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:43.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:43 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:43.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:43 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:43.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:43 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:43.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:43 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:43.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:43 smithi170 ceph-mon[32807]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:43.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:43 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:43.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:43 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:43.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:43 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:43.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:43 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:43.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:43 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:43.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:43 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:43.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:43 smithi107 ceph-mon[29153]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:43.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:43 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:43.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:43 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:43.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:43 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:43.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:43 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:43.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:43 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:43.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:43 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:43.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.999+0000 7ff213963640 1 -- 172.21.15.107:0/3166761035 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff20c02d3c0 con 0x7ff20c101870 2024-04-25T00:13:43.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:43.999+0000 7ff2027fc640 1 -- 172.21.15.107:0/3166761035 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6361 (secure 0 0 0) 0x7ff20809bcc0 con 0x7ff20c101870 2024-04-25T00:13:44.000 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:13:44.000 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T00:13:44.000 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 8 2024-04-25T00:13:44.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:44.001+0000 7ff213963640 1 -- 172.21.15.107:0/3166761035 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff1f80a2b90 msgr2=0x7ff1f80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:44.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:44.001+0000 7ff213963640 1 --2- 172.21.15.107:0/3166761035 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff1f80a2b90 0x7ff1f80a5050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc005ab0 tx=0x7ff1fc000bc0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:44.001+0000 7ff213963640 1 -- 172.21.15.107:0/3166761035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff20c101870 msgr2=0x7ff20c19c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:44.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:44.001+0000 7ff213963640 1 --2- 172.21.15.107:0/3166761035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff20c101870 0x7ff20c19c820 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff20800e9f0 tx=0x7ff20800edf0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:44.001+0000 7ff213963640 1 -- 172.21.15.107:0/3166761035 shutdown_connections 2024-04-25T00:13:44.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:44.001+0000 7ff213963640 1 --2- 172.21.15.107:0/3166761035 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff1f80a2b90 0x7ff1f80a5050 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:44.001+0000 7ff213963640 1 --2- 172.21.15.107:0/3166761035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff20c102230 0x7ff20c19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:44.001+0000 7ff213963640 1 --2- 172.21.15.107:0/3166761035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff20c101870 0x7ff20c19c820 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:44.001+0000 7ff213963640 1 --2- 172.21.15.107:0/3166761035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff20c100680 0x7ff20c069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:44.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:44.001+0000 7ff213963640 1 -- 172.21.15.107:0/3166761035 >> 172.21.15.107:0/3166761035 conn(0x7ff20c0fbbf0 msgr2=0x7ff20c104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:44.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:44.001+0000 7ff213963640 1 -- 172.21.15.107:0/3166761035 shutdown_connections 2024-04-25T00:13:44.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:44.001+0000 7ff213963640 1 -- 172.21.15.107:0/3166761035 wait complete. 2024-04-25T00:13:44.621 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:44 smithi195 ceph-mon[32833]: Deploying daemon mds.h on smithi170 2024-04-25T00:13:44.621 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:44 smithi195 ceph-mon[32833]: mds.? [v2:172.21.15.195:6836/2245799531,v1:172.21.15.195:6837/2245799531] up:boot 2024-04-25T00:13:44.621 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:44 smithi195 ceph-mon[32833]: fsmap 7 up:standby 2024-04-25T00:13:44.621 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:44 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:13:44.621 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:44 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3166761035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:44.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:44 smithi170 ceph-mon[32807]: Deploying daemon mds.h on smithi170 2024-04-25T00:13:44.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:44 smithi170 ceph-mon[32807]: mds.? [v2:172.21.15.195:6836/2245799531,v1:172.21.15.195:6837/2245799531] up:boot 2024-04-25T00:13:44.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:44 smithi170 ceph-mon[32807]: fsmap 7 up:standby 2024-04-25T00:13:44.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:44 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:13:44.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:44 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3166761035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:44.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:44 smithi107 ceph-mon[29153]: Deploying daemon mds.h on smithi170 2024-04-25T00:13:44.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:44 smithi107 ceph-mon[29153]: mds.? [v2:172.21.15.195:6836/2245799531,v1:172.21.15.195:6837/2245799531] up:boot 2024-04-25T00:13:44.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:44 smithi107 ceph-mon[29153]: fsmap 7 up:standby 2024-04-25T00:13:44.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:44 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:13:44.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:44 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3166761035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:45.764 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:45 smithi107 ceph-mon[29153]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:45.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:45 smithi195 ceph-mon[32833]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:45.875 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:45 smithi170 ceph-mon[32807]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:45.875 INFO:journalctl@ceph.mds.h.smithi170.stdout:Apr 25 00:13:45 smithi170 systemd[1]: Starting Ceph mds.h for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:13:46.013 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:46.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.114+0000 7fbfb78d4640 1 -- 172.21.15.107:0/2433342789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb0102990 msgr2=0x7fbfb0102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:46.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.114+0000 7fbfb78d4640 1 --2- 172.21.15.107:0/2433342789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb0102990 0x7fbfb0102e10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009900 tx=0x7fbfa002a410 comp rx=0 tx=0).stop 2024-04-25T00:13:46.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.114+0000 7fbfb78d4640 1 -- 172.21.15.107:0/2433342789 shutdown_connections 2024-04-25T00:13:46.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.114+0000 7fbfb78d4640 1 --2- 172.21.15.107:0/2433342789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfb0103350 0x7fbfb0109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:46.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.114+0000 7fbfb78d4640 1 --2- 172.21.15.107:0/2433342789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb0102990 0x7fbfb0102e10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:46.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.114+0000 7fbfb78d4640 1 --2- 172.21.15.107:0/2433342789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfb01017a0 0x7fbfb0101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:46.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.114+0000 7fbfb78d4640 1 -- 172.21.15.107:0/2433342789 >> 172.21.15.107:0/2433342789 conn(0x7fbfb00fccf0 msgr2=0x7fbfb00ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:46.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.114+0000 7fbfb78d4640 1 -- 172.21.15.107:0/2433342789 shutdown_connections 2024-04-25T00:13:46.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.114+0000 7fbfb78d4640 1 -- 172.21.15.107:0/2433342789 wait complete. 2024-04-25T00:13:46.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.115+0000 7fbfb78d4640 1 Processor -- start 2024-04-25T00:13:46.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.115+0000 7fbfb78d4640 1 -- start start 2024-04-25T00:13:46.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.115+0000 7fbfb78d4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb01017a0 0x7fbfb0197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:46.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.115+0000 7fbfb78d4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfb0102990 0x7fbfb0197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:46.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.115+0000 7fbfb78d4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfb0103350 0x7fbfb019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:46.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.115+0000 7fbfb78d4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb010d420 con 0x7fbfb01017a0 2024-04-25T00:13:46.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.115+0000 7fbfb78d4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb010d2a0 con 0x7fbfb0103350 2024-04-25T00:13:46.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.115+0000 7fbfb78d4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb010d5a0 con 0x7fbfb0102990 2024-04-25T00:13:46.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.115+0000 7fbfb5670640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb01017a0 0x7fbfb0197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:46.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.115+0000 7fbfb5670640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb01017a0 0x7fbfb0197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53192/0 (socket says 172.21.15.107:53192) 2024-04-25T00:13:46.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.115+0000 7fbfb5670640 1 -- 172.21.15.107:0/3349374042 learned_addr learned my addr 172.21.15.107:0/3349374042 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:46.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.115+0000 7fbfb5e71640 1 --2- 172.21.15.107:0/3349374042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfb0103350 0x7fbfb019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:46.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.115+0000 7fbfb4e6f640 1 --2- 172.21.15.107:0/3349374042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfb0102990 0x7fbfb0197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:46.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.115+0000 7fbfb5670640 1 -- 172.21.15.107:0/3349374042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfb0102990 msgr2=0x7fbfb0197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:46.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.115+0000 7fbfb5670640 1 --2- 172.21.15.107:0/3349374042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfb0102990 0x7fbfb0197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:46.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.115+0000 7fbfb5670640 1 -- 172.21.15.107:0/3349374042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfb0103350 msgr2=0x7fbfb019f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:46.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.115+0000 7fbfb5670640 1 --2- 172.21.15.107:0/3349374042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfb0103350 0x7fbfb019f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:46.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.115+0000 7fbfb5670640 1 -- 172.21.15.107:0/3349374042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb019f6b0 con 0x7fbfb01017a0 2024-04-25T00:13:46.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.115+0000 7fbfb5670640 1 --2- 172.21.15.107:0/3349374042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb01017a0 0x7fbfb0197a90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00c970 tx=0x7fbfac00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:46.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.116+0000 7fbfa67fc640 1 -- 172.21.15.107:0/3349374042 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac004220 con 0x7fbfb01017a0 2024-04-25T00:13:46.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.116+0000 7fbfa67fc640 1 -- 172.21.15.107:0/3349374042 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfac0043c0 con 0x7fbfb01017a0 2024-04-25T00:13:46.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.116+0000 7fbfa67fc640 1 -- 172.21.15.107:0/3349374042 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac004a00 con 0x7fbfb01017a0 2024-04-25T00:13:46.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.116+0000 7fbfb78d4640 1 -- 172.21.15.107:0/3349374042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb019f9a0 con 0x7fbfb01017a0 2024-04-25T00:13:46.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.116+0000 7fbfb78d4640 1 -- 172.21.15.107:0/3349374042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb019fe70 con 0x7fbfb01017a0 2024-04-25T00:13:46.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.117+0000 7fbfb78d4640 1 -- 172.21.15.107:0/3349374042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfb0108b10 con 0x7fbfb01017a0 2024-04-25T00:13:46.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.118+0000 7fbfa67fc640 1 -- 172.21.15.107:0/3349374042 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fbfac0079b0 con 0x7fbfb01017a0 2024-04-25T00:13:46.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.118+0000 7fbfa67fc640 1 --2- 172.21.15.107:0/3349374042 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fbf9c0a2c10 0x7fbf9c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:46.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.119+0000 7fbfa67fc640 1 -- 172.21.15.107:0/3349374042 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fbfac0cec00 con 0x7fbfb01017a0 2024-04-25T00:13:46.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.119+0000 7fbfb4e6f640 1 --2- 172.21.15.107:0/3349374042 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fbf9c0a2c10 0x7fbf9c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:46.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.119+0000 7fbfb4e6f640 1 --2- 172.21.15.107:0/3349374042 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fbf9c0a2c10 0x7fbf9c0a50d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbfa00314e0 tx=0x7fbfa0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:46.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.122+0000 7fbfa67fc640 1 -- 172.21.15.107:0/3349374042 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbfac014030 con 0x7fbfb01017a0 2024-04-25T00:13:46.145 INFO:journalctl@ceph.mds.h.smithi170.stdout:Apr 25 00:13:45 smithi170 podman[62331]: 2024-04-25 00:13:45.778306644 +0000 UTC m=+0.017987172 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:13:46.145 INFO:journalctl@ceph.mds.h.smithi170.stdout:Apr 25 00:13:45 smithi170 podman[62331]: 2024-04-25 00:13:45.909189346 +0000 UTC m=+0.148869892 container create bf9945b981a9839a10bd91042134e6734c0dff8fb3a097bb68b35a0099f75b7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-h, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:13:46.145 INFO:journalctl@ceph.mds.h.smithi170.stdout:Apr 25 00:13:46 smithi170 podman[62331]: 2024-04-25 00:13:46.145404756 +0000 UTC m=+0.385085264 container init bf9945b981a9839a10bd91042134e6734c0dff8fb3a097bb68b35a0099f75b7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T00:13:46.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.365+0000 7fbfb78d4640 1 -- 172.21.15.107:0/3349374042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbfb002d170 con 0x7fbfb01017a0 2024-04-25T00:13:46.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.366+0000 7fbfa67fc640 1 -- 172.21.15.107:0/3349374042 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6361 (secure 0 0 0) 0x7fbfb002d170 con 0x7fbfb01017a0 2024-04-25T00:13:46.367 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:13:46.367 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T00:13:46.368 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 8 2024-04-25T00:13:46.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.368+0000 7fbfb78d4640 1 -- 172.21.15.107:0/3349374042 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fbf9c0a2c10 msgr2=0x7fbf9c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:46.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.368+0000 7fbfb78d4640 1 --2- 172.21.15.107:0/3349374042 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fbf9c0a2c10 0x7fbf9c0a50d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbfa00314e0 tx=0x7fbfa0000bc0 comp rx=0 tx=0).stop 2024-04-25T00:13:46.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.368+0000 7fbfb78d4640 1 -- 172.21.15.107:0/3349374042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb01017a0 msgr2=0x7fbfb0197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:46.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.368+0000 7fbfb78d4640 1 --2- 172.21.15.107:0/3349374042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb01017a0 0x7fbfb0197a90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbfac00c970 tx=0x7fbfac00ce30 comp rx=0 tx=0).stop 2024-04-25T00:13:46.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.368+0000 7fbfb78d4640 1 -- 172.21.15.107:0/3349374042 shutdown_connections 2024-04-25T00:13:46.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.368+0000 7fbfb78d4640 1 --2- 172.21.15.107:0/3349374042 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fbf9c0a2c10 0x7fbf9c0a50d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:46.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.368+0000 7fbfb78d4640 1 --2- 172.21.15.107:0/3349374042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfb0103350 0x7fbfb019f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:46.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.368+0000 7fbfb78d4640 1 --2- 172.21.15.107:0/3349374042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbfb0102990 0x7fbfb0197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:46.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.368+0000 7fbfb78d4640 1 --2- 172.21.15.107:0/3349374042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfb01017a0 0x7fbfb0197a90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:46.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.368+0000 7fbfb78d4640 1 -- 172.21.15.107:0/3349374042 >> 172.21.15.107:0/3349374042 conn(0x7fbfb00fccf0 msgr2=0x7fbfb0105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:46.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.368+0000 7fbfb78d4640 1 -- 172.21.15.107:0/3349374042 shutdown_connections 2024-04-25T00:13:46.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:46.368+0000 7fbfb78d4640 1 -- 172.21.15.107:0/3349374042 wait complete. 2024-04-25T00:13:46.435 INFO:journalctl@ceph.mds.h.smithi170.stdout:Apr 25 00:13:46 smithi170 podman[62331]: 2024-04-25 00:13:46.148851858 +0000 UTC m=+0.388532363 container start bf9945b981a9839a10bd91042134e6734c0dff8fb3a097bb68b35a0099f75b7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-25T00:13:46.435 INFO:journalctl@ceph.mds.h.smithi170.stdout:Apr 25 00:13:46 smithi170 ceph-mds[62350]: -- 172.21.15.170:0/927642741 <== mon.2 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5641d2677a40 con 0x5641d264ac00 2024-04-25T00:13:46.435 INFO:journalctl@ceph.mds.h.smithi170.stdout:Apr 25 00:13:46 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-h[62346]: starting mds.h at 2024-04-25T00:13:46.435 INFO:journalctl@ceph.mds.h.smithi170.stdout:Apr 25 00:13:46 smithi170 bash[62331]: bf9945b981a9839a10bd91042134e6734c0dff8fb3a097bb68b35a0099f75b7d 2024-04-25T00:13:46.435 INFO:journalctl@ceph.mds.h.smithi170.stdout:Apr 25 00:13:46 smithi170 systemd[1]: Started Ceph mds.h for 79602f58-0297-11ef-bc93-c7b262605968. 2024-04-25T00:13:47.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:47.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:47.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:47.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:47.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:47 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3349374042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:47.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:47.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:47.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:47 smithi195 ceph-mon[32833]: Deploying daemon mds.i on smithi195 2024-04-25T00:13:47.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:47 smithi195 ceph-mon[32833]: mds.? [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] up:boot 2024-04-25T00:13:47.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:47 smithi195 ceph-mon[32833]: fsmap 8 up:standby 2024-04-25T00:13:47.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:47 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:13:47.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:47 smithi195 ceph-mon[32833]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:47.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:47.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:47.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:47.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:47.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:47 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3349374042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:47.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:47.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:47.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:47 smithi170 ceph-mon[32807]: Deploying daemon mds.i on smithi195 2024-04-25T00:13:47.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:47 smithi170 ceph-mon[32807]: mds.? [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] up:boot 2024-04-25T00:13:47.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:47 smithi170 ceph-mon[32807]: fsmap 8 up:standby 2024-04-25T00:13:47.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:47 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:13:47.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:47 smithi170 ceph-mon[32807]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:47.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:47.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:47.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:47.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:47.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:47 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3349374042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:47.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:47.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:47.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:47 smithi107 ceph-mon[29153]: Deploying daemon mds.i on smithi195 2024-04-25T00:13:47.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:47 smithi107 ceph-mon[29153]: mds.? [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] up:boot 2024-04-25T00:13:47.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:47 smithi107 ceph-mon[29153]: fsmap 8 up:standby 2024-04-25T00:13:47.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:47 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:13:47.735 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:47 smithi107 ceph-mon[29153]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:48.381 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:48.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.492+0000 7fa6bdfe3640 1 -- 172.21.15.107:0/3261810041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b8103360 msgr2=0x7fa6b8109bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:48.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.492+0000 7fa6bdfe3640 1 --2- 172.21.15.107:0/3261810041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b8103360 0x7fa6b8109bf0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0009ca0 tx=0x7fa6b002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:13:48.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.492+0000 7fa6bdfe3640 1 -- 172.21.15.107:0/3261810041 shutdown_connections 2024-04-25T00:13:48.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.492+0000 7fa6bdfe3640 1 --2- 172.21.15.107:0/3261810041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b8103360 0x7fa6b8109bf0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:48.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.492+0000 7fa6bdfe3640 1 --2- 172.21.15.107:0/3261810041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6b81029a0 0x7fa6b8102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:48.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.492+0000 7fa6bdfe3640 1 --2- 172.21.15.107:0/3261810041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6b81017b0 0x7fa6b8101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:48.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.492+0000 7fa6bdfe3640 1 -- 172.21.15.107:0/3261810041 >> 172.21.15.107:0/3261810041 conn(0x7fa6b80fcd40 msgr2=0x7fa6b80ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:48.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.492+0000 7fa6bdfe3640 1 -- 172.21.15.107:0/3261810041 shutdown_connections 2024-04-25T00:13:48.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.492+0000 7fa6bdfe3640 1 -- 172.21.15.107:0/3261810041 wait complete. 2024-04-25T00:13:48.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.493+0000 7fa6bdfe3640 1 Processor -- start 2024-04-25T00:13:48.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.493+0000 7fa6bdfe3640 1 -- start start 2024-04-25T00:13:48.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.493+0000 7fa6bdfe3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6b81017b0 0x7fa6b8197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:48.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.493+0000 7fa6bdfe3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6b81029a0 0x7fa6b8197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:48.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.493+0000 7fa6bdfe3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b8103360 0x7fa6b819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:48.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.493+0000 7fa6bdfe3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b810d430 con 0x7fa6b8103360 2024-04-25T00:13:48.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.493+0000 7fa6bdfe3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b810d2b0 con 0x7fa6b81029a0 2024-04-25T00:13:48.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.493+0000 7fa6bdfe3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b810d5b0 con 0x7fa6b81017b0 2024-04-25T00:13:48.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.493+0000 7fa6b7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b8103360 0x7fa6b819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:48.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.493+0000 7fa6b7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b8103360 0x7fa6b819f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53212/0 (socket says 172.21.15.107:53212) 2024-04-25T00:13:48.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.493+0000 7fa6b7fff640 1 -- 172.21.15.107:0/838216629 learned_addr learned my addr 172.21.15.107:0/838216629 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:48.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.493+0000 7fa6b77fe640 1 --2- 172.21.15.107:0/838216629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6b81017b0 0x7fa6b8197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:48.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.493+0000 7fa6b6ffd640 1 --2- 172.21.15.107:0/838216629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6b81029a0 0x7fa6b8197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:48.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.494+0000 7fa6b7fff640 1 -- 172.21.15.107:0/838216629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6b81017b0 msgr2=0x7fa6b8197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:48.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.494+0000 7fa6b7fff640 1 --2- 172.21.15.107:0/838216629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6b81017b0 0x7fa6b8197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:48.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.494+0000 7fa6b7fff640 1 -- 172.21.15.107:0/838216629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6b81029a0 msgr2=0x7fa6b8197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:48.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.494+0000 7fa6b7fff640 1 --2- 172.21.15.107:0/838216629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6b81029a0 0x7fa6b8197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:48.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.494+0000 7fa6b7fff640 1 -- 172.21.15.107:0/838216629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b819f6c0 con 0x7fa6b8103360 2024-04-25T00:13:48.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.494+0000 7fa6b77fe640 1 --2- 172.21.15.107:0/838216629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6b81017b0 0x7fa6b8197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:13:48.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.494+0000 7fa6b7fff640 1 --2- 172.21.15.107:0/838216629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b8103360 0x7fa6b819f050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0031ae0 tx=0x7fa6b0031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:48.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.494+0000 7fa6b4ff9640 1 -- 172.21.15.107:0/838216629 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0003b70 con 0x7fa6b8103360 2024-04-25T00:13:48.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.494+0000 7fa6b4ff9640 1 -- 172.21.15.107:0/838216629 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6b0003d10 con 0x7fa6b8103360 2024-04-25T00:13:48.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.494+0000 7fa6b4ff9640 1 -- 172.21.15.107:0/838216629 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b002fc40 con 0x7fa6b8103360 2024-04-25T00:13:48.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.494+0000 7fa6bdfe3640 1 -- 172.21.15.107:0/838216629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b819f950 con 0x7fa6b8103360 2024-04-25T00:13:48.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.494+0000 7fa6bdfe3640 1 -- 172.21.15.107:0/838216629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b819fea0 con 0x7fa6b8103360 2024-04-25T00:13:48.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.495+0000 7fa6bdfe3640 1 -- 172.21.15.107:0/838216629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b8108ba0 con 0x7fa6b8103360 2024-04-25T00:13:48.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.496+0000 7fa6b4ff9640 1 -- 172.21.15.107:0/838216629 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa6b0004050 con 0x7fa6b8103360 2024-04-25T00:13:48.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.497+0000 7fa6b4ff9640 1 --2- 172.21.15.107:0/838216629 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa6a00a2b90 0x7fa6a00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:48.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.497+0000 7fa6b77fe640 1 --2- 172.21.15.107:0/838216629 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa6a00a2b90 0x7fa6a00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:48.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.497+0000 7fa6b4ff9640 1 -- 172.21.15.107:0/838216629 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa6b00efcd0 con 0x7fa6b8103360 2024-04-25T00:13:48.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.497+0000 7fa6b77fe640 1 --2- 172.21.15.107:0/838216629 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa6a00a2b90 0x7fa6a00a5050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8102800 tx=0x7fa6ac005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:48.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.501+0000 7fa6b4ff9640 1 -- 172.21.15.107:0/838216629 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa6b00bd1f0 con 0x7fa6b8103360 2024-04-25T00:13:48.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.716+0000 7fa6bdfe3640 1 -- 172.21.15.107:0/838216629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6b802d390 con 0x7fa6b8103360 2024-04-25T00:13:48.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.717+0000 7fa6b4ff9640 1 -- 172.21.15.107:0/838216629 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7189 (secure 0 0 0) 0x7fa6b00bde20 con 0x7fa6b8103360 2024-04-25T00:13:48.717 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:13:48.718 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T00:13:48.718 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 9 2024-04-25T00:13:48.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.719+0000 7fa6bdfe3640 1 -- 172.21.15.107:0/838216629 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa6a00a2b90 msgr2=0x7fa6a00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:48.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.719+0000 7fa6bdfe3640 1 --2- 172.21.15.107:0/838216629 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa6a00a2b90 0x7fa6a00a5050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa6b8102800 tx=0x7fa6ac005f90 comp rx=0 tx=0).stop 2024-04-25T00:13:48.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.719+0000 7fa6bdfe3640 1 -- 172.21.15.107:0/838216629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b8103360 msgr2=0x7fa6b819f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:48.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.719+0000 7fa6bdfe3640 1 --2- 172.21.15.107:0/838216629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b8103360 0x7fa6b819f050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0031ae0 tx=0x7fa6b0031ee0 comp rx=0 tx=0).stop 2024-04-25T00:13:48.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.719+0000 7fa6bdfe3640 1 -- 172.21.15.107:0/838216629 shutdown_connections 2024-04-25T00:13:48.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.719+0000 7fa6bdfe3640 1 --2- 172.21.15.107:0/838216629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b8103360 0x7fa6b819f050 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:48.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.719+0000 7fa6bdfe3640 1 --2- 172.21.15.107:0/838216629 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa6a00a2b90 0x7fa6a00a5050 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:48.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.719+0000 7fa6bdfe3640 1 --2- 172.21.15.107:0/838216629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6b81029a0 0x7fa6b8197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:48.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.719+0000 7fa6bdfe3640 1 --2- 172.21.15.107:0/838216629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa6b81017b0 0x7fa6b8197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:48.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.719+0000 7fa6bdfe3640 1 -- 172.21.15.107:0/838216629 >> 172.21.15.107:0/838216629 conn(0x7fa6b80fcd40 msgr2=0x7fa6b8105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:48.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.719+0000 7fa6bdfe3640 1 -- 172.21.15.107:0/838216629 shutdown_connections 2024-04-25T00:13:48.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:48.719+0000 7fa6bdfe3640 1 -- 172.21.15.107:0/838216629 wait complete. 2024-04-25T00:13:48.769 INFO:journalctl@ceph.mds.i.smithi195.stdout:Apr 25 00:13:48 smithi195 podman[64089]: 2024-04-25 00:13:48.568030978 +0000 UTC m=+0.018246283 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:13:48.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:48 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/838216629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:49.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:48 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/838216629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:49.089 INFO:journalctl@ceph.mds.i.smithi195.stdout:Apr 25 00:13:48 smithi195 podman[64089]: 2024-04-25 00:13:48.895140986 +0000 UTC m=+0.345356274 container init d7c9c88192ac0cac3d749e83156dc942abd0eb1b47c6c5428d063ea06dd6904c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-i, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-25T00:13:49.089 INFO:journalctl@ceph.mds.i.smithi195.stdout:Apr 25 00:13:48 smithi195 podman[64089]: 2024-04-25 00:13:48.897639745 +0000 UTC m=+0.347855033 container start d7c9c88192ac0cac3d749e83156dc942abd0eb1b47c6c5428d063ea06dd6904c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T00:13:49.089 INFO:journalctl@ceph.mds.i.smithi195.stdout:Apr 25 00:13:48 smithi195 ceph-mds[64108]: -- 172.21.15.195:0/32465789 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5570a33d9a40 con 0x5570a33ad000 2024-04-25T00:13:49.090 INFO:journalctl@ceph.mds.i.smithi195.stdout:Apr 25 00:13:48 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-i[64104]: starting mds.i at 2024-04-25T00:13:49.090 INFO:journalctl@ceph.mds.i.smithi195.stdout:Apr 25 00:13:48 smithi195 bash[64089]: d7c9c88192ac0cac3d749e83156dc942abd0eb1b47c6c5428d063ea06dd6904c 2024-04-25T00:13:49.090 INFO:journalctl@ceph.mds.i.smithi195.stdout:Apr 25 00:13:48 smithi195 systemd[1]: Started Ceph mds.i for 79602f58-0297-11ef-bc93-c7b262605968. 2024-04-25T00:13:49.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:48 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/838216629' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:50.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:50 smithi107 ceph-mon[29153]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:50.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:50.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:50.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:50.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:50.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:50.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:50.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:50 smithi107 ceph-mon[29153]: Deploying daemon mds.g on smithi107 2024-04-25T00:13:50.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:50 smithi107 ceph-mon[29153]: mds.? [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] up:boot 2024-04-25T00:13:50.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:50 smithi107 ceph-mon[29153]: fsmap 9 up:standby 2024-04-25T00:13:50.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:50 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:13:50.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:50 smithi195 ceph-mon[32833]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:50.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:50.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:50.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:50.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:50.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:50.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:50.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:50 smithi195 ceph-mon[32833]: Deploying daemon mds.g on smithi107 2024-04-25T00:13:50.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:50 smithi195 ceph-mon[32833]: mds.? [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] up:boot 2024-04-25T00:13:50.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:50 smithi195 ceph-mon[32833]: fsmap 9 up:standby 2024-04-25T00:13:50.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:50 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:13:50.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:50 smithi170 ceph-mon[32807]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:50.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:50 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:50.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:50 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:50.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:50 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:50.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:50 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:50.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:50 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:50.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:50 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:50.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:50 smithi170 ceph-mon[32807]: Deploying daemon mds.g on smithi107 2024-04-25T00:13:50.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:50 smithi170 ceph-mon[32807]: mds.? [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] up:boot 2024-04-25T00:13:50.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:50 smithi170 ceph-mon[32807]: fsmap 9 up:standby 2024-04-25T00:13:50.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:50 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:13:50.732 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:50.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.847+0000 7f0998c8c640 1 -- 172.21.15.107:0/181651703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0994107460 msgr2=0x7f09941098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:50.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.847+0000 7f0998c8c640 1 --2- 172.21.15.107:0/181651703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0994107460 0x7f09941098b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f098c009ca0 tx=0x7f098c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:13:50.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.847+0000 7f0998c8c640 1 -- 172.21.15.107:0/181651703 shutdown_connections 2024-04-25T00:13:50.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.847+0000 7f0998c8c640 1 --2- 172.21.15.107:0/181651703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0994107460 0x7f09941098b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:50.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.847+0000 7f0998c8c640 1 --2- 172.21.15.107:0/181651703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0994104b30 0x7f0994106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:50.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.847+0000 7f0998c8c640 1 --2- 172.21.15.107:0/181651703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0994102200 0x7f09941045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:50.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.847+0000 7f0998c8c640 1 -- 172.21.15.107:0/181651703 >> 172.21.15.107:0/181651703 conn(0x7f09940fbbf0 msgr2=0x7f09940fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:50.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.847+0000 7f0998c8c640 1 -- 172.21.15.107:0/181651703 shutdown_connections 2024-04-25T00:13:50.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.847+0000 7f0998c8c640 1 -- 172.21.15.107:0/181651703 wait complete. 2024-04-25T00:13:50.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.848+0000 7f0998c8c640 1 Processor -- start 2024-04-25T00:13:50.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.848+0000 7f0998c8c640 1 -- start start 2024-04-25T00:13:50.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.848+0000 7f0998c8c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0994102200 0x7f0994197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:50.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.848+0000 7f0998c8c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0994104b30 0x7f0994197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:50.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.848+0000 7f0998c8c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0994107460 0x7f099419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:50.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.848+0000 7f0998c8c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f099410d420 con 0x7f0994104b30 2024-04-25T00:13:50.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.848+0000 7f0998c8c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f099410d2a0 con 0x7f0994107460 2024-04-25T00:13:50.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.848+0000 7f0998c8c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f099410d5a0 con 0x7f0994102200 2024-04-25T00:13:50.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.848+0000 7f098bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0994104b30 0x7f0994197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:50.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.848+0000 7f098bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0994104b30 0x7f0994197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36154/0 (socket says 172.21.15.107:36154) 2024-04-25T00:13:50.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.848+0000 7f098bfff640 1 -- 172.21.15.107:0/1754464433 learned_addr learned my addr 172.21.15.107:0/1754464433 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:50.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.848+0000 7f099259c640 1 --2- 172.21.15.107:0/1754464433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0994102200 0x7f0994197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:50.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.848+0000 7f0992d9d640 1 --2- 172.21.15.107:0/1754464433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0994107460 0x7f099419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:50.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.848+0000 7f098bfff640 1 -- 172.21.15.107:0/1754464433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0994102200 msgr2=0x7f0994197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:50.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.848+0000 7f098bfff640 1 --2- 172.21.15.107:0/1754464433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0994102200 0x7f0994197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:50.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.848+0000 7f098bfff640 1 -- 172.21.15.107:0/1754464433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0994107460 msgr2=0x7f099419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:50.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.848+0000 7f098bfff640 1 --2- 172.21.15.107:0/1754464433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0994107460 0x7f099419eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:50.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.849+0000 7f098bfff640 1 -- 172.21.15.107:0/1754464433 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f099419f660 con 0x7f0994104b30 2024-04-25T00:13:50.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.849+0000 7f099259c640 1 --2- 172.21.15.107:0/1754464433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0994102200 0x7f0994197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:50.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.849+0000 7f098bfff640 1 --2- 172.21.15.107:0/1754464433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0994104b30 0x7f0994197f80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f097c00e9f0 tx=0x7f097c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:50.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.849+0000 7f098b7fe640 1 -- 172.21.15.107:0/1754464433 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c00ccc0 con 0x7f0994104b30 2024-04-25T00:13:50.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.849+0000 7f0998c8c640 1 -- 172.21.15.107:0/1754464433 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f099419f950 con 0x7f0994104b30 2024-04-25T00:13:50.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.849+0000 7f098b7fe640 1 -- 172.21.15.107:0/1754464433 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f097c005500 con 0x7f0994104b30 2024-04-25T00:13:50.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.849+0000 7f098b7fe640 1 -- 172.21.15.107:0/1754464433 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c010690 con 0x7f0994104b30 2024-04-25T00:13:50.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.849+0000 7f0998c8c640 1 -- 172.21.15.107:0/1754464433 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f099419fea0 con 0x7f0994104b30 2024-04-25T00:13:50.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.850+0000 7f0998c8c640 1 -- 172.21.15.107:0/1754464433 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09940fd6e0 con 0x7f0994104b30 2024-04-25T00:13:50.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.851+0000 7f098b7fe640 1 -- 172.21.15.107:0/1754464433 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f097c0046c0 con 0x7f0994104b30 2024-04-25T00:13:50.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.852+0000 7f098b7fe640 1 --2- 172.21.15.107:0/1754464433 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f09800a2b90 0x7f09800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:50.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.852+0000 7f099259c640 1 --2- 172.21.15.107:0/1754464433 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f09800a2b90 0x7f09800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:50.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.852+0000 7f098b7fe640 1 -- 172.21.15.107:0/1754464433 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f097c0cf080 con 0x7f0994104b30 2024-04-25T00:13:50.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.852+0000 7f099259c640 1 --2- 172.21.15.107:0/1754464433 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f09800a2b90 0x7f09800a5050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0984009c20 tx=0x7f0984005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:50.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:50.856+0000 7f098b7fe640 1 -- 172.21.15.107:0/1754464433 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f097c09c480 con 0x7f0994104b30 2024-04-25T00:13:51.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:51.101+0000 7f0998c8c640 1 -- 172.21.15.107:0/1754464433 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f099402d330 con 0x7f0994104b30 2024-04-25T00:13:51.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:51.102+0000 7f098b7fe640 1 -- 172.21.15.107:0/1754464433 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8019 (secure 0 0 0) 0x7f097c09bd80 con 0x7f0994104b30 2024-04-25T00:13:51.103 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:13:51.103 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T00:13:51.103 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 10 2024-04-25T00:13:51.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:51.104+0000 7f0998c8c640 1 -- 172.21.15.107:0/1754464433 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f09800a2b90 msgr2=0x7f09800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:51.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:51.104+0000 7f0998c8c640 1 --2- 172.21.15.107:0/1754464433 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f09800a2b90 0x7f09800a5050 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0984009c20 tx=0x7f0984005f90 comp rx=0 tx=0).stop 2024-04-25T00:13:51.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:51.104+0000 7f0998c8c640 1 -- 172.21.15.107:0/1754464433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0994104b30 msgr2=0x7f0994197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:51.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:51.104+0000 7f0998c8c640 1 --2- 172.21.15.107:0/1754464433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0994104b30 0x7f0994197f80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f097c00e9f0 tx=0x7f097c00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:13:51.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:51.104+0000 7f0998c8c640 1 -- 172.21.15.107:0/1754464433 shutdown_connections 2024-04-25T00:13:51.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:51.104+0000 7f0998c8c640 1 --2- 172.21.15.107:0/1754464433 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f09800a2b90 0x7f09800a5050 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:51.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:51.104+0000 7f0998c8c640 1 --2- 172.21.15.107:0/1754464433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0994107460 0x7f099419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:51.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:51.105+0000 7f0998c8c640 1 --2- 172.21.15.107:0/1754464433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0994104b30 0x7f0994197f80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:51.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:51.105+0000 7f0998c8c640 1 --2- 172.21.15.107:0/1754464433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0994102200 0x7f0994197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:51.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:51.105+0000 7f0998c8c640 1 -- 172.21.15.107:0/1754464433 >> 172.21.15.107:0/1754464433 conn(0x7f09940fbbf0 msgr2=0x7f0994102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:51.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:51.105+0000 7f0998c8c640 1 -- 172.21.15.107:0/1754464433 shutdown_connections 2024-04-25T00:13:51.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:51.105+0000 7f0998c8c640 1 -- 172.21.15.107:0/1754464433 wait complete. 2024-04-25T00:13:51.439 INFO:journalctl@ceph.mds.g.smithi107.stdout:Apr 25 00:13:51 smithi107 systemd[1]: Starting Ceph mds.g for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:13:51.697 INFO:journalctl@ceph.mds.g.smithi107.stdout:Apr 25 00:13:51 smithi107 podman[67591]: 2024-04-25 00:13:51.342048116 +0000 UTC m=+0.017936881 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:13:51.697 INFO:journalctl@ceph.mds.g.smithi107.stdout:Apr 25 00:13:51 smithi107 podman[67591]: 2024-04-25 00:13:51.443925976 +0000 UTC m=+0.119814777 container create 631b408df0e5cb9cc4ef3baa7c6bd4def536a5325bd09299d2210b52f25ebe1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-g, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424) 2024-04-25T00:13:51.984 INFO:journalctl@ceph.mds.g.smithi107.stdout:Apr 25 00:13:51 smithi107 podman[67591]: 2024-04-25 00:13:51.697183995 +0000 UTC m=+0.373072746 container init 631b408df0e5cb9cc4ef3baa7c6bd4def536a5325bd09299d2210b52f25ebe1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9) 2024-04-25T00:13:51.984 INFO:journalctl@ceph.mds.g.smithi107.stdout:Apr 25 00:13:51 smithi107 podman[67591]: 2024-04-25 00:13:51.699889854 +0000 UTC m=+0.375778606 container start 631b408df0e5cb9cc4ef3baa7c6bd4def536a5325bd09299d2210b52f25ebe1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-g, RELEASE=squid-d2c45d9, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:13:51.984 INFO:journalctl@ceph.mds.g.smithi107.stdout:Apr 25 00:13:51 smithi107 ceph-mds[67611]: -- 172.21.15.107:0/1003830647 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a78f56da40 con 0x55a78f541000 2024-04-25T00:13:51.984 INFO:journalctl@ceph.mds.g.smithi107.stdout:Apr 25 00:13:51 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-g[67607]: starting mds.g at 2024-04-25T00:13:51.984 INFO:journalctl@ceph.mds.g.smithi107.stdout:Apr 25 00:13:51 smithi107 bash[67591]: 631b408df0e5cb9cc4ef3baa7c6bd4def536a5325bd09299d2210b52f25ebe1d 2024-04-25T00:13:51.984 INFO:journalctl@ceph.mds.g.smithi107.stdout:Apr 25 00:13:51 smithi107 systemd[1]: Started Ceph mds.g for 79602f58-0297-11ef-bc93-c7b262605968. 2024-04-25T00:13:52.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:52 smithi195 ceph-mon[32833]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:52.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:52 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1754464433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:52.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:52 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:52.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:52 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:52.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:52 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:52.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:52 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:52.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:52 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:52.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:52 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:52.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:52 smithi170 ceph-mon[32807]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:52.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:52 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1754464433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:52.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:52 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:52.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:52 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:52.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:52 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:52.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:52 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:52.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:52 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:52.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:52 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:52.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:52 smithi107 ceph-mon[29153]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:52.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:52 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1754464433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:52.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:52 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:52.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:52 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:52.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:52 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:52.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:52 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:52.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:52 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:52.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:52 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:53.117 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:53.139 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:53 smithi107 ceph-mon[29153]: Deploying daemon mds.a on smithi107 2024-04-25T00:13:53.139 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:53 smithi107 ceph-mon[29153]: mds.? [v2:172.21.15.107:6838/2028240372,v1:172.21.15.107:6839/2028240372] up:boot 2024-04-25T00:13:53.139 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:53 smithi107 ceph-mon[29153]: fsmap 10 up:standby 2024-04-25T00:13:53.139 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:53 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T00:13:53.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.222+0000 7fcf7a2a7640 1 -- 172.21.15.107:0/602752228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf74103340 msgr2=0x7fcf74109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:53.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.222+0000 7fcf7a2a7640 1 --2- 172.21.15.107:0/602752228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf74103340 0x7fcf74109bd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009ca0 tx=0x7fcf6c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.222+0000 7fcf7a2a7640 1 -- 172.21.15.107:0/602752228 shutdown_connections 2024-04-25T00:13:53.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.222+0000 7fcf7a2a7640 1 --2- 172.21.15.107:0/602752228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf74103340 0x7fcf74109bd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.222+0000 7fcf7a2a7640 1 --2- 172.21.15.107:0/602752228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf74102980 0x7fcf74102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.222+0000 7fcf7a2a7640 1 --2- 172.21.15.107:0/602752228 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf74101790 0x7fcf74101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.222+0000 7fcf7a2a7640 1 -- 172.21.15.107:0/602752228 >> 172.21.15.107:0/602752228 conn(0x7fcf740fcd40 msgr2=0x7fcf740ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:53.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.222+0000 7fcf7a2a7640 1 -- 172.21.15.107:0/602752228 shutdown_connections 2024-04-25T00:13:53.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.222+0000 7fcf7a2a7640 1 -- 172.21.15.107:0/602752228 wait complete. 2024-04-25T00:13:53.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.222+0000 7fcf7a2a7640 1 Processor -- start 2024-04-25T00:13:53.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.223+0000 7fcf7a2a7640 1 -- start start 2024-04-25T00:13:53.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.223+0000 7fcf7a2a7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf74101790 0x7fcf74197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:53.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.223+0000 7fcf7a2a7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf74102980 0x7fcf74197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:53.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.223+0000 7fcf7a2a7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf74103340 0x7fcf7419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:53.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.223+0000 7fcf7a2a7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7410d410 con 0x7fcf74102980 2024-04-25T00:13:53.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.223+0000 7fcf7a2a7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7410d290 con 0x7fcf74101790 2024-04-25T00:13:53.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.223+0000 7fcf7a2a7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7410d590 con 0x7fcf74103340 2024-04-25T00:13:53.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.223+0000 7fcf737fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf74102980 0x7fcf74197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:53.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.223+0000 7fcf737fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf74102980 0x7fcf74197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36192/0 (socket says 172.21.15.107:36192) 2024-04-25T00:13:53.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.223+0000 7fcf737fe640 1 -- 172.21.15.107:0/3572741740 learned_addr learned my addr 172.21.15.107:0/3572741740 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:53.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.223+0000 7fcf73fff640 1 --2- 172.21.15.107:0/3572741740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf74101790 0x7fcf74197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:53.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.223+0000 7fcf78844640 1 --2- 172.21.15.107:0/3572741740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf74103340 0x7fcf7419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:53.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.223+0000 7fcf737fe640 1 -- 172.21.15.107:0/3572741740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf74103340 msgr2=0x7fcf7419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:53.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.223+0000 7fcf737fe640 1 --2- 172.21.15.107:0/3572741740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf74103340 0x7fcf7419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.223+0000 7fcf737fe640 1 -- 172.21.15.107:0/3572741740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf74101790 msgr2=0x7fcf74197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:53.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.223+0000 7fcf737fe640 1 --2- 172.21.15.107:0/3572741740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf74101790 0x7fcf74197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.223+0000 7fcf737fe640 1 -- 172.21.15.107:0/3572741740 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7419f680 con 0x7fcf74102980 2024-04-25T00:13:53.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.223+0000 7fcf737fe640 1 --2- 172.21.15.107:0/3572741740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf74102980 0x7fcf74197fa0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400d950 tx=0x7fcf6400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:53.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.223+0000 7fcf717fa640 1 -- 172.21.15.107:0/3572741740 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf64017070 con 0x7fcf74102980 2024-04-25T00:13:53.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.223+0000 7fcf7a2a7640 1 -- 172.21.15.107:0/3572741740 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf7419f970 con 0x7fcf74102980 2024-04-25T00:13:53.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.224+0000 7fcf717fa640 1 -- 172.21.15.107:0/3572741740 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf64009c30 con 0x7fcf74102980 2024-04-25T00:13:53.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.224+0000 7fcf717fa640 1 -- 172.21.15.107:0/3572741740 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6400f6b0 con 0x7fcf74102980 2024-04-25T00:13:53.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.224+0000 7fcf7a2a7640 1 -- 172.21.15.107:0/3572741740 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf7419ff10 con 0x7fcf74102980 2024-04-25T00:13:53.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.224+0000 7fcf7a2a7640 1 -- 172.21.15.107:0/3572741740 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf74108ab0 con 0x7fcf74102980 2024-04-25T00:13:53.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.225+0000 7fcf717fa640 1 -- 172.21.15.107:0/3572741740 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fcf640109d0 con 0x7fcf74102980 2024-04-25T00:13:53.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.226+0000 7fcf717fa640 1 --2- 172.21.15.107:0/3572741740 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcf5c0a2b90 0x7fcf5c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:53.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.226+0000 7fcf73fff640 1 --2- 172.21.15.107:0/3572741740 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcf5c0a2b90 0x7fcf5c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:53.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.226+0000 7fcf717fa640 1 -- 172.21.15.107:0/3572741740 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fcf640ce810 con 0x7fcf74102980 2024-04-25T00:13:53.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.226+0000 7fcf73fff640 1 --2- 172.21.15.107:0/3572741740 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcf5c0a2b90 0x7fcf5c0a5050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcf68005fd0 tx=0x7fcf6800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:53.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.231+0000 7fcf717fa640 1 -- 172.21.15.107:0/3572741740 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcf64009de0 con 0x7fcf74102980 2024-04-25T00:13:53.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:53 smithi195 ceph-mon[32833]: Deploying daemon mds.a on smithi107 2024-04-25T00:13:53.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:53 smithi195 ceph-mon[32833]: mds.? [v2:172.21.15.107:6838/2028240372,v1:172.21.15.107:6839/2028240372] up:boot 2024-04-25T00:13:53.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:53 smithi195 ceph-mon[32833]: fsmap 10 up:standby 2024-04-25T00:13:53.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:53 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T00:13:53.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:53 smithi170 ceph-mon[32807]: Deploying daemon mds.a on smithi107 2024-04-25T00:13:53.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:53 smithi170 ceph-mon[32807]: mds.? [v2:172.21.15.107:6838/2028240372,v1:172.21.15.107:6839/2028240372] up:boot 2024-04-25T00:13:53.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:53 smithi170 ceph-mon[32807]: fsmap 10 up:standby 2024-04-25T00:13:53.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:53 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T00:13:53.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.450+0000 7fcf7a2a7640 1 -- 172.21.15.107:0/3572741740 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf7402d320 con 0x7fcf74102980 2024-04-25T00:13:53.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.451+0000 7fcf717fa640 1 -- 172.21.15.107:0/3572741740 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8848 (secure 0 0 0) 0x7fcf6409bc10 con 0x7fcf74102980 2024-04-25T00:13:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:13:53.452 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T00:13:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 11 2024-04-25T00:13:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.453+0000 7fcf7a2a7640 1 -- 172.21.15.107:0/3572741740 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcf5c0a2b90 msgr2=0x7fcf5c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.453+0000 7fcf7a2a7640 1 --2- 172.21.15.107:0/3572741740 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcf5c0a2b90 0x7fcf5c0a5050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcf68005fd0 tx=0x7fcf6800b390 comp rx=0 tx=0).stop 2024-04-25T00:13:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.453+0000 7fcf7a2a7640 1 -- 172.21.15.107:0/3572741740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf74102980 msgr2=0x7fcf74197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.453+0000 7fcf7a2a7640 1 --2- 172.21.15.107:0/3572741740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf74102980 0x7fcf74197fa0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400d950 tx=0x7fcf6400de10 comp rx=0 tx=0).stop 2024-04-25T00:13:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.453+0000 7fcf7a2a7640 1 -- 172.21.15.107:0/3572741740 shutdown_connections 2024-04-25T00:13:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.453+0000 7fcf7a2a7640 1 --2- 172.21.15.107:0/3572741740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf74103340 0x7fcf7419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.453+0000 7fcf7a2a7640 1 --2- 172.21.15.107:0/3572741740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf74102980 0x7fcf74197fa0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.453+0000 7fcf7a2a7640 1 --2- 172.21.15.107:0/3572741740 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcf5c0a2b90 0x7fcf5c0a5050 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.453+0000 7fcf7a2a7640 1 --2- 172.21.15.107:0/3572741740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf74101790 0x7fcf74197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.453+0000 7fcf7a2a7640 1 -- 172.21.15.107:0/3572741740 >> 172.21.15.107:0/3572741740 conn(0x7fcf740fcd40 msgr2=0x7fcf74105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.453+0000 7fcf7a2a7640 1 -- 172.21.15.107:0/3572741740 shutdown_connections 2024-04-25T00:13:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:53.453+0000 7fcf7a2a7640 1 -- 172.21.15.107:0/3572741740 wait complete. 2024-04-25T00:13:54.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:54 smithi107 ceph-mon[29153]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:54.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:54 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3572741740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:54.227 INFO:journalctl@ceph.mds.a.smithi107.stdout:Apr 25 00:13:54 smithi107 systemd[1]: Starting Ceph mds.a for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:13:54.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:54 smithi195 ceph-mon[32833]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:54.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:54 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3572741740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:54.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:54 smithi170 ceph-mon[32807]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:54.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:54 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3572741740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:54.480 INFO:journalctl@ceph.mds.a.smithi107.stdout:Apr 25 00:13:54 smithi107 podman[67948]: 2024-04-25 00:13:54.227105672 +0000 UTC m=+0.102526147 container create 956c23a13e3b12b48ec09edff62cdd9e21e53f008c0d337d324df9fe733f5df3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-a, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T00:13:54.480 INFO:journalctl@ceph.mds.a.smithi107.stdout:Apr 25 00:13:54 smithi107 podman[67948]: 2024-04-25 00:13:54.142559442 +0000 UTC m=+0.017979917 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:13:54.480 INFO:journalctl@ceph.mds.a.smithi107.stdout:Apr 25 00:13:54 smithi107 podman[67948]: 2024-04-25 00:13:54.480508275 +0000 UTC m=+0.355928750 container init 956c23a13e3b12b48ec09edff62cdd9e21e53f008c0d337d324df9fe733f5df3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:13:54.735 INFO:journalctl@ceph.mds.a.smithi107.stdout:Apr 25 00:13:54 smithi107 podman[67948]: 2024-04-25 00:13:54.483397755 +0000 UTC m=+0.358818229 container start 956c23a13e3b12b48ec09edff62cdd9e21e53f008c0d337d324df9fe733f5df3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-a, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T00:13:54.735 INFO:journalctl@ceph.mds.a.smithi107.stdout:Apr 25 00:13:54 smithi107 ceph-mds[67967]: -- 172.21.15.107:0/1586611094 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559ba4c3fa40 con 0x559ba4c12c00 2024-04-25T00:13:54.735 INFO:journalctl@ceph.mds.a.smithi107.stdout:Apr 25 00:13:54 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-a[67963]: starting mds.a at 2024-04-25T00:13:54.735 INFO:journalctl@ceph.mds.a.smithi107.stdout:Apr 25 00:13:54 smithi107 bash[67948]: 956c23a13e3b12b48ec09edff62cdd9e21e53f008c0d337d324df9fe733f5df3 2024-04-25T00:13:54.735 INFO:journalctl@ceph.mds.a.smithi107.stdout:Apr 25 00:13:54 smithi107 systemd[1]: Started Ceph mds.a for 79602f58-0297-11ef-bc93-c7b262605968. 2024-04-25T00:13:55.465 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:55.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.577+0000 7f3f3aeba640 1 -- 172.21.15.107:0/2372560650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f34101870 msgr2=0x7f3f34101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:55.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.577+0000 7f3f3aeba640 1 --2- 172.21.15.107:0/2372560650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f34101870 0x7f3f34101cf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c009900 tx=0x7f3f1c02a410 comp rx=0 tx=0).stop 2024-04-25T00:13:55.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.577+0000 7f3f3aeba640 1 -- 172.21.15.107:0/2372560650 shutdown_connections 2024-04-25T00:13:55.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.577+0000 7f3f3aeba640 1 --2- 172.21.15.107:0/2372560650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f34102230 0x7f3f34108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:55.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.577+0000 7f3f3aeba640 1 --2- 172.21.15.107:0/2372560650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f34101870 0x7f3f34101cf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:55.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.577+0000 7f3f3aeba640 1 --2- 172.21.15.107:0/2372560650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f34100680 0x7f3f34100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:55.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.577+0000 7f3f3aeba640 1 -- 172.21.15.107:0/2372560650 >> 172.21.15.107:0/2372560650 conn(0x7f3f340fbbf0 msgr2=0x7f3f340fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:55.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.577+0000 7f3f3aeba640 1 -- 172.21.15.107:0/2372560650 shutdown_connections 2024-04-25T00:13:55.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.577+0000 7f3f3aeba640 1 -- 172.21.15.107:0/2372560650 wait complete. 2024-04-25T00:13:55.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.578+0000 7f3f3aeba640 1 Processor -- start 2024-04-25T00:13:55.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.578+0000 7f3f3aeba640 1 -- start start 2024-04-25T00:13:55.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.578+0000 7f3f3aeba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f34100680 0x7f3f34199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:55.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.578+0000 7f3f3aeba640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f34101870 0x7f3f3419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:55.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.578+0000 7f3f3aeba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f34102230 0x7f3f341a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:55.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.578+0000 7f3f3aeba640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3410d420 con 0x7f3f34100680 2024-04-25T00:13:55.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.578+0000 7f3f3aeba640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3410d2a0 con 0x7f3f34101870 2024-04-25T00:13:55.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.578+0000 7f3f3aeba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3410d5a0 con 0x7f3f34102230 2024-04-25T00:13:55.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.578+0000 7f3f38c56640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f34100680 0x7f3f34199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:55.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.578+0000 7f3f38c56640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f34100680 0x7f3f34199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36224/0 (socket says 172.21.15.107:36224) 2024-04-25T00:13:55.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.578+0000 7f3f38c56640 1 -- 172.21.15.107:0/4171646007 learned_addr learned my addr 172.21.15.107:0/4171646007 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:55.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.578+0000 7f3f33fff640 1 --2- 172.21.15.107:0/4171646007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f34101870 0x7f3f3419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:55.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.578+0000 7f3f39457640 1 --2- 172.21.15.107:0/4171646007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f34102230 0x7f3f341a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:55.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.578+0000 7f3f38c56640 1 -- 172.21.15.107:0/4171646007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f34102230 msgr2=0x7f3f341a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:55.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.578+0000 7f3f38c56640 1 --2- 172.21.15.107:0/4171646007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f34102230 0x7f3f341a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:55.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.578+0000 7f3f38c56640 1 -- 172.21.15.107:0/4171646007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f34101870 msgr2=0x7f3f3419a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:55.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.578+0000 7f3f38c56640 1 --2- 172.21.15.107:0/4171646007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f34101870 0x7f3f3419a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:55.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.578+0000 7f3f38c56640 1 -- 172.21.15.107:0/4171646007 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f341a1910 con 0x7f3f34100680 2024-04-25T00:13:55.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.578+0000 7f3f38c56640 1 --2- 172.21.15.107:0/4171646007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f34100680 0x7f3f34199d00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00ea80 tx=0x7f3f2c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:55.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.578+0000 7f3f31ffb640 1 -- 172.21.15.107:0/4171646007 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c00ce40 con 0x7f3f34100680 2024-04-25T00:13:55.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.579+0000 7f3f31ffb640 1 -- 172.21.15.107:0/4171646007 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f2c011950 con 0x7f3f34100680 2024-04-25T00:13:55.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.579+0000 7f3f3aeba640 1 -- 172.21.15.107:0/4171646007 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f341a1c00 con 0x7f3f34100680 2024-04-25T00:13:55.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.579+0000 7f3f31ffb640 1 -- 172.21.15.107:0/4171646007 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c01ab80 con 0x7f3f34100680 2024-04-25T00:13:55.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.579+0000 7f3f3aeba640 1 -- 172.21.15.107:0/4171646007 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f341a2150 con 0x7f3f34100680 2024-04-25T00:13:55.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.580+0000 7f3f3aeba640 1 -- 172.21.15.107:0/4171646007 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f341a2320 con 0x7f3f34100680 2024-04-25T00:13:55.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.581+0000 7f3f31ffb640 1 -- 172.21.15.107:0/4171646007 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f3f2c011b00 con 0x7f3f34100680 2024-04-25T00:13:55.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.581+0000 7f3f31ffb640 1 --2- 172.21.15.107:0/4171646007 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f3f200a2b90 0x7f3f200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:55.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.581+0000 7f3f33fff640 1 --2- 172.21.15.107:0/4171646007 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f3f200a2b90 0x7f3f200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:55.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.581+0000 7f3f31ffb640 1 -- 172.21.15.107:0/4171646007 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f3f2c0cf1b0 con 0x7f3f34100680 2024-04-25T00:13:55.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.582+0000 7f3f33fff640 1 --2- 172.21.15.107:0/4171646007 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f3f200a2b90 0x7f3f200a5050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c0314e0 tx=0x7f3f1c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:55.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.586+0000 7f3f31ffb640 1 -- 172.21.15.107:0/4171646007 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3f2c016030 con 0x7f3f34100680 2024-04-25T00:13:55.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.800+0000 7f3f3aeba640 1 -- 172.21.15.107:0/4171646007 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f341a24f0 con 0x7f3f34100680 2024-04-25T00:13:55.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.800+0000 7f3f31ffb640 1 -- 172.21.15.107:0/4171646007 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9677 (secure 0 0 0) 0x7f3f2c09c5b0 con 0x7f3f34100680 2024-04-25T00:13:55.801 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:13:55.801 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T00:13:55.801 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 12 2024-04-25T00:13:55.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.802+0000 7f3f3aeba640 1 -- 172.21.15.107:0/4171646007 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f3f200a2b90 msgr2=0x7f3f200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:55.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.802+0000 7f3f3aeba640 1 --2- 172.21.15.107:0/4171646007 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f3f200a2b90 0x7f3f200a5050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c0314e0 tx=0x7f3f1c000bc0 comp rx=0 tx=0).stop 2024-04-25T00:13:55.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.802+0000 7f3f3aeba640 1 -- 172.21.15.107:0/4171646007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f34100680 msgr2=0x7f3f34199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:55.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.802+0000 7f3f3aeba640 1 --2- 172.21.15.107:0/4171646007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f34100680 0x7f3f34199d00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00ea80 tx=0x7f3f2c00ef40 comp rx=0 tx=0).stop 2024-04-25T00:13:55.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.803+0000 7f3f3aeba640 1 -- 172.21.15.107:0/4171646007 shutdown_connections 2024-04-25T00:13:55.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.803+0000 7f3f3aeba640 1 --2- 172.21.15.107:0/4171646007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f34102230 0x7f3f341a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:55.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.803+0000 7f3f3aeba640 1 --2- 172.21.15.107:0/4171646007 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f3f200a2b90 0x7f3f200a5050 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:55.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.803+0000 7f3f3aeba640 1 --2- 172.21.15.107:0/4171646007 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f34101870 0x7f3f3419a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:55.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.803+0000 7f3f3aeba640 1 --2- 172.21.15.107:0/4171646007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f34100680 0x7f3f34199d00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:55.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.803+0000 7f3f3aeba640 1 -- 172.21.15.107:0/4171646007 >> 172.21.15.107:0/4171646007 conn(0x7f3f340fbbf0 msgr2=0x7f3f34109480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:55.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.803+0000 7f3f3aeba640 1 -- 172.21.15.107:0/4171646007 shutdown_connections 2024-04-25T00:13:55.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:55.803+0000 7f3f3aeba640 1 -- 172.21.15.107:0/4171646007 wait complete. 2024-04-25T00:13:55.813 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:55 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:55.813 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:55 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:55.813 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:55 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:55.813 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:55 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:55.813 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:55 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:55.813 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:55 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:55.813 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:55 smithi107 ceph-mon[29153]: Deploying daemon mds.b on smithi170 2024-04-25T00:13:55.813 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:55 smithi107 ceph-mon[29153]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:55.813 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:55 smithi107 ceph-mon[29153]: mds.? [v2:172.21.15.107:6840/3282278035,v1:172.21.15.107:6841/3282278035] up:boot 2024-04-25T00:13:55.813 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:55 smithi107 ceph-mon[29153]: fsmap 11 up:standby 2024-04-25T00:13:55.813 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:55 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:13:55.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:55 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:55.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:55 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:55.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:55 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:55.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:55 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:55.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:55 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:55.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:55 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:55.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:55 smithi170 ceph-mon[32807]: Deploying daemon mds.b on smithi170 2024-04-25T00:13:55.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:55 smithi170 ceph-mon[32807]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:55.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:55 smithi170 ceph-mon[32807]: mds.? [v2:172.21.15.107:6840/3282278035,v1:172.21.15.107:6841/3282278035] up:boot 2024-04-25T00:13:55.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:55 smithi170 ceph-mon[32807]: fsmap 11 up:standby 2024-04-25T00:13:55.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:55 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:13:56.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:55 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:56.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:55 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:56.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:55 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:56.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:55 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:13:56.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:55 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:13:56.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:55 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:56.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:55 smithi195 ceph-mon[32833]: Deploying daemon mds.b on smithi170 2024-04-25T00:13:56.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:55 smithi195 ceph-mon[32833]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:56.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:55 smithi195 ceph-mon[32833]: mds.? [v2:172.21.15.107:6840/3282278035,v1:172.21.15.107:6841/3282278035] up:boot 2024-04-25T00:13:56.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:55 smithi195 ceph-mon[32833]: fsmap 11 up:standby 2024-04-25T00:13:56.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:55 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:13:56.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:56 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/4171646007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:56.935 INFO:journalctl@ceph.mds.b.smithi170.stdout:Apr 25 00:13:56 smithi170 systemd[1]: Starting Ceph mds.b for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:13:56.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:56 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/4171646007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:57.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:56 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/4171646007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:57.359 INFO:journalctl@ceph.mds.b.smithi170.stdout:Apr 25 00:13:57 smithi170 podman[62641]: 2024-04-25 00:13:57.008498469 +0000 UTC m=+0.018380505 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:13:57.360 INFO:journalctl@ceph.mds.b.smithi170.stdout:Apr 25 00:13:57 smithi170 podman[62641]: 2024-04-25 00:13:57.115493628 +0000 UTC m=+0.125375688 container create d651c23f6a0c494ae26f34eaae19664c3242e2a6f296953ee11b75324e176a78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:13:57.360 INFO:journalctl@ceph.mds.b.smithi170.stdout:Apr 25 00:13:57 smithi170 podman[62641]: 2024-04-25 00:13:57.359960252 +0000 UTC m=+0.369842270 container init d651c23f6a0c494ae26f34eaae19664c3242e2a6f296953ee11b75324e176a78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T00:13:57.664 INFO:journalctl@ceph.mds.b.smithi170.stdout:Apr 25 00:13:57 smithi170 podman[62641]: 2024-04-25 00:13:57.362668234 +0000 UTC m=+0.372550266 container start d651c23f6a0c494ae26f34eaae19664c3242e2a6f296953ee11b75324e176a78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T00:13:57.664 INFO:journalctl@ceph.mds.b.smithi170.stdout:Apr 25 00:13:57 smithi170 ceph-mds[62660]: -- 172.21.15.170:0/3583545866 <== mon.2 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55852e909a40 con 0x55852e8dd000 2024-04-25T00:13:57.664 INFO:journalctl@ceph.mds.b.smithi170.stdout:Apr 25 00:13:57 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-b[62656]: starting mds.b at 2024-04-25T00:13:57.664 INFO:journalctl@ceph.mds.b.smithi170.stdout:Apr 25 00:13:57 smithi170 bash[62641]: d651c23f6a0c494ae26f34eaae19664c3242e2a6f296953ee11b75324e176a78 2024-04-25T00:13:57.664 INFO:journalctl@ceph.mds.b.smithi170.stdout:Apr 25 00:13:57 smithi170 systemd[1]: Started Ceph mds.b for 79602f58-0297-11ef-bc93-c7b262605968. 2024-04-25T00:13:57.816 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:13:57.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.916+0000 7ff66f3db640 1 -- 172.21.15.107:0/1379839917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff668100680 msgr2=0x7ff668100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:57.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.916+0000 7ff66f3db640 1 --2- 172.21.15.107:0/1379839917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff668100680 0x7ff668100aa0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff664009900 tx=0x7ff66402a410 comp rx=0 tx=0).stop 2024-04-25T00:13:57.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.916+0000 7ff66f3db640 1 -- 172.21.15.107:0/1379839917 shutdown_connections 2024-04-25T00:13:57.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.916+0000 7ff66f3db640 1 --2- 172.21.15.107:0/1379839917 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff668102230 0x7ff668108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.916+0000 7ff66f3db640 1 --2- 172.21.15.107:0/1379839917 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff668101870 0x7ff668101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.916+0000 7ff66f3db640 1 --2- 172.21.15.107:0/1379839917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff668100680 0x7ff668100aa0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.916+0000 7ff66f3db640 1 -- 172.21.15.107:0/1379839917 >> 172.21.15.107:0/1379839917 conn(0x7ff6680fbbf0 msgr2=0x7ff6680fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:57.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.917+0000 7ff66f3db640 1 -- 172.21.15.107:0/1379839917 shutdown_connections 2024-04-25T00:13:57.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.917+0000 7ff66f3db640 1 -- 172.21.15.107:0/1379839917 wait complete. 2024-04-25T00:13:57.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.917+0000 7ff66f3db640 1 Processor -- start 2024-04-25T00:13:57.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.917+0000 7ff66f3db640 1 -- start start 2024-04-25T00:13:57.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.917+0000 7ff66f3db640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff668100680 0x7ff668199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.917+0000 7ff66f3db640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff668101870 0x7ff66819a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.917+0000 7ff66f3db640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff668102230 0x7ff6681a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.917+0000 7ff66f3db640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff66810c320 con 0x7ff668100680 2024-04-25T00:13:57.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.917+0000 7ff66f3db640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff66810c1a0 con 0x7ff668101870 2024-04-25T00:13:57.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.917+0000 7ff66f3db640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff66810c4a0 con 0x7ff668102230 2024-04-25T00:13:57.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.917+0000 7ff66d177640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff668100680 0x7ff668199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:57.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.917+0000 7ff66d177640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff668100680 0x7ff668199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36230/0 (socket says 172.21.15.107:36230) 2024-04-25T00:13:57.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.917+0000 7ff66d177640 1 -- 172.21.15.107:0/113441169 learned_addr learned my addr 172.21.15.107:0/113441169 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:57.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.917+0000 7ff66d978640 1 --2- 172.21.15.107:0/113441169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff668102230 0x7ff6681a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:57.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.917+0000 7ff66c976640 1 --2- 172.21.15.107:0/113441169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff668101870 0x7ff66819a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:57.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.918+0000 7ff66d177640 1 -- 172.21.15.107:0/113441169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff668102230 msgr2=0x7ff6681a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:57.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.918+0000 7ff66d177640 1 --2- 172.21.15.107:0/113441169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff668102230 0x7ff6681a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.918+0000 7ff66d177640 1 -- 172.21.15.107:0/113441169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff668101870 msgr2=0x7ff66819a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:57.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.918+0000 7ff66d177640 1 --2- 172.21.15.107:0/113441169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff668101870 0x7ff66819a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:57.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.918+0000 7ff66d177640 1 -- 172.21.15.107:0/113441169 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6681a1910 con 0x7ff668100680 2024-04-25T00:13:57.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.918+0000 7ff66d177640 1 --2- 172.21.15.107:0/113441169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff668100680 0x7ff668199d00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff664031440 tx=0x7ff664032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:57.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.918+0000 7ff65e7fc640 1 -- 172.21.15.107:0/113441169 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66403f030 con 0x7ff668100680 2024-04-25T00:13:57.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.918+0000 7ff65e7fc640 1 -- 172.21.15.107:0/113441169 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff664003ce0 con 0x7ff668100680 2024-04-25T00:13:57.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.918+0000 7ff66f3db640 1 -- 172.21.15.107:0/113441169 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6681a1ba0 con 0x7ff668100680 2024-04-25T00:13:57.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.918+0000 7ff65e7fc640 1 -- 172.21.15.107:0/113441169 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664035070 con 0x7ff668100680 2024-04-25T00:13:57.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.918+0000 7ff66f3db640 1 -- 172.21.15.107:0/113441169 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6681a2090 con 0x7ff668100680 2024-04-25T00:13:57.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.919+0000 7ff66f3db640 1 -- 172.21.15.107:0/113441169 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6681a2260 con 0x7ff668100680 2024-04-25T00:13:57.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.920+0000 7ff65e7fc640 1 -- 172.21.15.107:0/113441169 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff664042030 con 0x7ff668100680 2024-04-25T00:13:57.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.920+0000 7ff65e7fc640 1 --2- 172.21.15.107:0/113441169 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff6540a2b90 0x7ff6540a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:57.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.921+0000 7ff66c976640 1 --2- 172.21.15.107:0/113441169 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff6540a2b90 0x7ff6540a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:57.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.921+0000 7ff65e7fc640 1 -- 172.21.15.107:0/113441169 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff6640ef210 con 0x7ff668100680 2024-04-25T00:13:57.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.921+0000 7ff66c976640 1 --2- 172.21.15.107:0/113441169 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff6540a2b90 0x7ff6540a5050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff658005d00 tx=0x7ff65800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:57.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:57.925+0000 7ff65e7fc640 1 -- 172.21.15.107:0/113441169 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff6640bbfc0 con 0x7ff668100680 2024-04-25T00:13:57.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:57 smithi170 ceph-mon[32807]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:57.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:57 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:57.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:57 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:57.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:57 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:57.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:57 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:57.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:57 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:57.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:57 smithi107 ceph-mon[29153]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:57.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:57 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:57.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:57 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:57.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:57 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:57.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:57 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:57.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:57 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:58.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:57 smithi195 ceph-mon[32833]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:58.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:57 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:58.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:57 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:58.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:57 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:58.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:57 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:58.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:57 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:58.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.146+0000 7ff66f3db640 1 -- 172.21.15.107:0/113441169 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6681a2430 con 0x7ff668100680 2024-04-25T00:13:58.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.147+0000 7ff65e7fc640 1 -- 172.21.15.107:0/113441169 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10506 (secure 0 0 0) 0x7ff6640c02c0 con 0x7ff668100680 2024-04-25T00:13:58.148 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:13:58.148 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T00:13:58.148 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 13 2024-04-25T00:13:58.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.149+0000 7ff66f3db640 1 -- 172.21.15.107:0/113441169 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff6540a2b90 msgr2=0x7ff6540a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.149+0000 7ff66f3db640 1 --2- 172.21.15.107:0/113441169 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff6540a2b90 0x7ff6540a5050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff658005d00 tx=0x7ff65800b410 comp rx=0 tx=0).stop 2024-04-25T00:13:58.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.149+0000 7ff66f3db640 1 -- 172.21.15.107:0/113441169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff668100680 msgr2=0x7ff668199d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.149+0000 7ff66f3db640 1 --2- 172.21.15.107:0/113441169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff668100680 0x7ff668199d00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff664031440 tx=0x7ff664032e20 comp rx=0 tx=0).stop 2024-04-25T00:13:58.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.149+0000 7ff66f3db640 1 -- 172.21.15.107:0/113441169 shutdown_connections 2024-04-25T00:13:58.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.150+0000 7ff66f3db640 1 --2- 172.21.15.107:0/113441169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff668102230 0x7ff6681a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.150+0000 7ff66f3db640 1 --2- 172.21.15.107:0/113441169 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff6540a2b90 0x7ff6540a5050 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.150+0000 7ff66f3db640 1 --2- 172.21.15.107:0/113441169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff668101870 0x7ff66819a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.150+0000 7ff66f3db640 1 --2- 172.21.15.107:0/113441169 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff668100680 0x7ff668199d00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.150+0000 7ff66f3db640 1 -- 172.21.15.107:0/113441169 >> 172.21.15.107:0/113441169 conn(0x7ff6680fbbf0 msgr2=0x7ff6680fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:58.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.150+0000 7ff66f3db640 1 -- 172.21.15.107:0/113441169 shutdown_connections 2024-04-25T00:13:58.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.150+0000 7ff66f3db640 1 -- 172.21.15.107:0/113441169 wait complete. 2024-04-25T00:13:58.162 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:13:58.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.273+0000 7fc101509640 1 -- 172.21.15.107:0/3064757121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0fc104b30 msgr2=0x7fc0fc106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.273+0000 7fc101509640 1 --2- 172.21.15.107:0/3064757121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0fc104b30 0x7fc0fc106f20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009900 tx=0x7fc0e402a400 comp rx=0 tx=0).stop 2024-04-25T00:13:58.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.273+0000 7fc101509640 1 -- 172.21.15.107:0/3064757121 shutdown_connections 2024-04-25T00:13:58.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.273+0000 7fc101509640 1 --2- 172.21.15.107:0/3064757121 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0fc107460 0x7fc0fc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.273+0000 7fc101509640 1 --2- 172.21.15.107:0/3064757121 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0fc104b30 0x7fc0fc106f20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.273+0000 7fc101509640 1 --2- 172.21.15.107:0/3064757121 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0fc102200 0x7fc0fc1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.273+0000 7fc101509640 1 -- 172.21.15.107:0/3064757121 >> 172.21.15.107:0/3064757121 conn(0x7fc0fc0fbbf0 msgr2=0x7fc0fc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:58.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.273+0000 7fc101509640 1 -- 172.21.15.107:0/3064757121 shutdown_connections 2024-04-25T00:13:58.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.273+0000 7fc101509640 1 -- 172.21.15.107:0/3064757121 wait complete. 2024-04-25T00:13:58.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.273+0000 7fc101509640 1 Processor -- start 2024-04-25T00:13:58.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.273+0000 7fc101509640 1 -- start start 2024-04-25T00:13:58.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.274+0000 7fc101509640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0fc102200 0x7fc0fc101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.274+0000 7fc101509640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0fc104b30 0x7fc0fc1001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.274+0000 7fc101509640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0fc107460 0x7fc0fc1006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.274+0000 7fc101509640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc10d460 con 0x7fc0fc104b30 2024-04-25T00:13:58.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.274+0000 7fc101509640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc10d2e0 con 0x7fc0fc107460 2024-04-25T00:13:58.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.274+0000 7fc101509640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc10d5e0 con 0x7fc0fc102200 2024-04-25T00:13:58.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.274+0000 7fc0f3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0fc104b30 0x7fc0fc1001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.274+0000 7fc0f3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0fc104b30 0x7fc0fc1001b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36256/0 (socket says 172.21.15.107:36256) 2024-04-25T00:13:58.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.274+0000 7fc0f3fff640 1 -- 172.21.15.107:0/3282745729 learned_addr learned my addr 172.21.15.107:0/3282745729 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:58.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.274+0000 7fc0fb7fe640 1 --2- 172.21.15.107:0/3282745729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0fc107460 0x7fc0fc1006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.274+0000 7fc0faffd640 1 --2- 172.21.15.107:0/3282745729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0fc102200 0x7fc0fc101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.274+0000 7fc0f3fff640 1 -- 172.21.15.107:0/3282745729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0fc102200 msgr2=0x7fc0fc101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.274+0000 7fc0f3fff640 1 --2- 172.21.15.107:0/3282745729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0fc102200 0x7fc0fc101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.274+0000 7fc0f3fff640 1 -- 172.21.15.107:0/3282745729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0fc107460 msgr2=0x7fc0fc1006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.274+0000 7fc0f3fff640 1 --2- 172.21.15.107:0/3282745729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0fc107460 0x7fc0fc1006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.274+0000 7fc0f3fff640 1 -- 172.21.15.107:0/3282745729 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0fc100f80 con 0x7fc0fc104b30 2024-04-25T00:13:58.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.274+0000 7fc0faffd640 1 --2- 172.21.15.107:0/3282745729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0fc102200 0x7fc0fc101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:58.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.274+0000 7fc0f3fff640 1 --2- 172.21.15.107:0/3282745729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0fc104b30 0x7fc0fc1001b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4031610 tx=0x7fc0e4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:58.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.274+0000 7fc0f8ff9640 1 -- 172.21.15.107:0/3282745729 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e403f030 con 0x7fc0fc104b30 2024-04-25T00:13:58.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.274+0000 7fc0f8ff9640 1 -- 172.21.15.107:0/3282745729 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0e4003ad0 con 0x7fc0fc104b30 2024-04-25T00:13:58.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.274+0000 7fc101509640 1 -- 172.21.15.107:0/3282745729 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0fc1a1bf0 con 0x7fc0fc104b30 2024-04-25T00:13:58.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.274+0000 7fc0f8ff9640 1 -- 172.21.15.107:0/3282745729 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4035070 con 0x7fc0fc104b30 2024-04-25T00:13:58.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.274+0000 7fc101509640 1 -- 172.21.15.107:0/3282745729 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0fc1a2140 con 0x7fc0fc104b30 2024-04-25T00:13:58.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.275+0000 7fc101509640 1 -- 172.21.15.107:0/3282745729 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0fc0fd6e0 con 0x7fc0fc104b30 2024-04-25T00:13:58.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.277+0000 7fc0f8ff9640 1 -- 172.21.15.107:0/3282745729 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc0e4042030 con 0x7fc0fc104b30 2024-04-25T00:13:58.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.277+0000 7fc0f8ff9640 1 --2- 172.21.15.107:0/3282745729 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc0e80a2b90 0x7fc0e80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.278+0000 7fc0faffd640 1 --2- 172.21.15.107:0/3282745729 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc0e80a2b90 0x7fc0e80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.278+0000 7fc0f8ff9640 1 -- 172.21.15.107:0/3282745729 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc0e40ef5b0 con 0x7fc0fc104b30 2024-04-25T00:13:58.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.278+0000 7fc0faffd640 1 --2- 172.21.15.107:0/3282745729 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc0e80a2b90 0x7fc0e80a5050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec009930 tx=0x7fc0ec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:58.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.281+0000 7fc0f8ff9640 1 -- 172.21.15.107:0/3282745729 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc0e40bc910 con 0x7fc0fc104b30 2024-04-25T00:13:58.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.474+0000 7fc101509640 1 -- 172.21.15.107:0/3282745729 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0fc02d370 con 0x7fc0fc104b30 2024-04-25T00:13:58.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.475+0000 7fc0f8ff9640 1 -- 172.21.15.107:0/3282745729 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18462 (secure 0 0 0) 0x7fc0e40bc210 con 0x7fc0fc104b30 2024-04-25T00:13:58.475 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:13:58.475 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":82,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:13:12.278840+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:13:58.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.477+0000 7fc101509640 1 -- 172.21.15.107:0/3282745729 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc0e80a2b90 msgr2=0x7fc0e80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.477+0000 7fc101509640 1 --2- 172.21.15.107:0/3282745729 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc0e80a2b90 0x7fc0e80a5050 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec009930 tx=0x7fc0ec008040 comp rx=0 tx=0).stop 2024-04-25T00:13:58.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.477+0000 7fc101509640 1 -- 172.21.15.107:0/3282745729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0fc104b30 msgr2=0x7fc0fc1001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.477+0000 7fc101509640 1 --2- 172.21.15.107:0/3282745729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0fc104b30 0x7fc0fc1001b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4031610 tx=0x7fc0e4004270 comp rx=0 tx=0).stop 2024-04-25T00:13:58.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.477+0000 7fc101509640 1 -- 172.21.15.107:0/3282745729 shutdown_connections 2024-04-25T00:13:58.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.477+0000 7fc101509640 1 --2- 172.21.15.107:0/3282745729 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc0e80a2b90 0x7fc0e80a5050 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.477+0000 7fc101509640 1 --2- 172.21.15.107:0/3282745729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0fc107460 0x7fc0fc1006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.477+0000 7fc101509640 1 --2- 172.21.15.107:0/3282745729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0fc104b30 0x7fc0fc1001b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.477+0000 7fc101509640 1 --2- 172.21.15.107:0/3282745729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc0fc102200 0x7fc0fc101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.477+0000 7fc101509640 1 -- 172.21.15.107:0/3282745729 >> 172.21.15.107:0/3282745729 conn(0x7fc0fc0fbbf0 msgr2=0x7fc0fc102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:58.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.477+0000 7fc101509640 1 -- 172.21.15.107:0/3282745729 shutdown_connections 2024-04-25T00:13:58.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.477+0000 7fc101509640 1 -- 172.21.15.107:0/3282745729 wait complete. 2024-04-25T00:13:58.488 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:09:25.160329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:13:58.488 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:13:58.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.590+0000 7f8378629640 1 -- 172.21.15.107:0/551627336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8370100680 msgr2=0x7f8370100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.590+0000 7f8378629640 1 --2- 172.21.15.107:0/551627336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8370100680 0x7f8370100aa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f836c009900 tx=0x7f836c02a410 comp rx=0 tx=0).stop 2024-04-25T00:13:58.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.591+0000 7f8378629640 1 -- 172.21.15.107:0/551627336 shutdown_connections 2024-04-25T00:13:58.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.591+0000 7f8378629640 1 --2- 172.21.15.107:0/551627336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8370102230 0x7f8370108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.591+0000 7f8378629640 1 --2- 172.21.15.107:0/551627336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8370101870 0x7f8370101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.591+0000 7f8378629640 1 --2- 172.21.15.107:0/551627336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8370100680 0x7f8370100aa0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.591+0000 7f8378629640 1 -- 172.21.15.107:0/551627336 >> 172.21.15.107:0/551627336 conn(0x7f83700fbbf0 msgr2=0x7f83700fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:58.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.591+0000 7f8378629640 1 -- 172.21.15.107:0/551627336 shutdown_connections 2024-04-25T00:13:58.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.591+0000 7f8378629640 1 -- 172.21.15.107:0/551627336 wait complete. 2024-04-25T00:13:58.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.591+0000 7f8378629640 1 Processor -- start 2024-04-25T00:13:58.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.591+0000 7f8378629640 1 -- start start 2024-04-25T00:13:58.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.591+0000 7f8378629640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8370100680 0x7f8370197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.591+0000 7f8378629640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8370101870 0x7f8370198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.591+0000 7f8378629640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8370102230 0x7f837019f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.591+0000 7f8378629640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f837010c300 con 0x7f8370101870 2024-04-25T00:13:58.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.591+0000 7f8378629640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f837010c180 con 0x7f8370100680 2024-04-25T00:13:58.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.591+0000 7f8378629640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f837010c480 con 0x7f8370102230 2024-04-25T00:13:58.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.591+0000 7f8375bc4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8370101870 0x7f8370198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.591+0000 7f8375bc4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8370101870 0x7f8370198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36264/0 (socket says 172.21.15.107:36264) 2024-04-25T00:13:58.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.591+0000 7f8375bc4640 1 -- 172.21.15.107:0/797561132 learned_addr learned my addr 172.21.15.107:0/797561132 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:58.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.591+0000 7f83763c5640 1 --2- 172.21.15.107:0/797561132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8370100680 0x7f8370197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.591+0000 7f8376bc6640 1 --2- 172.21.15.107:0/797561132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8370102230 0x7f837019f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.592+0000 7f8375bc4640 1 -- 172.21.15.107:0/797561132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8370102230 msgr2=0x7f837019f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.592+0000 7f8375bc4640 1 --2- 172.21.15.107:0/797561132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8370102230 0x7f837019f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.592+0000 7f8375bc4640 1 -- 172.21.15.107:0/797561132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8370100680 msgr2=0x7f8370197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.592+0000 7f8375bc4640 1 --2- 172.21.15.107:0/797561132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8370100680 0x7f8370197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.592+0000 7f8375bc4640 1 -- 172.21.15.107:0/797561132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f837019f770 con 0x7f8370101870 2024-04-25T00:13:58.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.592+0000 7f8375bc4640 1 --2- 172.21.15.107:0/797561132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8370101870 0x7f8370198090 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f836000d770 tx=0x7f836000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:58.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.592+0000 7f83677fe640 1 -- 172.21.15.107:0/797561132 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360017070 con 0x7f8370101870 2024-04-25T00:13:58.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.592+0000 7f83677fe640 1 -- 172.21.15.107:0/797561132 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8360005500 con 0x7f8370101870 2024-04-25T00:13:58.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.592+0000 7f8378629640 1 -- 172.21.15.107:0/797561132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f837019fa60 con 0x7f8370101870 2024-04-25T00:13:58.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.592+0000 7f83677fe640 1 -- 172.21.15.107:0/797561132 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8360004e00 con 0x7f8370101870 2024-04-25T00:13:58.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.592+0000 7f8378629640 1 -- 172.21.15.107:0/797561132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f837019ffb0 con 0x7f8370101870 2024-04-25T00:13:58.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.593+0000 7f8378629640 1 -- 172.21.15.107:0/797561132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83701a0180 con 0x7f8370101870 2024-04-25T00:13:58.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.594+0000 7f83677fe640 1 -- 172.21.15.107:0/797561132 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f8360004fa0 con 0x7f8370101870 2024-04-25T00:13:58.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.595+0000 7f83677fe640 1 --2- 172.21.15.107:0/797561132 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f835c0a2c10 0x7f835c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.595+0000 7f83763c5640 1 --2- 172.21.15.107:0/797561132 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f835c0a2c10 0x7f835c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.595+0000 7f83677fe640 1 -- 172.21.15.107:0/797561132 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f83600ce930 con 0x7f8370101870 2024-04-25T00:13:58.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.595+0000 7f83763c5640 1 --2- 172.21.15.107:0/797561132 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f835c0a2c10 0x7f835c0a50d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f836c005ab0 tx=0x7f836c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:58.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.598+0000 7f83677fe640 1 -- 172.21.15.107:0/797561132 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f836009b760 con 0x7f8370101870 2024-04-25T00:13:58.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.782+0000 7f8378629640 1 -- 172.21.15.107:0/797561132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f83701a0470 con 0x7f8370101870 2024-04-25T00:13:58.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.782+0000 7f83677fe640 1 -- 172.21.15.107:0/797561132 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f836009fa60 con 0x7f8370101870 2024-04-25T00:13:58.783 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-04-25T00:13:58.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.784+0000 7f8378629640 1 -- 172.21.15.107:0/797561132 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f835c0a2c10 msgr2=0x7f835c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.785+0000 7f8378629640 1 --2- 172.21.15.107:0/797561132 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f835c0a2c10 0x7f835c0a50d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f836c005ab0 tx=0x7f836c000bc0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.785+0000 7f8378629640 1 -- 172.21.15.107:0/797561132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8370101870 msgr2=0x7f8370198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.785+0000 7f8378629640 1 --2- 172.21.15.107:0/797561132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8370101870 0x7f8370198090 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f836000d770 tx=0x7f836000dc30 comp rx=0 tx=0).stop 2024-04-25T00:13:58.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.785+0000 7f8378629640 1 -- 172.21.15.107:0/797561132 shutdown_connections 2024-04-25T00:13:58.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.785+0000 7f8378629640 1 --2- 172.21.15.107:0/797561132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8370102230 0x7f837019f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.785+0000 7f8378629640 1 --2- 172.21.15.107:0/797561132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8370101870 0x7f8370198090 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.785+0000 7f8378629640 1 --2- 172.21.15.107:0/797561132 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f835c0a2c10 0x7f835c0a50d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.785+0000 7f8378629640 1 --2- 172.21.15.107:0/797561132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8370100680 0x7f8370197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.785+0000 7f8378629640 1 -- 172.21.15.107:0/797561132 >> 172.21.15.107:0/797561132 conn(0x7f83700fbbf0 msgr2=0x7f8370068980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:58.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.785+0000 7f8378629640 1 -- 172.21.15.107:0/797561132 shutdown_connections 2024-04-25T00:13:58.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.785+0000 7f8378629640 1 -- 172.21.15.107:0/797561132 wait complete. 2024-04-25T00:13:58.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:58 smithi107 ceph-mon[29153]: mds.? [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] up:boot 2024-04-25T00:13:58.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:58 smithi107 ceph-mon[29153]: fsmap 12 up:standby 2024-04-25T00:13:58.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:58 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:13:58.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:58 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:58.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:58 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/113441169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:58.795 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:58 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3282745729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:13:58.796 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:13:58.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:58 smithi195 ceph-mon[32833]: mds.? [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] up:boot 2024-04-25T00:13:58.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:58 smithi195 ceph-mon[32833]: fsmap 12 up:standby 2024-04-25T00:13:58.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:58 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:13:58.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:58 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:58.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:58 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/113441169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:58.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:58 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3282745729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:13:58.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.895+0000 7ff31169a640 1 -- 172.21.15.107:0/875233585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff30c104b30 msgr2=0x7ff30c106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.895+0000 7ff31169a640 1 --2- 172.21.15.107:0/875233585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff30c104b30 0x7ff30c106f20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff300009900 tx=0x7ff30002a410 comp rx=0 tx=0).stop 2024-04-25T00:13:58.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.895+0000 7ff31169a640 1 -- 172.21.15.107:0/875233585 shutdown_connections 2024-04-25T00:13:58.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.895+0000 7ff31169a640 1 --2- 172.21.15.107:0/875233585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff30c107460 0x7ff30c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.895+0000 7ff31169a640 1 --2- 172.21.15.107:0/875233585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff30c104b30 0x7ff30c106f20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.895+0000 7ff31169a640 1 --2- 172.21.15.107:0/875233585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff30c102200 0x7ff30c1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.895+0000 7ff31169a640 1 -- 172.21.15.107:0/875233585 >> 172.21.15.107:0/875233585 conn(0x7ff30c0fbbf0 msgr2=0x7ff30c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:58.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.895+0000 7ff31169a640 1 -- 172.21.15.107:0/875233585 shutdown_connections 2024-04-25T00:13:58.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.895+0000 7ff31169a640 1 -- 172.21.15.107:0/875233585 wait complete. 2024-04-25T00:13:58.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.896+0000 7ff31169a640 1 Processor -- start 2024-04-25T00:13:58.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.896+0000 7ff31169a640 1 -- start start 2024-04-25T00:13:58.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.896+0000 7ff31169a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff30c102200 0x7ff30c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.896+0000 7ff31169a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff30c104b30 0x7ff30c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.896+0000 7ff31169a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff30c107460 0x7ff30c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.896+0000 7ff31169a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c10d460 con 0x7ff30c102200 2024-04-25T00:13:58.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.896+0000 7ff31169a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c10d2e0 con 0x7ff30c107460 2024-04-25T00:13:58.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.896+0000 7ff31169a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff30c10d5e0 con 0x7ff30c104b30 2024-04-25T00:13:58.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.896+0000 7ff30affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff30c102200 0x7ff30c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.896+0000 7ff30affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff30c102200 0x7ff30c197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36284/0 (socket says 172.21.15.107:36284) 2024-04-25T00:13:58.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.896+0000 7ff30affd640 1 -- 172.21.15.107:0/979697219 learned_addr learned my addr 172.21.15.107:0/979697219 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:58.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.896+0000 7ff30a7fc640 1 --2- 172.21.15.107:0/979697219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff30c104b30 0x7ff30c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.896+0000 7ff30b7fe640 1 --2- 172.21.15.107:0/979697219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff30c107460 0x7ff30c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.896+0000 7ff30affd640 1 -- 172.21.15.107:0/979697219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff30c104b30 msgr2=0x7ff30c198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.896+0000 7ff30affd640 1 --2- 172.21.15.107:0/979697219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff30c104b30 0x7ff30c198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.896+0000 7ff30affd640 1 -- 172.21.15.107:0/979697219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff30c107460 msgr2=0x7ff30c19f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:58.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.896+0000 7ff30affd640 1 --2- 172.21.15.107:0/979697219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff30c107460 0x7ff30c19f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:58.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.896+0000 7ff30affd640 1 -- 172.21.15.107:0/979697219 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30c19f6f0 con 0x7ff30c102200 2024-04-25T00:13:58.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.896+0000 7ff30a7fc640 1 --2- 172.21.15.107:0/979697219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff30c104b30 0x7ff30c198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:58.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.897+0000 7ff30affd640 1 --2- 172.21.15.107:0/979697219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff30c102200 0x7ff30c197ad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00ea80 tx=0x7ff2fc00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:58.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.897+0000 7ff2f3fff640 1 -- 172.21.15.107:0/979697219 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc00ce40 con 0x7ff30c102200 2024-04-25T00:13:58.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.897+0000 7ff31169a640 1 -- 172.21.15.107:0/979697219 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30c19f9e0 con 0x7ff30c102200 2024-04-25T00:13:58.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.897+0000 7ff2f3fff640 1 -- 172.21.15.107:0/979697219 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2fc005020 con 0x7ff30c102200 2024-04-25T00:13:58.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.897+0000 7ff2f3fff640 1 -- 172.21.15.107:0/979697219 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc011760 con 0x7ff30c102200 2024-04-25T00:13:58.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.897+0000 7ff31169a640 1 -- 172.21.15.107:0/979697219 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff30c19ff00 con 0x7ff30c102200 2024-04-25T00:13:58.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.898+0000 7ff31169a640 1 -- 172.21.15.107:0/979697219 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff30c068560 con 0x7ff30c102200 2024-04-25T00:13:58.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.899+0000 7ff2f3fff640 1 -- 172.21.15.107:0/979697219 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7ff2fc0051d0 con 0x7ff30c102200 2024-04-25T00:13:58.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.899+0000 7ff2f3fff640 1 --2- 172.21.15.107:0/979697219 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff2f40a2b90 0x7ff2f40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:58.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.900+0000 7ff30a7fc640 1 --2- 172.21.15.107:0/979697219 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff2f40a2b90 0x7ff2f40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:58.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.900+0000 7ff2f3fff640 1 -- 172.21.15.107:0/979697219 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff2fc0cf470 con 0x7ff30c102200 2024-04-25T00:13:58.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.900+0000 7ff30a7fc640 1 --2- 172.21.15.107:0/979697219 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff2f40a2b90 0x7ff2f40a5050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff30002aa00 tx=0x7ff300000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:58.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:58.903+0000 7ff2f3fff640 1 -- 172.21.15.107:0/979697219 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff2fc016030 con 0x7ff30c102200 2024-04-25T00:13:58.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:58 smithi170 ceph-mon[32807]: mds.? [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] up:boot 2024-04-25T00:13:58.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:58 smithi170 ceph-mon[32807]: fsmap 12 up:standby 2024-04-25T00:13:58.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:58 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:13:58.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:58 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:58.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:58 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/113441169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:13:58.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:58 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3282745729' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:13:59.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.088+0000 7ff31169a640 1 -- 172.21.15.107:0/979697219 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff30c02d300 con 0x7ff30c102200 2024-04-25T00:13:59.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.088+0000 7ff2f3fff640 1 -- 172.21.15.107:0/979697219 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7ff2fc09c870 con 0x7ff30c102200 2024-04-25T00:13:59.089 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:13:59.089 INFO:teuthology.orchestra.run.smithi107.stdout:[ 2024-04-25T00:13:59.089 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:13:59.089 INFO:teuthology.orchestra.run.smithi107.stdout: "poolnum": 1, 2024-04-25T00:13:59.089 INFO:teuthology.orchestra.run.smithi107.stdout: "poolname": ".mgr" 2024-04-25T00:13:59.089 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:13:59.089 INFO:teuthology.orchestra.run.smithi107.stdout:] 2024-04-25T00:13:59.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.090+0000 7ff31169a640 1 -- 172.21.15.107:0/979697219 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff2f40a2b90 msgr2=0x7ff2f40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:59.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.090+0000 7ff31169a640 1 --2- 172.21.15.107:0/979697219 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff2f40a2b90 0x7ff2f40a5050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff30002aa00 tx=0x7ff300000bc0 comp rx=0 tx=0).stop 2024-04-25T00:13:59.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.090+0000 7ff31169a640 1 -- 172.21.15.107:0/979697219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff30c102200 msgr2=0x7ff30c197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:59.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.090+0000 7ff31169a640 1 --2- 172.21.15.107:0/979697219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff30c102200 0x7ff30c197ad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00ea80 tx=0x7ff2fc00ef40 comp rx=0 tx=0).stop 2024-04-25T00:13:59.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.091+0000 7ff31169a640 1 -- 172.21.15.107:0/979697219 shutdown_connections 2024-04-25T00:13:59.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.091+0000 7ff31169a640 1 --2- 172.21.15.107:0/979697219 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff2f40a2b90 0x7ff2f40a5050 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:59.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.091+0000 7ff31169a640 1 --2- 172.21.15.107:0/979697219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff30c107460 0x7ff30c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:59.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.091+0000 7ff31169a640 1 --2- 172.21.15.107:0/979697219 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff30c104b30 0x7ff30c198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:59.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.091+0000 7ff31169a640 1 --2- 172.21.15.107:0/979697219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff30c102200 0x7ff30c197ad0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:59.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.091+0000 7ff31169a640 1 -- 172.21.15.107:0/979697219 >> 172.21.15.107:0/979697219 conn(0x7ff30c0fbbf0 msgr2=0x7ff30c102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:59.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.091+0000 7ff31169a640 1 -- 172.21.15.107:0/979697219 shutdown_connections 2024-04-25T00:13:59.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.091+0000 7ff31169a640 1 -- 172.21.15.107:0/979697219 wait complete. 2024-04-25T00:13:59.103 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:13:59.103 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:13:59.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.218+0000 7f6694f30640 1 -- 172.21.15.107:0/2848325755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6690101890 msgr2=0x7f6690101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:59.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.218+0000 7f6694f30640 1 --2- 172.21.15.107:0/2848325755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6690101890 0x7f6690101d10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6684009900 tx=0x7f668402a410 comp rx=0 tx=0).stop 2024-04-25T00:13:59.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.218+0000 7f6694f30640 1 -- 172.21.15.107:0/2848325755 shutdown_connections 2024-04-25T00:13:59.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.218+0000 7f6694f30640 1 --2- 172.21.15.107:0/2848325755 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6690102250 0x7f6690108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:59.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.218+0000 7f6694f30640 1 --2- 172.21.15.107:0/2848325755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6690101890 0x7f6690101d10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:59.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.218+0000 7f6694f30640 1 --2- 172.21.15.107:0/2848325755 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f66901006a0 0x7f6690100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:59.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.218+0000 7f6694f30640 1 -- 172.21.15.107:0/2848325755 >> 172.21.15.107:0/2848325755 conn(0x7f66900fbc30 msgr2=0x7f66900fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:59.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.218+0000 7f6694f30640 1 -- 172.21.15.107:0/2848325755 shutdown_connections 2024-04-25T00:13:59.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.218+0000 7f6694f30640 1 -- 172.21.15.107:0/2848325755 wait complete. 2024-04-25T00:13:59.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.219+0000 7f6694f30640 1 Processor -- start 2024-04-25T00:13:59.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.219+0000 7f6694f30640 1 -- start start 2024-04-25T00:13:59.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.219+0000 7f6694f30640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66901006a0 0x7f669006a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:59.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.219+0000 7f6694f30640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6690101890 0x7f669019c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:59.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.219+0000 7f6694f30640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6690102250 0x7f669019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:59.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.219+0000 7f6694f30640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f669010c300 con 0x7f6690101890 2024-04-25T00:13:59.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.219+0000 7f6694f30640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f669010c180 con 0x7f66901006a0 2024-04-25T00:13:59.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.219+0000 7f6694f30640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f669010c480 con 0x7f6690102250 2024-04-25T00:13:59.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.219+0000 7f668dd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6690101890 0x7f669019c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:59.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.219+0000 7f668dd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6690101890 0x7f669019c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36304/0 (socket says 172.21.15.107:36304) 2024-04-25T00:13:59.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.219+0000 7f668dd9b640 1 -- 172.21.15.107:0/1146646800 learned_addr learned my addr 172.21.15.107:0/1146646800 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:13:59.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.219+0000 7f668e59c640 1 --2- 172.21.15.107:0/1146646800 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66901006a0 0x7f669006a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:59.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.219+0000 7f668ed9d640 1 --2- 172.21.15.107:0/1146646800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6690102250 0x7f669019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:59.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.220+0000 7f668dd9b640 1 -- 172.21.15.107:0/1146646800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6690102250 msgr2=0x7f669019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:59.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.220+0000 7f668dd9b640 1 --2- 172.21.15.107:0/1146646800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6690102250 0x7f669019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:59.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.220+0000 7f668dd9b640 1 -- 172.21.15.107:0/1146646800 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66901006a0 msgr2=0x7f669006a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:59.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.220+0000 7f668dd9b640 1 --2- 172.21.15.107:0/1146646800 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66901006a0 0x7f669006a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:59.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.220+0000 7f668dd9b640 1 -- 172.21.15.107:0/1146646800 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f669019f780 con 0x7f6690101890 2024-04-25T00:13:59.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.220+0000 7f668ed9d640 1 --2- 172.21.15.107:0/1146646800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6690102250 0x7f669019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:13:59.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.220+0000 7f668dd9b640 1 --2- 172.21.15.107:0/1146646800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6690101890 0x7f669019c850 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6684005940 tx=0x7f6684004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:59.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.220+0000 7f667f7fe640 1 -- 172.21.15.107:0/1146646800 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668403f030 con 0x7f6690101890 2024-04-25T00:13:59.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.220+0000 7f6694f30640 1 -- 172.21.15.107:0/1146646800 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f669019f9b0 con 0x7f6690101890 2024-04-25T00:13:59.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.220+0000 7f667f7fe640 1 -- 172.21.15.107:0/1146646800 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6684003ce0 con 0x7f6690101890 2024-04-25T00:13:59.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.220+0000 7f667f7fe640 1 -- 172.21.15.107:0/1146646800 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6684035070 con 0x7f6690101890 2024-04-25T00:13:59.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.220+0000 7f6694f30640 1 -- 172.21.15.107:0/1146646800 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f669019fec0 con 0x7f6690101890 2024-04-25T00:13:59.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.221+0000 7f6694f30640 1 -- 172.21.15.107:0/1146646800 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6690107a60 con 0x7f6690101890 2024-04-25T00:13:59.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.222+0000 7f667f7fe640 1 -- 172.21.15.107:0/1146646800 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6684042030 con 0x7f6690101890 2024-04-25T00:13:59.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.223+0000 7f667f7fe640 1 --2- 172.21.15.107:0/1146646800 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f66780a2b90 0x7f66780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:59.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.223+0000 7f668e59c640 1 --2- 172.21.15.107:0/1146646800 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f66780a2b90 0x7f66780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:59.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.223+0000 7f667f7fe640 1 -- 172.21.15.107:0/1146646800 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f66840ef730 con 0x7f6690101890 2024-04-25T00:13:59.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.223+0000 7f668e59c640 1 --2- 172.21.15.107:0/1146646800 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f66780a2b90 0x7f66780a5050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f66901016f0 tx=0x7f668000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:59.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.226+0000 7f667f7fe640 1 -- 172.21.15.107:0/1146646800 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f66840bcb30 con 0x7f6690101890 2024-04-25T00:13:59.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:13:59.437+0000 7f6694f30640 1 -- 172.21.15.107:0/1146646800 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f669002d170 con 0x7f6690101890 2024-04-25T00:13:59.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:59 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/797561132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:13:59.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:59 smithi170 ceph-mon[32807]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:13:59.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:59 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/979697219' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T00:13:59.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:59 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:59.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:59 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:59.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:59 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:59.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:13:59 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:13:59.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:59 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/797561132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:14:00.056 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:59 smithi107 ceph-mon[29153]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:14:00.056 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:59 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/979697219' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T00:14:00.056 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:59 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:00.056 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:59 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:00.056 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:59 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:00.056 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:13:59 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:00.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:59 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/797561132' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:14:00.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:59 smithi195 ceph-mon[32833]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:14:00.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:59 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/979697219' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T00:14:00.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:59 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:00.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:59 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:00.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:59 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:00.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:13:59 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:00.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.387+0000 7f667f7fe640 1 -- 172.21.15.107:0/1146646800 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f66840bc430 con 0x7f6690101890 2024-04-25T00:14:00.387 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_metadata' created 2024-04-25T00:14:00.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.389+0000 7f6694f30640 1 -- 172.21.15.107:0/1146646800 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f66780a2b90 msgr2=0x7f66780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:00.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.389+0000 7f6694f30640 1 --2- 172.21.15.107:0/1146646800 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f66780a2b90 0x7f66780a5050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f66901016f0 tx=0x7f668000b390 comp rx=0 tx=0).stop 2024-04-25T00:14:00.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.389+0000 7f6694f30640 1 -- 172.21.15.107:0/1146646800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6690101890 msgr2=0x7f669019c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:00.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.389+0000 7f6694f30640 1 --2- 172.21.15.107:0/1146646800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6690101890 0x7f669019c850 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6684005940 tx=0x7f6684004360 comp rx=0 tx=0).stop 2024-04-25T00:14:00.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.389+0000 7f6694f30640 1 -- 172.21.15.107:0/1146646800 shutdown_connections 2024-04-25T00:14:00.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.389+0000 7f6694f30640 1 --2- 172.21.15.107:0/1146646800 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6690102250 0x7f669019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.390+0000 7f6694f30640 1 --2- 172.21.15.107:0/1146646800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6690101890 0x7f669019c850 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.390+0000 7f6694f30640 1 --2- 172.21.15.107:0/1146646800 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f66780a2b90 0x7f66780a5050 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.390+0000 7f6694f30640 1 --2- 172.21.15.107:0/1146646800 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f66901006a0 0x7f669006a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.390+0000 7f6694f30640 1 -- 172.21.15.107:0/1146646800 >> 172.21.15.107:0/1146646800 conn(0x7f66900fbc30 msgr2=0x7f6690104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:00.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.390+0000 7f6694f30640 1 -- 172.21.15.107:0/1146646800 shutdown_connections 2024-04-25T00:14:00.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.390+0000 7f6694f30640 1 -- 172.21.15.107:0/1146646800 wait complete. 2024-04-25T00:14:00.404 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T00:14:00.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.510+0000 7fdeb12bd640 1 -- 172.21.15.107:0/2117374869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdeac101790 msgr2=0x7fdeac101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:00.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.510+0000 7fdeb12bd640 1 --2- 172.21.15.107:0/2117374869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdeac101790 0x7fdeac101bb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009900 tx=0x7fdea402a410 comp rx=0 tx=0).stop 2024-04-25T00:14:00.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.510+0000 7fdeb12bd640 1 -- 172.21.15.107:0/2117374869 shutdown_connections 2024-04-25T00:14:00.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.511+0000 7fdeb12bd640 1 --2- 172.21.15.107:0/2117374869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdeac103340 0x7fdeac109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.511+0000 7fdeb12bd640 1 --2- 172.21.15.107:0/2117374869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeac102980 0x7fdeac102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.511+0000 7fdeb12bd640 1 --2- 172.21.15.107:0/2117374869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdeac101790 0x7fdeac101bb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.511+0000 7fdeb12bd640 1 -- 172.21.15.107:0/2117374869 >> 172.21.15.107:0/2117374869 conn(0x7fdeac0fcd20 msgr2=0x7fdeac0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:00.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.511+0000 7fdeb12bd640 1 -- 172.21.15.107:0/2117374869 shutdown_connections 2024-04-25T00:14:00.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.511+0000 7fdeb12bd640 1 -- 172.21.15.107:0/2117374869 wait complete. 2024-04-25T00:14:00.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.511+0000 7fdeb12bd640 1 Processor -- start 2024-04-25T00:14:00.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.511+0000 7fdeb12bd640 1 -- start start 2024-04-25T00:14:00.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.511+0000 7fdeb12bd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdeac101790 0x7fdeac197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.511+0000 7fdeb12bd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdeac102980 0x7fdeac198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.511+0000 7fdeb12bd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeac103340 0x7fdeac19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.511+0000 7fdeb12bd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac10d3e0 con 0x7fdeac101790 2024-04-25T00:14:00.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.511+0000 7fdeb12bd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac10d260 con 0x7fdeac103340 2024-04-25T00:14:00.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.511+0000 7fdeb12bd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac10d560 con 0x7fdeac102980 2024-04-25T00:14:00.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.511+0000 7fdeaaffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdeac101790 0x7fdeac197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.512+0000 7fdeaaffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdeac101790 0x7fdeac197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59410/0 (socket says 172.21.15.107:59410) 2024-04-25T00:14:00.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.512+0000 7fdeaaffd640 1 -- 172.21.15.107:0/3885171626 learned_addr learned my addr 172.21.15.107:0/3885171626 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:00.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.512+0000 7fdeaa7fc640 1 --2- 172.21.15.107:0/3885171626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdeac102980 0x7fdeac198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.512+0000 7fdeab7fe640 1 --2- 172.21.15.107:0/3885171626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeac103340 0x7fdeac19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.512+0000 7fdeaaffd640 1 -- 172.21.15.107:0/3885171626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdeac102980 msgr2=0x7fdeac198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:00.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.512+0000 7fdeaaffd640 1 --2- 172.21.15.107:0/3885171626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdeac102980 0x7fdeac198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.512+0000 7fdeaaffd640 1 -- 172.21.15.107:0/3885171626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeac103340 msgr2=0x7fdeac19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:00.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.512+0000 7fdeaaffd640 1 --2- 172.21.15.107:0/3885171626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeac103340 0x7fdeac19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:00.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.512+0000 7fdeaaffd640 1 -- 172.21.15.107:0/3885171626 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeac19f700 con 0x7fdeac101790 2024-04-25T00:14:00.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.512+0000 7fdeaaffd640 1 --2- 172.21.15.107:0/3885171626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdeac101790 0x7fdeac197ae0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdea4031610 tx=0x7fdea40043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:00.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.512+0000 7fde93fff640 1 -- 172.21.15.107:0/3885171626 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea403f030 con 0x7fdeac101790 2024-04-25T00:14:00.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.512+0000 7fde93fff640 1 -- 172.21.15.107:0/3885171626 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdea4003ce0 con 0x7fdeac101790 2024-04-25T00:14:00.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.512+0000 7fdeb12bd640 1 -- 172.21.15.107:0/3885171626 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeac19f990 con 0x7fdeac101790 2024-04-25T00:14:00.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.512+0000 7fde93fff640 1 -- 172.21.15.107:0/3885171626 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea4035070 con 0x7fdeac101790 2024-04-25T00:14:00.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.512+0000 7fdeb12bd640 1 -- 172.21.15.107:0/3885171626 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeac19fe80 con 0x7fdeac101790 2024-04-25T00:14:00.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.513+0000 7fdeb12bd640 1 -- 172.21.15.107:0/3885171626 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeac108b20 con 0x7fdeac101790 2024-04-25T00:14:00.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.514+0000 7fde93fff640 1 -- 172.21.15.107:0/3885171626 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fdea4042030 con 0x7fdeac101790 2024-04-25T00:14:00.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.515+0000 7fde93fff640 1 --2- 172.21.15.107:0/3885171626 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fde940a2b90 0x7fde940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:00.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.515+0000 7fde93fff640 1 -- 172.21.15.107:0/3885171626 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7fdea40ef470 con 0x7fdeac101790 2024-04-25T00:14:00.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.515+0000 7fdeaa7fc640 1 --2- 172.21.15.107:0/3885171626 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fde940a2b90 0x7fde940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:00.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.516+0000 7fdeaa7fc640 1 --2- 172.21.15.107:0/3885171626 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fde940a2b90 0x7fde940a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdeac198ff0 tx=0x7fdea0009760 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:00.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.519+0000 7fde93fff640 1 -- 172.21.15.107:0/3885171626 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdea40bc0b0 con 0x7fdeac101790 2024-04-25T00:14:00.679 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:00 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1146646800' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T00:14:00.679 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:00 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:00.679 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:00 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:00.679 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:00 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:00.679 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:00 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:00.680 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:00 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:00.680 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:00 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:00.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:00.717+0000 7fdeb12bd640 1 -- 172.21.15.107:0/3885171626 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fdeac02d170 con 0x7fdeac101790 2024-04-25T00:14:00.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:00 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1146646800' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T00:14:00.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:00 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:00.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:00 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:00.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:00 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:00.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:00 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:00.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:00 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:00.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:00 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:00.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:00 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1146646800' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T00:14:00.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:00 smithi170 ceph-mon[32807]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T00:14:00.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:00 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1146646800' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T00:14:00.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:00 smithi107 ceph-mon[29153]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T00:14:01.029 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:00 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1146646800' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T00:14:01.029 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:00 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:01.030 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:00 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:01.030 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:00 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:01.030 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:00 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:01.030 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:00 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:01.030 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:00 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:01.030 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:00 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1146646800' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T00:14:01.030 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:00 smithi195 ceph-mon[32833]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T00:14:01.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.389+0000 7fde93fff640 1 -- 172.21.15.107:0/3885171626 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v84) ==== 171+0+0 (secure 0 0 0) 0x7fdea40c03b0 con 0x7fdeac101790 2024-04-25T00:14:01.389 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_data' created 2024-04-25T00:14:01.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.392+0000 7fdeb12bd640 1 -- 172.21.15.107:0/3885171626 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fde940a2b90 msgr2=0x7fde940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:01.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.392+0000 7fdeb12bd640 1 --2- 172.21.15.107:0/3885171626 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fde940a2b90 0x7fde940a5050 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdeac198ff0 tx=0x7fdea0009760 comp rx=0 tx=0).stop 2024-04-25T00:14:01.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.392+0000 7fdeb12bd640 1 -- 172.21.15.107:0/3885171626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdeac101790 msgr2=0x7fdeac197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:01.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.392+0000 7fdeb12bd640 1 --2- 172.21.15.107:0/3885171626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdeac101790 0x7fdeac197ae0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdea4031610 tx=0x7fdea40043b0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.394+0000 7fdeb12bd640 1 -- 172.21.15.107:0/3885171626 shutdown_connections 2024-04-25T00:14:01.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.395+0000 7fdeb12bd640 1 --2- 172.21.15.107:0/3885171626 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fde940a2b90 0x7fde940a5050 secure :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdeac198ff0 tx=0x7fdea0009760 comp rx=0 tx=0).stop 2024-04-25T00:14:01.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.395+0000 7fdeb12bd640 1 --2- 172.21.15.107:0/3885171626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeac103340 0x7fdeac19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.395+0000 7fdeb12bd640 1 --2- 172.21.15.107:0/3885171626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdeac102980 0x7fdeac198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.395+0000 7fdeb12bd640 1 --2- 172.21.15.107:0/3885171626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdeac101790 0x7fdeac197ae0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.395+0000 7fdeb12bd640 1 -- 172.21.15.107:0/3885171626 >> 172.21.15.107:0/3885171626 conn(0x7fdeac0fcd20 msgr2=0x7fdeac105cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:01.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.396+0000 7fdeb12bd640 1 -- 172.21.15.107:0/3885171626 shutdown_connections 2024-04-25T00:14:01.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.396+0000 7fdeb12bd640 1 -- 172.21.15.107:0/3885171626 wait complete. 2024-04-25T00:14:01.419 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:14:01.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.526+0000 7fa3ae0df640 1 -- 172.21.15.107:0/1853885836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3a8102230 msgr2=0x7fa3a8108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:01.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.526+0000 7fa3ae0df640 1 --2- 172.21.15.107:0/1853885836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3a8102230 0x7fa3a8108ac0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009ca0 tx=0x7fa3a002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.526+0000 7fa3ae0df640 1 -- 172.21.15.107:0/1853885836 shutdown_connections 2024-04-25T00:14:01.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.526+0000 7fa3ae0df640 1 --2- 172.21.15.107:0/1853885836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3a8102230 0x7fa3a8108ac0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.526+0000 7fa3ae0df640 1 --2- 172.21.15.107:0/1853885836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3a8101870 0x7fa3a8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.526+0000 7fa3ae0df640 1 --2- 172.21.15.107:0/1853885836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3a8100680 0x7fa3a8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.526+0000 7fa3ae0df640 1 -- 172.21.15.107:0/1853885836 >> 172.21.15.107:0/1853885836 conn(0x7fa3a80fbc30 msgr2=0x7fa3a80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:01.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.526+0000 7fa3ae0df640 1 -- 172.21.15.107:0/1853885836 shutdown_connections 2024-04-25T00:14:01.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.526+0000 7fa3ae0df640 1 -- 172.21.15.107:0/1853885836 wait complete. 2024-04-25T00:14:01.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.527+0000 7fa3ae0df640 1 Processor -- start 2024-04-25T00:14:01.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.527+0000 7fa3ae0df640 1 -- start start 2024-04-25T00:14:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.527+0000 7fa3ae0df640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3a8100680 0x7fa3a8196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.527+0000 7fa3ae0df640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3a8101870 0x7fa3a8196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.527+0000 7fa3ae0df640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3a8102230 0x7fa3a819def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.527+0000 7fa3ae0df640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a810c300 con 0x7fa3a8101870 2024-04-25T00:14:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.527+0000 7fa3ae0df640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a810c180 con 0x7fa3a8102230 2024-04-25T00:14:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.527+0000 7fa3ae0df640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a810c480 con 0x7fa3a8100680 2024-04-25T00:14:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.527+0000 7fa3a77fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3a8100680 0x7fa3a8196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.527+0000 7fa3a6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3a8101870 0x7fa3a8196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.527+0000 7fa3a77fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3a8100680 0x7fa3a8196940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:41386/0 (socket says 172.21.15.107:41386) 2024-04-25T00:14:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.527+0000 7fa3a6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3a8101870 0x7fa3a8196e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59424/0 (socket says 172.21.15.107:59424) 2024-04-25T00:14:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.527+0000 7fa3a77fe640 1 -- 172.21.15.107:0/1275388719 learned_addr learned my addr 172.21.15.107:0/1275388719 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.527+0000 7fa3a7fff640 1 --2- 172.21.15.107:0/1275388719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3a8102230 0x7fa3a819def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:01.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.528+0000 7fa3a77fe640 1 -- 172.21.15.107:0/1275388719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3a8102230 msgr2=0x7fa3a819def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:01.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.528+0000 7fa3a77fe640 1 --2- 172.21.15.107:0/1275388719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3a8102230 0x7fa3a819def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.528+0000 7fa3a77fe640 1 -- 172.21.15.107:0/1275388719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3a8101870 msgr2=0x7fa3a8196e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:01.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.528+0000 7fa3a77fe640 1 --2- 172.21.15.107:0/1275388719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3a8101870 0x7fa3a8196e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:01.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.528+0000 7fa3a77fe640 1 -- 172.21.15.107:0/1275388719 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a819e560 con 0x7fa3a8100680 2024-04-25T00:14:01.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.528+0000 7fa3a77fe640 1 --2- 172.21.15.107:0/1275388719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3a8100680 0x7fa3a8196940 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa39800e9c0 tx=0x7fa39800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:01.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.528+0000 7fa3a4ff9640 1 -- 172.21.15.107:0/1275388719 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39800cd80 con 0x7fa3a8100680 2024-04-25T00:14:01.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.528+0000 7fa3ae0df640 1 -- 172.21.15.107:0/1275388719 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a819e850 con 0x7fa3a8100680 2024-04-25T00:14:01.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.528+0000 7fa3a4ff9640 1 -- 172.21.15.107:0/1275388719 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3980035f0 con 0x7fa3a8100680 2024-04-25T00:14:01.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.528+0000 7fa3a4ff9640 1 -- 172.21.15.107:0/1275388719 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398010690 con 0x7fa3a8100680 2024-04-25T00:14:01.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.528+0000 7fa3ae0df640 1 -- 172.21.15.107:0/1275388719 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a819edf0 con 0x7fa3a8100680 2024-04-25T00:14:01.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.529+0000 7fa3ae0df640 1 -- 172.21.15.107:0/1275388719 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3a81079a0 con 0x7fa3a8100680 2024-04-25T00:14:01.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.530+0000 7fa3a4ff9640 1 -- 172.21.15.107:0/1275388719 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fa3980037a0 con 0x7fa3a8100680 2024-04-25T00:14:01.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.531+0000 7fa3a4ff9640 1 --2- 172.21.15.107:0/1275388719 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa3900a2b90 0x7fa3900a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:01.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.531+0000 7fa3a6ffd640 1 --2- 172.21.15.107:0/1275388719 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa3900a2b90 0x7fa3900a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:01.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.531+0000 7fa3a4ff9640 1 -- 172.21.15.107:0/1275388719 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7fa3980cf300 con 0x7fa3a8100680 2024-04-25T00:14:01.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.531+0000 7fa3a6ffd640 1 --2- 172.21.15.107:0/1275388719 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa3900a2b90 0x7fa3900a5050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8197e50 tx=0x7fa39c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:01.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.536+0000 7fa3a4ff9640 1 -- 172.21.15.107:0/1275388719 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa39809c370 con 0x7fa3a8100680 2024-04-25T00:14:01.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:01.781+0000 7fa3ae0df640 1 -- 172.21.15.107:0/1275388719 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa3a802d290 con 0x7fa3a8100680 2024-04-25T00:14:01.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:01 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3885171626' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-25T00:14:01.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:01 smithi107 ceph-mon[29153]: pgmap v283: 33 pgs: 1 creating+peering, 1 active+clean, 31 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:14:01.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:01.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:01 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3885171626' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-25T00:14:01.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:01 smithi107 ceph-mon[29153]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T00:14:02.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:01 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3885171626' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-25T00:14:02.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:01 smithi195 ceph-mon[32833]: pgmap v283: 33 pgs: 1 creating+peering, 1 active+clean, 31 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:14:02.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:02.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:01 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3885171626' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-25T00:14:02.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:01 smithi195 ceph-mon[32833]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T00:14:02.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:01 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3885171626' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-25T00:14:02.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:01 smithi170 ceph-mon[32807]: pgmap v283: 33 pgs: 1 creating+peering, 1 active+clean, 31 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:14:02.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:02.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:01 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3885171626' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-25T00:14:02.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:01 smithi170 ceph-mon[32807]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T00:14:02.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.787+0000 7fa3a4ff9640 1 -- 172.21.15.107:0/1275388719 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fa39809bc70 con 0x7fa3a8100680 2024-04-25T00:14:02.787 INFO:teuthology.orchestra.run.smithi107.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T00:14:02.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.789+0000 7fa3ae0df640 1 -- 172.21.15.107:0/1275388719 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa3900a2b90 msgr2=0x7fa3900a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.789+0000 7fa3ae0df640 1 --2- 172.21.15.107:0/1275388719 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa3900a2b90 0x7fa3900a5050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8197e50 tx=0x7fa39c005f90 comp rx=0 tx=0).stop 2024-04-25T00:14:02.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.789+0000 7fa3ae0df640 1 -- 172.21.15.107:0/1275388719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3a8100680 msgr2=0x7fa3a8196940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.789+0000 7fa3ae0df640 1 --2- 172.21.15.107:0/1275388719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3a8100680 0x7fa3a8196940 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa39800e9c0 tx=0x7fa39800ee80 comp rx=0 tx=0).stop 2024-04-25T00:14:02.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.789+0000 7fa3ae0df640 1 -- 172.21.15.107:0/1275388719 shutdown_connections 2024-04-25T00:14:02.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.789+0000 7fa3ae0df640 1 --2- 172.21.15.107:0/1275388719 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa3900a2b90 0x7fa3900a5050 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.789+0000 7fa3ae0df640 1 --2- 172.21.15.107:0/1275388719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3a8102230 0x7fa3a819def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.789+0000 7fa3ae0df640 1 --2- 172.21.15.107:0/1275388719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3a8101870 0x7fa3a8196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.789+0000 7fa3ae0df640 1 --2- 172.21.15.107:0/1275388719 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3a8100680 0x7fa3a8196940 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.790+0000 7fa3ae0df640 1 -- 172.21.15.107:0/1275388719 >> 172.21.15.107:0/1275388719 conn(0x7fa3a80fbc30 msgr2=0x7fa3a8104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:02.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.790+0000 7fa3ae0df640 1 -- 172.21.15.107:0/1275388719 shutdown_connections 2024-04-25T00:14:02.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.790+0000 7fa3ae0df640 1 -- 172.21.15.107:0/1275388719 wait complete. 2024-04-25T00:14:02.805 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T00:14:02.805 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T00:14:02.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.912+0000 7f6d8ebce640 1 -- 172.21.15.107:0/2067662653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d88100680 msgr2=0x7f6d88100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.912+0000 7f6d8ebce640 1 --2- 172.21.15.107:0/2067662653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d88100680 0x7f6d88100aa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009900 tx=0x7f6d8402a410 comp rx=0 tx=0).stop 2024-04-25T00:14:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.912+0000 7f6d8ebce640 1 -- 172.21.15.107:0/2067662653 shutdown_connections 2024-04-25T00:14:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.912+0000 7f6d8ebce640 1 --2- 172.21.15.107:0/2067662653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d88102230 0x7f6d88108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.912+0000 7f6d8ebce640 1 --2- 172.21.15.107:0/2067662653 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d88101870 0x7f6d88101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.912+0000 7f6d8ebce640 1 --2- 172.21.15.107:0/2067662653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d88100680 0x7f6d88100aa0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.912+0000 7f6d8ebce640 1 -- 172.21.15.107:0/2067662653 >> 172.21.15.107:0/2067662653 conn(0x7f6d880fbbf0 msgr2=0x7f6d880fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.912+0000 7f6d8ebce640 1 -- 172.21.15.107:0/2067662653 shutdown_connections 2024-04-25T00:14:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.912+0000 7f6d8ebce640 1 -- 172.21.15.107:0/2067662653 wait complete. 2024-04-25T00:14:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.913+0000 7f6d8ebce640 1 Processor -- start 2024-04-25T00:14:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.913+0000 7f6d8ebce640 1 -- start start 2024-04-25T00:14:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.913+0000 7f6d8ebce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d88100680 0x7f6d8806a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.913+0000 7f6d8ebce640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d88101870 0x7f6d8819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.913+0000 7f6d8ebce640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d88102230 0x7f6d8819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:02.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.913+0000 7f6d8ebce640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8810c2d0 con 0x7f6d88100680 2024-04-25T00:14:02.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.913+0000 7f6d8ebce640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8810c150 con 0x7f6d88101870 2024-04-25T00:14:02.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.913+0000 7f6d8ebce640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8810c450 con 0x7f6d88102230 2024-04-25T00:14:02.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.913+0000 7f6d8c96a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d88100680 0x7f6d8806a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:02.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.913+0000 7f6d8c96a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d88100680 0x7f6d8806a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59444/0 (socket says 172.21.15.107:59444) 2024-04-25T00:14:02.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.913+0000 7f6d8c96a640 1 -- 172.21.15.107:0/454094638 learned_addr learned my addr 172.21.15.107:0/454094638 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:02.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.913+0000 7f6d8d16b640 1 --2- 172.21.15.107:0/454094638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d88102230 0x7f6d8819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:02.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.913+0000 7f6d7ffff640 1 --2- 172.21.15.107:0/454094638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d88101870 0x7f6d8819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:02.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.913+0000 7f6d8c96a640 1 -- 172.21.15.107:0/454094638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d88102230 msgr2=0x7f6d8819ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.913+0000 7f6d8c96a640 1 --2- 172.21.15.107:0/454094638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d88102230 0x7f6d8819ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.913+0000 7f6d8c96a640 1 -- 172.21.15.107:0/454094638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d88101870 msgr2=0x7f6d8819c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:02.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.913+0000 7f6d8c96a640 1 --2- 172.21.15.107:0/454094638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d88101870 0x7f6d8819c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:02.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.913+0000 7f6d8c96a640 1 -- 172.21.15.107:0/454094638 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8819f6a0 con 0x7f6d88100680 2024-04-25T00:14:02.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.913+0000 7f6d7ffff640 1 --2- 172.21.15.107:0/454094638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d88101870 0x7f6d8819c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:02.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.913+0000 7f6d8c96a640 1 --2- 172.21.15.107:0/454094638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d88100680 0x7f6d8806a8f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6d84031610 tx=0x7f6d840043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:02.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.914+0000 7f6d7dffb640 1 -- 172.21.15.107:0/454094638 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8403f030 con 0x7f6d88100680 2024-04-25T00:14:02.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.914+0000 7f6d8ebce640 1 -- 172.21.15.107:0/454094638 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d8819f930 con 0x7f6d88100680 2024-04-25T00:14:02.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.914+0000 7f6d7dffb640 1 -- 172.21.15.107:0/454094638 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d84003ce0 con 0x7f6d88100680 2024-04-25T00:14:02.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.914+0000 7f6d7dffb640 1 -- 172.21.15.107:0/454094638 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d84035070 con 0x7f6d88100680 2024-04-25T00:14:02.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.914+0000 7f6d8ebce640 1 -- 172.21.15.107:0/454094638 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d8819fe20 con 0x7f6d88100680 2024-04-25T00:14:02.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.915+0000 7f6d8ebce640 1 -- 172.21.15.107:0/454094638 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d881079a0 con 0x7f6d88100680 2024-04-25T00:14:02.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.916+0000 7f6d7dffb640 1 -- 172.21.15.107:0/454094638 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f6d84042030 con 0x7f6d88100680 2024-04-25T00:14:02.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.916+0000 7f6d7dffb640 1 --2- 172.21.15.107:0/454094638 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f6d700a2b90 0x7f6d700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:02.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.917+0000 7f6d7ffff640 1 --2- 172.21.15.107:0/454094638 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f6d700a2b90 0x7f6d700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:02.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.917+0000 7f6d7dffb640 1 -- 172.21.15.107:0/454094638 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f6d840ef7e0 con 0x7f6d88100680 2024-04-25T00:14:02.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.917+0000 7f6d7ffff640 1 --2- 172.21.15.107:0/454094638 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f6d700a2b90 0x7f6d700a5050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6d8806a660 tx=0x7f6d78005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:02.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:02.920+0000 7f6d7dffb640 1 -- 172.21.15.107:0/454094638 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6d840bc890 con 0x7f6d88100680 2024-04-25T00:14:02.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:02 smithi107 ceph-mon[29153]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T00:14:02.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:02.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:02.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:02.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:02 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1275388719' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T00:14:02.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:02.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:02 smithi107 ceph-mon[29153]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T00:14:02.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:02 smithi107 ceph-mon[29153]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T00:14:02.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:02.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:02 smithi107 ceph-mon[29153]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T00:14:02.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:02.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:02.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:02.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:03.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:02 smithi195 ceph-mon[32833]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T00:14:03.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:03.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:03.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:03.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:02 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1275388719' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T00:14:03.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:03.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:02 smithi195 ceph-mon[32833]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T00:14:03.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:02 smithi195 ceph-mon[32833]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T00:14:03.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:03.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:02 smithi195 ceph-mon[32833]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T00:14:03.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:03.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:03.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:03.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:03.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.106+0000 7f6d8ebce640 1 -- 172.21.15.107:0/454094638 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f6d8802d370 con 0x7f6d88100680 2024-04-25T00:14:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:02 smithi170 ceph-mon[32807]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T00:14:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:02 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1275388719' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T00:14:03.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:03.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:02 smithi170 ceph-mon[32807]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T00:14:03.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:02 smithi170 ceph-mon[32807]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T00:14:03.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:03.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:02 smithi170 ceph-mon[32807]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T00:14:03.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:03.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:03.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:03.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:14:03.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.821+0000 7f6d7dffb640 1 -- 172.21.15.107:0/454094638 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v87) ==== 157+0+0 (secure 0 0 0) 0x7f6d840bc190 con 0x7f6d88100680 2024-04-25T00:14:03.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.824+0000 7f6d8ebce640 1 -- 172.21.15.107:0/454094638 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f6d700a2b90 msgr2=0x7f6d700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:03.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.824+0000 7f6d8ebce640 1 --2- 172.21.15.107:0/454094638 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f6d700a2b90 0x7f6d700a5050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6d8806a660 tx=0x7f6d78005e50 comp rx=0 tx=0).stop 2024-04-25T00:14:03.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.824+0000 7f6d8ebce640 1 -- 172.21.15.107:0/454094638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d88100680 msgr2=0x7f6d8806a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:03.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.824+0000 7f6d8ebce640 1 --2- 172.21.15.107:0/454094638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d88100680 0x7f6d8806a8f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6d84031610 tx=0x7f6d840043b0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.824+0000 7f6d8ebce640 1 -- 172.21.15.107:0/454094638 shutdown_connections 2024-04-25T00:14:03.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.824+0000 7f6d8ebce640 1 --2- 172.21.15.107:0/454094638 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d88102230 0x7f6d8819ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.824+0000 7f6d8ebce640 1 --2- 172.21.15.107:0/454094638 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f6d700a2b90 0x7f6d700a5050 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.824+0000 7f6d8ebce640 1 --2- 172.21.15.107:0/454094638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6d88101870 0x7f6d8819c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.824+0000 7f6d8ebce640 1 --2- 172.21.15.107:0/454094638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d88100680 0x7f6d8806a8f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.824+0000 7f6d8ebce640 1 -- 172.21.15.107:0/454094638 >> 172.21.15.107:0/454094638 conn(0x7f6d880fbbf0 msgr2=0x7f6d88104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:03.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.825+0000 7f6d8ebce640 1 -- 172.21.15.107:0/454094638 shutdown_connections 2024-04-25T00:14:03.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.825+0000 7f6d8ebce640 1 -- 172.21.15.107:0/454094638 wait complete. 2024-04-25T00:14:03.839 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:14:03.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.950+0000 7f44212fc640 1 -- 172.21.15.107:0/1819785948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f441c100700 msgr2=0x7f441c100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:03.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.950+0000 7f44212fc640 1 --2- 172.21.15.107:0/1819785948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f441c100700 0x7f441c100b20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f440c009ca0 tx=0x7f440c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.950+0000 7f44212fc640 1 -- 172.21.15.107:0/1819785948 shutdown_connections 2024-04-25T00:14:03.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.950+0000 7f44212fc640 1 --2- 172.21.15.107:0/1819785948 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f441c1022b0 0x7f441c108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.950+0000 7f44212fc640 1 --2- 172.21.15.107:0/1819785948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f441c1018f0 0x7f441c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.950+0000 7f44212fc640 1 --2- 172.21.15.107:0/1819785948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f441c100700 0x7f441c100b20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.950+0000 7f44212fc640 1 -- 172.21.15.107:0/1819785948 >> 172.21.15.107:0/1819785948 conn(0x7f441c0fbc90 msgr2=0x7f441c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:03.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.951+0000 7f44212fc640 1 -- 172.21.15.107:0/1819785948 shutdown_connections 2024-04-25T00:14:03.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.951+0000 7f44212fc640 1 -- 172.21.15.107:0/1819785948 wait complete. 2024-04-25T00:14:03.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.951+0000 7f44212fc640 1 Processor -- start 2024-04-25T00:14:03.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.951+0000 7f44212fc640 1 -- start start 2024-04-25T00:14:03.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.951+0000 7f44212fc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f441c100700 0x7f441c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:03.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.951+0000 7f44212fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f441c1018f0 0x7f441c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:03.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.951+0000 7f44212fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f441c1022b0 0x7f441c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:03.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.951+0000 7f44212fc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f441c10d470 con 0x7f441c1022b0 2024-04-25T00:14:03.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.951+0000 7f44212fc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f441c10d2f0 con 0x7f441c1018f0 2024-04-25T00:14:03.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.951+0000 7f44212fc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f441c10d5f0 con 0x7f441c100700 2024-04-25T00:14:03.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.952+0000 7f441b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f441c1022b0 0x7f441c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:03.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.952+0000 7f441b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f441c1022b0 0x7f441c19f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59452/0 (socket says 172.21.15.107:59452) 2024-04-25T00:14:03.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.952+0000 7f441b7fe640 1 -- 172.21.15.107:0/1592713022 learned_addr learned my addr 172.21.15.107:0/1592713022 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:03.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.952+0000 7f441affd640 1 --2- 172.21.15.107:0/1592713022 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f441c100700 0x7f441c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:03.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.952+0000 7f441a7fc640 1 --2- 172.21.15.107:0/1592713022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f441c1018f0 0x7f441c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:03.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.952+0000 7f441b7fe640 1 -- 172.21.15.107:0/1592713022 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f441c100700 msgr2=0x7f441c197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:03.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.952+0000 7f441b7fe640 1 --2- 172.21.15.107:0/1592713022 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f441c100700 0x7f441c197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.952+0000 7f441b7fe640 1 -- 172.21.15.107:0/1592713022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f441c1018f0 msgr2=0x7f441c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:03.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.952+0000 7f441b7fe640 1 --2- 172.21.15.107:0/1592713022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f441c1018f0 0x7f441c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:03.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.952+0000 7f441b7fe640 1 -- 172.21.15.107:0/1592713022 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f441c19f770 con 0x7f441c1022b0 2024-04-25T00:14:03.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.952+0000 7f441a7fc640 1 --2- 172.21.15.107:0/1592713022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f441c1018f0 0x7f441c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:03.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.952+0000 7f441b7fe640 1 --2- 172.21.15.107:0/1592713022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f441c1022b0 0x7f441c19f100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f441400c970 tx=0x7f441400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:03.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.952+0000 7f4403fff640 1 -- 172.21.15.107:0/1592713022 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414007bb0 con 0x7f441c1022b0 2024-04-25T00:14:03.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.952+0000 7f4403fff640 1 -- 172.21.15.107:0/1592713022 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4414007d50 con 0x7f441c1022b0 2024-04-25T00:14:03.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.952+0000 7f44212fc640 1 -- 172.21.15.107:0/1592713022 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f441c19fa60 con 0x7f441c1022b0 2024-04-25T00:14:03.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.952+0000 7f4403fff640 1 -- 172.21.15.107:0/1592713022 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441400f450 con 0x7f441c1022b0 2024-04-25T00:14:03.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.952+0000 7f44212fc640 1 -- 172.21.15.107:0/1592713022 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f441c19ffe0 con 0x7f441c1022b0 2024-04-25T00:14:03.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.953+0000 7f44212fc640 1 -- 172.21.15.107:0/1592713022 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f441c107a40 con 0x7f441c1022b0 2024-04-25T00:14:03.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.954+0000 7f4403fff640 1 -- 172.21.15.107:0/1592713022 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f4414004f00 con 0x7f441c1022b0 2024-04-25T00:14:03.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.955+0000 7f4403fff640 1 --2- 172.21.15.107:0/1592713022 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f44040a2b90 0x7f44040a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:03.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.955+0000 7f441affd640 1 --2- 172.21.15.107:0/1592713022 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f44040a2b90 0x7f44040a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:03.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.955+0000 7f4403fff640 1 -- 172.21.15.107:0/1592713022 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8277+0+0 (secure 0 0 0) 0x7f44140cf1b0 con 0x7f441c1022b0 2024-04-25T00:14:03.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.955+0000 7f441affd640 1 --2- 172.21.15.107:0/1592713022 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f44040a2b90 0x7f44040a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f440c006010 tx=0x7f440c005f80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:03.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:03.958+0000 7f4403fff640 1 -- 172.21.15.107:0/1592713022 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4414014030 con 0x7f441c1022b0 2024-04-25T00:14:04.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:03 smithi195 ceph-mon[32833]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T00:14:04.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:03 smithi195 ceph-mon[32833]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T00:14:04.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:03 smithi195 ceph-mon[32833]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T00:14:04.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:03 smithi195 ceph-mon[32833]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T00:14:04.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:03 smithi195 ceph-mon[32833]: pgmap v287: 97 pgs: 8 creating+peering, 19 active+clean, 70 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-25T00:14:04.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:03 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/454094638' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-25T00:14:04.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:04.135+0000 7f44212fc640 1 -- 172.21.15.107:0/1592713022 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f441c02d170 con 0x7f441c1022b0 2024-04-25T00:14:04.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:03 smithi170 ceph-mon[32807]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T00:14:04.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:03 smithi170 ceph-mon[32807]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T00:14:04.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:03 smithi170 ceph-mon[32807]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T00:14:04.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:03 smithi170 ceph-mon[32807]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T00:14:04.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:03 smithi170 ceph-mon[32807]: pgmap v287: 97 pgs: 8 creating+peering, 19 active+clean, 70 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-25T00:14:04.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:03 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/454094638' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-25T00:14:04.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:03 smithi107 ceph-mon[29153]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T00:14:04.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:03 smithi107 ceph-mon[29153]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T00:14:04.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:03 smithi107 ceph-mon[29153]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T00:14:04.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:03 smithi107 ceph-mon[29153]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T00:14:04.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:03 smithi107 ceph-mon[29153]: pgmap v287: 97 pgs: 8 creating+peering, 19 active+clean, 70 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-25T00:14:04.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:03 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/454094638' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-25T00:14:05.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:04 smithi195 ceph-mon[32833]: mds.? [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] up:active 2024-04-25T00:14:05.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:04 smithi195 ceph-mon[32833]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:14:05.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:04 smithi195 ceph-mon[32833]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:14:05.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:04 smithi195 ceph-mon[32833]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T00:14:05.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:04 smithi195 ceph-mon[32833]: Cluster is now healthy 2024-04-25T00:14:05.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:04 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/454094638' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-25T00:14:05.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:04 smithi195 ceph-mon[32833]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T00:14:05.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:04 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1592713022' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T00:14:05.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:04 smithi170 ceph-mon[32807]: mds.? [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] up:active 2024-04-25T00:14:05.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:04 smithi170 ceph-mon[32807]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:14:05.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:04 smithi170 ceph-mon[32807]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:14:05.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:04 smithi170 ceph-mon[32807]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T00:14:05.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:04 smithi170 ceph-mon[32807]: Cluster is now healthy 2024-04-25T00:14:05.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:04 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/454094638' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-25T00:14:05.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:04 smithi170 ceph-mon[32807]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T00:14:05.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:04 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1592713022' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T00:14:05.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:04 smithi107 ceph-mon[29153]: mds.? [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] up:active 2024-04-25T00:14:05.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:04 smithi107 ceph-mon[29153]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:14:05.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:04 smithi107 ceph-mon[29153]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:14:05.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:04 smithi107 ceph-mon[29153]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T00:14:05.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:04 smithi107 ceph-mon[29153]: Cluster is now healthy 2024-04-25T00:14:05.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:04 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/454094638' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-25T00:14:05.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:04 smithi107 ceph-mon[29153]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T00:14:05.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:04 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1592713022' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T00:14:05.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.831+0000 7f4403fff640 1 -- 172.21.15.107:0/1592713022 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v89) ==== 228+0+0 (secure 0 0 0) 0x7f441409c0e0 con 0x7f441c1022b0 2024-04-25T00:14:05.831 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_data_ec' created 2024-04-25T00:14:05.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.833+0000 7f44212fc640 1 -- 172.21.15.107:0/1592713022 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f44040a2b90 msgr2=0x7f44040a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.833+0000 7f44212fc640 1 --2- 172.21.15.107:0/1592713022 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f44040a2b90 0x7f44040a5050 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f440c006010 tx=0x7f440c005f80 comp rx=0 tx=0).stop 2024-04-25T00:14:05.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.833+0000 7f44212fc640 1 -- 172.21.15.107:0/1592713022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f441c1022b0 msgr2=0x7f441c19f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.833+0000 7f44212fc640 1 --2- 172.21.15.107:0/1592713022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f441c1022b0 0x7f441c19f100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f441400c970 tx=0x7f441400ce30 comp rx=0 tx=0).stop 2024-04-25T00:14:05.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.834+0000 7f44212fc640 1 -- 172.21.15.107:0/1592713022 shutdown_connections 2024-04-25T00:14:05.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.834+0000 7f44212fc640 1 --2- 172.21.15.107:0/1592713022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f441c1022b0 0x7f441c19f100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.834+0000 7f44212fc640 1 --2- 172.21.15.107:0/1592713022 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f44040a2b90 0x7f44040a5050 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.834+0000 7f44212fc640 1 --2- 172.21.15.107:0/1592713022 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f441c1018f0 0x7f441c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.834+0000 7f44212fc640 1 --2- 172.21.15.107:0/1592713022 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f441c100700 0x7f441c197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.834+0000 7f44212fc640 1 -- 172.21.15.107:0/1592713022 >> 172.21.15.107:0/1592713022 conn(0x7f441c0fbc90 msgr2=0x7f441c104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:05.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.834+0000 7f44212fc640 1 -- 172.21.15.107:0/1592713022 shutdown_connections 2024-04-25T00:14:05.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.834+0000 7f44212fc640 1 -- 172.21.15.107:0/1592713022 wait complete. 2024-04-25T00:14:05.852 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T00:14:05.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.958+0000 7f20179fb640 1 -- 172.21.15.107:0/3882457210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2010102230 msgr2=0x7f2010108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.958+0000 7f20179fb640 1 --2- 172.21.15.107:0/3882457210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2010102230 0x7f2010108ac0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2000009900 tx=0x7f200002a410 comp rx=0 tx=0).stop 2024-04-25T00:14:05.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.959+0000 7f20179fb640 1 -- 172.21.15.107:0/3882457210 shutdown_connections 2024-04-25T00:14:05.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.959+0000 7f20179fb640 1 --2- 172.21.15.107:0/3882457210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2010102230 0x7f2010108ac0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.959+0000 7f20179fb640 1 --2- 172.21.15.107:0/3882457210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2010101870 0x7f2010101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.959+0000 7f20179fb640 1 --2- 172.21.15.107:0/3882457210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2010100680 0x7f2010100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.959+0000 7f20179fb640 1 -- 172.21.15.107:0/3882457210 >> 172.21.15.107:0/3882457210 conn(0x7f20100fbbf0 msgr2=0x7f20100fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:05.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.959+0000 7f20179fb640 1 -- 172.21.15.107:0/3882457210 shutdown_connections 2024-04-25T00:14:05.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.959+0000 7f20179fb640 1 -- 172.21.15.107:0/3882457210 wait complete. 2024-04-25T00:14:05.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.959+0000 7f20179fb640 1 Processor -- start 2024-04-25T00:14:05.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.959+0000 7f20179fb640 1 -- start start 2024-04-25T00:14:05.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.959+0000 7f20179fb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2010100680 0x7f2010196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.959+0000 7f20179fb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2010101870 0x7f2010196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.959+0000 7f20179fb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2010102230 0x7f201019df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.959+0000 7f20179fb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f201010c2e0 con 0x7f2010101870 2024-04-25T00:14:05.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.959+0000 7f20179fb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f201010c160 con 0x7f2010102230 2024-04-25T00:14:05.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.959+0000 7f20179fb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f201010c460 con 0x7f2010100680 2024-04-25T00:14:05.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.959+0000 7f2014f96640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2010101870 0x7f2010196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.959+0000 7f2014f96640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2010101870 0x7f2010196e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59480/0 (socket says 172.21.15.107:59480) 2024-04-25T00:14:05.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.959+0000 7f2014f96640 1 -- 172.21.15.107:0/2106240418 learned_addr learned my addr 172.21.15.107:0/2106240418 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:05.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.960+0000 7f2015f98640 1 --2- 172.21.15.107:0/2106240418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2010102230 0x7f201019df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.960+0000 7f2015797640 1 --2- 172.21.15.107:0/2106240418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2010100680 0x7f2010196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.960+0000 7f2014f96640 1 -- 172.21.15.107:0/2106240418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2010100680 msgr2=0x7f2010196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.960+0000 7f2014f96640 1 --2- 172.21.15.107:0/2106240418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2010100680 0x7f2010196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.960+0000 7f2014f96640 1 -- 172.21.15.107:0/2106240418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2010102230 msgr2=0x7f201019df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:05.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.960+0000 7f2014f96640 1 --2- 172.21.15.107:0/2106240418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2010102230 0x7f201019df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:05.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.960+0000 7f2014f96640 1 -- 172.21.15.107:0/2106240418 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201019e570 con 0x7f2010101870 2024-04-25T00:14:05.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.960+0000 7f2015f98640 1 --2- 172.21.15.107:0/2106240418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2010102230 0x7f201019df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:05.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.960+0000 7f2014f96640 1 --2- 172.21.15.107:0/2106240418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2010101870 0x7f2010196e90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f200400eab0 tx=0x7f200400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:05.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.960+0000 7f200e7fc640 1 -- 172.21.15.107:0/2106240418 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200400ce40 con 0x7f2010101870 2024-04-25T00:14:05.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.960+0000 7f200e7fc640 1 -- 172.21.15.107:0/2106240418 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2004011950 con 0x7f2010101870 2024-04-25T00:14:05.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.960+0000 7f20179fb640 1 -- 172.21.15.107:0/2106240418 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f201019e860 con 0x7f2010101870 2024-04-25T00:14:05.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.960+0000 7f200e7fc640 1 -- 172.21.15.107:0/2106240418 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2004010ae0 con 0x7f2010101870 2024-04-25T00:14:05.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.960+0000 7f20179fb640 1 -- 172.21.15.107:0/2106240418 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f201019edb0 con 0x7f2010101870 2024-04-25T00:14:05.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.961+0000 7f20179fb640 1 -- 172.21.15.107:0/2106240418 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2010107a10 con 0x7f2010101870 2024-04-25T00:14:05.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.962+0000 7f200e7fc640 1 -- 172.21.15.107:0/2106240418 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f2004011b00 con 0x7f2010101870 2024-04-25T00:14:05.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.963+0000 7f200e7fc640 1 --2- 172.21.15.107:0/2106240418 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f1ff80a2b90 0x7f1ff80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:05.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.963+0000 7f2015797640 1 --2- 172.21.15.107:0/2106240418 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f1ff80a2b90 0x7f1ff80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:05.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.963+0000 7f200e7fc640 1 -- 172.21.15.107:0/2106240418 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7f20040cf920 con 0x7f2010101870 2024-04-25T00:14:05.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.963+0000 7f2015797640 1 --2- 172.21.15.107:0/2106240418 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f1ff80a2b90 0x7f1ff80a5050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2008005fd0 tx=0x7f2008005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:05.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:05.966+0000 7f200e7fc640 1 -- 172.21.15.107:0/2106240418 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2004016030 con 0x7f2010101870 2024-04-25T00:14:06.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:05 smithi195 ceph-mon[32833]: pgmap v290: 97 pgs: 8 creating+peering, 71 active+clean, 18 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 9 op/s 2024-04-25T00:14:06.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:05 smithi195 ceph-mon[32833]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T00:14:06.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.141+0000 7f20179fb640 1 -- 172.21.15.107:0/2106240418 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f201002d170 con 0x7f2010101870 2024-04-25T00:14:06.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:05 smithi170 ceph-mon[32807]: pgmap v290: 97 pgs: 8 creating+peering, 71 active+clean, 18 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 9 op/s 2024-04-25T00:14:06.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:05 smithi170 ceph-mon[32807]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T00:14:06.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:05 smithi107 ceph-mon[29153]: pgmap v290: 97 pgs: 8 creating+peering, 71 active+clean, 18 unknown; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 9 op/s 2024-04-25T00:14:06.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:05 smithi107 ceph-mon[29153]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T00:14:06.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.837+0000 7f200e7fc640 1 -- 172.21.15.107:0/2106240418 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v90) ==== 169+0+0 (secure 0 0 0) 0x7f200409c660 con 0x7f2010101870 2024-04-25T00:14:06.837 INFO:teuthology.orchestra.run.smithi107.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T00:14:06.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.840+0000 7f20179fb640 1 -- 172.21.15.107:0/2106240418 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f1ff80a2b90 msgr2=0x7f1ff80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:06.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.840+0000 7f20179fb640 1 --2- 172.21.15.107:0/2106240418 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f1ff80a2b90 0x7f1ff80a5050 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2008005fd0 tx=0x7f2008005c70 comp rx=0 tx=0).stop 2024-04-25T00:14:06.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.840+0000 7f20179fb640 1 -- 172.21.15.107:0/2106240418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2010101870 msgr2=0x7f2010196e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:06.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.840+0000 7f20179fb640 1 --2- 172.21.15.107:0/2106240418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2010101870 0x7f2010196e90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f200400eab0 tx=0x7f200400ef70 comp rx=0 tx=0).stop 2024-04-25T00:14:06.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.841+0000 7f20179fb640 1 -- 172.21.15.107:0/2106240418 shutdown_connections 2024-04-25T00:14:06.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.841+0000 7f20179fb640 1 --2- 172.21.15.107:0/2106240418 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f1ff80a2b90 0x7f1ff80a5050 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.841+0000 7f20179fb640 1 --2- 172.21.15.107:0/2106240418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2010102230 0x7f201019df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.841+0000 7f20179fb640 1 --2- 172.21.15.107:0/2106240418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2010101870 0x7f2010196e90 secure :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f200400eab0 tx=0x7f200400ef70 comp rx=0 tx=0).stop 2024-04-25T00:14:06.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.841+0000 7f20179fb640 1 --2- 172.21.15.107:0/2106240418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2010100680 0x7f2010196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.841+0000 7f20179fb640 1 -- 172.21.15.107:0/2106240418 >> 172.21.15.107:0/2106240418 conn(0x7f20100fbbf0 msgr2=0x7f2010104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:06.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.841+0000 7f20179fb640 1 -- 172.21.15.107:0/2106240418 shutdown_connections 2024-04-25T00:14:06.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.841+0000 7f20179fb640 1 -- 172.21.15.107:0/2106240418 wait complete. 2024-04-25T00:14:06.861 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T00:14:06.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.963+0000 7f193ee9e640 1 -- 172.21.15.107:0/90548003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1938104bb0 msgr2=0x7f1938106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:06.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.963+0000 7f193ee9e640 1 --2- 172.21.15.107:0/90548003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1938104bb0 0x7f1938106fa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1924009900 tx=0x7f192402a410 comp rx=0 tx=0).stop 2024-04-25T00:14:06.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.963+0000 7f193ee9e640 1 -- 172.21.15.107:0/90548003 shutdown_connections 2024-04-25T00:14:06.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.963+0000 7f193ee9e640 1 --2- 172.21.15.107:0/90548003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19381074e0 0x7f19381098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.963+0000 7f193ee9e640 1 --2- 172.21.15.107:0/90548003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1938104bb0 0x7f1938106fa0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.963+0000 7f193ee9e640 1 --2- 172.21.15.107:0/90548003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1938102280 0x7f1938104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.963+0000 7f193ee9e640 1 -- 172.21.15.107:0/90548003 >> 172.21.15.107:0/90548003 conn(0x7f19380fbc30 msgr2=0x7f19380fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:06.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.963+0000 7f193ee9e640 1 -- 172.21.15.107:0/90548003 shutdown_connections 2024-04-25T00:14:06.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.963+0000 7f193ee9e640 1 -- 172.21.15.107:0/90548003 wait complete. 2024-04-25T00:14:06.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.963+0000 7f193ee9e640 1 Processor -- start 2024-04-25T00:14:06.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.963+0000 7f193ee9e640 1 -- start start 2024-04-25T00:14:06.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.964+0000 7f193ee9e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1938102280 0x7f1938197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:06.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.964+0000 7f193ee9e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1938104bb0 0x7f1938198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:06.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.964+0000 7f193ee9e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19381074e0 0x7f193819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:06.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.964+0000 7f193ee9e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f193810d3f0 con 0x7f19381074e0 2024-04-25T00:14:06.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.964+0000 7f193ee9e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f193810d270 con 0x7f1938104bb0 2024-04-25T00:14:06.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.964+0000 7f193ee9e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f193810d570 con 0x7f1938102280 2024-04-25T00:14:06.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.964+0000 7f193d43b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19381074e0 0x7f193819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:06.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.964+0000 7f193d43b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19381074e0 0x7f193819f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59512/0 (socket says 172.21.15.107:59512) 2024-04-25T00:14:06.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.964+0000 7f193d43b640 1 -- 172.21.15.107:0/3156202237 learned_addr learned my addr 172.21.15.107:0/3156202237 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:06.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.964+0000 7f193cc3a640 1 --2- 172.21.15.107:0/3156202237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1938102280 0x7f1938197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:06.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.964+0000 7f1937fff640 1 --2- 172.21.15.107:0/3156202237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1938104bb0 0x7f1938198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:06.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.964+0000 7f193d43b640 1 -- 172.21.15.107:0/3156202237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1938102280 msgr2=0x7f1938197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:06.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.964+0000 7f193d43b640 1 --2- 172.21.15.107:0/3156202237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1938102280 0x7f1938197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.964+0000 7f193d43b640 1 -- 172.21.15.107:0/3156202237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1938104bb0 msgr2=0x7f1938198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:06.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.964+0000 7f193d43b640 1 --2- 172.21.15.107:0/3156202237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1938104bb0 0x7f1938198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:06.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.964+0000 7f193d43b640 1 -- 172.21.15.107:0/3156202237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193819f720 con 0x7f19381074e0 2024-04-25T00:14:06.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.964+0000 7f1937fff640 1 --2- 172.21.15.107:0/3156202237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1938104bb0 0x7f1938198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:06.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.964+0000 7f193d43b640 1 --2- 172.21.15.107:0/3156202237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19381074e0 0x7f193819f0b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f193000eaa0 tx=0x7f193000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:06.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.964+0000 7f1935ffb640 1 -- 172.21.15.107:0/3156202237 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193000cdb0 con 0x7f19381074e0 2024-04-25T00:14:06.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.964+0000 7f1935ffb640 1 -- 172.21.15.107:0/3156202237 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1930005020 con 0x7f19381074e0 2024-04-25T00:14:06.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.964+0000 7f1935ffb640 1 -- 172.21.15.107:0/3156202237 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19300116b0 con 0x7f19381074e0 2024-04-25T00:14:06.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.964+0000 7f193ee9e640 1 -- 172.21.15.107:0/3156202237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193819fa10 con 0x7f19381074e0 2024-04-25T00:14:06.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.964+0000 7f193ee9e640 1 -- 172.21.15.107:0/3156202237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f193819ff10 con 0x7f19381074e0 2024-04-25T00:14:06.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.965+0000 7f193ee9e640 1 -- 172.21.15.107:0/3156202237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19380fd720 con 0x7f19381074e0 2024-04-25T00:14:06.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.966+0000 7f1935ffb640 1 -- 172.21.15.107:0/3156202237 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1930005560 con 0x7f19381074e0 2024-04-25T00:14:06.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.967+0000 7f1935ffb640 1 --2- 172.21.15.107:0/3156202237 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f19280a2b90 0x7f19280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:06.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.967+0000 7f193cc3a640 1 --2- 172.21.15.107:0/3156202237 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f19280a2b90 0x7f19280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:06.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.967+0000 7f1935ffb640 1 -- 172.21.15.107:0/3156202237 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8770+0+0 (secure 0 0 0) 0x7f19300cf590 con 0x7f19381074e0 2024-04-25T00:14:06.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.967+0000 7f193cc3a640 1 --2- 172.21.15.107:0/3156202237 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f19280a2b90 0x7f19280a5050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f192c005fd0 tx=0x7f192c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:06.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:06.971+0000 7f1935ffb640 1 -- 172.21.15.107:0/3156202237 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1930016030 con 0x7f19381074e0 2024-04-25T00:14:07.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:06 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1592713022' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T00:14:07.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:06 smithi195 ceph-mon[32833]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T00:14:07.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:06 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2106240418' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-25T00:14:07.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:06 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1592713022' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T00:14:07.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:06 smithi170 ceph-mon[32807]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T00:14:07.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:06 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2106240418' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-25T00:14:07.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.186+0000 7f193ee9e640 1 -- 172.21.15.107:0/3156202237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f1938062340 con 0x7f19381074e0 2024-04-25T00:14:07.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:06 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1592713022' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T00:14:07.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:06 smithi107 ceph-mon[29153]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T00:14:07.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:06 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2106240418' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-25T00:14:07.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.844+0000 7f1935ffb640 1 -- 172.21.15.107:0/3156202237 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f193009c2d0 con 0x7f19381074e0 2024-04-25T00:14:07.844 INFO:teuthology.orchestra.run.smithi107.stderr:added data pool 4 to fsmap 2024-04-25T00:14:07.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.847+0000 7f193ee9e640 1 -- 172.21.15.107:0/3156202237 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f19280a2b90 msgr2=0x7f19280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:07.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.847+0000 7f193ee9e640 1 --2- 172.21.15.107:0/3156202237 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f19280a2b90 0x7f19280a5050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f192c005fd0 tx=0x7f192c005850 comp rx=0 tx=0).stop 2024-04-25T00:14:07.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.847+0000 7f193ee9e640 1 -- 172.21.15.107:0/3156202237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19381074e0 msgr2=0x7f193819f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:07.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.847+0000 7f193ee9e640 1 --2- 172.21.15.107:0/3156202237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19381074e0 0x7f193819f0b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f193000eaa0 tx=0x7f193000ef60 comp rx=0 tx=0).stop 2024-04-25T00:14:07.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.847+0000 7f193ee9e640 1 -- 172.21.15.107:0/3156202237 shutdown_connections 2024-04-25T00:14:07.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.847+0000 7f193ee9e640 1 --2- 172.21.15.107:0/3156202237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19381074e0 0x7f193819f0b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.847+0000 7f193ee9e640 1 --2- 172.21.15.107:0/3156202237 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f19280a2b90 0x7f19280a5050 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.847+0000 7f193ee9e640 1 --2- 172.21.15.107:0/3156202237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1938104bb0 0x7f1938198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.847+0000 7f193ee9e640 1 --2- 172.21.15.107:0/3156202237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1938102280 0x7f1938197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.847+0000 7f193ee9e640 1 -- 172.21.15.107:0/3156202237 >> 172.21.15.107:0/3156202237 conn(0x7f19380fbc30 msgr2=0x7f1938102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:07.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.847+0000 7f193ee9e640 1 -- 172.21.15.107:0/3156202237 shutdown_connections 2024-04-25T00:14:07.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.847+0000 7f193ee9e640 1 -- 172.21.15.107:0/3156202237 wait complete. 2024-04-25T00:14:07.861 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:14:07.935 INFO:journalctl@ceph.mds.b.smithi170.stdout:Apr 25 00:14:07 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-b[62656]: 2024-04-25T00:14:07.669+0000 7febf57a6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:14:07.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:07 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2106240418' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-25T00:14:07.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:07 smithi170 ceph-mon[32807]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T00:14:07.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:07 smithi170 ceph-mon[32807]: pgmap v293: 129 pgs: 3 creating+peering, 29 unknown, 97 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-04-25T00:14:07.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:07 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3156202237' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-25T00:14:07.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.960+0000 7fafb524d640 1 -- 172.21.15.107:0/766433003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb0100700 msgr2=0x7fafb0100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:07.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.960+0000 7fafb524d640 1 --2- 172.21.15.107:0/766433003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb0100700 0x7fafb0100b20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fafa0009900 tx=0x7fafa002a410 comp rx=0 tx=0).stop 2024-04-25T00:14:07.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.960+0000 7fafb524d640 1 -- 172.21.15.107:0/766433003 shutdown_connections 2024-04-25T00:14:07.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.960+0000 7fafb524d640 1 --2- 172.21.15.107:0/766433003 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafb01022b0 0x7fafb0108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.960+0000 7fafb524d640 1 --2- 172.21.15.107:0/766433003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafb01018f0 0x7fafb0101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.960+0000 7fafb524d640 1 --2- 172.21.15.107:0/766433003 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb0100700 0x7fafb0100b20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.960+0000 7fafb524d640 1 -- 172.21.15.107:0/766433003 >> 172.21.15.107:0/766433003 conn(0x7fafb00fbc90 msgr2=0x7fafb00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:07.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.960+0000 7fafb524d640 1 -- 172.21.15.107:0/766433003 shutdown_connections 2024-04-25T00:14:07.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.960+0000 7fafb524d640 1 -- 172.21.15.107:0/766433003 wait complete. 2024-04-25T00:14:07.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.960+0000 7fafb524d640 1 Processor -- start 2024-04-25T00:14:07.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.961+0000 7fafb524d640 1 -- start start 2024-04-25T00:14:07.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.961+0000 7fafb524d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb0100700 0x7fafb006a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:07.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.961+0000 7fafb524d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafb01018f0 0x7fafb019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:07.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.961+0000 7fafb524d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafb01022b0 0x7fafb019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:07.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.961+0000 7fafb524d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fafb010c430 con 0x7fafb0100700 2024-04-25T00:14:07.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.961+0000 7fafb524d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fafb010c2b0 con 0x7fafb01022b0 2024-04-25T00:14:07.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.961+0000 7fafb524d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fafb010c5b0 con 0x7fafb01018f0 2024-04-25T00:14:07.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.961+0000 7fafaed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb0100700 0x7fafb006a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:07.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.961+0000 7fafaed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb0100700 0x7fafb006a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59522/0 (socket says 172.21.15.107:59522) 2024-04-25T00:14:07.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.961+0000 7fafaed9d640 1 -- 172.21.15.107:0/2946935920 learned_addr learned my addr 172.21.15.107:0/2946935920 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:07.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.961+0000 7fafaf59e640 1 --2- 172.21.15.107:0/2946935920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafb01022b0 0x7fafb019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:07.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.961+0000 7fafa7fff640 1 --2- 172.21.15.107:0/2946935920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafb01018f0 0x7fafb019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:07.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.961+0000 7fafaed9d640 1 -- 172.21.15.107:0/2946935920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafb01018f0 msgr2=0x7fafb019c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:07.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.961+0000 7fafaed9d640 1 --2- 172.21.15.107:0/2946935920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafb01018f0 0x7fafb019c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.961+0000 7fafaed9d640 1 -- 172.21.15.107:0/2946935920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafb01022b0 msgr2=0x7fafb019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:07.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.961+0000 7fafaed9d640 1 --2- 172.21.15.107:0/2946935920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafb01022b0 0x7fafb019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:07.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.961+0000 7fafaed9d640 1 -- 172.21.15.107:0/2946935920 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb019f860 con 0x7fafb0100700 2024-04-25T00:14:07.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.961+0000 7fafaed9d640 1 --2- 172.21.15.107:0/2946935920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb0100700 0x7fafb006a990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fafa002aa00 tx=0x7fafa0032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:07.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.961+0000 7fafacd99640 1 -- 172.21.15.107:0/2946935920 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa003f030 con 0x7fafb0100700 2024-04-25T00:14:07.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.961+0000 7fafacd99640 1 -- 172.21.15.107:0/2946935920 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafa0003ce0 con 0x7fafb0100700 2024-04-25T00:14:07.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.961+0000 7fafb524d640 1 -- 172.21.15.107:0/2946935920 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb019faf0 con 0x7fafb0100700 2024-04-25T00:14:07.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.961+0000 7fafacd99640 1 -- 172.21.15.107:0/2946935920 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa0035070 con 0x7fafb0100700 2024-04-25T00:14:07.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.961+0000 7fafb524d640 1 -- 172.21.15.107:0/2946935920 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb019ffe0 con 0x7fafb0100700 2024-04-25T00:14:07.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.962+0000 7fafb524d640 1 -- 172.21.15.107:0/2946935920 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb0107a70 con 0x7fafb0100700 2024-04-25T00:14:07.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.963+0000 7fafacd99640 1 -- 172.21.15.107:0/2946935920 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fafa0042030 con 0x7fafb0100700 2024-04-25T00:14:07.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.964+0000 7fafacd99640 1 --2- 172.21.15.107:0/2946935920 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7faf9c0a2b90 0x7faf9c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:07.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.964+0000 7fafa7fff640 1 --2- 172.21.15.107:0/2946935920 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7faf9c0a2b90 0x7faf9c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:07.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.964+0000 7fafacd99640 1 -- 172.21.15.107:0/2946935920 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7fafa00efa80 con 0x7fafb0100700 2024-04-25T00:14:07.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.965+0000 7fafa7fff640 1 --2- 172.21.15.107:0/2946935920 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7faf9c0a2b90 0x7faf9c0a5050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fafb006a700 tx=0x7faf98008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:07.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:07.969+0000 7fafacd99640 1 -- 172.21.15.107:0/2946935920 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fafa00bc7a0 con 0x7fafb0100700 2024-04-25T00:14:08.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.179+0000 7fafb524d640 1 -- 172.21.15.107:0/2946935920 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafb0062370 con 0x7fafb0100700 2024-04-25T00:14:08.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.180+0000 7fafacd99640 1 -- 172.21.15.107:0/2946935920 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11824 (secure 0 0 0) 0x7fafa00bc0a0 con 0x7fafb0100700 2024-04-25T00:14:08.180 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:14:08.180 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:14:07.188031+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24521,"qdb_cluster":[24521]},"id":1}]} 2024-04-25T00:14:08.181 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 17 2024-04-25T00:14:08.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.182+0000 7fafb524d640 1 -- 172.21.15.107:0/2946935920 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7faf9c0a2b90 msgr2=0x7faf9c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.182+0000 7fafb524d640 1 --2- 172.21.15.107:0/2946935920 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7faf9c0a2b90 0x7faf9c0a5050 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fafb006a700 tx=0x7faf98008040 comp rx=0 tx=0).stop 2024-04-25T00:14:08.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.182+0000 7fafb524d640 1 -- 172.21.15.107:0/2946935920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb0100700 msgr2=0x7fafb006a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.182+0000 7fafb524d640 1 --2- 172.21.15.107:0/2946935920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb0100700 0x7fafb006a990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fafa002aa00 tx=0x7fafa0032da0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.182+0000 7fafb524d640 1 -- 172.21.15.107:0/2946935920 shutdown_connections 2024-04-25T00:14:08.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.182+0000 7fafb524d640 1 --2- 172.21.15.107:0/2946935920 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7faf9c0a2b90 0x7faf9c0a5050 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.182+0000 7fafb524d640 1 --2- 172.21.15.107:0/2946935920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fafb01022b0 0x7fafb019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.182+0000 7fafb524d640 1 --2- 172.21.15.107:0/2946935920 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fafb01018f0 0x7fafb019c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.182+0000 7fafb524d640 1 --2- 172.21.15.107:0/2946935920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb0100700 0x7fafb006a990 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.182+0000 7fafb524d640 1 -- 172.21.15.107:0/2946935920 >> 172.21.15.107:0/2946935920 conn(0x7fafb00fbc90 msgr2=0x7fafb0104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:08.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.182+0000 7fafb524d640 1 -- 172.21.15.107:0/2946935920 shutdown_connections 2024-04-25T00:14:08.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.182+0000 7fafb524d640 1 -- 172.21.15.107:0/2946935920 wait complete. 2024-04-25T00:14:08.192 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:07 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2106240418' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-25T00:14:08.192 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:07 smithi107 ceph-mon[29153]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T00:14:08.192 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:07 smithi107 ceph-mon[29153]: pgmap v293: 129 pgs: 3 creating+peering, 29 unknown, 97 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-04-25T00:14:08.192 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:07 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3156202237' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-25T00:14:08.194 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:08.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.290+0000 7f96ceb10640 1 -- 172.21.15.107:0/164825958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96c8102980 msgr2=0x7f96c8102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.290+0000 7f96ceb10640 1 --2- 172.21.15.107:0/164825958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96c8102980 0x7f96c8102e00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009900 tx=0x7f96b402a410 comp rx=0 tx=0).stop 2024-04-25T00:14:08.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.290+0000 7f96ceb10640 1 -- 172.21.15.107:0/164825958 shutdown_connections 2024-04-25T00:14:08.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.290+0000 7f96ceb10640 1 --2- 172.21.15.107:0/164825958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96c8103340 0x7f96c8109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.290+0000 7f96ceb10640 1 --2- 172.21.15.107:0/164825958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96c8102980 0x7f96c8102e00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.290+0000 7f96ceb10640 1 --2- 172.21.15.107:0/164825958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96c8101790 0x7f96c8101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.290+0000 7f96ceb10640 1 -- 172.21.15.107:0/164825958 >> 172.21.15.107:0/164825958 conn(0x7f96c80fcd20 msgr2=0x7f96c80ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:08.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.290+0000 7f96ceb10640 1 -- 172.21.15.107:0/164825958 shutdown_connections 2024-04-25T00:14:08.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.290+0000 7f96ceb10640 1 -- 172.21.15.107:0/164825958 wait complete. 2024-04-25T00:14:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.290+0000 7f96ceb10640 1 Processor -- start 2024-04-25T00:14:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.290+0000 7f96ceb10640 1 -- start start 2024-04-25T00:14:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.291+0000 7f96ceb10640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96c8101790 0x7f96c8197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.291+0000 7f96ceb10640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96c8102980 0x7f96c8197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.291+0000 7f96ceb10640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96c8103340 0x7f96c819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.291+0000 7f96ceb10640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f96c810d410 con 0x7f96c8103340 2024-04-25T00:14:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.291+0000 7f96ceb10640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f96c810d290 con 0x7f96c8101790 2024-04-25T00:14:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.291+0000 7f96ceb10640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f96c810d590 con 0x7f96c8102980 2024-04-25T00:14:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.291+0000 7f96cd0ad640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96c8103340 0x7f96c819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.291+0000 7f96cd0ad640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96c8103340 0x7f96c819f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59542/0 (socket says 172.21.15.107:59542) 2024-04-25T00:14:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.291+0000 7f96cd0ad640 1 -- 172.21.15.107:0/3268943654 learned_addr learned my addr 172.21.15.107:0/3268943654 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:08.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.291+0000 7f96cc8ac640 1 --2- 172.21.15.107:0/3268943654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96c8101790 0x7f96c8197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:08.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.291+0000 7f96c7fff640 1 --2- 172.21.15.107:0/3268943654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96c8102980 0x7f96c8197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:08.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.291+0000 7f96cd0ad640 1 -- 172.21.15.107:0/3268943654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96c8102980 msgr2=0x7f96c8197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.291+0000 7f96cd0ad640 1 --2- 172.21.15.107:0/3268943654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96c8102980 0x7f96c8197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.291+0000 7f96cd0ad640 1 -- 172.21.15.107:0/3268943654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96c8101790 msgr2=0x7f96c8197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.291+0000 7f96cd0ad640 1 --2- 172.21.15.107:0/3268943654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96c8101790 0x7f96c8197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.291+0000 7f96cd0ad640 1 -- 172.21.15.107:0/3268943654 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c819f6d0 con 0x7f96c8103340 2024-04-25T00:14:08.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.291+0000 7f96cc8ac640 1 --2- 172.21.15.107:0/3268943654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96c8101790 0x7f96c8197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:08.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.291+0000 7f96cd0ad640 1 --2- 172.21.15.107:0/3268943654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96c8103340 0x7f96c819f060 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f96c000eaa0 tx=0x7f96c000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:08.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.291+0000 7f96c5ffb640 1 -- 172.21.15.107:0/3268943654 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c000cdb0 con 0x7f96c8103340 2024-04-25T00:14:08.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.291+0000 7f96ceb10640 1 -- 172.21.15.107:0/3268943654 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c819f9c0 con 0x7f96c8103340 2024-04-25T00:14:08.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.291+0000 7f96c5ffb640 1 -- 172.21.15.107:0/3268943654 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96c0005020 con 0x7f96c8103340 2024-04-25T00:14:08.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.291+0000 7f96c5ffb640 1 -- 172.21.15.107:0/3268943654 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c00116b0 con 0x7f96c8103340 2024-04-25T00:14:08.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.291+0000 7f96ceb10640 1 -- 172.21.15.107:0/3268943654 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c819fec0 con 0x7f96c8103340 2024-04-25T00:14:08.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.293+0000 7f96ceb10640 1 -- 172.21.15.107:0/3268943654 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c8108ab0 con 0x7f96c8103340 2024-04-25T00:14:08.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.293+0000 7f96c5ffb640 1 -- 172.21.15.107:0/3268943654 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f96c0005560 con 0x7f96c8103340 2024-04-25T00:14:08.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.294+0000 7f96c5ffb640 1 --2- 172.21.15.107:0/3268943654 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f96b80a2b90 0x7f96b80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:08.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.294+0000 7f96cc8ac640 1 --2- 172.21.15.107:0/3268943654 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f96b80a2b90 0x7f96b80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:08.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.294+0000 7f96c5ffb640 1 -- 172.21.15.107:0/3268943654 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f96c00cf5b0 con 0x7f96c8103340 2024-04-25T00:14:08.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.295+0000 7f96cc8ac640 1 --2- 172.21.15.107:0/3268943654 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f96b80a2b90 0x7f96b80a5050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f96bc0058c0 tx=0x7f96bc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:08.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.299+0000 7f96c5ffb640 1 -- 172.21.15.107:0/3268943654 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f96c0016030 con 0x7f96c8103340 2024-04-25T00:14:08.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:07 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2106240418' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-25T00:14:08.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:07 smithi195 ceph-mon[32833]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T00:14:08.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:07 smithi195 ceph-mon[32833]: pgmap v293: 129 pgs: 3 creating+peering, 29 unknown, 97 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2024-04-25T00:14:08.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:07 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3156202237' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-25T00:14:08.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.475+0000 7f96ceb10640 1 -- 172.21.15.107:0/3268943654 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96c8062340 con 0x7f96c8103340 2024-04-25T00:14:08.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.477+0000 7f96c5ffb640 1 -- 172.21.15.107:0/3268943654 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24147 (secure 0 0 0) 0x7f96c009c2d0 con 0x7f96c8103340 2024-04-25T00:14:08.477 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:14:08.478 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":91,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:14:07.187816+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:59.672801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:00.957782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:14:05.065846+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:08.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.479+0000 7f96ceb10640 1 -- 172.21.15.107:0/3268943654 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f96b80a2b90 msgr2=0x7f96b80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.479+0000 7f96ceb10640 1 --2- 172.21.15.107:0/3268943654 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f96b80a2b90 0x7f96b80a5050 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f96bc0058c0 tx=0x7f96bc005850 comp rx=0 tx=0).stop 2024-04-25T00:14:08.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.479+0000 7f96ceb10640 1 -- 172.21.15.107:0/3268943654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96c8103340 msgr2=0x7f96c819f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.479+0000 7f96ceb10640 1 --2- 172.21.15.107:0/3268943654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96c8103340 0x7f96c819f060 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f96c000eaa0 tx=0x7f96c000ef60 comp rx=0 tx=0).stop 2024-04-25T00:14:08.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.479+0000 7f96ceb10640 1 -- 172.21.15.107:0/3268943654 shutdown_connections 2024-04-25T00:14:08.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.479+0000 7f96ceb10640 1 --2- 172.21.15.107:0/3268943654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96c8103340 0x7f96c819f060 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.479+0000 7f96ceb10640 1 --2- 172.21.15.107:0/3268943654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f96c8102980 0x7f96c8197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.479+0000 7f96ceb10640 1 --2- 172.21.15.107:0/3268943654 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f96b80a2b90 0x7f96b80a5050 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.479+0000 7f96ceb10640 1 --2- 172.21.15.107:0/3268943654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f96c8101790 0x7f96c8197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.479+0000 7f96ceb10640 1 -- 172.21.15.107:0/3268943654 >> 172.21.15.107:0/3268943654 conn(0x7f96c80fcd20 msgr2=0x7f96c8105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:08.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.479+0000 7f96ceb10640 1 -- 172.21.15.107:0/3268943654 shutdown_connections 2024-04-25T00:14:08.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.479+0000 7f96ceb10640 1 -- 172.21.15.107:0/3268943654 wait complete. 2024-04-25T00:14:08.491 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:08.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.589+0000 7f48dbc9e640 1 -- 172.21.15.107:0/2408409168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48d4102980 msgr2=0x7f48d4102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.589+0000 7f48dbc9e640 1 --2- 172.21.15.107:0/2408409168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48d4102980 0x7f48d4102e00 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f48c4009900 tx=0x7f48c402a410 comp rx=0 tx=0).stop 2024-04-25T00:14:08.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.589+0000 7f48dbc9e640 1 -- 172.21.15.107:0/2408409168 shutdown_connections 2024-04-25T00:14:08.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.589+0000 7f48dbc9e640 1 --2- 172.21.15.107:0/2408409168 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48d4103340 0x7f48d4109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.589+0000 7f48dbc9e640 1 --2- 172.21.15.107:0/2408409168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48d4102980 0x7f48d4102e00 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.589+0000 7f48dbc9e640 1 --2- 172.21.15.107:0/2408409168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d4101790 0x7f48d4101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.589+0000 7f48dbc9e640 1 -- 172.21.15.107:0/2408409168 >> 172.21.15.107:0/2408409168 conn(0x7f48d40fcd20 msgr2=0x7f48d40ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:08.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.589+0000 7f48dbc9e640 1 -- 172.21.15.107:0/2408409168 shutdown_connections 2024-04-25T00:14:08.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.589+0000 7f48dbc9e640 1 -- 172.21.15.107:0/2408409168 wait complete. 2024-04-25T00:14:08.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.589+0000 7f48dbc9e640 1 Processor -- start 2024-04-25T00:14:08.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.589+0000 7f48dbc9e640 1 -- start start 2024-04-25T00:14:08.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.589+0000 7f48dbc9e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48d4101790 0x7f48d4197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:08.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.589+0000 7f48dbc9e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48d4102980 0x7f48d4197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:08.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.589+0000 7f48dbc9e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d4103340 0x7f48d419f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:08.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.590+0000 7f48dbc9e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f48d410d410 con 0x7f48d4101790 2024-04-25T00:14:08.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.590+0000 7f48dbc9e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f48d410d290 con 0x7f48d4103340 2024-04-25T00:14:08.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.590+0000 7f48dbc9e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f48d410d590 con 0x7f48d4102980 2024-04-25T00:14:08.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.590+0000 7f48d9a3a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48d4101790 0x7f48d4197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:08.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.590+0000 7f48d9a3a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48d4101790 0x7f48d4197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59564/0 (socket says 172.21.15.107:59564) 2024-04-25T00:14:08.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.590+0000 7f48d9a3a640 1 -- 172.21.15.107:0/2308275747 learned_addr learned my addr 172.21.15.107:0/2308275747 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:08.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.590+0000 7f48da23b640 1 --2- 172.21.15.107:0/2308275747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d4103340 0x7f48d419f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:08.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.590+0000 7f48d9239640 1 --2- 172.21.15.107:0/2308275747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48d4102980 0x7f48d4197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:08.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.590+0000 7f48d9a3a640 1 -- 172.21.15.107:0/2308275747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48d4102980 msgr2=0x7f48d4197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.590+0000 7f48d9a3a640 1 --2- 172.21.15.107:0/2308275747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48d4102980 0x7f48d4197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.590+0000 7f48d9a3a640 1 -- 172.21.15.107:0/2308275747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d4103340 msgr2=0x7f48d419f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.590+0000 7f48d9a3a640 1 --2- 172.21.15.107:0/2308275747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d4103340 0x7f48d419f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.590+0000 7f48d9a3a640 1 -- 172.21.15.107:0/2308275747 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d419f6b0 con 0x7f48d4101790 2024-04-25T00:14:08.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.590+0000 7f48da23b640 1 --2- 172.21.15.107:0/2308275747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d4103340 0x7f48d419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:08.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.590+0000 7f48d9239640 1 --2- 172.21.15.107:0/2308275747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48d4102980 0x7f48d4197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:08.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.590+0000 7f48d9a3a640 1 --2- 172.21.15.107:0/2308275747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48d4101790 0x7f48d4197a90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f48cc00ebc0 tx=0x7f48cc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:08.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.590+0000 7f48caffd640 1 -- 172.21.15.107:0/2308275747 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc003b70 con 0x7f48d4101790 2024-04-25T00:14:08.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.590+0000 7f48caffd640 1 -- 172.21.15.107:0/2308275747 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48cc003d10 con 0x7f48d4101790 2024-04-25T00:14:08.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.590+0000 7f48caffd640 1 -- 172.21.15.107:0/2308275747 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48cc019980 con 0x7f48d4101790 2024-04-25T00:14:08.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.590+0000 7f48dbc9e640 1 -- 172.21.15.107:0/2308275747 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d419f9a0 con 0x7f48d4101790 2024-04-25T00:14:08.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.590+0000 7f48dbc9e640 1 -- 172.21.15.107:0/2308275747 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d419fef0 con 0x7f48d4101790 2024-04-25T00:14:08.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.591+0000 7f48dbc9e640 1 -- 172.21.15.107:0/2308275747 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48d40689b0 con 0x7f48d4101790 2024-04-25T00:14:08.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.592+0000 7f48caffd640 1 -- 172.21.15.107:0/2308275747 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f48cc019b20 con 0x7f48d4101790 2024-04-25T00:14:08.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.593+0000 7f48caffd640 1 --2- 172.21.15.107:0/2308275747 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f48c00a2b90 0x7f48c00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:08.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.593+0000 7f48d9239640 1 --2- 172.21.15.107:0/2308275747 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f48c00a2b90 0x7f48c00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:08.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.593+0000 7f48caffd640 1 -- 172.21.15.107:0/2308275747 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8802+0+0 (secure 0 0 0) 0x7f48cc0cfbf0 con 0x7f48d4101790 2024-04-25T00:14:08.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.593+0000 7f48d9239640 1 --2- 172.21.15.107:0/2308275747 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f48c00a2b90 0x7f48c00a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f48c4000c00 tx=0x7f48c4031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:08.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.598+0000 7f48caffd640 1 -- 172.21.15.107:0/2308275747 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f48cc09c910 con 0x7f48d4101790 2024-04-25T00:14:08.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.774+0000 7f48dbc9e640 1 -- 172.21.15.107:0/2308275747 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48d4062370 con 0x7f48d4101790 2024-04-25T00:14:08.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.775+0000 7f48caffd640 1 -- 172.21.15.107:0/2308275747 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24147 (secure 0 0 0) 0x7f48cc09c210 con 0x7f48d4101790 2024-04-25T00:14:08.776 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:14:08.777 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":91,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:14:07.187816+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:59.672801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:00.957782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:14:05.065846+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:08.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.777+0000 7f48dbc9e640 1 -- 172.21.15.107:0/2308275747 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f48c00a2b90 msgr2=0x7f48c00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.777+0000 7f48dbc9e640 1 --2- 172.21.15.107:0/2308275747 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f48c00a2b90 0x7f48c00a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f48c4000c00 tx=0x7f48c4031400 comp rx=0 tx=0).stop 2024-04-25T00:14:08.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.777+0000 7f48dbc9e640 1 -- 172.21.15.107:0/2308275747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48d4101790 msgr2=0x7f48d4197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.777+0000 7f48dbc9e640 1 --2- 172.21.15.107:0/2308275747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48d4101790 0x7f48d4197a90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f48cc00ebc0 tx=0x7f48cc00c6a0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.777+0000 7f48dbc9e640 1 -- 172.21.15.107:0/2308275747 shutdown_connections 2024-04-25T00:14:08.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.777+0000 7f48dbc9e640 1 --2- 172.21.15.107:0/2308275747 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f48c00a2b90 0x7f48c00a5050 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.777+0000 7f48dbc9e640 1 --2- 172.21.15.107:0/2308275747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48d4103340 0x7f48d419f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.777+0000 7f48dbc9e640 1 --2- 172.21.15.107:0/2308275747 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f48d4102980 0x7f48d4197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.777+0000 7f48dbc9e640 1 --2- 172.21.15.107:0/2308275747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48d4101790 0x7f48d4197a90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.777+0000 7f48dbc9e640 1 -- 172.21.15.107:0/2308275747 >> 172.21.15.107:0/2308275747 conn(0x7f48d40fcd20 msgr2=0x7f48d4105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:08.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.777+0000 7f48dbc9e640 1 -- 172.21.15.107:0/2308275747 shutdown_connections 2024-04-25T00:14:08.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.777+0000 7f48dbc9e640 1 -- 172.21.15.107:0/2308275747 wait complete. 2024-04-25T00:14:08.814 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:14:08.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.913+0000 7fcafd310640 1 -- 172.21.15.107:0/2188178579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaf8101790 msgr2=0x7fcaf8101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.913+0000 7fcafd310640 1 --2- 172.21.15.107:0/2188178579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaf8101790 0x7fcaf8101bb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009900 tx=0x7fcae802a410 comp rx=0 tx=0).stop 2024-04-25T00:14:08.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.913+0000 7fcafd310640 1 -- 172.21.15.107:0/2188178579 shutdown_connections 2024-04-25T00:14:08.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.913+0000 7fcafd310640 1 --2- 172.21.15.107:0/2188178579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaf8103340 0x7fcaf8109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.913+0000 7fcafd310640 1 --2- 172.21.15.107:0/2188178579 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcaf8102980 0x7fcaf8102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.913+0000 7fcafd310640 1 --2- 172.21.15.107:0/2188178579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaf8101790 0x7fcaf8101bb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.913+0000 7fcafd310640 1 -- 172.21.15.107:0/2188178579 >> 172.21.15.107:0/2188178579 conn(0x7fcaf80fcd40 msgr2=0x7fcaf80ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:08.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.913+0000 7fcafd310640 1 -- 172.21.15.107:0/2188178579 shutdown_connections 2024-04-25T00:14:08.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.913+0000 7fcafd310640 1 -- 172.21.15.107:0/2188178579 wait complete. 2024-04-25T00:14:08.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.913+0000 7fcafd310640 1 Processor -- start 2024-04-25T00:14:08.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.914+0000 7fcafd310640 1 -- start start 2024-04-25T00:14:08.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.914+0000 7fcafd310640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaf8101790 0x7fcaf8197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:08.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.914+0000 7fcafd310640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaf8102980 0x7fcaf8197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:08.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.914+0000 7fcafd310640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcaf8103340 0x7fcaf819f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:08.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.914+0000 7fcafd310640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf810d410 con 0x7fcaf8101790 2024-04-25T00:14:08.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.914+0000 7fcafd310640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf810d290 con 0x7fcaf8103340 2024-04-25T00:14:08.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.914+0000 7fcafd310640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf810d590 con 0x7fcaf8102980 2024-04-25T00:14:08.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.914+0000 7fcaf6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaf8101790 0x7fcaf8197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:08.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.914+0000 7fcaf6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaf8101790 0x7fcaf8197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59584/0 (socket says 172.21.15.107:59584) 2024-04-25T00:14:08.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.914+0000 7fcaf6ffd640 1 -- 172.21.15.107:0/3590448859 learned_addr learned my addr 172.21.15.107:0/3590448859 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:08.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.914+0000 7fcaf77fe640 1 --2- 172.21.15.107:0/3590448859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcaf8103340 0x7fcaf819f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:08.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.914+0000 7fcaf6ffd640 1 -- 172.21.15.107:0/3590448859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaf8102980 msgr2=0x7fcaf8197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.914+0000 7fcaeffff640 1 --2- 172.21.15.107:0/3590448859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaf8102980 0x7fcaf8197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:08.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.914+0000 7fcaf6ffd640 1 --2- 172.21.15.107:0/3590448859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaf8102980 0x7fcaf8197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.914+0000 7fcaf6ffd640 1 -- 172.21.15.107:0/3590448859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcaf8103340 msgr2=0x7fcaf819f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:08.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.914+0000 7fcaf6ffd640 1 --2- 172.21.15.107:0/3590448859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcaf8103340 0x7fcaf819f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:08.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.914+0000 7fcaf6ffd640 1 -- 172.21.15.107:0/3590448859 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf819f6b0 con 0x7fcaf8101790 2024-04-25T00:14:08.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.914+0000 7fcaf77fe640 1 --2- 172.21.15.107:0/3590448859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcaf8103340 0x7fcaf819f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:08.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.914+0000 7fcaf6ffd640 1 --2- 172.21.15.107:0/3590448859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaf8101790 0x7fcaf8197a90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcae8031610 tx=0x7fcae80043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:08.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.914+0000 7fcaf4ff9640 1 -- 172.21.15.107:0/3590448859 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae803f030 con 0x7fcaf8101790 2024-04-25T00:14:08.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.914+0000 7fcaf4ff9640 1 -- 172.21.15.107:0/3590448859 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcae8003ce0 con 0x7fcaf8101790 2024-04-25T00:14:08.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.914+0000 7fcafd310640 1 -- 172.21.15.107:0/3590448859 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf819f940 con 0x7fcaf8101790 2024-04-25T00:14:08.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.914+0000 7fcaf4ff9640 1 -- 172.21.15.107:0/3590448859 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8035070 con 0x7fcaf8101790 2024-04-25T00:14:08.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.914+0000 7fcafd310640 1 -- 172.21.15.107:0/3590448859 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf819fe80 con 0x7fcaf8101790 2024-04-25T00:14:08.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.915+0000 7fcafd310640 1 -- 172.21.15.107:0/3590448859 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf8108ab0 con 0x7fcaf8101790 2024-04-25T00:14:08.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.916+0000 7fcaf4ff9640 1 -- 172.21.15.107:0/3590448859 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fcae8042030 con 0x7fcaf8101790 2024-04-25T00:14:08.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.917+0000 7fcaf4ff9640 1 --2- 172.21.15.107:0/3590448859 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcae40a2c10 0x7fcae40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:08.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.917+0000 7fcaeffff640 1 --2- 172.21.15.107:0/3590448859 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcae40a2c10 0x7fcae40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:08.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.917+0000 7fcaf4ff9640 1 -- 172.21.15.107:0/3590448859 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fcae80ef880 con 0x7fcaf8101790 2024-04-25T00:14:08.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.917+0000 7fcaeffff640 1 --2- 172.21.15.107:0/3590448859 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcae40a2c10 0x7fcae40a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcae0005fd0 tx=0x7fcae0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:08.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:08.921+0000 7fcaf4ff9640 1 -- 172.21.15.107:0/3590448859 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcae80bc5a0 con 0x7fcaf8101790 2024-04-25T00:14:09.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.135+0000 7fcafd310640 1 -- 172.21.15.107:0/3590448859 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcaf802d170 con 0x7fcaf8101790 2024-04-25T00:14:09.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.136+0000 7fcaf4ff9640 1 -- 172.21.15.107:0/3590448859 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11824 (secure 0 0 0) 0x7fcae80bbea0 con 0x7fcaf8101790 2024-04-25T00:14:09.136 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:14:09.136 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24463,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:14:07.188031+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24521,"qdb_cluster":[24521]},"id":1}]} 2024-04-25T00:14:09.137 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 17 2024-04-25T00:14:09.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.138+0000 7fcafd310640 1 -- 172.21.15.107:0/3590448859 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcae40a2c10 msgr2=0x7fcae40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.138+0000 7fcafd310640 1 --2- 172.21.15.107:0/3590448859 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcae40a2c10 0x7fcae40a50d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcae0005fd0 tx=0x7fcae0005850 comp rx=0 tx=0).stop 2024-04-25T00:14:09.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.138+0000 7fcafd310640 1 -- 172.21.15.107:0/3590448859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaf8101790 msgr2=0x7fcaf8197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.138+0000 7fcafd310640 1 --2- 172.21.15.107:0/3590448859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaf8101790 0x7fcaf8197a90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcae8031610 tx=0x7fcae80043b0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.138+0000 7fcafd310640 1 -- 172.21.15.107:0/3590448859 shutdown_connections 2024-04-25T00:14:09.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.138+0000 7fcafd310640 1 --2- 172.21.15.107:0/3590448859 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcae40a2c10 0x7fcae40a50d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.138+0000 7fcafd310640 1 --2- 172.21.15.107:0/3590448859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcaf8103340 0x7fcaf819f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.138+0000 7fcafd310640 1 --2- 172.21.15.107:0/3590448859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcaf8102980 0x7fcaf8197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.138+0000 7fcafd310640 1 --2- 172.21.15.107:0/3590448859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcaf8101790 0x7fcaf8197a90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.138+0000 7fcafd310640 1 -- 172.21.15.107:0/3590448859 >> 172.21.15.107:0/3590448859 conn(0x7fcaf80fcd40 msgr2=0x7fcaf8105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:09.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.138+0000 7fcafd310640 1 -- 172.21.15.107:0/3590448859 shutdown_connections 2024-04-25T00:14:09.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.138+0000 7fcafd310640 1 -- 172.21.15.107:0/3590448859 wait complete. 2024-04-25T00:14:09.148 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:08 smithi107 ceph-mon[29153]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T00:14:09.148 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:08 smithi107 ceph-mon[29153]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T00:14:09.148 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:08 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3156202237' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-25T00:14:09.148 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:08 smithi107 ceph-mon[29153]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:14:09.148 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:08 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2946935920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:14:09.149 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:08 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3268943654' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:09.149 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:08 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2308275747' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:09.149 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:08 smithi107 ceph-mon[29153]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T00:14:09.150 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:09.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:08 smithi170 ceph-mon[32807]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T00:14:09.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:08 smithi170 ceph-mon[32807]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T00:14:09.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:08 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3156202237' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-25T00:14:09.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:08 smithi170 ceph-mon[32807]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:14:09.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:08 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2946935920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:14:09.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:08 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3268943654' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:09.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:08 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2308275747' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:09.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:08 smithi170 ceph-mon[32807]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T00:14:09.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.248+0000 7fdf2841c640 1 -- 172.21.15.107:0/2685076858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf20102980 msgr2=0x7fdf20102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.248+0000 7fdf2841c640 1 --2- 172.21.15.107:0/2685076858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf20102980 0x7fdf20102e00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009900 tx=0x7fdf1002a410 comp rx=0 tx=0).stop 2024-04-25T00:14:09.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.248+0000 7fdf2841c640 1 -- 172.21.15.107:0/2685076858 shutdown_connections 2024-04-25T00:14:09.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.248+0000 7fdf2841c640 1 --2- 172.21.15.107:0/2685076858 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf20103340 0x7fdf20109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.248+0000 7fdf2841c640 1 --2- 172.21.15.107:0/2685076858 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf20102980 0x7fdf20102e00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.248+0000 7fdf2841c640 1 --2- 172.21.15.107:0/2685076858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf20101790 0x7fdf20101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.248+0000 7fdf2841c640 1 -- 172.21.15.107:0/2685076858 >> 172.21.15.107:0/2685076858 conn(0x7fdf200fcd20 msgr2=0x7fdf200ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:09.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.248+0000 7fdf2841c640 1 -- 172.21.15.107:0/2685076858 shutdown_connections 2024-04-25T00:14:09.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.248+0000 7fdf2841c640 1 -- 172.21.15.107:0/2685076858 wait complete. 2024-04-25T00:14:09.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.248+0000 7fdf2841c640 1 Processor -- start 2024-04-25T00:14:09.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.249+0000 7fdf2841c640 1 -- start start 2024-04-25T00:14:09.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.249+0000 7fdf2841c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf20101790 0x7fdf20197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.249+0000 7fdf2841c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf20102980 0x7fdf20197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.249+0000 7fdf2841c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf20103340 0x7fdf2019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.249+0000 7fdf2841c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2010d410 con 0x7fdf20103340 2024-04-25T00:14:09.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.249+0000 7fdf2841c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2010d290 con 0x7fdf20101790 2024-04-25T00:14:09.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.249+0000 7fdf2841c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2010d590 con 0x7fdf20102980 2024-04-25T00:14:09.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.249+0000 7fdf269b9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf20103340 0x7fdf2019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.249+0000 7fdf269b9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf20103340 0x7fdf2019f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59606/0 (socket says 172.21.15.107:59606) 2024-04-25T00:14:09.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.249+0000 7fdf269b9640 1 -- 172.21.15.107:0/778838434 learned_addr learned my addr 172.21.15.107:0/778838434 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:09.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.249+0000 7fdf259b7640 1 --2- 172.21.15.107:0/778838434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf20102980 0x7fdf20197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.249+0000 7fdf261b8640 1 --2- 172.21.15.107:0/778838434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf20101790 0x7fdf20197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.249+0000 7fdf269b9640 1 -- 172.21.15.107:0/778838434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf20102980 msgr2=0x7fdf20197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.249+0000 7fdf269b9640 1 --2- 172.21.15.107:0/778838434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf20102980 0x7fdf20197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.249+0000 7fdf269b9640 1 -- 172.21.15.107:0/778838434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf20101790 msgr2=0x7fdf20197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.249+0000 7fdf269b9640 1 --2- 172.21.15.107:0/778838434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf20101790 0x7fdf20197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.249+0000 7fdf269b9640 1 -- 172.21.15.107:0/778838434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2019f6b0 con 0x7fdf20103340 2024-04-25T00:14:09.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.249+0000 7fdf269b9640 1 --2- 172.21.15.107:0/778838434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf20103340 0x7fdf2019f040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800eaa0 tx=0x7fdf1800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:09.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.249+0000 7fdf177fe640 1 -- 172.21.15.107:0/778838434 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1800cdb0 con 0x7fdf20103340 2024-04-25T00:14:09.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.249+0000 7fdf2841c640 1 -- 172.21.15.107:0/778838434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf2019f9a0 con 0x7fdf20103340 2024-04-25T00:14:09.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.249+0000 7fdf177fe640 1 -- 172.21.15.107:0/778838434 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf18005020 con 0x7fdf20103340 2024-04-25T00:14:09.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.249+0000 7fdf177fe640 1 -- 172.21.15.107:0/778838434 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf180116d0 con 0x7fdf20103340 2024-04-25T00:14:09.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.249+0000 7fdf2841c640 1 -- 172.21.15.107:0/778838434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf2019ff10 con 0x7fdf20103340 2024-04-25T00:14:09.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.250+0000 7fdf2841c640 1 -- 172.21.15.107:0/778838434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf20108ab0 con 0x7fdf20103340 2024-04-25T00:14:09.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.252+0000 7fdf177fe640 1 -- 172.21.15.107:0/778838434 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fdf180054d0 con 0x7fdf20103340 2024-04-25T00:14:09.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.252+0000 7fdf177fe640 1 --2- 172.21.15.107:0/778838434 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fdf0c0a2b90 0x7fdf0c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.252+0000 7fdf261b8640 1 --2- 172.21.15.107:0/778838434 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fdf0c0a2b90 0x7fdf0c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.252+0000 7fdf177fe640 1 -- 172.21.15.107:0/778838434 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fdf180cf7a0 con 0x7fdf20103340 2024-04-25T00:14:09.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.252+0000 7fdf261b8640 1 --2- 172.21.15.107:0/778838434 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fdf0c0a2b90 0x7fdf0c0a5050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdf201027e0 tx=0x7fdf1c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:09.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.256+0000 7fdf177fe640 1 -- 172.21.15.107:0/778838434 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdf18016030 con 0x7fdf20103340 2024-04-25T00:14:09.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:08 smithi195 ceph-mon[32833]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T00:14:09.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:08 smithi195 ceph-mon[32833]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T00:14:09.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:08 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3156202237' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-25T00:14:09.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:08 smithi195 ceph-mon[32833]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:14:09.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:08 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2946935920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:14:09.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:08 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3268943654' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:09.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:08 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2308275747' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:09.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:08 smithi195 ceph-mon[32833]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T00:14:09.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.438+0000 7fdf2841c640 1 -- 172.21.15.107:0/778838434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf2002d170 con 0x7fdf20103340 2024-04-25T00:14:09.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.440+0000 7fdf177fe640 1 -- 172.21.15.107:0/778838434 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24134 (secure 0 0 0) 0x7fdf1809be70 con 0x7fdf20103340 2024-04-25T00:14:09.440 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:14:09.440 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":92,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:14:08.839737+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:59.672801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:00.957782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:14:05.065846+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:09.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.442+0000 7fdf2841c640 1 -- 172.21.15.107:0/778838434 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fdf0c0a2b90 msgr2=0x7fdf0c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.442+0000 7fdf2841c640 1 --2- 172.21.15.107:0/778838434 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fdf0c0a2b90 0x7fdf0c0a5050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdf201027e0 tx=0x7fdf1c00b390 comp rx=0 tx=0).stop 2024-04-25T00:14:09.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.442+0000 7fdf2841c640 1 -- 172.21.15.107:0/778838434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf20103340 msgr2=0x7fdf2019f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.442+0000 7fdf2841c640 1 --2- 172.21.15.107:0/778838434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf20103340 0x7fdf2019f040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800eaa0 tx=0x7fdf1800ef60 comp rx=0 tx=0).stop 2024-04-25T00:14:09.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.442+0000 7fdf2841c640 1 -- 172.21.15.107:0/778838434 shutdown_connections 2024-04-25T00:14:09.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.442+0000 7fdf2841c640 1 --2- 172.21.15.107:0/778838434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf20103340 0x7fdf2019f040 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.442+0000 7fdf2841c640 1 --2- 172.21.15.107:0/778838434 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf20102980 0x7fdf20197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.442+0000 7fdf2841c640 1 --2- 172.21.15.107:0/778838434 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fdf0c0a2b90 0x7fdf0c0a5050 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.442+0000 7fdf2841c640 1 --2- 172.21.15.107:0/778838434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf20101790 0x7fdf20197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.442+0000 7fdf2841c640 1 -- 172.21.15.107:0/778838434 >> 172.21.15.107:0/778838434 conn(0x7fdf200fcd20 msgr2=0x7fdf20105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:09.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.442+0000 7fdf2841c640 1 -- 172.21.15.107:0/778838434 shutdown_connections 2024-04-25T00:14:09.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.442+0000 7fdf2841c640 1 -- 172.21.15.107:0/778838434 wait complete. 2024-04-25T00:14:09.454 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T00:14:01.788955+0000', 'modified': '2024-04-25T00:14:07.188031+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24521}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24521': {'gid': 24521, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.170:6839/3558784302', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 3558784302}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 3558784302}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24521, 'qdb_cluster': [24521]} 2024-04-25T00:14:09.454 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T00:14:09.454 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T00:14:09.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.552+0000 7f08d62a5640 1 -- 172.21.15.107:0/1708547738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08d0101890 msgr2=0x7f08d0101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.552+0000 7f08d62a5640 1 --2- 172.21.15.107:0/1708547738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08d0101890 0x7f08d0101d10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009900 tx=0x7f08c002a410 comp rx=0 tx=0).stop 2024-04-25T00:14:09.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.552+0000 7f08d62a5640 1 -- 172.21.15.107:0/1708547738 shutdown_connections 2024-04-25T00:14:09.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.552+0000 7f08d62a5640 1 --2- 172.21.15.107:0/1708547738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08d0102250 0x7f08d0108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.552+0000 7f08d62a5640 1 --2- 172.21.15.107:0/1708547738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08d0101890 0x7f08d0101d10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.552+0000 7f08d62a5640 1 --2- 172.21.15.107:0/1708547738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08d01006a0 0x7f08d0100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.552+0000 7f08d62a5640 1 -- 172.21.15.107:0/1708547738 >> 172.21.15.107:0/1708547738 conn(0x7f08d00fbc10 msgr2=0x7f08d00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:09.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.552+0000 7f08d62a5640 1 -- 172.21.15.107:0/1708547738 shutdown_connections 2024-04-25T00:14:09.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.552+0000 7f08d62a5640 1 -- 172.21.15.107:0/1708547738 wait complete. 2024-04-25T00:14:09.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.553+0000 7f08d62a5640 1 Processor -- start 2024-04-25T00:14:09.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.553+0000 7f08d62a5640 1 -- start start 2024-04-25T00:14:09.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.553+0000 7f08d62a5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08d01006a0 0x7f08d01969f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.553+0000 7f08d62a5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08d0101890 0x7f08d0196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.553+0000 7f08d62a5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08d0102250 0x7f08d019dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.553+0000 7f08d62a5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f08d010c320 con 0x7f08d0101890 2024-04-25T00:14:09.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.553+0000 7f08d62a5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f08d010c1a0 con 0x7f08d01006a0 2024-04-25T00:14:09.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.553+0000 7f08d62a5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f08d010c4a0 con 0x7f08d0102250 2024-04-25T00:14:09.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.553+0000 7f08cf7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08d0101890 0x7f08d0196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.553+0000 7f08cf7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08d0101890 0x7f08d0196f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46680/0 (socket says 172.21.15.107:46680) 2024-04-25T00:14:09.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.553+0000 7f08cf7fe640 1 -- 172.21.15.107:0/634659206 learned_addr learned my addr 172.21.15.107:0/634659206 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:09.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.553+0000 7f08d4842640 1 --2- 172.21.15.107:0/634659206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08d0102250 0x7f08d019dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.553+0000 7f08cffff640 1 --2- 172.21.15.107:0/634659206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08d01006a0 0x7f08d01969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.553+0000 7f08cf7fe640 1 -- 172.21.15.107:0/634659206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08d0102250 msgr2=0x7f08d019dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.553+0000 7f08cf7fe640 1 --2- 172.21.15.107:0/634659206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08d0102250 0x7f08d019dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.553+0000 7f08cf7fe640 1 -- 172.21.15.107:0/634659206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08d01006a0 msgr2=0x7f08d01969f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.553+0000 7f08cf7fe640 1 --2- 172.21.15.107:0/634659206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08d01006a0 0x7f08d01969f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.553+0000 7f08cf7fe640 1 -- 172.21.15.107:0/634659206 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d019e610 con 0x7f08d0101890 2024-04-25T00:14:09.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.553+0000 7f08d4842640 1 --2- 172.21.15.107:0/634659206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08d0102250 0x7f08d019dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:09.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.553+0000 7f08cffff640 1 --2- 172.21.15.107:0/634659206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08d01006a0 0x7f08d01969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:09.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.553+0000 7f08cf7fe640 1 --2- 172.21.15.107:0/634659206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08d0101890 0x7f08d0196f30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009230 tx=0x7f08c0004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:09.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.554+0000 7f08cd7fa640 1 -- 172.21.15.107:0/634659206 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c003f030 con 0x7f08d0101890 2024-04-25T00:14:09.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.554+0000 7f08cd7fa640 1 -- 172.21.15.107:0/634659206 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08c0003ce0 con 0x7f08d0101890 2024-04-25T00:14:09.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.554+0000 7f08d62a5640 1 -- 172.21.15.107:0/634659206 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d019e8a0 con 0x7f08d0101890 2024-04-25T00:14:09.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.554+0000 7f08cd7fa640 1 -- 172.21.15.107:0/634659206 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c0035070 con 0x7f08d0101890 2024-04-25T00:14:09.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.554+0000 7f08d62a5640 1 -- 172.21.15.107:0/634659206 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d019ede0 con 0x7f08d0101890 2024-04-25T00:14:09.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.556+0000 7f08cd7fa640 1 -- 172.21.15.107:0/634659206 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f08c0042030 con 0x7f08d0101890 2024-04-25T00:14:09.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.556+0000 7f08d8fb3740 1 Processor -- start 2024-04-25T00:14:09.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.556+0000 7f08d8fb3740 1 -- start start 2024-04-25T00:14:09.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.556+0000 7f08d8fb3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61dcb0 0x562d7b61e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.556+0000 7f08d8fb3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d7b61e6a0 0x562d7b622d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.556+0000 7f08d8fb3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d7b6233b0 0x562d7b625840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.556+0000 7f08d8fb3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562d7b57cf00 con 0x562d7b61dcb0 2024-04-25T00:14:09.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.556+0000 7f08d8fb3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562d7b525750 con 0x562d7b6233b0 2024-04-25T00:14:09.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.556+0000 7f08d8fb3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562d7b578ec0 con 0x562d7b61e6a0 2024-04-25T00:14:09.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.556+0000 7f08cd7fa640 1 --2- 172.21.15.107:0/634659206 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f08b80a2c10 0x7f08b80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.556+0000 7f08d4842640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61dcb0 0x562d7b61e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.556+0000 7f08d4842640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61dcb0 0x562d7b61e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46692/0 (socket says 172.21.15.107:46692) 2024-04-25T00:14:09.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.556+0000 7f08d4842640 1 -- 172.21.15.107:0/1543520014 learned_addr learned my addr 172.21.15.107:0/1543520014 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:09.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.556+0000 7f08d4842640 1 --2- 172.21.15.107:0/634659206 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f08b80a2c10 0x7f08b80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.556+0000 7f08cd7fa640 1 -- 172.21.15.107:0/634659206 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f08c00efcd0 con 0x7f08d0101890 2024-04-25T00:14:09.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.556+0000 7f08cf7fe640 1 --2- 172.21.15.107:0/1543520014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d7b6233b0 0x562d7b625840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.556+0000 7f08cffff640 1 --2- 172.21.15.107:0/1543520014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d7b61e6a0 0x562d7b622d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.557+0000 7f08d4842640 1 -- 172.21.15.107:0/1543520014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d7b61e6a0 msgr2=0x562d7b622d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:09.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.557+0000 7f08d4842640 1 --2- 172.21.15.107:0/1543520014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d7b61e6a0 0x562d7b622d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.557+0000 7f08d4842640 1 -- 172.21.15.107:0/1543520014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d7b6233b0 msgr2=0x562d7b625840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:09.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.557+0000 7f08d4842640 1 --2- 172.21.15.107:0/1543520014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d7b6233b0 0x562d7b625840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.557+0000 7f08d4842640 1 -- 172.21.15.107:0/1543520014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d7b2912d0 con 0x562d7b61dcb0 2024-04-25T00:14:09.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.557+0000 7f08d4842640 1 --2- 172.21.15.107:0/1543520014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61dcb0 0x562d7b61e0d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f08c8014e40 tx=0x7f08c8011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37570cc18ab1707f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:09.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.557+0000 7f08d4842640 1 --2- 172.21.15.107:0/634659206 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f08b80a2c10 0x7f08b80a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009fd0 tx=0x7f08c8016000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:09.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.557+0000 7f08b67fc640 1 -- 172.21.15.107:0/1543520014 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c800cc00 con 0x562d7b61dcb0 2024-04-25T00:14:09.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.557+0000 7f08b67fc640 1 -- 172.21.15.107:0/1543520014 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08c800cda0 con 0x562d7b61dcb0 2024-04-25T00:14:09.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.557+0000 7f08b67fc640 1 -- 172.21.15.107:0/1543520014 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c8005310 con 0x562d7b61dcb0 2024-04-25T00:14:09.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.558+0000 7f08d8fb3740 1 -- 172.21.15.107:0/1543520014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61dcb0 msgr2=0x562d7b61e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.558+0000 7f08d8fb3740 1 --2- 172.21.15.107:0/1543520014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61dcb0 0x562d7b61e0d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f08c8014e40 tx=0x7f08c8011750 comp rx=0 tx=0).stop 2024-04-25T00:14:09.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.558+0000 7f08d8fb3740 1 -- 172.21.15.107:0/1543520014 shutdown_connections 2024-04-25T00:14:09.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.558+0000 7f08d8fb3740 1 --2- 172.21.15.107:0/1543520014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d7b6233b0 0x562d7b625840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.558+0000 7f08d8fb3740 1 --2- 172.21.15.107:0/1543520014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d7b61e6a0 0x562d7b622d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.558+0000 7f08d8fb3740 1 --2- 172.21.15.107:0/1543520014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61dcb0 0x562d7b61e0d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.558+0000 7f08d8fb3740 1 -- 172.21.15.107:0/1543520014 >> 172.21.15.107:0/1543520014 conn(0x562d7b615ff0 msgr2=0x562d7b6145a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:09.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.558+0000 7f08d8fb3740 1 -- 172.21.15.107:0/1543520014 shutdown_connections 2024-04-25T00:14:09.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.558+0000 7f08d8fb3740 1 -- 172.21.15.107:0/1543520014 wait complete. 2024-04-25T00:14:09.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.558+0000 7f08d8fb3740 1 -- start start 2024-04-25T00:14:09.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.558+0000 7f08d8fb3740 1 Processor -- start 2024-04-25T00:14:09.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.558+0000 7f08d8fb3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61dcb0 0x562d7b627d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.558+0000 7f08d8fb3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d7b61e6a0 0x562d7b628290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.558+0000 7f08d8fb3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d7b6233b0 0x562d7b62f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.558+0000 7f08d8fb3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562d7b626300 con 0x562d7b61dcb0 2024-04-25T00:14:09.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.558+0000 7f08d8fb3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562d7b538cf0 con 0x562d7b61e6a0 2024-04-25T00:14:09.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.558+0000 7f08d8fb3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562d7b626480 con 0x562d7b6233b0 2024-04-25T00:14:09.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.558+0000 7f08d8fb3740 10 client.0 _ll_register_callbacks cb 0x562d7b5386e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:14:09.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.558+0000 7f08cffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61dcb0 0x562d7b627d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.559+0000 7f08cffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61dcb0 0x562d7b627d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46706/0 (socket says 172.21.15.107:46706) 2024-04-25T00:14:09.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.559+0000 7f08cffff640 1 -- 172.21.15.107:0/4255757586 learned_addr learned my addr 172.21.15.107:0/4255757586 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:09.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.559+0000 7f08d4842640 1 --2- 172.21.15.107:0/4255757586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d7b6233b0 0x562d7b62f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.559+0000 7f08cffff640 1 -- 172.21.15.107:0/4255757586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d7b6233b0 msgr2=0x562d7b62f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:09.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.559+0000 7f08cffff640 1 --2- 172.21.15.107:0/4255757586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d7b6233b0 0x562d7b62f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.559+0000 7f08cffff640 1 -- 172.21.15.107:0/4255757586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d7b61e6a0 msgr2=0x562d7b628290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:09.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.559+0000 7f08cffff640 1 --2- 172.21.15.107:0/4255757586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d7b61e6a0 0x562d7b628290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.559+0000 7f08cffff640 1 -- 172.21.15.107:0/4255757586 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d7b35b6b0 con 0x562d7b61dcb0 2024-04-25T00:14:09.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.559+0000 7f08cffff640 1 --2- 172.21.15.107:0/4255757586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61dcb0 0x562d7b627d50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f08c400f6c0 tx=0x7f08c400fb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366d19268f68b3c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:09.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.559+0000 7f08af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:14:09.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.559+0000 7f08af7fe640 1 -- 172.21.15.107:0/4255757586 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c4016070 con 0x562d7b61dcb0 2024-04-25T00:14:09.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.559+0000 7f08af7fe640 1 -- 172.21.15.107:0/4255757586 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f08c400baa0 con 0x562d7b61dcb0 2024-04-25T00:14:09.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.559+0000 7f08d8fb3740 1 -- 172.21.15.107:0/4255757586 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08c800cda0 con 0x562d7b61dcb0 2024-04-25T00:14:09.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.559+0000 7f08af7fe640 1 -- 172.21.15.107:0/4255757586 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f08c4025050 con 0x562d7b61dcb0 2024-04-25T00:14:09.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.559+0000 7f08d8fb3740 10 client.14751 fetch_fsmap learned FSMap version 17 2024-04-25T00:14:09.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.559+0000 7f08d8fb3740 1 -- 172.21.15.107:0/4255757586 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x562d7b632310 con 0x562d7b61dcb0 2024-04-25T00:14:09.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.559+0000 7f08af7fe640 1 -- 172.21.15.107:0/4255757586 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08c800cda0 con 0x562d7b61dcb0 2024-04-25T00:14:09.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.560+0000 7f08af7fe640 1 -- 172.21.15.107:0/4255757586 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f08c401a400 con 0x562d7b61dcb0 2024-04-25T00:14:09.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.560+0000 7f08d8fb3740 10 client.14751 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T00:14:09.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.560+0000 7f08d8fb3740 10 client.14751 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:14:09.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.560+0000 7f08d8fb3740 20 client.14751 populate_metadata read hostname 'smithi107' 2024-04-25T00:14:09.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.560+0000 7f08d8fb3740 1 --2- 172.21.15.107:0/4255757586 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562d7b634a90 0x562d7b635f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.560+0000 7f08d8fb3740 4 client.14751 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:14:09.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.560+0000 7f08d8fb3740 1 -- 172.21.15.107:0/4255757586 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d7b58a440 con 0x562d7b634a90 2024-04-25T00:14:09.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.560+0000 7f08cf7fe640 1 --2- 172.21.15.107:0/4255757586 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562d7b634a90 0x562d7b635f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.560+0000 7f08af7fe640 10 client.14751 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:14:09.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.560+0000 7f08cf7fe640 1 --2- 172.21.15.107:0/4255757586 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562d7b634a90 0x562d7b635f00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e89afc930c0ffc39 server_cookie=244720af6249a818 in_seq=0 out_seq=0 2024-04-25T00:14:09.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.561+0000 7f08af7fe640 1 -- 172.21.15.107:0/4255757586 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562d7b58a440 con 0x562d7b634a90 2024-04-25T00:14:09.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.561+0000 7f08af7fe640 10 client.14751 handle_command_reply: tid=0 2024-04-25T00:14:09.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.561+0000 7f08d8fb3740 1 client.14751 shutdown 2024-04-25T00:14:09.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.561+0000 7f08d8fb3740 1 -- 172.21.15.107:0/4255757586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61dcb0 msgr2=0x562d7b627d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.561+0000 7f08d8fb3740 1 --2- 172.21.15.107:0/4255757586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61dcb0 0x562d7b627d50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f08c400f6c0 tx=0x7f08c400fb80 comp rx=0 tx=0).stop 2024-04-25T00:14:09.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.561+0000 7f08d8fb3740 1 -- 172.21.15.107:0/4255757586 shutdown_connections 2024-04-25T00:14:09.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.561+0000 7f08d8fb3740 1 --2- 172.21.15.107:0/4255757586 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562d7b634a90 0x562d7b635f00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.561+0000 7f08d8fb3740 1 --2- 172.21.15.107:0/4255757586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d7b6233b0 0x562d7b62f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.561+0000 7f08af7fe640 0 client.14751 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:14:09.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.561+0000 7f08d8fb3740 1 --2- 172.21.15.107:0/4255757586 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d7b61e6a0 0x562d7b628290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.561+0000 7f08d8fb3740 1 --2- 172.21.15.107:0/4255757586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61dcb0 0x562d7b627d50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.561+0000 7f08d8fb3740 1 -- 172.21.15.107:0/4255757586 >> 172.21.15.107:0/4255757586 conn(0x562d7b615ff0 msgr2=0x562d7b613ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:09.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.561+0000 7f08d8fb3740 1 -- 172.21.15.107:0/4255757586 shutdown_connections 2024-04-25T00:14:09.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.561+0000 7f08d8fb3740 1 -- 172.21.15.107:0/4255757586 wait complete. 2024-04-25T00:14:09.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.561+0000 7f08d8fb3740 20 client.14751 trim_cache size 0 max 16384 2024-04-25T00:14:09.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.579+0000 7f08d8fb3740 1 Processor -- start 2024-04-25T00:14:09.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.579+0000 7f08d8fb3740 1 -- start start 2024-04-25T00:14:09.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.579+0000 7f08d8fb3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d7b61dcb0 0x562d7b613250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.579+0000 7f08d8fb3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61e6a0 0x562d7b62f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.580+0000 7f08d8fb3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d7b6233b0 0x562d7b613790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.580+0000 7f08d8fb3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562d7b634240 con 0x562d7b61e6a0 2024-04-25T00:14:09.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.580+0000 7f08d8fb3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562d7b634910 con 0x562d7b61dcb0 2024-04-25T00:14:09.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.580+0000 7f08d8fb3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562d7b634fe0 con 0x562d7b6233b0 2024-04-25T00:14:09.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.580+0000 7f08cf7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61e6a0 0x562d7b62f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.580+0000 7f08cf7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61e6a0 0x562d7b62f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46712/0 (socket says 172.21.15.107:46712) 2024-04-25T00:14:09.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.580+0000 7f08cf7fe640 1 -- 172.21.15.107:0/1951298683 learned_addr learned my addr 172.21.15.107:0/1951298683 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:09.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.580+0000 7f08cffff640 1 --2- 172.21.15.107:0/1951298683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d7b61dcb0 0x562d7b613250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.580+0000 7f08cf7fe640 1 -- 172.21.15.107:0/1951298683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d7b6233b0 msgr2=0x562d7b613790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:09.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.580+0000 7f08cf7fe640 1 --2- 172.21.15.107:0/1951298683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d7b6233b0 0x562d7b613790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.580+0000 7f08cf7fe640 1 -- 172.21.15.107:0/1951298683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d7b61dcb0 msgr2=0x562d7b613250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:09.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.580+0000 7f08cf7fe640 1 --2- 172.21.15.107:0/1951298683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d7b61dcb0 0x562d7b613250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.580+0000 7f08cf7fe640 1 -- 172.21.15.107:0/1951298683 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d7b6150c0 con 0x562d7b61e6a0 2024-04-25T00:14:09.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.580+0000 7f08cf7fe640 1 --2- 172.21.15.107:0/1951298683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61e6a0 0x562d7b62f220 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009a20 tx=0x7f08c00dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0903c8cd8e959 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:09.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.580+0000 7f08b5ffb640 1 -- 172.21.15.107:0/1951298683 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c0049040 con 0x562d7b61e6a0 2024-04-25T00:14:09.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.580+0000 7f08b5ffb640 1 -- 172.21.15.107:0/1951298683 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08c00c06a0 con 0x562d7b61e6a0 2024-04-25T00:14:09.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.580+0000 7f08b5ffb640 1 -- 172.21.15.107:0/1951298683 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c00b6e60 con 0x562d7b61e6a0 2024-04-25T00:14:09.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.581+0000 7f08d8fb3740 1 -- 172.21.15.107:0/1951298683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61e6a0 msgr2=0x562d7b62f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.581+0000 7f08d8fb3740 1 --2- 172.21.15.107:0/1951298683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61e6a0 0x562d7b62f220 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009a20 tx=0x7f08c00dc040 comp rx=0 tx=0).stop 2024-04-25T00:14:09.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.581+0000 7f08d8fb3740 1 -- 172.21.15.107:0/1951298683 shutdown_connections 2024-04-25T00:14:09.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.581+0000 7f08d8fb3740 1 --2- 172.21.15.107:0/1951298683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d7b6233b0 0x562d7b613790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.581+0000 7f08d8fb3740 1 --2- 172.21.15.107:0/1951298683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61e6a0 0x562d7b62f220 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.581+0000 7f08d8fb3740 1 --2- 172.21.15.107:0/1951298683 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d7b61dcb0 0x562d7b613250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.581+0000 7f08d8fb3740 1 -- 172.21.15.107:0/1951298683 >> 172.21.15.107:0/1951298683 conn(0x562d7b615ff0 msgr2=0x562d7b614510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:09.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.581+0000 7f08d8fb3740 1 -- 172.21.15.107:0/1951298683 shutdown_connections 2024-04-25T00:14:09.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.581+0000 7f08d8fb3740 1 -- 172.21.15.107:0/1951298683 wait complete. 2024-04-25T00:14:09.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.581+0000 7f08d8fb3740 1 -- start start 2024-04-25T00:14:09.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.581+0000 7f08d8fb3740 1 Processor -- start 2024-04-25T00:14:09.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.581+0000 7f08d8fb3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d7b61dcb0 0x562d7b62b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.581+0000 7f08d8fb3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61e6a0 0x562d7b62b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.581+0000 7f08d8fb3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d7b6233b0 0x562d7b62e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.581+0000 7f08d8fb3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562d7b635d80 con 0x562d7b61e6a0 2024-04-25T00:14:09.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.581+0000 7f08d8fb3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562d7b6356b0 con 0x562d7b61dcb0 2024-04-25T00:14:09.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.581+0000 7f08d8fb3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562d7b636450 con 0x562d7b6233b0 2024-04-25T00:14:09.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.581+0000 7f08d8fb3740 10 client.0 _ll_register_callbacks cb 0x7f08c400de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:14:09.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.581+0000 7f08cf7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61e6a0 0x562d7b62b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.582+0000 7f08cf7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61e6a0 0x562d7b62b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46716/0 (socket says 172.21.15.107:46716) 2024-04-25T00:14:09.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.582+0000 7f08cf7fe640 1 -- 172.21.15.107:0/1049012470 learned_addr learned my addr 172.21.15.107:0/1049012470 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:09.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.582+0000 7f08cffff640 1 --2- 172.21.15.107:0/1049012470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d7b61dcb0 0x562d7b62b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.582+0000 7f08d4842640 1 --2- 172.21.15.107:0/1049012470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d7b6233b0 0x562d7b62e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.582+0000 7f08cf7fe640 1 -- 172.21.15.107:0/1049012470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d7b6233b0 msgr2=0x562d7b62e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:09.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.582+0000 7f08cf7fe640 1 --2- 172.21.15.107:0/1049012470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d7b6233b0 0x562d7b62e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.582+0000 7f08cf7fe640 1 -- 172.21.15.107:0/1049012470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d7b61dcb0 msgr2=0x562d7b62b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:14:09.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.582+0000 7f08cf7fe640 1 --2- 172.21.15.107:0/1049012470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d7b61dcb0 0x562d7b62b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.582+0000 7f08cf7fe640 1 -- 172.21.15.107:0/1049012470 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d7b62e940 con 0x562d7b61e6a0 2024-04-25T00:14:09.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.582+0000 7f08cf7fe640 1 --2- 172.21.15.107:0/1049012470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61e6a0 0x562d7b62b990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f08c00bbff0 tx=0x7f08c0009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2594aa43f04852f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:09.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.582+0000 7f08af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:14:09.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.582+0000 7f08af7fe640 1 -- 172.21.15.107:0/1049012470 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c00d4030 con 0x562d7b61e6a0 2024-04-25T00:14:09.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.582+0000 7f08af7fe640 1 -- 172.21.15.107:0/1049012470 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f08c00d5270 con 0x562d7b61e6a0 2024-04-25T00:14:09.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.582+0000 7f08d8fb3740 1 -- 172.21.15.107:0/1049012470 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08c00c06a0 con 0x562d7b61e6a0 2024-04-25T00:14:09.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.582+0000 7f08af7fe640 1 -- 172.21.15.107:0/1049012470 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f08c00b6e60 con 0x562d7b61e6a0 2024-04-25T00:14:09.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.582+0000 7f08d8fb3740 10 client.14757 fetch_fsmap learned FSMap version 17 2024-04-25T00:14:09.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.582+0000 7f08d8fb3740 1 -- 172.21.15.107:0/1049012470 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x562d7b69aef0 con 0x562d7b61e6a0 2024-04-25T00:14:09.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.582+0000 7f08af7fe640 1 -- 172.21.15.107:0/1049012470 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08c00c06a0 con 0x562d7b61e6a0 2024-04-25T00:14:09.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.582+0000 7f08af7fe640 1 -- 172.21.15.107:0/1049012470 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f08c00bc550 con 0x562d7b61e6a0 2024-04-25T00:14:09.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.582+0000 7f08d8fb3740 10 client.14757 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T00:14:09.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.583+0000 7f08d8fb3740 10 client.14757 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:14:09.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.583+0000 7f08d8fb3740 20 client.14757 populate_metadata read hostname 'smithi107' 2024-04-25T00:14:09.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.583+0000 7f08d8fb3740 1 --2- 172.21.15.107:0/1049012470 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562d7b69f930 0x562d7b69e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:09.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.583+0000 7f08d8fb3740 4 client.14757 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:14:09.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.583+0000 7f08d8fb3740 1 -- 172.21.15.107:0/1049012470 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "status"}) -- 0x562d7b584790 con 0x562d7b69f930 2024-04-25T00:14:09.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.583+0000 7f08cffff640 1 --2- 172.21.15.107:0/1049012470 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562d7b69f930 0x562d7b69e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:09.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.583+0000 7f08cffff640 1 --2- 172.21.15.107:0/1049012470 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562d7b69f930 0x562d7b69e640 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=806989021f9fefd7 server_cookie=837a5c5d9ea4ddef in_seq=0 out_seq=0 2024-04-25T00:14:09.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.583+0000 7f08af7fe640 10 client.14757 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:14:09.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.583+0000 7f08af7fe640 1 -- 172.21.15.107:0/1049012470 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x562d7b584790 con 0x562d7b69f930 2024-04-25T00:14:09.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.583+0000 7f08af7fe640 10 client.14757 handle_command_reply: tid=0 2024-04-25T00:14:09.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.583+0000 7f08d8fb3740 1 client.14757 shutdown 2024-04-25T00:14:09.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.584+0000 7f08d8fb3740 1 -- 172.21.15.107:0/1049012470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61e6a0 msgr2=0x562d7b62b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.584+0000 7f08d8fb3740 1 --2- 172.21.15.107:0/1049012470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61e6a0 0x562d7b62b990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f08c00bbff0 tx=0x7f08c0009680 comp rx=0 tx=0).stop 2024-04-25T00:14:09.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.584+0000 7f08d8fb3740 1 -- 172.21.15.107:0/1049012470 shutdown_connections 2024-04-25T00:14:09.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.584+0000 7f08d8fb3740 1 --2- 172.21.15.107:0/1049012470 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562d7b69f930 0x562d7b69e640 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.584+0000 7f08d8fb3740 1 --2- 172.21.15.107:0/1049012470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562d7b6233b0 0x562d7b62e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.584+0000 7f08af7fe640 0 client.14757 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:14:09.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.584+0000 7f08d8fb3740 1 --2- 172.21.15.107:0/1049012470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d7b61e6a0 0x562d7b62b990 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.584+0000 7f08d8fb3740 1 --2- 172.21.15.107:0/1049012470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562d7b61dcb0 0x562d7b62b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.584+0000 7f08d8fb3740 1 -- 172.21.15.107:0/1049012470 >> 172.21.15.107:0/1049012470 conn(0x562d7b615ff0 msgr2=0x562d7b613250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:09.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.584+0000 7f08d8fb3740 1 -- 172.21.15.107:0/1049012470 shutdown_connections 2024-04-25T00:14:09.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.584+0000 7f08d8fb3740 1 -- 172.21.15.107:0/1049012470 wait complete. 2024-04-25T00:14:09.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.584+0000 7f08d8fb3740 20 client.14757 trim_cache size 0 max 16384 2024-04-25T00:14:09.593 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:14:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "79602f58-0297-11ef-bc93-c7b262605968", 2024-04-25T00:14:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 0, 2024-04-25T00:14:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24521, 2024-04-25T00:14:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-04-25T00:14:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-04-25T00:14:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-04-25T00:14:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 6.7973320309999998, 2024-04-25T00:14:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 17, 2024-04-25T00:14:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 92, 2024-04-25T00:14:09.593 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 91, 2024-04-25T00:14:09.594 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 12.195980097 2024-04-25T00:14:09.594 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:14:09.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.584+0000 7f08d62a5640 1 -- 172.21.15.107:0/634659206 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f08b80a2c10 msgr2=0x7f08b80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.584+0000 7f08d62a5640 1 --2- 172.21.15.107:0/634659206 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f08b80a2c10 0x7f08b80a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009fd0 tx=0x7f08c8016000 comp rx=0 tx=0).stop 2024-04-25T00:14:09.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.584+0000 7f08d62a5640 1 -- 172.21.15.107:0/634659206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08d0101890 msgr2=0x7f08d0196f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:09.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.584+0000 7f08d62a5640 1 --2- 172.21.15.107:0/634659206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08d0101890 0x7f08d0196f30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009230 tx=0x7f08c0004360 comp rx=0 tx=0).stop 2024-04-25T00:14:09.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.584+0000 7f08d62a5640 1 -- 172.21.15.107:0/634659206 shutdown_connections 2024-04-25T00:14:09.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.584+0000 7f08d62a5640 1 --2- 172.21.15.107:0/634659206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f08d0102250 0x7f08d019dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.584+0000 7f08d62a5640 1 --2- 172.21.15.107:0/634659206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08d0101890 0x7f08d0196f30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.584+0000 7f08d62a5640 1 --2- 172.21.15.107:0/634659206 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f08b80a2c10 0x7f08b80a50d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.584+0000 7f08d62a5640 1 --2- 172.21.15.107:0/634659206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08d01006a0 0x7f08d01969f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:09.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.584+0000 7f08d62a5640 1 -- 172.21.15.107:0/634659206 >> 172.21.15.107:0/634659206 conn(0x7f08d00fbc10 msgr2=0x7f08d0104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:09.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.584+0000 7f08d62a5640 1 -- 172.21.15.107:0/634659206 shutdown_connections 2024-04-25T00:14:09.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:09.584+0000 7f08d62a5640 1 -- 172.21.15.107:0/634659206 wait complete. 2024-04-25T00:14:09.596 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T00:14:09.596 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T00:14:09.597 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T00:14:09.830 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:14:10.118 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:09 smithi107 ceph-mon[29153]: pgmap v296: 129 pgs: 3 creating+peering, 20 unknown, 106 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 2 op/s 2024-04-25T00:14:10.118 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:09 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3590448859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:14:10.119 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:09 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/778838434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:10.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:09 smithi170 ceph-mon[32807]: pgmap v296: 129 pgs: 3 creating+peering, 20 unknown, 106 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 2 op/s 2024-04-25T00:14:10.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:09 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3590448859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:14:10.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:09 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/778838434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:10.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:09 smithi195 ceph-mon[32833]: pgmap v296: 129 pgs: 3 creating+peering, 20 unknown, 106 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 2 op/s 2024-04-25T00:14:10.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:09 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3590448859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:14:10.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:09 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/778838434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:11.175 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:10 smithi107 ceph-mon[29153]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T00:14:11.175 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:10 smithi107 ceph-mon[29153]: Cluster is now healthy 2024-04-25T00:14:11.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:10 smithi170 ceph-mon[32807]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T00:14:11.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:10 smithi170 ceph-mon[32807]: Cluster is now healthy 2024-04-25T00:14:11.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:10 smithi195 ceph-mon[32833]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T00:14:11.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:10 smithi195 ceph-mon[32833]: Cluster is now healthy 2024-04-25T00:14:11.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.570+0000 7fe8084bd700 1 -- 172.21.15.107:0/763050172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe800107090 msgr2=0x7fe800107470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:11.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.570+0000 7fe8084bd700 1 --2- 172.21.15.107:0/763050172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe800107090 0x7fe800107470 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009990 tx=0x7fe7f0009ca0 comp rx=0 tx=0).stop 2024-04-25T00:14:11.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.570+0000 7fe8084bd700 1 -- 172.21.15.107:0/763050172 shutdown_connections 2024-04-25T00:14:11.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.570+0000 7fe8084bd700 1 --2- 172.21.15.107:0/763050172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe800106470 0x7fe80010e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:11.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.570+0000 7fe8084bd700 1 --2- 172.21.15.107:0/763050172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe800107090 0x7fe800107470 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:11.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.570+0000 7fe8084bd700 1 --2- 172.21.15.107:0/763050172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe800107a40 0x7fe800105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:11.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.570+0000 7fe8084bd700 1 -- 172.21.15.107:0/763050172 >> 172.21.15.107:0/763050172 conn(0x7fe800077330 msgr2=0x7fe8000fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:11.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.571+0000 7fe8084bd700 1 -- 172.21.15.107:0/763050172 shutdown_connections 2024-04-25T00:14:11.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.571+0000 7fe8084bd700 1 -- 172.21.15.107:0/763050172 wait complete. 2024-04-25T00:14:11.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.571+0000 7fe8084bd700 1 Processor -- start 2024-04-25T00:14:11.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.571+0000 7fe8084bd700 1 -- start start 2024-04-25T00:14:11.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.571+0000 7fe8084bd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe800106470 0x7fe80019d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:11.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.571+0000 7fe8084bd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe800107090 0x7fe80019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:11.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.571+0000 7fe8084bd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe800107a40 0x7fe8001977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:11.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.571+0000 7fe8084bd700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe8001114b0 con 0x7fe800106470 2024-04-25T00:14:11.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.571+0000 7fe8084bd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe800111330 con 0x7fe800107a40 2024-04-25T00:14:11.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.571+0000 7fe8084bd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe800111630 con 0x7fe800107090 2024-04-25T00:14:11.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.571+0000 7fe806259700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe800106470 0x7fe80019d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:11.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.571+0000 7fe806259700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe800106470 0x7fe80019d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46730/0 (socket says 172.21.15.107:46730) 2024-04-25T00:14:11.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.571+0000 7fe806259700 1 -- 172.21.15.107:0/3288664170 learned_addr learned my addr 172.21.15.107:0/3288664170 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:11.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.572+0000 7fe806259700 1 -- 172.21.15.107:0/3288664170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe800107090 msgr2=0x7fe80019db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:11.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.572+0000 7fe806259700 1 --2- 172.21.15.107:0/3288664170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe800107090 0x7fe80019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:11.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.572+0000 7fe806259700 1 -- 172.21.15.107:0/3288664170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe800107a40 msgr2=0x7fe8001977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:11.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.572+0000 7fe806259700 1 --2- 172.21.15.107:0/3288664170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe800107a40 0x7fe8001977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:11.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.572+0000 7fe806259700 1 -- 172.21.15.107:0/3288664170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe800197f90 con 0x7fe800106470 2024-04-25T00:14:11.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.572+0000 7fe806259700 1 --2- 172.21.15.107:0/3288664170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe800106470 0x7fe80019d630 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0000c00 tx=0x7fe7f00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:11.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.572+0000 7fe7f77fe700 1 -- 172.21.15.107:0/3288664170 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0027070 con 0x7fe800106470 2024-04-25T00:14:11.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.572+0000 7fe7f77fe700 1 -- 172.21.15.107:0/3288664170 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7f0004050 con 0x7fe800106470 2024-04-25T00:14:11.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.572+0000 7fe8084bd700 1 -- 172.21.15.107:0/3288664170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe800198220 con 0x7fe800106470 2024-04-25T00:14:11.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.572+0000 7fe7f77fe700 1 -- 172.21.15.107:0/3288664170 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f000f460 con 0x7fe800106470 2024-04-25T00:14:11.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.572+0000 7fe8084bd700 1 -- 172.21.15.107:0/3288664170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8001a4490 con 0x7fe800106470 2024-04-25T00:14:11.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.573+0000 7fe8084bd700 1 -- 172.21.15.107:0/3288664170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe800109690 con 0x7fe800106470 2024-04-25T00:14:11.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.574+0000 7fe7f77fe700 1 -- 172.21.15.107:0/3288664170 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fe7f002a070 con 0x7fe800106470 2024-04-25T00:14:11.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.575+0000 7fe7f77fe700 1 --2- 172.21.15.107:0/3288664170 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe7ec0a2c20 0x7fe7ec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:11.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.575+0000 7fe805a58700 1 --2- 172.21.15.107:0/3288664170 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe7ec0a2c20 0x7fe7ec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:11.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.575+0000 7fe7f77fe700 1 -- 172.21.15.107:0/3288664170 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fe7f00d0650 con 0x7fe800106470 2024-04-25T00:14:11.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.575+0000 7fe805a58700 1 --2- 172.21.15.107:0/3288664170 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe7ec0a2c20 0x7fe7ec0a50e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe8000ffa00 tx=0x7fe7fc006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:11.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.578+0000 7fe7f77fe700 1 -- 172.21.15.107:0/3288664170 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe7f0031070 con 0x7fe800106470 2024-04-25T00:14:11.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.818+0000 7fe8084bd700 1 -- 172.21.15.107:0/3288664170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe800198ce0 con 0x7fe800106470 2024-04-25T00:14:11.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.819+0000 7fe7f77fe700 1 -- 172.21.15.107:0/3288664170 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fe7f0013070 con 0x7fe800106470 2024-04-25T00:14:11.819 INFO:teuthology.orchestra.run.smithi107.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T00:14:11.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.821+0000 7fe8084bd700 1 -- 172.21.15.107:0/3288664170 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe7ec0a2c20 msgr2=0x7fe7ec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:11.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.821+0000 7fe8084bd700 1 --2- 172.21.15.107:0/3288664170 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe7ec0a2c20 0x7fe7ec0a50e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe8000ffa00 tx=0x7fe7fc006d20 comp rx=0 tx=0).stop 2024-04-25T00:14:11.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.821+0000 7fe8084bd700 1 -- 172.21.15.107:0/3288664170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe800106470 msgr2=0x7fe80019d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:11.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.821+0000 7fe8084bd700 1 --2- 172.21.15.107:0/3288664170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe800106470 0x7fe80019d630 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0000c00 tx=0x7fe7f00047b0 comp rx=0 tx=0).stop 2024-04-25T00:14:11.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.821+0000 7fe8084bd700 1 -- 172.21.15.107:0/3288664170 shutdown_connections 2024-04-25T00:14:11.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.821+0000 7fe8084bd700 1 --2- 172.21.15.107:0/3288664170 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe7ec0a2c20 0x7fe7ec0a50e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:11.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.821+0000 7fe8084bd700 1 --2- 172.21.15.107:0/3288664170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe800107a40 0x7fe8001977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:11.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.821+0000 7fe8084bd700 1 --2- 172.21.15.107:0/3288664170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe800106470 0x7fe80019d630 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:11.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.821+0000 7fe8084bd700 1 --2- 172.21.15.107:0/3288664170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe800107090 0x7fe80019db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:11.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.821+0000 7fe8084bd700 1 -- 172.21.15.107:0/3288664170 >> 172.21.15.107:0/3288664170 conn(0x7fe800077330 msgr2=0x7fe8001040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:11.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.821+0000 7fe8084bd700 1 -- 172.21.15.107:0/3288664170 shutdown_connections 2024-04-25T00:14:11.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:11.821+0000 7fe8084bd700 1 -- 172.21.15.107:0/3288664170 wait complete. 2024-04-25T00:14:12.183 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:11 smithi107 ceph-mon[29153]: pgmap v297: 129 pgs: 3 creating+peering, 126 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:14:12.183 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:11 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3288664170' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T00:14:12.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:11 smithi170 ceph-mon[32807]: pgmap v297: 129 pgs: 3 creating+peering, 126 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:14:12.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:11 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3288664170' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T00:14:12.239 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:11 smithi195 ceph-mon[32833]: pgmap v297: 129 pgs: 3 creating+peering, 126 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:14:12.239 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:11 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3288664170' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T00:14:12.531 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T00:14:12.531 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:14:12.531 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi107.front.sepia.ceph.com 2024-04-25T00:14:12.531 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T00:14:12.531 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T00:14:12.531 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T00:14:12.531 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:14:12.531 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:14:12.531 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T00:14:12.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:12.532 DEBUG:teuthology.orchestra.run.smithi107:> ip addr 2024-04-25T00:14:12.548 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:14:12.548 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:14:12.548 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:14:12.548 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:12.548 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2024-04-25T00:14:12.548 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:12.548 INFO:teuthology.orchestra.run.smithi107.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:14:12.548 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:12.548 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f0 2024-04-25T00:14:12.548 INFO:teuthology.orchestra.run.smithi107.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:14:12.548 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0b brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:12.549 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f1 2024-04-25T00:14:12.549 INFO:teuthology.orchestra.run.smithi107.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:14:12.549 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:ea brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:12.549 INFO:teuthology.orchestra.run.smithi107.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:14:12.549 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:eb brd ff:ff:ff:ff:ff:ff 2024-04-25T00:14:12.549 INFO:teuthology.orchestra.run.smithi107.stdout: inet 172.21.15.107/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T00:14:12.549 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft 42222sec preferred_lft 42222sec 2024-04-25T00:14:12.549 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::d21f:963:7e10:29d9/64 scope link noprefixroute 2024-04-25T00:14:12.549 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:14:12.550 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T00:14:12.550 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:14:12.550 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-25T00:14:12.550 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add name ceph-brx type bridge 2024-04-25T00:14:12.550 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr flush dev ceph-brx 2024-04-25T00:14:12.550 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set ceph-brx up 2024-04-25T00:14:12.550 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T00:14:12.550 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-25T00:14:12.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:14:12.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:14:12.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:12.732 DEBUG:teuthology.orchestra.run.smithi107:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:14:12.756 INFO:teuthology.orchestra.run.smithi107.stdout:1 2024-04-25T00:14:12.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:12.758 DEBUG:teuthology.orchestra.run.smithi107:> route 2024-04-25T00:14:12.831 INFO:teuthology.orchestra.run.smithi107.stdout:Kernel IP routing table 2024-04-25T00:14:12.831 INFO:teuthology.orchestra.run.smithi107.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:14:12.832 INFO:teuthology.orchestra.run.smithi107.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:14:12.832 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:14:12.832 INFO:teuthology.orchestra.run.smithi107.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:14:12.832 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:14:12.832 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-25T00:14:12.832 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:14:12.832 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:14:12.832 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:14:12.832 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-25T00:14:12.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:14:13.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:14:13.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:13.108 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-04-25T00:14:13.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:13.123 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list-id 2024-04-25T00:14:13.177 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:14:13.178 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-25T00:14:13.178 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T00:14:13.178 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T00:14:13.178 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-25T00:14:13.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:14:13.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:14:13.294 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T00:14:13.294 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:14:13.294 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-25T00:14:13.294 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T00:14:13.294 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T00:14:13.294 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T00:14:13.295 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T00:14:13.295 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T00:14:13.295 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-25T00:14:13.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:14:13.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:14:13.424 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:14:13.424 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-25T00:14:13.424 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set brx.0 up 2024-04-25T00:14:13.424 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T00:14:13.424 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-25T00:14:13.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:14:13.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:14:13.498 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T00:14:13.498 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:14:13.498 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T00:14:13.512 INFO:teuthology.orchestra.run.smithi107.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T00:14:13.512 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:14:13.512 DEBUG:teuthology.orchestra.run.smithi107:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T00:14:13.566 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2024-04-25T00:14:13.630 DEBUG:teuthology.orchestra.run.smithi107:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:14:13.686 INFO:teuthology.orchestra.run.smithi107.stdout:/proc 2024-04-25T00:14:13.686 INFO:teuthology.orchestra.run.smithi107.stdout:/sys 2024-04-25T00:14:13.686 INFO:teuthology.orchestra.run.smithi107.stdout:/dev 2024-04-25T00:14:13.686 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/security 2024-04-25T00:14:13.686 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/shm 2024-04-25T00:14:13.686 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/pts 2024-04-25T00:14:13.686 INFO:teuthology.orchestra.run.smithi107.stdout:/run 2024-04-25T00:14:13.686 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/cgroup 2024-04-25T00:14:13.686 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/pstore 2024-04-25T00:14:13.686 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/bpf 2024-04-25T00:14:13.686 INFO:teuthology.orchestra.run.smithi107.stdout:/ 2024-04-25T00:14:13.687 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/selinux 2024-04-25T00:14:13.687 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:14:13.687 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/hugepages 2024-04-25T00:14:13.687 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/mqueue 2024-04-25T00:14:13.687 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/debug 2024-04-25T00:14:13.687 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/tracing 2024-04-25T00:14:13.687 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/config 2024-04-25T00:14:13.687 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:14:13.687 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/fuse/connections 2024-04-25T00:14:13.687 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T00:14:13.687 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T00:14:13.687 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T00:14:13.687 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T00:14:13.687 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/1000 2024-04-25T00:14:13.688 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph 2024-04-25T00:14:13.688 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay 2024-04-25T00:14:13.688 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/5a5e1b99cffd80ba8f2f1faa5f246d318e452a0befcc227f3628ea02a88aa8ab/merged 2024-04-25T00:14:13.688 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/ee2c0ca72035a81b89585dca41acf0cbde32978924e59bb3007edb9bd9c8a62f/merged 2024-04-25T00:14:13.688 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/0 2024-04-25T00:14:13.688 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:14:13.688 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/be6a2fa59b4fb8fd05c02ee09bdbee9ef60286fdebdc1883222bcd6df6383d5c/merged 2024-04-25T00:14:13.688 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/6f431f46b0fc3c6e4e29cc53306b66fc941c2945a0a6f7ccb8b78a3bec907c4c/merged 2024-04-25T00:14:13.688 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/6e631c222104f93e8f86efebfa94da072b2852292b303d636a3f9831b6619688/merged 2024-04-25T00:14:13.688 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/324f01322043d5394fda5956726ca6b50e2f24885b5924de21bde2d41ecc3159/merged 2024-04-25T00:14:13.688 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/f621bb73fbd9712475c748415b80e0f413cad5ce420dc0129d87689f2c8ad443/merged 2024-04-25T00:14:13.688 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/17e1e16a60b89806d05b9ac803d8e23edc238e87c0551479a2b4043379b99e6b/merged 2024-04-25T00:14:13.688 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/2b80767e51de34a9189951242c617078e25100c83d3950d7f6e676687154b796/merged 2024-04-25T00:14:13.688 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/30efbdfb1aaf033f50ceca01ef8d6b25747e9678f137a084bd986cdd3e1f5056/merged 2024-04-25T00:14:13.688 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns 2024-04-25T00:14:13.689 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T00:14:13.689 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T00:14:13.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:13.689 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2024-04-25T00:14:13.741 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T00:14:13.741 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T00:14:13.783 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2024-04-25T00:14:13.806 DEBUG:teuthology.orchestra.run.smithi107:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T00:14:13.862 INFO:teuthology.orchestra.run.smithi107.stdout:/proc 2024-04-25T00:14:13.862 INFO:teuthology.orchestra.run.smithi107.stdout:/sys 2024-04-25T00:14:13.863 INFO:teuthology.orchestra.run.smithi107.stdout:/dev 2024-04-25T00:14:13.863 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/security 2024-04-25T00:14:13.863 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/shm 2024-04-25T00:14:13.863 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/pts 2024-04-25T00:14:13.863 INFO:teuthology.orchestra.run.smithi107.stdout:/run 2024-04-25T00:14:13.863 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/cgroup 2024-04-25T00:14:13.863 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/pstore 2024-04-25T00:14:13.863 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/bpf 2024-04-25T00:14:13.863 INFO:teuthology.orchestra.run.smithi107.stdout:/ 2024-04-25T00:14:13.863 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/selinux 2024-04-25T00:14:13.863 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:14:13.863 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/hugepages 2024-04-25T00:14:13.863 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/mqueue 2024-04-25T00:14:13.864 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/debug 2024-04-25T00:14:13.864 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/tracing 2024-04-25T00:14:13.864 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/config 2024-04-25T00:14:13.864 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T00:14:13.864 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/fuse/connections 2024-04-25T00:14:13.864 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T00:14:13.864 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T00:14:13.864 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T00:14:13.864 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T00:14:13.864 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/1000 2024-04-25T00:14:13.864 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph 2024-04-25T00:14:13.864 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay 2024-04-25T00:14:13.864 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/5a5e1b99cffd80ba8f2f1faa5f246d318e452a0befcc227f3628ea02a88aa8ab/merged 2024-04-25T00:14:13.864 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/ee2c0ca72035a81b89585dca41acf0cbde32978924e59bb3007edb9bd9c8a62f/merged 2024-04-25T00:14:13.865 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/0 2024-04-25T00:14:13.865 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T00:14:13.865 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/be6a2fa59b4fb8fd05c02ee09bdbee9ef60286fdebdc1883222bcd6df6383d5c/merged 2024-04-25T00:14:13.865 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/6f431f46b0fc3c6e4e29cc53306b66fc941c2945a0a6f7ccb8b78a3bec907c4c/merged 2024-04-25T00:14:13.865 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/6e631c222104f93e8f86efebfa94da072b2852292b303d636a3f9831b6619688/merged 2024-04-25T00:14:13.865 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/324f01322043d5394fda5956726ca6b50e2f24885b5924de21bde2d41ecc3159/merged 2024-04-25T00:14:13.865 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/f621bb73fbd9712475c748415b80e0f413cad5ce420dc0129d87689f2c8ad443/merged 2024-04-25T00:14:13.865 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/17e1e16a60b89806d05b9ac803d8e23edc238e87c0551479a2b4043379b99e6b/merged 2024-04-25T00:14:13.865 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/2b80767e51de34a9189951242c617078e25100c83d3950d7f6e676687154b796/merged 2024-04-25T00:14:13.865 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/30efbdfb1aaf033f50ceca01ef8d6b25747e9678f137a084bd986cdd3e1f5056/merged 2024-04-25T00:14:13.865 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns 2024-04-25T00:14:13.865 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T00:14:13.865 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T00:14:13.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:13.866 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2024-04-25T00:14:13.885 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi107.stderr:2024-04-25T00:14:13.884+0000 7f218fb55480 -1 init, newargv = 0x55c203217250 newargc=15 2024-04-25T00:14:13.885 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi107.stderr:ceph-fuse[70079]: starting ceph client 2024-04-25T00:14:13.890 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi107.stderr:ceph-fuse[70079]: starting fuse 2024-04-25T00:14:13.899 INFO:teuthology.orchestra.run.smithi107.stdout:115 2024-04-25T00:14:13.899 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [115] 2024-04-25T00:14:13.899 DEBUG:teuthology.orchestra.run.smithi107:> sudo stdin-killer -- python3 -c ' 2024-04-25T00:14:13.899 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2024-04-25T00:14:13.899 DEBUG:teuthology.orchestra.run.smithi107:> import re 2024-04-25T00:14:13.900 DEBUG:teuthology.orchestra.run.smithi107:> import os 2024-04-25T00:14:13.900 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2024-04-25T00:14:13.900 DEBUG:teuthology.orchestra.run.smithi107:> 2024-04-25T00:14:13.900 DEBUG:teuthology.orchestra.run.smithi107:> def _find_admin_socket(client_name): 2024-04-25T00:14:13.900 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T00:14:13.900 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2024-04-25T00:14:13.900 DEBUG:teuthology.orchestra.run.smithi107:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T00:14:13.900 DEBUG:teuthology.orchestra.run.smithi107:> 2024-04-25T00:14:13.900 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2024-04-25T00:14:13.900 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2024-04-25T00:14:13.900 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2024-04-25T00:14:13.900 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2024-04-25T00:14:13.900 DEBUG:teuthology.orchestra.run.smithi107:> 2024-04-25T00:14:13.900 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2024-04-25T00:14:13.900 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T00:14:13.900 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T00:14:13.901 DEBUG:teuthology.orchestra.run.smithi107:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T00:14:13.901 DEBUG:teuthology.orchestra.run.smithi107:> contents = proc_f.read() 2024-04-25T00:14:13.901 DEBUG:teuthology.orchestra.run.smithi107:> if mountpoint in contents: 2024-04-25T00:14:13.901 DEBUG:teuthology.orchestra.run.smithi107:> return f 2024-04-25T00:14:13.901 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T00:14:13.901 DEBUG:teuthology.orchestra.run.smithi107:> 2024-04-25T00:14:13.901 DEBUG:teuthology.orchestra.run.smithi107:> print(_find_admin_socket("client.admin")) 2024-04-25T00:14:13.901 DEBUG:teuthology.orchestra.run.smithi107:> ' 2024-04-25T00:14:14.017 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.admin.70079.asok 2024-04-25T00:14:14.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:14:14.027 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.70079.asok 2024-04-25T00:14:14.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:14.027 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.70079.asok status 2024-04-25T00:14:14.160 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:14:14.160 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2024-04-25T00:14:14.160 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T00:14:14.160 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:14:14.160 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "admin", 2024-04-25T00:14:14.160 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2024-04-25T00:14:14.161 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T00:14:14.161 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "70079", 2024-04-25T00:14:14.161 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/" 2024-04-25T00:14:14.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:14:14.161 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2024-04-25T00:14:14.161 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2024-04-25T00:14:14.161 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 14769, 2024-04-25T00:14:14.161 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2024-04-25T00:14:14.161 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2024-04-25T00:14:14.161 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2024-04-25T00:14:14.161 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 14769 2024-04-25T00:14:14.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:14:14.161 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2024-04-25T00:14:14.161 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-04-25T00:14:14.161 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:14:14.162 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 44451526 2024-04-25T00:14:14.162 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:14:14.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:14:14.162 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2024-04-25T00:14:14.162 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-04-25T00:14:14.162 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "192.168.0.1:0", 2024-04-25T00:14:14.162 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 44451526 2024-04-25T00:14:14.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:14:14.162 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.14769 192.168.0.1:0/44451526", 2024-04-25T00:14:14.162 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "192.168.0.1:0/44451526", 2024-04-25T00:14:14.162 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 1, 2024-04-25T00:14:14.162 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 17, 2024-04-25T00:14:14.162 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 92, 2024-04-25T00:14:14.162 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2024-04-25T00:14:14.162 INFO:teuthology.orchestra.run.smithi107.stdout: "blocklisted": false, 2024-04-25T00:14:14.163 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2024-04-25T00:14:14.163 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:14:14.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:14.169 DEBUG:teuthology.orchestra.run.smithi107:> stat --file-system '--printf=%T 2024-04-25T00:14:14.169 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T00:14:14.184 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2024-04-25T00:14:14.184 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T00:14:14.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:14.184 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T00:14:14.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:13 smithi107 ceph-mon[29153]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:14:14.254 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T00:14:14.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:13 smithi195 ceph-mon[32833]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:14:14.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:14:14.361 INFO:teuthology.orchestra.run.smithi107.stdout:# file: . 2024-04-25T00:14:14.361 INFO:teuthology.orchestra.run.smithi107.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T00:14:14.361 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:14:14.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:14:14.367 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T00:14:14.367 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:14.383 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:14.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:13 smithi170 ceph-mon[32807]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:14:14.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.525+0000 7f71b69db640 1 -- 172.21.15.107:0/2220321807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71b0101870 msgr2=0x7f71b0101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:14.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.525+0000 7f71b69db640 1 --2- 172.21.15.107:0/2220321807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71b0101870 0x7f71b0101cf0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009900 tx=0x7f71a002a410 comp rx=0 tx=0).stop 2024-04-25T00:14:14.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.525+0000 7f71b69db640 1 -- 172.21.15.107:0/2220321807 shutdown_connections 2024-04-25T00:14:14.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.525+0000 7f71b69db640 1 --2- 172.21.15.107:0/2220321807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71b0102230 0x7f71b0108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:14.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.525+0000 7f71b69db640 1 --2- 172.21.15.107:0/2220321807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71b0101870 0x7f71b0101cf0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:14.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.525+0000 7f71b69db640 1 --2- 172.21.15.107:0/2220321807 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b0100680 0x7f71b0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:14.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.525+0000 7f71b69db640 1 -- 172.21.15.107:0/2220321807 >> 172.21.15.107:0/2220321807 conn(0x7f71b00fbbf0 msgr2=0x7f71b00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:14.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.525+0000 7f71b69db640 1 -- 172.21.15.107:0/2220321807 shutdown_connections 2024-04-25T00:14:14.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.525+0000 7f71b69db640 1 -- 172.21.15.107:0/2220321807 wait complete. 2024-04-25T00:14:14.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.525+0000 7f71b69db640 1 Processor -- start 2024-04-25T00:14:14.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.525+0000 7f71b69db640 1 -- start start 2024-04-25T00:14:14.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.526+0000 7f71b69db640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71b0100680 0x7f71b0069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:14.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.526+0000 7f71b69db640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b0101870 0x7f71b019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:14.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.526+0000 7f71b69db640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71b0102230 0x7f71b019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:14.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.526+0000 7f71b69db640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f71b010c320 con 0x7f71b0100680 2024-04-25T00:14:14.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.526+0000 7f71b69db640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f71b010c1a0 con 0x7f71b0101870 2024-04-25T00:14:14.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.526+0000 7f71b69db640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f71b010c4a0 con 0x7f71b0102230 2024-04-25T00:14:14.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.526+0000 7f71affff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71b0100680 0x7f71b0069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:14.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.526+0000 7f71affff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71b0100680 0x7f71b0069460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46744/0 (socket says 172.21.15.107:46744) 2024-04-25T00:14:14.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.526+0000 7f71affff640 1 -- 172.21.15.107:0/2336628462 learned_addr learned my addr 172.21.15.107:0/2336628462 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:14.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.526+0000 7f71af7fe640 1 --2- 172.21.15.107:0/2336628462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b0101870 0x7f71b019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:14.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.526+0000 7f71b4f78640 1 --2- 172.21.15.107:0/2336628462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71b0102230 0x7f71b019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:14.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.526+0000 7f71affff640 1 -- 172.21.15.107:0/2336628462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71b0102230 msgr2=0x7f71b019ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:14.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.526+0000 7f71affff640 1 --2- 172.21.15.107:0/2336628462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71b0102230 0x7f71b019ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:14.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.526+0000 7f71affff640 1 -- 172.21.15.107:0/2336628462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b0101870 msgr2=0x7f71b019c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:14.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.526+0000 7f71affff640 1 --2- 172.21.15.107:0/2336628462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b0101870 0x7f71b019c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:14.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.526+0000 7f71affff640 1 -- 172.21.15.107:0/2336628462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b019f6f0 con 0x7f71b0100680 2024-04-25T00:14:14.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.526+0000 7f71affff640 1 --2- 172.21.15.107:0/2336628462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71b0100680 0x7f71b0069460 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f71a400d950 tx=0x7f71a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:14.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.526+0000 7f71ad7fa640 1 -- 172.21.15.107:0/2336628462 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4017070 con 0x7f71b0100680 2024-04-25T00:14:14.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.527+0000 7f71ad7fa640 1 -- 172.21.15.107:0/2336628462 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71a4009c30 con 0x7f71b0100680 2024-04-25T00:14:14.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.527+0000 7f71b69db640 1 -- 172.21.15.107:0/2336628462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b019f9e0 con 0x7f71b0100680 2024-04-25T00:14:14.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.527+0000 7f71ad7fa640 1 -- 172.21.15.107:0/2336628462 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a400f600 con 0x7f71b0100680 2024-04-25T00:14:14.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.527+0000 7f71b69db640 1 -- 172.21.15.107:0/2336628462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b019feb0 con 0x7f71b0100680 2024-04-25T00:14:14.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.528+0000 7f71b69db640 1 -- 172.21.15.107:0/2336628462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71b01079a0 con 0x7f71b0100680 2024-04-25T00:14:14.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.529+0000 7f71ad7fa640 1 -- 172.21.15.107:0/2336628462 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f71a401a070 con 0x7f71b0100680 2024-04-25T00:14:14.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.529+0000 7f71ad7fa640 1 --2- 172.21.15.107:0/2336628462 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f71980a2b90 0x7f71980a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:14.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.529+0000 7f71af7fe640 1 --2- 172.21.15.107:0/2336628462 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f71980a2b90 0x7f71980a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:14.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.529+0000 7f71ad7fa640 1 -- 172.21.15.107:0/2336628462 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f71a40cec50 con 0x7f71b0100680 2024-04-25T00:14:14.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.530+0000 7f71af7fe640 1 --2- 172.21.15.107:0/2336628462 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f71980a2b90 0x7f71980a5050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f71a00314e0 tx=0x7f71a0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:14.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.537+0000 7f71ad7fa640 1 -- 172.21.15.107:0/2336628462 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f71a4009de0 con 0x7f71b0100680 2024-04-25T00:14:14.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.722+0000 7f71b69db640 1 -- 172.21.15.107:0/2336628462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71b002d330 con 0x7f71b0100680 2024-04-25T00:14:14.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.723+0000 7f71ad7fa640 1 -- 172.21.15.107:0/2336628462 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24134 (secure 0 0 0) 0x7f71a409b320 con 0x7f71b0100680 2024-04-25T00:14:14.724 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:14:14.724 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":92,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:14:08.839737+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:59.672801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:00.957782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:14:05.065846+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:14.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.725+0000 7f71b69db640 1 -- 172.21.15.107:0/2336628462 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f71980a2b90 msgr2=0x7f71980a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:14.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.725+0000 7f71b69db640 1 --2- 172.21.15.107:0/2336628462 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f71980a2b90 0x7f71980a5050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f71a00314e0 tx=0x7f71a0000bc0 comp rx=0 tx=0).stop 2024-04-25T00:14:14.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.725+0000 7f71b69db640 1 -- 172.21.15.107:0/2336628462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71b0100680 msgr2=0x7f71b0069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:14.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.725+0000 7f71b69db640 1 --2- 172.21.15.107:0/2336628462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71b0100680 0x7f71b0069460 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f71a400d950 tx=0x7f71a400de10 comp rx=0 tx=0).stop 2024-04-25T00:14:14.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.725+0000 7f71b69db640 1 -- 172.21.15.107:0/2336628462 shutdown_connections 2024-04-25T00:14:14.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.725+0000 7f71b69db640 1 --2- 172.21.15.107:0/2336628462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71b0102230 0x7f71b019ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:14.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.725+0000 7f71b69db640 1 --2- 172.21.15.107:0/2336628462 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f71980a2b90 0x7f71980a5050 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:14.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.725+0000 7f71b69db640 1 --2- 172.21.15.107:0/2336628462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71b0101870 0x7f71b019c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:14.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.725+0000 7f71b69db640 1 --2- 172.21.15.107:0/2336628462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71b0100680 0x7f71b0069460 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:14.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.725+0000 7f71b69db640 1 -- 172.21.15.107:0/2336628462 >> 172.21.15.107:0/2336628462 conn(0x7f71b00fbbf0 msgr2=0x7f71b0104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:14.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.726+0000 7f71b69db640 1 -- 172.21.15.107:0/2336628462 shutdown_connections 2024-04-25T00:14:14.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.726+0000 7f71b69db640 1 -- 172.21.15.107:0/2336628462 wait complete. 2024-04-25T00:14:14.737 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:09:25.160329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:13:59.672801+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:14:00.957782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:14:05.065846+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:14:14.738 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:14:14.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.841+0000 7f62d72d4640 1 -- 172.21.15.107:0/3250686785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62d0101870 msgr2=0x7f62d0101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:14.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.841+0000 7f62d72d4640 1 --2- 172.21.15.107:0/3250686785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62d0101870 0x7f62d0101cf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009900 tx=0x7f62cc02a410 comp rx=0 tx=0).stop 2024-04-25T00:14:14.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.841+0000 7f62d72d4640 1 -- 172.21.15.107:0/3250686785 shutdown_connections 2024-04-25T00:14:14.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.841+0000 7f62d72d4640 1 --2- 172.21.15.107:0/3250686785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62d0102230 0x7f62d0108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:14.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.841+0000 7f62d72d4640 1 --2- 172.21.15.107:0/3250686785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62d0101870 0x7f62d0101cf0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:14.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.841+0000 7f62d72d4640 1 --2- 172.21.15.107:0/3250686785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d0100680 0x7f62d0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:14.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.841+0000 7f62d72d4640 1 -- 172.21.15.107:0/3250686785 >> 172.21.15.107:0/3250686785 conn(0x7f62d00fbbf0 msgr2=0x7f62d00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:14.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.841+0000 7f62d72d4640 1 -- 172.21.15.107:0/3250686785 shutdown_connections 2024-04-25T00:14:14.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.841+0000 7f62d72d4640 1 -- 172.21.15.107:0/3250686785 wait complete. 2024-04-25T00:14:14.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.842+0000 7f62d72d4640 1 Processor -- start 2024-04-25T00:14:14.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.842+0000 7f62d72d4640 1 -- start start 2024-04-25T00:14:14.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.842+0000 7f62d72d4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d0100680 0x7f62d0197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:14.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.842+0000 7f62d72d4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62d0101870 0x7f62d0198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:14.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.842+0000 7f62d72d4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62d0102230 0x7f62d019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:14.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.842+0000 7f62d72d4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f62d010c300 con 0x7f62d0101870 2024-04-25T00:14:14.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.842+0000 7f62d72d4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f62d010c180 con 0x7f62d0100680 2024-04-25T00:14:14.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.842+0000 7f62d72d4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f62d010c480 con 0x7f62d0102230 2024-04-25T00:14:14.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.842+0000 7f62d486f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62d0101870 0x7f62d0198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:14.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.842+0000 7f62d486f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62d0101870 0x7f62d0198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46766/0 (socket says 172.21.15.107:46766) 2024-04-25T00:14:14.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.842+0000 7f62d486f640 1 -- 172.21.15.107:0/3786831033 learned_addr learned my addr 172.21.15.107:0/3786831033 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:14.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.842+0000 7f62d5070640 1 --2- 172.21.15.107:0/3786831033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d0100680 0x7f62d0197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:14.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.842+0000 7f62d5871640 1 --2- 172.21.15.107:0/3786831033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62d0102230 0x7f62d019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:14.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.842+0000 7f62d486f640 1 -- 172.21.15.107:0/3786831033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62d0102230 msgr2=0x7f62d019f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:14.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.842+0000 7f62d486f640 1 --2- 172.21.15.107:0/3786831033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62d0102230 0x7f62d019f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:14.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.842+0000 7f62d486f640 1 -- 172.21.15.107:0/3786831033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d0100680 msgr2=0x7f62d0197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:14.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.842+0000 7f62d486f640 1 --2- 172.21.15.107:0/3786831033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d0100680 0x7f62d0197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:14.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.842+0000 7f62d486f640 1 -- 172.21.15.107:0/3786831033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d019f6f0 con 0x7f62d0101870 2024-04-25T00:14:14.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.842+0000 7f62d5871640 1 --2- 172.21.15.107:0/3786831033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62d0102230 0x7f62d019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:14.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.842+0000 7f62d486f640 1 --2- 172.21.15.107:0/3786831033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62d0101870 0x7f62d0198010 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009230 tx=0x7f62cc004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:14.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.842+0000 7f62c67fc640 1 -- 172.21.15.107:0/3786831033 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc03f030 con 0x7f62d0101870 2024-04-25T00:14:14.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.843+0000 7f62d72d4640 1 -- 172.21.15.107:0/3786831033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d019f980 con 0x7f62d0101870 2024-04-25T00:14:14.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.843+0000 7f62c67fc640 1 -- 172.21.15.107:0/3786831033 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62cc003ce0 con 0x7f62d0101870 2024-04-25T00:14:14.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.843+0000 7f62c67fc640 1 -- 172.21.15.107:0/3786831033 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62cc035070 con 0x7f62d0101870 2024-04-25T00:14:14.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.843+0000 7f62d72d4640 1 -- 172.21.15.107:0/3786831033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d019fec0 con 0x7f62d0101870 2024-04-25T00:14:14.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.844+0000 7f62d72d4640 1 -- 172.21.15.107:0/3786831033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62d01a00f0 con 0x7f62d0101870 2024-04-25T00:14:14.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.845+0000 7f62c67fc640 1 -- 172.21.15.107:0/3786831033 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f62cc042030 con 0x7f62d0101870 2024-04-25T00:14:14.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.845+0000 7f62c67fc640 1 --2- 172.21.15.107:0/3786831033 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f62bc0a2b90 0x7f62bc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:14.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.845+0000 7f62d5070640 1 --2- 172.21.15.107:0/3786831033 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f62bc0a2b90 0x7f62bc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:14.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.845+0000 7f62c67fc640 1 -- 172.21.15.107:0/3786831033 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f62cc0efc20 con 0x7f62d0101870 2024-04-25T00:14:14.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.846+0000 7f62d5070640 1 --2- 172.21.15.107:0/3786831033 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f62bc0a2b90 0x7f62bc0a5050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f62d01016d0 tx=0x7f62c000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:14.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:14.849+0000 7f62c67fc640 1 -- 172.21.15.107:0/3786831033 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f62cc0bc940 con 0x7f62d0101870 2024-04-25T00:14:15.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.035+0000 7f62d72d4640 1 -- 172.21.15.107:0/3786831033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f62d01a02c0 con 0x7f62d0101870 2024-04-25T00:14:15.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.035+0000 7f62c67fc640 1 -- 172.21.15.107:0/3786831033 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f62cc0bc240 con 0x7f62d0101870 2024-04-25T00:14:15.036 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-04-25T00:14:15.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.037+0000 7f62d72d4640 1 -- 172.21.15.107:0/3786831033 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f62bc0a2b90 msgr2=0x7f62bc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.037+0000 7f62d72d4640 1 --2- 172.21.15.107:0/3786831033 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f62bc0a2b90 0x7f62bc0a5050 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f62d01016d0 tx=0x7f62c000b390 comp rx=0 tx=0).stop 2024-04-25T00:14:15.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.037+0000 7f62d72d4640 1 -- 172.21.15.107:0/3786831033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62d0101870 msgr2=0x7f62d0198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.037+0000 7f62d72d4640 1 --2- 172.21.15.107:0/3786831033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62d0101870 0x7f62d0198010 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f62cc009230 tx=0x7f62cc004360 comp rx=0 tx=0).stop 2024-04-25T00:14:15.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.037+0000 7f62d72d4640 1 -- 172.21.15.107:0/3786831033 shutdown_connections 2024-04-25T00:14:15.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.037+0000 7f62d72d4640 1 --2- 172.21.15.107:0/3786831033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62d0102230 0x7f62d019f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.037+0000 7f62d72d4640 1 --2- 172.21.15.107:0/3786831033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62d0101870 0x7f62d0198010 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.037+0000 7f62d72d4640 1 --2- 172.21.15.107:0/3786831033 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f62bc0a2b90 0x7f62bc0a5050 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.037+0000 7f62d72d4640 1 --2- 172.21.15.107:0/3786831033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f62d0100680 0x7f62d0197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.037+0000 7f62d72d4640 1 -- 172.21.15.107:0/3786831033 >> 172.21.15.107:0/3786831033 conn(0x7f62d00fbbf0 msgr2=0x7f62d00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:15.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.037+0000 7f62d72d4640 1 -- 172.21.15.107:0/3786831033 shutdown_connections 2024-04-25T00:14:15.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.037+0000 7f62d72d4640 1 -- 172.21.15.107:0/3786831033 wait complete. 2024-04-25T00:14:15.047 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:14 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2336628462' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:15.048 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:14:15.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.151+0000 7fb079ed4640 1 -- 172.21.15.107:0/1293681255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb074102200 msgr2=0x7fb0741045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.151+0000 7fb079ed4640 1 --2- 172.21.15.107:0/1293681255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb074102200 0x7fb0741045f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb064009900 tx=0x7fb06402a410 comp rx=0 tx=0).stop 2024-04-25T00:14:15.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.151+0000 7fb079ed4640 1 -- 172.21.15.107:0/1293681255 shutdown_connections 2024-04-25T00:14:15.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.151+0000 7fb079ed4640 1 --2- 172.21.15.107:0/1293681255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb074107460 0x7fb0741098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.151+0000 7fb079ed4640 1 --2- 172.21.15.107:0/1293681255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb074104b30 0x7fb074106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.151+0000 7fb079ed4640 1 --2- 172.21.15.107:0/1293681255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb074102200 0x7fb0741045f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.151+0000 7fb079ed4640 1 -- 172.21.15.107:0/1293681255 >> 172.21.15.107:0/1293681255 conn(0x7fb0740fbbf0 msgr2=0x7fb0740fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:15.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.151+0000 7fb079ed4640 1 -- 172.21.15.107:0/1293681255 shutdown_connections 2024-04-25T00:14:15.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.151+0000 7fb079ed4640 1 -- 172.21.15.107:0/1293681255 wait complete. 2024-04-25T00:14:15.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.152+0000 7fb079ed4640 1 Processor -- start 2024-04-25T00:14:15.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.152+0000 7fb079ed4640 1 -- start start 2024-04-25T00:14:15.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.152+0000 7fb079ed4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb074102200 0x7fb074197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.152+0000 7fb079ed4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb074104b30 0x7fb074198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.152+0000 7fb079ed4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb074107460 0x7fb07419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.152+0000 7fb079ed4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb07410d430 con 0x7fb074104b30 2024-04-25T00:14:15.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.152+0000 7fb079ed4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb07410d2b0 con 0x7fb074107460 2024-04-25T00:14:15.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.152+0000 7fb079ed4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb07410d5b0 con 0x7fb074102200 2024-04-25T00:14:15.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.152+0000 7fb072ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb074104b30 0x7fb074198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.152+0000 7fb072ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb074104b30 0x7fb074198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46774/0 (socket says 172.21.15.107:46774) 2024-04-25T00:14:15.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.152+0000 7fb072ffd640 1 -- 172.21.15.107:0/2079447095 learned_addr learned my addr 172.21.15.107:0/2079447095 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.153+0000 7fb0737fe640 1 --2- 172.21.15.107:0/2079447095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb074102200 0x7fb074197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.153+0000 7fb073fff640 1 --2- 172.21.15.107:0/2079447095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb074107460 0x7fb07419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.153+0000 7fb072ffd640 1 -- 172.21.15.107:0/2079447095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb074102200 msgr2=0x7fb074197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.153+0000 7fb072ffd640 1 --2- 172.21.15.107:0/2079447095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb074102200 0x7fb074197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.153+0000 7fb072ffd640 1 -- 172.21.15.107:0/2079447095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb074107460 msgr2=0x7fb07419f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.153+0000 7fb072ffd640 1 --2- 172.21.15.107:0/2079447095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb074107460 0x7fb07419f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.153+0000 7fb072ffd640 1 -- 172.21.15.107:0/2079447095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07419f6e0 con 0x7fb074104b30 2024-04-25T00:14:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.153+0000 7fb073fff640 1 --2- 172.21.15.107:0/2079447095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb074107460 0x7fb07419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.153+0000 7fb0737fe640 1 --2- 172.21.15.107:0/2079447095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb074102200 0x7fb074197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.153+0000 7fb072ffd640 1 --2- 172.21.15.107:0/2079447095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb074104b30 0x7fb074198000 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb06800d740 tx=0x7fb06800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.153+0000 7fb070ff9640 1 -- 172.21.15.107:0/2079447095 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb068017070 con 0x7fb074104b30 2024-04-25T00:14:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.153+0000 7fb079ed4640 1 -- 172.21.15.107:0/2079447095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb07419f9d0 con 0x7fb074104b30 2024-04-25T00:14:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.153+0000 7fb070ff9640 1 -- 172.21.15.107:0/2079447095 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb06800de50 con 0x7fb074104b30 2024-04-25T00:14:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.153+0000 7fb070ff9640 1 -- 172.21.15.107:0/2079447095 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb068004e00 con 0x7fb074104b30 2024-04-25T00:14:15.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.153+0000 7fb079ed4640 1 -- 172.21.15.107:0/2079447095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb07419fed0 con 0x7fb074104b30 2024-04-25T00:14:15.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.154+0000 7fb079ed4640 1 -- 172.21.15.107:0/2079447095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0740fd6e0 con 0x7fb074104b30 2024-04-25T00:14:15.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.156+0000 7fb070ff9640 1 -- 172.21.15.107:0/2079447095 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb068004fa0 con 0x7fb074104b30 2024-04-25T00:14:15.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.157+0000 7fb070ff9640 1 --2- 172.21.15.107:0/2079447095 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb05c0a2b90 0x7fb05c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.157+0000 7fb0737fe640 1 --2- 172.21.15.107:0/2079447095 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb05c0a2b90 0x7fb05c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.157+0000 7fb070ff9640 1 -- 172.21.15.107:0/2079447095 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fb0680cf0e0 con 0x7fb074104b30 2024-04-25T00:14:15.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.157+0000 7fb0737fe640 1 --2- 172.21.15.107:0/2079447095 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb05c0a2b90 0x7fb05c0a5050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb064000c00 tx=0x7fb0640095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.160+0000 7fb070ff9640 1 -- 172.21.15.107:0/2079447095 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb06809be00 con 0x7fb074104b30 2024-04-25T00:14:15.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:14 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2336628462' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:15.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.341+0000 7fb079ed4640 1 -- 172.21.15.107:0/2079447095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb07402d3c0 con 0x7fb074104b30 2024-04-25T00:14:15.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.342+0000 7fb070ff9640 1 -- 172.21.15.107:0/2079447095 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7fb06809b700 con 0x7fb074104b30 2024-04-25T00:14:15.342 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-25T00:14:15.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.344+0000 7fb079ed4640 1 -- 172.21.15.107:0/2079447095 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb05c0a2b90 msgr2=0x7fb05c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.344+0000 7fb079ed4640 1 --2- 172.21.15.107:0/2079447095 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb05c0a2b90 0x7fb05c0a5050 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb064000c00 tx=0x7fb0640095f0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.344+0000 7fb079ed4640 1 -- 172.21.15.107:0/2079447095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb074104b30 msgr2=0x7fb074198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.344+0000 7fb079ed4640 1 --2- 172.21.15.107:0/2079447095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb074104b30 0x7fb074198000 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb06800d740 tx=0x7fb06800dc00 comp rx=0 tx=0).stop 2024-04-25T00:14:15.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.344+0000 7fb079ed4640 1 -- 172.21.15.107:0/2079447095 shutdown_connections 2024-04-25T00:14:15.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.344+0000 7fb079ed4640 1 --2- 172.21.15.107:0/2079447095 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb05c0a2b90 0x7fb05c0a5050 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.344+0000 7fb079ed4640 1 --2- 172.21.15.107:0/2079447095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb074107460 0x7fb07419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.344+0000 7fb079ed4640 1 --2- 172.21.15.107:0/2079447095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb074104b30 0x7fb074198000 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.344+0000 7fb079ed4640 1 --2- 172.21.15.107:0/2079447095 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb074102200 0x7fb074197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.344+0000 7fb079ed4640 1 -- 172.21.15.107:0/2079447095 >> 172.21.15.107:0/2079447095 conn(0x7fb0740fbbf0 msgr2=0x7fb074102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:15.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.344+0000 7fb079ed4640 1 -- 172.21.15.107:0/2079447095 shutdown_connections 2024-04-25T00:14:15.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.344+0000 7fb079ed4640 1 -- 172.21.15.107:0/2079447095 wait complete. 2024-04-25T00:14:15.355 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:14:15.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:14 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2336628462' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:15.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.457+0000 7f428401c640 1 -- 172.21.15.107:0/4260237770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f427c100680 msgr2=0x7f427c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.457+0000 7f428401c640 1 --2- 172.21.15.107:0/4260237770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f427c100680 0x7f427c100aa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4278009900 tx=0x7f427802a410 comp rx=0 tx=0).stop 2024-04-25T00:14:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.457+0000 7f428401c640 1 -- 172.21.15.107:0/4260237770 shutdown_connections 2024-04-25T00:14:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.457+0000 7f428401c640 1 --2- 172.21.15.107:0/4260237770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f427c102230 0x7f427c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.457+0000 7f428401c640 1 --2- 172.21.15.107:0/4260237770 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f427c101870 0x7f427c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.457+0000 7f428401c640 1 --2- 172.21.15.107:0/4260237770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f427c100680 0x7f427c100aa0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.457+0000 7f428401c640 1 -- 172.21.15.107:0/4260237770 >> 172.21.15.107:0/4260237770 conn(0x7f427c0fbbf0 msgr2=0x7f427c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.457+0000 7f428401c640 1 -- 172.21.15.107:0/4260237770 shutdown_connections 2024-04-25T00:14:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.457+0000 7f428401c640 1 -- 172.21.15.107:0/4260237770 wait complete. 2024-04-25T00:14:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.458+0000 7f428401c640 1 Processor -- start 2024-04-25T00:14:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.458+0000 7f428401c640 1 -- start start 2024-04-25T00:14:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.458+0000 7f428401c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f427c100680 0x7f427c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.458+0000 7f428401c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f427c101870 0x7f427c19c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.458+0000 7f428401c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f427c102230 0x7f427c19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.458+0000 7f428401c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f427c10c300 con 0x7f427c101870 2024-04-25T00:14:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.458+0000 7f428401c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f427c10c180 con 0x7f427c100680 2024-04-25T00:14:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.458+0000 7f428401c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f427c10c480 con 0x7f427c102230 2024-04-25T00:14:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.458+0000 7f42815b7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f427c101870 0x7f427c19c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.458+0000 7f42815b7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f427c101870 0x7f427c19c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46788/0 (socket says 172.21.15.107:46788) 2024-04-25T00:14:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.458+0000 7f42815b7640 1 -- 172.21.15.107:0/1248634532 learned_addr learned my addr 172.21.15.107:0/1248634532 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.458+0000 7f42825b9640 1 --2- 172.21.15.107:0/1248634532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f427c102230 0x7f427c19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.458+0000 7f4281db8640 1 --2- 172.21.15.107:0/1248634532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f427c100680 0x7f427c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.458+0000 7f42815b7640 1 -- 172.21.15.107:0/1248634532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f427c102230 msgr2=0x7f427c19ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.458+0000 7f42815b7640 1 --2- 172.21.15.107:0/1248634532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f427c102230 0x7f427c19ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.458+0000 7f42815b7640 1 -- 172.21.15.107:0/1248634532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f427c100680 msgr2=0x7f427c069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.458+0000 7f42815b7640 1 --2- 172.21.15.107:0/1248634532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f427c100680 0x7f427c069460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.458+0000 7f42815b7640 1 -- 172.21.15.107:0/1248634532 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427c19f730 con 0x7f427c101870 2024-04-25T00:14:15.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.459+0000 7f42815b7640 1 --2- 172.21.15.107:0/1248634532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f427c101870 0x7f427c19c830 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f426c00e9f0 tx=0x7f426c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:15.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.459+0000 7f4272ffd640 1 -- 172.21.15.107:0/1248634532 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c00cd80 con 0x7f427c101870 2024-04-25T00:14:15.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.459+0000 7f428401c640 1 -- 172.21.15.107:0/1248634532 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427c19fa20 con 0x7f427c101870 2024-04-25T00:14:15.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.459+0000 7f4272ffd640 1 -- 172.21.15.107:0/1248634532 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f426c005500 con 0x7f427c101870 2024-04-25T00:14:15.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.459+0000 7f4272ffd640 1 -- 172.21.15.107:0/1248634532 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c010690 con 0x7f427c101870 2024-04-25T00:14:15.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.459+0000 7f428401c640 1 -- 172.21.15.107:0/1248634532 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f427c19ff20 con 0x7f427c101870 2024-04-25T00:14:15.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.460+0000 7f428401c640 1 -- 172.21.15.107:0/1248634532 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f427c107a70 con 0x7f427c101870 2024-04-25T00:14:15.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.461+0000 7f4272ffd640 1 -- 172.21.15.107:0/1248634532 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f426c0046c0 con 0x7f427c101870 2024-04-25T00:14:15.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.462+0000 7f4272ffd640 1 --2- 172.21.15.107:0/1248634532 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f42680a2b90 0x7f42680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.462+0000 7f4281db8640 1 --2- 172.21.15.107:0/1248634532 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f42680a2b90 0x7f42680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.462+0000 7f4272ffd640 1 -- 172.21.15.107:0/1248634532 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f426c0cf660 con 0x7f427c101870 2024-04-25T00:14:15.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.463+0000 7f4281db8640 1 --2- 172.21.15.107:0/1248634532 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f42680a2b90 0x7f42680a5050 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4278005ab0 tx=0x7f4278000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:15.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.467+0000 7f4272ffd640 1 -- 172.21.15.107:0/1248634532 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f426c09c380 con 0x7f427c101870 2024-04-25T00:14:15.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.642+0000 7f428401c640 1 -- 172.21.15.107:0/1248634532 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f427c062340 con 0x7f427c101870 2024-04-25T00:14:15.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.643+0000 7f4272ffd640 1 -- 172.21.15.107:0/1248634532 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f426c09bc80 con 0x7f427c101870 2024-04-25T00:14:15.643 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-04-25T00:14:15.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.644+0000 7f428401c640 1 -- 172.21.15.107:0/1248634532 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f42680a2b90 msgr2=0x7f42680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.644+0000 7f428401c640 1 --2- 172.21.15.107:0/1248634532 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f42680a2b90 0x7f42680a5050 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4278005ab0 tx=0x7f4278000bc0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.644+0000 7f428401c640 1 -- 172.21.15.107:0/1248634532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f427c101870 msgr2=0x7f427c19c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.644+0000 7f428401c640 1 --2- 172.21.15.107:0/1248634532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f427c101870 0x7f427c19c830 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f426c00e9f0 tx=0x7f426c00edf0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.644+0000 7f428401c640 1 -- 172.21.15.107:0/1248634532 shutdown_connections 2024-04-25T00:14:15.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.644+0000 7f428401c640 1 --2- 172.21.15.107:0/1248634532 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f427c102230 0x7f427c19ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.645+0000 7f428401c640 1 --2- 172.21.15.107:0/1248634532 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f427c101870 0x7f427c19c830 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.645+0000 7f428401c640 1 --2- 172.21.15.107:0/1248634532 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f42680a2b90 0x7f42680a5050 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.645+0000 7f428401c640 1 --2- 172.21.15.107:0/1248634532 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f427c100680 0x7f427c069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.645+0000 7f428401c640 1 -- 172.21.15.107:0/1248634532 >> 172.21.15.107:0/1248634532 conn(0x7f427c0fbbf0 msgr2=0x7f427c104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:15.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.645+0000 7f428401c640 1 -- 172.21.15.107:0/1248634532 shutdown_connections 2024-04-25T00:14:15.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.645+0000 7f428401c640 1 -- 172.21.15.107:0/1248634532 wait complete. 2024-04-25T00:14:15.655 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:14:15.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.757+0000 7fae25990640 1 -- 172.21.15.107:0/1354186806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae20100690 msgr2=0x7fae20100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.757+0000 7fae25990640 1 --2- 172.21.15.107:0/1354186806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae20100690 0x7fae20100ab0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fae10009900 tx=0x7fae1002a410 comp rx=0 tx=0).stop 2024-04-25T00:14:15.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.757+0000 7fae25990640 1 -- 172.21.15.107:0/1354186806 shutdown_connections 2024-04-25T00:14:15.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.757+0000 7fae25990640 1 --2- 172.21.15.107:0/1354186806 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae20102240 0x7fae20108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.757+0000 7fae25990640 1 --2- 172.21.15.107:0/1354186806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae20101880 0x7fae20101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.757+0000 7fae25990640 1 --2- 172.21.15.107:0/1354186806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae20100690 0x7fae20100ab0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.757+0000 7fae25990640 1 -- 172.21.15.107:0/1354186806 >> 172.21.15.107:0/1354186806 conn(0x7fae200fbc00 msgr2=0x7fae200fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:15.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.757+0000 7fae25990640 1 -- 172.21.15.107:0/1354186806 shutdown_connections 2024-04-25T00:14:15.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.757+0000 7fae25990640 1 -- 172.21.15.107:0/1354186806 wait complete. 2024-04-25T00:14:15.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.757+0000 7fae25990640 1 Processor -- start 2024-04-25T00:14:15.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.758+0000 7fae25990640 1 -- start start 2024-04-25T00:14:15.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.758+0000 7fae25990640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae20100690 0x7fae2006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.758+0000 7fae25990640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae20101880 0x7fae2019c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.758+0000 7fae25990640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae20102240 0x7fae2019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.758+0000 7fae25990640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fae2010c2e0 con 0x7fae20101880 2024-04-25T00:14:15.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.758+0000 7fae25990640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fae2010c160 con 0x7fae20100690 2024-04-25T00:14:15.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.758+0000 7fae25990640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fae2010c460 con 0x7fae20102240 2024-04-25T00:14:15.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.758+0000 7fae17fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae20101880 0x7fae2019c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.758+0000 7fae17fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae20101880 0x7fae2019c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46814/0 (socket says 172.21.15.107:46814) 2024-04-25T00:14:15.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.758+0000 7fae17fff640 1 -- 172.21.15.107:0/1225473790 learned_addr learned my addr 172.21.15.107:0/1225473790 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:15.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.758+0000 7fae1effd640 1 --2- 172.21.15.107:0/1225473790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae20100690 0x7fae2006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.758+0000 7fae1f7fe640 1 --2- 172.21.15.107:0/1225473790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae20102240 0x7fae2019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.758+0000 7fae17fff640 1 -- 172.21.15.107:0/1225473790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae20102240 msgr2=0x7fae2019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.758+0000 7fae17fff640 1 --2- 172.21.15.107:0/1225473790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae20102240 0x7fae2019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.758+0000 7fae17fff640 1 -- 172.21.15.107:0/1225473790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae20100690 msgr2=0x7fae2006a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.758+0000 7fae17fff640 1 --2- 172.21.15.107:0/1225473790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae20100690 0x7fae2006a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.758+0000 7fae17fff640 1 -- 172.21.15.107:0/1225473790 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae2019f760 con 0x7fae20101880 2024-04-25T00:14:15.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.758+0000 7fae1effd640 1 --2- 172.21.15.107:0/1225473790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae20100690 0x7fae2006a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:15.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.758+0000 7fae17fff640 1 --2- 172.21.15.107:0/1225473790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae20101880 0x7fae2019c860 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fae0800d950 tx=0x7fae0800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:15.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.758+0000 7fae2498e640 1 -- 172.21.15.107:0/1225473790 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae08017070 con 0x7fae20101880 2024-04-25T00:14:15.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.758+0000 7fae2498e640 1 -- 172.21.15.107:0/1225473790 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae08009c30 con 0x7fae20101880 2024-04-25T00:14:15.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.758+0000 7fae25990640 1 -- 172.21.15.107:0/1225473790 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae2019fa50 con 0x7fae20101880 2024-04-25T00:14:15.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.758+0000 7fae2498e640 1 -- 172.21.15.107:0/1225473790 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0800f6b0 con 0x7fae20101880 2024-04-25T00:14:15.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.758+0000 7fae25990640 1 -- 172.21.15.107:0/1225473790 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae2019ffa0 con 0x7fae20101880 2024-04-25T00:14:15.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.759+0000 7fae25990640 1 -- 172.21.15.107:0/1225473790 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae20107a70 con 0x7fae20101880 2024-04-25T00:14:15.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.760+0000 7fae2498e640 1 -- 172.21.15.107:0/1225473790 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fae080109d0 con 0x7fae20101880 2024-04-25T00:14:15.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.761+0000 7fae2498e640 1 --2- 172.21.15.107:0/1225473790 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fae0c0a2b90 0x7fae0c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:15.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.761+0000 7fae1effd640 1 --2- 172.21.15.107:0/1225473790 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fae0c0a2b90 0x7fae0c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:15.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.761+0000 7fae2498e640 1 -- 172.21.15.107:0/1225473790 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fae080cefc0 con 0x7fae20101880 2024-04-25T00:14:15.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.761+0000 7fae1effd640 1 --2- 172.21.15.107:0/1225473790 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fae0c0a2b90 0x7fae0c0a5050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fae10006010 tx=0x7fae10032cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:15.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.765+0000 7fae2498e640 1 -- 172.21.15.107:0/1225473790 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fae08009de0 con 0x7fae20101880 2024-04-25T00:14:15.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.941+0000 7fae25990640 1 -- 172.21.15.107:0/1225473790 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fae20062370 con 0x7fae20101880 2024-04-25T00:14:15.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.941+0000 7fae2498e640 1 -- 172.21.15.107:0/1225473790 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7fae0809bce0 con 0x7fae20101880 2024-04-25T00:14:15.942 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-25T00:14:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.943+0000 7fae25990640 1 -- 172.21.15.107:0/1225473790 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fae0c0a2b90 msgr2=0x7fae0c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.943+0000 7fae25990640 1 --2- 172.21.15.107:0/1225473790 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fae0c0a2b90 0x7fae0c0a5050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fae10006010 tx=0x7fae10032cf0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.943+0000 7fae25990640 1 -- 172.21.15.107:0/1225473790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae20101880 msgr2=0x7fae2019c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.943+0000 7fae25990640 1 --2- 172.21.15.107:0/1225473790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae20101880 0x7fae2019c860 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fae0800d950 tx=0x7fae0800de10 comp rx=0 tx=0).stop 2024-04-25T00:14:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.943+0000 7fae25990640 1 -- 172.21.15.107:0/1225473790 shutdown_connections 2024-04-25T00:14:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.943+0000 7fae25990640 1 --2- 172.21.15.107:0/1225473790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae20102240 0x7fae2019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.943+0000 7fae25990640 1 --2- 172.21.15.107:0/1225473790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae20101880 0x7fae2019c860 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.943+0000 7fae25990640 1 --2- 172.21.15.107:0/1225473790 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fae0c0a2b90 0x7fae0c0a5050 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.943+0000 7fae25990640 1 --2- 172.21.15.107:0/1225473790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae20100690 0x7fae2006a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.943+0000 7fae25990640 1 -- 172.21.15.107:0/1225473790 >> 172.21.15.107:0/1225473790 conn(0x7fae200fbc00 msgr2=0x7fae20104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:15.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.943+0000 7fae25990640 1 -- 172.21.15.107:0/1225473790 shutdown_connections 2024-04-25T00:14:15.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:15.943+0000 7fae25990640 1 -- 172.21.15.107:0/1225473790 wait complete. 2024-04-25T00:14:15.955 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:14:16.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.053+0000 7f63b8412640 1 -- 172.21.15.107:0/2914538019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63b0101790 msgr2=0x7f63b0101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:16.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.053+0000 7f63b8412640 1 --2- 172.21.15.107:0/2914538019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63b0101790 0x7f63b0101bb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009900 tx=0x7f63ac02a410 comp rx=0 tx=0).stop 2024-04-25T00:14:16.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.053+0000 7f63b8412640 1 -- 172.21.15.107:0/2914538019 shutdown_connections 2024-04-25T00:14:16.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.053+0000 7f63b8412640 1 --2- 172.21.15.107:0/2914538019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63b0103340 0x7f63b0109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.053+0000 7f63b8412640 1 --2- 172.21.15.107:0/2914538019 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63b0102980 0x7f63b0102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.053+0000 7f63b8412640 1 --2- 172.21.15.107:0/2914538019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63b0101790 0x7f63b0101bb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.053+0000 7f63b8412640 1 -- 172.21.15.107:0/2914538019 >> 172.21.15.107:0/2914538019 conn(0x7f63b00fcd20 msgr2=0x7f63b00ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:16.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.053+0000 7f63b8412640 1 -- 172.21.15.107:0/2914538019 shutdown_connections 2024-04-25T00:14:16.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.053+0000 7f63b8412640 1 -- 172.21.15.107:0/2914538019 wait complete. 2024-04-25T00:14:16.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.053+0000 7f63b8412640 1 Processor -- start 2024-04-25T00:14:16.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.053+0000 7f63b8412640 1 -- start start 2024-04-25T00:14:16.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.054+0000 7f63b8412640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63b0101790 0x7f63b0197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:16.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.054+0000 7f63b8412640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63b0102980 0x7f63b01981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.054+0000 7f63b8412640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63b0103340 0x7f63b019f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.054+0000 7f63b8412640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f63b010d4e0 con 0x7f63b0103340 2024-04-25T00:14:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.054+0000 7f63b8412640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f63b010d360 con 0x7f63b0102980 2024-04-25T00:14:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.054+0000 7f63b8412640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f63b010d660 con 0x7f63b0101790 2024-04-25T00:14:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.054+0000 7f63b69af640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63b0103340 0x7f63b019f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.054+0000 7f63b69af640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63b0103340 0x7f63b019f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46826/0 (socket says 172.21.15.107:46826) 2024-04-25T00:14:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.054+0000 7f63b69af640 1 -- 172.21.15.107:0/3402710031 learned_addr learned my addr 172.21.15.107:0/3402710031 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.054+0000 7f63b61ae640 1 --2- 172.21.15.107:0/3402710031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63b0101790 0x7f63b0197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.054+0000 7f63b59ad640 1 --2- 172.21.15.107:0/3402710031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63b0102980 0x7f63b01981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.054+0000 7f63b69af640 1 -- 172.21.15.107:0/3402710031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63b0101790 msgr2=0x7f63b0197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.054+0000 7f63b69af640 1 --2- 172.21.15.107:0/3402710031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63b0101790 0x7f63b0197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.054+0000 7f63b69af640 1 -- 172.21.15.107:0/3402710031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63b0102980 msgr2=0x7f63b01981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.054+0000 7f63b69af640 1 --2- 172.21.15.107:0/3402710031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63b0102980 0x7f63b01981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.054+0000 7f63b69af640 1 -- 172.21.15.107:0/3402710031 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b019f8d0 con 0x7f63b0103340 2024-04-25T00:14:16.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.054+0000 7f63b61ae640 1 --2- 172.21.15.107:0/3402710031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63b0101790 0x7f63b0197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:16.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.054+0000 7f63b69af640 1 --2- 172.21.15.107:0/3402710031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63b0103340 0x7f63b019f230 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f63a800b970 tx=0x7f63a800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:16.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.054+0000 7f63a77fe640 1 -- 172.21.15.107:0/3402710031 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8019070 con 0x7f63b0103340 2024-04-25T00:14:16.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.054+0000 7f63b8412640 1 -- 172.21.15.107:0/3402710031 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b019fbc0 con 0x7f63b0103340 2024-04-25T00:14:16.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.054+0000 7f63a77fe640 1 -- 172.21.15.107:0/3402710031 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a800cb40 con 0x7f63b0103340 2024-04-25T00:14:16.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.054+0000 7f63a77fe640 1 -- 172.21.15.107:0/3402710031 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8007650 con 0x7f63b0103340 2024-04-25T00:14:16.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.054+0000 7f63b8412640 1 -- 172.21.15.107:0/3402710031 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b01a0140 con 0x7f63b0103340 2024-04-25T00:14:16.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.055+0000 7f63b8412640 1 -- 172.21.15.107:0/3402710031 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b0108ba0 con 0x7f63b0103340 2024-04-25T00:14:16.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.057+0000 7f63a77fe640 1 -- 172.21.15.107:0/3402710031 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f63a801c070 con 0x7f63b0103340 2024-04-25T00:14:16.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.057+0000 7f63a77fe640 1 --2- 172.21.15.107:0/3402710031 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f639c0a2b90 0x7f639c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:16.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.057+0000 7f63b61ae640 1 --2- 172.21.15.107:0/3402710031 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f639c0a2b90 0x7f639c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:16.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.057+0000 7f63a77fe640 1 -- 172.21.15.107:0/3402710031 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f63a80ced90 con 0x7f63b0103340 2024-04-25T00:14:16.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.058+0000 7f63b61ae640 1 --2- 172.21.15.107:0/3402710031 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f639c0a2b90 0x7f639c0a5050 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0314e0 tx=0x7f63ac000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:16.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.061+0000 7f63a77fe640 1 -- 172.21.15.107:0/3402710031 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f63a809bab0 con 0x7f63b0103340 2024-04-25T00:14:16.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:15 smithi107 ceph-mon[29153]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:14:16.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:15 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3786831033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:14:16.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:15 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2079447095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:14:16.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:15 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1248634532' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:14:16.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:15 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1225473790' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:14:16.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.250+0000 7f63b8412640 1 -- 172.21.15.107:0/3402710031 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f63b002d170 con 0x7f63b0103340 2024-04-25T00:14:16.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.250+0000 7f63a77fe640 1 -- 172.21.15.107:0/3402710031 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) ==== 81+0+596 (secure 0 0 0) 0x7f63a809b3b0 con 0x7f63b0103340 2024-04-25T00:14:16.251 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1174178948 2024-04-26T00:07:00.838246+0000 2024-04-25T00:14:16.251 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/153169302 2024-04-26T00:07:00.838246+0000 2024-04-25T00:14:16.251 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/120616774 2024-04-26T00:06:34.124795+0000 2024-04-25T00:14:16.251 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/120616774 2024-04-26T00:06:34.124795+0000 2024-04-25T00:14:16.251 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/971124970 2024-04-26T00:06:34.124795+0000 2024-04-25T00:14:16.251 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3022844652 2024-04-26T00:06:34.124795+0000 2024-04-25T00:14:16.251 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/153169302 2024-04-26T00:07:00.838246+0000 2024-04-25T00:14:16.251 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/4215203700 2024-04-26T00:07:00.838246+0000 2024-04-25T00:14:16.251 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/750288374 2024-04-26T00:06:34.124795+0000 2024-04-25T00:14:16.251 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3718680642 2024-04-26T00:07:00.838246+0000 2024-04-25T00:14:16.251 INFO:teuthology.orchestra.run.smithi107.stderr:listed 10 entries 2024-04-25T00:14:16.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.252+0000 7f63b8412640 1 -- 172.21.15.107:0/3402710031 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f639c0a2b90 msgr2=0x7f639c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:16.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.252+0000 7f63b8412640 1 --2- 172.21.15.107:0/3402710031 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f639c0a2b90 0x7f639c0a5050 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0314e0 tx=0x7f63ac000bc0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.252+0000 7f63b8412640 1 -- 172.21.15.107:0/3402710031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63b0103340 msgr2=0x7f63b019f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:16.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.252+0000 7f63b8412640 1 --2- 172.21.15.107:0/3402710031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63b0103340 0x7f63b019f230 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f63a800b970 tx=0x7f63a800be30 comp rx=0 tx=0).stop 2024-04-25T00:14:16.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.252+0000 7f63b8412640 1 -- 172.21.15.107:0/3402710031 shutdown_connections 2024-04-25T00:14:16.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.252+0000 7f63b8412640 1 --2- 172.21.15.107:0/3402710031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63b0103340 0x7f63b019f230 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.252+0000 7f63b8412640 1 --2- 172.21.15.107:0/3402710031 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f639c0a2b90 0x7f639c0a5050 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.252+0000 7f63b8412640 1 --2- 172.21.15.107:0/3402710031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63b0102980 0x7f63b01981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.252+0000 7f63b8412640 1 --2- 172.21.15.107:0/3402710031 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f63b0101790 0x7f63b0197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.252+0000 7f63b8412640 1 -- 172.21.15.107:0/3402710031 >> 172.21.15.107:0/3402710031 conn(0x7f63b00fcd20 msgr2=0x7f63b0105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:16.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.252+0000 7f63b8412640 1 -- 172.21.15.107:0/3402710031 shutdown_connections 2024-04-25T00:14:16.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.252+0000 7f63b8412640 1 -- 172.21.15.107:0/3402710031 wait complete. 2024-04-25T00:14:16.264 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T00:14:16.264 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:14:16.280 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:16.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:15 smithi195 ceph-mon[32833]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:14:16.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:15 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3786831033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:14:16.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:15 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2079447095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:14:16.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:15 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1248634532' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:14:16.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:15 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1225473790' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:14:16.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.417+0000 7fcf147d5640 1 -- 172.21.15.107:0/2136683206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf0c104bb0 msgr2=0x7fcf0c106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:16.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.417+0000 7fcf147d5640 1 --2- 172.21.15.107:0/2136683206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf0c104bb0 0x7fcf0c106fa0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009900 tx=0x7fcefc02a410 comp rx=0 tx=0).stop 2024-04-25T00:14:16.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.417+0000 7fcf147d5640 1 -- 172.21.15.107:0/2136683206 shutdown_connections 2024-04-25T00:14:16.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.417+0000 7fcf147d5640 1 --2- 172.21.15.107:0/2136683206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf0c1074e0 0x7fcf0c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.417+0000 7fcf147d5640 1 --2- 172.21.15.107:0/2136683206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf0c104bb0 0x7fcf0c106fa0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.417+0000 7fcf147d5640 1 --2- 172.21.15.107:0/2136683206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf0c102280 0x7fcf0c104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.417+0000 7fcf147d5640 1 -- 172.21.15.107:0/2136683206 >> 172.21.15.107:0/2136683206 conn(0x7fcf0c0fbc30 msgr2=0x7fcf0c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:16.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.417+0000 7fcf147d5640 1 -- 172.21.15.107:0/2136683206 shutdown_connections 2024-04-25T00:14:16.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.417+0000 7fcf147d5640 1 -- 172.21.15.107:0/2136683206 wait complete. 2024-04-25T00:14:16.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.418+0000 7fcf147d5640 1 Processor -- start 2024-04-25T00:14:16.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.418+0000 7fcf147d5640 1 -- start start 2024-04-25T00:14:16.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.418+0000 7fcf147d5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf0c102280 0x7fcf0c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:16.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.418+0000 7fcf147d5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf0c104bb0 0x7fcf0c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:16.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.418+0000 7fcf147d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf0c1074e0 0x7fcf0c19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:16.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.418+0000 7fcf147d5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c10d410 con 0x7fcf0c1074e0 2024-04-25T00:14:16.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.418+0000 7fcf147d5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c10d290 con 0x7fcf0c102280 2024-04-25T00:14:16.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.418+0000 7fcf147d5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c10d590 con 0x7fcf0c104bb0 2024-04-25T00:14:16.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.418+0000 7fcf12d72640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf0c1074e0 0x7fcf0c19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:16.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.418+0000 7fcf12d72640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf0c1074e0 0x7fcf0c19f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46842/0 (socket says 172.21.15.107:46842) 2024-04-25T00:14:16.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.418+0000 7fcf12d72640 1 -- 172.21.15.107:0/2215195076 learned_addr learned my addr 172.21.15.107:0/2215195076 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:16.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.418+0000 7fcf11d70640 1 --2- 172.21.15.107:0/2215195076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf0c104bb0 0x7fcf0c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:16.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.418+0000 7fcf12571640 1 --2- 172.21.15.107:0/2215195076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf0c102280 0x7fcf0c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:16.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.418+0000 7fcf12d72640 1 -- 172.21.15.107:0/2215195076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf0c104bb0 msgr2=0x7fcf0c198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:16.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.418+0000 7fcf12d72640 1 --2- 172.21.15.107:0/2215195076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf0c104bb0 0x7fcf0c198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.418+0000 7fcf12d72640 1 -- 172.21.15.107:0/2215195076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf0c102280 msgr2=0x7fcf0c197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:16.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.418+0000 7fcf12d72640 1 --2- 172.21.15.107:0/2215195076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf0c102280 0x7fcf0c197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.418+0000 7fcf12d72640 1 -- 172.21.15.107:0/2215195076 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0c19f710 con 0x7fcf0c1074e0 2024-04-25T00:14:16.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.418+0000 7fcf12571640 1 --2- 172.21.15.107:0/2215195076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf0c102280 0x7fcf0c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:16.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.418+0000 7fcf12d72640 1 --2- 172.21.15.107:0/2215195076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf0c1074e0 0x7fcf0c19f0a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400eaa0 tx=0x7fcf0400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:16.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.418+0000 7fcf037fe640 1 -- 172.21.15.107:0/2215195076 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0400cdb0 con 0x7fcf0c1074e0 2024-04-25T00:14:16.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.418+0000 7fcf147d5640 1 -- 172.21.15.107:0/2215195076 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf0c19fa00 con 0x7fcf0c1074e0 2024-04-25T00:14:16.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.418+0000 7fcf037fe640 1 -- 172.21.15.107:0/2215195076 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf04005020 con 0x7fcf0c1074e0 2024-04-25T00:14:16.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.418+0000 7fcf037fe640 1 -- 172.21.15.107:0/2215195076 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf040116b0 con 0x7fcf0c1074e0 2024-04-25T00:14:16.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.419+0000 7fcf147d5640 1 -- 172.21.15.107:0/2215195076 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf0c19fed0 con 0x7fcf0c1074e0 2024-04-25T00:14:16.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.420+0000 7fcf147d5640 1 -- 172.21.15.107:0/2215195076 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf0c0fd720 con 0x7fcf0c1074e0 2024-04-25T00:14:16.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.420+0000 7fcf037fe640 1 -- 172.21.15.107:0/2215195076 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fcf04005560 con 0x7fcf0c1074e0 2024-04-25T00:14:16.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.421+0000 7fcf037fe640 1 --2- 172.21.15.107:0/2215195076 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcef80a2b90 0x7fcef80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:16.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.421+0000 7fcf12571640 1 --2- 172.21.15.107:0/2215195076 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcef80a2b90 0x7fcef80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:16.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.421+0000 7fcf037fe640 1 -- 172.21.15.107:0/2215195076 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fcf040cf4d0 con 0x7fcf0c1074e0 2024-04-25T00:14:16.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.421+0000 7fcf12571640 1 --2- 172.21.15.107:0/2215195076 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcef80a2b90 0x7fcef80a5050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcf08005fd0 tx=0x7fcf08005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:16.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.426+0000 7fcf037fe640 1 -- 172.21.15.107:0/2215195076 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fcf04016030 con 0x7fcf0c1074e0 2024-04-25T00:14:16.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:15 smithi170 ceph-mon[32807]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:14:16.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:15 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3786831033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:14:16.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:15 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2079447095' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:14:16.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:15 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1248634532' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:14:16.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:15 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1225473790' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:14:16.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.603+0000 7fcf147d5640 1 -- 172.21.15.107:0/2215195076 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf0c0623e0 con 0x7fcf0c1074e0 2024-04-25T00:14:16.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.605+0000 7fcf037fe640 1 -- 172.21.15.107:0/2215195076 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24134 (secure 0 0 0) 0x7fcf0409c1f0 con 0x7fcf0c1074e0 2024-04-25T00:14:16.605 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:14:16.605 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":92,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:14:08.839737+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:59.672801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:00.957782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:14:05.065846+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:16.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.606+0000 7fcf147d5640 1 -- 172.21.15.107:0/2215195076 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcef80a2b90 msgr2=0x7fcef80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:16.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.606+0000 7fcf147d5640 1 --2- 172.21.15.107:0/2215195076 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcef80a2b90 0x7fcef80a5050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcf08005fd0 tx=0x7fcf08005ea0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.606+0000 7fcf147d5640 1 -- 172.21.15.107:0/2215195076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf0c1074e0 msgr2=0x7fcf0c19f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:16.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.606+0000 7fcf147d5640 1 --2- 172.21.15.107:0/2215195076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf0c1074e0 0x7fcf0c19f0a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400eaa0 tx=0x7fcf0400ef60 comp rx=0 tx=0).stop 2024-04-25T00:14:16.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.607+0000 7fcf147d5640 1 -- 172.21.15.107:0/2215195076 shutdown_connections 2024-04-25T00:14:16.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.607+0000 7fcf147d5640 1 --2- 172.21.15.107:0/2215195076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf0c1074e0 0x7fcf0c19f0a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.607+0000 7fcf147d5640 1 --2- 172.21.15.107:0/2215195076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf0c104bb0 0x7fcf0c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.607+0000 7fcf147d5640 1 --2- 172.21.15.107:0/2215195076 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fcef80a2b90 0x7fcef80a5050 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.607+0000 7fcf147d5640 1 --2- 172.21.15.107:0/2215195076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf0c102280 0x7fcf0c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.607+0000 7fcf147d5640 1 -- 172.21.15.107:0/2215195076 >> 172.21.15.107:0/2215195076 conn(0x7fcf0c0fbc30 msgr2=0x7fcf0c102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:16.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.607+0000 7fcf147d5640 1 -- 172.21.15.107:0/2215195076 shutdown_connections 2024-04-25T00:14:16.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.607+0000 7fcf147d5640 1 -- 172.21.15.107:0/2215195076 wait complete. 2024-04-25T00:14:16.618 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:09:25.160329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:13:59.672801+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:14:00.957782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:14:05.065846+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:14:16.618 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:14:16.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.718+0000 7f237e231640 1 -- 172.21.15.107:0/1272302126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2378100680 msgr2=0x7f2378100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:16.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.718+0000 7f237e231640 1 --2- 172.21.15.107:0/1272302126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2378100680 0x7f2378100aa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2368009900 tx=0x7f236802a410 comp rx=0 tx=0).stop 2024-04-25T00:14:16.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.718+0000 7f237e231640 1 -- 172.21.15.107:0/1272302126 shutdown_connections 2024-04-25T00:14:16.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.718+0000 7f237e231640 1 --2- 172.21.15.107:0/1272302126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2378102230 0x7f2378108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.718+0000 7f237e231640 1 --2- 172.21.15.107:0/1272302126 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2378101870 0x7f2378101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.718+0000 7f237e231640 1 --2- 172.21.15.107:0/1272302126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2378100680 0x7f2378100aa0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.718+0000 7f237e231640 1 -- 172.21.15.107:0/1272302126 >> 172.21.15.107:0/1272302126 conn(0x7f23780fbbf0 msgr2=0x7f23780fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:16.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.718+0000 7f237e231640 1 -- 172.21.15.107:0/1272302126 shutdown_connections 2024-04-25T00:14:16.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.718+0000 7f237e231640 1 -- 172.21.15.107:0/1272302126 wait complete. 2024-04-25T00:14:16.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.718+0000 7f237e231640 1 Processor -- start 2024-04-25T00:14:16.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.719+0000 7f237e231640 1 -- start start 2024-04-25T00:14:16.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.719+0000 7f237e231640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2378100680 0x7f237806a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:16.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.719+0000 7f237e231640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2378101870 0x7f237819c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:16.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.719+0000 7f237e231640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2378102230 0x7f237819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:16.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.719+0000 7f237e231640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f237810c300 con 0x7f2378100680 2024-04-25T00:14:16.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.719+0000 7f237e231640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f237810c180 con 0x7f2378101870 2024-04-25T00:14:16.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.719+0000 7f237e231640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f237810c480 con 0x7f2378102230 2024-04-25T00:14:16.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.719+0000 7f23777fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2378100680 0x7f237806a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:16.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.719+0000 7f23777fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2378100680 0x7f237806a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46866/0 (socket says 172.21.15.107:46866) 2024-04-25T00:14:16.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.719+0000 7f23777fe640 1 -- 172.21.15.107:0/2196417454 learned_addr learned my addr 172.21.15.107:0/2196417454 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:16.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.719+0000 7f236ffff640 1 --2- 172.21.15.107:0/2196417454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2378101870 0x7f237819c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:16.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.719+0000 7f2377fff640 1 --2- 172.21.15.107:0/2196417454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2378102230 0x7f237819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:16.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.719+0000 7f23777fe640 1 -- 172.21.15.107:0/2196417454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2378102230 msgr2=0x7f237819ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:16.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.719+0000 7f23777fe640 1 --2- 172.21.15.107:0/2196417454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2378102230 0x7f237819ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.719+0000 7f23777fe640 1 -- 172.21.15.107:0/2196417454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2378101870 msgr2=0x7f237819c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:16.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.719+0000 7f23777fe640 1 --2- 172.21.15.107:0/2196417454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2378101870 0x7f237819c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.719+0000 7f23777fe640 1 -- 172.21.15.107:0/2196417454 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f237819f730 con 0x7f2378100680 2024-04-25T00:14:16.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.719+0000 7f23777fe640 1 --2- 172.21.15.107:0/2196417454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2378100680 0x7f237806a8d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2368031610 tx=0x7f2368004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:16.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.719+0000 7f23757fa640 1 -- 172.21.15.107:0/2196417454 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236803f030 con 0x7f2378100680 2024-04-25T00:14:16.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.719+0000 7f23757fa640 1 -- 172.21.15.107:0/2196417454 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2368003ad0 con 0x7f2378100680 2024-04-25T00:14:16.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.719+0000 7f237e231640 1 -- 172.21.15.107:0/2196417454 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f237819f960 con 0x7f2378100680 2024-04-25T00:14:16.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.719+0000 7f23757fa640 1 -- 172.21.15.107:0/2196417454 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368035070 con 0x7f2378100680 2024-04-25T00:14:16.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.719+0000 7f237e231640 1 -- 172.21.15.107:0/2196417454 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f237819fe50 con 0x7f2378100680 2024-04-25T00:14:16.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.720+0000 7f237e231640 1 -- 172.21.15.107:0/2196417454 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2378107a20 con 0x7f2378100680 2024-04-25T00:14:16.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.721+0000 7f23757fa640 1 -- 172.21.15.107:0/2196417454 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f2368042030 con 0x7f2378100680 2024-04-25T00:14:16.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.722+0000 7f23757fa640 1 --2- 172.21.15.107:0/2196417454 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f23640a2b90 0x7f23640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:16.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.722+0000 7f236ffff640 1 --2- 172.21.15.107:0/2196417454 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f23640a2b90 0x7f23640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:16.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.722+0000 7f23757fa640 1 -- 172.21.15.107:0/2196417454 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f23680ef9b0 con 0x7f2378100680 2024-04-25T00:14:16.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.723+0000 7f236ffff640 1 --2- 172.21.15.107:0/2196417454 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f23640a2b90 0x7f23640a5050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f237806a640 tx=0x7f236000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:16.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.727+0000 7f23757fa640 1 -- 172.21.15.107:0/2196417454 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f23680bc6d0 con 0x7f2378100680 2024-04-25T00:14:16.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.902+0000 7f237e231640 1 -- 172.21.15.107:0/2196417454 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2378062340 con 0x7f2378100680 2024-04-25T00:14:16.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.903+0000 7f23757fa640 1 -- 172.21.15.107:0/2196417454 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f23680bbfd0 con 0x7f2378100680 2024-04-25T00:14:16.903 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-04-25T00:14:16.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.905+0000 7f237e231640 1 -- 172.21.15.107:0/2196417454 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f23640a2b90 msgr2=0x7f23640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:16.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.905+0000 7f237e231640 1 --2- 172.21.15.107:0/2196417454 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f23640a2b90 0x7f23640a5050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f237806a640 tx=0x7f236000b410 comp rx=0 tx=0).stop 2024-04-25T00:14:16.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.905+0000 7f237e231640 1 -- 172.21.15.107:0/2196417454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2378100680 msgr2=0x7f237806a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:16.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.905+0000 7f237e231640 1 --2- 172.21.15.107:0/2196417454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2378100680 0x7f237806a8d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2368031610 tx=0x7f2368004270 comp rx=0 tx=0).stop 2024-04-25T00:14:16.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.905+0000 7f237e231640 1 -- 172.21.15.107:0/2196417454 shutdown_connections 2024-04-25T00:14:16.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.905+0000 7f237e231640 1 --2- 172.21.15.107:0/2196417454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2378102230 0x7f237819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.905+0000 7f237e231640 1 --2- 172.21.15.107:0/2196417454 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f23640a2b90 0x7f23640a5050 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.905+0000 7f237e231640 1 --2- 172.21.15.107:0/2196417454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2378101870 0x7f237819c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.905+0000 7f237e231640 1 --2- 172.21.15.107:0/2196417454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2378100680 0x7f237806a8d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:16.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.905+0000 7f237e231640 1 -- 172.21.15.107:0/2196417454 >> 172.21.15.107:0/2196417454 conn(0x7f23780fbbf0 msgr2=0x7f2378104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:16.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.905+0000 7f237e231640 1 -- 172.21.15.107:0/2196417454 shutdown_connections 2024-04-25T00:14:16.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:16.905+0000 7f237e231640 1 -- 172.21.15.107:0/2196417454 wait complete. 2024-04-25T00:14:16.916 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:14:17.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.017+0000 7fc5d32cd640 1 -- 172.21.15.107:0/3550148098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5cc102980 msgr2=0x7fc5cc102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.017+0000 7fc5d32cd640 1 --2- 172.21.15.107:0/3550148098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5cc102980 0x7fc5cc102e00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009900 tx=0x7fc5bc02a410 comp rx=0 tx=0).stop 2024-04-25T00:14:17.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.017+0000 7fc5d32cd640 1 -- 172.21.15.107:0/3550148098 shutdown_connections 2024-04-25T00:14:17.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.017+0000 7fc5d32cd640 1 --2- 172.21.15.107:0/3550148098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5cc103340 0x7fc5cc109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.017+0000 7fc5d32cd640 1 --2- 172.21.15.107:0/3550148098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5cc102980 0x7fc5cc102e00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.017+0000 7fc5d32cd640 1 --2- 172.21.15.107:0/3550148098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5cc101790 0x7fc5cc101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.017+0000 7fc5d32cd640 1 -- 172.21.15.107:0/3550148098 >> 172.21.15.107:0/3550148098 conn(0x7fc5cc0fcd20 msgr2=0x7fc5cc0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:17.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.017+0000 7fc5d32cd640 1 -- 172.21.15.107:0/3550148098 shutdown_connections 2024-04-25T00:14:17.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.017+0000 7fc5d32cd640 1 -- 172.21.15.107:0/3550148098 wait complete. 2024-04-25T00:14:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.018+0000 7fc5d32cd640 1 Processor -- start 2024-04-25T00:14:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.018+0000 7fc5d32cd640 1 -- start start 2024-04-25T00:14:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.018+0000 7fc5d32cd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5cc101790 0x7fc5cc197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.018+0000 7fc5d32cd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5cc102980 0x7fc5cc1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.018+0000 7fc5d32cd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5cc103340 0x7fc5cc19f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.018+0000 7fc5d32cd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc10d500 con 0x7fc5cc101790 2024-04-25T00:14:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.018+0000 7fc5d32cd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc10d380 con 0x7fc5cc102980 2024-04-25T00:14:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.018+0000 7fc5d32cd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc10d680 con 0x7fc5cc103340 2024-04-25T00:14:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.018+0000 7fc5d1069640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5cc101790 0x7fc5cc197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.018+0000 7fc5d1069640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5cc101790 0x7fc5cc197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46874/0 (socket says 172.21.15.107:46874) 2024-04-25T00:14:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.018+0000 7fc5d1069640 1 -- 172.21.15.107:0/391529550 learned_addr learned my addr 172.21.15.107:0/391529550 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.018+0000 7fc5d1069640 1 -- 172.21.15.107:0/391529550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5cc103340 msgr2=0x7fc5cc19f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.018+0000 7fc5d1069640 1 --2- 172.21.15.107:0/391529550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5cc103340 0x7fc5cc19f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.018+0000 7fc5d0868640 1 --2- 172.21.15.107:0/391529550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5cc102980 0x7fc5cc1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.018+0000 7fc5d1069640 1 -- 172.21.15.107:0/391529550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5cc102980 msgr2=0x7fc5cc1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.018+0000 7fc5d1069640 1 --2- 172.21.15.107:0/391529550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5cc102980 0x7fc5cc1981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.018+0000 7fc5d1069640 1 -- 172.21.15.107:0/391529550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5cc19f880 con 0x7fc5cc101790 2024-04-25T00:14:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.018+0000 7fc5d1069640 1 --2- 172.21.15.107:0/391529550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5cc101790 0x7fc5cc197c70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800b450 tx=0x7fc5c800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.018+0000 7fc5c27fc640 1 -- 172.21.15.107:0/391529550 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8015020 con 0x7fc5cc101790 2024-04-25T00:14:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.018+0000 7fc5d32cd640 1 -- 172.21.15.107:0/391529550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5cc19fb70 con 0x7fc5cc101790 2024-04-25T00:14:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.018+0000 7fc5c27fc640 1 -- 172.21.15.107:0/391529550 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5c8011d70 con 0x7fc5cc101790 2024-04-25T00:14:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.018+0000 7fc5c27fc640 1 -- 172.21.15.107:0/391529550 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c800f340 con 0x7fc5cc101790 2024-04-25T00:14:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.019+0000 7fc5d32cd640 1 -- 172.21.15.107:0/391529550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5cc1a00c0 con 0x7fc5cc101790 2024-04-25T00:14:17.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.020+0000 7fc5d32cd640 1 -- 172.21.15.107:0/391529550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5cc108b80 con 0x7fc5cc101790 2024-04-25T00:14:17.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.020+0000 7fc5c27fc640 1 -- 172.21.15.107:0/391529550 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fc5c8018020 con 0x7fc5cc101790 2024-04-25T00:14:17.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.021+0000 7fc5c27fc640 1 --2- 172.21.15.107:0/391529550 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc5b80a2b90 0x7fc5b80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:17.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.021+0000 7fc5d0868640 1 --2- 172.21.15.107:0/391529550 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc5b80a2b90 0x7fc5b80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:17.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.021+0000 7fc5c27fc640 1 -- 172.21.15.107:0/391529550 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fc5c80cee70 con 0x7fc5cc101790 2024-04-25T00:14:17.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.021+0000 7fc5d0868640 1 --2- 172.21.15.107:0/391529550 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc5b80a2b90 0x7fc5b80a5050 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc031610 tx=0x7fc5bc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:17.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.026+0000 7fc5c27fc640 1 -- 172.21.15.107:0/391529550 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc5c801f020 con 0x7fc5cc101790 2024-04-25T00:14:17.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.200+0000 7fc5d32cd640 1 -- 172.21.15.107:0/391529550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc5cc062340 con 0x7fc5cc101790 2024-04-25T00:14:17.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.200+0000 7fc5c27fc640 1 -- 172.21.15.107:0/391529550 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7fc5c800d040 con 0x7fc5cc101790 2024-04-25T00:14:17.201 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-25T00:14:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.202+0000 7fc5d32cd640 1 -- 172.21.15.107:0/391529550 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc5b80a2b90 msgr2=0x7fc5b80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.202+0000 7fc5d32cd640 1 --2- 172.21.15.107:0/391529550 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc5b80a2b90 0x7fc5b80a5050 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc031610 tx=0x7fc5bc000bc0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.202+0000 7fc5d32cd640 1 -- 172.21.15.107:0/391529550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5cc101790 msgr2=0x7fc5cc197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.202+0000 7fc5d32cd640 1 --2- 172.21.15.107:0/391529550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5cc101790 0x7fc5cc197c70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800b450 tx=0x7fc5c800b910 comp rx=0 tx=0).stop 2024-04-25T00:14:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.202+0000 7fc5d32cd640 1 -- 172.21.15.107:0/391529550 shutdown_connections 2024-04-25T00:14:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.202+0000 7fc5d32cd640 1 --2- 172.21.15.107:0/391529550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5cc103340 0x7fc5cc19f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.202+0000 7fc5d32cd640 1 --2- 172.21.15.107:0/391529550 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc5b80a2b90 0x7fc5b80a5050 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.202+0000 7fc5d32cd640 1 --2- 172.21.15.107:0/391529550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc5cc102980 0x7fc5cc1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.202+0000 7fc5d32cd640 1 --2- 172.21.15.107:0/391529550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5cc101790 0x7fc5cc197c70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.202+0000 7fc5d32cd640 1 -- 172.21.15.107:0/391529550 >> 172.21.15.107:0/391529550 conn(0x7fc5cc0fcd20 msgr2=0x7fc5cc105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:17.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.202+0000 7fc5d32cd640 1 -- 172.21.15.107:0/391529550 shutdown_connections 2024-04-25T00:14:17.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.202+0000 7fc5d32cd640 1 -- 172.21.15.107:0/391529550 wait complete. 2024-04-25T00:14:17.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:16 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3402710031' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:14:17.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:16 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2215195076' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:17.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:16 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2196417454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:14:17.213 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:14:17.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.314+0000 7f58a7f90640 1 -- 172.21.15.107:0/1257333039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58a0101870 msgr2=0x7f58a0101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.314+0000 7f58a7f90640 1 --2- 172.21.15.107:0/1257333039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58a0101870 0x7f58a0101cf0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5890009900 tx=0x7f589002a410 comp rx=0 tx=0).stop 2024-04-25T00:14:17.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.314+0000 7f58a7f90640 1 -- 172.21.15.107:0/1257333039 shutdown_connections 2024-04-25T00:14:17.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.314+0000 7f58a7f90640 1 --2- 172.21.15.107:0/1257333039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58a0102230 0x7f58a0108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.314+0000 7f58a7f90640 1 --2- 172.21.15.107:0/1257333039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58a0101870 0x7f58a0101cf0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.314+0000 7f58a7f90640 1 --2- 172.21.15.107:0/1257333039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58a0100680 0x7f58a0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.314+0000 7f58a7f90640 1 -- 172.21.15.107:0/1257333039 >> 172.21.15.107:0/1257333039 conn(0x7f58a00fbbf0 msgr2=0x7f58a00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:17.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.314+0000 7f58a7f90640 1 -- 172.21.15.107:0/1257333039 shutdown_connections 2024-04-25T00:14:17.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.314+0000 7f58a7f90640 1 -- 172.21.15.107:0/1257333039 wait complete. 2024-04-25T00:14:17.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.314+0000 7f58a7f90640 1 Processor -- start 2024-04-25T00:14:17.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.314+0000 7f58a7f90640 1 -- start start 2024-04-25T00:14:17.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.315+0000 7f58a7f90640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58a0100680 0x7f58a006a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:17.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.315+0000 7f58a7f90640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58a0101870 0x7f58a019c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:17.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.315+0000 7f58a7f90640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58a0102230 0x7f58a019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:17.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.315+0000 7f58a7f90640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f58a010c350 con 0x7f58a0102230 2024-04-25T00:14:17.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.315+0000 7f58a7f90640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f58a010c1d0 con 0x7f58a0101870 2024-04-25T00:14:17.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.315+0000 7f58a7f90640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f58a010c4d0 con 0x7f58a0100680 2024-04-25T00:14:17.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.315+0000 7f58a652d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58a0102230 0x7f58a019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:17.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.315+0000 7f58a652d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58a0102230 0x7f58a019ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46896/0 (socket says 172.21.15.107:46896) 2024-04-25T00:14:17.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.315+0000 7f58a652d640 1 -- 172.21.15.107:0/3808002431 learned_addr learned my addr 172.21.15.107:0/3808002431 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:17.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.315+0000 7f58a552b640 1 --2- 172.21.15.107:0/3808002431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58a0101870 0x7f58a019c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:17.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.315+0000 7f58a5d2c640 1 --2- 172.21.15.107:0/3808002431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58a0100680 0x7f58a006a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:17.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.315+0000 7f58a652d640 1 -- 172.21.15.107:0/3808002431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58a0100680 msgr2=0x7f58a006a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.315+0000 7f58a652d640 1 --2- 172.21.15.107:0/3808002431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58a0100680 0x7f58a006a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.315+0000 7f58a652d640 1 -- 172.21.15.107:0/3808002431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58a0101870 msgr2=0x7f58a019c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.315+0000 7f58a652d640 1 --2- 172.21.15.107:0/3808002431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58a0101870 0x7f58a019c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.315+0000 7f58a652d640 1 -- 172.21.15.107:0/3808002431 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a019f770 con 0x7f58a0102230 2024-04-25T00:14:17.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.315+0000 7f58a552b640 1 --2- 172.21.15.107:0/3808002431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58a0101870 0x7f58a019c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:17.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.315+0000 7f58a652d640 1 --2- 172.21.15.107:0/3808002431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58a0102230 0x7f58a019ee60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f589c00cad0 tx=0x7f589c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:17.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.315+0000 7f5896ffd640 1 -- 172.21.15.107:0/3808002431 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c00ae20 con 0x7f58a0102230 2024-04-25T00:14:17.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.315+0000 7f5896ffd640 1 -- 172.21.15.107:0/3808002431 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f589c005020 con 0x7f58a0102230 2024-04-25T00:14:17.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.315+0000 7f58a7f90640 1 -- 172.21.15.107:0/3808002431 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a019fa00 con 0x7f58a0102230 2024-04-25T00:14:17.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.315+0000 7f5896ffd640 1 -- 172.21.15.107:0/3808002431 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c00f6f0 con 0x7f58a0102230 2024-04-25T00:14:17.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.315+0000 7f58a7f90640 1 -- 172.21.15.107:0/3808002431 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a019ff20 con 0x7f58a0102230 2024-04-25T00:14:17.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.316+0000 7f58a7f90640 1 -- 172.21.15.107:0/3808002431 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58a01079a0 con 0x7f58a0102230 2024-04-25T00:14:17.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.317+0000 7f5896ffd640 1 -- 172.21.15.107:0/3808002431 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f589c0051d0 con 0x7f58a0102230 2024-04-25T00:14:17.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.318+0000 7f5896ffd640 1 --2- 172.21.15.107:0/3808002431 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f588c0a2c10 0x7f588c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:17.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.318+0000 7f58a5d2c640 1 --2- 172.21.15.107:0/3808002431 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f588c0a2c10 0x7f588c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:17.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.318+0000 7f5896ffd640 1 -- 172.21.15.107:0/3808002431 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f589c0cf350 con 0x7f58a0102230 2024-04-25T00:14:17.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.319+0000 7f58a5d2c640 1 --2- 172.21.15.107:0/3808002431 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f588c0a2c10 0x7f588c0a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f58980058c0 tx=0x7f589800a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.323+0000 7f5896ffd640 1 -- 172.21.15.107:0/3808002431 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f589c014030 con 0x7f58a0102230 2024-04-25T00:14:17.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:16 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3402710031' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:14:17.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:16 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2215195076' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:17.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:16 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2196417454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:14:17.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:16 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3402710031' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:14:17.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:16 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2215195076' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:17.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:16 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2196417454' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:14:17.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.503+0000 7f58a7f90640 1 -- 172.21.15.107:0/3808002431 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f58a0062340 con 0x7f58a0102230 2024-04-25T00:14:17.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.504+0000 7f5896ffd640 1 -- 172.21.15.107:0/3808002431 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f589c09c070 con 0x7f58a0102230 2024-04-25T00:14:17.504 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-04-25T00:14:17.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.505+0000 7f58a7f90640 1 -- 172.21.15.107:0/3808002431 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f588c0a2c10 msgr2=0x7f588c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.505+0000 7f58a7f90640 1 --2- 172.21.15.107:0/3808002431 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f588c0a2c10 0x7f588c0a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f58980058c0 tx=0x7f589800a4c0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.506+0000 7f58a7f90640 1 -- 172.21.15.107:0/3808002431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58a0102230 msgr2=0x7f58a019ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.506+0000 7f58a7f90640 1 --2- 172.21.15.107:0/3808002431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58a0102230 0x7f58a019ee60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f589c00cad0 tx=0x7f589c00cf90 comp rx=0 tx=0).stop 2024-04-25T00:14:17.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.506+0000 7f58a7f90640 1 -- 172.21.15.107:0/3808002431 shutdown_connections 2024-04-25T00:14:17.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.506+0000 7f58a7f90640 1 --2- 172.21.15.107:0/3808002431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58a0102230 0x7f58a019ee60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.506+0000 7f58a7f90640 1 --2- 172.21.15.107:0/3808002431 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f588c0a2c10 0x7f588c0a50d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.506+0000 7f58a7f90640 1 --2- 172.21.15.107:0/3808002431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58a0101870 0x7f58a019c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.506+0000 7f58a7f90640 1 --2- 172.21.15.107:0/3808002431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58a0100680 0x7f58a006a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.506+0000 7f58a7f90640 1 -- 172.21.15.107:0/3808002431 >> 172.21.15.107:0/3808002431 conn(0x7f58a00fbbf0 msgr2=0x7f58a0104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:17.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.506+0000 7f58a7f90640 1 -- 172.21.15.107:0/3808002431 shutdown_connections 2024-04-25T00:14:17.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.506+0000 7f58a7f90640 1 -- 172.21.15.107:0/3808002431 wait complete. 2024-04-25T00:14:17.517 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:14:17.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.617+0000 7f045b0d4640 1 -- 172.21.15.107:0/4131511851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0454102980 msgr2=0x7f0454102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.617+0000 7f045b0d4640 1 --2- 172.21.15.107:0/4131511851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0454102980 0x7f0454102e00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0444009ca0 tx=0x7f044402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.617+0000 7f045b0d4640 1 -- 172.21.15.107:0/4131511851 shutdown_connections 2024-04-25T00:14:17.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.617+0000 7f045b0d4640 1 --2- 172.21.15.107:0/4131511851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0454103340 0x7f0454109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.617+0000 7f045b0d4640 1 --2- 172.21.15.107:0/4131511851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0454102980 0x7f0454102e00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.617+0000 7f045b0d4640 1 --2- 172.21.15.107:0/4131511851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0454101790 0x7f0454101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.617+0000 7f045b0d4640 1 -- 172.21.15.107:0/4131511851 >> 172.21.15.107:0/4131511851 conn(0x7f04540fcd20 msgr2=0x7f04540ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:17.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.617+0000 7f045b0d4640 1 -- 172.21.15.107:0/4131511851 shutdown_connections 2024-04-25T00:14:17.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.617+0000 7f045b0d4640 1 -- 172.21.15.107:0/4131511851 wait complete. 2024-04-25T00:14:17.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.617+0000 7f045b0d4640 1 Processor -- start 2024-04-25T00:14:17.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.617+0000 7f045b0d4640 1 -- start start 2024-04-25T00:14:17.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.617+0000 7f045b0d4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0454101790 0x7f0454197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:17.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.617+0000 7f045b0d4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0454102980 0x7f0454197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:17.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.618+0000 7f045b0d4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0454103340 0x7f045419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:17.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.618+0000 7f045b0d4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f045410d410 con 0x7f0454103340 2024-04-25T00:14:17.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.618+0000 7f045b0d4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f045410d290 con 0x7f0454102980 2024-04-25T00:14:17.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.618+0000 7f045b0d4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f045410d590 con 0x7f0454101790 2024-04-25T00:14:17.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.618+0000 7f0459671640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0454103340 0x7f045419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:17.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.618+0000 7f0459671640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0454103340 0x7f045419f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46908/0 (socket says 172.21.15.107:46908) 2024-04-25T00:14:17.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.618+0000 7f0459671640 1 -- 172.21.15.107:0/2229551576 learned_addr learned my addr 172.21.15.107:0/2229551576 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:17.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.618+0000 7f0458e70640 1 --2- 172.21.15.107:0/2229551576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0454101790 0x7f0454197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:17.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.618+0000 7f0459671640 1 -- 172.21.15.107:0/2229551576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0454101790 msgr2=0x7f0454197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.618+0000 7f0459671640 1 --2- 172.21.15.107:0/2229551576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0454101790 0x7f0454197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.618+0000 7f0453fff640 1 --2- 172.21.15.107:0/2229551576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0454102980 0x7f0454197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:17.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.618+0000 7f0459671640 1 -- 172.21.15.107:0/2229551576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0454102980 msgr2=0x7f0454197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.618+0000 7f0459671640 1 --2- 172.21.15.107:0/2229551576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0454102980 0x7f0454197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.618+0000 7f0459671640 1 -- 172.21.15.107:0/2229551576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045419f6a0 con 0x7f0454103340 2024-04-25T00:14:17.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.618+0000 7f0453fff640 1 --2- 172.21.15.107:0/2229551576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0454102980 0x7f0454197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:17.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.618+0000 7f0459671640 1 --2- 172.21.15.107:0/2229551576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0454103340 0x7f045419f030 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f043c00c970 tx=0x7f043c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:17.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.618+0000 7f0451ffb640 1 -- 172.21.15.107:0/2229551576 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c007bb0 con 0x7f0454103340 2024-04-25T00:14:17.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.618+0000 7f0451ffb640 1 -- 172.21.15.107:0/2229551576 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f043c007d50 con 0x7f0454103340 2024-04-25T00:14:17.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.618+0000 7f045b0d4640 1 -- 172.21.15.107:0/2229551576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f045419f990 con 0x7f0454103340 2024-04-25T00:14:17.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.618+0000 7f0451ffb640 1 -- 172.21.15.107:0/2229551576 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f043c018800 con 0x7f0454103340 2024-04-25T00:14:17.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.619+0000 7f045b0d4640 1 -- 172.21.15.107:0/2229551576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f045419fe90 con 0x7f0454103340 2024-04-25T00:14:17.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.619+0000 7f045b0d4640 1 -- 172.21.15.107:0/2229551576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0454108b30 con 0x7f0454103340 2024-04-25T00:14:17.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.620+0000 7f0451ffb640 1 -- 172.21.15.107:0/2229551576 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f043c00fa50 con 0x7f0454103340 2024-04-25T00:14:17.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.621+0000 7f0451ffb640 1 --2- 172.21.15.107:0/2229551576 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f04400a2b90 0x7f04400a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:17.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.621+0000 7f0458e70640 1 --2- 172.21.15.107:0/2229551576 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f04400a2b90 0x7f04400a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:17.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.621+0000 7f0451ffb640 1 -- 172.21.15.107:0/2229551576 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f043c0cf220 con 0x7f0454103340 2024-04-25T00:14:17.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.621+0000 7f0458e70640 1 --2- 172.21.15.107:0/2229551576 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f04400a2b90 0x7f04400a5050 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f044c005fd0 tx=0x7f044c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:17.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.625+0000 7f0451ffb640 1 -- 172.21.15.107:0/2229551576 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f043c014030 con 0x7f0454103340 2024-04-25T00:14:17.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.802+0000 7f045b0d4640 1 -- 172.21.15.107:0/2229551576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f045402d170 con 0x7f0454103340 2024-04-25T00:14:17.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.803+0000 7f0451ffb640 1 -- 172.21.15.107:0/2229551576 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7f043c09bfe0 con 0x7f0454103340 2024-04-25T00:14:17.803 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-25T00:14:17.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.805+0000 7f045b0d4640 1 -- 172.21.15.107:0/2229551576 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f04400a2b90 msgr2=0x7f04400a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.805+0000 7f045b0d4640 1 --2- 172.21.15.107:0/2229551576 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f04400a2b90 0x7f04400a5050 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f044c005fd0 tx=0x7f044c00b390 comp rx=0 tx=0).stop 2024-04-25T00:14:17.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.805+0000 7f045b0d4640 1 -- 172.21.15.107:0/2229551576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0454103340 msgr2=0x7f045419f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.805+0000 7f045b0d4640 1 --2- 172.21.15.107:0/2229551576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0454103340 0x7f045419f030 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f043c00c970 tx=0x7f043c00ce30 comp rx=0 tx=0).stop 2024-04-25T00:14:17.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.805+0000 7f045b0d4640 1 -- 172.21.15.107:0/2229551576 shutdown_connections 2024-04-25T00:14:17.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.805+0000 7f045b0d4640 1 --2- 172.21.15.107:0/2229551576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0454103340 0x7f045419f030 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.805+0000 7f045b0d4640 1 --2- 172.21.15.107:0/2229551576 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f04400a2b90 0x7f04400a5050 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.805+0000 7f045b0d4640 1 --2- 172.21.15.107:0/2229551576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0454102980 0x7f0454197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.805+0000 7f045b0d4640 1 --2- 172.21.15.107:0/2229551576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0454101790 0x7f0454197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.805+0000 7f045b0d4640 1 -- 172.21.15.107:0/2229551576 >> 172.21.15.107:0/2229551576 conn(0x7f04540fcd20 msgr2=0x7f0454105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:17.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.805+0000 7f045b0d4640 1 -- 172.21.15.107:0/2229551576 shutdown_connections 2024-04-25T00:14:17.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.805+0000 7f045b0d4640 1 -- 172.21.15.107:0/2229551576 wait complete. 2024-04-25T00:14:17.816 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:14:17.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.916+0000 7f88ac3a2640 1 -- 172.21.15.107:0/331344429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88a4102230 msgr2=0x7f88a4108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.916+0000 7f88ac3a2640 1 --2- 172.21.15.107:0/331344429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88a4102230 0x7f88a4108ac0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f889c009ca0 tx=0x7f889c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.916+0000 7f88ac3a2640 1 -- 172.21.15.107:0/331344429 shutdown_connections 2024-04-25T00:14:17.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.916+0000 7f88ac3a2640 1 --2- 172.21.15.107:0/331344429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88a4102230 0x7f88a4108ac0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.916+0000 7f88ac3a2640 1 --2- 172.21.15.107:0/331344429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88a4101870 0x7f88a4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.916+0000 7f88ac3a2640 1 --2- 172.21.15.107:0/331344429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a4100680 0x7f88a4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.917+0000 7f88ac3a2640 1 -- 172.21.15.107:0/331344429 >> 172.21.15.107:0/331344429 conn(0x7f88a40fbbf0 msgr2=0x7f88a40fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:17.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.917+0000 7f88ac3a2640 1 -- 172.21.15.107:0/331344429 shutdown_connections 2024-04-25T00:14:17.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.917+0000 7f88ac3a2640 1 -- 172.21.15.107:0/331344429 wait complete. 2024-04-25T00:14:17.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.917+0000 7f88ac3a2640 1 Processor -- start 2024-04-25T00:14:17.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.917+0000 7f88ac3a2640 1 -- start start 2024-04-25T00:14:17.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.917+0000 7f88ac3a2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a4100680 0x7f88a4069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:17.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.917+0000 7f88ac3a2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88a4101870 0x7f88a419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:17.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.917+0000 7f88ac3a2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88a4102230 0x7f88a419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:17.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.917+0000 7f88ac3a2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f88a410c300 con 0x7f88a4101870 2024-04-25T00:14:17.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.917+0000 7f88ac3a2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f88a410c180 con 0x7f88a4102230 2024-04-25T00:14:17.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.917+0000 7f88ac3a2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f88a410c480 con 0x7f88a4100680 2024-04-25T00:14:17.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.917+0000 7f88a993d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88a4101870 0x7f88a419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:17.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.917+0000 7f88a993d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88a4101870 0x7f88a419c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46918/0 (socket says 172.21.15.107:46918) 2024-04-25T00:14:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.917+0000 7f88a993d640 1 -- 172.21.15.107:0/1595230435 learned_addr learned my addr 172.21.15.107:0/1595230435 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.917+0000 7f88aa93f640 1 --2- 172.21.15.107:0/1595230435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88a4102230 0x7f88a419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.918+0000 7f88a993d640 1 -- 172.21.15.107:0/1595230435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a4100680 msgr2=0x7f88a4069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.918+0000 7f88a993d640 1 --2- 172.21.15.107:0/1595230435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a4100680 0x7f88a4069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.918+0000 7f88a993d640 1 -- 172.21.15.107:0/1595230435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88a4102230 msgr2=0x7f88a419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.918+0000 7f88a993d640 1 --2- 172.21.15.107:0/1595230435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88a4102230 0x7f88a419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.918+0000 7f88a993d640 1 -- 172.21.15.107:0/1595230435 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a419f6a0 con 0x7f88a4101870 2024-04-25T00:14:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.918+0000 7f88a993d640 1 --2- 172.21.15.107:0/1595230435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88a4101870 0x7f88a419c7a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f889400d950 tx=0x7f889400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.918+0000 7f889b7fe640 1 -- 172.21.15.107:0/1595230435 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8894017070 con 0x7f88a4101870 2024-04-25T00:14:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.918+0000 7f889b7fe640 1 -- 172.21.15.107:0/1595230435 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8894009c30 con 0x7f88a4101870 2024-04-25T00:14:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.918+0000 7f88ac3a2640 1 -- 172.21.15.107:0/1595230435 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a419f990 con 0x7f88a4101870 2024-04-25T00:14:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.918+0000 7f889b7fe640 1 -- 172.21.15.107:0/1595230435 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889400f6b0 con 0x7f88a4101870 2024-04-25T00:14:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.918+0000 7f88ac3a2640 1 -- 172.21.15.107:0/1595230435 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a419fee0 con 0x7f88a4101870 2024-04-25T00:14:17.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.919+0000 7f88ac3a2640 1 -- 172.21.15.107:0/1595230435 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88a41079a0 con 0x7f88a4101870 2024-04-25T00:14:17.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.920+0000 7f889b7fe640 1 -- 172.21.15.107:0/1595230435 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f88940109d0 con 0x7f88a4101870 2024-04-25T00:14:17.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.921+0000 7f889b7fe640 1 --2- 172.21.15.107:0/1595230435 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f88900a2b90 0x7f88900a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:17.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.921+0000 7f88aa13e640 1 --2- 172.21.15.107:0/1595230435 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f88900a2b90 0x7f88900a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:17.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.921+0000 7f889b7fe640 1 -- 172.21.15.107:0/1595230435 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f88940cef70 con 0x7f88a4101870 2024-04-25T00:14:17.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.921+0000 7f88aa13e640 1 --2- 172.21.15.107:0/1595230435 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f88900a2b90 0x7f88900a5050 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f88a41016d0 tx=0x7f88a0005c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:17.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:17.924+0000 7f889b7fe640 1 -- 172.21.15.107:0/1595230435 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8894009de0 con 0x7f88a4101870 2024-04-25T00:14:18.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:18.115+0000 7f88ac3a2640 1 -- 172.21.15.107:0/1595230435 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f88a4062340 con 0x7f88a4101870 2024-04-25T00:14:18.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:18.115+0000 7f889b7fe640 1 -- 172.21.15.107:0/1595230435 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) ==== 81+0+596 (secure 0 0 0) 0x7f88940bb070 con 0x7f88a4101870 2024-04-25T00:14:18.116 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1174178948 2024-04-26T00:07:00.838246+0000 2024-04-25T00:14:18.116 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/153169302 2024-04-26T00:07:00.838246+0000 2024-04-25T00:14:18.116 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/120616774 2024-04-26T00:06:34.124795+0000 2024-04-25T00:14:18.116 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/120616774 2024-04-26T00:06:34.124795+0000 2024-04-25T00:14:18.116 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/971124970 2024-04-26T00:06:34.124795+0000 2024-04-25T00:14:18.116 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3022844652 2024-04-26T00:06:34.124795+0000 2024-04-25T00:14:18.116 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/153169302 2024-04-26T00:07:00.838246+0000 2024-04-25T00:14:18.116 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/4215203700 2024-04-26T00:07:00.838246+0000 2024-04-25T00:14:18.116 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/750288374 2024-04-26T00:06:34.124795+0000 2024-04-25T00:14:18.116 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3718680642 2024-04-26T00:07:00.838246+0000 2024-04-25T00:14:18.117 INFO:teuthology.orchestra.run.smithi107.stderr:listed 10 entries 2024-04-25T00:14:18.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:18.117+0000 7f88ac3a2640 1 -- 172.21.15.107:0/1595230435 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f88900a2b90 msgr2=0x7f88900a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:18.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:18.117+0000 7f88ac3a2640 1 --2- 172.21.15.107:0/1595230435 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f88900a2b90 0x7f88900a5050 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f88a41016d0 tx=0x7f88a0005c10 comp rx=0 tx=0).stop 2024-04-25T00:14:18.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:18.117+0000 7f88ac3a2640 1 -- 172.21.15.107:0/1595230435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88a4101870 msgr2=0x7f88a419c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:18.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:18.117+0000 7f88ac3a2640 1 --2- 172.21.15.107:0/1595230435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88a4101870 0x7f88a419c7a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f889400d950 tx=0x7f889400de10 comp rx=0 tx=0).stop 2024-04-25T00:14:18.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:18.117+0000 7f88ac3a2640 1 -- 172.21.15.107:0/1595230435 shutdown_connections 2024-04-25T00:14:18.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:18.117+0000 7f88ac3a2640 1 --2- 172.21.15.107:0/1595230435 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f88900a2b90 0x7f88900a5050 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:18.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:18.117+0000 7f88ac3a2640 1 --2- 172.21.15.107:0/1595230435 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88a4102230 0x7f88a419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:18.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:18.117+0000 7f88ac3a2640 1 --2- 172.21.15.107:0/1595230435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88a4101870 0x7f88a419c7a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:18.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:18.117+0000 7f88ac3a2640 1 --2- 172.21.15.107:0/1595230435 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88a4100680 0x7f88a4069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:18.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:18.117+0000 7f88ac3a2640 1 -- 172.21.15.107:0/1595230435 >> 172.21.15.107:0/1595230435 conn(0x7f88a40fbbf0 msgr2=0x7f88a4104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:18.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:18.117+0000 7f88ac3a2640 1 -- 172.21.15.107:0/1595230435 shutdown_connections 2024-04-25T00:14:18.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:18.117+0000 7f88ac3a2640 1 -- 172.21.15.107:0/1595230435 wait complete. 2024-04-25T00:14:18.127 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:17 smithi107 ceph-mon[29153]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 338 B/s wr, 0 op/s 2024-04-25T00:14:18.127 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:17 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/391529550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:14:18.127 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:17 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3808002431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:14:18.127 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:17 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2229551576' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:14:18.128 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2024-04-25T00:14:18.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:18.128 DEBUG:teuthology.orchestra.run.smithi107:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T00:14:18.150 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi107.stderr:ceph-fuse[70079]: fuse finished with error 0 and tester_r 0 2024-04-25T00:14:18.152 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T00:14:18.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:17 smithi195 ceph-mon[32833]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 338 B/s wr, 0 op/s 2024-04-25T00:14:18.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:17 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/391529550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:14:18.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:17 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3808002431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:14:18.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:17 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2229551576' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:14:18.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:17 smithi170 ceph-mon[32807]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 338 B/s wr, 0 op/s 2024-04-25T00:14:18.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:17 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/391529550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:14:18.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:17 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3808002431' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:14:18.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:17 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2229551576' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:14:19.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:18 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1595230435' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:14:19.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:18 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1595230435' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:14:19.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:18 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1595230435' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:14:20.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:19 smithi195 ceph-mon[32833]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 304 B/s wr, 0 op/s 2024-04-25T00:14:20.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:19 smithi170 ceph-mon[32807]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 304 B/s wr, 0 op/s 2024-04-25T00:14:20.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:19 smithi107 ceph-mon[29153]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 304 B/s wr, 0 op/s 2024-04-25T00:14:22.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:21 smithi195 ceph-mon[32833]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T00:14:22.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:21 smithi170 ceph-mon[32807]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T00:14:22.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:21 smithi107 ceph-mon[29153]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T00:14:24.254 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T00:14:24.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:14:24.254 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T00:14:24.293 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:24.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:24 smithi195 ceph-mon[32833]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T00:14:24.357 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:24 smithi107 ceph-mon[29153]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T00:14:24.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:24 smithi170 ceph-mon[32807]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T00:14:24.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.442+0000 7f5d56ebc640 1 -- 172.21.15.107:0/3495628948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d50102230 msgr2=0x7f5d50108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:24.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.442+0000 7f5d56ebc640 1 --2- 172.21.15.107:0/3495628948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d50102230 0x7f5d50108ac0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5d48009ca0 tx=0x7f5d4802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.442+0000 7f5d56ebc640 1 -- 172.21.15.107:0/3495628948 shutdown_connections 2024-04-25T00:14:24.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.442+0000 7f5d56ebc640 1 --2- 172.21.15.107:0/3495628948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d50102230 0x7f5d50108ac0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.442+0000 7f5d56ebc640 1 --2- 172.21.15.107:0/3495628948 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d50101870 0x7f5d50101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.442+0000 7f5d56ebc640 1 --2- 172.21.15.107:0/3495628948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d50100680 0x7f5d50100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.442+0000 7f5d56ebc640 1 -- 172.21.15.107:0/3495628948 >> 172.21.15.107:0/3495628948 conn(0x7f5d500fbbf0 msgr2=0x7f5d500fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:24.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.442+0000 7f5d56ebc640 1 -- 172.21.15.107:0/3495628948 shutdown_connections 2024-04-25T00:14:24.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.442+0000 7f5d56ebc640 1 -- 172.21.15.107:0/3495628948 wait complete. 2024-04-25T00:14:24.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.442+0000 7f5d56ebc640 1 Processor -- start 2024-04-25T00:14:24.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.442+0000 7f5d56ebc640 1 -- start start 2024-04-25T00:14:24.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.442+0000 7f5d56ebc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d50100680 0x7f5d500694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:24.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.442+0000 7f5d56ebc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d50101870 0x7f5d5019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:24.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.442+0000 7f5d56ebc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d50102230 0x7f5d5019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:24.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.442+0000 7f5d56ebc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5010c370 con 0x7f5d50102230 2024-04-25T00:14:24.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.442+0000 7f5d56ebc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5010c1f0 con 0x7f5d50100680 2024-04-25T00:14:24.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.442+0000 7f5d56ebc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5010c4f0 con 0x7f5d50101870 2024-04-25T00:14:24.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.443+0000 7f5d55459640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d50102230 0x7f5d5019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:24.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.443+0000 7f5d55459640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d50102230 0x7f5d5019ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59134/0 (socket says 172.21.15.107:59134) 2024-04-25T00:14:24.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.443+0000 7f5d55459640 1 -- 172.21.15.107:0/2707996401 learned_addr learned my addr 172.21.15.107:0/2707996401 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:24.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.443+0000 7f5d54c58640 1 --2- 172.21.15.107:0/2707996401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d50100680 0x7f5d500694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:24.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.443+0000 7f5d55459640 1 -- 172.21.15.107:0/2707996401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d50101870 msgr2=0x7f5d5019c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:24.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.443+0000 7f5d55459640 1 --2- 172.21.15.107:0/2707996401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d50101870 0x7f5d5019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.443+0000 7f5d55459640 1 -- 172.21.15.107:0/2707996401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d50100680 msgr2=0x7f5d500694a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:24.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.443+0000 7f5d55459640 1 --2- 172.21.15.107:0/2707996401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d50100680 0x7f5d500694a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.443+0000 7f5d55459640 1 -- 172.21.15.107:0/2707996401 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d5019f620 con 0x7f5d50102230 2024-04-25T00:14:24.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.443+0000 7f5d55459640 1 --2- 172.21.15.107:0/2707996401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d50102230 0x7f5d5019ed60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5d480098d0 tx=0x7f5d4802f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:24.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.443+0000 7f5d4dffb640 1 -- 172.21.15.107:0/2707996401 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4802fc30 con 0x7f5d50102230 2024-04-25T00:14:24.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.443+0000 7f5d4dffb640 1 -- 172.21.15.107:0/2707996401 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d48003e50 con 0x7f5d50102230 2024-04-25T00:14:24.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.443+0000 7f5d56ebc640 1 -- 172.21.15.107:0/2707996401 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d5019f8b0 con 0x7f5d50102230 2024-04-25T00:14:24.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.443+0000 7f5d4dffb640 1 -- 172.21.15.107:0/2707996401 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d48004810 con 0x7f5d50102230 2024-04-25T00:14:24.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.444+0000 7f5d56ebc640 1 -- 172.21.15.107:0/2707996401 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d5019fe80 con 0x7f5d50102230 2024-04-25T00:14:24.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.445+0000 7f5d56ebc640 1 -- 172.21.15.107:0/2707996401 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d501078d0 con 0x7f5d50102230 2024-04-25T00:14:24.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.446+0000 7f5d4dffb640 1 -- 172.21.15.107:0/2707996401 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5d48003970 con 0x7f5d50102230 2024-04-25T00:14:24.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.446+0000 7f5d4dffb640 1 --2- 172.21.15.107:0/2707996401 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5d400a2b90 0x7f5d400a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:24.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.447+0000 7f5d54c58640 1 --2- 172.21.15.107:0/2707996401 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5d400a2b90 0x7f5d400a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:24.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.447+0000 7f5d4dffb640 1 -- 172.21.15.107:0/2707996401 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f5d480f0640 con 0x7f5d50102230 2024-04-25T00:14:24.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.447+0000 7f5d54c58640 1 --2- 172.21.15.107:0/2707996401 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5d400a2b90 0x7f5d400a5050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5d501016d0 tx=0x7f5d44009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:24.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.451+0000 7f5d4dffb640 1 -- 172.21.15.107:0/2707996401 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5d480bd480 con 0x7f5d50102230 2024-04-25T00:14:24.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.627+0000 7f5d56ebc640 1 -- 172.21.15.107:0/2707996401 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d50062340 con 0x7f5d50102230 2024-04-25T00:14:24.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.629+0000 7f5d4dffb640 1 -- 172.21.15.107:0/2707996401 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24134 (secure 0 0 0) 0x7f5d480bcd80 con 0x7f5d50102230 2024-04-25T00:14:24.630 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:14:24.631 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":92,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:14:08.839737+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:59.672801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:00.957782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:14:05.065846+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:24.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.631+0000 7f5d56ebc640 1 -- 172.21.15.107:0/2707996401 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5d400a2b90 msgr2=0x7f5d400a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:24.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.631+0000 7f5d56ebc640 1 --2- 172.21.15.107:0/2707996401 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5d400a2b90 0x7f5d400a5050 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5d501016d0 tx=0x7f5d44009110 comp rx=0 tx=0).stop 2024-04-25T00:14:24.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.631+0000 7f5d56ebc640 1 -- 172.21.15.107:0/2707996401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d50102230 msgr2=0x7f5d5019ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:24.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.631+0000 7f5d56ebc640 1 --2- 172.21.15.107:0/2707996401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d50102230 0x7f5d5019ed60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5d480098d0 tx=0x7f5d4802f840 comp rx=0 tx=0).stop 2024-04-25T00:14:24.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.631+0000 7f5d56ebc640 1 -- 172.21.15.107:0/2707996401 shutdown_connections 2024-04-25T00:14:24.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.631+0000 7f5d56ebc640 1 --2- 172.21.15.107:0/2707996401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d50102230 0x7f5d5019ed60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.631+0000 7f5d56ebc640 1 --2- 172.21.15.107:0/2707996401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d50101870 0x7f5d5019c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.631+0000 7f5d56ebc640 1 --2- 172.21.15.107:0/2707996401 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5d400a2b90 0x7f5d400a5050 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.631+0000 7f5d56ebc640 1 --2- 172.21.15.107:0/2707996401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5d50100680 0x7f5d500694a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.631+0000 7f5d56ebc640 1 -- 172.21.15.107:0/2707996401 >> 172.21.15.107:0/2707996401 conn(0x7f5d500fbbf0 msgr2=0x7f5d50104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:24.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.631+0000 7f5d56ebc640 1 -- 172.21.15.107:0/2707996401 shutdown_connections 2024-04-25T00:14:24.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.631+0000 7f5d56ebc640 1 -- 172.21.15.107:0/2707996401 wait complete. 2024-04-25T00:14:24.644 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:24.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.779+0000 7f1c351c8640 1 -- 172.21.15.107:0/999234630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c30101880 msgr2=0x7f1c30101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:24.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.779+0000 7f1c351c8640 1 --2- 172.21.15.107:0/999234630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c30101880 0x7f1c30101d00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009900 tx=0x7f1c2002a3f0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.779+0000 7f1c351c8640 1 -- 172.21.15.107:0/999234630 shutdown_connections 2024-04-25T00:14:24.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.779+0000 7f1c351c8640 1 --2- 172.21.15.107:0/999234630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c30102240 0x7f1c30108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.779+0000 7f1c351c8640 1 --2- 172.21.15.107:0/999234630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c30101880 0x7f1c30101d00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.779+0000 7f1c351c8640 1 --2- 172.21.15.107:0/999234630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c30100690 0x7f1c30100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.779+0000 7f1c351c8640 1 -- 172.21.15.107:0/999234630 >> 172.21.15.107:0/999234630 conn(0x7f1c300fbc20 msgr2=0x7f1c300fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:24.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.779+0000 7f1c351c8640 1 -- 172.21.15.107:0/999234630 shutdown_connections 2024-04-25T00:14:24.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.779+0000 7f1c351c8640 1 -- 172.21.15.107:0/999234630 wait complete. 2024-04-25T00:14:24.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.780+0000 7f1c351c8640 1 Processor -- start 2024-04-25T00:14:24.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.780+0000 7f1c351c8640 1 -- start start 2024-04-25T00:14:24.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.780+0000 7f1c351c8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c30100690 0x7f1c3006a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:24.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.780+0000 7f1c351c8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c30101880 0x7f1c3019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:24.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.780+0000 7f1c351c8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c30102240 0x7f1c3019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:24.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.780+0000 7f1c351c8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3010c340 con 0x7f1c30102240 2024-04-25T00:14:24.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.781+0000 7f1c351c8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3010c1c0 con 0x7f1c30101880 2024-04-25T00:14:24.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.781+0000 7f1c351c8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3010c4c0 con 0x7f1c30100690 2024-04-25T00:14:24.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.781+0000 7f1c2f59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c30102240 0x7f1c3019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:24.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.781+0000 7f1c2f59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c30102240 0x7f1c3019ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59164/0 (socket says 172.21.15.107:59164) 2024-04-25T00:14:24.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.781+0000 7f1c2f59e640 1 -- 172.21.15.107:0/1705057982 learned_addr learned my addr 172.21.15.107:0/1705057982 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:24.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.781+0000 7f1c2e59c640 1 --2- 172.21.15.107:0/1705057982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c30101880 0x7f1c3019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:24.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.781+0000 7f1c2ed9d640 1 --2- 172.21.15.107:0/1705057982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c30100690 0x7f1c3006a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:24.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.781+0000 7f1c2f59e640 1 -- 172.21.15.107:0/1705057982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c30100690 msgr2=0x7f1c3006a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:24.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.781+0000 7f1c2f59e640 1 --2- 172.21.15.107:0/1705057982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c30100690 0x7f1c3006a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.781+0000 7f1c2f59e640 1 -- 172.21.15.107:0/1705057982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c30101880 msgr2=0x7f1c3019c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:24.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.781+0000 7f1c2f59e640 1 --2- 172.21.15.107:0/1705057982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c30101880 0x7f1c3019c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.781+0000 7f1c2f59e640 1 -- 172.21.15.107:0/1705057982 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3019f640 con 0x7f1c30102240 2024-04-25T00:14:24.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.781+0000 7f1c2f59e640 1 --2- 172.21.15.107:0/1705057982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c30102240 0x7f1c3019ed30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800ccb0 tx=0x7f1c2800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:24.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.781+0000 7f1c17fff640 1 -- 172.21.15.107:0/1705057982 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28004b90 con 0x7f1c30102240 2024-04-25T00:14:24.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.781+0000 7f1c351c8640 1 -- 172.21.15.107:0/1705057982 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c3019f8d0 con 0x7f1c30102240 2024-04-25T00:14:24.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.781+0000 7f1c17fff640 1 -- 172.21.15.107:0/1705057982 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c28004d30 con 0x7f1c30102240 2024-04-25T00:14:24.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.781+0000 7f1c17fff640 1 -- 172.21.15.107:0/1705057982 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c28005370 con 0x7f1c30102240 2024-04-25T00:14:24.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.781+0000 7f1c351c8640 1 -- 172.21.15.107:0/1705057982 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c3019fe40 con 0x7f1c30102240 2024-04-25T00:14:24.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.782+0000 7f1c351c8640 1 -- 172.21.15.107:0/1705057982 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c301078e0 con 0x7f1c30102240 2024-04-25T00:14:24.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.784+0000 7f1c17fff640 1 -- 172.21.15.107:0/1705057982 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f1c2800f450 con 0x7f1c30102240 2024-04-25T00:14:24.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.784+0000 7f1c17fff640 1 --2- 172.21.15.107:0/1705057982 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f1c180a2b90 0x7f1c180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:24.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.784+0000 7f1c2ed9d640 1 --2- 172.21.15.107:0/1705057982 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f1c180a2b90 0x7f1c180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:24.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.784+0000 7f1c17fff640 1 -- 172.21.15.107:0/1705057982 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f1c280cf670 con 0x7f1c30102240 2024-04-25T00:14:24.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.785+0000 7f1c2ed9d640 1 --2- 172.21.15.107:0/1705057982 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f1c180a2b90 0x7f1c180a5050 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1c301016e0 tx=0x7f1c24005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:24.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.788+0000 7f1c17fff640 1 -- 172.21.15.107:0/1705057982 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1c28014070 con 0x7f1c30102240 2024-04-25T00:14:24.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.968+0000 7f1c351c8640 1 -- 172.21.15.107:0/1705057982 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c3002d170 con 0x7f1c30102240 2024-04-25T00:14:24.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.969+0000 7f1c17fff640 1 -- 172.21.15.107:0/1705057982 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24134 (secure 0 0 0) 0x7f1c2809bd40 con 0x7f1c30102240 2024-04-25T00:14:24.971 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:14:24.971 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":92,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:14:08.839737+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:59.672801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:00.957782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:14:05.065846+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:24.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.971+0000 7f1c351c8640 1 -- 172.21.15.107:0/1705057982 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f1c180a2b90 msgr2=0x7f1c180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:24.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.971+0000 7f1c351c8640 1 --2- 172.21.15.107:0/1705057982 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f1c180a2b90 0x7f1c180a5050 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1c301016e0 tx=0x7f1c24005f90 comp rx=0 tx=0).stop 2024-04-25T00:14:24.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.971+0000 7f1c351c8640 1 -- 172.21.15.107:0/1705057982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c30102240 msgr2=0x7f1c3019ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:24.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.971+0000 7f1c351c8640 1 --2- 172.21.15.107:0/1705057982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c30102240 0x7f1c3019ed30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800ccb0 tx=0x7f1c2800a620 comp rx=0 tx=0).stop 2024-04-25T00:14:24.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.971+0000 7f1c351c8640 1 -- 172.21.15.107:0/1705057982 shutdown_connections 2024-04-25T00:14:24.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.971+0000 7f1c351c8640 1 --2- 172.21.15.107:0/1705057982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1c30102240 0x7f1c3019ed30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.971+0000 7f1c351c8640 1 --2- 172.21.15.107:0/1705057982 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f1c180a2b90 0x7f1c180a5050 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.971+0000 7f1c351c8640 1 --2- 172.21.15.107:0/1705057982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1c30101880 0x7f1c3019c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.971+0000 7f1c351c8640 1 --2- 172.21.15.107:0/1705057982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1c30100690 0x7f1c3006a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:24.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.971+0000 7f1c351c8640 1 -- 172.21.15.107:0/1705057982 >> 172.21.15.107:0/1705057982 conn(0x7f1c300fbc20 msgr2=0x7f1c30104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:24.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.972+0000 7f1c351c8640 1 -- 172.21.15.107:0/1705057982 shutdown_connections 2024-04-25T00:14:24.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:24.972+0000 7f1c351c8640 1 -- 172.21.15.107:0/1705057982 wait complete. 2024-04-25T00:14:24.985 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:14:25.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.084+0000 7f5cb2ab3640 1 -- 172.21.15.107:0/2169609387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cac102200 msgr2=0x7f5cac1045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:25.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.084+0000 7f5cb2ab3640 1 --2- 172.21.15.107:0/2169609387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cac102200 0x7f5cac1045f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009900 tx=0x7f5c9c02a410 comp rx=0 tx=0).stop 2024-04-25T00:14:25.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.084+0000 7f5cb2ab3640 1 -- 172.21.15.107:0/2169609387 shutdown_connections 2024-04-25T00:14:25.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.084+0000 7f5cb2ab3640 1 --2- 172.21.15.107:0/2169609387 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cac107460 0x7f5cac1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:25.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.084+0000 7f5cb2ab3640 1 --2- 172.21.15.107:0/2169609387 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cac104b30 0x7f5cac106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:25.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.084+0000 7f5cb2ab3640 1 --2- 172.21.15.107:0/2169609387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cac102200 0x7f5cac1045f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:25.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.084+0000 7f5cb2ab3640 1 -- 172.21.15.107:0/2169609387 >> 172.21.15.107:0/2169609387 conn(0x7f5cac0fbbf0 msgr2=0x7f5cac0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:25.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.084+0000 7f5cb2ab3640 1 -- 172.21.15.107:0/2169609387 shutdown_connections 2024-04-25T00:14:25.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.084+0000 7f5cb2ab3640 1 -- 172.21.15.107:0/2169609387 wait complete. 2024-04-25T00:14:25.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.085+0000 7f5cb2ab3640 1 Processor -- start 2024-04-25T00:14:25.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.085+0000 7f5cb2ab3640 1 -- start start 2024-04-25T00:14:25.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.085+0000 7f5cb2ab3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cac102200 0x7f5cac197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:25.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.085+0000 7f5cb2ab3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cac104b30 0x7f5cac1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:25.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.085+0000 7f5cb2ab3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cac107460 0x7f5cac19f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:25.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.085+0000 7f5cb2ab3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac10d470 con 0x7f5cac107460 2024-04-25T00:14:25.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.085+0000 7f5cb2ab3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac10d2f0 con 0x7f5cac104b30 2024-04-25T00:14:25.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.085+0000 7f5cb2ab3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac10d5f0 con 0x7f5cac102200 2024-04-25T00:14:25.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.085+0000 7f5cb1050640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cac107460 0x7f5cac19f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:25.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.085+0000 7f5cb1050640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cac107460 0x7f5cac19f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59190/0 (socket says 172.21.15.107:59190) 2024-04-25T00:14:25.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.085+0000 7f5cb1050640 1 -- 172.21.15.107:0/838147601 learned_addr learned my addr 172.21.15.107:0/838147601 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:25.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.085+0000 7f5cabfff640 1 --2- 172.21.15.107:0/838147601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cac104b30 0x7f5cac1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:25.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.085+0000 7f5cb1050640 1 -- 172.21.15.107:0/838147601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cac102200 msgr2=0x7f5cac197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:25.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.085+0000 7f5cb084f640 1 --2- 172.21.15.107:0/838147601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cac102200 0x7f5cac197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:25.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.085+0000 7f5cb1050640 1 --2- 172.21.15.107:0/838147601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cac102200 0x7f5cac197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:25.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.085+0000 7f5cb1050640 1 -- 172.21.15.107:0/838147601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cac104b30 msgr2=0x7f5cac1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:25.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.085+0000 7f5cb1050640 1 --2- 172.21.15.107:0/838147601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cac104b30 0x7f5cac1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:25.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.085+0000 7f5cb1050640 1 -- 172.21.15.107:0/838147601 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cac19f790 con 0x7f5cac107460 2024-04-25T00:14:25.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.086+0000 7f5cb084f640 1 --2- 172.21.15.107:0/838147601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cac102200 0x7f5cac197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:25.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.086+0000 7f5cb1050640 1 --2- 172.21.15.107:0/838147601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cac107460 0x7f5cac19f120 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400b970 tx=0x7f5ca400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:25.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.086+0000 7f5ca9ffb640 1 -- 172.21.15.107:0/838147601 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4019070 con 0x7f5cac107460 2024-04-25T00:14:25.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.086+0000 7f5ca9ffb640 1 -- 172.21.15.107:0/838147601 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ca400cb40 con 0x7f5cac107460 2024-04-25T00:14:25.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.086+0000 7f5cb2ab3640 1 -- 172.21.15.107:0/838147601 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cac19fa80 con 0x7f5cac107460 2024-04-25T00:14:25.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.086+0000 7f5ca9ffb640 1 -- 172.21.15.107:0/838147601 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4007650 con 0x7f5cac107460 2024-04-25T00:14:25.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.086+0000 7f5cb2ab3640 1 -- 172.21.15.107:0/838147601 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cac1a0020 con 0x7f5cac107460 2024-04-25T00:14:25.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.087+0000 7f5cb2ab3640 1 -- 172.21.15.107:0/838147601 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cac0fd6e0 con 0x7f5cac107460 2024-04-25T00:14:25.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.088+0000 7f5ca9ffb640 1 -- 172.21.15.107:0/838147601 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f5ca401c070 con 0x7f5cac107460 2024-04-25T00:14:25.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.088+0000 7f5ca9ffb640 1 --2- 172.21.15.107:0/838147601 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5c940a2c10 0x7f5c940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:25.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.089+0000 7f5ca9ffb640 1 -- 172.21.15.107:0/838147601 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f5ca40ce790 con 0x7f5cac107460 2024-04-25T00:14:25.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.089+0000 7f5cb084f640 1 --2- 172.21.15.107:0/838147601 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5c940a2c10 0x7f5c940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:25.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.089+0000 7f5cb084f640 1 --2- 172.21.15.107:0/838147601 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5c940a2c10 0x7f5c940a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c000c00 tx=0x7f5c9c031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:25.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.093+0000 7f5ca9ffb640 1 -- 172.21.15.107:0/838147601 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5ca409ae60 con 0x7f5cac107460 2024-04-25T00:14:25.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:25 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2707996401' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:25.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:25 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1705057982' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:25.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:25.304+0000 7f5cb2ab3640 1 -- 172.21.15.107:0/838147601 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5cac0623e0 con 0x7f5cac107460 2024-04-25T00:14:25.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:25 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2707996401' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:25.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:25 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1705057982' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:25.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:25 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2707996401' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:25.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:25 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1705057982' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:26.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.006+0000 7f5ca9ffb640 1 -- 172.21.15.107:0/838147601 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f5ca409f160 con 0x7f5cac107460 2024-04-25T00:14:26.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.009+0000 7f5cb2ab3640 1 -- 172.21.15.107:0/838147601 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5c940a2c10 msgr2=0x7f5c940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.009+0000 7f5cb2ab3640 1 --2- 172.21.15.107:0/838147601 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5c940a2c10 0x7f5c940a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c000c00 tx=0x7f5c9c031400 comp rx=0 tx=0).stop 2024-04-25T00:14:26.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.009+0000 7f5cb2ab3640 1 -- 172.21.15.107:0/838147601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cac107460 msgr2=0x7f5cac19f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.009+0000 7f5cb2ab3640 1 --2- 172.21.15.107:0/838147601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cac107460 0x7f5cac19f120 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400b970 tx=0x7f5ca400be30 comp rx=0 tx=0).stop 2024-04-25T00:14:26.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.009+0000 7f5cb2ab3640 1 -- 172.21.15.107:0/838147601 shutdown_connections 2024-04-25T00:14:26.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.009+0000 7f5cb2ab3640 1 --2- 172.21.15.107:0/838147601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cac107460 0x7f5cac19f120 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.009+0000 7f5cb2ab3640 1 --2- 172.21.15.107:0/838147601 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5c940a2c10 0x7f5c940a50d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.009+0000 7f5cb2ab3640 1 --2- 172.21.15.107:0/838147601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cac104b30 0x7f5cac1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.009+0000 7f5cb2ab3640 1 --2- 172.21.15.107:0/838147601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cac102200 0x7f5cac197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.009+0000 7f5cb2ab3640 1 -- 172.21.15.107:0/838147601 >> 172.21.15.107:0/838147601 conn(0x7f5cac0fbbf0 msgr2=0x7f5cac102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:26.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.009+0000 7f5cb2ab3640 1 -- 172.21.15.107:0/838147601 shutdown_connections 2024-04-25T00:14:26.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.009+0000 7f5cb2ab3640 1 -- 172.21.15.107:0/838147601 wait complete. 2024-04-25T00:14:26.023 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-04-25T00:14:26.023 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-25T00:14:26.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.123+0000 7fdd3dacd640 1 -- 172.21.15.107:0/353461895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd38103340 msgr2=0x7fdd38109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.123+0000 7fdd3dacd640 1 --2- 172.21.15.107:0/353461895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd38103340 0x7fdd38109bd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fdd30009ca0 tx=0x7fdd3002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.123+0000 7fdd3dacd640 1 -- 172.21.15.107:0/353461895 shutdown_connections 2024-04-25T00:14:26.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.123+0000 7fdd3dacd640 1 --2- 172.21.15.107:0/353461895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd38103340 0x7fdd38109bd0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.123+0000 7fdd3dacd640 1 --2- 172.21.15.107:0/353461895 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd38102980 0x7fdd38102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.123+0000 7fdd3dacd640 1 --2- 172.21.15.107:0/353461895 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd38101790 0x7fdd38101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.123+0000 7fdd3dacd640 1 -- 172.21.15.107:0/353461895 >> 172.21.15.107:0/353461895 conn(0x7fdd380fcd20 msgr2=0x7fdd380ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:26.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.123+0000 7fdd3dacd640 1 -- 172.21.15.107:0/353461895 shutdown_connections 2024-04-25T00:14:26.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.123+0000 7fdd3dacd640 1 -- 172.21.15.107:0/353461895 wait complete. 2024-04-25T00:14:26.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.123+0000 7fdd3dacd640 1 Processor -- start 2024-04-25T00:14:26.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.123+0000 7fdd3dacd640 1 -- start start 2024-04-25T00:14:26.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.124+0000 7fdd3dacd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd38101790 0x7fdd38197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.124+0000 7fdd3dacd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd38102980 0x7fdd38198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.124+0000 7fdd3dacd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd38103340 0x7fdd3819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.124+0000 7fdd3dacd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3810d470 con 0x7fdd38101790 2024-04-25T00:14:26.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.124+0000 7fdd3dacd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3810d2f0 con 0x7fdd38102980 2024-04-25T00:14:26.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.124+0000 7fdd3dacd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3810d5f0 con 0x7fdd38103340 2024-04-25T00:14:26.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.124+0000 7fdd377fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd38101790 0x7fdd38197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.124+0000 7fdd377fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd38101790 0x7fdd38197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59214/0 (socket says 172.21.15.107:59214) 2024-04-25T00:14:26.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.124+0000 7fdd377fe640 1 -- 172.21.15.107:0/1071785004 learned_addr learned my addr 172.21.15.107:0/1071785004 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:26.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.124+0000 7fdd36ffd640 1 --2- 172.21.15.107:0/1071785004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd38102980 0x7fdd38198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.124+0000 7fdd37fff640 1 --2- 172.21.15.107:0/1071785004 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd38103340 0x7fdd3819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.124+0000 7fdd377fe640 1 -- 172.21.15.107:0/1071785004 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd38103340 msgr2=0x7fdd3819f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.124+0000 7fdd377fe640 1 --2- 172.21.15.107:0/1071785004 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd38103340 0x7fdd3819f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.124+0000 7fdd377fe640 1 -- 172.21.15.107:0/1071785004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd38102980 msgr2=0x7fdd38198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.124+0000 7fdd377fe640 1 --2- 172.21.15.107:0/1071785004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd38102980 0x7fdd38198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.124+0000 7fdd377fe640 1 -- 172.21.15.107:0/1071785004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd3819f6e0 con 0x7fdd38101790 2024-04-25T00:14:26.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.124+0000 7fdd37fff640 1 --2- 172.21.15.107:0/1071785004 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd38103340 0x7fdd3819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:26.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.124+0000 7fdd377fe640 1 --2- 172.21.15.107:0/1071785004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd38101790 0x7fdd38197ac0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c00d770 tx=0x7fdd2c00db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:26.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.124+0000 7fdd34ff9640 1 -- 172.21.15.107:0/1071785004 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c017070 con 0x7fdd38101790 2024-04-25T00:14:26.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.124+0000 7fdd34ff9640 1 -- 172.21.15.107:0/1071785004 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd2c009d00 con 0x7fdd38101790 2024-04-25T00:14:26.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.124+0000 7fdd3dacd640 1 -- 172.21.15.107:0/1071785004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd3819f9d0 con 0x7fdd38101790 2024-04-25T00:14:26.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.124+0000 7fdd34ff9640 1 -- 172.21.15.107:0/1071785004 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c003d30 con 0x7fdd38101790 2024-04-25T00:14:26.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.124+0000 7fdd3dacd640 1 -- 172.21.15.107:0/1071785004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd3819ffa0 con 0x7fdd38101790 2024-04-25T00:14:26.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.125+0000 7fdd3dacd640 1 -- 172.21.15.107:0/1071785004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd38108ab0 con 0x7fdd38101790 2024-04-25T00:14:26.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.126+0000 7fdd34ff9640 1 -- 172.21.15.107:0/1071785004 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fdd2c009820 con 0x7fdd38101790 2024-04-25T00:14:26.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.127+0000 7fdd34ff9640 1 --2- 172.21.15.107:0/1071785004 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fdd200a2b90 0x7fdd200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.127+0000 7fdd34ff9640 1 -- 172.21.15.107:0/1071785004 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fdd2c0cec20 con 0x7fdd38101790 2024-04-25T00:14:26.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.127+0000 7fdd36ffd640 1 --2- 172.21.15.107:0/1071785004 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fdd200a2b90 0x7fdd200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.127+0000 7fdd36ffd640 1 --2- 172.21.15.107:0/1071785004 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fdd200a2b90 0x7fdd200a5050 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdd38198fd0 tx=0x7fdd28009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:26.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.132+0000 7fdd34ff9640 1 -- 172.21.15.107:0/1071785004 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdd2c09b940 con 0x7fdd38101790 2024-04-25T00:14:26.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:26 smithi195 ceph-mon[32833]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T00:14:26.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:26 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/838147601' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T00:14:26.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.346+0000 7fdd3dacd640 1 -- 172.21.15.107:0/1071785004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fdd38062340 con 0x7fdd38101790 2024-04-25T00:14:26.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:26 smithi170 ceph-mon[32807]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T00:14:26.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:26 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/838147601' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T00:14:26.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:26 smithi107 ceph-mon[29153]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T00:14:26.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:26 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/838147601' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T00:14:26.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.516+0000 7fdd34ff9640 1 -- 172.21.15.107:0/1071785004 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7fdd2c09b240 con 0x7fdd38101790 2024-04-25T00:14:26.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.519+0000 7fdd3dacd640 1 -- 172.21.15.107:0/1071785004 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fdd200a2b90 msgr2=0x7fdd200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.519+0000 7fdd3dacd640 1 --2- 172.21.15.107:0/1071785004 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fdd200a2b90 0x7fdd200a5050 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdd38198fd0 tx=0x7fdd28009110 comp rx=0 tx=0).stop 2024-04-25T00:14:26.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.519+0000 7fdd3dacd640 1 -- 172.21.15.107:0/1071785004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd38101790 msgr2=0x7fdd38197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.519+0000 7fdd3dacd640 1 --2- 172.21.15.107:0/1071785004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd38101790 0x7fdd38197ac0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c00d770 tx=0x7fdd2c00db70 comp rx=0 tx=0).stop 2024-04-25T00:14:26.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.519+0000 7fdd3dacd640 1 -- 172.21.15.107:0/1071785004 shutdown_connections 2024-04-25T00:14:26.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.519+0000 7fdd3dacd640 1 --2- 172.21.15.107:0/1071785004 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdd38103340 0x7fdd3819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.519+0000 7fdd3dacd640 1 --2- 172.21.15.107:0/1071785004 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fdd200a2b90 0x7fdd200a5050 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.519+0000 7fdd3dacd640 1 --2- 172.21.15.107:0/1071785004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdd38102980 0x7fdd38198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.519+0000 7fdd3dacd640 1 --2- 172.21.15.107:0/1071785004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd38101790 0x7fdd38197ac0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.519+0000 7fdd3dacd640 1 -- 172.21.15.107:0/1071785004 >> 172.21.15.107:0/1071785004 conn(0x7fdd380fcd20 msgr2=0x7fdd38105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:26.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.519+0000 7fdd3dacd640 1 -- 172.21.15.107:0/1071785004 shutdown_connections 2024-04-25T00:14:26.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.519+0000 7fdd3dacd640 1 -- 172.21.15.107:0/1071785004 wait complete. 2024-04-25T00:14:26.534 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T00:14:26.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.634+0000 7f2d3144d640 1 -- 172.21.15.107:0/1926697909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d2c103330 msgr2=0x7f2d2c109bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.634+0000 7f2d3144d640 1 --2- 172.21.15.107:0/1926697909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d2c103330 0x7f2d2c109bc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009900 tx=0x7f2d1c02a410 comp rx=0 tx=0).stop 2024-04-25T00:14:26.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.634+0000 7f2d3144d640 1 -- 172.21.15.107:0/1926697909 shutdown_connections 2024-04-25T00:14:26.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.634+0000 7f2d3144d640 1 --2- 172.21.15.107:0/1926697909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d2c103330 0x7f2d2c109bc0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.634+0000 7f2d3144d640 1 --2- 172.21.15.107:0/1926697909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d2c102970 0x7f2d2c102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.634+0000 7f2d3144d640 1 --2- 172.21.15.107:0/1926697909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d2c101780 0x7f2d2c101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.634+0000 7f2d3144d640 1 -- 172.21.15.107:0/1926697909 >> 172.21.15.107:0/1926697909 conn(0x7f2d2c0fccf0 msgr2=0x7f2d2c0ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:26.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.634+0000 7f2d3144d640 1 -- 172.21.15.107:0/1926697909 shutdown_connections 2024-04-25T00:14:26.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.634+0000 7f2d3144d640 1 -- 172.21.15.107:0/1926697909 wait complete. 2024-04-25T00:14:26.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.634+0000 7f2d3144d640 1 Processor -- start 2024-04-25T00:14:26.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.635+0000 7f2d3144d640 1 -- start start 2024-04-25T00:14:26.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.635+0000 7f2d3144d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d2c101780 0x7f2d2c0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.635+0000 7f2d3144d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d2c102970 0x7f2d2c073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.635+0000 7f2d3144d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d2c103330 0x7f2d2c074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.635+0000 7f2d3144d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c10d430 con 0x7f2d2c103330 2024-04-25T00:14:26.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.635+0000 7f2d3144d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c10d2b0 con 0x7f2d2c101780 2024-04-25T00:14:26.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.635+0000 7f2d3144d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c10d5b0 con 0x7f2d2c102970 2024-04-25T00:14:26.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.635+0000 7f2d2b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d2c103330 0x7f2d2c074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.635+0000 7f2d2b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d2c103330 0x7f2d2c074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59232/0 (socket says 172.21.15.107:59232) 2024-04-25T00:14:26.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.635+0000 7f2d2b7fe640 1 -- 172.21.15.107:0/1699155996 learned_addr learned my addr 172.21.15.107:0/1699155996 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:26.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.635+0000 7f2d2a7fc640 1 --2- 172.21.15.107:0/1699155996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d2c102970 0x7f2d2c073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.635+0000 7f2d2affd640 1 --2- 172.21.15.107:0/1699155996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d2c101780 0x7f2d2c0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.635+0000 7f2d2b7fe640 1 -- 172.21.15.107:0/1699155996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d2c102970 msgr2=0x7f2d2c073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.635+0000 7f2d2b7fe640 1 --2- 172.21.15.107:0/1699155996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d2c102970 0x7f2d2c073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.635+0000 7f2d2b7fe640 1 -- 172.21.15.107:0/1699155996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d2c101780 msgr2=0x7f2d2c0754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:26.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.635+0000 7f2d2b7fe640 1 --2- 172.21.15.107:0/1699155996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d2c101780 0x7f2d2c0754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:26.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.635+0000 7f2d2b7fe640 1 -- 172.21.15.107:0/1699155996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2c074900 con 0x7f2d2c103330 2024-04-25T00:14:26.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.635+0000 7f2d2b7fe640 1 --2- 172.21.15.107:0/1699155996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d2c103330 0x7f2d2c074070 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c02a930 tx=0x7f2d1c032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:26.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.635+0000 7f2d13fff640 1 -- 172.21.15.107:0/1699155996 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c03f030 con 0x7f2d2c103330 2024-04-25T00:14:26.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.635+0000 7f2d13fff640 1 -- 172.21.15.107:0/1699155996 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d1c032e50 con 0x7f2d2c103330 2024-04-25T00:14:26.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.636+0000 7f2d3144d640 1 -- 172.21.15.107:0/1699155996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d2c1a1b10 con 0x7f2d2c103330 2024-04-25T00:14:26.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.636+0000 7f2d13fff640 1 -- 172.21.15.107:0/1699155996 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c035070 con 0x7f2d2c103330 2024-04-25T00:14:26.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.636+0000 7f2d3144d640 1 -- 172.21.15.107:0/1699155996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d2c1a2060 con 0x7f2d2c103330 2024-04-25T00:14:26.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.637+0000 7f2d3144d640 1 -- 172.21.15.107:0/1699155996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d2c108aa0 con 0x7f2d2c103330 2024-04-25T00:14:26.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.638+0000 7f2d13fff640 1 -- 172.21.15.107:0/1699155996 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f2d1c042030 con 0x7f2d2c103330 2024-04-25T00:14:26.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.638+0000 7f2d13fff640 1 --2- 172.21.15.107:0/1699155996 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f2d140a2b90 0x7f2d140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:26.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.638+0000 7f2d2affd640 1 --2- 172.21.15.107:0/1699155996 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f2d140a2b90 0x7f2d140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:26.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.638+0000 7f2d13fff640 1 -- 172.21.15.107:0/1699155996 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f2d1c0ef740 con 0x7f2d2c103330 2024-04-25T00:14:26.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.639+0000 7f2d2affd640 1 --2- 172.21.15.107:0/1699155996 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f2d140a2b90 0x7f2d140a5050 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c1027d0 tx=0x7f2d24009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:26.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.643+0000 7f2d13fff640 1 -- 172.21.15.107:0/1699155996 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2d1c0bbe10 con 0x7f2d2c103330 2024-04-25T00:14:26.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:26.858+0000 7f2d3144d640 1 -- 172.21.15.107:0/1699155996 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f2d2c062340 con 0x7f2d2c103330 2024-04-25T00:14:27.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:27 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/838147601' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T00:14:27.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:27 smithi195 ceph-mon[32833]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:14:27.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:27 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1071785004' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T00:14:27.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:27 smithi195 ceph-mon[32833]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T00:14:27.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:27 smithi195 ceph-mon[32833]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T00:14:27.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:27 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1071785004' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T00:14:27.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:27 smithi195 ceph-mon[32833]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T00:14:27.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:27 smithi195 ceph-mon[32833]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T00:14:27.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:27 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1699155996' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T00:14:27.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:27 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/838147601' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T00:14:27.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:27 smithi170 ceph-mon[32807]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:14:27.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:27 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1071785004' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T00:14:27.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:27 smithi170 ceph-mon[32807]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T00:14:27.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:27 smithi170 ceph-mon[32807]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T00:14:27.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:27 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1071785004' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T00:14:27.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:27 smithi170 ceph-mon[32807]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T00:14:27.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:27 smithi170 ceph-mon[32807]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T00:14:27.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:27 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1699155996' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T00:14:27.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:27 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/838147601' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T00:14:27.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:27 smithi107 ceph-mon[29153]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:14:27.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:27 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1071785004' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T00:14:27.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:27 smithi107 ceph-mon[29153]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T00:14:27.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:27 smithi107 ceph-mon[29153]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T00:14:27.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:27 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1071785004' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T00:14:27.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:27 smithi107 ceph-mon[29153]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T00:14:27.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:27 smithi107 ceph-mon[29153]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T00:14:27.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:27 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1699155996' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T00:14:27.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.518+0000 7f2d13fff640 1 -- 172.21.15.107:0/1699155996 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f2d1c0c0110 con 0x7f2d2c103330 2024-04-25T00:14:27.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.520+0000 7f2d3144d640 1 -- 172.21.15.107:0/1699155996 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f2d140a2b90 msgr2=0x7f2d140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:27.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.520+0000 7f2d3144d640 1 --2- 172.21.15.107:0/1699155996 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f2d140a2b90 0x7f2d140a5050 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c1027d0 tx=0x7f2d24009500 comp rx=0 tx=0).stop 2024-04-25T00:14:27.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.520+0000 7f2d3144d640 1 -- 172.21.15.107:0/1699155996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d2c103330 msgr2=0x7f2d2c074070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:27.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.520+0000 7f2d3144d640 1 --2- 172.21.15.107:0/1699155996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d2c103330 0x7f2d2c074070 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c02a930 tx=0x7f2d1c032c60 comp rx=0 tx=0).stop 2024-04-25T00:14:27.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.521+0000 7f2d3144d640 1 -- 172.21.15.107:0/1699155996 shutdown_connections 2024-04-25T00:14:27.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.521+0000 7f2d3144d640 1 --2- 172.21.15.107:0/1699155996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d2c103330 0x7f2d2c074070 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.521+0000 7f2d3144d640 1 --2- 172.21.15.107:0/1699155996 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2d2c102970 0x7f2d2c073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.521+0000 7f2d3144d640 1 --2- 172.21.15.107:0/1699155996 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f2d140a2b90 0x7f2d140a5050 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.521+0000 7f2d3144d640 1 --2- 172.21.15.107:0/1699155996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2d2c101780 0x7f2d2c0754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.521+0000 7f2d3144d640 1 -- 172.21.15.107:0/1699155996 >> 172.21.15.107:0/1699155996 conn(0x7f2d2c0fccf0 msgr2=0x7f2d2c105c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:27.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.521+0000 7f2d3144d640 1 -- 172.21.15.107:0/1699155996 shutdown_connections 2024-04-25T00:14:27.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.521+0000 7f2d3144d640 1 -- 172.21.15.107:0/1699155996 wait complete. 2024-04-25T00:14:27.538 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T00:14:27.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.640+0000 7f755829a640 1 -- 172.21.15.107:0/1490175177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7550102230 msgr2=0x7f7550108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:27.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.640+0000 7f755829a640 1 --2- 172.21.15.107:0/1490175177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7550102230 0x7f7550108ac0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7548009ca0 tx=0x7f754802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.641+0000 7f755829a640 1 -- 172.21.15.107:0/1490175177 shutdown_connections 2024-04-25T00:14:27.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.641+0000 7f755829a640 1 --2- 172.21.15.107:0/1490175177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7550102230 0x7f7550108ac0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.641+0000 7f755829a640 1 --2- 172.21.15.107:0/1490175177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7550101870 0x7f7550101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.641+0000 7f755829a640 1 --2- 172.21.15.107:0/1490175177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7550100680 0x7f7550100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.641+0000 7f755829a640 1 -- 172.21.15.107:0/1490175177 >> 172.21.15.107:0/1490175177 conn(0x7f75500fbbf0 msgr2=0x7f75500fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:27.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.641+0000 7f755829a640 1 -- 172.21.15.107:0/1490175177 shutdown_connections 2024-04-25T00:14:27.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.641+0000 7f755829a640 1 -- 172.21.15.107:0/1490175177 wait complete. 2024-04-25T00:14:27.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.641+0000 7f755829a640 1 Processor -- start 2024-04-25T00:14:27.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.641+0000 7f755829a640 1 -- start start 2024-04-25T00:14:27.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.641+0000 7f755829a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7550100680 0x7f7550069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:27.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.641+0000 7f755829a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7550101870 0x7f755019c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:27.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.641+0000 7f755829a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7550102230 0x7f755019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:27.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.641+0000 7f755829a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f755010c330 con 0x7f7550102230 2024-04-25T00:14:27.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.641+0000 7f755829a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f755010c1b0 con 0x7f7550101870 2024-04-25T00:14:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.641+0000 7f755829a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f755010c4b0 con 0x7f7550100680 2024-04-25T00:14:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.641+0000 7f7556837640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7550102230 0x7f755019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.641+0000 7f7556837640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7550102230 0x7f755019ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59248/0 (socket says 172.21.15.107:59248) 2024-04-25T00:14:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.641+0000 7f7556837640 1 -- 172.21.15.107:0/355733161 learned_addr learned my addr 172.21.15.107:0/355733161 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.641+0000 7f7555835640 1 --2- 172.21.15.107:0/355733161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7550101870 0x7f755019c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.642+0000 7f7556036640 1 --2- 172.21.15.107:0/355733161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7550100680 0x7f7550069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.642+0000 7f7556837640 1 -- 172.21.15.107:0/355733161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7550100680 msgr2=0x7f7550069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.642+0000 7f7556837640 1 --2- 172.21.15.107:0/355733161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7550100680 0x7f7550069460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.642+0000 7f7556837640 1 -- 172.21.15.107:0/355733161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7550101870 msgr2=0x7f755019c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.642+0000 7f7556837640 1 --2- 172.21.15.107:0/355733161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7550101870 0x7f755019c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.642+0000 7f7556837640 1 -- 172.21.15.107:0/355733161 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f755019f6d0 con 0x7f7550102230 2024-04-25T00:14:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.642+0000 7f7555835640 1 --2- 172.21.15.107:0/355733161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7550101870 0x7f755019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:27.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.642+0000 7f7556837640 1 --2- 172.21.15.107:0/355733161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7550102230 0x7f755019ee10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7548009430 tx=0x7f75480319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:27.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.642+0000 7f75437fe640 1 -- 172.21.15.107:0/355733161 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7548003b70 con 0x7f7550102230 2024-04-25T00:14:27.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.642+0000 7f75437fe640 1 -- 172.21.15.107:0/355733161 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7548003d10 con 0x7f7550102230 2024-04-25T00:14:27.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.642+0000 7f755829a640 1 -- 172.21.15.107:0/355733161 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f755019f960 con 0x7f7550102230 2024-04-25T00:14:27.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.642+0000 7f75437fe640 1 -- 172.21.15.107:0/355733161 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f754802fc50 con 0x7f7550102230 2024-04-25T00:14:27.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.642+0000 7f755829a640 1 -- 172.21.15.107:0/355733161 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f755019feb0 con 0x7f7550102230 2024-04-25T00:14:27.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.643+0000 7f755829a640 1 -- 172.21.15.107:0/355733161 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75501079a0 con 0x7f7550102230 2024-04-25T00:14:27.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.644+0000 7f75437fe640 1 -- 172.21.15.107:0/355733161 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f7548004040 con 0x7f7550102230 2024-04-25T00:14:27.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.645+0000 7f75437fe640 1 --2- 172.21.15.107:0/355733161 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f753c0a2b90 0x7f753c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:27.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.645+0000 7f7556036640 1 --2- 172.21.15.107:0/355733161 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f753c0a2b90 0x7f753c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:27.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.645+0000 7f75437fe640 1 -- 172.21.15.107:0/355733161 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f75480f0730 con 0x7f7550102230 2024-04-25T00:14:27.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.646+0000 7f7556036640 1 --2- 172.21.15.107:0/355733161 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f753c0a2b90 0x7f753c0a5050 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f75501016d0 tx=0x7f7544005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:27.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.650+0000 7f75437fe640 1 -- 172.21.15.107:0/355733161 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f75480bd450 con 0x7f7550102230 2024-04-25T00:14:27.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:27.863+0000 7f755829a640 1 -- 172.21.15.107:0/355733161 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f7550062340 con 0x7f7550102230 2024-04-25T00:14:27.934 INFO:journalctl@ceph.mds.b.smithi170.stdout:Apr 25 00:14:27 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-b[62656]: 2024-04-25T00:14:27.670+0000 7febf57a6700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T00:14:28.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:28 smithi195 ceph-mon[32833]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T00:14:28.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:28 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1699155996' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T00:14:28.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:28 smithi195 ceph-mon[32833]: mds.? [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] up:active 2024-04-25T00:14:28.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:28 smithi195 ceph-mon[32833]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T00:14:28.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:28 smithi195 ceph-mon[32833]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T00:14:28.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:28 smithi195 ceph-mon[32833]: Cluster is now healthy 2024-04-25T00:14:28.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:28 smithi195 ceph-mon[32833]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-25T00:14:28.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:28 smithi195 ceph-mon[32833]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-25T00:14:28.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:28 smithi195 ceph-mon[32833]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-25T00:14:28.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:28 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/355733161' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T00:14:28.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:28 smithi170 ceph-mon[32807]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T00:14:28.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:28 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1699155996' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T00:14:28.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:28 smithi170 ceph-mon[32807]: mds.? [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] up:active 2024-04-25T00:14:28.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:28 smithi170 ceph-mon[32807]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T00:14:28.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:28 smithi170 ceph-mon[32807]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T00:14:28.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:28 smithi170 ceph-mon[32807]: Cluster is now healthy 2024-04-25T00:14:28.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:28 smithi170 ceph-mon[32807]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-25T00:14:28.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:28 smithi170 ceph-mon[32807]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-25T00:14:28.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:28 smithi170 ceph-mon[32807]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-25T00:14:28.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:28 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/355733161' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T00:14:28.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:28 smithi107 ceph-mon[29153]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-04-25T00:14:28.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:28 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1699155996' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T00:14:28.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:28 smithi107 ceph-mon[29153]: mds.? [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] up:active 2024-04-25T00:14:28.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:28 smithi107 ceph-mon[29153]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T00:14:28.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:28 smithi107 ceph-mon[29153]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T00:14:28.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:28 smithi107 ceph-mon[29153]: Cluster is now healthy 2024-04-25T00:14:28.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:28 smithi107 ceph-mon[29153]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-25T00:14:28.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:28 smithi107 ceph-mon[29153]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-25T00:14:28.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:28 smithi107 ceph-mon[29153]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-25T00:14:28.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:28 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/355733161' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T00:14:28.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.524+0000 7f75437fe640 1 -- 172.21.15.107:0/355733161 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7f75480bcd50 con 0x7f7550102230 2024-04-25T00:14:28.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.527+0000 7f755829a640 1 -- 172.21.15.107:0/355733161 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f753c0a2b90 msgr2=0x7f753c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:28.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.527+0000 7f755829a640 1 --2- 172.21.15.107:0/355733161 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f753c0a2b90 0x7f753c0a5050 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f75501016d0 tx=0x7f7544005f90 comp rx=0 tx=0).stop 2024-04-25T00:14:28.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.527+0000 7f755829a640 1 -- 172.21.15.107:0/355733161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7550102230 msgr2=0x7f755019ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:28.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.527+0000 7f755829a640 1 --2- 172.21.15.107:0/355733161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7550102230 0x7f755019ee10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7548009430 tx=0x7f75480319f0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.527+0000 7f755829a640 1 -- 172.21.15.107:0/355733161 shutdown_connections 2024-04-25T00:14:28.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.527+0000 7f755829a640 1 --2- 172.21.15.107:0/355733161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7550102230 0x7f755019ee10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.527+0000 7f755829a640 1 --2- 172.21.15.107:0/355733161 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f753c0a2b90 0x7f753c0a5050 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.527+0000 7f755829a640 1 --2- 172.21.15.107:0/355733161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7550101870 0x7f755019c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.527+0000 7f755829a640 1 --2- 172.21.15.107:0/355733161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7550100680 0x7f7550069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.527+0000 7f755829a640 1 -- 172.21.15.107:0/355733161 >> 172.21.15.107:0/355733161 conn(0x7f75500fbbf0 msgr2=0x7f7550104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:28.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.527+0000 7f755829a640 1 -- 172.21.15.107:0/355733161 shutdown_connections 2024-04-25T00:14:28.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.527+0000 7f755829a640 1 -- 172.21.15.107:0/355733161 wait complete. 2024-04-25T00:14:28.542 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-04-25T00:14:28.542 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T00:14:28.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.648+0000 7f15c6509640 1 -- 172.21.15.107:0/3546962036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15c01006a0 msgr2=0x7f15c0100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:28.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.648+0000 7f15c6509640 1 --2- 172.21.15.107:0/3546962036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15c01006a0 0x7f15c0100ac0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009900 tx=0x7f15a802a410 comp rx=0 tx=0).stop 2024-04-25T00:14:28.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.648+0000 7f15c6509640 1 -- 172.21.15.107:0/3546962036 shutdown_connections 2024-04-25T00:14:28.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.648+0000 7f15c6509640 1 --2- 172.21.15.107:0/3546962036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15c0102250 0x7f15c0108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.648+0000 7f15c6509640 1 --2- 172.21.15.107:0/3546962036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15c0101890 0x7f15c0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.648+0000 7f15c6509640 1 --2- 172.21.15.107:0/3546962036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15c01006a0 0x7f15c0100ac0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.648+0000 7f15c6509640 1 -- 172.21.15.107:0/3546962036 >> 172.21.15.107:0/3546962036 conn(0x7f15c00fbc10 msgr2=0x7f15c00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:28.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.648+0000 7f15c6509640 1 -- 172.21.15.107:0/3546962036 shutdown_connections 2024-04-25T00:14:28.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.648+0000 7f15c6509640 1 -- 172.21.15.107:0/3546962036 wait complete. 2024-04-25T00:14:28.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.648+0000 7f15c6509640 1 Processor -- start 2024-04-25T00:14:28.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.648+0000 7f15c6509640 1 -- start start 2024-04-25T00:14:28.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.649+0000 7f15c6509640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15c01006a0 0x7f15c0196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:28.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.649+0000 7f15c6509640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15c0101890 0x7f15c0196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:28.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.649+0000 7f15c6509640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15c0102250 0x7f15c019dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:28.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.649+0000 7f15c6509640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f15c010c2f0 con 0x7f15c0101890 2024-04-25T00:14:28.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.649+0000 7f15c6509640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f15c010c170 con 0x7f15c01006a0 2024-04-25T00:14:28.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.649+0000 7f15c6509640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f15c010c470 con 0x7f15c0102250 2024-04-25T00:14:28.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.649+0000 7f15bf7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15c0101890 0x7f15c0196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:28.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.649+0000 7f15bf7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15c0101890 0x7f15c0196f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59274/0 (socket says 172.21.15.107:59274) 2024-04-25T00:14:28.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.649+0000 7f15bf7fe640 1 -- 172.21.15.107:0/4253066608 learned_addr learned my addr 172.21.15.107:0/4253066608 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:28.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.649+0000 7f15bffff640 1 --2- 172.21.15.107:0/4253066608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15c01006a0 0x7f15c0196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:28.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.649+0000 7f15c4aa6640 1 --2- 172.21.15.107:0/4253066608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15c0102250 0x7f15c019dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:28.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.649+0000 7f15bf7fe640 1 -- 172.21.15.107:0/4253066608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15c0102250 msgr2=0x7f15c019dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:28.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.649+0000 7f15bf7fe640 1 --2- 172.21.15.107:0/4253066608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15c0102250 0x7f15c019dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.649+0000 7f15bf7fe640 1 -- 172.21.15.107:0/4253066608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15c01006a0 msgr2=0x7f15c0196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:28.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.649+0000 7f15bf7fe640 1 --2- 172.21.15.107:0/4253066608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15c01006a0 0x7f15c0196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.649+0000 7f15bf7fe640 1 -- 172.21.15.107:0/4253066608 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c019e630 con 0x7f15c0101890 2024-04-25T00:14:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.649+0000 7f15bffff640 1 --2- 172.21.15.107:0/4253066608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15c01006a0 0x7f15c0196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.649+0000 7f15bf7fe640 1 --2- 172.21.15.107:0/4253066608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15c0101890 0x7f15c0196f50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f15b000d950 tx=0x7f15b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.649+0000 7f15bd7fa640 1 -- 172.21.15.107:0/4253066608 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b0017070 con 0x7f15c0101890 2024-04-25T00:14:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.649+0000 7f15bd7fa640 1 -- 172.21.15.107:0/4253066608 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15b0009c30 con 0x7f15c0101890 2024-04-25T00:14:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.649+0000 7f15c6509640 1 -- 172.21.15.107:0/4253066608 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c019e920 con 0x7f15c0101890 2024-04-25T00:14:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.649+0000 7f15bd7fa640 1 -- 172.21.15.107:0/4253066608 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b000f6b0 con 0x7f15c0101890 2024-04-25T00:14:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.649+0000 7f15c6509640 1 -- 172.21.15.107:0/4253066608 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c019ee70 con 0x7f15c0101890 2024-04-25T00:14:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.650+0000 7f15c6509640 1 -- 172.21.15.107:0/4253066608 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15c0107a90 con 0x7f15c0101890 2024-04-25T00:14:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.651+0000 7f15bd7fa640 1 -- 172.21.15.107:0/4253066608 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f15b00109d0 con 0x7f15c0101890 2024-04-25T00:14:28.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.652+0000 7f15bd7fa640 1 --2- 172.21.15.107:0/4253066608 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f15ac0a2b90 0x7f15ac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:28.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.652+0000 7f15bffff640 1 --2- 172.21.15.107:0/4253066608 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f15ac0a2b90 0x7f15ac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:28.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.652+0000 7f15bd7fa640 1 -- 172.21.15.107:0/4253066608 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f15b00cef10 con 0x7f15c0101890 2024-04-25T00:14:28.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.652+0000 7f15bffff640 1 --2- 172.21.15.107:0/4253066608 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f15ac0a2b90 0x7f15ac0a5050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f15a8005ab0 tx=0x7f15a8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:28.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.656+0000 7f15bd7fa640 1 -- 172.21.15.107:0/4253066608 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f15b0009de0 con 0x7f15c0101890 2024-04-25T00:14:28.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.857+0000 7f15c6509640 1 -- 172.21.15.107:0/4253066608 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f15c0061b80 con 0x7f15ac0a2b90 2024-04-25T00:14:28.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.880+0000 7f15bd7fa640 1 -- 172.21.15.107:0/4253066608 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f15c0061b80 con 0x7f15ac0a2b90 2024-04-25T00:14:28.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.882+0000 7f15c6509640 1 -- 172.21.15.107:0/4253066608 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f15ac0a2b90 msgr2=0x7f15ac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:28.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.882+0000 7f15c6509640 1 --2- 172.21.15.107:0/4253066608 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f15ac0a2b90 0x7f15ac0a5050 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f15a8005ab0 tx=0x7f15a8000bc0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.882+0000 7f15c6509640 1 -- 172.21.15.107:0/4253066608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15c0101890 msgr2=0x7f15c0196f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:28.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.882+0000 7f15c6509640 1 --2- 172.21.15.107:0/4253066608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15c0101890 0x7f15c0196f50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f15b000d950 tx=0x7f15b000de10 comp rx=0 tx=0).stop 2024-04-25T00:14:28.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.882+0000 7f15c6509640 1 -- 172.21.15.107:0/4253066608 shutdown_connections 2024-04-25T00:14:28.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.882+0000 7f15c6509640 1 --2- 172.21.15.107:0/4253066608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15c0102250 0x7f15c019dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.882+0000 7f15c6509640 1 --2- 172.21.15.107:0/4253066608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15c0101890 0x7f15c0196f50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.882+0000 7f15c6509640 1 --2- 172.21.15.107:0/4253066608 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f15ac0a2b90 0x7f15ac0a5050 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.882+0000 7f15c6509640 1 --2- 172.21.15.107:0/4253066608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15c01006a0 0x7f15c0196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:28.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.882+0000 7f15c6509640 1 -- 172.21.15.107:0/4253066608 >> 172.21.15.107:0/4253066608 conn(0x7f15c00fbc10 msgr2=0x7f15c0104c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:28.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.882+0000 7f15c6509640 1 -- 172.21.15.107:0/4253066608 shutdown_connections 2024-04-25T00:14:28.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.882+0000 7f15c6509640 1 -- 172.21.15.107:0/4253066608 wait complete. 2024-04-25T00:14:28.895 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T00:14:29.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.996+0000 7fb77fd70640 1 -- 172.21.15.107:0/1013980799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb778102980 msgr2=0x7fb778102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.996+0000 7fb77fd70640 1 --2- 172.21.15.107:0/1013980799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb778102980 0x7fb778102e00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009900 tx=0x7fb76c02a410 comp rx=0 tx=0).stop 2024-04-25T00:14:29.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.996+0000 7fb77fd70640 1 -- 172.21.15.107:0/1013980799 shutdown_connections 2024-04-25T00:14:29.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.996+0000 7fb77fd70640 1 --2- 172.21.15.107:0/1013980799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb778103340 0x7fb778109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.996+0000 7fb77fd70640 1 --2- 172.21.15.107:0/1013980799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb778102980 0x7fb778102e00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.996+0000 7fb77fd70640 1 --2- 172.21.15.107:0/1013980799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb778101790 0x7fb778101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.996+0000 7fb77fd70640 1 -- 172.21.15.107:0/1013980799 >> 172.21.15.107:0/1013980799 conn(0x7fb7780fcd20 msgr2=0x7fb7780ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:29.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.996+0000 7fb77fd70640 1 -- 172.21.15.107:0/1013980799 shutdown_connections 2024-04-25T00:14:29.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.996+0000 7fb77fd70640 1 -- 172.21.15.107:0/1013980799 wait complete. 2024-04-25T00:14:29.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.997+0000 7fb77fd70640 1 Processor -- start 2024-04-25T00:14:29.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.997+0000 7fb77fd70640 1 -- start start 2024-04-25T00:14:29.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.997+0000 7fb77fd70640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb778101790 0x7fb778197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.997+0000 7fb77fd70640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb778102980 0x7fb778197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.997+0000 7fb77fd70640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb778103340 0x7fb77819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.997+0000 7fb77fd70640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb77810d410 con 0x7fb778102980 2024-04-25T00:14:29.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.997+0000 7fb77fd70640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb77810d290 con 0x7fb778101790 2024-04-25T00:14:29.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.997+0000 7fb77fd70640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb77810d590 con 0x7fb778103340 2024-04-25T00:14:29.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.997+0000 7fb77d30b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb778102980 0x7fb778197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.997+0000 7fb77d30b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb778102980 0x7fb778197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59300/0 (socket says 172.21.15.107:59300) 2024-04-25T00:14:29.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.997+0000 7fb77d30b640 1 -- 172.21.15.107:0/3376632218 learned_addr learned my addr 172.21.15.107:0/3376632218 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:29.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.997+0000 7fb77db0c640 1 --2- 172.21.15.107:0/3376632218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb778101790 0x7fb778197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.997+0000 7fb77d30b640 1 -- 172.21.15.107:0/3376632218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb778103340 msgr2=0x7fb77819f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.997+0000 7fb77d30b640 1 --2- 172.21.15.107:0/3376632218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb778103340 0x7fb77819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.997+0000 7fb77d30b640 1 -- 172.21.15.107:0/3376632218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb778101790 msgr2=0x7fb778197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.997+0000 7fb77d30b640 1 --2- 172.21.15.107:0/3376632218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb778101790 0x7fb778197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.997+0000 7fb77d30b640 1 -- 172.21.15.107:0/3376632218 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77819f6d0 con 0x7fb778102980 2024-04-25T00:14:29.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.998+0000 7fb77d30b640 1 --2- 172.21.15.107:0/3376632218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb778102980 0x7fb778197ff0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009230 tx=0x7fb76c031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:29.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.998+0000 7fb776ffd640 1 -- 172.21.15.107:0/3376632218 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c03f030 con 0x7fb778102980 2024-04-25T00:14:29.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.998+0000 7fb77fd70640 1 -- 172.21.15.107:0/3376632218 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77819f960 con 0x7fb778102980 2024-04-25T00:14:29.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.998+0000 7fb776ffd640 1 -- 172.21.15.107:0/3376632218 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb76c003ad0 con 0x7fb778102980 2024-04-25T00:14:29.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.998+0000 7fb776ffd640 1 -- 172.21.15.107:0/3376632218 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c035070 con 0x7fb778102980 2024-04-25T00:14:29.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.998+0000 7fb77fd70640 1 -- 172.21.15.107:0/3376632218 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb77819fe50 con 0x7fb778102980 2024-04-25T00:14:29.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:28.999+0000 7fb77fd70640 1 -- 172.21.15.107:0/3376632218 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb778108b30 con 0x7fb778102980 2024-04-25T00:14:29.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.000+0000 7fb776ffd640 1 -- 172.21.15.107:0/3376632218 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7fb76c042030 con 0x7fb778102980 2024-04-25T00:14:29.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.000+0000 7fb776ffd640 1 --2- 172.21.15.107:0/3376632218 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb7600a2b90 0x7fb7600a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.001+0000 7fb77db0c640 1 --2- 172.21.15.107:0/3376632218 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb7600a2b90 0x7fb7600a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.001+0000 7fb776ffd640 1 -- 172.21.15.107:0/3376632218 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fb76c0efc10 con 0x7fb778102980 2024-04-25T00:14:29.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.001+0000 7fb77db0c640 1 --2- 172.21.15.107:0/3376632218 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb7600a2b90 0x7fb7600a5050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb768005c40 tx=0x7fb76800a590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:29.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.004+0000 7fb776ffd640 1 -- 172.21.15.107:0/3376632218 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb76c0bc930 con 0x7fb778102980 2024-04-25T00:14:29.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.206+0000 7fb77fd70640 1 -- 172.21.15.107:0/3376632218 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb778061b80 con 0x7fb7600a2b90 2024-04-25T00:14:29.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.229+0000 7fb776ffd640 1 -- 172.21.15.107:0/3376632218 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb778061b80 con 0x7fb7600a2b90 2024-04-25T00:14:29.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.231+0000 7fb77fd70640 1 -- 172.21.15.107:0/3376632218 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb7600a2b90 msgr2=0x7fb7600a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.231+0000 7fb77fd70640 1 --2- 172.21.15.107:0/3376632218 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb7600a2b90 0x7fb7600a5050 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb768005c40 tx=0x7fb76800a590 comp rx=0 tx=0).stop 2024-04-25T00:14:29.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.231+0000 7fb77fd70640 1 -- 172.21.15.107:0/3376632218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb778102980 msgr2=0x7fb778197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.231+0000 7fb77fd70640 1 --2- 172.21.15.107:0/3376632218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb778102980 0x7fb778197ff0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009230 tx=0x7fb76c031b60 comp rx=0 tx=0).stop 2024-04-25T00:14:29.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.231+0000 7fb77fd70640 1 -- 172.21.15.107:0/3376632218 shutdown_connections 2024-04-25T00:14:29.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.231+0000 7fb77fd70640 1 --2- 172.21.15.107:0/3376632218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb778103340 0x7fb77819f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.231+0000 7fb77fd70640 1 --2- 172.21.15.107:0/3376632218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb778102980 0x7fb778197ff0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.231+0000 7fb77fd70640 1 --2- 172.21.15.107:0/3376632218 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb7600a2b90 0x7fb7600a5050 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.231+0000 7fb77fd70640 1 --2- 172.21.15.107:0/3376632218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb778101790 0x7fb778197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.231+0000 7fb77fd70640 1 -- 172.21.15.107:0/3376632218 >> 172.21.15.107:0/3376632218 conn(0x7fb7780fcd20 msgr2=0x7fb778105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:29.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.231+0000 7fb77fd70640 1 -- 172.21.15.107:0/3376632218 shutdown_connections 2024-04-25T00:14:29.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.231+0000 7fb77fd70640 1 -- 172.21.15.107:0/3376632218 wait complete. 2024-04-25T00:14:29.234 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:14:28 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:14:28.862+0000 7efe7ae05700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:14:29.234 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:14:28 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:14:28.862+0000 7efe7ae05700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:14:29.234 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:14:28 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:14:28.862+0000 7efe7ae05700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:14:29.234 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:14:28 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:14:28.862+0000 7efe7ae05700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:14:29.234 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:14:28 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:14:28.862+0000 7efe7ae05700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:14:29.244 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T00:14:29.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.345+0000 7f2a9d3dc640 1 -- 172.21.15.107:0/4105822396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a98102200 msgr2=0x7f2a981045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.345+0000 7f2a9d3dc640 1 --2- 172.21.15.107:0/4105822396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a98102200 0x7f2a981045f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2a88009900 tx=0x7f2a8802a410 comp rx=0 tx=0).stop 2024-04-25T00:14:29.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.345+0000 7f2a9d3dc640 1 -- 172.21.15.107:0/4105822396 shutdown_connections 2024-04-25T00:14:29.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.345+0000 7f2a9d3dc640 1 --2- 172.21.15.107:0/4105822396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a98107460 0x7f2a981098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.345+0000 7f2a9d3dc640 1 --2- 172.21.15.107:0/4105822396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a98104b30 0x7f2a98106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.345+0000 7f2a9d3dc640 1 --2- 172.21.15.107:0/4105822396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a98102200 0x7f2a981045f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.345+0000 7f2a9d3dc640 1 -- 172.21.15.107:0/4105822396 >> 172.21.15.107:0/4105822396 conn(0x7f2a980fbbf0 msgr2=0x7f2a980fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:29.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.345+0000 7f2a9d3dc640 1 -- 172.21.15.107:0/4105822396 shutdown_connections 2024-04-25T00:14:29.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.345+0000 7f2a9d3dc640 1 -- 172.21.15.107:0/4105822396 wait complete. 2024-04-25T00:14:29.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.346+0000 7f2a9d3dc640 1 Processor -- start 2024-04-25T00:14:29.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.346+0000 7f2a9d3dc640 1 -- start start 2024-04-25T00:14:29.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.346+0000 7f2a9d3dc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a98102200 0x7f2a98197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.346+0000 7f2a9d3dc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a98104b30 0x7f2a981980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.346+0000 7f2a9d3dc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a98107460 0x7f2a9819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.346+0000 7f2a9d3dc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9810d480 con 0x7f2a98107460 2024-04-25T00:14:29.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.346+0000 7f2a9d3dc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9810d300 con 0x7f2a98102200 2024-04-25T00:14:29.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.346+0000 7f2a9d3dc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9810d600 con 0x7f2a98104b30 2024-04-25T00:14:29.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.346+0000 7f2a977fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a98107460 0x7f2a9819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.346+0000 7f2a977fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a98107460 0x7f2a9819f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59320/0 (socket says 172.21.15.107:59320) 2024-04-25T00:14:29.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.346+0000 7f2a977fe640 1 -- 172.21.15.107:0/2824967821 learned_addr learned my addr 172.21.15.107:0/2824967821 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:29.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.346+0000 7f2a96ffd640 1 --2- 172.21.15.107:0/2824967821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a98102200 0x7f2a98197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.346+0000 7f2a967fc640 1 --2- 172.21.15.107:0/2824967821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a98104b30 0x7f2a981980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.346+0000 7f2a977fe640 1 -- 172.21.15.107:0/2824967821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a98104b30 msgr2=0x7f2a981980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.346+0000 7f2a977fe640 1 --2- 172.21.15.107:0/2824967821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a98104b30 0x7f2a981980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.346+0000 7f2a977fe640 1 -- 172.21.15.107:0/2824967821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a98102200 msgr2=0x7f2a98197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.347+0000 7f2a977fe640 1 --2- 172.21.15.107:0/2824967821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a98102200 0x7f2a98197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.347+0000 7f2a977fe640 1 -- 172.21.15.107:0/2824967821 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9819f7b0 con 0x7f2a98107460 2024-04-25T00:14:29.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.347+0000 7f2a967fc640 1 --2- 172.21.15.107:0/2824967821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a98104b30 0x7f2a981980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:29.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.347+0000 7f2a977fe640 1 --2- 172.21.15.107:0/2824967821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a98107460 0x7f2a9819f140 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000eb10 tx=0x7f2a9000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:29.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.347+0000 7f2a7ffff640 1 -- 172.21.15.107:0/2824967821 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90004b20 con 0x7f2a98107460 2024-04-25T00:14:29.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.347+0000 7f2a7ffff640 1 -- 172.21.15.107:0/2824967821 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a90004cc0 con 0x7f2a98107460 2024-04-25T00:14:29.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.347+0000 7f2a9d3dc640 1 -- 172.21.15.107:0/2824967821 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a9819faa0 con 0x7f2a98107460 2024-04-25T00:14:29.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.347+0000 7f2a7ffff640 1 -- 172.21.15.107:0/2824967821 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a90010760 con 0x7f2a98107460 2024-04-25T00:14:29.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.347+0000 7f2a9d3dc640 1 -- 172.21.15.107:0/2824967821 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a981a0020 con 0x7f2a98107460 2024-04-25T00:14:29.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.348+0000 7f2a9d3dc640 1 -- 172.21.15.107:0/2824967821 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a980fd6e0 con 0x7f2a98107460 2024-04-25T00:14:29.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.349+0000 7f2a7ffff640 1 -- 172.21.15.107:0/2824967821 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 137834+0+0 (secure 0 0 0) 0x7f2a90005020 con 0x7f2a98107460 2024-04-25T00:14:29.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.349+0000 7f2a7ffff640 1 --2- 172.21.15.107:0/2824967821 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f2a800a2b90 0x7f2a800a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.349+0000 7f2a96ffd640 1 --2- 172.21.15.107:0/2824967821 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f2a800a2b90 0x7f2a800a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.349+0000 7f2a7ffff640 1 -- 172.21.15.107:0/2824967821 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f2a900cf8e0 con 0x7f2a98107460 2024-04-25T00:14:29.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.350+0000 7f2a96ffd640 1 --2- 172.21.15.107:0/2824967821 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f2a800a2b90 0x7f2a800a5050 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2a880314e0 tx=0x7f2a88000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:29.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.354+0000 7f2a7ffff640 1 -- 172.21.15.107:0/2824967821 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2a9009c600 con 0x7f2a98107460 2024-04-25T00:14:29.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.544+0000 7f2a7ffff640 1 -- 172.21.15.107:0/2824967821 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f2a90098570 con 0x7f2a98107460 2024-04-25T00:14:29.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.562+0000 7f2a9d3dc640 1 -- 172.21.15.107:0/2824967821 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f2a98061b80 con 0x7f2a800a2b90 2024-04-25T00:14:29.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.568+0000 7f2a7ffff640 1 -- 172.21.15.107:0/2824967821 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f2a98061b80 con 0x7f2a800a2b90 2024-04-25T00:14:29.569 INFO:teuthology.orchestra.run.smithi107.stdout:/volumes/qa/sv_0/37effdcc-0d9c-45c9-8f41-164935f0522c 2024-04-25T00:14:29.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.569+0000 7f2a9d3dc640 1 -- 172.21.15.107:0/2824967821 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f2a800a2b90 msgr2=0x7f2a800a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.569+0000 7f2a9d3dc640 1 --2- 172.21.15.107:0/2824967821 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f2a800a2b90 0x7f2a800a5050 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2a880314e0 tx=0x7f2a88000bc0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.569+0000 7f2a9d3dc640 1 -- 172.21.15.107:0/2824967821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a98107460 msgr2=0x7f2a9819f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.569+0000 7f2a9d3dc640 1 --2- 172.21.15.107:0/2824967821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a98107460 0x7f2a9819f140 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2a9000eb10 tx=0x7f2a9000c2d0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.570+0000 7f2a9d3dc640 1 -- 172.21.15.107:0/2824967821 shutdown_connections 2024-04-25T00:14:29.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.570+0000 7f2a9d3dc640 1 --2- 172.21.15.107:0/2824967821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a98107460 0x7f2a9819f140 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.570+0000 7f2a9d3dc640 1 --2- 172.21.15.107:0/2824967821 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2a98104b30 0x7f2a981980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.570+0000 7f2a9d3dc640 1 --2- 172.21.15.107:0/2824967821 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f2a800a2b90 0x7f2a800a5050 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.570+0000 7f2a9d3dc640 1 --2- 172.21.15.107:0/2824967821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a98102200 0x7f2a98197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.570+0000 7f2a9d3dc640 1 -- 172.21.15.107:0/2824967821 >> 172.21.15.107:0/2824967821 conn(0x7f2a980fbbf0 msgr2=0x7f2a98102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:29.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.570+0000 7f2a9d3dc640 1 -- 172.21.15.107:0/2824967821 shutdown_connections 2024-04-25T00:14:29.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.570+0000 7f2a9d3dc640 1 -- 172.21.15.107:0/2824967821 wait complete. 2024-04-25T00:14:29.582 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T00:14:29.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.683+0000 7fa3d969a640 1 -- 172.21.15.107:0/4177058636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3d40ffe10 msgr2=0x7fa3d4100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.683+0000 7fa3d969a640 1 --2- 172.21.15.107:0/4177058636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3d40ffe10 0x7fa3d4100230 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4009900 tx=0x7fa3c402a410 comp rx=0 tx=0).stop 2024-04-25T00:14:29.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.683+0000 7fa3d969a640 1 -- 172.21.15.107:0/4177058636 shutdown_connections 2024-04-25T00:14:29.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.683+0000 7fa3d969a640 1 --2- 172.21.15.107:0/4177058636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3d4101920 0x7fa3d4108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.683+0000 7fa3d969a640 1 --2- 172.21.15.107:0/4177058636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3d4100f60 0x7fa3d41013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.683+0000 7fa3d969a640 1 --2- 172.21.15.107:0/4177058636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3d40ffe10 0x7fa3d4100230 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.683+0000 7fa3d969a640 1 -- 172.21.15.107:0/4177058636 >> 172.21.15.107:0/4177058636 conn(0x7fa3d40fb380 msgr2=0x7fa3d40fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:29.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.683+0000 7fa3d969a640 1 -- 172.21.15.107:0/4177058636 shutdown_connections 2024-04-25T00:14:29.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.683+0000 7fa3d969a640 1 -- 172.21.15.107:0/4177058636 wait complete. 2024-04-25T00:14:29.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.683+0000 7fa3d969a640 1 Processor -- start 2024-04-25T00:14:29.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.683+0000 7fa3d969a640 1 -- start start 2024-04-25T00:14:29.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.684+0000 7fa3d969a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3d40ffe10 0x7fa3d4197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.684+0000 7fa3d969a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3d4100f60 0x7fa3d4198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.684+0000 7fa3d969a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3d4101920 0x7fa3d419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.684+0000 7fa3d969a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d410d4b0 con 0x7fa3d40ffe10 2024-04-25T00:14:29.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.684+0000 7fa3d969a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d410d330 con 0x7fa3d4101920 2024-04-25T00:14:29.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.684+0000 7fa3d969a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa3d410d630 con 0x7fa3d4100f60 2024-04-25T00:14:29.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.684+0000 7fa3d2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3d40ffe10 0x7fa3d4197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.684+0000 7fa3d2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3d40ffe10 0x7fa3d4197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38070/0 (socket says 172.21.15.107:38070) 2024-04-25T00:14:29.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.684+0000 7fa3d2ffd640 1 -- 172.21.15.107:0/891339185 learned_addr learned my addr 172.21.15.107:0/891339185 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:29.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.684+0000 7fa3d27fc640 1 --2- 172.21.15.107:0/891339185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3d4100f60 0x7fa3d4198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.684+0000 7fa3d37fe640 1 --2- 172.21.15.107:0/891339185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3d4101920 0x7fa3d419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.684+0000 7fa3d2ffd640 1 -- 172.21.15.107:0/891339185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3d4100f60 msgr2=0x7fa3d4198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.684+0000 7fa3d2ffd640 1 --2- 172.21.15.107:0/891339185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3d4100f60 0x7fa3d4198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.684+0000 7fa3d2ffd640 1 -- 172.21.15.107:0/891339185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3d4101920 msgr2=0x7fa3d419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.684+0000 7fa3d2ffd640 1 --2- 172.21.15.107:0/891339185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3d4101920 0x7fa3d419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.684+0000 7fa3d2ffd640 1 -- 172.21.15.107:0/891339185 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3d419f770 con 0x7fa3d40ffe10 2024-04-25T00:14:29.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.684+0000 7fa3d37fe640 1 --2- 172.21.15.107:0/891339185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3d4101920 0x7fa3d419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:29.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.684+0000 7fa3d2ffd640 1 --2- 172.21.15.107:0/891339185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3d40ffe10 0x7fa3d4197b50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4009230 tx=0x7fa3c4004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:29.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.684+0000 7fa3bbfff640 1 -- 172.21.15.107:0/891339185 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c403f030 con 0x7fa3d40ffe10 2024-04-25T00:14:29.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.684+0000 7fa3d969a640 1 -- 172.21.15.107:0/891339185 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3d419fa00 con 0x7fa3d40ffe10 2024-04-25T00:14:29.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.684+0000 7fa3bbfff640 1 -- 172.21.15.107:0/891339185 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3c4003ce0 con 0x7fa3d40ffe10 2024-04-25T00:14:29.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.684+0000 7fa3bbfff640 1 -- 172.21.15.107:0/891339185 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c4035070 con 0x7fa3d40ffe10 2024-04-25T00:14:29.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.684+0000 7fa3d969a640 1 -- 172.21.15.107:0/891339185 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3d419ff70 con 0x7fa3d40ffe10 2024-04-25T00:14:29.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.685+0000 7fa3d969a640 1 -- 172.21.15.107:0/891339185 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3d40689b0 con 0x7fa3d40ffe10 2024-04-25T00:14:29.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.686+0000 7fa3bbfff640 1 -- 172.21.15.107:0/891339185 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa3c4042030 con 0x7fa3d40ffe10 2024-04-25T00:14:29.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.687+0000 7fa3bbfff640 1 --2- 172.21.15.107:0/891339185 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa3bc0a2c20 0x7fa3bc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:29.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.687+0000 7fa3d27fc640 1 --2- 172.21.15.107:0/891339185 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa3bc0a2c20 0x7fa3bc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:29.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.687+0000 7fa3bbfff640 1 -- 172.21.15.107:0/891339185 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fa3c40efa70 con 0x7fa3d40ffe10 2024-04-25T00:14:29.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.687+0000 7fa3d27fc640 1 --2- 172.21.15.107:0/891339185 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa3bc0a2c20 0x7fa3bc0a50e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4199060 tx=0x7fa3c8009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:29.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.691+0000 7fa3bbfff640 1 -- 172.21.15.107:0/891339185 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa3c40bc5d0 con 0x7fa3d40ffe10 2024-04-25T00:14:29.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:29 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/355733161' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T00:14:29.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:29 smithi195 ceph-mon[32833]: mds.? [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] up:active 2024-04-25T00:14:29.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:29 smithi195 ceph-mon[32833]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-25T00:14:29.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:29 smithi195 ceph-mon[32833]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:14:29.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:29 smithi195 ceph-mon[32833]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:29.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:29 smithi195 ceph-mon[32833]: pgmap v306: 129 pgs: 129 active+clean; 582 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-25T00:14:29.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.894+0000 7fa3d969a640 1 -- 172.21.15.107:0/891339185 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa3d4061cd0 con 0x7fa3bc0a2c20 2024-04-25T00:14:29.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.917+0000 7fa3bbfff640 1 -- 172.21.15.107:0/891339185 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa3d4061cd0 con 0x7fa3bc0a2c20 2024-04-25T00:14:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.919+0000 7fa3d969a640 1 -- 172.21.15.107:0/891339185 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa3bc0a2c20 msgr2=0x7fa3bc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.919+0000 7fa3d969a640 1 --2- 172.21.15.107:0/891339185 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa3bc0a2c20 0x7fa3bc0a50e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4199060 tx=0x7fa3c8009110 comp rx=0 tx=0).stop 2024-04-25T00:14:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.919+0000 7fa3d969a640 1 -- 172.21.15.107:0/891339185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3d40ffe10 msgr2=0x7fa3d4197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.919+0000 7fa3d969a640 1 --2- 172.21.15.107:0/891339185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3d40ffe10 0x7fa3d4197b50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4009230 tx=0x7fa3c4004360 comp rx=0 tx=0).stop 2024-04-25T00:14:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.919+0000 7fa3d969a640 1 -- 172.21.15.107:0/891339185 shutdown_connections 2024-04-25T00:14:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.919+0000 7fa3d969a640 1 --2- 172.21.15.107:0/891339185 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa3bc0a2c20 0x7fa3bc0a50e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.919+0000 7fa3d969a640 1 --2- 172.21.15.107:0/891339185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3d4101920 0x7fa3d419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.919+0000 7fa3d969a640 1 --2- 172.21.15.107:0/891339185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3d4100f60 0x7fa3d4198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.919+0000 7fa3d969a640 1 --2- 172.21.15.107:0/891339185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3d40ffe10 0x7fa3d4197b50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:29.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.919+0000 7fa3d969a640 1 -- 172.21.15.107:0/891339185 >> 172.21.15.107:0/891339185 conn(0x7fa3d40fb380 msgr2=0x7fa3d41042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:29.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.920+0000 7fa3d969a640 1 -- 172.21.15.107:0/891339185 shutdown_connections 2024-04-25T00:14:29.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:29.920+0000 7fa3d969a640 1 -- 172.21.15.107:0/891339185 wait complete. 2024-04-25T00:14:29.931 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:29 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/355733161' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T00:14:29.931 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:29 smithi107 ceph-mon[29153]: mds.? [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] up:active 2024-04-25T00:14:29.931 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:29 smithi107 ceph-mon[29153]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-25T00:14:29.931 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:29 smithi107 ceph-mon[29153]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:14:29.931 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:29 smithi107 ceph-mon[29153]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:29.931 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:29 smithi107 ceph-mon[29153]: pgmap v306: 129 pgs: 129 active+clean; 582 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-25T00:14:29.932 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T00:14:29.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:29 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/355733161' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T00:14:29.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:29 smithi170 ceph-mon[32807]: mds.? [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] up:active 2024-04-25T00:14:29.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:29 smithi170 ceph-mon[32807]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-25T00:14:29.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:29 smithi170 ceph-mon[32807]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:14:29.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:29 smithi170 ceph-mon[32807]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:29.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:29 smithi170 ceph-mon[32807]: pgmap v306: 129 pgs: 129 active+clean; 582 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-25T00:14:30.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.034+0000 7efe6cd4f640 1 -- 172.21.15.107:0/2303118044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe681006a0 msgr2=0x7efe68100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:30.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.034+0000 7efe6cd4f640 1 --2- 172.21.15.107:0/2303118044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe681006a0 0x7efe68100ac0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efe58009900 tx=0x7efe5802a410 comp rx=0 tx=0).stop 2024-04-25T00:14:30.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.034+0000 7efe6cd4f640 1 -- 172.21.15.107:0/2303118044 shutdown_connections 2024-04-25T00:14:30.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.034+0000 7efe6cd4f640 1 --2- 172.21.15.107:0/2303118044 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe68102250 0x7efe68108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.034+0000 7efe6cd4f640 1 --2- 172.21.15.107:0/2303118044 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe68101890 0x7efe68101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.034+0000 7efe6cd4f640 1 --2- 172.21.15.107:0/2303118044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe681006a0 0x7efe68100ac0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.034+0000 7efe6cd4f640 1 -- 172.21.15.107:0/2303118044 >> 172.21.15.107:0/2303118044 conn(0x7efe680fbc30 msgr2=0x7efe680fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:30.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.034+0000 7efe6cd4f640 1 -- 172.21.15.107:0/2303118044 shutdown_connections 2024-04-25T00:14:30.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.034+0000 7efe6cd4f640 1 -- 172.21.15.107:0/2303118044 wait complete. 2024-04-25T00:14:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.035+0000 7efe6cd4f640 1 Processor -- start 2024-04-25T00:14:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.035+0000 7efe6cd4f640 1 -- start start 2024-04-25T00:14:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.035+0000 7efe6cd4f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe681006a0 0x7efe68197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.035+0000 7efe6cd4f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe68101890 0x7efe68198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.035+0000 7efe6cd4f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe68102250 0x7efe6819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.035+0000 7efe6cd4f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efe6810d440 con 0x7efe68101890 2024-04-25T00:14:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.035+0000 7efe6cd4f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efe6810d2c0 con 0x7efe681006a0 2024-04-25T00:14:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.035+0000 7efe6cd4f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efe6810d5c0 con 0x7efe68102250 2024-04-25T00:14:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.035+0000 7efe65d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe68101890 0x7efe68198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.035+0000 7efe65d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe68101890 0x7efe68198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38086/0 (socket says 172.21.15.107:38086) 2024-04-25T00:14:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.035+0000 7efe65d9b640 1 -- 172.21.15.107:0/2959567842 learned_addr learned my addr 172.21.15.107:0/2959567842 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.035+0000 7efe6659c640 1 --2- 172.21.15.107:0/2959567842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe681006a0 0x7efe68197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.035+0000 7efe66d9d640 1 --2- 172.21.15.107:0/2959567842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe68102250 0x7efe6819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.035+0000 7efe65d9b640 1 -- 172.21.15.107:0/2959567842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe68102250 msgr2=0x7efe6819f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.035+0000 7efe65d9b640 1 --2- 172.21.15.107:0/2959567842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe68102250 0x7efe6819f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.035+0000 7efe65d9b640 1 -- 172.21.15.107:0/2959567842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe681006a0 msgr2=0x7efe68197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.035+0000 7efe65d9b640 1 --2- 172.21.15.107:0/2959567842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe681006a0 0x7efe68197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.035+0000 7efe65d9b640 1 -- 172.21.15.107:0/2959567842 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6819f6f0 con 0x7efe68101890 2024-04-25T00:14:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.035+0000 7efe65d9b640 1 --2- 172.21.15.107:0/2959567842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe68101890 0x7efe68198010 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00d950 tx=0x7efe5c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.036+0000 7efe577fe640 1 -- 172.21.15.107:0/2959567842 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c017070 con 0x7efe68101890 2024-04-25T00:14:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.036+0000 7efe6cd4f640 1 -- 172.21.15.107:0/2959567842 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe6819f9e0 con 0x7efe68101890 2024-04-25T00:14:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.036+0000 7efe577fe640 1 -- 172.21.15.107:0/2959567842 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe5c009c30 con 0x7efe68101890 2024-04-25T00:14:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.036+0000 7efe577fe640 1 -- 172.21.15.107:0/2959567842 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c00f6b0 con 0x7efe68101890 2024-04-25T00:14:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.036+0000 7efe6cd4f640 1 -- 172.21.15.107:0/2959567842 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe6819ff80 con 0x7efe68101890 2024-04-25T00:14:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.037+0000 7efe6cd4f640 1 -- 172.21.15.107:0/2959567842 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe68107a40 con 0x7efe68101890 2024-04-25T00:14:30.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.038+0000 7efe577fe640 1 -- 172.21.15.107:0/2959567842 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7efe5c0109d0 con 0x7efe68101890 2024-04-25T00:14:30.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.038+0000 7efe577fe640 1 --2- 172.21.15.107:0/2959567842 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7efe500a2c20 0x7efe500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:30.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.038+0000 7efe6659c640 1 --2- 172.21.15.107:0/2959567842 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7efe500a2c20 0x7efe500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:30.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.039+0000 7efe577fe640 1 -- 172.21.15.107:0/2959567842 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7efe5c0cf0a0 con 0x7efe68101890 2024-04-25T00:14:30.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.039+0000 7efe6659c640 1 --2- 172.21.15.107:0/2959567842 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7efe500a2c20 0x7efe500a50e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efe58005ab0 tx=0x7efe58000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:30.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.043+0000 7efe577fe640 1 -- 172.21.15.107:0/2959567842 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efe5c009de0 con 0x7efe68101890 2024-04-25T00:14:30.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.245+0000 7efe6cd4f640 1 -- 172.21.15.107:0/2959567842 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7efe68061c10 con 0x7efe500a2c20 2024-04-25T00:14:30.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.250+0000 7efe577fe640 1 -- 172.21.15.107:0/2959567842 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7efe68061c10 con 0x7efe500a2c20 2024-04-25T00:14:30.251 INFO:teuthology.orchestra.run.smithi107.stdout:/volumes/qa/sv_1/026a421b-09bd-4435-ad20-5fa0f0e1edd5 2024-04-25T00:14:30.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.252+0000 7efe6cd4f640 1 -- 172.21.15.107:0/2959567842 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7efe500a2c20 msgr2=0x7efe500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:30.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.252+0000 7efe6cd4f640 1 --2- 172.21.15.107:0/2959567842 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7efe500a2c20 0x7efe500a50e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efe58005ab0 tx=0x7efe58000bc0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.252+0000 7efe6cd4f640 1 -- 172.21.15.107:0/2959567842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe68101890 msgr2=0x7efe68198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:30.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.252+0000 7efe6cd4f640 1 --2- 172.21.15.107:0/2959567842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe68101890 0x7efe68198010 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00d950 tx=0x7efe5c00de10 comp rx=0 tx=0).stop 2024-04-25T00:14:30.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.252+0000 7efe6cd4f640 1 -- 172.21.15.107:0/2959567842 shutdown_connections 2024-04-25T00:14:30.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.252+0000 7efe6cd4f640 1 --2- 172.21.15.107:0/2959567842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe68102250 0x7efe6819f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.252+0000 7efe6cd4f640 1 --2- 172.21.15.107:0/2959567842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe68101890 0x7efe68198010 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.252+0000 7efe6cd4f640 1 --2- 172.21.15.107:0/2959567842 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7efe500a2c20 0x7efe500a50e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.252+0000 7efe6cd4f640 1 --2- 172.21.15.107:0/2959567842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe681006a0 0x7efe68197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.252+0000 7efe6cd4f640 1 -- 172.21.15.107:0/2959567842 >> 172.21.15.107:0/2959567842 conn(0x7efe680fbc30 msgr2=0x7efe68104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:30.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.252+0000 7efe6cd4f640 1 -- 172.21.15.107:0/2959567842 shutdown_connections 2024-04-25T00:14:30.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.252+0000 7efe6cd4f640 1 -- 172.21.15.107:0/2959567842 wait complete. 2024-04-25T00:14:30.264 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:14:30.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.361+0000 7fe12f6f5640 1 -- 172.21.15.107:0/3851477493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1280ffff0 msgr2=0x7fe128102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:30.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.361+0000 7fe12f6f5640 1 --2- 172.21.15.107:0/3851477493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1280ffff0 0x7fe128102420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe124009900 tx=0x7fe12402a410 comp rx=0 tx=0).stop 2024-04-25T00:14:30.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.361+0000 7fe12f6f5640 1 -- 172.21.15.107:0/3851477493 shutdown_connections 2024-04-25T00:14:30.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.361+0000 7fe12f6f5640 1 --2- 172.21.15.107:0/3851477493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe128105310 0x7fe128105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.361+0000 7fe12f6f5640 1 --2- 172.21.15.107:0/3851477493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe128102960 0x7fe128104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.361+0000 7fe12f6f5640 1 --2- 172.21.15.107:0/3851477493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1280ffff0 0x7fe128102420 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.361+0000 7fe12f6f5640 1 -- 172.21.15.107:0/3851477493 >> 172.21.15.107:0/3851477493 conn(0x7fe1280fbbf0 msgr2=0x7fe1280fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:30.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.361+0000 7fe12f6f5640 1 -- 172.21.15.107:0/3851477493 shutdown_connections 2024-04-25T00:14:30.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.361+0000 7fe12f6f5640 1 -- 172.21.15.107:0/3851477493 wait complete. 2024-04-25T00:14:30.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.361+0000 7fe12f6f5640 1 Processor -- start 2024-04-25T00:14:30.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.362+0000 7fe12f6f5640 1 -- start start 2024-04-25T00:14:30.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.362+0000 7fe12f6f5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1280ffff0 0x7fe12819e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:30.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.362+0000 7fe12f6f5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe128102960 0x7fe12819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:30.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.362+0000 7fe12f6f5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe128105310 0x7fe1281a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:30.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.362+0000 7fe12f6f5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe1281139f0 con 0x7fe128105310 2024-04-25T00:14:30.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.362+0000 7fe12f6f5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe128113870 con 0x7fe128102960 2024-04-25T00:14:30.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.362+0000 7fe12f6f5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe128113b70 con 0x7fe1280ffff0 2024-04-25T00:14:30.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.362+0000 7fe12dc92640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe128105310 0x7fe1281a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:30.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.362+0000 7fe12dc92640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe128105310 0x7fe1281a5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38114/0 (socket says 172.21.15.107:38114) 2024-04-25T00:14:30.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.362+0000 7fe12dc92640 1 -- 172.21.15.107:0/3631618189 learned_addr learned my addr 172.21.15.107:0/3631618189 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:30.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.362+0000 7fe12cc90640 1 --2- 172.21.15.107:0/3631618189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe128102960 0x7fe12819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:30.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.362+0000 7fe12d491640 1 --2- 172.21.15.107:0/3631618189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1280ffff0 0x7fe12819e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:30.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.362+0000 7fe12dc92640 1 -- 172.21.15.107:0/3631618189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1280ffff0 msgr2=0x7fe12819e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:30.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.362+0000 7fe12dc92640 1 --2- 172.21.15.107:0/3631618189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1280ffff0 0x7fe12819e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.362+0000 7fe12dc92640 1 -- 172.21.15.107:0/3631618189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe128102960 msgr2=0x7fe12819e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:30.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.362+0000 7fe12dc92640 1 --2- 172.21.15.107:0/3631618189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe128102960 0x7fe12819e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.362+0000 7fe12dc92640 1 -- 172.21.15.107:0/3631618189 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1281a5cd0 con 0x7fe128105310 2024-04-25T00:14:30.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.362+0000 7fe12dc92640 1 --2- 172.21.15.107:0/3631618189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe128105310 0x7fe1281a5660 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe12000eaa0 tx=0x7fe12000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:30.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.362+0000 7fe11e7fc640 1 -- 172.21.15.107:0/3631618189 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12000cdb0 con 0x7fe128105310 2024-04-25T00:14:30.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.362+0000 7fe11e7fc640 1 -- 172.21.15.107:0/3631618189 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe120005020 con 0x7fe128105310 2024-04-25T00:14:30.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.362+0000 7fe12f6f5640 1 -- 172.21.15.107:0/3631618189 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1281a5fc0 con 0x7fe128105310 2024-04-25T00:14:30.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.362+0000 7fe11e7fc640 1 -- 172.21.15.107:0/3631618189 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1200116b0 con 0x7fe128105310 2024-04-25T00:14:30.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.363+0000 7fe12f6f5640 1 -- 172.21.15.107:0/3631618189 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1281a6430 con 0x7fe128105310 2024-04-25T00:14:30.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.364+0000 7fe12f6f5640 1 -- 172.21.15.107:0/3631618189 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1280fd6e0 con 0x7fe128105310 2024-04-25T00:14:30.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.364+0000 7fe11e7fc640 1 -- 172.21.15.107:0/3631618189 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe120005560 con 0x7fe128105310 2024-04-25T00:14:30.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.365+0000 7fe11e7fc640 1 --2- 172.21.15.107:0/3631618189 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe1140a2c20 0x7fe1140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:30.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.365+0000 7fe12d491640 1 --2- 172.21.15.107:0/3631618189 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe1140a2c20 0x7fe1140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:30.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.365+0000 7fe11e7fc640 1 -- 172.21.15.107:0/3631618189 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fe1200cf660 con 0x7fe128105310 2024-04-25T00:14:30.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.366+0000 7fe12d491640 1 --2- 172.21.15.107:0/3631618189 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe1140a2c20 0x7fe1140a50e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe124000c00 tx=0x7fe1240315d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:30.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.370+0000 7fe11e7fc640 1 -- 172.21.15.107:0/3631618189 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe120016030 con 0x7fe128105310 2024-04-25T00:14:30.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.583+0000 7fe12f6f5640 1 -- 172.21.15.107:0/3631618189 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe12802d170 con 0x7fe128105310 2024-04-25T00:14:30.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.584+0000 7fe11e7fc640 1 -- 172.21.15.107:0/3631618189 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11897 (secure 0 0 0) 0x7fe12009c220 con 0x7fe128105310 2024-04-25T00:14:30.586 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:14:30.586 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:14:28.529865+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24521,24493]},"id":1}]} 2024-04-25T00:14:30.586 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 23 2024-04-25T00:14:30.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.586+0000 7fe12f6f5640 1 -- 172.21.15.107:0/3631618189 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe1140a2c20 msgr2=0x7fe1140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:30.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.586+0000 7fe12f6f5640 1 --2- 172.21.15.107:0/3631618189 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe1140a2c20 0x7fe1140a50e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe124000c00 tx=0x7fe1240315d0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.586+0000 7fe12f6f5640 1 -- 172.21.15.107:0/3631618189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe128105310 msgr2=0x7fe1281a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:30.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.586+0000 7fe12f6f5640 1 --2- 172.21.15.107:0/3631618189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe128105310 0x7fe1281a5660 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe12000eaa0 tx=0x7fe12000ef60 comp rx=0 tx=0).stop 2024-04-25T00:14:30.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.586+0000 7fe12f6f5640 1 -- 172.21.15.107:0/3631618189 shutdown_connections 2024-04-25T00:14:30.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.586+0000 7fe12f6f5640 1 --2- 172.21.15.107:0/3631618189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe128105310 0x7fe1281a5660 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.586+0000 7fe12f6f5640 1 --2- 172.21.15.107:0/3631618189 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe1140a2c20 0x7fe1140a50e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.586+0000 7fe12f6f5640 1 --2- 172.21.15.107:0/3631618189 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe128102960 0x7fe12819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.586+0000 7fe12f6f5640 1 --2- 172.21.15.107:0/3631618189 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe1280ffff0 0x7fe12819e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.586+0000 7fe12f6f5640 1 -- 172.21.15.107:0/3631618189 >> 172.21.15.107:0/3631618189 conn(0x7fe1280fbbf0 msgr2=0x7fe128103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:30.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.586+0000 7fe12f6f5640 1 -- 172.21.15.107:0/3631618189 shutdown_connections 2024-04-25T00:14:30.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.586+0000 7fe12f6f5640 1 -- 172.21.15.107:0/3631618189 wait complete. 2024-04-25T00:14:30.599 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:14:30.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.695+0000 7fe939db0640 1 -- 172.21.15.107:0/605908887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9341018b0 msgr2=0x7fe934101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:30.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.695+0000 7fe939db0640 1 --2- 172.21.15.107:0/605908887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9341018b0 0x7fe934101d30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe928009900 tx=0x7fe92802a410 comp rx=0 tx=0).stop 2024-04-25T00:14:30.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.695+0000 7fe939db0640 1 -- 172.21.15.107:0/605908887 shutdown_connections 2024-04-25T00:14:30.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.695+0000 7fe939db0640 1 --2- 172.21.15.107:0/605908887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe934102270 0x7fe934108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.695+0000 7fe939db0640 1 --2- 172.21.15.107:0/605908887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9341018b0 0x7fe934101d30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.695+0000 7fe939db0640 1 --2- 172.21.15.107:0/605908887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9341006c0 0x7fe934100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.695+0000 7fe939db0640 1 -- 172.21.15.107:0/605908887 >> 172.21.15.107:0/605908887 conn(0x7fe9340fbc30 msgr2=0x7fe9340fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:30.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.695+0000 7fe939db0640 1 -- 172.21.15.107:0/605908887 shutdown_connections 2024-04-25T00:14:30.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.695+0000 7fe939db0640 1 -- 172.21.15.107:0/605908887 wait complete. 2024-04-25T00:14:30.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.695+0000 7fe939db0640 1 Processor -- start 2024-04-25T00:14:30.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.695+0000 7fe939db0640 1 -- start start 2024-04-25T00:14:30.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.696+0000 7fe939db0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9341006c0 0x7fe9341046f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:30.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.696+0000 7fe939db0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9341018b0 0x7fe934102d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:30.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.696+0000 7fe939db0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe934102270 0x7fe9341032c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:30.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.696+0000 7fe939db0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe93410c3a0 con 0x7fe934102270 2024-04-25T00:14:30.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.696+0000 7fe939db0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe93410c220 con 0x7fe9341006c0 2024-04-25T00:14:30.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.696+0000 7fe939db0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe93410c520 con 0x7fe9341018b0 2024-04-25T00:14:30.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.696+0000 7fe933fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe934102270 0x7fe9341032c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:30.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.696+0000 7fe933fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe934102270 0x7fe9341032c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38124/0 (socket says 172.21.15.107:38124) 2024-04-25T00:14:30.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.696+0000 7fe933fff640 1 -- 172.21.15.107:0/2108874213 learned_addr learned my addr 172.21.15.107:0/2108874213 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:30.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.696+0000 7fe9337fe640 1 --2- 172.21.15.107:0/2108874213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9341006c0 0x7fe9341046f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:30.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.696+0000 7fe932ffd640 1 --2- 172.21.15.107:0/2108874213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9341018b0 0x7fe934102d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:30.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.696+0000 7fe933fff640 1 -- 172.21.15.107:0/2108874213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9341018b0 msgr2=0x7fe934102d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:30.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.696+0000 7fe933fff640 1 --2- 172.21.15.107:0/2108874213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9341018b0 0x7fe934102d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.696+0000 7fe933fff640 1 -- 172.21.15.107:0/2108874213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9341006c0 msgr2=0x7fe9341046f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:30.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.696+0000 7fe933fff640 1 --2- 172.21.15.107:0/2108874213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9341006c0 0x7fe9341046f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.696+0000 7fe933fff640 1 -- 172.21.15.107:0/2108874213 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe934103b80 con 0x7fe934102270 2024-04-25T00:14:30.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.696+0000 7fe933fff640 1 --2- 172.21.15.107:0/2108874213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe934102270 0x7fe9341032c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00b970 tx=0x7fe92c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:30.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.696+0000 7fe930ff9640 1 -- 172.21.15.107:0/2108874213 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c019070 con 0x7fe934102270 2024-04-25T00:14:30.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.696+0000 7fe930ff9640 1 -- 172.21.15.107:0/2108874213 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe92c00cb40 con 0x7fe934102270 2024-04-25T00:14:30.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.696+0000 7fe939db0640 1 -- 172.21.15.107:0/2108874213 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9341a1d30 con 0x7fe934102270 2024-04-25T00:14:30.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.696+0000 7fe930ff9640 1 -- 172.21.15.107:0/2108874213 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c007650 con 0x7fe934102270 2024-04-25T00:14:30.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.696+0000 7fe939db0640 1 -- 172.21.15.107:0/2108874213 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9341a2240 con 0x7fe934102270 2024-04-25T00:14:30.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.697+0000 7fe939db0640 1 -- 172.21.15.107:0/2108874213 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9341a24d0 con 0x7fe934102270 2024-04-25T00:14:30.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.698+0000 7fe930ff9640 1 -- 172.21.15.107:0/2108874213 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe92c01c070 con 0x7fe934102270 2024-04-25T00:14:30.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.699+0000 7fe930ff9640 1 --2- 172.21.15.107:0/2108874213 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe91c0a2ca0 0x7fe91c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:30.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.699+0000 7fe9337fe640 1 --2- 172.21.15.107:0/2108874213 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe91c0a2ca0 0x7fe91c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:30.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.699+0000 7fe930ff9640 1 -- 172.21.15.107:0/2108874213 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fe92c0ce860 con 0x7fe934102270 2024-04-25T00:14:30.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.699+0000 7fe9337fe640 1 --2- 172.21.15.107:0/2108874213 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe91c0a2ca0 0x7fe91c0a5160 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe924005d00 tx=0x7fe924005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:30.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.704+0000 7fe930ff9640 1 -- 172.21.15.107:0/2108874213 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe92c09b530 con 0x7fe934102270 2024-04-25T00:14:30.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:30 smithi195 ceph-mon[32833]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:30.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:30 smithi195 ceph-mon[32833]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T00:14:30.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:30 smithi195 ceph-mon[32833]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:30.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:30 smithi195 ceph-mon[32833]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:30.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.886+0000 7fe939db0640 1 -- 172.21.15.107:0/2108874213 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9341a2830 con 0x7fe934102270 2024-04-25T00:14:30.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.887+0000 7fe930ff9640 1 -- 172.21.15.107:0/2108874213 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24134 (secure 0 0 0) 0x7fe92c09ae30 con 0x7fe934102270 2024-04-25T00:14:30.889 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:14:30.889 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":92,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:14:08.839737+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:59.672801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:00.957782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:14:05.065846+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:30.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.889+0000 7fe939db0640 1 -- 172.21.15.107:0/2108874213 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe91c0a2ca0 msgr2=0x7fe91c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:30.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.889+0000 7fe939db0640 1 --2- 172.21.15.107:0/2108874213 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe91c0a2ca0 0x7fe91c0a5160 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe924005d00 tx=0x7fe924005c50 comp rx=0 tx=0).stop 2024-04-25T00:14:30.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.889+0000 7fe939db0640 1 -- 172.21.15.107:0/2108874213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe934102270 msgr2=0x7fe9341032c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:30.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.889+0000 7fe939db0640 1 --2- 172.21.15.107:0/2108874213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe934102270 0x7fe9341032c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00b970 tx=0x7fe92c00be30 comp rx=0 tx=0).stop 2024-04-25T00:14:30.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.889+0000 7fe939db0640 1 -- 172.21.15.107:0/2108874213 shutdown_connections 2024-04-25T00:14:30.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.890+0000 7fe939db0640 1 --2- 172.21.15.107:0/2108874213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe934102270 0x7fe9341032c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.890+0000 7fe939db0640 1 --2- 172.21.15.107:0/2108874213 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe9341018b0 0x7fe934102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.890+0000 7fe939db0640 1 --2- 172.21.15.107:0/2108874213 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe91c0a2ca0 0x7fe91c0a5160 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.890+0000 7fe939db0640 1 --2- 172.21.15.107:0/2108874213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe9341006c0 0x7fe9341046f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:30.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.890+0000 7fe939db0640 1 -- 172.21.15.107:0/2108874213 >> 172.21.15.107:0/2108874213 conn(0x7fe9340fbc30 msgr2=0x7fe934069a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:30.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.890+0000 7fe939db0640 1 -- 172.21.15.107:0/2108874213 shutdown_connections 2024-04-25T00:14:30.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:30.890+0000 7fe939db0640 1 -- 172.21.15.107:0/2108874213 wait complete. 2024-04-25T00:14:30.901 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:30 smithi107 ceph-mon[29153]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:30.901 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:30 smithi107 ceph-mon[29153]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T00:14:30.902 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:30 smithi107 ceph-mon[29153]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:30.902 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:30 smithi107 ceph-mon[29153]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:30.904 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:14:30.904 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:14:30.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:30 smithi170 ceph-mon[32807]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:30.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:30 smithi170 ceph-mon[32807]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T00:14:30.936 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:30 smithi170 ceph-mon[32807]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:30.936 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:30 smithi170 ceph-mon[32807]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:31.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.005+0000 7f4977bd5640 1 -- 172.21.15.107:0/2544260263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49701006a0 msgr2=0x7f4970100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:31.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.005+0000 7f4977bd5640 1 --2- 172.21.15.107:0/2544260263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49701006a0 0x7f4970100ac0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f496c009900 tx=0x7f496c02a410 comp rx=0 tx=0).stop 2024-04-25T00:14:31.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.005+0000 7f4977bd5640 1 -- 172.21.15.107:0/2544260263 shutdown_connections 2024-04-25T00:14:31.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.005+0000 7f4977bd5640 1 --2- 172.21.15.107:0/2544260263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4970102250 0x7f4970108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.005+0000 7f4977bd5640 1 --2- 172.21.15.107:0/2544260263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4970101890 0x7f4970101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.005+0000 7f4977bd5640 1 --2- 172.21.15.107:0/2544260263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49701006a0 0x7f4970100ac0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.005+0000 7f4977bd5640 1 -- 172.21.15.107:0/2544260263 >> 172.21.15.107:0/2544260263 conn(0x7f49700fbc50 msgr2=0x7f49700fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:31.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.005+0000 7f4977bd5640 1 -- 172.21.15.107:0/2544260263 shutdown_connections 2024-04-25T00:14:31.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.005+0000 7f4977bd5640 1 -- 172.21.15.107:0/2544260263 wait complete. 2024-04-25T00:14:31.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.005+0000 7f4977bd5640 1 Processor -- start 2024-04-25T00:14:31.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.006+0000 7f4977bd5640 1 -- start start 2024-04-25T00:14:31.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.006+0000 7f4977bd5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49701006a0 0x7f497006a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:31.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.006+0000 7f4977bd5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4970101890 0x7f497019c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:31.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.006+0000 7f4977bd5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4970102250 0x7f497019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:31.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.006+0000 7f4977bd5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f497010c320 con 0x7f4970102250 2024-04-25T00:14:31.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.006+0000 7f4977bd5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f497010c1a0 con 0x7f4970101890 2024-04-25T00:14:31.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.006+0000 7f4977bd5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f497010c4a0 con 0x7f49701006a0 2024-04-25T00:14:31.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.006+0000 7f4976172640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4970102250 0x7f497019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:31.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.006+0000 7f4976172640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4970102250 0x7f497019ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38148/0 (socket says 172.21.15.107:38148) 2024-04-25T00:14:31.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.006+0000 7f4976172640 1 -- 172.21.15.107:0/3232647966 learned_addr learned my addr 172.21.15.107:0/3232647966 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:31.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.006+0000 7f4975971640 1 --2- 172.21.15.107:0/3232647966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49701006a0 0x7f497006a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:31.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.006+0000 7f4975170640 1 --2- 172.21.15.107:0/3232647966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4970101890 0x7f497019c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:31.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.006+0000 7f4976172640 1 -- 172.21.15.107:0/3232647966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49701006a0 msgr2=0x7f497006a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:31.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.006+0000 7f4976172640 1 --2- 172.21.15.107:0/3232647966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49701006a0 0x7f497006a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.006+0000 7f4976172640 1 -- 172.21.15.107:0/3232647966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4970101890 msgr2=0x7f497019c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:31.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.006+0000 7f4976172640 1 --2- 172.21.15.107:0/3232647966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4970101890 0x7f497019c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.006+0000 7f4976172640 1 -- 172.21.15.107:0/3232647966 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497019f710 con 0x7f4970102250 2024-04-25T00:14:31.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.006+0000 7f4976172640 1 --2- 172.21.15.107:0/3232647966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4970102250 0x7f497019ee50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f496800cad0 tx=0x7f496800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:31.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.006+0000 7f4966ffd640 1 -- 172.21.15.107:0/3232647966 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496800ae20 con 0x7f4970102250 2024-04-25T00:14:31.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.006+0000 7f4966ffd640 1 -- 172.21.15.107:0/3232647966 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4968005020 con 0x7f4970102250 2024-04-25T00:14:31.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.006+0000 7f4977bd5640 1 -- 172.21.15.107:0/3232647966 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f497019fa00 con 0x7f4970102250 2024-04-25T00:14:31.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.006+0000 7f4966ffd640 1 -- 172.21.15.107:0/3232647966 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496800f6f0 con 0x7f4970102250 2024-04-25T00:14:31.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.006+0000 7f4977bd5640 1 -- 172.21.15.107:0/3232647966 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f497019ff00 con 0x7f4970102250 2024-04-25T00:14:31.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.007+0000 7f4977bd5640 1 -- 172.21.15.107:0/3232647966 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49701079c0 con 0x7f4970102250 2024-04-25T00:14:31.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.008+0000 7f4966ffd640 1 -- 172.21.15.107:0/3232647966 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f49680051d0 con 0x7f4970102250 2024-04-25T00:14:31.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.009+0000 7f4966ffd640 1 --2- 172.21.15.107:0/3232647966 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f495c0a2c20 0x7f495c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:31.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.009+0000 7f4975971640 1 --2- 172.21.15.107:0/3232647966 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f495c0a2c20 0x7f495c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:31.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.009+0000 7f4966ffd640 1 -- 172.21.15.107:0/3232647966 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f49680cf810 con 0x7f4970102250 2024-04-25T00:14:31.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.009+0000 7f4975971640 1 --2- 172.21.15.107:0/3232647966 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f495c0a2c20 0x7f495c0a50e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f496c005ab0 tx=0x7f496c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:31.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.013+0000 7f4966ffd640 1 -- 172.21.15.107:0/3232647966 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4968014030 con 0x7f4970102250 2024-04-25T00:14:31.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.184+0000 7f4977bd5640 1 -- 172.21.15.107:0/3232647966 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4970061b80 con 0x7f495c0a2c20 2024-04-25T00:14:31.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.194+0000 7f4966ffd640 1 -- 172.21.15.107:0/3232647966 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340310 (secure 0 0 0) 0x7f4970061b80 con 0x7f495c0a2c20 2024-04-25T00:14:31.197 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:14:31.203 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-04-25T00:14:31.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.198+0000 7f4977bd5640 1 -- 172.21.15.107:0/3232647966 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f495c0a2c20 msgr2=0x7f495c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:31.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.198+0000 7f4977bd5640 1 --2- 172.21.15.107:0/3232647966 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f495c0a2c20 0x7f495c0a50e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f496c005ab0 tx=0x7f496c000bc0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.198+0000 7f4977bd5640 1 -- 172.21.15.107:0/3232647966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4970102250 msgr2=0x7f497019ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:31.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.198+0000 7f4977bd5640 1 --2- 172.21.15.107:0/3232647966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4970102250 0x7f497019ee50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f496800cad0 tx=0x7f496800cf90 comp rx=0 tx=0).stop 2024-04-25T00:14:31.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.198+0000 7f4977bd5640 1 -- 172.21.15.107:0/3232647966 shutdown_connections 2024-04-25T00:14:31.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.198+0000 7f4977bd5640 1 --2- 172.21.15.107:0/3232647966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4970102250 0x7f497019ee50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.198+0000 7f4977bd5640 1 --2- 172.21.15.107:0/3232647966 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f495c0a2c20 0x7f495c0a50e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.198+0000 7f4977bd5640 1 --2- 172.21.15.107:0/3232647966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4970101890 0x7f497019c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.198+0000 7f4977bd5640 1 --2- 172.21.15.107:0/3232647966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49701006a0 0x7f497006a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.198+0000 7f4977bd5640 1 -- 172.21.15.107:0/3232647966 >> 172.21.15.107:0/3232647966 conn(0x7f49700fbc50 msgr2=0x7f4970104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:31.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.198+0000 7f4977bd5640 1 -- 172.21.15.107:0/3232647966 shutdown_connections 2024-04-25T00:14:31.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.198+0000 7f4977bd5640 1 -- 172.21.15.107:0/3232647966 wait complete. 2024-04-25T00:14:31.210 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-04-25T00:14:30.925042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":634449,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":161,"num_read_kb":162,"num_write":323,"num_write_kb":3022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":262,"ondisk_log_size":262,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388120,"kb_used_data":14000,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553704,"statfs":{"total":1151940427776,"available":1151542992896,"internally_reserved":0,"allocated":14336000,"data_stored":7854759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":39582,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":25,"num_read_kb":43,"num_write":95,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003400"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307882+0000","last_change":"2024-04-25T00:14:01.835737+0000","last_active":"2024-04-25T00:14:09.307882+0000","last_peered":"2024-04-25T00:14:09.307882+0000","last_clean":"2024-04-25T00:14:09.307882+0000","last_became_active":"2024-04-25T00:14:01.835428+0000","last_became_peered":"2024-04-25T00:14:01.835428+0000","last_unstale":"2024-04-25T00:14:09.307882+0000","last_undegraded":"2024-04-25T00:14:09.307882+0000","last_fullsized":"2024-04-25T00:14:09.307882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:33.974656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306530+0000","last_change":"2024-04-25T00:14:01.837321+0000","last_active":"2024-04-25T00:14:09.306530+0000","last_peered":"2024-04-25T00:14:09.306530+0000","last_clean":"2024-04-25T00:14:09.306530+0000","last_became_active":"2024-04-25T00:14:01.836762+0000","last_became_peered":"2024-04-25T00:14:01.836762+0000","last_unstale":"2024-04-25T00:14:09.306530+0000","last_undegraded":"2024-04-25T00:14:09.306530+0000","last_fullsized":"2024-04-25T00:14:09.306530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:34:17.342967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305939+0000","last_change":"2024-04-25T00:14:01.834504+0000","last_active":"2024-04-25T00:14:09.305939+0000","last_peered":"2024-04-25T00:14:09.305939+0000","last_clean":"2024-04-25T00:14:09.305939+0000","last_became_active":"2024-04-25T00:14:01.834182+0000","last_became_peered":"2024-04-25T00:14:01.834182+0000","last_unstale":"2024-04-25T00:14:09.305939+0000","last_undegraded":"2024-04-25T00:14:09.305939+0000","last_fullsized":"2024-04-25T00:14:09.305939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:12.991170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306554+0000","last_change":"2024-04-25T00:14:01.838257+0000","last_active":"2024-04-25T00:14:09.306554+0000","last_peered":"2024-04-25T00:14:09.306554+0000","last_clean":"2024-04-25T00:14:09.306554+0000","last_became_active":"2024-04-25T00:14:01.837909+0000","last_became_peered":"2024-04-25T00:14:01.837909+0000","last_unstale":"2024-04-25T00:14:09.306554+0000","last_undegraded":"2024-04-25T00:14:09.306554+0000","last_fullsized":"2024-04-25T00:14:09.306554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:14.287154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843302+0000","last_change":"2024-04-25T00:14:01.834173+0000","last_active":"2024-04-25T00:14:08.843302+0000","last_peered":"2024-04-25T00:14:08.843302+0000","last_clean":"2024-04-25T00:14:08.843302+0000","last_became_active":"2024-04-25T00:14:01.833835+0000","last_became_peered":"2024-04-25T00:14:01.833835+0000","last_unstale":"2024-04-25T00:14:08.843302+0000","last_undegraded":"2024-04-25T00:14:08.843302+0000","last_fullsized":"2024-04-25T00:14:08.843302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:30.046629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843535+0000","last_change":"2024-04-25T00:14:01.832113+0000","last_active":"2024-04-25T00:14:08.843535+0000","last_peered":"2024-04-25T00:14:08.843535+0000","last_clean":"2024-04-25T00:14:08.843535+0000","last_became_active":"2024-04-25T00:14:01.831800+0000","last_became_peered":"2024-04-25T00:14:01.831800+0000","last_unstale":"2024-04-25T00:14:08.843535+0000","last_undegraded":"2024-04-25T00:14:08.843535+0000","last_fullsized":"2024-04-25T00:14:08.843535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:54.469994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305996+0000","last_change":"2024-04-25T00:14:01.839166+0000","last_active":"2024-04-25T00:14:09.305996+0000","last_peered":"2024-04-25T00:14:09.305996+0000","last_clean":"2024-04-25T00:14:09.305996+0000","last_became_active":"2024-04-25T00:14:01.838851+0000","last_became_peered":"2024-04-25T00:14:01.838851+0000","last_unstale":"2024-04-25T00:14:09.305996+0000","last_undegraded":"2024-04-25T00:14:09.305996+0000","last_fullsized":"2024-04-25T00:14:09.305996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:39:48.125363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305229+0000","last_change":"2024-04-25T00:14:01.828230+0000","last_active":"2024-04-25T00:14:09.305229+0000","last_peered":"2024-04-25T00:14:09.305229+0000","last_clean":"2024-04-25T00:14:09.305229+0000","last_became_active":"2024-04-25T00:14:01.827847+0000","last_became_peered":"2024-04-25T00:14:01.827847+0000","last_unstale":"2024-04-25T00:14:09.305229+0000","last_undegraded":"2024-04-25T00:14:09.305229+0000","last_fullsized":"2024-04-25T00:14:09.305229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:33.906668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307268+0000","last_change":"2024-04-25T00:14:01.829803+0000","last_active":"2024-04-25T00:14:09.307268+0000","last_peered":"2024-04-25T00:14:09.307268+0000","last_clean":"2024-04-25T00:14:09.307268+0000","last_became_active":"2024-04-25T00:14:01.827761+0000","last_became_peered":"2024-04-25T00:14:01.827761+0000","last_unstale":"2024-04-25T00:14:09.307268+0000","last_undegraded":"2024-04-25T00:14:09.307268+0000","last_fullsized":"2024-04-25T00:14:09.307268+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:38.862505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843989+0000","last_change":"2024-04-25T00:14:01.834269+0000","last_active":"2024-04-25T00:14:08.843989+0000","last_peered":"2024-04-25T00:14:08.843989+0000","last_clean":"2024-04-25T00:14:08.843989+0000","last_became_active":"2024-04-25T00:14:01.833964+0000","last_became_peered":"2024-04-25T00:14:01.833964+0000","last_unstale":"2024-04-25T00:14:08.843989+0000","last_undegraded":"2024-04-25T00:14:08.843989+0000","last_fullsized":"2024-04-25T00:14:08.843989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:29.501513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307776+0000","last_change":"2024-04-25T00:14:01.839510+0000","last_active":"2024-04-25T00:14:09.307776+0000","last_peered":"2024-04-25T00:14:09.307776+0000","last_clean":"2024-04-25T00:14:09.307776+0000","last_became_active":"2024-04-25T00:14:01.839000+0000","last_became_peered":"2024-04-25T00:14:01.839000+0000","last_unstale":"2024-04-25T00:14:09.307776+0000","last_undegraded":"2024-04-25T00:14:09.307776+0000","last_fullsized":"2024-04-25T00:14:09.307776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:20.809277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306399+0000","last_change":"2024-04-25T00:14:01.835821+0000","last_active":"2024-04-25T00:14:09.306399+0000","last_peered":"2024-04-25T00:14:09.306399+0000","last_clean":"2024-04-25T00:14:09.306399+0000","last_became_active":"2024-04-25T00:14:01.835525+0000","last_became_peered":"2024-04-25T00:14:01.835525+0000","last_unstale":"2024-04-25T00:14:09.306399+0000","last_undegraded":"2024-04-25T00:14:09.306399+0000","last_fullsized":"2024-04-25T00:14:09.306399+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:17.082562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305921+0000","last_change":"2024-04-25T00:14:01.831564+0000","last_active":"2024-04-25T00:14:09.305921+0000","last_peered":"2024-04-25T00:14:09.305921+0000","last_clean":"2024-04-25T00:14:09.305921+0000","last_became_active":"2024-04-25T00:14:01.828539+0000","last_became_peered":"2024-04-25T00:14:01.828539+0000","last_unstale":"2024-04-25T00:14:09.305921+0000","last_undegraded":"2024-04-25T00:14:09.305921+0000","last_fullsized":"2024-04-25T00:14:09.305921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:41:52.098299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305009+0000","last_change":"2024-04-25T00:14:01.835815+0000","last_active":"2024-04-25T00:14:09.305009+0000","last_peered":"2024-04-25T00:14:09.305009+0000","last_clean":"2024-04-25T00:14:09.305009+0000","last_became_active":"2024-04-25T00:14:01.835498+0000","last_became_peered":"2024-04-25T00:14:01.835498+0000","last_unstale":"2024-04-25T00:14:09.305009+0000","last_undegraded":"2024-04-25T00:14:09.305009+0000","last_fullsized":"2024-04-25T00:14:09.305009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:14:02.914149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305256+0000","last_change":"2024-04-25T00:14:01.839542+0000","last_active":"2024-04-25T00:14:09.305256+0000","last_peered":"2024-04-25T00:14:09.305256+0000","last_clean":"2024-04-25T00:14:09.305256+0000","last_became_active":"2024-04-25T00:14:01.839234+0000","last_became_peered":"2024-04-25T00:14:01.839234+0000","last_unstale":"2024-04-25T00:14:09.305256+0000","last_undegraded":"2024-04-25T00:14:09.305256+0000","last_fullsized":"2024-04-25T00:14:09.305256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:00.437311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305618+0000","last_change":"2024-04-25T00:14:01.832265+0000","last_active":"2024-04-25T00:14:09.305618+0000","last_peered":"2024-04-25T00:14:09.305618+0000","last_clean":"2024-04-25T00:14:09.305618+0000","last_became_active":"2024-04-25T00:14:01.831963+0000","last_became_peered":"2024-04-25T00:14:01.831963+0000","last_unstale":"2024-04-25T00:14:09.305618+0000","last_undegraded":"2024-04-25T00:14:09.305618+0000","last_fullsized":"2024-04-25T00:14:09.305618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:55.402265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307658+0000","last_change":"2024-04-25T00:14:01.835444+0000","last_active":"2024-04-25T00:14:09.307658+0000","last_peered":"2024-04-25T00:14:09.307658+0000","last_clean":"2024-04-25T00:14:09.307658+0000","last_became_active":"2024-04-25T00:14:01.835126+0000","last_became_peered":"2024-04-25T00:14:01.835126+0000","last_unstale":"2024-04-25T00:14:09.307658+0000","last_undegraded":"2024-04-25T00:14:09.307658+0000","last_fullsized":"2024-04-25T00:14:09.307658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:48.997924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843003+0000","last_change":"2024-04-25T00:14:01.834982+0000","last_active":"2024-04-25T00:14:08.843003+0000","last_peered":"2024-04-25T00:14:08.843003+0000","last_clean":"2024-04-25T00:14:08.843003+0000","last_became_active":"2024-04-25T00:14:01.834671+0000","last_became_peered":"2024-04-25T00:14:01.834671+0000","last_unstale":"2024-04-25T00:14:08.843003+0000","last_undegraded":"2024-04-25T00:14:08.843003+0000","last_fullsized":"2024-04-25T00:14:08.843003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:28.254408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306725+0000","last_change":"2024-04-25T00:14:01.824432+0000","last_active":"2024-04-25T00:14:09.306725+0000","last_peered":"2024-04-25T00:14:09.306725+0000","last_clean":"2024-04-25T00:14:09.306725+0000","last_became_active":"2024-04-25T00:14:01.823963+0000","last_became_peered":"2024-04-25T00:14:01.823963+0000","last_unstale":"2024-04-25T00:14:09.306725+0000","last_undegraded":"2024-04-25T00:14:09.306725+0000","last_fullsized":"2024-04-25T00:14:09.306725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:35.880932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306217+0000","last_change":"2024-04-25T00:14:06.861892+0000","last_active":"2024-04-25T00:14:09.306217+0000","last_peered":"2024-04-25T00:14:09.306217+0000","last_clean":"2024-04-25T00:14:09.306217+0000","last_became_active":"2024-04-25T00:14:06.861470+0000","last_became_peered":"2024-04-25T00:14:06.861470+0000","last_unstale":"2024-04-25T00:14:09.306217+0000","last_undegraded":"2024-04-25T00:14:09.306217+0000","last_fullsized":"2024-04-25T00:14:09.306217+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:06.192954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:27.536314+0000","last_change":"2024-04-25T00:14:01.412535+0000","last_active":"2024-04-25T00:14:27.536314+0000","last_peered":"2024-04-25T00:14:27.536314+0000","last_clean":"2024-04-25T00:14:27.536314+0000","last_became_active":"2024-04-25T00:14:01.412206+0000","last_became_peered":"2024-04-25T00:14:01.412206+0000","last_unstale":"2024-04-25T00:14:27.536314+0000","last_undegraded":"2024-04-25T00:14:27.536314+0000","last_fullsized":"2024-04-25T00:14:27.536314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:40.893987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306310+0000","last_change":"2024-04-25T00:14:01.840814+0000","last_active":"2024-04-25T00:14:09.306310+0000","last_peered":"2024-04-25T00:14:09.306310+0000","last_clean":"2024-04-25T00:14:09.306310+0000","last_became_active":"2024-04-25T00:14:01.840179+0000","last_became_peered":"2024-04-25T00:14:01.840179+0000","last_unstale":"2024-04-25T00:14:09.306310+0000","last_undegraded":"2024-04-25T00:14:09.306310+0000","last_fullsized":"2024-04-25T00:14:09.306310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:27.383918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305817+0000","last_change":"2024-04-25T00:14:06.857877+0000","last_active":"2024-04-25T00:14:09.305817+0000","last_peered":"2024-04-25T00:14:09.305817+0000","last_clean":"2024-04-25T00:14:09.305817+0000","last_became_active":"2024-04-25T00:14:06.857420+0000","last_became_peered":"2024-04-25T00:14:06.857420+0000","last_unstale":"2024-04-25T00:14:09.305817+0000","last_undegraded":"2024-04-25T00:14:09.305817+0000","last_fullsized":"2024-04-25T00:14:09.305817+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:54.424024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306621+0000","last_change":"2024-04-25T00:14:01.416560+0000","last_active":"2024-04-25T00:14:09.306621+0000","last_peered":"2024-04-25T00:14:09.306621+0000","last_clean":"2024-04-25T00:14:09.306621+0000","last_became_active":"2024-04-25T00:14:01.416246+0000","last_became_peered":"2024-04-25T00:14:01.416246+0000","last_unstale":"2024-04-25T00:14:09.306621+0000","last_undegraded":"2024-04-25T00:14:09.306621+0000","last_fullsized":"2024-04-25T00:14:09.306621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:03:47.938452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306055+0000","last_change":"2024-04-25T00:14:01.832607+0000","last_active":"2024-04-25T00:14:09.306055+0000","last_peered":"2024-04-25T00:14:09.306055+0000","last_clean":"2024-04-25T00:14:09.306055+0000","last_became_active":"2024-04-25T00:14:01.832300+0000","last_became_peered":"2024-04-25T00:14:01.832300+0000","last_unstale":"2024-04-25T00:14:09.306055+0000","last_undegraded":"2024-04-25T00:14:09.306055+0000","last_fullsized":"2024-04-25T00:14:09.306055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:22.045171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306107+0000","last_change":"2024-04-25T00:14:06.863642+0000","last_active":"2024-04-25T00:14:09.306107+0000","last_peered":"2024-04-25T00:14:09.306107+0000","last_clean":"2024-04-25T00:14:09.306107+0000","last_became_active":"2024-04-25T00:14:06.863284+0000","last_became_peered":"2024-04-25T00:14:06.863284+0000","last_unstale":"2024-04-25T00:14:09.306107+0000","last_undegraded":"2024-04-25T00:14:09.306107+0000","last_fullsized":"2024-04-25T00:14:09.306107+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:20.818625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:27.533684+0000","last_change":"2024-04-25T00:14:01.411874+0000","last_active":"2024-04-25T00:14:27.533684+0000","last_peered":"2024-04-25T00:14:27.533684+0000","last_clean":"2024-04-25T00:14:27.533684+0000","last_became_active":"2024-04-25T00:14:01.409568+0000","last_became_peered":"2024-04-25T00:14:01.409568+0000","last_unstale":"2024-04-25T00:14:27.533684+0000","last_undegraded":"2024-04-25T00:14:27.533684+0000","last_fullsized":"2024-04-25T00:14:27.533684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:22.415631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306790+0000","last_change":"2024-04-25T00:14:01.822471+0000","last_active":"2024-04-25T00:14:09.306790+0000","last_peered":"2024-04-25T00:14:09.306790+0000","last_clean":"2024-04-25T00:14:09.306790+0000","last_became_active":"2024-04-25T00:14:01.822181+0000","last_became_peered":"2024-04-25T00:14:01.822181+0000","last_unstale":"2024-04-25T00:14:09.306790+0000","last_undegraded":"2024-04-25T00:14:09.306790+0000","last_fullsized":"2024-04-25T00:14:09.306790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:44.857036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305663+0000","last_change":"2024-04-25T00:14:06.856987+0000","last_active":"2024-04-25T00:14:09.305663+0000","last_peered":"2024-04-25T00:14:09.305663+0000","last_clean":"2024-04-25T00:14:09.305663+0000","last_became_active":"2024-04-25T00:14:06.856053+0000","last_became_peered":"2024-04-25T00:14:06.856053+0000","last_unstale":"2024-04-25T00:14:09.305663+0000","last_undegraded":"2024-04-25T00:14:09.305663+0000","last_fullsized":"2024-04-25T00:14:09.305663+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:16.829969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843469+0000","last_change":"2024-04-25T00:14:01.410156+0000","last_active":"2024-04-25T00:14:08.843469+0000","last_peered":"2024-04-25T00:14:08.843469+0000","last_clean":"2024-04-25T00:14:08.843469+0000","last_became_active":"2024-04-25T00:14:01.409809+0000","last_became_peered":"2024-04-25T00:14:01.409809+0000","last_unstale":"2024-04-25T00:14:08.843469+0000","last_undegraded":"2024-04-25T00:14:08.843469+0000","last_fullsized":"2024-04-25T00:14:08.843469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:43.160419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306881+0000","last_change":"2024-04-25T00:14:01.836090+0000","last_active":"2024-04-25T00:14:09.306881+0000","last_peered":"2024-04-25T00:14:09.306881+0000","last_clean":"2024-04-25T00:14:09.306881+0000","last_became_active":"2024-04-25T00:14:01.835733+0000","last_became_peered":"2024-04-25T00:14:01.835733+0000","last_unstale":"2024-04-25T00:14:09.306881+0000","last_undegraded":"2024-04-25T00:14:09.306881+0000","last_fullsized":"2024-04-25T00:14:09.306881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:02.368893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306553+0000","last_change":"2024-04-25T00:14:06.851496+0000","last_active":"2024-04-25T00:14:09.306553+0000","last_peered":"2024-04-25T00:14:09.306553+0000","last_clean":"2024-04-25T00:14:09.306553+0000","last_became_active":"2024-04-25T00:14:06.851007+0000","last_became_peered":"2024-04-25T00:14:06.851007+0000","last_unstale":"2024-04-25T00:14:09.306553+0000","last_undegraded":"2024-04-25T00:14:09.306553+0000","last_fullsized":"2024-04-25T00:14:09.306553+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:50.543348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:26.530014+0000","last_change":"2024-04-25T00:14:01.414760+0000","last_active":"2024-04-25T00:14:26.530014+0000","last_peered":"2024-04-25T00:14:26.530014+0000","last_clean":"2024-04-25T00:14:26.530014+0000","last_became_active":"2024-04-25T00:14:01.414482+0000","last_became_peered":"2024-04-25T00:14:01.414482+0000","last_unstale":"2024-04-25T00:14:26.530014+0000","last_undegraded":"2024-04-25T00:14:26.530014+0000","last_fullsized":"2024-04-25T00:14:26.530014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:32.631933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843310+0000","last_change":"2024-04-25T00:14:01.832516+0000","last_active":"2024-04-25T00:14:08.843310+0000","last_peered":"2024-04-25T00:14:08.843310+0000","last_clean":"2024-04-25T00:14:08.843310+0000","last_became_active":"2024-04-25T00:14:01.832106+0000","last_became_peered":"2024-04-25T00:14:01.832106+0000","last_unstale":"2024-04-25T00:14:08.843310+0000","last_undegraded":"2024-04-25T00:14:08.843310+0000","last_fullsized":"2024-04-25T00:14:08.843310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:55.590486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306193+0000","last_change":"2024-04-25T00:14:06.861984+0000","last_active":"2024-04-25T00:14:09.306193+0000","last_peered":"2024-04-25T00:14:09.306193+0000","last_clean":"2024-04-25T00:14:09.306193+0000","last_became_active":"2024-04-25T00:14:06.861418+0000","last_became_peered":"2024-04-25T00:14:06.861418+0000","last_unstale":"2024-04-25T00:14:09.306193+0000","last_undegraded":"2024-04-25T00:14:09.306193+0000","last_fullsized":"2024-04-25T00:14:09.306193+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:27.559704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:27.534036+0000","last_change":"2024-04-25T00:14:01.407648+0000","last_active":"2024-04-25T00:14:27.534036+0000","last_peered":"2024-04-25T00:14:27.534036+0000","last_clean":"2024-04-25T00:14:27.534036+0000","last_became_active":"2024-04-25T00:14:01.407318+0000","last_became_peered":"2024-04-25T00:14:01.407318+0000","last_unstale":"2024-04-25T00:14:27.534036+0000","last_undegraded":"2024-04-25T00:14:27.534036+0000","last_fullsized":"2024-04-25T00:14:27.534036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:08:10.942814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306945+0000","last_change":"2024-04-25T00:14:01.833073+0000","last_active":"2024-04-25T00:14:09.306945+0000","last_peered":"2024-04-25T00:14:09.306945+0000","last_clean":"2024-04-25T00:14:09.306945+0000","last_became_active":"2024-04-25T00:14:01.832545+0000","last_became_peered":"2024-04-25T00:14:01.832545+0000","last_unstale":"2024-04-25T00:14:09.306945+0000","last_undegraded":"2024-04-25T00:14:09.306945+0000","last_fullsized":"2024-04-25T00:14:09.306945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:39.772781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307729+0000","last_change":"2024-04-25T00:14:06.862011+0000","last_active":"2024-04-25T00:14:09.307729+0000","last_peered":"2024-04-25T00:14:09.307729+0000","last_clean":"2024-04-25T00:14:09.307729+0000","last_became_active":"2024-04-25T00:14:06.861544+0000","last_became_peered":"2024-04-25T00:14:06.861544+0000","last_unstale":"2024-04-25T00:14:09.307729+0000","last_undegraded":"2024-04-25T00:14:09.307729+0000","last_fullsized":"2024-04-25T00:14:09.307729+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:37.736266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:27.535635+0000","last_change":"2024-04-25T00:14:01.397747+0000","last_active":"2024-04-25T00:14:27.535635+0000","last_peered":"2024-04-25T00:14:27.535635+0000","last_clean":"2024-04-25T00:14:27.535635+0000","last_became_active":"2024-04-25T00:14:01.397446+0000","last_became_peered":"2024-04-25T00:14:01.397446+0000","last_unstale":"2024-04-25T00:14:27.535635+0000","last_undegraded":"2024-04-25T00:14:27.535635+0000","last_fullsized":"2024-04-25T00:14:27.535635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:27.240034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305461+0000","last_change":"2024-04-25T00:14:01.833128+0000","last_active":"2024-04-25T00:14:09.305461+0000","last_peered":"2024-04-25T00:14:09.305461+0000","last_clean":"2024-04-25T00:14:09.305461+0000","last_became_active":"2024-04-25T00:14:01.832780+0000","last_became_peered":"2024-04-25T00:14:01.832780+0000","last_unstale":"2024-04-25T00:14:09.305461+0000","last_undegraded":"2024-04-25T00:14:09.305461+0000","last_fullsized":"2024-04-25T00:14:09.305461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.023737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306433+0000","last_change":"2024-04-25T00:14:06.860436+0000","last_active":"2024-04-25T00:14:09.306433+0000","last_peered":"2024-04-25T00:14:09.306433+0000","last_clean":"2024-04-25T00:14:09.306433+0000","last_became_active":"2024-04-25T00:14:06.860047+0000","last_became_peered":"2024-04-25T00:14:06.860047+0000","last_unstale":"2024-04-25T00:14:09.306433+0000","last_undegraded":"2024-04-25T00:14:09.306433+0000","last_fullsized":"2024-04-25T00:14:09.306433+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:08:41.246549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306233+0000","last_change":"2024-04-25T00:14:01.415939+0000","last_active":"2024-04-25T00:14:09.306233+0000","last_peered":"2024-04-25T00:14:09.306233+0000","last_clean":"2024-04-25T00:14:09.306233+0000","last_became_active":"2024-04-25T00:14:01.415513+0000","last_became_peered":"2024-04-25T00:14:01.415513+0000","last_unstale":"2024-04-25T00:14:09.306233+0000","last_undegraded":"2024-04-25T00:14:09.306233+0000","last_fullsized":"2024-04-25T00:14:09.306233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:57:23.588868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.842554+0000","last_change":"2024-04-25T00:14:01.838793+0000","last_active":"2024-04-25T00:14:08.842554+0000","last_peered":"2024-04-25T00:14:08.842554+0000","last_clean":"2024-04-25T00:14:08.842554+0000","last_became_active":"2024-04-25T00:14:01.838453+0000","last_became_peered":"2024-04-25T00:14:01.838453+0000","last_unstale":"2024-04-25T00:14:08.842554+0000","last_undegraded":"2024-04-25T00:14:08.842554+0000","last_fullsized":"2024-04-25T00:14:08.842554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:18.674009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305319+0000","last_change":"2024-04-25T00:14:06.856431+0000","last_active":"2024-04-25T00:14:09.305319+0000","last_peered":"2024-04-25T00:14:09.305319+0000","last_clean":"2024-04-25T00:14:09.305319+0000","last_became_active":"2024-04-25T00:14:06.856047+0000","last_became_peered":"2024-04-25T00:14:06.856047+0000","last_unstale":"2024-04-25T00:14:09.305319+0000","last_undegraded":"2024-04-25T00:14:09.305319+0000","last_fullsized":"2024-04-25T00:14:09.305319+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:20.646673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:28.538057+0000","last_change":"2024-04-25T00:14:01.399054+0000","last_active":"2024-04-25T00:14:28.538057+0000","last_peered":"2024-04-25T00:14:28.538057+0000","last_clean":"2024-04-25T00:14:28.538057+0000","last_became_active":"2024-04-25T00:14:01.398711+0000","last_became_peered":"2024-04-25T00:14:01.398711+0000","last_unstale":"2024-04-25T00:14:28.538057+0000","last_undegraded":"2024-04-25T00:14:28.538057+0000","last_fullsized":"2024-04-25T00:14:28.538057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:26.434014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307281+0000","last_change":"2024-04-25T00:14:01.829132+0000","last_active":"2024-04-25T00:14:09.307281+0000","last_peered":"2024-04-25T00:14:09.307281+0000","last_clean":"2024-04-25T00:14:09.307281+0000","last_became_active":"2024-04-25T00:14:01.828446+0000","last_became_peered":"2024-04-25T00:14:01.828446+0000","last_unstale":"2024-04-25T00:14:09.307281+0000","last_undegraded":"2024-04-25T00:14:09.307281+0000","last_fullsized":"2024-04-25T00:14:09.307281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:35.788478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.842621+0000","last_change":"2024-04-25T00:14:06.861884+0000","last_active":"2024-04-25T00:14:08.842621+0000","last_peered":"2024-04-25T00:14:08.842621+0000","last_clean":"2024-04-25T00:14:08.842621+0000","last_became_active":"2024-04-25T00:14:06.861413+0000","last_became_peered":"2024-04-25T00:14:06.861413+0000","last_unstale":"2024-04-25T00:14:08.842621+0000","last_undegraded":"2024-04-25T00:14:08.842621+0000","last_fullsized":"2024-04-25T00:14:08.842621+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:53.876308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:26.530769+0000","last_change":"2024-04-25T00:14:01.410372+0000","last_active":"2024-04-25T00:14:26.530769+0000","last_peered":"2024-04-25T00:14:26.530769+0000","last_clean":"2024-04-25T00:14:26.530769+0000","last_became_active":"2024-04-25T00:14:01.410074+0000","last_became_peered":"2024-04-25T00:14:01.410074+0000","last_unstale":"2024-04-25T00:14:26.530769+0000","last_undegraded":"2024-04-25T00:14:26.530769+0000","last_fullsized":"2024-04-25T00:14:26.530769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:00.964580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306219+0000","last_change":"2024-04-25T00:14:01.839723+0000","last_active":"2024-04-25T00:14:09.306219+0000","last_peered":"2024-04-25T00:14:09.306219+0000","last_clean":"2024-04-25T00:14:09.306219+0000","last_became_active":"2024-04-25T00:14:01.839199+0000","last_became_peered":"2024-04-25T00:14:01.839199+0000","last_unstale":"2024-04-25T00:14:09.306219+0000","last_undegraded":"2024-04-25T00:14:09.306219+0000","last_fullsized":"2024-04-25T00:14:09.306219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:32.087459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307445+0000","last_change":"2024-04-25T00:14:06.857525+0000","last_active":"2024-04-25T00:14:09.307445+0000","last_peered":"2024-04-25T00:14:09.307445+0000","last_clean":"2024-04-25T00:14:09.307445+0000","last_became_active":"2024-04-25T00:14:06.856886+0000","last_became_peered":"2024-04-25T00:14:06.856886+0000","last_unstale":"2024-04-25T00:14:09.307445+0000","last_undegraded":"2024-04-25T00:14:09.307445+0000","last_fullsized":"2024-04-25T00:14:09.307445+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:05.366093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306893+0000","last_change":"2024-04-25T00:14:01.401128+0000","last_active":"2024-04-25T00:14:09.306893+0000","last_peered":"2024-04-25T00:14:09.306893+0000","last_clean":"2024-04-25T00:14:09.306893+0000","last_became_active":"2024-04-25T00:14:01.400125+0000","last_became_peered":"2024-04-25T00:14:01.400125+0000","last_unstale":"2024-04-25T00:14:09.306893+0000","last_undegraded":"2024-04-25T00:14:09.306893+0000","last_fullsized":"2024-04-25T00:14:09.306893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:38:43.787855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307142+0000","last_change":"2024-04-25T00:14:01.830649+0000","last_active":"2024-04-25T00:14:09.307142+0000","last_peered":"2024-04-25T00:14:09.307142+0000","last_clean":"2024-04-25T00:14:09.307142+0000","last_became_active":"2024-04-25T00:14:01.830253+0000","last_became_peered":"2024-04-25T00:14:01.830253+0000","last_unstale":"2024-04-25T00:14:09.307142+0000","last_undegraded":"2024-04-25T00:14:09.307142+0000","last_fullsized":"2024-04-25T00:14:09.307142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:37.474833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.842752+0000","last_change":"2024-04-25T00:14:06.861275+0000","last_active":"2024-04-25T00:14:08.842752+0000","last_peered":"2024-04-25T00:14:08.842752+0000","last_clean":"2024-04-25T00:14:08.842752+0000","last_became_active":"2024-04-25T00:14:06.860891+0000","last_became_peered":"2024-04-25T00:14:06.860891+0000","last_unstale":"2024-04-25T00:14:08.842752+0000","last_undegraded":"2024-04-25T00:14:08.842752+0000","last_fullsized":"2024-04-25T00:14:08.842752+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:38.685837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'23","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:29.910371+0000","last_change":"2024-04-25T00:14:01.404706+0000","last_active":"2024-04-25T00:14:29.910371+0000","last_peered":"2024-04-25T00:14:29.910371+0000","last_clean":"2024-04-25T00:14:29.910371+0000","last_became_active":"2024-04-25T00:14:01.404231+0000","last_became_peered":"2024-04-25T00:14:01.404231+0000","last_unstale":"2024-04-25T00:14:29.910371+0000","last_undegraded":"2024-04-25T00:14:29.910371+0000","last_fullsized":"2024-04-25T00:14:29.910371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:36.437491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40471,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":26,"num_write":24,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843685+0000","last_change":"2024-04-25T00:14:01.828312+0000","last_active":"2024-04-25T00:14:08.843685+0000","last_peered":"2024-04-25T00:14:08.843685+0000","last_clean":"2024-04-25T00:14:08.843685+0000","last_became_active":"2024-04-25T00:14:01.827947+0000","last_became_peered":"2024-04-25T00:14:01.827947+0000","last_unstale":"2024-04-25T00:14:08.843685+0000","last_undegraded":"2024-04-25T00:14:08.843685+0000","last_fullsized":"2024-04-25T00:14:08.843685+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.782796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307156+0000","last_change":"2024-04-25T00:14:06.854379+0000","last_active":"2024-04-25T00:14:09.307156+0000","last_peered":"2024-04-25T00:14:09.307156+0000","last_clean":"2024-04-25T00:14:09.307156+0000","last_became_active":"2024-04-25T00:14:06.853307+0000","last_became_peered":"2024-04-25T00:14:06.853307+0000","last_unstale":"2024-04-25T00:14:09.307156+0000","last_undegraded":"2024-04-25T00:14:09.307156+0000","last_fullsized":"2024-04-25T00:14:09.307156+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:24:11.804189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:27.534855+0000","last_change":"2024-04-25T00:14:01.410395+0000","last_active":"2024-04-25T00:14:27.534855+0000","last_peered":"2024-04-25T00:14:27.534855+0000","last_clean":"2024-04-25T00:14:27.534855+0000","last_became_active":"2024-04-25T00:14:01.410039+0000","last_became_peered":"2024-04-25T00:14:01.410039+0000","last_unstale":"2024-04-25T00:14:27.534855+0000","last_undegraded":"2024-04-25T00:14:27.534855+0000","last_fullsized":"2024-04-25T00:14:27.534855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:29.496519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305998+0000","last_change":"2024-04-25T00:14:01.828403+0000","last_active":"2024-04-25T00:14:09.305998+0000","last_peered":"2024-04-25T00:14:09.305998+0000","last_clean":"2024-04-25T00:14:09.305998+0000","last_became_active":"2024-04-25T00:14:01.828107+0000","last_became_peered":"2024-04-25T00:14:01.828107+0000","last_unstale":"2024-04-25T00:14:09.305998+0000","last_undegraded":"2024-04-25T00:14:09.305998+0000","last_fullsized":"2024-04-25T00:14:09.305998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:29.417639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307322+0000","last_change":"2024-04-25T00:14:06.861845+0000","last_active":"2024-04-25T00:14:09.307322+0000","last_peered":"2024-04-25T00:14:09.307322+0000","last_clean":"2024-04-25T00:14:09.307322+0000","last_became_active":"2024-04-25T00:14:06.861451+0000","last_became_peered":"2024-04-25T00:14:06.861451+0000","last_unstale":"2024-04-25T00:14:09.307322+0000","last_undegraded":"2024-04-25T00:14:09.307322+0000","last_fullsized":"2024-04-25T00:14:09.307322+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:21.473628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:26.529581+0000","last_change":"2024-04-25T00:14:01.414406+0000","last_active":"2024-04-25T00:14:26.529581+0000","last_peered":"2024-04-25T00:14:26.529581+0000","last_clean":"2024-04-25T00:14:26.529581+0000","last_became_active":"2024-04-25T00:14:01.414085+0000","last_became_peered":"2024-04-25T00:14:01.414085+0000","last_unstale":"2024-04-25T00:14:26.529581+0000","last_undegraded":"2024-04-25T00:14:26.529581+0000","last_fullsized":"2024-04-25T00:14:26.529581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:22.565371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307277+0000","last_change":"2024-04-25T00:14:01.833652+0000","last_active":"2024-04-25T00:14:09.307277+0000","last_peered":"2024-04-25T00:14:09.307277+0000","last_clean":"2024-04-25T00:14:09.307277+0000","last_became_active":"2024-04-25T00:14:01.833307+0000","last_became_peered":"2024-04-25T00:14:01.833307+0000","last_unstale":"2024-04-25T00:14:09.307277+0000","last_undegraded":"2024-04-25T00:14:09.307277+0000","last_fullsized":"2024-04-25T00:14:09.307277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:55:41.261994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843067+0000","last_change":"2024-04-25T00:14:06.860167+0000","last_active":"2024-04-25T00:14:08.843067+0000","last_peered":"2024-04-25T00:14:08.843067+0000","last_clean":"2024-04-25T00:14:08.843067+0000","last_became_active":"2024-04-25T00:14:06.858651+0000","last_became_peered":"2024-04-25T00:14:06.858651+0000","last_unstale":"2024-04-25T00:14:08.843067+0000","last_undegraded":"2024-04-25T00:14:08.843067+0000","last_fullsized":"2024-04-25T00:14:08.843067+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:11.775434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:28.536263+0000","last_change":"2024-04-25T00:14:01.406267+0000","last_active":"2024-04-25T00:14:28.536263+0000","last_peered":"2024-04-25T00:14:28.536263+0000","last_clean":"2024-04-25T00:14:28.536263+0000","last_became_active":"2024-04-25T00:14:01.405985+0000","last_became_peered":"2024-04-25T00:14:01.405985+0000","last_unstale":"2024-04-25T00:14:28.536263+0000","last_undegraded":"2024-04-25T00:14:28.536263+0000","last_fullsized":"2024-04-25T00:14:28.536263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:17.253805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306649+0000","last_change":"2024-04-25T00:14:01.838019+0000","last_active":"2024-04-25T00:14:09.306649+0000","last_peered":"2024-04-25T00:14:09.306649+0000","last_clean":"2024-04-25T00:14:09.306649+0000","last_became_active":"2024-04-25T00:14:01.837691+0000","last_became_peered":"2024-04-25T00:14:01.837691+0000","last_unstale":"2024-04-25T00:14:09.306649+0000","last_undegraded":"2024-04-25T00:14:09.306649+0000","last_fullsized":"2024-04-25T00:14:09.306649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:26.677039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306288+0000","last_change":"2024-04-25T00:14:06.857061+0000","last_active":"2024-04-25T00:14:09.306288+0000","last_peered":"2024-04-25T00:14:09.306288+0000","last_clean":"2024-04-25T00:14:09.306288+0000","last_became_active":"2024-04-25T00:14:06.856678+0000","last_became_peered":"2024-04-25T00:14:06.856678+0000","last_unstale":"2024-04-25T00:14:09.306288+0000","last_undegraded":"2024-04-25T00:14:09.306288+0000","last_fullsized":"2024-04-25T00:14:09.306288+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:36:52.730861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:27.537047+0000","last_change":"2024-04-25T00:14:01.408385+0000","last_active":"2024-04-25T00:14:27.537047+0000","last_peered":"2024-04-25T00:14:27.537047+0000","last_clean":"2024-04-25T00:14:27.537047+0000","last_became_active":"2024-04-25T00:14:01.408049+0000","last_became_peered":"2024-04-25T00:14:01.408049+0000","last_unstale":"2024-04-25T00:14:27.537047+0000","last_undegraded":"2024-04-25T00:14:27.537047+0000","last_fullsized":"2024-04-25T00:14:27.537047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:19.765467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305958+0000","last_change":"2024-04-25T00:14:01.835113+0000","last_active":"2024-04-25T00:14:09.305958+0000","last_peered":"2024-04-25T00:14:09.305958+0000","last_clean":"2024-04-25T00:14:09.305958+0000","last_became_active":"2024-04-25T00:14:01.834782+0000","last_became_peered":"2024-04-25T00:14:01.834782+0000","last_unstale":"2024-04-25T00:14:09.305958+0000","last_undegraded":"2024-04-25T00:14:09.305958+0000","last_fullsized":"2024-04-25T00:14:09.305958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:25.595476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305839+0000","last_change":"2024-04-25T00:14:06.854192+0000","last_active":"2024-04-25T00:14:09.305839+0000","last_peered":"2024-04-25T00:14:09.305839+0000","last_clean":"2024-04-25T00:14:09.305839+0000","last_became_active":"2024-04-25T00:14:06.853325+0000","last_became_peered":"2024-04-25T00:14:06.853325+0000","last_unstale":"2024-04-25T00:14:09.305839+0000","last_undegraded":"2024-04-25T00:14:09.305839+0000","last_fullsized":"2024-04-25T00:14:09.305839+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:15.308169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:28.530098+0000","last_change":"2024-04-25T00:14:01.407247+0000","last_active":"2024-04-25T00:14:28.530098+0000","last_peered":"2024-04-25T00:14:28.530098+0000","last_clean":"2024-04-25T00:14:28.530098+0000","last_became_active":"2024-04-25T00:14:01.406891+0000","last_became_peered":"2024-04-25T00:14:01.406891+0000","last_unstale":"2024-04-25T00:14:28.530098+0000","last_undegraded":"2024-04-25T00:14:28.530098+0000","last_fullsized":"2024-04-25T00:14:28.530098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:43.294562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307381+0000","last_change":"2024-04-25T00:14:01.829364+0000","last_active":"2024-04-25T00:14:09.307381+0000","last_peered":"2024-04-25T00:14:09.307381+0000","last_clean":"2024-04-25T00:14:09.307381+0000","last_became_active":"2024-04-25T00:14:01.829087+0000","last_became_peered":"2024-04-25T00:14:01.829087+0000","last_unstale":"2024-04-25T00:14:09.307381+0000","last_undegraded":"2024-04-25T00:14:09.307381+0000","last_fullsized":"2024-04-25T00:14:09.307381+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:17.651815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843462+0000","last_change":"2024-04-25T00:14:07.323442+0000","last_active":"2024-04-25T00:14:08.843462+0000","last_peered":"2024-04-25T00:14:08.843462+0000","last_clean":"2024-04-25T00:14:08.843462+0000","last_became_active":"2024-04-25T00:14:07.322943+0000","last_became_peered":"2024-04-25T00:14:07.322943+0000","last_unstale":"2024-04-25T00:14:08.843462+0000","last_undegraded":"2024-04-25T00:14:08.843462+0000","last_fullsized":"2024-04-25T00:14:08.843462+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:49.166240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843045+0000","last_change":"2024-04-25T00:14:01.410936+0000","last_active":"2024-04-25T00:14:08.843045+0000","last_peered":"2024-04-25T00:14:08.843045+0000","last_clean":"2024-04-25T00:14:08.843045+0000","last_became_active":"2024-04-25T00:14:01.410321+0000","last_became_peered":"2024-04-25T00:14:01.410321+0000","last_unstale":"2024-04-25T00:14:08.843045+0000","last_undegraded":"2024-04-25T00:14:08.843045+0000","last_fullsized":"2024-04-25T00:14:08.843045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:44.603199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843739+0000","last_change":"2024-04-25T00:14:01.830842+0000","last_active":"2024-04-25T00:14:08.843739+0000","last_peered":"2024-04-25T00:14:08.843739+0000","last_clean":"2024-04-25T00:14:08.843739+0000","last_became_active":"2024-04-25T00:14:01.830543+0000","last_became_peered":"2024-04-25T00:14:01.830543+0000","last_unstale":"2024-04-25T00:14:08.843739+0000","last_undegraded":"2024-04-25T00:14:08.843739+0000","last_fullsized":"2024-04-25T00:14:08.843739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:59.897133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306636+0000","last_change":"2024-04-25T00:14:06.856841+0000","last_active":"2024-04-25T00:14:09.306636+0000","last_peered":"2024-04-25T00:14:09.306636+0000","last_clean":"2024-04-25T00:14:09.306636+0000","last_became_active":"2024-04-25T00:14:06.856416+0000","last_became_peered":"2024-04-25T00:14:06.856416+0000","last_unstale":"2024-04-25T00:14:09.306636+0000","last_undegraded":"2024-04-25T00:14:09.306636+0000","last_fullsized":"2024-04-25T00:14:09.306636+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:27.181175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307062+0000","last_change":"2024-04-25T00:14:01.409627+0000","last_active":"2024-04-25T00:14:09.307062+0000","last_peered":"2024-04-25T00:14:09.307062+0000","last_clean":"2024-04-25T00:14:09.307062+0000","last_became_active":"2024-04-25T00:14:01.409304+0000","last_became_peered":"2024-04-25T00:14:01.409304+0000","last_unstale":"2024-04-25T00:14:09.307062+0000","last_undegraded":"2024-04-25T00:14:09.307062+0000","last_fullsized":"2024-04-25T00:14:09.307062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:27.242004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843827+0000","last_change":"2024-04-25T00:14:01.829544+0000","last_active":"2024-04-25T00:14:08.843827+0000","last_peered":"2024-04-25T00:14:08.843827+0000","last_clean":"2024-04-25T00:14:08.843827+0000","last_became_active":"2024-04-25T00:14:01.829233+0000","last_became_peered":"2024-04-25T00:14:01.829233+0000","last_unstale":"2024-04-25T00:14:08.843827+0000","last_undegraded":"2024-04-25T00:14:08.843827+0000","last_fullsized":"2024-04-25T00:14:08.843827+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:16.102041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306455+0000","last_change":"2024-04-25T00:14:07.323213+0000","last_active":"2024-04-25T00:14:09.306455+0000","last_peered":"2024-04-25T00:14:09.306455+0000","last_clean":"2024-04-25T00:14:09.306455+0000","last_became_active":"2024-04-25T00:14:07.322763+0000","last_became_peered":"2024-04-25T00:14:07.322763+0000","last_unstale":"2024-04-25T00:14:09.306455+0000","last_undegraded":"2024-04-25T00:14:09.306455+0000","last_fullsized":"2024-04-25T00:14:09.306455+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:45.269314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306420+0000","last_change":"2024-04-25T00:14:01.832836+0000","last_active":"2024-04-25T00:14:09.306420+0000","last_peered":"2024-04-25T00:14:09.306420+0000","last_clean":"2024-04-25T00:14:09.306420+0000","last_became_active":"2024-04-25T00:14:01.830652+0000","last_became_peered":"2024-04-25T00:14:01.830652+0000","last_unstale":"2024-04-25T00:14:09.306420+0000","last_undegraded":"2024-04-25T00:14:09.306420+0000","last_fullsized":"2024-04-25T00:14:09.306420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:10:21.803200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:27.535012+0000","last_change":"2024-04-25T00:14:01.397066+0000","last_active":"2024-04-25T00:14:27.535012+0000","last_peered":"2024-04-25T00:14:27.535012+0000","last_clean":"2024-04-25T00:14:27.535012+0000","last_became_active":"2024-04-25T00:14:01.396728+0000","last_became_peered":"2024-04-25T00:14:01.396728+0000","last_unstale":"2024-04-25T00:14:27.535012+0000","last_undegraded":"2024-04-25T00:14:27.535012+0000","last_fullsized":"2024-04-25T00:14:27.535012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:28.334329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307671+0000","last_change":"2024-04-25T00:14:06.853039+0000","last_active":"2024-04-25T00:14:09.307671+0000","last_peered":"2024-04-25T00:14:09.307671+0000","last_clean":"2024-04-25T00:14:09.307671+0000","last_became_active":"2024-04-25T00:14:06.852568+0000","last_became_peered":"2024-04-25T00:14:06.852568+0000","last_unstale":"2024-04-25T00:14:09.307671+0000","last_undegraded":"2024-04-25T00:14:09.307671+0000","last_fullsized":"2024-04-25T00:14:09.307671+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:59.604160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305343+0000","last_change":"2024-04-25T00:14:01.840315+0000","last_active":"2024-04-25T00:14:09.305343+0000","last_peered":"2024-04-25T00:14:09.305343+0000","last_clean":"2024-04-25T00:14:09.305343+0000","last_became_active":"2024-04-25T00:14:01.839537+0000","last_became_peered":"2024-04-25T00:14:01.839537+0000","last_unstale":"2024-04-25T00:14:09.305343+0000","last_undegraded":"2024-04-25T00:14:09.305343+0000","last_fullsized":"2024-04-25T00:14:09.305343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:34.126164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:28.534457+0000","last_change":"2024-04-25T00:14:01.412493+0000","last_active":"2024-04-25T00:14:28.534457+0000","last_peered":"2024-04-25T00:14:28.534457+0000","last_clean":"2024-04-25T00:14:28.534457+0000","last_became_active":"2024-04-25T00:14:01.412133+0000","last_became_peered":"2024-04-25T00:14:01.412133+0000","last_unstale":"2024-04-25T00:14:28.534457+0000","last_undegraded":"2024-04-25T00:14:28.534457+0000","last_fullsized":"2024-04-25T00:14:28.534457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:44.933728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":360,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306136+0000","last_change":"2024-04-25T00:12:49.169294+0000","last_active":"2024-04-25T00:14:09.306136+0000","last_peered":"2024-04-25T00:14:09.306136+0000","last_clean":"2024-04-25T00:14:09.306136+0000","last_became_active":"2024-04-25T00:12:49.168783+0000","last_became_peered":"2024-04-25T00:12:49.168783+0000","last_unstale":"2024-04-25T00:14:09.306136+0000","last_undegraded":"2024-04-25T00:14:09.306136+0000","last_fullsized":"2024-04-25T00:14:09.306136+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:25.202280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:25.202280+0000","last_clean_scrub_stamp":"2024-04-25T00:09:25.202280+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:04.713986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306639+0000","last_change":"2024-04-25T00:14:07.325530+0000","last_active":"2024-04-25T00:14:09.306639+0000","last_peered":"2024-04-25T00:14:09.306639+0000","last_clean":"2024-04-25T00:14:09.306639+0000","last_became_active":"2024-04-25T00:14:07.325152+0000","last_became_peered":"2024-04-25T00:14:07.325152+0000","last_unstale":"2024-04-25T00:14:09.306639+0000","last_undegraded":"2024-04-25T00:14:09.306639+0000","last_fullsized":"2024-04-25T00:14:09.306639+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:22:48.136526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305666+0000","last_change":"2024-04-25T00:14:01.826418+0000","last_active":"2024-04-25T00:14:09.305666+0000","last_peered":"2024-04-25T00:14:09.305666+0000","last_clean":"2024-04-25T00:14:09.305666+0000","last_became_active":"2024-04-25T00:14:01.825783+0000","last_became_peered":"2024-04-25T00:14:01.825783+0000","last_unstale":"2024-04-25T00:14:09.305666+0000","last_undegraded":"2024-04-25T00:14:09.305666+0000","last_fullsized":"2024-04-25T00:14:09.305666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:05.981359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:27.534416+0000","last_change":"2024-04-25T00:14:01.417022+0000","last_active":"2024-04-25T00:14:27.534416+0000","last_peered":"2024-04-25T00:14:27.534416+0000","last_clean":"2024-04-25T00:14:27.534416+0000","last_became_active":"2024-04-25T00:14:01.416804+0000","last_became_peered":"2024-04-25T00:14:01.416804+0000","last_unstale":"2024-04-25T00:14:27.534416+0000","last_undegraded":"2024-04-25T00:14:27.534416+0000","last_fullsized":"2024-04-25T00:14:27.534416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:31.561242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307500+0000","last_change":"2024-04-25T00:14:06.856500+0000","last_active":"2024-04-25T00:14:09.307500+0000","last_peered":"2024-04-25T00:14:09.307500+0000","last_clean":"2024-04-25T00:14:09.307500+0000","last_became_active":"2024-04-25T00:14:06.856077+0000","last_became_peered":"2024-04-25T00:14:06.856077+0000","last_unstale":"2024-04-25T00:14:09.307500+0000","last_undegraded":"2024-04-25T00:14:09.307500+0000","last_fullsized":"2024-04-25T00:14:09.307500+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:10:04.129533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:26.529730+0000","last_change":"2024-04-25T00:14:01.414599+0000","last_active":"2024-04-25T00:14:26.529730+0000","last_peered":"2024-04-25T00:14:26.529730+0000","last_clean":"2024-04-25T00:14:26.529730+0000","last_became_active":"2024-04-25T00:14:01.413796+0000","last_became_peered":"2024-04-25T00:14:01.413796+0000","last_unstale":"2024-04-25T00:14:26.529730+0000","last_undegraded":"2024-04-25T00:14:26.529730+0000","last_fullsized":"2024-04-25T00:14:26.529730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:44.981848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306903+0000","last_change":"2024-04-25T00:14:01.827718+0000","last_active":"2024-04-25T00:14:09.306903+0000","last_peered":"2024-04-25T00:14:09.306903+0000","last_clean":"2024-04-25T00:14:09.306903+0000","last_became_active":"2024-04-25T00:14:01.827410+0000","last_became_peered":"2024-04-25T00:14:01.827410+0000","last_unstale":"2024-04-25T00:14:09.306903+0000","last_undegraded":"2024-04-25T00:14:09.306903+0000","last_fullsized":"2024-04-25T00:14:09.306903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:32.428137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305389+0000","last_change":"2024-04-25T00:14:07.323304+0000","last_active":"2024-04-25T00:14:09.305389+0000","last_peered":"2024-04-25T00:14:09.305389+0000","last_clean":"2024-04-25T00:14:09.305389+0000","last_became_active":"2024-04-25T00:14:07.322852+0000","last_became_peered":"2024-04-25T00:14:07.322852+0000","last_unstale":"2024-04-25T00:14:09.305389+0000","last_undegraded":"2024-04-25T00:14:09.305389+0000","last_fullsized":"2024-04-25T00:14:09.305389+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:35.173934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:27.535521+0000","last_change":"2024-04-25T00:14:01.413209+0000","last_active":"2024-04-25T00:14:27.535521+0000","last_peered":"2024-04-25T00:14:27.535521+0000","last_clean":"2024-04-25T00:14:27.535521+0000","last_became_active":"2024-04-25T00:14:01.412892+0000","last_became_peered":"2024-04-25T00:14:01.412892+0000","last_unstale":"2024-04-25T00:14:27.535521+0000","last_undegraded":"2024-04-25T00:14:27.535521+0000","last_fullsized":"2024-04-25T00:14:27.535521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:26.152962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306583+0000","last_change":"2024-04-25T00:14:01.834372+0000","last_active":"2024-04-25T00:14:09.306583+0000","last_peered":"2024-04-25T00:14:09.306583+0000","last_clean":"2024-04-25T00:14:09.306583+0000","last_became_active":"2024-04-25T00:14:01.834062+0000","last_became_peered":"2024-04-25T00:14:01.834062+0000","last_unstale":"2024-04-25T00:14:09.306583+0000","last_undegraded":"2024-04-25T00:14:09.306583+0000","last_fullsized":"2024-04-25T00:14:09.306583+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:05.580604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305626+0000","last_change":"2024-04-25T00:14:06.852303+0000","last_active":"2024-04-25T00:14:09.305626+0000","last_peered":"2024-04-25T00:14:09.305626+0000","last_clean":"2024-04-25T00:14:09.305626+0000","last_became_active":"2024-04-25T00:14:06.850957+0000","last_became_peered":"2024-04-25T00:14:06.850957+0000","last_unstale":"2024-04-25T00:14:09.305626+0000","last_undegraded":"2024-04-25T00:14:09.305626+0000","last_fullsized":"2024-04-25T00:14:09.305626+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:47.208880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:26.529886+0000","last_change":"2024-04-25T00:14:01.405538+0000","last_active":"2024-04-25T00:14:26.529886+0000","last_peered":"2024-04-25T00:14:26.529886+0000","last_clean":"2024-04-25T00:14:26.529886+0000","last_became_active":"2024-04-25T00:14:01.405155+0000","last_became_peered":"2024-04-25T00:14:01.405155+0000","last_unstale":"2024-04-25T00:14:26.529886+0000","last_undegraded":"2024-04-25T00:14:26.529886+0000","last_fullsized":"2024-04-25T00:14:26.529886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:20.864753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306618+0000","last_change":"2024-04-25T00:14:01.837058+0000","last_active":"2024-04-25T00:14:09.306618+0000","last_peered":"2024-04-25T00:14:09.306618+0000","last_clean":"2024-04-25T00:14:09.306618+0000","last_became_active":"2024-04-25T00:14:01.835838+0000","last_became_peered":"2024-04-25T00:14:01.835838+0000","last_unstale":"2024-04-25T00:14:09.306618+0000","last_undegraded":"2024-04-25T00:14:09.306618+0000","last_fullsized":"2024-04-25T00:14:09.306618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:21.696448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306347+0000","last_change":"2024-04-25T00:14:07.323870+0000","last_active":"2024-04-25T00:14:09.306347+0000","last_peered":"2024-04-25T00:14:09.306347+0000","last_clean":"2024-04-25T00:14:09.306347+0000","last_became_active":"2024-04-25T00:14:07.323356+0000","last_became_peered":"2024-04-25T00:14:07.323356+0000","last_unstale":"2024-04-25T00:14:09.306347+0000","last_undegraded":"2024-04-25T00:14:09.306347+0000","last_fullsized":"2024-04-25T00:14:09.306347+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:35.160618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:26.528549+0000","last_change":"2024-04-25T00:14:01.409453+0000","last_active":"2024-04-25T00:14:26.528549+0000","last_peered":"2024-04-25T00:14:26.528549+0000","last_clean":"2024-04-25T00:14:26.528549+0000","last_became_active":"2024-04-25T00:14:01.409164+0000","last_became_peered":"2024-04-25T00:14:01.409164+0000","last_unstale":"2024-04-25T00:14:26.528549+0000","last_undegraded":"2024-04-25T00:14:26.528549+0000","last_fullsized":"2024-04-25T00:14:26.528549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:23.266090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843228+0000","last_change":"2024-04-25T00:14:01.826790+0000","last_active":"2024-04-25T00:14:08.843228+0000","last_peered":"2024-04-25T00:14:08.843228+0000","last_clean":"2024-04-25T00:14:08.843228+0000","last_became_active":"2024-04-25T00:14:01.826457+0000","last_became_peered":"2024-04-25T00:14:01.826457+0000","last_unstale":"2024-04-25T00:14:08.843228+0000","last_undegraded":"2024-04-25T00:14:08.843228+0000","last_fullsized":"2024-04-25T00:14:08.843228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:15.356448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843512+0000","last_change":"2024-04-25T00:14:06.863280+0000","last_active":"2024-04-25T00:14:08.843512+0000","last_peered":"2024-04-25T00:14:08.843512+0000","last_clean":"2024-04-25T00:14:08.843512+0000","last_became_active":"2024-04-25T00:14:06.862725+0000","last_became_peered":"2024-04-25T00:14:06.862725+0000","last_unstale":"2024-04-25T00:14:08.843512+0000","last_undegraded":"2024-04-25T00:14:08.843512+0000","last_fullsized":"2024-04-25T00:14:08.843512+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:12.791744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:26.529922+0000","last_change":"2024-04-25T00:14:01.414856+0000","last_active":"2024-04-25T00:14:26.529922+0000","last_peered":"2024-04-25T00:14:26.529922+0000","last_clean":"2024-04-25T00:14:26.529922+0000","last_became_active":"2024-04-25T00:14:01.414533+0000","last_became_peered":"2024-04-25T00:14:01.414533+0000","last_unstale":"2024-04-25T00:14:26.529922+0000","last_undegraded":"2024-04-25T00:14:26.529922+0000","last_fullsized":"2024-04-25T00:14:26.529922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:01.090539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307007+0000","last_change":"2024-04-25T00:14:01.831452+0000","last_active":"2024-04-25T00:14:09.307007+0000","last_peered":"2024-04-25T00:14:09.307007+0000","last_clean":"2024-04-25T00:14:09.307007+0000","last_became_active":"2024-04-25T00:14:01.831151+0000","last_became_peered":"2024-04-25T00:14:01.831151+0000","last_unstale":"2024-04-25T00:14:09.307007+0000","last_undegraded":"2024-04-25T00:14:09.307007+0000","last_fullsized":"2024-04-25T00:14:09.307007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:53:07.659508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306452+0000","last_change":"2024-04-25T00:14:06.857601+0000","last_active":"2024-04-25T00:14:09.306452+0000","last_peered":"2024-04-25T00:14:09.306452+0000","last_clean":"2024-04-25T00:14:09.306452+0000","last_became_active":"2024-04-25T00:14:06.856685+0000","last_became_peered":"2024-04-25T00:14:06.856685+0000","last_unstale":"2024-04-25T00:14:09.306452+0000","last_undegraded":"2024-04-25T00:14:09.306452+0000","last_fullsized":"2024-04-25T00:14:09.306452+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:20.250053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305861+0000","last_change":"2024-04-25T00:14:01.408610+0000","last_active":"2024-04-25T00:14:09.305861+0000","last_peered":"2024-04-25T00:14:09.305861+0000","last_clean":"2024-04-25T00:14:09.305861+0000","last_became_active":"2024-04-25T00:14:01.408279+0000","last_became_peered":"2024-04-25T00:14:01.408279+0000","last_unstale":"2024-04-25T00:14:09.305861+0000","last_undegraded":"2024-04-25T00:14:09.305861+0000","last_fullsized":"2024-04-25T00:14:09.305861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:09.566769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305944+0000","last_change":"2024-04-25T00:14:01.832342+0000","last_active":"2024-04-25T00:14:09.305944+0000","last_peered":"2024-04-25T00:14:09.305944+0000","last_clean":"2024-04-25T00:14:09.305944+0000","last_became_active":"2024-04-25T00:14:01.831997+0000","last_became_peered":"2024-04-25T00:14:01.831997+0000","last_unstale":"2024-04-25T00:14:09.305944+0000","last_undegraded":"2024-04-25T00:14:09.305944+0000","last_fullsized":"2024-04-25T00:14:09.305944+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:12.335884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305855+0000","last_change":"2024-04-25T00:14:07.323628+0000","last_active":"2024-04-25T00:14:09.305855+0000","last_peered":"2024-04-25T00:14:09.305855+0000","last_clean":"2024-04-25T00:14:09.305855+0000","last_became_active":"2024-04-25T00:14:07.322996+0000","last_became_peered":"2024-04-25T00:14:07.322996+0000","last_unstale":"2024-04-25T00:14:09.305855+0000","last_undegraded":"2024-04-25T00:14:09.305855+0000","last_fullsized":"2024-04-25T00:14:09.305855+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:32.803216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:26.531006+0000","last_change":"2024-04-25T00:14:01.407715+0000","last_active":"2024-04-25T00:14:26.531006+0000","last_peered":"2024-04-25T00:14:26.531006+0000","last_clean":"2024-04-25T00:14:26.531006+0000","last_became_active":"2024-04-25T00:14:01.407439+0000","last_became_peered":"2024-04-25T00:14:01.407439+0000","last_unstale":"2024-04-25T00:14:26.531006+0000","last_undegraded":"2024-04-25T00:14:26.531006+0000","last_fullsized":"2024-04-25T00:14:26.531006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:33.550379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307062+0000","last_change":"2024-04-25T00:14:01.837733+0000","last_active":"2024-04-25T00:14:09.307062+0000","last_peered":"2024-04-25T00:14:09.307062+0000","last_clean":"2024-04-25T00:14:09.307062+0000","last_became_active":"2024-04-25T00:14:01.837395+0000","last_became_peered":"2024-04-25T00:14:01.837395+0000","last_unstale":"2024-04-25T00:14:09.307062+0000","last_undegraded":"2024-04-25T00:14:09.307062+0000","last_fullsized":"2024-04-25T00:14:09.307062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:50.186834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306774+0000","last_change":"2024-04-25T00:14:06.848700+0000","last_active":"2024-04-25T00:14:09.306774+0000","last_peered":"2024-04-25T00:14:09.306774+0000","last_clean":"2024-04-25T00:14:09.306774+0000","last_became_active":"2024-04-25T00:14:06.848252+0000","last_became_peered":"2024-04-25T00:14:06.848252+0000","last_unstale":"2024-04-25T00:14:09.306774+0000","last_undegraded":"2024-04-25T00:14:09.306774+0000","last_fullsized":"2024-04-25T00:14:09.306774+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:07.653668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:27.535458+0000","last_change":"2024-04-25T00:14:01.405785+0000","last_active":"2024-04-25T00:14:27.535458+0000","last_peered":"2024-04-25T00:14:27.535458+0000","last_clean":"2024-04-25T00:14:27.535458+0000","last_became_active":"2024-04-25T00:14:01.405475+0000","last_became_peered":"2024-04-25T00:14:01.405475+0000","last_unstale":"2024-04-25T00:14:27.535458+0000","last_undegraded":"2024-04-25T00:14:27.535458+0000","last_fullsized":"2024-04-25T00:14:27.535458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:32.487795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307808+0000","last_change":"2024-04-25T00:14:01.832393+0000","last_active":"2024-04-25T00:14:09.307808+0000","last_peered":"2024-04-25T00:14:09.307808+0000","last_clean":"2024-04-25T00:14:09.307808+0000","last_became_active":"2024-04-25T00:14:01.832096+0000","last_became_peered":"2024-04-25T00:14:01.832096+0000","last_unstale":"2024-04-25T00:14:09.307808+0000","last_undegraded":"2024-04-25T00:14:09.307808+0000","last_fullsized":"2024-04-25T00:14:09.307808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:41.749627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306849+0000","last_change":"2024-04-25T00:14:06.855224+0000","last_active":"2024-04-25T00:14:09.306849+0000","last_peered":"2024-04-25T00:14:09.306849+0000","last_clean":"2024-04-25T00:14:09.306849+0000","last_became_active":"2024-04-25T00:14:06.854847+0000","last_became_peered":"2024-04-25T00:14:06.854847+0000","last_unstale":"2024-04-25T00:14:09.306849+0000","last_undegraded":"2024-04-25T00:14:09.306849+0000","last_fullsized":"2024-04-25T00:14:09.306849+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:13.520448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306976+0000","last_change":"2024-04-25T00:14:01.401145+0000","last_active":"2024-04-25T00:14:09.306976+0000","last_peered":"2024-04-25T00:14:09.306976+0000","last_clean":"2024-04-25T00:14:09.306976+0000","last_became_active":"2024-04-25T00:14:01.400807+0000","last_became_peered":"2024-04-25T00:14:01.400807+0000","last_unstale":"2024-04-25T00:14:09.306976+0000","last_undegraded":"2024-04-25T00:14:09.306976+0000","last_fullsized":"2024-04-25T00:14:09.306976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:52:04.674677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307497+0000","last_change":"2024-04-25T00:14:01.838033+0000","last_active":"2024-04-25T00:14:09.307497+0000","last_peered":"2024-04-25T00:14:09.307497+0000","last_clean":"2024-04-25T00:14:09.307497+0000","last_became_active":"2024-04-25T00:14:01.837657+0000","last_became_peered":"2024-04-25T00:14:01.837657+0000","last_unstale":"2024-04-25T00:14:09.307497+0000","last_undegraded":"2024-04-25T00:14:09.307497+0000","last_fullsized":"2024-04-25T00:14:09.307497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:51.895086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305593+0000","last_change":"2024-04-25T00:14:06.854885+0000","last_active":"2024-04-25T00:14:09.305593+0000","last_peered":"2024-04-25T00:14:09.305593+0000","last_clean":"2024-04-25T00:14:09.305593+0000","last_became_active":"2024-04-25T00:14:06.854200+0000","last_became_peered":"2024-04-25T00:14:06.854200+0000","last_unstale":"2024-04-25T00:14:09.305593+0000","last_undegraded":"2024-04-25T00:14:09.305593+0000","last_fullsized":"2024-04-25T00:14:09.305593+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:43.627742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306265+0000","last_change":"2024-04-25T00:14:01.403789+0000","last_active":"2024-04-25T00:14:09.306265+0000","last_peered":"2024-04-25T00:14:09.306265+0000","last_clean":"2024-04-25T00:14:09.306265+0000","last_became_active":"2024-04-25T00:14:01.403464+0000","last_became_peered":"2024-04-25T00:14:01.403464+0000","last_unstale":"2024-04-25T00:14:09.306265+0000","last_undegraded":"2024-04-25T00:14:09.306265+0000","last_fullsized":"2024-04-25T00:14:09.306265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:48.276536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307534+0000","last_change":"2024-04-25T00:14:01.839988+0000","last_active":"2024-04-25T00:14:09.307534+0000","last_peered":"2024-04-25T00:14:09.307534+0000","last_clean":"2024-04-25T00:14:09.307534+0000","last_became_active":"2024-04-25T00:14:01.839516+0000","last_became_peered":"2024-04-25T00:14:01.839516+0000","last_unstale":"2024-04-25T00:14:09.307534+0000","last_undegraded":"2024-04-25T00:14:09.307534+0000","last_fullsized":"2024-04-25T00:14:09.307534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:05.875127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306885+0000","last_change":"2024-04-25T00:14:01.828638+0000","last_active":"2024-04-25T00:14:09.306885+0000","last_peered":"2024-04-25T00:14:09.306885+0000","last_clean":"2024-04-25T00:14:09.306885+0000","last_became_active":"2024-04-25T00:14:01.828266+0000","last_became_peered":"2024-04-25T00:14:01.828266+0000","last_unstale":"2024-04-25T00:14:09.306885+0000","last_undegraded":"2024-04-25T00:14:09.306885+0000","last_fullsized":"2024-04-25T00:14:09.306885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:34:22.619522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306476+0000","last_change":"2024-04-25T00:14:01.828199+0000","last_active":"2024-04-25T00:14:09.306476+0000","last_peered":"2024-04-25T00:14:09.306476+0000","last_clean":"2024-04-25T00:14:09.306476+0000","last_became_active":"2024-04-25T00:14:01.827579+0000","last_became_peered":"2024-04-25T00:14:01.827579+0000","last_unstale":"2024-04-25T00:14:09.306476+0000","last_undegraded":"2024-04-25T00:14:09.306476+0000","last_fullsized":"2024-04-25T00:14:09.306476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:00.708928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305712+0000","last_change":"2024-04-25T00:14:01.825620+0000","last_active":"2024-04-25T00:14:09.305712+0000","last_peered":"2024-04-25T00:14:09.305712+0000","last_clean":"2024-04-25T00:14:09.305712+0000","last_became_active":"2024-04-25T00:14:01.825310+0000","last_became_peered":"2024-04-25T00:14:01.825310+0000","last_unstale":"2024-04-25T00:14:09.305712+0000","last_undegraded":"2024-04-25T00:14:09.305712+0000","last_fullsized":"2024-04-25T00:14:09.305712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:35.075355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306986+0000","last_change":"2024-04-25T00:14:01.836537+0000","last_active":"2024-04-25T00:14:09.306986+0000","last_peered":"2024-04-25T00:14:09.306986+0000","last_clean":"2024-04-25T00:14:09.306986+0000","last_became_active":"2024-04-25T00:14:01.836234+0000","last_became_peered":"2024-04-25T00:14:01.836234+0000","last_unstale":"2024-04-25T00:14:09.306986+0000","last_undegraded":"2024-04-25T00:14:09.306986+0000","last_fullsized":"2024-04-25T00:14:09.306986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:15.037498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306802+0000","last_change":"2024-04-25T00:14:01.825937+0000","last_active":"2024-04-25T00:14:09.306802+0000","last_peered":"2024-04-25T00:14:09.306802+0000","last_clean":"2024-04-25T00:14:09.306802+0000","last_became_active":"2024-04-25T00:14:01.825605+0000","last_became_peered":"2024-04-25T00:14:01.825605+0000","last_unstale":"2024-04-25T00:14:09.306802+0000","last_undegraded":"2024-04-25T00:14:09.306802+0000","last_fullsized":"2024-04-25T00:14:09.306802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:06.918000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307561+0000","last_change":"2024-04-25T00:14:01.830004+0000","last_active":"2024-04-25T00:14:09.307561+0000","last_peered":"2024-04-25T00:14:09.307561+0000","last_clean":"2024-04-25T00:14:09.307561+0000","last_became_active":"2024-04-25T00:14:01.829588+0000","last_became_peered":"2024-04-25T00:14:01.829588+0000","last_unstale":"2024-04-25T00:14:09.307561+0000","last_undegraded":"2024-04-25T00:14:09.307561+0000","last_fullsized":"2024-04-25T00:14:09.307561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:08.005995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306368+0000","last_change":"2024-04-25T00:14:01.825762+0000","last_active":"2024-04-25T00:14:09.306368+0000","last_peered":"2024-04-25T00:14:09.306368+0000","last_clean":"2024-04-25T00:14:09.306368+0000","last_became_active":"2024-04-25T00:14:01.825471+0000","last_became_peered":"2024-04-25T00:14:01.825471+0000","last_unstale":"2024-04-25T00:14:09.306368+0000","last_undegraded":"2024-04-25T00:14:09.306368+0000","last_fullsized":"2024-04-25T00:14:09.306368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:33.716875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306695+0000","last_change":"2024-04-25T00:14:01.836103+0000","last_active":"2024-04-25T00:14:09.306695+0000","last_peered":"2024-04-25T00:14:09.306695+0000","last_clean":"2024-04-25T00:14:09.306695+0000","last_became_active":"2024-04-25T00:14:01.834591+0000","last_became_peered":"2024-04-25T00:14:01.834591+0000","last_unstale":"2024-04-25T00:14:09.306695+0000","last_undegraded":"2024-04-25T00:14:09.306695+0000","last_fullsized":"2024-04-25T00:14:09.306695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:55.130978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306293+0000","last_change":"2024-04-25T00:14:01.833179+0000","last_active":"2024-04-25T00:14:09.306293+0000","last_peered":"2024-04-25T00:14:09.306293+0000","last_clean":"2024-04-25T00:14:09.306293+0000","last_became_active":"2024-04-25T00:14:01.832856+0000","last_became_peered":"2024-04-25T00:14:01.832856+0000","last_unstale":"2024-04-25T00:14:09.306293+0000","last_undegraded":"2024-04-25T00:14:09.306293+0000","last_fullsized":"2024-04-25T00:14:09.306293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:07.153356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305673+0000","last_change":"2024-04-25T00:14:01.834751+0000","last_active":"2024-04-25T00:14:09.305673+0000","last_peered":"2024-04-25T00:14:09.305673+0000","last_clean":"2024-04-25T00:14:09.305673+0000","last_became_active":"2024-04-25T00:14:01.834413+0000","last_became_peered":"2024-04-25T00:14:01.834413+0000","last_unstale":"2024-04-25T00:14:09.305673+0000","last_undegraded":"2024-04-25T00:14:09.305673+0000","last_fullsized":"2024-04-25T00:14:09.305673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:52.341565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307026+0000","last_change":"2024-04-25T00:14:01.832944+0000","last_active":"2024-04-25T00:14:09.307026+0000","last_peered":"2024-04-25T00:14:09.307026+0000","last_clean":"2024-04-25T00:14:09.307026+0000","last_became_active":"2024-04-25T00:14:01.832394+0000","last_became_peered":"2024-04-25T00:14:01.832394+0000","last_unstale":"2024-04-25T00:14:09.307026+0000","last_undegraded":"2024-04-25T00:14:09.307026+0000","last_fullsized":"2024-04-25T00:14:09.307026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:26.957786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305963+0000","last_change":"2024-04-25T00:14:01.831492+0000","last_active":"2024-04-25T00:14:09.305963+0000","last_peered":"2024-04-25T00:14:09.305963+0000","last_clean":"2024-04-25T00:14:09.305963+0000","last_became_active":"2024-04-25T00:14:01.829557+0000","last_became_peered":"2024-04-25T00:14:01.829557+0000","last_unstale":"2024-04-25T00:14:09.305963+0000","last_undegraded":"2024-04-25T00:14:09.305963+0000","last_fullsized":"2024-04-25T00:14:09.305963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:45.116188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307577+0000","last_change":"2024-04-25T00:14:01.839323+0000","last_active":"2024-04-25T00:14:09.307577+0000","last_peered":"2024-04-25T00:14:09.307577+0000","last_clean":"2024-04-25T00:14:09.307577+0000","last_became_active":"2024-04-25T00:14:01.838999+0000","last_became_peered":"2024-04-25T00:14:01.838999+0000","last_unstale":"2024-04-25T00:14:09.307577+0000","last_undegraded":"2024-04-25T00:14:09.307577+0000","last_fullsized":"2024-04-25T00:14:09.307577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:20.652123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":44081,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":25,"num_read_kb":43,"num_write":142,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":107795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125,"ondisk_log_size":125,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":497852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612631,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36992,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708160,"statfs":{"total":95995035648,"available":95957155840,"internally_reserved":0,"allocated":1695744,"data_stored":1130269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776157,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":506862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037794,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":498590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299430,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1630208,"data_stored":1090012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462956,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":497942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":538323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":498546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":510035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149754,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":498136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888118,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40504,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704648,"statfs":{"total":95995035648,"available":95953559552,"internally_reserved":0,"allocated":1032192,"data_stored":498496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724592,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1642496,"data_stored":1089696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":40471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":18739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":42049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:14:31.218 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:14:31.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.349+0000 7f69d011b640 1 -- 172.21.15.107:0/1929273947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69c8102240 msgr2=0x7f69c8108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:31.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.349+0000 7f69d011b640 1 --2- 172.21.15.107:0/1929273947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69c8102240 0x7f69c8108ad0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009900 tx=0x7f69c402a410 comp rx=0 tx=0).stop 2024-04-25T00:14:31.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.349+0000 7f69d011b640 1 -- 172.21.15.107:0/1929273947 shutdown_connections 2024-04-25T00:14:31.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.349+0000 7f69d011b640 1 --2- 172.21.15.107:0/1929273947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69c8102240 0x7f69c8108ad0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.349+0000 7f69d011b640 1 --2- 172.21.15.107:0/1929273947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69c8101880 0x7f69c8101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.349+0000 7f69d011b640 1 --2- 172.21.15.107:0/1929273947 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69c8100690 0x7f69c8100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.349+0000 7f69d011b640 1 -- 172.21.15.107:0/1929273947 >> 172.21.15.107:0/1929273947 conn(0x7f69c80fbc40 msgr2=0x7f69c80fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:31.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.349+0000 7f69d011b640 1 -- 172.21.15.107:0/1929273947 shutdown_connections 2024-04-25T00:14:31.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.349+0000 7f69d011b640 1 -- 172.21.15.107:0/1929273947 wait complete. 2024-04-25T00:14:31.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.349+0000 7f69d011b640 1 Processor -- start 2024-04-25T00:14:31.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.349+0000 7f69d011b640 1 -- start start 2024-04-25T00:14:31.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.349+0000 7f69d011b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69c8100690 0x7f69c806a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:31.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.349+0000 7f69d011b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69c8101880 0x7f69c819c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:31.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.349+0000 7f69d011b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69c8102240 0x7f69c819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:31.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.349+0000 7f69d011b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f69c810c380 con 0x7f69c8100690 2024-04-25T00:14:31.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.349+0000 7f69d011b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f69c810c200 con 0x7f69c8102240 2024-04-25T00:14:31.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.349+0000 7f69d011b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f69c810c500 con 0x7f69c8101880 2024-04-25T00:14:31.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.349+0000 7f69cdeb7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69c8100690 0x7f69c806a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:31.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.350+0000 7f69cdeb7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69c8100690 0x7f69c806a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38160/0 (socket says 172.21.15.107:38160) 2024-04-25T00:14:31.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.350+0000 7f69cdeb7640 1 -- 172.21.15.107:0/2177032852 learned_addr learned my addr 172.21.15.107:0/2177032852 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:31.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.350+0000 7f69cdeb7640 1 -- 172.21.15.107:0/2177032852 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69c8101880 msgr2=0x7f69c819c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:31.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.350+0000 7f69cdeb7640 1 --2- 172.21.15.107:0/2177032852 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69c8101880 0x7f69c819c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.350+0000 7f69cdeb7640 1 -- 172.21.15.107:0/2177032852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69c8102240 msgr2=0x7f69c819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:31.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.350+0000 7f69cdeb7640 1 --2- 172.21.15.107:0/2177032852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69c8102240 0x7f69c819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.350+0000 7f69cdeb7640 1 -- 172.21.15.107:0/2177032852 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c819f690 con 0x7f69c8100690 2024-04-25T00:14:31.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.350+0000 7f69cdeb7640 1 --2- 172.21.15.107:0/2177032852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69c8100690 0x7f69c806a950 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f69c000b9c0 tx=0x7f69c000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:31.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.350+0000 7f69beffd640 1 -- 172.21.15.107:0/2177032852 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c0019070 con 0x7f69c8100690 2024-04-25T00:14:31.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.350+0000 7f69beffd640 1 -- 172.21.15.107:0/2177032852 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69c000cba0 con 0x7f69c8100690 2024-04-25T00:14:31.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.350+0000 7f69d011b640 1 -- 172.21.15.107:0/2177032852 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c819f920 con 0x7f69c8100690 2024-04-25T00:14:31.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.350+0000 7f69beffd640 1 -- 172.21.15.107:0/2177032852 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69c0012640 con 0x7f69c8100690 2024-04-25T00:14:31.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.350+0000 7f69d011b640 1 -- 172.21.15.107:0/2177032852 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c819fef0 con 0x7f69c8100690 2024-04-25T00:14:31.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.351+0000 7f69d011b640 1 -- 172.21.15.107:0/2177032852 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69c81078e0 con 0x7f69c8100690 2024-04-25T00:14:31.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.352+0000 7f69beffd640 1 -- 172.21.15.107:0/2177032852 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f69c001c070 con 0x7f69c8100690 2024-04-25T00:14:31.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.353+0000 7f69beffd640 1 --2- 172.21.15.107:0/2177032852 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f69b40a2ca0 0x7f69b40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:31.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.353+0000 7f69cd6b6640 1 --2- 172.21.15.107:0/2177032852 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f69b40a2ca0 0x7f69b40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:31.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.353+0000 7f69beffd640 1 -- 172.21.15.107:0/2177032852 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f69c00ce950 con 0x7f69c8100690 2024-04-25T00:14:31.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.353+0000 7f69cd6b6640 1 --2- 172.21.15.107:0/2177032852 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f69b40a2ca0 0x7f69b40a5160 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f69c806a6c0 tx=0x7f69b8005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:31.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.357+0000 7f69beffd640 1 -- 172.21.15.107:0/2177032852 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f69c009b620 con 0x7f69c8100690 2024-04-25T00:14:31.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.532+0000 7f69d011b640 1 -- 172.21.15.107:0/2177032852 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f69c8061b80 con 0x7f69b40a2ca0 2024-04-25T00:14:31.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.542+0000 7f69beffd640 1 -- 172.21.15.107:0/2177032852 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340310 (secure 0 0 0) 0x7f69c8061b80 con 0x7f69b40a2ca0 2024-04-25T00:14:31.547 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:14:31.549 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-04-25T00:14:31.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.545+0000 7f69d011b640 1 -- 172.21.15.107:0/2177032852 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f69b40a2ca0 msgr2=0x7f69b40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:31.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.545+0000 7f69d011b640 1 --2- 172.21.15.107:0/2177032852 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f69b40a2ca0 0x7f69b40a5160 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f69c806a6c0 tx=0x7f69b8005c70 comp rx=0 tx=0).stop 2024-04-25T00:14:31.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.545+0000 7f69d011b640 1 -- 172.21.15.107:0/2177032852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69c8100690 msgr2=0x7f69c806a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:31.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.545+0000 7f69d011b640 1 --2- 172.21.15.107:0/2177032852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69c8100690 0x7f69c806a950 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f69c000b9c0 tx=0x7f69c000be80 comp rx=0 tx=0).stop 2024-04-25T00:14:31.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.545+0000 7f69d011b640 1 -- 172.21.15.107:0/2177032852 shutdown_connections 2024-04-25T00:14:31.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.545+0000 7f69d011b640 1 --2- 172.21.15.107:0/2177032852 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f69b40a2ca0 0x7f69b40a5160 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.545+0000 7f69d011b640 1 --2- 172.21.15.107:0/2177032852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f69c8102240 0x7f69c819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.545+0000 7f69d011b640 1 --2- 172.21.15.107:0/2177032852 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69c8101880 0x7f69c819c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.545+0000 7f69d011b640 1 --2- 172.21.15.107:0/2177032852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69c8100690 0x7f69c806a950 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.545+0000 7f69d011b640 1 -- 172.21.15.107:0/2177032852 >> 172.21.15.107:0/2177032852 conn(0x7f69c80fbc40 msgr2=0x7f69c8104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:31.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.546+0000 7f69d011b640 1 -- 172.21.15.107:0/2177032852 shutdown_connections 2024-04-25T00:14:31.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:31.546+0000 7f69d011b640 1 -- 172.21.15.107:0/2177032852 wait complete. 2024-04-25T00:14:31.558 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-04-25T00:14:30.925042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":634449,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":161,"num_read_kb":162,"num_write":323,"num_write_kb":3022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":262,"ondisk_log_size":262,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388120,"kb_used_data":14000,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553704,"statfs":{"total":1151940427776,"available":1151542992896,"internally_reserved":0,"allocated":14336000,"data_stored":7854759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":39582,"num_objects":38,"num_object_clones":0,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":0,"num_read":25,"num_read_kb":43,"num_write":95,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003400"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307882+0000","last_change":"2024-04-25T00:14:01.835737+0000","last_active":"2024-04-25T00:14:09.307882+0000","last_peered":"2024-04-25T00:14:09.307882+0000","last_clean":"2024-04-25T00:14:09.307882+0000","last_became_active":"2024-04-25T00:14:01.835428+0000","last_became_peered":"2024-04-25T00:14:01.835428+0000","last_unstale":"2024-04-25T00:14:09.307882+0000","last_undegraded":"2024-04-25T00:14:09.307882+0000","last_fullsized":"2024-04-25T00:14:09.307882+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:33.974656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306530+0000","last_change":"2024-04-25T00:14:01.837321+0000","last_active":"2024-04-25T00:14:09.306530+0000","last_peered":"2024-04-25T00:14:09.306530+0000","last_clean":"2024-04-25T00:14:09.306530+0000","last_became_active":"2024-04-25T00:14:01.836762+0000","last_became_peered":"2024-04-25T00:14:01.836762+0000","last_unstale":"2024-04-25T00:14:09.306530+0000","last_undegraded":"2024-04-25T00:14:09.306530+0000","last_fullsized":"2024-04-25T00:14:09.306530+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:34:17.342967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305939+0000","last_change":"2024-04-25T00:14:01.834504+0000","last_active":"2024-04-25T00:14:09.305939+0000","last_peered":"2024-04-25T00:14:09.305939+0000","last_clean":"2024-04-25T00:14:09.305939+0000","last_became_active":"2024-04-25T00:14:01.834182+0000","last_became_peered":"2024-04-25T00:14:01.834182+0000","last_unstale":"2024-04-25T00:14:09.305939+0000","last_undegraded":"2024-04-25T00:14:09.305939+0000","last_fullsized":"2024-04-25T00:14:09.305939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:12.991170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306554+0000","last_change":"2024-04-25T00:14:01.838257+0000","last_active":"2024-04-25T00:14:09.306554+0000","last_peered":"2024-04-25T00:14:09.306554+0000","last_clean":"2024-04-25T00:14:09.306554+0000","last_became_active":"2024-04-25T00:14:01.837909+0000","last_became_peered":"2024-04-25T00:14:01.837909+0000","last_unstale":"2024-04-25T00:14:09.306554+0000","last_undegraded":"2024-04-25T00:14:09.306554+0000","last_fullsized":"2024-04-25T00:14:09.306554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:14.287154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843302+0000","last_change":"2024-04-25T00:14:01.834173+0000","last_active":"2024-04-25T00:14:08.843302+0000","last_peered":"2024-04-25T00:14:08.843302+0000","last_clean":"2024-04-25T00:14:08.843302+0000","last_became_active":"2024-04-25T00:14:01.833835+0000","last_became_peered":"2024-04-25T00:14:01.833835+0000","last_unstale":"2024-04-25T00:14:08.843302+0000","last_undegraded":"2024-04-25T00:14:08.843302+0000","last_fullsized":"2024-04-25T00:14:08.843302+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:30.046629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843535+0000","last_change":"2024-04-25T00:14:01.832113+0000","last_active":"2024-04-25T00:14:08.843535+0000","last_peered":"2024-04-25T00:14:08.843535+0000","last_clean":"2024-04-25T00:14:08.843535+0000","last_became_active":"2024-04-25T00:14:01.831800+0000","last_became_peered":"2024-04-25T00:14:01.831800+0000","last_unstale":"2024-04-25T00:14:08.843535+0000","last_undegraded":"2024-04-25T00:14:08.843535+0000","last_fullsized":"2024-04-25T00:14:08.843535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:54.469994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305996+0000","last_change":"2024-04-25T00:14:01.839166+0000","last_active":"2024-04-25T00:14:09.305996+0000","last_peered":"2024-04-25T00:14:09.305996+0000","last_clean":"2024-04-25T00:14:09.305996+0000","last_became_active":"2024-04-25T00:14:01.838851+0000","last_became_peered":"2024-04-25T00:14:01.838851+0000","last_unstale":"2024-04-25T00:14:09.305996+0000","last_undegraded":"2024-04-25T00:14:09.305996+0000","last_fullsized":"2024-04-25T00:14:09.305996+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:39:48.125363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305229+0000","last_change":"2024-04-25T00:14:01.828230+0000","last_active":"2024-04-25T00:14:09.305229+0000","last_peered":"2024-04-25T00:14:09.305229+0000","last_clean":"2024-04-25T00:14:09.305229+0000","last_became_active":"2024-04-25T00:14:01.827847+0000","last_became_peered":"2024-04-25T00:14:01.827847+0000","last_unstale":"2024-04-25T00:14:09.305229+0000","last_undegraded":"2024-04-25T00:14:09.305229+0000","last_fullsized":"2024-04-25T00:14:09.305229+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:33.906668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307268+0000","last_change":"2024-04-25T00:14:01.829803+0000","last_active":"2024-04-25T00:14:09.307268+0000","last_peered":"2024-04-25T00:14:09.307268+0000","last_clean":"2024-04-25T00:14:09.307268+0000","last_became_active":"2024-04-25T00:14:01.827761+0000","last_became_peered":"2024-04-25T00:14:01.827761+0000","last_unstale":"2024-04-25T00:14:09.307268+0000","last_undegraded":"2024-04-25T00:14:09.307268+0000","last_fullsized":"2024-04-25T00:14:09.307268+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:38.862505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843989+0000","last_change":"2024-04-25T00:14:01.834269+0000","last_active":"2024-04-25T00:14:08.843989+0000","last_peered":"2024-04-25T00:14:08.843989+0000","last_clean":"2024-04-25T00:14:08.843989+0000","last_became_active":"2024-04-25T00:14:01.833964+0000","last_became_peered":"2024-04-25T00:14:01.833964+0000","last_unstale":"2024-04-25T00:14:08.843989+0000","last_undegraded":"2024-04-25T00:14:08.843989+0000","last_fullsized":"2024-04-25T00:14:08.843989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:29.501513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307776+0000","last_change":"2024-04-25T00:14:01.839510+0000","last_active":"2024-04-25T00:14:09.307776+0000","last_peered":"2024-04-25T00:14:09.307776+0000","last_clean":"2024-04-25T00:14:09.307776+0000","last_became_active":"2024-04-25T00:14:01.839000+0000","last_became_peered":"2024-04-25T00:14:01.839000+0000","last_unstale":"2024-04-25T00:14:09.307776+0000","last_undegraded":"2024-04-25T00:14:09.307776+0000","last_fullsized":"2024-04-25T00:14:09.307776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:20.809277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306399+0000","last_change":"2024-04-25T00:14:01.835821+0000","last_active":"2024-04-25T00:14:09.306399+0000","last_peered":"2024-04-25T00:14:09.306399+0000","last_clean":"2024-04-25T00:14:09.306399+0000","last_became_active":"2024-04-25T00:14:01.835525+0000","last_became_peered":"2024-04-25T00:14:01.835525+0000","last_unstale":"2024-04-25T00:14:09.306399+0000","last_undegraded":"2024-04-25T00:14:09.306399+0000","last_fullsized":"2024-04-25T00:14:09.306399+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:17.082562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305921+0000","last_change":"2024-04-25T00:14:01.831564+0000","last_active":"2024-04-25T00:14:09.305921+0000","last_peered":"2024-04-25T00:14:09.305921+0000","last_clean":"2024-04-25T00:14:09.305921+0000","last_became_active":"2024-04-25T00:14:01.828539+0000","last_became_peered":"2024-04-25T00:14:01.828539+0000","last_unstale":"2024-04-25T00:14:09.305921+0000","last_undegraded":"2024-04-25T00:14:09.305921+0000","last_fullsized":"2024-04-25T00:14:09.305921+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:41:52.098299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305009+0000","last_change":"2024-04-25T00:14:01.835815+0000","last_active":"2024-04-25T00:14:09.305009+0000","last_peered":"2024-04-25T00:14:09.305009+0000","last_clean":"2024-04-25T00:14:09.305009+0000","last_became_active":"2024-04-25T00:14:01.835498+0000","last_became_peered":"2024-04-25T00:14:01.835498+0000","last_unstale":"2024-04-25T00:14:09.305009+0000","last_undegraded":"2024-04-25T00:14:09.305009+0000","last_fullsized":"2024-04-25T00:14:09.305009+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:14:02.914149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305256+0000","last_change":"2024-04-25T00:14:01.839542+0000","last_active":"2024-04-25T00:14:09.305256+0000","last_peered":"2024-04-25T00:14:09.305256+0000","last_clean":"2024-04-25T00:14:09.305256+0000","last_became_active":"2024-04-25T00:14:01.839234+0000","last_became_peered":"2024-04-25T00:14:01.839234+0000","last_unstale":"2024-04-25T00:14:09.305256+0000","last_undegraded":"2024-04-25T00:14:09.305256+0000","last_fullsized":"2024-04-25T00:14:09.305256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:00.437311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305618+0000","last_change":"2024-04-25T00:14:01.832265+0000","last_active":"2024-04-25T00:14:09.305618+0000","last_peered":"2024-04-25T00:14:09.305618+0000","last_clean":"2024-04-25T00:14:09.305618+0000","last_became_active":"2024-04-25T00:14:01.831963+0000","last_became_peered":"2024-04-25T00:14:01.831963+0000","last_unstale":"2024-04-25T00:14:09.305618+0000","last_undegraded":"2024-04-25T00:14:09.305618+0000","last_fullsized":"2024-04-25T00:14:09.305618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:55.402265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307658+0000","last_change":"2024-04-25T00:14:01.835444+0000","last_active":"2024-04-25T00:14:09.307658+0000","last_peered":"2024-04-25T00:14:09.307658+0000","last_clean":"2024-04-25T00:14:09.307658+0000","last_became_active":"2024-04-25T00:14:01.835126+0000","last_became_peered":"2024-04-25T00:14:01.835126+0000","last_unstale":"2024-04-25T00:14:09.307658+0000","last_undegraded":"2024-04-25T00:14:09.307658+0000","last_fullsized":"2024-04-25T00:14:09.307658+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:48.997924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843003+0000","last_change":"2024-04-25T00:14:01.834982+0000","last_active":"2024-04-25T00:14:08.843003+0000","last_peered":"2024-04-25T00:14:08.843003+0000","last_clean":"2024-04-25T00:14:08.843003+0000","last_became_active":"2024-04-25T00:14:01.834671+0000","last_became_peered":"2024-04-25T00:14:01.834671+0000","last_unstale":"2024-04-25T00:14:08.843003+0000","last_undegraded":"2024-04-25T00:14:08.843003+0000","last_fullsized":"2024-04-25T00:14:08.843003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:28.254408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306725+0000","last_change":"2024-04-25T00:14:01.824432+0000","last_active":"2024-04-25T00:14:09.306725+0000","last_peered":"2024-04-25T00:14:09.306725+0000","last_clean":"2024-04-25T00:14:09.306725+0000","last_became_active":"2024-04-25T00:14:01.823963+0000","last_became_peered":"2024-04-25T00:14:01.823963+0000","last_unstale":"2024-04-25T00:14:09.306725+0000","last_undegraded":"2024-04-25T00:14:09.306725+0000","last_fullsized":"2024-04-25T00:14:09.306725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:35.880932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306217+0000","last_change":"2024-04-25T00:14:06.861892+0000","last_active":"2024-04-25T00:14:09.306217+0000","last_peered":"2024-04-25T00:14:09.306217+0000","last_clean":"2024-04-25T00:14:09.306217+0000","last_became_active":"2024-04-25T00:14:06.861470+0000","last_became_peered":"2024-04-25T00:14:06.861470+0000","last_unstale":"2024-04-25T00:14:09.306217+0000","last_undegraded":"2024-04-25T00:14:09.306217+0000","last_fullsized":"2024-04-25T00:14:09.306217+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:06.192954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:27.536314+0000","last_change":"2024-04-25T00:14:01.412535+0000","last_active":"2024-04-25T00:14:27.536314+0000","last_peered":"2024-04-25T00:14:27.536314+0000","last_clean":"2024-04-25T00:14:27.536314+0000","last_became_active":"2024-04-25T00:14:01.412206+0000","last_became_peered":"2024-04-25T00:14:01.412206+0000","last_unstale":"2024-04-25T00:14:27.536314+0000","last_undegraded":"2024-04-25T00:14:27.536314+0000","last_fullsized":"2024-04-25T00:14:27.536314+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:40.893987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306310+0000","last_change":"2024-04-25T00:14:01.840814+0000","last_active":"2024-04-25T00:14:09.306310+0000","last_peered":"2024-04-25T00:14:09.306310+0000","last_clean":"2024-04-25T00:14:09.306310+0000","last_became_active":"2024-04-25T00:14:01.840179+0000","last_became_peered":"2024-04-25T00:14:01.840179+0000","last_unstale":"2024-04-25T00:14:09.306310+0000","last_undegraded":"2024-04-25T00:14:09.306310+0000","last_fullsized":"2024-04-25T00:14:09.306310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:27.383918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305817+0000","last_change":"2024-04-25T00:14:06.857877+0000","last_active":"2024-04-25T00:14:09.305817+0000","last_peered":"2024-04-25T00:14:09.305817+0000","last_clean":"2024-04-25T00:14:09.305817+0000","last_became_active":"2024-04-25T00:14:06.857420+0000","last_became_peered":"2024-04-25T00:14:06.857420+0000","last_unstale":"2024-04-25T00:14:09.305817+0000","last_undegraded":"2024-04-25T00:14:09.305817+0000","last_fullsized":"2024-04-25T00:14:09.305817+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:54.424024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306621+0000","last_change":"2024-04-25T00:14:01.416560+0000","last_active":"2024-04-25T00:14:09.306621+0000","last_peered":"2024-04-25T00:14:09.306621+0000","last_clean":"2024-04-25T00:14:09.306621+0000","last_became_active":"2024-04-25T00:14:01.416246+0000","last_became_peered":"2024-04-25T00:14:01.416246+0000","last_unstale":"2024-04-25T00:14:09.306621+0000","last_undegraded":"2024-04-25T00:14:09.306621+0000","last_fullsized":"2024-04-25T00:14:09.306621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:03:47.938452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306055+0000","last_change":"2024-04-25T00:14:01.832607+0000","last_active":"2024-04-25T00:14:09.306055+0000","last_peered":"2024-04-25T00:14:09.306055+0000","last_clean":"2024-04-25T00:14:09.306055+0000","last_became_active":"2024-04-25T00:14:01.832300+0000","last_became_peered":"2024-04-25T00:14:01.832300+0000","last_unstale":"2024-04-25T00:14:09.306055+0000","last_undegraded":"2024-04-25T00:14:09.306055+0000","last_fullsized":"2024-04-25T00:14:09.306055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:22.045171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306107+0000","last_change":"2024-04-25T00:14:06.863642+0000","last_active":"2024-04-25T00:14:09.306107+0000","last_peered":"2024-04-25T00:14:09.306107+0000","last_clean":"2024-04-25T00:14:09.306107+0000","last_became_active":"2024-04-25T00:14:06.863284+0000","last_became_peered":"2024-04-25T00:14:06.863284+0000","last_unstale":"2024-04-25T00:14:09.306107+0000","last_undegraded":"2024-04-25T00:14:09.306107+0000","last_fullsized":"2024-04-25T00:14:09.306107+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:20.818625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:27.533684+0000","last_change":"2024-04-25T00:14:01.411874+0000","last_active":"2024-04-25T00:14:27.533684+0000","last_peered":"2024-04-25T00:14:27.533684+0000","last_clean":"2024-04-25T00:14:27.533684+0000","last_became_active":"2024-04-25T00:14:01.409568+0000","last_became_peered":"2024-04-25T00:14:01.409568+0000","last_unstale":"2024-04-25T00:14:27.533684+0000","last_undegraded":"2024-04-25T00:14:27.533684+0000","last_fullsized":"2024-04-25T00:14:27.533684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:22.415631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306790+0000","last_change":"2024-04-25T00:14:01.822471+0000","last_active":"2024-04-25T00:14:09.306790+0000","last_peered":"2024-04-25T00:14:09.306790+0000","last_clean":"2024-04-25T00:14:09.306790+0000","last_became_active":"2024-04-25T00:14:01.822181+0000","last_became_peered":"2024-04-25T00:14:01.822181+0000","last_unstale":"2024-04-25T00:14:09.306790+0000","last_undegraded":"2024-04-25T00:14:09.306790+0000","last_fullsized":"2024-04-25T00:14:09.306790+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:44.857036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305663+0000","last_change":"2024-04-25T00:14:06.856987+0000","last_active":"2024-04-25T00:14:09.305663+0000","last_peered":"2024-04-25T00:14:09.305663+0000","last_clean":"2024-04-25T00:14:09.305663+0000","last_became_active":"2024-04-25T00:14:06.856053+0000","last_became_peered":"2024-04-25T00:14:06.856053+0000","last_unstale":"2024-04-25T00:14:09.305663+0000","last_undegraded":"2024-04-25T00:14:09.305663+0000","last_fullsized":"2024-04-25T00:14:09.305663+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:16.829969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843469+0000","last_change":"2024-04-25T00:14:01.410156+0000","last_active":"2024-04-25T00:14:08.843469+0000","last_peered":"2024-04-25T00:14:08.843469+0000","last_clean":"2024-04-25T00:14:08.843469+0000","last_became_active":"2024-04-25T00:14:01.409809+0000","last_became_peered":"2024-04-25T00:14:01.409809+0000","last_unstale":"2024-04-25T00:14:08.843469+0000","last_undegraded":"2024-04-25T00:14:08.843469+0000","last_fullsized":"2024-04-25T00:14:08.843469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:43.160419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306881+0000","last_change":"2024-04-25T00:14:01.836090+0000","last_active":"2024-04-25T00:14:09.306881+0000","last_peered":"2024-04-25T00:14:09.306881+0000","last_clean":"2024-04-25T00:14:09.306881+0000","last_became_active":"2024-04-25T00:14:01.835733+0000","last_became_peered":"2024-04-25T00:14:01.835733+0000","last_unstale":"2024-04-25T00:14:09.306881+0000","last_undegraded":"2024-04-25T00:14:09.306881+0000","last_fullsized":"2024-04-25T00:14:09.306881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:02.368893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306553+0000","last_change":"2024-04-25T00:14:06.851496+0000","last_active":"2024-04-25T00:14:09.306553+0000","last_peered":"2024-04-25T00:14:09.306553+0000","last_clean":"2024-04-25T00:14:09.306553+0000","last_became_active":"2024-04-25T00:14:06.851007+0000","last_became_peered":"2024-04-25T00:14:06.851007+0000","last_unstale":"2024-04-25T00:14:09.306553+0000","last_undegraded":"2024-04-25T00:14:09.306553+0000","last_fullsized":"2024-04-25T00:14:09.306553+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:50.543348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'5","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:26.530014+0000","last_change":"2024-04-25T00:14:01.414760+0000","last_active":"2024-04-25T00:14:26.530014+0000","last_peered":"2024-04-25T00:14:26.530014+0000","last_clean":"2024-04-25T00:14:26.530014+0000","last_became_active":"2024-04-25T00:14:01.414482+0000","last_became_peered":"2024-04-25T00:14:01.414482+0000","last_unstale":"2024-04-25T00:14:26.530014+0000","last_undegraded":"2024-04-25T00:14:26.530014+0000","last_fullsized":"2024-04-25T00:14:26.530014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:32.631933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843310+0000","last_change":"2024-04-25T00:14:01.832516+0000","last_active":"2024-04-25T00:14:08.843310+0000","last_peered":"2024-04-25T00:14:08.843310+0000","last_clean":"2024-04-25T00:14:08.843310+0000","last_became_active":"2024-04-25T00:14:01.832106+0000","last_became_peered":"2024-04-25T00:14:01.832106+0000","last_unstale":"2024-04-25T00:14:08.843310+0000","last_undegraded":"2024-04-25T00:14:08.843310+0000","last_fullsized":"2024-04-25T00:14:08.843310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:55.590486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306193+0000","last_change":"2024-04-25T00:14:06.861984+0000","last_active":"2024-04-25T00:14:09.306193+0000","last_peered":"2024-04-25T00:14:09.306193+0000","last_clean":"2024-04-25T00:14:09.306193+0000","last_became_active":"2024-04-25T00:14:06.861418+0000","last_became_peered":"2024-04-25T00:14:06.861418+0000","last_unstale":"2024-04-25T00:14:09.306193+0000","last_undegraded":"2024-04-25T00:14:09.306193+0000","last_fullsized":"2024-04-25T00:14:09.306193+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:27.559704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:27.534036+0000","last_change":"2024-04-25T00:14:01.407648+0000","last_active":"2024-04-25T00:14:27.534036+0000","last_peered":"2024-04-25T00:14:27.534036+0000","last_clean":"2024-04-25T00:14:27.534036+0000","last_became_active":"2024-04-25T00:14:01.407318+0000","last_became_peered":"2024-04-25T00:14:01.407318+0000","last_unstale":"2024-04-25T00:14:27.534036+0000","last_undegraded":"2024-04-25T00:14:27.534036+0000","last_fullsized":"2024-04-25T00:14:27.534036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:08:10.942814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306945+0000","last_change":"2024-04-25T00:14:01.833073+0000","last_active":"2024-04-25T00:14:09.306945+0000","last_peered":"2024-04-25T00:14:09.306945+0000","last_clean":"2024-04-25T00:14:09.306945+0000","last_became_active":"2024-04-25T00:14:01.832545+0000","last_became_peered":"2024-04-25T00:14:01.832545+0000","last_unstale":"2024-04-25T00:14:09.306945+0000","last_undegraded":"2024-04-25T00:14:09.306945+0000","last_fullsized":"2024-04-25T00:14:09.306945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:39.772781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307729+0000","last_change":"2024-04-25T00:14:06.862011+0000","last_active":"2024-04-25T00:14:09.307729+0000","last_peered":"2024-04-25T00:14:09.307729+0000","last_clean":"2024-04-25T00:14:09.307729+0000","last_became_active":"2024-04-25T00:14:06.861544+0000","last_became_peered":"2024-04-25T00:14:06.861544+0000","last_unstale":"2024-04-25T00:14:09.307729+0000","last_undegraded":"2024-04-25T00:14:09.307729+0000","last_fullsized":"2024-04-25T00:14:09.307729+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:37.736266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:27.535635+0000","last_change":"2024-04-25T00:14:01.397747+0000","last_active":"2024-04-25T00:14:27.535635+0000","last_peered":"2024-04-25T00:14:27.535635+0000","last_clean":"2024-04-25T00:14:27.535635+0000","last_became_active":"2024-04-25T00:14:01.397446+0000","last_became_peered":"2024-04-25T00:14:01.397446+0000","last_unstale":"2024-04-25T00:14:27.535635+0000","last_undegraded":"2024-04-25T00:14:27.535635+0000","last_fullsized":"2024-04-25T00:14:27.535635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:27.240034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305461+0000","last_change":"2024-04-25T00:14:01.833128+0000","last_active":"2024-04-25T00:14:09.305461+0000","last_peered":"2024-04-25T00:14:09.305461+0000","last_clean":"2024-04-25T00:14:09.305461+0000","last_became_active":"2024-04-25T00:14:01.832780+0000","last_became_peered":"2024-04-25T00:14:01.832780+0000","last_unstale":"2024-04-25T00:14:09.305461+0000","last_undegraded":"2024-04-25T00:14:09.305461+0000","last_fullsized":"2024-04-25T00:14:09.305461+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.023737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306433+0000","last_change":"2024-04-25T00:14:06.860436+0000","last_active":"2024-04-25T00:14:09.306433+0000","last_peered":"2024-04-25T00:14:09.306433+0000","last_clean":"2024-04-25T00:14:09.306433+0000","last_became_active":"2024-04-25T00:14:06.860047+0000","last_became_peered":"2024-04-25T00:14:06.860047+0000","last_unstale":"2024-04-25T00:14:09.306433+0000","last_undegraded":"2024-04-25T00:14:09.306433+0000","last_fullsized":"2024-04-25T00:14:09.306433+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:08:41.246549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306233+0000","last_change":"2024-04-25T00:14:01.415939+0000","last_active":"2024-04-25T00:14:09.306233+0000","last_peered":"2024-04-25T00:14:09.306233+0000","last_clean":"2024-04-25T00:14:09.306233+0000","last_became_active":"2024-04-25T00:14:01.415513+0000","last_became_peered":"2024-04-25T00:14:01.415513+0000","last_unstale":"2024-04-25T00:14:09.306233+0000","last_undegraded":"2024-04-25T00:14:09.306233+0000","last_fullsized":"2024-04-25T00:14:09.306233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:57:23.588868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.842554+0000","last_change":"2024-04-25T00:14:01.838793+0000","last_active":"2024-04-25T00:14:08.842554+0000","last_peered":"2024-04-25T00:14:08.842554+0000","last_clean":"2024-04-25T00:14:08.842554+0000","last_became_active":"2024-04-25T00:14:01.838453+0000","last_became_peered":"2024-04-25T00:14:01.838453+0000","last_unstale":"2024-04-25T00:14:08.842554+0000","last_undegraded":"2024-04-25T00:14:08.842554+0000","last_fullsized":"2024-04-25T00:14:08.842554+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:18.674009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305319+0000","last_change":"2024-04-25T00:14:06.856431+0000","last_active":"2024-04-25T00:14:09.305319+0000","last_peered":"2024-04-25T00:14:09.305319+0000","last_clean":"2024-04-25T00:14:09.305319+0000","last_became_active":"2024-04-25T00:14:06.856047+0000","last_became_peered":"2024-04-25T00:14:06.856047+0000","last_unstale":"2024-04-25T00:14:09.305319+0000","last_undegraded":"2024-04-25T00:14:09.305319+0000","last_fullsized":"2024-04-25T00:14:09.305319+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:20.646673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"92'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:28.538057+0000","last_change":"2024-04-25T00:14:01.399054+0000","last_active":"2024-04-25T00:14:28.538057+0000","last_peered":"2024-04-25T00:14:28.538057+0000","last_clean":"2024-04-25T00:14:28.538057+0000","last_became_active":"2024-04-25T00:14:01.398711+0000","last_became_peered":"2024-04-25T00:14:01.398711+0000","last_unstale":"2024-04-25T00:14:28.538057+0000","last_undegraded":"2024-04-25T00:14:28.538057+0000","last_fullsized":"2024-04-25T00:14:28.538057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:26.434014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307281+0000","last_change":"2024-04-25T00:14:01.829132+0000","last_active":"2024-04-25T00:14:09.307281+0000","last_peered":"2024-04-25T00:14:09.307281+0000","last_clean":"2024-04-25T00:14:09.307281+0000","last_became_active":"2024-04-25T00:14:01.828446+0000","last_became_peered":"2024-04-25T00:14:01.828446+0000","last_unstale":"2024-04-25T00:14:09.307281+0000","last_undegraded":"2024-04-25T00:14:09.307281+0000","last_fullsized":"2024-04-25T00:14:09.307281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:35.788478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.842621+0000","last_change":"2024-04-25T00:14:06.861884+0000","last_active":"2024-04-25T00:14:08.842621+0000","last_peered":"2024-04-25T00:14:08.842621+0000","last_clean":"2024-04-25T00:14:08.842621+0000","last_became_active":"2024-04-25T00:14:06.861413+0000","last_became_peered":"2024-04-25T00:14:06.861413+0000","last_unstale":"2024-04-25T00:14:08.842621+0000","last_undegraded":"2024-04-25T00:14:08.842621+0000","last_fullsized":"2024-04-25T00:14:08.842621+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:53.876308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:26.530769+0000","last_change":"2024-04-25T00:14:01.410372+0000","last_active":"2024-04-25T00:14:26.530769+0000","last_peered":"2024-04-25T00:14:26.530769+0000","last_clean":"2024-04-25T00:14:26.530769+0000","last_became_active":"2024-04-25T00:14:01.410074+0000","last_became_peered":"2024-04-25T00:14:01.410074+0000","last_unstale":"2024-04-25T00:14:26.530769+0000","last_undegraded":"2024-04-25T00:14:26.530769+0000","last_fullsized":"2024-04-25T00:14:26.530769+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:00.964580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306219+0000","last_change":"2024-04-25T00:14:01.839723+0000","last_active":"2024-04-25T00:14:09.306219+0000","last_peered":"2024-04-25T00:14:09.306219+0000","last_clean":"2024-04-25T00:14:09.306219+0000","last_became_active":"2024-04-25T00:14:01.839199+0000","last_became_peered":"2024-04-25T00:14:01.839199+0000","last_unstale":"2024-04-25T00:14:09.306219+0000","last_undegraded":"2024-04-25T00:14:09.306219+0000","last_fullsized":"2024-04-25T00:14:09.306219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:55:32.087459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307445+0000","last_change":"2024-04-25T00:14:06.857525+0000","last_active":"2024-04-25T00:14:09.307445+0000","last_peered":"2024-04-25T00:14:09.307445+0000","last_clean":"2024-04-25T00:14:09.307445+0000","last_became_active":"2024-04-25T00:14:06.856886+0000","last_became_peered":"2024-04-25T00:14:06.856886+0000","last_unstale":"2024-04-25T00:14:09.307445+0000","last_undegraded":"2024-04-25T00:14:09.307445+0000","last_fullsized":"2024-04-25T00:14:09.307445+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:29:05.366093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306893+0000","last_change":"2024-04-25T00:14:01.401128+0000","last_active":"2024-04-25T00:14:09.306893+0000","last_peered":"2024-04-25T00:14:09.306893+0000","last_clean":"2024-04-25T00:14:09.306893+0000","last_became_active":"2024-04-25T00:14:01.400125+0000","last_became_peered":"2024-04-25T00:14:01.400125+0000","last_unstale":"2024-04-25T00:14:09.306893+0000","last_undegraded":"2024-04-25T00:14:09.306893+0000","last_fullsized":"2024-04-25T00:14:09.306893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:38:43.787855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307142+0000","last_change":"2024-04-25T00:14:01.830649+0000","last_active":"2024-04-25T00:14:09.307142+0000","last_peered":"2024-04-25T00:14:09.307142+0000","last_clean":"2024-04-25T00:14:09.307142+0000","last_became_active":"2024-04-25T00:14:01.830253+0000","last_became_peered":"2024-04-25T00:14:01.830253+0000","last_unstale":"2024-04-25T00:14:09.307142+0000","last_undegraded":"2024-04-25T00:14:09.307142+0000","last_fullsized":"2024-04-25T00:14:09.307142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:37.474833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.842752+0000","last_change":"2024-04-25T00:14:06.861275+0000","last_active":"2024-04-25T00:14:08.842752+0000","last_peered":"2024-04-25T00:14:08.842752+0000","last_clean":"2024-04-25T00:14:08.842752+0000","last_became_active":"2024-04-25T00:14:06.860891+0000","last_became_peered":"2024-04-25T00:14:06.860891+0000","last_unstale":"2024-04-25T00:14:08.842752+0000","last_undegraded":"2024-04-25T00:14:08.842752+0000","last_fullsized":"2024-04-25T00:14:08.842752+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:38.685837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"92'23","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:29.910371+0000","last_change":"2024-04-25T00:14:01.404706+0000","last_active":"2024-04-25T00:14:29.910371+0000","last_peered":"2024-04-25T00:14:29.910371+0000","last_clean":"2024-04-25T00:14:29.910371+0000","last_became_active":"2024-04-25T00:14:01.404231+0000","last_became_peered":"2024-04-25T00:14:01.404231+0000","last_unstale":"2024-04-25T00:14:29.910371+0000","last_undegraded":"2024-04-25T00:14:29.910371+0000","last_fullsized":"2024-04-25T00:14:29.910371+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:36.437491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40471,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":26,"num_write":24,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843685+0000","last_change":"2024-04-25T00:14:01.828312+0000","last_active":"2024-04-25T00:14:08.843685+0000","last_peered":"2024-04-25T00:14:08.843685+0000","last_clean":"2024-04-25T00:14:08.843685+0000","last_became_active":"2024-04-25T00:14:01.827947+0000","last_became_peered":"2024-04-25T00:14:01.827947+0000","last_unstale":"2024-04-25T00:14:08.843685+0000","last_undegraded":"2024-04-25T00:14:08.843685+0000","last_fullsized":"2024-04-25T00:14:08.843685+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.782796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307156+0000","last_change":"2024-04-25T00:14:06.854379+0000","last_active":"2024-04-25T00:14:09.307156+0000","last_peered":"2024-04-25T00:14:09.307156+0000","last_clean":"2024-04-25T00:14:09.307156+0000","last_became_active":"2024-04-25T00:14:06.853307+0000","last_became_peered":"2024-04-25T00:14:06.853307+0000","last_unstale":"2024-04-25T00:14:09.307156+0000","last_undegraded":"2024-04-25T00:14:09.307156+0000","last_fullsized":"2024-04-25T00:14:09.307156+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:24:11.804189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:27.534855+0000","last_change":"2024-04-25T00:14:01.410395+0000","last_active":"2024-04-25T00:14:27.534855+0000","last_peered":"2024-04-25T00:14:27.534855+0000","last_clean":"2024-04-25T00:14:27.534855+0000","last_became_active":"2024-04-25T00:14:01.410039+0000","last_became_peered":"2024-04-25T00:14:01.410039+0000","last_unstale":"2024-04-25T00:14:27.534855+0000","last_undegraded":"2024-04-25T00:14:27.534855+0000","last_fullsized":"2024-04-25T00:14:27.534855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:29.496519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305998+0000","last_change":"2024-04-25T00:14:01.828403+0000","last_active":"2024-04-25T00:14:09.305998+0000","last_peered":"2024-04-25T00:14:09.305998+0000","last_clean":"2024-04-25T00:14:09.305998+0000","last_became_active":"2024-04-25T00:14:01.828107+0000","last_became_peered":"2024-04-25T00:14:01.828107+0000","last_unstale":"2024-04-25T00:14:09.305998+0000","last_undegraded":"2024-04-25T00:14:09.305998+0000","last_fullsized":"2024-04-25T00:14:09.305998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:29.417639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307322+0000","last_change":"2024-04-25T00:14:06.861845+0000","last_active":"2024-04-25T00:14:09.307322+0000","last_peered":"2024-04-25T00:14:09.307322+0000","last_clean":"2024-04-25T00:14:09.307322+0000","last_became_active":"2024-04-25T00:14:06.861451+0000","last_became_peered":"2024-04-25T00:14:06.861451+0000","last_unstale":"2024-04-25T00:14:09.307322+0000","last_undegraded":"2024-04-25T00:14:09.307322+0000","last_fullsized":"2024-04-25T00:14:09.307322+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:21.473628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:26.529581+0000","last_change":"2024-04-25T00:14:01.414406+0000","last_active":"2024-04-25T00:14:26.529581+0000","last_peered":"2024-04-25T00:14:26.529581+0000","last_clean":"2024-04-25T00:14:26.529581+0000","last_became_active":"2024-04-25T00:14:01.414085+0000","last_became_peered":"2024-04-25T00:14:01.414085+0000","last_unstale":"2024-04-25T00:14:26.529581+0000","last_undegraded":"2024-04-25T00:14:26.529581+0000","last_fullsized":"2024-04-25T00:14:26.529581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:22.565371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307277+0000","last_change":"2024-04-25T00:14:01.833652+0000","last_active":"2024-04-25T00:14:09.307277+0000","last_peered":"2024-04-25T00:14:09.307277+0000","last_clean":"2024-04-25T00:14:09.307277+0000","last_became_active":"2024-04-25T00:14:01.833307+0000","last_became_peered":"2024-04-25T00:14:01.833307+0000","last_unstale":"2024-04-25T00:14:09.307277+0000","last_undegraded":"2024-04-25T00:14:09.307277+0000","last_fullsized":"2024-04-25T00:14:09.307277+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:55:41.261994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843067+0000","last_change":"2024-04-25T00:14:06.860167+0000","last_active":"2024-04-25T00:14:08.843067+0000","last_peered":"2024-04-25T00:14:08.843067+0000","last_clean":"2024-04-25T00:14:08.843067+0000","last_became_active":"2024-04-25T00:14:06.858651+0000","last_became_peered":"2024-04-25T00:14:06.858651+0000","last_unstale":"2024-04-25T00:14:08.843067+0000","last_undegraded":"2024-04-25T00:14:08.843067+0000","last_fullsized":"2024-04-25T00:14:08.843067+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:11.775434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'3","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:28.536263+0000","last_change":"2024-04-25T00:14:01.406267+0000","last_active":"2024-04-25T00:14:28.536263+0000","last_peered":"2024-04-25T00:14:28.536263+0000","last_clean":"2024-04-25T00:14:28.536263+0000","last_became_active":"2024-04-25T00:14:01.405985+0000","last_became_peered":"2024-04-25T00:14:01.405985+0000","last_unstale":"2024-04-25T00:14:28.536263+0000","last_undegraded":"2024-04-25T00:14:28.536263+0000","last_fullsized":"2024-04-25T00:14:28.536263+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:17.253805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306649+0000","last_change":"2024-04-25T00:14:01.838019+0000","last_active":"2024-04-25T00:14:09.306649+0000","last_peered":"2024-04-25T00:14:09.306649+0000","last_clean":"2024-04-25T00:14:09.306649+0000","last_became_active":"2024-04-25T00:14:01.837691+0000","last_became_peered":"2024-04-25T00:14:01.837691+0000","last_unstale":"2024-04-25T00:14:09.306649+0000","last_undegraded":"2024-04-25T00:14:09.306649+0000","last_fullsized":"2024-04-25T00:14:09.306649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:26.677039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306288+0000","last_change":"2024-04-25T00:14:06.857061+0000","last_active":"2024-04-25T00:14:09.306288+0000","last_peered":"2024-04-25T00:14:09.306288+0000","last_clean":"2024-04-25T00:14:09.306288+0000","last_became_active":"2024-04-25T00:14:06.856678+0000","last_became_peered":"2024-04-25T00:14:06.856678+0000","last_unstale":"2024-04-25T00:14:09.306288+0000","last_undegraded":"2024-04-25T00:14:09.306288+0000","last_fullsized":"2024-04-25T00:14:09.306288+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:36:52.730861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:27.537047+0000","last_change":"2024-04-25T00:14:01.408385+0000","last_active":"2024-04-25T00:14:27.537047+0000","last_peered":"2024-04-25T00:14:27.537047+0000","last_clean":"2024-04-25T00:14:27.537047+0000","last_became_active":"2024-04-25T00:14:01.408049+0000","last_became_peered":"2024-04-25T00:14:01.408049+0000","last_unstale":"2024-04-25T00:14:27.537047+0000","last_undegraded":"2024-04-25T00:14:27.537047+0000","last_fullsized":"2024-04-25T00:14:27.537047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:19.765467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305958+0000","last_change":"2024-04-25T00:14:01.835113+0000","last_active":"2024-04-25T00:14:09.305958+0000","last_peered":"2024-04-25T00:14:09.305958+0000","last_clean":"2024-04-25T00:14:09.305958+0000","last_became_active":"2024-04-25T00:14:01.834782+0000","last_became_peered":"2024-04-25T00:14:01.834782+0000","last_unstale":"2024-04-25T00:14:09.305958+0000","last_undegraded":"2024-04-25T00:14:09.305958+0000","last_fullsized":"2024-04-25T00:14:09.305958+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:25.595476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305839+0000","last_change":"2024-04-25T00:14:06.854192+0000","last_active":"2024-04-25T00:14:09.305839+0000","last_peered":"2024-04-25T00:14:09.305839+0000","last_clean":"2024-04-25T00:14:09.305839+0000","last_became_active":"2024-04-25T00:14:06.853325+0000","last_became_peered":"2024-04-25T00:14:06.853325+0000","last_unstale":"2024-04-25T00:14:09.305839+0000","last_undegraded":"2024-04-25T00:14:09.305839+0000","last_fullsized":"2024-04-25T00:14:09.305839+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:15.308169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:28.530098+0000","last_change":"2024-04-25T00:14:01.407247+0000","last_active":"2024-04-25T00:14:28.530098+0000","last_peered":"2024-04-25T00:14:28.530098+0000","last_clean":"2024-04-25T00:14:28.530098+0000","last_became_active":"2024-04-25T00:14:01.406891+0000","last_became_peered":"2024-04-25T00:14:01.406891+0000","last_unstale":"2024-04-25T00:14:28.530098+0000","last_undegraded":"2024-04-25T00:14:28.530098+0000","last_fullsized":"2024-04-25T00:14:28.530098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:43.294562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307381+0000","last_change":"2024-04-25T00:14:01.829364+0000","last_active":"2024-04-25T00:14:09.307381+0000","last_peered":"2024-04-25T00:14:09.307381+0000","last_clean":"2024-04-25T00:14:09.307381+0000","last_became_active":"2024-04-25T00:14:01.829087+0000","last_became_peered":"2024-04-25T00:14:01.829087+0000","last_unstale":"2024-04-25T00:14:09.307381+0000","last_undegraded":"2024-04-25T00:14:09.307381+0000","last_fullsized":"2024-04-25T00:14:09.307381+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:17.651815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843462+0000","last_change":"2024-04-25T00:14:07.323442+0000","last_active":"2024-04-25T00:14:08.843462+0000","last_peered":"2024-04-25T00:14:08.843462+0000","last_clean":"2024-04-25T00:14:08.843462+0000","last_became_active":"2024-04-25T00:14:07.322943+0000","last_became_peered":"2024-04-25T00:14:07.322943+0000","last_unstale":"2024-04-25T00:14:08.843462+0000","last_undegraded":"2024-04-25T00:14:08.843462+0000","last_fullsized":"2024-04-25T00:14:08.843462+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:49.166240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843045+0000","last_change":"2024-04-25T00:14:01.410936+0000","last_active":"2024-04-25T00:14:08.843045+0000","last_peered":"2024-04-25T00:14:08.843045+0000","last_clean":"2024-04-25T00:14:08.843045+0000","last_became_active":"2024-04-25T00:14:01.410321+0000","last_became_peered":"2024-04-25T00:14:01.410321+0000","last_unstale":"2024-04-25T00:14:08.843045+0000","last_undegraded":"2024-04-25T00:14:08.843045+0000","last_fullsized":"2024-04-25T00:14:08.843045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:44.603199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843739+0000","last_change":"2024-04-25T00:14:01.830842+0000","last_active":"2024-04-25T00:14:08.843739+0000","last_peered":"2024-04-25T00:14:08.843739+0000","last_clean":"2024-04-25T00:14:08.843739+0000","last_became_active":"2024-04-25T00:14:01.830543+0000","last_became_peered":"2024-04-25T00:14:01.830543+0000","last_unstale":"2024-04-25T00:14:08.843739+0000","last_undegraded":"2024-04-25T00:14:08.843739+0000","last_fullsized":"2024-04-25T00:14:08.843739+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:59.897133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306636+0000","last_change":"2024-04-25T00:14:06.856841+0000","last_active":"2024-04-25T00:14:09.306636+0000","last_peered":"2024-04-25T00:14:09.306636+0000","last_clean":"2024-04-25T00:14:09.306636+0000","last_became_active":"2024-04-25T00:14:06.856416+0000","last_became_peered":"2024-04-25T00:14:06.856416+0000","last_unstale":"2024-04-25T00:14:09.306636+0000","last_undegraded":"2024-04-25T00:14:09.306636+0000","last_fullsized":"2024-04-25T00:14:09.306636+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:27.181175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307062+0000","last_change":"2024-04-25T00:14:01.409627+0000","last_active":"2024-04-25T00:14:09.307062+0000","last_peered":"2024-04-25T00:14:09.307062+0000","last_clean":"2024-04-25T00:14:09.307062+0000","last_became_active":"2024-04-25T00:14:01.409304+0000","last_became_peered":"2024-04-25T00:14:01.409304+0000","last_unstale":"2024-04-25T00:14:09.307062+0000","last_undegraded":"2024-04-25T00:14:09.307062+0000","last_fullsized":"2024-04-25T00:14:09.307062+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:27.242004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843827+0000","last_change":"2024-04-25T00:14:01.829544+0000","last_active":"2024-04-25T00:14:08.843827+0000","last_peered":"2024-04-25T00:14:08.843827+0000","last_clean":"2024-04-25T00:14:08.843827+0000","last_became_active":"2024-04-25T00:14:01.829233+0000","last_became_peered":"2024-04-25T00:14:01.829233+0000","last_unstale":"2024-04-25T00:14:08.843827+0000","last_undegraded":"2024-04-25T00:14:08.843827+0000","last_fullsized":"2024-04-25T00:14:08.843827+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:16.102041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306455+0000","last_change":"2024-04-25T00:14:07.323213+0000","last_active":"2024-04-25T00:14:09.306455+0000","last_peered":"2024-04-25T00:14:09.306455+0000","last_clean":"2024-04-25T00:14:09.306455+0000","last_became_active":"2024-04-25T00:14:07.322763+0000","last_became_peered":"2024-04-25T00:14:07.322763+0000","last_unstale":"2024-04-25T00:14:09.306455+0000","last_undegraded":"2024-04-25T00:14:09.306455+0000","last_fullsized":"2024-04-25T00:14:09.306455+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:45.269314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306420+0000","last_change":"2024-04-25T00:14:01.832836+0000","last_active":"2024-04-25T00:14:09.306420+0000","last_peered":"2024-04-25T00:14:09.306420+0000","last_clean":"2024-04-25T00:14:09.306420+0000","last_became_active":"2024-04-25T00:14:01.830652+0000","last_became_peered":"2024-04-25T00:14:01.830652+0000","last_unstale":"2024-04-25T00:14:09.306420+0000","last_undegraded":"2024-04-25T00:14:09.306420+0000","last_fullsized":"2024-04-25T00:14:09.306420+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:10:21.803200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:27.535012+0000","last_change":"2024-04-25T00:14:01.397066+0000","last_active":"2024-04-25T00:14:27.535012+0000","last_peered":"2024-04-25T00:14:27.535012+0000","last_clean":"2024-04-25T00:14:27.535012+0000","last_became_active":"2024-04-25T00:14:01.396728+0000","last_became_peered":"2024-04-25T00:14:01.396728+0000","last_unstale":"2024-04-25T00:14:27.535012+0000","last_undegraded":"2024-04-25T00:14:27.535012+0000","last_fullsized":"2024-04-25T00:14:27.535012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:28.334329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307671+0000","last_change":"2024-04-25T00:14:06.853039+0000","last_active":"2024-04-25T00:14:09.307671+0000","last_peered":"2024-04-25T00:14:09.307671+0000","last_clean":"2024-04-25T00:14:09.307671+0000","last_became_active":"2024-04-25T00:14:06.852568+0000","last_became_peered":"2024-04-25T00:14:06.852568+0000","last_unstale":"2024-04-25T00:14:09.307671+0000","last_undegraded":"2024-04-25T00:14:09.307671+0000","last_fullsized":"2024-04-25T00:14:09.307671+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:59.604160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305343+0000","last_change":"2024-04-25T00:14:01.840315+0000","last_active":"2024-04-25T00:14:09.305343+0000","last_peered":"2024-04-25T00:14:09.305343+0000","last_clean":"2024-04-25T00:14:09.305343+0000","last_became_active":"2024-04-25T00:14:01.839537+0000","last_became_peered":"2024-04-25T00:14:01.839537+0000","last_unstale":"2024-04-25T00:14:09.305343+0000","last_undegraded":"2024-04-25T00:14:09.305343+0000","last_fullsized":"2024-04-25T00:14:09.305343+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:34.126164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:28.534457+0000","last_change":"2024-04-25T00:14:01.412493+0000","last_active":"2024-04-25T00:14:28.534457+0000","last_peered":"2024-04-25T00:14:28.534457+0000","last_clean":"2024-04-25T00:14:28.534457+0000","last_became_active":"2024-04-25T00:14:01.412133+0000","last_became_peered":"2024-04-25T00:14:01.412133+0000","last_unstale":"2024-04-25T00:14:28.534457+0000","last_undegraded":"2024-04-25T00:14:28.534457+0000","last_fullsized":"2024-04-25T00:14:28.534457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:44.933728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":360,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306136+0000","last_change":"2024-04-25T00:12:49.169294+0000","last_active":"2024-04-25T00:14:09.306136+0000","last_peered":"2024-04-25T00:14:09.306136+0000","last_clean":"2024-04-25T00:14:09.306136+0000","last_became_active":"2024-04-25T00:12:49.168783+0000","last_became_peered":"2024-04-25T00:12:49.168783+0000","last_unstale":"2024-04-25T00:14:09.306136+0000","last_undegraded":"2024-04-25T00:14:09.306136+0000","last_fullsized":"2024-04-25T00:14:09.306136+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:25.202280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:25.202280+0000","last_clean_scrub_stamp":"2024-04-25T00:09:25.202280+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:04.713986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306639+0000","last_change":"2024-04-25T00:14:07.325530+0000","last_active":"2024-04-25T00:14:09.306639+0000","last_peered":"2024-04-25T00:14:09.306639+0000","last_clean":"2024-04-25T00:14:09.306639+0000","last_became_active":"2024-04-25T00:14:07.325152+0000","last_became_peered":"2024-04-25T00:14:07.325152+0000","last_unstale":"2024-04-25T00:14:09.306639+0000","last_undegraded":"2024-04-25T00:14:09.306639+0000","last_fullsized":"2024-04-25T00:14:09.306639+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:22:48.136526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305666+0000","last_change":"2024-04-25T00:14:01.826418+0000","last_active":"2024-04-25T00:14:09.305666+0000","last_peered":"2024-04-25T00:14:09.305666+0000","last_clean":"2024-04-25T00:14:09.305666+0000","last_became_active":"2024-04-25T00:14:01.825783+0000","last_became_peered":"2024-04-25T00:14:01.825783+0000","last_unstale":"2024-04-25T00:14:09.305666+0000","last_undegraded":"2024-04-25T00:14:09.305666+0000","last_fullsized":"2024-04-25T00:14:09.305666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:05.981359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:27.534416+0000","last_change":"2024-04-25T00:14:01.417022+0000","last_active":"2024-04-25T00:14:27.534416+0000","last_peered":"2024-04-25T00:14:27.534416+0000","last_clean":"2024-04-25T00:14:27.534416+0000","last_became_active":"2024-04-25T00:14:01.416804+0000","last_became_peered":"2024-04-25T00:14:01.416804+0000","last_unstale":"2024-04-25T00:14:27.534416+0000","last_undegraded":"2024-04-25T00:14:27.534416+0000","last_fullsized":"2024-04-25T00:14:27.534416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:31.561242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307500+0000","last_change":"2024-04-25T00:14:06.856500+0000","last_active":"2024-04-25T00:14:09.307500+0000","last_peered":"2024-04-25T00:14:09.307500+0000","last_clean":"2024-04-25T00:14:09.307500+0000","last_became_active":"2024-04-25T00:14:06.856077+0000","last_became_peered":"2024-04-25T00:14:06.856077+0000","last_unstale":"2024-04-25T00:14:09.307500+0000","last_undegraded":"2024-04-25T00:14:09.307500+0000","last_fullsized":"2024-04-25T00:14:09.307500+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:10:04.129533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:26.529730+0000","last_change":"2024-04-25T00:14:01.414599+0000","last_active":"2024-04-25T00:14:26.529730+0000","last_peered":"2024-04-25T00:14:26.529730+0000","last_clean":"2024-04-25T00:14:26.529730+0000","last_became_active":"2024-04-25T00:14:01.413796+0000","last_became_peered":"2024-04-25T00:14:01.413796+0000","last_unstale":"2024-04-25T00:14:26.529730+0000","last_undegraded":"2024-04-25T00:14:26.529730+0000","last_fullsized":"2024-04-25T00:14:26.529730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:44.981848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306903+0000","last_change":"2024-04-25T00:14:01.827718+0000","last_active":"2024-04-25T00:14:09.306903+0000","last_peered":"2024-04-25T00:14:09.306903+0000","last_clean":"2024-04-25T00:14:09.306903+0000","last_became_active":"2024-04-25T00:14:01.827410+0000","last_became_peered":"2024-04-25T00:14:01.827410+0000","last_unstale":"2024-04-25T00:14:09.306903+0000","last_undegraded":"2024-04-25T00:14:09.306903+0000","last_fullsized":"2024-04-25T00:14:09.306903+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:32.428137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305389+0000","last_change":"2024-04-25T00:14:07.323304+0000","last_active":"2024-04-25T00:14:09.305389+0000","last_peered":"2024-04-25T00:14:09.305389+0000","last_clean":"2024-04-25T00:14:09.305389+0000","last_became_active":"2024-04-25T00:14:07.322852+0000","last_became_peered":"2024-04-25T00:14:07.322852+0000","last_unstale":"2024-04-25T00:14:09.305389+0000","last_undegraded":"2024-04-25T00:14:09.305389+0000","last_fullsized":"2024-04-25T00:14:09.305389+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:35.173934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:27.535521+0000","last_change":"2024-04-25T00:14:01.413209+0000","last_active":"2024-04-25T00:14:27.535521+0000","last_peered":"2024-04-25T00:14:27.535521+0000","last_clean":"2024-04-25T00:14:27.535521+0000","last_became_active":"2024-04-25T00:14:01.412892+0000","last_became_peered":"2024-04-25T00:14:01.412892+0000","last_unstale":"2024-04-25T00:14:27.535521+0000","last_undegraded":"2024-04-25T00:14:27.535521+0000","last_fullsized":"2024-04-25T00:14:27.535521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:26.152962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306583+0000","last_change":"2024-04-25T00:14:01.834372+0000","last_active":"2024-04-25T00:14:09.306583+0000","last_peered":"2024-04-25T00:14:09.306583+0000","last_clean":"2024-04-25T00:14:09.306583+0000","last_became_active":"2024-04-25T00:14:01.834062+0000","last_became_peered":"2024-04-25T00:14:01.834062+0000","last_unstale":"2024-04-25T00:14:09.306583+0000","last_undegraded":"2024-04-25T00:14:09.306583+0000","last_fullsized":"2024-04-25T00:14:09.306583+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:05.580604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305626+0000","last_change":"2024-04-25T00:14:06.852303+0000","last_active":"2024-04-25T00:14:09.305626+0000","last_peered":"2024-04-25T00:14:09.305626+0000","last_clean":"2024-04-25T00:14:09.305626+0000","last_became_active":"2024-04-25T00:14:06.850957+0000","last_became_peered":"2024-04-25T00:14:06.850957+0000","last_unstale":"2024-04-25T00:14:09.305626+0000","last_undegraded":"2024-04-25T00:14:09.305626+0000","last_fullsized":"2024-04-25T00:14:09.305626+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:47.208880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:26.529886+0000","last_change":"2024-04-25T00:14:01.405538+0000","last_active":"2024-04-25T00:14:26.529886+0000","last_peered":"2024-04-25T00:14:26.529886+0000","last_clean":"2024-04-25T00:14:26.529886+0000","last_became_active":"2024-04-25T00:14:01.405155+0000","last_became_peered":"2024-04-25T00:14:01.405155+0000","last_unstale":"2024-04-25T00:14:26.529886+0000","last_undegraded":"2024-04-25T00:14:26.529886+0000","last_fullsized":"2024-04-25T00:14:26.529886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:20.864753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306618+0000","last_change":"2024-04-25T00:14:01.837058+0000","last_active":"2024-04-25T00:14:09.306618+0000","last_peered":"2024-04-25T00:14:09.306618+0000","last_clean":"2024-04-25T00:14:09.306618+0000","last_became_active":"2024-04-25T00:14:01.835838+0000","last_became_peered":"2024-04-25T00:14:01.835838+0000","last_unstale":"2024-04-25T00:14:09.306618+0000","last_undegraded":"2024-04-25T00:14:09.306618+0000","last_fullsized":"2024-04-25T00:14:09.306618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:21.696448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306347+0000","last_change":"2024-04-25T00:14:07.323870+0000","last_active":"2024-04-25T00:14:09.306347+0000","last_peered":"2024-04-25T00:14:09.306347+0000","last_clean":"2024-04-25T00:14:09.306347+0000","last_became_active":"2024-04-25T00:14:07.323356+0000","last_became_peered":"2024-04-25T00:14:07.323356+0000","last_unstale":"2024-04-25T00:14:09.306347+0000","last_undegraded":"2024-04-25T00:14:09.306347+0000","last_fullsized":"2024-04-25T00:14:09.306347+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:35.160618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:26.528549+0000","last_change":"2024-04-25T00:14:01.409453+0000","last_active":"2024-04-25T00:14:26.528549+0000","last_peered":"2024-04-25T00:14:26.528549+0000","last_clean":"2024-04-25T00:14:26.528549+0000","last_became_active":"2024-04-25T00:14:01.409164+0000","last_became_peered":"2024-04-25T00:14:01.409164+0000","last_unstale":"2024-04-25T00:14:26.528549+0000","last_undegraded":"2024-04-25T00:14:26.528549+0000","last_fullsized":"2024-04-25T00:14:26.528549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:23.266090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843228+0000","last_change":"2024-04-25T00:14:01.826790+0000","last_active":"2024-04-25T00:14:08.843228+0000","last_peered":"2024-04-25T00:14:08.843228+0000","last_clean":"2024-04-25T00:14:08.843228+0000","last_became_active":"2024-04-25T00:14:01.826457+0000","last_became_peered":"2024-04-25T00:14:01.826457+0000","last_unstale":"2024-04-25T00:14:08.843228+0000","last_undegraded":"2024-04-25T00:14:08.843228+0000","last_fullsized":"2024-04-25T00:14:08.843228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:15.356448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:08.843512+0000","last_change":"2024-04-25T00:14:06.863280+0000","last_active":"2024-04-25T00:14:08.843512+0000","last_peered":"2024-04-25T00:14:08.843512+0000","last_clean":"2024-04-25T00:14:08.843512+0000","last_became_active":"2024-04-25T00:14:06.862725+0000","last_became_peered":"2024-04-25T00:14:06.862725+0000","last_unstale":"2024-04-25T00:14:08.843512+0000","last_undegraded":"2024-04-25T00:14:08.843512+0000","last_fullsized":"2024-04-25T00:14:08.843512+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:12.791744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:26.529922+0000","last_change":"2024-04-25T00:14:01.414856+0000","last_active":"2024-04-25T00:14:26.529922+0000","last_peered":"2024-04-25T00:14:26.529922+0000","last_clean":"2024-04-25T00:14:26.529922+0000","last_became_active":"2024-04-25T00:14:01.414533+0000","last_became_peered":"2024-04-25T00:14:01.414533+0000","last_unstale":"2024-04-25T00:14:26.529922+0000","last_undegraded":"2024-04-25T00:14:26.529922+0000","last_fullsized":"2024-04-25T00:14:26.529922+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:01.090539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307007+0000","last_change":"2024-04-25T00:14:01.831452+0000","last_active":"2024-04-25T00:14:09.307007+0000","last_peered":"2024-04-25T00:14:09.307007+0000","last_clean":"2024-04-25T00:14:09.307007+0000","last_became_active":"2024-04-25T00:14:01.831151+0000","last_became_peered":"2024-04-25T00:14:01.831151+0000","last_unstale":"2024-04-25T00:14:09.307007+0000","last_undegraded":"2024-04-25T00:14:09.307007+0000","last_fullsized":"2024-04-25T00:14:09.307007+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:53:07.659508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306452+0000","last_change":"2024-04-25T00:14:06.857601+0000","last_active":"2024-04-25T00:14:09.306452+0000","last_peered":"2024-04-25T00:14:09.306452+0000","last_clean":"2024-04-25T00:14:09.306452+0000","last_became_active":"2024-04-25T00:14:06.856685+0000","last_became_peered":"2024-04-25T00:14:06.856685+0000","last_unstale":"2024-04-25T00:14:09.306452+0000","last_undegraded":"2024-04-25T00:14:09.306452+0000","last_fullsized":"2024-04-25T00:14:09.306452+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:20.250053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305861+0000","last_change":"2024-04-25T00:14:01.408610+0000","last_active":"2024-04-25T00:14:09.305861+0000","last_peered":"2024-04-25T00:14:09.305861+0000","last_clean":"2024-04-25T00:14:09.305861+0000","last_became_active":"2024-04-25T00:14:01.408279+0000","last_became_peered":"2024-04-25T00:14:01.408279+0000","last_unstale":"2024-04-25T00:14:09.305861+0000","last_undegraded":"2024-04-25T00:14:09.305861+0000","last_fullsized":"2024-04-25T00:14:09.305861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:09.566769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305944+0000","last_change":"2024-04-25T00:14:01.832342+0000","last_active":"2024-04-25T00:14:09.305944+0000","last_peered":"2024-04-25T00:14:09.305944+0000","last_clean":"2024-04-25T00:14:09.305944+0000","last_became_active":"2024-04-25T00:14:01.831997+0000","last_became_peered":"2024-04-25T00:14:01.831997+0000","last_unstale":"2024-04-25T00:14:09.305944+0000","last_undegraded":"2024-04-25T00:14:09.305944+0000","last_fullsized":"2024-04-25T00:14:09.305944+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:12.335884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305855+0000","last_change":"2024-04-25T00:14:07.323628+0000","last_active":"2024-04-25T00:14:09.305855+0000","last_peered":"2024-04-25T00:14:09.305855+0000","last_clean":"2024-04-25T00:14:09.305855+0000","last_became_active":"2024-04-25T00:14:07.322996+0000","last_became_peered":"2024-04-25T00:14:07.322996+0000","last_unstale":"2024-04-25T00:14:09.305855+0000","last_undegraded":"2024-04-25T00:14:09.305855+0000","last_fullsized":"2024-04-25T00:14:09.305855+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:32.803216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:26.531006+0000","last_change":"2024-04-25T00:14:01.407715+0000","last_active":"2024-04-25T00:14:26.531006+0000","last_peered":"2024-04-25T00:14:26.531006+0000","last_clean":"2024-04-25T00:14:26.531006+0000","last_became_active":"2024-04-25T00:14:01.407439+0000","last_became_peered":"2024-04-25T00:14:01.407439+0000","last_unstale":"2024-04-25T00:14:26.531006+0000","last_undegraded":"2024-04-25T00:14:26.531006+0000","last_fullsized":"2024-04-25T00:14:26.531006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:33.550379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307062+0000","last_change":"2024-04-25T00:14:01.837733+0000","last_active":"2024-04-25T00:14:09.307062+0000","last_peered":"2024-04-25T00:14:09.307062+0000","last_clean":"2024-04-25T00:14:09.307062+0000","last_became_active":"2024-04-25T00:14:01.837395+0000","last_became_peered":"2024-04-25T00:14:01.837395+0000","last_unstale":"2024-04-25T00:14:09.307062+0000","last_undegraded":"2024-04-25T00:14:09.307062+0000","last_fullsized":"2024-04-25T00:14:09.307062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:50.186834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306774+0000","last_change":"2024-04-25T00:14:06.848700+0000","last_active":"2024-04-25T00:14:09.306774+0000","last_peered":"2024-04-25T00:14:09.306774+0000","last_clean":"2024-04-25T00:14:09.306774+0000","last_became_active":"2024-04-25T00:14:06.848252+0000","last_became_peered":"2024-04-25T00:14:06.848252+0000","last_unstale":"2024-04-25T00:14:09.306774+0000","last_undegraded":"2024-04-25T00:14:09.306774+0000","last_fullsized":"2024-04-25T00:14:09.306774+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:07.653668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:27.535458+0000","last_change":"2024-04-25T00:14:01.405785+0000","last_active":"2024-04-25T00:14:27.535458+0000","last_peered":"2024-04-25T00:14:27.535458+0000","last_clean":"2024-04-25T00:14:27.535458+0000","last_became_active":"2024-04-25T00:14:01.405475+0000","last_became_peered":"2024-04-25T00:14:01.405475+0000","last_unstale":"2024-04-25T00:14:27.535458+0000","last_undegraded":"2024-04-25T00:14:27.535458+0000","last_fullsized":"2024-04-25T00:14:27.535458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:32.487795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307808+0000","last_change":"2024-04-25T00:14:01.832393+0000","last_active":"2024-04-25T00:14:09.307808+0000","last_peered":"2024-04-25T00:14:09.307808+0000","last_clean":"2024-04-25T00:14:09.307808+0000","last_became_active":"2024-04-25T00:14:01.832096+0000","last_became_peered":"2024-04-25T00:14:01.832096+0000","last_unstale":"2024-04-25T00:14:09.307808+0000","last_undegraded":"2024-04-25T00:14:09.307808+0000","last_fullsized":"2024-04-25T00:14:09.307808+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:41.749627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306849+0000","last_change":"2024-04-25T00:14:06.855224+0000","last_active":"2024-04-25T00:14:09.306849+0000","last_peered":"2024-04-25T00:14:09.306849+0000","last_clean":"2024-04-25T00:14:09.306849+0000","last_became_active":"2024-04-25T00:14:06.854847+0000","last_became_peered":"2024-04-25T00:14:06.854847+0000","last_unstale":"2024-04-25T00:14:09.306849+0000","last_undegraded":"2024-04-25T00:14:09.306849+0000","last_fullsized":"2024-04-25T00:14:09.306849+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:13.520448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306976+0000","last_change":"2024-04-25T00:14:01.401145+0000","last_active":"2024-04-25T00:14:09.306976+0000","last_peered":"2024-04-25T00:14:09.306976+0000","last_clean":"2024-04-25T00:14:09.306976+0000","last_became_active":"2024-04-25T00:14:01.400807+0000","last_became_peered":"2024-04-25T00:14:01.400807+0000","last_unstale":"2024-04-25T00:14:09.306976+0000","last_undegraded":"2024-04-25T00:14:09.306976+0000","last_fullsized":"2024-04-25T00:14:09.306976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:52:04.674677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307497+0000","last_change":"2024-04-25T00:14:01.838033+0000","last_active":"2024-04-25T00:14:09.307497+0000","last_peered":"2024-04-25T00:14:09.307497+0000","last_clean":"2024-04-25T00:14:09.307497+0000","last_became_active":"2024-04-25T00:14:01.837657+0000","last_became_peered":"2024-04-25T00:14:01.837657+0000","last_unstale":"2024-04-25T00:14:09.307497+0000","last_undegraded":"2024-04-25T00:14:09.307497+0000","last_fullsized":"2024-04-25T00:14:09.307497+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:51.895086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305593+0000","last_change":"2024-04-25T00:14:06.854885+0000","last_active":"2024-04-25T00:14:09.305593+0000","last_peered":"2024-04-25T00:14:09.305593+0000","last_clean":"2024-04-25T00:14:09.305593+0000","last_became_active":"2024-04-25T00:14:06.854200+0000","last_became_peered":"2024-04-25T00:14:06.854200+0000","last_unstale":"2024-04-25T00:14:09.305593+0000","last_undegraded":"2024-04-25T00:14:09.305593+0000","last_fullsized":"2024-04-25T00:14:09.305593+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:43.627742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306265+0000","last_change":"2024-04-25T00:14:01.403789+0000","last_active":"2024-04-25T00:14:09.306265+0000","last_peered":"2024-04-25T00:14:09.306265+0000","last_clean":"2024-04-25T00:14:09.306265+0000","last_became_active":"2024-04-25T00:14:01.403464+0000","last_became_peered":"2024-04-25T00:14:01.403464+0000","last_unstale":"2024-04-25T00:14:09.306265+0000","last_undegraded":"2024-04-25T00:14:09.306265+0000","last_fullsized":"2024-04-25T00:14:09.306265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:48.276536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307534+0000","last_change":"2024-04-25T00:14:01.839988+0000","last_active":"2024-04-25T00:14:09.307534+0000","last_peered":"2024-04-25T00:14:09.307534+0000","last_clean":"2024-04-25T00:14:09.307534+0000","last_became_active":"2024-04-25T00:14:01.839516+0000","last_became_peered":"2024-04-25T00:14:01.839516+0000","last_unstale":"2024-04-25T00:14:09.307534+0000","last_undegraded":"2024-04-25T00:14:09.307534+0000","last_fullsized":"2024-04-25T00:14:09.307534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:05.875127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306885+0000","last_change":"2024-04-25T00:14:01.828638+0000","last_active":"2024-04-25T00:14:09.306885+0000","last_peered":"2024-04-25T00:14:09.306885+0000","last_clean":"2024-04-25T00:14:09.306885+0000","last_became_active":"2024-04-25T00:14:01.828266+0000","last_became_peered":"2024-04-25T00:14:01.828266+0000","last_unstale":"2024-04-25T00:14:09.306885+0000","last_undegraded":"2024-04-25T00:14:09.306885+0000","last_fullsized":"2024-04-25T00:14:09.306885+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:34:22.619522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306476+0000","last_change":"2024-04-25T00:14:01.828199+0000","last_active":"2024-04-25T00:14:09.306476+0000","last_peered":"2024-04-25T00:14:09.306476+0000","last_clean":"2024-04-25T00:14:09.306476+0000","last_became_active":"2024-04-25T00:14:01.827579+0000","last_became_peered":"2024-04-25T00:14:01.827579+0000","last_unstale":"2024-04-25T00:14:09.306476+0000","last_undegraded":"2024-04-25T00:14:09.306476+0000","last_fullsized":"2024-04-25T00:14:09.306476+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:00.708928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305712+0000","last_change":"2024-04-25T00:14:01.825620+0000","last_active":"2024-04-25T00:14:09.305712+0000","last_peered":"2024-04-25T00:14:09.305712+0000","last_clean":"2024-04-25T00:14:09.305712+0000","last_became_active":"2024-04-25T00:14:01.825310+0000","last_became_peered":"2024-04-25T00:14:01.825310+0000","last_unstale":"2024-04-25T00:14:09.305712+0000","last_undegraded":"2024-04-25T00:14:09.305712+0000","last_fullsized":"2024-04-25T00:14:09.305712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:35.075355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306986+0000","last_change":"2024-04-25T00:14:01.836537+0000","last_active":"2024-04-25T00:14:09.306986+0000","last_peered":"2024-04-25T00:14:09.306986+0000","last_clean":"2024-04-25T00:14:09.306986+0000","last_became_active":"2024-04-25T00:14:01.836234+0000","last_became_peered":"2024-04-25T00:14:01.836234+0000","last_unstale":"2024-04-25T00:14:09.306986+0000","last_undegraded":"2024-04-25T00:14:09.306986+0000","last_fullsized":"2024-04-25T00:14:09.306986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:15.037498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306802+0000","last_change":"2024-04-25T00:14:01.825937+0000","last_active":"2024-04-25T00:14:09.306802+0000","last_peered":"2024-04-25T00:14:09.306802+0000","last_clean":"2024-04-25T00:14:09.306802+0000","last_became_active":"2024-04-25T00:14:01.825605+0000","last_became_peered":"2024-04-25T00:14:01.825605+0000","last_unstale":"2024-04-25T00:14:09.306802+0000","last_undegraded":"2024-04-25T00:14:09.306802+0000","last_fullsized":"2024-04-25T00:14:09.306802+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:06.918000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307561+0000","last_change":"2024-04-25T00:14:01.830004+0000","last_active":"2024-04-25T00:14:09.307561+0000","last_peered":"2024-04-25T00:14:09.307561+0000","last_clean":"2024-04-25T00:14:09.307561+0000","last_became_active":"2024-04-25T00:14:01.829588+0000","last_became_peered":"2024-04-25T00:14:01.829588+0000","last_unstale":"2024-04-25T00:14:09.307561+0000","last_undegraded":"2024-04-25T00:14:09.307561+0000","last_fullsized":"2024-04-25T00:14:09.307561+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:08.005995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306368+0000","last_change":"2024-04-25T00:14:01.825762+0000","last_active":"2024-04-25T00:14:09.306368+0000","last_peered":"2024-04-25T00:14:09.306368+0000","last_clean":"2024-04-25T00:14:09.306368+0000","last_became_active":"2024-04-25T00:14:01.825471+0000","last_became_peered":"2024-04-25T00:14:01.825471+0000","last_unstale":"2024-04-25T00:14:09.306368+0000","last_undegraded":"2024-04-25T00:14:09.306368+0000","last_fullsized":"2024-04-25T00:14:09.306368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:33.716875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306695+0000","last_change":"2024-04-25T00:14:01.836103+0000","last_active":"2024-04-25T00:14:09.306695+0000","last_peered":"2024-04-25T00:14:09.306695+0000","last_clean":"2024-04-25T00:14:09.306695+0000","last_became_active":"2024-04-25T00:14:01.834591+0000","last_became_peered":"2024-04-25T00:14:01.834591+0000","last_unstale":"2024-04-25T00:14:09.306695+0000","last_undegraded":"2024-04-25T00:14:09.306695+0000","last_fullsized":"2024-04-25T00:14:09.306695+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:55.130978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.306293+0000","last_change":"2024-04-25T00:14:01.833179+0000","last_active":"2024-04-25T00:14:09.306293+0000","last_peered":"2024-04-25T00:14:09.306293+0000","last_clean":"2024-04-25T00:14:09.306293+0000","last_became_active":"2024-04-25T00:14:01.832856+0000","last_became_peered":"2024-04-25T00:14:01.832856+0000","last_unstale":"2024-04-25T00:14:09.306293+0000","last_undegraded":"2024-04-25T00:14:09.306293+0000","last_fullsized":"2024-04-25T00:14:09.306293+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:07.153356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305673+0000","last_change":"2024-04-25T00:14:01.834751+0000","last_active":"2024-04-25T00:14:09.305673+0000","last_peered":"2024-04-25T00:14:09.305673+0000","last_clean":"2024-04-25T00:14:09.305673+0000","last_became_active":"2024-04-25T00:14:01.834413+0000","last_became_peered":"2024-04-25T00:14:01.834413+0000","last_unstale":"2024-04-25T00:14:09.305673+0000","last_undegraded":"2024-04-25T00:14:09.305673+0000","last_fullsized":"2024-04-25T00:14:09.305673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:52.341565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307026+0000","last_change":"2024-04-25T00:14:01.832944+0000","last_active":"2024-04-25T00:14:09.307026+0000","last_peered":"2024-04-25T00:14:09.307026+0000","last_clean":"2024-04-25T00:14:09.307026+0000","last_became_active":"2024-04-25T00:14:01.832394+0000","last_became_peered":"2024-04-25T00:14:01.832394+0000","last_unstale":"2024-04-25T00:14:09.307026+0000","last_undegraded":"2024-04-25T00:14:09.307026+0000","last_fullsized":"2024-04-25T00:14:09.307026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:26.957786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.305963+0000","last_change":"2024-04-25T00:14:01.831492+0000","last_active":"2024-04-25T00:14:09.305963+0000","last_peered":"2024-04-25T00:14:09.305963+0000","last_clean":"2024-04-25T00:14:09.305963+0000","last_became_active":"2024-04-25T00:14:01.829557+0000","last_became_peered":"2024-04-25T00:14:01.829557+0000","last_unstale":"2024-04-25T00:14:09.305963+0000","last_undegraded":"2024-04-25T00:14:09.305963+0000","last_fullsized":"2024-04-25T00:14:09.305963+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:45.116188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T00:14:09.307577+0000","last_change":"2024-04-25T00:14:01.839323+0000","last_active":"2024-04-25T00:14:09.307577+0000","last_peered":"2024-04-25T00:14:09.307577+0000","last_clean":"2024-04-25T00:14:09.307577+0000","last_became_active":"2024-04-25T00:14:01.838999+0000","last_became_peered":"2024-04-25T00:14:01.838999+0000","last_unstale":"2024-04-25T00:14:09.307577+0000","last_undegraded":"2024-04-25T00:14:09.307577+0000","last_fullsized":"2024-04-25T00:14:09.307577+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:20.652123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":44081,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":25,"num_read_kb":43,"num_write":142,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":107795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125,"ondisk_log_size":125,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":497852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612631,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36992,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708160,"statfs":{"total":95995035648,"available":95957155840,"internally_reserved":0,"allocated":1695744,"data_stored":1130269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776157,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":506862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037794,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":498590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299430,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28608,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716544,"statfs":{"total":95995035648,"available":95965741056,"internally_reserved":0,"allocated":1630208,"data_stored":1090012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462956,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1024000,"data_stored":497942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":538323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1032192,"data_stored":498546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":510035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149754,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":498136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888118,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40504,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704648,"statfs":{"total":95995035648,"available":95953559552,"internally_reserved":0,"allocated":1032192,"data_stored":498496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724592,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1642496,"data_stored":1089696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":40471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":18739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":42049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:14:31.563 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:14:31.564 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T00:14:31.564 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T00:14:31.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:31 smithi195 ceph-mon[32833]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:31.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:31 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3631618189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:14:31.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:31 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2108874213' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:31.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:31 smithi195 ceph-mon[32833]: pgmap v307: 129 pgs: 129 active+clean; 620 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5.6 KiB/s wr, 9 op/s 2024-04-25T00:14:31.840 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:14:31.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:31 smithi170 ceph-mon[32807]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:31.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:31 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3631618189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:14:31.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:31 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2108874213' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:31.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:31 smithi170 ceph-mon[32807]: pgmap v307: 129 pgs: 129 active+clean; 620 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5.6 KiB/s wr, 9 op/s 2024-04-25T00:14:31.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:31 smithi107 ceph-mon[29153]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:31.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:31 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3631618189' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:14:31.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:31 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2108874213' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:31.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:31 smithi107 ceph-mon[29153]: pgmap v307: 129 pgs: 129 active+clean; 620 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5.6 KiB/s wr, 9 op/s 2024-04-25T00:14:32.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:32 smithi195 ceph-mon[32833]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:14:32.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:32 smithi195 ceph-mon[32833]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:14:32.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:32 smithi170 ceph-mon[32807]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:14:32.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:32 smithi170 ceph-mon[32807]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:14:32.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:32 smithi107 ceph-mon[29153]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:14:32.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:32 smithi107 ceph-mon[29153]: from='client.14937 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:14:33.710 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:33 smithi107 ceph-mon[29153]: pgmap v308: 129 pgs: 129 active+clean; 620 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.5 KiB/s wr, 10 op/s 2024-04-25T00:14:33.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.746+0000 7f9939556700 1 -- 172.21.15.107:0/3444909862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99341011b0 msgr2=0x7f99341015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:33.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.746+0000 7f9939556700 1 --2- 172.21.15.107:0/3444909862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99341011b0 0x7f99341015d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f99240099c0 tx=0x7f9924009cd0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.746+0000 7f9939556700 1 -- 172.21.15.107:0/3444909862 shutdown_connections 2024-04-25T00:14:33.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.746+0000 7f9939556700 1 --2- 172.21.15.107:0/3444909862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9934102d50 0x7f9934109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.746+0000 7f9939556700 1 --2- 172.21.15.107:0/3444909862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99341011b0 0x7f99341015d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.746+0000 7f9939556700 1 --2- 172.21.15.107:0/3444909862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99341023b0 0x7f9934102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.746+0000 7f9939556700 1 -- 172.21.15.107:0/3444909862 >> 172.21.15.107:0/3444909862 conn(0x7f99340fc750 msgr2=0x7f99340feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:33.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.746+0000 7f9939556700 1 -- 172.21.15.107:0/3444909862 shutdown_connections 2024-04-25T00:14:33.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.746+0000 7f9939556700 1 -- 172.21.15.107:0/3444909862 wait complete. 2024-04-25T00:14:33.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.746+0000 7f9939556700 1 Processor -- start 2024-04-25T00:14:33.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.747+0000 7f9939556700 1 -- start start 2024-04-25T00:14:33.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.747+0000 7f9939556700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99341011b0 0x7f99341982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.747+0000 7f9939556700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99341023b0 0x7f9934198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.747+0000 7f9939556700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9934102d50 0x7f993419f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.747+0000 7f9939556700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f993410cd70 con 0x7f9934102d50 2024-04-25T00:14:33.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.747+0000 7f9939556700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f993410cbf0 con 0x7f99341023b0 2024-04-25T00:14:33.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.747+0000 7f9939556700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f993410cef0 con 0x7f99341011b0 2024-04-25T00:14:33.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.747+0000 7f99337fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9934102d50 0x7f993419f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.747+0000 7f99337fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9934102d50 0x7f993419f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38182/0 (socket says 172.21.15.107:38182) 2024-04-25T00:14:33.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.747+0000 7f99337fe700 1 -- 172.21.15.107:0/1927793734 learned_addr learned my addr 172.21.15.107:0/1927793734 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:33.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.747+0000 7f992a7fc700 1 --2- 172.21.15.107:0/1927793734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99341023b0 0x7f9934198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.747+0000 7f9932ffd700 1 --2- 172.21.15.107:0/1927793734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99341011b0 0x7f99341982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.747+0000 7f99337fe700 1 -- 172.21.15.107:0/1927793734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99341011b0 msgr2=0x7f99341982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:33.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.747+0000 7f99337fe700 1 --2- 172.21.15.107:0/1927793734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99341011b0 0x7f99341982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.747+0000 7f99337fe700 1 -- 172.21.15.107:0/1927793734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99341023b0 msgr2=0x7f9934198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:33.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.747+0000 7f99337fe700 1 --2- 172.21.15.107:0/1927793734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99341023b0 0x7f9934198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:33.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.747+0000 7f99337fe700 1 -- 172.21.15.107:0/1927793734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993419ff30 con 0x7f9934102d50 2024-04-25T00:14:33.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.747+0000 7f99337fe700 1 --2- 172.21.15.107:0/1927793734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9934102d50 0x7f993419f890 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f992c007da0 tx=0x7f992c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:33.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.748+0000 7f9930ff9700 1 -- 172.21.15.107:0/1927793734 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c016020 con 0x7f9934102d50 2024-04-25T00:14:33.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.748+0000 7f9930ff9700 1 -- 172.21.15.107:0/1927793734 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f992c00ac50 con 0x7f9934102d50 2024-04-25T00:14:33.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.748+0000 7f9939556700 1 -- 172.21.15.107:0/1927793734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99341a0220 con 0x7f9934102d50 2024-04-25T00:14:33.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.748+0000 7f9930ff9700 1 -- 172.21.15.107:0/1927793734 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c0115e0 con 0x7f9934102d50 2024-04-25T00:14:33.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.748+0000 7f9939556700 1 -- 172.21.15.107:0/1927793734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99341a0760 con 0x7f9934102d50 2024-04-25T00:14:33.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.749+0000 7f9939556700 1 -- 172.21.15.107:0/1927793734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99340507a0 con 0x7f9934102d50 2024-04-25T00:14:33.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.750+0000 7f9930ff9700 1 -- 172.21.15.107:0/1927793734 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f992c019020 con 0x7f9934102d50 2024-04-25T00:14:33.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.750+0000 7f9930ff9700 1 --2- 172.21.15.107:0/1927793734 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f99200a2de0 0x7f99200a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:33.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.751+0000 7f9932ffd700 1 --2- 172.21.15.107:0/1927793734 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f99200a2de0 0x7f99200a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:33.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.751+0000 7f9930ff9700 1 -- 172.21.15.107:0/1927793734 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f992c0cf560 con 0x7f9934102d50 2024-04-25T00:14:33.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.751+0000 7f9932ffd700 1 --2- 172.21.15.107:0/1927793734 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f99200a2de0 0x7f99200a52a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f992400b580 tx=0x7f9924005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:33.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:33.754+0000 7f9930ff9700 1 -- 172.21.15.107:0/1927793734 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f992c020020 con 0x7f9934102d50 2024-04-25T00:14:33.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:33 smithi195 ceph-mon[32833]: pgmap v308: 129 pgs: 129 active+clean; 620 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.5 KiB/s wr, 10 op/s 2024-04-25T00:14:33.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:33 smithi170 ceph-mon[32807]: pgmap v308: 129 pgs: 129 active+clean; 620 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5.5 KiB/s wr, 10 op/s 2024-04-25T00:14:34.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:34.036+0000 7f9939556700 1 -- 172.21.15.107:0/1927793734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f99341a0be0 con 0x7f9934102d50 2024-04-25T00:14:34.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:34.038+0000 7f9930ff9700 1 -- 172.21.15.107:0/1927793734 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f992c00ae00 con 0x7f9934102d50 2024-04-25T00:14:34.038 INFO:teuthology.orchestra.run.smithi107.stdout:[client.0] 2024-04-25T00:14:34.038 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQBqoClm1p05AhAA1NIbU33vTubkn4XO0i9usQ== 2024-04-25T00:14:34.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:34.040+0000 7f9939556700 1 -- 172.21.15.107:0/1927793734 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f99200a2de0 msgr2=0x7f99200a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:34.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:34.040+0000 7f9939556700 1 --2- 172.21.15.107:0/1927793734 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f99200a2de0 0x7f99200a52a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f992400b580 tx=0x7f9924005fb0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:34.040+0000 7f9939556700 1 -- 172.21.15.107:0/1927793734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9934102d50 msgr2=0x7f993419f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:34.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:34.040+0000 7f9939556700 1 --2- 172.21.15.107:0/1927793734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9934102d50 0x7f993419f890 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f992c007da0 tx=0x7f992c00a340 comp rx=0 tx=0).stop 2024-04-25T00:14:34.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:34.040+0000 7f9939556700 1 -- 172.21.15.107:0/1927793734 shutdown_connections 2024-04-25T00:14:34.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:34.040+0000 7f9939556700 1 --2- 172.21.15.107:0/1927793734 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f99200a2de0 0x7f99200a52a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:34.040+0000 7f9939556700 1 --2- 172.21.15.107:0/1927793734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9934102d50 0x7f993419f890 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:34.040+0000 7f9939556700 1 --2- 172.21.15.107:0/1927793734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99341011b0 0x7f99341982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:34.040+0000 7f9939556700 1 --2- 172.21.15.107:0/1927793734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99341023b0 0x7f9934198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:34.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:34.040+0000 7f9939556700 1 -- 172.21.15.107:0/1927793734 >> 172.21.15.107:0/1927793734 conn(0x7f99340fc750 msgr2=0x7f99340fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:34.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:34.040+0000 7f9939556700 1 -- 172.21.15.107:0/1927793734 shutdown_connections 2024-04-25T00:14:34.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:34.040+0000 7f9939556700 1 -- 172.21.15.107:0/1927793734 wait complete. 2024-04-25T00:14:34.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:34 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1927793734' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T00:14:34.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:34 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1927793734' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T00:14:34.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:34 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1927793734' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T00:14:34.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:34 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1927793734' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T00:14:34.861 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T00:14:34.861 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T00:14:34.861 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T00:14:34.897 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T00:14:34.897 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T00:14:34.897 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T00:14:34.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:34 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1927793734' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T00:14:34.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:34 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1927793734' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T00:14:35.185 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:14:35.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:35 smithi195 ceph-mon[32833]: pgmap v309: 129 pgs: 129 active+clean; 620 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-04-25T00:14:35.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:35 smithi170 ceph-mon[32807]: pgmap v309: 129 pgs: 129 active+clean; 620 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-04-25T00:14:35.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:35 smithi107 ceph-mon[29153]: pgmap v309: 129 pgs: 129 active+clean; 620 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-04-25T00:14:37.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.000+0000 7f0c92a48700 1 -- 172.21.15.107:0/1357399715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c8c06abe0 msgr2=0x7f0c8c06afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.000+0000 7f0c92a48700 1 --2- 172.21.15.107:0/1357399715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c8c06abe0 0x7f0c8c06afc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009990 tx=0x7f0c7c009ca0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.000+0000 7f0c92a48700 1 -- 172.21.15.107:0/1357399715 shutdown_connections 2024-04-25T00:14:37.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.000+0000 7f0c92a48700 1 --2- 172.21.15.107:0/1357399715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c8c107570 0x7f0c8c10f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.000+0000 7f0c92a48700 1 --2- 172.21.15.107:0/1357399715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c8c06abe0 0x7f0c8c06afc0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.000+0000 7f0c92a48700 1 --2- 172.21.15.107:0/1357399715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c8c06b590 0x7f0c8c106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.000+0000 7f0c92a48700 1 -- 172.21.15.107:0/1357399715 >> 172.21.15.107:0/1357399715 conn(0x7f0c8c078130 msgr2=0x7f0c8c078540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:37.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.000+0000 7f0c92a48700 1 -- 172.21.15.107:0/1357399715 shutdown_connections 2024-04-25T00:14:37.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.000+0000 7f0c92a48700 1 -- 172.21.15.107:0/1357399715 wait complete. 2024-04-25T00:14:37.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.001+0000 7f0c92a48700 1 Processor -- start 2024-04-25T00:14:37.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.001+0000 7f0c92a48700 1 -- start start 2024-04-25T00:14:37.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.001+0000 7f0c92a48700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c8c06abe0 0x7f0c8c19e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.001+0000 7f0c92a48700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c8c06b590 0x7f0c8c19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.001+0000 7f0c92a48700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c8c107570 0x7f0c8c1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.001+0000 7f0c92a48700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c112530 con 0x7f0c8c06b590 2024-04-25T00:14:37.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.001+0000 7f0c92a48700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c1123b0 con 0x7f0c8c107570 2024-04-25T00:14:37.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.001+0000 7f0c92a48700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c1126b0 con 0x7f0c8c06abe0 2024-04-25T00:14:37.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.001+0000 7f0c8b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c8c06b590 0x7f0c8c19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.001+0000 7f0c8b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c8c06b590 0x7f0c8c19ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38190/0 (socket says 172.21.15.107:38190) 2024-04-25T00:14:37.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.001+0000 7f0c8b7fe700 1 -- 172.21.15.107:0/2576064215 learned_addr learned my addr 172.21.15.107:0/2576064215 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:37.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.001+0000 7f0c8bfff700 1 --2- 172.21.15.107:0/2576064215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c8c06abe0 0x7f0c8c19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.001+0000 7f0c90fe5700 1 --2- 172.21.15.107:0/2576064215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c8c107570 0x7f0c8c1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.001+0000 7f0c8b7fe700 1 -- 172.21.15.107:0/2576064215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c8c06abe0 msgr2=0x7f0c8c19e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.001+0000 7f0c8b7fe700 1 --2- 172.21.15.107:0/2576064215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c8c06abe0 0x7f0c8c19e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.001+0000 7f0c8b7fe700 1 -- 172.21.15.107:0/2576064215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c8c107570 msgr2=0x7f0c8c1987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.001+0000 7f0c8b7fe700 1 --2- 172.21.15.107:0/2576064215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c8c107570 0x7f0c8c1987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.001+0000 7f0c8b7fe700 1 -- 172.21.15.107:0/2576064215 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8c198fd0 con 0x7f0c8c06b590 2024-04-25T00:14:37.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.002+0000 7f0c90fe5700 1 --2- 172.21.15.107:0/2576064215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c8c107570 0x7f0c8c1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:14:37.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.002+0000 7f0c8b7fe700 1 --2- 172.21.15.107:0/2576064215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c8c06b590 0x7f0c8c19ebb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000d980 tx=0x7f0c8000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:37.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.002+0000 7f0c897fa700 1 -- 172.21.15.107:0/2576064215 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c80017070 con 0x7f0c8c06b590 2024-04-25T00:14:37.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.002+0000 7f0c92a48700 1 -- 172.21.15.107:0/2576064215 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c8c1992c0 con 0x7f0c8c06b590 2024-04-25T00:14:37.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.002+0000 7f0c897fa700 1 -- 172.21.15.107:0/2576064215 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c80009e10 con 0x7f0c8c06b590 2024-04-25T00:14:37.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.002+0000 7f0c897fa700 1 -- 172.21.15.107:0/2576064215 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8000f620 con 0x7f0c8c06b590 2024-04-25T00:14:37.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.002+0000 7f0c92a48700 1 -- 172.21.15.107:0/2576064215 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c8c1a54d0 con 0x7f0c8c06b590 2024-04-25T00:14:37.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.003+0000 7f0c92a48700 1 -- 172.21.15.107:0/2576064215 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c8c103970 con 0x7f0c8c06b590 2024-04-25T00:14:37.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.004+0000 7f0c897fa700 1 -- 172.21.15.107:0/2576064215 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f0c80010460 con 0x7f0c8c06b590 2024-04-25T00:14:37.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.004+0000 7f0c897fa700 1 --2- 172.21.15.107:0/2576064215 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f0c740a2cb0 0x7f0c740a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.005+0000 7f0c897fa700 1 -- 172.21.15.107:0/2576064215 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f0c800cef20 con 0x7f0c8c06b590 2024-04-25T00:14:37.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.005+0000 7f0c8bfff700 1 --2- 172.21.15.107:0/2576064215 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f0c740a2cb0 0x7f0c740a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.005+0000 7f0c8bfff700 1 --2- 172.21.15.107:0/2576064215 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f0c740a2cb0 0x7f0c740a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c00b5c0 tx=0x7f0c7c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:37.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.008+0000 7f0c897fa700 1 -- 172.21.15.107:0/2576064215 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0c8009bae0 con 0x7f0c8c06b590 2024-04-25T00:14:37.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.263+0000 7f0c92a48700 1 -- 172.21.15.107:0/2576064215 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f0c8c02db30 con 0x7f0c8c06b590 2024-04-25T00:14:37.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.268+0000 7f0c897fa700 1 -- 172.21.15.107:0/2576064215 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263208 (secure 0 0 0) 0x7f0c8009b3e0 con 0x7f0c8c06b590 2024-04-25T00:14:37.272 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:14:37.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.270+0000 7f0c92a48700 1 -- 172.21.15.107:0/2576064215 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f0c740a2cb0 msgr2=0x7f0c740a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.270+0000 7f0c92a48700 1 --2- 172.21.15.107:0/2576064215 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f0c740a2cb0 0x7f0c740a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c00b5c0 tx=0x7f0c7c005960 comp rx=0 tx=0).stop 2024-04-25T00:14:37.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.270+0000 7f0c92a48700 1 -- 172.21.15.107:0/2576064215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c8c06b590 msgr2=0x7f0c8c19ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.270+0000 7f0c92a48700 1 --2- 172.21.15.107:0/2576064215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c8c06b590 0x7f0c8c19ebb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000d980 tx=0x7f0c8000dc90 comp rx=0 tx=0).stop 2024-04-25T00:14:37.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.270+0000 7f0c92a48700 1 -- 172.21.15.107:0/2576064215 shutdown_connections 2024-04-25T00:14:37.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.270+0000 7f0c92a48700 1 --2- 172.21.15.107:0/2576064215 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f0c740a2cb0 0x7f0c740a5170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.270+0000 7f0c92a48700 1 --2- 172.21.15.107:0/2576064215 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c8c107570 0x7f0c8c1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.270+0000 7f0c92a48700 1 --2- 172.21.15.107:0/2576064215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c8c06abe0 0x7f0c8c19e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.270+0000 7f0c92a48700 1 --2- 172.21.15.107:0/2576064215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c8c06b590 0x7f0c8c19ebb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.270+0000 7f0c92a48700 1 -- 172.21.15.107:0/2576064215 >> 172.21.15.107:0/2576064215 conn(0x7f0c8c078130 msgr2=0x7f0c8c1008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:37.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.271+0000 7f0c92a48700 1 -- 172.21.15.107:0/2576064215 shutdown_connections 2024-04-25T00:14:37.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:37.271+0000 7f0c92a48700 1 -- 172.21.15.107:0/2576064215 wait complete. 2024-04-25T00:14:38.045 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1644942185},{"type":"v1","addr":"172.21.15.107:6801","nonce":1644942185}]},"active_addr":"172.21.15.107:6801/1644942185","active_change":"2024-04-25T00:07:00.838400+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.107:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":111369358}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":151603643}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":1884923543}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":3936375305}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":2189889155}]}]} 2024-04-25T00:14:38.050 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T00:14:38.051 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T00:14:38.051 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T00:14:38.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:37 smithi107 ceph-mon[29153]: pgmap v310: 129 pgs: 129 active+clean; 625 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.3 KiB/s wr, 13 op/s 2024-04-25T00:14:38.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:37 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2576064215' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T00:14:38.293 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:14:38.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:37 smithi195 ceph-mon[32833]: pgmap v310: 129 pgs: 129 active+clean; 625 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.3 KiB/s wr, 13 op/s 2024-04-25T00:14:38.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:37 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2576064215' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T00:14:38.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:37 smithi170 ceph-mon[32807]: pgmap v310: 129 pgs: 129 active+clean; 625 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.3 KiB/s wr, 13 op/s 2024-04-25T00:14:38.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:37 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2576064215' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T00:14:40.171 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:39 smithi107 ceph-mon[29153]: pgmap v311: 129 pgs: 129 active+clean; 625 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.3 KiB/s wr, 13 op/s 2024-04-25T00:14:40.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.171+0000 7f892e4bb700 1 -- 172.21.15.107:0/343515703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89281011c0 msgr2=0x7f892810f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:40.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.171+0000 7f892e4bb700 1 --2- 172.21.15.107:0/343515703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89281011c0 0x7f892810f100 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f892000b0a0 tx=0x7f892000b3b0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.171+0000 7f892e4bb700 1 -- 172.21.15.107:0/343515703 shutdown_connections 2024-04-25T00:14:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.171+0000 7f892e4bb700 1 --2- 172.21.15.107:0/343515703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89281011c0 0x7f892810f100 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.171+0000 7f892e4bb700 1 --2- 172.21.15.107:0/343515703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89281082c0 0x7f89281086a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.171+0000 7f892e4bb700 1 --2- 172.21.15.107:0/343515703 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8928100820 0x7f8928100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.171+0000 7f892e4bb700 1 -- 172.21.15.107:0/343515703 >> 172.21.15.107:0/343515703 conn(0x7f89280777a0 msgr2=0x7f89280feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.171+0000 7f892e4bb700 1 -- 172.21.15.107:0/343515703 shutdown_connections 2024-04-25T00:14:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.171+0000 7f892e4bb700 1 -- 172.21.15.107:0/343515703 wait complete. 2024-04-25T00:14:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.171+0000 7f892e4bb700 1 Processor -- start 2024-04-25T00:14:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.171+0000 7f892e4bb700 1 -- start start 2024-04-25T00:14:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.172+0000 7f892e4bb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8928100820 0x7f8928101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.172+0000 7f892e4bb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89281011c0 0x7f8928101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.172+0000 7f892e4bb700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89281082c0 0x7f8928102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.172+0000 7f892e4bb700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f89281125c0 con 0x7f89281011c0 2024-04-25T00:14:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.172+0000 7f892e4bb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8928112440 con 0x7f8928100820 2024-04-25T00:14:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.172+0000 7f892e4bb700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8928112740 con 0x7f89281082c0 2024-04-25T00:14:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.172+0000 7f89277fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89281011c0 0x7f8928101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.172+0000 7f89277fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89281011c0 0x7f8928101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43730/0 (socket says 172.21.15.107:43730) 2024-04-25T00:14:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.172+0000 7f89277fe700 1 -- 172.21.15.107:0/3795326970 learned_addr learned my addr 172.21.15.107:0/3795326970 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.172+0000 7f892ca58700 1 --2- 172.21.15.107:0/3795326970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89281082c0 0x7f8928102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.172+0000 7f8927fff700 1 --2- 172.21.15.107:0/3795326970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8928100820 0x7f8928101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.172+0000 7f89277fe700 1 -- 172.21.15.107:0/3795326970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89281082c0 msgr2=0x7f8928102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.172+0000 7f89277fe700 1 --2- 172.21.15.107:0/3795326970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89281082c0 0x7f8928102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.172+0000 7f89277fe700 1 -- 172.21.15.107:0/3795326970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8928100820 msgr2=0x7f8928101a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.172+0000 7f89277fe700 1 --2- 172.21.15.107:0/3795326970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8928100820 0x7f8928101a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.172+0000 7f89277fe700 1 -- 172.21.15.107:0/3795326970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89281a6f20 con 0x7f89281011c0 2024-04-25T00:14:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.172+0000 7f89277fe700 1 --2- 172.21.15.107:0/3795326970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89281011c0 0x7f8928101f40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f891c00b530 tx=0x7f891c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.172+0000 7f89257fa700 1 -- 172.21.15.107:0/3795326970 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c015020 con 0x7f89281011c0 2024-04-25T00:14:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.173+0000 7f892e4bb700 1 -- 172.21.15.107:0/3795326970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89281a7210 con 0x7f89281011c0 2024-04-25T00:14:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.173+0000 7f89257fa700 1 -- 172.21.15.107:0/3795326970 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f891c011d10 con 0x7f89281011c0 2024-04-25T00:14:40.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.173+0000 7f89257fa700 1 -- 172.21.15.107:0/3795326970 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c00f2e0 con 0x7f89281011c0 2024-04-25T00:14:40.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.173+0000 7f892e4bb700 1 -- 172.21.15.107:0/3795326970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89281a77a0 con 0x7f89281011c0 2024-04-25T00:14:40.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.173+0000 7f892e4bb700 1 -- 172.21.15.107:0/3795326970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8928103b40 con 0x7f89281011c0 2024-04-25T00:14:40.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.175+0000 7f89257fa700 1 -- 172.21.15.107:0/3795326970 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f891c018020 con 0x7f89281011c0 2024-04-25T00:14:40.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.175+0000 7f89257fa700 1 --2- 172.21.15.107:0/3795326970 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f89100a2cb0 0x7f89100a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:40.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.175+0000 7f8927fff700 1 --2- 172.21.15.107:0/3795326970 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f89100a2cb0 0x7f89100a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:40.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.175+0000 7f89257fa700 1 -- 172.21.15.107:0/3795326970 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f891c0cef60 con 0x7f89281011c0 2024-04-25T00:14:40.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.176+0000 7f8927fff700 1 --2- 172.21.15.107:0/3795326970 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f89100a2cb0 0x7f89100a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8918005950 tx=0x7f891800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:40.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.179+0000 7f89257fa700 1 -- 172.21.15.107:0/3795326970 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f891c01f020 con 0x7f89281011c0 2024-04-25T00:14:40.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:39 smithi195 ceph-mon[32833]: pgmap v311: 129 pgs: 129 active+clean; 625 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.3 KiB/s wr, 13 op/s 2024-04-25T00:14:40.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.376+0000 7f892e4bb700 1 -- 172.21.15.107:0/3795326970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f892802db30 con 0x7f89281011c0 2024-04-25T00:14:40.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.378+0000 7f89257fa700 1 -- 172.21.15.107:0/3795326970 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24134 (secure 0 0 0) 0x7f891c09bc30 con 0x7f89281011c0 2024-04-25T00:14:40.378 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:14:40.379 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":92,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:14:08.839737+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:59.672801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:00.957782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:14:05.065846+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:40.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.380+0000 7f892e4bb700 1 -- 172.21.15.107:0/3795326970 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f89100a2cb0 msgr2=0x7f89100a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:40.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.380+0000 7f892e4bb700 1 --2- 172.21.15.107:0/3795326970 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f89100a2cb0 0x7f89100a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8918005950 tx=0x7f891800a300 comp rx=0 tx=0).stop 2024-04-25T00:14:40.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.380+0000 7f892e4bb700 1 -- 172.21.15.107:0/3795326970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89281011c0 msgr2=0x7f8928101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:40.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.380+0000 7f892e4bb700 1 --2- 172.21.15.107:0/3795326970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89281011c0 0x7f8928101f40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f891c00b530 tx=0x7f891c00b840 comp rx=0 tx=0).stop 2024-04-25T00:14:40.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.380+0000 7f892e4bb700 1 -- 172.21.15.107:0/3795326970 shutdown_connections 2024-04-25T00:14:40.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.380+0000 7f892e4bb700 1 --2- 172.21.15.107:0/3795326970 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f89100a2cb0 0x7f89100a5170 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.380+0000 7f892e4bb700 1 --2- 172.21.15.107:0/3795326970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f89281082c0 0x7f8928102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.380+0000 7f892e4bb700 1 --2- 172.21.15.107:0/3795326970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8928100820 0x7f8928101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.380+0000 7f892e4bb700 1 --2- 172.21.15.107:0/3795326970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f89281011c0 0x7f8928101f40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:40.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.380+0000 7f892e4bb700 1 -- 172.21.15.107:0/3795326970 >> 172.21.15.107:0/3795326970 conn(0x7f89280777a0 msgr2=0x7f89280feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:40.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.380+0000 7f892e4bb700 1 -- 172.21.15.107:0/3795326970 shutdown_connections 2024-04-25T00:14:40.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:40.380+0000 7f892e4bb700 1 -- 172.21.15.107:0/3795326970 wait complete. 2024-04-25T00:14:40.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:39 smithi170 ceph-mon[32807]: pgmap v311: 129 pgs: 129 active+clean; 625 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.3 KiB/s wr, 13 op/s 2024-04-25T00:14:41.170 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:40 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3795326970' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:41.196 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T00:14:41.196 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T00:14:41.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:40 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3795326970' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:41.434 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:14:41.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:40 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3795326970' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:42.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:42 smithi107 ceph-mon[29153]: pgmap v312: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-04-25T00:14:42.336 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:42 smithi170 ceph-mon[32807]: pgmap v312: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-04-25T00:14:42.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:42 smithi195 ceph-mon[32833]: pgmap v312: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-04-25T00:14:43.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.209+0000 7ffac128d700 1 -- 172.21.15.107:0/3291463463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffabc101800 msgr2=0x7ffabc106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:43.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.209+0000 7ffac128d700 1 --2- 172.21.15.107:0/3291463463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffabc101800 0x7ffabc106f40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0099c0 tx=0x7ffaac009cd0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.209+0000 7ffac128d700 1 -- 172.21.15.107:0/3291463463 shutdown_connections 2024-04-25T00:14:43.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.209+0000 7ffac128d700 1 --2- 172.21.15.107:0/3291463463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffabc107670 0x7ffabc10f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.209+0000 7ffac128d700 1 --2- 172.21.15.107:0/3291463463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffabc100e50 0x7ffabc101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.209+0000 7ffac128d700 1 --2- 172.21.15.107:0/3291463463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffabc101800 0x7ffabc106f40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.209+0000 7ffac128d700 1 -- 172.21.15.107:0/3291463463 >> 172.21.15.107:0/3291463463 conn(0x7ffabc076450 msgr2=0x7ffabc076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:43.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.209+0000 7ffac128d700 1 -- 172.21.15.107:0/3291463463 shutdown_connections 2024-04-25T00:14:43.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.209+0000 7ffac128d700 1 -- 172.21.15.107:0/3291463463 wait complete. 2024-04-25T00:14:43.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.209+0000 7ffac128d700 1 Processor -- start 2024-04-25T00:14:43.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.209+0000 7ffac128d700 1 -- start start 2024-04-25T00:14:43.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.210+0000 7ffac128d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffabc100e50 0x7ffabc19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:43.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.210+0000 7ffac128d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffabc101800 0x7ffabc19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:43.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.210+0000 7ffac128d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffabc107670 0x7ffabc1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:43.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.210+0000 7ffac128d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc112660 con 0x7ffabc107670 2024-04-25T00:14:43.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.210+0000 7ffac128d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc1124e0 con 0x7ffabc101800 2024-04-25T00:14:43.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.210+0000 7ffac128d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc1127e0 con 0x7ffabc100e50 2024-04-25T00:14:43.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.210+0000 7ffabb7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffabc107670 0x7ffabc1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:43.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.210+0000 7ffabb7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffabc107670 0x7ffabc1988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43748/0 (socket says 172.21.15.107:43748) 2024-04-25T00:14:43.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.210+0000 7ffabb7fe700 1 -- 172.21.15.107:0/2403495900 learned_addr learned my addr 172.21.15.107:0/2403495900 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:43.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.210+0000 7ffabaffd700 1 --2- 172.21.15.107:0/2403495900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffabc100e50 0x7ffabc19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:43.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.210+0000 7ffaba7fc700 1 --2- 172.21.15.107:0/2403495900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffabc101800 0x7ffabc19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:43.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.210+0000 7ffabb7fe700 1 -- 172.21.15.107:0/2403495900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffabc100e50 msgr2=0x7ffabc19e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:43.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.210+0000 7ffabb7fe700 1 --2- 172.21.15.107:0/2403495900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffabc100e50 0x7ffabc19e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.210+0000 7ffabb7fe700 1 -- 172.21.15.107:0/2403495900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffabc101800 msgr2=0x7ffabc19ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:43.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.210+0000 7ffabb7fe700 1 --2- 172.21.15.107:0/2403495900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffabc101800 0x7ffabc19ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.210+0000 7ffabb7fe700 1 -- 172.21.15.107:0/2403495900 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffabc199040 con 0x7ffabc107670 2024-04-25T00:14:43.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.210+0000 7ffabb7fe700 1 --2- 172.21.15.107:0/2403495900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffabc107670 0x7ffabc1988e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffab400b970 tx=0x7ffab400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:43.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.210+0000 7ffaa3fff700 1 -- 172.21.15.107:0/2403495900 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab4019070 con 0x7ffabc107670 2024-04-25T00:14:43.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.210+0000 7ffac128d700 1 -- 172.21.15.107:0/2403495900 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffabc199330 con 0x7ffabc107670 2024-04-25T00:14:43.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.210+0000 7ffaa3fff700 1 -- 172.21.15.107:0/2403495900 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffab4014440 con 0x7ffabc107670 2024-04-25T00:14:43.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.210+0000 7ffaa3fff700 1 -- 172.21.15.107:0/2403495900 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab40075d0 con 0x7ffabc107670 2024-04-25T00:14:43.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.211+0000 7ffac128d700 1 -- 172.21.15.107:0/2403495900 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffabc1a55f0 con 0x7ffabc107670 2024-04-25T00:14:43.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.211+0000 7ffac128d700 1 -- 172.21.15.107:0/2403495900 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffabc10a820 con 0x7ffabc107670 2024-04-25T00:14:43.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.213+0000 7ffaa3fff700 1 -- 172.21.15.107:0/2403495900 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ffab401c070 con 0x7ffabc107670 2024-04-25T00:14:43.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.213+0000 7ffaa3fff700 1 --2- 172.21.15.107:0/2403495900 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ffaa40a2cb0 0x7ffaa40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:43.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.213+0000 7ffabaffd700 1 --2- 172.21.15.107:0/2403495900 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ffaa40a2cb0 0x7ffaa40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:43.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.213+0000 7ffaa3fff700 1 -- 172.21.15.107:0/2403495900 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7ffab40cebf0 con 0x7ffabc107670 2024-04-25T00:14:43.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.214+0000 7ffabaffd700 1 --2- 172.21.15.107:0/2403495900 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ffaa40a2cb0 0x7ffaa40a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffab0006fd0 tx=0x7ffab0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:43.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.217+0000 7ffaa3fff700 1 -- 172.21.15.107:0/2403495900 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffab409b8c0 con 0x7ffabc107670 2024-04-25T00:14:43.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.412+0000 7ffac128d700 1 -- 172.21.15.107:0/2403495900 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffabc10a200 con 0x7ffabc107670 2024-04-25T00:14:43.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.414+0000 7ffaa3fff700 1 -- 172.21.15.107:0/2403495900 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24134 (secure 0 0 0) 0x7ffab409b1c0 con 0x7ffabc107670 2024-04-25T00:14:43.415 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:14:43.415 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":92,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:14:08.839737+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:59.672801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:00.957782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:14:05.065846+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:14:43.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.416+0000 7ffac128d700 1 -- 172.21.15.107:0/2403495900 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ffaa40a2cb0 msgr2=0x7ffaa40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:43.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.416+0000 7ffac128d700 1 --2- 172.21.15.107:0/2403495900 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ffaa40a2cb0 0x7ffaa40a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffab0006fd0 tx=0x7ffab0008040 comp rx=0 tx=0).stop 2024-04-25T00:14:43.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.416+0000 7ffac128d700 1 -- 172.21.15.107:0/2403495900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffabc107670 msgr2=0x7ffabc1988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:43.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.416+0000 7ffac128d700 1 --2- 172.21.15.107:0/2403495900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffabc107670 0x7ffabc1988e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffab400b970 tx=0x7ffab400bd30 comp rx=0 tx=0).stop 2024-04-25T00:14:43.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.416+0000 7ffac128d700 1 -- 172.21.15.107:0/2403495900 shutdown_connections 2024-04-25T00:14:43.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.416+0000 7ffac128d700 1 --2- 172.21.15.107:0/2403495900 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ffaa40a2cb0 0x7ffaa40a5170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.416+0000 7ffac128d700 1 --2- 172.21.15.107:0/2403495900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffabc107670 0x7ffabc1988e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.416+0000 7ffac128d700 1 --2- 172.21.15.107:0/2403495900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffabc100e50 0x7ffabc19e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.416+0000 7ffac128d700 1 --2- 172.21.15.107:0/2403495900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffabc101800 0x7ffabc19ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:43.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.416+0000 7ffac128d700 1 -- 172.21.15.107:0/2403495900 >> 172.21.15.107:0/2403495900 conn(0x7ffabc076450 msgr2=0x7ffabc0fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:43.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.416+0000 7ffac128d700 1 -- 172.21.15.107:0/2403495900 shutdown_connections 2024-04-25T00:14:43.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:43.416+0000 7ffac128d700 1 -- 172.21.15.107:0/2403495900 wait complete. 2024-04-25T00:14:44.095 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T00:14:44.096 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T00:14:44.096 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T00:14:44.096 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T00:14:44.097 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T00:14:44.097 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T00:14:44.097 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T00:14:44.098 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T00:14:44.098 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T00:14:44.099 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T00:14:44.099 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T00:14:44.099 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T00:14:44.140 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:44 smithi107 ceph-mon[29153]: pgmap v313: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-25T00:14:44.141 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:44 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2403495900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:44.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:44 smithi195 ceph-mon[32833]: pgmap v313: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-25T00:14:44.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:44 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2403495900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:44.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:44 smithi170 ceph-mon[32807]: pgmap v313: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-25T00:14:44.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:44 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2403495900' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:14:44.748 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:14:44.748 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:14:44.758 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:14:44.767 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:14:45.590 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:14:46.330 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:46 smithi107 ceph-mon[29153]: pgmap v314: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-25T00:14:46.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:46 smithi195 ceph-mon[32833]: pgmap v314: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-25T00:14:46.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:46 smithi170 ceph-mon[32807]: pgmap v314: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-25T00:14:47.122 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:14:47.122 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:14:47.122 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:14:47.544 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:14:47.544 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:14:47.544 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:14:48.305 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:48 smithi107 ceph-mon[29153]: pgmap v315: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T00:14:48.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:48 smithi195 ceph-mon[32833]: pgmap v315: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T00:14:48.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:48 smithi170 ceph-mon[32807]: pgmap v315: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T00:14:50.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:50 smithi195 ceph-mon[32833]: pgmap v316: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-25T00:14:50.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:50 smithi170 ceph-mon[32807]: pgmap v316: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-25T00:14:50.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:50 smithi107 ceph-mon[29153]: pgmap v316: 129 pgs: 129 active+clean; 637 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-25T00:14:52.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:52 smithi107 ceph-mon[29153]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T00:14:52.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:52 smithi195 ceph-mon[32833]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T00:14:52.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:52 smithi170 ceph-mon[32807]: pgmap v317: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T00:14:54.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:54 smithi195 ceph-mon[32833]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-25T00:14:54.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:54 smithi170 ceph-mon[32807]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-25T00:14:54.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:54 smithi107 ceph-mon[29153]: pgmap v318: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-25T00:14:54.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.622+0000 7ff755646700 1 -- 172.21.15.107:0/3507650206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff750069f40 msgr2=0x7ff75006a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:54.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.622+0000 7ff755646700 1 --2- 172.21.15.107:0/3507650206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff750069f40 0x7ff75006a3a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff744009990 tx=0x7ff744009ca0 comp rx=0 tx=0).stop 2024-04-25T00:14:54.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.622+0000 7ff755646700 1 -- 172.21.15.107:0/3507650206 shutdown_connections 2024-04-25T00:14:54.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.622+0000 7ff755646700 1 --2- 172.21.15.107:0/3507650206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff75006a8e0 0x7ff75010f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:54.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.622+0000 7ff755646700 1 --2- 172.21.15.107:0/3507650206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7501082b0 0x7ff750108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:54.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.622+0000 7ff755646700 1 --2- 172.21.15.107:0/3507650206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff750069f40 0x7ff75006a3a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:54.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.622+0000 7ff755646700 1 -- 172.21.15.107:0/3507650206 >> 172.21.15.107:0/3507650206 conn(0x7ff7500780e0 msgr2=0x7ff7500784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:54.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.622+0000 7ff755646700 1 -- 172.21.15.107:0/3507650206 shutdown_connections 2024-04-25T00:14:54.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.622+0000 7ff755646700 1 -- 172.21.15.107:0/3507650206 wait complete. 2024-04-25T00:14:54.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.622+0000 7ff755646700 1 Processor -- start 2024-04-25T00:14:54.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.623+0000 7ff755646700 1 -- start start 2024-04-25T00:14:54.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.623+0000 7ff755646700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff750069f40 0x7ff750101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:54.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.623+0000 7ff755646700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff75006a8e0 0x7ff750101740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:54.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.623+0000 7ff755646700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7501082b0 0x7ff750101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:54.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.623+0000 7ff755646700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff750112560 con 0x7ff750069f40 2024-04-25T00:14:54.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.623+0000 7ff755646700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff7501123e0 con 0x7ff75006a8e0 2024-04-25T00:14:54.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.623+0000 7ff755646700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff7501126e0 con 0x7ff7501082b0 2024-04-25T00:14:54.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.623+0000 7ff74effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff750069f40 0x7ff750101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:54.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.623+0000 7ff74effd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff750069f40 0x7ff750101200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58928/0 (socket says 172.21.15.107:58928) 2024-04-25T00:14:54.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.623+0000 7ff74effd700 1 -- 172.21.15.107:0/2356452218 learned_addr learned my addr 172.21.15.107:0/2356452218 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:54.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.623+0000 7ff74f7fe700 1 --2- 172.21.15.107:0/2356452218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7501082b0 0x7ff750101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:54.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.623+0000 7ff74e7fc700 1 --2- 172.21.15.107:0/2356452218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff75006a8e0 0x7ff750101740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:54.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.623+0000 7ff74effd700 1 -- 172.21.15.107:0/2356452218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7501082b0 msgr2=0x7ff750101c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:54.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.623+0000 7ff74effd700 1 --2- 172.21.15.107:0/2356452218 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7501082b0 0x7ff750101c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:54.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.623+0000 7ff74effd700 1 -- 172.21.15.107:0/2356452218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff75006a8e0 msgr2=0x7ff750101740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:54.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.623+0000 7ff74effd700 1 --2- 172.21.15.107:0/2356452218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff75006a8e0 0x7ff750101740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:54.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.623+0000 7ff74effd700 1 -- 172.21.15.107:0/2356452218 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7501a7020 con 0x7ff750069f40 2024-04-25T00:14:54.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.623+0000 7ff74e7fc700 1 --2- 172.21.15.107:0/2356452218 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff75006a8e0 0x7ff750101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:54.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.623+0000 7ff74effd700 1 --2- 172.21.15.107:0/2356452218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff750069f40 0x7ff750101200 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff74000d950 tx=0x7ff74000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:54.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.624+0000 7ff737fff700 1 -- 172.21.15.107:0/2356452218 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740016070 con 0x7ff750069f40 2024-04-25T00:14:54.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.624+0000 7ff755646700 1 -- 172.21.15.107:0/2356452218 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7501a7310 con 0x7ff750069f40 2024-04-25T00:14:54.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.624+0000 7ff737fff700 1 -- 172.21.15.107:0/2356452218 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff740009b40 con 0x7ff750069f40 2024-04-25T00:14:54.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.624+0000 7ff737fff700 1 -- 172.21.15.107:0/2356452218 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74000f620 con 0x7ff750069f40 2024-04-25T00:14:54.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.624+0000 7ff755646700 1 -- 172.21.15.107:0/2356452218 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7501a78a0 con 0x7ff750069f40 2024-04-25T00:14:54.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.624+0000 7ff755646700 1 -- 172.21.15.107:0/2356452218 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff750062940 con 0x7ff750069f40 2024-04-25T00:14:54.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.626+0000 7ff737fff700 1 -- 172.21.15.107:0/2356452218 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff740019070 con 0x7ff750069f40 2024-04-25T00:14:54.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.626+0000 7ff737fff700 1 --2- 172.21.15.107:0/2356452218 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff7380a2cb0 0x7ff7380a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:54.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.626+0000 7ff74e7fc700 1 --2- 172.21.15.107:0/2356452218 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff7380a2cb0 0x7ff7380a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:54.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.626+0000 7ff737fff700 1 -- 172.21.15.107:0/2356452218 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7ff7400cf4c0 con 0x7ff750069f40 2024-04-25T00:14:54.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.627+0000 7ff737fff700 1 --2- 172.21.15.107:0/2356452218 >> [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415] conn(0x7ff7380ad970 0x7ff7380afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:54.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.627+0000 7ff737fff700 1 -- 172.21.15.107:0/2356452218 --> [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff740014e80 con 0x7ff7380ad970 2024-04-25T00:14:54.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.627+0000 7ff737fff700 1 -- 172.21.15.107:0/2356452218 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7ff7400cf910 con 0x7ff750069f40 2024-04-25T00:14:54.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.627+0000 7ff74f7fe700 1 --2- 172.21.15.107:0/2356452218 >> [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415] conn(0x7ff7380ad970 0x7ff7380afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:54.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.627+0000 7ff74e7fc700 1 --2- 172.21.15.107:0/2356452218 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff7380a2cb0 0x7ff7380a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff74400b5c0 tx=0x7ff744005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:54.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.627+0000 7ff74f7fe700 1 --2- 172.21.15.107:0/2356452218 >> [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415] conn(0x7ff7380ad970 0x7ff7380afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:54.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.628+0000 7ff737fff700 1 -- 172.21.15.107:0/2356452218 <== osd.6 v2:172.21.15.107:6818/251620415 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff740014e80 con 0x7ff7380ad970 2024-04-25T00:14:54.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.646+0000 7ff755646700 1 -- 172.21.15.107:0/2356452218 --> [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff750062d40 con 0x7ff7380ad970 2024-04-25T00:14:54.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.646+0000 7ff737fff700 1 -- 172.21.15.107:0/2356452218 <== osd.6 v2:172.21.15.107:6818/251620415 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff750062d40 con 0x7ff7380ad970 2024-04-25T00:14:54.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.646+0000 7ff755646700 1 -- 172.21.15.107:0/2356452218 >> [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415] conn(0x7ff7380ad970 msgr2=0x7ff7380afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:54.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.646+0000 7ff755646700 1 --2- 172.21.15.107:0/2356452218 >> [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415] conn(0x7ff7380ad970 0x7ff7380afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:54.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.646+0000 7ff755646700 1 -- 172.21.15.107:0/2356452218 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff7380a2cb0 msgr2=0x7ff7380a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:54.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.646+0000 7ff755646700 1 --2- 172.21.15.107:0/2356452218 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff7380a2cb0 0x7ff7380a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff74400b5c0 tx=0x7ff744005fd0 comp rx=0 tx=0).stop 2024-04-25T00:14:54.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.646+0000 7ff755646700 1 -- 172.21.15.107:0/2356452218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff750069f40 msgr2=0x7ff750101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:54.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.646+0000 7ff755646700 1 --2- 172.21.15.107:0/2356452218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff750069f40 0x7ff750101200 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff74000d950 tx=0x7ff74000dd10 comp rx=0 tx=0).stop 2024-04-25T00:14:54.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.646+0000 7ff74f7fe700 1 -- 172.21.15.107:0/2356452218 reap_dead start 2024-04-25T00:14:54.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.646+0000 7ff755646700 1 -- 172.21.15.107:0/2356452218 shutdown_connections 2024-04-25T00:14:54.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.646+0000 7ff755646700 1 -- 172.21.15.107:0/2356452218 >> 172.21.15.107:0/2356452218 conn(0x7ff7500780e0 msgr2=0x7ff7500fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:54.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.647+0000 7ff755646700 1 -- 172.21.15.107:0/2356452218 shutdown_connections 2024-04-25T00:14:54.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:54.647+0000 7ff755646700 1 -- 172.21.15.107:0/2356452218 wait complete. 2024-04-25T00:14:56.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:56 smithi107 ceph-mon[29153]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-25T00:14:56.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:56 smithi195 ceph-mon[32833]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-25T00:14:56.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:56 smithi170 ceph-mon[32807]: pgmap v319: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-25T00:14:56.654 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:14:57.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.346+0000 7f15c2dea700 1 -- 172.21.15.107:0/1906961292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15bc1022b0 msgr2=0x7f15bc102710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.346+0000 7f15c2dea700 1 --2- 172.21.15.107:0/1906961292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15bc1022b0 0x7f15bc102710 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009990 tx=0x7f15a8009ca0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.346+0000 7f15c2dea700 1 -- 172.21.15.107:0/1906961292 shutdown_connections 2024-04-25T00:14:57.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.346+0000 7f15c2dea700 1 --2- 172.21.15.107:0/1906961292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15bc102c50 0x7f15bc10f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.346+0000 7f15c2dea700 1 --2- 172.21.15.107:0/1906961292 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15bc1082f0 0x7f15bc1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.346+0000 7f15c2dea700 1 --2- 172.21.15.107:0/1906961292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15bc1022b0 0x7f15bc102710 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.346+0000 7f15c2dea700 1 -- 172.21.15.107:0/1906961292 >> 172.21.15.107:0/1906961292 conn(0x7f15bc0793e0 msgr2=0x7f15bc0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:57.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.346+0000 7f15c2dea700 1 -- 172.21.15.107:0/1906961292 shutdown_connections 2024-04-25T00:14:57.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.346+0000 7f15c2dea700 1 -- 172.21.15.107:0/1906961292 wait complete. 2024-04-25T00:14:57.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.347+0000 7f15c2dea700 1 Processor -- start 2024-04-25T00:14:57.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.347+0000 7f15c2dea700 1 -- start start 2024-04-25T00:14:57.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.347+0000 7f15c2dea700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15bc1022b0 0x7f15bc19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.347+0000 7f15c2dea700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15bc102c50 0x7f15bc19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.347+0000 7f15c2dea700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15bc1082f0 0x7f15bc198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.347+0000 7f15c2dea700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc112550 con 0x7f15bc1082f0 2024-04-25T00:14:57.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.347+0000 7f15c2dea700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc1123d0 con 0x7f15bc1022b0 2024-04-25T00:14:57.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.347+0000 7f15c2dea700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc1126d0 con 0x7f15bc102c50 2024-04-25T00:14:57.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.347+0000 7f15c1387700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15bc1082f0 0x7f15bc198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.347+0000 7f15c1387700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15bc1082f0 0x7f15bc198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58958/0 (socket says 172.21.15.107:58958) 2024-04-25T00:14:57.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.347+0000 7f15c1387700 1 -- 172.21.15.107:0/1930724429 learned_addr learned my addr 172.21.15.107:0/1930724429 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:57.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.347+0000 7f15c0b86700 1 --2- 172.21.15.107:0/1930724429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15bc1022b0 0x7f15bc19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.347+0000 7f15bbfff700 1 --2- 172.21.15.107:0/1930724429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15bc102c50 0x7f15bc19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.347+0000 7f15c1387700 1 -- 172.21.15.107:0/1930724429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15bc102c50 msgr2=0x7f15bc19ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.347+0000 7f15c1387700 1 --2- 172.21.15.107:0/1930724429 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15bc102c50 0x7f15bc19ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.347+0000 7f15c1387700 1 -- 172.21.15.107:0/1930724429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15bc1022b0 msgr2=0x7f15bc19e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.347+0000 7f15c1387700 1 --2- 172.21.15.107:0/1930724429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15bc1022b0 0x7f15bc19e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.347+0000 7f15c1387700 1 -- 172.21.15.107:0/1930724429 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15bc198ff0 con 0x7f15bc1082f0 2024-04-25T00:14:57.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.347+0000 7f15c1387700 1 --2- 172.21.15.107:0/1930724429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15bc1082f0 0x7f15bc198810 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f15b4007f10 tx=0x7f15b400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:57.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.348+0000 7f15b9ffb700 1 -- 172.21.15.107:0/1930724429 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4009e20 con 0x7f15bc1082f0 2024-04-25T00:14:57.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.348+0000 7f15b9ffb700 1 -- 172.21.15.107:0/1930724429 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15b400dbd0 con 0x7f15bc1082f0 2024-04-25T00:14:57.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.348+0000 7f15b9ffb700 1 -- 172.21.15.107:0/1930724429 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4005a60 con 0x7f15bc1082f0 2024-04-25T00:14:57.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.348+0000 7f15c2dea700 1 -- 172.21.15.107:0/1930724429 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15bc1992e0 con 0x7f15bc1082f0 2024-04-25T00:14:57.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.348+0000 7f15c2dea700 1 -- 172.21.15.107:0/1930724429 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15bc1a54f0 con 0x7f15bc1082f0 2024-04-25T00:14:57.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.349+0000 7f15c2dea700 1 -- 172.21.15.107:0/1930724429 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f15bc0629c0 con 0x7f15bc1082f0 2024-04-25T00:14:57.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.350+0000 7f15b9ffb700 1 -- 172.21.15.107:0/1930724429 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f15b401a020 con 0x7f15bc1082f0 2024-04-25T00:14:57.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.350+0000 7f15b9ffb700 1 --2- 172.21.15.107:0/1930724429 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f15ac0a2cb0 0x7f15ac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.350+0000 7f15c0b86700 1 --2- 172.21.15.107:0/1930724429 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f15ac0a2cb0 0x7f15ac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.351+0000 7f15b9ffb700 1 -- 172.21.15.107:0/1930724429 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f15b40cf200 con 0x7f15bc1082f0 2024-04-25T00:14:57.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.351+0000 7f15b9ffb700 1 --2- 172.21.15.107:0/1930724429 >> [v2:172.21.15.195:6800/90978480,v1:172.21.15.195:6801/90978480] conn(0x7f15ac0ad970 0x7f15ac0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.351+0000 7f15b9ffb700 1 -- 172.21.15.107:0/1930724429 --> [v2:172.21.15.195:6800/90978480,v1:172.21.15.195:6801/90978480] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f15ac0b0480 con 0x7f15ac0ad970 2024-04-25T00:14:57.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.351+0000 7f15b9ffb700 1 -- 172.21.15.107:0/1930724429 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f15b40cf6e0 con 0x7f15bc1082f0 2024-04-25T00:14:57.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.351+0000 7f15c0b86700 1 --2- 172.21.15.107:0/1930724429 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f15ac0a2cb0 0x7f15ac0a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f15b0005950 tx=0x7f15b000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:57.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.351+0000 7f15bbfff700 1 --2- 172.21.15.107:0/1930724429 >> [v2:172.21.15.195:6800/90978480,v1:172.21.15.195:6801/90978480] conn(0x7f15ac0ad970 0x7f15ac0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.351+0000 7f15bbfff700 1 --2- 172.21.15.107:0/1930724429 >> [v2:172.21.15.195:6800/90978480,v1:172.21.15.195:6801/90978480] conn(0x7f15ac0ad970 0x7f15ac0afdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:57.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.352+0000 7f15b9ffb700 1 -- 172.21.15.107:0/1930724429 <== osd.2 v2:172.21.15.195:6800/90978480 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f15ac0b0480 con 0x7f15ac0ad970 2024-04-25T00:14:57.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.370+0000 7f15c2dea700 1 -- 172.21.15.107:0/1930724429 --> [v2:172.21.15.195:6800/90978480,v1:172.21.15.195:6801/90978480] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f15bc062e40 con 0x7f15ac0ad970 2024-04-25T00:14:57.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.371+0000 7f15b9ffb700 1 -- 172.21.15.107:0/1930724429 <== osd.2 v2:172.21.15.195:6800/90978480 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f15bc062e40 con 0x7f15ac0ad970 2024-04-25T00:14:57.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.371+0000 7f15c2dea700 1 -- 172.21.15.107:0/1930724429 >> [v2:172.21.15.195:6800/90978480,v1:172.21.15.195:6801/90978480] conn(0x7f15ac0ad970 msgr2=0x7f15ac0afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.372+0000 7f15c2dea700 1 --2- 172.21.15.107:0/1930724429 >> [v2:172.21.15.195:6800/90978480,v1:172.21.15.195:6801/90978480] conn(0x7f15ac0ad970 0x7f15ac0afdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.372+0000 7f15c2dea700 1 -- 172.21.15.107:0/1930724429 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f15ac0a2cb0 msgr2=0x7f15ac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.372+0000 7f15c2dea700 1 --2- 172.21.15.107:0/1930724429 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f15ac0a2cb0 0x7f15ac0a5170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f15b0005950 tx=0x7f15b000a300 comp rx=0 tx=0).stop 2024-04-25T00:14:57.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.372+0000 7f15c2dea700 1 -- 172.21.15.107:0/1930724429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15bc1082f0 msgr2=0x7f15bc198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.372+0000 7f15c2dea700 1 --2- 172.21.15.107:0/1930724429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15bc1082f0 0x7f15bc198810 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f15b4007f10 tx=0x7f15b400d3b0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.372+0000 7f15c1387700 1 -- 172.21.15.107:0/1930724429 reap_dead start 2024-04-25T00:14:57.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.372+0000 7f15c2dea700 1 -- 172.21.15.107:0/1930724429 shutdown_connections 2024-04-25T00:14:57.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.372+0000 7f15c2dea700 1 -- 172.21.15.107:0/1930724429 >> 172.21.15.107:0/1930724429 conn(0x7f15bc0793e0 msgr2=0x7f15bc100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:57.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.372+0000 7f15c2dea700 1 -- 172.21.15.107:0/1930724429 shutdown_connections 2024-04-25T00:14:57.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:56.372+0000 7f15c2dea700 1 -- 172.21.15.107:0/1930724429 wait complete. 2024-04-25T00:14:57.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.430+0000 7f07be6fa700 1 -- 172.21.15.107:0/869503489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07b8107670 msgr2=0x7f07b810f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.430+0000 7f07be6fa700 1 --2- 172.21.15.107:0/869503489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07b8107670 0x7f07b810f1d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f07b000b0d0 tx=0x7f07b000b3e0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.430+0000 7f07be6fa700 1 -- 172.21.15.107:0/869503489 shutdown_connections 2024-04-25T00:14:57.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.430+0000 7f07be6fa700 1 --2- 172.21.15.107:0/869503489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07b8107670 0x7f07b810f1d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.430+0000 7f07be6fa700 1 --2- 172.21.15.107:0/869503489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07b8100e50 0x7f07b8101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.430+0000 7f07be6fa700 1 --2- 172.21.15.107:0/869503489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07b8101800 0x7f07b8106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.430+0000 7f07be6fa700 1 -- 172.21.15.107:0/869503489 >> 172.21.15.107:0/869503489 conn(0x7f07b8076450 msgr2=0x7f07b8076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:57.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.430+0000 7f07be6fa700 1 -- 172.21.15.107:0/869503489 shutdown_connections 2024-04-25T00:14:57.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.430+0000 7f07be6fa700 1 -- 172.21.15.107:0/869503489 wait complete. 2024-04-25T00:14:57.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.431+0000 7f07be6fa700 1 Processor -- start 2024-04-25T00:14:57.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.431+0000 7f07be6fa700 1 -- start start 2024-04-25T00:14:57.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.431+0000 7f07be6fa700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07b8100e50 0x7f07b819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.431+0000 7f07be6fa700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07b8101800 0x7f07b819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.431+0000 7f07be6fa700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07b8107670 0x7f07b8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.432+0000 7f07be6fa700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f07b8112660 con 0x7f07b8101800 2024-04-25T00:14:57.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.432+0000 7f07be6fa700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81124e0 con 0x7f07b8107670 2024-04-25T00:14:57.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.432+0000 7f07be6fa700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f07b81127e0 con 0x7f07b8100e50 2024-04-25T00:14:57.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.432+0000 7f07b77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07b8101800 0x7f07b819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.432+0000 7f07b77fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07b8101800 0x7f07b819eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58988/0 (socket says 172.21.15.107:58988) 2024-04-25T00:14:57.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.432+0000 7f07b77fe700 1 -- 172.21.15.107:0/333080366 learned_addr learned my addr 172.21.15.107:0/333080366 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:57.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.432+0000 7f07b7fff700 1 --2- 172.21.15.107:0/333080366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07b8100e50 0x7f07b819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.432+0000 7f07bcc97700 1 --2- 172.21.15.107:0/333080366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07b8107670 0x7f07b8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.432+0000 7f07b77fe700 1 -- 172.21.15.107:0/333080366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07b8100e50 msgr2=0x7f07b819e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.432+0000 7f07b77fe700 1 --2- 172.21.15.107:0/333080366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07b8100e50 0x7f07b819e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.432+0000 7f07b77fe700 1 -- 172.21.15.107:0/333080366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07b8107670 msgr2=0x7f07b8198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.432+0000 7f07b77fe700 1 --2- 172.21.15.107:0/333080366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f07b8107670 0x7f07b8198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.432+0000 7f07b77fe700 1 -- 172.21.15.107:0/333080366 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b8199080 con 0x7f07b8101800 2024-04-25T00:14:57.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.432+0000 7f07b77fe700 1 --2- 172.21.15.107:0/333080366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07b8101800 0x7f07b819eca0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00b530 tx=0x7f07ac00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:57.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.432+0000 7f07b57fa700 1 -- 172.21.15.107:0/333080366 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac015020 con 0x7f07b8101800 2024-04-25T00:14:57.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.432+0000 7f07b57fa700 1 -- 172.21.15.107:0/333080366 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07ac011d10 con 0x7f07b8101800 2024-04-25T00:14:57.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.432+0000 7f07be6fa700 1 -- 172.21.15.107:0/333080366 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b8199370 con 0x7f07b8101800 2024-04-25T00:14:57.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.433+0000 7f07b57fa700 1 -- 172.21.15.107:0/333080366 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac00f2e0 con 0x7f07b8101800 2024-04-25T00:14:57.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.433+0000 7f07be6fa700 1 -- 172.21.15.107:0/333080366 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b81a5630 con 0x7f07b8101800 2024-04-25T00:14:57.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.433+0000 7f07be6fa700 1 -- 172.21.15.107:0/333080366 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f07b80629c0 con 0x7f07b8101800 2024-04-25T00:14:57.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.435+0000 7f07b57fa700 1 -- 172.21.15.107:0/333080366 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f07ac018020 con 0x7f07b8101800 2024-04-25T00:14:57.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.436+0000 7f07b57fa700 1 --2- 172.21.15.107:0/333080366 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f07a00a2cb0 0x7f07a00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.436+0000 7f07b7fff700 1 --2- 172.21.15.107:0/333080366 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f07a00a2cb0 0x7f07a00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.436+0000 7f07b57fa700 1 -- 172.21.15.107:0/333080366 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f07ac0cf100 con 0x7f07b8101800 2024-04-25T00:14:57.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.436+0000 7f07b57fa700 1 --2- 172.21.15.107:0/333080366 >> [v2:172.21.15.195:6816/2391866428,v1:172.21.15.195:6817/2391866428] conn(0x7f07a00ad970 0x7f07a00afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.436+0000 7f07b57fa700 1 -- 172.21.15.107:0/333080366 --> [v2:172.21.15.195:6816/2391866428,v1:172.21.15.195:6817/2391866428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f07a00b0480 con 0x7f07a00ad970 2024-04-25T00:14:57.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.436+0000 7f07b57fa700 1 -- 172.21.15.107:0/333080366 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f07ac0cf4b0 con 0x7f07b8101800 2024-04-25T00:14:57.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.436+0000 7f07b7fff700 1 --2- 172.21.15.107:0/333080366 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f07a00a2cb0 0x7f07a00a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f07b8199a60 tx=0x7f07a8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:57.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.437+0000 7f07bcc97700 1 --2- 172.21.15.107:0/333080366 >> [v2:172.21.15.195:6816/2391866428,v1:172.21.15.195:6817/2391866428] conn(0x7f07a00ad970 0x7f07a00afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.437+0000 7f07bcc97700 1 --2- 172.21.15.107:0/333080366 >> [v2:172.21.15.195:6816/2391866428,v1:172.21.15.195:6817/2391866428] conn(0x7f07a00ad970 0x7f07a00afdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:57.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.438+0000 7f07b57fa700 1 -- 172.21.15.107:0/333080366 <== osd.8 v2:172.21.15.195:6816/2391866428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f07a00b0480 con 0x7f07a00ad970 2024-04-25T00:14:57.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.454+0000 7f07be6fa700 1 -- 172.21.15.107:0/333080366 --> [v2:172.21.15.195:6816/2391866428,v1:172.21.15.195:6817/2391866428] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f07b8062e40 con 0x7f07a00ad970 2024-04-25T00:14:57.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.455+0000 7f07b57fa700 1 -- 172.21.15.107:0/333080366 <== osd.8 v2:172.21.15.195:6816/2391866428 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f07b8062e40 con 0x7f07a00ad970 2024-04-25T00:14:57.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.455+0000 7f07be6fa700 1 -- 172.21.15.107:0/333080366 >> [v2:172.21.15.195:6816/2391866428,v1:172.21.15.195:6817/2391866428] conn(0x7f07a00ad970 msgr2=0x7f07a00afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.455+0000 7f07be6fa700 1 --2- 172.21.15.107:0/333080366 >> [v2:172.21.15.195:6816/2391866428,v1:172.21.15.195:6817/2391866428] conn(0x7f07a00ad970 0x7f07a00afdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.455+0000 7f07be6fa700 1 -- 172.21.15.107:0/333080366 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f07a00a2cb0 msgr2=0x7f07a00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.455+0000 7f07be6fa700 1 --2- 172.21.15.107:0/333080366 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f07a00a2cb0 0x7f07a00a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f07b8199a60 tx=0x7f07a8009190 comp rx=0 tx=0).stop 2024-04-25T00:14:57.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.455+0000 7f07be6fa700 1 -- 172.21.15.107:0/333080366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07b8101800 msgr2=0x7f07b819eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.455+0000 7f07be6fa700 1 --2- 172.21.15.107:0/333080366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07b8101800 0x7f07b819eca0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00b530 tx=0x7f07ac00b840 comp rx=0 tx=0).stop 2024-04-25T00:14:57.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.455+0000 7f07bcc97700 1 -- 172.21.15.107:0/333080366 reap_dead start 2024-04-25T00:14:57.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.455+0000 7f07be6fa700 1 -- 172.21.15.107:0/333080366 shutdown_connections 2024-04-25T00:14:57.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.455+0000 7f07be6fa700 1 -- 172.21.15.107:0/333080366 >> 172.21.15.107:0/333080366 conn(0x7f07b8076450 msgr2=0x7f07b80fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:57.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.455+0000 7f07be6fa700 1 -- 172.21.15.107:0/333080366 shutdown_connections 2024-04-25T00:14:57.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.455+0000 7f07be6fa700 1 -- 172.21.15.107:0/333080366 wait complete. 2024-04-25T00:14:57.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.664+0000 7f800cca7700 1 -- 172.21.15.107:0/1704621898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8008101dd0 msgr2=0x7f8008105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.664+0000 7f800cca7700 1 --2- 172.21.15.107:0/1704621898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8008101dd0 0x7f8008105d90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0099c0 tx=0x7f7ffc009cd0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.664+0000 7f800cca7700 1 -- 172.21.15.107:0/1704621898 shutdown_connections 2024-04-25T00:14:57.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.665+0000 7f800cca7700 1 --2- 172.21.15.107:0/1704621898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8008106470 0x7f800810e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.665+0000 7f800cca7700 1 --2- 172.21.15.107:0/1704621898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8008101420 0x7f8008101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.665+0000 7f800cca7700 1 --2- 172.21.15.107:0/1704621898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8008101dd0 0x7f8008105d90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.665+0000 7f800cca7700 1 -- 172.21.15.107:0/1704621898 >> 172.21.15.107:0/1704621898 conn(0x7f8008077180 msgr2=0x7f8008077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:57.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.665+0000 7f800cca7700 1 -- 172.21.15.107:0/1704621898 shutdown_connections 2024-04-25T00:14:57.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.665+0000 7f800cca7700 1 -- 172.21.15.107:0/1704621898 wait complete. 2024-04-25T00:14:57.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.665+0000 7f800cca7700 1 Processor -- start 2024-04-25T00:14:57.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.665+0000 7f800cca7700 1 -- start start 2024-04-25T00:14:57.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.666+0000 7f800cca7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8008101420 0x7f8008102e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.666+0000 7f800cca7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8008101dd0 0x7f8008103370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.666+0000 7f800cca7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8008106470 0x7f80081038b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.666+0000 7f800cca7700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f80081114d0 con 0x7f8008106470 2024-04-25T00:14:57.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.666+0000 7f800cca7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8008111350 con 0x7f8008101dd0 2024-04-25T00:14:57.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.666+0000 7f800cca7700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8008111650 con 0x7f8008101420 2024-04-25T00:14:57.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.666+0000 7f8006d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8008106470 0x7f80081038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.666+0000 7f800659c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8008101420 0x7f8008102e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.666+0000 7f8006d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8008106470 0x7f80081038b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58998/0 (socket says 172.21.15.107:58998) 2024-04-25T00:14:57.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.666+0000 7f8006d9d700 1 -- 172.21.15.107:0/2415745664 learned_addr learned my addr 172.21.15.107:0/2415745664 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:57.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.666+0000 7f8005d9b700 1 --2- 172.21.15.107:0/2415745664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8008101dd0 0x7f8008103370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.666+0000 7f8006d9d700 1 -- 172.21.15.107:0/2415745664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8008101420 msgr2=0x7f8008102e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.666+0000 7f8006d9d700 1 --2- 172.21.15.107:0/2415745664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8008101420 0x7f8008102e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.666+0000 7f8006d9d700 1 -- 172.21.15.107:0/2415745664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8008101dd0 msgr2=0x7f8008103370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.666+0000 7f8006d9d700 1 --2- 172.21.15.107:0/2415745664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8008101dd0 0x7f8008103370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.666+0000 7f8006d9d700 1 -- 172.21.15.107:0/2415745664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80081a6fa0 con 0x7f8008106470 2024-04-25T00:14:57.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.666+0000 7f8005d9b700 1 --2- 172.21.15.107:0/2415745664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8008101dd0 0x7f8008103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:57.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.666+0000 7f800659c700 1 --2- 172.21.15.107:0/2415745664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8008101420 0x7f8008102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:14:57.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.666+0000 7f8006d9d700 1 --2- 172.21.15.107:0/2415745664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8008106470 0x7f80081038b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f800000b970 tx=0x7f800000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:57.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.666+0000 7f7ff77fe700 1 -- 172.21.15.107:0/2415745664 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800000d860 con 0x7f8008106470 2024-04-25T00:14:57.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.667+0000 7f800cca7700 1 -- 172.21.15.107:0/2415745664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80081a7290 con 0x7f8008106470 2024-04-25T00:14:57.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.667+0000 7f7ff77fe700 1 -- 172.21.15.107:0/2415745664 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f800000e430 con 0x7f8008106470 2024-04-25T00:14:57.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.667+0000 7f7ff77fe700 1 -- 172.21.15.107:0/2415745664 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000017660 con 0x7f8008106470 2024-04-25T00:14:57.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.667+0000 7f800cca7700 1 -- 172.21.15.107:0/2415745664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80081a77a0 con 0x7f8008106470 2024-04-25T00:14:57.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.667+0000 7f800cca7700 1 -- 172.21.15.107:0/2415745664 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f800806ac70 con 0x7f8008106470 2024-04-25T00:14:57.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.669+0000 7f7ff77fe700 1 -- 172.21.15.107:0/2415745664 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f800000e5e0 con 0x7f8008106470 2024-04-25T00:14:57.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.669+0000 7f7ff77fe700 1 --2- 172.21.15.107:0/2415745664 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7ff00a2cb0 0x7f7ff00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.669+0000 7f800659c700 1 --2- 172.21.15.107:0/2415745664 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7ff00a2cb0 0x7f7ff00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.669+0000 7f7ff77fe700 1 -- 172.21.15.107:0/2415745664 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f80000cf430 con 0x7f8008106470 2024-04-25T00:14:57.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.669+0000 7f7ff77fe700 1 --2- 172.21.15.107:0/2415745664 >> [v2:172.21.15.170:6816/3231152590,v1:172.21.15.170:6817/3231152590] conn(0x7f7ff00ad970 0x7f7ff00afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:57.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.669+0000 7f7ff77fe700 1 -- 172.21.15.107:0/2415745664 --> [v2:172.21.15.170:6816/3231152590,v1:172.21.15.170:6817/3231152590] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8000011ec0 con 0x7f7ff00ad970 2024-04-25T00:14:57.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.669+0000 7f7ff77fe700 1 -- 172.21.15.107:0/2415745664 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f80000cf880 con 0x7f8008106470 2024-04-25T00:14:57.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.670+0000 7f800659c700 1 --2- 172.21.15.107:0/2415745664 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7ff00a2cb0 0x7f7ff00a5170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80104e0 tx=0x7f7ff800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:57.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.670+0000 7f8005d9b700 1 --2- 172.21.15.107:0/2415745664 >> [v2:172.21.15.170:6816/3231152590,v1:172.21.15.170:6817/3231152590] conn(0x7f7ff00ad970 0x7f7ff00afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:57.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.670+0000 7f8005d9b700 1 --2- 172.21.15.107:0/2415745664 >> [v2:172.21.15.170:6816/3231152590,v1:172.21.15.170:6817/3231152590] conn(0x7f7ff00ad970 0x7f7ff00afdb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:57.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.671+0000 7f7ff77fe700 1 -- 172.21.15.107:0/2415745664 <== osd.7 v2:172.21.15.170:6816/3231152590 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8000011ec0 con 0x7f7ff00ad970 2024-04-25T00:14:57.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.687+0000 7f800cca7700 1 -- 172.21.15.107:0/2415745664 --> [v2:172.21.15.170:6816/3231152590,v1:172.21.15.170:6817/3231152590] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8008102870 con 0x7f7ff00ad970 2024-04-25T00:14:57.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.687+0000 7f7ff77fe700 1 -- 172.21.15.107:0/2415745664 <== osd.7 v2:172.21.15.170:6816/3231152590 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8008102870 con 0x7f7ff00ad970 2024-04-25T00:14:57.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.688+0000 7f800cca7700 1 -- 172.21.15.107:0/2415745664 >> [v2:172.21.15.170:6816/3231152590,v1:172.21.15.170:6817/3231152590] conn(0x7f7ff00ad970 msgr2=0x7f7ff00afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.688+0000 7f800cca7700 1 --2- 172.21.15.107:0/2415745664 >> [v2:172.21.15.170:6816/3231152590,v1:172.21.15.170:6817/3231152590] conn(0x7f7ff00ad970 0x7f7ff00afdb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.688+0000 7f800cca7700 1 -- 172.21.15.107:0/2415745664 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7ff00a2cb0 msgr2=0x7f7ff00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.688+0000 7f800cca7700 1 --2- 172.21.15.107:0/2415745664 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7ff00a2cb0 0x7f7ff00a5170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80104e0 tx=0x7f7ff800b3c0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.688+0000 7f800cca7700 1 -- 172.21.15.107:0/2415745664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8008106470 msgr2=0x7f80081038b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.688+0000 7f800cca7700 1 --2- 172.21.15.107:0/2415745664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8008106470 0x7f80081038b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f800000b970 tx=0x7f800000bc80 comp rx=0 tx=0).stop 2024-04-25T00:14:57.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.688+0000 7f8006d9d700 1 -- 172.21.15.107:0/2415745664 reap_dead start 2024-04-25T00:14:57.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.688+0000 7f800cca7700 1 -- 172.21.15.107:0/2415745664 shutdown_connections 2024-04-25T00:14:57.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.688+0000 7f800cca7700 1 -- 172.21.15.107:0/2415745664 >> 172.21.15.107:0/2415745664 conn(0x7f8008077180 msgr2=0x7f8008105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:57.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.688+0000 7f800cca7700 1 -- 172.21.15.107:0/2415745664 shutdown_connections 2024-04-25T00:14:57.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:57.688+0000 7f800cca7700 1 -- 172.21.15.107:0/2415745664 wait complete. 2024-04-25T00:14:58.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:14:58 smithi107 ceph-mon[29153]: pgmap v320: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-25T00:14:58.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:14:58 smithi195 ceph-mon[32833]: pgmap v320: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-25T00:14:58.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:14:58 smithi170 ceph-mon[32807]: pgmap v320: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-25T00:14:58.776 INFO:teuthology.orchestra.run.smithi107.stdout:201863462962 2024-04-25T00:14:58.777 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T00:14:59.200 INFO:teuthology.orchestra.run.smithi107.stdout:77309411398 2024-04-25T00:14:59.201 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T00:14:59.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.376+0000 7f819ef9a700 1 -- 172.21.15.107:0/2702176843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81981075b0 msgr2=0x7f819810f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:59.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.376+0000 7f819ef9a700 1 --2- 172.21.15.107:0/2702176843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81981075b0 0x7f819810f180 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f819000b0a0 tx=0x7f819000b3b0 comp rx=0 tx=0).stop 2024-04-25T00:14:59.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.376+0000 7f819ef9a700 1 -- 172.21.15.107:0/2702176843 shutdown_connections 2024-04-25T00:14:59.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.376+0000 7f819ef9a700 1 --2- 172.21.15.107:0/2702176843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81981075b0 0x7f819810f180 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:59.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.376+0000 7f819ef9a700 1 --2- 172.21.15.107:0/2702176843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f819806abc0 0x7f819806afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:59.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.376+0000 7f819ef9a700 1 --2- 172.21.15.107:0/2702176843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f819806b570 0x7f8198106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:59.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.376+0000 7f819ef9a700 1 -- 172.21.15.107:0/2702176843 >> 172.21.15.107:0/2702176843 conn(0x7f81980771d0 msgr2=0x7f81980775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:59.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.376+0000 7f819ef9a700 1 -- 172.21.15.107:0/2702176843 shutdown_connections 2024-04-25T00:14:59.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.376+0000 7f819ef9a700 1 -- 172.21.15.107:0/2702176843 wait complete. 2024-04-25T00:14:59.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.377+0000 7f819ef9a700 1 Processor -- start 2024-04-25T00:14:59.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.377+0000 7f819ef9a700 1 -- start start 2024-04-25T00:14:59.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.377+0000 7f819ef9a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f819806abc0 0x7f819819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:59.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.377+0000 7f819ef9a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819806b570 0x7f819819ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:59.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.377+0000 7f819ef9a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81981075b0 0x7f8198198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:59.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.377+0000 7f819ef9a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8198112590 con 0x7f819806b570 2024-04-25T00:14:59.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.377+0000 7f819ef9a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8198112410 con 0x7f81981075b0 2024-04-25T00:14:59.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.377+0000 7f819ef9a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8198112710 con 0x7f819806abc0 2024-04-25T00:14:59.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.377+0000 7f8197fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819806b570 0x7f819819ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:59.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.377+0000 7f8197fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819806b570 0x7f819819ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59018/0 (socket says 172.21.15.107:59018) 2024-04-25T00:14:59.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.377+0000 7f8197fff700 1 -- 172.21.15.107:0/1227849712 learned_addr learned my addr 172.21.15.107:0/1227849712 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:14:59.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.377+0000 7f819cd36700 1 --2- 172.21.15.107:0/1227849712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f819806abc0 0x7f819819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:59.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.378+0000 7f819d537700 1 --2- 172.21.15.107:0/1227849712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81981075b0 0x7f8198198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:59.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.378+0000 7f8197fff700 1 -- 172.21.15.107:0/1227849712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f819806abc0 msgr2=0x7f819819e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:59.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.378+0000 7f8197fff700 1 --2- 172.21.15.107:0/1227849712 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f819806abc0 0x7f819819e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:59.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.378+0000 7f8197fff700 1 -- 172.21.15.107:0/1227849712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81981075b0 msgr2=0x7f8198198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:59.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.378+0000 7f8197fff700 1 --2- 172.21.15.107:0/1227849712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81981075b0 0x7f8198198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:59.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.378+0000 7f8197fff700 1 -- 172.21.15.107:0/1227849712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8198199030 con 0x7f819806b570 2024-04-25T00:14:59.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.378+0000 7f819d537700 1 --2- 172.21.15.107:0/1227849712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81981075b0 0x7f8198198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:14:59.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.378+0000 7f8197fff700 1 --2- 172.21.15.107:0/1227849712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819806b570 0x7f819819ec10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f818400d950 tx=0x7f818400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:59.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.378+0000 7f8195ffb700 1 -- 172.21.15.107:0/1227849712 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184016070 con 0x7f819806b570 2024-04-25T00:14:59.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.378+0000 7f819ef9a700 1 -- 172.21.15.107:0/1227849712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8198199320 con 0x7f819806b570 2024-04-25T00:14:59.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.378+0000 7f8195ffb700 1 -- 172.21.15.107:0/1227849712 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8184004d10 con 0x7f819806b570 2024-04-25T00:14:59.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.378+0000 7f8195ffb700 1 -- 172.21.15.107:0/1227849712 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81840055e0 con 0x7f819806b570 2024-04-25T00:14:59.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.378+0000 7f819ef9a700 1 -- 172.21.15.107:0/1227849712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81981a5530 con 0x7f819806b570 2024-04-25T00:14:59.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.379+0000 7f819ef9a700 1 -- 172.21.15.107:0/1227849712 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8198062940 con 0x7f819806b570 2024-04-25T00:14:59.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.380+0000 7f8195ffb700 1 -- 172.21.15.107:0/1227849712 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f8184004750 con 0x7f819806b570 2024-04-25T00:14:59.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.381+0000 7f8195ffb700 1 --2- 172.21.15.107:0/1227849712 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f81880a2cb0 0x7f81880a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:59.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.381+0000 7f819cd36700 1 --2- 172.21.15.107:0/1227849712 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f81880a2cb0 0x7f81880a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:59.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.381+0000 7f8195ffb700 1 -- 172.21.15.107:0/1227849712 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f81840cf7d0 con 0x7f819806b570 2024-04-25T00:14:59.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.381+0000 7f8195ffb700 1 --2- 172.21.15.107:0/1227849712 >> [v2:172.21.15.195:6824/645474884,v1:172.21.15.195:6825/645474884] conn(0x7f81880ad970 0x7f81880afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:59.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.381+0000 7f8195ffb700 1 -- 172.21.15.107:0/1227849712 --> [v2:172.21.15.195:6824/645474884,v1:172.21.15.195:6825/645474884] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f818400d130 con 0x7f81880ad970 2024-04-25T00:14:59.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.381+0000 7f8195ffb700 1 -- 172.21.15.107:0/1227849712 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f81840cfc20 con 0x7f819806b570 2024-04-25T00:14:59.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.381+0000 7f819cd36700 1 --2- 172.21.15.107:0/1227849712 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f81880a2cb0 0x7f81880a5170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f818c005950 tx=0x7f818c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:59.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.382+0000 7f819d537700 1 --2- 172.21.15.107:0/1227849712 >> [v2:172.21.15.195:6824/645474884,v1:172.21.15.195:6825/645474884] conn(0x7f81880ad970 0x7f81880afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:59.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.382+0000 7f819d537700 1 --2- 172.21.15.107:0/1227849712 >> [v2:172.21.15.195:6824/645474884,v1:172.21.15.195:6825/645474884] conn(0x7f81880ad970 0x7f81880afdb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:59.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.383+0000 7f8195ffb700 1 -- 172.21.15.107:0/1227849712 <== osd.11 v2:172.21.15.195:6824/645474884 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f818400d130 con 0x7f81880ad970 2024-04-25T00:14:59.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.399+0000 7f819ef9a700 1 -- 172.21.15.107:0/1227849712 --> [v2:172.21.15.195:6824/645474884,v1:172.21.15.195:6825/645474884] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8198062d40 con 0x7f81880ad970 2024-04-25T00:14:59.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.400+0000 7f8195ffb700 1 -- 172.21.15.107:0/1227849712 <== osd.11 v2:172.21.15.195:6824/645474884 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8198062d40 con 0x7f81880ad970 2024-04-25T00:14:59.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.400+0000 7f819ef9a700 1 -- 172.21.15.107:0/1227849712 >> [v2:172.21.15.195:6824/645474884,v1:172.21.15.195:6825/645474884] conn(0x7f81880ad970 msgr2=0x7f81880afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:59.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.400+0000 7f819ef9a700 1 --2- 172.21.15.107:0/1227849712 >> [v2:172.21.15.195:6824/645474884,v1:172.21.15.195:6825/645474884] conn(0x7f81880ad970 0x7f81880afdb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:59.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.400+0000 7f819ef9a700 1 -- 172.21.15.107:0/1227849712 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f81880a2cb0 msgr2=0x7f81880a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:59.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.400+0000 7f819ef9a700 1 --2- 172.21.15.107:0/1227849712 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f81880a2cb0 0x7f81880a5170 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f818c005950 tx=0x7f818c00a300 comp rx=0 tx=0).stop 2024-04-25T00:14:59.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.400+0000 7f819ef9a700 1 -- 172.21.15.107:0/1227849712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819806b570 msgr2=0x7f819819ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:59.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.400+0000 7f819ef9a700 1 --2- 172.21.15.107:0/1227849712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819806b570 0x7f819819ec10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f818400d950 tx=0x7f818400dc60 comp rx=0 tx=0).stop 2024-04-25T00:14:59.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.400+0000 7f819d537700 1 -- 172.21.15.107:0/1227849712 reap_dead start 2024-04-25T00:14:59.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.400+0000 7f819ef9a700 1 -- 172.21.15.107:0/1227849712 shutdown_connections 2024-04-25T00:14:59.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.400+0000 7f819ef9a700 1 -- 172.21.15.107:0/1227849712 >> 172.21.15.107:0/1227849712 conn(0x7f81980771d0 msgr2=0x7f81980ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:59.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.401+0000 7f819ef9a700 1 -- 172.21.15.107:0/1227849712 shutdown_connections 2024-04-25T00:14:59.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:14:59.401+0000 7f819ef9a700 1 -- 172.21.15.107:0/1227849712 wait complete. 2024-04-25T00:15:00.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:00 smithi107 ceph-mon[29153]: pgmap v321: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-25T00:15:00.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:00 smithi195 ceph-mon[32833]: pgmap v321: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-25T00:15:00.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:00 smithi170 ceph-mon[32807]: pgmap v321: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-25T00:15:00.509 INFO:teuthology.orchestra.run.smithi107.stdout:347892351001 2024-04-25T00:15:00.509 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T00:15:00.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.694+0000 7f9a1d0cc700 1 -- 172.21.15.107:0/1351618024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a18102540 msgr2=0x7f9a18102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:00.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.694+0000 7f9a1d0cc700 1 --2- 172.21.15.107:0/1351618024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a18102540 0x7f9a18102920 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9a08009990 tx=0x7f9a08009ca0 comp rx=0 tx=0).stop 2024-04-25T00:15:00.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.694+0000 7f9a1d0cc700 1 -- 172.21.15.107:0/1351618024 shutdown_connections 2024-04-25T00:15:00.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.694+0000 7f9a1d0cc700 1 --2- 172.21.15.107:0/1351618024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a181075a0 0x7f9a1810f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:00.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.694+0000 7f9a1d0cc700 1 --2- 172.21.15.107:0/1351618024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a18102540 0x7f9a18102920 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:00.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.694+0000 7f9a1d0cc700 1 --2- 172.21.15.107:0/1351618024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a18102ef0 0x7f9a18106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:00.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.694+0000 7f9a1d0cc700 1 -- 172.21.15.107:0/1351618024 >> 172.21.15.107:0/1351618024 conn(0x7f9a18079310 msgr2=0x7f9a18079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:00.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.694+0000 7f9a1d0cc700 1 -- 172.21.15.107:0/1351618024 shutdown_connections 2024-04-25T00:15:00.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.694+0000 7f9a1d0cc700 1 -- 172.21.15.107:0/1351618024 wait complete. 2024-04-25T00:15:00.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.695+0000 7f9a1d0cc700 1 Processor -- start 2024-04-25T00:15:00.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.695+0000 7f9a1d0cc700 1 -- start start 2024-04-25T00:15:00.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.695+0000 7f9a1d0cc700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a18102540 0x7f9a1819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:00.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.695+0000 7f9a1d0cc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a18102ef0 0x7f9a1819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:00.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.695+0000 7f9a1d0cc700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a181075a0 0x7f9a18198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:00.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.695+0000 7f9a1d0cc700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9a18112560 con 0x7f9a18102540 2024-04-25T00:15:00.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.695+0000 7f9a1d0cc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9a181123e0 con 0x7f9a18102ef0 2024-04-25T00:15:00.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.695+0000 7f9a1d0cc700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9a181126e0 con 0x7f9a181075a0 2024-04-25T00:15:00.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.695+0000 7f9a16d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a18102540 0x7f9a1819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:00.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.696+0000 7f9a16d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a18102540 0x7f9a1819e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60908/0 (socket says 172.21.15.107:60908) 2024-04-25T00:15:00.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.696+0000 7f9a16d9d700 1 -- 172.21.15.107:0/2143024427 learned_addr learned my addr 172.21.15.107:0/2143024427 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:00.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.696+0000 7f9a1759e700 1 --2- 172.21.15.107:0/2143024427 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a181075a0 0x7f9a18198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:00.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.696+0000 7f9a16d9d700 1 -- 172.21.15.107:0/2143024427 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a181075a0 msgr2=0x7f9a18198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:00.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.696+0000 7f9a16d9d700 1 --2- 172.21.15.107:0/2143024427 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9a181075a0 0x7f9a18198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:00.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.696+0000 7f9a1659c700 1 --2- 172.21.15.107:0/2143024427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a18102ef0 0x7f9a1819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:00.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.696+0000 7f9a16d9d700 1 -- 172.21.15.107:0/2143024427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a18102ef0 msgr2=0x7f9a1819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:00.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.696+0000 7f9a16d9d700 1 --2- 172.21.15.107:0/2143024427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a18102ef0 0x7f9a1819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:00.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.696+0000 7f9a16d9d700 1 -- 172.21.15.107:0/2143024427 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a18199000 con 0x7f9a18102540 2024-04-25T00:15:00.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.696+0000 7f9a1659c700 1 --2- 172.21.15.107:0/2143024427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9a18102ef0 0x7f9a1819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:00.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.696+0000 7f9a16d9d700 1 --2- 172.21.15.107:0/2143024427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a18102540 0x7f9a1819e650 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9a080038b0 tx=0x7f9a08005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:00.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.696+0000 7f99fffff700 1 -- 172.21.15.107:0/2143024427 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a08027070 con 0x7f9a18102540 2024-04-25T00:15:00.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.696+0000 7f99fffff700 1 -- 172.21.15.107:0/2143024427 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a080040d0 con 0x7f9a18102540 2024-04-25T00:15:00.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.696+0000 7f99fffff700 1 -- 172.21.15.107:0/2143024427 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a08022dd0 con 0x7f9a18102540 2024-04-25T00:15:00.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.696+0000 7f9a1d0cc700 1 -- 172.21.15.107:0/2143024427 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a18199290 con 0x7f9a18102540 2024-04-25T00:15:00.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.696+0000 7f9a1d0cc700 1 -- 172.21.15.107:0/2143024427 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a181a54b0 con 0x7f9a18102540 2024-04-25T00:15:00.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.697+0000 7f9a1d0cc700 1 -- 172.21.15.107:0/2143024427 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9a180629c0 con 0x7f9a18102540 2024-04-25T00:15:00.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.699+0000 7f99fffff700 1 -- 172.21.15.107:0/2143024427 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f9a0802a070 con 0x7f9a18102540 2024-04-25T00:15:00.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.699+0000 7f99fffff700 1 --2- 172.21.15.107:0/2143024427 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f9a000a2d30 0x7f9a000a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:00.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.699+0000 7f99fffff700 1 -- 172.21.15.107:0/2143024427 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f9a080d0a90 con 0x7f9a18102540 2024-04-25T00:15:00.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.699+0000 7f9a1659c700 1 --2- 172.21.15.107:0/2143024427 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f9a000a2d30 0x7f9a000a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:00.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.700+0000 7f99fffff700 1 --2- 172.21.15.107:0/2143024427 >> [v2:172.21.15.170:6800/453760841,v1:172.21.15.170:6801/453760841] conn(0x7f9a000ad9f0 0x7f9a000afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:00.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.700+0000 7f99fffff700 1 -- 172.21.15.107:0/2143024427 --> [v2:172.21.15.170:6800/453760841,v1:172.21.15.170:6801/453760841] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9a08010ec0 con 0x7f9a000ad9f0 2024-04-25T00:15:00.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.700+0000 7f99fffff700 1 -- 172.21.15.107:0/2143024427 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f9a080d0ee0 con 0x7f9a18102540 2024-04-25T00:15:00.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.700+0000 7f9a1659c700 1 --2- 172.21.15.107:0/2143024427 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f9a000a2d30 0x7f9a000a51f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c005fd0 tx=0x7f9a0c005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:00.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.700+0000 7f9a1759e700 1 --2- 172.21.15.107:0/2143024427 >> [v2:172.21.15.170:6800/453760841,v1:172.21.15.170:6801/453760841] conn(0x7f9a000ad9f0 0x7f9a000afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:00.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.700+0000 7f9a1759e700 1 --2- 172.21.15.107:0/2143024427 >> [v2:172.21.15.170:6800/453760841,v1:172.21.15.170:6801/453760841] conn(0x7f9a000ad9f0 0x7f9a000afe30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:00.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.701+0000 7f99fffff700 1 -- 172.21.15.107:0/2143024427 <== osd.1 v2:172.21.15.170:6800/453760841 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9a08010ec0 con 0x7f9a000ad9f0 2024-04-25T00:15:00.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.723+0000 7f9a1d0cc700 1 -- 172.21.15.107:0/2143024427 --> [v2:172.21.15.170:6800/453760841,v1:172.21.15.170:6801/453760841] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9a18062e90 con 0x7f9a000ad9f0 2024-04-25T00:15:00.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.724+0000 7f99fffff700 1 -- 172.21.15.107:0/2143024427 <== osd.1 v2:172.21.15.170:6800/453760841 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9a18062e90 con 0x7f9a000ad9f0 2024-04-25T00:15:00.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.724+0000 7f9a1d0cc700 1 -- 172.21.15.107:0/2143024427 >> [v2:172.21.15.170:6800/453760841,v1:172.21.15.170:6801/453760841] conn(0x7f9a000ad9f0 msgr2=0x7f9a000afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:00.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.724+0000 7f9a1d0cc700 1 --2- 172.21.15.107:0/2143024427 >> [v2:172.21.15.170:6800/453760841,v1:172.21.15.170:6801/453760841] conn(0x7f9a000ad9f0 0x7f9a000afe30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:00.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.724+0000 7f9a1d0cc700 1 -- 172.21.15.107:0/2143024427 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f9a000a2d30 msgr2=0x7f9a000a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:00.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.724+0000 7f9a1d0cc700 1 --2- 172.21.15.107:0/2143024427 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f9a000a2d30 0x7f9a000a51f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c005fd0 tx=0x7f9a0c005e20 comp rx=0 tx=0).stop 2024-04-25T00:15:00.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.724+0000 7f9a1d0cc700 1 -- 172.21.15.107:0/2143024427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a18102540 msgr2=0x7f9a1819e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:00.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.724+0000 7f9a1d0cc700 1 --2- 172.21.15.107:0/2143024427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a18102540 0x7f9a1819e650 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9a080038b0 tx=0x7f9a08005590 comp rx=0 tx=0).stop 2024-04-25T00:15:00.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.724+0000 7f9a1759e700 1 -- 172.21.15.107:0/2143024427 reap_dead start 2024-04-25T00:15:00.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.724+0000 7f9a1d0cc700 1 -- 172.21.15.107:0/2143024427 shutdown_connections 2024-04-25T00:15:00.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.724+0000 7f9a1d0cc700 1 -- 172.21.15.107:0/2143024427 >> 172.21.15.107:0/2143024427 conn(0x7f9a18079310 msgr2=0x7f9a181003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:00.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.724+0000 7f9a1d0cc700 1 -- 172.21.15.107:0/2143024427 shutdown_connections 2024-04-25T00:15:00.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:00.724+0000 7f9a1d0cc700 1 -- 172.21.15.107:0/2143024427 wait complete. 2024-04-25T00:15:01.330 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T00:15:01.330 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-25T00:15:01.330 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-04-25T00:15:01.330 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-25T00:15:01.330 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-25T00:15:01.330 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-25T00:15:01.330 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8, 9, 11]}]: dispatch 2024-04-25T00:15:01.330 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-04-25T00:15:01.331 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:01 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-04-25T00:15:01.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T00:15:01.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-25T00:15:01.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-04-25T00:15:01.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-25T00:15:01.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-25T00:15:01.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-25T00:15:01.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8, 9, 11]}]: dispatch 2024-04-25T00:15:01.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-04-25T00:15:01.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:01 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-04-25T00:15:01.357 INFO:teuthology.orchestra.run.smithi107.stdout:257698037800 2024-04-25T00:15:01.357 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T00:15:01.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T00:15:01.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-25T00:15:01.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-04-25T00:15:01.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]: dispatch 2024-04-25T00:15:01.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-25T00:15:01.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-25T00:15:01.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8, 9, 11]}]: dispatch 2024-04-25T00:15:01.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]: dispatch 2024-04-25T00:15:01.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:01 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-04-25T00:15:01.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.896+0000 7fb7afcae700 1 -- 172.21.15.107:0/2395800604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7a8101120 msgr2=0x7fb7a81015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:01.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.896+0000 7fb7afcae700 1 --2- 172.21.15.107:0/2395800604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7a8101120 0x7fb7a81015a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb798009990 tx=0x7fb798009ca0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.896+0000 7fb7afcae700 1 -- 172.21.15.107:0/2395800604 shutdown_connections 2024-04-25T00:15:01.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.896+0000 7fb7afcae700 1 --2- 172.21.15.107:0/2395800604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7a8101ae0 0x7fb7a810dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.896+0000 7fb7afcae700 1 --2- 172.21.15.107:0/2395800604 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7a8107190 0x7fb7a8107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.896+0000 7fb7afcae700 1 --2- 172.21.15.107:0/2395800604 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7a8101120 0x7fb7a81015a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.896+0000 7fb7afcae700 1 -- 172.21.15.107:0/2395800604 >> 172.21.15.107:0/2395800604 conn(0x7fb7a80780a0 msgr2=0x7fb7a80784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:01.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.896+0000 7fb7afcae700 1 -- 172.21.15.107:0/2395800604 shutdown_connections 2024-04-25T00:15:01.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.896+0000 7fb7afcae700 1 -- 172.21.15.107:0/2395800604 wait complete. 2024-04-25T00:15:01.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.896+0000 7fb7afcae700 1 Processor -- start 2024-04-25T00:15:01.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.897+0000 7fb7afcae700 1 -- start start 2024-04-25T00:15:01.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.897+0000 7fb7afcae700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7a8101120 0x7fb7a8102e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:01.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.897+0000 7fb7afcae700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7a8101ae0 0x7fb7a8103340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:01.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.897+0000 7fb7afcae700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7a8107190 0x7fb7a8103880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:01.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.897+0000 7fb7afcae700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a8111400 con 0x7fb7a8107190 2024-04-25T00:15:01.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.897+0000 7fb7afcae700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a8111280 con 0x7fb7a8101ae0 2024-04-25T00:15:01.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.897+0000 7fb7afcae700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a8111580 con 0x7fb7a8101120 2024-04-25T00:15:01.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.897+0000 7fb7ae24b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7a8107190 0x7fb7a8103880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:01.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.897+0000 7fb7ae24b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7a8107190 0x7fb7a8103880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60940/0 (socket says 172.21.15.107:60940) 2024-04-25T00:15:01.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.897+0000 7fb7ae24b700 1 -- 172.21.15.107:0/2219766339 learned_addr learned my addr 172.21.15.107:0/2219766339 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:01.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.897+0000 7fb7ada4a700 1 --2- 172.21.15.107:0/2219766339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7a8101120 0x7fb7a8102e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:01.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.897+0000 7fb7ad249700 1 --2- 172.21.15.107:0/2219766339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7a8101ae0 0x7fb7a8103340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:01.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.897+0000 7fb7ae24b700 1 -- 172.21.15.107:0/2219766339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7a8101120 msgr2=0x7fb7a8102e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:01.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.897+0000 7fb7ae24b700 1 --2- 172.21.15.107:0/2219766339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7a8101120 0x7fb7a8102e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.897+0000 7fb7ae24b700 1 -- 172.21.15.107:0/2219766339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7a8101ae0 msgr2=0x7fb7a8103340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:01.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.897+0000 7fb7ae24b700 1 --2- 172.21.15.107:0/2219766339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7a8101ae0 0x7fb7a8103340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.897+0000 7fb7ae24b700 1 -- 172.21.15.107:0/2219766339 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a81a6e80 con 0x7fb7a8107190 2024-04-25T00:15:01.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.897+0000 7fb7ada4a700 1 --2- 172.21.15.107:0/2219766339 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7a8101120 0x7fb7a8102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:01.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.897+0000 7fb7ae24b700 1 --2- 172.21.15.107:0/2219766339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7a8107190 0x7fb7a8103880 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000ef30 tx=0x7fb7a000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:01.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.898+0000 7fb79effd700 1 -- 172.21.15.107:0/2219766339 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a00092e0 con 0x7fb7a8107190 2024-04-25T00:15:01.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.898+0000 7fb79effd700 1 -- 172.21.15.107:0/2219766339 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a0004bb0 con 0x7fb7a8107190 2024-04-25T00:15:01.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.898+0000 7fb7afcae700 1 -- 172.21.15.107:0/2219766339 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a81a7110 con 0x7fb7a8107190 2024-04-25T00:15:01.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.898+0000 7fb79effd700 1 -- 172.21.15.107:0/2219766339 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a0004280 con 0x7fb7a8107190 2024-04-25T00:15:01.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.898+0000 7fb7afcae700 1 -- 172.21.15.107:0/2219766339 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a8198690 con 0x7fb7a8107190 2024-04-25T00:15:01.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.899+0000 7fb7afcae700 1 -- 172.21.15.107:0/2219766339 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb7a8105210 con 0x7fb7a8107190 2024-04-25T00:15:01.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.900+0000 7fb79effd700 1 -- 172.21.15.107:0/2219766339 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb7a001b030 con 0x7fb7a8107190 2024-04-25T00:15:01.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.900+0000 7fb79effd700 1 --2- 172.21.15.107:0/2219766339 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb7940ab540 0x7fb7940ada00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:01.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.901+0000 7fb7ada4a700 1 --2- 172.21.15.107:0/2219766339 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb7940ab540 0x7fb7940ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:01.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.901+0000 7fb79effd700 1 -- 172.21.15.107:0/2219766339 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9071+0+0 (secure 0 0 0) 0x7fb7a00cfa00 con 0x7fb7a8107190 2024-04-25T00:15:01.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.901+0000 7fb79effd700 1 --2- 172.21.15.107:0/2219766339 >> [v2:172.21.15.170:6808/3780266808,v1:172.21.15.170:6809/3780266808] conn(0x7fb7940b6620 0x7fb7940b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:01.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.901+0000 7fb79effd700 1 -- 172.21.15.107:0/2219766339 --> [v2:172.21.15.170:6808/3780266808,v1:172.21.15.170:6809/3780266808] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb7a001fe50 con 0x7fb7940b6620 2024-04-25T00:15:01.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.901+0000 7fb79effd700 1 -- 172.21.15.107:0/2219766339 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7fb7a00cfe50 con 0x7fb7a8107190 2024-04-25T00:15:01.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.901+0000 7fb7ada4a700 1 --2- 172.21.15.107:0/2219766339 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb7940ab540 0x7fb7940ada00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400baa0 tx=0x7fb7a4005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:01.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.901+0000 7fb7ad249700 1 --2- 172.21.15.107:0/2219766339 >> [v2:172.21.15.170:6808/3780266808,v1:172.21.15.170:6809/3780266808] conn(0x7fb7940b6620 0x7fb7940b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:01.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.902+0000 7fb7ad249700 1 --2- 172.21.15.107:0/2219766339 >> [v2:172.21.15.170:6808/3780266808,v1:172.21.15.170:6809/3780266808] conn(0x7fb7940b6620 0x7fb7940b8ac0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:01.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.902+0000 7fb79effd700 1 -- 172.21.15.107:0/2219766339 <== osd.4 v2:172.21.15.170:6808/3780266808 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb7a001fe50 con 0x7fb7940b6620 2024-04-25T00:15:01.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.918+0000 7fb7afcae700 1 -- 172.21.15.107:0/2219766339 --> [v2:172.21.15.170:6808/3780266808,v1:172.21.15.170:6809/3780266808] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb7a8102840 con 0x7fb7940b6620 2024-04-25T00:15:01.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.919+0000 7fb79effd700 1 -- 172.21.15.107:0/2219766339 <== osd.4 v2:172.21.15.170:6808/3780266808 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb7a8102840 con 0x7fb7940b6620 2024-04-25T00:15:01.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.919+0000 7fb7afcae700 1 -- 172.21.15.107:0/2219766339 >> [v2:172.21.15.170:6808/3780266808,v1:172.21.15.170:6809/3780266808] conn(0x7fb7940b6620 msgr2=0x7fb7940b8ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:01.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.919+0000 7fb7afcae700 1 --2- 172.21.15.107:0/2219766339 >> [v2:172.21.15.170:6808/3780266808,v1:172.21.15.170:6809/3780266808] conn(0x7fb7940b6620 0x7fb7940b8ac0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.919+0000 7fb7afcae700 1 -- 172.21.15.107:0/2219766339 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb7940ab540 msgr2=0x7fb7940ada00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:01.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.919+0000 7fb7afcae700 1 --2- 172.21.15.107:0/2219766339 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb7940ab540 0x7fb7940ada00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400baa0 tx=0x7fb7a4005db0 comp rx=0 tx=0).stop 2024-04-25T00:15:01.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.919+0000 7fb7afcae700 1 -- 172.21.15.107:0/2219766339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7a8107190 msgr2=0x7fb7a8103880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:01.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.919+0000 7fb7afcae700 1 --2- 172.21.15.107:0/2219766339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7a8107190 0x7fb7a8103880 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000ef30 tx=0x7fb7a000c450 comp rx=0 tx=0).stop 2024-04-25T00:15:01.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.919+0000 7fb7ae24b700 1 -- 172.21.15.107:0/2219766339 reap_dead start 2024-04-25T00:15:01.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.919+0000 7fb7afcae700 1 -- 172.21.15.107:0/2219766339 shutdown_connections 2024-04-25T00:15:01.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.919+0000 7fb7afcae700 1 -- 172.21.15.107:0/2219766339 >> 172.21.15.107:0/2219766339 conn(0x7fb7a80780a0 msgr2=0x7fb7a8100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:01.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.919+0000 7fb7afcae700 1 -- 172.21.15.107:0/2219766339 shutdown_connections 2024-04-25T00:15:01.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.919+0000 7fb7afcae700 1 -- 172.21.15.107:0/2219766339 wait complete. 2024-04-25T00:15:02.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.819+0000 7f425bbe3700 1 -- 172.21.15.107:0/2627626987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4254100820 msgr2=0x7f4254100c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.819+0000 7f425bbe3700 1 --2- 172.21.15.107:0/2627626987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4254100820 0x7f4254100c80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4244009990 tx=0x7f4244009ca0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.819+0000 7f425bbe3700 1 -- 172.21.15.107:0/2627626987 shutdown_connections 2024-04-25T00:15:02.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.819+0000 7f425bbe3700 1 --2- 172.21.15.107:0/2627626987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42541011c0 0x7f425410f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.819+0000 7f425bbe3700 1 --2- 172.21.15.107:0/2627626987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42541082c0 0x7f42541086a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.819+0000 7f425bbe3700 1 --2- 172.21.15.107:0/2627626987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4254100820 0x7f4254100c80 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.819+0000 7f425bbe3700 1 -- 172.21.15.107:0/2627626987 >> 172.21.15.107:0/2627626987 conn(0x7f42540777a0 msgr2=0x7f42540feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:02.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.819+0000 7f425bbe3700 1 -- 172.21.15.107:0/2627626987 shutdown_connections 2024-04-25T00:15:02.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.819+0000 7f425bbe3700 1 -- 172.21.15.107:0/2627626987 wait complete. 2024-04-25T00:15:02.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.819+0000 7f425bbe3700 1 Processor -- start 2024-04-25T00:15:02.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.819+0000 7f425bbe3700 1 -- start start 2024-04-25T00:15:02.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.820+0000 7f425bbe3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4254100820 0x7f4254101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.820+0000 7f425bbe3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42541011c0 0x7f4254101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.820+0000 7f425bbe3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42541082c0 0x7f4254102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.820+0000 7f425bbe3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f42541125c0 con 0x7f42541082c0 2024-04-25T00:15:02.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.820+0000 7f425bbe3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4254112440 con 0x7f4254100820 2024-04-25T00:15:02.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.820+0000 7f425bbe3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4254112740 con 0x7f42541011c0 2024-04-25T00:15:02.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.820+0000 7f425a180700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42541082c0 0x7f4254102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.820+0000 7f425a180700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42541082c0 0x7f4254102480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60920/0 (socket says 172.21.15.107:60920) 2024-04-25T00:15:02.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.820+0000 7f425a180700 1 -- 172.21.15.107:0/1736972490 learned_addr learned my addr 172.21.15.107:0/1736972490 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:02.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.820+0000 7f425917e700 1 --2- 172.21.15.107:0/1736972490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42541011c0 0x7f4254101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.820+0000 7f425997f700 1 --2- 172.21.15.107:0/1736972490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4254100820 0x7f4254101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.820+0000 7f425a180700 1 -- 172.21.15.107:0/1736972490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42541011c0 msgr2=0x7f4254101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.820+0000 7f425a180700 1 --2- 172.21.15.107:0/1736972490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42541011c0 0x7f4254101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.820+0000 7f425a180700 1 -- 172.21.15.107:0/1736972490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4254100820 msgr2=0x7f4254101a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.820+0000 7f425a180700 1 --2- 172.21.15.107:0/1736972490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4254100820 0x7f4254101a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.820+0000 7f425a180700 1 -- 172.21.15.107:0/1736972490 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42541a6fa0 con 0x7f42541082c0 2024-04-25T00:15:02.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.820+0000 7f425917e700 1 --2- 172.21.15.107:0/1736972490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f42541011c0 0x7f4254101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:02.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.820+0000 7f425a180700 1 --2- 172.21.15.107:0/1736972490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42541082c0 0x7f4254102480 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f424c00b970 tx=0x7f424c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:02.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.820+0000 7f424affd700 1 -- 172.21.15.107:0/1736972490 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c019070 con 0x7f42541082c0 2024-04-25T00:15:02.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.821+0000 7f424affd700 1 -- 172.21.15.107:0/1736972490 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f424c014440 con 0x7f42541082c0 2024-04-25T00:15:02.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.821+0000 7f425bbe3700 1 -- 172.21.15.107:0/1736972490 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42541a7290 con 0x7f42541082c0 2024-04-25T00:15:02.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.821+0000 7f424affd700 1 -- 172.21.15.107:0/1736972490 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c0075d0 con 0x7f42541082c0 2024-04-25T00:15:02.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.821+0000 7f425bbe3700 1 -- 172.21.15.107:0/1736972490 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42541a77d0 con 0x7f42541082c0 2024-04-25T00:15:02.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.822+0000 7f425bbe3700 1 -- 172.21.15.107:0/1736972490 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4254062940 con 0x7f42541082c0 2024-04-25T00:15:02.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.823+0000 7f424affd700 1 -- 172.21.15.107:0/1736972490 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f424c01c070 con 0x7f42541082c0 2024-04-25T00:15:02.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.823+0000 7f424affd700 1 --2- 172.21.15.107:0/1736972490 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f42400a2cb0 0x7f42400a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.823+0000 7f425997f700 1 --2- 172.21.15.107:0/1736972490 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f42400a2cb0 0x7f42400a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.823+0000 7f424affd700 1 -- 172.21.15.107:0/1736972490 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9071+0+0 (secure 0 0 0) 0x7f424c0ce840 con 0x7f42541082c0 2024-04-25T00:15:02.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.823+0000 7f424affd700 1 --2- 172.21.15.107:0/1736972490 >> [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257] conn(0x7f42400add90 0x7f42400b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.823+0000 7f424affd700 1 -- 172.21.15.107:0/1736972490 --> [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f42400b0900 con 0x7f42400add90 2024-04-25T00:15:02.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.823+0000 7f424affd700 1 -- 172.21.15.107:0/1736972490 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f424c0d32c0 con 0x7f42541082c0 2024-04-25T00:15:02.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.824+0000 7f425917e700 1 --2- 172.21.15.107:0/1736972490 >> [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257] conn(0x7f42400add90 0x7f42400b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.824+0000 7f425997f700 1 --2- 172.21.15.107:0/1736972490 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f42400a2cb0 0x7f42400a5170 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4250005fd0 tx=0x7f425000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:02.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.824+0000 7f425917e700 1 --2- 172.21.15.107:0/1736972490 >> [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257] conn(0x7f42400add90 0x7f42400b0230 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:02.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.825+0000 7f424affd700 1 -- 172.21.15.107:0/1736972490 <== osd.9 v2:172.21.15.107:6826/174023257 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f42400b0900 con 0x7f42400add90 2024-04-25T00:15:02.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.841+0000 7f425bbe3700 1 -- 172.21.15.107:0/1736972490 --> [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4254062dc0 con 0x7f42400add90 2024-04-25T00:15:02.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.841+0000 7f424affd700 1 -- 172.21.15.107:0/1736972490 <== osd.9 v2:172.21.15.107:6826/174023257 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4254062dc0 con 0x7f42400add90 2024-04-25T00:15:02.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.842+0000 7f425bbe3700 1 -- 172.21.15.107:0/1736972490 >> [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257] conn(0x7f42400add90 msgr2=0x7f42400b0230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.842+0000 7f425bbe3700 1 --2- 172.21.15.107:0/1736972490 >> [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257] conn(0x7f42400add90 0x7f42400b0230 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.842+0000 7f425bbe3700 1 -- 172.21.15.107:0/1736972490 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f42400a2cb0 msgr2=0x7f42400a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.842+0000 7f425bbe3700 1 --2- 172.21.15.107:0/1736972490 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f42400a2cb0 0x7f42400a5170 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4250005fd0 tx=0x7f425000a560 comp rx=0 tx=0).stop 2024-04-25T00:15:02.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.842+0000 7f425bbe3700 1 -- 172.21.15.107:0/1736972490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42541082c0 msgr2=0x7f4254102480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.842+0000 7f425bbe3700 1 --2- 172.21.15.107:0/1736972490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42541082c0 0x7f4254102480 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f424c00b970 tx=0x7f424c00bd30 comp rx=0 tx=0).stop 2024-04-25T00:15:02.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.842+0000 7f425a180700 1 -- 172.21.15.107:0/1736972490 reap_dead start 2024-04-25T00:15:02.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.842+0000 7f425bbe3700 1 -- 172.21.15.107:0/1736972490 shutdown_connections 2024-04-25T00:15:02.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.842+0000 7f425bbe3700 1 -- 172.21.15.107:0/1736972490 >> 172.21.15.107:0/1736972490 conn(0x7f42540777a0 msgr2=0x7f42540feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:02.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.842+0000 7f425bbe3700 1 -- 172.21.15.107:0/1736972490 shutdown_connections 2024-04-25T00:15:02.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:01.842+0000 7f425bbe3700 1 -- 172.21.15.107:0/1736972490 wait complete. 2024-04-25T00:15:02.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:02 smithi107 ceph-mon[29153]: pgmap v322: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T00:15:02.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T00:15:02.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-25T00:15:02.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-04-25T00:15:02.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-25T00:15:02.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-25T00:15:02.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-25T00:15:02.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8, 9, 11]}]': finished 2024-04-25T00:15:02.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-04-25T00:15:02.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-04-25T00:15:02.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:02 smithi107 ceph-mon[29153]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T00:15:02.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:02.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:02 smithi195 ceph-mon[32833]: pgmap v322: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T00:15:02.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T00:15:02.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-25T00:15:02.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-04-25T00:15:02.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-25T00:15:02.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-25T00:15:02.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-25T00:15:02.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8, 9, 11]}]': finished 2024-04-25T00:15:02.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-04-25T00:15:02.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-04-25T00:15:02.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:02 smithi195 ceph-mon[32833]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T00:15:02.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:02.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:02 smithi170 ceph-mon[32807]: pgmap v322: 129 pgs: 129 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T00:15:02.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T00:15:02.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-25T00:15:02.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-04-25T00:15:02.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-04-25T00:15:02.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-25T00:15:02.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-25T00:15:02.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [5, 8, 9, 11]}]': finished 2024-04-25T00:15:02.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 8]}]': finished 2024-04-25T00:15:02.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-04-25T00:15:02.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:02 smithi170 ceph-mon[32807]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T00:15:02.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:03.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:03 smithi107 ceph-mon[29153]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T00:15:03.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:03 smithi107 ceph-mon[29153]: pgmap v325: 129 pgs: 6 peering, 123 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 6 op/s 2024-04-25T00:15:03.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:03 smithi195 ceph-mon[32833]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T00:15:03.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:03 smithi195 ceph-mon[32833]: pgmap v325: 129 pgs: 6 peering, 123 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 6 op/s 2024-04-25T00:15:03.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:03 smithi170 ceph-mon[32807]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T00:15:03.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:03 smithi170 ceph-mon[32807]: pgmap v325: 129 pgs: 6 peering, 123 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 383 B/s wr, 6 op/s 2024-04-25T00:15:03.585 INFO:teuthology.orchestra.run.smithi107.stdout:292057776164 2024-04-25T00:15:03.585 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T00:15:04.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:04 smithi107 ceph-mon[29153]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-04-25T00:15:04.188 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:04 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:15:04.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:04 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:15:04.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:04 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:15:04.189 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:04 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:15:04.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:04 smithi195 ceph-mon[32833]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-04-25T00:15:04.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:04 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:15:04.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:04 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:15:04.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:04 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:15:04.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:04 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:15:04.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:04 smithi170 ceph-mon[32807]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-04-25T00:15:04.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:04 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:15:04.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:04 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:15:04.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:04 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:15:04.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:04 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:15:05.088 INFO:teuthology.orchestra.run.smithi107.stdout:223338299436 2024-04-25T00:15:05.088 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T00:15:05.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.270+0000 7f859181d700 1 -- 172.21.15.107:0/3251482265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f858c102540 msgr2=0x7f858c105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:05.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.270+0000 7f859181d700 1 --2- 172.21.15.107:0/3251482265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f858c102540 0x7f858c105dd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f858000b0d0 tx=0x7f858000b3e0 comp rx=0 tx=0).stop 2024-04-25T00:15:05.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.270+0000 7f859181d700 1 -- 172.21.15.107:0/3251482265 shutdown_connections 2024-04-25T00:15:05.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.270+0000 7f859181d700 1 --2- 172.21.15.107:0/3251482265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f858c1064b0 0x7f858c10e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:05.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.270+0000 7f859181d700 1 --2- 172.21.15.107:0/3251482265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f858c101b90 0x7f858c101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:05.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.270+0000 7f859181d700 1 --2- 172.21.15.107:0/3251482265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f858c102540 0x7f858c105dd0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:05.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.270+0000 7f859181d700 1 -- 172.21.15.107:0/3251482265 >> 172.21.15.107:0/3251482265 conn(0x7f858c0777a0 msgr2=0x7f858c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:05.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.270+0000 7f859181d700 1 -- 172.21.15.107:0/3251482265 shutdown_connections 2024-04-25T00:15:05.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.270+0000 7f859181d700 1 -- 172.21.15.107:0/3251482265 wait complete. 2024-04-25T00:15:05.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.271+0000 7f859181d700 1 Processor -- start 2024-04-25T00:15:05.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.271+0000 7f859181d700 1 -- start start 2024-04-25T00:15:05.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.271+0000 7f859181d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f858c101b90 0x7f858c19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:05.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.271+0000 7f859181d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f858c102540 0x7f858c19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:05.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.271+0000 7f859181d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f858c1064b0 0x7f858c1977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:05.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.272+0000 7f859181d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f858c1114d0 con 0x7f858c101b90 2024-04-25T00:15:05.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.272+0000 7f859181d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f858c111350 con 0x7f858c1064b0 2024-04-25T00:15:05.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.272+0000 7f859181d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f858c111650 con 0x7f858c102540 2024-04-25T00:15:05.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.272+0000 7f858affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f858c101b90 0x7f858c19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:05.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.272+0000 7f858affd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f858c101b90 0x7f858c19d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60962/0 (socket says 172.21.15.107:60962) 2024-04-25T00:15:05.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.272+0000 7f858affd700 1 -- 172.21.15.107:0/311606036 learned_addr learned my addr 172.21.15.107:0/311606036 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:05.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.272+0000 7f858a7fc700 1 --2- 172.21.15.107:0/311606036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f858c102540 0x7f858c19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:05.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.272+0000 7f858b7fe700 1 --2- 172.21.15.107:0/311606036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f858c1064b0 0x7f858c1977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:05.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.272+0000 7f858affd700 1 -- 172.21.15.107:0/311606036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f858c102540 msgr2=0x7f858c19db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:05.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.272+0000 7f858affd700 1 --2- 172.21.15.107:0/311606036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f858c102540 0x7f858c19db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:05.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.272+0000 7f858affd700 1 -- 172.21.15.107:0/311606036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f858c1064b0 msgr2=0x7f858c1977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:05.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.272+0000 7f858affd700 1 --2- 172.21.15.107:0/311606036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f858c1064b0 0x7f858c1977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:05.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.272+0000 7f858affd700 1 -- 172.21.15.107:0/311606036 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858c197f90 con 0x7f858c101b90 2024-04-25T00:15:05.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.272+0000 7f858b7fe700 1 --2- 172.21.15.107:0/311606036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f858c1064b0 0x7f858c1977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:05.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.272+0000 7f858affd700 1 --2- 172.21.15.107:0/311606036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f858c101b90 0x7f858c19d630 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f857c00ea40 tx=0x7f857c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:05.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.272+0000 7f859081b700 1 -- 172.21.15.107:0/311606036 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c013be0 con 0x7f858c101b90 2024-04-25T00:15:05.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.272+0000 7f859181d700 1 -- 172.21.15.107:0/311606036 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f858c198280 con 0x7f858c101b90 2024-04-25T00:15:05.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.272+0000 7f859081b700 1 -- 172.21.15.107:0/311606036 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f857c013d80 con 0x7f858c101b90 2024-04-25T00:15:05.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.272+0000 7f859081b700 1 -- 172.21.15.107:0/311606036 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857c005020 con 0x7f858c101b90 2024-04-25T00:15:05.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.273+0000 7f859181d700 1 -- 172.21.15.107:0/311606036 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f858c100820 con 0x7f858c101b90 2024-04-25T00:15:05.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.273+0000 7f859181d700 1 -- 172.21.15.107:0/311606036 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f858c100dd0 con 0x7f858c101b90 2024-04-25T00:15:05.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.275+0000 7f859081b700 1 -- 172.21.15.107:0/311606036 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f857c0051c0 con 0x7f858c101b90 2024-04-25T00:15:05.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.276+0000 7f859081b700 1 --2- 172.21.15.107:0/311606036 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f85740a2cb0 0x7f85740a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:05.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.276+0000 7f858a7fc700 1 --2- 172.21.15.107:0/311606036 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f85740a2cb0 0x7f85740a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:05.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.276+0000 7f859081b700 1 -- 172.21.15.107:0/311606036 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f857c0d3eb0 con 0x7f858c101b90 2024-04-25T00:15:05.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.276+0000 7f859081b700 1 --2- 172.21.15.107:0/311606036 >> [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444] conn(0x7f85740add90 0x7f85740b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:05.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.276+0000 7f859081b700 1 -- 172.21.15.107:0/311606036 --> [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f857c01de70 con 0x7f85740add90 2024-04-25T00:15:05.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.276+0000 7f859081b700 1 -- 172.21.15.107:0/311606036 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f857c0d4300 con 0x7f858c101b90 2024-04-25T00:15:05.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.276+0000 7f858b7fe700 1 --2- 172.21.15.107:0/311606036 >> [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444] conn(0x7f85740add90 0x7f85740b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:05.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.276+0000 7f858a7fc700 1 --2- 172.21.15.107:0/311606036 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f85740a2cb0 0x7f85740a5170 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8580009fd0 tx=0x7f858000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:05.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.276+0000 7f858b7fe700 1 --2- 172.21.15.107:0/311606036 >> [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444] conn(0x7f85740add90 0x7f85740b0230 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:05.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.278+0000 7f859081b700 1 -- 172.21.15.107:0/311606036 <== osd.3 v2:172.21.15.107:6810/2902386444 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f857c01de70 con 0x7f85740add90 2024-04-25T00:15:05.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.294+0000 7f859181d700 1 -- 172.21.15.107:0/311606036 --> [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f858c101200 con 0x7f85740add90 2024-04-25T00:15:05.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.294+0000 7f859081b700 1 -- 172.21.15.107:0/311606036 <== osd.3 v2:172.21.15.107:6810/2902386444 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f858c101200 con 0x7f85740add90 2024-04-25T00:15:05.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.294+0000 7f859181d700 1 -- 172.21.15.107:0/311606036 >> [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444] conn(0x7f85740add90 msgr2=0x7f85740b0230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:05.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.294+0000 7f859181d700 1 --2- 172.21.15.107:0/311606036 >> [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444] conn(0x7f85740add90 0x7f85740b0230 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:05.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.294+0000 7f859181d700 1 -- 172.21.15.107:0/311606036 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f85740a2cb0 msgr2=0x7f85740a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:05.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.295+0000 7f859181d700 1 --2- 172.21.15.107:0/311606036 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f85740a2cb0 0x7f85740a5170 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8580009fd0 tx=0x7f858000b040 comp rx=0 tx=0).stop 2024-04-25T00:15:05.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.295+0000 7f859181d700 1 -- 172.21.15.107:0/311606036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f858c101b90 msgr2=0x7f858c19d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:05.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.295+0000 7f859181d700 1 --2- 172.21.15.107:0/311606036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f858c101b90 0x7f858c19d630 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f857c00ea40 tx=0x7f857c00ed50 comp rx=0 tx=0).stop 2024-04-25T00:15:05.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.295+0000 7f858b7fe700 1 -- 172.21.15.107:0/311606036 reap_dead start 2024-04-25T00:15:05.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.295+0000 7f859181d700 1 -- 172.21.15.107:0/311606036 shutdown_connections 2024-04-25T00:15:05.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.295+0000 7f859181d700 1 -- 172.21.15.107:0/311606036 >> 172.21.15.107:0/311606036 conn(0x7f858c0777a0 msgr2=0x7f858c104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:05.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.295+0000 7f859181d700 1 -- 172.21.15.107:0/311606036 shutdown_connections 2024-04-25T00:15:05.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:05.295+0000 7f859181d700 1 -- 172.21.15.107:0/311606036 wait complete. 2024-04-25T00:15:05.431 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:05 smithi107 ceph-mon[29153]: pgmap v326: 129 pgs: 5 peering, 124 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 383 B/s wr, 5 op/s 2024-04-25T00:15:05.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:05 smithi170 ceph-mon[32807]: pgmap v326: 129 pgs: 5 peering, 124 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 383 B/s wr, 5 op/s 2024-04-25T00:15:05.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:05 smithi195 ceph-mon[32833]: pgmap v326: 129 pgs: 5 peering, 124 active+clean; 643 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 383 B/s wr, 5 op/s 2024-04-25T00:15:06.118 INFO:teuthology.orchestra.run.smithi107.stdout:55834574922 2024-04-25T00:15:06.119 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T00:15:06.549 INFO:teuthology.orchestra.run.smithi107.stdout:146028888125 2024-04-25T00:15:06.549 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T00:15:07.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:07 smithi107 ceph-mon[29153]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-04-25T00:15:07.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:07 smithi107 ceph-mon[29153]: Cluster is now healthy 2024-04-25T00:15:07.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:07 smithi195 ceph-mon[32833]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-04-25T00:15:07.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:07 smithi195 ceph-mon[32833]: Cluster is now healthy 2024-04-25T00:15:07.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:07 smithi170 ceph-mon[32807]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-04-25T00:15:07.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:07 smithi170 ceph-mon[32807]: Cluster is now healthy 2024-04-25T00:15:08.160 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:08 smithi107 ceph-mon[29153]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:15:08.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:08 smithi195 ceph-mon[32833]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:15:08.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:08 smithi170 ceph-mon[32807]: pgmap v327: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:15:08.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.519+0000 7ff286bef700 1 -- 172.21.15.107:0/617052286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff280101b10 msgr2=0x7ff28010dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:08.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.519+0000 7ff286bef700 1 --2- 172.21.15.107:0/617052286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff280101b10 0x7ff28010dff0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff27800b0a0 tx=0x7ff27800b3b0 comp rx=0 tx=0).stop 2024-04-25T00:15:08.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.519+0000 7ff286bef700 1 -- 172.21.15.107:0/617052286 shutdown_connections 2024-04-25T00:15:08.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.519+0000 7ff286bef700 1 --2- 172.21.15.107:0/617052286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff280101b10 0x7ff28010dff0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:08.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.519+0000 7ff286bef700 1 --2- 172.21.15.107:0/617052286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2801071b0 0x7ff280107590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:08.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.519+0000 7ff286bef700 1 --2- 172.21.15.107:0/617052286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff280101170 0x7ff2801015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:08.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.519+0000 7ff286bef700 1 -- 172.21.15.107:0/617052286 >> 172.21.15.107:0/617052286 conn(0x7ff280078060 msgr2=0x7ff280078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:08.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.520+0000 7ff286bef700 1 -- 172.21.15.107:0/617052286 shutdown_connections 2024-04-25T00:15:08.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.520+0000 7ff286bef700 1 -- 172.21.15.107:0/617052286 wait complete. 2024-04-25T00:15:08.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.520+0000 7ff286bef700 1 Processor -- start 2024-04-25T00:15:08.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.520+0000 7ff286bef700 1 -- start start 2024-04-25T00:15:08.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.520+0000 7ff286bef700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff280101170 0x7ff280199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:08.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.520+0000 7ff286bef700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff280101b10 0x7ff280199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:08.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.520+0000 7ff286bef700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2801071b0 0x7ff2801a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:08.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.520+0000 7ff286bef700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff280111470 con 0x7ff280101b10 2024-04-25T00:15:08.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.520+0000 7ff286bef700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801112f0 con 0x7ff280101170 2024-04-25T00:15:08.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.520+0000 7ff286bef700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801115f0 con 0x7ff2801071b0 2024-04-25T00:15:08.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.520+0000 7ff27ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff280101b10 0x7ff280199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:08.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.521+0000 7ff27ffff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff280101b10 0x7ff280199cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60974/0 (socket says 172.21.15.107:60974) 2024-04-25T00:15:08.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.521+0000 7ff27ffff700 1 -- 172.21.15.107:0/4149146106 learned_addr learned my addr 172.21.15.107:0/4149146106 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:08.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.521+0000 7ff28498b700 1 --2- 172.21.15.107:0/4149146106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff280101170 0x7ff280199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:08.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.521+0000 7ff28518c700 1 --2- 172.21.15.107:0/4149146106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2801071b0 0x7ff2801a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:08.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.521+0000 7ff27ffff700 1 -- 172.21.15.107:0/4149146106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2801071b0 msgr2=0x7ff2801a4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:08.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.521+0000 7ff27ffff700 1 --2- 172.21.15.107:0/4149146106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff2801071b0 0x7ff2801a4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:08.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.521+0000 7ff27ffff700 1 -- 172.21.15.107:0/4149146106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff280101170 msgr2=0x7ff280199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:08.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.521+0000 7ff27ffff700 1 --2- 172.21.15.107:0/4149146106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff280101170 0x7ff280199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:08.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.521+0000 7ff27ffff700 1 -- 172.21.15.107:0/4149146106 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2801a4d50 con 0x7ff280101b10 2024-04-25T00:15:08.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.521+0000 7ff28498b700 1 --2- 172.21.15.107:0/4149146106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff280101170 0x7ff280199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:08.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.521+0000 7ff27ffff700 1 --2- 172.21.15.107:0/4149146106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff280101b10 0x7ff280199cd0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00c9b0 tx=0x7ff26c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:08.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.521+0000 7ff27dffb700 1 -- 172.21.15.107:0/4149146106 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c007b00 con 0x7ff280101b10 2024-04-25T00:15:08.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.521+0000 7ff27dffb700 1 -- 172.21.15.107:0/4149146106 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff26c007ca0 con 0x7ff280101b10 2024-04-25T00:15:08.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.521+0000 7ff286bef700 1 -- 172.21.15.107:0/4149146106 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2801a4f80 con 0x7ff280101b10 2024-04-25T00:15:08.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.521+0000 7ff27dffb700 1 -- 172.21.15.107:0/4149146106 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c018780 con 0x7ff280101b10 2024-04-25T00:15:08.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.521+0000 7ff286bef700 1 -- 172.21.15.107:0/4149146106 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2801a53e0 con 0x7ff280101b10 2024-04-25T00:15:08.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.522+0000 7ff286bef700 1 -- 172.21.15.107:0/4149146106 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff280062940 con 0x7ff280101b10 2024-04-25T00:15:08.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.523+0000 7ff27dffb700 1 -- 172.21.15.107:0/4149146106 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7ff26c00f450 con 0x7ff280101b10 2024-04-25T00:15:08.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.524+0000 7ff27dffb700 1 --2- 172.21.15.107:0/4149146106 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff2700a2cb0 0x7ff2700a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:08.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.524+0000 7ff28498b700 1 --2- 172.21.15.107:0/4149146106 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff2700a2cb0 0x7ff2700a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:08.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.524+0000 7ff27dffb700 1 -- 172.21.15.107:0/4149146106 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7ff26c0cf960 con 0x7ff280101b10 2024-04-25T00:15:08.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.524+0000 7ff27dffb700 1 --2- 172.21.15.107:0/4149146106 >> [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636] conn(0x7ff2700add90 0x7ff2700b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:08.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.524+0000 7ff27dffb700 1 -- 172.21.15.107:0/4149146106 --> [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff26c015e70 con 0x7ff2700add90 2024-04-25T00:15:08.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.524+0000 7ff27dffb700 1 -- 172.21.15.107:0/4149146106 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7ff26c0cfdb0 con 0x7ff280101b10 2024-04-25T00:15:08.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.524+0000 7ff28518c700 1 --2- 172.21.15.107:0/4149146106 >> [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636] conn(0x7ff2700add90 0x7ff2700b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:08.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.524+0000 7ff28498b700 1 --2- 172.21.15.107:0/4149146106 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff2700a2cb0 0x7ff2700a5170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff274007900 tx=0x7ff274008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:08.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.525+0000 7ff28518c700 1 --2- 172.21.15.107:0/4149146106 >> [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636] conn(0x7ff2700add90 0x7ff2700b0230 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:08.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.525+0000 7ff27dffb700 1 -- 172.21.15.107:0/4149146106 <== osd.0 v2:172.21.15.107:6802/331467636 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff26c015e70 con 0x7ff2700add90 2024-04-25T00:15:08.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.542+0000 7ff286bef700 1 -- 172.21.15.107:0/4149146106 --> [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff280062dc0 con 0x7ff2700add90 2024-04-25T00:15:08.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.542+0000 7ff27dffb700 1 -- 172.21.15.107:0/4149146106 <== osd.0 v2:172.21.15.107:6802/331467636 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff280062dc0 con 0x7ff2700add90 2024-04-25T00:15:08.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.542+0000 7ff286bef700 1 -- 172.21.15.107:0/4149146106 >> [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636] conn(0x7ff2700add90 msgr2=0x7ff2700b0230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:08.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.542+0000 7ff286bef700 1 --2- 172.21.15.107:0/4149146106 >> [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636] conn(0x7ff2700add90 0x7ff2700b0230 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:08.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.542+0000 7ff286bef700 1 -- 172.21.15.107:0/4149146106 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff2700a2cb0 msgr2=0x7ff2700a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:08.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.542+0000 7ff286bef700 1 --2- 172.21.15.107:0/4149146106 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7ff2700a2cb0 0x7ff2700a5170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff274007900 tx=0x7ff274008040 comp rx=0 tx=0).stop 2024-04-25T00:15:08.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.542+0000 7ff286bef700 1 -- 172.21.15.107:0/4149146106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff280101b10 msgr2=0x7ff280199cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:08.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.542+0000 7ff286bef700 1 --2- 172.21.15.107:0/4149146106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff280101b10 0x7ff280199cd0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00c9b0 tx=0x7ff26c00cd70 comp rx=0 tx=0).stop 2024-04-25T00:15:08.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.542+0000 7ff28518c700 1 -- 172.21.15.107:0/4149146106 reap_dead start 2024-04-25T00:15:08.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.543+0000 7ff286bef700 1 -- 172.21.15.107:0/4149146106 shutdown_connections 2024-04-25T00:15:08.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.543+0000 7ff286bef700 1 -- 172.21.15.107:0/4149146106 >> 172.21.15.107:0/4149146106 conn(0x7ff280078060 msgr2=0x7ff2800ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:08.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.543+0000 7ff286bef700 1 -- 172.21.15.107:0/4149146106 shutdown_connections 2024-04-25T00:15:08.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.543+0000 7ff286bef700 1 -- 172.21.15.107:0/4149146106 wait complete. 2024-04-25T00:15:08.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.749+0000 7f959e59f700 1 -- 172.21.15.107:0/65138202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9598101c70 msgr2=0x7f9598105cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:08.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.749+0000 7f959e59f700 1 --2- 172.21.15.107:0/65138202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9598101c70 0x7f9598105cc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f958c0099c0 tx=0x7f958c009cd0 comp rx=0 tx=0).stop 2024-04-25T00:15:08.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.749+0000 7f959e59f700 1 -- 172.21.15.107:0/65138202 shutdown_connections 2024-04-25T00:15:08.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.749+0000 7f959e59f700 1 --2- 172.21.15.107:0/65138202 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95981063a0 0x7f959810df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:08.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.749+0000 7f959e59f700 1 --2- 172.21.15.107:0/65138202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95981012c0 0x7f95981016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:08.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.749+0000 7f959e59f700 1 --2- 172.21.15.107:0/65138202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9598101c70 0x7f9598105cc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:08.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.749+0000 7f959e59f700 1 -- 172.21.15.107:0/65138202 >> 172.21.15.107:0/65138202 conn(0x7f95980fcb50 msgr2=0x7f95980fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:08.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.749+0000 7f959e59f700 1 -- 172.21.15.107:0/65138202 shutdown_connections 2024-04-25T00:15:08.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.749+0000 7f959e59f700 1 -- 172.21.15.107:0/65138202 wait complete. 2024-04-25T00:15:08.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.750+0000 7f959e59f700 1 Processor -- start 2024-04-25T00:15:08.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.750+0000 7f959e59f700 1 -- start start 2024-04-25T00:15:08.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.750+0000 7f959e59f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95981012c0 0x7f959819e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:08.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.750+0000 7f959e59f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9598101c70 0x7f959819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:08.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.750+0000 7f959e59f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95981063a0 0x7f9598198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:08.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.750+0000 7f959e59f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f95981113e0 con 0x7f9598101c70 2024-04-25T00:15:08.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.750+0000 7f959e59f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9598111260 con 0x7f95981063a0 2024-04-25T00:15:08.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.750+0000 7f959e59f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9598111560 con 0x7f95981012c0 2024-04-25T00:15:08.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.750+0000 7f959dd9e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95981063a0 0x7f9598198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:08.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.750+0000 7f959dd9e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95981063a0 0x7f9598198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.107:46724/0 (socket says 172.21.15.107:46724) 2024-04-25T00:15:08.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.750+0000 7f959d59d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95981012c0 0x7f959819e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:08.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.750+0000 7f959dd9e700 1 -- 172.21.15.107:0/2810985795 learned_addr learned my addr 172.21.15.107:0/2810985795 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:08.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.750+0000 7f959dd9e700 1 -- 172.21.15.107:0/2810985795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95981012c0 msgr2=0x7f959819e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:08.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.750+0000 7f959dd9e700 1 --2- 172.21.15.107:0/2810985795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95981012c0 0x7f959819e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:08.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.750+0000 7f959dd9e700 1 -- 172.21.15.107:0/2810985795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9598101c70 msgr2=0x7f959819eb90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:15:08.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.751+0000 7f959dd9e700 1 --2- 172.21.15.107:0/2810985795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9598101c70 0x7f959819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:08.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.751+0000 7f959dd9e700 1 -- 172.21.15.107:0/2810985795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9598199000 con 0x7f95981063a0 2024-04-25T00:15:08.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.751+0000 7f959dd9e700 1 --2- 172.21.15.107:0/2810985795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95981063a0 0x7f9598198720 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f959000b970 tx=0x7f959000bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:08.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.751+0000 7f958a7fc700 1 -- 172.21.15.107:0/2810985795 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590019070 con 0x7f95981063a0 2024-04-25T00:15:08.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.751+0000 7f958a7fc700 1 -- 172.21.15.107:0/2810985795 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f959000ce10 con 0x7f95981063a0 2024-04-25T00:15:08.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.751+0000 7f959e59f700 1 -- 172.21.15.107:0/2810985795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95981992f0 con 0x7f95981063a0 2024-04-25T00:15:08.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.751+0000 7f958a7fc700 1 -- 172.21.15.107:0/2810985795 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590007370 con 0x7f95981063a0 2024-04-25T00:15:08.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.751+0000 7f959e59f700 1 -- 172.21.15.107:0/2810985795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95981a55d0 con 0x7f95981063a0 2024-04-25T00:15:08.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.752+0000 7f959e59f700 1 -- 172.21.15.107:0/2810985795 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f957c001030 con 0x7f95981063a0 2024-04-25T00:15:08.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.754+0000 7f958a7fc700 1 -- 172.21.15.107:0/2810985795 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f959001c070 con 0x7f95981063a0 2024-04-25T00:15:08.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.755+0000 7f958a7fc700 1 --2- 172.21.15.107:0/2810985795 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f95840a2cb0 0x7f95840a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:08.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.755+0000 7f958a7fc700 1 -- 172.21.15.107:0/2810985795 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f95900cf1c0 con 0x7f95981063a0 2024-04-25T00:15:08.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.755+0000 7f959d59d700 1 --2- 172.21.15.107:0/2810985795 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f95840a2cb0 0x7f95840a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:08.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.755+0000 7f958a7fc700 1 --2- 172.21.15.107:0/2810985795 >> [v2:172.21.15.195:6808/2322632437,v1:172.21.15.195:6809/2322632437] conn(0x7f95840add90 0x7f95840b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:08.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.755+0000 7f958a7fc700 1 -- 172.21.15.107:0/2810985795 --> [v2:172.21.15.195:6808/2322632437,v1:172.21.15.195:6809/2322632437] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9590020e50 con 0x7f95840add90 2024-04-25T00:15:08.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.755+0000 7f958a7fc700 1 -- 172.21.15.107:0/2810985795 <== mon.2 v2:172.21.15.170:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f95900cf610 con 0x7f95981063a0 2024-04-25T00:15:08.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.756+0000 7f959cd9c700 1 --2- 172.21.15.107:0/2810985795 >> [v2:172.21.15.195:6808/2322632437,v1:172.21.15.195:6809/2322632437] conn(0x7f95840add90 0x7f95840b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:08.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.756+0000 7f959d59d700 1 --2- 172.21.15.107:0/2810985795 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f95840a2cb0 0x7f95840a5170 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9594005950 tx=0x7f959400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:08.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.756+0000 7f959cd9c700 1 --2- 172.21.15.107:0/2810985795 >> [v2:172.21.15.195:6808/2322632437,v1:172.21.15.195:6809/2322632437] conn(0x7f95840add90 0x7f95840b0230 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:08.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.757+0000 7f958a7fc700 1 -- 172.21.15.107:0/2810985795 <== osd.5 v2:172.21.15.195:6808/2322632437 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9590020e50 con 0x7f95840add90 2024-04-25T00:15:08.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.775+0000 7f959e59f700 1 -- 172.21.15.107:0/2810985795 --> [v2:172.21.15.195:6808/2322632437,v1:172.21.15.195:6809/2322632437] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f957c002e60 con 0x7f95840add90 2024-04-25T00:15:08.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.775+0000 7f958a7fc700 1 -- 172.21.15.107:0/2810985795 <== osd.5 v2:172.21.15.195:6808/2322632437 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f957c002e60 con 0x7f95840add90 2024-04-25T00:15:08.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.776+0000 7f959e59f700 1 -- 172.21.15.107:0/2810985795 >> [v2:172.21.15.195:6808/2322632437,v1:172.21.15.195:6809/2322632437] conn(0x7f95840add90 msgr2=0x7f95840b0230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:08.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.776+0000 7f959e59f700 1 --2- 172.21.15.107:0/2810985795 >> [v2:172.21.15.195:6808/2322632437,v1:172.21.15.195:6809/2322632437] conn(0x7f95840add90 0x7f95840b0230 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:08.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.776+0000 7f959e59f700 1 -- 172.21.15.107:0/2810985795 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f95840a2cb0 msgr2=0x7f95840a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:08.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.776+0000 7f959e59f700 1 --2- 172.21.15.107:0/2810985795 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f95840a2cb0 0x7f95840a5170 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9594005950 tx=0x7f959400b410 comp rx=0 tx=0).stop 2024-04-25T00:15:08.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.776+0000 7f959e59f700 1 -- 172.21.15.107:0/2810985795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95981063a0 msgr2=0x7f9598198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:08.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.776+0000 7f959e59f700 1 --2- 172.21.15.107:0/2810985795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95981063a0 0x7f9598198720 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f959000b970 tx=0x7f959000bc80 comp rx=0 tx=0).stop 2024-04-25T00:15:08.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.776+0000 7f959dd9e700 1 -- 172.21.15.107:0/2810985795 reap_dead start 2024-04-25T00:15:08.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.776+0000 7f959e59f700 1 -- 172.21.15.107:0/2810985795 shutdown_connections 2024-04-25T00:15:08.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.776+0000 7f959e59f700 1 -- 172.21.15.107:0/2810985795 >> 172.21.15.107:0/2810985795 conn(0x7f95980fcb50 msgr2=0x7f959806a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:08.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.776+0000 7f959e59f700 1 -- 172.21.15.107:0/2810985795 shutdown_connections 2024-04-25T00:15:08.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:08.776+0000 7f959e59f700 1 -- 172.21.15.107:0/2810985795 wait complete. 2024-04-25T00:15:09.659 INFO:teuthology.orchestra.run.smithi107.stdout:34359738450 2024-04-25T00:15:09.659 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T00:15:09.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.839+0000 7fb9ef6d3700 1 -- 172.21.15.107:0/4163654798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e81071a0 msgr2=0x7fb9e8107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:09.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.839+0000 7fb9ef6d3700 1 --2- 172.21.15.107:0/4163654798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e81071a0 0x7fb9e8107580 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009990 tx=0x7fb9e4009ca0 comp rx=0 tx=0).stop 2024-04-25T00:15:09.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.839+0000 7fb9ef6d3700 1 -- 172.21.15.107:0/4163654798 shutdown_connections 2024-04-25T00:15:09.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.839+0000 7fb9ef6d3700 1 --2- 172.21.15.107:0/4163654798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9e8101af0 0x7fb9e810dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:09.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.839+0000 7fb9ef6d3700 1 --2- 172.21.15.107:0/4163654798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e81071a0 0x7fb9e8107580 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:09.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.839+0000 7fb9ef6d3700 1 --2- 172.21.15.107:0/4163654798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9e8101130 0x7fb9e81015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:09.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.839+0000 7fb9ef6d3700 1 -- 172.21.15.107:0/4163654798 >> 172.21.15.107:0/4163654798 conn(0x7fb9e80780e0 msgr2=0x7fb9e80784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:09.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.839+0000 7fb9ef6d3700 1 -- 172.21.15.107:0/4163654798 shutdown_connections 2024-04-25T00:15:09.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.839+0000 7fb9ef6d3700 1 -- 172.21.15.107:0/4163654798 wait complete. 2024-04-25T00:15:09.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.840+0000 7fb9ef6d3700 1 Processor -- start 2024-04-25T00:15:09.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.840+0000 7fb9ef6d3700 1 -- start start 2024-04-25T00:15:09.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.840+0000 7fb9ef6d3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e8101130 0x7fb9e819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:09.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.840+0000 7fb9ef6d3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9e8101af0 0x7fb9e819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:09.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.840+0000 7fb9ef6d3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9e81071a0 0x7fb9e8198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:09.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.840+0000 7fb9ef6d3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e8111490 con 0x7fb9e8101130 2024-04-25T00:15:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.840+0000 7fb9ef6d3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e8111310 con 0x7fb9e8101af0 2024-04-25T00:15:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.840+0000 7fb9ef6d3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb9e8111610 con 0x7fb9e81071a0 2024-04-25T00:15:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.840+0000 7fb9ed46f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e8101130 0x7fb9e819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.840+0000 7fb9ed46f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e8101130 0x7fb9e819e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47900/0 (socket says 172.21.15.107:47900) 2024-04-25T00:15:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.840+0000 7fb9ed46f700 1 -- 172.21.15.107:0/3999578201 learned_addr learned my addr 172.21.15.107:0/3999578201 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.840+0000 7fb9ecc6e700 1 --2- 172.21.15.107:0/3999578201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9e8101af0 0x7fb9e819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.840+0000 7fb9edc70700 1 --2- 172.21.15.107:0/3999578201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9e81071a0 0x7fb9e8198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.840+0000 7fb9ed46f700 1 -- 172.21.15.107:0/3999578201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9e81071a0 msgr2=0x7fb9e8198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.840+0000 7fb9ed46f700 1 --2- 172.21.15.107:0/3999578201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9e81071a0 0x7fb9e8198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.840+0000 7fb9ed46f700 1 -- 172.21.15.107:0/3999578201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9e8101af0 msgr2=0x7fb9e819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.841+0000 7fb9ed46f700 1 --2- 172.21.15.107:0/3999578201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9e8101af0 0x7fb9e819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.841+0000 7fb9ed46f700 1 -- 172.21.15.107:0/3999578201 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9e8198fd0 con 0x7fb9e8101130 2024-04-25T00:15:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.841+0000 7fb9ed46f700 1 --2- 172.21.15.107:0/3999578201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e8101130 0x7fb9e819e6b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4005910 tx=0x7fb9e400bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.841+0000 7fb9de7fc700 1 -- 172.21.15.107:0/3999578201 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4027070 con 0x7fb9e8101130 2024-04-25T00:15:09.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.841+0000 7fb9ef6d3700 1 -- 172.21.15.107:0/3999578201 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9e8199260 con 0x7fb9e8101130 2024-04-25T00:15:09.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.841+0000 7fb9de7fc700 1 -- 172.21.15.107:0/3999578201 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9e4003d70 con 0x7fb9e8101130 2024-04-25T00:15:09.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.841+0000 7fb9de7fc700 1 -- 172.21.15.107:0/3999578201 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e400f460 con 0x7fb9e8101130 2024-04-25T00:15:09.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.841+0000 7fb9ef6d3700 1 -- 172.21.15.107:0/3999578201 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9e81a5580 con 0x7fb9e8101130 2024-04-25T00:15:09.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.842+0000 7fb9ef6d3700 1 -- 172.21.15.107:0/3999578201 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb9e8105260 con 0x7fb9e8101130 2024-04-25T00:15:09.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.843+0000 7fb9de7fc700 1 -- 172.21.15.107:0/3999578201 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fb9e402a070 con 0x7fb9e8101130 2024-04-25T00:15:09.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.844+0000 7fb9de7fc700 1 --2- 172.21.15.107:0/3999578201 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb9d40a2cb0 0x7fb9d40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:09.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.844+0000 7fb9ecc6e700 1 --2- 172.21.15.107:0/3999578201 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb9d40a2cb0 0x7fb9d40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:09.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.844+0000 7fb9de7fc700 1 -- 172.21.15.107:0/3999578201 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fb9e40d0b00 con 0x7fb9e8101130 2024-04-25T00:15:09.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.844+0000 7fb9de7fc700 1 --2- 172.21.15.107:0/3999578201 >> [v2:172.21.15.170:6824/1847892477,v1:172.21.15.170:6825/1847892477] conn(0x7fb9d40add90 0x7fb9d40b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:09.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.844+0000 7fb9de7fc700 1 -- 172.21.15.107:0/3999578201 --> [v2:172.21.15.170:6824/1847892477,v1:172.21.15.170:6825/1847892477] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb9e400eea0 con 0x7fb9d40add90 2024-04-25T00:15:09.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.844+0000 7fb9de7fc700 1 -- 172.21.15.107:0/3999578201 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fb9e40d0f50 con 0x7fb9e8101130 2024-04-25T00:15:09.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.844+0000 7fb9ecc6e700 1 --2- 172.21.15.107:0/3999578201 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb9d40a2cb0 0x7fb9d40a5170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8078670 tx=0x7fb9d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:09.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.844+0000 7fb9edc70700 1 --2- 172.21.15.107:0/3999578201 >> [v2:172.21.15.170:6824/1847892477,v1:172.21.15.170:6825/1847892477] conn(0x7fb9d40add90 0x7fb9d40b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:09.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.845+0000 7fb9edc70700 1 --2- 172.21.15.107:0/3999578201 >> [v2:172.21.15.170:6824/1847892477,v1:172.21.15.170:6825/1847892477] conn(0x7fb9d40add90 0x7fb9d40b0230 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:09.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.846+0000 7fb9de7fc700 1 -- 172.21.15.107:0/3999578201 <== osd.10 v2:172.21.15.170:6824/1847892477 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb9e400eea0 con 0x7fb9d40add90 2024-04-25T00:15:09.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.861+0000 7fb9ef6d3700 1 -- 172.21.15.107:0/3999578201 --> [v2:172.21.15.170:6824/1847892477,v1:172.21.15.170:6825/1847892477] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb9e8102850 con 0x7fb9d40add90 2024-04-25T00:15:09.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.862+0000 7fb9de7fc700 1 -- 172.21.15.107:0/3999578201 <== osd.10 v2:172.21.15.170:6824/1847892477 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb9e8102850 con 0x7fb9d40add90 2024-04-25T00:15:09.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.862+0000 7fb9ef6d3700 1 -- 172.21.15.107:0/3999578201 >> [v2:172.21.15.170:6824/1847892477,v1:172.21.15.170:6825/1847892477] conn(0x7fb9d40add90 msgr2=0x7fb9d40b0230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:09.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.862+0000 7fb9ef6d3700 1 --2- 172.21.15.107:0/3999578201 >> [v2:172.21.15.170:6824/1847892477,v1:172.21.15.170:6825/1847892477] conn(0x7fb9d40add90 0x7fb9d40b0230 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:09.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.862+0000 7fb9ef6d3700 1 -- 172.21.15.107:0/3999578201 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb9d40a2cb0 msgr2=0x7fb9d40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:09.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.862+0000 7fb9ef6d3700 1 --2- 172.21.15.107:0/3999578201 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb9d40a2cb0 0x7fb9d40a5170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8078670 tx=0x7fb9d8009260 comp rx=0 tx=0).stop 2024-04-25T00:15:09.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.862+0000 7fb9ef6d3700 1 -- 172.21.15.107:0/3999578201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e8101130 msgr2=0x7fb9e819e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:09.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.862+0000 7fb9ef6d3700 1 --2- 172.21.15.107:0/3999578201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9e8101130 0x7fb9e819e6b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4005910 tx=0x7fb9e400bab0 comp rx=0 tx=0).stop 2024-04-25T00:15:09.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.863+0000 7fb9edc70700 1 -- 172.21.15.107:0/3999578201 reap_dead start 2024-04-25T00:15:09.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.863+0000 7fb9ef6d3700 1 -- 172.21.15.107:0/3999578201 shutdown_connections 2024-04-25T00:15:09.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.863+0000 7fb9ef6d3700 1 -- 172.21.15.107:0/3999578201 >> 172.21.15.107:0/3999578201 conn(0x7fb9e80780e0 msgr2=0x7fb9e8100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:09.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.863+0000 7fb9ef6d3700 1 -- 172.21.15.107:0/3999578201 shutdown_connections 2024-04-25T00:15:09.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:09.863+0000 7fb9ef6d3700 1 -- 172.21.15.107:0/3999578201 wait complete. 2024-04-25T00:15:10.317 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:10 smithi107 ceph-mon[29153]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:15:10.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:10 smithi195 ceph-mon[32833]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:15:10.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:10 smithi170 ceph-mon[32807]: pgmap v328: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:15:11.029 INFO:teuthology.orchestra.run.smithi107.stdout:111669149762 2024-04-25T00:15:11.030 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T00:15:11.431 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:15:11.437 INFO:teuthology.orchestra.run.smithi107.stdout:167503724601 2024-04-25T00:15:11.437 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T00:15:11.438 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:15:11.472 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:15:12.236 INFO:teuthology.orchestra.run.smithi107.stdout:313532612639 2024-04-25T00:15:12.237 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T00:15:12.298 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:12 smithi107 ceph-mon[29153]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:15:12.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:12 smithi195 ceph-mon[32833]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:15:12.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:12 smithi170 ceph-mon[32807]: pgmap v329: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:15:13.705 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:15:13.712 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:15:13.712 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:15:14.212 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:14 smithi107 ceph-mon[29153]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:15:14.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:14 smithi195 ceph-mon[32833]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:15:14.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:14 smithi170 ceph-mon[32807]: pgmap v330: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:15:14.965 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:15:16.086 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:15:16.087 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:15:16.300 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:16 smithi107 ceph-mon[29153]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:15:16.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:16 smithi195 ceph-mon[32833]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:15:16.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:16 smithi170 ceph-mon[32807]: pgmap v331: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:15:17.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.446+0000 7f5accf8a700 1 -- 172.21.15.107:0/2499031154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ac81071b0 msgr2=0x7f5ac8107590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:17.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.446+0000 7f5accf8a700 1 --2- 172.21.15.107:0/2499031154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ac81071b0 0x7f5ac8107590 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8009990 tx=0x7f5ab8009ca0 comp rx=0 tx=0).stop 2024-04-25T00:15:17.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.447+0000 7f5accf8a700 1 -- 172.21.15.107:0/2499031154 shutdown_connections 2024-04-25T00:15:17.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.447+0000 7f5accf8a700 1 --2- 172.21.15.107:0/2499031154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ac8101b10 0x7f5ac810dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:17.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.447+0000 7f5accf8a700 1 --2- 172.21.15.107:0/2499031154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ac81071b0 0x7f5ac8107590 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:17.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.447+0000 7f5accf8a700 1 --2- 172.21.15.107:0/2499031154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ac8101170 0x7f5ac81015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:17.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.447+0000 7f5accf8a700 1 -- 172.21.15.107:0/2499031154 >> 172.21.15.107:0/2499031154 conn(0x7f5ac8078060 msgr2=0x7f5ac8078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:17.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.447+0000 7f5accf8a700 1 -- 172.21.15.107:0/2499031154 shutdown_connections 2024-04-25T00:15:17.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.447+0000 7f5accf8a700 1 -- 172.21.15.107:0/2499031154 wait complete. 2024-04-25T00:15:17.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.447+0000 7f5accf8a700 1 Processor -- start 2024-04-25T00:15:17.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.447+0000 7f5accf8a700 1 -- start start 2024-04-25T00:15:17.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.448+0000 7f5accf8a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ac8101170 0x7f5ac8102e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:17.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.448+0000 7f5accf8a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ac8101b10 0x7f5ac8103360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:17.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.448+0000 7f5accf8a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ac81071b0 0x7f5ac81038a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:17.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.448+0000 7f5accf8a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8111470 con 0x7f5ac81071b0 2024-04-25T00:15:17.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.448+0000 7f5accf8a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81112f0 con 0x7f5ac8101b10 2024-04-25T00:15:17.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.448+0000 7f5accf8a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81115f0 con 0x7f5ac8101170 2024-04-25T00:15:17.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.448+0000 7f5ac6d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ac81071b0 0x7f5ac81038a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:17.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.448+0000 7f5ac6d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ac81071b0 0x7f5ac81038a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47920/0 (socket says 172.21.15.107:47920) 2024-04-25T00:15:17.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.448+0000 7f5ac6d9d700 1 -- 172.21.15.107:0/3827642765 learned_addr learned my addr 172.21.15.107:0/3827642765 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:17.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.448+0000 7f5ac5d9b700 1 --2- 172.21.15.107:0/3827642765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ac8101b10 0x7f5ac8103360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:17.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.448+0000 7f5ac659c700 1 --2- 172.21.15.107:0/3827642765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ac8101170 0x7f5ac8102e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:17.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.448+0000 7f5ac6d9d700 1 -- 172.21.15.107:0/3827642765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ac8101170 msgr2=0x7f5ac8102e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:17.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.448+0000 7f5ac6d9d700 1 --2- 172.21.15.107:0/3827642765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ac8101170 0x7f5ac8102e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:17.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.448+0000 7f5ac6d9d700 1 -- 172.21.15.107:0/3827642765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ac8101b10 msgr2=0x7f5ac8103360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:17.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.448+0000 7f5ac6d9d700 1 --2- 172.21.15.107:0/3827642765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ac8101b10 0x7f5ac8103360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:17.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.448+0000 7f5ac6d9d700 1 -- 172.21.15.107:0/3827642765 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac81a6f10 con 0x7f5ac81071b0 2024-04-25T00:15:17.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.448+0000 7f5ac6d9d700 1 --2- 172.21.15.107:0/3827642765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ac81071b0 0x7f5ac81038a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000b970 tx=0x7f5ac000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:17.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.448+0000 7f5ab77fe700 1 -- 172.21.15.107:0/3827642765 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0019070 con 0x7f5ac81071b0 2024-04-25T00:15:17.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.448+0000 7f5ab77fe700 1 -- 172.21.15.107:0/3827642765 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ac000ce10 con 0x7f5ac81071b0 2024-04-25T00:15:17.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.448+0000 7f5accf8a700 1 -- 172.21.15.107:0/3827642765 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac81a7200 con 0x7f5ac81071b0 2024-04-25T00:15:17.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.448+0000 7f5ab77fe700 1 -- 172.21.15.107:0/3827642765 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0007370 con 0x7f5ac81071b0 2024-04-25T00:15:17.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.449+0000 7f5accf8a700 1 -- 172.21.15.107:0/3827642765 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac81a7740 con 0x7f5ac81071b0 2024-04-25T00:15:17.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.449+0000 7f5accf8a700 1 -- 172.21.15.107:0/3827642765 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac8102860 con 0x7f5ac81071b0 2024-04-25T00:15:17.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.451+0000 7f5ab77fe700 1 -- 172.21.15.107:0/3827642765 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f5ac001c070 con 0x7f5ac81071b0 2024-04-25T00:15:17.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.451+0000 7f5ab77fe700 1 --2- 172.21.15.107:0/3827642765 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5ab00a2cb0 0x7f5ab00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:17.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.451+0000 7f5ac659c700 1 --2- 172.21.15.107:0/3827642765 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5ab00a2cb0 0x7f5ab00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:17.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.451+0000 7f5ab77fe700 1 -- 172.21.15.107:0/3827642765 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f5ac00cec10 con 0x7f5ac81071b0 2024-04-25T00:15:17.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.452+0000 7f5ac659c700 1 --2- 172.21.15.107:0/3827642765 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5ab00a2cb0 0x7f5ab00a5170 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8009750 tx=0x7f5ab800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:17.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.455+0000 7f5ab77fe700 1 -- 172.21.15.107:0/3827642765 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5ac009b7e0 con 0x7f5ac81071b0 2024-04-25T00:15:17.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.663+0000 7f5accf8a700 1 -- 172.21.15.107:0/3827642765 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f5ac8068f20 con 0x7f5ac81071b0 2024-04-25T00:15:17.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.663+0000 7f5ab77fe700 1 -- 172.21.15.107:0/3827642765 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5ac8068f20 con 0x7f5ac81071b0 2024-04-25T00:15:17.664 INFO:teuthology.orchestra.run.smithi107.stdout:201863462966 2024-04-25T00:15:17.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.666+0000 7f5accf8a700 1 -- 172.21.15.107:0/3827642765 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5ab00a2cb0 msgr2=0x7f5ab00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:17.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.666+0000 7f5accf8a700 1 --2- 172.21.15.107:0/3827642765 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5ab00a2cb0 0x7f5ab00a5170 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8009750 tx=0x7f5ab800b540 comp rx=0 tx=0).stop 2024-04-25T00:15:17.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.666+0000 7f5accf8a700 1 -- 172.21.15.107:0/3827642765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ac81071b0 msgr2=0x7f5ac81038a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:17.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.666+0000 7f5accf8a700 1 --2- 172.21.15.107:0/3827642765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ac81071b0 0x7f5ac81038a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000b970 tx=0x7f5ac000bc80 comp rx=0 tx=0).stop 2024-04-25T00:15:17.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.667+0000 7f5accf8a700 1 -- 172.21.15.107:0/3827642765 shutdown_connections 2024-04-25T00:15:17.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.667+0000 7f5accf8a700 1 --2- 172.21.15.107:0/3827642765 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5ab00a2cb0 0x7f5ab00a5170 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:17.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.667+0000 7f5accf8a700 1 --2- 172.21.15.107:0/3827642765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ac81071b0 0x7f5ac81038a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:17.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.667+0000 7f5accf8a700 1 --2- 172.21.15.107:0/3827642765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5ac8101170 0x7f5ac8102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:17.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.667+0000 7f5accf8a700 1 --2- 172.21.15.107:0/3827642765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5ac8101b10 0x7f5ac8103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:17.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.667+0000 7f5accf8a700 1 -- 172.21.15.107:0/3827642765 >> 172.21.15.107:0/3827642765 conn(0x7f5ac8078060 msgr2=0x7f5ac8100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:17.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.667+0000 7f5accf8a700 1 -- 172.21.15.107:0/3827642765 shutdown_connections 2024-04-25T00:15:17.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:17.667+0000 7f5accf8a700 1 -- 172.21.15.107:0/3827642765 wait complete. 2024-04-25T00:15:17.670 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:15:18.108 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:18 smithi107 ceph-mon[29153]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:15:18.108 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:18 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3827642765' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T00:15:18.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:18 smithi195 ceph-mon[32833]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:15:18.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:18 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3827642765' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T00:15:18.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:18 smithi170 ceph-mon[32807]: pgmap v332: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:15:18.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:18 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3827642765' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T00:15:20.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:20 smithi195 ceph-mon[32833]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:15:20.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:20 smithi170 ceph-mon[32807]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:15:20.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:20 smithi107 ceph-mon[29153]: pgmap v333: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:15:21.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.818+0000 7f1011a5a700 1 -- 172.21.15.107:0/3954025343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f100c1024e0 msgr2=0x7f100c1028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:21.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.818+0000 7f1011a5a700 1 --2- 172.21.15.107:0/3954025343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f100c1024e0 0x7f100c1028c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc009990 tx=0x7f0ffc009ca0 comp rx=0 tx=0).stop 2024-04-25T00:15:21.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.818+0000 7f1011a5a700 1 -- 172.21.15.107:0/3954025343 shutdown_connections 2024-04-25T00:15:21.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.818+0000 7f1011a5a700 1 --2- 172.21.15.107:0/3954025343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f100c107570 0x7f100c10f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:21.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.818+0000 7f1011a5a700 1 --2- 172.21.15.107:0/3954025343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f100c1024e0 0x7f100c1028c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:21.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.818+0000 7f1011a5a700 1 --2- 172.21.15.107:0/3954025343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f100c102e90 0x7f100c106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:21.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.818+0000 7f1011a5a700 1 -- 172.21.15.107:0/3954025343 >> 172.21.15.107:0/3954025343 conn(0x7f100c079390 msgr2=0x7f100c0797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:21.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.818+0000 7f1011a5a700 1 -- 172.21.15.107:0/3954025343 shutdown_connections 2024-04-25T00:15:21.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.818+0000 7f1011a5a700 1 -- 172.21.15.107:0/3954025343 wait complete. 2024-04-25T00:15:21.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.819+0000 7f1011a5a700 1 Processor -- start 2024-04-25T00:15:21.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.819+0000 7f1011a5a700 1 -- start start 2024-04-25T00:15:21.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.819+0000 7f1011a5a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f100c1024e0 0x7f100c19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:21.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.819+0000 7f1011a5a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f100c102e90 0x7f100c19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:21.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.819+0000 7f1011a5a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f100c107570 0x7f100c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:21.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.819+0000 7f1011a5a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f100c1125b0 con 0x7f100c107570 2024-04-25T00:15:21.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.819+0000 7f1011a5a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f100c112430 con 0x7f100c1024e0 2024-04-25T00:15:21.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.819+0000 7f1011a5a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f100c112730 con 0x7f100c102e90 2024-04-25T00:15:21.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.819+0000 7f100b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f100c107570 0x7f100c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:21.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.819+0000 7f100b7fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f100c107570 0x7f100c198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54384/0 (socket says 172.21.15.107:54384) 2024-04-25T00:15:21.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.819+0000 7f100b7fe700 1 -- 172.21.15.107:0/1409406995 learned_addr learned my addr 172.21.15.107:0/1409406995 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:21.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.819+0000 7f100affd700 1 --2- 172.21.15.107:0/1409406995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f100c1024e0 0x7f100c19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:21.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.820+0000 7f10027fc700 1 --2- 172.21.15.107:0/1409406995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f100c102e90 0x7f100c19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:21.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.820+0000 7f100b7fe700 1 -- 172.21.15.107:0/1409406995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f100c102e90 msgr2=0x7f100c19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:21.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.820+0000 7f100b7fe700 1 --2- 172.21.15.107:0/1409406995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f100c102e90 0x7f100c19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:21.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.820+0000 7f100b7fe700 1 -- 172.21.15.107:0/1409406995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f100c1024e0 msgr2=0x7f100c19e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:21.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.820+0000 7f100b7fe700 1 --2- 172.21.15.107:0/1409406995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f100c1024e0 0x7f100c19e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:21.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.820+0000 7f100b7fe700 1 -- 172.21.15.107:0/1409406995 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f100c198fd0 con 0x7f100c107570 2024-04-25T00:15:21.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.820+0000 7f100b7fe700 1 --2- 172.21.15.107:0/1409406995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f100c107570 0x7f100c198870 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1004007ee0 tx=0x7f100400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:21.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.820+0000 7f1010a58700 1 -- 172.21.15.107:0/1409406995 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100400dbd0 con 0x7f100c107570 2024-04-25T00:15:21.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.820+0000 7f1010a58700 1 -- 172.21.15.107:0/1409406995 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f100400dd70 con 0x7f100c107570 2024-04-25T00:15:21.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.820+0000 7f1011a5a700 1 -- 172.21.15.107:0/1409406995 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f100c1992c0 con 0x7f100c107570 2024-04-25T00:15:21.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.820+0000 7f1010a58700 1 -- 172.21.15.107:0/1409406995 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1004005a60 con 0x7f100c107570 2024-04-25T00:15:21.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.820+0000 7f1011a5a700 1 -- 172.21.15.107:0/1409406995 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f100c1a5580 con 0x7f100c107570 2024-04-25T00:15:21.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.821+0000 7f1011a5a700 1 -- 172.21.15.107:0/1409406995 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f100c10a790 con 0x7f100c107570 2024-04-25T00:15:21.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.822+0000 7f1010a58700 1 -- 172.21.15.107:0/1409406995 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f100401a020 con 0x7f100c107570 2024-04-25T00:15:21.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.823+0000 7f1010a58700 1 --2- 172.21.15.107:0/1409406995 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f0ff80a2cb0 0x7f0ff80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:21.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.823+0000 7f100affd700 1 --2- 172.21.15.107:0/1409406995 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f0ff80a2cb0 0x7f0ff80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:21.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.823+0000 7f1010a58700 1 -- 172.21.15.107:0/1409406995 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f10040cec40 con 0x7f100c107570 2024-04-25T00:15:21.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.823+0000 7f100affd700 1 --2- 172.21.15.107:0/1409406995 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f0ff80a2cb0 0x7f0ff80a5170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00b5c0 tx=0x7f0ffc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:21.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:21.828+0000 7f1010a58700 1 -- 172.21.15.107:0/1409406995 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f10040d5080 con 0x7f100c107570 2024-04-25T00:15:22.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:22.037+0000 7f1011a5a700 1 -- 172.21.15.107:0/1409406995 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f100c10a170 con 0x7f100c107570 2024-04-25T00:15:22.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:22.037+0000 7f1010a58700 1 -- 172.21.15.107:0/1409406995 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1004010070 con 0x7f100c107570 2024-04-25T00:15:22.039 INFO:teuthology.orchestra.run.smithi107.stdout:146028888129 2024-04-25T00:15:22.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:22.039+0000 7f1011a5a700 1 -- 172.21.15.107:0/1409406995 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f0ff80a2cb0 msgr2=0x7f0ff80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:22.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:22.039+0000 7f1011a5a700 1 --2- 172.21.15.107:0/1409406995 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f0ff80a2cb0 0x7f0ff80a5170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00b5c0 tx=0x7f0ffc005960 comp rx=0 tx=0).stop 2024-04-25T00:15:22.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:22.039+0000 7f1011a5a700 1 -- 172.21.15.107:0/1409406995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f100c107570 msgr2=0x7f100c198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:22.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:22.039+0000 7f1011a5a700 1 --2- 172.21.15.107:0/1409406995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f100c107570 0x7f100c198870 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1004007ee0 tx=0x7f100400d3b0 comp rx=0 tx=0).stop 2024-04-25T00:15:22.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:22.039+0000 7f1011a5a700 1 -- 172.21.15.107:0/1409406995 shutdown_connections 2024-04-25T00:15:22.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:22.039+0000 7f1011a5a700 1 --2- 172.21.15.107:0/1409406995 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f0ff80a2cb0 0x7f0ff80a5170 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:22.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:22.039+0000 7f1011a5a700 1 --2- 172.21.15.107:0/1409406995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f100c107570 0x7f100c198870 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:22.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:22.039+0000 7f1011a5a700 1 --2- 172.21.15.107:0/1409406995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f100c1024e0 0x7f100c19e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:22.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:22.039+0000 7f1011a5a700 1 --2- 172.21.15.107:0/1409406995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f100c102e90 0x7f100c19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:22.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:22.039+0000 7f1011a5a700 1 -- 172.21.15.107:0/1409406995 >> 172.21.15.107:0/1409406995 conn(0x7f100c079390 msgr2=0x7f100c101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:22.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:22.039+0000 7f1011a5a700 1 -- 172.21.15.107:0/1409406995 shutdown_connections 2024-04-25T00:15:22.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:22.040+0000 7f1011a5a700 1 -- 172.21.15.107:0/1409406995 wait complete. 2024-04-25T00:15:22.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:22 smithi195 ceph-mon[32833]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T00:15:22.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:22 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1409406995' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T00:15:22.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:22 smithi170 ceph-mon[32807]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T00:15:22.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:22 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1409406995' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T00:15:22.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:22 smithi107 ceph-mon[29153]: pgmap v334: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T00:15:22.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:22 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1409406995' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T00:15:24.316 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:24 smithi107 ceph-mon[29153]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:15:24.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:24 smithi195 ceph-mon[32833]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:15:24.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:24 smithi170 ceph-mon[32807]: pgmap v335: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:15:24.667 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462962 got 201863462966 for osd.6 2024-04-25T00:15:24.668 DEBUG:teuthology.parallel:result is None 2024-04-25T00:15:24.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.314+0000 7fa42e674700 1 -- 172.21.15.107:0/2771273931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa428102540 msgr2=0x7fa428105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:24.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.314+0000 7fa42e674700 1 --2- 172.21.15.107:0/2771273931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa428102540 0x7fa428105dd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0099c0 tx=0x7fa41c009cd0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.314+0000 7fa42e674700 1 -- 172.21.15.107:0/2771273931 shutdown_connections 2024-04-25T00:15:24.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.314+0000 7fa42e674700 1 --2- 172.21.15.107:0/2771273931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa4281064b0 0x7fa42810e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.314+0000 7fa42e674700 1 --2- 172.21.15.107:0/2771273931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa428101b90 0x7fa428101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.314+0000 7fa42e674700 1 --2- 172.21.15.107:0/2771273931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa428102540 0x7fa428105dd0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.314+0000 7fa42e674700 1 -- 172.21.15.107:0/2771273931 >> 172.21.15.107:0/2771273931 conn(0x7fa4280777a0 msgr2=0x7fa4280feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:24.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.315+0000 7fa42e674700 1 -- 172.21.15.107:0/2771273931 shutdown_connections 2024-04-25T00:15:24.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.315+0000 7fa42e674700 1 -- 172.21.15.107:0/2771273931 wait complete. 2024-04-25T00:15:24.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.315+0000 7fa42e674700 1 Processor -- start 2024-04-25T00:15:24.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.315+0000 7fa42e674700 1 -- start start 2024-04-25T00:15:24.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.316+0000 7fa42e674700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa428101b90 0x7fa4280fcc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:24.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.316+0000 7fa42e674700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa428102540 0x7fa4280fd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:24.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.316+0000 7fa42e674700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4281064b0 0x7fa4280fd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:24.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.316+0000 7fa42e674700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa4281114d0 con 0x7fa4281064b0 2024-04-25T00:15:24.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.316+0000 7fa42e674700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa428111350 con 0x7fa428101b90 2024-04-25T00:15:24.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.316+0000 7fa42e674700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa428111650 con 0x7fa428102540 2024-04-25T00:15:24.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.316+0000 7fa42cc11700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4281064b0 0x7fa4280fd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:24.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.316+0000 7fa42cc11700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4281064b0 0x7fa4280fd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54416/0 (socket says 172.21.15.107:54416) 2024-04-25T00:15:24.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.316+0000 7fa42cc11700 1 -- 172.21.15.107:0/2555229485 learned_addr learned my addr 172.21.15.107:0/2555229485 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:24.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.316+0000 7fa427fff700 1 --2- 172.21.15.107:0/2555229485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa428101b90 0x7fa4280fcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:24.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.316+0000 7fa4277fe700 1 --2- 172.21.15.107:0/2555229485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa428102540 0x7fa4280fd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:24.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.316+0000 7fa42cc11700 1 -- 172.21.15.107:0/2555229485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa428102540 msgr2=0x7fa4280fd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:24.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.316+0000 7fa42cc11700 1 --2- 172.21.15.107:0/2555229485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa428102540 0x7fa4280fd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.316+0000 7fa42cc11700 1 -- 172.21.15.107:0/2555229485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa428101b90 msgr2=0x7fa4280fcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:24.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.316+0000 7fa42cc11700 1 --2- 172.21.15.107:0/2555229485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa428101b90 0x7fa4280fcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.316+0000 7fa42cc11700 1 -- 172.21.15.107:0/2555229485 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4281a6fe0 con 0x7fa4281064b0 2024-04-25T00:15:24.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.316+0000 7fa42cc11700 1 --2- 172.21.15.107:0/2555229485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4281064b0 0x7fa4280fd690 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa420007ee0 tx=0x7fa42000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:24.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.317+0000 7fa4257fa700 1 -- 172.21.15.107:0/2555229485 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420004bb0 con 0x7fa4281064b0 2024-04-25T00:15:24.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.317+0000 7fa42e674700 1 -- 172.21.15.107:0/2555229485 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4281a72d0 con 0x7fa4281064b0 2024-04-25T00:15:24.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.317+0000 7fa4257fa700 1 -- 172.21.15.107:0/2555229485 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa420004d50 con 0x7fa4281064b0 2024-04-25T00:15:24.754 INFO:teuthology.orchestra.run.smithi107.stdout:257698037805 2024-04-25T00:15:24.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.317+0000 7fa4257fa700 1 -- 172.21.15.107:0/2555229485 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa420005960 con 0x7fa4281064b0 2024-04-25T00:15:24.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.317+0000 7fa42e674700 1 -- 172.21.15.107:0/2555229485 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4281a7830 con 0x7fa4281064b0 2024-04-25T00:15:24.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.318+0000 7fa42e674700 1 -- 172.21.15.107:0/2555229485 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa428068f20 con 0x7fa4281064b0 2024-04-25T00:15:24.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.319+0000 7fa4257fa700 1 -- 172.21.15.107:0/2555229485 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fa42001a020 con 0x7fa4281064b0 2024-04-25T00:15:24.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.319+0000 7fa4257fa700 1 --2- 172.21.15.107:0/2555229485 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa4100a2cb0 0x7fa4100a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:24.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.320+0000 7fa427fff700 1 --2- 172.21.15.107:0/2555229485 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa4100a2cb0 0x7fa4100a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:24.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.320+0000 7fa4257fa700 1 -- 172.21.15.107:0/2555229485 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fa4200cf1d0 con 0x7fa4281064b0 2024-04-25T00:15:24.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.320+0000 7fa427fff700 1 --2- 172.21.15.107:0/2555229485 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa4100a2cb0 0x7fa4100a5170 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa418005950 tx=0x7fa41800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:24.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.323+0000 7fa4257fa700 1 -- 172.21.15.107:0/2555229485 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa420010070 con 0x7fa4281064b0 2024-04-25T00:15:24.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.527+0000 7fa42e674700 1 -- 172.21.15.107:0/2555229485 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fa4280fe140 con 0x7fa4281064b0 2024-04-25T00:15:24.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.527+0000 7fa4257fa700 1 -- 172.21.15.107:0/2555229485 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa4280fe140 con 0x7fa4281064b0 2024-04-25T00:15:24.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.529+0000 7fa42e674700 1 -- 172.21.15.107:0/2555229485 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa4100a2cb0 msgr2=0x7fa4100a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:24.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.529+0000 7fa42e674700 1 --2- 172.21.15.107:0/2555229485 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa4100a2cb0 0x7fa4100a5170 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa418005950 tx=0x7fa41800b410 comp rx=0 tx=0).stop 2024-04-25T00:15:24.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.529+0000 7fa42e674700 1 -- 172.21.15.107:0/2555229485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4281064b0 msgr2=0x7fa4280fd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:24.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.529+0000 7fa42e674700 1 --2- 172.21.15.107:0/2555229485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4281064b0 0x7fa4280fd690 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa420007ee0 tx=0x7fa42000d3b0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.530+0000 7fa42e674700 1 -- 172.21.15.107:0/2555229485 shutdown_connections 2024-04-25T00:15:24.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.530+0000 7fa42e674700 1 --2- 172.21.15.107:0/2555229485 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa4100a2cb0 0x7fa4100a5170 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.530+0000 7fa42e674700 1 --2- 172.21.15.107:0/2555229485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4281064b0 0x7fa4280fd690 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.530+0000 7fa42e674700 1 --2- 172.21.15.107:0/2555229485 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa428101b90 0x7fa4280fcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.530+0000 7fa42e674700 1 --2- 172.21.15.107:0/2555229485 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa428102540 0x7fa4280fd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:24.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.530+0000 7fa42e674700 1 -- 172.21.15.107:0/2555229485 >> 172.21.15.107:0/2555229485 conn(0x7fa4280777a0 msgr2=0x7fa428100a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:24.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.530+0000 7fa42e674700 1 -- 172.21.15.107:0/2555229485 shutdown_connections 2024-04-25T00:15:24.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.530+0000 7fa42e674700 1 -- 172.21.15.107:0/2555229485 wait complete. 2024-04-25T00:15:25.323 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:25 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3974978309' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T00:15:25.323 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:25 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2555229485' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T00:15:25.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:25 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3974978309' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T00:15:25.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:25 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2555229485' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T00:15:25.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:25 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3974978309' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T00:15:25.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:25 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2555229485' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T00:15:26.027 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:15:26.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:26 smithi107 ceph-mon[29153]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T00:15:26.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:26 smithi195 ceph-mon[32833]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T00:15:26.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:26 smithi170 ceph-mon[32807]: pgmap v336: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T00:15:27.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.479+0000 7efe7dd1a700 1 -- 172.21.15.107:0/3353714092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe78101ae0 msgr2=0x7efe7810dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.479+0000 7efe7dd1a700 1 --2- 172.21.15.107:0/3353714092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe78101ae0 0x7efe7810dfd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efe7000b970 tx=0x7efe70009260 comp rx=0 tx=0).stop 2024-04-25T00:15:27.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.479+0000 7efe7dd1a700 1 -- 172.21.15.107:0/3353714092 shutdown_connections 2024-04-25T00:15:27.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.479+0000 7efe7dd1a700 1 --2- 172.21.15.107:0/3353714092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe78101ae0 0x7efe7810dfd0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.479+0000 7efe7dd1a700 1 --2- 172.21.15.107:0/3353714092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe78107190 0x7efe78107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.479+0000 7efe7dd1a700 1 --2- 172.21.15.107:0/3353714092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe78101120 0x7efe781015a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.479+0000 7efe7dd1a700 1 -- 172.21.15.107:0/3353714092 >> 172.21.15.107:0/3353714092 conn(0x7efe780780a0 msgr2=0x7efe780784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:27.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.480+0000 7efe7dd1a700 1 -- 172.21.15.107:0/3353714092 shutdown_connections 2024-04-25T00:15:27.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.480+0000 7efe7dd1a700 1 -- 172.21.15.107:0/3353714092 wait complete. 2024-04-25T00:15:27.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.481+0000 7efe7dd1a700 1 Processor -- start 2024-04-25T00:15:27.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.481+0000 7efe7dd1a700 1 -- start start 2024-04-25T00:15:27.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.481+0000 7efe7dd1a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe78101120 0x7efe7819c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.482+0000 7efe7dd1a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe78101ae0 0x7efe7819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.482+0000 7efe7dd1a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe78107190 0x7efe781965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.482+0000 7efe7dd1a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efe78110360 con 0x7efe78101ae0 2024-04-25T00:15:27.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.482+0000 7efe7dd1a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efe781101e0 con 0x7efe78107190 2024-04-25T00:15:27.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.482+0000 7efe7dd1a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efe781104e0 con 0x7efe78101120 2024-04-25T00:15:27.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.482+0000 7efe76ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe78101ae0 0x7efe7819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.482+0000 7efe76ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe78101ae0 0x7efe7819c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54450/0 (socket says 172.21.15.107:54450) 2024-04-25T00:15:27.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.482+0000 7efe76ffd700 1 -- 172.21.15.107:0/2031515566 learned_addr learned my addr 172.21.15.107:0/2031515566 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:27.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.482+0000 7efe77fff700 1 --2- 172.21.15.107:0/2031515566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe78107190 0x7efe781965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.482+0000 7efe777fe700 1 --2- 172.21.15.107:0/2031515566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe78101120 0x7efe7819c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.482+0000 7efe76ffd700 1 -- 172.21.15.107:0/2031515566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe78101120 msgr2=0x7efe7819c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.482+0000 7efe76ffd700 1 --2- 172.21.15.107:0/2031515566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe78101120 0x7efe7819c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.482+0000 7efe76ffd700 1 -- 172.21.15.107:0/2031515566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe78107190 msgr2=0x7efe781965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.482+0000 7efe76ffd700 1 --2- 172.21.15.107:0/2031515566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe78107190 0x7efe781965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.482+0000 7efe76ffd700 1 -- 172.21.15.107:0/2031515566 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe78196d40 con 0x7efe78101ae0 2024-04-25T00:15:27.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.483+0000 7efe76ffd700 1 --2- 172.21.15.107:0/2031515566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe78101ae0 0x7efe7819c960 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efe6000bc50 tx=0x7efe6000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:27.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.483+0000 7efe74ff9700 1 -- 172.21.15.107:0/2031515566 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60008a20 con 0x7efe78101ae0 2024-04-25T00:15:27.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.483+0000 7efe7dd1a700 1 -- 172.21.15.107:0/2031515566 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe78197030 con 0x7efe78101ae0 2024-04-25T00:15:27.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.483+0000 7efe74ff9700 1 -- 172.21.15.107:0/2031515566 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe60008bc0 con 0x7efe78101ae0 2024-04-25T00:15:27.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.483+0000 7efe74ff9700 1 -- 172.21.15.107:0/2031515566 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60014600 con 0x7efe78101ae0 2024-04-25T00:15:27.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.483+0000 7efe7dd1a700 1 -- 172.21.15.107:0/2031515566 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe781a32f0 con 0x7efe78101ae0 2024-04-25T00:15:27.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.484+0000 7efe7dd1a700 1 -- 172.21.15.107:0/2031515566 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe78068f20 con 0x7efe78101ae0 2024-04-25T00:15:27.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.487+0000 7efe74ff9700 1 -- 172.21.15.107:0/2031515566 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7efe60015400 con 0x7efe78101ae0 2024-04-25T00:15:27.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.487+0000 7efe74ff9700 1 --2- 172.21.15.107:0/2031515566 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7efe640a2cb0 0x7efe640a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.488+0000 7efe74ff9700 1 -- 172.21.15.107:0/2031515566 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7efe600d0110 con 0x7efe78101ae0 2024-04-25T00:15:27.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.488+0000 7efe777fe700 1 --2- 172.21.15.107:0/2031515566 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7efe640a2cb0 0x7efe640a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.488+0000 7efe777fe700 1 --2- 172.21.15.107:0/2031515566 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7efe640a2cb0 0x7efe640a5170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efe68009e60 tx=0x7efe68015040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:27.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.491+0000 7efe74ff9700 1 -- 172.21.15.107:0/2031515566 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efe600252e0 con 0x7efe78101ae0 2024-04-25T00:15:27.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.697+0000 7efe7dd1a700 1 -- 172.21.15.107:0/2031515566 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7efe78197b10 con 0x7efe78101ae0 2024-04-25T00:15:27.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.697+0000 7efe74ff9700 1 -- 172.21.15.107:0/2031515566 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7efe6009cbd0 con 0x7efe78101ae0 2024-04-25T00:15:27.700 INFO:teuthology.orchestra.run.smithi107.stdout:347892351007 2024-04-25T00:15:27.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.699+0000 7efe7dd1a700 1 -- 172.21.15.107:0/2031515566 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7efe640a2cb0 msgr2=0x7efe640a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.699+0000 7efe7dd1a700 1 --2- 172.21.15.107:0/2031515566 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7efe640a2cb0 0x7efe640a5170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efe68009e60 tx=0x7efe68015040 comp rx=0 tx=0).stop 2024-04-25T00:15:27.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.699+0000 7efe7dd1a700 1 -- 172.21.15.107:0/2031515566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe78101ae0 msgr2=0x7efe7819c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.699+0000 7efe7dd1a700 1 --2- 172.21.15.107:0/2031515566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe78101ae0 0x7efe7819c960 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7efe6000bc50 tx=0x7efe6000bf60 comp rx=0 tx=0).stop 2024-04-25T00:15:27.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.699+0000 7efe7dd1a700 1 -- 172.21.15.107:0/2031515566 shutdown_connections 2024-04-25T00:15:27.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.699+0000 7efe7dd1a700 1 --2- 172.21.15.107:0/2031515566 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7efe640a2cb0 0x7efe640a5170 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.699+0000 7efe7dd1a700 1 --2- 172.21.15.107:0/2031515566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe78107190 0x7efe781965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.699+0000 7efe7dd1a700 1 --2- 172.21.15.107:0/2031515566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe78101120 0x7efe7819c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.699+0000 7efe7dd1a700 1 --2- 172.21.15.107:0/2031515566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe78101ae0 0x7efe7819c960 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.699+0000 7efe7dd1a700 1 -- 172.21.15.107:0/2031515566 >> 172.21.15.107:0/2031515566 conn(0x7efe780780a0 msgr2=0x7efe780ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:27.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.699+0000 7efe7dd1a700 1 -- 172.21.15.107:0/2031515566 shutdown_connections 2024-04-25T00:15:27.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.699+0000 7efe7dd1a700 1 -- 172.21.15.107:0/2031515566 wait complete. 2024-04-25T00:15:27.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.368+0000 7f1dcb1bd700 1 -- 172.21.15.107:0/1962631620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dc4102e90 msgr2=0x7f1dc4106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.368+0000 7f1dcb1bd700 1 --2- 172.21.15.107:0/1962631620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dc4102e90 0x7f1dc4106e90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1db0009990 tx=0x7f1db0009ca0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.368+0000 7f1dcb1bd700 1 -- 172.21.15.107:0/1962631620 shutdown_connections 2024-04-25T00:15:27.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.368+0000 7f1dcb1bd700 1 --2- 172.21.15.107:0/1962631620 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dc4107570 0x7f1dc410f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.368+0000 7f1dcb1bd700 1 --2- 172.21.15.107:0/1962631620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dc41024e0 0x7f1dc41028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.368+0000 7f1dcb1bd700 1 --2- 172.21.15.107:0/1962631620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dc4102e90 0x7f1dc4106e90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.368+0000 7f1dcb1bd700 1 -- 172.21.15.107:0/1962631620 >> 172.21.15.107:0/1962631620 conn(0x7f1dc4079390 msgr2=0x7f1dc40797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:27.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.369+0000 7f1dcb1bd700 1 -- 172.21.15.107:0/1962631620 shutdown_connections 2024-04-25T00:15:27.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.369+0000 7f1dcb1bd700 1 -- 172.21.15.107:0/1962631620 wait complete. 2024-04-25T00:15:27.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.369+0000 7f1dcb1bd700 1 Processor -- start 2024-04-25T00:15:27.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.369+0000 7f1dcb1bd700 1 -- start start 2024-04-25T00:15:27.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.369+0000 7f1dcb1bd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dc41024e0 0x7f1dc4074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.369+0000 7f1dcb1bd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dc4102e90 0x7f1dc4074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.369+0000 7f1dcb1bd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dc4107570 0x7f1dc4074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.369+0000 7f1dcb1bd700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc4112630 con 0x7f1dc41024e0 2024-04-25T00:15:27.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.369+0000 7f1dcb1bd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc41124b0 con 0x7f1dc4107570 2024-04-25T00:15:27.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.369+0000 7f1dcb1bd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc41127b0 con 0x7f1dc4102e90 2024-04-25T00:15:27.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.369+0000 7f1dc8f59700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dc41024e0 0x7f1dc4074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.369+0000 7f1dc8f59700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dc41024e0 0x7f1dc4074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54432/0 (socket says 172.21.15.107:54432) 2024-04-25T00:15:27.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.369+0000 7f1dc8f59700 1 -- 172.21.15.107:0/624966630 learned_addr learned my addr 172.21.15.107:0/624966630 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:27.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.370+0000 7f1dc3fff700 1 --2- 172.21.15.107:0/624966630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dc4102e90 0x7f1dc4074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.370+0000 7f1dc8f59700 1 -- 172.21.15.107:0/624966630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dc4102e90 msgr2=0x7f1dc4074a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.370+0000 7f1dc8f59700 1 --2- 172.21.15.107:0/624966630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dc4102e90 0x7f1dc4074a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.370+0000 7f1dc8f59700 1 -- 172.21.15.107:0/624966630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dc4107570 msgr2=0x7f1dc4074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.370+0000 7f1dc975a700 1 --2- 172.21.15.107:0/624966630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dc4107570 0x7f1dc4074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.370+0000 7f1dc8f59700 1 --2- 172.21.15.107:0/624966630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dc4107570 0x7f1dc4074fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.370+0000 7f1dc8f59700 1 -- 172.21.15.107:0/624966630 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc41a7030 con 0x7f1dc41024e0 2024-04-25T00:15:27.904 INFO:teuthology.orchestra.run.smithi107.stdout:292057776169 2024-04-25T00:15:27.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.370+0000 7f1dc3fff700 1 --2- 172.21.15.107:0/624966630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dc4102e90 0x7f1dc4074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:27.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.370+0000 7f1dc975a700 1 --2- 172.21.15.107:0/624966630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dc4107570 0x7f1dc4074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:27.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.370+0000 7f1dc8f59700 1 --2- 172.21.15.107:0/624966630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dc41024e0 0x7f1dc4074530 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1db800b550 tx=0x7f1db800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:27.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.370+0000 7f1dc1ffb700 1 -- 172.21.15.107:0/624966630 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8015020 con 0x7f1dc41024e0 2024-04-25T00:15:27.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.370+0000 7f1dc1ffb700 1 -- 172.21.15.107:0/624966630 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1db8011d10 con 0x7f1dc41024e0 2024-04-25T00:15:27.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.370+0000 7f1dc1ffb700 1 -- 172.21.15.107:0/624966630 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8010e50 con 0x7f1dc41024e0 2024-04-25T00:15:27.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.370+0000 7f1dcb1bd700 1 -- 172.21.15.107:0/624966630 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc41a7320 con 0x7f1dc41024e0 2024-04-25T00:15:27.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.370+0000 7f1dcb1bd700 1 -- 172.21.15.107:0/624966630 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc41a7860 con 0x7f1dc41024e0 2024-04-25T00:15:27.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.371+0000 7f1dcb1bd700 1 -- 172.21.15.107:0/624966630 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc410a800 con 0x7f1dc41024e0 2024-04-25T00:15:27.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.372+0000 7f1dc1ffb700 1 -- 172.21.15.107:0/624966630 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f1db8018020 con 0x7f1dc41024e0 2024-04-25T00:15:27.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.373+0000 7f1dc1ffb700 1 --2- 172.21.15.107:0/624966630 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f1db40a2e50 0x7f1db40a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:27.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.373+0000 7f1dc3fff700 1 --2- 172.21.15.107:0/624966630 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f1db40a2e50 0x7f1db40a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:27.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.373+0000 7f1dc1ffb700 1 -- 172.21.15.107:0/624966630 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f1db80cec80 con 0x7f1dc41024e0 2024-04-25T00:15:27.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.373+0000 7f1dc3fff700 1 --2- 172.21.15.107:0/624966630 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f1db40a2e50 0x7f1db40a5310 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1db0006010 tx=0x7f1db000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:27.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.378+0000 7f1dc1ffb700 1 -- 172.21.15.107:0/624966630 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1db801f020 con 0x7f1dc41024e0 2024-04-25T00:15:27.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.591+0000 7f1dcb1bd700 1 -- 172.21.15.107:0/624966630 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f1dc410a1e0 con 0x7f1dc41024e0 2024-04-25T00:15:27.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.591+0000 7f1dc1ffb700 1 -- 172.21.15.107:0/624966630 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1db809b850 con 0x7f1dc41024e0 2024-04-25T00:15:27.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.593+0000 7f1dcb1bd700 1 -- 172.21.15.107:0/624966630 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f1db40a2e50 msgr2=0x7f1db40a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.593+0000 7f1dcb1bd700 1 --2- 172.21.15.107:0/624966630 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f1db40a2e50 0x7f1db40a5310 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1db0006010 tx=0x7f1db000b540 comp rx=0 tx=0).stop 2024-04-25T00:15:27.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.593+0000 7f1dcb1bd700 1 -- 172.21.15.107:0/624966630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dc41024e0 msgr2=0x7f1dc4074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:27.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.593+0000 7f1dcb1bd700 1 --2- 172.21.15.107:0/624966630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dc41024e0 0x7f1dc4074530 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1db800b550 tx=0x7f1db800b910 comp rx=0 tx=0).stop 2024-04-25T00:15:27.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.593+0000 7f1dcb1bd700 1 -- 172.21.15.107:0/624966630 shutdown_connections 2024-04-25T00:15:27.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.593+0000 7f1dcb1bd700 1 --2- 172.21.15.107:0/624966630 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f1db40a2e50 0x7f1db40a5310 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.593+0000 7f1dcb1bd700 1 --2- 172.21.15.107:0/624966630 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dc4107570 0x7f1dc4074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.593+0000 7f1dcb1bd700 1 --2- 172.21.15.107:0/624966630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dc41024e0 0x7f1dc4074530 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.593+0000 7f1dcb1bd700 1 --2- 172.21.15.107:0/624966630 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dc4102e90 0x7f1dc4074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:27.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.593+0000 7f1dcb1bd700 1 -- 172.21.15.107:0/624966630 >> 172.21.15.107:0/624966630 conn(0x7f1dc4079390 msgr2=0x7f1dc4101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:27.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.593+0000 7f1dcb1bd700 1 -- 172.21.15.107:0/624966630 shutdown_connections 2024-04-25T00:15:27.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:27.593+0000 7f1dcb1bd700 1 -- 172.21.15.107:0/624966630 wait complete. 2024-04-25T00:15:28.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:28 smithi195 ceph-mon[32833]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:15:28.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:28 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/624966630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T00:15:28.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:28 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2031515566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T00:15:28.363 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:28 smithi107 ceph-mon[29153]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:15:28.363 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:28 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/624966630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T00:15:28.364 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:28 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2031515566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T00:15:28.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:28 smithi170 ceph-mon[32807]: pgmap v337: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:15:28.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:28 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/624966630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T00:15:28.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:28 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2031515566' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T00:15:28.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.204+0000 7f72e05c3700 1 -- 172.21.15.107:0/3852910482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72d8107590 msgr2=0x7f72d810f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.204+0000 7f72e05c3700 1 --2- 172.21.15.107:0/3852910482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72d8107590 0x7f72d810f160 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f72d000b0a0 tx=0x7f72d000b3b0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.204+0000 7f72e05c3700 1 -- 172.21.15.107:0/3852910482 shutdown_connections 2024-04-25T00:15:28.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.204+0000 7f72e05c3700 1 --2- 172.21.15.107:0/3852910482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72d8107590 0x7f72d810f160 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.204+0000 7f72e05c3700 1 --2- 172.21.15.107:0/3852910482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72d8102510 0x7f72d81028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.204+0000 7f72e05c3700 1 --2- 172.21.15.107:0/3852910482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72d8102ec0 0x7f72d8106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.204+0000 7f72e05c3700 1 -- 172.21.15.107:0/3852910482 >> 172.21.15.107:0/3852910482 conn(0x7f72d80793a0 msgr2=0x7f72d80797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:28.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.204+0000 7f72e05c3700 1 -- 172.21.15.107:0/3852910482 shutdown_connections 2024-04-25T00:15:28.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.204+0000 7f72e05c3700 1 -- 172.21.15.107:0/3852910482 wait complete. 2024-04-25T00:15:28.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.204+0000 7f72e05c3700 1 Processor -- start 2024-04-25T00:15:28.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.204+0000 7f72e05c3700 1 -- start start 2024-04-25T00:15:28.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.205+0000 7f72e05c3700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72d8102510 0x7f72d819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.205+0000 7f72e05c3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72d8102ec0 0x7f72d819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.205+0000 7f72e05c3700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72d8107590 0x7f72d8198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.205+0000 7f72e05c3700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f72d8112650 con 0x7f72d8102510 2024-04-25T00:15:28.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.205+0000 7f72e05c3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f72d81124d0 con 0x7f72d8102ec0 2024-04-25T00:15:28.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.205+0000 7f72e05c3700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f72d81127d0 con 0x7f72d8107590 2024-04-25T00:15:28.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.205+0000 7f72de35f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72d8102510 0x7f72d819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.205+0000 7f72de35f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72d8102510 0x7f72d819e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54402/0 (socket says 172.21.15.107:54402) 2024-04-25T00:15:28.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.205+0000 7f72de35f700 1 -- 172.21.15.107:0/3974978309 learned_addr learned my addr 172.21.15.107:0/3974978309 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:28.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.205+0000 7f72deb60700 1 --2- 172.21.15.107:0/3974978309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72d8107590 0x7f72d8198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.205+0000 7f72de35f700 1 -- 172.21.15.107:0/3974978309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72d8107590 msgr2=0x7f72d8198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.205+0000 7f72de35f700 1 --2- 172.21.15.107:0/3974978309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72d8107590 0x7f72d8198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.205+0000 7f72ddb5e700 1 --2- 172.21.15.107:0/3974978309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72d8102ec0 0x7f72d819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.205+0000 7f72de35f700 1 -- 172.21.15.107:0/3974978309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72d8102ec0 msgr2=0x7f72d819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.205+0000 7f72de35f700 1 --2- 172.21.15.107:0/3974978309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72d8102ec0 0x7f72d819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.633 INFO:teuthology.orchestra.run.smithi107.stdout:223338299441 2024-04-25T00:15:28.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.205+0000 7f72de35f700 1 -- 172.21.15.107:0/3974978309 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d8199010 con 0x7f72d8102510 2024-04-25T00:15:28.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.205+0000 7f72ddb5e700 1 --2- 172.21.15.107:0/3974978309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72d8102ec0 0x7f72d819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:28.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.205+0000 7f72de35f700 1 --2- 172.21.15.107:0/3974978309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72d8102510 0x7f72d819e6b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f72d400b530 tx=0x7f72d400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:28.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.205+0000 7f72cf7fe700 1 -- 172.21.15.107:0/3974978309 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d4015020 con 0x7f72d8102510 2024-04-25T00:15:28.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.205+0000 7f72cf7fe700 1 -- 172.21.15.107:0/3974978309 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72d4011d10 con 0x7f72d8102510 2024-04-25T00:15:28.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.205+0000 7f72e05c3700 1 -- 172.21.15.107:0/3974978309 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d8199300 con 0x7f72d8102510 2024-04-25T00:15:28.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.205+0000 7f72cf7fe700 1 -- 172.21.15.107:0/3974978309 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d4010e50 con 0x7f72d8102510 2024-04-25T00:15:28.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.205+0000 7f72e05c3700 1 -- 172.21.15.107:0/3974978309 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d81a5510 con 0x7f72d8102510 2024-04-25T00:15:28.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.206+0000 7f72e05c3700 1 -- 172.21.15.107:0/3974978309 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72d8103990 con 0x7f72d8102510 2024-04-25T00:15:28.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.207+0000 7f72cf7fe700 1 -- 172.21.15.107:0/3974978309 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f72d4018020 con 0x7f72d8102510 2024-04-25T00:15:28.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.208+0000 7f72cf7fe700 1 --2- 172.21.15.107:0/3974978309 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f72c40a2e50 0x7f72c40a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.208+0000 7f72ddb5e700 1 --2- 172.21.15.107:0/3974978309 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f72c40a2e50 0x7f72c40a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.208+0000 7f72cf7fe700 1 -- 172.21.15.107:0/3974978309 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f72d40cf250 con 0x7f72d8102510 2024-04-25T00:15:28.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.208+0000 7f72ddb5e700 1 --2- 172.21.15.107:0/3974978309 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f72c40a2e50 0x7f72c40a5310 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f72d8108930 tx=0x7f72c8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:28.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.212+0000 7f72cf7fe700 1 -- 172.21.15.107:0/3974978309 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f72d401f020 con 0x7f72d8102510 2024-04-25T00:15:28.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.426+0000 7f72e05c3700 1 -- 172.21.15.107:0/3974978309 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f72d802db30 con 0x7f72d8102510 2024-04-25T00:15:28.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.427+0000 7f72cf7fe700 1 -- 172.21.15.107:0/3974978309 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f72d409be20 con 0x7f72d8102510 2024-04-25T00:15:28.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.429+0000 7f72e05c3700 1 -- 172.21.15.107:0/3974978309 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f72c40a2e50 msgr2=0x7f72c40a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.429+0000 7f72e05c3700 1 --2- 172.21.15.107:0/3974978309 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f72c40a2e50 0x7f72c40a5310 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f72d8108930 tx=0x7f72c8008040 comp rx=0 tx=0).stop 2024-04-25T00:15:28.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.429+0000 7f72e05c3700 1 -- 172.21.15.107:0/3974978309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72d8102510 msgr2=0x7f72d819e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.429+0000 7f72e05c3700 1 --2- 172.21.15.107:0/3974978309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72d8102510 0x7f72d819e6b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f72d400b530 tx=0x7f72d400b8f0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.429+0000 7f72e05c3700 1 -- 172.21.15.107:0/3974978309 shutdown_connections 2024-04-25T00:15:28.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.429+0000 7f72e05c3700 1 --2- 172.21.15.107:0/3974978309 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f72c40a2e50 0x7f72c40a5310 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.429+0000 7f72e05c3700 1 --2- 172.21.15.107:0/3974978309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72d8107590 0x7f72d8198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.429+0000 7f72e05c3700 1 --2- 172.21.15.107:0/3974978309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72d8102510 0x7f72d819e6b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.429+0000 7f72e05c3700 1 --2- 172.21.15.107:0/3974978309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72d8102ec0 0x7f72d819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.429+0000 7f72e05c3700 1 -- 172.21.15.107:0/3974978309 >> 172.21.15.107:0/3974978309 conn(0x7f72d80793a0 msgr2=0x7f72d81002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:28.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.429+0000 7f72e05c3700 1 -- 172.21.15.107:0/3974978309 shutdown_connections 2024-04-25T00:15:28.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:24.429+0000 7f72e05c3700 1 -- 172.21.15.107:0/3974978309 wait complete. 2024-04-25T00:15:29.287 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299436 got 223338299441 for osd.7 2024-04-25T00:15:29.287 DEBUG:teuthology.parallel:result is None 2024-04-25T00:15:29.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:29 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1035544829' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T00:15:29.366 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:29 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1035544829' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T00:15:29.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:29 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1035544829' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T00:15:29.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.639+0000 7fc25272c700 1 -- 172.21.15.107:0/1805859314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc24c1071d0 msgr2=0x7fc24c1075b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.639+0000 7fc25272c700 1 --2- 172.21.15.107:0/1805859314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc24c1071d0 0x7fc24c1075b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009990 tx=0x7fc23c009ca0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.639+0000 7fc25272c700 1 -- 172.21.15.107:0/1805859314 shutdown_connections 2024-04-25T00:15:29.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.639+0000 7fc25272c700 1 --2- 172.21.15.107:0/1805859314 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc24c101b20 0x7fc24c10e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.639+0000 7fc25272c700 1 --2- 172.21.15.107:0/1805859314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc24c1071d0 0x7fc24c1075b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.639+0000 7fc25272c700 1 --2- 172.21.15.107:0/1805859314 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc24c101160 0x7fc24c1015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.639+0000 7fc25272c700 1 -- 172.21.15.107:0/1805859314 >> 172.21.15.107:0/1805859314 conn(0x7fc24c078170 msgr2=0x7fc24c078580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:29.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.639+0000 7fc25272c700 1 -- 172.21.15.107:0/1805859314 shutdown_connections 2024-04-25T00:15:29.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.639+0000 7fc25272c700 1 -- 172.21.15.107:0/1805859314 wait complete. 2024-04-25T00:15:29.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.640+0000 7fc25272c700 1 Processor -- start 2024-04-25T00:15:29.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.640+0000 7fc25272c700 1 -- start start 2024-04-25T00:15:29.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.640+0000 7fc25272c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc24c101160 0x7fc24c199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:29.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.640+0000 7fc25272c700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc24c101b20 0x7fc24c199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:29.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.640+0000 7fc25272c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc24c1071d0 0x7fc24c1a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:29.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.640+0000 7fc25272c700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c111470 con 0x7fc24c1071d0 2024-04-25T00:15:29.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.640+0000 7fc25272c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c1112f0 con 0x7fc24c101160 2024-04-25T00:15:29.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.640+0000 7fc25272c700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c1115f0 con 0x7fc24c101b20 2024-04-25T00:15:29.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.640+0000 7fc250cc9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc24c1071d0 0x7fc24c1a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:29.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.640+0000 7fc250cc9700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc24c1071d0 0x7fc24c1a48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54472/0 (socket says 172.21.15.107:54472) 2024-04-25T00:15:29.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.640+0000 7fc250cc9700 1 -- 172.21.15.107:0/1035544829 learned_addr learned my addr 172.21.15.107:0/1035544829 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:29.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.640+0000 7fc24bfff700 1 --2- 172.21.15.107:0/1035544829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc24c101160 0x7fc24c199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:29.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.640+0000 7fc24b7fe700 1 --2- 172.21.15.107:0/1035544829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc24c101b20 0x7fc24c199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:29.465 INFO:teuthology.orchestra.run.smithi107.stdout:77309411404 2024-04-25T00:15:29.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.640+0000 7fc250cc9700 1 -- 172.21.15.107:0/1035544829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc24c101b20 msgr2=0x7fc24c199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.640+0000 7fc250cc9700 1 --2- 172.21.15.107:0/1035544829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc24c101b20 0x7fc24c199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.640+0000 7fc250cc9700 1 -- 172.21.15.107:0/1035544829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc24c101160 msgr2=0x7fc24c199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.640+0000 7fc250cc9700 1 --2- 172.21.15.107:0/1035544829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc24c101160 0x7fc24c199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.640+0000 7fc250cc9700 1 -- 172.21.15.107:0/1035544829 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc24c1a4e30 con 0x7fc24c1071d0 2024-04-25T00:15:29.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.640+0000 7fc24b7fe700 1 --2- 172.21.15.107:0/1035544829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc24c101b20 0x7fc24c199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:29.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.641+0000 7fc250cc9700 1 --2- 172.21.15.107:0/1035544829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc24c1071d0 0x7fc24c1a48f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc244007ee0 tx=0x7fc24400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:29.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.641+0000 7fc2497fa700 1 -- 172.21.15.107:0/1035544829 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24400dbd0 con 0x7fc24c1071d0 2024-04-25T00:15:29.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.641+0000 7fc25272c700 1 -- 172.21.15.107:0/1035544829 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc24c1a5000 con 0x7fc24c1071d0 2024-04-25T00:15:29.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.641+0000 7fc2497fa700 1 -- 172.21.15.107:0/1035544829 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc24400dd70 con 0x7fc24c1071d0 2024-04-25T00:15:29.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.641+0000 7fc2497fa700 1 -- 172.21.15.107:0/1035544829 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc244005a60 con 0x7fc24c1071d0 2024-04-25T00:15:29.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.641+0000 7fc25272c700 1 -- 172.21.15.107:0/1035544829 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc24c1a5540 con 0x7fc24c1071d0 2024-04-25T00:15:29.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.642+0000 7fc25272c700 1 -- 172.21.15.107:0/1035544829 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc24c102880 con 0x7fc24c1071d0 2024-04-25T00:15:29.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.643+0000 7fc2497fa700 1 -- 172.21.15.107:0/1035544829 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fc24401a020 con 0x7fc24c1071d0 2024-04-25T00:15:29.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.644+0000 7fc2497fa700 1 --2- 172.21.15.107:0/1035544829 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc2340a2cb0 0x7fc2340a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:29.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.644+0000 7fc24bfff700 1 --2- 172.21.15.107:0/1035544829 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc2340a2cb0 0x7fc2340a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:29.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.644+0000 7fc2497fa700 1 -- 172.21.15.107:0/1035544829 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fc2440ceba0 con 0x7fc24c1071d0 2024-04-25T00:15:29.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.644+0000 7fc24bfff700 1 --2- 172.21.15.107:0/1035544829 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc2340a2cb0 0x7fc2340a5170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00b5c0 tx=0x7fc23c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:29.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.648+0000 7fc2497fa700 1 -- 172.21.15.107:0/1035544829 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc244010070 con 0x7fc24c1071d0 2024-04-25T00:15:29.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.849+0000 7fc25272c700 1 -- 172.21.15.107:0/1035544829 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc24c02db30 con 0x7fc24c1071d0 2024-04-25T00:15:29.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.849+0000 7fc2497fa700 1 -- 172.21.15.107:0/1035544829 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc24409b660 con 0x7fc24c1071d0 2024-04-25T00:15:29.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.851+0000 7fc25272c700 1 -- 172.21.15.107:0/1035544829 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc2340a2cb0 msgr2=0x7fc2340a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.851+0000 7fc25272c700 1 --2- 172.21.15.107:0/1035544829 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc2340a2cb0 0x7fc2340a5170 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00b5c0 tx=0x7fc23c005960 comp rx=0 tx=0).stop 2024-04-25T00:15:29.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.851+0000 7fc25272c700 1 -- 172.21.15.107:0/1035544829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc24c1071d0 msgr2=0x7fc24c1a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.851+0000 7fc25272c700 1 --2- 172.21.15.107:0/1035544829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc24c1071d0 0x7fc24c1a48f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc244007ee0 tx=0x7fc24400d3b0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.851+0000 7fc25272c700 1 -- 172.21.15.107:0/1035544829 shutdown_connections 2024-04-25T00:15:29.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.851+0000 7fc25272c700 1 --2- 172.21.15.107:0/1035544829 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc2340a2cb0 0x7fc2340a5170 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.851+0000 7fc25272c700 1 --2- 172.21.15.107:0/1035544829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc24c1071d0 0x7fc24c1a48f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.851+0000 7fc25272c700 1 --2- 172.21.15.107:0/1035544829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc24c101160 0x7fc24c199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.851+0000 7fc25272c700 1 --2- 172.21.15.107:0/1035544829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc24c101b20 0x7fc24c199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.851+0000 7fc25272c700 1 -- 172.21.15.107:0/1035544829 >> 172.21.15.107:0/1035544829 conn(0x7fc24c078170 msgr2=0x7fc24c0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:29.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.851+0000 7fc25272c700 1 -- 172.21.15.107:0/1035544829 shutdown_connections 2024-04-25T00:15:29.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:28.851+0000 7fc25272c700 1 -- 172.21.15.107:0/1035544829 wait complete. 2024-04-25T00:15:30.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:30 smithi195 ceph-mon[32833]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:15:30.367 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:30 smithi107 ceph-mon[29153]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:15:30.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:30 smithi170 ceph-mon[32807]: pgmap v338: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:15:30.464 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351001 got 347892351007 for osd.11 2024-04-25T00:15:30.465 DEBUG:teuthology.parallel:result is None 2024-04-25T00:15:30.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.756+0000 7fe4aa821700 1 -- 172.21.15.107:0/1523482862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4a4101130 msgr2=0x7fe4a41015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:30.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.756+0000 7fe4aa821700 1 --2- 172.21.15.107:0/1523482862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4a4101130 0x7fe4a41015b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe498009990 tx=0x7fe498009ca0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.756+0000 7fe4aa821700 1 -- 172.21.15.107:0/1523482862 shutdown_connections 2024-04-25T00:15:30.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.756+0000 7fe4aa821700 1 --2- 172.21.15.107:0/1523482862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4a4101af0 0x7fe4a410dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.756+0000 7fe4aa821700 1 --2- 172.21.15.107:0/1523482862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4a41071a0 0x7fe4a4107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.756+0000 7fe4aa821700 1 --2- 172.21.15.107:0/1523482862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4a4101130 0x7fe4a41015b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.756+0000 7fe4aa821700 1 -- 172.21.15.107:0/1523482862 >> 172.21.15.107:0/1523482862 conn(0x7fe4a40780e0 msgr2=0x7fe4a40784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:30.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.756+0000 7fe4aa821700 1 -- 172.21.15.107:0/1523482862 shutdown_connections 2024-04-25T00:15:30.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.756+0000 7fe4aa821700 1 -- 172.21.15.107:0/1523482862 wait complete. 2024-04-25T00:15:30.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.757+0000 7fe4aa821700 1 Processor -- start 2024-04-25T00:15:30.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.757+0000 7fe4aa821700 1 -- start start 2024-04-25T00:15:30.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.757+0000 7fe4aa821700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4a4101130 0x7fe4a419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:30.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.757+0000 7fe4aa821700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4a4101af0 0x7fe4a419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:30.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.757+0000 7fe4aa821700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4a41071a0 0x7fe4a41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:30.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.757+0000 7fe4aa821700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a4111510 con 0x7fe4a4101af0 2024-04-25T00:15:30.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.757+0000 7fe4aa821700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a4111390 con 0x7fe4a4101130 2024-04-25T00:15:30.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.757+0000 7fe4aa821700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a4111690 con 0x7fe4a41071a0 2024-04-25T00:15:30.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.757+0000 7fe4a37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4a4101af0 0x7fe4a419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:30.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.757+0000 7fe4a37fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4a4101af0 0x7fe4a419ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56460/0 (socket says 172.21.15.107:56460) 2024-04-25T00:15:30.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.757+0000 7fe4a37fe700 1 -- 172.21.15.107:0/3148300183 learned_addr learned my addr 172.21.15.107:0/3148300183 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:30.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.757+0000 7fe4a3fff700 1 --2- 172.21.15.107:0/3148300183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4a4101130 0x7fe4a419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:30.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.757+0000 7fe4a8dbe700 1 --2- 172.21.15.107:0/3148300183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4a41071a0 0x7fe4a41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:30.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.757+0000 7fe4a37fe700 1 -- 172.21.15.107:0/3148300183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4a41071a0 msgr2=0x7fe4a41989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:30.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.757+0000 7fe4a37fe700 1 --2- 172.21.15.107:0/3148300183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4a41071a0 0x7fe4a41989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.758+0000 7fe4a37fe700 1 -- 172.21.15.107:0/3148300183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4a4101130 msgr2=0x7fe4a419e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:30.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.758+0000 7fe4a37fe700 1 --2- 172.21.15.107:0/3148300183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4a4101130 0x7fe4a419e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.758+0000 7fe4a37fe700 1 -- 172.21.15.107:0/3148300183 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a4199100 con 0x7fe4a4101af0 2024-04-25T00:15:30.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.758+0000 7fe4a37fe700 1 --2- 172.21.15.107:0/3148300183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4a4101af0 0x7fe4a419ed20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe498000c00 tx=0x7fe498003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:30.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.758+0000 7fe4a17fa700 1 -- 172.21.15.107:0/3148300183 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498027070 con 0x7fe4a4101af0 2024-04-25T00:15:30.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.758+0000 7fe4aa821700 1 -- 172.21.15.107:0/3148300183 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a4199390 con 0x7fe4a4101af0 2024-04-25T00:15:30.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.758+0000 7fe4a17fa700 1 -- 172.21.15.107:0/3148300183 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe498004110 con 0x7fe4a4101af0 2024-04-25T00:15:30.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.758+0000 7fe4a17fa700 1 -- 172.21.15.107:0/3148300183 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49800f460 con 0x7fe4a4101af0 2024-04-25T00:15:30.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.758+0000 7fe4aa821700 1 -- 172.21.15.107:0/3148300183 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a41a56b0 con 0x7fe4a4101af0 2024-04-25T00:15:30.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.759+0000 7fe4aa821700 1 -- 172.21.15.107:0/3148300183 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4a4068f20 con 0x7fe4a4101af0 2024-04-25T00:15:30.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.760+0000 7fe4a17fa700 1 -- 172.21.15.107:0/3148300183 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7fe49802a070 con 0x7fe4a4101af0 2024-04-25T00:15:30.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.761+0000 7fe4a17fa700 1 --2- 172.21.15.107:0/3148300183 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe48c0a2cb0 0x7fe48c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:30.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.761+0000 7fe4a3fff700 1 --2- 172.21.15.107:0/3148300183 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe48c0a2cb0 0x7fe48c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:30.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.761+0000 7fe4a17fa700 1 -- 172.21.15.107:0/3148300183 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fe4980d0910 con 0x7fe4a4101af0 2024-04-25T00:15:30.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.761+0000 7fe4a3fff700 1 --2- 172.21.15.107:0/3148300183 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe48c0a2cb0 0x7fe48c0a5170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4199ae0 tx=0x7fe494009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:30.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.765+0000 7fe4a17fa700 1 -- 172.21.15.107:0/3148300183 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe498031070 con 0x7fe4a4101af0 2024-04-25T00:15:30.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.985+0000 7f64692fd700 1 -- 172.21.15.107:0/225771463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6464069f40 msgr2=0x7f646406a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:30.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.985+0000 7f64692fd700 1 --2- 172.21.15.107:0/225771463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6464069f40 0x7f646406a320 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6454009990 tx=0x7f6454009ca0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.985+0000 7f64692fd700 1 -- 172.21.15.107:0/225771463 shutdown_connections 2024-04-25T00:15:30.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.985+0000 7f64692fd700 1 --2- 172.21.15.107:0/225771463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f646410ece0 0x7f64641110d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.985+0000 7f64692fd700 1 --2- 172.21.15.107:0/225771463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6464069f40 0x7f646406a320 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.985+0000 7f64692fd700 1 --2- 172.21.15.107:0/225771463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f646406a860 0x7f646410e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.985+0000 7f64692fd700 1 -- 172.21.15.107:0/225771463 >> 172.21.15.107:0/225771463 conn(0x7f64640780e0 msgr2=0x7f64640784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:30.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.985+0000 7f64692fd700 1 -- 172.21.15.107:0/225771463 shutdown_connections 2024-04-25T00:15:30.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.985+0000 7f64692fd700 1 -- 172.21.15.107:0/225771463 wait complete. 2024-04-25T00:15:30.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.986+0000 7f64692fd700 1 Processor -- start 2024-04-25T00:15:30.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.987+0000 7f64692fd700 1 -- start start 2024-04-25T00:15:30.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.987+0000 7f64692fd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6464069f40 0x7f64641a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:30.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.987+0000 7f64692fd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f646406a860 0x7f64641a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:30.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.987+0000 7f64692fd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f646410ece0 0x7f646419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:30.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.987+0000 7f64692fd700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6464114740 con 0x7f646410ece0 2024-04-25T00:15:30.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.987+0000 7f64692fd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f64641145c0 con 0x7f646406a860 2024-04-25T00:15:30.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.987+0000 7f64692fd700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f64641148c0 con 0x7f6464069f40 2024-04-25T00:15:30.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.987+0000 7f6462ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6464069f40 0x7f64641a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:30.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.987+0000 7f64637fe700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f646410ece0 0x7f646419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:30.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.987+0000 7f6462ffd700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6464069f40 0x7f64641a08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:52456/0 (socket says 172.21.15.107:52456) 2024-04-25T00:15:30.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.987+0000 7f6462ffd700 1 -- 172.21.15.107:0/2614428300 learned_addr learned my addr 172.21.15.107:0/2614428300 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:30.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.987+0000 7f64627fc700 1 --2- 172.21.15.107:0/2614428300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f646406a860 0x7f64641a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:30.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.988+0000 7f64637fe700 1 -- 172.21.15.107:0/2614428300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6464069f40 msgr2=0x7f64641a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:30.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.988+0000 7f64637fe700 1 --2- 172.21.15.107:0/2614428300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6464069f40 0x7f64641a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.988+0000 7f64637fe700 1 -- 172.21.15.107:0/2614428300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f646406a860 msgr2=0x7f64641a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:30.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.988+0000 7f64637fe700 1 --2- 172.21.15.107:0/2614428300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f646406a860 0x7f64641a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:30.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.988+0000 7f64637fe700 1 -- 172.21.15.107:0/2614428300 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f646419b1e0 con 0x7f646410ece0 2024-04-25T00:15:30.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.988+0000 7f64627fc700 1 --2- 172.21.15.107:0/2614428300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f646406a860 0x7f64641a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:30.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.988+0000 7f64637fe700 1 --2- 172.21.15.107:0/2614428300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f646410ece0 0x7f646419aa80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f645c007ee0 tx=0x7f645c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:30.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.988+0000 7f644bfff700 1 -- 172.21.15.107:0/2614428300 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c004db0 con 0x7f646410ece0 2024-04-25T00:15:30.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.988+0000 7f64692fd700 1 -- 172.21.15.107:0/2614428300 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f646419b4d0 con 0x7f646410ece0 2024-04-25T00:15:30.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.988+0000 7f64692fd700 1 -- 172.21.15.107:0/2614428300 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64641a7790 con 0x7f646410ece0 2024-04-25T00:15:30.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.988+0000 7f644bfff700 1 -- 172.21.15.107:0/2614428300 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f645c009e20 con 0x7f646410ece0 2024-04-25T00:15:30.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.989+0000 7f644bfff700 1 -- 172.21.15.107:0/2614428300 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c015400 con 0x7f646410ece0 2024-04-25T00:15:30.996 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:15:30.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.990+0000 7f64692fd700 1 -- 172.21.15.107:0/2614428300 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6450005330 con 0x7f646410ece0 2024-04-25T00:15:30.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.991+0000 7f644bfff700 1 -- 172.21.15.107:0/2614428300 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 137885+0+0 (secure 0 0 0) 0x7f645c00db20 con 0x7f646410ece0 2024-04-25T00:15:30.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.992+0000 7f644bfff700 1 --2- 172.21.15.107:0/2614428300 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f644c0a2cb0 0x7f644c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:30.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.992+0000 7f644bfff700 1 -- 172.21.15.107:0/2614428300 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f645c0ced70 con 0x7f646410ece0 2024-04-25T00:15:30.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.992+0000 7f6462ffd700 1 --2- 172.21.15.107:0/2614428300 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f644c0a2cb0 0x7f644c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:30.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:30.992+0000 7f6462ffd700 1 --2- 172.21.15.107:0/2614428300 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f644c0a2cb0 0x7f644c0a5170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6454009750 tx=0x7f645401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:31.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.001+0000 7f644bfff700 1 -- 172.21.15.107:0/2614428300 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f645c010070 con 0x7f646410ece0 2024-04-25T00:15:31.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.006+0000 7fe4aa821700 1 -- 172.21.15.107:0/3148300183 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe4a41a5880 con 0x7fe4a4101af0 2024-04-25T00:15:31.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.007+0000 7fe4a17fa700 1 -- 172.21.15.107:0/3148300183 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe498013070 con 0x7fe4a4101af0 2024-04-25T00:15:31.008 INFO:teuthology.orchestra.run.smithi107.stdout:111669149767 2024-04-25T00:15:31.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.009+0000 7fe4aa821700 1 -- 172.21.15.107:0/3148300183 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe48c0a2cb0 msgr2=0x7fe48c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:31.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.009+0000 7fe4aa821700 1 --2- 172.21.15.107:0/3148300183 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe48c0a2cb0 0x7fe48c0a5170 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4199ae0 tx=0x7fe494009190 comp rx=0 tx=0).stop 2024-04-25T00:15:31.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.009+0000 7fe4aa821700 1 -- 172.21.15.107:0/3148300183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4a4101af0 msgr2=0x7fe4a419ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:31.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.009+0000 7fe4aa821700 1 --2- 172.21.15.107:0/3148300183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4a4101af0 0x7fe4a419ed20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe498000c00 tx=0x7fe498003680 comp rx=0 tx=0).stop 2024-04-25T00:15:31.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.009+0000 7fe4aa821700 1 -- 172.21.15.107:0/3148300183 shutdown_connections 2024-04-25T00:15:31.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.009+0000 7fe4aa821700 1 --2- 172.21.15.107:0/3148300183 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe48c0a2cb0 0x7fe48c0a5170 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:31.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.009+0000 7fe4aa821700 1 --2- 172.21.15.107:0/3148300183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4a41071a0 0x7fe4a41989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:31.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.009+0000 7fe4aa821700 1 --2- 172.21.15.107:0/3148300183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4a4101130 0x7fe4a419e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:31.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.009+0000 7fe4aa821700 1 --2- 172.21.15.107:0/3148300183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4a4101af0 0x7fe4a419ed20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:31.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.009+0000 7fe4aa821700 1 -- 172.21.15.107:0/3148300183 >> 172.21.15.107:0/3148300183 conn(0x7fe4a40780e0 msgr2=0x7fe4a41004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:31.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.009+0000 7fe4aa821700 1 -- 172.21.15.107:0/3148300183 shutdown_connections 2024-04-25T00:15:31.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.009+0000 7fe4aa821700 1 -- 172.21.15.107:0/3148300183 wait complete. 2024-04-25T00:15:31.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.096+0000 7f644bfff700 1 -- 172.21.15.107:0/2614428300 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f645c0977a0 con 0x7f646410ece0 2024-04-25T00:15:31.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.218+0000 7f64692fd700 1 -- 172.21.15.107:0/2614428300 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f64500058f0 con 0x7f646410ece0 2024-04-25T00:15:31.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.218+0000 7f644bfff700 1 -- 172.21.15.107:0/2614428300 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f645c09b830 con 0x7f646410ece0 2024-04-25T00:15:31.220 INFO:teuthology.orchestra.run.smithi107.stdout:55834574929 2024-04-25T00:15:31.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.221+0000 7f64692fd700 1 -- 172.21.15.107:0/2614428300 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f644c0a2cb0 msgr2=0x7f644c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:31.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.221+0000 7f64692fd700 1 --2- 172.21.15.107:0/2614428300 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f644c0a2cb0 0x7f644c0a5170 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6454009750 tx=0x7f645401a040 comp rx=0 tx=0).stop 2024-04-25T00:15:31.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.221+0000 7f64692fd700 1 -- 172.21.15.107:0/2614428300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f646410ece0 msgr2=0x7f646419aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:31.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.221+0000 7f64692fd700 1 --2- 172.21.15.107:0/2614428300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f646410ece0 0x7f646419aa80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f645c007ee0 tx=0x7f645c00d3b0 comp rx=0 tx=0).stop 2024-04-25T00:15:31.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.221+0000 7f64692fd700 1 -- 172.21.15.107:0/2614428300 shutdown_connections 2024-04-25T00:15:31.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.221+0000 7f64692fd700 1 --2- 172.21.15.107:0/2614428300 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f644c0a2cb0 0x7f644c0a5170 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:31.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.221+0000 7f64692fd700 1 --2- 172.21.15.107:0/2614428300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f646410ece0 0x7f646419aa80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:31.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.221+0000 7f64692fd700 1 --2- 172.21.15.107:0/2614428300 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6464069f40 0x7f64641a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:31.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.221+0000 7f64692fd700 1 --2- 172.21.15.107:0/2614428300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f646406a860 0x7f64641a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:31.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.221+0000 7f64692fd700 1 -- 172.21.15.107:0/2614428300 >> 172.21.15.107:0/2614428300 conn(0x7f64640780e0 msgr2=0x7f6464110190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:31.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.221+0000 7f64692fd700 1 -- 172.21.15.107:0/2614428300 shutdown_connections 2024-04-25T00:15:31.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.221+0000 7f64692fd700 1 -- 172.21.15.107:0/2614428300 wait complete. 2024-04-25T00:15:31.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:31 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3148300183' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T00:15:31.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:31 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3148300183' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T00:15:31.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:31 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3148300183' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T00:15:31.491 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888125 got 146028888129 for osd.4 2024-04-25T00:15:31.491 DEBUG:teuthology.parallel:result is None 2024-04-25T00:15:31.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.933+0000 7f8fcca8e700 1 -- 172.21.15.107:0/1020874547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc81011c0 msgr2=0x7f8fc810f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:31.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.933+0000 7f8fcca8e700 1 --2- 172.21.15.107:0/1020874547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc81011c0 0x7f8fc810f100 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009990 tx=0x7f8fb8009ca0 comp rx=0 tx=0).stop 2024-04-25T00:15:31.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.933+0000 7f8fcca8e700 1 -- 172.21.15.107:0/1020874547 shutdown_connections 2024-04-25T00:15:31.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.933+0000 7f8fcca8e700 1 --2- 172.21.15.107:0/1020874547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc81011c0 0x7f8fc810f100 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:31.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.933+0000 7f8fcca8e700 1 --2- 172.21.15.107:0/1020874547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fc81082c0 0x7f8fc81086a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:31.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.933+0000 7f8fcca8e700 1 --2- 172.21.15.107:0/1020874547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fc8100820 0x7f8fc8100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:31.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.933+0000 7f8fcca8e700 1 -- 172.21.15.107:0/1020874547 >> 172.21.15.107:0/1020874547 conn(0x7f8fc80777a0 msgr2=0x7f8fc80feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:31.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.933+0000 7f8fcca8e700 1 -- 172.21.15.107:0/1020874547 shutdown_connections 2024-04-25T00:15:31.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.933+0000 7f8fcca8e700 1 -- 172.21.15.107:0/1020874547 wait complete. 2024-04-25T00:15:31.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.934+0000 7f8fcca8e700 1 Processor -- start 2024-04-25T00:15:31.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.934+0000 7f8fcca8e700 1 -- start start 2024-04-25T00:15:31.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.934+0000 7f8fcca8e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc8100820 0x7f8fc81a0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:31.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.935+0000 7f8fcca8e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fc81011c0 0x7f8fc81a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:31.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.935+0000 7f8fcca8e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fc81082c0 0x7f8fc819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:31.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.935+0000 7f8fcca8e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc8112510 con 0x7f8fc8100820 2024-04-25T00:15:31.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.935+0000 7f8fcca8e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc8112390 con 0x7f8fc81011c0 2024-04-25T00:15:31.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.935+0000 7f8fcca8e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc8112690 con 0x7f8fc81082c0 2024-04-25T00:15:31.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.935+0000 7f8fc659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc8100820 0x7f8fc81a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:31.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.935+0000 7f8fc659c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc8100820 0x7f8fc81a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56508/0 (socket says 172.21.15.107:56508) 2024-04-25T00:15:31.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.935+0000 7f8fc659c700 1 -- 172.21.15.107:0/1483585609 learned_addr learned my addr 172.21.15.107:0/1483585609 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:31.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.935+0000 7f8fc6d9d700 1 --2- 172.21.15.107:0/1483585609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fc81082c0 0x7f8fc819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:31.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.935+0000 7f8fc5d9b700 1 --2- 172.21.15.107:0/1483585609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fc81011c0 0x7f8fc81a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:31.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.935+0000 7f8fc659c700 1 -- 172.21.15.107:0/1483585609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fc81082c0 msgr2=0x7f8fc819a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:31.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.935+0000 7f8fc659c700 1 --2- 172.21.15.107:0/1483585609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fc81082c0 0x7f8fc819a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:31.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.935+0000 7f8fc659c700 1 -- 172.21.15.107:0/1483585609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fc81011c0 msgr2=0x7f8fc81a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:31.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.935+0000 7f8fc659c700 1 --2- 172.21.15.107:0/1483585609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fc81011c0 0x7f8fc81a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:31.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.935+0000 7f8fc659c700 1 -- 172.21.15.107:0/1483585609 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc819b070 con 0x7f8fc8100820 2024-04-25T00:15:31.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.935+0000 7f8fc659c700 1 --2- 172.21.15.107:0/1483585609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc8100820 0x7f8fc81a0750 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0007e20 tx=0x7f8fc000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:31.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.935+0000 7f8fbf7fe700 1 -- 172.21.15.107:0/1483585609 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0016020 con 0x7f8fc8100820 2024-04-25T00:15:31.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.935+0000 7f8fcca8e700 1 -- 172.21.15.107:0/1483585609 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc819b360 con 0x7f8fc8100820 2024-04-25T00:15:31.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.936+0000 7f8fbf7fe700 1 -- 172.21.15.107:0/1483585609 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc000ad40 con 0x7f8fc8100820 2024-04-25T00:15:31.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.936+0000 7f8fbf7fe700 1 -- 172.21.15.107:0/1483585609 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0011700 con 0x7f8fc8100820 2024-04-25T00:15:31.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.936+0000 7f8fcca8e700 1 -- 172.21.15.107:0/1483585609 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc81a7620 con 0x7f8fc8100820 2024-04-25T00:15:31.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.937+0000 7f8fcca8e700 1 -- 172.21.15.107:0/1483585609 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc8103b40 con 0x7f8fc8100820 2024-04-25T00:15:31.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.938+0000 7f8fbf7fe700 1 -- 172.21.15.107:0/1483585609 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8fc0019020 con 0x7f8fc8100820 2024-04-25T00:15:31.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.939+0000 7f8fbf7fe700 1 --2- 172.21.15.107:0/1483585609 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8fb40a2ca0 0x7f8fb40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:31.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.939+0000 7f8fc5d9b700 1 --2- 172.21.15.107:0/1483585609 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8fb40a2ca0 0x7f8fb40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:31.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.939+0000 7f8fbf7fe700 1 -- 172.21.15.107:0/1483585609 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f8fc00cf000 con 0x7f8fc8100820 2024-04-25T00:15:31.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.940+0000 7f8fc5d9b700 1 --2- 172.21.15.107:0/1483585609 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8fb40a2ca0 0x7f8fb40a5160 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0005ea0 tx=0x7f8fb0005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:31.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:31.948+0000 7f8fbf7fe700 1 -- 172.21.15.107:0/1483585609 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8fc009bc20 con 0x7f8fc8100820 2024-04-25T00:15:31.956 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037800 got 257698037805 for osd.8 2024-04-25T00:15:31.956 DEBUG:teuthology.parallel:result is None 2024-04-25T00:15:32.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:32.151+0000 7f8fcca8e700 1 -- 172.21.15.107:0/1483585609 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f8fc819be40 con 0x7f8fc8100820 2024-04-25T00:15:32.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:32.151+0000 7f8fbf7fe700 1 -- 172.21.15.107:0/1483585609 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8fc009b520 con 0x7f8fc8100820 2024-04-25T00:15:32.153 INFO:teuthology.orchestra.run.smithi107.stdout:34359738454 2024-04-25T00:15:32.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:32.154+0000 7f8fcca8e700 1 -- 172.21.15.107:0/1483585609 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8fb40a2ca0 msgr2=0x7f8fb40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:32.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:32.154+0000 7f8fcca8e700 1 --2- 172.21.15.107:0/1483585609 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8fb40a2ca0 0x7f8fb40a5160 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0005ea0 tx=0x7f8fb0005e30 comp rx=0 tx=0).stop 2024-04-25T00:15:32.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:32.154+0000 7f8fcca8e700 1 -- 172.21.15.107:0/1483585609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc8100820 msgr2=0x7f8fc81a0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:32.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:32.154+0000 7f8fcca8e700 1 --2- 172.21.15.107:0/1483585609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc8100820 0x7f8fc81a0750 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0007e20 tx=0x7f8fc000a340 comp rx=0 tx=0).stop 2024-04-25T00:15:32.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:32.154+0000 7f8fcca8e700 1 -- 172.21.15.107:0/1483585609 shutdown_connections 2024-04-25T00:15:32.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:32.154+0000 7f8fcca8e700 1 --2- 172.21.15.107:0/1483585609 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8fb40a2ca0 0x7f8fb40a5160 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:32.154+0000 7f8fcca8e700 1 --2- 172.21.15.107:0/1483585609 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fc81082c0 0x7f8fc819a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:32.154+0000 7f8fcca8e700 1 --2- 172.21.15.107:0/1483585609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc8100820 0x7f8fc81a0750 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:32.154+0000 7f8fcca8e700 1 --2- 172.21.15.107:0/1483585609 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fc81011c0 0x7f8fc81a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:32.154+0000 7f8fcca8e700 1 -- 172.21.15.107:0/1483585609 >> 172.21.15.107:0/1483585609 conn(0x7f8fc80777a0 msgr2=0x7f8fc810dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:32.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:32.154+0000 7f8fcca8e700 1 -- 172.21.15.107:0/1483585609 shutdown_connections 2024-04-25T00:15:32.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:32.154+0000 7f8fcca8e700 1 -- 172.21.15.107:0/1483585609 wait complete. 2024-04-25T00:15:32.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:32 smithi195 ceph-mon[32833]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:15:32.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:32 smithi195 ceph-mon[32833]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T00:15:32.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:32 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2614428300' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T00:15:32.367 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:32 smithi107 ceph-mon[29153]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:15:32.367 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:32 smithi107 ceph-mon[29153]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T00:15:32.367 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:32 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2614428300' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T00:15:32.421 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411398 got 77309411404 for osd.2 2024-04-25T00:15:32.421 DEBUG:teuthology.parallel:result is None 2024-04-25T00:15:32.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:32 smithi170 ceph-mon[32807]: pgmap v339: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:15:32.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:32 smithi170 ceph-mon[32807]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-25T00:15:32.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:32 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2614428300' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T00:15:33.060 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738450 got 34359738454 for osd.0 2024-04-25T00:15:33.061 DEBUG:teuthology.parallel:result is None 2024-04-25T00:15:33.378 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:33 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1483585609' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T00:15:33.379 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:33 smithi107 ceph-mon[29153]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T00:15:33.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:33 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1483585609' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T00:15:33.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:33 smithi170 ceph-mon[32807]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T00:15:33.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:33 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1483585609' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T00:15:33.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:33 smithi195 ceph-mon[32833]: pgmap v340: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T00:15:33.952 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149762 got 111669149767 for osd.3 2024-04-25T00:15:33.952 DEBUG:teuthology.parallel:result is None 2024-04-25T00:15:35.057 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574922 got 55834574929 for osd.1 2024-04-25T00:15:35.057 DEBUG:teuthology.parallel:result is None 2024-04-25T00:15:36.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:35 smithi195 ceph-mon[32833]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T00:15:36.423 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:35 smithi107 ceph-mon[29153]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T00:15:36.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:35 smithi170 ceph-mon[32807]: pgmap v341: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T00:15:36.933 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776164 got 292057776169 for osd.9 2024-04-25T00:15:36.934 DEBUG:teuthology.parallel:result is None 2024-04-25T00:15:37.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.444+0000 7f2784280700 1 -- 172.21.15.107:0/3299511557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f277c1082c0 msgr2=0x7f277c1086a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:37.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.444+0000 7f2784280700 1 --2- 172.21.15.107:0/3299511557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f277c1082c0 0x7f277c1086a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2778009990 tx=0x7f2778009ca0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.444+0000 7f2784280700 1 -- 172.21.15.107:0/3299511557 shutdown_connections 2024-04-25T00:15:37.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.444+0000 7f2784280700 1 --2- 172.21.15.107:0/3299511557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f277c1011c0 0x7f277c10f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.444+0000 7f2784280700 1 --2- 172.21.15.107:0/3299511557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f277c1082c0 0x7f277c1086a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.444+0000 7f2784280700 1 --2- 172.21.15.107:0/3299511557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f277c100820 0x7f277c100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.444+0000 7f2784280700 1 -- 172.21.15.107:0/3299511557 >> 172.21.15.107:0/3299511557 conn(0x7f277c0777a0 msgr2=0x7f277c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:37.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.444+0000 7f2784280700 1 -- 172.21.15.107:0/3299511557 shutdown_connections 2024-04-25T00:15:37.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.444+0000 7f2784280700 1 -- 172.21.15.107:0/3299511557 wait complete. 2024-04-25T00:15:37.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.445+0000 7f2784280700 1 Processor -- start 2024-04-25T00:15:37.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.445+0000 7f2784280700 1 -- start start 2024-04-25T00:15:37.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.445+0000 7f2784280700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f277c100820 0x7f277c19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:37.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.445+0000 7f2784280700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f277c1011c0 0x7f277c19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:37.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.445+0000 7f2784280700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f277c1082c0 0x7f277c198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:37.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.445+0000 7f2784280700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f277c112500 con 0x7f277c100820 2024-04-25T00:15:37.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.445+0000 7f2784280700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f277c112380 con 0x7f277c1082c0 2024-04-25T00:15:37.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.445+0000 7f2784280700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f277c112680 con 0x7f277c1011c0 2024-04-25T00:15:37.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.445+0000 7f278281d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f277c1082c0 0x7f277c198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:37.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.445+0000 7f278281d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f277c1082c0 0x7f277c198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.107:50382/0 (socket says 172.21.15.107:50382) 2024-04-25T00:15:37.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.445+0000 7f278281d700 1 -- 172.21.15.107:0/3440290492 learned_addr learned my addr 172.21.15.107:0/3440290492 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:37.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.445+0000 7f278181b700 1 --2- 172.21.15.107:0/3440290492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f277c1011c0 0x7f277c19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:37.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.445+0000 7f278201c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f277c100820 0x7f277c19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:37.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.446+0000 7f278201c700 1 -- 172.21.15.107:0/3440290492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f277c1011c0 msgr2=0x7f277c19eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:37.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.446+0000 7f278201c700 1 --2- 172.21.15.107:0/3440290492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f277c1011c0 0x7f277c19eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.446+0000 7f278201c700 1 -- 172.21.15.107:0/3440290492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f277c1082c0 msgr2=0x7f277c198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:37.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.446+0000 7f278201c700 1 --2- 172.21.15.107:0/3440290492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f277c1082c0 0x7f277c198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.446+0000 7f278201c700 1 -- 172.21.15.107:0/3440290492 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277c199000 con 0x7f277c100820 2024-04-25T00:15:37.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.446+0000 7f278281d700 1 --2- 172.21.15.107:0/3440290492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f277c1082c0 0x7f277c198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:37.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.446+0000 7f278201c700 1 --2- 172.21.15.107:0/3440290492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f277c100820 0x7f277c19e650 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f27780059d0 tx=0x7f27780047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:37.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.446+0000 7f27737fe700 1 -- 172.21.15.107:0/3440290492 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778027070 con 0x7f277c100820 2024-04-25T00:15:37.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.446+0000 7f2784280700 1 -- 172.21.15.107:0/3440290492 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f277c199290 con 0x7f277c100820 2024-04-25T00:15:37.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.446+0000 7f27737fe700 1 -- 172.21.15.107:0/3440290492 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2778003740 con 0x7f277c100820 2024-04-25T00:15:37.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.446+0000 7f27737fe700 1 -- 172.21.15.107:0/3440290492 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778022b50 con 0x7f277c100820 2024-04-25T00:15:37.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.446+0000 7f2784280700 1 -- 172.21.15.107:0/3440290492 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f277c1a54b0 con 0x7f277c100820 2024-04-25T00:15:37.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.447+0000 7f2784280700 1 -- 172.21.15.107:0/3440290492 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f277c103b40 con 0x7f277c100820 2024-04-25T00:15:37.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.448+0000 7f27737fe700 1 -- 172.21.15.107:0/3440290492 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f2778022d10 con 0x7f277c100820 2024-04-25T00:15:37.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.449+0000 7f27737fe700 1 --2- 172.21.15.107:0/3440290492 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f27680a2c20 0x7f27680a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:37.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.449+0000 7f27737fe700 1 -- 172.21.15.107:0/3440290492 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f27780d0a80 con 0x7f277c100820 2024-04-25T00:15:37.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.449+0000 7f278181b700 1 --2- 172.21.15.107:0/3440290492 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f27680a2c20 0x7f27680a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:37.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.450+0000 7f278181b700 1 --2- 172.21.15.107:0/3440290492 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f27680a2c20 0x7f27680a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f277c107a30 tx=0x7f276c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:37.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.453+0000 7f27737fe700 1 -- 172.21.15.107:0/3440290492 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2778031070 con 0x7f277c100820 2024-04-25T00:15:37.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.667+0000 7f2784280700 1 -- 172.21.15.107:0/3440290492 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f277c199d00 con 0x7f277c100820 2024-04-25T00:15:37.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.668+0000 7f27737fe700 1 -- 172.21.15.107:0/3440290492 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2778013070 con 0x7f277c100820 2024-04-25T00:15:37.668 INFO:teuthology.orchestra.run.smithi107.stdout:167503724606 2024-04-25T00:15:37.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.671+0000 7f2784280700 1 -- 172.21.15.107:0/3440290492 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f27680a2c20 msgr2=0x7f27680a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:37.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.671+0000 7f2784280700 1 --2- 172.21.15.107:0/3440290492 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f27680a2c20 0x7f27680a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f277c107a30 tx=0x7f276c00b410 comp rx=0 tx=0).stop 2024-04-25T00:15:37.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.671+0000 7f2784280700 1 -- 172.21.15.107:0/3440290492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f277c100820 msgr2=0x7f277c19e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:37.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.671+0000 7f2784280700 1 --2- 172.21.15.107:0/3440290492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f277c100820 0x7f277c19e650 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f27780059d0 tx=0x7f27780047b0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.671+0000 7f2784280700 1 -- 172.21.15.107:0/3440290492 shutdown_connections 2024-04-25T00:15:37.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.671+0000 7f2784280700 1 --2- 172.21.15.107:0/3440290492 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f27680a2c20 0x7f27680a50e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.671+0000 7f2784280700 1 --2- 172.21.15.107:0/3440290492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f277c1082c0 0x7f277c198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.671+0000 7f2784280700 1 --2- 172.21.15.107:0/3440290492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f277c100820 0x7f277c19e650 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.671+0000 7f2784280700 1 --2- 172.21.15.107:0/3440290492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f277c1011c0 0x7f277c19eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.671+0000 7f2784280700 1 -- 172.21.15.107:0/3440290492 >> 172.21.15.107:0/3440290492 conn(0x7f277c0777a0 msgr2=0x7f277c0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:37.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.671+0000 7f2784280700 1 -- 172.21.15.107:0/3440290492 shutdown_connections 2024-04-25T00:15:37.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:37.671+0000 7f2784280700 1 -- 172.21.15.107:0/3440290492 wait complete. 2024-04-25T00:15:38.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:38 smithi195 ceph-mon[32833]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T00:15:38.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:38 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3440290492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T00:15:38.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:38 smithi170 ceph-mon[32807]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T00:15:38.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:38 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3440290492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T00:15:38.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:38 smithi107 ceph-mon[29153]: pgmap v342: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T00:15:38.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:38 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3440290492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T00:15:38.641 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724601 got 167503724606 for osd.5 2024-04-25T00:15:38.641 DEBUG:teuthology.parallel:result is None 2024-04-25T00:15:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.884+0000 7fa60bf7f700 1 -- 172.21.15.107:0/4173718767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa604115380 msgr2=0x7fa604117770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.884+0000 7fa60bf7f700 1 --2- 172.21.15.107:0/4173718767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa604115380 0x7fa604117770 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4009990 tx=0x7fa5f4009ca0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.884+0000 7fa60bf7f700 1 -- 172.21.15.107:0/4173718767 shutdown_connections 2024-04-25T00:15:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.884+0000 7fa60bf7f700 1 --2- 172.21.15.107:0/4173718767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa604115380 0x7fa604117770 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.884+0000 7fa60bf7f700 1 --2- 172.21.15.107:0/4173718767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa604109230 0x7fa604109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.884+0000 7fa60bf7f700 1 --2- 172.21.15.107:0/4173718767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa604109b50 0x7fa604114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.884+0000 7fa60bf7f700 1 -- 172.21.15.107:0/4173718767 >> 172.21.15.107:0/4173718767 conn(0x7fa60406d3a0 msgr2=0x7fa60406d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.884+0000 7fa60bf7f700 1 -- 172.21.15.107:0/4173718767 shutdown_connections 2024-04-25T00:15:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.884+0000 7fa60bf7f700 1 -- 172.21.15.107:0/4173718767 wait complete. 2024-04-25T00:15:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.884+0000 7fa60bf7f700 1 Processor -- start 2024-04-25T00:15:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.884+0000 7fa60bf7f700 1 -- start start 2024-04-25T00:15:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.885+0000 7fa60bf7f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa604109230 0x7fa6041a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.885+0000 7fa60bf7f700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa604109b50 0x7fa6041a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.885+0000 7fa60bf7f700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa604115380 0x7fa6041a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.885+0000 7fa60bf7f700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa60411ad90 con 0x7fa604115380 2024-04-25T00:15:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.885+0000 7fa60bf7f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa60411ac10 con 0x7fa604109230 2024-04-25T00:15:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.885+0000 7fa60bf7f700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa60411af10 con 0x7fa604109b50 2024-04-25T00:15:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.885+0000 7fa60a51c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa604115380 0x7fa6041a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.885+0000 7fa60a51c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa604115380 0x7fa6041a5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56544/0 (socket says 172.21.15.107:56544) 2024-04-25T00:15:38.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.885+0000 7fa60a51c700 1 -- 172.21.15.107:0/3009171253 learned_addr learned my addr 172.21.15.107:0/3009171253 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:38.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.885+0000 7fa609d1b700 1 --2- 172.21.15.107:0/3009171253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa604109230 0x7fa6041a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.885+0000 7fa60951a700 1 --2- 172.21.15.107:0/3009171253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa604109b50 0x7fa6041a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.885+0000 7fa60a51c700 1 -- 172.21.15.107:0/3009171253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa604109b50 msgr2=0x7fa6041a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:38.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.885+0000 7fa60a51c700 1 --2- 172.21.15.107:0/3009171253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa604109b50 0x7fa6041a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.885+0000 7fa60a51c700 1 -- 172.21.15.107:0/3009171253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa604109230 msgr2=0x7fa6041a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:38.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.885+0000 7fa60a51c700 1 --2- 172.21.15.107:0/3009171253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa604109230 0x7fa6041a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.885+0000 7fa60a51c700 1 -- 172.21.15.107:0/3009171253 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6041a6170 con 0x7fa604115380 2024-04-25T00:15:38.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.885+0000 7fa60a51c700 1 --2- 172.21.15.107:0/3009171253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa604115380 0x7fa6041a5a30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40038b0 tx=0x7fa5f4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:38.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.885+0000 7fa5faffd700 1 -- 172.21.15.107:0/3009171253 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4027070 con 0x7fa604115380 2024-04-25T00:15:38.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.885+0000 7fa5faffd700 1 -- 172.21.15.107:0/3009171253 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5f4005690 con 0x7fa604115380 2024-04-25T00:15:38.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.885+0000 7fa60bf7f700 1 -- 172.21.15.107:0/3009171253 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6041a6400 con 0x7fa604115380 2024-04-25T00:15:38.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.885+0000 7fa5faffd700 1 -- 172.21.15.107:0/3009171253 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4022da0 con 0x7fa604115380 2024-04-25T00:15:38.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.885+0000 7fa60bf7f700 1 -- 172.21.15.107:0/3009171253 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6041adce0 con 0x7fa604115380 2024-04-25T00:15:38.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.887+0000 7fa60bf7f700 1 -- 172.21.15.107:0/3009171253 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa60406e930 con 0x7fa604115380 2024-04-25T00:15:38.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.887+0000 7fa5faffd700 1 -- 172.21.15.107:0/3009171253 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fa5f402a070 con 0x7fa604115380 2024-04-25T00:15:38.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.888+0000 7fa5faffd700 1 --2- 172.21.15.107:0/3009171253 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa5f00a2c20 0x7fa5f00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:38.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.888+0000 7fa609d1b700 1 --2- 172.21.15.107:0/3009171253 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa5f00a2c20 0x7fa5f00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:38.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.888+0000 7fa5faffd700 1 -- 172.21.15.107:0/3009171253 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fa5f40d0120 con 0x7fa604115380 2024-04-25T00:15:38.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.888+0000 7fa609d1b700 1 --2- 172.21.15.107:0/3009171253 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa5f00a2c20 0x7fa5f00a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00f440 tx=0x7fa5fc0079e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:38.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:38.893+0000 7fa5faffd700 1 -- 172.21.15.107:0/3009171253 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa5f405e740 con 0x7fa604115380 2024-04-25T00:15:39.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:39.114+0000 7fa60bf7f700 1 -- 172.21.15.107:0/3009171253 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fa6041adeb0 con 0x7fa604115380 2024-04-25T00:15:39.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:39.114+0000 7fa5faffd700 1 -- 172.21.15.107:0/3009171253 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fa5f4013070 con 0x7fa604115380 2024-04-25T00:15:39.115 INFO:teuthology.orchestra.run.smithi107.stdout:313532612645 2024-04-25T00:15:39.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:39.116+0000 7fa60bf7f700 1 -- 172.21.15.107:0/3009171253 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa5f00a2c20 msgr2=0x7fa5f00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:39.116+0000 7fa60bf7f700 1 --2- 172.21.15.107:0/3009171253 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa5f00a2c20 0x7fa5f00a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00f440 tx=0x7fa5fc0079e0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:39.116+0000 7fa60bf7f700 1 -- 172.21.15.107:0/3009171253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa604115380 msgr2=0x7fa6041a5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:39.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:39.116+0000 7fa60bf7f700 1 --2- 172.21.15.107:0/3009171253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa604115380 0x7fa6041a5a30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa5f40038b0 tx=0x7fa5f4005e80 comp rx=0 tx=0).stop 2024-04-25T00:15:39.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:39.116+0000 7fa60bf7f700 1 -- 172.21.15.107:0/3009171253 shutdown_connections 2024-04-25T00:15:39.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:39.116+0000 7fa60bf7f700 1 --2- 172.21.15.107:0/3009171253 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fa5f00a2c20 0x7fa5f00a50e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:39.116+0000 7fa60bf7f700 1 --2- 172.21.15.107:0/3009171253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa604115380 0x7fa6041a5a30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:39.116+0000 7fa60bf7f700 1 --2- 172.21.15.107:0/3009171253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa604109230 0x7fa6041a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:39.116+0000 7fa60bf7f700 1 --2- 172.21.15.107:0/3009171253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa604109b50 0x7fa6041a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:39.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:39.116+0000 7fa60bf7f700 1 -- 172.21.15.107:0/3009171253 >> 172.21.15.107:0/3009171253 conn(0x7fa60406d3a0 msgr2=0x7fa604070f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:39.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:39.116+0000 7fa60bf7f700 1 -- 172.21.15.107:0/3009171253 shutdown_connections 2024-04-25T00:15:39.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:39.116+0000 7fa60bf7f700 1 -- 172.21.15.107:0/3009171253 wait complete. 2024-04-25T00:15:39.821 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612639 got 313532612645 for osd.10 2024-04-25T00:15:39.821 DEBUG:teuthology.parallel:result is None 2024-04-25T00:15:39.821 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T00:15:39.822 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T00:15:40.075 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:15:40.196 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:40 smithi107 ceph-mon[29153]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T00:15:40.197 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:40 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3009171253' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T00:15:40.197 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:40 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:15:40.197 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:40 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:15:40.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:40 smithi195 ceph-mon[32833]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T00:15:40.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:40 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3009171253' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T00:15:40.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:40 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:15:40.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:40 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:15:40.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:40 smithi170 ceph-mon[32807]: pgmap v343: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T00:15:40.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:40 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3009171253' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T00:15:40.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:40 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:15:40.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:40 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:15:41.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:41 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:41.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:41 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:41.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:41 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:15:41.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:41 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:41.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:41 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:41.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:41 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:15:41.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:41 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:41.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:41 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:41.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:41 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' 2024-04-25T00:15:41.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.973+0000 7f473dd88700 1 -- 172.21.15.107:0/4270096167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4738101dd0 msgr2=0x7f4738105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:41.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.973+0000 7f473dd88700 1 --2- 172.21.15.107:0/4270096167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4738101dd0 0x7f4738105d90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f472c0099c0 tx=0x7f472c009cd0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.973+0000 7f473dd88700 1 -- 172.21.15.107:0/4270096167 shutdown_connections 2024-04-25T00:15:41.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.973+0000 7f473dd88700 1 --2- 172.21.15.107:0/4270096167 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4738106470 0x7f473810e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.973+0000 7f473dd88700 1 --2- 172.21.15.107:0/4270096167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4738101420 0x7f4738101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.973+0000 7f473dd88700 1 --2- 172.21.15.107:0/4270096167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4738101dd0 0x7f4738105d90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.973+0000 7f473dd88700 1 -- 172.21.15.107:0/4270096167 >> 172.21.15.107:0/4270096167 conn(0x7f4738077180 msgr2=0x7f4738077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:41.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.974+0000 7f473dd88700 1 -- 172.21.15.107:0/4270096167 shutdown_connections 2024-04-25T00:15:41.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.974+0000 7f473dd88700 1 -- 172.21.15.107:0/4270096167 wait complete. 2024-04-25T00:15:41.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.974+0000 7f473dd88700 1 Processor -- start 2024-04-25T00:15:41.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.975+0000 7f473dd88700 1 -- start start 2024-04-25T00:15:41.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.975+0000 7f473dd88700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4738101420 0x7f47381997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.975+0000 7f473dd88700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4738101dd0 0x7f4738199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.975+0000 7f473dd88700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4738106470 0x7f47381a4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.975+0000 7f473dd88700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4738111430 con 0x7f4738101dd0 2024-04-25T00:15:41.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.975+0000 7f473dd88700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f47381112b0 con 0x7f4738101420 2024-04-25T00:15:41.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.975+0000 7f473dd88700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f47381115b0 con 0x7f4738106470 2024-04-25T00:15:41.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.975+0000 7f4736ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4738101dd0 0x7f4738199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.975+0000 7f4736ffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4738101dd0 0x7f4738199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38160/0 (socket says 172.21.15.107:38160) 2024-04-25T00:15:41.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.975+0000 7f4736ffd700 1 -- 172.21.15.107:0/2675364860 learned_addr learned my addr 172.21.15.107:0/2675364860 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:41.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.975+0000 7f47377fe700 1 --2- 172.21.15.107:0/2675364860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4738101420 0x7f47381997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.975+0000 7f4737fff700 1 --2- 172.21.15.107:0/2675364860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4738106470 0x7f47381a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.975+0000 7f4736ffd700 1 -- 172.21.15.107:0/2675364860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4738106470 msgr2=0x7f47381a4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:41.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.975+0000 7f4736ffd700 1 --2- 172.21.15.107:0/2675364860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4738106470 0x7f47381a4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.975+0000 7f4736ffd700 1 -- 172.21.15.107:0/2675364860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4738101420 msgr2=0x7f47381997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:41.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.975+0000 7f4736ffd700 1 --2- 172.21.15.107:0/2675364860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4738101420 0x7f47381997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:41.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.975+0000 7f4736ffd700 1 -- 172.21.15.107:0/2675364860 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47381a4db0 con 0x7f4738101dd0 2024-04-25T00:15:41.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.976+0000 7f4736ffd700 1 --2- 172.21.15.107:0/2675364860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4738101dd0 0x7f4738199d30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f472c00c010 tx=0x7f472c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:41.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.976+0000 7f4734ff9700 1 -- 172.21.15.107:0/2675364860 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c027070 con 0x7f4738101dd0 2024-04-25T00:15:41.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.976+0000 7f4734ff9700 1 -- 172.21.15.107:0/2675364860 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f472c003730 con 0x7f4738101dd0 2024-04-25T00:15:41.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.976+0000 7f473dd88700 1 -- 172.21.15.107:0/2675364860 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47381a4f80 con 0x7f4738101dd0 2024-04-25T00:15:41.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.976+0000 7f4734ff9700 1 -- 172.21.15.107:0/2675364860 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c022d40 con 0x7f4738101dd0 2024-04-25T00:15:41.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.976+0000 7f473dd88700 1 -- 172.21.15.107:0/2675364860 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47381a5320 con 0x7f4738101dd0 2024-04-25T00:15:41.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.977+0000 7f473dd88700 1 -- 172.21.15.107:0/2675364860 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4738068f20 con 0x7f4738101dd0 2024-04-25T00:15:41.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.978+0000 7f4734ff9700 1 -- 172.21.15.107:0/2675364860 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f472c02a070 con 0x7f4738101dd0 2024-04-25T00:15:41.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.978+0000 7f4734ff9700 1 --2- 172.21.15.107:0/2675364860 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f47200a2c20 0x7f47200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:41.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.978+0000 7f47377fe700 1 --2- 172.21.15.107:0/2675364860 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f47200a2c20 0x7f47200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:41.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.978+0000 7f4734ff9700 1 -- 172.21.15.107:0/2675364860 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f472c0d0160 con 0x7f4738101dd0 2024-04-25T00:15:41.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.979+0000 7f47377fe700 1 --2- 172.21.15.107:0/2675364860 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f47200a2c20 0x7f47200a50e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f472800ac50 tx=0x7f472800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:41.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:41.983+0000 7f4734ff9700 1 -- 172.21.15.107:0/2675364860 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f472c05e740 con 0x7f4738101dd0 2024-04-25T00:15:42.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:42.175+0000 7f473dd88700 1 -- 172.21.15.107:0/2675364860 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4738062670 con 0x7f47200a2c20 2024-04-25T00:15:42.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:42.185+0000 7f4734ff9700 1 -- 172.21.15.107:0/2675364860 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340459 (secure 0 0 0) 0x7f4738062670 con 0x7f47200a2c20 2024-04-25T00:15:42.190 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:15:42.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:42.189+0000 7f473dd88700 1 -- 172.21.15.107:0/2675364860 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f47200a2c20 msgr2=0x7f47200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:42.189+0000 7f473dd88700 1 --2- 172.21.15.107:0/2675364860 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f47200a2c20 0x7f47200a50e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f472800ac50 tx=0x7f472800a380 comp rx=0 tx=0).stop 2024-04-25T00:15:42.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:42.189+0000 7f473dd88700 1 -- 172.21.15.107:0/2675364860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4738101dd0 msgr2=0x7f4738199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:42.189+0000 7f473dd88700 1 --2- 172.21.15.107:0/2675364860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4738101dd0 0x7f4738199d30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f472c00c010 tx=0x7f472c0047b0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:42.189+0000 7f473dd88700 1 -- 172.21.15.107:0/2675364860 shutdown_connections 2024-04-25T00:15:42.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:42.190+0000 7f473dd88700 1 --2- 172.21.15.107:0/2675364860 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f47200a2c20 0x7f47200a50e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:42.190+0000 7f473dd88700 1 --2- 172.21.15.107:0/2675364860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4738106470 0x7f47381a4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:42.190+0000 7f473dd88700 1 --2- 172.21.15.107:0/2675364860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4738101420 0x7f47381997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:42.190+0000 7f473dd88700 1 --2- 172.21.15.107:0/2675364860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4738101dd0 0x7f4738199d30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:42.190+0000 7f473dd88700 1 -- 172.21.15.107:0/2675364860 >> 172.21.15.107:0/2675364860 conn(0x7f4738077180 msgr2=0x7f47380ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:42.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:42.190+0000 7f473dd88700 1 -- 172.21.15.107:0/2675364860 shutdown_connections 2024-04-25T00:15:42.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:42.190+0000 7f473dd88700 1 -- 172.21.15.107:0/2675364860 wait complete. 2024-04-25T00:15:42.195 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-04-25T00:15:42.336 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:42 smithi170 ceph-mon[32807]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T00:15:42.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:42 smithi195 ceph-mon[32833]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T00:15:42.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:42 smithi107 ceph-mon[29153]: pgmap v344: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T00:15:42.824 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-04-25T00:15:40.946009+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658349,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":593,"num_read_kb":421,"num_write":351,"num_write_kb":3061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":785,"ondisk_log_size":785,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388640,"kb_used_data":14528,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553184,"statfs":{"total":1151940427776,"available":1151542460416,"internally_reserved":0,"allocated":14876672,"data_stored":8207645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":47,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004263"},"pg_stats":[{"pgid":"3.3f","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.955552+0000","last_change":"2024-04-25T00:14:01.835737+0000","last_active":"2024-04-25T00:15:30.955552+0000","last_peered":"2024-04-25T00:15:30.955552+0000","last_clean":"2024-04-25T00:15:30.955552+0000","last_became_active":"2024-04-25T00:14:01.835428+0000","last_became_peered":"2024-04-25T00:14:01.835428+0000","last_unstale":"2024-04-25T00:15:30.955552+0000","last_undegraded":"2024-04-25T00:15:30.955552+0000","last_fullsized":"2024-04-25T00:15:30.955552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:33.974656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.958231+0000","last_change":"2024-04-25T00:14:01.837321+0000","last_active":"2024-04-25T00:15:30.958231+0000","last_peered":"2024-04-25T00:15:30.958231+0000","last_clean":"2024-04-25T00:15:30.958231+0000","last_became_active":"2024-04-25T00:14:01.836762+0000","last_became_peered":"2024-04-25T00:14:01.836762+0000","last_unstale":"2024-04-25T00:15:30.958231+0000","last_undegraded":"2024-04-25T00:15:30.958231+0000","last_fullsized":"2024-04-25T00:15:30.958231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:34:17.342967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.960471+0000","last_change":"2024-04-25T00:14:01.834504+0000","last_active":"2024-04-25T00:15:30.960471+0000","last_peered":"2024-04-25T00:15:30.960471+0000","last_clean":"2024-04-25T00:15:30.960471+0000","last_became_active":"2024-04-25T00:14:01.834182+0000","last_became_peered":"2024-04-25T00:14:01.834182+0000","last_unstale":"2024-04-25T00:15:30.960471+0000","last_undegraded":"2024-04-25T00:15:30.960471+0000","last_fullsized":"2024-04-25T00:15:30.960471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:12.991170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.949289+0000","last_change":"2024-04-25T00:15:02.080211+0000","last_active":"2024-04-25T00:15:30.949289+0000","last_peered":"2024-04-25T00:15:30.949289+0000","last_clean":"2024-04-25T00:15:30.949289+0000","last_became_active":"2024-04-25T00:15:02.079849+0000","last_became_peered":"2024-04-25T00:15:02.079849+0000","last_unstale":"2024-04-25T00:15:30.949289+0000","last_undegraded":"2024-04-25T00:15:30.949289+0000","last_fullsized":"2024-04-25T00:15:30.949289+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:56.804782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.965280+0000","last_change":"2024-04-25T00:14:01.834173+0000","last_active":"2024-04-25T00:15:30.965280+0000","last_peered":"2024-04-25T00:15:30.965280+0000","last_clean":"2024-04-25T00:15:30.965280+0000","last_became_active":"2024-04-25T00:14:01.833835+0000","last_became_peered":"2024-04-25T00:14:01.833835+0000","last_unstale":"2024-04-25T00:15:30.965280+0000","last_undegraded":"2024-04-25T00:15:30.965280+0000","last_fullsized":"2024-04-25T00:15:30.965280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:30.046629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.950213+0000","last_change":"2024-04-25T00:14:01.832113+0000","last_active":"2024-04-25T00:15:30.950213+0000","last_peered":"2024-04-25T00:15:30.950213+0000","last_clean":"2024-04-25T00:15:30.950213+0000","last_became_active":"2024-04-25T00:14:01.831800+0000","last_became_peered":"2024-04-25T00:14:01.831800+0000","last_unstale":"2024-04-25T00:15:30.950213+0000","last_undegraded":"2024-04-25T00:15:30.950213+0000","last_fullsized":"2024-04-25T00:15:30.950213+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:54.469994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.954336+0000","last_change":"2024-04-25T00:14:01.839166+0000","last_active":"2024-04-25T00:15:30.954336+0000","last_peered":"2024-04-25T00:15:30.954336+0000","last_clean":"2024-04-25T00:15:30.954336+0000","last_became_active":"2024-04-25T00:14:01.838851+0000","last_became_peered":"2024-04-25T00:14:01.838851+0000","last_unstale":"2024-04-25T00:15:30.954336+0000","last_undegraded":"2024-04-25T00:15:30.954336+0000","last_fullsized":"2024-04-25T00:15:30.954336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:39:48.125363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.954451+0000","last_change":"2024-04-25T00:15:02.083386+0000","last_active":"2024-04-25T00:15:30.954451+0000","last_peered":"2024-04-25T00:15:30.954451+0000","last_clean":"2024-04-25T00:15:30.954451+0000","last_became_active":"2024-04-25T00:15:02.083106+0000","last_became_peered":"2024-04-25T00:15:02.083106+0000","last_unstale":"2024-04-25T00:15:30.954451+0000","last_undegraded":"2024-04-25T00:15:30.954451+0000","last_fullsized":"2024-04-25T00:15:30.954451+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:58.543667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.959137+0000","last_change":"2024-04-25T00:14:01.829803+0000","last_active":"2024-04-25T00:15:30.959137+0000","last_peered":"2024-04-25T00:15:30.959137+0000","last_clean":"2024-04-25T00:15:30.959137+0000","last_became_active":"2024-04-25T00:14:01.827761+0000","last_became_peered":"2024-04-25T00:14:01.827761+0000","last_unstale":"2024-04-25T00:15:30.959137+0000","last_undegraded":"2024-04-25T00:15:30.959137+0000","last_fullsized":"2024-04-25T00:15:30.959137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:38.862505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.948779+0000","last_change":"2024-04-25T00:14:01.834269+0000","last_active":"2024-04-25T00:15:30.948779+0000","last_peered":"2024-04-25T00:15:30.948779+0000","last_clean":"2024-04-25T00:15:30.948779+0000","last_became_active":"2024-04-25T00:14:01.833964+0000","last_became_peered":"2024-04-25T00:14:01.833964+0000","last_unstale":"2024-04-25T00:15:30.948779+0000","last_undegraded":"2024-04-25T00:15:30.948779+0000","last_fullsized":"2024-04-25T00:15:30.948779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:29.501513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.959542+0000","last_change":"2024-04-25T00:14:01.839510+0000","last_active":"2024-04-25T00:15:30.959542+0000","last_peered":"2024-04-25T00:15:30.959542+0000","last_clean":"2024-04-25T00:15:30.959542+0000","last_became_active":"2024-04-25T00:14:01.839000+0000","last_became_peered":"2024-04-25T00:14:01.839000+0000","last_unstale":"2024-04-25T00:15:30.959542+0000","last_undegraded":"2024-04-25T00:15:30.959542+0000","last_fullsized":"2024-04-25T00:15:30.959542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:20.809277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.933166+0000","last_change":"2024-04-25T00:14:01.835821+0000","last_active":"2024-04-25T00:15:30.933166+0000","last_peered":"2024-04-25T00:15:30.933166+0000","last_clean":"2024-04-25T00:15:30.933166+0000","last_became_active":"2024-04-25T00:14:01.835525+0000","last_became_peered":"2024-04-25T00:14:01.835525+0000","last_unstale":"2024-04-25T00:15:30.933166+0000","last_undegraded":"2024-04-25T00:15:30.933166+0000","last_fullsized":"2024-04-25T00:15:30.933166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:17.082562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.960479+0000","last_change":"2024-04-25T00:14:01.831564+0000","last_active":"2024-04-25T00:15:30.960479+0000","last_peered":"2024-04-25T00:15:30.960479+0000","last_clean":"2024-04-25T00:15:30.960479+0000","last_became_active":"2024-04-25T00:14:01.828539+0000","last_became_peered":"2024-04-25T00:14:01.828539+0000","last_unstale":"2024-04-25T00:15:30.960479+0000","last_undegraded":"2024-04-25T00:15:30.960479+0000","last_fullsized":"2024-04-25T00:15:30.960479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:41:52.098299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.960792+0000","last_change":"2024-04-25T00:14:01.835815+0000","last_active":"2024-04-25T00:15:30.960792+0000","last_peered":"2024-04-25T00:15:30.960792+0000","last_clean":"2024-04-25T00:15:30.960792+0000","last_became_active":"2024-04-25T00:14:01.835498+0000","last_became_peered":"2024-04-25T00:14:01.835498+0000","last_unstale":"2024-04-25T00:15:30.960792+0000","last_undegraded":"2024-04-25T00:15:30.960792+0000","last_fullsized":"2024-04-25T00:15:30.960792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:14:02.914149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.962247+0000","last_change":"2024-04-25T00:14:01.839542+0000","last_active":"2024-04-25T00:15:30.962247+0000","last_peered":"2024-04-25T00:15:30.962247+0000","last_clean":"2024-04-25T00:15:30.962247+0000","last_became_active":"2024-04-25T00:14:01.839234+0000","last_became_peered":"2024-04-25T00:14:01.839234+0000","last_unstale":"2024-04-25T00:15:30.962247+0000","last_undegraded":"2024-04-25T00:15:30.962247+0000","last_fullsized":"2024-04-25T00:15:30.962247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:00.437311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.958957+0000","last_change":"2024-04-25T00:14:01.832265+0000","last_active":"2024-04-25T00:15:30.958957+0000","last_peered":"2024-04-25T00:15:30.958957+0000","last_clean":"2024-04-25T00:15:30.958957+0000","last_became_active":"2024-04-25T00:14:01.831963+0000","last_became_peered":"2024-04-25T00:14:01.831963+0000","last_unstale":"2024-04-25T00:15:30.958957+0000","last_undegraded":"2024-04-25T00:15:30.958957+0000","last_fullsized":"2024-04-25T00:15:30.958957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:55.402265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.955690+0000","last_change":"2024-04-25T00:14:01.835444+0000","last_active":"2024-04-25T00:15:30.955690+0000","last_peered":"2024-04-25T00:15:30.955690+0000","last_clean":"2024-04-25T00:15:30.955690+0000","last_became_active":"2024-04-25T00:14:01.835126+0000","last_became_peered":"2024-04-25T00:14:01.835126+0000","last_unstale":"2024-04-25T00:15:30.955690+0000","last_undegraded":"2024-04-25T00:15:30.955690+0000","last_fullsized":"2024-04-25T00:15:30.955690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:48.997924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.955107+0000","last_change":"2024-04-25T00:14:01.834982+0000","last_active":"2024-04-25T00:15:30.955107+0000","last_peered":"2024-04-25T00:15:30.955107+0000","last_clean":"2024-04-25T00:15:30.955107+0000","last_became_active":"2024-04-25T00:14:01.834671+0000","last_became_peered":"2024-04-25T00:14:01.834671+0000","last_unstale":"2024-04-25T00:15:30.955107+0000","last_undegraded":"2024-04-25T00:15:30.955107+0000","last_fullsized":"2024-04-25T00:15:30.955107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:28.254408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.946674+0000","last_change":"2024-04-25T00:14:01.824432+0000","last_active":"2024-04-25T00:15:30.946674+0000","last_peered":"2024-04-25T00:15:30.946674+0000","last_clean":"2024-04-25T00:15:30.946674+0000","last_became_active":"2024-04-25T00:14:01.823963+0000","last_became_peered":"2024-04-25T00:14:01.823963+0000","last_unstale":"2024-04-25T00:15:30.946674+0000","last_undegraded":"2024-04-25T00:15:30.946674+0000","last_fullsized":"2024-04-25T00:15:30.946674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:35.880932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.073688+0000","last_change":"2024-04-25T00:14:06.861892+0000","last_active":"2024-04-25T00:15:02.073688+0000","last_peered":"2024-04-25T00:15:02.073688+0000","last_clean":"2024-04-25T00:15:02.073688+0000","last_became_active":"2024-04-25T00:14:06.861470+0000","last_became_peered":"2024-04-25T00:14:06.861470+0000","last_unstale":"2024-04-25T00:15:02.073688+0000","last_undegraded":"2024-04-25T00:15:02.073688+0000","last_fullsized":"2024-04-25T00:15:02.073688+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:06.192954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076691+0000","last_change":"2024-04-25T00:14:01.412535+0000","last_active":"2024-04-25T00:15:02.076691+0000","last_peered":"2024-04-25T00:15:02.076691+0000","last_clean":"2024-04-25T00:15:02.076691+0000","last_became_active":"2024-04-25T00:14:01.412206+0000","last_became_peered":"2024-04-25T00:14:01.412206+0000","last_unstale":"2024-04-25T00:15:02.076691+0000","last_undegraded":"2024-04-25T00:15:02.076691+0000","last_fullsized":"2024-04-25T00:15:02.076691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:40.893987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.963511+0000","last_change":"2024-04-25T00:14:01.840814+0000","last_active":"2024-04-25T00:15:30.963511+0000","last_peered":"2024-04-25T00:15:30.963511+0000","last_clean":"2024-04-25T00:15:30.963511+0000","last_became_active":"2024-04-25T00:14:01.840179+0000","last_became_peered":"2024-04-25T00:14:01.840179+0000","last_unstale":"2024-04-25T00:15:30.963511+0000","last_undegraded":"2024-04-25T00:15:30.963511+0000","last_fullsized":"2024-04-25T00:15:30.963511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:27.383918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.075532+0000","last_change":"2024-04-25T00:14:06.857877+0000","last_active":"2024-04-25T00:15:02.075532+0000","last_peered":"2024-04-25T00:15:02.075532+0000","last_clean":"2024-04-25T00:15:02.075532+0000","last_became_active":"2024-04-25T00:14:06.857420+0000","last_became_peered":"2024-04-25T00:14:06.857420+0000","last_unstale":"2024-04-25T00:15:02.075532+0000","last_undegraded":"2024-04-25T00:15:02.075532+0000","last_fullsized":"2024-04-25T00:15:02.075532+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:54.424024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.074359+0000","last_change":"2024-04-25T00:14:01.416560+0000","last_active":"2024-04-25T00:15:02.074359+0000","last_peered":"2024-04-25T00:15:02.074359+0000","last_clean":"2024-04-25T00:15:02.074359+0000","last_became_active":"2024-04-25T00:14:01.416246+0000","last_became_peered":"2024-04-25T00:14:01.416246+0000","last_unstale":"2024-04-25T00:15:02.074359+0000","last_undegraded":"2024-04-25T00:15:02.074359+0000","last_fullsized":"2024-04-25T00:15:02.074359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:03:47.938452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.960439+0000","last_change":"2024-04-25T00:14:01.832607+0000","last_active":"2024-04-25T00:15:30.960439+0000","last_peered":"2024-04-25T00:15:30.960439+0000","last_clean":"2024-04-25T00:15:30.960439+0000","last_became_active":"2024-04-25T00:14:01.832300+0000","last_became_peered":"2024-04-25T00:14:01.832300+0000","last_unstale":"2024-04-25T00:15:30.960439+0000","last_undegraded":"2024-04-25T00:15:30.960439+0000","last_fullsized":"2024-04-25T00:15:30.960439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:22.045171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.075719+0000","last_change":"2024-04-25T00:14:06.863642+0000","last_active":"2024-04-25T00:15:02.075719+0000","last_peered":"2024-04-25T00:15:02.075719+0000","last_clean":"2024-04-25T00:15:02.075719+0000","last_became_active":"2024-04-25T00:14:06.863284+0000","last_became_peered":"2024-04-25T00:14:06.863284+0000","last_unstale":"2024-04-25T00:15:02.075719+0000","last_undegraded":"2024-04-25T00:15:02.075719+0000","last_fullsized":"2024-04-25T00:15:02.075719+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:20.818625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076385+0000","last_change":"2024-04-25T00:14:01.411874+0000","last_active":"2024-04-25T00:15:02.076385+0000","last_peered":"2024-04-25T00:15:02.076385+0000","last_clean":"2024-04-25T00:15:02.076385+0000","last_became_active":"2024-04-25T00:14:01.409568+0000","last_became_peered":"2024-04-25T00:14:01.409568+0000","last_unstale":"2024-04-25T00:15:02.076385+0000","last_undegraded":"2024-04-25T00:15:02.076385+0000","last_fullsized":"2024-04-25T00:15:02.076385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:22.415631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.952225+0000","last_change":"2024-04-25T00:14:01.822471+0000","last_active":"2024-04-25T00:15:30.952225+0000","last_peered":"2024-04-25T00:15:30.952225+0000","last_clean":"2024-04-25T00:15:30.952225+0000","last_became_active":"2024-04-25T00:14:01.822181+0000","last_became_peered":"2024-04-25T00:14:01.822181+0000","last_unstale":"2024-04-25T00:15:30.952225+0000","last_undegraded":"2024-04-25T00:15:30.952225+0000","last_fullsized":"2024-04-25T00:15:30.952225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:44.857036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.075572+0000","last_change":"2024-04-25T00:14:06.856987+0000","last_active":"2024-04-25T00:15:02.075572+0000","last_peered":"2024-04-25T00:15:02.075572+0000","last_clean":"2024-04-25T00:15:02.075572+0000","last_became_active":"2024-04-25T00:14:06.856053+0000","last_became_peered":"2024-04-25T00:14:06.856053+0000","last_unstale":"2024-04-25T00:15:02.075572+0000","last_undegraded":"2024-04-25T00:15:02.075572+0000","last_fullsized":"2024-04-25T00:15:02.075572+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:16.829969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.077677+0000","last_change":"2024-04-25T00:14:01.410156+0000","last_active":"2024-04-25T00:15:02.077677+0000","last_peered":"2024-04-25T00:15:02.077677+0000","last_clean":"2024-04-25T00:15:02.077677+0000","last_became_active":"2024-04-25T00:14:01.409809+0000","last_became_peered":"2024-04-25T00:14:01.409809+0000","last_unstale":"2024-04-25T00:15:02.077677+0000","last_undegraded":"2024-04-25T00:15:02.077677+0000","last_fullsized":"2024-04-25T00:15:02.077677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:43.160419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.955674+0000","last_change":"2024-04-25T00:14:01.836090+0000","last_active":"2024-04-25T00:15:30.955674+0000","last_peered":"2024-04-25T00:15:30.955674+0000","last_clean":"2024-04-25T00:15:30.955674+0000","last_became_active":"2024-04-25T00:14:01.835733+0000","last_became_peered":"2024-04-25T00:14:01.835733+0000","last_unstale":"2024-04-25T00:15:30.955674+0000","last_undegraded":"2024-04-25T00:15:30.955674+0000","last_fullsized":"2024-04-25T00:15:30.955674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:02.368893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.074149+0000","last_change":"2024-04-25T00:14:06.851496+0000","last_active":"2024-04-25T00:15:02.074149+0000","last_peered":"2024-04-25T00:15:02.074149+0000","last_clean":"2024-04-25T00:15:02.074149+0000","last_became_active":"2024-04-25T00:14:06.851007+0000","last_became_peered":"2024-04-25T00:14:06.851007+0000","last_unstale":"2024-04-25T00:15:02.074149+0000","last_undegraded":"2024-04-25T00:15:02.074149+0000","last_fullsized":"2024-04-25T00:15:02.074149+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:50.543348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'7","reported_seq":112,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:36.644130+0000","last_change":"2024-04-25T00:14:01.414760+0000","last_active":"2024-04-25T00:15:36.644130+0000","last_peered":"2024-04-25T00:15:36.644130+0000","last_clean":"2024-04-25T00:15:36.644130+0000","last_became_active":"2024-04-25T00:14:01.414482+0000","last_became_peered":"2024-04-25T00:14:01.414482+0000","last_unstale":"2024-04-25T00:15:36.644130+0000","last_undegraded":"2024-04-25T00:15:36.644130+0000","last_fullsized":"2024-04-25T00:15:36.644130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:32.631933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.930160+0000","last_change":"2024-04-25T00:14:01.832516+0000","last_active":"2024-04-25T00:15:30.930160+0000","last_peered":"2024-04-25T00:15:30.930160+0000","last_clean":"2024-04-25T00:15:30.930160+0000","last_became_active":"2024-04-25T00:14:01.832106+0000","last_became_peered":"2024-04-25T00:14:01.832106+0000","last_unstale":"2024-04-25T00:15:30.930160+0000","last_undegraded":"2024-04-25T00:15:30.930160+0000","last_fullsized":"2024-04-25T00:15:30.930160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:55.590486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.073810+0000","last_change":"2024-04-25T00:14:06.861984+0000","last_active":"2024-04-25T00:15:02.073810+0000","last_peered":"2024-04-25T00:15:02.073810+0000","last_clean":"2024-04-25T00:15:02.073810+0000","last_became_active":"2024-04-25T00:14:06.861418+0000","last_became_peered":"2024-04-25T00:14:06.861418+0000","last_unstale":"2024-04-25T00:15:02.073810+0000","last_undegraded":"2024-04-25T00:15:02.073810+0000","last_fullsized":"2024-04-25T00:15:02.073810+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:27.559704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.073710+0000","last_change":"2024-04-25T00:14:01.407648+0000","last_active":"2024-04-25T00:15:02.073710+0000","last_peered":"2024-04-25T00:15:02.073710+0000","last_clean":"2024-04-25T00:15:02.073710+0000","last_became_active":"2024-04-25T00:14:01.407318+0000","last_became_peered":"2024-04-25T00:14:01.407318+0000","last_unstale":"2024-04-25T00:15:02.073710+0000","last_undegraded":"2024-04-25T00:15:02.073710+0000","last_fullsized":"2024-04-25T00:15:02.073710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:08:10.942814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.965829+0000","last_change":"2024-04-25T00:14:01.833073+0000","last_active":"2024-04-25T00:15:30.965829+0000","last_peered":"2024-04-25T00:15:30.965829+0000","last_clean":"2024-04-25T00:15:30.965829+0000","last_became_active":"2024-04-25T00:14:01.832545+0000","last_became_peered":"2024-04-25T00:14:01.832545+0000","last_unstale":"2024-04-25T00:15:30.965829+0000","last_undegraded":"2024-04-25T00:15:30.965829+0000","last_fullsized":"2024-04-25T00:15:30.965829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:39.772781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076033+0000","last_change":"2024-04-25T00:14:06.862011+0000","last_active":"2024-04-25T00:15:02.076033+0000","last_peered":"2024-04-25T00:15:02.076033+0000","last_clean":"2024-04-25T00:15:02.076033+0000","last_became_active":"2024-04-25T00:14:06.861544+0000","last_became_peered":"2024-04-25T00:14:06.861544+0000","last_unstale":"2024-04-25T00:15:02.076033+0000","last_undegraded":"2024-04-25T00:15:02.076033+0000","last_fullsized":"2024-04-25T00:15:02.076033+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:37.736266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.370974+0000","last_change":"2024-04-25T00:14:01.397747+0000","last_active":"2024-04-25T00:15:02.370974+0000","last_peered":"2024-04-25T00:15:02.370974+0000","last_clean":"2024-04-25T00:15:02.370974+0000","last_became_active":"2024-04-25T00:14:01.397446+0000","last_became_peered":"2024-04-25T00:14:01.397446+0000","last_unstale":"2024-04-25T00:15:02.370974+0000","last_undegraded":"2024-04-25T00:15:02.370974+0000","last_fullsized":"2024-04-25T00:15:02.370974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:27.240034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.964056+0000","last_change":"2024-04-25T00:14:01.833128+0000","last_active":"2024-04-25T00:15:30.964056+0000","last_peered":"2024-04-25T00:15:30.964056+0000","last_clean":"2024-04-25T00:15:30.964056+0000","last_became_active":"2024-04-25T00:14:01.832780+0000","last_became_peered":"2024-04-25T00:14:01.832780+0000","last_unstale":"2024-04-25T00:15:30.964056+0000","last_undegraded":"2024-04-25T00:15:30.964056+0000","last_fullsized":"2024-04-25T00:15:30.964056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.023737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076514+0000","last_change":"2024-04-25T00:14:06.860436+0000","last_active":"2024-04-25T00:15:02.076514+0000","last_peered":"2024-04-25T00:15:02.076514+0000","last_clean":"2024-04-25T00:15:02.076514+0000","last_became_active":"2024-04-25T00:14:06.860047+0000","last_became_peered":"2024-04-25T00:14:06.860047+0000","last_unstale":"2024-04-25T00:15:02.076514+0000","last_undegraded":"2024-04-25T00:15:02.076514+0000","last_fullsized":"2024-04-25T00:15:02.076514+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:08:41.246549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.082288+0000","last_change":"2024-04-25T00:14:01.415939+0000","last_active":"2024-04-25T00:15:02.082288+0000","last_peered":"2024-04-25T00:15:02.082288+0000","last_clean":"2024-04-25T00:15:02.082288+0000","last_became_active":"2024-04-25T00:14:01.415513+0000","last_became_peered":"2024-04-25T00:14:01.415513+0000","last_unstale":"2024-04-25T00:15:02.082288+0000","last_undegraded":"2024-04-25T00:15:02.082288+0000","last_fullsized":"2024-04-25T00:15:02.082288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:57:23.588868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.955256+0000","last_change":"2024-04-25T00:14:01.838793+0000","last_active":"2024-04-25T00:15:30.955256+0000","last_peered":"2024-04-25T00:15:30.955256+0000","last_clean":"2024-04-25T00:15:30.955256+0000","last_became_active":"2024-04-25T00:14:01.838453+0000","last_became_peered":"2024-04-25T00:14:01.838453+0000","last_unstale":"2024-04-25T00:15:30.955256+0000","last_undegraded":"2024-04-25T00:15:30.955256+0000","last_fullsized":"2024-04-25T00:15:30.955256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:18.674009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.073683+0000","last_change":"2024-04-25T00:14:06.856431+0000","last_active":"2024-04-25T00:15:02.073683+0000","last_peered":"2024-04-25T00:15:02.073683+0000","last_clean":"2024-04-25T00:15:02.073683+0000","last_became_active":"2024-04-25T00:14:06.856047+0000","last_became_peered":"2024-04-25T00:14:06.856047+0000","last_unstale":"2024-04-25T00:15:02.073683+0000","last_undegraded":"2024-04-25T00:15:02.073683+0000","last_fullsized":"2024-04-25T00:15:02.073683+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:20.646673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":107,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:38.646457+0000","last_change":"2024-04-25T00:14:01.399054+0000","last_active":"2024-04-25T00:15:38.646457+0000","last_peered":"2024-04-25T00:15:38.646457+0000","last_clean":"2024-04-25T00:15:38.646457+0000","last_became_active":"2024-04-25T00:14:01.398711+0000","last_became_peered":"2024-04-25T00:14:01.398711+0000","last_unstale":"2024-04-25T00:15:38.646457+0000","last_undegraded":"2024-04-25T00:15:38.646457+0000","last_fullsized":"2024-04-25T00:15:38.646457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:26.434014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.957701+0000","last_change":"2024-04-25T00:14:01.829132+0000","last_active":"2024-04-25T00:15:30.957701+0000","last_peered":"2024-04-25T00:15:30.957701+0000","last_clean":"2024-04-25T00:15:30.957701+0000","last_became_active":"2024-04-25T00:14:01.828446+0000","last_became_peered":"2024-04-25T00:14:01.828446+0000","last_unstale":"2024-04-25T00:15:30.957701+0000","last_undegraded":"2024-04-25T00:15:30.957701+0000","last_fullsized":"2024-04-25T00:15:30.957701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:35.788478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076380+0000","last_change":"2024-04-25T00:14:06.861884+0000","last_active":"2024-04-25T00:15:02.076380+0000","last_peered":"2024-04-25T00:15:02.076380+0000","last_clean":"2024-04-25T00:15:02.076380+0000","last_became_active":"2024-04-25T00:14:06.861413+0000","last_became_peered":"2024-04-25T00:14:06.861413+0000","last_unstale":"2024-04-25T00:15:02.076380+0000","last_undegraded":"2024-04-25T00:15:02.076380+0000","last_fullsized":"2024-04-25T00:15:02.076380+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:53.876308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.371309+0000","last_change":"2024-04-25T00:14:01.410372+0000","last_active":"2024-04-25T00:15:02.371309+0000","last_peered":"2024-04-25T00:15:02.371309+0000","last_clean":"2024-04-25T00:15:02.371309+0000","last_became_active":"2024-04-25T00:14:01.410074+0000","last_became_peered":"2024-04-25T00:14:01.410074+0000","last_unstale":"2024-04-25T00:15:02.371309+0000","last_undegraded":"2024-04-25T00:15:02.371309+0000","last_fullsized":"2024-04-25T00:15:02.371309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:00.964580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.962909+0000","last_change":"2024-04-25T00:15:02.372269+0000","last_active":"2024-04-25T00:15:30.962909+0000","last_peered":"2024-04-25T00:15:30.962909+0000","last_clean":"2024-04-25T00:15:30.962909+0000","last_became_active":"2024-04-25T00:15:02.371962+0000","last_became_peered":"2024-04-25T00:15:02.371962+0000","last_unstale":"2024-04-25T00:15:30.962909+0000","last_undegraded":"2024-04-25T00:15:30.962909+0000","last_fullsized":"2024-04-25T00:15:30.962909+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:46.597638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.372941+0000","last_change":"2024-04-25T00:15:02.372941+0000","last_active":"2024-04-25T00:15:02.372941+0000","last_peered":"2024-04-25T00:15:02.372941+0000","last_clean":"2024-04-25T00:15:02.372941+0000","last_became_active":"2024-04-25T00:15:02.372504+0000","last_became_peered":"2024-04-25T00:15:02.372504+0000","last_unstale":"2024-04-25T00:15:02.372941+0000","last_undegraded":"2024-04-25T00:15:02.372941+0000","last_fullsized":"2024-04-25T00:15:02.372941+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:58.202708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.081403+0000","last_change":"2024-04-25T00:14:01.401128+0000","last_active":"2024-04-25T00:15:02.081403+0000","last_peered":"2024-04-25T00:15:02.081403+0000","last_clean":"2024-04-25T00:15:02.081403+0000","last_became_active":"2024-04-25T00:14:01.400125+0000","last_became_peered":"2024-04-25T00:14:01.400125+0000","last_unstale":"2024-04-25T00:15:02.081403+0000","last_undegraded":"2024-04-25T00:15:02.081403+0000","last_fullsized":"2024-04-25T00:15:02.081403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:38:43.787855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.960159+0000","last_change":"2024-04-25T00:14:01.830649+0000","last_active":"2024-04-25T00:15:30.960159+0000","last_peered":"2024-04-25T00:15:30.960159+0000","last_clean":"2024-04-25T00:15:30.960159+0000","last_became_active":"2024-04-25T00:14:01.830253+0000","last_became_peered":"2024-04-25T00:14:01.830253+0000","last_unstale":"2024-04-25T00:15:30.960159+0000","last_undegraded":"2024-04-25T00:15:30.960159+0000","last_fullsized":"2024-04-25T00:15:30.960159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:37.474833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.075600+0000","last_change":"2024-04-25T00:14:06.861275+0000","last_active":"2024-04-25T00:15:02.075600+0000","last_peered":"2024-04-25T00:15:02.075600+0000","last_clean":"2024-04-25T00:15:02.075600+0000","last_became_active":"2024-04-25T00:14:06.860891+0000","last_became_peered":"2024-04-25T00:14:06.860891+0000","last_unstale":"2024-04-25T00:15:02.075600+0000","last_undegraded":"2024-04-25T00:15:02.075600+0000","last_fullsized":"2024-04-25T00:15:02.075600+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:38.685837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'36","reported_seq":220,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:39.653642+0000","last_change":"2024-04-25T00:14:01.404706+0000","last_active":"2024-04-25T00:15:39.653642+0000","last_peered":"2024-04-25T00:15:39.653642+0000","last_clean":"2024-04-25T00:15:39.653642+0000","last_became_active":"2024-04-25T00:14:01.404231+0000","last_became_peered":"2024-04-25T00:14:01.404231+0000","last_unstale":"2024-04-25T00:15:39.653642+0000","last_undegraded":"2024-04-25T00:15:39.653642+0000","last_fullsized":"2024-04-25T00:15:39.653642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:36.437491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63605,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":153,"num_read_kb":139,"num_write":37,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.941762+0000","last_change":"2024-04-25T00:14:01.828312+0000","last_active":"2024-04-25T00:15:30.941762+0000","last_peered":"2024-04-25T00:15:30.941762+0000","last_clean":"2024-04-25T00:15:30.941762+0000","last_became_active":"2024-04-25T00:14:01.827947+0000","last_became_peered":"2024-04-25T00:14:01.827947+0000","last_unstale":"2024-04-25T00:15:30.941762+0000","last_undegraded":"2024-04-25T00:15:30.941762+0000","last_fullsized":"2024-04-25T00:15:30.941762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.782796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.081614+0000","last_change":"2024-04-25T00:14:06.854379+0000","last_active":"2024-04-25T00:15:02.081614+0000","last_peered":"2024-04-25T00:15:02.081614+0000","last_clean":"2024-04-25T00:15:02.081614+0000","last_became_active":"2024-04-25T00:14:06.853307+0000","last_became_peered":"2024-04-25T00:14:06.853307+0000","last_unstale":"2024-04-25T00:15:02.081614+0000","last_undegraded":"2024-04-25T00:15:02.081614+0000","last_fullsized":"2024-04-25T00:15:02.081614+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:24:11.804189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.074161+0000","last_change":"2024-04-25T00:14:01.410395+0000","last_active":"2024-04-25T00:15:02.074161+0000","last_peered":"2024-04-25T00:15:02.074161+0000","last_clean":"2024-04-25T00:15:02.074161+0000","last_became_active":"2024-04-25T00:14:01.410039+0000","last_became_peered":"2024-04-25T00:14:01.410039+0000","last_unstale":"2024-04-25T00:15:02.074161+0000","last_undegraded":"2024-04-25T00:15:02.074161+0000","last_fullsized":"2024-04-25T00:15:02.074161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:29.496519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.959939+0000","last_change":"2024-04-25T00:14:01.828403+0000","last_active":"2024-04-25T00:15:30.959939+0000","last_peered":"2024-04-25T00:15:30.959939+0000","last_clean":"2024-04-25T00:15:30.959939+0000","last_became_active":"2024-04-25T00:14:01.828107+0000","last_became_peered":"2024-04-25T00:14:01.828107+0000","last_unstale":"2024-04-25T00:15:30.959939+0000","last_undegraded":"2024-04-25T00:15:30.959939+0000","last_fullsized":"2024-04-25T00:15:30.959939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:29.417639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.374105+0000","last_change":"2024-04-25T00:15:02.374105+0000","last_active":"2024-04-25T00:15:02.374105+0000","last_peered":"2024-04-25T00:15:02.374105+0000","last_clean":"2024-04-25T00:15:02.374105+0000","last_became_active":"2024-04-25T00:15:02.373628+0000","last_became_peered":"2024-04-25T00:15:02.373628+0000","last_unstale":"2024-04-25T00:15:02.374105+0000","last_undegraded":"2024-04-25T00:15:02.374105+0000","last_fullsized":"2024-04-25T00:15:02.374105+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:05.542358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.080742+0000","last_change":"2024-04-25T00:14:01.414406+0000","last_active":"2024-04-25T00:15:02.080742+0000","last_peered":"2024-04-25T00:15:02.080742+0000","last_clean":"2024-04-25T00:15:02.080742+0000","last_became_active":"2024-04-25T00:14:01.414085+0000","last_became_peered":"2024-04-25T00:14:01.414085+0000","last_unstale":"2024-04-25T00:15:02.080742+0000","last_undegraded":"2024-04-25T00:15:02.080742+0000","last_fullsized":"2024-04-25T00:15:02.080742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:22.565371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.961060+0000","last_change":"2024-04-25T00:14:01.833652+0000","last_active":"2024-04-25T00:15:30.961060+0000","last_peered":"2024-04-25T00:15:30.961060+0000","last_clean":"2024-04-25T00:15:30.961060+0000","last_became_active":"2024-04-25T00:14:01.833307+0000","last_became_peered":"2024-04-25T00:14:01.833307+0000","last_unstale":"2024-04-25T00:15:30.961060+0000","last_undegraded":"2024-04-25T00:15:30.961060+0000","last_fullsized":"2024-04-25T00:15:30.961060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:55:41.261994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.075757+0000","last_change":"2024-04-25T00:14:06.860167+0000","last_active":"2024-04-25T00:15:02.075757+0000","last_peered":"2024-04-25T00:15:02.075757+0000","last_clean":"2024-04-25T00:15:02.075757+0000","last_became_active":"2024-04-25T00:14:06.858651+0000","last_became_peered":"2024-04-25T00:14:06.858651+0000","last_unstale":"2024-04-25T00:15:02.075757+0000","last_undegraded":"2024-04-25T00:15:02.075757+0000","last_fullsized":"2024-04-25T00:15:02.075757+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:11.775434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'3","reported_seq":107,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:37.643220+0000","last_change":"2024-04-25T00:14:01.406267+0000","last_active":"2024-04-25T00:15:37.643220+0000","last_peered":"2024-04-25T00:15:37.643220+0000","last_clean":"2024-04-25T00:15:37.643220+0000","last_became_active":"2024-04-25T00:14:01.405985+0000","last_became_peered":"2024-04-25T00:14:01.405985+0000","last_unstale":"2024-04-25T00:15:37.643220+0000","last_undegraded":"2024-04-25T00:15:37.643220+0000","last_fullsized":"2024-04-25T00:15:37.643220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:17.253805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.952457+0000","last_change":"2024-04-25T00:14:01.838019+0000","last_active":"2024-04-25T00:15:30.952457+0000","last_peered":"2024-04-25T00:15:30.952457+0000","last_clean":"2024-04-25T00:15:30.952457+0000","last_became_active":"2024-04-25T00:14:01.837691+0000","last_became_peered":"2024-04-25T00:14:01.837691+0000","last_unstale":"2024-04-25T00:15:30.952457+0000","last_undegraded":"2024-04-25T00:15:30.952457+0000","last_fullsized":"2024-04-25T00:15:30.952457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:26.677039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.371358+0000","last_change":"2024-04-25T00:14:06.857061+0000","last_active":"2024-04-25T00:15:02.371358+0000","last_peered":"2024-04-25T00:15:02.371358+0000","last_clean":"2024-04-25T00:15:02.371358+0000","last_became_active":"2024-04-25T00:14:06.856678+0000","last_became_peered":"2024-04-25T00:14:06.856678+0000","last_unstale":"2024-04-25T00:15:02.371358+0000","last_undegraded":"2024-04-25T00:15:02.371358+0000","last_fullsized":"2024-04-25T00:15:02.371358+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:36:52.730861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.075817+0000","last_change":"2024-04-25T00:14:01.408385+0000","last_active":"2024-04-25T00:15:02.075817+0000","last_peered":"2024-04-25T00:15:02.075817+0000","last_clean":"2024-04-25T00:15:02.075817+0000","last_became_active":"2024-04-25T00:14:01.408049+0000","last_became_peered":"2024-04-25T00:14:01.408049+0000","last_unstale":"2024-04-25T00:15:02.075817+0000","last_undegraded":"2024-04-25T00:15:02.075817+0000","last_fullsized":"2024-04-25T00:15:02.075817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:19.765467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.955201+0000","last_change":"2024-04-25T00:15:02.372426+0000","last_active":"2024-04-25T00:15:30.955201+0000","last_peered":"2024-04-25T00:15:30.955201+0000","last_clean":"2024-04-25T00:15:30.955201+0000","last_became_active":"2024-04-25T00:15:02.372044+0000","last_became_peered":"2024-04-25T00:15:02.372044+0000","last_unstale":"2024-04-25T00:15:30.955201+0000","last_undegraded":"2024-04-25T00:15:30.955201+0000","last_fullsized":"2024-04-25T00:15:30.955201+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:20.965683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.082497+0000","last_change":"2024-04-25T00:14:06.854192+0000","last_active":"2024-04-25T00:15:02.082497+0000","last_peered":"2024-04-25T00:15:02.082497+0000","last_clean":"2024-04-25T00:15:02.082497+0000","last_became_active":"2024-04-25T00:14:06.853325+0000","last_became_peered":"2024-04-25T00:14:06.853325+0000","last_unstale":"2024-04-25T00:15:02.082497+0000","last_undegraded":"2024-04-25T00:15:02.082497+0000","last_fullsized":"2024-04-25T00:15:02.082497+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:15.308169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076565+0000","last_change":"2024-04-25T00:14:01.407247+0000","last_active":"2024-04-25T00:15:02.076565+0000","last_peered":"2024-04-25T00:15:02.076565+0000","last_clean":"2024-04-25T00:15:02.076565+0000","last_became_active":"2024-04-25T00:14:01.406891+0000","last_became_peered":"2024-04-25T00:14:01.406891+0000","last_unstale":"2024-04-25T00:15:02.076565+0000","last_undegraded":"2024-04-25T00:15:02.076565+0000","last_fullsized":"2024-04-25T00:15:02.076565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:43.294562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.945357+0000","last_change":"2024-04-25T00:14:01.829364+0000","last_active":"2024-04-25T00:15:30.945357+0000","last_peered":"2024-04-25T00:15:30.945357+0000","last_clean":"2024-04-25T00:15:30.945357+0000","last_became_active":"2024-04-25T00:14:01.829087+0000","last_became_peered":"2024-04-25T00:14:01.829087+0000","last_unstale":"2024-04-25T00:15:30.945357+0000","last_undegraded":"2024-04-25T00:15:30.945357+0000","last_fullsized":"2024-04-25T00:15:30.945357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:17.651815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076133+0000","last_change":"2024-04-25T00:14:07.323442+0000","last_active":"2024-04-25T00:15:02.076133+0000","last_peered":"2024-04-25T00:15:02.076133+0000","last_clean":"2024-04-25T00:15:02.076133+0000","last_became_active":"2024-04-25T00:14:07.322943+0000","last_became_peered":"2024-04-25T00:14:07.322943+0000","last_unstale":"2024-04-25T00:15:02.076133+0000","last_undegraded":"2024-04-25T00:15:02.076133+0000","last_fullsized":"2024-04-25T00:15:02.076133+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:49.166240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076222+0000","last_change":"2024-04-25T00:14:01.410936+0000","last_active":"2024-04-25T00:15:02.076222+0000","last_peered":"2024-04-25T00:15:02.076222+0000","last_clean":"2024-04-25T00:15:02.076222+0000","last_became_active":"2024-04-25T00:14:01.410321+0000","last_became_peered":"2024-04-25T00:14:01.410321+0000","last_unstale":"2024-04-25T00:15:02.076222+0000","last_undegraded":"2024-04-25T00:15:02.076222+0000","last_fullsized":"2024-04-25T00:15:02.076222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:44.603199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"94'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.938542+0000","last_change":"2024-04-25T00:14:01.830842+0000","last_active":"2024-04-25T00:15:30.938542+0000","last_peered":"2024-04-25T00:15:30.938542+0000","last_clean":"2024-04-25T00:15:30.938542+0000","last_became_active":"2024-04-25T00:14:01.830543+0000","last_became_peered":"2024-04-25T00:14:01.830543+0000","last_unstale":"2024-04-25T00:15:30.938542+0000","last_undegraded":"2024-04-25T00:15:30.938542+0000","last_fullsized":"2024-04-25T00:15:30.938542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:59.897133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.371351+0000","last_change":"2024-04-25T00:14:06.856841+0000","last_active":"2024-04-25T00:15:02.371351+0000","last_peered":"2024-04-25T00:15:02.371351+0000","last_clean":"2024-04-25T00:15:02.371351+0000","last_became_active":"2024-04-25T00:14:06.856416+0000","last_became_peered":"2024-04-25T00:14:06.856416+0000","last_unstale":"2024-04-25T00:15:02.371351+0000","last_undegraded":"2024-04-25T00:15:02.371351+0000","last_fullsized":"2024-04-25T00:15:02.371351+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:27.181175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.371225+0000","last_change":"2024-04-25T00:14:01.409627+0000","last_active":"2024-04-25T00:15:02.371225+0000","last_peered":"2024-04-25T00:15:02.371225+0000","last_clean":"2024-04-25T00:15:02.371225+0000","last_became_active":"2024-04-25T00:14:01.409304+0000","last_became_peered":"2024-04-25T00:14:01.409304+0000","last_unstale":"2024-04-25T00:15:02.371225+0000","last_undegraded":"2024-04-25T00:15:02.371225+0000","last_fullsized":"2024-04-25T00:15:02.371225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:27.242004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.958599+0000","last_change":"2024-04-25T00:14:01.829544+0000","last_active":"2024-04-25T00:15:30.958599+0000","last_peered":"2024-04-25T00:15:30.958599+0000","last_clean":"2024-04-25T00:15:30.958599+0000","last_became_active":"2024-04-25T00:14:01.829233+0000","last_became_peered":"2024-04-25T00:14:01.829233+0000","last_unstale":"2024-04-25T00:15:30.958599+0000","last_undegraded":"2024-04-25T00:15:30.958599+0000","last_fullsized":"2024-04-25T00:15:30.958599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:16.102041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.371517+0000","last_change":"2024-04-25T00:14:07.323213+0000","last_active":"2024-04-25T00:15:02.371517+0000","last_peered":"2024-04-25T00:15:02.371517+0000","last_clean":"2024-04-25T00:15:02.371517+0000","last_became_active":"2024-04-25T00:14:07.322763+0000","last_became_peered":"2024-04-25T00:14:07.322763+0000","last_unstale":"2024-04-25T00:15:02.371517+0000","last_undegraded":"2024-04-25T00:15:02.371517+0000","last_fullsized":"2024-04-25T00:15:02.371517+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:45.269314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.947333+0000","last_change":"2024-04-25T00:15:02.079745+0000","last_active":"2024-04-25T00:15:30.947333+0000","last_peered":"2024-04-25T00:15:30.947333+0000","last_clean":"2024-04-25T00:15:30.947333+0000","last_became_active":"2024-04-25T00:15:02.079405+0000","last_became_peered":"2024-04-25T00:15:02.079405+0000","last_unstale":"2024-04-25T00:15:30.947333+0000","last_undegraded":"2024-04-25T00:15:30.947333+0000","last_fullsized":"2024-04-25T00:15:30.947333+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:38.197048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"92'4","reported_seq":106,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:37.642606+0000","last_change":"2024-04-25T00:14:01.397066+0000","last_active":"2024-04-25T00:15:37.642606+0000","last_peered":"2024-04-25T00:15:37.642606+0000","last_clean":"2024-04-25T00:15:37.642606+0000","last_became_active":"2024-04-25T00:14:01.396728+0000","last_became_peered":"2024-04-25T00:14:01.396728+0000","last_unstale":"2024-04-25T00:15:37.642606+0000","last_undegraded":"2024-04-25T00:15:37.642606+0000","last_fullsized":"2024-04-25T00:15:37.642606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:28.334329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"92'1","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.371995+0000","last_change":"2024-04-25T00:14:06.853039+0000","last_active":"2024-04-25T00:15:02.371995+0000","last_peered":"2024-04-25T00:15:02.371995+0000","last_clean":"2024-04-25T00:15:02.371995+0000","last_became_active":"2024-04-25T00:14:06.852568+0000","last_became_peered":"2024-04-25T00:14:06.852568+0000","last_unstale":"2024-04-25T00:15:02.371995+0000","last_undegraded":"2024-04-25T00:15:02.371995+0000","last_fullsized":"2024-04-25T00:15:02.371995+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:59.604160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.965920+0000","last_change":"2024-04-25T00:14:01.840315+0000","last_active":"2024-04-25T00:15:30.965920+0000","last_peered":"2024-04-25T00:15:30.965920+0000","last_clean":"2024-04-25T00:15:30.965920+0000","last_became_active":"2024-04-25T00:14:01.839537+0000","last_became_peered":"2024-04-25T00:14:01.839537+0000","last_unstale":"2024-04-25T00:15:30.965920+0000","last_undegraded":"2024-04-25T00:15:30.965920+0000","last_fullsized":"2024-04-25T00:15:30.965920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:34.126164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.075830+0000","last_change":"2024-04-25T00:14:01.412493+0000","last_active":"2024-04-25T00:15:02.075830+0000","last_peered":"2024-04-25T00:15:02.075830+0000","last_clean":"2024-04-25T00:15:02.075830+0000","last_became_active":"2024-04-25T00:14:01.412133+0000","last_became_peered":"2024-04-25T00:14:01.412133+0000","last_unstale":"2024-04-25T00:15:02.075830+0000","last_undegraded":"2024-04-25T00:15:02.075830+0000","last_fullsized":"2024-04-25T00:15:02.075830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:44.933728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":365,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.370851+0000","last_change":"2024-04-25T00:12:49.169294+0000","last_active":"2024-04-25T00:15:02.370851+0000","last_peered":"2024-04-25T00:15:02.370851+0000","last_clean":"2024-04-25T00:15:02.370851+0000","last_became_active":"2024-04-25T00:12:49.168783+0000","last_became_peered":"2024-04-25T00:12:49.168783+0000","last_unstale":"2024-04-25T00:15:02.370851+0000","last_undegraded":"2024-04-25T00:15:02.370851+0000","last_fullsized":"2024-04-25T00:15:02.370851+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:25.202280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:25.202280+0000","last_clean_scrub_stamp":"2024-04-25T00:09:25.202280+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:04.713986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.373148+0000","last_change":"2024-04-25T00:15:02.372570+0000","last_active":"2024-04-25T00:15:02.373148+0000","last_peered":"2024-04-25T00:15:02.373148+0000","last_clean":"2024-04-25T00:15:02.373148+0000","last_became_active":"2024-04-25T00:15:02.372066+0000","last_became_peered":"2024-04-25T00:15:02.372066+0000","last_unstale":"2024-04-25T00:15:02.373148+0000","last_undegraded":"2024-04-25T00:15:02.373148+0000","last_fullsized":"2024-04-25T00:15:02.373148+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:59.470049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,6,10],"acting":[8,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.960856+0000","last_change":"2024-04-25T00:14:01.826418+0000","last_active":"2024-04-25T00:15:30.960856+0000","last_peered":"2024-04-25T00:15:30.960856+0000","last_clean":"2024-04-25T00:15:30.960856+0000","last_became_active":"2024-04-25T00:14:01.825783+0000","last_became_peered":"2024-04-25T00:14:01.825783+0000","last_unstale":"2024-04-25T00:15:30.960856+0000","last_undegraded":"2024-04-25T00:15:30.960856+0000","last_fullsized":"2024-04-25T00:15:30.960856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:05.981359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.370975+0000","last_change":"2024-04-25T00:14:01.417022+0000","last_active":"2024-04-25T00:15:02.370975+0000","last_peered":"2024-04-25T00:15:02.370975+0000","last_clean":"2024-04-25T00:15:02.370975+0000","last_became_active":"2024-04-25T00:14:01.416804+0000","last_became_peered":"2024-04-25T00:14:01.416804+0000","last_unstale":"2024-04-25T00:15:02.370975+0000","last_undegraded":"2024-04-25T00:15:02.370975+0000","last_fullsized":"2024-04-25T00:15:02.370975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:31.561242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.075667+0000","last_change":"2024-04-25T00:14:06.856500+0000","last_active":"2024-04-25T00:15:02.075667+0000","last_peered":"2024-04-25T00:15:02.075667+0000","last_clean":"2024-04-25T00:15:02.075667+0000","last_became_active":"2024-04-25T00:14:06.856077+0000","last_became_peered":"2024-04-25T00:14:06.856077+0000","last_unstale":"2024-04-25T00:15:02.075667+0000","last_undegraded":"2024-04-25T00:15:02.075667+0000","last_fullsized":"2024-04-25T00:15:02.075667+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:10:04.129533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076218+0000","last_change":"2024-04-25T00:14:01.414599+0000","last_active":"2024-04-25T00:15:02.076218+0000","last_peered":"2024-04-25T00:15:02.076218+0000","last_clean":"2024-04-25T00:15:02.076218+0000","last_became_active":"2024-04-25T00:14:01.413796+0000","last_became_peered":"2024-04-25T00:14:01.413796+0000","last_unstale":"2024-04-25T00:15:02.076218+0000","last_undegraded":"2024-04-25T00:15:02.076218+0000","last_fullsized":"2024-04-25T00:15:02.076218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:44.981848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.960865+0000","last_change":"2024-04-25T00:14:01.827718+0000","last_active":"2024-04-25T00:15:30.960865+0000","last_peered":"2024-04-25T00:15:30.960865+0000","last_clean":"2024-04-25T00:15:30.960865+0000","last_became_active":"2024-04-25T00:14:01.827410+0000","last_became_peered":"2024-04-25T00:14:01.827410+0000","last_unstale":"2024-04-25T00:15:30.960865+0000","last_undegraded":"2024-04-25T00:15:30.960865+0000","last_fullsized":"2024-04-25T00:15:30.960865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:32.428137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.073685+0000","last_change":"2024-04-25T00:14:07.323304+0000","last_active":"2024-04-25T00:15:02.073685+0000","last_peered":"2024-04-25T00:15:02.073685+0000","last_clean":"2024-04-25T00:15:02.073685+0000","last_became_active":"2024-04-25T00:14:07.322852+0000","last_became_peered":"2024-04-25T00:14:07.322852+0000","last_unstale":"2024-04-25T00:15:02.073685+0000","last_undegraded":"2024-04-25T00:15:02.073685+0000","last_fullsized":"2024-04-25T00:15:02.073685+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:35.173934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076228+0000","last_change":"2024-04-25T00:14:01.413209+0000","last_active":"2024-04-25T00:15:02.076228+0000","last_peered":"2024-04-25T00:15:02.076228+0000","last_clean":"2024-04-25T00:15:02.076228+0000","last_became_active":"2024-04-25T00:14:01.412892+0000","last_became_peered":"2024-04-25T00:14:01.412892+0000","last_unstale":"2024-04-25T00:15:02.076228+0000","last_undegraded":"2024-04-25T00:15:02.076228+0000","last_fullsized":"2024-04-25T00:15:02.076228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:26.152962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.961212+0000","last_change":"2024-04-25T00:14:01.834372+0000","last_active":"2024-04-25T00:15:30.961212+0000","last_peered":"2024-04-25T00:15:30.961212+0000","last_clean":"2024-04-25T00:15:30.961212+0000","last_became_active":"2024-04-25T00:14:01.834062+0000","last_became_peered":"2024-04-25T00:14:01.834062+0000","last_unstale":"2024-04-25T00:15:30.961212+0000","last_undegraded":"2024-04-25T00:15:30.961212+0000","last_fullsized":"2024-04-25T00:15:30.961212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:05.580604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.073535+0000","last_change":"2024-04-25T00:14:06.852303+0000","last_active":"2024-04-25T00:15:02.073535+0000","last_peered":"2024-04-25T00:15:02.073535+0000","last_clean":"2024-04-25T00:15:02.073535+0000","last_became_active":"2024-04-25T00:14:06.850957+0000","last_became_peered":"2024-04-25T00:14:06.850957+0000","last_unstale":"2024-04-25T00:15:02.073535+0000","last_undegraded":"2024-04-25T00:15:02.073535+0000","last_fullsized":"2024-04-25T00:15:02.073535+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:47.208880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076101+0000","last_change":"2024-04-25T00:14:01.405538+0000","last_active":"2024-04-25T00:15:02.076101+0000","last_peered":"2024-04-25T00:15:02.076101+0000","last_clean":"2024-04-25T00:15:02.076101+0000","last_became_active":"2024-04-25T00:14:01.405155+0000","last_became_peered":"2024-04-25T00:14:01.405155+0000","last_unstale":"2024-04-25T00:15:02.076101+0000","last_undegraded":"2024-04-25T00:15:02.076101+0000","last_fullsized":"2024-04-25T00:15:02.076101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:20.864753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.962393+0000","last_change":"2024-04-25T00:14:01.837058+0000","last_active":"2024-04-25T00:15:30.962393+0000","last_peered":"2024-04-25T00:15:30.962393+0000","last_clean":"2024-04-25T00:15:30.962393+0000","last_became_active":"2024-04-25T00:14:01.835838+0000","last_became_peered":"2024-04-25T00:14:01.835838+0000","last_unstale":"2024-04-25T00:15:30.962393+0000","last_undegraded":"2024-04-25T00:15:30.962393+0000","last_fullsized":"2024-04-25T00:15:30.962393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:21.696448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.075742+0000","last_change":"2024-04-25T00:14:07.323870+0000","last_active":"2024-04-25T00:15:02.075742+0000","last_peered":"2024-04-25T00:15:02.075742+0000","last_clean":"2024-04-25T00:15:02.075742+0000","last_became_active":"2024-04-25T00:14:07.323356+0000","last_became_peered":"2024-04-25T00:14:07.323356+0000","last_unstale":"2024-04-25T00:15:02.075742+0000","last_undegraded":"2024-04-25T00:15:02.075742+0000","last_fullsized":"2024-04-25T00:15:02.075742+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:35.160618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.371557+0000","last_change":"2024-04-25T00:14:01.409453+0000","last_active":"2024-04-25T00:15:02.371557+0000","last_peered":"2024-04-25T00:15:02.371557+0000","last_clean":"2024-04-25T00:15:02.371557+0000","last_became_active":"2024-04-25T00:14:01.409164+0000","last_became_peered":"2024-04-25T00:14:01.409164+0000","last_unstale":"2024-04-25T00:15:02.371557+0000","last_undegraded":"2024-04-25T00:15:02.371557+0000","last_fullsized":"2024-04-25T00:15:02.371557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:23.266090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.910070+0000","last_change":"2024-04-25T00:14:01.826790+0000","last_active":"2024-04-25T00:15:30.910070+0000","last_peered":"2024-04-25T00:15:30.910070+0000","last_clean":"2024-04-25T00:15:30.910070+0000","last_became_active":"2024-04-25T00:14:01.826457+0000","last_became_peered":"2024-04-25T00:14:01.826457+0000","last_unstale":"2024-04-25T00:15:30.910070+0000","last_undegraded":"2024-04-25T00:15:30.910070+0000","last_fullsized":"2024-04-25T00:15:30.910070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:15.356448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076592+0000","last_change":"2024-04-25T00:14:06.863280+0000","last_active":"2024-04-25T00:15:02.076592+0000","last_peered":"2024-04-25T00:15:02.076592+0000","last_clean":"2024-04-25T00:15:02.076592+0000","last_became_active":"2024-04-25T00:14:06.862725+0000","last_became_peered":"2024-04-25T00:14:06.862725+0000","last_unstale":"2024-04-25T00:15:02.076592+0000","last_undegraded":"2024-04-25T00:15:02.076592+0000","last_fullsized":"2024-04-25T00:15:02.076592+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:12.791744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.073815+0000","last_change":"2024-04-25T00:14:01.414856+0000","last_active":"2024-04-25T00:15:02.073815+0000","last_peered":"2024-04-25T00:15:02.073815+0000","last_clean":"2024-04-25T00:15:02.073815+0000","last_became_active":"2024-04-25T00:14:01.414533+0000","last_became_peered":"2024-04-25T00:14:01.414533+0000","last_unstale":"2024-04-25T00:15:02.073815+0000","last_undegraded":"2024-04-25T00:15:02.073815+0000","last_fullsized":"2024-04-25T00:15:02.073815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:01.090539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.952846+0000","last_change":"2024-04-25T00:14:01.831452+0000","last_active":"2024-04-25T00:15:30.952846+0000","last_peered":"2024-04-25T00:15:30.952846+0000","last_clean":"2024-04-25T00:15:30.952846+0000","last_became_active":"2024-04-25T00:14:01.831151+0000","last_became_peered":"2024-04-25T00:14:01.831151+0000","last_unstale":"2024-04-25T00:15:30.952846+0000","last_undegraded":"2024-04-25T00:15:30.952846+0000","last_fullsized":"2024-04-25T00:15:30.952846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:53:07.659508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"92'1","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076010+0000","last_change":"2024-04-25T00:14:06.857601+0000","last_active":"2024-04-25T00:15:02.076010+0000","last_peered":"2024-04-25T00:15:02.076010+0000","last_clean":"2024-04-25T00:15:02.076010+0000","last_became_active":"2024-04-25T00:14:06.856685+0000","last_became_peered":"2024-04-25T00:14:06.856685+0000","last_unstale":"2024-04-25T00:15:02.076010+0000","last_undegraded":"2024-04-25T00:15:02.076010+0000","last_fullsized":"2024-04-25T00:15:02.076010+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:20.250053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.085142+0000","last_change":"2024-04-25T00:15:02.085142+0000","last_active":"2024-04-25T00:15:02.085142+0000","last_peered":"2024-04-25T00:15:02.085142+0000","last_clean":"2024-04-25T00:15:02.085142+0000","last_became_active":"2024-04-25T00:15:02.080336+0000","last_became_peered":"2024-04-25T00:15:02.080336+0000","last_unstale":"2024-04-25T00:15:02.085142+0000","last_undegraded":"2024-04-25T00:15:02.085142+0000","last_fullsized":"2024-04-25T00:15:02.085142+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:32.811681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.954566+0000","last_change":"2024-04-25T00:14:01.832342+0000","last_active":"2024-04-25T00:15:30.954566+0000","last_peered":"2024-04-25T00:15:30.954566+0000","last_clean":"2024-04-25T00:15:30.954566+0000","last_became_active":"2024-04-25T00:14:01.831997+0000","last_became_peered":"2024-04-25T00:14:01.831997+0000","last_unstale":"2024-04-25T00:15:30.954566+0000","last_undegraded":"2024-04-25T00:15:30.954566+0000","last_fullsized":"2024-04-25T00:15:30.954566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:12.335884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.075708+0000","last_change":"2024-04-25T00:14:07.323628+0000","last_active":"2024-04-25T00:15:02.075708+0000","last_peered":"2024-04-25T00:15:02.075708+0000","last_clean":"2024-04-25T00:15:02.075708+0000","last_became_active":"2024-04-25T00:14:07.322996+0000","last_became_peered":"2024-04-25T00:14:07.322996+0000","last_unstale":"2024-04-25T00:15:02.075708+0000","last_undegraded":"2024-04-25T00:15:02.075708+0000","last_fullsized":"2024-04-25T00:15:02.075708+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:32.803216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.075953+0000","last_change":"2024-04-25T00:14:01.407715+0000","last_active":"2024-04-25T00:15:02.075953+0000","last_peered":"2024-04-25T00:15:02.075953+0000","last_clean":"2024-04-25T00:15:02.075953+0000","last_became_active":"2024-04-25T00:14:01.407439+0000","last_became_peered":"2024-04-25T00:14:01.407439+0000","last_unstale":"2024-04-25T00:15:02.075953+0000","last_undegraded":"2024-04-25T00:15:02.075953+0000","last_fullsized":"2024-04-25T00:15:02.075953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:33.550379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.960664+0000","last_change":"2024-04-25T00:14:01.837733+0000","last_active":"2024-04-25T00:15:30.960664+0000","last_peered":"2024-04-25T00:15:30.960664+0000","last_clean":"2024-04-25T00:15:30.960664+0000","last_became_active":"2024-04-25T00:14:01.837395+0000","last_became_peered":"2024-04-25T00:14:01.837395+0000","last_unstale":"2024-04-25T00:15:30.960664+0000","last_undegraded":"2024-04-25T00:15:30.960664+0000","last_fullsized":"2024-04-25T00:15:30.960664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:50.186834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.370740+0000","last_change":"2024-04-25T00:14:06.848700+0000","last_active":"2024-04-25T00:15:02.370740+0000","last_peered":"2024-04-25T00:15:02.370740+0000","last_clean":"2024-04-25T00:15:02.370740+0000","last_became_active":"2024-04-25T00:14:06.848252+0000","last_became_peered":"2024-04-25T00:14:06.848252+0000","last_unstale":"2024-04-25T00:15:02.370740+0000","last_undegraded":"2024-04-25T00:15:02.370740+0000","last_fullsized":"2024-04-25T00:15:02.370740+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:07.653668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.074029+0000","last_change":"2024-04-25T00:14:01.405785+0000","last_active":"2024-04-25T00:15:02.074029+0000","last_peered":"2024-04-25T00:15:02.074029+0000","last_clean":"2024-04-25T00:15:02.074029+0000","last_became_active":"2024-04-25T00:14:01.405475+0000","last_became_peered":"2024-04-25T00:14:01.405475+0000","last_unstale":"2024-04-25T00:15:02.074029+0000","last_undegraded":"2024-04-25T00:15:02.074029+0000","last_fullsized":"2024-04-25T00:15:02.074029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:32.487795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.959655+0000","last_change":"2024-04-25T00:14:01.832393+0000","last_active":"2024-04-25T00:15:30.959655+0000","last_peered":"2024-04-25T00:15:30.959655+0000","last_clean":"2024-04-25T00:15:30.959655+0000","last_became_active":"2024-04-25T00:14:01.832096+0000","last_became_peered":"2024-04-25T00:14:01.832096+0000","last_unstale":"2024-04-25T00:15:30.959655+0000","last_undegraded":"2024-04-25T00:15:30.959655+0000","last_fullsized":"2024-04-25T00:15:30.959655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:41.749627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076472+0000","last_change":"2024-04-25T00:14:06.855224+0000","last_active":"2024-04-25T00:15:02.076472+0000","last_peered":"2024-04-25T00:15:02.076472+0000","last_clean":"2024-04-25T00:15:02.076472+0000","last_became_active":"2024-04-25T00:14:06.854847+0000","last_became_peered":"2024-04-25T00:14:06.854847+0000","last_unstale":"2024-04-25T00:15:02.076472+0000","last_undegraded":"2024-04-25T00:15:02.076472+0000","last_fullsized":"2024-04-25T00:15:02.076472+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:13.520448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.081520+0000","last_change":"2024-04-25T00:14:01.401145+0000","last_active":"2024-04-25T00:15:02.081520+0000","last_peered":"2024-04-25T00:15:02.081520+0000","last_clean":"2024-04-25T00:15:02.081520+0000","last_became_active":"2024-04-25T00:14:01.400807+0000","last_became_peered":"2024-04-25T00:14:01.400807+0000","last_unstale":"2024-04-25T00:15:02.081520+0000","last_undegraded":"2024-04-25T00:15:02.081520+0000","last_fullsized":"2024-04-25T00:15:02.081520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:52:04.674677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"94'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.958132+0000","last_change":"2024-04-25T00:14:01.838033+0000","last_active":"2024-04-25T00:15:30.958132+0000","last_peered":"2024-04-25T00:15:30.958132+0000","last_clean":"2024-04-25T00:15:30.958132+0000","last_became_active":"2024-04-25T00:14:01.837657+0000","last_became_peered":"2024-04-25T00:14:01.837657+0000","last_unstale":"2024-04-25T00:15:30.958132+0000","last_undegraded":"2024-04-25T00:15:30.958132+0000","last_fullsized":"2024-04-25T00:15:30.958132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:51.895086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.082042+0000","last_change":"2024-04-25T00:14:06.854885+0000","last_active":"2024-04-25T00:15:02.082042+0000","last_peered":"2024-04-25T00:15:02.082042+0000","last_clean":"2024-04-25T00:15:02.082042+0000","last_became_active":"2024-04-25T00:14:06.854200+0000","last_became_peered":"2024-04-25T00:14:06.854200+0000","last_unstale":"2024-04-25T00:15:02.082042+0000","last_undegraded":"2024-04-25T00:15:02.082042+0000","last_fullsized":"2024-04-25T00:15:02.082042+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:43.627742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.371379+0000","last_change":"2024-04-25T00:14:01.403789+0000","last_active":"2024-04-25T00:15:02.371379+0000","last_peered":"2024-04-25T00:15:02.371379+0000","last_clean":"2024-04-25T00:15:02.371379+0000","last_became_active":"2024-04-25T00:14:01.403464+0000","last_became_peered":"2024-04-25T00:14:01.403464+0000","last_unstale":"2024-04-25T00:15:02.371379+0000","last_undegraded":"2024-04-25T00:15:02.371379+0000","last_fullsized":"2024-04-25T00:15:02.371379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:48.276536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.950980+0000","last_change":"2024-04-25T00:14:01.839988+0000","last_active":"2024-04-25T00:15:30.950980+0000","last_peered":"2024-04-25T00:15:30.950980+0000","last_clean":"2024-04-25T00:15:30.950980+0000","last_became_active":"2024-04-25T00:14:01.839516+0000","last_became_peered":"2024-04-25T00:14:01.839516+0000","last_unstale":"2024-04-25T00:15:30.950980+0000","last_undegraded":"2024-04-25T00:15:30.950980+0000","last_fullsized":"2024-04-25T00:15:30.950980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:05.875127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.943071+0000","last_change":"2024-04-25T00:14:01.828638+0000","last_active":"2024-04-25T00:15:30.943071+0000","last_peered":"2024-04-25T00:15:30.943071+0000","last_clean":"2024-04-25T00:15:30.943071+0000","last_became_active":"2024-04-25T00:14:01.828266+0000","last_became_peered":"2024-04-25T00:14:01.828266+0000","last_unstale":"2024-04-25T00:15:30.943071+0000","last_undegraded":"2024-04-25T00:15:30.943071+0000","last_fullsized":"2024-04-25T00:15:30.943071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:34:22.619522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.947867+0000","last_change":"2024-04-25T00:14:01.828199+0000","last_active":"2024-04-25T00:15:30.947867+0000","last_peered":"2024-04-25T00:15:30.947867+0000","last_clean":"2024-04-25T00:15:30.947867+0000","last_became_active":"2024-04-25T00:14:01.827579+0000","last_became_peered":"2024-04-25T00:14:01.827579+0000","last_unstale":"2024-04-25T00:15:30.947867+0000","last_undegraded":"2024-04-25T00:15:30.947867+0000","last_fullsized":"2024-04-25T00:15:30.947867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:00.708928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.958158+0000","last_change":"2024-04-25T00:14:01.825620+0000","last_active":"2024-04-25T00:15:30.958158+0000","last_peered":"2024-04-25T00:15:30.958158+0000","last_clean":"2024-04-25T00:15:30.958158+0000","last_became_active":"2024-04-25T00:14:01.825310+0000","last_became_peered":"2024-04-25T00:14:01.825310+0000","last_unstale":"2024-04-25T00:15:30.958158+0000","last_undegraded":"2024-04-25T00:15:30.958158+0000","last_fullsized":"2024-04-25T00:15:30.958158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:35.075355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.962619+0000","last_change":"2024-04-25T00:14:01.836537+0000","last_active":"2024-04-25T00:15:30.962619+0000","last_peered":"2024-04-25T00:15:30.962619+0000","last_clean":"2024-04-25T00:15:30.962619+0000","last_became_active":"2024-04-25T00:14:01.836234+0000","last_became_peered":"2024-04-25T00:14:01.836234+0000","last_unstale":"2024-04-25T00:15:30.962619+0000","last_undegraded":"2024-04-25T00:15:30.962619+0000","last_fullsized":"2024-04-25T00:15:30.962619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:15.037498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.963606+0000","last_change":"2024-04-25T00:14:01.825937+0000","last_active":"2024-04-25T00:15:30.963606+0000","last_peered":"2024-04-25T00:15:30.963606+0000","last_clean":"2024-04-25T00:15:30.963606+0000","last_became_active":"2024-04-25T00:14:01.825605+0000","last_became_peered":"2024-04-25T00:14:01.825605+0000","last_unstale":"2024-04-25T00:15:30.963606+0000","last_undegraded":"2024-04-25T00:15:30.963606+0000","last_fullsized":"2024-04-25T00:15:30.963606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:06.918000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.940251+0000","last_change":"2024-04-25T00:14:01.830004+0000","last_active":"2024-04-25T00:15:30.940251+0000","last_peered":"2024-04-25T00:15:30.940251+0000","last_clean":"2024-04-25T00:15:30.940251+0000","last_became_active":"2024-04-25T00:14:01.829588+0000","last_became_peered":"2024-04-25T00:14:01.829588+0000","last_unstale":"2024-04-25T00:15:30.940251+0000","last_undegraded":"2024-04-25T00:15:30.940251+0000","last_fullsized":"2024-04-25T00:15:30.940251+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:08.005995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.940551+0000","last_change":"2024-04-25T00:14:01.825762+0000","last_active":"2024-04-25T00:15:30.940551+0000","last_peered":"2024-04-25T00:15:30.940551+0000","last_clean":"2024-04-25T00:15:30.940551+0000","last_became_active":"2024-04-25T00:14:01.825471+0000","last_became_peered":"2024-04-25T00:14:01.825471+0000","last_unstale":"2024-04-25T00:15:30.940551+0000","last_undegraded":"2024-04-25T00:15:30.940551+0000","last_fullsized":"2024-04-25T00:15:30.940551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:33.716875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.956340+0000","last_change":"2024-04-25T00:14:01.836103+0000","last_active":"2024-04-25T00:15:30.956340+0000","last_peered":"2024-04-25T00:15:30.956340+0000","last_clean":"2024-04-25T00:15:30.956340+0000","last_became_active":"2024-04-25T00:14:01.834591+0000","last_became_peered":"2024-04-25T00:14:01.834591+0000","last_unstale":"2024-04-25T00:15:30.956340+0000","last_undegraded":"2024-04-25T00:15:30.956340+0000","last_fullsized":"2024-04-25T00:15:30.956340+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:55.130978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.965741+0000","last_change":"2024-04-25T00:14:01.833179+0000","last_active":"2024-04-25T00:15:30.965741+0000","last_peered":"2024-04-25T00:15:30.965741+0000","last_clean":"2024-04-25T00:15:30.965741+0000","last_became_active":"2024-04-25T00:14:01.832856+0000","last_became_peered":"2024-04-25T00:14:01.832856+0000","last_unstale":"2024-04-25T00:15:30.965741+0000","last_undegraded":"2024-04-25T00:15:30.965741+0000","last_fullsized":"2024-04-25T00:15:30.965741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:07.153356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.955106+0000","last_change":"2024-04-25T00:14:01.834751+0000","last_active":"2024-04-25T00:15:30.955106+0000","last_peered":"2024-04-25T00:15:30.955106+0000","last_clean":"2024-04-25T00:15:30.955106+0000","last_became_active":"2024-04-25T00:14:01.834413+0000","last_became_peered":"2024-04-25T00:14:01.834413+0000","last_unstale":"2024-04-25T00:15:30.955106+0000","last_undegraded":"2024-04-25T00:15:30.955106+0000","last_fullsized":"2024-04-25T00:15:30.955106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:52.341565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.966591+0000","last_change":"2024-04-25T00:14:01.832944+0000","last_active":"2024-04-25T00:15:30.966591+0000","last_peered":"2024-04-25T00:15:30.966591+0000","last_clean":"2024-04-25T00:15:30.966591+0000","last_became_active":"2024-04-25T00:14:01.832394+0000","last_became_peered":"2024-04-25T00:14:01.832394+0000","last_unstale":"2024-04-25T00:15:30.966591+0000","last_undegraded":"2024-04-25T00:15:30.966591+0000","last_fullsized":"2024-04-25T00:15:30.966591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:26.957786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.960241+0000","last_change":"2024-04-25T00:14:01.831492+0000","last_active":"2024-04-25T00:15:30.960241+0000","last_peered":"2024-04-25T00:15:30.960241+0000","last_clean":"2024-04-25T00:15:30.960241+0000","last_became_active":"2024-04-25T00:14:01.829557+0000","last_became_peered":"2024-04-25T00:14:01.829557+0000","last_unstale":"2024-04-25T00:15:30.960241+0000","last_undegraded":"2024-04-25T00:15:30.960241+0000","last_fullsized":"2024-04-25T00:15:30.960241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:45.116188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.945618+0000","last_change":"2024-04-25T00:14:01.839323+0000","last_active":"2024-04-25T00:15:30.945618+0000","last_peered":"2024-04-25T00:15:30.945618+0000","last_clean":"2024-04-25T00:15:30.945618+0000","last_became_active":"2024-04-25T00:14:01.838999+0000","last_became_peered":"2024-04-25T00:14:01.838999+0000","last_unstale":"2024-04-25T00:15:30.945618+0000","last_undegraded":"2024-04-25T00:15:30.945618+0000","last_fullsized":"2024-04-25T00:15:30.945618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:20.652123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67747,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":457,"num_read_kb":302,"num_write":168,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":203241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351009,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":521389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612646,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37048,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708104,"statfs":{"total":95995035648,"available":95957098496,"internally_reserved":0,"allocated":1753088,"data_stored":1172262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776172,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":525721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037809,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":521545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299445,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1671168,"data_stored":1117595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462971,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":516801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":580316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1081344,"data_stored":518189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1150976,"data_stored":582198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149769,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1069056,"data_stored":516995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888133,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40540,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704612,"statfs":{"total":95995035648,"available":95953522688,"internally_reserved":0,"allocated":1069056,"data_stored":521451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724607,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1679360,"data_stored":1113183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":63605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:15:42.830 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T00:15:43.074 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:15:44.204 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:44 smithi107 ceph-mon[29153]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:15:44.205 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:44 smithi107 ceph-mon[29153]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T00:15:44.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:44 smithi195 ceph-mon[32833]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:15:44.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:44 smithi195 ceph-mon[32833]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T00:15:44.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:44 smithi170 ceph-mon[32807]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:15:44.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:44 smithi170 ceph-mon[32807]: pgmap v345: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T00:15:44.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.732+0000 7f8fc700a700 1 -- 172.21.15.107:0/4161829687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc0102e90 msgr2=0x7f8fc0106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:44.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.732+0000 7f8fc700a700 1 --2- 172.21.15.107:0/4161829687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc0102e90 0x7f8fc0106e90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8fac009990 tx=0x7f8fac009ca0 comp rx=0 tx=0).stop 2024-04-25T00:15:44.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.733+0000 7f8fc700a700 1 -- 172.21.15.107:0/4161829687 shutdown_connections 2024-04-25T00:15:44.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.733+0000 7f8fc700a700 1 --2- 172.21.15.107:0/4161829687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fc0107570 0x7f8fc010f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:44.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.733+0000 7f8fc700a700 1 --2- 172.21.15.107:0/4161829687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fc01024e0 0x7f8fc01028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:44.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.733+0000 7f8fc700a700 1 --2- 172.21.15.107:0/4161829687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc0102e90 0x7f8fc0106e90 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:44.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.733+0000 7f8fc700a700 1 -- 172.21.15.107:0/4161829687 >> 172.21.15.107:0/4161829687 conn(0x7f8fc0079390 msgr2=0x7f8fc00797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:44.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.733+0000 7f8fc700a700 1 -- 172.21.15.107:0/4161829687 shutdown_connections 2024-04-25T00:15:44.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.733+0000 7f8fc700a700 1 -- 172.21.15.107:0/4161829687 wait complete. 2024-04-25T00:15:44.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.733+0000 7f8fc700a700 1 Processor -- start 2024-04-25T00:15:44.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.733+0000 7f8fc700a700 1 -- start start 2024-04-25T00:15:44.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.733+0000 7f8fc700a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fc01024e0 0x7f8fc019e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:44.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.733+0000 7f8fc700a700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fc0102e90 0x7f8fc019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:44.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.733+0000 7f8fc700a700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc0107570 0x7f8fc01987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:44.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.733+0000 7f8fc700a700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc0112510 con 0x7f8fc0107570 2024-04-25T00:15:44.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.733+0000 7f8fc700a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc0112390 con 0x7f8fc01024e0 2024-04-25T00:15:44.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.733+0000 7f8fc700a700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc0112690 con 0x7f8fc0102e90 2024-04-25T00:15:44.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.733+0000 7f8fc55a7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc0107570 0x7f8fc01987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:44.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.733+0000 7f8fc55a7700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc0107570 0x7f8fc01987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38176/0 (socket says 172.21.15.107:38176) 2024-04-25T00:15:44.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.733+0000 7f8fc55a7700 1 -- 172.21.15.107:0/547244866 learned_addr learned my addr 172.21.15.107:0/547244866 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:44.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.734+0000 7f8fc4da6700 1 --2- 172.21.15.107:0/547244866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fc01024e0 0x7f8fc019e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:44.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.734+0000 7f8fbffff700 1 --2- 172.21.15.107:0/547244866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fc0102e90 0x7f8fc019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:44.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.734+0000 7f8fc55a7700 1 -- 172.21.15.107:0/547244866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fc0102e90 msgr2=0x7f8fc019eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:44.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.734+0000 7f8fc55a7700 1 --2- 172.21.15.107:0/547244866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fc0102e90 0x7f8fc019eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:44.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.734+0000 7f8fc55a7700 1 -- 172.21.15.107:0/547244866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fc01024e0 msgr2=0x7f8fc019e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:44.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.734+0000 7f8fc55a7700 1 --2- 172.21.15.107:0/547244866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fc01024e0 0x7f8fc019e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:44.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.734+0000 7f8fc55a7700 1 -- 172.21.15.107:0/547244866 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc0198f90 con 0x7f8fc0107570 2024-04-25T00:15:44.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.734+0000 7f8fc4da6700 1 --2- 172.21.15.107:0/547244866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fc01024e0 0x7f8fc019e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:44.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.734+0000 7f8fbffff700 1 --2- 172.21.15.107:0/547244866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fc0102e90 0x7f8fc019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:44.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.734+0000 7f8fc55a7700 1 --2- 172.21.15.107:0/547244866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc0107570 0x7f8fc01987b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8007d70 tx=0x7f8fb800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:44.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.734+0000 7f8fbdffb700 1 -- 172.21.15.107:0/547244866 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8016020 con 0x7f8fc0107570 2024-04-25T00:15:44.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.734+0000 7f8fbdffb700 1 -- 172.21.15.107:0/547244866 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fb800ad40 con 0x7f8fc0107570 2024-04-25T00:15:44.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.734+0000 7f8fc700a700 1 -- 172.21.15.107:0/547244866 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc0199280 con 0x7f8fc0107570 2024-04-25T00:15:44.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.734+0000 7f8fbdffb700 1 -- 172.21.15.107:0/547244866 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb80116f0 con 0x7f8fc0107570 2024-04-25T00:15:44.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.734+0000 7f8fc700a700 1 -- 172.21.15.107:0/547244866 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc01a5490 con 0x7f8fc0107570 2024-04-25T00:15:44.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.735+0000 7f8fc700a700 1 -- 172.21.15.107:0/547244866 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc0103970 con 0x7f8fc0107570 2024-04-25T00:15:44.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.736+0000 7f8fbdffb700 1 -- 172.21.15.107:0/547244866 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8fb8019020 con 0x7f8fc0107570 2024-04-25T00:15:44.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.737+0000 7f8fbdffb700 1 --2- 172.21.15.107:0/547244866 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8fb00a2c20 0x7f8fb00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:44.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.737+0000 7f8fc4da6700 1 --2- 172.21.15.107:0/547244866 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8fb00a2c20 0x7f8fb00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:44.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.737+0000 7f8fbdffb700 1 -- 172.21.15.107:0/547244866 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f8fb80cf2e0 con 0x7f8fc0107570 2024-04-25T00:15:44.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.737+0000 7f8fc4da6700 1 --2- 172.21.15.107:0/547244866 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8fb00a2c20 0x7f8fb00a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4005fd0 tx=0x7f8fb4005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:44.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.741+0000 7f8fbdffb700 1 -- 172.21.15.107:0/547244866 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8fb8020020 con 0x7f8fc0107570 2024-04-25T00:15:44.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.933+0000 7f8fc700a700 1 -- 172.21.15.107:0/547244866 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8fc0062670 con 0x7f8fb00a2c20 2024-04-25T00:15:44.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.942+0000 7f8fbdffb700 1 -- 172.21.15.107:0/547244866 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340459 (secure 0 0 0) 0x7f8fc0062670 con 0x7f8fb00a2c20 2024-04-25T00:15:44.949 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:15:44.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.946+0000 7f8fc700a700 1 -- 172.21.15.107:0/547244866 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8fb00a2c20 msgr2=0x7f8fb00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:44.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.946+0000 7f8fc700a700 1 --2- 172.21.15.107:0/547244866 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8fb00a2c20 0x7f8fb00a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4005fd0 tx=0x7f8fb4005ee0 comp rx=0 tx=0).stop 2024-04-25T00:15:44.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.946+0000 7f8fc700a700 1 -- 172.21.15.107:0/547244866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc0107570 msgr2=0x7f8fc01987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:44.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.946+0000 7f8fc700a700 1 --2- 172.21.15.107:0/547244866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc0107570 0x7f8fc01987b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8007d70 tx=0x7f8fb800a340 comp rx=0 tx=0).stop 2024-04-25T00:15:44.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.946+0000 7f8fc700a700 1 -- 172.21.15.107:0/547244866 shutdown_connections 2024-04-25T00:15:44.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.946+0000 7f8fc700a700 1 --2- 172.21.15.107:0/547244866 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8fb00a2c20 0x7f8fb00a50e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:44.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.946+0000 7f8fc700a700 1 --2- 172.21.15.107:0/547244866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc0107570 0x7f8fc01987b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:44.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.946+0000 7f8fc700a700 1 --2- 172.21.15.107:0/547244866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fc01024e0 0x7f8fc019e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:44.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.946+0000 7f8fc700a700 1 --2- 172.21.15.107:0/547244866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fc0102e90 0x7f8fc019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:44.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.946+0000 7f8fc700a700 1 -- 172.21.15.107:0/547244866 >> 172.21.15.107:0/547244866 conn(0x7f8fc0079390 msgr2=0x7f8fc0100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:44.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.946+0000 7f8fc700a700 1 -- 172.21.15.107:0/547244866 shutdown_connections 2024-04-25T00:15:44.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:44.946+0000 7f8fc700a700 1 -- 172.21.15.107:0/547244866 wait complete. 2024-04-25T00:15:44.954 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-04-25T00:15:45.623 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-04-25T00:15:42.946539+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658349,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":598,"num_read_kb":421,"num_write":351,"num_write_kb":3061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":785,"ondisk_log_size":785,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388640,"kb_used_data":14528,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553184,"statfs":{"total":1151940427776,"available":1151542460416,"internally_reserved":0,"allocated":14876672,"data_stored":8207645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":32,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003706"},"pg_stats":[{"pgid":"3.3f","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.955552+0000","last_change":"2024-04-25T00:14:01.835737+0000","last_active":"2024-04-25T00:15:30.955552+0000","last_peered":"2024-04-25T00:15:30.955552+0000","last_clean":"2024-04-25T00:15:30.955552+0000","last_became_active":"2024-04-25T00:14:01.835428+0000","last_became_peered":"2024-04-25T00:14:01.835428+0000","last_unstale":"2024-04-25T00:15:30.955552+0000","last_undegraded":"2024-04-25T00:15:30.955552+0000","last_fullsized":"2024-04-25T00:15:30.955552+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:33.974656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.958231+0000","last_change":"2024-04-25T00:14:01.837321+0000","last_active":"2024-04-25T00:15:30.958231+0000","last_peered":"2024-04-25T00:15:30.958231+0000","last_clean":"2024-04-25T00:15:30.958231+0000","last_became_active":"2024-04-25T00:14:01.836762+0000","last_became_peered":"2024-04-25T00:14:01.836762+0000","last_unstale":"2024-04-25T00:15:30.958231+0000","last_undegraded":"2024-04-25T00:15:30.958231+0000","last_fullsized":"2024-04-25T00:15:30.958231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:34:17.342967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.960471+0000","last_change":"2024-04-25T00:14:01.834504+0000","last_active":"2024-04-25T00:15:30.960471+0000","last_peered":"2024-04-25T00:15:30.960471+0000","last_clean":"2024-04-25T00:15:30.960471+0000","last_became_active":"2024-04-25T00:14:01.834182+0000","last_became_peered":"2024-04-25T00:14:01.834182+0000","last_unstale":"2024-04-25T00:15:30.960471+0000","last_undegraded":"2024-04-25T00:15:30.960471+0000","last_fullsized":"2024-04-25T00:15:30.960471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:12.991170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.949289+0000","last_change":"2024-04-25T00:15:02.080211+0000","last_active":"2024-04-25T00:15:30.949289+0000","last_peered":"2024-04-25T00:15:30.949289+0000","last_clean":"2024-04-25T00:15:30.949289+0000","last_became_active":"2024-04-25T00:15:02.079849+0000","last_became_peered":"2024-04-25T00:15:02.079849+0000","last_unstale":"2024-04-25T00:15:30.949289+0000","last_undegraded":"2024-04-25T00:15:30.949289+0000","last_fullsized":"2024-04-25T00:15:30.949289+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:56.804782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.965280+0000","last_change":"2024-04-25T00:14:01.834173+0000","last_active":"2024-04-25T00:15:30.965280+0000","last_peered":"2024-04-25T00:15:30.965280+0000","last_clean":"2024-04-25T00:15:30.965280+0000","last_became_active":"2024-04-25T00:14:01.833835+0000","last_became_peered":"2024-04-25T00:14:01.833835+0000","last_unstale":"2024-04-25T00:15:30.965280+0000","last_undegraded":"2024-04-25T00:15:30.965280+0000","last_fullsized":"2024-04-25T00:15:30.965280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:30.046629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.950213+0000","last_change":"2024-04-25T00:14:01.832113+0000","last_active":"2024-04-25T00:15:30.950213+0000","last_peered":"2024-04-25T00:15:30.950213+0000","last_clean":"2024-04-25T00:15:30.950213+0000","last_became_active":"2024-04-25T00:14:01.831800+0000","last_became_peered":"2024-04-25T00:14:01.831800+0000","last_unstale":"2024-04-25T00:15:30.950213+0000","last_undegraded":"2024-04-25T00:15:30.950213+0000","last_fullsized":"2024-04-25T00:15:30.950213+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:54.469994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.954336+0000","last_change":"2024-04-25T00:14:01.839166+0000","last_active":"2024-04-25T00:15:30.954336+0000","last_peered":"2024-04-25T00:15:30.954336+0000","last_clean":"2024-04-25T00:15:30.954336+0000","last_became_active":"2024-04-25T00:14:01.838851+0000","last_became_peered":"2024-04-25T00:14:01.838851+0000","last_unstale":"2024-04-25T00:15:30.954336+0000","last_undegraded":"2024-04-25T00:15:30.954336+0000","last_fullsized":"2024-04-25T00:15:30.954336+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:39:48.125363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.954451+0000","last_change":"2024-04-25T00:15:02.083386+0000","last_active":"2024-04-25T00:15:30.954451+0000","last_peered":"2024-04-25T00:15:30.954451+0000","last_clean":"2024-04-25T00:15:30.954451+0000","last_became_active":"2024-04-25T00:15:02.083106+0000","last_became_peered":"2024-04-25T00:15:02.083106+0000","last_unstale":"2024-04-25T00:15:30.954451+0000","last_undegraded":"2024-04-25T00:15:30.954451+0000","last_fullsized":"2024-04-25T00:15:30.954451+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:58.543667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.959137+0000","last_change":"2024-04-25T00:14:01.829803+0000","last_active":"2024-04-25T00:15:30.959137+0000","last_peered":"2024-04-25T00:15:30.959137+0000","last_clean":"2024-04-25T00:15:30.959137+0000","last_became_active":"2024-04-25T00:14:01.827761+0000","last_became_peered":"2024-04-25T00:14:01.827761+0000","last_unstale":"2024-04-25T00:15:30.959137+0000","last_undegraded":"2024-04-25T00:15:30.959137+0000","last_fullsized":"2024-04-25T00:15:30.959137+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:38.862505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.948779+0000","last_change":"2024-04-25T00:14:01.834269+0000","last_active":"2024-04-25T00:15:30.948779+0000","last_peered":"2024-04-25T00:15:30.948779+0000","last_clean":"2024-04-25T00:15:30.948779+0000","last_became_active":"2024-04-25T00:14:01.833964+0000","last_became_peered":"2024-04-25T00:14:01.833964+0000","last_unstale":"2024-04-25T00:15:30.948779+0000","last_undegraded":"2024-04-25T00:15:30.948779+0000","last_fullsized":"2024-04-25T00:15:30.948779+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:29.501513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.959542+0000","last_change":"2024-04-25T00:14:01.839510+0000","last_active":"2024-04-25T00:15:30.959542+0000","last_peered":"2024-04-25T00:15:30.959542+0000","last_clean":"2024-04-25T00:15:30.959542+0000","last_became_active":"2024-04-25T00:14:01.839000+0000","last_became_peered":"2024-04-25T00:14:01.839000+0000","last_unstale":"2024-04-25T00:15:30.959542+0000","last_undegraded":"2024-04-25T00:15:30.959542+0000","last_fullsized":"2024-04-25T00:15:30.959542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:20.809277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.933166+0000","last_change":"2024-04-25T00:14:01.835821+0000","last_active":"2024-04-25T00:15:30.933166+0000","last_peered":"2024-04-25T00:15:30.933166+0000","last_clean":"2024-04-25T00:15:30.933166+0000","last_became_active":"2024-04-25T00:14:01.835525+0000","last_became_peered":"2024-04-25T00:14:01.835525+0000","last_unstale":"2024-04-25T00:15:30.933166+0000","last_undegraded":"2024-04-25T00:15:30.933166+0000","last_fullsized":"2024-04-25T00:15:30.933166+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:17.082562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.960479+0000","last_change":"2024-04-25T00:14:01.831564+0000","last_active":"2024-04-25T00:15:30.960479+0000","last_peered":"2024-04-25T00:15:30.960479+0000","last_clean":"2024-04-25T00:15:30.960479+0000","last_became_active":"2024-04-25T00:14:01.828539+0000","last_became_peered":"2024-04-25T00:14:01.828539+0000","last_unstale":"2024-04-25T00:15:30.960479+0000","last_undegraded":"2024-04-25T00:15:30.960479+0000","last_fullsized":"2024-04-25T00:15:30.960479+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:41:52.098299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.960792+0000","last_change":"2024-04-25T00:14:01.835815+0000","last_active":"2024-04-25T00:15:30.960792+0000","last_peered":"2024-04-25T00:15:30.960792+0000","last_clean":"2024-04-25T00:15:30.960792+0000","last_became_active":"2024-04-25T00:14:01.835498+0000","last_became_peered":"2024-04-25T00:14:01.835498+0000","last_unstale":"2024-04-25T00:15:30.960792+0000","last_undegraded":"2024-04-25T00:15:30.960792+0000","last_fullsized":"2024-04-25T00:15:30.960792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:14:02.914149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.962247+0000","last_change":"2024-04-25T00:14:01.839542+0000","last_active":"2024-04-25T00:15:30.962247+0000","last_peered":"2024-04-25T00:15:30.962247+0000","last_clean":"2024-04-25T00:15:30.962247+0000","last_became_active":"2024-04-25T00:14:01.839234+0000","last_became_peered":"2024-04-25T00:14:01.839234+0000","last_unstale":"2024-04-25T00:15:30.962247+0000","last_undegraded":"2024-04-25T00:15:30.962247+0000","last_fullsized":"2024-04-25T00:15:30.962247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:00.437311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.958957+0000","last_change":"2024-04-25T00:14:01.832265+0000","last_active":"2024-04-25T00:15:30.958957+0000","last_peered":"2024-04-25T00:15:30.958957+0000","last_clean":"2024-04-25T00:15:30.958957+0000","last_became_active":"2024-04-25T00:14:01.831963+0000","last_became_peered":"2024-04-25T00:14:01.831963+0000","last_unstale":"2024-04-25T00:15:30.958957+0000","last_undegraded":"2024-04-25T00:15:30.958957+0000","last_fullsized":"2024-04-25T00:15:30.958957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:55.402265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.955690+0000","last_change":"2024-04-25T00:14:01.835444+0000","last_active":"2024-04-25T00:15:30.955690+0000","last_peered":"2024-04-25T00:15:30.955690+0000","last_clean":"2024-04-25T00:15:30.955690+0000","last_became_active":"2024-04-25T00:14:01.835126+0000","last_became_peered":"2024-04-25T00:14:01.835126+0000","last_unstale":"2024-04-25T00:15:30.955690+0000","last_undegraded":"2024-04-25T00:15:30.955690+0000","last_fullsized":"2024-04-25T00:15:30.955690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:48.997924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.955107+0000","last_change":"2024-04-25T00:14:01.834982+0000","last_active":"2024-04-25T00:15:30.955107+0000","last_peered":"2024-04-25T00:15:30.955107+0000","last_clean":"2024-04-25T00:15:30.955107+0000","last_became_active":"2024-04-25T00:14:01.834671+0000","last_became_peered":"2024-04-25T00:14:01.834671+0000","last_unstale":"2024-04-25T00:15:30.955107+0000","last_undegraded":"2024-04-25T00:15:30.955107+0000","last_fullsized":"2024-04-25T00:15:30.955107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:28.254408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.946674+0000","last_change":"2024-04-25T00:14:01.824432+0000","last_active":"2024-04-25T00:15:30.946674+0000","last_peered":"2024-04-25T00:15:30.946674+0000","last_clean":"2024-04-25T00:15:30.946674+0000","last_became_active":"2024-04-25T00:14:01.823963+0000","last_became_peered":"2024-04-25T00:14:01.823963+0000","last_unstale":"2024-04-25T00:15:30.946674+0000","last_undegraded":"2024-04-25T00:15:30.946674+0000","last_fullsized":"2024-04-25T00:15:30.946674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:35.880932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.073688+0000","last_change":"2024-04-25T00:14:06.861892+0000","last_active":"2024-04-25T00:15:02.073688+0000","last_peered":"2024-04-25T00:15:02.073688+0000","last_clean":"2024-04-25T00:15:02.073688+0000","last_became_active":"2024-04-25T00:14:06.861470+0000","last_became_peered":"2024-04-25T00:14:06.861470+0000","last_unstale":"2024-04-25T00:15:02.073688+0000","last_undegraded":"2024-04-25T00:15:02.073688+0000","last_fullsized":"2024-04-25T00:15:02.073688+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:06.192954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076691+0000","last_change":"2024-04-25T00:14:01.412535+0000","last_active":"2024-04-25T00:15:02.076691+0000","last_peered":"2024-04-25T00:15:02.076691+0000","last_clean":"2024-04-25T00:15:02.076691+0000","last_became_active":"2024-04-25T00:14:01.412206+0000","last_became_peered":"2024-04-25T00:14:01.412206+0000","last_unstale":"2024-04-25T00:15:02.076691+0000","last_undegraded":"2024-04-25T00:15:02.076691+0000","last_fullsized":"2024-04-25T00:15:02.076691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:40.893987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.963511+0000","last_change":"2024-04-25T00:14:01.840814+0000","last_active":"2024-04-25T00:15:30.963511+0000","last_peered":"2024-04-25T00:15:30.963511+0000","last_clean":"2024-04-25T00:15:30.963511+0000","last_became_active":"2024-04-25T00:14:01.840179+0000","last_became_peered":"2024-04-25T00:14:01.840179+0000","last_unstale":"2024-04-25T00:15:30.963511+0000","last_undegraded":"2024-04-25T00:15:30.963511+0000","last_fullsized":"2024-04-25T00:15:30.963511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:27.383918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.075532+0000","last_change":"2024-04-25T00:14:06.857877+0000","last_active":"2024-04-25T00:15:02.075532+0000","last_peered":"2024-04-25T00:15:02.075532+0000","last_clean":"2024-04-25T00:15:02.075532+0000","last_became_active":"2024-04-25T00:14:06.857420+0000","last_became_peered":"2024-04-25T00:14:06.857420+0000","last_unstale":"2024-04-25T00:15:02.075532+0000","last_undegraded":"2024-04-25T00:15:02.075532+0000","last_fullsized":"2024-04-25T00:15:02.075532+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:54.424024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.074359+0000","last_change":"2024-04-25T00:14:01.416560+0000","last_active":"2024-04-25T00:15:02.074359+0000","last_peered":"2024-04-25T00:15:02.074359+0000","last_clean":"2024-04-25T00:15:02.074359+0000","last_became_active":"2024-04-25T00:14:01.416246+0000","last_became_peered":"2024-04-25T00:14:01.416246+0000","last_unstale":"2024-04-25T00:15:02.074359+0000","last_undegraded":"2024-04-25T00:15:02.074359+0000","last_fullsized":"2024-04-25T00:15:02.074359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:03:47.938452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.960439+0000","last_change":"2024-04-25T00:14:01.832607+0000","last_active":"2024-04-25T00:15:30.960439+0000","last_peered":"2024-04-25T00:15:30.960439+0000","last_clean":"2024-04-25T00:15:30.960439+0000","last_became_active":"2024-04-25T00:14:01.832300+0000","last_became_peered":"2024-04-25T00:14:01.832300+0000","last_unstale":"2024-04-25T00:15:30.960439+0000","last_undegraded":"2024-04-25T00:15:30.960439+0000","last_fullsized":"2024-04-25T00:15:30.960439+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:22.045171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.075719+0000","last_change":"2024-04-25T00:14:06.863642+0000","last_active":"2024-04-25T00:15:02.075719+0000","last_peered":"2024-04-25T00:15:02.075719+0000","last_clean":"2024-04-25T00:15:02.075719+0000","last_became_active":"2024-04-25T00:14:06.863284+0000","last_became_peered":"2024-04-25T00:14:06.863284+0000","last_unstale":"2024-04-25T00:15:02.075719+0000","last_undegraded":"2024-04-25T00:15:02.075719+0000","last_fullsized":"2024-04-25T00:15:02.075719+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:20.818625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076385+0000","last_change":"2024-04-25T00:14:01.411874+0000","last_active":"2024-04-25T00:15:02.076385+0000","last_peered":"2024-04-25T00:15:02.076385+0000","last_clean":"2024-04-25T00:15:02.076385+0000","last_became_active":"2024-04-25T00:14:01.409568+0000","last_became_peered":"2024-04-25T00:14:01.409568+0000","last_unstale":"2024-04-25T00:15:02.076385+0000","last_undegraded":"2024-04-25T00:15:02.076385+0000","last_fullsized":"2024-04-25T00:15:02.076385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:22.415631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.952225+0000","last_change":"2024-04-25T00:14:01.822471+0000","last_active":"2024-04-25T00:15:30.952225+0000","last_peered":"2024-04-25T00:15:30.952225+0000","last_clean":"2024-04-25T00:15:30.952225+0000","last_became_active":"2024-04-25T00:14:01.822181+0000","last_became_peered":"2024-04-25T00:14:01.822181+0000","last_unstale":"2024-04-25T00:15:30.952225+0000","last_undegraded":"2024-04-25T00:15:30.952225+0000","last_fullsized":"2024-04-25T00:15:30.952225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:44.857036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.075572+0000","last_change":"2024-04-25T00:14:06.856987+0000","last_active":"2024-04-25T00:15:02.075572+0000","last_peered":"2024-04-25T00:15:02.075572+0000","last_clean":"2024-04-25T00:15:02.075572+0000","last_became_active":"2024-04-25T00:14:06.856053+0000","last_became_peered":"2024-04-25T00:14:06.856053+0000","last_unstale":"2024-04-25T00:15:02.075572+0000","last_undegraded":"2024-04-25T00:15:02.075572+0000","last_fullsized":"2024-04-25T00:15:02.075572+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:16.829969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.077677+0000","last_change":"2024-04-25T00:14:01.410156+0000","last_active":"2024-04-25T00:15:02.077677+0000","last_peered":"2024-04-25T00:15:02.077677+0000","last_clean":"2024-04-25T00:15:02.077677+0000","last_became_active":"2024-04-25T00:14:01.409809+0000","last_became_peered":"2024-04-25T00:14:01.409809+0000","last_unstale":"2024-04-25T00:15:02.077677+0000","last_undegraded":"2024-04-25T00:15:02.077677+0000","last_fullsized":"2024-04-25T00:15:02.077677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:43.160419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.955674+0000","last_change":"2024-04-25T00:14:01.836090+0000","last_active":"2024-04-25T00:15:30.955674+0000","last_peered":"2024-04-25T00:15:30.955674+0000","last_clean":"2024-04-25T00:15:30.955674+0000","last_became_active":"2024-04-25T00:14:01.835733+0000","last_became_peered":"2024-04-25T00:14:01.835733+0000","last_unstale":"2024-04-25T00:15:30.955674+0000","last_undegraded":"2024-04-25T00:15:30.955674+0000","last_fullsized":"2024-04-25T00:15:30.955674+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:02.368893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.074149+0000","last_change":"2024-04-25T00:14:06.851496+0000","last_active":"2024-04-25T00:15:02.074149+0000","last_peered":"2024-04-25T00:15:02.074149+0000","last_clean":"2024-04-25T00:15:02.074149+0000","last_became_active":"2024-04-25T00:14:06.851007+0000","last_became_peered":"2024-04-25T00:14:06.851007+0000","last_unstale":"2024-04-25T00:15:02.074149+0000","last_undegraded":"2024-04-25T00:15:02.074149+0000","last_fullsized":"2024-04-25T00:15:02.074149+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:50.543348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'7","reported_seq":117,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:41.651705+0000","last_change":"2024-04-25T00:14:01.414760+0000","last_active":"2024-04-25T00:15:41.651705+0000","last_peered":"2024-04-25T00:15:41.651705+0000","last_clean":"2024-04-25T00:15:41.651705+0000","last_became_active":"2024-04-25T00:14:01.414482+0000","last_became_peered":"2024-04-25T00:14:01.414482+0000","last_unstale":"2024-04-25T00:15:41.651705+0000","last_undegraded":"2024-04-25T00:15:41.651705+0000","last_fullsized":"2024-04-25T00:15:41.651705+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:32.631933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.930160+0000","last_change":"2024-04-25T00:14:01.832516+0000","last_active":"2024-04-25T00:15:30.930160+0000","last_peered":"2024-04-25T00:15:30.930160+0000","last_clean":"2024-04-25T00:15:30.930160+0000","last_became_active":"2024-04-25T00:14:01.832106+0000","last_became_peered":"2024-04-25T00:14:01.832106+0000","last_unstale":"2024-04-25T00:15:30.930160+0000","last_undegraded":"2024-04-25T00:15:30.930160+0000","last_fullsized":"2024-04-25T00:15:30.930160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:55.590486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.073810+0000","last_change":"2024-04-25T00:14:06.861984+0000","last_active":"2024-04-25T00:15:02.073810+0000","last_peered":"2024-04-25T00:15:02.073810+0000","last_clean":"2024-04-25T00:15:02.073810+0000","last_became_active":"2024-04-25T00:14:06.861418+0000","last_became_peered":"2024-04-25T00:14:06.861418+0000","last_unstale":"2024-04-25T00:15:02.073810+0000","last_undegraded":"2024-04-25T00:15:02.073810+0000","last_fullsized":"2024-04-25T00:15:02.073810+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:27.559704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.073710+0000","last_change":"2024-04-25T00:14:01.407648+0000","last_active":"2024-04-25T00:15:02.073710+0000","last_peered":"2024-04-25T00:15:02.073710+0000","last_clean":"2024-04-25T00:15:02.073710+0000","last_became_active":"2024-04-25T00:14:01.407318+0000","last_became_peered":"2024-04-25T00:14:01.407318+0000","last_unstale":"2024-04-25T00:15:02.073710+0000","last_undegraded":"2024-04-25T00:15:02.073710+0000","last_fullsized":"2024-04-25T00:15:02.073710+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:08:10.942814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.965829+0000","last_change":"2024-04-25T00:14:01.833073+0000","last_active":"2024-04-25T00:15:30.965829+0000","last_peered":"2024-04-25T00:15:30.965829+0000","last_clean":"2024-04-25T00:15:30.965829+0000","last_became_active":"2024-04-25T00:14:01.832545+0000","last_became_peered":"2024-04-25T00:14:01.832545+0000","last_unstale":"2024-04-25T00:15:30.965829+0000","last_undegraded":"2024-04-25T00:15:30.965829+0000","last_fullsized":"2024-04-25T00:15:30.965829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:39.772781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076033+0000","last_change":"2024-04-25T00:14:06.862011+0000","last_active":"2024-04-25T00:15:02.076033+0000","last_peered":"2024-04-25T00:15:02.076033+0000","last_clean":"2024-04-25T00:15:02.076033+0000","last_became_active":"2024-04-25T00:14:06.861544+0000","last_became_peered":"2024-04-25T00:14:06.861544+0000","last_unstale":"2024-04-25T00:15:02.076033+0000","last_undegraded":"2024-04-25T00:15:02.076033+0000","last_fullsized":"2024-04-25T00:15:02.076033+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:37.736266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.370974+0000","last_change":"2024-04-25T00:14:01.397747+0000","last_active":"2024-04-25T00:15:02.370974+0000","last_peered":"2024-04-25T00:15:02.370974+0000","last_clean":"2024-04-25T00:15:02.370974+0000","last_became_active":"2024-04-25T00:14:01.397446+0000","last_became_peered":"2024-04-25T00:14:01.397446+0000","last_unstale":"2024-04-25T00:15:02.370974+0000","last_undegraded":"2024-04-25T00:15:02.370974+0000","last_fullsized":"2024-04-25T00:15:02.370974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:27.240034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.964056+0000","last_change":"2024-04-25T00:14:01.833128+0000","last_active":"2024-04-25T00:15:30.964056+0000","last_peered":"2024-04-25T00:15:30.964056+0000","last_clean":"2024-04-25T00:15:30.964056+0000","last_became_active":"2024-04-25T00:14:01.832780+0000","last_became_peered":"2024-04-25T00:14:01.832780+0000","last_unstale":"2024-04-25T00:15:30.964056+0000","last_undegraded":"2024-04-25T00:15:30.964056+0000","last_fullsized":"2024-04-25T00:15:30.964056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.023737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076514+0000","last_change":"2024-04-25T00:14:06.860436+0000","last_active":"2024-04-25T00:15:02.076514+0000","last_peered":"2024-04-25T00:15:02.076514+0000","last_clean":"2024-04-25T00:15:02.076514+0000","last_became_active":"2024-04-25T00:14:06.860047+0000","last_became_peered":"2024-04-25T00:14:06.860047+0000","last_unstale":"2024-04-25T00:15:02.076514+0000","last_undegraded":"2024-04-25T00:15:02.076514+0000","last_fullsized":"2024-04-25T00:15:02.076514+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:08:41.246549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.082288+0000","last_change":"2024-04-25T00:14:01.415939+0000","last_active":"2024-04-25T00:15:02.082288+0000","last_peered":"2024-04-25T00:15:02.082288+0000","last_clean":"2024-04-25T00:15:02.082288+0000","last_became_active":"2024-04-25T00:14:01.415513+0000","last_became_peered":"2024-04-25T00:14:01.415513+0000","last_unstale":"2024-04-25T00:15:02.082288+0000","last_undegraded":"2024-04-25T00:15:02.082288+0000","last_fullsized":"2024-04-25T00:15:02.082288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:57:23.588868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.955256+0000","last_change":"2024-04-25T00:14:01.838793+0000","last_active":"2024-04-25T00:15:30.955256+0000","last_peered":"2024-04-25T00:15:30.955256+0000","last_clean":"2024-04-25T00:15:30.955256+0000","last_became_active":"2024-04-25T00:14:01.838453+0000","last_became_peered":"2024-04-25T00:14:01.838453+0000","last_unstale":"2024-04-25T00:15:30.955256+0000","last_undegraded":"2024-04-25T00:15:30.955256+0000","last_fullsized":"2024-04-25T00:15:30.955256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:18.674009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.073683+0000","last_change":"2024-04-25T00:14:06.856431+0000","last_active":"2024-04-25T00:15:02.073683+0000","last_peered":"2024-04-25T00:15:02.073683+0000","last_clean":"2024-04-25T00:15:02.073683+0000","last_became_active":"2024-04-25T00:14:06.856047+0000","last_became_peered":"2024-04-25T00:14:06.856047+0000","last_unstale":"2024-04-25T00:15:02.073683+0000","last_undegraded":"2024-04-25T00:15:02.073683+0000","last_fullsized":"2024-04-25T00:15:02.073683+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:20.646673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":107,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:38.646457+0000","last_change":"2024-04-25T00:14:01.399054+0000","last_active":"2024-04-25T00:15:38.646457+0000","last_peered":"2024-04-25T00:15:38.646457+0000","last_clean":"2024-04-25T00:15:38.646457+0000","last_became_active":"2024-04-25T00:14:01.398711+0000","last_became_peered":"2024-04-25T00:14:01.398711+0000","last_unstale":"2024-04-25T00:15:38.646457+0000","last_undegraded":"2024-04-25T00:15:38.646457+0000","last_fullsized":"2024-04-25T00:15:38.646457+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:26.434014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.957701+0000","last_change":"2024-04-25T00:14:01.829132+0000","last_active":"2024-04-25T00:15:30.957701+0000","last_peered":"2024-04-25T00:15:30.957701+0000","last_clean":"2024-04-25T00:15:30.957701+0000","last_became_active":"2024-04-25T00:14:01.828446+0000","last_became_peered":"2024-04-25T00:14:01.828446+0000","last_unstale":"2024-04-25T00:15:30.957701+0000","last_undegraded":"2024-04-25T00:15:30.957701+0000","last_fullsized":"2024-04-25T00:15:30.957701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:35.788478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076380+0000","last_change":"2024-04-25T00:14:06.861884+0000","last_active":"2024-04-25T00:15:02.076380+0000","last_peered":"2024-04-25T00:15:02.076380+0000","last_clean":"2024-04-25T00:15:02.076380+0000","last_became_active":"2024-04-25T00:14:06.861413+0000","last_became_peered":"2024-04-25T00:14:06.861413+0000","last_unstale":"2024-04-25T00:15:02.076380+0000","last_undegraded":"2024-04-25T00:15:02.076380+0000","last_fullsized":"2024-04-25T00:15:02.076380+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:53.876308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.371309+0000","last_change":"2024-04-25T00:14:01.410372+0000","last_active":"2024-04-25T00:15:02.371309+0000","last_peered":"2024-04-25T00:15:02.371309+0000","last_clean":"2024-04-25T00:15:02.371309+0000","last_became_active":"2024-04-25T00:14:01.410074+0000","last_became_peered":"2024-04-25T00:14:01.410074+0000","last_unstale":"2024-04-25T00:15:02.371309+0000","last_undegraded":"2024-04-25T00:15:02.371309+0000","last_fullsized":"2024-04-25T00:15:02.371309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:00.964580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.962909+0000","last_change":"2024-04-25T00:15:02.372269+0000","last_active":"2024-04-25T00:15:30.962909+0000","last_peered":"2024-04-25T00:15:30.962909+0000","last_clean":"2024-04-25T00:15:30.962909+0000","last_became_active":"2024-04-25T00:15:02.371962+0000","last_became_peered":"2024-04-25T00:15:02.371962+0000","last_unstale":"2024-04-25T00:15:30.962909+0000","last_undegraded":"2024-04-25T00:15:30.962909+0000","last_fullsized":"2024-04-25T00:15:30.962909+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:46.597638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.372941+0000","last_change":"2024-04-25T00:15:02.372941+0000","last_active":"2024-04-25T00:15:02.372941+0000","last_peered":"2024-04-25T00:15:02.372941+0000","last_clean":"2024-04-25T00:15:02.372941+0000","last_became_active":"2024-04-25T00:15:02.372504+0000","last_became_peered":"2024-04-25T00:15:02.372504+0000","last_unstale":"2024-04-25T00:15:02.372941+0000","last_undegraded":"2024-04-25T00:15:02.372941+0000","last_fullsized":"2024-04-25T00:15:02.372941+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:58.202708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.081403+0000","last_change":"2024-04-25T00:14:01.401128+0000","last_active":"2024-04-25T00:15:02.081403+0000","last_peered":"2024-04-25T00:15:02.081403+0000","last_clean":"2024-04-25T00:15:02.081403+0000","last_became_active":"2024-04-25T00:14:01.400125+0000","last_became_peered":"2024-04-25T00:14:01.400125+0000","last_unstale":"2024-04-25T00:15:02.081403+0000","last_undegraded":"2024-04-25T00:15:02.081403+0000","last_fullsized":"2024-04-25T00:15:02.081403+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:38:43.787855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.960159+0000","last_change":"2024-04-25T00:14:01.830649+0000","last_active":"2024-04-25T00:15:30.960159+0000","last_peered":"2024-04-25T00:15:30.960159+0000","last_clean":"2024-04-25T00:15:30.960159+0000","last_became_active":"2024-04-25T00:14:01.830253+0000","last_became_peered":"2024-04-25T00:14:01.830253+0000","last_unstale":"2024-04-25T00:15:30.960159+0000","last_undegraded":"2024-04-25T00:15:30.960159+0000","last_fullsized":"2024-04-25T00:15:30.960159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:37.474833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.075600+0000","last_change":"2024-04-25T00:14:06.861275+0000","last_active":"2024-04-25T00:15:02.075600+0000","last_peered":"2024-04-25T00:15:02.075600+0000","last_clean":"2024-04-25T00:15:02.075600+0000","last_became_active":"2024-04-25T00:14:06.860891+0000","last_became_peered":"2024-04-25T00:14:06.860891+0000","last_unstale":"2024-04-25T00:15:02.075600+0000","last_undegraded":"2024-04-25T00:15:02.075600+0000","last_fullsized":"2024-04-25T00:15:02.075600+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:38.685837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'36","reported_seq":220,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:39.653642+0000","last_change":"2024-04-25T00:14:01.404706+0000","last_active":"2024-04-25T00:15:39.653642+0000","last_peered":"2024-04-25T00:15:39.653642+0000","last_clean":"2024-04-25T00:15:39.653642+0000","last_became_active":"2024-04-25T00:14:01.404231+0000","last_became_peered":"2024-04-25T00:14:01.404231+0000","last_unstale":"2024-04-25T00:15:39.653642+0000","last_undegraded":"2024-04-25T00:15:39.653642+0000","last_fullsized":"2024-04-25T00:15:39.653642+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:36.437491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63605,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":153,"num_read_kb":139,"num_write":37,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.941762+0000","last_change":"2024-04-25T00:14:01.828312+0000","last_active":"2024-04-25T00:15:30.941762+0000","last_peered":"2024-04-25T00:15:30.941762+0000","last_clean":"2024-04-25T00:15:30.941762+0000","last_became_active":"2024-04-25T00:14:01.827947+0000","last_became_peered":"2024-04-25T00:14:01.827947+0000","last_unstale":"2024-04-25T00:15:30.941762+0000","last_undegraded":"2024-04-25T00:15:30.941762+0000","last_fullsized":"2024-04-25T00:15:30.941762+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.782796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.081614+0000","last_change":"2024-04-25T00:14:06.854379+0000","last_active":"2024-04-25T00:15:02.081614+0000","last_peered":"2024-04-25T00:15:02.081614+0000","last_clean":"2024-04-25T00:15:02.081614+0000","last_became_active":"2024-04-25T00:14:06.853307+0000","last_became_peered":"2024-04-25T00:14:06.853307+0000","last_unstale":"2024-04-25T00:15:02.081614+0000","last_undegraded":"2024-04-25T00:15:02.081614+0000","last_fullsized":"2024-04-25T00:15:02.081614+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:24:11.804189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.074161+0000","last_change":"2024-04-25T00:14:01.410395+0000","last_active":"2024-04-25T00:15:02.074161+0000","last_peered":"2024-04-25T00:15:02.074161+0000","last_clean":"2024-04-25T00:15:02.074161+0000","last_became_active":"2024-04-25T00:14:01.410039+0000","last_became_peered":"2024-04-25T00:14:01.410039+0000","last_unstale":"2024-04-25T00:15:02.074161+0000","last_undegraded":"2024-04-25T00:15:02.074161+0000","last_fullsized":"2024-04-25T00:15:02.074161+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:29.496519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.959939+0000","last_change":"2024-04-25T00:14:01.828403+0000","last_active":"2024-04-25T00:15:30.959939+0000","last_peered":"2024-04-25T00:15:30.959939+0000","last_clean":"2024-04-25T00:15:30.959939+0000","last_became_active":"2024-04-25T00:14:01.828107+0000","last_became_peered":"2024-04-25T00:14:01.828107+0000","last_unstale":"2024-04-25T00:15:30.959939+0000","last_undegraded":"2024-04-25T00:15:30.959939+0000","last_fullsized":"2024-04-25T00:15:30.959939+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:29.417639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.374105+0000","last_change":"2024-04-25T00:15:02.374105+0000","last_active":"2024-04-25T00:15:02.374105+0000","last_peered":"2024-04-25T00:15:02.374105+0000","last_clean":"2024-04-25T00:15:02.374105+0000","last_became_active":"2024-04-25T00:15:02.373628+0000","last_became_peered":"2024-04-25T00:15:02.373628+0000","last_unstale":"2024-04-25T00:15:02.374105+0000","last_undegraded":"2024-04-25T00:15:02.374105+0000","last_fullsized":"2024-04-25T00:15:02.374105+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:05.542358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.080742+0000","last_change":"2024-04-25T00:14:01.414406+0000","last_active":"2024-04-25T00:15:02.080742+0000","last_peered":"2024-04-25T00:15:02.080742+0000","last_clean":"2024-04-25T00:15:02.080742+0000","last_became_active":"2024-04-25T00:14:01.414085+0000","last_became_peered":"2024-04-25T00:14:01.414085+0000","last_unstale":"2024-04-25T00:15:02.080742+0000","last_undegraded":"2024-04-25T00:15:02.080742+0000","last_fullsized":"2024-04-25T00:15:02.080742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:22.565371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.961060+0000","last_change":"2024-04-25T00:14:01.833652+0000","last_active":"2024-04-25T00:15:30.961060+0000","last_peered":"2024-04-25T00:15:30.961060+0000","last_clean":"2024-04-25T00:15:30.961060+0000","last_became_active":"2024-04-25T00:14:01.833307+0000","last_became_peered":"2024-04-25T00:14:01.833307+0000","last_unstale":"2024-04-25T00:15:30.961060+0000","last_undegraded":"2024-04-25T00:15:30.961060+0000","last_fullsized":"2024-04-25T00:15:30.961060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:55:41.261994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.075757+0000","last_change":"2024-04-25T00:14:06.860167+0000","last_active":"2024-04-25T00:15:02.075757+0000","last_peered":"2024-04-25T00:15:02.075757+0000","last_clean":"2024-04-25T00:15:02.075757+0000","last_became_active":"2024-04-25T00:14:06.858651+0000","last_became_peered":"2024-04-25T00:14:06.858651+0000","last_unstale":"2024-04-25T00:15:02.075757+0000","last_undegraded":"2024-04-25T00:15:02.075757+0000","last_fullsized":"2024-04-25T00:15:02.075757+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:11.775434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"92'3","reported_seq":107,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:37.643220+0000","last_change":"2024-04-25T00:14:01.406267+0000","last_active":"2024-04-25T00:15:37.643220+0000","last_peered":"2024-04-25T00:15:37.643220+0000","last_clean":"2024-04-25T00:15:37.643220+0000","last_became_active":"2024-04-25T00:14:01.405985+0000","last_became_peered":"2024-04-25T00:14:01.405985+0000","last_unstale":"2024-04-25T00:15:37.643220+0000","last_undegraded":"2024-04-25T00:15:37.643220+0000","last_fullsized":"2024-04-25T00:15:37.643220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:17.253805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.952457+0000","last_change":"2024-04-25T00:14:01.838019+0000","last_active":"2024-04-25T00:15:30.952457+0000","last_peered":"2024-04-25T00:15:30.952457+0000","last_clean":"2024-04-25T00:15:30.952457+0000","last_became_active":"2024-04-25T00:14:01.837691+0000","last_became_peered":"2024-04-25T00:14:01.837691+0000","last_unstale":"2024-04-25T00:15:30.952457+0000","last_undegraded":"2024-04-25T00:15:30.952457+0000","last_fullsized":"2024-04-25T00:15:30.952457+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:26.677039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.371358+0000","last_change":"2024-04-25T00:14:06.857061+0000","last_active":"2024-04-25T00:15:02.371358+0000","last_peered":"2024-04-25T00:15:02.371358+0000","last_clean":"2024-04-25T00:15:02.371358+0000","last_became_active":"2024-04-25T00:14:06.856678+0000","last_became_peered":"2024-04-25T00:14:06.856678+0000","last_unstale":"2024-04-25T00:15:02.371358+0000","last_undegraded":"2024-04-25T00:15:02.371358+0000","last_fullsized":"2024-04-25T00:15:02.371358+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:36:52.730861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.075817+0000","last_change":"2024-04-25T00:14:01.408385+0000","last_active":"2024-04-25T00:15:02.075817+0000","last_peered":"2024-04-25T00:15:02.075817+0000","last_clean":"2024-04-25T00:15:02.075817+0000","last_became_active":"2024-04-25T00:14:01.408049+0000","last_became_peered":"2024-04-25T00:14:01.408049+0000","last_unstale":"2024-04-25T00:15:02.075817+0000","last_undegraded":"2024-04-25T00:15:02.075817+0000","last_fullsized":"2024-04-25T00:15:02.075817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:19.765467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.955201+0000","last_change":"2024-04-25T00:15:02.372426+0000","last_active":"2024-04-25T00:15:30.955201+0000","last_peered":"2024-04-25T00:15:30.955201+0000","last_clean":"2024-04-25T00:15:30.955201+0000","last_became_active":"2024-04-25T00:15:02.372044+0000","last_became_peered":"2024-04-25T00:15:02.372044+0000","last_unstale":"2024-04-25T00:15:30.955201+0000","last_undegraded":"2024-04-25T00:15:30.955201+0000","last_fullsized":"2024-04-25T00:15:30.955201+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:20.965683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.082497+0000","last_change":"2024-04-25T00:14:06.854192+0000","last_active":"2024-04-25T00:15:02.082497+0000","last_peered":"2024-04-25T00:15:02.082497+0000","last_clean":"2024-04-25T00:15:02.082497+0000","last_became_active":"2024-04-25T00:14:06.853325+0000","last_became_peered":"2024-04-25T00:14:06.853325+0000","last_unstale":"2024-04-25T00:15:02.082497+0000","last_undegraded":"2024-04-25T00:15:02.082497+0000","last_fullsized":"2024-04-25T00:15:02.082497+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:15.308169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076565+0000","last_change":"2024-04-25T00:14:01.407247+0000","last_active":"2024-04-25T00:15:02.076565+0000","last_peered":"2024-04-25T00:15:02.076565+0000","last_clean":"2024-04-25T00:15:02.076565+0000","last_became_active":"2024-04-25T00:14:01.406891+0000","last_became_peered":"2024-04-25T00:14:01.406891+0000","last_unstale":"2024-04-25T00:15:02.076565+0000","last_undegraded":"2024-04-25T00:15:02.076565+0000","last_fullsized":"2024-04-25T00:15:02.076565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:43.294562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.945357+0000","last_change":"2024-04-25T00:14:01.829364+0000","last_active":"2024-04-25T00:15:30.945357+0000","last_peered":"2024-04-25T00:15:30.945357+0000","last_clean":"2024-04-25T00:15:30.945357+0000","last_became_active":"2024-04-25T00:14:01.829087+0000","last_became_peered":"2024-04-25T00:14:01.829087+0000","last_unstale":"2024-04-25T00:15:30.945357+0000","last_undegraded":"2024-04-25T00:15:30.945357+0000","last_fullsized":"2024-04-25T00:15:30.945357+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:17.651815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076133+0000","last_change":"2024-04-25T00:14:07.323442+0000","last_active":"2024-04-25T00:15:02.076133+0000","last_peered":"2024-04-25T00:15:02.076133+0000","last_clean":"2024-04-25T00:15:02.076133+0000","last_became_active":"2024-04-25T00:14:07.322943+0000","last_became_peered":"2024-04-25T00:14:07.322943+0000","last_unstale":"2024-04-25T00:15:02.076133+0000","last_undegraded":"2024-04-25T00:15:02.076133+0000","last_fullsized":"2024-04-25T00:15:02.076133+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:49.166240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076222+0000","last_change":"2024-04-25T00:14:01.410936+0000","last_active":"2024-04-25T00:15:02.076222+0000","last_peered":"2024-04-25T00:15:02.076222+0000","last_clean":"2024-04-25T00:15:02.076222+0000","last_became_active":"2024-04-25T00:14:01.410321+0000","last_became_peered":"2024-04-25T00:14:01.410321+0000","last_unstale":"2024-04-25T00:15:02.076222+0000","last_undegraded":"2024-04-25T00:15:02.076222+0000","last_fullsized":"2024-04-25T00:15:02.076222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:44.603199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"94'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.938542+0000","last_change":"2024-04-25T00:14:01.830842+0000","last_active":"2024-04-25T00:15:30.938542+0000","last_peered":"2024-04-25T00:15:30.938542+0000","last_clean":"2024-04-25T00:15:30.938542+0000","last_became_active":"2024-04-25T00:14:01.830543+0000","last_became_peered":"2024-04-25T00:14:01.830543+0000","last_unstale":"2024-04-25T00:15:30.938542+0000","last_undegraded":"2024-04-25T00:15:30.938542+0000","last_fullsized":"2024-04-25T00:15:30.938542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:59.897133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.371351+0000","last_change":"2024-04-25T00:14:06.856841+0000","last_active":"2024-04-25T00:15:02.371351+0000","last_peered":"2024-04-25T00:15:02.371351+0000","last_clean":"2024-04-25T00:15:02.371351+0000","last_became_active":"2024-04-25T00:14:06.856416+0000","last_became_peered":"2024-04-25T00:14:06.856416+0000","last_unstale":"2024-04-25T00:15:02.371351+0000","last_undegraded":"2024-04-25T00:15:02.371351+0000","last_fullsized":"2024-04-25T00:15:02.371351+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:27.181175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.371225+0000","last_change":"2024-04-25T00:14:01.409627+0000","last_active":"2024-04-25T00:15:02.371225+0000","last_peered":"2024-04-25T00:15:02.371225+0000","last_clean":"2024-04-25T00:15:02.371225+0000","last_became_active":"2024-04-25T00:14:01.409304+0000","last_became_peered":"2024-04-25T00:14:01.409304+0000","last_unstale":"2024-04-25T00:15:02.371225+0000","last_undegraded":"2024-04-25T00:15:02.371225+0000","last_fullsized":"2024-04-25T00:15:02.371225+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:27.242004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.958599+0000","last_change":"2024-04-25T00:14:01.829544+0000","last_active":"2024-04-25T00:15:30.958599+0000","last_peered":"2024-04-25T00:15:30.958599+0000","last_clean":"2024-04-25T00:15:30.958599+0000","last_became_active":"2024-04-25T00:14:01.829233+0000","last_became_peered":"2024-04-25T00:14:01.829233+0000","last_unstale":"2024-04-25T00:15:30.958599+0000","last_undegraded":"2024-04-25T00:15:30.958599+0000","last_fullsized":"2024-04-25T00:15:30.958599+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:16.102041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.371517+0000","last_change":"2024-04-25T00:14:07.323213+0000","last_active":"2024-04-25T00:15:02.371517+0000","last_peered":"2024-04-25T00:15:02.371517+0000","last_clean":"2024-04-25T00:15:02.371517+0000","last_became_active":"2024-04-25T00:14:07.322763+0000","last_became_peered":"2024-04-25T00:14:07.322763+0000","last_unstale":"2024-04-25T00:15:02.371517+0000","last_undegraded":"2024-04-25T00:15:02.371517+0000","last_fullsized":"2024-04-25T00:15:02.371517+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:45.269314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.947333+0000","last_change":"2024-04-25T00:15:02.079745+0000","last_active":"2024-04-25T00:15:30.947333+0000","last_peered":"2024-04-25T00:15:30.947333+0000","last_clean":"2024-04-25T00:15:30.947333+0000","last_became_active":"2024-04-25T00:15:02.079405+0000","last_became_peered":"2024-04-25T00:15:02.079405+0000","last_unstale":"2024-04-25T00:15:30.947333+0000","last_undegraded":"2024-04-25T00:15:30.947333+0000","last_fullsized":"2024-04-25T00:15:30.947333+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:38.197048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"92'4","reported_seq":106,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:37.642606+0000","last_change":"2024-04-25T00:14:01.397066+0000","last_active":"2024-04-25T00:15:37.642606+0000","last_peered":"2024-04-25T00:15:37.642606+0000","last_clean":"2024-04-25T00:15:37.642606+0000","last_became_active":"2024-04-25T00:14:01.396728+0000","last_became_peered":"2024-04-25T00:14:01.396728+0000","last_unstale":"2024-04-25T00:15:37.642606+0000","last_undegraded":"2024-04-25T00:15:37.642606+0000","last_fullsized":"2024-04-25T00:15:37.642606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:28.334329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"92'1","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.371995+0000","last_change":"2024-04-25T00:14:06.853039+0000","last_active":"2024-04-25T00:15:02.371995+0000","last_peered":"2024-04-25T00:15:02.371995+0000","last_clean":"2024-04-25T00:15:02.371995+0000","last_became_active":"2024-04-25T00:14:06.852568+0000","last_became_peered":"2024-04-25T00:14:06.852568+0000","last_unstale":"2024-04-25T00:15:02.371995+0000","last_undegraded":"2024-04-25T00:15:02.371995+0000","last_fullsized":"2024-04-25T00:15:02.371995+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:59.604160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.965920+0000","last_change":"2024-04-25T00:14:01.840315+0000","last_active":"2024-04-25T00:15:30.965920+0000","last_peered":"2024-04-25T00:15:30.965920+0000","last_clean":"2024-04-25T00:15:30.965920+0000","last_became_active":"2024-04-25T00:14:01.839537+0000","last_became_peered":"2024-04-25T00:14:01.839537+0000","last_unstale":"2024-04-25T00:15:30.965920+0000","last_undegraded":"2024-04-25T00:15:30.965920+0000","last_fullsized":"2024-04-25T00:15:30.965920+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:34.126164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.075830+0000","last_change":"2024-04-25T00:14:01.412493+0000","last_active":"2024-04-25T00:15:02.075830+0000","last_peered":"2024-04-25T00:15:02.075830+0000","last_clean":"2024-04-25T00:15:02.075830+0000","last_became_active":"2024-04-25T00:14:01.412133+0000","last_became_peered":"2024-04-25T00:14:01.412133+0000","last_unstale":"2024-04-25T00:15:02.075830+0000","last_undegraded":"2024-04-25T00:15:02.075830+0000","last_fullsized":"2024-04-25T00:15:02.075830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:44.933728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":365,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.370851+0000","last_change":"2024-04-25T00:12:49.169294+0000","last_active":"2024-04-25T00:15:02.370851+0000","last_peered":"2024-04-25T00:15:02.370851+0000","last_clean":"2024-04-25T00:15:02.370851+0000","last_became_active":"2024-04-25T00:12:49.168783+0000","last_became_peered":"2024-04-25T00:12:49.168783+0000","last_unstale":"2024-04-25T00:15:02.370851+0000","last_undegraded":"2024-04-25T00:15:02.370851+0000","last_fullsized":"2024-04-25T00:15:02.370851+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:25.202280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:25.202280+0000","last_clean_scrub_stamp":"2024-04-25T00:09:25.202280+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:04.713986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.373148+0000","last_change":"2024-04-25T00:15:02.372570+0000","last_active":"2024-04-25T00:15:02.373148+0000","last_peered":"2024-04-25T00:15:02.373148+0000","last_clean":"2024-04-25T00:15:02.373148+0000","last_became_active":"2024-04-25T00:15:02.372066+0000","last_became_peered":"2024-04-25T00:15:02.372066+0000","last_unstale":"2024-04-25T00:15:02.373148+0000","last_undegraded":"2024-04-25T00:15:02.373148+0000","last_fullsized":"2024-04-25T00:15:02.373148+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:59.470049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,6,10],"acting":[8,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.960856+0000","last_change":"2024-04-25T00:14:01.826418+0000","last_active":"2024-04-25T00:15:30.960856+0000","last_peered":"2024-04-25T00:15:30.960856+0000","last_clean":"2024-04-25T00:15:30.960856+0000","last_became_active":"2024-04-25T00:14:01.825783+0000","last_became_peered":"2024-04-25T00:14:01.825783+0000","last_unstale":"2024-04-25T00:15:30.960856+0000","last_undegraded":"2024-04-25T00:15:30.960856+0000","last_fullsized":"2024-04-25T00:15:30.960856+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:05.981359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.370975+0000","last_change":"2024-04-25T00:14:01.417022+0000","last_active":"2024-04-25T00:15:02.370975+0000","last_peered":"2024-04-25T00:15:02.370975+0000","last_clean":"2024-04-25T00:15:02.370975+0000","last_became_active":"2024-04-25T00:14:01.416804+0000","last_became_peered":"2024-04-25T00:14:01.416804+0000","last_unstale":"2024-04-25T00:15:02.370975+0000","last_undegraded":"2024-04-25T00:15:02.370975+0000","last_fullsized":"2024-04-25T00:15:02.370975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:31.561242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.075667+0000","last_change":"2024-04-25T00:14:06.856500+0000","last_active":"2024-04-25T00:15:02.075667+0000","last_peered":"2024-04-25T00:15:02.075667+0000","last_clean":"2024-04-25T00:15:02.075667+0000","last_became_active":"2024-04-25T00:14:06.856077+0000","last_became_peered":"2024-04-25T00:14:06.856077+0000","last_unstale":"2024-04-25T00:15:02.075667+0000","last_undegraded":"2024-04-25T00:15:02.075667+0000","last_fullsized":"2024-04-25T00:15:02.075667+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:10:04.129533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076218+0000","last_change":"2024-04-25T00:14:01.414599+0000","last_active":"2024-04-25T00:15:02.076218+0000","last_peered":"2024-04-25T00:15:02.076218+0000","last_clean":"2024-04-25T00:15:02.076218+0000","last_became_active":"2024-04-25T00:14:01.413796+0000","last_became_peered":"2024-04-25T00:14:01.413796+0000","last_unstale":"2024-04-25T00:15:02.076218+0000","last_undegraded":"2024-04-25T00:15:02.076218+0000","last_fullsized":"2024-04-25T00:15:02.076218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:44.981848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.960865+0000","last_change":"2024-04-25T00:14:01.827718+0000","last_active":"2024-04-25T00:15:30.960865+0000","last_peered":"2024-04-25T00:15:30.960865+0000","last_clean":"2024-04-25T00:15:30.960865+0000","last_became_active":"2024-04-25T00:14:01.827410+0000","last_became_peered":"2024-04-25T00:14:01.827410+0000","last_unstale":"2024-04-25T00:15:30.960865+0000","last_undegraded":"2024-04-25T00:15:30.960865+0000","last_fullsized":"2024-04-25T00:15:30.960865+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:32.428137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.073685+0000","last_change":"2024-04-25T00:14:07.323304+0000","last_active":"2024-04-25T00:15:02.073685+0000","last_peered":"2024-04-25T00:15:02.073685+0000","last_clean":"2024-04-25T00:15:02.073685+0000","last_became_active":"2024-04-25T00:14:07.322852+0000","last_became_peered":"2024-04-25T00:14:07.322852+0000","last_unstale":"2024-04-25T00:15:02.073685+0000","last_undegraded":"2024-04-25T00:15:02.073685+0000","last_fullsized":"2024-04-25T00:15:02.073685+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:35.173934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076228+0000","last_change":"2024-04-25T00:14:01.413209+0000","last_active":"2024-04-25T00:15:02.076228+0000","last_peered":"2024-04-25T00:15:02.076228+0000","last_clean":"2024-04-25T00:15:02.076228+0000","last_became_active":"2024-04-25T00:14:01.412892+0000","last_became_peered":"2024-04-25T00:14:01.412892+0000","last_unstale":"2024-04-25T00:15:02.076228+0000","last_undegraded":"2024-04-25T00:15:02.076228+0000","last_fullsized":"2024-04-25T00:15:02.076228+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:26.152962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.961212+0000","last_change":"2024-04-25T00:14:01.834372+0000","last_active":"2024-04-25T00:15:30.961212+0000","last_peered":"2024-04-25T00:15:30.961212+0000","last_clean":"2024-04-25T00:15:30.961212+0000","last_became_active":"2024-04-25T00:14:01.834062+0000","last_became_peered":"2024-04-25T00:14:01.834062+0000","last_unstale":"2024-04-25T00:15:30.961212+0000","last_undegraded":"2024-04-25T00:15:30.961212+0000","last_fullsized":"2024-04-25T00:15:30.961212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:05.580604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.073535+0000","last_change":"2024-04-25T00:14:06.852303+0000","last_active":"2024-04-25T00:15:02.073535+0000","last_peered":"2024-04-25T00:15:02.073535+0000","last_clean":"2024-04-25T00:15:02.073535+0000","last_became_active":"2024-04-25T00:14:06.850957+0000","last_became_peered":"2024-04-25T00:14:06.850957+0000","last_unstale":"2024-04-25T00:15:02.073535+0000","last_undegraded":"2024-04-25T00:15:02.073535+0000","last_fullsized":"2024-04-25T00:15:02.073535+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:47.208880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076101+0000","last_change":"2024-04-25T00:14:01.405538+0000","last_active":"2024-04-25T00:15:02.076101+0000","last_peered":"2024-04-25T00:15:02.076101+0000","last_clean":"2024-04-25T00:15:02.076101+0000","last_became_active":"2024-04-25T00:14:01.405155+0000","last_became_peered":"2024-04-25T00:14:01.405155+0000","last_unstale":"2024-04-25T00:15:02.076101+0000","last_undegraded":"2024-04-25T00:15:02.076101+0000","last_fullsized":"2024-04-25T00:15:02.076101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:20.864753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.962393+0000","last_change":"2024-04-25T00:14:01.837058+0000","last_active":"2024-04-25T00:15:30.962393+0000","last_peered":"2024-04-25T00:15:30.962393+0000","last_clean":"2024-04-25T00:15:30.962393+0000","last_became_active":"2024-04-25T00:14:01.835838+0000","last_became_peered":"2024-04-25T00:14:01.835838+0000","last_unstale":"2024-04-25T00:15:30.962393+0000","last_undegraded":"2024-04-25T00:15:30.962393+0000","last_fullsized":"2024-04-25T00:15:30.962393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:21.696448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.075742+0000","last_change":"2024-04-25T00:14:07.323870+0000","last_active":"2024-04-25T00:15:02.075742+0000","last_peered":"2024-04-25T00:15:02.075742+0000","last_clean":"2024-04-25T00:15:02.075742+0000","last_became_active":"2024-04-25T00:14:07.323356+0000","last_became_peered":"2024-04-25T00:14:07.323356+0000","last_unstale":"2024-04-25T00:15:02.075742+0000","last_undegraded":"2024-04-25T00:15:02.075742+0000","last_fullsized":"2024-04-25T00:15:02.075742+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:35.160618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.371557+0000","last_change":"2024-04-25T00:14:01.409453+0000","last_active":"2024-04-25T00:15:02.371557+0000","last_peered":"2024-04-25T00:15:02.371557+0000","last_clean":"2024-04-25T00:15:02.371557+0000","last_became_active":"2024-04-25T00:14:01.409164+0000","last_became_peered":"2024-04-25T00:14:01.409164+0000","last_unstale":"2024-04-25T00:15:02.371557+0000","last_undegraded":"2024-04-25T00:15:02.371557+0000","last_fullsized":"2024-04-25T00:15:02.371557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:23.266090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.910070+0000","last_change":"2024-04-25T00:14:01.826790+0000","last_active":"2024-04-25T00:15:30.910070+0000","last_peered":"2024-04-25T00:15:30.910070+0000","last_clean":"2024-04-25T00:15:30.910070+0000","last_became_active":"2024-04-25T00:14:01.826457+0000","last_became_peered":"2024-04-25T00:14:01.826457+0000","last_unstale":"2024-04-25T00:15:30.910070+0000","last_undegraded":"2024-04-25T00:15:30.910070+0000","last_fullsized":"2024-04-25T00:15:30.910070+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:15.356448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076592+0000","last_change":"2024-04-25T00:14:06.863280+0000","last_active":"2024-04-25T00:15:02.076592+0000","last_peered":"2024-04-25T00:15:02.076592+0000","last_clean":"2024-04-25T00:15:02.076592+0000","last_became_active":"2024-04-25T00:14:06.862725+0000","last_became_peered":"2024-04-25T00:14:06.862725+0000","last_unstale":"2024-04-25T00:15:02.076592+0000","last_undegraded":"2024-04-25T00:15:02.076592+0000","last_fullsized":"2024-04-25T00:15:02.076592+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:12.791744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.073815+0000","last_change":"2024-04-25T00:14:01.414856+0000","last_active":"2024-04-25T00:15:02.073815+0000","last_peered":"2024-04-25T00:15:02.073815+0000","last_clean":"2024-04-25T00:15:02.073815+0000","last_became_active":"2024-04-25T00:14:01.414533+0000","last_became_peered":"2024-04-25T00:14:01.414533+0000","last_unstale":"2024-04-25T00:15:02.073815+0000","last_undegraded":"2024-04-25T00:15:02.073815+0000","last_fullsized":"2024-04-25T00:15:02.073815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:01.090539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.952846+0000","last_change":"2024-04-25T00:14:01.831452+0000","last_active":"2024-04-25T00:15:30.952846+0000","last_peered":"2024-04-25T00:15:30.952846+0000","last_clean":"2024-04-25T00:15:30.952846+0000","last_became_active":"2024-04-25T00:14:01.831151+0000","last_became_peered":"2024-04-25T00:14:01.831151+0000","last_unstale":"2024-04-25T00:15:30.952846+0000","last_undegraded":"2024-04-25T00:15:30.952846+0000","last_fullsized":"2024-04-25T00:15:30.952846+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:53:07.659508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"92'1","reported_seq":22,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076010+0000","last_change":"2024-04-25T00:14:06.857601+0000","last_active":"2024-04-25T00:15:02.076010+0000","last_peered":"2024-04-25T00:15:02.076010+0000","last_clean":"2024-04-25T00:15:02.076010+0000","last_became_active":"2024-04-25T00:14:06.856685+0000","last_became_peered":"2024-04-25T00:14:06.856685+0000","last_unstale":"2024-04-25T00:15:02.076010+0000","last_undegraded":"2024-04-25T00:15:02.076010+0000","last_fullsized":"2024-04-25T00:15:02.076010+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:20.250053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.085142+0000","last_change":"2024-04-25T00:15:02.085142+0000","last_active":"2024-04-25T00:15:02.085142+0000","last_peered":"2024-04-25T00:15:02.085142+0000","last_clean":"2024-04-25T00:15:02.085142+0000","last_became_active":"2024-04-25T00:15:02.080336+0000","last_became_peered":"2024-04-25T00:15:02.080336+0000","last_unstale":"2024-04-25T00:15:02.085142+0000","last_undegraded":"2024-04-25T00:15:02.085142+0000","last_fullsized":"2024-04-25T00:15:02.085142+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:32.811681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.954566+0000","last_change":"2024-04-25T00:14:01.832342+0000","last_active":"2024-04-25T00:15:30.954566+0000","last_peered":"2024-04-25T00:15:30.954566+0000","last_clean":"2024-04-25T00:15:30.954566+0000","last_became_active":"2024-04-25T00:14:01.831997+0000","last_became_peered":"2024-04-25T00:14:01.831997+0000","last_unstale":"2024-04-25T00:15:30.954566+0000","last_undegraded":"2024-04-25T00:15:30.954566+0000","last_fullsized":"2024-04-25T00:15:30.954566+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:12.335884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.075708+0000","last_change":"2024-04-25T00:14:07.323628+0000","last_active":"2024-04-25T00:15:02.075708+0000","last_peered":"2024-04-25T00:15:02.075708+0000","last_clean":"2024-04-25T00:15:02.075708+0000","last_became_active":"2024-04-25T00:14:07.322996+0000","last_became_peered":"2024-04-25T00:14:07.322996+0000","last_unstale":"2024-04-25T00:15:02.075708+0000","last_undegraded":"2024-04-25T00:15:02.075708+0000","last_fullsized":"2024-04-25T00:15:02.075708+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:32.803216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.075953+0000","last_change":"2024-04-25T00:14:01.407715+0000","last_active":"2024-04-25T00:15:02.075953+0000","last_peered":"2024-04-25T00:15:02.075953+0000","last_clean":"2024-04-25T00:15:02.075953+0000","last_became_active":"2024-04-25T00:14:01.407439+0000","last_became_peered":"2024-04-25T00:14:01.407439+0000","last_unstale":"2024-04-25T00:15:02.075953+0000","last_undegraded":"2024-04-25T00:15:02.075953+0000","last_fullsized":"2024-04-25T00:15:02.075953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:33.550379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.960664+0000","last_change":"2024-04-25T00:14:01.837733+0000","last_active":"2024-04-25T00:15:30.960664+0000","last_peered":"2024-04-25T00:15:30.960664+0000","last_clean":"2024-04-25T00:15:30.960664+0000","last_became_active":"2024-04-25T00:14:01.837395+0000","last_became_peered":"2024-04-25T00:14:01.837395+0000","last_unstale":"2024-04-25T00:15:30.960664+0000","last_undegraded":"2024-04-25T00:15:30.960664+0000","last_fullsized":"2024-04-25T00:15:30.960664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:50.186834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.370740+0000","last_change":"2024-04-25T00:14:06.848700+0000","last_active":"2024-04-25T00:15:02.370740+0000","last_peered":"2024-04-25T00:15:02.370740+0000","last_clean":"2024-04-25T00:15:02.370740+0000","last_became_active":"2024-04-25T00:14:06.848252+0000","last_became_peered":"2024-04-25T00:14:06.848252+0000","last_unstale":"2024-04-25T00:15:02.370740+0000","last_undegraded":"2024-04-25T00:15:02.370740+0000","last_fullsized":"2024-04-25T00:15:02.370740+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:07.653668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.074029+0000","last_change":"2024-04-25T00:14:01.405785+0000","last_active":"2024-04-25T00:15:02.074029+0000","last_peered":"2024-04-25T00:15:02.074029+0000","last_clean":"2024-04-25T00:15:02.074029+0000","last_became_active":"2024-04-25T00:14:01.405475+0000","last_became_peered":"2024-04-25T00:14:01.405475+0000","last_unstale":"2024-04-25T00:15:02.074029+0000","last_undegraded":"2024-04-25T00:15:02.074029+0000","last_fullsized":"2024-04-25T00:15:02.074029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:32.487795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.959655+0000","last_change":"2024-04-25T00:14:01.832393+0000","last_active":"2024-04-25T00:15:30.959655+0000","last_peered":"2024-04-25T00:15:30.959655+0000","last_clean":"2024-04-25T00:15:30.959655+0000","last_became_active":"2024-04-25T00:14:01.832096+0000","last_became_peered":"2024-04-25T00:14:01.832096+0000","last_unstale":"2024-04-25T00:15:30.959655+0000","last_undegraded":"2024-04-25T00:15:30.959655+0000","last_fullsized":"2024-04-25T00:15:30.959655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:41.749627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.076472+0000","last_change":"2024-04-25T00:14:06.855224+0000","last_active":"2024-04-25T00:15:02.076472+0000","last_peered":"2024-04-25T00:15:02.076472+0000","last_clean":"2024-04-25T00:15:02.076472+0000","last_became_active":"2024-04-25T00:14:06.854847+0000","last_became_peered":"2024-04-25T00:14:06.854847+0000","last_unstale":"2024-04-25T00:15:02.076472+0000","last_undegraded":"2024-04-25T00:15:02.076472+0000","last_fullsized":"2024-04-25T00:15:02.076472+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:13.520448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.081520+0000","last_change":"2024-04-25T00:14:01.401145+0000","last_active":"2024-04-25T00:15:02.081520+0000","last_peered":"2024-04-25T00:15:02.081520+0000","last_clean":"2024-04-25T00:15:02.081520+0000","last_became_active":"2024-04-25T00:14:01.400807+0000","last_became_peered":"2024-04-25T00:14:01.400807+0000","last_unstale":"2024-04-25T00:15:02.081520+0000","last_undegraded":"2024-04-25T00:15:02.081520+0000","last_fullsized":"2024-04-25T00:15:02.081520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:52:04.674677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"94'6","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.958132+0000","last_change":"2024-04-25T00:14:01.838033+0000","last_active":"2024-04-25T00:15:30.958132+0000","last_peered":"2024-04-25T00:15:30.958132+0000","last_clean":"2024-04-25T00:15:30.958132+0000","last_became_active":"2024-04-25T00:14:01.837657+0000","last_became_peered":"2024-04-25T00:14:01.837657+0000","last_unstale":"2024-04-25T00:15:30.958132+0000","last_undegraded":"2024-04-25T00:15:30.958132+0000","last_fullsized":"2024-04-25T00:15:30.958132+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:51.895086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.082042+0000","last_change":"2024-04-25T00:14:06.854885+0000","last_active":"2024-04-25T00:15:02.082042+0000","last_peered":"2024-04-25T00:15:02.082042+0000","last_clean":"2024-04-25T00:15:02.082042+0000","last_became_active":"2024-04-25T00:14:06.854200+0000","last_became_peered":"2024-04-25T00:14:06.854200+0000","last_unstale":"2024-04-25T00:15:02.082042+0000","last_undegraded":"2024-04-25T00:15:02.082042+0000","last_fullsized":"2024-04-25T00:15:02.082042+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:43.627742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:02.371379+0000","last_change":"2024-04-25T00:14:01.403789+0000","last_active":"2024-04-25T00:15:02.371379+0000","last_peered":"2024-04-25T00:15:02.371379+0000","last_clean":"2024-04-25T00:15:02.371379+0000","last_became_active":"2024-04-25T00:14:01.403464+0000","last_became_peered":"2024-04-25T00:14:01.403464+0000","last_unstale":"2024-04-25T00:15:02.371379+0000","last_undegraded":"2024-04-25T00:15:02.371379+0000","last_fullsized":"2024-04-25T00:15:02.371379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:48.276536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.950980+0000","last_change":"2024-04-25T00:14:01.839988+0000","last_active":"2024-04-25T00:15:30.950980+0000","last_peered":"2024-04-25T00:15:30.950980+0000","last_clean":"2024-04-25T00:15:30.950980+0000","last_became_active":"2024-04-25T00:14:01.839516+0000","last_became_peered":"2024-04-25T00:14:01.839516+0000","last_unstale":"2024-04-25T00:15:30.950980+0000","last_undegraded":"2024-04-25T00:15:30.950980+0000","last_fullsized":"2024-04-25T00:15:30.950980+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:05.875127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.943071+0000","last_change":"2024-04-25T00:14:01.828638+0000","last_active":"2024-04-25T00:15:30.943071+0000","last_peered":"2024-04-25T00:15:30.943071+0000","last_clean":"2024-04-25T00:15:30.943071+0000","last_became_active":"2024-04-25T00:14:01.828266+0000","last_became_peered":"2024-04-25T00:14:01.828266+0000","last_unstale":"2024-04-25T00:15:30.943071+0000","last_undegraded":"2024-04-25T00:15:30.943071+0000","last_fullsized":"2024-04-25T00:15:30.943071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:34:22.619522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.947867+0000","last_change":"2024-04-25T00:14:01.828199+0000","last_active":"2024-04-25T00:15:30.947867+0000","last_peered":"2024-04-25T00:15:30.947867+0000","last_clean":"2024-04-25T00:15:30.947867+0000","last_became_active":"2024-04-25T00:14:01.827579+0000","last_became_peered":"2024-04-25T00:14:01.827579+0000","last_unstale":"2024-04-25T00:15:30.947867+0000","last_undegraded":"2024-04-25T00:15:30.947867+0000","last_fullsized":"2024-04-25T00:15:30.947867+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:00.708928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.958158+0000","last_change":"2024-04-25T00:14:01.825620+0000","last_active":"2024-04-25T00:15:30.958158+0000","last_peered":"2024-04-25T00:15:30.958158+0000","last_clean":"2024-04-25T00:15:30.958158+0000","last_became_active":"2024-04-25T00:14:01.825310+0000","last_became_peered":"2024-04-25T00:14:01.825310+0000","last_unstale":"2024-04-25T00:15:30.958158+0000","last_undegraded":"2024-04-25T00:15:30.958158+0000","last_fullsized":"2024-04-25T00:15:30.958158+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:35.075355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.962619+0000","last_change":"2024-04-25T00:14:01.836537+0000","last_active":"2024-04-25T00:15:30.962619+0000","last_peered":"2024-04-25T00:15:30.962619+0000","last_clean":"2024-04-25T00:15:30.962619+0000","last_became_active":"2024-04-25T00:14:01.836234+0000","last_became_peered":"2024-04-25T00:14:01.836234+0000","last_unstale":"2024-04-25T00:15:30.962619+0000","last_undegraded":"2024-04-25T00:15:30.962619+0000","last_fullsized":"2024-04-25T00:15:30.962619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:15.037498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.963606+0000","last_change":"2024-04-25T00:14:01.825937+0000","last_active":"2024-04-25T00:15:30.963606+0000","last_peered":"2024-04-25T00:15:30.963606+0000","last_clean":"2024-04-25T00:15:30.963606+0000","last_became_active":"2024-04-25T00:14:01.825605+0000","last_became_peered":"2024-04-25T00:14:01.825605+0000","last_unstale":"2024-04-25T00:15:30.963606+0000","last_undegraded":"2024-04-25T00:15:30.963606+0000","last_fullsized":"2024-04-25T00:15:30.963606+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:06.918000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.940251+0000","last_change":"2024-04-25T00:14:01.830004+0000","last_active":"2024-04-25T00:15:30.940251+0000","last_peered":"2024-04-25T00:15:30.940251+0000","last_clean":"2024-04-25T00:15:30.940251+0000","last_became_active":"2024-04-25T00:14:01.829588+0000","last_became_peered":"2024-04-25T00:14:01.829588+0000","last_unstale":"2024-04-25T00:15:30.940251+0000","last_undegraded":"2024-04-25T00:15:30.940251+0000","last_fullsized":"2024-04-25T00:15:30.940251+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:08.005995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.940551+0000","last_change":"2024-04-25T00:14:01.825762+0000","last_active":"2024-04-25T00:15:30.940551+0000","last_peered":"2024-04-25T00:15:30.940551+0000","last_clean":"2024-04-25T00:15:30.940551+0000","last_became_active":"2024-04-25T00:14:01.825471+0000","last_became_peered":"2024-04-25T00:14:01.825471+0000","last_unstale":"2024-04-25T00:15:30.940551+0000","last_undegraded":"2024-04-25T00:15:30.940551+0000","last_fullsized":"2024-04-25T00:15:30.940551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:33.716875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.956340+0000","last_change":"2024-04-25T00:14:01.836103+0000","last_active":"2024-04-25T00:15:30.956340+0000","last_peered":"2024-04-25T00:15:30.956340+0000","last_clean":"2024-04-25T00:15:30.956340+0000","last_became_active":"2024-04-25T00:14:01.834591+0000","last_became_peered":"2024-04-25T00:14:01.834591+0000","last_unstale":"2024-04-25T00:15:30.956340+0000","last_undegraded":"2024-04-25T00:15:30.956340+0000","last_fullsized":"2024-04-25T00:15:30.956340+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:55.130978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.965741+0000","last_change":"2024-04-25T00:14:01.833179+0000","last_active":"2024-04-25T00:15:30.965741+0000","last_peered":"2024-04-25T00:15:30.965741+0000","last_clean":"2024-04-25T00:15:30.965741+0000","last_became_active":"2024-04-25T00:14:01.832856+0000","last_became_peered":"2024-04-25T00:14:01.832856+0000","last_unstale":"2024-04-25T00:15:30.965741+0000","last_undegraded":"2024-04-25T00:15:30.965741+0000","last_fullsized":"2024-04-25T00:15:30.965741+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:07.153356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.955106+0000","last_change":"2024-04-25T00:14:01.834751+0000","last_active":"2024-04-25T00:15:30.955106+0000","last_peered":"2024-04-25T00:15:30.955106+0000","last_clean":"2024-04-25T00:15:30.955106+0000","last_became_active":"2024-04-25T00:14:01.834413+0000","last_became_peered":"2024-04-25T00:14:01.834413+0000","last_unstale":"2024-04-25T00:15:30.955106+0000","last_undegraded":"2024-04-25T00:15:30.955106+0000","last_fullsized":"2024-04-25T00:15:30.955106+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:52.341565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.966591+0000","last_change":"2024-04-25T00:14:01.832944+0000","last_active":"2024-04-25T00:15:30.966591+0000","last_peered":"2024-04-25T00:15:30.966591+0000","last_clean":"2024-04-25T00:15:30.966591+0000","last_became_active":"2024-04-25T00:14:01.832394+0000","last_became_peered":"2024-04-25T00:14:01.832394+0000","last_unstale":"2024-04-25T00:15:30.966591+0000","last_undegraded":"2024-04-25T00:15:30.966591+0000","last_fullsized":"2024-04-25T00:15:30.966591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:26.957786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.960241+0000","last_change":"2024-04-25T00:14:01.831492+0000","last_active":"2024-04-25T00:15:30.960241+0000","last_peered":"2024-04-25T00:15:30.960241+0000","last_clean":"2024-04-25T00:15:30.960241+0000","last_became_active":"2024-04-25T00:14:01.829557+0000","last_became_peered":"2024-04-25T00:14:01.829557+0000","last_unstale":"2024-04-25T00:15:30.960241+0000","last_undegraded":"2024-04-25T00:15:30.960241+0000","last_fullsized":"2024-04-25T00:15:30.960241+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:45.116188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T00:15:30.945618+0000","last_change":"2024-04-25T00:14:01.839323+0000","last_active":"2024-04-25T00:15:30.945618+0000","last_peered":"2024-04-25T00:15:30.945618+0000","last_clean":"2024-04-25T00:15:30.945618+0000","last_became_active":"2024-04-25T00:14:01.838999+0000","last_became_peered":"2024-04-25T00:14:01.838999+0000","last_unstale":"2024-04-25T00:15:30.945618+0000","last_undegraded":"2024-04-25T00:15:30.945618+0000","last_fullsized":"2024-04-25T00:15:30.945618+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:20.652123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67747,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":462,"num_read_kb":302,"num_write":168,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":203241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351010,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":521389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612646,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37048,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708104,"statfs":{"total":95995035648,"available":95957098496,"internally_reserved":0,"allocated":1753088,"data_stored":1172262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776172,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":525721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037809,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28068,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717084,"statfs":{"total":95995035648,"available":95966294016,"internally_reserved":0,"allocated":1077248,"data_stored":521545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299445,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1671168,"data_stored":1117595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462971,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":516801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":580316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1081344,"data_stored":518189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1150976,"data_stored":582198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149769,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1069056,"data_stored":516995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888133,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40540,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704612,"statfs":{"total":95995035648,"available":95953522688,"internally_reserved":0,"allocated":1069056,"data_stored":521451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724608,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1679360,"data_stored":1113183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":63605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:15:45.628 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T00:15:45.629 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T00:15:45.629 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T00:15:45.629 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T00:15:45.870 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:15:46.249 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:46 smithi107 ceph-mon[29153]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:15:46.249 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:46 smithi107 ceph-mon[29153]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-25T00:15:46.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:46 smithi195 ceph-mon[32833]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:15:46.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:46 smithi195 ceph-mon[32833]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-25T00:15:46.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:46 smithi170 ceph-mon[32807]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:15:46.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:46 smithi170 ceph-mon[32807]: pgmap v346: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-25T00:15:47.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.706+0000 7fcccf54e700 1 -- 172.21.15.107:0/1660444869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccc8101800 msgr2=0x7fccc8106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:47.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.706+0000 7fcccf54e700 1 --2- 172.21.15.107:0/1660444869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccc8101800 0x7fccc8106f40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fccb80099c0 tx=0x7fccb8009cd0 comp rx=0 tx=0).stop 2024-04-25T00:15:47.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.706+0000 7fcccf54e700 1 -- 172.21.15.107:0/1660444869 shutdown_connections 2024-04-25T00:15:47.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.706+0000 7fcccf54e700 1 --2- 172.21.15.107:0/1660444869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccc8107670 0x7fccc810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:47.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.706+0000 7fcccf54e700 1 --2- 172.21.15.107:0/1660444869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccc8100e50 0x7fccc8101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:47.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.706+0000 7fcccf54e700 1 --2- 172.21.15.107:0/1660444869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccc8101800 0x7fccc8106f40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:47.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.706+0000 7fcccf54e700 1 -- 172.21.15.107:0/1660444869 >> 172.21.15.107:0/1660444869 conn(0x7fccc8076450 msgr2=0x7fccc8076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:47.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.706+0000 7fcccf54e700 1 -- 172.21.15.107:0/1660444869 shutdown_connections 2024-04-25T00:15:47.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.706+0000 7fcccf54e700 1 -- 172.21.15.107:0/1660444869 wait complete. 2024-04-25T00:15:47.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.707+0000 7fcccf54e700 1 Processor -- start 2024-04-25T00:15:47.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.707+0000 7fcccf54e700 1 -- start start 2024-04-25T00:15:47.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.707+0000 7fcccf54e700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccc8100e50 0x7fccc819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:47.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.707+0000 7fcccf54e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccc8101800 0x7fccc819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:47.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.707+0000 7fcccf54e700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccc8107670 0x7fccc81988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:47.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.707+0000 7fcccf54e700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fccc8112660 con 0x7fccc8107670 2024-04-25T00:15:47.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.707+0000 7fcccf54e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fccc81124e0 con 0x7fccc8101800 2024-04-25T00:15:47.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.707+0000 7fcccf54e700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fccc81127e0 con 0x7fccc8100e50 2024-04-25T00:15:47.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.707+0000 7fcccdaeb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccc8107670 0x7fccc81988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:47.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.707+0000 7fcccdaeb700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccc8107670 0x7fccc81988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38186/0 (socket says 172.21.15.107:38186) 2024-04-25T00:15:47.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.707+0000 7fcccdaeb700 1 -- 172.21.15.107:0/4211255749 learned_addr learned my addr 172.21.15.107:0/4211255749 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:47.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.707+0000 7fccccae9700 1 --2- 172.21.15.107:0/4211255749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccc8101800 0x7fccc819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:47.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.707+0000 7fcccd2ea700 1 --2- 172.21.15.107:0/4211255749 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccc8100e50 0x7fccc819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:47.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.707+0000 7fcccdaeb700 1 -- 172.21.15.107:0/4211255749 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccc8100e50 msgr2=0x7fccc819e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:47.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.707+0000 7fcccdaeb700 1 --2- 172.21.15.107:0/4211255749 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccc8100e50 0x7fccc819e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:47.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.707+0000 7fcccdaeb700 1 -- 172.21.15.107:0/4211255749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccc8101800 msgr2=0x7fccc819ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:47.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.707+0000 7fcccdaeb700 1 --2- 172.21.15.107:0/4211255749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccc8101800 0x7fccc819ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:47.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.707+0000 7fcccdaeb700 1 -- 172.21.15.107:0/4211255749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc8199040 con 0x7fccc8107670 2024-04-25T00:15:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.707+0000 7fccccae9700 1 --2- 172.21.15.107:0/4211255749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccc8101800 0x7fccc819ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.708+0000 7fcccdaeb700 1 --2- 172.21.15.107:0/4211255749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccc8107670 0x7fccc81988e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fccc000b970 tx=0x7fccc000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.708+0000 7fccbe7fc700 1 -- 172.21.15.107:0/4211255749 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc0019070 con 0x7fccc8107670 2024-04-25T00:15:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.708+0000 7fccbe7fc700 1 -- 172.21.15.107:0/4211255749 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc0014440 con 0x7fccc8107670 2024-04-25T00:15:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.708+0000 7fcccf54e700 1 -- 172.21.15.107:0/4211255749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc8199330 con 0x7fccc8107670 2024-04-25T00:15:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.708+0000 7fccbe7fc700 1 -- 172.21.15.107:0/4211255749 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc00075d0 con 0x7fccc8107670 2024-04-25T00:15:47.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.708+0000 7fcccf54e700 1 -- 172.21.15.107:0/4211255749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc81a55f0 con 0x7fccc8107670 2024-04-25T00:15:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.709+0000 7fcccf54e700 1 -- 172.21.15.107:0/4211255749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccc8068f20 con 0x7fccc8107670 2024-04-25T00:15:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.710+0000 7fccbe7fc700 1 -- 172.21.15.107:0/4211255749 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fccc001c070 con 0x7fccc8107670 2024-04-25T00:15:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.711+0000 7fccbe7fc700 1 --2- 172.21.15.107:0/4211255749 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fccb40a2c20 0x7fccb40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:47.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.711+0000 7fcccd2ea700 1 --2- 172.21.15.107:0/4211255749 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fccb40a2c20 0x7fccb40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:47.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.711+0000 7fccbe7fc700 1 -- 172.21.15.107:0/4211255749 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fccc00cec30 con 0x7fccc8107670 2024-04-25T00:15:47.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.712+0000 7fcccd2ea700 1 --2- 172.21.15.107:0/4211255749 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fccb40a2c20 0x7fccb40a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fccc8199a20 tx=0x7fccc4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:47.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.715+0000 7fccbe7fc700 1 -- 172.21.15.107:0/4211255749 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fccc009b850 con 0x7fccc8107670 2024-04-25T00:15:47.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.953+0000 7fcccf54e700 1 -- 172.21.15.107:0/4211255749 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fccc810a820 con 0x7fccc8107670 2024-04-25T00:15:47.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.953+0000 7fccbe7fc700 1 -- 172.21.15.107:0/4211255749 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fccc009b150 con 0x7fccc8107670 2024-04-25T00:15:47.953 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:15:47.954 INFO:teuthology.orchestra.run.smithi107.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T00:15:47.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.955+0000 7fcccf54e700 1 -- 172.21.15.107:0/4211255749 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fccb40a2c20 msgr2=0x7fccb40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:47.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.955+0000 7fcccf54e700 1 --2- 172.21.15.107:0/4211255749 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fccb40a2c20 0x7fccb40a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fccc8199a20 tx=0x7fccc4008040 comp rx=0 tx=0).stop 2024-04-25T00:15:47.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.955+0000 7fcccf54e700 1 -- 172.21.15.107:0/4211255749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccc8107670 msgr2=0x7fccc81988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:47.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.955+0000 7fcccf54e700 1 --2- 172.21.15.107:0/4211255749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccc8107670 0x7fccc81988e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fccc000b970 tx=0x7fccc000bd30 comp rx=0 tx=0).stop 2024-04-25T00:15:47.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.955+0000 7fcccf54e700 1 -- 172.21.15.107:0/4211255749 shutdown_connections 2024-04-25T00:15:47.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.955+0000 7fcccf54e700 1 --2- 172.21.15.107:0/4211255749 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fccb40a2c20 0x7fccb40a50e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:47.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.955+0000 7fcccf54e700 1 --2- 172.21.15.107:0/4211255749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccc8107670 0x7fccc81988e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:47.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.955+0000 7fcccf54e700 1 --2- 172.21.15.107:0/4211255749 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fccc8100e50 0x7fccc819e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:47.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.955+0000 7fcccf54e700 1 --2- 172.21.15.107:0/4211255749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccc8101800 0x7fccc819ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:47.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.955+0000 7fcccf54e700 1 -- 172.21.15.107:0/4211255749 >> 172.21.15.107:0/4211255749 conn(0x7fccc8076450 msgr2=0x7fccc80fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:47.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.955+0000 7fcccf54e700 1 -- 172.21.15.107:0/4211255749 shutdown_connections 2024-04-25T00:15:47.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:47.955+0000 7fcccf54e700 1 -- 172.21.15.107:0/4211255749 wait complete. 2024-04-25T00:15:48.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:48 smithi107 ceph-mon[29153]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-25T00:15:48.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:48 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/4211255749' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T00:15:48.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:48 smithi195 ceph-mon[32833]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-25T00:15:48.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:48 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/4211255749' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T00:15:48.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:48 smithi170 ceph-mon[32807]: pgmap v347: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 6 op/s 2024-04-25T00:15:48.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:48 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/4211255749' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T00:15:48.733 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T00:15:48.734 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:15:48.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.836+0000 7f9ad613b640 1 -- 172.21.15.107:0/1126445182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ad0102240 msgr2=0x7f9ad0108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:48.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.836+0000 7f9ad613b640 1 --2- 172.21.15.107:0/1126445182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ad0102240 0x7f9ad0108ad0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009ca0 tx=0x7f9ac802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:15:48.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.836+0000 7f9ad613b640 1 -- 172.21.15.107:0/1126445182 shutdown_connections 2024-04-25T00:15:48.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.836+0000 7f9ad613b640 1 --2- 172.21.15.107:0/1126445182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ad0102240 0x7f9ad0108ad0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:48.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.836+0000 7f9ad613b640 1 --2- 172.21.15.107:0/1126445182 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ad0101880 0x7f9ad0101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:48.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.836+0000 7f9ad613b640 1 --2- 172.21.15.107:0/1126445182 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ad0100690 0x7f9ad0100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:48.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.836+0000 7f9ad613b640 1 -- 172.21.15.107:0/1126445182 >> 172.21.15.107:0/1126445182 conn(0x7f9ad00fbc20 msgr2=0x7f9ad00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:48.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.836+0000 7f9ad613b640 1 -- 172.21.15.107:0/1126445182 shutdown_connections 2024-04-25T00:15:48.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.836+0000 7f9ad613b640 1 -- 172.21.15.107:0/1126445182 wait complete. 2024-04-25T00:15:48.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.837+0000 7f9ad613b640 1 Processor -- start 2024-04-25T00:15:48.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.837+0000 7f9ad613b640 1 -- start start 2024-04-25T00:15:48.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.837+0000 7f9ad613b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ad0100690 0x7f9ad006a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:48.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.837+0000 7f9ad613b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ad0101880 0x7f9ad019c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:48.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.837+0000 7f9ad613b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ad0102240 0x7f9ad019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:48.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.837+0000 7f9ad613b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad010c310 con 0x7f9ad0100690 2024-04-25T00:15:48.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.837+0000 7f9ad613b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad010c190 con 0x7f9ad0101880 2024-04-25T00:15:48.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.837+0000 7f9ad613b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad010c490 con 0x7f9ad0102240 2024-04-25T00:15:48.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.837+0000 7f9acf7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ad0100690 0x7f9ad006a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:48.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.837+0000 7f9acf7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ad0100690 0x7f9ad006a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38210/0 (socket says 172.21.15.107:38210) 2024-04-25T00:15:48.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.837+0000 7f9acf7fe640 1 -- 172.21.15.107:0/2462134916 learned_addr learned my addr 172.21.15.107:0/2462134916 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:48.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.837+0000 7f9acffff640 1 --2- 172.21.15.107:0/2462134916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ad0102240 0x7f9ad019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:48.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.837+0000 7f9aceffd640 1 --2- 172.21.15.107:0/2462134916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ad0101880 0x7f9ad019c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:48.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.837+0000 7f9acf7fe640 1 -- 172.21.15.107:0/2462134916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ad0102240 msgr2=0x7f9ad019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:48.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.837+0000 7f9acf7fe640 1 --2- 172.21.15.107:0/2462134916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ad0102240 0x7f9ad019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:48.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.837+0000 7f9acf7fe640 1 -- 172.21.15.107:0/2462134916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ad0101880 msgr2=0x7f9ad019c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:48.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.837+0000 7f9acf7fe640 1 --2- 172.21.15.107:0/2462134916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ad0101880 0x7f9ad019c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:48.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.837+0000 7f9acf7fe640 1 -- 172.21.15.107:0/2462134916 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad019f6b0 con 0x7f9ad0100690 2024-04-25T00:15:48.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.838+0000 7f9acf7fe640 1 --2- 172.21.15.107:0/2462134916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ad0100690 0x7f9ad006a910 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400d950 tx=0x7f9ac400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:48.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.838+0000 7f9accff9640 1 -- 172.21.15.107:0/2462134916 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4017070 con 0x7f9ad0100690 2024-04-25T00:15:48.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.838+0000 7f9ad613b640 1 -- 172.21.15.107:0/2462134916 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad019f940 con 0x7f9ad0100690 2024-04-25T00:15:48.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.838+0000 7f9accff9640 1 -- 172.21.15.107:0/2462134916 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ac4009c30 con 0x7f9ad0100690 2024-04-25T00:15:48.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.838+0000 7f9accff9640 1 -- 172.21.15.107:0/2462134916 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400f600 con 0x7f9ad0100690 2024-04-25T00:15:48.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.838+0000 7f9ad613b640 1 -- 172.21.15.107:0/2462134916 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad019fe90 con 0x7f9ad0100690 2024-04-25T00:15:48.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.839+0000 7f9ad613b640 1 -- 172.21.15.107:0/2462134916 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ad00689b0 con 0x7f9ad0100690 2024-04-25T00:15:48.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.840+0000 7f9accff9640 1 -- 172.21.15.107:0/2462134916 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9ac401a070 con 0x7f9ad0100690 2024-04-25T00:15:48.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.841+0000 7f9accff9640 1 --2- 172.21.15.107:0/2462134916 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f9ab80a2b90 0x7f9ab80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:48.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.841+0000 7f9aceffd640 1 --2- 172.21.15.107:0/2462134916 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f9ab80a2b90 0x7f9ab80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:48.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.841+0000 7f9accff9640 1 -- 172.21.15.107:0/2462134916 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f9ac40cee00 con 0x7f9ad0100690 2024-04-25T00:15:48.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.841+0000 7f9aceffd640 1 --2- 172.21.15.107:0/2462134916 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f9ab80a2b90 0x7f9ab80a5050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0005fd0 tx=0x7f9ac0005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:48.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:48.845+0000 7f9accff9640 1 -- 172.21.15.107:0/2462134916 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9ac4009de0 con 0x7f9ad0100690 2024-04-25T00:15:49.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.025+0000 7f9ad613b640 1 -- 172.21.15.107:0/2462134916 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ad0062340 con 0x7f9ad0100690 2024-04-25T00:15:49.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.027+0000 7f9accff9640 1 -- 172.21.15.107:0/2462134916 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24576 (secure 0 0 0) 0x7f9ac409ba20 con 0x7f9ad0100690 2024-04-25T00:15:49.028 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:15:49.028 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":94,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:15:02.070002+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:59.672801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:00.957782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:14:05.065846+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8},{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:15:49.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.029+0000 7f9ad613b640 1 -- 172.21.15.107:0/2462134916 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f9ab80a2b90 msgr2=0x7f9ab80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:49.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.029+0000 7f9ad613b640 1 --2- 172.21.15.107:0/2462134916 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f9ab80a2b90 0x7f9ab80a5050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9ac0005fd0 tx=0x7f9ac0005850 comp rx=0 tx=0).stop 2024-04-25T00:15:49.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.029+0000 7f9ad613b640 1 -- 172.21.15.107:0/2462134916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ad0100690 msgr2=0x7f9ad006a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:49.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.029+0000 7f9ad613b640 1 --2- 172.21.15.107:0/2462134916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ad0100690 0x7f9ad006a910 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400d950 tx=0x7f9ac400de10 comp rx=0 tx=0).stop 2024-04-25T00:15:49.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.029+0000 7f9ad613b640 1 -- 172.21.15.107:0/2462134916 shutdown_connections 2024-04-25T00:15:49.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.029+0000 7f9ad613b640 1 --2- 172.21.15.107:0/2462134916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ad0102240 0x7f9ad019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.029+0000 7f9ad613b640 1 --2- 172.21.15.107:0/2462134916 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f9ab80a2b90 0x7f9ab80a5050 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.029+0000 7f9ad613b640 1 --2- 172.21.15.107:0/2462134916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ad0101880 0x7f9ad019c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.029+0000 7f9ad613b640 1 --2- 172.21.15.107:0/2462134916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ad0100690 0x7f9ad006a910 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.029+0000 7f9ad613b640 1 -- 172.21.15.107:0/2462134916 >> 172.21.15.107:0/2462134916 conn(0x7f9ad00fbc20 msgr2=0x7f9ad0104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:49.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.029+0000 7f9ad613b640 1 -- 172.21.15.107:0/2462134916 shutdown_connections 2024-04-25T00:15:49.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.029+0000 7f9ad613b640 1 -- 172.21.15.107:0/2462134916 wait complete. 2024-04-25T00:15:49.041 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:09:25.160329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:13:59.672801+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:14:00.957782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:14:05.065846+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:15:49.041 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:15:49.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.142+0000 7fbe17a2a640 1 -- 172.21.15.107:0/1407856212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe10102230 msgr2=0x7fbe10108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:49.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.142+0000 7fbe17a2a640 1 --2- 172.21.15.107:0/1407856212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe10102230 0x7fbe10108ac0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fbe08009ca0 tx=0x7fbe0802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.142+0000 7fbe17a2a640 1 -- 172.21.15.107:0/1407856212 shutdown_connections 2024-04-25T00:15:49.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.142+0000 7fbe17a2a640 1 --2- 172.21.15.107:0/1407856212 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe10102230 0x7fbe10108ac0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.142+0000 7fbe17a2a640 1 --2- 172.21.15.107:0/1407856212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe10101870 0x7fbe10101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.142+0000 7fbe17a2a640 1 --2- 172.21.15.107:0/1407856212 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe10100680 0x7fbe10100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.142+0000 7fbe17a2a640 1 -- 172.21.15.107:0/1407856212 >> 172.21.15.107:0/1407856212 conn(0x7fbe100fbbf0 msgr2=0x7fbe100fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:49.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.142+0000 7fbe17a2a640 1 -- 172.21.15.107:0/1407856212 shutdown_connections 2024-04-25T00:15:49.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.142+0000 7fbe17a2a640 1 -- 172.21.15.107:0/1407856212 wait complete. 2024-04-25T00:15:49.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.143+0000 7fbe17a2a640 1 Processor -- start 2024-04-25T00:15:49.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.143+0000 7fbe17a2a640 1 -- start start 2024-04-25T00:15:49.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.143+0000 7fbe17a2a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe10100680 0x7fbe10069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:49.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.143+0000 7fbe17a2a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe10101870 0x7fbe1019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:49.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.143+0000 7fbe17a2a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe10102230 0x7fbe1019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:49.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.143+0000 7fbe17a2a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1010c360 con 0x7fbe10102230 2024-04-25T00:15:49.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.143+0000 7fbe17a2a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1010c1e0 con 0x7fbe10100680 2024-04-25T00:15:49.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.143+0000 7fbe17a2a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1010c4e0 con 0x7fbe10101870 2024-04-25T00:15:49.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.143+0000 7fbe15fc7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe10102230 0x7fbe1019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:49.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.143+0000 7fbe15fc7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe10102230 0x7fbe1019ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38224/0 (socket says 172.21.15.107:38224) 2024-04-25T00:15:49.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.143+0000 7fbe15fc7640 1 -- 172.21.15.107:0/1824652042 learned_addr learned my addr 172.21.15.107:0/1824652042 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:49.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.144+0000 7fbe157c6640 1 --2- 172.21.15.107:0/1824652042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe10100680 0x7fbe10069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:49.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.144+0000 7fbe14fc5640 1 --2- 172.21.15.107:0/1824652042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe10101870 0x7fbe1019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:49.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.144+0000 7fbe15fc7640 1 -- 172.21.15.107:0/1824652042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe10101870 msgr2=0x7fbe1019c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:49.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.144+0000 7fbe15fc7640 1 --2- 172.21.15.107:0/1824652042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe10101870 0x7fbe1019c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.144+0000 7fbe15fc7640 1 -- 172.21.15.107:0/1824652042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe10100680 msgr2=0x7fbe10069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:49.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.144+0000 7fbe15fc7640 1 --2- 172.21.15.107:0/1824652042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe10100680 0x7fbe10069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.144+0000 7fbe15fc7640 1 -- 172.21.15.107:0/1824652042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1019f6f0 con 0x7fbe10102230 2024-04-25T00:15:49.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.144+0000 7fbe157c6640 1 --2- 172.21.15.107:0/1824652042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe10100680 0x7fbe10069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:49.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.144+0000 7fbe15fc7640 1 --2- 172.21.15.107:0/1824652042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe10102230 0x7fbe1019ede0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbe080098d0 tx=0x7fbe0802f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:49.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.144+0000 7fbe067fc640 1 -- 172.21.15.107:0/1824652042 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0802fc30 con 0x7fbe10102230 2024-04-25T00:15:49.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.144+0000 7fbe17a2a640 1 -- 172.21.15.107:0/1824652042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe1019f920 con 0x7fbe10102230 2024-04-25T00:15:49.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.144+0000 7fbe067fc640 1 -- 172.21.15.107:0/1824652042 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe08003e50 con 0x7fbe10102230 2024-04-25T00:15:49.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.144+0000 7fbe067fc640 1 -- 172.21.15.107:0/1824652042 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe080305c0 con 0x7fbe10102230 2024-04-25T00:15:49.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.144+0000 7fbe17a2a640 1 -- 172.21.15.107:0/1824652042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe1019fef0 con 0x7fbe10102230 2024-04-25T00:15:49.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.145+0000 7fbe17a2a640 1 -- 172.21.15.107:0/1824652042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe10107940 con 0x7fbe10102230 2024-04-25T00:15:49.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.146+0000 7fbe067fc640 1 -- 172.21.15.107:0/1824652042 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbe08003970 con 0x7fbe10102230 2024-04-25T00:15:49.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.147+0000 7fbe067fc640 1 --2- 172.21.15.107:0/1824652042 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fbdfc0a2b90 0x7fbdfc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:49.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.147+0000 7fbe157c6640 1 --2- 172.21.15.107:0/1824652042 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fbdfc0a2b90 0x7fbdfc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:49.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.147+0000 7fbe067fc640 1 -- 172.21.15.107:0/1824652042 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fbe080f0990 con 0x7fbe10102230 2024-04-25T00:15:49.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.147+0000 7fbe157c6640 1 --2- 172.21.15.107:0/1824652042 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fbdfc0a2b90 0x7fbdfc0a5050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbe101016d0 tx=0x7fbe0c005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:49.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.150+0000 7fbe067fc640 1 -- 172.21.15.107:0/1824652042 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbe080bd5b0 con 0x7fbe10102230 2024-04-25T00:15:49.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.326+0000 7fbe17a2a640 1 -- 172.21.15.107:0/1824652042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbe1002d300 con 0x7fbe10102230 2024-04-25T00:15:49.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.326+0000 7fbe067fc640 1 -- 172.21.15.107:0/1824652042 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fbe080bceb0 con 0x7fbe10102230 2024-04-25T00:15:49.327 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-04-25T00:15:49.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.328+0000 7fbe17a2a640 1 -- 172.21.15.107:0/1824652042 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fbdfc0a2b90 msgr2=0x7fbdfc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:49.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.328+0000 7fbe17a2a640 1 --2- 172.21.15.107:0/1824652042 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fbdfc0a2b90 0x7fbdfc0a5050 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbe101016d0 tx=0x7fbe0c005db0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.328+0000 7fbe17a2a640 1 -- 172.21.15.107:0/1824652042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe10102230 msgr2=0x7fbe1019ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:49.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.328+0000 7fbe17a2a640 1 --2- 172.21.15.107:0/1824652042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe10102230 0x7fbe1019ede0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbe080098d0 tx=0x7fbe0802f840 comp rx=0 tx=0).stop 2024-04-25T00:15:49.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.328+0000 7fbe17a2a640 1 -- 172.21.15.107:0/1824652042 shutdown_connections 2024-04-25T00:15:49.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.328+0000 7fbe17a2a640 1 --2- 172.21.15.107:0/1824652042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe10102230 0x7fbe1019ede0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.328+0000 7fbe17a2a640 1 --2- 172.21.15.107:0/1824652042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbe10101870 0x7fbe1019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.328+0000 7fbe17a2a640 1 --2- 172.21.15.107:0/1824652042 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fbdfc0a2b90 0x7fbdfc0a5050 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.328+0000 7fbe17a2a640 1 --2- 172.21.15.107:0/1824652042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe10100680 0x7fbe10069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.328+0000 7fbe17a2a640 1 -- 172.21.15.107:0/1824652042 >> 172.21.15.107:0/1824652042 conn(0x7fbe100fbbf0 msgr2=0x7fbe10104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:49.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.328+0000 7fbe17a2a640 1 -- 172.21.15.107:0/1824652042 shutdown_connections 2024-04-25T00:15:49.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.328+0000 7fbe17a2a640 1 -- 172.21.15.107:0/1824652042 wait complete. 2024-04-25T00:15:49.340 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:15:49.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.440+0000 7f371bb7e640 1 -- 172.21.15.107:0/1575677244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37141006a0 msgr2=0x7f3714100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:49.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.440+0000 7f371bb7e640 1 --2- 172.21.15.107:0/1575677244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37141006a0 0x7f3714100ac0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3704009900 tx=0x7f370402a410 comp rx=0 tx=0).stop 2024-04-25T00:15:49.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.440+0000 7f371bb7e640 1 -- 172.21.15.107:0/1575677244 shutdown_connections 2024-04-25T00:15:49.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.440+0000 7f371bb7e640 1 --2- 172.21.15.107:0/1575677244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3714102250 0x7f3714108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.440+0000 7f371bb7e640 1 --2- 172.21.15.107:0/1575677244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3714101890 0x7f3714101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.440+0000 7f371bb7e640 1 --2- 172.21.15.107:0/1575677244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37141006a0 0x7f3714100ac0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.440+0000 7f371bb7e640 1 -- 172.21.15.107:0/1575677244 >> 172.21.15.107:0/1575677244 conn(0x7f37140fbc30 msgr2=0x7f37140fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:49.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.440+0000 7f371bb7e640 1 -- 172.21.15.107:0/1575677244 shutdown_connections 2024-04-25T00:15:49.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.440+0000 7f371bb7e640 1 -- 172.21.15.107:0/1575677244 wait complete. 2024-04-25T00:15:49.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.441+0000 7f371bb7e640 1 Processor -- start 2024-04-25T00:15:49.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.441+0000 7f371bb7e640 1 -- start start 2024-04-25T00:15:49.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.441+0000 7f371bb7e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37141006a0 0x7f3714196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:49.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.441+0000 7f371bb7e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3714101890 0x7f3714196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:49.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.441+0000 7f371bb7e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3714102250 0x7f371419e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:49.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.441+0000 7f371bb7e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f371410c3c0 con 0x7f3714102250 2024-04-25T00:15:49.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.441+0000 7f371bb7e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f371410c240 con 0x7f3714101890 2024-04-25T00:15:49.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.441+0000 7f371bb7e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f371410c540 con 0x7f37141006a0 2024-04-25T00:15:49.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.441+0000 7f371a11b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3714102250 0x7f371419e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:49.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.441+0000 7f371a11b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3714102250 0x7f371419e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40136/0 (socket says 172.21.15.107:40136) 2024-04-25T00:15:49.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.441+0000 7f371a11b640 1 -- 172.21.15.107:0/4020941227 learned_addr learned my addr 172.21.15.107:0/4020941227 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:49.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.441+0000 7f371991a640 1 --2- 172.21.15.107:0/4020941227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37141006a0 0x7f3714196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:49.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.441+0000 7f3719119640 1 --2- 172.21.15.107:0/4020941227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3714101890 0x7f3714196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:49.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.441+0000 7f371a11b640 1 -- 172.21.15.107:0/4020941227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37141006a0 msgr2=0x7f3714196a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:49.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.441+0000 7f371a11b640 1 --2- 172.21.15.107:0/4020941227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37141006a0 0x7f3714196a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.441+0000 7f371a11b640 1 -- 172.21.15.107:0/4020941227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3714101890 msgr2=0x7f3714196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:49.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.441+0000 7f371a11b640 1 --2- 172.21.15.107:0/4020941227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3714101890 0x7f3714196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.441+0000 7f371a11b640 1 -- 172.21.15.107:0/4020941227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f371419e690 con 0x7f3714102250 2024-04-25T00:15:49.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.441+0000 7f371a11b640 1 --2- 172.21.15.107:0/4020941227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3714102250 0x7f371419e020 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f370c00cce0 tx=0x7f370c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:49.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.441+0000 7f370affd640 1 -- 172.21.15.107:0/4020941227 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c00f450 con 0x7f3714102250 2024-04-25T00:15:49.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.442+0000 7f370affd640 1 -- 172.21.15.107:0/4020941227 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f370c00fad0 con 0x7f3714102250 2024-04-25T00:15:49.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.442+0000 7f371bb7e640 1 -- 172.21.15.107:0/4020941227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f371419e980 con 0x7f3714102250 2024-04-25T00:15:49.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.442+0000 7f370affd640 1 -- 172.21.15.107:0/4020941227 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c018b60 con 0x7f3714102250 2024-04-25T00:15:49.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.442+0000 7f371bb7e640 1 -- 172.21.15.107:0/4020941227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f371419ef00 con 0x7f3714102250 2024-04-25T00:15:49.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.442+0000 7f371bb7e640 1 -- 172.21.15.107:0/4020941227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3714107960 con 0x7f3714102250 2024-04-25T00:15:49.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.444+0000 7f370affd640 1 -- 172.21.15.107:0/4020941227 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f370c00fc80 con 0x7f3714102250 2024-04-25T00:15:49.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.444+0000 7f370affd640 1 --2- 172.21.15.107:0/4020941227 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f37000a2b90 0x7f37000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:49.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.444+0000 7f371991a640 1 --2- 172.21.15.107:0/4020941227 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f37000a2b90 0x7f37000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:49.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.444+0000 7f370affd640 1 -- 172.21.15.107:0/4020941227 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f370c0cfc30 con 0x7f3714102250 2024-04-25T00:15:49.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.445+0000 7f371991a640 1 --2- 172.21.15.107:0/4020941227 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f37000a2b90 0x7f37000a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3704031440 tx=0x7f37040315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:49.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.448+0000 7f370affd640 1 -- 172.21.15.107:0/4020941227 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f370c014030 con 0x7f3714102250 2024-04-25T00:15:49.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.622+0000 7f371bb7e640 1 -- 172.21.15.107:0/4020941227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3714062340 con 0x7f3714102250 2024-04-25T00:15:49.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.623+0000 7f370affd640 1 -- 172.21.15.107:0/4020941227 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f370c09c850 con 0x7f3714102250 2024-04-25T00:15:49.623 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-25T00:15:49.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.625+0000 7f371bb7e640 1 -- 172.21.15.107:0/4020941227 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f37000a2b90 msgr2=0x7f37000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:49.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.625+0000 7f371bb7e640 1 --2- 172.21.15.107:0/4020941227 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f37000a2b90 0x7f37000a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3704031440 tx=0x7f37040315b0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.625+0000 7f371bb7e640 1 -- 172.21.15.107:0/4020941227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3714102250 msgr2=0x7f371419e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:49.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.625+0000 7f371bb7e640 1 --2- 172.21.15.107:0/4020941227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3714102250 0x7f371419e020 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f370c00cce0 tx=0x7f370c00a620 comp rx=0 tx=0).stop 2024-04-25T00:15:49.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.625+0000 7f371bb7e640 1 -- 172.21.15.107:0/4020941227 shutdown_connections 2024-04-25T00:15:49.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.625+0000 7f371bb7e640 1 --2- 172.21.15.107:0/4020941227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3714102250 0x7f371419e020 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.625+0000 7f371bb7e640 1 --2- 172.21.15.107:0/4020941227 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f37000a2b90 0x7f37000a5050 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.625+0000 7f371bb7e640 1 --2- 172.21.15.107:0/4020941227 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3714101890 0x7f3714196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.625+0000 7f371bb7e640 1 --2- 172.21.15.107:0/4020941227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f37141006a0 0x7f3714196a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.625+0000 7f371bb7e640 1 -- 172.21.15.107:0/4020941227 >> 172.21.15.107:0/4020941227 conn(0x7f37140fbc30 msgr2=0x7f3714104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:49.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.625+0000 7f371bb7e640 1 -- 172.21.15.107:0/4020941227 shutdown_connections 2024-04-25T00:15:49.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.625+0000 7f371bb7e640 1 -- 172.21.15.107:0/4020941227 wait complete. 2024-04-25T00:15:49.636 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:15:49.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.740+0000 7efe837c1640 1 -- 172.21.15.107:0/2351856162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe7c101870 msgr2=0x7efe7c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:49.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.740+0000 7efe837c1640 1 --2- 172.21.15.107:0/2351856162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe7c101870 0x7efe7c101cf0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009900 tx=0x7efe6c02a410 comp rx=0 tx=0).stop 2024-04-25T00:15:49.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.740+0000 7efe837c1640 1 -- 172.21.15.107:0/2351856162 shutdown_connections 2024-04-25T00:15:49.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.740+0000 7efe837c1640 1 --2- 172.21.15.107:0/2351856162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe7c102230 0x7efe7c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.740+0000 7efe837c1640 1 --2- 172.21.15.107:0/2351856162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe7c101870 0x7efe7c101cf0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.740+0000 7efe837c1640 1 --2- 172.21.15.107:0/2351856162 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe7c100680 0x7efe7c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.740+0000 7efe837c1640 1 -- 172.21.15.107:0/2351856162 >> 172.21.15.107:0/2351856162 conn(0x7efe7c0fbbf0 msgr2=0x7efe7c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.740+0000 7efe837c1640 1 -- 172.21.15.107:0/2351856162 shutdown_connections 2024-04-25T00:15:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.740+0000 7efe837c1640 1 -- 172.21.15.107:0/2351856162 wait complete. 2024-04-25T00:15:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.741+0000 7efe837c1640 1 Processor -- start 2024-04-25T00:15:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.741+0000 7efe837c1640 1 -- start start 2024-04-25T00:15:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.741+0000 7efe837c1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe7c100680 0x7efe7c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.741+0000 7efe837c1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe7c101870 0x7efe7c19c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.741+0000 7efe837c1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe7c102230 0x7efe7c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.741+0000 7efe837c1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c10c3c0 con 0x7efe7c101870 2024-04-25T00:15:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.741+0000 7efe837c1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c10c240 con 0x7efe7c102230 2024-04-25T00:15:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.741+0000 7efe837c1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efe7c10c540 con 0x7efe7c100680 2024-04-25T00:15:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.741+0000 7efe80d5c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe7c101870 0x7efe7c19c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.741+0000 7efe80d5c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe7c101870 0x7efe7c19c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40144/0 (socket says 172.21.15.107:40144) 2024-04-25T00:15:49.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.741+0000 7efe80d5c640 1 -- 172.21.15.107:0/1334071397 learned_addr learned my addr 172.21.15.107:0/1334071397 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.741+0000 7efe81d5e640 1 --2- 172.21.15.107:0/1334071397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe7c102230 0x7efe7c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.741+0000 7efe8155d640 1 --2- 172.21.15.107:0/1334071397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe7c100680 0x7efe7c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.741+0000 7efe80d5c640 1 -- 172.21.15.107:0/1334071397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe7c100680 msgr2=0x7efe7c069460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.741+0000 7efe80d5c640 1 --2- 172.21.15.107:0/1334071397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe7c100680 0x7efe7c069460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.741+0000 7efe80d5c640 1 -- 172.21.15.107:0/1334071397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe7c102230 msgr2=0x7efe7c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.741+0000 7efe80d5c640 1 --2- 172.21.15.107:0/1334071397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe7c102230 0x7efe7c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.741+0000 7efe80d5c640 1 -- 172.21.15.107:0/1334071397 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7c19f7c0 con 0x7efe7c101870 2024-04-25T00:15:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.741+0000 7efe8155d640 1 --2- 172.21.15.107:0/1334071397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe7c100680 0x7efe7c069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.741+0000 7efe80d5c640 1 --2- 172.21.15.107:0/1334071397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe7c101870 0x7efe7c19c8c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7efe6c031440 tx=0x7efe6c032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.741+0000 7efe727fc640 1 -- 172.21.15.107:0/1334071397 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c03f030 con 0x7efe7c101870 2024-04-25T00:15:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.741+0000 7efe727fc640 1 -- 172.21.15.107:0/1334071397 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe6c003ce0 con 0x7efe7c101870 2024-04-25T00:15:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.741+0000 7efe837c1640 1 -- 172.21.15.107:0/1334071397 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe7c19fa50 con 0x7efe7c101870 2024-04-25T00:15:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.741+0000 7efe727fc640 1 -- 172.21.15.107:0/1334071397 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c035070 con 0x7efe7c101870 2024-04-25T00:15:49.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.742+0000 7efe837c1640 1 -- 172.21.15.107:0/1334071397 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe7c19ffc0 con 0x7efe7c101870 2024-04-25T00:15:49.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.743+0000 7efe837c1640 1 -- 172.21.15.107:0/1334071397 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe7c1079f0 con 0x7efe7c101870 2024-04-25T00:15:49.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.744+0000 7efe727fc640 1 -- 172.21.15.107:0/1334071397 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7efe6c042030 con 0x7efe7c101870 2024-04-25T00:15:49.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.744+0000 7efe727fc640 1 --2- 172.21.15.107:0/1334071397 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7efe680a2b90 0x7efe680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:49.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.745+0000 7efe8155d640 1 --2- 172.21.15.107:0/1334071397 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7efe680a2b90 0x7efe680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:49.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.745+0000 7efe727fc640 1 -- 172.21.15.107:0/1334071397 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7efe6c0efa90 con 0x7efe7c101870 2024-04-25T00:15:49.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.745+0000 7efe8155d640 1 --2- 172.21.15.107:0/1334071397 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7efe680a2b90 0x7efe680a5050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efe78006fd0 tx=0x7efe78008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:49.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.749+0000 7efe727fc640 1 -- 172.21.15.107:0/1334071397 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efe6c0bc6b0 con 0x7efe7c101870 2024-04-25T00:15:49.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.923+0000 7efe837c1640 1 -- 172.21.15.107:0/1334071397 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efe7c062340 con 0x7efe7c101870 2024-04-25T00:15:49.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.924+0000 7efe727fc640 1 -- 172.21.15.107:0/1334071397 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7efe6c0bbfb0 con 0x7efe7c101870 2024-04-25T00:15:49.924 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-04-25T00:15:49.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.926+0000 7efe837c1640 1 -- 172.21.15.107:0/1334071397 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7efe680a2b90 msgr2=0x7efe680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:49.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.926+0000 7efe837c1640 1 --2- 172.21.15.107:0/1334071397 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7efe680a2b90 0x7efe680a5050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efe78006fd0 tx=0x7efe78008040 comp rx=0 tx=0).stop 2024-04-25T00:15:49.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.926+0000 7efe837c1640 1 -- 172.21.15.107:0/1334071397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe7c101870 msgr2=0x7efe7c19c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:49.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.926+0000 7efe837c1640 1 --2- 172.21.15.107:0/1334071397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe7c101870 0x7efe7c19c8c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7efe6c031440 tx=0x7efe6c032e20 comp rx=0 tx=0).stop 2024-04-25T00:15:49.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.926+0000 7efe837c1640 1 -- 172.21.15.107:0/1334071397 shutdown_connections 2024-04-25T00:15:49.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.926+0000 7efe837c1640 1 --2- 172.21.15.107:0/1334071397 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7efe680a2b90 0x7efe680a5050 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.926+0000 7efe837c1640 1 --2- 172.21.15.107:0/1334071397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe7c102230 0x7efe7c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.926+0000 7efe837c1640 1 --2- 172.21.15.107:0/1334071397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe7c101870 0x7efe7c19c8c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.926+0000 7efe837c1640 1 --2- 172.21.15.107:0/1334071397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe7c100680 0x7efe7c069460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:49.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.926+0000 7efe837c1640 1 -- 172.21.15.107:0/1334071397 >> 172.21.15.107:0/1334071397 conn(0x7efe7c0fbbf0 msgr2=0x7efe7c104bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:49.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.926+0000 7efe837c1640 1 -- 172.21.15.107:0/1334071397 shutdown_connections 2024-04-25T00:15:49.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:49.926+0000 7efe837c1640 1 -- 172.21.15.107:0/1334071397 wait complete. 2024-04-25T00:15:49.937 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:15:50.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.038+0000 7fe7e29c8640 1 -- 172.21.15.107:0/2379394065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7dc1001a0 msgr2=0x7fe7dc109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:50.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.038+0000 7fe7e29c8640 1 --2- 172.21.15.107:0/2379394065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7dc1001a0 0x7fe7dc109d50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4009ca0 tx=0x7fe7d402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.038+0000 7fe7e29c8640 1 -- 172.21.15.107:0/2379394065 shutdown_connections 2024-04-25T00:15:50.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.038+0000 7fe7e29c8640 1 --2- 172.21.15.107:0/2379394065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7dc1001a0 0x7fe7dc109d50 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.038+0000 7fe7e29c8640 1 --2- 172.21.15.107:0/2379394065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7dc0ff7e0 0x7fe7dc0ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.038+0000 7fe7e29c8640 1 --2- 172.21.15.107:0/2379394065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7dc102d70 0x7fe7dc103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.038+0000 7fe7e29c8640 1 -- 172.21.15.107:0/2379394065 >> 172.21.15.107:0/2379394065 conn(0x7fe7dc0fb380 msgr2=0x7fe7dc0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:50.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.038+0000 7fe7e29c8640 1 -- 172.21.15.107:0/2379394065 shutdown_connections 2024-04-25T00:15:50.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.038+0000 7fe7e29c8640 1 -- 172.21.15.107:0/2379394065 wait complete. 2024-04-25T00:15:50.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.039+0000 7fe7e29c8640 1 Processor -- start 2024-04-25T00:15:50.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.039+0000 7fe7e29c8640 1 -- start start 2024-04-25T00:15:50.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.039+0000 7fe7e29c8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7dc0ff7e0 0x7fe7dc102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:50.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.039+0000 7fe7e29c8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7dc1001a0 0x7fe7dc100ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:50.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.039+0000 7fe7e29c8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7dc102d70 0x7fe7dc100ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:50.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.039+0000 7fe7e29c8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc10d3f0 con 0x7fe7dc1001a0 2024-04-25T00:15:50.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.039+0000 7fe7e29c8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc10d270 con 0x7fe7dc102d70 2024-04-25T00:15:50.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.039+0000 7fe7e29c8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe7dc10d570 con 0x7fe7dc0ff7e0 2024-04-25T00:15:50.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.039+0000 7fe7db7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7dc1001a0 0x7fe7dc100ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:50.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.039+0000 7fe7db7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7dc1001a0 0x7fe7dc100ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40168/0 (socket says 172.21.15.107:40168) 2024-04-25T00:15:50.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.039+0000 7fe7db7fe640 1 -- 172.21.15.107:0/3965240229 learned_addr learned my addr 172.21.15.107:0/3965240229 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:50.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.040+0000 7fe7e0f65640 1 --2- 172.21.15.107:0/3965240229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7dc102d70 0x7fe7dc100ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:50.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.040+0000 7fe7db7fe640 1 -- 172.21.15.107:0/3965240229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7dc0ff7e0 msgr2=0x7fe7dc102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:50.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.040+0000 7fe7dbfff640 1 --2- 172.21.15.107:0/3965240229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7dc0ff7e0 0x7fe7dc102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:50.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.040+0000 7fe7db7fe640 1 --2- 172.21.15.107:0/3965240229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7dc0ff7e0 0x7fe7dc102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.040+0000 7fe7db7fe640 1 -- 172.21.15.107:0/3965240229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7dc102d70 msgr2=0x7fe7dc100ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:50.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.040+0000 7fe7db7fe640 1 --2- 172.21.15.107:0/3965240229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7dc102d70 0x7fe7dc100ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.040+0000 7fe7db7fe640 1 -- 172.21.15.107:0/3965240229 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7dc101880 con 0x7fe7dc1001a0 2024-04-25T00:15:50.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.040+0000 7fe7e0f65640 1 --2- 172.21.15.107:0/3965240229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7dc102d70 0x7fe7dc100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:50.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.040+0000 7fe7dbfff640 1 --2- 172.21.15.107:0/3965240229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7dc0ff7e0 0x7fe7dc102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:50.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.040+0000 7fe7db7fe640 1 --2- 172.21.15.107:0/3965240229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7dc1001a0 0x7fe7dc100ab0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000d950 tx=0x7fe7d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:50.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.040+0000 7fe7d97fa640 1 -- 172.21.15.107:0/3965240229 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0017030 con 0x7fe7dc1001a0 2024-04-25T00:15:50.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.040+0000 7fe7d97fa640 1 -- 172.21.15.107:0/3965240229 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7d0004020 con 0x7fe7dc1001a0 2024-04-25T00:15:50.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.040+0000 7fe7e29c8640 1 -- 172.21.15.107:0/3965240229 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7dc1a1b20 con 0x7fe7dc1001a0 2024-04-25T00:15:50.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.040+0000 7fe7d97fa640 1 -- 172.21.15.107:0/3965240229 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d00047a0 con 0x7fe7dc1001a0 2024-04-25T00:15:50.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.040+0000 7fe7e29c8640 1 -- 172.21.15.107:0/3965240229 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7dc1a2070 con 0x7fe7dc1001a0 2024-04-25T00:15:50.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.041+0000 7fe7e29c8640 1 -- 172.21.15.107:0/3965240229 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7dc108c30 con 0x7fe7dc1001a0 2024-04-25T00:15:50.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.042+0000 7fe7d97fa640 1 -- 172.21.15.107:0/3965240229 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe7d00041d0 con 0x7fe7dc1001a0 2024-04-25T00:15:50.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.043+0000 7fe7d97fa640 1 --2- 172.21.15.107:0/3965240229 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe7c40a2b90 0x7fe7c40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:50.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.043+0000 7fe7dbfff640 1 --2- 172.21.15.107:0/3965240229 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe7c40a2b90 0x7fe7c40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:50.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.043+0000 7fe7d97fa640 1 -- 172.21.15.107:0/3965240229 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fe7d00ceee0 con 0x7fe7dc1001a0 2024-04-25T00:15:50.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.043+0000 7fe7dbfff640 1 --2- 172.21.15.107:0/3965240229 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe7c40a2b90 0x7fe7c40a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc069ae0 tx=0x7fe7cc005cb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:50.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.047+0000 7fe7d97fa640 1 -- 172.21.15.107:0/3965240229 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe7d009bb00 con 0x7fe7dc1001a0 2024-04-25T00:15:50.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.225+0000 7fe7e29c8640 1 -- 172.21.15.107:0/3965240229 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe7dc0623e0 con 0x7fe7dc1001a0 2024-04-25T00:15:50.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.225+0000 7fe7d97fa640 1 -- 172.21.15.107:0/3965240229 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fe7d009b400 con 0x7fe7dc1001a0 2024-04-25T00:15:50.226 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-25T00:15:50.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.228+0000 7fe7e29c8640 1 -- 172.21.15.107:0/3965240229 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe7c40a2b90 msgr2=0x7fe7c40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:50.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.228+0000 7fe7e29c8640 1 --2- 172.21.15.107:0/3965240229 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe7c40a2b90 0x7fe7c40a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc069ae0 tx=0x7fe7cc005cb0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.228+0000 7fe7e29c8640 1 -- 172.21.15.107:0/3965240229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7dc1001a0 msgr2=0x7fe7dc100ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:50.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.228+0000 7fe7e29c8640 1 --2- 172.21.15.107:0/3965240229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7dc1001a0 0x7fe7dc100ab0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000d950 tx=0x7fe7d000de10 comp rx=0 tx=0).stop 2024-04-25T00:15:50.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.228+0000 7fe7e29c8640 1 -- 172.21.15.107:0/3965240229 shutdown_connections 2024-04-25T00:15:50.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.228+0000 7fe7e29c8640 1 --2- 172.21.15.107:0/3965240229 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe7c40a2b90 0x7fe7c40a5050 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.228+0000 7fe7e29c8640 1 --2- 172.21.15.107:0/3965240229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7dc102d70 0x7fe7dc100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.228+0000 7fe7e29c8640 1 --2- 172.21.15.107:0/3965240229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7dc1001a0 0x7fe7dc100ab0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.228+0000 7fe7e29c8640 1 --2- 172.21.15.107:0/3965240229 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7dc0ff7e0 0x7fe7dc102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.228+0000 7fe7e29c8640 1 -- 172.21.15.107:0/3965240229 >> 172.21.15.107:0/3965240229 conn(0x7fe7dc0fb380 msgr2=0x7fe7dc105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:50.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.228+0000 7fe7e29c8640 1 -- 172.21.15.107:0/3965240229 shutdown_connections 2024-04-25T00:15:50.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.228+0000 7fe7e29c8640 1 -- 172.21.15.107:0/3965240229 wait complete. 2024-04-25T00:15:50.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:50 smithi107 ceph-mon[29153]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:15:50.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:50 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2462134916' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:15:50.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:50 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1824652042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:15:50.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:50 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/4020941227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:15:50.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:50 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1334071397' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:15:50.239 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:15:50.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.337+0000 7fe286bc8640 1 -- 172.21.15.107:0/3744102919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe280101890 msgr2=0x7fe280101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:50.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.337+0000 7fe286bc8640 1 --2- 172.21.15.107:0/3744102919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe280101890 0x7fe280101d10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009900 tx=0x7fe26c02a410 comp rx=0 tx=0).stop 2024-04-25T00:15:50.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.337+0000 7fe286bc8640 1 -- 172.21.15.107:0/3744102919 shutdown_connections 2024-04-25T00:15:50.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.337+0000 7fe286bc8640 1 --2- 172.21.15.107:0/3744102919 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe280102250 0x7fe280108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.337+0000 7fe286bc8640 1 --2- 172.21.15.107:0/3744102919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe280101890 0x7fe280101d10 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.337+0000 7fe286bc8640 1 --2- 172.21.15.107:0/3744102919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2801006a0 0x7fe280100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.337+0000 7fe286bc8640 1 -- 172.21.15.107:0/3744102919 >> 172.21.15.107:0/3744102919 conn(0x7fe2800fbc30 msgr2=0x7fe2800fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:50.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.337+0000 7fe286bc8640 1 -- 172.21.15.107:0/3744102919 shutdown_connections 2024-04-25T00:15:50.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.337+0000 7fe286bc8640 1 -- 172.21.15.107:0/3744102919 wait complete. 2024-04-25T00:15:50.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.337+0000 7fe286bc8640 1 Processor -- start 2024-04-25T00:15:50.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.337+0000 7fe286bc8640 1 -- start start 2024-04-25T00:15:50.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.338+0000 7fe286bc8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2801006a0 0x7fe280197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:50.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.338+0000 7fe286bc8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe280101890 0x7fe280197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:50.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.338+0000 7fe286bc8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe280102250 0x7fe28019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:50.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.338+0000 7fe286bc8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe28010c300 con 0x7fe2801006a0 2024-04-25T00:15:50.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.338+0000 7fe286bc8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe28010c180 con 0x7fe280102250 2024-04-25T00:15:50.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.338+0000 7fe286bc8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe28010c480 con 0x7fe280101890 2024-04-25T00:15:50.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.338+0000 7fe284964640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2801006a0 0x7fe280197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:50.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.338+0000 7fe284964640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2801006a0 0x7fe280197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40196/0 (socket says 172.21.15.107:40196) 2024-04-25T00:15:50.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.338+0000 7fe284964640 1 -- 172.21.15.107:0/4098961014 learned_addr learned my addr 172.21.15.107:0/4098961014 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:50.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.338+0000 7fe285165640 1 --2- 172.21.15.107:0/4098961014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe280102250 0x7fe28019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:50.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.338+0000 7fe27ffff640 1 --2- 172.21.15.107:0/4098961014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe280101890 0x7fe280197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:50.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.338+0000 7fe284964640 1 -- 172.21.15.107:0/4098961014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe280101890 msgr2=0x7fe280197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:50.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.338+0000 7fe284964640 1 --2- 172.21.15.107:0/4098961014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe280101890 0x7fe280197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.338+0000 7fe284964640 1 -- 172.21.15.107:0/4098961014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe280102250 msgr2=0x7fe28019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:50.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.338+0000 7fe284964640 1 --2- 172.21.15.107:0/4098961014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe280102250 0x7fe28019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.338+0000 7fe284964640 1 -- 172.21.15.107:0/4098961014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe28019f670 con 0x7fe2801006a0 2024-04-25T00:15:50.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.338+0000 7fe284964640 1 --2- 172.21.15.107:0/4098961014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2801006a0 0x7fe280197aa0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe27400d770 tx=0x7fe27400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:50.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.338+0000 7fe27dffb640 1 -- 172.21.15.107:0/4098961014 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe274017070 con 0x7fe2801006a0 2024-04-25T00:15:50.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.338+0000 7fe27dffb640 1 -- 172.21.15.107:0/4098961014 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe274009c30 con 0x7fe2801006a0 2024-04-25T00:15:50.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.338+0000 7fe286bc8640 1 -- 172.21.15.107:0/4098961014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe28019f960 con 0x7fe2801006a0 2024-04-25T00:15:50.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.338+0000 7fe27dffb640 1 -- 172.21.15.107:0/4098961014 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe274003dd0 con 0x7fe2801006a0 2024-04-25T00:15:50.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.339+0000 7fe286bc8640 1 -- 172.21.15.107:0/4098961014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe28019fe80 con 0x7fe2801006a0 2024-04-25T00:15:50.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.339+0000 7fe286bc8640 1 -- 172.21.15.107:0/4098961014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2801a0050 con 0x7fe2801006a0 2024-04-25T00:15:50.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:50 smithi195 ceph-mon[32833]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:15:50.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:50 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2462134916' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:15:50.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:50 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1824652042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:15:50.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:50 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/4020941227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:15:50.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:50 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1334071397' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:15:50.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.341+0000 7fe27dffb640 1 -- 172.21.15.107:0/4098961014 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe274003fb0 con 0x7fe2801006a0 2024-04-25T00:15:50.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.341+0000 7fe27dffb640 1 --2- 172.21.15.107:0/4098961014 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe2700a2b90 0x7fe2700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:50.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.341+0000 7fe27ffff640 1 --2- 172.21.15.107:0/4098961014 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe2700a2b90 0x7fe2700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:50.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.342+0000 7fe27dffb640 1 -- 172.21.15.107:0/4098961014 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fe2740ced10 con 0x7fe2801006a0 2024-04-25T00:15:50.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.342+0000 7fe27ffff640 1 --2- 172.21.15.107:0/4098961014 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe2700a2b90 0x7fe2700a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe26c000c00 tx=0x7fe26c0315d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:50.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.346+0000 7fe27dffb640 1 -- 172.21.15.107:0/4098961014 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe27409b930 con 0x7fe2801006a0 2024-04-25T00:15:50.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:50 smithi170 ceph-mon[32807]: pgmap v348: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:15:50.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:50 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2462134916' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:15:50.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:50 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1824652042' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:15:50.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:50 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/4020941227' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:15:50.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:50 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1334071397' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:15:50.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.558+0000 7fe286bc8640 1 -- 172.21.15.107:0/4098961014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2801a0220 con 0x7fe2801006a0 2024-04-25T00:15:50.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.558+0000 7fe27dffb640 1 -- 172.21.15.107:0/4098961014 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11897 (secure 0 0 0) 0x7fe274009de0 con 0x7fe2801006a0 2024-04-25T00:15:50.559 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:15:50.559 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:14:28.529865+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24521,24493]},"id":1}]} 2024-04-25T00:15:50.559 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 23 2024-04-25T00:15:50.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.560+0000 7fe286bc8640 1 -- 172.21.15.107:0/4098961014 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe2700a2b90 msgr2=0x7fe2700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:50.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.560+0000 7fe286bc8640 1 --2- 172.21.15.107:0/4098961014 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe2700a2b90 0x7fe2700a5050 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe26c000c00 tx=0x7fe26c0315d0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.560+0000 7fe286bc8640 1 -- 172.21.15.107:0/4098961014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2801006a0 msgr2=0x7fe280197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:50.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.560+0000 7fe286bc8640 1 --2- 172.21.15.107:0/4098961014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2801006a0 0x7fe280197aa0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe27400d770 tx=0x7fe27400dc30 comp rx=0 tx=0).stop 2024-04-25T00:15:50.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.561+0000 7fe286bc8640 1 -- 172.21.15.107:0/4098961014 shutdown_connections 2024-04-25T00:15:50.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.561+0000 7fe286bc8640 1 --2- 172.21.15.107:0/4098961014 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe2700a2b90 0x7fe2700a5050 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.561+0000 7fe286bc8640 1 --2- 172.21.15.107:0/4098961014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe280102250 0x7fe28019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.561+0000 7fe286bc8640 1 --2- 172.21.15.107:0/4098961014 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe280101890 0x7fe280197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.561+0000 7fe286bc8640 1 --2- 172.21.15.107:0/4098961014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2801006a0 0x7fe280197aa0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.561+0000 7fe286bc8640 1 -- 172.21.15.107:0/4098961014 >> 172.21.15.107:0/4098961014 conn(0x7fe2800fbc30 msgr2=0x7fe280069da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:50.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.561+0000 7fe286bc8640 1 -- 172.21.15.107:0/4098961014 shutdown_connections 2024-04-25T00:15:50.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.561+0000 7fe286bc8640 1 -- 172.21.15.107:0/4098961014 wait complete. 2024-04-25T00:15:50.573 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:15:50.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.671+0000 7f783e94f640 1 -- 172.21.15.107:0/2526350073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7838101790 msgr2=0x7f7838101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:50.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.671+0000 7f783e94f640 1 --2- 172.21.15.107:0/2526350073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7838101790 0x7f7838101bb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7828009900 tx=0x7f782802a410 comp rx=0 tx=0).stop 2024-04-25T00:15:50.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.671+0000 7f783e94f640 1 -- 172.21.15.107:0/2526350073 shutdown_connections 2024-04-25T00:15:50.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.671+0000 7f783e94f640 1 --2- 172.21.15.107:0/2526350073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7838103340 0x7f7838109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.671+0000 7f783e94f640 1 --2- 172.21.15.107:0/2526350073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7838102980 0x7f7838102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.671+0000 7f783e94f640 1 --2- 172.21.15.107:0/2526350073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7838101790 0x7f7838101bb0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.671+0000 7f783e94f640 1 -- 172.21.15.107:0/2526350073 >> 172.21.15.107:0/2526350073 conn(0x7f78380fcd20 msgr2=0x7f78380ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:50.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.671+0000 7f783e94f640 1 -- 172.21.15.107:0/2526350073 shutdown_connections 2024-04-25T00:15:50.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.671+0000 7f783e94f640 1 -- 172.21.15.107:0/2526350073 wait complete. 2024-04-25T00:15:50.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.672+0000 7f783e94f640 1 Processor -- start 2024-04-25T00:15:50.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.672+0000 7f783e94f640 1 -- start start 2024-04-25T00:15:50.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.672+0000 7f783e94f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7838101790 0x7f7838197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:50.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.672+0000 7f783e94f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7838102980 0x7f7838197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:50.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.672+0000 7f783e94f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7838103340 0x7f783819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:50.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.672+0000 7f783e94f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f783810d400 con 0x7f7838101790 2024-04-25T00:15:50.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.672+0000 7f783e94f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f783810d280 con 0x7f7838102980 2024-04-25T00:15:50.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.672+0000 7f783e94f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f783810d580 con 0x7f7838103340 2024-04-25T00:15:50.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.672+0000 7f7837fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7838101790 0x7f7838197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:50.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.672+0000 7f7837fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7838101790 0x7f7838197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40202/0 (socket says 172.21.15.107:40202) 2024-04-25T00:15:50.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.672+0000 7f7837fff640 1 -- 172.21.15.107:0/2953649163 learned_addr learned my addr 172.21.15.107:0/2953649163 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:50.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.672+0000 7f78377fe640 1 --2- 172.21.15.107:0/2953649163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7838102980 0x7f7838197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:50.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.672+0000 7f783ceec640 1 --2- 172.21.15.107:0/2953649163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7838103340 0x7f783819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:50.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.672+0000 7f7837fff640 1 -- 172.21.15.107:0/2953649163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7838103340 msgr2=0x7f783819f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:50.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.672+0000 7f7837fff640 1 --2- 172.21.15.107:0/2953649163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7838103340 0x7f783819f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.672+0000 7f7837fff640 1 -- 172.21.15.107:0/2953649163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7838102980 msgr2=0x7f7838197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:50.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.672+0000 7f7837fff640 1 --2- 172.21.15.107:0/2953649163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7838102980 0x7f7838197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.672+0000 7f7837fff640 1 -- 172.21.15.107:0/2953649163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783819f6d0 con 0x7f7838101790 2024-04-25T00:15:50.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.673+0000 7f7837fff640 1 --2- 172.21.15.107:0/2953649163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7838101790 0x7f7838197ab0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7828031610 tx=0x7f7828004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:50.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.673+0000 7f78357fa640 1 -- 172.21.15.107:0/2953649163 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782803f030 con 0x7f7838101790 2024-04-25T00:15:50.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.673+0000 7f78357fa640 1 -- 172.21.15.107:0/2953649163 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7828003ad0 con 0x7f7838101790 2024-04-25T00:15:50.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.673+0000 7f783e94f640 1 -- 172.21.15.107:0/2953649163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f783819f960 con 0x7f7838101790 2024-04-25T00:15:50.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.673+0000 7f78357fa640 1 -- 172.21.15.107:0/2953649163 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828035070 con 0x7f7838101790 2024-04-25T00:15:50.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.673+0000 7f783e94f640 1 -- 172.21.15.107:0/2953649163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f783819fe50 con 0x7f7838101790 2024-04-25T00:15:50.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.674+0000 7f783e94f640 1 -- 172.21.15.107:0/2953649163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7838108ad0 con 0x7f7838101790 2024-04-25T00:15:50.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.675+0000 7f78357fa640 1 -- 172.21.15.107:0/2953649163 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7828042030 con 0x7f7838101790 2024-04-25T00:15:50.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.675+0000 7f78357fa640 1 --2- 172.21.15.107:0/2953649163 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f78200a2b90 0x7f78200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:50.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.675+0000 7f78377fe640 1 --2- 172.21.15.107:0/2953649163 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f78200a2b90 0x7f78200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:50.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.675+0000 7f78357fa640 1 -- 172.21.15.107:0/2953649163 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f78280ef910 con 0x7f7838101790 2024-04-25T00:15:50.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.676+0000 7f78377fe640 1 --2- 172.21.15.107:0/2953649163 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f78200a2b90 0x7f78200a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7838198fc0 tx=0x7f782c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:50.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.680+0000 7f78357fa640 1 -- 172.21.15.107:0/2953649163 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f78280bbee0 con 0x7f7838101790 2024-04-25T00:15:50.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.855+0000 7f783e94f640 1 -- 172.21.15.107:0/2953649163 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7838062340 con 0x7f7838101790 2024-04-25T00:15:50.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.857+0000 7f78357fa640 1 -- 172.21.15.107:0/2953649163 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24576 (secure 0 0 0) 0x7f78280c01e0 con 0x7f7838101790 2024-04-25T00:15:50.858 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:15:50.858 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":94,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:15:02.070002+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:59.672801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:00.957782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:14:05.065846+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8},{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:15:50.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.859+0000 7f783e94f640 1 -- 172.21.15.107:0/2953649163 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f78200a2b90 msgr2=0x7f78200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:50.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.859+0000 7f783e94f640 1 --2- 172.21.15.107:0/2953649163 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f78200a2b90 0x7f78200a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7838198fc0 tx=0x7f782c005f90 comp rx=0 tx=0).stop 2024-04-25T00:15:50.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.859+0000 7f783e94f640 1 -- 172.21.15.107:0/2953649163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7838101790 msgr2=0x7f7838197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:50.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.859+0000 7f783e94f640 1 --2- 172.21.15.107:0/2953649163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7838101790 0x7f7838197ab0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7828031610 tx=0x7f7828004270 comp rx=0 tx=0).stop 2024-04-25T00:15:50.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.859+0000 7f783e94f640 1 -- 172.21.15.107:0/2953649163 shutdown_connections 2024-04-25T00:15:50.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.859+0000 7f783e94f640 1 --2- 172.21.15.107:0/2953649163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7838103340 0x7f783819f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.859+0000 7f783e94f640 1 --2- 172.21.15.107:0/2953649163 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f78200a2b90 0x7f78200a5050 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.859+0000 7f783e94f640 1 --2- 172.21.15.107:0/2953649163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7838102980 0x7f7838197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.859+0000 7f783e94f640 1 --2- 172.21.15.107:0/2953649163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7838101790 0x7f7838197ab0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.859+0000 7f783e94f640 1 -- 172.21.15.107:0/2953649163 >> 172.21.15.107:0/2953649163 conn(0x7f78380fcd20 msgr2=0x7f7838105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:50.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.860+0000 7f783e94f640 1 -- 172.21.15.107:0/2953649163 shutdown_connections 2024-04-25T00:15:50.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.860+0000 7f783e94f640 1 -- 172.21.15.107:0/2953649163 wait complete. 2024-04-25T00:15:50.873 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:09:25.160329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:13:59.672801+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:14:00.957782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:14:05.065846+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:15:50.874 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:15:50.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.974+0000 7fc9b17dc640 1 -- 172.21.15.107:0/255880939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9ac107460 msgr2=0x7fc9ac1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:50.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.974+0000 7fc9b17dc640 1 --2- 172.21.15.107:0/255880939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9ac107460 0x7fc9ac1098b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4009ca0 tx=0x7fc9a402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.974+0000 7fc9b17dc640 1 -- 172.21.15.107:0/255880939 shutdown_connections 2024-04-25T00:15:50.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.974+0000 7fc9b17dc640 1 --2- 172.21.15.107:0/255880939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9ac107460 0x7fc9ac1098b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.974+0000 7fc9b17dc640 1 --2- 172.21.15.107:0/255880939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9ac104b30 0x7fc9ac106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.974+0000 7fc9b17dc640 1 --2- 172.21.15.107:0/255880939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9ac102200 0x7fc9ac1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.974+0000 7fc9b17dc640 1 -- 172.21.15.107:0/255880939 >> 172.21.15.107:0/255880939 conn(0x7fc9ac0fbbf0 msgr2=0x7fc9ac0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:50.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.974+0000 7fc9b17dc640 1 -- 172.21.15.107:0/255880939 shutdown_connections 2024-04-25T00:15:50.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.974+0000 7fc9b17dc640 1 -- 172.21.15.107:0/255880939 wait complete. 2024-04-25T00:15:50.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.975+0000 7fc9b17dc640 1 Processor -- start 2024-04-25T00:15:50.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.975+0000 7fc9b17dc640 1 -- start start 2024-04-25T00:15:50.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.975+0000 7fc9b17dc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9ac102200 0x7fc9ac101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:50.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.975+0000 7fc9b17dc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9ac104b30 0x7fc9ac1001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:50.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.975+0000 7fc9b17dc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9ac107460 0x7fc9ac1006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:50.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.975+0000 7fc9b17dc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac10d470 con 0x7fc9ac107460 2024-04-25T00:15:50.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.975+0000 7fc9b17dc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac10d2f0 con 0x7fc9ac104b30 2024-04-25T00:15:50.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.975+0000 7fc9b17dc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac10d5f0 con 0x7fc9ac102200 2024-04-25T00:15:50.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.975+0000 7fc9ab7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9ac107460 0x7fc9ac1006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:50.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.975+0000 7fc9ab7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9ac107460 0x7fc9ac1006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40222/0 (socket says 172.21.15.107:40222) 2024-04-25T00:15:50.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.975+0000 7fc9ab7fe640 1 -- 172.21.15.107:0/3818629948 learned_addr learned my addr 172.21.15.107:0/3818629948 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:50.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.975+0000 7fc9aaffd640 1 --2- 172.21.15.107:0/3818629948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9ac102200 0x7fc9ac101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:50.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.975+0000 7fc9aa7fc640 1 --2- 172.21.15.107:0/3818629948 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9ac104b30 0x7fc9ac1001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:50.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.975+0000 7fc9ab7fe640 1 -- 172.21.15.107:0/3818629948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9ac102200 msgr2=0x7fc9ac101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:50.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.975+0000 7fc9ab7fe640 1 --2- 172.21.15.107:0/3818629948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9ac102200 0x7fc9ac101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.975+0000 7fc9ab7fe640 1 -- 172.21.15.107:0/3818629948 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9ac104b30 msgr2=0x7fc9ac1001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:50.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.975+0000 7fc9ab7fe640 1 --2- 172.21.15.107:0/3818629948 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9ac104b30 0x7fc9ac1001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.975+0000 7fc9ab7fe640 1 -- 172.21.15.107:0/3818629948 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ac100f80 con 0x7fc9ac107460 2024-04-25T00:15:50.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.975+0000 7fc9aaffd640 1 --2- 172.21.15.107:0/3818629948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9ac102200 0x7fc9ac101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:50.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.975+0000 7fc9ab7fe640 1 --2- 172.21.15.107:0/3818629948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9ac107460 0x7fc9ac1006f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4005e30 tx=0x7fc9a4031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:50.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.976+0000 7fc993fff640 1 -- 172.21.15.107:0/3818629948 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a4003c30 con 0x7fc9ac107460 2024-04-25T00:15:50.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.976+0000 7fc993fff640 1 -- 172.21.15.107:0/3818629948 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9a4003dd0 con 0x7fc9ac107460 2024-04-25T00:15:50.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.976+0000 7fc9b17dc640 1 -- 172.21.15.107:0/3818629948 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9ac1a1bf0 con 0x7fc9ac107460 2024-04-25T00:15:50.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.976+0000 7fc993fff640 1 -- 172.21.15.107:0/3818629948 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a402fde0 con 0x7fc9ac107460 2024-04-25T00:15:50.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.976+0000 7fc9b17dc640 1 -- 172.21.15.107:0/3818629948 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9ac1a2190 con 0x7fc9ac107460 2024-04-25T00:15:50.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.977+0000 7fc9b17dc640 1 -- 172.21.15.107:0/3818629948 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9ac0fd6e0 con 0x7fc9ac107460 2024-04-25T00:15:50.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.978+0000 7fc993fff640 1 -- 172.21.15.107:0/3818629948 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc9a4004050 con 0x7fc9ac107460 2024-04-25T00:15:50.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.978+0000 7fc993fff640 1 --2- 172.21.15.107:0/3818629948 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc9940a2c10 0x7fc9940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:50.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.978+0000 7fc9aaffd640 1 --2- 172.21.15.107:0/3818629948 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc9940a2c10 0x7fc9940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:50.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.978+0000 7fc993fff640 1 -- 172.21.15.107:0/3818629948 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fc9a40f07c0 con 0x7fc9ac107460 2024-04-25T00:15:50.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.979+0000 7fc9aaffd640 1 --2- 172.21.15.107:0/3818629948 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc9940a2c10 0x7fc9940a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009930 tx=0x7fc99c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:50.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:50.983+0000 7fc993fff640 1 -- 172.21.15.107:0/3818629948 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc9a40bd500 con 0x7fc9ac107460 2024-04-25T00:15:51.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.159+0000 7fc9b17dc640 1 -- 172.21.15.107:0/3818629948 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc9ac062340 con 0x7fc9ac107460 2024-04-25T00:15:51.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.160+0000 7fc993fff640 1 -- 172.21.15.107:0/3818629948 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fc9a40bce00 con 0x7fc9ac107460 2024-04-25T00:15:51.161 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-04-25T00:15:51.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.162+0000 7fc9b17dc640 1 -- 172.21.15.107:0/3818629948 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc9940a2c10 msgr2=0x7fc9940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:51.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.162+0000 7fc9b17dc640 1 --2- 172.21.15.107:0/3818629948 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc9940a2c10 0x7fc9940a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009930 tx=0x7fc99c008040 comp rx=0 tx=0).stop 2024-04-25T00:15:51.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.162+0000 7fc9b17dc640 1 -- 172.21.15.107:0/3818629948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9ac107460 msgr2=0x7fc9ac1006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:51.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.162+0000 7fc9b17dc640 1 --2- 172.21.15.107:0/3818629948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9ac107460 0x7fc9ac1006f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4005e30 tx=0x7fc9a4031ee0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.162+0000 7fc9b17dc640 1 -- 172.21.15.107:0/3818629948 shutdown_connections 2024-04-25T00:15:51.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.162+0000 7fc9b17dc640 1 --2- 172.21.15.107:0/3818629948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9ac107460 0x7fc9ac1006f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.162+0000 7fc9b17dc640 1 --2- 172.21.15.107:0/3818629948 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc9940a2c10 0x7fc9940a50d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.162+0000 7fc9b17dc640 1 --2- 172.21.15.107:0/3818629948 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9ac104b30 0x7fc9ac1001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.162+0000 7fc9b17dc640 1 --2- 172.21.15.107:0/3818629948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9ac102200 0x7fc9ac101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.162+0000 7fc9b17dc640 1 -- 172.21.15.107:0/3818629948 >> 172.21.15.107:0/3818629948 conn(0x7fc9ac0fbbf0 msgr2=0x7fc9ac102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:51.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.162+0000 7fc9b17dc640 1 -- 172.21.15.107:0/3818629948 shutdown_connections 2024-04-25T00:15:51.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.162+0000 7fc9b17dc640 1 -- 172.21.15.107:0/3818629948 wait complete. 2024-04-25T00:15:51.172 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:51 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3965240229' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:15:51.172 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:51 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/4098961014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:51.172 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:51 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2953649163' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:15:51.174 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:15:51.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.274+0000 7f500069e640 1 -- 172.21.15.107:0/668297304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff81022b0 msgr2=0x7f4ff8108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:51.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.274+0000 7f500069e640 1 --2- 172.21.15.107:0/668297304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff81022b0 0x7f4ff8108b40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0009900 tx=0x7f4ff002a410 comp rx=0 tx=0).stop 2024-04-25T00:15:51.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.274+0000 7f500069e640 1 -- 172.21.15.107:0/668297304 shutdown_connections 2024-04-25T00:15:51.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.274+0000 7f500069e640 1 --2- 172.21.15.107:0/668297304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff81022b0 0x7f4ff8108b40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.274+0000 7f500069e640 1 --2- 172.21.15.107:0/668297304 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ff81018f0 0x7f4ff8101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.274+0000 7f500069e640 1 --2- 172.21.15.107:0/668297304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ff8100700 0x7f4ff8100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.274+0000 7f500069e640 1 -- 172.21.15.107:0/668297304 >> 172.21.15.107:0/668297304 conn(0x7f4ff80fbc90 msgr2=0x7f4ff80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:51.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.274+0000 7f500069e640 1 -- 172.21.15.107:0/668297304 shutdown_connections 2024-04-25T00:15:51.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.274+0000 7f500069e640 1 -- 172.21.15.107:0/668297304 wait complete. 2024-04-25T00:15:51.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.275+0000 7f500069e640 1 Processor -- start 2024-04-25T00:15:51.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.275+0000 7f500069e640 1 -- start start 2024-04-25T00:15:51.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.275+0000 7f500069e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ff8100700 0x7f4ff8197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:51.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.275+0000 7f500069e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ff81018f0 0x7f4ff8198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:51.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.275+0000 7f500069e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff81022b0 0x7f4ff819f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:51.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.275+0000 7f500069e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff810c370 con 0x7f4ff81022b0 2024-04-25T00:15:51.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.275+0000 7f500069e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff810c1f0 con 0x7f4ff81018f0 2024-04-25T00:15:51.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.275+0000 7f500069e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff810c4f0 con 0x7f4ff8100700 2024-04-25T00:15:51.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.275+0000 7f4ffec3b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff81022b0 0x7f4ff819f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:51.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.275+0000 7f4ffec3b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff81022b0 0x7f4ff819f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40250/0 (socket says 172.21.15.107:40250) 2024-04-25T00:15:51.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.275+0000 7f4ffec3b640 1 -- 172.21.15.107:0/1628317550 learned_addr learned my addr 172.21.15.107:0/1628317550 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:51.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.275+0000 7f4ffe43a640 1 --2- 172.21.15.107:0/1628317550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ff8100700 0x7f4ff8197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:51.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.275+0000 7f4ffdc39640 1 --2- 172.21.15.107:0/1628317550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ff81018f0 0x7f4ff8198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:51.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.275+0000 7f4ffec3b640 1 -- 172.21.15.107:0/1628317550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ff8100700 msgr2=0x7f4ff8197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:51.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.275+0000 7f4ffec3b640 1 --2- 172.21.15.107:0/1628317550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ff8100700 0x7f4ff8197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.275+0000 7f4ffec3b640 1 -- 172.21.15.107:0/1628317550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ff81018f0 msgr2=0x7f4ff8198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:51.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.275+0000 7f4ffec3b640 1 --2- 172.21.15.107:0/1628317550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ff81018f0 0x7f4ff8198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.275+0000 7f4ffec3b640 1 -- 172.21.15.107:0/1628317550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff819f6f0 con 0x7f4ff81022b0 2024-04-25T00:15:51.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.275+0000 7f4ffe43a640 1 --2- 172.21.15.107:0/1628317550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ff8100700 0x7f4ff8197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:51.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.275+0000 7f4ffdc39640 1 --2- 172.21.15.107:0/1628317550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ff81018f0 0x7f4ff8198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:51.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.276+0000 7f4ffec3b640 1 --2- 172.21.15.107:0/1628317550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff81022b0 0x7f4ff819f080 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00314e0 tx=0x7f4ff0004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:51.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.276+0000 7f4fef7fe640 1 -- 172.21.15.107:0/1628317550 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff003f030 con 0x7f4ff81022b0 2024-04-25T00:15:51.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.276+0000 7f500069e640 1 -- 172.21.15.107:0/1628317550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff819f980 con 0x7f4ff81022b0 2024-04-25T00:15:51.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.276+0000 7f4fef7fe640 1 -- 172.21.15.107:0/1628317550 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ff0003ce0 con 0x7f4ff81022b0 2024-04-25T00:15:51.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.276+0000 7f4fef7fe640 1 -- 172.21.15.107:0/1628317550 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0035070 con 0x7f4ff81022b0 2024-04-25T00:15:51.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.276+0000 7f500069e640 1 -- 172.21.15.107:0/1628317550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff819fef0 con 0x7f4ff81022b0 2024-04-25T00:15:51.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.277+0000 7f500069e640 1 -- 172.21.15.107:0/1628317550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ff81a0120 con 0x7f4ff81022b0 2024-04-25T00:15:51.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.278+0000 7f4fef7fe640 1 -- 172.21.15.107:0/1628317550 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4ff0042030 con 0x7f4ff81022b0 2024-04-25T00:15:51.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.278+0000 7f4fef7fe640 1 --2- 172.21.15.107:0/1628317550 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f4fe40a2b90 0x7f4fe40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:51.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.278+0000 7f4ffe43a640 1 --2- 172.21.15.107:0/1628317550 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f4fe40a2b90 0x7f4fe40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:51.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.279+0000 7f4fef7fe640 1 -- 172.21.15.107:0/1628317550 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f4ff00efde0 con 0x7f4ff81022b0 2024-04-25T00:15:51.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.279+0000 7f4ffe43a640 1 --2- 172.21.15.107:0/1628317550 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f4fe40a2b90 0x7f4fe40a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8101750 tx=0x7f4ff4009400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:51.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.283+0000 7f4fef7fe640 1 -- 172.21.15.107:0/1628317550 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4ff00bca00 con 0x7f4ff81022b0 2024-04-25T00:15:51.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:51 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3965240229' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:15:51.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:51 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/4098961014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:51.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:51 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2953649163' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:15:51.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:51 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3965240229' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:15:51.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:51 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/4098961014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:51.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:51 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2953649163' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:15:51.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.458+0000 7f500069e640 1 -- 172.21.15.107:0/1628317550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4ff81a02f0 con 0x7f4ff81022b0 2024-04-25T00:15:51.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.459+0000 7f4fef7fe640 1 -- 172.21.15.107:0/1628317550 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f4ff00bc300 con 0x7f4ff81022b0 2024-04-25T00:15:51.459 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-25T00:15:51.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.461+0000 7f500069e640 1 -- 172.21.15.107:0/1628317550 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f4fe40a2b90 msgr2=0x7f4fe40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:51.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.461+0000 7f500069e640 1 --2- 172.21.15.107:0/1628317550 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f4fe40a2b90 0x7f4fe40a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8101750 tx=0x7f4ff4009400 comp rx=0 tx=0).stop 2024-04-25T00:15:51.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.461+0000 7f500069e640 1 -- 172.21.15.107:0/1628317550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff81022b0 msgr2=0x7f4ff819f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:51.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.461+0000 7f500069e640 1 --2- 172.21.15.107:0/1628317550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff81022b0 0x7f4ff819f080 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00314e0 tx=0x7f4ff0004360 comp rx=0 tx=0).stop 2024-04-25T00:15:51.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.461+0000 7f500069e640 1 -- 172.21.15.107:0/1628317550 shutdown_connections 2024-04-25T00:15:51.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.461+0000 7f500069e640 1 --2- 172.21.15.107:0/1628317550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ff81022b0 0x7f4ff819f080 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.461+0000 7f500069e640 1 --2- 172.21.15.107:0/1628317550 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f4fe40a2b90 0x7f4fe40a5050 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.461+0000 7f500069e640 1 --2- 172.21.15.107:0/1628317550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ff81018f0 0x7f4ff8198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.461+0000 7f500069e640 1 --2- 172.21.15.107:0/1628317550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ff8100700 0x7f4ff8197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.461+0000 7f500069e640 1 -- 172.21.15.107:0/1628317550 >> 172.21.15.107:0/1628317550 conn(0x7f4ff80fbc90 msgr2=0x7f4ff80fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:51.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.461+0000 7f500069e640 1 -- 172.21.15.107:0/1628317550 shutdown_connections 2024-04-25T00:15:51.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.461+0000 7f500069e640 1 -- 172.21.15.107:0/1628317550 wait complete. 2024-04-25T00:15:51.475 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:15:51.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.576+0000 7f6065368640 1 -- 172.21.15.107:0/1584392866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6060107460 msgr2=0x7f60601098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:51.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.576+0000 7f6065368640 1 --2- 172.21.15.107:0/1584392866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6060107460 0x7f60601098b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6058009900 tx=0x7f605802a410 comp rx=0 tx=0).stop 2024-04-25T00:15:51.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.576+0000 7f6065368640 1 -- 172.21.15.107:0/1584392866 shutdown_connections 2024-04-25T00:15:51.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.576+0000 7f6065368640 1 --2- 172.21.15.107:0/1584392866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6060107460 0x7f60601098b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.576+0000 7f6065368640 1 --2- 172.21.15.107:0/1584392866 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6060104b30 0x7f6060106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.576+0000 7f6065368640 1 --2- 172.21.15.107:0/1584392866 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6060102200 0x7f60601045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.576+0000 7f6065368640 1 -- 172.21.15.107:0/1584392866 >> 172.21.15.107:0/1584392866 conn(0x7f60600fbbf0 msgr2=0x7f60600fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:51.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.576+0000 7f6065368640 1 -- 172.21.15.107:0/1584392866 shutdown_connections 2024-04-25T00:15:51.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.576+0000 7f6065368640 1 -- 172.21.15.107:0/1584392866 wait complete. 2024-04-25T00:15:51.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.577+0000 7f6065368640 1 Processor -- start 2024-04-25T00:15:51.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.577+0000 7f6065368640 1 -- start start 2024-04-25T00:15:51.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.577+0000 7f6065368640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6060102200 0x7f6060101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:51.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.577+0000 7f6065368640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6060104b30 0x7f60601001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:51.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.577+0000 7f6065368640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6060107460 0x7f60601006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:51.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.577+0000 7f6065368640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f606010d4f0 con 0x7f6060102200 2024-04-25T00:15:51.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.577+0000 7f6065368640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f606010d370 con 0x7f6060104b30 2024-04-25T00:15:51.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.577+0000 7f6065368640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f606010d670 con 0x7f6060107460 2024-04-25T00:15:51.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.577+0000 7f605effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6060102200 0x7f6060101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:51.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.577+0000 7f605effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6060102200 0x7f6060101b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40260/0 (socket says 172.21.15.107:40260) 2024-04-25T00:15:51.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.577+0000 7f605effd640 1 -- 172.21.15.107:0/3668341985 learned_addr learned my addr 172.21.15.107:0/3668341985 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:51.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.578+0000 7f605f7fe640 1 --2- 172.21.15.107:0/3668341985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6060107460 0x7f60601006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:51.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.578+0000 7f605effd640 1 -- 172.21.15.107:0/3668341985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6060107460 msgr2=0x7f60601006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:51.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.578+0000 7f605effd640 1 --2- 172.21.15.107:0/3668341985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6060107460 0x7f60601006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.578+0000 7f605e7fc640 1 --2- 172.21.15.107:0/3668341985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6060104b30 0x7f60601001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:51.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.578+0000 7f605effd640 1 -- 172.21.15.107:0/3668341985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6060104b30 msgr2=0x7f60601001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:51.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.578+0000 7f605effd640 1 --2- 172.21.15.107:0/3668341985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6060104b30 0x7f60601001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.578+0000 7f605effd640 1 -- 172.21.15.107:0/3668341985 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6060100f80 con 0x7f6060102200 2024-04-25T00:15:51.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.578+0000 7f605f7fe640 1 --2- 172.21.15.107:0/3668341985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6060107460 0x7f60601006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:51.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.578+0000 7f605e7fc640 1 --2- 172.21.15.107:0/3668341985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6060104b30 0x7f60601001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:51.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.578+0000 7f605effd640 1 --2- 172.21.15.107:0/3668341985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6060102200 0x7f6060101b70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f605000ba50 tx=0x7f605000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:51.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.578+0000 7f6047fff640 1 -- 172.21.15.107:0/3668341985 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050019070 con 0x7f6060102200 2024-04-25T00:15:51.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.578+0000 7f6065368640 1 -- 172.21.15.107:0/3668341985 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60601a1d10 con 0x7f6060102200 2024-04-25T00:15:51.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.578+0000 7f6047fff640 1 -- 172.21.15.107:0/3668341985 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f605000cbe0 con 0x7f6060102200 2024-04-25T00:15:51.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.578+0000 7f6047fff640 1 -- 172.21.15.107:0/3668341985 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60500076e0 con 0x7f6060102200 2024-04-25T00:15:51.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.578+0000 7f6065368640 1 -- 172.21.15.107:0/3668341985 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60601a21b0 con 0x7f6060102200 2024-04-25T00:15:51.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.579+0000 7f6065368640 1 -- 172.21.15.107:0/3668341985 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60600fd6e0 con 0x7f6060102200 2024-04-25T00:15:51.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.580+0000 7f6047fff640 1 -- 172.21.15.107:0/3668341985 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f605001c070 con 0x7f6060102200 2024-04-25T00:15:51.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.581+0000 7f6047fff640 1 --2- 172.21.15.107:0/3668341985 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f60480a2b90 0x7f60480a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:51.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.581+0000 7f605e7fc640 1 --2- 172.21.15.107:0/3668341985 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f60480a2b90 0x7f60480a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:51.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.581+0000 7f6047fff640 1 -- 172.21.15.107:0/3668341985 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f60500cefb0 con 0x7f6060102200 2024-04-25T00:15:51.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.581+0000 7f605e7fc640 1 --2- 172.21.15.107:0/3668341985 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f60480a2b90 0x7f60480a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6054006fd0 tx=0x7f6054008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:51.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.585+0000 7f6047fff640 1 -- 172.21.15.107:0/3668341985 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f605009bbd0 con 0x7f6060102200 2024-04-25T00:15:51.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.760+0000 7f6065368640 1 -- 172.21.15.107:0/3668341985 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f606002d370 con 0x7f6060102200 2024-04-25T00:15:51.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.761+0000 7f6047fff640 1 -- 172.21.15.107:0/3668341985 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f605009b4d0 con 0x7f6060102200 2024-04-25T00:15:51.761 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-04-25T00:15:51.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.763+0000 7f6065368640 1 -- 172.21.15.107:0/3668341985 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f60480a2b90 msgr2=0x7f60480a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:51.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.763+0000 7f6065368640 1 --2- 172.21.15.107:0/3668341985 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f60480a2b90 0x7f60480a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6054006fd0 tx=0x7f6054008040 comp rx=0 tx=0).stop 2024-04-25T00:15:51.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.763+0000 7f6065368640 1 -- 172.21.15.107:0/3668341985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6060102200 msgr2=0x7f6060101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:51.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.763+0000 7f6065368640 1 --2- 172.21.15.107:0/3668341985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6060102200 0x7f6060101b70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f605000ba50 tx=0x7f605000bf10 comp rx=0 tx=0).stop 2024-04-25T00:15:51.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.763+0000 7f6065368640 1 -- 172.21.15.107:0/3668341985 shutdown_connections 2024-04-25T00:15:51.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.763+0000 7f6065368640 1 --2- 172.21.15.107:0/3668341985 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6060107460 0x7f60601006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.763+0000 7f6065368640 1 --2- 172.21.15.107:0/3668341985 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f60480a2b90 0x7f60480a5050 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.763+0000 7f6065368640 1 --2- 172.21.15.107:0/3668341985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6060104b30 0x7f60601001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.763+0000 7f6065368640 1 --2- 172.21.15.107:0/3668341985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6060102200 0x7f6060101b70 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.763+0000 7f6065368640 1 -- 172.21.15.107:0/3668341985 >> 172.21.15.107:0/3668341985 conn(0x7f60600fbbf0 msgr2=0x7f6060102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:51.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.763+0000 7f6065368640 1 -- 172.21.15.107:0/3668341985 shutdown_connections 2024-04-25T00:15:51.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.763+0000 7f6065368640 1 -- 172.21.15.107:0/3668341985 wait complete. 2024-04-25T00:15:51.774 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:15:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.876+0000 7f8dfe1c1640 1 -- 172.21.15.107:0/1094466848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8df81006a0 msgr2=0x7f8df8100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.876+0000 7f8dfe1c1640 1 --2- 172.21.15.107:0/1094466848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8df81006a0 0x7f8df8100ac0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8de8009900 tx=0x7f8de802a410 comp rx=0 tx=0).stop 2024-04-25T00:15:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.876+0000 7f8dfe1c1640 1 -- 172.21.15.107:0/1094466848 shutdown_connections 2024-04-25T00:15:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.876+0000 7f8dfe1c1640 1 --2- 172.21.15.107:0/1094466848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8df8102250 0x7f8df8108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.876+0000 7f8dfe1c1640 1 --2- 172.21.15.107:0/1094466848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8df8101890 0x7f8df8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.876+0000 7f8dfe1c1640 1 --2- 172.21.15.107:0/1094466848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8df81006a0 0x7f8df8100ac0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.876+0000 7f8dfe1c1640 1 -- 172.21.15.107:0/1094466848 >> 172.21.15.107:0/1094466848 conn(0x7f8df80fbc10 msgr2=0x7f8df80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.876+0000 7f8dfe1c1640 1 -- 172.21.15.107:0/1094466848 shutdown_connections 2024-04-25T00:15:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.876+0000 7f8dfe1c1640 1 -- 172.21.15.107:0/1094466848 wait complete. 2024-04-25T00:15:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.876+0000 7f8dfe1c1640 1 Processor -- start 2024-04-25T00:15:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.877+0000 7f8dfe1c1640 1 -- start start 2024-04-25T00:15:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.877+0000 7f8dfe1c1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8df81006a0 0x7f8df81969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.877+0000 7f8dfe1c1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8df8101890 0x7f8df8196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.877+0000 7f8dfe1c1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8df8102250 0x7f8df819df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.877+0000 7f8dfe1c1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8df810c320 con 0x7f8df81006a0 2024-04-25T00:15:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.877+0000 7f8dfe1c1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8df810c1a0 con 0x7f8df8101890 2024-04-25T00:15:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.877+0000 7f8dfe1c1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8df810c4a0 con 0x7f8df8102250 2024-04-25T00:15:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.877+0000 7f8df77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8df81006a0 0x7f8df81969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.877+0000 7f8df77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8df81006a0 0x7f8df81969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40286/0 (socket says 172.21.15.107:40286) 2024-04-25T00:15:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.877+0000 7f8df77fe640 1 -- 172.21.15.107:0/2250576812 learned_addr learned my addr 172.21.15.107:0/2250576812 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.877+0000 7f8df7fff640 1 --2- 172.21.15.107:0/2250576812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8df8102250 0x7f8df819df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.877+0000 7f8df6ffd640 1 --2- 172.21.15.107:0/2250576812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8df8101890 0x7f8df8196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.877+0000 7f8df77fe640 1 -- 172.21.15.107:0/2250576812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8df8102250 msgr2=0x7f8df819df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.877+0000 7f8df77fe640 1 --2- 172.21.15.107:0/2250576812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8df8102250 0x7f8df819df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.877+0000 7f8df77fe640 1 -- 172.21.15.107:0/2250576812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8df8101890 msgr2=0x7f8df8196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.877+0000 7f8df77fe640 1 --2- 172.21.15.107:0/2250576812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8df8101890 0x7f8df8196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.877+0000 7f8df77fe640 1 -- 172.21.15.107:0/2250576812 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df819e5e0 con 0x7f8df81006a0 2024-04-25T00:15:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.877+0000 7f8df77fe640 1 --2- 172.21.15.107:0/2250576812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8df81006a0 0x7f8df81969c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8de8031610 tx=0x7f8de80043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.878+0000 7f8df4ff9640 1 -- 172.21.15.107:0/2250576812 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de803f030 con 0x7f8df81006a0 2024-04-25T00:15:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.878+0000 7f8df4ff9640 1 -- 172.21.15.107:0/2250576812 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8de8003ce0 con 0x7f8df81006a0 2024-04-25T00:15:51.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.878+0000 7f8df4ff9640 1 -- 172.21.15.107:0/2250576812 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de8035070 con 0x7f8df81006a0 2024-04-25T00:15:51.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.878+0000 7f8dfe1c1640 1 -- 172.21.15.107:0/2250576812 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8df819e870 con 0x7f8df81006a0 2024-04-25T00:15:51.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.878+0000 7f8dfe1c1640 1 -- 172.21.15.107:0/2250576812 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8df819ed10 con 0x7f8df81006a0 2024-04-25T00:15:51.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.879+0000 7f8dfe1c1640 1 -- 172.21.15.107:0/2250576812 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8df8107a40 con 0x7f8df81006a0 2024-04-25T00:15:51.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.880+0000 7f8df4ff9640 1 -- 172.21.15.107:0/2250576812 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8de8042030 con 0x7f8df81006a0 2024-04-25T00:15:51.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.880+0000 7f8df4ff9640 1 --2- 172.21.15.107:0/2250576812 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8de00a2b90 0x7f8de00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:51.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.880+0000 7f8df6ffd640 1 --2- 172.21.15.107:0/2250576812 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8de00a2b90 0x7f8de00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:51.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.880+0000 7f8df4ff9640 1 -- 172.21.15.107:0/2250576812 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f8de80ef5f0 con 0x7f8df81006a0 2024-04-25T00:15:51.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.881+0000 7f8df6ffd640 1 --2- 172.21.15.107:0/2250576812 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8de00a2b90 0x7f8de00a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8dec005fd0 tx=0x7f8dec005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:51.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:51.885+0000 7f8df4ff9640 1 -- 172.21.15.107:0/2250576812 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8de80bc210 con 0x7f8df81006a0 2024-04-25T00:15:52.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.060+0000 7f8dfe1c1640 1 -- 172.21.15.107:0/2250576812 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8df8062340 con 0x7f8df81006a0 2024-04-25T00:15:52.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.060+0000 7f8df4ff9640 1 -- 172.21.15.107:0/2250576812 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f8de80bbb10 con 0x7f8df81006a0 2024-04-25T00:15:52.061 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-25T00:15:52.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.062+0000 7f8dfe1c1640 1 -- 172.21.15.107:0/2250576812 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8de00a2b90 msgr2=0x7f8de00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:52.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.062+0000 7f8dfe1c1640 1 --2- 172.21.15.107:0/2250576812 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8de00a2b90 0x7f8de00a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8dec005fd0 tx=0x7f8dec005850 comp rx=0 tx=0).stop 2024-04-25T00:15:52.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.062+0000 7f8dfe1c1640 1 -- 172.21.15.107:0/2250576812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8df81006a0 msgr2=0x7f8df81969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:52.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.062+0000 7f8dfe1c1640 1 --2- 172.21.15.107:0/2250576812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8df81006a0 0x7f8df81969c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8de8031610 tx=0x7f8de80043b0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.062+0000 7f8dfe1c1640 1 -- 172.21.15.107:0/2250576812 shutdown_connections 2024-04-25T00:15:52.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.062+0000 7f8dfe1c1640 1 --2- 172.21.15.107:0/2250576812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8df8102250 0x7f8df819df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.062+0000 7f8dfe1c1640 1 --2- 172.21.15.107:0/2250576812 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8de00a2b90 0x7f8de00a5050 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.062+0000 7f8dfe1c1640 1 --2- 172.21.15.107:0/2250576812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8df8101890 0x7f8df8196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.062+0000 7f8dfe1c1640 1 --2- 172.21.15.107:0/2250576812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8df81006a0 0x7f8df81969c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.062+0000 7f8dfe1c1640 1 -- 172.21.15.107:0/2250576812 >> 172.21.15.107:0/2250576812 conn(0x7f8df80fbc10 msgr2=0x7f8df8104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:52.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.062+0000 7f8dfe1c1640 1 -- 172.21.15.107:0/2250576812 shutdown_connections 2024-04-25T00:15:52.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.062+0000 7f8dfe1c1640 1 -- 172.21.15.107:0/2250576812 wait complete. 2024-04-25T00:15:52.073 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:52 smithi107 ceph-mon[29153]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:15:52.073 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:52 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3818629948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:15:52.073 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:52 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1628317550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:15:52.073 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:52 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3668341985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:15:52.074 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:15:52.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.171+0000 7fdaf2542640 1 -- 172.21.15.107:0/1282450969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaec102980 msgr2=0x7fdaec102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:52.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.171+0000 7fdaf2542640 1 --2- 172.21.15.107:0/1282450969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaec102980 0x7fdaec102e00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdae0009900 tx=0x7fdae002a410 comp rx=0 tx=0).stop 2024-04-25T00:15:52.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.171+0000 7fdaf2542640 1 -- 172.21.15.107:0/1282450969 shutdown_connections 2024-04-25T00:15:52.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.171+0000 7fdaf2542640 1 --2- 172.21.15.107:0/1282450969 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaec103340 0x7fdaec109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.171+0000 7fdaf2542640 1 --2- 172.21.15.107:0/1282450969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaec102980 0x7fdaec102e00 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.171+0000 7fdaf2542640 1 --2- 172.21.15.107:0/1282450969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaec101790 0x7fdaec101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.171+0000 7fdaf2542640 1 -- 172.21.15.107:0/1282450969 >> 172.21.15.107:0/1282450969 conn(0x7fdaec0fcd20 msgr2=0x7fdaec0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:52.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.171+0000 7fdaf2542640 1 -- 172.21.15.107:0/1282450969 shutdown_connections 2024-04-25T00:15:52.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.171+0000 7fdaf2542640 1 -- 172.21.15.107:0/1282450969 wait complete. 2024-04-25T00:15:52.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.172+0000 7fdaf2542640 1 Processor -- start 2024-04-25T00:15:52.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.172+0000 7fdaf2542640 1 -- start start 2024-04-25T00:15:52.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.172+0000 7fdaf2542640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaec101790 0x7fdaec197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:52.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.172+0000 7fdaf2542640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaec102980 0x7fdaec198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:52.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.172+0000 7fdaf2542640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaec103340 0x7fdaec19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:52.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.172+0000 7fdaf2542640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec10d470 con 0x7fdaec103340 2024-04-25T00:15:52.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.172+0000 7fdaf2542640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec10d2f0 con 0x7fdaec102980 2024-04-25T00:15:52.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.172+0000 7fdaf2542640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdaec10d5f0 con 0x7fdaec101790 2024-04-25T00:15:52.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.172+0000 7fdaf0adf640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaec103340 0x7fdaec19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:52.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.172+0000 7fdaf0adf640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaec103340 0x7fdaec19f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40304/0 (socket says 172.21.15.107:40304) 2024-04-25T00:15:52.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.172+0000 7fdaf0adf640 1 -- 172.21.15.107:0/1830839248 learned_addr learned my addr 172.21.15.107:0/1830839248 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:52.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.172+0000 7fdaeb7fe640 1 --2- 172.21.15.107:0/1830839248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaec102980 0x7fdaec198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:52.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.172+0000 7fdaebfff640 1 --2- 172.21.15.107:0/1830839248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaec101790 0x7fdaec197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:52.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.172+0000 7fdaf0adf640 1 -- 172.21.15.107:0/1830839248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaec101790 msgr2=0x7fdaec197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:52.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.172+0000 7fdaf0adf640 1 --2- 172.21.15.107:0/1830839248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaec101790 0x7fdaec197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.172+0000 7fdaf0adf640 1 -- 172.21.15.107:0/1830839248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaec102980 msgr2=0x7fdaec198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:52.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.172+0000 7fdaf0adf640 1 --2- 172.21.15.107:0/1830839248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaec102980 0x7fdaec198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.172+0000 7fdaf0adf640 1 -- 172.21.15.107:0/1830839248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaec19f740 con 0x7fdaec103340 2024-04-25T00:15:52.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.172+0000 7fdaebfff640 1 --2- 172.21.15.107:0/1830839248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaec101790 0x7fdaec197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:52.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.173+0000 7fdaf0adf640 1 --2- 172.21.15.107:0/1830839248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaec103340 0x7fdaec19f0d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdae400b970 tx=0x7fdae400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:52.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.173+0000 7fdae97fa640 1 -- 172.21.15.107:0/1830839248 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4019070 con 0x7fdaec103340 2024-04-25T00:15:52.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.173+0000 7fdaf2542640 1 -- 172.21.15.107:0/1830839248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaec19fa30 con 0x7fdaec103340 2024-04-25T00:15:52.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.173+0000 7fdae97fa640 1 -- 172.21.15.107:0/1830839248 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdae400cb40 con 0x7fdaec103340 2024-04-25T00:15:52.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.173+0000 7fdae97fa640 1 -- 172.21.15.107:0/1830839248 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4007650 con 0x7fdaec103340 2024-04-25T00:15:52.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.173+0000 7fdaf2542640 1 -- 172.21.15.107:0/1830839248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaec19ff80 con 0x7fdaec103340 2024-04-25T00:15:52.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.174+0000 7fdaf2542640 1 -- 172.21.15.107:0/1830839248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaec108ab0 con 0x7fdaec103340 2024-04-25T00:15:52.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.175+0000 7fdae97fa640 1 -- 172.21.15.107:0/1830839248 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fdae401c070 con 0x7fdaec103340 2024-04-25T00:15:52.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.175+0000 7fdae97fa640 1 --2- 172.21.15.107:0/1830839248 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fdad40a2b90 0x7fdad40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:52.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.176+0000 7fdae97fa640 1 -- 172.21.15.107:0/1830839248 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fdae40cede0 con 0x7fdaec103340 2024-04-25T00:15:52.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.176+0000 7fdaebfff640 1 --2- 172.21.15.107:0/1830839248 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fdad40a2b90 0x7fdad40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:52.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.176+0000 7fdaebfff640 1 --2- 172.21.15.107:0/1830839248 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fdad40a2b90 0x7fdad40a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0058c0 tx=0x7fdadc005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:52.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.179+0000 7fdae97fa640 1 -- 172.21.15.107:0/1830839248 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdae409b3b0 con 0x7fdaec103340 2024-04-25T00:15:52.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:52 smithi195 ceph-mon[32833]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:15:52.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:52 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3818629948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:15:52.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:52 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1628317550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:15:52.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:52 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3668341985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:15:52.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.389+0000 7fdaf2542640 1 -- 172.21.15.107:0/1830839248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdaec02d170 con 0x7fdaec103340 2024-04-25T00:15:52.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.390+0000 7fdae97fa640 1 -- 172.21.15.107:0/1830839248 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11897 (secure 0 0 0) 0x7fdae409f6b0 con 0x7fdaec103340 2024-04-25T00:15:52.391 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:15:52.391 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:14:28.529865+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24521,24493]},"id":1}]} 2024-04-25T00:15:52.391 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 23 2024-04-25T00:15:52.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.392+0000 7fdaf2542640 1 -- 172.21.15.107:0/1830839248 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fdad40a2b90 msgr2=0x7fdad40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:52.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.392+0000 7fdaf2542640 1 --2- 172.21.15.107:0/1830839248 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fdad40a2b90 0x7fdad40a5050 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0058c0 tx=0x7fdadc005850 comp rx=0 tx=0).stop 2024-04-25T00:15:52.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.392+0000 7fdaf2542640 1 -- 172.21.15.107:0/1830839248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaec103340 msgr2=0x7fdaec19f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:52.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.392+0000 7fdaf2542640 1 --2- 172.21.15.107:0/1830839248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaec103340 0x7fdaec19f0d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdae400b970 tx=0x7fdae400be30 comp rx=0 tx=0).stop 2024-04-25T00:15:52.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.392+0000 7fdaf2542640 1 -- 172.21.15.107:0/1830839248 shutdown_connections 2024-04-25T00:15:52.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.392+0000 7fdaf2542640 1 --2- 172.21.15.107:0/1830839248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaec103340 0x7fdaec19f0d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.392+0000 7fdaf2542640 1 --2- 172.21.15.107:0/1830839248 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fdad40a2b90 0x7fdad40a5050 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.392+0000 7fdaf2542640 1 --2- 172.21.15.107:0/1830839248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaec102980 0x7fdaec198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.392+0000 7fdaf2542640 1 --2- 172.21.15.107:0/1830839248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdaec101790 0x7fdaec197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.392+0000 7fdaf2542640 1 -- 172.21.15.107:0/1830839248 >> 172.21.15.107:0/1830839248 conn(0x7fdaec0fcd20 msgr2=0x7fdaec105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:52.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.392+0000 7fdaf2542640 1 -- 172.21.15.107:0/1830839248 shutdown_connections 2024-04-25T00:15:52.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.392+0000 7fdaf2542640 1 -- 172.21.15.107:0/1830839248 wait complete. 2024-04-25T00:15:52.404 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:15:52.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:52 smithi170 ceph-mon[32807]: pgmap v349: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:15:52.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:52 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3818629948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:15:52.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:52 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1628317550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:15:52.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:52 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3668341985' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:15:52.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.503+0000 7fb4cd0e1640 1 -- 172.21.15.107:0/776994643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c8102240 msgr2=0x7fb4c8108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:52.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.503+0000 7fb4cd0e1640 1 --2- 172.21.15.107:0/776994643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c8102240 0x7fb4c8108ad0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009ca0 tx=0x7fb4c002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.503+0000 7fb4cd0e1640 1 -- 172.21.15.107:0/776994643 shutdown_connections 2024-04-25T00:15:52.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.503+0000 7fb4cd0e1640 1 --2- 172.21.15.107:0/776994643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c8102240 0x7fb4c8108ad0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.503+0000 7fb4cd0e1640 1 --2- 172.21.15.107:0/776994643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4c8101880 0x7fb4c8101d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.503+0000 7fb4cd0e1640 1 --2- 172.21.15.107:0/776994643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4c8100690 0x7fb4c8100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.503+0000 7fb4cd0e1640 1 -- 172.21.15.107:0/776994643 >> 172.21.15.107:0/776994643 conn(0x7fb4c80fbc20 msgr2=0x7fb4c80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:52.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.503+0000 7fb4cd0e1640 1 -- 172.21.15.107:0/776994643 shutdown_connections 2024-04-25T00:15:52.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.503+0000 7fb4cd0e1640 1 -- 172.21.15.107:0/776994643 wait complete. 2024-04-25T00:15:52.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.503+0000 7fb4cd0e1640 1 Processor -- start 2024-04-25T00:15:52.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.503+0000 7fb4cd0e1640 1 -- start start 2024-04-25T00:15:52.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.504+0000 7fb4cd0e1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4c8100690 0x7fb4c8197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:52.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.504+0000 7fb4cd0e1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4c8101880 0x7fb4c8197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:52.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.504+0000 7fb4cd0e1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c8102240 0x7fb4c819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:52.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.504+0000 7fb4cd0e1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c810d430 con 0x7fb4c8102240 2024-04-25T00:15:52.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.504+0000 7fb4cd0e1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c810d2b0 con 0x7fb4c8100690 2024-04-25T00:15:52.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.504+0000 7fb4cd0e1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c810d5b0 con 0x7fb4c8101880 2024-04-25T00:15:52.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.504+0000 7fb4c759e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c8102240 0x7fb4c819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:52.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.504+0000 7fb4c759e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c8102240 0x7fb4c819efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40312/0 (socket says 172.21.15.107:40312) 2024-04-25T00:15:52.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.504+0000 7fb4c759e640 1 -- 172.21.15.107:0/2220923054 learned_addr learned my addr 172.21.15.107:0/2220923054 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:52.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.504+0000 7fb4c659c640 1 --2- 172.21.15.107:0/2220923054 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4c8101880 0x7fb4c8197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:52.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.504+0000 7fb4c6d9d640 1 --2- 172.21.15.107:0/2220923054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4c8100690 0x7fb4c8197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:52.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.504+0000 7fb4c759e640 1 -- 172.21.15.107:0/2220923054 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4c8101880 msgr2=0x7fb4c8197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:52.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.504+0000 7fb4c759e640 1 --2- 172.21.15.107:0/2220923054 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4c8101880 0x7fb4c8197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.504+0000 7fb4c759e640 1 -- 172.21.15.107:0/2220923054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4c8100690 msgr2=0x7fb4c8197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:52.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.504+0000 7fb4c759e640 1 --2- 172.21.15.107:0/2220923054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4c8100690 0x7fb4c8197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.504+0000 7fb4c759e640 1 -- 172.21.15.107:0/2220923054 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c819f650 con 0x7fb4c8102240 2024-04-25T00:15:52.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.504+0000 7fb4c759e640 1 --2- 172.21.15.107:0/2220923054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c8102240 0x7fb4c819efe0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0005e30 tx=0x7fb4c0031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:52.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.504+0000 7fb4affff640 1 -- 172.21.15.107:0/2220923054 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c0003c30 con 0x7fb4c8102240 2024-04-25T00:15:52.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.504+0000 7fb4cd0e1640 1 -- 172.21.15.107:0/2220923054 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c819f8e0 con 0x7fb4c8102240 2024-04-25T00:15:52.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.504+0000 7fb4affff640 1 -- 172.21.15.107:0/2220923054 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4c0003dd0 con 0x7fb4c8102240 2024-04-25T00:15:52.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.504+0000 7fb4affff640 1 -- 172.21.15.107:0/2220923054 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c002fde0 con 0x7fb4c8102240 2024-04-25T00:15:52.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.504+0000 7fb4cd0e1640 1 -- 172.21.15.107:0/2220923054 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c819fe80 con 0x7fb4c8102240 2024-04-25T00:15:52.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.505+0000 7fb4cd0e1640 1 -- 172.21.15.107:0/2220923054 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4c81079b0 con 0x7fb4c8102240 2024-04-25T00:15:52.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.506+0000 7fb4affff640 1 -- 172.21.15.107:0/2220923054 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb4c0004050 con 0x7fb4c8102240 2024-04-25T00:15:52.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.507+0000 7fb4affff640 1 --2- 172.21.15.107:0/2220923054 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb4b00a2b90 0x7fb4b00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:52.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.507+0000 7fb4c6d9d640 1 --2- 172.21.15.107:0/2220923054 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb4b00a2b90 0x7fb4b00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:52.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.507+0000 7fb4affff640 1 -- 172.21.15.107:0/2220923054 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fb4c00f06f0 con 0x7fb4c8102240 2024-04-25T00:15:52.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.507+0000 7fb4c6d9d640 1 --2- 172.21.15.107:0/2220923054 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb4b00a2b90 0x7fb4b00a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb4c81016e0 tx=0x7fb4bc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:52.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.511+0000 7fb4affff640 1 -- 172.21.15.107:0/2220923054 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb4c00bd310 con 0x7fb4c8102240 2024-04-25T00:15:52.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.689+0000 7fb4cd0e1640 1 -- 172.21.15.107:0/2220923054 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4c8062340 con 0x7fb4c8102240 2024-04-25T00:15:52.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.691+0000 7fb4affff640 1 -- 172.21.15.107:0/2220923054 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24576 (secure 0 0 0) 0x7fb4c00bcc10 con 0x7fb4c8102240 2024-04-25T00:15:52.692 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:15:52.692 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":94,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:15:02.070002+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:59.672801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:00.957782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:14:05.065846+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8},{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:15:52.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.693+0000 7fb4cd0e1640 1 -- 172.21.15.107:0/2220923054 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb4b00a2b90 msgr2=0x7fb4b00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:52.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.693+0000 7fb4cd0e1640 1 --2- 172.21.15.107:0/2220923054 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb4b00a2b90 0x7fb4b00a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb4c81016e0 tx=0x7fb4bc00b390 comp rx=0 tx=0).stop 2024-04-25T00:15:52.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.693+0000 7fb4cd0e1640 1 -- 172.21.15.107:0/2220923054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c8102240 msgr2=0x7fb4c819efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:52.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.693+0000 7fb4cd0e1640 1 --2- 172.21.15.107:0/2220923054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c8102240 0x7fb4c819efe0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0005e30 tx=0x7fb4c0031ee0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.693+0000 7fb4cd0e1640 1 -- 172.21.15.107:0/2220923054 shutdown_connections 2024-04-25T00:15:52.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.693+0000 7fb4cd0e1640 1 --2- 172.21.15.107:0/2220923054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4c8102240 0x7fb4c819efe0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.693+0000 7fb4cd0e1640 1 --2- 172.21.15.107:0/2220923054 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4c8101880 0x7fb4c8197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.693+0000 7fb4cd0e1640 1 --2- 172.21.15.107:0/2220923054 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb4b00a2b90 0x7fb4b00a5050 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.693+0000 7fb4cd0e1640 1 --2- 172.21.15.107:0/2220923054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4c8100690 0x7fb4c8197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.693+0000 7fb4cd0e1640 1 -- 172.21.15.107:0/2220923054 >> 172.21.15.107:0/2220923054 conn(0x7fb4c80fbc20 msgr2=0x7fb4c8104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:52.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.694+0000 7fb4cd0e1640 1 -- 172.21.15.107:0/2220923054 shutdown_connections 2024-04-25T00:15:52.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.694+0000 7fb4cd0e1640 1 -- 172.21.15.107:0/2220923054 wait complete. 2024-04-25T00:15:52.706 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:15:52.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.803+0000 7f162f330640 1 -- 172.21.15.107:0/1604369175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16281018f0 msgr2=0x7f1628101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:52.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.803+0000 7f162f330640 1 --2- 172.21.15.107:0/1604369175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16281018f0 0x7f1628101d70 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f161c009900 tx=0x7f161c02a410 comp rx=0 tx=0).stop 2024-04-25T00:15:52.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.803+0000 7f162f330640 1 -- 172.21.15.107:0/1604369175 shutdown_connections 2024-04-25T00:15:52.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.803+0000 7f162f330640 1 --2- 172.21.15.107:0/1604369175 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f16281022b0 0x7f1628108b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.803+0000 7f162f330640 1 --2- 172.21.15.107:0/1604369175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16281018f0 0x7f1628101d70 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.803+0000 7f162f330640 1 --2- 172.21.15.107:0/1604369175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1628100700 0x7f1628100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.803+0000 7f162f330640 1 -- 172.21.15.107:0/1604369175 >> 172.21.15.107:0/1604369175 conn(0x7f16280fbc90 msgr2=0x7f16280fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:52.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.803+0000 7f162f330640 1 -- 172.21.15.107:0/1604369175 shutdown_connections 2024-04-25T00:15:52.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.803+0000 7f162f330640 1 -- 172.21.15.107:0/1604369175 wait complete. 2024-04-25T00:15:52.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.803+0000 7f162f330640 1 Processor -- start 2024-04-25T00:15:52.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.803+0000 7f162f330640 1 -- start start 2024-04-25T00:15:52.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.804+0000 7f162f330640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1628100700 0x7f1628197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:52.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.804+0000 7f162f330640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16281018f0 0x7f1628197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:52.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.804+0000 7f162f330640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16281022b0 0x7f162819f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:52.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.804+0000 7f162f330640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f162810c310 con 0x7f16281022b0 2024-04-25T00:15:52.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.804+0000 7f162f330640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f162810c190 con 0x7f16281018f0 2024-04-25T00:15:52.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.804+0000 7f162f330640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f162810c490 con 0x7f1628100700 2024-04-25T00:15:52.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.804+0000 7f162d8cd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16281022b0 0x7f162819f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:52.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.804+0000 7f162d8cd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16281022b0 0x7f162819f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40330/0 (socket says 172.21.15.107:40330) 2024-04-25T00:15:52.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.804+0000 7f162d8cd640 1 -- 172.21.15.107:0/1803030337 learned_addr learned my addr 172.21.15.107:0/1803030337 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:52.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.804+0000 7f162d0cc640 1 --2- 172.21.15.107:0/1803030337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1628100700 0x7f1628197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:52.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.804+0000 7f162d8cd640 1 -- 172.21.15.107:0/1803030337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1628100700 msgr2=0x7f1628197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:52.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.804+0000 7f162d8cd640 1 --2- 172.21.15.107:0/1803030337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1628100700 0x7f1628197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.804+0000 7f162c8cb640 1 --2- 172.21.15.107:0/1803030337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16281018f0 0x7f1628197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:52.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.804+0000 7f162d8cd640 1 -- 172.21.15.107:0/1803030337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16281018f0 msgr2=0x7f1628197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:52.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.804+0000 7f162d8cd640 1 --2- 172.21.15.107:0/1803030337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16281018f0 0x7f1628197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.804+0000 7f162d8cd640 1 -- 172.21.15.107:0/1803030337 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162819f6d0 con 0x7f16281022b0 2024-04-25T00:15:52.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.804+0000 7f162d8cd640 1 --2- 172.21.15.107:0/1803030337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16281022b0 0x7f162819f060 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f161800d750 tx=0x7f161800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:52.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.804+0000 7f16267fc640 1 -- 172.21.15.107:0/1803030337 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618017070 con 0x7f16281022b0 2024-04-25T00:15:52.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.804+0000 7f16267fc640 1 -- 172.21.15.107:0/1803030337 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1618005500 con 0x7f16281022b0 2024-04-25T00:15:52.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.804+0000 7f16267fc640 1 -- 172.21.15.107:0/1803030337 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618004d90 con 0x7f16281022b0 2024-04-25T00:15:52.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.804+0000 7f162f330640 1 -- 172.21.15.107:0/1803030337 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162819f9c0 con 0x7f16281022b0 2024-04-25T00:15:52.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.804+0000 7f162f330640 1 -- 172.21.15.107:0/1803030337 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f162819ff10 con 0x7f16281022b0 2024-04-25T00:15:52.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.805+0000 7f162f330640 1 -- 172.21.15.107:0/1803030337 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16281a01a0 con 0x7f16281022b0 2024-04-25T00:15:52.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.806+0000 7f16267fc640 1 -- 172.21.15.107:0/1803030337 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f1618004f30 con 0x7f16281022b0 2024-04-25T00:15:52.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.807+0000 7f16267fc640 1 --2- 172.21.15.107:0/1803030337 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f16100a2c10 0x7f16100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:52.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.807+0000 7f162d0cc640 1 --2- 172.21.15.107:0/1803030337 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f16100a2c10 0x7f16100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:52.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.807+0000 7f16267fc640 1 -- 172.21.15.107:0/1803030337 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f16180cf170 con 0x7f16281022b0 2024-04-25T00:15:52.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.807+0000 7f162d0cc640 1 --2- 172.21.15.107:0/1803030337 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f16100a2c10 0x7f16100a50d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f162000a300 tx=0x7f162000a290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:52.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:52.810+0000 7f16267fc640 1 -- 172.21.15.107:0/1803030337 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f161809be10 con 0x7f16281022b0 2024-04-25T00:15:53.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.022+0000 7f162f330640 1 -- 172.21.15.107:0/1803030337 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16281a0370 con 0x7f16281022b0 2024-04-25T00:15:53.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.023+0000 7f16267fc640 1 -- 172.21.15.107:0/1803030337 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11897 (secure 0 0 0) 0x7f161809b710 con 0x7f16281022b0 2024-04-25T00:15:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:15:53.023 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:14:28.529865+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24521,24493]},"id":1}]} 2024-04-25T00:15:53.023 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 23 2024-04-25T00:15:53.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.025+0000 7f162f330640 1 -- 172.21.15.107:0/1803030337 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f16100a2c10 msgr2=0x7f16100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.025+0000 7f162f330640 1 --2- 172.21.15.107:0/1803030337 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f16100a2c10 0x7f16100a50d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f162000a300 tx=0x7f162000a290 comp rx=0 tx=0).stop 2024-04-25T00:15:53.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.025+0000 7f162f330640 1 -- 172.21.15.107:0/1803030337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16281022b0 msgr2=0x7f162819f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.025+0000 7f162f330640 1 --2- 172.21.15.107:0/1803030337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16281022b0 0x7f162819f060 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f161800d750 tx=0x7f161800dc10 comp rx=0 tx=0).stop 2024-04-25T00:15:53.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.025+0000 7f162f330640 1 -- 172.21.15.107:0/1803030337 shutdown_connections 2024-04-25T00:15:53.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.025+0000 7f162f330640 1 --2- 172.21.15.107:0/1803030337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16281022b0 0x7f162819f060 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.025+0000 7f162f330640 1 --2- 172.21.15.107:0/1803030337 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f16100a2c10 0x7f16100a50d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.025+0000 7f162f330640 1 --2- 172.21.15.107:0/1803030337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16281018f0 0x7f1628197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.025+0000 7f162f330640 1 --2- 172.21.15.107:0/1803030337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1628100700 0x7f1628197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.025+0000 7f162f330640 1 -- 172.21.15.107:0/1803030337 >> 172.21.15.107:0/1803030337 conn(0x7f16280fbc90 msgr2=0x7f16280fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:53.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.025+0000 7f162f330640 1 -- 172.21.15.107:0/1803030337 shutdown_connections 2024-04-25T00:15:53.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.025+0000 7f162f330640 1 -- 172.21.15.107:0/1803030337 wait complete. 2024-04-25T00:15:53.037 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T00:14:01.788955+0000', 'modified': '2024-04-25T00:14:28.529865+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24521, 'mds_1': 24493, 'mds_2': 24485}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6835/2926641872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 2926641872}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 2926641872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.195:6835/2828335061', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 2828335061}, {'type': 'v1', 'addr': '172.21.15.195:6835', 'nonce': 2828335061}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.195:6837/2245799531', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6836', 'nonce': 2245799531}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 2245799531}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.170:6837/3616315471', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6836', 'nonce': 3616315471}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 3616315471}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.195:6839/4133864926', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6838', 'nonce': 4133864926}, {'type': 'v1', 'addr': '172.21.15.195:6839', 'nonce': 4133864926}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24521': {'gid': 24521, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.170:6839/3558784302', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 3558784302}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 3558784302}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24521, 'qdb_cluster': [24485, 24521, 24493]} 2024-04-25T00:15:53.037 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T00:15:53.038 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-25T00:15:53.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.135+0000 7fe145215640 1 -- 172.21.15.107:0/1806180714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe140101870 msgr2=0x7fe140101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.135+0000 7fe145215640 1 --2- 172.21.15.107:0/1806180714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe140101870 0x7fe140101cf0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe134009900 tx=0x7fe13402a410 comp rx=0 tx=0).stop 2024-04-25T00:15:53.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.135+0000 7fe145215640 1 -- 172.21.15.107:0/1806180714 shutdown_connections 2024-04-25T00:15:53.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.135+0000 7fe145215640 1 --2- 172.21.15.107:0/1806180714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe140102230 0x7fe140108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.135+0000 7fe145215640 1 --2- 172.21.15.107:0/1806180714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe140101870 0x7fe140101cf0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.135+0000 7fe145215640 1 --2- 172.21.15.107:0/1806180714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe140100680 0x7fe140100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.135+0000 7fe145215640 1 -- 172.21.15.107:0/1806180714 >> 172.21.15.107:0/1806180714 conn(0x7fe1400fbbf0 msgr2=0x7fe1400fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:53.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.135+0000 7fe145215640 1 -- 172.21.15.107:0/1806180714 shutdown_connections 2024-04-25T00:15:53.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.135+0000 7fe145215640 1 -- 172.21.15.107:0/1806180714 wait complete. 2024-04-25T00:15:53.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.136+0000 7fe145215640 1 Processor -- start 2024-04-25T00:15:53.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.136+0000 7fe145215640 1 -- start start 2024-04-25T00:15:53.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.136+0000 7fe145215640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe140100680 0x7fe140069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.136+0000 7fe145215640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe140101870 0x7fe14019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.136+0000 7fe145215640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe140102230 0x7fe14019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.136+0000 7fe145215640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe14010c370 con 0x7fe140102230 2024-04-25T00:15:53.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.136+0000 7fe145215640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe14010c1f0 con 0x7fe140101870 2024-04-25T00:15:53.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.136+0000 7fe145215640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe14010c4f0 con 0x7fe140100680 2024-04-25T00:15:53.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.136+0000 7fe13f59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe140102230 0x7fe14019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.137+0000 7fe13f59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe140102230 0x7fe14019eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40352/0 (socket says 172.21.15.107:40352) 2024-04-25T00:15:53.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.137+0000 7fe13f59e640 1 -- 172.21.15.107:0/70365785 learned_addr learned my addr 172.21.15.107:0/70365785 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:53.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.137+0000 7fe13ed9d640 1 --2- 172.21.15.107:0/70365785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe140100680 0x7fe140069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.137+0000 7fe13e59c640 1 --2- 172.21.15.107:0/70365785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe140101870 0x7fe14019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.137+0000 7fe13f59e640 1 -- 172.21.15.107:0/70365785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe140100680 msgr2=0x7fe140069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.137+0000 7fe13f59e640 1 --2- 172.21.15.107:0/70365785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe140100680 0x7fe140069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.137+0000 7fe13f59e640 1 -- 172.21.15.107:0/70365785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe140101870 msgr2=0x7fe14019c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.137+0000 7fe13f59e640 1 --2- 172.21.15.107:0/70365785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe140101870 0x7fe14019c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.137+0000 7fe13f59e640 1 -- 172.21.15.107:0/70365785 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe14019f7c0 con 0x7fe140102230 2024-04-25T00:15:53.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.137+0000 7fe13e59c640 1 --2- 172.21.15.107:0/70365785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe140101870 0x7fe14019c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:53.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.137+0000 7fe13ed9d640 1 --2- 172.21.15.107:0/70365785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe140100680 0x7fe140069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:53.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.137+0000 7fe13f59e640 1 --2- 172.21.15.107:0/70365785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe140102230 0x7fe14019eeb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe13800eaa0 tx=0x7fe13800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:53.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.137+0000 7fe127fff640 1 -- 172.21.15.107:0/70365785 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe138004d20 con 0x7fe140102230 2024-04-25T00:15:53.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.137+0000 7fe127fff640 1 -- 172.21.15.107:0/70365785 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe138005020 con 0x7fe140102230 2024-04-25T00:15:53.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.137+0000 7fe145215640 1 -- 172.21.15.107:0/70365785 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe14019fa50 con 0x7fe140102230 2024-04-25T00:15:53.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.137+0000 7fe127fff640 1 -- 172.21.15.107:0/70365785 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1380116d0 con 0x7fe140102230 2024-04-25T00:15:53.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.137+0000 7fe145215640 1 -- 172.21.15.107:0/70365785 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe14019ffa0 con 0x7fe140102230 2024-04-25T00:15:53.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.139+0000 7fe127fff640 1 -- 172.21.15.107:0/70365785 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe1380051d0 con 0x7fe140102230 2024-04-25T00:15:53.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.140+0000 7fe148003740 1 Processor -- start 2024-04-25T00:15:53.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.140+0000 7fe148003740 1 -- start start 2024-04-25T00:15:53.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.140+0000 7fe148003740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c0690440 0x55f9c0690860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.140+0000 7fe127fff640 1 --2- 172.21.15.107:0/70365785 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe1280a2b90 0x7fe1280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.140+0000 7fe148003740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0690e30 0x55f9c0695930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.140+0000 7fe148003740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c0695fa0 0x55f9c0698430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.140+0000 7fe148003740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f9c05f3f00 con 0x55f9c0690e30 2024-04-25T00:15:53.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.140+0000 7fe148003740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f9c059c750 con 0x55f9c0690440 2024-04-25T00:15:53.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.140+0000 7fe148003740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f9c05efec0 con 0x55f9c0695fa0 2024-04-25T00:15:53.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.140+0000 7fe13e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0690e30 0x55f9c0695930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.140+0000 7fe13e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0690e30 0x55f9c0695930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40366/0 (socket says 172.21.15.107:40366) 2024-04-25T00:15:53.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.140+0000 7fe13e59c640 1 -- 172.21.15.107:0/1159046582 learned_addr learned my addr 172.21.15.107:0/1159046582 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:53.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.140+0000 7fe13ed9d640 1 --2- 172.21.15.107:0/70365785 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe1280a2b90 0x7fe1280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.140+0000 7fe13e59c640 1 --2- 172.21.15.107:0/1159046582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c0690440 0x55f9c0690860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.140+0000 7fe127fff640 1 -- 172.21.15.107:0/70365785 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fe1380cf5f0 con 0x7fe140102230 2024-04-25T00:15:53.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.140+0000 7fe13f59e640 1 --2- 172.21.15.107:0/1159046582 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c0695fa0 0x55f9c0698430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.140+0000 7fe13e59c640 1 -- 172.21.15.107:0/1159046582 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c0695fa0 msgr2=0x55f9c0698430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:53.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.140+0000 7fe13e59c640 1 --2- 172.21.15.107:0/1159046582 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c0695fa0 0x55f9c0698430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.140+0000 7fe13e59c640 1 -- 172.21.15.107:0/1159046582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c0690440 msgr2=0x55f9c0690860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:53.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.140+0000 7fe13e59c640 1 --2- 172.21.15.107:0/1159046582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c0690440 0x55f9c0690860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.140+0000 7fe13e59c640 1 -- 172.21.15.107:0/1159046582 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9c03082d0 con 0x55f9c0690e30 2024-04-25T00:15:53.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.141+0000 7fe13ed9d640 1 --2- 172.21.15.107:0/70365785 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe1280a2b90 0x7fe1280a5050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe1401016d0 tx=0x7fe130009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:53.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.141+0000 7fe13e59c640 1 --2- 172.21.15.107:0/1159046582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0690e30 0x55f9c0695930 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe134005940 tx=0x7fe1340058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4331b4f5dbfd98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:53.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.141+0000 7fe1257fa640 1 -- 172.21.15.107:0/1159046582 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe134035040 con 0x55f9c0690e30 2024-04-25T00:15:53.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.141+0000 7fe1257fa640 1 -- 172.21.15.107:0/1159046582 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe134004230 con 0x55f9c0690e30 2024-04-25T00:15:53.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.141+0000 7fe1257fa640 1 -- 172.21.15.107:0/1159046582 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13403d6c0 con 0x55f9c0690e30 2024-04-25T00:15:53.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.141+0000 7fe148003740 1 -- 172.21.15.107:0/1159046582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0690e30 msgr2=0x55f9c0695930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.141+0000 7fe148003740 1 --2- 172.21.15.107:0/1159046582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0690e30 0x55f9c0695930 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe134005940 tx=0x7fe1340058b0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.141+0000 7fe148003740 1 -- 172.21.15.107:0/1159046582 shutdown_connections 2024-04-25T00:15:53.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.141+0000 7fe148003740 1 --2- 172.21.15.107:0/1159046582 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c0695fa0 0x55f9c0698430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.141+0000 7fe148003740 1 --2- 172.21.15.107:0/1159046582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0690e30 0x55f9c0695930 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.141+0000 7fe148003740 1 --2- 172.21.15.107:0/1159046582 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c0690440 0x55f9c0690860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.141+0000 7fe148003740 1 -- 172.21.15.107:0/1159046582 >> 172.21.15.107:0/1159046582 conn(0x55f9c068bab0 msgr2=0x55f9c068beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:53.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.141+0000 7fe148003740 1 -- 172.21.15.107:0/1159046582 shutdown_connections 2024-04-25T00:15:53.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.141+0000 7fe148003740 1 -- 172.21.15.107:0/1159046582 wait complete. 2024-04-25T00:15:53.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.142+0000 7fe148003740 1 -- start start 2024-04-25T00:15:53.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.142+0000 7fe148003740 1 Processor -- start 2024-04-25T00:15:53.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.142+0000 7fe148003740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c0690440 0x55f9c069a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.142+0000 7fe148003740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0690e30 0x55f9c069ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.142+0000 7fe148003740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c0695fa0 0x55f9c06a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.142+0000 7fe148003740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f9c0698ef0 con 0x55f9c0690e30 2024-04-25T00:15:53.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.142+0000 7fe148003740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f9c05afcf0 con 0x55f9c0695fa0 2024-04-25T00:15:53.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.142+0000 7fe148003740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f9c0699070 con 0x55f9c0690440 2024-04-25T00:15:53.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.142+0000 7fe148003740 10 client.0 _ll_register_callbacks cb 0x55f9c05af6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:15:53.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.142+0000 7fe13e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0690e30 0x55f9c069ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.142+0000 7fe13e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0690e30 0x55f9c069ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40368/0 (socket says 172.21.15.107:40368) 2024-04-25T00:15:53.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.142+0000 7fe13e59c640 1 -- 172.21.15.107:0/1116587042 learned_addr learned my addr 172.21.15.107:0/1116587042 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:53.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.142+0000 7fe13ed9d640 1 --2- 172.21.15.107:0/1116587042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c0690440 0x55f9c069a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.142+0000 7fe13e59c640 1 -- 172.21.15.107:0/1116587042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c0690440 msgr2=0x55f9c069a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:53.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.142+0000 7fe13e59c640 1 --2- 172.21.15.107:0/1116587042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c0690440 0x55f9c069a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.142+0000 7fe13e59c640 1 -- 172.21.15.107:0/1116587042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c0695fa0 msgr2=0x55f9c06a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:53.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.142+0000 7fe13e59c640 1 --2- 172.21.15.107:0/1116587042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c0695fa0 0x55f9c06a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.142+0000 7fe13e59c640 1 -- 172.21.15.107:0/1116587042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9c03d26b0 con 0x55f9c0690e30 2024-04-25T00:15:53.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.142+0000 7fe13e59c640 1 --2- 172.21.15.107:0/1116587042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0690e30 0x55f9c069ad70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe1340314e0 tx=0x7fe134003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231faddfdd22486e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:53.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.142+0000 7fe122ff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:15:53.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.143+0000 7fe122ff5640 1 -- 172.21.15.107:0/1116587042 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe134000e60 con 0x55f9c0690e30 2024-04-25T00:15:53.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.143+0000 7fe122ff5640 1 -- 172.21.15.107:0/1116587042 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fe13403d480 con 0x55f9c0690e30 2024-04-25T00:15:53.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.143+0000 7fe148003740 1 -- 172.21.15.107:0/1116587042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe134004230 con 0x55f9c0690e30 2024-04-25T00:15:53.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.143+0000 7fe122ff5640 1 -- 172.21.15.107:0/1116587042 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe134035050 con 0x55f9c0690e30 2024-04-25T00:15:53.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.143+0000 7fe148003740 10 client.15216 fetch_fsmap learned FSMap version 23 2024-04-25T00:15:53.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.143+0000 7fe148003740 1 -- 172.21.15.107:0/1116587042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55f9c06a4d90 con 0x55f9c0690e30 2024-04-25T00:15:53.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.143+0000 7fe122ff5640 1 -- 172.21.15.107:0/1116587042 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe134004230 con 0x55f9c0690e30 2024-04-25T00:15:53.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.143+0000 7fe122ff5640 1 -- 172.21.15.107:0/1116587042 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fe13403a040 con 0x55f9c0690e30 2024-04-25T00:15:53.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.143+0000 7fe148003740 10 client.15216 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:15:53.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.143+0000 7fe148003740 10 client.15216 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T00:15:53.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.143+0000 7fe148003740 20 client.15216 populate_metadata read hostname 'smithi107' 2024-04-25T00:15:53.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.143+0000 7fe148003740 1 --2- 172.21.15.107:0/1116587042 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55f9c06a60e0 0x55f9c06a82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.143+0000 7fe148003740 4 client.15216 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:15:53.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.143+0000 7fe148003740 1 -- 172.21.15.107:0/1116587042 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9c0601440 con 0x55f9c06a60e0 2024-04-25T00:15:53.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.143+0000 7fe13ed9d640 1 --2- 172.21.15.107:0/1116587042 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55f9c06a60e0 0x55f9c06a82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.144+0000 7fe122ff5640 10 client.15216 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:15:53.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.144+0000 7fe13ed9d640 1 --2- 172.21.15.107:0/1116587042 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55f9c06a60e0 0x55f9c06a82b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9871056be6743d21 server_cookie=8819f8db21dc5774 in_seq=0 out_seq=0 2024-04-25T00:15:53.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.145+0000 7fe122ff5640 1 -- 172.21.15.107:0/1116587042 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f9c0601440 con 0x55f9c06a60e0 2024-04-25T00:15:53.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.145+0000 7fe122ff5640 10 client.15216 handle_command_reply: tid=0 2024-04-25T00:15:53.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.145+0000 7fe148003740 1 client.15216 shutdown 2024-04-25T00:15:53.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.145+0000 7fe148003740 1 -- 172.21.15.107:0/1116587042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0690e30 msgr2=0x55f9c069ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.145+0000 7fe148003740 1 --2- 172.21.15.107:0/1116587042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0690e30 0x55f9c069ad70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe1340314e0 tx=0x7fe134003aa0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.145+0000 7fe148003740 1 -- 172.21.15.107:0/1116587042 shutdown_connections 2024-04-25T00:15:53.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.145+0000 7fe148003740 1 --2- 172.21.15.107:0/1116587042 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55f9c06a60e0 0x55f9c06a82b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.145+0000 7fe148003740 1 --2- 172.21.15.107:0/1116587042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c0695fa0 0x55f9c06a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.145+0000 7fe122ff5640 0 client.15216 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:15:53.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.145+0000 7fe148003740 1 --2- 172.21.15.107:0/1116587042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0690e30 0x55f9c069ad70 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.145+0000 7fe148003740 1 --2- 172.21.15.107:0/1116587042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c0690440 0x55f9c069a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.145+0000 7fe148003740 1 -- 172.21.15.107:0/1116587042 >> 172.21.15.107:0/1116587042 conn(0x55f9c068bab0 msgr2=0x55f9c068aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:53.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.145+0000 7fe148003740 1 -- 172.21.15.107:0/1116587042 shutdown_connections 2024-04-25T00:15:53.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.145+0000 7fe148003740 1 -- 172.21.15.107:0/1116587042 wait complete. 2024-04-25T00:15:53.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.145+0000 7fe148003740 20 client.15216 trim_cache size 0 max 16384 2024-04-25T00:15:53.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.163+0000 7fe148003740 1 Processor -- start 2024-04-25T00:15:53.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.163+0000 7fe148003740 1 -- start start 2024-04-25T00:15:53.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.163+0000 7fe148003740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c0690440 0x55f9c06918a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.163+0000 7fe148003740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c0690e30 0x55f9c06a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.163+0000 7fe148003740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0695fa0 0x55f9c0686690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.163+0000 7fe148003740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f9c06a65f0 con 0x55f9c0695fa0 2024-04-25T00:15:53.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.163+0000 7fe148003740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f9c06a6cc0 con 0x55f9c0690440 2024-04-25T00:15:53.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.163+0000 7fe148003740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f9c06a7390 con 0x55f9c0690e30 2024-04-25T00:15:53.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.163+0000 7fe13f59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0695fa0 0x55f9c0686690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.163+0000 7fe13f59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0695fa0 0x55f9c0686690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40380/0 (socket says 172.21.15.107:40380) 2024-04-25T00:15:53.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.163+0000 7fe13f59e640 1 -- 172.21.15.107:0/2939676177 learned_addr learned my addr 172.21.15.107:0/2939676177 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:53.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.163+0000 7fe13e59c640 1 --2- 172.21.15.107:0/2939676177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c0690e30 0x55f9c06a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.163+0000 7fe13f59e640 1 -- 172.21.15.107:0/2939676177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c0690e30 msgr2=0x55f9c06a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:53.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.163+0000 7fe13f59e640 1 --2- 172.21.15.107:0/2939676177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c0690e30 0x55f9c06a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.163+0000 7fe13ed9d640 1 --2- 172.21.15.107:0/2939676177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c0690440 0x55f9c06918a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.163+0000 7fe13f59e640 1 -- 172.21.15.107:0/2939676177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c0690440 msgr2=0x55f9c06918a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:53.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.163+0000 7fe13f59e640 1 --2- 172.21.15.107:0/2939676177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c0690440 0x55f9c06918a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.163+0000 7fe13f59e640 1 -- 172.21.15.107:0/2939676177 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9c06a7f10 con 0x55f9c0695fa0 2024-04-25T00:15:53.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.163+0000 7fe13f59e640 1 --2- 172.21.15.107:0/2939676177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0695fa0 0x55f9c0686690 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe138097f70 tx=0x7fe13809cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fdaf9cd6342ebf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:53.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.163+0000 7fe124ff9640 1 -- 172.21.15.107:0/2939676177 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13801f070 con 0x55f9c0695fa0 2024-04-25T00:15:53.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.163+0000 7fe124ff9640 1 -- 172.21.15.107:0/2939676177 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe13809ce40 con 0x55f9c0695fa0 2024-04-25T00:15:53.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.163+0000 7fe124ff9640 1 -- 172.21.15.107:0/2939676177 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe138005560 con 0x55f9c0695fa0 2024-04-25T00:15:53.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.164+0000 7fe148003740 1 -- 172.21.15.107:0/2939676177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0695fa0 msgr2=0x55f9c0686690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.164+0000 7fe148003740 1 --2- 172.21.15.107:0/2939676177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0695fa0 0x55f9c0686690 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe138097f70 tx=0x7fe13809cd00 comp rx=0 tx=0).stop 2024-04-25T00:15:53.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.164+0000 7fe148003740 1 -- 172.21.15.107:0/2939676177 shutdown_connections 2024-04-25T00:15:53.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.164+0000 7fe148003740 1 --2- 172.21.15.107:0/2939676177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0695fa0 0x55f9c0686690 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.164+0000 7fe148003740 1 --2- 172.21.15.107:0/2939676177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c0690e30 0x55f9c06a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.164+0000 7fe148003740 1 --2- 172.21.15.107:0/2939676177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c0690440 0x55f9c06918a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.164+0000 7fe148003740 1 -- 172.21.15.107:0/2939676177 >> 172.21.15.107:0/2939676177 conn(0x55f9c068bab0 msgr2=0x55f9c068be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:53.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.164+0000 7fe148003740 1 -- 172.21.15.107:0/2939676177 shutdown_connections 2024-04-25T00:15:53.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.164+0000 7fe148003740 1 -- 172.21.15.107:0/2939676177 wait complete. 2024-04-25T00:15:53.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.164+0000 7fe148003740 1 -- start start 2024-04-25T00:15:53.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.164+0000 7fe148003740 1 Processor -- start 2024-04-25T00:15:53.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe148003740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c0690440 0x55f9c069da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe148003740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c0690e30 0x55f9c069dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe148003740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0695fa0 0x55f9c06a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe148003740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f9c06a8130 con 0x55f9c0695fa0 2024-04-25T00:15:53.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe148003740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f9c06a7a60 con 0x55f9c0690440 2024-04-25T00:15:53.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe148003740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f9c06a8800 con 0x55f9c0690e30 2024-04-25T00:15:53.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe148003740 10 client.0 _ll_register_callbacks cb 0x7fe134003940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:15:53.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe13f59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0695fa0 0x55f9c06a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe13f59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0695fa0 0x55f9c06a0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40388/0 (socket says 172.21.15.107:40388) 2024-04-25T00:15:53.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe13f59e640 1 -- 172.21.15.107:0/724811365 learned_addr learned my addr 172.21.15.107:0/724811365 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:53.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe13e59c640 1 --2- 172.21.15.107:0/724811365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c0690e30 0x55f9c069dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe13ed9d640 1 --2- 172.21.15.107:0/724811365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c0690440 0x55f9c069da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe13f59e640 1 -- 172.21.15.107:0/724811365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c0690e30 msgr2=0x55f9c069dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:53.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe13f59e640 1 --2- 172.21.15.107:0/724811365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c0690e30 0x55f9c069dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe13f59e640 1 -- 172.21.15.107:0/724811365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c0690440 msgr2=0x55f9c069da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:53.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe13f59e640 1 --2- 172.21.15.107:0/724811365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c0690440 0x55f9c069da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe13f59e640 1 -- 172.21.15.107:0/724811365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9c06a15d0 con 0x55f9c0695fa0 2024-04-25T00:15:53.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe13e59c640 1 --2- 172.21.15.107:0/724811365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c0690e30 0x55f9c069dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:53.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe13f59e640 1 --2- 172.21.15.107:0/724811365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0695fa0 0x55f9c06a0fe0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe13809b810 tx=0x7fe1380ccd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390e2e08bfa8a76d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:53.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe122ff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:15:53.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe122ff5640 1 -- 172.21.15.107:0/724811365 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1380b3070 con 0x55f9c0695fa0 2024-04-25T00:15:53.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe122ff5640 1 -- 172.21.15.107:0/724811365 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fe1380cf350 con 0x55f9c0695fa0 2024-04-25T00:15:53.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe148003740 1 -- 172.21.15.107:0/724811365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe13809ce40 con 0x55f9c0695fa0 2024-04-25T00:15:53.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe122ff5640 1 -- 172.21.15.107:0/724811365 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe13801f040 con 0x55f9c0695fa0 2024-04-25T00:15:53.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe148003740 10 client.15222 fetch_fsmap learned FSMap version 23 2024-04-25T00:15:53.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.165+0000 7fe148003740 1 -- 172.21.15.107:0/724811365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55f9c070d410 con 0x55f9c0695fa0 2024-04-25T00:15:53.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.166+0000 7fe122ff5640 1 -- 172.21.15.107:0/724811365 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe13809ce40 con 0x55f9c0695fa0 2024-04-25T00:15:53.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.166+0000 7fe122ff5640 1 -- 172.21.15.107:0/724811365 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fe13800f040 con 0x55f9c0695fa0 2024-04-25T00:15:53.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.166+0000 7fe148003740 10 client.15222 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:15:53.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.166+0000 7fe148003740 10 client.15222 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T00:15:53.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.166+0000 7fe148003740 20 client.15222 populate_metadata read hostname 'smithi107' 2024-04-25T00:15:53.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.166+0000 7fe148003740 1 --2- 172.21.15.107:0/724811365 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55f9c0711b10 0x55f9c070e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.166+0000 7fe148003740 4 client.15222 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:15:53.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.166+0000 7fe148003740 1 -- 172.21.15.107:0/724811365 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "status"}) -- 0x55f9c05fb790 con 0x55f9c0711b10 2024-04-25T00:15:53.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.166+0000 7fe13ed9d640 1 --2- 172.21.15.107:0/724811365 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55f9c0711b10 0x55f9c070e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.167+0000 7fe122ff5640 10 client.15222 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:15:53.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.167+0000 7fe13ed9d640 1 --2- 172.21.15.107:0/724811365 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55f9c0711b10 0x55f9c070e620 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=edd637cbde7d4b11 server_cookie=f1dba2d14dd292a in_seq=0 out_seq=0 2024-04-25T00:15:53.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.167+0000 7fe122ff5640 1 -- 172.21.15.107:0/724811365 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f9c05fb790 con 0x55f9c0711b10 2024-04-25T00:15:53.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.167+0000 7fe122ff5640 10 client.15222 handle_command_reply: tid=0 2024-04-25T00:15:53.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.167+0000 7fe148003740 1 client.15222 shutdown 2024-04-25T00:15:53.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.167+0000 7fe148003740 1 -- 172.21.15.107:0/724811365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0695fa0 msgr2=0x55f9c06a0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.167+0000 7fe148003740 1 --2- 172.21.15.107:0/724811365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0695fa0 0x55f9c06a0fe0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe13809b810 tx=0x7fe1380ccd50 comp rx=0 tx=0).stop 2024-04-25T00:15:53.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.167+0000 7fe148003740 1 -- 172.21.15.107:0/724811365 shutdown_connections 2024-04-25T00:15:53.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.167+0000 7fe148003740 1 --2- 172.21.15.107:0/724811365 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55f9c0711b10 0x55f9c070e620 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.167+0000 7fe148003740 1 --2- 172.21.15.107:0/724811365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f9c0695fa0 0x55f9c06a0fe0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.167+0000 7fe122ff5640 0 client.15222 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:15:53.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.167+0000 7fe148003740 1 --2- 172.21.15.107:0/724811365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f9c0690e30 0x55f9c069dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.167+0000 7fe148003740 1 --2- 172.21.15.107:0/724811365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f9c0690440 0x55f9c069da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.167+0000 7fe148003740 1 -- 172.21.15.107:0/724811365 >> 172.21.15.107:0/724811365 conn(0x55f9c068bab0 msgr2=0x55f9c0686690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:53.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.167+0000 7fe148003740 1 -- 172.21.15.107:0/724811365 shutdown_connections 2024-04-25T00:15:53.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.167+0000 7fe148003740 1 -- 172.21.15.107:0/724811365 wait complete. 2024-04-25T00:15:53.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.167+0000 7fe148003740 20 client.15222 trim_cache size 0 max 16384 2024-04-25T00:15:53.179 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:15:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "79602f58-0297-11ef-bc93-c7b262605968", 2024-04-25T00:15:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 2, 2024-04-25T00:15:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24485, 2024-04-25T00:15:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-04-25T00:15:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-04-25T00:15:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-04-25T00:15:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 85.642363576999998, 2024-04-25T00:15:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 23, 2024-04-25T00:15:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 94, 2024-04-25T00:15:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 92, 2024-04-25T00:15:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 126.99373715199999 2024-04-25T00:15:53.180 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:15:53.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.167+0000 7fe145215640 1 -- 172.21.15.107:0/70365785 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe1280a2b90 msgr2=0x7fe1280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.167+0000 7fe145215640 1 --2- 172.21.15.107:0/70365785 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe1280a2b90 0x7fe1280a5050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe1401016d0 tx=0x7fe130009110 comp rx=0 tx=0).stop 2024-04-25T00:15:53.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.167+0000 7fe145215640 1 -- 172.21.15.107:0/70365785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe140102230 msgr2=0x7fe14019eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.167+0000 7fe145215640 1 --2- 172.21.15.107:0/70365785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe140102230 0x7fe14019eeb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe13800eaa0 tx=0x7fe13800eea0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.168+0000 7fe145215640 1 -- 172.21.15.107:0/70365785 shutdown_connections 2024-04-25T00:15:53.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.168+0000 7fe145215640 1 --2- 172.21.15.107:0/70365785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe140102230 0x7fe14019eeb0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.168+0000 7fe145215640 1 --2- 172.21.15.107:0/70365785 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe1280a2b90 0x7fe1280a5050 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.168+0000 7fe145215640 1 --2- 172.21.15.107:0/70365785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe140101870 0x7fe14019c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.168+0000 7fe145215640 1 --2- 172.21.15.107:0/70365785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe140100680 0x7fe140069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.168+0000 7fe145215640 1 -- 172.21.15.107:0/70365785 >> 172.21.15.107:0/70365785 conn(0x7fe1400fbbf0 msgr2=0x7fe140104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:53.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.168+0000 7fe145215640 1 -- 172.21.15.107:0/70365785 shutdown_connections 2024-04-25T00:15:53.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.168+0000 7fe145215640 1 -- 172.21.15.107:0/70365785 wait complete. 2024-04-25T00:15:53.182 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T00:15:53.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.280+0000 7f9424553640 1 -- 172.21.15.107:0/3605203129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f941c100680 msgr2=0x7f941c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.280+0000 7f9424553640 1 --2- 172.21.15.107:0/3605203129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f941c100680 0x7f941c100aa0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9418009ca0 tx=0x7f941802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.280+0000 7f9424553640 1 -- 172.21.15.107:0/3605203129 shutdown_connections 2024-04-25T00:15:53.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.280+0000 7f9424553640 1 --2- 172.21.15.107:0/3605203129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f941c102230 0x7f941c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.280+0000 7f9424553640 1 --2- 172.21.15.107:0/3605203129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f941c101870 0x7f941c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.280+0000 7f9424553640 1 --2- 172.21.15.107:0/3605203129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f941c100680 0x7f941c100aa0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.280+0000 7f9424553640 1 -- 172.21.15.107:0/3605203129 >> 172.21.15.107:0/3605203129 conn(0x7f941c0fbbf0 msgr2=0x7f941c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.280+0000 7f9424553640 1 -- 172.21.15.107:0/3605203129 shutdown_connections 2024-04-25T00:15:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.280+0000 7f9424553640 1 -- 172.21.15.107:0/3605203129 wait complete. 2024-04-25T00:15:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.281+0000 7f9424553640 1 Processor -- start 2024-04-25T00:15:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.281+0000 7f9424553640 1 -- start start 2024-04-25T00:15:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.281+0000 7f9424553640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f941c100680 0x7f941c069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.281+0000 7f9424553640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f941c101870 0x7f941c19c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.281+0000 7f9424553640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f941c102230 0x7f941c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.281+0000 7f9424553640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f941c10c2d0 con 0x7f941c101870 2024-04-25T00:15:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.281+0000 7f9424553640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f941c10c150 con 0x7f941c100680 2024-04-25T00:15:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.281+0000 7f9424553640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f941c10c450 con 0x7f941c102230 2024-04-25T00:15:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.281+0000 7f9421aee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f941c101870 0x7f941c19c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.281+0000 7f9421aee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f941c101870 0x7f941c19c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40398/0 (socket says 172.21.15.107:40398) 2024-04-25T00:15:53.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.281+0000 7f9421aee640 1 -- 172.21.15.107:0/3339183308 learned_addr learned my addr 172.21.15.107:0/3339183308 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:53.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.281+0000 7f94222ef640 1 --2- 172.21.15.107:0/3339183308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f941c100680 0x7f941c069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.281+0000 7f9422af0640 1 --2- 172.21.15.107:0/3339183308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f941c102230 0x7f941c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.282+0000 7f9421aee640 1 -- 172.21.15.107:0/3339183308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f941c102230 msgr2=0x7f941c19ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.282+0000 7f9421aee640 1 --2- 172.21.15.107:0/3339183308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f941c102230 0x7f941c19ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.282+0000 7f9421aee640 1 -- 172.21.15.107:0/3339183308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f941c100680 msgr2=0x7f941c069440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.282+0000 7f9421aee640 1 --2- 172.21.15.107:0/3339183308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f941c100680 0x7f941c069440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.282+0000 7f9421aee640 1 -- 172.21.15.107:0/3339183308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f941c19f720 con 0x7f941c101870 2024-04-25T00:15:53.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.282+0000 7f9422af0640 1 --2- 172.21.15.107:0/3339183308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f941c102230 0x7f941c19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:53.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.282+0000 7f9421aee640 1 --2- 172.21.15.107:0/3339183308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f941c101870 0x7f941c19c7f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f940c00c920 tx=0x7f940c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:53.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.282+0000 7f94137fe640 1 -- 172.21.15.107:0/3339183308 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c007c70 con 0x7f941c101870 2024-04-25T00:15:53.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.282+0000 7f94137fe640 1 -- 172.21.15.107:0/3339183308 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f940c007e10 con 0x7f941c101870 2024-04-25T00:15:53.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.282+0000 7f9424553640 1 -- 172.21.15.107:0/3339183308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f941c19f9b0 con 0x7f941c101870 2024-04-25T00:15:53.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.282+0000 7f94137fe640 1 -- 172.21.15.107:0/3339183308 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940c018890 con 0x7f941c101870 2024-04-25T00:15:53.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.282+0000 7f9424553640 1 -- 172.21.15.107:0/3339183308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f941c19ff00 con 0x7f941c101870 2024-04-25T00:15:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.284+0000 7f94137fe640 1 -- 172.21.15.107:0/3339183308 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f940c018a70 con 0x7f941c101870 2024-04-25T00:15:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.284+0000 7f942725d740 1 Processor -- start 2024-04-25T00:15:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.285+0000 7f942725d740 1 -- start start 2024-04-25T00:15:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.285+0000 7f942725d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6c024c440 0x55e6c024c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.285+0000 7f942725d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c024ce30 0x55e6c0251930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.285+0000 7f942725d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c0251fa0 0x55e6c0254430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.285+0000 7f942725d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e6c01aff00 con 0x55e6c024ce30 2024-04-25T00:15:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.285+0000 7f942725d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e6c0158750 con 0x55e6c024c440 2024-04-25T00:15:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.285+0000 7f942725d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e6c01abec0 con 0x55e6c0251fa0 2024-04-25T00:15:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.285+0000 7f94137fe640 1 --2- 172.21.15.107:0/3339183308 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f94080a2b90 0x7f94080a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.285+0000 7f94222ef640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c024ce30 0x55e6c0251930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.285+0000 7f94222ef640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c024ce30 0x55e6c0251930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40408/0 (socket says 172.21.15.107:40408) 2024-04-25T00:15:53.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.285+0000 7f94222ef640 1 -- 172.21.15.107:0/1802849583 learned_addr learned my addr 172.21.15.107:0/1802849583 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:53.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.285+0000 7f9422af0640 1 --2- 172.21.15.107:0/1802849583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6c024c440 0x55e6c024c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.285+0000 7f9422af0640 1 --2- 172.21.15.107:0/3339183308 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f94080a2b90 0x7f94080a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.285+0000 7f94137fe640 1 -- 172.21.15.107:0/3339183308 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f940c0cf4d0 con 0x7f941c101870 2024-04-25T00:15:53.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.285+0000 7f9421aee640 1 --2- 172.21.15.107:0/1802849583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c0251fa0 0x55e6c0254430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.285+0000 7f94222ef640 1 -- 172.21.15.107:0/1802849583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c0251fa0 msgr2=0x55e6c0254430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:53.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.285+0000 7f94222ef640 1 --2- 172.21.15.107:0/1802849583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c0251fa0 0x55e6c0254430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.285+0000 7f94222ef640 1 -- 172.21.15.107:0/1802849583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6c024c440 msgr2=0x55e6c024c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:53.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.285+0000 7f94222ef640 1 --2- 172.21.15.107:0/1802849583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6c024c440 0x55e6c024c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.285+0000 7f94222ef640 1 -- 172.21.15.107:0/1802849583 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6bfec42d0 con 0x55e6c024ce30 2024-04-25T00:15:53.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.285+0000 7f94222ef640 1 --2- 172.21.15.107:0/1802849583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c024ce30 0x55e6c0251930 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9418009ca0 tx=0x7f941802f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce5a8fc7fddc66f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:53.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.286+0000 7f9422af0640 1 --2- 172.21.15.107:0/3339183308 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f94080a2b90 0x7f94080a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55e6c024d690 tx=0x7f9414005b30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:53.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.286+0000 7f9410bfe640 1 -- 172.21.15.107:0/1802849583 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9418049070 con 0x55e6c024ce30 2024-04-25T00:15:53.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.286+0000 7f9410bfe640 1 -- 172.21.15.107:0/1802849583 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9418036040 con 0x55e6c024ce30 2024-04-25T00:15:53.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.286+0000 7f9410bfe640 1 -- 172.21.15.107:0/1802849583 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941802fde0 con 0x55e6c024ce30 2024-04-25T00:15:53.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.286+0000 7f942725d740 1 -- 172.21.15.107:0/1802849583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c024ce30 msgr2=0x55e6c0251930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.286+0000 7f942725d740 1 --2- 172.21.15.107:0/1802849583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c024ce30 0x55e6c0251930 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9418009ca0 tx=0x7f941802f310 comp rx=0 tx=0).stop 2024-04-25T00:15:53.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.286+0000 7f942725d740 1 -- 172.21.15.107:0/1802849583 shutdown_connections 2024-04-25T00:15:53.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.286+0000 7f942725d740 1 --2- 172.21.15.107:0/1802849583 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c0251fa0 0x55e6c0254430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.286+0000 7f942725d740 1 --2- 172.21.15.107:0/1802849583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c024ce30 0x55e6c0251930 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.286+0000 7f942725d740 1 --2- 172.21.15.107:0/1802849583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6c024c440 0x55e6c024c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.286+0000 7f942725d740 1 -- 172.21.15.107:0/1802849583 >> 172.21.15.107:0/1802849583 conn(0x55e6c0247ab0 msgr2=0x55e6c0247eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.286+0000 7f942725d740 1 -- 172.21.15.107:0/1802849583 shutdown_connections 2024-04-25T00:15:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.286+0000 7f942725d740 1 -- 172.21.15.107:0/1802849583 wait complete. 2024-04-25T00:15:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.287+0000 7f942725d740 1 -- start start 2024-04-25T00:15:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.287+0000 7f942725d740 1 Processor -- start 2024-04-25T00:15:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.287+0000 7f942725d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c024c440 0x55e6c0256690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.287+0000 7f942725d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6c024ce30 0x55e6c0256bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.287+0000 7f942725d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c0251fa0 0x55e6c025dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.287+0000 7f942725d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e6c0254d70 con 0x55e6c024c440 2024-04-25T00:15:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.287+0000 7f942725d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e6c016bcf0 con 0x55e6c024ce30 2024-04-25T00:15:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.287+0000 7f942725d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e6c0254ef0 con 0x55e6c0251fa0 2024-04-25T00:15:53.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.287+0000 7f942725d740 10 client.0 _ll_register_callbacks cb 0x55e6c016b6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:15:53.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.287+0000 7f94222ef640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c024c440 0x55e6c0256690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.287+0000 7f94222ef640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c024c440 0x55e6c0256690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40416/0 (socket says 172.21.15.107:40416) 2024-04-25T00:15:53.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.287+0000 7f94222ef640 1 -- 172.21.15.107:0/1642997503 learned_addr learned my addr 172.21.15.107:0/1642997503 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:53.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.287+0000 7f9421aee640 1 --2- 172.21.15.107:0/1642997503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6c024ce30 0x55e6c0256bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.287+0000 7f94222ef640 1 -- 172.21.15.107:0/1642997503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c0251fa0 msgr2=0x55e6c025dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:53.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.287+0000 7f94222ef640 1 --2- 172.21.15.107:0/1642997503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c0251fa0 0x55e6c025dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.287+0000 7f94222ef640 1 -- 172.21.15.107:0/1642997503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6c024ce30 msgr2=0x55e6c0256bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:53.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.287+0000 7f94222ef640 1 --2- 172.21.15.107:0/1642997503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6c024ce30 0x55e6c0256bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.287+0000 7f94222ef640 1 -- 172.21.15.107:0/1642997503 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6bff8e6b0 con 0x55e6c024c440 2024-04-25T00:15:53.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.287+0000 7f94222ef640 1 --2- 172.21.15.107:0/1642997503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c024c440 0x55e6c0256690 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9418031ae0 tx=0x7f9418031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17273f1446a012d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:53.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.287+0000 7f9405ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:15:53.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.288+0000 7f9405ffb640 1 -- 172.21.15.107:0/1642997503 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9418054060 con 0x55e6c024c440 2024-04-25T00:15:53.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.288+0000 7f9405ffb640 1 -- 172.21.15.107:0/1642997503 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f9418031660 con 0x55e6c024c440 2024-04-25T00:15:53.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.288+0000 7f942725d740 1 -- 172.21.15.107:0/1642997503 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9418036040 con 0x55e6c024c440 2024-04-25T00:15:53.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.288+0000 7f9405ffb640 1 -- 172.21.15.107:0/1642997503 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f941805b050 con 0x55e6c024c440 2024-04-25T00:15:53.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.288+0000 7f942725d740 10 client.15234 fetch_fsmap learned FSMap version 23 2024-04-25T00:15:53.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.288+0000 7f942725d740 1 -- 172.21.15.107:0/1642997503 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55e6c0254d70 con 0x55e6c024c440 2024-04-25T00:15:53.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.288+0000 7f9405ffb640 1 -- 172.21.15.107:0/1642997503 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9418036040 con 0x55e6c024c440 2024-04-25T00:15:53.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.288+0000 7f9405ffb640 1 -- 172.21.15.107:0/1642997503 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f941803b400 con 0x55e6c024c440 2024-04-25T00:15:53.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.288+0000 7f942725d740 10 client.15234 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:15:53.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.288+0000 7f942725d740 10 client.15234 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T00:15:53.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.288+0000 7f942725d740 20 client.15234 populate_metadata read hostname 'smithi107' 2024-04-25T00:15:53.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.288+0000 7f942725d740 1 --2- 172.21.15.107:0/1642997503 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55e6c02630b0 0x55e6c0264520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.288+0000 7f942725d740 4 client.15234 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:15:53.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.288+0000 7f942725d740 1 -- 172.21.15.107:0/1642997503 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6c01bd440 con 0x55e6c02630b0 2024-04-25T00:15:53.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.288+0000 7f9421aee640 1 --2- 172.21.15.107:0/1642997503 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55e6c02630b0 0x55e6c0264520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.289+0000 7f9405ffb640 10 client.15234 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:15:53.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.289+0000 7f9421aee640 1 --2- 172.21.15.107:0/1642997503 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55e6c02630b0 0x55e6c0264520 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53df4f720435a995 server_cookie=81dc70525734be73 in_seq=0 out_seq=0 2024-04-25T00:15:53.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.290+0000 7f9405ffb640 1 -- 172.21.15.107:0/1642997503 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e6c01bd440 con 0x55e6c02630b0 2024-04-25T00:15:53.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.290+0000 7f9405ffb640 10 client.15234 handle_command_reply: tid=0 2024-04-25T00:15:53.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.290+0000 7f942725d740 1 client.15234 shutdown 2024-04-25T00:15:53.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.290+0000 7f942725d740 1 -- 172.21.15.107:0/1642997503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c024c440 msgr2=0x55e6c0256690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.290+0000 7f942725d740 1 --2- 172.21.15.107:0/1642997503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c024c440 0x55e6c0256690 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9418031ae0 tx=0x7f9418031ee0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.290+0000 7f942725d740 1 -- 172.21.15.107:0/1642997503 shutdown_connections 2024-04-25T00:15:53.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.290+0000 7f942725d740 1 --2- 172.21.15.107:0/1642997503 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c0251fa0 0x55e6c025dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.290+0000 7f942725d740 1 --2- 172.21.15.107:0/1642997503 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6c024ce30 0x55e6c0256bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.290+0000 7f942725d740 1 --2- 172.21.15.107:0/1642997503 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55e6c02630b0 0x55e6c0264520 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.290+0000 7f942725d740 1 --2- 172.21.15.107:0/1642997503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c024c440 0x55e6c0256690 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.290+0000 7f9405ffb640 0 client.15234 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:15:53.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.290+0000 7f942725d740 1 -- 172.21.15.107:0/1642997503 >> 172.21.15.107:0/1642997503 conn(0x55e6c0247ab0 msgr2=0x55e6c0246ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:53.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.290+0000 7f942725d740 1 -- 172.21.15.107:0/1642997503 shutdown_connections 2024-04-25T00:15:53.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.290+0000 7f942725d740 1 -- 172.21.15.107:0/1642997503 wait complete. 2024-04-25T00:15:53.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.290+0000 7f942725d740 20 client.15234 trim_cache size 0 max 16384 2024-04-25T00:15:53.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.308+0000 7f942725d740 1 Processor -- start 2024-04-25T00:15:53.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.308+0000 7f942725d740 1 -- start start 2024-04-25T00:15:53.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.308+0000 7f942725d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c024c440 0x55e6c024e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.308+0000 7f942725d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c024ce30 0x55e6c024e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.308+0000 7f942725d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6c0251fa0 0x55e6c0246250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.308+0000 7f942725d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e6c0262860 con 0x55e6c024c440 2024-04-25T00:15:53.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.308+0000 7f942725d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e6c0262f30 con 0x55e6c0251fa0 2024-04-25T00:15:53.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.308+0000 7f942725d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e6c0263600 con 0x55e6c024ce30 2024-04-25T00:15:53.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.308+0000 7f94222ef640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c024c440 0x55e6c024e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.308+0000 7f94222ef640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c024c440 0x55e6c024e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40420/0 (socket says 172.21.15.107:40420) 2024-04-25T00:15:53.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.308+0000 7f94222ef640 1 -- 172.21.15.107:0/2380999790 learned_addr learned my addr 172.21.15.107:0/2380999790 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:53.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.308+0000 7f94222ef640 1 -- 172.21.15.107:0/2380999790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c024ce30 msgr2=0x55e6c024e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:53.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.308+0000 7f94222ef640 1 --2- 172.21.15.107:0/2380999790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c024ce30 0x55e6c024e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.308+0000 7f94222ef640 1 -- 172.21.15.107:0/2380999790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6c0251fa0 msgr2=0x55e6c0246250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:53.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.308+0000 7f94222ef640 1 --2- 172.21.15.107:0/2380999790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6c0251fa0 0x55e6c0246250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.308+0000 7f94222ef640 1 -- 172.21.15.107:0/2380999790 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6c024d210 con 0x55e6c024c440 2024-04-25T00:15:53.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.308+0000 7f94222ef640 1 --2- 172.21.15.107:0/2380999790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c024c440 0x55e6c024e290 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9418009bf0 tx=0x7f94180098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a9203c6e62f0eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:53.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.308+0000 7f9407fff640 1 -- 172.21.15.107:0/2380999790 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9418054040 con 0x55e6c024c440 2024-04-25T00:15:53.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.308+0000 7f9407fff640 1 -- 172.21.15.107:0/2380999790 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9418036070 con 0x55e6c024c440 2024-04-25T00:15:53.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.308+0000 7f9407fff640 1 -- 172.21.15.107:0/2380999790 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9418049070 con 0x55e6c024c440 2024-04-25T00:15:53.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.309+0000 7f942725d740 1 -- 172.21.15.107:0/2380999790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c024c440 msgr2=0x55e6c024e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.309+0000 7f942725d740 1 --2- 172.21.15.107:0/2380999790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c024c440 0x55e6c024e290 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9418009bf0 tx=0x7f94180098e0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.309+0000 7f942725d740 1 -- 172.21.15.107:0/2380999790 shutdown_connections 2024-04-25T00:15:53.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.309+0000 7f942725d740 1 --2- 172.21.15.107:0/2380999790 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6c0251fa0 0x55e6c0246250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.309+0000 7f942725d740 1 --2- 172.21.15.107:0/2380999790 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c024ce30 0x55e6c024e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.309+0000 7f942725d740 1 --2- 172.21.15.107:0/2380999790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c024c440 0x55e6c024e290 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.309+0000 7f942725d740 1 -- 172.21.15.107:0/2380999790 >> 172.21.15.107:0/2380999790 conn(0x55e6c0247ab0 msgr2=0x55e6c0261d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:53.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.309+0000 7f942725d740 1 -- 172.21.15.107:0/2380999790 shutdown_connections 2024-04-25T00:15:53.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.309+0000 7f942725d740 1 -- 172.21.15.107:0/2380999790 wait complete. 2024-04-25T00:15:53.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.309+0000 7f942725d740 1 -- start start 2024-04-25T00:15:53.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.309+0000 7f942725d740 1 Processor -- start 2024-04-25T00:15:53.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.309+0000 7f942725d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6c024c440 0x55e6c025bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.309+0000 7f942725d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c024ce30 0x55e6c025c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.309+0000 7f942725d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c0251fa0 0x55e6c025c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.309+0000 7f942725d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e6c02643a0 con 0x55e6c0251fa0 2024-04-25T00:15:53.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.309+0000 7f942725d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e6c0263cd0 con 0x55e6c024c440 2024-04-25T00:15:53.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.309+0000 7f942725d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e6c0264a70 con 0x55e6c024ce30 2024-04-25T00:15:53.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.309+0000 7f942725d740 10 client.0 _ll_register_callbacks cb 0x7f941802fe30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:15:53.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.310+0000 7f9422af0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c0251fa0 0x55e6c025c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.310+0000 7f9422af0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c0251fa0 0x55e6c025c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40432/0 (socket says 172.21.15.107:40432) 2024-04-25T00:15:53.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.310+0000 7f9422af0640 1 -- 172.21.15.107:0/4204871369 learned_addr learned my addr 172.21.15.107:0/4204871369 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:53.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.310+0000 7f94222ef640 1 --2- 172.21.15.107:0/4204871369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6c024c440 0x55e6c025bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.310+0000 7f9421aee640 1 --2- 172.21.15.107:0/4204871369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c024ce30 0x55e6c025c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.310+0000 7f9422af0640 1 -- 172.21.15.107:0/4204871369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c024ce30 msgr2=0x55e6c025c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:53.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.310+0000 7f9422af0640 1 --2- 172.21.15.107:0/4204871369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c024ce30 0x55e6c025c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.310+0000 7f9422af0640 1 -- 172.21.15.107:0/4204871369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6c024c440 msgr2=0x55e6c025bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:53.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.310+0000 7f9422af0640 1 --2- 172.21.15.107:0/4204871369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6c024c440 0x55e6c025bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.310+0000 7f9422af0640 1 -- 172.21.15.107:0/4204871369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6c025ce40 con 0x55e6c0251fa0 2024-04-25T00:15:53.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.310+0000 7f94222ef640 1 --2- 172.21.15.107:0/4204871369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6c024c440 0x55e6c025bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:53.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.310+0000 7f9421aee640 1 --2- 172.21.15.107:0/4204871369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c024ce30 0x55e6c025c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:53.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.310+0000 7f9422af0640 1 --2- 172.21.15.107:0/4204871369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c0251fa0 0x55e6c025c640 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f941400fa50 tx=0x7f941400d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5838b953073c6c15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:53.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.310+0000 7f9405ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:15:53.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.310+0000 7f9405ffb640 1 -- 172.21.15.107:0/4204871369 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941401cc80 con 0x55e6c0251fa0 2024-04-25T00:15:53.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.310+0000 7f9405ffb640 1 -- 172.21.15.107:0/4204871369 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f941401e450 con 0x55e6c0251fa0 2024-04-25T00:15:53.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.310+0000 7f942725d740 1 -- 172.21.15.107:0/4204871369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9418036070 con 0x55e6c0251fa0 2024-04-25T00:15:53.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.310+0000 7f9405ffb640 1 -- 172.21.15.107:0/4204871369 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f941401ce20 con 0x55e6c0251fa0 2024-04-25T00:15:53.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.310+0000 7f942725d740 10 client.15240 fetch_fsmap learned FSMap version 23 2024-04-25T00:15:53.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.310+0000 7f942725d740 1 -- 172.21.15.107:0/4204871369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55e6c02cdfe0 con 0x55e6c0251fa0 2024-04-25T00:15:53.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.310+0000 7f9405ffb640 1 -- 172.21.15.107:0/4204871369 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9418036070 con 0x55e6c0251fa0 2024-04-25T00:15:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.311+0000 7f9405ffb640 1 -- 172.21.15.107:0/4204871369 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f941401b040 con 0x55e6c0251fa0 2024-04-25T00:15:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.311+0000 7f942725d740 10 client.15240 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:15:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.311+0000 7f942725d740 10 client.15240 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T00:15:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.311+0000 7f942725d740 20 client.15240 populate_metadata read hostname 'smithi107' 2024-04-25T00:15:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.311+0000 7f942725d740 1 --2- 172.21.15.107:0/4204871369 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55e6c02d2c70 0x55e6c02d1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.311+0000 7f942725d740 4 client.15240 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:15:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.311+0000 7f942725d740 1 -- 172.21.15.107:0/4204871369 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "status"}) -- 0x55e6c01b7790 con 0x55e6c02d2c70 2024-04-25T00:15:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.311+0000 7f94222ef640 1 --2- 172.21.15.107:0/4204871369 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55e6c02d2c70 0x55e6c02d1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.311+0000 7f94222ef640 1 --2- 172.21.15.107:0/4204871369 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55e6c02d2c70 0x55e6c02d1980 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=467108f4b4430307 server_cookie=2e92d73e1a97a4fe in_seq=0 out_seq=0 2024-04-25T00:15:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.311+0000 7f9405ffb640 10 client.15240 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:15:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.311+0000 7f9405ffb640 1 -- 172.21.15.107:0/4204871369 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e6c01b7790 con 0x55e6c02d2c70 2024-04-25T00:15:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.311+0000 7f9405ffb640 10 client.15240 handle_command_reply: tid=0 2024-04-25T00:15:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.311+0000 7f942725d740 1 client.15240 shutdown 2024-04-25T00:15:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.312+0000 7f942725d740 1 -- 172.21.15.107:0/4204871369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c0251fa0 msgr2=0x55e6c025c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.312+0000 7f942725d740 1 --2- 172.21.15.107:0/4204871369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c0251fa0 0x55e6c025c640 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f941400fa50 tx=0x7f941400d940 comp rx=0 tx=0).stop 2024-04-25T00:15:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.312+0000 7f942725d740 1 -- 172.21.15.107:0/4204871369 shutdown_connections 2024-04-25T00:15:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.312+0000 7f942725d740 1 --2- 172.21.15.107:0/4204871369 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55e6c02d2c70 0x55e6c02d1980 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.312+0000 7f942725d740 1 --2- 172.21.15.107:0/4204871369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6c0251fa0 0x55e6c025c640 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.312+0000 7f942725d740 1 --2- 172.21.15.107:0/4204871369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6c024ce30 0x55e6c025c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.312+0000 7f9405ffb640 0 client.15240 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:15:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.312+0000 7f942725d740 1 --2- 172.21.15.107:0/4204871369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6c024c440 0x55e6c025bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.312+0000 7f942725d740 1 -- 172.21.15.107:0/4204871369 >> 172.21.15.107:0/4204871369 conn(0x55e6c0247ab0 msgr2=0x55e6c0261d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.312+0000 7f942725d740 1 -- 172.21.15.107:0/4204871369 shutdown_connections 2024-04-25T00:15:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.312+0000 7f942725d740 1 -- 172.21.15.107:0/4204871369 wait complete. 2024-04-25T00:15:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.312+0000 7f942725d740 20 client.15240 trim_cache size 0 max 16384 2024-04-25T00:15:53.328 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:15:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "79602f58-0297-11ef-bc93-c7b262605968", 2024-04-25T00:15:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 1, 2024-04-25T00:15:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24493, 2024-04-25T00:15:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-04-25T00:15:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-04-25T00:15:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-04-25T00:15:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 86.792306668999998, 2024-04-25T00:15:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 23, 2024-04-25T00:15:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 94, 2024-04-25T00:15:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 92, 2024-04-25T00:15:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 124.389409133 2024-04-25T00:15:53.328 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:15:53.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.312+0000 7f9424553640 1 -- 172.21.15.107:0/3339183308 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f94080a2b90 msgr2=0x7f94080a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.312+0000 7f9424553640 1 --2- 172.21.15.107:0/3339183308 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f94080a2b90 0x7f94080a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55e6c024d690 tx=0x7f9414005b30 comp rx=0 tx=0).stop 2024-04-25T00:15:53.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.312+0000 7f9424553640 1 -- 172.21.15.107:0/3339183308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f941c101870 msgr2=0x7f941c19c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.312+0000 7f9424553640 1 --2- 172.21.15.107:0/3339183308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f941c101870 0x7f941c19c7f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f940c00c920 tx=0x7f940c00cde0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.312+0000 7f9424553640 1 -- 172.21.15.107:0/3339183308 shutdown_connections 2024-04-25T00:15:53.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.312+0000 7f9424553640 1 --2- 172.21.15.107:0/3339183308 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f941c102230 0x7f941c19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.312+0000 7f9424553640 1 --2- 172.21.15.107:0/3339183308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f941c101870 0x7f941c19c7f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.312+0000 7f9424553640 1 --2- 172.21.15.107:0/3339183308 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f94080a2b90 0x7f94080a5050 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.312+0000 7f9424553640 1 --2- 172.21.15.107:0/3339183308 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f941c100680 0x7f941c069440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.312+0000 7f9424553640 1 -- 172.21.15.107:0/3339183308 >> 172.21.15.107:0/3339183308 conn(0x7f941c0fbbf0 msgr2=0x7f941c104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:53.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.313+0000 7f9424553640 1 -- 172.21.15.107:0/3339183308 shutdown_connections 2024-04-25T00:15:53.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.313+0000 7f9424553640 1 -- 172.21.15.107:0/3339183308 wait complete. 2024-04-25T00:15:53.331 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:53 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2250576812' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:15:53.331 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:53 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1830839248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:53.331 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:53 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2220923054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:15:53.331 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:53 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1803030337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:53.332 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T00:15:53.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:53 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2250576812' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:15:53.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:53 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1830839248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:53.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:53 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2220923054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:15:53.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:53 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1803030337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:53.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.432+0000 7f321c390640 1 -- 172.21.15.107:0/750097590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3214102250 msgr2=0x7f3214108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.432+0000 7f321c390640 1 --2- 172.21.15.107:0/750097590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3214102250 0x7f3214108ae0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f320c009ca0 tx=0x7f320c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.432+0000 7f321c390640 1 -- 172.21.15.107:0/750097590 shutdown_connections 2024-04-25T00:15:53.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.432+0000 7f321c390640 1 --2- 172.21.15.107:0/750097590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3214102250 0x7f3214108ae0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.432+0000 7f321c390640 1 --2- 172.21.15.107:0/750097590 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3214101890 0x7f3214101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.432+0000 7f321c390640 1 --2- 172.21.15.107:0/750097590 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f32141006a0 0x7f3214100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.432+0000 7f321c390640 1 -- 172.21.15.107:0/750097590 >> 172.21.15.107:0/750097590 conn(0x7f32140fbc30 msgr2=0x7f32140fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:53.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.432+0000 7f321c390640 1 -- 172.21.15.107:0/750097590 shutdown_connections 2024-04-25T00:15:53.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.432+0000 7f321c390640 1 -- 172.21.15.107:0/750097590 wait complete. 2024-04-25T00:15:53.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.433+0000 7f321c390640 1 Processor -- start 2024-04-25T00:15:53.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.433+0000 7f321c390640 1 -- start start 2024-04-25T00:15:53.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.433+0000 7f321c390640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32141006a0 0x7f3214197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.433+0000 7f321c390640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3214101890 0x7f3214197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.433+0000 7f321c390640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3214102250 0x7f321419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.433+0000 7f321c390640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f321410c320 con 0x7f32141006a0 2024-04-25T00:15:53.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.433+0000 7f321c390640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f321410c1a0 con 0x7f3214101890 2024-04-25T00:15:53.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.433+0000 7f321c390640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f321410c4a0 con 0x7f3214102250 2024-04-25T00:15:53.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.433+0000 7f321a12c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32141006a0 0x7f3214197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.433+0000 7f321a12c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32141006a0 0x7f3214197a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40450/0 (socket says 172.21.15.107:40450) 2024-04-25T00:15:53.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.433+0000 7f321a12c640 1 -- 172.21.15.107:0/1414026264 learned_addr learned my addr 172.21.15.107:0/1414026264 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:53.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.433+0000 7f321a92d640 1 --2- 172.21.15.107:0/1414026264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3214102250 0x7f321419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.433+0000 7f321992b640 1 --2- 172.21.15.107:0/1414026264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3214101890 0x7f3214197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.433+0000 7f321a12c640 1 -- 172.21.15.107:0/1414026264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3214102250 msgr2=0x7f321419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.433+0000 7f321a12c640 1 --2- 172.21.15.107:0/1414026264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3214102250 0x7f321419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.433+0000 7f321a12c640 1 -- 172.21.15.107:0/1414026264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3214101890 msgr2=0x7f3214197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.433+0000 7f321a12c640 1 --2- 172.21.15.107:0/1414026264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3214101890 0x7f3214197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.433+0000 7f321a12c640 1 -- 172.21.15.107:0/1414026264 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321419f680 con 0x7f32141006a0 2024-04-25T00:15:53.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.433+0000 7f321a92d640 1 --2- 172.21.15.107:0/1414026264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3214102250 0x7f321419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:53.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.433+0000 7f321992b640 1 --2- 172.21.15.107:0/1414026264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3214101890 0x7f3214197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:53.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.434+0000 7f321a12c640 1 --2- 172.21.15.107:0/1414026264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32141006a0 0x7f3214197a60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f321000d950 tx=0x7f321000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:53.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.434+0000 7f320b7fe640 1 -- 172.21.15.107:0/1414026264 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3210017070 con 0x7f32141006a0 2024-04-25T00:15:53.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.434+0000 7f321c390640 1 -- 172.21.15.107:0/1414026264 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321419f970 con 0x7f32141006a0 2024-04-25T00:15:53.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.434+0000 7f320b7fe640 1 -- 172.21.15.107:0/1414026264 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3210009c30 con 0x7f32141006a0 2024-04-25T00:15:53.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.434+0000 7f320b7fe640 1 -- 172.21.15.107:0/1414026264 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321000f600 con 0x7f32141006a0 2024-04-25T00:15:53.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.434+0000 7f321c390640 1 -- 172.21.15.107:0/1414026264 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f321419fec0 con 0x7f32141006a0 2024-04-25T00:15:53.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:53 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2250576812' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:15:53.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:53 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1830839248' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:53.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:53 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2220923054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:15:53.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:53 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1803030337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:53.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.436+0000 7f320b7fe640 1 -- 172.21.15.107:0/1414026264 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f321001a070 con 0x7f32141006a0 2024-04-25T00:15:53.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.436+0000 7f321f17e740 1 Processor -- start 2024-04-25T00:15:53.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.437+0000 7f321f17e740 1 -- start start 2024-04-25T00:15:53.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.437+0000 7f321f17e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8a1c4b440 0x55d8a1c4b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.437+0000 7f321f17e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d8a1c4be30 0x55d8a1c50930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.437+0000 7f321f17e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c50fa0 0x55d8a1c53430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.437+0000 7f321f17e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d8a1baef00 con 0x55d8a1c50fa0 2024-04-25T00:15:53.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.437+0000 7f321f17e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d8a1b57750 con 0x55d8a1c4b440 2024-04-25T00:15:53.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.437+0000 7f321f17e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d8a1baaec0 con 0x55d8a1c4be30 2024-04-25T00:15:53.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.437+0000 7f320b7fe640 1 --2- 172.21.15.107:0/1414026264 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f32000a2b90 0x7f32000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.437+0000 7f321992b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c50fa0 0x55d8a1c53430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.437+0000 7f321a92d640 1 --2- 172.21.15.107:0/1414026264 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f32000a2b90 0x7f32000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.437+0000 7f321992b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c50fa0 0x55d8a1c53430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40464/0 (socket says 172.21.15.107:40464) 2024-04-25T00:15:53.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.437+0000 7f321992b640 1 -- 172.21.15.107:0/3466160027 learned_addr learned my addr 172.21.15.107:0/3466160027 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:53.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.437+0000 7f321a92d640 1 --2- 172.21.15.107:0/3466160027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8a1c4b440 0x55d8a1c4b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.437+0000 7f320b7fe640 1 -- 172.21.15.107:0/1414026264 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f32100ceed0 con 0x7f32141006a0 2024-04-25T00:15:53.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.437+0000 7f321a12c640 1 --2- 172.21.15.107:0/3466160027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d8a1c4be30 0x55d8a1c50930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.437+0000 7f321992b640 1 -- 172.21.15.107:0/3466160027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d8a1c4be30 msgr2=0x55d8a1c50930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:53.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.437+0000 7f321992b640 1 --2- 172.21.15.107:0/3466160027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d8a1c4be30 0x55d8a1c50930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.437+0000 7f321992b640 1 -- 172.21.15.107:0/3466160027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8a1c4b440 msgr2=0x55d8a1c4b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:53.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.437+0000 7f321992b640 1 --2- 172.21.15.107:0/3466160027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8a1c4b440 0x55d8a1c4b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.437+0000 7f321992b640 1 -- 172.21.15.107:0/3466160027 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8a18c32d0 con 0x55d8a1c50fa0 2024-04-25T00:15:53.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.437+0000 7f321a92d640 1 --2- 172.21.15.107:0/3466160027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8a1c4b440 0x55d8a1c4b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:53.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.437+0000 7f321a92d640 1 --2- 172.21.15.107:0/1414026264 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f32000a2b90 0x7f32000a5050 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f320c02f540 tx=0x7f320c031bb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:53.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.437+0000 7f321992b640 1 --2- 172.21.15.107:0/3466160027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c50fa0 0x55d8a1c53430 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f320400bd30 tx=0x7f320400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32415cd7f99def8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:53.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.437+0000 7f3208bfe640 1 -- 172.21.15.107:0/3466160027 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204015070 con 0x55d8a1c50fa0 2024-04-25T00:15:53.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.438+0000 7f3208bfe640 1 -- 172.21.15.107:0/3466160027 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f320400be20 con 0x55d8a1c50fa0 2024-04-25T00:15:53.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.438+0000 7f3208bfe640 1 -- 172.21.15.107:0/3466160027 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204011aa0 con 0x55d8a1c50fa0 2024-04-25T00:15:53.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.438+0000 7f321f17e740 1 -- 172.21.15.107:0/3466160027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c50fa0 msgr2=0x55d8a1c53430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.438+0000 7f321f17e740 1 --2- 172.21.15.107:0/3466160027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c50fa0 0x55d8a1c53430 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f320400bd30 tx=0x7f320400f730 comp rx=0 tx=0).stop 2024-04-25T00:15:53.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.438+0000 7f321f17e740 1 -- 172.21.15.107:0/3466160027 shutdown_connections 2024-04-25T00:15:53.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.438+0000 7f321f17e740 1 --2- 172.21.15.107:0/3466160027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c50fa0 0x55d8a1c53430 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.438+0000 7f321f17e740 1 --2- 172.21.15.107:0/3466160027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d8a1c4be30 0x55d8a1c50930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.438+0000 7f321f17e740 1 --2- 172.21.15.107:0/3466160027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8a1c4b440 0x55d8a1c4b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.438+0000 7f321f17e740 1 -- 172.21.15.107:0/3466160027 >> 172.21.15.107:0/3466160027 conn(0x55d8a1c46ab0 msgr2=0x55d8a1c46eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:53.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.438+0000 7f321f17e740 1 -- 172.21.15.107:0/3466160027 shutdown_connections 2024-04-25T00:15:53.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.438+0000 7f321f17e740 1 -- 172.21.15.107:0/3466160027 wait complete. 2024-04-25T00:15:53.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.439+0000 7f321f17e740 1 -- start start 2024-04-25T00:15:53.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.439+0000 7f321f17e740 1 Processor -- start 2024-04-25T00:15:53.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.439+0000 7f321f17e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c4b440 0x55d8a1c557c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.439+0000 7f321f17e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8a1c4be30 0x55d8a1c55d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.439+0000 7f321f17e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d8a1c50fa0 0x55d8a1c5cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.439+0000 7f321f17e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d8a1c53f50 con 0x55d8a1c4b440 2024-04-25T00:15:53.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.439+0000 7f321f17e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d8a1b6acf0 con 0x55d8a1c4be30 2024-04-25T00:15:53.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.439+0000 7f321f17e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d8a1c540d0 con 0x55d8a1c50fa0 2024-04-25T00:15:53.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.439+0000 7f321f17e740 10 client.0 _ll_register_callbacks cb 0x55d8a1b6a6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:15:53.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.439+0000 7f321a12c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c4b440 0x55d8a1c557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.439+0000 7f321a12c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c4b440 0x55d8a1c557c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40472/0 (socket says 172.21.15.107:40472) 2024-04-25T00:15:53.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.439+0000 7f321a12c640 1 -- 172.21.15.107:0/891158829 learned_addr learned my addr 172.21.15.107:0/891158829 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:53.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.439+0000 7f321a12c640 1 -- 172.21.15.107:0/891158829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d8a1c50fa0 msgr2=0x55d8a1c5cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:53.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.439+0000 7f321a12c640 1 --2- 172.21.15.107:0/891158829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d8a1c50fa0 0x55d8a1c5cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.439+0000 7f321a12c640 1 -- 172.21.15.107:0/891158829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8a1c4be30 msgr2=0x55d8a1c55d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:53.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.439+0000 7f321a12c640 1 --2- 172.21.15.107:0/891158829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8a1c4be30 0x55d8a1c55d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.439+0000 7f321a12c640 1 -- 172.21.15.107:0/891158829 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8a198d6b0 con 0x55d8a1c4b440 2024-04-25T00:15:53.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.439+0000 7f321a12c640 1 --2- 172.21.15.107:0/891158829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c4b440 0x55d8a1c557c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3210097dc0 tx=0x7f321009b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91f9bfb3ff4ce74f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:53.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.439+0000 7f31fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:15:53.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.440+0000 7f31fdffb640 1 -- 172.21.15.107:0/891158829 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3210021040 con 0x55d8a1c4b440 2024-04-25T00:15:53.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.440+0000 7f31fdffb640 1 -- 172.21.15.107:0/891158829 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f32100ab260 con 0x55d8a1c4b440 2024-04-25T00:15:53.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.440+0000 7f321f17e740 1 -- 172.21.15.107:0/891158829 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f320400be20 con 0x55d8a1c4b440 2024-04-25T00:15:53.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.440+0000 7f31fdffb640 1 -- 172.21.15.107:0/891158829 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f32100ab460 con 0x55d8a1c4b440 2024-04-25T00:15:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.440+0000 7f321f17e740 10 client.15252 fetch_fsmap learned FSMap version 23 2024-04-25T00:15:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.440+0000 7f321f17e740 1 -- 172.21.15.107:0/891158829 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55d8a1c5fd60 con 0x55d8a1c4b440 2024-04-25T00:15:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.440+0000 7f31fdffb640 1 -- 172.21.15.107:0/891158829 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f320400be20 con 0x55d8a1c4b440 2024-04-25T00:15:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.440+0000 7f31fdffb640 1 -- 172.21.15.107:0/891158829 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f32100b41d0 con 0x55d8a1c4b440 2024-04-25T00:15:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.440+0000 7f321f17e740 10 client.15252 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:15:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.440+0000 7f321f17e740 10 client.15252 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:15:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.440+0000 7f321f17e740 20 client.15252 populate_metadata read hostname 'smithi107' 2024-04-25T00:15:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.440+0000 7f321f17e740 1 --2- 172.21.15.107:0/891158829 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d8a1c629d0 0x55d8a1c63e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.440+0000 7f321f17e740 4 client.15252 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:15:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.440+0000 7f321f17e740 1 -- 172.21.15.107:0/891158829 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8a1bbc440 con 0x55d8a1c629d0 2024-04-25T00:15:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.440+0000 7f321992b640 1 --2- 172.21.15.107:0/891158829 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d8a1c629d0 0x55d8a1c63e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.441+0000 7f31fdffb640 10 client.15252 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:15:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.441+0000 7f321992b640 1 --2- 172.21.15.107:0/891158829 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d8a1c629d0 0x55d8a1c63e40 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bb43c7f9d100f9b server_cookie=1e188a61c1e886da in_seq=0 out_seq=0 2024-04-25T00:15:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.442+0000 7f31fdffb640 1 -- 172.21.15.107:0/891158829 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d8a1bbc440 con 0x55d8a1c629d0 2024-04-25T00:15:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.442+0000 7f31fdffb640 10 client.15252 handle_command_reply: tid=0 2024-04-25T00:15:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.442+0000 7f321f17e740 1 client.15252 shutdown 2024-04-25T00:15:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.442+0000 7f321f17e740 1 -- 172.21.15.107:0/891158829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c4b440 msgr2=0x55d8a1c557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.442+0000 7f321f17e740 1 --2- 172.21.15.107:0/891158829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c4b440 0x55d8a1c557c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3210097dc0 tx=0x7f321009b050 comp rx=0 tx=0).stop 2024-04-25T00:15:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.442+0000 7f321f17e740 1 -- 172.21.15.107:0/891158829 shutdown_connections 2024-04-25T00:15:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.442+0000 7f321f17e740 1 --2- 172.21.15.107:0/891158829 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d8a1c629d0 0x55d8a1c63e40 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.442+0000 7f321f17e740 1 --2- 172.21.15.107:0/891158829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d8a1c50fa0 0x55d8a1c5cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.442+0000 7f321f17e740 1 --2- 172.21.15.107:0/891158829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8a1c4be30 0x55d8a1c55d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.442+0000 7f31fdffb640 0 client.15252 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:15:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.442+0000 7f321f17e740 1 --2- 172.21.15.107:0/891158829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c4b440 0x55d8a1c557c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.442+0000 7f321f17e740 1 -- 172.21.15.107:0/891158829 >> 172.21.15.107:0/891158829 conn(0x55d8a1c46ab0 msgr2=0x55d8a1c45ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.442+0000 7f321f17e740 1 -- 172.21.15.107:0/891158829 shutdown_connections 2024-04-25T00:15:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.442+0000 7f321f17e740 1 -- 172.21.15.107:0/891158829 wait complete. 2024-04-25T00:15:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.442+0000 7f321f17e740 20 client.15252 trim_cache size 0 max 16384 2024-04-25T00:15:53.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.460+0000 7f321f17e740 1 Processor -- start 2024-04-25T00:15:53.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.460+0000 7f321f17e740 1 -- start start 2024-04-25T00:15:53.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.460+0000 7f321f17e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8a1c4b440 0x55d8a1c41690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.460+0000 7f321f17e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c4be30 0x55d8a1c5cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.460+0000 7f321f17e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d8a1c50fa0 0x55d8a1c553d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.460+0000 7f321f17e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d8a1c62180 con 0x55d8a1c4be30 2024-04-25T00:15:53.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.460+0000 7f321f17e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d8a1c62850 con 0x55d8a1c4b440 2024-04-25T00:15:53.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.460+0000 7f321f17e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d8a1c62f20 con 0x55d8a1c50fa0 2024-04-25T00:15:53.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.460+0000 7f321992b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c4be30 0x55d8a1c5cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.460+0000 7f321992b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c4be30 0x55d8a1c5cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40478/0 (socket says 172.21.15.107:40478) 2024-04-25T00:15:53.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.460+0000 7f321992b640 1 -- 172.21.15.107:0/672260357 learned_addr learned my addr 172.21.15.107:0/672260357 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:53.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.460+0000 7f321a92d640 1 --2- 172.21.15.107:0/672260357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d8a1c50fa0 0x55d8a1c553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.460+0000 7f321a12c640 1 --2- 172.21.15.107:0/672260357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8a1c4b440 0x55d8a1c41690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.460+0000 7f321992b640 1 -- 172.21.15.107:0/672260357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d8a1c50fa0 msgr2=0x55d8a1c553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.460+0000 7f321992b640 1 --2- 172.21.15.107:0/672260357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d8a1c50fa0 0x55d8a1c553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.460+0000 7f321992b640 1 -- 172.21.15.107:0/672260357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8a1c4b440 msgr2=0x55d8a1c41690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.460+0000 7f321992b640 1 --2- 172.21.15.107:0/672260357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8a1c4b440 0x55d8a1c41690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.460+0000 7f321992b640 1 -- 172.21.15.107:0/672260357 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8a1c47780 con 0x55d8a1c4be30 2024-04-25T00:15:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.460+0000 7f321a92d640 1 --2- 172.21.15.107:0/672260357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d8a1c50fa0 0x55d8a1c553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.460+0000 7f321a12c640 1 --2- 172.21.15.107:0/672260357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8a1c4b440 0x55d8a1c41690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.460+0000 7f321992b640 1 --2- 172.21.15.107:0/672260357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c4be30 0x55d8a1c5cd00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x55d8a1c47980 tx=0x7f320400b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f2073a11ac204c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.461+0000 7f31fffff640 1 -- 172.21.15.107:0/672260357 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320402c070 con 0x55d8a1c4be30 2024-04-25T00:15:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.461+0000 7f31fffff640 1 -- 172.21.15.107:0/672260357 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32040050c0 con 0x55d8a1c4be30 2024-04-25T00:15:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.461+0000 7f31fffff640 1 -- 172.21.15.107:0/672260357 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3204015040 con 0x55d8a1c4be30 2024-04-25T00:15:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.461+0000 7f321f17e740 1 -- 172.21.15.107:0/672260357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c4be30 msgr2=0x55d8a1c5cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.461+0000 7f321f17e740 1 --2- 172.21.15.107:0/672260357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c4be30 0x55d8a1c5cd00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x55d8a1c47980 tx=0x7f320400b470 comp rx=0 tx=0).stop 2024-04-25T00:15:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.461+0000 7f321f17e740 1 -- 172.21.15.107:0/672260357 shutdown_connections 2024-04-25T00:15:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.461+0000 7f321f17e740 1 --2- 172.21.15.107:0/672260357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d8a1c50fa0 0x55d8a1c553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.461+0000 7f321f17e740 1 --2- 172.21.15.107:0/672260357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c4be30 0x55d8a1c5cd00 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.461+0000 7f321f17e740 1 --2- 172.21.15.107:0/672260357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8a1c4b440 0x55d8a1c41690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.461+0000 7f321f17e740 1 -- 172.21.15.107:0/672260357 >> 172.21.15.107:0/672260357 conn(0x55d8a1c46ab0 msgr2=0x55d8a1c46e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.461+0000 7f321f17e740 1 -- 172.21.15.107:0/672260357 shutdown_connections 2024-04-25T00:15:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.461+0000 7f321f17e740 1 -- 172.21.15.107:0/672260357 wait complete. 2024-04-25T00:15:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.461+0000 7f321f17e740 1 -- start start 2024-04-25T00:15:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.461+0000 7f321f17e740 1 Processor -- start 2024-04-25T00:15:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.462+0000 7f321f17e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d8a1c4b440 0x55d8a1c5ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.462+0000 7f321f17e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8a1c4be30 0x55d8a1cc4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.462+0000 7f321f17e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c50fa0 0x55d8a1cc6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.462+0000 7f321f17e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d8a1c63cc0 con 0x55d8a1c50fa0 2024-04-25T00:15:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.462+0000 7f321f17e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d8a1c635f0 con 0x55d8a1c4be30 2024-04-25T00:15:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.462+0000 7f321f17e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d8a1c64390 con 0x55d8a1c4b440 2024-04-25T00:15:53.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.462+0000 7f321f17e740 10 client.0 _ll_register_callbacks cb 0x7f321009af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:15:53.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.462+0000 7f321a92d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c50fa0 0x55d8a1cc6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.462+0000 7f321a92d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c50fa0 0x55d8a1cc6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40494/0 (socket says 172.21.15.107:40494) 2024-04-25T00:15:53.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.462+0000 7f321a92d640 1 -- 172.21.15.107:0/51550802 learned_addr learned my addr 172.21.15.107:0/51550802 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:53.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.462+0000 7f321a12c640 1 --2- 172.21.15.107:0/51550802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d8a1c4b440 0x55d8a1c5ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.462+0000 7f321992b640 1 --2- 172.21.15.107:0/51550802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8a1c4be30 0x55d8a1cc4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.462+0000 7f321a92d640 1 -- 172.21.15.107:0/51550802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d8a1c4b440 msgr2=0x55d8a1c5ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:53.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.462+0000 7f321a92d640 1 --2- 172.21.15.107:0/51550802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d8a1c4b440 0x55d8a1c5ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.462+0000 7f321a92d640 1 -- 172.21.15.107:0/51550802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8a1c4be30 msgr2=0x55d8a1cc4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:15:53.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.462+0000 7f321a92d640 1 --2- 172.21.15.107:0/51550802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8a1c4be30 0x55d8a1cc4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.462+0000 7f321a92d640 1 -- 172.21.15.107:0/51550802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8a1cc73f0 con 0x55d8a1c50fa0 2024-04-25T00:15:53.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.462+0000 7f321992b640 1 --2- 172.21.15.107:0/51550802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8a1c4be30 0x55d8a1cc4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:53.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.462+0000 7f321a92d640 1 --2- 172.21.15.107:0/51550802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c50fa0 0x55d8a1cc6eb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f320c009ca0 tx=0x7f320c02f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cec2e803812988f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.462+0000 7f31fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:15:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.462+0000 7f31fdffb640 1 -- 172.21.15.107:0/51550802 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c0305c0 con 0x55d8a1c50fa0 2024-04-25T00:15:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.462+0000 7f31fdffb640 1 -- 172.21.15.107:0/51550802 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f320c030760 con 0x55d8a1c50fa0 2024-04-25T00:15:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.462+0000 7f321f17e740 1 -- 172.21.15.107:0/51550802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32040050c0 con 0x55d8a1c50fa0 2024-04-25T00:15:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.462+0000 7f31fdffb640 1 -- 172.21.15.107:0/51550802 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f320c030960 con 0x55d8a1c50fa0 2024-04-25T00:15:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.463+0000 7f321f17e740 10 client.15258 fetch_fsmap learned FSMap version 23 2024-04-25T00:15:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.463+0000 7f321f17e740 1 -- 172.21.15.107:0/51550802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55d8a1cc8560 con 0x55d8a1c50fa0 2024-04-25T00:15:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.463+0000 7f31fdffb640 1 -- 172.21.15.107:0/51550802 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32040050c0 con 0x55d8a1c50fa0 2024-04-25T00:15:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.463+0000 7f31fdffb640 1 -- 172.21.15.107:0/51550802 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f320c049b30 con 0x55d8a1c50fa0 2024-04-25T00:15:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.463+0000 7f321f17e740 10 client.15258 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:15:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.463+0000 7f321f17e740 10 client.15258 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:15:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.463+0000 7f321f17e740 20 client.15258 populate_metadata read hostname 'smithi107' 2024-04-25T00:15:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.463+0000 7f321f17e740 1 --2- 172.21.15.107:0/51550802 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d8a1ccd280 0x55d8a1ccbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.463+0000 7f321f17e740 4 client.15258 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:15:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.463+0000 7f321f17e740 1 -- 172.21.15.107:0/51550802 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "status"}) -- 0x55d8a1bb6790 con 0x55d8a1ccd280 2024-04-25T00:15:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.463+0000 7f321a12c640 1 --2- 172.21.15.107:0/51550802 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d8a1ccd280 0x55d8a1ccbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.463+0000 7f321a12c640 1 --2- 172.21.15.107:0/51550802 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d8a1ccd280 0x55d8a1ccbf90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8a37d7b687c38f7 server_cookie=4a45e991bebc4b57 in_seq=0 out_seq=0 2024-04-25T00:15:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.463+0000 7f31fdffb640 10 client.15258 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:15:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f31fdffb640 1 -- 172.21.15.107:0/51550802 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55d8a1bb6790 con 0x55d8a1ccd280 2024-04-25T00:15:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f31fdffb640 10 client.15258 handle_command_reply: tid=0 2024-04-25T00:15:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f321f17e740 1 client.15258 shutdown 2024-04-25T00:15:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f321f17e740 1 -- 172.21.15.107:0/51550802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c50fa0 msgr2=0x55d8a1cc6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f321f17e740 1 --2- 172.21.15.107:0/51550802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c50fa0 0x55d8a1cc6eb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f320c009ca0 tx=0x7f320c02f310 comp rx=0 tx=0).stop 2024-04-25T00:15:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f321f17e740 1 -- 172.21.15.107:0/51550802 shutdown_connections 2024-04-25T00:15:53.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f321f17e740 1 --2- 172.21.15.107:0/51550802 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d8a1ccd280 0x55d8a1ccbf90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f321f17e740 1 --2- 172.21.15.107:0/51550802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d8a1c50fa0 0x55d8a1cc6eb0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f31fdffb640 0 client.15258 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:15:53.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f321f17e740 1 --2- 172.21.15.107:0/51550802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d8a1c4be30 0x55d8a1cc4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f321f17e740 1 --2- 172.21.15.107:0/51550802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d8a1c4b440 0x55d8a1c5ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f321f17e740 1 -- 172.21.15.107:0/51550802 >> 172.21.15.107:0/51550802 conn(0x55d8a1c46ab0 msgr2=0x55d8a1c41690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:53.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f321f17e740 1 -- 172.21.15.107:0/51550802 shutdown_connections 2024-04-25T00:15:53.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f321f17e740 1 -- 172.21.15.107:0/51550802 wait complete. 2024-04-25T00:15:53.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f321f17e740 20 client.15258 trim_cache size 0 max 16384 2024-04-25T00:15:53.474 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:15:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "79602f58-0297-11ef-bc93-c7b262605968", 2024-04-25T00:15:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 0, 2024-04-25T00:15:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24521, 2024-04-25T00:15:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-04-25T00:15:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-04-25T00:15:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-04-25T00:15:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 110.677521802, 2024-04-25T00:15:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 23, 2024-04-25T00:15:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 94, 2024-04-25T00:15:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 92, 2024-04-25T00:15:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 116.076170006 2024-04-25T00:15:53.475 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:15:53.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f321c390640 1 -- 172.21.15.107:0/1414026264 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f32000a2b90 msgr2=0x7f32000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f321c390640 1 --2- 172.21.15.107:0/1414026264 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f32000a2b90 0x7f32000a5050 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f320c02f540 tx=0x7f320c031bb0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f321c390640 1 -- 172.21.15.107:0/1414026264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32141006a0 msgr2=0x7f3214197a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f321c390640 1 --2- 172.21.15.107:0/1414026264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32141006a0 0x7f3214197a60 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f321000d950 tx=0x7f321000de10 comp rx=0 tx=0).stop 2024-04-25T00:15:53.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f321c390640 1 -- 172.21.15.107:0/1414026264 shutdown_connections 2024-04-25T00:15:53.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f321c390640 1 --2- 172.21.15.107:0/1414026264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3214102250 0x7f321419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f321c390640 1 --2- 172.21.15.107:0/1414026264 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f32000a2b90 0x7f32000a5050 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f321c390640 1 --2- 172.21.15.107:0/1414026264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3214101890 0x7f3214197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f321c390640 1 --2- 172.21.15.107:0/1414026264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32141006a0 0x7f3214197a60 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f321c390640 1 -- 172.21.15.107:0/1414026264 >> 172.21.15.107:0/1414026264 conn(0x7f32140fbc30 msgr2=0x7f32140fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:53.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.464+0000 7f321c390640 1 -- 172.21.15.107:0/1414026264 shutdown_connections 2024-04-25T00:15:53.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:53.465+0000 7f321c390640 1 -- 172.21.15.107:0/1414026264 wait complete. 2024-04-25T00:15:53.478 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T00:15:53.478 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T00:15:53.490 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T00:15:53.490 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T00:15:53.726 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:15:54.184 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:54 smithi107 ceph-mon[29153]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:15:54.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:54 smithi195 ceph-mon[32833]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:15:54.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:54 smithi170 ceph-mon[32807]: pgmap v350: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:15:55.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.495+0000 7f9d9cb07700 1 -- 172.21.15.107:0/3574541421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d98101470 msgr2=0x7f9d981018d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:55.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.495+0000 7f9d9cb07700 1 --2- 172.21.15.107:0/3574541421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d98101470 0x7f9d981018d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009990 tx=0x7f9d8c009ca0 comp rx=0 tx=0).stop 2024-04-25T00:15:55.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.495+0000 7f9d9cb07700 1 -- 172.21.15.107:0/3574541421 shutdown_connections 2024-04-25T00:15:55.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.495+0000 7f9d9cb07700 1 --2- 172.21.15.107:0/3574541421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d98101e10 0x7f9d981096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:55.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.495+0000 7f9d9cb07700 1 --2- 172.21.15.107:0/3574541421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d981027c0 0x7f9d98102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:55.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.495+0000 7f9d9cb07700 1 --2- 172.21.15.107:0/3574541421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d98101470 0x7f9d981018d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:55.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.495+0000 7f9d9cb07700 1 -- 172.21.15.107:0/3574541421 >> 172.21.15.107:0/3574541421 conn(0x7f9d980fbf00 msgr2=0x7f9d980fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:55.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.495+0000 7f9d9cb07700 1 -- 172.21.15.107:0/3574541421 shutdown_connections 2024-04-25T00:15:55.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.495+0000 7f9d9cb07700 1 -- 172.21.15.107:0/3574541421 wait complete. 2024-04-25T00:15:55.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.496+0000 7f9d9cb07700 1 Processor -- start 2024-04-25T00:15:55.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.496+0000 7f9d9cb07700 1 -- start start 2024-04-25T00:15:55.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.496+0000 7f9d9cb07700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d98101470 0x7f9d98105210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:55.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.496+0000 7f9d9cb07700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d98101e10 0x7f9d98103860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:55.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.496+0000 7f9d9cb07700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d981027c0 0x7f9d98103da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:55.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.496+0000 7f9d9cb07700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9810cdd0 con 0x7f9d981027c0 2024-04-25T00:15:55.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.496+0000 7f9d9cb07700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9810cc50 con 0x7f9d98101470 2024-04-25T00:15:55.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.496+0000 7f9d9cb07700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9810cf50 con 0x7f9d98101e10 2024-04-25T00:15:55.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.496+0000 7f9d96d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d981027c0 0x7f9d98103da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:55.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.496+0000 7f9d96d9d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d981027c0 0x7f9d98103da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40512/0 (socket says 172.21.15.107:40512) 2024-04-25T00:15:55.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.496+0000 7f9d96d9d700 1 -- 172.21.15.107:0/3870921670 learned_addr learned my addr 172.21.15.107:0/3870921670 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:55.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.496+0000 7f9d9659c700 1 --2- 172.21.15.107:0/3870921670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d98101470 0x7f9d98105210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:55.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.496+0000 7f9d95d9b700 1 --2- 172.21.15.107:0/3870921670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d98101e10 0x7f9d98103860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:55.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.496+0000 7f9d96d9d700 1 -- 172.21.15.107:0/3870921670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d98101e10 msgr2=0x7f9d98103860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:55.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.496+0000 7f9d96d9d700 1 --2- 172.21.15.107:0/3870921670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d98101e10 0x7f9d98103860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:55.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.496+0000 7f9d96d9d700 1 -- 172.21.15.107:0/3870921670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d98101470 msgr2=0x7f9d98105210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:55.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.496+0000 7f9d96d9d700 1 --2- 172.21.15.107:0/3870921670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d98101470 0x7f9d98105210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:55.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.496+0000 7f9d96d9d700 1 -- 172.21.15.107:0/3870921670 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d98104650 con 0x7f9d981027c0 2024-04-25T00:15:55.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.496+0000 7f9d96d9d700 1 --2- 172.21.15.107:0/3870921670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d981027c0 0x7f9d98103da0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9d90007ee0 tx=0x7f9d9000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:55.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.497+0000 7f9d877fe700 1 -- 172.21.15.107:0/3870921670 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9000dbd0 con 0x7f9d981027c0 2024-04-25T00:15:55.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.497+0000 7f9d877fe700 1 -- 172.21.15.107:0/3870921670 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d9000dd70 con 0x7f9d981027c0 2024-04-25T00:15:55.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.497+0000 7f9d9cb07700 1 -- 172.21.15.107:0/3870921670 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d981a2610 con 0x7f9d981027c0 2024-04-25T00:15:55.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.497+0000 7f9d877fe700 1 -- 172.21.15.107:0/3870921670 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d90005a60 con 0x7f9d981027c0 2024-04-25T00:15:55.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.497+0000 7f9d9cb07700 1 -- 172.21.15.107:0/3870921670 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d981a2b20 con 0x7f9d981027c0 2024-04-25T00:15:55.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.498+0000 7f9d9cb07700 1 -- 172.21.15.107:0/3870921670 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d98068f20 con 0x7f9d981027c0 2024-04-25T00:15:55.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.500+0000 7f9d877fe700 1 -- 172.21.15.107:0/3870921670 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f9d9001a020 con 0x7f9d981027c0 2024-04-25T00:15:55.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.500+0000 7f9d877fe700 1 --2- 172.21.15.107:0/3870921670 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f9d800a2c20 0x7f9d800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:55.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.500+0000 7f9d9659c700 1 --2- 172.21.15.107:0/3870921670 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f9d800a2c20 0x7f9d800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:55.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.500+0000 7f9d877fe700 1 -- 172.21.15.107:0/3870921670 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f9d900cf0d0 con 0x7f9d981027c0 2024-04-25T00:15:55.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.501+0000 7f9d9659c700 1 --2- 172.21.15.107:0/3870921670 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f9d800a2c20 0x7f9d800a50e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9d9806a740 tx=0x7f9d88007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:55.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.504+0000 7f9d877fe700 1 -- 172.21.15.107:0/3870921670 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9d90010070 con 0x7f9d981027c0 2024-04-25T00:15:55.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.749+0000 7f9d9cb07700 1 -- 172.21.15.107:0/3870921670 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f9d980fe060 con 0x7f9d800a2c20 2024-04-25T00:15:55.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.750+0000 7f9d877fe700 1 -- 172.21.15.107:0/3870921670 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f9d980fe060 con 0x7f9d800a2c20 2024-04-25T00:15:55.751 INFO:teuthology.orchestra.run.smithi107.stdout:Backend: cephadm 2024-04-25T00:15:55.751 INFO:teuthology.orchestra.run.smithi107.stdout:Available: Yes 2024-04-25T00:15:55.751 INFO:teuthology.orchestra.run.smithi107.stdout:Paused: No 2024-04-25T00:15:55.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.752+0000 7f9d9cb07700 1 -- 172.21.15.107:0/3870921670 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f9d800a2c20 msgr2=0x7f9d800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:55.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.752+0000 7f9d9cb07700 1 --2- 172.21.15.107:0/3870921670 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f9d800a2c20 0x7f9d800a50e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9d9806a740 tx=0x7f9d88007400 comp rx=0 tx=0).stop 2024-04-25T00:15:55.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.753+0000 7f9d9cb07700 1 -- 172.21.15.107:0/3870921670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d981027c0 msgr2=0x7f9d98103da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:55.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.753+0000 7f9d9cb07700 1 --2- 172.21.15.107:0/3870921670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d981027c0 0x7f9d98103da0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9d90007ee0 tx=0x7f9d9000d3b0 comp rx=0 tx=0).stop 2024-04-25T00:15:55.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.753+0000 7f9d9cb07700 1 -- 172.21.15.107:0/3870921670 shutdown_connections 2024-04-25T00:15:55.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.753+0000 7f9d9cb07700 1 --2- 172.21.15.107:0/3870921670 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f9d800a2c20 0x7f9d800a50e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:55.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.753+0000 7f9d9cb07700 1 --2- 172.21.15.107:0/3870921670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d981027c0 0x7f9d98103da0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:55.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.753+0000 7f9d9cb07700 1 --2- 172.21.15.107:0/3870921670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d98101470 0x7f9d98105210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:55.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.753+0000 7f9d9cb07700 1 --2- 172.21.15.107:0/3870921670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d98101e10 0x7f9d98103860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:55.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.753+0000 7f9d9cb07700 1 -- 172.21.15.107:0/3870921670 >> 172.21.15.107:0/3870921670 conn(0x7f9d980fbf00 msgr2=0x7f9d980fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:55.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.753+0000 7f9d9cb07700 1 -- 172.21.15.107:0/3870921670 shutdown_connections 2024-04-25T00:15:55.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:55.753+0000 7f9d9cb07700 1 -- 172.21.15.107:0/3870921670 wait complete. 2024-04-25T00:15:56.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:56 smithi195 ceph-mon[32833]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T00:15:56.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:56 smithi170 ceph-mon[32807]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T00:15:56.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:56 smithi107 ceph-mon[29153]: pgmap v351: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T00:15:56.608 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T00:15:56.849 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:15:57.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:57 smithi107 ceph-mon[29153]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:15:57.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:57 smithi195 ceph-mon[32833]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:15:57.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:57 smithi170 ceph-mon[32807]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:15:58.284 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:15:58 smithi107 ceph-mon[29153]: pgmap v352: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:15:58.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:15:58 smithi195 ceph-mon[32833]: pgmap v352: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:15:58.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:15:58 smithi170 ceph-mon[32807]: pgmap v352: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:15:58.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.671+0000 7f436e2ea700 1 -- 172.21.15.107:0/4117037850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4368101180 msgr2=0x7f43681015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:58.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.671+0000 7f436e2ea700 1 --2- 172.21.15.107:0/4117037850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4368101180 0x7f43681015a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f435c0099c0 tx=0x7f435c009cd0 comp rx=0 tx=0).stop 2024-04-25T00:15:58.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.671+0000 7f436e2ea700 1 -- 172.21.15.107:0/4117037850 shutdown_connections 2024-04-25T00:15:58.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.671+0000 7f436e2ea700 1 --2- 172.21.15.107:0/4117037850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4368102d20 0x7f43681095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:58.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.671+0000 7f436e2ea700 1 --2- 172.21.15.107:0/4117037850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4368101180 0x7f43681015a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:58.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.671+0000 7f436e2ea700 1 --2- 172.21.15.107:0/4117037850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4368102380 0x7f43681027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:58.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.671+0000 7f436e2ea700 1 -- 172.21.15.107:0/4117037850 >> 172.21.15.107:0/4117037850 conn(0x7f43680fc720 msgr2=0x7f43680feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:58.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.671+0000 7f436e2ea700 1 -- 172.21.15.107:0/4117037850 shutdown_connections 2024-04-25T00:15:58.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.671+0000 7f436e2ea700 1 -- 172.21.15.107:0/4117037850 wait complete. 2024-04-25T00:15:58.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.672+0000 7f436e2ea700 1 Processor -- start 2024-04-25T00:15:58.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.672+0000 7f436e2ea700 1 -- start start 2024-04-25T00:15:58.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.673+0000 7f436e2ea700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4368101180 0x7f4368198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:58.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.673+0000 7f436e2ea700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4368102380 0x7f4368198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:58.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.673+0000 7f436e2ea700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4368102d20 0x7f436819fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:58.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.673+0000 7f436e2ea700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f436810cd40 con 0x7f4368101180 2024-04-25T00:15:58.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.673+0000 7f436e2ea700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f436810cbc0 con 0x7f4368102380 2024-04-25T00:15:58.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.673+0000 7f436e2ea700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f436810cec0 con 0x7f4368102d20 2024-04-25T00:15:58.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.673+0000 7f4367fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4368101180 0x7f4368198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:58.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.673+0000 7f4367fff700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4368101180 0x7f4368198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40536/0 (socket says 172.21.15.107:40536) 2024-04-25T00:15:58.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.673+0000 7f4367fff700 1 -- 172.21.15.107:0/2252091291 learned_addr learned my addr 172.21.15.107:0/2252091291 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:15:58.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.673+0000 7f43677fe700 1 --2- 172.21.15.107:0/2252091291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4368102380 0x7f4368198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:58.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.673+0000 7f436c887700 1 --2- 172.21.15.107:0/2252091291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4368102d20 0x7f436819fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:58.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.673+0000 7f4367fff700 1 -- 172.21.15.107:0/2252091291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4368102d20 msgr2=0x7f436819fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:58.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.673+0000 7f4367fff700 1 --2- 172.21.15.107:0/2252091291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4368102d20 0x7f436819fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:58.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.673+0000 7f4367fff700 1 -- 172.21.15.107:0/2252091291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4368102380 msgr2=0x7f4368198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:58.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.673+0000 7f4367fff700 1 --2- 172.21.15.107:0/2252091291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4368102380 0x7f4368198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:58.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.673+0000 7f4367fff700 1 -- 172.21.15.107:0/2252091291 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43681a00b0 con 0x7f4368101180 2024-04-25T00:15:58.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.673+0000 7f436c887700 1 --2- 172.21.15.107:0/2252091291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4368102d20 0x7f436819fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:58.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.674+0000 7f4367fff700 1 --2- 172.21.15.107:0/2252091291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4368101180 0x7f4368198450 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f435c009510 tx=0x7f435c009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:58.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.674+0000 7f43657fa700 1 -- 172.21.15.107:0/2252091291 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c026070 con 0x7f4368101180 2024-04-25T00:15:58.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.674+0000 7f436e2ea700 1 -- 172.21.15.107:0/2252091291 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43681a0340 con 0x7f4368101180 2024-04-25T00:15:58.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.674+0000 7f43657fa700 1 -- 172.21.15.107:0/2252091291 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f435c005940 con 0x7f4368101180 2024-04-25T00:15:58.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.674+0000 7f43657fa700 1 -- 172.21.15.107:0/2252091291 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c00fdc0 con 0x7f4368101180 2024-04-25T00:15:58.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.674+0000 7f436e2ea700 1 -- 172.21.15.107:0/2252091291 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43681a0820 con 0x7f4368101180 2024-04-25T00:15:58.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.675+0000 7f436e2ea700 1 -- 172.21.15.107:0/2252091291 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4368068f20 con 0x7f4368101180 2024-04-25T00:15:58.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.676+0000 7f43657fa700 1 -- 172.21.15.107:0/2252091291 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f435c029070 con 0x7f4368101180 2024-04-25T00:15:58.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.676+0000 7f43657fa700 1 --2- 172.21.15.107:0/2252091291 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f43500a2c20 0x7f43500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:58.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.676+0000 7f43677fe700 1 --2- 172.21.15.107:0/2252091291 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f43500a2c20 0x7f43500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:58.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.676+0000 7f43657fa700 1 -- 172.21.15.107:0/2252091291 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f435c0cffe0 con 0x7f4368101180 2024-04-25T00:15:58.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.677+0000 7f43677fe700 1 --2- 172.21.15.107:0/2252091291 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f43500a2c20 0x7f43500a50e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f435800ac50 tx=0x7f435800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:58.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.680+0000 7f43657fa700 1 -- 172.21.15.107:0/2252091291 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f435c030080 con 0x7f4368101180 2024-04-25T00:15:58.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.890+0000 7f436e2ea700 1 -- 172.21.15.107:0/2252091291 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f43680fe890 con 0x7f43500a2c20 2024-04-25T00:15:58.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.904+0000 7f43657fa700 1 -- 172.21.15.107:0/2252091291 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7f43680fe890 con 0x7f43500a2c20 2024-04-25T00:15:58.905 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T00:15:58.905 INFO:teuthology.orchestra.run.smithi107.stdout:mds.a smithi107 running (2m) 19s ago 2m 13.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 956c23a13e3b 2024-04-25T00:15:58.906 INFO:teuthology.orchestra.run.smithi107.stdout:mds.b smithi170 running (2m) 55s ago 2m 25.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef d651c23f6a0c 2024-04-25T00:15:58.906 INFO:teuthology.orchestra.run.smithi107.stdout:mds.c smithi195 running (2m) 55s ago 2m 17.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 2eabc1c06945 2024-04-25T00:15:58.906 INFO:teuthology.orchestra.run.smithi107.stdout:mds.d smithi107 running (2m) 19s ago 2m 15.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef df3dd6c9d0de 2024-04-25T00:15:58.906 INFO:teuthology.orchestra.run.smithi107.stdout:mds.e smithi170 running (2m) 55s ago 2m 16.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 2bd248e19770 2024-04-25T00:15:58.906 INFO:teuthology.orchestra.run.smithi107.stdout:mds.f smithi195 running (2m) 55s ago 2m 16.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef ad2dd935d23d 2024-04-25T00:15:58.906 INFO:teuthology.orchestra.run.smithi107.stdout:mds.g smithi107 running (2m) 19s ago 2m 13.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef 631b408df0e5 2024-04-25T00:15:58.906 INFO:teuthology.orchestra.run.smithi107.stdout:mds.h smithi170 running (2m) 55s ago 2m 16.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef bf9945b981a9 2024-04-25T00:15:58.906 INFO:teuthology.orchestra.run.smithi107.stdout:mds.i smithi195 running (2m) 55s ago 2m 19.6M - 19.0.0-2445-gd2c45d97 f68e37b807ef d7c9c88192ac 2024-04-25T00:15:58.907 INFO:teuthology.orchestra.run.smithi107.stdout:mds.j smithi107 running (2m) 19s ago 2m 16.2M - 19.0.0-2445-gd2c45d97 f68e37b807ef 260019258da8 2024-04-25T00:15:58.907 INFO:teuthology.orchestra.run.smithi107.stdout:mds.k smithi170 running (2m) 55s ago 2m 15.2M - 19.0.0-2445-gd2c45d97 f68e37b807ef 94fc407f5344 2024-04-25T00:15:58.907 INFO:teuthology.orchestra.run.smithi107.stdout:mds.l smithi195 running (2m) 55s ago 2m 13.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 261749b799af 2024-04-25T00:15:58.907 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.x smithi107 *:9283,8765 running (9m) 19s ago 9m 515M - 19.0.0-2445-gd2c45d97 f68e37b807ef 081648f9ff72 2024-04-25T00:15:58.907 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.y smithi170 *:8443,8765 running (7m) 55s ago 7m 452M - 19.0.0-2445-gd2c45d97 f68e37b807ef 655ed70eabc8 2024-04-25T00:15:58.907 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.z smithi195 *:8443,8765 running (7m) 55s ago 7m 453M - 19.0.0-2445-gd2c45d97 f68e37b807ef 52ec1f7c266a 2024-04-25T00:15:58.907 INFO:teuthology.orchestra.run.smithi107.stdout:mon.a smithi107 running (9m) 19s ago 10m 50.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 8662034e7721 2024-04-25T00:15:58.908 INFO:teuthology.orchestra.run.smithi107.stdout:mon.b smithi170 running (8m) 55s ago 8m 39.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 21037d335149 2024-04-25T00:15:58.908 INFO:teuthology.orchestra.run.smithi107.stdout:mon.c smithi195 running (8m) 55s ago 8m 41.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef a6bea1777566 2024-04-25T00:15:58.908 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (7m) 19s ago 7m 53.8M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 0196ae328dc9 2024-04-25T00:15:58.908 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi170 running (7m) 55s ago 7m 47.2M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 49eca87544e9 2024-04-25T00:15:58.908 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi195 running (6m) 55s ago 6m 49.2M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 4b76fac40e5e 2024-04-25T00:15:58.908 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi107 running (6m) 19s ago 6m 52.5M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 0029e29e7d9f 2024-04-25T00:15:58.909 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi170 running (5m) 55s ago 5m 50.8M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 89fdbcd6a628 2024-04-25T00:15:58.909 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi195 running (5m) 55s ago 5m 47.3M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef b66de97a4bce 2024-04-25T00:15:58.909 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6 smithi107 running (5m) 19s ago 5m 50.2M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef d0939b4021da 2024-04-25T00:15:58.909 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7 smithi170 running (4m) 55s ago 4m 43.5M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef ddb46930f962 2024-04-25T00:15:58.909 INFO:teuthology.orchestra.run.smithi107.stdout:osd.8 smithi195 running (4m) 55s ago 4m 43.9M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 65e510050ab0 2024-04-25T00:15:58.909 INFO:teuthology.orchestra.run.smithi107.stdout:osd.9 smithi107 running (3m) 19s ago 3m 50.0M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef bf0a854faba3 2024-04-25T00:15:58.910 INFO:teuthology.orchestra.run.smithi107.stdout:osd.10 smithi170 running (3m) 55s ago 3m 50.0M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 41f0ab43c0d1 2024-04-25T00:15:58.910 INFO:teuthology.orchestra.run.smithi107.stdout:osd.11 smithi195 running (2m) 55s ago 2m 44.1M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 72d775667ced 2024-04-25T00:15:58.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.906+0000 7f436e2ea700 1 -- 172.21.15.107:0/2252091291 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f43500a2c20 msgr2=0x7f43500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:58.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.906+0000 7f436e2ea700 1 --2- 172.21.15.107:0/2252091291 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f43500a2c20 0x7f43500a50e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f435800ac50 tx=0x7f435800a380 comp rx=0 tx=0).stop 2024-04-25T00:15:58.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.906+0000 7f436e2ea700 1 -- 172.21.15.107:0/2252091291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4368101180 msgr2=0x7f4368198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:58.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.906+0000 7f436e2ea700 1 --2- 172.21.15.107:0/2252091291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4368101180 0x7f4368198450 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f435c009510 tx=0x7f435c009540 comp rx=0 tx=0).stop 2024-04-25T00:15:58.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.906+0000 7f436e2ea700 1 -- 172.21.15.107:0/2252091291 shutdown_connections 2024-04-25T00:15:58.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.906+0000 7f436e2ea700 1 --2- 172.21.15.107:0/2252091291 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f43500a2c20 0x7f43500a50e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:58.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.906+0000 7f436e2ea700 1 --2- 172.21.15.107:0/2252091291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4368102d20 0x7f436819fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:58.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.906+0000 7f436e2ea700 1 --2- 172.21.15.107:0/2252091291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4368101180 0x7f4368198450 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:58.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.906+0000 7f436e2ea700 1 --2- 172.21.15.107:0/2252091291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4368102380 0x7f4368198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:58.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.906+0000 7f436e2ea700 1 -- 172.21.15.107:0/2252091291 >> 172.21.15.107:0/2252091291 conn(0x7f43680fc720 msgr2=0x7f43680fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:58.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.906+0000 7f436e2ea700 1 -- 172.21.15.107:0/2252091291 shutdown_connections 2024-04-25T00:15:58.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:15:58.906+0000 7f436e2ea700 1 -- 172.21.15.107:0/2252091291 wait complete. 2024-04-25T00:15:59.597 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T00:15:59.835 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:16:00.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:00 smithi107 ceph-mon[29153]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:00.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:00 smithi107 ceph-mon[29153]: pgmap v353: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:16:00.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:00 smithi195 ceph-mon[32833]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:00.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:00 smithi195 ceph-mon[32833]: pgmap v353: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:16:00.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:00 smithi170 ceph-mon[32807]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:00.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:00 smithi170 ceph-mon[32807]: pgmap v353: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:16:01.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.675+0000 7fbb09b61700 1 -- 172.21.15.107:0/93027525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb04101190 msgr2=0x7fbb041015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:01.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.675+0000 7fbb09b61700 1 --2- 172.21.15.107:0/93027525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb04101190 0x7fbb041015b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40099c0 tx=0x7fbaf4009cd0 comp rx=0 tx=0).stop 2024-04-25T00:16:01.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.675+0000 7fbb09b61700 1 -- 172.21.15.107:0/93027525 shutdown_connections 2024-04-25T00:16:01.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.675+0000 7fbb09b61700 1 --2- 172.21.15.107:0/93027525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb04102d30 0x7fbb041095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:01.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.675+0000 7fbb09b61700 1 --2- 172.21.15.107:0/93027525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb04101190 0x7fbb041015b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:01.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.675+0000 7fbb09b61700 1 --2- 172.21.15.107:0/93027525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb04102390 0x7fbb041027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:01.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.675+0000 7fbb09b61700 1 -- 172.21.15.107:0/93027525 >> 172.21.15.107:0/93027525 conn(0x7fbb040fc730 msgr2=0x7fbb040feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:01.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.675+0000 7fbb09b61700 1 -- 172.21.15.107:0/93027525 shutdown_connections 2024-04-25T00:16:01.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.675+0000 7fbb09b61700 1 -- 172.21.15.107:0/93027525 wait complete. 2024-04-25T00:16:01.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.675+0000 7fbb09b61700 1 Processor -- start 2024-04-25T00:16:01.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.675+0000 7fbb09b61700 1 -- start start 2024-04-25T00:16:01.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.676+0000 7fbb09b61700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb04101190 0x7fbb0406b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:01.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.676+0000 7fbb09b61700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb04102390 0x7fbb0419d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:01.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.676+0000 7fbb09b61700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb04102d30 0x7fbb0419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:01.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.676+0000 7fbb09b61700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0410cd70 con 0x7fbb04102390 2024-04-25T00:16:01.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.676+0000 7fbb09b61700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0410cbf0 con 0x7fbb04102d30 2024-04-25T00:16:01.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.676+0000 7fbb09b61700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0410cef0 con 0x7fbb04101190 2024-04-25T00:16:01.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.676+0000 7fbafaffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb04102390 0x7fbb0419d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:01.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.676+0000 7fbafaffd700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb04102390 0x7fbb0419d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36758/0 (socket says 172.21.15.107:36758) 2024-04-25T00:16:01.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.676+0000 7fbafaffd700 1 -- 172.21.15.107:0/381741689 learned_addr learned my addr 172.21.15.107:0/381741689 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:01.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.676+0000 7fbb03fff700 1 --2- 172.21.15.107:0/381741689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb04102d30 0x7fbb0419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:01.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.676+0000 7fbb037fe700 1 --2- 172.21.15.107:0/381741689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb04101190 0x7fbb0406b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:01.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.676+0000 7fbafaffd700 1 -- 172.21.15.107:0/381741689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb04101190 msgr2=0x7fbb0406b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:01.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.676+0000 7fbafaffd700 1 --2- 172.21.15.107:0/381741689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb04101190 0x7fbb0406b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:01.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.676+0000 7fbafaffd700 1 -- 172.21.15.107:0/381741689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb04102d30 msgr2=0x7fbb0419f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:01.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.676+0000 7fbafaffd700 1 --2- 172.21.15.107:0/381741689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb04102d30 0x7fbb0419f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:01.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.676+0000 7fbafaffd700 1 -- 172.21.15.107:0/381741689 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb041a0110 con 0x7fbb04102390 2024-04-25T00:16:01.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.676+0000 7fbafaffd700 1 --2- 172.21.15.107:0/381741689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb04102390 0x7fbb0419d1b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00d980 tx=0x7fbaec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:01.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.677+0000 7fbb017fa700 1 -- 172.21.15.107:0/381741689 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec017070 con 0x7fbb04102390 2024-04-25T00:16:01.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.677+0000 7fbb017fa700 1 -- 172.21.15.107:0/381741689 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaec005020 con 0x7fbb04102390 2024-04-25T00:16:01.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.677+0000 7fbb017fa700 1 -- 172.21.15.107:0/381741689 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaec00fbf0 con 0x7fbb04102390 2024-04-25T00:16:01.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.677+0000 7fbb09b61700 1 -- 172.21.15.107:0/381741689 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb041a0400 con 0x7fbb04102390 2024-04-25T00:16:01.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.677+0000 7fbb09b61700 1 -- 172.21.15.107:0/381741689 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb041a08f0 con 0x7fbb04102390 2024-04-25T00:16:01.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.678+0000 7fbb09b61700 1 -- 172.21.15.107:0/381741689 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb04068f20 con 0x7fbb04102390 2024-04-25T00:16:01.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.679+0000 7fbb017fa700 1 -- 172.21.15.107:0/381741689 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fbaec0051d0 con 0x7fbb04102390 2024-04-25T00:16:01.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.679+0000 7fbb017fa700 1 --2- 172.21.15.107:0/381741689 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fbaf00a2ca0 0x7fbaf00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:01.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.680+0000 7fbb017fa700 1 -- 172.21.15.107:0/381741689 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fbaec0cf520 con 0x7fbb04102390 2024-04-25T00:16:01.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.680+0000 7fbb037fe700 1 --2- 172.21.15.107:0/381741689 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fbaf00a2ca0 0x7fbaf00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:01.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.680+0000 7fbb037fe700 1 --2- 172.21.15.107:0/381741689 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fbaf00a2ca0 0x7fbaf00a5160 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400b580 tx=0x7fbaf4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:01.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.683+0000 7fbb017fa700 1 -- 172.21.15.107:0/381741689 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbaec020020 con 0x7fbb04102390 2024-04-25T00:16:01.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.898+0000 7fbb09b61700 1 -- 172.21.15.107:0/381741689 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fbb04062670 con 0x7fbaf00a2ca0 2024-04-25T00:16:01.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.904+0000 7fbb017fa700 1 -- 172.21.15.107:0/381741689 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fbb04062670 con 0x7fbaf00a2ca0 2024-04-25T00:16:01.906 INFO:teuthology.orchestra.run.smithi107.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T00:16:01.906 INFO:teuthology.orchestra.run.smithi107.stdout:mds.all 12/12 58s ago 2m smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi170=b;smithi170=e;smithi170=h;smithi170=k;smithi195=c;smithi195=f;smithi195=i;smithi195=l;count:12 2024-04-25T00:16:01.906 INFO:teuthology.orchestra.run.smithi107.stdout:mgr 3/3 58s ago 7m smithi107=x;smithi170=y;smithi195=z;count:3 2024-04-25T00:16:01.906 INFO:teuthology.orchestra.run.smithi107.stdout:mon 3/3 58s ago 8m smithi107:172.21.15.107=a;smithi170:172.21.15.170=b;smithi195:172.21.15.195=c;count:3 2024-04-25T00:16:01.906 INFO:teuthology.orchestra.run.smithi107.stdout:osd 12 58s ago - 2024-04-25T00:16:01.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.906+0000 7fbb09b61700 1 -- 172.21.15.107:0/381741689 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fbaf00a2ca0 msgr2=0x7fbaf00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:01.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.906+0000 7fbb09b61700 1 --2- 172.21.15.107:0/381741689 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fbaf00a2ca0 0x7fbaf00a5160 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400b580 tx=0x7fbaf4005fb0 comp rx=0 tx=0).stop 2024-04-25T00:16:01.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.906+0000 7fbb09b61700 1 -- 172.21.15.107:0/381741689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb04102390 msgr2=0x7fbb0419d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:01.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.906+0000 7fbb09b61700 1 --2- 172.21.15.107:0/381741689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb04102390 0x7fbb0419d1b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00d980 tx=0x7fbaec00dc90 comp rx=0 tx=0).stop 2024-04-25T00:16:01.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.906+0000 7fbb09b61700 1 -- 172.21.15.107:0/381741689 shutdown_connections 2024-04-25T00:16:01.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.906+0000 7fbb09b61700 1 --2- 172.21.15.107:0/381741689 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fbaf00a2ca0 0x7fbaf00a5160 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:01.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.906+0000 7fbb09b61700 1 --2- 172.21.15.107:0/381741689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb04102d30 0x7fbb0419f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:01.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.906+0000 7fbb09b61700 1 --2- 172.21.15.107:0/381741689 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb04101190 0x7fbb0406b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:01.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.906+0000 7fbb09b61700 1 --2- 172.21.15.107:0/381741689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb04102390 0x7fbb0419d1b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:01.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.906+0000 7fbb09b61700 1 -- 172.21.15.107:0/381741689 >> 172.21.15.107:0/381741689 conn(0x7fbb040fc730 msgr2=0x7fbb04105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:01.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.906+0000 7fbb09b61700 1 -- 172.21.15.107:0/381741689 shutdown_connections 2024-04-25T00:16:01.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:01.906+0000 7fbb09b61700 1 -- 172.21.15.107:0/381741689 wait complete. 2024-04-25T00:16:02.179 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:02 smithi170 ceph-mon[32807]: pgmap v354: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:16:02.179 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-25T00:16:02.179 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]: dispatch 2024-04-25T00:16:02.179 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:02 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-25T00:16:02.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:02 smithi107 ceph-mon[29153]: pgmap v354: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:16:02.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-25T00:16:02.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]: dispatch 2024-04-25T00:16:02.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:02 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-25T00:16:02.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:02 smithi195 ceph-mon[32833]: pgmap v354: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:16:02.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-25T00:16:02.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]: dispatch 2024-04-25T00:16:02.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:02 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-04-25T00:16:02.724 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T00:16:02.962 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:16:03.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:03 smithi195 ceph-mon[32833]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:03.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:03 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-25T00:16:03.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:03 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]': finished 2024-04-25T00:16:03.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:03 smithi195 ceph-mon[32833]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-25T00:16:03.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:03 smithi195 ceph-mon[32833]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T00:16:03.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:03 smithi195 ceph-mon[32833]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T00:16:03.350 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:03 smithi107 ceph-mon[29153]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:03.351 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:03 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-25T00:16:03.351 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:03 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]': finished 2024-04-25T00:16:03.351 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:03 smithi107 ceph-mon[29153]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-25T00:16:03.351 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:03 smithi107 ceph-mon[29153]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T00:16:03.351 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:03 smithi107 ceph-mon[29153]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T00:16:03.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:03 smithi170 ceph-mon[32807]: from='client.15276 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:03.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:03 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-25T00:16:03.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:03 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 1]}]': finished 2024-04-25T00:16:03.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:03 smithi170 ceph-mon[32807]: from='mgr.14152 172.21.15.107:0/4187826792' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-04-25T00:16:03.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:03 smithi170 ceph-mon[32807]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T00:16:03.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:03 smithi170 ceph-mon[32807]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T00:16:04.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:04 smithi195 ceph-mon[32833]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T00:16:04.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:04 smithi107 ceph-mon[29153]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T00:16:04.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:04 smithi170 ceph-mon[32807]: pgmap v356: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T00:16:04.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.753+0000 7fd8f410d700 1 -- 172.21.15.107:0/3975538812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8ec101190 msgr2=0x7fd8ec1015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:04.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.753+0000 7fd8f410d700 1 --2- 172.21.15.107:0/3975538812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8ec101190 0x7fd8ec1015b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc0099c0 tx=0x7fd8dc009cd0 comp rx=0 tx=0).stop 2024-04-25T00:16:04.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.753+0000 7fd8f410d700 1 -- 172.21.15.107:0/3975538812 shutdown_connections 2024-04-25T00:16:04.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.753+0000 7fd8f410d700 1 --2- 172.21.15.107:0/3975538812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8ec102d30 0x7fd8ec1095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:04.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.753+0000 7fd8f410d700 1 --2- 172.21.15.107:0/3975538812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8ec101190 0x7fd8ec1015b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:04.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.753+0000 7fd8f410d700 1 --2- 172.21.15.107:0/3975538812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8ec102390 0x7fd8ec1027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:04.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.753+0000 7fd8f410d700 1 -- 172.21.15.107:0/3975538812 >> 172.21.15.107:0/3975538812 conn(0x7fd8ec0fc750 msgr2=0x7fd8ec0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:04.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.754+0000 7fd8f410d700 1 -- 172.21.15.107:0/3975538812 shutdown_connections 2024-04-25T00:16:04.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.754+0000 7fd8f410d700 1 -- 172.21.15.107:0/3975538812 wait complete. 2024-04-25T00:16:04.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.754+0000 7fd8f410d700 1 Processor -- start 2024-04-25T00:16:04.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.754+0000 7fd8f410d700 1 -- start start 2024-04-25T00:16:04.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.754+0000 7fd8f410d700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8ec101190 0x7fd8ec1050b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:04.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.754+0000 7fd8f410d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8ec102390 0x7fd8ec103770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:04.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.754+0000 7fd8f410d700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8ec102d30 0x7fd8ec103cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:04.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.754+0000 7fd8f410d700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec10cd50 con 0x7fd8ec102d30 2024-04-25T00:16:04.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.754+0000 7fd8f410d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec10cbd0 con 0x7fd8ec102390 2024-04-25T00:16:04.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.754+0000 7fd8f410d700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ec10ced0 con 0x7fd8ec101190 2024-04-25T00:16:04.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.755+0000 7fd8f26aa700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8ec102d30 0x7fd8ec103cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:04.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.755+0000 7fd8f26aa700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8ec102d30 0x7fd8ec103cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36776/0 (socket says 172.21.15.107:36776) 2024-04-25T00:16:04.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.755+0000 7fd8f26aa700 1 -- 172.21.15.107:0/4074836813 learned_addr learned my addr 172.21.15.107:0/4074836813 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:04.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.755+0000 7fd8f16a8700 1 --2- 172.21.15.107:0/4074836813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8ec102390 0x7fd8ec103770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:04.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.755+0000 7fd8f1ea9700 1 --2- 172.21.15.107:0/4074836813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8ec101190 0x7fd8ec1050b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:04.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.755+0000 7fd8f26aa700 1 -- 172.21.15.107:0/4074836813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8ec101190 msgr2=0x7fd8ec1050b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:04.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.755+0000 7fd8f26aa700 1 --2- 172.21.15.107:0/4074836813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8ec101190 0x7fd8ec1050b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:04.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.755+0000 7fd8f26aa700 1 -- 172.21.15.107:0/4074836813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8ec102390 msgr2=0x7fd8ec103770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:04.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.755+0000 7fd8f26aa700 1 --2- 172.21.15.107:0/4074836813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8ec102390 0x7fd8ec103770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:04.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.755+0000 7fd8f26aa700 1 -- 172.21.15.107:0/4074836813 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ec1044f0 con 0x7fd8ec102d30 2024-04-25T00:16:04.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.755+0000 7fd8f16a8700 1 --2- 172.21.15.107:0/4074836813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8ec102390 0x7fd8ec103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:04.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.755+0000 7fd8f26aa700 1 --2- 172.21.15.107:0/4074836813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8ec102d30 0x7fd8ec103cb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400ada0 tx=0x7fd8e400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:04.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.755+0000 7fd8e2ffd700 1 -- 172.21.15.107:0/4074836813 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4017020 con 0x7fd8ec102d30 2024-04-25T00:16:04.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.755+0000 7fd8e2ffd700 1 -- 172.21.15.107:0/4074836813 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8e4004bb0 con 0x7fd8ec102d30 2024-04-25T00:16:04.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.755+0000 7fd8f410d700 1 -- 172.21.15.107:0/4074836813 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ec1a2630 con 0x7fd8ec102d30 2024-04-25T00:16:04.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.755+0000 7fd8e2ffd700 1 -- 172.21.15.107:0/4074836813 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4005520 con 0x7fd8ec102d30 2024-04-25T00:16:04.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.755+0000 7fd8f410d700 1 -- 172.21.15.107:0/4074836813 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ec1a2b20 con 0x7fd8ec102d30 2024-04-25T00:16:04.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.756+0000 7fd8f410d700 1 -- 172.21.15.107:0/4074836813 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8ec068f20 con 0x7fd8ec102d30 2024-04-25T00:16:04.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.757+0000 7fd8e2ffd700 1 -- 172.21.15.107:0/4074836813 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fd8e40056c0 con 0x7fd8ec102d30 2024-04-25T00:16:04.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.758+0000 7fd8e2ffd700 1 --2- 172.21.15.107:0/4074836813 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd8d80a2c20 0x7fd8d80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:04.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.758+0000 7fd8f1ea9700 1 --2- 172.21.15.107:0/4074836813 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd8d80a2c20 0x7fd8d80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:04.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.758+0000 7fd8e2ffd700 1 -- 172.21.15.107:0/4074836813 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7fd8e40cf150 con 0x7fd8ec102d30 2024-04-25T00:16:04.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.758+0000 7fd8f1ea9700 1 --2- 172.21.15.107:0/4074836813 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd8d80a2c20 0x7fd8d80a50e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc0039c0 tx=0x7fd8dc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:04.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.761+0000 7fd8e2ffd700 1 -- 172.21.15.107:0/4074836813 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd8e4021020 con 0x7fd8ec102d30 2024-04-25T00:16:04.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.961+0000 7fd8f410d700 1 -- 172.21.15.107:0/4074836813 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fd8ec0fe910 con 0x7fd8d80a2c20 2024-04-25T00:16:04.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.963+0000 7fd8e2ffd700 1 -- 172.21.15.107:0/4074836813 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fd8ec0fe910 con 0x7fd8d80a2c20 2024-04-25T00:16:04.963 INFO:teuthology.orchestra.run.smithi107.stdout:HOST ADDR LABELS STATUS 2024-04-25T00:16:04.963 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107 172.21.15.107 2024-04-25T00:16:04.963 INFO:teuthology.orchestra.run.smithi107.stdout:smithi170 172.21.15.170 2024-04-25T00:16:04.963 INFO:teuthology.orchestra.run.smithi107.stdout:smithi195 172.21.15.195 2024-04-25T00:16:04.963 INFO:teuthology.orchestra.run.smithi107.stdout:3 hosts in cluster 2024-04-25T00:16:04.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.965+0000 7fd8f410d700 1 -- 172.21.15.107:0/4074836813 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd8d80a2c20 msgr2=0x7fd8d80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:04.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.965+0000 7fd8f410d700 1 --2- 172.21.15.107:0/4074836813 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd8d80a2c20 0x7fd8d80a50e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc0039c0 tx=0x7fd8dc000bc0 comp rx=0 tx=0).stop 2024-04-25T00:16:04.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.965+0000 7fd8f410d700 1 -- 172.21.15.107:0/4074836813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8ec102d30 msgr2=0x7fd8ec103cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:04.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.965+0000 7fd8f410d700 1 --2- 172.21.15.107:0/4074836813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8ec102d30 0x7fd8ec103cb0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400ada0 tx=0x7fd8e400c330 comp rx=0 tx=0).stop 2024-04-25T00:16:04.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.965+0000 7fd8f410d700 1 -- 172.21.15.107:0/4074836813 shutdown_connections 2024-04-25T00:16:04.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.965+0000 7fd8f410d700 1 --2- 172.21.15.107:0/4074836813 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fd8d80a2c20 0x7fd8d80a50e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:04.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.965+0000 7fd8f410d700 1 --2- 172.21.15.107:0/4074836813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8ec102d30 0x7fd8ec103cb0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:04.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.965+0000 7fd8f410d700 1 --2- 172.21.15.107:0/4074836813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8ec101190 0x7fd8ec1050b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:04.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.965+0000 7fd8f410d700 1 --2- 172.21.15.107:0/4074836813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8ec102390 0x7fd8ec103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:04.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.965+0000 7fd8f410d700 1 -- 172.21.15.107:0/4074836813 >> 172.21.15.107:0/4074836813 conn(0x7fd8ec0fc750 msgr2=0x7fd8ec0fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:04.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.965+0000 7fd8f410d700 1 -- 172.21.15.107:0/4074836813 shutdown_connections 2024-04-25T00:16:04.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:04.965+0000 7fd8f410d700 1 -- 172.21.15.107:0/4074836813 wait complete. 2024-04-25T00:16:05.662 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T00:16:05.905 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:16:06.277 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:06 smithi107 ceph-mon[29153]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:16:06.277 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:06 smithi107 ceph-mon[29153]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:06.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:06 smithi195 ceph-mon[32833]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:16:06.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:06 smithi195 ceph-mon[32833]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:06.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:06 smithi170 ceph-mon[32807]: pgmap v358: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:16:06.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:06 smithi170 ceph-mon[32807]: from='client.15282 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:07.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.613+0000 7f8240830700 1 -- 172.21.15.107:0/1962820032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f823c105650 msgr2=0x7f823c107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:07.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.613+0000 7f8240830700 1 --2- 172.21.15.107:0/1962820032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f823c105650 0x7f823c107a40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8230009990 tx=0x7f8230009ca0 comp rx=0 tx=0).stop 2024-04-25T00:16:07.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.613+0000 7f8240830700 1 -- 172.21.15.107:0/1962820032 shutdown_connections 2024-04-25T00:16:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.613+0000 7f8240830700 1 --2- 172.21.15.107:0/1962820032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f823c107f80 0x7f823c10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.613+0000 7f8240830700 1 --2- 172.21.15.107:0/1962820032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f823c102d20 0x7f823c105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.613+0000 7f8240830700 1 --2- 172.21.15.107:0/1962820032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f823c105650 0x7f823c107a40 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.613+0000 7f8240830700 1 -- 172.21.15.107:0/1962820032 >> 172.21.15.107:0/1962820032 conn(0x7f823c0fc750 msgr2=0x7f823c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.613+0000 7f8240830700 1 -- 172.21.15.107:0/1962820032 shutdown_connections 2024-04-25T00:16:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.613+0000 7f8240830700 1 -- 172.21.15.107:0/1962820032 wait complete. 2024-04-25T00:16:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.613+0000 7f8240830700 1 Processor -- start 2024-04-25T00:16:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.614+0000 7f8240830700 1 -- start start 2024-04-25T00:16:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.614+0000 7f8240830700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f823c102d20 0x7f823c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.614+0000 7f8240830700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f823c105650 0x7f823c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.614+0000 7f8240830700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f823c107f80 0x7f823c19fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.614+0000 7f8240830700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f823c10dea0 con 0x7f823c102d20 2024-04-25T00:16:07.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.614+0000 7f8240830700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f823c10dd20 con 0x7f823c105650 2024-04-25T00:16:07.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.614+0000 7f8240830700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f823c10e020 con 0x7f823c107f80 2024-04-25T00:16:07.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.614+0000 7f823a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f823c102d20 0x7f823c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:07.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.614+0000 7f823a59c700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f823c102d20 0x7f823c1984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36786/0 (socket says 172.21.15.107:36786) 2024-04-25T00:16:07.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.614+0000 7f823a59c700 1 -- 172.21.15.107:0/239983280 learned_addr learned my addr 172.21.15.107:0/239983280 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:07.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.614+0000 7f8239d9b700 1 --2- 172.21.15.107:0/239983280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f823c105650 0x7f823c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:07.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.614+0000 7f823ad9d700 1 --2- 172.21.15.107:0/239983280 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f823c107f80 0x7f823c19fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:07.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.614+0000 7f823a59c700 1 -- 172.21.15.107:0/239983280 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f823c107f80 msgr2=0x7f823c19fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:07.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.614+0000 7f823a59c700 1 --2- 172.21.15.107:0/239983280 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f823c107f80 0x7f823c19fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:07.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.614+0000 7f823a59c700 1 -- 172.21.15.107:0/239983280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f823c105650 msgr2=0x7f823c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:07.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.614+0000 7f823a59c700 1 --2- 172.21.15.107:0/239983280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f823c105650 0x7f823c1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:07.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.614+0000 7f823a59c700 1 -- 172.21.15.107:0/239983280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823c1a0110 con 0x7f823c102d20 2024-04-25T00:16:07.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.614+0000 7f8239d9b700 1 --2- 172.21.15.107:0/239983280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f823c105650 0x7f823c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:07.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.614+0000 7f823ad9d700 1 --2- 172.21.15.107:0/239983280 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f823c107f80 0x7f823c19fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:07.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.614+0000 7f823a59c700 1 --2- 172.21.15.107:0/239983280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f823c102d20 0x7f823c1984b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f822c00d980 tx=0x7f822c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:07.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.614+0000 7f822b7fe700 1 -- 172.21.15.107:0/239983280 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c016070 con 0x7f823c102d20 2024-04-25T00:16:07.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.615+0000 7f8240830700 1 -- 172.21.15.107:0/239983280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f823c1a0400 con 0x7f823c102d20 2024-04-25T00:16:07.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.615+0000 7f822b7fe700 1 -- 172.21.15.107:0/239983280 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f822c009b40 con 0x7f823c102d20 2024-04-25T00:16:07.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.615+0000 7f822b7fe700 1 -- 172.21.15.107:0/239983280 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c00f620 con 0x7f823c102d20 2024-04-25T00:16:07.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.615+0000 7f8240830700 1 -- 172.21.15.107:0/239983280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f823c1a0940 con 0x7f823c102d20 2024-04-25T00:16:07.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.616+0000 7f8240830700 1 -- 172.21.15.107:0/239983280 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f823c068f20 con 0x7f823c102d20 2024-04-25T00:16:07.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.617+0000 7f822b7fe700 1 -- 172.21.15.107:0/239983280 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f822c019070 con 0x7f823c102d20 2024-04-25T00:16:07.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.617+0000 7f822b7fe700 1 --2- 172.21.15.107:0/239983280 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f82240a2c20 0x7f82240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:07.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.618+0000 7f8239d9b700 1 --2- 172.21.15.107:0/239983280 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f82240a2c20 0x7f82240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:07.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.618+0000 7f822b7fe700 1 -- 172.21.15.107:0/239983280 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7f822c0cefc0 con 0x7f823c102d20 2024-04-25T00:16:07.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.618+0000 7f8239d9b700 1 --2- 172.21.15.107:0/239983280 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f82240a2c20 0x7f82240a50e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f823000b5c0 tx=0x7f8230005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:07.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.622+0000 7f822b7fe700 1 -- 172.21.15.107:0/239983280 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f822c09bb90 con 0x7f823c102d20 2024-04-25T00:16:07.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.831+0000 7f8240830700 1 -- 172.21.15.107:0/239983280 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f823c062670 con 0x7f82240a2c20 2024-04-25T00:16:07.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.833+0000 7f822b7fe700 1 -- 172.21.15.107:0/239983280 <== mgr.14152 v2:172.21.15.107:6800/1644942185 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f823c062670 con 0x7f82240a2c20 2024-04-25T00:16:07.833 INFO:teuthology.orchestra.run.smithi107.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T00:16:07.834 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NK400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T00:16:07.834 INFO:teuthology.orchestra.run.smithi107.stdout:smithi170 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002R480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-25T00:16:07.834 INFO:teuthology.orchestra.run.smithi107.stdout:smithi195 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300FS480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-04-25T00:16:07.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.835+0000 7f8240830700 1 -- 172.21.15.107:0/239983280 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f82240a2c20 msgr2=0x7f82240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:07.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.835+0000 7f8240830700 1 --2- 172.21.15.107:0/239983280 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f82240a2c20 0x7f82240a50e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f823000b5c0 tx=0x7f8230005fb0 comp rx=0 tx=0).stop 2024-04-25T00:16:07.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.835+0000 7f8240830700 1 -- 172.21.15.107:0/239983280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f823c102d20 msgr2=0x7f823c1984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:07.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.835+0000 7f8240830700 1 --2- 172.21.15.107:0/239983280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f823c102d20 0x7f823c1984b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f822c00d980 tx=0x7f822c00dd40 comp rx=0 tx=0).stop 2024-04-25T00:16:07.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.835+0000 7f8240830700 1 -- 172.21.15.107:0/239983280 shutdown_connections 2024-04-25T00:16:07.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.835+0000 7f8240830700 1 --2- 172.21.15.107:0/239983280 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f82240a2c20 0x7f82240a50e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:07.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.835+0000 7f8240830700 1 --2- 172.21.15.107:0/239983280 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f823c107f80 0x7f823c19fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:07.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.835+0000 7f8240830700 1 --2- 172.21.15.107:0/239983280 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f823c102d20 0x7f823c1984b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:07.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.835+0000 7f8240830700 1 --2- 172.21.15.107:0/239983280 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f823c105650 0x7f823c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:07.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.835+0000 7f8240830700 1 -- 172.21.15.107:0/239983280 >> 172.21.15.107:0/239983280 conn(0x7f823c0fc750 msgr2=0x7f823c103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:07.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.835+0000 7f8240830700 1 -- 172.21.15.107:0/239983280 shutdown_connections 2024-04-25T00:16:07.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:07.835+0000 7f8240830700 1 -- 172.21.15.107:0/239983280 wait complete. 2024-04-25T00:16:08.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:08 smithi195 ceph-mon[32833]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:16:08.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:08 smithi170 ceph-mon[32807]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:16:08.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:08 smithi107 ceph-mon[29153]: pgmap v359: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:16:08.527 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T00:16:08.541 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T00:16:08.541 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T00:16:08.791 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:16:09.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:09 smithi195 ceph-mon[32833]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:09.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:09 smithi170 ceph-mon[32807]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:09.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:09 smithi107 ceph-mon[29153]: from='client.15288 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:10.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:10 smithi195 ceph-mon[32833]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:16:10.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:10 smithi107 ceph-mon[29153]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:16:10.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:10 smithi170 ceph-mon[32807]: pgmap v360: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:16:10.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.488+0000 7f7fca58b700 1 -- 172.21.15.107:0/937625533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fc4103490 msgr2=0x7f7fc41038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:10.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.488+0000 7f7fca58b700 1 --2- 172.21.15.107:0/937625533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fc4103490 0x7f7fc41038f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0099c0 tx=0x7f7fac009cd0 comp rx=0 tx=0).stop 2024-04-25T00:16:10.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.488+0000 7f7fca58b700 1 -- 172.21.15.107:0/937625533 shutdown_connections 2024-04-25T00:16:10.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.488+0000 7f7fca58b700 1 --2- 172.21.15.107:0/937625533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7fc4103e30 0x7f7fc410a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:10.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.488+0000 7f7fca58b700 1 --2- 172.21.15.107:0/937625533 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc4102290 0x7f7fc41026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:10.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.488+0000 7f7fca58b700 1 --2- 172.21.15.107:0/937625533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fc4103490 0x7f7fc41038f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:10.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.488+0000 7f7fca58b700 1 -- 172.21.15.107:0/937625533 >> 172.21.15.107:0/937625533 conn(0x7f7fc40fd830 msgr2=0x7f7fc40ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:10.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.488+0000 7f7fca58b700 1 -- 172.21.15.107:0/937625533 shutdown_connections 2024-04-25T00:16:10.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.488+0000 7f7fca58b700 1 -- 172.21.15.107:0/937625533 wait complete. 2024-04-25T00:16:10.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.489+0000 7f7fca58b700 1 Processor -- start 2024-04-25T00:16:10.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.489+0000 7f7fca58b700 1 -- start start 2024-04-25T00:16:10.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.489+0000 7f7fca58b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc4102290 0x7f7fc4075fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:10.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.489+0000 7f7fca58b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7fc4103490 0x7f7fc4074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:10.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.489+0000 7f7fca58b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fc4103e30 0x7f7fc4074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:10.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.489+0000 7f7fca58b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc410de50 con 0x7f7fc4103e30 2024-04-25T00:16:10.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.489+0000 7f7fca58b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc410dcd0 con 0x7f7fc4103490 2024-04-25T00:16:10.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.489+0000 7f7fca58b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc410dfd0 con 0x7f7fc4102290 2024-04-25T00:16:10.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.489+0000 7f7fc8b28700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fc4103e30 0x7f7fc4074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:10.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.490+0000 7f7fc8b28700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fc4103e30 0x7f7fc4074b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56854/0 (socket says 172.21.15.107:56854) 2024-04-25T00:16:10.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.490+0000 7f7fc8b28700 1 -- 172.21.15.107:0/2968424232 learned_addr learned my addr 172.21.15.107:0/2968424232 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:10.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.490+0000 7f7fc8b28700 1 -- 172.21.15.107:0/2968424232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc4102290 msgr2=0x7f7fc4075fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:10.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.490+0000 7f7fc8b28700 1 --2- 172.21.15.107:0/2968424232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc4102290 0x7f7fc4075fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:10.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.490+0000 7f7fc37fe700 1 --2- 172.21.15.107:0/2968424232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7fc4103490 0x7f7fc4074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:10.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.490+0000 7f7fc8b28700 1 -- 172.21.15.107:0/2968424232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7fc4103490 msgr2=0x7f7fc4074620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:10.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.490+0000 7f7fc8b28700 1 --2- 172.21.15.107:0/2968424232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7fc4103490 0x7f7fc4074620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:10.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.490+0000 7f7fc8b28700 1 -- 172.21.15.107:0/2968424232 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc4075410 con 0x7f7fc4103e30 2024-04-25T00:16:10.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.490+0000 7f7fc8b28700 1 --2- 172.21.15.107:0/2968424232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fc4103e30 0x7f7fc4074b60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00ada0 tx=0x7f7fbc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:10.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.490+0000 7f7fc17fa700 1 -- 172.21.15.107:0/2968424232 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc017020 con 0x7f7fc4103e30 2024-04-25T00:16:10.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.490+0000 7f7fc17fa700 1 -- 172.21.15.107:0/2968424232 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fbc004bb0 con 0x7f7fc4103e30 2024-04-25T00:16:10.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.490+0000 7f7fca58b700 1 -- 172.21.15.107:0/2968424232 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc41a2650 con 0x7f7fc4103e30 2024-04-25T00:16:10.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.490+0000 7f7fc17fa700 1 -- 172.21.15.107:0/2968424232 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc005520 con 0x7f7fc4103e30 2024-04-25T00:16:10.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.490+0000 7f7fca58b700 1 -- 172.21.15.107:0/2968424232 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc41a2b10 con 0x7f7fc4103e30 2024-04-25T00:16:10.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.491+0000 7f7fca58b700 1 -- 172.21.15.107:0/2968424232 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fc4068f20 con 0x7f7fc4103e30 2024-04-25T00:16:10.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.492+0000 7f7fc17fa700 1 -- 172.21.15.107:0/2968424232 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7fbc0056c0 con 0x7f7fc4103e30 2024-04-25T00:16:10.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.493+0000 7f7fc17fa700 1 --2- 172.21.15.107:0/2968424232 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7fb00a2c20 0x7f7fb00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:10.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.493+0000 7f7fc3fff700 1 --2- 172.21.15.107:0/2968424232 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7fb00a2c20 0x7f7fb00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:10.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.493+0000 7f7fc17fa700 1 -- 172.21.15.107:0/2968424232 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7f7fbc0ceb90 con 0x7f7fc4103e30 2024-04-25T00:16:10.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.493+0000 7f7fc3fff700 1 --2- 172.21.15.107:0/2968424232 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7fb00a2c20 0x7f7fb00a50e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7fc41032f0 tx=0x7f7fb4009470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:10.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.497+0000 7f7fc17fa700 1 -- 172.21.15.107:0/2968424232 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7fbc021020 con 0x7f7fc4103e30 2024-04-25T00:16:10.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.737+0000 7f7fca58b700 1 -- 172.21.15.107:0/2968424232 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f7fc4062e40 con 0x7f7fc4103e30 2024-04-25T00:16:10.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.738+0000 7f7fc17fa700 1 -- 172.21.15.107:0/2968424232 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) ==== 76+0+3068 (secure 0 0 0) 0x7f7fbc09b7f0 con 0x7f7fc4103e30 2024-04-25T00:16:10.738 INFO:teuthology.orchestra.run.smithi107.stdout:e23 2024-04-25T00:16:10.739 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T00:16:10.739 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T00:16:10.739 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-04-25T00:16:10.739 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:16:10.739 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-04-25T00:16:10.739 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-04-25T00:16:10.739 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 23 2024-04-25T00:16:10.739 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T00:16:10.739 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-25T00:14:01.788955+0000 2024-04-25T00:16:10.739 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-25T00:14:28.529865+0000 2024-04-25T00:16:10.739 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-04-25T00:16:10.739 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-04-25T00:16:10.739 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 300 2024-04-25T00:16:10.739 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-04-25T00:16:10.740 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-04-25T00:16:10.740 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-04-25T00:16:10.740 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-04-25T00:16:10.740 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-04-25T00:16:10.740 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 0 2024-04-25T00:16:10.740 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T00:16:10.740 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 3 2024-04-25T00:16:10.740 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1,2 2024-04-25T00:16:10.740 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24521,1=24493,2=24485} 2024-04-25T00:16:10.740 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-04-25T00:16:10.740 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-04-25T00:16:10.740 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-04-25T00:16:10.740 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3,4] 2024-04-25T00:16:10.740 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-04-25T00:16:10.741 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-04-25T00:16:10.741 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-04-25T00:16:10.741 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-04-25T00:16:10.741 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 0 2024-04-25T00:16:10.741 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 24521 members: 24485,24521,24493 2024-04-25T00:16:10.741 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.b{0:24521} state up:active seq 3 addr [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:16:10.741 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.c{0:24472} state up:standby-replay seq 1 addr [v2:172.21.15.195:6836/2245799531,v1:172.21.15.195:6837/2245799531] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:16:10.741 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.i{1:24493} state up:active seq 11 addr [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:16:10.741 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.f{1:24463} state up:standby-replay seq 1 addr [v2:172.21.15.195:6834/2828335061,v1:172.21.15.195:6835/2828335061] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:16:10.741 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.h{2:24485} state up:active seq 12 addr [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:16:10.741 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.e{2:24443} state up:standby-replay seq 1 addr [v2:172.21.15.170:6834/2926641872,v1:172.21.15.170:6835/2926641872] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:16:10.741 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:16:10.741 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:16:10.741 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-04-25T00:16:10.741 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:16:10.742 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.j{-1:14544} state up:standby seq 1 addr [v2:172.21.15.107:6834/2628342623,v1:172.21.15.107:6835/2628342623] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:16:10.742 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.d{-1:14556} state up:standby seq 1 addr [v2:172.21.15.107:6836/2695944193,v1:172.21.15.107:6837/2695944193] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:16:10.742 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.g{-1:14619} state up:standby seq 1 addr [v2:172.21.15.107:6838/2028240372,v1:172.21.15.107:6839/2028240372] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:16:10.742 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.a{-1:14631} state up:standby seq 1 addr [v2:172.21.15.107:6840/3282278035,v1:172.21.15.107:6841/3282278035] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:16:10.742 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.k{-1:24413} state up:standby seq 1 addr [v2:172.21.15.170:6832/674283763,v1:172.21.15.170:6833/674283763] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:16:10.742 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.l{-1:24421} state up:standby seq 1 addr [v2:172.21.15.195:6832/2162039457,v1:172.21.15.195:6833/2162039457] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:16:10.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.740+0000 7f7fca58b700 1 -- 172.21.15.107:0/2968424232 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7fb00a2c20 msgr2=0x7f7fb00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:10.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.740+0000 7f7fca58b700 1 --2- 172.21.15.107:0/2968424232 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7fb00a2c20 0x7f7fb00a50e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7fc41032f0 tx=0x7f7fb4009470 comp rx=0 tx=0).stop 2024-04-25T00:16:10.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.740+0000 7f7fca58b700 1 -- 172.21.15.107:0/2968424232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fc4103e30 msgr2=0x7f7fc4074b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:10.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.740+0000 7f7fca58b700 1 --2- 172.21.15.107:0/2968424232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fc4103e30 0x7f7fc4074b60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00ada0 tx=0x7f7fbc00c330 comp rx=0 tx=0).stop 2024-04-25T00:16:10.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.740+0000 7f7fca58b700 1 -- 172.21.15.107:0/2968424232 shutdown_connections 2024-04-25T00:16:10.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.740+0000 7f7fca58b700 1 --2- 172.21.15.107:0/2968424232 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7fb00a2c20 0x7f7fb00a50e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:10.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.740+0000 7f7fca58b700 1 --2- 172.21.15.107:0/2968424232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7fc4103e30 0x7f7fc4074b60 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:10.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.740+0000 7f7fca58b700 1 --2- 172.21.15.107:0/2968424232 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc4102290 0x7f7fc4075fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:10.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.740+0000 7f7fca58b700 1 --2- 172.21.15.107:0/2968424232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7fc4103490 0x7f7fc4074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:10.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.740+0000 7f7fca58b700 1 -- 172.21.15.107:0/2968424232 >> 172.21.15.107:0/2968424232 conn(0x7f7fc40fd830 msgr2=0x7f7fc41067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:10.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.740+0000 7f7fca58b700 1 -- 172.21.15.107:0/2968424232 shutdown_connections 2024-04-25T00:16:10.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:10.740+0000 7f7fca58b700 1 -- 172.21.15.107:0/2968424232 wait complete. 2024-04-25T00:16:10.744 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 23 2024-04-25T00:16:11.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:11 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2968424232' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T00:16:11.376 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:11 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2968424232' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T00:16:11.405 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T00:16:11.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:11 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2968424232' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T00:16:11.646 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:16:12.336 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:12 smithi170 ceph-mon[32807]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:16:12.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:12 smithi195 ceph-mon[32833]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:16:12.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:12 smithi107 ceph-mon[29153]: pgmap v361: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:16:13.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.325+0000 7f5613c1b700 1 -- 172.21.15.107:0/2241696788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f560c103e50 msgr2=0x7f560c10a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:13.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.325+0000 7f5613c1b700 1 --2- 172.21.15.107:0/2241696788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f560c103e50 0x7f560c10a700 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f560400b0d0 tx=0x7f560400b3e0 comp rx=0 tx=0).stop 2024-04-25T00:16:13.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.325+0000 7f5613c1b700 1 -- 172.21.15.107:0/2241696788 shutdown_connections 2024-04-25T00:16:13.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.325+0000 7f5613c1b700 1 --2- 172.21.15.107:0/2241696788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f560c103e50 0x7f560c10a700 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:13.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.325+0000 7f5613c1b700 1 --2- 172.21.15.107:0/2241696788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f560c1022b0 0x7f560c1026d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:13.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.325+0000 7f5613c1b700 1 --2- 172.21.15.107:0/2241696788 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f560c1034b0 0x7f560c103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:13.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.325+0000 7f5613c1b700 1 -- 172.21.15.107:0/2241696788 >> 172.21.15.107:0/2241696788 conn(0x7f560c0fd850 msgr2=0x7f560c0ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:13.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.326+0000 7f5613c1b700 1 -- 172.21.15.107:0/2241696788 shutdown_connections 2024-04-25T00:16:13.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.326+0000 7f5613c1b700 1 -- 172.21.15.107:0/2241696788 wait complete. 2024-04-25T00:16:13.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.326+0000 7f5613c1b700 1 Processor -- start 2024-04-25T00:16:13.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.326+0000 7f5613c1b700 1 -- start start 2024-04-25T00:16:13.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.326+0000 7f5613c1b700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f560c1022b0 0x7f560c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:13.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.326+0000 7f5613c1b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f560c1034b0 0x7f560c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:13.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.326+0000 7f5613c1b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f560c103e50 0x7f560c19f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:13.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.326+0000 7f5613c1b700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f560c10de70 con 0x7f560c103e50 2024-04-25T00:16:13.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.326+0000 7f5613c1b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f560c10dcf0 con 0x7f560c1034b0 2024-04-25T00:16:13.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.326+0000 7f5613c1b700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f560c10dff0 con 0x7f560c1022b0 2024-04-25T00:16:13.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.326+0000 7f56121b8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f560c103e50 0x7f560c19f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:13.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.326+0000 7f56119b7700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f560c1022b0 0x7f560c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:13.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.326+0000 7f56121b8700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f560c103e50 0x7f560c19f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56860/0 (socket says 172.21.15.107:56860) 2024-04-25T00:16:13.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.326+0000 7f56121b8700 1 -- 172.21.15.107:0/915324667 learned_addr learned my addr 172.21.15.107:0/915324667 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:13.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.327+0000 7f56111b6700 1 --2- 172.21.15.107:0/915324667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f560c1034b0 0x7f560c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:13.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.327+0000 7f56121b8700 1 -- 172.21.15.107:0/915324667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f560c1022b0 msgr2=0x7f560c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:13.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.327+0000 7f56121b8700 1 --2- 172.21.15.107:0/915324667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f560c1022b0 0x7f560c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:13.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.327+0000 7f56121b8700 1 -- 172.21.15.107:0/915324667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f560c1034b0 msgr2=0x7f560c198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:13.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.327+0000 7f56121b8700 1 --2- 172.21.15.107:0/915324667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f560c1034b0 0x7f560c198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:13.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.327+0000 7f56121b8700 1 -- 172.21.15.107:0/915324667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560c1a0030 con 0x7f560c103e50 2024-04-25T00:16:13.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.327+0000 7f56121b8700 1 --2- 172.21.15.107:0/915324667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f560c103e50 0x7f560c19f990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5604006070 tx=0x7f5604007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:13.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.327+0000 7f5602ffd700 1 -- 172.21.15.107:0/915324667 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604003c00 con 0x7f560c103e50 2024-04-25T00:16:13.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.327+0000 7f5602ffd700 1 -- 172.21.15.107:0/915324667 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5604003da0 con 0x7f560c103e50 2024-04-25T00:16:13.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.327+0000 7f5613c1b700 1 -- 172.21.15.107:0/915324667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560c1a02c0 con 0x7f560c103e50 2024-04-25T00:16:13.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.327+0000 7f5602ffd700 1 -- 172.21.15.107:0/915324667 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56040054b0 con 0x7f560c103e50 2024-04-25T00:16:13.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.327+0000 7f5613c1b700 1 -- 172.21.15.107:0/915324667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f560c1a07a0 con 0x7f560c103e50 2024-04-25T00:16:13.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.328+0000 7f5613c1b700 1 -- 172.21.15.107:0/915324667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f560c068f20 con 0x7f560c103e50 2024-04-25T00:16:13.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.329+0000 7f5602ffd700 1 -- 172.21.15.107:0/915324667 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f5604005650 con 0x7f560c103e50 2024-04-25T00:16:13.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.330+0000 7f5602ffd700 1 --2- 172.21.15.107:0/915324667 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f55f80a2c20 0x7f55f80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:13.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.330+0000 7f56119b7700 1 --2- 172.21.15.107:0/915324667 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f55f80a2c20 0x7f55f80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:13.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.330+0000 7f5602ffd700 1 -- 172.21.15.107:0/915324667 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7f560400f020 con 0x7f560c103e50 2024-04-25T00:16:13.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.330+0000 7f56119b7700 1 --2- 172.21.15.107:0/915324667 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f55f80a2c20 0x7f55f80a50e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f560c103310 tx=0x7f560800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:13.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.334+0000 7f5602ffd700 1 -- 172.21.15.107:0/915324667 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f560402b020 con 0x7f560c103e50 2024-04-25T00:16:13.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.533+0000 7f5613c1b700 1 -- 172.21.15.107:0/915324667 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f560c062e40 con 0x7f560c103e50 2024-04-25T00:16:13.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.534+0000 7f5602ffd700 1 -- 172.21.15.107:0/915324667 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v96) ==== 56+0+5642 (secure 0 0 0) 0x7f560409d200 con 0x7f560c103e50 2024-04-25T00:16:13.534 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 96 2024-04-25T00:16:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:fsid 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:16:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-04-25T00:05:56.053373+0000 2024-04-25T00:16:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-04-25T00:16:03.045295+0000 2024-04-25T00:16:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T00:16:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:crush_version 27 2024-04-25T00:16:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:full_ratio 0.95 2024-04-25T00:16:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:backfillfull_ratio 0.9 2024-04-25T00:16:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:nearfull_ratio 0.85 2024-04-25T00:16:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:require_min_compat_client luminous 2024-04-25T00:16:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:min_compat_client luminous 2024-04-25T00:16:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:require_osd_release squid 2024-04-25T00:16:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:stretch_mode_enabled false 2024-04-25T00:16:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T00:16:13.535 INFO:teuthology.orchestra.run.smithi107.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-04-25T00:16:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-25T00:16:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 92 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-25T00:16:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:max_osd 12 2024-04-25T00:16:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 up in weight 1 up_from 8 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6802/331467636,v1:172.21.15.107:6803/331467636] [v2:172.21.15.107:6804/331467636,v1:172.21.15.107:6805/331467636] exists,up 5a74e9b9-fc11-4472-913b-ea0084ac1492 2024-04-25T00:16:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 up in weight 1 up_from 13 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6800/453760841,v1:172.21.15.170:6801/453760841] [v2:172.21.15.170:6802/453760841,v1:172.21.15.170:6803/453760841] exists,up d009a8aa-9bf8-44c8-b873-ca2ab7c0587c 2024-04-25T00:16:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 up in weight 1 up_from 18 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6800/90978480,v1:172.21.15.195:6801/90978480] [v2:172.21.15.195:6802/90978480,v1:172.21.15.195:6803/90978480] exists,up 4df1456c-7657-4efd-8295-bf8248fcbcaf 2024-04-25T00:16:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 up in weight 1 up_from 26 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6810/2902386444,v1:172.21.15.107:6811/2902386444] [v2:172.21.15.107:6812/2902386444,v1:172.21.15.107:6813/2902386444] exists,up 2c75ecce-0fee-4e65-8442-71125e8e5ad3 2024-04-25T00:16:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 up in weight 1 up_from 34 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6808/3780266808,v1:172.21.15.170:6809/3780266808] [v2:172.21.15.170:6810/3780266808,v1:172.21.15.170:6811/3780266808] exists,up 6f61a3aa-62cc-43aa-b4b0-8931c9837b57 2024-04-25T00:16:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 up in weight 1 up_from 39 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6808/2322632437,v1:172.21.15.195:6809/2322632437] [v2:172.21.15.195:6810/2322632437,v1:172.21.15.195:6811/2322632437] exists,up 104da40f-019b-4dc3-8c62-090920ebe24f 2024-04-25T00:16:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6 up in weight 1 up_from 47 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6818/251620415,v1:172.21.15.107:6819/251620415] [v2:172.21.15.107:6820/251620415,v1:172.21.15.107:6821/251620415] exists,up 543e0a0a-23fc-402f-8a71-3bcd276999db 2024-04-25T00:16:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7 up in weight 1 up_from 52 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6816/3231152590,v1:172.21.15.170:6817/3231152590] [v2:172.21.15.170:6818/3231152590,v1:172.21.15.170:6819/3231152590] exists,up aa97be8a-73cd-4452-b143-825370290232 2024-04-25T00:16:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:osd.8 up in weight 1 up_from 60 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6816/2391866428,v1:172.21.15.195:6817/2391866428] [v2:172.21.15.195:6818/2391866428,v1:172.21.15.195:6819/2391866428] exists,up ebddecde-4dac-4b5e-af58-f01f285aa7c5 2024-04-25T00:16:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:osd.9 up in weight 1 up_from 68 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6826/174023257,v1:172.21.15.107:6827/174023257] [v2:172.21.15.107:6828/174023257,v1:172.21.15.107:6829/174023257] exists,up f85eab7a-7430-4520-9b2d-932398199b4c 2024-04-25T00:16:13.536 INFO:teuthology.orchestra.run.smithi107.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6824/1847892477,v1:172.21.15.170:6825/1847892477] [v2:172.21.15.170:6826/1847892477,v1:172.21.15.170:6827/1847892477] exists,up 4acd7b2f-39b0-44d8-8d3a-69d58cbd2226 2024-04-25T00:16:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:osd.11 up in weight 1 up_from 81 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.195:6824/645474884,v1:172.21.15.195:6825/645474884] [v2:172.21.15.195:6826/645474884,v1:172.21.15.195:6827/645474884] exists,up 61c096c8-f55a-442c-9fa8-00231fdbd258 2024-04-25T00:16:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 2.6 [8,11] 2024-04-25T00:16:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 2.a [2,11,8,1] 2024-04-25T00:16:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 2.16 [2,4] 2024-04-25T00:16:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.0 [5,9] 2024-04-25T00:16:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.11 [4,8] 2024-04-25T00:16:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.17 [4,6] 2024-04-25T00:16:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.38 [3,9] 2024-04-25T00:16:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.3c [4,9] 2024-04-25T00:16:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 4.6 [5,8,9,11] 2024-04-25T00:16:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 4.13 [9,8] 2024-04-25T00:16:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 4.14 [9,8] 2024-04-25T00:16:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/1174178948 expires 2024-04-26T00:07:00.838246+0000 2024-04-25T00:16:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:6800/153169302 expires 2024-04-26T00:07:00.838246+0000 2024-04-25T00:16:13.537 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:6800/120616774 expires 2024-04-26T00:06:34.124795+0000 2024-04-25T00:16:13.538 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:6801/120616774 expires 2024-04-26T00:06:34.124795+0000 2024-04-25T00:16:13.538 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/971124970 expires 2024-04-26T00:06:34.124795+0000 2024-04-25T00:16:13.538 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/3022844652 expires 2024-04-26T00:06:34.124795+0000 2024-04-25T00:16:13.538 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:6801/153169302 expires 2024-04-26T00:07:00.838246+0000 2024-04-25T00:16:13.538 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/4215203700 expires 2024-04-26T00:07:00.838246+0000 2024-04-25T00:16:13.538 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/750288374 expires 2024-04-26T00:06:34.124795+0000 2024-04-25T00:16:13.538 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/3718680642 expires 2024-04-26T00:07:00.838246+0000 2024-04-25T00:16:13.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.536+0000 7f5613c1b700 1 -- 172.21.15.107:0/915324667 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f55f80a2c20 msgr2=0x7f55f80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:13.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.536+0000 7f5613c1b700 1 --2- 172.21.15.107:0/915324667 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f55f80a2c20 0x7f55f80a50e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f560c103310 tx=0x7f560800a380 comp rx=0 tx=0).stop 2024-04-25T00:16:13.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.536+0000 7f5613c1b700 1 -- 172.21.15.107:0/915324667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f560c103e50 msgr2=0x7f560c19f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:13.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.536+0000 7f5613c1b700 1 --2- 172.21.15.107:0/915324667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f560c103e50 0x7f560c19f990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5604006070 tx=0x7f5604007d90 comp rx=0 tx=0).stop 2024-04-25T00:16:13.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.536+0000 7f5613c1b700 1 -- 172.21.15.107:0/915324667 shutdown_connections 2024-04-25T00:16:13.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.536+0000 7f5613c1b700 1 --2- 172.21.15.107:0/915324667 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f55f80a2c20 0x7f55f80a50e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:13.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.536+0000 7f5613c1b700 1 --2- 172.21.15.107:0/915324667 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f560c103e50 0x7f560c19f990 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:13.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.536+0000 7f5613c1b700 1 --2- 172.21.15.107:0/915324667 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f560c1022b0 0x7f560c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:13.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.536+0000 7f5613c1b700 1 --2- 172.21.15.107:0/915324667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f560c1034b0 0x7f560c198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:13.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.536+0000 7f5613c1b700 1 -- 172.21.15.107:0/915324667 >> 172.21.15.107:0/915324667 conn(0x7f560c0fd850 msgr2=0x7f560c106750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:13.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.536+0000 7f5613c1b700 1 -- 172.21.15.107:0/915324667 shutdown_connections 2024-04-25T00:16:13.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:13.536+0000 7f5613c1b700 1 -- 172.21.15.107:0/915324667 wait complete. 2024-04-25T00:16:14.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:14 smithi107 ceph-mon[29153]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:16:14.261 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:14 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/915324667' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T00:16:14.262 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T00:16:14.278 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:16:14.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:14 smithi195 ceph-mon[32833]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:16:14.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:14 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/915324667' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T00:16:14.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.380+0000 7f70d4729640 1 -- 172.21.15.107:0/1953246330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70cc101870 msgr2=0x7f70cc101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:14.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.380+0000 7f70d4729640 1 --2- 172.21.15.107:0/1953246330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70cc101870 0x7f70cc101cf0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f70bc009900 tx=0x7f70bc02a410 comp rx=0 tx=0).stop 2024-04-25T00:16:14.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.380+0000 7f70d4729640 1 -- 172.21.15.107:0/1953246330 shutdown_connections 2024-04-25T00:16:14.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.380+0000 7f70d4729640 1 --2- 172.21.15.107:0/1953246330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70cc102230 0x7f70cc108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.380+0000 7f70d4729640 1 --2- 172.21.15.107:0/1953246330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70cc101870 0x7f70cc101cf0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.380+0000 7f70d4729640 1 --2- 172.21.15.107:0/1953246330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70cc100680 0x7f70cc100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.380+0000 7f70d4729640 1 -- 172.21.15.107:0/1953246330 >> 172.21.15.107:0/1953246330 conn(0x7f70cc0fbbf0 msgr2=0x7f70cc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:14.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.380+0000 7f70d4729640 1 -- 172.21.15.107:0/1953246330 shutdown_connections 2024-04-25T00:16:14.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.380+0000 7f70d4729640 1 -- 172.21.15.107:0/1953246330 wait complete. 2024-04-25T00:16:14.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70d4729640 1 Processor -- start 2024-04-25T00:16:14.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70d4729640 1 -- start start 2024-04-25T00:16:14.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70d4729640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70cc100680 0x7f70cc1969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:14.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70d4729640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70cc101870 0x7f70cc196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:14.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70d4729640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70cc102230 0x7f70cc19df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:14.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70d4729640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f70cc10c350 con 0x7f70cc100680 2024-04-25T00:16:14.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70d4729640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f70cc10c1d0 con 0x7f70cc101870 2024-04-25T00:16:14.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70d4729640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f70cc10c4d0 con 0x7f70cc102230 2024-04-25T00:16:14.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70d24c5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70cc100680 0x7f70cc1969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:14.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70d24c5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70cc100680 0x7f70cc1969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56880/0 (socket says 172.21.15.107:56880) 2024-04-25T00:16:14.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70d24c5640 1 -- 172.21.15.107:0/2564648792 learned_addr learned my addr 172.21.15.107:0/2564648792 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:14.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70d1cc4640 1 --2- 172.21.15.107:0/2564648792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70cc101870 0x7f70cc196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:14.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70d2cc6640 1 --2- 172.21.15.107:0/2564648792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70cc102230 0x7f70cc19df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:14.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70d24c5640 1 -- 172.21.15.107:0/2564648792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70cc102230 msgr2=0x7f70cc19df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:14.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70d24c5640 1 --2- 172.21.15.107:0/2564648792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70cc102230 0x7f70cc19df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70d24c5640 1 -- 172.21.15.107:0/2564648792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70cc101870 msgr2=0x7f70cc196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:14.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70d24c5640 1 --2- 172.21.15.107:0/2564648792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70cc101870 0x7f70cc196ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70d24c5640 1 -- 172.21.15.107:0/2564648792 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70cc19e5c0 con 0x7f70cc100680 2024-04-25T00:16:14.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70d2cc6640 1 --2- 172.21.15.107:0/2564648792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70cc102230 0x7f70cc19df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:14.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70d1cc4640 1 --2- 172.21.15.107:0/2564648792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70cc101870 0x7f70cc196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:14.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70d24c5640 1 --2- 172.21.15.107:0/2564648792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70cc100680 0x7f70cc1969a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f70c800c970 tx=0x7f70c800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:14.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70c37fe640 1 -- 172.21.15.107:0/2564648792 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c8004220 con 0x7f70cc100680 2024-04-25T00:16:14.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70c37fe640 1 -- 172.21.15.107:0/2564648792 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70c80043c0 con 0x7f70cc100680 2024-04-25T00:16:14.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70c37fe640 1 -- 172.21.15.107:0/2564648792 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c800f450 con 0x7f70cc100680 2024-04-25T00:16:14.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.381+0000 7f70d4729640 1 -- 172.21.15.107:0/2564648792 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70cc19e8b0 con 0x7f70cc100680 2024-04-25T00:16:14.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.382+0000 7f70d4729640 1 -- 172.21.15.107:0/2564648792 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70cc19ee20 con 0x7f70cc100680 2024-04-25T00:16:14.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.383+0000 7f70d4729640 1 -- 172.21.15.107:0/2564648792 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70cc1079a0 con 0x7f70cc100680 2024-04-25T00:16:14.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.383+0000 7f70c37fe640 1 -- 172.21.15.107:0/2564648792 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f70c80079b0 con 0x7f70cc100680 2024-04-25T00:16:14.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.384+0000 7f70c37fe640 1 --2- 172.21.15.107:0/2564648792 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f70b80a2c10 0x7f70b80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:14.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.384+0000 7f70d1cc4640 1 --2- 172.21.15.107:0/2564648792 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f70b80a2c10 0x7f70b80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:14.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.384+0000 7f70c37fe640 1 -- 172.21.15.107:0/2564648792 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7f70c80cf1c0 con 0x7f70cc100680 2024-04-25T00:16:14.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.384+0000 7f70d1cc4640 1 --2- 172.21.15.107:0/2564648792 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f70b80a2c10 0x7f70b80a50d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f70bc006010 tx=0x7f70bc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:14.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.388+0000 7f70c37fe640 1 -- 172.21.15.107:0/2564648792 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f70c8014030 con 0x7f70cc100680 2024-04-25T00:16:14.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:14 smithi170 ceph-mon[32807]: pgmap v362: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:16:14.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:14 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/915324667' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T00:16:14.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.567+0000 7f70d4729640 1 -- 172.21.15.107:0/2564648792 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f70cc02d170 con 0x7f70cc100680 2024-04-25T00:16:14.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.569+0000 7f70c37fe640 1 -- 172.21.15.107:0/2564648792 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24688 (secure 0 0 0) 0x7f70c809bd90 con 0x7f70cc100680 2024-04-25T00:16:14.569 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:16:14.570 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":96,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:16:03.045295+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:59.672801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:00.957782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:14:05.065846+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8},{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:16:14.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.571+0000 7f70d4729640 1 -- 172.21.15.107:0/2564648792 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f70b80a2c10 msgr2=0x7f70b80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:14.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.571+0000 7f70d4729640 1 --2- 172.21.15.107:0/2564648792 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f70b80a2c10 0x7f70b80a50d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f70bc006010 tx=0x7f70bc000bc0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.571+0000 7f70d4729640 1 -- 172.21.15.107:0/2564648792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70cc100680 msgr2=0x7f70cc1969a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:14.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.571+0000 7f70d4729640 1 --2- 172.21.15.107:0/2564648792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70cc100680 0x7f70cc1969a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f70c800c970 tx=0x7f70c800ce30 comp rx=0 tx=0).stop 2024-04-25T00:16:14.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.571+0000 7f70d4729640 1 -- 172.21.15.107:0/2564648792 shutdown_connections 2024-04-25T00:16:14.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.571+0000 7f70d4729640 1 --2- 172.21.15.107:0/2564648792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70cc102230 0x7f70cc19df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.571+0000 7f70d4729640 1 --2- 172.21.15.107:0/2564648792 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f70b80a2c10 0x7f70b80a50d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.571+0000 7f70d4729640 1 --2- 172.21.15.107:0/2564648792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70cc101870 0x7f70cc196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.571+0000 7f70d4729640 1 --2- 172.21.15.107:0/2564648792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70cc100680 0x7f70cc1969a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.571+0000 7f70d4729640 1 -- 172.21.15.107:0/2564648792 >> 172.21.15.107:0/2564648792 conn(0x7f70cc0fbbf0 msgr2=0x7f70cc104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:14.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.571+0000 7f70d4729640 1 -- 172.21.15.107:0/2564648792 shutdown_connections 2024-04-25T00:16:14.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.571+0000 7f70d4729640 1 -- 172.21.15.107:0/2564648792 wait complete. 2024-04-25T00:16:14.583 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:09:25.160329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:13:59.672801+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:14:00.957782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:14:05.065846+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:16:14.583 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:16:14.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.683+0000 7f4169c72640 1 -- 172.21.15.107:0/1899451893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4164102a20 msgr2=0x7f4164102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:14.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.683+0000 7f4169c72640 1 --2- 172.21.15.107:0/1899451893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4164102a20 0x7f4164102ea0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4158009930 tx=0x7f415802a420 comp rx=0 tx=0).stop 2024-04-25T00:16:14.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.683+0000 7f4169c72640 1 -- 172.21.15.107:0/1899451893 shutdown_connections 2024-04-25T00:16:14.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.683+0000 7f4169c72640 1 --2- 172.21.15.107:0/1899451893 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41641033e0 0x7f4164109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.683+0000 7f4169c72640 1 --2- 172.21.15.107:0/1899451893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4164102a20 0x7f4164102ea0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.683+0000 7f4169c72640 1 --2- 172.21.15.107:0/1899451893 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41640ffa50 0x7f41640ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.683+0000 7f4169c72640 1 -- 172.21.15.107:0/1899451893 >> 172.21.15.107:0/1899451893 conn(0x7f41640fb380 msgr2=0x7f41640fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:14.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.683+0000 7f4169c72640 1 -- 172.21.15.107:0/1899451893 shutdown_connections 2024-04-25T00:16:14.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.683+0000 7f4169c72640 1 -- 172.21.15.107:0/1899451893 wait complete. 2024-04-25T00:16:14.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.684+0000 7f4169c72640 1 Processor -- start 2024-04-25T00:16:14.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.684+0000 7f4169c72640 1 -- start start 2024-04-25T00:16:14.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.684+0000 7f4169c72640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41640ffa50 0x7f4164197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:14.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.684+0000 7f4169c72640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4164102a20 0x7f4164198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:14.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.684+0000 7f4169c72640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41641033e0 0x7f416419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:14.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.684+0000 7f4169c72640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f416410d430 con 0x7f4164102a20 2024-04-25T00:16:14.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.684+0000 7f4169c72640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f416410d2b0 con 0x7f41640ffa50 2024-04-25T00:16:14.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.684+0000 7f4169c72640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f416410d5b0 con 0x7f41641033e0 2024-04-25T00:16:14.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.684+0000 7f4162ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4164102a20 0x7f4164198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:14.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.684+0000 7f4162ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4164102a20 0x7f4164198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56890/0 (socket says 172.21.15.107:56890) 2024-04-25T00:16:14.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.684+0000 7f4162ffd640 1 -- 172.21.15.107:0/3361651096 learned_addr learned my addr 172.21.15.107:0/3361651096 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:14.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.684+0000 7f4163fff640 1 --2- 172.21.15.107:0/3361651096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41641033e0 0x7f416419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:14.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.685+0000 7f41637fe640 1 --2- 172.21.15.107:0/3361651096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41640ffa50 0x7f4164197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:14.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.685+0000 7f4162ffd640 1 -- 172.21.15.107:0/3361651096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41641033e0 msgr2=0x7f416419f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:14.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.685+0000 7f4162ffd640 1 --2- 172.21.15.107:0/3361651096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41641033e0 0x7f416419f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.685+0000 7f4162ffd640 1 -- 172.21.15.107:0/3361651096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41640ffa50 msgr2=0x7f4164197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:14.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.685+0000 7f4162ffd640 1 --2- 172.21.15.107:0/3361651096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41640ffa50 0x7f4164197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.685+0000 7f4162ffd640 1 -- 172.21.15.107:0/3361651096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416419f720 con 0x7f4164102a20 2024-04-25T00:16:14.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.685+0000 7f4162ffd640 1 --2- 172.21.15.107:0/3361651096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4164102a20 0x7f4164198040 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4158005ab0 tx=0x7f41580042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:14.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.685+0000 7f4160ff9640 1 -- 172.21.15.107:0/3361651096 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415803f030 con 0x7f4164102a20 2024-04-25T00:16:14.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.685+0000 7f4160ff9640 1 -- 172.21.15.107:0/3361651096 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41580044d0 con 0x7f4164102a20 2024-04-25T00:16:14.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.685+0000 7f4169c72640 1 -- 172.21.15.107:0/3361651096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416419f9b0 con 0x7f4164102a20 2024-04-25T00:16:14.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.685+0000 7f4160ff9640 1 -- 172.21.15.107:0/3361651096 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158035070 con 0x7f4164102a20 2024-04-25T00:16:14.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.685+0000 7f4169c72640 1 -- 172.21.15.107:0/3361651096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416419fea0 con 0x7f4164102a20 2024-04-25T00:16:14.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.686+0000 7f4169c72640 1 -- 172.21.15.107:0/3361651096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4164108ba0 con 0x7f4164102a20 2024-04-25T00:16:14.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.687+0000 7f4160ff9640 1 -- 172.21.15.107:0/3361651096 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f4158042030 con 0x7f4164102a20 2024-04-25T00:16:14.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.687+0000 7f4160ff9640 1 --2- 172.21.15.107:0/3361651096 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f414c0a2b90 0x7f414c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:14.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.688+0000 7f41637fe640 1 --2- 172.21.15.107:0/3361651096 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f414c0a2b90 0x7f414c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:14.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.688+0000 7f4160ff9640 1 -- 172.21.15.107:0/3361651096 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7f41580ef930 con 0x7f4164102a20 2024-04-25T00:16:14.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.688+0000 7f41637fe640 1 --2- 172.21.15.107:0/3361651096 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f414c0a2b90 0x7f414c0a5050 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4164069b80 tx=0x7f4154005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:14.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.692+0000 7f4160ff9640 1 -- 172.21.15.107:0/3361651096 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f41580bc500 con 0x7f4164102a20 2024-04-25T00:16:14.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.871+0000 7f4169c72640 1 -- 172.21.15.107:0/3361651096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f41640623e0 con 0x7f4164102a20 2024-04-25T00:16:14.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.871+0000 7f4160ff9640 1 -- 172.21.15.107:0/3361651096 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7f41580bbe00 con 0x7f4164102a20 2024-04-25T00:16:14.872 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-04-25T00:16:14.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.873+0000 7f4169c72640 1 -- 172.21.15.107:0/3361651096 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f414c0a2b90 msgr2=0x7f414c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:14.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.873+0000 7f4169c72640 1 --2- 172.21.15.107:0/3361651096 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f414c0a2b90 0x7f414c0a5050 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4164069b80 tx=0x7f4154005f90 comp rx=0 tx=0).stop 2024-04-25T00:16:14.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.873+0000 7f4169c72640 1 -- 172.21.15.107:0/3361651096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4164102a20 msgr2=0x7f4164198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:14.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.873+0000 7f4169c72640 1 --2- 172.21.15.107:0/3361651096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4164102a20 0x7f4164198040 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4158005ab0 tx=0x7f41580042e0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.873+0000 7f4169c72640 1 -- 172.21.15.107:0/3361651096 shutdown_connections 2024-04-25T00:16:14.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.873+0000 7f4169c72640 1 --2- 172.21.15.107:0/3361651096 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41641033e0 0x7f416419f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.873+0000 7f4169c72640 1 --2- 172.21.15.107:0/3361651096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4164102a20 0x7f4164198040 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.873+0000 7f4169c72640 1 --2- 172.21.15.107:0/3361651096 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f414c0a2b90 0x7f414c0a5050 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.873+0000 7f4169c72640 1 --2- 172.21.15.107:0/3361651096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41640ffa50 0x7f4164197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.873+0000 7f4169c72640 1 -- 172.21.15.107:0/3361651096 >> 172.21.15.107:0/3361651096 conn(0x7f41640fb380 msgr2=0x7f4164105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:14.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.874+0000 7f4169c72640 1 -- 172.21.15.107:0/3361651096 shutdown_connections 2024-04-25T00:16:14.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.874+0000 7f4169c72640 1 -- 172.21.15.107:0/3361651096 wait complete. 2024-04-25T00:16:14.885 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:16:14.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.987+0000 7f3e96615640 1 -- 172.21.15.107:0/296994842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e90104b30 msgr2=0x7f3e90106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:14.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.987+0000 7f3e96615640 1 --2- 172.21.15.107:0/296994842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e90104b30 0x7f3e90106f20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3e84009900 tx=0x7f3e8402a410 comp rx=0 tx=0).stop 2024-04-25T00:16:14.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.987+0000 7f3e96615640 1 -- 172.21.15.107:0/296994842 shutdown_connections 2024-04-25T00:16:14.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.987+0000 7f3e96615640 1 --2- 172.21.15.107:0/296994842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e90107460 0x7f3e901098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.987+0000 7f3e96615640 1 --2- 172.21.15.107:0/296994842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e90104b30 0x7f3e90106f20 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.987+0000 7f3e96615640 1 --2- 172.21.15.107:0/296994842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e90102200 0x7f3e901045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.987+0000 7f3e96615640 1 -- 172.21.15.107:0/296994842 >> 172.21.15.107:0/296994842 conn(0x7f3e900fbbf0 msgr2=0x7f3e900fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:14.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.987+0000 7f3e96615640 1 -- 172.21.15.107:0/296994842 shutdown_connections 2024-04-25T00:16:14.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.987+0000 7f3e96615640 1 -- 172.21.15.107:0/296994842 wait complete. 2024-04-25T00:16:14.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.987+0000 7f3e96615640 1 Processor -- start 2024-04-25T00:16:14.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.987+0000 7f3e96615640 1 -- start start 2024-04-25T00:16:14.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.988+0000 7f3e96615640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e90102200 0x7f3e90101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:14.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.988+0000 7f3e96615640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e90104b30 0x7f3e901001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:14.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.988+0000 7f3e96615640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e90107460 0x7f3e901006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:14.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.988+0000 7f3e96615640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9010d400 con 0x7f3e90104b30 2024-04-25T00:16:14.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.988+0000 7f3e96615640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9010d280 con 0x7f3e90107460 2024-04-25T00:16:14.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.988+0000 7f3e96615640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9010d580 con 0x7f3e90102200 2024-04-25T00:16:14.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.988+0000 7f3e8f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e90104b30 0x7f3e901001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:14.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.988+0000 7f3e8f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e90104b30 0x7f3e901001b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56904/0 (socket says 172.21.15.107:56904) 2024-04-25T00:16:14.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.988+0000 7f3e8f7fe640 1 -- 172.21.15.107:0/3176611692 learned_addr learned my addr 172.21.15.107:0/3176611692 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:14.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.988+0000 7f3e8ffff640 1 --2- 172.21.15.107:0/3176611692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e90102200 0x7f3e90101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:14.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.988+0000 7f3e94bb2640 1 --2- 172.21.15.107:0/3176611692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e90107460 0x7f3e901006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:14.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.988+0000 7f3e8f7fe640 1 -- 172.21.15.107:0/3176611692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e90102200 msgr2=0x7f3e90101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:14.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.988+0000 7f3e8f7fe640 1 --2- 172.21.15.107:0/3176611692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e90102200 0x7f3e90101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.988+0000 7f3e8f7fe640 1 -- 172.21.15.107:0/3176611692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e90107460 msgr2=0x7f3e901006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:14.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.988+0000 7f3e8f7fe640 1 --2- 172.21.15.107:0/3176611692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e90107460 0x7f3e901006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:14.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.988+0000 7f3e8f7fe640 1 -- 172.21.15.107:0/3176611692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e90100f80 con 0x7f3e90104b30 2024-04-25T00:16:14.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.988+0000 7f3e8f7fe640 1 --2- 172.21.15.107:0/3176611692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e90104b30 0x7f3e901001b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3e84031610 tx=0x7f3e840043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:14.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.988+0000 7f3e8d7fa640 1 -- 172.21.15.107:0/3176611692 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8403f030 con 0x7f3e90104b30 2024-04-25T00:16:14.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.988+0000 7f3e96615640 1 -- 172.21.15.107:0/3176611692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e901a1b70 con 0x7f3e90104b30 2024-04-25T00:16:14.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.988+0000 7f3e8d7fa640 1 -- 172.21.15.107:0/3176611692 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e84003ce0 con 0x7f3e90104b30 2024-04-25T00:16:14.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.988+0000 7f3e96615640 1 -- 172.21.15.107:0/3176611692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e901a2110 con 0x7f3e90104b30 2024-04-25T00:16:14.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.988+0000 7f3e8d7fa640 1 -- 172.21.15.107:0/3176611692 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e84049060 con 0x7f3e90104b30 2024-04-25T00:16:14.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.990+0000 7f3e96615640 1 -- 172.21.15.107:0/3176611692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e900fd6e0 con 0x7f3e90104b30 2024-04-25T00:16:14.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.990+0000 7f3e8d7fa640 1 -- 172.21.15.107:0/3176611692 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3e84042030 con 0x7f3e90104b30 2024-04-25T00:16:14.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.991+0000 7f3e8d7fa640 1 --2- 172.21.15.107:0/3176611692 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f3e780a2b90 0x7f3e780a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:14.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.991+0000 7f3e8ffff640 1 --2- 172.21.15.107:0/3176611692 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f3e780a2b90 0x7f3e780a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:14.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.991+0000 7f3e8d7fa640 1 -- 172.21.15.107:0/3176611692 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7f3e840efbe0 con 0x7f3e90104b30 2024-04-25T00:16:14.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.991+0000 7f3e8ffff640 1 --2- 172.21.15.107:0/3176611692 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f3e780a2b90 0x7f3e780a5050 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3e8000bbe0 tx=0x7f3e8000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:14.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:14.995+0000 7f3e8d7fa640 1 -- 172.21.15.107:0/3176611692 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3e840bc160 con 0x7f3e90104b30 2024-04-25T00:16:15.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.173+0000 7f3e96615640 1 -- 172.21.15.107:0/3176611692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3e90062340 con 0x7f3e90104b30 2024-04-25T00:16:15.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.174+0000 7f3e8d7fa640 1 -- 172.21.15.107:0/3176611692 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7f3e840c0460 con 0x7f3e90104b30 2024-04-25T00:16:15.174 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-25T00:16:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.176+0000 7f3e96615640 1 -- 172.21.15.107:0/3176611692 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f3e780a2b90 msgr2=0x7f3e780a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.176+0000 7f3e96615640 1 --2- 172.21.15.107:0/3176611692 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f3e780a2b90 0x7f3e780a5050 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3e8000bbe0 tx=0x7f3e8000b390 comp rx=0 tx=0).stop 2024-04-25T00:16:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.176+0000 7f3e96615640 1 -- 172.21.15.107:0/3176611692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e90104b30 msgr2=0x7f3e901001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.176+0000 7f3e96615640 1 --2- 172.21.15.107:0/3176611692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e90104b30 0x7f3e901001b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3e84031610 tx=0x7f3e840043b0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.176+0000 7f3e96615640 1 -- 172.21.15.107:0/3176611692 shutdown_connections 2024-04-25T00:16:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.176+0000 7f3e96615640 1 --2- 172.21.15.107:0/3176611692 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f3e780a2b90 0x7f3e780a5050 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.176+0000 7f3e96615640 1 --2- 172.21.15.107:0/3176611692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3e90107460 0x7f3e901006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.176+0000 7f3e96615640 1 --2- 172.21.15.107:0/3176611692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e90104b30 0x7f3e901001b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.176+0000 7f3e96615640 1 --2- 172.21.15.107:0/3176611692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e90102200 0x7f3e90101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.176+0000 7f3e96615640 1 -- 172.21.15.107:0/3176611692 >> 172.21.15.107:0/3176611692 conn(0x7f3e900fbbf0 msgr2=0x7f3e90102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.176+0000 7f3e96615640 1 -- 172.21.15.107:0/3176611692 shutdown_connections 2024-04-25T00:16:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.176+0000 7f3e96615640 1 -- 172.21.15.107:0/3176611692 wait complete. 2024-04-25T00:16:15.186 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:15 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2564648792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:15.186 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:15 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3361651096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:16:15.188 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:16:15.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.289+0000 7fde3437e640 1 -- 172.21.15.107:0/796732758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde2c100680 msgr2=0x7fde2c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:15.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.289+0000 7fde3437e640 1 --2- 172.21.15.107:0/796732758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde2c100680 0x7fde2c100aa0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fde28009ca0 tx=0x7fde2802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.289+0000 7fde3437e640 1 -- 172.21.15.107:0/796732758 shutdown_connections 2024-04-25T00:16:15.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.289+0000 7fde3437e640 1 --2- 172.21.15.107:0/796732758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde2c102230 0x7fde2c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.289+0000 7fde3437e640 1 --2- 172.21.15.107:0/796732758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde2c101870 0x7fde2c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.289+0000 7fde3437e640 1 --2- 172.21.15.107:0/796732758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde2c100680 0x7fde2c100aa0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.289+0000 7fde3437e640 1 -- 172.21.15.107:0/796732758 >> 172.21.15.107:0/796732758 conn(0x7fde2c0fbc30 msgr2=0x7fde2c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:15.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.289+0000 7fde3437e640 1 -- 172.21.15.107:0/796732758 shutdown_connections 2024-04-25T00:16:15.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.289+0000 7fde3437e640 1 -- 172.21.15.107:0/796732758 wait complete. 2024-04-25T00:16:15.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.290+0000 7fde3437e640 1 Processor -- start 2024-04-25T00:16:15.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.290+0000 7fde3437e640 1 -- start start 2024-04-25T00:16:15.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.290+0000 7fde3437e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde2c100680 0x7fde2c197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:15.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.290+0000 7fde3437e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde2c101870 0x7fde2c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:15.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.290+0000 7fde3437e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde2c102230 0x7fde2c19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:15.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.290+0000 7fde3437e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fde2c10d450 con 0x7fde2c102230 2024-04-25T00:16:15.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.290+0000 7fde3437e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fde2c10d2d0 con 0x7fde2c100680 2024-04-25T00:16:15.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.290+0000 7fde3437e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fde2c10d5d0 con 0x7fde2c101870 2024-04-25T00:16:15.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.290+0000 7fde3291b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde2c102230 0x7fde2c19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:15.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.290+0000 7fde3291b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde2c102230 0x7fde2c19f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56916/0 (socket says 172.21.15.107:56916) 2024-04-25T00:16:15.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.290+0000 7fde3291b640 1 -- 172.21.15.107:0/3867009876 learned_addr learned my addr 172.21.15.107:0/3867009876 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:15.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.290+0000 7fde31919640 1 --2- 172.21.15.107:0/3867009876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde2c101870 0x7fde2c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:15.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.290+0000 7fde3211a640 1 --2- 172.21.15.107:0/3867009876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde2c100680 0x7fde2c197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:15.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.290+0000 7fde3291b640 1 -- 172.21.15.107:0/3867009876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde2c101870 msgr2=0x7fde2c198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:15.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.290+0000 7fde3291b640 1 --2- 172.21.15.107:0/3867009876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde2c101870 0x7fde2c198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.290+0000 7fde3291b640 1 -- 172.21.15.107:0/3867009876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde2c100680 msgr2=0x7fde2c197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:15.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.290+0000 7fde3291b640 1 --2- 172.21.15.107:0/3867009876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde2c100680 0x7fde2c197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.290+0000 7fde3291b640 1 -- 172.21.15.107:0/3867009876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2c19f740 con 0x7fde2c102230 2024-04-25T00:16:15.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.290+0000 7fde31919640 1 --2- 172.21.15.107:0/3867009876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde2c101870 0x7fde2c198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:15.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.291+0000 7fde3291b640 1 --2- 172.21.15.107:0/3867009876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde2c102230 0x7fde2c19f0d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fde24009fd0 tx=0x7fde2400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:15.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.291+0000 7fde237fe640 1 -- 172.21.15.107:0/3867009876 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde24011830 con 0x7fde2c102230 2024-04-25T00:16:15.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.291+0000 7fde237fe640 1 -- 172.21.15.107:0/3867009876 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde24016070 con 0x7fde2c102230 2024-04-25T00:16:15.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.291+0000 7fde3437e640 1 -- 172.21.15.107:0/3867009876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde2c19fa30 con 0x7fde2c102230 2024-04-25T00:16:15.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.291+0000 7fde237fe640 1 -- 172.21.15.107:0/3867009876 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2400f2e0 con 0x7fde2c102230 2024-04-25T00:16:15.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.291+0000 7fde3437e640 1 -- 172.21.15.107:0/3867009876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde2c19ffb0 con 0x7fde2c102230 2024-04-25T00:16:15.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.292+0000 7fde3437e640 1 -- 172.21.15.107:0/3867009876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde2c107a40 con 0x7fde2c102230 2024-04-25T00:16:15.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.293+0000 7fde237fe640 1 -- 172.21.15.107:0/3867009876 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fde240119d0 con 0x7fde2c102230 2024-04-25T00:16:15.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.293+0000 7fde237fe640 1 --2- 172.21.15.107:0/3867009876 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fde180a2b90 0x7fde180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:15.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.293+0000 7fde3211a640 1 --2- 172.21.15.107:0/3867009876 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fde180a2b90 0x7fde180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:15.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.294+0000 7fde237fe640 1 -- 172.21.15.107:0/3867009876 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7fde240ceb00 con 0x7fde2c102230 2024-04-25T00:16:15.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.294+0000 7fde3211a640 1 --2- 172.21.15.107:0/3867009876 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fde180a2b90 0x7fde180a5050 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fde28006010 tx=0x7fde28009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:15.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.298+0000 7fde237fe640 1 -- 172.21.15.107:0/3867009876 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fde2409b6d0 con 0x7fde2c102230 2024-04-25T00:16:15.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:15 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2564648792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:15.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:15 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3361651096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:16:15.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:15 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2564648792' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:15.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:15 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3361651096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:16:15.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.476+0000 7fde3437e640 1 -- 172.21.15.107:0/3867009876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fde2c0623e0 con 0x7fde2c102230 2024-04-25T00:16:15.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.477+0000 7fde237fe640 1 -- 172.21.15.107:0/3867009876 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7fde2409afd0 con 0x7fde2c102230 2024-04-25T00:16:15.477 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-04-25T00:16:15.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.479+0000 7fde3437e640 1 -- 172.21.15.107:0/3867009876 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fde180a2b90 msgr2=0x7fde180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:15.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.479+0000 7fde3437e640 1 --2- 172.21.15.107:0/3867009876 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fde180a2b90 0x7fde180a5050 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fde28006010 tx=0x7fde28009360 comp rx=0 tx=0).stop 2024-04-25T00:16:15.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.479+0000 7fde3437e640 1 -- 172.21.15.107:0/3867009876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde2c102230 msgr2=0x7fde2c19f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:15.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.479+0000 7fde3437e640 1 --2- 172.21.15.107:0/3867009876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde2c102230 0x7fde2c19f0d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fde24009fd0 tx=0x7fde2400b8e0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.479+0000 7fde3437e640 1 -- 172.21.15.107:0/3867009876 shutdown_connections 2024-04-25T00:16:15.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.479+0000 7fde3437e640 1 --2- 172.21.15.107:0/3867009876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde2c102230 0x7fde2c19f0d0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.479+0000 7fde3437e640 1 --2- 172.21.15.107:0/3867009876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fde2c101870 0x7fde2c198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.479+0000 7fde3437e640 1 --2- 172.21.15.107:0/3867009876 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fde180a2b90 0x7fde180a5050 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.479+0000 7fde3437e640 1 --2- 172.21.15.107:0/3867009876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fde2c100680 0x7fde2c197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.479+0000 7fde3437e640 1 -- 172.21.15.107:0/3867009876 >> 172.21.15.107:0/3867009876 conn(0x7fde2c0fbc30 msgr2=0x7fde2c104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:15.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.479+0000 7fde3437e640 1 -- 172.21.15.107:0/3867009876 shutdown_connections 2024-04-25T00:16:15.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.479+0000 7fde3437e640 1 -- 172.21.15.107:0/3867009876 wait complete. 2024-04-25T00:16:15.490 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:16:15.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.591+0000 7fc6e6b9e640 1 -- 172.21.15.107:0/2457064019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6e0101870 msgr2=0x7fc6e0101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:15.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.591+0000 7fc6e6b9e640 1 --2- 172.21.15.107:0/2457064019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6e0101870 0x7fc6e0101cf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0009900 tx=0x7fc6d002a410 comp rx=0 tx=0).stop 2024-04-25T00:16:15.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.591+0000 7fc6e6b9e640 1 -- 172.21.15.107:0/2457064019 shutdown_connections 2024-04-25T00:16:15.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.591+0000 7fc6e6b9e640 1 --2- 172.21.15.107:0/2457064019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6e0102230 0x7fc6e0108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.591+0000 7fc6e6b9e640 1 --2- 172.21.15.107:0/2457064019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6e0101870 0x7fc6e0101cf0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.591+0000 7fc6e6b9e640 1 --2- 172.21.15.107:0/2457064019 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6e0100680 0x7fc6e0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.591+0000 7fc6e6b9e640 1 -- 172.21.15.107:0/2457064019 >> 172.21.15.107:0/2457064019 conn(0x7fc6e00fbc30 msgr2=0x7fc6e00fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:15.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.591+0000 7fc6e6b9e640 1 -- 172.21.15.107:0/2457064019 shutdown_connections 2024-04-25T00:16:15.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.591+0000 7fc6e6b9e640 1 -- 172.21.15.107:0/2457064019 wait complete. 2024-04-25T00:16:15.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.591+0000 7fc6e6b9e640 1 Processor -- start 2024-04-25T00:16:15.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.592+0000 7fc6e6b9e640 1 -- start start 2024-04-25T00:16:15.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.592+0000 7fc6e6b9e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6e0100680 0x7fc6e006a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:15.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.592+0000 7fc6e6b9e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6e0101870 0x7fc6e019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:15.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.592+0000 7fc6e6b9e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6e0102230 0x7fc6e019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:15.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.592+0000 7fc6e6b9e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e010c3f0 con 0x7fc6e0100680 2024-04-25T00:16:15.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.592+0000 7fc6e6b9e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e010c270 con 0x7fc6e0102230 2024-04-25T00:16:15.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.592+0000 7fc6e6b9e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e010c570 con 0x7fc6e0101870 2024-04-25T00:16:15.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.592+0000 7fc6e493a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6e0100680 0x7fc6e006a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:15.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.592+0000 7fc6e493a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6e0100680 0x7fc6e006a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56928/0 (socket says 172.21.15.107:56928) 2024-04-25T00:16:15.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.592+0000 7fc6e493a640 1 -- 172.21.15.107:0/995363561 learned_addr learned my addr 172.21.15.107:0/995363561 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:15.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.592+0000 7fc6d7fff640 1 --2- 172.21.15.107:0/995363561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6e0101870 0x7fc6e019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:15.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.592+0000 7fc6e493a640 1 -- 172.21.15.107:0/995363561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6e0101870 msgr2=0x7fc6e019c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:15.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.592+0000 7fc6e513b640 1 --2- 172.21.15.107:0/995363561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6e0102230 0x7fc6e019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:15.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.592+0000 7fc6e493a640 1 --2- 172.21.15.107:0/995363561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6e0101870 0x7fc6e019c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.592+0000 7fc6e493a640 1 -- 172.21.15.107:0/995363561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6e0102230 msgr2=0x7fc6e019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:15.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.592+0000 7fc6e493a640 1 --2- 172.21.15.107:0/995363561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6e0102230 0x7fc6e019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.592+0000 7fc6e493a640 1 -- 172.21.15.107:0/995363561 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e019f880 con 0x7fc6e0100680 2024-04-25T00:16:15.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.592+0000 7fc6e513b640 1 --2- 172.21.15.107:0/995363561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6e0102230 0x7fc6e019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:15.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.592+0000 7fc6e493a640 1 --2- 172.21.15.107:0/995363561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6e0100680 0x7fc6e006a910 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc00b450 tx=0x7fc6dc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:15.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.592+0000 7fc6d5ffb640 1 -- 172.21.15.107:0/995363561 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc015020 con 0x7fc6e0100680 2024-04-25T00:16:15.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.592+0000 7fc6d5ffb640 1 -- 172.21.15.107:0/995363561 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6dc011d70 con 0x7fc6e0100680 2024-04-25T00:16:15.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.592+0000 7fc6e6b9e640 1 -- 172.21.15.107:0/995363561 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e019fb70 con 0x7fc6e0100680 2024-04-25T00:16:15.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.592+0000 7fc6d5ffb640 1 -- 172.21.15.107:0/995363561 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc00f340 con 0x7fc6e0100680 2024-04-25T00:16:15.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.593+0000 7fc6e6b9e640 1 -- 172.21.15.107:0/995363561 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e01a0070 con 0x7fc6e0100680 2024-04-25T00:16:15.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.593+0000 7fc6e6b9e640 1 -- 172.21.15.107:0/995363561 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6e0107ac0 con 0x7fc6e0100680 2024-04-25T00:16:15.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.594+0000 7fc6d5ffb640 1 -- 172.21.15.107:0/995363561 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fc6dc018020 con 0x7fc6e0100680 2024-04-25T00:16:15.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.595+0000 7fc6d5ffb640 1 --2- 172.21.15.107:0/995363561 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc6c80a2b90 0x7fc6c80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:15.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.595+0000 7fc6d7fff640 1 --2- 172.21.15.107:0/995363561 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc6c80a2b90 0x7fc6c80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:15.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.595+0000 7fc6d5ffb640 1 -- 172.21.15.107:0/995363561 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7fc6dc0cef70 con 0x7fc6e0100680 2024-04-25T00:16:15.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.595+0000 7fc6d7fff640 1 --2- 172.21.15.107:0/995363561 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc6c80a2b90 0x7fc6c80a5050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0031610 tx=0x7fc6d0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:15.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.599+0000 7fc6d5ffb640 1 -- 172.21.15.107:0/995363561 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc6dc01f020 con 0x7fc6e0100680 2024-04-25T00:16:15.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.773+0000 7fc6e6b9e640 1 -- 172.21.15.107:0/995363561 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc6e0062370 con 0x7fc6e0100680 2024-04-25T00:16:15.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.774+0000 7fc6d5ffb640 1 -- 172.21.15.107:0/995363561 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v96) ==== 103+0+11 (secure 0 0 0) 0x7fc6dc00d040 con 0x7fc6e0100680 2024-04-25T00:16:15.774 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-25T00:16:15.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.775+0000 7fc6e6b9e640 1 -- 172.21.15.107:0/995363561 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc6c80a2b90 msgr2=0x7fc6c80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:15.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.775+0000 7fc6e6b9e640 1 --2- 172.21.15.107:0/995363561 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc6c80a2b90 0x7fc6c80a5050 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0031610 tx=0x7fc6d0000bc0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.775+0000 7fc6e6b9e640 1 -- 172.21.15.107:0/995363561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6e0100680 msgr2=0x7fc6e006a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:15.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.775+0000 7fc6e6b9e640 1 --2- 172.21.15.107:0/995363561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6e0100680 0x7fc6e006a910 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc00b450 tx=0x7fc6dc00b910 comp rx=0 tx=0).stop 2024-04-25T00:16:15.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.776+0000 7fc6e6b9e640 1 -- 172.21.15.107:0/995363561 shutdown_connections 2024-04-25T00:16:15.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.776+0000 7fc6e6b9e640 1 --2- 172.21.15.107:0/995363561 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc6c80a2b90 0x7fc6c80a5050 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.776+0000 7fc6e6b9e640 1 --2- 172.21.15.107:0/995363561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6e0102230 0x7fc6e019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.776+0000 7fc6e6b9e640 1 --2- 172.21.15.107:0/995363561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc6e0101870 0x7fc6e019c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.776+0000 7fc6e6b9e640 1 --2- 172.21.15.107:0/995363561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6e0100680 0x7fc6e006a910 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.776+0000 7fc6e6b9e640 1 -- 172.21.15.107:0/995363561 >> 172.21.15.107:0/995363561 conn(0x7fc6e00fbc30 msgr2=0x7fc6e0104c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:15.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.776+0000 7fc6e6b9e640 1 -- 172.21.15.107:0/995363561 shutdown_connections 2024-04-25T00:16:15.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.776+0000 7fc6e6b9e640 1 -- 172.21.15.107:0/995363561 wait complete. 2024-04-25T00:16:15.788 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:16:15.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.886+0000 7f827ff9d640 1 -- 172.21.15.107:0/2053898939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8278102200 msgr2=0x7f82781045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:15.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.886+0000 7f827ff9d640 1 --2- 172.21.15.107:0/2053898939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8278102200 0x7f82781045f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8274009900 tx=0x7f827402a410 comp rx=0 tx=0).stop 2024-04-25T00:16:15.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.886+0000 7f827ff9d640 1 -- 172.21.15.107:0/2053898939 shutdown_connections 2024-04-25T00:16:15.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.886+0000 7f827ff9d640 1 --2- 172.21.15.107:0/2053898939 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8278107460 0x7f82781098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.886+0000 7f827ff9d640 1 --2- 172.21.15.107:0/2053898939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8278104b30 0x7f8278106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.886+0000 7f827ff9d640 1 --2- 172.21.15.107:0/2053898939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8278102200 0x7f82781045f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.886+0000 7f827ff9d640 1 -- 172.21.15.107:0/2053898939 >> 172.21.15.107:0/2053898939 conn(0x7f82780fbbf0 msgr2=0x7f82780fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:15.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.886+0000 7f827ff9d640 1 -- 172.21.15.107:0/2053898939 shutdown_connections 2024-04-25T00:16:15.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.886+0000 7f827ff9d640 1 -- 172.21.15.107:0/2053898939 wait complete. 2024-04-25T00:16:15.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.887+0000 7f827ff9d640 1 Processor -- start 2024-04-25T00:16:15.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.887+0000 7f827ff9d640 1 -- start start 2024-04-25T00:16:15.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.887+0000 7f827ff9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8278102200 0x7f8278197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:15.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.887+0000 7f827ff9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8278104b30 0x7f8278197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:15.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.887+0000 7f827ff9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8278107460 0x7f827819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:15.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.887+0000 7f827ff9d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f827810d420 con 0x7f8278104b30 2024-04-25T00:16:15.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.887+0000 7f827ff9d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f827810d2a0 con 0x7f8278107460 2024-04-25T00:16:15.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.887+0000 7f827ff9d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f827810d5a0 con 0x7f8278102200 2024-04-25T00:16:15.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.887+0000 7f827d538640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8278104b30 0x7f8278197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:15.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.887+0000 7f827d538640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8278104b30 0x7f8278197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56950/0 (socket says 172.21.15.107:56950) 2024-04-25T00:16:15.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.887+0000 7f827d538640 1 -- 172.21.15.107:0/2678928639 learned_addr learned my addr 172.21.15.107:0/2678928639 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:15.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.887+0000 7f827e53a640 1 --2- 172.21.15.107:0/2678928639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8278107460 0x7f827819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:15.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.887+0000 7f827dd39640 1 --2- 172.21.15.107:0/2678928639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8278102200 0x7f8278197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:15.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.887+0000 7f827d538640 1 -- 172.21.15.107:0/2678928639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8278102200 msgr2=0x7f8278197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:15.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.887+0000 7f827d538640 1 --2- 172.21.15.107:0/2678928639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8278102200 0x7f8278197aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.887+0000 7f827d538640 1 -- 172.21.15.107:0/2678928639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8278107460 msgr2=0x7f827819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:15.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.887+0000 7f827d538640 1 --2- 172.21.15.107:0/2678928639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8278107460 0x7f827819f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.887+0000 7f827d538640 1 -- 172.21.15.107:0/2678928639 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827819f6c0 con 0x7f8278104b30 2024-04-25T00:16:15.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.887+0000 7f827d538640 1 --2- 172.21.15.107:0/2678928639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8278104b30 0x7f8278197fe0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8268005fd0 tx=0x7f826800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:15.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.888+0000 7f826effd640 1 -- 172.21.15.107:0/2678928639 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8268017070 con 0x7f8278104b30 2024-04-25T00:16:15.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.888+0000 7f826effd640 1 -- 172.21.15.107:0/2678928639 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8268009c30 con 0x7f8278104b30 2024-04-25T00:16:15.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.888+0000 7f827ff9d640 1 -- 172.21.15.107:0/2678928639 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827819f9b0 con 0x7f8278104b30 2024-04-25T00:16:15.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.888+0000 7f826effd640 1 -- 172.21.15.107:0/2678928639 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8268010460 con 0x7f8278104b30 2024-04-25T00:16:15.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.888+0000 7f827ff9d640 1 -- 172.21.15.107:0/2678928639 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f827819ff50 con 0x7f8278104b30 2024-04-25T00:16:15.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.889+0000 7f827ff9d640 1 -- 172.21.15.107:0/2678928639 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82780fd6e0 con 0x7f8278104b30 2024-04-25T00:16:15.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.890+0000 7f826effd640 1 -- 172.21.15.107:0/2678928639 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8268003e70 con 0x7f8278104b30 2024-04-25T00:16:15.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.890+0000 7f826effd640 1 --2- 172.21.15.107:0/2678928639 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f82640a2c10 0x7f82640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:15.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.890+0000 7f827dd39640 1 --2- 172.21.15.107:0/2678928639 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f82640a2c10 0x7f82640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:15.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.890+0000 7f826effd640 1 -- 172.21.15.107:0/2678928639 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7f82680cf120 con 0x7f8278104b30 2024-04-25T00:16:15.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.891+0000 7f827dd39640 1 --2- 172.21.15.107:0/2678928639 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f82640a2c10 0x7f82640a50d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8274031610 tx=0x7f8274000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:15.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:15.895+0000 7f826effd640 1 -- 172.21.15.107:0/2678928639 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8268020030 con 0x7f8278104b30 2024-04-25T00:16:16.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.109+0000 7f827ff9d640 1 -- 172.21.15.107:0/2678928639 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f827802d170 con 0x7f8278104b30 2024-04-25T00:16:16.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.110+0000 7f826effd640 1 -- 172.21.15.107:0/2678928639 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11897 (secure 0 0 0) 0x7f826809bcf0 con 0x7f8278104b30 2024-04-25T00:16:16.110 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:16:16.110 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:14:28.529865+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24521,24493]},"id":1}]} 2024-04-25T00:16:16.111 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 23 2024-04-25T00:16:16.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.112+0000 7f827ff9d640 1 -- 172.21.15.107:0/2678928639 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f82640a2c10 msgr2=0x7f82640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.112+0000 7f827ff9d640 1 --2- 172.21.15.107:0/2678928639 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f82640a2c10 0x7f82640a50d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8274031610 tx=0x7f8274000bc0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.112+0000 7f827ff9d640 1 -- 172.21.15.107:0/2678928639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8278104b30 msgr2=0x7f8278197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.112+0000 7f827ff9d640 1 --2- 172.21.15.107:0/2678928639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8278104b30 0x7f8278197fe0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8268005fd0 tx=0x7f826800dde0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.112+0000 7f827ff9d640 1 -- 172.21.15.107:0/2678928639 shutdown_connections 2024-04-25T00:16:16.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.112+0000 7f827ff9d640 1 --2- 172.21.15.107:0/2678928639 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f82640a2c10 0x7f82640a50d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.112+0000 7f827ff9d640 1 --2- 172.21.15.107:0/2678928639 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8278107460 0x7f827819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.112+0000 7f827ff9d640 1 --2- 172.21.15.107:0/2678928639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8278104b30 0x7f8278197fe0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.112+0000 7f827ff9d640 1 --2- 172.21.15.107:0/2678928639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8278102200 0x7f8278197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.112+0000 7f827ff9d640 1 -- 172.21.15.107:0/2678928639 >> 172.21.15.107:0/2678928639 conn(0x7f82780fbbf0 msgr2=0x7f8278102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:16.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.112+0000 7f827ff9d640 1 -- 172.21.15.107:0/2678928639 shutdown_connections 2024-04-25T00:16:16.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.112+0000 7f827ff9d640 1 -- 172.21.15.107:0/2678928639 wait complete. 2024-04-25T00:16:16.124 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:16:16.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.222+0000 7fcec54a5640 1 -- 172.21.15.107:0/2637793307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcec01018b0 msgr2=0x7fcec0101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.222+0000 7fcec54a5640 1 --2- 172.21.15.107:0/2637793307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcec01018b0 0x7fcec0101d30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009900 tx=0x7fcea802a410 comp rx=0 tx=0).stop 2024-04-25T00:16:16.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.222+0000 7fcec54a5640 1 -- 172.21.15.107:0/2637793307 shutdown_connections 2024-04-25T00:16:16.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.222+0000 7fcec54a5640 1 --2- 172.21.15.107:0/2637793307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcec0102270 0x7fcec0108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.222+0000 7fcec54a5640 1 --2- 172.21.15.107:0/2637793307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcec01018b0 0x7fcec0101d30 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.222+0000 7fcec54a5640 1 --2- 172.21.15.107:0/2637793307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcec01006c0 0x7fcec0100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.222+0000 7fcec54a5640 1 -- 172.21.15.107:0/2637793307 >> 172.21.15.107:0/2637793307 conn(0x7fcec00fbc30 msgr2=0x7fcec00fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:16.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.222+0000 7fcec54a5640 1 -- 172.21.15.107:0/2637793307 shutdown_connections 2024-04-25T00:16:16.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.222+0000 7fcec54a5640 1 -- 172.21.15.107:0/2637793307 wait complete. 2024-04-25T00:16:16.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.223+0000 7fcec54a5640 1 Processor -- start 2024-04-25T00:16:16.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.223+0000 7fcec54a5640 1 -- start start 2024-04-25T00:16:16.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.223+0000 7fcec54a5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcec01006c0 0x7fcec0197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:16.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.223+0000 7fcec54a5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcec01018b0 0x7fcec0198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:16.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.223+0000 7fcec54a5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcec0102270 0x7fcec019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:16.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.223+0000 7fcec54a5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcec010d460 con 0x7fcec01006c0 2024-04-25T00:16:16.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.223+0000 7fcec54a5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcec010d2e0 con 0x7fcec0102270 2024-04-25T00:16:16.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.223+0000 7fcec54a5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcec010d5e0 con 0x7fcec01018b0 2024-04-25T00:16:16.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.223+0000 7fcebeffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcec01006c0 0x7fcec0197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:16.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.223+0000 7fcebeffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcec01006c0 0x7fcec0197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56974/0 (socket says 172.21.15.107:56974) 2024-04-25T00:16:16.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.223+0000 7fcebeffd640 1 -- 172.21.15.107:0/2332178551 learned_addr learned my addr 172.21.15.107:0/2332178551 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:16.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.223+0000 7fcebf7fe640 1 --2- 172.21.15.107:0/2332178551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcec0102270 0x7fcec019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:16.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.223+0000 7fceb7fff640 1 --2- 172.21.15.107:0/2332178551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcec01018b0 0x7fcec0198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:16.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.223+0000 7fcebeffd640 1 -- 172.21.15.107:0/2332178551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcec01018b0 msgr2=0x7fcec0198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.223+0000 7fcebeffd640 1 --2- 172.21.15.107:0/2332178551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcec01018b0 0x7fcec0198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.223+0000 7fcebeffd640 1 -- 172.21.15.107:0/2332178551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcec0102270 msgr2=0x7fcec019f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.223+0000 7fcebeffd640 1 --2- 172.21.15.107:0/2332178551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcec0102270 0x7fcec019f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.223+0000 7fcebeffd640 1 -- 172.21.15.107:0/2332178551 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec019f730 con 0x7fcec01006c0 2024-04-25T00:16:16.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.223+0000 7fcebf7fe640 1 --2- 172.21.15.107:0/2332178551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcec0102270 0x7fcec019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:16.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.223+0000 7fcebeffd640 1 --2- 172.21.15.107:0/2332178551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcec01006c0 0x7fcec0197b10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fceb000c970 tx=0x7fceb000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:16.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.223+0000 7fcebcff9640 1 -- 172.21.15.107:0/2332178551 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb0004220 con 0x7fcec01006c0 2024-04-25T00:16:16.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.224+0000 7fcebcff9640 1 -- 172.21.15.107:0/2332178551 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fceb00043c0 con 0x7fcec01006c0 2024-04-25T00:16:16.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.224+0000 7fcec54a5640 1 -- 172.21.15.107:0/2332178551 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec019fa20 con 0x7fcec01006c0 2024-04-25T00:16:16.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.224+0000 7fcebcff9640 1 -- 172.21.15.107:0/2332178551 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb000f450 con 0x7fcec01006c0 2024-04-25T00:16:16.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.224+0000 7fcec54a5640 1 -- 172.21.15.107:0/2332178551 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec019ff90 con 0x7fcec01006c0 2024-04-25T00:16:16.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.224+0000 7fcec54a5640 1 -- 172.21.15.107:0/2332178551 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcec0107a50 con 0x7fcec01006c0 2024-04-25T00:16:16.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.226+0000 7fcebcff9640 1 -- 172.21.15.107:0/2332178551 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fceb00079b0 con 0x7fcec01006c0 2024-04-25T00:16:16.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.227+0000 7fcebcff9640 1 --2- 172.21.15.107:0/2332178551 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fceac0a2c10 0x7fceac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:16.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.227+0000 7fcebcff9640 1 -- 172.21.15.107:0/2332178551 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7fceb00cf4a0 con 0x7fcec01006c0 2024-04-25T00:16:16.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.227+0000 7fceb7fff640 1 --2- 172.21.15.107:0/2332178551 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fceac0a2c10 0x7fceac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:16.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.227+0000 7fceb7fff640 1 --2- 172.21.15.107:0/2332178551 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fceac0a2c10 0x7fceac0a50d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcea8000c00 tx=0x7fcea80315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:16.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.230+0000 7fcebcff9640 1 -- 172.21.15.107:0/2332178551 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fceb0014030 con 0x7fcec01006c0 2024-04-25T00:16:16.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:16 smithi195 ceph-mon[32833]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:16:16.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:16 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3176611692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:16:16.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:16 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3867009876' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:16:16.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:16 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/995363561' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:16:16.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.406+0000 7fcec54a5640 1 -- 172.21.15.107:0/2332178551 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcec002d310 con 0x7fcec01006c0 2024-04-25T00:16:16.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.408+0000 7fcebcff9640 1 -- 172.21.15.107:0/2332178551 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24688 (secure 0 0 0) 0x7fceb009c0f0 con 0x7fcec01006c0 2024-04-25T00:16:16.409 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:16:16.409 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":96,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:16:03.045295+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:59.672801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:00.957782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:14:05.065846+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8},{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:16:16.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.410+0000 7fcec54a5640 1 -- 172.21.15.107:0/2332178551 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fceac0a2c10 msgr2=0x7fceac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.410+0000 7fcec54a5640 1 --2- 172.21.15.107:0/2332178551 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fceac0a2c10 0x7fceac0a50d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcea8000c00 tx=0x7fcea80315b0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.410+0000 7fcec54a5640 1 -- 172.21.15.107:0/2332178551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcec01006c0 msgr2=0x7fcec0197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.410+0000 7fcec54a5640 1 --2- 172.21.15.107:0/2332178551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcec01006c0 0x7fcec0197b10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fceb000c970 tx=0x7fceb000ce30 comp rx=0 tx=0).stop 2024-04-25T00:16:16.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.410+0000 7fcec54a5640 1 -- 172.21.15.107:0/2332178551 shutdown_connections 2024-04-25T00:16:16.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.410+0000 7fcec54a5640 1 --2- 172.21.15.107:0/2332178551 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fceac0a2c10 0x7fceac0a50d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.410+0000 7fcec54a5640 1 --2- 172.21.15.107:0/2332178551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcec0102270 0x7fcec019f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.410+0000 7fcec54a5640 1 --2- 172.21.15.107:0/2332178551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcec01018b0 0x7fcec0198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.410+0000 7fcec54a5640 1 --2- 172.21.15.107:0/2332178551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcec01006c0 0x7fcec0197b10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.410+0000 7fcec54a5640 1 -- 172.21.15.107:0/2332178551 >> 172.21.15.107:0/2332178551 conn(0x7fcec00fbc30 msgr2=0x7fcec0104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:16.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.410+0000 7fcec54a5640 1 -- 172.21.15.107:0/2332178551 shutdown_connections 2024-04-25T00:16:16.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.410+0000 7fcec54a5640 1 -- 172.21.15.107:0/2332178551 wait complete. 2024-04-25T00:16:16.420 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:16 smithi107 ceph-mon[29153]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:16:16.420 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:16 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3176611692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:16:16.421 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:16 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3867009876' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:16:16.421 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:16 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/995363561' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:16:16.422 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:09:25.160329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:13:59.672801+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:14:00.957782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:14:05.065846+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:16:16.422 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:16:16.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:16 smithi170 ceph-mon[32807]: pgmap v363: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:16:16.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:16 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3176611692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:16:16.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:16 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3867009876' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:16:16.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:16 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/995363561' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:16:16.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.523+0000 7efe8f7b7640 1 -- 172.21.15.107:0/4102384342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe88100680 msgr2=0x7efe88100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.523+0000 7efe8f7b7640 1 --2- 172.21.15.107:0/4102384342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe88100680 0x7efe88100aa0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efe84009900 tx=0x7efe8402a410 comp rx=0 tx=0).stop 2024-04-25T00:16:16.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.523+0000 7efe8f7b7640 1 -- 172.21.15.107:0/4102384342 shutdown_connections 2024-04-25T00:16:16.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.523+0000 7efe8f7b7640 1 --2- 172.21.15.107:0/4102384342 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe88102230 0x7efe88108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.523+0000 7efe8f7b7640 1 --2- 172.21.15.107:0/4102384342 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe88101870 0x7efe88101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.523+0000 7efe8f7b7640 1 --2- 172.21.15.107:0/4102384342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe88100680 0x7efe88100aa0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.523+0000 7efe8f7b7640 1 -- 172.21.15.107:0/4102384342 >> 172.21.15.107:0/4102384342 conn(0x7efe880fbbf0 msgr2=0x7efe880fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:16.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.524+0000 7efe8f7b7640 1 -- 172.21.15.107:0/4102384342 shutdown_connections 2024-04-25T00:16:16.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.524+0000 7efe8f7b7640 1 -- 172.21.15.107:0/4102384342 wait complete. 2024-04-25T00:16:16.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.524+0000 7efe8f7b7640 1 Processor -- start 2024-04-25T00:16:16.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.524+0000 7efe8f7b7640 1 -- start start 2024-04-25T00:16:16.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.524+0000 7efe8f7b7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe88100680 0x7efe881969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:16.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.524+0000 7efe8f7b7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe88101870 0x7efe88196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:16.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.524+0000 7efe8f7b7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe88102230 0x7efe8819df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:16.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.524+0000 7efe8f7b7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efe8810c350 con 0x7efe88102230 2024-04-25T00:16:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.524+0000 7efe8f7b7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efe8810c1d0 con 0x7efe88100680 2024-04-25T00:16:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.524+0000 7efe8f7b7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efe8810c4d0 con 0x7efe88101870 2024-04-25T00:16:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.524+0000 7efe8dd54640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe88102230 0x7efe8819df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.524+0000 7efe8dd54640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe88102230 0x7efe8819df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56996/0 (socket says 172.21.15.107:56996) 2024-04-25T00:16:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.524+0000 7efe8dd54640 1 -- 172.21.15.107:0/511001078 learned_addr learned my addr 172.21.15.107:0/511001078 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.524+0000 7efe8cd52640 1 --2- 172.21.15.107:0/511001078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe88101870 0x7efe88196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.524+0000 7efe8d553640 1 --2- 172.21.15.107:0/511001078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe88100680 0x7efe881969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.525+0000 7efe8dd54640 1 -- 172.21.15.107:0/511001078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe88101870 msgr2=0x7efe88196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.525+0000 7efe8dd54640 1 --2- 172.21.15.107:0/511001078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe88101870 0x7efe88196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.525+0000 7efe8dd54640 1 -- 172.21.15.107:0/511001078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe88100680 msgr2=0x7efe881969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.525+0000 7efe8dd54640 1 --2- 172.21.15.107:0/511001078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe88100680 0x7efe881969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.525+0000 7efe8dd54640 1 -- 172.21.15.107:0/511001078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8819e5e0 con 0x7efe88102230 2024-04-25T00:16:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.525+0000 7efe8dd54640 1 --2- 172.21.15.107:0/511001078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe88102230 0x7efe8819df70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7efe8000b970 tx=0x7efe8000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:16.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.525+0000 7efe7e7fc640 1 -- 172.21.15.107:0/511001078 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe80019070 con 0x7efe88102230 2024-04-25T00:16:16.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.525+0000 7efe7e7fc640 1 -- 172.21.15.107:0/511001078 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe80014440 con 0x7efe88102230 2024-04-25T00:16:16.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.525+0000 7efe8f7b7640 1 -- 172.21.15.107:0/511001078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8819e8d0 con 0x7efe88102230 2024-04-25T00:16:16.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.525+0000 7efe7e7fc640 1 -- 172.21.15.107:0/511001078 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe800075d0 con 0x7efe88102230 2024-04-25T00:16:16.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.525+0000 7efe8f7b7640 1 -- 172.21.15.107:0/511001078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe8819eea0 con 0x7efe88102230 2024-04-25T00:16:16.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.526+0000 7efe8f7b7640 1 -- 172.21.15.107:0/511001078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe88107a20 con 0x7efe88102230 2024-04-25T00:16:16.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.527+0000 7efe7e7fc640 1 -- 172.21.15.107:0/511001078 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7efe8001c070 con 0x7efe88102230 2024-04-25T00:16:16.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.527+0000 7efe7e7fc640 1 --2- 172.21.15.107:0/511001078 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7efe740a2c10 0x7efe740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:16.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.528+0000 7efe8d553640 1 --2- 172.21.15.107:0/511001078 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7efe740a2c10 0x7efe740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:16.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.528+0000 7efe7e7fc640 1 -- 172.21.15.107:0/511001078 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7efe800ce8f0 con 0x7efe88102230 2024-04-25T00:16:16.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.528+0000 7efe8d553640 1 --2- 172.21.15.107:0/511001078 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7efe740a2c10 0x7efe740a50d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efe84005ab0 tx=0x7efe84000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:16.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.532+0000 7efe7e7fc640 1 -- 172.21.15.107:0/511001078 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efe8009ae70 con 0x7efe88102230 2024-04-25T00:16:16.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.706+0000 7efe8f7b7640 1 -- 172.21.15.107:0/511001078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efe88062340 con 0x7efe88102230 2024-04-25T00:16:16.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.707+0000 7efe7e7fc640 1 -- 172.21.15.107:0/511001078 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7efe8009f170 con 0x7efe88102230 2024-04-25T00:16:16.708 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-04-25T00:16:16.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.709+0000 7efe8f7b7640 1 -- 172.21.15.107:0/511001078 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7efe740a2c10 msgr2=0x7efe740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.709+0000 7efe8f7b7640 1 --2- 172.21.15.107:0/511001078 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7efe740a2c10 0x7efe740a50d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7efe84005ab0 tx=0x7efe84000bc0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.709+0000 7efe8f7b7640 1 -- 172.21.15.107:0/511001078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe88102230 msgr2=0x7efe8819df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.709+0000 7efe8f7b7640 1 --2- 172.21.15.107:0/511001078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe88102230 0x7efe8819df70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7efe8000b970 tx=0x7efe8000bd70 comp rx=0 tx=0).stop 2024-04-25T00:16:16.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.709+0000 7efe8f7b7640 1 -- 172.21.15.107:0/511001078 shutdown_connections 2024-04-25T00:16:16.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.709+0000 7efe8f7b7640 1 --2- 172.21.15.107:0/511001078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe88102230 0x7efe8819df70 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.709+0000 7efe8f7b7640 1 --2- 172.21.15.107:0/511001078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe88101870 0x7efe88196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.709+0000 7efe8f7b7640 1 --2- 172.21.15.107:0/511001078 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7efe740a2c10 0x7efe740a50d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.709+0000 7efe8f7b7640 1 --2- 172.21.15.107:0/511001078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe88100680 0x7efe881969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.709+0000 7efe8f7b7640 1 -- 172.21.15.107:0/511001078 >> 172.21.15.107:0/511001078 conn(0x7efe880fbbf0 msgr2=0x7efe88104bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:16.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.709+0000 7efe8f7b7640 1 -- 172.21.15.107:0/511001078 shutdown_connections 2024-04-25T00:16:16.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.709+0000 7efe8f7b7640 1 -- 172.21.15.107:0/511001078 wait complete. 2024-04-25T00:16:16.721 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:16:16.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.821+0000 7f7d37c4d640 1 -- 172.21.15.107:0/2262229070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d301033e0 msgr2=0x7f7d30109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.821+0000 7f7d37c4d640 1 --2- 172.21.15.107:0/2262229070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d301033e0 0x7f7d30109c70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7d28009cd0 tx=0x7f7d2802a8e0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.821+0000 7f7d37c4d640 1 -- 172.21.15.107:0/2262229070 shutdown_connections 2024-04-25T00:16:16.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.821+0000 7f7d37c4d640 1 --2- 172.21.15.107:0/2262229070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d301033e0 0x7f7d30109c70 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.821+0000 7f7d37c4d640 1 --2- 172.21.15.107:0/2262229070 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d30102a20 0x7f7d30102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.821+0000 7f7d37c4d640 1 --2- 172.21.15.107:0/2262229070 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d300ffa50 0x7f7d300ffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.821+0000 7f7d37c4d640 1 -- 172.21.15.107:0/2262229070 >> 172.21.15.107:0/2262229070 conn(0x7f7d300fb380 msgr2=0x7f7d300fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:16.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.821+0000 7f7d37c4d640 1 -- 172.21.15.107:0/2262229070 shutdown_connections 2024-04-25T00:16:16.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.821+0000 7f7d37c4d640 1 -- 172.21.15.107:0/2262229070 wait complete. 2024-04-25T00:16:16.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.822+0000 7f7d37c4d640 1 Processor -- start 2024-04-25T00:16:16.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.822+0000 7f7d37c4d640 1 -- start start 2024-04-25T00:16:16.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.822+0000 7f7d37c4d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d300ffa50 0x7f7d30197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:16.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.822+0000 7f7d37c4d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d30102a20 0x7f7d30197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:16.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.822+0000 7f7d37c4d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d301033e0 0x7f7d3019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:16.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.822+0000 7f7d37c4d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3010d420 con 0x7f7d301033e0 2024-04-25T00:16:16.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.822+0000 7f7d37c4d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3010d2a0 con 0x7f7d300ffa50 2024-04-25T00:16:16.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.822+0000 7f7d37c4d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3010d5a0 con 0x7f7d30102a20 2024-04-25T00:16:16.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.822+0000 7f7d361ea640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d301033e0 0x7f7d3019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:16.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.822+0000 7f7d361ea640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d301033e0 0x7f7d3019f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57024/0 (socket says 172.21.15.107:57024) 2024-04-25T00:16:16.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.822+0000 7f7d361ea640 1 -- 172.21.15.107:0/2253793571 learned_addr learned my addr 172.21.15.107:0/2253793571 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:16.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.822+0000 7f7d359e9640 1 --2- 172.21.15.107:0/2253793571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d300ffa50 0x7f7d30197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:16.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.822+0000 7f7d351e8640 1 --2- 172.21.15.107:0/2253793571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d30102a20 0x7f7d30197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:16.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.822+0000 7f7d361ea640 1 -- 172.21.15.107:0/2253793571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d30102a20 msgr2=0x7f7d30197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.822+0000 7f7d361ea640 1 --2- 172.21.15.107:0/2253793571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d30102a20 0x7f7d30197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.822+0000 7f7d361ea640 1 -- 172.21.15.107:0/2253793571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d300ffa50 msgr2=0x7f7d30197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.822+0000 7f7d361ea640 1 --2- 172.21.15.107:0/2253793571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d300ffa50 0x7f7d30197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.822+0000 7f7d361ea640 1 -- 172.21.15.107:0/2253793571 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3019f680 con 0x7f7d301033e0 2024-04-25T00:16:16.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.822+0000 7f7d359e9640 1 --2- 172.21.15.107:0/2253793571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d300ffa50 0x7f7d30197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:16.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.823+0000 7f7d361ea640 1 --2- 172.21.15.107:0/2253793571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d301033e0 0x7f7d3019f010 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7d28031920 tx=0x7f7d28003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:16.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.823+0000 7f7d26ffd640 1 -- 172.21.15.107:0/2253793571 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28004450 con 0x7f7d301033e0 2024-04-25T00:16:16.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.823+0000 7f7d37c4d640 1 -- 172.21.15.107:0/2253793571 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3019f910 con 0x7f7d301033e0 2024-04-25T00:16:16.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.823+0000 7f7d26ffd640 1 -- 172.21.15.107:0/2253793571 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d28003e30 con 0x7f7d301033e0 2024-04-25T00:16:16.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.823+0000 7f7d26ffd640 1 -- 172.21.15.107:0/2253793571 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d280466e0 con 0x7f7d301033e0 2024-04-25T00:16:16.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.823+0000 7f7d37c4d640 1 -- 172.21.15.107:0/2253793571 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d3019fe60 con 0x7f7d301033e0 2024-04-25T00:16:16.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.824+0000 7f7d37c4d640 1 -- 172.21.15.107:0/2253793571 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d30108b50 con 0x7f7d301033e0 2024-04-25T00:16:16.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.825+0000 7f7d26ffd640 1 -- 172.21.15.107:0/2253793571 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7d280305a0 con 0x7f7d301033e0 2024-04-25T00:16:16.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.825+0000 7f7d26ffd640 1 --2- 172.21.15.107:0/2253793571 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7d1c0a2b90 0x7f7d1c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:16.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.826+0000 7f7d359e9640 1 --2- 172.21.15.107:0/2253793571 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7d1c0a2b90 0x7f7d1c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:16.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.826+0000 7f7d26ffd640 1 -- 172.21.15.107:0/2253793571 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7f7d2802e030 con 0x7f7d301033e0 2024-04-25T00:16:16.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.826+0000 7f7d359e9640 1 --2- 172.21.15.107:0/2253793571 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7d1c0a2b90 0x7f7d1c0a5050 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0058c0 tx=0x7f7d2c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:16.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:16.829+0000 7f7d26ffd640 1 -- 172.21.15.107:0/2253793571 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7d2804b060 con 0x7f7d301033e0 2024-04-25T00:16:17.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.007+0000 7f7d37c4d640 1 -- 172.21.15.107:0/2253793571 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7d3002d170 con 0x7f7d301033e0 2024-04-25T00:16:17.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.007+0000 7f7d26ffd640 1 -- 172.21.15.107:0/2253793571 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7f7d280bd810 con 0x7f7d301033e0 2024-04-25T00:16:17.008 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-25T00:16:17.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.009+0000 7f7d37c4d640 1 -- 172.21.15.107:0/2253793571 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7d1c0a2b90 msgr2=0x7f7d1c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:17.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.009+0000 7f7d37c4d640 1 --2- 172.21.15.107:0/2253793571 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7d1c0a2b90 0x7f7d1c0a5050 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c0058c0 tx=0x7f7d2c005850 comp rx=0 tx=0).stop 2024-04-25T00:16:17.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.009+0000 7f7d37c4d640 1 -- 172.21.15.107:0/2253793571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d301033e0 msgr2=0x7f7d3019f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:17.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.009+0000 7f7d37c4d640 1 --2- 172.21.15.107:0/2253793571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d301033e0 0x7f7d3019f010 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7d28031920 tx=0x7f7d28003b80 comp rx=0 tx=0).stop 2024-04-25T00:16:17.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.009+0000 7f7d37c4d640 1 -- 172.21.15.107:0/2253793571 shutdown_connections 2024-04-25T00:16:17.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.009+0000 7f7d37c4d640 1 --2- 172.21.15.107:0/2253793571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d301033e0 0x7f7d3019f010 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.009+0000 7f7d37c4d640 1 --2- 172.21.15.107:0/2253793571 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d30102a20 0x7f7d30197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.009+0000 7f7d37c4d640 1 --2- 172.21.15.107:0/2253793571 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7d1c0a2b90 0x7f7d1c0a5050 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.009+0000 7f7d37c4d640 1 --2- 172.21.15.107:0/2253793571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d300ffa50 0x7f7d30197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.009+0000 7f7d37c4d640 1 -- 172.21.15.107:0/2253793571 >> 172.21.15.107:0/2253793571 conn(0x7f7d300fb380 msgr2=0x7f7d30105d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:17.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.009+0000 7f7d37c4d640 1 -- 172.21.15.107:0/2253793571 shutdown_connections 2024-04-25T00:16:17.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.009+0000 7f7d37c4d640 1 -- 172.21.15.107:0/2253793571 wait complete. 2024-04-25T00:16:17.021 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:16:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.125+0000 7f771c88c640 1 -- 172.21.15.107:0/2898266497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7718102250 msgr2=0x7f7718108ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.125+0000 7f771c88c640 1 --2- 172.21.15.107:0/2898266497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7718102250 0x7f7718108ae0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7710009ca0 tx=0x7f771002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.125+0000 7f771c88c640 1 -- 172.21.15.107:0/2898266497 shutdown_connections 2024-04-25T00:16:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.125+0000 7f771c88c640 1 --2- 172.21.15.107:0/2898266497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7718102250 0x7f7718108ae0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.125+0000 7f771c88c640 1 --2- 172.21.15.107:0/2898266497 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7718101890 0x7f7718101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.125+0000 7f771c88c640 1 --2- 172.21.15.107:0/2898266497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f77181006a0 0x7f7718100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.125+0000 7f771c88c640 1 -- 172.21.15.107:0/2898266497 >> 172.21.15.107:0/2898266497 conn(0x7f77180fbc30 msgr2=0x7f77180fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.125+0000 7f771c88c640 1 -- 172.21.15.107:0/2898266497 shutdown_connections 2024-04-25T00:16:17.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.125+0000 7f771c88c640 1 -- 172.21.15.107:0/2898266497 wait complete. 2024-04-25T00:16:17.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.125+0000 7f771c88c640 1 Processor -- start 2024-04-25T00:16:17.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.125+0000 7f771c88c640 1 -- start start 2024-04-25T00:16:17.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.126+0000 7f771c88c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77181006a0 0x7f7718197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:17.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.126+0000 7f771c88c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7718101890 0x7f7718197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:17.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.126+0000 7f771c88c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7718102250 0x7f771819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:17.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.126+0000 7f771c88c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f771810d440 con 0x7f7718102250 2024-04-25T00:16:17.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.126+0000 7f771c88c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f771810d2c0 con 0x7f7718101890 2024-04-25T00:16:17.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.126+0000 7f771c88c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f771810d5c0 con 0x7f77181006a0 2024-04-25T00:16:17.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.126+0000 7f7716d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7718102250 0x7f771819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:17.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.126+0000 7f7716d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7718102250 0x7f771819eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57042/0 (socket says 172.21.15.107:57042) 2024-04-25T00:16:17.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.126+0000 7f7716d9d640 1 -- 172.21.15.107:0/3686589652 learned_addr learned my addr 172.21.15.107:0/3686589652 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:17.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.126+0000 7f771659c640 1 --2- 172.21.15.107:0/3686589652 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77181006a0 0x7f7718197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:17.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.126+0000 7f770ffff640 1 --2- 172.21.15.107:0/3686589652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7718101890 0x7f7718197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:17.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.126+0000 7f7716d9d640 1 -- 172.21.15.107:0/3686589652 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77181006a0 msgr2=0x7f7718197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:17.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.126+0000 7f7716d9d640 1 --2- 172.21.15.107:0/3686589652 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77181006a0 0x7f7718197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.126+0000 7f7716d9d640 1 -- 172.21.15.107:0/3686589652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7718101890 msgr2=0x7f7718197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:17.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.126+0000 7f7716d9d640 1 --2- 172.21.15.107:0/3686589652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7718101890 0x7f7718197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.126+0000 7f7716d9d640 1 -- 172.21.15.107:0/3686589652 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771819f660 con 0x7f7718102250 2024-04-25T00:16:17.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.126+0000 7f771659c640 1 --2- 172.21.15.107:0/3686589652 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77181006a0 0x7f7718197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:17.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.126+0000 7f770ffff640 1 --2- 172.21.15.107:0/3686589652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7718101890 0x7f7718197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:17.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.126+0000 7f7716d9d640 1 --2- 172.21.15.107:0/3686589652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7718102250 0x7f771819eff0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f77100098d0 tx=0x7f771002f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:17.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.126+0000 7f770f7fe640 1 -- 172.21.15.107:0/3686589652 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771002fc30 con 0x7f7718102250 2024-04-25T00:16:17.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.126+0000 7f771c88c640 1 -- 172.21.15.107:0/3686589652 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f771819f8f0 con 0x7f7718102250 2024-04-25T00:16:17.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.126+0000 7f770f7fe640 1 -- 172.21.15.107:0/3686589652 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7710003e50 con 0x7f7718102250 2024-04-25T00:16:17.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.126+0000 7f770f7fe640 1 -- 172.21.15.107:0/3686589652 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77100305c0 con 0x7f7718102250 2024-04-25T00:16:17.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.126+0000 7f771c88c640 1 -- 172.21.15.107:0/3686589652 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f771819fe90 con 0x7f7718102250 2024-04-25T00:16:17.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.128+0000 7f771c88c640 1 -- 172.21.15.107:0/3686589652 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7718107960 con 0x7f7718102250 2024-04-25T00:16:17.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.129+0000 7f770f7fe640 1 -- 172.21.15.107:0/3686589652 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7710003970 con 0x7f7718102250 2024-04-25T00:16:17.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.129+0000 7f770f7fe640 1 --2- 172.21.15.107:0/3686589652 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f77040a2b90 0x7f77040a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:17.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.129+0000 7f771659c640 1 --2- 172.21.15.107:0/3686589652 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f77040a2b90 0x7f77040a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:17.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.129+0000 7f770f7fe640 1 -- 172.21.15.107:0/3686589652 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7f77100f0860 con 0x7f7718102250 2024-04-25T00:16:17.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.130+0000 7f771659c640 1 --2- 172.21.15.107:0/3686589652 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f77040a2b90 0x7f77040a5050 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f77181016f0 tx=0x7f770800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:17.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.133+0000 7f770f7fe640 1 -- 172.21.15.107:0/3686589652 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f77100bd550 con 0x7f7718102250 2024-04-25T00:16:17.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.307+0000 7f771c88c640 1 -- 172.21.15.107:0/3686589652 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f771802d170 con 0x7f7718102250 2024-04-25T00:16:17.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.307+0000 7f770f7fe640 1 -- 172.21.15.107:0/3686589652 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7f77100bce50 con 0x7f7718102250 2024-04-25T00:16:17.308 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-04-25T00:16:17.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.309+0000 7f771c88c640 1 -- 172.21.15.107:0/3686589652 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f77040a2b90 msgr2=0x7f77040a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:17.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.309+0000 7f771c88c640 1 --2- 172.21.15.107:0/3686589652 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f77040a2b90 0x7f77040a5050 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f77181016f0 tx=0x7f770800a560 comp rx=0 tx=0).stop 2024-04-25T00:16:17.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.309+0000 7f771c88c640 1 -- 172.21.15.107:0/3686589652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7718102250 msgr2=0x7f771819eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:17.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.309+0000 7f771c88c640 1 --2- 172.21.15.107:0/3686589652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7718102250 0x7f771819eff0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f77100098d0 tx=0x7f771002f840 comp rx=0 tx=0).stop 2024-04-25T00:16:17.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.309+0000 7f771c88c640 1 -- 172.21.15.107:0/3686589652 shutdown_connections 2024-04-25T00:16:17.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.309+0000 7f771c88c640 1 --2- 172.21.15.107:0/3686589652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7718102250 0x7f771819eff0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.309+0000 7f771c88c640 1 --2- 172.21.15.107:0/3686589652 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f77040a2b90 0x7f77040a5050 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.309+0000 7f771c88c640 1 --2- 172.21.15.107:0/3686589652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7718101890 0x7f7718197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.309+0000 7f771c88c640 1 --2- 172.21.15.107:0/3686589652 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f77181006a0 0x7f7718197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.309+0000 7f771c88c640 1 -- 172.21.15.107:0/3686589652 >> 172.21.15.107:0/3686589652 conn(0x7f77180fbc30 msgr2=0x7f7718104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:17.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.309+0000 7f771c88c640 1 -- 172.21.15.107:0/3686589652 shutdown_connections 2024-04-25T00:16:17.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.309+0000 7f771c88c640 1 -- 172.21.15.107:0/3686589652 wait complete. 2024-04-25T00:16:17.319 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:17 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2678928639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:17.319 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:17 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2332178551' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:17.319 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:17 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/511001078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:16:17.319 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:17 smithi107 ceph-mon[29153]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:16:17.320 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:17 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2253793571' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:16:17.321 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:16:17.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.423+0000 7f7f31922640 1 -- 172.21.15.107:0/2797855550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f2c0ffa60 msgr2=0x7f7f2c0ffe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:17.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.423+0000 7f7f31922640 1 --2- 172.21.15.107:0/2797855550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f2c0ffa60 0x7f7f2c0ffe80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009930 tx=0x7f7f1c02a420 comp rx=0 tx=0).stop 2024-04-25T00:16:17.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.423+0000 7f7f31922640 1 -- 172.21.15.107:0/2797855550 shutdown_connections 2024-04-25T00:16:17.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.423+0000 7f7f31922640 1 --2- 172.21.15.107:0/2797855550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f2c1033f0 0x7f7f2c109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.423+0000 7f7f31922640 1 --2- 172.21.15.107:0/2797855550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f2c102a30 0x7f7f2c102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.423+0000 7f7f31922640 1 --2- 172.21.15.107:0/2797855550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f2c0ffa60 0x7f7f2c0ffe80 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.423+0000 7f7f31922640 1 -- 172.21.15.107:0/2797855550 >> 172.21.15.107:0/2797855550 conn(0x7f7f2c0fb370 msgr2=0x7f7f2c0fd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:17.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.423+0000 7f7f31922640 1 -- 172.21.15.107:0/2797855550 shutdown_connections 2024-04-25T00:16:17.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.423+0000 7f7f31922640 1 -- 172.21.15.107:0/2797855550 wait complete. 2024-04-25T00:16:17.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.424+0000 7f7f31922640 1 Processor -- start 2024-04-25T00:16:17.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.424+0000 7f7f31922640 1 -- start start 2024-04-25T00:16:17.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.424+0000 7f7f31922640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f2c0ffa60 0x7f7f2c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:17.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.424+0000 7f7f31922640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f2c102a30 0x7f7f2c198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:17.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.424+0000 7f7f31922640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f2c1033f0 0x7f7f2c19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:17.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.424+0000 7f7f31922640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c10d410 con 0x7f7f2c102a30 2024-04-25T00:16:17.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.424+0000 7f7f31922640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c10d290 con 0x7f7f2c1033f0 2024-04-25T00:16:17.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.424+0000 7f7f31922640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c10d590 con 0x7f7f2c0ffa60 2024-04-25T00:16:17.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.424+0000 7f7f2a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f2c102a30 0x7f7f2c198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:17.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.424+0000 7f7f2a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f2c102a30 0x7f7f2c198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57066/0 (socket says 172.21.15.107:57066) 2024-04-25T00:16:17.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.424+0000 7f7f2a7fc640 1 -- 172.21.15.107:0/3879268060 learned_addr learned my addr 172.21.15.107:0/3879268060 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:17.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.424+0000 7f7f2affd640 1 --2- 172.21.15.107:0/3879268060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f2c0ffa60 0x7f7f2c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:17.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.424+0000 7f7f2b7fe640 1 --2- 172.21.15.107:0/3879268060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f2c1033f0 0x7f7f2c19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:17.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.424+0000 7f7f2a7fc640 1 -- 172.21.15.107:0/3879268060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f2c0ffa60 msgr2=0x7f7f2c197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:17.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.424+0000 7f7f2a7fc640 1 --2- 172.21.15.107:0/3879268060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f2c0ffa60 0x7f7f2c197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.424+0000 7f7f2a7fc640 1 -- 172.21.15.107:0/3879268060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f2c1033f0 msgr2=0x7f7f2c19f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:17.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.424+0000 7f7f2a7fc640 1 --2- 172.21.15.107:0/3879268060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f2c1033f0 0x7f7f2c19f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.424+0000 7f7f2a7fc640 1 -- 172.21.15.107:0/3879268060 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2c19f720 con 0x7f7f2c102a30 2024-04-25T00:16:17.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.424+0000 7f7f2affd640 1 --2- 172.21.15.107:0/3879268060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f2c0ffa60 0x7f7f2c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:17.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.424+0000 7f7f2a7fc640 1 --2- 172.21.15.107:0/3879268060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f2c102a30 0x7f7f2c198040 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000d950 tx=0x7f7f2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:17.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.425+0000 7f7f30920640 1 -- 172.21.15.107:0/3879268060 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f20017030 con 0x7f7f2c102a30 2024-04-25T00:16:17.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.425+0000 7f7f30920640 1 -- 172.21.15.107:0/3879268060 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f20004020 con 0x7f7f2c102a30 2024-04-25T00:16:17.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.425+0000 7f7f31922640 1 -- 172.21.15.107:0/3879268060 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f2c19fa10 con 0x7f7f2c102a30 2024-04-25T00:16:17.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.425+0000 7f7f30920640 1 -- 172.21.15.107:0/3879268060 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f200047a0 con 0x7f7f2c102a30 2024-04-25T00:16:17.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.425+0000 7f7f31922640 1 -- 172.21.15.107:0/3879268060 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f2c19ff60 con 0x7f7f2c102a30 2024-04-25T00:16:17.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.426+0000 7f7f31922640 1 -- 172.21.15.107:0/3879268060 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f2c108c30 con 0x7f7f2c102a30 2024-04-25T00:16:17.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.427+0000 7f7f30920640 1 -- 172.21.15.107:0/3879268060 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f7f200041d0 con 0x7f7f2c102a30 2024-04-25T00:16:17.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.427+0000 7f7f30920640 1 --2- 172.21.15.107:0/3879268060 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7f140a2b90 0x7f7f140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:17.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.427+0000 7f7f2affd640 1 --2- 172.21.15.107:0/3879268060 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7f140a2b90 0x7f7f140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:17.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.427+0000 7f7f30920640 1 -- 172.21.15.107:0/3879268060 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7f7f200ceef0 con 0x7f7f2c102a30 2024-04-25T00:16:17.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.428+0000 7f7f2affd640 1 --2- 172.21.15.107:0/3879268060 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7f140a2b90 0x7f7f140a5050 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c006010 tx=0x7f7f1c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:17.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.431+0000 7f7f30920640 1 -- 172.21.15.107:0/3879268060 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7f2009bac0 con 0x7f7f2c102a30 2024-04-25T00:16:17.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:17 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2678928639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:17.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:17 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2332178551' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:17.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:17 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/511001078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:16:17.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:17 smithi170 ceph-mon[32807]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:16:17.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:17 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2253793571' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:16:17.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:17 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2678928639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:17.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:17 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2332178551' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:17.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:17 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/511001078' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:16:17.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:17 smithi195 ceph-mon[32833]: pgmap v364: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:16:17.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:17 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2253793571' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:16:17.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.606+0000 7f7f31922640 1 -- 172.21.15.107:0/3879268060 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7f2c062340 con 0x7f7f2c102a30 2024-04-25T00:16:17.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.607+0000 7f7f30920640 1 -- 172.21.15.107:0/3879268060 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v96) ==== 103+0+11 (secure 0 0 0) 0x7f7f2009b3c0 con 0x7f7f2c102a30 2024-04-25T00:16:17.607 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-25T00:16:17.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.609+0000 7f7f31922640 1 -- 172.21.15.107:0/3879268060 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7f140a2b90 msgr2=0x7f7f140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:17.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.609+0000 7f7f31922640 1 --2- 172.21.15.107:0/3879268060 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7f140a2b90 0x7f7f140a5050 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c006010 tx=0x7f7f1c000bc0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.609+0000 7f7f31922640 1 -- 172.21.15.107:0/3879268060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f2c102a30 msgr2=0x7f7f2c198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:17.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.609+0000 7f7f31922640 1 --2- 172.21.15.107:0/3879268060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f2c102a30 0x7f7f2c198040 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7f2000d950 tx=0x7f7f2000de10 comp rx=0 tx=0).stop 2024-04-25T00:16:17.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.609+0000 7f7f31922640 1 -- 172.21.15.107:0/3879268060 shutdown_connections 2024-04-25T00:16:17.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.609+0000 7f7f31922640 1 --2- 172.21.15.107:0/3879268060 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f7f140a2b90 0x7f7f140a5050 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.609+0000 7f7f31922640 1 --2- 172.21.15.107:0/3879268060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f2c1033f0 0x7f7f2c19f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.609+0000 7f7f31922640 1 --2- 172.21.15.107:0/3879268060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f2c102a30 0x7f7f2c198040 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.609+0000 7f7f31922640 1 --2- 172.21.15.107:0/3879268060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7f2c0ffa60 0x7f7f2c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.609+0000 7f7f31922640 1 -- 172.21.15.107:0/3879268060 >> 172.21.15.107:0/3879268060 conn(0x7f7f2c0fb370 msgr2=0x7f7f2c105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:17.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.609+0000 7f7f31922640 1 -- 172.21.15.107:0/3879268060 shutdown_connections 2024-04-25T00:16:17.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.609+0000 7f7f31922640 1 -- 172.21.15.107:0/3879268060 wait complete. 2024-04-25T00:16:17.620 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:16:17.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.718+0000 7f8134b96640 1 -- 172.21.15.107:0/3137720968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81301017b0 msgr2=0x7f8130101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:17.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.718+0000 7f8134b96640 1 --2- 172.21.15.107:0/3137720968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81301017b0 0x7f8130101bd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8120009ca0 tx=0x7f812002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.718+0000 7f8134b96640 1 -- 172.21.15.107:0/3137720968 shutdown_connections 2024-04-25T00:16:17.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.718+0000 7f8134b96640 1 --2- 172.21.15.107:0/3137720968 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8130103360 0x7f8130109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.718+0000 7f8134b96640 1 --2- 172.21.15.107:0/3137720968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81301029a0 0x7f8130102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.718+0000 7f8134b96640 1 --2- 172.21.15.107:0/3137720968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81301017b0 0x7f8130101bd0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.718+0000 7f8134b96640 1 -- 172.21.15.107:0/3137720968 >> 172.21.15.107:0/3137720968 conn(0x7f81300fcd20 msgr2=0x7f81300ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:17.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.718+0000 7f8134b96640 1 -- 172.21.15.107:0/3137720968 shutdown_connections 2024-04-25T00:16:17.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.718+0000 7f8134b96640 1 -- 172.21.15.107:0/3137720968 wait complete. 2024-04-25T00:16:17.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.719+0000 7f8134b96640 1 Processor -- start 2024-04-25T00:16:17.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.719+0000 7f8134b96640 1 -- start start 2024-04-25T00:16:17.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.719+0000 7f8134b96640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81301017b0 0x7f8130197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:17.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.719+0000 7f8134b96640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81301029a0 0x7f8130198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:17.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.719+0000 7f8134b96640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8130103360 0x7f813019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:17.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.719+0000 7f8134b96640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f813010d480 con 0x7f8130103360 2024-04-25T00:16:17.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.719+0000 7f8134b96640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f813010d300 con 0x7f81301017b0 2024-04-25T00:16:17.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.719+0000 7f8134b96640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f813010d600 con 0x7f81301029a0 2024-04-25T00:16:17.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.719+0000 7f812ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8130103360 0x7f813019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:17.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.720+0000 7f812ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8130103360 0x7f813019f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57084/0 (socket says 172.21.15.107:57084) 2024-04-25T00:16:17.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.720+0000 7f812ed9d640 1 -- 172.21.15.107:0/3638941736 learned_addr learned my addr 172.21.15.107:0/3638941736 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:17.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.720+0000 7f812dd9b640 1 --2- 172.21.15.107:0/3638941736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81301029a0 0x7f8130198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:17.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.720+0000 7f812e59c640 1 --2- 172.21.15.107:0/3638941736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81301017b0 0x7f8130197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:17.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.720+0000 7f812ed9d640 1 -- 172.21.15.107:0/3638941736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81301029a0 msgr2=0x7f8130198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:17.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.720+0000 7f812ed9d640 1 --2- 172.21.15.107:0/3638941736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81301029a0 0x7f8130198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.720+0000 7f812ed9d640 1 -- 172.21.15.107:0/3638941736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81301017b0 msgr2=0x7f8130197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:17.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.720+0000 7f812ed9d640 1 --2- 172.21.15.107:0/3638941736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81301017b0 0x7f8130197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.720+0000 7f812ed9d640 1 -- 172.21.15.107:0/3638941736 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813019f750 con 0x7f8130103360 2024-04-25T00:16:17.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.720+0000 7f812dd9b640 1 --2- 172.21.15.107:0/3638941736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81301029a0 0x7f8130198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:17.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.720+0000 7f812ed9d640 1 --2- 172.21.15.107:0/3638941736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8130103360 0x7f813019f0e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8128009fd0 tx=0x7f812800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:17.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.720+0000 7f811f7fe640 1 -- 172.21.15.107:0/3638941736 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128011830 con 0x7f8130103360 2024-04-25T00:16:17.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.720+0000 7f811f7fe640 1 -- 172.21.15.107:0/3638941736 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8128016070 con 0x7f8130103360 2024-04-25T00:16:17.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.720+0000 7f8134b96640 1 -- 172.21.15.107:0/3638941736 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f813019fa40 con 0x7f8130103360 2024-04-25T00:16:17.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.720+0000 7f811f7fe640 1 -- 172.21.15.107:0/3638941736 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812800f2e0 con 0x7f8130103360 2024-04-25T00:16:17.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.720+0000 7f8134b96640 1 -- 172.21.15.107:0/3638941736 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81301a0010 con 0x7f8130103360 2024-04-25T00:16:17.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.721+0000 7f8134b96640 1 -- 172.21.15.107:0/3638941736 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8130108ad0 con 0x7f8130103360 2024-04-25T00:16:17.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.722+0000 7f811f7fe640 1 -- 172.21.15.107:0/3638941736 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f81280119d0 con 0x7f8130103360 2024-04-25T00:16:17.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.723+0000 7f811f7fe640 1 --2- 172.21.15.107:0/3638941736 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f81180a2b90 0x7f81180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:17.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.723+0000 7f812e59c640 1 --2- 172.21.15.107:0/3638941736 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f81180a2b90 0x7f81180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:17.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.723+0000 7f811f7fe640 1 -- 172.21.15.107:0/3638941736 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7f81280ceab0 con 0x7f8130103360 2024-04-25T00:16:17.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.723+0000 7f812e59c640 1 --2- 172.21.15.107:0/3638941736 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f81180a2b90 0x7f81180a5050 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8120006010 tx=0x7f8120009360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:17.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.727+0000 7f811f7fe640 1 -- 172.21.15.107:0/3638941736 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f812809b680 con 0x7f8130103360 2024-04-25T00:16:17.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.944+0000 7f8134b96640 1 -- 172.21.15.107:0/3638941736 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8130062340 con 0x7f8130103360 2024-04-25T00:16:17.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.945+0000 7f811f7fe640 1 -- 172.21.15.107:0/3638941736 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11897 (secure 0 0 0) 0x7f812809af80 con 0x7f8130103360 2024-04-25T00:16:17.946 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:16:17.946 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:14:28.529865+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24521,24493]},"id":1}]} 2024-04-25T00:16:17.946 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 23 2024-04-25T00:16:17.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.947+0000 7f8134b96640 1 -- 172.21.15.107:0/3638941736 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f81180a2b90 msgr2=0x7f81180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:17.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.947+0000 7f8134b96640 1 --2- 172.21.15.107:0/3638941736 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f81180a2b90 0x7f81180a5050 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8120006010 tx=0x7f8120009360 comp rx=0 tx=0).stop 2024-04-25T00:16:17.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.947+0000 7f8134b96640 1 -- 172.21.15.107:0/3638941736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8130103360 msgr2=0x7f813019f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:17.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.947+0000 7f8134b96640 1 --2- 172.21.15.107:0/3638941736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8130103360 0x7f813019f0e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8128009fd0 tx=0x7f812800b8e0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.947+0000 7f8134b96640 1 -- 172.21.15.107:0/3638941736 shutdown_connections 2024-04-25T00:16:17.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.947+0000 7f8134b96640 1 --2- 172.21.15.107:0/3638941736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8130103360 0x7f813019f0e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.947+0000 7f8134b96640 1 --2- 172.21.15.107:0/3638941736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81301029a0 0x7f8130198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.947+0000 7f8134b96640 1 --2- 172.21.15.107:0/3638941736 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f81180a2b90 0x7f81180a5050 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.947+0000 7f8134b96640 1 --2- 172.21.15.107:0/3638941736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81301017b0 0x7f8130197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.947+0000 7f8134b96640 1 -- 172.21.15.107:0/3638941736 >> 172.21.15.107:0/3638941736 conn(0x7f81300fcd20 msgr2=0x7f8130105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:17.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.947+0000 7f8134b96640 1 -- 172.21.15.107:0/3638941736 shutdown_connections 2024-04-25T00:16:17.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:17.947+0000 7f8134b96640 1 -- 172.21.15.107:0/3638941736 wait complete. 2024-04-25T00:16:17.959 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:16:18.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.057+0000 7fe82ad4f640 1 -- 172.21.15.107:0/2278778792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe824104b90 msgr2=0x7fe824106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.057+0000 7fe82ad4f640 1 --2- 172.21.15.107:0/2278778792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe824104b90 0x7fe824106f80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe810009900 tx=0x7fe81002a410 comp rx=0 tx=0).stop 2024-04-25T00:16:18.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.057+0000 7fe82ad4f640 1 -- 172.21.15.107:0/2278778792 shutdown_connections 2024-04-25T00:16:18.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.057+0000 7fe82ad4f640 1 --2- 172.21.15.107:0/2278778792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8241074c0 0x7fe8241098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.057+0000 7fe82ad4f640 1 --2- 172.21.15.107:0/2278778792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe824104b90 0x7fe824106f80 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.057+0000 7fe82ad4f640 1 --2- 172.21.15.107:0/2278778792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe824102260 0x7fe824104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.057+0000 7fe82ad4f640 1 -- 172.21.15.107:0/2278778792 >> 172.21.15.107:0/2278778792 conn(0x7fe8240fbbf0 msgr2=0x7fe8240fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:18.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.057+0000 7fe82ad4f640 1 -- 172.21.15.107:0/2278778792 shutdown_connections 2024-04-25T00:16:18.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.057+0000 7fe82ad4f640 1 -- 172.21.15.107:0/2278778792 wait complete. 2024-04-25T00:16:18.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.057+0000 7fe82ad4f640 1 Processor -- start 2024-04-25T00:16:18.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.058+0000 7fe82ad4f640 1 -- start start 2024-04-25T00:16:18.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.058+0000 7fe82ad4f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe824102260 0x7fe824197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.058+0000 7fe82ad4f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe824104b90 0x7fe824198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.058+0000 7fe82ad4f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8241074c0 0x7fe82419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.058+0000 7fe82ad4f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe82410d400 con 0x7fe824102260 2024-04-25T00:16:18.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.058+0000 7fe82ad4f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe82410d280 con 0x7fe8241074c0 2024-04-25T00:16:18.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.058+0000 7fe82ad4f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe82410d580 con 0x7fe824104b90 2024-04-25T00:16:18.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.058+0000 7fe828aeb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe824102260 0x7fe824197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.058+0000 7fe828aeb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe824102260 0x7fe824197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57088/0 (socket says 172.21.15.107:57088) 2024-04-25T00:16:18.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.058+0000 7fe828aeb640 1 -- 172.21.15.107:0/1732453822 learned_addr learned my addr 172.21.15.107:0/1732453822 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:18.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.058+0000 7fe823fff640 1 --2- 172.21.15.107:0/1732453822 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe824104b90 0x7fe824198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.058+0000 7fe8292ec640 1 --2- 172.21.15.107:0/1732453822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8241074c0 0x7fe82419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.058+0000 7fe828aeb640 1 -- 172.21.15.107:0/1732453822 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe824104b90 msgr2=0x7fe824198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.058+0000 7fe828aeb640 1 --2- 172.21.15.107:0/1732453822 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe824104b90 0x7fe824198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.058+0000 7fe828aeb640 1 -- 172.21.15.107:0/1732453822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8241074c0 msgr2=0x7fe82419f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.058+0000 7fe828aeb640 1 --2- 172.21.15.107:0/1732453822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8241074c0 0x7fe82419f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.058+0000 7fe828aeb640 1 -- 172.21.15.107:0/1732453822 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe82419f6f0 con 0x7fe824102260 2024-04-25T00:16:18.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.058+0000 7fe8292ec640 1 --2- 172.21.15.107:0/1732453822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8241074c0 0x7fe82419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:18.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.058+0000 7fe828aeb640 1 --2- 172.21.15.107:0/1732453822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe824102260 0x7fe824197ad0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe81800d950 tx=0x7fe81800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:18.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.058+0000 7fe821ffb640 1 -- 172.21.15.107:0/1732453822 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe818017070 con 0x7fe824102260 2024-04-25T00:16:18.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.058+0000 7fe821ffb640 1 -- 172.21.15.107:0/1732453822 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe818009c30 con 0x7fe824102260 2024-04-25T00:16:18.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.058+0000 7fe82ad4f640 1 -- 172.21.15.107:0/1732453822 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe82419f9e0 con 0x7fe824102260 2024-04-25T00:16:18.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.058+0000 7fe821ffb640 1 -- 172.21.15.107:0/1732453822 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81800f6b0 con 0x7fe824102260 2024-04-25T00:16:18.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.059+0000 7fe82ad4f640 1 -- 172.21.15.107:0/1732453822 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe82419ff00 con 0x7fe824102260 2024-04-25T00:16:18.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.060+0000 7fe82ad4f640 1 -- 172.21.15.107:0/1732453822 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8240fd6e0 con 0x7fe824102260 2024-04-25T00:16:18.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.060+0000 7fe821ffb640 1 -- 172.21.15.107:0/1732453822 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fe8180109d0 con 0x7fe824102260 2024-04-25T00:16:18.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.061+0000 7fe821ffb640 1 --2- 172.21.15.107:0/1732453822 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe8140a2b90 0x7fe8140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.061+0000 7fe823fff640 1 --2- 172.21.15.107:0/1732453822 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe8140a2b90 0x7fe8140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.061+0000 7fe821ffb640 1 -- 172.21.15.107:0/1732453822 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7fe8180cf060 con 0x7fe824102260 2024-04-25T00:16:18.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.061+0000 7fe823fff640 1 --2- 172.21.15.107:0/1732453822 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe8140a2b90 0x7fe8140a5050 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe810005ab0 tx=0x7fe810000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:18.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.066+0000 7fe821ffb640 1 -- 172.21.15.107:0/1732453822 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe818009de0 con 0x7fe824102260 2024-04-25T00:16:18.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:18 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3686589652' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:16:18.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:18 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3879268060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:16:18.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:18 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3638941736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:18.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.245+0000 7fe82ad4f640 1 -- 172.21.15.107:0/1732453822 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe824062340 con 0x7fe824102260 2024-04-25T00:16:18.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.246+0000 7fe821ffb640 1 -- 172.21.15.107:0/1732453822 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24688 (secure 0 0 0) 0x7fe81809bc30 con 0x7fe824102260 2024-04-25T00:16:18.247 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:16:18.247 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":96,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:16:03.045295+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:59.672801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:00.957782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:14:05.065846+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8},{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:16:18.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.249+0000 7fe82ad4f640 1 -- 172.21.15.107:0/1732453822 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe8140a2b90 msgr2=0x7fe8140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.249+0000 7fe82ad4f640 1 --2- 172.21.15.107:0/1732453822 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe8140a2b90 0x7fe8140a5050 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe810005ab0 tx=0x7fe810000bc0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.249+0000 7fe82ad4f640 1 -- 172.21.15.107:0/1732453822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe824102260 msgr2=0x7fe824197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.249+0000 7fe82ad4f640 1 --2- 172.21.15.107:0/1732453822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe824102260 0x7fe824197ad0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe81800d950 tx=0x7fe81800de10 comp rx=0 tx=0).stop 2024-04-25T00:16:18.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.249+0000 7fe82ad4f640 1 -- 172.21.15.107:0/1732453822 shutdown_connections 2024-04-25T00:16:18.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.249+0000 7fe82ad4f640 1 --2- 172.21.15.107:0/1732453822 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fe8140a2b90 0x7fe8140a5050 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.249+0000 7fe82ad4f640 1 --2- 172.21.15.107:0/1732453822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8241074c0 0x7fe82419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.249+0000 7fe82ad4f640 1 --2- 172.21.15.107:0/1732453822 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe824104b90 0x7fe824198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.249+0000 7fe82ad4f640 1 --2- 172.21.15.107:0/1732453822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe824102260 0x7fe824197ad0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.249+0000 7fe82ad4f640 1 -- 172.21.15.107:0/1732453822 >> 172.21.15.107:0/1732453822 conn(0x7fe8240fbbf0 msgr2=0x7fe824102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:18.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.249+0000 7fe82ad4f640 1 -- 172.21.15.107:0/1732453822 shutdown_connections 2024-04-25T00:16:18.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.249+0000 7fe82ad4f640 1 -- 172.21.15.107:0/1732453822 wait complete. 2024-04-25T00:16:18.261 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T00:14:01.788955+0000', 'modified': '2024-04-25T00:14:28.529865+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24521, 'mds_1': 24493, 'mds_2': 24485}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.170:6835/2926641872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 2926641872}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 2926641872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.195:6835/2828335061', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 2828335061}, {'type': 'v1', 'addr': '172.21.15.195:6835', 'nonce': 2828335061}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.195:6837/2245799531', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6836', 'nonce': 2245799531}, {'type': 'v1', 'addr': '172.21.15.195:6837', 'nonce': 2245799531}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.170:6837/3616315471', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6836', 'nonce': 3616315471}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 3616315471}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.195:6839/4133864926', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6838', 'nonce': 4133864926}, {'type': 'v1', 'addr': '172.21.15.195:6839', 'nonce': 4133864926}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24521': {'gid': 24521, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.170:6839/3558784302', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 3558784302}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 3558784302}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24521, 'qdb_cluster': [24485, 24521, 24493]} 2024-04-25T00:16:18.261 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T00:16:18.261 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-25T00:16:18.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.360+0000 7f8ce1d12640 1 -- 172.21.15.107:0/1703346774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cdc101890 msgr2=0x7f8cdc101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.360+0000 7f8ce1d12640 1 --2- 172.21.15.107:0/1703346774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cdc101890 0x7f8cdc101d10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0009900 tx=0x7f8cd002a410 comp rx=0 tx=0).stop 2024-04-25T00:16:18.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.360+0000 7f8ce1d12640 1 -- 172.21.15.107:0/1703346774 shutdown_connections 2024-04-25T00:16:18.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.360+0000 7f8ce1d12640 1 --2- 172.21.15.107:0/1703346774 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cdc102250 0x7f8cdc108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.360+0000 7f8ce1d12640 1 --2- 172.21.15.107:0/1703346774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cdc101890 0x7f8cdc101d10 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.360+0000 7f8ce1d12640 1 --2- 172.21.15.107:0/1703346774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cdc1006a0 0x7f8cdc100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.360+0000 7f8ce1d12640 1 -- 172.21.15.107:0/1703346774 >> 172.21.15.107:0/1703346774 conn(0x7f8cdc0fbc30 msgr2=0x7f8cdc0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:18.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.360+0000 7f8ce1d12640 1 -- 172.21.15.107:0/1703346774 shutdown_connections 2024-04-25T00:16:18.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.360+0000 7f8ce1d12640 1 -- 172.21.15.107:0/1703346774 wait complete. 2024-04-25T00:16:18.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.360+0000 7f8ce1d12640 1 Processor -- start 2024-04-25T00:16:18.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.360+0000 7f8ce1d12640 1 -- start start 2024-04-25T00:16:18.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.360+0000 7f8ce1d12640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cdc1006a0 0x7f8cdc196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.360+0000 7f8ce1d12640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cdc101890 0x7f8cdc196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.361+0000 7f8ce1d12640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cdc102250 0x7f8cdc19df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.361+0000 7f8ce1d12640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc10c380 con 0x7f8cdc1006a0 2024-04-25T00:16:18.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.361+0000 7f8ce1d12640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc10c200 con 0x7f8cdc102250 2024-04-25T00:16:18.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.361+0000 7f8ce1d12640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc10c500 con 0x7f8cdc101890 2024-04-25T00:16:18.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.361+0000 7f8cdb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cdc1006a0 0x7f8cdc196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.361+0000 7f8cdb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cdc1006a0 0x7f8cdc196990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57098/0 (socket says 172.21.15.107:57098) 2024-04-25T00:16:18.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.361+0000 7f8cdb7fe640 1 -- 172.21.15.107:0/1479408351 learned_addr learned my addr 172.21.15.107:0/1479408351 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:18.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.361+0000 7f8cdaffd640 1 --2- 172.21.15.107:0/1479408351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cdc101890 0x7f8cdc196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.361+0000 7f8cdbfff640 1 --2- 172.21.15.107:0/1479408351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cdc102250 0x7f8cdc19df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.361+0000 7f8cdb7fe640 1 -- 172.21.15.107:0/1479408351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cdc101890 msgr2=0x7f8cdc196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.361+0000 7f8cdb7fe640 1 --2- 172.21.15.107:0/1479408351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cdc101890 0x7f8cdc196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.361+0000 7f8cdb7fe640 1 -- 172.21.15.107:0/1479408351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cdc102250 msgr2=0x7f8cdc19df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.361+0000 7f8cdb7fe640 1 --2- 172.21.15.107:0/1479408351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cdc102250 0x7f8cdc19df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.361+0000 7f8cdb7fe640 1 -- 172.21.15.107:0/1479408351 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cdc19e5b0 con 0x7f8cdc1006a0 2024-04-25T00:16:18.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.361+0000 7f8cdaffd640 1 --2- 172.21.15.107:0/1479408351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cdc101890 0x7f8cdc196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:18.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.361+0000 7f8cdbfff640 1 --2- 172.21.15.107:0/1479408351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cdc102250 0x7f8cdc19df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:18.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.361+0000 7f8cdb7fe640 1 --2- 172.21.15.107:0/1479408351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cdc1006a0 0x7f8cdc196990 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00c970 tx=0x7f8ccc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:18.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.361+0000 7f8cd8ff9640 1 -- 172.21.15.107:0/1479408351 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc007c10 con 0x7f8cdc1006a0 2024-04-25T00:16:18.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.361+0000 7f8ce1d12640 1 -- 172.21.15.107:0/1479408351 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cdc19e8a0 con 0x7f8cdc1006a0 2024-04-25T00:16:18.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.361+0000 7f8cd8ff9640 1 -- 172.21.15.107:0/1479408351 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ccc007db0 con 0x7f8cdc1006a0 2024-04-25T00:16:18.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.361+0000 7f8cd8ff9640 1 -- 172.21.15.107:0/1479408351 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc00f8a0 con 0x7f8cdc1006a0 2024-04-25T00:16:18.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.361+0000 7f8ce1d12640 1 -- 172.21.15.107:0/1479408351 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cdc19eec0 con 0x7f8cdc1006a0 2024-04-25T00:16:18.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.363+0000 7f8cd8ff9640 1 -- 172.21.15.107:0/1479408351 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f8ccc00fb00 con 0x7f8cdc1006a0 2024-04-25T00:16:18.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.364+0000 7f8ce4a16740 1 Processor -- start 2024-04-25T00:16:18.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.364+0000 7f8ce4a16740 1 -- start start 2024-04-25T00:16:18.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.364+0000 7f8ce4a16740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56397178b650 0x56397178ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.364+0000 7f8ce4a16740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56397178c040 0x563971790b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.364+0000 7f8ce4a16740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639717911b0 0x563971793640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.364+0000 7f8ce4a16740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5639716ecf00 con 0x5639717911b0 2024-04-25T00:16:18.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.364+0000 7f8ce4a16740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563971695750 con 0x56397178c040 2024-04-25T00:16:18.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.364+0000 7f8ce4a16740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5639716e8ec0 con 0x56397178b650 2024-04-25T00:16:18.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.364+0000 7f8cdaffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639717911b0 0x563971793640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.364+0000 7f8cdaffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639717911b0 0x563971793640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57102/0 (socket says 172.21.15.107:57102) 2024-04-25T00:16:18.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.364+0000 7f8cdaffd640 1 -- 172.21.15.107:0/2772021176 learned_addr learned my addr 172.21.15.107:0/2772021176 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:18.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.364+0000 7f8cd8ff9640 1 --2- 172.21.15.107:0/1479408351 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8cc40a2b90 0x7f8cc40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.364+0000 7f8cdbfff640 1 --2- 172.21.15.107:0/2772021176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56397178b650 0x56397178ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.364+0000 7f8cdb7fe640 1 --2- 172.21.15.107:0/2772021176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56397178c040 0x563971790b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.364+0000 7f8cdbfff640 1 --2- 172.21.15.107:0/1479408351 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8cc40a2b90 0x7f8cc40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.364+0000 7f8cd8ff9640 1 -- 172.21.15.107:0/1479408351 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7f8ccc0cf7a0 con 0x7f8cdc1006a0 2024-04-25T00:16:18.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.364+0000 7f8cdaffd640 1 -- 172.21.15.107:0/2772021176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56397178b650 msgr2=0x56397178ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:18.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.364+0000 7f8cdaffd640 1 --2- 172.21.15.107:0/2772021176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56397178b650 0x56397178ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.364+0000 7f8cdaffd640 1 -- 172.21.15.107:0/2772021176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56397178c040 msgr2=0x563971790b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:18.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.364+0000 7f8cdaffd640 1 --2- 172.21.15.107:0/2772021176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56397178c040 0x563971790b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.364+0000 7f8cdaffd640 1 -- 172.21.15.107:0/2772021176 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639714012d0 con 0x5639717911b0 2024-04-25T00:16:18.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.364+0000 7f8cdb7fe640 1 --2- 172.21.15.107:0/2772021176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56397178c040 0x563971790b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:18.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.364+0000 7f8cdaffd640 1 --2- 172.21.15.107:0/2772021176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639717911b0 0x563971793640 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0000c00 tx=0x7f8cd0032df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5384db2d248ee9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:18.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.365+0000 7f8cdbfff640 1 --2- 172.21.15.107:0/1479408351 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8cc40a2b90 0x7f8cc40a5050 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x56397178c8a0 tx=0x7f8cd4010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:18.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.365+0000 7f8cc1ffb640 1 -- 172.21.15.107:0/2772021176 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd003f070 con 0x5639717911b0 2024-04-25T00:16:18.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.365+0000 7f8cc1ffb640 1 -- 172.21.15.107:0/2772021176 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cd00318b0 con 0x5639717911b0 2024-04-25T00:16:18.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.365+0000 7f8cc1ffb640 1 -- 172.21.15.107:0/2772021176 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd0035070 con 0x5639717911b0 2024-04-25T00:16:18.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.365+0000 7f8ce4a16740 1 -- 172.21.15.107:0/2772021176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639717911b0 msgr2=0x563971793640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.365+0000 7f8ce4a16740 1 --2- 172.21.15.107:0/2772021176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639717911b0 0x563971793640 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0000c00 tx=0x7f8cd0032df0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.365+0000 7f8ce4a16740 1 -- 172.21.15.107:0/2772021176 shutdown_connections 2024-04-25T00:16:18.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.365+0000 7f8ce4a16740 1 --2- 172.21.15.107:0/2772021176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639717911b0 0x563971793640 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.365+0000 7f8ce4a16740 1 --2- 172.21.15.107:0/2772021176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56397178c040 0x563971790b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.365+0000 7f8ce4a16740 1 --2- 172.21.15.107:0/2772021176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56397178b650 0x56397178ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.365+0000 7f8ce4a16740 1 -- 172.21.15.107:0/2772021176 >> 172.21.15.107:0/2772021176 conn(0x563971786cc0 msgr2=0x5639717870c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:18.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.365+0000 7f8ce4a16740 1 -- 172.21.15.107:0/2772021176 shutdown_connections 2024-04-25T00:16:18.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.365+0000 7f8ce4a16740 1 -- 172.21.15.107:0/2772021176 wait complete. 2024-04-25T00:16:18.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.366+0000 7f8ce4a16740 1 -- start start 2024-04-25T00:16:18.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.366+0000 7f8ce4a16740 1 Processor -- start 2024-04-25T00:16:18.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.366+0000 7f8ce4a16740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56397178b650 0x5639717959e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.366+0000 7f8ce4a16740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56397178c040 0x563971795f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.366+0000 7f8ce4a16740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639717911b0 0x56397179cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.366+0000 7f8ce4a16740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563971794100 con 0x56397178c040 2024-04-25T00:16:18.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.366+0000 7f8ce4a16740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5639716a8cf0 con 0x5639717911b0 2024-04-25T00:16:18.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.366+0000 7f8ce4a16740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563971794280 con 0x56397178b650 2024-04-25T00:16:18.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.366+0000 7f8ce4a16740 10 client.0 _ll_register_callbacks cb 0x5639716a86e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:16:18.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.366+0000 7f8cdaffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56397178c040 0x563971795f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.366+0000 7f8cdaffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56397178c040 0x563971795f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57112/0 (socket says 172.21.15.107:57112) 2024-04-25T00:16:18.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.366+0000 7f8cdaffd640 1 -- 172.21.15.107:0/2720607717 learned_addr learned my addr 172.21.15.107:0/2720607717 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:18.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.366+0000 7f8cdb7fe640 1 --2- 172.21.15.107:0/2720607717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56397178b650 0x5639717959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.366+0000 7f8cdbfff640 1 --2- 172.21.15.107:0/2720607717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639717911b0 0x56397179cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.366+0000 7f8cdaffd640 1 -- 172.21.15.107:0/2720607717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56397178b650 msgr2=0x5639717959e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:18.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.366+0000 7f8cdaffd640 1 --2- 172.21.15.107:0/2720607717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56397178b650 0x5639717959e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.366+0000 7f8cdaffd640 1 -- 172.21.15.107:0/2720607717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639717911b0 msgr2=0x56397179cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:18.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.366+0000 7f8cdaffd640 1 --2- 172.21.15.107:0/2720607717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639717911b0 0x56397179cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.366+0000 7f8cdaffd640 1 -- 172.21.15.107:0/2720607717 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639714cb6b0 con 0x56397178c040 2024-04-25T00:16:18.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.367+0000 7f8cdaffd640 1 --2- 172.21.15.107:0/2720607717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56397178c040 0x563971795f20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8cd002a450 tx=0x7f8cd00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91a5e9f600db341 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:18.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.367+0000 7f8cbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:16:18.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.367+0000 7f8cbaffd640 1 -- 172.21.15.107:0/2720607717 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd003f050 con 0x56397178c040 2024-04-25T00:16:18.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.367+0000 7f8cbaffd640 1 -- 172.21.15.107:0/2720607717 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7f8cd0003a00 con 0x56397178c040 2024-04-25T00:16:18.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.367+0000 7f8ce4a16740 1 -- 172.21.15.107:0/2720607717 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cd00318b0 con 0x56397178c040 2024-04-25T00:16:18.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.367+0000 7f8cbaffd640 1 -- 172.21.15.107:0/2720607717 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8cd0035040 con 0x56397178c040 2024-04-25T00:16:18.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.367+0000 7f8ce4a16740 10 client.15390 fetch_fsmap learned FSMap version 23 2024-04-25T00:16:18.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.367+0000 7f8ce4a16740 1 -- 172.21.15.107:0/2720607717 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x56397179feb0 con 0x56397178c040 2024-04-25T00:16:18.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.367+0000 7f8cbaffd640 1 -- 172.21.15.107:0/2720607717 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cd00318b0 con 0x56397178c040 2024-04-25T00:16:18.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.367+0000 7f8cbaffd640 1 -- 172.21.15.107:0/2720607717 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f8cd004dc50 con 0x56397178c040 2024-04-25T00:16:18.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.367+0000 7f8ce4a16740 10 client.15390 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:16:18.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.367+0000 7f8ce4a16740 10 client.15390 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T00:16:18.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.367+0000 7f8ce4a16740 20 client.15390 populate_metadata read hostname 'smithi107' 2024-04-25T00:16:18.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.367+0000 7f8ce4a16740 1 --2- 172.21.15.107:0/2720607717 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5639717a2070 0x5639717a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.367+0000 7f8ce4a16740 4 client.15390 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:16:18.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.367+0000 7f8ce4a16740 1 -- 172.21.15.107:0/2720607717 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639716fa440 con 0x5639717a2070 2024-04-25T00:16:18.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.368+0000 7f8cdb7fe640 1 --2- 172.21.15.107:0/2720607717 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5639717a2070 0x5639717a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.368+0000 7f8cbaffd640 10 client.15390 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:16:18.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.368+0000 7f8cdb7fe640 1 --2- 172.21.15.107:0/2720607717 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5639717a2070 0x5639717a34e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1651f78d655f4ff3 server_cookie=fb3bdf15c27a9233 in_seq=0 out_seq=0 2024-04-25T00:16:18.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.369+0000 7f8cbaffd640 1 -- 172.21.15.107:0/2720607717 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5639716fa440 con 0x5639717a2070 2024-04-25T00:16:18.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.369+0000 7f8cbaffd640 10 client.15390 handle_command_reply: tid=0 2024-04-25T00:16:18.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.369+0000 7f8ce4a16740 1 client.15390 shutdown 2024-04-25T00:16:18.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.369+0000 7f8ce4a16740 1 -- 172.21.15.107:0/2720607717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56397178c040 msgr2=0x563971795f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.369+0000 7f8ce4a16740 1 --2- 172.21.15.107:0/2720607717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56397178c040 0x563971795f20 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8cd002a450 tx=0x7f8cd00042f0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.369+0000 7f8ce4a16740 1 -- 172.21.15.107:0/2720607717 shutdown_connections 2024-04-25T00:16:18.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.369+0000 7f8ce4a16740 1 --2- 172.21.15.107:0/2720607717 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5639717a2070 0x5639717a34e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.369+0000 7f8ce4a16740 1 --2- 172.21.15.107:0/2720607717 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639717911b0 0x56397179cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.369+0000 7f8ce4a16740 1 --2- 172.21.15.107:0/2720607717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56397178c040 0x563971795f20 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.369+0000 7f8cbaffd640 0 client.15390 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:16:18.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.369+0000 7f8ce4a16740 1 --2- 172.21.15.107:0/2720607717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56397178b650 0x5639717959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.369+0000 7f8ce4a16740 1 -- 172.21.15.107:0/2720607717 >> 172.21.15.107:0/2720607717 conn(0x563971786cc0 msgr2=0x563971783ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:18.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.369+0000 7f8ce4a16740 1 -- 172.21.15.107:0/2720607717 shutdown_connections 2024-04-25T00:16:18.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.369+0000 7f8ce4a16740 1 -- 172.21.15.107:0/2720607717 wait complete. 2024-04-25T00:16:18.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.370+0000 7f8ce4a16740 20 client.15390 trim_cache size 0 max 16384 2024-04-25T00:16:18.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.387+0000 7f8ce4a16740 1 Processor -- start 2024-04-25T00:16:18.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.387+0000 7f8ce4a16740 1 -- start start 2024-04-25T00:16:18.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.387+0000 7f8ce4a16740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56397178b650 0x56397177f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.387+0000 7f8ce4a16740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56397178c040 0x56397179cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.387+0000 7f8ce4a16740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639717911b0 0x563971783250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.387+0000 7f8ce4a16740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5639717a1820 con 0x5639717911b0 2024-04-25T00:16:18.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.387+0000 7f8ce4a16740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5639717a1ef0 con 0x56397178b650 2024-04-25T00:16:18.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.387+0000 7f8ce4a16740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5639717a25c0 con 0x56397178c040 2024-04-25T00:16:18.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.387+0000 7f8cdbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639717911b0 0x563971783250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.388+0000 7f8cdbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639717911b0 0x563971783250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57122/0 (socket says 172.21.15.107:57122) 2024-04-25T00:16:18.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.388+0000 7f8cdbfff640 1 -- 172.21.15.107:0/4276937597 learned_addr learned my addr 172.21.15.107:0/4276937597 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:18.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.388+0000 7f8cdb7fe640 1 --2- 172.21.15.107:0/4276937597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56397178b650 0x56397177f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.388+0000 7f8cdaffd640 1 --2- 172.21.15.107:0/4276937597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56397178c040 0x56397179cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.388+0000 7f8cdbfff640 1 -- 172.21.15.107:0/4276937597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56397178c040 msgr2=0x56397179cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:18.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.388+0000 7f8cdbfff640 1 --2- 172.21.15.107:0/4276937597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56397178c040 0x56397179cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.388+0000 7f8cdbfff640 1 -- 172.21.15.107:0/4276937597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56397178b650 msgr2=0x56397177f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:18.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.388+0000 7f8cdbfff640 1 --2- 172.21.15.107:0/4276937597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56397178b650 0x56397177f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.388+0000 7f8cdbfff640 1 -- 172.21.15.107:0/4276937597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56397178c8d0 con 0x5639717911b0 2024-04-25T00:16:18.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.388+0000 7f8cdbfff640 1 --2- 172.21.15.107:0/4276937597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639717911b0 0x563971783250 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8cd400b870 tx=0x7f8cd4009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f987255fc934bfd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:18.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.388+0000 7f8cc17fa640 1 -- 172.21.15.107:0/4276937597 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4018980 con 0x5639717911b0 2024-04-25T00:16:18.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.388+0000 7f8cc17fa640 1 -- 172.21.15.107:0/4276937597 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cd4027070 con 0x5639717911b0 2024-04-25T00:16:18.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.388+0000 7f8cc17fa640 1 -- 172.21.15.107:0/4276937597 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4017390 con 0x5639717911b0 2024-04-25T00:16:18.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.389+0000 7f8ce4a16740 1 -- 172.21.15.107:0/4276937597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639717911b0 msgr2=0x563971783250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.389+0000 7f8ce4a16740 1 --2- 172.21.15.107:0/4276937597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639717911b0 0x563971783250 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8cd400b870 tx=0x7f8cd4009380 comp rx=0 tx=0).stop 2024-04-25T00:16:18.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.389+0000 7f8ce4a16740 1 -- 172.21.15.107:0/4276937597 shutdown_connections 2024-04-25T00:16:18.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.389+0000 7f8ce4a16740 1 --2- 172.21.15.107:0/4276937597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639717911b0 0x563971783250 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.389+0000 7f8ce4a16740 1 --2- 172.21.15.107:0/4276937597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56397178c040 0x56397179cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.389+0000 7f8ce4a16740 1 --2- 172.21.15.107:0/4276937597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56397178b650 0x56397177f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.389+0000 7f8ce4a16740 1 -- 172.21.15.107:0/4276937597 >> 172.21.15.107:0/4276937597 conn(0x563971786cc0 msgr2=0x5639717870a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:18.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.389+0000 7f8ce4a16740 1 -- 172.21.15.107:0/4276937597 shutdown_connections 2024-04-25T00:16:18.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.389+0000 7f8ce4a16740 1 -- 172.21.15.107:0/4276937597 wait complete. 2024-04-25T00:16:18.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.389+0000 7f8ce4a16740 1 -- start start 2024-04-25T00:16:18.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.389+0000 7f8ce4a16740 1 Processor -- start 2024-04-25T00:16:18.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.389+0000 7f8ce4a16740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56397178b650 0x563971799150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.389+0000 7f8ce4a16740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56397178c040 0x5639717996f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.389+0000 7f8ce4a16740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639717911b0 0x56397179c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.389+0000 7f8ce4a16740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5639717a3360 con 0x5639717911b0 2024-04-25T00:16:18.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.389+0000 7f8ce4a16740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5639717a2c90 con 0x56397178b650 2024-04-25T00:16:18.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.389+0000 7f8ce4a16740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5639717a3a30 con 0x56397178c040 2024-04-25T00:16:18.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.389+0000 7f8ce4a16740 10 client.0 _ll_register_callbacks cb 0x7f8cd00316b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:16:18.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.390+0000 7f8cdbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639717911b0 0x56397179c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.390+0000 7f8cdbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639717911b0 0x56397179c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57134/0 (socket says 172.21.15.107:57134) 2024-04-25T00:16:18.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.390+0000 7f8cdbfff640 1 -- 172.21.15.107:0/2076219162 learned_addr learned my addr 172.21.15.107:0/2076219162 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:18.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.390+0000 7f8cdb7fe640 1 --2- 172.21.15.107:0/2076219162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56397178b650 0x563971799150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.390+0000 7f8cdaffd640 1 --2- 172.21.15.107:0/2076219162 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56397178c040 0x5639717996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.390+0000 7f8cdbfff640 1 -- 172.21.15.107:0/2076219162 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56397178c040 msgr2=0x5639717996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:18.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.390+0000 7f8cdbfff640 1 --2- 172.21.15.107:0/2076219162 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56397178c040 0x5639717996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.390+0000 7f8cdbfff640 1 -- 172.21.15.107:0/2076219162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56397178b650 msgr2=0x563971799150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:18.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.390+0000 7f8cdbfff640 1 --2- 172.21.15.107:0/2076219162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56397178b650 0x563971799150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.390+0000 7f8cdbfff640 1 -- 172.21.15.107:0/2076219162 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56397179ca40 con 0x5639717911b0 2024-04-25T00:16:18.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.390+0000 7f8cdaffd640 1 --2- 172.21.15.107:0/2076219162 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56397178c040 0x5639717996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:18.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.390+0000 7f8cdbfff640 1 --2- 172.21.15.107:0/2076219162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639717911b0 0x56397179c480 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4022b70 tx=0x7f8cd4022ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca98789395c9b570 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:18.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.390+0000 7f8cbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:16:18.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.390+0000 7f8cbaffd640 1 -- 172.21.15.107:0/2076219162 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4031070 con 0x5639717911b0 2024-04-25T00:16:18.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.390+0000 7f8cbaffd640 1 -- 172.21.15.107:0/2076219162 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7f8cd4022400 con 0x5639717911b0 2024-04-25T00:16:18.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.390+0000 7f8ce4a16740 1 -- 172.21.15.107:0/2076219162 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cd4027070 con 0x5639717911b0 2024-04-25T00:16:18.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.390+0000 7f8cbaffd640 1 -- 172.21.15.107:0/2076219162 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f8cd403d050 con 0x5639717911b0 2024-04-25T00:16:18.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.390+0000 7f8ce4a16740 10 client.15396 fetch_fsmap learned FSMap version 23 2024-04-25T00:16:18.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.390+0000 7f8ce4a16740 1 -- 172.21.15.107:0/2076219162 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x563971808900 con 0x5639717911b0 2024-04-25T00:16:18.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.390+0000 7f8cbaffd640 1 -- 172.21.15.107:0/2076219162 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cd4027070 con 0x5639717911b0 2024-04-25T00:16:18.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.391+0000 7f8cbaffd640 1 -- 172.21.15.107:0/2076219162 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f8cd4017070 con 0x5639717911b0 2024-04-25T00:16:18.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.391+0000 7f8ce4a16740 10 client.15396 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:16:18.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.391+0000 7f8ce4a16740 10 client.15396 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T00:16:18.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.391+0000 7f8ce4a16740 20 client.15396 populate_metadata read hostname 'smithi107' 2024-04-25T00:16:18.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.391+0000 7f8ce4a16740 1 --2- 172.21.15.107:0/2076219162 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x56397180d1a0 0x56397180beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.391+0000 7f8ce4a16740 4 client.15396 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:16:18.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.391+0000 7f8ce4a16740 1 -- 172.21.15.107:0/2076219162 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "status"}) -- 0x5639716f4790 con 0x56397180d1a0 2024-04-25T00:16:18.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.391+0000 7f8cdb7fe640 1 --2- 172.21.15.107:0/2076219162 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x56397180d1a0 0x56397180beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.391+0000 7f8cbaffd640 10 client.15396 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:16:18.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.391+0000 7f8cdb7fe640 1 --2- 172.21.15.107:0/2076219162 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x56397180d1a0 0x56397180beb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=570f7312925b1abf server_cookie=fa2482797d55b657 in_seq=0 out_seq=0 2024-04-25T00:16:18.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.392+0000 7f8cbaffd640 1 -- 172.21.15.107:0/2076219162 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5639716f4790 con 0x56397180d1a0 2024-04-25T00:16:18.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.392+0000 7f8cbaffd640 10 client.15396 handle_command_reply: tid=0 2024-04-25T00:16:18.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.392+0000 7f8ce4a16740 1 client.15396 shutdown 2024-04-25T00:16:18.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.392+0000 7f8ce4a16740 1 -- 172.21.15.107:0/2076219162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639717911b0 msgr2=0x56397179c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.392+0000 7f8ce4a16740 1 --2- 172.21.15.107:0/2076219162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639717911b0 0x56397179c480 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4022b70 tx=0x7f8cd4022ed0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.392+0000 7f8ce4a16740 1 -- 172.21.15.107:0/2076219162 shutdown_connections 2024-04-25T00:16:18.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.392+0000 7f8ce4a16740 1 --2- 172.21.15.107:0/2076219162 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x56397180d1a0 0x56397180beb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.392+0000 7f8ce4a16740 1 --2- 172.21.15.107:0/2076219162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639717911b0 0x56397179c480 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.392+0000 7f8ce4a16740 1 --2- 172.21.15.107:0/2076219162 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56397178c040 0x5639717996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.392+0000 7f8cbaffd640 0 client.15396 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:16:18.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.392+0000 7f8ce4a16740 1 --2- 172.21.15.107:0/2076219162 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56397178b650 0x563971799150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.392+0000 7f8ce4a16740 1 -- 172.21.15.107:0/2076219162 >> 172.21.15.107:0/2076219162 conn(0x563971786cc0 msgr2=0x56397177f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:18.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.392+0000 7f8ce4a16740 1 -- 172.21.15.107:0/2076219162 shutdown_connections 2024-04-25T00:16:18.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.392+0000 7f8ce4a16740 1 -- 172.21.15.107:0/2076219162 wait complete. 2024-04-25T00:16:18.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.392+0000 7f8ce4a16740 20 client.15396 trim_cache size 0 max 16384 2024-04-25T00:16:18.400 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:16:18.400 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "79602f58-0297-11ef-bc93-c7b262605968", 2024-04-25T00:16:18.400 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 2, 2024-04-25T00:16:18.400 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24485, 2024-04-25T00:16:18.400 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-04-25T00:16:18.400 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-04-25T00:16:18.400 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-04-25T00:16:18.400 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 110.867250208, 2024-04-25T00:16:18.401 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 23, 2024-04-25T00:16:18.401 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 96, 2024-04-25T00:16:18.401 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 92, 2024-04-25T00:16:18.401 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 152.21862052099999 2024-04-25T00:16:18.401 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:16:18.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.393+0000 7f8ce1d12640 1 -- 172.21.15.107:0/1479408351 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8cc40a2b90 msgr2=0x7f8cc40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.393+0000 7f8ce1d12640 1 --2- 172.21.15.107:0/1479408351 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8cc40a2b90 0x7f8cc40a5050 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x56397178c8a0 tx=0x7f8cd4010040 comp rx=0 tx=0).stop 2024-04-25T00:16:18.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.393+0000 7f8ce1d12640 1 -- 172.21.15.107:0/1479408351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cdc1006a0 msgr2=0x7f8cdc196990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.393+0000 7f8ce1d12640 1 --2- 172.21.15.107:0/1479408351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cdc1006a0 0x7f8cdc196990 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00c970 tx=0x7f8ccc00ce30 comp rx=0 tx=0).stop 2024-04-25T00:16:18.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.393+0000 7f8ce1d12640 1 -- 172.21.15.107:0/1479408351 shutdown_connections 2024-04-25T00:16:18.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.393+0000 7f8ce1d12640 1 --2- 172.21.15.107:0/1479408351 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8cc40a2b90 0x7f8cc40a5050 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.393+0000 7f8ce1d12640 1 --2- 172.21.15.107:0/1479408351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8cdc102250 0x7f8cdc19df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.393+0000 7f8ce1d12640 1 --2- 172.21.15.107:0/1479408351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8cdc101890 0x7f8cdc196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.393+0000 7f8ce1d12640 1 --2- 172.21.15.107:0/1479408351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8cdc1006a0 0x7f8cdc196990 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.393+0000 7f8ce1d12640 1 -- 172.21.15.107:0/1479408351 >> 172.21.15.107:0/1479408351 conn(0x7f8cdc0fbc30 msgr2=0x7f8cdc104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:18.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.393+0000 7f8ce1d12640 1 -- 172.21.15.107:0/1479408351 shutdown_connections 2024-04-25T00:16:18.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.393+0000 7f8ce1d12640 1 -- 172.21.15.107:0/1479408351 wait complete. 2024-04-25T00:16:18.404 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-25T00:16:18.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:18 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3686589652' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:16:18.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:18 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3879268060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:16:18.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:18 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3638941736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:18.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.504+0000 7f0104b42640 1 -- 172.21.15.107:0/3775066731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01001006a0 msgr2=0x7f0100100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.504+0000 7f0104b42640 1 --2- 172.21.15.107:0/3775066731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01001006a0 0x7f0100100ac0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009900 tx=0x7f00f402a410 comp rx=0 tx=0).stop 2024-04-25T00:16:18.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.504+0000 7f0104b42640 1 -- 172.21.15.107:0/3775066731 shutdown_connections 2024-04-25T00:16:18.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.504+0000 7f0104b42640 1 --2- 172.21.15.107:0/3775066731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0100102250 0x7f0100108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.504+0000 7f0104b42640 1 --2- 172.21.15.107:0/3775066731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0100101890 0x7f0100101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.504+0000 7f0104b42640 1 --2- 172.21.15.107:0/3775066731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01001006a0 0x7f0100100ac0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.504+0000 7f0104b42640 1 -- 172.21.15.107:0/3775066731 >> 172.21.15.107:0/3775066731 conn(0x7f01000fbc10 msgr2=0x7f01000fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:18.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.504+0000 7f0104b42640 1 -- 172.21.15.107:0/3775066731 shutdown_connections 2024-04-25T00:16:18.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.504+0000 7f0104b42640 1 -- 172.21.15.107:0/3775066731 wait complete. 2024-04-25T00:16:18.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.504+0000 7f0104b42640 1 Processor -- start 2024-04-25T00:16:18.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.504+0000 7f0104b42640 1 -- start start 2024-04-25T00:16:18.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.505+0000 7f0104b42640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01001006a0 0x7f0100197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.505+0000 7f0104b42640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0100101890 0x7f01001980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.505+0000 7f0104b42640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0100102250 0x7f010019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.505+0000 7f0104b42640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f010010c320 con 0x7f0100102250 2024-04-25T00:16:18.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.505+0000 7f0104b42640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f010010c1a0 con 0x7f0100101890 2024-04-25T00:16:18.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.505+0000 7f0104b42640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f010010c4a0 con 0x7f01001006a0 2024-04-25T00:16:18.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.505+0000 7f00fed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0100102250 0x7f010019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.505+0000 7f00fed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0100102250 0x7f010019f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57144/0 (socket says 172.21.15.107:57144) 2024-04-25T00:16:18.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.505+0000 7f00fed9d640 1 -- 172.21.15.107:0/1208141106 learned_addr learned my addr 172.21.15.107:0/1208141106 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:18.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.505+0000 7f00fe59c640 1 --2- 172.21.15.107:0/1208141106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01001006a0 0x7f0100197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.505+0000 7f00fdd9b640 1 --2- 172.21.15.107:0/1208141106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0100101890 0x7f01001980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.505+0000 7f00fed9d640 1 -- 172.21.15.107:0/1208141106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01001006a0 msgr2=0x7f0100197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.505+0000 7f00fed9d640 1 --2- 172.21.15.107:0/1208141106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01001006a0 0x7f0100197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.505+0000 7f00fed9d640 1 -- 172.21.15.107:0/1208141106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0100101890 msgr2=0x7f01001980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.505+0000 7f00fed9d640 1 --2- 172.21.15.107:0/1208141106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0100101890 0x7f01001980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.505+0000 7f00fed9d640 1 -- 172.21.15.107:0/1208141106 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010019f7a0 con 0x7f0100102250 2024-04-25T00:16:18.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.505+0000 7f00fdd9b640 1 --2- 172.21.15.107:0/1208141106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0100101890 0x7f01001980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:18.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.505+0000 7f00fed9d640 1 --2- 172.21.15.107:0/1208141106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0100102250 0x7f010019f130 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f00f800eaa0 tx=0x7f00f800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:18.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.505+0000 7f00ef7fe640 1 -- 172.21.15.107:0/1208141106 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f800cdb0 con 0x7f0100102250 2024-04-25T00:16:18.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.505+0000 7f00ef7fe640 1 -- 172.21.15.107:0/1208141106 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00f8005020 con 0x7f0100102250 2024-04-25T00:16:18.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.505+0000 7f0104b42640 1 -- 172.21.15.107:0/1208141106 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f010019fa90 con 0x7f0100102250 2024-04-25T00:16:18.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.505+0000 7f00ef7fe640 1 -- 172.21.15.107:0/1208141106 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f80116b0 con 0x7f0100102250 2024-04-25T00:16:18.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.505+0000 7f0104b42640 1 -- 172.21.15.107:0/1208141106 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f010019ffe0 con 0x7f0100102250 2024-04-25T00:16:18.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.508+0000 7f00ef7fe640 1 -- 172.21.15.107:0/1208141106 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f00f8005560 con 0x7f0100102250 2024-04-25T00:16:18.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.508+0000 7f010784d740 1 Processor -- start 2024-04-25T00:16:18.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.508+0000 7f010784d740 1 -- start start 2024-04-25T00:16:18.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.508+0000 7f010784d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5642840b5440 0x5642840b5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.508+0000 7f010784d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642840b5e30 0x5642840ba930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.509+0000 7f010784d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840bafa0 0x5642840bd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.509+0000 7f010784d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564284018f00 con 0x5642840bafa0 2024-04-25T00:16:18.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.509+0000 7f010784d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564283fc1750 con 0x5642840b5e30 2024-04-25T00:16:18.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.509+0000 7f010784d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564284014ec0 con 0x5642840b5440 2024-04-25T00:16:18.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.509+0000 7f00fdd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840bafa0 0x5642840bd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.509+0000 7f00fdd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840bafa0 0x5642840bd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57150/0 (socket says 172.21.15.107:57150) 2024-04-25T00:16:18.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.509+0000 7f00fdd9b640 1 -- 172.21.15.107:0/679372960 learned_addr learned my addr 172.21.15.107:0/679372960 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:18.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.509+0000 7f00ef7fe640 1 --2- 172.21.15.107:0/1208141106 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f00e80a2b90 0x7f00e80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.509+0000 7f00fe59c640 1 --2- 172.21.15.107:0/679372960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642840b5e30 0x5642840ba930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.509+0000 7f00fed9d640 1 --2- 172.21.15.107:0/1208141106 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f00e80a2b90 0x7f00e80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.509+0000 7f00fdd9b640 1 --2- 172.21.15.107:0/679372960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5642840b5440 0x5642840b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.509+0000 7f00fdd9b640 1 -- 172.21.15.107:0/679372960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5642840b5440 msgr2=0x5642840b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:18.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.509+0000 7f00fdd9b640 1 --2- 172.21.15.107:0/679372960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5642840b5440 0x5642840b5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.509+0000 7f00ef7fe640 1 -- 172.21.15.107:0/1208141106 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7f00f80cf6f0 con 0x7f0100102250 2024-04-25T00:16:18.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.509+0000 7f00fdd9b640 1 -- 172.21.15.107:0/679372960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642840b5e30 msgr2=0x5642840ba930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:18.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.509+0000 7f00fdd9b640 1 --2- 172.21.15.107:0/679372960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642840b5e30 0x5642840ba930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.509+0000 7f00fdd9b640 1 -- 172.21.15.107:0/679372960 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564283d2d2d0 con 0x5642840bafa0 2024-04-25T00:16:18.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.509+0000 7f00fdd9b640 1 --2- 172.21.15.107:0/679372960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840bafa0 0x5642840bd430 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f00f0005d00 tx=0x7f00f0009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9791de84750a86d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:18.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.509+0000 7f00fed9d640 1 --2- 172.21.15.107:0/1208141106 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f00e80a2b90 0x7f00e80a5050 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f00f809bb50 tx=0x7f00f806ec80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:18.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.509+0000 7f00ecff9640 1 -- 172.21.15.107:0/679372960 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f00150f0 con 0x5642840bafa0 2024-04-25T00:16:18.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.509+0000 7f00ecff9640 1 -- 172.21.15.107:0/679372960 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00f0005500 con 0x5642840bafa0 2024-04-25T00:16:18.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.509+0000 7f00ecff9640 1 -- 172.21.15.107:0/679372960 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f00136c0 con 0x5642840bafa0 2024-04-25T00:16:18.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.510+0000 7f010784d740 1 -- 172.21.15.107:0/679372960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840bafa0 msgr2=0x5642840bd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.510+0000 7f010784d740 1 --2- 172.21.15.107:0/679372960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840bafa0 0x5642840bd430 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f00f0005d00 tx=0x7f00f0009640 comp rx=0 tx=0).stop 2024-04-25T00:16:18.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.510+0000 7f010784d740 1 -- 172.21.15.107:0/679372960 shutdown_connections 2024-04-25T00:16:18.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.510+0000 7f010784d740 1 --2- 172.21.15.107:0/679372960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840bafa0 0x5642840bd430 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.510+0000 7f010784d740 1 --2- 172.21.15.107:0/679372960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642840b5e30 0x5642840ba930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.510+0000 7f010784d740 1 --2- 172.21.15.107:0/679372960 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5642840b5440 0x5642840b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.510+0000 7f010784d740 1 -- 172.21.15.107:0/679372960 >> 172.21.15.107:0/679372960 conn(0x5642840b0ab0 msgr2=0x5642840b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:18.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.510+0000 7f010784d740 1 -- 172.21.15.107:0/679372960 shutdown_connections 2024-04-25T00:16:18.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.510+0000 7f010784d740 1 -- 172.21.15.107:0/679372960 wait complete. 2024-04-25T00:16:18.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.510+0000 7f010784d740 1 -- start start 2024-04-25T00:16:18.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.511+0000 7f010784d740 1 Processor -- start 2024-04-25T00:16:18.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.511+0000 7f010784d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5642840b5440 0x5642840bf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.511+0000 7f010784d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840b5e30 0x5642840bfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.511+0000 7f010784d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642840bafa0 0x5642840c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.511+0000 7f010784d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5642840bdf50 con 0x5642840b5e30 2024-04-25T00:16:18.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.511+0000 7f010784d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564283fd4cf0 con 0x5642840bafa0 2024-04-25T00:16:18.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.511+0000 7f010784d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5642840be0d0 con 0x5642840b5440 2024-04-25T00:16:18.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.511+0000 7f010784d740 10 client.0 _ll_register_callbacks cb 0x564283fd46e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:16:18.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.511+0000 7f00fe59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840b5e30 0x5642840bfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.511+0000 7f00fe59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840b5e30 0x5642840bfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57160/0 (socket says 172.21.15.107:57160) 2024-04-25T00:16:18.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.511+0000 7f00fe59c640 1 -- 172.21.15.107:0/1616881058 learned_addr learned my addr 172.21.15.107:0/1616881058 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:18.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.511+0000 7f00fdd9b640 1 --2- 172.21.15.107:0/1616881058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642840bafa0 0x5642840c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.511+0000 7f00fdd9b640 1 --2- 172.21.15.107:0/1616881058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5642840b5440 0x5642840bf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.511+0000 7f00fe59c640 1 -- 172.21.15.107:0/1616881058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5642840b5440 msgr2=0x5642840bf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:18.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.511+0000 7f00fe59c640 1 --2- 172.21.15.107:0/1616881058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5642840b5440 0x5642840bf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.511+0000 7f00fe59c640 1 -- 172.21.15.107:0/1616881058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642840bafa0 msgr2=0x5642840c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:18.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.511+0000 7f00fe59c640 1 --2- 172.21.15.107:0/1616881058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642840bafa0 0x5642840c6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.511+0000 7f00fe59c640 1 -- 172.21.15.107:0/1616881058 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564283df76b0 con 0x5642840b5e30 2024-04-25T00:16:18.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.511+0000 7f00fdd9b640 1 --2- 172.21.15.107:0/1616881058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5642840b5440 0x5642840bf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:18.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.511+0000 7f00fe59c640 1 --2- 172.21.15.107:0/1616881058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840b5e30 0x5642840bfda0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f01001016f0 tx=0x7f00f40316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1725cd70090e0bc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:18.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.511+0000 7f00e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:16:18.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.512+0000 7f00e5ffb640 1 -- 172.21.15.107:0/1616881058 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f4040070 con 0x5642840b5e30 2024-04-25T00:16:18.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.512+0000 7f00e5ffb640 1 -- 172.21.15.107:0/1616881058 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7f00f4004240 con 0x5642840b5e30 2024-04-25T00:16:18.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.512+0000 7f010784d740 1 -- 172.21.15.107:0/1616881058 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00f0005500 con 0x5642840b5e30 2024-04-25T00:16:18.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.512+0000 7f00e5ffb640 1 -- 172.21.15.107:0/1616881058 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f00f404d050 con 0x5642840b5e30 2024-04-25T00:16:18.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.512+0000 7f010784d740 10 client.15408 fetch_fsmap learned FSMap version 23 2024-04-25T00:16:18.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.512+0000 7f010784d740 1 -- 172.21.15.107:0/1616881058 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x5642840c9dc0 con 0x5642840b5e30 2024-04-25T00:16:18.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.512+0000 7f00e5ffb640 1 -- 172.21.15.107:0/1616881058 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00f0005500 con 0x5642840b5e30 2024-04-25T00:16:18.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.512+0000 7f00e5ffb640 1 -- 172.21.15.107:0/1616881058 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f00f4004ae0 con 0x5642840b5e30 2024-04-25T00:16:18.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.512+0000 7f010784d740 10 client.15408 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:16:18.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.512+0000 7f010784d740 10 client.15408 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T00:16:18.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.512+0000 7f010784d740 20 client.15408 populate_metadata read hostname 'smithi107' 2024-04-25T00:16:18.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.512+0000 7f010784d740 1 --2- 172.21.15.107:0/1616881058 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5642840cc470 0x5642840cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.512+0000 7f010784d740 4 client.15408 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:16:18.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.512+0000 7f010784d740 1 -- 172.21.15.107:0/1616881058 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564284026440 con 0x5642840cc470 2024-04-25T00:16:18.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.512+0000 7f00fdd9b640 1 --2- 172.21.15.107:0/1616881058 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5642840cc470 0x5642840cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.513+0000 7f00fdd9b640 1 --2- 172.21.15.107:0/1616881058 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5642840cc470 0x5642840cd8e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=19d5b0d4d14278b9 server_cookie=450beaa10e86138 in_seq=0 out_seq=0 2024-04-25T00:16:18.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.513+0000 7f00e5ffb640 10 client.15408 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:16:18.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.514+0000 7f00e5ffb640 1 -- 172.21.15.107:0/1616881058 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564284026440 con 0x5642840cc470 2024-04-25T00:16:18.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.514+0000 7f00e5ffb640 10 client.15408 handle_command_reply: tid=0 2024-04-25T00:16:18.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.514+0000 7f010784d740 1 client.15408 shutdown 2024-04-25T00:16:18.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.514+0000 7f010784d740 1 -- 172.21.15.107:0/1616881058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840b5e30 msgr2=0x5642840bfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.514+0000 7f010784d740 1 --2- 172.21.15.107:0/1616881058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840b5e30 0x5642840bfda0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f01001016f0 tx=0x7f00f40316b0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.514+0000 7f010784d740 1 -- 172.21.15.107:0/1616881058 shutdown_connections 2024-04-25T00:16:18.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.514+0000 7f010784d740 1 --2- 172.21.15.107:0/1616881058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642840bafa0 0x5642840c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.514+0000 7f010784d740 1 --2- 172.21.15.107:0/1616881058 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5642840cc470 0x5642840cd8e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.514+0000 7f010784d740 1 --2- 172.21.15.107:0/1616881058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840b5e30 0x5642840bfda0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.514+0000 7f00e5ffb640 0 client.15408 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:16:18.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.514+0000 7f010784d740 1 --2- 172.21.15.107:0/1616881058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5642840b5440 0x5642840bf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.514+0000 7f010784d740 1 -- 172.21.15.107:0/1616881058 >> 172.21.15.107:0/1616881058 conn(0x5642840b0ab0 msgr2=0x5642840afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:18.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.514+0000 7f010784d740 1 -- 172.21.15.107:0/1616881058 shutdown_connections 2024-04-25T00:16:18.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.514+0000 7f010784d740 1 -- 172.21.15.107:0/1616881058 wait complete. 2024-04-25T00:16:18.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.514+0000 7f010784d740 20 client.15408 trim_cache size 0 max 16384 2024-04-25T00:16:18.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.532+0000 7f010784d740 1 Processor -- start 2024-04-25T00:16:18.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.532+0000 7f010784d740 1 -- start start 2024-04-25T00:16:18.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.532+0000 7f010784d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5642840b5440 0x5642840b3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.532+0000 7f010784d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642840b5e30 0x5642840c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.532+0000 7f010784d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840bafa0 0x5642840b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.532+0000 7f010784d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5642840cbc20 con 0x5642840bafa0 2024-04-25T00:16:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.532+0000 7f010784d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5642840cc2f0 con 0x5642840b5e30 2024-04-25T00:16:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.532+0000 7f010784d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5642840cc9c0 con 0x5642840b5440 2024-04-25T00:16:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.532+0000 7f00fdd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840bafa0 0x5642840b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.532+0000 7f00fdd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840bafa0 0x5642840b3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57170/0 (socket says 172.21.15.107:57170) 2024-04-25T00:16:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.532+0000 7f00fdd9b640 1 -- 172.21.15.107:0/1008195616 learned_addr learned my addr 172.21.15.107:0/1008195616 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.532+0000 7f00fe59c640 1 --2- 172.21.15.107:0/1008195616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642840b5e30 0x5642840c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.532+0000 7f00fdd9b640 1 -- 172.21.15.107:0/1008195616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5642840b5440 msgr2=0x5642840b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.532+0000 7f00fdd9b640 1 --2- 172.21.15.107:0/1008195616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5642840b5440 0x5642840b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.532+0000 7f00fdd9b640 1 -- 172.21.15.107:0/1008195616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642840b5e30 msgr2=0x5642840c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.532+0000 7f00fdd9b640 1 --2- 172.21.15.107:0/1008195616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642840b5e30 0x5642840c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.532+0000 7f00fdd9b640 1 -- 172.21.15.107:0/1008195616 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642840b6210 con 0x5642840bafa0 2024-04-25T00:16:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.532+0000 7f00fdd9b640 1 --2- 172.21.15.107:0/1008195616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840bafa0 0x5642840b3cd0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f00f0015e60 tx=0x7f00f0009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec242764c2a3af5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.533+0000 7f00e7fff640 1 -- 172.21.15.107:0/1008195616 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0004c10 con 0x5642840bafa0 2024-04-25T00:16:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.533+0000 7f00e7fff640 1 -- 172.21.15.107:0/1008195616 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00f00150d0 con 0x5642840bafa0 2024-04-25T00:16:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.533+0000 7f00e7fff640 1 -- 172.21.15.107:0/1008195616 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0004020 con 0x5642840bafa0 2024-04-25T00:16:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.533+0000 7f010784d740 1 -- 172.21.15.107:0/1008195616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840bafa0 msgr2=0x5642840b3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.533+0000 7f010784d740 1 --2- 172.21.15.107:0/1008195616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840bafa0 0x5642840b3cd0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f00f0015e60 tx=0x7f00f0009640 comp rx=0 tx=0).stop 2024-04-25T00:16:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.533+0000 7f010784d740 1 -- 172.21.15.107:0/1008195616 shutdown_connections 2024-04-25T00:16:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.533+0000 7f010784d740 1 --2- 172.21.15.107:0/1008195616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840bafa0 0x5642840b3cd0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.533+0000 7f010784d740 1 --2- 172.21.15.107:0/1008195616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642840b5e30 0x5642840c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.533+0000 7f010784d740 1 --2- 172.21.15.107:0/1008195616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5642840b5440 0x5642840b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.533+0000 7f010784d740 1 -- 172.21.15.107:0/1008195616 >> 172.21.15.107:0/1008195616 conn(0x5642840b0ab0 msgr2=0x5642840cb280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.533+0000 7f010784d740 1 -- 172.21.15.107:0/1008195616 shutdown_connections 2024-04-25T00:16:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.533+0000 7f010784d740 1 -- 172.21.15.107:0/1008195616 wait complete. 2024-04-25T00:16:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.534+0000 7f010784d740 1 -- start start 2024-04-25T00:16:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.534+0000 7f010784d740 1 Processor -- start 2024-04-25T00:16:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.534+0000 7f010784d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5642840b5440 0x5642840c2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.534+0000 7f010784d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642840b5e30 0x5642840c34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.534+0000 7f010784d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840bafa0 0x5642840c6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.534+0000 7f010784d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5642840cd760 con 0x5642840bafa0 2024-04-25T00:16:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.534+0000 7f010784d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5642840cd090 con 0x5642840b5e30 2024-04-25T00:16:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.534+0000 7f010784d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5642840cde30 con 0x5642840b5440 2024-04-25T00:16:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.534+0000 7f010784d740 10 client.0 _ll_register_callbacks cb 0x7f00f4032c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:16:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.534+0000 7f00fdd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840bafa0 0x5642840c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.534+0000 7f00fdd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840bafa0 0x5642840c6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57184/0 (socket says 172.21.15.107:57184) 2024-04-25T00:16:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.534+0000 7f00fdd9b640 1 -- 172.21.15.107:0/945817973 learned_addr learned my addr 172.21.15.107:0/945817973 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.534+0000 7f00fe59c640 1 --2- 172.21.15.107:0/945817973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642840b5e30 0x5642840c34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.534+0000 7f00fdd9b640 1 --2- 172.21.15.107:0/945817973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5642840b5440 0x5642840c2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.534+0000 7f00fdd9b640 1 -- 172.21.15.107:0/945817973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5642840b5440 msgr2=0x5642840c2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.534+0000 7f00fdd9b640 1 --2- 172.21.15.107:0/945817973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5642840b5440 0x5642840c2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.534+0000 7f00fdd9b640 1 -- 172.21.15.107:0/945817973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642840b5e30 msgr2=0x5642840c34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.534+0000 7f00fdd9b640 1 --2- 172.21.15.107:0/945817973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642840b5e30 0x5642840c34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.534+0000 7f00fdd9b640 1 -- 172.21.15.107:0/945817973 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642840c6920 con 0x5642840bafa0 2024-04-25T00:16:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.534+0000 7f00fdd9b640 1 --2- 172.21.15.107:0/945817973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840bafa0 0x5642840c6330 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f00f002e040 tx=0x7f00f0009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c68afaac827a739 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.534+0000 7f00e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:16:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.534+0000 7f00e5ffb640 1 -- 172.21.15.107:0/945817973 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0030070 con 0x5642840bafa0 2024-04-25T00:16:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.535+0000 7f010784d740 1 -- 172.21.15.107:0/945817973 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00f00150d0 con 0x5642840bafa0 2024-04-25T00:16:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.535+0000 7f00e5ffb640 1 -- 172.21.15.107:0/945817973 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7f00f0029070 con 0x5642840bafa0 2024-04-25T00:16:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.535+0000 7f00e5ffb640 1 -- 172.21.15.107:0/945817973 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f00f0004200 con 0x5642840bafa0 2024-04-25T00:16:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.535+0000 7f010784d740 10 client.15414 fetch_fsmap learned FSMap version 23 2024-04-25T00:16:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.535+0000 7f010784d740 1 -- 172.21.15.107:0/945817973 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x564284132d80 con 0x5642840bafa0 2024-04-25T00:16:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.535+0000 7f00e5ffb640 1 -- 172.21.15.107:0/945817973 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00f00150d0 con 0x5642840bafa0 2024-04-25T00:16:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.535+0000 7f00e5ffb640 1 -- 172.21.15.107:0/945817973 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7f00f000c070 con 0x5642840bafa0 2024-04-25T00:16:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.535+0000 7f010784d740 10 client.15414 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:16:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.535+0000 7f010784d740 10 client.15414 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T00:16:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.535+0000 7f010784d740 20 client.15414 populate_metadata read hostname 'smithi107' 2024-04-25T00:16:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.535+0000 7f010784d740 1 --2- 172.21.15.107:0/945817973 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5642841372d0 0x564284133de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.535+0000 7f010784d740 4 client.15414 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:16:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.535+0000 7f010784d740 1 -- 172.21.15.107:0/945817973 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "status"}) -- 0x564284020790 con 0x5642841372d0 2024-04-25T00:16:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.535+0000 7f00fe59c640 1 --2- 172.21.15.107:0/945817973 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5642841372d0 0x564284133de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.536+0000 7f00fe59c640 1 --2- 172.21.15.107:0/945817973 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5642841372d0 0x564284133de0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2a2ecf29a58acd10 server_cookie=a7b06cc5c0ab3ac9 in_seq=0 out_seq=0 2024-04-25T00:16:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.536+0000 7f00e5ffb640 10 client.15414 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:16:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.536+0000 7f00e5ffb640 1 -- 172.21.15.107:0/945817973 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564284020790 con 0x5642841372d0 2024-04-25T00:16:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.536+0000 7f00e5ffb640 10 client.15414 handle_command_reply: tid=0 2024-04-25T00:16:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.536+0000 7f010784d740 1 client.15414 shutdown 2024-04-25T00:16:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.536+0000 7f010784d740 1 -- 172.21.15.107:0/945817973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840bafa0 msgr2=0x5642840c6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.536+0000 7f010784d740 1 --2- 172.21.15.107:0/945817973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840bafa0 0x5642840c6330 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f00f002e040 tx=0x7f00f0009800 comp rx=0 tx=0).stop 2024-04-25T00:16:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.536+0000 7f010784d740 1 -- 172.21.15.107:0/945817973 shutdown_connections 2024-04-25T00:16:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.536+0000 7f010784d740 1 --2- 172.21.15.107:0/945817973 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5642841372d0 0x564284133de0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.536+0000 7f010784d740 1 --2- 172.21.15.107:0/945817973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5642840bafa0 0x5642840c6330 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.536+0000 7f010784d740 1 --2- 172.21.15.107:0/945817973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5642840b5e30 0x5642840c34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.536+0000 7f00e5ffb640 0 client.15414 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:16:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.536+0000 7f010784d740 1 --2- 172.21.15.107:0/945817973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5642840b5440 0x5642840c2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.536+0000 7f010784d740 1 -- 172.21.15.107:0/945817973 >> 172.21.15.107:0/945817973 conn(0x5642840b0ab0 msgr2=0x5642840af250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.536+0000 7f010784d740 1 -- 172.21.15.107:0/945817973 shutdown_connections 2024-04-25T00:16:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.536+0000 7f010784d740 1 -- 172.21.15.107:0/945817973 wait complete. 2024-04-25T00:16:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.536+0000 7f010784d740 20 client.15414 trim_cache size 0 max 16384 2024-04-25T00:16:18.544 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:16:18.544 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "79602f58-0297-11ef-bc93-c7b262605968", 2024-04-25T00:16:18.545 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 1, 2024-04-25T00:16:18.545 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24493, 2024-04-25T00:16:18.545 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-04-25T00:16:18.545 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-04-25T00:16:18.545 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-04-25T00:16:18.545 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 112.01691802400001, 2024-04-25T00:16:18.545 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 23, 2024-04-25T00:16:18.545 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 96, 2024-04-25T00:16:18.545 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 92, 2024-04-25T00:16:18.545 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 149.614020224 2024-04-25T00:16:18.545 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:16:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.537+0000 7f0104b42640 1 -- 172.21.15.107:0/1208141106 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f00e80a2b90 msgr2=0x7f00e80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.537+0000 7f0104b42640 1 --2- 172.21.15.107:0/1208141106 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f00e80a2b90 0x7f00e80a5050 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f00f809bb50 tx=0x7f00f806ec80 comp rx=0 tx=0).stop 2024-04-25T00:16:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.537+0000 7f0104b42640 1 -- 172.21.15.107:0/1208141106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0100102250 msgr2=0x7f010019f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.537+0000 7f0104b42640 1 --2- 172.21.15.107:0/1208141106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0100102250 0x7f010019f130 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f00f800eaa0 tx=0x7f00f800ef60 comp rx=0 tx=0).stop 2024-04-25T00:16:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.537+0000 7f0104b42640 1 -- 172.21.15.107:0/1208141106 shutdown_connections 2024-04-25T00:16:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.537+0000 7f0104b42640 1 --2- 172.21.15.107:0/1208141106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0100102250 0x7f010019f130 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.537+0000 7f0104b42640 1 --2- 172.21.15.107:0/1208141106 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f00e80a2b90 0x7f00e80a5050 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.537+0000 7f0104b42640 1 --2- 172.21.15.107:0/1208141106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0100101890 0x7f01001980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.537+0000 7f0104b42640 1 --2- 172.21.15.107:0/1208141106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01001006a0 0x7f0100197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.537+0000 7f0104b42640 1 -- 172.21.15.107:0/1208141106 >> 172.21.15.107:0/1208141106 conn(0x7f01000fbc10 msgr2=0x7f01000fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:18.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.537+0000 7f0104b42640 1 -- 172.21.15.107:0/1208141106 shutdown_connections 2024-04-25T00:16:18.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.537+0000 7f0104b42640 1 -- 172.21.15.107:0/1208141106 wait complete. 2024-04-25T00:16:18.548 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T00:16:18.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:18 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3686589652' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:16:18.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:18 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3879268060' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:16:18.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:18 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3638941736' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:18.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.646+0000 7fb0e8bc1640 1 -- 172.21.15.107:0/3199808632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0e4102230 msgr2=0x7fb0e4108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.646+0000 7fb0e8bc1640 1 --2- 172.21.15.107:0/3199808632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0e4102230 0x7fb0e4108ac0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009ca0 tx=0x7fb0dc02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.646+0000 7fb0e8bc1640 1 -- 172.21.15.107:0/3199808632 shutdown_connections 2024-04-25T00:16:18.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.646+0000 7fb0e8bc1640 1 --2- 172.21.15.107:0/3199808632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0e4102230 0x7fb0e4108ac0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.646+0000 7fb0e8bc1640 1 --2- 172.21.15.107:0/3199808632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0e4101870 0x7fb0e4101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.646+0000 7fb0e8bc1640 1 --2- 172.21.15.107:0/3199808632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e4100680 0x7fb0e4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.646+0000 7fb0e8bc1640 1 -- 172.21.15.107:0/3199808632 >> 172.21.15.107:0/3199808632 conn(0x7fb0e40fbc30 msgr2=0x7fb0e40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:18.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.646+0000 7fb0e8bc1640 1 -- 172.21.15.107:0/3199808632 shutdown_connections 2024-04-25T00:16:18.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.646+0000 7fb0e8bc1640 1 -- 172.21.15.107:0/3199808632 wait complete. 2024-04-25T00:16:18.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.647+0000 7fb0e8bc1640 1 Processor -- start 2024-04-25T00:16:18.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.647+0000 7fb0e8bc1640 1 -- start start 2024-04-25T00:16:18.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.647+0000 7fb0e8bc1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0e4100680 0x7fb0e406a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.647+0000 7fb0e8bc1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e4101870 0x7fb0e419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.647+0000 7fb0e8bc1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0e4102230 0x7fb0e419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.647+0000 7fb0e8bc1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e410c300 con 0x7fb0e4102230 2024-04-25T00:16:18.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.647+0000 7fb0e8bc1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e410c180 con 0x7fb0e4101870 2024-04-25T00:16:18.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.647+0000 7fb0e8bc1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e410c480 con 0x7fb0e4100680 2024-04-25T00:16:18.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.647+0000 7fb0e2d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0e4102230 0x7fb0e419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.647+0000 7fb0e2d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0e4102230 0x7fb0e419ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57196/0 (socket says 172.21.15.107:57196) 2024-04-25T00:16:18.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.647+0000 7fb0e2d9d640 1 -- 172.21.15.107:0/119338369 learned_addr learned my addr 172.21.15.107:0/119338369 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:18.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.647+0000 7fb0e259c640 1 --2- 172.21.15.107:0/119338369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0e4100680 0x7fb0e406a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.648+0000 7fb0dbfff640 1 --2- 172.21.15.107:0/119338369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e4101870 0x7fb0e419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.648+0000 7fb0e2d9d640 1 -- 172.21.15.107:0/119338369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0e4100680 msgr2=0x7fb0e406a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.648+0000 7fb0e2d9d640 1 --2- 172.21.15.107:0/119338369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0e4100680 0x7fb0e406a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.648+0000 7fb0e2d9d640 1 -- 172.21.15.107:0/119338369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e4101870 msgr2=0x7fb0e419c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.648+0000 7fb0e2d9d640 1 --2- 172.21.15.107:0/119338369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e4101870 0x7fb0e419c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.648+0000 7fb0e2d9d640 1 -- 172.21.15.107:0/119338369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e419f6f0 con 0x7fb0e4102230 2024-04-25T00:16:18.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.648+0000 7fb0e259c640 1 --2- 172.21.15.107:0/119338369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0e4100680 0x7fb0e406a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:18.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.648+0000 7fb0e2d9d640 1 --2- 172.21.15.107:0/119338369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0e4102230 0x7fb0e419ede0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc031ae0 tx=0x7fb0dc031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:18.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.648+0000 7fb0db7fe640 1 -- 172.21.15.107:0/119338369 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc003c30 con 0x7fb0e4102230 2024-04-25T00:16:18.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.648+0000 7fb0db7fe640 1 -- 172.21.15.107:0/119338369 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0dc003dd0 con 0x7fb0e4102230 2024-04-25T00:16:18.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.648+0000 7fb0e8bc1640 1 -- 172.21.15.107:0/119338369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e419f920 con 0x7fb0e4102230 2024-04-25T00:16:18.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.648+0000 7fb0db7fe640 1 -- 172.21.15.107:0/119338369 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc02fcb0 con 0x7fb0e4102230 2024-04-25T00:16:18.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.648+0000 7fb0e8bc1640 1 -- 172.21.15.107:0/119338369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e419fec0 con 0x7fb0e4102230 2024-04-25T00:16:18.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.650+0000 7fb0db7fe640 1 -- 172.21.15.107:0/119338369 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7fb0dc004050 con 0x7fb0e4102230 2024-04-25T00:16:18.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.650+0000 7fb0eb9ad740 1 Processor -- start 2024-04-25T00:16:18.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.651+0000 7fb0eb9ad740 1 -- start start 2024-04-25T00:16:18.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.651+0000 7fb0db7fe640 1 --2- 172.21.15.107:0/119338369 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb0d00a2c10 0x7fb0d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.651+0000 7fb0eb9ad740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e74850 0x562ba1e74c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.651+0000 7fb0eb9ad740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ba1e75240 0x562ba1e79d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.651+0000 7fb0eb9ad740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba1e7a3b0 0x562ba1e7c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.651+0000 7fb0eb9ad740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562ba1dd3f00 con 0x562ba1e74850 2024-04-25T00:16:18.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.651+0000 7fb0eb9ad740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562ba1d7c750 con 0x562ba1e75240 2024-04-25T00:16:18.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.651+0000 7fb0eb9ad740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562ba1dcfec0 con 0x562ba1e7a3b0 2024-04-25T00:16:18.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.651+0000 7fb0e259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e74850 0x562ba1e74c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.651+0000 7fb0dbfff640 1 --2- 172.21.15.107:0/119338369 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb0d00a2c10 0x7fb0d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.651+0000 7fb0e259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e74850 0x562ba1e74c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57208/0 (socket says 172.21.15.107:57208) 2024-04-25T00:16:18.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.651+0000 7fb0e259c640 1 -- 172.21.15.107:0/3908504603 learned_addr learned my addr 172.21.15.107:0/3908504603 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:18.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.651+0000 7fb0db7fe640 1 -- 172.21.15.107:0/119338369 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7fb0dc0f06f0 con 0x7fb0e4102230 2024-04-25T00:16:18.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.651+0000 7fb0e2d9d640 1 --2- 172.21.15.107:0/3908504603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba1e7a3b0 0x562ba1e7c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.651+0000 7fb0dbfff640 1 --2- 172.21.15.107:0/3908504603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ba1e75240 0x562ba1e79d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.651+0000 7fb0e259c640 1 -- 172.21.15.107:0/3908504603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba1e7a3b0 msgr2=0x562ba1e7c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:18.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.651+0000 7fb0e259c640 1 --2- 172.21.15.107:0/3908504603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba1e7a3b0 0x562ba1e7c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.651+0000 7fb0e259c640 1 -- 172.21.15.107:0/3908504603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ba1e75240 msgr2=0x562ba1e79d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:18.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.651+0000 7fb0e259c640 1 --2- 172.21.15.107:0/3908504603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ba1e75240 0x562ba1e79d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.651+0000 7fb0e259c640 1 -- 172.21.15.107:0/3908504603 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ba1ae82d0 con 0x562ba1e74850 2024-04-25T00:16:18.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.651+0000 7fb0dbfff640 1 --2- 172.21.15.107:0/119338369 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb0d00a2c10 0x7fb0d00a50d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb0e406a660 tx=0x7fb0cc0147e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:18.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.651+0000 7fb0e259c640 1 --2- 172.21.15.107:0/3908504603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e74850 0x562ba1e74c70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb0d400cc50 tx=0x7fb0d400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a526d6109e07d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:18.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.652+0000 7fb0d8ff9640 1 -- 172.21.15.107:0/3908504603 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4014070 con 0x562ba1e74850 2024-04-25T00:16:18.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.652+0000 7fb0d8ff9640 1 -- 172.21.15.107:0/3908504603 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d40047a0 con 0x562ba1e74850 2024-04-25T00:16:18.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.652+0000 7fb0d8ff9640 1 -- 172.21.15.107:0/3908504603 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d40107d0 con 0x562ba1e74850 2024-04-25T00:16:18.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.652+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/3908504603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e74850 msgr2=0x562ba1e74c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.652+0000 7fb0eb9ad740 1 --2- 172.21.15.107:0/3908504603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e74850 0x562ba1e74c70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb0d400cc50 tx=0x7fb0d400e720 comp rx=0 tx=0).stop 2024-04-25T00:16:18.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.652+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/3908504603 shutdown_connections 2024-04-25T00:16:18.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.652+0000 7fb0eb9ad740 1 --2- 172.21.15.107:0/3908504603 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba1e7a3b0 0x562ba1e7c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.652+0000 7fb0eb9ad740 1 --2- 172.21.15.107:0/3908504603 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ba1e75240 0x562ba1e79d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.652+0000 7fb0eb9ad740 1 --2- 172.21.15.107:0/3908504603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e74850 0x562ba1e74c70 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.652+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/3908504603 >> 172.21.15.107:0/3908504603 conn(0x562ba1e6bab0 msgr2=0x562ba1e6beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:18.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.652+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/3908504603 shutdown_connections 2024-04-25T00:16:18.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.652+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/3908504603 wait complete. 2024-04-25T00:16:18.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.653+0000 7fb0eb9ad740 1 -- start start 2024-04-25T00:16:18.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.653+0000 7fb0eb9ad740 1 Processor -- start 2024-04-25T00:16:18.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.653+0000 7fb0eb9ad740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ba1e74850 0x562ba1e7ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.653+0000 7fb0eb9ad740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e75240 0x562ba1e7f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.653+0000 7fb0eb9ad740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba1e7a3b0 0x562ba1e861f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.653+0000 7fb0eb9ad740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562ba1e7d360 con 0x562ba1e75240 2024-04-25T00:16:18.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.653+0000 7fb0eb9ad740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562ba1d8fcf0 con 0x562ba1e74850 2024-04-25T00:16:18.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.653+0000 7fb0eb9ad740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562ba1e7d4e0 con 0x562ba1e7a3b0 2024-04-25T00:16:18.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.653+0000 7fb0eb9ad740 10 client.0 _ll_register_callbacks cb 0x562ba1d8f6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:16:18.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.653+0000 7fb0dbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e75240 0x562ba1e7f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.653+0000 7fb0dbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e75240 0x562ba1e7f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57224/0 (socket says 172.21.15.107:57224) 2024-04-25T00:16:18.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.653+0000 7fb0dbfff640 1 -- 172.21.15.107:0/1205486033 learned_addr learned my addr 172.21.15.107:0/1205486033 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:18.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.653+0000 7fb0e259c640 1 --2- 172.21.15.107:0/1205486033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ba1e74850 0x562ba1e7ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.653+0000 7fb0e2d9d640 1 --2- 172.21.15.107:0/1205486033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba1e7a3b0 0x562ba1e861f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.653+0000 7fb0dbfff640 1 -- 172.21.15.107:0/1205486033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba1e7a3b0 msgr2=0x562ba1e861f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:18.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.653+0000 7fb0dbfff640 1 --2- 172.21.15.107:0/1205486033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba1e7a3b0 0x562ba1e861f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.653+0000 7fb0dbfff640 1 -- 172.21.15.107:0/1205486033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ba1e74850 msgr2=0x562ba1e7ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:18.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.653+0000 7fb0dbfff640 1 --2- 172.21.15.107:0/1205486033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ba1e74850 0x562ba1e7ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.653+0000 7fb0dbfff640 1 -- 172.21.15.107:0/1205486033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ba1bb26b0 con 0x562ba1e75240 2024-04-25T00:16:18.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.653+0000 7fb0e2d9d640 1 --2- 172.21.15.107:0/1205486033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba1e7a3b0 0x562ba1e861f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:18.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.654+0000 7fb0dbfff640 1 --2- 172.21.15.107:0/1205486033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e75240 0x562ba1e7f1b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc004170 tx=0x7fb0cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e110f300bb83d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:18.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.654+0000 7fb0c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:16:18.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.654+0000 7fb0c9ffb640 1 -- 172.21.15.107:0/1205486033 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc026030 con 0x562ba1e75240 2024-04-25T00:16:18.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.654+0000 7fb0c9ffb640 1 -- 172.21.15.107:0/1205486033 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7fb0cc005020 con 0x562ba1e75240 2024-04-25T00:16:18.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.654+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/1205486033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0d40047a0 con 0x562ba1e75240 2024-04-25T00:16:18.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.654+0000 7fb0c9ffb640 1 -- 172.21.15.107:0/1205486033 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb0cc030050 con 0x562ba1e75240 2024-04-25T00:16:18.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.654+0000 7fb0eb9ad740 10 client.15426 fetch_fsmap learned FSMap version 23 2024-04-25T00:16:18.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.654+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/1205486033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x562ba1e89290 con 0x562ba1e75240 2024-04-25T00:16:18.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.654+0000 7fb0c9ffb640 1 -- 172.21.15.107:0/1205486033 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d40047a0 con 0x562ba1e75240 2024-04-25T00:16:18.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.654+0000 7fb0c9ffb640 1 -- 172.21.15.107:0/1205486033 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fb0cc00f040 con 0x562ba1e75240 2024-04-25T00:16:18.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.654+0000 7fb0eb9ad740 10 client.15426 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:16:18.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.654+0000 7fb0eb9ad740 10 client.15426 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:16:18.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.654+0000 7fb0eb9ad740 20 client.15426 populate_metadata read hostname 'smithi107' 2024-04-25T00:16:18.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.654+0000 7fb0eb9ad740 1 --2- 172.21.15.107:0/1205486033 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562ba1e8ba20 0x562ba1e8ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.654+0000 7fb0eb9ad740 4 client.15426 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:16:18.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.654+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/1205486033 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ba1de1440 con 0x562ba1e8ba20 2024-04-25T00:16:18.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.655+0000 7fb0e259c640 1 --2- 172.21.15.107:0/1205486033 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562ba1e8ba20 0x562ba1e8ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.655+0000 7fb0e259c640 1 --2- 172.21.15.107:0/1205486033 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562ba1e8ba20 0x562ba1e8ce90 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=849bfbac26e21749 server_cookie=2ec19b7516848214 in_seq=0 out_seq=0 2024-04-25T00:16:18.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.655+0000 7fb0c9ffb640 10 client.15426 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:16:18.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.656+0000 7fb0c9ffb640 1 -- 172.21.15.107:0/1205486033 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562ba1de1440 con 0x562ba1e8ba20 2024-04-25T00:16:18.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.656+0000 7fb0c9ffb640 10 client.15426 handle_command_reply: tid=0 2024-04-25T00:16:18.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.656+0000 7fb0eb9ad740 1 client.15426 shutdown 2024-04-25T00:16:18.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.656+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/1205486033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e75240 msgr2=0x562ba1e7f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.656+0000 7fb0eb9ad740 1 --2- 172.21.15.107:0/1205486033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e75240 0x562ba1e7f1b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc004170 tx=0x7fb0cc004730 comp rx=0 tx=0).stop 2024-04-25T00:16:18.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.656+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/1205486033 shutdown_connections 2024-04-25T00:16:18.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.656+0000 7fb0eb9ad740 1 --2- 172.21.15.107:0/1205486033 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562ba1e8ba20 0x562ba1e8ce90 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.656+0000 7fb0eb9ad740 1 --2- 172.21.15.107:0/1205486033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba1e7a3b0 0x562ba1e861f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.656+0000 7fb0c9ffb640 0 client.15426 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:16:18.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.656+0000 7fb0eb9ad740 1 --2- 172.21.15.107:0/1205486033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e75240 0x562ba1e7f1b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.656+0000 7fb0eb9ad740 1 --2- 172.21.15.107:0/1205486033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ba1e74850 0x562ba1e7ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.656+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/1205486033 >> 172.21.15.107:0/1205486033 conn(0x562ba1e6bab0 msgr2=0x562ba1e6aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:18.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.656+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/1205486033 shutdown_connections 2024-04-25T00:16:18.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.656+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/1205486033 wait complete. 2024-04-25T00:16:18.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.656+0000 7fb0eb9ad740 20 client.15426 trim_cache size 0 max 16384 2024-04-25T00:16:18.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.674+0000 7fb0eb9ad740 1 Processor -- start 2024-04-25T00:16:18.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.674+0000 7fb0eb9ad740 1 -- start start 2024-04-25T00:16:18.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.674+0000 7fb0eb9ad740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ba1e74850 0x562ba1e6af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.674+0000 7fb0eb9ad740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba1e75240 0x562ba1e861b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.674+0000 7fb0eb9ad740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e7a3b0 0x562ba1e8a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.674+0000 7fb0eb9ad740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562ba1e8b1d0 con 0x562ba1e7a3b0 2024-04-25T00:16:18.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.674+0000 7fb0eb9ad740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562ba1e8b8a0 con 0x562ba1e74850 2024-04-25T00:16:18.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.674+0000 7fb0eb9ad740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562ba1e8bf70 con 0x562ba1e75240 2024-04-25T00:16:18.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.674+0000 7fb0e2d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e7a3b0 0x562ba1e8a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.674+0000 7fb0e2d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e7a3b0 0x562ba1e8a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57234/0 (socket says 172.21.15.107:57234) 2024-04-25T00:16:18.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.674+0000 7fb0e2d9d640 1 -- 172.21.15.107:0/214236176 learned_addr learned my addr 172.21.15.107:0/214236176 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:18.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.674+0000 7fb0e259c640 1 --2- 172.21.15.107:0/214236176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ba1e74850 0x562ba1e6af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.674+0000 7fb0dbfff640 1 --2- 172.21.15.107:0/214236176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba1e75240 0x562ba1e861b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.674+0000 7fb0e2d9d640 1 -- 172.21.15.107:0/214236176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba1e75240 msgr2=0x562ba1e861b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:18.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.674+0000 7fb0e2d9d640 1 --2- 172.21.15.107:0/214236176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba1e75240 0x562ba1e861b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.674+0000 7fb0e2d9d640 1 -- 172.21.15.107:0/214236176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ba1e74850 msgr2=0x562ba1e6af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:18.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.674+0000 7fb0e2d9d640 1 --2- 172.21.15.107:0/214236176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ba1e74850 0x562ba1e6af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.674+0000 7fb0e2d9d640 1 -- 172.21.15.107:0/214236176 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ba1e75620 con 0x562ba1e7a3b0 2024-04-25T00:16:18.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.674+0000 7fb0dbfff640 1 --2- 172.21.15.107:0/214236176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba1e75240 0x562ba1e861b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:18.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.674+0000 7fb0e2d9d640 1 --2- 172.21.15.107:0/214236176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e7a3b0 0x562ba1e8a750 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0edeb0 tx=0x7fb0dc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c53fce78ab67d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:18.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.674+0000 7fb0cbfff640 1 -- 172.21.15.107:0/214236176 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc0bcbc0 con 0x562ba1e7a3b0 2024-04-25T00:16:18.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.675+0000 7fb0cbfff640 1 -- 172.21.15.107:0/214236176 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0dc036040 con 0x562ba1e7a3b0 2024-04-25T00:16:18.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.675+0000 7fb0cbfff640 1 -- 172.21.15.107:0/214236176 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc0de0b0 con 0x562ba1e7a3b0 2024-04-25T00:16:18.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.675+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/214236176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e7a3b0 msgr2=0x562ba1e8a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.675+0000 7fb0eb9ad740 1 --2- 172.21.15.107:0/214236176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e7a3b0 0x562ba1e8a750 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0edeb0 tx=0x7fb0dc02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.675+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/214236176 shutdown_connections 2024-04-25T00:16:18.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.675+0000 7fb0eb9ad740 1 --2- 172.21.15.107:0/214236176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e7a3b0 0x562ba1e8a750 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.675+0000 7fb0eb9ad740 1 --2- 172.21.15.107:0/214236176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba1e75240 0x562ba1e861b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.675+0000 7fb0eb9ad740 1 --2- 172.21.15.107:0/214236176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ba1e74850 0x562ba1e6af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.675+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/214236176 >> 172.21.15.107:0/214236176 conn(0x562ba1e6bab0 msgr2=0x562ba1e66690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:18.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.675+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/214236176 shutdown_connections 2024-04-25T00:16:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.675+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/214236176 wait complete. 2024-04-25T00:16:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.675+0000 7fb0eb9ad740 1 -- start start 2024-04-25T00:16:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.675+0000 7fb0eb9ad740 1 Processor -- start 2024-04-25T00:16:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0eb9ad740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba1e74850 0x562ba1e78ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0eb9ad740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e75240 0x562ba1e861b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0eb9ad740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ba1e7a3b0 0x562ba1eef9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0eb9ad740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562ba1e8cd10 con 0x562ba1e75240 2024-04-25T00:16:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0eb9ad740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562ba1e8c640 con 0x562ba1e7a3b0 2024-04-25T00:16:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0eb9ad740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562ba1e8d3e0 con 0x562ba1e74850 2024-04-25T00:16:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0eb9ad740 10 client.0 _ll_register_callbacks cb 0x7fb0cc004030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:16:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0dbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e75240 0x562ba1e861b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0dbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e75240 0x562ba1e861b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57238/0 (socket says 172.21.15.107:57238) 2024-04-25T00:16:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0dbfff640 1 -- 172.21.15.107:0/555510886 learned_addr learned my addr 172.21.15.107:0/555510886 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:18.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0e259c640 1 --2- 172.21.15.107:0/555510886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba1e74850 0x562ba1e78ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0e2d9d640 1 --2- 172.21.15.107:0/555510886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ba1e7a3b0 0x562ba1eef9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0dbfff640 1 -- 172.21.15.107:0/555510886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba1e74850 msgr2=0x562ba1e78ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0dbfff640 1 --2- 172.21.15.107:0/555510886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba1e74850 0x562ba1e78ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0dbfff640 1 -- 172.21.15.107:0/555510886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ba1e7a3b0 msgr2=0x562ba1eef9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0dbfff640 1 --2- 172.21.15.107:0/555510886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ba1e7a3b0 0x562ba1eef9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0dbfff640 1 -- 172.21.15.107:0/555510886 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ba1ef0170 con 0x562ba1e75240 2024-04-25T00:16:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0dbfff640 1 --2- 172.21.15.107:0/555510886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e75240 0x562ba1e861b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc005d80 tx=0x7fb0cc009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309afc685bf724dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:16:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0c9ffb640 1 -- 172.21.15.107:0/555510886 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc026050 con 0x562ba1e75240 2024-04-25T00:16:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0c9ffb640 1 -- 172.21.15.107:0/555510886 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7fb0cc021bf0 con 0x562ba1e75240 2024-04-25T00:16:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/555510886 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0dc036040 con 0x562ba1e75240 2024-04-25T00:16:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0c9ffb640 1 -- 172.21.15.107:0/555510886 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb0cc042050 con 0x562ba1e75240 2024-04-25T00:16:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0eb9ad740 10 client.15432 fetch_fsmap learned FSMap version 23 2024-04-25T00:16:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.676+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/555510886 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x562ba1ef1450 con 0x562ba1e75240 2024-04-25T00:16:18.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.677+0000 7fb0c9ffb640 1 -- 172.21.15.107:0/555510886 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0dc036040 con 0x562ba1e75240 2024-04-25T00:16:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.677+0000 7fb0c9ffb640 1 -- 172.21.15.107:0/555510886 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 23) ==== 8346+0+0 (secure 0 0 0) 0x7fb0cc0052c0 con 0x562ba1e75240 2024-04-25T00:16:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.677+0000 7fb0eb9ad740 10 client.15432 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:16:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.677+0000 7fb0eb9ad740 10 client.15432 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:16:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.677+0000 7fb0eb9ad740 20 client.15432 populate_metadata read hostname 'smithi107' 2024-04-25T00:16:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.677+0000 7fb0eb9ad740 1 --2- 172.21.15.107:0/555510886 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562ba1e8ba20 0x562ba1ef4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.677+0000 7fb0eb9ad740 4 client.15432 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:16:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.677+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/555510886 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "status"}) -- 0x562ba1ddb790 con 0x562ba1e8ba20 2024-04-25T00:16:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.677+0000 7fb0e259c640 1 --2- 172.21.15.107:0/555510886 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562ba1e8ba20 0x562ba1ef4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.677+0000 7fb0c9ffb640 10 client.15432 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:16:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.677+0000 7fb0e259c640 1 --2- 172.21.15.107:0/555510886 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562ba1e8ba20 0x562ba1ef4d40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3564d1c606b596b server_cookie=1cfb432573fdf8b8 in_seq=0 out_seq=0 2024-04-25T00:16:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0c9ffb640 1 -- 172.21.15.107:0/555510886 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562ba1ddb790 con 0x562ba1e8ba20 2024-04-25T00:16:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0c9ffb640 10 client.15432 handle_command_reply: tid=0 2024-04-25T00:16:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0eb9ad740 1 client.15432 shutdown 2024-04-25T00:16:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/555510886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e75240 msgr2=0x562ba1e861b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0eb9ad740 1 --2- 172.21.15.107:0/555510886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e75240 0x562ba1e861b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc005d80 tx=0x7fb0cc009a10 comp rx=0 tx=0).stop 2024-04-25T00:16:18.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/555510886 shutdown_connections 2024-04-25T00:16:18.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0eb9ad740 1 --2- 172.21.15.107:0/555510886 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562ba1e8ba20 0x562ba1ef4d40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0eb9ad740 1 --2- 172.21.15.107:0/555510886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ba1e7a3b0 0x562ba1eef9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0eb9ad740 1 --2- 172.21.15.107:0/555510886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ba1e75240 0x562ba1e861b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0c9ffb640 0 client.15432 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:16:18.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0eb9ad740 1 --2- 172.21.15.107:0/555510886 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ba1e74850 0x562ba1e78ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/555510886 >> 172.21.15.107:0/555510886 conn(0x562ba1e6bab0 msgr2=0x562ba1e76600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:18.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/555510886 shutdown_connections 2024-04-25T00:16:18.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0eb9ad740 1 -- 172.21.15.107:0/555510886 wait complete. 2024-04-25T00:16:18.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0eb9ad740 20 client.15432 trim_cache size 0 max 16384 2024-04-25T00:16:18.686 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:16:18.687 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "79602f58-0297-11ef-bc93-c7b262605968", 2024-04-25T00:16:18.687 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 0, 2024-04-25T00:16:18.687 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24521, 2024-04-25T00:16:18.687 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-04-25T00:16:18.687 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-04-25T00:16:18.687 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-04-25T00:16:18.687 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 135.89149393299999, 2024-04-25T00:16:18.687 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 23, 2024-04-25T00:16:18.687 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 96, 2024-04-25T00:16:18.687 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 92, 2024-04-25T00:16:18.687 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 141.290154073 2024-04-25T00:16:18.687 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:16:18.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0e8bc1640 1 -- 172.21.15.107:0/119338369 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb0d00a2c10 msgr2=0x7fb0d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0e8bc1640 1 --2- 172.21.15.107:0/119338369 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb0d00a2c10 0x7fb0d00a50d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb0e406a660 tx=0x7fb0cc0147e0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0e8bc1640 1 -- 172.21.15.107:0/119338369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0e4102230 msgr2=0x7fb0e419ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0e8bc1640 1 --2- 172.21.15.107:0/119338369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0e4102230 0x7fb0e419ede0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc031ae0 tx=0x7fb0dc031ee0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0e8bc1640 1 -- 172.21.15.107:0/119338369 shutdown_connections 2024-04-25T00:16:18.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0e8bc1640 1 --2- 172.21.15.107:0/119338369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0e4102230 0x7fb0e419ede0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0e8bc1640 1 --2- 172.21.15.107:0/119338369 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fb0d00a2c10 0x7fb0d00a50d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0e8bc1640 1 --2- 172.21.15.107:0/119338369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb0e4101870 0x7fb0e419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0e8bc1640 1 --2- 172.21.15.107:0/119338369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb0e4100680 0x7fb0e406a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0e8bc1640 1 -- 172.21.15.107:0/119338369 >> 172.21.15.107:0/119338369 conn(0x7fb0e40fbc30 msgr2=0x7fb0e4104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:18.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0e8bc1640 1 -- 172.21.15.107:0/119338369 shutdown_connections 2024-04-25T00:16:18.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.678+0000 7fb0e8bc1640 1 -- 172.21.15.107:0/119338369 wait complete. 2024-04-25T00:16:18.690 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T00:16:18.701 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T00:16:18.701 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T00:16:18.702 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T00:16:18.702 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-25T00:16:18.703 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T00:16:18.703 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T00:16:18.703 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-25T00:16:18.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.800+0000 7f3aecf9e640 1 -- 172.21.15.107:0/2928967678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae8107380 msgr2=0x7f3ae8109770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.800+0000 7f3aecf9e640 1 --2- 172.21.15.107:0/2928967678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae8107380 0x7f3ae8109770 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0009ca0 tx=0x7f3ae002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.800+0000 7f3aecf9e640 1 -- 172.21.15.107:0/2928967678 shutdown_connections 2024-04-25T00:16:18.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.800+0000 7f3aecf9e640 1 --2- 172.21.15.107:0/2928967678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae8107380 0x7f3ae8109770 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.800+0000 7f3aecf9e640 1 --2- 172.21.15.107:0/2928967678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ae8104a50 0x7f3ae8106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.800+0000 7f3aecf9e640 1 --2- 172.21.15.107:0/2928967678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ae8102120 0x7f3ae8104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.800+0000 7f3aecf9e640 1 -- 172.21.15.107:0/2928967678 >> 172.21.15.107:0/2928967678 conn(0x7f3ae80fba70 msgr2=0x7f3ae80fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:18.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.800+0000 7f3aecf9e640 1 -- 172.21.15.107:0/2928967678 shutdown_connections 2024-04-25T00:16:18.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.800+0000 7f3aecf9e640 1 -- 172.21.15.107:0/2928967678 wait complete. 2024-04-25T00:16:18.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.801+0000 7f3aecf9e640 1 Processor -- start 2024-04-25T00:16:18.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.801+0000 7f3aecf9e640 1 -- start start 2024-04-25T00:16:18.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.801+0000 7f3aecf9e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ae8102120 0x7f3ae8101a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.801+0000 7f3aecf9e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ae8104a50 0x7f3ae81000d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.801+0000 7f3aecf9e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae8107380 0x7f3ae8100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.801+0000 7f3aecf9e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae810d280 con 0x7f3ae8107380 2024-04-25T00:16:18.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.801+0000 7f3aecf9e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae810d100 con 0x7f3ae8104a50 2024-04-25T00:16:18.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.801+0000 7f3aecf9e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae810d400 con 0x7f3ae8102120 2024-04-25T00:16:18.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.801+0000 7f3ae7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae8107380 0x7f3ae8100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.802+0000 7f3ae7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae8107380 0x7f3ae8100610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57258/0 (socket says 172.21.15.107:57258) 2024-04-25T00:16:18.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.802+0000 7f3ae7fff640 1 -- 172.21.15.107:0/3795059668 learned_addr learned my addr 172.21.15.107:0/3795059668 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:18.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.802+0000 7f3ae6ffd640 1 --2- 172.21.15.107:0/3795059668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ae8104a50 0x7f3ae81000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.802+0000 7f3ae77fe640 1 --2- 172.21.15.107:0/3795059668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ae8102120 0x7f3ae8101a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.802+0000 7f3ae7fff640 1 -- 172.21.15.107:0/3795059668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ae8102120 msgr2=0x7f3ae8101a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.802+0000 7f3ae7fff640 1 --2- 172.21.15.107:0/3795059668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ae8102120 0x7f3ae8101a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.802+0000 7f3ae7fff640 1 -- 172.21.15.107:0/3795059668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ae8104a50 msgr2=0x7f3ae81000d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.802+0000 7f3ae7fff640 1 --2- 172.21.15.107:0/3795059668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ae8104a50 0x7f3ae81000d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.802+0000 7f3ae7fff640 1 -- 172.21.15.107:0/3795059668 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae8100ed0 con 0x7f3ae8107380 2024-04-25T00:16:18.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.802+0000 7f3ae6ffd640 1 --2- 172.21.15.107:0/3795059668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ae8104a50 0x7f3ae81000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:18.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.802+0000 7f3ae7fff640 1 --2- 172.21.15.107:0/3795059668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae8107380 0x7f3ae8100610 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0005e30 tx=0x7f3ae0031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:18.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.802+0000 7f3ae4ff9640 1 -- 172.21.15.107:0/3795059668 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae0003c30 con 0x7f3ae8107380 2024-04-25T00:16:18.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.802+0000 7f3aecf9e640 1 -- 172.21.15.107:0/3795059668 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae81a19e0 con 0x7f3ae8107380 2024-04-25T00:16:18.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.802+0000 7f3ae4ff9640 1 -- 172.21.15.107:0/3795059668 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ae0003dd0 con 0x7f3ae8107380 2024-04-25T00:16:18.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.802+0000 7f3ae4ff9640 1 -- 172.21.15.107:0/3795059668 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae002fde0 con 0x7f3ae8107380 2024-04-25T00:16:18.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.802+0000 7f3aecf9e640 1 -- 172.21.15.107:0/3795059668 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae81a1f00 con 0x7f3ae8107380 2024-04-25T00:16:18.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.803+0000 7f3aecf9e640 1 -- 172.21.15.107:0/3795059668 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae80fd560 con 0x7f3ae8107380 2024-04-25T00:16:18.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.804+0000 7f3ae4ff9640 1 -- 172.21.15.107:0/3795059668 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 137834+0+0 (secure 0 0 0) 0x7f3ae0004050 con 0x7f3ae8107380 2024-04-25T00:16:18.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.805+0000 7f3ae4ff9640 1 --2- 172.21.15.107:0/3795059668 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f3ad40a2b80 0x7f3ad40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.805+0000 7f3ae77fe640 1 --2- 172.21.15.107:0/3795059668 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f3ad40a2b80 0x7f3ad40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.805+0000 7f3ae4ff9640 1 -- 172.21.15.107:0/3795059668 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7f3ae00f0940 con 0x7f3ae8107380 2024-04-25T00:16:18.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.805+0000 7f3ae77fe640 1 --2- 172.21.15.107:0/3795059668 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f3ad40a2b80 0x7f3ad40a5040 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8005d00 tx=0x7f3ad8005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:18.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:18.809+0000 7f3ae4ff9640 1 -- 172.21.15.107:0/3795059668 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3ae00bd630 con 0x7f3ae8107380 2024-04-25T00:16:19.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.034+0000 7f3aecf9e640 1 -- 172.21.15.107:0/3795059668 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f3ae8062340 con 0x7f3ae8107380 2024-04-25T00:16:19.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.100+0000 7f3ae4ff9640 1 -- 172.21.15.107:0/3795059668 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f3ae00bcf30 con 0x7f3ae8107380 2024-04-25T00:16:19.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.102+0000 7f3aecf9e640 1 -- 172.21.15.107:0/3795059668 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f3ad40a2b80 msgr2=0x7f3ad40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:19.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.102+0000 7f3aecf9e640 1 --2- 172.21.15.107:0/3795059668 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f3ad40a2b80 0x7f3ad40a5040 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8005d00 tx=0x7f3ad8005c50 comp rx=0 tx=0).stop 2024-04-25T00:16:19.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.102+0000 7f3aecf9e640 1 -- 172.21.15.107:0/3795059668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae8107380 msgr2=0x7f3ae8100610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:19.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.102+0000 7f3aecf9e640 1 --2- 172.21.15.107:0/3795059668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae8107380 0x7f3ae8100610 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3ae0005e30 tx=0x7f3ae0031ee0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.102+0000 7f3aecf9e640 1 -- 172.21.15.107:0/3795059668 shutdown_connections 2024-04-25T00:16:19.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.102+0000 7f3aecf9e640 1 --2- 172.21.15.107:0/3795059668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae8107380 0x7f3ae8100610 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.102+0000 7f3aecf9e640 1 --2- 172.21.15.107:0/3795059668 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f3ad40a2b80 0x7f3ad40a5040 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.102+0000 7f3aecf9e640 1 --2- 172.21.15.107:0/3795059668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3ae8104a50 0x7f3ae81000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.102+0000 7f3aecf9e640 1 --2- 172.21.15.107:0/3795059668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ae8102120 0x7f3ae8101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.102+0000 7f3aecf9e640 1 -- 172.21.15.107:0/3795059668 >> 172.21.15.107:0/3795059668 conn(0x7f3ae80fba70 msgr2=0x7f3ae8102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:19.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.102+0000 7f3aecf9e640 1 -- 172.21.15.107:0/3795059668 shutdown_connections 2024-04-25T00:16:19.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.102+0000 7f3aecf9e640 1 -- 172.21.15.107:0/3795059668 wait complete. 2024-04-25T00:16:19.120 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-25T00:16:19.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.223+0000 7f6c93df9640 1 -- 172.21.15.107:0/315535833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c8c101890 msgr2=0x7f6c8c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:19.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.223+0000 7f6c93df9640 1 --2- 172.21.15.107:0/315535833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c8c101890 0x7f6c8c101d10 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6c80009900 tx=0x7f6c8002a410 comp rx=0 tx=0).stop 2024-04-25T00:16:19.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.223+0000 7f6c93df9640 1 -- 172.21.15.107:0/315535833 shutdown_connections 2024-04-25T00:16:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.223+0000 7f6c93df9640 1 --2- 172.21.15.107:0/315535833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c8c102250 0x7f6c8c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.223+0000 7f6c93df9640 1 --2- 172.21.15.107:0/315535833 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c8c101890 0x7f6c8c101d10 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.223+0000 7f6c93df9640 1 --2- 172.21.15.107:0/315535833 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c8c1006a0 0x7f6c8c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.223+0000 7f6c93df9640 1 -- 172.21.15.107:0/315535833 >> 172.21.15.107:0/315535833 conn(0x7f6c8c0fbc10 msgr2=0x7f6c8c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.224+0000 7f6c93df9640 1 -- 172.21.15.107:0/315535833 shutdown_connections 2024-04-25T00:16:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.224+0000 7f6c93df9640 1 -- 172.21.15.107:0/315535833 wait complete. 2024-04-25T00:16:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.224+0000 7f6c93df9640 1 Processor -- start 2024-04-25T00:16:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.224+0000 7f6c93df9640 1 -- start start 2024-04-25T00:16:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.224+0000 7f6c93df9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c8c1006a0 0x7f6c8c10b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.224+0000 7f6c93df9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c8c101890 0x7f6c8c109660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.224+0000 7f6c93df9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c8c102250 0x7f6c8c109ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.224+0000 7f6c93df9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c10d440 con 0x7f6c8c101890 2024-04-25T00:16:19.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.224+0000 7f6c93df9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c10d2c0 con 0x7f6c8c102250 2024-04-25T00:16:19.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.224+0000 7f6c93df9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c10d5c0 con 0x7f6c8c1006a0 2024-04-25T00:16:19.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.224+0000 7f6c91394640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c8c101890 0x7f6c8c109660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:19.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.224+0000 7f6c91394640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c8c101890 0x7f6c8c109660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57302/0 (socket says 172.21.15.107:57302) 2024-04-25T00:16:19.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.224+0000 7f6c91394640 1 -- 172.21.15.107:0/2008270898 learned_addr learned my addr 172.21.15.107:0/2008270898 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:19.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.224+0000 7f6c91b95640 1 --2- 172.21.15.107:0/2008270898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c8c1006a0 0x7f6c8c10b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:19.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.224+0000 7f6c92396640 1 --2- 172.21.15.107:0/2008270898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c8c102250 0x7f6c8c109ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:19.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.225+0000 7f6c91394640 1 -- 172.21.15.107:0/2008270898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c8c1006a0 msgr2=0x7f6c8c10b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:19.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.225+0000 7f6c91394640 1 --2- 172.21.15.107:0/2008270898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c8c1006a0 0x7f6c8c10b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.225+0000 7f6c91394640 1 -- 172.21.15.107:0/2008270898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c8c102250 msgr2=0x7f6c8c109ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:19.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.225+0000 7f6c91394640 1 --2- 172.21.15.107:0/2008270898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c8c102250 0x7f6c8c109ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.225+0000 7f6c91394640 1 -- 172.21.15.107:0/2008270898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8c10a430 con 0x7f6c8c101890 2024-04-25T00:16:19.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.225+0000 7f6c91b95640 1 --2- 172.21.15.107:0/2008270898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c8c1006a0 0x7f6c8c10b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:19.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.225+0000 7f6c91394640 1 --2- 172.21.15.107:0/2008270898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c8c101890 0x7f6c8c109660 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6c80009230 tx=0x7f6c80004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:19.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.225+0000 7f6c8affd640 1 -- 172.21.15.107:0/2008270898 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8003f030 con 0x7f6c8c101890 2024-04-25T00:16:19.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.225+0000 7f6c93df9640 1 -- 172.21.15.107:0/2008270898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8c1a1af0 con 0x7f6c8c101890 2024-04-25T00:16:19.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.225+0000 7f6c8affd640 1 -- 172.21.15.107:0/2008270898 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c80003ce0 con 0x7f6c8c101890 2024-04-25T00:16:19.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.225+0000 7f6c8affd640 1 -- 172.21.15.107:0/2008270898 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c80035070 con 0x7f6c8c101890 2024-04-25T00:16:19.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.225+0000 7f6c93df9640 1 -- 172.21.15.107:0/2008270898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c8c1a2090 con 0x7f6c8c101890 2024-04-25T00:16:19.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.226+0000 7f6c93df9640 1 -- 172.21.15.107:0/2008270898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c8c107960 con 0x7f6c8c101890 2024-04-25T00:16:19.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.227+0000 7f6c8affd640 1 -- 172.21.15.107:0/2008270898 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7f6c80042030 con 0x7f6c8c101890 2024-04-25T00:16:19.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.228+0000 7f6c8affd640 1 --2- 172.21.15.107:0/2008270898 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f6c740a2c80 0x7f6c740a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:19.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.228+0000 7f6c91b95640 1 -- 172.21.15.107:0/2008270898 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f6c740a2c80 msgr2=0x7f6c740a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1644942185 2024-04-25T00:16:19.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.228+0000 7f6c8affd640 1 -- 172.21.15.107:0/2008270898 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7f6c800efda0 con 0x7f6c8c101890 2024-04-25T00:16:19.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.228+0000 7f6c91b95640 1 --2- 172.21.15.107:0/2008270898 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f6c740a2c80 0x7f6c740a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:16:19.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.232+0000 7f6c8affd640 1 -- 172.21.15.107:0/2008270898 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6c800bc320 con 0x7f6c8c101890 2024-04-25T00:16:19.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.412+0000 7f6c93df9640 1 -- 172.21.15.107:0/2008270898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f6c8c02d170 con 0x7f6c8c101890 2024-04-25T00:16:19.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.416+0000 7f6c8affd640 1 -- 172.21.15.107:0/2008270898 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f6c800c0620 con 0x7f6c8c101890 2024-04-25T00:16:19.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.419+0000 7f6c93df9640 1 -- 172.21.15.107:0/2008270898 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f6c740a2c80 msgr2=0x7f6c740a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:16:19.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.419+0000 7f6c93df9640 1 --2- 172.21.15.107:0/2008270898 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f6c740a2c80 0x7f6c740a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.419+0000 7f6c93df9640 1 -- 172.21.15.107:0/2008270898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c8c101890 msgr2=0x7f6c8c109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:19.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.419+0000 7f6c93df9640 1 --2- 172.21.15.107:0/2008270898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c8c101890 0x7f6c8c109660 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6c80009230 tx=0x7f6c80004360 comp rx=0 tx=0).stop 2024-04-25T00:16:19.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.419+0000 7f6c93df9640 1 -- 172.21.15.107:0/2008270898 shutdown_connections 2024-04-25T00:16:19.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.419+0000 7f6c93df9640 1 --2- 172.21.15.107:0/2008270898 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f6c740a2c80 0x7f6c740a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.419+0000 7f6c93df9640 1 --2- 172.21.15.107:0/2008270898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c8c102250 0x7f6c8c109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.419+0000 7f6c93df9640 1 --2- 172.21.15.107:0/2008270898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c8c101890 0x7f6c8c109660 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.419+0000 7f6c93df9640 1 --2- 172.21.15.107:0/2008270898 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6c8c1006a0 0x7f6c8c10b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.419+0000 7f6c93df9640 1 -- 172.21.15.107:0/2008270898 >> 172.21.15.107:0/2008270898 conn(0x7f6c8c0fbc10 msgr2=0x7f6c8c104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:19.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.419+0000 7f6c93df9640 1 -- 172.21.15.107:0/2008270898 shutdown_connections 2024-04-25T00:16:19.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.419+0000 7f6c93df9640 1 -- 172.21.15.107:0/2008270898 wait complete. 2024-04-25T00:16:19.433 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:19 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1732453822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:19.433 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:19 smithi107 ceph-mon[29153]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:16:19.433 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:19 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3795059668' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T00:16:19.433 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:19 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: ignoring --setuser ceph since I am not root 2024-04-25T00:16:19.434 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:19 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: ignoring --setgroup ceph since I am not root 2024-04-25T00:16:19.434 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:19 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:19.285+0000 7f75486bf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:16:19.435 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-25T00:16:19.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:19 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1732453822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:19.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:19 smithi170 ceph-mon[32807]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:16:19.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:19 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3795059668' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T00:16:19.436 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:19 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: ignoring --setuser ceph since I am not root 2024-04-25T00:16:19.436 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:19 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: ignoring --setgroup ceph since I am not root 2024-04-25T00:16:19.436 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:19 smithi170 ceph-mgr[34198]: -- 172.21.15.170:0/995488486 <== mon.2 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5628986005a0 con 0x5628985d4800 2024-04-25T00:16:19.436 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:19 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:19.280+0000 7fbcb1a3e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:16:19.462 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:19 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1732453822' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:19.462 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:19 smithi195 ceph-mon[32833]: pgmap v365: 129 pgs: 129 active+clean; 643 KiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:16:19.462 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:19 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3795059668' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T00:16:19.463 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:19 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: ignoring --setuser ceph since I am not root 2024-04-25T00:16:19.463 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:19 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: ignoring --setgroup ceph since I am not root 2024-04-25T00:16:19.463 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:19 smithi195 ceph-mgr[34706]: -- 172.21.15.195:0/3782976444 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x565369cb65a0 con 0x565369c8a800 2024-04-25T00:16:19.463 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:19 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:19.276+0000 7f7cf6a78200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:16:19.463 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:19 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:19.460+0000 7f7cf6a78200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:16:19.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.537+0000 7fc7f997e640 1 -- 172.21.15.107:0/241016636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7f4100f90 msgr2=0x7fc7f4101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:19.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.537+0000 7fc7f997e640 1 --2- 172.21.15.107:0/241016636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7f4100f90 0x7fc7f4101410 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8009900 tx=0x7fc7e802a410 comp rx=0 tx=0).stop 2024-04-25T00:16:19.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.537+0000 7fc7f997e640 1 -- 172.21.15.107:0/241016636 shutdown_connections 2024-04-25T00:16:19.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.537+0000 7fc7f997e640 1 --2- 172.21.15.107:0/241016636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7f4101950 0x7fc7f4108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.537+0000 7fc7f997e640 1 --2- 172.21.15.107:0/241016636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7f4100f90 0x7fc7f4101410 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.537+0000 7fc7f997e640 1 --2- 172.21.15.107:0/241016636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7f40ffe40 0x7fc7f4100260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.537+0000 7fc7f997e640 1 -- 172.21.15.107:0/241016636 >> 172.21.15.107:0/241016636 conn(0x7fc7f40fb3b0 msgr2=0x7fc7f40fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:19.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.537+0000 7fc7f997e640 1 -- 172.21.15.107:0/241016636 shutdown_connections 2024-04-25T00:16:19.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.537+0000 7fc7f997e640 1 -- 172.21.15.107:0/241016636 wait complete. 2024-04-25T00:16:19.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.537+0000 7fc7f997e640 1 Processor -- start 2024-04-25T00:16:19.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.538+0000 7fc7f997e640 1 -- start start 2024-04-25T00:16:19.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.538+0000 7fc7f997e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7f40ffe40 0x7fc7f41969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:19.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.538+0000 7fc7f997e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7f4100f90 0x7fc7f4196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:19.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.538+0000 7fc7f997e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7f4101950 0x7fc7f419df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:19.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.538+0000 7fc7f997e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f410c3a0 con 0x7fc7f40ffe40 2024-04-25T00:16:19.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.538+0000 7fc7f997e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f410c220 con 0x7fc7f4100f90 2024-04-25T00:16:19.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.538+0000 7fc7f997e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f410c520 con 0x7fc7f4101950 2024-04-25T00:16:19.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.538+0000 7fc7f2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7f40ffe40 0x7fc7f41969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:19.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.538+0000 7fc7f2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7f40ffe40 0x7fc7f41969d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59170/0 (socket says 172.21.15.107:59170) 2024-04-25T00:16:19.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.538+0000 7fc7f2ffd640 1 -- 172.21.15.107:0/3457246365 learned_addr learned my addr 172.21.15.107:0/3457246365 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:19.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.538+0000 7fc7f27fc640 1 --2- 172.21.15.107:0/3457246365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7f4100f90 0x7fc7f4196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:19.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.538+0000 7fc7f37fe640 1 --2- 172.21.15.107:0/3457246365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7f4101950 0x7fc7f419df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:19.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.538+0000 7fc7f27fc640 1 -- 172.21.15.107:0/3457246365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7f4101950 msgr2=0x7fc7f419df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:19.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.538+0000 7fc7f27fc640 1 --2- 172.21.15.107:0/3457246365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7f4101950 0x7fc7f419df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.538+0000 7fc7f27fc640 1 -- 172.21.15.107:0/3457246365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7f40ffe40 msgr2=0x7fc7f41969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:19.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.538+0000 7fc7f27fc640 1 --2- 172.21.15.107:0/3457246365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7f40ffe40 0x7fc7f41969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.538+0000 7fc7f27fc640 1 -- 172.21.15.107:0/3457246365 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f419e5f0 con 0x7fc7f4100f90 2024-04-25T00:16:19.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.538+0000 7fc7f2ffd640 1 --2- 172.21.15.107:0/3457246365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7f40ffe40 0x7fc7f41969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:19.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.538+0000 7fc7f27fc640 1 --2- 172.21.15.107:0/3457246365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7f4100f90 0x7fc7f4196f10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8009230 tx=0x7fc7e8032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:19.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.539+0000 7fc7f897c640 1 -- 172.21.15.107:0/3457246365 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e803f030 con 0x7fc7f4100f90 2024-04-25T00:16:19.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.539+0000 7fc7f997e640 1 -- 172.21.15.107:0/3457246365 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f419e880 con 0x7fc7f4100f90 2024-04-25T00:16:19.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.539+0000 7fc7f897c640 1 -- 172.21.15.107:0/3457246365 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7e8003ce0 con 0x7fc7f4100f90 2024-04-25T00:16:19.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.539+0000 7fc7f897c640 1 -- 172.21.15.107:0/3457246365 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8035070 con 0x7fc7f4100f90 2024-04-25T00:16:19.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.539+0000 7fc7f997e640 1 -- 172.21.15.107:0/3457246365 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f419ed90 con 0x7fc7f4100f90 2024-04-25T00:16:19.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.540+0000 7fc7f997e640 1 -- 172.21.15.107:0/3457246365 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f4107080 con 0x7fc7f4100f90 2024-04-25T00:16:19.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.541+0000 7fc7f897c640 1 -- 172.21.15.107:0/3457246365 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7fc7e8042030 con 0x7fc7f4100f90 2024-04-25T00:16:19.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.541+0000 7fc7f897c640 1 --2- 172.21.15.107:0/3457246365 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc7dc0a2c80 0x7fc7dc0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:19.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.541+0000 7fc7f2ffd640 1 -- 172.21.15.107:0/3457246365 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc7dc0a2c80 msgr2=0x7fc7dc0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1644942185 2024-04-25T00:16:19.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.541+0000 7fc7f897c640 1 -- 172.21.15.107:0/3457246365 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7fc7e80efad0 con 0x7fc7f4100f90 2024-04-25T00:16:19.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.541+0000 7fc7f2ffd640 1 --2- 172.21.15.107:0/3457246365 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc7dc0a2c80 0x7fc7dc0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:16:19.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.545+0000 7fc7f897c640 1 -- 172.21.15.107:0/3457246365 <== mon.2 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc7e80bbff0 con 0x7fc7f4100f90 2024-04-25T00:16:19.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.727+0000 7fc7f997e640 1 -- 172.21.15.107:0/3457246365 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fc7f402d170 con 0x7fc7f4100f90 2024-04-25T00:16:19.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.732+0000 7fc7f897c640 1 -- 172.21.15.107:0/3457246365 <== mon.2 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7fc7e80c02f0 con 0x7fc7f4100f90 2024-04-25T00:16:19.734 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:19 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:19.477+0000 7f75486bf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:16:19.734 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:19 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:19.586+0000 7f75486bf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:16:19.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.735+0000 7fc7f997e640 1 -- 172.21.15.107:0/3457246365 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc7dc0a2c80 msgr2=0x7fc7dc0a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:16:19.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.735+0000 7fc7f997e640 1 --2- 172.21.15.107:0/3457246365 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc7dc0a2c80 0x7fc7dc0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.735+0000 7fc7f997e640 1 -- 172.21.15.107:0/3457246365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7f4100f90 msgr2=0x7fc7f4196f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:19.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.735+0000 7fc7f997e640 1 --2- 172.21.15.107:0/3457246365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7f4100f90 0x7fc7f4196f10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8009230 tx=0x7fc7e8032ce0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.735+0000 7fc7f997e640 1 -- 172.21.15.107:0/3457246365 shutdown_connections 2024-04-25T00:16:19.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.735+0000 7fc7f997e640 1 --2- 172.21.15.107:0/3457246365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7f4101950 0x7fc7f419df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.735+0000 7fc7f997e640 1 --2- 172.21.15.107:0/3457246365 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7fc7dc0a2c80 0x7fc7dc0a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.735+0000 7fc7f997e640 1 --2- 172.21.15.107:0/3457246365 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7f4100f90 0x7fc7f4196f10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.735+0000 7fc7f997e640 1 --2- 172.21.15.107:0/3457246365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7f40ffe40 0x7fc7f41969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.735+0000 7fc7f997e640 1 -- 172.21.15.107:0/3457246365 >> 172.21.15.107:0/3457246365 conn(0x7fc7f40fb3b0 msgr2=0x7fc7f4104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:19.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.735+0000 7fc7f997e640 1 -- 172.21.15.107:0/3457246365 shutdown_connections 2024-04-25T00:16:19.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:19.735+0000 7fc7f997e640 1 -- 172.21.15.107:0/3457246365 wait complete. 2024-04-25T00:16:19.747 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T00:16:19.764 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T00:16:19.764 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T00:16:19.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:16:19.764 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-04-25T00:16:19.779 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T00:16:19.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:16:19.780 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T00:16:19.839 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:19 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:19.564+0000 7f7cf6a78200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:16:19.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:16:19.845 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2024-04-25T00:16:19.872 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:19 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:19.463+0000 7fbcb1a3e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:16:19.872 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:19 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:19.566+0000 7fbcb1a3e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:16:19.944 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-25T00:16:19.944 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T00:16:19.944 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:16:19.944 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T00:16:20.005 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:19 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:19.912+0000 7f75486bf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:16:20.005 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:20 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:20.004+0000 7f75486bf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:16:20.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:20 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3795059668' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T00:16:20.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:20 smithi170 ceph-mon[32807]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T00:16:20.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:20 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2008270898' entity='client.admin' 2024-04-25T00:16:20.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:20 smithi170 ceph-mon[32807]: from='client.? ' entity='client.admin' 2024-04-25T00:16:20.185 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:19 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:19.872+0000 7fbcb1a3e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:16:20.185 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:19 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:19.956+0000 7fbcb1a3e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:16:20.195 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:16:20.317 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:20 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3795059668' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T00:16:20.317 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:20 smithi107 ceph-mon[29153]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T00:16:20.317 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:20 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2008270898' entity='client.admin' 2024-04-25T00:16:20.317 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:20 smithi107 ceph-mon[29153]: from='client.? ' entity='client.admin' 2024-04-25T00:16:20.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:20 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3795059668' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T00:16:20.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:20 smithi195 ceph-mon[32833]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-25T00:16:20.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:20 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2008270898' entity='client.admin' 2024-04-25T00:16:20.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:20 smithi195 ceph-mon[32833]: from='client.? ' entity='client.admin' 2024-04-25T00:16:20.339 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:19 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:19.874+0000 7f7cf6a78200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:16:20.339 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:19 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:19.959+0000 7f7cf6a78200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:16:21.400 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:21 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:21.227+0000 7f75486bf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:16:21.400 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:21 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:21.399+0000 7f75486bf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:16:21.434 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:21 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:21.125+0000 7fbcb1a3e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:16:21.435 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:21 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:21.293+0000 7fbcb1a3e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:16:21.483 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:21 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:21.143+0000 7f7cf6a78200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:16:21.483 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:21 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:21.312+0000 7f7cf6a78200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:16:21.734 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:21 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:21.579+0000 7f75486bf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:16:21.748 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:21 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:21.463+0000 7fbcb1a3e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:16:21.774 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:21 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:21.482+0000 7f7cf6a78200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:16:22.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.011+0000 7f7124fa6700 1 -- 172.21.15.107:0/157838978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7120101d00 msgr2=0x7f7120102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:22.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.011+0000 7f7124fa6700 1 --2- 172.21.15.107:0/157838978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7120101d00 0x7f7120102180 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f71140099c0 tx=0x7f7114009cd0 comp rx=0 tx=0).stop 2024-04-25T00:16:22.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.011+0000 7f7124fa6700 1 -- 172.21.15.107:0/157838978 shutdown_connections 2024-04-25T00:16:22.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.011+0000 7f7124fa6700 1 --2- 172.21.15.107:0/157838978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f71201026c0 0x7f712010e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:22.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.011+0000 7f7124fa6700 1 --2- 172.21.15.107:0/157838978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7120107420 0x7f7120107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:22.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.011+0000 7f7124fa6700 1 --2- 172.21.15.107:0/157838978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7120101d00 0x7f7120102180 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:22.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.011+0000 7f7124fa6700 1 -- 172.21.15.107:0/157838978 >> 172.21.15.107:0/157838978 conn(0x7f7120076450 msgr2=0x7f7120076860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:22.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.011+0000 7f7124fa6700 1 -- 172.21.15.107:0/157838978 shutdown_connections 2024-04-25T00:16:22.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.011+0000 7f7124fa6700 1 -- 172.21.15.107:0/157838978 wait complete. 2024-04-25T00:16:22.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.012+0000 7f7124fa6700 1 Processor -- start 2024-04-25T00:16:22.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.012+0000 7f7124fa6700 1 -- start start 2024-04-25T00:16:22.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.012+0000 7f7124fa6700 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7120101d00 0x7f712019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:22.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.012+0000 7f7124fa6700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71201026c0 0x7f712019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:22.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.012+0000 7f7124fa6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7120107420 0x7f71201988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:22.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.012+0000 7f7124fa6700 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7120111590 con 0x7f71201026c0 2024-04-25T00:16:22.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.012+0000 7f7124fa6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7120111410 con 0x7f7120107420 2024-04-25T00:16:22.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.012+0000 7f7124fa6700 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7120111710 con 0x7f7120101d00 2024-04-25T00:16:22.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.012+0000 7f711dd9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71201026c0 0x7f712019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:22.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.012+0000 7f711dd9b700 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71201026c0 0x7f712019ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59180/0 (socket says 172.21.15.107:59180) 2024-04-25T00:16:22.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.012+0000 7f711dd9b700 1 -- 172.21.15.107:0/56214388 learned_addr learned my addr 172.21.15.107:0/56214388 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:22.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.012+0000 7f711ed9d700 1 --2- 172.21.15.107:0/56214388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7120107420 0x7f71201988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:22.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.012+0000 7f711e59c700 1 --2- 172.21.15.107:0/56214388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7120101d00 0x7f712019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:22.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.012+0000 7f711dd9b700 1 -- 172.21.15.107:0/56214388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7120101d00 msgr2=0x7f712019e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:22.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.012+0000 7f711dd9b700 1 --2- 172.21.15.107:0/56214388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7120101d00 0x7f712019e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:22.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.012+0000 7f711dd9b700 1 -- 172.21.15.107:0/56214388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7120107420 msgr2=0x7f71201988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:22.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.012+0000 7f711dd9b700 1 --2- 172.21.15.107:0/56214388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7120107420 0x7f71201988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:22.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.012+0000 7f711dd9b700 1 -- 172.21.15.107:0/56214388 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7120199000 con 0x7f71201026c0 2024-04-25T00:16:22.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.013+0000 7f711dd9b700 1 --2- 172.21.15.107:0/56214388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71201026c0 0x7f712019ec20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f71140059d0 tx=0x7f71140038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:22.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.013+0000 7f710f7fe700 1 -- 172.21.15.107:0/56214388 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114027070 con 0x7f71201026c0 2024-04-25T00:16:22.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.013+0000 7f7124fa6700 1 -- 172.21.15.107:0/56214388 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7120199290 con 0x7f71201026c0 2024-04-25T00:16:22.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.013+0000 7f710f7fe700 1 -- 172.21.15.107:0/56214388 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7114003e40 con 0x7f71201026c0 2024-04-25T00:16:22.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.013+0000 7f710f7fe700 1 -- 172.21.15.107:0/56214388 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7114022d00 con 0x7f71201026c0 2024-04-25T00:16:22.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.013+0000 7f7124fa6700 1 -- 172.21.15.107:0/56214388 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71201a55b0 con 0x7f71201026c0 2024-04-25T00:16:22.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.014+0000 7f7124fa6700 1 -- 172.21.15.107:0/56214388 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7120068f20 con 0x7f71201026c0 2024-04-25T00:16:22.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.015+0000 7f710f7fe700 1 -- 172.21.15.107:0/56214388 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7f711402a070 con 0x7f71201026c0 2024-04-25T00:16:22.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.016+0000 7f710f7fe700 1 --2- 172.21.15.107:0/56214388 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f71080a2c70 0x7f71080a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:22.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.016+0000 7f711e59c700 1 -- 172.21.15.107:0/56214388 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f71080a2c70 msgr2=0x7f71080a5130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1644942185 2024-04-25T00:16:22.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.016+0000 7f710f7fe700 1 -- 172.21.15.107:0/56214388 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7f71140d02e0 con 0x7f71201026c0 2024-04-25T00:16:22.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.016+0000 7f711e59c700 1 --2- 172.21.15.107:0/56214388 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f71080a2c70 0x7f71080a5130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:16:22.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.020+0000 7f710f7fe700 1 -- 172.21.15.107:0/56214388 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f711405e740 con 0x7f71201026c0 2024-04-25T00:16:22.022 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:21 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:21.887+0000 7f75486bf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:16:22.022 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:21 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:21.986+0000 7f75486bf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:16:22.043 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:21 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:21.747+0000 7fbcb1a3e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:16:22.043 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:21 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:21.838+0000 7fbcb1a3e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:16:22.043 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:22 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:22.042+0000 7fbcb1a3e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:16:22.072 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:21 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:21.774+0000 7f7cf6a78200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:16:22.072 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:21 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:21.867+0000 7f7cf6a78200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:16:22.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.217+0000 7f711e59c700 1 -- 172.21.15.107:0/56214388 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f71080a2c70 msgr2=0x7f71080a5130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1644942185 2024-04-25T00:16:22.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.217+0000 7f711e59c700 1 --2- 172.21.15.107:0/56214388 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f71080a2c70 0x7f71080a5130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T00:16:22.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.269+0000 7f7124fa6700 1 -- 172.21.15.107:0/56214388 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f71201a5780 con 0x7f71201026c0 2024-04-25T00:16:22.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.270+0000 7f710f7fe700 1 -- 172.21.15.107:0/56214388 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+88 (secure 0 0 0) 0x7f7114013070 con 0x7f71201026c0 2024-04-25T00:16:22.271 INFO:teuthology.orchestra.run.smithi107.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T00:16:22.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.272+0000 7f7124fa6700 1 -- 172.21.15.107:0/56214388 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f71080a2c70 msgr2=0x7f71080a5130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:16:22.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.272+0000 7f7124fa6700 1 --2- 172.21.15.107:0/56214388 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f71080a2c70 0x7f71080a5130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:22.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.272+0000 7f7124fa6700 1 -- 172.21.15.107:0/56214388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71201026c0 msgr2=0x7f712019ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:22.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.272+0000 7f7124fa6700 1 --2- 172.21.15.107:0/56214388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71201026c0 0x7f712019ec20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f71140059d0 tx=0x7f71140038b0 comp rx=0 tx=0).stop 2024-04-25T00:16:22.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.272+0000 7f7124fa6700 1 -- 172.21.15.107:0/56214388 shutdown_connections 2024-04-25T00:16:22.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.272+0000 7f7124fa6700 1 --2- 172.21.15.107:0/56214388 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f71080a2c70 0x7f71080a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:22.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.272+0000 7f7124fa6700 1 --2- 172.21.15.107:0/56214388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7120107420 0x7f71201988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:22.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.272+0000 7f7124fa6700 1 --2- 172.21.15.107:0/56214388 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7120101d00 0x7f712019e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:22.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.272+0000 7f7124fa6700 1 --2- 172.21.15.107:0/56214388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71201026c0 0x7f712019ec20 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:22.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.272+0000 7f7124fa6700 1 -- 172.21.15.107:0/56214388 >> 172.21.15.107:0/56214388 conn(0x7f7120076450 msgr2=0x7f7120100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:22.274 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:22 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:22.197+0000 7f75486bf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:16:22.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.272+0000 7f7124fa6700 1 -- 172.21.15.107:0/56214388 shutdown_connections 2024-04-25T00:16:22.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:22.272+0000 7f7124fa6700 1 -- 172.21.15.107:0/56214388 wait complete. 2024-04-25T00:16:22.323 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:22 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:22.233+0000 7fbcb1a3e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:16:22.323 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:22 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:22.071+0000 7f7cf6a78200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:16:22.323 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:22 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:22.264+0000 7f7cf6a78200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:16:22.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:22 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/56214388' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T00:16:22.637 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:22 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/56214388' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T00:16:22.637 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:22 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:22.398+0000 7f75486bf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:16:22.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:22 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/56214388' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T00:16:22.984 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:16:22.984 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:16:22.984 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi107.front.sepia.ceph.com 2024-04-25T00:16:22.984 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:16:22.984 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:16:22.984 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:16:22.984 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:16:22.985 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:16:22.985 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:16:22.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:16:22.985 DEBUG:teuthology.orchestra.run.smithi107:> ip addr 2024-04-25T00:16:23.002 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:16:23.002 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:16:23.002 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:16:23.003 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:16:23.003 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2024-04-25T00:16:23.003 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:16:23.003 INFO:teuthology.orchestra.run.smithi107.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:16:23.003 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0a brd ff:ff:ff:ff:ff:ff 2024-04-25T00:16:23.003 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f0 2024-04-25T00:16:23.004 INFO:teuthology.orchestra.run.smithi107.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:16:23.004 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0b brd ff:ff:ff:ff:ff:ff 2024-04-25T00:16:23.004 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f1 2024-04-25T00:16:23.004 INFO:teuthology.orchestra.run.smithi107.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:16:23.004 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:ea brd ff:ff:ff:ff:ff:ff 2024-04-25T00:16:23.004 INFO:teuthology.orchestra.run.smithi107.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:16:23.004 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:eb brd ff:ff:ff:ff:ff:ff 2024-04-25T00:16:23.005 INFO:teuthology.orchestra.run.smithi107.stdout: inet 172.21.15.107/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T00:16:23.005 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft 42091sec preferred_lft 42091sec 2024-04-25T00:16:23.005 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::d21f:963:7e10:29d9/64 scope link noprefixroute 2024-04-25T00:16:23.005 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:16:23.005 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T00:16:23.006 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:16:23.006 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-25T00:16:23.006 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add name ceph-brx type bridge 2024-04-25T00:16:23.006 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr flush dev ceph-brx 2024-04-25T00:16:23.007 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set ceph-brx up 2024-04-25T00:16:23.007 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T00:16:23.007 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-25T00:16:23.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:16:23.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:16:23.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:16:23.145 DEBUG:teuthology.orchestra.run.smithi107:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:16:23.170 INFO:teuthology.orchestra.run.smithi107.stdout:1 2024-04-25T00:16:23.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:16:23.171 DEBUG:teuthology.orchestra.run.smithi107:> route 2024-04-25T00:16:23.184 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:22 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:22.830+0000 7fbcb1a3e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:16:23.229 INFO:teuthology.orchestra.run.smithi107.stdout:Kernel IP routing table 2024-04-25T00:16:23.229 INFO:teuthology.orchestra.run.smithi107.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:16:23.229 INFO:teuthology.orchestra.run.smithi107.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T00:16:23.230 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T00:16:23.230 INFO:teuthology.orchestra.run.smithi107.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:16:23.230 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:16:23.230 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-25T00:16:23.230 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T00:16:23.230 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T00:16:23.231 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T00:16:23.231 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-25T00:16:23.234 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:23 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:23.009+0000 7f75486bf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:16:23.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:16:23.292 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:22 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:22.866+0000 7f7cf6a78200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:16:23.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:16:23.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:16:23.319 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-04-25T00:16:23.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:16:23.334 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list-id 2024-04-25T00:16:23.390 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:16:23.390 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-25T00:16:23.390 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:16:23.390 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T00:16:23.391 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-25T00:16:23.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:16:23.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:16:23.507 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T00:16:23.507 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:16:23.507 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-25T00:16:23.507 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T00:16:23.507 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T00:16:23.507 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T00:16:23.508 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T00:16:23.508 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T00:16:23.508 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-25T00:16:23.549 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:23 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:23.252+0000 7fbcb1a3e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:16:23.550 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:23 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:23.349+0000 7fbcb1a3e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:16:23.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:16:23.564 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:23 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:23.464+0000 7f75486bf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:16:23.589 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:23 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:23.292+0000 7f7cf6a78200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:16:23.590 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:23 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:23.390+0000 7f7cf6a78200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:16:23.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:16:23.620 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:16:23.620 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-25T00:16:23.620 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set brx.0 up 2024-04-25T00:16:23.620 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T00:16:23.620 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-25T00:16:23.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:16:23.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:16:23.695 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:16:23.696 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:16:23.710 INFO:teuthology.orchestra.run.smithi107.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:16:23.710 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:16:23.710 DEBUG:teuthology.orchestra.run.smithi107:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:16:23.765 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T00:16:23.765 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T00:16:23.765 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/026a421b-09bd-4435-ad20-5fa0f0e1edd5 2024-04-25T00:16:23.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:16:23.765 DEBUG:teuthology.orchestra.run.smithi107:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/026a421b-09bd-4435-ad20-5fa0f0e1edd5 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-25T00:16:23.825 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:23 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:23.569+0000 7f75486bf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:16:23.825 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:23 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:23.784+0000 7f75486bf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:16:23.846 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-25T00:16:23.846 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-04-25T00:16:23.846 INFO:teuthology.orchestra.run.smithi107.stdout:invalid new device string format 2024-04-25T00:16:23.934 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:23 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:23.549+0000 7fbcb1a3e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:16:23.970 INFO:teuthology.orchestra.run.smithi107.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-04-25T00:16:23.970 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-04-25T00:16:23.971 INFO:teuthology.orchestra.run.smithi107.stdout:invalid new device string format 2024-04-25T00:16:23.971 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: resolved to: "172.21.15.107:3300,172.21.15.170:3300,172.21.15.195:3300" 2024-04-25T00:16:23.971 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.107:3300,172.21.15.170:3300,172.21.15.195:3300:/volumes/qa/sv_1/026a421b-09bd-4435-ad20-5fa0f0e1edd5 2024-04-25T00:16:23.971 INFO:teuthology.orchestra.run.smithi107.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=79602f58-0297-11ef-bc93-c7b262605968" will pass to kernel 2024-04-25T00:16:23.971 INFO:teuthology.orchestra.run.smithi107.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T00:16:23.971 INFO:teuthology.orchestra.run.smithi107.stdout: You just mounted a file system that supports labels which does not 2024-04-25T00:16:23.972 INFO:teuthology.orchestra.run.smithi107.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T00:16:23.972 INFO:teuthology.orchestra.run.smithi107.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T00:16:23.972 INFO:teuthology.orchestra.run.smithi107.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T00:16:23.973 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:16:23.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:16:23.973 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:16:24.040 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:16:24.089 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:23 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:23.592+0000 7f7cf6a78200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:16:24.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:16:24.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:16:24.136 INFO:teuthology.orchestra.run.smithi107.stdout:client25106 2024-04-25T00:16:24.137 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T00:16:24.137 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:16:24.153 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:16:24.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:16:24.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:16:24.249 INFO:teuthology.orchestra.run.smithi107.stdout:client25106 2024-04-25T00:16:24.249 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:16:24.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:16:24.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:16:24.305 INFO:teuthology.orchestra.run.smithi107.stdout:client25106 2024-04-25T00:16:24.305 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/79602f58-0297-11ef-bc93-c7b262605968.client25106/status') 2024-04-25T00:16:24.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:16:24.362 INFO:teuthology.orchestra.run.smithi107.stdout:instance: client.25106 (3)172.21.15.107:0/1680170686 2024-04-25T00:16:24.363 INFO:teuthology.orchestra.run.smithi107.stdout:blocklisted: false 2024-04-25T00:16:24.363 INFO:teuthology.orchestra.run.smithi107.stderr:0+1 records in 2024-04-25T00:16:24.363 INFO:teuthology.orchestra.run.smithi107.stderr:0+1 records out 2024-04-25T00:16:24.363 INFO:teuthology.orchestra.run.smithi107.stderr:72 bytes copied, 2.2485e-05 s, 3.2 MB/s 2024-04-25T00:16:24.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:16:24.370 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T00:16:24.379 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T00:16:24.380 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T00:16:24.380 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-25T00:16:24.405 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:24 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:24.074+0000 7fbcb1a3e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:16:24.405 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:24 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:24.165+0000 7fbcb1a3e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:16:24.456 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:24 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:24.119+0000 7f7cf6a78200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:16:24.456 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:24 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:24.212+0000 7f7cf6a78200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:16:24.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.493+0000 7f8b522ba640 1 -- 172.21.15.107:0/4067305158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b4c104bb0 msgr2=0x7f8b4c106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:24.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.493+0000 7f8b522ba640 1 --2- 172.21.15.107:0/4067305158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b4c104bb0 0x7f8b4c106fa0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009900 tx=0x7f8b4002a410 comp rx=0 tx=0).stop 2024-04-25T00:16:24.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.493+0000 7f8b522ba640 1 -- 172.21.15.107:0/4067305158 shutdown_connections 2024-04-25T00:16:24.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.493+0000 7f8b522ba640 1 --2- 172.21.15.107:0/4067305158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b4c1074e0 0x7f8b4c1098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.493+0000 7f8b522ba640 1 --2- 172.21.15.107:0/4067305158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b4c104bb0 0x7f8b4c106fa0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.493+0000 7f8b522ba640 1 --2- 172.21.15.107:0/4067305158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b4c102280 0x7f8b4c104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.493+0000 7f8b522ba640 1 -- 172.21.15.107:0/4067305158 >> 172.21.15.107:0/4067305158 conn(0x7f8b4c0fbc10 msgr2=0x7f8b4c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:24.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.493+0000 7f8b522ba640 1 -- 172.21.15.107:0/4067305158 shutdown_connections 2024-04-25T00:16:24.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.493+0000 7f8b522ba640 1 -- 172.21.15.107:0/4067305158 wait complete. 2024-04-25T00:16:24.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.493+0000 7f8b522ba640 1 Processor -- start 2024-04-25T00:16:24.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.493+0000 7f8b522ba640 1 -- start start 2024-04-25T00:16:24.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.493+0000 7f8b522ba640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b4c102280 0x7f8b4c197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:24.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.494+0000 7f8b522ba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b4c104bb0 0x7f8b4c197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:24.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.494+0000 7f8b522ba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b4c1074e0 0x7f8b4c19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:24.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.494+0000 7f8b522ba640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c10d3f0 con 0x7f8b4c1074e0 2024-04-25T00:16:24.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.494+0000 7f8b522ba640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c10d270 con 0x7f8b4c102280 2024-04-25T00:16:24.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.494+0000 7f8b522ba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c10d570 con 0x7f8b4c104bb0 2024-04-25T00:16:24.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.494+0000 7f8b50857640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b4c1074e0 0x7f8b4c19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:24.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.494+0000 7f8b4bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b4c102280 0x7f8b4c197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:24.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.494+0000 7f8b50857640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b4c1074e0 0x7f8b4c19eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59202/0 (socket says 172.21.15.107:59202) 2024-04-25T00:16:24.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.494+0000 7f8b50857640 1 -- 172.21.15.107:0/615093808 learned_addr learned my addr 172.21.15.107:0/615093808 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:24.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.494+0000 7f8b4b7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b4c104bb0 0x7f8b4c197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:24.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.494+0000 7f8b4bfff640 1 -- 172.21.15.107:0/615093808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b4c104bb0 msgr2=0x7f8b4c197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:24.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.494+0000 7f8b4bfff640 1 --2- 172.21.15.107:0/615093808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b4c104bb0 0x7f8b4c197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.494+0000 7f8b4bfff640 1 -- 172.21.15.107:0/615093808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b4c1074e0 msgr2=0x7f8b4c19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:24.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.494+0000 7f8b4bfff640 1 --2- 172.21.15.107:0/615093808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b4c1074e0 0x7f8b4c19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.494+0000 7f8b4bfff640 1 -- 172.21.15.107:0/615093808 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4c19f660 con 0x7f8b4c102280 2024-04-25T00:16:24.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.494+0000 7f8b4b7fe640 1 --2- 172.21.15.107:0/615093808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b4c104bb0 0x7f8b4c197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:24.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.494+0000 7f8b50857640 1 --2- 172.21.15.107:0/615093808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b4c1074e0 0x7f8b4c19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:24.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.494+0000 7f8b4bfff640 1 --2- 172.21.15.107:0/615093808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b4c102280 0x7f8b4c197a40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00e9f0 tx=0x7f8b3c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:24.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.494+0000 7f8b497fa640 1 -- 172.21.15.107:0/615093808 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c00cd80 con 0x7f8b4c102280 2024-04-25T00:16:24.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.494+0000 7f8b522ba640 1 -- 172.21.15.107:0/615093808 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b4c19f950 con 0x7f8b4c102280 2024-04-25T00:16:24.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.494+0000 7f8b497fa640 1 -- 172.21.15.107:0/615093808 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b3c0035f0 con 0x7f8b4c102280 2024-04-25T00:16:24.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.494+0000 7f8b497fa640 1 -- 172.21.15.107:0/615093808 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c010670 con 0x7f8b4c102280 2024-04-25T00:16:24.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.494+0000 7f8b522ba640 1 -- 172.21.15.107:0/615093808 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b4c19fe70 con 0x7f8b4c102280 2024-04-25T00:16:24.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.495+0000 7f8b522ba640 1 -- 172.21.15.107:0/615093808 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b4c04f500 con 0x7f8b4c102280 2024-04-25T00:16:24.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.496+0000 7f8b497fa640 1 -- 172.21.15.107:0/615093808 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7f8b3c0037a0 con 0x7f8b4c102280 2024-04-25T00:16:24.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.497+0000 7f8b497fa640 1 --2- 172.21.15.107:0/615093808 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8b340a2c80 0x7f8b340a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:24.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.497+0000 7f8b4b7fe640 1 -- 172.21.15.107:0/615093808 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8b340a2c80 msgr2=0x7f8b340a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1644942185 2024-04-25T00:16:24.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.497+0000 7f8b497fa640 1 -- 172.21.15.107:0/615093808 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7f8b3c0cf740 con 0x7f8b4c102280 2024-04-25T00:16:24.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.497+0000 7f8b4b7fe640 1 --2- 172.21.15.107:0/615093808 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8b340a2c80 0x7f8b340a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:16:24.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.501+0000 7f8b497fa640 1 -- 172.21.15.107:0/615093808 <== mon.2 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8b3c09c2b0 con 0x7f8b4c102280 2024-04-25T00:16:24.684 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:24 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:24.403+0000 7fbcb1a3e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:16:24.685 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:24 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:24.538+0000 7fbcb1a3e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:16:24.685 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:24 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:24.633+0000 7fbcb1a3e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:16:24.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.698+0000 7f8b4b7fe640 1 -- 172.21.15.107:0/615093808 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8b340a2c80 msgr2=0x7f8b340a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1644942185 2024-04-25T00:16:24.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.698+0000 7f8b4b7fe640 1 --2- 172.21.15.107:0/615093808 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8b340a2c80 0x7f8b340a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T00:16:24.703 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:24 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:24.334+0000 7f75486bf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:16:24.703 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:24 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:24.446+0000 7f75486bf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:16:24.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.722+0000 7f8b522ba640 1 -- 172.21.15.107:0/615093808 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f8b4c0fd700 con 0x7f8b4c102280 2024-04-25T00:16:24.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.777+0000 7f8b497fa640 1 -- 172.21.15.107:0/615093808 <== mon.2 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v24) ==== 118+0+0 (secure 0 0 0) 0x7f8b3c09bbb0 con 0x7f8b4c102280 2024-04-25T00:16:24.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.780+0000 7f8b522ba640 1 -- 172.21.15.107:0/615093808 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8b340a2c80 msgr2=0x7f8b340a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:16:24.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.780+0000 7f8b522ba640 1 --2- 172.21.15.107:0/615093808 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8b340a2c80 0x7f8b340a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.780+0000 7f8b522ba640 1 -- 172.21.15.107:0/615093808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b4c102280 msgr2=0x7f8b4c197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:24.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.780+0000 7f8b522ba640 1 --2- 172.21.15.107:0/615093808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b4c102280 0x7f8b4c197a40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00e9f0 tx=0x7f8b3c00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.780+0000 7f8b522ba640 1 -- 172.21.15.107:0/615093808 shutdown_connections 2024-04-25T00:16:24.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.780+0000 7f8b522ba640 1 --2- 172.21.15.107:0/615093808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b4c1074e0 0x7f8b4c19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.780+0000 7f8b522ba640 1 --2- 172.21.15.107:0/615093808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b4c104bb0 0x7f8b4c197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.780+0000 7f8b522ba640 1 --2- 172.21.15.107:0/615093808 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f8b340a2c80 0x7f8b340a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.780+0000 7f8b522ba640 1 --2- 172.21.15.107:0/615093808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b4c102280 0x7f8b4c197a40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.780+0000 7f8b522ba640 1 -- 172.21.15.107:0/615093808 >> 172.21.15.107:0/615093808 conn(0x7f8b4c0fbc10 msgr2=0x7f8b4c102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:24.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.780+0000 7f8b522ba640 1 -- 172.21.15.107:0/615093808 shutdown_connections 2024-04-25T00:16:24.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.780+0000 7f8b522ba640 1 -- 172.21.15.107:0/615093808 wait complete. 2024-04-25T00:16:24.786 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:24 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:24.455+0000 7f7cf6a78200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:16:24.786 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:24 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:24.591+0000 7f7cf6a78200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:16:24.786 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:24 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:24.686+0000 7f7cf6a78200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:16:24.792 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-25T00:16:24.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.895+0000 7f5cf7dd5640 1 -- 172.21.15.107:0/1129429146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cf01018b0 msgr2=0x7f5cf0101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:24.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.895+0000 7f5cf7dd5640 1 --2- 172.21.15.107:0/1129429146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cf01018b0 0x7f5cf0101d30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5cec009900 tx=0x7f5cec02a410 comp rx=0 tx=0).stop 2024-04-25T00:16:24.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.895+0000 7f5cf7dd5640 1 -- 172.21.15.107:0/1129429146 shutdown_connections 2024-04-25T00:16:24.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.895+0000 7f5cf7dd5640 1 --2- 172.21.15.107:0/1129429146 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cf0102270 0x7f5cf0108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.895+0000 7f5cf7dd5640 1 --2- 172.21.15.107:0/1129429146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cf01018b0 0x7f5cf0101d30 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.895+0000 7f5cf7dd5640 1 --2- 172.21.15.107:0/1129429146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf01006c0 0x7f5cf0100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.895+0000 7f5cf7dd5640 1 -- 172.21.15.107:0/1129429146 >> 172.21.15.107:0/1129429146 conn(0x7f5cf00fbc50 msgr2=0x7f5cf00fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:24.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.895+0000 7f5cf7dd5640 1 -- 172.21.15.107:0/1129429146 shutdown_connections 2024-04-25T00:16:24.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.895+0000 7f5cf7dd5640 1 -- 172.21.15.107:0/1129429146 wait complete. 2024-04-25T00:16:24.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.896+0000 7f5cf7dd5640 1 Processor -- start 2024-04-25T00:16:24.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.896+0000 7f5cf7dd5640 1 -- start start 2024-04-25T00:16:24.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.896+0000 7f5cf7dd5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cf01006c0 0x7f5cf0197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:24.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.896+0000 7f5cf7dd5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cf01018b0 0x7f5cf0197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:24.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.896+0000 7f5cf7dd5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf0102270 0x7f5cf019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:24.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.896+0000 7f5cf7dd5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf010c340 con 0x7f5cf01006c0 2024-04-25T00:16:24.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.896+0000 7f5cf7dd5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf010c1c0 con 0x7f5cf01018b0 2024-04-25T00:16:24.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.896+0000 7f5cf7dd5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf010c4c0 con 0x7f5cf0102270 2024-04-25T00:16:24.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.896+0000 7f5cf5b71640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cf01006c0 0x7f5cf0197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:24.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.896+0000 7f5cf6372640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf0102270 0x7f5cf019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:24.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.896+0000 7f5cf6372640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf0102270 0x7f5cf019f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:50428/0 (socket says 172.21.15.107:50428) 2024-04-25T00:16:24.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.896+0000 7f5cf6372640 1 -- 172.21.15.107:0/389374685 learned_addr learned my addr 172.21.15.107:0/389374685 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:24.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.896+0000 7f5cf5370640 1 --2- 172.21.15.107:0/389374685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cf01018b0 0x7f5cf0197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:24.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.896+0000 7f5cf5b71640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cf01006c0 0x7f5cf0197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59232/0 (socket says 172.21.15.107:59232) 2024-04-25T00:16:24.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.896+0000 7f5cf6372640 1 -- 172.21.15.107:0/389374685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cf01018b0 msgr2=0x7f5cf0197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:24.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.896+0000 7f5cf6372640 1 --2- 172.21.15.107:0/389374685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5cf01018b0 0x7f5cf0197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.896+0000 7f5cf6372640 1 -- 172.21.15.107:0/389374685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cf01006c0 msgr2=0x7f5cf0197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:24.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.896+0000 7f5cf6372640 1 --2- 172.21.15.107:0/389374685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cf01006c0 0x7f5cf0197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.896+0000 7f5cf6372640 1 -- 172.21.15.107:0/389374685 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf019f670 con 0x7f5cf0102270 2024-04-25T00:16:24.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.896+0000 7f5cf5b71640 1 --2- 172.21.15.107:0/389374685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cf01006c0 0x7f5cf0197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:24.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.897+0000 7f5cf6372640 1 --2- 172.21.15.107:0/389374685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf0102270 0x7f5cf019f000 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800eb10 tx=0x7f5ce800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:24.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.897+0000 7f5ce6ffd640 1 -- 172.21.15.107:0/389374685 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8004b20 con 0x7f5cf0102270 2024-04-25T00:16:24.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.897+0000 7f5cf7dd5640 1 -- 172.21.15.107:0/389374685 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf019f960 con 0x7f5cf0102270 2024-04-25T00:16:24.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.897+0000 7f5ce6ffd640 1 -- 172.21.15.107:0/389374685 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ce8004cc0 con 0x7f5cf0102270 2024-04-25T00:16:24.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.897+0000 7f5ce6ffd640 1 -- 172.21.15.107:0/389374685 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8010700 con 0x7f5cf0102270 2024-04-25T00:16:24.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.897+0000 7f5cf7dd5640 1 -- 172.21.15.107:0/389374685 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf019fe80 con 0x7f5cf0102270 2024-04-25T00:16:24.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.898+0000 7f5cf7dd5640 1 -- 172.21.15.107:0/389374685 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf004f500 con 0x7f5cf0102270 2024-04-25T00:16:24.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.899+0000 7f5ce6ffd640 1 -- 172.21.15.107:0/389374685 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 19) ==== 137851+0+0 (secure 0 0 0) 0x7f5ce8005000 con 0x7f5cf0102270 2024-04-25T00:16:24.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.899+0000 7f5ce6ffd640 1 --2- 172.21.15.107:0/389374685 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5cdc0a2c80 0x7f5cdc0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:24.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.899+0000 7f5cf5b71640 1 -- 172.21.15.107:0/389374685 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5cdc0a2c80 msgr2=0x7f5cdc0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1644942185 2024-04-25T00:16:24.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.899+0000 7f5ce6ffd640 1 -- 172.21.15.107:0/389374685 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9137+0+0 (secure 0 0 0) 0x7f5ce80cfaf0 con 0x7f5cf0102270 2024-04-25T00:16:24.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.899+0000 7f5cf5b71640 1 --2- 172.21.15.107:0/389374685 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5cdc0a2c80 0x7f5cdc0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:16:24.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:24.904+0000 7f5ce6ffd640 1 -- 172.21.15.107:0/389374685 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5ce809c6c0 con 0x7f5cf0102270 2024-04-25T00:16:24.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:24 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/615093808' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T00:16:24.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:24 smithi107 ceph-mon[29153]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T00:16:24.984 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:24 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:24.700+0000 7f75486bf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:16:24.984 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:24 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:24.840+0000 7f75486bf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:16:24.985 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:24 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:24.945+0000 7f75486bf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:16:25.094 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:24 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/615093808' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T00:16:25.095 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:24 smithi195 ceph-mon[32833]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T00:16:25.095 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:24 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:24.804+0000 7f7cf6a78200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:16:25.095 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:24 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:24.897+0000 7f7cf6a78200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:16:25.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:25.100+0000 7f5cf5b71640 1 -- 172.21.15.107:0/389374685 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5cdc0a2c80 msgr2=0x7f5cdc0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1644942185 2024-04-25T00:16:25.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:25.100+0000 7f5cf5b71640 1 --2- 172.21.15.107:0/389374685 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5cdc0a2c80 0x7f5cdc0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T00:16:25.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:25.111+0000 7f5cf7dd5640 1 -- 172.21.15.107:0/389374685 --> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f5cf00fdd30 con 0x7f5cdc0a2c80 2024-04-25T00:16:25.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:24 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/615093808' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T00:16:25.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:24 smithi170 ceph-mon[32807]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T00:16:25.185 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:24 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:24.749+0000 7fbcb1a3e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:16:25.185 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:24 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:24.843+0000 7fbcb1a3e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:16:25.484 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:25 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:25.064+0000 7f75486bf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:16:25.484 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:25 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:25.159+0000 7f75486bf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:16:25.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:25.501+0000 7f5cf5b71640 1 -- 172.21.15.107:0/389374685 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5cdc0a2c80 msgr2=0x7f5cdc0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/1644942185 2024-04-25T00:16:25.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:25.501+0000 7f5cf5b71640 1 --2- 172.21.15.107:0/389374685 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5cdc0a2c80 0x7f5cdc0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T00:16:25.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:25 smithi170 ceph-mon[32807]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T00:16:25.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:25 smithi170 ceph-mon[32807]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:16:25.935 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:25 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:25.654+0000 7fbcb1a3e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:16:25.935 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:16:25 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y[34193]: 2024-04-25T00:16:25.872+0000 7fbcb1a3e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:16:26.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:25 smithi195 ceph-mon[32833]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T00:16:26.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:25 smithi195 ceph-mon[32833]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:16:26.090 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:25 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:25.715+0000 7f7cf6a78200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:16:26.090 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:16:25 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z[34702]: 2024-04-25T00:16:25.933+0000 7f7cf6a78200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:16:26.227 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:25 smithi107 ceph-mon[29153]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T00:16:26.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:25 smithi107 ceph-mon[29153]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:16:26.228 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:26.007+0000 7f75486bf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:16:26.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:26.239+0000 7f5ce6ffd640 1 -- 172.21.15.107:0/389374685 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mgrmap(e 20) ==== 137518+0+0 (secure 0 0 0) 0x7f5ce8098630 con 0x7f5cf0102270 2024-04-25T00:16:26.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:26.239+0000 7f5ce6ffd640 1 -- 172.21.15.107:0/389374685 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5cdc0a2c80 msgr2=0x7f5cdc0a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:16:26.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:26.239+0000 7f5ce6ffd640 1 --2- 172.21.15.107:0/389374685 >> [v2:172.21.15.107:6800/1644942185,v1:172.21.15.107:6801/1644942185] conn(0x7f5cdc0a2c80 0x7f5cdc0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:26.485 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:26.226+0000 7f75486bf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:16:26.485 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:26.464+0000 7f74a807d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:16:26.485 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:26.464+0000 7f74a807d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:16:26.485 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:26.464+0000 7f74a807d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:16:26.485 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:26.464+0000 7f74a807d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:16:26.485 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:26.464+0000 7f74a807d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:16:26.485 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:26.465+0000 7f74ab083700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:16:26.485 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:26.465+0000 7f74ab083700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:16:26.485 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:26.465+0000 7f74ab083700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:16:26.485 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:26.465+0000 7f74ab083700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:16:26.485 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:26.465+0000 7f74ab083700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:16:27.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: Standby manager daemon y restarted 2024-04-25T00:16:27.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: Standby manager daemon y started 2024-04-25T00:16:27.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.? 172.21.15.170:0/3638406770' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T00:16:27.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.? 172.21.15.170:0/3638406770' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:16:27.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.? 172.21.15.170:0/3638406770' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T00:16:27.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.? 172.21.15.170:0/3638406770' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:16:27.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: Standby manager daemon z restarted 2024-04-25T00:16:27.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: Standby manager daemon z started 2024-04-25T00:16:27.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.? 172.21.15.195:0/625366878' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T00:16:27.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.? 172.21.15.195:0/625366878' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:16:27.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.? 172.21.15.195:0/625366878' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T00:16:27.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.? 172.21.15.195:0/625366878' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:16:27.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: Active manager daemon x restarted 2024-04-25T00:16:27.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: Activating manager daemon x 2024-04-25T00:16:27.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T00:16:27.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: mgrmap e20: x(active, starting, since 0.0111528s), standbys: z, y 2024-04-25T00:16:27.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:16:27.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:16:27.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:16:27.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T00:16:27.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T00:16:27.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T00:16:27.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:16:27.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T00:16:27.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T00:16:27.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:16:27.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:16:27.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:16:27.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:16:27.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:16:27.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:16:27.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T00:16:27.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T00:16:27.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T00:16:27.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:16:27.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:16:27.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:16:27.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:16:27.093 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:16:27.093 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:16:27.093 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:16:27.093 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:16:27.093 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:16:27.093 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:16:27.093 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:16:27.093 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:16:27.093 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T00:16:27.093 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T00:16:27.093 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T00:16:27.093 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: Manager daemon x is now available 2024-04-25T00:16:27.093 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:27.093 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T00:16:27.093 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:26 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T00:16:27.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: Standby manager daemon y restarted 2024-04-25T00:16:27.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: Standby manager daemon y started 2024-04-25T00:16:27.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.? 172.21.15.170:0/3638406770' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T00:16:27.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.? 172.21.15.170:0/3638406770' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:16:27.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.? 172.21.15.170:0/3638406770' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T00:16:27.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.? 172.21.15.170:0/3638406770' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:16:27.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: Standby manager daemon z restarted 2024-04-25T00:16:27.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: Standby manager daemon z started 2024-04-25T00:16:27.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.? 172.21.15.195:0/625366878' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T00:16:27.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.? 172.21.15.195:0/625366878' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:16:27.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.? 172.21.15.195:0/625366878' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T00:16:27.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.? 172.21.15.195:0/625366878' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:16:27.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: Active manager daemon x restarted 2024-04-25T00:16:27.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: Activating manager daemon x 2024-04-25T00:16:27.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T00:16:27.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: mgrmap e20: x(active, starting, since 0.0111528s), standbys: z, y 2024-04-25T00:16:27.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:16:27.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:16:27.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:16:27.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T00:16:27.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T00:16:27.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T00:16:27.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:16:27.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T00:16:27.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T00:16:27.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:16:27.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:16:27.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:16:27.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:16:27.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:16:27.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:16:27.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T00:16:27.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T00:16:27.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T00:16:27.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:16:27.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:16:27.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:16:27.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:16:27.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:16:27.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:16:27.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:16:27.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:16:27.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:16:27.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:16:27.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:16:27.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:16:27.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T00:16:27.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T00:16:27.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T00:16:27.189 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: Manager daemon x is now available 2024-04-25T00:16:27.189 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:27.189 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T00:16:27.189 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:26 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T00:16:27.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: Standby manager daemon y restarted 2024-04-25T00:16:27.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: Standby manager daemon y started 2024-04-25T00:16:27.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.? 172.21.15.170:0/3638406770' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T00:16:27.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.? 172.21.15.170:0/3638406770' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:16:27.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.? 172.21.15.170:0/3638406770' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T00:16:27.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.? 172.21.15.170:0/3638406770' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:16:27.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: Standby manager daemon z restarted 2024-04-25T00:16:27.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: Standby manager daemon z started 2024-04-25T00:16:27.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.? 172.21.15.195:0/625366878' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T00:16:27.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.? 172.21.15.195:0/625366878' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:16:27.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.? 172.21.15.195:0/625366878' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T00:16:27.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.? 172.21.15.195:0/625366878' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:16:27.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: Active manager daemon x restarted 2024-04-25T00:16:27.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: Activating manager daemon x 2024-04-25T00:16:27.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T00:16:27.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: mgrmap e20: x(active, starting, since 0.0111528s), standbys: z, y 2024-04-25T00:16:27.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:16:27.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:16:27.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:16:27.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T00:16:27.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T00:16:27.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T00:16:27.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:16:27.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T00:16:27.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T00:16:27.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:16:27.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:16:27.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:16:27.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:16:27.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:16:27.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:16:27.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T00:16:27.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T00:16:27.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T00:16:27.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:16:27.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:16:27.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:16:27.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:16:27.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:16:27.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:16:27.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:16:27.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:16:27.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:16:27.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:16:27.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:16:27.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:16:27.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T00:16:27.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T00:16:27.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T00:16:27.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: Manager daemon x is now available 2024-04-25T00:16:27.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:27.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T00:16:27.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:26 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T00:16:27.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.245+0000 7f5ce6ffd640 1 -- 172.21.15.107:0/389374685 <== mon.1 v2:172.21.15.195:3300/0 8 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f5ce8045b10 con 0x7f5cf0102270 2024-04-25T00:16:27.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.245+0000 7f5ce6ffd640 1 --2- 172.21.15.107:0/389374685 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f5cdc051920 0x7f5cdc051d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:27.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.245+0000 7f5ce6ffd640 1 -- 172.21.15.107:0/389374685 --> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f5ce800e1a0 con 0x7f5cdc051920 2024-04-25T00:16:27.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.246+0000 7f5cf5b71640 1 --2- 172.21.15.107:0/389374685 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f5cdc051920 0x7f5cdc051d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:27.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.249+0000 7f5cf5b71640 1 --2- 172.21.15.107:0/389374685 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f5cdc051920 0x7f5cdc051d00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0101710 tx=0x7f5ce0005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:27.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.266+0000 7f5ce6ffd640 1 -- 172.21.15.107:0/389374685 <== mgr.15456 v2:172.21.15.107:6800/1021104172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f5ce800e1a0 con 0x7f5cdc051920 2024-04-25T00:16:27.268 INFO:teuthology.orchestra.run.smithi107.stdout:{} 2024-04-25T00:16:27.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.268+0000 7f5cf7dd5640 1 -- 172.21.15.107:0/389374685 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f5cdc051920 msgr2=0x7f5cdc051d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:27.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.268+0000 7f5cf7dd5640 1 --2- 172.21.15.107:0/389374685 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f5cdc051920 0x7f5cdc051d00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0101710 tx=0x7f5ce0005f90 comp rx=0 tx=0).stop 2024-04-25T00:16:27.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.268+0000 7f5cf7dd5640 1 -- 172.21.15.107:0/389374685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf0102270 msgr2=0x7f5cf019f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:27.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.268+0000 7f5cf7dd5640 1 --2- 172.21.15.107:0/389374685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf0102270 0x7f5cf019f000 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800eb10 tx=0x7f5ce800c2d0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.268+0000 7f5cf6372640 1 -- 172.21.15.107:0/389374685 reap_dead start 2024-04-25T00:16:27.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.268+0000 7f5cf7dd5640 1 -- 172.21.15.107:0/389374685 shutdown_connections 2024-04-25T00:16:27.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.268+0000 7f5cf7dd5640 1 -- 172.21.15.107:0/389374685 >> 172.21.15.107:0/389374685 conn(0x7f5cf00fbc50 msgr2=0x7f5cf00fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:27.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.268+0000 7f5cf7dd5640 1 -- 172.21.15.107:0/389374685 shutdown_connections 2024-04-25T00:16:27.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.268+0000 7f5cf7dd5640 1 -- 172.21.15.107:0/389374685 wait complete. 2024-04-25T00:16:27.280 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T00:16:27.292 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-25T00:16:27.343 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:16:27.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.444+0000 7f4bef458640 1 -- 172.21.15.107:0/2003376498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be81039c0 msgr2=0x7f4be8103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:27.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.444+0000 7f4bef458640 1 --2- 172.21.15.107:0/2003376498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be81039c0 0x7f4be8103de0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009900 tx=0x7f4be002a410 comp rx=0 tx=0).stop 2024-04-25T00:16:27.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.444+0000 7f4bef458640 1 -- 172.21.15.107:0/2003376498 shutdown_connections 2024-04-25T00:16:27.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.444+0000 7f4bef458640 1 --2- 172.21.15.107:0/2003376498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4be8105570 0x7f4be810be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.444+0000 7f4bef458640 1 --2- 172.21.15.107:0/2003376498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4be8104bb0 0x7f4be8105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.444+0000 7f4bef458640 1 --2- 172.21.15.107:0/2003376498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be81039c0 0x7f4be8103de0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.444+0000 7f4bef458640 1 -- 172.21.15.107:0/2003376498 >> 172.21.15.107:0/2003376498 conn(0x7f4be80fef30 msgr2=0x7f4be8101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:27.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.444+0000 7f4bef458640 1 -- 172.21.15.107:0/2003376498 shutdown_connections 2024-04-25T00:16:27.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.444+0000 7f4bef458640 1 -- 172.21.15.107:0/2003376498 wait complete. 2024-04-25T00:16:27.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.444+0000 7f4bef458640 1 Processor -- start 2024-04-25T00:16:27.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.444+0000 7f4bef458640 1 -- start start 2024-04-25T00:16:27.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.445+0000 7f4bef458640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be81039c0 0x7f4be8199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:27.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.445+0000 7f4bef458640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4be8104bb0 0x7f4be819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:27.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.445+0000 7f4bef458640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4be8105570 0x7f4be81a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:27.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.445+0000 7f4bef458640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4be810f610 con 0x7f4be81039c0 2024-04-25T00:16:27.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.445+0000 7f4bef458640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4be810f490 con 0x7f4be8105570 2024-04-25T00:16:27.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.445+0000 7f4bef458640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4be810f790 con 0x7f4be8104bb0 2024-04-25T00:16:27.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.445+0000 7f4bed1f4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be81039c0 0x7f4be8199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:27.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.445+0000 7f4bed1f4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be81039c0 0x7f4be8199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59370/0 (socket says 172.21.15.107:59370) 2024-04-25T00:16:27.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.445+0000 7f4bed1f4640 1 -- 172.21.15.107:0/3684256037 learned_addr learned my addr 172.21.15.107:0/3684256037 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:27.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.445+0000 7f4bec9f3640 1 --2- 172.21.15.107:0/3684256037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4be8104bb0 0x7f4be819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:27.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.445+0000 7f4bed9f5640 1 --2- 172.21.15.107:0/3684256037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4be8105570 0x7f4be81a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:27.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.445+0000 7f4bed1f4640 1 -- 172.21.15.107:0/3684256037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4be8104bb0 msgr2=0x7f4be819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:27.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.445+0000 7f4bed1f4640 1 --2- 172.21.15.107:0/3684256037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4be8104bb0 0x7f4be819a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.445+0000 7f4bed1f4640 1 -- 172.21.15.107:0/3684256037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4be8105570 msgr2=0x7f4be81a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:27.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.445+0000 7f4bed1f4640 1 --2- 172.21.15.107:0/3684256037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4be8105570 0x7f4be81a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.445+0000 7f4bed1f4640 1 -- 172.21.15.107:0/3684256037 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be81a1940 con 0x7f4be81039c0 2024-04-25T00:16:27.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.445+0000 7f4bed1f4640 1 --2- 172.21.15.107:0/3684256037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be81039c0 0x7f4be8199d30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4be0031610 tx=0x7f4be0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:27.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.445+0000 7f4be67fc640 1 -- 172.21.15.107:0/3684256037 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be003f030 con 0x7f4be81039c0 2024-04-25T00:16:27.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.445+0000 7f4bef458640 1 -- 172.21.15.107:0/3684256037 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be81a1bd0 con 0x7f4be81039c0 2024-04-25T00:16:27.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.445+0000 7f4be67fc640 1 -- 172.21.15.107:0/3684256037 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4be0003ad0 con 0x7f4be81039c0 2024-04-25T00:16:27.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.445+0000 7f4bef458640 1 -- 172.21.15.107:0/3684256037 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be81a2110 con 0x7f4be81039c0 2024-04-25T00:16:27.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.445+0000 7f4be67fc640 1 -- 172.21.15.107:0/3684256037 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be0035070 con 0x7f4be81039c0 2024-04-25T00:16:27.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.447+0000 7f4bef458640 1 -- 172.21.15.107:0/3684256037 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4be810ad30 con 0x7f4be81039c0 2024-04-25T00:16:27.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.448+0000 7f4be67fc640 1 -- 172.21.15.107:0/3684256037 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f4be0042030 con 0x7f4be81039c0 2024-04-25T00:16:27.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.448+0000 7f4be67fc640 1 --2- 172.21.15.107:0/3684256037 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4bd00a2990 0x7f4bd00a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:27.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.448+0000 7f4bec9f3640 1 --2- 172.21.15.107:0/3684256037 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4bd00a2990 0x7f4bd00a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:27.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.448+0000 7f4be67fc640 1 -- 172.21.15.107:0/3684256037 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f4be00ef970 con 0x7f4be81039c0 2024-04-25T00:16:27.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.449+0000 7f4bec9f3640 1 --2- 172.21.15.107:0/3684256037 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4bd00a2990 0x7f4bd00a4e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009bd0 tx=0x7f4bdc005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:27.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.453+0000 7f4be67fc640 1 -- 172.21.15.107:0/3684256037 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4be00bc0e0 con 0x7f4be81039c0 2024-04-25T00:16:27.585 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:27 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:27.255+0000 7f74a5077700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:16:27.585 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:27 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:27.256+0000 7f74a5077700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:16:27.585 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:27 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:27.256+0000 7f74a5077700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:16:27.585 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:27 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:27.256+0000 7f74a5077700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:16:27.586 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:16:27 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:16:27.256+0000 7f74a5077700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:16:27.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.650+0000 7f4bef458640 1 -- 172.21.15.107:0/3684256037 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4be8062340 con 0x7f4be81039c0 2024-04-25T00:16:27.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.652+0000 7f4be67fc640 1 -- 172.21.15.107:0/3684256037 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+25070 (secure 0 0 0) 0x7f4be00c03e0 con 0x7f4be81039c0 2024-04-25T00:16:27.654 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:16:27.654 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":97,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:16:26.229487+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:59.672801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:00.957782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:14:05.065846+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8},{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/151603643":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/1884923543":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/111369358":"2024-04-26T00:16:26.229442+0000","172.21.15.107:6800/1644942185":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3936375305":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/1644942185":"2024-04-26T00:16:26.229442+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:16:27.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.654+0000 7f4bef458640 1 -- 172.21.15.107:0/3684256037 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4bd00a2990 msgr2=0x7f4bd00a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:27.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.654+0000 7f4bef458640 1 --2- 172.21.15.107:0/3684256037 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4bd00a2990 0x7f4bd00a4e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009bd0 tx=0x7f4bdc005f90 comp rx=0 tx=0).stop 2024-04-25T00:16:27.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.654+0000 7f4bef458640 1 -- 172.21.15.107:0/3684256037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be81039c0 msgr2=0x7f4be8199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:27.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.654+0000 7f4bef458640 1 --2- 172.21.15.107:0/3684256037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be81039c0 0x7f4be8199d30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4be0031610 tx=0x7f4be0004270 comp rx=0 tx=0).stop 2024-04-25T00:16:27.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.654+0000 7f4bef458640 1 -- 172.21.15.107:0/3684256037 shutdown_connections 2024-04-25T00:16:27.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.654+0000 7f4bef458640 1 --2- 172.21.15.107:0/3684256037 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4bd00a2990 0x7f4bd00a4e50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.654+0000 7f4bef458640 1 --2- 172.21.15.107:0/3684256037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4be8105570 0x7f4be81a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.654+0000 7f4bef458640 1 --2- 172.21.15.107:0/3684256037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4be8104bb0 0x7f4be819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.654+0000 7f4bef458640 1 --2- 172.21.15.107:0/3684256037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be81039c0 0x7f4be8199d30 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.654+0000 7f4bef458640 1 -- 172.21.15.107:0/3684256037 >> 172.21.15.107:0/3684256037 conn(0x7f4be80fef30 msgr2=0x7f4be8107f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:27.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.654+0000 7f4bef458640 1 -- 172.21.15.107:0/3684256037 shutdown_connections 2024-04-25T00:16:27.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.654+0000 7f4bef458640 1 -- 172.21.15.107:0/3684256037 wait complete. 2024-04-25T00:16:27.667 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:09:25.160329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:13:59.672801+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:14:00.957782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:14:05.065846+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:16:27.667 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:16:27.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.770+0000 7f426e1ee640 1 -- 172.21.15.107:0/1456094819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4268100680 msgr2=0x7f4268100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:27.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.770+0000 7f426e1ee640 1 --2- 172.21.15.107:0/1456094819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4268100680 0x7f4268100aa0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f4258009900 tx=0x7f425802a410 comp rx=0 tx=0).stop 2024-04-25T00:16:27.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.770+0000 7f426e1ee640 1 -- 172.21.15.107:0/1456094819 shutdown_connections 2024-04-25T00:16:27.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.770+0000 7f426e1ee640 1 --2- 172.21.15.107:0/1456094819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4268102230 0x7f4268108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.770+0000 7f426e1ee640 1 --2- 172.21.15.107:0/1456094819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4268101870 0x7f4268101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.770+0000 7f426e1ee640 1 --2- 172.21.15.107:0/1456094819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4268100680 0x7f4268100aa0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.770+0000 7f426e1ee640 1 -- 172.21.15.107:0/1456094819 >> 172.21.15.107:0/1456094819 conn(0x7f42680fbbf0 msgr2=0x7f42680fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:27.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.771+0000 7f426e1ee640 1 -- 172.21.15.107:0/1456094819 shutdown_connections 2024-04-25T00:16:27.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.771+0000 7f426e1ee640 1 -- 172.21.15.107:0/1456094819 wait complete. 2024-04-25T00:16:27.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.771+0000 7f426e1ee640 1 Processor -- start 2024-04-25T00:16:27.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.771+0000 7f426e1ee640 1 -- start start 2024-04-25T00:16:27.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.771+0000 7f426e1ee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4268100680 0x7f42681047b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:27.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.771+0000 7f426e1ee640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4268101870 0x7f4268102df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:27.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.771+0000 7f426e1ee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4268102230 0x7f4268103330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:27.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.771+0000 7f426e1ee640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f426810c2d0 con 0x7f4268102230 2024-04-25T00:16:27.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.771+0000 7f426e1ee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f426810c150 con 0x7f4268100680 2024-04-25T00:16:27.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.771+0000 7f426e1ee640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f426810c450 con 0x7f4268101870 2024-04-25T00:16:27.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.772+0000 7f4267fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4268102230 0x7f4268103330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:27.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.772+0000 7f4266ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4268101870 0x7f4268102df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:27.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.772+0000 7f4267fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4268102230 0x7f4268103330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59386/0 (socket says 172.21.15.107:59386) 2024-04-25T00:16:27.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.772+0000 7f42677fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4268100680 0x7f42681047b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:27.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.772+0000 7f4267fff640 1 -- 172.21.15.107:0/73324549 learned_addr learned my addr 172.21.15.107:0/73324549 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:27.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.772+0000 7f4267fff640 1 -- 172.21.15.107:0/73324549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4268101870 msgr2=0x7f4268102df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:27.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.772+0000 7f4267fff640 1 --2- 172.21.15.107:0/73324549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4268101870 0x7f4268102df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.772+0000 7f4267fff640 1 -- 172.21.15.107:0/73324549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4268100680 msgr2=0x7f42681047b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:27.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.772+0000 7f4267fff640 1 --2- 172.21.15.107:0/73324549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4268100680 0x7f42681047b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.772+0000 7f4267fff640 1 -- 172.21.15.107:0/73324549 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4268103bc0 con 0x7f4268102230 2024-04-25T00:16:27.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.772+0000 7f4267fff640 1 --2- 172.21.15.107:0/73324549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4268102230 0x7f4268103330 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f426000cce0 tx=0x7f426000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:27.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.772+0000 7f4264ff9640 1 -- 172.21.15.107:0/73324549 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426000f450 con 0x7f4268102230 2024-04-25T00:16:27.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.772+0000 7f4264ff9640 1 -- 172.21.15.107:0/73324549 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f426000fad0 con 0x7f4268102230 2024-04-25T00:16:27.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.772+0000 7f426e1ee640 1 -- 172.21.15.107:0/73324549 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42681a1bc0 con 0x7f4268102230 2024-04-25T00:16:27.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.772+0000 7f4264ff9640 1 -- 172.21.15.107:0/73324549 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260018c10 con 0x7f4268102230 2024-04-25T00:16:27.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.772+0000 7f426e1ee640 1 -- 172.21.15.107:0/73324549 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42681a20c0 con 0x7f4268102230 2024-04-25T00:16:27.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.773+0000 7f426e1ee640 1 -- 172.21.15.107:0/73324549 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42681a22f0 con 0x7f4268102230 2024-04-25T00:16:27.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.774+0000 7f4264ff9640 1 -- 172.21.15.107:0/73324549 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f426000f5f0 con 0x7f4268102230 2024-04-25T00:16:27.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.775+0000 7f4264ff9640 1 --2- 172.21.15.107:0/73324549 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f42540a2990 0x7f42540a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:27.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.775+0000 7f4264ff9640 1 -- 172.21.15.107:0/73324549 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f42600cf760 con 0x7f4268102230 2024-04-25T00:16:27.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.775+0000 7f42677fe640 1 --2- 172.21.15.107:0/73324549 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f42540a2990 0x7f42540a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:27.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.775+0000 7f42677fe640 1 --2- 172.21.15.107:0/73324549 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f42540a2990 0x7f42540a4e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4258005ab0 tx=0x7f4258000bc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:27.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.779+0000 7f4264ff9640 1 -- 172.21.15.107:0/73324549 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4260014030 con 0x7f4268102230 2024-04-25T00:16:27.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.970+0000 7f426e1ee640 1 -- 172.21.15.107:0/73324549 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f42681a24c0 con 0x7f4268102230 2024-04-25T00:16:27.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.970+0000 7f4264ff9640 1 -- 172.21.15.107:0/73324549 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f426009c520 con 0x7f4268102230 2024-04-25T00:16:27.972 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-04-25T00:16:27.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.972+0000 7f426e1ee640 1 -- 172.21.15.107:0/73324549 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f42540a2990 msgr2=0x7f42540a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:27.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.972+0000 7f426e1ee640 1 --2- 172.21.15.107:0/73324549 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f42540a2990 0x7f42540a4e50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4258005ab0 tx=0x7f4258000bc0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.972+0000 7f426e1ee640 1 -- 172.21.15.107:0/73324549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4268102230 msgr2=0x7f4268103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:27.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.972+0000 7f426e1ee640 1 --2- 172.21.15.107:0/73324549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4268102230 0x7f4268103330 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f426000cce0 tx=0x7f426000a620 comp rx=0 tx=0).stop 2024-04-25T00:16:27.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.973+0000 7f426e1ee640 1 -- 172.21.15.107:0/73324549 shutdown_connections 2024-04-25T00:16:27.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.973+0000 7f426e1ee640 1 --2- 172.21.15.107:0/73324549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4268102230 0x7f4268103330 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.973+0000 7f426e1ee640 1 --2- 172.21.15.107:0/73324549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4268101870 0x7f4268102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.973+0000 7f426e1ee640 1 --2- 172.21.15.107:0/73324549 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f42540a2990 0x7f42540a4e50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.973+0000 7f426e1ee640 1 --2- 172.21.15.107:0/73324549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4268100680 0x7f42681047b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.973+0000 7f426e1ee640 1 -- 172.21.15.107:0/73324549 >> 172.21.15.107:0/73324549 conn(0x7f42680fbbf0 msgr2=0x7f42680688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:27.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.973+0000 7f426e1ee640 1 -- 172.21.15.107:0/73324549 shutdown_connections 2024-04-25T00:16:27.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:27.973+0000 7f426e1ee640 1 -- 172.21.15.107:0/73324549 wait complete. 2024-04-25T00:16:27.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:27 smithi107 ceph-mon[29153]: [25/Apr/2024:00:16:26] ENGINE Bus STARTING 2024-04-25T00:16:27.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:27 smithi107 ceph-mon[29153]: [25/Apr/2024:00:16:26] ENGINE Serving on http://172.21.15.107:8765 2024-04-25T00:16:27.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:27 smithi107 ceph-mon[29153]: [25/Apr/2024:00:16:26] ENGINE Serving on https://172.21.15.107:7150 2024-04-25T00:16:27.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:27 smithi107 ceph-mon[29153]: [25/Apr/2024:00:16:26] ENGINE Bus STARTED 2024-04-25T00:16:27.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:27 smithi107 ceph-mon[29153]: mgrmap e21: x(active, since 1.01839s), standbys: z, y 2024-04-25T00:16:27.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:27 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3684256037' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:27.986 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:16:28.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:27 smithi195 ceph-mon[32833]: [25/Apr/2024:00:16:26] ENGINE Bus STARTING 2024-04-25T00:16:28.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:27 smithi195 ceph-mon[32833]: [25/Apr/2024:00:16:26] ENGINE Serving on http://172.21.15.107:8765 2024-04-25T00:16:28.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:27 smithi195 ceph-mon[32833]: [25/Apr/2024:00:16:26] ENGINE Serving on https://172.21.15.107:7150 2024-04-25T00:16:28.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:27 smithi195 ceph-mon[32833]: [25/Apr/2024:00:16:26] ENGINE Bus STARTED 2024-04-25T00:16:28.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:27 smithi195 ceph-mon[32833]: mgrmap e21: x(active, since 1.01839s), standbys: z, y 2024-04-25T00:16:28.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:27 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3684256037' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.101+0000 7f1dc1148640 1 -- 172.21.15.107:0/3801882988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dbc107460 msgr2=0x7f1dbc1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.101+0000 7f1dc1148640 1 --2- 172.21.15.107:0/3801882988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dbc107460 0x7f1dbc1098b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009ca0 tx=0x7f1db402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.101+0000 7f1dc1148640 1 -- 172.21.15.107:0/3801882988 shutdown_connections 2024-04-25T00:16:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.101+0000 7f1dc1148640 1 --2- 172.21.15.107:0/3801882988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dbc107460 0x7f1dbc1098b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.101+0000 7f1dc1148640 1 --2- 172.21.15.107:0/3801882988 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dbc104b30 0x7f1dbc106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.102+0000 7f1dc1148640 1 --2- 172.21.15.107:0/3801882988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dbc102200 0x7f1dbc1045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.102+0000 7f1dc1148640 1 -- 172.21.15.107:0/3801882988 >> 172.21.15.107:0/3801882988 conn(0x7f1dbc0fbbf0 msgr2=0x7f1dbc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.102+0000 7f1dc1148640 1 -- 172.21.15.107:0/3801882988 shutdown_connections 2024-04-25T00:16:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.102+0000 7f1dc1148640 1 -- 172.21.15.107:0/3801882988 wait complete. 2024-04-25T00:16:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.102+0000 7f1dc1148640 1 Processor -- start 2024-04-25T00:16:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.102+0000 7f1dc1148640 1 -- start start 2024-04-25T00:16:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.103+0000 7f1dc1148640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dbc102200 0x7f1dbc197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.103+0000 7f1dc1148640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dbc104b30 0x7f1dbc197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.103+0000 7f1dc1148640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dbc107460 0x7f1dbc19f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.103+0000 7f1dc1148640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc10d400 con 0x7f1dbc104b30 2024-04-25T00:16:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.103+0000 7f1dc1148640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc10d280 con 0x7f1dbc102200 2024-04-25T00:16:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.103+0000 7f1dc1148640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1dbc10d580 con 0x7f1dbc107460 2024-04-25T00:16:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.103+0000 7f1db3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dbc104b30 0x7f1dbc197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.103+0000 7f1dbad9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dbc102200 0x7f1dbc197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.103+0000 7f1db3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dbc104b30 0x7f1dbc197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59394/0 (socket says 172.21.15.107:59394) 2024-04-25T00:16:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.103+0000 7f1db3fff640 1 -- 172.21.15.107:0/2195462694 learned_addr learned my addr 172.21.15.107:0/2195462694 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.103+0000 7f1dbb59e640 1 --2- 172.21.15.107:0/2195462694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dbc107460 0x7f1dbc19f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.103+0000 7f1dbad9d640 1 -- 172.21.15.107:0/2195462694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dbc107460 msgr2=0x7f1dbc19f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.103+0000 7f1dbad9d640 1 --2- 172.21.15.107:0/2195462694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dbc107460 0x7f1dbc19f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.103+0000 7f1dbad9d640 1 -- 172.21.15.107:0/2195462694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dbc104b30 msgr2=0x7f1dbc197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.103+0000 7f1dbad9d640 1 --2- 172.21.15.107:0/2195462694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dbc104b30 0x7f1dbc197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.103+0000 7f1dbad9d640 1 -- 172.21.15.107:0/2195462694 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dbc19f6c0 con 0x7f1dbc102200 2024-04-25T00:16:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.103+0000 7f1dbad9d640 1 --2- 172.21.15.107:0/2195462694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dbc102200 0x7f1dbc197aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00e9f0 tx=0x7f1dac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.103+0000 7f1db8d99640 1 -- 172.21.15.107:0/2195462694 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac00ccc0 con 0x7f1dbc102200 2024-04-25T00:16:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.103+0000 7f1dc1148640 1 -- 172.21.15.107:0/2195462694 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dbc19f9b0 con 0x7f1dbc102200 2024-04-25T00:16:28.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.103+0000 7f1db8d99640 1 -- 172.21.15.107:0/2195462694 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1dac010910 con 0x7f1dbc102200 2024-04-25T00:16:28.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.103+0000 7f1db8d99640 1 -- 172.21.15.107:0/2195462694 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dac018960 con 0x7f1dbc102200 2024-04-25T00:16:28.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.103+0000 7f1dc1148640 1 -- 172.21.15.107:0/2195462694 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dbc19ff00 con 0x7f1dbc102200 2024-04-25T00:16:28.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.104+0000 7f1dc1148640 1 -- 172.21.15.107:0/2195462694 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dbc068560 con 0x7f1dbc102200 2024-04-25T00:16:28.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.106+0000 7f1db8d99640 1 -- 172.21.15.107:0/2195462694 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f1dac010ac0 con 0x7f1dbc102200 2024-04-25T00:16:28.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.106+0000 7f1db8d99640 1 --2- 172.21.15.107:0/2195462694 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f1da80a2990 0x7f1da80a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:28.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.106+0000 7f1db3fff640 1 --2- 172.21.15.107:0/2195462694 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f1da80a2990 0x7f1da80a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:28.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.106+0000 7f1db8d99640 1 -- 172.21.15.107:0/2195462694 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f1dac0cf470 con 0x7f1dbc102200 2024-04-25T00:16:28.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.107+0000 7f1db3fff640 1 --2- 172.21.15.107:0/2195462694 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f1da80a2990 0x7f1da80a4e50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc198fb0 tx=0x7f1da4005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:28.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.110+0000 7f1db8d99640 1 -- 172.21.15.107:0/2195462694 <== mon.2 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1dac09bbe0 con 0x7f1dbc102200 2024-04-25T00:16:28.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:27 smithi170 ceph-mon[32807]: [25/Apr/2024:00:16:26] ENGINE Bus STARTING 2024-04-25T00:16:28.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:27 smithi170 ceph-mon[32807]: [25/Apr/2024:00:16:26] ENGINE Serving on http://172.21.15.107:8765 2024-04-25T00:16:28.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:27 smithi170 ceph-mon[32807]: [25/Apr/2024:00:16:26] ENGINE Serving on https://172.21.15.107:7150 2024-04-25T00:16:28.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:27 smithi170 ceph-mon[32807]: [25/Apr/2024:00:16:26] ENGINE Bus STARTED 2024-04-25T00:16:28.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:27 smithi170 ceph-mon[32807]: mgrmap e21: x(active, since 1.01839s), standbys: z, y 2024-04-25T00:16:28.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:27 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3684256037' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:28.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.318+0000 7f1dc1148640 1 -- 172.21.15.107:0/2195462694 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1dbc02d170 con 0x7f1dbc102200 2024-04-25T00:16:28.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.319+0000 7f1db8d99640 1 -- 172.21.15.107:0/2195462694 <== mon.2 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f1dbc02d170 con 0x7f1dbc102200 2024-04-25T00:16:28.320 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-25T00:16:28.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.321+0000 7f1dc1148640 1 -- 172.21.15.107:0/2195462694 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f1da80a2990 msgr2=0x7f1da80a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.321+0000 7f1dc1148640 1 --2- 172.21.15.107:0/2195462694 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f1da80a2990 0x7f1da80a4e50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc198fb0 tx=0x7f1da4005f90 comp rx=0 tx=0).stop 2024-04-25T00:16:28.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.321+0000 7f1dc1148640 1 -- 172.21.15.107:0/2195462694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dbc102200 msgr2=0x7f1dbc197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.321+0000 7f1dc1148640 1 --2- 172.21.15.107:0/2195462694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dbc102200 0x7f1dbc197aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1dac00e9f0 tx=0x7f1dac00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.321+0000 7f1dc1148640 1 -- 172.21.15.107:0/2195462694 shutdown_connections 2024-04-25T00:16:28.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.321+0000 7f1dc1148640 1 --2- 172.21.15.107:0/2195462694 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1dbc107460 0x7f1dbc19f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.321+0000 7f1dc1148640 1 --2- 172.21.15.107:0/2195462694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dbc104b30 0x7f1dbc197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.321+0000 7f1dc1148640 1 --2- 172.21.15.107:0/2195462694 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f1da80a2990 0x7f1da80a4e50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.321+0000 7f1dc1148640 1 --2- 172.21.15.107:0/2195462694 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1dbc102200 0x7f1dbc197aa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.321+0000 7f1dc1148640 1 -- 172.21.15.107:0/2195462694 >> 172.21.15.107:0/2195462694 conn(0x7f1dbc0fbbf0 msgr2=0x7f1dbc102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:28.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.321+0000 7f1dc1148640 1 -- 172.21.15.107:0/2195462694 shutdown_connections 2024-04-25T00:16:28.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.321+0000 7f1dc1148640 1 -- 172.21.15.107:0/2195462694 wait complete. 2024-04-25T00:16:28.340 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:16:28.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.444+0000 7f5108c15640 1 -- 172.21.15.107:0/2817048860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5104100680 msgr2=0x7f5104100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.444+0000 7f5108c15640 1 --2- 172.21.15.107:0/2817048860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5104100680 0x7f5104100aa0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009900 tx=0x7f50f402a410 comp rx=0 tx=0).stop 2024-04-25T00:16:28.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.444+0000 7f5108c15640 1 -- 172.21.15.107:0/2817048860 shutdown_connections 2024-04-25T00:16:28.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.444+0000 7f5108c15640 1 --2- 172.21.15.107:0/2817048860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5104102230 0x7f5104108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.444+0000 7f5108c15640 1 --2- 172.21.15.107:0/2817048860 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5104101870 0x7f5104101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.444+0000 7f5108c15640 1 --2- 172.21.15.107:0/2817048860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5104100680 0x7f5104100aa0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.444+0000 7f5108c15640 1 -- 172.21.15.107:0/2817048860 >> 172.21.15.107:0/2817048860 conn(0x7f51040fbbf0 msgr2=0x7f51040fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:28.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.444+0000 7f5108c15640 1 -- 172.21.15.107:0/2817048860 shutdown_connections 2024-04-25T00:16:28.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.444+0000 7f5108c15640 1 -- 172.21.15.107:0/2817048860 wait complete. 2024-04-25T00:16:28.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.445+0000 7f5108c15640 1 Processor -- start 2024-04-25T00:16:28.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.445+0000 7f5108c15640 1 -- start start 2024-04-25T00:16:28.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.445+0000 7f5108c15640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5104100680 0x7f5104197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:28.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.445+0000 7f5108c15640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5104101870 0x7f5104198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:28.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.445+0000 7f5108c15640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5104102230 0x7f510419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:28.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.445+0000 7f5108c15640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f510410c300 con 0x7f5104101870 2024-04-25T00:16:28.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.445+0000 7f5108c15640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f510410c180 con 0x7f5104102230 2024-04-25T00:16:28.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.445+0000 7f5108c15640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f510410c480 con 0x7f5104100680 2024-04-25T00:16:28.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.446+0000 7f5101d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5104101870 0x7f5104198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:28.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.446+0000 7f5102d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5104102230 0x7f510419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:28.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.446+0000 7f510259c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5104100680 0x7f5104197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:28.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.446+0000 7f5102d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5104102230 0x7f510419f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.107:37170/0 (socket says 172.21.15.107:37170) 2024-04-25T00:16:28.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.446+0000 7f5102d9d640 1 -- 172.21.15.107:0/2647063696 learned_addr learned my addr 172.21.15.107:0/2647063696 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:28.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.446+0000 7f5101d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5104101870 0x7f5104198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59418/0 (socket says 172.21.15.107:59418) 2024-04-25T00:16:28.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.446+0000 7f510259c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5104100680 0x7f5104197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:50642/0 (socket says 172.21.15.107:50642) 2024-04-25T00:16:28.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.446+0000 7f5102d9d640 1 -- 172.21.15.107:0/2647063696 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5104100680 msgr2=0x7f5104197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.446+0000 7f5102d9d640 1 --2- 172.21.15.107:0/2647063696 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5104100680 0x7f5104197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.446+0000 7f5102d9d640 1 -- 172.21.15.107:0/2647063696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5104101870 msgr2=0x7f5104198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.446+0000 7f5102d9d640 1 --2- 172.21.15.107:0/2647063696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5104101870 0x7f5104198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.446+0000 7f5102d9d640 1 -- 172.21.15.107:0/2647063696 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510419f6f0 con 0x7f5104102230 2024-04-25T00:16:28.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.446+0000 7f5101d9b640 1 --2- 172.21.15.107:0/2647063696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5104101870 0x7f5104198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:28.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.446+0000 7f5102d9d640 1 --2- 172.21.15.107:0/2647063696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5104102230 0x7f510419f080 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00eb10 tx=0x7f50fc00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:28.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.446+0000 7f50f37fe640 1 -- 172.21.15.107:0/2647063696 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc004b20 con 0x7f5104102230 2024-04-25T00:16:28.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.446+0000 7f5108c15640 1 -- 172.21.15.107:0/2647063696 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510419f9e0 con 0x7f5104102230 2024-04-25T00:16:28.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.446+0000 7f50f37fe640 1 -- 172.21.15.107:0/2647063696 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50fc004cc0 con 0x7f5104102230 2024-04-25T00:16:28.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.446+0000 7f50f37fe640 1 -- 172.21.15.107:0/2647063696 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc010760 con 0x7f5104102230 2024-04-25T00:16:28.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.446+0000 7f5108c15640 1 -- 172.21.15.107:0/2647063696 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f510419ff80 con 0x7f5104102230 2024-04-25T00:16:28.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.447+0000 7f5108c15640 1 -- 172.21.15.107:0/2647063696 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51041a0270 con 0x7f5104102230 2024-04-25T00:16:28.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.448+0000 7f50f37fe640 1 -- 172.21.15.107:0/2647063696 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f50fc045b10 con 0x7f5104102230 2024-04-25T00:16:28.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.449+0000 7f50f37fe640 1 --2- 172.21.15.107:0/2647063696 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f50ec0a2990 0x7f50ec0a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:28.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.449+0000 7f510259c640 1 --2- 172.21.15.107:0/2647063696 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f50ec0a2990 0x7f50ec0a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:28.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.449+0000 7f50f37fe640 1 -- 172.21.15.107:0/2647063696 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f50fc0cf780 con 0x7f5104102230 2024-04-25T00:16:28.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.449+0000 7f510259c640 1 --2- 172.21.15.107:0/2647063696 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f50ec0a2990 0x7f50ec0a4e50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f50f4005ab0 tx=0x7f50f4000bc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:28.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.452+0000 7f50f37fe640 1 -- 172.21.15.107:0/2647063696 <== mon.2 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f50fc09c540 con 0x7f5104102230 2024-04-25T00:16:28.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.632+0000 7f5108c15640 1 -- 172.21.15.107:0/2647063696 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f51041a0440 con 0x7f5104102230 2024-04-25T00:16:28.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.633+0000 7f50f37fe640 1 -- 172.21.15.107:0/2647063696 <== mon.2 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7f50fc09be40 con 0x7f5104102230 2024-04-25T00:16:28.634 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-04-25T00:16:28.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.635+0000 7f5108c15640 1 -- 172.21.15.107:0/2647063696 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f50ec0a2990 msgr2=0x7f50ec0a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.635+0000 7f5108c15640 1 --2- 172.21.15.107:0/2647063696 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f50ec0a2990 0x7f50ec0a4e50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f50f4005ab0 tx=0x7f50f4000bc0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.635+0000 7f5108c15640 1 -- 172.21.15.107:0/2647063696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5104102230 msgr2=0x7f510419f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.635+0000 7f5108c15640 1 --2- 172.21.15.107:0/2647063696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5104102230 0x7f510419f080 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00eb10 tx=0x7f50fc00c2d0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.635+0000 7f5108c15640 1 -- 172.21.15.107:0/2647063696 shutdown_connections 2024-04-25T00:16:28.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.635+0000 7f5108c15640 1 --2- 172.21.15.107:0/2647063696 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f50ec0a2990 0x7f50ec0a4e50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.635+0000 7f5108c15640 1 --2- 172.21.15.107:0/2647063696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5104102230 0x7f510419f080 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.635+0000 7f5108c15640 1 --2- 172.21.15.107:0/2647063696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5104101870 0x7f5104198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.635+0000 7f5108c15640 1 --2- 172.21.15.107:0/2647063696 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5104100680 0x7f5104197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.635+0000 7f5108c15640 1 -- 172.21.15.107:0/2647063696 >> 172.21.15.107:0/2647063696 conn(0x7f51040fbbf0 msgr2=0x7f51040fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:28.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.635+0000 7f5108c15640 1 -- 172.21.15.107:0/2647063696 shutdown_connections 2024-04-25T00:16:28.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.635+0000 7f5108c15640 1 -- 172.21.15.107:0/2647063696 wait complete. 2024-04-25T00:16:28.648 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:16:28.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.749+0000 7f919ee4c640 1 -- 172.21.15.107:0/203504935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9198104bb0 msgr2=0x7f9198106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.749+0000 7f919ee4c640 1 --2- 172.21.15.107:0/203504935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9198104bb0 0x7f9198106fa0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9184009900 tx=0x7f918402a410 comp rx=0 tx=0).stop 2024-04-25T00:16:28.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.749+0000 7f919ee4c640 1 -- 172.21.15.107:0/203504935 shutdown_connections 2024-04-25T00:16:28.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.749+0000 7f919ee4c640 1 --2- 172.21.15.107:0/203504935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91981074e0 0x7f91981098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.749+0000 7f919ee4c640 1 --2- 172.21.15.107:0/203504935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9198104bb0 0x7f9198106fa0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.749+0000 7f919ee4c640 1 --2- 172.21.15.107:0/203504935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9198102280 0x7f9198104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.749+0000 7f919ee4c640 1 -- 172.21.15.107:0/203504935 >> 172.21.15.107:0/203504935 conn(0x7f91980fbbf0 msgr2=0x7f91980fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:28.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.749+0000 7f919ee4c640 1 -- 172.21.15.107:0/203504935 shutdown_connections 2024-04-25T00:16:28.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.749+0000 7f919ee4c640 1 -- 172.21.15.107:0/203504935 wait complete. 2024-04-25T00:16:28.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.750+0000 7f919ee4c640 1 Processor -- start 2024-04-25T00:16:28.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.750+0000 7f919ee4c640 1 -- start start 2024-04-25T00:16:28.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.750+0000 7f919ee4c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9198102280 0x7f9198197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:28.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.750+0000 7f919ee4c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9198104bb0 0x7f9198198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:28.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.750+0000 7f919ee4c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91981074e0 0x7f919819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:28.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.750+0000 7f919ee4c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f919810d3d0 con 0x7f91981074e0 2024-04-25T00:16:28.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.750+0000 7f919ee4c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f919810d250 con 0x7f9198102280 2024-04-25T00:16:28.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.750+0000 7f919ee4c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f919810d550 con 0x7f9198104bb0 2024-04-25T00:16:28.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.750+0000 7f919d3e9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91981074e0 0x7f919819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:28.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.750+0000 7f919d3e9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91981074e0 0x7f919819f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59446/0 (socket says 172.21.15.107:59446) 2024-04-25T00:16:28.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.750+0000 7f919d3e9640 1 -- 172.21.15.107:0/1486103129 learned_addr learned my addr 172.21.15.107:0/1486103129 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:28.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.750+0000 7f919cbe8640 1 --2- 172.21.15.107:0/1486103129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9198102280 0x7f9198197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:28.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.750+0000 7f9197fff640 1 --2- 172.21.15.107:0/1486103129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9198104bb0 0x7f9198198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:28.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.750+0000 7f919d3e9640 1 -- 172.21.15.107:0/1486103129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9198104bb0 msgr2=0x7f9198198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.750+0000 7f919d3e9640 1 --2- 172.21.15.107:0/1486103129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9198104bb0 0x7f9198198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.750+0000 7f919d3e9640 1 -- 172.21.15.107:0/1486103129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9198102280 msgr2=0x7f9198197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.750+0000 7f919d3e9640 1 --2- 172.21.15.107:0/1486103129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9198102280 0x7f9198197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.751+0000 7f919d3e9640 1 -- 172.21.15.107:0/1486103129 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919819f730 con 0x7f91981074e0 2024-04-25T00:16:28.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.751+0000 7f919cbe8640 1 --2- 172.21.15.107:0/1486103129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9198102280 0x7f9198197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:28.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.751+0000 7f919d3e9640 1 --2- 172.21.15.107:0/1486103129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91981074e0 0x7f919819f0c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f919000efd0 tx=0x7f919000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:28.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.751+0000 7f9195ffb640 1 -- 172.21.15.107:0/1486103129 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9190003b70 con 0x7f91981074e0 2024-04-25T00:16:28.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.751+0000 7f9195ffb640 1 -- 172.21.15.107:0/1486103129 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9190003d10 con 0x7f91981074e0 2024-04-25T00:16:28.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.751+0000 7f919ee4c640 1 -- 172.21.15.107:0/1486103129 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f919819fa20 con 0x7f91981074e0 2024-04-25T00:16:28.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.751+0000 7f9195ffb640 1 -- 172.21.15.107:0/1486103129 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91900109f0 con 0x7f91981074e0 2024-04-25T00:16:28.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.751+0000 7f919ee4c640 1 -- 172.21.15.107:0/1486103129 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f919819fef0 con 0x7f91981074e0 2024-04-25T00:16:28.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.752+0000 7f919ee4c640 1 -- 172.21.15.107:0/1486103129 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91980fd6e0 con 0x7f91981074e0 2024-04-25T00:16:28.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.753+0000 7f9195ffb640 1 -- 172.21.15.107:0/1486103129 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7f9190004020 con 0x7f91981074e0 2024-04-25T00:16:28.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.754+0000 7f9195ffb640 1 --2- 172.21.15.107:0/1486103129 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f91880a2990 0x7f91880a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:28.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.754+0000 7f919cbe8640 1 --2- 172.21.15.107:0/1486103129 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f91880a2990 0x7f91880a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:28.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.754+0000 7f9195ffb640 1 -- 172.21.15.107:0/1486103129 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f91900cf440 con 0x7f91981074e0 2024-04-25T00:16:28.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.754+0000 7f919cbe8640 1 --2- 172.21.15.107:0/1486103129 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f91880a2990 0x7f91880a4e50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f918c009fc0 tx=0x7f918c005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:28.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.758+0000 7f9195ffb640 1 -- 172.21.15.107:0/1486103129 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f919009c2d0 con 0x7f91981074e0 2024-04-25T00:16:28.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.799+0000 7f9195ffb640 1 -- 172.21.15.107:0/1486103129 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 22) ==== 138005+0+0 (secure 0 0 0) 0x7f9190004310 con 0x7f91981074e0 2024-04-25T00:16:28.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.950+0000 7f919ee4c640 1 -- 172.21.15.107:0/1486103129 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9198062340 con 0x7f91981074e0 2024-04-25T00:16:28.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.951+0000 7f9195ffb640 1 -- 172.21.15.107:0/1486103129 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v97) ==== 103+0+11 (secure 0 0 0) 0x7f919008fbc0 con 0x7f91981074e0 2024-04-25T00:16:28.952 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-25T00:16:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.953+0000 7f919ee4c640 1 -- 172.21.15.107:0/1486103129 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f91880a2990 msgr2=0x7f91880a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.953+0000 7f919ee4c640 1 --2- 172.21.15.107:0/1486103129 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f91880a2990 0x7f91880a4e50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f918c009fc0 tx=0x7f918c005f90 comp rx=0 tx=0).stop 2024-04-25T00:16:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.953+0000 7f919ee4c640 1 -- 172.21.15.107:0/1486103129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91981074e0 msgr2=0x7f919819f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.953+0000 7f919ee4c640 1 --2- 172.21.15.107:0/1486103129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91981074e0 0x7f919819f0c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f919000efd0 tx=0x7f919000c6a0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.953+0000 7f919ee4c640 1 -- 172.21.15.107:0/1486103129 shutdown_connections 2024-04-25T00:16:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.953+0000 7f919ee4c640 1 --2- 172.21.15.107:0/1486103129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91981074e0 0x7f919819f0c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.953+0000 7f919ee4c640 1 --2- 172.21.15.107:0/1486103129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9198104bb0 0x7f9198198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.953+0000 7f919ee4c640 1 --2- 172.21.15.107:0/1486103129 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f91880a2990 0x7f91880a4e50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.953+0000 7f919ee4c640 1 --2- 172.21.15.107:0/1486103129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9198102280 0x7f9198197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.953+0000 7f919ee4c640 1 -- 172.21.15.107:0/1486103129 >> 172.21.15.107:0/1486103129 conn(0x7f91980fbbf0 msgr2=0x7f91981040d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.953+0000 7f919ee4c640 1 -- 172.21.15.107:0/1486103129 shutdown_connections 2024-04-25T00:16:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:28.953+0000 7f919ee4c640 1 -- 172.21.15.107:0/1486103129 wait complete. 2024-04-25T00:16:28.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:28 smithi107 ceph-mon[29153]: from='client.25048 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:28.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:28 smithi107 ceph-mon[29153]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:28.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:28 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/73324549' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:16:28.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:28 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:28.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:28 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:28.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:28 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2195462694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:16:28.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:28 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:28.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:28 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:28.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:28 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:28.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:28 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:28.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:28 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2647063696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:16:28.967 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:16:29.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.068+0000 7f677fd70640 1 -- 172.21.15.107:0/1689080522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67781029a0 msgr2=0x7f6778102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.068+0000 7f677fd70640 1 --2- 172.21.15.107:0/1689080522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67781029a0 0x7f6778102e20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6768009900 tx=0x7f676802a410 comp rx=0 tx=0).stop 2024-04-25T00:16:29.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.068+0000 7f677fd70640 1 -- 172.21.15.107:0/1689080522 shutdown_connections 2024-04-25T00:16:29.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.068+0000 7f677fd70640 1 --2- 172.21.15.107:0/1689080522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6778103360 0x7f6778109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.068+0000 7f677fd70640 1 --2- 172.21.15.107:0/1689080522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67781029a0 0x7f6778102e20 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.068+0000 7f677fd70640 1 --2- 172.21.15.107:0/1689080522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f67781017b0 0x7f6778101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.068+0000 7f677fd70640 1 -- 172.21.15.107:0/1689080522 >> 172.21.15.107:0/1689080522 conn(0x7f67780fcd20 msgr2=0x7f67780ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.068+0000 7f677fd70640 1 -- 172.21.15.107:0/1689080522 shutdown_connections 2024-04-25T00:16:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.068+0000 7f677fd70640 1 -- 172.21.15.107:0/1689080522 wait complete. 2024-04-25T00:16:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.068+0000 7f677fd70640 1 Processor -- start 2024-04-25T00:16:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.068+0000 7f677fd70640 1 -- start start 2024-04-25T00:16:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.069+0000 7f677fd70640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67781017b0 0x7f67780754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.069+0000 7f677fd70640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67781029a0 0x7f6778073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.069+0000 7f677fd70640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6778103360 0x7f6778074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.069+0000 7f677fd70640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f677810d480 con 0x7f67781029a0 2024-04-25T00:16:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.069+0000 7f677fd70640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f677810d300 con 0x7f6778103360 2024-04-25T00:16:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.069+0000 7f677fd70640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f677810d600 con 0x7f67781017b0 2024-04-25T00:16:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.069+0000 7f677d30b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67781029a0 0x7f6778073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.069+0000 7f677d30b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67781029a0 0x7f6778073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59464/0 (socket says 172.21.15.107:59464) 2024-04-25T00:16:29.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.069+0000 7f677d30b640 1 -- 172.21.15.107:0/1314700461 learned_addr learned my addr 172.21.15.107:0/1314700461 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:29.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.069+0000 7f677db0c640 1 --2- 172.21.15.107:0/1314700461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67781017b0 0x7f67780754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:29.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.069+0000 7f677e30d640 1 --2- 172.21.15.107:0/1314700461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6778103360 0x7f6778074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:29.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.069+0000 7f677d30b640 1 -- 172.21.15.107:0/1314700461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67781017b0 msgr2=0x7f67780754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.069+0000 7f677d30b640 1 --2- 172.21.15.107:0/1314700461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67781017b0 0x7f67780754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.069+0000 7f677d30b640 1 -- 172.21.15.107:0/1314700461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6778103360 msgr2=0x7f6778074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.069+0000 7f677d30b640 1 --2- 172.21.15.107:0/1314700461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6778103360 0x7f6778074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.069+0000 7f677d30b640 1 -- 172.21.15.107:0/1314700461 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6778074900 con 0x7f67781029a0 2024-04-25T00:16:29.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.069+0000 7f677d30b640 1 --2- 172.21.15.107:0/1314700461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67781029a0 0x7f6778073b30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6768031440 tx=0x7f6768032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:29.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.069+0000 7f676effd640 1 -- 172.21.15.107:0/1314700461 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676803f030 con 0x7f67781029a0 2024-04-25T00:16:29.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.069+0000 7f677fd70640 1 -- 172.21.15.107:0/1314700461 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67781a1bb0 con 0x7f67781029a0 2024-04-25T00:16:29.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.069+0000 7f676effd640 1 -- 172.21.15.107:0/1314700461 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6768003ce0 con 0x7f67781029a0 2024-04-25T00:16:29.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.069+0000 7f676effd640 1 -- 172.21.15.107:0/1314700461 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768035070 con 0x7f67781029a0 2024-04-25T00:16:29.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.070+0000 7f677fd70640 1 -- 172.21.15.107:0/1314700461 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67781a2150 con 0x7f67781029a0 2024-04-25T00:16:29.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.071+0000 7f677fd70640 1 -- 172.21.15.107:0/1314700461 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6778108ad0 con 0x7f67781029a0 2024-04-25T00:16:29.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.072+0000 7f676effd640 1 -- 172.21.15.107:0/1314700461 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 22) ==== 138005+0+0 (secure 0 0 0) 0x7f6768042030 con 0x7f67781029a0 2024-04-25T00:16:29.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.073+0000 7f676effd640 1 --2- 172.21.15.107:0/1314700461 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f67640a2e10 0x7f67640a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:29.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.073+0000 7f677db0c640 1 --2- 172.21.15.107:0/1314700461 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f67640a2e10 0x7f67640a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:29.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.073+0000 7f676effd640 1 -- 172.21.15.107:0/1314700461 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f67680f00c0 con 0x7f67781029a0 2024-04-25T00:16:29.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.073+0000 7f677db0c640 1 --2- 172.21.15.107:0/1314700461 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f67640a2e10 0x7f67640a52d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6778102800 tx=0x7f6774005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:29.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.078+0000 7f676effd640 1 -- 172.21.15.107:0/1314700461 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f67680bc1b0 con 0x7f67781029a0 2024-04-25T00:16:29.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:28 smithi195 ceph-mon[32833]: from='client.25048 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:29.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:28 smithi195 ceph-mon[32833]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:29.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:28 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/73324549' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:16:29.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:28 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:29.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:28 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:29.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:28 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2195462694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:16:29.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:28 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:29.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:28 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:29.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:28 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:29.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:28 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:29.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:28 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2647063696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:16:29.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:28 smithi170 ceph-mon[32807]: from='client.25048 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:29.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:28 smithi170 ceph-mon[32807]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:29.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:28 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/73324549' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:16:29.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:28 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:29.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:28 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:29.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:28 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2195462694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:16:29.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:28 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:29.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:28 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:29.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:28 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:29.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:28 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:29.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:28 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2647063696' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:16:29.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.264+0000 7f677fd70640 1 -- 172.21.15.107:0/1314700461 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f677802d3b0 con 0x7f67781029a0 2024-04-25T00:16:29.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.266+0000 7f676effd640 1 -- 172.21.15.107:0/1314700461 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+25070 (secure 0 0 0) 0x7f67680c04b0 con 0x7f67781029a0 2024-04-25T00:16:29.267 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:16:29.267 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":97,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:16:26.229487+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:59.672801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:00.957782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:14:05.065846+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8},{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/151603643":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/1884923543":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/111369358":"2024-04-26T00:16:26.229442+0000","172.21.15.107:6800/1644942185":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3936375305":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/1644942185":"2024-04-26T00:16:26.229442+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:16:29.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.268+0000 7f677fd70640 1 -- 172.21.15.107:0/1314700461 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f67640a2e10 msgr2=0x7f67640a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.268+0000 7f677fd70640 1 --2- 172.21.15.107:0/1314700461 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f67640a2e10 0x7f67640a52d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6778102800 tx=0x7f6774005f90 comp rx=0 tx=0).stop 2024-04-25T00:16:29.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.268+0000 7f677fd70640 1 -- 172.21.15.107:0/1314700461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67781029a0 msgr2=0x7f6778073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.268+0000 7f677fd70640 1 --2- 172.21.15.107:0/1314700461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67781029a0 0x7f6778073b30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6768031440 tx=0x7f6768032ce0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.268+0000 7f677fd70640 1 -- 172.21.15.107:0/1314700461 shutdown_connections 2024-04-25T00:16:29.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.268+0000 7f677fd70640 1 --2- 172.21.15.107:0/1314700461 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f67640a2e10 0x7f67640a52d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.268+0000 7f677fd70640 1 --2- 172.21.15.107:0/1314700461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6778103360 0x7f6778074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.268+0000 7f677fd70640 1 --2- 172.21.15.107:0/1314700461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67781029a0 0x7f6778073b30 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.268+0000 7f677fd70640 1 --2- 172.21.15.107:0/1314700461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f67781017b0 0x7f67780754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.268+0000 7f677fd70640 1 -- 172.21.15.107:0/1314700461 >> 172.21.15.107:0/1314700461 conn(0x7f67780fcd20 msgr2=0x7f6778105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:29.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.268+0000 7f677fd70640 1 -- 172.21.15.107:0/1314700461 shutdown_connections 2024-04-25T00:16:29.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.268+0000 7f677fd70640 1 -- 172.21.15.107:0/1314700461 wait complete. 2024-04-25T00:16:29.281 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:09:25.160329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:13:59.672801+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:14:00.957782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:14:05.065846+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:16:29.281 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:16:29.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.384+0000 7faaa1929640 1 -- 172.21.15.107:0/2202040243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa9c100680 msgr2=0x7faa9c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.384+0000 7faaa1929640 1 --2- 172.21.15.107:0/2202040243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa9c100680 0x7faa9c100aa0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7faa8c009900 tx=0x7faa8c02a410 comp rx=0 tx=0).stop 2024-04-25T00:16:29.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.385+0000 7faaa1929640 1 -- 172.21.15.107:0/2202040243 shutdown_connections 2024-04-25T00:16:29.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.385+0000 7faaa1929640 1 --2- 172.21.15.107:0/2202040243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa9c102230 0x7faa9c108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.385+0000 7faaa1929640 1 --2- 172.21.15.107:0/2202040243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa9c101870 0x7faa9c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.385+0000 7faaa1929640 1 --2- 172.21.15.107:0/2202040243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa9c100680 0x7faa9c100aa0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.385+0000 7faaa1929640 1 -- 172.21.15.107:0/2202040243 >> 172.21.15.107:0/2202040243 conn(0x7faa9c0fbbf0 msgr2=0x7faa9c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:29.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.385+0000 7faaa1929640 1 -- 172.21.15.107:0/2202040243 shutdown_connections 2024-04-25T00:16:29.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.385+0000 7faaa1929640 1 -- 172.21.15.107:0/2202040243 wait complete. 2024-04-25T00:16:29.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.385+0000 7faaa1929640 1 Processor -- start 2024-04-25T00:16:29.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.385+0000 7faaa1929640 1 -- start start 2024-04-25T00:16:29.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.385+0000 7faaa1929640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa9c100680 0x7faa9c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:29.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.385+0000 7faaa1929640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa9c101870 0x7faa9c198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:29.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.385+0000 7faaa1929640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa9c102230 0x7faa9c19f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:29.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.385+0000 7faaa1929640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c10c300 con 0x7faa9c101870 2024-04-25T00:16:29.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.385+0000 7faaa1929640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c10c180 con 0x7faa9c102230 2024-04-25T00:16:29.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.385+0000 7faaa1929640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c10c480 con 0x7faa9c100680 2024-04-25T00:16:29.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.385+0000 7faa93fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa9c101870 0x7faa9c198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:29.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.385+0000 7faa93fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa9c101870 0x7faa9c198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59468/0 (socket says 172.21.15.107:59468) 2024-04-25T00:16:29.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.385+0000 7faa93fff640 1 -- 172.21.15.107:0/3693606927 learned_addr learned my addr 172.21.15.107:0/3693606927 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:29.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.386+0000 7faa9affd640 1 --2- 172.21.15.107:0/3693606927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa9c100680 0x7faa9c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:29.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.386+0000 7faa9b7fe640 1 --2- 172.21.15.107:0/3693606927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa9c102230 0x7faa9c19f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:29.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.386+0000 7faa93fff640 1 -- 172.21.15.107:0/3693606927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa9c100680 msgr2=0x7faa9c197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.386+0000 7faa93fff640 1 --2- 172.21.15.107:0/3693606927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa9c100680 0x7faa9c197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.386+0000 7faa93fff640 1 -- 172.21.15.107:0/3693606927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa9c102230 msgr2=0x7faa9c19f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.386+0000 7faa93fff640 1 --2- 172.21.15.107:0/3693606927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa9c102230 0x7faa9c19f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.386+0000 7faa93fff640 1 -- 172.21.15.107:0/3693606927 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9c19f720 con 0x7faa9c101870 2024-04-25T00:16:29.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.386+0000 7faa93fff640 1 --2- 172.21.15.107:0/3693606927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa9c101870 0x7faa9c198040 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7faa8400e9f0 tx=0x7faa8400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:29.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.386+0000 7faaa0927640 1 -- 172.21.15.107:0/3693606927 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8400cd80 con 0x7faa9c101870 2024-04-25T00:16:29.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.386+0000 7faaa0927640 1 -- 172.21.15.107:0/3693606927 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa84005500 con 0x7faa9c101870 2024-04-25T00:16:29.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.386+0000 7faaa1929640 1 -- 172.21.15.107:0/3693606927 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa9c19fa10 con 0x7faa9c101870 2024-04-25T00:16:29.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.386+0000 7faaa0927640 1 -- 172.21.15.107:0/3693606927 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa84010690 con 0x7faa9c101870 2024-04-25T00:16:29.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.386+0000 7faaa1929640 1 -- 172.21.15.107:0/3693606927 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa9c19ff60 con 0x7faa9c101870 2024-04-25T00:16:29.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.387+0000 7faaa1929640 1 -- 172.21.15.107:0/3693606927 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa9c1a01f0 con 0x7faa9c101870 2024-04-25T00:16:29.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.388+0000 7faaa0927640 1 -- 172.21.15.107:0/3693606927 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 22) ==== 138005+0+0 (secure 0 0 0) 0x7faa840046c0 con 0x7faa9c101870 2024-04-25T00:16:29.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.389+0000 7faaa0927640 1 --2- 172.21.15.107:0/3693606927 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7faa880a2e10 0x7faa880a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:29.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.389+0000 7faa9affd640 1 --2- 172.21.15.107:0/3693606927 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7faa880a2e10 0x7faa880a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:29.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.389+0000 7faaa0927640 1 -- 172.21.15.107:0/3693606927 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7faa840cff50 con 0x7faa9c101870 2024-04-25T00:16:29.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.389+0000 7faa9affd640 1 --2- 172.21.15.107:0/3693606927 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7faa880a2e10 0x7faa880a52d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faa8c005ab0 tx=0x7faa8c000bc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:29.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.397+0000 7faaa0927640 1 -- 172.21.15.107:0/3693606927 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faa84053740 con 0x7faa9c101870 2024-04-25T00:16:29.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.606+0000 7faaa1929640 1 -- 172.21.15.107:0/3693606927 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faa9c1a03c0 con 0x7faa9c101870 2024-04-25T00:16:29.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.606+0000 7faaa0927640 1 -- 172.21.15.107:0/3693606927 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7faa8409c690 con 0x7faa9c101870 2024-04-25T00:16:29.608 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-04-25T00:16:29.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.610+0000 7faaa1929640 1 -- 172.21.15.107:0/3693606927 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7faa880a2e10 msgr2=0x7faa880a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.610+0000 7faaa1929640 1 --2- 172.21.15.107:0/3693606927 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7faa880a2e10 0x7faa880a52d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faa8c005ab0 tx=0x7faa8c000bc0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.610+0000 7faaa1929640 1 -- 172.21.15.107:0/3693606927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa9c101870 msgr2=0x7faa9c198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.610+0000 7faaa1929640 1 --2- 172.21.15.107:0/3693606927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa9c101870 0x7faa9c198040 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7faa8400e9f0 tx=0x7faa8400edf0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.610+0000 7faaa1929640 1 -- 172.21.15.107:0/3693606927 shutdown_connections 2024-04-25T00:16:29.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.610+0000 7faaa1929640 1 --2- 172.21.15.107:0/3693606927 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7faa880a2e10 0x7faa880a52d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.610+0000 7faaa1929640 1 --2- 172.21.15.107:0/3693606927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa9c102230 0x7faa9c19f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.610+0000 7faaa1929640 1 --2- 172.21.15.107:0/3693606927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa9c101870 0x7faa9c198040 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.610+0000 7faaa1929640 1 --2- 172.21.15.107:0/3693606927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa9c100680 0x7faa9c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.610+0000 7faaa1929640 1 -- 172.21.15.107:0/3693606927 >> 172.21.15.107:0/3693606927 conn(0x7faa9c0fbbf0 msgr2=0x7faa9c0688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:29.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.610+0000 7faaa1929640 1 -- 172.21.15.107:0/3693606927 shutdown_connections 2024-04-25T00:16:29.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.610+0000 7faaa1929640 1 -- 172.21.15.107:0/3693606927 wait complete. 2024-04-25T00:16:29.623 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:16:29.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.725+0000 7f2ebec3f640 1 -- 172.21.15.107:0/4292851862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eb8107460 msgr2=0x7f2eb81098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.725+0000 7f2ebec3f640 1 --2- 172.21.15.107:0/4292851862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eb8107460 0x7f2eb81098b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0009ca0 tx=0x7f2eb002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.725+0000 7f2ebec3f640 1 -- 172.21.15.107:0/4292851862 shutdown_connections 2024-04-25T00:16:29.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.725+0000 7f2ebec3f640 1 --2- 172.21.15.107:0/4292851862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eb8107460 0x7f2eb81098b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.725+0000 7f2ebec3f640 1 --2- 172.21.15.107:0/4292851862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2eb8104b30 0x7f2eb8106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.725+0000 7f2ebec3f640 1 --2- 172.21.15.107:0/4292851862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2eb8102200 0x7f2eb81045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.725+0000 7f2ebec3f640 1 -- 172.21.15.107:0/4292851862 >> 172.21.15.107:0/4292851862 conn(0x7f2eb80fbbf0 msgr2=0x7f2eb80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:29.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.725+0000 7f2ebec3f640 1 -- 172.21.15.107:0/4292851862 shutdown_connections 2024-04-25T00:16:29.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.725+0000 7f2ebec3f640 1 -- 172.21.15.107:0/4292851862 wait complete. 2024-04-25T00:16:29.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.726+0000 7f2ebec3f640 1 Processor -- start 2024-04-25T00:16:29.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.726+0000 7f2ebec3f640 1 -- start start 2024-04-25T00:16:29.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.726+0000 7f2ebec3f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2eb8102200 0x7f2eb8101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:29.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.726+0000 7f2ebec3f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2eb8104b30 0x7f2eb81001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:29.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.726+0000 7f2ebec3f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eb8107460 0x7f2eb81006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:29.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.726+0000 7f2ebec3f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb810d400 con 0x7f2eb8107460 2024-04-25T00:16:29.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.726+0000 7f2ebec3f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb810d280 con 0x7f2eb8102200 2024-04-25T00:16:29.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.726+0000 7f2ebec3f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb810d580 con 0x7f2eb8104b30 2024-04-25T00:16:29.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.726+0000 7f2ebd1dc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eb8107460 0x7f2eb81006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:29.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.726+0000 7f2ebd1dc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eb8107460 0x7f2eb81006f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38984/0 (socket says 172.21.15.107:38984) 2024-04-25T00:16:29.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.726+0000 7f2ebd1dc640 1 -- 172.21.15.107:0/2790949279 learned_addr learned my addr 172.21.15.107:0/2790949279 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:29.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.726+0000 7f2ebc9db640 1 --2- 172.21.15.107:0/2790949279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2eb8102200 0x7f2eb8101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:29.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.726+0000 7f2eb7fff640 1 --2- 172.21.15.107:0/2790949279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2eb8104b30 0x7f2eb81001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:29.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.727+0000 7f2ebd1dc640 1 -- 172.21.15.107:0/2790949279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2eb8104b30 msgr2=0x7f2eb81001b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.727+0000 7f2ebd1dc640 1 --2- 172.21.15.107:0/2790949279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2eb8104b30 0x7f2eb81001b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.727+0000 7f2ebd1dc640 1 -- 172.21.15.107:0/2790949279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2eb8102200 msgr2=0x7f2eb8101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.727+0000 7f2ebd1dc640 1 --2- 172.21.15.107:0/2790949279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2eb8102200 0x7f2eb8101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.727+0000 7f2ebd1dc640 1 -- 172.21.15.107:0/2790949279 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb8100f80 con 0x7f2eb8107460 2024-04-25T00:16:29.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.727+0000 7f2ebc9db640 1 --2- 172.21.15.107:0/2790949279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2eb8102200 0x7f2eb8101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:29.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.727+0000 7f2eb7fff640 1 --2- 172.21.15.107:0/2790949279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2eb8104b30 0x7f2eb81001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:29.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.727+0000 7f2ebd1dc640 1 --2- 172.21.15.107:0/2790949279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eb8107460 0x7f2eb81006f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0031ae0 tx=0x7f2eb0031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:29.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.727+0000 7f2eb5ffb640 1 -- 172.21.15.107:0/2790949279 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0003c30 con 0x7f2eb8107460 2024-04-25T00:16:29.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.727+0000 7f2ebec3f640 1 -- 172.21.15.107:0/2790949279 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb81a1bc0 con 0x7f2eb8107460 2024-04-25T00:16:29.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.727+0000 7f2eb5ffb640 1 -- 172.21.15.107:0/2790949279 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb0003dd0 con 0x7f2eb8107460 2024-04-25T00:16:29.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.727+0000 7f2eb5ffb640 1 -- 172.21.15.107:0/2790949279 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb002fcb0 con 0x7f2eb8107460 2024-04-25T00:16:29.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.727+0000 7f2ebec3f640 1 -- 172.21.15.107:0/2790949279 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb81a2160 con 0x7f2eb8107460 2024-04-25T00:16:29.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.728+0000 7f2ebec3f640 1 -- 172.21.15.107:0/2790949279 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb80fd6e0 con 0x7f2eb8107460 2024-04-25T00:16:29.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.730+0000 7f2eb5ffb640 1 -- 172.21.15.107:0/2790949279 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 22) ==== 138005+0+0 (secure 0 0 0) 0x7f2eb0004050 con 0x7f2eb8107460 2024-04-25T00:16:29.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.730+0000 7f2eb5ffb640 1 --2- 172.21.15.107:0/2790949279 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f2ea80a2d90 0x7f2ea80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:29.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.731+0000 7f2ebc9db640 1 --2- 172.21.15.107:0/2790949279 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f2ea80a2d90 0x7f2ea80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:29.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.731+0000 7f2eb5ffb640 1 -- 172.21.15.107:0/2790949279 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f2eb00f0d50 con 0x7f2eb8107460 2024-04-25T00:16:29.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.731+0000 7f2ebc9db640 1 --2- 172.21.15.107:0/2790949279 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f2ea80a2d90 0x7f2ea80a5250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2eac005d00 tx=0x7f2eac00b410 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:29.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.734+0000 7f2eb5ffb640 1 -- 172.21.15.107:0/2790949279 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2eb00bd5a0 con 0x7f2eb8107460 2024-04-25T00:16:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.925+0000 7f2ebec3f640 1 -- 172.21.15.107:0/2790949279 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2eb802d170 con 0x7f2eb8107460 2024-04-25T00:16:29.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.926+0000 7f2eb5ffb640 1 -- 172.21.15.107:0/2790949279 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f2eb00bcea0 con 0x7f2eb8107460 2024-04-25T00:16:29.927 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-25T00:16:29.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.927+0000 7f2ebec3f640 1 -- 172.21.15.107:0/2790949279 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f2ea80a2d90 msgr2=0x7f2ea80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.927+0000 7f2ebec3f640 1 --2- 172.21.15.107:0/2790949279 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f2ea80a2d90 0x7f2ea80a5250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2eac005d00 tx=0x7f2eac00b410 comp rx=0 tx=0).stop 2024-04-25T00:16:29.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.927+0000 7f2ebec3f640 1 -- 172.21.15.107:0/2790949279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eb8107460 msgr2=0x7f2eb81006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.927+0000 7f2ebec3f640 1 --2- 172.21.15.107:0/2790949279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eb8107460 0x7f2eb81006f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0031ae0 tx=0x7f2eb0031ee0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.928+0000 7f2ebec3f640 1 -- 172.21.15.107:0/2790949279 shutdown_connections 2024-04-25T00:16:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.928+0000 7f2ebec3f640 1 --2- 172.21.15.107:0/2790949279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2eb8107460 0x7f2eb81006f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.928+0000 7f2ebec3f640 1 --2- 172.21.15.107:0/2790949279 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2eb8104b30 0x7f2eb81001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.928+0000 7f2ebec3f640 1 --2- 172.21.15.107:0/2790949279 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f2ea80a2d90 0x7f2ea80a5250 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.928+0000 7f2ebec3f640 1 --2- 172.21.15.107:0/2790949279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2eb8102200 0x7f2eb8101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.928+0000 7f2ebec3f640 1 -- 172.21.15.107:0/2790949279 >> 172.21.15.107:0/2790949279 conn(0x7f2eb80fbbf0 msgr2=0x7f2eb8102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.928+0000 7f2ebec3f640 1 -- 172.21.15.107:0/2790949279 shutdown_connections 2024-04-25T00:16:29.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:29.928+0000 7f2ebec3f640 1 -- 172.21.15.107:0/2790949279 wait complete. 2024-04-25T00:16:29.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:29.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-04-25T00:16:29.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1486103129' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:16:29.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1314700461' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:29.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:29.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:29.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:29.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:29.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:29.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:29.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:29.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:29.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:29.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:29.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:29.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:29.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:29.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:29.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:29.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:29.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:29.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:29.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:29.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:29.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:29 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3693606927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:16:29.942 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:16:30.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.059+0000 7f3fa9348640 1 -- 172.21.15.107:0/3589965766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fa4101880 msgr2=0x7f3fa4101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:30.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.059+0000 7f3fa9348640 1 --2- 172.21.15.107:0/3589965766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fa4101880 0x7f3fa4101d00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009900 tx=0x7f3f8c02a410 comp rx=0 tx=0).stop 2024-04-25T00:16:30.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.059+0000 7f3fa9348640 1 -- 172.21.15.107:0/3589965766 shutdown_connections 2024-04-25T00:16:30.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.059+0000 7f3fa9348640 1 --2- 172.21.15.107:0/3589965766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fa4102240 0x7f3fa4108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.059+0000 7f3fa9348640 1 --2- 172.21.15.107:0/3589965766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fa4101880 0x7f3fa4101d00 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.059+0000 7f3fa9348640 1 --2- 172.21.15.107:0/3589965766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3fa4100690 0x7f3fa4100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.059+0000 7f3fa9348640 1 -- 172.21.15.107:0/3589965766 >> 172.21.15.107:0/3589965766 conn(0x7f3fa40fbc00 msgr2=0x7f3fa40fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.059+0000 7f3fa9348640 1 -- 172.21.15.107:0/3589965766 shutdown_connections 2024-04-25T00:16:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.059+0000 7f3fa9348640 1 -- 172.21.15.107:0/3589965766 wait complete. 2024-04-25T00:16:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.059+0000 7f3fa9348640 1 Processor -- start 2024-04-25T00:16:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.059+0000 7f3fa9348640 1 -- start start 2024-04-25T00:16:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.059+0000 7f3fa9348640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3fa4100690 0x7f3fa4197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.059+0000 7f3fa9348640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fa4101880 0x7f3fa4197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.060+0000 7f3fa9348640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fa4102240 0x7f3fa419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.060+0000 7f3fa9348640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa410d430 con 0x7f3fa4101880 2024-04-25T00:16:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.060+0000 7f3fa9348640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa410d2b0 con 0x7f3fa4102240 2024-04-25T00:16:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.060+0000 7f3fa9348640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa410d5b0 con 0x7f3fa4100690 2024-04-25T00:16:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.060+0000 7f3f9a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fa4101880 0x7f3fa4197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.060+0000 7f3f9a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fa4101880 0x7f3fa4197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38998/0 (socket says 172.21.15.107:38998) 2024-04-25T00:16:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.060+0000 7f3f9a7fc640 1 -- 172.21.15.107:0/473102243 learned_addr learned my addr 172.21.15.107:0/473102243 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.060+0000 7f3fa37fe640 1 --2- 172.21.15.107:0/473102243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fa4102240 0x7f3fa419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.060+0000 7f3fa2ffd640 1 --2- 172.21.15.107:0/473102243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3fa4100690 0x7f3fa4197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.060+0000 7f3f9a7fc640 1 -- 172.21.15.107:0/473102243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3fa4100690 msgr2=0x7f3fa4197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.060+0000 7f3f9a7fc640 1 --2- 172.21.15.107:0/473102243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3fa4100690 0x7f3fa4197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.060+0000 7f3f9a7fc640 1 -- 172.21.15.107:0/473102243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fa4102240 msgr2=0x7f3fa419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.060+0000 7f3f9a7fc640 1 --2- 172.21.15.107:0/473102243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fa4102240 0x7f3fa419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.060+0000 7f3f9a7fc640 1 -- 172.21.15.107:0/473102243 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa419f680 con 0x7f3fa4101880 2024-04-25T00:16:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.060+0000 7f3f9a7fc640 1 --2- 172.21.15.107:0/473102243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fa4101880 0x7f3fa4197fa0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009230 tx=0x7f3f8c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.060+0000 7f3fa0ff9640 1 -- 172.21.15.107:0/473102243 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c03f030 con 0x7f3fa4101880 2024-04-25T00:16:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.060+0000 7f3fa9348640 1 -- 172.21.15.107:0/473102243 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa419f910 con 0x7f3fa4101880 2024-04-25T00:16:30.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.060+0000 7f3fa0ff9640 1 -- 172.21.15.107:0/473102243 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f8c003ce0 con 0x7f3fa4101880 2024-04-25T00:16:30.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.060+0000 7f3fa0ff9640 1 -- 172.21.15.107:0/473102243 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c035070 con 0x7f3fa4101880 2024-04-25T00:16:30.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.060+0000 7f3fa9348640 1 -- 172.21.15.107:0/473102243 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa419fe50 con 0x7f3fa4101880 2024-04-25T00:16:30.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.061+0000 7f3fa9348640 1 -- 172.21.15.107:0/473102243 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fa41079b0 con 0x7f3fa4101880 2024-04-25T00:16:30.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.063+0000 7f3fa0ff9640 1 -- 172.21.15.107:0/473102243 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 22) ==== 138005+0+0 (secure 0 0 0) 0x7f3f8c042030 con 0x7f3fa4101880 2024-04-25T00:16:30.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.064+0000 7f3fa0ff9640 1 --2- 172.21.15.107:0/473102243 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f3f900a2d90 0x7f3f900a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:30.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.064+0000 7f3fa0ff9640 1 -- 172.21.15.107:0/473102243 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f3f8c0f03e0 con 0x7f3fa4101880 2024-04-25T00:16:30.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.064+0000 7f3fa2ffd640 1 --2- 172.21.15.107:0/473102243 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f3f900a2d90 0x7f3f900a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:30.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.064+0000 7f3fa2ffd640 1 --2- 172.21.15.107:0/473102243 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f3f900a2d90 0x7f3f900a5250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3fa41016e0 tx=0x7f3f94005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:30.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.067+0000 7f3fa0ff9640 1 -- 172.21.15.107:0/473102243 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3f8c0bc4d0 con 0x7f3fa4101880 2024-04-25T00:16:30.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:30.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-04-25T00:16:30.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1486103129' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:16:30.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1314700461' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:30.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:30.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:30.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:30.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:30.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:30.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:30.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:30.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:30.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:30.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:30.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:30.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:30.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:30.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:30.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:30.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:30.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:30.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:30.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:30.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:30.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:29 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3693606927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:16:30.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:30.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-04-25T00:16:30.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1486103129' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:16:30.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1314700461' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:30.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:30.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:30.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:30.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:30.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:30.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:30.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:30.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:30.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:30.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:30.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:30.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:30.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:30.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:30.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:30.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:30.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:30.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:16:30.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:30.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:30.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:29 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3693606927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:16:30.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.290+0000 7f3fa9348640 1 -- 172.21.15.107:0/473102243 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3fa402d170 con 0x7f3fa4101880 2024-04-25T00:16:30.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.290+0000 7f3fa0ff9640 1 -- 172.21.15.107:0/473102243 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7f3f8c0c07d0 con 0x7f3fa4101880 2024-04-25T00:16:30.292 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-04-25T00:16:30.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.293+0000 7f3fa9348640 1 -- 172.21.15.107:0/473102243 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f3f900a2d90 msgr2=0x7f3f900a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:30.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.293+0000 7f3fa9348640 1 --2- 172.21.15.107:0/473102243 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f3f900a2d90 0x7f3f900a5250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3fa41016e0 tx=0x7f3f94005f90 comp rx=0 tx=0).stop 2024-04-25T00:16:30.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.293+0000 7f3fa9348640 1 -- 172.21.15.107:0/473102243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fa4101880 msgr2=0x7f3fa4197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:30.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.293+0000 7f3fa9348640 1 --2- 172.21.15.107:0/473102243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fa4101880 0x7f3fa4197fa0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c009230 tx=0x7f3f8c004430 comp rx=0 tx=0).stop 2024-04-25T00:16:30.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.293+0000 7f3fa9348640 1 -- 172.21.15.107:0/473102243 shutdown_connections 2024-04-25T00:16:30.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.293+0000 7f3fa9348640 1 --2- 172.21.15.107:0/473102243 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f3f900a2d90 0x7f3f900a5250 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.293+0000 7f3fa9348640 1 --2- 172.21.15.107:0/473102243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3fa4102240 0x7f3fa419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.293+0000 7f3fa9348640 1 --2- 172.21.15.107:0/473102243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fa4101880 0x7f3fa4197fa0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.293+0000 7f3fa9348640 1 --2- 172.21.15.107:0/473102243 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3fa4100690 0x7f3fa4197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.293+0000 7f3fa9348640 1 -- 172.21.15.107:0/473102243 >> 172.21.15.107:0/473102243 conn(0x7f3fa40fbc00 msgr2=0x7f3fa4104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:30.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.293+0000 7f3fa9348640 1 -- 172.21.15.107:0/473102243 shutdown_connections 2024-04-25T00:16:30.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.293+0000 7f3fa9348640 1 -- 172.21.15.107:0/473102243 wait complete. 2024-04-25T00:16:30.305 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:16:30.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.422+0000 7f0960229640 1 -- 172.21.15.107:0/1836780053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09580ffc50 msgr2=0x7f09581000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:30.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.422+0000 7f0960229640 1 --2- 172.21.15.107:0/1836780053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09580ffc50 0x7f09581000b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0948009930 tx=0x7f094802a420 comp rx=0 tx=0).stop 2024-04-25T00:16:30.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.422+0000 7f0960229640 1 -- 172.21.15.107:0/1836780053 shutdown_connections 2024-04-25T00:16:30.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.422+0000 7f0960229640 1 --2- 172.21.15.107:0/1836780053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09581005f0 0x7f095810e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.422+0000 7f0960229640 1 --2- 172.21.15.107:0/1836780053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09580ffc50 0x7f09581000b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.422+0000 7f0960229640 1 --2- 172.21.15.107:0/1836780053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0958107710 0x7f0958107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.422+0000 7f0960229640 1 -- 172.21.15.107:0/1836780053 >> 172.21.15.107:0/1836780053 conn(0x7f0958076be0 msgr2=0x7f09580fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:30.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.422+0000 7f0960229640 1 -- 172.21.15.107:0/1836780053 shutdown_connections 2024-04-25T00:16:30.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.422+0000 7f0960229640 1 -- 172.21.15.107:0/1836780053 wait complete. 2024-04-25T00:16:30.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.423+0000 7f0960229640 1 Processor -- start 2024-04-25T00:16:30.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.423+0000 7f0960229640 1 -- start start 2024-04-25T00:16:30.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.423+0000 7f0960229640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09580ffc50 0x7f0958198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:30.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.423+0000 7f0960229640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09581005f0 0x7f09581985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:30.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.423+0000 7f0960229640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0958107710 0x7f095819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:30.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.423+0000 7f0960229640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0958111b50 con 0x7f09580ffc50 2024-04-25T00:16:30.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.423+0000 7f0960229640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f09581119d0 con 0x7f0958107710 2024-04-25T00:16:30.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.423+0000 7f0960229640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0958111cd0 con 0x7f09581005f0 2024-04-25T00:16:30.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.423+0000 7f095dfc5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09580ffc50 0x7f0958198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:30.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.423+0000 7f095dfc5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09580ffc50 0x7f0958198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39018/0 (socket says 172.21.15.107:39018) 2024-04-25T00:16:30.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.423+0000 7f095dfc5640 1 -- 172.21.15.107:0/461372428 learned_addr learned my addr 172.21.15.107:0/461372428 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:30.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.423+0000 7f095d7c4640 1 --2- 172.21.15.107:0/461372428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09581005f0 0x7f09581985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:30.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.423+0000 7f095e7c6640 1 --2- 172.21.15.107:0/461372428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0958107710 0x7f095819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:30.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.423+0000 7f095dfc5640 1 -- 172.21.15.107:0/461372428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09581005f0 msgr2=0x7f09581985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:30.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.423+0000 7f095dfc5640 1 --2- 172.21.15.107:0/461372428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09581005f0 0x7f09581985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.423+0000 7f095dfc5640 1 -- 172.21.15.107:0/461372428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0958107710 msgr2=0x7f095819c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:30.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.423+0000 7f095dfc5640 1 --2- 172.21.15.107:0/461372428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0958107710 0x7f095819c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.423+0000 7f095dfc5640 1 -- 172.21.15.107:0/461372428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095819d030 con 0x7f09580ffc50 2024-04-25T00:16:30.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.423+0000 7f095dfc5640 1 --2- 172.21.15.107:0/461372428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09580ffc50 0x7f0958198090 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f095400e9f0 tx=0x7f095400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:30.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.424+0000 7f094effd640 1 -- 172.21.15.107:0/461372428 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095400cd80 con 0x7f09580ffc50 2024-04-25T00:16:30.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.424+0000 7f094effd640 1 -- 172.21.15.107:0/461372428 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0954010910 con 0x7f09580ffc50 2024-04-25T00:16:30.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.424+0000 7f0960229640 1 -- 172.21.15.107:0/461372428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f095819d320 con 0x7f09580ffc50 2024-04-25T00:16:30.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.424+0000 7f094effd640 1 -- 172.21.15.107:0/461372428 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954018990 con 0x7f09580ffc50 2024-04-25T00:16:30.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.424+0000 7f0960229640 1 -- 172.21.15.107:0/461372428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09581a4bb0 con 0x7f09580ffc50 2024-04-25T00:16:30.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.425+0000 7f0960229640 1 -- 172.21.15.107:0/461372428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0958067cd0 con 0x7f09580ffc50 2024-04-25T00:16:30.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.426+0000 7f094effd640 1 -- 172.21.15.107:0/461372428 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 22) ==== 138005+0+0 (secure 0 0 0) 0x7f0954010ac0 con 0x7f09580ffc50 2024-04-25T00:16:30.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.426+0000 7f094effd640 1 --2- 172.21.15.107:0/461372428 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f09440a2d90 0x7f09440a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:30.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.427+0000 7f095d7c4640 1 --2- 172.21.15.107:0/461372428 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f09440a2d90 0x7f09440a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:30.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.427+0000 7f094effd640 1 -- 172.21.15.107:0/461372428 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f09540cfbf0 con 0x7f09580ffc50 2024-04-25T00:16:30.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.427+0000 7f095d7c4640 1 --2- 172.21.15.107:0/461372428 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f09440a2d90 0x7f09440a5250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0948009900 tx=0x7f09480058d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:30.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.431+0000 7f094effd640 1 -- 172.21.15.107:0/461372428 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f095409bce0 con 0x7f09580ffc50 2024-04-25T00:16:30.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.630+0000 7f0960229640 1 -- 172.21.15.107:0/461372428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f095802d170 con 0x7f09580ffc50 2024-04-25T00:16:30.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.631+0000 7f094effd640 1 -- 172.21.15.107:0/461372428 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v97) ==== 103+0+11 (secure 0 0 0) 0x7f095409ffe0 con 0x7f09580ffc50 2024-04-25T00:16:30.632 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-25T00:16:30.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.633+0000 7f0960229640 1 -- 172.21.15.107:0/461372428 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f09440a2d90 msgr2=0x7f09440a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:30.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.633+0000 7f0960229640 1 --2- 172.21.15.107:0/461372428 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f09440a2d90 0x7f09440a5250 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0948009900 tx=0x7f09480058d0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.633+0000 7f0960229640 1 -- 172.21.15.107:0/461372428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09580ffc50 msgr2=0x7f0958198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:30.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.633+0000 7f0960229640 1 --2- 172.21.15.107:0/461372428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09580ffc50 0x7f0958198090 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f095400e9f0 tx=0x7f095400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.633+0000 7f0960229640 1 -- 172.21.15.107:0/461372428 shutdown_connections 2024-04-25T00:16:30.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.633+0000 7f0960229640 1 --2- 172.21.15.107:0/461372428 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f09440a2d90 0x7f09440a5250 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.633+0000 7f0960229640 1 --2- 172.21.15.107:0/461372428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0958107710 0x7f095819c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.633+0000 7f0960229640 1 --2- 172.21.15.107:0/461372428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09581005f0 0x7f09581985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.633+0000 7f0960229640 1 --2- 172.21.15.107:0/461372428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09580ffc50 0x7f0958198090 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.633+0000 7f0960229640 1 -- 172.21.15.107:0/461372428 >> 172.21.15.107:0/461372428 conn(0x7f0958076be0 msgr2=0x7f09580fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:30.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.633+0000 7f0960229640 1 -- 172.21.15.107:0/461372428 shutdown_connections 2024-04-25T00:16:30.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.633+0000 7f0960229640 1 -- 172.21.15.107:0/461372428 wait complete. 2024-04-25T00:16:30.646 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-25T00:16:30.646 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:16:30.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.746+0000 7fa05ee93640 1 -- 172.21.15.107:0/2847142766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa058100fe0 msgr2=0x7fa05810d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:30.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.746+0000 7fa05ee93640 1 --2- 172.21.15.107:0/2847142766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa058100fe0 0x7fa05810d4f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa050009cd0 tx=0x7fa05002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.747+0000 7fa05ee93640 1 -- 172.21.15.107:0/2847142766 shutdown_connections 2024-04-25T00:16:30.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.747+0000 7fa05ee93640 1 --2- 172.21.15.107:0/2847142766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa058100fe0 0x7fa05810d4f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.747+0000 7fa05ee93640 1 --2- 172.21.15.107:0/2847142766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa058100620 0x7fa058100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.747+0000 7fa05ee93640 1 --2- 172.21.15.107:0/2847142766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa058106630 0x7fa058106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.747+0000 7fa05ee93640 1 -- 172.21.15.107:0/2847142766 >> 172.21.15.107:0/2847142766 conn(0x7fa058077740 msgr2=0x7fa058077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:30.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.747+0000 7fa05ee93640 1 -- 172.21.15.107:0/2847142766 shutdown_connections 2024-04-25T00:16:30.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.747+0000 7fa05ee93640 1 -- 172.21.15.107:0/2847142766 wait complete. 2024-04-25T00:16:30.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.747+0000 7fa05ee93640 1 Processor -- start 2024-04-25T00:16:30.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.747+0000 7fa05ee93640 1 -- start start 2024-04-25T00:16:30.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.747+0000 7fa05ee93640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa058100620 0x7fa058197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:30.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.747+0000 7fa05ee93640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa058100fe0 0x7fa058198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:30.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.747+0000 7fa05ee93640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa058106630 0x7fa05819c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:30.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.747+0000 7fa05ee93640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa058110a50 con 0x7fa058106630 2024-04-25T00:16:30.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.747+0000 7fa05ee93640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa0581108d0 con 0x7fa058100fe0 2024-04-25T00:16:30.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.747+0000 7fa05ee93640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa058110bd0 con 0x7fa058100620 2024-04-25T00:16:30.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.748+0000 7fa05d430640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa058106630 0x7fa05819c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:30.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.748+0000 7fa05d430640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa058106630 0x7fa05819c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39036/0 (socket says 172.21.15.107:39036) 2024-04-25T00:16:30.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.748+0000 7fa05d430640 1 -- 172.21.15.107:0/1132749210 learned_addr learned my addr 172.21.15.107:0/1132749210 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:30.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.748+0000 7fa057fff640 1 --2- 172.21.15.107:0/1132749210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa058100fe0 0x7fa058198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:30.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.748+0000 7fa05cc2f640 1 --2- 172.21.15.107:0/1132749210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa058100620 0x7fa058197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:30.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.748+0000 7fa05d430640 1 -- 172.21.15.107:0/1132749210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa058100620 msgr2=0x7fa058197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:30.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.748+0000 7fa05d430640 1 --2- 172.21.15.107:0/1132749210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa058100620 0x7fa058197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.748+0000 7fa05d430640 1 -- 172.21.15.107:0/1132749210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa058100fe0 msgr2=0x7fa058198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:30.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.748+0000 7fa05d430640 1 --2- 172.21.15.107:0/1132749210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa058100fe0 0x7fa058198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.748+0000 7fa05d430640 1 -- 172.21.15.107:0/1132749210 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05819cf60 con 0x7fa058106630 2024-04-25T00:16:30.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.748+0000 7fa05cc2f640 1 --2- 172.21.15.107:0/1132749210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa058100620 0x7fa058197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:30.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.748+0000 7fa05d430640 1 --2- 172.21.15.107:0/1132749210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa058106630 0x7fa05819c830 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa050033040 tx=0x7fa050031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:30.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.748+0000 7fa055ffb640 1 -- 172.21.15.107:0/1132749210 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050004040 con 0x7fa058106630 2024-04-25T00:16:30.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.748+0000 7fa05ee93640 1 -- 172.21.15.107:0/1132749210 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05819d1f0 con 0x7fa058106630 2024-04-25T00:16:30.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.748+0000 7fa055ffb640 1 -- 172.21.15.107:0/1132749210 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0500046c0 con 0x7fa058106630 2024-04-25T00:16:30.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.748+0000 7fa055ffb640 1 -- 172.21.15.107:0/1132749210 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050030ed0 con 0x7fa058106630 2024-04-25T00:16:30.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.748+0000 7fa05ee93640 1 -- 172.21.15.107:0/1132749210 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0581a4ae0 con 0x7fa058106630 2024-04-25T00:16:30.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.749+0000 7fa05ee93640 1 -- 172.21.15.107:0/1132749210 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa058068560 con 0x7fa058106630 2024-04-25T00:16:30.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.750+0000 7fa055ffb640 1 -- 172.21.15.107:0/1132749210 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 22) ==== 138005+0+0 (secure 0 0 0) 0x7fa0500041e0 con 0x7fa058106630 2024-04-25T00:16:30.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.751+0000 7fa055ffb640 1 --2- 172.21.15.107:0/1132749210 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa0480a2d90 0x7fa0480a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:30.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.751+0000 7fa05cc2f640 1 --2- 172.21.15.107:0/1132749210 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa0480a2d90 0x7fa0480a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:30.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.751+0000 7fa055ffb640 1 -- 172.21.15.107:0/1132749210 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fa05002e030 con 0x7fa058106630 2024-04-25T00:16:30.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.751+0000 7fa05cc2f640 1 --2- 172.21.15.107:0/1132749210 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa0480a2d90 0x7fa0480a5250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa04c009ba0 tx=0x7fa04c005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:30.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.756+0000 7fa055ffb640 1 -- 172.21.15.107:0/1132749210 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa05004b060 con 0x7fa058106630 2024-04-25T00:16:30.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.804+0000 7fa055ffb640 1 -- 172.21.15.107:0/1132749210 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7fa0500b96b0 con 0x7fa058106630 2024-04-25T00:16:30.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: Adjusting osd_memory_target on smithi170 to 176.2M 2024-04-25T00:16:30.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: Unable to set osd_memory_target on smithi170 to 184790323: error parsing value: Value '184790323' is below minimum 939524096 2024-04-25T00:16:30.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: Adjusting osd_memory_target on smithi195 to 176.2M 2024-04-25T00:16:30.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: Unable to set osd_memory_target on smithi195 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T00:16:30.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: Adjusting osd_memory_target on smithi107 to 176.1M 2024-04-25T00:16:30.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: Unable to set osd_memory_target on smithi107 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-04-25T00:16:30.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-25T00:16:30.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: Updating smithi170:/etc/ceph/ceph.conf 2024-04-25T00:16:30.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: Updating smithi195:/etc/ceph/ceph.conf 2024-04-25T00:16:30.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: Updating smithi170:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:16:30.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: Updating smithi107:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:16:30.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: Updating smithi195:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:16:30.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2790949279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:16:30.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:16:30.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:16:30.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: Updating smithi195:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:16:30.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/473102243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:16:30.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:30.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/461372428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:16:30.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:30.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:30.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:30.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:30.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:30.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:30 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:30.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.985+0000 7fa05ee93640 1 -- 172.21.15.107:0/1132749210 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0581a4d80 con 0x7fa058106630 2024-04-25T00:16:30.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.986+0000 7fa055ffb640 1 -- 172.21.15.107:0/1132749210 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11897 (secure 0 0 0) 0x7fa0500bd740 con 0x7fa058106630 2024-04-25T00:16:30.987 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:16:30.987 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:16:24.724978+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24493,24521]},"id":1}]} 2024-04-25T00:16:30.988 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 24 2024-04-25T00:16:30.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.988+0000 7fa05ee93640 1 -- 172.21.15.107:0/1132749210 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa0480a2d90 msgr2=0x7fa0480a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:30.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.988+0000 7fa05ee93640 1 --2- 172.21.15.107:0/1132749210 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa0480a2d90 0x7fa0480a5250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa04c009ba0 tx=0x7fa04c005f90 comp rx=0 tx=0).stop 2024-04-25T00:16:30.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.988+0000 7fa05ee93640 1 -- 172.21.15.107:0/1132749210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa058106630 msgr2=0x7fa05819c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:30.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.988+0000 7fa05ee93640 1 --2- 172.21.15.107:0/1132749210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa058106630 0x7fa05819c830 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa050033040 tx=0x7fa050031e60 comp rx=0 tx=0).stop 2024-04-25T00:16:30.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.988+0000 7fa05ee93640 1 -- 172.21.15.107:0/1132749210 shutdown_connections 2024-04-25T00:16:30.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.988+0000 7fa05ee93640 1 --2- 172.21.15.107:0/1132749210 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa058106630 0x7fa05819c830 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.988+0000 7fa05ee93640 1 --2- 172.21.15.107:0/1132749210 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa0480a2d90 0x7fa0480a5250 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.988+0000 7fa05ee93640 1 --2- 172.21.15.107:0/1132749210 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa058100fe0 0x7fa058198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.988+0000 7fa05ee93640 1 --2- 172.21.15.107:0/1132749210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa058100620 0x7fa058197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:30.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.988+0000 7fa05ee93640 1 -- 172.21.15.107:0/1132749210 >> 172.21.15.107:0/1132749210 conn(0x7fa058077740 msgr2=0x7fa0580ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:30.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.988+0000 7fa05ee93640 1 -- 172.21.15.107:0/1132749210 shutdown_connections 2024-04-25T00:16:30.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:30.988+0000 7fa05ee93640 1 -- 172.21.15.107:0/1132749210 wait complete. 2024-04-25T00:16:31.000 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-25T00:16:31.000 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-25T00:16:31.000 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:16:31.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: Adjusting osd_memory_target on smithi170 to 176.2M 2024-04-25T00:16:31.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: Unable to set osd_memory_target on smithi170 to 184790323: error parsing value: Value '184790323' is below minimum 939524096 2024-04-25T00:16:31.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: Adjusting osd_memory_target on smithi195 to 176.2M 2024-04-25T00:16:31.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: Unable to set osd_memory_target on smithi195 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T00:16:31.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: Adjusting osd_memory_target on smithi107 to 176.1M 2024-04-25T00:16:31.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: Unable to set osd_memory_target on smithi107 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-04-25T00:16:31.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-25T00:16:31.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: Updating smithi170:/etc/ceph/ceph.conf 2024-04-25T00:16:31.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: Updating smithi195:/etc/ceph/ceph.conf 2024-04-25T00:16:31.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: Updating smithi170:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:16:31.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: Updating smithi107:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:16:31.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: Updating smithi195:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:16:31.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2790949279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:16:31.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:16:31.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:16:31.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: Updating smithi195:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:16:31.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/473102243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:16:31.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:31.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/461372428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:16:31.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:31.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:31.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:31.093 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:31.093 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:31.093 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:30 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:31.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.097+0000 7f62a305d640 1 -- 172.21.15.107:0/1946388549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f629c101710 msgr2=0x7f629c101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:31.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.097+0000 7f62a305d640 1 --2- 172.21.15.107:0/1946388549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f629c101710 0x7f629c101b90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f6290009930 tx=0x7f629002a420 comp rx=0 tx=0).stop 2024-04-25T00:16:31.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.097+0000 7f62a305d640 1 -- 172.21.15.107:0/1946388549 shutdown_connections 2024-04-25T00:16:31.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.097+0000 7f62a305d640 1 --2- 172.21.15.107:0/1946388549 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f629c1020d0 0x7f629c10e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.097+0000 7f62a305d640 1 --2- 172.21.15.107:0/1946388549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f629c101710 0x7f629c101b90 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.097+0000 7f62a305d640 1 --2- 172.21.15.107:0/1946388549 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f629c107720 0x7f629c107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.097+0000 7f62a305d640 1 -- 172.21.15.107:0/1946388549 >> 172.21.15.107:0/1946388549 conn(0x7f629c078b00 msgr2=0x7f629c078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:31.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.098+0000 7f62a305d640 1 -- 172.21.15.107:0/1946388549 shutdown_connections 2024-04-25T00:16:31.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.098+0000 7f62a305d640 1 -- 172.21.15.107:0/1946388549 wait complete. 2024-04-25T00:16:31.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.098+0000 7f62a305d640 1 Processor -- start 2024-04-25T00:16:31.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.098+0000 7f62a305d640 1 -- start start 2024-04-25T00:16:31.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.098+0000 7f62a305d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f629c101710 0x7f629c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:31.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.098+0000 7f62a305d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f629c1020d0 0x7f629c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:31.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.098+0000 7f62a305d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f629c107720 0x7f629c19c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:31.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.098+0000 7f62a305d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f629c111b30 con 0x7f629c1020d0 2024-04-25T00:16:31.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.098+0000 7f62a305d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f629c1119b0 con 0x7f629c101710 2024-04-25T00:16:31.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.098+0000 7f62a305d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f629c111cb0 con 0x7f629c107720 2024-04-25T00:16:31.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.098+0000 7f629bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f629c1020d0 0x7f629c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:31.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.098+0000 7f629bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f629c1020d0 0x7f629c198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39050/0 (socket says 172.21.15.107:39050) 2024-04-25T00:16:31.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.098+0000 7f629bfff640 1 -- 172.21.15.107:0/322629970 learned_addr learned my addr 172.21.15.107:0/322629970 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:31.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.098+0000 7f62a15fa640 1 --2- 172.21.15.107:0/322629970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f629c107720 0x7f629c19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:31.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.099+0000 7f629bfff640 1 -- 172.21.15.107:0/322629970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f629c107720 msgr2=0x7f629c19c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:31.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.099+0000 7f62a0df9640 1 --2- 172.21.15.107:0/322629970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f629c101710 0x7f629c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:31.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.099+0000 7f629bfff640 1 --2- 172.21.15.107:0/322629970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f629c107720 0x7f629c19c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.099+0000 7f629bfff640 1 -- 172.21.15.107:0/322629970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f629c101710 msgr2=0x7f629c198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:31.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.099+0000 7f629bfff640 1 --2- 172.21.15.107:0/322629970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f629c101710 0x7f629c198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.099+0000 7f629bfff640 1 -- 172.21.15.107:0/322629970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f629c19d040 con 0x7f629c1020d0 2024-04-25T00:16:31.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.099+0000 7f62a0df9640 1 --2- 172.21.15.107:0/322629970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f629c101710 0x7f629c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:31.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.099+0000 7f629bfff640 1 --2- 172.21.15.107:0/322629970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f629c1020d0 0x7f629c198590 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6290009230 tx=0x7f6290004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:31.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.099+0000 7f6299ffb640 1 -- 172.21.15.107:0/322629970 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629003f030 con 0x7f629c1020d0 2024-04-25T00:16:31.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.099+0000 7f6299ffb640 1 -- 172.21.15.107:0/322629970 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6290003ad0 con 0x7f629c1020d0 2024-04-25T00:16:31.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.099+0000 7f62a305d640 1 -- 172.21.15.107:0/322629970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f629c19d2d0 con 0x7f629c1020d0 2024-04-25T00:16:31.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.099+0000 7f6299ffb640 1 -- 172.21.15.107:0/322629970 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6290035070 con 0x7f629c1020d0 2024-04-25T00:16:31.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.099+0000 7f62a305d640 1 -- 172.21.15.107:0/322629970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f629c1a4bc0 con 0x7f629c1020d0 2024-04-25T00:16:31.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.100+0000 7f62a305d640 1 -- 172.21.15.107:0/322629970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f629c0689b0 con 0x7f629c1020d0 2024-04-25T00:16:31.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.101+0000 7f6299ffb640 1 -- 172.21.15.107:0/322629970 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7f6290042030 con 0x7f629c1020d0 2024-04-25T00:16:31.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.102+0000 7f6299ffb640 1 --2- 172.21.15.107:0/322629970 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f62840a2e20 0x7f62840a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:31.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.102+0000 7f62a0df9640 1 --2- 172.21.15.107:0/322629970 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f62840a2e20 0x7f62840a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:31.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.102+0000 7f6299ffb640 1 -- 172.21.15.107:0/322629970 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f62900eff50 con 0x7f629c1020d0 2024-04-25T00:16:31.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.102+0000 7f62a0df9640 1 --2- 172.21.15.107:0/322629970 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f62840a2e20 0x7f62840a52e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f628c005fd0 tx=0x7f628c0094d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:31.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.107+0000 7f6299ffb640 1 -- 172.21.15.107:0/322629970 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f62900bc680 con 0x7f629c1020d0 2024-04-25T00:16:31.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: Adjusting osd_memory_target on smithi170 to 176.2M 2024-04-25T00:16:31.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: Unable to set osd_memory_target on smithi170 to 184790323: error parsing value: Value '184790323' is below minimum 939524096 2024-04-25T00:16:31.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: Adjusting osd_memory_target on smithi195 to 176.2M 2024-04-25T00:16:31.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: Unable to set osd_memory_target on smithi195 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T00:16:31.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: Adjusting osd_memory_target on smithi107 to 176.1M 2024-04-25T00:16:31.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: Unable to set osd_memory_target on smithi107 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-04-25T00:16:31.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: Updating smithi107:/etc/ceph/ceph.conf 2024-04-25T00:16:31.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: Updating smithi170:/etc/ceph/ceph.conf 2024-04-25T00:16:31.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: Updating smithi195:/etc/ceph/ceph.conf 2024-04-25T00:16:31.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: Updating smithi170:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:16:31.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: Updating smithi107:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:16:31.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: Updating smithi195:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:16:31.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2790949279' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:16:31.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:16:31.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:16:31.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: Updating smithi195:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:16:31.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/473102243' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:16:31.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:31.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/461372428' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:16:31.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:31.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:31.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:31.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:31.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:31.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:30 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:31.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.280+0000 7f62a305d640 1 -- 172.21.15.107:0/322629970 --> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f629c109cb0 con 0x7f62840a2e20 2024-04-25T00:16:31.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.290+0000 7f6299ffb640 1 -- 172.21.15.107:0/322629970 <== mgr.15456 v2:172.21.15.107:6800/1021104172 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339377 (secure 0 0 0) 0x7f629c109cb0 con 0x7f62840a2e20 2024-04-25T00:16:31.296 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:16:31.298 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-04-25T00:16:31.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.295+0000 7f62a305d640 1 -- 172.21.15.107:0/322629970 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f62840a2e20 msgr2=0x7f62840a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:31.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.295+0000 7f62a305d640 1 --2- 172.21.15.107:0/322629970 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f62840a2e20 0x7f62840a52e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f628c005fd0 tx=0x7f628c0094d0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.295+0000 7f62a305d640 1 -- 172.21.15.107:0/322629970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f629c1020d0 msgr2=0x7f629c198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:31.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.295+0000 7f62a305d640 1 --2- 172.21.15.107:0/322629970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f629c1020d0 0x7f629c198590 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6290009230 tx=0x7f6290004270 comp rx=0 tx=0).stop 2024-04-25T00:16:31.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.295+0000 7f62a305d640 1 -- 172.21.15.107:0/322629970 shutdown_connections 2024-04-25T00:16:31.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.295+0000 7f62a305d640 1 --2- 172.21.15.107:0/322629970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f629c107720 0x7f629c19c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.295+0000 7f62a305d640 1 --2- 172.21.15.107:0/322629970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f629c1020d0 0x7f629c198590 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.295+0000 7f62a305d640 1 --2- 172.21.15.107:0/322629970 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f62840a2e20 0x7f62840a52e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.295+0000 7f62a305d640 1 --2- 172.21.15.107:0/322629970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f629c101710 0x7f629c198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.295+0000 7f62a305d640 1 -- 172.21.15.107:0/322629970 >> 172.21.15.107:0/322629970 conn(0x7f629c078b00 msgr2=0x7f629c104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:31.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.295+0000 7f62a305d640 1 -- 172.21.15.107:0/322629970 shutdown_connections 2024-04-25T00:16:31.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.295+0000 7f62a305d640 1 -- 172.21.15.107:0/322629970 wait complete. 2024-04-25T00:16:31.306 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-25T00:16:30.246986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190410,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":974,"num_read_kb":651,"num_write":470,"num_write_kb":4381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":395080,"kb_used_data":16596,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124546744,"statfs":{"total":1151940427776,"available":1151535865856,"internally_reserved":0,"allocated":16994304,"data_stored":10101231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.987708"},"pg_stats":[{"pgid":"3.3f","version":"94'8","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328766+0000","last_change":"2024-04-25T00:14:01.835737+0000","last_active":"2024-04-25T00:16:26.328766+0000","last_peered":"2024-04-25T00:16:26.328766+0000","last_clean":"2024-04-25T00:16:26.328766+0000","last_became_active":"2024-04-25T00:14:01.835428+0000","last_became_peered":"2024-04-25T00:14:01.835428+0000","last_unstale":"2024-04-25T00:16:26.328766+0000","last_undegraded":"2024-04-25T00:16:26.328766+0000","last_fullsized":"2024-04-25T00:16:26.328766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:33.974656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261738+0000","last_change":"2024-04-25T00:14:01.837321+0000","last_active":"2024-04-25T00:16:26.261738+0000","last_peered":"2024-04-25T00:16:26.261738+0000","last_clean":"2024-04-25T00:16:26.261738+0000","last_became_active":"2024-04-25T00:14:01.836762+0000","last_became_peered":"2024-04-25T00:14:01.836762+0000","last_unstale":"2024-04-25T00:16:26.261738+0000","last_undegraded":"2024-04-25T00:16:26.261738+0000","last_fullsized":"2024-04-25T00:16:26.261738+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:34:17.342967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261705+0000","last_change":"2024-04-25T00:14:01.834504+0000","last_active":"2024-04-25T00:16:26.261705+0000","last_peered":"2024-04-25T00:16:26.261705+0000","last_clean":"2024-04-25T00:16:26.261705+0000","last_became_active":"2024-04-25T00:14:01.834182+0000","last_became_peered":"2024-04-25T00:14:01.834182+0000","last_unstale":"2024-04-25T00:16:26.261705+0000","last_undegraded":"2024-04-25T00:16:26.261705+0000","last_fullsized":"2024-04-25T00:16:26.261705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:12.991170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":25,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.234081+0000","last_change":"2024-04-25T00:15:02.080211+0000","last_active":"2024-04-25T00:16:26.234081+0000","last_peered":"2024-04-25T00:16:26.234081+0000","last_clean":"2024-04-25T00:16:26.234081+0000","last_became_active":"2024-04-25T00:15:02.079849+0000","last_became_peered":"2024-04-25T00:15:02.079849+0000","last_unstale":"2024-04-25T00:16:26.234081+0000","last_undegraded":"2024-04-25T00:16:26.234081+0000","last_fullsized":"2024-04-25T00:16:26.234081+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:56.804782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912156+0000","last_change":"2024-04-25T00:14:01.834173+0000","last_active":"2024-04-25T00:16:26.912156+0000","last_peered":"2024-04-25T00:16:26.912156+0000","last_clean":"2024-04-25T00:16:26.912156+0000","last_became_active":"2024-04-25T00:14:01.833835+0000","last_became_peered":"2024-04-25T00:14:01.833835+0000","last_unstale":"2024-04-25T00:16:26.912156+0000","last_undegraded":"2024-04-25T00:16:26.912156+0000","last_fullsized":"2024-04-25T00:16:26.912156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:30.046629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.338934+0000","last_change":"2024-04-25T00:14:01.832113+0000","last_active":"2024-04-25T00:16:26.338934+0000","last_peered":"2024-04-25T00:16:26.338934+0000","last_clean":"2024-04-25T00:16:26.338934+0000","last_became_active":"2024-04-25T00:14:01.831800+0000","last_became_peered":"2024-04-25T00:14:01.831800+0000","last_unstale":"2024-04-25T00:16:26.338934+0000","last_undegraded":"2024-04-25T00:16:26.338934+0000","last_fullsized":"2024-04-25T00:16:26.338934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:54.469994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472140+0000","last_change":"2024-04-25T00:14:01.839166+0000","last_active":"2024-04-25T00:16:26.472140+0000","last_peered":"2024-04-25T00:16:26.472140+0000","last_clean":"2024-04-25T00:16:26.472140+0000","last_became_active":"2024-04-25T00:14:01.838851+0000","last_became_peered":"2024-04-25T00:14:01.838851+0000","last_unstale":"2024-04-25T00:16:26.472140+0000","last_undegraded":"2024-04-25T00:16:26.472140+0000","last_fullsized":"2024-04-25T00:16:26.472140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:39:48.125363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":26,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.233734+0000","last_change":"2024-04-25T00:15:02.083386+0000","last_active":"2024-04-25T00:16:26.233734+0000","last_peered":"2024-04-25T00:16:26.233734+0000","last_clean":"2024-04-25T00:16:26.233734+0000","last_became_active":"2024-04-25T00:15:02.083106+0000","last_became_peered":"2024-04-25T00:15:02.083106+0000","last_unstale":"2024-04-25T00:16:26.233734+0000","last_undegraded":"2024-04-25T00:16:26.233734+0000","last_fullsized":"2024-04-25T00:16:26.233734+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:58.543667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.473534+0000","last_change":"2024-04-25T00:14:01.829803+0000","last_active":"2024-04-25T00:16:26.473534+0000","last_peered":"2024-04-25T00:16:26.473534+0000","last_clean":"2024-04-25T00:16:26.473534+0000","last_became_active":"2024-04-25T00:14:01.827761+0000","last_became_peered":"2024-04-25T00:14:01.827761+0000","last_unstale":"2024-04-25T00:16:26.473534+0000","last_undegraded":"2024-04-25T00:16:26.473534+0000","last_fullsized":"2024-04-25T00:16:26.473534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:38.862505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.339417+0000","last_change":"2024-04-25T00:14:01.834269+0000","last_active":"2024-04-25T00:16:26.339417+0000","last_peered":"2024-04-25T00:16:26.339417+0000","last_clean":"2024-04-25T00:16:26.339417+0000","last_became_active":"2024-04-25T00:14:01.833964+0000","last_became_peered":"2024-04-25T00:14:01.833964+0000","last_unstale":"2024-04-25T00:16:26.339417+0000","last_undegraded":"2024-04-25T00:16:26.339417+0000","last_fullsized":"2024-04-25T00:16:26.339417+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:29.501513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472579+0000","last_change":"2024-04-25T00:14:01.839510+0000","last_active":"2024-04-25T00:16:26.472579+0000","last_peered":"2024-04-25T00:16:26.472579+0000","last_clean":"2024-04-25T00:16:26.472579+0000","last_became_active":"2024-04-25T00:14:01.839000+0000","last_became_peered":"2024-04-25T00:14:01.839000+0000","last_unstale":"2024-04-25T00:16:26.472579+0000","last_undegraded":"2024-04-25T00:16:26.472579+0000","last_fullsized":"2024-04-25T00:16:26.472579+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:20.809277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235608+0000","last_change":"2024-04-25T00:14:01.835821+0000","last_active":"2024-04-25T00:16:26.235608+0000","last_peered":"2024-04-25T00:16:26.235608+0000","last_clean":"2024-04-25T00:16:26.235608+0000","last_became_active":"2024-04-25T00:14:01.835525+0000","last_became_peered":"2024-04-25T00:14:01.835525+0000","last_unstale":"2024-04-25T00:16:26.235608+0000","last_undegraded":"2024-04-25T00:16:26.235608+0000","last_fullsized":"2024-04-25T00:16:26.235608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:17.082562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472256+0000","last_change":"2024-04-25T00:14:01.831564+0000","last_active":"2024-04-25T00:16:26.472256+0000","last_peered":"2024-04-25T00:16:26.472256+0000","last_clean":"2024-04-25T00:16:26.472256+0000","last_became_active":"2024-04-25T00:14:01.828539+0000","last_became_peered":"2024-04-25T00:14:01.828539+0000","last_unstale":"2024-04-25T00:16:26.472256+0000","last_undegraded":"2024-04-25T00:16:26.472256+0000","last_fullsized":"2024-04-25T00:16:26.472256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:41:52.098299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261510+0000","last_change":"2024-04-25T00:14:01.835815+0000","last_active":"2024-04-25T00:16:26.261510+0000","last_peered":"2024-04-25T00:16:26.261510+0000","last_clean":"2024-04-25T00:16:26.261510+0000","last_became_active":"2024-04-25T00:14:01.835498+0000","last_became_peered":"2024-04-25T00:14:01.835498+0000","last_unstale":"2024-04-25T00:16:26.261510+0000","last_undegraded":"2024-04-25T00:16:26.261510+0000","last_fullsized":"2024-04-25T00:16:26.261510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:14:02.914149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261465+0000","last_change":"2024-04-25T00:14:01.839542+0000","last_active":"2024-04-25T00:16:26.261465+0000","last_peered":"2024-04-25T00:16:26.261465+0000","last_clean":"2024-04-25T00:16:26.261465+0000","last_became_active":"2024-04-25T00:14:01.839234+0000","last_became_peered":"2024-04-25T00:14:01.839234+0000","last_unstale":"2024-04-25T00:16:26.261465+0000","last_undegraded":"2024-04-25T00:16:26.261465+0000","last_fullsized":"2024-04-25T00:16:26.261465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:00.437311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472718+0000","last_change":"2024-04-25T00:14:01.832265+0000","last_active":"2024-04-25T00:16:26.472718+0000","last_peered":"2024-04-25T00:16:26.472718+0000","last_clean":"2024-04-25T00:16:26.472718+0000","last_became_active":"2024-04-25T00:14:01.831963+0000","last_became_peered":"2024-04-25T00:14:01.831963+0000","last_unstale":"2024-04-25T00:16:26.472718+0000","last_undegraded":"2024-04-25T00:16:26.472718+0000","last_fullsized":"2024-04-25T00:16:26.472718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:55.402265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328586+0000","last_change":"2024-04-25T00:14:01.835444+0000","last_active":"2024-04-25T00:16:26.328586+0000","last_peered":"2024-04-25T00:16:26.328586+0000","last_clean":"2024-04-25T00:16:26.328586+0000","last_became_active":"2024-04-25T00:14:01.835126+0000","last_became_peered":"2024-04-25T00:14:01.835126+0000","last_unstale":"2024-04-25T00:16:26.328586+0000","last_undegraded":"2024-04-25T00:16:26.328586+0000","last_fullsized":"2024-04-25T00:16:26.328586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:48.997924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.234333+0000","last_change":"2024-04-25T00:14:01.834982+0000","last_active":"2024-04-25T00:16:26.234333+0000","last_peered":"2024-04-25T00:16:26.234333+0000","last_clean":"2024-04-25T00:16:26.234333+0000","last_became_active":"2024-04-25T00:14:01.834671+0000","last_became_peered":"2024-04-25T00:14:01.834671+0000","last_unstale":"2024-04-25T00:16:26.234333+0000","last_undegraded":"2024-04-25T00:16:26.234333+0000","last_fullsized":"2024-04-25T00:16:26.234333+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:28.254408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235773+0000","last_change":"2024-04-25T00:14:01.824432+0000","last_active":"2024-04-25T00:16:26.235773+0000","last_peered":"2024-04-25T00:16:26.235773+0000","last_clean":"2024-04-25T00:16:26.235773+0000","last_became_active":"2024-04-25T00:14:01.823963+0000","last_became_peered":"2024-04-25T00:14:01.823963+0000","last_unstale":"2024-04-25T00:16:26.235773+0000","last_undegraded":"2024-04-25T00:16:26.235773+0000","last_fullsized":"2024-04-25T00:16:26.235773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:35.880932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235322+0000","last_change":"2024-04-25T00:14:06.861892+0000","last_active":"2024-04-25T00:16:26.235322+0000","last_peered":"2024-04-25T00:16:26.235322+0000","last_clean":"2024-04-25T00:16:26.235322+0000","last_became_active":"2024-04-25T00:14:06.861470+0000","last_became_peered":"2024-04-25T00:14:06.861470+0000","last_unstale":"2024-04-25T00:16:26.235322+0000","last_undegraded":"2024-04-25T00:16:26.235322+0000","last_fullsized":"2024-04-25T00:16:26.235322+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:06.192954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.234449+0000","last_change":"2024-04-25T00:14:01.412535+0000","last_active":"2024-04-25T00:16:26.234449+0000","last_peered":"2024-04-25T00:16:26.234449+0000","last_clean":"2024-04-25T00:16:26.234449+0000","last_became_active":"2024-04-25T00:14:01.412206+0000","last_became_peered":"2024-04-25T00:14:01.412206+0000","last_unstale":"2024-04-25T00:16:26.234449+0000","last_undegraded":"2024-04-25T00:16:26.234449+0000","last_fullsized":"2024-04-25T00:16:26.234449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:40.893987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.329130+0000","last_change":"2024-04-25T00:14:01.840814+0000","last_active":"2024-04-25T00:16:26.329130+0000","last_peered":"2024-04-25T00:16:26.329130+0000","last_clean":"2024-04-25T00:16:26.329130+0000","last_became_active":"2024-04-25T00:14:01.840179+0000","last_became_peered":"2024-04-25T00:14:01.840179+0000","last_unstale":"2024-04-25T00:16:26.329130+0000","last_undegraded":"2024-04-25T00:16:26.329130+0000","last_fullsized":"2024-04-25T00:16:26.329130+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:27.383918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235154+0000","last_change":"2024-04-25T00:14:06.857877+0000","last_active":"2024-04-25T00:16:26.235154+0000","last_peered":"2024-04-25T00:16:26.235154+0000","last_clean":"2024-04-25T00:16:26.235154+0000","last_became_active":"2024-04-25T00:14:06.857420+0000","last_became_peered":"2024-04-25T00:14:06.857420+0000","last_unstale":"2024-04-25T00:16:26.235154+0000","last_undegraded":"2024-04-25T00:16:26.235154+0000","last_fullsized":"2024-04-25T00:16:26.235154+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:54.424024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.329451+0000","last_change":"2024-04-25T00:14:01.416560+0000","last_active":"2024-04-25T00:16:26.329451+0000","last_peered":"2024-04-25T00:16:26.329451+0000","last_clean":"2024-04-25T00:16:26.329451+0000","last_became_active":"2024-04-25T00:14:01.416246+0000","last_became_peered":"2024-04-25T00:14:01.416246+0000","last_unstale":"2024-04-25T00:16:26.329451+0000","last_undegraded":"2024-04-25T00:16:26.329451+0000","last_fullsized":"2024-04-25T00:16:26.329451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:03:47.938452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261742+0000","last_change":"2024-04-25T00:14:01.832607+0000","last_active":"2024-04-25T00:16:26.261742+0000","last_peered":"2024-04-25T00:16:26.261742+0000","last_clean":"2024-04-25T00:16:26.261742+0000","last_became_active":"2024-04-25T00:14:01.832300+0000","last_became_peered":"2024-04-25T00:14:01.832300+0000","last_unstale":"2024-04-25T00:16:26.261742+0000","last_undegraded":"2024-04-25T00:16:26.261742+0000","last_fullsized":"2024-04-25T00:16:26.261742+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:22.045171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235355+0000","last_change":"2024-04-25T00:14:06.863642+0000","last_active":"2024-04-25T00:16:26.235355+0000","last_peered":"2024-04-25T00:16:26.235355+0000","last_clean":"2024-04-25T00:16:26.235355+0000","last_became_active":"2024-04-25T00:14:06.863284+0000","last_became_peered":"2024-04-25T00:14:06.863284+0000","last_unstale":"2024-04-25T00:16:26.235355+0000","last_undegraded":"2024-04-25T00:16:26.235355+0000","last_fullsized":"2024-04-25T00:16:26.235355+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:20.818625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912488+0000","last_change":"2024-04-25T00:14:01.411874+0000","last_active":"2024-04-25T00:16:26.912488+0000","last_peered":"2024-04-25T00:16:26.912488+0000","last_clean":"2024-04-25T00:16:26.912488+0000","last_became_active":"2024-04-25T00:14:01.409568+0000","last_became_peered":"2024-04-25T00:14:01.409568+0000","last_unstale":"2024-04-25T00:16:26.912488+0000","last_undegraded":"2024-04-25T00:16:26.912488+0000","last_fullsized":"2024-04-25T00:16:26.912488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:22.415631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.473104+0000","last_change":"2024-04-25T00:14:01.822471+0000","last_active":"2024-04-25T00:16:26.473104+0000","last_peered":"2024-04-25T00:16:26.473104+0000","last_clean":"2024-04-25T00:16:26.473104+0000","last_became_active":"2024-04-25T00:14:01.822181+0000","last_became_peered":"2024-04-25T00:14:01.822181+0000","last_unstale":"2024-04-25T00:16:26.473104+0000","last_undegraded":"2024-04-25T00:16:26.473104+0000","last_fullsized":"2024-04-25T00:16:26.473104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:44.857036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235154+0000","last_change":"2024-04-25T00:14:06.856987+0000","last_active":"2024-04-25T00:16:26.235154+0000","last_peered":"2024-04-25T00:16:26.235154+0000","last_clean":"2024-04-25T00:16:26.235154+0000","last_became_active":"2024-04-25T00:14:06.856053+0000","last_became_peered":"2024-04-25T00:14:06.856053+0000","last_unstale":"2024-04-25T00:16:26.235154+0000","last_undegraded":"2024-04-25T00:16:26.235154+0000","last_fullsized":"2024-04-25T00:16:26.235154+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:16.829969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912190+0000","last_change":"2024-04-25T00:14:01.410156+0000","last_active":"2024-04-25T00:16:26.912190+0000","last_peered":"2024-04-25T00:16:26.912190+0000","last_clean":"2024-04-25T00:16:26.912190+0000","last_became_active":"2024-04-25T00:14:01.409809+0000","last_became_peered":"2024-04-25T00:14:01.409809+0000","last_unstale":"2024-04-25T00:16:26.912190+0000","last_undegraded":"2024-04-25T00:16:26.912190+0000","last_fullsized":"2024-04-25T00:16:26.912190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:43.160419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.336265+0000","last_change":"2024-04-25T00:14:01.836090+0000","last_active":"2024-04-25T00:16:26.336265+0000","last_peered":"2024-04-25T00:16:26.336265+0000","last_clean":"2024-04-25T00:16:26.336265+0000","last_became_active":"2024-04-25T00:14:01.835733+0000","last_became_peered":"2024-04-25T00:14:01.835733+0000","last_unstale":"2024-04-25T00:16:26.336265+0000","last_undegraded":"2024-04-25T00:16:26.336265+0000","last_fullsized":"2024-04-25T00:16:26.336265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:02.368893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235589+0000","last_change":"2024-04-25T00:14:06.851496+0000","last_active":"2024-04-25T00:16:26.235589+0000","last_peered":"2024-04-25T00:16:26.235589+0000","last_clean":"2024-04-25T00:16:26.235589+0000","last_became_active":"2024-04-25T00:14:06.851007+0000","last_became_peered":"2024-04-25T00:14:06.851007+0000","last_unstale":"2024-04-25T00:16:26.235589+0000","last_undegraded":"2024-04-25T00:16:26.235589+0000","last_fullsized":"2024-04-25T00:16:26.235589+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:50.543348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'7","reported_seq":167,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.336089+0000","last_change":"2024-04-25T00:14:01.414760+0000","last_active":"2024-04-25T00:16:26.336089+0000","last_peered":"2024-04-25T00:16:26.336089+0000","last_clean":"2024-04-25T00:16:26.336089+0000","last_became_active":"2024-04-25T00:14:01.414482+0000","last_became_peered":"2024-04-25T00:14:01.414482+0000","last_unstale":"2024-04-25T00:16:26.336089+0000","last_undegraded":"2024-04-25T00:16:26.336089+0000","last_fullsized":"2024-04-25T00:16:26.336089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:32.631933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912017+0000","last_change":"2024-04-25T00:14:01.832516+0000","last_active":"2024-04-25T00:16:26.912017+0000","last_peered":"2024-04-25T00:16:26.912017+0000","last_clean":"2024-04-25T00:16:26.912017+0000","last_became_active":"2024-04-25T00:14:01.832106+0000","last_became_peered":"2024-04-25T00:14:01.832106+0000","last_unstale":"2024-04-25T00:16:26.912017+0000","last_undegraded":"2024-04-25T00:16:26.912017+0000","last_fullsized":"2024-04-25T00:16:26.912017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:55.590486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235703+0000","last_change":"2024-04-25T00:14:06.861984+0000","last_active":"2024-04-25T00:16:26.235703+0000","last_peered":"2024-04-25T00:16:26.235703+0000","last_clean":"2024-04-25T00:16:26.235703+0000","last_became_active":"2024-04-25T00:14:06.861418+0000","last_became_peered":"2024-04-25T00:14:06.861418+0000","last_unstale":"2024-04-25T00:16:26.235703+0000","last_undegraded":"2024-04-25T00:16:26.235703+0000","last_fullsized":"2024-04-25T00:16:26.235703+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:27.559704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.338926+0000","last_change":"2024-04-25T00:14:01.407648+0000","last_active":"2024-04-25T00:16:26.338926+0000","last_peered":"2024-04-25T00:16:26.338926+0000","last_clean":"2024-04-25T00:16:26.338926+0000","last_became_active":"2024-04-25T00:14:01.407318+0000","last_became_peered":"2024-04-25T00:14:01.407318+0000","last_unstale":"2024-04-25T00:16:26.338926+0000","last_undegraded":"2024-04-25T00:16:26.338926+0000","last_fullsized":"2024-04-25T00:16:26.338926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:08:10.942814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.336183+0000","last_change":"2024-04-25T00:14:01.833073+0000","last_active":"2024-04-25T00:16:26.336183+0000","last_peered":"2024-04-25T00:16:26.336183+0000","last_clean":"2024-04-25T00:16:26.336183+0000","last_became_active":"2024-04-25T00:14:01.832545+0000","last_became_peered":"2024-04-25T00:14:01.832545+0000","last_unstale":"2024-04-25T00:16:26.336183+0000","last_undegraded":"2024-04-25T00:16:26.336183+0000","last_fullsized":"2024-04-25T00:16:26.336183+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:39.772781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328531+0000","last_change":"2024-04-25T00:14:06.862011+0000","last_active":"2024-04-25T00:16:26.328531+0000","last_peered":"2024-04-25T00:16:26.328531+0000","last_clean":"2024-04-25T00:16:26.328531+0000","last_became_active":"2024-04-25T00:14:06.861544+0000","last_became_peered":"2024-04-25T00:14:06.861544+0000","last_unstale":"2024-04-25T00:16:26.328531+0000","last_undegraded":"2024-04-25T00:16:26.328531+0000","last_fullsized":"2024-04-25T00:16:26.328531+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:37.736266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471570+0000","last_change":"2024-04-25T00:14:01.397747+0000","last_active":"2024-04-25T00:16:26.471570+0000","last_peered":"2024-04-25T00:16:26.471570+0000","last_clean":"2024-04-25T00:16:26.471570+0000","last_became_active":"2024-04-25T00:14:01.397446+0000","last_became_peered":"2024-04-25T00:14:01.397446+0000","last_unstale":"2024-04-25T00:16:26.471570+0000","last_undegraded":"2024-04-25T00:16:26.471570+0000","last_fullsized":"2024-04-25T00:16:26.471570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:27.240034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328414+0000","last_change":"2024-04-25T00:14:01.833128+0000","last_active":"2024-04-25T00:16:26.328414+0000","last_peered":"2024-04-25T00:16:26.328414+0000","last_clean":"2024-04-25T00:16:26.328414+0000","last_became_active":"2024-04-25T00:14:01.832780+0000","last_became_peered":"2024-04-25T00:14:01.832780+0000","last_unstale":"2024-04-25T00:16:26.328414+0000","last_undegraded":"2024-04-25T00:16:26.328414+0000","last_fullsized":"2024-04-25T00:16:26.328414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.023737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261698+0000","last_change":"2024-04-25T00:14:06.860436+0000","last_active":"2024-04-25T00:16:26.261698+0000","last_peered":"2024-04-25T00:16:26.261698+0000","last_clean":"2024-04-25T00:16:26.261698+0000","last_became_active":"2024-04-25T00:14:06.860047+0000","last_became_peered":"2024-04-25T00:14:06.860047+0000","last_unstale":"2024-04-25T00:16:26.261698+0000","last_undegraded":"2024-04-25T00:16:26.261698+0000","last_fullsized":"2024-04-25T00:16:26.261698+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:08:41.246549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.335972+0000","last_change":"2024-04-25T00:14:01.415939+0000","last_active":"2024-04-25T00:16:26.335972+0000","last_peered":"2024-04-25T00:16:26.335972+0000","last_clean":"2024-04-25T00:16:26.335972+0000","last_became_active":"2024-04-25T00:14:01.415513+0000","last_became_peered":"2024-04-25T00:14:01.415513+0000","last_unstale":"2024-04-25T00:16:26.335972+0000","last_undegraded":"2024-04-25T00:16:26.335972+0000","last_fullsized":"2024-04-25T00:16:26.335972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:57:23.588868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.233864+0000","last_change":"2024-04-25T00:14:01.838793+0000","last_active":"2024-04-25T00:16:26.233864+0000","last_peered":"2024-04-25T00:16:26.233864+0000","last_clean":"2024-04-25T00:16:26.233864+0000","last_became_active":"2024-04-25T00:14:01.838453+0000","last_became_peered":"2024-04-25T00:14:01.838453+0000","last_unstale":"2024-04-25T00:16:26.233864+0000","last_undegraded":"2024-04-25T00:16:26.233864+0000","last_fullsized":"2024-04-25T00:16:26.233864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:18.674009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328492+0000","last_change":"2024-04-25T00:14:06.856431+0000","last_active":"2024-04-25T00:16:26.328492+0000","last_peered":"2024-04-25T00:16:26.328492+0000","last_clean":"2024-04-25T00:16:26.328492+0000","last_became_active":"2024-04-25T00:14:06.856047+0000","last_became_peered":"2024-04-25T00:14:06.856047+0000","last_unstale":"2024-04-25T00:16:26.328492+0000","last_undegraded":"2024-04-25T00:16:26.328492+0000","last_fullsized":"2024-04-25T00:16:26.328492+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:20.646673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":160,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.339454+0000","last_change":"2024-04-25T00:14:01.399054+0000","last_active":"2024-04-25T00:16:26.339454+0000","last_peered":"2024-04-25T00:16:26.339454+0000","last_clean":"2024-04-25T00:16:26.339454+0000","last_became_active":"2024-04-25T00:14:01.398711+0000","last_became_peered":"2024-04-25T00:14:01.398711+0000","last_unstale":"2024-04-25T00:16:26.339454+0000","last_undegraded":"2024-04-25T00:16:26.339454+0000","last_fullsized":"2024-04-25T00:16:26.339454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:26.434014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.473480+0000","last_change":"2024-04-25T00:14:01.829132+0000","last_active":"2024-04-25T00:16:26.473480+0000","last_peered":"2024-04-25T00:16:26.473480+0000","last_clean":"2024-04-25T00:16:26.473480+0000","last_became_active":"2024-04-25T00:14:01.828446+0000","last_became_peered":"2024-04-25T00:14:01.828446+0000","last_unstale":"2024-04-25T00:16:26.473480+0000","last_undegraded":"2024-04-25T00:16:26.473480+0000","last_fullsized":"2024-04-25T00:16:26.473480+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:35.788478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.233990+0000","last_change":"2024-04-25T00:14:06.861884+0000","last_active":"2024-04-25T00:16:26.233990+0000","last_peered":"2024-04-25T00:16:26.233990+0000","last_clean":"2024-04-25T00:16:26.233990+0000","last_became_active":"2024-04-25T00:14:06.861413+0000","last_became_peered":"2024-04-25T00:14:06.861413+0000","last_unstale":"2024-04-25T00:16:26.233990+0000","last_undegraded":"2024-04-25T00:16:26.233990+0000","last_fullsized":"2024-04-25T00:16:26.233990+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:53.876308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471856+0000","last_change":"2024-04-25T00:16:03.325276+0000","last_active":"2024-04-25T00:16:26.471856+0000","last_peered":"2024-04-25T00:16:26.471856+0000","last_clean":"2024-04-25T00:16:26.471856+0000","last_became_active":"2024-04-25T00:16:03.322660+0000","last_became_peered":"2024-04-25T00:16:03.322660+0000","last_unstale":"2024-04-25T00:16:26.471856+0000","last_undegraded":"2024-04-25T00:16:26.471856+0000","last_fullsized":"2024-04-25T00:16:26.471856+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:55.882768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261931+0000","last_change":"2024-04-25T00:15:02.372269+0000","last_active":"2024-04-25T00:16:26.261931+0000","last_peered":"2024-04-25T00:16:26.261931+0000","last_clean":"2024-04-25T00:16:26.261931+0000","last_became_active":"2024-04-25T00:15:02.371962+0000","last_became_peered":"2024-04-25T00:15:02.371962+0000","last_unstale":"2024-04-25T00:16:26.261931+0000","last_undegraded":"2024-04-25T00:16:26.261931+0000","last_fullsized":"2024-04-25T00:16:26.261931+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:46.597638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235686+0000","last_change":"2024-04-25T00:15:02.372941+0000","last_active":"2024-04-25T00:16:26.235686+0000","last_peered":"2024-04-25T00:16:26.235686+0000","last_clean":"2024-04-25T00:16:26.235686+0000","last_became_active":"2024-04-25T00:15:02.372504+0000","last_became_peered":"2024-04-25T00:15:02.372504+0000","last_unstale":"2024-04-25T00:16:26.235686+0000","last_undegraded":"2024-04-25T00:16:26.235686+0000","last_fullsized":"2024-04-25T00:16:26.235686+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:58.202708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.336237+0000","last_change":"2024-04-25T00:14:01.401128+0000","last_active":"2024-04-25T00:16:26.336237+0000","last_peered":"2024-04-25T00:16:26.336237+0000","last_clean":"2024-04-25T00:16:26.336237+0000","last_became_active":"2024-04-25T00:14:01.400125+0000","last_became_peered":"2024-04-25T00:14:01.400125+0000","last_unstale":"2024-04-25T00:16:26.336237+0000","last_undegraded":"2024-04-25T00:16:26.336237+0000","last_fullsized":"2024-04-25T00:16:26.336237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:38:43.787855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.473511+0000","last_change":"2024-04-25T00:14:01.830649+0000","last_active":"2024-04-25T00:16:26.473511+0000","last_peered":"2024-04-25T00:16:26.473511+0000","last_clean":"2024-04-25T00:16:26.473511+0000","last_became_active":"2024-04-25T00:14:01.830253+0000","last_became_peered":"2024-04-25T00:14:01.830253+0000","last_unstale":"2024-04-25T00:16:26.473511+0000","last_undegraded":"2024-04-25T00:16:26.473511+0000","last_fullsized":"2024-04-25T00:16:26.473511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:37.474833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.233945+0000","last_change":"2024-04-25T00:14:06.861275+0000","last_active":"2024-04-25T00:16:26.233945+0000","last_peered":"2024-04-25T00:16:26.233945+0000","last_clean":"2024-04-25T00:16:26.233945+0000","last_became_active":"2024-04-25T00:14:06.860891+0000","last_became_peered":"2024-04-25T00:14:06.860891+0000","last_unstale":"2024-04-25T00:16:26.233945+0000","last_undegraded":"2024-04-25T00:16:26.233945+0000","last_fullsized":"2024-04-25T00:16:26.233945+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:38.685837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"97'40","reported_seq":327,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.726401+0000","last_change":"2024-04-25T00:14:01.404706+0000","last_active":"2024-04-25T00:16:26.726401+0000","last_peered":"2024-04-25T00:16:26.726401+0000","last_clean":"2024-04-25T00:16:26.726401+0000","last_became_active":"2024-04-25T00:14:01.404231+0000","last_became_peered":"2024-04-25T00:14:01.404231+0000","last_unstale":"2024-04-25T00:16:26.726401+0000","last_undegraded":"2024-04-25T00:16:26.726401+0000","last_fullsized":"2024-04-25T00:16:26.726401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:36.437491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70068,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":249,"num_read_kb":194,"num_write":41,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912310+0000","last_change":"2024-04-25T00:14:01.828312+0000","last_active":"2024-04-25T00:16:26.912310+0000","last_peered":"2024-04-25T00:16:26.912310+0000","last_clean":"2024-04-25T00:16:26.912310+0000","last_became_active":"2024-04-25T00:14:01.827947+0000","last_became_peered":"2024-04-25T00:14:01.827947+0000","last_unstale":"2024-04-25T00:16:26.912310+0000","last_undegraded":"2024-04-25T00:16:26.912310+0000","last_fullsized":"2024-04-25T00:16:26.912310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.782796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.336543+0000","last_change":"2024-04-25T00:14:06.854379+0000","last_active":"2024-04-25T00:16:26.336543+0000","last_peered":"2024-04-25T00:16:26.336543+0000","last_clean":"2024-04-25T00:16:26.336543+0000","last_became_active":"2024-04-25T00:14:06.853307+0000","last_became_peered":"2024-04-25T00:14:06.853307+0000","last_unstale":"2024-04-25T00:16:26.336543+0000","last_undegraded":"2024-04-25T00:16:26.336543+0000","last_fullsized":"2024-04-25T00:16:26.336543+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:24:11.804189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235501+0000","last_change":"2024-04-25T00:14:01.410395+0000","last_active":"2024-04-25T00:16:26.235501+0000","last_peered":"2024-04-25T00:16:26.235501+0000","last_clean":"2024-04-25T00:16:26.235501+0000","last_became_active":"2024-04-25T00:14:01.410039+0000","last_became_peered":"2024-04-25T00:14:01.410039+0000","last_unstale":"2024-04-25T00:16:26.235501+0000","last_undegraded":"2024-04-25T00:16:26.235501+0000","last_fullsized":"2024-04-25T00:16:26.235501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:29.496519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471990+0000","last_change":"2024-04-25T00:14:01.828403+0000","last_active":"2024-04-25T00:16:26.471990+0000","last_peered":"2024-04-25T00:16:26.471990+0000","last_clean":"2024-04-25T00:16:26.471990+0000","last_became_active":"2024-04-25T00:14:01.828107+0000","last_became_peered":"2024-04-25T00:14:01.828107+0000","last_unstale":"2024-04-25T00:16:26.471990+0000","last_undegraded":"2024-04-25T00:16:26.471990+0000","last_fullsized":"2024-04-25T00:16:26.471990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:29.417639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328638+0000","last_change":"2024-04-25T00:15:02.374105+0000","last_active":"2024-04-25T00:16:26.328638+0000","last_peered":"2024-04-25T00:16:26.328638+0000","last_clean":"2024-04-25T00:16:26.328638+0000","last_became_active":"2024-04-25T00:15:02.373628+0000","last_became_peered":"2024-04-25T00:15:02.373628+0000","last_unstale":"2024-04-25T00:16:26.328638+0000","last_undegraded":"2024-04-25T00:16:26.328638+0000","last_fullsized":"2024-04-25T00:16:26.328638+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:05.542358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.336205+0000","last_change":"2024-04-25T00:14:01.414406+0000","last_active":"2024-04-25T00:16:26.336205+0000","last_peered":"2024-04-25T00:16:26.336205+0000","last_clean":"2024-04-25T00:16:26.336205+0000","last_became_active":"2024-04-25T00:14:01.414085+0000","last_became_peered":"2024-04-25T00:14:01.414085+0000","last_unstale":"2024-04-25T00:16:26.336205+0000","last_undegraded":"2024-04-25T00:16:26.336205+0000","last_fullsized":"2024-04-25T00:16:26.336205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:22.565371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471981+0000","last_change":"2024-04-25T00:14:01.833652+0000","last_active":"2024-04-25T00:16:26.471981+0000","last_peered":"2024-04-25T00:16:26.471981+0000","last_clean":"2024-04-25T00:16:26.471981+0000","last_became_active":"2024-04-25T00:14:01.833307+0000","last_became_peered":"2024-04-25T00:14:01.833307+0000","last_unstale":"2024-04-25T00:16:26.471981+0000","last_undegraded":"2024-04-25T00:16:26.471981+0000","last_fullsized":"2024-04-25T00:16:26.471981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:55:41.261994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.234672+0000","last_change":"2024-04-25T00:14:06.860167+0000","last_active":"2024-04-25T00:16:26.234672+0000","last_peered":"2024-04-25T00:16:26.234672+0000","last_clean":"2024-04-25T00:16:26.234672+0000","last_became_active":"2024-04-25T00:14:06.858651+0000","last_became_peered":"2024-04-25T00:14:06.858651+0000","last_unstale":"2024-04-25T00:16:26.234672+0000","last_undegraded":"2024-04-25T00:16:26.234672+0000","last_fullsized":"2024-04-25T00:16:26.234672+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:11.775434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"97'5","reported_seq":165,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.715411+0000","last_change":"2024-04-25T00:14:01.406267+0000","last_active":"2024-04-25T00:16:26.715411+0000","last_peered":"2024-04-25T00:16:26.715411+0000","last_clean":"2024-04-25T00:16:26.715411+0000","last_became_active":"2024-04-25T00:14:01.405985+0000","last_became_peered":"2024-04-25T00:14:01.405985+0000","last_unstale":"2024-04-25T00:16:26.715411+0000","last_undegraded":"2024-04-25T00:16:26.715411+0000","last_fullsized":"2024-04-25T00:16:26.715411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:17.253805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":4,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472054+0000","last_change":"2024-04-25T00:14:01.838019+0000","last_active":"2024-04-25T00:16:26.472054+0000","last_peered":"2024-04-25T00:16:26.472054+0000","last_clean":"2024-04-25T00:16:26.472054+0000","last_became_active":"2024-04-25T00:14:01.837691+0000","last_became_peered":"2024-04-25T00:14:01.837691+0000","last_unstale":"2024-04-25T00:16:26.472054+0000","last_undegraded":"2024-04-25T00:16:26.472054+0000","last_fullsized":"2024-04-25T00:16:26.472054+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:26.677039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472562+0000","last_change":"2024-04-25T00:14:06.857061+0000","last_active":"2024-04-25T00:16:26.472562+0000","last_peered":"2024-04-25T00:16:26.472562+0000","last_clean":"2024-04-25T00:16:26.472562+0000","last_became_active":"2024-04-25T00:14:06.856678+0000","last_became_peered":"2024-04-25T00:14:06.856678+0000","last_unstale":"2024-04-25T00:16:26.472562+0000","last_undegraded":"2024-04-25T00:16:26.472562+0000","last_fullsized":"2024-04-25T00:16:26.472562+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:36:52.730861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.911965+0000","last_change":"2024-04-25T00:14:01.408385+0000","last_active":"2024-04-25T00:16:26.911965+0000","last_peered":"2024-04-25T00:16:26.911965+0000","last_clean":"2024-04-25T00:16:26.911965+0000","last_became_active":"2024-04-25T00:14:01.408049+0000","last_became_peered":"2024-04-25T00:14:01.408049+0000","last_unstale":"2024-04-25T00:16:26.911965+0000","last_undegraded":"2024-04-25T00:16:26.911965+0000","last_fullsized":"2024-04-25T00:16:26.911965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:19.765467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.335937+0000","last_change":"2024-04-25T00:15:02.372426+0000","last_active":"2024-04-25T00:16:26.335937+0000","last_peered":"2024-04-25T00:16:26.335937+0000","last_clean":"2024-04-25T00:16:26.335937+0000","last_became_active":"2024-04-25T00:15:02.372044+0000","last_became_peered":"2024-04-25T00:15:02.372044+0000","last_unstale":"2024-04-25T00:16:26.335937+0000","last_undegraded":"2024-04-25T00:16:26.335937+0000","last_fullsized":"2024-04-25T00:16:26.335937+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:20.965683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.335834+0000","last_change":"2024-04-25T00:14:06.854192+0000","last_active":"2024-04-25T00:16:26.335834+0000","last_peered":"2024-04-25T00:16:26.335834+0000","last_clean":"2024-04-25T00:16:26.335834+0000","last_became_active":"2024-04-25T00:14:06.853325+0000","last_became_peered":"2024-04-25T00:14:06.853325+0000","last_unstale":"2024-04-25T00:16:26.335834+0000","last_undegraded":"2024-04-25T00:16:26.335834+0000","last_fullsized":"2024-04-25T00:16:26.335834+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:15.308169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912441+0000","last_change":"2024-04-25T00:14:01.407247+0000","last_active":"2024-04-25T00:16:26.912441+0000","last_peered":"2024-04-25T00:16:26.912441+0000","last_clean":"2024-04-25T00:16:26.912441+0000","last_became_active":"2024-04-25T00:14:01.406891+0000","last_became_peered":"2024-04-25T00:14:01.406891+0000","last_unstale":"2024-04-25T00:16:26.912441+0000","last_undegraded":"2024-04-25T00:16:26.912441+0000","last_fullsized":"2024-04-25T00:16:26.912441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:43.294562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328645+0000","last_change":"2024-04-25T00:14:01.829364+0000","last_active":"2024-04-25T00:16:26.328645+0000","last_peered":"2024-04-25T00:16:26.328645+0000","last_clean":"2024-04-25T00:16:26.328645+0000","last_became_active":"2024-04-25T00:14:01.829087+0000","last_became_peered":"2024-04-25T00:14:01.829087+0000","last_unstale":"2024-04-25T00:16:26.328645+0000","last_undegraded":"2024-04-25T00:16:26.328645+0000","last_fullsized":"2024-04-25T00:16:26.328645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:17.651815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.911928+0000","last_change":"2024-04-25T00:14:07.323442+0000","last_active":"2024-04-25T00:16:26.911928+0000","last_peered":"2024-04-25T00:16:26.911928+0000","last_clean":"2024-04-25T00:16:26.911928+0000","last_became_active":"2024-04-25T00:14:07.322943+0000","last_became_peered":"2024-04-25T00:14:07.322943+0000","last_unstale":"2024-04-25T00:16:26.911928+0000","last_undegraded":"2024-04-25T00:16:26.911928+0000","last_fullsized":"2024-04-25T00:16:26.911928+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:49.166240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.234040+0000","last_change":"2024-04-25T00:14:01.410936+0000","last_active":"2024-04-25T00:16:26.234040+0000","last_peered":"2024-04-25T00:16:26.234040+0000","last_clean":"2024-04-25T00:16:26.234040+0000","last_became_active":"2024-04-25T00:14:01.410321+0000","last_became_peered":"2024-04-25T00:14:01.410321+0000","last_unstale":"2024-04-25T00:16:26.234040+0000","last_undegraded":"2024-04-25T00:16:26.234040+0000","last_fullsized":"2024-04-25T00:16:26.234040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:44.603199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"94'4","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.339205+0000","last_change":"2024-04-25T00:14:01.830842+0000","last_active":"2024-04-25T00:16:26.339205+0000","last_peered":"2024-04-25T00:16:26.339205+0000","last_clean":"2024-04-25T00:16:26.339205+0000","last_became_active":"2024-04-25T00:14:01.830543+0000","last_became_peered":"2024-04-25T00:14:01.830543+0000","last_unstale":"2024-04-25T00:16:26.339205+0000","last_undegraded":"2024-04-25T00:16:26.339205+0000","last_fullsized":"2024-04-25T00:16:26.339205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:59.897133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471933+0000","last_change":"2024-04-25T00:14:06.856841+0000","last_active":"2024-04-25T00:16:26.471933+0000","last_peered":"2024-04-25T00:16:26.471933+0000","last_clean":"2024-04-25T00:16:26.471933+0000","last_became_active":"2024-04-25T00:14:06.856416+0000","last_became_peered":"2024-04-25T00:14:06.856416+0000","last_unstale":"2024-04-25T00:16:26.471933+0000","last_undegraded":"2024-04-25T00:16:26.471933+0000","last_fullsized":"2024-04-25T00:16:26.471933+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:27.181175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471815+0000","last_change":"2024-04-25T00:14:01.409627+0000","last_active":"2024-04-25T00:16:26.471815+0000","last_peered":"2024-04-25T00:16:26.471815+0000","last_clean":"2024-04-25T00:16:26.471815+0000","last_became_active":"2024-04-25T00:14:01.409304+0000","last_became_peered":"2024-04-25T00:14:01.409304+0000","last_unstale":"2024-04-25T00:16:26.471815+0000","last_undegraded":"2024-04-25T00:16:26.471815+0000","last_fullsized":"2024-04-25T00:16:26.471815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:27.242004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.339062+0000","last_change":"2024-04-25T00:14:01.829544+0000","last_active":"2024-04-25T00:16:26.339062+0000","last_peered":"2024-04-25T00:16:26.339062+0000","last_clean":"2024-04-25T00:16:26.339062+0000","last_became_active":"2024-04-25T00:14:01.829233+0000","last_became_peered":"2024-04-25T00:14:01.829233+0000","last_unstale":"2024-04-25T00:16:26.339062+0000","last_undegraded":"2024-04-25T00:16:26.339062+0000","last_fullsized":"2024-04-25T00:16:26.339062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:16.102041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472694+0000","last_change":"2024-04-25T00:14:07.323213+0000","last_active":"2024-04-25T00:16:26.472694+0000","last_peered":"2024-04-25T00:16:26.472694+0000","last_clean":"2024-04-25T00:16:26.472694+0000","last_became_active":"2024-04-25T00:14:07.322763+0000","last_became_peered":"2024-04-25T00:14:07.322763+0000","last_unstale":"2024-04-25T00:16:26.472694+0000","last_undegraded":"2024-04-25T00:16:26.472694+0000","last_fullsized":"2024-04-25T00:16:26.472694+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:45.269314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":25,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.233734+0000","last_change":"2024-04-25T00:15:02.079745+0000","last_active":"2024-04-25T00:16:26.233734+0000","last_peered":"2024-04-25T00:16:26.233734+0000","last_clean":"2024-04-25T00:16:26.233734+0000","last_became_active":"2024-04-25T00:15:02.079405+0000","last_became_peered":"2024-04-25T00:15:02.079405+0000","last_unstale":"2024-04-25T00:16:26.233734+0000","last_undegraded":"2024-04-25T00:16:26.233734+0000","last_fullsized":"2024-04-25T00:16:26.233734+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:38.197048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"97'5","reported_seq":162,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.473772+0000","last_change":"2024-04-25T00:14:01.397066+0000","last_active":"2024-04-25T00:16:26.473772+0000","last_peered":"2024-04-25T00:16:26.473772+0000","last_clean":"2024-04-25T00:16:26.473772+0000","last_became_active":"2024-04-25T00:14:01.396728+0000","last_became_peered":"2024-04-25T00:14:01.396728+0000","last_unstale":"2024-04-25T00:16:26.473772+0000","last_undegraded":"2024-04-25T00:16:26.473772+0000","last_fullsized":"2024-04-25T00:16:26.473772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:28.334329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"92'1","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472372+0000","last_change":"2024-04-25T00:14:06.853039+0000","last_active":"2024-04-25T00:16:26.472372+0000","last_peered":"2024-04-25T00:16:26.472372+0000","last_clean":"2024-04-25T00:16:26.472372+0000","last_became_active":"2024-04-25T00:14:06.852568+0000","last_became_peered":"2024-04-25T00:14:06.852568+0000","last_unstale":"2024-04-25T00:16:26.472372+0000","last_undegraded":"2024-04-25T00:16:26.472372+0000","last_fullsized":"2024-04-25T00:16:26.472372+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:59.604160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328701+0000","last_change":"2024-04-25T00:14:01.840315+0000","last_active":"2024-04-25T00:16:26.328701+0000","last_peered":"2024-04-25T00:16:26.328701+0000","last_clean":"2024-04-25T00:16:26.328701+0000","last_became_active":"2024-04-25T00:14:01.839537+0000","last_became_peered":"2024-04-25T00:14:01.839537+0000","last_unstale":"2024-04-25T00:16:26.328701+0000","last_undegraded":"2024-04-25T00:16:26.328701+0000","last_fullsized":"2024-04-25T00:16:26.328701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:34.126164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235345+0000","last_change":"2024-04-25T00:14:01.412493+0000","last_active":"2024-04-25T00:16:26.235345+0000","last_peered":"2024-04-25T00:16:26.235345+0000","last_clean":"2024-04-25T00:16:26.235345+0000","last_became_active":"2024-04-25T00:14:01.412133+0000","last_became_peered":"2024-04-25T00:14:01.412133+0000","last_unstale":"2024-04-25T00:16:26.235345+0000","last_undegraded":"2024-04-25T00:16:26.235345+0000","last_fullsized":"2024-04-25T00:16:26.235345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:44.933728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":371,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471716+0000","last_change":"2024-04-25T00:12:49.169294+0000","last_active":"2024-04-25T00:16:26.471716+0000","last_peered":"2024-04-25T00:16:26.471716+0000","last_clean":"2024-04-25T00:16:26.471716+0000","last_became_active":"2024-04-25T00:12:49.168783+0000","last_became_peered":"2024-04-25T00:12:49.168783+0000","last_unstale":"2024-04-25T00:16:26.471716+0000","last_undegraded":"2024-04-25T00:16:26.471716+0000","last_fullsized":"2024-04-25T00:16:26.471716+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:25.202280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:25.202280+0000","last_clean_scrub_stamp":"2024-04-25T00:09:25.202280+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:04.713986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912255+0000","last_change":"2024-04-25T00:15:02.372570+0000","last_active":"2024-04-25T00:16:26.912255+0000","last_peered":"2024-04-25T00:16:26.912255+0000","last_clean":"2024-04-25T00:16:26.912255+0000","last_became_active":"2024-04-25T00:15:02.372066+0000","last_became_peered":"2024-04-25T00:15:02.372066+0000","last_unstale":"2024-04-25T00:16:26.912255+0000","last_undegraded":"2024-04-25T00:16:26.912255+0000","last_fullsized":"2024-04-25T00:16:26.912255+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:59.470049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,6,10],"acting":[8,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471452+0000","last_change":"2024-04-25T00:14:01.826418+0000","last_active":"2024-04-25T00:16:26.471452+0000","last_peered":"2024-04-25T00:16:26.471452+0000","last_clean":"2024-04-25T00:16:26.471452+0000","last_became_active":"2024-04-25T00:14:01.825783+0000","last_became_peered":"2024-04-25T00:14:01.825783+0000","last_unstale":"2024-04-25T00:16:26.471452+0000","last_undegraded":"2024-04-25T00:16:26.471452+0000","last_fullsized":"2024-04-25T00:16:26.471452+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:05.981359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471893+0000","last_change":"2024-04-25T00:14:01.417022+0000","last_active":"2024-04-25T00:16:26.471893+0000","last_peered":"2024-04-25T00:16:26.471893+0000","last_clean":"2024-04-25T00:16:26.471893+0000","last_became_active":"2024-04-25T00:14:01.416804+0000","last_became_peered":"2024-04-25T00:14:01.416804+0000","last_unstale":"2024-04-25T00:16:26.471893+0000","last_undegraded":"2024-04-25T00:16:26.471893+0000","last_fullsized":"2024-04-25T00:16:26.471893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:31.561242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328438+0000","last_change":"2024-04-25T00:14:06.856500+0000","last_active":"2024-04-25T00:16:26.328438+0000","last_peered":"2024-04-25T00:16:26.328438+0000","last_clean":"2024-04-25T00:16:26.328438+0000","last_became_active":"2024-04-25T00:14:06.856077+0000","last_became_peered":"2024-04-25T00:14:06.856077+0000","last_unstale":"2024-04-25T00:16:26.328438+0000","last_undegraded":"2024-04-25T00:16:26.328438+0000","last_fullsized":"2024-04-25T00:16:26.328438+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:10:04.129533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912513+0000","last_change":"2024-04-25T00:14:01.414599+0000","last_active":"2024-04-25T00:16:26.912513+0000","last_peered":"2024-04-25T00:16:26.912513+0000","last_clean":"2024-04-25T00:16:26.912513+0000","last_became_active":"2024-04-25T00:14:01.413796+0000","last_became_peered":"2024-04-25T00:14:01.413796+0000","last_unstale":"2024-04-25T00:16:26.912513+0000","last_undegraded":"2024-04-25T00:16:26.912513+0000","last_fullsized":"2024-04-25T00:16:26.912513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:44.981848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472240+0000","last_change":"2024-04-25T00:14:01.827718+0000","last_active":"2024-04-25T00:16:26.472240+0000","last_peered":"2024-04-25T00:16:26.472240+0000","last_clean":"2024-04-25T00:16:26.472240+0000","last_became_active":"2024-04-25T00:14:01.827410+0000","last_became_peered":"2024-04-25T00:14:01.827410+0000","last_unstale":"2024-04-25T00:16:26.472240+0000","last_undegraded":"2024-04-25T00:16:26.472240+0000","last_fullsized":"2024-04-25T00:16:26.472240+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:32.428137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328411+0000","last_change":"2024-04-25T00:14:07.323304+0000","last_active":"2024-04-25T00:16:26.328411+0000","last_peered":"2024-04-25T00:16:26.328411+0000","last_clean":"2024-04-25T00:16:26.328411+0000","last_became_active":"2024-04-25T00:14:07.322852+0000","last_became_peered":"2024-04-25T00:14:07.322852+0000","last_unstale":"2024-04-25T00:16:26.328411+0000","last_undegraded":"2024-04-25T00:16:26.328411+0000","last_fullsized":"2024-04-25T00:16:26.328411+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:35.173934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'45","reported_seq":118,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.439157+0000","last_change":"2024-04-25T00:16:03.058662+0000","last_active":"2024-04-25T00:16:26.439157+0000","last_peered":"2024-04-25T00:16:26.439157+0000","last_clean":"2024-04-25T00:16:26.439157+0000","last_became_active":"2024-04-25T00:16:03.054623+0000","last_became_peered":"2024-04-25T00:16:03.054623+0000","last_unstale":"2024-04-25T00:16:26.439157+0000","last_undegraded":"2024-04-25T00:16:26.439157+0000","last_fullsized":"2024-04-25T00:16:26.439157+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:51.610854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.329382+0000","last_change":"2024-04-25T00:14:01.834372+0000","last_active":"2024-04-25T00:16:26.329382+0000","last_peered":"2024-04-25T00:16:26.329382+0000","last_clean":"2024-04-25T00:16:26.329382+0000","last_became_active":"2024-04-25T00:14:01.834062+0000","last_became_peered":"2024-04-25T00:14:01.834062+0000","last_unstale":"2024-04-25T00:16:26.329382+0000","last_undegraded":"2024-04-25T00:16:26.329382+0000","last_fullsized":"2024-04-25T00:16:26.329382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:05.580604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235191+0000","last_change":"2024-04-25T00:14:06.852303+0000","last_active":"2024-04-25T00:16:26.235191+0000","last_peered":"2024-04-25T00:16:26.235191+0000","last_clean":"2024-04-25T00:16:26.235191+0000","last_became_active":"2024-04-25T00:14:06.850957+0000","last_became_peered":"2024-04-25T00:14:06.850957+0000","last_unstale":"2024-04-25T00:16:26.235191+0000","last_undegraded":"2024-04-25T00:16:26.235191+0000","last_fullsized":"2024-04-25T00:16:26.235191+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:47.208880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235559+0000","last_change":"2024-04-25T00:14:01.405538+0000","last_active":"2024-04-25T00:16:26.235559+0000","last_peered":"2024-04-25T00:16:26.235559+0000","last_clean":"2024-04-25T00:16:26.235559+0000","last_became_active":"2024-04-25T00:14:01.405155+0000","last_became_peered":"2024-04-25T00:14:01.405155+0000","last_unstale":"2024-04-25T00:16:26.235559+0000","last_undegraded":"2024-04-25T00:16:26.235559+0000","last_fullsized":"2024-04-25T00:16:26.235559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:20.864753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261840+0000","last_change":"2024-04-25T00:14:01.837058+0000","last_active":"2024-04-25T00:16:26.261840+0000","last_peered":"2024-04-25T00:16:26.261840+0000","last_clean":"2024-04-25T00:16:26.261840+0000","last_became_active":"2024-04-25T00:14:01.835838+0000","last_became_peered":"2024-04-25T00:14:01.835838+0000","last_unstale":"2024-04-25T00:16:26.261840+0000","last_undegraded":"2024-04-25T00:16:26.261840+0000","last_fullsized":"2024-04-25T00:16:26.261840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:21.696448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235583+0000","last_change":"2024-04-25T00:14:07.323870+0000","last_active":"2024-04-25T00:16:26.235583+0000","last_peered":"2024-04-25T00:16:26.235583+0000","last_clean":"2024-04-25T00:16:26.235583+0000","last_became_active":"2024-04-25T00:14:07.323356+0000","last_became_peered":"2024-04-25T00:14:07.323356+0000","last_unstale":"2024-04-25T00:16:26.235583+0000","last_undegraded":"2024-04-25T00:16:26.235583+0000","last_fullsized":"2024-04-25T00:16:26.235583+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:35.160618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472790+0000","last_change":"2024-04-25T00:14:01.409453+0000","last_active":"2024-04-25T00:16:26.472790+0000","last_peered":"2024-04-25T00:16:26.472790+0000","last_clean":"2024-04-25T00:16:26.472790+0000","last_became_active":"2024-04-25T00:14:01.409164+0000","last_became_peered":"2024-04-25T00:14:01.409164+0000","last_unstale":"2024-04-25T00:16:26.472790+0000","last_undegraded":"2024-04-25T00:16:26.472790+0000","last_fullsized":"2024-04-25T00:16:26.472790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:23.266090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.911889+0000","last_change":"2024-04-25T00:14:01.826790+0000","last_active":"2024-04-25T00:16:26.911889+0000","last_peered":"2024-04-25T00:16:26.911889+0000","last_clean":"2024-04-25T00:16:26.911889+0000","last_became_active":"2024-04-25T00:14:01.826457+0000","last_became_peered":"2024-04-25T00:14:01.826457+0000","last_unstale":"2024-04-25T00:16:26.911889+0000","last_undegraded":"2024-04-25T00:16:26.911889+0000","last_fullsized":"2024-04-25T00:16:26.911889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:15.356448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.234393+0000","last_change":"2024-04-25T00:14:06.863280+0000","last_active":"2024-04-25T00:16:26.234393+0000","last_peered":"2024-04-25T00:16:26.234393+0000","last_clean":"2024-04-25T00:16:26.234393+0000","last_became_active":"2024-04-25T00:14:06.862725+0000","last_became_peered":"2024-04-25T00:14:06.862725+0000","last_unstale":"2024-04-25T00:16:26.234393+0000","last_undegraded":"2024-04-25T00:16:26.234393+0000","last_fullsized":"2024-04-25T00:16:26.234393+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:12.791744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328567+0000","last_change":"2024-04-25T00:14:01.414856+0000","last_active":"2024-04-25T00:16:26.328567+0000","last_peered":"2024-04-25T00:16:26.328567+0000","last_clean":"2024-04-25T00:16:26.328567+0000","last_became_active":"2024-04-25T00:14:01.414533+0000","last_became_peered":"2024-04-25T00:14:01.414533+0000","last_unstale":"2024-04-25T00:16:26.328567+0000","last_undegraded":"2024-04-25T00:16:26.328567+0000","last_fullsized":"2024-04-25T00:16:26.328567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:01.090539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328304+0000","last_change":"2024-04-25T00:14:01.831452+0000","last_active":"2024-04-25T00:16:26.328304+0000","last_peered":"2024-04-25T00:16:26.328304+0000","last_clean":"2024-04-25T00:16:26.328304+0000","last_became_active":"2024-04-25T00:14:01.831151+0000","last_became_peered":"2024-04-25T00:14:01.831151+0000","last_unstale":"2024-04-25T00:16:26.328304+0000","last_undegraded":"2024-04-25T00:16:26.328304+0000","last_fullsized":"2024-04-25T00:16:26.328304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:53:07.659508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"92'1","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235539+0000","last_change":"2024-04-25T00:14:06.857601+0000","last_active":"2024-04-25T00:16:26.235539+0000","last_peered":"2024-04-25T00:16:26.235539+0000","last_clean":"2024-04-25T00:16:26.235539+0000","last_became_active":"2024-04-25T00:14:06.856685+0000","last_became_peered":"2024-04-25T00:14:06.856685+0000","last_unstale":"2024-04-25T00:16:26.235539+0000","last_undegraded":"2024-04-25T00:16:26.235539+0000","last_fullsized":"2024-04-25T00:16:26.235539+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:20.250053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328326+0000","last_change":"2024-04-25T00:16:03.060574+0000","last_active":"2024-04-25T00:16:26.328326+0000","last_peered":"2024-04-25T00:16:26.328326+0000","last_clean":"2024-04-25T00:16:26.328326+0000","last_became_active":"2024-04-25T00:16:03.055792+0000","last_became_peered":"2024-04-25T00:16:03.055792+0000","last_unstale":"2024-04-25T00:16:26.328326+0000","last_undegraded":"2024-04-25T00:16:26.328326+0000","last_fullsized":"2024-04-25T00:16:26.328326+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:28.112894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328959+0000","last_change":"2024-04-25T00:14:01.832342+0000","last_active":"2024-04-25T00:16:26.328959+0000","last_peered":"2024-04-25T00:16:26.328959+0000","last_clean":"2024-04-25T00:16:26.328959+0000","last_became_active":"2024-04-25T00:14:01.831997+0000","last_became_peered":"2024-04-25T00:14:01.831997+0000","last_unstale":"2024-04-25T00:16:26.328959+0000","last_undegraded":"2024-04-25T00:16:26.328959+0000","last_fullsized":"2024-04-25T00:16:26.328959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:12.335884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235429+0000","last_change":"2024-04-25T00:14:07.323628+0000","last_active":"2024-04-25T00:16:26.235429+0000","last_peered":"2024-04-25T00:16:26.235429+0000","last_clean":"2024-04-25T00:16:26.235429+0000","last_became_active":"2024-04-25T00:14:07.322996+0000","last_became_peered":"2024-04-25T00:14:07.322996+0000","last_unstale":"2024-04-25T00:16:26.235429+0000","last_undegraded":"2024-04-25T00:16:26.235429+0000","last_fullsized":"2024-04-25T00:16:26.235429+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:32.803216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912115+0000","last_change":"2024-04-25T00:14:01.407715+0000","last_active":"2024-04-25T00:16:26.912115+0000","last_peered":"2024-04-25T00:16:26.912115+0000","last_clean":"2024-04-25T00:16:26.912115+0000","last_became_active":"2024-04-25T00:14:01.407439+0000","last_became_peered":"2024-04-25T00:14:01.407439+0000","last_unstale":"2024-04-25T00:16:26.912115+0000","last_undegraded":"2024-04-25T00:16:26.912115+0000","last_fullsized":"2024-04-25T00:16:26.912115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:33.550379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471832+0000","last_change":"2024-04-25T00:14:01.837733+0000","last_active":"2024-04-25T00:16:26.471832+0000","last_peered":"2024-04-25T00:16:26.471832+0000","last_clean":"2024-04-25T00:16:26.471832+0000","last_became_active":"2024-04-25T00:14:01.837395+0000","last_became_peered":"2024-04-25T00:14:01.837395+0000","last_unstale":"2024-04-25T00:16:26.471832+0000","last_undegraded":"2024-04-25T00:16:26.471832+0000","last_fullsized":"2024-04-25T00:16:26.471832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:50.186834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472922+0000","last_change":"2024-04-25T00:14:06.848700+0000","last_active":"2024-04-25T00:16:26.472922+0000","last_peered":"2024-04-25T00:16:26.472922+0000","last_clean":"2024-04-25T00:16:26.472922+0000","last_became_active":"2024-04-25T00:14:06.848252+0000","last_became_peered":"2024-04-25T00:14:06.848252+0000","last_unstale":"2024-04-25T00:16:26.472922+0000","last_undegraded":"2024-04-25T00:16:26.472922+0000","last_fullsized":"2024-04-25T00:16:26.472922+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:07.653668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235943+0000","last_change":"2024-04-25T00:14:01.405785+0000","last_active":"2024-04-25T00:16:26.235943+0000","last_peered":"2024-04-25T00:16:26.235943+0000","last_clean":"2024-04-25T00:16:26.235943+0000","last_became_active":"2024-04-25T00:14:01.405475+0000","last_became_peered":"2024-04-25T00:14:01.405475+0000","last_unstale":"2024-04-25T00:16:26.235943+0000","last_undegraded":"2024-04-25T00:16:26.235943+0000","last_fullsized":"2024-04-25T00:16:26.235943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:32.487795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472556+0000","last_change":"2024-04-25T00:14:01.832393+0000","last_active":"2024-04-25T00:16:26.472556+0000","last_peered":"2024-04-25T00:16:26.472556+0000","last_clean":"2024-04-25T00:16:26.472556+0000","last_became_active":"2024-04-25T00:14:01.832096+0000","last_became_peered":"2024-04-25T00:14:01.832096+0000","last_unstale":"2024-04-25T00:16:26.472556+0000","last_undegraded":"2024-04-25T00:16:26.472556+0000","last_fullsized":"2024-04-25T00:16:26.472556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:41.749627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261459+0000","last_change":"2024-04-25T00:14:06.855224+0000","last_active":"2024-04-25T00:16:26.261459+0000","last_peered":"2024-04-25T00:16:26.261459+0000","last_clean":"2024-04-25T00:16:26.261459+0000","last_became_active":"2024-04-25T00:14:06.854847+0000","last_became_peered":"2024-04-25T00:14:06.854847+0000","last_unstale":"2024-04-25T00:16:26.261459+0000","last_undegraded":"2024-04-25T00:16:26.261459+0000","last_fullsized":"2024-04-25T00:16:26.261459+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:13.520448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"97'39","reported_seq":85,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.433764+0000","last_change":"2024-04-25T00:14:01.401145+0000","last_active":"2024-04-25T00:16:26.433764+0000","last_peered":"2024-04-25T00:16:26.433764+0000","last_clean":"2024-04-25T00:16:26.433764+0000","last_became_active":"2024-04-25T00:14:01.400807+0000","last_became_peered":"2024-04-25T00:14:01.400807+0000","last_unstale":"2024-04-25T00:16:26.433764+0000","last_undegraded":"2024-04-25T00:16:26.433764+0000","last_fullsized":"2024-04-25T00:16:26.433764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:52:04.674677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"94'6","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472317+0000","last_change":"2024-04-25T00:14:01.838033+0000","last_active":"2024-04-25T00:16:26.472317+0000","last_peered":"2024-04-25T00:16:26.472317+0000","last_clean":"2024-04-25T00:16:26.472317+0000","last_became_active":"2024-04-25T00:14:01.837657+0000","last_became_peered":"2024-04-25T00:14:01.837657+0000","last_unstale":"2024-04-25T00:16:26.472317+0000","last_undegraded":"2024-04-25T00:16:26.472317+0000","last_fullsized":"2024-04-25T00:16:26.472317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:51.895086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.335703+0000","last_change":"2024-04-25T00:14:06.854885+0000","last_active":"2024-04-25T00:16:26.335703+0000","last_peered":"2024-04-25T00:16:26.335703+0000","last_clean":"2024-04-25T00:16:26.335703+0000","last_became_active":"2024-04-25T00:14:06.854200+0000","last_became_peered":"2024-04-25T00:14:06.854200+0000","last_unstale":"2024-04-25T00:16:26.335703+0000","last_undegraded":"2024-04-25T00:16:26.335703+0000","last_fullsized":"2024-04-25T00:16:26.335703+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:43.627742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472665+0000","last_change":"2024-04-25T00:14:01.403789+0000","last_active":"2024-04-25T00:16:26.472665+0000","last_peered":"2024-04-25T00:16:26.472665+0000","last_clean":"2024-04-25T00:16:26.472665+0000","last_became_active":"2024-04-25T00:14:01.403464+0000","last_became_peered":"2024-04-25T00:14:01.403464+0000","last_unstale":"2024-04-25T00:16:26.472665+0000","last_undegraded":"2024-04-25T00:16:26.472665+0000","last_fullsized":"2024-04-25T00:16:26.472665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:48.276536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472074+0000","last_change":"2024-04-25T00:14:01.839988+0000","last_active":"2024-04-25T00:16:26.472074+0000","last_peered":"2024-04-25T00:16:26.472074+0000","last_clean":"2024-04-25T00:16:26.472074+0000","last_became_active":"2024-04-25T00:14:01.839516+0000","last_became_peered":"2024-04-25T00:14:01.839516+0000","last_unstale":"2024-04-25T00:16:26.472074+0000","last_undegraded":"2024-04-25T00:16:26.472074+0000","last_fullsized":"2024-04-25T00:16:26.472074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:05.875127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328368+0000","last_change":"2024-04-25T00:14:01.828638+0000","last_active":"2024-04-25T00:16:26.328368+0000","last_peered":"2024-04-25T00:16:26.328368+0000","last_clean":"2024-04-25T00:16:26.328368+0000","last_became_active":"2024-04-25T00:14:01.828266+0000","last_became_peered":"2024-04-25T00:14:01.828266+0000","last_unstale":"2024-04-25T00:16:26.328368+0000","last_undegraded":"2024-04-25T00:16:26.328368+0000","last_fullsized":"2024-04-25T00:16:26.328368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:34:22.619522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235391+0000","last_change":"2024-04-25T00:14:01.828199+0000","last_active":"2024-04-25T00:16:26.235391+0000","last_peered":"2024-04-25T00:16:26.235391+0000","last_clean":"2024-04-25T00:16:26.235391+0000","last_became_active":"2024-04-25T00:14:01.827579+0000","last_became_peered":"2024-04-25T00:14:01.827579+0000","last_unstale":"2024-04-25T00:16:26.235391+0000","last_undegraded":"2024-04-25T00:16:26.235391+0000","last_fullsized":"2024-04-25T00:16:26.235391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:00.708928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235265+0000","last_change":"2024-04-25T00:14:01.825620+0000","last_active":"2024-04-25T00:16:26.235265+0000","last_peered":"2024-04-25T00:16:26.235265+0000","last_clean":"2024-04-25T00:16:26.235265+0000","last_became_active":"2024-04-25T00:14:01.825310+0000","last_became_peered":"2024-04-25T00:14:01.825310+0000","last_unstale":"2024-04-25T00:16:26.235265+0000","last_undegraded":"2024-04-25T00:16:26.235265+0000","last_fullsized":"2024-04-25T00:16:26.235265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:35.075355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261644+0000","last_change":"2024-04-25T00:14:01.836537+0000","last_active":"2024-04-25T00:16:26.261644+0000","last_peered":"2024-04-25T00:16:26.261644+0000","last_clean":"2024-04-25T00:16:26.261644+0000","last_became_active":"2024-04-25T00:14:01.836234+0000","last_became_peered":"2024-04-25T00:14:01.836234+0000","last_unstale":"2024-04-25T00:16:26.261644+0000","last_undegraded":"2024-04-25T00:16:26.261644+0000","last_fullsized":"2024-04-25T00:16:26.261644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:15.037498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472108+0000","last_change":"2024-04-25T00:14:01.825937+0000","last_active":"2024-04-25T00:16:26.472108+0000","last_peered":"2024-04-25T00:16:26.472108+0000","last_clean":"2024-04-25T00:16:26.472108+0000","last_became_active":"2024-04-25T00:14:01.825605+0000","last_became_peered":"2024-04-25T00:14:01.825605+0000","last_unstale":"2024-04-25T00:16:26.472108+0000","last_undegraded":"2024-04-25T00:16:26.472108+0000","last_fullsized":"2024-04-25T00:16:26.472108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:06.918000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328697+0000","last_change":"2024-04-25T00:14:01.830004+0000","last_active":"2024-04-25T00:16:26.328697+0000","last_peered":"2024-04-25T00:16:26.328697+0000","last_clean":"2024-04-25T00:16:26.328697+0000","last_became_active":"2024-04-25T00:14:01.829588+0000","last_became_peered":"2024-04-25T00:14:01.829588+0000","last_unstale":"2024-04-25T00:16:26.328697+0000","last_undegraded":"2024-04-25T00:16:26.328697+0000","last_fullsized":"2024-04-25T00:16:26.328697+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:08.005995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235672+0000","last_change":"2024-04-25T00:14:01.825762+0000","last_active":"2024-04-25T00:16:26.235672+0000","last_peered":"2024-04-25T00:16:26.235672+0000","last_clean":"2024-04-25T00:16:26.235672+0000","last_became_active":"2024-04-25T00:14:01.825471+0000","last_became_peered":"2024-04-25T00:14:01.825471+0000","last_unstale":"2024-04-25T00:16:26.235672+0000","last_undegraded":"2024-04-25T00:16:26.235672+0000","last_fullsized":"2024-04-25T00:16:26.235672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:33.716875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.329464+0000","last_change":"2024-04-25T00:14:01.836103+0000","last_active":"2024-04-25T00:16:26.329464+0000","last_peered":"2024-04-25T00:16:26.329464+0000","last_clean":"2024-04-25T00:16:26.329464+0000","last_became_active":"2024-04-25T00:14:01.834591+0000","last_became_peered":"2024-04-25T00:14:01.834591+0000","last_unstale":"2024-04-25T00:16:26.329464+0000","last_undegraded":"2024-04-25T00:16:26.329464+0000","last_fullsized":"2024-04-25T00:16:26.329464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:55.130978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.335893+0000","last_change":"2024-04-25T00:14:01.833179+0000","last_active":"2024-04-25T00:16:26.335893+0000","last_peered":"2024-04-25T00:16:26.335893+0000","last_clean":"2024-04-25T00:16:26.335893+0000","last_became_active":"2024-04-25T00:14:01.832856+0000","last_became_peered":"2024-04-25T00:14:01.832856+0000","last_unstale":"2024-04-25T00:16:26.335893+0000","last_undegraded":"2024-04-25T00:16:26.335893+0000","last_fullsized":"2024-04-25T00:16:26.335893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:07.153356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328555+0000","last_change":"2024-04-25T00:14:01.834751+0000","last_active":"2024-04-25T00:16:26.328555+0000","last_peered":"2024-04-25T00:16:26.328555+0000","last_clean":"2024-04-25T00:16:26.328555+0000","last_became_active":"2024-04-25T00:14:01.834413+0000","last_became_peered":"2024-04-25T00:14:01.834413+0000","last_unstale":"2024-04-25T00:16:26.328555+0000","last_undegraded":"2024-04-25T00:16:26.328555+0000","last_fullsized":"2024-04-25T00:16:26.328555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:52.341565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.473303+0000","last_change":"2024-04-25T00:14:01.832944+0000","last_active":"2024-04-25T00:16:26.473303+0000","last_peered":"2024-04-25T00:16:26.473303+0000","last_clean":"2024-04-25T00:16:26.473303+0000","last_became_active":"2024-04-25T00:14:01.832394+0000","last_became_peered":"2024-04-25T00:14:01.832394+0000","last_unstale":"2024-04-25T00:16:26.473303+0000","last_undegraded":"2024-04-25T00:16:26.473303+0000","last_fullsized":"2024-04-25T00:16:26.473303+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:26.957786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472374+0000","last_change":"2024-04-25T00:14:01.831492+0000","last_active":"2024-04-25T00:16:26.472374+0000","last_peered":"2024-04-25T00:16:26.472374+0000","last_clean":"2024-04-25T00:16:26.472374+0000","last_became_active":"2024-04-25T00:14:01.829557+0000","last_became_peered":"2024-04-25T00:14:01.829557+0000","last_unstale":"2024-04-25T00:16:26.472374+0000","last_undegraded":"2024-04-25T00:16:26.472374+0000","last_fullsized":"2024-04-25T00:16:26.472374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:45.116188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328652+0000","last_change":"2024-04-25T00:14:01.839323+0000","last_active":"2024-04-25T00:16:26.328652+0000","last_peered":"2024-04-25T00:16:26.328652+0000","last_clean":"2024-04-25T00:16:26.328652+0000","last_became_active":"2024-04-25T00:14:01.838999+0000","last_became_peered":"2024-04-25T00:14:01.838999+0000","last_unstale":"2024-04-25T00:16:26.328652+0000","last_undegraded":"2024-04-25T00:16:26.328652+0000","last_fullsized":"2024-04-25T00:16:26.328652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:20.652123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599808,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":838,"num_read_kb":532,"num_write":287,"num_write_kb":1441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1799424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351018,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1503232,"data_stored":943251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612654,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37232,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707920,"statfs":{"total":95995035648,"available":95956910080,"internally_reserved":0,"allocated":1941504,"data_stored":1339753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776180,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36828,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708324,"statfs":{"total":95995035648,"available":95957323776,"internally_reserved":0,"allocated":1523712,"data_stored":947583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037817,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":540338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299454,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28684,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716468,"statfs":{"total":95995035648,"available":95965663232,"internally_reserved":0,"allocated":1703936,"data_stored":1136388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462980,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1089536,"data_stored":535594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":599417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1130496,"data_stored":546835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28196,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716956,"statfs":{"total":95995035648,"available":95966162944,"internally_reserved":0,"allocated":1204224,"data_stored":617307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149778,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36944,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708208,"statfs":{"total":95995035648,"available":95957204992,"internally_reserved":0,"allocated":1646592,"data_stored":1070457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888141,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40592,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704560,"statfs":{"total":95995035648,"available":95953469440,"internally_reserved":0,"allocated":1118208,"data_stored":550879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724616,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29016,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716136,"statfs":{"total":95995035648,"available":95965323264,"internally_reserved":0,"allocated":1863680,"data_stored":1273429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":70068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":204028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:16:31.313 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:16:31.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.444+0000 7efe9a861640 1 -- 172.21.15.107:0/2647275573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe941029a0 msgr2=0x7efe94102d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:31.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.444+0000 7efe9a861640 1 --2- 172.21.15.107:0/2647275573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe941029a0 0x7efe94102d80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7efe84009900 tx=0x7efe8402a480 comp rx=0 tx=0).stop 2024-04-25T00:16:31.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.444+0000 7efe9a861640 1 -- 172.21.15.107:0/2647275573 shutdown_connections 2024-04-25T00:16:31.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.444+0000 7efe9a861640 1 --2- 172.21.15.107:0/2647275573 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe9410bd70 0x7efe9410e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.444+0000 7efe9a861640 1 --2- 172.21.15.107:0/2647275573 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe941032c0 0x7efe9410b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.444+0000 7efe9a861640 1 --2- 172.21.15.107:0/2647275573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe941029a0 0x7efe94102d80 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.444+0000 7efe9a861640 1 -- 172.21.15.107:0/2647275573 >> 172.21.15.107:0/2647275573 conn(0x7efe940769c0 msgr2=0x7efe94076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:31.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.444+0000 7efe9a861640 1 -- 172.21.15.107:0/2647275573 shutdown_connections 2024-04-25T00:16:31.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.444+0000 7efe9a861640 1 -- 172.21.15.107:0/2647275573 wait complete. 2024-04-25T00:16:31.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.444+0000 7efe9a861640 1 Processor -- start 2024-04-25T00:16:31.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.445+0000 7efe9a861640 1 -- start start 2024-04-25T00:16:31.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.445+0000 7efe9a861640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe941029a0 0x7efe94198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:31.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.445+0000 7efe9a861640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe941032c0 0x7efe941985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:31.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.445+0000 7efe9a861640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe9410bd70 0x7efe9419c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:31.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.445+0000 7efe9a861640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efe94111b60 con 0x7efe9410bd70 2024-04-25T00:16:31.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.445+0000 7efe9a861640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efe941119e0 con 0x7efe941032c0 2024-04-25T00:16:31.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.445+0000 7efe9a861640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efe94111ce0 con 0x7efe941029a0 2024-04-25T00:16:31.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.445+0000 7efe98dfe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe9410bd70 0x7efe9419c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:31.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.445+0000 7efe98dfe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe9410bd70 0x7efe9419c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39064/0 (socket says 172.21.15.107:39064) 2024-04-25T00:16:31.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.445+0000 7efe98dfe640 1 -- 172.21.15.107:0/1641741715 learned_addr learned my addr 172.21.15.107:0/1641741715 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:31.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.445+0000 7efe937fe640 1 --2- 172.21.15.107:0/1641741715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe941032c0 0x7efe941985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:31.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.445+0000 7efe93fff640 1 --2- 172.21.15.107:0/1641741715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe941029a0 0x7efe94198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:31.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.445+0000 7efe98dfe640 1 -- 172.21.15.107:0/1641741715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe941029a0 msgr2=0x7efe94198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:31.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.445+0000 7efe98dfe640 1 --2- 172.21.15.107:0/1641741715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe941029a0 0x7efe94198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.445+0000 7efe98dfe640 1 -- 172.21.15.107:0/1641741715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe941032c0 msgr2=0x7efe941985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:31.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.445+0000 7efe98dfe640 1 --2- 172.21.15.107:0/1641741715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe941032c0 0x7efe941985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.445+0000 7efe98dfe640 1 -- 172.21.15.107:0/1641741715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9419d070 con 0x7efe9410bd70 2024-04-25T00:16:31.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.445+0000 7efe937fe640 1 --2- 172.21.15.107:0/1641741715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe941032c0 0x7efe941985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:31.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.445+0000 7efe93fff640 1 --2- 172.21.15.107:0/1641741715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe941029a0 0x7efe94198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:31.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.445+0000 7efe98dfe640 1 --2- 172.21.15.107:0/1641741715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe9410bd70 0x7efe9419c940 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7efe8c00e9d0 tx=0x7efe8c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:31.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.446+0000 7efe917fa640 1 -- 172.21.15.107:0/1641741715 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8c00cd80 con 0x7efe9410bd70 2024-04-25T00:16:31.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.446+0000 7efe917fa640 1 -- 172.21.15.107:0/1641741715 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe8c005020 con 0x7efe9410bd70 2024-04-25T00:16:31.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.446+0000 7efe9a861640 1 -- 172.21.15.107:0/1641741715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe9419d360 con 0x7efe9410bd70 2024-04-25T00:16:31.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.446+0000 7efe917fa640 1 -- 172.21.15.107:0/1641741715 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8c0116d0 con 0x7efe9410bd70 2024-04-25T00:16:31.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.446+0000 7efe9a861640 1 -- 172.21.15.107:0/1641741715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe941a4bf0 con 0x7efe9410bd70 2024-04-25T00:16:31.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.447+0000 7efe9a861640 1 -- 172.21.15.107:0/1641741715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe9406a5c0 con 0x7efe9410bd70 2024-04-25T00:16:31.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.448+0000 7efe917fa640 1 -- 172.21.15.107:0/1641741715 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7efe8c0051d0 con 0x7efe9410bd70 2024-04-25T00:16:31.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.448+0000 7efe917fa640 1 --2- 172.21.15.107:0/1641741715 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7efe7c0a2ea0 0x7efe7c0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:31.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.448+0000 7efe93fff640 1 --2- 172.21.15.107:0/1641741715 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7efe7c0a2ea0 0x7efe7c0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:31.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.448+0000 7efe917fa640 1 -- 172.21.15.107:0/1641741715 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7efe8c0d0080 con 0x7efe9410bd70 2024-04-25T00:16:31.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.449+0000 7efe93fff640 1 --2- 172.21.15.107:0/1641741715 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7efe7c0a2ea0 0x7efe7c0a5360 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efe84000c00 tx=0x7efe84032c70 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:31.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.453+0000 7efe917fa640 1 -- 172.21.15.107:0/1641741715 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efe8c016030 con 0x7efe9410bd70 2024-04-25T00:16:31.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.626+0000 7efe9a861640 1 -- 172.21.15.107:0/1641741715 --> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efe94061b80 con 0x7efe7c0a2ea0 2024-04-25T00:16:31.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.635+0000 7efe917fa640 1 -- 172.21.15.107:0/1641741715 <== mgr.15456 v2:172.21.15.107:6800/1021104172 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339377 (secure 0 0 0) 0x7efe94061b80 con 0x7efe7c0a2ea0 2024-04-25T00:16:31.640 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:16:31.642 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-04-25T00:16:31.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.638+0000 7efe9a861640 1 -- 172.21.15.107:0/1641741715 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7efe7c0a2ea0 msgr2=0x7efe7c0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:31.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.638+0000 7efe9a861640 1 --2- 172.21.15.107:0/1641741715 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7efe7c0a2ea0 0x7efe7c0a5360 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7efe84000c00 tx=0x7efe84032c70 comp rx=0 tx=0).stop 2024-04-25T00:16:31.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.638+0000 7efe9a861640 1 -- 172.21.15.107:0/1641741715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe9410bd70 msgr2=0x7efe9419c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:31.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.638+0000 7efe9a861640 1 --2- 172.21.15.107:0/1641741715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe9410bd70 0x7efe9419c940 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7efe8c00e9d0 tx=0x7efe8c00ee90 comp rx=0 tx=0).stop 2024-04-25T00:16:31.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.638+0000 7efe9a861640 1 -- 172.21.15.107:0/1641741715 shutdown_connections 2024-04-25T00:16:31.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.638+0000 7efe9a861640 1 --2- 172.21.15.107:0/1641741715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe9410bd70 0x7efe9419c940 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.638+0000 7efe9a861640 1 --2- 172.21.15.107:0/1641741715 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7efe7c0a2ea0 0x7efe7c0a5360 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.638+0000 7efe9a861640 1 --2- 172.21.15.107:0/1641741715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efe941032c0 0x7efe941985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.638+0000 7efe9a861640 1 --2- 172.21.15.107:0/1641741715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe941029a0 0x7efe94198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.638+0000 7efe9a861640 1 -- 172.21.15.107:0/1641741715 >> 172.21.15.107:0/1641741715 conn(0x7efe940769c0 msgr2=0x7efe940fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:31.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.638+0000 7efe9a861640 1 -- 172.21.15.107:0/1641741715 shutdown_connections 2024-04-25T00:16:31.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.638+0000 7efe9a861640 1 -- 172.21.15.107:0/1641741715 wait complete. 2024-04-25T00:16:31.651 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-25T00:16:30.246986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190410,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":974,"num_read_kb":651,"num_write":470,"num_write_kb":4381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":395080,"kb_used_data":16596,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124546744,"statfs":{"total":1151940427776,"available":1151535865856,"internally_reserved":0,"allocated":16994304,"data_stored":10101231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.987708"},"pg_stats":[{"pgid":"3.3f","version":"94'8","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328766+0000","last_change":"2024-04-25T00:14:01.835737+0000","last_active":"2024-04-25T00:16:26.328766+0000","last_peered":"2024-04-25T00:16:26.328766+0000","last_clean":"2024-04-25T00:16:26.328766+0000","last_became_active":"2024-04-25T00:14:01.835428+0000","last_became_peered":"2024-04-25T00:14:01.835428+0000","last_unstale":"2024-04-25T00:16:26.328766+0000","last_undegraded":"2024-04-25T00:16:26.328766+0000","last_fullsized":"2024-04-25T00:16:26.328766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:33.974656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261738+0000","last_change":"2024-04-25T00:14:01.837321+0000","last_active":"2024-04-25T00:16:26.261738+0000","last_peered":"2024-04-25T00:16:26.261738+0000","last_clean":"2024-04-25T00:16:26.261738+0000","last_became_active":"2024-04-25T00:14:01.836762+0000","last_became_peered":"2024-04-25T00:14:01.836762+0000","last_unstale":"2024-04-25T00:16:26.261738+0000","last_undegraded":"2024-04-25T00:16:26.261738+0000","last_fullsized":"2024-04-25T00:16:26.261738+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:34:17.342967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261705+0000","last_change":"2024-04-25T00:14:01.834504+0000","last_active":"2024-04-25T00:16:26.261705+0000","last_peered":"2024-04-25T00:16:26.261705+0000","last_clean":"2024-04-25T00:16:26.261705+0000","last_became_active":"2024-04-25T00:14:01.834182+0000","last_became_peered":"2024-04-25T00:14:01.834182+0000","last_unstale":"2024-04-25T00:16:26.261705+0000","last_undegraded":"2024-04-25T00:16:26.261705+0000","last_fullsized":"2024-04-25T00:16:26.261705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:12.991170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":25,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.234081+0000","last_change":"2024-04-25T00:15:02.080211+0000","last_active":"2024-04-25T00:16:26.234081+0000","last_peered":"2024-04-25T00:16:26.234081+0000","last_clean":"2024-04-25T00:16:26.234081+0000","last_became_active":"2024-04-25T00:15:02.079849+0000","last_became_peered":"2024-04-25T00:15:02.079849+0000","last_unstale":"2024-04-25T00:16:26.234081+0000","last_undegraded":"2024-04-25T00:16:26.234081+0000","last_fullsized":"2024-04-25T00:16:26.234081+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:56.804782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912156+0000","last_change":"2024-04-25T00:14:01.834173+0000","last_active":"2024-04-25T00:16:26.912156+0000","last_peered":"2024-04-25T00:16:26.912156+0000","last_clean":"2024-04-25T00:16:26.912156+0000","last_became_active":"2024-04-25T00:14:01.833835+0000","last_became_peered":"2024-04-25T00:14:01.833835+0000","last_unstale":"2024-04-25T00:16:26.912156+0000","last_undegraded":"2024-04-25T00:16:26.912156+0000","last_fullsized":"2024-04-25T00:16:26.912156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:30.046629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.338934+0000","last_change":"2024-04-25T00:14:01.832113+0000","last_active":"2024-04-25T00:16:26.338934+0000","last_peered":"2024-04-25T00:16:26.338934+0000","last_clean":"2024-04-25T00:16:26.338934+0000","last_became_active":"2024-04-25T00:14:01.831800+0000","last_became_peered":"2024-04-25T00:14:01.831800+0000","last_unstale":"2024-04-25T00:16:26.338934+0000","last_undegraded":"2024-04-25T00:16:26.338934+0000","last_fullsized":"2024-04-25T00:16:26.338934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:54.469994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472140+0000","last_change":"2024-04-25T00:14:01.839166+0000","last_active":"2024-04-25T00:16:26.472140+0000","last_peered":"2024-04-25T00:16:26.472140+0000","last_clean":"2024-04-25T00:16:26.472140+0000","last_became_active":"2024-04-25T00:14:01.838851+0000","last_became_peered":"2024-04-25T00:14:01.838851+0000","last_unstale":"2024-04-25T00:16:26.472140+0000","last_undegraded":"2024-04-25T00:16:26.472140+0000","last_fullsized":"2024-04-25T00:16:26.472140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:39:48.125363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":26,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.233734+0000","last_change":"2024-04-25T00:15:02.083386+0000","last_active":"2024-04-25T00:16:26.233734+0000","last_peered":"2024-04-25T00:16:26.233734+0000","last_clean":"2024-04-25T00:16:26.233734+0000","last_became_active":"2024-04-25T00:15:02.083106+0000","last_became_peered":"2024-04-25T00:15:02.083106+0000","last_unstale":"2024-04-25T00:16:26.233734+0000","last_undegraded":"2024-04-25T00:16:26.233734+0000","last_fullsized":"2024-04-25T00:16:26.233734+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:58.543667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.473534+0000","last_change":"2024-04-25T00:14:01.829803+0000","last_active":"2024-04-25T00:16:26.473534+0000","last_peered":"2024-04-25T00:16:26.473534+0000","last_clean":"2024-04-25T00:16:26.473534+0000","last_became_active":"2024-04-25T00:14:01.827761+0000","last_became_peered":"2024-04-25T00:14:01.827761+0000","last_unstale":"2024-04-25T00:16:26.473534+0000","last_undegraded":"2024-04-25T00:16:26.473534+0000","last_fullsized":"2024-04-25T00:16:26.473534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:38.862505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.339417+0000","last_change":"2024-04-25T00:14:01.834269+0000","last_active":"2024-04-25T00:16:26.339417+0000","last_peered":"2024-04-25T00:16:26.339417+0000","last_clean":"2024-04-25T00:16:26.339417+0000","last_became_active":"2024-04-25T00:14:01.833964+0000","last_became_peered":"2024-04-25T00:14:01.833964+0000","last_unstale":"2024-04-25T00:16:26.339417+0000","last_undegraded":"2024-04-25T00:16:26.339417+0000","last_fullsized":"2024-04-25T00:16:26.339417+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:29.501513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472579+0000","last_change":"2024-04-25T00:14:01.839510+0000","last_active":"2024-04-25T00:16:26.472579+0000","last_peered":"2024-04-25T00:16:26.472579+0000","last_clean":"2024-04-25T00:16:26.472579+0000","last_became_active":"2024-04-25T00:14:01.839000+0000","last_became_peered":"2024-04-25T00:14:01.839000+0000","last_unstale":"2024-04-25T00:16:26.472579+0000","last_undegraded":"2024-04-25T00:16:26.472579+0000","last_fullsized":"2024-04-25T00:16:26.472579+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:20.809277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235608+0000","last_change":"2024-04-25T00:14:01.835821+0000","last_active":"2024-04-25T00:16:26.235608+0000","last_peered":"2024-04-25T00:16:26.235608+0000","last_clean":"2024-04-25T00:16:26.235608+0000","last_became_active":"2024-04-25T00:14:01.835525+0000","last_became_peered":"2024-04-25T00:14:01.835525+0000","last_unstale":"2024-04-25T00:16:26.235608+0000","last_undegraded":"2024-04-25T00:16:26.235608+0000","last_fullsized":"2024-04-25T00:16:26.235608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:17.082562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472256+0000","last_change":"2024-04-25T00:14:01.831564+0000","last_active":"2024-04-25T00:16:26.472256+0000","last_peered":"2024-04-25T00:16:26.472256+0000","last_clean":"2024-04-25T00:16:26.472256+0000","last_became_active":"2024-04-25T00:14:01.828539+0000","last_became_peered":"2024-04-25T00:14:01.828539+0000","last_unstale":"2024-04-25T00:16:26.472256+0000","last_undegraded":"2024-04-25T00:16:26.472256+0000","last_fullsized":"2024-04-25T00:16:26.472256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:41:52.098299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261510+0000","last_change":"2024-04-25T00:14:01.835815+0000","last_active":"2024-04-25T00:16:26.261510+0000","last_peered":"2024-04-25T00:16:26.261510+0000","last_clean":"2024-04-25T00:16:26.261510+0000","last_became_active":"2024-04-25T00:14:01.835498+0000","last_became_peered":"2024-04-25T00:14:01.835498+0000","last_unstale":"2024-04-25T00:16:26.261510+0000","last_undegraded":"2024-04-25T00:16:26.261510+0000","last_fullsized":"2024-04-25T00:16:26.261510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:14:02.914149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261465+0000","last_change":"2024-04-25T00:14:01.839542+0000","last_active":"2024-04-25T00:16:26.261465+0000","last_peered":"2024-04-25T00:16:26.261465+0000","last_clean":"2024-04-25T00:16:26.261465+0000","last_became_active":"2024-04-25T00:14:01.839234+0000","last_became_peered":"2024-04-25T00:14:01.839234+0000","last_unstale":"2024-04-25T00:16:26.261465+0000","last_undegraded":"2024-04-25T00:16:26.261465+0000","last_fullsized":"2024-04-25T00:16:26.261465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:00.437311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472718+0000","last_change":"2024-04-25T00:14:01.832265+0000","last_active":"2024-04-25T00:16:26.472718+0000","last_peered":"2024-04-25T00:16:26.472718+0000","last_clean":"2024-04-25T00:16:26.472718+0000","last_became_active":"2024-04-25T00:14:01.831963+0000","last_became_peered":"2024-04-25T00:14:01.831963+0000","last_unstale":"2024-04-25T00:16:26.472718+0000","last_undegraded":"2024-04-25T00:16:26.472718+0000","last_fullsized":"2024-04-25T00:16:26.472718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:55.402265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328586+0000","last_change":"2024-04-25T00:14:01.835444+0000","last_active":"2024-04-25T00:16:26.328586+0000","last_peered":"2024-04-25T00:16:26.328586+0000","last_clean":"2024-04-25T00:16:26.328586+0000","last_became_active":"2024-04-25T00:14:01.835126+0000","last_became_peered":"2024-04-25T00:14:01.835126+0000","last_unstale":"2024-04-25T00:16:26.328586+0000","last_undegraded":"2024-04-25T00:16:26.328586+0000","last_fullsized":"2024-04-25T00:16:26.328586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:48.997924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.234333+0000","last_change":"2024-04-25T00:14:01.834982+0000","last_active":"2024-04-25T00:16:26.234333+0000","last_peered":"2024-04-25T00:16:26.234333+0000","last_clean":"2024-04-25T00:16:26.234333+0000","last_became_active":"2024-04-25T00:14:01.834671+0000","last_became_peered":"2024-04-25T00:14:01.834671+0000","last_unstale":"2024-04-25T00:16:26.234333+0000","last_undegraded":"2024-04-25T00:16:26.234333+0000","last_fullsized":"2024-04-25T00:16:26.234333+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:28.254408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235773+0000","last_change":"2024-04-25T00:14:01.824432+0000","last_active":"2024-04-25T00:16:26.235773+0000","last_peered":"2024-04-25T00:16:26.235773+0000","last_clean":"2024-04-25T00:16:26.235773+0000","last_became_active":"2024-04-25T00:14:01.823963+0000","last_became_peered":"2024-04-25T00:14:01.823963+0000","last_unstale":"2024-04-25T00:16:26.235773+0000","last_undegraded":"2024-04-25T00:16:26.235773+0000","last_fullsized":"2024-04-25T00:16:26.235773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:35.880932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235322+0000","last_change":"2024-04-25T00:14:06.861892+0000","last_active":"2024-04-25T00:16:26.235322+0000","last_peered":"2024-04-25T00:16:26.235322+0000","last_clean":"2024-04-25T00:16:26.235322+0000","last_became_active":"2024-04-25T00:14:06.861470+0000","last_became_peered":"2024-04-25T00:14:06.861470+0000","last_unstale":"2024-04-25T00:16:26.235322+0000","last_undegraded":"2024-04-25T00:16:26.235322+0000","last_fullsized":"2024-04-25T00:16:26.235322+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:06.192954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.234449+0000","last_change":"2024-04-25T00:14:01.412535+0000","last_active":"2024-04-25T00:16:26.234449+0000","last_peered":"2024-04-25T00:16:26.234449+0000","last_clean":"2024-04-25T00:16:26.234449+0000","last_became_active":"2024-04-25T00:14:01.412206+0000","last_became_peered":"2024-04-25T00:14:01.412206+0000","last_unstale":"2024-04-25T00:16:26.234449+0000","last_undegraded":"2024-04-25T00:16:26.234449+0000","last_fullsized":"2024-04-25T00:16:26.234449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:40.893987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.329130+0000","last_change":"2024-04-25T00:14:01.840814+0000","last_active":"2024-04-25T00:16:26.329130+0000","last_peered":"2024-04-25T00:16:26.329130+0000","last_clean":"2024-04-25T00:16:26.329130+0000","last_became_active":"2024-04-25T00:14:01.840179+0000","last_became_peered":"2024-04-25T00:14:01.840179+0000","last_unstale":"2024-04-25T00:16:26.329130+0000","last_undegraded":"2024-04-25T00:16:26.329130+0000","last_fullsized":"2024-04-25T00:16:26.329130+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:27.383918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235154+0000","last_change":"2024-04-25T00:14:06.857877+0000","last_active":"2024-04-25T00:16:26.235154+0000","last_peered":"2024-04-25T00:16:26.235154+0000","last_clean":"2024-04-25T00:16:26.235154+0000","last_became_active":"2024-04-25T00:14:06.857420+0000","last_became_peered":"2024-04-25T00:14:06.857420+0000","last_unstale":"2024-04-25T00:16:26.235154+0000","last_undegraded":"2024-04-25T00:16:26.235154+0000","last_fullsized":"2024-04-25T00:16:26.235154+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:54.424024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.329451+0000","last_change":"2024-04-25T00:14:01.416560+0000","last_active":"2024-04-25T00:16:26.329451+0000","last_peered":"2024-04-25T00:16:26.329451+0000","last_clean":"2024-04-25T00:16:26.329451+0000","last_became_active":"2024-04-25T00:14:01.416246+0000","last_became_peered":"2024-04-25T00:14:01.416246+0000","last_unstale":"2024-04-25T00:16:26.329451+0000","last_undegraded":"2024-04-25T00:16:26.329451+0000","last_fullsized":"2024-04-25T00:16:26.329451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:03:47.938452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261742+0000","last_change":"2024-04-25T00:14:01.832607+0000","last_active":"2024-04-25T00:16:26.261742+0000","last_peered":"2024-04-25T00:16:26.261742+0000","last_clean":"2024-04-25T00:16:26.261742+0000","last_became_active":"2024-04-25T00:14:01.832300+0000","last_became_peered":"2024-04-25T00:14:01.832300+0000","last_unstale":"2024-04-25T00:16:26.261742+0000","last_undegraded":"2024-04-25T00:16:26.261742+0000","last_fullsized":"2024-04-25T00:16:26.261742+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:22.045171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235355+0000","last_change":"2024-04-25T00:14:06.863642+0000","last_active":"2024-04-25T00:16:26.235355+0000","last_peered":"2024-04-25T00:16:26.235355+0000","last_clean":"2024-04-25T00:16:26.235355+0000","last_became_active":"2024-04-25T00:14:06.863284+0000","last_became_peered":"2024-04-25T00:14:06.863284+0000","last_unstale":"2024-04-25T00:16:26.235355+0000","last_undegraded":"2024-04-25T00:16:26.235355+0000","last_fullsized":"2024-04-25T00:16:26.235355+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:20.818625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912488+0000","last_change":"2024-04-25T00:14:01.411874+0000","last_active":"2024-04-25T00:16:26.912488+0000","last_peered":"2024-04-25T00:16:26.912488+0000","last_clean":"2024-04-25T00:16:26.912488+0000","last_became_active":"2024-04-25T00:14:01.409568+0000","last_became_peered":"2024-04-25T00:14:01.409568+0000","last_unstale":"2024-04-25T00:16:26.912488+0000","last_undegraded":"2024-04-25T00:16:26.912488+0000","last_fullsized":"2024-04-25T00:16:26.912488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:22.415631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.473104+0000","last_change":"2024-04-25T00:14:01.822471+0000","last_active":"2024-04-25T00:16:26.473104+0000","last_peered":"2024-04-25T00:16:26.473104+0000","last_clean":"2024-04-25T00:16:26.473104+0000","last_became_active":"2024-04-25T00:14:01.822181+0000","last_became_peered":"2024-04-25T00:14:01.822181+0000","last_unstale":"2024-04-25T00:16:26.473104+0000","last_undegraded":"2024-04-25T00:16:26.473104+0000","last_fullsized":"2024-04-25T00:16:26.473104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:44.857036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235154+0000","last_change":"2024-04-25T00:14:06.856987+0000","last_active":"2024-04-25T00:16:26.235154+0000","last_peered":"2024-04-25T00:16:26.235154+0000","last_clean":"2024-04-25T00:16:26.235154+0000","last_became_active":"2024-04-25T00:14:06.856053+0000","last_became_peered":"2024-04-25T00:14:06.856053+0000","last_unstale":"2024-04-25T00:16:26.235154+0000","last_undegraded":"2024-04-25T00:16:26.235154+0000","last_fullsized":"2024-04-25T00:16:26.235154+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:16.829969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912190+0000","last_change":"2024-04-25T00:14:01.410156+0000","last_active":"2024-04-25T00:16:26.912190+0000","last_peered":"2024-04-25T00:16:26.912190+0000","last_clean":"2024-04-25T00:16:26.912190+0000","last_became_active":"2024-04-25T00:14:01.409809+0000","last_became_peered":"2024-04-25T00:14:01.409809+0000","last_unstale":"2024-04-25T00:16:26.912190+0000","last_undegraded":"2024-04-25T00:16:26.912190+0000","last_fullsized":"2024-04-25T00:16:26.912190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:43.160419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.336265+0000","last_change":"2024-04-25T00:14:01.836090+0000","last_active":"2024-04-25T00:16:26.336265+0000","last_peered":"2024-04-25T00:16:26.336265+0000","last_clean":"2024-04-25T00:16:26.336265+0000","last_became_active":"2024-04-25T00:14:01.835733+0000","last_became_peered":"2024-04-25T00:14:01.835733+0000","last_unstale":"2024-04-25T00:16:26.336265+0000","last_undegraded":"2024-04-25T00:16:26.336265+0000","last_fullsized":"2024-04-25T00:16:26.336265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:02.368893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235589+0000","last_change":"2024-04-25T00:14:06.851496+0000","last_active":"2024-04-25T00:16:26.235589+0000","last_peered":"2024-04-25T00:16:26.235589+0000","last_clean":"2024-04-25T00:16:26.235589+0000","last_became_active":"2024-04-25T00:14:06.851007+0000","last_became_peered":"2024-04-25T00:14:06.851007+0000","last_unstale":"2024-04-25T00:16:26.235589+0000","last_undegraded":"2024-04-25T00:16:26.235589+0000","last_fullsized":"2024-04-25T00:16:26.235589+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:50.543348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'7","reported_seq":167,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.336089+0000","last_change":"2024-04-25T00:14:01.414760+0000","last_active":"2024-04-25T00:16:26.336089+0000","last_peered":"2024-04-25T00:16:26.336089+0000","last_clean":"2024-04-25T00:16:26.336089+0000","last_became_active":"2024-04-25T00:14:01.414482+0000","last_became_peered":"2024-04-25T00:14:01.414482+0000","last_unstale":"2024-04-25T00:16:26.336089+0000","last_undegraded":"2024-04-25T00:16:26.336089+0000","last_fullsized":"2024-04-25T00:16:26.336089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:32.631933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912017+0000","last_change":"2024-04-25T00:14:01.832516+0000","last_active":"2024-04-25T00:16:26.912017+0000","last_peered":"2024-04-25T00:16:26.912017+0000","last_clean":"2024-04-25T00:16:26.912017+0000","last_became_active":"2024-04-25T00:14:01.832106+0000","last_became_peered":"2024-04-25T00:14:01.832106+0000","last_unstale":"2024-04-25T00:16:26.912017+0000","last_undegraded":"2024-04-25T00:16:26.912017+0000","last_fullsized":"2024-04-25T00:16:26.912017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:55.590486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235703+0000","last_change":"2024-04-25T00:14:06.861984+0000","last_active":"2024-04-25T00:16:26.235703+0000","last_peered":"2024-04-25T00:16:26.235703+0000","last_clean":"2024-04-25T00:16:26.235703+0000","last_became_active":"2024-04-25T00:14:06.861418+0000","last_became_peered":"2024-04-25T00:14:06.861418+0000","last_unstale":"2024-04-25T00:16:26.235703+0000","last_undegraded":"2024-04-25T00:16:26.235703+0000","last_fullsized":"2024-04-25T00:16:26.235703+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:27.559704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.338926+0000","last_change":"2024-04-25T00:14:01.407648+0000","last_active":"2024-04-25T00:16:26.338926+0000","last_peered":"2024-04-25T00:16:26.338926+0000","last_clean":"2024-04-25T00:16:26.338926+0000","last_became_active":"2024-04-25T00:14:01.407318+0000","last_became_peered":"2024-04-25T00:14:01.407318+0000","last_unstale":"2024-04-25T00:16:26.338926+0000","last_undegraded":"2024-04-25T00:16:26.338926+0000","last_fullsized":"2024-04-25T00:16:26.338926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:08:10.942814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.336183+0000","last_change":"2024-04-25T00:14:01.833073+0000","last_active":"2024-04-25T00:16:26.336183+0000","last_peered":"2024-04-25T00:16:26.336183+0000","last_clean":"2024-04-25T00:16:26.336183+0000","last_became_active":"2024-04-25T00:14:01.832545+0000","last_became_peered":"2024-04-25T00:14:01.832545+0000","last_unstale":"2024-04-25T00:16:26.336183+0000","last_undegraded":"2024-04-25T00:16:26.336183+0000","last_fullsized":"2024-04-25T00:16:26.336183+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:39.772781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328531+0000","last_change":"2024-04-25T00:14:06.862011+0000","last_active":"2024-04-25T00:16:26.328531+0000","last_peered":"2024-04-25T00:16:26.328531+0000","last_clean":"2024-04-25T00:16:26.328531+0000","last_became_active":"2024-04-25T00:14:06.861544+0000","last_became_peered":"2024-04-25T00:14:06.861544+0000","last_unstale":"2024-04-25T00:16:26.328531+0000","last_undegraded":"2024-04-25T00:16:26.328531+0000","last_fullsized":"2024-04-25T00:16:26.328531+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:37.736266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471570+0000","last_change":"2024-04-25T00:14:01.397747+0000","last_active":"2024-04-25T00:16:26.471570+0000","last_peered":"2024-04-25T00:16:26.471570+0000","last_clean":"2024-04-25T00:16:26.471570+0000","last_became_active":"2024-04-25T00:14:01.397446+0000","last_became_peered":"2024-04-25T00:14:01.397446+0000","last_unstale":"2024-04-25T00:16:26.471570+0000","last_undegraded":"2024-04-25T00:16:26.471570+0000","last_fullsized":"2024-04-25T00:16:26.471570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:27.240034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328414+0000","last_change":"2024-04-25T00:14:01.833128+0000","last_active":"2024-04-25T00:16:26.328414+0000","last_peered":"2024-04-25T00:16:26.328414+0000","last_clean":"2024-04-25T00:16:26.328414+0000","last_became_active":"2024-04-25T00:14:01.832780+0000","last_became_peered":"2024-04-25T00:14:01.832780+0000","last_unstale":"2024-04-25T00:16:26.328414+0000","last_undegraded":"2024-04-25T00:16:26.328414+0000","last_fullsized":"2024-04-25T00:16:26.328414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.023737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261698+0000","last_change":"2024-04-25T00:14:06.860436+0000","last_active":"2024-04-25T00:16:26.261698+0000","last_peered":"2024-04-25T00:16:26.261698+0000","last_clean":"2024-04-25T00:16:26.261698+0000","last_became_active":"2024-04-25T00:14:06.860047+0000","last_became_peered":"2024-04-25T00:14:06.860047+0000","last_unstale":"2024-04-25T00:16:26.261698+0000","last_undegraded":"2024-04-25T00:16:26.261698+0000","last_fullsized":"2024-04-25T00:16:26.261698+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:08:41.246549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.335972+0000","last_change":"2024-04-25T00:14:01.415939+0000","last_active":"2024-04-25T00:16:26.335972+0000","last_peered":"2024-04-25T00:16:26.335972+0000","last_clean":"2024-04-25T00:16:26.335972+0000","last_became_active":"2024-04-25T00:14:01.415513+0000","last_became_peered":"2024-04-25T00:14:01.415513+0000","last_unstale":"2024-04-25T00:16:26.335972+0000","last_undegraded":"2024-04-25T00:16:26.335972+0000","last_fullsized":"2024-04-25T00:16:26.335972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:57:23.588868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.233864+0000","last_change":"2024-04-25T00:14:01.838793+0000","last_active":"2024-04-25T00:16:26.233864+0000","last_peered":"2024-04-25T00:16:26.233864+0000","last_clean":"2024-04-25T00:16:26.233864+0000","last_became_active":"2024-04-25T00:14:01.838453+0000","last_became_peered":"2024-04-25T00:14:01.838453+0000","last_unstale":"2024-04-25T00:16:26.233864+0000","last_undegraded":"2024-04-25T00:16:26.233864+0000","last_fullsized":"2024-04-25T00:16:26.233864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:18.674009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328492+0000","last_change":"2024-04-25T00:14:06.856431+0000","last_active":"2024-04-25T00:16:26.328492+0000","last_peered":"2024-04-25T00:16:26.328492+0000","last_clean":"2024-04-25T00:16:26.328492+0000","last_became_active":"2024-04-25T00:14:06.856047+0000","last_became_peered":"2024-04-25T00:14:06.856047+0000","last_unstale":"2024-04-25T00:16:26.328492+0000","last_undegraded":"2024-04-25T00:16:26.328492+0000","last_fullsized":"2024-04-25T00:16:26.328492+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:20.646673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":160,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.339454+0000","last_change":"2024-04-25T00:14:01.399054+0000","last_active":"2024-04-25T00:16:26.339454+0000","last_peered":"2024-04-25T00:16:26.339454+0000","last_clean":"2024-04-25T00:16:26.339454+0000","last_became_active":"2024-04-25T00:14:01.398711+0000","last_became_peered":"2024-04-25T00:14:01.398711+0000","last_unstale":"2024-04-25T00:16:26.339454+0000","last_undegraded":"2024-04-25T00:16:26.339454+0000","last_fullsized":"2024-04-25T00:16:26.339454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:26.434014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.473480+0000","last_change":"2024-04-25T00:14:01.829132+0000","last_active":"2024-04-25T00:16:26.473480+0000","last_peered":"2024-04-25T00:16:26.473480+0000","last_clean":"2024-04-25T00:16:26.473480+0000","last_became_active":"2024-04-25T00:14:01.828446+0000","last_became_peered":"2024-04-25T00:14:01.828446+0000","last_unstale":"2024-04-25T00:16:26.473480+0000","last_undegraded":"2024-04-25T00:16:26.473480+0000","last_fullsized":"2024-04-25T00:16:26.473480+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:35.788478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.233990+0000","last_change":"2024-04-25T00:14:06.861884+0000","last_active":"2024-04-25T00:16:26.233990+0000","last_peered":"2024-04-25T00:16:26.233990+0000","last_clean":"2024-04-25T00:16:26.233990+0000","last_became_active":"2024-04-25T00:14:06.861413+0000","last_became_peered":"2024-04-25T00:14:06.861413+0000","last_unstale":"2024-04-25T00:16:26.233990+0000","last_undegraded":"2024-04-25T00:16:26.233990+0000","last_fullsized":"2024-04-25T00:16:26.233990+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:53.876308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471856+0000","last_change":"2024-04-25T00:16:03.325276+0000","last_active":"2024-04-25T00:16:26.471856+0000","last_peered":"2024-04-25T00:16:26.471856+0000","last_clean":"2024-04-25T00:16:26.471856+0000","last_became_active":"2024-04-25T00:16:03.322660+0000","last_became_peered":"2024-04-25T00:16:03.322660+0000","last_unstale":"2024-04-25T00:16:26.471856+0000","last_undegraded":"2024-04-25T00:16:26.471856+0000","last_fullsized":"2024-04-25T00:16:26.471856+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:55.882768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261931+0000","last_change":"2024-04-25T00:15:02.372269+0000","last_active":"2024-04-25T00:16:26.261931+0000","last_peered":"2024-04-25T00:16:26.261931+0000","last_clean":"2024-04-25T00:16:26.261931+0000","last_became_active":"2024-04-25T00:15:02.371962+0000","last_became_peered":"2024-04-25T00:15:02.371962+0000","last_unstale":"2024-04-25T00:16:26.261931+0000","last_undegraded":"2024-04-25T00:16:26.261931+0000","last_fullsized":"2024-04-25T00:16:26.261931+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:46.597638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235686+0000","last_change":"2024-04-25T00:15:02.372941+0000","last_active":"2024-04-25T00:16:26.235686+0000","last_peered":"2024-04-25T00:16:26.235686+0000","last_clean":"2024-04-25T00:16:26.235686+0000","last_became_active":"2024-04-25T00:15:02.372504+0000","last_became_peered":"2024-04-25T00:15:02.372504+0000","last_unstale":"2024-04-25T00:16:26.235686+0000","last_undegraded":"2024-04-25T00:16:26.235686+0000","last_fullsized":"2024-04-25T00:16:26.235686+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:58.202708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.336237+0000","last_change":"2024-04-25T00:14:01.401128+0000","last_active":"2024-04-25T00:16:26.336237+0000","last_peered":"2024-04-25T00:16:26.336237+0000","last_clean":"2024-04-25T00:16:26.336237+0000","last_became_active":"2024-04-25T00:14:01.400125+0000","last_became_peered":"2024-04-25T00:14:01.400125+0000","last_unstale":"2024-04-25T00:16:26.336237+0000","last_undegraded":"2024-04-25T00:16:26.336237+0000","last_fullsized":"2024-04-25T00:16:26.336237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:38:43.787855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.473511+0000","last_change":"2024-04-25T00:14:01.830649+0000","last_active":"2024-04-25T00:16:26.473511+0000","last_peered":"2024-04-25T00:16:26.473511+0000","last_clean":"2024-04-25T00:16:26.473511+0000","last_became_active":"2024-04-25T00:14:01.830253+0000","last_became_peered":"2024-04-25T00:14:01.830253+0000","last_unstale":"2024-04-25T00:16:26.473511+0000","last_undegraded":"2024-04-25T00:16:26.473511+0000","last_fullsized":"2024-04-25T00:16:26.473511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:37.474833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.233945+0000","last_change":"2024-04-25T00:14:06.861275+0000","last_active":"2024-04-25T00:16:26.233945+0000","last_peered":"2024-04-25T00:16:26.233945+0000","last_clean":"2024-04-25T00:16:26.233945+0000","last_became_active":"2024-04-25T00:14:06.860891+0000","last_became_peered":"2024-04-25T00:14:06.860891+0000","last_unstale":"2024-04-25T00:16:26.233945+0000","last_undegraded":"2024-04-25T00:16:26.233945+0000","last_fullsized":"2024-04-25T00:16:26.233945+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:38.685837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"97'40","reported_seq":327,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.726401+0000","last_change":"2024-04-25T00:14:01.404706+0000","last_active":"2024-04-25T00:16:26.726401+0000","last_peered":"2024-04-25T00:16:26.726401+0000","last_clean":"2024-04-25T00:16:26.726401+0000","last_became_active":"2024-04-25T00:14:01.404231+0000","last_became_peered":"2024-04-25T00:14:01.404231+0000","last_unstale":"2024-04-25T00:16:26.726401+0000","last_undegraded":"2024-04-25T00:16:26.726401+0000","last_fullsized":"2024-04-25T00:16:26.726401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:36.437491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70068,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":249,"num_read_kb":194,"num_write":41,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912310+0000","last_change":"2024-04-25T00:14:01.828312+0000","last_active":"2024-04-25T00:16:26.912310+0000","last_peered":"2024-04-25T00:16:26.912310+0000","last_clean":"2024-04-25T00:16:26.912310+0000","last_became_active":"2024-04-25T00:14:01.827947+0000","last_became_peered":"2024-04-25T00:14:01.827947+0000","last_unstale":"2024-04-25T00:16:26.912310+0000","last_undegraded":"2024-04-25T00:16:26.912310+0000","last_fullsized":"2024-04-25T00:16:26.912310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.782796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.336543+0000","last_change":"2024-04-25T00:14:06.854379+0000","last_active":"2024-04-25T00:16:26.336543+0000","last_peered":"2024-04-25T00:16:26.336543+0000","last_clean":"2024-04-25T00:16:26.336543+0000","last_became_active":"2024-04-25T00:14:06.853307+0000","last_became_peered":"2024-04-25T00:14:06.853307+0000","last_unstale":"2024-04-25T00:16:26.336543+0000","last_undegraded":"2024-04-25T00:16:26.336543+0000","last_fullsized":"2024-04-25T00:16:26.336543+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:24:11.804189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235501+0000","last_change":"2024-04-25T00:14:01.410395+0000","last_active":"2024-04-25T00:16:26.235501+0000","last_peered":"2024-04-25T00:16:26.235501+0000","last_clean":"2024-04-25T00:16:26.235501+0000","last_became_active":"2024-04-25T00:14:01.410039+0000","last_became_peered":"2024-04-25T00:14:01.410039+0000","last_unstale":"2024-04-25T00:16:26.235501+0000","last_undegraded":"2024-04-25T00:16:26.235501+0000","last_fullsized":"2024-04-25T00:16:26.235501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:29.496519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471990+0000","last_change":"2024-04-25T00:14:01.828403+0000","last_active":"2024-04-25T00:16:26.471990+0000","last_peered":"2024-04-25T00:16:26.471990+0000","last_clean":"2024-04-25T00:16:26.471990+0000","last_became_active":"2024-04-25T00:14:01.828107+0000","last_became_peered":"2024-04-25T00:14:01.828107+0000","last_unstale":"2024-04-25T00:16:26.471990+0000","last_undegraded":"2024-04-25T00:16:26.471990+0000","last_fullsized":"2024-04-25T00:16:26.471990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:29.417639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328638+0000","last_change":"2024-04-25T00:15:02.374105+0000","last_active":"2024-04-25T00:16:26.328638+0000","last_peered":"2024-04-25T00:16:26.328638+0000","last_clean":"2024-04-25T00:16:26.328638+0000","last_became_active":"2024-04-25T00:15:02.373628+0000","last_became_peered":"2024-04-25T00:15:02.373628+0000","last_unstale":"2024-04-25T00:16:26.328638+0000","last_undegraded":"2024-04-25T00:16:26.328638+0000","last_fullsized":"2024-04-25T00:16:26.328638+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:05.542358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.336205+0000","last_change":"2024-04-25T00:14:01.414406+0000","last_active":"2024-04-25T00:16:26.336205+0000","last_peered":"2024-04-25T00:16:26.336205+0000","last_clean":"2024-04-25T00:16:26.336205+0000","last_became_active":"2024-04-25T00:14:01.414085+0000","last_became_peered":"2024-04-25T00:14:01.414085+0000","last_unstale":"2024-04-25T00:16:26.336205+0000","last_undegraded":"2024-04-25T00:16:26.336205+0000","last_fullsized":"2024-04-25T00:16:26.336205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:22.565371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471981+0000","last_change":"2024-04-25T00:14:01.833652+0000","last_active":"2024-04-25T00:16:26.471981+0000","last_peered":"2024-04-25T00:16:26.471981+0000","last_clean":"2024-04-25T00:16:26.471981+0000","last_became_active":"2024-04-25T00:14:01.833307+0000","last_became_peered":"2024-04-25T00:14:01.833307+0000","last_unstale":"2024-04-25T00:16:26.471981+0000","last_undegraded":"2024-04-25T00:16:26.471981+0000","last_fullsized":"2024-04-25T00:16:26.471981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:55:41.261994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.234672+0000","last_change":"2024-04-25T00:14:06.860167+0000","last_active":"2024-04-25T00:16:26.234672+0000","last_peered":"2024-04-25T00:16:26.234672+0000","last_clean":"2024-04-25T00:16:26.234672+0000","last_became_active":"2024-04-25T00:14:06.858651+0000","last_became_peered":"2024-04-25T00:14:06.858651+0000","last_unstale":"2024-04-25T00:16:26.234672+0000","last_undegraded":"2024-04-25T00:16:26.234672+0000","last_fullsized":"2024-04-25T00:16:26.234672+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:11.775434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"97'5","reported_seq":165,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.715411+0000","last_change":"2024-04-25T00:14:01.406267+0000","last_active":"2024-04-25T00:16:26.715411+0000","last_peered":"2024-04-25T00:16:26.715411+0000","last_clean":"2024-04-25T00:16:26.715411+0000","last_became_active":"2024-04-25T00:14:01.405985+0000","last_became_peered":"2024-04-25T00:14:01.405985+0000","last_unstale":"2024-04-25T00:16:26.715411+0000","last_undegraded":"2024-04-25T00:16:26.715411+0000","last_fullsized":"2024-04-25T00:16:26.715411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:17.253805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":4,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472054+0000","last_change":"2024-04-25T00:14:01.838019+0000","last_active":"2024-04-25T00:16:26.472054+0000","last_peered":"2024-04-25T00:16:26.472054+0000","last_clean":"2024-04-25T00:16:26.472054+0000","last_became_active":"2024-04-25T00:14:01.837691+0000","last_became_peered":"2024-04-25T00:14:01.837691+0000","last_unstale":"2024-04-25T00:16:26.472054+0000","last_undegraded":"2024-04-25T00:16:26.472054+0000","last_fullsized":"2024-04-25T00:16:26.472054+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:26.677039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472562+0000","last_change":"2024-04-25T00:14:06.857061+0000","last_active":"2024-04-25T00:16:26.472562+0000","last_peered":"2024-04-25T00:16:26.472562+0000","last_clean":"2024-04-25T00:16:26.472562+0000","last_became_active":"2024-04-25T00:14:06.856678+0000","last_became_peered":"2024-04-25T00:14:06.856678+0000","last_unstale":"2024-04-25T00:16:26.472562+0000","last_undegraded":"2024-04-25T00:16:26.472562+0000","last_fullsized":"2024-04-25T00:16:26.472562+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:36:52.730861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.911965+0000","last_change":"2024-04-25T00:14:01.408385+0000","last_active":"2024-04-25T00:16:26.911965+0000","last_peered":"2024-04-25T00:16:26.911965+0000","last_clean":"2024-04-25T00:16:26.911965+0000","last_became_active":"2024-04-25T00:14:01.408049+0000","last_became_peered":"2024-04-25T00:14:01.408049+0000","last_unstale":"2024-04-25T00:16:26.911965+0000","last_undegraded":"2024-04-25T00:16:26.911965+0000","last_fullsized":"2024-04-25T00:16:26.911965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:19.765467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.335937+0000","last_change":"2024-04-25T00:15:02.372426+0000","last_active":"2024-04-25T00:16:26.335937+0000","last_peered":"2024-04-25T00:16:26.335937+0000","last_clean":"2024-04-25T00:16:26.335937+0000","last_became_active":"2024-04-25T00:15:02.372044+0000","last_became_peered":"2024-04-25T00:15:02.372044+0000","last_unstale":"2024-04-25T00:16:26.335937+0000","last_undegraded":"2024-04-25T00:16:26.335937+0000","last_fullsized":"2024-04-25T00:16:26.335937+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:20.965683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.335834+0000","last_change":"2024-04-25T00:14:06.854192+0000","last_active":"2024-04-25T00:16:26.335834+0000","last_peered":"2024-04-25T00:16:26.335834+0000","last_clean":"2024-04-25T00:16:26.335834+0000","last_became_active":"2024-04-25T00:14:06.853325+0000","last_became_peered":"2024-04-25T00:14:06.853325+0000","last_unstale":"2024-04-25T00:16:26.335834+0000","last_undegraded":"2024-04-25T00:16:26.335834+0000","last_fullsized":"2024-04-25T00:16:26.335834+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:15.308169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912441+0000","last_change":"2024-04-25T00:14:01.407247+0000","last_active":"2024-04-25T00:16:26.912441+0000","last_peered":"2024-04-25T00:16:26.912441+0000","last_clean":"2024-04-25T00:16:26.912441+0000","last_became_active":"2024-04-25T00:14:01.406891+0000","last_became_peered":"2024-04-25T00:14:01.406891+0000","last_unstale":"2024-04-25T00:16:26.912441+0000","last_undegraded":"2024-04-25T00:16:26.912441+0000","last_fullsized":"2024-04-25T00:16:26.912441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:43.294562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328645+0000","last_change":"2024-04-25T00:14:01.829364+0000","last_active":"2024-04-25T00:16:26.328645+0000","last_peered":"2024-04-25T00:16:26.328645+0000","last_clean":"2024-04-25T00:16:26.328645+0000","last_became_active":"2024-04-25T00:14:01.829087+0000","last_became_peered":"2024-04-25T00:14:01.829087+0000","last_unstale":"2024-04-25T00:16:26.328645+0000","last_undegraded":"2024-04-25T00:16:26.328645+0000","last_fullsized":"2024-04-25T00:16:26.328645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:17.651815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.911928+0000","last_change":"2024-04-25T00:14:07.323442+0000","last_active":"2024-04-25T00:16:26.911928+0000","last_peered":"2024-04-25T00:16:26.911928+0000","last_clean":"2024-04-25T00:16:26.911928+0000","last_became_active":"2024-04-25T00:14:07.322943+0000","last_became_peered":"2024-04-25T00:14:07.322943+0000","last_unstale":"2024-04-25T00:16:26.911928+0000","last_undegraded":"2024-04-25T00:16:26.911928+0000","last_fullsized":"2024-04-25T00:16:26.911928+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:49.166240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.234040+0000","last_change":"2024-04-25T00:14:01.410936+0000","last_active":"2024-04-25T00:16:26.234040+0000","last_peered":"2024-04-25T00:16:26.234040+0000","last_clean":"2024-04-25T00:16:26.234040+0000","last_became_active":"2024-04-25T00:14:01.410321+0000","last_became_peered":"2024-04-25T00:14:01.410321+0000","last_unstale":"2024-04-25T00:16:26.234040+0000","last_undegraded":"2024-04-25T00:16:26.234040+0000","last_fullsized":"2024-04-25T00:16:26.234040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:44.603199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"94'4","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.339205+0000","last_change":"2024-04-25T00:14:01.830842+0000","last_active":"2024-04-25T00:16:26.339205+0000","last_peered":"2024-04-25T00:16:26.339205+0000","last_clean":"2024-04-25T00:16:26.339205+0000","last_became_active":"2024-04-25T00:14:01.830543+0000","last_became_peered":"2024-04-25T00:14:01.830543+0000","last_unstale":"2024-04-25T00:16:26.339205+0000","last_undegraded":"2024-04-25T00:16:26.339205+0000","last_fullsized":"2024-04-25T00:16:26.339205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:59.897133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471933+0000","last_change":"2024-04-25T00:14:06.856841+0000","last_active":"2024-04-25T00:16:26.471933+0000","last_peered":"2024-04-25T00:16:26.471933+0000","last_clean":"2024-04-25T00:16:26.471933+0000","last_became_active":"2024-04-25T00:14:06.856416+0000","last_became_peered":"2024-04-25T00:14:06.856416+0000","last_unstale":"2024-04-25T00:16:26.471933+0000","last_undegraded":"2024-04-25T00:16:26.471933+0000","last_fullsized":"2024-04-25T00:16:26.471933+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:27.181175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471815+0000","last_change":"2024-04-25T00:14:01.409627+0000","last_active":"2024-04-25T00:16:26.471815+0000","last_peered":"2024-04-25T00:16:26.471815+0000","last_clean":"2024-04-25T00:16:26.471815+0000","last_became_active":"2024-04-25T00:14:01.409304+0000","last_became_peered":"2024-04-25T00:14:01.409304+0000","last_unstale":"2024-04-25T00:16:26.471815+0000","last_undegraded":"2024-04-25T00:16:26.471815+0000","last_fullsized":"2024-04-25T00:16:26.471815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:27.242004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.339062+0000","last_change":"2024-04-25T00:14:01.829544+0000","last_active":"2024-04-25T00:16:26.339062+0000","last_peered":"2024-04-25T00:16:26.339062+0000","last_clean":"2024-04-25T00:16:26.339062+0000","last_became_active":"2024-04-25T00:14:01.829233+0000","last_became_peered":"2024-04-25T00:14:01.829233+0000","last_unstale":"2024-04-25T00:16:26.339062+0000","last_undegraded":"2024-04-25T00:16:26.339062+0000","last_fullsized":"2024-04-25T00:16:26.339062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:16.102041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472694+0000","last_change":"2024-04-25T00:14:07.323213+0000","last_active":"2024-04-25T00:16:26.472694+0000","last_peered":"2024-04-25T00:16:26.472694+0000","last_clean":"2024-04-25T00:16:26.472694+0000","last_became_active":"2024-04-25T00:14:07.322763+0000","last_became_peered":"2024-04-25T00:14:07.322763+0000","last_unstale":"2024-04-25T00:16:26.472694+0000","last_undegraded":"2024-04-25T00:16:26.472694+0000","last_fullsized":"2024-04-25T00:16:26.472694+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:45.269314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":25,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.233734+0000","last_change":"2024-04-25T00:15:02.079745+0000","last_active":"2024-04-25T00:16:26.233734+0000","last_peered":"2024-04-25T00:16:26.233734+0000","last_clean":"2024-04-25T00:16:26.233734+0000","last_became_active":"2024-04-25T00:15:02.079405+0000","last_became_peered":"2024-04-25T00:15:02.079405+0000","last_unstale":"2024-04-25T00:16:26.233734+0000","last_undegraded":"2024-04-25T00:16:26.233734+0000","last_fullsized":"2024-04-25T00:16:26.233734+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:38.197048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"97'5","reported_seq":162,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.473772+0000","last_change":"2024-04-25T00:14:01.397066+0000","last_active":"2024-04-25T00:16:26.473772+0000","last_peered":"2024-04-25T00:16:26.473772+0000","last_clean":"2024-04-25T00:16:26.473772+0000","last_became_active":"2024-04-25T00:14:01.396728+0000","last_became_peered":"2024-04-25T00:14:01.396728+0000","last_unstale":"2024-04-25T00:16:26.473772+0000","last_undegraded":"2024-04-25T00:16:26.473772+0000","last_fullsized":"2024-04-25T00:16:26.473772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:28.334329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"92'1","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472372+0000","last_change":"2024-04-25T00:14:06.853039+0000","last_active":"2024-04-25T00:16:26.472372+0000","last_peered":"2024-04-25T00:16:26.472372+0000","last_clean":"2024-04-25T00:16:26.472372+0000","last_became_active":"2024-04-25T00:14:06.852568+0000","last_became_peered":"2024-04-25T00:14:06.852568+0000","last_unstale":"2024-04-25T00:16:26.472372+0000","last_undegraded":"2024-04-25T00:16:26.472372+0000","last_fullsized":"2024-04-25T00:16:26.472372+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:59.604160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328701+0000","last_change":"2024-04-25T00:14:01.840315+0000","last_active":"2024-04-25T00:16:26.328701+0000","last_peered":"2024-04-25T00:16:26.328701+0000","last_clean":"2024-04-25T00:16:26.328701+0000","last_became_active":"2024-04-25T00:14:01.839537+0000","last_became_peered":"2024-04-25T00:14:01.839537+0000","last_unstale":"2024-04-25T00:16:26.328701+0000","last_undegraded":"2024-04-25T00:16:26.328701+0000","last_fullsized":"2024-04-25T00:16:26.328701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:34.126164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235345+0000","last_change":"2024-04-25T00:14:01.412493+0000","last_active":"2024-04-25T00:16:26.235345+0000","last_peered":"2024-04-25T00:16:26.235345+0000","last_clean":"2024-04-25T00:16:26.235345+0000","last_became_active":"2024-04-25T00:14:01.412133+0000","last_became_peered":"2024-04-25T00:14:01.412133+0000","last_unstale":"2024-04-25T00:16:26.235345+0000","last_undegraded":"2024-04-25T00:16:26.235345+0000","last_fullsized":"2024-04-25T00:16:26.235345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:44.933728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":371,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471716+0000","last_change":"2024-04-25T00:12:49.169294+0000","last_active":"2024-04-25T00:16:26.471716+0000","last_peered":"2024-04-25T00:16:26.471716+0000","last_clean":"2024-04-25T00:16:26.471716+0000","last_became_active":"2024-04-25T00:12:49.168783+0000","last_became_peered":"2024-04-25T00:12:49.168783+0000","last_unstale":"2024-04-25T00:16:26.471716+0000","last_undegraded":"2024-04-25T00:16:26.471716+0000","last_fullsized":"2024-04-25T00:16:26.471716+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:25.202280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:25.202280+0000","last_clean_scrub_stamp":"2024-04-25T00:09:25.202280+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:04.713986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912255+0000","last_change":"2024-04-25T00:15:02.372570+0000","last_active":"2024-04-25T00:16:26.912255+0000","last_peered":"2024-04-25T00:16:26.912255+0000","last_clean":"2024-04-25T00:16:26.912255+0000","last_became_active":"2024-04-25T00:15:02.372066+0000","last_became_peered":"2024-04-25T00:15:02.372066+0000","last_unstale":"2024-04-25T00:16:26.912255+0000","last_undegraded":"2024-04-25T00:16:26.912255+0000","last_fullsized":"2024-04-25T00:16:26.912255+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:59.470049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,6,10],"acting":[8,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471452+0000","last_change":"2024-04-25T00:14:01.826418+0000","last_active":"2024-04-25T00:16:26.471452+0000","last_peered":"2024-04-25T00:16:26.471452+0000","last_clean":"2024-04-25T00:16:26.471452+0000","last_became_active":"2024-04-25T00:14:01.825783+0000","last_became_peered":"2024-04-25T00:14:01.825783+0000","last_unstale":"2024-04-25T00:16:26.471452+0000","last_undegraded":"2024-04-25T00:16:26.471452+0000","last_fullsized":"2024-04-25T00:16:26.471452+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:05.981359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471893+0000","last_change":"2024-04-25T00:14:01.417022+0000","last_active":"2024-04-25T00:16:26.471893+0000","last_peered":"2024-04-25T00:16:26.471893+0000","last_clean":"2024-04-25T00:16:26.471893+0000","last_became_active":"2024-04-25T00:14:01.416804+0000","last_became_peered":"2024-04-25T00:14:01.416804+0000","last_unstale":"2024-04-25T00:16:26.471893+0000","last_undegraded":"2024-04-25T00:16:26.471893+0000","last_fullsized":"2024-04-25T00:16:26.471893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:31.561242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328438+0000","last_change":"2024-04-25T00:14:06.856500+0000","last_active":"2024-04-25T00:16:26.328438+0000","last_peered":"2024-04-25T00:16:26.328438+0000","last_clean":"2024-04-25T00:16:26.328438+0000","last_became_active":"2024-04-25T00:14:06.856077+0000","last_became_peered":"2024-04-25T00:14:06.856077+0000","last_unstale":"2024-04-25T00:16:26.328438+0000","last_undegraded":"2024-04-25T00:16:26.328438+0000","last_fullsized":"2024-04-25T00:16:26.328438+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:10:04.129533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912513+0000","last_change":"2024-04-25T00:14:01.414599+0000","last_active":"2024-04-25T00:16:26.912513+0000","last_peered":"2024-04-25T00:16:26.912513+0000","last_clean":"2024-04-25T00:16:26.912513+0000","last_became_active":"2024-04-25T00:14:01.413796+0000","last_became_peered":"2024-04-25T00:14:01.413796+0000","last_unstale":"2024-04-25T00:16:26.912513+0000","last_undegraded":"2024-04-25T00:16:26.912513+0000","last_fullsized":"2024-04-25T00:16:26.912513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:44.981848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472240+0000","last_change":"2024-04-25T00:14:01.827718+0000","last_active":"2024-04-25T00:16:26.472240+0000","last_peered":"2024-04-25T00:16:26.472240+0000","last_clean":"2024-04-25T00:16:26.472240+0000","last_became_active":"2024-04-25T00:14:01.827410+0000","last_became_peered":"2024-04-25T00:14:01.827410+0000","last_unstale":"2024-04-25T00:16:26.472240+0000","last_undegraded":"2024-04-25T00:16:26.472240+0000","last_fullsized":"2024-04-25T00:16:26.472240+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:32.428137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328411+0000","last_change":"2024-04-25T00:14:07.323304+0000","last_active":"2024-04-25T00:16:26.328411+0000","last_peered":"2024-04-25T00:16:26.328411+0000","last_clean":"2024-04-25T00:16:26.328411+0000","last_became_active":"2024-04-25T00:14:07.322852+0000","last_became_peered":"2024-04-25T00:14:07.322852+0000","last_unstale":"2024-04-25T00:16:26.328411+0000","last_undegraded":"2024-04-25T00:16:26.328411+0000","last_fullsized":"2024-04-25T00:16:26.328411+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:35.173934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'45","reported_seq":118,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.439157+0000","last_change":"2024-04-25T00:16:03.058662+0000","last_active":"2024-04-25T00:16:26.439157+0000","last_peered":"2024-04-25T00:16:26.439157+0000","last_clean":"2024-04-25T00:16:26.439157+0000","last_became_active":"2024-04-25T00:16:03.054623+0000","last_became_peered":"2024-04-25T00:16:03.054623+0000","last_unstale":"2024-04-25T00:16:26.439157+0000","last_undegraded":"2024-04-25T00:16:26.439157+0000","last_fullsized":"2024-04-25T00:16:26.439157+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:51.610854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.329382+0000","last_change":"2024-04-25T00:14:01.834372+0000","last_active":"2024-04-25T00:16:26.329382+0000","last_peered":"2024-04-25T00:16:26.329382+0000","last_clean":"2024-04-25T00:16:26.329382+0000","last_became_active":"2024-04-25T00:14:01.834062+0000","last_became_peered":"2024-04-25T00:14:01.834062+0000","last_unstale":"2024-04-25T00:16:26.329382+0000","last_undegraded":"2024-04-25T00:16:26.329382+0000","last_fullsized":"2024-04-25T00:16:26.329382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:05.580604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235191+0000","last_change":"2024-04-25T00:14:06.852303+0000","last_active":"2024-04-25T00:16:26.235191+0000","last_peered":"2024-04-25T00:16:26.235191+0000","last_clean":"2024-04-25T00:16:26.235191+0000","last_became_active":"2024-04-25T00:14:06.850957+0000","last_became_peered":"2024-04-25T00:14:06.850957+0000","last_unstale":"2024-04-25T00:16:26.235191+0000","last_undegraded":"2024-04-25T00:16:26.235191+0000","last_fullsized":"2024-04-25T00:16:26.235191+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:47.208880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235559+0000","last_change":"2024-04-25T00:14:01.405538+0000","last_active":"2024-04-25T00:16:26.235559+0000","last_peered":"2024-04-25T00:16:26.235559+0000","last_clean":"2024-04-25T00:16:26.235559+0000","last_became_active":"2024-04-25T00:14:01.405155+0000","last_became_peered":"2024-04-25T00:14:01.405155+0000","last_unstale":"2024-04-25T00:16:26.235559+0000","last_undegraded":"2024-04-25T00:16:26.235559+0000","last_fullsized":"2024-04-25T00:16:26.235559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:20.864753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261840+0000","last_change":"2024-04-25T00:14:01.837058+0000","last_active":"2024-04-25T00:16:26.261840+0000","last_peered":"2024-04-25T00:16:26.261840+0000","last_clean":"2024-04-25T00:16:26.261840+0000","last_became_active":"2024-04-25T00:14:01.835838+0000","last_became_peered":"2024-04-25T00:14:01.835838+0000","last_unstale":"2024-04-25T00:16:26.261840+0000","last_undegraded":"2024-04-25T00:16:26.261840+0000","last_fullsized":"2024-04-25T00:16:26.261840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:21.696448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235583+0000","last_change":"2024-04-25T00:14:07.323870+0000","last_active":"2024-04-25T00:16:26.235583+0000","last_peered":"2024-04-25T00:16:26.235583+0000","last_clean":"2024-04-25T00:16:26.235583+0000","last_became_active":"2024-04-25T00:14:07.323356+0000","last_became_peered":"2024-04-25T00:14:07.323356+0000","last_unstale":"2024-04-25T00:16:26.235583+0000","last_undegraded":"2024-04-25T00:16:26.235583+0000","last_fullsized":"2024-04-25T00:16:26.235583+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:35.160618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472790+0000","last_change":"2024-04-25T00:14:01.409453+0000","last_active":"2024-04-25T00:16:26.472790+0000","last_peered":"2024-04-25T00:16:26.472790+0000","last_clean":"2024-04-25T00:16:26.472790+0000","last_became_active":"2024-04-25T00:14:01.409164+0000","last_became_peered":"2024-04-25T00:14:01.409164+0000","last_unstale":"2024-04-25T00:16:26.472790+0000","last_undegraded":"2024-04-25T00:16:26.472790+0000","last_fullsized":"2024-04-25T00:16:26.472790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:23.266090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.911889+0000","last_change":"2024-04-25T00:14:01.826790+0000","last_active":"2024-04-25T00:16:26.911889+0000","last_peered":"2024-04-25T00:16:26.911889+0000","last_clean":"2024-04-25T00:16:26.911889+0000","last_became_active":"2024-04-25T00:14:01.826457+0000","last_became_peered":"2024-04-25T00:14:01.826457+0000","last_unstale":"2024-04-25T00:16:26.911889+0000","last_undegraded":"2024-04-25T00:16:26.911889+0000","last_fullsized":"2024-04-25T00:16:26.911889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:15.356448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.234393+0000","last_change":"2024-04-25T00:14:06.863280+0000","last_active":"2024-04-25T00:16:26.234393+0000","last_peered":"2024-04-25T00:16:26.234393+0000","last_clean":"2024-04-25T00:16:26.234393+0000","last_became_active":"2024-04-25T00:14:06.862725+0000","last_became_peered":"2024-04-25T00:14:06.862725+0000","last_unstale":"2024-04-25T00:16:26.234393+0000","last_undegraded":"2024-04-25T00:16:26.234393+0000","last_fullsized":"2024-04-25T00:16:26.234393+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:12.791744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328567+0000","last_change":"2024-04-25T00:14:01.414856+0000","last_active":"2024-04-25T00:16:26.328567+0000","last_peered":"2024-04-25T00:16:26.328567+0000","last_clean":"2024-04-25T00:16:26.328567+0000","last_became_active":"2024-04-25T00:14:01.414533+0000","last_became_peered":"2024-04-25T00:14:01.414533+0000","last_unstale":"2024-04-25T00:16:26.328567+0000","last_undegraded":"2024-04-25T00:16:26.328567+0000","last_fullsized":"2024-04-25T00:16:26.328567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:01.090539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328304+0000","last_change":"2024-04-25T00:14:01.831452+0000","last_active":"2024-04-25T00:16:26.328304+0000","last_peered":"2024-04-25T00:16:26.328304+0000","last_clean":"2024-04-25T00:16:26.328304+0000","last_became_active":"2024-04-25T00:14:01.831151+0000","last_became_peered":"2024-04-25T00:14:01.831151+0000","last_unstale":"2024-04-25T00:16:26.328304+0000","last_undegraded":"2024-04-25T00:16:26.328304+0000","last_fullsized":"2024-04-25T00:16:26.328304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:53:07.659508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"92'1","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235539+0000","last_change":"2024-04-25T00:14:06.857601+0000","last_active":"2024-04-25T00:16:26.235539+0000","last_peered":"2024-04-25T00:16:26.235539+0000","last_clean":"2024-04-25T00:16:26.235539+0000","last_became_active":"2024-04-25T00:14:06.856685+0000","last_became_peered":"2024-04-25T00:14:06.856685+0000","last_unstale":"2024-04-25T00:16:26.235539+0000","last_undegraded":"2024-04-25T00:16:26.235539+0000","last_fullsized":"2024-04-25T00:16:26.235539+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:20.250053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328326+0000","last_change":"2024-04-25T00:16:03.060574+0000","last_active":"2024-04-25T00:16:26.328326+0000","last_peered":"2024-04-25T00:16:26.328326+0000","last_clean":"2024-04-25T00:16:26.328326+0000","last_became_active":"2024-04-25T00:16:03.055792+0000","last_became_peered":"2024-04-25T00:16:03.055792+0000","last_unstale":"2024-04-25T00:16:26.328326+0000","last_undegraded":"2024-04-25T00:16:26.328326+0000","last_fullsized":"2024-04-25T00:16:26.328326+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:28.112894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328959+0000","last_change":"2024-04-25T00:14:01.832342+0000","last_active":"2024-04-25T00:16:26.328959+0000","last_peered":"2024-04-25T00:16:26.328959+0000","last_clean":"2024-04-25T00:16:26.328959+0000","last_became_active":"2024-04-25T00:14:01.831997+0000","last_became_peered":"2024-04-25T00:14:01.831997+0000","last_unstale":"2024-04-25T00:16:26.328959+0000","last_undegraded":"2024-04-25T00:16:26.328959+0000","last_fullsized":"2024-04-25T00:16:26.328959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:12.335884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235429+0000","last_change":"2024-04-25T00:14:07.323628+0000","last_active":"2024-04-25T00:16:26.235429+0000","last_peered":"2024-04-25T00:16:26.235429+0000","last_clean":"2024-04-25T00:16:26.235429+0000","last_became_active":"2024-04-25T00:14:07.322996+0000","last_became_peered":"2024-04-25T00:14:07.322996+0000","last_unstale":"2024-04-25T00:16:26.235429+0000","last_undegraded":"2024-04-25T00:16:26.235429+0000","last_fullsized":"2024-04-25T00:16:26.235429+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:32.803216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912115+0000","last_change":"2024-04-25T00:14:01.407715+0000","last_active":"2024-04-25T00:16:26.912115+0000","last_peered":"2024-04-25T00:16:26.912115+0000","last_clean":"2024-04-25T00:16:26.912115+0000","last_became_active":"2024-04-25T00:14:01.407439+0000","last_became_peered":"2024-04-25T00:14:01.407439+0000","last_unstale":"2024-04-25T00:16:26.912115+0000","last_undegraded":"2024-04-25T00:16:26.912115+0000","last_fullsized":"2024-04-25T00:16:26.912115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:33.550379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471832+0000","last_change":"2024-04-25T00:14:01.837733+0000","last_active":"2024-04-25T00:16:26.471832+0000","last_peered":"2024-04-25T00:16:26.471832+0000","last_clean":"2024-04-25T00:16:26.471832+0000","last_became_active":"2024-04-25T00:14:01.837395+0000","last_became_peered":"2024-04-25T00:14:01.837395+0000","last_unstale":"2024-04-25T00:16:26.471832+0000","last_undegraded":"2024-04-25T00:16:26.471832+0000","last_fullsized":"2024-04-25T00:16:26.471832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:50.186834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472922+0000","last_change":"2024-04-25T00:14:06.848700+0000","last_active":"2024-04-25T00:16:26.472922+0000","last_peered":"2024-04-25T00:16:26.472922+0000","last_clean":"2024-04-25T00:16:26.472922+0000","last_became_active":"2024-04-25T00:14:06.848252+0000","last_became_peered":"2024-04-25T00:14:06.848252+0000","last_unstale":"2024-04-25T00:16:26.472922+0000","last_undegraded":"2024-04-25T00:16:26.472922+0000","last_fullsized":"2024-04-25T00:16:26.472922+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:07.653668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235943+0000","last_change":"2024-04-25T00:14:01.405785+0000","last_active":"2024-04-25T00:16:26.235943+0000","last_peered":"2024-04-25T00:16:26.235943+0000","last_clean":"2024-04-25T00:16:26.235943+0000","last_became_active":"2024-04-25T00:14:01.405475+0000","last_became_peered":"2024-04-25T00:14:01.405475+0000","last_unstale":"2024-04-25T00:16:26.235943+0000","last_undegraded":"2024-04-25T00:16:26.235943+0000","last_fullsized":"2024-04-25T00:16:26.235943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:32.487795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472556+0000","last_change":"2024-04-25T00:14:01.832393+0000","last_active":"2024-04-25T00:16:26.472556+0000","last_peered":"2024-04-25T00:16:26.472556+0000","last_clean":"2024-04-25T00:16:26.472556+0000","last_became_active":"2024-04-25T00:14:01.832096+0000","last_became_peered":"2024-04-25T00:14:01.832096+0000","last_unstale":"2024-04-25T00:16:26.472556+0000","last_undegraded":"2024-04-25T00:16:26.472556+0000","last_fullsized":"2024-04-25T00:16:26.472556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:41.749627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261459+0000","last_change":"2024-04-25T00:14:06.855224+0000","last_active":"2024-04-25T00:16:26.261459+0000","last_peered":"2024-04-25T00:16:26.261459+0000","last_clean":"2024-04-25T00:16:26.261459+0000","last_became_active":"2024-04-25T00:14:06.854847+0000","last_became_peered":"2024-04-25T00:14:06.854847+0000","last_unstale":"2024-04-25T00:16:26.261459+0000","last_undegraded":"2024-04-25T00:16:26.261459+0000","last_fullsized":"2024-04-25T00:16:26.261459+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:13.520448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"97'39","reported_seq":85,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.433764+0000","last_change":"2024-04-25T00:14:01.401145+0000","last_active":"2024-04-25T00:16:26.433764+0000","last_peered":"2024-04-25T00:16:26.433764+0000","last_clean":"2024-04-25T00:16:26.433764+0000","last_became_active":"2024-04-25T00:14:01.400807+0000","last_became_peered":"2024-04-25T00:14:01.400807+0000","last_unstale":"2024-04-25T00:16:26.433764+0000","last_undegraded":"2024-04-25T00:16:26.433764+0000","last_fullsized":"2024-04-25T00:16:26.433764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:52:04.674677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"94'6","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472317+0000","last_change":"2024-04-25T00:14:01.838033+0000","last_active":"2024-04-25T00:16:26.472317+0000","last_peered":"2024-04-25T00:16:26.472317+0000","last_clean":"2024-04-25T00:16:26.472317+0000","last_became_active":"2024-04-25T00:14:01.837657+0000","last_became_peered":"2024-04-25T00:14:01.837657+0000","last_unstale":"2024-04-25T00:16:26.472317+0000","last_undegraded":"2024-04-25T00:16:26.472317+0000","last_fullsized":"2024-04-25T00:16:26.472317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:51.895086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.335703+0000","last_change":"2024-04-25T00:14:06.854885+0000","last_active":"2024-04-25T00:16:26.335703+0000","last_peered":"2024-04-25T00:16:26.335703+0000","last_clean":"2024-04-25T00:16:26.335703+0000","last_became_active":"2024-04-25T00:14:06.854200+0000","last_became_peered":"2024-04-25T00:14:06.854200+0000","last_unstale":"2024-04-25T00:16:26.335703+0000","last_undegraded":"2024-04-25T00:16:26.335703+0000","last_fullsized":"2024-04-25T00:16:26.335703+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:43.627742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472665+0000","last_change":"2024-04-25T00:14:01.403789+0000","last_active":"2024-04-25T00:16:26.472665+0000","last_peered":"2024-04-25T00:16:26.472665+0000","last_clean":"2024-04-25T00:16:26.472665+0000","last_became_active":"2024-04-25T00:14:01.403464+0000","last_became_peered":"2024-04-25T00:14:01.403464+0000","last_unstale":"2024-04-25T00:16:26.472665+0000","last_undegraded":"2024-04-25T00:16:26.472665+0000","last_fullsized":"2024-04-25T00:16:26.472665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:48.276536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472074+0000","last_change":"2024-04-25T00:14:01.839988+0000","last_active":"2024-04-25T00:16:26.472074+0000","last_peered":"2024-04-25T00:16:26.472074+0000","last_clean":"2024-04-25T00:16:26.472074+0000","last_became_active":"2024-04-25T00:14:01.839516+0000","last_became_peered":"2024-04-25T00:14:01.839516+0000","last_unstale":"2024-04-25T00:16:26.472074+0000","last_undegraded":"2024-04-25T00:16:26.472074+0000","last_fullsized":"2024-04-25T00:16:26.472074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:05.875127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328368+0000","last_change":"2024-04-25T00:14:01.828638+0000","last_active":"2024-04-25T00:16:26.328368+0000","last_peered":"2024-04-25T00:16:26.328368+0000","last_clean":"2024-04-25T00:16:26.328368+0000","last_became_active":"2024-04-25T00:14:01.828266+0000","last_became_peered":"2024-04-25T00:14:01.828266+0000","last_unstale":"2024-04-25T00:16:26.328368+0000","last_undegraded":"2024-04-25T00:16:26.328368+0000","last_fullsized":"2024-04-25T00:16:26.328368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:34:22.619522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235391+0000","last_change":"2024-04-25T00:14:01.828199+0000","last_active":"2024-04-25T00:16:26.235391+0000","last_peered":"2024-04-25T00:16:26.235391+0000","last_clean":"2024-04-25T00:16:26.235391+0000","last_became_active":"2024-04-25T00:14:01.827579+0000","last_became_peered":"2024-04-25T00:14:01.827579+0000","last_unstale":"2024-04-25T00:16:26.235391+0000","last_undegraded":"2024-04-25T00:16:26.235391+0000","last_fullsized":"2024-04-25T00:16:26.235391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:00.708928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235265+0000","last_change":"2024-04-25T00:14:01.825620+0000","last_active":"2024-04-25T00:16:26.235265+0000","last_peered":"2024-04-25T00:16:26.235265+0000","last_clean":"2024-04-25T00:16:26.235265+0000","last_became_active":"2024-04-25T00:14:01.825310+0000","last_became_peered":"2024-04-25T00:14:01.825310+0000","last_unstale":"2024-04-25T00:16:26.235265+0000","last_undegraded":"2024-04-25T00:16:26.235265+0000","last_fullsized":"2024-04-25T00:16:26.235265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:35.075355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261644+0000","last_change":"2024-04-25T00:14:01.836537+0000","last_active":"2024-04-25T00:16:26.261644+0000","last_peered":"2024-04-25T00:16:26.261644+0000","last_clean":"2024-04-25T00:16:26.261644+0000","last_became_active":"2024-04-25T00:14:01.836234+0000","last_became_peered":"2024-04-25T00:14:01.836234+0000","last_unstale":"2024-04-25T00:16:26.261644+0000","last_undegraded":"2024-04-25T00:16:26.261644+0000","last_fullsized":"2024-04-25T00:16:26.261644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:15.037498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472108+0000","last_change":"2024-04-25T00:14:01.825937+0000","last_active":"2024-04-25T00:16:26.472108+0000","last_peered":"2024-04-25T00:16:26.472108+0000","last_clean":"2024-04-25T00:16:26.472108+0000","last_became_active":"2024-04-25T00:14:01.825605+0000","last_became_peered":"2024-04-25T00:14:01.825605+0000","last_unstale":"2024-04-25T00:16:26.472108+0000","last_undegraded":"2024-04-25T00:16:26.472108+0000","last_fullsized":"2024-04-25T00:16:26.472108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:06.918000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328697+0000","last_change":"2024-04-25T00:14:01.830004+0000","last_active":"2024-04-25T00:16:26.328697+0000","last_peered":"2024-04-25T00:16:26.328697+0000","last_clean":"2024-04-25T00:16:26.328697+0000","last_became_active":"2024-04-25T00:14:01.829588+0000","last_became_peered":"2024-04-25T00:14:01.829588+0000","last_unstale":"2024-04-25T00:16:26.328697+0000","last_undegraded":"2024-04-25T00:16:26.328697+0000","last_fullsized":"2024-04-25T00:16:26.328697+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:08.005995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235672+0000","last_change":"2024-04-25T00:14:01.825762+0000","last_active":"2024-04-25T00:16:26.235672+0000","last_peered":"2024-04-25T00:16:26.235672+0000","last_clean":"2024-04-25T00:16:26.235672+0000","last_became_active":"2024-04-25T00:14:01.825471+0000","last_became_peered":"2024-04-25T00:14:01.825471+0000","last_unstale":"2024-04-25T00:16:26.235672+0000","last_undegraded":"2024-04-25T00:16:26.235672+0000","last_fullsized":"2024-04-25T00:16:26.235672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:33.716875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.329464+0000","last_change":"2024-04-25T00:14:01.836103+0000","last_active":"2024-04-25T00:16:26.329464+0000","last_peered":"2024-04-25T00:16:26.329464+0000","last_clean":"2024-04-25T00:16:26.329464+0000","last_became_active":"2024-04-25T00:14:01.834591+0000","last_became_peered":"2024-04-25T00:14:01.834591+0000","last_unstale":"2024-04-25T00:16:26.329464+0000","last_undegraded":"2024-04-25T00:16:26.329464+0000","last_fullsized":"2024-04-25T00:16:26.329464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:55.130978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.335893+0000","last_change":"2024-04-25T00:14:01.833179+0000","last_active":"2024-04-25T00:16:26.335893+0000","last_peered":"2024-04-25T00:16:26.335893+0000","last_clean":"2024-04-25T00:16:26.335893+0000","last_became_active":"2024-04-25T00:14:01.832856+0000","last_became_peered":"2024-04-25T00:14:01.832856+0000","last_unstale":"2024-04-25T00:16:26.335893+0000","last_undegraded":"2024-04-25T00:16:26.335893+0000","last_fullsized":"2024-04-25T00:16:26.335893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:07.153356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328555+0000","last_change":"2024-04-25T00:14:01.834751+0000","last_active":"2024-04-25T00:16:26.328555+0000","last_peered":"2024-04-25T00:16:26.328555+0000","last_clean":"2024-04-25T00:16:26.328555+0000","last_became_active":"2024-04-25T00:14:01.834413+0000","last_became_peered":"2024-04-25T00:14:01.834413+0000","last_unstale":"2024-04-25T00:16:26.328555+0000","last_undegraded":"2024-04-25T00:16:26.328555+0000","last_fullsized":"2024-04-25T00:16:26.328555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:52.341565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.473303+0000","last_change":"2024-04-25T00:14:01.832944+0000","last_active":"2024-04-25T00:16:26.473303+0000","last_peered":"2024-04-25T00:16:26.473303+0000","last_clean":"2024-04-25T00:16:26.473303+0000","last_became_active":"2024-04-25T00:14:01.832394+0000","last_became_peered":"2024-04-25T00:14:01.832394+0000","last_unstale":"2024-04-25T00:16:26.473303+0000","last_undegraded":"2024-04-25T00:16:26.473303+0000","last_fullsized":"2024-04-25T00:16:26.473303+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:26.957786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472374+0000","last_change":"2024-04-25T00:14:01.831492+0000","last_active":"2024-04-25T00:16:26.472374+0000","last_peered":"2024-04-25T00:16:26.472374+0000","last_clean":"2024-04-25T00:16:26.472374+0000","last_became_active":"2024-04-25T00:14:01.829557+0000","last_became_peered":"2024-04-25T00:14:01.829557+0000","last_unstale":"2024-04-25T00:16:26.472374+0000","last_undegraded":"2024-04-25T00:16:26.472374+0000","last_fullsized":"2024-04-25T00:16:26.472374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:45.116188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328652+0000","last_change":"2024-04-25T00:14:01.839323+0000","last_active":"2024-04-25T00:16:26.328652+0000","last_peered":"2024-04-25T00:16:26.328652+0000","last_clean":"2024-04-25T00:16:26.328652+0000","last_became_active":"2024-04-25T00:14:01.838999+0000","last_became_peered":"2024-04-25T00:14:01.838999+0000","last_unstale":"2024-04-25T00:16:26.328652+0000","last_undegraded":"2024-04-25T00:16:26.328652+0000","last_fullsized":"2024-04-25T00:16:26.328652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:20.652123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599808,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":838,"num_read_kb":532,"num_write":287,"num_write_kb":1441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1799424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351018,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1503232,"data_stored":943251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612654,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37232,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707920,"statfs":{"total":95995035648,"available":95956910080,"internally_reserved":0,"allocated":1941504,"data_stored":1339753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776180,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36828,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708324,"statfs":{"total":95995035648,"available":95957323776,"internally_reserved":0,"allocated":1523712,"data_stored":947583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037817,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":540338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299454,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28684,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716468,"statfs":{"total":95995035648,"available":95965663232,"internally_reserved":0,"allocated":1703936,"data_stored":1136388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462980,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1089536,"data_stored":535594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":599417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1130496,"data_stored":546835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28196,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716956,"statfs":{"total":95995035648,"available":95966162944,"internally_reserved":0,"allocated":1204224,"data_stored":617307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149778,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36944,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708208,"statfs":{"total":95995035648,"available":95957204992,"internally_reserved":0,"allocated":1646592,"data_stored":1070457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888141,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40592,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704560,"statfs":{"total":95995035648,"available":95953469440,"internally_reserved":0,"allocated":1118208,"data_stored":550879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724616,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29016,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716136,"statfs":{"total":95995035648,"available":95965323264,"internally_reserved":0,"allocated":1863680,"data_stored":1273429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":70068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":204028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:16:31.656 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-25T00:16:31.656 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:16:31.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.789+0000 7fe339a38640 1 -- 172.21.15.107:0/4288533857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe334100fd0 msgr2=0x7fe33410d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:31.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.789+0000 7fe339a38640 1 --2- 172.21.15.107:0/4288533857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe334100fd0 0x7fe33410d4e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009cd0 tx=0x7fe32c02a8e0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.789+0000 7fe339a38640 1 -- 172.21.15.107:0/4288533857 shutdown_connections 2024-04-25T00:16:31.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.789+0000 7fe339a38640 1 --2- 172.21.15.107:0/4288533857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe334100fd0 0x7fe33410d4e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.789+0000 7fe339a38640 1 --2- 172.21.15.107:0/4288533857 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe334100610 0x7fe334100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.789+0000 7fe339a38640 1 --2- 172.21.15.107:0/4288533857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3341066b0 0x7fe334106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.789+0000 7fe339a38640 1 -- 172.21.15.107:0/4288533857 >> 172.21.15.107:0/4288533857 conn(0x7fe3340769c0 msgr2=0x7fe334076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:31.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.789+0000 7fe339a38640 1 -- 172.21.15.107:0/4288533857 shutdown_connections 2024-04-25T00:16:31.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.789+0000 7fe339a38640 1 -- 172.21.15.107:0/4288533857 wait complete. 2024-04-25T00:16:31.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.789+0000 7fe339a38640 1 Processor -- start 2024-04-25T00:16:31.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.789+0000 7fe339a38640 1 -- start start 2024-04-25T00:16:31.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.790+0000 7fe339a38640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe334100610 0x7fe334196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:31.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.790+0000 7fe339a38640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe334100fd0 0x7fe334197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:31.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.790+0000 7fe339a38640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3341066b0 0x7fe33419b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:31.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.790+0000 7fe339a38640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe3341109f0 con 0x7fe3341066b0 2024-04-25T00:16:31.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.790+0000 7fe339a38640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe334110870 con 0x7fe334100fd0 2024-04-25T00:16:31.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.790+0000 7fe339a38640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe334110b70 con 0x7fe334100610 2024-04-25T00:16:31.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.790+0000 7fe3337fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3341066b0 0x7fe33419b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:31.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.790+0000 7fe3337fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3341066b0 0x7fe33419b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39076/0 (socket says 172.21.15.107:39076) 2024-04-25T00:16:31.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.790+0000 7fe3337fe640 1 -- 172.21.15.107:0/3689271636 learned_addr learned my addr 172.21.15.107:0/3689271636 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:31.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.790+0000 7fe3327fc640 1 --2- 172.21.15.107:0/3689271636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe334100fd0 0x7fe334197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:31.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.790+0000 7fe332ffd640 1 --2- 172.21.15.107:0/3689271636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe334100610 0x7fe334196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:31.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.790+0000 7fe3337fe640 1 -- 172.21.15.107:0/3689271636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe334100610 msgr2=0x7fe334196e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:31.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.790+0000 7fe3337fe640 1 --2- 172.21.15.107:0/3689271636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe334100610 0x7fe334196e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.790+0000 7fe3337fe640 1 -- 172.21.15.107:0/3689271636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe334100fd0 msgr2=0x7fe334197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:31.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.790+0000 7fe3337fe640 1 --2- 172.21.15.107:0/3689271636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe334100fd0 0x7fe334197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.790+0000 7fe3337fe640 1 -- 172.21.15.107:0/3689271636 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33419be00 con 0x7fe3341066b0 2024-04-25T00:16:31.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.790+0000 7fe3337fe640 1 --2- 172.21.15.107:0/3689271636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3341066b0 0x7fe33419b6d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe32c005cc0 tx=0x7fe32c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:31.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.790+0000 7fe338a36640 1 -- 172.21.15.107:0/3689271636 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c004040 con 0x7fe3341066b0 2024-04-25T00:16:31.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.790+0000 7fe338a36640 1 -- 172.21.15.107:0/3689271636 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe32c0046c0 con 0x7fe3341066b0 2024-04-25T00:16:31.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.790+0000 7fe338a36640 1 -- 172.21.15.107:0/3689271636 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32c030e00 con 0x7fe3341066b0 2024-04-25T00:16:31.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.790+0000 7fe339a38640 1 -- 172.21.15.107:0/3689271636 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe33419c090 con 0x7fe3341066b0 2024-04-25T00:16:31.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.790+0000 7fe339a38640 1 -- 172.21.15.107:0/3689271636 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3341a3980 con 0x7fe3341066b0 2024-04-25T00:16:31.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.791+0000 7fe339a38640 1 -- 172.21.15.107:0/3689271636 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe334101d50 con 0x7fe3341066b0 2024-04-25T00:16:31.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.793+0000 7fe338a36640 1 -- 172.21.15.107:0/3689271636 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7fe32c0305a0 con 0x7fe3341066b0 2024-04-25T00:16:31.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.793+0000 7fe338a36640 1 --2- 172.21.15.107:0/3689271636 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe31c0a2e20 0x7fe31c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:31.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.794+0000 7fe332ffd640 1 --2- 172.21.15.107:0/3689271636 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe31c0a2e20 0x7fe31c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:31.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.794+0000 7fe338a36640 1 -- 172.21.15.107:0/3689271636 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fe32c02e030 con 0x7fe3341066b0 2024-04-25T00:16:31.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.794+0000 7fe332ffd640 1 --2- 172.21.15.107:0/3689271636 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe31c0a2e20 0x7fe31c0a52e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe334101750 tx=0x7fe328005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:31.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.798+0000 7fe338a36640 1 -- 172.21.15.107:0/3689271636 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe32c04b060 con 0x7fe3341066b0 2024-04-25T00:16:31.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.971+0000 7fe339a38640 1 -- 172.21.15.107:0/3689271636 --> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe3340ff4b0 con 0x7fe31c0a2e20 2024-04-25T00:16:31.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.980+0000 7fe338a36640 1 -- 172.21.15.107:0/3689271636 <== mgr.15456 v2:172.21.15.107:6800/1021104172 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339377 (secure 0 0 0) 0x7fe3340ff4b0 con 0x7fe31c0a2e20 2024-04-25T00:16:31.985 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:16:31.988 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-04-25T00:16:31.989 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:31 smithi107 ceph-mon[29153]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:31.989 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:31 smithi107 ceph-mon[29153]: Updating smithi170:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:16:31.989 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:31 smithi107 ceph-mon[29153]: Updating smithi107:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:16:31.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:31 smithi107 ceph-mon[29153]: Updating smithi195:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:16:31.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:31 smithi107 ceph-mon[29153]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-04-25T00:16:31.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:31 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:31.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:31 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:31.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:31 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:31.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:31 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:31.990 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:31 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1132749210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:31.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.984+0000 7fe339a38640 1 -- 172.21.15.107:0/3689271636 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe31c0a2e20 msgr2=0x7fe31c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:31.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.984+0000 7fe339a38640 1 --2- 172.21.15.107:0/3689271636 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe31c0a2e20 0x7fe31c0a52e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe334101750 tx=0x7fe328005f90 comp rx=0 tx=0).stop 2024-04-25T00:16:31.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.984+0000 7fe339a38640 1 -- 172.21.15.107:0/3689271636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3341066b0 msgr2=0x7fe33419b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:31.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.984+0000 7fe339a38640 1 --2- 172.21.15.107:0/3689271636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3341066b0 0x7fe33419b6d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe32c005cc0 tx=0x7fe32c032f40 comp rx=0 tx=0).stop 2024-04-25T00:16:31.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.984+0000 7fe339a38640 1 -- 172.21.15.107:0/3689271636 shutdown_connections 2024-04-25T00:16:31.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.984+0000 7fe339a38640 1 --2- 172.21.15.107:0/3689271636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3341066b0 0x7fe33419b6d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.984+0000 7fe339a38640 1 --2- 172.21.15.107:0/3689271636 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe31c0a2e20 0x7fe31c0a52e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.984+0000 7fe339a38640 1 --2- 172.21.15.107:0/3689271636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe334100fd0 0x7fe334197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.984+0000 7fe339a38640 1 --2- 172.21.15.107:0/3689271636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe334100610 0x7fe334196e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:31.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.984+0000 7fe339a38640 1 -- 172.21.15.107:0/3689271636 >> 172.21.15.107:0/3689271636 conn(0x7fe3340769c0 msgr2=0x7fe3340feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:31.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.984+0000 7fe339a38640 1 -- 172.21.15.107:0/3689271636 shutdown_connections 2024-04-25T00:16:31.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:31.984+0000 7fe339a38640 1 -- 172.21.15.107:0/3689271636 wait complete. 2024-04-25T00:16:31.996 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-25T00:16:30.246986+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1190410,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":974,"num_read_kb":651,"num_write":470,"num_write_kb":4381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":395080,"kb_used_data":16596,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124546744,"statfs":{"total":1151940427776,"available":1151535865856,"internally_reserved":0,"allocated":16994304,"data_stored":10101231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.987708"},"pg_stats":[{"pgid":"3.3f","version":"94'8","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328766+0000","last_change":"2024-04-25T00:14:01.835737+0000","last_active":"2024-04-25T00:16:26.328766+0000","last_peered":"2024-04-25T00:16:26.328766+0000","last_clean":"2024-04-25T00:16:26.328766+0000","last_became_active":"2024-04-25T00:14:01.835428+0000","last_became_peered":"2024-04-25T00:14:01.835428+0000","last_unstale":"2024-04-25T00:16:26.328766+0000","last_undegraded":"2024-04-25T00:16:26.328766+0000","last_fullsized":"2024-04-25T00:16:26.328766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:33.974656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261738+0000","last_change":"2024-04-25T00:14:01.837321+0000","last_active":"2024-04-25T00:16:26.261738+0000","last_peered":"2024-04-25T00:16:26.261738+0000","last_clean":"2024-04-25T00:16:26.261738+0000","last_became_active":"2024-04-25T00:14:01.836762+0000","last_became_peered":"2024-04-25T00:14:01.836762+0000","last_unstale":"2024-04-25T00:16:26.261738+0000","last_undegraded":"2024-04-25T00:16:26.261738+0000","last_fullsized":"2024-04-25T00:16:26.261738+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:34:17.342967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261705+0000","last_change":"2024-04-25T00:14:01.834504+0000","last_active":"2024-04-25T00:16:26.261705+0000","last_peered":"2024-04-25T00:16:26.261705+0000","last_clean":"2024-04-25T00:16:26.261705+0000","last_became_active":"2024-04-25T00:14:01.834182+0000","last_became_peered":"2024-04-25T00:14:01.834182+0000","last_unstale":"2024-04-25T00:16:26.261705+0000","last_undegraded":"2024-04-25T00:16:26.261705+0000","last_fullsized":"2024-04-25T00:16:26.261705+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:12.991170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":25,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.234081+0000","last_change":"2024-04-25T00:15:02.080211+0000","last_active":"2024-04-25T00:16:26.234081+0000","last_peered":"2024-04-25T00:16:26.234081+0000","last_clean":"2024-04-25T00:16:26.234081+0000","last_became_active":"2024-04-25T00:15:02.079849+0000","last_became_peered":"2024-04-25T00:15:02.079849+0000","last_unstale":"2024-04-25T00:16:26.234081+0000","last_undegraded":"2024-04-25T00:16:26.234081+0000","last_fullsized":"2024-04-25T00:16:26.234081+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:56.804782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912156+0000","last_change":"2024-04-25T00:14:01.834173+0000","last_active":"2024-04-25T00:16:26.912156+0000","last_peered":"2024-04-25T00:16:26.912156+0000","last_clean":"2024-04-25T00:16:26.912156+0000","last_became_active":"2024-04-25T00:14:01.833835+0000","last_became_peered":"2024-04-25T00:14:01.833835+0000","last_unstale":"2024-04-25T00:16:26.912156+0000","last_undegraded":"2024-04-25T00:16:26.912156+0000","last_fullsized":"2024-04-25T00:16:26.912156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:30.046629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.338934+0000","last_change":"2024-04-25T00:14:01.832113+0000","last_active":"2024-04-25T00:16:26.338934+0000","last_peered":"2024-04-25T00:16:26.338934+0000","last_clean":"2024-04-25T00:16:26.338934+0000","last_became_active":"2024-04-25T00:14:01.831800+0000","last_became_peered":"2024-04-25T00:14:01.831800+0000","last_unstale":"2024-04-25T00:16:26.338934+0000","last_undegraded":"2024-04-25T00:16:26.338934+0000","last_fullsized":"2024-04-25T00:16:26.338934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:06:54.469994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472140+0000","last_change":"2024-04-25T00:14:01.839166+0000","last_active":"2024-04-25T00:16:26.472140+0000","last_peered":"2024-04-25T00:16:26.472140+0000","last_clean":"2024-04-25T00:16:26.472140+0000","last_became_active":"2024-04-25T00:14:01.838851+0000","last_became_peered":"2024-04-25T00:14:01.838851+0000","last_unstale":"2024-04-25T00:16:26.472140+0000","last_undegraded":"2024-04-25T00:16:26.472140+0000","last_fullsized":"2024-04-25T00:16:26.472140+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:39:48.125363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":26,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.233734+0000","last_change":"2024-04-25T00:15:02.083386+0000","last_active":"2024-04-25T00:16:26.233734+0000","last_peered":"2024-04-25T00:16:26.233734+0000","last_clean":"2024-04-25T00:16:26.233734+0000","last_became_active":"2024-04-25T00:15:02.083106+0000","last_became_peered":"2024-04-25T00:15:02.083106+0000","last_unstale":"2024-04-25T00:16:26.233734+0000","last_undegraded":"2024-04-25T00:16:26.233734+0000","last_fullsized":"2024-04-25T00:16:26.233734+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:58.543667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.473534+0000","last_change":"2024-04-25T00:14:01.829803+0000","last_active":"2024-04-25T00:16:26.473534+0000","last_peered":"2024-04-25T00:16:26.473534+0000","last_clean":"2024-04-25T00:16:26.473534+0000","last_became_active":"2024-04-25T00:14:01.827761+0000","last_became_peered":"2024-04-25T00:14:01.827761+0000","last_unstale":"2024-04-25T00:16:26.473534+0000","last_undegraded":"2024-04-25T00:16:26.473534+0000","last_fullsized":"2024-04-25T00:16:26.473534+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:38.862505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.339417+0000","last_change":"2024-04-25T00:14:01.834269+0000","last_active":"2024-04-25T00:16:26.339417+0000","last_peered":"2024-04-25T00:16:26.339417+0000","last_clean":"2024-04-25T00:16:26.339417+0000","last_became_active":"2024-04-25T00:14:01.833964+0000","last_became_peered":"2024-04-25T00:14:01.833964+0000","last_unstale":"2024-04-25T00:16:26.339417+0000","last_undegraded":"2024-04-25T00:16:26.339417+0000","last_fullsized":"2024-04-25T00:16:26.339417+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:29.501513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472579+0000","last_change":"2024-04-25T00:14:01.839510+0000","last_active":"2024-04-25T00:16:26.472579+0000","last_peered":"2024-04-25T00:16:26.472579+0000","last_clean":"2024-04-25T00:16:26.472579+0000","last_became_active":"2024-04-25T00:14:01.839000+0000","last_became_peered":"2024-04-25T00:14:01.839000+0000","last_unstale":"2024-04-25T00:16:26.472579+0000","last_undegraded":"2024-04-25T00:16:26.472579+0000","last_fullsized":"2024-04-25T00:16:26.472579+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:20.809277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235608+0000","last_change":"2024-04-25T00:14:01.835821+0000","last_active":"2024-04-25T00:16:26.235608+0000","last_peered":"2024-04-25T00:16:26.235608+0000","last_clean":"2024-04-25T00:16:26.235608+0000","last_became_active":"2024-04-25T00:14:01.835525+0000","last_became_peered":"2024-04-25T00:14:01.835525+0000","last_unstale":"2024-04-25T00:16:26.235608+0000","last_undegraded":"2024-04-25T00:16:26.235608+0000","last_fullsized":"2024-04-25T00:16:26.235608+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:17.082562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472256+0000","last_change":"2024-04-25T00:14:01.831564+0000","last_active":"2024-04-25T00:16:26.472256+0000","last_peered":"2024-04-25T00:16:26.472256+0000","last_clean":"2024-04-25T00:16:26.472256+0000","last_became_active":"2024-04-25T00:14:01.828539+0000","last_became_peered":"2024-04-25T00:14:01.828539+0000","last_unstale":"2024-04-25T00:16:26.472256+0000","last_undegraded":"2024-04-25T00:16:26.472256+0000","last_fullsized":"2024-04-25T00:16:26.472256+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:41:52.098299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261510+0000","last_change":"2024-04-25T00:14:01.835815+0000","last_active":"2024-04-25T00:16:26.261510+0000","last_peered":"2024-04-25T00:16:26.261510+0000","last_clean":"2024-04-25T00:16:26.261510+0000","last_became_active":"2024-04-25T00:14:01.835498+0000","last_became_peered":"2024-04-25T00:14:01.835498+0000","last_unstale":"2024-04-25T00:16:26.261510+0000","last_undegraded":"2024-04-25T00:16:26.261510+0000","last_fullsized":"2024-04-25T00:16:26.261510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:14:02.914149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261465+0000","last_change":"2024-04-25T00:14:01.839542+0000","last_active":"2024-04-25T00:16:26.261465+0000","last_peered":"2024-04-25T00:16:26.261465+0000","last_clean":"2024-04-25T00:16:26.261465+0000","last_became_active":"2024-04-25T00:14:01.839234+0000","last_became_peered":"2024-04-25T00:14:01.839234+0000","last_unstale":"2024-04-25T00:16:26.261465+0000","last_undegraded":"2024-04-25T00:16:26.261465+0000","last_fullsized":"2024-04-25T00:16:26.261465+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:13:00.437311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472718+0000","last_change":"2024-04-25T00:14:01.832265+0000","last_active":"2024-04-25T00:16:26.472718+0000","last_peered":"2024-04-25T00:16:26.472718+0000","last_clean":"2024-04-25T00:16:26.472718+0000","last_became_active":"2024-04-25T00:14:01.831963+0000","last_became_peered":"2024-04-25T00:14:01.831963+0000","last_unstale":"2024-04-25T00:16:26.472718+0000","last_undegraded":"2024-04-25T00:16:26.472718+0000","last_fullsized":"2024-04-25T00:16:26.472718+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:55.402265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328586+0000","last_change":"2024-04-25T00:14:01.835444+0000","last_active":"2024-04-25T00:16:26.328586+0000","last_peered":"2024-04-25T00:16:26.328586+0000","last_clean":"2024-04-25T00:16:26.328586+0000","last_became_active":"2024-04-25T00:14:01.835126+0000","last_became_peered":"2024-04-25T00:14:01.835126+0000","last_unstale":"2024-04-25T00:16:26.328586+0000","last_undegraded":"2024-04-25T00:16:26.328586+0000","last_fullsized":"2024-04-25T00:16:26.328586+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:04:48.997924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.234333+0000","last_change":"2024-04-25T00:14:01.834982+0000","last_active":"2024-04-25T00:16:26.234333+0000","last_peered":"2024-04-25T00:16:26.234333+0000","last_clean":"2024-04-25T00:16:26.234333+0000","last_became_active":"2024-04-25T00:14:01.834671+0000","last_became_peered":"2024-04-25T00:14:01.834671+0000","last_unstale":"2024-04-25T00:16:26.234333+0000","last_undegraded":"2024-04-25T00:16:26.234333+0000","last_fullsized":"2024-04-25T00:16:26.234333+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:28.254408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235773+0000","last_change":"2024-04-25T00:14:01.824432+0000","last_active":"2024-04-25T00:16:26.235773+0000","last_peered":"2024-04-25T00:16:26.235773+0000","last_clean":"2024-04-25T00:16:26.235773+0000","last_became_active":"2024-04-25T00:14:01.823963+0000","last_became_peered":"2024-04-25T00:14:01.823963+0000","last_unstale":"2024-04-25T00:16:26.235773+0000","last_undegraded":"2024-04-25T00:16:26.235773+0000","last_fullsized":"2024-04-25T00:16:26.235773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:35.880932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235322+0000","last_change":"2024-04-25T00:14:06.861892+0000","last_active":"2024-04-25T00:16:26.235322+0000","last_peered":"2024-04-25T00:16:26.235322+0000","last_clean":"2024-04-25T00:16:26.235322+0000","last_became_active":"2024-04-25T00:14:06.861470+0000","last_became_peered":"2024-04-25T00:14:06.861470+0000","last_unstale":"2024-04-25T00:16:26.235322+0000","last_undegraded":"2024-04-25T00:16:26.235322+0000","last_fullsized":"2024-04-25T00:16:26.235322+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:58:06.192954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.234449+0000","last_change":"2024-04-25T00:14:01.412535+0000","last_active":"2024-04-25T00:16:26.234449+0000","last_peered":"2024-04-25T00:16:26.234449+0000","last_clean":"2024-04-25T00:16:26.234449+0000","last_became_active":"2024-04-25T00:14:01.412206+0000","last_became_peered":"2024-04-25T00:14:01.412206+0000","last_unstale":"2024-04-25T00:16:26.234449+0000","last_undegraded":"2024-04-25T00:16:26.234449+0000","last_fullsized":"2024-04-25T00:16:26.234449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:03:40.893987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.329130+0000","last_change":"2024-04-25T00:14:01.840814+0000","last_active":"2024-04-25T00:16:26.329130+0000","last_peered":"2024-04-25T00:16:26.329130+0000","last_clean":"2024-04-25T00:16:26.329130+0000","last_became_active":"2024-04-25T00:14:01.840179+0000","last_became_peered":"2024-04-25T00:14:01.840179+0000","last_unstale":"2024-04-25T00:16:26.329130+0000","last_undegraded":"2024-04-25T00:16:26.329130+0000","last_fullsized":"2024-04-25T00:16:26.329130+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:27.383918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235154+0000","last_change":"2024-04-25T00:14:06.857877+0000","last_active":"2024-04-25T00:16:26.235154+0000","last_peered":"2024-04-25T00:16:26.235154+0000","last_clean":"2024-04-25T00:16:26.235154+0000","last_became_active":"2024-04-25T00:14:06.857420+0000","last_became_peered":"2024-04-25T00:14:06.857420+0000","last_unstale":"2024-04-25T00:16:26.235154+0000","last_undegraded":"2024-04-25T00:16:26.235154+0000","last_fullsized":"2024-04-25T00:16:26.235154+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:22:54.424024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.329451+0000","last_change":"2024-04-25T00:14:01.416560+0000","last_active":"2024-04-25T00:16:26.329451+0000","last_peered":"2024-04-25T00:16:26.329451+0000","last_clean":"2024-04-25T00:16:26.329451+0000","last_became_active":"2024-04-25T00:14:01.416246+0000","last_became_peered":"2024-04-25T00:14:01.416246+0000","last_unstale":"2024-04-25T00:16:26.329451+0000","last_undegraded":"2024-04-25T00:16:26.329451+0000","last_fullsized":"2024-04-25T00:16:26.329451+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:03:47.938452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261742+0000","last_change":"2024-04-25T00:14:01.832607+0000","last_active":"2024-04-25T00:16:26.261742+0000","last_peered":"2024-04-25T00:16:26.261742+0000","last_clean":"2024-04-25T00:16:26.261742+0000","last_became_active":"2024-04-25T00:14:01.832300+0000","last_became_peered":"2024-04-25T00:14:01.832300+0000","last_unstale":"2024-04-25T00:16:26.261742+0000","last_undegraded":"2024-04-25T00:16:26.261742+0000","last_fullsized":"2024-04-25T00:16:26.261742+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:26:22.045171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235355+0000","last_change":"2024-04-25T00:14:06.863642+0000","last_active":"2024-04-25T00:16:26.235355+0000","last_peered":"2024-04-25T00:16:26.235355+0000","last_clean":"2024-04-25T00:16:26.235355+0000","last_became_active":"2024-04-25T00:14:06.863284+0000","last_became_peered":"2024-04-25T00:14:06.863284+0000","last_unstale":"2024-04-25T00:16:26.235355+0000","last_undegraded":"2024-04-25T00:16:26.235355+0000","last_fullsized":"2024-04-25T00:16:26.235355+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:11:20.818625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912488+0000","last_change":"2024-04-25T00:14:01.411874+0000","last_active":"2024-04-25T00:16:26.912488+0000","last_peered":"2024-04-25T00:16:26.912488+0000","last_clean":"2024-04-25T00:16:26.912488+0000","last_became_active":"2024-04-25T00:14:01.409568+0000","last_became_peered":"2024-04-25T00:14:01.409568+0000","last_unstale":"2024-04-25T00:16:26.912488+0000","last_undegraded":"2024-04-25T00:16:26.912488+0000","last_fullsized":"2024-04-25T00:16:26.912488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:22.415631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.473104+0000","last_change":"2024-04-25T00:14:01.822471+0000","last_active":"2024-04-25T00:16:26.473104+0000","last_peered":"2024-04-25T00:16:26.473104+0000","last_clean":"2024-04-25T00:16:26.473104+0000","last_became_active":"2024-04-25T00:14:01.822181+0000","last_became_peered":"2024-04-25T00:14:01.822181+0000","last_unstale":"2024-04-25T00:16:26.473104+0000","last_undegraded":"2024-04-25T00:16:26.473104+0000","last_fullsized":"2024-04-25T00:16:26.473104+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:21:44.857036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235154+0000","last_change":"2024-04-25T00:14:06.856987+0000","last_active":"2024-04-25T00:16:26.235154+0000","last_peered":"2024-04-25T00:16:26.235154+0000","last_clean":"2024-04-25T00:16:26.235154+0000","last_became_active":"2024-04-25T00:14:06.856053+0000","last_became_peered":"2024-04-25T00:14:06.856053+0000","last_unstale":"2024-04-25T00:16:26.235154+0000","last_undegraded":"2024-04-25T00:16:26.235154+0000","last_fullsized":"2024-04-25T00:16:26.235154+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:16.829969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912190+0000","last_change":"2024-04-25T00:14:01.410156+0000","last_active":"2024-04-25T00:16:26.912190+0000","last_peered":"2024-04-25T00:16:26.912190+0000","last_clean":"2024-04-25T00:16:26.912190+0000","last_became_active":"2024-04-25T00:14:01.409809+0000","last_became_peered":"2024-04-25T00:14:01.409809+0000","last_unstale":"2024-04-25T00:16:26.912190+0000","last_undegraded":"2024-04-25T00:16:26.912190+0000","last_fullsized":"2024-04-25T00:16:26.912190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:43.160419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.336265+0000","last_change":"2024-04-25T00:14:01.836090+0000","last_active":"2024-04-25T00:16:26.336265+0000","last_peered":"2024-04-25T00:16:26.336265+0000","last_clean":"2024-04-25T00:16:26.336265+0000","last_became_active":"2024-04-25T00:14:01.835733+0000","last_became_peered":"2024-04-25T00:14:01.835733+0000","last_unstale":"2024-04-25T00:16:26.336265+0000","last_undegraded":"2024-04-25T00:16:26.336265+0000","last_fullsized":"2024-04-25T00:16:26.336265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:02.368893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235589+0000","last_change":"2024-04-25T00:14:06.851496+0000","last_active":"2024-04-25T00:16:26.235589+0000","last_peered":"2024-04-25T00:16:26.235589+0000","last_clean":"2024-04-25T00:16:26.235589+0000","last_became_active":"2024-04-25T00:14:06.851007+0000","last_became_peered":"2024-04-25T00:14:06.851007+0000","last_unstale":"2024-04-25T00:16:26.235589+0000","last_undegraded":"2024-04-25T00:16:26.235589+0000","last_fullsized":"2024-04-25T00:16:26.235589+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:50.543348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'7","reported_seq":167,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.336089+0000","last_change":"2024-04-25T00:14:01.414760+0000","last_active":"2024-04-25T00:16:26.336089+0000","last_peered":"2024-04-25T00:16:26.336089+0000","last_clean":"2024-04-25T00:16:26.336089+0000","last_became_active":"2024-04-25T00:14:01.414482+0000","last_became_peered":"2024-04-25T00:14:01.414482+0000","last_unstale":"2024-04-25T00:16:26.336089+0000","last_undegraded":"2024-04-25T00:16:26.336089+0000","last_fullsized":"2024-04-25T00:16:26.336089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:47:32.631933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912017+0000","last_change":"2024-04-25T00:14:01.832516+0000","last_active":"2024-04-25T00:16:26.912017+0000","last_peered":"2024-04-25T00:16:26.912017+0000","last_clean":"2024-04-25T00:16:26.912017+0000","last_became_active":"2024-04-25T00:14:01.832106+0000","last_became_peered":"2024-04-25T00:14:01.832106+0000","last_unstale":"2024-04-25T00:16:26.912017+0000","last_undegraded":"2024-04-25T00:16:26.912017+0000","last_fullsized":"2024-04-25T00:16:26.912017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:55.590486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235703+0000","last_change":"2024-04-25T00:14:06.861984+0000","last_active":"2024-04-25T00:16:26.235703+0000","last_peered":"2024-04-25T00:16:26.235703+0000","last_clean":"2024-04-25T00:16:26.235703+0000","last_became_active":"2024-04-25T00:14:06.861418+0000","last_became_peered":"2024-04-25T00:14:06.861418+0000","last_unstale":"2024-04-25T00:16:26.235703+0000","last_undegraded":"2024-04-25T00:16:26.235703+0000","last_fullsized":"2024-04-25T00:16:26.235703+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:27.559704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.338926+0000","last_change":"2024-04-25T00:14:01.407648+0000","last_active":"2024-04-25T00:16:26.338926+0000","last_peered":"2024-04-25T00:16:26.338926+0000","last_clean":"2024-04-25T00:16:26.338926+0000","last_became_active":"2024-04-25T00:14:01.407318+0000","last_became_peered":"2024-04-25T00:14:01.407318+0000","last_unstale":"2024-04-25T00:16:26.338926+0000","last_undegraded":"2024-04-25T00:16:26.338926+0000","last_fullsized":"2024-04-25T00:16:26.338926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:08:10.942814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.336183+0000","last_change":"2024-04-25T00:14:01.833073+0000","last_active":"2024-04-25T00:16:26.336183+0000","last_peered":"2024-04-25T00:16:26.336183+0000","last_clean":"2024-04-25T00:16:26.336183+0000","last_became_active":"2024-04-25T00:14:01.832545+0000","last_became_peered":"2024-04-25T00:14:01.832545+0000","last_unstale":"2024-04-25T00:16:26.336183+0000","last_undegraded":"2024-04-25T00:16:26.336183+0000","last_fullsized":"2024-04-25T00:16:26.336183+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:39.772781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328531+0000","last_change":"2024-04-25T00:14:06.862011+0000","last_active":"2024-04-25T00:16:26.328531+0000","last_peered":"2024-04-25T00:16:26.328531+0000","last_clean":"2024-04-25T00:16:26.328531+0000","last_became_active":"2024-04-25T00:14:06.861544+0000","last_became_peered":"2024-04-25T00:14:06.861544+0000","last_unstale":"2024-04-25T00:16:26.328531+0000","last_undegraded":"2024-04-25T00:16:26.328531+0000","last_fullsized":"2024-04-25T00:16:26.328531+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:37.736266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471570+0000","last_change":"2024-04-25T00:14:01.397747+0000","last_active":"2024-04-25T00:16:26.471570+0000","last_peered":"2024-04-25T00:16:26.471570+0000","last_clean":"2024-04-25T00:16:26.471570+0000","last_became_active":"2024-04-25T00:14:01.397446+0000","last_became_peered":"2024-04-25T00:14:01.397446+0000","last_unstale":"2024-04-25T00:16:26.471570+0000","last_undegraded":"2024-04-25T00:16:26.471570+0000","last_fullsized":"2024-04-25T00:16:26.471570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:27.240034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328414+0000","last_change":"2024-04-25T00:14:01.833128+0000","last_active":"2024-04-25T00:16:26.328414+0000","last_peered":"2024-04-25T00:16:26.328414+0000","last_clean":"2024-04-25T00:16:26.328414+0000","last_became_active":"2024-04-25T00:14:01.832780+0000","last_became_peered":"2024-04-25T00:14:01.832780+0000","last_unstale":"2024-04-25T00:16:26.328414+0000","last_undegraded":"2024-04-25T00:16:26.328414+0000","last_fullsized":"2024-04-25T00:16:26.328414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:13:43.023737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261698+0000","last_change":"2024-04-25T00:14:06.860436+0000","last_active":"2024-04-25T00:16:26.261698+0000","last_peered":"2024-04-25T00:16:26.261698+0000","last_clean":"2024-04-25T00:16:26.261698+0000","last_became_active":"2024-04-25T00:14:06.860047+0000","last_became_peered":"2024-04-25T00:14:06.860047+0000","last_unstale":"2024-04-25T00:16:26.261698+0000","last_undegraded":"2024-04-25T00:16:26.261698+0000","last_fullsized":"2024-04-25T00:16:26.261698+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:08:41.246549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":39,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.335972+0000","last_change":"2024-04-25T00:14:01.415939+0000","last_active":"2024-04-25T00:16:26.335972+0000","last_peered":"2024-04-25T00:16:26.335972+0000","last_clean":"2024-04-25T00:16:26.335972+0000","last_became_active":"2024-04-25T00:14:01.415513+0000","last_became_peered":"2024-04-25T00:14:01.415513+0000","last_unstale":"2024-04-25T00:16:26.335972+0000","last_undegraded":"2024-04-25T00:16:26.335972+0000","last_fullsized":"2024-04-25T00:16:26.335972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:57:23.588868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.233864+0000","last_change":"2024-04-25T00:14:01.838793+0000","last_active":"2024-04-25T00:16:26.233864+0000","last_peered":"2024-04-25T00:16:26.233864+0000","last_clean":"2024-04-25T00:16:26.233864+0000","last_became_active":"2024-04-25T00:14:01.838453+0000","last_became_peered":"2024-04-25T00:14:01.838453+0000","last_unstale":"2024-04-25T00:16:26.233864+0000","last_undegraded":"2024-04-25T00:16:26.233864+0000","last_fullsized":"2024-04-25T00:16:26.233864+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:18.674009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328492+0000","last_change":"2024-04-25T00:14:06.856431+0000","last_active":"2024-04-25T00:16:26.328492+0000","last_peered":"2024-04-25T00:16:26.328492+0000","last_clean":"2024-04-25T00:16:26.328492+0000","last_became_active":"2024-04-25T00:14:06.856047+0000","last_became_peered":"2024-04-25T00:14:06.856047+0000","last_unstale":"2024-04-25T00:16:26.328492+0000","last_undegraded":"2024-04-25T00:16:26.328492+0000","last_fullsized":"2024-04-25T00:16:26.328492+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:13:20.646673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":160,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.339454+0000","last_change":"2024-04-25T00:14:01.399054+0000","last_active":"2024-04-25T00:16:26.339454+0000","last_peered":"2024-04-25T00:16:26.339454+0000","last_clean":"2024-04-25T00:16:26.339454+0000","last_became_active":"2024-04-25T00:14:01.398711+0000","last_became_peered":"2024-04-25T00:14:01.398711+0000","last_unstale":"2024-04-25T00:16:26.339454+0000","last_undegraded":"2024-04-25T00:16:26.339454+0000","last_fullsized":"2024-04-25T00:16:26.339454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:26.434014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.473480+0000","last_change":"2024-04-25T00:14:01.829132+0000","last_active":"2024-04-25T00:16:26.473480+0000","last_peered":"2024-04-25T00:16:26.473480+0000","last_clean":"2024-04-25T00:16:26.473480+0000","last_became_active":"2024-04-25T00:14:01.828446+0000","last_became_peered":"2024-04-25T00:14:01.828446+0000","last_unstale":"2024-04-25T00:16:26.473480+0000","last_undegraded":"2024-04-25T00:16:26.473480+0000","last_fullsized":"2024-04-25T00:16:26.473480+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:35.788478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.233990+0000","last_change":"2024-04-25T00:14:06.861884+0000","last_active":"2024-04-25T00:16:26.233990+0000","last_peered":"2024-04-25T00:16:26.233990+0000","last_clean":"2024-04-25T00:16:26.233990+0000","last_became_active":"2024-04-25T00:14:06.861413+0000","last_became_peered":"2024-04-25T00:14:06.861413+0000","last_unstale":"2024-04-25T00:16:26.233990+0000","last_undegraded":"2024-04-25T00:16:26.233990+0000","last_fullsized":"2024-04-25T00:16:26.233990+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:08:53.876308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":49,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471856+0000","last_change":"2024-04-25T00:16:03.325276+0000","last_active":"2024-04-25T00:16:26.471856+0000","last_peered":"2024-04-25T00:16:26.471856+0000","last_clean":"2024-04-25T00:16:26.471856+0000","last_became_active":"2024-04-25T00:16:03.322660+0000","last_became_peered":"2024-04-25T00:16:03.322660+0000","last_unstale":"2024-04-25T00:16:26.471856+0000","last_undegraded":"2024-04-25T00:16:26.471856+0000","last_fullsized":"2024-04-25T00:16:26.471856+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:55.882768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261931+0000","last_change":"2024-04-25T00:15:02.372269+0000","last_active":"2024-04-25T00:16:26.261931+0000","last_peered":"2024-04-25T00:16:26.261931+0000","last_clean":"2024-04-25T00:16:26.261931+0000","last_became_active":"2024-04-25T00:15:02.371962+0000","last_became_peered":"2024-04-25T00:15:02.371962+0000","last_unstale":"2024-04-25T00:16:26.261931+0000","last_undegraded":"2024-04-25T00:16:26.261931+0000","last_fullsized":"2024-04-25T00:16:26.261931+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:46.597638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":34,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235686+0000","last_change":"2024-04-25T00:15:02.372941+0000","last_active":"2024-04-25T00:16:26.235686+0000","last_peered":"2024-04-25T00:16:26.235686+0000","last_clean":"2024-04-25T00:16:26.235686+0000","last_became_active":"2024-04-25T00:15:02.372504+0000","last_became_peered":"2024-04-25T00:15:02.372504+0000","last_unstale":"2024-04-25T00:16:26.235686+0000","last_undegraded":"2024-04-25T00:16:26.235686+0000","last_fullsized":"2024-04-25T00:16:26.235686+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:44:58.202708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,8],"acting":[4,1,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.336237+0000","last_change":"2024-04-25T00:14:01.401128+0000","last_active":"2024-04-25T00:16:26.336237+0000","last_peered":"2024-04-25T00:16:26.336237+0000","last_clean":"2024-04-25T00:16:26.336237+0000","last_became_active":"2024-04-25T00:14:01.400125+0000","last_became_peered":"2024-04-25T00:14:01.400125+0000","last_unstale":"2024-04-25T00:16:26.336237+0000","last_undegraded":"2024-04-25T00:16:26.336237+0000","last_fullsized":"2024-04-25T00:16:26.336237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:38:43.787855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.473511+0000","last_change":"2024-04-25T00:14:01.830649+0000","last_active":"2024-04-25T00:16:26.473511+0000","last_peered":"2024-04-25T00:16:26.473511+0000","last_clean":"2024-04-25T00:16:26.473511+0000","last_became_active":"2024-04-25T00:14:01.830253+0000","last_became_peered":"2024-04-25T00:14:01.830253+0000","last_unstale":"2024-04-25T00:16:26.473511+0000","last_undegraded":"2024-04-25T00:16:26.473511+0000","last_fullsized":"2024-04-25T00:16:26.473511+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:37.474833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.233945+0000","last_change":"2024-04-25T00:14:06.861275+0000","last_active":"2024-04-25T00:16:26.233945+0000","last_peered":"2024-04-25T00:16:26.233945+0000","last_clean":"2024-04-25T00:16:26.233945+0000","last_became_active":"2024-04-25T00:14:06.860891+0000","last_became_peered":"2024-04-25T00:14:06.860891+0000","last_unstale":"2024-04-25T00:16:26.233945+0000","last_undegraded":"2024-04-25T00:16:26.233945+0000","last_fullsized":"2024-04-25T00:16:26.233945+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:09:38.685837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"97'40","reported_seq":327,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.726401+0000","last_change":"2024-04-25T00:14:01.404706+0000","last_active":"2024-04-25T00:16:26.726401+0000","last_peered":"2024-04-25T00:16:26.726401+0000","last_clean":"2024-04-25T00:16:26.726401+0000","last_became_active":"2024-04-25T00:14:01.404231+0000","last_became_peered":"2024-04-25T00:14:01.404231+0000","last_unstale":"2024-04-25T00:16:26.726401+0000","last_undegraded":"2024-04-25T00:16:26.726401+0000","last_fullsized":"2024-04-25T00:16:26.726401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:36.437491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70068,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":249,"num_read_kb":194,"num_write":41,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912310+0000","last_change":"2024-04-25T00:14:01.828312+0000","last_active":"2024-04-25T00:16:26.912310+0000","last_peered":"2024-04-25T00:16:26.912310+0000","last_clean":"2024-04-25T00:16:26.912310+0000","last_became_active":"2024-04-25T00:14:01.827947+0000","last_became_peered":"2024-04-25T00:14:01.827947+0000","last_unstale":"2024-04-25T00:16:26.912310+0000","last_undegraded":"2024-04-25T00:16:26.912310+0000","last_fullsized":"2024-04-25T00:16:26.912310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:32.782796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.336543+0000","last_change":"2024-04-25T00:14:06.854379+0000","last_active":"2024-04-25T00:16:26.336543+0000","last_peered":"2024-04-25T00:16:26.336543+0000","last_clean":"2024-04-25T00:16:26.336543+0000","last_became_active":"2024-04-25T00:14:06.853307+0000","last_became_peered":"2024-04-25T00:14:06.853307+0000","last_unstale":"2024-04-25T00:16:26.336543+0000","last_undegraded":"2024-04-25T00:16:26.336543+0000","last_fullsized":"2024-04-25T00:16:26.336543+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:24:11.804189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235501+0000","last_change":"2024-04-25T00:14:01.410395+0000","last_active":"2024-04-25T00:16:26.235501+0000","last_peered":"2024-04-25T00:16:26.235501+0000","last_clean":"2024-04-25T00:16:26.235501+0000","last_became_active":"2024-04-25T00:14:01.410039+0000","last_became_peered":"2024-04-25T00:14:01.410039+0000","last_unstale":"2024-04-25T00:16:26.235501+0000","last_undegraded":"2024-04-25T00:16:26.235501+0000","last_fullsized":"2024-04-25T00:16:26.235501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:21:29.496519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471990+0000","last_change":"2024-04-25T00:14:01.828403+0000","last_active":"2024-04-25T00:16:26.471990+0000","last_peered":"2024-04-25T00:16:26.471990+0000","last_clean":"2024-04-25T00:16:26.471990+0000","last_became_active":"2024-04-25T00:14:01.828107+0000","last_became_peered":"2024-04-25T00:14:01.828107+0000","last_unstale":"2024-04-25T00:16:26.471990+0000","last_undegraded":"2024-04-25T00:16:26.471990+0000","last_fullsized":"2024-04-25T00:16:26.471990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:47:29.417639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":35,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328638+0000","last_change":"2024-04-25T00:15:02.374105+0000","last_active":"2024-04-25T00:16:26.328638+0000","last_peered":"2024-04-25T00:16:26.328638+0000","last_clean":"2024-04-25T00:16:26.328638+0000","last_became_active":"2024-04-25T00:15:02.373628+0000","last_became_peered":"2024-04-25T00:15:02.373628+0000","last_unstale":"2024-04-25T00:16:26.328638+0000","last_undegraded":"2024-04-25T00:16:26.328638+0000","last_fullsized":"2024-04-25T00:16:26.328638+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:05:05.542358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.336205+0000","last_change":"2024-04-25T00:14:01.414406+0000","last_active":"2024-04-25T00:16:26.336205+0000","last_peered":"2024-04-25T00:16:26.336205+0000","last_clean":"2024-04-25T00:16:26.336205+0000","last_became_active":"2024-04-25T00:14:01.414085+0000","last_became_peered":"2024-04-25T00:14:01.414085+0000","last_unstale":"2024-04-25T00:16:26.336205+0000","last_undegraded":"2024-04-25T00:16:26.336205+0000","last_fullsized":"2024-04-25T00:16:26.336205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:22.565371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":50,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471981+0000","last_change":"2024-04-25T00:14:01.833652+0000","last_active":"2024-04-25T00:16:26.471981+0000","last_peered":"2024-04-25T00:16:26.471981+0000","last_clean":"2024-04-25T00:16:26.471981+0000","last_became_active":"2024-04-25T00:14:01.833307+0000","last_became_peered":"2024-04-25T00:14:01.833307+0000","last_unstale":"2024-04-25T00:16:26.471981+0000","last_undegraded":"2024-04-25T00:16:26.471981+0000","last_fullsized":"2024-04-25T00:16:26.471981+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:55:41.261994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.234672+0000","last_change":"2024-04-25T00:14:06.860167+0000","last_active":"2024-04-25T00:16:26.234672+0000","last_peered":"2024-04-25T00:16:26.234672+0000","last_clean":"2024-04-25T00:16:26.234672+0000","last_became_active":"2024-04-25T00:14:06.858651+0000","last_became_peered":"2024-04-25T00:14:06.858651+0000","last_unstale":"2024-04-25T00:16:26.234672+0000","last_undegraded":"2024-04-25T00:16:26.234672+0000","last_fullsized":"2024-04-25T00:16:26.234672+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:11.775434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"97'5","reported_seq":165,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.715411+0000","last_change":"2024-04-25T00:14:01.406267+0000","last_active":"2024-04-25T00:16:26.715411+0000","last_peered":"2024-04-25T00:16:26.715411+0000","last_clean":"2024-04-25T00:16:26.715411+0000","last_became_active":"2024-04-25T00:14:01.405985+0000","last_became_peered":"2024-04-25T00:14:01.405985+0000","last_unstale":"2024-04-25T00:16:26.715411+0000","last_undegraded":"2024-04-25T00:16:26.715411+0000","last_fullsized":"2024-04-25T00:16:26.715411+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:17.253805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":4,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472054+0000","last_change":"2024-04-25T00:14:01.838019+0000","last_active":"2024-04-25T00:16:26.472054+0000","last_peered":"2024-04-25T00:16:26.472054+0000","last_clean":"2024-04-25T00:16:26.472054+0000","last_became_active":"2024-04-25T00:14:01.837691+0000","last_became_peered":"2024-04-25T00:14:01.837691+0000","last_unstale":"2024-04-25T00:16:26.472054+0000","last_undegraded":"2024-04-25T00:16:26.472054+0000","last_fullsized":"2024-04-25T00:16:26.472054+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:26.677039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472562+0000","last_change":"2024-04-25T00:14:06.857061+0000","last_active":"2024-04-25T00:16:26.472562+0000","last_peered":"2024-04-25T00:16:26.472562+0000","last_clean":"2024-04-25T00:16:26.472562+0000","last_became_active":"2024-04-25T00:14:06.856678+0000","last_became_peered":"2024-04-25T00:14:06.856678+0000","last_unstale":"2024-04-25T00:16:26.472562+0000","last_undegraded":"2024-04-25T00:16:26.472562+0000","last_fullsized":"2024-04-25T00:16:26.472562+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:36:52.730861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":54,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.911965+0000","last_change":"2024-04-25T00:14:01.408385+0000","last_active":"2024-04-25T00:16:26.911965+0000","last_peered":"2024-04-25T00:16:26.911965+0000","last_clean":"2024-04-25T00:16:26.911965+0000","last_became_active":"2024-04-25T00:14:01.408049+0000","last_became_peered":"2024-04-25T00:14:01.408049+0000","last_unstale":"2024-04-25T00:16:26.911965+0000","last_undegraded":"2024-04-25T00:16:26.911965+0000","last_fullsized":"2024-04-25T00:16:26.911965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:01:19.765467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.335937+0000","last_change":"2024-04-25T00:15:02.372426+0000","last_active":"2024-04-25T00:16:26.335937+0000","last_peered":"2024-04-25T00:16:26.335937+0000","last_clean":"2024-04-25T00:16:26.335937+0000","last_became_active":"2024-04-25T00:15:02.372044+0000","last_became_peered":"2024-04-25T00:15:02.372044+0000","last_unstale":"2024-04-25T00:16:26.335937+0000","last_undegraded":"2024-04-25T00:16:26.335937+0000","last_fullsized":"2024-04-25T00:16:26.335937+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:53:20.965683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.335834+0000","last_change":"2024-04-25T00:14:06.854192+0000","last_active":"2024-04-25T00:16:26.335834+0000","last_peered":"2024-04-25T00:16:26.335834+0000","last_clean":"2024-04-25T00:16:26.335834+0000","last_became_active":"2024-04-25T00:14:06.853325+0000","last_became_peered":"2024-04-25T00:14:06.853325+0000","last_unstale":"2024-04-25T00:16:26.335834+0000","last_undegraded":"2024-04-25T00:16:26.335834+0000","last_fullsized":"2024-04-25T00:16:26.335834+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:15.308169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912441+0000","last_change":"2024-04-25T00:14:01.407247+0000","last_active":"2024-04-25T00:16:26.912441+0000","last_peered":"2024-04-25T00:16:26.912441+0000","last_clean":"2024-04-25T00:16:26.912441+0000","last_became_active":"2024-04-25T00:14:01.406891+0000","last_became_peered":"2024-04-25T00:14:01.406891+0000","last_unstale":"2024-04-25T00:16:26.912441+0000","last_undegraded":"2024-04-25T00:16:26.912441+0000","last_fullsized":"2024-04-25T00:16:26.912441+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:31:43.294562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328645+0000","last_change":"2024-04-25T00:14:01.829364+0000","last_active":"2024-04-25T00:16:26.328645+0000","last_peered":"2024-04-25T00:16:26.328645+0000","last_clean":"2024-04-25T00:16:26.328645+0000","last_became_active":"2024-04-25T00:14:01.829087+0000","last_became_peered":"2024-04-25T00:14:01.829087+0000","last_unstale":"2024-04-25T00:16:26.328645+0000","last_undegraded":"2024-04-25T00:16:26.328645+0000","last_fullsized":"2024-04-25T00:16:26.328645+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:25:17.651815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.911928+0000","last_change":"2024-04-25T00:14:07.323442+0000","last_active":"2024-04-25T00:16:26.911928+0000","last_peered":"2024-04-25T00:16:26.911928+0000","last_clean":"2024-04-25T00:16:26.911928+0000","last_became_active":"2024-04-25T00:14:07.322943+0000","last_became_peered":"2024-04-25T00:14:07.322943+0000","last_unstale":"2024-04-25T00:16:26.911928+0000","last_undegraded":"2024-04-25T00:16:26.911928+0000","last_fullsized":"2024-04-25T00:16:26.911928+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:49.166240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.234040+0000","last_change":"2024-04-25T00:14:01.410936+0000","last_active":"2024-04-25T00:16:26.234040+0000","last_peered":"2024-04-25T00:16:26.234040+0000","last_clean":"2024-04-25T00:16:26.234040+0000","last_became_active":"2024-04-25T00:14:01.410321+0000","last_became_peered":"2024-04-25T00:14:01.410321+0000","last_unstale":"2024-04-25T00:16:26.234040+0000","last_undegraded":"2024-04-25T00:16:26.234040+0000","last_fullsized":"2024-04-25T00:16:26.234040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:44.603199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"94'4","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.339205+0000","last_change":"2024-04-25T00:14:01.830842+0000","last_active":"2024-04-25T00:16:26.339205+0000","last_peered":"2024-04-25T00:16:26.339205+0000","last_clean":"2024-04-25T00:16:26.339205+0000","last_became_active":"2024-04-25T00:14:01.830543+0000","last_became_peered":"2024-04-25T00:14:01.830543+0000","last_unstale":"2024-04-25T00:16:26.339205+0000","last_undegraded":"2024-04-25T00:16:26.339205+0000","last_fullsized":"2024-04-25T00:16:26.339205+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:59.897133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471933+0000","last_change":"2024-04-25T00:14:06.856841+0000","last_active":"2024-04-25T00:16:26.471933+0000","last_peered":"2024-04-25T00:16:26.471933+0000","last_clean":"2024-04-25T00:16:26.471933+0000","last_became_active":"2024-04-25T00:14:06.856416+0000","last_became_peered":"2024-04-25T00:14:06.856416+0000","last_unstale":"2024-04-25T00:16:26.471933+0000","last_undegraded":"2024-04-25T00:16:26.471933+0000","last_fullsized":"2024-04-25T00:16:26.471933+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:27.181175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471815+0000","last_change":"2024-04-25T00:14:01.409627+0000","last_active":"2024-04-25T00:16:26.471815+0000","last_peered":"2024-04-25T00:16:26.471815+0000","last_clean":"2024-04-25T00:16:26.471815+0000","last_became_active":"2024-04-25T00:14:01.409304+0000","last_became_peered":"2024-04-25T00:14:01.409304+0000","last_unstale":"2024-04-25T00:16:26.471815+0000","last_undegraded":"2024-04-25T00:16:26.471815+0000","last_fullsized":"2024-04-25T00:16:26.471815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:46:27.242004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.339062+0000","last_change":"2024-04-25T00:14:01.829544+0000","last_active":"2024-04-25T00:16:26.339062+0000","last_peered":"2024-04-25T00:16:26.339062+0000","last_clean":"2024-04-25T00:16:26.339062+0000","last_became_active":"2024-04-25T00:14:01.829233+0000","last_became_peered":"2024-04-25T00:14:01.829233+0000","last_unstale":"2024-04-25T00:16:26.339062+0000","last_undegraded":"2024-04-25T00:16:26.339062+0000","last_fullsized":"2024-04-25T00:16:26.339062+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:16.102041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472694+0000","last_change":"2024-04-25T00:14:07.323213+0000","last_active":"2024-04-25T00:16:26.472694+0000","last_peered":"2024-04-25T00:16:26.472694+0000","last_clean":"2024-04-25T00:16:26.472694+0000","last_became_active":"2024-04-25T00:14:07.322763+0000","last_became_peered":"2024-04-25T00:14:07.322763+0000","last_unstale":"2024-04-25T00:16:26.472694+0000","last_undegraded":"2024-04-25T00:16:26.472694+0000","last_fullsized":"2024-04-25T00:16:26.472694+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:45.269314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":25,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.233734+0000","last_change":"2024-04-25T00:15:02.079745+0000","last_active":"2024-04-25T00:16:26.233734+0000","last_peered":"2024-04-25T00:16:26.233734+0000","last_clean":"2024-04-25T00:16:26.233734+0000","last_became_active":"2024-04-25T00:15:02.079405+0000","last_became_peered":"2024-04-25T00:15:02.079405+0000","last_unstale":"2024-04-25T00:16:26.233734+0000","last_undegraded":"2024-04-25T00:16:26.233734+0000","last_fullsized":"2024-04-25T00:16:26.233734+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:01:38.197048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"97'5","reported_seq":162,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.473772+0000","last_change":"2024-04-25T00:14:01.397066+0000","last_active":"2024-04-25T00:16:26.473772+0000","last_peered":"2024-04-25T00:16:26.473772+0000","last_clean":"2024-04-25T00:16:26.473772+0000","last_became_active":"2024-04-25T00:14:01.396728+0000","last_became_peered":"2024-04-25T00:14:01.396728+0000","last_unstale":"2024-04-25T00:16:26.473772+0000","last_undegraded":"2024-04-25T00:16:26.473772+0000","last_fullsized":"2024-04-25T00:16:26.473772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:10:28.334329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"92'1","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472372+0000","last_change":"2024-04-25T00:14:06.853039+0000","last_active":"2024-04-25T00:16:26.472372+0000","last_peered":"2024-04-25T00:16:26.472372+0000","last_clean":"2024-04-25T00:16:26.472372+0000","last_became_active":"2024-04-25T00:14:06.852568+0000","last_became_peered":"2024-04-25T00:14:06.852568+0000","last_unstale":"2024-04-25T00:16:26.472372+0000","last_undegraded":"2024-04-25T00:16:26.472372+0000","last_fullsized":"2024-04-25T00:16:26.472372+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:59.604160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328701+0000","last_change":"2024-04-25T00:14:01.840315+0000","last_active":"2024-04-25T00:16:26.328701+0000","last_peered":"2024-04-25T00:16:26.328701+0000","last_clean":"2024-04-25T00:16:26.328701+0000","last_became_active":"2024-04-25T00:14:01.839537+0000","last_became_peered":"2024-04-25T00:14:01.839537+0000","last_unstale":"2024-04-25T00:16:26.328701+0000","last_undegraded":"2024-04-25T00:16:26.328701+0000","last_fullsized":"2024-04-25T00:16:26.328701+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:34.126164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235345+0000","last_change":"2024-04-25T00:14:01.412493+0000","last_active":"2024-04-25T00:16:26.235345+0000","last_peered":"2024-04-25T00:16:26.235345+0000","last_clean":"2024-04-25T00:16:26.235345+0000","last_became_active":"2024-04-25T00:14:01.412133+0000","last_became_peered":"2024-04-25T00:14:01.412133+0000","last_unstale":"2024-04-25T00:16:26.235345+0000","last_undegraded":"2024-04-25T00:16:26.235345+0000","last_fullsized":"2024-04-25T00:16:26.235345+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:31:44.933728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":371,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471716+0000","last_change":"2024-04-25T00:12:49.169294+0000","last_active":"2024-04-25T00:16:26.471716+0000","last_peered":"2024-04-25T00:16:26.471716+0000","last_clean":"2024-04-25T00:16:26.471716+0000","last_became_active":"2024-04-25T00:12:49.168783+0000","last_became_peered":"2024-04-25T00:12:49.168783+0000","last_unstale":"2024-04-25T00:16:26.471716+0000","last_undegraded":"2024-04-25T00:16:26.471716+0000","last_fullsized":"2024-04-25T00:16:26.471716+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:09:25.202280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:09:25.202280+0000","last_clean_scrub_stamp":"2024-04-25T00:09:25.202280+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:43:04.713986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912255+0000","last_change":"2024-04-25T00:15:02.372570+0000","last_active":"2024-04-25T00:16:26.912255+0000","last_peered":"2024-04-25T00:16:26.912255+0000","last_clean":"2024-04-25T00:16:26.912255+0000","last_became_active":"2024-04-25T00:15:02.372066+0000","last_became_peered":"2024-04-25T00:15:02.372066+0000","last_unstale":"2024-04-25T00:16:26.912255+0000","last_undegraded":"2024-04-25T00:16:26.912255+0000","last_fullsized":"2024-04-25T00:16:26.912255+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:56:59.470049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,6,10],"acting":[8,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471452+0000","last_change":"2024-04-25T00:14:01.826418+0000","last_active":"2024-04-25T00:16:26.471452+0000","last_peered":"2024-04-25T00:16:26.471452+0000","last_clean":"2024-04-25T00:16:26.471452+0000","last_became_active":"2024-04-25T00:14:01.825783+0000","last_became_peered":"2024-04-25T00:14:01.825783+0000","last_unstale":"2024-04-25T00:16:26.471452+0000","last_undegraded":"2024-04-25T00:16:26.471452+0000","last_fullsized":"2024-04-25T00:16:26.471452+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:49:05.981359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471893+0000","last_change":"2024-04-25T00:14:01.417022+0000","last_active":"2024-04-25T00:16:26.471893+0000","last_peered":"2024-04-25T00:16:26.471893+0000","last_clean":"2024-04-25T00:16:26.471893+0000","last_became_active":"2024-04-25T00:14:01.416804+0000","last_became_peered":"2024-04-25T00:14:01.416804+0000","last_unstale":"2024-04-25T00:16:26.471893+0000","last_undegraded":"2024-04-25T00:16:26.471893+0000","last_fullsized":"2024-04-25T00:16:26.471893+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:31.561242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328438+0000","last_change":"2024-04-25T00:14:06.856500+0000","last_active":"2024-04-25T00:16:26.328438+0000","last_peered":"2024-04-25T00:16:26.328438+0000","last_clean":"2024-04-25T00:16:26.328438+0000","last_became_active":"2024-04-25T00:14:06.856077+0000","last_became_peered":"2024-04-25T00:14:06.856077+0000","last_unstale":"2024-04-25T00:16:26.328438+0000","last_undegraded":"2024-04-25T00:16:26.328438+0000","last_fullsized":"2024-04-25T00:16:26.328438+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:10:04.129533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912513+0000","last_change":"2024-04-25T00:14:01.414599+0000","last_active":"2024-04-25T00:16:26.912513+0000","last_peered":"2024-04-25T00:16:26.912513+0000","last_clean":"2024-04-25T00:16:26.912513+0000","last_became_active":"2024-04-25T00:14:01.413796+0000","last_became_peered":"2024-04-25T00:14:01.413796+0000","last_unstale":"2024-04-25T00:16:26.912513+0000","last_undegraded":"2024-04-25T00:16:26.912513+0000","last_fullsized":"2024-04-25T00:16:26.912513+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:44.981848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472240+0000","last_change":"2024-04-25T00:14:01.827718+0000","last_active":"2024-04-25T00:16:26.472240+0000","last_peered":"2024-04-25T00:16:26.472240+0000","last_clean":"2024-04-25T00:16:26.472240+0000","last_became_active":"2024-04-25T00:14:01.827410+0000","last_became_peered":"2024-04-25T00:14:01.827410+0000","last_unstale":"2024-04-25T00:16:26.472240+0000","last_undegraded":"2024-04-25T00:16:26.472240+0000","last_fullsized":"2024-04-25T00:16:26.472240+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:32.428137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328411+0000","last_change":"2024-04-25T00:14:07.323304+0000","last_active":"2024-04-25T00:16:26.328411+0000","last_peered":"2024-04-25T00:16:26.328411+0000","last_clean":"2024-04-25T00:16:26.328411+0000","last_became_active":"2024-04-25T00:14:07.322852+0000","last_became_peered":"2024-04-25T00:14:07.322852+0000","last_unstale":"2024-04-25T00:16:26.328411+0000","last_undegraded":"2024-04-25T00:16:26.328411+0000","last_fullsized":"2024-04-25T00:16:26.328411+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:35.173934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"97'45","reported_seq":118,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.439157+0000","last_change":"2024-04-25T00:16:03.058662+0000","last_active":"2024-04-25T00:16:26.439157+0000","last_peered":"2024-04-25T00:16:26.439157+0000","last_clean":"2024-04-25T00:16:26.439157+0000","last_became_active":"2024-04-25T00:16:03.054623+0000","last_became_peered":"2024-04-25T00:16:03.054623+0000","last_unstale":"2024-04-25T00:16:26.439157+0000","last_undegraded":"2024-04-25T00:16:26.439157+0000","last_fullsized":"2024-04-25T00:16:26.439157+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:08:51.610854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":80,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.329382+0000","last_change":"2024-04-25T00:14:01.834372+0000","last_active":"2024-04-25T00:16:26.329382+0000","last_peered":"2024-04-25T00:16:26.329382+0000","last_clean":"2024-04-25T00:16:26.329382+0000","last_became_active":"2024-04-25T00:14:01.834062+0000","last_became_peered":"2024-04-25T00:14:01.834062+0000","last_unstale":"2024-04-25T00:16:26.329382+0000","last_undegraded":"2024-04-25T00:16:26.329382+0000","last_fullsized":"2024-04-25T00:16:26.329382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:05.580604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235191+0000","last_change":"2024-04-25T00:14:06.852303+0000","last_active":"2024-04-25T00:16:26.235191+0000","last_peered":"2024-04-25T00:16:26.235191+0000","last_clean":"2024-04-25T00:16:26.235191+0000","last_became_active":"2024-04-25T00:14:06.850957+0000","last_became_peered":"2024-04-25T00:14:06.850957+0000","last_unstale":"2024-04-25T00:16:26.235191+0000","last_undegraded":"2024-04-25T00:16:26.235191+0000","last_fullsized":"2024-04-25T00:16:26.235191+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:47.208880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235559+0000","last_change":"2024-04-25T00:14:01.405538+0000","last_active":"2024-04-25T00:16:26.235559+0000","last_peered":"2024-04-25T00:16:26.235559+0000","last_clean":"2024-04-25T00:16:26.235559+0000","last_became_active":"2024-04-25T00:14:01.405155+0000","last_became_peered":"2024-04-25T00:14:01.405155+0000","last_unstale":"2024-04-25T00:16:26.235559+0000","last_undegraded":"2024-04-25T00:16:26.235559+0000","last_fullsized":"2024-04-25T00:16:26.235559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:22:20.864753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261840+0000","last_change":"2024-04-25T00:14:01.837058+0000","last_active":"2024-04-25T00:16:26.261840+0000","last_peered":"2024-04-25T00:16:26.261840+0000","last_clean":"2024-04-25T00:16:26.261840+0000","last_became_active":"2024-04-25T00:14:01.835838+0000","last_became_peered":"2024-04-25T00:14:01.835838+0000","last_unstale":"2024-04-25T00:16:26.261840+0000","last_undegraded":"2024-04-25T00:16:26.261840+0000","last_fullsized":"2024-04-25T00:16:26.261840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:44:21.696448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235583+0000","last_change":"2024-04-25T00:14:07.323870+0000","last_active":"2024-04-25T00:16:26.235583+0000","last_peered":"2024-04-25T00:16:26.235583+0000","last_clean":"2024-04-25T00:16:26.235583+0000","last_became_active":"2024-04-25T00:14:07.323356+0000","last_became_peered":"2024-04-25T00:14:07.323356+0000","last_unstale":"2024-04-25T00:16:26.235583+0000","last_undegraded":"2024-04-25T00:16:26.235583+0000","last_fullsized":"2024-04-25T00:16:26.235583+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:35.160618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472790+0000","last_change":"2024-04-25T00:14:01.409453+0000","last_active":"2024-04-25T00:16:26.472790+0000","last_peered":"2024-04-25T00:16:26.472790+0000","last_clean":"2024-04-25T00:16:26.472790+0000","last_became_active":"2024-04-25T00:14:01.409164+0000","last_became_peered":"2024-04-25T00:14:01.409164+0000","last_unstale":"2024-04-25T00:16:26.472790+0000","last_undegraded":"2024-04-25T00:16:26.472790+0000","last_fullsized":"2024-04-25T00:16:26.472790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:23.266090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.911889+0000","last_change":"2024-04-25T00:14:01.826790+0000","last_active":"2024-04-25T00:16:26.911889+0000","last_peered":"2024-04-25T00:16:26.911889+0000","last_clean":"2024-04-25T00:16:26.911889+0000","last_became_active":"2024-04-25T00:14:01.826457+0000","last_became_peered":"2024-04-25T00:14:01.826457+0000","last_unstale":"2024-04-25T00:16:26.911889+0000","last_undegraded":"2024-04-25T00:16:26.911889+0000","last_fullsized":"2024-04-25T00:16:26.911889+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:21:15.356448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.234393+0000","last_change":"2024-04-25T00:14:06.863280+0000","last_active":"2024-04-25T00:16:26.234393+0000","last_peered":"2024-04-25T00:16:26.234393+0000","last_clean":"2024-04-25T00:16:26.234393+0000","last_became_active":"2024-04-25T00:14:06.862725+0000","last_became_peered":"2024-04-25T00:14:06.862725+0000","last_unstale":"2024-04-25T00:16:26.234393+0000","last_undegraded":"2024-04-25T00:16:26.234393+0000","last_fullsized":"2024-04-25T00:16:26.234393+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:43:12.791744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328567+0000","last_change":"2024-04-25T00:14:01.414856+0000","last_active":"2024-04-25T00:16:26.328567+0000","last_peered":"2024-04-25T00:16:26.328567+0000","last_clean":"2024-04-25T00:16:26.328567+0000","last_became_active":"2024-04-25T00:14:01.414533+0000","last_became_peered":"2024-04-25T00:14:01.414533+0000","last_unstale":"2024-04-25T00:16:26.328567+0000","last_undegraded":"2024-04-25T00:16:26.328567+0000","last_fullsized":"2024-04-25T00:16:26.328567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:01.090539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328304+0000","last_change":"2024-04-25T00:14:01.831452+0000","last_active":"2024-04-25T00:16:26.328304+0000","last_peered":"2024-04-25T00:16:26.328304+0000","last_clean":"2024-04-25T00:16:26.328304+0000","last_became_active":"2024-04-25T00:14:01.831151+0000","last_became_peered":"2024-04-25T00:14:01.831151+0000","last_unstale":"2024-04-25T00:16:26.328304+0000","last_undegraded":"2024-04-25T00:16:26.328304+0000","last_fullsized":"2024-04-25T00:16:26.328304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:53:07.659508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"92'1","reported_seq":29,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235539+0000","last_change":"2024-04-25T00:14:06.857601+0000","last_active":"2024-04-25T00:16:26.235539+0000","last_peered":"2024-04-25T00:16:26.235539+0000","last_clean":"2024-04-25T00:16:26.235539+0000","last_became_active":"2024-04-25T00:14:06.856685+0000","last_became_peered":"2024-04-25T00:14:06.856685+0000","last_unstale":"2024-04-25T00:16:26.235539+0000","last_undegraded":"2024-04-25T00:16:26.235539+0000","last_fullsized":"2024-04-25T00:16:26.235539+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:20.250053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":31,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328326+0000","last_change":"2024-04-25T00:16:03.060574+0000","last_active":"2024-04-25T00:16:26.328326+0000","last_peered":"2024-04-25T00:16:26.328326+0000","last_clean":"2024-04-25T00:16:26.328326+0000","last_became_active":"2024-04-25T00:16:03.055792+0000","last_became_peered":"2024-04-25T00:16:03.055792+0000","last_unstale":"2024-04-25T00:16:26.328326+0000","last_undegraded":"2024-04-25T00:16:26.328326+0000","last_fullsized":"2024-04-25T00:16:26.328326+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:37:28.112894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,1],"acting":[11,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328959+0000","last_change":"2024-04-25T00:14:01.832342+0000","last_active":"2024-04-25T00:16:26.328959+0000","last_peered":"2024-04-25T00:16:26.328959+0000","last_clean":"2024-04-25T00:16:26.328959+0000","last_became_active":"2024-04-25T00:14:01.831997+0000","last_became_peered":"2024-04-25T00:14:01.831997+0000","last_unstale":"2024-04-25T00:16:26.328959+0000","last_undegraded":"2024-04-25T00:16:26.328959+0000","last_fullsized":"2024-04-25T00:16:26.328959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:16:12.335884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235429+0000","last_change":"2024-04-25T00:14:07.323628+0000","last_active":"2024-04-25T00:16:26.235429+0000","last_peered":"2024-04-25T00:16:26.235429+0000","last_clean":"2024-04-25T00:16:26.235429+0000","last_became_active":"2024-04-25T00:14:07.322996+0000","last_became_peered":"2024-04-25T00:14:07.322996+0000","last_unstale":"2024-04-25T00:16:26.235429+0000","last_undegraded":"2024-04-25T00:16:26.235429+0000","last_fullsized":"2024-04-25T00:16:26.235429+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:19:32.803216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":43,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.912115+0000","last_change":"2024-04-25T00:14:01.407715+0000","last_active":"2024-04-25T00:16:26.912115+0000","last_peered":"2024-04-25T00:16:26.912115+0000","last_clean":"2024-04-25T00:16:26.912115+0000","last_became_active":"2024-04-25T00:14:01.407439+0000","last_became_peered":"2024-04-25T00:14:01.407439+0000","last_unstale":"2024-04-25T00:16:26.912115+0000","last_undegraded":"2024-04-25T00:16:26.912115+0000","last_fullsized":"2024-04-25T00:16:26.912115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:33.550379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.471832+0000","last_change":"2024-04-25T00:14:01.837733+0000","last_active":"2024-04-25T00:16:26.471832+0000","last_peered":"2024-04-25T00:16:26.471832+0000","last_clean":"2024-04-25T00:16:26.471832+0000","last_became_active":"2024-04-25T00:14:01.837395+0000","last_became_peered":"2024-04-25T00:14:01.837395+0000","last_unstale":"2024-04-25T00:16:26.471832+0000","last_undegraded":"2024-04-25T00:16:26.471832+0000","last_fullsized":"2024-04-25T00:16:26.471832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:50.186834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472922+0000","last_change":"2024-04-25T00:14:06.848700+0000","last_active":"2024-04-25T00:16:26.472922+0000","last_peered":"2024-04-25T00:16:26.472922+0000","last_clean":"2024-04-25T00:16:26.472922+0000","last_became_active":"2024-04-25T00:14:06.848252+0000","last_became_peered":"2024-04-25T00:14:06.848252+0000","last_unstale":"2024-04-25T00:16:26.472922+0000","last_undegraded":"2024-04-25T00:16:26.472922+0000","last_fullsized":"2024-04-25T00:16:26.472922+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:07.653668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235943+0000","last_change":"2024-04-25T00:14:01.405785+0000","last_active":"2024-04-25T00:16:26.235943+0000","last_peered":"2024-04-25T00:16:26.235943+0000","last_clean":"2024-04-25T00:16:26.235943+0000","last_became_active":"2024-04-25T00:14:01.405475+0000","last_became_peered":"2024-04-25T00:14:01.405475+0000","last_unstale":"2024-04-25T00:16:26.235943+0000","last_undegraded":"2024-04-25T00:16:26.235943+0000","last_fullsized":"2024-04-25T00:16:26.235943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:02:32.487795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472556+0000","last_change":"2024-04-25T00:14:01.832393+0000","last_active":"2024-04-25T00:16:26.472556+0000","last_peered":"2024-04-25T00:16:26.472556+0000","last_clean":"2024-04-25T00:16:26.472556+0000","last_became_active":"2024-04-25T00:14:01.832096+0000","last_became_peered":"2024-04-25T00:14:01.832096+0000","last_unstale":"2024-04-25T00:16:26.472556+0000","last_undegraded":"2024-04-25T00:16:26.472556+0000","last_fullsized":"2024-04-25T00:16:26.472556+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:06:41.749627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261459+0000","last_change":"2024-04-25T00:14:06.855224+0000","last_active":"2024-04-25T00:16:26.261459+0000","last_peered":"2024-04-25T00:16:26.261459+0000","last_clean":"2024-04-25T00:16:26.261459+0000","last_became_active":"2024-04-25T00:14:06.854847+0000","last_became_peered":"2024-04-25T00:14:06.854847+0000","last_unstale":"2024-04-25T00:16:26.261459+0000","last_undegraded":"2024-04-25T00:16:26.261459+0000","last_fullsized":"2024-04-25T00:16:26.261459+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:12:13.520448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"97'39","reported_seq":85,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.433764+0000","last_change":"2024-04-25T00:14:01.401145+0000","last_active":"2024-04-25T00:16:26.433764+0000","last_peered":"2024-04-25T00:16:26.433764+0000","last_clean":"2024-04-25T00:16:26.433764+0000","last_became_active":"2024-04-25T00:14:01.400807+0000","last_became_peered":"2024-04-25T00:14:01.400807+0000","last_unstale":"2024-04-25T00:16:26.433764+0000","last_undegraded":"2024-04-25T00:16:26.433764+0000","last_fullsized":"2024-04-25T00:16:26.433764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:52:04.674677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"94'6","reported_seq":41,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472317+0000","last_change":"2024-04-25T00:14:01.838033+0000","last_active":"2024-04-25T00:16:26.472317+0000","last_peered":"2024-04-25T00:16:26.472317+0000","last_clean":"2024-04-25T00:16:26.472317+0000","last_became_active":"2024-04-25T00:14:01.837657+0000","last_became_peered":"2024-04-25T00:14:01.837657+0000","last_unstale":"2024-04-25T00:16:26.472317+0000","last_undegraded":"2024-04-25T00:16:26.472317+0000","last_fullsized":"2024-04-25T00:16:26.472317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:51.895086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":28,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.335703+0000","last_change":"2024-04-25T00:14:06.854885+0000","last_active":"2024-04-25T00:16:26.335703+0000","last_peered":"2024-04-25T00:16:26.335703+0000","last_clean":"2024-04-25T00:16:26.335703+0000","last_became_active":"2024-04-25T00:14:06.854200+0000","last_became_peered":"2024-04-25T00:14:06.854200+0000","last_unstale":"2024-04-25T00:16:26.335703+0000","last_undegraded":"2024-04-25T00:16:26.335703+0000","last_fullsized":"2024-04-25T00:16:26.335703+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:05.824472+0000","last_clean_scrub_stamp":"2024-04-25T00:14:05.824472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:43.627742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472665+0000","last_change":"2024-04-25T00:14:01.403789+0000","last_active":"2024-04-25T00:16:26.472665+0000","last_peered":"2024-04-25T00:16:26.472665+0000","last_clean":"2024-04-25T00:16:26.472665+0000","last_became_active":"2024-04-25T00:14:01.403464+0000","last_became_peered":"2024-04-25T00:14:01.403464+0000","last_unstale":"2024-04-25T00:16:26.472665+0000","last_undegraded":"2024-04-25T00:16:26.472665+0000","last_fullsized":"2024-04-25T00:16:26.472665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:00.386280+0000","last_clean_scrub_stamp":"2024-04-25T00:14:00.386280+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:48.276536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472074+0000","last_change":"2024-04-25T00:14:01.839988+0000","last_active":"2024-04-25T00:16:26.472074+0000","last_peered":"2024-04-25T00:16:26.472074+0000","last_clean":"2024-04-25T00:16:26.472074+0000","last_became_active":"2024-04-25T00:14:01.839516+0000","last_became_peered":"2024-04-25T00:14:01.839516+0000","last_unstale":"2024-04-25T00:16:26.472074+0000","last_undegraded":"2024-04-25T00:16:26.472074+0000","last_fullsized":"2024-04-25T00:16:26.472074+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:56:05.875127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":45,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328368+0000","last_change":"2024-04-25T00:14:01.828638+0000","last_active":"2024-04-25T00:16:26.328368+0000","last_peered":"2024-04-25T00:16:26.328368+0000","last_clean":"2024-04-25T00:16:26.328368+0000","last_became_active":"2024-04-25T00:14:01.828266+0000","last_became_peered":"2024-04-25T00:14:01.828266+0000","last_unstale":"2024-04-25T00:16:26.328368+0000","last_undegraded":"2024-04-25T00:16:26.328368+0000","last_fullsized":"2024-04-25T00:16:26.328368+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:34:22.619522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":48,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235391+0000","last_change":"2024-04-25T00:14:01.828199+0000","last_active":"2024-04-25T00:16:26.235391+0000","last_peered":"2024-04-25T00:16:26.235391+0000","last_clean":"2024-04-25T00:16:26.235391+0000","last_became_active":"2024-04-25T00:14:01.827579+0000","last_became_peered":"2024-04-25T00:14:01.827579+0000","last_unstale":"2024-04-25T00:16:26.235391+0000","last_undegraded":"2024-04-25T00:16:26.235391+0000","last_fullsized":"2024-04-25T00:16:26.235391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:00.708928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235265+0000","last_change":"2024-04-25T00:14:01.825620+0000","last_active":"2024-04-25T00:16:26.235265+0000","last_peered":"2024-04-25T00:16:26.235265+0000","last_clean":"2024-04-25T00:16:26.235265+0000","last_became_active":"2024-04-25T00:14:01.825310+0000","last_became_peered":"2024-04-25T00:14:01.825310+0000","last_unstale":"2024-04-25T00:16:26.235265+0000","last_undegraded":"2024-04-25T00:16:26.235265+0000","last_fullsized":"2024-04-25T00:16:26.235265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:45:35.075355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.261644+0000","last_change":"2024-04-25T00:14:01.836537+0000","last_active":"2024-04-25T00:16:26.261644+0000","last_peered":"2024-04-25T00:16:26.261644+0000","last_clean":"2024-04-25T00:16:26.261644+0000","last_became_active":"2024-04-25T00:14:01.836234+0000","last_became_peered":"2024-04-25T00:14:01.836234+0000","last_unstale":"2024-04-25T00:16:26.261644+0000","last_undegraded":"2024-04-25T00:16:26.261644+0000","last_fullsized":"2024-04-25T00:16:26.261644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:07:15.037498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":40,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472108+0000","last_change":"2024-04-25T00:14:01.825937+0000","last_active":"2024-04-25T00:16:26.472108+0000","last_peered":"2024-04-25T00:16:26.472108+0000","last_clean":"2024-04-25T00:16:26.472108+0000","last_became_active":"2024-04-25T00:14:01.825605+0000","last_became_peered":"2024-04-25T00:14:01.825605+0000","last_unstale":"2024-04-25T00:16:26.472108+0000","last_undegraded":"2024-04-25T00:16:26.472108+0000","last_fullsized":"2024-04-25T00:16:26.472108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:13:06.918000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328697+0000","last_change":"2024-04-25T00:14:01.830004+0000","last_active":"2024-04-25T00:16:26.328697+0000","last_peered":"2024-04-25T00:16:26.328697+0000","last_clean":"2024-04-25T00:16:26.328697+0000","last_became_active":"2024-04-25T00:14:01.829588+0000","last_became_peered":"2024-04-25T00:14:01.829588+0000","last_unstale":"2024-04-25T00:16:26.328697+0000","last_undegraded":"2024-04-25T00:16:26.328697+0000","last_fullsized":"2024-04-25T00:16:26.328697+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:08.005995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":51,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.235672+0000","last_change":"2024-04-25T00:14:01.825762+0000","last_active":"2024-04-25T00:16:26.235672+0000","last_peered":"2024-04-25T00:16:26.235672+0000","last_clean":"2024-04-25T00:16:26.235672+0000","last_became_active":"2024-04-25T00:14:01.825471+0000","last_became_peered":"2024-04-25T00:14:01.825471+0000","last_unstale":"2024-04-25T00:16:26.235672+0000","last_undegraded":"2024-04-25T00:16:26.235672+0000","last_fullsized":"2024-04-25T00:16:26.235672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:33.716875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":38,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.329464+0000","last_change":"2024-04-25T00:14:01.836103+0000","last_active":"2024-04-25T00:16:26.329464+0000","last_peered":"2024-04-25T00:16:26.329464+0000","last_clean":"2024-04-25T00:16:26.329464+0000","last_became_active":"2024-04-25T00:14:01.834591+0000","last_became_peered":"2024-04-25T00:14:01.834591+0000","last_unstale":"2024-04-25T00:16:26.329464+0000","last_undegraded":"2024-04-25T00:16:26.329464+0000","last_fullsized":"2024-04-25T00:16:26.329464+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:37:55.130978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":42,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.335893+0000","last_change":"2024-04-25T00:14:01.833179+0000","last_active":"2024-04-25T00:16:26.335893+0000","last_peered":"2024-04-25T00:16:26.335893+0000","last_clean":"2024-04-25T00:16:26.335893+0000","last_became_active":"2024-04-25T00:14:01.832856+0000","last_became_peered":"2024-04-25T00:14:01.832856+0000","last_unstale":"2024-04-25T00:16:26.335893+0000","last_undegraded":"2024-04-25T00:16:26.335893+0000","last_fullsized":"2024-04-25T00:16:26.335893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:39:07.153356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328555+0000","last_change":"2024-04-25T00:14:01.834751+0000","last_active":"2024-04-25T00:16:26.328555+0000","last_peered":"2024-04-25T00:16:26.328555+0000","last_clean":"2024-04-25T00:16:26.328555+0000","last_became_active":"2024-04-25T00:14:01.834413+0000","last_became_peered":"2024-04-25T00:14:01.834413+0000","last_unstale":"2024-04-25T00:16:26.328555+0000","last_undegraded":"2024-04-25T00:16:26.328555+0000","last_fullsized":"2024-04-25T00:16:26.328555+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:52.341565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":46,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.473303+0000","last_change":"2024-04-25T00:14:01.832944+0000","last_active":"2024-04-25T00:16:26.473303+0000","last_peered":"2024-04-25T00:16:26.473303+0000","last_clean":"2024-04-25T00:16:26.473303+0000","last_became_active":"2024-04-25T00:14:01.832394+0000","last_became_peered":"2024-04-25T00:14:01.832394+0000","last_unstale":"2024-04-25T00:16:26.473303+0000","last_undegraded":"2024-04-25T00:16:26.473303+0000","last_fullsized":"2024-04-25T00:16:26.473303+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:26.957786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":47,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.472374+0000","last_change":"2024-04-25T00:14:01.831492+0000","last_active":"2024-04-25T00:16:26.472374+0000","last_peered":"2024-04-25T00:16:26.472374+0000","last_clean":"2024-04-25T00:16:26.472374+0000","last_became_active":"2024-04-25T00:14:01.829557+0000","last_became_peered":"2024-04-25T00:14:01.829557+0000","last_unstale":"2024-04-25T00:16:26.472374+0000","last_undegraded":"2024-04-25T00:16:26.472374+0000","last_fullsized":"2024-04-25T00:16:26.472374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:45.116188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":44,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.328652+0000","last_change":"2024-04-25T00:14:01.839323+0000","last_active":"2024-04-25T00:16:26.328652+0000","last_peered":"2024-04-25T00:16:26.328652+0000","last_clean":"2024-04-25T00:16:26.328652+0000","last_became_active":"2024-04-25T00:14:01.838999+0000","last_became_peered":"2024-04-25T00:14:01.838999+0000","last_unstale":"2024-04-25T00:16:26.328652+0000","last_undegraded":"2024-04-25T00:16:26.328652+0000","last_fullsized":"2024-04-25T00:16:26.328652+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:14:01.387805+0000","last_clean_scrub_stamp":"2024-04-25T00:14:01.387805+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:20.652123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599808,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":838,"num_read_kb":532,"num_write":287,"num_write_kb":1441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1799424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351018,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1503232,"data_stored":943251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612654,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37232,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707920,"statfs":{"total":95995035648,"available":95956910080,"internally_reserved":0,"allocated":1941504,"data_stored":1339753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776180,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36828,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708324,"statfs":{"total":95995035648,"available":95957323776,"internally_reserved":0,"allocated":1523712,"data_stored":947583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037817,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1110016,"data_stored":540338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299454,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28684,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716468,"statfs":{"total":95995035648,"available":95965663232,"internally_reserved":0,"allocated":1703936,"data_stored":1136388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462980,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1089536,"data_stored":535594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":599417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36444,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708708,"statfs":{"total":95995035648,"available":95957716992,"internally_reserved":0,"allocated":1130496,"data_stored":546835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28196,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716956,"statfs":{"total":95995035648,"available":95966162944,"internally_reserved":0,"allocated":1204224,"data_stored":617307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149778,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36944,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708208,"statfs":{"total":95995035648,"available":95957204992,"internally_reserved":0,"allocated":1646592,"data_stored":1070457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888141,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40592,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704560,"statfs":{"total":95995035648,"available":95953469440,"internally_reserved":0,"allocated":1118208,"data_stored":550879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724616,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29016,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716136,"statfs":{"total":95995035648,"available":95965323264,"internally_reserved":0,"allocated":1863680,"data_stored":1273429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":70068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":133608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":204028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:16:32.001 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:16:32.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:31 smithi195 ceph-mon[32833]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:32.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:31 smithi195 ceph-mon[32833]: Updating smithi170:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:16:32.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:31 smithi195 ceph-mon[32833]: Updating smithi107:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:16:32.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:31 smithi195 ceph-mon[32833]: Updating smithi195:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:16:32.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:31 smithi195 ceph-mon[32833]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-04-25T00:16:32.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:31 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:32.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:31 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:32.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:31 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:32.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:31 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:32.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:31 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1132749210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:32.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.132+0000 7fb02d589640 1 -- 172.21.15.107:0/1974052771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0281066b0 msgr2=0x7fb028106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:32.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.132+0000 7fb02d589640 1 --2- 172.21.15.107:0/1974052771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0281066b0 0x7fb028106a90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb010009930 tx=0x7fb01002a420 comp rx=0 tx=0).stop 2024-04-25T00:16:32.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.133+0000 7fb02d589640 1 -- 172.21.15.107:0/1974052771 shutdown_connections 2024-04-25T00:16:32.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.133+0000 7fb02d589640 1 --2- 172.21.15.107:0/1974052771 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb028100fd0 0x7fb02810d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.133+0000 7fb02d589640 1 --2- 172.21.15.107:0/1974052771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb028100610 0x7fb028100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.133+0000 7fb02d589640 1 --2- 172.21.15.107:0/1974052771 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0281066b0 0x7fb028106a90 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.133+0000 7fb02d589640 1 -- 172.21.15.107:0/1974052771 >> 172.21.15.107:0/1974052771 conn(0x7fb0280769c0 msgr2=0x7fb028076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:32.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.133+0000 7fb02d589640 1 -- 172.21.15.107:0/1974052771 shutdown_connections 2024-04-25T00:16:32.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.133+0000 7fb02d589640 1 -- 172.21.15.107:0/1974052771 wait complete. 2024-04-25T00:16:32.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.133+0000 7fb02d589640 1 Processor -- start 2024-04-25T00:16:32.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.133+0000 7fb02d589640 1 -- start start 2024-04-25T00:16:32.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.133+0000 7fb02d589640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb028100610 0x7fb028198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:32.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.134+0000 7fb02d589640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb028100fd0 0x7fb028199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:32.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.134+0000 7fb02d589640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0281066b0 0x7fb0281a3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:32.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.134+0000 7fb02d589640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb028110a30 con 0x7fb0281066b0 2024-04-25T00:16:32.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.134+0000 7fb02d589640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb0281108b0 con 0x7fb028100fd0 2024-04-25T00:16:32.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.134+0000 7fb02d589640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb028110bb0 con 0x7fb028100610 2024-04-25T00:16:32.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.134+0000 7fb0277fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0281066b0 0x7fb0281a3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:32.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.134+0000 7fb0277fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0281066b0 0x7fb0281a3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39104/0 (socket says 172.21.15.107:39104) 2024-04-25T00:16:32.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.134+0000 7fb0277fe640 1 -- 172.21.15.107:0/3184672373 learned_addr learned my addr 172.21.15.107:0/3184672373 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:32.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.134+0000 7fb026ffd640 1 --2- 172.21.15.107:0/3184672373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb028100610 0x7fb028198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:32.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.134+0000 7fb0267fc640 1 --2- 172.21.15.107:0/3184672373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb028100fd0 0x7fb028199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:32.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.134+0000 7fb0277fe640 1 -- 172.21.15.107:0/3184672373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb028100610 msgr2=0x7fb028198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:32.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.134+0000 7fb0277fe640 1 --2- 172.21.15.107:0/3184672373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb028100610 0x7fb028198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.134+0000 7fb0277fe640 1 -- 172.21.15.107:0/3184672373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb028100fd0 msgr2=0x7fb028199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:32.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.134+0000 7fb0277fe640 1 --2- 172.21.15.107:0/3184672373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb028100fd0 0x7fb028199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.134+0000 7fb0277fe640 1 -- 172.21.15.107:0/3184672373 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0281a43b0 con 0x7fb0281066b0 2024-04-25T00:16:32.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.134+0000 7fb0277fe640 1 --2- 172.21.15.107:0/3184672373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0281066b0 0x7fb0281a3e70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb02000eaa0 tx=0x7fb02000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:32.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.134+0000 7fb01ffff640 1 -- 172.21.15.107:0/3184672373 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02000cdb0 con 0x7fb0281066b0 2024-04-25T00:16:32.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.134+0000 7fb01ffff640 1 -- 172.21.15.107:0/3184672373 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb020005020 con 0x7fb0281066b0 2024-04-25T00:16:32.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.134+0000 7fb02d589640 1 -- 172.21.15.107:0/3184672373 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0281a4580 con 0x7fb0281066b0 2024-04-25T00:16:32.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.134+0000 7fb01ffff640 1 -- 172.21.15.107:0/3184672373 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0200116b0 con 0x7fb0281066b0 2024-04-25T00:16:32.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.134+0000 7fb02d589640 1 -- 172.21.15.107:0/3184672373 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0281a4af0 con 0x7fb0281066b0 2024-04-25T00:16:32.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.135+0000 7fb02d589640 1 -- 172.21.15.107:0/3184672373 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb028101d50 con 0x7fb0281066b0 2024-04-25T00:16:32.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.136+0000 7fb01ffff640 1 -- 172.21.15.107:0/3184672373 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7fb020005560 con 0x7fb0281066b0 2024-04-25T00:16:32.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.137+0000 7fb01ffff640 1 --2- 172.21.15.107:0/3184672373 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb0140a2ea0 0x7fb0140a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:32.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.137+0000 7fb026ffd640 1 --2- 172.21.15.107:0/3184672373 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb0140a2ea0 0x7fb0140a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:32.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.137+0000 7fb01ffff640 1 -- 172.21.15.107:0/3184672373 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fb0200cfb00 con 0x7fb0281066b0 2024-04-25T00:16:32.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.137+0000 7fb026ffd640 1 --2- 172.21.15.107:0/3184672373 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb0140a2ea0 0x7fb0140a5360 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb010004770 tx=0x7fb0100058d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:32.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.142+0000 7fb01ffff640 1 -- 172.21.15.107:0/3184672373 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb020016030 con 0x7fb0281066b0 2024-04-25T00:16:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:31 smithi170 ceph-mon[32807]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:31 smithi170 ceph-mon[32807]: Updating smithi170:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:16:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:31 smithi170 ceph-mon[32807]: Updating smithi107:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:16:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:31 smithi170 ceph-mon[32807]: Updating smithi195:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:16:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:31 smithi170 ceph-mon[32807]: mgrmap e23: x(active, since 4s), standbys: z, y 2024-04-25T00:16:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:31 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:31 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:31 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:31 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:16:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:31 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1132749210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:32.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.325+0000 7fb02d589640 1 -- 172.21.15.107:0/3184672373 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb028108bb0 con 0x7fb0281066b0 2024-04-25T00:16:32.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.326+0000 7fb01ffff640 1 -- 172.21.15.107:0/3184672373 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+25070 (secure 0 0 0) 0x7fb02009bb10 con 0x7fb0281066b0 2024-04-25T00:16:32.328 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:16:32.328 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":97,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:16:26.229487+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:59.672801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:00.957782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:14:05.065846+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8},{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/151603643":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/1884923543":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/111369358":"2024-04-26T00:16:26.229442+0000","172.21.15.107:6800/1644942185":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3936375305":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/1644942185":"2024-04-26T00:16:26.229442+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:16:32.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.329+0000 7fb02d589640 1 -- 172.21.15.107:0/3184672373 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb0140a2ea0 msgr2=0x7fb0140a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:32.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.329+0000 7fb02d589640 1 --2- 172.21.15.107:0/3184672373 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb0140a2ea0 0x7fb0140a5360 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb010004770 tx=0x7fb0100058d0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.329+0000 7fb02d589640 1 -- 172.21.15.107:0/3184672373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0281066b0 msgr2=0x7fb0281a3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:32.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.329+0000 7fb02d589640 1 --2- 172.21.15.107:0/3184672373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0281066b0 0x7fb0281a3e70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb02000eaa0 tx=0x7fb02000ef60 comp rx=0 tx=0).stop 2024-04-25T00:16:32.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.329+0000 7fb02d589640 1 -- 172.21.15.107:0/3184672373 shutdown_connections 2024-04-25T00:16:32.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.329+0000 7fb02d589640 1 --2- 172.21.15.107:0/3184672373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0281066b0 0x7fb0281a3e70 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.329+0000 7fb02d589640 1 --2- 172.21.15.107:0/3184672373 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb0140a2ea0 0x7fb0140a5360 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.329+0000 7fb02d589640 1 --2- 172.21.15.107:0/3184672373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb028100fd0 0x7fb028199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.329+0000 7fb02d589640 1 --2- 172.21.15.107:0/3184672373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb028100610 0x7fb028198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.329+0000 7fb02d589640 1 -- 172.21.15.107:0/3184672373 >> 172.21.15.107:0/3184672373 conn(0x7fb0280769c0 msgr2=0x7fb0280fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:32.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.329+0000 7fb02d589640 1 -- 172.21.15.107:0/3184672373 shutdown_connections 2024-04-25T00:16:32.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.329+0000 7fb02d589640 1 -- 172.21.15.107:0/3184672373 wait complete. 2024-04-25T00:16:32.341 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:09:25.160329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:13:59.672801+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:14:00.957782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:14:05.065846+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:16:32.341 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:16:32.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.442+0000 7fbac61c1640 1 -- 172.21.15.107:0/130820916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbac0107720 msgr2=0x7fbac0107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:32.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.442+0000 7fbac61c1640 1 --2- 172.21.15.107:0/130820916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbac0107720 0x7fbac0107b00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009930 tx=0x7fbab002a420 comp rx=0 tx=0).stop 2024-04-25T00:16:32.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.442+0000 7fbac61c1640 1 -- 172.21.15.107:0/130820916 shutdown_connections 2024-04-25T00:16:32.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.442+0000 7fbac61c1640 1 --2- 172.21.15.107:0/130820916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbac01020d0 0x7fbac010e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.442+0000 7fbac61c1640 1 --2- 172.21.15.107:0/130820916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbac0101710 0x7fbac0101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.442+0000 7fbac61c1640 1 --2- 172.21.15.107:0/130820916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbac0107720 0x7fbac0107b00 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.442+0000 7fbac61c1640 1 -- 172.21.15.107:0/130820916 >> 172.21.15.107:0/130820916 conn(0x7fbac0078b90 msgr2=0x7fbac0078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:32.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.442+0000 7fbac61c1640 1 -- 172.21.15.107:0/130820916 shutdown_connections 2024-04-25T00:16:32.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.442+0000 7fbac61c1640 1 -- 172.21.15.107:0/130820916 wait complete. 2024-04-25T00:16:32.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.442+0000 7fbac61c1640 1 Processor -- start 2024-04-25T00:16:32.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.442+0000 7fbac61c1640 1 -- start start 2024-04-25T00:16:32.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.442+0000 7fbac61c1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbac0101710 0x7fbac0198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:32.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.442+0000 7fbac61c1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbac01020d0 0x7fbac0198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:32.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.442+0000 7fbac61c1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbac0107720 0x7fbac019c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:32.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.442+0000 7fbac61c1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbac0111b30 con 0x7fbac01020d0 2024-04-25T00:16:32.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.443+0000 7fbac61c1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbac01119b0 con 0x7fbac0107720 2024-04-25T00:16:32.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.443+0000 7fbac61c1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbac0111cb0 con 0x7fbac0101710 2024-04-25T00:16:32.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.443+0000 7fbab7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbac01020d0 0x7fbac0198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:32.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.443+0000 7fbab7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbac01020d0 0x7fbac0198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39110/0 (socket says 172.21.15.107:39110) 2024-04-25T00:16:32.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.443+0000 7fbab7fff640 1 -- 172.21.15.107:0/1301381040 learned_addr learned my addr 172.21.15.107:0/1301381040 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:32.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.443+0000 7fbabf7fe640 1 --2- 172.21.15.107:0/1301381040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbac0101710 0x7fbac0198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:32.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.443+0000 7fbab7fff640 1 -- 172.21.15.107:0/1301381040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbac0101710 msgr2=0x7fbac0198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:32.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.443+0000 7fbab7fff640 1 --2- 172.21.15.107:0/1301381040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbac0101710 0x7fbac0198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.443+0000 7fbabffff640 1 --2- 172.21.15.107:0/1301381040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbac0107720 0x7fbac019c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:32.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.443+0000 7fbab7fff640 1 -- 172.21.15.107:0/1301381040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbac0107720 msgr2=0x7fbac019c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:32.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.443+0000 7fbab7fff640 1 --2- 172.21.15.107:0/1301381040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbac0107720 0x7fbac019c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.443+0000 7fbab7fff640 1 -- 172.21.15.107:0/1301381040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac019d040 con 0x7fbac01020d0 2024-04-25T00:16:32.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.443+0000 7fbabf7fe640 1 --2- 172.21.15.107:0/1301381040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbac0101710 0x7fbac0198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:32.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.443+0000 7fbab7fff640 1 --2- 172.21.15.107:0/1301381040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbac01020d0 0x7fbac0198590 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbaa800e9f0 tx=0x7fbaa800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:32.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.443+0000 7fbabd7fa640 1 -- 172.21.15.107:0/1301381040 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa800cd80 con 0x7fbac01020d0 2024-04-25T00:16:32.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.443+0000 7fbac61c1640 1 -- 172.21.15.107:0/1301381040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac019d330 con 0x7fbac01020d0 2024-04-25T00:16:32.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.443+0000 7fbabd7fa640 1 -- 172.21.15.107:0/1301381040 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbaa8005500 con 0x7fbac01020d0 2024-04-25T00:16:32.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.443+0000 7fbabd7fa640 1 -- 172.21.15.107:0/1301381040 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8010690 con 0x7fbac01020d0 2024-04-25T00:16:32.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.443+0000 7fbac61c1640 1 -- 172.21.15.107:0/1301381040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac01a4bc0 con 0x7fbac01020d0 2024-04-25T00:16:32.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.444+0000 7fbac61c1640 1 -- 172.21.15.107:0/1301381040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac00689b0 con 0x7fbac01020d0 2024-04-25T00:16:32.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.445+0000 7fbabd7fa640 1 -- 172.21.15.107:0/1301381040 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7fbaa80046c0 con 0x7fbac01020d0 2024-04-25T00:16:32.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.446+0000 7fbabd7fa640 1 --2- 172.21.15.107:0/1301381040 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbaac0a2e20 0x7fbaac0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:32.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.446+0000 7fbabf7fe640 1 --2- 172.21.15.107:0/1301381040 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbaac0a2e20 0x7fbaac0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:32.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.446+0000 7fbabd7fa640 1 -- 172.21.15.107:0/1301381040 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fbaa80d01d0 con 0x7fbac01020d0 2024-04-25T00:16:32.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.446+0000 7fbabf7fe640 1 --2- 172.21.15.107:0/1301381040 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbaac0a2e20 0x7fbaac0a52e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbab0000c00 tx=0x7fbab0005a20 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:32.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.449+0000 7fbabd7fa640 1 -- 172.21.15.107:0/1301381040 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbaa8053740 con 0x7fbac01020d0 2024-04-25T00:16:32.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.623+0000 7fbac61c1640 1 -- 172.21.15.107:0/1301381040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbac002d170 con 0x7fbac01020d0 2024-04-25T00:16:32.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.624+0000 7fbabd7fa640 1 -- 172.21.15.107:0/1301381040 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7fbaa809c830 con 0x7fbac01020d0 2024-04-25T00:16:32.624 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-04-25T00:16:32.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.626+0000 7fbac61c1640 1 -- 172.21.15.107:0/1301381040 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbaac0a2e20 msgr2=0x7fbaac0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:32.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.626+0000 7fbac61c1640 1 --2- 172.21.15.107:0/1301381040 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbaac0a2e20 0x7fbaac0a52e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbab0000c00 tx=0x7fbab0005a20 comp rx=0 tx=0).stop 2024-04-25T00:16:32.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.626+0000 7fbac61c1640 1 -- 172.21.15.107:0/1301381040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbac01020d0 msgr2=0x7fbac0198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:32.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.626+0000 7fbac61c1640 1 --2- 172.21.15.107:0/1301381040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbac01020d0 0x7fbac0198590 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbaa800e9f0 tx=0x7fbaa800edf0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.626+0000 7fbac61c1640 1 -- 172.21.15.107:0/1301381040 shutdown_connections 2024-04-25T00:16:32.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.626+0000 7fbac61c1640 1 --2- 172.21.15.107:0/1301381040 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbaac0a2e20 0x7fbaac0a52e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.626+0000 7fbac61c1640 1 --2- 172.21.15.107:0/1301381040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbac0107720 0x7fbac019c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.626+0000 7fbac61c1640 1 --2- 172.21.15.107:0/1301381040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbac01020d0 0x7fbac0198590 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.626+0000 7fbac61c1640 1 --2- 172.21.15.107:0/1301381040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbac0101710 0x7fbac0198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.626+0000 7fbac61c1640 1 -- 172.21.15.107:0/1301381040 >> 172.21.15.107:0/1301381040 conn(0x7fbac0078b90 msgr2=0x7fbac00ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:32.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.626+0000 7fbac61c1640 1 -- 172.21.15.107:0/1301381040 shutdown_connections 2024-04-25T00:16:32.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.626+0000 7fbac61c1640 1 -- 172.21.15.107:0/1301381040 wait complete. 2024-04-25T00:16:32.637 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:16:32.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.738+0000 7f44fefb0640 1 -- 172.21.15.107:0/3203585452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44f8106690 msgr2=0x7f44f8106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:32.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.738+0000 7f44fefb0640 1 --2- 172.21.15.107:0/3203585452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44f8106690 0x7f44f8106a70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009900 tx=0x7f44ec02a410 comp rx=0 tx=0).stop 2024-04-25T00:16:32.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.738+0000 7f44fefb0640 1 -- 172.21.15.107:0/3203585452 shutdown_connections 2024-04-25T00:16:32.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.738+0000 7f44fefb0640 1 --2- 172.21.15.107:0/3203585452 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44f8101330 0x7f44f810d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.738+0000 7f44fefb0640 1 --2- 172.21.15.107:0/3203585452 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44f8100990 0x7f44f8100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.738+0000 7f44fefb0640 1 --2- 172.21.15.107:0/3203585452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44f8106690 0x7f44f8106a70 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.738+0000 7f44fefb0640 1 -- 172.21.15.107:0/3203585452 >> 172.21.15.107:0/3203585452 conn(0x7f44f8076440 msgr2=0x7f44f8076850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:32.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.738+0000 7f44fefb0640 1 -- 172.21.15.107:0/3203585452 shutdown_connections 2024-04-25T00:16:32.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.738+0000 7f44fefb0640 1 -- 172.21.15.107:0/3203585452 wait complete. 2024-04-25T00:16:32.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.738+0000 7f44fefb0640 1 Processor -- start 2024-04-25T00:16:32.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.738+0000 7f44fefb0640 1 -- start start 2024-04-25T00:16:32.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.739+0000 7f44fefb0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44f8100990 0x7f44f8197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:32.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.739+0000 7f44fefb0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44f8101330 0x7f44f81984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:32.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.739+0000 7f44fefb0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44f8106690 0x7f44f819c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:32.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.739+0000 7f44fefb0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f44f8111ad0 con 0x7f44f8106690 2024-04-25T00:16:32.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.739+0000 7f44fefb0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f44f8111950 con 0x7f44f8101330 2024-04-25T00:16:32.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.739+0000 7f44fefb0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f44f8111c50 con 0x7f44f8100990 2024-04-25T00:16:32.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.739+0000 7f44fd54d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44f8106690 0x7f44f819c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:32.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.739+0000 7f44fd54d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44f8106690 0x7f44f819c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39134/0 (socket says 172.21.15.107:39134) 2024-04-25T00:16:32.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.739+0000 7f44fd54d640 1 -- 172.21.15.107:0/2847826629 learned_addr learned my addr 172.21.15.107:0/2847826629 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:32.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.739+0000 7f44f7fff640 1 --2- 172.21.15.107:0/2847826629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44f8101330 0x7f44f81984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:32.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.739+0000 7f44fcd4c640 1 --2- 172.21.15.107:0/2847826629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44f8100990 0x7f44f8197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:32.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.739+0000 7f44fd54d640 1 -- 172.21.15.107:0/2847826629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44f8100990 msgr2=0x7f44f8197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:32.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.739+0000 7f44fd54d640 1 --2- 172.21.15.107:0/2847826629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44f8100990 0x7f44f8197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.739+0000 7f44fd54d640 1 -- 172.21.15.107:0/2847826629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44f8101330 msgr2=0x7f44f81984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:32.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.739+0000 7f44fd54d640 1 --2- 172.21.15.107:0/2847826629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44f8101330 0x7f44f81984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.739+0000 7f44fd54d640 1 -- 172.21.15.107:0/2847826629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f819cf30 con 0x7f44f8106690 2024-04-25T00:16:32.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.739+0000 7f44fcd4c640 1 --2- 172.21.15.107:0/2847826629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44f8100990 0x7f44f8197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:32.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.739+0000 7f44fd54d640 1 --2- 172.21.15.107:0/2847826629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44f8106690 0x7f44f819c800 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f44f000eb10 tx=0x7f44f000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:32.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.739+0000 7f44f5ffb640 1 -- 172.21.15.107:0/2847826629 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0004b20 con 0x7f44f8106690 2024-04-25T00:16:32.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.739+0000 7f44f5ffb640 1 -- 172.21.15.107:0/2847826629 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44f0004cc0 con 0x7f44f8106690 2024-04-25T00:16:32.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.739+0000 7f44f5ffb640 1 -- 172.21.15.107:0/2847826629 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0010760 con 0x7f44f8106690 2024-04-25T00:16:32.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.739+0000 7f44fefb0640 1 -- 172.21.15.107:0/2847826629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f819d220 con 0x7f44f8106690 2024-04-25T00:16:32.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.740+0000 7f44fefb0640 1 -- 172.21.15.107:0/2847826629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f81a4ab0 con 0x7f44f8106690 2024-04-25T00:16:32.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.740+0000 7f44fefb0640 1 -- 172.21.15.107:0/2847826629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44f8101d80 con 0x7f44f8106690 2024-04-25T00:16:32.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.741+0000 7f44f5ffb640 1 -- 172.21.15.107:0/2847826629 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7f44f0005020 con 0x7f44f8106690 2024-04-25T00:16:32.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.742+0000 7f44f5ffb640 1 --2- 172.21.15.107:0/2847826629 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f44e80a2e20 0x7f44e80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:32.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.742+0000 7f44fcd4c640 1 --2- 172.21.15.107:0/2847826629 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f44e80a2e20 0x7f44e80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:32.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.742+0000 7f44f5ffb640 1 -- 172.21.15.107:0/2847826629 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f44f00d0150 con 0x7f44f8106690 2024-04-25T00:16:32.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.743+0000 7f44fcd4c640 1 --2- 172.21.15.107:0/2847826629 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f44e80a2e20 0x7f44e80a52e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f44ec000c00 tx=0x7f44ec032c70 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:32.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.747+0000 7f44f5ffb640 1 -- 172.21.15.107:0/2847826629 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f44f009c7b0 con 0x7f44f8106690 2024-04-25T00:16:32.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.926+0000 7f44fefb0640 1 -- 172.21.15.107:0/2847826629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f44f8199290 con 0x7f44f8106690 2024-04-25T00:16:32.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.926+0000 7f44f5ffb640 1 -- 172.21.15.107:0/2847826629 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) ==== 104+0+11 (secure 0 0 0) 0x7f44f009c0b0 con 0x7f44f8106690 2024-04-25T00:16:32.927 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-25T00:16:32.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.928+0000 7f44fefb0640 1 -- 172.21.15.107:0/2847826629 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f44e80a2e20 msgr2=0x7f44e80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:32.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.928+0000 7f44fefb0640 1 --2- 172.21.15.107:0/2847826629 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f44e80a2e20 0x7f44e80a52e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f44ec000c00 tx=0x7f44ec032c70 comp rx=0 tx=0).stop 2024-04-25T00:16:32.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.928+0000 7f44fefb0640 1 -- 172.21.15.107:0/2847826629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44f8106690 msgr2=0x7f44f819c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:32.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.928+0000 7f44fefb0640 1 --2- 172.21.15.107:0/2847826629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44f8106690 0x7f44f819c800 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f44f000eb10 tx=0x7f44f000c2d0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.928+0000 7f44fefb0640 1 -- 172.21.15.107:0/2847826629 shutdown_connections 2024-04-25T00:16:32.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.928+0000 7f44fefb0640 1 --2- 172.21.15.107:0/2847826629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44f8106690 0x7f44f819c800 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.928+0000 7f44fefb0640 1 --2- 172.21.15.107:0/2847826629 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f44e80a2e20 0x7f44e80a52e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.928+0000 7f44fefb0640 1 --2- 172.21.15.107:0/2847826629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44f8101330 0x7f44f81984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.928+0000 7f44fefb0640 1 --2- 172.21.15.107:0/2847826629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44f8100990 0x7f44f8197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.928+0000 7f44fefb0640 1 -- 172.21.15.107:0/2847826629 >> 172.21.15.107:0/2847826629 conn(0x7f44f8076440 msgr2=0x7f44f80ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:32.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.929+0000 7f44fefb0640 1 -- 172.21.15.107:0/2847826629 shutdown_connections 2024-04-25T00:16:32.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:32.929+0000 7f44fefb0640 1 -- 172.21.15.107:0/2847826629 wait complete. 2024-04-25T00:16:32.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:32 smithi107 ceph-mon[29153]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:16:32.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:32 smithi107 ceph-mon[29153]: from='client.15609 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:16:32.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:32 smithi107 ceph-mon[29153]: from='client.15615 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:16:32.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:32 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3184672373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:32.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:32 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1301381040' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:16:32.940 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:16:33.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.040+0000 7fce31672640 1 -- 172.21.15.107:0/3868098423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce2c100610 msgr2=0x7fce2c100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:33.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.040+0000 7fce31672640 1 --2- 172.21.15.107:0/3868098423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce2c100610 0x7fce2c100a90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fce20009930 tx=0x7fce2002a420 comp rx=0 tx=0).stop 2024-04-25T00:16:33.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.040+0000 7fce31672640 1 -- 172.21.15.107:0/3868098423 shutdown_connections 2024-04-25T00:16:33.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.040+0000 7fce31672640 1 --2- 172.21.15.107:0/3868098423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce2c100fd0 0x7fce2c10d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.040+0000 7fce31672640 1 --2- 172.21.15.107:0/3868098423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce2c100610 0x7fce2c100a90 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.040+0000 7fce31672640 1 --2- 172.21.15.107:0/3868098423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce2c1066b0 0x7fce2c106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.040+0000 7fce31672640 1 -- 172.21.15.107:0/3868098423 >> 172.21.15.107:0/3868098423 conn(0x7fce2c077740 msgr2=0x7fce2c077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:33.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.041+0000 7fce31672640 1 -- 172.21.15.107:0/3868098423 shutdown_connections 2024-04-25T00:16:33.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.041+0000 7fce31672640 1 -- 172.21.15.107:0/3868098423 wait complete. 2024-04-25T00:16:33.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.041+0000 7fce31672640 1 Processor -- start 2024-04-25T00:16:33.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.041+0000 7fce31672640 1 -- start start 2024-04-25T00:16:33.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.041+0000 7fce31672640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce2c100610 0x7fce2c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:33.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.041+0000 7fce31672640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce2c100fd0 0x7fce2c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:33.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.041+0000 7fce31672640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce2c1066b0 0x7fce2c19c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:33.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.041+0000 7fce31672640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c110a30 con 0x7fce2c100610 2024-04-25T00:16:33.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.041+0000 7fce31672640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c1108b0 con 0x7fce2c1066b0 2024-04-25T00:16:33.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.041+0000 7fce31672640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fce2c110bb0 con 0x7fce2c100fd0 2024-04-25T00:16:33.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.042+0000 7fce2affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce2c100610 0x7fce2c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:33.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.042+0000 7fce2affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce2c100610 0x7fce2c198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39150/0 (socket says 172.21.15.107:39150) 2024-04-25T00:16:33.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.042+0000 7fce2affd640 1 -- 172.21.15.107:0/1706270017 learned_addr learned my addr 172.21.15.107:0/1706270017 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:33.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.042+0000 7fce2b7fe640 1 --2- 172.21.15.107:0/1706270017 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce2c1066b0 0x7fce2c19c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:33.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.042+0000 7fce2a7fc640 1 --2- 172.21.15.107:0/1706270017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce2c100fd0 0x7fce2c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:33.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.042+0000 7fce2affd640 1 -- 172.21.15.107:0/1706270017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce2c100fd0 msgr2=0x7fce2c198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:33.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.042+0000 7fce2affd640 1 --2- 172.21.15.107:0/1706270017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce2c100fd0 0x7fce2c198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.042+0000 7fce2affd640 1 -- 172.21.15.107:0/1706270017 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce2c1066b0 msgr2=0x7fce2c19c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:33.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.042+0000 7fce2affd640 1 --2- 172.21.15.107:0/1706270017 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce2c1066b0 0x7fce2c19c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.042+0000 7fce2affd640 1 -- 172.21.15.107:0/1706270017 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2c19cfb0 con 0x7fce2c100610 2024-04-25T00:16:33.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.042+0000 7fce2a7fc640 1 --2- 172.21.15.107:0/1706270017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce2c100fd0 0x7fce2c198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:33.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.042+0000 7fce2affd640 1 --2- 172.21.15.107:0/1706270017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce2c100610 0x7fce2c198010 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fce1c00ea80 tx=0x7fce1c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:33.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.042+0000 7fce13fff640 1 -- 172.21.15.107:0/1706270017 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1c00ce40 con 0x7fce2c100610 2024-04-25T00:16:33.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.042+0000 7fce31672640 1 -- 172.21.15.107:0/1706270017 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce2c19d2a0 con 0x7fce2c100610 2024-04-25T00:16:33.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.042+0000 7fce13fff640 1 -- 172.21.15.107:0/1706270017 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce1c011950 con 0x7fce2c100610 2024-04-25T00:16:33.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.042+0000 7fce13fff640 1 -- 172.21.15.107:0/1706270017 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1c01ab80 con 0x7fce2c100610 2024-04-25T00:16:33.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.042+0000 7fce31672640 1 -- 172.21.15.107:0/1706270017 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce2c1a4b30 con 0x7fce2c100610 2024-04-25T00:16:33.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.043+0000 7fce31672640 1 -- 172.21.15.107:0/1706270017 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce2c068560 con 0x7fce2c100610 2024-04-25T00:16:33.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.044+0000 7fce13fff640 1 -- 172.21.15.107:0/1706270017 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7fce1c011b00 con 0x7fce2c100610 2024-04-25T00:16:33.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.045+0000 7fce13fff640 1 --2- 172.21.15.107:0/1706270017 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fce140a2e20 0x7fce140a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:33.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.045+0000 7fce2a7fc640 1 --2- 172.21.15.107:0/1706270017 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fce140a2e20 0x7fce140a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:33.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.045+0000 7fce13fff640 1 -- 172.21.15.107:0/1706270017 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fce1c0d02d0 con 0x7fce2c100610 2024-04-25T00:16:33.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.045+0000 7fce2a7fc640 1 --2- 172.21.15.107:0/1706270017 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fce140a2e20 0x7fce140a52e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fce20000c00 tx=0x7fce20032c70 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:33.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.048+0000 7fce13fff640 1 -- 172.21.15.107:0/1706270017 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fce1c016030 con 0x7fce2c100610 2024-04-25T00:16:33.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:32 smithi195 ceph-mon[32833]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:16:33.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:32 smithi195 ceph-mon[32833]: from='client.15609 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:16:33.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:32 smithi195 ceph-mon[32833]: from='client.15615 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:16:33.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:32 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3184672373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:33.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:32 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1301381040' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:16:33.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:32 smithi170 ceph-mon[32807]: from='client.15603 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:16:33.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:32 smithi170 ceph-mon[32807]: from='client.15609 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:16:33.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:32 smithi170 ceph-mon[32807]: from='client.15615 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:16:33.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:32 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3184672373' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:33.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:32 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1301381040' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:16:33.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.226+0000 7fce31672640 1 -- 172.21.15.107:0/1706270017 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fce2c199310 con 0x7fce2c100610 2024-04-25T00:16:33.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.226+0000 7fce13fff640 1 -- 172.21.15.107:0/1706270017 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) ==== 100+0+11 (secure 0 0 0) 0x7fce1c09c930 con 0x7fce2c100610 2024-04-25T00:16:33.226 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-04-25T00:16:33.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.228+0000 7fce31672640 1 -- 172.21.15.107:0/1706270017 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fce140a2e20 msgr2=0x7fce140a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:33.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.228+0000 7fce31672640 1 --2- 172.21.15.107:0/1706270017 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fce140a2e20 0x7fce140a52e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fce20000c00 tx=0x7fce20032c70 comp rx=0 tx=0).stop 2024-04-25T00:16:33.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.228+0000 7fce31672640 1 -- 172.21.15.107:0/1706270017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce2c100610 msgr2=0x7fce2c198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:33.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.228+0000 7fce31672640 1 --2- 172.21.15.107:0/1706270017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce2c100610 0x7fce2c198010 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fce1c00ea80 tx=0x7fce1c00ef40 comp rx=0 tx=0).stop 2024-04-25T00:16:33.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.228+0000 7fce31672640 1 -- 172.21.15.107:0/1706270017 shutdown_connections 2024-04-25T00:16:33.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.228+0000 7fce31672640 1 --2- 172.21.15.107:0/1706270017 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fce140a2e20 0x7fce140a52e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.228+0000 7fce31672640 1 --2- 172.21.15.107:0/1706270017 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce2c1066b0 0x7fce2c19c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.228+0000 7fce31672640 1 --2- 172.21.15.107:0/1706270017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fce2c100fd0 0x7fce2c198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.228+0000 7fce31672640 1 --2- 172.21.15.107:0/1706270017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce2c100610 0x7fce2c198010 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.228+0000 7fce31672640 1 -- 172.21.15.107:0/1706270017 >> 172.21.15.107:0/1706270017 conn(0x7fce2c077740 msgr2=0x7fce2c0ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:33.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.228+0000 7fce31672640 1 -- 172.21.15.107:0/1706270017 shutdown_connections 2024-04-25T00:16:33.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.228+0000 7fce31672640 1 -- 172.21.15.107:0/1706270017 wait complete. 2024-04-25T00:16:33.239 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:16:33.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.340+0000 7f67449ee640 1 -- 172.21.15.107:0/1795653731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6740100620 msgr2=0x7f6740100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:33.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.340+0000 7f67449ee640 1 --2- 172.21.15.107:0/1795653731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6740100620 0x7f6740100aa0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6730009930 tx=0x7f673002a420 comp rx=0 tx=0).stop 2024-04-25T00:16:33.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.340+0000 7f67449ee640 1 -- 172.21.15.107:0/1795653731 shutdown_connections 2024-04-25T00:16:33.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.340+0000 7f67449ee640 1 --2- 172.21.15.107:0/1795653731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6740100fe0 0x7f674010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.340+0000 7f67449ee640 1 --2- 172.21.15.107:0/1795653731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6740100620 0x7f6740100aa0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.340+0000 7f67449ee640 1 --2- 172.21.15.107:0/1795653731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6740106630 0x7f6740106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.340+0000 7f67449ee640 1 -- 172.21.15.107:0/1795653731 >> 172.21.15.107:0/1795653731 conn(0x7f67400776a0 msgr2=0x7f6740077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:33.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.340+0000 7f67449ee640 1 -- 172.21.15.107:0/1795653731 shutdown_connections 2024-04-25T00:16:33.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.340+0000 7f67449ee640 1 -- 172.21.15.107:0/1795653731 wait complete. 2024-04-25T00:16:33.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.341+0000 7f67449ee640 1 Processor -- start 2024-04-25T00:16:33.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.341+0000 7f67449ee640 1 -- start start 2024-04-25T00:16:33.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.341+0000 7f67449ee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6740100620 0x7f6740198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:33.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.341+0000 7f67449ee640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6740100fe0 0x7f67401993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:33.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.341+0000 7f67449ee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6740106630 0x7f67401a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:33.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.341+0000 7f67449ee640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6740110a40 con 0x7f6740100620 2024-04-25T00:16:33.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.341+0000 7f67449ee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f67401108c0 con 0x7f6740106630 2024-04-25T00:16:33.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.341+0000 7f67449ee640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6740110bc0 con 0x7f6740100fe0 2024-04-25T00:16:33.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.341+0000 7f673e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6740100620 0x7f6740198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:33.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.341+0000 7f673e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6740100620 0x7f6740198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39174/0 (socket says 172.21.15.107:39174) 2024-04-25T00:16:33.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.341+0000 7f673e59c640 1 -- 172.21.15.107:0/3117626313 learned_addr learned my addr 172.21.15.107:0/3117626313 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:33.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.341+0000 7f673ed9d640 1 --2- 172.21.15.107:0/3117626313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6740106630 0x7f67401a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:33.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.341+0000 7f673dd9b640 1 --2- 172.21.15.107:0/3117626313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6740100fe0 0x7f67401993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:33.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.341+0000 7f673e59c640 1 -- 172.21.15.107:0/3117626313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6740100fe0 msgr2=0x7f67401993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:33.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.341+0000 7f673e59c640 1 --2- 172.21.15.107:0/3117626313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6740100fe0 0x7f67401993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.341+0000 7f673e59c640 1 -- 172.21.15.107:0/3117626313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6740106630 msgr2=0x7f67401a3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:33.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.341+0000 7f673e59c640 1 --2- 172.21.15.107:0/3117626313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6740106630 0x7f67401a3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.341+0000 7f673e59c640 1 -- 172.21.15.107:0/3117626313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67401a4420 con 0x7f6740100620 2024-04-25T00:16:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.341+0000 7f673dd9b640 1 --2- 172.21.15.107:0/3117626313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6740100fe0 0x7f67401993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.341+0000 7f673e59c640 1 --2- 172.21.15.107:0/3117626313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6740100620 0x7f6740198e60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f672800d770 tx=0x7f672800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.342+0000 7f67377fe640 1 -- 172.21.15.107:0/3117626313 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728017070 con 0x7f6740100620 2024-04-25T00:16:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.342+0000 7f67449ee640 1 -- 172.21.15.107:0/3117626313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67401a45f0 con 0x7f6740100620 2024-04-25T00:16:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.342+0000 7f67377fe640 1 -- 172.21.15.107:0/3117626313 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6728005500 con 0x7f6740100620 2024-04-25T00:16:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.342+0000 7f67377fe640 1 -- 172.21.15.107:0/3117626313 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728004e00 con 0x7f6740100620 2024-04-25T00:16:33.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.342+0000 7f67449ee640 1 -- 172.21.15.107:0/3117626313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67401a4ac0 con 0x7f6740100620 2024-04-25T00:16:33.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.343+0000 7f67449ee640 1 -- 172.21.15.107:0/3117626313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67400689b0 con 0x7f6740100620 2024-04-25T00:16:33.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.344+0000 7f67377fe640 1 -- 172.21.15.107:0/3117626313 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7f6728004fa0 con 0x7f6740100620 2024-04-25T00:16:33.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.344+0000 7f67377fe640 1 --2- 172.21.15.107:0/3117626313 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f672c0a2e20 0x7f672c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:33.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.345+0000 7f673dd9b640 1 --2- 172.21.15.107:0/3117626313 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f672c0a2e20 0x7f672c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:33.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.345+0000 7f67377fe640 1 -- 172.21.15.107:0/3117626313 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f67280cf460 con 0x7f6740100620 2024-04-25T00:16:33.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.345+0000 7f673dd9b640 1 --2- 172.21.15.107:0/3117626313 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f672c0a2e20 0x7f672c0a52e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6730000c00 tx=0x7f6730032c70 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:33.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.349+0000 7f67377fe640 1 -- 172.21.15.107:0/3117626313 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f672809bac0 con 0x7f6740100620 2024-04-25T00:16:33.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.527+0000 7f67449ee640 1 -- 172.21.15.107:0/3117626313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6740108bb0 con 0x7f6740100620 2024-04-25T00:16:33.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.528+0000 7f67377fe640 1 -- 172.21.15.107:0/3117626313 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v97) ==== 103+0+11 (secure 0 0 0) 0x7f672809b3c0 con 0x7f6740100620 2024-04-25T00:16:33.529 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-25T00:16:33.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.530+0000 7f67449ee640 1 -- 172.21.15.107:0/3117626313 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f672c0a2e20 msgr2=0x7f672c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:33.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.530+0000 7f67449ee640 1 --2- 172.21.15.107:0/3117626313 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f672c0a2e20 0x7f672c0a52e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6730000c00 tx=0x7f6730032c70 comp rx=0 tx=0).stop 2024-04-25T00:16:33.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.530+0000 7f67449ee640 1 -- 172.21.15.107:0/3117626313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6740100620 msgr2=0x7f6740198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:33.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.530+0000 7f67449ee640 1 --2- 172.21.15.107:0/3117626313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6740100620 0x7f6740198e60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f672800d770 tx=0x7f672800dc30 comp rx=0 tx=0).stop 2024-04-25T00:16:33.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.530+0000 7f67449ee640 1 -- 172.21.15.107:0/3117626313 shutdown_connections 2024-04-25T00:16:33.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.530+0000 7f67449ee640 1 --2- 172.21.15.107:0/3117626313 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f672c0a2e20 0x7f672c0a52e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.530+0000 7f67449ee640 1 --2- 172.21.15.107:0/3117626313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6740106630 0x7f67401a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.530+0000 7f67449ee640 1 --2- 172.21.15.107:0/3117626313 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6740100fe0 0x7f67401993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.530+0000 7f67449ee640 1 --2- 172.21.15.107:0/3117626313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6740100620 0x7f6740198e60 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.530+0000 7f67449ee640 1 -- 172.21.15.107:0/3117626313 >> 172.21.15.107:0/3117626313 conn(0x7f67400776a0 msgr2=0x7f67401031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:33.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.530+0000 7f67449ee640 1 -- 172.21.15.107:0/3117626313 shutdown_connections 2024-04-25T00:16:33.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.530+0000 7f67449ee640 1 -- 172.21.15.107:0/3117626313 wait complete. 2024-04-25T00:16:33.542 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:16:33.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.640+0000 7fb8864a7640 1 -- 172.21.15.107:0/1904516127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8801029d0 msgr2=0x7fb880102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:33.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.640+0000 7fb8864a7640 1 --2- 172.21.15.107:0/1904516127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8801029d0 0x7fb880102db0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb870009900 tx=0x7fb87002a510 comp rx=0 tx=0).stop 2024-04-25T00:16:33.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.640+0000 7fb8864a7640 1 -- 172.21.15.107:0/1904516127 shutdown_connections 2024-04-25T00:16:33.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.640+0000 7fb8864a7640 1 --2- 172.21.15.107:0/1904516127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb88010bd10 0x7fb88010e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.640+0000 7fb8864a7640 1 --2- 172.21.15.107:0/1904516127 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8801032f0 0x7fb88010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.640+0000 7fb8864a7640 1 --2- 172.21.15.107:0/1904516127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8801029d0 0x7fb880102db0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.640+0000 7fb8864a7640 1 -- 172.21.15.107:0/1904516127 >> 172.21.15.107:0/1904516127 conn(0x7fb880077740 msgr2=0x7fb880077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:33.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.640+0000 7fb8864a7640 1 -- 172.21.15.107:0/1904516127 shutdown_connections 2024-04-25T00:16:33.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.640+0000 7fb8864a7640 1 -- 172.21.15.107:0/1904516127 wait complete. 2024-04-25T00:16:33.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.640+0000 7fb8864a7640 1 Processor -- start 2024-04-25T00:16:33.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.640+0000 7fb8864a7640 1 -- start start 2024-04-25T00:16:33.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.641+0000 7fb8864a7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8801029d0 0x7fb880197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:33.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.641+0000 7fb8864a7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8801032f0 0x7fb880198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:33.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.641+0000 7fb8864a7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb88010bd10 0x7fb88019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:33.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.641+0000 7fb8864a7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb880111b60 con 0x7fb88010bd10 2024-04-25T00:16:33.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.641+0000 7fb8864a7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb8801119e0 con 0x7fb8801032f0 2024-04-25T00:16:33.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.641+0000 7fb8864a7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb880111ce0 con 0x7fb8801029d0 2024-04-25T00:16:33.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.641+0000 7fb884a44640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb88010bd10 0x7fb88019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:33.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.641+0000 7fb884a44640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb88010bd10 0x7fb88019c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39186/0 (socket says 172.21.15.107:39186) 2024-04-25T00:16:33.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.641+0000 7fb884a44640 1 -- 172.21.15.107:0/930702207 learned_addr learned my addr 172.21.15.107:0/930702207 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:33.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.641+0000 7fb87ffff640 1 --2- 172.21.15.107:0/930702207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8801029d0 0x7fb880197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:33.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.641+0000 7fb87f7fe640 1 --2- 172.21.15.107:0/930702207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8801032f0 0x7fb880198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:33.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.641+0000 7fb884a44640 1 -- 172.21.15.107:0/930702207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8801029d0 msgr2=0x7fb880197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:33.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.641+0000 7fb884a44640 1 --2- 172.21.15.107:0/930702207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8801029d0 0x7fb880197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.641+0000 7fb884a44640 1 -- 172.21.15.107:0/930702207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8801032f0 msgr2=0x7fb880198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:33.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.641+0000 7fb884a44640 1 --2- 172.21.15.107:0/930702207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8801032f0 0x7fb880198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.641+0000 7fb884a44640 1 -- 172.21.15.107:0/930702207 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88019cfd0 con 0x7fb88010bd10 2024-04-25T00:16:33.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.641+0000 7fb87ffff640 1 --2- 172.21.15.107:0/930702207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8801029d0 0x7fb880197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:33.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.641+0000 7fb884a44640 1 --2- 172.21.15.107:0/930702207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb88010bd10 0x7fb88019c8a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb87800cce0 tx=0x7fb87800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:33.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.641+0000 7fb87d7fa640 1 -- 172.21.15.107:0/930702207 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87800f450 con 0x7fb88010bd10 2024-04-25T00:16:33.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.641+0000 7fb87d7fa640 1 -- 172.21.15.107:0/930702207 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb87800fad0 con 0x7fb88010bd10 2024-04-25T00:16:33.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.641+0000 7fb8864a7640 1 -- 172.21.15.107:0/930702207 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb88019d2c0 con 0x7fb88010bd10 2024-04-25T00:16:33.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.641+0000 7fb87d7fa640 1 -- 172.21.15.107:0/930702207 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878018c10 con 0x7fb88010bd10 2024-04-25T00:16:33.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.641+0000 7fb8864a7640 1 -- 172.21.15.107:0/930702207 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8801a4b50 con 0x7fb88010bd10 2024-04-25T00:16:33.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.643+0000 7fb8864a7640 1 -- 172.21.15.107:0/930702207 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb880068560 con 0x7fb88010bd10 2024-04-25T00:16:33.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.644+0000 7fb87d7fa640 1 -- 172.21.15.107:0/930702207 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7fb87800f5f0 con 0x7fb88010bd10 2024-04-25T00:16:33.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.644+0000 7fb87d7fa640 1 --2- 172.21.15.107:0/930702207 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb8680a2ea0 0x7fb8680a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:33.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.644+0000 7fb87ffff640 1 --2- 172.21.15.107:0/930702207 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb8680a2ea0 0x7fb8680a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:33.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.644+0000 7fb87d7fa640 1 -- 172.21.15.107:0/930702207 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fb8780d0140 con 0x7fb88010bd10 2024-04-25T00:16:33.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.645+0000 7fb87ffff640 1 --2- 172.21.15.107:0/930702207 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb8680a2ea0 0x7fb8680a5360 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb870005ab0 tx=0x7fb870000bc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:33.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.648+0000 7fb87d7fa640 1 -- 172.21.15.107:0/930702207 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb878014030 con 0x7fb88010bd10 2024-04-25T00:16:33.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.866+0000 7fb8864a7640 1 -- 172.21.15.107:0/930702207 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8801a4e50 con 0x7fb88010bd10 2024-04-25T00:16:33.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.866+0000 7fb87d7fa640 1 -- 172.21.15.107:0/930702207 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11897 (secure 0 0 0) 0x7fb87809c740 con 0x7fb88010bd10 2024-04-25T00:16:33.867 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:16:33.867 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:16:24.724978+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24493,24521]},"id":1}]} 2024-04-25T00:16:33.868 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 24 2024-04-25T00:16:33.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.868+0000 7fb8864a7640 1 -- 172.21.15.107:0/930702207 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb8680a2ea0 msgr2=0x7fb8680a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:33.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.868+0000 7fb8864a7640 1 --2- 172.21.15.107:0/930702207 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb8680a2ea0 0x7fb8680a5360 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb870005ab0 tx=0x7fb870000bc0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.868+0000 7fb8864a7640 1 -- 172.21.15.107:0/930702207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb88010bd10 msgr2=0x7fb88019c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:33.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.868+0000 7fb8864a7640 1 --2- 172.21.15.107:0/930702207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb88010bd10 0x7fb88019c8a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb87800cce0 tx=0x7fb87800a620 comp rx=0 tx=0).stop 2024-04-25T00:16:33.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.869+0000 7fb8864a7640 1 -- 172.21.15.107:0/930702207 shutdown_connections 2024-04-25T00:16:33.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.869+0000 7fb8864a7640 1 --2- 172.21.15.107:0/930702207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb88010bd10 0x7fb88019c8a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.869+0000 7fb8864a7640 1 --2- 172.21.15.107:0/930702207 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb8680a2ea0 0x7fb8680a5360 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.869+0000 7fb8864a7640 1 --2- 172.21.15.107:0/930702207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8801032f0 0x7fb880198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.869+0000 7fb8864a7640 1 --2- 172.21.15.107:0/930702207 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8801029d0 0x7fb880197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.869+0000 7fb8864a7640 1 -- 172.21.15.107:0/930702207 >> 172.21.15.107:0/930702207 conn(0x7fb880077740 msgr2=0x7fb8800fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:33.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.869+0000 7fb8864a7640 1 -- 172.21.15.107:0/930702207 shutdown_connections 2024-04-25T00:16:33.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.869+0000 7fb8864a7640 1 -- 172.21.15.107:0/930702207 wait complete. 2024-04-25T00:16:33.880 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:16:33.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.977+0000 7f7684da9640 1 -- 172.21.15.107:0/33623749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f768010bd10 msgr2=0x7f768010e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:33.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.977+0000 7f7684da9640 1 --2- 172.21.15.107:0/33623749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f768010bd10 0x7f768010e1b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7678009ca0 tx=0x7f767802a9e0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.977+0000 7f7684da9640 1 -- 172.21.15.107:0/33623749 shutdown_connections 2024-04-25T00:16:33.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.977+0000 7f7684da9640 1 --2- 172.21.15.107:0/33623749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f768010bd10 0x7f768010e1b0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.977+0000 7f7684da9640 1 --2- 172.21.15.107:0/33623749 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76801032f0 0x7f768010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.977+0000 7f7684da9640 1 --2- 172.21.15.107:0/33623749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76801029d0 0x7f7680102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.977+0000 7f7684da9640 1 -- 172.21.15.107:0/33623749 >> 172.21.15.107:0/33623749 conn(0x7f7680077740 msgr2=0x7f7680077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:33.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.978+0000 7f7684da9640 1 -- 172.21.15.107:0/33623749 shutdown_connections 2024-04-25T00:16:33.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.978+0000 7f7684da9640 1 -- 172.21.15.107:0/33623749 wait complete. 2024-04-25T00:16:33.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.978+0000 7f7684da9640 1 Processor -- start 2024-04-25T00:16:33.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.978+0000 7f7684da9640 1 -- start start 2024-04-25T00:16:33.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.978+0000 7f7684da9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76801029d0 0x7f7680198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:33.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.978+0000 7f7684da9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76801032f0 0x7f7680198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:33.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.978+0000 7f7684da9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f768010bd10 0x7f768019c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:33.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.978+0000 7f7684da9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7680111b50 con 0x7f76801029d0 2024-04-25T00:16:33.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.978+0000 7f7684da9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f76801119d0 con 0x7f76801032f0 2024-04-25T00:16:33.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.978+0000 7f7684da9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7680111cd0 con 0x7f768010bd10 2024-04-25T00:16:33.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.978+0000 7f767e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76801029d0 0x7f7680198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:33.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.979+0000 7f767e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76801029d0 0x7f7680198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39192/0 (socket says 172.21.15.107:39192) 2024-04-25T00:16:33.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.979+0000 7f767e59c640 1 -- 172.21.15.107:0/2401048682 learned_addr learned my addr 172.21.15.107:0/2401048682 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:33.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.979+0000 7f767dd9b640 1 --2- 172.21.15.107:0/2401048682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76801032f0 0x7f7680198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:33.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.979+0000 7f767e59c640 1 -- 172.21.15.107:0/2401048682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f768010bd10 msgr2=0x7f768019c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:33.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.979+0000 7f767e59c640 1 --2- 172.21.15.107:0/2401048682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f768010bd10 0x7f768019c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.979+0000 7f767e59c640 1 -- 172.21.15.107:0/2401048682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76801032f0 msgr2=0x7f7680198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:33.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.979+0000 7f767e59c640 1 --2- 172.21.15.107:0/2401048682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76801032f0 0x7f7680198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.979+0000 7f767e59c640 1 -- 172.21.15.107:0/2401048682 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768019cfd0 con 0x7f76801029d0 2024-04-25T00:16:33.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.979+0000 7f767dd9b640 1 --2- 172.21.15.107:0/2401048682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76801032f0 0x7f7680198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:33.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.979+0000 7f767e59c640 1 --2- 172.21.15.107:0/2401048682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76801029d0 0x7f7680198030 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f766800b4a0 tx=0x7f766800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:33.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.979+0000 7f76777fe640 1 -- 172.21.15.107:0/2401048682 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668014070 con 0x7f76801029d0 2024-04-25T00:16:33.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.979+0000 7f76777fe640 1 -- 172.21.15.107:0/2401048682 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7668010af0 con 0x7f76801029d0 2024-04-25T00:16:33.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.979+0000 7f7684da9640 1 -- 172.21.15.107:0/2401048682 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f768019d2c0 con 0x7f76801029d0 2024-04-25T00:16:33.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.979+0000 7f76777fe640 1 -- 172.21.15.107:0/2401048682 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668017d10 con 0x7f76801029d0 2024-04-25T00:16:33.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.979+0000 7f7684da9640 1 -- 172.21.15.107:0/2401048682 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76801a4b50 con 0x7f76801029d0 2024-04-25T00:16:33.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.980+0000 7f7684da9640 1 -- 172.21.15.107:0/2401048682 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7680068560 con 0x7f76801029d0 2024-04-25T00:16:33.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.981+0000 7f76777fe640 1 -- 172.21.15.107:0/2401048682 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7f7668017070 con 0x7f76801029d0 2024-04-25T00:16:33.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.982+0000 7f76777fe640 1 --2- 172.21.15.107:0/2401048682 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f766c0a2e20 0x7f766c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:33.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.982+0000 7f767dd9b640 1 --2- 172.21.15.107:0/2401048682 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f766c0a2e20 0x7f766c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:33.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.982+0000 7f76777fe640 1 -- 172.21.15.107:0/2401048682 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f76680ced80 con 0x7f76801029d0 2024-04-25T00:16:33.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.982+0000 7f767dd9b640 1 --2- 172.21.15.107:0/2401048682 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f766c0a2e20 0x7f766c0a52e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f768010db70 tx=0x7f7670005d30 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:33.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:33.986+0000 7f76777fe640 1 -- 172.21.15.107:0/2401048682 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f766809b4f0 con 0x7f76801029d0 2024-04-25T00:16:34.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:33 smithi195 ceph-mon[32833]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-25T00:16:34.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:33 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2847826629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:16:34.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:33 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1706270017' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:16:34.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:33 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3117626313' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:16:34.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.161+0000 7f7684da9640 1 -- 172.21.15.107:0/2401048682 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7680106900 con 0x7f76801029d0 2024-04-25T00:16:34.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.163+0000 7f76777fe640 1 -- 172.21.15.107:0/2401048682 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+25070 (secure 0 0 0) 0x7f766809adf0 con 0x7f76801029d0 2024-04-25T00:16:34.164 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:16:34.164 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":97,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:16:26.229487+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:59.672801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:00.957782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:14:05.065846+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8},{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/151603643":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/1884923543":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/111369358":"2024-04-26T00:16:26.229442+0000","172.21.15.107:6800/1644942185":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3936375305":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/1644942185":"2024-04-26T00:16:26.229442+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:16:34.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.165+0000 7f7684da9640 1 -- 172.21.15.107:0/2401048682 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f766c0a2e20 msgr2=0x7f766c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.165+0000 7f7684da9640 1 --2- 172.21.15.107:0/2401048682 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f766c0a2e20 0x7f766c0a52e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f768010db70 tx=0x7f7670005d30 comp rx=0 tx=0).stop 2024-04-25T00:16:34.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.165+0000 7f7684da9640 1 -- 172.21.15.107:0/2401048682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76801029d0 msgr2=0x7f7680198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.165+0000 7f7684da9640 1 --2- 172.21.15.107:0/2401048682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76801029d0 0x7f7680198030 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f766800b4a0 tx=0x7f766800b960 comp rx=0 tx=0).stop 2024-04-25T00:16:34.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.165+0000 7f7684da9640 1 -- 172.21.15.107:0/2401048682 shutdown_connections 2024-04-25T00:16:34.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.165+0000 7f7684da9640 1 --2- 172.21.15.107:0/2401048682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f768010bd10 0x7f768019c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.165+0000 7f7684da9640 1 --2- 172.21.15.107:0/2401048682 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f766c0a2e20 0x7f766c0a52e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.165+0000 7f7684da9640 1 --2- 172.21.15.107:0/2401048682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f76801032f0 0x7f7680198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.165+0000 7f7684da9640 1 --2- 172.21.15.107:0/2401048682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76801029d0 0x7f7680198030 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.165+0000 7f7684da9640 1 -- 172.21.15.107:0/2401048682 >> 172.21.15.107:0/2401048682 conn(0x7f7680077740 msgr2=0x7f76800fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:34.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.165+0000 7f7684da9640 1 -- 172.21.15.107:0/2401048682 shutdown_connections 2024-04-25T00:16:34.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.165+0000 7f7684da9640 1 -- 172.21.15.107:0/2401048682 wait complete. 2024-04-25T00:16:34.176 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:33 smithi107 ceph-mon[29153]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-25T00:16:34.176 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:33 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2847826629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:16:34.176 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:33 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1706270017' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:16:34.176 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:33 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3117626313' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:16:34.178 DEBUG:tasks.fwd_scrub:Yielding 2024-04-25T00:16:34.178 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T00:16:34.188 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T00:16:34.188 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T00:16:34.189 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-25T00:16:34.189 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-25T00:16:34.189 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:16:34.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:16:34.189 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:16:34.190 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:33 smithi170 ceph-mon[32807]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-25T00:16:34.191 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:33 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2847826629' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:16:34.191 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:33 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1706270017' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:16:34.191 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:33 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3117626313' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:16:34.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.292+0000 7f24be0a5640 1 -- 172.21.15.107:0/4035508781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24b810bd10 msgr2=0x7f24b810e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.292+0000 7f24be0a5640 1 --2- 172.21.15.107:0/4035508781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24b810bd10 0x7f24b810e1b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009ca0 tx=0x7f24b002a9e0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.292+0000 7f24be0a5640 1 -- 172.21.15.107:0/4035508781 shutdown_connections 2024-04-25T00:16:34.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.292+0000 7f24be0a5640 1 --2- 172.21.15.107:0/4035508781 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24b810bd10 0x7f24b810e1b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.292+0000 7f24be0a5640 1 --2- 172.21.15.107:0/4035508781 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24b81032f0 0x7f24b810b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.292+0000 7f24be0a5640 1 --2- 172.21.15.107:0/4035508781 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24b81029d0 0x7f24b8102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.292+0000 7f24be0a5640 1 -- 172.21.15.107:0/4035508781 >> 172.21.15.107:0/4035508781 conn(0x7f24b80777c0 msgr2=0x7f24b8077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:34.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.292+0000 7f7911661640 1 -- 172.21.15.107:0/2579349053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f790c10bd00 msgr2=0x7f790c10e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.292+0000 7f7911661640 1 --2- 172.21.15.107:0/2579349053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f790c10bd00 0x7f790c10e1a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7904009be0 tx=0x7f790402a7e0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.292+0000 7f24be0a5640 1 -- 172.21.15.107:0/4035508781 shutdown_connections 2024-04-25T00:16:34.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.292+0000 7f7911661640 1 -- 172.21.15.107:0/2579349053 shutdown_connections 2024-04-25T00:16:34.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.292+0000 7f7911661640 1 --2- 172.21.15.107:0/2579349053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f790c10bd00 0x7f790c10e1a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.292+0000 7f7911661640 1 --2- 172.21.15.107:0/2579349053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f790c1032e0 0x7f790c10b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.292+0000 7f7911661640 1 --2- 172.21.15.107:0/2579349053 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f790c1029c0 0x7f790c102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.292+0000 7f7911661640 1 -- 172.21.15.107:0/2579349053 >> 172.21.15.107:0/2579349053 conn(0x7f790c077740 msgr2=0x7f790c077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:34.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.292+0000 7f24be0a5640 1 -- 172.21.15.107:0/4035508781 wait complete. 2024-04-25T00:16:34.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.293+0000 7f7911661640 1 -- 172.21.15.107:0/2579349053 shutdown_connections 2024-04-25T00:16:34.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.293+0000 7f7911661640 1 -- 172.21.15.107:0/2579349053 wait complete. 2024-04-25T00:16:34.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.293+0000 7f24be0a5640 1 Processor -- start 2024-04-25T00:16:34.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.293+0000 7f7911661640 1 Processor -- start 2024-04-25T00:16:34.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.293+0000 7f24be0a5640 1 -- start start 2024-04-25T00:16:34.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.293+0000 7f7911661640 1 -- start start 2024-04-25T00:16:34.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.293+0000 7f24be0a5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24b81029d0 0x7f24b8197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.293+0000 7f24be0a5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24b81032f0 0x7f24b8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.293+0000 7f24be0a5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24b810bd10 0x7f24b819c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.293+0000 7f24be0a5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f24b8111b10 con 0x7f24b81032f0 2024-04-25T00:16:34.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.293+0000 7f24be0a5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f24b8111990 con 0x7f24b81029d0 2024-04-25T00:16:34.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.293+0000 7f24be0a5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f24b8111c90 con 0x7f24b810bd10 2024-04-25T00:16:34.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.293+0000 7f7911661640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f790c1029c0 0x7f790c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.293+0000 7f7911661640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f790c1032e0 0x7f790c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.293+0000 7f7911661640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f790c10bd00 0x7f790c19c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.293+0000 7f7911661640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f790c111b00 con 0x7f790c10bd00 2024-04-25T00:16:34.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.293+0000 7f7911661640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f790c111980 con 0x7f790c1029c0 2024-04-25T00:16:34.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f7911661640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f790c111c80 con 0x7f790c1032e0 2024-04-25T00:16:34.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f24b6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24b81032f0 0x7f24b8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f790b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f790c10bd00 0x7f790c19c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f24b6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24b81032f0 0x7f24b8198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39226/0 (socket says 172.21.15.107:39226) 2024-04-25T00:16:34.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f24b77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24b81029d0 0x7f24b8197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f24b77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24b81029d0 0x7f24b8197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.107:60274/0 (socket says 172.21.15.107:60274) 2024-04-25T00:16:34.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f24b6ffd640 1 -- 172.21.15.107:0/3052808682 learned_addr learned my addr 172.21.15.107:0/3052808682 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:34.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f790affd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f790c1029c0 0x7f790c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f790b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f790c10bd00 0x7f790c19c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39236/0 (socket says 172.21.15.107:39236) 2024-04-25T00:16:34.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f790b7fe640 1 -- 172.21.15.107:0/3629683613 learned_addr learned my addr 172.21.15.107:0/3629683613 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:34.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f790a7fc640 1 --2- 172.21.15.107:0/3629683613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f790c1032e0 0x7f790c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f24b7fff640 1 --2- 172.21.15.107:0/3052808682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24b810bd10 0x7f24b819c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f790b7fe640 1 -- 172.21.15.107:0/3629683613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f790c1032e0 msgr2=0x7f790c198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f790b7fe640 1 --2- 172.21.15.107:0/3629683613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f790c1032e0 0x7f790c198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f790b7fe640 1 -- 172.21.15.107:0/3629683613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f790c1029c0 msgr2=0x7f790c197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f790b7fe640 1 --2- 172.21.15.107:0/3629683613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f790c1029c0 0x7f790c197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f790b7fe640 1 -- 172.21.15.107:0/3629683613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790c19cfd0 con 0x7f790c10bd00 2024-04-25T00:16:34.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f24b6ffd640 1 -- 172.21.15.107:0/3052808682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24b810bd10 msgr2=0x7f24b819c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f24b6ffd640 1 --2- 172.21.15.107:0/3052808682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24b810bd10 0x7f24b819c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f24b6ffd640 1 -- 172.21.15.107:0/3052808682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24b81029d0 msgr2=0x7f24b8197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f24b6ffd640 1 --2- 172.21.15.107:0/3052808682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24b81029d0 0x7f24b8197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f24b6ffd640 1 -- 172.21.15.107:0/3052808682 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b819ce90 con 0x7f24b81032f0 2024-04-25T00:16:34.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f24b77fe640 1 --2- 172.21.15.107:0/3052808682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24b81029d0 0x7f24b8197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:34.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f790affd640 1 --2- 172.21.15.107:0/3629683613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f790c1029c0 0x7f790c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:34.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f790b7fe640 1 --2- 172.21.15.107:0/3629683613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f790c10bd00 0x7f790c19c8a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7904034040 tx=0x7f79040305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:34.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f24b6ffd640 1 --2- 172.21.15.107:0/3052808682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24b81032f0 0x7f24b8198430 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f24ac00c920 tx=0x7f24ac00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:34.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f78f3fff640 1 -- 172.21.15.107:0/3629683613 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904040020 con 0x7f790c10bd00 2024-04-25T00:16:34.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f78f3fff640 1 -- 172.21.15.107:0/3629683613 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7904003e50 con 0x7f790c10bd00 2024-04-25T00:16:34.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f78f3fff640 1 -- 172.21.15.107:0/3629683613 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904036040 con 0x7f790c10bd00 2024-04-25T00:16:34.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f7911661640 1 -- 172.21.15.107:0/3629683613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f790c19d260 con 0x7f790c10bd00 2024-04-25T00:16:34.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f24b4ff9640 1 -- 172.21.15.107:0/3052808682 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ac007c70 con 0x7f24b81032f0 2024-04-25T00:16:34.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f24b4ff9640 1 -- 172.21.15.107:0/3052808682 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24ac007e10 con 0x7f24b81032f0 2024-04-25T00:16:34.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f24b4ff9640 1 -- 172.21.15.107:0/3052808682 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ac018890 con 0x7f24b81032f0 2024-04-25T00:16:34.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.294+0000 7f7911661640 1 -- 172.21.15.107:0/3629683613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f790c1a4b50 con 0x7f790c10bd00 2024-04-25T00:16:34.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.295+0000 7f24be0a5640 1 -- 172.21.15.107:0/3052808682 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b819d180 con 0x7f24b81032f0 2024-04-25T00:16:34.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.295+0000 7f24be0a5640 1 -- 172.21.15.107:0/3052808682 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b81a4a10 con 0x7f24b81032f0 2024-04-25T00:16:34.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.296+0000 7f24be0a5640 1 -- 172.21.15.107:0/3052808682 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24b804f500 con 0x7f24b81032f0 2024-04-25T00:16:34.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.297+0000 7f78f3fff640 1 -- 172.21.15.107:0/3629683613 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7f7904043020 con 0x7f790c10bd00 2024-04-25T00:16:34.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.297+0000 7f7914368740 1 Processor -- start 2024-04-25T00:16:34.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.298+0000 7f7914368740 1 -- start start 2024-04-25T00:16:34.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.298+0000 7f7914368740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c479980 0x55b90c477cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.298+0000 7f7914368740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b90c478290 0x55b90c478710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.298+0000 7f7914368740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b90c487420 0x55b90c489810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.298+0000 7f7914368740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b90c3dc2c0 con 0x55b90c479980 2024-04-25T00:16:34.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.298+0000 7f7914368740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b90c384910 con 0x55b90c478290 2024-04-25T00:16:34.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.298+0000 7f7914368740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b90c3d8280 con 0x55b90c487420 2024-04-25T00:16:34.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.298+0000 7f24b4ff9640 1 -- 172.21.15.107:0/3052808682 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7f24ac018a50 con 0x7f24b81032f0 2024-04-25T00:16:34.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.298+0000 7f790a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c479980 0x55b90c477cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.298+0000 7f78f3fff640 1 --2- 172.21.15.107:0/3629683613 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f78f40a2e20 0x7f78f40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.298+0000 7f790a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c479980 0x55b90c477cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39238/0 (socket says 172.21.15.107:39238) 2024-04-25T00:16:34.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.298+0000 7f790a7fc640 1 -- 172.21.15.107:0/3098008552 learned_addr learned my addr 172.21.15.107:0/3098008552 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:34.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.298+0000 7f790affd640 1 --2- 172.21.15.107:0/3098008552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b90c478290 0x55b90c478710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.298+0000 7f790a7fc640 1 --2- 172.21.15.107:0/3098008552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b90c487420 0x55b90c489810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.298+0000 7f790b7fe640 1 --2- 172.21.15.107:0/3629683613 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f78f40a2e20 0x7f78f40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.298+0000 7f790a7fc640 1 -- 172.21.15.107:0/3098008552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b90c487420 msgr2=0x55b90c489810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:34.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.298+0000 7f790a7fc640 1 --2- 172.21.15.107:0/3098008552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b90c487420 0x55b90c489810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.298+0000 7f78f3fff640 1 -- 172.21.15.107:0/3629683613 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f79040f0300 con 0x7f790c10bd00 2024-04-25T00:16:34.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.298+0000 7f790a7fc640 1 -- 172.21.15.107:0/3098008552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b90c478290 msgr2=0x55b90c478710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:34.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.298+0000 7f790a7fc640 1 --2- 172.21.15.107:0/3098008552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b90c478290 0x55b90c478710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.298+0000 7f790a7fc640 1 -- 172.21.15.107:0/3098008552 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b90c395fb0 con 0x55b90c479980 2024-04-25T00:16:34.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.298+0000 7f790affd640 1 --2- 172.21.15.107:0/3098008552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b90c478290 0x55b90c478710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:34.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.298+0000 7f790a7fc640 1 --2- 172.21.15.107:0/3098008552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c479980 0x55b90c477cc0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7900009f80 tx=0x7f7900009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7bea9b18e078e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:34.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.299+0000 7f24b4ff9640 1 --2- 172.21.15.107:0/3052808682 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f24a00a2f40 0x7f24a00a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.299+0000 7f790b7fe640 1 --2- 172.21.15.107:0/3629683613 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f78f40a2e20 0x7f78f40a52e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f79040ed5c0 tx=0x7f790404ab90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:34.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.299+0000 7f24b77fe640 1 --2- 172.21.15.107:0/3052808682 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f24a00a2f40 0x7f24a00a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.299+0000 7f24b4ff9640 1 -- 172.21.15.107:0/3052808682 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f24ac0cfdb0 con 0x7f24b81032f0 2024-04-25T00:16:34.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.299+0000 7f24b77fe640 1 --2- 172.21.15.107:0/3052808682 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f24a00a2f40 0x7f24a00a5400 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f24a8009fc0 tx=0x7f24a8005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:34.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.302+0000 7f78f17fa640 1 -- 172.21.15.107:0/3098008552 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790000f480 con 0x55b90c479980 2024-04-25T00:16:34.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.302+0000 7f78f17fa640 1 -- 172.21.15.107:0/3098008552 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f790000fb00 con 0x55b90c479980 2024-04-25T00:16:34.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.302+0000 7f78f17fa640 1 -- 172.21.15.107:0/3098008552 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900010ab0 con 0x55b90c479980 2024-04-25T00:16:34.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.302+0000 7f24b4ff9640 1 -- 172.21.15.107:0/3052808682 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f24ac014030 con 0x7f24b81032f0 2024-04-25T00:16:34.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.303+0000 7f7914368740 1 -- 172.21.15.107:0/3098008552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c479980 msgr2=0x55b90c477cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.303+0000 7f7914368740 1 --2- 172.21.15.107:0/3098008552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c479980 0x55b90c477cc0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7900009f80 tx=0x7f7900009790 comp rx=0 tx=0).stop 2024-04-25T00:16:34.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.303+0000 7f7914368740 1 -- 172.21.15.107:0/3098008552 shutdown_connections 2024-04-25T00:16:34.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.303+0000 7f7914368740 1 --2- 172.21.15.107:0/3098008552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b90c487420 0x55b90c489810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.303+0000 7f7914368740 1 --2- 172.21.15.107:0/3098008552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b90c478290 0x55b90c478710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.303+0000 7f7914368740 1 --2- 172.21.15.107:0/3098008552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c479980 0x55b90c477cc0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.303+0000 7f7914368740 1 -- 172.21.15.107:0/3098008552 >> 172.21.15.107:0/3098008552 conn(0x55b90c440950 msgr2=0x55b90c4747f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:34.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.303+0000 7f7914368740 1 -- 172.21.15.107:0/3098008552 shutdown_connections 2024-04-25T00:16:34.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.303+0000 7f7914368740 1 -- 172.21.15.107:0/3098008552 wait complete. 2024-04-25T00:16:34.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.303+0000 7f7914368740 1 -- start start 2024-04-25T00:16:34.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.303+0000 7f7914368740 1 Processor -- start 2024-04-25T00:16:34.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.304+0000 7f7914368740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b90c478290 0x55b90c484340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.304+0000 7f7914368740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b90c479980 0x55b90c484880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.304+0000 7f7914368740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c487420 0x55b90c484dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.304+0000 7f7914368740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b90c48a2b0 con 0x55b90c487420 2024-04-25T00:16:34.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.304+0000 7f7914368740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b90c397e90 con 0x55b90c478290 2024-04-25T00:16:34.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.304+0000 7f7914368740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b90c48a430 con 0x55b90c479980 2024-04-25T00:16:34.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.304+0000 7f7914368740 10 client.0 _ll_register_callbacks cb 0x55b90c3d3970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:16:34.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.304+0000 7f790a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c487420 0x55b90c484dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.304+0000 7f790a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c487420 0x55b90c484dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39242/0 (socket says 172.21.15.107:39242) 2024-04-25T00:16:34.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.304+0000 7f790a7fc640 1 -- 172.21.15.107:0/1161520499 learned_addr learned my addr 172.21.15.107:0/1161520499 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:34.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.304+0000 7f790affd640 1 --2- 172.21.15.107:0/1161520499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b90c479980 0x55b90c484880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.304+0000 7f790a7fc640 1 --2- 172.21.15.107:0/1161520499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b90c478290 0x55b90c484340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.304+0000 7f790a7fc640 1 -- 172.21.15.107:0/1161520499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b90c479980 msgr2=0x55b90c484880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:34.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.304+0000 7f790a7fc640 1 --2- 172.21.15.107:0/1161520499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b90c479980 0x55b90c484880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.304+0000 7f790a7fc640 1 -- 172.21.15.107:0/1161520499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b90c478290 msgr2=0x55b90c484340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:34.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.304+0000 7f790a7fc640 1 --2- 172.21.15.107:0/1161520499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b90c478290 0x55b90c484340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.304+0000 7f790a7fc640 1 -- 172.21.15.107:0/1161520499 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b90c1b87e0 con 0x55b90c487420 2024-04-25T00:16:34.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.304+0000 7f790affd640 1 --2- 172.21.15.107:0/1161520499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b90c479980 0x55b90c484880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:34.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.304+0000 7f790a7fc640 1 --2- 172.21.15.107:0/1161520499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c487420 0x55b90c484dc0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7900012180 tx=0x7f7900009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08407897607fa70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:34.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.304+0000 7f78eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:16:34.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.304+0000 7f78eeff5640 1 -- 172.21.15.107:0/1161520499 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79000121e0 con 0x55b90c487420 2024-04-25T00:16:34.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.304+0000 7f7914368740 1 -- 172.21.15.107:0/1161520499 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f790000fb00 con 0x55b90c487420 2024-04-25T00:16:34.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.304+0000 7f78eeff5640 1 -- 172.21.15.107:0/1161520499 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f7900004260 con 0x55b90c487420 2024-04-25T00:16:34.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.305+0000 7f78eeff5640 1 -- 172.21.15.107:0/1161520499 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7900004460 con 0x55b90c487420 2024-04-25T00:16:34.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.305+0000 7f7914368740 10 client.15675 fetch_fsmap learned FSMap version 24 2024-04-25T00:16:34.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.305+0000 7f7914368740 1 -- 172.21.15.107:0/1161520499 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b90c218f70 con 0x55b90c487420 2024-04-25T00:16:34.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.305+0000 7f78eeff5640 1 -- 172.21.15.107:0/1161520499 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f790000fb00 con 0x55b90c487420 2024-04-25T00:16:34.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.305+0000 7f78eeff5640 1 -- 172.21.15.107:0/1161520499 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f7900010850 con 0x55b90c487420 2024-04-25T00:16:34.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.305+0000 7f7914368740 10 client.15675 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:16:34.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.305+0000 7f7914368740 10 client.15675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:16:34.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.305+0000 7f7914368740 20 client.15675 populate_metadata read hostname 'smithi107' 2024-04-25T00:16:34.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.305+0000 7f7914368740 1 --2- 172.21.15.107:0/1161520499 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b90c4973d0 0x55b90c4997c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.305+0000 7f7914368740 4 client.15675 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:16:34.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.305+0000 7f7914368740 1 -- 172.21.15.107:0/1161520499 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b90c3b05a0 con 0x55b90c4973d0 2024-04-25T00:16:34.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.305+0000 7f790affd640 1 --2- 172.21.15.107:0/1161520499 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b90c4973d0 0x55b90c4997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.306+0000 7f790affd640 1 --2- 172.21.15.107:0/1161520499 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b90c4973d0 0x55b90c4997c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7571967892d6e90b server_cookie=f7fe07222d1887fd in_seq=0 out_seq=0 2024-04-25T00:16:34.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.306+0000 7f78eeff5640 10 client.15675 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:16:34.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.307+0000 7f78eeff5640 1 -- 172.21.15.107:0/1161520499 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b90c3b05a0 con 0x55b90c4973d0 2024-04-25T00:16:34.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.307+0000 7f78eeff5640 10 client.15675 handle_command_reply: tid=0 2024-04-25T00:16:34.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.307+0000 7f7914368740 1 client.15675 shutdown 2024-04-25T00:16:34.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.307+0000 7f7914368740 1 -- 172.21.15.107:0/1161520499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c487420 msgr2=0x55b90c484dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.307+0000 7f7914368740 1 --2- 172.21.15.107:0/1161520499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c487420 0x55b90c484dc0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7900012180 tx=0x7f7900009240 comp rx=0 tx=0).stop 2024-04-25T00:16:34.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.307+0000 7f7914368740 1 -- 172.21.15.107:0/1161520499 shutdown_connections 2024-04-25T00:16:34.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.307+0000 7f7914368740 1 --2- 172.21.15.107:0/1161520499 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b90c4973d0 0x55b90c4997c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.307+0000 7f7914368740 1 --2- 172.21.15.107:0/1161520499 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c487420 0x55b90c484dc0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.307+0000 7f7914368740 1 --2- 172.21.15.107:0/1161520499 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b90c479980 0x55b90c484880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.307+0000 7f7914368740 1 --2- 172.21.15.107:0/1161520499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b90c478290 0x55b90c484340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.307+0000 7f78eeff5640 0 client.15675 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:16:34.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.307+0000 7f7914368740 1 -- 172.21.15.107:0/1161520499 >> 172.21.15.107:0/1161520499 conn(0x55b90c440950 msgr2=0x55b90c47a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:34.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.307+0000 7f7914368740 1 -- 172.21.15.107:0/1161520499 shutdown_connections 2024-04-25T00:16:34.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.307+0000 7f7914368740 1 -- 172.21.15.107:0/1161520499 wait complete. 2024-04-25T00:16:34.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.307+0000 7f7914368740 20 client.15675 trim_cache size 0 max 16384 2024-04-25T00:16:34.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.324+0000 7f7914368740 1 Processor -- start 2024-04-25T00:16:34.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.324+0000 7f7914368740 1 -- start start 2024-04-25T00:16:34.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.325+0000 7f7914368740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b90c478290 0x55b90c4931e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.325+0000 7f7914368740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c479980 0x55b90c49efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.325+0000 7f7914368740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b90c487420 0x55b90c49f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.325+0000 7f7914368740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f790c1af240 con 0x55b90c479980 2024-04-25T00:16:34.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.325+0000 7f7914368740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f790c1acb10 con 0x55b90c487420 2024-04-25T00:16:34.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.325+0000 7f7914368740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f790c1aa960 con 0x55b90c478290 2024-04-25T00:16:34.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.325+0000 7f790affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c479980 0x55b90c49efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.325+0000 7f790affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c479980 0x55b90c49efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39246/0 (socket says 172.21.15.107:39246) 2024-04-25T00:16:34.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.325+0000 7f790affd640 1 -- 172.21.15.107:0/1275161863 learned_addr learned my addr 172.21.15.107:0/1275161863 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:34.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.325+0000 7f790a7fc640 1 --2- 172.21.15.107:0/1275161863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b90c478290 0x55b90c4931e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.325+0000 7f790a7fc640 1 --2- 172.21.15.107:0/1275161863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b90c487420 0x55b90c49f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.325+0000 7f790affd640 1 -- 172.21.15.107:0/1275161863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b90c478290 msgr2=0x55b90c4931e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:34.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.325+0000 7f790affd640 1 --2- 172.21.15.107:0/1275161863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b90c478290 0x55b90c4931e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.325+0000 7f790affd640 1 -- 172.21.15.107:0/1275161863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b90c487420 msgr2=0x55b90c49f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:34.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.325+0000 7f790affd640 1 --2- 172.21.15.107:0/1275161863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b90c487420 0x55b90c49f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.325+0000 7f790affd640 1 -- 172.21.15.107:0/1275161863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b90c0b0170 con 0x55b90c479980 2024-04-25T00:16:34.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.325+0000 7f790a7fc640 1 --2- 172.21.15.107:0/1275161863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b90c487420 0x55b90c49f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:34.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.325+0000 7f790affd640 1 --2- 172.21.15.107:0/1275161863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c479980 0x55b90c49efa0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f78fc012700 tx=0x7f78fc005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7b7110a48b7865 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:34.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.325+0000 7f78f0ff9640 1 -- 172.21.15.107:0/1275161863 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc00fd00 con 0x55b90c479980 2024-04-25T00:16:34.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.326+0000 7f78f0ff9640 1 -- 172.21.15.107:0/1275161863 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78fc027070 con 0x55b90c479980 2024-04-25T00:16:34.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.326+0000 7f78f0ff9640 1 -- 172.21.15.107:0/1275161863 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc010690 con 0x55b90c479980 2024-04-25T00:16:34.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.326+0000 7f7914368740 1 -- 172.21.15.107:0/1275161863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c479980 msgr2=0x55b90c49efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.326+0000 7f7914368740 1 --2- 172.21.15.107:0/1275161863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c479980 0x55b90c49efa0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f78fc012700 tx=0x7f78fc005f90 comp rx=0 tx=0).stop 2024-04-25T00:16:34.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.326+0000 7f7914368740 1 -- 172.21.15.107:0/1275161863 shutdown_connections 2024-04-25T00:16:34.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.326+0000 7f7914368740 1 --2- 172.21.15.107:0/1275161863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b90c487420 0x55b90c49f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.326+0000 7f7914368740 1 --2- 172.21.15.107:0/1275161863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c479980 0x55b90c49efa0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.326+0000 7f7914368740 1 --2- 172.21.15.107:0/1275161863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b90c478290 0x55b90c4931e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.326+0000 7f7914368740 1 -- 172.21.15.107:0/1275161863 >> 172.21.15.107:0/1275161863 conn(0x55b90c440950 msgr2=0x55b90c473950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:34.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.326+0000 7f7914368740 1 -- 172.21.15.107:0/1275161863 shutdown_connections 2024-04-25T00:16:34.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.326+0000 7f7914368740 1 -- 172.21.15.107:0/1275161863 wait complete. 2024-04-25T00:16:34.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.327+0000 7f7914368740 1 -- start start 2024-04-25T00:16:34.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.327+0000 7f7914368740 1 Processor -- start 2024-04-25T00:16:34.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.327+0000 7f7914368740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b90c478290 0x55b90c48fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.327+0000 7f7914368740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b90c479980 0x55b90c4818d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.327+0000 7f7914368740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c487420 0x55b90c481f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.327+0000 7f7914368740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f790c1aafd0 con 0x55b90c487420 2024-04-25T00:16:34.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.327+0000 7f7914368740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f790c1ac440 con 0x55b90c478290 2024-04-25T00:16:34.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.327+0000 7f7914368740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f790c1ab6a0 con 0x55b90c479980 2024-04-25T00:16:34.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.327+0000 7f7914368740 10 client.0 _ll_register_callbacks cb 0x7f79000123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:16:34.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.327+0000 7f790a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c487420 0x55b90c481f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.327+0000 7f790a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c487420 0x55b90c481f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39250/0 (socket says 172.21.15.107:39250) 2024-04-25T00:16:34.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.327+0000 7f790a7fc640 1 -- 172.21.15.107:0/3933595579 learned_addr learned my addr 172.21.15.107:0/3933595579 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:34.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.327+0000 7f790a7fc640 1 --2- 172.21.15.107:0/3933595579 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b90c478290 0x55b90c48fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.327+0000 7f790affd640 1 --2- 172.21.15.107:0/3933595579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b90c479980 0x55b90c4818d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.327+0000 7f790a7fc640 1 -- 172.21.15.107:0/3933595579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b90c479980 msgr2=0x55b90c4818d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:34.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.327+0000 7f790a7fc640 1 --2- 172.21.15.107:0/3933595579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b90c479980 0x55b90c4818d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.327+0000 7f790a7fc640 1 -- 172.21.15.107:0/3933595579 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b90c478290 msgr2=0x55b90c48fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:34.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.327+0000 7f790a7fc640 1 --2- 172.21.15.107:0/3933595579 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b90c478290 0x55b90c48fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.327+0000 7f790a7fc640 1 -- 172.21.15.107:0/3933595579 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b90c488150 con 0x55b90c487420 2024-04-25T00:16:34.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.327+0000 7f790a7fc640 1 --2- 172.21.15.107:0/3933595579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c487420 0x55b90c481f00 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7900010cf0 tx=0x7f7900009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2cd7d423688dd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:34.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.327+0000 7f78eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:16:34.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.328+0000 7f78eeff5640 1 -- 172.21.15.107:0/3933595579 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79000121b0 con 0x55b90c487420 2024-04-25T00:16:34.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.328+0000 7f78eeff5640 1 -- 172.21.15.107:0/3933595579 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f79000092b0 con 0x55b90c487420 2024-04-25T00:16:34.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.328+0000 7f7914368740 1 -- 172.21.15.107:0/3933595579 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78fc027070 con 0x55b90c487420 2024-04-25T00:16:34.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.328+0000 7f78eeff5640 1 -- 172.21.15.107:0/3933595579 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f790000f480 con 0x55b90c487420 2024-04-25T00:16:34.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.328+0000 7f7914368740 10 client.15681 fetch_fsmap learned FSMap version 24 2024-04-25T00:16:34.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.328+0000 7f7914368740 1 -- 172.21.15.107:0/3933595579 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b90c4fdc20 con 0x55b90c487420 2024-04-25T00:16:34.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.328+0000 7f78eeff5640 1 -- 172.21.15.107:0/3933595579 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78fc027070 con 0x55b90c487420 2024-04-25T00:16:34.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.328+0000 7f78eeff5640 1 -- 172.21.15.107:0/3933595579 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f7900003c60 con 0x55b90c487420 2024-04-25T00:16:34.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.328+0000 7f7914368740 10 client.15681 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:16:34.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.328+0000 7f7914368740 10 client.15681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:16:34.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.328+0000 7f7914368740 20 client.15681 populate_metadata read hostname 'smithi107' 2024-04-25T00:16:34.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.328+0000 7f7914368740 1 --2- 172.21.15.107:0/3933595579 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b90c4fe120 0x55b90c500540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.328+0000 7f7914368740 4 client.15681 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:16:34.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.328+0000 7f7914368740 1 -- 172.21.15.107:0/3933595579 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b90c3e9800 con 0x55b90c4fe120 2024-04-25T00:16:34.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.328+0000 7f790affd640 1 --2- 172.21.15.107:0/3933595579 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b90c4fe120 0x55b90c500540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.329+0000 7f78eeff5640 10 client.15681 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:16:34.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.329+0000 7f790affd640 1 --2- 172.21.15.107:0/3933595579 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b90c4fe120 0x55b90c500540 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=749059d08fb16794 server_cookie=bc1d5251e19c8fda in_seq=0 out_seq=0 2024-04-25T00:16:34.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.330+0000 7f78eeff5640 1 -- 172.21.15.107:0/3933595579 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b90c3e9800 con 0x55b90c4fe120 2024-04-25T00:16:34.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.330+0000 7f78eeff5640 10 client.15681 handle_command_reply: tid=0 2024-04-25T00:16:34.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.330+0000 7f7914368740 1 client.15681 shutdown 2024-04-25T00:16:34.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.330+0000 7f7914368740 1 -- 172.21.15.107:0/3933595579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c487420 msgr2=0x55b90c481f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.330+0000 7f7914368740 1 --2- 172.21.15.107:0/3933595579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c487420 0x55b90c481f00 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7900010cf0 tx=0x7f7900009240 comp rx=0 tx=0).stop 2024-04-25T00:16:34.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.330+0000 7f7914368740 1 -- 172.21.15.107:0/3933595579 shutdown_connections 2024-04-25T00:16:34.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.330+0000 7f7914368740 1 --2- 172.21.15.107:0/3933595579 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b90c4fe120 0x55b90c500540 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.330+0000 7f7914368740 1 --2- 172.21.15.107:0/3933595579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b90c487420 0x55b90c481f00 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.330+0000 7f7914368740 1 --2- 172.21.15.107:0/3933595579 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b90c479980 0x55b90c4818d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.330+0000 7f7914368740 1 --2- 172.21.15.107:0/3933595579 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b90c478290 0x55b90c48fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.330+0000 7f78eeff5640 0 client.15681 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:16:34.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.330+0000 7f7914368740 1 -- 172.21.15.107:0/3933595579 >> 172.21.15.107:0/3933595579 conn(0x55b90c440950 msgr2=0x55b90c482ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:34.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.331+0000 7f7914368740 1 -- 172.21.15.107:0/3933595579 shutdown_connections 2024-04-25T00:16:34.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.331+0000 7f7914368740 1 -- 172.21.15.107:0/3933595579 wait complete. 2024-04-25T00:16:34.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.331+0000 7f7914368740 20 client.15681 trim_cache size 0 max 16384 2024-04-25T00:16:34.357 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:16:34.357 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-04-25T00:16:34.358 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "549e423b-455f-47f2-b0e7-fd93cf70e437", 2024-04-25T00:16:34.358 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-04-25T00:16:34.358 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:16:34.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.331+0000 7f7911661640 1 -- 172.21.15.107:0/3629683613 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f78f40a2e20 msgr2=0x7f78f40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.331+0000 7f7911661640 1 --2- 172.21.15.107:0/3629683613 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f78f40a2e20 0x7f78f40a52e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f79040ed5c0 tx=0x7f790404ab90 comp rx=0 tx=0).stop 2024-04-25T00:16:34.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.331+0000 7f7911661640 1 -- 172.21.15.107:0/3629683613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f790c10bd00 msgr2=0x7f790c19c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.331+0000 7f7911661640 1 --2- 172.21.15.107:0/3629683613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f790c10bd00 0x7f790c19c8a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7904034040 tx=0x7f79040305d0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.331+0000 7f7911661640 1 -- 172.21.15.107:0/3629683613 shutdown_connections 2024-04-25T00:16:34.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.331+0000 7f7911661640 1 --2- 172.21.15.107:0/3629683613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f790c10bd00 0x7f790c19c8a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.331+0000 7f7911661640 1 --2- 172.21.15.107:0/3629683613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f790c1032e0 0x7f790c198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.331+0000 7f7911661640 1 --2- 172.21.15.107:0/3629683613 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f78f40a2e20 0x7f78f40a52e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.331+0000 7f7911661640 1 --2- 172.21.15.107:0/3629683613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f790c1029c0 0x7f790c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.331+0000 7f7911661640 1 -- 172.21.15.107:0/3629683613 >> 172.21.15.107:0/3629683613 conn(0x7f790c077740 msgr2=0x7f790c0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:34.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.331+0000 7f7911661640 1 -- 172.21.15.107:0/3629683613 shutdown_connections 2024-04-25T00:16:34.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.331+0000 7f7911661640 1 -- 172.21.15.107:0/3629683613 wait complete. 2024-04-25T00:16:34.363 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:16:34.363 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:16:34.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.464+0000 7fe04c6f5640 1 -- 172.21.15.107:0/1085317256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe044100610 msgr2=0x7fe044100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.464+0000 7fe04c6f5640 1 --2- 172.21.15.107:0/1085317256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe044100610 0x7fe044100a90 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fe034009930 tx=0x7fe03402a420 comp rx=0 tx=0).stop 2024-04-25T00:16:34.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.464+0000 7fe04c6f5640 1 -- 172.21.15.107:0/1085317256 shutdown_connections 2024-04-25T00:16:34.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.464+0000 7fe04c6f5640 1 --2- 172.21.15.107:0/1085317256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe044100fd0 0x7fe04410d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.464+0000 7fe04c6f5640 1 --2- 172.21.15.107:0/1085317256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe044100610 0x7fe044100a90 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.464+0000 7fe04c6f5640 1 --2- 172.21.15.107:0/1085317256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0441066b0 0x7fe044106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.464+0000 7fe04c6f5640 1 -- 172.21.15.107:0/1085317256 >> 172.21.15.107:0/1085317256 conn(0x7fe044077740 msgr2=0x7fe044077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:34.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.464+0000 7fe04c6f5640 1 -- 172.21.15.107:0/1085317256 shutdown_connections 2024-04-25T00:16:34.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.464+0000 7fe04c6f5640 1 -- 172.21.15.107:0/1085317256 wait complete. 2024-04-25T00:16:34.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.465+0000 7fe04c6f5640 1 Processor -- start 2024-04-25T00:16:34.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.465+0000 7fe04c6f5640 1 -- start start 2024-04-25T00:16:34.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.465+0000 7fe04c6f5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe044100610 0x7fe044198d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.465+0000 7fe04c6f5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe044100fd0 0x7fe0441992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.465+0000 7fe04c6f5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0441066b0 0x7fe0441a3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.465+0000 7fe04c6f5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe0441109f0 con 0x7fe044100fd0 2024-04-25T00:16:34.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.465+0000 7fe04c6f5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe044110870 con 0x7fe0441066b0 2024-04-25T00:16:34.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.465+0000 7fe04c6f5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe044110b70 con 0x7fe044100610 2024-04-25T00:16:34.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.465+0000 7fe049c90640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe044100fd0 0x7fe0441992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.465+0000 7fe049c90640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe044100fd0 0x7fe0441992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39270/0 (socket says 172.21.15.107:39270) 2024-04-25T00:16:34.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.465+0000 7fe049c90640 1 -- 172.21.15.107:0/269901656 learned_addr learned my addr 172.21.15.107:0/269901656 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:34.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.465+0000 7fe04a491640 1 --2- 172.21.15.107:0/269901656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe044100610 0x7fe044198d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.465+0000 7fe04ac92640 1 --2- 172.21.15.107:0/269901656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0441066b0 0x7fe0441a3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.465+0000 7fe049c90640 1 -- 172.21.15.107:0/269901656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe044100610 msgr2=0x7fe044198d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.465+0000 7fe049c90640 1 --2- 172.21.15.107:0/269901656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe044100610 0x7fe044198d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.465+0000 7fe049c90640 1 -- 172.21.15.107:0/269901656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0441066b0 msgr2=0x7fe0441a3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.465+0000 7fe049c90640 1 --2- 172.21.15.107:0/269901656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0441066b0 0x7fe0441a3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.465+0000 7fe049c90640 1 -- 172.21.15.107:0/269901656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0441a4350 con 0x7fe044100fd0 2024-04-25T00:16:34.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.465+0000 7fe04ac92640 1 --2- 172.21.15.107:0/269901656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0441066b0 0x7fe0441a3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:34.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.465+0000 7fe049c90640 1 --2- 172.21.15.107:0/269901656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe044100fd0 0x7fe0441992d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe034009230 tx=0x7fe034004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:34.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.466+0000 7fe03b7fe640 1 -- 172.21.15.107:0/269901656 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03403f030 con 0x7fe044100fd0 2024-04-25T00:16:34.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.466+0000 7fe04c6f5640 1 -- 172.21.15.107:0/269901656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0441a4520 con 0x7fe044100fd0 2024-04-25T00:16:34.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.466+0000 7fe03b7fe640 1 -- 172.21.15.107:0/269901656 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe034003ad0 con 0x7fe044100fd0 2024-04-25T00:16:34.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.466+0000 7fe03b7fe640 1 -- 172.21.15.107:0/269901656 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe034049060 con 0x7fe044100fd0 2024-04-25T00:16:34.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.466+0000 7fe04c6f5640 1 -- 172.21.15.107:0/269901656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0441a49e0 con 0x7fe044100fd0 2024-04-25T00:16:34.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.468+0000 7fe03b7fe640 1 -- 172.21.15.107:0/269901656 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7fe034042030 con 0x7fe044100fd0 2024-04-25T00:16:34.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.468+0000 7fe04f402740 1 Processor -- start 2024-04-25T00:16:34.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.468+0000 7fe04f402740 1 -- start start 2024-04-25T00:16:34.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.468+0000 7fe04f402740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0a3530 0x55bd8a0a3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.468+0000 7fe04f402740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd8a0a3e50 0x55bd8a0b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.468+0000 7fe04f402740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd8a0b2a30 0x55bd8a0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.468+0000 7fe04f402740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bd8a0086b0 con 0x55bd8a0a3530 2024-04-25T00:16:34.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.468+0000 7fe04f402740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bd89fb0880 con 0x55bd8a0b2a30 2024-04-25T00:16:34.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.468+0000 7fe04f402740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bd8a004670 con 0x55bd8a0a3e50 2024-04-25T00:16:34.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.469+0000 7fe04ac92640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0a3530 0x55bd8a0a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.469+0000 7fe04ac92640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0a3530 0x55bd8a0a3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39284/0 (socket says 172.21.15.107:39284) 2024-04-25T00:16:34.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.469+0000 7fe04ac92640 1 -- 172.21.15.107:0/3005237329 learned_addr learned my addr 172.21.15.107:0/3005237329 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:34.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.469+0000 7fe03b7fe640 1 --2- 172.21.15.107:0/269901656 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe0300a2e20 0x7fe0300a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.469+0000 7fe04a491640 1 --2- 172.21.15.107:0/3005237329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd8a0a3e50 0x55bd8a0b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.469+0000 7fe049c90640 1 --2- 172.21.15.107:0/3005237329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd8a0b2a30 0x55bd8a0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.469+0000 7fe04ac92640 1 --2- 172.21.15.107:0/269901656 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe0300a2e20 0x7fe0300a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.469+0000 7fe04ac92640 1 -- 172.21.15.107:0/3005237329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd8a0a3e50 msgr2=0x55bd8a0b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:34.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.469+0000 7fe04ac92640 1 --2- 172.21.15.107:0/3005237329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd8a0a3e50 0x55bd8a0b24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.469+0000 7fe04ac92640 1 -- 172.21.15.107:0/3005237329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd8a0b2a30 msgr2=0x55bd8a0b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:34.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.469+0000 7fe04ac92640 1 --2- 172.21.15.107:0/3005237329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd8a0b2a30 0x55bd8a0b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.469+0000 7fe04ac92640 1 -- 172.21.15.107:0/3005237329 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd89de67d0 con 0x55bd8a0a3530 2024-04-25T00:16:34.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.469+0000 7fe03b7fe640 1 -- 172.21.15.107:0/269901656 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fe0340f0240 con 0x7fe044100fd0 2024-04-25T00:16:34.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.469+0000 7fe049c90640 1 --2- 172.21.15.107:0/3005237329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd8a0b2a30 0x55bd8a0b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:34.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.469+0000 7fe04ac92640 1 --2- 172.21.15.107:0/3005237329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0a3530 0x55bd8a0a3910 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe03c0129e0 tx=0x7fe03c012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761a5dec21b0ab9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:34.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.469+0000 7fe038ff9640 1 -- 172.21.15.107:0/3005237329 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c0050e0 con 0x55bd8a0a3530 2024-04-25T00:16:34.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.469+0000 7fe038ff9640 1 -- 172.21.15.107:0/3005237329 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe03c005760 con 0x55bd8a0a3530 2024-04-25T00:16:34.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.469+0000 7fe038ff9640 1 -- 172.21.15.107:0/3005237329 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c01c930 con 0x55bd8a0a3530 2024-04-25T00:16:34.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.469+0000 7fe04ac92640 1 --2- 172.21.15.107:0/269901656 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe0300a2e20 0x7fe0300a52e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe03c01f000 tx=0x7fe03c01e000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:34.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.470+0000 7fe04f402740 1 -- 172.21.15.107:0/3005237329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0a3530 msgr2=0x55bd8a0a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.470+0000 7fe04f402740 1 --2- 172.21.15.107:0/3005237329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0a3530 0x55bd8a0a3910 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe03c0129e0 tx=0x7fe03c012ea0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.470+0000 7fe04f402740 1 -- 172.21.15.107:0/3005237329 shutdown_connections 2024-04-25T00:16:34.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.470+0000 7fe04f402740 1 --2- 172.21.15.107:0/3005237329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd8a0b2a30 0x55bd8a0b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.470+0000 7fe04f402740 1 --2- 172.21.15.107:0/3005237329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd8a0a3e50 0x55bd8a0b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.470+0000 7fe04f402740 1 --2- 172.21.15.107:0/3005237329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0a3530 0x55bd8a0a3910 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.470+0000 7fe04f402740 1 -- 172.21.15.107:0/3005237329 >> 172.21.15.107:0/3005237329 conn(0x55bd8a09f950 msgr2=0x55bd8a09fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:34.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.470+0000 7fe04f402740 1 -- 172.21.15.107:0/3005237329 shutdown_connections 2024-04-25T00:16:34.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.470+0000 7fe04f402740 1 -- 172.21.15.107:0/3005237329 wait complete. 2024-04-25T00:16:34.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.470+0000 7fe04f402740 1 -- start start 2024-04-25T00:16:34.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.471+0000 7fe04f402740 1 Processor -- start 2024-04-25T00:16:34.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.471+0000 7fe04f402740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd8a0a3530 0x55bd8a0af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.471+0000 7fe04f402740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd8a0a3e50 0x55bd8a0afe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.471+0000 7fe04f402740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0b2a30 0x55bd8a0b0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.471+0000 7fe04f402740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bd8a0b57f0 con 0x55bd8a0b2a30 2024-04-25T00:16:34.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.471+0000 7fe04f402740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bd89fc3f50 con 0x55bd8a0a3e50 2024-04-25T00:16:34.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.471+0000 7fe04f402740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bd8a0b5970 con 0x55bd8a0a3530 2024-04-25T00:16:34.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.471+0000 7fe04f402740 10 client.0 _ll_register_callbacks cb 0x55bd89fff000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:16:34.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.471+0000 7fe04ac92640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0b2a30 0x55bd8a0b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.471+0000 7fe04ac92640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0b2a30 0x55bd8a0b0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39288/0 (socket says 172.21.15.107:39288) 2024-04-25T00:16:34.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.471+0000 7fe04ac92640 1 -- 172.21.15.107:0/2020684597 learned_addr learned my addr 172.21.15.107:0/2020684597 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:34.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.471+0000 7fe049c90640 1 --2- 172.21.15.107:0/2020684597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd8a0a3e50 0x55bd8a0afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.471+0000 7fe04a491640 1 --2- 172.21.15.107:0/2020684597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd8a0a3530 0x55bd8a0af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.471+0000 7fe04ac92640 1 -- 172.21.15.107:0/2020684597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd8a0a3530 msgr2=0x55bd8a0af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:34.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.471+0000 7fe04ac92640 1 --2- 172.21.15.107:0/2020684597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd8a0a3530 0x55bd8a0af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.471+0000 7fe04ac92640 1 -- 172.21.15.107:0/2020684597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd8a0a3e50 msgr2=0x55bd8a0afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:34.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.471+0000 7fe04ac92640 1 --2- 172.21.15.107:0/2020684597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd8a0a3e50 0x55bd8a0afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.471+0000 7fe04ac92640 1 -- 172.21.15.107:0/2020684597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd89d1c3f0 con 0x55bd8a0b2a30 2024-04-25T00:16:34.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.471+0000 7fe04a491640 1 --2- 172.21.15.107:0/2020684597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd8a0a3530 0x55bd8a0af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:34.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.471+0000 7fe04ac92640 1 --2- 172.21.15.107:0/2020684597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0b2a30 0x55bd8a0b0390 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe03c005b40 tx=0x7fe03c005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0992f580fa8710 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:34.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.471+0000 7fe02dffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:16:34.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.471+0000 7fe02dffb640 1 -- 172.21.15.107:0/2020684597 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c035060 con 0x55bd8a0b2a30 2024-04-25T00:16:34.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.471+0000 7fe04f402740 1 -- 172.21.15.107:0/2020684597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe03c005760 con 0x55bd8a0b2a30 2024-04-25T00:16:34.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.471+0000 7fe02dffb640 1 -- 172.21.15.107:0/2020684597 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fe03c021040 con 0x55bd8a0b2a30 2024-04-25T00:16:34.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.472+0000 7fe02dffb640 1 -- 172.21.15.107:0/2020684597 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe03c03c050 con 0x55bd8a0b2a30 2024-04-25T00:16:34.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.472+0000 7fe04f402740 10 client.15693 fetch_fsmap learned FSMap version 24 2024-04-25T00:16:34.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.472+0000 7fe04f402740 1 -- 172.21.15.107:0/2020684597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bd89cdc0f0 con 0x55bd8a0b2a30 2024-04-25T00:16:34.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.472+0000 7fe02dffb640 1 -- 172.21.15.107:0/2020684597 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe03c005760 con 0x55bd8a0b2a30 2024-04-25T00:16:34.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.472+0000 7fe02dffb640 1 -- 172.21.15.107:0/2020684597 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fe03c025490 con 0x55bd8a0b2a30 2024-04-25T00:16:34.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.472+0000 7fe04f402740 10 client.15693 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:16:34.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.472+0000 7fe04f402740 10 client.15693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:16:34.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.472+0000 7fe04f402740 20 client.15693 populate_metadata read hostname 'smithi107' 2024-04-25T00:16:34.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.472+0000 7fe04f402740 1 --2- 172.21.15.107:0/2020684597 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55bd8a0c2c90 0x55bd8a0c5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.472+0000 7fe04f402740 4 client.15693 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:16:34.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.472+0000 7fe04f402740 1 -- 172.21.15.107:0/2020684597 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd8a015bf0 con 0x55bd8a0c2c90 2024-04-25T00:16:34.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.472+0000 7fe04a491640 1 --2- 172.21.15.107:0/2020684597 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55bd8a0c2c90 0x55bd8a0c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.473+0000 7fe02dffb640 10 client.15693 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:16:34.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.473+0000 7fe04a491640 1 --2- 172.21.15.107:0/2020684597 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55bd8a0c2c90 0x55bd8a0c5080 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=986598838a6bc860 server_cookie=11ca8098c457cb21 in_seq=0 out_seq=0 2024-04-25T00:16:34.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.473+0000 7fe02dffb640 1 -- 172.21.15.107:0/2020684597 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bd8a015bf0 con 0x55bd8a0c2c90 2024-04-25T00:16:34.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.473+0000 7fe02dffb640 10 client.15693 handle_command_reply: tid=0 2024-04-25T00:16:34.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.473+0000 7fe04f402740 1 client.15693 shutdown 2024-04-25T00:16:34.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.473+0000 7fe04f402740 1 -- 172.21.15.107:0/2020684597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0b2a30 msgr2=0x55bd8a0b0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.473+0000 7fe04f402740 1 --2- 172.21.15.107:0/2020684597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0b2a30 0x55bd8a0b0390 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe03c005b40 tx=0x7fe03c005b70 comp rx=0 tx=0).stop 2024-04-25T00:16:34.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.473+0000 7fe04f402740 1 -- 172.21.15.107:0/2020684597 shutdown_connections 2024-04-25T00:16:34.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.473+0000 7fe04f402740 1 --2- 172.21.15.107:0/2020684597 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55bd8a0c2c90 0x55bd8a0c5080 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.473+0000 7fe04f402740 1 --2- 172.21.15.107:0/2020684597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0b2a30 0x55bd8a0b0390 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.473+0000 7fe04f402740 1 --2- 172.21.15.107:0/2020684597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd8a0a3e50 0x55bd8a0afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.473+0000 7fe02dffb640 0 client.15693 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:16:34.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.473+0000 7fe04f402740 1 --2- 172.21.15.107:0/2020684597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd8a0a3530 0x55bd8a0af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.473+0000 7fe04f402740 1 -- 172.21.15.107:0/2020684597 >> 172.21.15.107:0/2020684597 conn(0x55bd8a09f950 msgr2=0x55bd8a0a56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:34.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.473+0000 7fe04f402740 1 -- 172.21.15.107:0/2020684597 shutdown_connections 2024-04-25T00:16:34.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.474+0000 7fe04f402740 1 -- 172.21.15.107:0/2020684597 wait complete. 2024-04-25T00:16:34.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.474+0000 7fe04f402740 20 client.15693 trim_cache size 0 max 16384 2024-04-25T00:16:34.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.491+0000 7fe04f402740 1 Processor -- start 2024-04-25T00:16:34.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.491+0000 7fe04f402740 1 -- start start 2024-04-25T00:16:34.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.491+0000 7fe04f402740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0a3530 0x55bd8a09f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.491+0000 7fe04f402740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd8a0a3e50 0x55bd8a0be720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.491+0000 7fe04f402740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd8a0b2a30 0x55bd8a0affe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.491+0000 7fe04f402740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe0441af010 con 0x55bd8a0a3530 2024-04-25T00:16:34.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.491+0000 7fe04f402740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe0441ac8e0 con 0x55bd8a0a3e50 2024-04-25T00:16:34.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.491+0000 7fe04f402740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe0441aa730 con 0x55bd8a0b2a30 2024-04-25T00:16:34.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.491+0000 7fe04a491640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0a3530 0x55bd8a09f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.491+0000 7fe04a491640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0a3530 0x55bd8a09f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39298/0 (socket says 172.21.15.107:39298) 2024-04-25T00:16:34.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.491+0000 7fe04a491640 1 -- 172.21.15.107:0/3086135744 learned_addr learned my addr 172.21.15.107:0/3086135744 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:34.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.492+0000 7fe04a491640 1 -- 172.21.15.107:0/3086135744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd8a0b2a30 msgr2=0x55bd8a0affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:34.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.492+0000 7fe049c90640 1 --2- 172.21.15.107:0/3086135744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd8a0a3e50 0x55bd8a0be720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.492+0000 7fe04a491640 1 --2- 172.21.15.107:0/3086135744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd8a0b2a30 0x55bd8a0affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.492+0000 7fe04a491640 1 -- 172.21.15.107:0/3086135744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd8a0a3e50 msgr2=0x55bd8a0be720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:34.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.492+0000 7fe04a491640 1 --2- 172.21.15.107:0/3086135744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd8a0a3e50 0x55bd8a0be720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.492+0000 7fe04a491640 1 -- 172.21.15.107:0/3086135744 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd8a091090 con 0x55bd8a0a3530 2024-04-25T00:16:34.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.492+0000 7fe049c90640 1 --2- 172.21.15.107:0/3086135744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd8a0a3e50 0x55bd8a0be720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:34.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.492+0000 7fe04a491640 1 --2- 172.21.15.107:0/3086135744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0a3530 0x55bd8a09f270 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55bd8a0c5650 tx=0x7fe04000bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bca820f8919e1ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:34.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.492+0000 7fe02ffff640 1 -- 172.21.15.107:0/3086135744 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe040021030 con 0x55bd8a0a3530 2024-04-25T00:16:34.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.492+0000 7fe02ffff640 1 -- 172.21.15.107:0/3086135744 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe04001d900 con 0x55bd8a0a3530 2024-04-25T00:16:34.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.492+0000 7fe02ffff640 1 -- 172.21.15.107:0/3086135744 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0400047a0 con 0x55bd8a0a3530 2024-04-25T00:16:34.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.493+0000 7fe04f402740 1 -- 172.21.15.107:0/3086135744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0a3530 msgr2=0x55bd8a09f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.493+0000 7fe04f402740 1 --2- 172.21.15.107:0/3086135744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0a3530 0x55bd8a09f270 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55bd8a0c5650 tx=0x7fe04000bdf0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.493+0000 7fe04f402740 1 -- 172.21.15.107:0/3086135744 shutdown_connections 2024-04-25T00:16:34.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.493+0000 7fe04f402740 1 --2- 172.21.15.107:0/3086135744 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd8a0b2a30 0x55bd8a0affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.493+0000 7fe04f402740 1 --2- 172.21.15.107:0/3086135744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd8a0a3e50 0x55bd8a0be720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.493+0000 7fe04f402740 1 --2- 172.21.15.107:0/3086135744 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0a3530 0x55bd8a09f270 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.493+0000 7fe04f402740 1 -- 172.21.15.107:0/3086135744 >> 172.21.15.107:0/3086135744 conn(0x55bd8a09f950 msgr2=0x55bd8a0af8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:34.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.493+0000 7fe04f402740 1 -- 172.21.15.107:0/3086135744 shutdown_connections 2024-04-25T00:16:34.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.493+0000 7fe04f402740 1 -- 172.21.15.107:0/3086135744 wait complete. 2024-04-25T00:16:34.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.493+0000 7fe04f402740 1 -- start start 2024-04-25T00:16:34.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.493+0000 7fe04f402740 1 Processor -- start 2024-04-25T00:16:34.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.494+0000 7fe04f402740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd8a0a3530 0x55bd8a0bb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.494+0000 7fe04f402740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0a3e50 0x55bd8a0ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.494+0000 7fe04f402740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd8a0b2a30 0x55bd8a0ac910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.494+0000 7fe04f402740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe0441aada0 con 0x55bd8a0a3e50 2024-04-25T00:16:34.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.494+0000 7fe04f402740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe0441ac210 con 0x55bd8a0a3530 2024-04-25T00:16:34.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.494+0000 7fe04f402740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe0441ab470 con 0x55bd8a0b2a30 2024-04-25T00:16:34.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.494+0000 7fe04f402740 10 client.0 _ll_register_callbacks cb 0x7fe04000b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:16:34.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.494+0000 7fe049c90640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0a3e50 0x55bd8a0ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.494+0000 7fe049c90640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0a3e50 0x55bd8a0ac3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39300/0 (socket says 172.21.15.107:39300) 2024-04-25T00:16:34.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.494+0000 7fe049c90640 1 -- 172.21.15.107:0/4018092324 learned_addr learned my addr 172.21.15.107:0/4018092324 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:34.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.494+0000 7fe04a491640 1 --2- 172.21.15.107:0/4018092324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd8a0a3530 0x55bd8a0bb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.494+0000 7fe04ac92640 1 --2- 172.21.15.107:0/4018092324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd8a0b2a30 0x55bd8a0ac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.494+0000 7fe049c90640 1 -- 172.21.15.107:0/4018092324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd8a0b2a30 msgr2=0x55bd8a0ac910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:34.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.494+0000 7fe049c90640 1 --2- 172.21.15.107:0/4018092324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd8a0b2a30 0x55bd8a0ac910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.494+0000 7fe049c90640 1 -- 172.21.15.107:0/4018092324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd8a0a3530 msgr2=0x55bd8a0bb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:16:34.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.494+0000 7fe049c90640 1 --2- 172.21.15.107:0/4018092324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd8a0a3530 0x55bd8a0bb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.494+0000 7fe049c90640 1 -- 172.21.15.107:0/4018092324 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd8a0ad140 con 0x55bd8a0a3e50 2024-04-25T00:16:34.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.494+0000 7fe049c90640 1 --2- 172.21.15.107:0/4018092324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0a3e50 0x55bd8a0ac3b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe0340ed5b0 tx=0x7fe03408c0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3e0e04c92011ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:34.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.494+0000 7fe02dffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:16:34.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.494+0000 7fe02dffb640 1 -- 172.21.15.107:0/4018092324 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe034035040 con 0x55bd8a0a3e50 2024-04-25T00:16:34.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.494+0000 7fe02dffb640 1 -- 172.21.15.107:0/4018092324 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fe034003c80 con 0x55bd8a0a3e50 2024-04-25T00:16:34.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.494+0000 7fe04f402740 1 -- 172.21.15.107:0/4018092324 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe04001d900 con 0x55bd8a0a3e50 2024-04-25T00:16:34.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.495+0000 7fe02dffb640 1 -- 172.21.15.107:0/4018092324 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fe0340c0620 con 0x55bd8a0a3e50 2024-04-25T00:16:34.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.495+0000 7fe04f402740 10 client.15699 fetch_fsmap learned FSMap version 24 2024-04-25T00:16:34.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.495+0000 7fe04f402740 1 -- 172.21.15.107:0/4018092324 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55bd8a128f60 con 0x55bd8a0a3e50 2024-04-25T00:16:34.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.495+0000 7fe02dffb640 1 -- 172.21.15.107:0/4018092324 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe04001d900 con 0x55bd8a0a3e50 2024-04-25T00:16:34.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.495+0000 7fe02dffb640 1 -- 172.21.15.107:0/4018092324 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fe034031bd0 con 0x55bd8a0a3e50 2024-04-25T00:16:34.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.495+0000 7fe04f402740 10 client.15699 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:16:34.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.495+0000 7fe04f402740 10 client.15699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:16:34.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.495+0000 7fe04f402740 20 client.15699 populate_metadata read hostname 'smithi107' 2024-04-25T00:16:34.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.495+0000 7fe04f402740 1 --2- 172.21.15.107:0/4018092324 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55bd8a1296d0 0x55bd8a12baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.495+0000 7fe04f402740 4 client.15699 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:16:34.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.495+0000 7fe04f402740 1 -- 172.21.15.107:0/4018092324 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bd8a00ff40 con 0x55bd8a1296d0 2024-04-25T00:16:34.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.495+0000 7fe04a491640 1 --2- 172.21.15.107:0/4018092324 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55bd8a1296d0 0x55bd8a12baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.496+0000 7fe04a491640 1 --2- 172.21.15.107:0/4018092324 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55bd8a1296d0 0x55bd8a12baf0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d64ec74c42ca909 server_cookie=b79f05c5f3eec6c6 in_seq=0 out_seq=0 2024-04-25T00:16:34.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.496+0000 7fe02dffb640 10 client.15699 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:16:34.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.496+0000 7fe02dffb640 1 -- 172.21.15.107:0/4018092324 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bd8a00ff40 con 0x55bd8a1296d0 2024-04-25T00:16:34.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.496+0000 7fe02dffb640 10 client.15699 handle_command_reply: tid=0 2024-04-25T00:16:34.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.496+0000 7fe04f402740 1 client.15699 shutdown 2024-04-25T00:16:34.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.496+0000 7fe04f402740 1 -- 172.21.15.107:0/4018092324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0a3e50 msgr2=0x55bd8a0ac3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.496+0000 7fe04f402740 1 --2- 172.21.15.107:0/4018092324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0a3e50 0x55bd8a0ac3b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe0340ed5b0 tx=0x7fe03408c0d0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.496+0000 7fe04f402740 1 -- 172.21.15.107:0/4018092324 shutdown_connections 2024-04-25T00:16:34.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.496+0000 7fe04f402740 1 --2- 172.21.15.107:0/4018092324 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55bd8a1296d0 0x55bd8a12baf0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.496+0000 7fe04f402740 1 --2- 172.21.15.107:0/4018092324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bd8a0b2a30 0x55bd8a0ac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.496+0000 7fe04f402740 1 --2- 172.21.15.107:0/4018092324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bd8a0a3e50 0x55bd8a0ac3b0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.496+0000 7fe02dffb640 0 client.15699 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:16:34.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.496+0000 7fe04f402740 1 --2- 172.21.15.107:0/4018092324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bd8a0a3530 0x55bd8a0bb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.496+0000 7fe04f402740 1 -- 172.21.15.107:0/4018092324 >> 172.21.15.107:0/4018092324 conn(0x55bd8a09f950 msgr2=0x55bd8a0b0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:34.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.496+0000 7fe04f402740 1 -- 172.21.15.107:0/4018092324 shutdown_connections 2024-04-25T00:16:34.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.496+0000 7fe04f402740 1 -- 172.21.15.107:0/4018092324 wait complete. 2024-04-25T00:16:34.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.496+0000 7fe04f402740 20 client.15699 trim_cache size 0 max 16384 2024-04-25T00:16:34.519 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:16:34.519 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:16:34.519 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-04-25T00:16:34.519 INFO:teuthology.orchestra.run.smithi107.stdout: "549e423b-455f-47f2-b0e7-fd93cf70e437": { 2024-04-25T00:16:34.519 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-04-25T00:16:34.519 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "549e423b-455f-47f2-b0e7-fd93cf70e437", 2024-04-25T00:16:34.520 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-04-25T00:16:34.520 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:16:34.520 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:16:34.520 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:16:34.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.496+0000 7fe04c6f5640 1 -- 172.21.15.107:0/269901656 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe0300a2e20 msgr2=0x7fe0300a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.496+0000 7fe04c6f5640 1 --2- 172.21.15.107:0/269901656 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe0300a2e20 0x7fe0300a52e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe03c01f000 tx=0x7fe03c01e000 comp rx=0 tx=0).stop 2024-04-25T00:16:34.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.496+0000 7fe04c6f5640 1 -- 172.21.15.107:0/269901656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe044100fd0 msgr2=0x7fe0441992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.496+0000 7fe04c6f5640 1 --2- 172.21.15.107:0/269901656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe044100fd0 0x7fe0441992d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe034009230 tx=0x7fe034004270 comp rx=0 tx=0).stop 2024-04-25T00:16:34.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.497+0000 7fe04c6f5640 1 -- 172.21.15.107:0/269901656 shutdown_connections 2024-04-25T00:16:34.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.497+0000 7fe04c6f5640 1 --2- 172.21.15.107:0/269901656 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe0300a2e20 0x7fe0300a52e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.497+0000 7fe04c6f5640 1 --2- 172.21.15.107:0/269901656 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0441066b0 0x7fe0441a3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.497+0000 7fe04c6f5640 1 --2- 172.21.15.107:0/269901656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe044100fd0 0x7fe0441992d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.497+0000 7fe04c6f5640 1 --2- 172.21.15.107:0/269901656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe044100610 0x7fe044198d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.497+0000 7fe04c6f5640 1 -- 172.21.15.107:0/269901656 >> 172.21.15.107:0/269901656 conn(0x7fe044077740 msgr2=0x7fe0440ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:34.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.497+0000 7fe04c6f5640 1 -- 172.21.15.107:0/269901656 shutdown_connections 2024-04-25T00:16:34.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.497+0000 7fe04c6f5640 1 -- 172.21.15.107:0/269901656 wait complete. 2024-04-25T00:16:34.522 INFO:tasks.cephfs.filesystem:scrub status for tag:549e423b-455f-47f2-b0e7-fd93cf70e437 - {'path': '/', 'tag': '549e423b-455f-47f2-b0e7-fd93cf70e437', 'options': 'recursive,force'} 2024-04-25T00:16:34.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.516+0000 7f24be0a5640 1 -- 172.21.15.107:0/3052808682 --> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f24b80fdd80 con 0x7f24a00a2f40 2024-04-25T00:16:34.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.579+0000 7f24b4ff9640 1 -- 172.21.15.107:0/3052808682 <== mgr.15456 v2:172.21.15.107:6800/1021104172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f24b80fdd80 con 0x7f24a00a2f40 2024-04-25T00:16:34.580 INFO:teuthology.orchestra.run.smithi107.stdout:Schedule set for path / 2024-04-25T00:16:34.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.581+0000 7f24be0a5640 1 -- 172.21.15.107:0/3052808682 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f24a00a2f40 msgr2=0x7f24a00a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.581+0000 7f24be0a5640 1 --2- 172.21.15.107:0/3052808682 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f24a00a2f40 0x7f24a00a5400 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f24a8009fc0 tx=0x7f24a8005f90 comp rx=0 tx=0).stop 2024-04-25T00:16:34.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.581+0000 7f24be0a5640 1 -- 172.21.15.107:0/3052808682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24b81032f0 msgr2=0x7f24b8198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.581+0000 7f24be0a5640 1 --2- 172.21.15.107:0/3052808682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24b81032f0 0x7f24b8198430 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f24ac00c920 tx=0x7f24ac00cde0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.581+0000 7f24be0a5640 1 -- 172.21.15.107:0/3052808682 shutdown_connections 2024-04-25T00:16:34.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.581+0000 7f24be0a5640 1 --2- 172.21.15.107:0/3052808682 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24b810bd10 0x7f24b819c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.581+0000 7f24be0a5640 1 --2- 172.21.15.107:0/3052808682 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24b81032f0 0x7f24b8198430 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.581+0000 7f24be0a5640 1 --2- 172.21.15.107:0/3052808682 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f24a00a2f40 0x7f24a00a5400 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.581+0000 7f24be0a5640 1 --2- 172.21.15.107:0/3052808682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24b81029d0 0x7f24b8197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.581+0000 7f24be0a5640 1 -- 172.21.15.107:0/3052808682 >> 172.21.15.107:0/3052808682 conn(0x7f24b80777c0 msgr2=0x7f24b80fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:34.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.582+0000 7f24be0a5640 1 -- 172.21.15.107:0/3052808682 shutdown_connections 2024-04-25T00:16:34.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.582+0000 7f24be0a5640 1 -- 172.21.15.107:0/3052808682 wait complete. 2024-04-25T00:16:34.592 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-25T00:16:34.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.693+0000 7fec16742640 1 -- 172.21.15.107:0/2175896130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec10106780 msgr2=0x7fec10106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.693+0000 7fec16742640 1 --2- 172.21.15.107:0/2175896130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec10106780 0x7fec10106b60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7febf8009900 tx=0x7febf802a410 comp rx=0 tx=0).stop 2024-04-25T00:16:34.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.693+0000 7fec16742640 1 -- 172.21.15.107:0/2175896130 shutdown_connections 2024-04-25T00:16:34.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.693+0000 7fec16742640 1 --2- 172.21.15.107:0/2175896130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec10101330 0x7fec1010d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.693+0000 7fec16742640 1 --2- 172.21.15.107:0/2175896130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec10100990 0x7fec10100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.693+0000 7fec16742640 1 --2- 172.21.15.107:0/2175896130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec10106780 0x7fec10106b60 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.693+0000 7fec16742640 1 -- 172.21.15.107:0/2175896130 >> 172.21.15.107:0/2175896130 conn(0x7fec10076440 msgr2=0x7fec10076850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:34.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.693+0000 7fec16742640 1 -- 172.21.15.107:0/2175896130 shutdown_connections 2024-04-25T00:16:34.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.693+0000 7fec16742640 1 -- 172.21.15.107:0/2175896130 wait complete. 2024-04-25T00:16:34.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.693+0000 7fec16742640 1 Processor -- start 2024-04-25T00:16:34.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.693+0000 7fec16742640 1 -- start start 2024-04-25T00:16:34.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.694+0000 7fec16742640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec10100990 0x7fec10197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.694+0000 7fec16742640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec10101330 0x7fec10198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.694+0000 7fec16742640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec10106780 0x7fec1019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.694+0000 7fec16742640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fec10111b60 con 0x7fec10100990 2024-04-25T00:16:34.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.694+0000 7fec16742640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fec101119e0 con 0x7fec10106780 2024-04-25T00:16:34.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.694+0000 7fec16742640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fec10111ce0 con 0x7fec10101330 2024-04-25T00:16:34.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.694+0000 7fec0ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec10100990 0x7fec10197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.694+0000 7fec0ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec10100990 0x7fec10197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39308/0 (socket says 172.21.15.107:39308) 2024-04-25T00:16:34.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.694+0000 7fec0ffff640 1 -- 172.21.15.107:0/2276155201 learned_addr learned my addr 172.21.15.107:0/2276155201 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:34.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.694+0000 7fec0f7fe640 1 --2- 172.21.15.107:0/2276155201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec10101330 0x7fec10198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.694+0000 7fec14cdf640 1 --2- 172.21.15.107:0/2276155201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec10106780 0x7fec1019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.694+0000 7fec0ffff640 1 -- 172.21.15.107:0/2276155201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec10101330 msgr2=0x7fec10198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.694+0000 7fec0ffff640 1 --2- 172.21.15.107:0/2276155201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec10101330 0x7fec10198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.694+0000 7fec0ffff640 1 -- 172.21.15.107:0/2276155201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec10106780 msgr2=0x7fec1019c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.694+0000 7fec0ffff640 1 --2- 172.21.15.107:0/2276155201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec10106780 0x7fec1019c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.694+0000 7fec0ffff640 1 -- 172.21.15.107:0/2276155201 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1019cdf0 con 0x7fec10100990 2024-04-25T00:16:34.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.694+0000 7fec14cdf640 1 --2- 172.21.15.107:0/2276155201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec10106780 0x7fec1019c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:34.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.694+0000 7fec0ffff640 1 --2- 172.21.15.107:0/2276155201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec10100990 0x7fec10197e00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7febf8006010 tx=0x7febf80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:34.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.694+0000 7fec0d7fa640 1 -- 172.21.15.107:0/2276155201 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf803f030 con 0x7fec10100990 2024-04-25T00:16:34.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.694+0000 7fec0d7fa640 1 -- 172.21.15.107:0/2276155201 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febf8003ce0 con 0x7fec10100990 2024-04-25T00:16:34.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.694+0000 7fec0d7fa640 1 -- 172.21.15.107:0/2276155201 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf8035070 con 0x7fec10100990 2024-04-25T00:16:34.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.694+0000 7fec16742640 1 -- 172.21.15.107:0/2276155201 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec1019d080 con 0x7fec10100990 2024-04-25T00:16:34.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.694+0000 7fec16742640 1 -- 172.21.15.107:0/2276155201 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec101a4970 con 0x7fec10100990 2024-04-25T00:16:34.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.695+0000 7fec16742640 1 -- 172.21.15.107:0/2276155201 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec10101d40 con 0x7fec10100990 2024-04-25T00:16:34.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.696+0000 7fec0d7fa640 1 -- 172.21.15.107:0/2276155201 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7febf8042030 con 0x7fec10100990 2024-04-25T00:16:34.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.697+0000 7fec0d7fa640 1 --2- 172.21.15.107:0/2276155201 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7febfc0a2ec0 0x7febfc0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:34.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.697+0000 7fec0f7fe640 1 --2- 172.21.15.107:0/2276155201 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7febfc0a2ec0 0x7febfc0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:34.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.697+0000 7fec0d7fa640 1 -- 172.21.15.107:0/2276155201 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7febf80efe10 con 0x7fec10100990 2024-04-25T00:16:34.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.697+0000 7fec0f7fe640 1 --2- 172.21.15.107:0/2276155201 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7febfc0a2ec0 0x7febfc0a5380 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fec10105d00 tx=0x7fec0000a2d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:34.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.701+0000 7fec0d7fa640 1 -- 172.21.15.107:0/2276155201 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7febf80bc540 con 0x7fec10100990 2024-04-25T00:16:34.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.901+0000 7fec16742640 1 -- 172.21.15.107:0/2276155201 --> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fec100ffc40 con 0x7febfc0a2ec0 2024-04-25T00:16:34.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.963+0000 7fec0d7fa640 1 -- 172.21.15.107:0/2276155201 <== mgr.15456 v2:172.21.15.107:6800/1021104172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fec100ffc40 con 0x7febfc0a2ec0 2024-04-25T00:16:34.963 INFO:teuthology.orchestra.run.smithi107.stdout:Retention added to path / 2024-04-25T00:16:34.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.965+0000 7fec16742640 1 -- 172.21.15.107:0/2276155201 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7febfc0a2ec0 msgr2=0x7febfc0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.965+0000 7fec16742640 1 --2- 172.21.15.107:0/2276155201 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7febfc0a2ec0 0x7febfc0a5380 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fec10105d00 tx=0x7fec0000a2d0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.965+0000 7fec16742640 1 -- 172.21.15.107:0/2276155201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec10100990 msgr2=0x7fec10197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:34.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.965+0000 7fec16742640 1 --2- 172.21.15.107:0/2276155201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec10100990 0x7fec10197e00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7febf8006010 tx=0x7febf80042f0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.965+0000 7fec16742640 1 -- 172.21.15.107:0/2276155201 shutdown_connections 2024-04-25T00:16:34.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.965+0000 7fec16742640 1 --2- 172.21.15.107:0/2276155201 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7febfc0a2ec0 0x7febfc0a5380 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.965+0000 7fec16742640 1 --2- 172.21.15.107:0/2276155201 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec10106780 0x7fec1019c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.965+0000 7fec16742640 1 --2- 172.21.15.107:0/2276155201 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fec10101330 0x7fec10198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.965+0000 7fec16742640 1 --2- 172.21.15.107:0/2276155201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec10100990 0x7fec10197e00 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:34.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.965+0000 7fec16742640 1 -- 172.21.15.107:0/2276155201 >> 172.21.15.107:0/2276155201 conn(0x7fec10076440 msgr2=0x7fec100ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:34.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.965+0000 7fec16742640 1 -- 172.21.15.107:0/2276155201 shutdown_connections 2024-04-25T00:16:34.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:34.965+0000 7fec16742640 1 -- 172.21.15.107:0/2276155201 wait complete. 2024-04-25T00:16:34.976 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T00:16:35.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.074+0000 7fa0adb42640 1 -- 172.21.15.107:0/2774552807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0a8100660 msgr2=0x7fa0a8100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:35.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.074+0000 7fa0adb42640 1 --2- 172.21.15.107:0/2774552807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0a8100660 0x7fa0a8100ae0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009930 tx=0x7fa09c02a420 comp rx=0 tx=0).stop 2024-04-25T00:16:35.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.075+0000 7fa0adb42640 1 -- 172.21.15.107:0/2774552807 shutdown_connections 2024-04-25T00:16:35.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.075+0000 7fa0adb42640 1 --2- 172.21.15.107:0/2774552807 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0a8101020 0x7fa0a810d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:35.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.075+0000 7fa0adb42640 1 --2- 172.21.15.107:0/2774552807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0a8100660 0x7fa0a8100ae0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:35.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.075+0000 7fa0adb42640 1 --2- 172.21.15.107:0/2774552807 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0a8106670 0x7fa0a8106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:35.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.075+0000 7fa0adb42640 1 -- 172.21.15.107:0/2774552807 >> 172.21.15.107:0/2774552807 conn(0x7fa0a80769c0 msgr2=0x7fa0a8076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:35.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.075+0000 7fa0adb42640 1 -- 172.21.15.107:0/2774552807 shutdown_connections 2024-04-25T00:16:35.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.075+0000 7fa0adb42640 1 -- 172.21.15.107:0/2774552807 wait complete. 2024-04-25T00:16:35.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.075+0000 7fa0adb42640 1 Processor -- start 2024-04-25T00:16:35.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.075+0000 7fa0adb42640 1 -- start start 2024-04-25T00:16:35.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.075+0000 7fa0adb42640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0a8100660 0x7fa0a8198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:35.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.075+0000 7fa0adb42640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0a8101020 0x7fa0a81991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:35.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.075+0000 7fa0adb42640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0a8106670 0x7fa0a81a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:35.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.076+0000 7fa0adb42640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a8110a40 con 0x7fa0a8100660 2024-04-25T00:16:35.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.076+0000 7fa0adb42640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a81108c0 con 0x7fa0a8101020 2024-04-25T00:16:35.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.076+0000 7fa0adb42640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a8110bc0 con 0x7fa0a8106670 2024-04-25T00:16:35.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.076+0000 7fa0a77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0a8100660 0x7fa0a8198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:35.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.076+0000 7fa0a77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0a8100660 0x7fa0a8198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39330/0 (socket says 172.21.15.107:39330) 2024-04-25T00:16:35.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.076+0000 7fa0a77fe640 1 -- 172.21.15.107:0/1810255935 learned_addr learned my addr 172.21.15.107:0/1810255935 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:35.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.076+0000 7fa0a7fff640 1 --2- 172.21.15.107:0/1810255935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0a8106670 0x7fa0a81a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:35.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.076+0000 7fa0a77fe640 1 -- 172.21.15.107:0/1810255935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0a8106670 msgr2=0x7fa0a81a3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:35.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.076+0000 7fa0a77fe640 1 --2- 172.21.15.107:0/1810255935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0a8106670 0x7fa0a81a3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:35.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.076+0000 7fa0a6ffd640 1 --2- 172.21.15.107:0/1810255935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0a8101020 0x7fa0a81991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:35.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.076+0000 7fa0a77fe640 1 -- 172.21.15.107:0/1810255935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0a8101020 msgr2=0x7fa0a81991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:35.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.076+0000 7fa0a77fe640 1 --2- 172.21.15.107:0/1810255935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0a8101020 0x7fa0a81991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:35.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.076+0000 7fa0a77fe640 1 -- 172.21.15.107:0/1810255935 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a81a4260 con 0x7fa0a8100660 2024-04-25T00:16:35.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.076+0000 7fa0a6ffd640 1 --2- 172.21.15.107:0/1810255935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0a8101020 0x7fa0a81991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:35.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.076+0000 7fa0a77fe640 1 --2- 172.21.15.107:0/1810255935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0a8100660 0x7fa0a8198ca0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa09800c920 tx=0x7fa09800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:35.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.076+0000 7fa0a4ff9640 1 -- 172.21.15.107:0/1810255935 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098007c70 con 0x7fa0a8100660 2024-04-25T00:16:35.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.076+0000 7fa0adb42640 1 -- 172.21.15.107:0/1810255935 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a81a4430 con 0x7fa0a8100660 2024-04-25T00:16:35.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.076+0000 7fa0a4ff9640 1 -- 172.21.15.107:0/1810255935 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa098007e10 con 0x7fa0a8100660 2024-04-25T00:16:35.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.076+0000 7fa0a4ff9640 1 -- 172.21.15.107:0/1810255935 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098018890 con 0x7fa0a8100660 2024-04-25T00:16:35.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.076+0000 7fa0adb42640 1 -- 172.21.15.107:0/1810255935 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a81a49d0 con 0x7fa0a8100660 2024-04-25T00:16:35.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.077+0000 7fa0adb42640 1 -- 172.21.15.107:0/1810255935 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0a804f500 con 0x7fa0a8100660 2024-04-25T00:16:35.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.078+0000 7fa0a4ff9640 1 -- 172.21.15.107:0/1810255935 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7fa098018a70 con 0x7fa0a8100660 2024-04-25T00:16:35.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.079+0000 7fa0a4ff9640 1 --2- 172.21.15.107:0/1810255935 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa0900a2ec0 0x7fa0900a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:35.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.079+0000 7fa0a6ffd640 1 --2- 172.21.15.107:0/1810255935 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa0900a2ec0 0x7fa0900a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:35.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.079+0000 7fa0a4ff9640 1 -- 172.21.15.107:0/1810255935 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fa0980cfbf0 con 0x7fa0a8100660 2024-04-25T00:16:35.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.079+0000 7fa0a6ffd640 1 --2- 172.21.15.107:0/1810255935 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa0900a2ec0 0x7fa0900a5380 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa09c000c00 tx=0x7fa09c032d50 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:35.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.083+0000 7fa0a4ff9640 1 -- 172.21.15.107:0/1810255935 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa098014030 con 0x7fa0a8100660 2024-04-25T00:16:35.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:34 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/930702207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:35.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:34 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2401048682' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:35.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:34 smithi195 ceph-mon[32833]: scrub queued for path: / 2024-04-25T00:16:35.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:34 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:16:35.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:34 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:16:35.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:34 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:16:35.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:34 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/930702207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:35.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:34 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2401048682' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:35.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:34 smithi170 ceph-mon[32807]: scrub queued for path: / 2024-04-25T00:16:35.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:34 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:16:35.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:34 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:16:35.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:34 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:16:35.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:34 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/930702207' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:35.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:34 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2401048682' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:35.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:34 smithi107 ceph-mon[29153]: scrub queued for path: / 2024-04-25T00:16:35.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:34 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:16:35.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:34 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:16:35.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:34 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:16:35.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.278+0000 7fa0adb42640 1 -- 172.21.15.107:0/1810255935 --> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fa0a80fe9d0 con 0x7fa0900a2ec0 2024-04-25T00:16:35.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.286+0000 7fa0a4ff9640 1 -- 172.21.15.107:0/1810255935 <== mgr.15456 v2:172.21.15.107:6800/1021104172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fa0a80fe9d0 con 0x7fa0900a2ec0 2024-04-25T00:16:35.286 INFO:teuthology.orchestra.run.smithi107.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T00:16:34", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-25T00:16:35.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.288+0000 7fa0adb42640 1 -- 172.21.15.107:0/1810255935 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa0900a2ec0 msgr2=0x7fa0900a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:35.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.288+0000 7fa0adb42640 1 --2- 172.21.15.107:0/1810255935 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa0900a2ec0 0x7fa0900a5380 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa09c000c00 tx=0x7fa09c032d50 comp rx=0 tx=0).stop 2024-04-25T00:16:35.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.288+0000 7fa0adb42640 1 -- 172.21.15.107:0/1810255935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0a8100660 msgr2=0x7fa0a8198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:35.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.288+0000 7fa0adb42640 1 --2- 172.21.15.107:0/1810255935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0a8100660 0x7fa0a8198ca0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa09800c920 tx=0x7fa09800cde0 comp rx=0 tx=0).stop 2024-04-25T00:16:35.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.288+0000 7fa0adb42640 1 -- 172.21.15.107:0/1810255935 shutdown_connections 2024-04-25T00:16:35.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.288+0000 7fa0adb42640 1 --2- 172.21.15.107:0/1810255935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa0a8106670 0x7fa0a81a3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:35.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.288+0000 7fa0adb42640 1 --2- 172.21.15.107:0/1810255935 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa0900a2ec0 0x7fa0900a5380 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:35.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.288+0000 7fa0adb42640 1 --2- 172.21.15.107:0/1810255935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0a8101020 0x7fa0a81991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:35.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.288+0000 7fa0adb42640 1 --2- 172.21.15.107:0/1810255935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0a8100660 0x7fa0a8198ca0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:35.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.288+0000 7fa0adb42640 1 -- 172.21.15.107:0/1810255935 >> 172.21.15.107:0/1810255935 conn(0x7fa0a80769c0 msgr2=0x7fa0a80fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:35.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.288+0000 7fa0adb42640 1 -- 172.21.15.107:0/1810255935 shutdown_connections 2024-04-25T00:16:35.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.288+0000 7fa0adb42640 1 -- 172.21.15.107:0/1810255935 wait complete. 2024-04-25T00:16:35.299 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T00:16:35.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.399+0000 7f5379cba640 1 -- 172.21.15.107:0/999237700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5374101020 msgr2=0x7f537410d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:35.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.399+0000 7f5379cba640 1 --2- 172.21.15.107:0/999237700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5374101020 0x7f537410d530 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f536c009cd0 tx=0x7f536c02a8e0 comp rx=0 tx=0).stop 2024-04-25T00:16:35.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.399+0000 7f5379cba640 1 -- 172.21.15.107:0/999237700 shutdown_connections 2024-04-25T00:16:35.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.399+0000 7f5379cba640 1 --2- 172.21.15.107:0/999237700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5374101020 0x7f537410d530 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:35.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.399+0000 7f5379cba640 1 --2- 172.21.15.107:0/999237700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5374100660 0x7f5374100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:35.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.399+0000 7f5379cba640 1 --2- 172.21.15.107:0/999237700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5374106670 0x7f5374106a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:35.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.399+0000 7f5379cba640 1 -- 172.21.15.107:0/999237700 >> 172.21.15.107:0/999237700 conn(0x7f53740769c0 msgr2=0x7f5374076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:35.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.399+0000 7f5379cba640 1 -- 172.21.15.107:0/999237700 shutdown_connections 2024-04-25T00:16:35.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.399+0000 7f5379cba640 1 -- 172.21.15.107:0/999237700 wait complete. 2024-04-25T00:16:35.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.399+0000 7f5379cba640 1 Processor -- start 2024-04-25T00:16:35.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.400+0000 7f5379cba640 1 -- start start 2024-04-25T00:16:35.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.400+0000 7f5379cba640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5374100660 0x7f5374196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:35.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.400+0000 7f5379cba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5374101020 0x7f5374197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:35.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.400+0000 7f5379cba640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5374106670 0x7f537419b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:35.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.400+0000 7f5379cba640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5374110a80 con 0x7f5374101020 2024-04-25T00:16:35.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.400+0000 7f5379cba640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5374110900 con 0x7f5374100660 2024-04-25T00:16:35.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.400+0000 7f5379cba640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5374110c00 con 0x7f5374106670 2024-04-25T00:16:35.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.400+0000 7f536bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5374101020 0x7f5374197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:35.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.400+0000 7f536bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5374101020 0x7f5374197400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39348/0 (socket says 172.21.15.107:39348) 2024-04-25T00:16:35.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.400+0000 7f536bfff640 1 -- 172.21.15.107:0/3702888931 learned_addr learned my addr 172.21.15.107:0/3702888931 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:16:35.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.400+0000 7f5373fff640 1 --2- 172.21.15.107:0/3702888931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5374106670 0x7f537419b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:35.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.400+0000 7f53737fe640 1 --2- 172.21.15.107:0/3702888931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5374100660 0x7f5374196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:35.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.400+0000 7f536bfff640 1 -- 172.21.15.107:0/3702888931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5374106670 msgr2=0x7f537419b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:35.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.400+0000 7f536bfff640 1 --2- 172.21.15.107:0/3702888931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5374106670 0x7f537419b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:35.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.400+0000 7f536bfff640 1 -- 172.21.15.107:0/3702888931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5374100660 msgr2=0x7f5374196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:35.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.400+0000 7f536bfff640 1 --2- 172.21.15.107:0/3702888931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5374100660 0x7f5374196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:35.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.400+0000 7f536bfff640 1 -- 172.21.15.107:0/3702888931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537419beb0 con 0x7f5374101020 2024-04-25T00:16:35.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.400+0000 7f5373fff640 1 --2- 172.21.15.107:0/3702888931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5374106670 0x7f537419b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:35.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.400+0000 7f53737fe640 1 --2- 172.21.15.107:0/3702888931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5374100660 0x7f5374196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:16:35.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.400+0000 7f536bfff640 1 --2- 172.21.15.107:0/3702888931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5374101020 0x7f5374197400 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f535c00d770 tx=0x7f535c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:35.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.400+0000 7f53717fa640 1 -- 172.21.15.107:0/3702888931 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c017070 con 0x7f5374101020 2024-04-25T00:16:35.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.401+0000 7f53717fa640 1 -- 172.21.15.107:0/3702888931 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f535c005500 con 0x7f5374101020 2024-04-25T00:16:35.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.401+0000 7f5379cba640 1 -- 172.21.15.107:0/3702888931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537419c1a0 con 0x7f5374101020 2024-04-25T00:16:35.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.401+0000 7f53717fa640 1 -- 172.21.15.107:0/3702888931 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c004e00 con 0x7f5374101020 2024-04-25T00:16:35.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.401+0000 7f5379cba640 1 -- 172.21.15.107:0/3702888931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53741a3a30 con 0x7f5374101020 2024-04-25T00:16:35.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.401+0000 7f5379cba640 1 -- 172.21.15.107:0/3702888931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f537404f500 con 0x7f5374101020 2024-04-25T00:16:35.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.403+0000 7f53717fa640 1 -- 172.21.15.107:0/3702888931 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 23) ==== 138061+0+0 (secure 0 0 0) 0x7f535c004fa0 con 0x7f5374101020 2024-04-25T00:16:35.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.403+0000 7f53717fa640 1 --2- 172.21.15.107:0/3702888931 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f53600a2ec0 0x7f53600a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:35.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.403+0000 7f53737fe640 1 --2- 172.21.15.107:0/3702888931 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f53600a2ec0 0x7f53600a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:35.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.403+0000 7f53717fa640 1 -- 172.21.15.107:0/3702888931 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f535c0cf950 con 0x7f5374101020 2024-04-25T00:16:35.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.404+0000 7f53737fe640 1 --2- 172.21.15.107:0/3702888931 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f53600a2ec0 0x7f53600a5380 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f53640058c0 tx=0x7f5364005cb0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:35.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.407+0000 7f53717fa640 1 -- 172.21.15.107:0/3702888931 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f535c09bfb0 con 0x7f5374101020 2024-04-25T00:16:35.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.604+0000 7f5379cba640 1 -- 172.21.15.107:0/3702888931 --> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f5374108c20 con 0x7f53600a2ec0 2024-04-25T00:16:35.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.610+0000 7f53717fa640 1 -- 172.21.15.107:0/3702888931 <== mgr.15456 v2:172.21.15.107:6800/1021104172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f5374108c20 con 0x7f53600a2ec0 2024-04-25T00:16:35.610 INFO:teuthology.orchestra.run.smithi107.stdout:/ 1m 6m3h 2024-04-25T00:16:35.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.612+0000 7f5379cba640 1 -- 172.21.15.107:0/3702888931 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f53600a2ec0 msgr2=0x7f53600a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:35.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.612+0000 7f5379cba640 1 --2- 172.21.15.107:0/3702888931 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f53600a2ec0 0x7f53600a5380 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f53640058c0 tx=0x7f5364005cb0 comp rx=0 tx=0).stop 2024-04-25T00:16:35.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.612+0000 7f5379cba640 1 -- 172.21.15.107:0/3702888931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5374101020 msgr2=0x7f5374197400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:35.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.612+0000 7f5379cba640 1 --2- 172.21.15.107:0/3702888931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5374101020 0x7f5374197400 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f535c00d770 tx=0x7f535c00dc30 comp rx=0 tx=0).stop 2024-04-25T00:16:35.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.612+0000 7f5379cba640 1 -- 172.21.15.107:0/3702888931 shutdown_connections 2024-04-25T00:16:35.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.612+0000 7f5379cba640 1 --2- 172.21.15.107:0/3702888931 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5374106670 0x7f537419b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:35.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.612+0000 7f5379cba640 1 --2- 172.21.15.107:0/3702888931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5374101020 0x7f5374197400 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:35.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.612+0000 7f5379cba640 1 --2- 172.21.15.107:0/3702888931 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f53600a2ec0 0x7f53600a5380 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:35.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.612+0000 7f5379cba640 1 --2- 172.21.15.107:0/3702888931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5374100660 0x7f5374196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:35.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.612+0000 7f5379cba640 1 -- 172.21.15.107:0/3702888931 >> 172.21.15.107:0/3702888931 conn(0x7f53740769c0 msgr2=0x7f53740fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:35.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.612+0000 7f5379cba640 1 -- 172.21.15.107:0/3702888931 shutdown_connections 2024-04-25T00:16:35.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:16:35.612+0000 7f5379cba640 1 -- 172.21.15.107:0/3702888931 wait complete. 2024-04-25T00:16:35.623 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-25T00:16:35.647 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-25T00:16:35.659 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T00:16:35.682 INFO:tasks.workunit:Pulling workunits from ref d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:16:35.683 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T00:16:35.683 INFO:tasks.workunit:timeout=6h 2024-04-25T00:16:35.683 INFO:tasks.workunit:cleanup=True 2024-04-25T00:16:35.683 DEBUG:teuthology.orchestra.run.smithi107:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T00:16:35.703 INFO:teuthology.orchestra.run.smithi107.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T00:16:35.703 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-25T00:16:35.703 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-25T00:16:35.703 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:16:35.703 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-25T00:16:35.703 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-04-25 00:14:29.899838315 +0000 2024-04-25T00:16:35.703 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-04-25 00:14:29.899838315 +0000 2024-04-25T00:16:35.704 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-04-25 00:16:24.036961713 +0000 2024-04-25T00:16:35.704 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-04-25 00:14:29.899838315 +0000 2024-04-25T00:16:35.704 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T00:16:35.704 DEBUG:teuthology.orchestra.run.smithi107:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T00:16:35.775 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:16:35.831 INFO:tasks.workunit.client.0.smithi107.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T00:16:35.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:35 smithi107 ceph-mon[29153]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.0 KiB/s wr, 22 op/s 2024-04-25T00:16:35.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:35 smithi107 ceph-mon[29153]: from='client.15669 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:35.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:35 smithi107 ceph-mon[29153]: from='client.15705 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:36.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:35 smithi195 ceph-mon[32833]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.0 KiB/s wr, 22 op/s 2024-04-25T00:16:36.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:35 smithi195 ceph-mon[32833]: from='client.15669 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:36.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:35 smithi195 ceph-mon[32833]: from='client.15705 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:36.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:35 smithi170 ceph-mon[32807]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.0 KiB/s wr, 22 op/s 2024-04-25T00:16:36.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:35 smithi170 ceph-mon[32807]: from='client.15669 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:36.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:35 smithi170 ceph-mon[32807]: from='client.15705 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:37.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:36 smithi195 ceph-mon[32833]: from='client.15711 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:37.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:36 smithi195 ceph-mon[32833]: from='client.15717 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:37.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:36 smithi170 ceph-mon[32807]: from='client.15711 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:37.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:36 smithi170 ceph-mon[32807]: from='client.15717 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:37.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:36 smithi107 ceph-mon[29153]: from='client.15711 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:37.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:36 smithi107 ceph-mon[29153]: from='client.15717 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:38.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:37 smithi195 ceph-mon[32833]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 797 B/s wr, 17 op/s 2024-04-25T00:16:38.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:37 smithi170 ceph-mon[32807]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 797 B/s wr, 17 op/s 2024-04-25T00:16:38.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:37 smithi107 ceph-mon[29153]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 797 B/s wr, 17 op/s 2024-04-25T00:16:40.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:39 smithi195 ceph-mon[32833]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 85 KiB/s wr, 52 op/s 2024-04-25T00:16:40.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:39 smithi170 ceph-mon[32807]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 85 KiB/s wr, 52 op/s 2024-04-25T00:16:40.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:39 smithi107 ceph-mon[29153]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 85 KiB/s wr, 52 op/s 2024-04-25T00:16:42.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:41 smithi195 ceph-mon[32833]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-04-25T00:16:42.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:41 smithi170 ceph-mon[32807]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-04-25T00:16:42.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:41 smithi107 ceph-mon[29153]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-04-25T00:16:44.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:43 smithi195 ceph-mon[32833]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-04-25T00:16:44.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:43 smithi170 ceph-mon[32807]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-04-25T00:16:44.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:43 smithi107 ceph-mon[29153]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-04-25T00:16:46.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:45 smithi195 ceph-mon[32833]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-25T00:16:46.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:45 smithi170 ceph-mon[32807]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-25T00:16:46.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:45 smithi107 ceph-mon[29153]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-04-25T00:16:48.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:47 smithi195 ceph-mon[32833]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-25T00:16:48.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:47 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:16:48.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:47 smithi170 ceph-mon[32807]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-25T00:16:48.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:47 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:16:48.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:47 smithi107 ceph-mon[29153]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-04-25T00:16:48.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:47 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:16:50.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:49 smithi195 ceph-mon[32833]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-04-25T00:16:50.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:49 smithi170 ceph-mon[32807]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-04-25T00:16:50.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:49 smithi107 ceph-mon[29153]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-04-25T00:16:52.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:51 smithi170 ceph-mon[32807]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:16:52.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:51 smithi107 ceph-mon[29153]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:16:52.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:51 smithi195 ceph-mon[32833]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:16:54.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:53 smithi170 ceph-mon[32807]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T00:16:54.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:53 smithi107 ceph-mon[29153]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T00:16:54.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:53 smithi195 ceph-mon[32833]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T00:16:56.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:55 smithi170 ceph-mon[32807]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-25T00:16:56.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:55 smithi107 ceph-mon[29153]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-25T00:16:56.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:55 smithi195 ceph-mon[32833]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-04-25T00:16:58.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:57 smithi170 ceph-mon[32807]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:16:58.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:57 smithi107 ceph-mon[29153]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:16:58.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:57 smithi195 ceph-mon[32833]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:17:00.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:16:59 smithi170 ceph-mon[32807]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:17:00.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:16:59 smithi107 ceph-mon[29153]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:17:00.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:16:59 smithi195 ceph-mon[32833]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:17:01.234 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:17:00 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:17:00.974+0000 7f748032e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:17:01.234 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:17:00 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:17:00.974+0000 7f748032e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:17:01.234 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:17:00 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:17:00.974+0000 7f748032e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:17:01.234 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:17:00 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:17:00.974+0000 7f748032e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:17:01.234 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:17:00 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x[29382]: 2024-04-25T00:17:00.974+0000 7f748032e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:17:02.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:01 smithi170 ceph-mon[32807]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:17:02.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:01 smithi107 ceph-mon[29153]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:17:02.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:01 smithi195 ceph-mon[32833]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:17:04.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:03 smithi170 ceph-mon[32807]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-04-25T00:17:04.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:03 smithi170 ceph-mon[32807]: mgrmap e24: x(active, since 36s), standbys: z, y 2024-04-25T00:17:04.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:03 smithi107 ceph-mon[29153]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-04-25T00:17:04.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:03 smithi107 ceph-mon[29153]: mgrmap e24: x(active, since 36s), standbys: z, y 2024-04-25T00:17:04.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:03 smithi195 ceph-mon[32833]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-04-25T00:17:04.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:03 smithi195 ceph-mon[32833]: mgrmap e24: x(active, since 36s), standbys: z, y 2024-04-25T00:17:04.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:17:04.523 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:17:04.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.637+0000 7f38cb491640 1 -- 172.21.15.107:0/229680404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38c41020d0 msgr2=0x7f38c410e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:04.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.637+0000 7f38cb491640 1 --2- 172.21.15.107:0/229680404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38c41020d0 0x7f38c410e5e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009cd0 tx=0x7f38bc02a8e0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.637+0000 7f38cb491640 1 -- 172.21.15.107:0/229680404 shutdown_connections 2024-04-25T00:17:04.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.637+0000 7f38cb491640 1 --2- 172.21.15.107:0/229680404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38c41020d0 0x7f38c410e5e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.637+0000 7f38cb491640 1 --2- 172.21.15.107:0/229680404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38c4101710 0x7f38c4101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.637+0000 7f38cb491640 1 --2- 172.21.15.107:0/229680404 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38c4107720 0x7f38c4107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.637+0000 7f38cb491640 1 -- 172.21.15.107:0/229680404 >> 172.21.15.107:0/229680404 conn(0x7f38c40fd180 msgr2=0x7f38c40ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:04.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.638+0000 7f38cb491640 1 -- 172.21.15.107:0/229680404 shutdown_connections 2024-04-25T00:17:04.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.638+0000 7f38cb491640 1 -- 172.21.15.107:0/229680404 wait complete. 2024-04-25T00:17:04.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.638+0000 7f38cb491640 1 Processor -- start 2024-04-25T00:17:04.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.638+0000 7f38cb491640 1 -- start start 2024-04-25T00:17:04.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.638+0000 7f38cb491640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38c4101710 0x7f38c4197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.638+0000 7f38cb491640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38c41020d0 0x7f38c4198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.638+0000 7f38cb491640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38c4107720 0x7f38c419c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.638+0000 7f38cb491640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f38c4111af0 con 0x7f38c4101710 2024-04-25T00:17:04.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.638+0000 7f38cb491640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f38c4111970 con 0x7f38c4107720 2024-04-25T00:17:04.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.638+0000 7f38cb491640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f38c4111c70 con 0x7f38c41020d0 2024-04-25T00:17:04.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.639+0000 7f38c922d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38c4101710 0x7f38c4197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.639+0000 7f38c922d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38c4101710 0x7f38c4197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37726/0 (socket says 172.21.15.107:37726) 2024-04-25T00:17:04.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.639+0000 7f38c922d640 1 -- 172.21.15.107:0/2427059221 learned_addr learned my addr 172.21.15.107:0/2427059221 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:04.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.639+0000 7f38c8a2c640 1 --2- 172.21.15.107:0/2427059221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38c41020d0 0x7f38c4198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.639+0000 7f38c9a2e640 1 --2- 172.21.15.107:0/2427059221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38c4107720 0x7f38c419c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.639+0000 7f38c922d640 1 -- 172.21.15.107:0/2427059221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38c41020d0 msgr2=0x7f38c4198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:04.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.639+0000 7f38c922d640 1 --2- 172.21.15.107:0/2427059221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38c41020d0 0x7f38c4198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.639+0000 7f38c922d640 1 -- 172.21.15.107:0/2427059221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38c4107720 msgr2=0x7f38c419c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:04.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.639+0000 7f38c922d640 1 --2- 172.21.15.107:0/2427059221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38c4107720 0x7f38c419c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.639+0000 7f38c922d640 1 -- 172.21.15.107:0/2427059221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c419cfe0 con 0x7f38c4101710 2024-04-25T00:17:04.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.639+0000 7f38c9a2e640 1 --2- 172.21.15.107:0/2427059221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38c4107720 0x7f38c419c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:17:04.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.639+0000 7f38c922d640 1 --2- 172.21.15.107:0/2427059221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38c4101710 0x7f38c4197ff0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f38b400c890 tx=0x7f38b400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:04.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.639+0000 7f38ba7fc640 1 -- 172.21.15.107:0/2427059221 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b4004220 con 0x7f38c4101710 2024-04-25T00:17:04.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.639+0000 7f38ba7fc640 1 -- 172.21.15.107:0/2427059221 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38b40043c0 con 0x7f38c4101710 2024-04-25T00:17:04.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.639+0000 7f38ba7fc640 1 -- 172.21.15.107:0/2427059221 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b40049e0 con 0x7f38c4101710 2024-04-25T00:17:04.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.639+0000 7f38cb491640 1 -- 172.21.15.107:0/2427059221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c419d2d0 con 0x7f38c4101710 2024-04-25T00:17:04.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.639+0000 7f38cb491640 1 -- 172.21.15.107:0/2427059221 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c41a4b60 con 0x7f38c4101710 2024-04-25T00:17:04.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.641+0000 7f38ba7fc640 1 -- 172.21.15.107:0/2427059221 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f38b40079b0 con 0x7f38c4101710 2024-04-25T00:17:04.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.642+0000 7f38ce195740 1 Processor -- start 2024-04-25T00:17:04.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.642+0000 7f38ce195740 1 -- start start 2024-04-25T00:17:04.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.642+0000 7f38ce195740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0645e3630 0x55e0645e3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.642+0000 7f38ce195740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0645e3f50 0x55e0645ee0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.642+0000 7f38ce195740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645ee620 0x55e0645f0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.642+0000 7f38ce195740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e0645486b0 con 0x55e0645ee620 2024-04-25T00:17:04.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.642+0000 7f38ce195740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e0644f0880 con 0x55e0645e3f50 2024-04-25T00:17:04.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.642+0000 7f38ce195740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e064544670 con 0x55e0645e3630 2024-04-25T00:17:04.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.642+0000 7f38ba7fc640 1 --2- 172.21.15.107:0/2427059221 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f38b00a2f30 0x7f38b00a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.642+0000 7f38c8a2c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645ee620 0x55e0645f0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.643+0000 7f38c8a2c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645ee620 0x55e0645f0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37734/0 (socket says 172.21.15.107:37734) 2024-04-25T00:17:04.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.643+0000 7f38c8a2c640 1 -- 172.21.15.107:0/1457916589 learned_addr learned my addr 172.21.15.107:0/1457916589 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:04.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.643+0000 7f38c9a2e640 1 --2- 172.21.15.107:0/2427059221 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f38b00a2f30 0x7f38b00a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.643+0000 7f38c922d640 1 --2- 172.21.15.107:0/1457916589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0645e3f50 0x55e0645ee0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.643+0000 7f38c9a2e640 1 --2- 172.21.15.107:0/1457916589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0645e3630 0x55e0645e3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.643+0000 7f38ba7fc640 1 -- 172.21.15.107:0/2427059221 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f38b40cfab0 con 0x7f38c4101710 2024-04-25T00:17:04.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.643+0000 7f38c8a2c640 1 -- 172.21.15.107:0/1457916589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0645e3630 msgr2=0x55e0645e3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:04.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.643+0000 7f38c8a2c640 1 --2- 172.21.15.107:0/1457916589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0645e3630 0x55e0645e3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.643+0000 7f38c8a2c640 1 -- 172.21.15.107:0/1457916589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0645e3f50 msgr2=0x55e0645ee0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:04.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.643+0000 7f38c8a2c640 1 --2- 172.21.15.107:0/1457916589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0645e3f50 0x55e0645ee0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.643+0000 7f38c8a2c640 1 -- 172.21.15.107:0/1457916589 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0643267d0 con 0x55e0645ee620 2024-04-25T00:17:04.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.643+0000 7f38c9a2e640 1 --2- 172.21.15.107:0/1457916589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0645e3630 0x55e0645e3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:17:04.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.643+0000 7f38c922d640 1 --2- 172.21.15.107:0/1457916589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0645e3f50 0x55e0645ee0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:17:04.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.643+0000 7f38c9a2e640 1 --2- 172.21.15.107:0/2427059221 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f38b00a2f30 0x7f38b00a53f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009a60 tx=0x7f38bc005da0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:04.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.643+0000 7f38c8a2c640 1 --2- 172.21.15.107:0/1457916589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645ee620 0x55e0645f0a10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f38c000cbc0 tx=0x7f38c000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72c6a1b4d25d998 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:04.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.643+0000 7f38af7fe640 1 -- 172.21.15.107:0/1457916589 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c0010630 con 0x55e0645ee620 2024-04-25T00:17:04.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.643+0000 7f38af7fe640 1 -- 172.21.15.107:0/1457916589 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38c0010cb0 con 0x55e0645ee620 2024-04-25T00:17:04.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.643+0000 7f38af7fe640 1 -- 172.21.15.107:0/1457916589 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c001ac00 con 0x55e0645ee620 2024-04-25T00:17:04.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.644+0000 7f38ce195740 1 -- 172.21.15.107:0/1457916589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645ee620 msgr2=0x55e0645f0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:04.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.644+0000 7f38ce195740 1 --2- 172.21.15.107:0/1457916589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645ee620 0x55e0645f0a10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f38c000cbc0 tx=0x7f38c000e720 comp rx=0 tx=0).stop 2024-04-25T00:17:04.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.644+0000 7f38ce195740 1 -- 172.21.15.107:0/1457916589 shutdown_connections 2024-04-25T00:17:04.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.644+0000 7f38ce195740 1 --2- 172.21.15.107:0/1457916589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645ee620 0x55e0645f0a10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.644+0000 7f38ce195740 1 --2- 172.21.15.107:0/1457916589 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0645e3f50 0x55e0645ee0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.644+0000 7f38ce195740 1 --2- 172.21.15.107:0/1457916589 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0645e3630 0x55e0645e3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.644+0000 7f38ce195740 1 -- 172.21.15.107:0/1457916589 >> 172.21.15.107:0/1457916589 conn(0x55e0645df950 msgr2=0x55e0645dfd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:04.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.644+0000 7f38ce195740 1 -- 172.21.15.107:0/1457916589 shutdown_connections 2024-04-25T00:17:04.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.644+0000 7f38ce195740 1 -- 172.21.15.107:0/1457916589 wait complete. 2024-04-25T00:17:04.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.644+0000 7f38ce195740 1 -- start start 2024-04-25T00:17:04.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.644+0000 7f38ce195740 1 Processor -- start 2024-04-25T00:17:04.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.645+0000 7f38ce195740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0645e3630 0x55e0645eb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.645+0000 7f38ce195740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0645e3f50 0x55e0645eba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.645+0000 7f38ce195740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645ee620 0x55e0645ebf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.645+0000 7f38ce195740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e0645f1410 con 0x55e0645ee620 2024-04-25T00:17:04.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.645+0000 7f38ce195740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e064503f50 con 0x55e0645e3f50 2024-04-25T00:17:04.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.645+0000 7f38ce195740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e0645f1590 con 0x55e0645e3630 2024-04-25T00:17:04.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.645+0000 7f38ce195740 10 client.0 _ll_register_callbacks cb 0x55e06453f000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:04.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.645+0000 7f38c9a2e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645ee620 0x55e0645ebf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.645+0000 7f38c9a2e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645ee620 0x55e0645ebf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37748/0 (socket says 172.21.15.107:37748) 2024-04-25T00:17:04.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.645+0000 7f38c9a2e640 1 -- 172.21.15.107:0/930651687 learned_addr learned my addr 172.21.15.107:0/930651687 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:04.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.645+0000 7f38c922d640 1 --2- 172.21.15.107:0/930651687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0645e3630 0x55e0645eb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.645+0000 7f38c8a2c640 1 --2- 172.21.15.107:0/930651687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0645e3f50 0x55e0645eba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.645+0000 7f38c9a2e640 1 -- 172.21.15.107:0/930651687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0645e3630 msgr2=0x55e0645eb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:04.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.645+0000 7f38c9a2e640 1 --2- 172.21.15.107:0/930651687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0645e3630 0x55e0645eb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.645+0000 7f38c9a2e640 1 -- 172.21.15.107:0/930651687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0645e3f50 msgr2=0x55e0645eba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:04.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.645+0000 7f38c9a2e640 1 --2- 172.21.15.107:0/930651687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0645e3f50 0x55e0645eba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.645+0000 7f38c9a2e640 1 -- 172.21.15.107:0/930651687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e06425c3f0 con 0x55e0645ee620 2024-04-25T00:17:04.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.646+0000 7f38c9a2e640 1 --2- 172.21.15.107:0/930651687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645ee620 0x55e0645ebf90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f38bc003bf0 tx=0x7f38bc02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f6066892671700 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:04.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.646+0000 7f38acff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:17:04.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.646+0000 7f38acff9640 1 -- 172.21.15.107:0/930651687 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc04f030 con 0x55e0645ee620 2024-04-25T00:17:04.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.646+0000 7f38acff9640 1 -- 172.21.15.107:0/930651687 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f38bc004450 con 0x55e0645ee620 2024-04-25T00:17:04.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.646+0000 7f38ce195740 1 -- 172.21.15.107:0/930651687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38c0010cb0 con 0x55e0645ee620 2024-04-25T00:17:04.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.646+0000 7f38acff9640 1 -- 172.21.15.107:0/930651687 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f38bc004650 con 0x55e0645ee620 2024-04-25T00:17:04.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.646+0000 7f38ce195740 10 client.15735 fetch_fsmap learned FSMap version 24 2024-04-25T00:17:04.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.646+0000 7f38ce195740 1 -- 172.21.15.107:0/930651687 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e06421c0f0 con 0x55e0645ee620 2024-04-25T00:17:04.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.646+0000 7f38acff9640 1 -- 172.21.15.107:0/930651687 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38c0010cb0 con 0x55e0645ee620 2024-04-25T00:17:04.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.646+0000 7f38acff9640 1 -- 172.21.15.107:0/930651687 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f38bc048050 con 0x55e0645ee620 2024-04-25T00:17:04.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.646+0000 7f38ce195740 10 client.15735 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:17:04.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.646+0000 7f38ce195740 10 client.15735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:04.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.646+0000 7f38ce195740 20 client.15735 populate_metadata read hostname 'smithi107' 2024-04-25T00:17:04.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.646+0000 7f38ce195740 1 --2- 172.21.15.107:0/930651687 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e0645fe8a0 0x55e064600cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.646+0000 7f38ce195740 4 client.15735 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:17:04.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.646+0000 7f38ce195740 1 -- 172.21.15.107:0/930651687 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e064555bf0 con 0x55e0645fe8a0 2024-04-25T00:17:04.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.647+0000 7f38c922d640 1 --2- 172.21.15.107:0/930651687 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e0645fe8a0 0x55e064600cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.647+0000 7f38c922d640 1 --2- 172.21.15.107:0/930651687 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e0645fe8a0 0x55e064600cf0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7df54306ea5f151 server_cookie=fb6fbbf95b80d6ab in_seq=0 out_seq=0 2024-04-25T00:17:04.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.647+0000 7f38acff9640 10 client.15735 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:04.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.648+0000 7f38acff9640 1 -- 172.21.15.107:0/930651687 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e064555bf0 con 0x55e0645fe8a0 2024-04-25T00:17:04.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.648+0000 7f38acff9640 10 client.15735 handle_command_reply: tid=0 2024-04-25T00:17:04.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.648+0000 7f38ce195740 1 client.15735 shutdown 2024-04-25T00:17:04.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.648+0000 7f38ce195740 1 -- 172.21.15.107:0/930651687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645ee620 msgr2=0x55e0645ebf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:04.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.648+0000 7f38ce195740 1 --2- 172.21.15.107:0/930651687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645ee620 0x55e0645ebf90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f38bc003bf0 tx=0x7f38bc02a8e0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.648+0000 7f38ce195740 1 -- 172.21.15.107:0/930651687 shutdown_connections 2024-04-25T00:17:04.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.648+0000 7f38ce195740 1 --2- 172.21.15.107:0/930651687 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e0645fe8a0 0x55e064600cf0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.648+0000 7f38ce195740 1 --2- 172.21.15.107:0/930651687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645ee620 0x55e0645ebf90 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.648+0000 7f38ce195740 1 --2- 172.21.15.107:0/930651687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0645e3f50 0x55e0645eba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.648+0000 7f38acff9640 0 client.15735 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:04.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.648+0000 7f38ce195740 1 --2- 172.21.15.107:0/930651687 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0645e3630 0x55e0645eb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.648+0000 7f38ce195740 1 -- 172.21.15.107:0/930651687 >> 172.21.15.107:0/930651687 conn(0x55e0645df950 msgr2=0x55e0645e5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:04.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.648+0000 7f38ce195740 1 -- 172.21.15.107:0/930651687 shutdown_connections 2024-04-25T00:17:04.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.648+0000 7f38ce195740 1 -- 172.21.15.107:0/930651687 wait complete. 2024-04-25T00:17:04.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.648+0000 7f38ce195740 20 client.15735 trim_cache size 0 max 16384 2024-04-25T00:17:04.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.665+0000 7f38ce195740 1 Processor -- start 2024-04-25T00:17:04.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.665+0000 7f38ce195740 1 -- start start 2024-04-25T00:17:04.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.665+0000 7f38ce195740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0645e3630 0x55e0645fa2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.665+0000 7f38ce195740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0645e3f50 0x55e0645ea470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.665+0000 7f38ce195740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645ee620 0x55e0645f6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.665+0000 7f38ce195740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f38c41b1490 con 0x55e0645ee620 2024-04-25T00:17:04.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.665+0000 7f38ce195740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f38c41aed60 con 0x55e0645e3f50 2024-04-25T00:17:04.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.665+0000 7f38ce195740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f38c41acbb0 con 0x55e0645e3630 2024-04-25T00:17:04.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.665+0000 7f38c9a2e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645ee620 0x55e0645f6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.666+0000 7f38c9a2e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645ee620 0x55e0645f6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37750/0 (socket says 172.21.15.107:37750) 2024-04-25T00:17:04.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.666+0000 7f38c9a2e640 1 -- 172.21.15.107:0/2257823291 learned_addr learned my addr 172.21.15.107:0/2257823291 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:04.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.666+0000 7f38c8a2c640 1 --2- 172.21.15.107:0/2257823291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0645e3f50 0x55e0645ea470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.666+0000 7f38c9a2e640 1 -- 172.21.15.107:0/2257823291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0645e3630 msgr2=0x55e0645fa2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:04.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.666+0000 7f38c922d640 1 --2- 172.21.15.107:0/2257823291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0645e3630 0x55e0645fa2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.666+0000 7f38c9a2e640 1 --2- 172.21.15.107:0/2257823291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0645e3630 0x55e0645fa2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.666+0000 7f38c9a2e640 1 -- 172.21.15.107:0/2257823291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0645e3f50 msgr2=0x55e0645ea470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:04.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.666+0000 7f38c9a2e640 1 --2- 172.21.15.107:0/2257823291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0645e3f50 0x55e0645ea470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.666+0000 7f38c9a2e640 1 -- 172.21.15.107:0/2257823291 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0645ee440 con 0x55e0645ee620 2024-04-25T00:17:04.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.666+0000 7f38c8a2c640 1 --2- 172.21.15.107:0/2257823291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0645e3f50 0x55e0645ea470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:17:04.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.666+0000 7f38c922d640 1 --2- 172.21.15.107:0/2257823291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0645e3630 0x55e0645fa2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:17:04.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.666+0000 7f38c9a2e640 1 --2- 172.21.15.107:0/2257823291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645ee620 0x55e0645f6bc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009600 tx=0x7f38bc02af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee725f8af3f785e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:04.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.666+0000 7f38aeffd640 1 -- 172.21.15.107:0/2257823291 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc04f070 con 0x55e0645ee620 2024-04-25T00:17:04.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.666+0000 7f38aeffd640 1 -- 172.21.15.107:0/2257823291 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38bc036040 con 0x55e0645ee620 2024-04-25T00:17:04.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.666+0000 7f38aeffd640 1 -- 172.21.15.107:0/2257823291 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38bc04b7e0 con 0x55e0645ee620 2024-04-25T00:17:04.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.666+0000 7f38ce195740 1 -- 172.21.15.107:0/2257823291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645ee620 msgr2=0x55e0645f6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:04.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.667+0000 7f38ce195740 1 --2- 172.21.15.107:0/2257823291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645ee620 0x55e0645f6bc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009600 tx=0x7f38bc02af60 comp rx=0 tx=0).stop 2024-04-25T00:17:04.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.667+0000 7f38ce195740 1 -- 172.21.15.107:0/2257823291 shutdown_connections 2024-04-25T00:17:04.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.667+0000 7f38ce195740 1 --2- 172.21.15.107:0/2257823291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645ee620 0x55e0645f6bc0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.667+0000 7f38ce195740 1 --2- 172.21.15.107:0/2257823291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0645e3f50 0x55e0645ea470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.667+0000 7f38ce195740 1 --2- 172.21.15.107:0/2257823291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0645e3630 0x55e0645fa2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.667+0000 7f38ce195740 1 -- 172.21.15.107:0/2257823291 >> 172.21.15.107:0/2257823291 conn(0x55e0645df950 msgr2=0x55e0645deaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:04.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.667+0000 7f38ce195740 1 -- 172.21.15.107:0/2257823291 shutdown_connections 2024-04-25T00:17:04.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.667+0000 7f38ce195740 1 -- 172.21.15.107:0/2257823291 wait complete. 2024-04-25T00:17:04.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.667+0000 7f38ce195740 1 -- start start 2024-04-25T00:17:04.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.667+0000 7f38ce195740 1 Processor -- start 2024-04-25T00:17:04.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.667+0000 7f38ce195740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0645e3630 0x55e0645e9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.667+0000 7f38ce195740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645e3f50 0x55e064605e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.667+0000 7f38ce195740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0645ee620 0x55e0646065c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.667+0000 7f38ce195740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f38c41ad220 con 0x55e0645e3f50 2024-04-25T00:17:04.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.667+0000 7f38ce195740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f38c41ae690 con 0x55e0645ee620 2024-04-25T00:17:04.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.667+0000 7f38ce195740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f38c41ad8f0 con 0x55e0645e3630 2024-04-25T00:17:04.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.667+0000 7f38ce195740 10 client.0 _ll_register_callbacks cb 0x7f38bc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:04.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.667+0000 7f38c8a2c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645e3f50 0x55e064605e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.667+0000 7f38c8a2c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645e3f50 0x55e064605e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37758/0 (socket says 172.21.15.107:37758) 2024-04-25T00:17:04.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.668+0000 7f38c8a2c640 1 -- 172.21.15.107:0/4126229709 learned_addr learned my addr 172.21.15.107:0/4126229709 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:04.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.668+0000 7f38c9a2e640 1 --2- 172.21.15.107:0/4126229709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0645ee620 0x55e0646065c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.668+0000 7f38c922d640 1 --2- 172.21.15.107:0/4126229709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0645e3630 0x55e0645e9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.668+0000 7f38c8a2c640 1 -- 172.21.15.107:0/4126229709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0645e3630 msgr2=0x55e0645e9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:04.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.668+0000 7f38c8a2c640 1 --2- 172.21.15.107:0/4126229709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0645e3630 0x55e0645e9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.668+0000 7f38c8a2c640 1 -- 172.21.15.107:0/4126229709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0645ee620 msgr2=0x55e0646065c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:04.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.668+0000 7f38c8a2c640 1 --2- 172.21.15.107:0/4126229709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0645ee620 0x55e0646065c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.668+0000 7f38c8a2c640 1 -- 172.21.15.107:0/4126229709 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e064667e60 con 0x55e0645e3f50 2024-04-25T00:17:04.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.668+0000 7f38c8a2c640 1 --2- 172.21.15.107:0/4126229709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645e3f50 0x55e064605e80 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f38c000cae0 tx=0x7f38c0005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc7e586323e45a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:04.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.668+0000 7f38acff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:17:04.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.668+0000 7f38acff9640 1 -- 172.21.15.107:0/4126229709 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c00055c0 con 0x55e0645e3f50 2024-04-25T00:17:04.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.668+0000 7f38acff9640 1 -- 172.21.15.107:0/4126229709 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f38c001a850 con 0x55e0645e3f50 2024-04-25T00:17:04.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.668+0000 7f38ce195740 1 -- 172.21.15.107:0/4126229709 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38bc036040 con 0x55e0645e3f50 2024-04-25T00:17:04.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.668+0000 7f38acff9640 1 -- 172.21.15.107:0/4126229709 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f38c0033050 con 0x55e0645e3f50 2024-04-25T00:17:04.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.668+0000 7f38ce195740 10 client.15741 fetch_fsmap learned FSMap version 24 2024-04-25T00:17:04.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.668+0000 7f38ce195740 1 -- 172.21.15.107:0/4126229709 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e064668fb0 con 0x55e0645e3f50 2024-04-25T00:17:04.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.668+0000 7f38acff9640 1 -- 172.21.15.107:0/4126229709 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38bc036040 con 0x55e0645e3f50 2024-04-25T00:17:04.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.669+0000 7f38acff9640 1 -- 172.21.15.107:0/4126229709 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f38c0025460 con 0x55e0645e3f50 2024-04-25T00:17:04.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.669+0000 7f38ce195740 10 client.15741 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:17:04.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.669+0000 7f38ce195740 10 client.15741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:04.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.669+0000 7f38ce195740 20 client.15741 populate_metadata read hostname 'smithi107' 2024-04-25T00:17:04.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.669+0000 7f38ce195740 1 --2- 172.21.15.107:0/4126229709 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e0645fe8a0 0x55e06466b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.669+0000 7f38ce195740 4 client.15741 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:17:04.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.669+0000 7f38ce195740 1 -- 172.21.15.107:0/4126229709 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e06454ff40 con 0x55e0645fe8a0 2024-04-25T00:17:04.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.669+0000 7f38c922d640 1 --2- 172.21.15.107:0/4126229709 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e0645fe8a0 0x55e06466b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.669+0000 7f38acff9640 10 client.15741 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:04.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.669+0000 7f38c922d640 1 --2- 172.21.15.107:0/4126229709 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e0645fe8a0 0x55e06466b600 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b69cd9fa8ce4fa6a server_cookie=8be4c5990140c444 in_seq=0 out_seq=0 2024-04-25T00:17:04.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.669+0000 7f38acff9640 1 -- 172.21.15.107:0/4126229709 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e06454ff40 con 0x55e0645fe8a0 2024-04-25T00:17:04.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.669+0000 7f38acff9640 10 client.15741 handle_command_reply: tid=0 2024-04-25T00:17:04.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.669+0000 7f38ce195740 1 client.15741 shutdown 2024-04-25T00:17:04.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.669+0000 7f38ce195740 1 -- 172.21.15.107:0/4126229709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645e3f50 msgr2=0x55e064605e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:04.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.669+0000 7f38ce195740 1 --2- 172.21.15.107:0/4126229709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645e3f50 0x55e064605e80 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f38c000cae0 tx=0x7f38c0005040 comp rx=0 tx=0).stop 2024-04-25T00:17:04.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.670+0000 7f38ce195740 1 -- 172.21.15.107:0/4126229709 shutdown_connections 2024-04-25T00:17:04.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.670+0000 7f38ce195740 1 --2- 172.21.15.107:0/4126229709 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e0645fe8a0 0x55e06466b600 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.670+0000 7f38ce195740 1 --2- 172.21.15.107:0/4126229709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0645ee620 0x55e0646065c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.670+0000 7f38ce195740 1 --2- 172.21.15.107:0/4126229709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e0645e3f50 0x55e064605e80 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.670+0000 7f38acff9640 0 client.15741 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:04.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.670+0000 7f38ce195740 1 --2- 172.21.15.107:0/4126229709 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e0645e3630 0x55e0645e9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.670+0000 7f38ce195740 1 -- 172.21.15.107:0/4126229709 >> 172.21.15.107:0/4126229709 conn(0x55e0645df950 msgr2=0x55e0645eb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:04.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.670+0000 7f38ce195740 1 -- 172.21.15.107:0/4126229709 shutdown_connections 2024-04-25T00:17:04.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.670+0000 7f38ce195740 1 -- 172.21.15.107:0/4126229709 wait complete. 2024-04-25T00:17:04.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.670+0000 7f38ce195740 20 client.15741 trim_cache size 0 max 16384 2024-04-25T00:17:04.678 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:17:04.678 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-04-25T00:17:04.679 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-04-25T00:17:04.679 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:17:04.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.670+0000 7f38cb491640 1 -- 172.21.15.107:0/2427059221 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f38b00a2f30 msgr2=0x7f38b00a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:04.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.670+0000 7f38cb491640 1 --2- 172.21.15.107:0/2427059221 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f38b00a2f30 0x7f38b00a53f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009a60 tx=0x7f38bc005da0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.670+0000 7f38cb491640 1 -- 172.21.15.107:0/2427059221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38c4101710 msgr2=0x7f38c4197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:04.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.670+0000 7f38cb491640 1 --2- 172.21.15.107:0/2427059221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38c4101710 0x7f38c4197ff0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f38b400c890 tx=0x7f38b400cd50 comp rx=0 tx=0).stop 2024-04-25T00:17:04.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.670+0000 7f38cb491640 1 -- 172.21.15.107:0/2427059221 shutdown_connections 2024-04-25T00:17:04.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.670+0000 7f38cb491640 1 --2- 172.21.15.107:0/2427059221 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f38b00a2f30 0x7f38b00a53f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.670+0000 7f38cb491640 1 --2- 172.21.15.107:0/2427059221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f38c4107720 0x7f38c419c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.670+0000 7f38cb491640 1 --2- 172.21.15.107:0/2427059221 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38c41020d0 0x7f38c4198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.670+0000 7f38cb491640 1 --2- 172.21.15.107:0/2427059221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38c4101710 0x7f38c4197ff0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.670+0000 7f38cb491640 1 -- 172.21.15.107:0/2427059221 >> 172.21.15.107:0/2427059221 conn(0x7f38c40fd180 msgr2=0x7f38c40fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:04.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.670+0000 7f38cb491640 1 -- 172.21.15.107:0/2427059221 shutdown_connections 2024-04-25T00:17:04.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.670+0000 7f38cb491640 1 -- 172.21.15.107:0/2427059221 wait complete. 2024-04-25T00:17:04.681 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:17:04.681 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:17:04.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.779+0000 7f5298283640 1 -- 172.21.15.107:0/3444422879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52901029d0 msgr2=0x7f5290102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:04.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.779+0000 7f5298283640 1 --2- 172.21.15.107:0/3444422879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52901029d0 0x7f5290102db0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f528c009900 tx=0x7f528c02a510 comp rx=0 tx=0).stop 2024-04-25T00:17:04.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.779+0000 7f5298283640 1 -- 172.21.15.107:0/3444422879 shutdown_connections 2024-04-25T00:17:04.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.779+0000 7f5298283640 1 --2- 172.21.15.107:0/3444422879 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f529010bd10 0x7f529010e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.779+0000 7f5298283640 1 --2- 172.21.15.107:0/3444422879 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52901032f0 0x7f529010b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.779+0000 7f5298283640 1 --2- 172.21.15.107:0/3444422879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52901029d0 0x7f5290102db0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.779+0000 7f5298283640 1 -- 172.21.15.107:0/3444422879 >> 172.21.15.107:0/3444422879 conn(0x7f5290077740 msgr2=0x7f5290077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:04.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.779+0000 7f5298283640 1 -- 172.21.15.107:0/3444422879 shutdown_connections 2024-04-25T00:17:04.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.779+0000 7f5298283640 1 -- 172.21.15.107:0/3444422879 wait complete. 2024-04-25T00:17:04.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.780+0000 7f5298283640 1 Processor -- start 2024-04-25T00:17:04.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.780+0000 7f5298283640 1 -- start start 2024-04-25T00:17:04.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.780+0000 7f5298283640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52901029d0 0x7f5290197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.780+0000 7f5298283640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52901032f0 0x7f5290198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.780+0000 7f5298283640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f529010bd10 0x7f529019c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.780+0000 7f5298283640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5290111b10 con 0x7f52901032f0 2024-04-25T00:17:04.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.780+0000 7f5298283640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5290111990 con 0x7f52901029d0 2024-04-25T00:17:04.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.780+0000 7f5298283640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5290111c90 con 0x7f529010bd10 2024-04-25T00:17:04.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.780+0000 7f529581e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52901032f0 0x7f5290198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.780+0000 7f529581e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52901032f0 0x7f5290198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37770/0 (socket says 172.21.15.107:37770) 2024-04-25T00:17:04.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.780+0000 7f529581e640 1 -- 172.21.15.107:0/2042429617 learned_addr learned my addr 172.21.15.107:0/2042429617 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:04.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.780+0000 7f5296820640 1 --2- 172.21.15.107:0/2042429617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f529010bd10 0x7f529019c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.780+0000 7f529601f640 1 --2- 172.21.15.107:0/2042429617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52901029d0 0x7f5290197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.781+0000 7f529581e640 1 -- 172.21.15.107:0/2042429617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f529010bd10 msgr2=0x7f529019c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:04.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.781+0000 7f529581e640 1 --2- 172.21.15.107:0/2042429617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f529010bd10 0x7f529019c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.781+0000 7f529581e640 1 -- 172.21.15.107:0/2042429617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52901029d0 msgr2=0x7f5290197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:04.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.781+0000 7f529581e640 1 --2- 172.21.15.107:0/2042429617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52901029d0 0x7f5290197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.781+0000 7f529581e640 1 -- 172.21.15.107:0/2042429617 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529019cfe0 con 0x7f52901032f0 2024-04-25T00:17:04.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.781+0000 7f529601f640 1 --2- 172.21.15.107:0/2042429617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52901029d0 0x7f5290197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:17:04.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.781+0000 7f529581e640 1 --2- 172.21.15.107:0/2042429617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52901032f0 0x7f5290198530 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f528000c890 tx=0x7f528000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:04.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.781+0000 7f52877fe640 1 -- 172.21.15.107:0/2042429617 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280004220 con 0x7f52901032f0 2024-04-25T00:17:04.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.781+0000 7f5298283640 1 -- 172.21.15.107:0/2042429617 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f529019d2d0 con 0x7f52901032f0 2024-04-25T00:17:04.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.781+0000 7f52877fe640 1 -- 172.21.15.107:0/2042429617 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52800043c0 con 0x7f52901032f0 2024-04-25T00:17:04.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.781+0000 7f52877fe640 1 -- 172.21.15.107:0/2042429617 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280004a00 con 0x7f52901032f0 2024-04-25T00:17:04.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.781+0000 7f5298283640 1 -- 172.21.15.107:0/2042429617 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52901a4b60 con 0x7f52901032f0 2024-04-25T00:17:04.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.782+0000 7f5298283640 1 -- 172.21.15.107:0/2042429617 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5290068560 con 0x7f52901032f0 2024-04-25T00:17:04.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.783+0000 7f52877fe640 1 -- 172.21.15.107:0/2042429617 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f52800079b0 con 0x7f52901032f0 2024-04-25T00:17:04.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.783+0000 7f52877fe640 1 --2- 172.21.15.107:0/2042429617 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f527c0a2eb0 0x7f527c0a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.784+0000 7f529601f640 1 --2- 172.21.15.107:0/2042429617 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f527c0a2eb0 0x7f527c0a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.784+0000 7f52877fe640 1 -- 172.21.15.107:0/2042429617 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f52800cf7a0 con 0x7f52901032f0 2024-04-25T00:17:04.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.784+0000 7f529601f640 1 --2- 172.21.15.107:0/2042429617 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f527c0a2eb0 0x7f527c0a5370 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f528c005ab0 tx=0x7f528c005a20 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:04.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:04.788+0000 7f52877fe640 1 -- 172.21.15.107:0/2042429617 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5280014030 con 0x7f52901032f0 2024-04-25T00:17:05.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.023+0000 7f5298283640 1 -- 172.21.15.107:0/2042429617 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5290106ef0 con 0x7f52901032f0 2024-04-25T00:17:05.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.024+0000 7f52877fe640 1 -- 172.21.15.107:0/2042429617 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11897 (secure 0 0 0) 0x7f528009be10 con 0x7f52901032f0 2024-04-25T00:17:05.024 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:17:05.024 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:16:24.724978+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24493,24521]},"id":1}]} 2024-04-25T00:17:05.025 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 24 2024-04-25T00:17:05.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.026+0000 7f5298283640 1 -- 172.21.15.107:0/2042429617 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f527c0a2eb0 msgr2=0x7f527c0a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.026+0000 7f5298283640 1 --2- 172.21.15.107:0/2042429617 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f527c0a2eb0 0x7f527c0a5370 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f528c005ab0 tx=0x7f528c005a20 comp rx=0 tx=0).stop 2024-04-25T00:17:05.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.026+0000 7f5298283640 1 -- 172.21.15.107:0/2042429617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52901032f0 msgr2=0x7f5290198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.026+0000 7f5298283640 1 --2- 172.21.15.107:0/2042429617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52901032f0 0x7f5290198530 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f528000c890 tx=0x7f528000cd50 comp rx=0 tx=0).stop 2024-04-25T00:17:05.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.026+0000 7f5298283640 1 -- 172.21.15.107:0/2042429617 shutdown_connections 2024-04-25T00:17:05.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.026+0000 7f5298283640 1 --2- 172.21.15.107:0/2042429617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f529010bd10 0x7f529019c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.026+0000 7f5298283640 1 --2- 172.21.15.107:0/2042429617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52901032f0 0x7f5290198530 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.026+0000 7f5298283640 1 --2- 172.21.15.107:0/2042429617 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f527c0a2eb0 0x7f527c0a5370 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.026+0000 7f5298283640 1 --2- 172.21.15.107:0/2042429617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52901029d0 0x7f5290197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.026+0000 7f5298283640 1 -- 172.21.15.107:0/2042429617 >> 172.21.15.107:0/2042429617 conn(0x7f5290077740 msgr2=0x7f52900fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.026+0000 7f5298283640 1 -- 172.21.15.107:0/2042429617 shutdown_connections 2024-04-25T00:17:05.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.026+0000 7f5298283640 1 -- 172.21.15.107:0/2042429617 wait complete. 2024-04-25T00:17:05.038 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:17:05.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.141+0000 7f9fc0a6b640 1 -- 172.21.15.107:0/468605088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fbc1066b0 msgr2=0x7f9fbc106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.141+0000 7f9fc0a6b640 1 --2- 172.21.15.107:0/468605088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fbc1066b0 0x7f9fbc106a90 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9fac009930 tx=0x7f9fac02a420 comp rx=0 tx=0).stop 2024-04-25T00:17:05.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.141+0000 7f9fc0a6b640 1 -- 172.21.15.107:0/468605088 shutdown_connections 2024-04-25T00:17:05.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.141+0000 7f9fc0a6b640 1 --2- 172.21.15.107:0/468605088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fbc100fd0 0x7f9fbc10d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.141+0000 7f9fc0a6b640 1 --2- 172.21.15.107:0/468605088 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fbc100610 0x7f9fbc100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.141+0000 7f9fc0a6b640 1 --2- 172.21.15.107:0/468605088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fbc1066b0 0x7f9fbc106a90 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.141+0000 7f9fc0a6b640 1 -- 172.21.15.107:0/468605088 >> 172.21.15.107:0/468605088 conn(0x7f9fbc077740 msgr2=0x7f9fbc077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.141+0000 7f9fc0a6b640 1 -- 172.21.15.107:0/468605088 shutdown_connections 2024-04-25T00:17:05.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.141+0000 7f9fc0a6b640 1 -- 172.21.15.107:0/468605088 wait complete. 2024-04-25T00:17:05.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.142+0000 7f9fc0a6b640 1 Processor -- start 2024-04-25T00:17:05.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.142+0000 7f9fc0a6b640 1 -- start start 2024-04-25T00:17:05.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.142+0000 7f9fc0a6b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fbc100610 0x7f9fbc198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.142+0000 7f9fc0a6b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fbc100fd0 0x7f9fbc198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.142+0000 7f9fc0a6b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fbc1066b0 0x7f9fbc19c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.142+0000 7f9fc0a6b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc110a30 con 0x7f9fbc100610 2024-04-25T00:17:05.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.142+0000 7f9fc0a6b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc1108b0 con 0x7f9fbc100fd0 2024-04-25T00:17:05.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.142+0000 7f9fc0a6b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc110bb0 con 0x7f9fbc1066b0 2024-04-25T00:17:05.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.142+0000 7f9fba59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fbc100610 0x7f9fbc198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.142+0000 7f9fba59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fbc100610 0x7f9fbc198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37790/0 (socket says 172.21.15.107:37790) 2024-04-25T00:17:05.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.142+0000 7f9fba59c640 1 -- 172.21.15.107:0/4223756673 learned_addr learned my addr 172.21.15.107:0/4223756673 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:05.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.142+0000 7f9fb3fff640 1 --2- 172.21.15.107:0/4223756673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fbc100fd0 0x7f9fbc198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.142+0000 7f9fba59c640 1 -- 172.21.15.107:0/4223756673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fbc1066b0 msgr2=0x7f9fbc19c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.142+0000 7f9fbad9d640 1 --2- 172.21.15.107:0/4223756673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fbc1066b0 0x7f9fbc19c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.142+0000 7f9fba59c640 1 --2- 172.21.15.107:0/4223756673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fbc1066b0 0x7f9fbc19c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.142+0000 7f9fba59c640 1 -- 172.21.15.107:0/4223756673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fbc100fd0 msgr2=0x7f9fbc198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.142+0000 7f9fba59c640 1 --2- 172.21.15.107:0/4223756673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fbc100fd0 0x7f9fbc198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.142+0000 7f9fba59c640 1 -- 172.21.15.107:0/4223756673 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fbc19d000 con 0x7f9fbc100610 2024-04-25T00:17:05.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.142+0000 7f9fb3fff640 1 --2- 172.21.15.107:0/4223756673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fbc100fd0 0x7f9fbc198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:17:05.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.142+0000 7f9fbad9d640 1 --2- 172.21.15.107:0/4223756673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fbc1066b0 0x7f9fbc19c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:17:05.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.142+0000 7f9fba59c640 1 --2- 172.21.15.107:0/4223756673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fbc100610 0x7f9fbc198010 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9fac005940 tx=0x7f9fac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:05.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.143+0000 7f9fb37fe640 1 -- 172.21.15.107:0/4223756673 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac03f030 con 0x7f9fbc100610 2024-04-25T00:17:05.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.143+0000 7f9fb37fe640 1 -- 172.21.15.107:0/4223756673 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fac003ce0 con 0x7f9fbc100610 2024-04-25T00:17:05.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.143+0000 7f9fc0a6b640 1 -- 172.21.15.107:0/4223756673 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fbc19d290 con 0x7f9fbc100610 2024-04-25T00:17:05.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.143+0000 7f9fb37fe640 1 -- 172.21.15.107:0/4223756673 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac035070 con 0x7f9fbc100610 2024-04-25T00:17:05.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.143+0000 7f9fc0a6b640 1 -- 172.21.15.107:0/4223756673 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fbc1a4b80 con 0x7f9fbc100610 2024-04-25T00:17:05.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.145+0000 7f9fb37fe640 1 -- 172.21.15.107:0/4223756673 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f9fac042030 con 0x7f9fbc100610 2024-04-25T00:17:05.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.145+0000 7f9fc377b740 1 Processor -- start 2024-04-25T00:17:05.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.145+0000 7f9fc377b740 1 -- start start 2024-04-25T00:17:05.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.146+0000 7f9fc377b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b7f580 0x55b621b7f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.146+0000 7f9fc377b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b621b7fea0 0x55b621b8a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.146+0000 7f9fc377b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b621b8a570 0x55b621b8c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.146+0000 7f9fc377b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b621ae4670 con 0x55b621b7f580 2024-04-25T00:17:05.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.146+0000 7f9fc377b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b621a8c970 con 0x55b621b8a570 2024-04-25T00:17:05.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.146+0000 7f9fc377b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b621ae0630 con 0x55b621b7fea0 2024-04-25T00:17:05.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.146+0000 7f9fb37fe640 1 --2- 172.21.15.107:0/4223756673 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f9fa80a2eb0 0x7f9fa80a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.146+0000 7f9fbad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b7f580 0x55b621b7f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.146+0000 7f9fba59c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b621b7fea0 0x55b621b8a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.146+0000 7f9fbad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b7f580 0x55b621b7f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37792/0 (socket says 172.21.15.107:37792) 2024-04-25T00:17:05.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.146+0000 7f9fbad9d640 1 -- 172.21.15.107:0/3864933073 learned_addr learned my addr 172.21.15.107:0/3864933073 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:05.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.146+0000 7f9fb3fff640 1 --2- 172.21.15.107:0/3864933073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b621b8a570 0x55b621b8c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.146+0000 7f9fbad9d640 1 --2- 172.21.15.107:0/4223756673 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f9fa80a2eb0 0x7f9fa80a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.146+0000 7f9fb37fe640 1 -- 172.21.15.107:0/4223756673 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f9fac0f0070 con 0x7f9fbc100610 2024-04-25T00:17:05.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.146+0000 7f9fba59c640 1 -- 172.21.15.107:0/3864933073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b621b8a570 msgr2=0x55b621b8c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.146+0000 7f9fba59c640 1 --2- 172.21.15.107:0/3864933073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b621b8a570 0x55b621b8c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.146+0000 7f9fba59c640 1 -- 172.21.15.107:0/3864933073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b7f580 msgr2=0x55b621b7f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.146+0000 7f9fba59c640 1 --2- 172.21.15.107:0/3864933073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b7f580 0x55b621b7f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.146+0000 7f9fba59c640 1 -- 172.21.15.107:0/3864933073 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b621ab0400 con 0x55b621b7fea0 2024-04-25T00:17:05.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.147+0000 7f9fbad9d640 1 --2- 172.21.15.107:0/4223756673 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f9fa80a2eb0 0x7f9fa80a5370 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400eca0 tx=0x7f9fb4009ad0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:05.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.147+0000 7f9fba59c640 1 --2- 172.21.15.107:0/3864933073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b621b7fea0 0x55b621b8a030 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0ed210 tx=0x7f9fac08f050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e48d84a8fbce906d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:05.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.147+0000 7f9fb0ff9640 1 -- 172.21.15.107:0/3864933073 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac049020 con 0x55b621b7fea0 2024-04-25T00:17:05.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.147+0000 7f9fb0ff9640 1 -- 172.21.15.107:0/3864933073 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fac0dc0c0 con 0x55b621b7fea0 2024-04-25T00:17:05.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.147+0000 7f9fb0ff9640 1 -- 172.21.15.107:0/3864933073 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac0c0390 con 0x55b621b7fea0 2024-04-25T00:17:05.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.147+0000 7f9fc377b740 1 -- 172.21.15.107:0/3864933073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b621b7fea0 msgr2=0x55b621b8a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.147+0000 7f9fc377b740 1 --2- 172.21.15.107:0/3864933073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b621b7fea0 0x55b621b8a030 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0ed210 tx=0x7f9fac08f050 comp rx=0 tx=0).stop 2024-04-25T00:17:05.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.148+0000 7f9fc377b740 1 -- 172.21.15.107:0/3864933073 shutdown_connections 2024-04-25T00:17:05.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.148+0000 7f9fc377b740 1 --2- 172.21.15.107:0/3864933073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b621b8a570 0x55b621b8c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.148+0000 7f9fc377b740 1 --2- 172.21.15.107:0/3864933073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b621b7fea0 0x55b621b8a030 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.148+0000 7f9fc377b740 1 --2- 172.21.15.107:0/3864933073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b7f580 0x55b621b7f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.148+0000 7f9fc377b740 1 -- 172.21.15.107:0/3864933073 >> 172.21.15.107:0/3864933073 conn(0x55b621b7b8a0 msgr2=0x55b621b7bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.148+0000 7f9fc377b740 1 -- 172.21.15.107:0/3864933073 shutdown_connections 2024-04-25T00:17:05.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.148+0000 7f9fc377b740 1 -- 172.21.15.107:0/3864933073 wait complete. 2024-04-25T00:17:05.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.148+0000 7f9fc377b740 1 -- start start 2024-04-25T00:17:05.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.148+0000 7f9fc377b740 1 Processor -- start 2024-04-25T00:17:05.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.148+0000 7f9fc377b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b7f580 0x55b621b89f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.148+0000 7f9fc377b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b621b7fea0 0x55b621b87200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.148+0000 7f9fc377b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b621b8a570 0x55b621b877a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.148+0000 7f9fc377b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b621ae0630 con 0x55b621b7f580 2024-04-25T00:17:05.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.148+0000 7f9fc377b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b621aa0050 con 0x55b621b8a570 2024-04-25T00:17:05.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.148+0000 7f9fc377b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b621b8d120 con 0x55b621b7fea0 2024-04-25T00:17:05.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.148+0000 7f9fc377b740 10 client.0 _ll_register_callbacks cb 0x55b6217f7be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:05.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.148+0000 7f9fba59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b7f580 0x55b621b89f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.148+0000 7f9fba59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b7f580 0x55b621b89f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37794/0 (socket says 172.21.15.107:37794) 2024-04-25T00:17:05.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.148+0000 7f9fba59c640 1 -- 172.21.15.107:0/3858426223 learned_addr learned my addr 172.21.15.107:0/3858426223 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:05.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.148+0000 7f9fbad9d640 1 --2- 172.21.15.107:0/3858426223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b621b8a570 0x55b621b877a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.148+0000 7f9fb3fff640 1 --2- 172.21.15.107:0/3858426223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b621b7fea0 0x55b621b87200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.149+0000 7f9fba59c640 1 -- 172.21.15.107:0/3858426223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b621b7fea0 msgr2=0x55b621b87200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.149+0000 7f9fba59c640 1 --2- 172.21.15.107:0/3858426223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b621b7fea0 0x55b621b87200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.149+0000 7f9fba59c640 1 -- 172.21.15.107:0/3858426223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b621b8a570 msgr2=0x55b621b877a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.149+0000 7f9fba59c640 1 --2- 172.21.15.107:0/3858426223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b621b8a570 0x55b621b877a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.149+0000 7f9fba59c640 1 -- 172.21.15.107:0/3858426223 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b621a71dc0 con 0x55b621b7f580 2024-04-25T00:17:05.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.149+0000 7f9fb3fff640 1 --2- 172.21.15.107:0/3858426223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b621b7fea0 0x55b621b87200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:17:05.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.149+0000 7f9fba59c640 1 --2- 172.21.15.107:0/3858426223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b7f580 0x55b621b89f10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0ed270 tx=0x7f9fac0095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ae739b4999cb98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:05.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.149+0000 7f9fa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:17:05.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.149+0000 7f9fa1ffb640 1 -- 172.21.15.107:0/3858426223 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac049040 con 0x55b621b7f580 2024-04-25T00:17:05.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.149+0000 7f9fa1ffb640 1 -- 172.21.15.107:0/3858426223 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f9fac03bc80 con 0x55b621b7f580 2024-04-25T00:17:05.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.149+0000 7f9fc377b740 1 -- 172.21.15.107:0/3858426223 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b621af1bb0 con 0x55b621b7f580 2024-04-25T00:17:05.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.149+0000 7f9fa1ffb640 1 -- 172.21.15.107:0/3858426223 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9fac0c4c50 con 0x55b621b7f580 2024-04-25T00:17:05.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.149+0000 7f9fc377b740 10 client.15759 fetch_fsmap learned FSMap version 24 2024-04-25T00:17:05.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.149+0000 7f9fc377b740 1 -- 172.21.15.107:0/3858426223 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b6217f83f0 con 0x55b621b7f580 2024-04-25T00:17:05.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.149+0000 7f9fa1ffb640 1 -- 172.21.15.107:0/3858426223 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b621af1bb0 con 0x55b621b7f580 2024-04-25T00:17:05.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.149+0000 7f9fa1ffb640 1 -- 172.21.15.107:0/3858426223 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f9fac031bd0 con 0x55b621b7f580 2024-04-25T00:17:05.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.149+0000 7f9fc377b740 10 client.15759 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:17:05.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.149+0000 7f9fc377b740 10 client.15759 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:17:05.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.149+0000 7f9fc377b740 20 client.15759 populate_metadata read hostname 'smithi107' 2024-04-25T00:17:05.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.149+0000 7f9fc377b740 1 --2- 172.21.15.107:0/3858426223 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55b621b9a0d0 0x55b621b9c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.149+0000 7f9fc377b740 4 client.15759 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:17:05.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.149+0000 7f9fc377b740 1 -- 172.21.15.107:0/3858426223 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b621aebf00 con 0x55b621b9a0d0 2024-04-25T00:17:05.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.150+0000 7f9fb3fff640 1 --2- 172.21.15.107:0/3858426223 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55b621b9a0d0 0x55b621b9c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.150+0000 7f9fa1ffb640 10 client.15759 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:17:05.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.150+0000 7f9fb3fff640 1 --2- 172.21.15.107:0/3858426223 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55b621b9a0d0 0x55b621b9c4c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d086954126b91 server_cookie=1dfc3d96be6e730f in_seq=0 out_seq=0 2024-04-25T00:17:05.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.151+0000 7f9fa1ffb640 1 -- 172.21.15.107:0/3858426223 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b621aebf00 con 0x55b621b9a0d0 2024-04-25T00:17:05.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.151+0000 7f9fa1ffb640 10 client.15759 handle_command_reply: tid=0 2024-04-25T00:17:05.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.151+0000 7f9fc377b740 1 client.15759 shutdown 2024-04-25T00:17:05.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.151+0000 7f9fc377b740 1 -- 172.21.15.107:0/3858426223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b7f580 msgr2=0x55b621b89f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.151+0000 7f9fc377b740 1 --2- 172.21.15.107:0/3858426223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b7f580 0x55b621b89f10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0ed270 tx=0x7f9fac0095e0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.151+0000 7f9fc377b740 1 -- 172.21.15.107:0/3858426223 shutdown_connections 2024-04-25T00:17:05.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.151+0000 7f9fc377b740 1 --2- 172.21.15.107:0/3858426223 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55b621b9a0d0 0x55b621b9c4c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.151+0000 7f9fc377b740 1 --2- 172.21.15.107:0/3858426223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b621b8a570 0x55b621b877a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.151+0000 7f9fc377b740 1 --2- 172.21.15.107:0/3858426223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b621b7fea0 0x55b621b87200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.151+0000 7f9fa1ffb640 0 client.15759 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:17:05.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.151+0000 7f9fc377b740 1 --2- 172.21.15.107:0/3858426223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b7f580 0x55b621b89f10 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.151+0000 7f9fc377b740 1 -- 172.21.15.107:0/3858426223 >> 172.21.15.107:0/3858426223 conn(0x55b621b7b8a0 msgr2=0x55b621b7ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.151+0000 7f9fc377b740 1 -- 172.21.15.107:0/3858426223 shutdown_connections 2024-04-25T00:17:05.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.151+0000 7f9fc377b740 1 -- 172.21.15.107:0/3858426223 wait complete. 2024-04-25T00:17:05.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.151+0000 7f9fc377b740 20 client.15759 trim_cache size 0 max 16384 2024-04-25T00:17:05.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.164+0000 7f9fc377b740 1 Processor -- start 2024-04-25T00:17:05.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.164+0000 7f9fc377b740 1 -- start start 2024-04-25T00:17:05.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.164+0000 7f9fc377b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b621b7f580 0x55b621b95e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.164+0000 7f9fc377b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b621b7fea0 0x55b621b7ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.164+0000 7f9fc377b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b8a570 0x55b621b89a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.164+0000 7f9fc377b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc1b13f0 con 0x55b621b8a570 2024-04-25T00:17:05.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.164+0000 7f9fc377b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc1aecc0 con 0x55b621b7fea0 2024-04-25T00:17:05.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.164+0000 7f9fc377b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc1acb10 con 0x55b621b7f580 2024-04-25T00:17:05.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.164+0000 7f9fbad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b8a570 0x55b621b89a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.164+0000 7f9fbad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b8a570 0x55b621b89a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37800/0 (socket says 172.21.15.107:37800) 2024-04-25T00:17:05.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.164+0000 7f9fbad9d640 1 -- 172.21.15.107:0/801567258 learned_addr learned my addr 172.21.15.107:0/801567258 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:05.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.164+0000 7f9fb3fff640 1 --2- 172.21.15.107:0/801567258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b621b7fea0 0x55b621b7ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.164+0000 7f9fbad9d640 1 -- 172.21.15.107:0/801567258 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b621b7f580 msgr2=0x55b621b95e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.164+0000 7f9fbad9d640 1 --2- 172.21.15.107:0/801567258 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b621b7f580 0x55b621b95e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.164+0000 7f9fbad9d640 1 -- 172.21.15.107:0/801567258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b621b7fea0 msgr2=0x55b621b7ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.164+0000 7f9fbad9d640 1 --2- 172.21.15.107:0/801567258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b621b7fea0 0x55b621b7ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.164+0000 7f9fbad9d640 1 -- 172.21.15.107:0/801567258 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b621b8ccd0 con 0x55b621b8a570 2024-04-25T00:17:05.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.164+0000 7f9fb3fff640 1 --2- 172.21.15.107:0/801567258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b621b7fea0 0x55b621b7ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:17:05.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.165+0000 7f9fbad9d640 1 --2- 172.21.15.107:0/801567258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b8a570 0x55b621b89a40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40095d0 tx=0x7f9fb401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76af7ba81b378dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:05.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.165+0000 7f9fa3fff640 1 -- 172.21.15.107:0/801567258 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb402b070 con 0x55b621b8a570 2024-04-25T00:17:05.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.165+0000 7f9fa3fff640 1 -- 172.21.15.107:0/801567258 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb401ddf0 con 0x55b621b8a570 2024-04-25T00:17:05.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.165+0000 7f9fa3fff640 1 -- 172.21.15.107:0/801567258 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4026bf0 con 0x55b621b8a570 2024-04-25T00:17:05.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.165+0000 7f9fc377b740 1 -- 172.21.15.107:0/801567258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b8a570 msgr2=0x55b621b89a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.165+0000 7f9fc377b740 1 --2- 172.21.15.107:0/801567258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b8a570 0x55b621b89a40 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40095d0 tx=0x7f9fb401b000 comp rx=0 tx=0).stop 2024-04-25T00:17:05.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.165+0000 7f9fc377b740 1 -- 172.21.15.107:0/801567258 shutdown_connections 2024-04-25T00:17:05.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.165+0000 7f9fc377b740 1 --2- 172.21.15.107:0/801567258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b8a570 0x55b621b89a40 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.165+0000 7f9fc377b740 1 --2- 172.21.15.107:0/801567258 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b621b7fea0 0x55b621b7ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.165+0000 7f9fc377b740 1 --2- 172.21.15.107:0/801567258 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b621b7f580 0x55b621b95e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.165+0000 7f9fc377b740 1 -- 172.21.15.107:0/801567258 >> 172.21.15.107:0/801567258 conn(0x55b621b7b8a0 msgr2=0x55b621b7bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.166+0000 7f9fc377b740 1 -- 172.21.15.107:0/801567258 shutdown_connections 2024-04-25T00:17:05.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.166+0000 7f9fc377b740 1 -- 172.21.15.107:0/801567258 wait complete. 2024-04-25T00:17:05.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.166+0000 7f9fc377b740 1 -- start start 2024-04-25T00:17:05.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.166+0000 7f9fc377b740 1 Processor -- start 2024-04-25T00:17:05.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.166+0000 7f9fc377b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b621b7f580 0x55b621bf8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.166+0000 7f9fc377b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b7fea0 0x55b621bf8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.166+0000 7f9fc377b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b621b8a570 0x55b621c035a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.166+0000 7f9fc377b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc1ad180 con 0x55b621b7fea0 2024-04-25T00:17:05.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.166+0000 7f9fc377b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc1ae5f0 con 0x55b621b7f580 2024-04-25T00:17:05.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.166+0000 7f9fc377b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc1ad850 con 0x55b621b8a570 2024-04-25T00:17:05.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.166+0000 7f9fc377b740 10 client.0 _ll_register_callbacks cb 0x7f9fb400e3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:05.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.166+0000 7f9fb3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b7fea0 0x55b621bf8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.166+0000 7f9fb3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b7fea0 0x55b621bf8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37810/0 (socket says 172.21.15.107:37810) 2024-04-25T00:17:05.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.166+0000 7f9fb3fff640 1 -- 172.21.15.107:0/3047730566 learned_addr learned my addr 172.21.15.107:0/3047730566 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:05.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.166+0000 7f9fba59c640 1 --2- 172.21.15.107:0/3047730566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b621b7f580 0x55b621bf8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.166+0000 7f9fb3fff640 1 -- 172.21.15.107:0/3047730566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b621b8a570 msgr2=0x55b621c035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.166+0000 7f9fb3fff640 1 --2- 172.21.15.107:0/3047730566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b621b8a570 0x55b621c035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.166+0000 7f9fb3fff640 1 -- 172.21.15.107:0/3047730566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b621b7f580 msgr2=0x55b621bf8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.166+0000 7f9fb3fff640 1 --2- 172.21.15.107:0/3047730566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b621b7f580 0x55b621bf8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.166+0000 7f9fb3fff640 1 -- 172.21.15.107:0/3047730566 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6218c27d0 con 0x55b621b7fea0 2024-04-25T00:17:05.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.167+0000 7f9fb3fff640 1 --2- 172.21.15.107:0/3047730566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b7fea0 0x55b621bf8a60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x55b621b7ca30 tx=0x7f9fa40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f94c2c8a0c8eb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:05.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.167+0000 7f9fa1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:17:05.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.167+0000 7f9fa1ffb640 1 -- 172.21.15.107:0/3047730566 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4023060 con 0x55b621b7fea0 2024-04-25T00:17:05.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.167+0000 7f9fa1ffb640 1 -- 172.21.15.107:0/3047730566 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f9fa401d630 con 0x55b621b7fea0 2024-04-25T00:17:05.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.167+0000 7f9fc377b740 1 -- 172.21.15.107:0/3047730566 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fb401ddf0 con 0x55b621b7fea0 2024-04-25T00:17:05.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.167+0000 7f9fa1ffb640 1 -- 172.21.15.107:0/3047730566 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f9fa402a050 con 0x55b621b7fea0 2024-04-25T00:17:05.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.167+0000 7f9fc377b740 10 client.15765 fetch_fsmap learned FSMap version 24 2024-04-25T00:17:05.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.167+0000 7f9fc377b740 1 -- 172.21.15.107:0/3047730566 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b621c04aa0 con 0x55b621b7fea0 2024-04-25T00:17:05.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.167+0000 7f9fa1ffb640 1 -- 172.21.15.107:0/3047730566 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb401ddf0 con 0x55b621b7fea0 2024-04-25T00:17:05.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.167+0000 7f9fa1ffb640 1 -- 172.21.15.107:0/3047730566 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f9fa4004aa0 con 0x55b621b7fea0 2024-04-25T00:17:05.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.167+0000 7f9fc377b740 10 client.15765 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:17:05.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.167+0000 7f9fc377b740 10 client.15765 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:17:05.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.167+0000 7f9fc377b740 20 client.15765 populate_metadata read hostname 'smithi107' 2024-04-25T00:17:05.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.167+0000 7f9fc377b740 1 --2- 172.21.15.107:0/3047730566 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55b621c05200 0x55b621c07620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.167+0000 7f9fc377b740 4 client.15765 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:17:05.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.167+0000 7f9fc377b740 1 -- 172.21.15.107:0/3047730566 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b621a69820 con 0x55b621c05200 2024-04-25T00:17:05.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.168+0000 7f9fba59c640 1 --2- 172.21.15.107:0/3047730566 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55b621c05200 0x55b621c07620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.168+0000 7f9fa1ffb640 10 client.15765 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:17:05.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.168+0000 7f9fba59c640 1 --2- 172.21.15.107:0/3047730566 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55b621c05200 0x55b621c07620 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f84a9157edf1dc42 server_cookie=5226b4b88dac01b2 in_seq=0 out_seq=0 2024-04-25T00:17:05.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.168+0000 7f9fa1ffb640 1 -- 172.21.15.107:0/3047730566 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b621a69820 con 0x55b621c05200 2024-04-25T00:17:05.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.168+0000 7f9fa1ffb640 10 client.15765 handle_command_reply: tid=0 2024-04-25T00:17:05.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.168+0000 7f9fc377b740 1 client.15765 shutdown 2024-04-25T00:17:05.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.168+0000 7f9fc377b740 1 -- 172.21.15.107:0/3047730566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b7fea0 msgr2=0x55b621bf8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.168+0000 7f9fc377b740 1 --2- 172.21.15.107:0/3047730566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b7fea0 0x55b621bf8a60 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x55b621b7ca30 tx=0x7f9fa40039c0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.168+0000 7f9fc377b740 1 -- 172.21.15.107:0/3047730566 shutdown_connections 2024-04-25T00:17:05.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.168+0000 7f9fc377b740 1 --2- 172.21.15.107:0/3047730566 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55b621c05200 0x55b621c07620 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.168+0000 7f9fc377b740 1 --2- 172.21.15.107:0/3047730566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b621b8a570 0x55b621c035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.168+0000 7f9fc377b740 1 --2- 172.21.15.107:0/3047730566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b621b7fea0 0x55b621bf8a60 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.168+0000 7f9fa1ffb640 0 client.15765 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:17:05.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.168+0000 7f9fc377b740 1 --2- 172.21.15.107:0/3047730566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b621b7f580 0x55b621bf8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.168+0000 7f9fc377b740 1 -- 172.21.15.107:0/3047730566 >> 172.21.15.107:0/3047730566 conn(0x55b621b7b8a0 msgr2=0x55b621b7ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.168+0000 7f9fc377b740 1 -- 172.21.15.107:0/3047730566 shutdown_connections 2024-04-25T00:17:05.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.168+0000 7f9fc377b740 1 -- 172.21.15.107:0/3047730566 wait complete. 2024-04-25T00:17:05.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.169+0000 7f9fc377b740 20 client.15765 trim_cache size 0 max 16384 2024-04-25T00:17:05.179 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:17:05.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.169+0000 7f9fc0a6b640 1 -- 172.21.15.107:0/4223756673 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f9fa80a2eb0 msgr2=0x7f9fa80a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.169+0000 7f9fc0a6b640 1 --2- 172.21.15.107:0/4223756673 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f9fa80a2eb0 0x7f9fa80a5370 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400eca0 tx=0x7f9fb4009ad0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.169+0000 7f9fc0a6b640 1 -- 172.21.15.107:0/4223756673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fbc100610 msgr2=0x7f9fbc198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.169+0000 7f9fc0a6b640 1 --2- 172.21.15.107:0/4223756673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fbc100610 0x7f9fbc198010 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9fac005940 tx=0x7f9fac0042f0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.169+0000 7f9fc0a6b640 1 -- 172.21.15.107:0/4223756673 shutdown_connections 2024-04-25T00:17:05.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.169+0000 7f9fc0a6b640 1 --2- 172.21.15.107:0/4223756673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9fbc1066b0 0x7f9fbc19c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.169+0000 7f9fc0a6b640 1 --2- 172.21.15.107:0/4223756673 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f9fa80a2eb0 0x7f9fa80a5370 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.169+0000 7f9fc0a6b640 1 --2- 172.21.15.107:0/4223756673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fbc100fd0 0x7f9fbc198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.169+0000 7f9fc0a6b640 1 --2- 172.21.15.107:0/4223756673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fbc100610 0x7f9fbc198010 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.169+0000 7f9fc0a6b640 1 -- 172.21.15.107:0/4223756673 >> 172.21.15.107:0/4223756673 conn(0x7f9fbc077740 msgr2=0x7f9fbc0ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.169+0000 7f9fc0a6b640 1 -- 172.21.15.107:0/4223756673 shutdown_connections 2024-04-25T00:17:05.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.169+0000 7f9fc0a6b640 1 -- 172.21.15.107:0/4223756673 wait complete. 2024-04-25T00:17:05.183 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:17:05.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.284+0000 7f264dba2640 1 -- 172.21.15.107:0/2009780575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2648100fd0 msgr2=0x7f264810d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.284+0000 7f264dba2640 1 --2- 172.21.15.107:0/2009780575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2648100fd0 0x7f264810d4e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2640009cd0 tx=0x7f264002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.284+0000 7f264dba2640 1 -- 172.21.15.107:0/2009780575 shutdown_connections 2024-04-25T00:17:05.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.284+0000 7f264dba2640 1 --2- 172.21.15.107:0/2009780575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2648100fd0 0x7f264810d4e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.284+0000 7f264dba2640 1 --2- 172.21.15.107:0/2009780575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2648100610 0x7f2648100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.284+0000 7f264dba2640 1 --2- 172.21.15.107:0/2009780575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f26481066b0 0x7f2648106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.284+0000 7f264dba2640 1 -- 172.21.15.107:0/2009780575 >> 172.21.15.107:0/2009780575 conn(0x7f2648077740 msgr2=0x7f2648077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.284+0000 7f264dba2640 1 -- 172.21.15.107:0/2009780575 shutdown_connections 2024-04-25T00:17:05.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.284+0000 7f264dba2640 1 -- 172.21.15.107:0/2009780575 wait complete. 2024-04-25T00:17:05.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.284+0000 7f264dba2640 1 Processor -- start 2024-04-25T00:17:05.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.284+0000 7f264dba2640 1 -- start start 2024-04-25T00:17:05.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.284+0000 7f264dba2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2648100610 0x7f2648198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.284+0000 7f264dba2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2648100fd0 0x7f26481993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.284+0000 7f264dba2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26481066b0 0x7f26481a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.284+0000 7f264dba2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2648110a70 con 0x7f2648100610 2024-04-25T00:17:05.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.284+0000 7f264dba2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f26481108f0 con 0x7f2648100fd0 2024-04-25T00:17:05.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.284+0000 7f264dba2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2648110bf0 con 0x7f26481066b0 2024-04-25T00:17:05.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.285+0000 7f26477fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2648100610 0x7f2648198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.285+0000 7f26477fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2648100610 0x7f2648198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37840/0 (socket says 172.21.15.107:37840) 2024-04-25T00:17:05.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.285+0000 7f26477fe640 1 -- 172.21.15.107:0/662957132 learned_addr learned my addr 172.21.15.107:0/662957132 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:05.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.285+0000 7f2647fff640 1 --2- 172.21.15.107:0/662957132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26481066b0 0x7f26481a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.285+0000 7f263ffff640 1 --2- 172.21.15.107:0/662957132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2648100fd0 0x7f26481993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.285+0000 7f26477fe640 1 -- 172.21.15.107:0/662957132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26481066b0 msgr2=0x7f26481a3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.285+0000 7f26477fe640 1 --2- 172.21.15.107:0/662957132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26481066b0 0x7f26481a3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.285+0000 7f26477fe640 1 -- 172.21.15.107:0/662957132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2648100fd0 msgr2=0x7f26481993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.285+0000 7f26477fe640 1 --2- 172.21.15.107:0/662957132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2648100fd0 0x7f26481993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.285+0000 7f26477fe640 1 -- 172.21.15.107:0/662957132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26481a4430 con 0x7f2648100610 2024-04-25T00:17:05.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.285+0000 7f26477fe640 1 --2- 172.21.15.107:0/662957132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2648100610 0x7f2648198e70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f263800b450 tx=0x7f263800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:05.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.285+0000 7f26457fa640 1 -- 172.21.15.107:0/662957132 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638015020 con 0x7f2648100610 2024-04-25T00:17:05.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.285+0000 7f26457fa640 1 -- 172.21.15.107:0/662957132 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2638011d70 con 0x7f2648100610 2024-04-25T00:17:05.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.285+0000 7f264dba2640 1 -- 172.21.15.107:0/662957132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26481a4600 con 0x7f2648100610 2024-04-25T00:17:05.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.285+0000 7f26457fa640 1 -- 172.21.15.107:0/662957132 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263800f340 con 0x7f2648100610 2024-04-25T00:17:05.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.285+0000 7f264dba2640 1 -- 172.21.15.107:0/662957132 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26481a4b20 con 0x7f2648100610 2024-04-25T00:17:05.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.288+0000 7f26457fa640 1 -- 172.21.15.107:0/662957132 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f2638018020 con 0x7f2648100610 2024-04-25T00:17:05.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.288+0000 7f2650994740 1 Processor -- start 2024-04-25T00:17:05.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.288+0000 7f2650994740 1 -- start start 2024-04-25T00:17:05.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.288+0000 7f2650994740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559a79a58580 0x559a79a58960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.288+0000 7f2650994740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a79a58ea0 0x559a79a63030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.288+0000 7f2650994740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a63570 0x559a79a65960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.288+0000 7f2650994740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559a799bd670 con 0x559a79a63570 2024-04-25T00:17:05.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.288+0000 7f2650994740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559a79965970 con 0x559a79a58580 2024-04-25T00:17:05.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.288+0000 7f2650994740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559a799b9630 con 0x559a79a58ea0 2024-04-25T00:17:05.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.288+0000 7f26457fa640 1 --2- 172.21.15.107:0/662957132 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f26340a2eb0 0x7f26340a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.289+0000 7f263ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a63570 0x559a79a65960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.289+0000 7f263ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a63570 0x559a79a65960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37848/0 (socket says 172.21.15.107:37848) 2024-04-25T00:17:05.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.289+0000 7f263ffff640 1 -- 172.21.15.107:0/1479990816 learned_addr learned my addr 172.21.15.107:0/1479990816 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:05.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.289+0000 7f2647fff640 1 --2- 172.21.15.107:0/662957132 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f26340a2eb0 0x7f26340a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.289+0000 7f26457fa640 1 -- 172.21.15.107:0/662957132 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f26380cf4a0 con 0x7f2648100610 2024-04-25T00:17:05.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.289+0000 7f26477fe640 1 --2- 172.21.15.107:0/1479990816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a79a58ea0 0x559a79a63030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.289+0000 7f2647fff640 1 --2- 172.21.15.107:0/1479990816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559a79a58580 0x559a79a58960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.289+0000 7f263ffff640 1 -- 172.21.15.107:0/1479990816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a79a58ea0 msgr2=0x559a79a63030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.289+0000 7f263ffff640 1 --2- 172.21.15.107:0/1479990816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a79a58ea0 0x559a79a63030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.289+0000 7f263ffff640 1 -- 172.21.15.107:0/1479990816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559a79a58580 msgr2=0x559a79a58960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.289+0000 7f263ffff640 1 --2- 172.21.15.107:0/1479990816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559a79a58580 0x559a79a58960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.289+0000 7f263ffff640 1 -- 172.21.15.107:0/1479990816 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a79989400 con 0x559a79a63570 2024-04-25T00:17:05.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.289+0000 7f2647fff640 1 --2- 172.21.15.107:0/1479990816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559a79a58580 0x559a79a58960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:17:05.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.289+0000 7f263ffff640 1 --2- 172.21.15.107:0/1479990816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a63570 0x559a79a65960 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f263000ba70 tx=0x7f263000bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c883b8295de22a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:05.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.289+0000 7f2647fff640 1 --2- 172.21.15.107:0/662957132 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f26340a2eb0 0x7f26340a5370 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2640006010 tx=0x7f2640009900 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:05.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.289+0000 7f263dffb640 1 -- 172.21.15.107:0/1479990816 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2630017420 con 0x559a79a63570 2024-04-25T00:17:05.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.289+0000 7f263dffb640 1 -- 172.21.15.107:0/1479990816 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2630017aa0 con 0x559a79a63570 2024-04-25T00:17:05.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.289+0000 7f263dffb640 1 -- 172.21.15.107:0/1479990816 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263000fa30 con 0x559a79a63570 2024-04-25T00:17:05.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.290+0000 7f2650994740 1 -- 172.21.15.107:0/1479990816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a63570 msgr2=0x559a79a65960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.290+0000 7f2650994740 1 --2- 172.21.15.107:0/1479990816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a63570 0x559a79a65960 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f263000ba70 tx=0x7f263000bf30 comp rx=0 tx=0).stop 2024-04-25T00:17:05.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.290+0000 7f2650994740 1 -- 172.21.15.107:0/1479990816 shutdown_connections 2024-04-25T00:17:05.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.290+0000 7f2650994740 1 --2- 172.21.15.107:0/1479990816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a63570 0x559a79a65960 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.290+0000 7f2650994740 1 --2- 172.21.15.107:0/1479990816 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a79a58ea0 0x559a79a63030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.290+0000 7f2650994740 1 --2- 172.21.15.107:0/1479990816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559a79a58580 0x559a79a58960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.290+0000 7f2650994740 1 -- 172.21.15.107:0/1479990816 >> 172.21.15.107:0/1479990816 conn(0x559a79a548a0 msgr2=0x559a79a54c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.290+0000 7f2650994740 1 -- 172.21.15.107:0/1479990816 shutdown_connections 2024-04-25T00:17:05.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.290+0000 7f2650994740 1 -- 172.21.15.107:0/1479990816 wait complete. 2024-04-25T00:17:05.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.290+0000 7f2650994740 1 -- start start 2024-04-25T00:17:05.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.290+0000 7f2650994740 1 Processor -- start 2024-04-25T00:17:05.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f2650994740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559a79a58580 0x559a79a62ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f2650994740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a58ea0 0x559a79a601b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f2650994740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a79a63570 0x559a79a60710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f2650994740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559a79a66360 con 0x559a79a58ea0 2024-04-25T00:17:05.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f2650994740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559a79979050 con 0x559a79a58580 2024-04-25T00:17:05.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f2650994740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559a79a664e0 con 0x559a79a63570 2024-04-25T00:17:05.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f2650994740 10 client.0 _ll_register_callbacks cb 0x559a796d0be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:05.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f263ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a58ea0 0x559a79a601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f263ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a58ea0 0x559a79a601b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37856/0 (socket says 172.21.15.107:37856) 2024-04-25T00:17:05.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f263ffff640 1 -- 172.21.15.107:0/2223269001 learned_addr learned my addr 172.21.15.107:0/2223269001 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:05.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f2647fff640 1 --2- 172.21.15.107:0/2223269001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a79a63570 0x559a79a60710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f26477fe640 1 --2- 172.21.15.107:0/2223269001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559a79a58580 0x559a79a62ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f263ffff640 1 -- 172.21.15.107:0/2223269001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a79a63570 msgr2=0x559a79a60710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f263ffff640 1 --2- 172.21.15.107:0/2223269001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a79a63570 0x559a79a60710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f263ffff640 1 -- 172.21.15.107:0/2223269001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559a79a58580 msgr2=0x559a79a62ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f263ffff640 1 --2- 172.21.15.107:0/2223269001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559a79a58580 0x559a79a62ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f263ffff640 1 -- 172.21.15.107:0/2223269001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a7994adc0 con 0x559a79a58ea0 2024-04-25T00:17:05.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f263ffff640 1 --2- 172.21.15.107:0/2223269001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a58ea0 0x559a79a601b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f263000f670 tx=0x7f2630017c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c62c2c5de6a519 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:05.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f2626ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:17:05.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f2626ffd640 1 -- 172.21.15.107:0/2223269001 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2630025070 con 0x559a79a58ea0 2024-04-25T00:17:05.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f2650994740 1 -- 172.21.15.107:0/2223269001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2630017aa0 con 0x559a79a58ea0 2024-04-25T00:17:05.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f2626ffd640 1 -- 172.21.15.107:0/2223269001 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f2630017830 con 0x559a79a58ea0 2024-04-25T00:17:05.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f2626ffd640 1 -- 172.21.15.107:0/2223269001 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f2630031050 con 0x559a79a58ea0 2024-04-25T00:17:05.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f2650994740 10 client.15777 fetch_fsmap learned FSMap version 24 2024-04-25T00:17:05.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.291+0000 7f2650994740 1 -- 172.21.15.107:0/2223269001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559a796d13f0 con 0x559a79a58ea0 2024-04-25T00:17:05.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.292+0000 7f2626ffd640 1 -- 172.21.15.107:0/2223269001 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2630017aa0 con 0x559a79a58ea0 2024-04-25T00:17:05.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.292+0000 7f2626ffd640 1 -- 172.21.15.107:0/2223269001 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f263000fcd0 con 0x559a79a58ea0 2024-04-25T00:17:05.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.292+0000 7f2650994740 10 client.15777 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:17:05.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.292+0000 7f2650994740 10 client.15777 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:17:05.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.292+0000 7f2650994740 20 client.15777 populate_metadata read hostname 'smithi107' 2024-04-25T00:17:05.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.292+0000 7f2650994740 1 --2- 172.21.15.107:0/2223269001 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x559a79a73480 0x559a79a75870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.292+0000 7f2650994740 4 client.15777 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:17:05.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.292+0000 7f2650994740 1 -- 172.21.15.107:0/2223269001 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a799cabb0 con 0x559a79a73480 2024-04-25T00:17:05.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.292+0000 7f26477fe640 1 --2- 172.21.15.107:0/2223269001 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x559a79a73480 0x559a79a75870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.293+0000 7f26477fe640 1 --2- 172.21.15.107:0/2223269001 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x559a79a73480 0x559a79a75870 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a6afe4c8b0ed73a0 server_cookie=14f23f70047cfd83 in_seq=0 out_seq=0 2024-04-25T00:17:05.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.293+0000 7f2626ffd640 10 client.15777 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:17:05.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.293+0000 7f2626ffd640 1 -- 172.21.15.107:0/2223269001 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a799cabb0 con 0x559a79a73480 2024-04-25T00:17:05.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.293+0000 7f2626ffd640 10 client.15777 handle_command_reply: tid=0 2024-04-25T00:17:05.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.293+0000 7f2650994740 1 client.15777 shutdown 2024-04-25T00:17:05.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.294+0000 7f2650994740 1 -- 172.21.15.107:0/2223269001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a58ea0 msgr2=0x559a79a601b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.294+0000 7f2650994740 1 --2- 172.21.15.107:0/2223269001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a58ea0 0x559a79a601b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f263000f670 tx=0x7f2630017c50 comp rx=0 tx=0).stop 2024-04-25T00:17:05.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.294+0000 7f2650994740 1 -- 172.21.15.107:0/2223269001 shutdown_connections 2024-04-25T00:17:05.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.294+0000 7f2650994740 1 --2- 172.21.15.107:0/2223269001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a79a63570 0x559a79a60710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.294+0000 7f2650994740 1 --2- 172.21.15.107:0/2223269001 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x559a79a73480 0x559a79a75870 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.294+0000 7f2650994740 1 --2- 172.21.15.107:0/2223269001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a58ea0 0x559a79a601b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.294+0000 7f2650994740 1 --2- 172.21.15.107:0/2223269001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559a79a58580 0x559a79a62ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.294+0000 7f2626ffd640 0 client.15777 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:17:05.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.294+0000 7f2650994740 1 -- 172.21.15.107:0/2223269001 >> 172.21.15.107:0/2223269001 conn(0x559a79a548a0 msgr2=0x559a79a5a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.294+0000 7f2650994740 1 -- 172.21.15.107:0/2223269001 shutdown_connections 2024-04-25T00:17:05.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.294+0000 7f2650994740 1 -- 172.21.15.107:0/2223269001 wait complete. 2024-04-25T00:17:05.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.294+0000 7f2650994740 20 client.15777 trim_cache size 0 max 16384 2024-04-25T00:17:05.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.307+0000 7f2650994740 1 Processor -- start 2024-04-25T00:17:05.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.307+0000 7f2650994740 1 -- start start 2024-04-25T00:17:05.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.307+0000 7f2650994740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a58580 0x559a79a6f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.307+0000 7f2650994740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559a79a58ea0 0x559a79a6bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.307+0000 7f2650994740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a79a63570 0x559a79a7af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.307+0000 7f2650994740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f26481b1490 con 0x559a79a58580 2024-04-25T00:17:05.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.307+0000 7f2650994740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f26481aed60 con 0x559a79a58ea0 2024-04-25T00:17:05.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.307+0000 7f2650994740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f26481acbb0 con 0x559a79a63570 2024-04-25T00:17:05.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.307+0000 7f26477fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a58580 0x559a79a6f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.308+0000 7f26477fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a58580 0x559a79a6f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37870/0 (socket says 172.21.15.107:37870) 2024-04-25T00:17:05.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.308+0000 7f26477fe640 1 -- 172.21.15.107:0/1202630970 learned_addr learned my addr 172.21.15.107:0/1202630970 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:05.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.308+0000 7f2647fff640 1 --2- 172.21.15.107:0/1202630970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a79a63570 0x559a79a7af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.308+0000 7f263ffff640 1 --2- 172.21.15.107:0/1202630970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559a79a58ea0 0x559a79a6bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.308+0000 7f26477fe640 1 -- 172.21.15.107:0/1202630970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a79a63570 msgr2=0x559a79a7af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.308+0000 7f26477fe640 1 --2- 172.21.15.107:0/1202630970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a79a63570 0x559a79a7af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.308+0000 7f26477fe640 1 -- 172.21.15.107:0/1202630970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559a79a58ea0 msgr2=0x559a79a6bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.308+0000 7f26477fe640 1 --2- 172.21.15.107:0/1202630970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559a79a58ea0 0x559a79a6bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.308+0000 7f26477fe640 1 -- 172.21.15.107:0/1202630970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a7979b7d0 con 0x559a79a58580 2024-04-25T00:17:05.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.308+0000 7f263ffff640 1 --2- 172.21.15.107:0/1202630970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559a79a58ea0 0x559a79a6bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:17:05.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.308+0000 7f26477fe640 1 --2- 172.21.15.107:0/1202630970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a58580 0x559a79a6f1b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2638009fd0 tx=0x7f26380970e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618b4bf47a23be7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:05.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.308+0000 7f263d7fa640 1 -- 172.21.15.107:0/1202630970 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26380b4070 con 0x559a79a58580 2024-04-25T00:17:05.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.308+0000 7f263d7fa640 1 -- 172.21.15.107:0/1202630970 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2638080e50 con 0x559a79a58580 2024-04-25T00:17:05.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.308+0000 7f263d7fa640 1 -- 172.21.15.107:0/1202630970 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638084070 con 0x559a79a58580 2024-04-25T00:17:05.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.309+0000 7f2650994740 1 -- 172.21.15.107:0/1202630970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a58580 msgr2=0x559a79a6f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.309+0000 7f2650994740 1 --2- 172.21.15.107:0/1202630970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a58580 0x559a79a6f1b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2638009fd0 tx=0x7f26380970e0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.309+0000 7f2650994740 1 -- 172.21.15.107:0/1202630970 shutdown_connections 2024-04-25T00:17:05.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.309+0000 7f2650994740 1 --2- 172.21.15.107:0/1202630970 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a79a63570 0x559a79a7af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.309+0000 7f2650994740 1 --2- 172.21.15.107:0/1202630970 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559a79a58ea0 0x559a79a6bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.309+0000 7f2650994740 1 --2- 172.21.15.107:0/1202630970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a58580 0x559a79a6f1b0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.309+0000 7f2650994740 1 -- 172.21.15.107:0/1202630970 >> 172.21.15.107:0/1202630970 conn(0x559a79a548a0 msgr2=0x559a79a6eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.309+0000 7f2650994740 1 -- 172.21.15.107:0/1202630970 shutdown_connections 2024-04-25T00:17:05.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.309+0000 7f2650994740 1 -- 172.21.15.107:0/1202630970 wait complete. 2024-04-25T00:17:05.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.309+0000 7f2650994740 1 -- start start 2024-04-25T00:17:05.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.309+0000 7f2650994740 1 Processor -- start 2024-04-25T00:17:05.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.309+0000 7f2650994740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a79a58580 0x559a79ad6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.309+0000 7f2650994740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559a79a58ea0 0x559a79ad6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.309+0000 7f2650994740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a63570 0x559a79ad0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.309+0000 7f2650994740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f26481ad220 con 0x559a79a63570 2024-04-25T00:17:05.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.309+0000 7f2650994740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f26481ae690 con 0x559a79a58ea0 2024-04-25T00:17:05.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.309+0000 7f2650994740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f26481ad8f0 con 0x559a79a58580 2024-04-25T00:17:05.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.310+0000 7f2650994740 10 client.0 _ll_register_callbacks cb 0x7f263000b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:05.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.310+0000 7f2647fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a63570 0x559a79ad0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.310+0000 7f2647fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a63570 0x559a79ad0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37884/0 (socket says 172.21.15.107:37884) 2024-04-25T00:17:05.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.310+0000 7f2647fff640 1 -- 172.21.15.107:0/1697690472 learned_addr learned my addr 172.21.15.107:0/1697690472 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:05.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.310+0000 7f26477fe640 1 --2- 172.21.15.107:0/1697690472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a79a58580 0x559a79ad6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.310+0000 7f263ffff640 1 --2- 172.21.15.107:0/1697690472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559a79a58ea0 0x559a79ad6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.310+0000 7f2647fff640 1 -- 172.21.15.107:0/1697690472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a79a58580 msgr2=0x559a79ad6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.310+0000 7f2647fff640 1 --2- 172.21.15.107:0/1697690472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a79a58580 0x559a79ad6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.310+0000 7f2647fff640 1 -- 172.21.15.107:0/1697690472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559a79a58ea0 msgr2=0x559a79ad6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.310+0000 7f2647fff640 1 --2- 172.21.15.107:0/1697690472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559a79a58ea0 0x559a79ad6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.310+0000 7f2647fff640 1 -- 172.21.15.107:0/1697690472 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a79ad1000 con 0x559a79a63570 2024-04-25T00:17:05.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.310+0000 7f2647fff640 1 --2- 172.21.15.107:0/1697690472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a63570 0x559a79ad0910 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2640009cd0 tx=0x7f26400095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172fbe3fa227d64a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:05.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.310+0000 7f2626ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:17:05.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.310+0000 7f2626ffd640 1 -- 172.21.15.107:0/1697690472 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640051030 con 0x559a79a63570 2024-04-25T00:17:05.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.310+0000 7f2626ffd640 1 -- 172.21.15.107:0/1697690472 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f26400047e0 con 0x559a79a63570 2024-04-25T00:17:05.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.310+0000 7f2650994740 1 -- 172.21.15.107:0/1697690472 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2638080e50 con 0x559a79a63570 2024-04-25T00:17:05.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.310+0000 7f2626ffd640 1 -- 172.21.15.107:0/1697690472 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f26400305a0 con 0x559a79a63570 2024-04-25T00:17:05.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.310+0000 7f2650994740 10 client.15783 fetch_fsmap learned FSMap version 24 2024-04-25T00:17:05.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.310+0000 7f2650994740 1 -- 172.21.15.107:0/1697690472 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559a79addc70 con 0x559a79a63570 2024-04-25T00:17:05.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.311+0000 7f2626ffd640 1 -- 172.21.15.107:0/1697690472 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2638080e50 con 0x559a79a63570 2024-04-25T00:17:05.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.311+0000 7f2626ffd640 1 -- 172.21.15.107:0/1697690472 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f2640030070 con 0x559a79a63570 2024-04-25T00:17:05.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.311+0000 7f2650994740 10 client.15783 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:17:05.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.311+0000 7f2650994740 10 client.15783 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:17:05.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.311+0000 7f2650994740 20 client.15783 populate_metadata read hostname 'smithi107' 2024-04-25T00:17:05.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.311+0000 7f2650994740 1 --2- 172.21.15.107:0/1697690472 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x559a79ade390 0x559a79ae07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.311+0000 7f2650994740 4 client.15783 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:17:05.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.311+0000 7f2650994740 1 -- 172.21.15.107:0/1697690472 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a799c4f00 con 0x559a79ade390 2024-04-25T00:17:05.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.311+0000 7f26477fe640 1 --2- 172.21.15.107:0/1697690472 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x559a79ade390 0x559a79ae07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.311+0000 7f2626ffd640 10 client.15783 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:17:05.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.311+0000 7f26477fe640 1 --2- 172.21.15.107:0/1697690472 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x559a79ade390 0x559a79ae07b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9e0c2feaec0d8897 server_cookie=dd369a89e2e0bd4a in_seq=0 out_seq=0 2024-04-25T00:17:05.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.312+0000 7f2626ffd640 1 -- 172.21.15.107:0/1697690472 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a799c4f00 con 0x559a79ade390 2024-04-25T00:17:05.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.312+0000 7f2626ffd640 10 client.15783 handle_command_reply: tid=0 2024-04-25T00:17:05.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.312+0000 7f2650994740 1 client.15783 shutdown 2024-04-25T00:17:05.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.312+0000 7f2650994740 1 -- 172.21.15.107:0/1697690472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a63570 msgr2=0x559a79ad0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.312+0000 7f2650994740 1 --2- 172.21.15.107:0/1697690472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a63570 0x559a79ad0910 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2640009cd0 tx=0x7f26400095c0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.312+0000 7f2650994740 1 -- 172.21.15.107:0/1697690472 shutdown_connections 2024-04-25T00:17:05.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.312+0000 7f2650994740 1 --2- 172.21.15.107:0/1697690472 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x559a79ade390 0x559a79ae07b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.312+0000 7f2650994740 1 --2- 172.21.15.107:0/1697690472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a79a63570 0x559a79ad0910 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.312+0000 7f2626ffd640 0 client.15783 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:17:05.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.312+0000 7f2650994740 1 --2- 172.21.15.107:0/1697690472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559a79a58ea0 0x559a79ad6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.312+0000 7f2650994740 1 --2- 172.21.15.107:0/1697690472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559a79a58580 0x559a79ad6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.312+0000 7f2650994740 1 -- 172.21.15.107:0/1697690472 >> 172.21.15.107:0/1697690472 conn(0x559a79a548a0 msgr2=0x559a79a6ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.312+0000 7f2650994740 1 -- 172.21.15.107:0/1697690472 shutdown_connections 2024-04-25T00:17:05.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.312+0000 7f2650994740 1 -- 172.21.15.107:0/1697690472 wait complete. 2024-04-25T00:17:05.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.312+0000 7f2650994740 20 client.15783 trim_cache size 0 max 16384 2024-04-25T00:17:05.320 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:17:05.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.312+0000 7f264dba2640 1 -- 172.21.15.107:0/662957132 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f26340a2eb0 msgr2=0x7f26340a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.312+0000 7f264dba2640 1 --2- 172.21.15.107:0/662957132 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f26340a2eb0 0x7f26340a5370 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2640006010 tx=0x7f2640009900 comp rx=0 tx=0).stop 2024-04-25T00:17:05.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.312+0000 7f264dba2640 1 -- 172.21.15.107:0/662957132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2648100610 msgr2=0x7f2648198e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.312+0000 7f264dba2640 1 --2- 172.21.15.107:0/662957132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2648100610 0x7f2648198e70 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f263800b450 tx=0x7f263800b910 comp rx=0 tx=0).stop 2024-04-25T00:17:05.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.312+0000 7f264dba2640 1 -- 172.21.15.107:0/662957132 shutdown_connections 2024-04-25T00:17:05.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.312+0000 7f264dba2640 1 --2- 172.21.15.107:0/662957132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f26481066b0 0x7f26481a3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.312+0000 7f264dba2640 1 --2- 172.21.15.107:0/662957132 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f26340a2eb0 0x7f26340a5370 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.312+0000 7f264dba2640 1 --2- 172.21.15.107:0/662957132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2648100fd0 0x7f26481993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.312+0000 7f264dba2640 1 --2- 172.21.15.107:0/662957132 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2648100610 0x7f2648198e70 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.313+0000 7f264dba2640 1 -- 172.21.15.107:0/662957132 >> 172.21.15.107:0/662957132 conn(0x7f2648077740 msgr2=0x7f26480ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.313+0000 7f264dba2640 1 -- 172.21.15.107:0/662957132 shutdown_connections 2024-04-25T00:17:05.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.313+0000 7f264dba2640 1 -- 172.21.15.107:0/662957132 wait complete. 2024-04-25T00:17:05.324 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:17:05.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.426+0000 7fd8616f5640 1 -- 172.21.15.107:0/176371130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd85c100610 msgr2=0x7fd85c100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.426+0000 7fd8616f5640 1 --2- 172.21.15.107:0/176371130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd85c100610 0x7fd85c100a90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd850009930 tx=0x7fd85002a420 comp rx=0 tx=0).stop 2024-04-25T00:17:05.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.426+0000 7fd8616f5640 1 -- 172.21.15.107:0/176371130 shutdown_connections 2024-04-25T00:17:05.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.426+0000 7fd8616f5640 1 --2- 172.21.15.107:0/176371130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd85c100fd0 0x7fd85c10d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.426+0000 7fd8616f5640 1 --2- 172.21.15.107:0/176371130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd85c100610 0x7fd85c100a90 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.426+0000 7fd8616f5640 1 --2- 172.21.15.107:0/176371130 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd85c1066b0 0x7fd85c106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.426+0000 7fd8616f5640 1 -- 172.21.15.107:0/176371130 >> 172.21.15.107:0/176371130 conn(0x7fd85c077740 msgr2=0x7fd85c077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.426+0000 7fd8616f5640 1 -- 172.21.15.107:0/176371130 shutdown_connections 2024-04-25T00:17:05.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.426+0000 7fd8616f5640 1 -- 172.21.15.107:0/176371130 wait complete. 2024-04-25T00:17:05.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.427+0000 7fd8616f5640 1 Processor -- start 2024-04-25T00:17:05.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.427+0000 7fd8616f5640 1 -- start start 2024-04-25T00:17:05.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.427+0000 7fd8616f5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd85c100610 0x7fd85c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.427+0000 7fd8616f5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd85c100fd0 0x7fd85c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.427+0000 7fd8616f5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd85c1066b0 0x7fd85c19c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.427+0000 7fd8616f5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c1109f0 con 0x7fd85c100fd0 2024-04-25T00:17:05.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.427+0000 7fd8616f5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c110870 con 0x7fd85c100610 2024-04-25T00:17:05.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.427+0000 7fd8616f5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c110b70 con 0x7fd85c1066b0 2024-04-25T00:17:05.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.427+0000 7fd85a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd85c100fd0 0x7fd85c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.427+0000 7fd85a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd85c100fd0 0x7fd85c198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37908/0 (socket says 172.21.15.107:37908) 2024-04-25T00:17:05.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.427+0000 7fd85a7fc640 1 -- 172.21.15.107:0/1999473888 learned_addr learned my addr 172.21.15.107:0/1999473888 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:05.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.427+0000 7fd85affd640 1 --2- 172.21.15.107:0/1999473888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd85c100610 0x7fd85c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.427+0000 7fd85a7fc640 1 -- 172.21.15.107:0/1999473888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd85c1066b0 msgr2=0x7fd85c19c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.427+0000 7fd85a7fc640 1 --2- 172.21.15.107:0/1999473888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd85c1066b0 0x7fd85c19c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.427+0000 7fd85a7fc640 1 -- 172.21.15.107:0/1999473888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd85c100610 msgr2=0x7fd85c198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.427+0000 7fd85a7fc640 1 --2- 172.21.15.107:0/1999473888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd85c100610 0x7fd85c198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.427+0000 7fd85a7fc640 1 -- 172.21.15.107:0/1999473888 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85c19d000 con 0x7fd85c100fd0 2024-04-25T00:17:05.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.427+0000 7fd85a7fc640 1 --2- 172.21.15.107:0/1999473888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd85c100fd0 0x7fd85c198550 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd850009230 tx=0x7fd850004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:05.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.428+0000 7fd843fff640 1 -- 172.21.15.107:0/1999473888 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85003f030 con 0x7fd85c100fd0 2024-04-25T00:17:05.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.428+0000 7fd8616f5640 1 -- 172.21.15.107:0/1999473888 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd85c19d290 con 0x7fd85c100fd0 2024-04-25T00:17:05.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.428+0000 7fd843fff640 1 -- 172.21.15.107:0/1999473888 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd850003ad0 con 0x7fd85c100fd0 2024-04-25T00:17:05.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.428+0000 7fd843fff640 1 -- 172.21.15.107:0/1999473888 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd850035070 con 0x7fd85c100fd0 2024-04-25T00:17:05.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.428+0000 7fd8616f5640 1 -- 172.21.15.107:0/1999473888 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd85c1a4b80 con 0x7fd85c100fd0 2024-04-25T00:17:05.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.430+0000 7fd843fff640 1 -- 172.21.15.107:0/1999473888 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7fd850042030 con 0x7fd85c100fd0 2024-04-25T00:17:05.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.431+0000 7fd864400740 1 Processor -- start 2024-04-25T00:17:05.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.431+0000 7fd864400740 1 -- start start 2024-04-25T00:17:05.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.431+0000 7fd864400740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55971b822480 0x55971b822860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.431+0000 7fd864400740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b822da0 0x55971b831440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.431+0000 7fd864400740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55971b831980 0x55971b833d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.431+0000 7fd864400740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55971b787670 con 0x55971b822da0 2024-04-25T00:17:05.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.431+0000 7fd864400740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55971b72f970 con 0x55971b831980 2024-04-25T00:17:05.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.431+0000 7fd864400740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55971b783630 con 0x55971b822480 2024-04-25T00:17:05.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.431+0000 7fd843fff640 1 --2- 172.21.15.107:0/1999473888 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd8440a2eb0 0x7fd8440a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.431+0000 7fd85affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b822da0 0x55971b831440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.431+0000 7fd85affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b822da0 0x55971b831440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37912/0 (socket says 172.21.15.107:37912) 2024-04-25T00:17:05.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.431+0000 7fd85affd640 1 -- 172.21.15.107:0/3030461993 learned_addr learned my addr 172.21.15.107:0/3030461993 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:05.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.431+0000 7fd85b7fe640 1 --2- 172.21.15.107:0/1999473888 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd8440a2eb0 0x7fd8440a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.431+0000 7fd85b7fe640 1 --2- 172.21.15.107:0/3030461993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55971b822480 0x55971b822860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.431+0000 7fd843fff640 1 -- 172.21.15.107:0/1999473888 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fd8500f0050 con 0x7fd85c100fd0 2024-04-25T00:17:05.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.431+0000 7fd85a7fc640 1 --2- 172.21.15.107:0/3030461993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55971b831980 0x55971b833d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.431+0000 7fd85affd640 1 -- 172.21.15.107:0/3030461993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55971b822480 msgr2=0x55971b822860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.431+0000 7fd85affd640 1 --2- 172.21.15.107:0/3030461993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55971b822480 0x55971b822860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.431+0000 7fd85affd640 1 -- 172.21.15.107:0/3030461993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55971b831980 msgr2=0x55971b833d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.431+0000 7fd85affd640 1 --2- 172.21.15.107:0/3030461993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55971b831980 0x55971b833d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.431+0000 7fd85affd640 1 -- 172.21.15.107:0/3030461993 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55971b753400 con 0x55971b822da0 2024-04-25T00:17:05.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.431+0000 7fd85a7fc640 1 --2- 172.21.15.107:0/3030461993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55971b831980 0x55971b833d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:17:05.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.431+0000 7fd85b7fe640 1 --2- 172.21.15.107:0/1999473888 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd8440a2eb0 0x7fd8440a5370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd854005fd0 tx=0x7fd854009520 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:05.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.431+0000 7fd85affd640 1 --2- 172.21.15.107:0/3030461993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b822da0 0x55971b831440 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd84c009af0 tx=0x7fd84c00e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b056e8b772b13a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:05.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.432+0000 7fd8417fa640 1 -- 172.21.15.107:0/3030461993 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c00ed20 con 0x55971b822da0 2024-04-25T00:17:05.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.432+0000 7fd8417fa640 1 -- 172.21.15.107:0/3030461993 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd84c005500 con 0x55971b822da0 2024-04-25T00:17:05.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.432+0000 7fd8417fa640 1 -- 172.21.15.107:0/3030461993 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd84c011690 con 0x55971b822da0 2024-04-25T00:17:05.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.432+0000 7fd864400740 1 -- 172.21.15.107:0/3030461993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b822da0 msgr2=0x55971b831440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.432+0000 7fd864400740 1 --2- 172.21.15.107:0/3030461993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b822da0 0x55971b831440 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd84c009af0 tx=0x7fd84c00e2f0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.432+0000 7fd864400740 1 -- 172.21.15.107:0/3030461993 shutdown_connections 2024-04-25T00:17:05.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.432+0000 7fd864400740 1 --2- 172.21.15.107:0/3030461993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55971b831980 0x55971b833d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.432+0000 7fd864400740 1 --2- 172.21.15.107:0/3030461993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b822da0 0x55971b831440 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.432+0000 7fd864400740 1 --2- 172.21.15.107:0/3030461993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55971b822480 0x55971b822860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.432+0000 7fd864400740 1 -- 172.21.15.107:0/3030461993 >> 172.21.15.107:0/3030461993 conn(0x55971b81e8a0 msgr2=0x55971b81ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.432+0000 7fd864400740 1 -- 172.21.15.107:0/3030461993 shutdown_connections 2024-04-25T00:17:05.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.432+0000 7fd864400740 1 -- 172.21.15.107:0/3030461993 wait complete. 2024-04-25T00:17:05.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.433+0000 7fd864400740 1 -- start start 2024-04-25T00:17:05.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.433+0000 7fd864400740 1 Processor -- start 2024-04-25T00:17:05.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.433+0000 7fd864400740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55971b822480 0x55971b8311f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.433+0000 7fd864400740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b822da0 0x55971b82e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.433+0000 7fd864400740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55971b831980 0x55971b82ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.433+0000 7fd864400740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55971b834740 con 0x55971b822da0 2024-04-25T00:17:05.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.433+0000 7fd864400740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55971b743050 con 0x55971b822480 2024-04-25T00:17:05.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.433+0000 7fd864400740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55971b8348c0 con 0x55971b831980 2024-04-25T00:17:05.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.433+0000 7fd864400740 10 client.0 _ll_register_callbacks cb 0x55971b49abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:05.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.433+0000 7fd85a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b822da0 0x55971b82e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.433+0000 7fd85a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b822da0 0x55971b82e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37918/0 (socket says 172.21.15.107:37918) 2024-04-25T00:17:05.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.433+0000 7fd85a7fc640 1 -- 172.21.15.107:0/3287544876 learned_addr learned my addr 172.21.15.107:0/3287544876 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:05.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.433+0000 7fd85a7fc640 1 -- 172.21.15.107:0/3287544876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55971b831980 msgr2=0x55971b82ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.433+0000 7fd85a7fc640 1 --2- 172.21.15.107:0/3287544876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55971b831980 0x55971b82ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.433+0000 7fd85affd640 1 --2- 172.21.15.107:0/3287544876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55971b822480 0x55971b8311f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.433+0000 7fd85a7fc640 1 -- 172.21.15.107:0/3287544876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55971b822480 msgr2=0x55971b8311f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.433+0000 7fd85a7fc640 1 --2- 172.21.15.107:0/3287544876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55971b822480 0x55971b8311f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.433+0000 7fd85a7fc640 1 -- 172.21.15.107:0/3287544876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55971b714dc0 con 0x55971b822da0 2024-04-25T00:17:05.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.433+0000 7fd85a7fc640 1 --2- 172.21.15.107:0/3287544876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b822da0 0x55971b82e4e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd850009900 tx=0x7fd8500045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0fc3ffb3daaf3ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.433+0000 7fd83eff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:17:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.434+0000 7fd83eff5640 1 -- 172.21.15.107:0/3287544876 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd850049040 con 0x55971b822da0 2024-04-25T00:17:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.434+0000 7fd83eff5640 1 -- 172.21.15.107:0/3287544876 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fd8500b8630 con 0x55971b822da0 2024-04-25T00:17:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.434+0000 7fd864400740 1 -- 172.21.15.107:0/3287544876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd84c005500 con 0x55971b822da0 2024-04-25T00:17:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.434+0000 7fd83eff5640 1 -- 172.21.15.107:0/3287544876 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd8500dc0b0 con 0x55971b822da0 2024-04-25T00:17:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.434+0000 7fd864400740 10 client.15795 fetch_fsmap learned FSMap version 24 2024-04-25T00:17:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.434+0000 7fd864400740 1 -- 172.21.15.107:0/3287544876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55971b49b3f0 con 0x55971b822da0 2024-04-25T00:17:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.434+0000 7fd83eff5640 1 -- 172.21.15.107:0/3287544876 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd84c005500 con 0x55971b822da0 2024-04-25T00:17:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.434+0000 7fd83eff5640 1 -- 172.21.15.107:0/3287544876 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fd8500be070 con 0x55971b822da0 2024-04-25T00:17:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.434+0000 7fd864400740 10 client.15795 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:17:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.434+0000 7fd864400740 10 client.15795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.434+0000 7fd864400740 20 client.15795 populate_metadata read hostname 'smithi107' 2024-04-25T00:17:05.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.434+0000 7fd864400740 1 --2- 172.21.15.107:0/3287544876 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55971b841950 0x55971b843dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.434+0000 7fd864400740 4 client.15795 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:17:05.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.434+0000 7fd864400740 1 -- 172.21.15.107:0/3287544876 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55971b794bb0 con 0x55971b841950 2024-04-25T00:17:05.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.435+0000 7fd85affd640 1 --2- 172.21.15.107:0/3287544876 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55971b841950 0x55971b843dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.435+0000 7fd85affd640 1 --2- 172.21.15.107:0/3287544876 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55971b841950 0x55971b843dc0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c46aff22877fc170 server_cookie=664479d69a25aca6 in_seq=0 out_seq=0 2024-04-25T00:17:05.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.435+0000 7fd83eff5640 10 client.15795 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:05.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.436+0000 7fd83eff5640 1 -- 172.21.15.107:0/3287544876 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55971b794bb0 con 0x55971b841950 2024-04-25T00:17:05.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.436+0000 7fd83eff5640 10 client.15795 handle_command_reply: tid=0 2024-04-25T00:17:05.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.436+0000 7fd864400740 1 client.15795 shutdown 2024-04-25T00:17:05.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.436+0000 7fd864400740 1 -- 172.21.15.107:0/3287544876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b822da0 msgr2=0x55971b82e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.436+0000 7fd864400740 1 --2- 172.21.15.107:0/3287544876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b822da0 0x55971b82e4e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd850009900 tx=0x7fd8500045f0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.436+0000 7fd864400740 1 -- 172.21.15.107:0/3287544876 shutdown_connections 2024-04-25T00:17:05.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.436+0000 7fd864400740 1 --2- 172.21.15.107:0/3287544876 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55971b841950 0x55971b843dc0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.436+0000 7fd864400740 1 --2- 172.21.15.107:0/3287544876 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55971b831980 0x55971b82ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.436+0000 7fd864400740 1 --2- 172.21.15.107:0/3287544876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b822da0 0x55971b82e4e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.436+0000 7fd83eff5640 0 client.15795 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:05.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.436+0000 7fd864400740 1 --2- 172.21.15.107:0/3287544876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55971b822480 0x55971b8311f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.436+0000 7fd864400740 1 -- 172.21.15.107:0/3287544876 >> 172.21.15.107:0/3287544876 conn(0x55971b81e8a0 msgr2=0x55971b81db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.436+0000 7fd864400740 1 -- 172.21.15.107:0/3287544876 shutdown_connections 2024-04-25T00:17:05.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.436+0000 7fd864400740 1 -- 172.21.15.107:0/3287544876 wait complete. 2024-04-25T00:17:05.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.436+0000 7fd864400740 20 client.15795 trim_cache size 0 max 16384 2024-04-25T00:17:05.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.449+0000 7fd864400740 1 Processor -- start 2024-04-25T00:17:05.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.449+0000 7fd864400740 1 -- start start 2024-04-25T00:17:05.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.449+0000 7fd864400740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55971b822480 0x55971b83d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.449+0000 7fd864400740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55971b822da0 0x55971b829af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.449+0000 7fd864400740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b831980 0x55971b830d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.449+0000 7fd864400740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c1af030 con 0x55971b831980 2024-04-25T00:17:05.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.449+0000 7fd864400740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c1ac900 con 0x55971b822480 2024-04-25T00:17:05.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.449+0000 7fd864400740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c1aa750 con 0x55971b822da0 2024-04-25T00:17:05.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.449+0000 7fd85b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b831980 0x55971b830d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.449+0000 7fd85b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b831980 0x55971b830d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37930/0 (socket says 172.21.15.107:37930) 2024-04-25T00:17:05.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.449+0000 7fd85b7fe640 1 -- 172.21.15.107:0/4287045440 learned_addr learned my addr 172.21.15.107:0/4287045440 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:05.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.450+0000 7fd85a7fc640 1 --2- 172.21.15.107:0/4287045440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55971b822da0 0x55971b829af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.450+0000 7fd85b7fe640 1 -- 172.21.15.107:0/4287045440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55971b822da0 msgr2=0x55971b829af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.450+0000 7fd85b7fe640 1 --2- 172.21.15.107:0/4287045440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55971b822da0 0x55971b829af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.450+0000 7fd85b7fe640 1 -- 172.21.15.107:0/4287045440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55971b822480 msgr2=0x55971b83d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.450+0000 7fd85affd640 1 --2- 172.21.15.107:0/4287045440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55971b822480 0x55971b83d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.450+0000 7fd85b7fe640 1 --2- 172.21.15.107:0/4287045440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55971b822480 0x55971b83d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.450+0000 7fd85b7fe640 1 -- 172.21.15.107:0/4287045440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55971b5657d0 con 0x55971b831980 2024-04-25T00:17:05.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.450+0000 7fd85b7fe640 1 --2- 172.21.15.107:0/4287045440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b831980 0x55971b830d20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd854005f40 tx=0x7fd854004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e559ee2f5107429 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:05.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.450+0000 7fd840ff9640 1 -- 172.21.15.107:0/4287045440 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd854023030 con 0x55971b831980 2024-04-25T00:17:05.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.450+0000 7fd840ff9640 1 -- 172.21.15.107:0/4287045440 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd854003e50 con 0x55971b831980 2024-04-25T00:17:05.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.450+0000 7fd840ff9640 1 -- 172.21.15.107:0/4287045440 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd854011470 con 0x55971b831980 2024-04-25T00:17:05.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.450+0000 7fd864400740 1 -- 172.21.15.107:0/4287045440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b831980 msgr2=0x55971b830d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.450+0000 7fd864400740 1 --2- 172.21.15.107:0/4287045440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b831980 0x55971b830d20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd854005f40 tx=0x7fd854004a40 comp rx=0 tx=0).stop 2024-04-25T00:17:05.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.451+0000 7fd864400740 1 -- 172.21.15.107:0/4287045440 shutdown_connections 2024-04-25T00:17:05.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.451+0000 7fd864400740 1 --2- 172.21.15.107:0/4287045440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b831980 0x55971b830d20 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.451+0000 7fd864400740 1 --2- 172.21.15.107:0/4287045440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55971b822da0 0x55971b829af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.451+0000 7fd864400740 1 --2- 172.21.15.107:0/4287045440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55971b822480 0x55971b83d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.451+0000 7fd864400740 1 -- 172.21.15.107:0/4287045440 >> 172.21.15.107:0/4287045440 conn(0x55971b81e8a0 msgr2=0x55971b8293e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.451+0000 7fd864400740 1 -- 172.21.15.107:0/4287045440 shutdown_connections 2024-04-25T00:17:05.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.451+0000 7fd864400740 1 -- 172.21.15.107:0/4287045440 wait complete. 2024-04-25T00:17:05.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.451+0000 7fd864400740 1 -- start start 2024-04-25T00:17:05.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.451+0000 7fd864400740 1 Processor -- start 2024-04-25T00:17:05.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.451+0000 7fd864400740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55971b822480 0x55971b839ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.451+0000 7fd864400740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55971b822da0 0x55971b82ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.451+0000 7fd864400740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b831980 0x55971b82b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.451+0000 7fd864400740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c1aadc0 con 0x55971b831980 2024-04-25T00:17:05.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.451+0000 7fd864400740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c1ac230 con 0x55971b822480 2024-04-25T00:17:05.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.451+0000 7fd864400740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c1ab490 con 0x55971b822da0 2024-04-25T00:17:05.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.451+0000 7fd864400740 10 client.0 _ll_register_callbacks cb 0x7fd850009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:05.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.451+0000 7fd85b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b831980 0x55971b82b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.451+0000 7fd85b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b831980 0x55971b82b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37938/0 (socket says 172.21.15.107:37938) 2024-04-25T00:17:05.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.451+0000 7fd85b7fe640 1 -- 172.21.15.107:0/4152216977 learned_addr learned my addr 172.21.15.107:0/4152216977 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:05.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.451+0000 7fd85affd640 1 --2- 172.21.15.107:0/4152216977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55971b822480 0x55971b839ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.451+0000 7fd85a7fc640 1 --2- 172.21.15.107:0/4152216977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55971b822da0 0x55971b82ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.452+0000 7fd85b7fe640 1 -- 172.21.15.107:0/4152216977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55971b822da0 msgr2=0x55971b82ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.452+0000 7fd85b7fe640 1 --2- 172.21.15.107:0/4152216977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55971b822da0 0x55971b82ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.452+0000 7fd85b7fe640 1 -- 172.21.15.107:0/4152216977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55971b822480 msgr2=0x55971b839ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:05.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.452+0000 7fd85b7fe640 1 --2- 172.21.15.107:0/4152216977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55971b822480 0x55971b839ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.452+0000 7fd85b7fe640 1 -- 172.21.15.107:0/4152216977 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55971b82bb60 con 0x55971b831980 2024-04-25T00:17:05.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.452+0000 7fd85b7fe640 1 --2- 172.21.15.107:0/4152216977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b831980 0x55971b82b390 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd85400dfe0 tx=0x7fd854012480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5a7a3bfcb6ca67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:05.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.452+0000 7fd83eff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:17:05.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.452+0000 7fd83eff5640 1 -- 172.21.15.107:0/4152216977 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd854023040 con 0x55971b831980 2024-04-25T00:17:05.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.452+0000 7fd83eff5640 1 -- 172.21.15.107:0/4152216977 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fd854004b20 con 0x55971b831980 2024-04-25T00:17:05.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.452+0000 7fd864400740 1 -- 172.21.15.107:0/4152216977 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd854003e50 con 0x55971b831980 2024-04-25T00:17:05.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.452+0000 7fd83eff5640 1 -- 172.21.15.107:0/4152216977 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd85403b050 con 0x55971b831980 2024-04-25T00:17:05.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.452+0000 7fd864400740 10 client.15801 fetch_fsmap learned FSMap version 24 2024-04-25T00:17:05.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.452+0000 7fd864400740 1 -- 172.21.15.107:0/4152216977 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55971b8a7ec0 con 0x55971b831980 2024-04-25T00:17:05.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.452+0000 7fd83eff5640 1 -- 172.21.15.107:0/4152216977 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd854003e50 con 0x55971b831980 2024-04-25T00:17:05.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.452+0000 7fd83eff5640 1 -- 172.21.15.107:0/4152216977 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fd8540117a0 con 0x55971b831980 2024-04-25T00:17:05.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.452+0000 7fd864400740 10 client.15801 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:17:05.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.452+0000 7fd864400740 10 client.15801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:05.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.452+0000 7fd864400740 20 client.15801 populate_metadata read hostname 'smithi107' 2024-04-25T00:17:05.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.452+0000 7fd864400740 1 --2- 172.21.15.107:0/4152216977 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55971b8a8600 0x55971b8aaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:05.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.452+0000 7fd864400740 4 client.15801 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:17:05.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.452+0000 7fd864400740 1 -- 172.21.15.107:0/4152216977 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55971b82edb0 con 0x55971b8a8600 2024-04-25T00:17:05.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.453+0000 7fd85affd640 1 --2- 172.21.15.107:0/4152216977 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55971b8a8600 0x55971b8aaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:05.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.453+0000 7fd83eff5640 10 client.15801 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:05.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.453+0000 7fd85affd640 1 --2- 172.21.15.107:0/4152216977 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55971b8a8600 0x55971b8aaa20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dce66e0dc4f9253 server_cookie=394e30529354ed6d in_seq=0 out_seq=0 2024-04-25T00:17:05.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.453+0000 7fd83eff5640 1 -- 172.21.15.107:0/4152216977 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55971b82edb0 con 0x55971b8a8600 2024-04-25T00:17:05.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.453+0000 7fd83eff5640 10 client.15801 handle_command_reply: tid=0 2024-04-25T00:17:05.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.453+0000 7fd864400740 1 client.15801 shutdown 2024-04-25T00:17:05.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.453+0000 7fd864400740 1 -- 172.21.15.107:0/4152216977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b831980 msgr2=0x55971b82b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.453+0000 7fd864400740 1 --2- 172.21.15.107:0/4152216977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b831980 0x55971b82b390 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd85400dfe0 tx=0x7fd854012480 comp rx=0 tx=0).stop 2024-04-25T00:17:05.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.453+0000 7fd864400740 1 -- 172.21.15.107:0/4152216977 shutdown_connections 2024-04-25T00:17:05.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.453+0000 7fd864400740 1 --2- 172.21.15.107:0/4152216977 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55971b8a8600 0x55971b8aaa20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.453+0000 7fd864400740 1 --2- 172.21.15.107:0/4152216977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55971b831980 0x55971b82b390 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.453+0000 7fd864400740 1 --2- 172.21.15.107:0/4152216977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55971b822da0 0x55971b82ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.453+0000 7fd83eff5640 0 client.15801 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:05.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.453+0000 7fd864400740 1 --2- 172.21.15.107:0/4152216977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55971b822480 0x55971b839ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.453+0000 7fd864400740 1 -- 172.21.15.107:0/4152216977 >> 172.21.15.107:0/4152216977 conn(0x55971b81e8a0 msgr2=0x55971b823d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.453+0000 7fd864400740 1 -- 172.21.15.107:0/4152216977 shutdown_connections 2024-04-25T00:17:05.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.453+0000 7fd864400740 1 -- 172.21.15.107:0/4152216977 wait complete. 2024-04-25T00:17:05.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.453+0000 7fd864400740 20 client.15801 trim_cache size 0 max 16384 2024-04-25T00:17:05.461 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:17:05.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.454+0000 7fd8616f5640 1 -- 172.21.15.107:0/1999473888 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd8440a2eb0 msgr2=0x7fd8440a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.454+0000 7fd8616f5640 1 --2- 172.21.15.107:0/1999473888 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd8440a2eb0 0x7fd8440a5370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd854005fd0 tx=0x7fd854009520 comp rx=0 tx=0).stop 2024-04-25T00:17:05.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.454+0000 7fd8616f5640 1 -- 172.21.15.107:0/1999473888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd85c100fd0 msgr2=0x7fd85c198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:05.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.454+0000 7fd8616f5640 1 --2- 172.21.15.107:0/1999473888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd85c100fd0 0x7fd85c198550 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd850009230 tx=0x7fd850004270 comp rx=0 tx=0).stop 2024-04-25T00:17:05.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.454+0000 7fd8616f5640 1 -- 172.21.15.107:0/1999473888 shutdown_connections 2024-04-25T00:17:05.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.454+0000 7fd8616f5640 1 --2- 172.21.15.107:0/1999473888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd85c1066b0 0x7fd85c19c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.454+0000 7fd8616f5640 1 --2- 172.21.15.107:0/1999473888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd85c100fd0 0x7fd85c198550 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.454+0000 7fd8616f5640 1 --2- 172.21.15.107:0/1999473888 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd8440a2eb0 0x7fd8440a5370 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.454+0000 7fd8616f5640 1 --2- 172.21.15.107:0/1999473888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd85c100610 0x7fd85c198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:05.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.454+0000 7fd8616f5640 1 -- 172.21.15.107:0/1999473888 >> 172.21.15.107:0/1999473888 conn(0x7fd85c077740 msgr2=0x7fd85c0ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:05.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.454+0000 7fd8616f5640 1 -- 172.21.15.107:0/1999473888 shutdown_connections 2024-04-25T00:17:05.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:05.454+0000 7fd8616f5640 1 -- 172.21.15.107:0/1999473888 wait complete. 2024-04-25T00:17:06.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:05 smithi170 ceph-mon[32807]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T00:17:06.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:05 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2042429617' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:17:06.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:05 smithi107 ceph-mon[29153]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T00:17:06.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:05 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2042429617' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:17:06.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:05 smithi195 ceph-mon[32833]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T00:17:06.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:05 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2042429617' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:17:06.465 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:17:06.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:17:06.466 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:17:06.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.575+0000 7fdf391e7640 1 -- 172.21.15.107:0/1886870237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf341029c0 msgr2=0x7fdf34102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:06.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.575+0000 7fdf391e7640 1 --2- 172.21.15.107:0/1886870237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf341029c0 0x7fdf34102da0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdf24009900 tx=0x7fdf2402a510 comp rx=0 tx=0).stop 2024-04-25T00:17:06.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.575+0000 7fdf391e7640 1 -- 172.21.15.107:0/1886870237 shutdown_connections 2024-04-25T00:17:06.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.575+0000 7fdf391e7640 1 --2- 172.21.15.107:0/1886870237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf3410bd00 0x7fdf3410e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.575+0000 7fdf391e7640 1 --2- 172.21.15.107:0/1886870237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf341032e0 0x7fdf3410b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.575+0000 7fdf391e7640 1 --2- 172.21.15.107:0/1886870237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf341029c0 0x7fdf34102da0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.575+0000 7fdf391e7640 1 -- 172.21.15.107:0/1886870237 >> 172.21.15.107:0/1886870237 conn(0x7fdf34077740 msgr2=0x7fdf34077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:06.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.575+0000 7fdf391e7640 1 -- 172.21.15.107:0/1886870237 shutdown_connections 2024-04-25T00:17:06.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.575+0000 7fdf391e7640 1 -- 172.21.15.107:0/1886870237 wait complete. 2024-04-25T00:17:06.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.576+0000 7fdf391e7640 1 Processor -- start 2024-04-25T00:17:06.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.576+0000 7fdf391e7640 1 -- start start 2024-04-25T00:17:06.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.576+0000 7fdf391e7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf341029c0 0x7fdf34198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.576+0000 7fdf391e7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf341032e0 0x7fdf341985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.576+0000 7fdf391e7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf3410bd00 0x7fdf3419c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.576+0000 7fdf391e7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdf34111b60 con 0x7fdf341029c0 2024-04-25T00:17:06.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.576+0000 7fdf391e7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdf341119e0 con 0x7fdf3410bd00 2024-04-25T00:17:06.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.576+0000 7fdf391e7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdf34111ce0 con 0x7fdf341032e0 2024-04-25T00:17:06.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.577+0000 7fdf32d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf341029c0 0x7fdf34198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.577+0000 7fdf32d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf341029c0 0x7fdf34198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37962/0 (socket says 172.21.15.107:37962) 2024-04-25T00:17:06.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.577+0000 7fdf32d9d640 1 -- 172.21.15.107:0/1841237580 learned_addr learned my addr 172.21.15.107:0/1841237580 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:06.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.577+0000 7fdf3259c640 1 --2- 172.21.15.107:0/1841237580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf341032e0 0x7fdf341985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.577+0000 7fdf3359e640 1 --2- 172.21.15.107:0/1841237580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf3410bd00 0x7fdf3419c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.577+0000 7fdf32d9d640 1 -- 172.21.15.107:0/1841237580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf341032e0 msgr2=0x7fdf341985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:06.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.577+0000 7fdf32d9d640 1 --2- 172.21.15.107:0/1841237580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf341032e0 0x7fdf341985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.577+0000 7fdf32d9d640 1 -- 172.21.15.107:0/1841237580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf3410bd00 msgr2=0x7fdf3419c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:06.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.577+0000 7fdf32d9d640 1 --2- 172.21.15.107:0/1841237580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf3410bd00 0x7fdf3419c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.577+0000 7fdf32d9d640 1 -- 172.21.15.107:0/1841237580 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf3419d070 con 0x7fdf341029c0 2024-04-25T00:17:06.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.577+0000 7fdf32d9d640 1 --2- 172.21.15.107:0/1841237580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf341029c0 0x7fdf34198080 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdf24006010 tx=0x7fdf240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:06.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.577+0000 7fdf1bfff640 1 -- 172.21.15.107:0/1841237580 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2403f030 con 0x7fdf341029c0 2024-04-25T00:17:06.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.577+0000 7fdf1bfff640 1 -- 172.21.15.107:0/1841237580 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf24003ce0 con 0x7fdf341029c0 2024-04-25T00:17:06.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.577+0000 7fdf391e7640 1 -- 172.21.15.107:0/1841237580 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf3419d300 con 0x7fdf341029c0 2024-04-25T00:17:06.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.577+0000 7fdf1bfff640 1 -- 172.21.15.107:0/1841237580 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf24035070 con 0x7fdf341029c0 2024-04-25T00:17:06.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.577+0000 7fdf391e7640 1 -- 172.21.15.107:0/1841237580 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf341a4bf0 con 0x7fdf341029c0 2024-04-25T00:17:06.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.580+0000 7fdf1bfff640 1 -- 172.21.15.107:0/1841237580 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7fdf24042030 con 0x7fdf341029c0 2024-04-25T00:17:06.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.580+0000 7fdf3bfd3740 1 Processor -- start 2024-04-25T00:17:06.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.580+0000 7fdf3bfd3740 1 -- start start 2024-04-25T00:17:06.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.580+0000 7fdf1bfff640 1 --2- 172.21.15.107:0/1841237580 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdf1c0a2eb0 0x7fdf1c0a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.580+0000 7fdf3bfd3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ecde7b4980 0x55ecde7b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.580+0000 7fdf3bfd3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7b3290 0x55ecde7b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.580+0000 7fdf3bfd3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecde7be010 0x55ecde7c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.580+0000 7fdf3bfd3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ecde7172c0 con 0x55ecde7b3290 2024-04-25T00:17:06.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.580+0000 7fdf3bfd3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ecde6bf910 con 0x55ecde7b4980 2024-04-25T00:17:06.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.580+0000 7fdf3bfd3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ecde713280 con 0x55ecde7be010 2024-04-25T00:17:06.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.580+0000 7fdf3359e640 1 --2- 172.21.15.107:0/1841237580 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdf1c0a2eb0 0x7fdf1c0a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.580+0000 7fdf3259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7b3290 0x55ecde7b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.580+0000 7fdf1bfff640 1 -- 172.21.15.107:0/1841237580 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fdf240eff10 con 0x7fdf341029c0 2024-04-25T00:17:06.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.580+0000 7fdf3259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7b3290 0x55ecde7b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37972/0 (socket says 172.21.15.107:37972) 2024-04-25T00:17:06.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.580+0000 7fdf3259c640 1 -- 172.21.15.107:0/940435766 learned_addr learned my addr 172.21.15.107:0/940435766 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:06.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.581+0000 7fdf32d9d640 1 --2- 172.21.15.107:0/940435766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ecde7b4980 0x55ecde7b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.581+0000 7fdf3359e640 1 --2- 172.21.15.107:0/940435766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecde7be010 0x55ecde7c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.581+0000 7fdf3259c640 1 -- 172.21.15.107:0/940435766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecde7be010 msgr2=0x55ecde7c0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:06.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.581+0000 7fdf3259c640 1 --2- 172.21.15.107:0/940435766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecde7be010 0x55ecde7c0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.581+0000 7fdf3259c640 1 -- 172.21.15.107:0/940435766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ecde7b4980 msgr2=0x55ecde7b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:06.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.581+0000 7fdf3259c640 1 --2- 172.21.15.107:0/940435766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ecde7b4980 0x55ecde7b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.581+0000 7fdf3259c640 1 -- 172.21.15.107:0/940435766 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecde6d0fb0 con 0x55ecde7b3290 2024-04-25T00:17:06.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.581+0000 7fdf3359e640 1 --2- 172.21.15.107:0/1841237580 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdf1c0a2eb0 0x7fdf1c0a5370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c005fd0 tx=0x7fdf2c0147e0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:06.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.581+0000 7fdf3259c640 1 --2- 172.21.15.107:0/940435766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7b3290 0x55ecde7b3710 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdf28010b80 tx=0x7fdf28009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9590329205bf1cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:06.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.581+0000 7fdf197fa640 1 -- 172.21.15.107:0/940435766 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2801e070 con 0x55ecde7b3290 2024-04-25T00:17:06.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.581+0000 7fdf197fa640 1 -- 172.21.15.107:0/940435766 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf280198f0 con 0x55ecde7b3290 2024-04-25T00:17:06.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.581+0000 7fdf197fa640 1 -- 172.21.15.107:0/940435766 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf28018850 con 0x55ecde7b3290 2024-04-25T00:17:06.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.582+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/940435766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7b3290 msgr2=0x55ecde7b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:06.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.582+0000 7fdf3bfd3740 1 --2- 172.21.15.107:0/940435766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7b3290 0x55ecde7b3710 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdf28010b80 tx=0x7fdf28009770 comp rx=0 tx=0).stop 2024-04-25T00:17:06.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.582+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/940435766 shutdown_connections 2024-04-25T00:17:06.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.582+0000 7fdf3bfd3740 1 --2- 172.21.15.107:0/940435766 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecde7be010 0x55ecde7c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.582+0000 7fdf3bfd3740 1 --2- 172.21.15.107:0/940435766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7b3290 0x55ecde7b3710 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.582+0000 7fdf3bfd3740 1 --2- 172.21.15.107:0/940435766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ecde7b4980 0x55ecde7b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.582+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/940435766 >> 172.21.15.107:0/940435766 conn(0x55ecde77b950 msgr2=0x55ecde7af7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:06.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.582+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/940435766 shutdown_connections 2024-04-25T00:17:06.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.582+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/940435766 wait complete. 2024-04-25T00:17:06.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.582+0000 7fdf3bfd3740 1 -- start start 2024-04-25T00:17:06.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.582+0000 7fdf3bfd3740 1 Processor -- start 2024-04-25T00:17:06.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.583+0000 7fdf3bfd3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ecde7b3290 0x55ecde7baf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.583+0000 7fdf3bfd3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecde7b4980 0x55ecde7bb460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.583+0000 7fdf3bfd3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7be010 0x55ecde7bb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.583+0000 7fdf3bfd3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ecde7c0ea0 con 0x55ecde7be010 2024-04-25T00:17:06.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.583+0000 7fdf3bfd3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ecde6d2e90 con 0x55ecde7b3290 2024-04-25T00:17:06.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.583+0000 7fdf3bfd3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55ecde7c1020 con 0x55ecde7b4980 2024-04-25T00:17:06.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.583+0000 7fdf3bfd3740 10 client.0 _ll_register_callbacks cb 0x55ecde70e970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:06.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.583+0000 7fdf3359e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7be010 0x55ecde7bb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.583+0000 7fdf3359e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7be010 0x55ecde7bb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37982/0 (socket says 172.21.15.107:37982) 2024-04-25T00:17:06.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.583+0000 7fdf3359e640 1 -- 172.21.15.107:0/3614055250 learned_addr learned my addr 172.21.15.107:0/3614055250 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:06.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.583+0000 7fdf3359e640 1 -- 172.21.15.107:0/3614055250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecde7b4980 msgr2=0x55ecde7bb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:06.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.583+0000 7fdf3359e640 1 --2- 172.21.15.107:0/3614055250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecde7b4980 0x55ecde7bb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.583+0000 7fdf3359e640 1 -- 172.21.15.107:0/3614055250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ecde7b3290 msgr2=0x55ecde7baf20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:17:06.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.583+0000 7fdf3359e640 1 --2- 172.21.15.107:0/3614055250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ecde7b3290 0x55ecde7baf20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.583+0000 7fdf3359e640 1 -- 172.21.15.107:0/3614055250 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecde4f37e0 con 0x55ecde7be010 2024-04-25T00:17:06.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.583+0000 7fdf3359e640 1 --2- 172.21.15.107:0/3614055250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7be010 0x55ecde7bb9a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0129b0 tx=0x7fdf2c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec649e97d269a76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:06.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.583+0000 7fdf16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:17:06.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.583+0000 7fdf16ff5640 1 -- 172.21.15.107:0/3614055250 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c025070 con 0x55ecde7be010 2024-04-25T00:17:06.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.583+0000 7fdf16ff5640 1 -- 172.21.15.107:0/3614055250 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fdf2c01ed50 con 0x55ecde7be010 2024-04-25T00:17:06.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.583+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/3614055250 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf280198f0 con 0x55ecde7be010 2024-04-25T00:17:06.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.583+0000 7fdf16ff5640 1 -- 172.21.15.107:0/3614055250 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdf2c034050 con 0x55ecde7be010 2024-04-25T00:17:06.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.584+0000 7fdf3bfd3740 10 client.15813 fetch_fsmap learned FSMap version 24 2024-04-25T00:17:06.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.584+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/3614055250 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ecde553f70 con 0x55ecde7be010 2024-04-25T00:17:06.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.584+0000 7fdf16ff5640 1 -- 172.21.15.107:0/3614055250 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf280198f0 con 0x55ecde7be010 2024-04-25T00:17:06.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.584+0000 7fdf16ff5640 1 -- 172.21.15.107:0/3614055250 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fdf2c00f040 con 0x55ecde7be010 2024-04-25T00:17:06.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.584+0000 7fdf3bfd3740 10 client.15813 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:17:06.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.584+0000 7fdf3bfd3740 10 client.15813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:06.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.584+0000 7fdf3bfd3740 20 client.15813 populate_metadata read hostname 'smithi107' 2024-04-25T00:17:06.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.584+0000 7fdf3bfd3740 1 --2- 172.21.15.107:0/3614055250 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55ecde7ce1d0 0x55ecde7d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.584+0000 7fdf3bfd3740 4 client.15813 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:17:06.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.584+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/3614055250 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecde6eb5a0 con 0x55ecde7ce1d0 2024-04-25T00:17:06.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.584+0000 7fdf32d9d640 1 --2- 172.21.15.107:0/3614055250 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55ecde7ce1d0 0x55ecde7d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.585+0000 7fdf16ff5640 10 client.15813 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:06.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.585+0000 7fdf32d9d640 1 --2- 172.21.15.107:0/3614055250 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55ecde7ce1d0 0x55ecde7d0690 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7e33ac3aec4b698 server_cookie=62bc702e77e1e677 in_seq=0 out_seq=0 2024-04-25T00:17:06.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.585+0000 7fdf16ff5640 1 -- 172.21.15.107:0/3614055250 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ecde6eb5a0 con 0x55ecde7ce1d0 2024-04-25T00:17:06.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.585+0000 7fdf16ff5640 10 client.15813 handle_command_reply: tid=0 2024-04-25T00:17:06.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.585+0000 7fdf3bfd3740 1 client.15813 shutdown 2024-04-25T00:17:06.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.585+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/3614055250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7be010 msgr2=0x55ecde7bb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:06.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.585+0000 7fdf3bfd3740 1 --2- 172.21.15.107:0/3614055250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7be010 0x55ecde7bb9a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0129b0 tx=0x7fdf2c004ab0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.585+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/3614055250 shutdown_connections 2024-04-25T00:17:06.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.585+0000 7fdf3bfd3740 1 --2- 172.21.15.107:0/3614055250 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55ecde7ce1d0 0x55ecde7d0690 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.585+0000 7fdf3bfd3740 1 --2- 172.21.15.107:0/3614055250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7be010 0x55ecde7bb9a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.586+0000 7fdf3bfd3740 1 --2- 172.21.15.107:0/3614055250 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecde7b4980 0x55ecde7bb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.586+0000 7fdf16ff5640 0 client.15813 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:06.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.586+0000 7fdf3bfd3740 1 --2- 172.21.15.107:0/3614055250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ecde7b3290 0x55ecde7baf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.586+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/3614055250 >> 172.21.15.107:0/3614055250 conn(0x55ecde77b950 msgr2=0x55ecde7b55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:06.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.586+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/3614055250 shutdown_connections 2024-04-25T00:17:06.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.586+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/3614055250 wait complete. 2024-04-25T00:17:06.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.586+0000 7fdf3bfd3740 20 client.15813 trim_cache size 0 max 16384 2024-04-25T00:17:06.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.603+0000 7fdf3bfd3740 1 Processor -- start 2024-04-25T00:17:06.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.603+0000 7fdf3bfd3740 1 -- start start 2024-04-25T00:17:06.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.604+0000 7fdf3bfd3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecde7b3290 0x55ecde7ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.604+0000 7fdf3bfd3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ecde7b4980 0x55ecde7c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.604+0000 7fdf3bfd3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7be010 0x55ecde7bb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.604+0000 7fdf3bfd3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdf341af1f0 con 0x55ecde7be010 2024-04-25T00:17:06.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.604+0000 7fdf3bfd3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdf341acac0 con 0x55ecde7b4980 2024-04-25T00:17:06.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.604+0000 7fdf3bfd3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdf341aa910 con 0x55ecde7b3290 2024-04-25T00:17:06.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.604+0000 7fdf3359e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7be010 0x55ecde7bb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.604+0000 7fdf3359e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7be010 0x55ecde7bb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37996/0 (socket says 172.21.15.107:37996) 2024-04-25T00:17:06.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.604+0000 7fdf3359e640 1 -- 172.21.15.107:0/3402645042 learned_addr learned my addr 172.21.15.107:0/3402645042 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:06.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.604+0000 7fdf32d9d640 1 --2- 172.21.15.107:0/3402645042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecde7b3290 0x55ecde7ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.604+0000 7fdf3359e640 1 -- 172.21.15.107:0/3402645042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecde7b3290 msgr2=0x55ecde7ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:06.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.604+0000 7fdf3359e640 1 --2- 172.21.15.107:0/3402645042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecde7b3290 0x55ecde7ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.604+0000 7fdf3359e640 1 -- 172.21.15.107:0/3402645042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ecde7b4980 msgr2=0x55ecde7c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:06.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.604+0000 7fdf3359e640 1 --2- 172.21.15.107:0/3402645042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ecde7b4980 0x55ecde7c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.604+0000 7fdf3359e640 1 -- 172.21.15.107:0/3402645042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecde7afa00 con 0x55ecde7be010 2024-04-25T00:17:06.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.604+0000 7fdf32d9d640 1 --2- 172.21.15.107:0/3402645042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecde7b3290 0x55ecde7ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:17:06.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.604+0000 7fdf3359e640 1 --2- 172.21.15.107:0/3402645042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7be010 0x55ecde7bb5f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c009980 tx=0x7fdf2c005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b3bcbc778a0c55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:06.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.604+0000 7fdf18ff9640 1 -- 172.21.15.107:0/3402645042 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c03d070 con 0x55ecde7be010 2024-04-25T00:17:06.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.604+0000 7fdf18ff9640 1 -- 172.21.15.107:0/3402645042 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf2c01e790 con 0x55ecde7be010 2024-04-25T00:17:06.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.604+0000 7fdf18ff9640 1 -- 172.21.15.107:0/3402645042 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c025040 con 0x55ecde7be010 2024-04-25T00:17:06.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.605+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/3402645042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7be010 msgr2=0x55ecde7bb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:06.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.605+0000 7fdf3bfd3740 1 --2- 172.21.15.107:0/3402645042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7be010 0x55ecde7bb5f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c009980 tx=0x7fdf2c005cd0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.605+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/3402645042 shutdown_connections 2024-04-25T00:17:06.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.605+0000 7fdf3bfd3740 1 --2- 172.21.15.107:0/3402645042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7be010 0x55ecde7bb5f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.605+0000 7fdf3bfd3740 1 --2- 172.21.15.107:0/3402645042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ecde7b4980 0x55ecde7c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.605+0000 7fdf3bfd3740 1 --2- 172.21.15.107:0/3402645042 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecde7b3290 0x55ecde7ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.605+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/3402645042 >> 172.21.15.107:0/3402645042 conn(0x55ecde77b950 msgr2=0x55ecde7baee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:06.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.605+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/3402645042 shutdown_connections 2024-04-25T00:17:06.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.605+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/3402645042 wait complete. 2024-04-25T00:17:06.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.605+0000 7fdf3bfd3740 1 -- start start 2024-04-25T00:17:06.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.606+0000 7fdf3bfd3740 1 Processor -- start 2024-04-25T00:17:06.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.606+0000 7fdf3bfd3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ecde7b3290 0x55ecde7c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.606+0000 7fdf3bfd3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecde7b4980 0x55ecde7c32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.606+0000 7fdf3bfd3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7be010 0x55ecde7c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.606+0000 7fdf3bfd3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdf341aaf80 con 0x55ecde7be010 2024-04-25T00:17:06.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.606+0000 7fdf3bfd3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdf341ac3f0 con 0x55ecde7b3290 2024-04-25T00:17:06.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.606+0000 7fdf3bfd3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdf341ab650 con 0x55ecde7b4980 2024-04-25T00:17:06.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.606+0000 7fdf3bfd3740 10 client.0 _ll_register_callbacks cb 0x7fdf2c009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:06.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.606+0000 7fdf3359e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7be010 0x55ecde7c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.606+0000 7fdf3359e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7be010 0x55ecde7c37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38008/0 (socket says 172.21.15.107:38008) 2024-04-25T00:17:06.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.606+0000 7fdf3359e640 1 -- 172.21.15.107:0/692032617 learned_addr learned my addr 172.21.15.107:0/692032617 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:06.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.606+0000 7fdf3259c640 1 --2- 172.21.15.107:0/692032617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecde7b4980 0x55ecde7c32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.606+0000 7fdf32d9d640 1 --2- 172.21.15.107:0/692032617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ecde7b3290 0x55ecde7c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.606+0000 7fdf3359e640 1 -- 172.21.15.107:0/692032617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecde7b4980 msgr2=0x55ecde7c32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:06.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.606+0000 7fdf3359e640 1 --2- 172.21.15.107:0/692032617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecde7b4980 0x55ecde7c32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.606+0000 7fdf3359e640 1 -- 172.21.15.107:0/692032617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ecde7b3290 msgr2=0x55ecde7c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:06.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.606+0000 7fdf3359e640 1 --2- 172.21.15.107:0/692032617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ecde7b3290 0x55ecde7c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.606+0000 7fdf3359e640 1 -- 172.21.15.107:0/692032617 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecde3eb170 con 0x55ecde7be010 2024-04-25T00:17:06.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.606+0000 7fdf32d9d640 1 --2- 172.21.15.107:0/692032617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ecde7b3290 0x55ecde7c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:17:06.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.606+0000 7fdf3359e640 1 --2- 172.21.15.107:0/692032617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7be010 0x55ecde7c37f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c014e00 tx=0x7fdf2c01ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fc166ffa5b7d3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:06.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.606+0000 7fdf16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:17:06.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.606+0000 7fdf16ff5640 1 -- 172.21.15.107:0/692032617 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c03d030 con 0x55ecde7be010 2024-04-25T00:17:06.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.606+0000 7fdf16ff5640 1 -- 172.21.15.107:0/692032617 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fdf2c004c50 con 0x55ecde7be010 2024-04-25T00:17:06.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.606+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/692032617 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf2c01e790 con 0x55ecde7be010 2024-04-25T00:17:06.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.607+0000 7fdf16ff5640 1 -- 172.21.15.107:0/692032617 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdf2c04a050 con 0x55ecde7be010 2024-04-25T00:17:06.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.607+0000 7fdf3bfd3740 10 client.15819 fetch_fsmap learned FSMap version 24 2024-04-25T00:17:06.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.607+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/692032617 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ecde8354d0 con 0x55ecde7be010 2024-04-25T00:17:06.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.607+0000 7fdf16ff5640 1 -- 172.21.15.107:0/692032617 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf2c01e790 con 0x55ecde7be010 2024-04-25T00:17:06.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.607+0000 7fdf16ff5640 1 -- 172.21.15.107:0/692032617 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fdf2c005460 con 0x55ecde7be010 2024-04-25T00:17:06.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.607+0000 7fdf3bfd3740 10 client.15819 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:17:06.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.607+0000 7fdf3bfd3740 10 client.15819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:06.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.607+0000 7fdf3bfd3740 20 client.15819 populate_metadata read hostname 'smithi107' 2024-04-25T00:17:06.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.607+0000 7fdf3bfd3740 1 --2- 172.21.15.107:0/692032617 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55ecde835b30 0x55ecde837f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.607+0000 7fdf3bfd3740 4 client.15819 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:17:06.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.607+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/692032617 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ecde724800 con 0x55ecde835b30 2024-04-25T00:17:06.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.607+0000 7fdf32d9d640 1 --2- 172.21.15.107:0/692032617 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55ecde835b30 0x55ecde837f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.608+0000 7fdf16ff5640 10 client.15819 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:06.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.608+0000 7fdf32d9d640 1 --2- 172.21.15.107:0/692032617 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55ecde835b30 0x55ecde837f20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33eb16dccea86b70 server_cookie=2abe086a193d5bc8 in_seq=0 out_seq=0 2024-04-25T00:17:06.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.609+0000 7fdf16ff5640 1 -- 172.21.15.107:0/692032617 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ecde724800 con 0x55ecde835b30 2024-04-25T00:17:06.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.609+0000 7fdf16ff5640 10 client.15819 handle_command_reply: tid=0 2024-04-25T00:17:06.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.609+0000 7fdf3bfd3740 1 client.15819 shutdown 2024-04-25T00:17:06.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.609+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/692032617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7be010 msgr2=0x55ecde7c37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:06.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.609+0000 7fdf3bfd3740 1 --2- 172.21.15.107:0/692032617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7be010 0x55ecde7c37f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c014e00 tx=0x7fdf2c01ed80 comp rx=0 tx=0).stop 2024-04-25T00:17:06.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.609+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/692032617 shutdown_connections 2024-04-25T00:17:06.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.609+0000 7fdf3bfd3740 1 --2- 172.21.15.107:0/692032617 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55ecde835b30 0x55ecde837f20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.609+0000 7fdf16ff5640 0 client.15819 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:06.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.609+0000 7fdf3bfd3740 1 --2- 172.21.15.107:0/692032617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ecde7be010 0x55ecde7c37f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.609+0000 7fdf3bfd3740 1 --2- 172.21.15.107:0/692032617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55ecde7b4980 0x55ecde7c32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.609+0000 7fdf3bfd3740 1 --2- 172.21.15.107:0/692032617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ecde7b3290 0x55ecde7c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.609+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/692032617 >> 172.21.15.107:0/692032617 conn(0x55ecde77b950 msgr2=0x55ecde7b55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:06.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.609+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/692032617 shutdown_connections 2024-04-25T00:17:06.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.609+0000 7fdf3bfd3740 1 -- 172.21.15.107:0/692032617 wait complete. 2024-04-25T00:17:06.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.609+0000 7fdf3bfd3740 20 client.15819 trim_cache size 0 max 16384 2024-04-25T00:17:06.618 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:17:06.619 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-04-25T00:17:06.619 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "8d4089b7-4d8b-4532-be5c-cc1730e8b4a6", 2024-04-25T00:17:06.619 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-04-25T00:17:06.619 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:17:06.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.610+0000 7fdf391e7640 1 -- 172.21.15.107:0/1841237580 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdf1c0a2eb0 msgr2=0x7fdf1c0a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:06.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.610+0000 7fdf391e7640 1 --2- 172.21.15.107:0/1841237580 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdf1c0a2eb0 0x7fdf1c0a5370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c005fd0 tx=0x7fdf2c0147e0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.610+0000 7fdf391e7640 1 -- 172.21.15.107:0/1841237580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf341029c0 msgr2=0x7fdf34198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:06.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.610+0000 7fdf391e7640 1 --2- 172.21.15.107:0/1841237580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf341029c0 0x7fdf34198080 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdf24006010 tx=0x7fdf240042f0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.610+0000 7fdf391e7640 1 -- 172.21.15.107:0/1841237580 shutdown_connections 2024-04-25T00:17:06.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.610+0000 7fdf391e7640 1 --2- 172.21.15.107:0/1841237580 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdf1c0a2eb0 0x7fdf1c0a5370 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.610+0000 7fdf391e7640 1 --2- 172.21.15.107:0/1841237580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdf3410bd00 0x7fdf3419c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.610+0000 7fdf391e7640 1 --2- 172.21.15.107:0/1841237580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdf341032e0 0x7fdf341985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.610+0000 7fdf391e7640 1 --2- 172.21.15.107:0/1841237580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf341029c0 0x7fdf34198080 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.610+0000 7fdf391e7640 1 -- 172.21.15.107:0/1841237580 >> 172.21.15.107:0/1841237580 conn(0x7fdf34077740 msgr2=0x7fdf340fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:06.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.610+0000 7fdf391e7640 1 -- 172.21.15.107:0/1841237580 shutdown_connections 2024-04-25T00:17:06.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.610+0000 7fdf391e7640 1 -- 172.21.15.107:0/1841237580 wait complete. 2024-04-25T00:17:06.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:17:06.624 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:17:06.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.726+0000 7feb8a3e7640 1 -- 172.21.15.107:0/295602282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84100fd0 msgr2=0x7feb8410d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:06.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.726+0000 7feb8a3e7640 1 --2- 172.21.15.107:0/295602282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84100fd0 0x7feb8410d4e0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009cd0 tx=0x7feb7c02a8e0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.726+0000 7feb8a3e7640 1 -- 172.21.15.107:0/295602282 shutdown_connections 2024-04-25T00:17:06.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.726+0000 7feb8a3e7640 1 --2- 172.21.15.107:0/295602282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84100fd0 0x7feb8410d4e0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.726+0000 7feb8a3e7640 1 --2- 172.21.15.107:0/295602282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb84100610 0x7feb84100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.726+0000 7feb8a3e7640 1 --2- 172.21.15.107:0/295602282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb841066b0 0x7feb84106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.726+0000 7feb8a3e7640 1 -- 172.21.15.107:0/295602282 >> 172.21.15.107:0/295602282 conn(0x7feb84077740 msgr2=0x7feb84077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:06.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.726+0000 7feb8a3e7640 1 -- 172.21.15.107:0/295602282 shutdown_connections 2024-04-25T00:17:06.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.726+0000 7feb8a3e7640 1 -- 172.21.15.107:0/295602282 wait complete. 2024-04-25T00:17:06.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.727+0000 7feb8a3e7640 1 Processor -- start 2024-04-25T00:17:06.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.727+0000 7feb8a3e7640 1 -- start start 2024-04-25T00:17:06.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.727+0000 7feb8a3e7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb84100610 0x7feb8419a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.727+0000 7feb8a3e7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84100fd0 0x7feb8419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.727+0000 7feb8a3e7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb841066b0 0x7feb8419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.727+0000 7feb8a3e7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feb841109f0 con 0x7feb84100fd0 2024-04-25T00:17:06.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.727+0000 7feb8a3e7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7feb84110870 con 0x7feb84100610 2024-04-25T00:17:06.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.727+0000 7feb8a3e7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feb84110b70 con 0x7feb841066b0 2024-04-25T00:17:06.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.727+0000 7feb837fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84100fd0 0x7feb8419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.727+0000 7feb837fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84100fd0 0x7feb8419a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38030/0 (socket says 172.21.15.107:38030) 2024-04-25T00:17:06.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.727+0000 7feb837fe640 1 -- 172.21.15.107:0/3718903034 learned_addr learned my addr 172.21.15.107:0/3718903034 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:06.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.727+0000 7feb83fff640 1 --2- 172.21.15.107:0/3718903034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb84100610 0x7feb8419a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.727+0000 7feb88984640 1 --2- 172.21.15.107:0/3718903034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb841066b0 0x7feb8419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.727+0000 7feb837fe640 1 -- 172.21.15.107:0/3718903034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb841066b0 msgr2=0x7feb8419e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:06.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.727+0000 7feb837fe640 1 --2- 172.21.15.107:0/3718903034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb841066b0 0x7feb8419e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.727+0000 7feb837fe640 1 -- 172.21.15.107:0/3718903034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb84100610 msgr2=0x7feb8419a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:06.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.727+0000 7feb837fe640 1 --2- 172.21.15.107:0/3718903034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb84100610 0x7feb8419a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.727+0000 7feb837fe640 1 -- 172.21.15.107:0/3718903034 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb8419f090 con 0x7feb84100fd0 2024-04-25T00:17:06.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.727+0000 7feb88984640 1 --2- 172.21.15.107:0/3718903034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb841066b0 0x7feb8419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:17:06.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.727+0000 7feb83fff640 1 --2- 172.21.15.107:0/3718903034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb84100610 0x7feb8419a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:17:06.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.728+0000 7feb837fe640 1 --2- 172.21.15.107:0/3718903034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84100fd0 0x7feb8419a5e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7feb7800d950 tx=0x7feb7800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:06.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.728+0000 7feb817fa640 1 -- 172.21.15.107:0/3718903034 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb78017070 con 0x7feb84100fd0 2024-04-25T00:17:06.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.728+0000 7feb817fa640 1 -- 172.21.15.107:0/3718903034 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb78009c30 con 0x7feb84100fd0 2024-04-25T00:17:06.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.728+0000 7feb8a3e7640 1 -- 172.21.15.107:0/3718903034 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb8419f380 con 0x7feb84100fd0 2024-04-25T00:17:06.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.728+0000 7feb817fa640 1 -- 172.21.15.107:0/3718903034 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7800f6b0 con 0x7feb84100fd0 2024-04-25T00:17:06.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.728+0000 7feb8a3e7640 1 -- 172.21.15.107:0/3718903034 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb841a6c10 con 0x7feb84100fd0 2024-04-25T00:17:06.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.730+0000 7feb817fa640 1 -- 172.21.15.107:0/3718903034 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7feb780109d0 con 0x7feb84100fd0 2024-04-25T00:17:06.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.730+0000 7feb817fa640 1 --2- 172.21.15.107:0/3718903034 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7feb6c0a2eb0 0x7feb6c0a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.731+0000 7feb83fff640 1 --2- 172.21.15.107:0/3718903034 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7feb6c0a2eb0 0x7feb6c0a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.731+0000 7feb817fa640 1 -- 172.21.15.107:0/3718903034 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7feb780cf880 con 0x7feb84100fd0 2024-04-25T00:17:06.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.731+0000 7feb83fff640 1 --2- 172.21.15.107:0/3718903034 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7feb6c0a2eb0 0x7feb6c0a5370 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feb84101750 tx=0x7feb74005db0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:06.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.731+0000 7feb8d1d1740 1 Processor -- start 2024-04-25T00:17:06.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.731+0000 7feb8d1d1740 1 -- start start 2024-04-25T00:17:06.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.731+0000 7feb8d1d1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901908c630 0x55901908ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.732+0000 7feb8d1d1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55901908cf50 0x5590190970e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.732+0000 7feb8d1d1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559019097620 0x559019099a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.732+0000 7feb8d1d1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559018ff16b0 con 0x55901908c630 2024-04-25T00:17:06.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.732+0000 7feb8d1d1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559018f99880 con 0x559019097620 2024-04-25T00:17:06.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.732+0000 7feb8d1d1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559018fed670 con 0x55901908cf50 2024-04-25T00:17:06.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.732+0000 7feb83fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901908c630 0x55901908ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.732+0000 7feb83fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901908c630 0x55901908ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38036/0 (socket says 172.21.15.107:38036) 2024-04-25T00:17:06.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.732+0000 7feb83fff640 1 -- 172.21.15.107:0/1924330108 learned_addr learned my addr 172.21.15.107:0/1924330108 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:06.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.732+0000 7feb837fe640 1 --2- 172.21.15.107:0/1924330108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55901908cf50 0x5590190970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.732+0000 7feb83fff640 1 -- 172.21.15.107:0/1924330108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55901908cf50 msgr2=0x5590190970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:06.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.732+0000 7feb83fff640 1 --2- 172.21.15.107:0/1924330108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55901908cf50 0x5590190970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.732+0000 7feb83fff640 1 -- 172.21.15.107:0/1924330108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559019097620 msgr2=0x559019099a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:06.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.732+0000 7feb83fff640 1 --2- 172.21.15.107:0/1924330108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559019097620 0x559019099a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.732+0000 7feb83fff640 1 -- 172.21.15.107:0/1924330108 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559018dcf7d0 con 0x55901908c630 2024-04-25T00:17:06.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.732+0000 7feb83fff640 1 --2- 172.21.15.107:0/1924330108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901908c630 0x55901908ca10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7feb7400ddc0 tx=0x7feb74010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2468af95181ca8b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:06.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.732+0000 7feb6a7fc640 1 -- 172.21.15.107:0/1924330108 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb74020070 con 0x55901908c630 2024-04-25T00:17:06.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.732+0000 7feb6a7fc640 1 -- 172.21.15.107:0/1924330108 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb7400fe30 con 0x55901908c630 2024-04-25T00:17:06.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.732+0000 7feb6a7fc640 1 -- 172.21.15.107:0/1924330108 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7401bbe0 con 0x55901908c630 2024-04-25T00:17:06.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.733+0000 7feb8d1d1740 1 -- 172.21.15.107:0/1924330108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901908c630 msgr2=0x55901908ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:06.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.733+0000 7feb8d1d1740 1 --2- 172.21.15.107:0/1924330108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901908c630 0x55901908ca10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7feb7400ddc0 tx=0x7feb74010000 comp rx=0 tx=0).stop 2024-04-25T00:17:06.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.733+0000 7feb8d1d1740 1 -- 172.21.15.107:0/1924330108 shutdown_connections 2024-04-25T00:17:06.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.733+0000 7feb8d1d1740 1 --2- 172.21.15.107:0/1924330108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559019097620 0x559019099a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.733+0000 7feb8d1d1740 1 --2- 172.21.15.107:0/1924330108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55901908cf50 0x5590190970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.733+0000 7feb8d1d1740 1 --2- 172.21.15.107:0/1924330108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901908c630 0x55901908ca10 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.733+0000 7feb8d1d1740 1 -- 172.21.15.107:0/1924330108 >> 172.21.15.107:0/1924330108 conn(0x559019088950 msgr2=0x559019088d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:06.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.733+0000 7feb8d1d1740 1 -- 172.21.15.107:0/1924330108 shutdown_connections 2024-04-25T00:17:06.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.733+0000 7feb8d1d1740 1 -- 172.21.15.107:0/1924330108 wait complete. 2024-04-25T00:17:06.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.733+0000 7feb8d1d1740 1 -- start start 2024-04-25T00:17:06.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.734+0000 7feb8d1d1740 1 Processor -- start 2024-04-25T00:17:06.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.734+0000 7feb8d1d1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55901908c630 0x5590190945a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.734+0000 7feb8d1d1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55901908cf50 0x559019094ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.734+0000 7feb8d1d1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559019097620 0x559019095020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.734+0000 7feb8d1d1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55901909a410 con 0x559019097620 2024-04-25T00:17:06.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.734+0000 7feb8d1d1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559018facf50 con 0x55901908cf50 2024-04-25T00:17:06.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.734+0000 7feb8d1d1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55901909a590 con 0x55901908c630 2024-04-25T00:17:06.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.734+0000 7feb8d1d1740 10 client.0 _ll_register_callbacks cb 0x559018fe8000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:06.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.734+0000 7feb88984640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559019097620 0x559019095020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.734+0000 7feb88984640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559019097620 0x559019095020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38042/0 (socket says 172.21.15.107:38042) 2024-04-25T00:17:06.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.734+0000 7feb88984640 1 -- 172.21.15.107:0/1999013623 learned_addr learned my addr 172.21.15.107:0/1999013623 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:06.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.734+0000 7feb837fe640 1 --2- 172.21.15.107:0/1999013623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55901908cf50 0x559019094ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.734+0000 7feb83fff640 1 --2- 172.21.15.107:0/1999013623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55901908c630 0x5590190945a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.734+0000 7feb88984640 1 -- 172.21.15.107:0/1999013623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55901908c630 msgr2=0x5590190945a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:06.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.734+0000 7feb88984640 1 --2- 172.21.15.107:0/1999013623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55901908c630 0x5590190945a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.734+0000 7feb88984640 1 -- 172.21.15.107:0/1999013623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55901908cf50 msgr2=0x559019094ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:06.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.734+0000 7feb88984640 1 --2- 172.21.15.107:0/1999013623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55901908cf50 0x559019094ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.734+0000 7feb88984640 1 -- 172.21.15.107:0/1999013623 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559018d053f0 con 0x559019097620 2024-04-25T00:17:06.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.734+0000 7feb88984640 1 --2- 172.21.15.107:0/1999013623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559019097620 0x559019095020 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009430 tx=0x7feb7c031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae7eca16d212efd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:06.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.734+0000 7feb637fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:17:06.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.735+0000 7feb637fe640 1 -- 172.21.15.107:0/1999013623 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c003b70 con 0x559019097620 2024-04-25T00:17:06.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.735+0000 7feb637fe640 1 -- 172.21.15.107:0/1999013623 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7feb7c003d10 con 0x559019097620 2024-04-25T00:17:06.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.735+0000 7feb8d1d1740 1 -- 172.21.15.107:0/1999013623 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb7400fe30 con 0x559019097620 2024-04-25T00:17:06.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.735+0000 7feb637fe640 1 -- 172.21.15.107:0/1999013623 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7feb7c04e050 con 0x559019097620 2024-04-25T00:17:06.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.735+0000 7feb8d1d1740 10 client.15831 fetch_fsmap learned FSMap version 24 2024-04-25T00:17:06.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.735+0000 7feb8d1d1740 1 -- 172.21.15.107:0/1999013623 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559018cc50f0 con 0x559019097620 2024-04-25T00:17:06.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.735+0000 7feb637fe640 1 -- 172.21.15.107:0/1999013623 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb7400fe30 con 0x559019097620 2024-04-25T00:17:06.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.735+0000 7feb637fe640 1 -- 172.21.15.107:0/1999013623 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7feb7c030070 con 0x559019097620 2024-04-25T00:17:06.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.735+0000 7feb8d1d1740 10 client.15831 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:17:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.735+0000 7feb8d1d1740 10 client.15831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.735+0000 7feb8d1d1740 20 client.15831 populate_metadata read hostname 'smithi107' 2024-04-25T00:17:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.735+0000 7feb8d1d1740 1 --2- 172.21.15.107:0/1999013623 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5590190a7850 0x5590190a9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.735+0000 7feb8d1d1740 4 client.15831 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:17:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.735+0000 7feb8d1d1740 1 -- 172.21.15.107:0/1999013623 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559018ffebf0 con 0x5590190a7850 2024-04-25T00:17:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.736+0000 7feb83fff640 1 --2- 172.21.15.107:0/1999013623 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5590190a7850 0x5590190a9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.736+0000 7feb637fe640 10 client.15831 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.736+0000 7feb83fff640 1 --2- 172.21.15.107:0/1999013623 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5590190a7850 0x5590190a9c40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4cbc2b7678ca420 server_cookie=2d85e26a2b9062a2 in_seq=0 out_seq=0 2024-04-25T00:17:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.737+0000 7feb637fe640 1 -- 172.21.15.107:0/1999013623 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559018ffebf0 con 0x5590190a7850 2024-04-25T00:17:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.737+0000 7feb637fe640 10 client.15831 handle_command_reply: tid=0 2024-04-25T00:17:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.737+0000 7feb8d1d1740 1 client.15831 shutdown 2024-04-25T00:17:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.737+0000 7feb8d1d1740 1 -- 172.21.15.107:0/1999013623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559019097620 msgr2=0x559019095020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.737+0000 7feb8d1d1740 1 --2- 172.21.15.107:0/1999013623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559019097620 0x559019095020 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009430 tx=0x7feb7c031af0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.737+0000 7feb8d1d1740 1 -- 172.21.15.107:0/1999013623 shutdown_connections 2024-04-25T00:17:06.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.737+0000 7feb8d1d1740 1 --2- 172.21.15.107:0/1999013623 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5590190a7850 0x5590190a9c40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.737+0000 7feb8d1d1740 1 --2- 172.21.15.107:0/1999013623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559019097620 0x559019095020 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.737+0000 7feb8d1d1740 1 --2- 172.21.15.107:0/1999013623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55901908cf50 0x559019094ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.737+0000 7feb637fe640 0 client.15831 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:06.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.737+0000 7feb8d1d1740 1 --2- 172.21.15.107:0/1999013623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55901908c630 0x5590190945a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.737+0000 7feb8d1d1740 1 -- 172.21.15.107:0/1999013623 >> 172.21.15.107:0/1999013623 conn(0x559019088950 msgr2=0x55901908e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:06.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.737+0000 7feb8d1d1740 1 -- 172.21.15.107:0/1999013623 shutdown_connections 2024-04-25T00:17:06.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.737+0000 7feb8d1d1740 1 -- 172.21.15.107:0/1999013623 wait complete. 2024-04-25T00:17:06.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.737+0000 7feb8d1d1740 20 client.15831 trim_cache size 0 max 16384 2024-04-25T00:17:06.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.755+0000 7feb8d1d1740 1 Processor -- start 2024-04-25T00:17:06.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.755+0000 7feb8d1d1740 1 -- start start 2024-04-25T00:17:06.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.755+0000 7feb8d1d1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559019088950 0x5590190a32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.755+0000 7feb8d1d1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55901908c630 0x5590190af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.755+0000 7feb8d1d1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55901908cf50 0x5590190afa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.755+0000 7feb8d1d1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feb841b35c0 con 0x559019088950 2024-04-25T00:17:06.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.755+0000 7feb8d1d1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7feb841b0e90 con 0x55901908c630 2024-04-25T00:17:06.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.755+0000 7feb8d1d1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feb841aece0 con 0x55901908cf50 2024-04-25T00:17:06.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.755+0000 7feb83fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559019088950 0x5590190a32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.755+0000 7feb83fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559019088950 0x5590190a32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38058/0 (socket says 172.21.15.107:38058) 2024-04-25T00:17:06.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.755+0000 7feb83fff640 1 -- 172.21.15.107:0/2984870062 learned_addr learned my addr 172.21.15.107:0/2984870062 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:06.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.755+0000 7feb88984640 1 --2- 172.21.15.107:0/2984870062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55901908cf50 0x5590190afa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.755+0000 7feb837fe640 1 --2- 172.21.15.107:0/2984870062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55901908c630 0x5590190af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.755+0000 7feb83fff640 1 -- 172.21.15.107:0/2984870062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55901908cf50 msgr2=0x5590190afa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:06.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.755+0000 7feb83fff640 1 --2- 172.21.15.107:0/2984870062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55901908cf50 0x5590190afa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.755+0000 7feb83fff640 1 -- 172.21.15.107:0/2984870062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55901908c630 msgr2=0x5590190af4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:06.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.755+0000 7feb83fff640 1 --2- 172.21.15.107:0/2984870062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55901908c630 0x5590190af4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.755+0000 7feb83fff640 1 -- 172.21.15.107:0/2984870062 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55901908cd80 con 0x559019088950 2024-04-25T00:17:06.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.756+0000 7feb83fff640 1 --2- 172.21.15.107:0/2984870062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559019088950 0x5590190a32d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7feb7400dd90 tx=0x7feb7401b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7ac5e564c461ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:06.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.756+0000 7feb69ffb640 1 -- 172.21.15.107:0/2984870062 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7403a070 con 0x559019088950 2024-04-25T00:17:06.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.756+0000 7feb69ffb640 1 -- 172.21.15.107:0/2984870062 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb7401bdd0 con 0x559019088950 2024-04-25T00:17:06.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.756+0000 7feb69ffb640 1 -- 172.21.15.107:0/2984870062 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb74020040 con 0x559019088950 2024-04-25T00:17:06.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.756+0000 7feb8d1d1740 1 -- 172.21.15.107:0/2984870062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559019088950 msgr2=0x5590190a32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:06.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.756+0000 7feb8d1d1740 1 --2- 172.21.15.107:0/2984870062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559019088950 0x5590190a32d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7feb7400dd90 tx=0x7feb7401b7c0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.756+0000 7feb8d1d1740 1 -- 172.21.15.107:0/2984870062 shutdown_connections 2024-04-25T00:17:06.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.756+0000 7feb8d1d1740 1 --2- 172.21.15.107:0/2984870062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55901908cf50 0x5590190afa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.756+0000 7feb8d1d1740 1 --2- 172.21.15.107:0/2984870062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55901908c630 0x5590190af4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.756+0000 7feb8d1d1740 1 --2- 172.21.15.107:0/2984870062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559019088950 0x5590190a32d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.756+0000 7feb8d1d1740 1 -- 172.21.15.107:0/2984870062 >> 172.21.15.107:0/2984870062 conn(0x5590190931e0 msgr2=0x559019087af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:06.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.757+0000 7feb8d1d1740 1 -- 172.21.15.107:0/2984870062 shutdown_connections 2024-04-25T00:17:06.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.757+0000 7feb8d1d1740 1 -- 172.21.15.107:0/2984870062 wait complete. 2024-04-25T00:17:06.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.757+0000 7feb8d1d1740 1 -- start start 2024-04-25T00:17:06.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.757+0000 7feb8d1d1740 1 Processor -- start 2024-04-25T00:17:06.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.757+0000 7feb8d1d1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559019088950 0x55901909fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.757+0000 7feb8d1d1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901908c630 0x55901909aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.757+0000 7feb8d1d1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55901908cf50 0x55901909b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.757+0000 7feb8d1d1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feb841af350 con 0x55901908c630 2024-04-25T00:17:06.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.757+0000 7feb8d1d1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7feb841b07c0 con 0x55901908cf50 2024-04-25T00:17:06.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.757+0000 7feb8d1d1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feb841afa20 con 0x559019088950 2024-04-25T00:17:06.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.757+0000 7feb8d1d1740 10 client.0 _ll_register_callbacks cb 0x7feb7c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:06.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.757+0000 7feb837fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901908c630 0x55901909aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.757+0000 7feb837fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901908c630 0x55901909aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38068/0 (socket says 172.21.15.107:38068) 2024-04-25T00:17:06.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.757+0000 7feb837fe640 1 -- 172.21.15.107:0/1756928640 learned_addr learned my addr 172.21.15.107:0/1756928640 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:06.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.758+0000 7feb83fff640 1 --2- 172.21.15.107:0/1756928640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559019088950 0x55901909fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.758+0000 7feb88984640 1 --2- 172.21.15.107:0/1756928640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55901908cf50 0x55901909b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.758+0000 7feb837fe640 1 -- 172.21.15.107:0/1756928640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559019088950 msgr2=0x55901909fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:06.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.758+0000 7feb837fe640 1 --2- 172.21.15.107:0/1756928640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559019088950 0x55901909fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.758+0000 7feb837fe640 1 -- 172.21.15.107:0/1756928640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55901908cf50 msgr2=0x55901909b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:06.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.758+0000 7feb837fe640 1 --2- 172.21.15.107:0/1756928640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55901908cf50 0x55901909b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.758+0000 7feb837fe640 1 -- 172.21.15.107:0/1756928640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55901909bbd0 con 0x55901908c630 2024-04-25T00:17:06.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.758+0000 7feb88984640 1 --2- 172.21.15.107:0/1756928640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55901908cf50 0x55901909b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:17:06.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.758+0000 7feb837fe640 1 --2- 172.21.15.107:0/1756928640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901908c630 0x55901909aee0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7feb780980e0 tx=0x7feb78009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746541e6f6000169 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:06.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.758+0000 7feb637fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:17:06.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.758+0000 7feb637fe640 1 -- 172.21.15.107:0/1756928640 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb78020040 con 0x55901908c630 2024-04-25T00:17:06.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.758+0000 7feb637fe640 1 -- 172.21.15.107:0/1756928640 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7feb780cf430 con 0x55901908c630 2024-04-25T00:17:06.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.758+0000 7feb8d1d1740 1 -- 172.21.15.107:0/1756928640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb7401bdd0 con 0x55901908c630 2024-04-25T00:17:06.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.758+0000 7feb637fe640 1 -- 172.21.15.107:0/1756928640 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7feb780b9050 con 0x55901908c630 2024-04-25T00:17:06.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.758+0000 7feb8d1d1740 10 client.15837 fetch_fsmap learned FSMap version 24 2024-04-25T00:17:06.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.758+0000 7feb8d1d1740 1 -- 172.21.15.107:0/1756928640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55901910dcf0 con 0x55901908c630 2024-04-25T00:17:06.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.758+0000 7feb637fe640 1 -- 172.21.15.107:0/1756928640 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb7401bdd0 con 0x55901908c630 2024-04-25T00:17:06.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.758+0000 7feb637fe640 1 -- 172.21.15.107:0/1756928640 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7feb78097460 con 0x55901908c630 2024-04-25T00:17:06.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.758+0000 7feb8d1d1740 10 client.15837 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:17:06.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.758+0000 7feb8d1d1740 10 client.15837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:06.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.758+0000 7feb8d1d1740 20 client.15837 populate_metadata read hostname 'smithi107' 2024-04-25T00:17:06.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.758+0000 7feb8d1d1740 1 --2- 172.21.15.107:0/1756928640 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x559019097620 0x5590191104b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:06.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.758+0000 7feb8d1d1740 4 client.15837 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:17:06.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.758+0000 7feb8d1d1740 1 -- 172.21.15.107:0/1756928640 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559018ff8f40 con 0x559019097620 2024-04-25T00:17:06.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.759+0000 7feb83fff640 1 --2- 172.21.15.107:0/1756928640 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x559019097620 0x5590191104b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:06.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.759+0000 7feb83fff640 1 --2- 172.21.15.107:0/1756928640 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x559019097620 0x5590191104b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3874a6f00350604 server_cookie=e77f8d5f303a3a0e in_seq=0 out_seq=0 2024-04-25T00:17:06.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.759+0000 7feb637fe640 10 client.15837 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:06.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.759+0000 7feb637fe640 1 -- 172.21.15.107:0/1756928640 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559018ff8f40 con 0x559019097620 2024-04-25T00:17:06.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.759+0000 7feb637fe640 10 client.15837 handle_command_reply: tid=0 2024-04-25T00:17:06.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.759+0000 7feb8d1d1740 1 client.15837 shutdown 2024-04-25T00:17:06.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.760+0000 7feb8d1d1740 1 -- 172.21.15.107:0/1756928640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901908c630 msgr2=0x55901909aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:06.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.760+0000 7feb8d1d1740 1 --2- 172.21.15.107:0/1756928640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901908c630 0x55901909aee0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7feb780980e0 tx=0x7feb78009ea0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.760+0000 7feb8d1d1740 1 -- 172.21.15.107:0/1756928640 shutdown_connections 2024-04-25T00:17:06.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.760+0000 7feb8d1d1740 1 --2- 172.21.15.107:0/1756928640 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x559019097620 0x5590191104b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.760+0000 7feb8d1d1740 1 --2- 172.21.15.107:0/1756928640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55901908cf50 0x55901909b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.760+0000 7feb8d1d1740 1 --2- 172.21.15.107:0/1756928640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901908c630 0x55901909aee0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.760+0000 7feb8d1d1740 1 --2- 172.21.15.107:0/1756928640 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x559019088950 0x55901909fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.760+0000 7feb637fe640 0 client.15837 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:06.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.760+0000 7feb8d1d1740 1 -- 172.21.15.107:0/1756928640 >> 172.21.15.107:0/1756928640 conn(0x5590190931e0 msgr2=0x55901908dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:06.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.760+0000 7feb8d1d1740 1 -- 172.21.15.107:0/1756928640 shutdown_connections 2024-04-25T00:17:06.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.760+0000 7feb8d1d1740 1 -- 172.21.15.107:0/1756928640 wait complete. 2024-04-25T00:17:06.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.760+0000 7feb8d1d1740 20 client.15837 trim_cache size 0 max 16384 2024-04-25T00:17:06.768 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:17:06.768 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:17:06.768 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-04-25T00:17:06.769 INFO:teuthology.orchestra.run.smithi107.stdout: "8d4089b7-4d8b-4532-be5c-cc1730e8b4a6": { 2024-04-25T00:17:06.769 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-04-25T00:17:06.769 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "8d4089b7-4d8b-4532-be5c-cc1730e8b4a6", 2024-04-25T00:17:06.769 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-04-25T00:17:06.769 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:17:06.769 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:17:06.769 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:17:06.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.760+0000 7feb8a3e7640 1 -- 172.21.15.107:0/3718903034 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7feb6c0a2eb0 msgr2=0x7feb6c0a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:06.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.760+0000 7feb8a3e7640 1 --2- 172.21.15.107:0/3718903034 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7feb6c0a2eb0 0x7feb6c0a5370 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feb84101750 tx=0x7feb74005db0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.760+0000 7feb8a3e7640 1 -- 172.21.15.107:0/3718903034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84100fd0 msgr2=0x7feb8419a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:06.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.760+0000 7feb8a3e7640 1 --2- 172.21.15.107:0/3718903034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84100fd0 0x7feb8419a5e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7feb7800d950 tx=0x7feb7800de10 comp rx=0 tx=0).stop 2024-04-25T00:17:06.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.760+0000 7feb8a3e7640 1 -- 172.21.15.107:0/3718903034 shutdown_connections 2024-04-25T00:17:06.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.760+0000 7feb8a3e7640 1 --2- 172.21.15.107:0/3718903034 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb841066b0 0x7feb8419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.760+0000 7feb8a3e7640 1 --2- 172.21.15.107:0/3718903034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb84100fd0 0x7feb8419a5e0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.760+0000 7feb8a3e7640 1 --2- 172.21.15.107:0/3718903034 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7feb6c0a2eb0 0x7feb6c0a5370 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.760+0000 7feb8a3e7640 1 --2- 172.21.15.107:0/3718903034 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb84100610 0x7feb8419a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:06.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.760+0000 7feb8a3e7640 1 -- 172.21.15.107:0/3718903034 >> 172.21.15.107:0/3718903034 conn(0x7feb84077740 msgr2=0x7feb840688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:06.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.760+0000 7feb8a3e7640 1 -- 172.21.15.107:0/3718903034 shutdown_connections 2024-04-25T00:17:06.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:06.760+0000 7feb8a3e7640 1 -- 172.21.15.107:0/3718903034 wait complete. 2024-04-25T00:17:06.772 INFO:tasks.cephfs.filesystem:scrub status for tag:8d4089b7-4d8b-4532-be5c-cc1730e8b4a6 - {'path': '/', 'tag': '8d4089b7-4d8b-4532-be5c-cc1730e8b4a6', 'options': 'recursive,force'} 2024-04-25T00:17:08.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:07 smithi170 ceph-mon[32807]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T00:17:08.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:07 smithi170 ceph-mon[32807]: scrub queued for path: / 2024-04-25T00:17:08.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:07 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:17:08.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:07 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:17:08.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:07 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:17:08.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:07 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:17:08.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:07 smithi107 ceph-mon[29153]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T00:17:08.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:07 smithi107 ceph-mon[29153]: scrub queued for path: / 2024-04-25T00:17:08.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:07 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:17:08.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:07 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:17:08.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:07 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:17:08.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:07 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:17:08.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:07 smithi195 ceph-mon[32833]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T00:17:08.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:07 smithi195 ceph-mon[32833]: scrub queued for path: / 2024-04-25T00:17:08.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:07 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:17:08.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:07 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:17:08.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:07 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:17:08.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:07 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:17:10.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:09 smithi170 ceph-mon[32807]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-25T00:17:10.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:09 smithi107 ceph-mon[29153]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-25T00:17:10.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:09 smithi195 ceph-mon[32833]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-25T00:17:12.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:11 smithi170 ceph-mon[32807]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T00:17:12.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:11 smithi107 ceph-mon[29153]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T00:17:12.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:11 smithi195 ceph-mon[32833]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T00:17:14.141 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:13 smithi195 ceph-mon[32833]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-25T00:17:14.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:13 smithi170 ceph-mon[32807]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-25T00:17:14.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:13 smithi107 ceph-mon[29153]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-25T00:17:16.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:15 smithi170 ceph-mon[32807]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-04-25T00:17:16.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:15 smithi107 ceph-mon[29153]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-04-25T00:17:16.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:15 smithi195 ceph-mon[32833]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-04-25T00:17:18.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:17 smithi170 ceph-mon[32807]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-25T00:17:18.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:17 smithi107 ceph-mon[29153]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-25T00:17:18.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:17 smithi195 ceph-mon[32833]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-25T00:17:20.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:19 smithi170 ceph-mon[32807]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-25T00:17:20.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:19 smithi107 ceph-mon[29153]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-25T00:17:20.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:19 smithi195 ceph-mon[32833]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-04-25T00:17:22.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:21 smithi170 ceph-mon[32807]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-25T00:17:22.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:21 smithi107 ceph-mon[29153]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-25T00:17:22.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:21 smithi195 ceph-mon[32833]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-25T00:17:24.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:23 smithi170 ceph-mon[32807]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-25T00:17:24.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:23 smithi107 ceph-mon[29153]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-25T00:17:24.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:23 smithi195 ceph-mon[32833]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-25T00:17:25.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:25 smithi170 ceph-mon[32807]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:17:26.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:25 smithi107 ceph-mon[29153]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:17:26.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:25 smithi195 ceph-mon[32833]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:17:28.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:27 smithi170 ceph-mon[32807]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:17:28.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:27 smithi107 ceph-mon[29153]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:17:28.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:27 smithi195 ceph-mon[32833]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:17:30.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:29 smithi170 ceph-mon[32807]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:17:30.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:29 smithi107 ceph-mon[29153]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:17:30.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:29 smithi195 ceph-mon[32833]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:17:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:31 smithi170 ceph-mon[32807]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:17:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:31 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:17:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:31 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:17:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:31 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:17:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:31 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:17:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:31 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:17:32.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:31 smithi107 ceph-mon[29153]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:17:32.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:31 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:17:32.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:31 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:17:32.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:31 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:17:32.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:31 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:17:32.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:31 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:17:32.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:31 smithi195 ceph-mon[32833]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:17:32.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:31 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:17:32.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:31 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:17:32.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:31 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:17:32.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:31 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:17:32.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:31 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:17:33.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:32 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:33.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:32 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:17:33.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:32 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:17:33.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:32 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:33.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:32 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:17:33.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:32 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:17:33.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:32 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:33.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:32 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:17:33.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:32 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:17:34.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:33 smithi170 ceph-mon[32807]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:17:34.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:33 smithi107 ceph-mon[29153]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:17:34.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:33 smithi195 ceph-mon[32833]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:17:36.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:35 smithi170 ceph-mon[32807]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:17:36.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:35 smithi107 ceph-mon[29153]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:17:36.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:35 smithi195 ceph-mon[32833]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:17:36.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:17:36.773 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:17:36.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.929+0000 7fb3709dc640 1 -- 172.21.15.107:0/427746980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb36c1066a0 msgr2=0x7fb36c106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:36.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.929+0000 7fb3709dc640 1 --2- 172.21.15.107:0/427746980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb36c1066a0 0x7fb36c106a80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009930 tx=0x7fb35c02a420 comp rx=0 tx=0).stop 2024-04-25T00:17:36.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.929+0000 7fb3709dc640 1 -- 172.21.15.107:0/427746980 shutdown_connections 2024-04-25T00:17:36.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.929+0000 7fb3709dc640 1 --2- 172.21.15.107:0/427746980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb36c1014e0 0x7fb36c10d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.929+0000 7fb3709dc640 1 --2- 172.21.15.107:0/427746980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb36c100b20 0x7fb36c100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.929+0000 7fb3709dc640 1 --2- 172.21.15.107:0/427746980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb36c1066a0 0x7fb36c106a80 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.929+0000 7fb3709dc640 1 -- 172.21.15.107:0/427746980 >> 172.21.15.107:0/427746980 conn(0x7fb36c0761c0 msgr2=0x7fb36c0765d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:36.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.929+0000 7fb3709dc640 1 -- 172.21.15.107:0/427746980 shutdown_connections 2024-04-25T00:17:36.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.929+0000 7fb3709dc640 1 -- 172.21.15.107:0/427746980 wait complete. 2024-04-25T00:17:36.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.930+0000 7fb3709dc640 1 Processor -- start 2024-04-25T00:17:36.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.930+0000 7fb3709dc640 1 -- start start 2024-04-25T00:17:36.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.930+0000 7fb3709dc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb36c100b20 0x7fb36c19a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:36.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.930+0000 7fb3709dc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb36c1014e0 0x7fb36c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:36.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.930+0000 7fb3709dc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb36c1066a0 0x7fb36c19ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:36.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.930+0000 7fb3709dc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb36c110a70 con 0x7fb36c1014e0 2024-04-25T00:17:36.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.930+0000 7fb3709dc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb36c1108f0 con 0x7fb36c100b20 2024-04-25T00:17:36.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.930+0000 7fb3709dc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb36c110bf0 con 0x7fb36c1066a0 2024-04-25T00:17:36.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.930+0000 7fb369d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb36c1014e0 0x7fb36c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:36.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.930+0000 7fb369d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb36c1014e0 0x7fb36c19a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51886/0 (socket says 172.21.15.107:51886) 2024-04-25T00:17:36.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.930+0000 7fb369d9b640 1 -- 172.21.15.107:0/1756605654 learned_addr learned my addr 172.21.15.107:0/1756605654 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:36.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.930+0000 7fb36a59c640 1 --2- 172.21.15.107:0/1756605654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb36c100b20 0x7fb36c19a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:36.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.930+0000 7fb369d9b640 1 -- 172.21.15.107:0/1756605654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb36c1066a0 msgr2=0x7fb36c19ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:36.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.930+0000 7fb369d9b640 1 --2- 172.21.15.107:0/1756605654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb36c1066a0 0x7fb36c19ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.930+0000 7fb369d9b640 1 -- 172.21.15.107:0/1756605654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb36c100b20 msgr2=0x7fb36c19a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:36.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.930+0000 7fb369d9b640 1 --2- 172.21.15.107:0/1756605654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb36c100b20 0x7fb36c19a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.931+0000 7fb369d9b640 1 -- 172.21.15.107:0/1756605654 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36c19f1c0 con 0x7fb36c1014e0 2024-04-25T00:17:36.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.931+0000 7fb369d9b640 1 --2- 172.21.15.107:0/1756605654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb36c1014e0 0x7fb36c19a710 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb35400e9f0 tx=0x7fb35400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:36.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.931+0000 7fb3637fe640 1 -- 172.21.15.107:0/1756605654 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35400cd80 con 0x7fb36c1014e0 2024-04-25T00:17:36.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.931+0000 7fb3637fe640 1 -- 172.21.15.107:0/1756605654 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb354005500 con 0x7fb36c1014e0 2024-04-25T00:17:36.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.932+0000 7fb3637fe640 1 -- 172.21.15.107:0/1756605654 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb354010690 con 0x7fb36c1014e0 2024-04-25T00:17:36.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.932+0000 7fb3709dc640 1 -- 172.21.15.107:0/1756605654 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb36c19f4b0 con 0x7fb36c1014e0 2024-04-25T00:17:36.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.932+0000 7fb3709dc640 1 -- 172.21.15.107:0/1756605654 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb36c1a6d40 con 0x7fb36c1014e0 2024-04-25T00:17:36.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.935+0000 7fb3737c9740 1 Processor -- start 2024-04-25T00:17:36.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.935+0000 7fb3737c9740 1 -- start start 2024-04-25T00:17:36.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.936+0000 7fb3637fe640 1 -- 172.21.15.107:0/1756605654 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7fb354004680 con 0x7fb36c1014e0 2024-04-25T00:17:36.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.936+0000 7fb3737c9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558821c4f630 0x558821c4fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:36.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.936+0000 7fb3737c9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c4ff50 0x558821c5a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:36.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.936+0000 7fb3737c9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821c5a620 0x558821c5ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:36.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.936+0000 7fb36a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c4ff50 0x558821c5a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:36.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.936+0000 7fb36a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c4ff50 0x558821c5a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51890/0 (socket says 172.21.15.107:51890) 2024-04-25T00:17:36.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.936+0000 7fb36a59c640 1 -- 172.21.15.107:0/2319363928 learned_addr learned my addr 172.21.15.107:0/2319363928 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:36.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.936+0000 7fb36ad9d640 1 --2- 172.21.15.107:0/2319363928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558821c4f630 0x558821c4fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:36.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.936+0000 7fb3737c9740 1 -- 172.21.15.107:0/2319363928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558821bb46b0 con 0x558821c4ff50 2024-04-25T00:17:36.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.936+0000 7fb3737c9740 1 -- 172.21.15.107:0/2319363928 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558821b5c880 con 0x558821c5a620 2024-04-25T00:17:36.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.936+0000 7fb3737c9740 1 -- 172.21.15.107:0/2319363928 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558821bb0670 con 0x558821c4f630 2024-04-25T00:17:36.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.937+0000 7fb369d9b640 1 --2- 172.21.15.107:0/2319363928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821c5a620 0x558821c5ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:36.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.937+0000 7fb36a59c640 1 -- 172.21.15.107:0/2319363928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558821c4f630 msgr2=0x558821c4fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:36.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.937+0000 7fb36a59c640 1 --2- 172.21.15.107:0/2319363928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558821c4f630 0x558821c4fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.937+0000 7fb36a59c640 1 -- 172.21.15.107:0/2319363928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821c5a620 msgr2=0x558821c5ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:36.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.937+0000 7fb36a59c640 1 --2- 172.21.15.107:0/2319363928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821c5a620 0x558821c5ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.937+0000 7fb36a59c640 1 -- 172.21.15.107:0/2319363928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588219927d0 con 0x558821c4ff50 2024-04-25T00:17:36.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.937+0000 7fb369d9b640 1 --2- 172.21.15.107:0/2319363928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821c5a620 0x558821c5ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:17:36.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.937+0000 7fb3637fe640 1 --2- 172.21.15.107:0/1756605654 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb3580a2d90 0x7fb3580a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:36.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.937+0000 7fb36a59c640 1 --2- 172.21.15.107:0/2319363928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c4ff50 0x558821c5a0e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb35c005ab0 tx=0x7fb35c005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79391019bdbdba1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:36.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.937+0000 7fb36ad9d640 1 --2- 172.21.15.107:0/1756605654 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb3580a2d90 0x7fb3580a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:36.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.937+0000 7fb360ff9640 1 -- 172.21.15.107:0/2319363928 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c03f070 con 0x558821c4ff50 2024-04-25T00:17:36.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.937+0000 7fb360ff9640 1 -- 172.21.15.107:0/2319363928 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb35c0043c0 con 0x558821c4ff50 2024-04-25T00:17:36.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.937+0000 7fb360ff9640 1 -- 172.21.15.107:0/2319363928 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35c035030 con 0x558821c4ff50 2024-04-25T00:17:36.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.937+0000 7fb3637fe640 1 -- 172.21.15.107:0/1756605654 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fb3540d22f0 con 0x7fb36c1014e0 2024-04-25T00:17:36.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.938+0000 7fb3737c9740 1 -- 172.21.15.107:0/2319363928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c4ff50 msgr2=0x558821c5a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:36.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.938+0000 7fb36ad9d640 1 --2- 172.21.15.107:0/1756605654 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb3580a2d90 0x7fb3580a5250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb364009270 tx=0x7fb364009200 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:36.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.938+0000 7fb3737c9740 1 --2- 172.21.15.107:0/2319363928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c4ff50 0x558821c5a0e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb35c005ab0 tx=0x7fb35c005de0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.938+0000 7fb3737c9740 1 -- 172.21.15.107:0/2319363928 shutdown_connections 2024-04-25T00:17:36.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.938+0000 7fb3737c9740 1 --2- 172.21.15.107:0/2319363928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821c5a620 0x558821c5ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.938+0000 7fb3737c9740 1 --2- 172.21.15.107:0/2319363928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c4ff50 0x558821c5a0e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.938+0000 7fb3737c9740 1 --2- 172.21.15.107:0/2319363928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558821c4f630 0x558821c4fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.938+0000 7fb3737c9740 1 -- 172.21.15.107:0/2319363928 >> 172.21.15.107:0/2319363928 conn(0x558821c4b950 msgr2=0x558821c4bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:36.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.939+0000 7fb3737c9740 1 -- 172.21.15.107:0/2319363928 shutdown_connections 2024-04-25T00:17:36.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.939+0000 7fb3737c9740 1 -- 172.21.15.107:0/2319363928 wait complete. 2024-04-25T00:17:36.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.939+0000 7fb3737c9740 1 -- start start 2024-04-25T00:17:36.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.939+0000 7fb3737c9740 1 Processor -- start 2024-04-25T00:17:36.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.940+0000 7fb3737c9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558821c4f630 0x558821c57580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:36.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.940+0000 7fb3737c9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821c4ff50 0x558821c57ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:36.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.940+0000 7fb3737c9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c5a620 0x558821c58000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:36.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.940+0000 7fb3737c9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558821c5d3b0 con 0x558821c5a620 2024-04-25T00:17:36.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.940+0000 7fb3737c9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558821b6ff50 con 0x558821c4ff50 2024-04-25T00:17:36.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.940+0000 7fb3737c9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558821c5d530 con 0x558821c4f630 2024-04-25T00:17:36.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.940+0000 7fb36ad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c5a620 0x558821c58000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:36.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.940+0000 7fb369d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821c4ff50 0x558821c57ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:36.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.940+0000 7fb36ad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c5a620 0x558821c58000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51904/0 (socket says 172.21.15.107:51904) 2024-04-25T00:17:36.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.940+0000 7fb36ad9d640 1 -- 172.21.15.107:0/314230522 learned_addr learned my addr 172.21.15.107:0/314230522 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:36.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.940+0000 7fb36a59c640 1 --2- 172.21.15.107:0/314230522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558821c4f630 0x558821c57580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:36.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.940+0000 7fb3737c9740 10 client.0 _ll_register_callbacks cb 0x558821bab000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:36.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.940+0000 7fb36ad9d640 1 -- 172.21.15.107:0/314230522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558821c4f630 msgr2=0x558821c57580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:36.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.940+0000 7fb36ad9d640 1 --2- 172.21.15.107:0/314230522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558821c4f630 0x558821c57580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.941+0000 7fb36ad9d640 1 -- 172.21.15.107:0/314230522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821c4ff50 msgr2=0x558821c57ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:36.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.941+0000 7fb36ad9d640 1 --2- 172.21.15.107:0/314230522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821c4ff50 0x558821c57ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.941+0000 7fb36ad9d640 1 -- 172.21.15.107:0/314230522 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588218c83f0 con 0x558821c5a620 2024-04-25T00:17:36.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.941+0000 7fb369d9b640 1 --2- 172.21.15.107:0/314230522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821c4ff50 0x558821c57ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:36.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.941+0000 7fb36a59c640 1 --2- 172.21.15.107:0/314230522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558821c4f630 0x558821c57580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:36.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.941+0000 7fb36ad9d640 1 --2- 172.21.15.107:0/314230522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c5a620 0x558821c58000 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb364004090 tx=0x7fb364004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7ddf5f91b0ccee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:36.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.941+0000 7fb351ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:17:36.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.941+0000 7fb351ffb640 1 -- 172.21.15.107:0/314230522 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb364004d20 con 0x558821c5a620 2024-04-25T00:17:36.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.941+0000 7fb3737c9740 1 -- 172.21.15.107:0/314230522 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb35c0043c0 con 0x558821c5a620 2024-04-25T00:17:36.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.941+0000 7fb351ffb640 1 -- 172.21.15.107:0/314230522 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fb36400f850 con 0x558821c5a620 2024-04-25T00:17:36.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.941+0000 7fb351ffb640 1 -- 172.21.15.107:0/314230522 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb36400fa50 con 0x558821c5a620 2024-04-25T00:17:36.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.941+0000 7fb3737c9740 10 client.15849 fetch_fsmap learned FSMap version 24 2024-04-25T00:17:36.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.941+0000 7fb3737c9740 1 -- 172.21.15.107:0/314230522 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5588218880f0 con 0x558821c5a620 2024-04-25T00:17:36.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.942+0000 7fb351ffb640 1 -- 172.21.15.107:0/314230522 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb35c0043c0 con 0x558821c5a620 2024-04-25T00:17:36.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.942+0000 7fb351ffb640 1 -- 172.21.15.107:0/314230522 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb364030420 con 0x558821c5a620 2024-04-25T00:17:36.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.942+0000 7fb3737c9740 10 client.15849 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:17:36.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.942+0000 7fb3737c9740 10 client.15849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:36.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.942+0000 7fb3737c9740 20 client.15849 populate_metadata read hostname 'smithi107' 2024-04-25T00:17:36.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.942+0000 7fb3737c9740 1 --2- 172.21.15.107:0/314230522 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x558821c6a800 0x558821c6cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:36.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.942+0000 7fb3737c9740 4 client.15849 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:17:36.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.942+0000 7fb3737c9740 1 -- 172.21.15.107:0/314230522 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558821bc1bf0 con 0x558821c6a800 2024-04-25T00:17:36.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.942+0000 7fb36a59c640 1 --2- 172.21.15.107:0/314230522 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x558821c6a800 0x558821c6cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:36.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.943+0000 7fb36a59c640 1 --2- 172.21.15.107:0/314230522 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x558821c6a800 0x558821c6cc80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2582d4a8caa45bfb server_cookie=dfe1bb2e3a0c5d14 in_seq=0 out_seq=0 2024-04-25T00:17:36.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.943+0000 7fb351ffb640 10 client.15849 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:36.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.944+0000 7fb351ffb640 1 -- 172.21.15.107:0/314230522 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558821bc1bf0 con 0x558821c6a800 2024-04-25T00:17:36.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.944+0000 7fb351ffb640 10 client.15849 handle_command_reply: tid=0 2024-04-25T00:17:36.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.944+0000 7fb3737c9740 1 client.15849 shutdown 2024-04-25T00:17:36.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.944+0000 7fb3737c9740 1 -- 172.21.15.107:0/314230522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c5a620 msgr2=0x558821c58000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:36.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.944+0000 7fb3737c9740 1 --2- 172.21.15.107:0/314230522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c5a620 0x558821c58000 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb364004090 tx=0x7fb364004a40 comp rx=0 tx=0).stop 2024-04-25T00:17:36.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.944+0000 7fb3737c9740 1 -- 172.21.15.107:0/314230522 shutdown_connections 2024-04-25T00:17:36.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.944+0000 7fb3737c9740 1 --2- 172.21.15.107:0/314230522 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x558821c6a800 0x558821c6cc80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.944+0000 7fb3737c9740 1 --2- 172.21.15.107:0/314230522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c5a620 0x558821c58000 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.944+0000 7fb3737c9740 1 --2- 172.21.15.107:0/314230522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821c4ff50 0x558821c57ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.944+0000 7fb3737c9740 1 --2- 172.21.15.107:0/314230522 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558821c4f630 0x558821c57580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.944+0000 7fb3737c9740 1 -- 172.21.15.107:0/314230522 >> 172.21.15.107:0/314230522 conn(0x558821c4b950 msgr2=0x558821c51750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:36.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.944+0000 7fb351ffb640 0 client.15849 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:36.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.944+0000 7fb3737c9740 1 -- 172.21.15.107:0/314230522 shutdown_connections 2024-04-25T00:17:36.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.944+0000 7fb3737c9740 1 -- 172.21.15.107:0/314230522 wait complete. 2024-04-25T00:17:36.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.944+0000 7fb3737c9740 20 client.15849 trim_cache size 0 max 16384 2024-04-25T00:17:36.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.979+0000 7fb3737c9740 1 Processor -- start 2024-04-25T00:17:36.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.979+0000 7fb3737c9740 1 -- start start 2024-04-25T00:17:36.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.979+0000 7fb3737c9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558821c4f630 0x558821c66270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:36.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.979+0000 7fb3737c9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c4ff50 0x558821c51ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:36.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.979+0000 7fb3737c9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821c5a620 0x558821c72550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:36.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.979+0000 7fb3737c9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb36c1b33a0 con 0x558821c4ff50 2024-04-25T00:17:36.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.979+0000 7fb3737c9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb36c1b0c70 con 0x558821c5a620 2024-04-25T00:17:36.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.979+0000 7fb3737c9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb36c1aeac0 con 0x558821c4f630 2024-04-25T00:17:36.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.979+0000 7fb369d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c4ff50 0x558821c51ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:36.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.979+0000 7fb369d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c4ff50 0x558821c51ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51912/0 (socket says 172.21.15.107:51912) 2024-04-25T00:17:36.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.979+0000 7fb369d9b640 1 -- 172.21.15.107:0/1090202151 learned_addr learned my addr 172.21.15.107:0/1090202151 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:36.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.979+0000 7fb36a59c640 1 --2- 172.21.15.107:0/1090202151 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558821c4f630 0x558821c66270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:36.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.979+0000 7fb36ad9d640 1 --2- 172.21.15.107:0/1090202151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821c5a620 0x558821c72550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:36.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.979+0000 7fb369d9b640 1 -- 172.21.15.107:0/1090202151 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558821c4f630 msgr2=0x558821c66270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:36.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.979+0000 7fb369d9b640 1 --2- 172.21.15.107:0/1090202151 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558821c4f630 0x558821c66270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.979+0000 7fb369d9b640 1 -- 172.21.15.107:0/1090202151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821c5a620 msgr2=0x558821c72550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:36.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.979+0000 7fb369d9b640 1 --2- 172.21.15.107:0/1090202151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821c5a620 0x558821c72550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.979+0000 7fb369d9b640 1 -- 172.21.15.107:0/1090202151 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558821c4fd80 con 0x558821c4ff50 2024-04-25T00:17:36.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.979+0000 7fb36a59c640 1 --2- 172.21.15.107:0/1090202151 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558821c4f630 0x558821c66270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:36.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.980+0000 7fb369d9b640 1 --2- 172.21.15.107:0/1090202151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c4ff50 0x558821c51ea0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb35400c6a0 tx=0x7fb354086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e3184e7d168287 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:36.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.980+0000 7fb353fff640 1 -- 172.21.15.107:0/1090202151 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3540b4070 con 0x558821c4ff50 2024-04-25T00:17:36.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.980+0000 7fb353fff640 1 -- 172.21.15.107:0/1090202151 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb354004970 con 0x558821c4ff50 2024-04-25T00:17:36.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.980+0000 7fb353fff640 1 -- 172.21.15.107:0/1090202151 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35409a960 con 0x558821c4ff50 2024-04-25T00:17:36.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.981+0000 7fb3737c9740 1 -- 172.21.15.107:0/1090202151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c4ff50 msgr2=0x558821c51ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:36.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.981+0000 7fb3737c9740 1 --2- 172.21.15.107:0/1090202151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c4ff50 0x558821c51ea0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb35400c6a0 tx=0x7fb354086040 comp rx=0 tx=0).stop 2024-04-25T00:17:36.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.981+0000 7fb3737c9740 1 -- 172.21.15.107:0/1090202151 shutdown_connections 2024-04-25T00:17:36.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.981+0000 7fb3737c9740 1 --2- 172.21.15.107:0/1090202151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821c5a620 0x558821c72550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.981+0000 7fb3737c9740 1 --2- 172.21.15.107:0/1090202151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c4ff50 0x558821c51ea0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.981+0000 7fb3737c9740 1 --2- 172.21.15.107:0/1090202151 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558821c4f630 0x558821c66270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.981+0000 7fb3737c9740 1 -- 172.21.15.107:0/1090202151 >> 172.21.15.107:0/1090202151 conn(0x558821c4b950 msgr2=0x558821c55e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:36.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.981+0000 7fb3737c9740 1 -- 172.21.15.107:0/1090202151 shutdown_connections 2024-04-25T00:17:36.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.981+0000 7fb3737c9740 1 -- 172.21.15.107:0/1090202151 wait complete. 2024-04-25T00:17:36.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.981+0000 7fb3737c9740 1 -- start start 2024-04-25T00:17:36.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.981+0000 7fb3737c9740 1 Processor -- start 2024-04-25T00:17:36.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.982+0000 7fb3737c9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558821c4f630 0x558821c55260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:36.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.982+0000 7fb3737c9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c4ff50 0x558821c62780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:36.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.982+0000 7fb3737c9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821c5a620 0x558821c53830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:36.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.982+0000 7fb3737c9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb36c1af130 con 0x558821c4ff50 2024-04-25T00:17:36.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.982+0000 7fb3737c9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb36c1b05a0 con 0x558821c5a620 2024-04-25T00:17:36.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.982+0000 7fb3737c9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb36c1af800 con 0x558821c4f630 2024-04-25T00:17:36.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.982+0000 7fb369d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c4ff50 0x558821c62780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:36.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.982+0000 7fb369d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c4ff50 0x558821c62780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51926/0 (socket says 172.21.15.107:51926) 2024-04-25T00:17:36.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.982+0000 7fb369d9b640 1 -- 172.21.15.107:0/2164706570 learned_addr learned my addr 172.21.15.107:0/2164706570 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:36.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.982+0000 7fb3737c9740 10 client.0 _ll_register_callbacks cb 0x7fb364009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:36.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.982+0000 7fb36a59c640 1 --2- 172.21.15.107:0/2164706570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558821c4f630 0x558821c55260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:36.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.982+0000 7fb369d9b640 1 -- 172.21.15.107:0/2164706570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558821c4f630 msgr2=0x558821c55260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:36.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.982+0000 7fb369d9b640 1 --2- 172.21.15.107:0/2164706570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558821c4f630 0x558821c55260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.982+0000 7fb369d9b640 1 -- 172.21.15.107:0/2164706570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821c5a620 msgr2=0x558821c53830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:17:36.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.982+0000 7fb369d9b640 1 --2- 172.21.15.107:0/2164706570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821c5a620 0x558821c53830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.982+0000 7fb369d9b640 1 -- 172.21.15.107:0/2164706570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558821c53d70 con 0x558821c4ff50 2024-04-25T00:17:36.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.982+0000 7fb369d9b640 1 --2- 172.21.15.107:0/2164706570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c4ff50 0x558821c62780 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb35409b4a0 tx=0x7fb354097170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc012b88f1a936b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:36.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.982+0000 7fb351ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:17:36.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.982+0000 7fb351ffb640 1 -- 172.21.15.107:0/2164706570 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3540b8070 con 0x558821c4ff50 2024-04-25T00:17:36.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.982+0000 7fb351ffb640 1 -- 172.21.15.107:0/2164706570 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fb354097460 con 0x558821c4ff50 2024-04-25T00:17:36.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.982+0000 7fb3737c9740 1 -- 172.21.15.107:0/2164706570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb354004970 con 0x558821c4ff50 2024-04-25T00:17:36.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.982+0000 7fb351ffb640 1 -- 172.21.15.107:0/2164706570 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb3540ae050 con 0x558821c4ff50 2024-04-25T00:17:36.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.983+0000 7fb3737c9740 10 client.15855 fetch_fsmap learned FSMap version 24 2024-04-25T00:17:36.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.983+0000 7fb3737c9740 1 -- 172.21.15.107:0/2164706570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x558821cd8c00 con 0x558821c4ff50 2024-04-25T00:17:36.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.983+0000 7fb351ffb640 1 -- 172.21.15.107:0/2164706570 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb354004970 con 0x558821c4ff50 2024-04-25T00:17:36.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.983+0000 7fb351ffb640 1 -- 172.21.15.107:0/2164706570 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb35409a3d0 con 0x558821c4ff50 2024-04-25T00:17:36.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.983+0000 7fb3737c9740 10 client.15855 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:17:36.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.983+0000 7fb3737c9740 10 client.15855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:36.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.983+0000 7fb3737c9740 20 client.15855 populate_metadata read hostname 'smithi107' 2024-04-25T00:17:36.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.983+0000 7fb3737c9740 1 --2- 172.21.15.107:0/2164706570 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x558821cd9240 0x558821cdb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:36.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.983+0000 7fb3737c9740 4 client.15855 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:17:36.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.983+0000 7fb3737c9740 1 -- 172.21.15.107:0/2164706570 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558821bbbf40 con 0x558821cd9240 2024-04-25T00:17:36.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.983+0000 7fb36a59c640 1 --2- 172.21.15.107:0/2164706570 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x558821cd9240 0x558821cdb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:36.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.984+0000 7fb36a59c640 1 --2- 172.21.15.107:0/2164706570 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x558821cd9240 0x558821cdb660 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7da871f48502d750 server_cookie=43d651e1c7db5bdd in_seq=0 out_seq=0 2024-04-25T00:17:36.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.984+0000 7fb351ffb640 10 client.15855 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:36.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.984+0000 7fb351ffb640 1 -- 172.21.15.107:0/2164706570 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558821bbbf40 con 0x558821cd9240 2024-04-25T00:17:36.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.984+0000 7fb351ffb640 10 client.15855 handle_command_reply: tid=0 2024-04-25T00:17:36.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.984+0000 7fb3737c9740 1 client.15855 shutdown 2024-04-25T00:17:36.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.984+0000 7fb3737c9740 1 -- 172.21.15.107:0/2164706570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c4ff50 msgr2=0x558821c62780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.984+0000 7fb3737c9740 1 --2- 172.21.15.107:0/2164706570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c4ff50 0x558821c62780 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb35409b4a0 tx=0x7fb354097170 comp rx=0 tx=0).stop 2024-04-25T00:17:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.984+0000 7fb3737c9740 1 -- 172.21.15.107:0/2164706570 shutdown_connections 2024-04-25T00:17:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.984+0000 7fb3737c9740 1 --2- 172.21.15.107:0/2164706570 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x558821cd9240 0x558821cdb660 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.984+0000 7fb351ffb640 0 client.15855 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.984+0000 7fb3737c9740 1 --2- 172.21.15.107:0/2164706570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558821c5a620 0x558821c53830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.984+0000 7fb3737c9740 1 --2- 172.21.15.107:0/2164706570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558821c4ff50 0x558821c62780 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.984+0000 7fb3737c9740 1 --2- 172.21.15.107:0/2164706570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558821c4f630 0x558821c55260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.984+0000 7fb3737c9740 1 -- 172.21.15.107:0/2164706570 >> 172.21.15.107:0/2164706570 conn(0x558821c4b950 msgr2=0x558821c57540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.984+0000 7fb3737c9740 1 -- 172.21.15.107:0/2164706570 shutdown_connections 2024-04-25T00:17:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.984+0000 7fb3737c9740 1 -- 172.21.15.107:0/2164706570 wait complete. 2024-04-25T00:17:36.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.985+0000 7fb3737c9740 20 client.15855 trim_cache size 0 max 16384 2024-04-25T00:17:36.992 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:17:36.992 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-04-25T00:17:36.992 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-04-25T00:17:36.992 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:17:36.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.985+0000 7fb3709dc640 1 -- 172.21.15.107:0/1756605654 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb3580a2d90 msgr2=0x7fb3580a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:36.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.985+0000 7fb3709dc640 1 --2- 172.21.15.107:0/1756605654 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb3580a2d90 0x7fb3580a5250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb364009270 tx=0x7fb364009200 comp rx=0 tx=0).stop 2024-04-25T00:17:36.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.985+0000 7fb3709dc640 1 -- 172.21.15.107:0/1756605654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb36c1014e0 msgr2=0x7fb36c19a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:36.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.985+0000 7fb3709dc640 1 --2- 172.21.15.107:0/1756605654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb36c1014e0 0x7fb36c19a710 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb35400e9f0 tx=0x7fb35400edf0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.985+0000 7fb3709dc640 1 -- 172.21.15.107:0/1756605654 shutdown_connections 2024-04-25T00:17:36.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.985+0000 7fb3709dc640 1 --2- 172.21.15.107:0/1756605654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb36c1066a0 0x7fb36c19ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.985+0000 7fb3709dc640 1 --2- 172.21.15.107:0/1756605654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb36c1014e0 0x7fb36c19a710 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.985+0000 7fb3709dc640 1 --2- 172.21.15.107:0/1756605654 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb3580a2d90 0x7fb3580a5250 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.985+0000 7fb3709dc640 1 --2- 172.21.15.107:0/1756605654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb36c100b20 0x7fb36c19a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.985+0000 7fb3709dc640 1 -- 172.21.15.107:0/1756605654 >> 172.21.15.107:0/1756605654 conn(0x7fb36c0761c0 msgr2=0x7fb36c0699d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:36.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.986+0000 7fb3709dc640 1 -- 172.21.15.107:0/1756605654 shutdown_connections 2024-04-25T00:17:36.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:36.986+0000 7fb3709dc640 1 -- 172.21.15.107:0/1756605654 wait complete. 2024-04-25T00:17:37.000 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:17:37.000 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:17:37.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.101+0000 7efdaf3e9640 1 -- 172.21.15.107:0/2298122628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efda8103340 msgr2=0x7efda810b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.101+0000 7efdaf3e9640 1 --2- 172.21.15.107:0/2298122628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efda8103340 0x7efda810b820 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7efd98009900 tx=0x7efd9802a480 comp rx=0 tx=0).stop 2024-04-25T00:17:37.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.101+0000 7efdaf3e9640 1 -- 172.21.15.107:0/2298122628 shutdown_connections 2024-04-25T00:17:37.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.101+0000 7efdaf3e9640 1 --2- 172.21.15.107:0/2298122628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efda810bd60 0x7efda810e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.101+0000 7efdaf3e9640 1 --2- 172.21.15.107:0/2298122628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efda8103340 0x7efda810b820 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.101+0000 7efdaf3e9640 1 --2- 172.21.15.107:0/2298122628 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efda8102a20 0x7efda8102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.101+0000 7efdaf3e9640 1 -- 172.21.15.107:0/2298122628 >> 172.21.15.107:0/2298122628 conn(0x7efda80777a0 msgr2=0x7efda8077bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:37.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.101+0000 7efdaf3e9640 1 -- 172.21.15.107:0/2298122628 shutdown_connections 2024-04-25T00:17:37.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.101+0000 7efdaf3e9640 1 -- 172.21.15.107:0/2298122628 wait complete. 2024-04-25T00:17:37.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.102+0000 7efdaf3e9640 1 Processor -- start 2024-04-25T00:17:37.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.102+0000 7efdaf3e9640 1 -- start start 2024-04-25T00:17:37.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.102+0000 7efdaf3e9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efda8102a20 0x7efda8198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.102+0000 7efdaf3e9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efda8103340 0x7efda8198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.102+0000 7efdaf3e9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efda810bd60 0x7efda819c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.102+0000 7efdaf3e9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efda8111b30 con 0x7efda810bd60 2024-04-25T00:17:37.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.102+0000 7efdaf3e9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efda81119b0 con 0x7efda8102a20 2024-04-25T00:17:37.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.102+0000 7efdaf3e9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efda8111cb0 con 0x7efda8103340 2024-04-25T00:17:37.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.102+0000 7efdad986640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efda810bd60 0x7efda819c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.102+0000 7efdad986640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efda810bd60 0x7efda819c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51948/0 (socket says 172.21.15.107:51948) 2024-04-25T00:17:37.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.102+0000 7efdad986640 1 -- 172.21.15.107:0/2711038863 learned_addr learned my addr 172.21.15.107:0/2711038863 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:37.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.102+0000 7efdad185640 1 --2- 172.21.15.107:0/2711038863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efda8102a20 0x7efda8198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.102+0000 7efdac984640 1 --2- 172.21.15.107:0/2711038863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efda8103340 0x7efda8198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.103+0000 7efdad185640 1 -- 172.21.15.107:0/2711038863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efda8103340 msgr2=0x7efda8198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.103+0000 7efdad185640 1 --2- 172.21.15.107:0/2711038863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efda8103340 0x7efda8198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.103+0000 7efdad185640 1 -- 172.21.15.107:0/2711038863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efda810bd60 msgr2=0x7efda819c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.103+0000 7efdad185640 1 --2- 172.21.15.107:0/2711038863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efda810bd60 0x7efda819c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.103+0000 7efdad185640 1 -- 172.21.15.107:0/2711038863 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda819cff0 con 0x7efda8102a20 2024-04-25T00:17:37.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.103+0000 7efdad185640 1 --2- 172.21.15.107:0/2711038863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efda8102a20 0x7efda8198050 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efda400e9f0 tx=0x7efda400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:37.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.103+0000 7efd9e7fc640 1 -- 172.21.15.107:0/2711038863 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda400ccc0 con 0x7efda8102a20 2024-04-25T00:17:37.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.103+0000 7efdaf3e9640 1 -- 172.21.15.107:0/2711038863 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda819d2e0 con 0x7efda8102a20 2024-04-25T00:17:37.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.103+0000 7efd9e7fc640 1 -- 172.21.15.107:0/2711038863 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efda4005020 con 0x7efda8102a20 2024-04-25T00:17:37.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.103+0000 7efd9e7fc640 1 -- 172.21.15.107:0/2711038863 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda40049e0 con 0x7efda8102a20 2024-04-25T00:17:37.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.103+0000 7efdaf3e9640 1 -- 172.21.15.107:0/2711038863 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda81005a0 con 0x7efda8102a20 2024-04-25T00:17:37.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.105+0000 7efdaf3e9640 1 -- 172.21.15.107:0/2711038863 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efda80689b0 con 0x7efda8102a20 2024-04-25T00:17:37.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.105+0000 7efd9e7fc640 1 -- 172.21.15.107:0/2711038863 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7efda4005560 con 0x7efda8102a20 2024-04-25T00:17:37.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.106+0000 7efd9e7fc640 1 --2- 172.21.15.107:0/2711038863 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7efd940a2eb0 0x7efd940a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.106+0000 7efdac984640 1 --2- 172.21.15.107:0/2711038863 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7efd940a2eb0 0x7efd940a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.106+0000 7efd9e7fc640 1 -- 172.21.15.107:0/2711038863 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7efda40cfaa0 con 0x7efda8102a20 2024-04-25T00:17:37.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.107+0000 7efdac984640 1 --2- 172.21.15.107:0/2711038863 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7efd940a2eb0 0x7efd940a5370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efda810daa0 tx=0x7efd9803c040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:37.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.111+0000 7efd9e7fc640 1 -- 172.21.15.107:0/2711038863 <== mon.2 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efda40051d0 con 0x7efda8102a20 2024-04-25T00:17:37.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.436+0000 7efdaf3e9640 1 -- 172.21.15.107:0/2711038863 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efda8106ee0 con 0x7efda8102a20 2024-04-25T00:17:37.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.437+0000 7efd9e7fc640 1 -- 172.21.15.107:0/2711038863 <== mon.2 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11897 (secure 0 0 0) 0x7efda409c110 con 0x7efda8102a20 2024-04-25T00:17:37.437 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:17:37.437 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:16:24.724978+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24493,24521]},"id":1}]} 2024-04-25T00:17:37.437 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 24 2024-04-25T00:17:37.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.439+0000 7efdaf3e9640 1 -- 172.21.15.107:0/2711038863 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7efd940a2eb0 msgr2=0x7efd940a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.439+0000 7efdaf3e9640 1 --2- 172.21.15.107:0/2711038863 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7efd940a2eb0 0x7efd940a5370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efda810daa0 tx=0x7efd9803c040 comp rx=0 tx=0).stop 2024-04-25T00:17:37.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.439+0000 7efdaf3e9640 1 -- 172.21.15.107:0/2711038863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efda8102a20 msgr2=0x7efda8198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.439+0000 7efdaf3e9640 1 --2- 172.21.15.107:0/2711038863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efda8102a20 0x7efda8198050 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efda400e9f0 tx=0x7efda400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.439+0000 7efdaf3e9640 1 -- 172.21.15.107:0/2711038863 shutdown_connections 2024-04-25T00:17:37.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.439+0000 7efdaf3e9640 1 --2- 172.21.15.107:0/2711038863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efda810bd60 0x7efda819c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.439+0000 7efdaf3e9640 1 --2- 172.21.15.107:0/2711038863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efda8103340 0x7efda8198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.439+0000 7efdaf3e9640 1 --2- 172.21.15.107:0/2711038863 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7efd940a2eb0 0x7efd940a5370 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.439+0000 7efdaf3e9640 1 --2- 172.21.15.107:0/2711038863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efda8102a20 0x7efda8198050 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.439+0000 7efdaf3e9640 1 -- 172.21.15.107:0/2711038863 >> 172.21.15.107:0/2711038863 conn(0x7efda80777a0 msgr2=0x7efda80fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:37.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.439+0000 7efdaf3e9640 1 -- 172.21.15.107:0/2711038863 shutdown_connections 2024-04-25T00:17:37.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.439+0000 7efdaf3e9640 1 -- 172.21.15.107:0/2711038863 wait complete. 2024-04-25T00:17:37.454 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:17:37.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.556+0000 7f763142a640 1 -- 172.21.15.107:0/494999881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f762c101790 msgr2=0x7f762c101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.556+0000 7f763142a640 1 --2- 172.21.15.107:0/494999881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f762c101790 0x7f762c101c10 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7620009930 tx=0x7f762002a420 comp rx=0 tx=0).stop 2024-04-25T00:17:37.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.556+0000 7f763142a640 1 -- 172.21.15.107:0/494999881 shutdown_connections 2024-04-25T00:17:37.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.556+0000 7f763142a640 1 --2- 172.21.15.107:0/494999881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f762c102150 0x7f762c10e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.556+0000 7f763142a640 1 --2- 172.21.15.107:0/494999881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f762c101790 0x7f762c101c10 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.556+0000 7f763142a640 1 --2- 172.21.15.107:0/494999881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f762c1077a0 0x7f762c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.556+0000 7f763142a640 1 -- 172.21.15.107:0/494999881 >> 172.21.15.107:0/494999881 conn(0x7f762c0fd2a0 msgr2=0x7f762c0ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:37.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.556+0000 7f763142a640 1 -- 172.21.15.107:0/494999881 shutdown_connections 2024-04-25T00:17:37.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.556+0000 7f763142a640 1 -- 172.21.15.107:0/494999881 wait complete. 2024-04-25T00:17:37.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.557+0000 7f763142a640 1 Processor -- start 2024-04-25T00:17:37.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.557+0000 7f763142a640 1 -- start start 2024-04-25T00:17:37.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.557+0000 7f763142a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f762c101790 0x7f762c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.557+0000 7f763142a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f762c102150 0x7f762c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.557+0000 7f763142a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f762c1077a0 0x7f762c19c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.557+0000 7f763142a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f762c111c00 con 0x7f762c102150 2024-04-25T00:17:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.557+0000 7f763142a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f762c111a80 con 0x7f762c101790 2024-04-25T00:17:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.557+0000 7f763142a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f762c111d80 con 0x7f762c1077a0 2024-04-25T00:17:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.557+0000 7f762a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f762c102150 0x7f762c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.557+0000 7f762a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f762c102150 0x7f762c198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51972/0 (socket says 172.21.15.107:51972) 2024-04-25T00:17:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.557+0000 7f762a7fc640 1 -- 172.21.15.107:0/2128228454 learned_addr learned my addr 172.21.15.107:0/2128228454 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.557+0000 7f762affd640 1 --2- 172.21.15.107:0/2128228454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f762c101790 0x7f762c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.557+0000 7f762b7fe640 1 --2- 172.21.15.107:0/2128228454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f762c1077a0 0x7f762c19c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.557+0000 7f762a7fc640 1 -- 172.21.15.107:0/2128228454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f762c1077a0 msgr2=0x7f762c19c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.557+0000 7f762a7fc640 1 --2- 172.21.15.107:0/2128228454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f762c1077a0 0x7f762c19c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.557+0000 7f762a7fc640 1 -- 172.21.15.107:0/2128228454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f762c101790 msgr2=0x7f762c197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.557+0000 7f762a7fc640 1 --2- 172.21.15.107:0/2128228454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f762c101790 0x7f762c197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.557+0000 7f762a7fc640 1 -- 172.21.15.107:0/2128228454 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762c19ceb0 con 0x7f762c102150 2024-04-25T00:17:37.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.558+0000 7f762a7fc640 1 --2- 172.21.15.107:0/2128228454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f762c102150 0x7f762c198400 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f762002a940 tx=0x7f76200309c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:37.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.558+0000 7f7613fff640 1 -- 172.21.15.107:0/2128228454 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762003f070 con 0x7f762c102150 2024-04-25T00:17:37.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.558+0000 7f763142a640 1 -- 172.21.15.107:0/2128228454 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f762c19d140 con 0x7f762c102150 2024-04-25T00:17:37.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.558+0000 7f7613fff640 1 -- 172.21.15.107:0/2128228454 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7620030dc0 con 0x7f762c102150 2024-04-25T00:17:37.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.558+0000 7f763142a640 1 -- 172.21.15.107:0/2128228454 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f762c1a4a30 con 0x7f762c102150 2024-04-25T00:17:37.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.558+0000 7f7613fff640 1 -- 172.21.15.107:0/2128228454 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7620035030 con 0x7f762c102150 2024-04-25T00:17:37.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.561+0000 7f7613fff640 1 -- 172.21.15.107:0/2128228454 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f7620042070 con 0x7f762c102150 2024-04-25T00:17:37.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.561+0000 7f7634212740 1 Processor -- start 2024-04-25T00:17:37.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.561+0000 7f7634212740 1 -- start start 2024-04-25T00:17:37.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.561+0000 7f7634212740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f8f580 0x55a015f8f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.561+0000 7f7634212740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a015f8fea0 0x55a015f9a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.562+0000 7f7613fff640 1 --2- 172.21.15.107:0/2128228454 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f76140a2fb0 0x7f76140a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.562+0000 7f7634212740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a015f9a570 0x55a015f9c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.562+0000 7f7634212740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a015ef4670 con 0x55a015f8f580 2024-04-25T00:17:37.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.562+0000 7f762b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f8f580 0x55a015f8f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.562+0000 7f762a7fc640 1 --2- 172.21.15.107:0/2128228454 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f76140a2fb0 0x7f76140a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.562+0000 7f762b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f8f580 0x55a015f8f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51984/0 (socket says 172.21.15.107:51984) 2024-04-25T00:17:37.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.562+0000 7f762b7fe640 1 -- 172.21.15.107:0/2430308163 learned_addr learned my addr 172.21.15.107:0/2430308163 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:37.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.562+0000 7f7613fff640 1 -- 172.21.15.107:0/2128228454 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f76200f05b0 con 0x7f762c102150 2024-04-25T00:17:37.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.562+0000 7f762affd640 1 --2- 172.21.15.107:0/2430308163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a015f8fea0 0x55a015f9a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.562+0000 7f7634212740 1 -- 172.21.15.107:0/2430308163 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a015e9c970 con 0x55a015f9a570 2024-04-25T00:17:37.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.562+0000 7f7634212740 1 -- 172.21.15.107:0/2430308163 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a015ef0630 con 0x55a015f8fea0 2024-04-25T00:17:37.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.562+0000 7f762a7fc640 1 --2- 172.21.15.107:0/2128228454 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f76140a2fb0 0x7f76140a5470 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f76200ed7b0 tx=0x7f7620030f70 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:37.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.562+0000 7f762affd640 1 -- 172.21.15.107:0/2430308163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a015f9a570 msgr2=0x55a015f9c960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:17:37.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.562+0000 7f762affd640 1 --2- 172.21.15.107:0/2430308163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a015f9a570 0x55a015f9c960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.562+0000 7f762affd640 1 -- 172.21.15.107:0/2430308163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f8f580 msgr2=0x55a015f8f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:37.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.563+0000 7f762affd640 1 --2- 172.21.15.107:0/2430308163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f8f580 0x55a015f8f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.563+0000 7f762affd640 1 -- 172.21.15.107:0/2430308163 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a015ec0400 con 0x55a015f8fea0 2024-04-25T00:17:37.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.563+0000 7f762b7fe640 1 --2- 172.21.15.107:0/2430308163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f8f580 0x55a015f8f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:37.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.563+0000 7f762affd640 1 --2- 172.21.15.107:0/2430308163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a015f8fea0 0x55a015f9a030 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f761c00bb70 tx=0x7f761c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73b931d0b808185d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:37.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.563+0000 7f76117fa640 1 -- 172.21.15.107:0/2430308163 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c004030 con 0x55a015f8fea0 2024-04-25T00:17:37.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.563+0000 7f76117fa640 1 -- 172.21.15.107:0/2430308163 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f761c0041d0 con 0x55a015f8fea0 2024-04-25T00:17:37.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.563+0000 7f76117fa640 1 -- 172.21.15.107:0/2430308163 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c0116a0 con 0x55a015f8fea0 2024-04-25T00:17:37.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.564+0000 7f7634212740 1 -- 172.21.15.107:0/2430308163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a015f8fea0 msgr2=0x55a015f9a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.564+0000 7f7634212740 1 --2- 172.21.15.107:0/2430308163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a015f8fea0 0x55a015f9a030 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f761c00bb70 tx=0x7f761c0107a0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.564+0000 7f7634212740 1 -- 172.21.15.107:0/2430308163 shutdown_connections 2024-04-25T00:17:37.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.564+0000 7f7634212740 1 --2- 172.21.15.107:0/2430308163 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a015f9a570 0x55a015f9c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.564+0000 7f7634212740 1 --2- 172.21.15.107:0/2430308163 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a015f8fea0 0x55a015f9a030 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.564+0000 7f7634212740 1 --2- 172.21.15.107:0/2430308163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f8f580 0x55a015f8f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.564+0000 7f7634212740 1 -- 172.21.15.107:0/2430308163 >> 172.21.15.107:0/2430308163 conn(0x55a015f8b8a0 msgr2=0x55a015f8bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:37.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.564+0000 7f7634212740 1 -- 172.21.15.107:0/2430308163 shutdown_connections 2024-04-25T00:17:37.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.564+0000 7f7634212740 1 -- 172.21.15.107:0/2430308163 wait complete. 2024-04-25T00:17:37.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.564+0000 7f7634212740 1 -- start start 2024-04-25T00:17:37.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.565+0000 7f7634212740 1 Processor -- start 2024-04-25T00:17:37.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.565+0000 7f7634212740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a015f8f580 0x55a015f99fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.565+0000 7f7634212740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a015f8fea0 0x55a015f97290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.565+0000 7f7634212740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f9a570 0x55a015f979d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.565+0000 7f7634212740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a015f9d360 con 0x55a015f9a570 2024-04-25T00:17:37.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.565+0000 7f7634212740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a015eb0050 con 0x55a015f8f580 2024-04-25T00:17:37.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.565+0000 7f7634212740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a015f9d4e0 con 0x55a015f8fea0 2024-04-25T00:17:37.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.565+0000 7f7634212740 10 client.0 _ll_register_callbacks cb 0x55a015c07be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:37.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.565+0000 7f762b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f9a570 0x55a015f979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.565+0000 7f762b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f9a570 0x55a015f979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52000/0 (socket says 172.21.15.107:52000) 2024-04-25T00:17:37.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.565+0000 7f762b7fe640 1 -- 172.21.15.107:0/3204027905 learned_addr learned my addr 172.21.15.107:0/3204027905 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:37.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.565+0000 7f762affd640 1 --2- 172.21.15.107:0/3204027905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a015f8fea0 0x55a015f97290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.565+0000 7f762b7fe640 1 -- 172.21.15.107:0/3204027905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a015f8fea0 msgr2=0x55a015f97290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:37.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.565+0000 7f762b7fe640 1 --2- 172.21.15.107:0/3204027905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a015f8fea0 0x55a015f97290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.565+0000 7f762b7fe640 1 -- 172.21.15.107:0/3204027905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a015f8f580 msgr2=0x55a015f99fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:17:37.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.565+0000 7f762b7fe640 1 --2- 172.21.15.107:0/3204027905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a015f8f580 0x55a015f99fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.565+0000 7f762b7fe640 1 -- 172.21.15.107:0/3204027905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a015e81dc0 con 0x55a015f9a570 2024-04-25T00:17:37.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.566+0000 7f762b7fe640 1 --2- 172.21.15.107:0/3204027905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f9a570 0x55a015f979d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7624004630 tx=0x7f7624011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486796fe703731ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:37.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.566+0000 7f760eff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:17:37.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.566+0000 7f760eff5640 1 -- 172.21.15.107:0/3204027905 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624011e60 con 0x55a015f9a570 2024-04-25T00:17:37.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.566+0000 7f760eff5640 1 -- 172.21.15.107:0/3204027905 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f7624009d70 con 0x55a015f9a570 2024-04-25T00:17:37.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.566+0000 7f7634212740 1 -- 172.21.15.107:0/3204027905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f761c0041d0 con 0x55a015f9a570 2024-04-25T00:17:37.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.566+0000 7f760eff5640 1 -- 172.21.15.107:0/3204027905 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7624004720 con 0x55a015f9a570 2024-04-25T00:17:37.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.566+0000 7f7634212740 10 client.15873 fetch_fsmap learned FSMap version 24 2024-04-25T00:17:37.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.566+0000 7f7634212740 1 -- 172.21.15.107:0/3204027905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a015c083f0 con 0x55a015f9a570 2024-04-25T00:17:37.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.566+0000 7f760eff5640 1 -- 172.21.15.107:0/3204027905 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f761c0041d0 con 0x55a015f9a570 2024-04-25T00:17:37.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.567+0000 7f760eff5640 1 -- 172.21.15.107:0/3204027905 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f7624010560 con 0x55a015f9a570 2024-04-25T00:17:37.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.567+0000 7f7634212740 10 client.15873 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:17:37.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.567+0000 7f7634212740 10 client.15873 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:17:37.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.567+0000 7f7634212740 20 client.15873 populate_metadata read hostname 'smithi107' 2024-04-25T00:17:37.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.567+0000 7f7634212740 1 --2- 172.21.15.107:0/3204027905 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55a015faeb30 0x55a015fb0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.567+0000 7f7634212740 4 client.15873 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:17:37.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.567+0000 7f7634212740 1 -- 172.21.15.107:0/3204027905 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a015f01bb0 con 0x55a015faeb30 2024-04-25T00:17:37.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.567+0000 7f762affd640 1 --2- 172.21.15.107:0/3204027905 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55a015faeb30 0x55a015fb0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.567+0000 7f760eff5640 10 client.15873 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:17:37.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.567+0000 7f762affd640 1 --2- 172.21.15.107:0/3204027905 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55a015faeb30 0x55a015fb0f20 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8368bd98954e29eb server_cookie=83a0e9e464359a77 in_seq=0 out_seq=0 2024-04-25T00:17:37.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.568+0000 7f760eff5640 1 -- 172.21.15.107:0/3204027905 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a015f01bb0 con 0x55a015faeb30 2024-04-25T00:17:37.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.568+0000 7f760eff5640 10 client.15873 handle_command_reply: tid=0 2024-04-25T00:17:37.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.568+0000 7f7634212740 1 client.15873 shutdown 2024-04-25T00:17:37.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.568+0000 7f7634212740 1 -- 172.21.15.107:0/3204027905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f9a570 msgr2=0x55a015f979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.568+0000 7f7634212740 1 --2- 172.21.15.107:0/3204027905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f9a570 0x55a015f979d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7624004630 tx=0x7f7624011810 comp rx=0 tx=0).stop 2024-04-25T00:17:37.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.568+0000 7f7634212740 1 -- 172.21.15.107:0/3204027905 shutdown_connections 2024-04-25T00:17:37.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.568+0000 7f7634212740 1 --2- 172.21.15.107:0/3204027905 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55a015faeb30 0x55a015fb0f20 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.568+0000 7f7634212740 1 --2- 172.21.15.107:0/3204027905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f9a570 0x55a015f979d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.568+0000 7f760eff5640 0 client.15873 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:17:37.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.568+0000 7f7634212740 1 --2- 172.21.15.107:0/3204027905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a015f8fea0 0x55a015f97290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.569+0000 7f7634212740 1 --2- 172.21.15.107:0/3204027905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a015f8f580 0x55a015f99fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.569+0000 7f7634212740 1 -- 172.21.15.107:0/3204027905 >> 172.21.15.107:0/3204027905 conn(0x55a015f8b8a0 msgr2=0x55a015f916a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:37.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.569+0000 7f7634212740 1 -- 172.21.15.107:0/3204027905 shutdown_connections 2024-04-25T00:17:37.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.569+0000 7f7634212740 1 -- 172.21.15.107:0/3204027905 wait complete. 2024-04-25T00:17:37.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.569+0000 7f7634212740 20 client.15873 trim_cache size 0 max 16384 2024-04-25T00:17:37.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.582+0000 7f7634212740 1 Processor -- start 2024-04-25T00:17:37.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.582+0000 7f7634212740 1 -- start start 2024-04-25T00:17:37.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.582+0000 7f7634212740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a015f8f580 0x55a015fa6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.582+0000 7f7634212740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f8fea0 0x55a015f962e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.582+0000 7f7634212740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a015f9a570 0x55a015f926e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.582+0000 7f7634212740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f762c1af0c0 con 0x55a015f8fea0 2024-04-25T00:17:37.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.582+0000 7f7634212740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f762c1ac990 con 0x55a015f8f580 2024-04-25T00:17:37.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.582+0000 7f7634212740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f762c1aa7e0 con 0x55a015f9a570 2024-04-25T00:17:37.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.582+0000 7f762affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f8fea0 0x55a015f962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.582+0000 7f762affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f8fea0 0x55a015f962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52006/0 (socket says 172.21.15.107:52006) 2024-04-25T00:17:37.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.582+0000 7f762affd640 1 -- 172.21.15.107:0/312077661 learned_addr learned my addr 172.21.15.107:0/312077661 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:37.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.582+0000 7f762b7fe640 1 --2- 172.21.15.107:0/312077661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a015f8f580 0x55a015fa6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.582+0000 7f762affd640 1 -- 172.21.15.107:0/312077661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a015f9a570 msgr2=0x55a015f926e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:37.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.582+0000 7f762affd640 1 --2- 172.21.15.107:0/312077661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a015f9a570 0x55a015f926e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.583+0000 7f762affd640 1 -- 172.21.15.107:0/312077661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a015f8f580 msgr2=0x55a015fa6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:37.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.583+0000 7f762affd640 1 --2- 172.21.15.107:0/312077661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a015f8f580 0x55a015fa6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.583+0000 7f762affd640 1 -- 172.21.15.107:0/312077661 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a015f8fcd0 con 0x55a015f8fea0 2024-04-25T00:17:37.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.583+0000 7f762b7fe640 1 --2- 172.21.15.107:0/312077661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a015f8f580 0x55a015fa6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:17:37.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.583+0000 7f762affd640 1 --2- 172.21.15.107:0/312077661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f8fea0 0x55a015f962e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f761c011010 tx=0x7f761c00beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b8ecd950aa8f11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:37.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.583+0000 7f7610ff9640 1 -- 172.21.15.107:0/312077661 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c0055f0 con 0x55a015f8fea0 2024-04-25T00:17:37.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.583+0000 7f7610ff9640 1 -- 172.21.15.107:0/312077661 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f761c003ad0 con 0x55a015f8fea0 2024-04-25T00:17:37.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.583+0000 7f7610ff9640 1 -- 172.21.15.107:0/312077661 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c011440 con 0x55a015f8fea0 2024-04-25T00:17:37.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.584+0000 7f7634212740 1 -- 172.21.15.107:0/312077661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f8fea0 msgr2=0x55a015f962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.584+0000 7f7634212740 1 --2- 172.21.15.107:0/312077661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f8fea0 0x55a015f962e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f761c011010 tx=0x7f761c00beb0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.584+0000 7f7634212740 1 -- 172.21.15.107:0/312077661 shutdown_connections 2024-04-25T00:17:37.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.584+0000 7f7634212740 1 --2- 172.21.15.107:0/312077661 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a015f9a570 0x55a015f926e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.584+0000 7f7634212740 1 --2- 172.21.15.107:0/312077661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f8fea0 0x55a015f962e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.584+0000 7f7634212740 1 --2- 172.21.15.107:0/312077661 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a015f8f580 0x55a015fa6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.584+0000 7f7634212740 1 -- 172.21.15.107:0/312077661 >> 172.21.15.107:0/312077661 conn(0x55a015f8b8a0 msgr2=0x55a015f91fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:37.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.584+0000 7f7634212740 1 -- 172.21.15.107:0/312077661 shutdown_connections 2024-04-25T00:17:37.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.584+0000 7f7634212740 1 -- 172.21.15.107:0/312077661 wait complete. 2024-04-25T00:17:37.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.584+0000 7f7634212740 1 -- start start 2024-04-25T00:17:37.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.584+0000 7f7634212740 1 Processor -- start 2024-04-25T00:17:37.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.585+0000 7f7634212740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f8f580 0x55a015fa2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.585+0000 7f762b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f8f580 0x55a015fa2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.585+0000 7f762b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f8f580 0x55a015fa2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52020/0 (socket says 172.21.15.107:52020) 2024-04-25T00:17:37.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.585+0000 7f762b7fe640 1 -- 172.21.15.107:0/1646922750 learned_addr learned my addr 172.21.15.107:0/1646922750 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:37.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.585+0000 7f7634212740 1 --2- 172.21.15.107:0/1646922750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a015f9a570 0x55a015fa2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.585+0000 7f7634212740 1 --2- 172.21.15.107:0/1646922750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a015f9dc60 0x55a015f9e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.585+0000 7f7634212740 1 -- 172.21.15.107:0/1646922750 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f762c1aae50 con 0x55a015f8f580 2024-04-25T00:17:37.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.585+0000 7f7634212740 1 -- 172.21.15.107:0/1646922750 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f762c1ac2c0 con 0x55a015f9dc60 2024-04-25T00:17:37.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.585+0000 7f7634212740 1 -- 172.21.15.107:0/1646922750 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f762c1ab520 con 0x55a015f9a570 2024-04-25T00:17:37.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.585+0000 7f7634212740 10 client.0 _ll_register_callbacks cb 0x7f7624009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:37.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.585+0000 7f762b7fe640 1 -- 172.21.15.107:0/1646922750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a015f9a570 msgr2=0x55a015fa2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:37.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.586+0000 7f762affd640 1 --2- 172.21.15.107:0/1646922750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a015f9a570 0x55a015fa2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.586+0000 7f762b7fe640 1 --2- 172.21.15.107:0/1646922750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a015f9a570 0x55a015fa2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.586+0000 7f762b7fe640 1 -- 172.21.15.107:0/1646922750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a015f9dc60 msgr2=0x55a015f9e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:37.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.586+0000 7f762b7fe640 1 --2- 172.21.15.107:0/1646922750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a015f9dc60 0x55a015f9e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.586+0000 7f762b7fe640 1 -- 172.21.15.107:0/1646922750 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a015cd27d0 con 0x55a015f8f580 2024-04-25T00:17:37.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.586+0000 7f762affd640 1 --2- 172.21.15.107:0/1646922750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a015f9a570 0x55a015fa2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:17:37.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.586+0000 7f762b7fe640 1 --2- 172.21.15.107:0/1646922750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f8f580 0x55a015fa2780 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x55a015f978b0 tx=0x7f762401bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f61ef074fa0a292 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:37.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.586+0000 7f760eff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:17:37.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.586+0000 7f760eff5640 1 -- 172.21.15.107:0/1646922750 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624010440 con 0x55a015f8f580 2024-04-25T00:17:37.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.586+0000 7f760eff5640 1 -- 172.21.15.107:0/1646922750 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f7624017030 con 0x55a015f8f580 2024-04-25T00:17:37.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.586+0000 7f7634212740 1 -- 172.21.15.107:0/1646922750 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f761c003ad0 con 0x55a015f8f580 2024-04-25T00:17:37.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.586+0000 7f760eff5640 1 -- 172.21.15.107:0/1646922750 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7624038050 con 0x55a015f8f580 2024-04-25T00:17:37.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.586+0000 7f7634212740 10 client.15879 fetch_fsmap learned FSMap version 24 2024-04-25T00:17:37.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.586+0000 7f7634212740 1 -- 172.21.15.107:0/1646922750 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a016014e90 con 0x55a015f8f580 2024-04-25T00:17:37.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.587+0000 7f760eff5640 1 -- 172.21.15.107:0/1646922750 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f761c003ad0 con 0x55a015f8f580 2024-04-25T00:17:37.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.587+0000 7f760eff5640 1 -- 172.21.15.107:0/1646922750 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f7624004b70 con 0x55a015f8f580 2024-04-25T00:17:37.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.587+0000 7f7634212740 10 client.15879 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:17:37.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.587+0000 7f7634212740 10 client.15879 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:17:37.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.587+0000 7f7634212740 20 client.15879 populate_metadata read hostname 'smithi107' 2024-04-25T00:17:37.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.587+0000 7f7634212740 1 --2- 172.21.15.107:0/1646922750 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55a016015610 0x55a016017a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.587+0000 7f7634212740 4 client.15879 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:17:37.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.587+0000 7f7634212740 1 -- 172.21.15.107:0/1646922750 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a015efbf00 con 0x55a016015610 2024-04-25T00:17:37.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.587+0000 7f762affd640 1 --2- 172.21.15.107:0/1646922750 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55a016015610 0x55a016017a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.588+0000 7f762affd640 1 --2- 172.21.15.107:0/1646922750 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55a016015610 0x55a016017a30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db914c8fd2314356 server_cookie=d87fe07104d4223a in_seq=0 out_seq=0 2024-04-25T00:17:37.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.588+0000 7f760eff5640 10 client.15879 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:17:37.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.588+0000 7f760eff5640 1 -- 172.21.15.107:0/1646922750 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a015efbf00 con 0x55a016015610 2024-04-25T00:17:37.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.588+0000 7f760eff5640 10 client.15879 handle_command_reply: tid=0 2024-04-25T00:17:37.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.588+0000 7f7634212740 1 client.15879 shutdown 2024-04-25T00:17:37.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.588+0000 7f7634212740 1 -- 172.21.15.107:0/1646922750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f8f580 msgr2=0x55a015fa2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.588+0000 7f7634212740 1 --2- 172.21.15.107:0/1646922750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f8f580 0x55a015fa2780 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x55a015f978b0 tx=0x7f762401bc30 comp rx=0 tx=0).stop 2024-04-25T00:17:37.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.588+0000 7f7634212740 1 -- 172.21.15.107:0/1646922750 shutdown_connections 2024-04-25T00:17:37.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.588+0000 7f7634212740 1 --2- 172.21.15.107:0/1646922750 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55a016015610 0x55a016017a30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.588+0000 7f7634212740 1 --2- 172.21.15.107:0/1646922750 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a015f9dc60 0x55a015f9e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.588+0000 7f7634212740 1 --2- 172.21.15.107:0/1646922750 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a015f9a570 0x55a015fa2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.588+0000 7f760eff5640 0 client.15879 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:17:37.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.588+0000 7f7634212740 1 --2- 172.21.15.107:0/1646922750 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a015f8f580 0x55a015fa2780 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.588+0000 7f7634212740 1 -- 172.21.15.107:0/1646922750 >> 172.21.15.107:0/1646922750 conn(0x55a015f8b8a0 msgr2=0x55a015f96160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:37.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.588+0000 7f7634212740 1 -- 172.21.15.107:0/1646922750 shutdown_connections 2024-04-25T00:17:37.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.588+0000 7f7634212740 1 -- 172.21.15.107:0/1646922750 wait complete. 2024-04-25T00:17:37.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.588+0000 7f7634212740 20 client.15879 trim_cache size 0 max 16384 2024-04-25T00:17:37.596 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:17:37.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.589+0000 7f763142a640 1 -- 172.21.15.107:0/2128228454 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f76140a2fb0 msgr2=0x7f76140a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.589+0000 7f763142a640 1 --2- 172.21.15.107:0/2128228454 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f76140a2fb0 0x7f76140a5470 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f76200ed7b0 tx=0x7f7620030f70 comp rx=0 tx=0).stop 2024-04-25T00:17:37.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.589+0000 7f763142a640 1 -- 172.21.15.107:0/2128228454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f762c102150 msgr2=0x7f762c198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.589+0000 7f763142a640 1 --2- 172.21.15.107:0/2128228454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f762c102150 0x7f762c198400 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f762002a940 tx=0x7f76200309c0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.589+0000 7f763142a640 1 -- 172.21.15.107:0/2128228454 shutdown_connections 2024-04-25T00:17:37.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.589+0000 7f763142a640 1 --2- 172.21.15.107:0/2128228454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f762c1077a0 0x7f762c19c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.589+0000 7f763142a640 1 --2- 172.21.15.107:0/2128228454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f762c102150 0x7f762c198400 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.589+0000 7f763142a640 1 --2- 172.21.15.107:0/2128228454 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f76140a2fb0 0x7f76140a5470 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.589+0000 7f763142a640 1 --2- 172.21.15.107:0/2128228454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f762c101790 0x7f762c197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.589+0000 7f763142a640 1 -- 172.21.15.107:0/2128228454 >> 172.21.15.107:0/2128228454 conn(0x7f762c0fd2a0 msgr2=0x7f762c0fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:37.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.589+0000 7f763142a640 1 -- 172.21.15.107:0/2128228454 shutdown_connections 2024-04-25T00:17:37.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.589+0000 7f763142a640 1 -- 172.21.15.107:0/2128228454 wait complete. 2024-04-25T00:17:37.600 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:17:37.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.701+0000 7fb9f88b3640 1 -- 172.21.15.107:0/831739888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f4106630 msgr2=0x7fb9f4106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.701+0000 7fb9f88b3640 1 --2- 172.21.15.107:0/831739888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f4106630 0x7fb9f4106a10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009930 tx=0x7fb9e402a420 comp rx=0 tx=0).stop 2024-04-25T00:17:37.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.701+0000 7fb9f88b3640 1 -- 172.21.15.107:0/831739888 shutdown_connections 2024-04-25T00:17:37.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.701+0000 7fb9f88b3640 1 --2- 172.21.15.107:0/831739888 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9f4100fe0 0x7fb9f410d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.701+0000 7fb9f88b3640 1 --2- 172.21.15.107:0/831739888 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9f4100620 0x7fb9f4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.701+0000 7fb9f88b3640 1 --2- 172.21.15.107:0/831739888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f4106630 0x7fb9f4106a10 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.701+0000 7fb9f88b3640 1 -- 172.21.15.107:0/831739888 >> 172.21.15.107:0/831739888 conn(0x7fb9f40769c0 msgr2=0x7fb9f4076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:37.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.701+0000 7fb9f88b3640 1 -- 172.21.15.107:0/831739888 shutdown_connections 2024-04-25T00:17:37.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.701+0000 7fb9f88b3640 1 -- 172.21.15.107:0/831739888 wait complete. 2024-04-25T00:17:37.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.702+0000 7fb9f88b3640 1 Processor -- start 2024-04-25T00:17:37.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.702+0000 7fb9f88b3640 1 -- start start 2024-04-25T00:17:37.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.703+0000 7fb9f88b3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9f4100620 0x7fb9f4196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.703+0000 7fb9f88b3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9f4100fe0 0x7fb9f4197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.703+0000 7fb9f88b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f4106630 0x7fb9f419b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.703+0000 7fb9f88b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f4110a00 con 0x7fb9f4106630 2024-04-25T00:17:37.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.703+0000 7fb9f88b3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f4110880 con 0x7fb9f4100620 2024-04-25T00:17:37.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.703+0000 7fb9f88b3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f4110b80 con 0x7fb9f4100fe0 2024-04-25T00:17:37.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.703+0000 7fb9f2d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f4106630 0x7fb9f419b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.703+0000 7fb9f2d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f4106630 0x7fb9f419b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52042/0 (socket says 172.21.15.107:52042) 2024-04-25T00:17:37.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.703+0000 7fb9f2d9d640 1 -- 172.21.15.107:0/2812629268 learned_addr learned my addr 172.21.15.107:0/2812629268 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:37.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.703+0000 7fb9f1d9b640 1 --2- 172.21.15.107:0/2812629268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9f4100fe0 0x7fb9f4197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.703+0000 7fb9f259c640 1 --2- 172.21.15.107:0/2812629268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9f4100620 0x7fb9f4196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.703+0000 7fb9f2d9d640 1 -- 172.21.15.107:0/2812629268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9f4100fe0 msgr2=0x7fb9f4197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.703+0000 7fb9f2d9d640 1 --2- 172.21.15.107:0/2812629268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9f4100fe0 0x7fb9f4197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.703+0000 7fb9f2d9d640 1 -- 172.21.15.107:0/2812629268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9f4100620 msgr2=0x7fb9f4196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.703+0000 7fb9f2d9d640 1 --2- 172.21.15.107:0/2812629268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9f4100620 0x7fb9f4196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.703+0000 7fb9f2d9d640 1 -- 172.21.15.107:0/2812629268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f419bf10 con 0x7fb9f4106630 2024-04-25T00:17:37.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.703+0000 7fb9f2d9d640 1 --2- 172.21.15.107:0/2812629268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f4106630 0x7fb9f419b7e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00cce0 tx=0x7fb9ec00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:37.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.704+0000 7fb9e37fe640 1 -- 172.21.15.107:0/2812629268 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec00f450 con 0x7fb9f4106630 2024-04-25T00:17:37.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.704+0000 7fb9e37fe640 1 -- 172.21.15.107:0/2812629268 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9ec00fad0 con 0x7fb9f4106630 2024-04-25T00:17:37.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.704+0000 7fb9f88b3640 1 -- 172.21.15.107:0/2812629268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9f419c200 con 0x7fb9f4106630 2024-04-25T00:17:37.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.704+0000 7fb9e37fe640 1 -- 172.21.15.107:0/2812629268 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec018c10 con 0x7fb9f4106630 2024-04-25T00:17:37.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.704+0000 7fb9f88b3640 1 -- 172.21.15.107:0/2812629268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9f4069d80 con 0x7fb9f4106630 2024-04-25T00:17:37.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.707+0000 7fb9fb5bf740 1 Processor -- start 2024-04-25T00:17:37.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.707+0000 7fb9e37fe640 1 -- 172.21.15.107:0/2812629268 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7fb9ec00f5f0 con 0x7fb9f4106630 2024-04-25T00:17:37.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.708+0000 7fb9fb5bf740 1 -- start start 2024-04-25T00:17:37.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.708+0000 7fb9fb5bf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1a42d0 0x55566c1a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.708+0000 7fb9fb5bf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55566c1a4c80 0x55566c1b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.708+0000 7fb9fb5bf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55566c1b3980 0x55566c1b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.708+0000 7fb9fb5bf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55566c109670 con 0x55566c1a42d0 2024-04-25T00:17:37.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.708+0000 7fb9fb5bf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55566c0b1970 con 0x55566c1b3980 2024-04-25T00:17:37.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.708+0000 7fb9fb5bf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55566c105630 con 0x55566c1a4c80 2024-04-25T00:17:37.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.708+0000 7fb9e37fe640 1 --2- 172.21.15.107:0/2812629268 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb9dc0a2eb0 0x7fb9dc0a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.708+0000 7fb9f1d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1a42d0 0x55566c1a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.708+0000 7fb9f1d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1a42d0 0x55566c1a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52052/0 (socket says 172.21.15.107:52052) 2024-04-25T00:17:37.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.708+0000 7fb9f1d9b640 1 -- 172.21.15.107:0/3422171454 learned_addr learned my addr 172.21.15.107:0/3422171454 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:37.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.708+0000 7fb9f259c640 1 --2- 172.21.15.107:0/3422171454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55566c1a4c80 0x55566c1b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.708+0000 7fb9e37fe640 1 -- 172.21.15.107:0/2812629268 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fb9ec0d0020 con 0x7fb9f4106630 2024-04-25T00:17:37.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.708+0000 7fb9f2d9d640 1 --2- 172.21.15.107:0/2812629268 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb9dc0a2eb0 0x7fb9dc0a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.708+0000 7fb9f259c640 1 -- 172.21.15.107:0/3422171454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55566c1b3980 msgr2=0x55566c1b5d70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:17:37.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.708+0000 7fb9f259c640 1 --2- 172.21.15.107:0/3422171454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55566c1b3980 0x55566c1b5d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.708+0000 7fb9f259c640 1 -- 172.21.15.107:0/3422171454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1a42d0 msgr2=0x55566c1a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:37.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.709+0000 7fb9f259c640 1 --2- 172.21.15.107:0/3422171454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1a42d0 0x55566c1a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.709+0000 7fb9f259c640 1 -- 172.21.15.107:0/3422171454 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55566c0d5400 con 0x55566c1a4c80 2024-04-25T00:17:37.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.709+0000 7fb9f1d9b640 1 --2- 172.21.15.107:0/3422171454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1a42d0 0x55566c1a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:37.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.709+0000 7fb9f259c640 1 --2- 172.21.15.107:0/3422171454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55566c1a4c80 0x55566c1b3440 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4005940 tx=0x7fb9e4031630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a1945ae79f6cc5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:37.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.709+0000 7fb9f2d9d640 1 --2- 172.21.15.107:0/2812629268 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb9dc0a2eb0 0x7fb9dc0a5370 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0cd220 tx=0x7fb9ec0185b0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:37.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.709+0000 7fb9e0ff9640 1 -- 172.21.15.107:0/3422171454 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4040070 con 0x55566c1a4c80 2024-04-25T00:17:37.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.709+0000 7fb9e0ff9640 1 -- 172.21.15.107:0/3422171454 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9e4005500 con 0x55566c1a4c80 2024-04-25T00:17:37.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.709+0000 7fb9e0ff9640 1 -- 172.21.15.107:0/3422171454 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4035040 con 0x55566c1a4c80 2024-04-25T00:17:37.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.710+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/3422171454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55566c1a4c80 msgr2=0x55566c1b3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.710+0000 7fb9fb5bf740 1 --2- 172.21.15.107:0/3422171454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55566c1a4c80 0x55566c1b3440 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4005940 tx=0x7fb9e4031630 comp rx=0 tx=0).stop 2024-04-25T00:17:37.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.710+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/3422171454 shutdown_connections 2024-04-25T00:17:37.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.710+0000 7fb9fb5bf740 1 --2- 172.21.15.107:0/3422171454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55566c1b3980 0x55566c1b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.710+0000 7fb9fb5bf740 1 --2- 172.21.15.107:0/3422171454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55566c1a4c80 0x55566c1b3440 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.710+0000 7fb9fb5bf740 1 --2- 172.21.15.107:0/3422171454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1a42d0 0x55566c1a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.710+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/3422171454 >> 172.21.15.107:0/3422171454 conn(0x55566c1a08a0 msgr2=0x55566c1a0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:37.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.710+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/3422171454 shutdown_connections 2024-04-25T00:17:37.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.710+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/3422171454 wait complete. 2024-04-25T00:17:37.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.711+0000 7fb9fb5bf740 1 -- start start 2024-04-25T00:17:37.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.711+0000 7fb9fb5bf740 1 Processor -- start 2024-04-25T00:17:37.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.711+0000 7fb9fb5bf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1a42d0 0x55566c1b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.711+0000 7fb9fb5bf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55566c1a4c80 0x55566c1b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.711+0000 7fb9fb5bf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55566c1b3980 0x55566c1b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.711+0000 7fb9f1d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1a42d0 0x55566c1b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.711+0000 7fb9f1d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1a42d0 0x55566c1b32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52064/0 (socket says 172.21.15.107:52064) 2024-04-25T00:17:37.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.711+0000 7fb9f1d9b640 1 -- 172.21.15.107:0/1706512464 learned_addr learned my addr 172.21.15.107:0/1706512464 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:37.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.711+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/1706512464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55566c1b66e0 con 0x55566c1a42d0 2024-04-25T00:17:37.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.711+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/1706512464 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55566c0c5050 con 0x55566c1b3980 2024-04-25T00:17:37.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.711+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/1706512464 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55566c1b6860 con 0x55566c1a4c80 2024-04-25T00:17:37.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.711+0000 7fb9f259c640 1 --2- 172.21.15.107:0/1706512464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55566c1a4c80 0x55566c1b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.711+0000 7fb9f1d9b640 1 --2- 172.21.15.107:0/1706512464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55566c1b3980 0x55566c1b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.712+0000 7fb9fb5bf740 10 client.0 _ll_register_callbacks cb 0x55566be1cbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:37.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.712+0000 7fb9f1d9b640 1 -- 172.21.15.107:0/1706512464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55566c1a4c80 msgr2=0x55566c1b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:37.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.712+0000 7fb9f1d9b640 1 --2- 172.21.15.107:0/1706512464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55566c1a4c80 0x55566c1b05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.712+0000 7fb9f1d9b640 1 -- 172.21.15.107:0/1706512464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55566c1b3980 msgr2=0x55566c1b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:37.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.712+0000 7fb9f1d9b640 1 --2- 172.21.15.107:0/1706512464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55566c1b3980 0x55566c1b0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.712+0000 7fb9f1d9b640 1 -- 172.21.15.107:0/1706512464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55566c096dc0 con 0x55566c1a42d0 2024-04-25T00:17:37.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.712+0000 7fb9f1d9b640 1 --2- 172.21.15.107:0/1706512464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1a42d0 0x55566c1b32e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8004650 tx=0x7fb9e800d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99625aa9aad6e45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:37.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.712+0000 7fb9d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:17:37.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.712+0000 7fb9d9ffb640 1 -- 172.21.15.107:0/1706512464 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8005020 con 0x55566c1a42d0 2024-04-25T00:17:37.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.712+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/1706512464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9e4005500 con 0x55566c1a42d0 2024-04-25T00:17:37.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.712+0000 7fb9d9ffb640 1 -- 172.21.15.107:0/1706512464 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fb9e80051c0 con 0x55566c1a42d0 2024-04-25T00:17:37.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.713+0000 7fb9d9ffb640 1 -- 172.21.15.107:0/1706512464 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb9e80053c0 con 0x55566c1a42d0 2024-04-25T00:17:37.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.713+0000 7fb9fb5bf740 10 client.15891 fetch_fsmap learned FSMap version 24 2024-04-25T00:17:37.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.713+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/1706512464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55566be1d3f0 con 0x55566c1a42d0 2024-04-25T00:17:37.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.713+0000 7fb9d9ffb640 1 -- 172.21.15.107:0/1706512464 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9e4005500 con 0x55566c1a42d0 2024-04-25T00:17:37.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.713+0000 7fb9d9ffb640 1 -- 172.21.15.107:0/1706512464 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb9e801aab0 con 0x55566c1a42d0 2024-04-25T00:17:37.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.713+0000 7fb9fb5bf740 10 client.15891 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:17:37.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.713+0000 7fb9fb5bf740 10 client.15891 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:17:37.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.713+0000 7fb9fb5bf740 20 client.15891 populate_metadata read hostname 'smithi107' 2024-04-25T00:17:37.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.713+0000 7fb9fb5bf740 1 --2- 172.21.15.107:0/1706512464 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55566c1c38c0 0x55566c1c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.713+0000 7fb9fb5bf740 4 client.15891 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:17:37.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.713+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/1706512464 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55566c116bb0 con 0x55566c1c38c0 2024-04-25T00:17:37.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.714+0000 7fb9f259c640 1 --2- 172.21.15.107:0/1706512464 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55566c1c38c0 0x55566c1c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.714+0000 7fb9f259c640 1 --2- 172.21.15.107:0/1706512464 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55566c1c38c0 0x55566c1c5d60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=39c8f8b28b5bf9ab server_cookie=b68079e77f877bcf in_seq=0 out_seq=0 2024-04-25T00:17:37.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.714+0000 7fb9d9ffb640 10 client.15891 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:17:37.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.715+0000 7fb9d9ffb640 1 -- 172.21.15.107:0/1706512464 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55566c116bb0 con 0x55566c1c38c0 2024-04-25T00:17:37.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.715+0000 7fb9d9ffb640 10 client.15891 handle_command_reply: tid=0 2024-04-25T00:17:37.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.715+0000 7fb9fb5bf740 1 client.15891 shutdown 2024-04-25T00:17:37.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.715+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/1706512464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1a42d0 msgr2=0x55566c1b32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.715+0000 7fb9fb5bf740 1 --2- 172.21.15.107:0/1706512464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1a42d0 0x55566c1b32e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8004650 tx=0x7fb9e800d320 comp rx=0 tx=0).stop 2024-04-25T00:17:37.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.715+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/1706512464 shutdown_connections 2024-04-25T00:17:37.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.715+0000 7fb9fb5bf740 1 --2- 172.21.15.107:0/1706512464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55566c1b3980 0x55566c1b0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.715+0000 7fb9fb5bf740 1 --2- 172.21.15.107:0/1706512464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55566c1a4c80 0x55566c1b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.715+0000 7fb9fb5bf740 1 --2- 172.21.15.107:0/1706512464 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55566c1c38c0 0x55566c1c5d60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.715+0000 7fb9fb5bf740 1 --2- 172.21.15.107:0/1706512464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1a42d0 0x55566c1b32e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.715+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/1706512464 >> 172.21.15.107:0/1706512464 conn(0x55566c1a08a0 msgr2=0x55566c1a6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:37.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.715+0000 7fb9d9ffb640 0 client.15891 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:17:37.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.716+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/1706512464 shutdown_connections 2024-04-25T00:17:37.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.716+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/1706512464 wait complete. 2024-04-25T00:17:37.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.716+0000 7fb9fb5bf740 20 client.15891 trim_cache size 0 max 16384 2024-04-25T00:17:37.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.738+0000 7fb9fb5bf740 1 Processor -- start 2024-04-25T00:17:37.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.738+0000 7fb9fb5bf740 1 -- start start 2024-04-25T00:17:37.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.738+0000 7fb9fb5bf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55566c1a42d0 0x55566c1bf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.738+0000 7fb9fb5bf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55566c1a4c80 0x55566c1af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.738+0000 7fb9fb5bf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1b3980 0x55566c1abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.738+0000 7fb9fb5bf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f41af1e0 con 0x55566c1b3980 2024-04-25T00:17:37.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.738+0000 7fb9fb5bf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f41acab0 con 0x55566c1a42d0 2024-04-25T00:17:37.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.738+0000 7fb9fb5bf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f41aa900 con 0x55566c1a4c80 2024-04-25T00:17:37.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.738+0000 7fb9f1d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1b3980 0x55566c1abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.738+0000 7fb9f1d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1b3980 0x55566c1abaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52066/0 (socket says 172.21.15.107:52066) 2024-04-25T00:17:37.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.738+0000 7fb9f1d9b640 1 -- 172.21.15.107:0/3510076671 learned_addr learned my addr 172.21.15.107:0/3510076671 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:37.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.738+0000 7fb9f1d9b640 1 --2- 172.21.15.107:0/3510076671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55566c1a42d0 0x55566c1bf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.738+0000 7fb9f259c640 1 --2- 172.21.15.107:0/3510076671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55566c1a4c80 0x55566c1af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.739+0000 7fb9f1d9b640 1 -- 172.21.15.107:0/3510076671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55566c1a4c80 msgr2=0x55566c1af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.739+0000 7fb9f1d9b640 1 --2- 172.21.15.107:0/3510076671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55566c1a4c80 0x55566c1af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.739+0000 7fb9f1d9b640 1 -- 172.21.15.107:0/3510076671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55566c1a42d0 msgr2=0x55566c1bf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.739+0000 7fb9f1d9b640 1 --2- 172.21.15.107:0/3510076671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55566c1a42d0 0x55566c1bf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.739+0000 7fb9f1d9b640 1 -- 172.21.15.107:0/3510076671 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55566bee77d0 con 0x55566c1b3980 2024-04-25T00:17:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.739+0000 7fb9f1d9b640 1 --2- 172.21.15.107:0/3510076671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1b3980 0x55566c1abaf0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8004490 tx=0x7fb9e8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e3112b276e59a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.739+0000 7fb9dbfff640 1 -- 172.21.15.107:0/3510076671 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8005000 con 0x55566c1b3980 2024-04-25T00:17:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.739+0000 7fb9dbfff640 1 -- 172.21.15.107:0/3510076671 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9e8004ba0 con 0x55566c1b3980 2024-04-25T00:17:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.739+0000 7fb9dbfff640 1 -- 172.21.15.107:0/3510076671 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e801a6b0 con 0x55566c1b3980 2024-04-25T00:17:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.740+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/3510076671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1b3980 msgr2=0x55566c1abaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.740+0000 7fb9fb5bf740 1 --2- 172.21.15.107:0/3510076671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1b3980 0x55566c1abaf0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8004490 tx=0x7fb9e8000bc0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.740+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/3510076671 shutdown_connections 2024-04-25T00:17:37.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.740+0000 7fb9fb5bf740 1 --2- 172.21.15.107:0/3510076671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1b3980 0x55566c1abaf0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.740+0000 7fb9fb5bf740 1 --2- 172.21.15.107:0/3510076671 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55566c1a4c80 0x55566c1af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.740+0000 7fb9fb5bf740 1 --2- 172.21.15.107:0/3510076671 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55566c1a42d0 0x55566c1bf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.740+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/3510076671 >> 172.21.15.107:0/3510076671 conn(0x55566c1a08a0 msgr2=0x55566c1ab3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.740+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/3510076671 shutdown_connections 2024-04-25T00:17:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.740+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/3510076671 wait complete. 2024-04-25T00:17:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.740+0000 7fb9fb5bf740 1 -- start start 2024-04-25T00:17:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.741+0000 7fb9fb5bf740 1 Processor -- start 2024-04-25T00:17:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.741+0000 7fb9fb5bf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1a42d0 0x55566c222210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.741+0000 7fb9fb5bf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55566c1a4c80 0x55566c222750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.741+0000 7fb9fb5bf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55566c1b6c50 0x55566c1b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.741+0000 7fb9fb5bf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f41aaf70 con 0x55566c1a42d0 2024-04-25T00:17:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.741+0000 7fb9fb5bf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f41ac3e0 con 0x55566c1a4c80 2024-04-25T00:17:37.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.741+0000 7fb9fb5bf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f41ab640 con 0x55566c1b6c50 2024-04-25T00:17:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.741+0000 7fb9fb5bf740 10 client.0 _ll_register_callbacks cb 0x7fb9e8004550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.741+0000 7fb9f1d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1a42d0 0x55566c222210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.741+0000 7fb9f1d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1a42d0 0x55566c222210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52070/0 (socket says 172.21.15.107:52070) 2024-04-25T00:17:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.741+0000 7fb9f1d9b640 1 -- 172.21.15.107:0/4116662100 learned_addr learned my addr 172.21.15.107:0/4116662100 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.741+0000 7fb9f259c640 1 --2- 172.21.15.107:0/4116662100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55566c1a4c80 0x55566c222750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.741+0000 7fb9f1d9b640 1 -- 172.21.15.107:0/4116662100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55566c1b6c50 msgr2=0x55566c1b70d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:17:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.741+0000 7fb9f1d9b640 1 --2- 172.21.15.107:0/4116662100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55566c1b6c50 0x55566c1b70d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.741+0000 7fb9f1d9b640 1 -- 172.21.15.107:0/4116662100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55566c1a4c80 msgr2=0x55566c222750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.741+0000 7fb9f1d9b640 1 --2- 172.21.15.107:0/4116662100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55566c1a4c80 0x55566c222750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.741+0000 7fb9f1d9b640 1 -- 172.21.15.107:0/4116662100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55566c1b7710 con 0x55566c1a42d0 2024-04-25T00:17:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.741+0000 7fb9f259c640 1 --2- 172.21.15.107:0/4116662100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55566c1a4c80 0x55566c222750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.742+0000 7fb9f1d9b640 1 --2- 172.21.15.107:0/4116662100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1a42d0 0x55566c222210 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8004320 tx=0x7fb9e801ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb478ec5c0c16487 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.742+0000 7fb9d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:17:37.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.742+0000 7fb9d9ffb640 1 -- 172.21.15.107:0/4116662100 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e803f060 con 0x55566c1a42d0 2024-04-25T00:17:37.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.742+0000 7fb9d9ffb640 1 -- 172.21.15.107:0/4116662100 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fb9e8004630 con 0x55566c1a42d0 2024-04-25T00:17:37.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.742+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/4116662100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55566c110f00 con 0x55566c1a42d0 2024-04-25T00:17:37.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.742+0000 7fb9d9ffb640 1 -- 172.21.15.107:0/4116662100 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb9e8005b90 con 0x55566c1a42d0 2024-04-25T00:17:37.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.742+0000 7fb9fb5bf740 10 client.15897 fetch_fsmap learned FSMap version 24 2024-04-25T00:17:37.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.742+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/4116662100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55566c229d50 con 0x55566c1a42d0 2024-04-25T00:17:37.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.742+0000 7fb9d9ffb640 1 -- 172.21.15.107:0/4116662100 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55566c110f00 con 0x55566c1a42d0 2024-04-25T00:17:37.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.742+0000 7fb9d9ffb640 1 -- 172.21.15.107:0/4116662100 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb9e8004830 con 0x55566c1a42d0 2024-04-25T00:17:37.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.743+0000 7fb9fb5bf740 10 client.15897 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:17:37.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.743+0000 7fb9fb5bf740 10 client.15897 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:17:37.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.743+0000 7fb9fb5bf740 20 client.15897 populate_metadata read hostname 'smithi107' 2024-04-25T00:17:37.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.743+0000 7fb9fb5bf740 1 --2- 172.21.15.107:0/4116662100 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55566c22a2d0 0x55566c22c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.743+0000 7fb9fb5bf740 4 client.15897 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:17:37.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.743+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/4116662100 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55566c08e820 con 0x55566c22a2d0 2024-04-25T00:17:37.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.743+0000 7fb9f259c640 1 --2- 172.21.15.107:0/4116662100 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55566c22a2d0 0x55566c22c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.744+0000 7fb9f259c640 1 --2- 172.21.15.107:0/4116662100 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55566c22a2d0 0x55566c22c6c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4be55b6f4e178f88 server_cookie=8610502d19c08911 in_seq=0 out_seq=0 2024-04-25T00:17:37.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.744+0000 7fb9d9ffb640 10 client.15897 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:17:37.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.744+0000 7fb9d9ffb640 1 -- 172.21.15.107:0/4116662100 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55566c08e820 con 0x55566c22a2d0 2024-04-25T00:17:37.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.744+0000 7fb9d9ffb640 10 client.15897 handle_command_reply: tid=0 2024-04-25T00:17:37.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.744+0000 7fb9fb5bf740 1 client.15897 shutdown 2024-04-25T00:17:37.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.744+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/4116662100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1a42d0 msgr2=0x55566c222210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.744+0000 7fb9fb5bf740 1 --2- 172.21.15.107:0/4116662100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1a42d0 0x55566c222210 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8004320 tx=0x7fb9e801ea30 comp rx=0 tx=0).stop 2024-04-25T00:17:37.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.744+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/4116662100 shutdown_connections 2024-04-25T00:17:37.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.744+0000 7fb9fb5bf740 1 --2- 172.21.15.107:0/4116662100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55566c1b6c50 0x55566c1b70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.744+0000 7fb9fb5bf740 1 --2- 172.21.15.107:0/4116662100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55566c1a4c80 0x55566c222750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.744+0000 7fb9fb5bf740 1 --2- 172.21.15.107:0/4116662100 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55566c22a2d0 0x55566c22c6c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.744+0000 7fb9d9ffb640 0 client.15897 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:17:37.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.744+0000 7fb9fb5bf740 1 --2- 172.21.15.107:0/4116662100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55566c1a42d0 0x55566c222210 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.744+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/4116662100 >> 172.21.15.107:0/4116662100 conn(0x55566c1a08a0 msgr2=0x55566c1abc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:37.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.745+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/4116662100 shutdown_connections 2024-04-25T00:17:37.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.745+0000 7fb9fb5bf740 1 -- 172.21.15.107:0/4116662100 wait complete. 2024-04-25T00:17:37.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.745+0000 7fb9fb5bf740 20 client.15897 trim_cache size 0 max 16384 2024-04-25T00:17:37.752 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:17:37.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.745+0000 7fb9f88b3640 1 -- 172.21.15.107:0/2812629268 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb9dc0a2eb0 msgr2=0x7fb9dc0a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.745+0000 7fb9f88b3640 1 --2- 172.21.15.107:0/2812629268 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb9dc0a2eb0 0x7fb9dc0a5370 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec0cd220 tx=0x7fb9ec0185b0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.746+0000 7fb9f88b3640 1 -- 172.21.15.107:0/2812629268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f4106630 msgr2=0x7fb9f419b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.746+0000 7fb9f88b3640 1 --2- 172.21.15.107:0/2812629268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f4106630 0x7fb9f419b7e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec00cce0 tx=0x7fb9ec00a620 comp rx=0 tx=0).stop 2024-04-25T00:17:37.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.746+0000 7fb9f88b3640 1 -- 172.21.15.107:0/2812629268 shutdown_connections 2024-04-25T00:17:37.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.746+0000 7fb9f88b3640 1 --2- 172.21.15.107:0/2812629268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f4106630 0x7fb9f419b7e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.746+0000 7fb9f88b3640 1 --2- 172.21.15.107:0/2812629268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9f4100fe0 0x7fb9f4197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.746+0000 7fb9f88b3640 1 --2- 172.21.15.107:0/2812629268 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb9dc0a2eb0 0x7fb9dc0a5370 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.746+0000 7fb9f88b3640 1 --2- 172.21.15.107:0/2812629268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9f4100620 0x7fb9f4196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.746+0000 7fb9f88b3640 1 -- 172.21.15.107:0/2812629268 >> 172.21.15.107:0/2812629268 conn(0x7fb9f40769c0 msgr2=0x7fb9f40fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:37.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.746+0000 7fb9f88b3640 1 -- 172.21.15.107:0/2812629268 shutdown_connections 2024-04-25T00:17:37.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.746+0000 7fb9f88b3640 1 -- 172.21.15.107:0/2812629268 wait complete. 2024-04-25T00:17:37.758 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:17:37.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.875+0000 7f74acb22640 1 -- 172.21.15.107:0/4106973143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74a8100610 msgr2=0x7f74a8100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.875+0000 7f74acb22640 1 --2- 172.21.15.107:0/4106973143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74a8100610 0x7f74a8100a90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f749c009930 tx=0x7f749c02a420 comp rx=0 tx=0).stop 2024-04-25T00:17:37.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.875+0000 7f74acb22640 1 -- 172.21.15.107:0/4106973143 shutdown_connections 2024-04-25T00:17:37.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.875+0000 7f74acb22640 1 --2- 172.21.15.107:0/4106973143 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74a8100fd0 0x7f74a810d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.875+0000 7f74acb22640 1 --2- 172.21.15.107:0/4106973143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74a8100610 0x7f74a8100a90 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.875+0000 7f74acb22640 1 --2- 172.21.15.107:0/4106973143 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74a81066b0 0x7f74a8106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.875+0000 7f74acb22640 1 -- 172.21.15.107:0/4106973143 >> 172.21.15.107:0/4106973143 conn(0x7f74a8077740 msgr2=0x7f74a8077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:37.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.875+0000 7f74acb22640 1 -- 172.21.15.107:0/4106973143 shutdown_connections 2024-04-25T00:17:37.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.875+0000 7f74acb22640 1 -- 172.21.15.107:0/4106973143 wait complete. 2024-04-25T00:17:37.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.876+0000 7f74acb22640 1 Processor -- start 2024-04-25T00:17:37.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.876+0000 7f74acb22640 1 -- start start 2024-04-25T00:17:37.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.877+0000 7f74acb22640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74a8100610 0x7f74a8195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.877+0000 7f74acb22640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74a8100fd0 0x7f74a81962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.877+0000 7f74acb22640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74a81066b0 0x7f74a819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.877+0000 7f74acb22640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f74a810f920 con 0x7f74a8100610 2024-04-25T00:17:37.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.877+0000 7f74a659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74a8100610 0x7f74a8195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.877+0000 7f74acb22640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f74a810f7a0 con 0x7f74a81066b0 2024-04-25T00:17:37.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.877+0000 7f74a659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74a8100610 0x7f74a8195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52086/0 (socket says 172.21.15.107:52086) 2024-04-25T00:17:37.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.877+0000 7f74acb22640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f74a810faa0 con 0x7f74a8100fd0 2024-04-25T00:17:37.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.877+0000 7f74a659c640 1 -- 172.21.15.107:0/1398451507 learned_addr learned my addr 172.21.15.107:0/1398451507 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:37.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.877+0000 7f74a5d9b640 1 --2- 172.21.15.107:0/1398451507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74a8100fd0 0x7f74a81962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.877+0000 7f74a6d9d640 1 --2- 172.21.15.107:0/1398451507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74a81066b0 0x7f74a819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.877+0000 7f74a659c640 1 -- 172.21.15.107:0/1398451507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74a8100fd0 msgr2=0x7f74a81962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.877+0000 7f74a659c640 1 --2- 172.21.15.107:0/1398451507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74a8100fd0 0x7f74a81962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.877+0000 7f74a659c640 1 -- 172.21.15.107:0/1398451507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74a81066b0 msgr2=0x7f74a819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.877+0000 7f74a659c640 1 --2- 172.21.15.107:0/1398451507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74a81066b0 0x7f74a819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.877+0000 7f74a659c640 1 -- 172.21.15.107:0/1398451507 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a819ad70 con 0x7f74a8100610 2024-04-25T00:17:37.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.878+0000 7f74a659c640 1 --2- 172.21.15.107:0/1398451507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74a8100610 0x7f74a8195d80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f749800d770 tx=0x7f749800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:37.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.878+0000 7f74977fe640 1 -- 172.21.15.107:0/1398451507 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498017070 con 0x7f74a8100610 2024-04-25T00:17:37.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.878+0000 7f74977fe640 1 -- 172.21.15.107:0/1398451507 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7498009c30 con 0x7f74a8100610 2024-04-25T00:17:37.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.878+0000 7f74acb22640 1 -- 172.21.15.107:0/1398451507 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a819b060 con 0x7f74a8100610 2024-04-25T00:17:37.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.878+0000 7f74acb22640 1 -- 172.21.15.107:0/1398451507 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a81a28f0 con 0x7f74a8100610 2024-04-25T00:17:37.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.878+0000 7f74977fe640 1 -- 172.21.15.107:0/1398451507 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498003dd0 con 0x7f74a8100610 2024-04-25T00:17:37.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.881+0000 7f74977fe640 1 -- 172.21.15.107:0/1398451507 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f7498003f70 con 0x7f74a8100610 2024-04-25T00:17:37.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.882+0000 7f74977fe640 1 --2- 172.21.15.107:0/1398451507 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f74900a5140 0x7f74900a7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.882+0000 7f74af833740 1 Processor -- start 2024-04-25T00:17:37.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.882+0000 7f74af833740 1 -- start start 2024-04-25T00:17:37.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.882+0000 7f74a5d9b640 1 --2- 172.21.15.107:0/1398451507 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f74900a5140 0x7f74900a7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.883+0000 7f74977fe640 1 -- 172.21.15.107:0/1398451507 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f74980cf300 con 0x7f74a8100610 2024-04-25T00:17:37.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.883+0000 7f74af833740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc140580 0x55befc140960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.883+0000 7f74af833740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55befc140ea0 0x55befc14b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.883+0000 7f74af833740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55befc14b570 0x55befc14d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.883+0000 7f74af833740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55befc0a5670 con 0x55befc140580 2024-04-25T00:17:37.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.883+0000 7f74af833740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55befc04d970 con 0x55befc140ea0 2024-04-25T00:17:37.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.883+0000 7f74af833740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55befc0a1630 con 0x55befc14b570 2024-04-25T00:17:37.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.883+0000 7f74a659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc140580 0x55befc140960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.883+0000 7f74a659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc140580 0x55befc140960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52094/0 (socket says 172.21.15.107:52094) 2024-04-25T00:17:37.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.883+0000 7f74a659c640 1 -- 172.21.15.107:0/3304068623 learned_addr learned my addr 172.21.15.107:0/3304068623 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:37.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.883+0000 7f74a6d9d640 1 --2- 172.21.15.107:0/3304068623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55befc14b570 0x55befc14d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.883+0000 7f74a659c640 1 -- 172.21.15.107:0/3304068623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55befc14b570 msgr2=0x55befc14d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:37.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.883+0000 7f74a659c640 1 --2- 172.21.15.107:0/3304068623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55befc14b570 0x55befc14d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.883+0000 7f74a659c640 1 -- 172.21.15.107:0/3304068623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55befc140ea0 msgr2=0x55befc14b030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:17:37.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.883+0000 7f74a659c640 1 --2- 172.21.15.107:0/3304068623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55befc140ea0 0x55befc14b030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.883+0000 7f74a659c640 1 -- 172.21.15.107:0/3304068623 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55befc071400 con 0x55befc140580 2024-04-25T00:17:37.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.883+0000 7f74a5d9b640 1 --2- 172.21.15.107:0/1398451507 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f74900a5140 0x7f74900a7600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f749c009900 tx=0x7f749c0058d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:37.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.883+0000 7f74a6d9d640 1 --2- 172.21.15.107:0/3304068623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55befc14b570 0x55befc14d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:17:37.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.884+0000 7f74a659c640 1 --2- 172.21.15.107:0/3304068623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc140580 0x55befc140960 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7498097cf0 tx=0x7f749809ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3f7b597c7c37d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:37.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.884+0000 7f7494ff9640 1 -- 172.21.15.107:0/3304068623 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498020040 con 0x55befc140580 2024-04-25T00:17:37.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.884+0000 7f7494ff9640 1 -- 172.21.15.107:0/3304068623 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f749809f620 con 0x55befc140580 2024-04-25T00:17:37.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.884+0000 7f7494ff9640 1 -- 172.21.15.107:0/3304068623 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749809f160 con 0x55befc140580 2024-04-25T00:17:37.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.885+0000 7f74af833740 1 -- 172.21.15.107:0/3304068623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc140580 msgr2=0x55befc140960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.885+0000 7f74af833740 1 --2- 172.21.15.107:0/3304068623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc140580 0x55befc140960 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7498097cf0 tx=0x7f749809ac80 comp rx=0 tx=0).stop 2024-04-25T00:17:37.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.885+0000 7f74af833740 1 -- 172.21.15.107:0/3304068623 shutdown_connections 2024-04-25T00:17:37.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.885+0000 7f74af833740 1 --2- 172.21.15.107:0/3304068623 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55befc14b570 0x55befc14d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.885+0000 7f74af833740 1 --2- 172.21.15.107:0/3304068623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55befc140ea0 0x55befc14b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.885+0000 7f74af833740 1 --2- 172.21.15.107:0/3304068623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc140580 0x55befc140960 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.885+0000 7f74af833740 1 -- 172.21.15.107:0/3304068623 >> 172.21.15.107:0/3304068623 conn(0x55befc13c8a0 msgr2=0x55befc13cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:37.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.885+0000 7f74af833740 1 -- 172.21.15.107:0/3304068623 shutdown_connections 2024-04-25T00:17:37.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.885+0000 7f74af833740 1 -- 172.21.15.107:0/3304068623 wait complete. 2024-04-25T00:17:37.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.885+0000 7f74af833740 1 -- start start 2024-04-25T00:17:37.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.886+0000 7f74af833740 1 Processor -- start 2024-04-25T00:17:37.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.886+0000 7f74af833740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55befc140580 0x55befc14afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.886+0000 7f74af833740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc140ea0 0x55befc1482d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.886+0000 7f74a5d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc140ea0 0x55befc1482d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.886+0000 7f74a659c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55befc140580 0x55befc14afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.886+0000 7f74a659c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55befc140580 0x55befc14afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:51668/0 (socket says 172.21.15.107:51668) 2024-04-25T00:17:37.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.886+0000 7f74a5d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc140ea0 0x55befc1482d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52110/0 (socket says 172.21.15.107:52110) 2024-04-25T00:17:37.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.886+0000 7f74a659c640 1 -- 172.21.15.107:0/364210675 learned_addr learned my addr 172.21.15.107:0/364210675 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:37.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.886+0000 7f74af833740 1 --2- 172.21.15.107:0/364210675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55befc14b570 0x55befc148a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.886+0000 7f74af833740 1 -- 172.21.15.107:0/364210675 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55befc14e360 con 0x55befc140ea0 2024-04-25T00:17:37.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.886+0000 7f74af833740 1 -- 172.21.15.107:0/364210675 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55befc061050 con 0x55befc14b570 2024-04-25T00:17:37.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.886+0000 7f74af833740 1 -- 172.21.15.107:0/364210675 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55befc14e4e0 con 0x55befc140580 2024-04-25T00:17:37.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.886+0000 7f74a6d9d640 1 --2- 172.21.15.107:0/364210675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55befc14b570 0x55befc148a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.887+0000 7f74a659c640 1 -- 172.21.15.107:0/364210675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55befc14b570 msgr2=0x55befc148a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:37.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.887+0000 7f74a659c640 1 --2- 172.21.15.107:0/364210675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55befc14b570 0x55befc148a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.887+0000 7f74a659c640 1 -- 172.21.15.107:0/364210675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc140ea0 msgr2=0x55befc1482d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:37.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.887+0000 7f74a659c640 1 --2- 172.21.15.107:0/364210675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc140ea0 0x55befc1482d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.887+0000 7f74a659c640 1 -- 172.21.15.107:0/364210675 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55befc032dc0 con 0x55befc140580 2024-04-25T00:17:37.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.887+0000 7f74a6d9d640 1 --2- 172.21.15.107:0/364210675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55befc14b570 0x55befc148a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:17:37.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.887+0000 7f74a5d9b640 1 --2- 172.21.15.107:0/364210675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc140ea0 0x55befc1482d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:37.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.887+0000 7f74a659c640 1 --2- 172.21.15.107:0/364210675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55befc140580 0x55befc14afe0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f74980cc590 tx=0x7f749809b3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d463c43ff246e4d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:37.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.887+0000 7f748dffb640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-04-25T00:17:37.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.887+0000 7f748dffb640 1 -- 172.21.15.107:0/364210675 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74980b9060 con 0x55befc140580 2024-04-25T00:17:37.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.887+0000 7f74af833740 10 client.0 _ll_register_callbacks cb 0x55befbdb8be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:37.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.887+0000 7f748dffb640 1 -- 172.21.15.107:0/364210675 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f7498099620 con 0x55befc140580 2024-04-25T00:17:37.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.887+0000 7f74af833740 1 -- 172.21.15.107:0/364210675 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f749809f620 con 0x55befc140580 2024-04-25T00:17:37.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.888+0000 7f748dffb640 1 -- 172.21.15.107:0/364210675 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7498081050 con 0x55befc140580 2024-04-25T00:17:37.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.888+0000 7f74af833740 10 client.25354 fetch_fsmap learned FSMap version 24 2024-04-25T00:17:37.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.888+0000 7f74af833740 1 -- 172.21.15.107:0/364210675 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({fsmap=24}) -- 0x55befbdb93f0 con 0x55befc140580 2024-04-25T00:17:37.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.888+0000 7f748dffb640 1 -- 172.21.15.107:0/364210675 <== mon.1 v2:172.21.15.195:3300/0 4 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f74980bb0c0 con 0x55befc140580 2024-04-25T00:17:37.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.888+0000 7f74af833740 10 client.25354 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:17:37.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.888+0000 7f74af833740 10 client.25354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:37.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.889+0000 7f74af833740 20 client.25354 populate_metadata read hostname 'smithi107' 2024-04-25T00:17:37.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.889+0000 7f74af833740 1 --2- 172.21.15.107:0/364210675 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55befc15af10 0x55befc15d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.889+0000 7f74af833740 4 client.25354 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:17:37.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.889+0000 7f74af833740 1 -- 172.21.15.107:0/364210675 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55befc0b2bb0 con 0x55befc15af10 2024-04-25T00:17:37.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.889+0000 7f74a5d9b640 1 --2- 172.21.15.107:0/364210675 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55befc15af10 0x55befc15d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.889+0000 7f748dffb640 10 client.25354 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:37.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.889+0000 7f74a5d9b640 1 --2- 172.21.15.107:0/364210675 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55befc15af10 0x55befc15d300 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b48caf922e857e3d server_cookie=8c92d94247a1eed0 in_seq=0 out_seq=0 2024-04-25T00:17:37.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.890+0000 7f748dffb640 1 -- 172.21.15.107:0/364210675 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55befc0b2bb0 con 0x55befc15af10 2024-04-25T00:17:37.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.890+0000 7f748dffb640 10 client.25354 handle_command_reply: tid=0 2024-04-25T00:17:37.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.890+0000 7f74af833740 1 client.25354 shutdown 2024-04-25T00:17:37.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.890+0000 7f74af833740 1 -- 172.21.15.107:0/364210675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55befc140580 msgr2=0x55befc14afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.890+0000 7f74af833740 1 --2- 172.21.15.107:0/364210675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55befc140580 0x55befc14afe0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f74980cc590 tx=0x7f749809b3e0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.890+0000 7f74af833740 1 -- 172.21.15.107:0/364210675 shutdown_connections 2024-04-25T00:17:37.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.890+0000 7f74af833740 1 --2- 172.21.15.107:0/364210675 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55befc15af10 0x55befc15d300 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.890+0000 7f74af833740 1 --2- 172.21.15.107:0/364210675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55befc14b570 0x55befc148a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.890+0000 7f74af833740 1 --2- 172.21.15.107:0/364210675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc140ea0 0x55befc1482d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.890+0000 7f74af833740 1 --2- 172.21.15.107:0/364210675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55befc140580 0x55befc14afe0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.890+0000 7f74af833740 1 -- 172.21.15.107:0/364210675 >> 172.21.15.107:0/364210675 conn(0x55befc13c8a0 msgr2=0x55befc13bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:37.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.890+0000 7f748dffb640 0 client.25354 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:37.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.891+0000 7f74af833740 1 -- 172.21.15.107:0/364210675 shutdown_connections 2024-04-25T00:17:37.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.891+0000 7f74af833740 1 -- 172.21.15.107:0/364210675 wait complete. 2024-04-25T00:17:37.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.891+0000 7f74af833740 20 client.25354 trim_cache size 0 max 16384 2024-04-25T00:17:37.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.915+0000 7f74af833740 1 Processor -- start 2024-04-25T00:17:37.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.915+0000 7f74af833740 1 -- start start 2024-04-25T00:17:37.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.916+0000 7f74af833740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55befc140580 0x55befc1571b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.916+0000 7f74af833740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc140ea0 0x55befc147bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.916+0000 7f74af833740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55befc14b570 0x55befc13c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.916+0000 7f74af833740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f74a81af110 con 0x55befc140ea0 2024-04-25T00:17:37.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.916+0000 7f74af833740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f74a81ac9e0 con 0x55befc140580 2024-04-25T00:17:37.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.916+0000 7f74af833740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f74a81aa830 con 0x55befc14b570 2024-04-25T00:17:37.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.916+0000 7f74a5d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc140ea0 0x55befc147bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.916+0000 7f74a5d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc140ea0 0x55befc147bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52126/0 (socket says 172.21.15.107:52126) 2024-04-25T00:17:37.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.916+0000 7f74a5d9b640 1 -- 172.21.15.107:0/3451814422 learned_addr learned my addr 172.21.15.107:0/3451814422 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:37.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.916+0000 7f74a659c640 1 --2- 172.21.15.107:0/3451814422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55befc140580 0x55befc1571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.916+0000 7f74a6d9d640 1 --2- 172.21.15.107:0/3451814422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55befc14b570 0x55befc13c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.916+0000 7f74a5d9b640 1 -- 172.21.15.107:0/3451814422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55befc14b570 msgr2=0x55befc13c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:37.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.916+0000 7f74a5d9b640 1 --2- 172.21.15.107:0/3451814422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55befc14b570 0x55befc13c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.916+0000 7f74a5d9b640 1 -- 172.21.15.107:0/3451814422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55befc140580 msgr2=0x55befc1571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:37.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.916+0000 7f74a5d9b640 1 --2- 172.21.15.107:0/3451814422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55befc140580 0x55befc1571b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.916+0000 7f74a5d9b640 1 -- 172.21.15.107:0/3451814422 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55befbe837d0 con 0x55befc140ea0 2024-04-25T00:17:37.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.917+0000 7f74a5d9b640 1 --2- 172.21.15.107:0/3451814422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc140ea0 0x55befc147bf0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f749c03f040 tx=0x7f749c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f49991d14cdce8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:37.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.917+0000 7f748ffff640 1 -- 172.21.15.107:0/3451814422 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c035040 con 0x55befc140ea0 2024-04-25T00:17:37.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.917+0000 7f748ffff640 1 -- 172.21.15.107:0/3451814422 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f749c056070 con 0x55befc140ea0 2024-04-25T00:17:37.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.917+0000 7f748ffff640 1 -- 172.21.15.107:0/3451814422 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c003770 con 0x55befc140ea0 2024-04-25T00:17:37.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.918+0000 7f74af833740 1 -- 172.21.15.107:0/3451814422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc140ea0 msgr2=0x55befc147bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.918+0000 7f74af833740 1 --2- 172.21.15.107:0/3451814422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc140ea0 0x55befc147bf0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f749c03f040 tx=0x7f749c040000 comp rx=0 tx=0).stop 2024-04-25T00:17:37.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.918+0000 7f74af833740 1 -- 172.21.15.107:0/3451814422 shutdown_connections 2024-04-25T00:17:37.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.918+0000 7f74af833740 1 --2- 172.21.15.107:0/3451814422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55befc14b570 0x55befc13c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.918+0000 7f74af833740 1 --2- 172.21.15.107:0/3451814422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc140ea0 0x55befc147bf0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.918+0000 7f74af833740 1 --2- 172.21.15.107:0/3451814422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55befc140580 0x55befc1571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.918+0000 7f74af833740 1 -- 172.21.15.107:0/3451814422 >> 172.21.15.107:0/3451814422 conn(0x55befc13c8a0 msgr2=0x55befc156aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:37.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.918+0000 7f74af833740 1 -- 172.21.15.107:0/3451814422 shutdown_connections 2024-04-25T00:17:37.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.918+0000 7f74af833740 1 -- 172.21.15.107:0/3451814422 wait complete. 2024-04-25T00:17:37.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.918+0000 7f74af833740 1 -- start start 2024-04-25T00:17:37.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.918+0000 7f74af833740 1 Processor -- start 2024-04-25T00:17:37.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.919+0000 7f74af833740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55befc140580 0x55befc1538d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.919+0000 7f74af833740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55befc14b570 0x55befc150d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.919+0000 7f74af833740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc151240 0x55befc1516c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.919+0000 7f74af833740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f74a81aaea0 con 0x55befc151240 2024-04-25T00:17:37.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.919+0000 7f74af833740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f74a81ac310 con 0x55befc14b570 2024-04-25T00:17:37.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.919+0000 7f74af833740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f74a81ab570 con 0x55befc140580 2024-04-25T00:17:37.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.919+0000 7f74af833740 10 client.0 _ll_register_callbacks cb 0x7f7498097690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:37.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.919+0000 7f74a6d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc151240 0x55befc1516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.919+0000 7f74a659c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55befc140580 0x55befc1538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.919+0000 7f74a5d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55befc14b570 0x55befc150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.919+0000 7f74a5d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55befc14b570 0x55befc150d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.107:45820/0 (socket says 172.21.15.107:45820) 2024-04-25T00:17:37.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.919+0000 7f74a5d9b640 1 -- 172.21.15.107:0/2016600998 learned_addr learned my addr 172.21.15.107:0/2016600998 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:37.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.920+0000 7f74a659c640 1 -- 172.21.15.107:0/2016600998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55befc14b570 msgr2=0x55befc150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:37.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.920+0000 7f74a659c640 1 --2- 172.21.15.107:0/2016600998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55befc14b570 0x55befc150d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.920+0000 7f74a659c640 1 -- 172.21.15.107:0/2016600998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc151240 msgr2=0x55befc1516c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:37.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.920+0000 7f74a659c640 1 --2- 172.21.15.107:0/2016600998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc151240 0x55befc1516c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.920+0000 7f74a659c640 1 -- 172.21.15.107:0/2016600998 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55befc1c03a0 con 0x55befc140580 2024-04-25T00:17:37.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.920+0000 7f74a659c640 1 --2- 172.21.15.107:0/2016600998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55befc140580 0x55befc1538d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f74980995d0 tx=0x7f749809d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37334a88b3f86286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:37.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.920+0000 7f748dffb640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-04-25T00:17:37.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.920+0000 7f748dffb640 1 -- 172.21.15.107:0/2016600998 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74980b9050 con 0x55befc140580 2024-04-25T00:17:37.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.920+0000 7f748dffb640 1 -- 172.21.15.107:0/2016600998 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f7498099620 con 0x55befc140580 2024-04-25T00:17:37.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.921+0000 7f74af833740 1 -- 172.21.15.107:0/2016600998 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f749c056070 con 0x55befc140580 2024-04-25T00:17:37.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.927+0000 7f748dffb640 1 -- 172.21.15.107:0/2016600998 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f74980b3030 con 0x55befc140580 2024-04-25T00:17:37.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.927+0000 7f74af833740 10 client.25360 fetch_fsmap learned FSMap version 24 2024-04-25T00:17:37.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.927+0000 7f74af833740 1 -- 172.21.15.107:0/2016600998 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55befc1c03a0 con 0x55befc140580 2024-04-25T00:17:37.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.927+0000 7f748dffb640 1 -- 172.21.15.107:0/2016600998 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f749c056070 con 0x55befc140580 2024-04-25T00:17:37.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.927+0000 7f748dffb640 1 -- 172.21.15.107:0/2016600998 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f749809a770 con 0x55befc140580 2024-04-25T00:17:37.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.927+0000 7f74af833740 10 client.25360 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:17:37.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.927+0000 7f74af833740 10 client.25360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:37.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.927+0000 7f74af833740 20 client.25360 populate_metadata read hostname 'smithi107' 2024-04-25T00:17:37.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.927+0000 7f74af833740 1 --2- 172.21.15.107:0/2016600998 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55befc1c1800 0x55befc1c3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:37.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.927+0000 7f74af833740 4 client.25360 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:17:37.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.927+0000 7f74af833740 1 -- 172.21.15.107:0/2016600998 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55befc0acf00 con 0x55befc1c1800 2024-04-25T00:17:37.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.928+0000 7f74a5d9b640 1 --2- 172.21.15.107:0/2016600998 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55befc1c1800 0x55befc1c3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:37.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.928+0000 7f748dffb640 10 client.25360 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:37.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.928+0000 7f74a5d9b640 1 --2- 172.21.15.107:0/2016600998 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55befc1c1800 0x55befc1c3bf0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7224b4b1c7150ec8 server_cookie=9d5b389faa5aab34 in_seq=0 out_seq=0 2024-04-25T00:17:37.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.929+0000 7f748dffb640 1 -- 172.21.15.107:0/2016600998 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55befc0acf00 con 0x55befc1c1800 2024-04-25T00:17:37.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.929+0000 7f748dffb640 10 client.25360 handle_command_reply: tid=0 2024-04-25T00:17:37.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.929+0000 7f74af833740 1 client.25360 shutdown 2024-04-25T00:17:37.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.929+0000 7f74af833740 1 -- 172.21.15.107:0/2016600998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55befc140580 msgr2=0x55befc1538d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.929+0000 7f74af833740 1 --2- 172.21.15.107:0/2016600998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55befc140580 0x55befc1538d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f74980995d0 tx=0x7f749809d000 comp rx=0 tx=0).stop 2024-04-25T00:17:37.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.929+0000 7f74af833740 1 -- 172.21.15.107:0/2016600998 shutdown_connections 2024-04-25T00:17:37.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.929+0000 7f74af833740 1 --2- 172.21.15.107:0/2016600998 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55befc1c1800 0x55befc1c3bf0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.929+0000 7f74af833740 1 --2- 172.21.15.107:0/2016600998 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55befc151240 0x55befc1516c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.929+0000 7f74af833740 1 --2- 172.21.15.107:0/2016600998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55befc14b570 0x55befc150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.929+0000 7f748dffb640 0 client.25360 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:37.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.929+0000 7f74af833740 1 --2- 172.21.15.107:0/2016600998 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55befc140580 0x55befc1538d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.929+0000 7f74af833740 1 -- 172.21.15.107:0/2016600998 >> 172.21.15.107:0/2016600998 conn(0x55befc13c8a0 msgr2=0x55befc146be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:37.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.929+0000 7f74af833740 1 -- 172.21.15.107:0/2016600998 shutdown_connections 2024-04-25T00:17:37.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.929+0000 7f74af833740 1 -- 172.21.15.107:0/2016600998 wait complete. 2024-04-25T00:17:37.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.929+0000 7f74af833740 20 client.25360 trim_cache size 0 max 16384 2024-04-25T00:17:37.932 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:17:37.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.930+0000 7f74acb22640 1 -- 172.21.15.107:0/1398451507 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f74900a5140 msgr2=0x7f74900a7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.930+0000 7f74acb22640 1 --2- 172.21.15.107:0/1398451507 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f74900a5140 0x7f74900a7600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f749c009900 tx=0x7f749c0058d0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.930+0000 7f74acb22640 1 -- 172.21.15.107:0/1398451507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74a8100610 msgr2=0x7f74a8195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:37.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.930+0000 7f74acb22640 1 --2- 172.21.15.107:0/1398451507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74a8100610 0x7f74a8195d80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f749800d770 tx=0x7f749800dc30 comp rx=0 tx=0).stop 2024-04-25T00:17:37.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.930+0000 7f74acb22640 1 -- 172.21.15.107:0/1398451507 shutdown_connections 2024-04-25T00:17:37.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.930+0000 7f74acb22640 1 --2- 172.21.15.107:0/1398451507 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f74900a5140 0x7f74900a7600 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.930+0000 7f74acb22640 1 --2- 172.21.15.107:0/1398451507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f74a81066b0 0x7f74a819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.930+0000 7f74acb22640 1 --2- 172.21.15.107:0/1398451507 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f74a8100fd0 0x7f74a81962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.930+0000 7f74acb22640 1 --2- 172.21.15.107:0/1398451507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74a8100610 0x7f74a8195d80 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:37.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.930+0000 7f74acb22640 1 -- 172.21.15.107:0/1398451507 >> 172.21.15.107:0/1398451507 conn(0x7f74a8077740 msgr2=0x7f74a80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:37.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.930+0000 7f74acb22640 1 -- 172.21.15.107:0/1398451507 shutdown_connections 2024-04-25T00:17:37.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:37.930+0000 7f74acb22640 1 -- 172.21.15.107:0/1398451507 wait complete. 2024-04-25T00:17:38.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:37 smithi170 ceph-mon[32807]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:17:38.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:37 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2711038863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:17:38.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:37 smithi107 ceph-mon[29153]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:17:38.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:37 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2711038863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:17:38.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:37 smithi195 ceph-mon[32833]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:17:38.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:37 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2711038863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:17:38.946 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:17:38.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:17:38.946 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:17:39.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.060+0000 7f45e6438640 1 -- 172.21.15.107:0/1230317177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45e0101740 msgr2=0x7f45e0101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.060+0000 7f45e6438640 1 --2- 172.21.15.107:0/1230317177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45e0101740 0x7f45e0101bc0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f45d4009930 tx=0x7f45d402a420 comp rx=0 tx=0).stop 2024-04-25T00:17:39.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.060+0000 7f45e6438640 1 -- 172.21.15.107:0/1230317177 shutdown_connections 2024-04-25T00:17:39.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.060+0000 7f45e6438640 1 --2- 172.21.15.107:0/1230317177 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45e0102100 0x7f45e010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.060+0000 7f45e6438640 1 --2- 172.21.15.107:0/1230317177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45e0101740 0x7f45e0101bc0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.060+0000 7f45e6438640 1 --2- 172.21.15.107:0/1230317177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45e0107750 0x7f45e0107b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.060+0000 7f45e6438640 1 -- 172.21.15.107:0/1230317177 >> 172.21.15.107:0/1230317177 conn(0x7f45e0078bd0 msgr2=0x7f45e0078fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:39.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.061+0000 7f45e6438640 1 -- 172.21.15.107:0/1230317177 shutdown_connections 2024-04-25T00:17:39.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.061+0000 7f45e6438640 1 -- 172.21.15.107:0/1230317177 wait complete. 2024-04-25T00:17:39.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.062+0000 7f45e6438640 1 Processor -- start 2024-04-25T00:17:39.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.062+0000 7f45e6438640 1 -- start start 2024-04-25T00:17:39.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.062+0000 7f45e6438640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45e0101740 0x7f45e01980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.062+0000 7f45e6438640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45e0102100 0x7f45e01985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.062+0000 7f45e6438640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45e0107750 0x7f45e019c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.062+0000 7f45e6438640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f45e0111b20 con 0x7f45e0101740 2024-04-25T00:17:39.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.062+0000 7f45e6438640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f45e01119a0 con 0x7f45e0107750 2024-04-25T00:17:39.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.062+0000 7f45e6438640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f45e0111ca0 con 0x7f45e0102100 2024-04-25T00:17:39.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.062+0000 7f45dffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45e0101740 0x7f45e01980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.062+0000 7f45dffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45e0101740 0x7f45e01980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52144/0 (socket says 172.21.15.107:52144) 2024-04-25T00:17:39.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.062+0000 7f45dffff640 1 -- 172.21.15.107:0/3045041794 learned_addr learned my addr 172.21.15.107:0/3045041794 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:39.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.063+0000 7f45df7fe640 1 --2- 172.21.15.107:0/3045041794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45e0102100 0x7f45e01985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.063+0000 7f45e49d5640 1 --2- 172.21.15.107:0/3045041794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45e0107750 0x7f45e019c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.063+0000 7f45dffff640 1 -- 172.21.15.107:0/3045041794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45e0102100 msgr2=0x7f45e01985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.063+0000 7f45dffff640 1 --2- 172.21.15.107:0/3045041794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45e0102100 0x7f45e01985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.063+0000 7f45dffff640 1 -- 172.21.15.107:0/3045041794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45e0107750 msgr2=0x7f45e019c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.063+0000 7f45dffff640 1 --2- 172.21.15.107:0/3045041794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45e0107750 0x7f45e019c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.063+0000 7f45dffff640 1 -- 172.21.15.107:0/3045041794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e019d040 con 0x7f45e0101740 2024-04-25T00:17:39.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.063+0000 7f45df7fe640 1 --2- 172.21.15.107:0/3045041794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45e0102100 0x7f45e01985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:17:39.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.063+0000 7f45dffff640 1 --2- 172.21.15.107:0/3045041794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45e0101740 0x7f45e01980a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f45d000ea60 tx=0x7f45d000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:39.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.063+0000 7f45dd7fa640 1 -- 172.21.15.107:0/3045041794 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d000cde0 con 0x7f45e0101740 2024-04-25T00:17:39.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.063+0000 7f45e6438640 1 -- 172.21.15.107:0/3045041794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e019d330 con 0x7f45e0101740 2024-04-25T00:17:39.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.063+0000 7f45dd7fa640 1 -- 172.21.15.107:0/3045041794 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d00035f0 con 0x7f45e0101740 2024-04-25T00:17:39.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.063+0000 7f45e6438640 1 -- 172.21.15.107:0/3045041794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e0073970 con 0x7f45e0101740 2024-04-25T00:17:39.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.064+0000 7f45dd7fa640 1 -- 172.21.15.107:0/3045041794 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0010670 con 0x7f45e0101740 2024-04-25T00:17:39.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.066+0000 7f45e9223740 1 Processor -- start 2024-04-25T00:17:39.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.066+0000 7f45e9223740 1 -- start start 2024-04-25T00:17:39.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.066+0000 7f45e9223740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb2297e980 0x55eb2297ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.066+0000 7f45e9223740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb2297d290 0x55eb2297d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.066+0000 7f45e9223740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb22988010 0x55eb2298a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.066+0000 7f45e9223740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55eb228e12c0 con 0x55eb22988010 2024-04-25T00:17:39.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.066+0000 7f45e9223740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55eb22889910 con 0x55eb2297d290 2024-04-25T00:17:39.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.066+0000 7f45e9223740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55eb228dd280 con 0x55eb2297e980 2024-04-25T00:17:39.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.066+0000 7f45df7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb22988010 0x55eb2298a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.067+0000 7f45df7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb22988010 0x55eb2298a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52160/0 (socket says 172.21.15.107:52160) 2024-04-25T00:17:39.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.067+0000 7f45df7fe640 1 -- 172.21.15.107:0/1376959404 learned_addr learned my addr 172.21.15.107:0/1376959404 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:39.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.067+0000 7f45e49d5640 1 --2- 172.21.15.107:0/1376959404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb2297e980 0x55eb2297ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.067+0000 7f45dd7fa640 1 -- 172.21.15.107:0/3045041794 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f45d0004000 con 0x7f45e0101740 2024-04-25T00:17:39.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.067+0000 7f45e49d5640 1 -- 172.21.15.107:0/1376959404 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb2297d290 msgr2=0x55eb2297d710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:17:39.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.067+0000 7f45e49d5640 1 --2- 172.21.15.107:0/1376959404 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb2297d290 0x55eb2297d710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.067+0000 7f45e49d5640 1 -- 172.21.15.107:0/1376959404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb22988010 msgr2=0x55eb2298a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:39.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.067+0000 7f45e49d5640 1 --2- 172.21.15.107:0/1376959404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb22988010 0x55eb2298a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.067+0000 7f45e49d5640 1 -- 172.21.15.107:0/1376959404 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb2289afb0 con 0x55eb2297e980 2024-04-25T00:17:39.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.067+0000 7f45df7fe640 1 --2- 172.21.15.107:0/1376959404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb22988010 0x55eb2298a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:39.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.067+0000 7f45e49d5640 1 --2- 172.21.15.107:0/1376959404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb2297e980 0x55eb2297ccc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f45d8009e10 tx=0x7f45d8010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=742187f03a431c63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:39.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.067+0000 7f45c67fc640 1 -- 172.21.15.107:0/1376959404 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8016070 con 0x55eb2297e980 2024-04-25T00:17:39.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.068+0000 7f45c67fc640 1 -- 172.21.15.107:0/1376959404 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d8004d20 con 0x55eb2297e980 2024-04-25T00:17:39.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.068+0000 7f45dd7fa640 1 --2- 172.21.15.107:0/3045041794 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f45c80a2e10 0x7f45c80a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.068+0000 7f45c67fc640 1 -- 172.21.15.107:0/1376959404 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8012790 con 0x55eb2297e980 2024-04-25T00:17:39.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.068+0000 7f45dd7fa640 1 -- 172.21.15.107:0/3045041794 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f45d00cfe70 con 0x7f45e0101740 2024-04-25T00:17:39.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.068+0000 7f45dffff640 1 --2- 172.21.15.107:0/3045041794 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f45c80a2e10 0x7f45c80a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.068+0000 7f45e9223740 1 -- 172.21.15.107:0/1376959404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb2297e980 msgr2=0x55eb2297ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.068+0000 7f45e9223740 1 --2- 172.21.15.107:0/1376959404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb2297e980 0x55eb2297ccc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f45d8009e10 tx=0x7f45d8010740 comp rx=0 tx=0).stop 2024-04-25T00:17:39.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.068+0000 7f45e9223740 1 -- 172.21.15.107:0/1376959404 shutdown_connections 2024-04-25T00:17:39.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.068+0000 7f45e9223740 1 --2- 172.21.15.107:0/1376959404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb22988010 0x55eb2298a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.068+0000 7f45e9223740 1 --2- 172.21.15.107:0/1376959404 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb2297d290 0x55eb2297d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.068+0000 7f45e9223740 1 --2- 172.21.15.107:0/1376959404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb2297e980 0x55eb2297ccc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.068+0000 7f45e9223740 1 -- 172.21.15.107:0/1376959404 >> 172.21.15.107:0/1376959404 conn(0x55eb22945950 msgr2=0x55eb229797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:39.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.069+0000 7f45dffff640 1 --2- 172.21.15.107:0/3045041794 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f45c80a2e10 0x7f45c80a52d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f45d00cd070 tx=0x7f45d0098080 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:39.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.069+0000 7f45e9223740 1 -- 172.21.15.107:0/1376959404 shutdown_connections 2024-04-25T00:17:39.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.069+0000 7f45e9223740 1 -- 172.21.15.107:0/1376959404 wait complete. 2024-04-25T00:17:39.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.069+0000 7f45e9223740 1 -- start start 2024-04-25T00:17:39.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.069+0000 7f45e9223740 1 Processor -- start 2024-04-25T00:17:39.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.070+0000 7f45e9223740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb2297d290 0x55eb22984f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.070+0000 7f45e9223740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb2297e980 0x55eb229854d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.070+0000 7f45e9223740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb22988010 0x55eb22985a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.070+0000 7f45e9223740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55eb2298aea0 con 0x55eb22988010 2024-04-25T00:17:39.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.070+0000 7f45e9223740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55eb2289ce90 con 0x55eb2297d290 2024-04-25T00:17:39.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.070+0000 7f45e9223740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55eb2298b020 con 0x55eb2297e980 2024-04-25T00:17:39.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.070+0000 7f45e9223740 10 client.0 _ll_register_callbacks cb 0x55eb228d8970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:39.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.070+0000 7f45e49d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb22988010 0x55eb22985a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.070+0000 7f45e49d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb22988010 0x55eb22985a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52172/0 (socket says 172.21.15.107:52172) 2024-04-25T00:17:39.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.070+0000 7f45e49d5640 1 -- 172.21.15.107:0/3955847327 learned_addr learned my addr 172.21.15.107:0/3955847327 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:39.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.070+0000 7f45df7fe640 1 --2- 172.21.15.107:0/3955847327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb2297e980 0x55eb229854d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.070+0000 7f45e49d5640 1 --2- 172.21.15.107:0/3955847327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb2297d290 0x55eb22984f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.070+0000 7f45e49d5640 1 -- 172.21.15.107:0/3955847327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb2297e980 msgr2=0x55eb229854d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:39.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.070+0000 7f45e49d5640 1 --2- 172.21.15.107:0/3955847327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb2297e980 0x55eb229854d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.070+0000 7f45e49d5640 1 -- 172.21.15.107:0/3955847327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb2297d290 msgr2=0x55eb22984f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:39.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.070+0000 7f45e49d5640 1 --2- 172.21.15.107:0/3955847327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb2297d290 0x55eb22984f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.070+0000 7f45e49d5640 1 -- 172.21.15.107:0/3955847327 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb226bd7e0 con 0x55eb22988010 2024-04-25T00:17:39.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.070+0000 7f45e49d5640 1 --2- 172.21.15.107:0/3955847327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb22988010 0x55eb22985a10 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f45d8012490 tx=0x7f45d8010f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2516b1a555777209 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:39.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.070+0000 7f45bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:17:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.070+0000 7f45bf7fe640 1 -- 172.21.15.107:0/3955847327 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8022040 con 0x55eb22988010 2024-04-25T00:17:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.071+0000 7f45bf7fe640 1 -- 172.21.15.107:0/3955847327 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f45d8012a60 con 0x55eb22988010 2024-04-25T00:17:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.071+0000 7f45e9223740 1 -- 172.21.15.107:0/3955847327 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45d8004d20 con 0x55eb22988010 2024-04-25T00:17:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.071+0000 7f45bf7fe640 1 -- 172.21.15.107:0/3955847327 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f45d8016040 con 0x55eb22988010 2024-04-25T00:17:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.071+0000 7f45e9223740 10 client.15927 fetch_fsmap learned FSMap version 24 2024-04-25T00:17:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.071+0000 7f45e9223740 1 -- 172.21.15.107:0/3955847327 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55eb2271df70 con 0x55eb22988010 2024-04-25T00:17:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.071+0000 7f45bf7fe640 1 -- 172.21.15.107:0/3955847327 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d8004d20 con 0x55eb22988010 2024-04-25T00:17:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.071+0000 7f45bf7fe640 1 -- 172.21.15.107:0/3955847327 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f45d800f040 con 0x55eb22988010 2024-04-25T00:17:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.071+0000 7f45e9223740 10 client.15927 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:17:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.071+0000 7f45e9223740 10 client.15927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.071+0000 7f45e9223740 20 client.15927 populate_metadata read hostname 'smithi107' 2024-04-25T00:17:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.071+0000 7f45e9223740 1 --2- 172.21.15.107:0/3955847327 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55eb22998060 0x55eb2299a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.071+0000 7f45e9223740 4 client.15927 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:17:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.071+0000 7f45e9223740 1 -- 172.21.15.107:0/3955847327 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb228b55a0 con 0x55eb22998060 2024-04-25T00:17:39.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.072+0000 7f45df7fe640 1 --2- 172.21.15.107:0/3955847327 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55eb22998060 0x55eb2299a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.072+0000 7f45df7fe640 1 --2- 172.21.15.107:0/3955847327 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55eb22998060 0x55eb2299a450 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cff47b40b7c107f1 server_cookie=f02793f42c2a6901 in_seq=0 out_seq=0 2024-04-25T00:17:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.072+0000 7f45bf7fe640 10 client.15927 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.073+0000 7f45bf7fe640 1 -- 172.21.15.107:0/3955847327 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb228b55a0 con 0x55eb22998060 2024-04-25T00:17:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.073+0000 7f45bf7fe640 10 client.15927 handle_command_reply: tid=0 2024-04-25T00:17:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.073+0000 7f45e9223740 1 client.15927 shutdown 2024-04-25T00:17:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.073+0000 7f45e9223740 1 -- 172.21.15.107:0/3955847327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb22988010 msgr2=0x55eb22985a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.073+0000 7f45e9223740 1 --2- 172.21.15.107:0/3955847327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb22988010 0x55eb22985a10 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f45d8012490 tx=0x7f45d8010f80 comp rx=0 tx=0).stop 2024-04-25T00:17:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.073+0000 7f45e9223740 1 -- 172.21.15.107:0/3955847327 shutdown_connections 2024-04-25T00:17:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.073+0000 7f45e9223740 1 --2- 172.21.15.107:0/3955847327 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55eb22998060 0x55eb2299a450 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.073+0000 7f45e9223740 1 --2- 172.21.15.107:0/3955847327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb22988010 0x55eb22985a10 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.073+0000 7f45e9223740 1 --2- 172.21.15.107:0/3955847327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb2297e980 0x55eb229854d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.073+0000 7f45bf7fe640 0 client.15927 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:39.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.073+0000 7f45e9223740 1 --2- 172.21.15.107:0/3955847327 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb2297d290 0x55eb22984f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.073+0000 7f45e9223740 1 -- 172.21.15.107:0/3955847327 >> 172.21.15.107:0/3955847327 conn(0x55eb22945950 msgr2=0x55eb2297f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:39.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.073+0000 7f45e9223740 1 -- 172.21.15.107:0/3955847327 shutdown_connections 2024-04-25T00:17:39.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.073+0000 7f45e9223740 1 -- 172.21.15.107:0/3955847327 wait complete. 2024-04-25T00:17:39.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.073+0000 7f45e9223740 20 client.15927 trim_cache size 0 max 16384 2024-04-25T00:17:39.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.090+0000 7f45e9223740 1 Processor -- start 2024-04-25T00:17:39.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.090+0000 7f45e9223740 1 -- start start 2024-04-25T00:17:39.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.091+0000 7f45e9223740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb2297d290 0x55eb22993e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.091+0000 7f45e9223740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb2297e980 0x55eb2299f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.091+0000 7f45e9223740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb22988010 0x55eb2299ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.091+0000 7f45e9223740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc0126b0 con 0x55eb2297d290 2024-04-25T00:17:39.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.091+0000 7f45e9223740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc00ff80 con 0x55eb22988010 2024-04-25T00:17:39.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.091+0000 7f45e9223740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc00ddd0 con 0x55eb2297e980 2024-04-25T00:17:39.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.091+0000 7f45e49d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb2297d290 0x55eb22993e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.091+0000 7f45e49d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb2297d290 0x55eb22993e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52184/0 (socket says 172.21.15.107:52184) 2024-04-25T00:17:39.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.091+0000 7f45e49d5640 1 -- 172.21.15.107:0/106890311 learned_addr learned my addr 172.21.15.107:0/106890311 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:39.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.091+0000 7f45e49d5640 1 -- 172.21.15.107:0/106890311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb2297e980 msgr2=0x55eb2299f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:39.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.091+0000 7f45df7fe640 1 --2- 172.21.15.107:0/106890311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb2297e980 0x55eb2299f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.091+0000 7f45e49d5640 1 --2- 172.21.15.107:0/106890311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb2297e980 0x55eb2299f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.091+0000 7f45e49d5640 1 -- 172.21.15.107:0/106890311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb22988010 msgr2=0x55eb2299ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:39.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.091+0000 7f45e49d5640 1 --2- 172.21.15.107:0/106890311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb22988010 0x55eb2299ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.091+0000 7f45e49d5640 1 -- 172.21.15.107:0/106890311 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb225b5170 con 0x55eb2297d290 2024-04-25T00:17:39.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.091+0000 7f45e49d5640 1 --2- 172.21.15.107:0/106890311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb2297d290 0x55eb22993e00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f45d8012500 tx=0x7f45d80096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9ca664386ff3c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:39.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.091+0000 7f45c5ffb640 1 -- 172.21.15.107:0/106890311 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d803b070 con 0x55eb2297d290 2024-04-25T00:17:39.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.091+0000 7f45c5ffb640 1 -- 172.21.15.107:0/106890311 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d8005bd0 con 0x55eb2297d290 2024-04-25T00:17:39.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.091+0000 7f45c5ffb640 1 -- 172.21.15.107:0/106890311 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d8029460 con 0x55eb2297d290 2024-04-25T00:17:39.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.092+0000 7f45e9223740 1 -- 172.21.15.107:0/106890311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb2297d290 msgr2=0x55eb22993e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.092+0000 7f45e9223740 1 --2- 172.21.15.107:0/106890311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb2297d290 0x55eb22993e00 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f45d8012500 tx=0x7f45d80096d0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.092+0000 7f45e9223740 1 -- 172.21.15.107:0/106890311 shutdown_connections 2024-04-25T00:17:39.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.092+0000 7f45e9223740 1 --2- 172.21.15.107:0/106890311 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb22988010 0x55eb2299ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.092+0000 7f45e9223740 1 --2- 172.21.15.107:0/106890311 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb2297e980 0x55eb2299f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.092+0000 7f45e9223740 1 --2- 172.21.15.107:0/106890311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb2297d290 0x55eb22993e00 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.092+0000 7f45e9223740 1 -- 172.21.15.107:0/106890311 >> 172.21.15.107:0/106890311 conn(0x55eb22945950 msgr2=0x55eb22978950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:39.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.092+0000 7f45e9223740 1 -- 172.21.15.107:0/106890311 shutdown_connections 2024-04-25T00:17:39.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.092+0000 7f45e9223740 1 -- 172.21.15.107:0/106890311 wait complete. 2024-04-25T00:17:39.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.093+0000 7f45e9223740 1 -- start start 2024-04-25T00:17:39.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.093+0000 7f45e9223740 1 Processor -- start 2024-04-25T00:17:39.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.093+0000 7f45e9223740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb2297d290 0x55eb22990310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.093+0000 7f45e9223740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb2297e980 0x55eb22990850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.093+0000 7f45e9223740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb22988010 0x55eb2298d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.093+0000 7f45e9223740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc00e440 con 0x55eb2297e980 2024-04-25T00:17:39.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.093+0000 7f45e9223740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc00f8b0 con 0x55eb22988010 2024-04-25T00:17:39.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.093+0000 7f45e9223740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc00eb10 con 0x55eb2297d290 2024-04-25T00:17:39.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.093+0000 7f45e9223740 10 client.0 _ll_register_callbacks cb 0x7f45d0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:39.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.093+0000 7f45df7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb2297e980 0x55eb22990850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.093+0000 7f45df7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb2297e980 0x55eb22990850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52200/0 (socket says 172.21.15.107:52200) 2024-04-25T00:17:39.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.093+0000 7f45df7fe640 1 -- 172.21.15.107:0/176597052 learned_addr learned my addr 172.21.15.107:0/176597052 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:39.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.093+0000 7f45e49d5640 1 --2- 172.21.15.107:0/176597052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb22988010 0x55eb2298d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.093+0000 7f45e49d5640 1 --2- 172.21.15.107:0/176597052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb2297d290 0x55eb22990310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.094+0000 7f45df7fe640 1 -- 172.21.15.107:0/176597052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb2297d290 msgr2=0x55eb22990310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:39.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.094+0000 7f45df7fe640 1 --2- 172.21.15.107:0/176597052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb2297d290 0x55eb22990310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.094+0000 7f45df7fe640 1 -- 172.21.15.107:0/176597052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb22988010 msgr2=0x55eb2298d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:39.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.094+0000 7f45df7fe640 1 --2- 172.21.15.107:0/176597052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb22988010 0x55eb2298d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.094+0000 7f45df7fe640 1 -- 172.21.15.107:0/176597052 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb2298daf0 con 0x55eb2297e980 2024-04-25T00:17:39.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.094+0000 7f45e49d5640 1 --2- 172.21.15.107:0/176597052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb22988010 0x55eb2298d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:17:39.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.094+0000 7f45df7fe640 1 --2- 172.21.15.107:0/176597052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb2297e980 0x55eb22990850 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f45d4009230 tx=0x7f45d40040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93231b25ff0cad74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:39.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.094+0000 7f45bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:17:39.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.094+0000 7f45bf7fe640 1 -- 172.21.15.107:0/176597052 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d4035040 con 0x55eb2297e980 2024-04-25T00:17:39.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.094+0000 7f45bf7fe640 1 -- 172.21.15.107:0/176597052 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f45d4045dc0 con 0x55eb2297e980 2024-04-25T00:17:39.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.094+0000 7f45e9223740 1 -- 172.21.15.107:0/176597052 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45d8005bd0 con 0x55eb2297e980 2024-04-25T00:17:39.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.094+0000 7f45bf7fe640 1 -- 172.21.15.107:0/176597052 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f45d4052050 con 0x55eb2297e980 2024-04-25T00:17:39.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.094+0000 7f45e9223740 10 client.15933 fetch_fsmap learned FSMap version 24 2024-04-25T00:17:39.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.094+0000 7f45e9223740 1 -- 172.21.15.107:0/176597052 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55eb229fee10 con 0x55eb2297e980 2024-04-25T00:17:39.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.094+0000 7f45bf7fe640 1 -- 172.21.15.107:0/176597052 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d8005bd0 con 0x55eb2297e980 2024-04-25T00:17:39.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.095+0000 7f45bf7fe640 1 -- 172.21.15.107:0/176597052 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f45d404d030 con 0x55eb2297e980 2024-04-25T00:17:39.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.095+0000 7f45e9223740 10 client.15933 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:17:39.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.095+0000 7f45e9223740 10 client.15933 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:39.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.095+0000 7f45e9223740 20 client.15933 populate_metadata read hostname 'smithi107' 2024-04-25T00:17:39.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.095+0000 7f45e9223740 1 --2- 172.21.15.107:0/176597052 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55eb229ff5c0 0x55eb22a019e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.095+0000 7f45e9223740 4 client.15933 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:17:39.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.095+0000 7f45e9223740 1 -- 172.21.15.107:0/176597052 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55eb228ee800 con 0x55eb229ff5c0 2024-04-25T00:17:39.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.095+0000 7f45e49d5640 1 --2- 172.21.15.107:0/176597052 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55eb229ff5c0 0x55eb22a019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.095+0000 7f45bf7fe640 10 client.15933 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:39.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.095+0000 7f45e49d5640 1 --2- 172.21.15.107:0/176597052 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55eb229ff5c0 0x55eb22a019e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=450695c05289d6e6 server_cookie=57944d40b467f839 in_seq=0 out_seq=0 2024-04-25T00:17:39.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.097+0000 7f45bf7fe640 1 -- 172.21.15.107:0/176597052 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55eb228ee800 con 0x55eb229ff5c0 2024-04-25T00:17:39.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.097+0000 7f45bf7fe640 10 client.15933 handle_command_reply: tid=0 2024-04-25T00:17:39.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.097+0000 7f45e9223740 1 client.15933 shutdown 2024-04-25T00:17:39.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.097+0000 7f45e9223740 1 -- 172.21.15.107:0/176597052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb2297e980 msgr2=0x55eb22990850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.097+0000 7f45e9223740 1 --2- 172.21.15.107:0/176597052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb2297e980 0x55eb22990850 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f45d4009230 tx=0x7f45d40040b0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.097+0000 7f45e9223740 1 -- 172.21.15.107:0/176597052 shutdown_connections 2024-04-25T00:17:39.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.097+0000 7f45e9223740 1 --2- 172.21.15.107:0/176597052 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55eb229ff5c0 0x55eb22a019e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.097+0000 7f45e9223740 1 --2- 172.21.15.107:0/176597052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55eb22988010 0x55eb2298d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.097+0000 7f45bf7fe640 0 client.15933 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:39.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.097+0000 7f45e9223740 1 --2- 172.21.15.107:0/176597052 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eb2297e980 0x55eb22990850 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.097+0000 7f45e9223740 1 --2- 172.21.15.107:0/176597052 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55eb2297d290 0x55eb22990310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.097+0000 7f45e9223740 1 -- 172.21.15.107:0/176597052 >> 172.21.15.107:0/176597052 conn(0x55eb22945950 msgr2=0x55eb2297ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:39.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.098+0000 7f45e9223740 1 -- 172.21.15.107:0/176597052 shutdown_connections 2024-04-25T00:17:39.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.098+0000 7f45e9223740 1 -- 172.21.15.107:0/176597052 wait complete. 2024-04-25T00:17:39.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.098+0000 7f45e9223740 20 client.15933 trim_cache size 0 max 16384 2024-04-25T00:17:39.102 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:17:39.103 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-04-25T00:17:39.103 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "e2127913-239c-45ab-a769-084d3587eb3e", 2024-04-25T00:17:39.103 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-04-25T00:17:39.103 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:17:39.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.099+0000 7f45e6438640 1 -- 172.21.15.107:0/3045041794 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f45c80a2e10 msgr2=0x7f45c80a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.099+0000 7f45e6438640 1 --2- 172.21.15.107:0/3045041794 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f45c80a2e10 0x7f45c80a52d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f45d00cd070 tx=0x7f45d0098080 comp rx=0 tx=0).stop 2024-04-25T00:17:39.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.099+0000 7f45e6438640 1 -- 172.21.15.107:0/3045041794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45e0101740 msgr2=0x7f45e01980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.099+0000 7f45e6438640 1 --2- 172.21.15.107:0/3045041794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45e0101740 0x7f45e01980a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f45d000ea60 tx=0x7f45d000ef20 comp rx=0 tx=0).stop 2024-04-25T00:17:39.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.099+0000 7f45e6438640 1 -- 172.21.15.107:0/3045041794 shutdown_connections 2024-04-25T00:17:39.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.099+0000 7f45e6438640 1 --2- 172.21.15.107:0/3045041794 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f45c80a2e10 0x7f45c80a52d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.099+0000 7f45e6438640 1 --2- 172.21.15.107:0/3045041794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45e0107750 0x7f45e019c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.099+0000 7f45e6438640 1 --2- 172.21.15.107:0/3045041794 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45e0102100 0x7f45e01985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.099+0000 7f45e6438640 1 --2- 172.21.15.107:0/3045041794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45e0101740 0x7f45e01980a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.099+0000 7f45e6438640 1 -- 172.21.15.107:0/3045041794 >> 172.21.15.107:0/3045041794 conn(0x7f45e0078bd0 msgr2=0x7f45e00ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:39.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.099+0000 7f45e6438640 1 -- 172.21.15.107:0/3045041794 shutdown_connections 2024-04-25T00:17:39.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.099+0000 7f45e6438640 1 -- 172.21.15.107:0/3045041794 wait complete. 2024-04-25T00:17:39.111 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:17:39.111 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:17:39.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.212+0000 7f4904e5d640 1 -- 172.21.15.107:0/3949817578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49001032e0 msgr2=0x7f490010b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.212+0000 7f4904e5d640 1 --2- 172.21.15.107:0/3949817578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49001032e0 0x7f490010b7c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009900 tx=0x7f48f402a510 comp rx=0 tx=0).stop 2024-04-25T00:17:39.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.213+0000 7f4904e5d640 1 -- 172.21.15.107:0/3949817578 shutdown_connections 2024-04-25T00:17:39.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.213+0000 7f4904e5d640 1 --2- 172.21.15.107:0/3949817578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f490010bd00 0x7f490010e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.213+0000 7f4904e5d640 1 --2- 172.21.15.107:0/3949817578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49001032e0 0x7f490010b7c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.213+0000 7f4904e5d640 1 --2- 172.21.15.107:0/3949817578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49001029c0 0x7f4900102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.213+0000 7f4904e5d640 1 -- 172.21.15.107:0/3949817578 >> 172.21.15.107:0/3949817578 conn(0x7f4900077740 msgr2=0x7f4900077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:39.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.213+0000 7f4904e5d640 1 -- 172.21.15.107:0/3949817578 shutdown_connections 2024-04-25T00:17:39.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.213+0000 7f4904e5d640 1 -- 172.21.15.107:0/3949817578 wait complete. 2024-04-25T00:17:39.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.214+0000 7f4904e5d640 1 Processor -- start 2024-04-25T00:17:39.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.214+0000 7f4904e5d640 1 -- start start 2024-04-25T00:17:39.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.214+0000 7f4904e5d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49001029c0 0x7f4900198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.214+0000 7f4904e5d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49001032e0 0x7f4900198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.214+0000 7f4904e5d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f490010bd00 0x7f490019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.214+0000 7f4904e5d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4900111b00 con 0x7f490010bd00 2024-04-25T00:17:39.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.214+0000 7f4904e5d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4900111980 con 0x7f49001032e0 2024-04-25T00:17:39.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.214+0000 7f4904e5d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4900111c80 con 0x7f49001029c0 2024-04-25T00:17:39.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.214+0000 7f48fe59c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49001029c0 0x7f4900198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.214+0000 7f48fed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f490010bd00 0x7f490019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.214+0000 7f48fe59c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49001029c0 0x7f4900198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:51732/0 (socket says 172.21.15.107:51732) 2024-04-25T00:17:39.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.214+0000 7f48fe59c640 1 -- 172.21.15.107:0/2892786170 learned_addr learned my addr 172.21.15.107:0/2892786170 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:39.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.214+0000 7f48fdd9b640 1 --2- 172.21.15.107:0/2892786170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49001032e0 0x7f4900198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.214+0000 7f48fe59c640 1 -- 172.21.15.107:0/2892786170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49001032e0 msgr2=0x7f4900198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.214+0000 7f48fe59c640 1 --2- 172.21.15.107:0/2892786170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49001032e0 0x7f4900198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.214+0000 7f48fe59c640 1 -- 172.21.15.107:0/2892786170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f490010bd00 msgr2=0x7f490019c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.214+0000 7f48fe59c640 1 --2- 172.21.15.107:0/2892786170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f490010bd00 0x7f490019c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.214+0000 7f48fe59c640 1 -- 172.21.15.107:0/2892786170 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490019cfc0 con 0x7f49001029c0 2024-04-25T00:17:39.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.214+0000 7f48fed9d640 1 --2- 172.21.15.107:0/2892786170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f490010bd00 0x7f490019c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:39.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.215+0000 7f48fe59c640 1 --2- 172.21.15.107:0/2892786170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49001029c0 0x7f4900198020 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f48f000d770 tx=0x7f48f000dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:39.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.215+0000 7f48ef7fe640 1 -- 172.21.15.107:0/2892786170 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f0017070 con 0x7f49001029c0 2024-04-25T00:17:39.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.215+0000 7f48ef7fe640 1 -- 172.21.15.107:0/2892786170 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48f0005500 con 0x7f49001029c0 2024-04-25T00:17:39.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.215+0000 7f4904e5d640 1 -- 172.21.15.107:0/2892786170 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f490019d2b0 con 0x7f49001029c0 2024-04-25T00:17:39.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.215+0000 7f48ef7fe640 1 -- 172.21.15.107:0/2892786170 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f0017070 con 0x7f49001029c0 2024-04-25T00:17:39.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.215+0000 7f4904e5d640 1 -- 172.21.15.107:0/2892786170 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49001a4b40 con 0x7f49001029c0 2024-04-25T00:17:39.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.218+0000 7f4907b6c740 1 Processor -- start 2024-04-25T00:17:39.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.219+0000 7f4907b6c740 1 -- start start 2024-04-25T00:17:39.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.219+0000 7f48ef7fe640 1 -- 172.21.15.107:0/2892786170 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 24) ==== 138118+0+0 (secure 0 0 0) 0x7f48f0004c00 con 0x7f49001029c0 2024-04-25T00:17:39.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.219+0000 7f4907b6c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af546630 0x5574af546a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.219+0000 7f4907b6c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574af546f50 0x5574af5510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.219+0000 7f4907b6c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574af551620 0x5574af553a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.219+0000 7f4907b6c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5574af4ab6b0 con 0x5574af546630 2024-04-25T00:17:39.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.219+0000 7f4907b6c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5574af453880 con 0x5574af551620 2024-04-25T00:17:39.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.219+0000 7f4907b6c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5574af4a7670 con 0x5574af546f50 2024-04-25T00:17:39.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.219+0000 7f48fed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af546630 0x5574af546a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.219+0000 7f48fed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af546630 0x5574af546a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52232/0 (socket says 172.21.15.107:52232) 2024-04-25T00:17:39.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.219+0000 7f48fed9d640 1 -- 172.21.15.107:0/446699158 learned_addr learned my addr 172.21.15.107:0/446699158 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:39.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.219+0000 7f48fdd9b640 1 --2- 172.21.15.107:0/446699158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574af551620 0x5574af553a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.220+0000 7f48fe59c640 1 --2- 172.21.15.107:0/446699158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574af546f50 0x5574af5510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.220+0000 7f48fed9d640 1 -- 172.21.15.107:0/446699158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574af546f50 msgr2=0x5574af5510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:39.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.220+0000 7f48ef7fe640 1 --2- 172.21.15.107:0/2892786170 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f48e80a30b0 0x7f48e80a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.220+0000 7f48ef7fe640 1 -- 172.21.15.107:0/2892786170 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f48f00cf090 con 0x7f49001029c0 2024-04-25T00:17:39.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.220+0000 7f48fed9d640 1 --2- 172.21.15.107:0/446699158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574af546f50 0x5574af5510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.220+0000 7f48fed9d640 1 -- 172.21.15.107:0/446699158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574af551620 msgr2=0x5574af553a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:39.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.220+0000 7f48fed9d640 1 --2- 172.21.15.107:0/446699158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574af551620 0x5574af553a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.220+0000 7f48fed9d640 1 -- 172.21.15.107:0/446699158 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574af2897d0 con 0x5574af546630 2024-04-25T00:17:39.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.220+0000 7f48fdd9b640 1 --2- 172.21.15.107:0/446699158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574af551620 0x5574af553a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:39.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.220+0000 7f48fe59c640 1 --2- 172.21.15.107:0/446699158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574af546f50 0x5574af5510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:17:39.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.220+0000 7f48fed9d640 1 --2- 172.21.15.107:0/446699158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af546630 0x5574af546a10 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f48f800f9f0 tx=0x7f48f800feb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f9637443fe750a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:39.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.221+0000 7f48fed9d640 1 --2- 172.21.15.107:0/2892786170 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f48e80a30b0 0x7f48e80a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.221+0000 7f48ecff9640 1 -- 172.21.15.107:0/446699158 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8010900 con 0x5574af546630 2024-04-25T00:17:39.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.221+0000 7f48ecff9640 1 -- 172.21.15.107:0/446699158 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48f801a400 con 0x5574af546630 2024-04-25T00:17:39.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.221+0000 7f48ecff9640 1 -- 172.21.15.107:0/446699158 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8019600 con 0x5574af546630 2024-04-25T00:17:39.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.222+0000 7f4907b6c740 1 -- 172.21.15.107:0/446699158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af546630 msgr2=0x5574af546a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.222+0000 7f48fed9d640 1 --2- 172.21.15.107:0/2892786170 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f48e80a30b0 0x7f48e80a5570 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f48f801e000 tx=0x7f48f801f000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:39.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.222+0000 7f4907b6c740 1 --2- 172.21.15.107:0/446699158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af546630 0x5574af546a10 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f48f800f9f0 tx=0x7f48f800feb0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.222+0000 7f4907b6c740 1 -- 172.21.15.107:0/446699158 shutdown_connections 2024-04-25T00:17:39.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.222+0000 7f4907b6c740 1 --2- 172.21.15.107:0/446699158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574af551620 0x5574af553a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.222+0000 7f4907b6c740 1 --2- 172.21.15.107:0/446699158 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574af546f50 0x5574af5510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.222+0000 7f4907b6c740 1 --2- 172.21.15.107:0/446699158 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af546630 0x5574af546a10 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.222+0000 7f4907b6c740 1 -- 172.21.15.107:0/446699158 >> 172.21.15.107:0/446699158 conn(0x5574af542950 msgr2=0x5574af542d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:39.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.223+0000 7f4907b6c740 1 -- 172.21.15.107:0/446699158 shutdown_connections 2024-04-25T00:17:39.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.223+0000 7f4907b6c740 1 -- 172.21.15.107:0/446699158 wait complete. 2024-04-25T00:17:39.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.223+0000 7f4907b6c740 1 -- start start 2024-04-25T00:17:39.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.223+0000 7f4907b6c740 1 Processor -- start 2024-04-25T00:17:39.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.223+0000 7f4907b6c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574af546630 0x5574af551060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.223+0000 7f4907b6c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af546f50 0x5574af54e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.223+0000 7f4907b6c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574af551620 0x5574af54eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.223+0000 7f4907b6c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5574af558820 con 0x5574af546f50 2024-04-25T00:17:39.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.224+0000 7f4907b6c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5574af466f50 con 0x5574af551620 2024-04-25T00:17:39.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.224+0000 7f4907b6c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5574af5589a0 con 0x5574af546630 2024-04-25T00:17:39.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.224+0000 7f4907b6c740 10 client.0 _ll_register_callbacks cb 0x5574af4a2000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:39.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.224+0000 7f48fdd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af546f50 0x5574af54e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.224+0000 7f48fdd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af546f50 0x5574af54e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52238/0 (socket says 172.21.15.107:52238) 2024-04-25T00:17:39.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.224+0000 7f48fdd9b640 1 -- 172.21.15.107:0/1499299869 learned_addr learned my addr 172.21.15.107:0/1499299869 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:39.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.224+0000 7f48fe59c640 1 --2- 172.21.15.107:0/1499299869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574af546630 0x5574af551060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.224+0000 7f48fed9d640 1 --2- 172.21.15.107:0/1499299869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574af551620 0x5574af54eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.224+0000 7f48fdd9b640 1 -- 172.21.15.107:0/1499299869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574af546630 msgr2=0x5574af551060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:39.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.224+0000 7f48fdd9b640 1 --2- 172.21.15.107:0/1499299869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574af546630 0x5574af551060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.224+0000 7f48fdd9b640 1 -- 172.21.15.107:0/1499299869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574af551620 msgr2=0x5574af54eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:39.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.224+0000 7f48fdd9b640 1 --2- 172.21.15.107:0/1499299869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574af551620 0x5574af54eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.224+0000 7f48fdd9b640 1 -- 172.21.15.107:0/1499299869 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574af1bf3f0 con 0x5574af546f50 2024-04-25T00:17:39.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.224+0000 7f48fed9d640 1 --2- 172.21.15.107:0/1499299869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574af551620 0x5574af54eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:17:39.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.224+0000 7f48fdd9b640 1 --2- 172.21.15.107:0/1499299869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af546f50 0x5574af54e650 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f48f402aa30 tx=0x7f48f40314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53560fb1a0e9b9e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:39.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.224+0000 7f48e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:17:39.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.224+0000 7f48e5ffb640 1 -- 172.21.15.107:0/1499299869 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f4035040 con 0x5574af546f50 2024-04-25T00:17:39.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.225+0000 7f48e5ffb640 1 -- 172.21.15.107:0/1499299869 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f48f4004480 con 0x5574af546f50 2024-04-25T00:17:39.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.225+0000 7f4907b6c740 1 -- 172.21.15.107:0/1499299869 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48f801a400 con 0x5574af546f50 2024-04-25T00:17:39.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.225+0000 7f48e5ffb640 1 -- 172.21.15.107:0/1499299869 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f48f4005020 con 0x5574af546f50 2024-04-25T00:17:39.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.225+0000 7f4907b6c740 10 client.15945 fetch_fsmap learned FSMap version 24 2024-04-25T00:17:39.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.225+0000 7f4907b6c740 1 -- 172.21.15.107:0/1499299869 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5574af17f0f0 con 0x5574af546f50 2024-04-25T00:17:39.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.225+0000 7f48e5ffb640 1 -- 172.21.15.107:0/1499299869 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48f801a400 con 0x5574af546f50 2024-04-25T00:17:39.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.226+0000 7f48e5ffb640 1 -- 172.21.15.107:0/1499299869 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f48f4003630 con 0x5574af546f50 2024-04-25T00:17:39.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.226+0000 7f4907b6c740 10 client.15945 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:17:39.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.226+0000 7f4907b6c740 10 client.15945 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:39.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.226+0000 7f4907b6c740 20 client.15945 populate_metadata read hostname 'smithi107' 2024-04-25T00:17:39.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.226+0000 7f4907b6c740 1 --2- 172.21.15.107:0/1499299869 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5574af55b980 0x5574af55dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.226+0000 7f4907b6c740 4 client.15945 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:17:39.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.226+0000 7f4907b6c740 1 -- 172.21.15.107:0/1499299869 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574af4b8bf0 con 0x5574af55b980 2024-04-25T00:17:39.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.226+0000 7f48fe59c640 1 --2- 172.21.15.107:0/1499299869 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5574af55b980 0x5574af55dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.226+0000 7f48e5ffb640 10 client.15945 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:39.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.226+0000 7f48fe59c640 1 --2- 172.21.15.107:0/1499299869 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5574af55b980 0x5574af55dd90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c191402247d146de server_cookie=bdb068021a3a27fd in_seq=0 out_seq=0 2024-04-25T00:17:39.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.227+0000 7f48e5ffb640 1 -- 172.21.15.107:0/1499299869 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5574af4b8bf0 con 0x5574af55b980 2024-04-25T00:17:39.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.227+0000 7f48e5ffb640 10 client.15945 handle_command_reply: tid=0 2024-04-25T00:17:39.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.227+0000 7f4907b6c740 1 client.15945 shutdown 2024-04-25T00:17:39.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.227+0000 7f4907b6c740 1 -- 172.21.15.107:0/1499299869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af546f50 msgr2=0x5574af54e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.227+0000 7f4907b6c740 1 --2- 172.21.15.107:0/1499299869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af546f50 0x5574af54e650 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f48f402aa30 tx=0x7f48f40314e0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.227+0000 7f4907b6c740 1 -- 172.21.15.107:0/1499299869 shutdown_connections 2024-04-25T00:17:39.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.227+0000 7f4907b6c740 1 --2- 172.21.15.107:0/1499299869 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5574af55b980 0x5574af55dd90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.227+0000 7f48e5ffb640 0 client.15945 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:39.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.227+0000 7f4907b6c740 1 --2- 172.21.15.107:0/1499299869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574af551620 0x5574af54eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.227+0000 7f4907b6c740 1 --2- 172.21.15.107:0/1499299869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af546f50 0x5574af54e650 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.227+0000 7f4907b6c740 1 --2- 172.21.15.107:0/1499299869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574af546630 0x5574af551060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.227+0000 7f4907b6c740 1 -- 172.21.15.107:0/1499299869 >> 172.21.15.107:0/1499299869 conn(0x5574af542950 msgr2=0x5574af548750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:39.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.227+0000 7f4907b6c740 1 -- 172.21.15.107:0/1499299869 shutdown_connections 2024-04-25T00:17:39.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.227+0000 7f4907b6c740 1 -- 172.21.15.107:0/1499299869 wait complete. 2024-04-25T00:17:39.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.227+0000 7f4907b6c740 20 client.15945 trim_cache size 0 max 16384 2024-04-25T00:17:39.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.262+0000 7f4907b6c740 1 Processor -- start 2024-04-25T00:17:39.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.262+0000 7f4907b6c740 1 -- start start 2024-04-25T00:17:39.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.262+0000 7f4907b6c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574af546630 0x5574af548ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.262+0000 7f4907b6c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574af551620 0x5574af57d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.262+0000 7f4907b6c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af55b980 0x5574af57d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.262+0000 7f4907b6c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f49001af160 con 0x5574af55b980 2024-04-25T00:17:39.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.262+0000 7f4907b6c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f49001aca30 con 0x5574af551620 2024-04-25T00:17:39.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.262+0000 7f4907b6c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f49001aa880 con 0x5574af546630 2024-04-25T00:17:39.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.262+0000 7f48fed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af55b980 0x5574af57d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.262+0000 7f48fed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af55b980 0x5574af57d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52246/0 (socket says 172.21.15.107:52246) 2024-04-25T00:17:39.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.262+0000 7f48fed9d640 1 -- 172.21.15.107:0/2204741778 learned_addr learned my addr 172.21.15.107:0/2204741778 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:39.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.262+0000 7f48fe59c640 1 --2- 172.21.15.107:0/2204741778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574af546630 0x5574af548ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.263+0000 7f48fdd9b640 1 --2- 172.21.15.107:0/2204741778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574af551620 0x5574af57d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.263+0000 7f48fed9d640 1 -- 172.21.15.107:0/2204741778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574af546630 msgr2=0x5574af548ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:39.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.263+0000 7f48fed9d640 1 --2- 172.21.15.107:0/2204741778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574af546630 0x5574af548ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.263+0000 7f48fed9d640 1 -- 172.21.15.107:0/2204741778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574af551620 msgr2=0x5574af57d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:39.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.263+0000 7f48fed9d640 1 --2- 172.21.15.107:0/2204741778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574af551620 0x5574af57d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.263+0000 7f48fed9d640 1 -- 172.21.15.107:0/2204741778 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574af553d80 con 0x5574af55b980 2024-04-25T00:17:39.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.263+0000 7f48fdd9b640 1 --2- 172.21.15.107:0/2204741778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574af551620 0x5574af57d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:17:39.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.263+0000 7f48fed9d640 1 --2- 172.21.15.107:0/2204741778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af55b980 0x5574af57d9f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f48f800f9f0 tx=0x7f48f8025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b549ccecab354aa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:39.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.263+0000 7f48e7fff640 1 -- 172.21.15.107:0/2204741778 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f802c040 con 0x5574af55b980 2024-04-25T00:17:39.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.263+0000 7f48e7fff640 1 -- 172.21.15.107:0/2204741778 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48f8003e50 con 0x5574af55b980 2024-04-25T00:17:39.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.263+0000 7f48e7fff640 1 -- 172.21.15.107:0/2204741778 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f801a660 con 0x5574af55b980 2024-04-25T00:17:39.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.264+0000 7f4907b6c740 1 -- 172.21.15.107:0/2204741778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af55b980 msgr2=0x5574af57d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.264+0000 7f4907b6c740 1 --2- 172.21.15.107:0/2204741778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af55b980 0x5574af57d9f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f48f800f9f0 tx=0x7f48f8025000 comp rx=0 tx=0).stop 2024-04-25T00:17:39.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.264+0000 7f4907b6c740 1 -- 172.21.15.107:0/2204741778 shutdown_connections 2024-04-25T00:17:39.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.264+0000 7f4907b6c740 1 --2- 172.21.15.107:0/2204741778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af55b980 0x5574af57d9f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.264+0000 7f4907b6c740 1 --2- 172.21.15.107:0/2204741778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574af551620 0x5574af57d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.264+0000 7f4907b6c740 1 --2- 172.21.15.107:0/2204741778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574af546630 0x5574af548ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.264+0000 7f4907b6c740 1 -- 172.21.15.107:0/2204741778 >> 172.21.15.107:0/2204741778 conn(0x5574af542950 msgr2=0x5574af54cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:39.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.264+0000 7f4907b6c740 1 -- 172.21.15.107:0/2204741778 shutdown_connections 2024-04-25T00:17:39.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.264+0000 7f4907b6c740 1 -- 172.21.15.107:0/2204741778 wait complete. 2024-04-25T00:17:39.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.265+0000 7f4907b6c740 1 -- start start 2024-04-25T00:17:39.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.265+0000 7f4907b6c740 1 Processor -- start 2024-04-25T00:17:39.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.265+0000 7f4907b6c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574af546630 0x5574af563670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.265+0000 7f4907b6c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574af551620 0x5574af563bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.265+0000 7f4907b6c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af55b980 0x5574af54aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.266+0000 7f48fed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af55b980 0x5574af54aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.266+0000 7f48fed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af55b980 0x5574af54aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52258/0 (socket says 172.21.15.107:52258) 2024-04-25T00:17:39.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.266+0000 7f48fed9d640 1 -- 172.21.15.107:0/2552199678 learned_addr learned my addr 172.21.15.107:0/2552199678 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:17:39.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.266+0000 7f4907b6c740 1 -- 172.21.15.107:0/2552199678 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f49001aaef0 con 0x5574af55b980 2024-04-25T00:17:39.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.266+0000 7f4907b6c740 1 -- 172.21.15.107:0/2552199678 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f49001ac360 con 0x5574af551620 2024-04-25T00:17:39.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.266+0000 7f4907b6c740 1 -- 172.21.15.107:0/2552199678 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f49001ab5c0 con 0x5574af546630 2024-04-25T00:17:39.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.266+0000 7f48fe59c640 1 --2- 172.21.15.107:0/2552199678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574af546630 0x5574af563670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.266+0000 7f4907b6c740 10 client.0 _ll_register_callbacks cb 0x7f48f40315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:17:39.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.266+0000 7f48fdd9b640 1 --2- 172.21.15.107:0/2552199678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574af551620 0x5574af563bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.266+0000 7f48fed9d640 1 -- 172.21.15.107:0/2552199678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574af546630 msgr2=0x5574af563670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:39.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.266+0000 7f48fed9d640 1 --2- 172.21.15.107:0/2552199678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574af546630 0x5574af563670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.266+0000 7f48fed9d640 1 -- 172.21.15.107:0/2552199678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574af551620 msgr2=0x5574af563bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:17:39.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.266+0000 7f48fed9d640 1 --2- 172.21.15.107:0/2552199678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574af551620 0x5574af563bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.266+0000 7f48fed9d640 1 -- 172.21.15.107:0/2552199678 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574af5577c0 con 0x5574af55b980 2024-04-25T00:17:39.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.266+0000 7f48fdd9b640 1 --2- 172.21.15.107:0/2552199678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574af551620 0x5574af563bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:17:39.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.266+0000 7f48fe59c640 1 --2- 172.21.15.107:0/2552199678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574af546630 0x5574af563670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:39.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.266+0000 7f48fed9d640 1 --2- 172.21.15.107:0/2552199678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af55b980 0x5574af54aac0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f48f8007560 tx=0x7f48f8007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb853068fc491a9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:39.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.266+0000 7f48e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:17:39.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.267+0000 7f48e5ffb640 1 -- 172.21.15.107:0/2552199678 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f801a400 con 0x5574af55b980 2024-04-25T00:17:39.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.267+0000 7f48e5ffb640 1 -- 172.21.15.107:0/2552199678 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f48f801a5a0 con 0x5574af55b980 2024-04-25T00:17:39.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.267+0000 7f4907b6c740 1 -- 172.21.15.107:0/2552199678 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48f8003e50 con 0x5574af55b980 2024-04-25T00:17:39.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.267+0000 7f48e5ffb640 1 -- 172.21.15.107:0/2552199678 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f48f8048050 con 0x5574af55b980 2024-04-25T00:17:39.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.267+0000 7f4907b6c740 10 client.15951 fetch_fsmap learned FSMap version 24 2024-04-25T00:17:39.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.267+0000 7f4907b6c740 1 -- 172.21.15.107:0/2552199678 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5574af5d9580 con 0x5574af55b980 2024-04-25T00:17:39.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.267+0000 7f48e5ffb640 1 -- 172.21.15.107:0/2552199678 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48f8003e50 con 0x5574af55b980 2024-04-25T00:17:39.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.267+0000 7f48e5ffb640 1 -- 172.21.15.107:0/2552199678 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f48f802f070 con 0x5574af55b980 2024-04-25T00:17:39.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.267+0000 7f4907b6c740 10 client.15951 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:17:39.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.267+0000 7f4907b6c740 10 client.15951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:17:39.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.267+0000 7f4907b6c740 20 client.15951 populate_metadata read hostname 'smithi107' 2024-04-25T00:17:39.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.267+0000 7f4907b6c740 1 --2- 172.21.15.107:0/2552199678 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5574af5d9b80 0x5574af5dbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.267+0000 7f4907b6c740 4 client.15951 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:17:39.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.267+0000 7f4907b6c740 1 -- 172.21.15.107:0/2552199678 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5574af4b2f40 con 0x5574af5d9b80 2024-04-25T00:17:39.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.268+0000 7f48fe59c640 1 --2- 172.21.15.107:0/2552199678 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5574af5d9b80 0x5574af5dbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.268+0000 7f48fe59c640 1 --2- 172.21.15.107:0/2552199678 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5574af5d9b80 0x5574af5dbfa0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d64d4b81fffec5e3 server_cookie=2c668be24f3a07d1 in_seq=0 out_seq=0 2024-04-25T00:17:39.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.268+0000 7f48e5ffb640 10 client.15951 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:39.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.269+0000 7f48e5ffb640 1 -- 172.21.15.107:0/2552199678 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5574af4b2f40 con 0x5574af5d9b80 2024-04-25T00:17:39.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.269+0000 7f48e5ffb640 10 client.15951 handle_command_reply: tid=0 2024-04-25T00:17:39.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.269+0000 7f4907b6c740 1 client.15951 shutdown 2024-04-25T00:17:39.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.269+0000 7f4907b6c740 1 -- 172.21.15.107:0/2552199678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af55b980 msgr2=0x5574af54aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.269+0000 7f4907b6c740 1 --2- 172.21.15.107:0/2552199678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af55b980 0x5574af54aac0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f48f8007560 tx=0x7f48f8007de0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.269+0000 7f4907b6c740 1 -- 172.21.15.107:0/2552199678 shutdown_connections 2024-04-25T00:17:39.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.269+0000 7f4907b6c740 1 --2- 172.21.15.107:0/2552199678 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5574af5d9b80 0x5574af5dbfa0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.269+0000 7f4907b6c740 1 --2- 172.21.15.107:0/2552199678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5574af55b980 0x5574af54aac0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.269+0000 7f4907b6c740 1 --2- 172.21.15.107:0/2552199678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5574af551620 0x5574af563bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.269+0000 7f48e5ffb640 0 client.15951 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:17:39.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.269+0000 7f4907b6c740 1 --2- 172.21.15.107:0/2552199678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5574af546630 0x5574af563670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.269+0000 7f4907b6c740 1 -- 172.21.15.107:0/2552199678 >> 172.21.15.107:0/2552199678 conn(0x5574af542950 msgr2=0x5574af541260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:39.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.269+0000 7f4907b6c740 1 -- 172.21.15.107:0/2552199678 shutdown_connections 2024-04-25T00:17:39.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.269+0000 7f4907b6c740 1 -- 172.21.15.107:0/2552199678 wait complete. 2024-04-25T00:17:39.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.269+0000 7f4907b6c740 20 client.15951 trim_cache size 0 max 16384 2024-04-25T00:17:39.275 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:17:39.275 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:17:39.275 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-04-25T00:17:39.275 INFO:teuthology.orchestra.run.smithi107.stdout: "e2127913-239c-45ab-a769-084d3587eb3e": { 2024-04-25T00:17:39.275 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-04-25T00:17:39.275 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "e2127913-239c-45ab-a769-084d3587eb3e", 2024-04-25T00:17:39.275 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-04-25T00:17:39.276 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:17:39.276 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:17:39.276 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:17:39.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.269+0000 7f4904e5d640 1 -- 172.21.15.107:0/2892786170 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f48e80a30b0 msgr2=0x7f48e80a5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.269+0000 7f4904e5d640 1 --2- 172.21.15.107:0/2892786170 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f48e80a30b0 0x7f48e80a5570 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f48f801e000 tx=0x7f48f801f000 comp rx=0 tx=0).stop 2024-04-25T00:17:39.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.270+0000 7f4904e5d640 1 -- 172.21.15.107:0/2892786170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49001029c0 msgr2=0x7f4900198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.270+0000 7f4904e5d640 1 --2- 172.21.15.107:0/2892786170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49001029c0 0x7f4900198020 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f48f000d770 tx=0x7f48f000dc30 comp rx=0 tx=0).stop 2024-04-25T00:17:39.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.270+0000 7f4904e5d640 1 -- 172.21.15.107:0/2892786170 shutdown_connections 2024-04-25T00:17:39.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.270+0000 7f4904e5d640 1 --2- 172.21.15.107:0/2892786170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f490010bd00 0x7f490019c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.270+0000 7f4904e5d640 1 --2- 172.21.15.107:0/2892786170 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f48e80a30b0 0x7f48e80a5570 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.270+0000 7f4904e5d640 1 --2- 172.21.15.107:0/2892786170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f49001032e0 0x7f4900198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.270+0000 7f4904e5d640 1 --2- 172.21.15.107:0/2892786170 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f49001029c0 0x7f4900198020 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.270+0000 7f4904e5d640 1 -- 172.21.15.107:0/2892786170 >> 172.21.15.107:0/2892786170 conn(0x7f4900077740 msgr2=0x7f49000fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:39.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.270+0000 7f4904e5d640 1 -- 172.21.15.107:0/2892786170 shutdown_connections 2024-04-25T00:17:39.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:17:39.270+0000 7f4904e5d640 1 -- 172.21.15.107:0/2892786170 wait complete. 2024-04-25T00:17:39.282 INFO:tasks.cephfs.filesystem:scrub status for tag:e2127913-239c-45ab-a769-084d3587eb3e - {'path': '/', 'tag': 'e2127913-239c-45ab-a769-084d3587eb3e', 'options': 'recursive,force'} 2024-04-25T00:17:40.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:39 smithi170 ceph-mon[32807]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:17:40.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:39 smithi170 ceph-mon[32807]: scrub queued for path: / 2024-04-25T00:17:40.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:39 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:17:40.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:39 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:17:40.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:39 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:17:40.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:39 smithi107 ceph-mon[29153]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:17:40.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:39 smithi107 ceph-mon[29153]: scrub queued for path: / 2024-04-25T00:17:40.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:39 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:17:40.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:39 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:17:40.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:39 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:17:40.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:39 smithi195 ceph-mon[32833]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:17:40.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:39 smithi195 ceph-mon[32833]: scrub queued for path: / 2024-04-25T00:17:40.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:39 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:17:40.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:39 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:17:40.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:39 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:17:42.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:41 smithi170 ceph-mon[32807]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:17:42.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:41 smithi107 ceph-mon[29153]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:17:42.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:41 smithi195 ceph-mon[32833]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:17:43.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:42 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:17:43.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:42 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:17:43.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:42 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:17:44.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:43 smithi107 ceph-mon[29153]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:17:44.240 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:43 smithi195 ceph-mon[32833]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:17:44.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:43 smithi170 ceph-mon[32807]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:17:46.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:45 smithi107 ceph-mon[29153]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:17:46.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:45 smithi195 ceph-mon[32833]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:17:46.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:45 smithi170 ceph-mon[32807]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:17:46.680 INFO:tasks.workunit.client.0.smithi107.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-25T00:17:46.681 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-04-25T00:17:46.681 INFO:tasks.workunit.client.0.smithi107.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T00:17:46.681 INFO:tasks.workunit.client.0.smithi107.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T00:17:46.681 INFO:tasks.workunit.client.0.smithi107.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T00:17:46.681 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-04-25T00:17:46.681 INFO:tasks.workunit.client.0.smithi107.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T00:17:46.681 INFO:tasks.workunit.client.0.smithi107.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T00:17:46.681 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-04-25T00:17:46.681 INFO:tasks.workunit.client.0.smithi107.stderr: git switch -c 2024-04-25T00:17:46.681 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-04-25T00:17:46.681 INFO:tasks.workunit.client.0.smithi107.stderr:Or undo this operation with: 2024-04-25T00:17:46.682 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-04-25T00:17:46.682 INFO:tasks.workunit.client.0.smithi107.stderr: git switch - 2024-04-25T00:17:46.682 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-04-25T00:17:46.682 INFO:tasks.workunit.client.0.smithi107.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T00:17:46.682 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-04-25T00:17:46.682 INFO:tasks.workunit.client.0.smithi107.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-25T00:17:46.684 DEBUG:teuthology.orchestra.run.smithi107:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T00:17:46.707 INFO:tasks.workunit.client.0.smithi107.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T00:17:46.710 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T00:17:46.710 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T00:17:47.366 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T00:17:47.419 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T00:17:47.459 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T00:17:47.461 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T00:17:47.461 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T00:17:47.499 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T00:17:47.502 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T00:17:47.502 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T00:17:47.558 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-04-25T00:17:47.559 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-25T00:17:47.560 DEBUG:teuthology.orchestra.run.smithi107:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-04-25T00:17:47.618 INFO:tasks.workunit.client.0.smithi107.stderr:+ mkdir -p fsstress 2024-04-25T00:17:47.620 INFO:tasks.workunit.client.0.smithi107.stderr:+ pushd fsstress 2024-04-25T00:17:47.620 INFO:tasks.workunit.client.0.smithi107.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-25T00:17:47.620 INFO:tasks.workunit.client.0.smithi107.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-25T00:17:48.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:47 smithi107 ceph-mon[29153]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:17:48.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:47 smithi195 ceph-mon[32833]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:17:48.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:47 smithi170 ceph-mon[32807]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:17:49.090 INFO:tasks.workunit.client.0.smithi107.stderr:+ tar xzf ltp-full.tgz 2024-04-25T00:17:50.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:49 smithi107 ceph-mon[29153]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:17:50.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:49 smithi195 ceph-mon[32833]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:17:50.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:49 smithi170 ceph-mon[32807]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:17:52.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:51 smithi107 ceph-mon[29153]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:17:52.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:51 smithi195 ceph-mon[32833]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:17:52.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:51 smithi170 ceph-mon[32807]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:17:54.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:53 smithi107 ceph-mon[29153]: pgmap v46: 129 pgs: 129 active+clean; 5.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 362 KiB/s wr, 11 op/s 2024-04-25T00:17:54.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:53 smithi107 ceph-mon[29153]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:17:54.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:53 smithi195 ceph-mon[32833]: pgmap v46: 129 pgs: 129 active+clean; 5.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 362 KiB/s wr, 11 op/s 2024-04-25T00:17:54.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:53 smithi195 ceph-mon[32833]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:17:54.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:53 smithi170 ceph-mon[32807]: pgmap v46: 129 pgs: 129 active+clean; 5.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 362 KiB/s wr, 11 op/s 2024-04-25T00:17:54.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:53 smithi170 ceph-mon[32807]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:17:56.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:55 smithi107 ceph-mon[29153]: pgmap v47: 129 pgs: 129 active+clean; 5.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 362 KiB/s wr, 8 op/s 2024-04-25T00:17:56.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:55 smithi195 ceph-mon[32833]: pgmap v47: 129 pgs: 129 active+clean; 5.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 362 KiB/s wr, 8 op/s 2024-04-25T00:17:56.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:55 smithi170 ceph-mon[32807]: pgmap v47: 129 pgs: 129 active+clean; 5.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 362 KiB/s wr, 8 op/s 2024-04-25T00:17:58.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:57 smithi107 ceph-mon[29153]: pgmap v48: 129 pgs: 129 active+clean; 5.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 362 KiB/s wr, 8 op/s 2024-04-25T00:17:58.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:57 smithi107 ceph-mon[29153]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:17:58.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:57 smithi195 ceph-mon[32833]: pgmap v48: 129 pgs: 129 active+clean; 5.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 362 KiB/s wr, 8 op/s 2024-04-25T00:17:58.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:57 smithi195 ceph-mon[32833]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:17:58.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:57 smithi170 ceph-mon[32807]: pgmap v48: 129 pgs: 129 active+clean; 5.3 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 362 KiB/s wr, 8 op/s 2024-04-25T00:17:58.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:57 smithi170 ceph-mon[32807]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:18:00.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:59 smithi107 ceph-mon[29153]: pgmap v49: 129 pgs: 129 active+clean; 38 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 257 op/s 2024-04-25T00:18:00.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:17:59 smithi107 ceph-mon[29153]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-04-25T00:18:00.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:59 smithi195 ceph-mon[32833]: pgmap v49: 129 pgs: 129 active+clean; 38 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 257 op/s 2024-04-25T00:18:00.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:17:59 smithi195 ceph-mon[32833]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-04-25T00:18:00.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:59 smithi170 ceph-mon[32807]: pgmap v49: 129 pgs: 129 active+clean; 38 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 257 op/s 2024-04-25T00:18:00.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:17:59 smithi170 ceph-mon[32807]: mgrmap e25: x(active, since 92s), standbys: z, y 2024-04-25T00:18:02.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:01 smithi107 ceph-mon[29153]: pgmap v50: 129 pgs: 129 active+clean; 38 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 254 op/s 2024-04-25T00:18:02.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:01 smithi195 ceph-mon[32833]: pgmap v50: 129 pgs: 129 active+clean; 38 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 254 op/s 2024-04-25T00:18:02.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:01 smithi170 ceph-mon[32807]: pgmap v50: 129 pgs: 129 active+clean; 38 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 254 op/s 2024-04-25T00:18:04.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:03 smithi107 ceph-mon[29153]: pgmap v51: 129 pgs: 129 active+clean; 46 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 304 op/s 2024-04-25T00:18:04.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:03 smithi195 ceph-mon[32833]: pgmap v51: 129 pgs: 129 active+clean; 46 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 304 op/s 2024-04-25T00:18:04.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:03 smithi170 ceph-mon[32807]: pgmap v51: 129 pgs: 129 active+clean; 46 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 304 op/s 2024-04-25T00:18:06.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:05 smithi107 ceph-mon[29153]: pgmap v52: 129 pgs: 129 active+clean; 46 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 298 op/s 2024-04-25T00:18:06.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:05 smithi195 ceph-mon[32833]: pgmap v52: 129 pgs: 129 active+clean; 46 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 298 op/s 2024-04-25T00:18:06.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:05 smithi170 ceph-mon[32807]: pgmap v52: 129 pgs: 129 active+clean; 46 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 298 op/s 2024-04-25T00:18:08.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:07 smithi107 ceph-mon[29153]: pgmap v53: 129 pgs: 129 active+clean; 46 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 298 op/s 2024-04-25T00:18:08.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:07 smithi195 ceph-mon[32833]: pgmap v53: 129 pgs: 129 active+clean; 46 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 298 op/s 2024-04-25T00:18:08.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:07 smithi170 ceph-mon[32807]: pgmap v53: 129 pgs: 129 active+clean; 46 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.7 MiB/s wr, 298 op/s 2024-04-25T00:18:08.828 INFO:tasks.workunit.client.0.smithi107.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-25T00:18:08.830 INFO:tasks.workunit.client.0.smithi107.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-25T00:18:08.830 INFO:tasks.workunit.client.0.smithi107.stderr:+ make 2024-04-25T00:18:08.939 INFO:tasks.workunit.client.0.smithi107.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-25T00:18:09.283 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:18:09.283 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:18:09.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.465+0000 7f8a9f590640 1 -- 172.21.15.107:0/1389904874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a98104160 msgr2=0x7f8a98104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:09.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.465+0000 7f8a9f590640 1 --2- 172.21.15.107:0/1389904874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a98104160 0x7f8a98104610 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009900 tx=0x7f8a9402a440 comp rx=0 tx=0).stop 2024-04-25T00:18:09.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.465+0000 7f8a9f590640 1 -- 172.21.15.107:0/1389904874 shutdown_connections 2024-04-25T00:18:09.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.465+0000 7f8a9f590640 1 --2- 172.21.15.107:0/1389904874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a98104160 0x7f8a98104610 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.465+0000 7f8a9f590640 1 --2- 172.21.15.107:0/1389904874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a9806a2d0 0x7f8a9806a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.465+0000 7f8a9f590640 1 --2- 172.21.15.107:0/1389904874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a98106660 0x7f8a98069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.465+0000 7f8a9f590640 1 -- 172.21.15.107:0/1389904874 >> 172.21.15.107:0/1389904874 conn(0x7f8a980776d0 msgr2=0x7f8a98077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:09.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.465+0000 7f8a9f590640 1 -- 172.21.15.107:0/1389904874 shutdown_connections 2024-04-25T00:18:09.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.465+0000 7f8a9f590640 1 -- 172.21.15.107:0/1389904874 wait complete. 2024-04-25T00:18:09.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.466+0000 7f8a9f590640 1 Processor -- start 2024-04-25T00:18:09.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.466+0000 7f8a9f590640 1 -- start start 2024-04-25T00:18:09.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.469+0000 7f8a9f590640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a9806a2d0 0x7f8a980721e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.469+0000 7f8a9f590640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a98104160 0x7f8a98072720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.469+0000 7f8a9f590640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a98106660 0x7f8a9810aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.469+0000 7f8a9f590640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8a981075d0 con 0x7f8a9806a2d0 2024-04-25T00:18:09.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.469+0000 7f8a9f590640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8a98107450 con 0x7f8a98106660 2024-04-25T00:18:09.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.469+0000 7f8a9f590640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8a98107750 con 0x7f8a98104160 2024-04-25T00:18:09.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.469+0000 7f8a9d32c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a9806a2d0 0x7f8a980721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:09.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.469+0000 7f8a9d32c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a9806a2d0 0x7f8a980721e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33748/0 (socket says 172.21.15.107:33748) 2024-04-25T00:18:09.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.469+0000 7f8a9db2d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a98106660 0x7f8a9810aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:09.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.469+0000 7f8a9d32c640 1 -- 172.21.15.107:0/271118814 learned_addr learned my addr 172.21.15.107:0/271118814 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:09.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.469+0000 7f8a9cb2b640 1 --2- 172.21.15.107:0/271118814 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a98104160 0x7f8a98072720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:09.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.469+0000 7f8a9d32c640 1 -- 172.21.15.107:0/271118814 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a98104160 msgr2=0x7f8a98072720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:09.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.469+0000 7f8a9d32c640 1 --2- 172.21.15.107:0/271118814 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a98104160 0x7f8a98072720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.469+0000 7f8a9d32c640 1 -- 172.21.15.107:0/271118814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a98106660 msgr2=0x7f8a9810aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:09.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.470+0000 7f8a9d32c640 1 --2- 172.21.15.107:0/271118814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a98106660 0x7f8a9810aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.470+0000 7f8a9d32c640 1 -- 172.21.15.107:0/271118814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9810b600 con 0x7f8a9806a2d0 2024-04-25T00:18:09.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.470+0000 7f8a9d32c640 1 --2- 172.21.15.107:0/271118814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a9806a2d0 0x7f8a980721e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800c890 tx=0x7f8a8800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:09.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.470+0000 7f8a8e7fc640 1 -- 172.21.15.107:0/271118814 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88004220 con 0x7f8a9806a2d0 2024-04-25T00:18:09.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.470+0000 7f8a8e7fc640 1 -- 172.21.15.107:0/271118814 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a880043c0 con 0x7f8a9806a2d0 2024-04-25T00:18:09.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.470+0000 7f8a8e7fc640 1 -- 172.21.15.107:0/271118814 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88004a00 con 0x7f8a9806a2d0 2024-04-25T00:18:09.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.472+0000 7f8a9f590640 1 -- 172.21.15.107:0/271118814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a9810b8f0 con 0x7f8a9806a2d0 2024-04-25T00:18:09.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.472+0000 7f8a9f590640 1 -- 172.21.15.107:0/271118814 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a981131a0 con 0x7f8a9806a2d0 2024-04-25T00:18:09.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.477+0000 7f8a8e7fc640 1 -- 172.21.15.107:0/271118814 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f8a880079b0 con 0x7f8a9806a2d0 2024-04-25T00:18:09.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.477+0000 7f8aa237c740 1 Processor -- start 2024-04-25T00:18:09.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.478+0000 7f8aa237c740 1 -- start start 2024-04-25T00:18:09.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.478+0000 7f8aa237c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604f630 0x55800604fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.478+0000 7f8a8e7fc640 1 --2- 172.21.15.107:0/271118814 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8a840a2e40 0x7f8a840a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.478+0000 7f8aa237c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55800604ff50 0x55800605a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.478+0000 7f8aa237c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55800605a620 0x55800605ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.478+0000 7f8aa237c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558005fb46b0 con 0x55800604ff50 2024-04-25T00:18:09.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.478+0000 7f8aa237c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558005f5c880 con 0x55800605a620 2024-04-25T00:18:09.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.478+0000 7f8aa237c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558005fb0670 con 0x55800604f630 2024-04-25T00:18:09.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.478+0000 7f8a8e7fc640 1 -- 172.21.15.107:0/271118814 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f8a880cf300 con 0x7f8a9806a2d0 2024-04-25T00:18:09.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.478+0000 7f8a9d32c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604f630 0x55800604fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:09.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.478+0000 7f8a9d32c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604f630 0x55800604fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:37924/0 (socket says 172.21.15.107:37924) 2024-04-25T00:18:09.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.478+0000 7f8a9d32c640 1 -- 172.21.15.107:0/1623791775 learned_addr learned my addr 172.21.15.107:0/1623791775 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:09.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.480+0000 7f8a9d32c640 1 -- 172.21.15.107:0/1623791775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55800605a620 msgr2=0x55800605ca10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:18:09.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.480+0000 7f8a9d32c640 1 --2- 172.21.15.107:0/1623791775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55800605a620 0x55800605ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.480+0000 7f8a9d32c640 1 -- 172.21.15.107:0/1623791775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55800604ff50 msgr2=0x55800605a0e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:18:09.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.480+0000 7f8a9d32c640 1 --2- 172.21.15.107:0/1623791775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55800604ff50 0x55800605a0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.480+0000 7f8a9d32c640 1 -- 172.21.15.107:0/1623791775 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558005d927d0 con 0x55800604f630 2024-04-25T00:18:09.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.480+0000 7f8a9d32c640 1 --2- 172.21.15.107:0/1623791775 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604f630 0x55800604fa10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8a88097ed0 tx=0x7f8a880978d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=634cbc64f898140 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:09.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.481+0000 7f8a837fe640 1 -- 172.21.15.107:0/1623791775 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8801d040 con 0x55800604f630 2024-04-25T00:18:09.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.483+0000 7f8a9db2d640 1 --2- 172.21.15.107:0/271118814 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8a840a2e40 0x7f8a840a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:09.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.483+0000 7f8a837fe640 1 -- 172.21.15.107:0/1623791775 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a880673f0 con 0x55800604f630 2024-04-25T00:18:09.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.483+0000 7f8a837fe640 1 -- 172.21.15.107:0/1623791775 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88007ca0 con 0x55800604f630 2024-04-25T00:18:09.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.484+0000 7f8aa237c740 1 -- 172.21.15.107:0/1623791775 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604f630 msgr2=0x55800604fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:09.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.484+0000 7f8aa237c740 1 --2- 172.21.15.107:0/1623791775 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604f630 0x55800604fa10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8a88097ed0 tx=0x7f8a880978d0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.485+0000 7f8aa237c740 1 -- 172.21.15.107:0/1623791775 shutdown_connections 2024-04-25T00:18:09.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.485+0000 7f8aa237c740 1 --2- 172.21.15.107:0/1623791775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55800605a620 0x55800605ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.485+0000 7f8aa237c740 1 --2- 172.21.15.107:0/1623791775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55800604ff50 0x55800605a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.485 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.485+0000 7f8aa237c740 1 --2- 172.21.15.107:0/1623791775 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604f630 0x55800604fa10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.485+0000 7f8aa237c740 1 -- 172.21.15.107:0/1623791775 >> 172.21.15.107:0/1623791775 conn(0x55800604b950 msgr2=0x55800604bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:09.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.485+0000 7f8a9db2d640 1 --2- 172.21.15.107:0/271118814 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8a840a2e40 0x7f8a840a5300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009530 tx=0x7f8a9402a940 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:09.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.486+0000 7f8aa237c740 1 -- 172.21.15.107:0/1623791775 shutdown_connections 2024-04-25T00:18:09.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.486+0000 7f8aa237c740 1 -- 172.21.15.107:0/1623791775 wait complete. 2024-04-25T00:18:09.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.486+0000 7f8aa237c740 1 -- start start 2024-04-25T00:18:09.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.486+0000 7f8aa237c740 1 Processor -- start 2024-04-25T00:18:09.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.487+0000 7f8aa237c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604f630 0x558006057510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.487+0000 7f8aa237c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55800604ff50 0x558006057a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.487+0000 7f8aa237c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55800605a620 0x558006057f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.487+0000 7f8aa237c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55800605d350 con 0x55800605a620 2024-04-25T00:18:09.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.487+0000 7f8aa237c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558005f6ff50 con 0x55800604ff50 2024-04-25T00:18:09.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.487+0000 7f8aa237c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55800605d4d0 con 0x55800604f630 2024-04-25T00:18:09.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.487+0000 7f8aa237c740 10 client.0 _ll_register_callbacks cb 0x558005fab000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:09.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.487+0000 7f8a9d32c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604f630 0x558006057510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:09.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.487+0000 7f8a9d32c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604f630 0x558006057510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:37938/0 (socket says 172.21.15.107:37938) 2024-04-25T00:18:09.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.487+0000 7f8a9d32c640 1 -- 172.21.15.107:0/128213230 learned_addr learned my addr 172.21.15.107:0/128213230 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:09.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.487+0000 7f8a9db2d640 1 --2- 172.21.15.107:0/128213230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55800605a620 0x558006057f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:09.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.488+0000 7f8a9d32c640 1 -- 172.21.15.107:0/128213230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55800604ff50 msgr2=0x558006057a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:09.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.488+0000 7f8a9d32c640 1 --2- 172.21.15.107:0/128213230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55800604ff50 0x558006057a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.488+0000 7f8a9d32c640 1 -- 172.21.15.107:0/128213230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55800605a620 msgr2=0x558006057f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:09.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.488+0000 7f8a9d32c640 1 --2- 172.21.15.107:0/128213230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55800605a620 0x558006057f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.488+0000 7f8a9d32c640 1 -- 172.21.15.107:0/128213230 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558005cc83f0 con 0x55800604f630 2024-04-25T00:18:09.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.488+0000 7f8a9d32c640 1 --2- 172.21.15.107:0/128213230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604f630 0x558006057510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8a8809b490 tx=0x7f8a880a40c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cc91db81c5a1b24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:09.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.488+0000 7f8a80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-04-25T00:18:09.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.489+0000 7f8a80ff9640 1 -- 172.21.15.107:0/128213230 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8801d040 con 0x55800604f630 2024-04-25T00:18:09.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.489+0000 7f8a80ff9640 1 -- 172.21.15.107:0/128213230 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f8a880a3c30 con 0x55800604f630 2024-04-25T00:18:09.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.489+0000 7f8aa237c740 1 -- 172.21.15.107:0/128213230 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a880673f0 con 0x55800604f630 2024-04-25T00:18:09.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.489+0000 7f8a80ff9640 1 -- 172.21.15.107:0/128213230 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8a880ccb00 con 0x55800604f630 2024-04-25T00:18:09.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.489+0000 7f8aa237c740 10 client.25399 fetch_fsmap learned FSMap version 26 2024-04-25T00:18:09.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.489+0000 7f8aa237c740 1 -- 172.21.15.107:0/128213230 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558005c880f0 con 0x55800604f630 2024-04-25T00:18:09.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.491+0000 7f8a80ff9640 1 -- 172.21.15.107:0/128213230 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a880673f0 con 0x55800604f630 2024-04-25T00:18:09.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.491+0000 7f8a80ff9640 1 -- 172.21.15.107:0/128213230 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f8a8806f470 con 0x55800604f630 2024-04-25T00:18:09.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.491+0000 7f8aa237c740 10 client.25399 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:18:09.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.491+0000 7f8aa237c740 10 client.25399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:18:09.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.491+0000 7f8aa237c740 20 client.25399 populate_metadata read hostname 'smithi107' 2024-04-25T00:18:09.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.491+0000 7f8aa237c740 1 --2- 172.21.15.107:0/128213230 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55800606a440 0x55800606c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.491+0000 7f8aa237c740 4 client.25399 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:18:09.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.491+0000 7f8aa237c740 1 -- 172.21.15.107:0/128213230 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558005fc1bf0 con 0x55800606a440 2024-04-25T00:18:09.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.492+0000 7f8a9cb2b640 1 --2- 172.21.15.107:0/128213230 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55800606a440 0x55800606c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:09.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.495+0000 7f8a9cb2b640 1 --2- 172.21.15.107:0/128213230 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55800606a440 0x55800606c830 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54924009c90cfe3d server_cookie=a645e33c537e1b16 in_seq=0 out_seq=0 2024-04-25T00:18:09.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.495+0000 7f8a80ff9640 10 client.25399 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:09.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.498+0000 7f8a80ff9640 1 -- 172.21.15.107:0/128213230 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558005fc1bf0 con 0x55800606a440 2024-04-25T00:18:09.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.498+0000 7f8a80ff9640 10 client.25399 handle_command_reply: tid=0 2024-04-25T00:18:09.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.498+0000 7f8aa237c740 1 client.25399 shutdown 2024-04-25T00:18:09.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.499+0000 7f8aa237c740 1 -- 172.21.15.107:0/128213230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604f630 msgr2=0x558006057510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:09.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.499+0000 7f8aa237c740 1 --2- 172.21.15.107:0/128213230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604f630 0x558006057510 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8a8809b490 tx=0x7f8a880a40c0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.501+0000 7f8aa237c740 1 -- 172.21.15.107:0/128213230 shutdown_connections 2024-04-25T00:18:09.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.501+0000 7f8aa237c740 1 --2- 172.21.15.107:0/128213230 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55800606a440 0x55800606c830 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.501+0000 7f8aa237c740 1 --2- 172.21.15.107:0/128213230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55800605a620 0x558006057f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.501+0000 7f8aa237c740 1 --2- 172.21.15.107:0/128213230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55800604ff50 0x558006057a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.501+0000 7f8aa237c740 1 --2- 172.21.15.107:0/128213230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604f630 0x558006057510 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.501+0000 7f8aa237c740 1 -- 172.21.15.107:0/128213230 >> 172.21.15.107:0/128213230 conn(0x55800604b950 msgr2=0x558006051630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:09.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.503+0000 7f8a80ff9640 0 client.25399 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:09.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.503+0000 7f8aa237c740 1 -- 172.21.15.107:0/128213230 shutdown_connections 2024-04-25T00:18:09.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.503+0000 7f8aa237c740 1 -- 172.21.15.107:0/128213230 wait complete. 2024-04-25T00:18:09.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.504+0000 7f8aa237c740 20 client.25399 trim_cache size 0 max 16384 2024-04-25T00:18:09.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.540+0000 7f8aa237c740 1 Processor -- start 2024-04-25T00:18:09.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.541+0000 7f8aa237c740 1 -- start start 2024-04-25T00:18:09.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.541+0000 7f8aa237c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604f630 0x558006059c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.541+0000 7f8aa237c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55800604ff50 0x558006066240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.541+0000 7f8aa237c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55800605a620 0x558006055e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.541+0000 7f8aa237c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c012850 con 0x55800604ff50 2024-04-25T00:18:09.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.541+0000 7f8aa237c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c0100c0 con 0x55800605a620 2024-04-25T00:18:09.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.541+0000 7f8aa237c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c00df10 con 0x55800604f630 2024-04-25T00:18:09.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.541+0000 7f8a9d32c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604f630 0x558006059c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:09.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.541+0000 7f8a9d32c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604f630 0x558006059c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:37950/0 (socket says 172.21.15.107:37950) 2024-04-25T00:18:09.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.541+0000 7f8a9d32c640 1 -- 172.21.15.107:0/350830111 learned_addr learned my addr 172.21.15.107:0/350830111 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:09.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.541+0000 7f8a9cb2b640 1 --2- 172.21.15.107:0/350830111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55800604ff50 0x558006066240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:09.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.541+0000 7f8a9db2d640 1 --2- 172.21.15.107:0/350830111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55800605a620 0x558006055e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:09.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.542+0000 7f8a9d32c640 1 -- 172.21.15.107:0/350830111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55800605a620 msgr2=0x558006055e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:09.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.542+0000 7f8a9d32c640 1 --2- 172.21.15.107:0/350830111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55800605a620 0x558006055e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.542+0000 7f8a9d32c640 1 -- 172.21.15.107:0/350830111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55800604ff50 msgr2=0x558006066240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:09.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.542+0000 7f8a9d32c640 1 --2- 172.21.15.107:0/350830111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55800604ff50 0x558006066240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.542+0000 7f8a9d32c640 1 -- 172.21.15.107:0/350830111 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55800603d090 con 0x55800604f630 2024-04-25T00:18:09.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.543+0000 7f8a9d32c640 1 --2- 172.21.15.107:0/350830111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604f630 0x558006059c40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8a88097e80 tx=0x7f8a88000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76e3a24cd9dc1374 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:09.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.544+0000 7f8a9cb2b640 1 --2- 172.21.15.107:0/350830111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55800604ff50 0x558006066240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:18:09.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.544+0000 7f8a82ffd640 1 -- 172.21.15.107:0/350830111 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a880b4180 con 0x55800604f630 2024-04-25T00:18:09.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.544+0000 7f8a82ffd640 1 -- 172.21.15.107:0/350830111 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a880cc900 con 0x55800604f630 2024-04-25T00:18:09.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.544+0000 7f8a82ffd640 1 -- 172.21.15.107:0/350830111 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a880a3c30 con 0x55800604f630 2024-04-25T00:18:09.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.545+0000 7f8aa237c740 1 -- 172.21.15.107:0/350830111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604f630 msgr2=0x558006059c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:09.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.545+0000 7f8aa237c740 1 --2- 172.21.15.107:0/350830111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604f630 0x558006059c40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8a88097e80 tx=0x7f8a88000bc0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.549+0000 7f8aa237c740 1 -- 172.21.15.107:0/350830111 shutdown_connections 2024-04-25T00:18:09.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.549+0000 7f8aa237c740 1 --2- 172.21.15.107:0/350830111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55800605a620 0x558006055e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.549+0000 7f8aa237c740 1 --2- 172.21.15.107:0/350830111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55800604ff50 0x558006066240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.549+0000 7f8aa237c740 1 --2- 172.21.15.107:0/350830111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604f630 0x558006059c40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.549+0000 7f8aa237c740 1 -- 172.21.15.107:0/350830111 >> 172.21.15.107:0/350830111 conn(0x55800604b950 msgr2=0x55800604aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:09.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.550+0000 7f8aa237c740 1 -- 172.21.15.107:0/350830111 shutdown_connections 2024-04-25T00:18:09.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.550+0000 7f8aa237c740 1 -- 172.21.15.107:0/350830111 wait complete. 2024-04-25T00:18:09.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.551+0000 7f8aa237c740 1 -- start start 2024-04-25T00:18:09.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.551+0000 7f8aa237c740 1 Processor -- start 2024-04-25T00:18:09.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.552+0000 7f8aa237c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55800604f630 0x5580060629b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.552+0000 7f8aa237c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604ff50 0x55800605f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.552+0000 7f8a9d32c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55800604f630 0x5580060629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:09.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.552+0000 7f8a9d32c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55800604f630 0x5580060629b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.107:46588/0 (socket says 172.21.15.107:46588) 2024-04-25T00:18:09.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.552+0000 7f8a9d32c640 1 -- 172.21.15.107:0/1173951208 learned_addr learned my addr 172.21.15.107:0/1173951208 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:09.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.553+0000 7f8aa237c740 1 --2- 172.21.15.107:0/1173951208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55800605a620 0x55800605fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.553+0000 7f8a9cb2b640 1 --2- 172.21.15.107:0/1173951208 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604ff50 0x55800605f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:09.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.553+0000 7f8aa237c740 1 -- 172.21.15.107:0/1173951208 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c00e580 con 0x55800605a620 2024-04-25T00:18:09.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.553+0000 7f8aa237c740 1 -- 172.21.15.107:0/1173951208 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c00f9f0 con 0x55800604f630 2024-04-25T00:18:09.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.553+0000 7f8aa237c740 1 -- 172.21.15.107:0/1173951208 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8a7c00ec50 con 0x55800604ff50 2024-04-25T00:18:09.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.553+0000 7f8aa237c740 10 client.0 _ll_register_callbacks cb 0x7f8a8809b0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:09.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.553+0000 7f8a9db2d640 1 --2- 172.21.15.107:0/1173951208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55800605a620 0x55800605fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:09.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.554+0000 7f8a9db2d640 1 -- 172.21.15.107:0/1173951208 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604ff50 msgr2=0x55800605f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:09.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.554+0000 7f8a9db2d640 1 --2- 172.21.15.107:0/1173951208 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604ff50 0x55800605f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.554+0000 7f8a9db2d640 1 -- 172.21.15.107:0/1173951208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55800604f630 msgr2=0x5580060629b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:09.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.554+0000 7f8a9db2d640 1 --2- 172.21.15.107:0/1173951208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55800604f630 0x5580060629b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.554+0000 7f8a9db2d640 1 -- 172.21.15.107:0/1173951208 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558006060300 con 0x55800605a620 2024-04-25T00:18:09.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.555+0000 7f8a9db2d640 1 --2- 172.21.15.107:0/1173951208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55800605a620 0x55800605fc60 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009850 tx=0x7f8a94009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0b988b4a6b424f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:09.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.556+0000 7f8a80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:18:09.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.556+0000 7f8a80ff9640 1 -- 172.21.15.107:0/1173951208 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a94035040 con 0x55800605a620 2024-04-25T00:18:09.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.556+0000 7f8a80ff9640 1 -- 172.21.15.107:0/1173951208 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f8a94031950 con 0x55800605a620 2024-04-25T00:18:09.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.556+0000 7f8aa237c740 1 -- 172.21.15.107:0/1173951208 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a880cc900 con 0x55800605a620 2024-04-25T00:18:09.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.556+0000 7f8a80ff9640 1 -- 172.21.15.107:0/1173951208 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8a94031b50 con 0x55800605a620 2024-04-25T00:18:09.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.557+0000 7f8aa237c740 10 client.15966 fetch_fsmap learned FSMap version 26 2024-04-25T00:18:09.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.557+0000 7f8aa237c740 1 -- 172.21.15.107:0/1173951208 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5580060d1600 con 0x55800605a620 2024-04-25T00:18:09.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.559+0000 7f8a80ff9640 1 -- 172.21.15.107:0/1173951208 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a880cc900 con 0x55800605a620 2024-04-25T00:18:09.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.559+0000 7f8a80ff9640 1 -- 172.21.15.107:0/1173951208 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f8a9403c440 con 0x55800605a620 2024-04-25T00:18:09.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.560+0000 7f8aa237c740 10 client.15966 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:18:09.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.560+0000 7f8aa237c740 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:18:09.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.560+0000 7f8aa237c740 20 client.15966 populate_metadata read hostname 'smithi107' 2024-04-25T00:18:09.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.560+0000 7f8aa237c740 1 --2- 172.21.15.107:0/1173951208 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5580060d1d60 0x5580060d4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.560+0000 7f8aa237c740 4 client.15966 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:18:09.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.560+0000 7f8aa237c740 1 -- 172.21.15.107:0/1173951208 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558005fbbf40 con 0x5580060d1d60 2024-04-25T00:18:09.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.560+0000 7f8a9d32c640 1 --2- 172.21.15.107:0/1173951208 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5580060d1d60 0x5580060d4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:09.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.564+0000 7f8a9d32c640 1 --2- 172.21.15.107:0/1173951208 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5580060d1d60 0x5580060d4180 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad6303f249a6b8fe server_cookie=d64fa41cb403b3b4 in_seq=0 out_seq=0 2024-04-25T00:18:09.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.565+0000 7f8a80ff9640 10 client.15966 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:09.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.565+0000 7f8a80ff9640 1 -- 172.21.15.107:0/1173951208 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558005fbbf40 con 0x5580060d1d60 2024-04-25T00:18:09.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.565+0000 7f8a80ff9640 10 client.15966 handle_command_reply: tid=0 2024-04-25T00:18:09.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.565+0000 7f8aa237c740 1 client.15966 shutdown 2024-04-25T00:18:09.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.567+0000 7f8aa237c740 1 -- 172.21.15.107:0/1173951208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55800605a620 msgr2=0x55800605fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:09.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.567+0000 7f8aa237c740 1 --2- 172.21.15.107:0/1173951208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55800605a620 0x55800605fc60 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009850 tx=0x7f8a94009260 comp rx=0 tx=0).stop 2024-04-25T00:18:09.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.569+0000 7f8aa237c740 1 -- 172.21.15.107:0/1173951208 shutdown_connections 2024-04-25T00:18:09.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.569+0000 7f8aa237c740 1 --2- 172.21.15.107:0/1173951208 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5580060d1d60 0x5580060d4180 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.569+0000 7f8aa237c740 1 --2- 172.21.15.107:0/1173951208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55800605a620 0x55800605fc60 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.569+0000 7f8aa237c740 1 --2- 172.21.15.107:0/1173951208 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55800604ff50 0x55800605f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.569+0000 7f8aa237c740 1 --2- 172.21.15.107:0/1173951208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55800604f630 0x5580060629b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.569+0000 7f8aa237c740 1 -- 172.21.15.107:0/1173951208 >> 172.21.15.107:0/1173951208 conn(0x55800604b950 msgr2=0x55800604aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:09.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.569+0000 7f8a80ff9640 0 client.15966 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:09.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.571+0000 7f8aa237c740 1 -- 172.21.15.107:0/1173951208 shutdown_connections 2024-04-25T00:18:09.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.573+0000 7f8aa237c740 1 -- 172.21.15.107:0/1173951208 wait complete. 2024-04-25T00:18:09.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.574+0000 7f8aa237c740 20 client.15966 trim_cache size 0 max 16384 2024-04-25T00:18:09.577 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:18:09.577 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-04-25T00:18:09.577 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-04-25T00:18:09.577 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:18:09.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.578+0000 7f8a9f590640 1 -- 172.21.15.107:0/271118814 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8a840a2e40 msgr2=0x7f8a840a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:09.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.578+0000 7f8a9f590640 1 --2- 172.21.15.107:0/271118814 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8a840a2e40 0x7f8a840a5300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009530 tx=0x7f8a9402a940 comp rx=0 tx=0).stop 2024-04-25T00:18:09.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.578+0000 7f8a9f590640 1 -- 172.21.15.107:0/271118814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a9806a2d0 msgr2=0x7f8a980721e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:09.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.578+0000 7f8a9f590640 1 --2- 172.21.15.107:0/271118814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a9806a2d0 0x7f8a980721e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800c890 tx=0x7f8a8800cd50 comp rx=0 tx=0).stop 2024-04-25T00:18:09.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.579+0000 7f8a9f590640 1 -- 172.21.15.107:0/271118814 shutdown_connections 2024-04-25T00:18:09.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.579+0000 7f8a9f590640 1 --2- 172.21.15.107:0/271118814 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8a840a2e40 0x7f8a840a5300 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.579+0000 7f8a9f590640 1 --2- 172.21.15.107:0/271118814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a98106660 0x7f8a9810aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.579+0000 7f8a9f590640 1 --2- 172.21.15.107:0/271118814 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8a98104160 0x7f8a98072720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.579+0000 7f8a9f590640 1 --2- 172.21.15.107:0/271118814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a9806a2d0 0x7f8a980721e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.579+0000 7f8a9f590640 1 -- 172.21.15.107:0/271118814 >> 172.21.15.107:0/271118814 conn(0x7f8a980776d0 msgr2=0x7f8a98103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:09.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.582+0000 7f8a9f590640 1 -- 172.21.15.107:0/271118814 shutdown_connections 2024-04-25T00:18:09.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.583+0000 7f8a9f590640 1 -- 172.21.15.107:0/271118814 wait complete. 2024-04-25T00:18:09.603 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:18:09.603 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:18:09.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.790+0000 7f6f14b10640 1 -- 172.21.15.107:0/2611592346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f101020e0 msgr2=0x7f6f1010e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:09.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.790+0000 7f6f0d59a640 1 -- 172.21.15.107:0/2611592346 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f080307f0 con 0x7f6f101020e0 2024-04-25T00:18:09.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.791+0000 7f6f14b10640 1 --2- 172.21.15.107:0/2611592346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f101020e0 0x7f6f1010e5f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009cd0 tx=0x7f6f0802a930 comp rx=0 tx=0).stop 2024-04-25T00:18:09.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.791+0000 7f6f14b10640 1 -- 172.21.15.107:0/2611592346 shutdown_connections 2024-04-25T00:18:09.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.791+0000 7f6f14b10640 1 --2- 172.21.15.107:0/2611592346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f101020e0 0x7f6f1010e5f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.791+0000 7f6f14b10640 1 --2- 172.21.15.107:0/2611592346 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f10101720 0x7f6f10101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.791+0000 7f6f14b10640 1 --2- 172.21.15.107:0/2611592346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f10107730 0x7f6f10107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.791+0000 7f6f14b10640 1 -- 172.21.15.107:0/2611592346 >> 172.21.15.107:0/2611592346 conn(0x7f6f10078b40 msgr2=0x7f6f10078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:09.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.792+0000 7f6f14b10640 1 -- 172.21.15.107:0/2611592346 shutdown_connections 2024-04-25T00:18:09.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.793+0000 7f6f14b10640 1 -- 172.21.15.107:0/2611592346 wait complete. 2024-04-25T00:18:09.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.794+0000 7f6f14b10640 1 Processor -- start 2024-04-25T00:18:09.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.795+0000 7f6f14b10640 1 -- start start 2024-04-25T00:18:09.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.795+0000 7f6f14b10640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f10101720 0x7f6f1010c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.796+0000 7f6f14b10640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f101020e0 0x7f6f1010ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.796+0000 7f6f14b10640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f10107730 0x7f6f1010d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.796+0000 7f6f14b10640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1010fa50 con 0x7f6f101020e0 2024-04-25T00:18:09.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.796+0000 7f6f14b10640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1010f8d0 con 0x7f6f10107730 2024-04-25T00:18:09.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.796+0000 7f6f14b10640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1010fbd0 con 0x7f6f10101720 2024-04-25T00:18:09.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.797+0000 7f6f0ed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f10107730 0x7f6f1010d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:09.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.797+0000 7f6f0ed9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f10107730 0x7f6f1010d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.107:46606/0 (socket says 172.21.15.107:46606) 2024-04-25T00:18:09.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.797+0000 7f6f0ed9d640 1 -- 172.21.15.107:0/2091968690 learned_addr learned my addr 172.21.15.107:0/2091968690 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:09.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.797+0000 7f6f0dd9b640 1 --2- 172.21.15.107:0/2091968690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f101020e0 0x7f6f1010ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:09.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.798+0000 7f6f0e59c640 1 --2- 172.21.15.107:0/2091968690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f10101720 0x7f6f1010c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:09.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.798+0000 7f6f0ed9d640 1 -- 172.21.15.107:0/2091968690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f10101720 msgr2=0x7f6f1010c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:09.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.798+0000 7f6f0ed9d640 1 --2- 172.21.15.107:0/2091968690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f10101720 0x7f6f1010c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.798+0000 7f6f0ed9d640 1 -- 172.21.15.107:0/2091968690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f101020e0 msgr2=0x7f6f1010ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:09.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.798+0000 7f6f0ed9d640 1 --2- 172.21.15.107:0/2091968690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f101020e0 0x7f6f1010ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.798+0000 7f6f0ed9d640 1 -- 172.21.15.107:0/2091968690 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f101aaa60 con 0x7f6f10107730 2024-04-25T00:18:09.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.799+0000 7f6f0ed9d640 1 --2- 172.21.15.107:0/2091968690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f10107730 0x7f6f1010d410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6f08006010 tx=0x7f6f080316b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:09.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.801+0000 7f6eff7fe640 1 -- 172.21.15.107:0/2091968690 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08004040 con 0x7f6f10107730 2024-04-25T00:18:09.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.801+0000 7f6eff7fe640 1 -- 172.21.15.107:0/2091968690 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f0802d070 con 0x7f6f10107730 2024-04-25T00:18:09.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.801+0000 7f6eff7fe640 1 -- 172.21.15.107:0/2091968690 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08030e30 con 0x7f6f10107730 2024-04-25T00:18:09.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.801+0000 7f6f14b10640 1 -- 172.21.15.107:0/2091968690 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f101aacf0 con 0x7f6f10107730 2024-04-25T00:18:09.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.801+0000 7f6f14b10640 1 -- 172.21.15.107:0/2091968690 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f101ab190 con 0x7f6f10107730 2024-04-25T00:18:09.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.803+0000 7f6f14b10640 1 -- 172.21.15.107:0/2091968690 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f10067cd0 con 0x7f6f10107730 2024-04-25T00:18:09.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.806+0000 7f6eff7fe640 1 -- 172.21.15.107:0/2091968690 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f6f080305f0 con 0x7f6f10107730 2024-04-25T00:18:09.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.807+0000 7f6eff7fe640 1 --2- 172.21.15.107:0/2091968690 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f6ef80a2f00 0x7f6ef80a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.807+0000 7f6eff7fe640 1 -- 172.21.15.107:0/2091968690 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f6f080f0da0 con 0x7f6f10107730 2024-04-25T00:18:09.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.808+0000 7f6f0e59c640 1 --2- 172.21.15.107:0/2091968690 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f6ef80a2f00 0x7f6ef80a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:09.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.811+0000 7f6f0e59c640 1 --2- 172.21.15.107:0/2091968690 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f6ef80a2f00 0x7f6ef80a53c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6f04006fd0 tx=0x7f6f040090e0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:09.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:09.816+0000 7f6eff7fe640 1 -- 172.21.15.107:0/2091968690 <== mon.2 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6f080bd780 con 0x7f6f10107730 2024-04-25T00:18:10.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:09 smithi107 ceph-mon[29153]: pgmap v54: 129 pgs: 129 active+clean; 55 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 324 op/s 2024-04-25T00:18:10.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.268+0000 7f6f14b10640 1 -- 172.21.15.107:0/2091968690 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f1010dec0 con 0x7f6f10107730 2024-04-25T00:18:10.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.269+0000 7f6eff7fe640 1 -- 172.21.15.107:0/2091968690 <== mon.2 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11903 (secure 0 0 0) 0x7f6f080bd080 con 0x7f6f10107730 2024-04-25T00:18:10.270 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:18:10.270 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:17:56.182729+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24493,24521]},"id":1}]} 2024-04-25T00:18:10.270 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 26 2024-04-25T00:18:10.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.272+0000 7f6f14b10640 1 -- 172.21.15.107:0/2091968690 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f6ef80a2f00 msgr2=0x7f6ef80a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.272+0000 7f6f14b10640 1 --2- 172.21.15.107:0/2091968690 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f6ef80a2f00 0x7f6ef80a53c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6f04006fd0 tx=0x7f6f040090e0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.272+0000 7f6f14b10640 1 -- 172.21.15.107:0/2091968690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f10107730 msgr2=0x7f6f1010d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.272+0000 7f6f14b10640 1 --2- 172.21.15.107:0/2091968690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f10107730 0x7f6f1010d410 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6f08006010 tx=0x7f6f080316b0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.273+0000 7f6f14b10640 1 -- 172.21.15.107:0/2091968690 shutdown_connections 2024-04-25T00:18:10.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.273+0000 7f6f14b10640 1 --2- 172.21.15.107:0/2091968690 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f6ef80a2f00 0x7f6ef80a53c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.273+0000 7f6f14b10640 1 --2- 172.21.15.107:0/2091968690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f10107730 0x7f6f1010d410 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.273+0000 7f6f14b10640 1 --2- 172.21.15.107:0/2091968690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f101020e0 0x7f6f1010ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.273+0000 7f6f14b10640 1 --2- 172.21.15.107:0/2091968690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f10101720 0x7f6f1010c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.273+0000 7f6f14b10640 1 -- 172.21.15.107:0/2091968690 >> 172.21.15.107:0/2091968690 conn(0x7f6f10078b40 msgr2=0x7f6f10111280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:10.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.273+0000 7f6f14b10640 1 -- 172.21.15.107:0/2091968690 shutdown_connections 2024-04-25T00:18:10.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.273+0000 7f6f14b10640 1 -- 172.21.15.107:0/2091968690 wait complete. 2024-04-25T00:18:10.290 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:18:10.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:09 smithi195 ceph-mon[32833]: pgmap v54: 129 pgs: 129 active+clean; 55 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 324 op/s 2024-04-25T00:18:10.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.399+0000 7f14beb09640 1 -- 172.21.15.107:0/105632369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14b81066b0 msgr2=0x7f14b8106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.399+0000 7f14beb09640 1 --2- 172.21.15.107:0/105632369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14b81066b0 0x7f14b8106a90 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009930 tx=0x7f14ac02a420 comp rx=0 tx=0).stop 2024-04-25T00:18:10.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.400+0000 7f14beb09640 1 -- 172.21.15.107:0/105632369 shutdown_connections 2024-04-25T00:18:10.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.400+0000 7f14beb09640 1 --2- 172.21.15.107:0/105632369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14b8100fd0 0x7f14b810d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.400+0000 7f14beb09640 1 --2- 172.21.15.107:0/105632369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14b8100610 0x7f14b8100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.400+0000 7f14beb09640 1 --2- 172.21.15.107:0/105632369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14b81066b0 0x7f14b8106a90 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.400+0000 7f14beb09640 1 -- 172.21.15.107:0/105632369 >> 172.21.15.107:0/105632369 conn(0x7f14b8077740 msgr2=0x7f14b8077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:10.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.400+0000 7f14beb09640 1 -- 172.21.15.107:0/105632369 shutdown_connections 2024-04-25T00:18:10.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.400+0000 7f14beb09640 1 -- 172.21.15.107:0/105632369 wait complete. 2024-04-25T00:18:10.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.400+0000 7f14beb09640 1 Processor -- start 2024-04-25T00:18:10.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.400+0000 7f14beb09640 1 -- start start 2024-04-25T00:18:10.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.400+0000 7f14beb09640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14b8100610 0x7f14b8196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.400+0000 7f14beb09640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14b8100fd0 0x7f14b8197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.400+0000 7f14beb09640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14b81066b0 0x7f14b819b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.400+0000 7f14beb09640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f14b8110a30 con 0x7f14b8100610 2024-04-25T00:18:10.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.400+0000 7f14beb09640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f14b81108b0 con 0x7f14b81066b0 2024-04-25T00:18:10.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.400+0000 7f14beb09640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f14b8110bb0 con 0x7f14b8100fd0 2024-04-25T00:18:10.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.400+0000 7f14bc8a5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14b8100610 0x7f14b8196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.400+0000 7f14bc8a5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14b8100610 0x7f14b8196f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33820/0 (socket says 172.21.15.107:33820) 2024-04-25T00:18:10.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.400+0000 7f14bc8a5640 1 -- 172.21.15.107:0/884809950 learned_addr learned my addr 172.21.15.107:0/884809950 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:10.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.401+0000 7f14bd0a6640 1 --2- 172.21.15.107:0/884809950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14b81066b0 0x7f14b819b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.401+0000 7f14b7fff640 1 --2- 172.21.15.107:0/884809950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14b8100fd0 0x7f14b8197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.401+0000 7f14bc8a5640 1 -- 172.21.15.107:0/884809950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14b8100fd0 msgr2=0x7f14b8197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.401+0000 7f14bc8a5640 1 --2- 172.21.15.107:0/884809950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14b8100fd0 0x7f14b8197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.401+0000 7f14bc8a5640 1 -- 172.21.15.107:0/884809950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14b81066b0 msgr2=0x7f14b819b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.401+0000 7f14bc8a5640 1 --2- 172.21.15.107:0/884809950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14b81066b0 0x7f14b819b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.401+0000 7f14bc8a5640 1 -- 172.21.15.107:0/884809950 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b819bf00 con 0x7f14b8100610 2024-04-25T00:18:10.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.401+0000 7f14bc8a5640 1 --2- 172.21.15.107:0/884809950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14b8100610 0x7f14b8196f10 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f14ac005940 tx=0x7f14ac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:10.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.401+0000 7f14b5ffb640 1 -- 172.21.15.107:0/884809950 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac03f030 con 0x7f14b8100610 2024-04-25T00:18:10.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.401+0000 7f14beb09640 1 -- 172.21.15.107:0/884809950 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b819c190 con 0x7f14b8100610 2024-04-25T00:18:10.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.401+0000 7f14b5ffb640 1 -- 172.21.15.107:0/884809950 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14ac003ce0 con 0x7f14b8100610 2024-04-25T00:18:10.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.401+0000 7f14b5ffb640 1 -- 172.21.15.107:0/884809950 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac035070 con 0x7f14b8100610 2024-04-25T00:18:10.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.401+0000 7f14beb09640 1 -- 172.21.15.107:0/884809950 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b81a3a80 con 0x7f14b8100610 2024-04-25T00:18:10.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.403+0000 7f14b5ffb640 1 -- 172.21.15.107:0/884809950 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f14ac042030 con 0x7f14b8100610 2024-04-25T00:18:10.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.404+0000 7f14c1814740 1 Processor -- start 2024-04-25T00:18:10.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.404+0000 7f14c1814740 1 -- start start 2024-04-25T00:18:10.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.404+0000 7f14b5ffb640 1 --2- 172.21.15.107:0/884809950 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f14a80a2d00 0x7f14a80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.404+0000 7f14c1814740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605093bf4a0 0x5605093bf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.404+0000 7f14c1814740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093bfdc0 0x5605093cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.404+0000 7f14c1814740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605093ccb70 0x5605093cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.404+0000 7f14c1814740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560509322670 con 0x5605093bfdc0 2024-04-25T00:18:10.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.404+0000 7f14c1814740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5605092ca970 con 0x5605093ccb70 2024-04-25T00:18:10.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.404+0000 7f14c1814740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56050931e630 con 0x5605093bf4a0 2024-04-25T00:18:10.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.404+0000 7f14bd0a6640 1 --2- 172.21.15.107:0/884809950 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f14a80a2d00 0x7f14a80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.404+0000 7f14b7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093bfdc0 0x5605093cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.404+0000 7f14b5ffb640 1 -- 172.21.15.107:0/884809950 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f14ac0efc30 con 0x7f14b8100610 2024-04-25T00:18:10.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.404+0000 7f14b7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093bfdc0 0x5605093cc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33822/0 (socket says 172.21.15.107:33822) 2024-04-25T00:18:10.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.404+0000 7f14b7fff640 1 -- 172.21.15.107:0/4176285028 learned_addr learned my addr 172.21.15.107:0/4176285028 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:10.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.404+0000 7f14bd0a6640 1 --2- 172.21.15.107:0/4176285028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605093ccb70 0x5605093cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.404+0000 7f14bc8a5640 1 --2- 172.21.15.107:0/4176285028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605093bf4a0 0x5605093bf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.405+0000 7f14b7fff640 1 -- 172.21.15.107:0/4176285028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605093bf4a0 msgr2=0x5605093bf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:10.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.405+0000 7f14b7fff640 1 --2- 172.21.15.107:0/4176285028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605093bf4a0 0x5605093bf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.405+0000 7f14b7fff640 1 -- 172.21.15.107:0/4176285028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605093ccb70 msgr2=0x5605093cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:10.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.405+0000 7f14b7fff640 1 --2- 172.21.15.107:0/4176285028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605093ccb70 0x5605093cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.405+0000 7f14b7fff640 1 -- 172.21.15.107:0/4176285028 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605092ee400 con 0x5605093bfdc0 2024-04-25T00:18:10.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.405+0000 7f14bd0a6640 1 --2- 172.21.15.107:0/884809950 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f14a80a2d00 0x7f14a80a51c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f14b0009fd0 tx=0x7f14b0009da0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:10.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.405+0000 7f14b7fff640 1 --2- 172.21.15.107:0/4176285028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093bfdc0 0x5605093cc630 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f14a400bd10 tx=0x7f14a40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c69aca44a1c5b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:10.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.405+0000 7f14a2ffd640 1 -- 172.21.15.107:0/4176285028 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a4005220 con 0x5605093bfdc0 2024-04-25T00:18:10.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.405+0000 7f14a2ffd640 1 -- 172.21.15.107:0/4176285028 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14a4004c80 con 0x5605093bfdc0 2024-04-25T00:18:10.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.405+0000 7f14a2ffd640 1 -- 172.21.15.107:0/4176285028 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a40117e0 con 0x5605093bfdc0 2024-04-25T00:18:10.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.405+0000 7f14c1814740 1 -- 172.21.15.107:0/4176285028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093bfdc0 msgr2=0x5605093cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.405+0000 7f14c1814740 1 --2- 172.21.15.107:0/4176285028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093bfdc0 0x5605093cc630 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f14a400bd10 tx=0x7f14a40107a0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.406+0000 7f14c1814740 1 -- 172.21.15.107:0/4176285028 shutdown_connections 2024-04-25T00:18:10.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.406+0000 7f14c1814740 1 --2- 172.21.15.107:0/4176285028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605093ccb70 0x5605093cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.406+0000 7f14c1814740 1 --2- 172.21.15.107:0/4176285028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093bfdc0 0x5605093cc630 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.406+0000 7f14c1814740 1 --2- 172.21.15.107:0/4176285028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605093bf4a0 0x5605093bf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.406+0000 7f14c1814740 1 -- 172.21.15.107:0/4176285028 >> 172.21.15.107:0/4176285028 conn(0x5605093bbab0 msgr2=0x5605093bbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:10.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.406+0000 7f14c1814740 1 -- 172.21.15.107:0/4176285028 shutdown_connections 2024-04-25T00:18:10.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.406+0000 7f14c1814740 1 -- 172.21.15.107:0/4176285028 wait complete. 2024-04-25T00:18:10.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.406+0000 7f14c1814740 1 -- start start 2024-04-25T00:18:10.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.406+0000 7f14c1814740 1 Processor -- start 2024-04-25T00:18:10.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.406+0000 7f14c1814740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605093bf4a0 0x5605093cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.406+0000 7f14c1814740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605093bfdc0 0x5605093c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.406+0000 7f14c1814740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093ccb70 0x5605093c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.406+0000 7f14c1814740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5605093cf7a0 con 0x5605093ccb70 2024-04-25T00:18:10.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.406+0000 7f14c1814740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5605092de050 con 0x5605093bfdc0 2024-04-25T00:18:10.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.406+0000 7f14c1814740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5605093cf920 con 0x5605093bf4a0 2024-04-25T00:18:10.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.406+0000 7f14c1814740 10 client.0 _ll_register_callbacks cb 0x560509035be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:10.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.406+0000 7f14bd0a6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093ccb70 0x5605093c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.407+0000 7f14bc8a5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605093bf4a0 0x5605093cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.407+0000 7f14bd0a6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093ccb70 0x5605093c9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33834/0 (socket says 172.21.15.107:33834) 2024-04-25T00:18:10.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.407+0000 7f14bd0a6640 1 -- 172.21.15.107:0/3969784200 learned_addr learned my addr 172.21.15.107:0/3969784200 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:10.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.407+0000 7f14b7fff640 1 --2- 172.21.15.107:0/3969784200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605093bfdc0 0x5605093c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.407+0000 7f14bc8a5640 1 -- 172.21.15.107:0/3969784200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605093bfdc0 msgr2=0x5605093c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:10.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.407+0000 7f14bc8a5640 1 --2- 172.21.15.107:0/3969784200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605093bfdc0 0x5605093c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.407+0000 7f14bc8a5640 1 -- 172.21.15.107:0/3969784200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093ccb70 msgr2=0x5605093c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:10.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.407+0000 7f14bc8a5640 1 --2- 172.21.15.107:0/3969784200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093ccb70 0x5605093c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.407+0000 7f14bc8a5640 1 -- 172.21.15.107:0/3969784200 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605092afdc0 con 0x5605093bf4a0 2024-04-25T00:18:10.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.407+0000 7f14bd0a6640 1 --2- 172.21.15.107:0/3969784200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093ccb70 0x5605093c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:10.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.407+0000 7f14bc8a5640 1 --2- 172.21.15.107:0/3969784200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605093bf4a0 0x5605093cc4f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0ed010 tx=0x7f14ac0bb8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1656c1a0da646da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:10.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.407+0000 7f1497fff640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-04-25T00:18:10.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.407+0000 7f1497fff640 1 -- 172.21.15.107:0/3969784200 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac049050 con 0x5605093bf4a0 2024-04-25T00:18:10.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.407+0000 7f1497fff640 1 -- 172.21.15.107:0/3969784200 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f14ac03b460 con 0x5605093bf4a0 2024-04-25T00:18:10.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.407+0000 7f14c1814740 1 -- 172.21.15.107:0/3969784200 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14a4004c80 con 0x5605093bf4a0 2024-04-25T00:18:10.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.407+0000 7f1497fff640 1 -- 172.21.15.107:0/3969784200 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f14ac0c0190 con 0x5605093bf4a0 2024-04-25T00:18:10.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.407+0000 7f14c1814740 10 client.25420 fetch_fsmap learned FSMap version 26 2024-04-25T00:18:10.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.407+0000 7f14c1814740 1 -- 172.21.15.107:0/3969784200 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5605090363f0 con 0x5605093bf4a0 2024-04-25T00:18:10.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.407+0000 7f1497fff640 1 -- 172.21.15.107:0/3969784200 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14a4004c80 con 0x5605093bf4a0 2024-04-25T00:18:10.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.408+0000 7f1497fff640 1 -- 172.21.15.107:0/3969784200 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f14ac0e9960 con 0x5605093bf4a0 2024-04-25T00:18:10.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.408+0000 7f14c1814740 10 client.25420 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:18:10.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.408+0000 7f14c1814740 10 client.25420 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:18:10.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.408+0000 7f14c1814740 20 client.25420 populate_metadata read hostname 'smithi107' 2024-04-25T00:18:10.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.408+0000 7f14c1814740 1 --2- 172.21.15.107:0/3969784200 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5605093dca00 0x5605093dee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.408+0000 7f14c1814740 4 client.25420 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:18:10.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.408+0000 7f14c1814740 1 -- 172.21.15.107:0/3969784200 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56050932fbb0 con 0x5605093dca00 2024-04-25T00:18:10.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.408+0000 7f14b7fff640 1 --2- 172.21.15.107:0/3969784200 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5605093dca00 0x5605093dee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.408+0000 7f14b7fff640 1 --2- 172.21.15.107:0/3969784200 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5605093dca00 0x5605093dee90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6eff6ee0cb94ea24 server_cookie=a337694f88ead417 in_seq=0 out_seq=0 2024-04-25T00:18:10.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.408+0000 7f1497fff640 10 client.25420 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:18:10.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.410+0000 7f1497fff640 1 -- 172.21.15.107:0/3969784200 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56050932fbb0 con 0x5605093dca00 2024-04-25T00:18:10.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.410+0000 7f1497fff640 10 client.25420 handle_command_reply: tid=0 2024-04-25T00:18:10.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.410+0000 7f14c1814740 1 client.25420 shutdown 2024-04-25T00:18:10.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.410+0000 7f14c1814740 1 -- 172.21.15.107:0/3969784200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605093bf4a0 msgr2=0x5605093cc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.410+0000 7f14c1814740 1 --2- 172.21.15.107:0/3969784200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605093bf4a0 0x5605093cc4f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0ed010 tx=0x7f14ac0bb8b0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.410+0000 7f14c1814740 1 -- 172.21.15.107:0/3969784200 shutdown_connections 2024-04-25T00:18:10.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.410+0000 7f14c1814740 1 --2- 172.21.15.107:0/3969784200 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5605093dca00 0x5605093dee90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.410+0000 7f14c1814740 1 --2- 172.21.15.107:0/3969784200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093ccb70 0x5605093c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.410+0000 7f1497fff640 0 client.25420 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:18:10.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.410+0000 7f14c1814740 1 --2- 172.21.15.107:0/3969784200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605093bfdc0 0x5605093c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.410+0000 7f14c1814740 1 --2- 172.21.15.107:0/3969784200 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605093bf4a0 0x5605093cc4f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.410+0000 7f14c1814740 1 -- 172.21.15.107:0/3969784200 >> 172.21.15.107:0/3969784200 conn(0x5605093bbab0 msgr2=0x5605093c1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:10.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.411+0000 7f14c1814740 1 -- 172.21.15.107:0/3969784200 shutdown_connections 2024-04-25T00:18:10.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.411+0000 7f14c1814740 1 -- 172.21.15.107:0/3969784200 wait complete. 2024-04-25T00:18:10.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.411+0000 7f14c1814740 20 client.25420 trim_cache size 0 max 16384 2024-04-25T00:18:10.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.424+0000 7f14c1814740 1 Processor -- start 2024-04-25T00:18:10.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.424+0000 7f14c1814740 1 -- start start 2024-04-25T00:18:10.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.425+0000 7f14c1814740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093bf4a0 0x5605093bb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.425+0000 7f14c1814740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605093bfdc0 0x5605093d86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.425+0000 7f14c1814740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605093ccb70 0x5605093c8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.425+0000 7f14c1814740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f14b81af140 con 0x5605093bf4a0 2024-04-25T00:18:10.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.425+0000 7f14c1814740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f14b81ac9b0 con 0x5605093bfdc0 2024-04-25T00:18:10.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.425+0000 7f14c1814740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f14b81aa800 con 0x5605093ccb70 2024-04-25T00:18:10.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.425+0000 7f14bc8a5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093bf4a0 0x5605093bb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.425+0000 7f14bc8a5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093bf4a0 0x5605093bb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33844/0 (socket says 172.21.15.107:33844) 2024-04-25T00:18:10.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.425+0000 7f14bc8a5640 1 -- 172.21.15.107:0/1346703317 learned_addr learned my addr 172.21.15.107:0/1346703317 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:10.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.425+0000 7f14b7fff640 1 --2- 172.21.15.107:0/1346703317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605093bfdc0 0x5605093d86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.425+0000 7f14bd0a6640 1 --2- 172.21.15.107:0/1346703317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605093ccb70 0x5605093c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.425+0000 7f14bc8a5640 1 -- 172.21.15.107:0/1346703317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605093ccb70 msgr2=0x5605093c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:10.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.425+0000 7f14bc8a5640 1 --2- 172.21.15.107:0/1346703317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605093ccb70 0x5605093c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.425+0000 7f14bc8a5640 1 -- 172.21.15.107:0/1346703317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605093bfdc0 msgr2=0x5605093d86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:10.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.425+0000 7f14bc8a5640 1 --2- 172.21.15.107:0/1346703317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605093bfdc0 0x5605093d86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.425+0000 7f14bc8a5640 1 -- 172.21.15.107:0/1346703317 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605093b8990 con 0x5605093bf4a0 2024-04-25T00:18:10.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.425+0000 7f14bc8a5640 1 --2- 172.21.15.107:0/1346703317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093bf4a0 0x5605093bb550 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0bbb90 tx=0x7f14ac0bc010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599f58707252848c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:10.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.425+0000 7f14a27fc640 1 -- 172.21.15.107:0/1346703317 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac049070 con 0x5605093bf4a0 2024-04-25T00:18:10.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.425+0000 7f14a27fc640 1 -- 172.21.15.107:0/1346703317 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14ac0ed470 con 0x5605093bf4a0 2024-04-25T00:18:10.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.425+0000 7f14a27fc640 1 -- 172.21.15.107:0/1346703317 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac0c45c0 con 0x5605093bf4a0 2024-04-25T00:18:10.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.426+0000 7f14c1814740 1 -- 172.21.15.107:0/1346703317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093bf4a0 msgr2=0x5605093bb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.426+0000 7f14c1814740 1 --2- 172.21.15.107:0/1346703317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093bf4a0 0x5605093bb550 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0bbb90 tx=0x7f14ac0bc010 comp rx=0 tx=0).stop 2024-04-25T00:18:10.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.426+0000 7f14c1814740 1 -- 172.21.15.107:0/1346703317 shutdown_connections 2024-04-25T00:18:10.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.426+0000 7f14c1814740 1 --2- 172.21.15.107:0/1346703317 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605093ccb70 0x5605093c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.426+0000 7f14c1814740 1 --2- 172.21.15.107:0/1346703317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605093bfdc0 0x5605093d86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.426+0000 7f14c1814740 1 --2- 172.21.15.107:0/1346703317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093bf4a0 0x5605093bb550 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.426+0000 7f14c1814740 1 -- 172.21.15.107:0/1346703317 >> 172.21.15.107:0/1346703317 conn(0x5605093bbab0 msgr2=0x5605093bbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:10.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.426+0000 7f14c1814740 1 -- 172.21.15.107:0/1346703317 shutdown_connections 2024-04-25T00:18:10.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.426+0000 7f14c1814740 1 -- 172.21.15.107:0/1346703317 wait complete. 2024-04-25T00:18:10.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.426+0000 7f14c1814740 1 -- start start 2024-04-25T00:18:10.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.426+0000 7f14c1814740 1 Processor -- start 2024-04-25T00:18:10.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f14c1814740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605093bf4a0 0x5605093e4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f14c1814740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093bfdc0 0x5605093e4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f14c1814740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605093ccb70 0x560509442010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f14c1814740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f14b81aae70 con 0x5605093bfdc0 2024-04-25T00:18:10.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f14c1814740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f14b81ac2e0 con 0x5605093ccb70 2024-04-25T00:18:10.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f14c1814740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f14b81ab540 con 0x5605093bf4a0 2024-04-25T00:18:10.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f14c1814740 10 client.0 _ll_register_callbacks cb 0x7f14ac009290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:10.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f14b7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093bfdc0 0x5605093e4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f14b7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093bfdc0 0x5605093e4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33856/0 (socket says 172.21.15.107:33856) 2024-04-25T00:18:10.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f14b7fff640 1 -- 172.21.15.107:0/2596894073 learned_addr learned my addr 172.21.15.107:0/2596894073 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:10.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f14bd0a6640 1 --2- 172.21.15.107:0/2596894073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605093ccb70 0x560509442010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f14bc8a5640 1 --2- 172.21.15.107:0/2596894073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605093bf4a0 0x5605093e4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f14b7fff640 1 -- 172.21.15.107:0/2596894073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605093bf4a0 msgr2=0x5605093e4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:10.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f14b7fff640 1 --2- 172.21.15.107:0/2596894073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605093bf4a0 0x5605093e4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f14b7fff640 1 -- 172.21.15.107:0/2596894073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605093ccb70 msgr2=0x560509442010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:10.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f14b7fff640 1 --2- 172.21.15.107:0/2596894073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605093ccb70 0x560509442010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f14b7fff640 1 -- 172.21.15.107:0/2596894073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605093bfbf0 con 0x5605093bfdc0 2024-04-25T00:18:10.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f14b7fff640 1 --2- 172.21.15.107:0/2596894073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093bfdc0 0x5605093e4890 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f14a400bd10 tx=0x7f14a4005f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8eca72f2e00f3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:10.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f1497fff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:18:10.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f1497fff640 1 -- 172.21.15.107:0/2596894073 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a402c070 con 0x5605093bfdc0 2024-04-25T00:18:10.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f1497fff640 1 -- 172.21.15.107:0/2596894073 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f14a4028840 con 0x5605093bfdc0 2024-04-25T00:18:10.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f14c1814740 1 -- 172.21.15.107:0/2596894073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14ac0ed470 con 0x5605093bfdc0 2024-04-25T00:18:10.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f1497fff640 1 -- 172.21.15.107:0/2596894073 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f14a4035050 con 0x5605093bfdc0 2024-04-25T00:18:10.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f14c1814740 10 client.15990 fetch_fsmap learned FSMap version 26 2024-04-25T00:18:10.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.427+0000 7f14c1814740 1 -- 172.21.15.107:0/2596894073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5605091007d0 con 0x5605093bfdc0 2024-04-25T00:18:10.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.428+0000 7f1497fff640 1 -- 172.21.15.107:0/2596894073 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14ac0ed470 con 0x5605093bfdc0 2024-04-25T00:18:10.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.428+0000 7f1497fff640 1 -- 172.21.15.107:0/2596894073 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f14a402f020 con 0x5605093bfdc0 2024-04-25T00:18:10.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.428+0000 7f14c1814740 10 client.15990 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:18:10.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.428+0000 7f14c1814740 10 client.15990 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:18:10.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.428+0000 7f14c1814740 20 client.15990 populate_metadata read hostname 'smithi107' 2024-04-25T00:18:10.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.428+0000 7f14c1814740 1 --2- 172.21.15.107:0/2596894073 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5605093dca00 0x560509445a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.428+0000 7f14c1814740 4 client.15990 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:18:10.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.428+0000 7f14c1814740 1 -- 172.21.15.107:0/2596894073 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560509329f00 con 0x5605093dca00 2024-04-25T00:18:10.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.428+0000 7f14bc8a5640 1 --2- 172.21.15.107:0/2596894073 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5605093dca00 0x560509445a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.428+0000 7f1497fff640 10 client.15990 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:18:10.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.428+0000 7f14bc8a5640 1 --2- 172.21.15.107:0/2596894073 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5605093dca00 0x560509445a50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e21d426dd576f639 server_cookie=6ac0a66370d6b902 in_seq=0 out_seq=0 2024-04-25T00:18:10.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:09 smithi170 ceph-mon[32807]: pgmap v54: 129 pgs: 129 active+clean; 55 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.5 MiB/s wr, 324 op/s 2024-04-25T00:18:10.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.430+0000 7f1497fff640 1 -- 172.21.15.107:0/2596894073 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560509329f00 con 0x5605093dca00 2024-04-25T00:18:10.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.430+0000 7f1497fff640 10 client.15990 handle_command_reply: tid=0 2024-04-25T00:18:10.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.430+0000 7f14c1814740 1 client.15990 shutdown 2024-04-25T00:18:10.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.431+0000 7f14c1814740 1 -- 172.21.15.107:0/2596894073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093bfdc0 msgr2=0x5605093e4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.431+0000 7f14c1814740 1 --2- 172.21.15.107:0/2596894073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093bfdc0 0x5605093e4890 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f14a400bd10 tx=0x7f14a4005f40 comp rx=0 tx=0).stop 2024-04-25T00:18:10.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.431+0000 7f14c1814740 1 -- 172.21.15.107:0/2596894073 shutdown_connections 2024-04-25T00:18:10.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.431+0000 7f14c1814740 1 --2- 172.21.15.107:0/2596894073 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5605093dca00 0x560509445a50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.431+0000 7f14c1814740 1 --2- 172.21.15.107:0/2596894073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5605093ccb70 0x560509442010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.431+0000 7f1497fff640 0 client.15990 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:18:10.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.431+0000 7f14c1814740 1 --2- 172.21.15.107:0/2596894073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5605093bfdc0 0x5605093e4890 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.431+0000 7f14c1814740 1 --2- 172.21.15.107:0/2596894073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5605093bf4a0 0x5605093e4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.431+0000 7f14c1814740 1 -- 172.21.15.107:0/2596894073 >> 172.21.15.107:0/2596894073 conn(0x5605093bbab0 msgr2=0x5605093c8940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:10.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.431+0000 7f14c1814740 1 -- 172.21.15.107:0/2596894073 shutdown_connections 2024-04-25T00:18:10.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.431+0000 7f14c1814740 1 -- 172.21.15.107:0/2596894073 wait complete. 2024-04-25T00:18:10.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.431+0000 7f14c1814740 20 client.15990 trim_cache size 0 max 16384 2024-04-25T00:18:10.437 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:18:10.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.431+0000 7f14beb09640 1 -- 172.21.15.107:0/884809950 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f14a80a2d00 msgr2=0x7f14a80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.431+0000 7f14beb09640 1 --2- 172.21.15.107:0/884809950 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f14a80a2d00 0x7f14a80a51c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f14b0009fd0 tx=0x7f14b0009da0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.431+0000 7f14beb09640 1 -- 172.21.15.107:0/884809950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14b8100610 msgr2=0x7f14b8196f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.431+0000 7f14beb09640 1 --2- 172.21.15.107:0/884809950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14b8100610 0x7f14b8196f10 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f14ac005940 tx=0x7f14ac0042f0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.431+0000 7f14beb09640 1 -- 172.21.15.107:0/884809950 shutdown_connections 2024-04-25T00:18:10.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.431+0000 7f14beb09640 1 --2- 172.21.15.107:0/884809950 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f14a80a2d00 0x7f14a80a51c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.431+0000 7f14beb09640 1 --2- 172.21.15.107:0/884809950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14b81066b0 0x7f14b819b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.431+0000 7f14beb09640 1 --2- 172.21.15.107:0/884809950 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14b8100fd0 0x7f14b8197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.431+0000 7f14beb09640 1 --2- 172.21.15.107:0/884809950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14b8100610 0x7f14b8196f10 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.431+0000 7f14beb09640 1 -- 172.21.15.107:0/884809950 >> 172.21.15.107:0/884809950 conn(0x7f14b8077740 msgr2=0x7f14b80ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:10.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.432+0000 7f14beb09640 1 -- 172.21.15.107:0/884809950 shutdown_connections 2024-04-25T00:18:10.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.432+0000 7f14beb09640 1 -- 172.21.15.107:0/884809950 wait complete. 2024-04-25T00:18:10.442 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:18:10.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.545+0000 7fa96db09640 1 -- 172.21.15.107:0/4154853685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa968107720 msgr2=0x7fa968107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.545+0000 7fa96db09640 1 --2- 172.21.15.107:0/4154853685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa968107720 0x7fa968107b00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa958009cd0 tx=0x7fa95802a8e0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.545+0000 7fa96db09640 1 -- 172.21.15.107:0/4154853685 shutdown_connections 2024-04-25T00:18:10.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.545+0000 7fa96db09640 1 --2- 172.21.15.107:0/4154853685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa968100600 0x7fa96810e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.545+0000 7fa96db09640 1 --2- 172.21.15.107:0/4154853685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9680ffc60 0x7fa9681000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.545+0000 7fa96db09640 1 --2- 172.21.15.107:0/4154853685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa968107720 0x7fa968107b00 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.545+0000 7fa96db09640 1 -- 172.21.15.107:0/4154853685 >> 172.21.15.107:0/4154853685 conn(0x7fa968076bd0 msgr2=0x7fa9680fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:10.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.545+0000 7fa96db09640 1 -- 172.21.15.107:0/4154853685 shutdown_connections 2024-04-25T00:18:10.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.545+0000 7fa96db09640 1 -- 172.21.15.107:0/4154853685 wait complete. 2024-04-25T00:18:10.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.545+0000 7fa96db09640 1 Processor -- start 2024-04-25T00:18:10.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.546+0000 7fa96db09640 1 -- start start 2024-04-25T00:18:10.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.546+0000 7fa96db09640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9680ffc60 0x7fa9681980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.546+0000 7fa96db09640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa968100600 0x7fa9681985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.546+0000 7fa96db09640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa968107720 0x7fa96819c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.546+0000 7fa96db09640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa968111b60 con 0x7fa9680ffc60 2024-04-25T00:18:10.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.546+0000 7fa96db09640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681119e0 con 0x7fa968107720 2024-04-25T00:18:10.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.546+0000 7fa96db09640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa968111ce0 con 0x7fa968100600 2024-04-25T00:18:10.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.546+0000 7fa9677fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9680ffc60 0x7fa9681980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.546+0000 7fa9677fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9680ffc60 0x7fa9681980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33878/0 (socket says 172.21.15.107:33878) 2024-04-25T00:18:10.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.546+0000 7fa9677fe640 1 -- 172.21.15.107:0/2133667298 learned_addr learned my addr 172.21.15.107:0/2133667298 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:10.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.546+0000 7fa967fff640 1 --2- 172.21.15.107:0/2133667298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa968107720 0x7fa96819c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.546+0000 7fa966ffd640 1 --2- 172.21.15.107:0/2133667298 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa968100600 0x7fa9681985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.546+0000 7fa9677fe640 1 -- 172.21.15.107:0/2133667298 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa968100600 msgr2=0x7fa9681985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.546+0000 7fa9677fe640 1 --2- 172.21.15.107:0/2133667298 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa968100600 0x7fa9681985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.546+0000 7fa9677fe640 1 -- 172.21.15.107:0/2133667298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa968107720 msgr2=0x7fa96819c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.546+0000 7fa9677fe640 1 --2- 172.21.15.107:0/2133667298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa968107720 0x7fa96819c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.546+0000 7fa9677fe640 1 -- 172.21.15.107:0/2133667298 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96819d040 con 0x7fa9680ffc60 2024-04-25T00:18:10.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.546+0000 7fa9677fe640 1 --2- 172.21.15.107:0/2133667298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9680ffc60 0x7fa9681980a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa95802ae00 tx=0x7fa958031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:10.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.547+0000 7fa964ff9640 1 -- 172.21.15.107:0/2133667298 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958003e00 con 0x7fa9680ffc60 2024-04-25T00:18:10.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.547+0000 7fa964ff9640 1 -- 172.21.15.107:0/2133667298 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa958004520 con 0x7fa9680ffc60 2024-04-25T00:18:10.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.547+0000 7fa96db09640 1 -- 172.21.15.107:0/2133667298 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96819d2d0 con 0x7fa9680ffc60 2024-04-25T00:18:10.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.547+0000 7fa964ff9640 1 -- 172.21.15.107:0/2133667298 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958030de0 con 0x7fa9680ffc60 2024-04-25T00:18:10.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.547+0000 7fa96db09640 1 -- 172.21.15.107:0/2133667298 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9681a4bc0 con 0x7fa9680ffc60 2024-04-25T00:18:10.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.549+0000 7fa964ff9640 1 -- 172.21.15.107:0/2133667298 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa9580046d0 con 0x7fa9680ffc60 2024-04-25T00:18:10.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.549+0000 7fa970814740 1 Processor -- start 2024-04-25T00:18:10.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.549+0000 7fa970814740 1 -- start start 2024-04-25T00:18:10.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.549+0000 7fa970814740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfbfc8580 0x55bcfbfc8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.549+0000 7fa970814740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcfbfc8ea0 0x55bcfbfd3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.549+0000 7fa970814740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfd3570 0x55bcfbfd5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.549+0000 7fa970814740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bcfbf2d670 con 0x55bcfbfd3570 2024-04-25T00:18:10.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.549+0000 7fa970814740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bcfbed5970 con 0x55bcfbfc8ea0 2024-04-25T00:18:10.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.549+0000 7fa970814740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bcfbf29630 con 0x55bcfbfc8580 2024-04-25T00:18:10.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.549+0000 7fa966ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfd3570 0x55bcfbfd5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.549+0000 7fa966ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfd3570 0x55bcfbfd5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33894/0 (socket says 172.21.15.107:33894) 2024-04-25T00:18:10.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.549+0000 7fa966ffd640 1 -- 172.21.15.107:0/1779251528 learned_addr learned my addr 172.21.15.107:0/1779251528 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:10.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.549+0000 7fa9677fe640 1 --2- 172.21.15.107:0/1779251528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcfbfc8ea0 0x55bcfbfd3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.550+0000 7fa964ff9640 1 --2- 172.21.15.107:0/2133667298 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa9500a2d00 0x7fa9500a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.550+0000 7fa967fff640 1 --2- 172.21.15.107:0/1779251528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfbfc8580 0x55bcfbfc8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.550+0000 7fa966ffd640 1 -- 172.21.15.107:0/1779251528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfbfc8580 msgr2=0x55bcfbfc8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:10.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.550+0000 7fa966ffd640 1 --2- 172.21.15.107:0/1779251528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfbfc8580 0x55bcfbfc8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.550+0000 7fa966ffd640 1 -- 172.21.15.107:0/1779251528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcfbfc8ea0 msgr2=0x55bcfbfd3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:10.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.550+0000 7fa966ffd640 1 --2- 172.21.15.107:0/1779251528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcfbfc8ea0 0x55bcfbfd3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.550+0000 7fa966ffd640 1 -- 172.21.15.107:0/1779251528 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcfbef9400 con 0x55bcfbfd3570 2024-04-25T00:18:10.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.550+0000 7fa967fff640 1 --2- 172.21.15.107:0/1779251528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfbfc8580 0x55bcfbfc8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:18:10.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.550+0000 7fa967fff640 1 --2- 172.21.15.107:0/2133667298 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa9500a2d00 0x7fa9500a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.550+0000 7fa964ff9640 1 -- 172.21.15.107:0/2133667298 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fa95802e030 con 0x7fa9680ffc60 2024-04-25T00:18:10.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.550+0000 7fa966ffd640 1 --2- 172.21.15.107:0/1779251528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfd3570 0x55bcfbfd5960 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009c30 tx=0x7fa95c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6747446252999c82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:10.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.550+0000 7fa94dffb640 1 -- 172.21.15.107:0/1779251528 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c012450 con 0x55bcfbfd3570 2024-04-25T00:18:10.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.550+0000 7fa94dffb640 1 -- 172.21.15.107:0/1779251528 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa95c012ad0 con 0x55bcfbfd3570 2024-04-25T00:18:10.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.550+0000 7fa94dffb640 1 -- 172.21.15.107:0/1779251528 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c00fbf0 con 0x55bcfbfd3570 2024-04-25T00:18:10.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.551+0000 7fa970814740 1 -- 172.21.15.107:0/1779251528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfd3570 msgr2=0x55bcfbfd5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.551+0000 7fa967fff640 1 --2- 172.21.15.107:0/2133667298 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa9500a2d00 0x7fa9500a51c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa960009ca0 tx=0x7fa9600046c0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:10.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.551+0000 7fa970814740 1 --2- 172.21.15.107:0/1779251528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfd3570 0x55bcfbfd5960 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009c30 tx=0x7fa95c0117b0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.551+0000 7fa970814740 1 -- 172.21.15.107:0/1779251528 shutdown_connections 2024-04-25T00:18:10.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.551+0000 7fa970814740 1 --2- 172.21.15.107:0/1779251528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfd3570 0x55bcfbfd5960 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.551+0000 7fa970814740 1 --2- 172.21.15.107:0/1779251528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcfbfc8ea0 0x55bcfbfd3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.551+0000 7fa970814740 1 --2- 172.21.15.107:0/1779251528 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfbfc8580 0x55bcfbfc8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.551+0000 7fa970814740 1 -- 172.21.15.107:0/1779251528 >> 172.21.15.107:0/1779251528 conn(0x55bcfbfc48a0 msgr2=0x55bcfbfc4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:10.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.551+0000 7fa970814740 1 -- 172.21.15.107:0/1779251528 shutdown_connections 2024-04-25T00:18:10.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.551+0000 7fa970814740 1 -- 172.21.15.107:0/1779251528 wait complete. 2024-04-25T00:18:10.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.551+0000 7fa970814740 1 -- start start 2024-04-25T00:18:10.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.551+0000 7fa970814740 1 Processor -- start 2024-04-25T00:18:10.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.552+0000 7fa970814740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfbfc8580 0x55bcfbfd2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.552+0000 7fa970814740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcfbfc8ea0 0x55bcfbfd0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.552+0000 7fa970814740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfd3570 0x55bcfbfd06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.552+0000 7fa970814740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bcfbfd6360 con 0x55bcfbfd3570 2024-04-25T00:18:10.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.552+0000 7fa970814740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bcfbee9050 con 0x55bcfbfc8ea0 2024-04-25T00:18:10.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.552+0000 7fa970814740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bcfbfd64e0 con 0x55bcfbfc8580 2024-04-25T00:18:10.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.552+0000 7fa970814740 10 client.0 _ll_register_callbacks cb 0x55bcfbc40be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:10.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.552+0000 7fa967fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfd3570 0x55bcfbfd06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.552+0000 7fa966ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcfbfc8ea0 0x55bcfbfd0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.552+0000 7fa967fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfd3570 0x55bcfbfd06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33906/0 (socket says 172.21.15.107:33906) 2024-04-25T00:18:10.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.552+0000 7fa967fff640 1 -- 172.21.15.107:0/1650956738 learned_addr learned my addr 172.21.15.107:0/1650956738 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:10.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.552+0000 7fa9677fe640 1 --2- 172.21.15.107:0/1650956738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfbfc8580 0x55bcfbfd2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.552+0000 7fa966ffd640 1 -- 172.21.15.107:0/1650956738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfbfc8580 msgr2=0x55bcfbfd2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:10.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.552+0000 7fa966ffd640 1 --2- 172.21.15.107:0/1650956738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfbfc8580 0x55bcfbfd2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.552+0000 7fa966ffd640 1 -- 172.21.15.107:0/1650956738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfd3570 msgr2=0x55bcfbfd06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:10.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.552+0000 7fa966ffd640 1 --2- 172.21.15.107:0/1650956738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfd3570 0x55bcfbfd06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.552+0000 7fa966ffd640 1 -- 172.21.15.107:0/1650956738 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcfbebadc0 con 0x55bcfbfc8ea0 2024-04-25T00:18:10.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.552+0000 7fa967fff640 1 --2- 172.21.15.107:0/1650956738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfd3570 0x55bcfbfd06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:10.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.552+0000 7fa966ffd640 1 --2- 172.21.15.107:0/1650956738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcfbfc8ea0 0x55bcfbfd0160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa95c01c040 tx=0x7fa95c012f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ee32253883b04bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:10.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.552+0000 7fa946ffd640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-04-25T00:18:10.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.552+0000 7fa946ffd640 1 -- 172.21.15.107:0/1650956738 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa95c028070 con 0x55bcfbfc8ea0 2024-04-25T00:18:10.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.552+0000 7fa946ffd640 1 -- 172.21.15.107:0/1650956738 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fa95c012860 con 0x55bcfbfc8ea0 2024-04-25T00:18:10.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.552+0000 7fa970814740 1 -- 172.21.15.107:0/1650956738 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa95c012ad0 con 0x55bcfbfc8ea0 2024-04-25T00:18:10.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.553+0000 7fa946ffd640 1 -- 172.21.15.107:0/1650956738 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa95c033050 con 0x55bcfbfc8ea0 2024-04-25T00:18:10.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.553+0000 7fa970814740 10 client.25478 fetch_fsmap learned FSMap version 26 2024-04-25T00:18:10.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.553+0000 7fa970814740 1 -- 172.21.15.107:0/1650956738 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bcfbc413f0 con 0x55bcfbfc8ea0 2024-04-25T00:18:10.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.553+0000 7fa946ffd640 1 -- 172.21.15.107:0/1650956738 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa95c012ad0 con 0x55bcfbfc8ea0 2024-04-25T00:18:10.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.553+0000 7fa946ffd640 1 -- 172.21.15.107:0/1650956738 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fa95c005160 con 0x55bcfbfc8ea0 2024-04-25T00:18:10.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.553+0000 7fa970814740 10 client.25478 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:18:10.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.553+0000 7fa970814740 10 client.25478 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:18:10.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.553+0000 7fa970814740 20 client.25478 populate_metadata read hostname 'smithi107' 2024-04-25T00:18:10.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.553+0000 7fa970814740 1 --2- 172.21.15.107:0/1650956738 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55bcfbfe3500 0x55bcfbfe58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.553+0000 7fa970814740 4 client.25478 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:18:10.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.553+0000 7fa970814740 1 -- 172.21.15.107:0/1650956738 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcfbf3abb0 con 0x55bcfbfe3500 2024-04-25T00:18:10.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.553+0000 7fa9677fe640 1 --2- 172.21.15.107:0/1650956738 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55bcfbfe3500 0x55bcfbfe58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.554+0000 7fa9677fe640 1 --2- 172.21.15.107:0/1650956738 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55bcfbfe3500 0x55bcfbfe58f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d4ce73bba8e9d179 server_cookie=1acab9d57bb35c0 in_seq=0 out_seq=0 2024-04-25T00:18:10.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.554+0000 7fa946ffd640 10 client.25478 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:18:10.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.554+0000 7fa946ffd640 1 -- 172.21.15.107:0/1650956738 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bcfbf3abb0 con 0x55bcfbfe3500 2024-04-25T00:18:10.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.554+0000 7fa946ffd640 10 client.25478 handle_command_reply: tid=0 2024-04-25T00:18:10.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.554+0000 7fa970814740 1 client.25478 shutdown 2024-04-25T00:18:10.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.554+0000 7fa970814740 1 -- 172.21.15.107:0/1650956738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcfbfc8ea0 msgr2=0x55bcfbfd0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.554+0000 7fa970814740 1 --2- 172.21.15.107:0/1650956738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcfbfc8ea0 0x55bcfbfd0160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa95c01c040 tx=0x7fa95c012f30 comp rx=0 tx=0).stop 2024-04-25T00:18:10.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.554+0000 7fa970814740 1 -- 172.21.15.107:0/1650956738 shutdown_connections 2024-04-25T00:18:10.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.554+0000 7fa970814740 1 --2- 172.21.15.107:0/1650956738 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55bcfbfe3500 0x55bcfbfe58f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.554+0000 7fa970814740 1 --2- 172.21.15.107:0/1650956738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfd3570 0x55bcfbfd06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.554+0000 7fa970814740 1 --2- 172.21.15.107:0/1650956738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcfbfc8ea0 0x55bcfbfd0160 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.554+0000 7fa970814740 1 --2- 172.21.15.107:0/1650956738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfbfc8580 0x55bcfbfd2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.554+0000 7fa946ffd640 0 client.25478 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:18:10.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.554+0000 7fa970814740 1 -- 172.21.15.107:0/1650956738 >> 172.21.15.107:0/1650956738 conn(0x55bcfbfc48a0 msgr2=0x55bcfbfca6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:10.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.554+0000 7fa970814740 1 -- 172.21.15.107:0/1650956738 shutdown_connections 2024-04-25T00:18:10.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.554+0000 7fa970814740 1 -- 172.21.15.107:0/1650956738 wait complete. 2024-04-25T00:18:10.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.555+0000 7fa970814740 20 client.25478 trim_cache size 0 max 16384 2024-04-25T00:18:10.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.568+0000 7fa970814740 1 Processor -- start 2024-04-25T00:18:10.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.568+0000 7fa970814740 1 -- start start 2024-04-25T00:18:10.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.568+0000 7fa970814740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfc8580 0x55bcfbfdf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.568+0000 7fa970814740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcfbfc8ea0 0x55bcfbfcb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.568+0000 7fa970814740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfbfd3570 0x55bcfbfd29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.568+0000 7fa970814740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681b1370 con 0x55bcfbfc8580 2024-04-25T00:18:10.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.568+0000 7fa970814740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681aebe0 con 0x55bcfbfc8ea0 2024-04-25T00:18:10.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.568+0000 7fa970814740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681aca30 con 0x55bcfbfd3570 2024-04-25T00:18:10.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.568+0000 7fa9677fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfc8580 0x55bcfbfdf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.568+0000 7fa9677fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfc8580 0x55bcfbfdf220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33914/0 (socket says 172.21.15.107:33914) 2024-04-25T00:18:10.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.568+0000 7fa9677fe640 1 -- 172.21.15.107:0/934886426 learned_addr learned my addr 172.21.15.107:0/934886426 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:10.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.568+0000 7fa966ffd640 1 --2- 172.21.15.107:0/934886426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcfbfc8ea0 0x55bcfbfcb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.568+0000 7fa967fff640 1 --2- 172.21.15.107:0/934886426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfbfd3570 0x55bcfbfd29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.568+0000 7fa9677fe640 1 -- 172.21.15.107:0/934886426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfbfd3570 msgr2=0x55bcfbfd29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:10.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.568+0000 7fa9677fe640 1 --2- 172.21.15.107:0/934886426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfbfd3570 0x55bcfbfd29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.568+0000 7fa9677fe640 1 -- 172.21.15.107:0/934886426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcfbfc8ea0 msgr2=0x55bcfbfcb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:10.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.568+0000 7fa9677fe640 1 --2- 172.21.15.107:0/934886426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcfbfc8ea0 0x55bcfbfcb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.568+0000 7fa9677fe640 1 -- 172.21.15.107:0/934886426 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcfbd0b7d0 con 0x55bcfbfc8580 2024-04-25T00:18:10.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.568+0000 7fa967fff640 1 --2- 172.21.15.107:0/934886426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfbfd3570 0x55bcfbfd29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:18:10.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.568+0000 7fa9677fe640 1 --2- 172.21.15.107:0/934886426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfc8580 0x55bcfbfdf220 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa958005e30 tx=0x7fa9580f5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f7ea1bb8781055 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:10.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.569+0000 7fa94d7fa640 1 -- 172.21.15.107:0/934886426 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9580bc740 con 0x55bcfbfc8580 2024-04-25T00:18:10.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.569+0000 7fa94d7fa640 1 -- 172.21.15.107:0/934886426 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9580c0040 con 0x55bcfbfc8580 2024-04-25T00:18:10.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.569+0000 7fa94d7fa640 1 -- 172.21.15.107:0/934886426 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa958004040 con 0x55bcfbfc8580 2024-04-25T00:18:10.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.569+0000 7fa970814740 1 -- 172.21.15.107:0/934886426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfc8580 msgr2=0x55bcfbfdf220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.569+0000 7fa970814740 1 --2- 172.21.15.107:0/934886426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfc8580 0x55bcfbfdf220 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa958005e30 tx=0x7fa9580f5040 comp rx=0 tx=0).stop 2024-04-25T00:18:10.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.569+0000 7fa970814740 1 -- 172.21.15.107:0/934886426 shutdown_connections 2024-04-25T00:18:10.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.569+0000 7fa970814740 1 --2- 172.21.15.107:0/934886426 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfbfd3570 0x55bcfbfd29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.569+0000 7fa970814740 1 --2- 172.21.15.107:0/934886426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcfbfc8ea0 0x55bcfbfcb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.569+0000 7fa970814740 1 --2- 172.21.15.107:0/934886426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfc8580 0x55bcfbfdf220 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.569+0000 7fa970814740 1 -- 172.21.15.107:0/934886426 >> 172.21.15.107:0/934886426 conn(0x55bcfbfc48a0 msgr2=0x55bcfbfcaeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:10.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.569+0000 7fa970814740 1 -- 172.21.15.107:0/934886426 shutdown_connections 2024-04-25T00:18:10.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.569+0000 7fa970814740 1 -- 172.21.15.107:0/934886426 wait complete. 2024-04-25T00:18:10.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.570+0000 7fa970814740 1 -- start start 2024-04-25T00:18:10.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.570+0000 7fa970814740 1 Processor -- start 2024-04-25T00:18:10.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.570+0000 7fa970814740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfbfc8580 0x55bcfbfdb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.570+0000 7fa970814740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcfbfc8ea0 0x55bcfbfdbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.570+0000 7fa970814740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfd3570 0x55bcfbfd6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.570+0000 7fa970814740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681ad0a0 con 0x55bcfbfd3570 2024-04-25T00:18:10.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.570+0000 7fa970814740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681ae510 con 0x55bcfbfc8ea0 2024-04-25T00:18:10.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.570+0000 7fa970814740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa9681ad770 con 0x55bcfbfc8580 2024-04-25T00:18:10.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.570+0000 7fa970814740 10 client.0 _ll_register_callbacks cb 0x7fa9580b9100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:10.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.570+0000 7fa967fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfd3570 0x55bcfbfd6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.570+0000 7fa966ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcfbfc8ea0 0x55bcfbfdbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.570+0000 7fa967fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfd3570 0x55bcfbfd6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33924/0 (socket says 172.21.15.107:33924) 2024-04-25T00:18:10.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.570+0000 7fa967fff640 1 -- 172.21.15.107:0/3387526321 learned_addr learned my addr 172.21.15.107:0/3387526321 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:10.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.570+0000 7fa9677fe640 1 --2- 172.21.15.107:0/3387526321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfbfc8580 0x55bcfbfdb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.570+0000 7fa967fff640 1 -- 172.21.15.107:0/3387526321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfbfc8580 msgr2=0x55bcfbfdb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:10.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.571+0000 7fa967fff640 1 --2- 172.21.15.107:0/3387526321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfbfc8580 0x55bcfbfdb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.571+0000 7fa967fff640 1 -- 172.21.15.107:0/3387526321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcfbfc8ea0 msgr2=0x55bcfbfdbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:10.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.571+0000 7fa967fff640 1 --2- 172.21.15.107:0/3387526321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcfbfc8ea0 0x55bcfbfdbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.571+0000 7fa967fff640 1 -- 172.21.15.107:0/3387526321 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcfbfd76b0 con 0x55bcfbfd3570 2024-04-25T00:18:10.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.571+0000 7fa967fff640 1 --2- 172.21.15.107:0/3387526321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfd3570 0x55bcfbfd6eb0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa96001ebf0 tx=0x7fa960004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b93d9b50157e2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:10.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.571+0000 7fa946ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:18:10.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.571+0000 7fa946ffd640 1 -- 172.21.15.107:0/3387526321 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96002e060 con 0x55bcfbfd3570 2024-04-25T00:18:10.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.571+0000 7fa946ffd640 1 -- 172.21.15.107:0/3387526321 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fa96001e0d0 con 0x55bcfbfd3570 2024-04-25T00:18:10.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.571+0000 7fa970814740 1 -- 172.21.15.107:0/3387526321 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9580c0040 con 0x55bcfbfd3570 2024-04-25T00:18:10.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.571+0000 7fa946ffd640 1 -- 172.21.15.107:0/3387526321 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa960035050 con 0x55bcfbfd3570 2024-04-25T00:18:10.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.571+0000 7fa970814740 10 client.16008 fetch_fsmap learned FSMap version 26 2024-04-25T00:18:10.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.571+0000 7fa970814740 1 -- 172.21.15.107:0/3387526321 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bcfc049500 con 0x55bcfbfd3570 2024-04-25T00:18:10.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.571+0000 7fa946ffd640 1 -- 172.21.15.107:0/3387526321 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9580c0040 con 0x55bcfbfd3570 2024-04-25T00:18:10.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.571+0000 7fa946ffd640 1 -- 172.21.15.107:0/3387526321 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fa96001a070 con 0x55bcfbfd3570 2024-04-25T00:18:10.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.571+0000 7fa970814740 10 client.16008 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:18:10.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.571+0000 7fa970814740 10 client.16008 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:18:10.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.571+0000 7fa970814740 20 client.16008 populate_metadata read hostname 'smithi107' 2024-04-25T00:18:10.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.571+0000 7fa970814740 1 --2- 172.21.15.107:0/3387526321 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55bcfc049ce0 0x55bcfc04c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.571+0000 7fa970814740 4 client.16008 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:18:10.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.571+0000 7fa970814740 1 -- 172.21.15.107:0/3387526321 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bcfbfd0a30 con 0x55bcfc049ce0 2024-04-25T00:18:10.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.572+0000 7fa9677fe640 1 --2- 172.21.15.107:0/3387526321 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55bcfc049ce0 0x55bcfc04c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.572+0000 7fa9677fe640 1 --2- 172.21.15.107:0/3387526321 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55bcfc049ce0 0x55bcfc04c100 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2c748ce77dcbe251 server_cookie=2d85b25692e8827f in_seq=0 out_seq=0 2024-04-25T00:18:10.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.572+0000 7fa946ffd640 10 client.16008 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:18:10.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.572+0000 7fa946ffd640 1 -- 172.21.15.107:0/3387526321 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bcfbfd0a30 con 0x55bcfc049ce0 2024-04-25T00:18:10.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.572+0000 7fa946ffd640 10 client.16008 handle_command_reply: tid=0 2024-04-25T00:18:10.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.572+0000 7fa970814740 1 client.16008 shutdown 2024-04-25T00:18:10.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.572+0000 7fa970814740 1 -- 172.21.15.107:0/3387526321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfd3570 msgr2=0x55bcfbfd6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.572+0000 7fa970814740 1 --2- 172.21.15.107:0/3387526321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfd3570 0x55bcfbfd6eb0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa96001ebf0 tx=0x7fa960004730 comp rx=0 tx=0).stop 2024-04-25T00:18:10.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.572+0000 7fa970814740 1 -- 172.21.15.107:0/3387526321 shutdown_connections 2024-04-25T00:18:10.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.572+0000 7fa970814740 1 --2- 172.21.15.107:0/3387526321 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55bcfc049ce0 0x55bcfc04c100 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.572+0000 7fa970814740 1 --2- 172.21.15.107:0/3387526321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bcfbfd3570 0x55bcfbfd6eb0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.572+0000 7fa946ffd640 0 client.16008 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:18:10.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.572+0000 7fa970814740 1 --2- 172.21.15.107:0/3387526321 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bcfbfc8ea0 0x55bcfbfdbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.572+0000 7fa970814740 1 --2- 172.21.15.107:0/3387526321 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bcfbfc8580 0x55bcfbfdb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.572+0000 7fa970814740 1 -- 172.21.15.107:0/3387526321 >> 172.21.15.107:0/3387526321 conn(0x55bcfbfc48a0 msgr2=0x55bcfbfca5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:10.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.572+0000 7fa970814740 1 -- 172.21.15.107:0/3387526321 shutdown_connections 2024-04-25T00:18:10.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.572+0000 7fa970814740 1 -- 172.21.15.107:0/3387526321 wait complete. 2024-04-25T00:18:10.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.573+0000 7fa970814740 20 client.16008 trim_cache size 0 max 16384 2024-04-25T00:18:10.581 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:18:10.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.573+0000 7fa96db09640 1 -- 172.21.15.107:0/2133667298 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa9500a2d00 msgr2=0x7fa9500a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.573+0000 7fa96db09640 1 --2- 172.21.15.107:0/2133667298 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa9500a2d00 0x7fa9500a51c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa960009ca0 tx=0x7fa9600046c0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.573+0000 7fa96db09640 1 -- 172.21.15.107:0/2133667298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9680ffc60 msgr2=0x7fa9681980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.573+0000 7fa96db09640 1 --2- 172.21.15.107:0/2133667298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9680ffc60 0x7fa9681980a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa95802ae00 tx=0x7fa958031e60 comp rx=0 tx=0).stop 2024-04-25T00:18:10.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.573+0000 7fa96db09640 1 -- 172.21.15.107:0/2133667298 shutdown_connections 2024-04-25T00:18:10.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.573+0000 7fa96db09640 1 --2- 172.21.15.107:0/2133667298 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa9500a2d00 0x7fa9500a51c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.573+0000 7fa96db09640 1 --2- 172.21.15.107:0/2133667298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa968107720 0x7fa96819c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.573+0000 7fa96db09640 1 --2- 172.21.15.107:0/2133667298 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa968100600 0x7fa9681985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.573+0000 7fa96db09640 1 --2- 172.21.15.107:0/2133667298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9680ffc60 0x7fa9681980a0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.573+0000 7fa96db09640 1 -- 172.21.15.107:0/2133667298 >> 172.21.15.107:0/2133667298 conn(0x7fa968076bd0 msgr2=0x7fa9680fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:10.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.573+0000 7fa96db09640 1 -- 172.21.15.107:0/2133667298 shutdown_connections 2024-04-25T00:18:10.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.573+0000 7fa96db09640 1 -- 172.21.15.107:0/2133667298 wait complete. 2024-04-25T00:18:10.584 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:18:10.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.687+0000 7f5047e7d640 1 -- 172.21.15.107:0/1766697374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50401020e0 msgr2=0x7f504010e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.687+0000 7f5047e7d640 1 --2- 172.21.15.107:0/1766697374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50401020e0 0x7f504010e5f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5038009930 tx=0x7f503802a420 comp rx=0 tx=0).stop 2024-04-25T00:18:10.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.687+0000 7f5047e7d640 1 -- 172.21.15.107:0/1766697374 shutdown_connections 2024-04-25T00:18:10.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.687+0000 7f5047e7d640 1 --2- 172.21.15.107:0/1766697374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50401020e0 0x7f504010e5f0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.687+0000 7f5047e7d640 1 --2- 172.21.15.107:0/1766697374 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5040101720 0x7f5040101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.687+0000 7f5047e7d640 1 --2- 172.21.15.107:0/1766697374 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5040107730 0x7f5040107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.687+0000 7f5047e7d640 1 -- 172.21.15.107:0/1766697374 >> 172.21.15.107:0/1766697374 conn(0x7f5040078b90 msgr2=0x7f5040078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:10.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.687+0000 7f5047e7d640 1 -- 172.21.15.107:0/1766697374 shutdown_connections 2024-04-25T00:18:10.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.687+0000 7f5047e7d640 1 -- 172.21.15.107:0/1766697374 wait complete. 2024-04-25T00:18:10.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.687+0000 7f5047e7d640 1 Processor -- start 2024-04-25T00:18:10.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.687+0000 7f5047e7d640 1 -- start start 2024-04-25T00:18:10.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.688+0000 7f5047e7d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5040101720 0x7f5040198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.688+0000 7f5047e7d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50401020e0 0x7f5040198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.688+0000 7f5047e7d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5040107730 0x7f504019c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.688+0000 7f5047e7d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5040111b40 con 0x7f5040107730 2024-04-25T00:18:10.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.688+0000 7f5047e7d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f50401119c0 con 0x7f5040101720 2024-04-25T00:18:10.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.688+0000 7f5047e7d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5040111cc0 con 0x7f50401020e0 2024-04-25T00:18:10.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.688+0000 7f504641a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5040107730 0x7f504019c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.688+0000 7f5045c19640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5040101720 0x7f5040198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.688+0000 7f504641a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5040107730 0x7f504019c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33934/0 (socket says 172.21.15.107:33934) 2024-04-25T00:18:10.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.688+0000 7f5045c19640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5040101720 0x7f5040198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.107:46722/0 (socket says 172.21.15.107:46722) 2024-04-25T00:18:10.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.688+0000 7f504641a640 1 -- 172.21.15.107:0/2653373141 learned_addr learned my addr 172.21.15.107:0/2653373141 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:10.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.688+0000 7f5045418640 1 --2- 172.21.15.107:0/2653373141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50401020e0 0x7f5040198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.688+0000 7f504641a640 1 -- 172.21.15.107:0/2653373141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50401020e0 msgr2=0x7f5040198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.688+0000 7f504641a640 1 --2- 172.21.15.107:0/2653373141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50401020e0 0x7f5040198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.688+0000 7f504641a640 1 -- 172.21.15.107:0/2653373141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5040101720 msgr2=0x7f5040198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.688+0000 7f504641a640 1 --2- 172.21.15.107:0/2653373141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5040101720 0x7f5040198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.688+0000 7f504641a640 1 -- 172.21.15.107:0/2653373141 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504019d010 con 0x7f5040107730 2024-04-25T00:18:10.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.688+0000 7f5045c19640 1 --2- 172.21.15.107:0/2653373141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5040101720 0x7f5040198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:10.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.688+0000 7f504641a640 1 --2- 172.21.15.107:0/2653373141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5040107730 0x7f504019c8e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f5038005ab0 tx=0x7f5038031b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:10.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.688+0000 7f5036ffd640 1 -- 172.21.15.107:0/2653373141 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503803f030 con 0x7f5040107730 2024-04-25T00:18:10.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.688+0000 7f5036ffd640 1 -- 172.21.15.107:0/2653373141 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50380043d0 con 0x7f5040107730 2024-04-25T00:18:10.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.688+0000 7f5047e7d640 1 -- 172.21.15.107:0/2653373141 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f504019d2a0 con 0x7f5040107730 2024-04-25T00:18:10.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.689+0000 7f5036ffd640 1 -- 172.21.15.107:0/2653373141 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038035070 con 0x7f5040107730 2024-04-25T00:18:10.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.689+0000 7f5047e7d640 1 -- 172.21.15.107:0/2653373141 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50401a4b90 con 0x7f5040107730 2024-04-25T00:18:10.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.691+0000 7f5036ffd640 1 -- 172.21.15.107:0/2653373141 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f5038042030 con 0x7f5040107730 2024-04-25T00:18:10.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.691+0000 7f504ab8e740 1 Processor -- start 2024-04-25T00:18:10.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.691+0000 7f504ab8e740 1 -- start start 2024-04-25T00:18:10.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.691+0000 7f5036ffd640 1 --2- 172.21.15.107:0/2653373141 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f502c0a2d00 0x7f502c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.691+0000 7f504ab8e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed67a580 0x55d5ed67a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.691+0000 7f504ab8e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5ed67aea0 0x55d5ed685030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.691+0000 7f504ab8e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d5ed685570 0x55d5ed687960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.691+0000 7f504ab8e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d5ed5df670 con 0x55d5ed67a580 2024-04-25T00:18:10.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.691+0000 7f504ab8e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d5ed587970 con 0x55d5ed685570 2024-04-25T00:18:10.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.691+0000 7f504ab8e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d5ed5db630 con 0x55d5ed67aea0 2024-04-25T00:18:10.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.692+0000 7f5045418640 1 --2- 172.21.15.107:0/2653373141 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f502c0a2d00 0x7f502c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.692+0000 7f5045c19640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed67a580 0x55d5ed67a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.692+0000 7f5045c19640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed67a580 0x55d5ed67a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33940/0 (socket says 172.21.15.107:33940) 2024-04-25T00:18:10.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.692+0000 7f5045c19640 1 -- 172.21.15.107:0/979812307 learned_addr learned my addr 172.21.15.107:0/979812307 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:10.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.692+0000 7f504641a640 1 --2- 172.21.15.107:0/979812307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d5ed685570 0x55d5ed687960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.692+0000 7f5036ffd640 1 -- 172.21.15.107:0/2653373141 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f50380ef870 con 0x7f5040107730 2024-04-25T00:18:10.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.692+0000 7f5045418640 1 --2- 172.21.15.107:0/979812307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5ed67aea0 0x55d5ed685030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.692+0000 7f5045c19640 1 -- 172.21.15.107:0/979812307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5ed67aea0 msgr2=0x55d5ed685030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:10.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.692+0000 7f5045c19640 1 --2- 172.21.15.107:0/979812307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5ed67aea0 0x55d5ed685030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.692+0000 7f5045c19640 1 -- 172.21.15.107:0/979812307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d5ed685570 msgr2=0x55d5ed687960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:10.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.692+0000 7f5045c19640 1 --2- 172.21.15.107:0/979812307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d5ed685570 0x55d5ed687960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.692+0000 7f5045c19640 1 -- 172.21.15.107:0/979812307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5ed5ab400 con 0x55d5ed67a580 2024-04-25T00:18:10.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.692+0000 7f5045418640 1 --2- 172.21.15.107:0/2653373141 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f502c0a2d00 0x7f502c0a51c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5040101030 tx=0x7f50300137d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:10.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.692+0000 7f5045c19640 1 --2- 172.21.15.107:0/979812307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed67a580 0x55d5ed67a960 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f503c010750 tx=0x7f503c010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53070e0b1c3b0bf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:10.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.692+0000 7f502bfff640 1 -- 172.21.15.107:0/979812307 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c00e8d0 con 0x55d5ed67a580 2024-04-25T00:18:10.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.692+0000 7f502bfff640 1 -- 172.21.15.107:0/979812307 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f503c012450 con 0x55d5ed67a580 2024-04-25T00:18:10.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.692+0000 7f502bfff640 1 -- 172.21.15.107:0/979812307 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c011640 con 0x55d5ed67a580 2024-04-25T00:18:10.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.693+0000 7f504ab8e740 1 -- 172.21.15.107:0/979812307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed67a580 msgr2=0x55d5ed67a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.693+0000 7f504ab8e740 1 --2- 172.21.15.107:0/979812307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed67a580 0x55d5ed67a960 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f503c010750 tx=0x7f503c010c10 comp rx=0 tx=0).stop 2024-04-25T00:18:10.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.693+0000 7f504ab8e740 1 -- 172.21.15.107:0/979812307 shutdown_connections 2024-04-25T00:18:10.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.693+0000 7f504ab8e740 1 --2- 172.21.15.107:0/979812307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d5ed685570 0x55d5ed687960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.693+0000 7f504ab8e740 1 --2- 172.21.15.107:0/979812307 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5ed67aea0 0x55d5ed685030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.693+0000 7f504ab8e740 1 --2- 172.21.15.107:0/979812307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed67a580 0x55d5ed67a960 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.693+0000 7f504ab8e740 1 -- 172.21.15.107:0/979812307 >> 172.21.15.107:0/979812307 conn(0x55d5ed6768a0 msgr2=0x55d5ed676c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:10.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.693+0000 7f504ab8e740 1 -- 172.21.15.107:0/979812307 shutdown_connections 2024-04-25T00:18:10.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.693+0000 7f504ab8e740 1 -- 172.21.15.107:0/979812307 wait complete. 2024-04-25T00:18:10.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.693+0000 7f504ab8e740 1 -- start start 2024-04-25T00:18:10.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.693+0000 7f504ab8e740 1 Processor -- start 2024-04-25T00:18:10.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.694+0000 7f504ab8e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5ed67a580 0x55d5ed684ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.694+0000 7f504ab8e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed67aea0 0x55d5ed682190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.694+0000 7f504ab8e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d5ed685570 0x55d5ed6826d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.694+0000 7f504ab8e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d5ed688360 con 0x55d5ed67aea0 2024-04-25T00:18:10.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.694+0000 7f504ab8e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d5ed59b050 con 0x55d5ed685570 2024-04-25T00:18:10.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.694+0000 7f504ab8e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d5ed6884e0 con 0x55d5ed67a580 2024-04-25T00:18:10.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.694+0000 7f504ab8e740 10 client.0 _ll_register_callbacks cb 0x55d5ed2f2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:10.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.694+0000 7f5045418640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed67aea0 0x55d5ed682190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.694+0000 7f5045418640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed67aea0 0x55d5ed682190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33946/0 (socket says 172.21.15.107:33946) 2024-04-25T00:18:10.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.694+0000 7f5045418640 1 -- 172.21.15.107:0/2455362370 learned_addr learned my addr 172.21.15.107:0/2455362370 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:10.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.694+0000 7f504641a640 1 --2- 172.21.15.107:0/2455362370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d5ed685570 0x55d5ed6826d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.694+0000 7f5045c19640 1 --2- 172.21.15.107:0/2455362370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5ed67a580 0x55d5ed684ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.694+0000 7f5045418640 1 -- 172.21.15.107:0/2455362370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5ed67a580 msgr2=0x55d5ed684ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:10.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.694+0000 7f5045418640 1 --2- 172.21.15.107:0/2455362370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5ed67a580 0x55d5ed684ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.694+0000 7f5045418640 1 -- 172.21.15.107:0/2455362370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d5ed685570 msgr2=0x55d5ed6826d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:10.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.694+0000 7f5045418640 1 --2- 172.21.15.107:0/2455362370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d5ed685570 0x55d5ed6826d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.694+0000 7f5045418640 1 -- 172.21.15.107:0/2455362370 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5ed56cdc0 con 0x55d5ed67aea0 2024-04-25T00:18:10.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.694+0000 7f5045418640 1 --2- 172.21.15.107:0/2455362370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed67aea0 0x55d5ed682190 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f503000f960 tx=0x7f5030004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea9b7c076186950 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:10.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.694+0000 7f50297fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:18:10.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.695+0000 7f50297fa640 1 -- 172.21.15.107:0/2455362370 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503000fe20 con 0x55d5ed67aea0 2024-04-25T00:18:10.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.695+0000 7f50297fa640 1 -- 172.21.15.107:0/2455362370 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f5030010b60 con 0x55d5ed67aea0 2024-04-25T00:18:10.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.695+0000 7f504ab8e740 1 -- 172.21.15.107:0/2455362370 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f503c012450 con 0x55d5ed67aea0 2024-04-25T00:18:10.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.695+0000 7f50297fa640 1 -- 172.21.15.107:0/2455362370 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5030010d60 con 0x55d5ed67aea0 2024-04-25T00:18:10.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.695+0000 7f504ab8e740 10 client.16020 fetch_fsmap learned FSMap version 26 2024-04-25T00:18:10.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.695+0000 7f504ab8e740 1 -- 172.21.15.107:0/2455362370 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d5ed2f33f0 con 0x55d5ed67aea0 2024-04-25T00:18:10.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.695+0000 7f50297fa640 1 -- 172.21.15.107:0/2455362370 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f503c012450 con 0x55d5ed67aea0 2024-04-25T00:18:10.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.695+0000 7f50297fa640 1 -- 172.21.15.107:0/2455362370 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f5030021490 con 0x55d5ed67aea0 2024-04-25T00:18:10.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.695+0000 7f504ab8e740 10 client.16020 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:18:10.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.695+0000 7f504ab8e740 10 client.16020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:18:10.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.695+0000 7f504ab8e740 20 client.16020 populate_metadata read hostname 'smithi107' 2024-04-25T00:18:10.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.695+0000 7f504ab8e740 1 --2- 172.21.15.107:0/2455362370 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d5ed695600 0x55d5ed697aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.695+0000 7f504ab8e740 4 client.16020 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:18:10.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.695+0000 7f504ab8e740 1 -- 172.21.15.107:0/2455362370 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5ed5ecbb0 con 0x55d5ed695600 2024-04-25T00:18:10.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.695+0000 7f5045c19640 1 --2- 172.21.15.107:0/2455362370 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d5ed695600 0x55d5ed697aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.696+0000 7f5045c19640 1 --2- 172.21.15.107:0/2455362370 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d5ed695600 0x55d5ed697aa0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31f60977d7f7db3c server_cookie=2775343261975839 in_seq=0 out_seq=0 2024-04-25T00:18:10.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.696+0000 7f50297fa640 10 client.16020 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:10.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.697+0000 7f50297fa640 1 -- 172.21.15.107:0/2455362370 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d5ed5ecbb0 con 0x55d5ed695600 2024-04-25T00:18:10.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.697+0000 7f50297fa640 10 client.16020 handle_command_reply: tid=0 2024-04-25T00:18:10.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.697+0000 7f504ab8e740 1 client.16020 shutdown 2024-04-25T00:18:10.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.697+0000 7f504ab8e740 1 -- 172.21.15.107:0/2455362370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed67aea0 msgr2=0x55d5ed682190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.697+0000 7f504ab8e740 1 --2- 172.21.15.107:0/2455362370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed67aea0 0x55d5ed682190 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f503000f960 tx=0x7f5030004730 comp rx=0 tx=0).stop 2024-04-25T00:18:10.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.697+0000 7f504ab8e740 1 -- 172.21.15.107:0/2455362370 shutdown_connections 2024-04-25T00:18:10.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.697+0000 7f504ab8e740 1 --2- 172.21.15.107:0/2455362370 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d5ed695600 0x55d5ed697aa0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.697+0000 7f504ab8e740 1 --2- 172.21.15.107:0/2455362370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d5ed685570 0x55d5ed6826d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.697+0000 7f504ab8e740 1 --2- 172.21.15.107:0/2455362370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed67aea0 0x55d5ed682190 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.697+0000 7f504ab8e740 1 --2- 172.21.15.107:0/2455362370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5ed67a580 0x55d5ed684ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.697+0000 7f50297fa640 0 client.16020 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:10.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.697+0000 7f504ab8e740 1 -- 172.21.15.107:0/2455362370 >> 172.21.15.107:0/2455362370 conn(0x55d5ed6768a0 msgr2=0x55d5ed67c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:10.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.697+0000 7f504ab8e740 1 -- 172.21.15.107:0/2455362370 shutdown_connections 2024-04-25T00:18:10.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.697+0000 7f504ab8e740 1 -- 172.21.15.107:0/2455362370 wait complete. 2024-04-25T00:18:10.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.697+0000 7f504ab8e740 20 client.16020 trim_cache size 0 max 16384 2024-04-25T00:18:10.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.710+0000 7f504ab8e740 1 Processor -- start 2024-04-25T00:18:10.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.710+0000 7f504ab8e740 1 -- start start 2024-04-25T00:18:10.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.710+0000 7f504ab8e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed67a580 0x55d5ed6849d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.711+0000 7f504ab8e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5ed67aea0 0x55d5ed68d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.711+0000 7f504ab8e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d5ed685570 0x55d5ed68dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.711+0000 7f504ab8e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f50401b1430 con 0x55d5ed67a580 2024-04-25T00:18:10.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.711+0000 7f504ab8e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f50401aeca0 con 0x55d5ed685570 2024-04-25T00:18:10.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.711+0000 7f504ab8e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f50401acaf0 con 0x55d5ed67aea0 2024-04-25T00:18:10.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.711+0000 7f5045c19640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed67a580 0x55d5ed6849d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.711+0000 7f5045c19640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed67a580 0x55d5ed6849d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33960/0 (socket says 172.21.15.107:33960) 2024-04-25T00:18:10.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.711+0000 7f5045c19640 1 -- 172.21.15.107:0/859365266 learned_addr learned my addr 172.21.15.107:0/859365266 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.711+0000 7f5045418640 1 --2- 172.21.15.107:0/859365266 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5ed67aea0 0x55d5ed68d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.711+0000 7f5045c19640 1 -- 172.21.15.107:0/859365266 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5ed67aea0 msgr2=0x55d5ed68d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.711+0000 7f5045c19640 1 --2- 172.21.15.107:0/859365266 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5ed67aea0 0x55d5ed68d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.711+0000 7f5045c19640 1 -- 172.21.15.107:0/859365266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d5ed685570 msgr2=0x55d5ed68dc70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:18:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.711+0000 7f5045c19640 1 --2- 172.21.15.107:0/859365266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d5ed685570 0x55d5ed68dc70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.711+0000 7f5045c19640 1 -- 172.21.15.107:0/859365266 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5ed3bd7d0 con 0x55d5ed67a580 2024-04-25T00:18:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.711+0000 7f5045c19640 1 --2- 172.21.15.107:0/859365266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed67a580 0x55d5ed6849d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f503c00e6c0 tx=0x7f503c012e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0eef85a0e3e1b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.711+0000 7f502b7fe640 1 -- 172.21.15.107:0/859365266 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c030070 con 0x55d5ed67a580 2024-04-25T00:18:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.711+0000 7f502b7fe640 1 -- 172.21.15.107:0/859365266 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f503c004db0 con 0x55d5ed67a580 2024-04-25T00:18:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.711+0000 7f502b7fe640 1 -- 172.21.15.107:0/859365266 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503c00ed30 con 0x55d5ed67a580 2024-04-25T00:18:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.712+0000 7f504ab8e740 1 -- 172.21.15.107:0/859365266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed67a580 msgr2=0x55d5ed6849d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.712+0000 7f504ab8e740 1 --2- 172.21.15.107:0/859365266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed67a580 0x55d5ed6849d0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f503c00e6c0 tx=0x7f503c012e30 comp rx=0 tx=0).stop 2024-04-25T00:18:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.712+0000 7f504ab8e740 1 -- 172.21.15.107:0/859365266 shutdown_connections 2024-04-25T00:18:10.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.712+0000 7f504ab8e740 1 --2- 172.21.15.107:0/859365266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d5ed685570 0x55d5ed68dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.712+0000 7f504ab8e740 1 --2- 172.21.15.107:0/859365266 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5ed67aea0 0x55d5ed68d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.712+0000 7f504ab8e740 1 --2- 172.21.15.107:0/859365266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed67a580 0x55d5ed6849d0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.712+0000 7f504ab8e740 1 -- 172.21.15.107:0/859365266 >> 172.21.15.107:0/859365266 conn(0x55d5ed6768a0 msgr2=0x55d5ed675d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:10.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.712+0000 7f504ab8e740 1 -- 172.21.15.107:0/859365266 shutdown_connections 2024-04-25T00:18:10.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.712+0000 7f504ab8e740 1 -- 172.21.15.107:0/859365266 wait complete. 2024-04-25T00:18:10.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.712+0000 7f504ab8e740 1 -- start start 2024-04-25T00:18:10.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.712+0000 7f504ab8e740 1 Processor -- start 2024-04-25T00:18:10.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f504ab8e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d5ed67a580 0x55d5ed68b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f504ab8e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5ed67aea0 0x55d5ed68b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f504ab8e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed685570 0x55d5ed68be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f504ab8e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f50401ad160 con 0x55d5ed685570 2024-04-25T00:18:10.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f504ab8e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f50401ae5d0 con 0x55d5ed67a580 2024-04-25T00:18:10.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f504ab8e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f50401ad830 con 0x55d5ed67aea0 2024-04-25T00:18:10.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f504ab8e740 10 client.0 _ll_register_callbacks cb 0x7f5030009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:10.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f504641a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed685570 0x55d5ed68be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f5045418640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5ed67aea0 0x55d5ed68b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f504641a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed685570 0x55d5ed68be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33974/0 (socket says 172.21.15.107:33974) 2024-04-25T00:18:10.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f504641a640 1 -- 172.21.15.107:0/2454782575 learned_addr learned my addr 172.21.15.107:0/2454782575 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:10.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f5045c19640 1 --2- 172.21.15.107:0/2454782575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d5ed67a580 0x55d5ed68b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f504641a640 1 -- 172.21.15.107:0/2454782575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5ed67aea0 msgr2=0x55d5ed68b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:10.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f504641a640 1 --2- 172.21.15.107:0/2454782575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5ed67aea0 0x55d5ed68b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f504641a640 1 -- 172.21.15.107:0/2454782575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d5ed67a580 msgr2=0x55d5ed68b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:10.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f504641a640 1 --2- 172.21.15.107:0/2454782575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d5ed67a580 0x55d5ed68b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f504641a640 1 -- 172.21.15.107:0/2454782575 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5ed6fae90 con 0x55d5ed685570 2024-04-25T00:18:10.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f504641a640 1 --2- 172.21.15.107:0/2454782575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed685570 0x55d5ed68be80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f50380ed250 tx=0x7f50380bb4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b088046ca1adde9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:10.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f50297fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:18:10.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f50297fa640 1 -- 172.21.15.107:0/2454782575 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038049040 con 0x55d5ed685570 2024-04-25T00:18:10.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f50297fa640 1 -- 172.21.15.107:0/2454782575 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f50380eced0 con 0x55d5ed685570 2024-04-25T00:18:10.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f504ab8e740 1 -- 172.21.15.107:0/2454782575 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f503c004db0 con 0x55d5ed685570 2024-04-25T00:18:10.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f50297fa640 1 -- 172.21.15.107:0/2454782575 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5038098dc0 con 0x55d5ed685570 2024-04-25T00:18:10.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f504ab8e740 10 client.16026 fetch_fsmap learned FSMap version 26 2024-04-25T00:18:10.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.713+0000 7f504ab8e740 1 -- 172.21.15.107:0/2454782575 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d5ed6fc1f0 con 0x55d5ed685570 2024-04-25T00:18:10.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.714+0000 7f50297fa640 1 -- 172.21.15.107:0/2454782575 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f503c004db0 con 0x55d5ed685570 2024-04-25T00:18:10.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.714+0000 7f50297fa640 1 -- 172.21.15.107:0/2454782575 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f50380b7ff0 con 0x55d5ed685570 2024-04-25T00:18:10.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.714+0000 7f504ab8e740 10 client.16026 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:18:10.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.714+0000 7f504ab8e740 10 client.16026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:18:10.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.714+0000 7f504ab8e740 20 client.16026 populate_metadata read hostname 'smithi107' 2024-04-25T00:18:10.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.714+0000 7f504ab8e740 1 --2- 172.21.15.107:0/2454782575 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d5ed6fc970 0x55d5ed6fed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:10.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.714+0000 7f504ab8e740 4 client.16026 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:18:10.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.714+0000 7f504ab8e740 1 -- 172.21.15.107:0/2454782575 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d5ed5e6f00 con 0x55d5ed6fc970 2024-04-25T00:18:10.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.714+0000 7f5045c19640 1 --2- 172.21.15.107:0/2454782575 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d5ed6fc970 0x55d5ed6fed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:10.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.714+0000 7f5045c19640 1 --2- 172.21.15.107:0/2454782575 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d5ed6fc970 0x55d5ed6fed90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a150d859002fe8f server_cookie=3fb1093900e59a1b in_seq=0 out_seq=0 2024-04-25T00:18:10.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.714+0000 7f50297fa640 10 client.16026 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:10.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.714+0000 7f50297fa640 1 -- 172.21.15.107:0/2454782575 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d5ed5e6f00 con 0x55d5ed6fc970 2024-04-25T00:18:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.714+0000 7f50297fa640 10 client.16026 handle_command_reply: tid=0 2024-04-25T00:18:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.714+0000 7f504ab8e740 1 client.16026 shutdown 2024-04-25T00:18:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.715+0000 7f504ab8e740 1 -- 172.21.15.107:0/2454782575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed685570 msgr2=0x55d5ed68be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.715+0000 7f504ab8e740 1 --2- 172.21.15.107:0/2454782575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed685570 0x55d5ed68be80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f50380ed250 tx=0x7f50380bb4c0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.715+0000 7f504ab8e740 1 -- 172.21.15.107:0/2454782575 shutdown_connections 2024-04-25T00:18:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.715+0000 7f504ab8e740 1 --2- 172.21.15.107:0/2454782575 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d5ed6fc970 0x55d5ed6fed90 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.715+0000 7f504ab8e740 1 --2- 172.21.15.107:0/2454782575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d5ed685570 0x55d5ed68be80 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.715+0000 7f50297fa640 0 client.16026 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.715+0000 7f504ab8e740 1 --2- 172.21.15.107:0/2454782575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d5ed67aea0 0x55d5ed68b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.715+0000 7f504ab8e740 1 --2- 172.21.15.107:0/2454782575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d5ed67a580 0x55d5ed68b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.715+0000 7f504ab8e740 1 -- 172.21.15.107:0/2454782575 >> 172.21.15.107:0/2454782575 conn(0x55d5ed6768a0 msgr2=0x55d5ed690aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.715+0000 7f504ab8e740 1 -- 172.21.15.107:0/2454782575 shutdown_connections 2024-04-25T00:18:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.715+0000 7f504ab8e740 1 -- 172.21.15.107:0/2454782575 wait complete. 2024-04-25T00:18:10.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.715+0000 7f504ab8e740 20 client.16026 trim_cache size 0 max 16384 2024-04-25T00:18:10.723 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:18:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.715+0000 7f5047e7d640 1 -- 172.21.15.107:0/2653373141 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f502c0a2d00 msgr2=0x7f502c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.715+0000 7f5047e7d640 1 --2- 172.21.15.107:0/2653373141 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f502c0a2d00 0x7f502c0a51c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5040101030 tx=0x7f50300137d0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.715+0000 7f5047e7d640 1 -- 172.21.15.107:0/2653373141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5040107730 msgr2=0x7f504019c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.715+0000 7f5047e7d640 1 --2- 172.21.15.107:0/2653373141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5040107730 0x7f504019c8e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f5038005ab0 tx=0x7f5038031b50 comp rx=0 tx=0).stop 2024-04-25T00:18:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.715+0000 7f5047e7d640 1 -- 172.21.15.107:0/2653373141 shutdown_connections 2024-04-25T00:18:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.715+0000 7f5047e7d640 1 --2- 172.21.15.107:0/2653373141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5040107730 0x7f504019c8e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.715+0000 7f5047e7d640 1 --2- 172.21.15.107:0/2653373141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50401020e0 0x7f5040198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.715+0000 7f5047e7d640 1 --2- 172.21.15.107:0/2653373141 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f502c0a2d00 0x7f502c0a51c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.715+0000 7f5047e7d640 1 --2- 172.21.15.107:0/2653373141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5040101720 0x7f5040198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.715+0000 7f5047e7d640 1 -- 172.21.15.107:0/2653373141 >> 172.21.15.107:0/2653373141 conn(0x7f5040078b90 msgr2=0x7f50400ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.715+0000 7f5047e7d640 1 -- 172.21.15.107:0/2653373141 shutdown_connections 2024-04-25T00:18:10.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:10.715+0000 7f5047e7d640 1 -- 172.21.15.107:0/2653373141 wait complete. 2024-04-25T00:18:11.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:10 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2091968690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:11.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:10 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2091968690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:11.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:10 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2091968690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:11.727 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:18:11.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:18:11.727 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:18:11.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.836+0000 7f2f28b41640 1 -- 172.21.15.107:0/3095661464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f24107740 msgr2=0x7f2f24107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:11.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.836+0000 7f2f28b41640 1 --2- 172.21.15.107:0/3095661464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f24107740 0x7f2f24107b20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2f14009900 tx=0x7f2f1402a410 comp rx=0 tx=0).stop 2024-04-25T00:18:11.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.836+0000 7f2f28b41640 1 -- 172.21.15.107:0/3095661464 shutdown_connections 2024-04-25T00:18:11.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.836+0000 7f2f28b41640 1 --2- 172.21.15.107:0/3095661464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f2406a6e0 0x7f2f2410e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.836+0000 7f2f28b41640 1 --2- 172.21.15.107:0/3095661464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f24069d40 0x7f2f2406a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.836+0000 7f2f28b41640 1 --2- 172.21.15.107:0/3095661464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f24107740 0x7f2f24107b20 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.836+0000 7f2f28b41640 1 -- 172.21.15.107:0/3095661464 >> 172.21.15.107:0/3095661464 conn(0x7f2f24076440 msgr2=0x7f2f24076850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:11.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.836+0000 7f2f28b41640 1 -- 172.21.15.107:0/3095661464 shutdown_connections 2024-04-25T00:18:11.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.836+0000 7f2f28b41640 1 -- 172.21.15.107:0/3095661464 wait complete. 2024-04-25T00:18:11.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.837+0000 7f2f28b41640 1 Processor -- start 2024-04-25T00:18:11.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.837+0000 7f2f28b41640 1 -- start start 2024-04-25T00:18:11.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.837+0000 7f2f28b41640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f24069d40 0x7f2f24198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.837+0000 7f2f28b41640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f2406a6e0 0x7f2f241985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.837+0000 7f2f28b41640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f24107740 0x7f2f2419c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.837+0000 7f2f28b41640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2f24111b20 con 0x7f2f24069d40 2024-04-25T00:18:11.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.837+0000 7f2f28b41640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2f241119a0 con 0x7f2f24107740 2024-04-25T00:18:11.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.837+0000 7f2f28b41640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2f24111ca0 con 0x7f2f2406a6e0 2024-04-25T00:18:11.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.837+0000 7f2f2259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f24069d40 0x7f2f24198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.837+0000 7f2f2259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f24069d40 0x7f2f24198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33998/0 (socket says 172.21.15.107:33998) 2024-04-25T00:18:11.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.837+0000 7f2f2259c640 1 -- 172.21.15.107:0/3389563329 learned_addr learned my addr 172.21.15.107:0/3389563329 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:11.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.837+0000 7f2f22d9d640 1 --2- 172.21.15.107:0/3389563329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f24107740 0x7f2f2419c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.837+0000 7f2f21d9b640 1 --2- 172.21.15.107:0/3389563329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f2406a6e0 0x7f2f241985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.837+0000 7f2f2259c640 1 -- 172.21.15.107:0/3389563329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f2406a6e0 msgr2=0x7f2f241985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:11.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.837+0000 7f2f2259c640 1 --2- 172.21.15.107:0/3389563329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f2406a6e0 0x7f2f241985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.837+0000 7f2f2259c640 1 -- 172.21.15.107:0/3389563329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f24107740 msgr2=0x7f2f2419c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:11.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.837+0000 7f2f2259c640 1 --2- 172.21.15.107:0/3389563329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f24107740 0x7f2f2419c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.837+0000 7f2f2259c640 1 -- 172.21.15.107:0/3389563329 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2419d050 con 0x7f2f24069d40 2024-04-25T00:18:11.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.837+0000 7f2f2259c640 1 --2- 172.21.15.107:0/3389563329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f24069d40 0x7f2f24198060 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2f14006010 tx=0x7f2f14003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:11.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.838+0000 7f2f137fe640 1 -- 172.21.15.107:0/3389563329 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1403f070 con 0x7f2f24069d40 2024-04-25T00:18:11.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.838+0000 7f2f28b41640 1 -- 172.21.15.107:0/3389563329 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f2419d2e0 con 0x7f2f24069d40 2024-04-25T00:18:11.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.838+0000 7f2f137fe640 1 -- 172.21.15.107:0/3389563329 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f14031c30 con 0x7f2f24069d40 2024-04-25T00:18:11.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.838+0000 7f2f137fe640 1 -- 172.21.15.107:0/3389563329 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f14035030 con 0x7f2f24069d40 2024-04-25T00:18:11.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.838+0000 7f2f28b41640 1 -- 172.21.15.107:0/3389563329 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f241a4bd0 con 0x7f2f24069d40 2024-04-25T00:18:11.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.840+0000 7f2f137fe640 1 -- 172.21.15.107:0/3389563329 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f2f14042070 con 0x7f2f24069d40 2024-04-25T00:18:11.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.840+0000 7f2f2b845740 1 Processor -- start 2024-04-25T00:18:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.840+0000 7f2f2b845740 1 -- start start 2024-04-25T00:18:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.840+0000 7f2f2b845740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfc3980 0x55d2fcfc1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.840+0000 7f2f2b845740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2fcfc2290 0x55d2fcfc2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.841+0000 7f2f2b845740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2fcfcd010 0x55d2fcfcf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.841+0000 7f2f2b845740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d2fcf262c0 con 0x55d2fcfc3980 2024-04-25T00:18:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.841+0000 7f2f2b845740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d2fcece910 con 0x55d2fcfc2290 2024-04-25T00:18:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.841+0000 7f2f2b845740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d2fcf22280 con 0x55d2fcfcd010 2024-04-25T00:18:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.841+0000 7f2f22d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfc3980 0x55d2fcfc1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.841+0000 7f2f137fe640 1 --2- 172.21.15.107:0/3389563329 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f2f0c0a2d00 0x7f2f0c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.841+0000 7f2f2259c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2fcfc2290 0x55d2fcfc2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.841+0000 7f2f22d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfc3980 0x55d2fcfc1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34000/0 (socket says 172.21.15.107:34000) 2024-04-25T00:18:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.841+0000 7f2f22d9d640 1 -- 172.21.15.107:0/1633947469 learned_addr learned my addr 172.21.15.107:0/1633947469 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:11.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.841+0000 7f2f22d9d640 1 --2- 172.21.15.107:0/3389563329 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f2f0c0a2d00 0x7f2f0c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.841+0000 7f2f21d9b640 1 --2- 172.21.15.107:0/1633947469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2fcfcd010 0x55d2fcfcf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.841+0000 7f2f137fe640 1 -- 172.21.15.107:0/3389563329 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f2f140efea0 con 0x7f2f24069d40 2024-04-25T00:18:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.841+0000 7f2f22d9d640 1 -- 172.21.15.107:0/1633947469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2fcfcd010 msgr2=0x55d2fcfcf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.841+0000 7f2f22d9d640 1 --2- 172.21.15.107:0/1633947469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2fcfcd010 0x55d2fcfcf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.841+0000 7f2f22d9d640 1 -- 172.21.15.107:0/1633947469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2fcfc2290 msgr2=0x55d2fcfc2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.841+0000 7f2f22d9d640 1 --2- 172.21.15.107:0/1633947469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2fcfc2290 0x55d2fcfc2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.841+0000 7f2f22d9d640 1 -- 172.21.15.107:0/1633947469 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2fcedffb0 con 0x55d2fcfc3980 2024-04-25T00:18:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.841+0000 7f2f22d9d640 1 --2- 172.21.15.107:0/1633947469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfc3980 0x55d2fcfc1cc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00afc0 tx=0x7f2f1c017700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701c4311de7f5472 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.841+0000 7f2f22d9d640 1 --2- 172.21.15.107:0/3389563329 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f2f0c0a2d00 0x7f2f0c0a51c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c017c00 tx=0x7f2f1c01b000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.841+0000 7f2f10ff9640 1 -- 172.21.15.107:0/1633947469 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c027070 con 0x55d2fcfc3980 2024-04-25T00:18:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.841+0000 7f2f10ff9640 1 -- 172.21.15.107:0/1633947469 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f1c009e10 con 0x55d2fcfc3980 2024-04-25T00:18:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.841+0000 7f2f10ff9640 1 -- 172.21.15.107:0/1633947469 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c014040 con 0x55d2fcfc3980 2024-04-25T00:18:11.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.842+0000 7f2f2b845740 1 -- 172.21.15.107:0/1633947469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfc3980 msgr2=0x55d2fcfc1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.842+0000 7f2f2b845740 1 --2- 172.21.15.107:0/1633947469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfc3980 0x55d2fcfc1cc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00afc0 tx=0x7f2f1c017700 comp rx=0 tx=0).stop 2024-04-25T00:18:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.842+0000 7f2f2b845740 1 -- 172.21.15.107:0/1633947469 shutdown_connections 2024-04-25T00:18:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.842+0000 7f2f2b845740 1 --2- 172.21.15.107:0/1633947469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2fcfcd010 0x55d2fcfcf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.842+0000 7f2f2b845740 1 --2- 172.21.15.107:0/1633947469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2fcfc2290 0x55d2fcfc2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.842+0000 7f2f2b845740 1 --2- 172.21.15.107:0/1633947469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfc3980 0x55d2fcfc1cc0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.842+0000 7f2f2b845740 1 -- 172.21.15.107:0/1633947469 >> 172.21.15.107:0/1633947469 conn(0x55d2fcf8a950 msgr2=0x55d2fcfbe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.842+0000 7f2f2b845740 1 -- 172.21.15.107:0/1633947469 shutdown_connections 2024-04-25T00:18:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.842+0000 7f2f2b845740 1 -- 172.21.15.107:0/1633947469 wait complete. 2024-04-25T00:18:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.843+0000 7f2f2b845740 1 -- start start 2024-04-25T00:18:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.843+0000 7f2f2b845740 1 Processor -- start 2024-04-25T00:18:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.843+0000 7f2f2b845740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfc2290 0x55d2fcfc9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.843+0000 7f2f2b845740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2fcfc3980 0x55d2fcfca460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.843+0000 7f2f2b845740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2fcfcd010 0x55d2fcfca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.843+0000 7f2f2b845740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d2fcfcfe40 con 0x55d2fcfc2290 2024-04-25T00:18:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.843+0000 7f2f2b845740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d2fcee1e90 con 0x55d2fcfcd010 2024-04-25T00:18:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.843+0000 7f2f2b845740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d2fcfcffc0 con 0x55d2fcfc3980 2024-04-25T00:18:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.843+0000 7f2f2b845740 10 client.0 _ll_register_callbacks cb 0x55d2fcf1d970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.843+0000 7f2f2259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfc2290 0x55d2fcfc9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.843+0000 7f2f2259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfc2290 0x55d2fcfc9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34014/0 (socket says 172.21.15.107:34014) 2024-04-25T00:18:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.843+0000 7f2f2259c640 1 -- 172.21.15.107:0/2544831598 learned_addr learned my addr 172.21.15.107:0/2544831598 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.843+0000 7f2f22d9d640 1 --2- 172.21.15.107:0/2544831598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2fcfcd010 0x55d2fcfca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.843+0000 7f2f21d9b640 1 --2- 172.21.15.107:0/2544831598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2fcfc3980 0x55d2fcfca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.843+0000 7f2f2259c640 1 -- 172.21.15.107:0/2544831598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2fcfc3980 msgr2=0x55d2fcfca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.843+0000 7f2f2259c640 1 --2- 172.21.15.107:0/2544831598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2fcfc3980 0x55d2fcfca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.843+0000 7f2f2259c640 1 -- 172.21.15.107:0/2544831598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2fcfcd010 msgr2=0x55d2fcfca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:11.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.843+0000 7f2f2259c640 1 --2- 172.21.15.107:0/2544831598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2fcfcd010 0x55d2fcfca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.843+0000 7f2f2259c640 1 -- 172.21.15.107:0/2544831598 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2fcd027e0 con 0x55d2fcfc2290 2024-04-25T00:18:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.843+0000 7f2f2259c640 1 --2- 172.21.15.107:0/2544831598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfc2290 0x55d2fcfc9f20 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2f140bbfe0 tx=0x7f2f1408beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538a0bf4ca07301c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.843+0000 7f2f09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:18:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.844+0000 7f2f09ffb640 1 -- 172.21.15.107:0/2544831598 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f14049040 con 0x55d2fcfc2290 2024-04-25T00:18:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.844+0000 7f2f09ffb640 1 -- 172.21.15.107:0/2544831598 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f2f1403b460 con 0x55d2fcfc2290 2024-04-25T00:18:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.844+0000 7f2f2b845740 1 -- 172.21.15.107:0/2544831598 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d2fcefa5a0 con 0x55d2fcfc2290 2024-04-25T00:18:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.844+0000 7f2f09ffb640 1 -- 172.21.15.107:0/2544831598 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2f140c0400 con 0x55d2fcfc2290 2024-04-25T00:18:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.844+0000 7f2f2b845740 10 client.16038 fetch_fsmap learned FSMap version 26 2024-04-25T00:18:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.844+0000 7f2f2b845740 1 -- 172.21.15.107:0/2544831598 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d2fcd62f70 con 0x55d2fcfc2290 2024-04-25T00:18:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.844+0000 7f2f09ffb640 1 -- 172.21.15.107:0/2544831598 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d2fcefa5a0 con 0x55d2fcfc2290 2024-04-25T00:18:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.844+0000 7f2f09ffb640 1 -- 172.21.15.107:0/2544831598 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f2f140b86c0 con 0x55d2fcfc2290 2024-04-25T00:18:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.844+0000 7f2f2b845740 10 client.16038 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:18:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.844+0000 7f2f2b845740 10 client.16038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:18:11.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.844+0000 7f2f2b845740 20 client.16038 populate_metadata read hostname 'smithi107' 2024-04-25T00:18:11.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.844+0000 7f2f2b845740 1 --2- 172.21.15.107:0/2544831598 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d2fcfdcfd0 0x55d2fcfdf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.844+0000 7f2f2b845740 4 client.16038 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:18:11.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.844+0000 7f2f2b845740 1 -- 172.21.15.107:0/2544831598 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2fcf33800 con 0x55d2fcfdcfd0 2024-04-25T00:18:11.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.844+0000 7f2f21d9b640 1 --2- 172.21.15.107:0/2544831598 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d2fcfdcfd0 0x55d2fcfdf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.845+0000 7f2f21d9b640 1 --2- 172.21.15.107:0/2544831598 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d2fcfdcfd0 0x55d2fcfdf450 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1339c60e9f0d0110 server_cookie=f7d9f62b2dfd3b72 in_seq=0 out_seq=0 2024-04-25T00:18:11.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.845+0000 7f2f09ffb640 10 client.16038 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:11.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.845+0000 7f2f09ffb640 1 -- 172.21.15.107:0/2544831598 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d2fcf33800 con 0x55d2fcfdcfd0 2024-04-25T00:18:11.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.845+0000 7f2f09ffb640 10 client.16038 handle_command_reply: tid=0 2024-04-25T00:18:11.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.845+0000 7f2f2b845740 1 client.16038 shutdown 2024-04-25T00:18:11.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.846+0000 7f2f2b845740 1 -- 172.21.15.107:0/2544831598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfc2290 msgr2=0x55d2fcfc9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:11.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.846+0000 7f2f2b845740 1 --2- 172.21.15.107:0/2544831598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfc2290 0x55d2fcfc9f20 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2f140bbfe0 tx=0x7f2f1408beb0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.846+0000 7f2f2b845740 1 -- 172.21.15.107:0/2544831598 shutdown_connections 2024-04-25T00:18:11.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.846+0000 7f2f2b845740 1 --2- 172.21.15.107:0/2544831598 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d2fcfdcfd0 0x55d2fcfdf450 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.846+0000 7f2f2b845740 1 --2- 172.21.15.107:0/2544831598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2fcfcd010 0x55d2fcfca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.846+0000 7f2f09ffb640 0 client.16038 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:11.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.846+0000 7f2f2b845740 1 --2- 172.21.15.107:0/2544831598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2fcfc3980 0x55d2fcfca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.846+0000 7f2f2b845740 1 --2- 172.21.15.107:0/2544831598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfc2290 0x55d2fcfc9f20 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.846+0000 7f2f2b845740 1 -- 172.21.15.107:0/2544831598 >> 172.21.15.107:0/2544831598 conn(0x55d2fcf8a950 msgr2=0x55d2fcfc3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:11.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.846+0000 7f2f2b845740 1 -- 172.21.15.107:0/2544831598 shutdown_connections 2024-04-25T00:18:11.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.846+0000 7f2f2b845740 1 -- 172.21.15.107:0/2544831598 wait complete. 2024-04-25T00:18:11.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.846+0000 7f2f2b845740 20 client.16038 trim_cache size 0 max 16384 2024-04-25T00:18:11.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.863+0000 7f2f2b845740 1 Processor -- start 2024-04-25T00:18:11.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.863+0000 7f2f2b845740 1 -- start start 2024-04-25T00:18:11.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.863+0000 7f2f2b845740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfc2290 0x55d2fcfd8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.863+0000 7f2f2b845740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2fcfc3980 0x55d2fcfc9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.864+0000 7f2f2b845740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2fcfcd010 0x55d2fcfbd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.864+0000 7f2f2b845740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2f241b14d0 con 0x55d2fcfc2290 2024-04-25T00:18:11.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.864+0000 7f2f2b845740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2f241aed40 con 0x55d2fcfcd010 2024-04-25T00:18:11.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.864+0000 7f2f2b845740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2f241acb90 con 0x55d2fcfc3980 2024-04-25T00:18:11.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.864+0000 7f2f2259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfc2290 0x55d2fcfd8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.864+0000 7f2f2259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfc2290 0x55d2fcfd8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34026/0 (socket says 172.21.15.107:34026) 2024-04-25T00:18:11.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.864+0000 7f2f2259c640 1 -- 172.21.15.107:0/536127935 learned_addr learned my addr 172.21.15.107:0/536127935 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:11.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.864+0000 7f2f22d9d640 1 --2- 172.21.15.107:0/536127935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2fcfcd010 0x55d2fcfbd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.864+0000 7f2f2259c640 1 -- 172.21.15.107:0/536127935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2fcfc3980 msgr2=0x55d2fcfc9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:11.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.864+0000 7f2f2259c640 1 --2- 172.21.15.107:0/536127935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2fcfc3980 0x55d2fcfc9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.864+0000 7f2f2259c640 1 -- 172.21.15.107:0/536127935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2fcfcd010 msgr2=0x55d2fcfbd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:11.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.864+0000 7f2f2259c640 1 --2- 172.21.15.107:0/536127935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2fcfcd010 0x55d2fcfbd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.864+0000 7f2f2259c640 1 -- 172.21.15.107:0/536127935 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2fcbfa170 con 0x55d2fcfc2290 2024-04-25T00:18:11.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.864+0000 7f2f2259c640 1 --2- 172.21.15.107:0/536127935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfc2290 0x55d2fcfd8d70 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2f140c05c0 tx=0x7f2f1403b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01e5ad0f142fbda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:11.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.864+0000 7f2f0bfff640 1 -- 172.21.15.107:0/536127935 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f140c00d0 con 0x55d2fcfc2290 2024-04-25T00:18:11.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.864+0000 7f2f0bfff640 1 -- 172.21.15.107:0/536127935 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f1408b070 con 0x55d2fcfc2290 2024-04-25T00:18:11.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.864+0000 7f2f0bfff640 1 -- 172.21.15.107:0/536127935 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f140b86c0 con 0x55d2fcfc2290 2024-04-25T00:18:11.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.865+0000 7f2f2b845740 1 -- 172.21.15.107:0/536127935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfc2290 msgr2=0x55d2fcfd8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:11.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.865+0000 7f2f2b845740 1 --2- 172.21.15.107:0/536127935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfc2290 0x55d2fcfd8d70 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f2f140c05c0 tx=0x7f2f1403b790 comp rx=0 tx=0).stop 2024-04-25T00:18:11.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.865+0000 7f2f2b845740 1 -- 172.21.15.107:0/536127935 shutdown_connections 2024-04-25T00:18:11.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.865+0000 7f2f2b845740 1 --2- 172.21.15.107:0/536127935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2fcfcd010 0x55d2fcfbd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.865+0000 7f2f2b845740 1 --2- 172.21.15.107:0/536127935 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2fcfc3980 0x55d2fcfc9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.865+0000 7f2f2b845740 1 --2- 172.21.15.107:0/536127935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfc2290 0x55d2fcfd8d70 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.865+0000 7f2f2b845740 1 -- 172.21.15.107:0/536127935 >> 172.21.15.107:0/536127935 conn(0x55d2fcf8a950 msgr2=0x55d2fcfbd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:11.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.865+0000 7f2f2b845740 1 -- 172.21.15.107:0/536127935 shutdown_connections 2024-04-25T00:18:11.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.865+0000 7f2f2b845740 1 -- 172.21.15.107:0/536127935 wait complete. 2024-04-25T00:18:11.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.865+0000 7f2f2b845740 1 -- start start 2024-04-25T00:18:11.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.865+0000 7f2f2b845740 1 Processor -- start 2024-04-25T00:18:11.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.865+0000 7f2f2b845740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2fcfc2290 0x55d2fcfcdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.865+0000 7f2f2b845740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2fcfc3980 0x55d2fcfce290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.865+0000 7f2f2b845740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfcd010 0x55d2fcfce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.865+0000 7f2f2b845740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2f241ad200 con 0x55d2fcfcd010 2024-04-25T00:18:11.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.865+0000 7f2f2b845740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2f241ae670 con 0x55d2fcfc3980 2024-04-25T00:18:11.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.865+0000 7f2f2b845740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2f241ad8d0 con 0x55d2fcfc2290 2024-04-25T00:18:11.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.865+0000 7f2f2b845740 10 client.0 _ll_register_callbacks cb 0x7f2f140bbed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:11.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.865+0000 7f2f22d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfcd010 0x55d2fcfce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.865+0000 7f2f22d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfcd010 0x55d2fcfce7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34034/0 (socket says 172.21.15.107:34034) 2024-04-25T00:18:11.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.865+0000 7f2f22d9d640 1 -- 172.21.15.107:0/1089589416 learned_addr learned my addr 172.21.15.107:0/1089589416 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:11.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.866+0000 7f2f21d9b640 1 --2- 172.21.15.107:0/1089589416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2fcfc3980 0x55d2fcfce290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.866+0000 7f2f2259c640 1 --2- 172.21.15.107:0/1089589416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2fcfc2290 0x55d2fcfcdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.866+0000 7f2f22d9d640 1 -- 172.21.15.107:0/1089589416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2fcfc2290 msgr2=0x55d2fcfcdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:11.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.866+0000 7f2f22d9d640 1 --2- 172.21.15.107:0/1089589416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2fcfc2290 0x55d2fcfcdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.866+0000 7f2f22d9d640 1 -- 172.21.15.107:0/1089589416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2fcfc3980 msgr2=0x55d2fcfce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:11.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.866+0000 7f2f22d9d640 1 --2- 172.21.15.107:0/1089589416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2fcfc3980 0x55d2fcfce290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.866+0000 7f2f22d9d640 1 -- 172.21.15.107:0/1089589416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2fcfd3970 con 0x55d2fcfcd010 2024-04-25T00:18:11.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.866+0000 7f2f22d9d640 1 --2- 172.21.15.107:0/1089589416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfcd010 0x55d2fcfce7d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c022c70 tx=0x7f2f1c00a4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e353cdf518a1a323 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:11.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.866+0000 7f2f09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:18:11.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.866+0000 7f2f09ffb640 1 -- 172.21.15.107:0/1089589416 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c032030 con 0x55d2fcfcd010 2024-04-25T00:18:11.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.866+0000 7f2f09ffb640 1 -- 172.21.15.107:0/1089589416 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f2f1c01ac50 con 0x55d2fcfcd010 2024-04-25T00:18:11.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.866+0000 7f2f2b845740 1 -- 172.21.15.107:0/1089589416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f1408b070 con 0x55d2fcfcd010 2024-04-25T00:18:11.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.866+0000 7f2f09ffb640 1 -- 172.21.15.107:0/1089589416 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2f1c03f050 con 0x55d2fcfcd010 2024-04-25T00:18:11.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.866+0000 7f2f2b845740 10 client.16044 fetch_fsmap learned FSMap version 26 2024-04-25T00:18:11.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.866+0000 7f2f2b845740 1 -- 172.21.15.107:0/1089589416 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d2fd04c3c0 con 0x55d2fcfcd010 2024-04-25T00:18:11.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.866+0000 7f2f09ffb640 1 -- 172.21.15.107:0/1089589416 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f1408b070 con 0x55d2fcfcd010 2024-04-25T00:18:11.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.866+0000 7f2f09ffb640 1 -- 172.21.15.107:0/1089589416 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f2f1c0394d0 con 0x55d2fcfcd010 2024-04-25T00:18:11.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.867+0000 7f2f2b845740 10 client.16044 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:18:11.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.867+0000 7f2f2b845740 10 client.16044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:18:11.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.867+0000 7f2f2b845740 20 client.16044 populate_metadata read hostname 'smithi107' 2024-04-25T00:18:11.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.867+0000 7f2f2b845740 1 --2- 172.21.15.107:0/1089589416 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d2fd04c8b0 0x55d2fd04eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.867+0000 7f2f2b845740 4 client.16044 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:18:11.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.867+0000 7f2f2b845740 1 -- 172.21.15.107:0/1089589416 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d2fcf2db50 con 0x55d2fd04c8b0 2024-04-25T00:18:11.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.867+0000 7f2f2259c640 1 --2- 172.21.15.107:0/1089589416 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d2fd04c8b0 0x55d2fd04eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.867+0000 7f2f09ffb640 10 client.16044 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:11.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.867+0000 7f2f2259c640 1 --2- 172.21.15.107:0/1089589416 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d2fd04c8b0 0x55d2fd04eca0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8387f19615ede9d2 server_cookie=4714e1fbbbae54b5 in_seq=0 out_seq=0 2024-04-25T00:18:11.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.868+0000 7f2f09ffb640 1 -- 172.21.15.107:0/1089589416 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d2fcf2db50 con 0x55d2fd04c8b0 2024-04-25T00:18:11.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.868+0000 7f2f09ffb640 10 client.16044 handle_command_reply: tid=0 2024-04-25T00:18:11.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.868+0000 7f2f2b845740 1 client.16044 shutdown 2024-04-25T00:18:11.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.869+0000 7f2f2b845740 1 -- 172.21.15.107:0/1089589416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfcd010 msgr2=0x55d2fcfce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:11.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.869+0000 7f2f2b845740 1 --2- 172.21.15.107:0/1089589416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfcd010 0x55d2fcfce7d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c022c70 tx=0x7f2f1c00a4a0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.869+0000 7f2f2b845740 1 -- 172.21.15.107:0/1089589416 shutdown_connections 2024-04-25T00:18:11.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.869+0000 7f2f2b845740 1 --2- 172.21.15.107:0/1089589416 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d2fd04c8b0 0x55d2fd04eca0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.869+0000 7f2f2b845740 1 --2- 172.21.15.107:0/1089589416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d2fcfcd010 0x55d2fcfce7d0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.869+0000 7f2f09ffb640 0 client.16044 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:11.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.869+0000 7f2f2b845740 1 --2- 172.21.15.107:0/1089589416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2fcfc3980 0x55d2fcfce290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.869+0000 7f2f2b845740 1 --2- 172.21.15.107:0/1089589416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d2fcfc2290 0x55d2fcfcdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.869+0000 7f2f2b845740 1 -- 172.21.15.107:0/1089589416 >> 172.21.15.107:0/1089589416 conn(0x55d2fcf8a950 msgr2=0x55d2fcfbd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:11.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.869+0000 7f2f2b845740 1 -- 172.21.15.107:0/1089589416 shutdown_connections 2024-04-25T00:18:11.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.869+0000 7f2f2b845740 1 -- 172.21.15.107:0/1089589416 wait complete. 2024-04-25T00:18:11.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.869+0000 7f2f2b845740 20 client.16044 trim_cache size 0 max 16384 2024-04-25T00:18:11.876 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:18:11.876 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-04-25T00:18:11.876 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "eb920132-669d-4a0e-8f0d-d40aa4a625db", 2024-04-25T00:18:11.876 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-04-25T00:18:11.876 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:18:11.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.869+0000 7f2f28b41640 1 -- 172.21.15.107:0/3389563329 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f2f0c0a2d00 msgr2=0x7f2f0c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:11.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.869+0000 7f2f28b41640 1 --2- 172.21.15.107:0/3389563329 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f2f0c0a2d00 0x7f2f0c0a51c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c017c00 tx=0x7f2f1c01b000 comp rx=0 tx=0).stop 2024-04-25T00:18:11.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.869+0000 7f2f28b41640 1 -- 172.21.15.107:0/3389563329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f24069d40 msgr2=0x7f2f24198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:11.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.869+0000 7f2f28b41640 1 --2- 172.21.15.107:0/3389563329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f24069d40 0x7f2f24198060 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2f14006010 tx=0x7f2f14003c10 comp rx=0 tx=0).stop 2024-04-25T00:18:11.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.869+0000 7f2f28b41640 1 -- 172.21.15.107:0/3389563329 shutdown_connections 2024-04-25T00:18:11.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.869+0000 7f2f28b41640 1 --2- 172.21.15.107:0/3389563329 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f2f0c0a2d00 0x7f2f0c0a51c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.869+0000 7f2f28b41640 1 --2- 172.21.15.107:0/3389563329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2f24107740 0x7f2f2419c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.869+0000 7f2f28b41640 1 --2- 172.21.15.107:0/3389563329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2f2406a6e0 0x7f2f241985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.869+0000 7f2f28b41640 1 --2- 172.21.15.107:0/3389563329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f24069d40 0x7f2f24198060 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.869+0000 7f2f28b41640 1 -- 172.21.15.107:0/3389563329 >> 172.21.15.107:0/3389563329 conn(0x7f2f24076440 msgr2=0x7f2f240fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:11.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.869+0000 7f2f28b41640 1 -- 172.21.15.107:0/3389563329 shutdown_connections 2024-04-25T00:18:11.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.869+0000 7f2f28b41640 1 -- 172.21.15.107:0/3389563329 wait complete. 2024-04-25T00:18:11.880 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:18:11.880 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:18:11.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.983+0000 7fdb73e66640 1 -- 172.21.15.107:0/501805029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb6c100620 msgr2=0x7fdb6c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:11.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.983+0000 7fdb73e66640 1 --2- 172.21.15.107:0/501805029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb6c100620 0x7fdb6c100aa0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009930 tx=0x7fdb6802a420 comp rx=0 tx=0).stop 2024-04-25T00:18:11.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.983+0000 7fdb73e66640 1 -- 172.21.15.107:0/501805029 shutdown_connections 2024-04-25T00:18:11.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.983+0000 7fdb73e66640 1 --2- 172.21.15.107:0/501805029 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb6c100fe0 0x7fdb6c10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.983+0000 7fdb73e66640 1 --2- 172.21.15.107:0/501805029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb6c100620 0x7fdb6c100aa0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.983+0000 7fdb73e66640 1 --2- 172.21.15.107:0/501805029 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb6c106630 0x7fdb6c106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.983+0000 7fdb73e66640 1 -- 172.21.15.107:0/501805029 >> 172.21.15.107:0/501805029 conn(0x7fdb6c0776a0 msgr2=0x7fdb6c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:11.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.983+0000 7fdb73e66640 1 -- 172.21.15.107:0/501805029 shutdown_connections 2024-04-25T00:18:11.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.983+0000 7fdb73e66640 1 -- 172.21.15.107:0/501805029 wait complete. 2024-04-25T00:18:11.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.984+0000 7fdb73e66640 1 Processor -- start 2024-04-25T00:18:11.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.984+0000 7fdb73e66640 1 -- start start 2024-04-25T00:18:11.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.984+0000 7fdb73e66640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb6c100620 0x7fdb6c196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.984+0000 7fdb73e66640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb6c100fe0 0x7fdb6c197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.984+0000 7fdb73e66640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb6c106630 0x7fdb6c19b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.984+0000 7fdb73e66640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c110a40 con 0x7fdb6c106630 2024-04-25T00:18:11.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.984+0000 7fdb73e66640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c1108c0 con 0x7fdb6c100620 2024-04-25T00:18:11.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.984+0000 7fdb73e66640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdb6c110bc0 con 0x7fdb6c100fe0 2024-04-25T00:18:11.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.984+0000 7fdb72403640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb6c106630 0x7fdb6c19b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.984+0000 7fdb72403640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb6c106630 0x7fdb6c19b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34046/0 (socket says 172.21.15.107:34046) 2024-04-25T00:18:11.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.984+0000 7fdb72403640 1 -- 172.21.15.107:0/2373876417 learned_addr learned my addr 172.21.15.107:0/2373876417 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:11.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.984+0000 7fdb71c02640 1 --2- 172.21.15.107:0/2373876417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb6c100620 0x7fdb6c196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.984+0000 7fdb71401640 1 --2- 172.21.15.107:0/2373876417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb6c100fe0 0x7fdb6c197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.984+0000 7fdb72403640 1 -- 172.21.15.107:0/2373876417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb6c100fe0 msgr2=0x7fdb6c197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:11.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.984+0000 7fdb72403640 1 --2- 172.21.15.107:0/2373876417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb6c100fe0 0x7fdb6c197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.984+0000 7fdb72403640 1 -- 172.21.15.107:0/2373876417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb6c100620 msgr2=0x7fdb6c196f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:11.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.984+0000 7fdb72403640 1 --2- 172.21.15.107:0/2373876417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb6c100620 0x7fdb6c196f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.985+0000 7fdb72403640 1 -- 172.21.15.107:0/2373876417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb6c19bf10 con 0x7fdb6c106630 2024-04-25T00:18:11.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.985+0000 7fdb72403640 1 --2- 172.21.15.107:0/2373876417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb6c106630 0x7fdb6c19b7e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400eb10 tx=0x7fdb6400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:11.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.985+0000 7fdb62ffd640 1 -- 172.21.15.107:0/2373876417 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb64004b20 con 0x7fdb6c106630 2024-04-25T00:18:11.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.985+0000 7fdb73e66640 1 -- 172.21.15.107:0/2373876417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb6c19c200 con 0x7fdb6c106630 2024-04-25T00:18:11.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.985+0000 7fdb73e66640 1 -- 172.21.15.107:0/2373876417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb6c1a3a90 con 0x7fdb6c106630 2024-04-25T00:18:11.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.985+0000 7fdb62ffd640 1 -- 172.21.15.107:0/2373876417 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb64004cc0 con 0x7fdb6c106630 2024-04-25T00:18:11.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.985+0000 7fdb62ffd640 1 -- 172.21.15.107:0/2373876417 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb640216a0 con 0x7fdb6c106630 2024-04-25T00:18:11.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.987+0000 7fdb62ffd640 1 -- 172.21.15.107:0/2373876417 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fdb64021840 con 0x7fdb6c106630 2024-04-25T00:18:11.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.988+0000 7fdb62ffd640 1 --2- 172.21.15.107:0/2373876417 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdb580a2d00 0x7fdb580a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.988+0000 7fdb76b70740 1 Processor -- start 2024-04-25T00:18:11.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.988+0000 7fdb62ffd640 1 -- 172.21.15.107:0/2373876417 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fdb640cfe20 con 0x7fdb6c106630 2024-04-25T00:18:11.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.988+0000 7fdb71c02640 1 --2- 172.21.15.107:0/2373876417 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdb580a2d00 0x7fdb580a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.988+0000 7fdb76b70740 1 -- start start 2024-04-25T00:18:11.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.989+0000 7fdb76b70740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f35d45630 0x564f35d45a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.989+0000 7fdb76b70740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f35d45f50 0x564f35d522f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.989+0000 7fdb76b70740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d52830 0x564f35d54c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:11.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.989+0000 7fdb76b70740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564f35caa6b0 con 0x564f35d52830 2024-04-25T00:18:11.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.989+0000 7fdb76b70740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564f35c52880 con 0x564f35d45630 2024-04-25T00:18:11.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.989+0000 7fdb76b70740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564f35ca6670 con 0x564f35d45f50 2024-04-25T00:18:11.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.989+0000 7fdb71c02640 1 --2- 172.21.15.107:0/2373876417 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdb580a2d00 0x7fdb580a51c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c005fd0 tx=0x7fdb5c00b390 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:11.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.989+0000 7fdb72403640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d52830 0x564f35d54c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.989+0000 7fdb72403640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d52830 0x564f35d54c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34054/0 (socket says 172.21.15.107:34054) 2024-04-25T00:18:11.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.989+0000 7fdb71c02640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f35d45630 0x564f35d45a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.989+0000 7fdb72403640 1 -- 172.21.15.107:0/3353237069 learned_addr learned my addr 172.21.15.107:0/3353237069 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:11.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.989+0000 7fdb71401640 1 --2- 172.21.15.107:0/3353237069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f35d45f50 0x564f35d522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:11.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.989+0000 7fdb72403640 1 -- 172.21.15.107:0/3353237069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f35d45f50 msgr2=0x564f35d522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:11.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.989+0000 7fdb72403640 1 --2- 172.21.15.107:0/3353237069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f35d45f50 0x564f35d522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.989+0000 7fdb72403640 1 -- 172.21.15.107:0/3353237069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f35d45630 msgr2=0x564f35d45a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:11.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.989+0000 7fdb72403640 1 --2- 172.21.15.107:0/3353237069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f35d45630 0x564f35d45a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.989+0000 7fdb72403640 1 -- 172.21.15.107:0/3353237069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f35a887d0 con 0x564f35d52830 2024-04-25T00:18:11.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.989+0000 7fdb71c02640 1 --2- 172.21.15.107:0/3353237069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f35d45630 0x564f35d45a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:11.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.989+0000 7fdb72403640 1 --2- 172.21.15.107:0/3353237069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d52830 0x564f35d54c20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdb640a4710 tx=0x7fdb6409bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89593c8c5df0b92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:11.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.990+0000 7fdb57fff640 1 -- 172.21.15.107:0/3353237069 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6409bfd0 con 0x564f35d52830 2024-04-25T00:18:11.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.990+0000 7fdb57fff640 1 -- 172.21.15.107:0/3353237069 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb640a0380 con 0x564f35d52830 2024-04-25T00:18:11.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.990+0000 7fdb57fff640 1 -- 172.21.15.107:0/3353237069 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb640a47b0 con 0x564f35d52830 2024-04-25T00:18:11.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.990+0000 7fdb76b70740 1 -- 172.21.15.107:0/3353237069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d52830 msgr2=0x564f35d54c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:11.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.990+0000 7fdb76b70740 1 --2- 172.21.15.107:0/3353237069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d52830 0x564f35d54c20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdb640a4710 tx=0x7fdb6409bc10 comp rx=0 tx=0).stop 2024-04-25T00:18:11.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.990+0000 7fdb76b70740 1 -- 172.21.15.107:0/3353237069 shutdown_connections 2024-04-25T00:18:11.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.990+0000 7fdb76b70740 1 --2- 172.21.15.107:0/3353237069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d52830 0x564f35d54c20 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.990+0000 7fdb76b70740 1 --2- 172.21.15.107:0/3353237069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f35d45f50 0x564f35d522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.990+0000 7fdb76b70740 1 --2- 172.21.15.107:0/3353237069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f35d45630 0x564f35d45a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:11.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.990+0000 7fdb76b70740 1 -- 172.21.15.107:0/3353237069 >> 172.21.15.107:0/3353237069 conn(0x564f35d41950 msgr2=0x564f35d41d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:11.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.991+0000 7fdb76b70740 1 -- 172.21.15.107:0/3353237069 shutdown_connections 2024-04-25T00:18:11.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.991+0000 7fdb76b70740 1 -- 172.21.15.107:0/3353237069 wait complete. 2024-04-25T00:18:11.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.991+0000 7fdb76b70740 1 -- start start 2024-04-25T00:18:11.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.991+0000 7fdb76b70740 1 Processor -- start 2024-04-25T00:18:11.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.991+0000 7fdb76b70740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f35d45630 0x564f35d567a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:12.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.991+0000 7fdb76b70740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f35d45f50 0x564f35d56ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:12.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.991+0000 7fdb76b70740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d52830 0x564f35d57220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:12.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.991+0000 7fdb76b70740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564f35d4e4a0 con 0x564f35d52830 2024-04-25T00:18:12.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.991+0000 7fdb76b70740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564f35c65f50 con 0x564f35d45630 2024-04-25T00:18:12.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.991+0000 7fdb76b70740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564f35d4e620 con 0x564f35d45f50 2024-04-25T00:18:12.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.991+0000 7fdb76b70740 10 client.0 _ll_register_callbacks cb 0x564f35ca1000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:12.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.991+0000 7fdb72403640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d52830 0x564f35d57220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:12.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.991+0000 7fdb72403640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d52830 0x564f35d57220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34066/0 (socket says 172.21.15.107:34066) 2024-04-25T00:18:12.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.992+0000 7fdb72403640 1 -- 172.21.15.107:0/1953863956 learned_addr learned my addr 172.21.15.107:0/1953863956 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:12.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.992+0000 7fdb71c02640 1 --2- 172.21.15.107:0/1953863956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f35d45630 0x564f35d567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:12.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.992+0000 7fdb71401640 1 --2- 172.21.15.107:0/1953863956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f35d45f50 0x564f35d56ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:12.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.992+0000 7fdb72403640 1 -- 172.21.15.107:0/1953863956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f35d45f50 msgr2=0x564f35d56ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:12.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.992+0000 7fdb72403640 1 --2- 172.21.15.107:0/1953863956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f35d45f50 0x564f35d56ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.992+0000 7fdb72403640 1 -- 172.21.15.107:0/1953863956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f35d45630 msgr2=0x564f35d567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:12.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.992+0000 7fdb72403640 1 --2- 172.21.15.107:0/1953863956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f35d45630 0x564f35d567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.992+0000 7fdb72403640 1 -- 172.21.15.107:0/1953863956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f359be3f0 con 0x564f35d52830 2024-04-25T00:18:12.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.992+0000 7fdb71c02640 1 --2- 172.21.15.107:0/1953863956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f35d45630 0x564f35d567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:12.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.992+0000 7fdb72403640 1 --2- 172.21.15.107:0/1953863956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d52830 0x564f35d57220 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fdb640049d0 tx=0x7fdb640a4cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1fb7d7183e7113f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:12.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.992+0000 7fdb557fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:18:12.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.992+0000 7fdb557fa640 1 -- 172.21.15.107:0/1953863956 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb640986b0 con 0x564f35d52830 2024-04-25T00:18:12.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.992+0000 7fdb557fa640 1 -- 172.21.15.107:0/1953863956 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fdb640a47b0 con 0x564f35d52830 2024-04-25T00:18:12.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.992+0000 7fdb76b70740 1 -- 172.21.15.107:0/1953863956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb640a0380 con 0x564f35d52830 2024-04-25T00:18:12.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.992+0000 7fdb557fa640 1 -- 172.21.15.107:0/1953863956 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdb640a49b0 con 0x564f35d52830 2024-04-25T00:18:12.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.992+0000 7fdb76b70740 10 client.16056 fetch_fsmap learned FSMap version 26 2024-04-25T00:18:12.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.992+0000 7fdb76b70740 1 -- 172.21.15.107:0/1953863956 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564f3597e0f0 con 0x564f35d52830 2024-04-25T00:18:12.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.992+0000 7fdb557fa640 1 -- 172.21.15.107:0/1953863956 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb640a0380 con 0x564f35d52830 2024-04-25T00:18:12.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.993+0000 7fdb557fa640 1 -- 172.21.15.107:0/1953863956 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fdb6409ae90 con 0x564f35d52830 2024-04-25T00:18:12.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.993+0000 7fdb76b70740 10 client.16056 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:18:12.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.993+0000 7fdb76b70740 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:18:12.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.993+0000 7fdb76b70740 20 client.16056 populate_metadata read hostname 'smithi107' 2024-04-25T00:18:12.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.993+0000 7fdb76b70740 1 --2- 172.21.15.107:0/1953863956 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x564f35d68f30 0x564f35d6b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:12.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.993+0000 7fdb76b70740 4 client.16056 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:18:12.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.993+0000 7fdb76b70740 1 -- 172.21.15.107:0/1953863956 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f35cb7bf0 con 0x564f35d68f30 2024-04-25T00:18:12.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.993+0000 7fdb71c02640 1 --2- 172.21.15.107:0/1953863956 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x564f35d68f30 0x564f35d6b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:12.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.993+0000 7fdb557fa640 10 client.16056 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:12.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.993+0000 7fdb71c02640 1 --2- 172.21.15.107:0/1953863956 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x564f35d68f30 0x564f35d6b320 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4ce02ea2f0ea488 server_cookie=da760b310ddcbfbd in_seq=0 out_seq=0 2024-04-25T00:18:12.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.994+0000 7fdb557fa640 1 -- 172.21.15.107:0/1953863956 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f35cb7bf0 con 0x564f35d68f30 2024-04-25T00:18:12.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.994+0000 7fdb557fa640 10 client.16056 handle_command_reply: tid=0 2024-04-25T00:18:12.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.994+0000 7fdb76b70740 1 client.16056 shutdown 2024-04-25T00:18:12.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.994+0000 7fdb76b70740 1 -- 172.21.15.107:0/1953863956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d52830 msgr2=0x564f35d57220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:12.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.994+0000 7fdb76b70740 1 --2- 172.21.15.107:0/1953863956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d52830 0x564f35d57220 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fdb640049d0 tx=0x7fdb640a4cc0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.994+0000 7fdb76b70740 1 -- 172.21.15.107:0/1953863956 shutdown_connections 2024-04-25T00:18:12.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.994+0000 7fdb76b70740 1 --2- 172.21.15.107:0/1953863956 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x564f35d68f30 0x564f35d6b320 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.994+0000 7fdb76b70740 1 --2- 172.21.15.107:0/1953863956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d52830 0x564f35d57220 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.994+0000 7fdb557fa640 0 client.16056 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:12.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.994+0000 7fdb76b70740 1 --2- 172.21.15.107:0/1953863956 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f35d45f50 0x564f35d56ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.994+0000 7fdb76b70740 1 --2- 172.21.15.107:0/1953863956 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f35d45630 0x564f35d567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.994+0000 7fdb76b70740 1 -- 172.21.15.107:0/1953863956 >> 172.21.15.107:0/1953863956 conn(0x564f35d41950 msgr2=0x564f35d47650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:12.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.994+0000 7fdb76b70740 1 -- 172.21.15.107:0/1953863956 shutdown_connections 2024-04-25T00:18:12.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.994+0000 7fdb76b70740 1 -- 172.21.15.107:0/1953863956 wait complete. 2024-04-25T00:18:12.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:11.994+0000 7fdb76b70740 20 client.16056 trim_cache size 0 max 16384 2024-04-25T00:18:12.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.012+0000 7fdb76b70740 1 Processor -- start 2024-04-25T00:18:12.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.012+0000 7fdb76b70740 1 -- start start 2024-04-25T00:18:12.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.012+0000 7fdb76b70740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f35d45630 0x564f35d58ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:12.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.012+0000 7fdb76b70740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f35d45f50 0x564f35d41d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:12.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.012+0000 7fdb76b70740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d52830 0x564f35d56e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:12.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.012+0000 7fdb76b70740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb500126c0 con 0x564f35d52830 2024-04-25T00:18:12.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.012+0000 7fdb76b70740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5000ff30 con 0x564f35d45f50 2024-04-25T00:18:12.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.012+0000 7fdb76b70740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5000dd80 con 0x564f35d45630 2024-04-25T00:18:12.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.012+0000 7fdb72403640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d52830 0x564f35d56e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:12.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.012+0000 7fdb72403640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d52830 0x564f35d56e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34068/0 (socket says 172.21.15.107:34068) 2024-04-25T00:18:12.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.012+0000 7fdb72403640 1 -- 172.21.15.107:0/3622769874 learned_addr learned my addr 172.21.15.107:0/3622769874 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:12.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.012+0000 7fdb71401640 1 --2- 172.21.15.107:0/3622769874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f35d45f50 0x564f35d41d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:12.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.012+0000 7fdb71c02640 1 --2- 172.21.15.107:0/3622769874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f35d45630 0x564f35d58ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:12.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.012+0000 7fdb72403640 1 -- 172.21.15.107:0/3622769874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f35d45630 msgr2=0x564f35d58ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:12.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.012+0000 7fdb72403640 1 --2- 172.21.15.107:0/3622769874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f35d45630 0x564f35d58ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.012+0000 7fdb72403640 1 -- 172.21.15.107:0/3622769874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f35d45f50 msgr2=0x564f35d41d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:12.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.012+0000 7fdb72403640 1 --2- 172.21.15.107:0/3622769874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f35d45f50 0x564f35d41d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.012+0000 7fdb72403640 1 -- 172.21.15.107:0/3622769874 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f35d33090 con 0x564f35d52830 2024-04-25T00:18:12.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.012+0000 7fdb72403640 1 --2- 172.21.15.107:0/3622769874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d52830 0x564f35d56e70 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fdb6409c190 tx=0x7fdb64009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b967870f9230ecd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:12.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.013+0000 7fdb577fe640 1 -- 172.21.15.107:0/3622769874 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb640a0380 con 0x564f35d52830 2024-04-25T00:18:12.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.013+0000 7fdb577fe640 1 -- 172.21.15.107:0/3622769874 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb640a47b0 con 0x564f35d52830 2024-04-25T00:18:12.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.013+0000 7fdb577fe640 1 -- 172.21.15.107:0/3622769874 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb64098640 con 0x564f35d52830 2024-04-25T00:18:12.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.013+0000 7fdb76b70740 1 -- 172.21.15.107:0/3622769874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d52830 msgr2=0x564f35d56e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:12.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.013+0000 7fdb76b70740 1 --2- 172.21.15.107:0/3622769874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d52830 0x564f35d56e70 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fdb6409c190 tx=0x7fdb64009a90 comp rx=0 tx=0).stop 2024-04-25T00:18:12.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.013+0000 7fdb76b70740 1 -- 172.21.15.107:0/3622769874 shutdown_connections 2024-04-25T00:18:12.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.013+0000 7fdb76b70740 1 --2- 172.21.15.107:0/3622769874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d52830 0x564f35d56e70 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.013+0000 7fdb76b70740 1 --2- 172.21.15.107:0/3622769874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f35d45f50 0x564f35d41d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.013+0000 7fdb76b70740 1 --2- 172.21.15.107:0/3622769874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f35d45630 0x564f35d58ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.013+0000 7fdb76b70740 1 -- 172.21.15.107:0/3622769874 >> 172.21.15.107:0/3622769874 conn(0x564f35d41950 msgr2=0x564f35d56760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:12.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.013+0000 7fdb76b70740 1 -- 172.21.15.107:0/3622769874 shutdown_connections 2024-04-25T00:18:12.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.013+0000 7fdb76b70740 1 -- 172.21.15.107:0/3622769874 wait complete. 2024-04-25T00:18:12.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.013+0000 7fdb76b70740 1 -- start start 2024-04-25T00:18:12.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.013+0000 7fdb76b70740 1 Processor -- start 2024-04-25T00:18:12.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.014+0000 7fdb76b70740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f35d45630 0x564f35d50e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:12.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.014+0000 7fdb76b70740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d45f50 0x564f35d51380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:12.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.014+0000 7fdb76b70740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f35d52830 0x564f35dcdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:12.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.014+0000 7fdb76b70740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5000e3f0 con 0x564f35d45f50 2024-04-25T00:18:12.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.014+0000 7fdb76b70740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5000f860 con 0x564f35d52830 2024-04-25T00:18:12.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.014+0000 7fdb76b70740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5000eac0 con 0x564f35d45630 2024-04-25T00:18:12.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.014+0000 7fdb76b70740 10 client.0 _ll_register_callbacks cb 0x7fdb640093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:12.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.014+0000 7fdb71401640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d45f50 0x564f35d51380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:12.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.014+0000 7fdb72403640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f35d52830 0x564f35dcdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.014+0000 7fdb71401640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d45f50 0x564f35d51380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34070/0 (socket says 172.21.15.107:34070) 2024-04-25T00:18:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.014+0000 7fdb71401640 1 -- 172.21.15.107:0/56736408 learned_addr learned my addr 172.21.15.107:0/56736408 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.014+0000 7fdb71c02640 1 --2- 172.21.15.107:0/56736408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f35d45630 0x564f35d50e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.014+0000 7fdb71401640 1 -- 172.21.15.107:0/56736408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f35d45630 msgr2=0x564f35d50e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.014+0000 7fdb71401640 1 --2- 172.21.15.107:0/56736408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f35d45630 0x564f35d50e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.014+0000 7fdb71401640 1 -- 172.21.15.107:0/56736408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f35d52830 msgr2=0x564f35dcdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.014+0000 7fdb71401640 1 --2- 172.21.15.107:0/56736408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f35d52830 0x564f35dcdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.014+0000 7fdb71401640 1 -- 172.21.15.107:0/56736408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f35d51c50 con 0x564f35d45f50 2024-04-25T00:18:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.014+0000 7fdb71c02640 1 --2- 172.21.15.107:0/56736408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f35d45630 0x564f35d50e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:18:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.014+0000 7fdb72403640 1 --2- 172.21.15.107:0/56736408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f35d52830 0x564f35dcdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.014+0000 7fdb71401640 1 --2- 172.21.15.107:0/56736408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d45f50 0x564f35d51380 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fdb68005c40 tx=0x7fdb680317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b218d63631651b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.014+0000 7fdb557fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:18:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.015+0000 7fdb557fa640 1 -- 172.21.15.107:0/56736408 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68035040 con 0x564f35d45f50 2024-04-25T00:18:12.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.015+0000 7fdb557fa640 1 -- 172.21.15.107:0/56736408 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fdb68004240 con 0x564f35d45f50 2024-04-25T00:18:12.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.015+0000 7fdb76b70740 1 -- 172.21.15.107:0/56736408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb640a47b0 con 0x564f35d45f50 2024-04-25T00:18:12.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.015+0000 7fdb557fa640 1 -- 172.21.15.107:0/56736408 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdb68050050 con 0x564f35d45f50 2024-04-25T00:18:12.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.015+0000 7fdb76b70740 10 client.16062 fetch_fsmap learned FSMap version 26 2024-04-25T00:18:12.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.015+0000 7fdb76b70740 1 -- 172.21.15.107:0/56736408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564f35dcf320 con 0x564f35d45f50 2024-04-25T00:18:12.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.015+0000 7fdb557fa640 1 -- 172.21.15.107:0/56736408 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb640a47b0 con 0x564f35d45f50 2024-04-25T00:18:12.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.015+0000 7fdb557fa640 1 -- 172.21.15.107:0/56736408 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fdb68004a50 con 0x564f35d45f50 2024-04-25T00:18:12.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.015+0000 7fdb76b70740 10 client.16062 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T00:18:12.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.015+0000 7fdb76b70740 10 client.16062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:18:12.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.015+0000 7fdb76b70740 20 client.16062 populate_metadata read hostname 'smithi107' 2024-04-25T00:18:12.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.015+0000 7fdb76b70740 1 --2- 172.21.15.107:0/56736408 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x564f35dcfb30 0x564f35dd1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:12.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.015+0000 7fdb76b70740 4 client.16062 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:18:12.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.015+0000 7fdb76b70740 1 -- 172.21.15.107:0/56736408 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564f35cb1f40 con 0x564f35dcfb30 2024-04-25T00:18:12.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.015+0000 7fdb71c02640 1 --2- 172.21.15.107:0/56736408 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x564f35dcfb30 0x564f35dd1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.016+0000 7fdb71c02640 1 --2- 172.21.15.107:0/56736408 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x564f35dcfb30 0x564f35dd1f50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=349058468ee65118 server_cookie=2643ccf3c17a836f in_seq=0 out_seq=0 2024-04-25T00:18:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.016+0000 7fdb557fa640 10 client.16062 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.016+0000 7fdb557fa640 1 -- 172.21.15.107:0/56736408 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x564f35cb1f40 con 0x564f35dcfb30 2024-04-25T00:18:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.016+0000 7fdb557fa640 10 client.16062 handle_command_reply: tid=0 2024-04-25T00:18:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.016+0000 7fdb76b70740 1 client.16062 shutdown 2024-04-25T00:18:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.016+0000 7fdb76b70740 1 -- 172.21.15.107:0/56736408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d45f50 msgr2=0x564f35d51380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.016+0000 7fdb76b70740 1 --2- 172.21.15.107:0/56736408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d45f50 0x564f35d51380 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fdb68005c40 tx=0x7fdb680317e0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.016+0000 7fdb76b70740 1 -- 172.21.15.107:0/56736408 shutdown_connections 2024-04-25T00:18:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.016+0000 7fdb76b70740 1 --2- 172.21.15.107:0/56736408 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x564f35dcfb30 0x564f35dd1f50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.016+0000 7fdb76b70740 1 --2- 172.21.15.107:0/56736408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f35d52830 0x564f35dcdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.016+0000 7fdb76b70740 1 --2- 172.21.15.107:0/56736408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f35d45f50 0x564f35d51380 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.016+0000 7fdb76b70740 1 --2- 172.21.15.107:0/56736408 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f35d45630 0x564f35d50e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.016+0000 7fdb557fa640 0 client.16062 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.016+0000 7fdb76b70740 1 -- 172.21.15.107:0/56736408 >> 172.21.15.107:0/56736408 conn(0x564f35d41950 msgr2=0x564f35d4c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:12.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.016+0000 7fdb76b70740 1 -- 172.21.15.107:0/56736408 shutdown_connections 2024-04-25T00:18:12.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.016+0000 7fdb76b70740 1 -- 172.21.15.107:0/56736408 wait complete. 2024-04-25T00:18:12.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.016+0000 7fdb76b70740 20 client.16062 trim_cache size 0 max 16384 2024-04-25T00:18:12.025 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:18:12.025 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "scrub active (0 inodes in the stack)", 2024-04-25T00:18:12.025 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-04-25T00:18:12.025 INFO:teuthology.orchestra.run.smithi107.stdout: "eb920132-669d-4a0e-8f0d-d40aa4a625db": { 2024-04-25T00:18:12.025 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-04-25T00:18:12.025 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "eb920132-669d-4a0e-8f0d-d40aa4a625db", 2024-04-25T00:18:12.025 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-04-25T00:18:12.026 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:18:12.026 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:18:12.026 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:18:12.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.017+0000 7fdb73e66640 1 -- 172.21.15.107:0/2373876417 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdb580a2d00 msgr2=0x7fdb580a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:12.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.017+0000 7fdb73e66640 1 --2- 172.21.15.107:0/2373876417 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdb580a2d00 0x7fdb580a51c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c005fd0 tx=0x7fdb5c00b390 comp rx=0 tx=0).stop 2024-04-25T00:18:12.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.017+0000 7fdb73e66640 1 -- 172.21.15.107:0/2373876417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb6c106630 msgr2=0x7fdb6c19b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:12.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.017+0000 7fdb73e66640 1 --2- 172.21.15.107:0/2373876417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb6c106630 0x7fdb6c19b7e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400eb10 tx=0x7fdb6400c2d0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.017+0000 7fdb73e66640 1 -- 172.21.15.107:0/2373876417 shutdown_connections 2024-04-25T00:18:12.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.017+0000 7fdb73e66640 1 --2- 172.21.15.107:0/2373876417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb6c106630 0x7fdb6c19b7e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.017+0000 7fdb73e66640 1 --2- 172.21.15.107:0/2373876417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb6c100fe0 0x7fdb6c197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.017+0000 7fdb73e66640 1 --2- 172.21.15.107:0/2373876417 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdb580a2d00 0x7fdb580a51c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.017+0000 7fdb73e66640 1 --2- 172.21.15.107:0/2373876417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb6c100620 0x7fdb6c196f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:12.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.017+0000 7fdb73e66640 1 -- 172.21.15.107:0/2373876417 >> 172.21.15.107:0/2373876417 conn(0x7fdb6c0776a0 msgr2=0x7fdb6c1031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:12.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.017+0000 7fdb73e66640 1 -- 172.21.15.107:0/2373876417 shutdown_connections 2024-04-25T00:18:12.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:12.017+0000 7fdb73e66640 1 -- 172.21.15.107:0/2373876417 wait complete. 2024-04-25T00:18:12.029 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:11 smithi107 ceph-mon[29153]: pgmap v55: 129 pgs: 129 active+clean; 55 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 76 op/s 2024-04-25T00:18:12.029 INFO:tasks.cephfs.filesystem:scrub status for tag:eb920132-669d-4a0e-8f0d-d40aa4a625db - {'path': '/', 'tag': 'eb920132-669d-4a0e-8f0d-d40aa4a625db', 'options': 'recursive,force'} 2024-04-25T00:18:12.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:11 smithi195 ceph-mon[32833]: pgmap v55: 129 pgs: 129 active+clean; 55 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 76 op/s 2024-04-25T00:18:12.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:11 smithi170 ceph-mon[32807]: pgmap v55: 129 pgs: 129 active+clean; 55 MiB data, 576 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 76 op/s 2024-04-25T00:18:13.336 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:12 smithi170 ceph-mon[32807]: scrub queued for path: / 2024-04-25T00:18:13.336 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:12 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:13.336 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:12 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:13.336 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:12 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:13.336 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:12 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:13.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:12 smithi195 ceph-mon[32833]: scrub queued for path: / 2024-04-25T00:18:13.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:12 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:13.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:12 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:13.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:12 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:13.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:12 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:13.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:12 smithi107 ceph-mon[29153]: scrub queued for path: / 2024-04-25T00:18:13.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:12 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:13.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:12 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:13.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:12 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:13.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:12 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:14.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:13 smithi195 ceph-mon[32833]: pgmap v56: 129 pgs: 129 active+clean; 91 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 476 op/s 2024-04-25T00:18:14.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:13 smithi107 ceph-mon[29153]: pgmap v56: 129 pgs: 129 active+clean; 91 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 476 op/s 2024-04-25T00:18:14.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:13 smithi170 ceph-mon[32807]: pgmap v56: 129 pgs: 129 active+clean; 91 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.8 MiB/s wr, 476 op/s 2024-04-25T00:18:16.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:15 smithi107 ceph-mon[29153]: pgmap v57: 129 pgs: 129 active+clean; 91 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 426 op/s 2024-04-25T00:18:16.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:15 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:16.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:15 smithi195 ceph-mon[32833]: pgmap v57: 129 pgs: 129 active+clean; 91 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 426 op/s 2024-04-25T00:18:16.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:15 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:16.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:15 smithi170 ceph-mon[32807]: pgmap v57: 129 pgs: 129 active+clean; 91 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 426 op/s 2024-04-25T00:18:16.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:15 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:17.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:17 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:17.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:17 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:17.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:17 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:17.733 INFO:tasks.workunit.client.0.smithi107.stderr:++ readlink -f fsstress 2024-04-25T00:18:17.733 INFO:tasks.workunit.client.0.smithi107.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-25T00:18:17.734 INFO:tasks.workunit.client.0.smithi107.stderr:+ popd 2024-04-25T00:18:17.734 INFO:tasks.workunit.client.0.smithi107.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-25T00:18:17.734 INFO:tasks.workunit.client.0.smithi107.stderr:+ popd 2024-04-25T00:18:17.734 INFO:tasks.workunit.client.0.smithi107.stdout:~/cephtest/mnt.0/client.0/tmp 2024-04-25T00:18:17.735 INFO:tasks.workunit.client.0.smithi107.stderr:++ mktemp -d -p . 2024-04-25T00:18:17.735 INFO:tasks.workunit.client.0.smithi107.stderr:+ T=./tmp.Rw3HFxaOXA 2024-04-25T00:18:17.736 INFO:tasks.workunit.client.0.smithi107.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.Rw3HFxaOXA -l 1 -n 1000 -p 10 -v 2024-04-25T00:18:17.737 INFO:tasks.workunit.client.0.smithi107.stdout:seed = 1713562553 2024-04-25T00:18:17.739 INFO:tasks.workunit.client.0.smithi107.stdout:0/0: dread - no filename 2024-04-25T00:18:17.740 INFO:tasks.workunit.client.0.smithi107.stdout:0/1: chown . 18613547 1 2024-04-25T00:18:17.741 INFO:tasks.workunit.client.0.smithi107.stdout:0/2: mknod c0 0 2024-04-25T00:18:17.741 INFO:tasks.workunit.client.0.smithi107.stdout:0/3: chown c0 0 1 2024-04-25T00:18:17.741 INFO:tasks.workunit.client.0.smithi107.stdout:0/4: readlink - no filename 2024-04-25T00:18:17.741 INFO:tasks.workunit.client.0.smithi107.stdout:0/5: dread - no filename 2024-04-25T00:18:17.742 INFO:tasks.workunit.client.0.smithi107.stdout:1/0: mkdir d0 0 2024-04-25T00:18:17.743 INFO:tasks.workunit.client.0.smithi107.stdout:0/6: creat f1 x:0 0 0 2024-04-25T00:18:17.744 INFO:tasks.workunit.client.0.smithi107.stdout:3/0: unlink - no file 2024-04-25T00:18:17.744 INFO:tasks.workunit.client.0.smithi107.stdout:3/1: dread - no filename 2024-04-25T00:18:17.744 INFO:tasks.workunit.client.0.smithi107.stdout:3/2: write - no filename 2024-04-25T00:18:17.744 INFO:tasks.workunit.client.0.smithi107.stdout:2/0: mknod c0 0 2024-04-25T00:18:17.745 INFO:tasks.workunit.client.0.smithi107.stdout:2/1: truncate - no filename 2024-04-25T00:18:17.745 INFO:tasks.workunit.client.0.smithi107.stdout:2/2: dwrite - no filename 2024-04-25T00:18:17.745 INFO:tasks.workunit.client.0.smithi107.stdout:2/3: rmdir - no directory 2024-04-25T00:18:17.745 INFO:tasks.workunit.client.0.smithi107.stdout:1/1: rmdir d0 0 2024-04-25T00:18:17.745 INFO:tasks.workunit.client.0.smithi107.stdout:1/2: dread - no filename 2024-04-25T00:18:17.746 INFO:tasks.workunit.client.0.smithi107.stdout:0/7: symlink l2 0 2024-04-25T00:18:17.746 INFO:tasks.workunit.client.0.smithi107.stdout:0/8: dread - f1 zero size 2024-04-25T00:18:17.748 INFO:tasks.workunit.client.0.smithi107.stdout:3/3: creat f0 x:0 0 0 2024-04-25T00:18:17.749 INFO:tasks.workunit.client.0.smithi107.stdout:2/4: unlink c0 0 2024-04-25T00:18:17.750 INFO:tasks.workunit.client.0.smithi107.stdout:0/9: symlink l3 0 2024-04-25T00:18:17.750 INFO:tasks.workunit.client.0.smithi107.stdout:0/10: dread - f1 zero size 2024-04-25T00:18:17.751 INFO:tasks.workunit.client.0.smithi107.stdout:3/4: symlink l1 0 2024-04-25T00:18:17.751 INFO:tasks.workunit.client.0.smithi107.stdout:3/5: write f0 [1001997,2239] 0 2024-04-25T00:18:17.752 INFO:tasks.workunit.client.0.smithi107.stdout:2/5: mknod c1 0 2024-04-25T00:18:17.752 INFO:tasks.workunit.client.0.smithi107.stdout:2/6: chown c1 60263 1 2024-04-25T00:18:17.754 INFO:tasks.workunit.client.0.smithi107.stdout:2/7: creat f2 x:0 0 0 2024-04-25T00:18:17.757 INFO:tasks.workunit.client.0.smithi107.stdout:2/8: unlink c1 0 2024-04-25T00:18:17.757 INFO:tasks.workunit.client.0.smithi107.stdout:2/9: chown f2 6 1 2024-04-25T00:18:17.757 INFO:tasks.workunit.client.0.smithi107.stdout:2/10: dread - f2 zero size 2024-04-25T00:18:17.757 INFO:tasks.workunit.client.0.smithi107.stdout:2/11: read - f2 zero size 2024-04-25T00:18:17.760 INFO:tasks.workunit.client.0.smithi107.stdout:3/6: dread f0 [0,4194304] 0 2024-04-25T00:18:17.760 INFO:tasks.workunit.client.0.smithi107.stdout:3/7: truncate f0 1064611 0 2024-04-25T00:18:17.760 INFO:tasks.workunit.client.0.smithi107.stdout:3/8: chown f0 165 1 2024-04-25T00:18:17.762 INFO:tasks.workunit.client.0.smithi107.stdout:3/9: read f0 [540847,21335] 0 2024-04-25T00:18:17.764 INFO:tasks.workunit.client.0.smithi107.stdout:3/10: mknod c2 0 2024-04-25T00:18:17.764 INFO:tasks.workunit.client.0.smithi107.stdout:3/11: chown c2 301596 1 2024-04-25T00:18:17.808 INFO:tasks.workunit.client.0.smithi107.stdout:8/0: dwrite - no filename 2024-04-25T00:18:17.808 INFO:tasks.workunit.client.0.smithi107.stdout:6/0: dread - no filename 2024-04-25T00:18:17.808 INFO:tasks.workunit.client.0.smithi107.stdout:6/1: fdatasync - no filename 2024-04-25T00:18:17.808 INFO:tasks.workunit.client.0.smithi107.stdout:4/0: chown . 115738 1 2024-04-25T00:18:17.808 INFO:tasks.workunit.client.0.smithi107.stdout:7/0: dread - no filename 2024-04-25T00:18:17.808 INFO:tasks.workunit.client.0.smithi107.stdout:6/2: unlink - no file 2024-04-25T00:18:17.808 INFO:tasks.workunit.client.0.smithi107.stdout:9/0: readlink - no filename 2024-04-25T00:18:17.808 INFO:tasks.workunit.client.0.smithi107.stdout:6/3: rename - no filename 2024-04-25T00:18:17.808 INFO:tasks.workunit.client.0.smithi107.stdout:4/1: read - no filename 2024-04-25T00:18:17.808 INFO:tasks.workunit.client.0.smithi107.stdout:9/1: write - no filename 2024-04-25T00:18:17.808 INFO:tasks.workunit.client.0.smithi107.stdout:6/4: dwrite - no filename 2024-04-25T00:18:17.809 INFO:tasks.workunit.client.0.smithi107.stdout:9/2: readlink - no filename 2024-04-25T00:18:17.809 INFO:tasks.workunit.client.0.smithi107.stdout:4/2: truncate - no filename 2024-04-25T00:18:17.809 INFO:tasks.workunit.client.0.smithi107.stdout:7/1: truncate - no filename 2024-04-25T00:18:17.809 INFO:tasks.workunit.client.0.smithi107.stdout:9/3: rename - no filename 2024-04-25T00:18:17.809 INFO:tasks.workunit.client.0.smithi107.stdout:4/3: write - no filename 2024-04-25T00:18:17.809 INFO:tasks.workunit.client.0.smithi107.stdout:5/0: stat - no entries 2024-04-25T00:18:17.809 INFO:tasks.workunit.client.0.smithi107.stdout:5/1: rename - no filename 2024-04-25T00:18:17.810 INFO:tasks.workunit.client.0.smithi107.stdout:9/4: mkdir d0 0 2024-04-25T00:18:17.810 INFO:tasks.workunit.client.0.smithi107.stdout:9/5: fdatasync - no filename 2024-04-25T00:18:17.810 INFO:tasks.workunit.client.0.smithi107.stdout:9/6: chown d0 62 1 2024-04-25T00:18:17.810 INFO:tasks.workunit.client.0.smithi107.stdout:9/7: dwrite - no filename 2024-04-25T00:18:17.811 INFO:tasks.workunit.client.0.smithi107.stdout:4/4: creat f0 x:0 0 0 2024-04-25T00:18:17.811 INFO:tasks.workunit.client.0.smithi107.stdout:4/5: write f0 [180387,28971] 0 2024-04-25T00:18:17.812 INFO:tasks.workunit.client.0.smithi107.stdout:8/1: creat f0 x:0 0 0 2024-04-25T00:18:17.812 INFO:tasks.workunit.client.0.smithi107.stdout:8/2: fdatasync f0 0 2024-04-25T00:18:17.812 INFO:tasks.workunit.client.0.smithi107.stdout:8/3: chown f0 3780 1 2024-04-25T00:18:17.812 INFO:tasks.workunit.client.0.smithi107.stdout:6/5: symlink l0 0 2024-04-25T00:18:17.813 INFO:tasks.workunit.client.0.smithi107.stdout:7/2: mkdir d0 0 2024-04-25T00:18:17.813 INFO:tasks.workunit.client.0.smithi107.stdout:7/3: dread - no filename 2024-04-25T00:18:17.813 INFO:tasks.workunit.client.0.smithi107.stdout:7/4: dwrite - no filename 2024-04-25T00:18:17.813 INFO:tasks.workunit.client.0.smithi107.stdout:7/5: chown d0 57681 1 2024-04-25T00:18:17.814 INFO:tasks.workunit.client.0.smithi107.stdout:5/2: symlink l0 0 2024-04-25T00:18:17.814 INFO:tasks.workunit.client.0.smithi107.stdout:9/8: mkdir d0/d1 0 2024-04-25T00:18:17.816 INFO:tasks.workunit.client.0.smithi107.stdout:8/4: link f0 f1 0 2024-04-25T00:18:17.817 INFO:tasks.workunit.client.0.smithi107.stdout:6/6: mkdir d1 0 2024-04-25T00:18:17.817 INFO:tasks.workunit.client.0.smithi107.stdout:6/7: chown l0 8034428 1 2024-04-25T00:18:17.817 INFO:tasks.workunit.client.0.smithi107.stdout:7/6: creat d0/f1 x:0 0 0 2024-04-25T00:18:17.820 INFO:tasks.workunit.client.0.smithi107.stdout:9/9: creat d0/f2 x:0 0 0 2024-04-25T00:18:17.821 INFO:tasks.workunit.client.0.smithi107.stdout:8/5: mkdir d2 0 2024-04-25T00:18:17.822 INFO:tasks.workunit.client.0.smithi107.stdout:6/8: mkdir d1/d2 0 2024-04-25T00:18:17.822 INFO:tasks.workunit.client.0.smithi107.stdout:6/9: dread - no filename 2024-04-25T00:18:17.823 INFO:tasks.workunit.client.0.smithi107.stdout:9/10: creat d0/d1/f3 x:0 0 0 2024-04-25T00:18:17.824 INFO:tasks.workunit.client.0.smithi107.stdout:6/10: mkdir d1/d2/d3 0 2024-04-25T00:18:17.825 INFO:tasks.workunit.client.0.smithi107.stdout:9/11: creat d0/d1/f4 x:0 0 0 2024-04-25T00:18:17.831 INFO:tasks.workunit.client.0.smithi107.stdout:6/11: unlink l0 0 2024-04-25T00:18:17.831 INFO:tasks.workunit.client.0.smithi107.stdout:9/12: creat d0/d1/f5 x:0 0 0 2024-04-25T00:18:17.831 INFO:tasks.workunit.client.0.smithi107.stdout:9/13: chown d0/d1/f4 174026711 1 2024-04-25T00:18:17.831 INFO:tasks.workunit.client.0.smithi107.stdout:9/14: write d0/d1/f5 [788613,55765] 0 2024-04-25T00:18:17.831 INFO:tasks.workunit.client.0.smithi107.stdout:9/15: chown d0/f2 0 1 2024-04-25T00:18:17.831 INFO:tasks.workunit.client.0.smithi107.stdout:9/16: truncate d0/d1/f3 440235 0 2024-04-25T00:18:17.831 INFO:tasks.workunit.client.0.smithi107.stdout:6/12: creat d1/d2/d3/f4 x:0 0 0 2024-04-25T00:18:17.831 INFO:tasks.workunit.client.0.smithi107.stdout:6/13: chown d1 4 1 2024-04-25T00:18:17.831 INFO:tasks.workunit.client.0.smithi107.stdout:6/14: dread - d1/d2/d3/f4 zero size 2024-04-25T00:18:17.832 INFO:tasks.workunit.client.0.smithi107.stdout:9/17: mknod d0/d1/c6 0 2024-04-25T00:18:17.832 INFO:tasks.workunit.client.0.smithi107.stdout:9/18: write d0/d1/f5 [1725276,64242] 0 2024-04-25T00:18:17.832 INFO:tasks.workunit.client.0.smithi107.stdout:9/19: stat d0/d1/c6 0 2024-04-25T00:18:17.833 INFO:tasks.workunit.client.0.smithi107.stdout:9/20: write d0/d1/f5 [363516,74325] 0 2024-04-25T00:18:17.833 INFO:tasks.workunit.client.0.smithi107.stdout:9/21: chown d0/d1 168492 1 2024-04-25T00:18:17.833 INFO:tasks.workunit.client.0.smithi107.stdout:9/22: write d0/d1/f5 [2026439,80036] 0 2024-04-25T00:18:17.833 INFO:tasks.workunit.client.0.smithi107.stdout:9/23: creat d0/d1/f7 x:0 0 0 2024-04-25T00:18:17.851 INFO:tasks.workunit.client.0.smithi107.stdout:6/15: dwrite d1/d2/d3/f4 [0,4194304] 0 2024-04-25T00:18:17.851 INFO:tasks.workunit.client.0.smithi107.stdout:6/16: stat d1 0 2024-04-25T00:18:17.854 INFO:tasks.workunit.client.0.smithi107.stdout:6/17: creat d1/d2/f5 x:0 0 0 2024-04-25T00:18:17.854 INFO:tasks.workunit.client.0.smithi107.stdout:6/18: dread - d1/d2/f5 zero size 2024-04-25T00:18:17.873 INFO:tasks.workunit.client.0.smithi107.stdout:6/19: dwrite d1/d2/d3/f4 [0,4194304] 0 2024-04-25T00:18:17.873 INFO:tasks.workunit.client.0.smithi107.stdout:6/20: chown d1/d2/d3 3780 1 2024-04-25T00:18:17.876 INFO:tasks.workunit.client.0.smithi107.stdout:6/21: unlink d1/d2/d3/f4 0 2024-04-25T00:18:17.876 INFO:tasks.workunit.client.0.smithi107.stdout:6/22: fdatasync d1/d2/f5 0 2024-04-25T00:18:17.879 INFO:tasks.workunit.client.0.smithi107.stdout:6/23: creat d1/d2/d3/f6 x:0 0 0 2024-04-25T00:18:17.881 INFO:tasks.workunit.client.0.smithi107.stdout:6/24: mknod d1/d2/c7 0 2024-04-25T00:18:17.881 INFO:tasks.workunit.client.0.smithi107.stdout:6/25: write d1/d2/d3/f6 [528957,16594] 0 2024-04-25T00:18:17.881 INFO:tasks.workunit.client.0.smithi107.stdout:6/26: write d1/d2/d3/f6 [119911,69732] 0 2024-04-25T00:18:17.884 INFO:tasks.workunit.client.0.smithi107.stdout:6/27: unlink d1/d2/f5 0 2024-04-25T00:18:17.892 INFO:tasks.workunit.client.0.smithi107.stdout:6/28: dread d1/d2/d3/f6 [0,4194304] 0 2024-04-25T00:18:17.893 INFO:tasks.workunit.client.0.smithi107.stdout:6/29: mkdir d1/d8 0 2024-04-25T00:18:17.894 INFO:tasks.workunit.client.0.smithi107.stdout:6/30: rmdir d1/d8 0 2024-04-25T00:18:17.894 INFO:tasks.workunit.client.0.smithi107.stdout:6/31: chown d1/d2/d3/f6 600194139 1 2024-04-25T00:18:17.895 INFO:tasks.workunit.client.0.smithi107.stdout:6/32: creat d1/d2/d3/f9 x:0 0 0 2024-04-25T00:18:17.896 INFO:tasks.workunit.client.0.smithi107.stdout:6/33: mkdir d1/d2/da 0 2024-04-25T00:18:17.896 INFO:tasks.workunit.client.0.smithi107.stdout:6/34: write d1/d2/d3/f6 [915538,116724] 0 2024-04-25T00:18:17.896 INFO:tasks.workunit.client.0.smithi107.stdout:6/35: write d1/d2/d3/f9 [114752,54831] 0 2024-04-25T00:18:17.897 INFO:tasks.workunit.client.0.smithi107.stdout:6/36: rmdir d1/d2/da 0 2024-04-25T00:18:17.905 INFO:tasks.workunit.client.0.smithi107.stdout:6/37: dread d1/d2/d3/f6 [0,4194304] 0 2024-04-25T00:18:17.911 INFO:tasks.workunit.client.0.smithi107.stdout:6/38: dread d1/d2/d3/f9 [0,4194304] 0 2024-04-25T00:18:17.927 INFO:tasks.workunit.client.0.smithi107.stdout:6/39: dwrite d1/d2/d3/f9 [0,4194304] 0 2024-04-25T00:18:17.927 INFO:tasks.workunit.client.0.smithi107.stdout:6/40: chown d1 166436 1 2024-04-25T00:18:17.935 INFO:tasks.workunit.client.0.smithi107.stdout:6/41: write d1/d2/d3/f9 [2027628,59677] 0 2024-04-25T00:18:17.936 INFO:tasks.workunit.client.0.smithi107.stdout:6/42: mkdir d1/d2/db 0 2024-04-25T00:18:17.938 INFO:tasks.workunit.client.0.smithi107.stdout:6/43: mkdir d1/d2/dc 0 2024-04-25T00:18:17.940 INFO:tasks.workunit.client.0.smithi107.stdout:6/44: rename d1/d2/d3/f9 to d1/d2/db/fd 0 2024-04-25T00:18:17.940 INFO:tasks.workunit.client.0.smithi107.stdout:6/45: write d1/d2/d3/f6 [1267088,12547] 0 2024-04-25T00:18:17.942 INFO:tasks.workunit.client.0.smithi107.stdout:6/46: mknod d1/d2/ce 0 2024-04-25T00:18:17.942 INFO:tasks.workunit.client.0.smithi107.stdout:6/47: write d1/d2/d3/f6 [2049606,44645] 0 2024-04-25T00:18:17.942 INFO:tasks.workunit.client.0.smithi107.stdout:6/48: write d1/d2/db/fd [1846659,69249] 0 2024-04-25T00:18:17.951 INFO:tasks.workunit.client.0.smithi107.stdout:6/49: dread d1/d2/db/fd [0,4194304] 0 2024-04-25T00:18:17.952 INFO:tasks.workunit.client.0.smithi107.stdout:6/50: link d1/d2/ce d1/d2/dc/cf 0 2024-04-25T00:18:17.954 INFO:tasks.workunit.client.0.smithi107.stdout:6/51: link d1/d2/c7 d1/d2/db/c10 0 2024-04-25T00:18:18.072 INFO:tasks.workunit.client.0.smithi107.stdout:0/11: getdents . 0 2024-04-25T00:18:18.073 INFO:tasks.workunit.client.0.smithi107.stdout:2/12: fsync f2 0 2024-04-25T00:18:18.075 INFO:tasks.workunit.client.0.smithi107.stdout:2/13: unlink f2 0 2024-04-25T00:18:18.077 INFO:tasks.workunit.client.0.smithi107.stdout:2/14: creat f3 x:0 0 0 2024-04-25T00:18:18.077 INFO:tasks.workunit.client.0.smithi107.stdout:2/15: readlink - no filename 2024-04-25T00:18:18.077 INFO:tasks.workunit.client.0.smithi107.stdout:2/16: write f3 [128387,35250] 0 2024-04-25T00:18:18.077 INFO:tasks.workunit.client.0.smithi107.stdout:2/17: write f3 [946802,35032] 0 2024-04-25T00:18:18.077 INFO:tasks.workunit.client.0.smithi107.stdout:2/18: rmdir - no directory 2024-04-25T00:18:18.079 INFO:tasks.workunit.client.0.smithi107.stdout:3/12: truncate f0 131091 0 2024-04-25T00:18:18.081 INFO:tasks.workunit.client.0.smithi107.stdout:2/19: fdatasync f3 0 2024-04-25T00:18:18.081 INFO:tasks.workunit.client.0.smithi107.stdout:2/20: chown f3 3608 1 2024-04-25T00:18:18.081 INFO:tasks.workunit.client.0.smithi107.stdout:2/21: chown f3 168506 1 2024-04-25T00:18:18.084 INFO:tasks.workunit.client.0.smithi107.stdout:2/22: dread f3 [0,4194304] 0 2024-04-25T00:18:18.085 INFO:tasks.workunit.client.0.smithi107.stdout:5/3: getdents . 0 2024-04-25T00:18:18.085 INFO:tasks.workunit.client.0.smithi107.stdout:5/4: truncate - no filename 2024-04-25T00:18:18.085 INFO:tasks.workunit.client.0.smithi107.stdout:5/5: fdatasync - no filename 2024-04-25T00:18:18.085 INFO:tasks.workunit.client.0.smithi107.stdout:5/6: stat l0 0 2024-04-25T00:18:18.086 INFO:tasks.workunit.client.0.smithi107.stdout:2/23: dread f3 [0,4194304] 0 2024-04-25T00:18:18.086 INFO:tasks.workunit.client.0.smithi107.stdout:2/24: rmdir - no directory 2024-04-25T00:18:18.086 INFO:tasks.workunit.client.0.smithi107.stdout:5/7: creat f1 x:0 0 0 2024-04-25T00:18:18.088 INFO:tasks.workunit.client.0.smithi107.stdout:2/25: creat f4 x:0 0 0 2024-04-25T00:18:18.088 INFO:tasks.workunit.client.0.smithi107.stdout:8/6: fsync f0 0 2024-04-25T00:18:18.088 INFO:tasks.workunit.client.0.smithi107.stdout:8/7: write f1 [759440,11921] 0 2024-04-25T00:18:18.090 INFO:tasks.workunit.client.0.smithi107.stdout:2/26: creat f5 x:0 0 0 2024-04-25T00:18:18.090 INFO:tasks.workunit.client.0.smithi107.stdout:2/27: write f4 [379664,56549] 0 2024-04-25T00:18:18.092 INFO:tasks.workunit.client.0.smithi107.stdout:8/8: link f1 d2/f3 0 2024-04-25T00:18:18.092 INFO:tasks.workunit.client.0.smithi107.stdout:7/7: fsync d0/f1 0 2024-04-25T00:18:18.092 INFO:tasks.workunit.client.0.smithi107.stdout:7/8: chown d0 186 1 2024-04-25T00:18:18.094 INFO:tasks.workunit.client.0.smithi107.stdout:2/28: mkdir d6 0 2024-04-25T00:18:18.094 INFO:tasks.workunit.client.0.smithi107.stdout:2/29: chown f4 5 1 2024-04-25T00:18:18.096 INFO:tasks.workunit.client.0.smithi107.stdout:8/9: creat d2/f4 x:0 0 0 2024-04-25T00:18:18.096 INFO:tasks.workunit.client.0.smithi107.stdout:8/10: dread - d2/f4 zero size 2024-04-25T00:18:18.096 INFO:tasks.workunit.client.0.smithi107.stdout:4/6: truncate f0 14821 0 2024-04-25T00:18:18.098 INFO:tasks.workunit.client.0.smithi107.stdout:2/30: rename f5 to d6/f7 0 2024-04-25T00:18:18.098 INFO:tasks.workunit.client.0.smithi107.stdout:8/11: rename d2 to d2/d5 22 2024-04-25T00:18:18.099 INFO:tasks.workunit.client.0.smithi107.stdout:4/7: symlink l1 0 2024-04-25T00:18:18.101 INFO:tasks.workunit.client.0.smithi107.stdout:5/8: dwrite f1 [0,4194304] 0 2024-04-25T00:18:18.102 INFO:tasks.workunit.client.0.smithi107.stdout:2/31: rmdir d6 39 2024-04-25T00:18:18.119 INFO:tasks.workunit.client.0.smithi107.stdout:7/9: dwrite d0/f1 [0,4194304] 0 2024-04-25T00:18:18.126 INFO:tasks.workunit.client.0.smithi107.stdout:8/12: dwrite d2/f4 [0,4194304] 0 2024-04-25T00:18:18.126 INFO:tasks.workunit.client.0.smithi107.stdout:8/13: truncate d2/f3 1296605 0 2024-04-25T00:18:18.128 INFO:tasks.workunit.client.0.smithi107.stdout:5/9: dwrite f1 [0,4194304] 0 2024-04-25T00:18:18.128 INFO:tasks.workunit.client.0.smithi107.stdout:5/10: stat f1 0 2024-04-25T00:18:18.132 INFO:tasks.workunit.client.0.smithi107.stdout:2/32: dwrite f4 [0,4194304] 0 2024-04-25T00:18:18.141 INFO:tasks.workunit.client.0.smithi107.stdout:8/14: symlink d2/l6 0 2024-04-25T00:18:18.146 INFO:tasks.workunit.client.0.smithi107.stdout:5/11: creat f2 x:0 0 0 2024-04-25T00:18:18.151 INFO:tasks.workunit.client.0.smithi107.stdout:8/15: dread f0 [0,4194304] 0 2024-04-25T00:18:18.161 INFO:tasks.workunit.client.0.smithi107.stdout:2/33: creat d6/f8 x:0 0 0 2024-04-25T00:18:18.166 INFO:tasks.workunit.client.0.smithi107.stdout:8/16: dwrite f1 [0,4194304] 0 2024-04-25T00:18:18.166 INFO:tasks.workunit.client.0.smithi107.stdout:8/17: fdatasync f0 0 2024-04-25T00:18:18.167 INFO:tasks.workunit.client.0.smithi107.stdout:8/18: chown d2/f4 7749063 1 2024-04-25T00:18:18.171 INFO:tasks.workunit.client.0.smithi107.stdout:6/52: truncate d1/d2/d3/f6 1601077 0 2024-04-25T00:18:18.171 INFO:tasks.workunit.client.0.smithi107.stdout:6/53: read d1/d2/db/fd [3943028,47471] 0 2024-04-25T00:18:18.174 INFO:tasks.workunit.client.0.smithi107.stdout:3/13: fsync f0 0 2024-04-25T00:18:18.174 INFO:tasks.workunit.client.0.smithi107.stdout:3/14: write f0 [88453,10991] 0 2024-04-25T00:18:18.177 INFO:tasks.workunit.client.0.smithi107.stdout:3/15: symlink l3 0 2024-04-25T00:18:18.178 INFO:tasks.workunit.client.0.smithi107.stdout:2/34: read f4 [2189261,24945] 0 2024-04-25T00:18:18.178 INFO:tasks.workunit.client.0.smithi107.stdout:2/35: fsync f4 0 2024-04-25T00:18:18.179 INFO:tasks.workunit.client.0.smithi107.stdout:3/16: mknod c4 0 2024-04-25T00:18:18.183 INFO:tasks.workunit.client.0.smithi107.stdout:3/17: dread f0 [0,4194304] 0 2024-04-25T00:18:18.187 INFO:tasks.workunit.client.0.smithi107.stdout:3/18: unlink l1 0 2024-04-25T00:18:18.189 INFO:tasks.workunit.client.0.smithi107.stdout:3/19: dread f0 [0,4194304] 0 2024-04-25T00:18:18.191 INFO:tasks.workunit.client.0.smithi107.stdout:3/20: dread f0 [0,4194304] 0 2024-04-25T00:18:18.191 INFO:tasks.workunit.client.0.smithi107.stdout:3/21: chown c4 95303 1 2024-04-25T00:18:18.194 INFO:tasks.workunit.client.0.smithi107.stdout:8/19: read d2/f3 [1238811,25634] 0 2024-04-25T00:18:18.196 INFO:tasks.workunit.client.0.smithi107.stdout:8/20: creat d2/f7 x:0 0 0 2024-04-25T00:18:18.200 INFO:tasks.workunit.client.0.smithi107.stdout:2/36: dwrite f4 [0,4194304] 0 2024-04-25T00:18:18.208 INFO:tasks.workunit.client.0.smithi107.stdout:4/8: dwrite f0 [0,4194304] 0 2024-04-25T00:18:18.212 INFO:tasks.workunit.client.0.smithi107.stdout:3/22: dwrite f0 [0,4194304] 0 2024-04-25T00:18:18.215 INFO:tasks.workunit.client.0.smithi107.stdout:4/9: getdents . 0 2024-04-25T00:18:18.221 INFO:tasks.workunit.client.0.smithi107.stdout:5/12: truncate f1 4089604 0 2024-04-25T00:18:18.223 INFO:tasks.workunit.client.0.smithi107.stdout:5/13: stat l0 0 2024-04-25T00:18:18.223 INFO:tasks.workunit.client.0.smithi107.stdout:2/37: dwrite d6/f8 [0,4194304] 0 2024-04-25T00:18:18.225 INFO:tasks.workunit.client.0.smithi107.stdout:7/10: dwrite d0/f1 [4194304,4194304] 0 2024-04-25T00:18:18.226 INFO:tasks.workunit.client.0.smithi107.stdout:5/14: unlink f2 0 2024-04-25T00:18:18.226 INFO:tasks.workunit.client.0.smithi107.stdout:5/15: stat l0 0 2024-04-25T00:18:18.228 INFO:tasks.workunit.client.0.smithi107.stdout:2/38: rmdir d6 39 2024-04-25T00:18:18.228 INFO:tasks.workunit.client.0.smithi107.stdout:2/39: stat f3 0 2024-04-25T00:18:18.231 INFO:tasks.workunit.client.0.smithi107.stdout:2/40: creat d6/f9 x:0 0 0 2024-04-25T00:18:18.233 INFO:tasks.workunit.client.0.smithi107.stdout:5/16: symlink l3 0 2024-04-25T00:18:18.233 INFO:tasks.workunit.client.0.smithi107.stdout:4/10: dwrite f0 [0,4194304] 0 2024-04-25T00:18:18.233 INFO:tasks.workunit.client.0.smithi107.stdout:4/11: chown l1 150488 1 2024-04-25T00:18:18.233 INFO:tasks.workunit.client.0.smithi107.stdout:4/12: fdatasync f0 0 2024-04-25T00:18:18.233 INFO:tasks.workunit.client.0.smithi107.stdout:4/13: readlink l1 0 2024-04-25T00:18:18.233 INFO:tasks.workunit.client.0.smithi107.stdout:2/41: symlink d6/la 0 2024-04-25T00:18:18.238 INFO:tasks.workunit.client.0.smithi107.stdout:2/42: rename d6/la to d6/lb 0 2024-04-25T00:18:18.238 INFO:tasks.workunit.client.0.smithi107.stdout:5/17: mkdir d4 0 2024-04-25T00:18:18.238 INFO:tasks.workunit.client.0.smithi107.stdout:2/43: mknod d6/cc 0 2024-04-25T00:18:18.240 INFO:tasks.workunit.client.0.smithi107.stdout:5/18: unlink l3 0 2024-04-25T00:18:18.241 INFO:tasks.workunit.client.0.smithi107.stdout:2/44: symlink d6/ld 0 2024-04-25T00:18:18.241 INFO:tasks.workunit.client.0.smithi107.stdout:2/45: write d6/f9 [1022963,60613] 0 2024-04-25T00:18:18.243 INFO:tasks.workunit.client.0.smithi107.stdout:7/11: dread d0/f1 [0,4194304] 0 2024-04-25T00:18:18.243 INFO:tasks.workunit.client.0.smithi107.stdout:4/14: read f0 [1209063,89454] 0 2024-04-25T00:18:18.243 INFO:tasks.workunit.client.0.smithi107.stdout:4/15: write f0 [1330211,57621] 0 2024-04-25T00:18:18.244 INFO:tasks.workunit.client.0.smithi107.stdout:2/46: dread f3 [0,4194304] 0 2024-04-25T00:18:18.247 INFO:tasks.workunit.client.0.smithi107.stdout:2/47: unlink d6/ld 0 2024-04-25T00:18:18.249 INFO:tasks.workunit.client.0.smithi107.stdout:2/48: creat d6/fe x:0 0 0 2024-04-25T00:18:18.249 INFO:tasks.workunit.client.0.smithi107.stdout:8/21: getdents d2 0 2024-04-25T00:18:18.250 INFO:tasks.workunit.client.0.smithi107.stdout:8/22: chown d2/l6 1800 1 2024-04-25T00:18:18.251 INFO:tasks.workunit.client.0.smithi107.stdout:2/49: creat d6/ff x:0 0 0 2024-04-25T00:18:18.253 INFO:tasks.workunit.client.0.smithi107.stdout:8/23: unlink d2/f4 0 2024-04-25T00:18:18.254 INFO:tasks.workunit.client.0.smithi107.stdout:8/24: stat f1 0 2024-04-25T00:18:18.255 INFO:tasks.workunit.client.0.smithi107.stdout:2/50: rename d6/f9 to d6/f10 0 2024-04-25T00:18:18.260 INFO:tasks.workunit.client.0.smithi107.stdout:3/23: write f0 [4496383,112854] 0 2024-04-25T00:18:18.260 INFO:tasks.workunit.client.0.smithi107.stdout:7/12: dwrite d0/f1 [4194304,4194304] 0 2024-04-25T00:18:18.263 INFO:tasks.workunit.client.0.smithi107.stdout:3/24: creat f5 x:0 0 0 2024-04-25T00:18:18.263 INFO:tasks.workunit.client.0.smithi107.stdout:3/25: chown c2 8356490 1 2024-04-25T00:18:18.266 INFO:tasks.workunit.client.0.smithi107.stdout:7/13: creat d0/f2 x:0 0 0 2024-04-25T00:18:18.270 INFO:tasks.workunit.client.0.smithi107.stdout:3/26: rename c2 to c6 0 2024-04-25T00:18:18.272 INFO:tasks.workunit.client.0.smithi107.stdout:8/25: dwrite f0 [0,4194304] 0 2024-04-25T00:18:18.274 INFO:tasks.workunit.client.0.smithi107.stdout:8/26: chown d2/l6 11722 1 2024-04-25T00:18:18.274 INFO:tasks.workunit.client.0.smithi107.stdout:8/27: chown d2/l6 3 1 2024-04-25T00:18:18.276 INFO:tasks.workunit.client.0.smithi107.stdout:5/19: dread f1 [0,4194304] 0 2024-04-25T00:18:18.280 INFO:tasks.workunit.client.0.smithi107.stdout:4/16: truncate f0 3632033 0 2024-04-25T00:18:18.280 INFO:tasks.workunit.client.0.smithi107.stdout:4/17: chown l1 2007827 1 2024-04-25T00:18:18.281 INFO:tasks.workunit.client.0.smithi107.stdout:6/54: truncate d1/d2/d3/f6 316975 0 2024-04-25T00:18:18.282 INFO:tasks.workunit.client.0.smithi107.stdout:5/20: dread f1 [0,4194304] 0 2024-04-25T00:18:18.283 INFO:tasks.workunit.client.0.smithi107.stdout:4/18: creat f2 x:0 0 0 2024-04-25T00:18:18.283 INFO:tasks.workunit.client.0.smithi107.stdout:4/19: chown f2 2837 1 2024-04-25T00:18:18.283 INFO:tasks.workunit.client.0.smithi107.stdout:4/20: dread - f2 zero size 2024-04-25T00:18:18.283 INFO:tasks.workunit.client.0.smithi107.stdout:6/55: creat d1/d2/db/f11 x:0 0 0 2024-04-25T00:18:18.284 INFO:tasks.workunit.client.0.smithi107.stdout:5/21: mknod d4/c5 0 2024-04-25T00:18:18.285 INFO:tasks.workunit.client.0.smithi107.stdout:6/56: link d1/d2/c7 d1/d2/dc/c12 0 2024-04-25T00:18:18.286 INFO:tasks.workunit.client.0.smithi107.stdout:3/27: dread f0 [0,4194304] 0 2024-04-25T00:18:18.286 INFO:tasks.workunit.client.0.smithi107.stdout:3/28: dread - f5 zero size 2024-04-25T00:18:18.286 INFO:tasks.workunit.client.0.smithi107.stdout:3/29: chown l3 48953066 1 2024-04-25T00:18:18.286 INFO:tasks.workunit.client.0.smithi107.stdout:6/57: creat d1/d2/d3/f13 x:0 0 0 2024-04-25T00:18:18.287 INFO:tasks.workunit.client.0.smithi107.stdout:3/30: creat f7 x:0 0 0 2024-04-25T00:18:18.287 INFO:tasks.workunit.client.0.smithi107.stdout:8/28: read f0 [2988151,45149] 0 2024-04-25T00:18:18.289 INFO:tasks.workunit.client.0.smithi107.stdout:3/31: symlink l8 0 2024-04-25T00:18:18.290 INFO:tasks.workunit.client.0.smithi107.stdout:8/29: creat d2/f8 x:0 0 0 2024-04-25T00:18:18.291 INFO:tasks.workunit.client.0.smithi107.stdout:3/32: mkdir d9 0 2024-04-25T00:18:18.294 INFO:tasks.workunit.client.0.smithi107.stdout:6/58: dread d1/d2/db/fd [0,4194304] 0 2024-04-25T00:18:18.300 INFO:tasks.workunit.client.0.smithi107.stdout:7/14: getdents d0 0 2024-04-25T00:18:18.302 INFO:tasks.workunit.client.0.smithi107.stdout:7/15: rename d0/f2 to d0/f3 0 2024-04-25T00:18:18.308 INFO:tasks.workunit.client.0.smithi107.stdout:3/33: dwrite f0 [4194304,4194304] 0 2024-04-25T00:18:18.308 INFO:tasks.workunit.client.0.smithi107.stdout:3/34: fdatasync f0 0 2024-04-25T00:18:18.311 INFO:tasks.workunit.client.0.smithi107.stdout:3/35: unlink f5 0 2024-04-25T00:18:18.314 INFO:tasks.workunit.client.0.smithi107.stdout:8/30: dwrite d2/f3 [4194304,4194304] 0 2024-04-25T00:18:18.314 INFO:tasks.workunit.client.0.smithi107.stdout:8/31: write f0 [730563,81750] 0 2024-04-25T00:18:18.314 INFO:tasks.workunit.client.0.smithi107.stdout:8/32: dread - d2/f8 zero size 2024-04-25T00:18:18.314 INFO:tasks.workunit.client.0.smithi107.stdout:8/33: readlink d2/l6 0 2024-04-25T00:18:18.315 INFO:tasks.workunit.client.0.smithi107.stdout:8/34: readlink d2/l6 0 2024-04-25T00:18:18.316 INFO:tasks.workunit.client.0.smithi107.stdout:6/59: dread d1/d2/d3/f6 [0,4194304] 0 2024-04-25T00:18:18.316 INFO:tasks.workunit.client.0.smithi107.stdout:6/60: truncate d1/d2/db/f11 580897 0 2024-04-25T00:18:18.317 INFO:tasks.workunit.client.0.smithi107.stdout:4/21: dread f0 [0,4194304] 0 2024-04-25T00:18:18.317 INFO:tasks.workunit.client.0.smithi107.stdout:4/22: write f0 [3614317,35448] 0 2024-04-25T00:18:18.318 INFO:tasks.workunit.client.0.smithi107.stdout:8/35: creat d2/f9 x:0 0 0 2024-04-25T00:18:18.322 INFO:tasks.workunit.client.0.smithi107.stdout:7/16: rename d0/f3 to d0/f4 0 2024-04-25T00:18:18.323 INFO:tasks.workunit.client.0.smithi107.stdout:4/23: mkdir d3 0 2024-04-25T00:18:18.324 INFO:tasks.workunit.client.0.smithi107.stdout:8/36: rename d2 to d2/da 22 2024-04-25T00:18:18.327 INFO:tasks.workunit.client.0.smithi107.stdout:8/37: creat d2/fb x:0 0 0 2024-04-25T00:18:18.329 INFO:tasks.workunit.client.0.smithi107.stdout:7/17: mknod d0/c5 0 2024-04-25T00:18:18.330 INFO:tasks.workunit.client.0.smithi107.stdout:3/36: dwrite f7 [0,4194304] 0 2024-04-25T00:18:18.331 INFO:tasks.workunit.client.0.smithi107.stdout:8/38: rmdir d2 39 2024-04-25T00:18:18.331 INFO:tasks.workunit.client.0.smithi107.stdout:8/39: chown d2/l6 3 1 2024-04-25T00:18:18.331 INFO:tasks.workunit.client.0.smithi107.stdout:8/40: fsync f0 0 2024-04-25T00:18:18.331 INFO:tasks.workunit.client.0.smithi107.stdout:8/41: stat f1 0 2024-04-25T00:18:18.333 INFO:tasks.workunit.client.0.smithi107.stdout:8/42: dread - d2/f7 zero size 2024-04-25T00:18:18.333 INFO:tasks.workunit.client.0.smithi107.stdout:8/43: read - d2/fb zero size 2024-04-25T00:18:18.337 INFO:tasks.workunit.client.0.smithi107.stdout:7/18: rename d0/f1 to d0/f6 0 2024-04-25T00:18:18.337 INFO:tasks.workunit.client.0.smithi107.stdout:7/19: read - d0/f4 zero size 2024-04-25T00:18:18.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:18 smithi195 ceph-mon[32833]: pgmap v58: 129 pgs: 129 active+clean; 91 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 426 op/s 2024-04-25T00:18:18.339 INFO:tasks.workunit.client.0.smithi107.stdout:3/37: dread f0 [0,4194304] 0 2024-04-25T00:18:18.349 INFO:tasks.workunit.client.0.smithi107.stdout:3/38: dread f7 [0,4194304] 0 2024-04-25T00:18:18.350 INFO:tasks.workunit.client.0.smithi107.stdout:4/24: dwrite f0 [0,4194304] 0 2024-04-25T00:18:18.353 INFO:tasks.workunit.client.0.smithi107.stdout:7/20: dwrite d0/f4 [0,4194304] 0 2024-04-25T00:18:18.354 INFO:tasks.workunit.client.0.smithi107.stdout:4/25: truncate f0 4193814 0 2024-04-25T00:18:18.357 INFO:tasks.workunit.client.0.smithi107.stdout:4/26: symlink d3/l4 0 2024-04-25T00:18:18.357 INFO:tasks.workunit.client.0.smithi107.stdout:7/21: mknod d0/c7 0 2024-04-25T00:18:18.361 INFO:tasks.workunit.client.0.smithi107.stdout:4/27: fsync f0 0 2024-04-25T00:18:18.366 INFO:tasks.workunit.client.0.smithi107.stdout:7/22: dread d0/f6 [0,4194304] 0 2024-04-25T00:18:18.367 INFO:tasks.workunit.client.0.smithi107.stdout:3/39: dwrite f7 [0,4194304] 0 2024-04-25T00:18:18.376 INFO:tasks.workunit.client.0.smithi107.stdout:4/28: dread f0 [0,4194304] 0 2024-04-25T00:18:18.377 INFO:tasks.workunit.client.0.smithi107.stdout:4/29: mkdir d3/d5 0 2024-04-25T00:18:18.381 INFO:tasks.workunit.client.0.smithi107.stdout:7/23: dread d0/f4 [0,4194304] 0 2024-04-25T00:18:18.382 INFO:tasks.workunit.client.0.smithi107.stdout:7/24: getdents d0 0 2024-04-25T00:18:18.383 INFO:tasks.workunit.client.0.smithi107.stdout:7/25: chown d0/c7 1979811 1 2024-04-25T00:18:18.383 INFO:tasks.workunit.client.0.smithi107.stdout:7/26: fsync d0/f6 0 2024-04-25T00:18:18.389 INFO:tasks.workunit.client.0.smithi107.stdout:3/40: dwrite f7 [0,4194304] 0 2024-04-25T00:18:18.391 INFO:tasks.workunit.client.0.smithi107.stdout:3/41: mknod d9/ca 0 2024-04-25T00:18:18.399 INFO:tasks.workunit.client.0.smithi107.stdout:3/42: dread f7 [0,4194304] 0 2024-04-25T00:18:18.399 INFO:tasks.workunit.client.0.smithi107.stdout:3/43: chown l8 362245 1 2024-04-25T00:18:18.400 INFO:tasks.workunit.client.0.smithi107.stdout:3/44: symlink d9/lb 0 2024-04-25T00:18:18.400 INFO:tasks.workunit.client.0.smithi107.stdout:7/27: dwrite d0/f4 [0,4194304] 0 2024-04-25T00:18:18.401 INFO:tasks.workunit.client.0.smithi107.stdout:7/28: rmdir d0 39 2024-04-25T00:18:18.404 INFO:tasks.workunit.client.0.smithi107.stdout:4/30: dwrite f2 [0,4194304] 0 2024-04-25T00:18:18.408 INFO:tasks.workunit.client.0.smithi107.stdout:3/45: dread f7 [0,4194304] 0 2024-04-25T00:18:18.409 INFO:tasks.workunit.client.0.smithi107.stdout:3/46: getdents d9 0 2024-04-25T00:18:18.409 INFO:tasks.workunit.client.0.smithi107.stdout:3/47: chown c4 62 1 2024-04-25T00:18:18.418 INFO:tasks.workunit.client.0.smithi107.stdout:3/48: write f0 [5360328,130248] 0 2024-04-25T00:18:18.418 INFO:tasks.workunit.client.0.smithi107.stdout:7/29: dwrite d0/f4 [0,4194304] 0 2024-04-25T00:18:18.420 INFO:tasks.workunit.client.0.smithi107.stdout:7/30: mkdir d0/d8 0 2024-04-25T00:18:18.420 INFO:tasks.workunit.client.0.smithi107.stdout:7/31: chown d0/c7 447 1 2024-04-25T00:18:18.421 INFO:tasks.workunit.client.0.smithi107.stdout:7/32: truncate d0/f6 2291437 0 2024-04-25T00:18:18.425 INFO:tasks.workunit.client.0.smithi107.stdout:3/49: write f7 [2270144,72816] 0 2024-04-25T00:18:18.431 INFO:tasks.workunit.client.0.smithi107.stdout:7/33: dread d0/f4 [0,4194304] 0 2024-04-25T00:18:18.431 INFO:tasks.workunit.client.0.smithi107.stdout:7/34: fdatasync d0/f6 0 2024-04-25T00:18:18.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:18 smithi170 ceph-mon[32807]: pgmap v58: 129 pgs: 129 active+clean; 91 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 426 op/s 2024-04-25T00:18:18.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:18 smithi107 ceph-mon[29153]: pgmap v58: 129 pgs: 129 active+clean; 91 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 426 op/s 2024-04-25T00:18:19.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:19.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:19.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:19.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:19.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: pgmap v59: 129 pgs: 129 active+clean; 133 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.8 MiB/s wr, 764 op/s 2024-04-25T00:18:20.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:20.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:20 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: pgmap v59: 129 pgs: 129 active+clean; 133 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.8 MiB/s wr, 764 op/s 2024-04-25T00:18:20.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.440 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.441 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.441 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.441 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.441 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.441 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.441 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.441 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.441 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.441 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.441 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.441 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.441 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.441 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.441 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.441 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.441 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:20.442 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:20 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: pgmap v59: 129 pgs: 129 active+clean; 133 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.8 MiB/s wr, 764 op/s 2024-04-25T00:18:20.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.488 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.490 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.490 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.490 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.490 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.490 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.490 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.490 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.490 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.490 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.490 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.490 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.490 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.490 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.490 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.491 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.491 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.491 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.491 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.491 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.491 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:20.491 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:20.491 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:20 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:21.155 INFO:tasks.workunit.client.0.smithi107.stdout:3/50: sync 2024-04-25T00:18:21.155 INFO:tasks.workunit.client.0.smithi107.stdout:1/3: sync 2024-04-25T00:18:21.155 INFO:tasks.workunit.client.0.smithi107.stdout:2/51: sync 2024-04-25T00:18:21.155 INFO:tasks.workunit.client.0.smithi107.stdout:1/4: truncate - no filename 2024-04-25T00:18:21.155 INFO:tasks.workunit.client.0.smithi107.stdout:1/5: write - no filename 2024-04-25T00:18:21.155 INFO:tasks.workunit.client.0.smithi107.stdout:1/6: unlink - no file 2024-04-25T00:18:21.155 INFO:tasks.workunit.client.0.smithi107.stdout:8/44: sync 2024-04-25T00:18:21.155 INFO:tasks.workunit.client.0.smithi107.stdout:5/22: sync 2024-04-25T00:18:21.155 INFO:tasks.workunit.client.0.smithi107.stdout:7/35: sync 2024-04-25T00:18:21.155 INFO:tasks.workunit.client.0.smithi107.stdout:0/12: sync 2024-04-25T00:18:21.155 INFO:tasks.workunit.client.0.smithi107.stdout:8/45: read f0 [2721756,21065] 0 2024-04-25T00:18:21.155 INFO:tasks.workunit.client.0.smithi107.stdout:0/13: dread - f1 zero size 2024-04-25T00:18:21.156 INFO:tasks.workunit.client.0.smithi107.stdout:4/31: sync 2024-04-25T00:18:21.156 INFO:tasks.workunit.client.0.smithi107.stdout:4/32: chown f2 3436 1 2024-04-25T00:18:21.156 INFO:tasks.workunit.client.0.smithi107.stdout:6/61: sync 2024-04-25T00:18:21.156 INFO:tasks.workunit.client.0.smithi107.stdout:9/24: sync 2024-04-25T00:18:21.156 INFO:tasks.workunit.client.0.smithi107.stdout:3/51: mknod d9/cc 0 2024-04-25T00:18:21.156 INFO:tasks.workunit.client.0.smithi107.stdout:2/52: creat d6/f11 x:0 0 0 2024-04-25T00:18:21.157 INFO:tasks.workunit.client.0.smithi107.stdout:1/7: getdents . 0 2024-04-25T00:18:21.157 INFO:tasks.workunit.client.0.smithi107.stdout:5/23: getdents d4 0 2024-04-25T00:18:21.158 INFO:tasks.workunit.client.0.smithi107.stdout:7/36: mknod d0/d8/c9 0 2024-04-25T00:18:21.158 INFO:tasks.workunit.client.0.smithi107.stdout:0/14: symlink l4 0 2024-04-25T00:18:21.159 INFO:tasks.workunit.client.0.smithi107.stdout:4/33: rename d3/l4 to d3/d5/l6 0 2024-04-25T00:18:21.159 INFO:tasks.workunit.client.0.smithi107.stdout:4/34: stat f2 0 2024-04-25T00:18:21.160 INFO:tasks.workunit.client.0.smithi107.stdout:9/25: mkdir d0/d8 0 2024-04-25T00:18:21.160 INFO:tasks.workunit.client.0.smithi107.stdout:3/52: mknod d9/cd 0 2024-04-25T00:18:21.162 INFO:tasks.workunit.client.0.smithi107.stdout:2/53: link d6/lb d6/l12 0 2024-04-25T00:18:21.162 INFO:tasks.workunit.client.0.smithi107.stdout:2/54: chown d6/l12 4121 1 2024-04-25T00:18:21.162 INFO:tasks.workunit.client.0.smithi107.stdout:1/8: creat f1 x:0 0 0 2024-04-25T00:18:21.163 INFO:tasks.workunit.client.0.smithi107.stdout:5/24: symlink d4/l6 0 2024-04-25T00:18:21.164 INFO:tasks.workunit.client.0.smithi107.stdout:8/46: link f1 d2/fc 0 2024-04-25T00:18:21.165 INFO:tasks.workunit.client.0.smithi107.stdout:7/37: symlink d0/d8/la 0 2024-04-25T00:18:21.166 INFO:tasks.workunit.client.0.smithi107.stdout:4/35: creat d3/d5/f7 x:0 0 0 2024-04-25T00:18:21.166 INFO:tasks.workunit.client.0.smithi107.stdout:2/55: creat d6/f13 x:0 0 0 2024-04-25T00:18:21.167 INFO:tasks.workunit.client.0.smithi107.stdout:9/26: dread d0/d1/f5 [0,4194304] 0 2024-04-25T00:18:21.168 INFO:tasks.workunit.client.0.smithi107.stdout:2/56: rename f3 to d6/f14 0 2024-04-25T00:18:21.169 INFO:tasks.workunit.client.0.smithi107.stdout:1/9: mkdir d2 0 2024-04-25T00:18:21.169 INFO:tasks.workunit.client.0.smithi107.stdout:1/10: truncate f1 598000 0 2024-04-25T00:18:21.169 INFO:tasks.workunit.client.0.smithi107.stdout:9/27: creat d0/d8/f9 x:0 0 0 2024-04-25T00:18:21.170 INFO:tasks.workunit.client.0.smithi107.stdout:2/57: rename d6/lb to d6/l15 0 2024-04-25T00:18:21.174 INFO:tasks.workunit.client.0.smithi107.stdout:9/28: unlink d0/d1/f5 0 2024-04-25T00:18:21.176 INFO:tasks.workunit.client.0.smithi107.stdout:6/62: dwrite d1/d2/db/f11 [0,4194304] 0 2024-04-25T00:18:21.176 INFO:tasks.workunit.client.0.smithi107.stdout:2/58: truncate d6/f8 111675 0 2024-04-25T00:18:21.176 INFO:tasks.workunit.client.0.smithi107.stdout:2/59: fdatasync d6/f14 0 2024-04-25T00:18:21.180 INFO:tasks.workunit.client.0.smithi107.stdout:9/29: unlink d0/d8/f9 0 2024-04-25T00:18:21.182 INFO:tasks.workunit.client.0.smithi107.stdout:9/30: dread d0/d1/f3 [0,4194304] 0 2024-04-25T00:18:21.182 INFO:tasks.workunit.client.0.smithi107.stdout:9/31: truncate d0/d1/f7 870747 0 2024-04-25T00:18:21.182 INFO:tasks.workunit.client.0.smithi107.stdout:9/32: write d0/f2 [317507,84560] 0 2024-04-25T00:18:21.183 INFO:tasks.workunit.client.0.smithi107.stdout:0/15: dwrite f1 [0,4194304] 0 2024-04-25T00:18:21.185 INFO:tasks.workunit.client.0.smithi107.stdout:9/33: symlink d0/la 0 2024-04-25T00:18:21.185 INFO:tasks.workunit.client.0.smithi107.stdout:3/53: dwrite f0 [4194304,4194304] 0 2024-04-25T00:18:21.185 INFO:tasks.workunit.client.0.smithi107.stdout:1/11: fsync f1 0 2024-04-25T00:18:21.192 INFO:tasks.workunit.client.0.smithi107.stdout:5/25: dwrite f1 [0,4194304] 0 2024-04-25T00:18:21.192 INFO:tasks.workunit.client.0.smithi107.stdout:6/63: dread d1/d2/db/f11 [0,4194304] 0 2024-04-25T00:18:21.192 INFO:tasks.workunit.client.0.smithi107.stdout:8/47: dwrite d2/f9 [0,4194304] 0 2024-04-25T00:18:21.197 INFO:tasks.workunit.client.0.smithi107.stdout:9/34: write d0/d1/f7 [759314,118338] 0 2024-04-25T00:18:21.201 INFO:tasks.workunit.client.0.smithi107.stdout:0/16: read f1 [3939066,76446] 0 2024-04-25T00:18:21.206 INFO:tasks.workunit.client.0.smithi107.stdout:2/60: dwrite d6/f7 [0,4194304] 0 2024-04-25T00:18:21.206 INFO:tasks.workunit.client.0.smithi107.stdout:2/61: write d6/ff [831334,30491] 0 2024-04-25T00:18:21.216 INFO:tasks.workunit.client.0.smithi107.stdout:7/38: getdents d0/d8 0 2024-04-25T00:18:21.217 INFO:tasks.workunit.client.0.smithi107.stdout:4/36: fsync d3/d5/f7 0 2024-04-25T00:18:21.218 INFO:tasks.workunit.client.0.smithi107.stdout:1/12: dwrite f1 [0,4194304] 0 2024-04-25T00:18:21.218 INFO:tasks.workunit.client.0.smithi107.stdout:3/54: truncate f7 1625582 0 2024-04-25T00:18:21.218 INFO:tasks.workunit.client.0.smithi107.stdout:6/64: dwrite d1/d2/d3/f6 [0,4194304] 0 2024-04-25T00:18:21.222 INFO:tasks.workunit.client.0.smithi107.stdout:5/26: creat d4/f7 x:0 0 0 2024-04-25T00:18:21.223 INFO:tasks.workunit.client.0.smithi107.stdout:9/35: dwrite d0/d1/f7 [0,4194304] 0 2024-04-25T00:18:21.230 INFO:tasks.workunit.client.0.smithi107.stdout:5/27: dread f1 [0,4194304] 0 2024-04-25T00:18:21.230 INFO:tasks.workunit.client.0.smithi107.stdout:5/28: chown d4/f7 247408 1 2024-04-25T00:18:21.236 INFO:tasks.workunit.client.0.smithi107.stdout:5/29: write f1 [412255,102897] 0 2024-04-25T00:18:21.240 INFO:tasks.workunit.client.0.smithi107.stdout:2/62: rename d6/cc to d6/c16 0 2024-04-25T00:18:21.240 INFO:tasks.workunit.client.0.smithi107.stdout:2/63: fdatasync d6/f11 0 2024-04-25T00:18:21.252 INFO:tasks.workunit.client.0.smithi107.stdout:1/13: mknod d2/c3 0 2024-04-25T00:18:21.254 INFO:tasks.workunit.client.0.smithi107.stdout:3/55: mknod d9/ce 0 2024-04-25T00:18:21.264 INFO:tasks.workunit.client.0.smithi107.stdout:9/36: rename d0/d1/c6 to d0/cb 0 2024-04-25T00:18:21.265 INFO:tasks.workunit.client.0.smithi107.stdout:8/48: mknod d2/cd 0 2024-04-25T00:18:21.267 INFO:tasks.workunit.client.0.smithi107.stdout:0/17: truncate f1 2035023 0 2024-04-25T00:18:21.268 INFO:tasks.workunit.client.0.smithi107.stdout:3/56: write f0 [6349989,74759] 0 2024-04-25T00:18:21.269 INFO:tasks.workunit.client.0.smithi107.stdout:3/57: truncate f7 1908518 0 2024-04-25T00:18:21.271 INFO:tasks.workunit.client.0.smithi107.stdout:2/64: creat d6/f17 x:0 0 0 2024-04-25T00:18:21.273 INFO:tasks.workunit.client.0.smithi107.stdout:1/14: creat d2/f4 x:0 0 0 2024-04-25T00:18:21.276 INFO:tasks.workunit.client.0.smithi107.stdout:2/65: dread d6/f14 [0,4194304] 0 2024-04-25T00:18:21.276 INFO:tasks.workunit.client.0.smithi107.stdout:9/37: creat d0/fc x:0 0 0 2024-04-25T00:18:21.276 INFO:tasks.workunit.client.0.smithi107.stdout:8/49: write f0 [8168527,48900] 0 2024-04-25T00:18:21.278 INFO:tasks.workunit.client.0.smithi107.stdout:3/58: creat d9/ff x:0 0 0 2024-04-25T00:18:21.279 INFO:tasks.workunit.client.0.smithi107.stdout:5/30: mknod d4/c8 0 2024-04-25T00:18:21.280 INFO:tasks.workunit.client.0.smithi107.stdout:5/31: truncate d4/f7 491114 0 2024-04-25T00:18:21.280 INFO:tasks.workunit.client.0.smithi107.stdout:2/66: mkdir d6/d18 0 2024-04-25T00:18:21.280 INFO:tasks.workunit.client.0.smithi107.stdout:2/67: dread - d6/fe zero size 2024-04-25T00:18:21.281 INFO:tasks.workunit.client.0.smithi107.stdout:6/65: dwrite d1/d2/db/fd [0,4194304] 0 2024-04-25T00:18:21.281 INFO:tasks.workunit.client.0.smithi107.stdout:9/38: creat d0/d1/fd x:0 0 0 2024-04-25T00:18:21.281 INFO:tasks.workunit.client.0.smithi107.stdout:9/39: fdatasync d0/d1/f3 0 2024-04-25T00:18:21.283 INFO:tasks.workunit.client.0.smithi107.stdout:1/15: read f1 [3653764,60829] 0 2024-04-25T00:18:21.283 INFO:tasks.workunit.client.0.smithi107.stdout:1/16: write d2/f4 [256705,33082] 0 2024-04-25T00:18:21.283 INFO:tasks.workunit.client.0.smithi107.stdout:3/59: creat d9/f10 x:0 0 0 2024-04-25T00:18:21.285 INFO:tasks.workunit.client.0.smithi107.stdout:2/68: read d6/f10 [748869,107216] 0 2024-04-25T00:18:21.285 INFO:tasks.workunit.client.0.smithi107.stdout:2/69: truncate d6/f17 289768 0 2024-04-25T00:18:21.287 INFO:tasks.workunit.client.0.smithi107.stdout:9/40: mkdir d0/d8/de 0 2024-04-25T00:18:21.289 INFO:tasks.workunit.client.0.smithi107.stdout:8/50: symlink d2/le 0 2024-04-25T00:18:21.290 INFO:tasks.workunit.client.0.smithi107.stdout:3/60: creat d9/f11 x:0 0 0 2024-04-25T00:18:21.291 INFO:tasks.workunit.client.0.smithi107.stdout:9/41: mknod d0/d8/de/cf 0 2024-04-25T00:18:21.294 INFO:tasks.workunit.client.0.smithi107.stdout:3/61: symlink d9/l12 0 2024-04-25T00:18:21.295 INFO:tasks.workunit.client.0.smithi107.stdout:9/42: creat d0/d1/f10 x:0 0 0 2024-04-25T00:18:21.304 INFO:tasks.workunit.client.0.smithi107.stdout:2/70: dwrite d6/fe [0,4194304] 0 2024-04-25T00:18:21.304 INFO:tasks.workunit.client.0.smithi107.stdout:8/51: dread f0 [4194304,4194304] 0 2024-04-25T00:18:21.308 INFO:tasks.workunit.client.0.smithi107.stdout:2/71: dread d6/ff [0,4194304] 0 2024-04-25T00:18:21.311 INFO:tasks.workunit.client.0.smithi107.stdout:2/72: mkdir d6/d18/d19 0 2024-04-25T00:18:21.311 INFO:tasks.workunit.client.0.smithi107.stdout:2/73: dread - d6/f13 zero size 2024-04-25T00:18:21.314 INFO:tasks.workunit.client.0.smithi107.stdout:5/32: truncate f1 1012929 0 2024-04-25T00:18:21.317 INFO:tasks.workunit.client.0.smithi107.stdout:3/62: dwrite d9/ff [0,4194304] 0 2024-04-25T00:18:21.318 INFO:tasks.workunit.client.0.smithi107.stdout:3/63: fdatasync d9/f11 0 2024-04-25T00:18:21.320 INFO:tasks.workunit.client.0.smithi107.stdout:3/64: mknod d9/c13 0 2024-04-25T00:18:21.322 INFO:tasks.workunit.client.0.smithi107.stdout:3/65: creat d9/f14 x:0 0 0 2024-04-25T00:18:21.322 INFO:tasks.workunit.client.0.smithi107.stdout:8/52: dwrite d2/f8 [0,4194304] 0 2024-04-25T00:18:21.322 INFO:tasks.workunit.client.0.smithi107.stdout:2/74: write f4 [951441,121779] 0 2024-04-25T00:18:21.322 INFO:tasks.workunit.client.0.smithi107.stdout:9/43: dwrite d0/f2 [0,4194304] 0 2024-04-25T00:18:21.325 INFO:tasks.workunit.client.0.smithi107.stdout:8/53: creat d2/ff x:0 0 0 2024-04-25T00:18:21.330 INFO:tasks.workunit.client.0.smithi107.stdout:2/75: creat d6/d18/d19/f1a x:0 0 0 2024-04-25T00:18:21.331 INFO:tasks.workunit.client.0.smithi107.stdout:2/76: read d6/f14 [184089,92063] 0 2024-04-25T00:18:21.332 INFO:tasks.workunit.client.0.smithi107.stdout:0/18: truncate f1 452178 0 2024-04-25T00:18:21.337 INFO:tasks.workunit.client.0.smithi107.stdout:2/77: link d6/f10 d6/f1b 0 2024-04-25T00:18:21.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:21 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:21.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:21 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:21.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:21 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:21.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:21 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:21.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:21 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:21.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:21 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:21.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:21 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:21.340 INFO:tasks.workunit.client.0.smithi107.stdout:8/54: mknod d2/c10 0 2024-04-25T00:18:21.341 INFO:tasks.workunit.client.0.smithi107.stdout:7/39: truncate d0/f6 932700 0 2024-04-25T00:18:21.343 INFO:tasks.workunit.client.0.smithi107.stdout:0/19: unlink l4 0 2024-04-25T00:18:21.344 INFO:tasks.workunit.client.0.smithi107.stdout:7/40: mkdir d0/d8/db 0 2024-04-25T00:18:21.345 INFO:tasks.workunit.client.0.smithi107.stdout:9/44: truncate d0/f2 1360814 0 2024-04-25T00:18:21.348 INFO:tasks.workunit.client.0.smithi107.stdout:8/55: creat d2/f11 x:0 0 0 2024-04-25T00:18:21.350 INFO:tasks.workunit.client.0.smithi107.stdout:0/20: mknod c5 0 2024-04-25T00:18:21.350 INFO:tasks.workunit.client.0.smithi107.stdout:0/21: write f1 [364169,104546] 0 2024-04-25T00:18:21.351 INFO:tasks.workunit.client.0.smithi107.stdout:2/78: fsync d6/f1b 0 2024-04-25T00:18:21.351 INFO:tasks.workunit.client.0.smithi107.stdout:2/79: chown d6/f1b 51544648 1 2024-04-25T00:18:21.351 INFO:tasks.workunit.client.0.smithi107.stdout:9/45: symlink d0/d1/l11 0 2024-04-25T00:18:21.352 INFO:tasks.workunit.client.0.smithi107.stdout:7/41: unlink d0/c7 0 2024-04-25T00:18:21.354 INFO:tasks.workunit.client.0.smithi107.stdout:4/37: truncate f2 3994491 0 2024-04-25T00:18:21.356 INFO:tasks.workunit.client.0.smithi107.stdout:2/80: getdents d6 0 2024-04-25T00:18:21.356 INFO:tasks.workunit.client.0.smithi107.stdout:2/81: chown d6/d18/d19 170 1 2024-04-25T00:18:21.356 INFO:tasks.workunit.client.0.smithi107.stdout:2/82: stat d6/f13 0 2024-04-25T00:18:21.363 INFO:tasks.workunit.client.0.smithi107.stdout:8/56: rename f1 to d2/f12 0 2024-04-25T00:18:21.363 INFO:tasks.workunit.client.0.smithi107.stdout:8/57: write d2/f7 [936582,24696] 0 2024-04-25T00:18:21.364 INFO:tasks.workunit.client.0.smithi107.stdout:2/83: creat d6/d18/d19/f1c x:0 0 0 2024-04-25T00:18:21.367 INFO:tasks.workunit.client.0.smithi107.stdout:2/84: getdents d6/d18 0 2024-04-25T00:18:21.367 INFO:tasks.workunit.client.0.smithi107.stdout:2/85: fsync d6/f14 0 2024-04-25T00:18:21.367 INFO:tasks.workunit.client.0.smithi107.stdout:6/66: dwrite d1/d2/d3/f6 [0,4194304] 0 2024-04-25T00:18:21.369 INFO:tasks.workunit.client.0.smithi107.stdout:8/58: mknod d2/c13 0 2024-04-25T00:18:21.369 INFO:tasks.workunit.client.0.smithi107.stdout:8/59: write d2/f9 [5038570,62663] 0 2024-04-25T00:18:21.371 INFO:tasks.workunit.client.0.smithi107.stdout:2/86: creat d6/d18/d19/f1d x:0 0 0 2024-04-25T00:18:21.372 INFO:tasks.workunit.client.0.smithi107.stdout:9/46: write d0/f2 [1331727,58784] 0 2024-04-25T00:18:21.372 INFO:tasks.workunit.client.0.smithi107.stdout:9/47: stat d0 0 2024-04-25T00:18:21.372 INFO:tasks.workunit.client.0.smithi107.stdout:0/22: dwrite f1 [0,4194304] 0 2024-04-25T00:18:21.372 INFO:tasks.workunit.client.0.smithi107.stdout:9/48: read d0/d1/f3 [351227,51404] 0 2024-04-25T00:18:21.373 INFO:tasks.workunit.client.0.smithi107.stdout:6/67: mkdir d1/d14 0 2024-04-25T00:18:21.379 INFO:tasks.workunit.client.0.smithi107.stdout:2/87: creat d6/f1e x:0 0 0 2024-04-25T00:18:21.380 INFO:tasks.workunit.client.0.smithi107.stdout:9/49: symlink d0/d1/l12 0 2024-04-25T00:18:21.380 INFO:tasks.workunit.client.0.smithi107.stdout:9/50: chown d0/f2 136947 1 2024-04-25T00:18:21.382 INFO:tasks.workunit.client.0.smithi107.stdout:0/23: dread f1 [0,4194304] 0 2024-04-25T00:18:21.382 INFO:tasks.workunit.client.0.smithi107.stdout:8/60: rename d2/f12 to d2/f14 0 2024-04-25T00:18:21.383 INFO:tasks.workunit.client.0.smithi107.stdout:7/42: stat d0 0 2024-04-25T00:18:21.383 INFO:tasks.workunit.client.0.smithi107.stdout:7/43: write d0/f6 [1956131,12285] 0 2024-04-25T00:18:21.388 INFO:tasks.workunit.client.0.smithi107.stdout:2/88: symlink d6/d18/l1f 0 2024-04-25T00:18:21.388 INFO:tasks.workunit.client.0.smithi107.stdout:2/89: write d6/f1b [387887,27339] 0 2024-04-25T00:18:21.389 INFO:tasks.workunit.client.0.smithi107.stdout:9/51: link d0/la d0/d8/de/l13 0 2024-04-25T00:18:21.391 INFO:tasks.workunit.client.0.smithi107.stdout:0/24: symlink l6 0 2024-04-25T00:18:21.391 INFO:tasks.workunit.client.0.smithi107.stdout:7/44: dread d0/f6 [0,4194304] 0 2024-04-25T00:18:21.391 INFO:tasks.workunit.client.0.smithi107.stdout:7/45: truncate d0/f6 2015198 0 2024-04-25T00:18:21.393 INFO:tasks.workunit.client.0.smithi107.stdout:2/90: mkdir d6/d20 0 2024-04-25T00:18:21.394 INFO:tasks.workunit.client.0.smithi107.stdout:0/25: link l3 l7 0 2024-04-25T00:18:21.394 INFO:tasks.workunit.client.0.smithi107.stdout:0/26: readlink l6 0 2024-04-25T00:18:21.394 INFO:tasks.workunit.client.0.smithi107.stdout:0/27: chown l7 2 1 2024-04-25T00:18:21.394 INFO:tasks.workunit.client.0.smithi107.stdout:0/28: readlink l2 0 2024-04-25T00:18:21.395 INFO:tasks.workunit.client.0.smithi107.stdout:0/29: readlink l6 0 2024-04-25T00:18:21.395 INFO:tasks.workunit.client.0.smithi107.stdout:0/30: creat f8 x:0 0 0 2024-04-25T00:18:21.395 INFO:tasks.workunit.client.0.smithi107.stdout:0/31: write f8 [545747,55399] 0 2024-04-25T00:18:21.396 INFO:tasks.workunit.client.0.smithi107.stdout:0/32: symlink l9 0 2024-04-25T00:18:21.397 INFO:tasks.workunit.client.0.smithi107.stdout:0/33: symlink la 0 2024-04-25T00:18:21.399 INFO:tasks.workunit.client.0.smithi107.stdout:4/38: rename f2 to d3/d5/f8 0 2024-04-25T00:18:21.399 INFO:tasks.workunit.client.0.smithi107.stdout:4/39: stat d3 0 2024-04-25T00:18:21.400 INFO:tasks.workunit.client.0.smithi107.stdout:6/68: rename d1/d2/dc/cf to d1/d2/db/c15 0 2024-04-25T00:18:21.401 INFO:tasks.workunit.client.0.smithi107.stdout:4/40: creat d3/f9 x:0 0 0 2024-04-25T00:18:21.403 INFO:tasks.workunit.client.0.smithi107.stdout:4/41: mknod d3/d5/ca 0 2024-04-25T00:18:21.403 INFO:tasks.workunit.client.0.smithi107.stdout:4/42: read - d3/d5/f7 zero size 2024-04-25T00:18:21.405 INFO:tasks.workunit.client.0.smithi107.stdout:4/43: mknod d3/cb 0 2024-04-25T00:18:21.407 INFO:tasks.workunit.client.0.smithi107.stdout:0/34: write f1 [1305612,91935] 0 2024-04-25T00:18:21.407 INFO:tasks.workunit.client.0.smithi107.stdout:2/91: fsync d6/f10 0 2024-04-25T00:18:21.407 INFO:tasks.workunit.client.0.smithi107.stdout:9/52: dwrite d0/d1/f10 [0,4194304] 0 2024-04-25T00:18:21.412 INFO:tasks.workunit.client.0.smithi107.stdout:2/92: link d6/ff d6/d18/d19/f21 0 2024-04-25T00:18:21.415 INFO:tasks.workunit.client.0.smithi107.stdout:9/53: mknod d0/c14 0 2024-04-25T00:18:21.415 INFO:tasks.workunit.client.0.smithi107.stdout:7/46: rename d0/f6 to d0/d8/db/fc 0 2024-04-25T00:18:21.415 INFO:tasks.workunit.client.0.smithi107.stdout:7/47: readlink d0/d8/la 0 2024-04-25T00:18:21.420 INFO:tasks.workunit.client.0.smithi107.stdout:2/93: dread d6/f1b [0,4194304] 0 2024-04-25T00:18:21.422 INFO:tasks.workunit.client.0.smithi107.stdout:4/44: dread d3/d5/f8 [0,4194304] 0 2024-04-25T00:18:21.422 INFO:tasks.workunit.client.0.smithi107.stdout:9/54: dread d0/f2 [0,4194304] 0 2024-04-25T00:18:21.423 INFO:tasks.workunit.client.0.smithi107.stdout:6/69: dwrite d1/d2/db/fd [4194304,4194304] 0 2024-04-25T00:18:21.423 INFO:tasks.workunit.client.0.smithi107.stdout:4/45: creat d3/d5/fc x:0 0 0 2024-04-25T00:18:21.423 INFO:tasks.workunit.client.0.smithi107.stdout:9/55: mknod d0/c15 0 2024-04-25T00:18:21.424 INFO:tasks.workunit.client.0.smithi107.stdout:7/48: dread d0/f4 [0,4194304] 0 2024-04-25T00:18:21.425 INFO:tasks.workunit.client.0.smithi107.stdout:6/70: creat d1/d2/f16 x:0 0 0 2024-04-25T00:18:21.428 INFO:tasks.workunit.client.0.smithi107.stdout:2/94: dread f4 [0,4194304] 0 2024-04-25T00:18:21.429 INFO:tasks.workunit.client.0.smithi107.stdout:2/95: unlink d6/f13 0 2024-04-25T00:18:21.429 INFO:tasks.workunit.client.0.smithi107.stdout:6/71: mknod d1/d2/c17 0 2024-04-25T00:18:21.429 INFO:tasks.workunit.client.0.smithi107.stdout:6/72: write d1/d2/f16 [7878,52402] 0 2024-04-25T00:18:21.430 INFO:tasks.workunit.client.0.smithi107.stdout:7/49: read d0/f4 [3642855,64019] 0 2024-04-25T00:18:21.431 INFO:tasks.workunit.client.0.smithi107.stdout:4/46: rename d3/d5/f8 to d3/fd 0 2024-04-25T00:18:21.431 INFO:tasks.workunit.client.0.smithi107.stdout:9/56: rename d0/d8/de to d0/d8/de/d16 22 2024-04-25T00:18:21.431 INFO:tasks.workunit.client.0.smithi107.stdout:6/73: unlink d1/d2/f16 0 2024-04-25T00:18:21.432 INFO:tasks.workunit.client.0.smithi107.stdout:9/57: symlink d0/d8/de/l17 0 2024-04-25T00:18:21.433 INFO:tasks.workunit.client.0.smithi107.stdout:7/50: creat d0/d8/fd x:0 0 0 2024-04-25T00:18:21.433 INFO:tasks.workunit.client.0.smithi107.stdout:9/58: rmdir d0/d8/de 39 2024-04-25T00:18:21.434 INFO:tasks.workunit.client.0.smithi107.stdout:6/74: truncate d1/d2/db/f11 1399692 0 2024-04-25T00:18:21.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:21 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:21.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:21 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:21.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:21 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:21.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:21 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:21.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:21 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:21.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:21 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:21.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:21 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:21.435 INFO:tasks.workunit.client.0.smithi107.stdout:4/47: link l1 d3/le 0 2024-04-25T00:18:21.436 INFO:tasks.workunit.client.0.smithi107.stdout:4/48: symlink d3/d5/lf 0 2024-04-25T00:18:21.437 INFO:tasks.workunit.client.0.smithi107.stdout:2/96: rename d6/d18/d19 to d6/d20/d22 0 2024-04-25T00:18:21.437 INFO:tasks.workunit.client.0.smithi107.stdout:2/97: truncate d6/f11 215658 0 2024-04-25T00:18:21.438 INFO:tasks.workunit.client.0.smithi107.stdout:9/59: symlink d0/l18 0 2024-04-25T00:18:21.438 INFO:tasks.workunit.client.0.smithi107.stdout:9/60: readlink d0/d8/de/l13 0 2024-04-25T00:18:21.439 INFO:tasks.workunit.client.0.smithi107.stdout:4/49: mkdir d3/d10 0 2024-04-25T00:18:21.439 INFO:tasks.workunit.client.0.smithi107.stdout:4/50: fsync d3/d5/f7 0 2024-04-25T00:18:21.439 INFO:tasks.workunit.client.0.smithi107.stdout:4/51: write d3/d5/fc [66726,35545] 0 2024-04-25T00:18:21.439 INFO:tasks.workunit.client.0.smithi107.stdout:6/75: mkdir d1/d14/d18 0 2024-04-25T00:18:21.439 INFO:tasks.workunit.client.0.smithi107.stdout:4/52: creat d3/d10/f11 x:0 0 0 2024-04-25T00:18:21.445 INFO:tasks.workunit.client.0.smithi107.stdout:4/53: truncate d3/fd 338233 0 2024-04-25T00:18:21.445 INFO:tasks.workunit.client.0.smithi107.stdout:4/54: read - d3/d10/f11 zero size 2024-04-25T00:18:21.446 INFO:tasks.workunit.client.0.smithi107.stdout:4/55: creat d3/d5/f12 x:0 0 0 2024-04-25T00:18:21.448 INFO:tasks.workunit.client.0.smithi107.stdout:4/56: link f0 d3/d10/f13 0 2024-04-25T00:18:21.448 INFO:tasks.workunit.client.0.smithi107.stdout:4/57: truncate d3/d5/f7 208303 0 2024-04-25T00:18:21.448 INFO:tasks.workunit.client.0.smithi107.stdout:4/58: write d3/d5/f12 [887101,53344] 0 2024-04-25T00:18:21.448 INFO:tasks.workunit.client.0.smithi107.stdout:4/59: chown d3/d10/f13 202166 1 2024-04-25T00:18:21.448 INFO:tasks.workunit.client.0.smithi107.stdout:4/60: chown d3/d5/f7 3 1 2024-04-25T00:18:21.449 INFO:tasks.workunit.client.0.smithi107.stdout:4/61: mknod d3/d5/c14 0 2024-04-25T00:18:21.449 INFO:tasks.workunit.client.0.smithi107.stdout:4/62: chown d3/d5/c14 3566762 1 2024-04-25T00:18:21.450 INFO:tasks.workunit.client.0.smithi107.stdout:4/63: creat d3/f15 x:0 0 0 2024-04-25T00:18:21.451 INFO:tasks.workunit.client.0.smithi107.stdout:7/51: dwrite d0/d8/fd [0,4194304] 0 2024-04-25T00:18:21.451 INFO:tasks.workunit.client.0.smithi107.stdout:7/52: chown d0/d8/db/fc 402 1 2024-04-25T00:18:21.451 INFO:tasks.workunit.client.0.smithi107.stdout:4/64: creat d3/f16 x:0 0 0 2024-04-25T00:18:21.453 INFO:tasks.workunit.client.0.smithi107.stdout:9/61: dread d0/d1/f7 [0,4194304] 0 2024-04-25T00:18:21.459 INFO:tasks.workunit.client.0.smithi107.stdout:7/53: dread d0/d8/db/fc [0,4194304] 0 2024-04-25T00:18:21.459 INFO:tasks.workunit.client.0.smithi107.stdout:7/54: truncate d0/d8/db/fc 2331014 0 2024-04-25T00:18:21.459 INFO:tasks.workunit.client.0.smithi107.stdout:2/98: dwrite d6/f14 [0,4194304] 0 2024-04-25T00:18:21.459 INFO:tasks.workunit.client.0.smithi107.stdout:2/99: fdatasync d6/d20/d22/f1d 0 2024-04-25T00:18:21.462 INFO:tasks.workunit.client.0.smithi107.stdout:2/100: symlink d6/l23 0 2024-04-25T00:18:21.470 INFO:tasks.workunit.client.0.smithi107.stdout:6/76: dwrite d1/d2/db/fd [0,4194304] 0 2024-04-25T00:18:21.471 INFO:tasks.workunit.client.0.smithi107.stdout:9/62: write d0/d1/f7 [288275,50803] 0 2024-04-25T00:18:21.472 INFO:tasks.workunit.client.0.smithi107.stdout:9/63: mknod d0/d1/c19 0 2024-04-25T00:18:21.473 INFO:tasks.workunit.client.0.smithi107.stdout:9/64: getdents d0/d8 0 2024-04-25T00:18:21.473 INFO:tasks.workunit.client.0.smithi107.stdout:9/65: chown d0/d1/fd 29024130 1 2024-04-25T00:18:21.474 INFO:tasks.workunit.client.0.smithi107.stdout:9/66: symlink d0/d1/l1a 0 2024-04-25T00:18:21.475 INFO:tasks.workunit.client.0.smithi107.stdout:9/67: dread - d0/d1/f4 zero size 2024-04-25T00:18:21.476 INFO:tasks.workunit.client.0.smithi107.stdout:9/68: mkdir d0/d1b 0 2024-04-25T00:18:21.476 INFO:tasks.workunit.client.0.smithi107.stdout:9/69: dread - d0/fc zero size 2024-04-25T00:18:21.477 INFO:tasks.workunit.client.0.smithi107.stdout:6/77: rmdir d1/d2/db 39 2024-04-25T00:18:21.477 INFO:tasks.workunit.client.0.smithi107.stdout:9/70: rmdir d0/d8/de 39 2024-04-25T00:18:21.478 INFO:tasks.workunit.client.0.smithi107.stdout:9/71: write d0/d1/f7 [2024376,11938] 0 2024-04-25T00:18:21.478 INFO:tasks.workunit.client.0.smithi107.stdout:9/72: getdents d0/d8/de 0 2024-04-25T00:18:21.479 INFO:tasks.workunit.client.0.smithi107.stdout:6/78: creat d1/d2/db/f19 x:0 0 0 2024-04-25T00:18:21.480 INFO:tasks.workunit.client.0.smithi107.stdout:9/73: creat d0/d1/f1c x:0 0 0 2024-04-25T00:18:21.481 INFO:tasks.workunit.client.0.smithi107.stdout:9/74: mknod d0/d1b/c1d 0 2024-04-25T00:18:21.481 INFO:tasks.workunit.client.0.smithi107.stdout:9/75: dread - d0/d1/f1c zero size 2024-04-25T00:18:21.481 INFO:tasks.workunit.client.0.smithi107.stdout:6/79: symlink d1/d2/dc/l1a 0 2024-04-25T00:18:21.481 INFO:tasks.workunit.client.0.smithi107.stdout:9/76: truncate d0/d1/f4 760135 0 2024-04-25T00:18:21.482 INFO:tasks.workunit.client.0.smithi107.stdout:6/80: creat d1/d14/f1b x:0 0 0 2024-04-25T00:18:21.483 INFO:tasks.workunit.client.0.smithi107.stdout:9/77: link d0/la d0/d8/l1e 0 2024-04-25T00:18:21.483 INFO:tasks.workunit.client.0.smithi107.stdout:9/78: write d0/d1/fd [542760,5159] 0 2024-04-25T00:18:21.483 INFO:tasks.workunit.client.0.smithi107.stdout:9/79: fdatasync d0/f2 0 2024-04-25T00:18:21.483 INFO:tasks.workunit.client.0.smithi107.stdout:9/80: stat d0/c15 0 2024-04-25T00:18:21.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:21 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:21.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:21 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:21.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:21 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:21.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:21 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:21.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:21 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:21.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:21 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:21.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:21 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:21.485 INFO:tasks.workunit.client.0.smithi107.stdout:9/81: mkdir d0/d8/d1f 0 2024-04-25T00:18:21.491 INFO:tasks.workunit.client.0.smithi107.stdout:6/81: read d1/d2/d3/f6 [845602,73967] 0 2024-04-25T00:18:21.496 INFO:tasks.workunit.client.0.smithi107.stdout:6/82: unlink d1/d2/d3/f6 0 2024-04-25T00:18:21.498 INFO:tasks.workunit.client.0.smithi107.stdout:6/83: mknod d1/d2/db/c1c 0 2024-04-25T00:18:21.498 INFO:tasks.workunit.client.0.smithi107.stdout:6/84: read - d1/d14/f1b zero size 2024-04-25T00:18:21.498 INFO:tasks.workunit.client.0.smithi107.stdout:6/85: write d1/d2/d3/f13 [454570,35085] 0 2024-04-25T00:18:21.499 INFO:tasks.workunit.client.0.smithi107.stdout:6/86: mkdir d1/d2/dc/d1d 0 2024-04-25T00:18:21.500 INFO:tasks.workunit.client.0.smithi107.stdout:6/87: mknod d1/d2/db/c1e 0 2024-04-25T00:18:21.501 INFO:tasks.workunit.client.0.smithi107.stdout:6/88: creat d1/f1f x:0 0 0 2024-04-25T00:18:21.502 INFO:tasks.workunit.client.0.smithi107.stdout:6/89: creat d1/d2/dc/d1d/f20 x:0 0 0 2024-04-25T00:18:21.502 INFO:tasks.workunit.client.0.smithi107.stdout:9/82: dwrite d0/d1/f7 [0,4194304] 0 2024-04-25T00:18:21.504 INFO:tasks.workunit.client.0.smithi107.stdout:6/90: mkdir d1/d2/d3/d21 0 2024-04-25T00:18:21.520 INFO:tasks.workunit.client.0.smithi107.stdout:9/83: rename d0/d1 to d0/d8/de/d20 0 2024-04-25T00:18:21.521 INFO:tasks.workunit.client.0.smithi107.stdout:7/55: dwrite d0/f4 [4194304,4194304] 0 2024-04-25T00:18:21.522 INFO:tasks.workunit.client.0.smithi107.stdout:9/84: creat d0/d8/de/f21 x:0 0 0 2024-04-25T00:18:21.523 INFO:tasks.workunit.client.0.smithi107.stdout:9/85: mknod d0/d1b/c22 0 2024-04-25T00:18:21.523 INFO:tasks.workunit.client.0.smithi107.stdout:6/91: dwrite d1/d2/db/f11 [0,4194304] 0 2024-04-25T00:18:21.524 INFO:tasks.workunit.client.0.smithi107.stdout:2/101: dwrite d6/ff [0,4194304] 0 2024-04-25T00:18:21.526 INFO:tasks.workunit.client.0.smithi107.stdout:9/86: rename d0/d8/de/l13 to d0/d1b/l23 0 2024-04-25T00:18:21.527 INFO:tasks.workunit.client.0.smithi107.stdout:9/87: readlink d0/d8/de/d20/l11 0 2024-04-25T00:18:21.527 INFO:tasks.workunit.client.0.smithi107.stdout:2/102: dread d6/f8 [0,4194304] 0 2024-04-25T00:18:21.527 INFO:tasks.workunit.client.0.smithi107.stdout:2/103: write f4 [4806428,113644] 0 2024-04-25T00:18:21.529 INFO:tasks.workunit.client.0.smithi107.stdout:6/92: fsync d1/f1f 0 2024-04-25T00:18:21.532 INFO:tasks.workunit.client.0.smithi107.stdout:7/56: truncate d0/f4 2105491 0 2024-04-25T00:18:21.536 INFO:tasks.workunit.client.0.smithi107.stdout:7/57: write d0/d8/db/fc [417718,44717] 0 2024-04-25T00:18:21.536 INFO:tasks.workunit.client.0.smithi107.stdout:2/104: mknod d6/d20/d22/c24 0 2024-04-25T00:18:21.537 INFO:tasks.workunit.client.0.smithi107.stdout:6/93: unlink d1/d14/f1b 0 2024-04-25T00:18:21.538 INFO:tasks.workunit.client.0.smithi107.stdout:2/105: symlink d6/l25 0 2024-04-25T00:18:21.539 INFO:tasks.workunit.client.0.smithi107.stdout:6/94: creat d1/d2/d3/f22 x:0 0 0 2024-04-25T00:18:21.539 INFO:tasks.workunit.client.0.smithi107.stdout:2/106: truncate d6/ff 5133637 0 2024-04-25T00:18:21.539 INFO:tasks.workunit.client.0.smithi107.stdout:6/95: fdatasync d1/f1f 0 2024-04-25T00:18:21.539 INFO:tasks.workunit.client.0.smithi107.stdout:2/107: truncate d6/d20/d22/f1d 412846 0 2024-04-25T00:18:21.539 INFO:tasks.workunit.client.0.smithi107.stdout:2/108: fsync d6/f7 0 2024-04-25T00:18:21.553 INFO:tasks.workunit.client.0.smithi107.stdout:7/58: dwrite d0/f4 [0,4194304] 0 2024-04-25T00:18:21.563 INFO:tasks.workunit.client.0.smithi107.stdout:7/59: write d0/d8/fd [278141,124549] 0 2024-04-25T00:18:21.570 INFO:tasks.workunit.client.0.smithi107.stdout:1/17: sync 2024-04-25T00:18:21.571 INFO:tasks.workunit.client.0.smithi107.stdout:1/18: creat d2/f5 x:0 0 0 2024-04-25T00:18:21.577 INFO:tasks.workunit.client.0.smithi107.stdout:1/19: getdents d2 0 2024-04-25T00:18:21.578 INFO:tasks.workunit.client.0.smithi107.stdout:1/20: mknod d2/c6 0 2024-04-25T00:18:21.581 INFO:tasks.workunit.client.0.smithi107.stdout:7/60: dwrite d0/d8/db/fc [0,4194304] 0 2024-04-25T00:18:21.582 INFO:tasks.workunit.client.0.smithi107.stdout:1/21: getdents d2 0 2024-04-25T00:18:21.582 INFO:tasks.workunit.client.0.smithi107.stdout:7/61: mknod d0/d8/ce 0 2024-04-25T00:18:21.583 INFO:tasks.workunit.client.0.smithi107.stdout:7/62: read d0/d8/fd [3845026,22048] 0 2024-04-25T00:18:21.583 INFO:tasks.workunit.client.0.smithi107.stdout:1/22: write d2/f4 [49468,22610] 0 2024-04-25T00:18:21.584 INFO:tasks.workunit.client.0.smithi107.stdout:7/63: rename d0/d8/c9 to d0/d8/cf 0 2024-04-25T00:18:21.584 INFO:tasks.workunit.client.0.smithi107.stdout:1/23: rename d2 to d2/d7 22 2024-04-25T00:18:21.584 INFO:tasks.workunit.client.0.smithi107.stdout:1/24: chown d2/f5 0 1 2024-04-25T00:18:21.585 INFO:tasks.workunit.client.0.smithi107.stdout:1/25: rmdir d2 39 2024-04-25T00:18:21.585 INFO:tasks.workunit.client.0.smithi107.stdout:1/26: truncate d2/f4 1226520 0 2024-04-25T00:18:21.586 INFO:tasks.workunit.client.0.smithi107.stdout:7/64: rmdir d0/d8 39 2024-04-25T00:18:21.587 INFO:tasks.workunit.client.0.smithi107.stdout:1/27: mknod d2/c8 0 2024-04-25T00:18:21.587 INFO:tasks.workunit.client.0.smithi107.stdout:1/28: chown d2/c8 164321034 1 2024-04-25T00:18:21.587 INFO:tasks.workunit.client.0.smithi107.stdout:7/65: chown d0/d8/ce 16244839 1 2024-04-25T00:18:21.587 INFO:tasks.workunit.client.0.smithi107.stdout:7/66: readlink d0/d8/la 0 2024-04-25T00:18:21.587 INFO:tasks.workunit.client.0.smithi107.stdout:7/67: fsync d0/f4 0 2024-04-25T00:18:21.588 INFO:tasks.workunit.client.0.smithi107.stdout:1/29: creat d2/f9 x:0 0 0 2024-04-25T00:18:21.593 INFO:tasks.workunit.client.0.smithi107.stdout:1/30: dread d2/f4 [0,4194304] 0 2024-04-25T00:18:21.593 INFO:tasks.workunit.client.0.smithi107.stdout:1/31: write f1 [1743402,24659] 0 2024-04-25T00:18:21.603 INFO:tasks.workunit.client.0.smithi107.stdout:7/68: dwrite d0/d8/db/fc [4194304,4194304] 0 2024-04-25T00:18:21.611 INFO:tasks.workunit.client.0.smithi107.stdout:1/32: dwrite d2/f5 [0,4194304] 0 2024-04-25T00:18:21.612 INFO:tasks.workunit.client.0.smithi107.stdout:7/69: write d0/d8/db/fc [8144601,86664] 0 2024-04-25T00:18:21.613 INFO:tasks.workunit.client.0.smithi107.stdout:7/70: creat d0/d8/db/f10 x:0 0 0 2024-04-25T00:18:21.615 INFO:tasks.workunit.client.0.smithi107.stdout:7/71: creat d0/f11 x:0 0 0 2024-04-25T00:18:21.617 INFO:tasks.workunit.client.0.smithi107.stdout:7/72: rename d0/c5 to d0/d8/c12 0 2024-04-25T00:18:21.619 INFO:tasks.workunit.client.0.smithi107.stdout:7/73: link d0/f4 d0/f13 0 2024-04-25T00:18:21.619 INFO:tasks.workunit.client.0.smithi107.stdout:7/74: dread - d0/d8/db/f10 zero size 2024-04-25T00:18:21.623 INFO:tasks.workunit.client.0.smithi107.stdout:1/33: dread d2/f5 [0,4194304] 0 2024-04-25T00:18:21.624 INFO:tasks.workunit.client.0.smithi107.stdout:1/34: mkdir d2/da 0 2024-04-25T00:18:21.626 INFO:tasks.workunit.client.0.smithi107.stdout:1/35: rename d2/c8 to d2/cb 0 2024-04-25T00:18:21.626 INFO:tasks.workunit.client.0.smithi107.stdout:7/75: unlink d0/d8/c12 0 2024-04-25T00:18:21.627 INFO:tasks.workunit.client.0.smithi107.stdout:1/36: rename d2/f5 to d2/fc 0 2024-04-25T00:18:21.629 INFO:tasks.workunit.client.0.smithi107.stdout:1/37: mkdir d2/dd 0 2024-04-25T00:18:21.629 INFO:tasks.workunit.client.0.smithi107.stdout:1/38: chown d2/da 3 1 2024-04-25T00:18:21.632 INFO:tasks.workunit.client.0.smithi107.stdout:1/39: link d2/cb d2/ce 0 2024-04-25T00:18:21.634 INFO:tasks.workunit.client.0.smithi107.stdout:1/40: symlink d2/da/lf 0 2024-04-25T00:18:21.634 INFO:tasks.workunit.client.0.smithi107.stdout:1/41: fsync d2/f4 0 2024-04-25T00:18:21.636 INFO:tasks.workunit.client.0.smithi107.stdout:1/42: rename d2/f4 to d2/dd/f10 0 2024-04-25T00:18:21.638 INFO:tasks.workunit.client.0.smithi107.stdout:1/43: dread d2/dd/f10 [0,4194304] 0 2024-04-25T00:18:21.639 INFO:tasks.workunit.client.0.smithi107.stdout:1/44: symlink d2/da/l11 0 2024-04-25T00:18:21.639 INFO:tasks.workunit.client.0.smithi107.stdout:1/45: fsync d2/fc 0 2024-04-25T00:18:21.639 INFO:tasks.workunit.client.0.smithi107.stdout:1/46: chown d2/f9 10456 1 2024-04-25T00:18:21.640 INFO:tasks.workunit.client.0.smithi107.stdout:7/76: dwrite d0/f4 [0,4194304] 0 2024-04-25T00:18:21.641 INFO:tasks.workunit.client.0.smithi107.stdout:7/77: mknod d0/d8/c14 0 2024-04-25T00:18:21.657 INFO:tasks.workunit.client.0.smithi107.stdout:1/47: dwrite d2/f9 [0,4194304] 0 2024-04-25T00:18:21.659 INFO:tasks.workunit.client.0.smithi107.stdout:7/78: dwrite d0/f13 [0,4194304] 0 2024-04-25T00:18:21.661 INFO:tasks.workunit.client.0.smithi107.stdout:7/79: rename d0/f4 to d0/d8/db/f15 0 2024-04-25T00:18:21.661 INFO:tasks.workunit.client.0.smithi107.stdout:7/80: chown d0/d8/db/f10 21130 1 2024-04-25T00:18:21.666 INFO:tasks.workunit.client.0.smithi107.stdout:1/48: dread f1 [0,4194304] 0 2024-04-25T00:18:21.666 INFO:tasks.workunit.client.0.smithi107.stdout:1/49: stat d2/da/l11 0 2024-04-25T00:18:21.667 INFO:tasks.workunit.client.0.smithi107.stdout:1/50: truncate f1 3781623 0 2024-04-25T00:18:21.668 INFO:tasks.workunit.client.0.smithi107.stdout:1/51: truncate d2/dd/f10 512900 0 2024-04-25T00:18:21.668 INFO:tasks.workunit.client.0.smithi107.stdout:1/52: rename d2 to d2/da/d12 22 2024-04-25T00:18:21.669 INFO:tasks.workunit.client.0.smithi107.stdout:1/53: rmdir d2/da 39 2024-04-25T00:18:21.669 INFO:tasks.workunit.client.0.smithi107.stdout:7/81: read d0/f13 [815628,103914] 0 2024-04-25T00:18:21.670 INFO:tasks.workunit.client.0.smithi107.stdout:7/82: truncate d0/d8/db/fc 9416418 0 2024-04-25T00:18:21.670 INFO:tasks.workunit.client.0.smithi107.stdout:1/54: truncate d2/fc 1525019 0 2024-04-25T00:18:21.670 INFO:tasks.workunit.client.0.smithi107.stdout:1/55: chown d2/cb 3564368 1 2024-04-25T00:18:21.670 INFO:tasks.workunit.client.0.smithi107.stdout:1/56: creat d2/f13 x:0 0 0 2024-04-25T00:18:21.672 INFO:tasks.workunit.client.0.smithi107.stdout:1/57: symlink d2/da/l14 0 2024-04-25T00:18:21.678 INFO:tasks.workunit.client.0.smithi107.stdout:1/58: getdents d2 0 2024-04-25T00:18:21.678 INFO:tasks.workunit.client.0.smithi107.stdout:1/59: stat d2/cb 0 2024-04-25T00:18:21.688 INFO:tasks.workunit.client.0.smithi107.stdout:7/83: dwrite d0/d8/db/fc [4194304,4194304] 0 2024-04-25T00:18:21.690 INFO:tasks.workunit.client.0.smithi107.stdout:7/84: creat d0/f16 x:0 0 0 2024-04-25T00:18:21.691 INFO:tasks.workunit.client.0.smithi107.stdout:7/85: write d0/f11 [72730,66708] 0 2024-04-25T00:18:21.692 INFO:tasks.workunit.client.0.smithi107.stdout:7/86: mknod d0/c17 0 2024-04-25T00:18:21.693 INFO:tasks.workunit.client.0.smithi107.stdout:7/87: mkdir d0/d8/db/d18 0 2024-04-25T00:18:21.693 INFO:tasks.workunit.client.0.smithi107.stdout:1/60: dwrite d2/fc [0,4194304] 0 2024-04-25T00:18:21.694 INFO:tasks.workunit.client.0.smithi107.stdout:7/88: mkdir d0/d8/db/d18/d19 0 2024-04-25T00:18:21.694 INFO:tasks.workunit.client.0.smithi107.stdout:7/89: dread - d0/d8/db/f10 zero size 2024-04-25T00:18:21.694 INFO:tasks.workunit.client.0.smithi107.stdout:7/90: fdatasync d0/d8/db/f10 0 2024-04-25T00:18:21.694 INFO:tasks.workunit.client.0.smithi107.stdout:7/91: chown d0/d8/db/f10 0 1 2024-04-25T00:18:21.703 INFO:tasks.workunit.client.0.smithi107.stdout:1/61: dread d2/f9 [0,4194304] 0 2024-04-25T00:18:21.704 INFO:tasks.workunit.client.0.smithi107.stdout:1/62: rename d2/f9 to d2/da/f15 0 2024-04-25T00:18:21.705 INFO:tasks.workunit.client.0.smithi107.stdout:1/63: fsync d2/fc 0 2024-04-25T00:18:21.705 INFO:tasks.workunit.client.0.smithi107.stdout:1/64: chown d2/c3 56457 1 2024-04-25T00:18:21.705 INFO:tasks.workunit.client.0.smithi107.stdout:1/65: truncate d2/f13 601778 0 2024-04-25T00:18:21.707 INFO:tasks.workunit.client.0.smithi107.stdout:1/66: dread d2/dd/f10 [0,4194304] 0 2024-04-25T00:18:21.708 INFO:tasks.workunit.client.0.smithi107.stdout:1/67: dread d2/f13 [0,4194304] 0 2024-04-25T00:18:21.709 INFO:tasks.workunit.client.0.smithi107.stdout:1/68: rmdir d2 39 2024-04-25T00:18:21.715 INFO:tasks.workunit.client.0.smithi107.stdout:1/69: write d2/da/f15 [1163679,19302] 0 2024-04-25T00:18:21.716 INFO:tasks.workunit.client.0.smithi107.stdout:1/70: write d2/da/f15 [3671163,43890] 0 2024-04-25T00:18:21.717 INFO:tasks.workunit.client.0.smithi107.stdout:1/71: rename d2/ce to d2/c16 0 2024-04-25T00:18:21.719 INFO:tasks.workunit.client.0.smithi107.stdout:1/72: truncate d2/fc 3687706 0 2024-04-25T00:18:21.729 INFO:tasks.workunit.client.0.smithi107.stdout:3/66: sync 2024-04-25T00:18:21.729 INFO:tasks.workunit.client.0.smithi107.stdout:5/33: sync 2024-04-25T00:18:21.729 INFO:tasks.workunit.client.0.smithi107.stdout:8/61: sync 2024-04-25T00:18:21.730 INFO:tasks.workunit.client.0.smithi107.stdout:5/34: getdents d4 0 2024-04-25T00:18:21.732 INFO:tasks.workunit.client.0.smithi107.stdout:3/67: mkdir d9/d15 0 2024-04-25T00:18:21.734 INFO:tasks.workunit.client.0.smithi107.stdout:8/62: rename d2/c13 to d2/c15 0 2024-04-25T00:18:21.736 INFO:tasks.workunit.client.0.smithi107.stdout:5/35: symlink d4/l9 0 2024-04-25T00:18:21.737 INFO:tasks.workunit.client.0.smithi107.stdout:5/36: rmdir d4 39 2024-04-25T00:18:21.737 INFO:tasks.workunit.client.0.smithi107.stdout:1/73: write d2/fc [650751,42448] 0 2024-04-25T00:18:21.739 INFO:tasks.workunit.client.0.smithi107.stdout:5/37: creat d4/fa x:0 0 0 2024-04-25T00:18:21.739 INFO:tasks.workunit.client.0.smithi107.stdout:5/38: readlink l0 0 2024-04-25T00:18:21.740 INFO:tasks.workunit.client.0.smithi107.stdout:1/74: symlink d2/dd/l17 0 2024-04-25T00:18:21.744 INFO:tasks.workunit.client.0.smithi107.stdout:1/75: rmdir d2/da 39 2024-04-25T00:18:21.744 INFO:tasks.workunit.client.0.smithi107.stdout:1/76: rename d2/da to d2/da/d18 22 2024-04-25T00:18:21.746 INFO:tasks.workunit.client.0.smithi107.stdout:5/39: truncate d4/f7 330632 0 2024-04-25T00:18:21.748 INFO:tasks.workunit.client.0.smithi107.stdout:3/68: dwrite d9/f11 [0,4194304] 0 2024-04-25T00:18:21.749 INFO:tasks.workunit.client.0.smithi107.stdout:1/77: creat d2/da/f19 x:0 0 0 2024-04-25T00:18:21.749 INFO:tasks.workunit.client.0.smithi107.stdout:1/78: fsync d2/da/f15 0 2024-04-25T00:18:21.752 INFO:tasks.workunit.client.0.smithi107.stdout:5/40: creat d4/fb x:0 0 0 2024-04-25T00:18:21.753 INFO:tasks.workunit.client.0.smithi107.stdout:8/63: dwrite d2/f14 [0,4194304] 0 2024-04-25T00:18:21.756 INFO:tasks.workunit.client.0.smithi107.stdout:1/79: mkdir d2/dd/d1a 0 2024-04-25T00:18:21.756 INFO:tasks.workunit.client.0.smithi107.stdout:1/80: chown d2/da/lf 19020 1 2024-04-25T00:18:21.758 INFO:tasks.workunit.client.0.smithi107.stdout:1/81: mkdir d2/dd/d1b 0 2024-04-25T00:18:21.758 INFO:tasks.workunit.client.0.smithi107.stdout:1/82: truncate d2/da/f19 677414 0 2024-04-25T00:18:21.758 INFO:tasks.workunit.client.0.smithi107.stdout:1/83: write d2/da/f15 [4069166,123530] 0 2024-04-25T00:18:21.759 INFO:tasks.workunit.client.0.smithi107.stdout:5/41: mkdir d4/dc 0 2024-04-25T00:18:21.760 INFO:tasks.workunit.client.0.smithi107.stdout:1/84: creat d2/da/f1c x:0 0 0 2024-04-25T00:18:21.761 INFO:tasks.workunit.client.0.smithi107.stdout:1/85: chown d2/dd/f10 1 1 2024-04-25T00:18:21.761 INFO:tasks.workunit.client.0.smithi107.stdout:1/86: read - d2/da/f1c zero size 2024-04-25T00:18:21.761 INFO:tasks.workunit.client.0.smithi107.stdout:3/69: read d9/ff [1078797,108530] 0 2024-04-25T00:18:21.761 INFO:tasks.workunit.client.0.smithi107.stdout:1/87: rmdir d2 39 2024-04-25T00:18:21.762 INFO:tasks.workunit.client.0.smithi107.stdout:5/42: dread f1 [0,4194304] 0 2024-04-25T00:18:21.762 INFO:tasks.workunit.client.0.smithi107.stdout:5/43: write d4/fb [991417,38145] 0 2024-04-25T00:18:21.764 INFO:tasks.workunit.client.0.smithi107.stdout:5/44: creat d4/dc/fd x:0 0 0 2024-04-25T00:18:21.772 INFO:tasks.workunit.client.0.smithi107.stdout:8/64: dwrite d2/ff [0,4194304] 0 2024-04-25T00:18:21.777 INFO:tasks.workunit.client.0.smithi107.stdout:8/65: rename f0 to d2/f16 0 2024-04-25T00:18:21.782 INFO:tasks.workunit.client.0.smithi107.stdout:3/70: dwrite d9/f11 [0,4194304] 0 2024-04-25T00:18:21.782 INFO:tasks.workunit.client.0.smithi107.stdout:1/88: dwrite d2/dd/f10 [0,4194304] 0 2024-04-25T00:18:21.783 INFO:tasks.workunit.client.0.smithi107.stdout:3/71: creat d9/d15/f16 x:0 0 0 2024-04-25T00:18:21.784 INFO:tasks.workunit.client.0.smithi107.stdout:8/66: dread d2/f9 [0,4194304] 0 2024-04-25T00:18:21.786 INFO:tasks.workunit.client.0.smithi107.stdout:1/89: creat d2/dd/d1a/f1d x:0 0 0 2024-04-25T00:18:21.794 INFO:tasks.workunit.client.0.smithi107.stdout:3/72: dread d9/ff [0,4194304] 0 2024-04-25T00:18:21.796 INFO:tasks.workunit.client.0.smithi107.stdout:3/73: mkdir d9/d15/d17 0 2024-04-25T00:18:21.796 INFO:tasks.workunit.client.0.smithi107.stdout:3/74: fsync d9/ff 0 2024-04-25T00:18:21.799 INFO:tasks.workunit.client.0.smithi107.stdout:3/75: creat d9/d15/d17/f18 x:0 0 0 2024-04-25T00:18:21.799 INFO:tasks.workunit.client.0.smithi107.stdout:3/76: write d9/f14 [498391,9768] 0 2024-04-25T00:18:21.799 INFO:tasks.workunit.client.0.smithi107.stdout:8/67: write d2/f14 [715459,111317] 0 2024-04-25T00:18:21.799 INFO:tasks.workunit.client.0.smithi107.stdout:5/45: dwrite d4/fb [0,4194304] 0 2024-04-25T00:18:21.799 INFO:tasks.workunit.client.0.smithi107.stdout:5/46: chown d4/fb 59696405 1 2024-04-25T00:18:21.799 INFO:tasks.workunit.client.0.smithi107.stdout:5/47: fdatasync d4/fa 0 2024-04-25T00:18:21.800 INFO:tasks.workunit.client.0.smithi107.stdout:3/77: mknod d9/d15/d17/c19 0 2024-04-25T00:18:21.802 INFO:tasks.workunit.client.0.smithi107.stdout:5/48: creat d4/dc/fe x:0 0 0 2024-04-25T00:18:21.802 INFO:tasks.workunit.client.0.smithi107.stdout:5/49: fdatasync f1 0 2024-04-25T00:18:21.802 INFO:tasks.workunit.client.0.smithi107.stdout:5/50: rename d4 to d4/dc/df 22 2024-04-25T00:18:21.802 INFO:tasks.workunit.client.0.smithi107.stdout:5/51: write d4/dc/fe [769158,100012] 0 2024-04-25T00:18:21.803 INFO:tasks.workunit.client.0.smithi107.stdout:3/78: getdents d9 0 2024-04-25T00:18:21.804 INFO:tasks.workunit.client.0.smithi107.stdout:5/52: symlink d4/dc/l10 0 2024-04-25T00:18:21.807 INFO:tasks.workunit.client.0.smithi107.stdout:3/79: rmdir d9 39 2024-04-25T00:18:21.808 INFO:tasks.workunit.client.0.smithi107.stdout:3/80: chown f7 286 1 2024-04-25T00:18:21.809 INFO:tasks.workunit.client.0.smithi107.stdout:5/53: chown d4/l6 107 1 2024-04-25T00:18:21.810 INFO:tasks.workunit.client.0.smithi107.stdout:8/68: dread d2/fc [4194304,4194304] 0 2024-04-25T00:18:21.811 INFO:tasks.workunit.client.0.smithi107.stdout:3/81: mkdir d9/d15/d1a 0 2024-04-25T00:18:21.811 INFO:tasks.workunit.client.0.smithi107.stdout:3/82: dread - d9/d15/f16 zero size 2024-04-25T00:18:21.813 INFO:tasks.workunit.client.0.smithi107.stdout:3/83: mkdir d9/d15/d17/d1b 0 2024-04-25T00:18:21.815 INFO:tasks.workunit.client.0.smithi107.stdout:8/69: symlink d2/l17 0 2024-04-25T00:18:21.815 INFO:tasks.workunit.client.0.smithi107.stdout:8/70: chown d2/l17 1703 1 2024-04-25T00:18:21.816 INFO:tasks.workunit.client.0.smithi107.stdout:3/84: creat d9/d15/d1a/f1c x:0 0 0 2024-04-25T00:18:21.816 INFO:tasks.workunit.client.0.smithi107.stdout:1/90: dwrite d2/da/f15 [0,4194304] 0 2024-04-25T00:18:21.818 INFO:tasks.workunit.client.0.smithi107.stdout:1/91: mknod d2/dd/c1e 0 2024-04-25T00:18:21.821 INFO:tasks.workunit.client.0.smithi107.stdout:1/92: mkdir d2/dd/d1a/d1f 0 2024-04-25T00:18:21.821 INFO:tasks.workunit.client.0.smithi107.stdout:1/93: readlink d2/da/l14 0 2024-04-25T00:18:21.821 INFO:tasks.workunit.client.0.smithi107.stdout:1/94: rename d2 to d2/dd/d1a/d20 22 2024-04-25T00:18:21.823 INFO:tasks.workunit.client.0.smithi107.stdout:1/95: link d2/da/f19 d2/dd/f21 0 2024-04-25T00:18:21.824 INFO:tasks.workunit.client.0.smithi107.stdout:1/96: write d2/f13 [559567,102407] 0 2024-04-25T00:18:21.828 INFO:tasks.workunit.client.0.smithi107.stdout:8/71: dread d2/f8 [0,4194304] 0 2024-04-25T00:18:21.828 INFO:tasks.workunit.client.0.smithi107.stdout:3/85: dread d9/ff [0,4194304] 0 2024-04-25T00:18:21.831 INFO:tasks.workunit.client.0.smithi107.stdout:5/54: dwrite d4/fa [0,4194304] 0 2024-04-25T00:18:21.836 INFO:tasks.workunit.client.0.smithi107.stdout:5/55: creat d4/dc/f11 x:0 0 0 2024-04-25T00:18:21.837 INFO:tasks.workunit.client.0.smithi107.stdout:3/86: write d9/f11 [2245467,16233] 0 2024-04-25T00:18:21.839 INFO:tasks.workunit.client.0.smithi107.stdout:5/56: creat d4/dc/f12 x:0 0 0 2024-04-25T00:18:21.839 INFO:tasks.workunit.client.0.smithi107.stdout:5/57: readlink d4/dc/l10 0 2024-04-25T00:18:21.839 INFO:tasks.workunit.client.0.smithi107.stdout:5/58: write d4/dc/f11 [236507,25051] 0 2024-04-25T00:18:21.839 INFO:tasks.workunit.client.0.smithi107.stdout:5/59: fsync d4/fb 0 2024-04-25T00:18:21.841 INFO:tasks.workunit.client.0.smithi107.stdout:3/87: mknod d9/d15/d1a/c1d 0 2024-04-25T00:18:21.842 INFO:tasks.workunit.client.0.smithi107.stdout:5/60: rmdir d4 39 2024-04-25T00:18:21.842 INFO:tasks.workunit.client.0.smithi107.stdout:5/61: write d4/dc/fd [662530,18387] 0 2024-04-25T00:18:21.842 INFO:tasks.workunit.client.0.smithi107.stdout:5/62: write d4/dc/fd [1056758,130776] 0 2024-04-25T00:18:21.842 INFO:tasks.workunit.client.0.smithi107.stdout:5/63: write d4/dc/fe [1224199,112441] 0 2024-04-25T00:18:21.844 INFO:tasks.workunit.client.0.smithi107.stdout:3/88: creat d9/d15/d17/f1e x:0 0 0 2024-04-25T00:18:21.845 INFO:tasks.workunit.client.0.smithi107.stdout:5/64: rmdir d4 39 2024-04-25T00:18:21.846 INFO:tasks.workunit.client.0.smithi107.stdout:3/89: mknod d9/d15/d17/d1b/c1f 0 2024-04-25T00:18:21.848 INFO:tasks.workunit.client.0.smithi107.stdout:3/90: rmdir d9 39 2024-04-25T00:18:21.851 INFO:tasks.workunit.client.0.smithi107.stdout:8/72: truncate d2/f14 1335324 0 2024-04-25T00:18:21.851 INFO:tasks.workunit.client.0.smithi107.stdout:8/73: dread - d2/fb zero size 2024-04-25T00:18:21.851 INFO:tasks.workunit.client.0.smithi107.stdout:8/74: dread - d2/f11 zero size 2024-04-25T00:18:21.852 INFO:tasks.workunit.client.0.smithi107.stdout:8/75: stat d2/le 0 2024-04-25T00:18:21.852 INFO:tasks.workunit.client.0.smithi107.stdout:8/76: chown d2 77879244 1 2024-04-25T00:18:21.852 INFO:tasks.workunit.client.0.smithi107.stdout:5/65: getdents d4/dc 0 2024-04-25T00:18:21.854 INFO:tasks.workunit.client.0.smithi107.stdout:8/77: mkdir d2/d18 0 2024-04-25T00:18:21.856 INFO:tasks.workunit.client.0.smithi107.stdout:5/66: creat d4/dc/f13 x:0 0 0 2024-04-25T00:18:21.858 INFO:tasks.workunit.client.0.smithi107.stdout:8/78: symlink d2/l19 0 2024-04-25T00:18:21.859 INFO:tasks.workunit.client.0.smithi107.stdout:0/35: sync 2024-04-25T00:18:21.859 INFO:tasks.workunit.client.0.smithi107.stdout:4/65: sync 2024-04-25T00:18:21.860 INFO:tasks.workunit.client.0.smithi107.stdout:4/66: chown d3/fd 3333 1 2024-04-25T00:18:21.860 INFO:tasks.workunit.client.0.smithi107.stdout:8/79: mkdir d2/d18/d1a 0 2024-04-25T00:18:21.860 INFO:tasks.workunit.client.0.smithi107.stdout:8/80: write d2/f11 [120544,44073] 0 2024-04-25T00:18:21.862 INFO:tasks.workunit.client.0.smithi107.stdout:0/36: link l3 lb 0 2024-04-25T00:18:21.864 INFO:tasks.workunit.client.0.smithi107.stdout:8/81: mkdir d2/d18/d1a/d1b 0 2024-04-25T00:18:21.864 INFO:tasks.workunit.client.0.smithi107.stdout:8/82: fsync d2/fb 0 2024-04-25T00:18:21.865 INFO:tasks.workunit.client.0.smithi107.stdout:3/91: dwrite d9/d15/f16 [0,4194304] 0 2024-04-25T00:18:21.867 INFO:tasks.workunit.client.0.smithi107.stdout:8/83: unlink d2/le 0 2024-04-25T00:18:21.867 INFO:tasks.workunit.client.0.smithi107.stdout:0/37: dread f8 [0,4194304] 0 2024-04-25T00:18:21.867 INFO:tasks.workunit.client.0.smithi107.stdout:8/84: fsync d2/fb 0 2024-04-25T00:18:21.867 INFO:tasks.workunit.client.0.smithi107.stdout:0/38: truncate f1 4667596 0 2024-04-25T00:18:21.868 INFO:tasks.workunit.client.0.smithi107.stdout:0/39: write f8 [826103,65227] 0 2024-04-25T00:18:21.869 INFO:tasks.workunit.client.0.smithi107.stdout:3/92: creat d9/f20 x:0 0 0 2024-04-25T00:18:21.869 INFO:tasks.workunit.client.0.smithi107.stdout:3/93: fsync f7 0 2024-04-25T00:18:21.869 INFO:tasks.workunit.client.0.smithi107.stdout:3/94: chown d9/c13 8231800 1 2024-04-25T00:18:21.869 INFO:tasks.workunit.client.0.smithi107.stdout:3/95: read - d9/d15/d17/f1e zero size 2024-04-25T00:18:21.869 INFO:tasks.workunit.client.0.smithi107.stdout:0/40: mkdir dc 0 2024-04-25T00:18:21.871 INFO:tasks.workunit.client.0.smithi107.stdout:0/41: mkdir dc/dd 0 2024-04-25T00:18:21.873 INFO:tasks.workunit.client.0.smithi107.stdout:8/85: dread d2/f8 [0,4194304] 0 2024-04-25T00:18:21.873 INFO:tasks.workunit.client.0.smithi107.stdout:8/86: fdatasync d2/f16 0 2024-04-25T00:18:21.875 INFO:tasks.workunit.client.0.smithi107.stdout:8/87: link d2/f3 d2/d18/d1a/d1b/f1c 0 2024-04-25T00:18:21.876 INFO:tasks.workunit.client.0.smithi107.stdout:8/88: mknod d2/d18/c1d 0 2024-04-25T00:18:21.877 INFO:tasks.workunit.client.0.smithi107.stdout:0/42: dread f1 [0,4194304] 0 2024-04-25T00:18:21.878 INFO:tasks.workunit.client.0.smithi107.stdout:8/89: rename d2/fc to d2/d18/d1a/f1e 0 2024-04-25T00:18:21.882 INFO:tasks.workunit.client.0.smithi107.stdout:0/43: dread f8 [0,4194304] 0 2024-04-25T00:18:21.883 INFO:tasks.workunit.client.0.smithi107.stdout:0/44: write f8 [1272099,67409] 0 2024-04-25T00:18:21.884 INFO:tasks.workunit.client.0.smithi107.stdout:0/45: link c0 dc/dd/ce 0 2024-04-25T00:18:21.884 INFO:tasks.workunit.client.0.smithi107.stdout:0/46: chown l2 133821425 1 2024-04-25T00:18:21.889 INFO:tasks.workunit.client.0.smithi107.stdout:0/47: truncate f1 2575177 0 2024-04-25T00:18:21.890 INFO:tasks.workunit.client.0.smithi107.stdout:0/48: mknod dc/dd/cf 0 2024-04-25T00:18:21.892 INFO:tasks.workunit.client.0.smithi107.stdout:8/90: dwrite d2/f16 [0,4194304] 0 2024-04-25T00:18:21.902 INFO:tasks.workunit.client.0.smithi107.stdout:0/49: dread f1 [0,4194304] 0 2024-04-25T00:18:21.904 INFO:tasks.workunit.client.0.smithi107.stdout:0/50: creat dc/f10 x:0 0 0 2024-04-25T00:18:21.904 INFO:tasks.workunit.client.0.smithi107.stdout:8/91: write d2/f14 [3135055,9252] 0 2024-04-25T00:18:21.904 INFO:tasks.workunit.client.0.smithi107.stdout:0/51: rmdir dc/dd 39 2024-04-25T00:18:21.906 INFO:tasks.workunit.client.0.smithi107.stdout:8/92: rename d2/l6 to d2/d18/d1a/d1b/l1f 0 2024-04-25T00:18:21.906 INFO:tasks.workunit.client.0.smithi107.stdout:8/93: write d2/d18/d1a/d1b/f1c [2687017,128729] 0 2024-04-25T00:18:21.906 INFO:tasks.workunit.client.0.smithi107.stdout:8/94: fsync d2/ff 0 2024-04-25T00:18:21.908 INFO:tasks.workunit.client.0.smithi107.stdout:0/52: truncate f1 948399 0 2024-04-25T00:18:21.911 INFO:tasks.workunit.client.0.smithi107.stdout:0/53: creat dc/f11 x:0 0 0 2024-04-25T00:18:21.912 INFO:tasks.workunit.client.0.smithi107.stdout:0/54: symlink dc/l12 0 2024-04-25T00:18:21.913 INFO:tasks.workunit.client.0.smithi107.stdout:8/95: unlink d2/d18/d1a/d1b/l1f 0 2024-04-25T00:18:21.918 INFO:tasks.workunit.client.0.smithi107.stdout:6/96: sync 2024-04-25T00:18:21.918 INFO:tasks.workunit.client.0.smithi107.stdout:9/88: sync 2024-04-25T00:18:21.918 INFO:tasks.workunit.client.0.smithi107.stdout:6/97: dread - d1/f1f zero size 2024-04-25T00:18:21.918 INFO:tasks.workunit.client.0.smithi107.stdout:9/89: fdatasync d0/d8/de/f21 0 2024-04-25T00:18:21.919 INFO:tasks.workunit.client.0.smithi107.stdout:8/96: write d2/f9 [2753710,36823] 0 2024-04-25T00:18:21.919 INFO:tasks.workunit.client.0.smithi107.stdout:8/97: fdatasync d2/d18/d1a/d1b/f1c 0 2024-04-25T00:18:21.919 INFO:tasks.workunit.client.0.smithi107.stdout:8/98: readlink d2/l19 0 2024-04-25T00:18:21.919 INFO:tasks.workunit.client.0.smithi107.stdout:8/99: fsync d2/f16 0 2024-04-25T00:18:21.920 INFO:tasks.workunit.client.0.smithi107.stdout:6/98: write d1/d2/d3/f13 [321729,77733] 0 2024-04-25T00:18:21.921 INFO:tasks.workunit.client.0.smithi107.stdout:0/55: dread f1 [0,4194304] 0 2024-04-25T00:18:21.921 INFO:tasks.workunit.client.0.smithi107.stdout:0/56: chown lb 170232932 1 2024-04-25T00:18:21.921 INFO:tasks.workunit.client.0.smithi107.stdout:6/99: mkdir d1/d14/d18/d23 0 2024-04-25T00:18:21.921 INFO:tasks.workunit.client.0.smithi107.stdout:6/100: write d1/d2/db/f11 [4288539,94941] 0 2024-04-25T00:18:21.922 INFO:tasks.workunit.client.0.smithi107.stdout:9/90: dread d0/d8/de/d20/fd [0,4194304] 0 2024-04-25T00:18:21.923 INFO:tasks.workunit.client.0.smithi107.stdout:0/57: rename l7 to dc/l13 0 2024-04-25T00:18:21.924 INFO:tasks.workunit.client.0.smithi107.stdout:6/101: mknod d1/d14/d18/d23/c24 0 2024-04-25T00:18:21.924 INFO:tasks.workunit.client.0.smithi107.stdout:6/102: truncate d1/f1f 672023 0 2024-04-25T00:18:21.926 INFO:tasks.workunit.client.0.smithi107.stdout:9/91: getdents d0/d1b 0 2024-04-25T00:18:21.928 INFO:tasks.workunit.client.0.smithi107.stdout:0/58: creat dc/dd/f14 x:0 0 0 2024-04-25T00:18:21.928 INFO:tasks.workunit.client.0.smithi107.stdout:8/100: dread d2/d18/d1a/d1b/f1c [0,4194304] 0 2024-04-25T00:18:21.929 INFO:tasks.workunit.client.0.smithi107.stdout:9/92: link d0/d8/de/d20/c19 d0/c24 0 2024-04-25T00:18:21.929 INFO:tasks.workunit.client.0.smithi107.stdout:0/59: creat dc/dd/f15 x:0 0 0 2024-04-25T00:18:21.931 INFO:tasks.workunit.client.0.smithi107.stdout:9/93: link d0/c24 d0/d8/de/d20/c25 0 2024-04-25T00:18:21.931 INFO:tasks.workunit.client.0.smithi107.stdout:9/94: write d0/d8/de/d20/f1c [781077,38120] 0 2024-04-25T00:18:21.931 INFO:tasks.workunit.client.0.smithi107.stdout:9/95: write d0/d8/de/d20/f4 [572091,81200] 0 2024-04-25T00:18:21.932 INFO:tasks.workunit.client.0.smithi107.stdout:0/60: mkdir dc/dd/d16 0 2024-04-25T00:18:21.932 INFO:tasks.workunit.client.0.smithi107.stdout:0/61: dread - dc/dd/f15 zero size 2024-04-25T00:18:21.933 INFO:tasks.workunit.client.0.smithi107.stdout:0/62: creat dc/dd/f17 x:0 0 0 2024-04-25T00:18:21.944 INFO:tasks.workunit.client.0.smithi107.stdout:6/103: dwrite d1/d2/d3/f13 [0,4194304] 0 2024-04-25T00:18:21.955 INFO:tasks.workunit.client.0.smithi107.stdout:8/101: dwrite d2/f16 [0,4194304] 0 2024-04-25T00:18:21.959 INFO:tasks.workunit.client.0.smithi107.stdout:8/102: rename d2/f7 to d2/d18/f20 0 2024-04-25T00:18:21.961 INFO:tasks.workunit.client.0.smithi107.stdout:8/103: creat d2/d18/d1a/f21 x:0 0 0 2024-04-25T00:18:21.961 INFO:tasks.workunit.client.0.smithi107.stdout:8/104: chown d2/fb 1325310881 1 2024-04-25T00:18:21.961 INFO:tasks.workunit.client.0.smithi107.stdout:8/105: chown d2/ff 647127234 1 2024-04-25T00:18:21.962 INFO:tasks.workunit.client.0.smithi107.stdout:9/96: dwrite d0/d8/de/d20/f4 [0,4194304] 0 2024-04-25T00:18:21.963 INFO:tasks.workunit.client.0.smithi107.stdout:9/97: truncate d0/d8/de/d20/f3 964792 0 2024-04-25T00:18:21.965 INFO:tasks.workunit.client.0.smithi107.stdout:9/98: creat d0/d8/f26 x:0 0 0 2024-04-25T00:18:21.965 INFO:tasks.workunit.client.0.smithi107.stdout:9/99: getdents d0/d8/d1f 0 2024-04-25T00:18:21.965 INFO:tasks.workunit.client.0.smithi107.stdout:6/104: dwrite d1/d2/dc/d1d/f20 [0,4194304] 0 2024-04-25T00:18:21.966 INFO:tasks.workunit.client.0.smithi107.stdout:0/63: dwrite dc/dd/f14 [0,4194304] 0 2024-04-25T00:18:21.967 INFO:tasks.workunit.client.0.smithi107.stdout:6/105: symlink d1/d14/l25 0 2024-04-25T00:18:21.972 INFO:tasks.workunit.client.0.smithi107.stdout:6/106: rename d1/d2/d3/d21 to d1/d2/d3/d26 0 2024-04-25T00:18:21.972 INFO:tasks.workunit.client.0.smithi107.stdout:6/107: chown d1/d2/db/c1c 408288 1 2024-04-25T00:18:21.972 INFO:tasks.workunit.client.0.smithi107.stdout:6/108: fsync d1/d2/d3/f13 0 2024-04-25T00:18:21.974 INFO:tasks.workunit.client.0.smithi107.stdout:6/109: rename d1/d2/db/f11 to d1/d2/dc/f27 0 2024-04-25T00:18:21.974 INFO:tasks.workunit.client.0.smithi107.stdout:6/110: dread - d1/d2/d3/f22 zero size 2024-04-25T00:18:21.974 INFO:tasks.workunit.client.0.smithi107.stdout:8/106: read d2/f14 [3299853,42574] 0 2024-04-25T00:18:21.976 INFO:tasks.workunit.client.0.smithi107.stdout:8/107: getdents d2/d18 0 2024-04-25T00:18:21.978 INFO:tasks.workunit.client.0.smithi107.stdout:8/108: mknod d2/d18/d1a/c22 0 2024-04-25T00:18:21.978 INFO:tasks.workunit.client.0.smithi107.stdout:8/109: rmdir d2/d18/d1a 39 2024-04-25T00:18:21.979 INFO:tasks.workunit.client.0.smithi107.stdout:8/110: readlink d2/l19 0 2024-04-25T00:18:21.979 INFO:tasks.workunit.client.0.smithi107.stdout:8/111: rmdir d2/d18/d1a 39 2024-04-25T00:18:21.979 INFO:tasks.workunit.client.0.smithi107.stdout:8/112: chown d2 789587 1 2024-04-25T00:18:21.980 INFO:tasks.workunit.client.0.smithi107.stdout:8/113: creat d2/f23 x:0 0 0 2024-04-25T00:18:21.982 INFO:tasks.workunit.client.0.smithi107.stdout:8/114: mknod d2/d18/d1a/d1b/c24 0 2024-04-25T00:18:21.984 INFO:tasks.workunit.client.0.smithi107.stdout:9/100: dwrite d0/fc [0,4194304] 0 2024-04-25T00:18:21.986 INFO:tasks.workunit.client.0.smithi107.stdout:9/101: truncate d0/d8/de/d20/f10 1271296 0 2024-04-25T00:18:21.986 INFO:tasks.workunit.client.0.smithi107.stdout:9/102: write d0/d8/de/d20/f1c [1784759,59595] 0 2024-04-25T00:18:21.987 INFO:tasks.workunit.client.0.smithi107.stdout:0/64: dwrite dc/f10 [0,4194304] 0 2024-04-25T00:18:21.987 INFO:tasks.workunit.client.0.smithi107.stdout:8/115: fdatasync d2/f9 0 2024-04-25T00:18:21.991 INFO:tasks.workunit.client.0.smithi107.stdout:0/65: unlink dc/f10 0 2024-04-25T00:18:21.991 INFO:tasks.workunit.client.0.smithi107.stdout:0/66: chown f1 39 1 2024-04-25T00:18:21.992 INFO:tasks.workunit.client.0.smithi107.stdout:0/67: rmdir dc/dd 39 2024-04-25T00:18:21.994 INFO:tasks.workunit.client.0.smithi107.stdout:0/68: creat dc/f18 x:0 0 0 2024-04-25T00:18:21.997 INFO:tasks.workunit.client.0.smithi107.stdout:0/69: creat dc/dd/f19 x:0 0 0 2024-04-25T00:18:22.000 INFO:tasks.workunit.client.0.smithi107.stdout:6/111: rmdir d1/d2 39 2024-04-25T00:18:22.002 INFO:tasks.workunit.client.0.smithi107.stdout:0/70: mkdir dc/dd/d16/d1a 0 2024-04-25T00:18:22.002 INFO:tasks.workunit.client.0.smithi107.stdout:0/71: fdatasync dc/f11 0 2024-04-25T00:18:22.002 INFO:tasks.workunit.client.0.smithi107.stdout:6/112: truncate d1/d2/dc/f27 1633666 0 2024-04-25T00:18:22.003 INFO:tasks.workunit.client.0.smithi107.stdout:6/113: read - d1/d2/db/f19 zero size 2024-04-25T00:18:22.003 INFO:tasks.workunit.client.0.smithi107.stdout:6/114: read - d1/d2/d3/f22 zero size 2024-04-25T00:18:22.003 INFO:tasks.workunit.client.0.smithi107.stdout:0/72: creat dc/dd/d16/f1b x:0 0 0 2024-04-25T00:18:22.004 INFO:tasks.workunit.client.0.smithi107.stdout:9/103: dwrite d0/f2 [0,4194304] 0 2024-04-25T00:18:22.009 INFO:tasks.workunit.client.0.smithi107.stdout:6/115: read d1/f1f [487597,126920] 0 2024-04-25T00:18:22.009 INFO:tasks.workunit.client.0.smithi107.stdout:6/116: chown d1/d2/db/fd 19605 1 2024-04-25T00:18:22.010 INFO:tasks.workunit.client.0.smithi107.stdout:6/117: getdents d1 0 2024-04-25T00:18:22.010 INFO:tasks.workunit.client.0.smithi107.stdout:6/118: dread - d1/d2/d3/f22 zero size 2024-04-25T00:18:22.013 INFO:tasks.workunit.client.0.smithi107.stdout:8/116: dwrite d2/f14 [0,4194304] 0 2024-04-25T00:18:22.014 INFO:tasks.workunit.client.0.smithi107.stdout:8/117: creat d2/d18/d1a/f25 x:0 0 0 2024-04-25T00:18:22.017 INFO:tasks.workunit.client.0.smithi107.stdout:8/118: rename d2/f14 to d2/f26 0 2024-04-25T00:18:22.017 INFO:tasks.workunit.client.0.smithi107.stdout:8/119: readlink d2/l17 0 2024-04-25T00:18:22.018 INFO:tasks.workunit.client.0.smithi107.stdout:0/73: dread dc/dd/f14 [0,4194304] 0 2024-04-25T00:18:22.019 INFO:tasks.workunit.client.0.smithi107.stdout:6/119: dread d1/f1f [0,4194304] 0 2024-04-25T00:18:22.021 INFO:tasks.workunit.client.0.smithi107.stdout:6/120: unlink d1/d2/dc/c12 0 2024-04-25T00:18:22.021 INFO:tasks.workunit.client.0.smithi107.stdout:0/74: unlink f8 0 2024-04-25T00:18:22.024 INFO:tasks.workunit.client.0.smithi107.stdout:0/75: mknod dc/c1c 0 2024-04-25T00:18:22.024 INFO:tasks.workunit.client.0.smithi107.stdout:8/120: getdents d2/d18/d1a 0 2024-04-25T00:18:22.026 INFO:tasks.workunit.client.0.smithi107.stdout:0/76: truncate dc/dd/f14 2435091 0 2024-04-25T00:18:22.027 INFO:tasks.workunit.client.0.smithi107.stdout:9/104: dwrite d0/d8/f26 [0,4194304] 0 2024-04-25T00:18:22.048 INFO:tasks.workunit.client.0.smithi107.stdout:8/121: dwrite d2/f3 [0,4194304] 0 2024-04-25T00:18:22.048 INFO:tasks.workunit.client.0.smithi107.stdout:9/105: dread d0/d8/de/d20/f7 [0,4194304] 0 2024-04-25T00:18:22.050 INFO:tasks.workunit.client.0.smithi107.stdout:9/106: creat d0/d1b/f27 x:0 0 0 2024-04-25T00:18:22.050 INFO:tasks.workunit.client.0.smithi107.stdout:8/122: write d2/f9 [4991076,113535] 0 2024-04-25T00:18:22.051 INFO:tasks.workunit.client.0.smithi107.stdout:9/107: creat d0/d8/de/d20/f28 x:0 0 0 2024-04-25T00:18:22.051 INFO:tasks.workunit.client.0.smithi107.stdout:9/108: dread - d0/d8/de/f21 zero size 2024-04-25T00:18:22.051 INFO:tasks.workunit.client.0.smithi107.stdout:9/109: chown d0/d8/de/d20/fd 6065770 1 2024-04-25T00:18:22.056 INFO:tasks.workunit.client.0.smithi107.stdout:8/123: dread d2/f9 [4194304,4194304] 0 2024-04-25T00:18:22.056 INFO:tasks.workunit.client.0.smithi107.stdout:8/124: chown d2/f9 17521861 1 2024-04-25T00:18:22.066 INFO:tasks.workunit.client.0.smithi107.stdout:9/110: dwrite d0/d8/de/d20/f4 [0,4194304] 0 2024-04-25T00:18:22.081 INFO:tasks.workunit.client.0.smithi107.stdout:9/111: dwrite d0/d8/de/d20/f4 [0,4194304] 0 2024-04-25T00:18:22.084 INFO:tasks.workunit.client.0.smithi107.stdout:9/112: symlink d0/l29 0 2024-04-25T00:18:22.085 INFO:tasks.workunit.client.0.smithi107.stdout:9/113: mkdir d0/d8/d1f/d2a 0 2024-04-25T00:18:22.089 INFO:tasks.workunit.client.0.smithi107.stdout:6/121: rename d1/d14/d18/d23 to d1/d2/d3/d28 0 2024-04-25T00:18:22.091 INFO:tasks.workunit.client.0.smithi107.stdout:6/122: mknod d1/d2/d3/c29 0 2024-04-25T00:18:22.094 INFO:tasks.workunit.client.0.smithi107.stdout:6/123: symlink d1/d2/d3/d28/l2a 0 2024-04-25T00:18:22.096 INFO:tasks.workunit.client.0.smithi107.stdout:6/124: rename d1/d2/dc to d1/d2/d2b 0 2024-04-25T00:18:22.097 INFO:tasks.workunit.client.0.smithi107.stdout:9/114: write d0/f2 [568916,130336] 0 2024-04-25T00:18:22.098 INFO:tasks.workunit.client.0.smithi107.stdout:9/115: mknod d0/d8/de/c2b 0 2024-04-25T00:18:22.103 INFO:tasks.workunit.client.0.smithi107.stdout:6/125: dread d1/d2/d2b/f27 [0,4194304] 0 2024-04-25T00:18:22.107 INFO:tasks.workunit.client.0.smithi107.stdout:9/116: read d0/d8/de/d20/f4 [4166436,31107] 0 2024-04-25T00:18:22.107 INFO:tasks.workunit.client.0.smithi107.stdout:6/126: dread d1/d2/d2b/f27 [0,4194304] 0 2024-04-25T00:18:22.107 INFO:tasks.workunit.client.0.smithi107.stdout:9/117: chown d0/f2 2028 1 2024-04-25T00:18:22.107 INFO:tasks.workunit.client.0.smithi107.stdout:6/127: truncate d1/d2/d3/f22 845194 0 2024-04-25T00:18:22.107 INFO:tasks.workunit.client.0.smithi107.stdout:9/118: write d0/d1b/f27 [147309,81953] 0 2024-04-25T00:18:22.107 INFO:tasks.workunit.client.0.smithi107.stdout:9/119: chown d0/f2 2 1 2024-04-25T00:18:22.107 INFO:tasks.workunit.client.0.smithi107.stdout:9/120: chown d0/d8/de/d20/f3 3 1 2024-04-25T00:18:22.108 INFO:tasks.workunit.client.0.smithi107.stdout:9/121: fsync d0/d8/de/d20/f10 0 2024-04-25T00:18:22.108 INFO:tasks.workunit.client.0.smithi107.stdout:9/122: rename d0/d8/de to d0/d8/de/d20/d2c 22 2024-04-25T00:18:22.108 INFO:tasks.workunit.client.0.smithi107.stdout:9/123: chown d0/cb 1 1 2024-04-25T00:18:22.108 INFO:tasks.workunit.client.0.smithi107.stdout:9/124: truncate d0/d8/de/f21 840137 0 2024-04-25T00:18:22.108 INFO:tasks.workunit.client.0.smithi107.stdout:9/125: chown d0/d8/de/d20/fd 387582 1 2024-04-25T00:18:22.108 INFO:tasks.workunit.client.0.smithi107.stdout:6/128: dread d1/d2/d3/f22 [0,4194304] 0 2024-04-25T00:18:22.108 INFO:tasks.workunit.client.0.smithi107.stdout:6/129: truncate d1/f1f 1665322 0 2024-04-25T00:18:22.117 INFO:tasks.workunit.client.0.smithi107.stdout:9/126: dread d0/d8/f26 [0,4194304] 0 2024-04-25T00:18:22.123 INFO:tasks.workunit.client.0.smithi107.stdout:6/130: rename d1/d2/d2b/d1d/f20 to d1/d14/d18/f2c 0 2024-04-25T00:18:22.126 INFO:tasks.workunit.client.0.smithi107.stdout:9/127: write d0/d8/f26 [3324339,108009] 0 2024-04-25T00:18:22.138 INFO:tasks.workunit.client.0.smithi107.stdout:1/97: sync 2024-04-25T00:18:22.138 INFO:tasks.workunit.client.0.smithi107.stdout:2/109: sync 2024-04-25T00:18:22.139 INFO:tasks.workunit.client.0.smithi107.stdout:5/67: sync 2024-04-25T00:18:22.139 INFO:tasks.workunit.client.0.smithi107.stdout:7/92: sync 2024-04-25T00:18:22.141 INFO:tasks.workunit.client.0.smithi107.stdout:2/110: link d6/f7 d6/d20/d22/f26 0 2024-04-25T00:18:22.141 INFO:tasks.workunit.client.0.smithi107.stdout:6/131: dwrite d1/f1f [0,4194304] 0 2024-04-25T00:18:22.142 INFO:tasks.workunit.client.0.smithi107.stdout:7/93: mknod d0/c1a 0 2024-04-25T00:18:22.142 INFO:tasks.workunit.client.0.smithi107.stdout:2/111: mknod d6/d20/c27 0 2024-04-25T00:18:22.145 INFO:tasks.workunit.client.0.smithi107.stdout:5/68: write d4/dc/fe [1064517,44257] 0 2024-04-25T00:18:22.145 INFO:tasks.workunit.client.0.smithi107.stdout:6/132: link d1/d2/d3/d28/l2a d1/d14/d18/l2d 0 2024-04-25T00:18:22.146 INFO:tasks.workunit.client.0.smithi107.stdout:9/128: dwrite d0/d8/de/d20/f1c [0,4194304] 0 2024-04-25T00:18:22.146 INFO:tasks.workunit.client.0.smithi107.stdout:2/112: rmdir d6/d20 39 2024-04-25T00:18:22.146 INFO:tasks.workunit.client.0.smithi107.stdout:2/113: write d6/d20/d22/f26 [4462861,18258] 0 2024-04-25T00:18:22.149 INFO:tasks.workunit.client.0.smithi107.stdout:7/94: dread d0/f13 [0,4194304] 0 2024-04-25T00:18:22.150 INFO:tasks.workunit.client.0.smithi107.stdout:6/133: dread d1/d2/d2b/f27 [0,4194304] 0 2024-04-25T00:18:22.151 INFO:tasks.workunit.client.0.smithi107.stdout:7/95: mknod d0/d8/db/d18/d19/c1b 0 2024-04-25T00:18:22.153 INFO:tasks.workunit.client.0.smithi107.stdout:2/114: rename d6/l25 to d6/d20/l28 0 2024-04-25T00:18:22.156 INFO:tasks.workunit.client.0.smithi107.stdout:1/98: dwrite d2/dd/f10 [0,4194304] 0 2024-04-25T00:18:22.156 INFO:tasks.workunit.client.0.smithi107.stdout:7/96: creat d0/f1c x:0 0 0 2024-04-25T00:18:22.158 INFO:tasks.workunit.client.0.smithi107.stdout:5/69: unlink d4/dc/f12 0 2024-04-25T00:18:22.159 INFO:tasks.workunit.client.0.smithi107.stdout:2/115: dread d6/f10 [0,4194304] 0 2024-04-25T00:18:22.159 INFO:tasks.workunit.client.0.smithi107.stdout:2/116: readlink d6/l12 0 2024-04-25T00:18:22.163 INFO:tasks.workunit.client.0.smithi107.stdout:7/97: dread d0/d8/db/f15 [0,4194304] 0 2024-04-25T00:18:22.163 INFO:tasks.workunit.client.0.smithi107.stdout:5/70: truncate d4/fb 3151318 0 2024-04-25T00:18:22.163 INFO:tasks.workunit.client.0.smithi107.stdout:9/129: dread d0/d8/f26 [0,4194304] 0 2024-04-25T00:18:22.164 INFO:tasks.workunit.client.0.smithi107.stdout:5/71: creat d4/dc/f14 x:0 0 0 2024-04-25T00:18:22.164 INFO:tasks.workunit.client.0.smithi107.stdout:5/72: read d4/f7 [80158,73714] 0 2024-04-25T00:18:22.165 INFO:tasks.workunit.client.0.smithi107.stdout:7/98: creat d0/f1d x:0 0 0 2024-04-25T00:18:22.168 INFO:tasks.workunit.client.0.smithi107.stdout:6/134: write d1/f1f [2847668,49734] 0 2024-04-25T00:18:22.168 INFO:tasks.workunit.client.0.smithi107.stdout:9/130: dread d0/d8/de/d20/fd [0,4194304] 0 2024-04-25T00:18:22.168 INFO:tasks.workunit.client.0.smithi107.stdout:5/73: truncate d4/dc/fd 1124414 0 2024-04-25T00:18:22.171 INFO:tasks.workunit.client.0.smithi107.stdout:9/131: mknod d0/d1b/c2d 0 2024-04-25T00:18:22.172 INFO:tasks.workunit.client.0.smithi107.stdout:7/99: dread d0/d8/fd [0,4194304] 0 2024-04-25T00:18:22.172 INFO:tasks.workunit.client.0.smithi107.stdout:7/100: fsync d0/d8/fd 0 2024-04-25T00:18:22.173 INFO:tasks.workunit.client.0.smithi107.stdout:7/101: creat d0/d8/f1e x:0 0 0 2024-04-25T00:18:22.173 INFO:tasks.workunit.client.0.smithi107.stdout:7/102: dread - d0/d8/f1e zero size 2024-04-25T00:18:22.173 INFO:tasks.workunit.client.0.smithi107.stdout:7/103: dread - d0/d8/f1e zero size 2024-04-25T00:18:22.173 INFO:tasks.workunit.client.0.smithi107.stdout:9/132: truncate d0/d1b/f27 24056 0 2024-04-25T00:18:22.174 INFO:tasks.workunit.client.0.smithi107.stdout:7/104: getdents d0/d8/db/d18/d19 0 2024-04-25T00:18:22.174 INFO:tasks.workunit.client.0.smithi107.stdout:7/105: fdatasync d0/d8/fd 0 2024-04-25T00:18:22.175 INFO:tasks.workunit.client.0.smithi107.stdout:6/135: rmdir d1/d2/d2b 39 2024-04-25T00:18:22.176 INFO:tasks.workunit.client.0.smithi107.stdout:5/74: mknod d4/c15 0 2024-04-25T00:18:22.176 INFO:tasks.workunit.client.0.smithi107.stdout:7/106: creat d0/d8/db/d18/d19/f1f x:0 0 0 2024-04-25T00:18:22.176 INFO:tasks.workunit.client.0.smithi107.stdout:7/107: write d0/f1d [845683,64220] 0 2024-04-25T00:18:22.177 INFO:tasks.workunit.client.0.smithi107.stdout:9/133: creat d0/d1b/f2e x:0 0 0 2024-04-25T00:18:22.177 INFO:tasks.workunit.client.0.smithi107.stdout:7/108: truncate d0/f13 4148187 0 2024-04-25T00:18:22.179 INFO:tasks.workunit.client.0.smithi107.stdout:2/117: write d6/f7 [1841836,26529] 0 2024-04-25T00:18:22.180 INFO:tasks.workunit.client.0.smithi107.stdout:6/136: symlink d1/l2e 0 2024-04-25T00:18:22.180 INFO:tasks.workunit.client.0.smithi107.stdout:2/118: stat d6/f1e 0 2024-04-25T00:18:22.181 INFO:tasks.workunit.client.0.smithi107.stdout:6/137: creat d1/d2/d3/f2f x:0 0 0 2024-04-25T00:18:22.181 INFO:tasks.workunit.client.0.smithi107.stdout:6/138: fdatasync d1/d14/d18/f2c 0 2024-04-25T00:18:22.182 INFO:tasks.workunit.client.0.smithi107.stdout:6/139: write d1/d2/d3/f22 [235154,122675] 0 2024-04-25T00:18:22.183 INFO:tasks.workunit.client.0.smithi107.stdout:5/75: unlink f1 0 2024-04-25T00:18:22.185 INFO:tasks.workunit.client.0.smithi107.stdout:9/134: creat d0/d1b/f2f x:0 0 0 2024-04-25T00:18:22.185 INFO:tasks.workunit.client.0.smithi107.stdout:9/135: chown d0/d8/de/d20/c19 77809817 1 2024-04-25T00:18:22.185 INFO:tasks.workunit.client.0.smithi107.stdout:1/99: dwrite d2/dd/d1a/f1d [0,4194304] 0 2024-04-25T00:18:22.187 INFO:tasks.workunit.client.0.smithi107.stdout:6/140: mknod d1/c30 0 2024-04-25T00:18:22.188 INFO:tasks.workunit.client.0.smithi107.stdout:5/76: creat d4/f16 x:0 0 0 2024-04-25T00:18:22.192 INFO:tasks.workunit.client.0.smithi107.stdout:9/136: dread d0/d8/de/d20/f10 [0,4194304] 0 2024-04-25T00:18:22.192 INFO:tasks.workunit.client.0.smithi107.stdout:1/100: link d2/dd/d1a/f1d d2/dd/f22 0 2024-04-25T00:18:22.194 INFO:tasks.workunit.client.0.smithi107.stdout:1/101: getdents d2 0 2024-04-25T00:18:22.196 INFO:tasks.workunit.client.0.smithi107.stdout:5/77: mkdir d4/d17 0 2024-04-25T00:18:22.204 INFO:tasks.workunit.client.0.smithi107.stdout:7/109: dwrite d0/d8/fd [0,4194304] 0 2024-04-25T00:18:22.205 INFO:tasks.workunit.client.0.smithi107.stdout:2/119: dwrite d6/d20/d22/f1a [0,4194304] 0 2024-04-25T00:18:22.208 INFO:tasks.workunit.client.0.smithi107.stdout:2/120: mkdir d6/d29 0 2024-04-25T00:18:22.213 INFO:tasks.workunit.client.0.smithi107.stdout:5/78: dwrite d4/dc/f14 [0,4194304] 0 2024-04-25T00:18:22.216 INFO:tasks.workunit.client.0.smithi107.stdout:3/96: sync 2024-04-25T00:18:22.216 INFO:tasks.workunit.client.0.smithi107.stdout:3/97: write d9/ff [3296493,31493] 0 2024-04-25T00:18:22.217 INFO:tasks.workunit.client.0.smithi107.stdout:3/98: getdents d9/d15/d17/d1b 0 2024-04-25T00:18:22.217 INFO:tasks.workunit.client.0.smithi107.stdout:3/99: chown d9/d15/d17 119224 1 2024-04-25T00:18:22.217 INFO:tasks.workunit.client.0.smithi107.stdout:3/100: chown c4 1891274522 1 2024-04-25T00:18:22.217 INFO:tasks.workunit.client.0.smithi107.stdout:3/101: chown l3 251911533 1 2024-04-25T00:18:22.217 INFO:tasks.workunit.client.0.smithi107.stdout:3/102: write f7 [111036,62126] 0 2024-04-25T00:18:22.218 INFO:tasks.workunit.client.0.smithi107.stdout:3/103: stat d9/d15/d1a/c1d 0 2024-04-25T00:18:22.219 INFO:tasks.workunit.client.0.smithi107.stdout:3/104: link d9/lb d9/d15/d17/d1b/l21 0 2024-04-25T00:18:22.219 INFO:tasks.workunit.client.0.smithi107.stdout:3/105: dread - d9/d15/d17/f18 zero size 2024-04-25T00:18:22.219 INFO:tasks.workunit.client.0.smithi107.stdout:4/67: sync 2024-04-25T00:18:22.219 INFO:tasks.workunit.client.0.smithi107.stdout:4/68: dread - d3/f16 zero size 2024-04-25T00:18:22.219 INFO:tasks.workunit.client.0.smithi107.stdout:4/69: fsync d3/d5/f7 0 2024-04-25T00:18:22.219 INFO:tasks.workunit.client.0.smithi107.stdout:4/70: readlink d3/le 0 2024-04-25T00:18:22.219 INFO:tasks.workunit.client.0.smithi107.stdout:4/71: write d3/f16 [334109,31444] 0 2024-04-25T00:18:22.219 INFO:tasks.workunit.client.0.smithi107.stdout:4/72: write d3/f9 [174671,49673] 0 2024-04-25T00:18:22.220 INFO:tasks.workunit.client.0.smithi107.stdout:4/73: chown d3/d5/ca 31 1 2024-04-25T00:18:22.222 INFO:tasks.workunit.client.0.smithi107.stdout:3/106: rename d9/d15/d17 to d9/d22 0 2024-04-25T00:18:22.222 INFO:tasks.workunit.client.0.smithi107.stdout:3/107: fdatasync d9/d15/d1a/f1c 0 2024-04-25T00:18:22.223 INFO:tasks.workunit.client.0.smithi107.stdout:3/108: rename d9/d22/d1b to d9/d22/d1b/d23 22 2024-04-25T00:18:22.223 INFO:tasks.workunit.client.0.smithi107.stdout:3/109: write d9/d22/f1e [511063,35310] 0 2024-04-25T00:18:22.223 INFO:tasks.workunit.client.0.smithi107.stdout:4/74: truncate d3/fd 840756 0 2024-04-25T00:18:22.223 INFO:tasks.workunit.client.0.smithi107.stdout:4/75: stat d3/d5/c14 0 2024-04-25T00:18:22.224 INFO:tasks.workunit.client.0.smithi107.stdout:4/76: stat d3/d5/f12 0 2024-04-25T00:18:22.224 INFO:tasks.workunit.client.0.smithi107.stdout:3/110: creat d9/d15/f24 x:0 0 0 2024-04-25T00:18:22.224 INFO:tasks.workunit.client.0.smithi107.stdout:3/111: dread - d9/d15/d1a/f1c zero size 2024-04-25T00:18:22.224 INFO:tasks.workunit.client.0.smithi107.stdout:4/77: rename d3/cb to d3/c17 0 2024-04-25T00:18:22.225 INFO:tasks.workunit.client.0.smithi107.stdout:4/78: read - d3/f15 zero size 2024-04-25T00:18:22.228 INFO:tasks.workunit.client.0.smithi107.stdout:6/141: dwrite d1/f1f [4194304,4194304] 0 2024-04-25T00:18:22.229 INFO:tasks.workunit.client.0.smithi107.stdout:6/142: fdatasync d1/d2/db/fd 0 2024-04-25T00:18:22.230 INFO:tasks.workunit.client.0.smithi107.stdout:6/143: creat d1/d2/d3/f31 x:0 0 0 2024-04-25T00:18:22.230 INFO:tasks.workunit.client.0.smithi107.stdout:6/144: write d1/d2/d3/f13 [4943308,113655] 0 2024-04-25T00:18:22.236 INFO:tasks.workunit.client.0.smithi107.stdout:5/79: dread d4/dc/fe [0,4194304] 0 2024-04-25T00:18:22.237 INFO:tasks.workunit.client.0.smithi107.stdout:2/121: dwrite d6/d20/d22/f1d [0,4194304] 0 2024-04-25T00:18:22.237 INFO:tasks.workunit.client.0.smithi107.stdout:2/122: fdatasync f4 0 2024-04-25T00:18:22.237 INFO:tasks.workunit.client.0.smithi107.stdout:6/145: rename d1/d2/d2b/d1d to d1/d2/db/d32 0 2024-04-25T00:18:22.237 INFO:tasks.workunit.client.0.smithi107.stdout:6/146: fdatasync d1/d14/d18/f2c 0 2024-04-25T00:18:22.238 INFO:tasks.workunit.client.0.smithi107.stdout:5/80: mknod d4/dc/c18 0 2024-04-25T00:18:22.239 INFO:tasks.workunit.client.0.smithi107.stdout:2/123: dread d6/d20/d22/f26 [4194304,4194304] 0 2024-04-25T00:18:22.239 INFO:tasks.workunit.client.0.smithi107.stdout:2/124: dread - d6/d20/d22/f1c zero size 2024-04-25T00:18:22.239 INFO:tasks.workunit.client.0.smithi107.stdout:2/125: dread - d6/f1e zero size 2024-04-25T00:18:22.241 INFO:tasks.workunit.client.0.smithi107.stdout:4/79: dwrite d3/d10/f13 [4194304,4194304] 0 2024-04-25T00:18:22.242 INFO:tasks.workunit.client.0.smithi107.stdout:6/147: truncate d1/d14/d18/f2c 1869218 0 2024-04-25T00:18:22.244 INFO:tasks.workunit.client.0.smithi107.stdout:4/80: rename d3/f15 to d3/f18 0 2024-04-25T00:18:22.244 INFO:tasks.workunit.client.0.smithi107.stdout:4/81: chown d3/le 8247 1 2024-04-25T00:18:22.244 INFO:tasks.workunit.client.0.smithi107.stdout:2/126: dread d6/f10 [0,4194304] 0 2024-04-25T00:18:22.245 INFO:tasks.workunit.client.0.smithi107.stdout:6/148: mkdir d1/d2/db/d32/d33 0 2024-04-25T00:18:22.245 INFO:tasks.workunit.client.0.smithi107.stdout:6/149: chown d1/d14/d18/f2c 10755 1 2024-04-25T00:18:22.245 INFO:tasks.workunit.client.0.smithi107.stdout:2/127: mkdir d6/d29/d2a 0 2024-04-25T00:18:22.246 INFO:tasks.workunit.client.0.smithi107.stdout:2/128: mkdir d6/d29/d2a/d2b 0 2024-04-25T00:18:22.247 INFO:tasks.workunit.client.0.smithi107.stdout:4/82: mknod d3/d10/c19 0 2024-04-25T00:18:22.249 INFO:tasks.workunit.client.0.smithi107.stdout:2/129: mkdir d6/d20/d22/d2c 0 2024-04-25T00:18:22.249 INFO:tasks.workunit.client.0.smithi107.stdout:6/150: creat d1/d2/db/f34 x:0 0 0 2024-04-25T00:18:22.249 INFO:tasks.workunit.client.0.smithi107.stdout:6/151: dread - d1/d2/d3/f2f zero size 2024-04-25T00:18:22.250 INFO:tasks.workunit.client.0.smithi107.stdout:6/152: readlink d1/l2e 0 2024-04-25T00:18:22.250 INFO:tasks.workunit.client.0.smithi107.stdout:6/153: chown d1/d2/db/fd 406084 1 2024-04-25T00:18:22.250 INFO:tasks.workunit.client.0.smithi107.stdout:6/154: dread - d1/d2/db/f19 zero size 2024-04-25T00:18:22.251 INFO:tasks.workunit.client.0.smithi107.stdout:2/130: symlink d6/d20/d22/d2c/l2d 0 2024-04-25T00:18:22.251 INFO:tasks.workunit.client.0.smithi107.stdout:2/131: write d6/d20/d22/f21 [5826301,107905] 0 2024-04-25T00:18:22.251 INFO:tasks.workunit.client.0.smithi107.stdout:6/155: truncate d1/d2/d3/f22 440026 0 2024-04-25T00:18:22.256 INFO:tasks.workunit.client.0.smithi107.stdout:4/83: getdents d3/d10 0 2024-04-25T00:18:22.257 INFO:tasks.workunit.client.0.smithi107.stdout:4/84: mknod d3/c1a 0 2024-04-25T00:18:22.259 INFO:tasks.workunit.client.0.smithi107.stdout:4/85: creat d3/d10/f1b x:0 0 0 2024-04-25T00:18:22.260 INFO:tasks.workunit.client.0.smithi107.stdout:4/86: rmdir d3/d10 39 2024-04-25T00:18:22.260 INFO:tasks.workunit.client.0.smithi107.stdout:2/132: dread d6/f7 [0,4194304] 0 2024-04-25T00:18:22.260 INFO:tasks.workunit.client.0.smithi107.stdout:5/81: dwrite d4/fa [0,4194304] 0 2024-04-25T00:18:22.261 INFO:tasks.workunit.client.0.smithi107.stdout:4/87: creat d3/d10/f1c x:0 0 0 2024-04-25T00:18:22.262 INFO:tasks.workunit.client.0.smithi107.stdout:5/82: symlink d4/d17/l19 0 2024-04-25T00:18:22.264 INFO:tasks.workunit.client.0.smithi107.stdout:4/88: dread d3/f16 [0,4194304] 0 2024-04-25T00:18:22.264 INFO:tasks.workunit.client.0.smithi107.stdout:4/89: chown d3 18611 1 2024-04-25T00:18:22.268 INFO:tasks.workunit.client.0.smithi107.stdout:2/133: dread f4 [0,4194304] 0 2024-04-25T00:18:22.268 INFO:tasks.workunit.client.0.smithi107.stdout:2/134: write d6/f10 [684215,20008] 0 2024-04-25T00:18:22.268 INFO:tasks.workunit.client.0.smithi107.stdout:2/135: truncate d6/d20/d22/f26 5093432 0 2024-04-25T00:18:22.269 INFO:tasks.workunit.client.0.smithi107.stdout:5/83: dread d4/fb [0,4194304] 0 2024-04-25T00:18:22.270 INFO:tasks.workunit.client.0.smithi107.stdout:2/136: write d6/ff [5491390,6687] 0 2024-04-25T00:18:22.271 INFO:tasks.workunit.client.0.smithi107.stdout:5/84: creat d4/dc/f1a x:0 0 0 2024-04-25T00:18:22.272 INFO:tasks.workunit.client.0.smithi107.stdout:2/137: creat d6/d18/f2e x:0 0 0 2024-04-25T00:18:22.274 INFO:tasks.workunit.client.0.smithi107.stdout:2/138: symlink d6/d29/d2a/d2b/l2f 0 2024-04-25T00:18:22.277 INFO:tasks.workunit.client.0.smithi107.stdout:2/139: mkdir d6/d29/d2a/d2b/d30 0 2024-04-25T00:18:22.277 INFO:tasks.workunit.client.0.smithi107.stdout:5/85: write d4/dc/fe [735308,81921] 0 2024-04-25T00:18:22.277 INFO:tasks.workunit.client.0.smithi107.stdout:5/86: write d4/dc/f13 [55122,95887] 0 2024-04-25T00:18:22.277 INFO:tasks.workunit.client.0.smithi107.stdout:5/87: truncate d4/dc/f1a 375044 0 2024-04-25T00:18:22.278 INFO:tasks.workunit.client.0.smithi107.stdout:5/88: creat d4/f1b x:0 0 0 2024-04-25T00:18:22.280 INFO:tasks.workunit.client.0.smithi107.stdout:6/156: dwrite d1/d2/d3/f22 [0,4194304] 0 2024-04-25T00:18:22.281 INFO:tasks.workunit.client.0.smithi107.stdout:6/157: truncate d1/d2/d3/f31 115782 0 2024-04-25T00:18:22.281 INFO:tasks.workunit.client.0.smithi107.stdout:6/158: truncate d1/d2/db/f34 1026154 0 2024-04-25T00:18:22.288 INFO:tasks.workunit.client.0.smithi107.stdout:4/90: dwrite d3/d10/f1b [0,4194304] 0 2024-04-25T00:18:22.291 INFO:tasks.workunit.client.0.smithi107.stdout:5/89: write d4/dc/f11 [1034999,52761] 0 2024-04-25T00:18:22.293 INFO:tasks.workunit.client.0.smithi107.stdout:6/159: dread d1/f1f [4194304,4194304] 0 2024-04-25T00:18:22.294 INFO:tasks.workunit.client.0.smithi107.stdout:5/90: mkdir d4/d1c 0 2024-04-25T00:18:22.294 INFO:tasks.workunit.client.0.smithi107.stdout:4/91: write f0 [624633,7695] 0 2024-04-25T00:18:22.294 INFO:tasks.workunit.client.0.smithi107.stdout:2/140: dread d6/d20/d22/f1a [0,4194304] 0 2024-04-25T00:18:22.295 INFO:tasks.workunit.client.0.smithi107.stdout:4/92: symlink d3/d10/l1d 0 2024-04-25T00:18:22.295 INFO:tasks.workunit.client.0.smithi107.stdout:5/91: link d4/d17/l19 d4/dc/l1d 0 2024-04-25T00:18:22.296 INFO:tasks.workunit.client.0.smithi107.stdout:4/93: symlink d3/l1e 0 2024-04-25T00:18:22.297 INFO:tasks.workunit.client.0.smithi107.stdout:2/141: link d6/d20/d22/c24 d6/d20/d22/d2c/c31 0 2024-04-25T00:18:22.298 INFO:tasks.workunit.client.0.smithi107.stdout:4/94: unlink d3/d10/f13 0 2024-04-25T00:18:22.299 INFO:tasks.workunit.client.0.smithi107.stdout:4/95: fdatasync d3/d10/f11 0 2024-04-25T00:18:22.299 INFO:tasks.workunit.client.0.smithi107.stdout:4/96: write d3/d5/f7 [580699,42898] 0 2024-04-25T00:18:22.300 INFO:tasks.workunit.client.0.smithi107.stdout:4/97: mknod d3/d10/c1f 0 2024-04-25T00:18:22.300 INFO:tasks.workunit.client.0.smithi107.stdout:2/142: rename d6/f14 to d6/d29/d2a/d2b/f32 0 2024-04-25T00:18:22.300 INFO:tasks.workunit.client.0.smithi107.stdout:4/98: read - d3/f18 zero size 2024-04-25T00:18:22.300 INFO:tasks.workunit.client.0.smithi107.stdout:5/92: symlink d4/dc/l1e 0 2024-04-25T00:18:22.301 INFO:tasks.workunit.client.0.smithi107.stdout:5/93: readlink d4/d17/l19 0 2024-04-25T00:18:22.301 INFO:tasks.workunit.client.0.smithi107.stdout:6/160: dread d1/d2/d3/f13 [4194304,4194304] 0 2024-04-25T00:18:22.301 INFO:tasks.workunit.client.0.smithi107.stdout:2/143: link d6/d20/d22/d2c/c31 d6/d29/d2a/d2b/c33 0 2024-04-25T00:18:22.302 INFO:tasks.workunit.client.0.smithi107.stdout:2/144: symlink d6/d20/d22/d2c/l34 0 2024-04-25T00:18:22.303 INFO:tasks.workunit.client.0.smithi107.stdout:2/145: read d6/d20/d22/f21 [5201642,129660] 0 2024-04-25T00:18:22.303 INFO:tasks.workunit.client.0.smithi107.stdout:2/146: chown d6/d20/d22/f1a 65 1 2024-04-25T00:18:22.304 INFO:tasks.workunit.client.0.smithi107.stdout:4/99: read d3/fd [533291,108547] 0 2024-04-25T00:18:22.304 INFO:tasks.workunit.client.0.smithi107.stdout:4/100: truncate d3/d5/fc 165669 0 2024-04-25T00:18:22.304 INFO:tasks.workunit.client.0.smithi107.stdout:4/101: read - d3/d10/f11 zero size 2024-04-25T00:18:22.315 INFO:tasks.workunit.client.0.smithi107.stdout:2/147: write d6/fe [2066600,110638] 0 2024-04-25T00:18:22.318 INFO:tasks.workunit.client.0.smithi107.stdout:2/148: dread d6/f8 [0,4194304] 0 2024-04-25T00:18:22.320 INFO:tasks.workunit.client.0.smithi107.stdout:6/161: dwrite d1/d14/d18/f2c [0,4194304] 0 2024-04-25T00:18:22.324 INFO:tasks.workunit.client.0.smithi107.stdout:5/94: dwrite d4/dc/f14 [4194304,4194304] 0 2024-04-25T00:18:22.325 INFO:tasks.workunit.client.0.smithi107.stdout:5/95: creat d4/dc/f1f x:0 0 0 2024-04-25T00:18:22.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:22 smithi195 ceph-mon[32833]: pgmap v60: 129 pgs: 129 active+clean; 133 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.0 MiB/s wr, 738 op/s 2024-04-25T00:18:22.341 INFO:tasks.workunit.client.0.smithi107.stdout:6/162: dwrite d1/d2/d3/f22 [0,4194304] 0 2024-04-25T00:18:22.342 INFO:tasks.workunit.client.0.smithi107.stdout:6/163: truncate d1/d14/d18/f2c 5052449 0 2024-04-25T00:18:22.342 INFO:tasks.workunit.client.0.smithi107.stdout:5/96: dread d4/fa [0,4194304] 0 2024-04-25T00:18:22.342 INFO:tasks.workunit.client.0.smithi107.stdout:5/97: fsync d4/dc/f14 0 2024-04-25T00:18:22.342 INFO:tasks.workunit.client.0.smithi107.stdout:5/98: chown l0 288281379 1 2024-04-25T00:18:22.344 INFO:tasks.workunit.client.0.smithi107.stdout:5/99: link d4/dc/f13 d4/dc/f20 0 2024-04-25T00:18:22.345 INFO:tasks.workunit.client.0.smithi107.stdout:5/100: chown d4/c5 686292492 1 2024-04-25T00:18:22.345 INFO:tasks.workunit.client.0.smithi107.stdout:5/101: truncate d4/dc/f11 1215079 0 2024-04-25T00:18:22.346 INFO:tasks.workunit.client.0.smithi107.stdout:5/102: creat d4/d17/f21 x:0 0 0 2024-04-25T00:18:22.347 INFO:tasks.workunit.client.0.smithi107.stdout:5/103: mknod d4/c22 0 2024-04-25T00:18:22.347 INFO:tasks.workunit.client.0.smithi107.stdout:5/104: fsync d4/dc/f1f 0 2024-04-25T00:18:22.347 INFO:tasks.workunit.client.0.smithi107.stdout:5/105: mknod d4/d17/c23 0 2024-04-25T00:18:22.363 INFO:tasks.workunit.client.0.smithi107.stdout:5/106: dread d4/fa [0,4194304] 0 2024-04-25T00:18:22.363 INFO:tasks.workunit.client.0.smithi107.stdout:5/107: chown d4/dc/fe 14785119 1 2024-04-25T00:18:22.363 INFO:tasks.workunit.client.0.smithi107.stdout:5/108: read - d4/f1b zero size 2024-04-25T00:18:22.363 INFO:tasks.workunit.client.0.smithi107.stdout:5/109: dread - d4/dc/f1f zero size 2024-04-25T00:18:22.363 INFO:tasks.workunit.client.0.smithi107.stdout:6/164: dwrite d1/f1f [4194304,4194304] 0 2024-04-25T00:18:22.364 INFO:tasks.workunit.client.0.smithi107.stdout:6/165: chown d1/f1f 66965355 1 2024-04-25T00:18:22.364 INFO:tasks.workunit.client.0.smithi107.stdout:5/110: mkdir d4/d1c/d24 0 2024-04-25T00:18:22.365 INFO:tasks.workunit.client.0.smithi107.stdout:6/166: creat d1/d2/d3/d26/f35 x:0 0 0 2024-04-25T00:18:22.366 INFO:tasks.workunit.client.0.smithi107.stdout:5/111: symlink d4/d1c/d24/l25 0 2024-04-25T00:18:22.366 INFO:tasks.workunit.client.0.smithi107.stdout:6/167: mkdir d1/d2/db/d32/d36 0 2024-04-25T00:18:22.371 INFO:tasks.workunit.client.0.smithi107.stdout:0/77: sync 2024-04-25T00:18:22.371 INFO:tasks.workunit.client.0.smithi107.stdout:8/125: sync 2024-04-25T00:18:22.372 INFO:tasks.workunit.client.0.smithi107.stdout:0/78: rename c0 to dc/dd/c1d 0 2024-04-25T00:18:22.373 INFO:tasks.workunit.client.0.smithi107.stdout:8/126: creat d2/f27 x:0 0 0 2024-04-25T00:18:22.374 INFO:tasks.workunit.client.0.smithi107.stdout:0/79: creat dc/dd/d16/d1a/f1e x:0 0 0 2024-04-25T00:18:22.375 INFO:tasks.workunit.client.0.smithi107.stdout:8/127: mknod d2/d18/d1a/d1b/c28 0 2024-04-25T00:18:22.375 INFO:tasks.workunit.client.0.smithi107.stdout:8/128: chown d2/f23 30203627 1 2024-04-25T00:18:22.375 INFO:tasks.workunit.client.0.smithi107.stdout:8/129: rename d2/d18/d1a to d2/d18/d1a/d1b/d29 22 2024-04-25T00:18:22.376 INFO:tasks.workunit.client.0.smithi107.stdout:0/80: rename dc/f11 to dc/f1f 0 2024-04-25T00:18:22.377 INFO:tasks.workunit.client.0.smithi107.stdout:8/130: symlink d2/d18/d1a/d1b/l2a 0 2024-04-25T00:18:22.379 INFO:tasks.workunit.client.0.smithi107.stdout:0/81: mkdir dc/dd/d20 0 2024-04-25T00:18:22.380 INFO:tasks.workunit.client.0.smithi107.stdout:0/82: write f1 [132148,21109] 0 2024-04-25T00:18:22.380 INFO:tasks.workunit.client.0.smithi107.stdout:0/83: dread - dc/dd/d16/d1a/f1e zero size 2024-04-25T00:18:22.380 INFO:tasks.workunit.client.0.smithi107.stdout:0/84: dread - dc/dd/f17 zero size 2024-04-25T00:18:22.381 INFO:tasks.workunit.client.0.smithi107.stdout:8/131: mknod d2/d18/d1a/d1b/c2b 0 2024-04-25T00:18:22.383 INFO:tasks.workunit.client.0.smithi107.stdout:8/132: mknod d2/d18/d1a/c2c 0 2024-04-25T00:18:22.385 INFO:tasks.workunit.client.0.smithi107.stdout:8/133: mknod d2/d18/d1a/d1b/c2d 0 2024-04-25T00:18:22.390 INFO:tasks.workunit.client.0.smithi107.stdout:5/112: dwrite d4/d17/f21 [0,4194304] 0 2024-04-25T00:18:22.390 INFO:tasks.workunit.client.0.smithi107.stdout:5/113: chown d4/dc/f13 22733506 1 2024-04-25T00:18:22.390 INFO:tasks.workunit.client.0.smithi107.stdout:5/114: dread - d4/dc/f1f zero size 2024-04-25T00:18:22.392 INFO:tasks.workunit.client.0.smithi107.stdout:5/115: rmdir d4 39 2024-04-25T00:18:22.394 INFO:tasks.workunit.client.0.smithi107.stdout:5/116: mkdir d4/dc/d26 0 2024-04-25T00:18:22.401 INFO:tasks.workunit.client.0.smithi107.stdout:6/168: dwrite d1/d2/db/f34 [0,4194304] 0 2024-04-25T00:18:22.401 INFO:tasks.workunit.client.0.smithi107.stdout:8/134: dread d2/f16 [0,4194304] 0 2024-04-25T00:18:22.403 INFO:tasks.workunit.client.0.smithi107.stdout:0/85: dwrite dc/dd/f19 [0,4194304] 0 2024-04-25T00:18:22.403 INFO:tasks.workunit.client.0.smithi107.stdout:0/86: getdents dc/dd/d20 0 2024-04-25T00:18:22.403 INFO:tasks.workunit.client.0.smithi107.stdout:8/135: mknod d2/c2e 0 2024-04-25T00:18:22.403 INFO:tasks.workunit.client.0.smithi107.stdout:8/136: dread - d2/d18/d1a/f21 zero size 2024-04-25T00:18:22.404 INFO:tasks.workunit.client.0.smithi107.stdout:0/87: mknod dc/c21 0 2024-04-25T00:18:22.404 INFO:tasks.workunit.client.0.smithi107.stdout:0/88: write dc/dd/d16/d1a/f1e [244240,91332] 0 2024-04-25T00:18:22.405 INFO:tasks.workunit.client.0.smithi107.stdout:8/137: creat d2/f2f x:0 0 0 2024-04-25T00:18:22.405 INFO:tasks.workunit.client.0.smithi107.stdout:8/138: truncate d2/f11 969668 0 2024-04-25T00:18:22.407 INFO:tasks.workunit.client.0.smithi107.stdout:0/89: creat dc/dd/d16/f22 x:0 0 0 2024-04-25T00:18:22.409 INFO:tasks.workunit.client.0.smithi107.stdout:0/90: mknod dc/dd/d16/d1a/c23 0 2024-04-25T00:18:22.410 INFO:tasks.workunit.client.0.smithi107.stdout:8/139: mknod d2/c30 0 2024-04-25T00:18:22.411 INFO:tasks.workunit.client.0.smithi107.stdout:0/91: mkdir dc/d24 0 2024-04-25T00:18:22.411 INFO:tasks.workunit.client.0.smithi107.stdout:0/92: chown dc/c1c 32311 1 2024-04-25T00:18:22.413 INFO:tasks.workunit.client.0.smithi107.stdout:5/117: dwrite d4/dc/f11 [0,4194304] 0 2024-04-25T00:18:22.413 INFO:tasks.workunit.client.0.smithi107.stdout:5/118: dread - d4/f1b zero size 2024-04-25T00:18:22.413 INFO:tasks.workunit.client.0.smithi107.stdout:5/119: chown d4/dc/c18 58 1 2024-04-25T00:18:22.413 INFO:tasks.workunit.client.0.smithi107.stdout:5/120: fsync d4/f7 0 2024-04-25T00:18:22.415 INFO:tasks.workunit.client.0.smithi107.stdout:8/140: dread d2/f8 [0,4194304] 0 2024-04-25T00:18:22.416 INFO:tasks.workunit.client.0.smithi107.stdout:5/121: mknod d4/d17/c27 0 2024-04-25T00:18:22.416 INFO:tasks.workunit.client.0.smithi107.stdout:5/122: chown d4/dc/f1f 1879581 1 2024-04-25T00:18:22.417 INFO:tasks.workunit.client.0.smithi107.stdout:8/141: creat d2/d18/d1a/f31 x:0 0 0 2024-04-25T00:18:22.417 INFO:tasks.workunit.client.0.smithi107.stdout:0/93: dread dc/dd/f14 [0,4194304] 0 2024-04-25T00:18:22.418 INFO:tasks.workunit.client.0.smithi107.stdout:5/123: link d4/fa d4/d1c/f28 0 2024-04-25T00:18:22.421 INFO:tasks.workunit.client.0.smithi107.stdout:8/142: mkdir d2/d18/d32 0 2024-04-25T00:18:22.421 INFO:tasks.workunit.client.0.smithi107.stdout:8/143: dread - d2/f27 zero size 2024-04-25T00:18:22.422 INFO:tasks.workunit.client.0.smithi107.stdout:8/144: rmdir d2/d18 39 2024-04-25T00:18:22.422 INFO:tasks.workunit.client.0.smithi107.stdout:8/145: write d2/d18/f20 [1263483,32679] 0 2024-04-25T00:18:22.426 INFO:tasks.workunit.client.0.smithi107.stdout:0/94: dread dc/dd/d16/d1a/f1e [0,4194304] 0 2024-04-25T00:18:22.430 INFO:tasks.workunit.client.0.smithi107.stdout:0/95: mkdir dc/d24/d25 0 2024-04-25T00:18:22.430 INFO:tasks.workunit.client.0.smithi107.stdout:0/96: fsync dc/f1f 0 2024-04-25T00:18:22.430 INFO:tasks.workunit.client.0.smithi107.stdout:0/97: truncate dc/dd/f15 585533 0 2024-04-25T00:18:22.431 INFO:tasks.workunit.client.0.smithi107.stdout:0/98: write dc/dd/f14 [2836824,372] 0 2024-04-25T00:18:22.431 INFO:tasks.workunit.client.0.smithi107.stdout:6/169: rename d1/d2/d3/d28/c24 to d1/d2/db/d32/d33/c37 0 2024-04-25T00:18:22.432 INFO:tasks.workunit.client.0.smithi107.stdout:6/170: creat d1/d2/db/f38 x:0 0 0 2024-04-25T00:18:22.433 INFO:tasks.workunit.client.0.smithi107.stdout:6/171: mknod d1/d2/db/d32/d33/c39 0 2024-04-25T00:18:22.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:22 smithi170 ceph-mon[32807]: pgmap v60: 129 pgs: 129 active+clean; 133 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.0 MiB/s wr, 738 op/s 2024-04-25T00:18:22.435 INFO:tasks.workunit.client.0.smithi107.stdout:6/172: mkdir d1/d2/db/d32/d36/d3a 0 2024-04-25T00:18:22.435 INFO:tasks.workunit.client.0.smithi107.stdout:6/173: mkdir d1/d2/db/d32/d33/d3b 0 2024-04-25T00:18:22.443 INFO:tasks.workunit.client.0.smithi107.stdout:5/124: dwrite d4/fa [0,4194304] 0 2024-04-25T00:18:22.443 INFO:tasks.workunit.client.0.smithi107.stdout:5/125: chown d4/fb 1 1 2024-04-25T00:18:22.443 INFO:tasks.workunit.client.0.smithi107.stdout:8/146: dwrite d2/f8 [0,4194304] 0 2024-04-25T00:18:22.444 INFO:tasks.workunit.client.0.smithi107.stdout:5/126: mkdir d4/dc/d26/d29 0 2024-04-25T00:18:22.445 INFO:tasks.workunit.client.0.smithi107.stdout:5/127: mknod d4/dc/d26/d29/c2a 0 2024-04-25T00:18:22.446 INFO:tasks.workunit.client.0.smithi107.stdout:5/128: write d4/dc/fe [2162216,125362] 0 2024-04-25T00:18:22.446 INFO:tasks.workunit.client.0.smithi107.stdout:5/129: chown d4/d1c/d24 3374174 1 2024-04-25T00:18:22.446 INFO:tasks.workunit.client.0.smithi107.stdout:5/130: chown l0 275 1 2024-04-25T00:18:22.446 INFO:tasks.workunit.client.0.smithi107.stdout:5/131: write d4/dc/f1f [35644,78272] 0 2024-04-25T00:18:22.446 INFO:tasks.workunit.client.0.smithi107.stdout:5/132: creat d4/d17/f2b x:0 0 0 2024-04-25T00:18:22.448 INFO:tasks.workunit.client.0.smithi107.stdout:5/133: link d4/c5 d4/c2c 0 2024-04-25T00:18:22.449 INFO:tasks.workunit.client.0.smithi107.stdout:5/134: dread d4/dc/f1a [0,4194304] 0 2024-04-25T00:18:22.451 INFO:tasks.workunit.client.0.smithi107.stdout:6/174: dwrite d1/d2/d2b/f27 [0,4194304] 0 2024-04-25T00:18:22.456 INFO:tasks.workunit.client.0.smithi107.stdout:5/135: dread d4/dc/fd [0,4194304] 0 2024-04-25T00:18:22.457 INFO:tasks.workunit.client.0.smithi107.stdout:5/136: rename d4/dc/d26/d29/c2a to d4/d1c/c2d 0 2024-04-25T00:18:22.459 INFO:tasks.workunit.client.0.smithi107.stdout:5/137: rename d4/d1c/d24 to d4/dc/d26/d29/d2e 0 2024-04-25T00:18:22.461 INFO:tasks.workunit.client.0.smithi107.stdout:8/147: dwrite d2/f9 [4194304,4194304] 0 2024-04-25T00:18:22.461 INFO:tasks.workunit.client.0.smithi107.stdout:8/148: readlink d2/d18/d1a/d1b/l2a 0 2024-04-25T00:18:22.461 INFO:tasks.workunit.client.0.smithi107.stdout:8/149: readlink d2/l19 0 2024-04-25T00:18:22.461 INFO:tasks.workunit.client.0.smithi107.stdout:8/150: write d2/f2f [181234,52792] 0 2024-04-25T00:18:22.464 INFO:tasks.workunit.client.0.smithi107.stdout:8/151: rename d2/d18/d32 to d2/d18/d1a/d33 0 2024-04-25T00:18:22.464 INFO:tasks.workunit.client.0.smithi107.stdout:5/138: dread d4/dc/fd [0,4194304] 0 2024-04-25T00:18:22.464 INFO:tasks.workunit.client.0.smithi107.stdout:5/139: write d4/dc/f1f [533759,56283] 0 2024-04-25T00:18:22.465 INFO:tasks.workunit.client.0.smithi107.stdout:8/152: unlink d2/f23 0 2024-04-25T00:18:22.466 INFO:tasks.workunit.client.0.smithi107.stdout:5/140: creat d4/dc/f2f x:0 0 0 2024-04-25T00:18:22.466 INFO:tasks.workunit.client.0.smithi107.stdout:5/141: write d4/dc/f13 [1128458,102010] 0 2024-04-25T00:18:22.466 INFO:tasks.workunit.client.0.smithi107.stdout:5/142: fdatasync d4/d17/f21 0 2024-04-25T00:18:22.466 INFO:tasks.workunit.client.0.smithi107.stdout:8/153: creat d2/d18/d1a/d33/f34 x:0 0 0 2024-04-25T00:18:22.467 INFO:tasks.workunit.client.0.smithi107.stdout:8/154: symlink d2/d18/d1a/d33/l35 0 2024-04-25T00:18:22.469 INFO:tasks.workunit.client.0.smithi107.stdout:6/175: dwrite d1/d2/db/fd [0,4194304] 0 2024-04-25T00:18:22.471 INFO:tasks.workunit.client.0.smithi107.stdout:6/176: mkdir d1/d2/db/d3c 0 2024-04-25T00:18:22.471 INFO:tasks.workunit.client.0.smithi107.stdout:6/177: truncate d1/d2/d3/f2f 658515 0 2024-04-25T00:18:22.472 INFO:tasks.workunit.client.0.smithi107.stdout:6/178: mkdir d1/d2/db/d32/d36/d3d 0 2024-04-25T00:18:22.474 INFO:tasks.workunit.client.0.smithi107.stdout:6/179: rename d1/d2/db/d32/d33/d3b to d1/d2/d2b/d3e 0 2024-04-25T00:18:22.476 INFO:tasks.workunit.client.0.smithi107.stdout:6/180: creat d1/d2/db/d32/d36/f3f x:0 0 0 2024-04-25T00:18:22.478 INFO:tasks.workunit.client.0.smithi107.stdout:8/155: dread d2/f8 [0,4194304] 0 2024-04-25T00:18:22.478 INFO:tasks.workunit.client.0.smithi107.stdout:8/156: read - d2/fb zero size 2024-04-25T00:18:22.482 INFO:tasks.workunit.client.0.smithi107.stdout:9/137: sync 2024-04-25T00:18:22.482 INFO:tasks.workunit.client.0.smithi107.stdout:9/138: readlink d0/d8/de/d20/l12 0 2024-04-25T00:18:22.483 INFO:tasks.workunit.client.0.smithi107.stdout:9/139: creat d0/f30 x:0 0 0 2024-04-25T00:18:22.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:22 smithi107 ceph-mon[29153]: pgmap v60: 129 pgs: 129 active+clean; 133 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.0 MiB/s wr, 738 op/s 2024-04-25T00:18:22.484 INFO:tasks.workunit.client.0.smithi107.stdout:9/140: symlink d0/d8/de/l31 0 2024-04-25T00:18:22.485 INFO:tasks.workunit.client.0.smithi107.stdout:9/141: mkdir d0/d8/d32 0 2024-04-25T00:18:22.487 INFO:tasks.workunit.client.0.smithi107.stdout:9/142: dread d0/d1b/f27 [0,4194304] 0 2024-04-25T00:18:22.489 INFO:tasks.workunit.client.0.smithi107.stdout:9/143: rmdir d0/d8/d1f/d2a 0 2024-04-25T00:18:22.490 INFO:tasks.workunit.client.0.smithi107.stdout:9/144: truncate d0/d1b/f2f 471991 0 2024-04-25T00:18:22.490 INFO:tasks.workunit.client.0.smithi107.stdout:9/145: read d0/d1b/f27 [3027,27918] 0 2024-04-25T00:18:22.491 INFO:tasks.workunit.client.0.smithi107.stdout:9/146: creat d0/d8/de/f33 x:0 0 0 2024-04-25T00:18:22.492 INFO:tasks.workunit.client.0.smithi107.stdout:8/157: dwrite d2/d18/d1a/f31 [0,4194304] 0 2024-04-25T00:18:22.494 INFO:tasks.workunit.client.0.smithi107.stdout:8/158: mknod d2/d18/d1a/c36 0 2024-04-25T00:18:22.509 INFO:tasks.workunit.client.0.smithi107.stdout:9/147: dwrite d0/d8/de/f21 [0,4194304] 0 2024-04-25T00:18:22.509 INFO:tasks.workunit.client.0.smithi107.stdout:9/148: stat d0/d1b/f27 0 2024-04-25T00:18:22.510 INFO:tasks.workunit.client.0.smithi107.stdout:8/159: dwrite d2/d18/d1a/d33/f34 [0,4194304] 0 2024-04-25T00:18:22.511 INFO:tasks.workunit.client.0.smithi107.stdout:9/149: mkdir d0/d8/d34 0 2024-04-25T00:18:22.514 INFO:tasks.workunit.client.0.smithi107.stdout:8/160: link d2/f16 d2/d18/d1a/f37 0 2024-04-25T00:18:22.516 INFO:tasks.workunit.client.0.smithi107.stdout:9/150: rename d0/d8/de/d20/c19 to d0/d8/d1f/c35 0 2024-04-25T00:18:22.516 INFO:tasks.workunit.client.0.smithi107.stdout:9/151: stat d0/f30 0 2024-04-25T00:18:22.517 INFO:tasks.workunit.client.0.smithi107.stdout:8/161: mknod d2/d18/d1a/d1b/c38 0 2024-04-25T00:18:22.517 INFO:tasks.workunit.client.0.smithi107.stdout:8/162: chown d2/d18/d1a/d1b/c2b 46 1 2024-04-25T00:18:22.518 INFO:tasks.workunit.client.0.smithi107.stdout:9/152: mkdir d0/d8/d34/d36 0 2024-04-25T00:18:22.518 INFO:tasks.workunit.client.0.smithi107.stdout:9/153: fdatasync d0/d8/de/d20/f28 0 2024-04-25T00:18:22.518 INFO:tasks.workunit.client.0.smithi107.stdout:9/154: write d0/f2 [2653133,18167] 0 2024-04-25T00:18:22.519 INFO:tasks.workunit.client.0.smithi107.stdout:8/163: creat d2/d18/d1a/d1b/f39 x:0 0 0 2024-04-25T00:18:22.520 INFO:tasks.workunit.client.0.smithi107.stdout:8/164: unlink d2/c10 0 2024-04-25T00:18:22.521 INFO:tasks.workunit.client.0.smithi107.stdout:8/165: creat d2/d18/d1a/f3a x:0 0 0 2024-04-25T00:18:22.548 INFO:tasks.workunit.client.0.smithi107.stdout:8/166: dwrite d2/d18/d1a/f3a [0,4194304] 0 2024-04-25T00:18:22.548 INFO:tasks.workunit.client.0.smithi107.stdout:9/155: dwrite d0/d8/de/d20/fd [0,4194304] 0 2024-04-25T00:18:22.550 INFO:tasks.workunit.client.0.smithi107.stdout:8/167: mknod d2/d18/d1a/c3b 0 2024-04-25T00:18:22.550 INFO:tasks.workunit.client.0.smithi107.stdout:8/168: dread - d2/f27 zero size 2024-04-25T00:18:22.552 INFO:tasks.workunit.client.0.smithi107.stdout:9/156: mknod d0/d8/de/c37 0 2024-04-25T00:18:22.554 INFO:tasks.workunit.client.0.smithi107.stdout:9/157: creat d0/d8/d32/f38 x:0 0 0 2024-04-25T00:18:22.556 INFO:tasks.workunit.client.0.smithi107.stdout:9/158: unlink d0/d8/l1e 0 2024-04-25T00:18:22.558 INFO:tasks.workunit.client.0.smithi107.stdout:8/169: write d2/f26 [1376198,108359] 0 2024-04-25T00:18:22.573 INFO:tasks.workunit.client.0.smithi107.stdout:8/170: dwrite d2/f9 [4194304,4194304] 0 2024-04-25T00:18:22.574 INFO:tasks.workunit.client.0.smithi107.stdout:8/171: write d2/d18/f20 [1915979,81829] 0 2024-04-25T00:18:22.574 INFO:tasks.workunit.client.0.smithi107.stdout:8/172: write d2/f16 [4542331,13134] 0 2024-04-25T00:18:22.574 INFO:tasks.workunit.client.0.smithi107.stdout:8/173: chown d2/d18/d1a/f1e 105330 1 2024-04-25T00:18:22.583 INFO:tasks.workunit.client.0.smithi107.stdout:8/174: dread d2/d18/d1a/f31 [0,4194304] 0 2024-04-25T00:18:22.639 INFO:tasks.workunit.client.0.smithi107.stdout:7/110: sync 2024-04-25T00:18:22.639 INFO:tasks.workunit.client.0.smithi107.stdout:1/102: sync 2024-04-25T00:18:22.639 INFO:tasks.workunit.client.0.smithi107.stdout:7/111: getdents d0/d8 0 2024-04-25T00:18:22.640 INFO:tasks.workunit.client.0.smithi107.stdout:1/103: mknod d2/dd/d1a/c23 0 2024-04-25T00:18:22.641 INFO:tasks.workunit.client.0.smithi107.stdout:7/112: creat d0/d8/db/f20 x:0 0 0 2024-04-25T00:18:22.641 INFO:tasks.workunit.client.0.smithi107.stdout:7/113: readlink d0/d8/la 0 2024-04-25T00:18:22.641 INFO:tasks.workunit.client.0.smithi107.stdout:7/114: dread - d0/f16 zero size 2024-04-25T00:18:22.641 INFO:tasks.workunit.client.0.smithi107.stdout:1/104: link d2/cb d2/da/c24 0 2024-04-25T00:18:22.642 INFO:tasks.workunit.client.0.smithi107.stdout:1/105: mkdir d2/dd/d25 0 2024-04-25T00:18:22.647 INFO:tasks.workunit.client.0.smithi107.stdout:8/175: dread d2/f16 [0,4194304] 0 2024-04-25T00:18:22.647 INFO:tasks.workunit.client.0.smithi107.stdout:8/176: dread - d2/d18/d1a/f21 zero size 2024-04-25T00:18:22.663 INFO:tasks.workunit.client.0.smithi107.stdout:1/106: dwrite f1 [0,4194304] 0 2024-04-25T00:18:22.674 INFO:tasks.workunit.client.0.smithi107.stdout:1/107: write d2/dd/d1a/f1d [1662839,7807] 0 2024-04-25T00:18:22.674 INFO:tasks.workunit.client.0.smithi107.stdout:1/108: fdatasync d2/dd/f10 0 2024-04-25T00:18:22.674 INFO:tasks.workunit.client.0.smithi107.stdout:1/109: write d2/dd/f22 [2753166,7878] 0 2024-04-25T00:18:22.674 INFO:tasks.workunit.client.0.smithi107.stdout:1/110: stat d2/da/l14 0 2024-04-25T00:18:22.676 INFO:tasks.workunit.client.0.smithi107.stdout:1/111: truncate d2/fc 1612267 0 2024-04-25T00:18:22.677 INFO:tasks.workunit.client.0.smithi107.stdout:1/112: rename d2/da/l11 to d2/dd/d1a/l26 0 2024-04-25T00:18:22.678 INFO:tasks.workunit.client.0.smithi107.stdout:1/113: creat d2/dd/d1a/f27 x:0 0 0 2024-04-25T00:18:22.678 INFO:tasks.workunit.client.0.smithi107.stdout:1/114: dread - d2/dd/d1a/f27 zero size 2024-04-25T00:18:22.730 INFO:tasks.workunit.client.0.smithi107.stdout:1/115: dwrite d2/dd/d1a/f27 [0,4194304] 0 2024-04-25T00:18:22.743 INFO:tasks.workunit.client.0.smithi107.stdout:1/116: dread d2/dd/f10 [0,4194304] 0 2024-04-25T00:18:22.744 INFO:tasks.workunit.client.0.smithi107.stdout:1/117: symlink d2/dd/d1a/d1f/l28 0 2024-04-25T00:18:22.744 INFO:tasks.workunit.client.0.smithi107.stdout:1/118: link d2/c6 d2/c29 0 2024-04-25T00:18:22.771 INFO:tasks.workunit.client.0.smithi107.stdout:4/102: sync 2024-04-25T00:18:22.771 INFO:tasks.workunit.client.0.smithi107.stdout:3/112: sync 2024-04-25T00:18:22.771 INFO:tasks.workunit.client.0.smithi107.stdout:3/113: truncate d9/f14 1461174 0 2024-04-25T00:18:22.771 INFO:tasks.workunit.client.0.smithi107.stdout:3/114: getdents d9/d15 0 2024-04-25T00:18:22.771 INFO:tasks.workunit.client.0.smithi107.stdout:3/115: chown d9/ce 235864704 1 2024-04-25T00:18:22.772 INFO:tasks.workunit.client.0.smithi107.stdout:4/103: link d3/d10/c1f d3/d5/c20 0 2024-04-25T00:18:22.773 INFO:tasks.workunit.client.0.smithi107.stdout:4/104: read d3/d5/fc [31619,32016] 0 2024-04-25T00:18:22.773 INFO:tasks.workunit.client.0.smithi107.stdout:3/116: write d9/d22/f1e [220918,75171] 0 2024-04-25T00:18:22.774 INFO:tasks.workunit.client.0.smithi107.stdout:3/117: mknod d9/d15/d1a/c25 0 2024-04-25T00:18:22.774 INFO:tasks.workunit.client.0.smithi107.stdout:3/118: dread - d9/d15/f24 zero size 2024-04-25T00:18:22.774 INFO:tasks.workunit.client.0.smithi107.stdout:3/119: chown d9/d22 4395874 1 2024-04-25T00:18:22.774 INFO:tasks.workunit.client.0.smithi107.stdout:4/105: creat d3/f21 x:0 0 0 2024-04-25T00:18:22.775 INFO:tasks.workunit.client.0.smithi107.stdout:4/106: fdatasync d3/d10/f1b 0 2024-04-25T00:18:22.775 INFO:tasks.workunit.client.0.smithi107.stdout:3/120: truncate f0 570297 0 2024-04-25T00:18:22.775 INFO:tasks.workunit.client.0.smithi107.stdout:4/107: creat d3/d5/f22 x:0 0 0 2024-04-25T00:18:22.777 INFO:tasks.workunit.client.0.smithi107.stdout:4/108: rename d3/f16 to d3/d5/f23 0 2024-04-25T00:18:22.778 INFO:tasks.workunit.client.0.smithi107.stdout:4/109: rmdir d3/d10 39 2024-04-25T00:18:22.795 INFO:tasks.workunit.client.0.smithi107.stdout:3/121: dwrite d9/d15/f16 [0,4194304] 0 2024-04-25T00:18:22.796 INFO:tasks.workunit.client.0.smithi107.stdout:3/122: unlink c4 0 2024-04-25T00:18:22.798 INFO:tasks.workunit.client.0.smithi107.stdout:3/123: unlink d9/d15/d1a/c1d 0 2024-04-25T00:18:22.798 INFO:tasks.workunit.client.0.smithi107.stdout:3/124: readlink d9/l12 0 2024-04-25T00:18:22.798 INFO:tasks.workunit.client.0.smithi107.stdout:3/125: write d9/f20 [971866,125532] 0 2024-04-25T00:18:22.798 INFO:tasks.workunit.client.0.smithi107.stdout:3/126: readlink l8 0 2024-04-25T00:18:22.799 INFO:tasks.workunit.client.0.smithi107.stdout:3/127: mknod d9/d15/d1a/c26 0 2024-04-25T00:18:22.799 INFO:tasks.workunit.client.0.smithi107.stdout:3/128: fsync f0 0 2024-04-25T00:18:22.799 INFO:tasks.workunit.client.0.smithi107.stdout:3/129: write d9/f10 [424584,75372] 0 2024-04-25T00:18:22.799 INFO:tasks.workunit.client.0.smithi107.stdout:4/110: dwrite d3/d10/f1c [0,4194304] 0 2024-04-25T00:18:22.801 INFO:tasks.workunit.client.0.smithi107.stdout:4/111: mkdir d3/d24 0 2024-04-25T00:18:22.801 INFO:tasks.workunit.client.0.smithi107.stdout:4/112: stat d3/d24 0 2024-04-25T00:18:22.802 INFO:tasks.workunit.client.0.smithi107.stdout:4/113: mkdir d3/d24/d25 0 2024-04-25T00:18:22.803 INFO:tasks.workunit.client.0.smithi107.stdout:4/114: creat d3/d24/d25/f26 x:0 0 0 2024-04-25T00:18:22.805 INFO:tasks.workunit.client.0.smithi107.stdout:4/115: rename d3/d10/c1f to d3/d24/c27 0 2024-04-25T00:18:22.806 INFO:tasks.workunit.client.0.smithi107.stdout:3/130: dread d9/f11 [0,4194304] 0 2024-04-25T00:18:22.806 INFO:tasks.workunit.client.0.smithi107.stdout:3/131: readlink d9/d22/d1b/l21 0 2024-04-25T00:18:22.807 INFO:tasks.workunit.client.0.smithi107.stdout:3/132: symlink d9/d22/d1b/l27 0 2024-04-25T00:18:22.808 INFO:tasks.workunit.client.0.smithi107.stdout:3/133: getdents d9/d22 0 2024-04-25T00:18:22.817 INFO:tasks.workunit.client.0.smithi107.stdout:3/134: dread d9/d15/f16 [0,4194304] 0 2024-04-25T00:18:22.819 INFO:tasks.workunit.client.0.smithi107.stdout:3/135: symlink d9/d22/l28 0 2024-04-25T00:18:22.820 INFO:tasks.workunit.client.0.smithi107.stdout:3/136: creat d9/f29 x:0 0 0 2024-04-25T00:18:22.820 INFO:tasks.workunit.client.0.smithi107.stdout:3/137: chown d9/d22/f1e 0 1 2024-04-25T00:18:22.820 INFO:tasks.workunit.client.0.smithi107.stdout:3/138: read f7 [1566440,79551] 0 2024-04-25T00:18:22.821 INFO:tasks.workunit.client.0.smithi107.stdout:3/139: symlink d9/d22/d1b/l2a 0 2024-04-25T00:18:22.826 INFO:tasks.workunit.client.0.smithi107.stdout:3/140: dread d9/f20 [0,4194304] 0 2024-04-25T00:18:22.826 INFO:tasks.workunit.client.0.smithi107.stdout:3/141: write f7 [230789,2037] 0 2024-04-25T00:18:22.827 INFO:tasks.workunit.client.0.smithi107.stdout:3/142: rename d9/d22/d1b/l2a to d9/d22/d1b/l2b 0 2024-04-25T00:18:22.828 INFO:tasks.workunit.client.0.smithi107.stdout:3/143: mkdir d9/d15/d1a/d2c 0 2024-04-25T00:18:22.830 INFO:tasks.workunit.client.0.smithi107.stdout:3/144: rename d9/d15/f16 to d9/d22/f2d 0 2024-04-25T00:18:22.831 INFO:tasks.workunit.client.0.smithi107.stdout:3/145: creat d9/d22/f2e x:0 0 0 2024-04-25T00:18:22.843 INFO:tasks.workunit.client.0.smithi107.stdout:2/149: sync 2024-04-25T00:18:22.844 INFO:tasks.workunit.client.0.smithi107.stdout:0/99: sync 2024-04-25T00:18:22.844 INFO:tasks.workunit.client.0.smithi107.stdout:5/143: sync 2024-04-25T00:18:22.844 INFO:tasks.workunit.client.0.smithi107.stdout:5/144: stat d4/dc/fd 0 2024-04-25T00:18:22.844 INFO:tasks.workunit.client.0.smithi107.stdout:2/150: symlink d6/d20/d22/l35 0 2024-04-25T00:18:22.845 INFO:tasks.workunit.client.0.smithi107.stdout:0/100: symlink dc/l26 0 2024-04-25T00:18:22.845 INFO:tasks.workunit.client.0.smithi107.stdout:0/101: getdents dc/d24/d25 0 2024-04-25T00:18:22.845 INFO:tasks.workunit.client.0.smithi107.stdout:5/145: rmdir d4/dc/d26/d29/d2e 39 2024-04-25T00:18:22.846 INFO:tasks.workunit.client.0.smithi107.stdout:2/151: mkdir d6/d18/d36 0 2024-04-25T00:18:22.847 INFO:tasks.workunit.client.0.smithi107.stdout:5/146: creat d4/dc/d26/f30 x:0 0 0 2024-04-25T00:18:22.848 INFO:tasks.workunit.client.0.smithi107.stdout:2/152: rename d6/c16 to d6/d29/d2a/c37 0 2024-04-25T00:18:22.848 INFO:tasks.workunit.client.0.smithi107.stdout:2/153: readlink d6/l23 0 2024-04-25T00:18:22.848 INFO:tasks.workunit.client.0.smithi107.stdout:2/154: stat d6/d20/d22/f1a 0 2024-04-25T00:18:22.849 INFO:tasks.workunit.client.0.smithi107.stdout:5/147: symlink d4/dc/l31 0 2024-04-25T00:18:22.850 INFO:tasks.workunit.client.0.smithi107.stdout:5/148: stat d4/dc/l1d 0 2024-04-25T00:18:22.851 INFO:tasks.workunit.client.0.smithi107.stdout:5/149: creat d4/d17/f32 x:0 0 0 2024-04-25T00:18:22.851 INFO:tasks.workunit.client.0.smithi107.stdout:5/150: chown d4/f16 110474255 1 2024-04-25T00:18:22.852 INFO:tasks.workunit.client.0.smithi107.stdout:5/151: symlink d4/d17/l33 0 2024-04-25T00:18:22.852 INFO:tasks.workunit.client.0.smithi107.stdout:5/152: write d4/dc/f1f [870810,91964] 0 2024-04-25T00:18:22.853 INFO:tasks.workunit.client.0.smithi107.stdout:2/155: dread d6/f1b [0,4194304] 0 2024-04-25T00:18:22.853 INFO:tasks.workunit.client.0.smithi107.stdout:2/156: write d6/f1e [923097,94898] 0 2024-04-25T00:18:22.853 INFO:tasks.workunit.client.0.smithi107.stdout:5/153: mknod d4/dc/c34 0 2024-04-25T00:18:22.853 INFO:tasks.workunit.client.0.smithi107.stdout:2/157: creat d6/d29/d2a/f38 x:0 0 0 2024-04-25T00:18:22.854 INFO:tasks.workunit.client.0.smithi107.stdout:2/158: creat d6/d29/d2a/d2b/f39 x:0 0 0 2024-04-25T00:18:22.855 INFO:tasks.workunit.client.0.smithi107.stdout:5/154: mkdir d4/d35 0 2024-04-25T00:18:22.855 INFO:tasks.workunit.client.0.smithi107.stdout:5/155: rename d4/dc/d26 to d4/dc/d26/d29/d36 22 2024-04-25T00:18:22.856 INFO:tasks.workunit.client.0.smithi107.stdout:5/156: truncate d4/dc/f20 603056 0 2024-04-25T00:18:22.856 INFO:tasks.workunit.client.0.smithi107.stdout:5/157: stat d4/fb 0 2024-04-25T00:18:22.856 INFO:tasks.workunit.client.0.smithi107.stdout:0/102: dread dc/dd/f19 [0,4194304] 0 2024-04-25T00:18:22.863 INFO:tasks.workunit.client.0.smithi107.stdout:0/103: write dc/dd/f14 [1920180,120565] 0 2024-04-25T00:18:22.866 INFO:tasks.workunit.client.0.smithi107.stdout:0/104: dread dc/dd/d16/d1a/f1e [0,4194304] 0 2024-04-25T00:18:22.866 INFO:tasks.workunit.client.0.smithi107.stdout:0/105: readlink lb 0 2024-04-25T00:18:22.867 INFO:tasks.workunit.client.0.smithi107.stdout:0/106: fsync dc/dd/d16/f1b 0 2024-04-25T00:18:22.867 INFO:tasks.workunit.client.0.smithi107.stdout:0/107: chown dc/dd 1078 1 2024-04-25T00:18:22.873 INFO:tasks.workunit.client.0.smithi107.stdout:2/159: dwrite d6/ff [0,4194304] 0 2024-04-25T00:18:22.876 INFO:tasks.workunit.client.0.smithi107.stdout:2/160: creat d6/d20/d22/f3a x:0 0 0 2024-04-25T00:18:22.876 INFO:tasks.workunit.client.0.smithi107.stdout:2/161: write d6/d29/d2a/f38 [49786,44966] 0 2024-04-25T00:18:22.876 INFO:tasks.workunit.client.0.smithi107.stdout:5/158: dread d4/dc/f13 [0,4194304] 0 2024-04-25T00:18:22.876 INFO:tasks.workunit.client.0.smithi107.stdout:2/162: creat d6/d29/d2a/f3b x:0 0 0 2024-04-25T00:18:22.877 INFO:tasks.workunit.client.0.smithi107.stdout:2/163: readlink d6/d20/d22/l35 0 2024-04-25T00:18:22.877 INFO:tasks.workunit.client.0.smithi107.stdout:5/159: creat d4/d35/f37 x:0 0 0 2024-04-25T00:18:22.880 INFO:tasks.workunit.client.0.smithi107.stdout:5/160: dread d4/dc/f13 [0,4194304] 0 2024-04-25T00:18:22.880 INFO:tasks.workunit.client.0.smithi107.stdout:5/161: chown d4/d17/f21 255 1 2024-04-25T00:18:22.881 INFO:tasks.workunit.client.0.smithi107.stdout:5/162: creat d4/d17/f38 x:0 0 0 2024-04-25T00:18:22.889 INFO:tasks.workunit.client.0.smithi107.stdout:5/163: write d4/fb [854290,128463] 0 2024-04-25T00:18:22.890 INFO:tasks.workunit.client.0.smithi107.stdout:0/108: dwrite dc/dd/f14 [0,4194304] 0 2024-04-25T00:18:22.890 INFO:tasks.workunit.client.0.smithi107.stdout:0/109: dread - dc/dd/d16/f1b zero size 2024-04-25T00:18:22.890 INFO:tasks.workunit.client.0.smithi107.stdout:0/110: fdatasync dc/f1f 0 2024-04-25T00:18:22.902 INFO:tasks.workunit.client.0.smithi107.stdout:2/164: dwrite d6/f1e [0,4194304] 0 2024-04-25T00:18:22.905 INFO:tasks.workunit.client.0.smithi107.stdout:2/165: rename d6/d29/d2a/f38 to d6/d29/d2a/f3c 0 2024-04-25T00:18:22.905 INFO:tasks.workunit.client.0.smithi107.stdout:2/166: write d6/d29/d2a/f3c [167052,42852] 0 2024-04-25T00:18:22.906 INFO:tasks.workunit.client.0.smithi107.stdout:5/164: dwrite d4/d17/f38 [0,4194304] 0 2024-04-25T00:18:22.907 INFO:tasks.workunit.client.0.smithi107.stdout:2/167: creat d6/f3d x:0 0 0 2024-04-25T00:18:22.907 INFO:tasks.workunit.client.0.smithi107.stdout:2/168: write d6/d20/d22/f1c [431369,83418] 0 2024-04-25T00:18:22.909 INFO:tasks.workunit.client.0.smithi107.stdout:0/111: dwrite dc/dd/d16/f1b [0,4194304] 0 2024-04-25T00:18:22.912 INFO:tasks.workunit.client.0.smithi107.stdout:0/112: dread f1 [0,4194304] 0 2024-04-25T00:18:22.920 INFO:tasks.workunit.client.0.smithi107.stdout:2/169: creat d6/d29/d2a/d2b/d30/f3e x:0 0 0 2024-04-25T00:18:22.922 INFO:tasks.workunit.client.0.smithi107.stdout:5/165: mknod d4/dc/c39 0 2024-04-25T00:18:22.924 INFO:tasks.workunit.client.0.smithi107.stdout:0/113: write dc/dd/f14 [3171303,41717] 0 2024-04-25T00:18:22.924 INFO:tasks.workunit.client.0.smithi107.stdout:5/166: creat d4/d1c/f3a x:0 0 0 2024-04-25T00:18:22.924 INFO:tasks.workunit.client.0.smithi107.stdout:5/167: write d4/dc/d26/f30 [618111,34292] 0 2024-04-25T00:18:22.926 INFO:tasks.workunit.client.0.smithi107.stdout:0/114: link lb dc/dd/l27 0 2024-04-25T00:18:22.933 INFO:tasks.workunit.client.0.smithi107.stdout:0/115: dread dc/dd/d16/f1b [0,4194304] 0 2024-04-25T00:18:22.942 INFO:tasks.workunit.client.0.smithi107.stdout:0/116: dread dc/dd/d16/d1a/f1e [0,4194304] 0 2024-04-25T00:18:22.942 INFO:tasks.workunit.client.0.smithi107.stdout:2/170: dwrite d6/f17 [0,4194304] 0 2024-04-25T00:18:22.944 INFO:tasks.workunit.client.0.smithi107.stdout:2/171: creat d6/d29/d2a/f3f x:0 0 0 2024-04-25T00:18:22.945 INFO:tasks.workunit.client.0.smithi107.stdout:2/172: rename d6/d18/d36 to d6/d18/d40 0 2024-04-25T00:18:22.949 INFO:tasks.workunit.client.0.smithi107.stdout:5/168: dwrite d4/d1c/f28 [0,4194304] 0 2024-04-25T00:18:22.953 INFO:tasks.workunit.client.0.smithi107.stdout:5/169: rename d4/d17/l33 to d4/l3b 0 2024-04-25T00:18:22.953 INFO:tasks.workunit.client.0.smithi107.stdout:6/181: sync 2024-04-25T00:18:22.953 INFO:tasks.workunit.client.0.smithi107.stdout:9/159: sync 2024-04-25T00:18:22.953 INFO:tasks.workunit.client.0.smithi107.stdout:7/115: sync 2024-04-25T00:18:22.954 INFO:tasks.workunit.client.0.smithi107.stdout:7/116: chown d0/d8/db/f10 1909 1 2024-04-25T00:18:22.954 INFO:tasks.workunit.client.0.smithi107.stdout:9/160: write d0/d8/de/d20/f4 [3918320,68392] 0 2024-04-25T00:18:22.954 INFO:tasks.workunit.client.0.smithi107.stdout:7/117: write d0/d8/db/d18/d19/f1f [713149,102166] 0 2024-04-25T00:18:22.954 INFO:tasks.workunit.client.0.smithi107.stdout:6/182: symlink d1/d2/db/d32/d36/d3a/l40 0 2024-04-25T00:18:22.960 INFO:tasks.workunit.client.0.smithi107.stdout:7/118: dread d0/f13 [0,4194304] 0 2024-04-25T00:18:22.962 INFO:tasks.workunit.client.0.smithi107.stdout:9/161: read d0/fc [1370827,27705] 0 2024-04-25T00:18:22.963 INFO:tasks.workunit.client.0.smithi107.stdout:9/162: stat d0/d8 0 2024-04-25T00:18:22.963 INFO:tasks.workunit.client.0.smithi107.stdout:9/163: chown d0/f30 18613 1 2024-04-25T00:18:22.963 INFO:tasks.workunit.client.0.smithi107.stdout:9/164: readlink d0/d8/de/l31 0 2024-04-25T00:18:22.963 INFO:tasks.workunit.client.0.smithi107.stdout:7/119: mkdir d0/d21 0 2024-04-25T00:18:22.963 INFO:tasks.workunit.client.0.smithi107.stdout:2/173: dread d6/d20/d22/f1d [0,4194304] 0 2024-04-25T00:18:22.963 INFO:tasks.workunit.client.0.smithi107.stdout:2/174: stat d6/d20/d22/d2c/l2d 0 2024-04-25T00:18:22.974 INFO:tasks.workunit.client.0.smithi107.stdout:5/170: dwrite d4/f7 [0,4194304] 0 2024-04-25T00:18:22.978 INFO:tasks.workunit.client.0.smithi107.stdout:5/171: creat d4/d35/f3c x:0 0 0 2024-04-25T00:18:22.978 INFO:tasks.workunit.client.0.smithi107.stdout:5/172: truncate d4/dc/f2f 379497 0 2024-04-25T00:18:22.981 INFO:tasks.workunit.client.0.smithi107.stdout:5/173: creat d4/dc/d26/f3d x:0 0 0 2024-04-25T00:18:22.989 INFO:tasks.workunit.client.0.smithi107.stdout:6/183: dwrite d1/d2/db/d32/d36/f3f [0,4194304] 0 2024-04-25T00:18:22.990 INFO:tasks.workunit.client.0.smithi107.stdout:2/175: dwrite d6/d20/d22/f1a [0,4194304] 0 2024-04-25T00:18:22.991 INFO:tasks.workunit.client.0.smithi107.stdout:2/176: creat d6/d29/d2a/d2b/f41 x:0 0 0 2024-04-25T00:18:22.992 INFO:tasks.workunit.client.0.smithi107.stdout:6/184: creat d1/d2/d3/f41 x:0 0 0 2024-04-25T00:18:22.993 INFO:tasks.workunit.client.0.smithi107.stdout:2/177: truncate d6/f17 99509 0 2024-04-25T00:18:22.995 INFO:tasks.workunit.client.0.smithi107.stdout:6/185: symlink d1/d2/d3/l42 0 2024-04-25T00:18:22.995 INFO:tasks.workunit.client.0.smithi107.stdout:6/186: readlink d1/d2/d2b/l1a 0 2024-04-25T00:18:22.997 INFO:tasks.workunit.client.0.smithi107.stdout:2/178: rename d6/d29/d2a to d6/d18/d40/d42 0 2024-04-25T00:18:22.997 INFO:tasks.workunit.client.0.smithi107.stdout:2/179: chown d6/d20/d22/f21 713 1 2024-04-25T00:18:22.997 INFO:tasks.workunit.client.0.smithi107.stdout:2/180: fsync d6/d18/d40/d42/f3b 0 2024-04-25T00:18:22.999 INFO:tasks.workunit.client.0.smithi107.stdout:2/181: rename d6/f1b to d6/f43 0 2024-04-25T00:18:22.999 INFO:tasks.workunit.client.0.smithi107.stdout:2/182: chown d6/d18/d40/d42/d2b/d30/f3e 1019872062 1 2024-04-25T00:18:22.999 INFO:tasks.workunit.client.0.smithi107.stdout:5/174: dwrite d4/d1c/f28 [0,4194304] 0 2024-04-25T00:18:23.016 INFO:tasks.workunit.client.0.smithi107.stdout:5/175: dwrite d4/d17/f32 [0,4194304] 0 2024-04-25T00:18:23.024 INFO:tasks.workunit.client.0.smithi107.stdout:2/183: dwrite d6/d20/d22/f1d [0,4194304] 0 2024-04-25T00:18:23.024 INFO:tasks.workunit.client.0.smithi107.stdout:5/176: dread d4/dc/f11 [0,4194304] 0 2024-04-25T00:18:23.026 INFO:tasks.workunit.client.0.smithi107.stdout:5/177: mknod d4/dc/c3e 0 2024-04-25T00:18:23.027 INFO:tasks.workunit.client.0.smithi107.stdout:5/178: link d4/c5 d4/dc/d26/d29/d2e/c3f 0 2024-04-25T00:18:23.038 INFO:tasks.workunit.client.0.smithi107.stdout:5/179: write d4/dc/f11 [1410929,31927] 0 2024-04-25T00:18:23.040 INFO:tasks.workunit.client.0.smithi107.stdout:5/180: rename d4/c2c to d4/c40 0 2024-04-25T00:18:23.040 INFO:tasks.workunit.client.0.smithi107.stdout:5/181: stat d4/dc/d26/d29/d2e/c3f 0 2024-04-25T00:18:23.041 INFO:tasks.workunit.client.0.smithi107.stdout:5/182: symlink d4/dc/d26/d29/d2e/l41 0 2024-04-25T00:18:23.042 INFO:tasks.workunit.client.0.smithi107.stdout:5/183: mkdir d4/dc/d26/d42 0 2024-04-25T00:18:23.042 INFO:tasks.workunit.client.0.smithi107.stdout:5/184: rmdir d4 39 2024-04-25T00:18:23.043 INFO:tasks.workunit.client.0.smithi107.stdout:5/185: write d4/dc/f2f [531099,102848] 0 2024-04-25T00:18:23.043 INFO:tasks.workunit.client.0.smithi107.stdout:5/186: stat d4/dc/f2f 0 2024-04-25T00:18:23.044 INFO:tasks.workunit.client.0.smithi107.stdout:5/187: rename d4/c22 to d4/d17/c43 0 2024-04-25T00:18:23.045 INFO:tasks.workunit.client.0.smithi107.stdout:5/188: creat d4/d1c/f44 x:0 0 0 2024-04-25T00:18:23.046 INFO:tasks.workunit.client.0.smithi107.stdout:5/189: symlink d4/dc/d26/d29/d2e/l45 0 2024-04-25T00:18:23.046 INFO:tasks.workunit.client.0.smithi107.stdout:5/190: mknod d4/dc/d26/c46 0 2024-04-25T00:18:23.048 INFO:tasks.workunit.client.0.smithi107.stdout:5/191: symlink d4/dc/l47 0 2024-04-25T00:18:23.049 INFO:tasks.workunit.client.0.smithi107.stdout:5/192: link d4/dc/fe d4/f48 0 2024-04-25T00:18:23.049 INFO:tasks.workunit.client.0.smithi107.stdout:5/193: dread - d4/d1c/f44 zero size 2024-04-25T00:18:23.050 INFO:tasks.workunit.client.0.smithi107.stdout:5/194: chown d4/dc/d26/d29/d2e/l25 1 1 2024-04-25T00:18:23.074 INFO:tasks.workunit.client.0.smithi107.stdout:1/119: sync 2024-04-25T00:18:23.074 INFO:tasks.workunit.client.0.smithi107.stdout:3/146: sync 2024-04-25T00:18:23.074 INFO:tasks.workunit.client.0.smithi107.stdout:4/116: sync 2024-04-25T00:18:23.074 INFO:tasks.workunit.client.0.smithi107.stdout:8/177: sync 2024-04-25T00:18:23.074 INFO:tasks.workunit.client.0.smithi107.stdout:4/117: fsync d3/d5/f22 0 2024-04-25T00:18:23.074 INFO:tasks.workunit.client.0.smithi107.stdout:3/147: truncate d9/f29 401378 0 2024-04-25T00:18:23.075 INFO:tasks.workunit.client.0.smithi107.stdout:4/118: fsync d3/d10/f11 0 2024-04-25T00:18:23.075 INFO:tasks.workunit.client.0.smithi107.stdout:1/120: dread d2/da/f19 [0,4194304] 0 2024-04-25T00:18:23.075 INFO:tasks.workunit.client.0.smithi107.stdout:1/121: stat d2/f13 0 2024-04-25T00:18:23.075 INFO:tasks.workunit.client.0.smithi107.stdout:3/148: mkdir d9/d2f 0 2024-04-25T00:18:23.075 INFO:tasks.workunit.client.0.smithi107.stdout:4/119: creat d3/d24/d25/f28 x:0 0 0 2024-04-25T00:18:23.076 INFO:tasks.workunit.client.0.smithi107.stdout:8/178: mknod d2/d18/d1a/d1b/c3c 0 2024-04-25T00:18:23.077 INFO:tasks.workunit.client.0.smithi107.stdout:8/179: mkdir d2/d18/d1a/d1b/d3d 0 2024-04-25T00:18:23.077 INFO:tasks.workunit.client.0.smithi107.stdout:4/120: dread d3/d5/f23 [0,4194304] 0 2024-04-25T00:18:23.078 INFO:tasks.workunit.client.0.smithi107.stdout:8/180: mkdir d2/d18/d1a/d1b/d3e 0 2024-04-25T00:18:23.079 INFO:tasks.workunit.client.0.smithi107.stdout:4/121: rename d3/d10/c19 to d3/d24/d25/c29 0 2024-04-25T00:18:23.081 INFO:tasks.workunit.client.0.smithi107.stdout:4/122: creat d3/f2a x:0 0 0 2024-04-25T00:18:23.081 INFO:tasks.workunit.client.0.smithi107.stdout:4/123: dread - d3/d5/f22 zero size 2024-04-25T00:18:23.081 INFO:tasks.workunit.client.0.smithi107.stdout:4/124: chown d3 528248149 1 2024-04-25T00:18:23.083 INFO:tasks.workunit.client.0.smithi107.stdout:4/125: rename d3/f9 to d3/d24/d25/f2b 0 2024-04-25T00:18:23.084 INFO:tasks.workunit.client.0.smithi107.stdout:4/126: creat d3/d24/f2c x:0 0 0 2024-04-25T00:18:23.085 INFO:tasks.workunit.client.0.smithi107.stdout:4/127: read d3/d24/d25/f2b [88844,36008] 0 2024-04-25T00:18:23.086 INFO:tasks.workunit.client.0.smithi107.stdout:2/184: dwrite d6/d18/d40/d42/d2b/f39 [0,4194304] 0 2024-04-25T00:18:23.087 INFO:tasks.workunit.client.0.smithi107.stdout:4/128: creat d3/d10/f2d x:0 0 0 2024-04-25T00:18:23.090 INFO:tasks.workunit.client.0.smithi107.stdout:2/185: dread d6/d20/d22/f1c [0,4194304] 0 2024-04-25T00:18:23.090 INFO:tasks.workunit.client.0.smithi107.stdout:5/195: dwrite d4/d17/f2b [0,4194304] 0 2024-04-25T00:18:23.100 INFO:tasks.workunit.client.0.smithi107.stdout:8/181: dread d2/f16 [0,4194304] 0 2024-04-25T00:18:23.100 INFO:tasks.workunit.client.0.smithi107.stdout:1/122: dwrite d2/f13 [0,4194304] 0 2024-04-25T00:18:23.100 INFO:tasks.workunit.client.0.smithi107.stdout:2/186: dread d6/f17 [0,4194304] 0 2024-04-25T00:18:23.102 INFO:tasks.workunit.client.0.smithi107.stdout:8/182: rename d2/f3 to d2/d18/d1a/d33/f3f 0 2024-04-25T00:18:23.103 INFO:tasks.workunit.client.0.smithi107.stdout:8/183: fdatasync d2/f16 0 2024-04-25T00:18:23.103 INFO:tasks.workunit.client.0.smithi107.stdout:8/184: fdatasync d2/d18/d1a/f25 0 2024-04-25T00:18:23.104 INFO:tasks.workunit.client.0.smithi107.stdout:3/149: dwrite d9/f29 [0,4194304] 0 2024-04-25T00:18:23.105 INFO:tasks.workunit.client.0.smithi107.stdout:4/129: write f0 [7721214,75933] 0 2024-04-25T00:18:23.106 INFO:tasks.workunit.client.0.smithi107.stdout:4/130: truncate d3/d10/f1b 3108048 0 2024-04-25T00:18:23.107 INFO:tasks.workunit.client.0.smithi107.stdout:5/196: dread d4/dc/f14 [0,4194304] 0 2024-04-25T00:18:23.107 INFO:tasks.workunit.client.0.smithi107.stdout:1/123: mkdir d2/d2a 0 2024-04-25T00:18:23.112 INFO:tasks.workunit.client.0.smithi107.stdout:2/187: rename d6/l23 to d6/d18/d40/d42/l44 0 2024-04-25T00:18:23.112 INFO:tasks.workunit.client.0.smithi107.stdout:3/150: unlink d9/d22/f2d 0 2024-04-25T00:18:23.112 INFO:tasks.workunit.client.0.smithi107.stdout:3/151: truncate d9/d22/f18 237873 0 2024-04-25T00:18:23.112 INFO:tasks.workunit.client.0.smithi107.stdout:8/185: read d2/d18/f20 [925007,98082] 0 2024-04-25T00:18:23.113 INFO:tasks.workunit.client.0.smithi107.stdout:5/197: readlink d4/dc/l10 0 2024-04-25T00:18:23.115 INFO:tasks.workunit.client.0.smithi107.stdout:1/124: symlink d2/dd/d1b/l2b 0 2024-04-25T00:18:23.117 INFO:tasks.workunit.client.0.smithi107.stdout:8/186: rename d2/d18/d1a/d1b/c2b to d2/d18/c40 0 2024-04-25T00:18:23.119 INFO:tasks.workunit.client.0.smithi107.stdout:5/198: mknod d4/dc/d26/d29/d2e/c49 0 2024-04-25T00:18:23.120 INFO:tasks.workunit.client.0.smithi107.stdout:1/125: symlink d2/dd/d25/l2c 0 2024-04-25T00:18:23.122 INFO:tasks.workunit.client.0.smithi107.stdout:2/188: dread d6/fe [0,4194304] 0 2024-04-25T00:18:23.122 INFO:tasks.workunit.client.0.smithi107.stdout:2/189: dread - d6/f3d zero size 2024-04-25T00:18:23.122 INFO:tasks.workunit.client.0.smithi107.stdout:8/187: creat d2/f41 x:0 0 0 2024-04-25T00:18:23.122 INFO:tasks.workunit.client.0.smithi107.stdout:8/188: chown d2/d18/d1a/f1e 75615 1 2024-04-25T00:18:23.123 INFO:tasks.workunit.client.0.smithi107.stdout:5/199: mknod d4/d35/c4a 0 2024-04-25T00:18:23.125 INFO:tasks.workunit.client.0.smithi107.stdout:1/126: rename d2/c6 to d2/dd/d1b/c2d 0 2024-04-25T00:18:23.127 INFO:tasks.workunit.client.0.smithi107.stdout:5/200: unlink d4/d1c/f44 0 2024-04-25T00:18:23.127 INFO:tasks.workunit.client.0.smithi107.stdout:4/131: dwrite d3/f2a [0,4194304] 0 2024-04-25T00:18:23.128 INFO:tasks.workunit.client.0.smithi107.stdout:1/127: getdents d2/dd 0 2024-04-25T00:18:23.128 INFO:tasks.workunit.client.0.smithi107.stdout:1/128: chown d2/dd/d1b/l2b 1117 1 2024-04-25T00:18:23.132 INFO:tasks.workunit.client.0.smithi107.stdout:8/189: read d2/ff [773707,8846] 0 2024-04-25T00:18:23.132 INFO:tasks.workunit.client.0.smithi107.stdout:2/190: dread d6/d20/d22/f21 [0,4194304] 0 2024-04-25T00:18:23.133 INFO:tasks.workunit.client.0.smithi107.stdout:1/129: getdents d2/dd/d1a/d1f 0 2024-04-25T00:18:23.135 INFO:tasks.workunit.client.0.smithi107.stdout:8/190: rmdir d2/d18/d1a 39 2024-04-25T00:18:23.138 INFO:tasks.workunit.client.0.smithi107.stdout:2/191: mkdir d6/d18/d40/d42/d2b/d30/d45 0 2024-04-25T00:18:23.138 INFO:tasks.workunit.client.0.smithi107.stdout:5/201: getdents d4/dc/d26/d29/d2e 0 2024-04-25T00:18:23.140 INFO:tasks.workunit.client.0.smithi107.stdout:1/130: truncate d2/dd/d1a/f27 3951691 0 2024-04-25T00:18:23.140 INFO:tasks.workunit.client.0.smithi107.stdout:1/131: write d2/dd/d1a/f1d [2417432,2079] 0 2024-04-25T00:18:23.142 INFO:tasks.workunit.client.0.smithi107.stdout:2/192: mknod d6/d18/d40/d42/d2b/d30/d45/c46 0 2024-04-25T00:18:23.143 INFO:tasks.workunit.client.0.smithi107.stdout:8/191: getdents d2/d18/d1a 0 2024-04-25T00:18:23.145 INFO:tasks.workunit.client.0.smithi107.stdout:2/193: rmdir d6/d18/d40/d42/d2b/d30 39 2024-04-25T00:18:23.145 INFO:tasks.workunit.client.0.smithi107.stdout:4/132: dwrite d3/d5/fc [0,4194304] 0 2024-04-25T00:18:23.150 INFO:tasks.workunit.client.0.smithi107.stdout:4/133: mknod d3/d24/d25/c2e 0 2024-04-25T00:18:23.151 INFO:tasks.workunit.client.0.smithi107.stdout:4/134: mknod d3/d24/d25/c2f 0 2024-04-25T00:18:23.151 INFO:tasks.workunit.client.0.smithi107.stdout:5/202: write d4/d17/f32 [3684238,80771] 0 2024-04-25T00:18:23.152 INFO:tasks.workunit.client.0.smithi107.stdout:4/135: creat d3/d24/d25/f30 x:0 0 0 2024-04-25T00:18:23.153 INFO:tasks.workunit.client.0.smithi107.stdout:5/203: creat d4/dc/d26/f4b x:0 0 0 2024-04-25T00:18:23.154 INFO:tasks.workunit.client.0.smithi107.stdout:4/136: symlink d3/d10/l31 0 2024-04-25T00:18:23.156 INFO:tasks.workunit.client.0.smithi107.stdout:1/132: dwrite d2/da/f1c [0,4194304] 0 2024-04-25T00:18:23.166 INFO:tasks.workunit.client.0.smithi107.stdout:5/204: dread d4/dc/f14 [0,4194304] 0 2024-04-25T00:18:23.166 INFO:tasks.workunit.client.0.smithi107.stdout:5/205: chown d4/d17/f32 12 1 2024-04-25T00:18:23.166 INFO:tasks.workunit.client.0.smithi107.stdout:8/192: dwrite d2/f9 [8388608,4194304] 0 2024-04-25T00:18:23.166 INFO:tasks.workunit.client.0.smithi107.stdout:8/193: chown d2/f27 213 1 2024-04-25T00:18:23.167 INFO:tasks.workunit.client.0.smithi107.stdout:8/194: write d2/d18/d1a/d33/f3f [50048,30524] 0 2024-04-25T00:18:23.169 INFO:tasks.workunit.client.0.smithi107.stdout:5/206: creat d4/dc/d26/d29/d2e/f4c x:0 0 0 2024-04-25T00:18:23.170 INFO:tasks.workunit.client.0.smithi107.stdout:8/195: mknod d2/d18/d1a/d1b/d3d/c42 0 2024-04-25T00:18:23.171 INFO:tasks.workunit.client.0.smithi107.stdout:5/207: unlink d4/c40 0 2024-04-25T00:18:23.171 INFO:tasks.workunit.client.0.smithi107.stdout:2/194: dwrite d6/d18/f2e [0,4194304] 0 2024-04-25T00:18:23.172 INFO:tasks.workunit.client.0.smithi107.stdout:2/195: fsync d6/f1e 0 2024-04-25T00:18:23.172 INFO:tasks.workunit.client.0.smithi107.stdout:2/196: fdatasync d6/f11 0 2024-04-25T00:18:23.174 INFO:tasks.workunit.client.0.smithi107.stdout:8/196: rename d2/d18/d1a/f21 to d2/d18/f43 0 2024-04-25T00:18:23.177 INFO:tasks.workunit.client.0.smithi107.stdout:5/208: write d4/dc/f1f [237820,91197] 0 2024-04-25T00:18:23.178 INFO:tasks.workunit.client.0.smithi107.stdout:4/137: dwrite d3/d10/f1c [0,4194304] 0 2024-04-25T00:18:23.179 INFO:tasks.workunit.client.0.smithi107.stdout:5/209: symlink d4/dc/d26/d29/d2e/l4d 0 2024-04-25T00:18:23.179 INFO:tasks.workunit.client.0.smithi107.stdout:5/210: write d4/dc/d26/d29/d2e/f4c [115232,83095] 0 2024-04-25T00:18:23.179 INFO:tasks.workunit.client.0.smithi107.stdout:5/211: write d4/d35/f37 [439872,84864] 0 2024-04-25T00:18:23.183 INFO:tasks.workunit.client.0.smithi107.stdout:1/133: dwrite d2/fc [0,4194304] 0 2024-04-25T00:18:23.188 INFO:tasks.workunit.client.0.smithi107.stdout:1/134: unlink d2/da/f1c 0 2024-04-25T00:18:23.195 INFO:tasks.workunit.client.0.smithi107.stdout:2/197: dwrite d6/d20/d22/f26 [4194304,4194304] 0 2024-04-25T00:18:23.195 INFO:tasks.workunit.client.0.smithi107.stdout:2/198: fsync d6/d20/d22/f1d 0 2024-04-25T00:18:23.196 INFO:tasks.workunit.client.0.smithi107.stdout:8/197: dwrite d2/d18/f43 [0,4194304] 0 2024-04-25T00:18:23.197 INFO:tasks.workunit.client.0.smithi107.stdout:2/199: creat d6/d18/d40/d42/f47 x:0 0 0 2024-04-25T00:18:23.198 INFO:tasks.workunit.client.0.smithi107.stdout:2/200: mknod d6/d18/d40/c48 0 2024-04-25T00:18:23.199 INFO:tasks.workunit.client.0.smithi107.stdout:2/201: dread - d6/f3d zero size 2024-04-25T00:18:23.199 INFO:tasks.workunit.client.0.smithi107.stdout:2/202: rmdir d6 39 2024-04-25T00:18:23.199 INFO:tasks.workunit.client.0.smithi107.stdout:1/135: read d2/dd/d1a/f27 [2024543,64465] 0 2024-04-25T00:18:23.200 INFO:tasks.workunit.client.0.smithi107.stdout:5/212: dwrite d4/d17/f38 [0,4194304] 0 2024-04-25T00:18:23.202 INFO:tasks.workunit.client.0.smithi107.stdout:8/198: dread d2/f8 [0,4194304] 0 2024-04-25T00:18:23.205 INFO:tasks.workunit.client.0.smithi107.stdout:8/199: mknod d2/d18/c44 0 2024-04-25T00:18:23.206 INFO:tasks.workunit.client.0.smithi107.stdout:8/200: creat d2/f45 x:0 0 0 2024-04-25T00:18:23.206 INFO:tasks.workunit.client.0.smithi107.stdout:8/201: write d2/d18/d1a/d1b/f39 [407019,55000] 0 2024-04-25T00:18:23.207 INFO:tasks.workunit.client.0.smithi107.stdout:5/213: dread d4/d35/f37 [0,4194304] 0 2024-04-25T00:18:23.208 INFO:tasks.workunit.client.0.smithi107.stdout:5/214: rmdir d4/dc/d26/d29 39 2024-04-25T00:18:23.208 INFO:tasks.workunit.client.0.smithi107.stdout:5/215: stat d4/d17/f32 0 2024-04-25T00:18:23.209 INFO:tasks.workunit.client.0.smithi107.stdout:8/202: mknod d2/d18/c46 0 2024-04-25T00:18:23.210 INFO:tasks.workunit.client.0.smithi107.stdout:5/216: creat d4/d1c/f4e x:0 0 0 2024-04-25T00:18:23.211 INFO:tasks.workunit.client.0.smithi107.stdout:2/203: dread d6/d18/f2e [0,4194304] 0 2024-04-25T00:18:23.211 INFO:tasks.workunit.client.0.smithi107.stdout:1/136: write d2/fc [3278860,57375] 0 2024-04-25T00:18:23.212 INFO:tasks.workunit.client.0.smithi107.stdout:8/203: creat d2/d18/d1a/d1b/f47 x:0 0 0 2024-04-25T00:18:23.212 INFO:tasks.workunit.client.0.smithi107.stdout:5/217: truncate d4/dc/f2f 365620 0 2024-04-25T00:18:23.213 INFO:tasks.workunit.client.0.smithi107.stdout:1/137: creat d2/dd/d25/f2e x:0 0 0 2024-04-25T00:18:23.214 INFO:tasks.workunit.client.0.smithi107.stdout:1/138: chown d2/c3 62399 1 2024-04-25T00:18:23.215 INFO:tasks.workunit.client.0.smithi107.stdout:8/204: link d2/ff d2/d18/d1a/f48 0 2024-04-25T00:18:23.216 INFO:tasks.workunit.client.0.smithi107.stdout:5/218: creat d4/dc/d26/d42/f4f x:0 0 0 2024-04-25T00:18:23.218 INFO:tasks.workunit.client.0.smithi107.stdout:8/205: symlink d2/d18/l49 0 2024-04-25T00:18:23.218 INFO:tasks.workunit.client.0.smithi107.stdout:8/206: chown d2/f16 138920 1 2024-04-25T00:18:23.219 INFO:tasks.workunit.client.0.smithi107.stdout:5/219: mknod d4/dc/d26/c50 0 2024-04-25T00:18:23.221 INFO:tasks.workunit.client.0.smithi107.stdout:8/207: mknod d2/d18/d1a/d1b/d3d/c4a 0 2024-04-25T00:18:23.221 INFO:tasks.workunit.client.0.smithi107.stdout:8/208: chown d2/d18/d1a/f37 549604 1 2024-04-25T00:18:23.221 INFO:tasks.workunit.client.0.smithi107.stdout:8/209: readlink d2/d18/d1a/d1b/l2a 0 2024-04-25T00:18:23.222 INFO:tasks.workunit.client.0.smithi107.stdout:5/220: unlink d4/d1c/f3a 0 2024-04-25T00:18:23.223 INFO:tasks.workunit.client.0.smithi107.stdout:8/210: symlink d2/d18/d1a/d1b/d3d/l4b 0 2024-04-25T00:18:23.226 INFO:tasks.workunit.client.0.smithi107.stdout:1/139: dread d2/da/f15 [0,4194304] 0 2024-04-25T00:18:23.226 INFO:tasks.workunit.client.0.smithi107.stdout:8/211: creat d2/d18/d1a/d1b/d3e/f4c x:0 0 0 2024-04-25T00:18:23.226 INFO:tasks.workunit.client.0.smithi107.stdout:8/212: truncate d2/f2f 1136280 0 2024-04-25T00:18:23.227 INFO:tasks.workunit.client.0.smithi107.stdout:8/213: stat d2/d18/d1a/d1b/c24 0 2024-04-25T00:18:23.227 INFO:tasks.workunit.client.0.smithi107.stdout:1/140: mkdir d2/da/d2f 0 2024-04-25T00:18:23.228 INFO:tasks.workunit.client.0.smithi107.stdout:8/214: symlink d2/d18/d1a/d1b/d3e/l4d 0 2024-04-25T00:18:23.228 INFO:tasks.workunit.client.0.smithi107.stdout:8/215: truncate d2/d18/f20 2228296 0 2024-04-25T00:18:23.230 INFO:tasks.workunit.client.0.smithi107.stdout:2/204: dwrite d6/f10 [0,4194304] 0 2024-04-25T00:18:23.232 INFO:tasks.workunit.client.0.smithi107.stdout:5/221: write d4/d17/f2b [2316515,27083] 0 2024-04-25T00:18:23.233 INFO:tasks.workunit.client.0.smithi107.stdout:1/141: rename d2/dd/f22 to d2/f30 0 2024-04-25T00:18:23.236 INFO:tasks.workunit.client.0.smithi107.stdout:2/205: creat d6/d20/d22/d2c/f49 x:0 0 0 2024-04-25T00:18:23.237 INFO:tasks.workunit.client.0.smithi107.stdout:5/222: write d4/dc/fd [989168,58621] 0 2024-04-25T00:18:23.239 INFO:tasks.workunit.client.0.smithi107.stdout:2/206: symlink d6/d18/d40/d42/d2b/l4a 0 2024-04-25T00:18:23.241 INFO:tasks.workunit.client.0.smithi107.stdout:2/207: creat d6/d18/d40/d42/d2b/d30/f4b x:0 0 0 2024-04-25T00:18:23.257 INFO:tasks.workunit.client.0.smithi107.stdout:8/216: dwrite d2/d18/d1a/f37 [0,4194304] 0 2024-04-25T00:18:23.258 INFO:tasks.workunit.client.0.smithi107.stdout:5/223: write d4/dc/f14 [122670,119267] 0 2024-04-25T00:18:23.258 INFO:tasks.workunit.client.0.smithi107.stdout:1/142: dwrite d2/fc [0,4194304] 0 2024-04-25T00:18:23.259 INFO:tasks.workunit.client.0.smithi107.stdout:8/217: symlink d2/d18/d1a/l4e 0 2024-04-25T00:18:23.259 INFO:tasks.workunit.client.0.smithi107.stdout:0/117: sync 2024-04-25T00:18:23.259 INFO:tasks.workunit.client.0.smithi107.stdout:7/120: sync 2024-04-25T00:18:23.259 INFO:tasks.workunit.client.0.smithi107.stdout:9/165: sync 2024-04-25T00:18:23.259 INFO:tasks.workunit.client.0.smithi107.stdout:1/143: link d2/dd/c1e d2/d2a/c31 0 2024-04-25T00:18:23.260 INFO:tasks.workunit.client.0.smithi107.stdout:7/121: symlink d0/d8/l22 0 2024-04-25T00:18:23.260 INFO:tasks.workunit.client.0.smithi107.stdout:7/122: write d0/d8/db/f15 [325136,28742] 0 2024-04-25T00:18:23.261 INFO:tasks.workunit.client.0.smithi107.stdout:7/123: write d0/d8/db/d18/d19/f1f [1052074,54572] 0 2024-04-25T00:18:23.261 INFO:tasks.workunit.client.0.smithi107.stdout:8/218: mkdir d2/d18/d1a/d1b/d4f 0 2024-04-25T00:18:23.262 INFO:tasks.workunit.client.0.smithi107.stdout:0/118: link dc/f1f dc/f28 0 2024-04-25T00:18:23.262 INFO:tasks.workunit.client.0.smithi107.stdout:0/119: dread - dc/f28 zero size 2024-04-25T00:18:23.262 INFO:tasks.workunit.client.0.smithi107.stdout:2/208: dwrite d6/d18/d40/d42/d2b/f41 [0,4194304] 0 2024-04-25T00:18:23.262 INFO:tasks.workunit.client.0.smithi107.stdout:2/209: chown d6/d18/d40/d42/d2b/l4a 5747 1 2024-04-25T00:18:23.262 INFO:tasks.workunit.client.0.smithi107.stdout:2/210: fsync d6/d20/d22/f1d 0 2024-04-25T00:18:23.262 INFO:tasks.workunit.client.0.smithi107.stdout:7/124: creat d0/d8/db/d18/d19/f23 x:0 0 0 2024-04-25T00:18:23.262 INFO:tasks.workunit.client.0.smithi107.stdout:1/144: rmdir d2/dd/d1a 39 2024-04-25T00:18:23.263 INFO:tasks.workunit.client.0.smithi107.stdout:8/219: creat d2/d18/d1a/d33/f50 x:0 0 0 2024-04-25T00:18:23.263 INFO:tasks.workunit.client.0.smithi107.stdout:7/125: creat d0/d21/f24 x:0 0 0 2024-04-25T00:18:23.263 INFO:tasks.workunit.client.0.smithi107.stdout:0/120: truncate dc/dd/f19 653369 0 2024-04-25T00:18:23.265 INFO:tasks.workunit.client.0.smithi107.stdout:0/121: read dc/dd/d16/d1a/f1e [135923,103813] 0 2024-04-25T00:18:23.265 INFO:tasks.workunit.client.0.smithi107.stdout:1/145: getdents d2 0 2024-04-25T00:18:23.266 INFO:tasks.workunit.client.0.smithi107.stdout:8/220: symlink d2/d18/d1a/d1b/d3e/l51 0 2024-04-25T00:18:23.268 INFO:tasks.workunit.client.0.smithi107.stdout:9/166: dread d0/d8/de/d20/f4 [0,4194304] 0 2024-04-25T00:18:23.268 INFO:tasks.workunit.client.0.smithi107.stdout:9/167: fsync d0/d8/d32/f38 0 2024-04-25T00:18:23.268 INFO:tasks.workunit.client.0.smithi107.stdout:0/122: rmdir dc/dd/d16 39 2024-04-25T00:18:23.271 INFO:tasks.workunit.client.0.smithi107.stdout:9/168: getdents d0/d8/de/d20 0 2024-04-25T00:18:23.273 INFO:tasks.workunit.client.0.smithi107.stdout:0/123: rename dc/c21 to dc/dd/d16/d1a/c29 0 2024-04-25T00:18:23.276 INFO:tasks.workunit.client.0.smithi107.stdout:9/169: unlink d0/d1b/f27 0 2024-04-25T00:18:23.277 INFO:tasks.workunit.client.0.smithi107.stdout:9/170: readlink d0/l18 0 2024-04-25T00:18:23.278 INFO:tasks.workunit.client.0.smithi107.stdout:5/224: dwrite d4/d17/f32 [4194304,4194304] 0 2024-04-25T00:18:23.278 INFO:tasks.workunit.client.0.smithi107.stdout:9/171: symlink d0/d8/d34/l39 0 2024-04-25T00:18:23.279 INFO:tasks.workunit.client.0.smithi107.stdout:9/172: chown d0/d8/d1f 495 1 2024-04-25T00:18:23.280 INFO:tasks.workunit.client.0.smithi107.stdout:5/225: mkdir d4/dc/d26/d29/d51 0 2024-04-25T00:18:23.280 INFO:tasks.workunit.client.0.smithi107.stdout:2/211: dread d6/d20/d22/f1d [0,4194304] 0 2024-04-25T00:18:23.282 INFO:tasks.workunit.client.0.smithi107.stdout:2/212: mkdir d6/d18/d40/d42/d2b/d30/d4c 0 2024-04-25T00:18:23.282 INFO:tasks.workunit.client.0.smithi107.stdout:2/213: fsync d6/d18/d40/d42/f3c 0 2024-04-25T00:18:23.284 INFO:tasks.workunit.client.0.smithi107.stdout:2/214: mkdir d6/d18/d4d 0 2024-04-25T00:18:23.286 INFO:tasks.workunit.client.0.smithi107.stdout:7/126: dwrite d0/f13 [0,4194304] 0 2024-04-25T00:18:23.290 INFO:tasks.workunit.client.0.smithi107.stdout:0/124: read dc/dd/f19 [23165,114629] 0 2024-04-25T00:18:23.290 INFO:tasks.workunit.client.0.smithi107.stdout:0/125: write dc/dd/d16/f22 [92673,71279] 0 2024-04-25T00:18:23.290 INFO:tasks.workunit.client.0.smithi107.stdout:0/126: truncate dc/dd/d16/d1a/f1e 606242 0 2024-04-25T00:18:23.291 INFO:tasks.workunit.client.0.smithi107.stdout:0/127: stat dc/d24/d25 0 2024-04-25T00:18:23.291 INFO:tasks.workunit.client.0.smithi107.stdout:0/128: stat dc 0 2024-04-25T00:18:23.300 INFO:tasks.workunit.client.0.smithi107.stdout:8/221: dwrite d2/d18/d1a/f3a [0,4194304] 0 2024-04-25T00:18:23.301 INFO:tasks.workunit.client.0.smithi107.stdout:5/226: dwrite d4/d17/f38 [0,4194304] 0 2024-04-25T00:18:23.304 INFO:tasks.workunit.client.0.smithi107.stdout:5/227: write d4/dc/fe [1209799,49569] 0 2024-04-25T00:18:23.304 INFO:tasks.workunit.client.0.smithi107.stdout:8/222: creat d2/d18/d1a/d1b/d4f/f52 x:0 0 0 2024-04-25T00:18:23.307 INFO:tasks.workunit.client.0.smithi107.stdout:5/228: link d4/dc/d26/c50 d4/c52 0 2024-04-25T00:18:23.307 INFO:tasks.workunit.client.0.smithi107.stdout:5/229: write d4/f7 [4225915,21226] 0 2024-04-25T00:18:23.308 INFO:tasks.workunit.client.0.smithi107.stdout:7/127: dwrite d0/d8/db/d18/d19/f1f [0,4194304] 0 2024-04-25T00:18:23.308 INFO:tasks.workunit.client.0.smithi107.stdout:7/128: chown d0/d8/cf 2070072 1 2024-04-25T00:18:23.310 INFO:tasks.workunit.client.0.smithi107.stdout:5/230: mknod d4/dc/c53 0 2024-04-25T00:18:23.310 INFO:tasks.workunit.client.0.smithi107.stdout:5/231: readlink d4/dc/d26/d29/d2e/l41 0 2024-04-25T00:18:23.311 INFO:tasks.workunit.client.0.smithi107.stdout:5/232: mkdir d4/dc/d54 0 2024-04-25T00:18:23.311 INFO:tasks.workunit.client.0.smithi107.stdout:5/233: dread - d4/f16 zero size 2024-04-25T00:18:23.313 INFO:tasks.workunit.client.0.smithi107.stdout:1/146: dwrite d2/dd/f21 [0,4194304] 0 2024-04-25T00:18:23.315 INFO:tasks.workunit.client.0.smithi107.stdout:2/215: dwrite d6/f8 [0,4194304] 0 2024-04-25T00:18:23.317 INFO:tasks.workunit.client.0.smithi107.stdout:1/147: mknod d2/dd/d25/c32 0 2024-04-25T00:18:23.317 INFO:tasks.workunit.client.0.smithi107.stdout:1/148: chown d2/da/l14 32 1 2024-04-25T00:18:23.317 INFO:tasks.workunit.client.0.smithi107.stdout:1/149: dread - d2/dd/d25/f2e zero size 2024-04-25T00:18:23.317 INFO:tasks.workunit.client.0.smithi107.stdout:1/150: fdatasync f1 0 2024-04-25T00:18:23.317 INFO:tasks.workunit.client.0.smithi107.stdout:1/151: chown d2/dd/d25 0 1 2024-04-25T00:18:23.320 INFO:tasks.workunit.client.0.smithi107.stdout:7/129: read d0/d8/db/f15 [4166432,103973] 0 2024-04-25T00:18:23.327 INFO:tasks.workunit.client.0.smithi107.stdout:1/152: creat d2/dd/d25/f33 x:0 0 0 2024-04-25T00:18:23.327 INFO:tasks.workunit.client.0.smithi107.stdout:1/153: chown d2/dd/d1b 99705 1 2024-04-25T00:18:23.337 INFO:tasks.workunit.client.0.smithi107.stdout:1/154: dread d2/f30 [0,4194304] 0 2024-04-25T00:18:23.347 INFO:tasks.workunit.client.0.smithi107.stdout:1/155: dread d2/dd/f21 [0,4194304] 0 2024-04-25T00:18:23.347 INFO:tasks.workunit.client.0.smithi107.stdout:1/156: readlink d2/dd/d1a/d1f/l28 0 2024-04-25T00:18:23.348 INFO:tasks.workunit.client.0.smithi107.stdout:1/157: creat d2/d2a/f34 x:0 0 0 2024-04-25T00:18:23.348 INFO:tasks.workunit.client.0.smithi107.stdout:1/158: write d2/dd/d25/f2e [779332,116916] 0 2024-04-25T00:18:23.348 INFO:tasks.workunit.client.0.smithi107.stdout:1/159: chown d2/f13 7930 1 2024-04-25T00:18:23.348 INFO:tasks.workunit.client.0.smithi107.stdout:1/160: readlink d2/da/lf 0 2024-04-25T00:18:23.349 INFO:tasks.workunit.client.0.smithi107.stdout:1/161: truncate d2/f30 718905 0 2024-04-25T00:18:23.349 INFO:tasks.workunit.client.0.smithi107.stdout:1/162: mkdir d2/da/d35 0 2024-04-25T00:18:23.350 INFO:tasks.workunit.client.0.smithi107.stdout:1/163: creat d2/dd/d1b/f36 x:0 0 0 2024-04-25T00:18:23.366 INFO:tasks.workunit.client.0.smithi107.stdout:1/164: dwrite d2/dd/d25/f33 [0,4194304] 0 2024-04-25T00:18:23.366 INFO:tasks.workunit.client.0.smithi107.stdout:1/165: write d2/dd/d25/f2e [19622,7311] 0 2024-04-25T00:18:23.366 INFO:tasks.workunit.client.0.smithi107.stdout:1/166: stat d2/dd/d25/c32 0 2024-04-25T00:18:23.367 INFO:tasks.workunit.client.0.smithi107.stdout:1/167: mkdir d2/d2a/d37 0 2024-04-25T00:18:23.368 INFO:tasks.workunit.client.0.smithi107.stdout:1/168: creat d2/d2a/f38 x:0 0 0 2024-04-25T00:18:23.369 INFO:tasks.workunit.client.0.smithi107.stdout:1/169: creat d2/dd/d1a/d1f/f39 x:0 0 0 2024-04-25T00:18:23.370 INFO:tasks.workunit.client.0.smithi107.stdout:1/170: fsync d2/dd/d25/f33 0 2024-04-25T00:18:23.371 INFO:tasks.workunit.client.0.smithi107.stdout:1/171: rename d2/d2a/f34 to d2/dd/d1b/f3a 0 2024-04-25T00:18:23.381 INFO:tasks.workunit.client.0.smithi107.stdout:8/223: dwrite d2/d18/d1a/f37 [0,4194304] 0 2024-04-25T00:18:23.383 INFO:tasks.workunit.client.0.smithi107.stdout:5/234: dwrite d4/f7 [0,4194304] 0 2024-04-25T00:18:23.383 INFO:tasks.workunit.client.0.smithi107.stdout:5/235: chown d4/dc/d26/d29/d2e/c49 141737 1 2024-04-25T00:18:23.394 INFO:tasks.workunit.client.0.smithi107.stdout:5/236: dread d4/dc/f11 [0,4194304] 0 2024-04-25T00:18:23.394 INFO:tasks.workunit.client.0.smithi107.stdout:1/172: dread d2/f13 [0,4194304] 0 2024-04-25T00:18:23.395 INFO:tasks.workunit.client.0.smithi107.stdout:5/237: mkdir d4/dc/d26/d29/d2e/d55 0 2024-04-25T00:18:23.396 INFO:tasks.workunit.client.0.smithi107.stdout:5/238: truncate d4/dc/f11 3376957 0 2024-04-25T00:18:23.396 INFO:tasks.workunit.client.0.smithi107.stdout:5/239: getdents d4/dc/d26/d29/d51 0 2024-04-25T00:18:23.398 INFO:tasks.workunit.client.0.smithi107.stdout:5/240: rename d4/dc/d26/d29/d2e/l41 to d4/l56 0 2024-04-25T00:18:23.398 INFO:tasks.workunit.client.0.smithi107.stdout:5/241: chown d4/dc/f20 42070 1 2024-04-25T00:18:23.398 INFO:tasks.workunit.client.0.smithi107.stdout:5/242: chown d4/d17/f32 1950 1 2024-04-25T00:18:23.399 INFO:tasks.workunit.client.0.smithi107.stdout:8/224: dread d2/d18/d1a/f37 [0,4194304] 0 2024-04-25T00:18:23.399 INFO:tasks.workunit.client.0.smithi107.stdout:8/225: write d2/fb [167070,4413] 0 2024-04-25T00:18:23.399 INFO:tasks.workunit.client.0.smithi107.stdout:1/173: dread d2/dd/d25/f2e [0,4194304] 0 2024-04-25T00:18:23.399 INFO:tasks.workunit.client.0.smithi107.stdout:5/243: read d4/dc/fd [1036557,69695] 0 2024-04-25T00:18:23.408 INFO:tasks.workunit.client.0.smithi107.stdout:1/174: write d2/da/f19 [1646694,113280] 0 2024-04-25T00:18:23.426 INFO:tasks.workunit.client.0.smithi107.stdout:8/226: dwrite d2/f45 [0,4194304] 0 2024-04-25T00:18:23.426 INFO:tasks.workunit.client.0.smithi107.stdout:5/244: dwrite d4/f7 [0,4194304] 0 2024-04-25T00:18:23.428 INFO:tasks.workunit.client.0.smithi107.stdout:1/175: dwrite d2/da/f15 [0,4194304] 0 2024-04-25T00:18:23.428 INFO:tasks.workunit.client.0.smithi107.stdout:1/176: write d2/da/f19 [3212422,119511] 0 2024-04-25T00:18:23.429 INFO:tasks.workunit.client.0.smithi107.stdout:8/227: rename d2/f9 to d2/d18/f53 0 2024-04-25T00:18:23.433 INFO:tasks.workunit.client.0.smithi107.stdout:1/177: creat d2/da/f3b x:0 0 0 2024-04-25T00:18:23.435 INFO:tasks.workunit.client.0.smithi107.stdout:5/245: rename d4/f7 to d4/dc/d26/d29/d51/f57 0 2024-04-25T00:18:23.437 INFO:tasks.workunit.client.0.smithi107.stdout:8/228: mknod d2/d18/d1a/d1b/d3e/c54 0 2024-04-25T00:18:23.437 INFO:tasks.workunit.client.0.smithi107.stdout:8/229: chown d2/fb 2031257 1 2024-04-25T00:18:23.440 INFO:tasks.workunit.client.0.smithi107.stdout:8/230: creat d2/f55 x:0 0 0 2024-04-25T00:18:23.441 INFO:tasks.workunit.client.0.smithi107.stdout:5/246: mknod d4/dc/c58 0 2024-04-25T00:18:23.442 INFO:tasks.workunit.client.0.smithi107.stdout:1/178: dread f1 [0,4194304] 0 2024-04-25T00:18:23.442 INFO:tasks.workunit.client.0.smithi107.stdout:1/179: chown d2/d2a/d37 30338163 1 2024-04-25T00:18:23.442 INFO:tasks.workunit.client.0.smithi107.stdout:1/180: stat d2/dd/l17 0 2024-04-25T00:18:23.444 INFO:tasks.workunit.client.0.smithi107.stdout:1/181: rmdir d2/da/d35 0 2024-04-25T00:18:23.447 INFO:tasks.workunit.client.0.smithi107.stdout:1/182: rename d2/dd/l17 to d2/l3c 0 2024-04-25T00:18:23.447 INFO:tasks.workunit.client.0.smithi107.stdout:1/183: chown d2/c3 1336479423 1 2024-04-25T00:18:23.450 INFO:tasks.workunit.client.0.smithi107.stdout:8/231: dread d2/f45 [0,4194304] 0 2024-04-25T00:18:23.451 INFO:tasks.workunit.client.0.smithi107.stdout:8/232: rmdir d2/d18/d1a 39 2024-04-25T00:18:23.461 INFO:tasks.workunit.client.0.smithi107.stdout:5/247: dwrite d4/d1c/f4e [0,4194304] 0 2024-04-25T00:18:23.461 INFO:tasks.workunit.client.0.smithi107.stdout:5/248: stat d4/dc/c39 0 2024-04-25T00:18:23.461 INFO:tasks.workunit.client.0.smithi107.stdout:8/233: dread d2/d18/d1a/d1b/f1c [0,4194304] 0 2024-04-25T00:18:23.461 INFO:tasks.workunit.client.0.smithi107.stdout:8/234: write d2/ff [1353527,68087] 0 2024-04-25T00:18:23.463 INFO:tasks.workunit.client.0.smithi107.stdout:8/235: creat d2/d18/d1a/d1b/f56 x:0 0 0 2024-04-25T00:18:23.464 INFO:tasks.workunit.client.0.smithi107.stdout:8/236: creat d2/f57 x:0 0 0 2024-04-25T00:18:23.465 INFO:tasks.workunit.client.0.smithi107.stdout:8/237: rename d2/d18/d1a/d1b/c2d to d2/d18/d1a/c58 0 2024-04-25T00:18:23.465 INFO:tasks.workunit.client.0.smithi107.stdout:8/238: write d2/ff [3301211,49026] 0 2024-04-25T00:18:23.466 INFO:tasks.workunit.client.0.smithi107.stdout:8/239: truncate d2/d18/d1a/d33/f34 1373879 0 2024-04-25T00:18:23.466 INFO:tasks.workunit.client.0.smithi107.stdout:8/240: truncate d2/d18/d1a/d1b/f47 312285 0 2024-04-25T00:18:23.469 INFO:tasks.workunit.client.0.smithi107.stdout:5/249: truncate d4/d1c/f4e 2691401 0 2024-04-25T00:18:23.471 INFO:tasks.workunit.client.0.smithi107.stdout:1/184: dwrite f1 [0,4194304] 0 2024-04-25T00:18:23.472 INFO:tasks.workunit.client.0.smithi107.stdout:1/185: mknod d2/da/c3d 0 2024-04-25T00:18:23.474 INFO:tasks.workunit.client.0.smithi107.stdout:1/186: link d2/fc d2/da/d2f/f3e 0 2024-04-25T00:18:23.478 INFO:tasks.workunit.client.0.smithi107.stdout:5/250: dread d4/dc/fe [0,4194304] 0 2024-04-25T00:18:23.478 INFO:tasks.workunit.client.0.smithi107.stdout:5/251: write d4/d17/f21 [4557565,96304] 0 2024-04-25T00:18:23.478 INFO:tasks.workunit.client.0.smithi107.stdout:1/187: rmdir d2/da 39 2024-04-25T00:18:23.478 INFO:tasks.workunit.client.0.smithi107.stdout:8/241: dread d2/ff [0,4194304] 0 2024-04-25T00:18:23.479 INFO:tasks.workunit.client.0.smithi107.stdout:5/252: link d4/dc/fe d4/dc/d26/d42/f59 0 2024-04-25T00:18:23.480 INFO:tasks.workunit.client.0.smithi107.stdout:1/188: mknod d2/dd/d1a/c3f 0 2024-04-25T00:18:23.481 INFO:tasks.workunit.client.0.smithi107.stdout:1/189: write d2/dd/d1a/d1f/f39 [298893,89390] 0 2024-04-25T00:18:23.501 INFO:tasks.workunit.client.0.smithi107.stdout:1/190: dwrite d2/f13 [0,4194304] 0 2024-04-25T00:18:23.501 INFO:tasks.workunit.client.0.smithi107.stdout:1/191: dread - d2/da/f3b zero size 2024-04-25T00:18:23.501 INFO:tasks.workunit.client.0.smithi107.stdout:8/242: dwrite d2/d18/d1a/d33/f50 [0,4194304] 0 2024-04-25T00:18:23.501 INFO:tasks.workunit.client.0.smithi107.stdout:5/253: dwrite d4/d1c/f4e [0,4194304] 0 2024-04-25T00:18:23.502 INFO:tasks.workunit.client.0.smithi107.stdout:5/254: fdatasync d4/dc/d26/d42/f4f 0 2024-04-25T00:18:23.503 INFO:tasks.workunit.client.0.smithi107.stdout:1/192: mkdir d2/da/d40 0 2024-04-25T00:18:23.506 INFO:tasks.workunit.client.0.smithi107.stdout:8/243: rename d2/d18/d1a/d1b/f1c to d2/d18/d1a/d33/f59 0 2024-04-25T00:18:23.509 INFO:tasks.workunit.client.0.smithi107.stdout:1/193: rename d2/dd/d1a/c23 to d2/dd/d25/c41 0 2024-04-25T00:18:23.509 INFO:tasks.workunit.client.0.smithi107.stdout:1/194: chown d2/cb 3 1 2024-04-25T00:18:23.509 INFO:tasks.workunit.client.0.smithi107.stdout:1/195: fsync d2/da/f19 0 2024-04-25T00:18:23.510 INFO:tasks.workunit.client.0.smithi107.stdout:1/196: fdatasync d2/f30 0 2024-04-25T00:18:23.510 INFO:tasks.workunit.client.0.smithi107.stdout:8/244: symlink d2/d18/d1a/d1b/d3d/l5a 0 2024-04-25T00:18:23.511 INFO:tasks.workunit.client.0.smithi107.stdout:1/197: truncate d2/dd/d1a/f27 3899897 0 2024-04-25T00:18:23.511 INFO:tasks.workunit.client.0.smithi107.stdout:1/198: chown d2/dd 3376 1 2024-04-25T00:18:23.514 INFO:tasks.workunit.client.0.smithi107.stdout:8/245: dread d2/f2f [0,4194304] 0 2024-04-25T00:18:23.515 INFO:tasks.workunit.client.0.smithi107.stdout:8/246: fsync d2/d18/d1a/f3a 0 2024-04-25T00:18:23.516 INFO:tasks.workunit.client.0.smithi107.stdout:8/247: link d2/d18/d1a/d1b/c24 d2/c5b 0 2024-04-25T00:18:23.516 INFO:tasks.workunit.client.0.smithi107.stdout:8/248: read d2/d18/f53 [1394747,96154] 0 2024-04-25T00:18:23.517 INFO:tasks.workunit.client.0.smithi107.stdout:8/249: chown d2/d18/d1a/d1b/d3d 6852646 1 2024-04-25T00:18:23.517 INFO:tasks.workunit.client.0.smithi107.stdout:8/250: mkdir d2/d18/d1a/d1b/d5c 0 2024-04-25T00:18:23.524 INFO:tasks.workunit.client.0.smithi107.stdout:5/255: dwrite d4/f48 [0,4194304] 0 2024-04-25T00:18:23.524 INFO:tasks.workunit.client.0.smithi107.stdout:5/256: stat d4/dc/f11 0 2024-04-25T00:18:23.536 INFO:tasks.workunit.client.0.smithi107.stdout:8/251: dwrite d2/d18/d1a/f3a [0,4194304] 0 2024-04-25T00:18:23.538 INFO:tasks.workunit.client.0.smithi107.stdout:8/252: symlink d2/d18/d1a/d1b/d3e/l5d 0 2024-04-25T00:18:23.543 INFO:tasks.workunit.client.0.smithi107.stdout:5/257: dwrite d4/dc/d26/f3d [0,4194304] 0 2024-04-25T00:18:23.546 INFO:tasks.workunit.client.0.smithi107.stdout:8/253: write d2/f26 [1857864,110354] 0 2024-04-25T00:18:23.546 INFO:tasks.workunit.client.0.smithi107.stdout:5/258: mknod d4/dc/d54/c5a 0 2024-04-25T00:18:23.548 INFO:tasks.workunit.client.0.smithi107.stdout:8/254: dread d2/f2f [0,4194304] 0 2024-04-25T00:18:23.548 INFO:tasks.workunit.client.0.smithi107.stdout:8/255: fsync d2/f2f 0 2024-04-25T00:18:23.548 INFO:tasks.workunit.client.0.smithi107.stdout:5/259: mknod d4/dc/d26/d29/c5b 0 2024-04-25T00:18:23.548 INFO:tasks.workunit.client.0.smithi107.stdout:5/260: truncate d4/d35/f37 884959 0 2024-04-25T00:18:23.562 INFO:tasks.workunit.client.0.smithi107.stdout:8/256: dwrite d2/d18/d1a/f31 [0,4194304] 0 2024-04-25T00:18:23.562 INFO:tasks.workunit.client.0.smithi107.stdout:8/257: write d2/d18/d1a/d1b/f39 [1394016,95719] 0 2024-04-25T00:18:23.569 INFO:tasks.workunit.client.0.smithi107.stdout:5/261: dwrite d4/dc/d26/d42/f4f [0,4194304] 0 2024-04-25T00:18:23.571 INFO:tasks.workunit.client.0.smithi107.stdout:5/262: mkdir d4/d1c/d5c 0 2024-04-25T00:18:23.591 INFO:tasks.workunit.client.0.smithi107.stdout:5/263: write d4/dc/d26/d42/f4f [1050392,30192] 0 2024-04-25T00:18:23.591 INFO:tasks.workunit.client.0.smithi107.stdout:5/264: chown d4/dc/c18 1169 1 2024-04-25T00:18:23.591 INFO:tasks.workunit.client.0.smithi107.stdout:8/258: dwrite d2/f16 [0,4194304] 0 2024-04-25T00:18:23.591 INFO:tasks.workunit.client.0.smithi107.stdout:8/259: dread - d2/f57 zero size 2024-04-25T00:18:23.591 INFO:tasks.workunit.client.0.smithi107.stdout:5/265: truncate d4/dc/f14 1363820 0 2024-04-25T00:18:23.594 INFO:tasks.workunit.client.0.smithi107.stdout:5/266: dread d4/dc/f2f [0,4194304] 0 2024-04-25T00:18:23.602 INFO:tasks.workunit.client.0.smithi107.stdout:8/260: dread d2/d18/d1a/f3a [0,4194304] 0 2024-04-25T00:18:23.602 INFO:tasks.workunit.client.0.smithi107.stdout:8/261: readlink d2/l19 0 2024-04-25T00:18:23.602 INFO:tasks.workunit.client.0.smithi107.stdout:8/262: readlink d2/d18/d1a/d1b/d3d/l4b 0 2024-04-25T00:18:23.607 INFO:tasks.workunit.client.0.smithi107.stdout:5/267: write d4/d17/f38 [1087942,70793] 0 2024-04-25T00:18:23.608 INFO:tasks.workunit.client.0.smithi107.stdout:5/268: symlink d4/d1c/d5c/l5d 0 2024-04-25T00:18:23.609 INFO:tasks.workunit.client.0.smithi107.stdout:5/269: rmdir d4/dc/d26/d29 39 2024-04-25T00:18:23.610 INFO:tasks.workunit.client.0.smithi107.stdout:5/270: creat d4/dc/d26/d42/f5e x:0 0 0 2024-04-25T00:18:23.611 INFO:tasks.workunit.client.0.smithi107.stdout:5/271: symlink d4/d17/l5f 0 2024-04-25T00:18:23.615 INFO:tasks.workunit.client.0.smithi107.stdout:8/263: dread d2/d18/d1a/f3a [0,4194304] 0 2024-04-25T00:18:23.616 INFO:tasks.workunit.client.0.smithi107.stdout:5/272: dread d4/dc/fd [0,4194304] 0 2024-04-25T00:18:23.617 INFO:tasks.workunit.client.0.smithi107.stdout:5/273: symlink d4/dc/d26/d42/l60 0 2024-04-25T00:18:23.618 INFO:tasks.workunit.client.0.smithi107.stdout:5/274: mkdir d4/d35/d61 0 2024-04-25T00:18:23.618 INFO:tasks.workunit.client.0.smithi107.stdout:5/275: chown d4/dc/c34 6530 1 2024-04-25T00:18:23.623 INFO:tasks.workunit.client.0.smithi107.stdout:8/264: write d2/d18/f43 [4079977,57301] 0 2024-04-25T00:18:23.625 INFO:tasks.workunit.client.0.smithi107.stdout:8/265: rename d2/d18/d1a/d1b to d2/d18/d1a/d5e 0 2024-04-25T00:18:23.625 INFO:tasks.workunit.client.0.smithi107.stdout:8/266: stat d2/d18/d1a/f1e 0 2024-04-25T00:18:23.625 INFO:tasks.workunit.client.0.smithi107.stdout:8/267: dread - d2/f41 zero size 2024-04-25T00:18:23.626 INFO:tasks.workunit.client.0.smithi107.stdout:8/268: getdents d2 0 2024-04-25T00:18:23.627 INFO:tasks.workunit.client.0.smithi107.stdout:8/269: read d2/d18/d1a/d5e/f39 [854923,86747] 0 2024-04-25T00:18:23.628 INFO:tasks.workunit.client.0.smithi107.stdout:8/270: creat d2/d18/d1a/d5e/d4f/f5f x:0 0 0 2024-04-25T00:18:23.629 INFO:tasks.workunit.client.0.smithi107.stdout:8/271: symlink d2/d18/d1a/l60 0 2024-04-25T00:18:23.629 INFO:tasks.workunit.client.0.smithi107.stdout:8/272: fdatasync d2/f41 0 2024-04-25T00:18:23.630 INFO:tasks.workunit.client.0.smithi107.stdout:8/273: link d2/c15 d2/d18/d1a/d5e/d3d/c61 0 2024-04-25T00:18:23.630 INFO:tasks.workunit.client.0.smithi107.stdout:8/274: truncate d2/d18/f20 3216076 0 2024-04-25T00:18:23.631 INFO:tasks.workunit.client.0.smithi107.stdout:8/275: creat d2/d18/d1a/f62 x:0 0 0 2024-04-25T00:18:23.642 INFO:tasks.workunit.client.0.smithi107.stdout:5/276: dwrite d4/f48 [0,4194304] 0 2024-04-25T00:18:23.642 INFO:tasks.workunit.client.0.smithi107.stdout:5/277: stat d4/d17/l19 0 2024-04-25T00:18:23.642 INFO:tasks.workunit.client.0.smithi107.stdout:5/278: dread - d4/f16 zero size 2024-04-25T00:18:23.642 INFO:tasks.workunit.client.0.smithi107.stdout:5/279: read - d4/f1b zero size 2024-04-25T00:18:23.646 INFO:tasks.workunit.client.0.smithi107.stdout:5/280: mkdir d4/dc/d26/d62 0 2024-04-25T00:18:23.647 INFO:tasks.workunit.client.0.smithi107.stdout:5/281: unlink d4/d1c/f4e 0 2024-04-25T00:18:23.648 INFO:tasks.workunit.client.0.smithi107.stdout:5/282: creat d4/dc/f63 x:0 0 0 2024-04-25T00:18:23.653 INFO:tasks.workunit.client.0.smithi107.stdout:8/276: dwrite d2/d18/d1a/d33/f59 [4194304,4194304] 0 2024-04-25T00:18:23.659 INFO:tasks.workunit.client.0.smithi107.stdout:5/283: dread d4/dc/fe [0,4194304] 0 2024-04-25T00:18:23.667 INFO:tasks.workunit.client.0.smithi107.stdout:8/277: dread d2/d18/d1a/d33/f3f [4194304,4194304] 0 2024-04-25T00:18:23.668 INFO:tasks.workunit.client.0.smithi107.stdout:8/278: getdents d2/d18/d1a 0 2024-04-25T00:18:23.682 INFO:tasks.workunit.client.0.smithi107.stdout:8/279: write d2/d18/d1a/f3a [148299,62443] 0 2024-04-25T00:18:23.682 INFO:tasks.workunit.client.0.smithi107.stdout:5/284: dwrite d4/fa [0,4194304] 0 2024-04-25T00:18:23.682 INFO:tasks.workunit.client.0.smithi107.stdout:8/280: dread - d2/f57 zero size 2024-04-25T00:18:23.682 INFO:tasks.workunit.client.0.smithi107.stdout:8/281: fsync d2/ff 0 2024-04-25T00:18:23.682 INFO:tasks.workunit.client.0.smithi107.stdout:8/282: readlink d2/d18/l49 0 2024-04-25T00:18:23.684 INFO:tasks.workunit.client.0.smithi107.stdout:5/285: creat d4/dc/d26/f64 x:0 0 0 2024-04-25T00:18:23.685 INFO:tasks.workunit.client.0.smithi107.stdout:5/286: symlink d4/dc/d26/d29/l65 0 2024-04-25T00:18:23.687 INFO:tasks.workunit.client.0.smithi107.stdout:5/287: write d4/dc/f2f [167837,76487] 0 2024-04-25T00:18:23.701 INFO:tasks.workunit.client.0.smithi107.stdout:5/288: write d4/d17/f21 [3007392,68880] 0 2024-04-25T00:18:23.701 INFO:tasks.workunit.client.0.smithi107.stdout:8/283: dwrite d2/d18/d1a/d5e/d3e/f4c [0,4194304] 0 2024-04-25T00:18:23.704 INFO:tasks.workunit.client.0.smithi107.stdout:8/284: creat d2/f63 x:0 0 0 2024-04-25T00:18:23.707 INFO:tasks.workunit.client.0.smithi107.stdout:8/285: creat d2/d18/d1a/d33/f64 x:0 0 0 2024-04-25T00:18:23.715 INFO:tasks.workunit.client.0.smithi107.stdout:8/286: dread d2/f16 [4194304,4194304] 0 2024-04-25T00:18:23.717 INFO:tasks.workunit.client.0.smithi107.stdout:8/287: dread d2/f2f [0,4194304] 0 2024-04-25T00:18:23.718 INFO:tasks.workunit.client.0.smithi107.stdout:8/288: mknod d2/d18/d1a/d5e/d5c/c65 0 2024-04-25T00:18:23.719 INFO:tasks.workunit.client.0.smithi107.stdout:8/289: unlink d2/d18/d1a/d5e/l2a 0 2024-04-25T00:18:23.720 INFO:tasks.workunit.client.0.smithi107.stdout:8/290: readlink d2/d18/l49 0 2024-04-25T00:18:23.720 INFO:tasks.workunit.client.0.smithi107.stdout:8/291: write d2/d18/d1a/d5e/d4f/f52 [11,23998] 0 2024-04-25T00:18:23.720 INFO:tasks.workunit.client.0.smithi107.stdout:8/292: creat d2/d18/d1a/d33/f66 x:0 0 0 2024-04-25T00:18:23.721 INFO:tasks.workunit.client.0.smithi107.stdout:8/293: link d2/d18/d1a/d5e/d3d/l4b d2/d18/d1a/d5e/d5c/l67 0 2024-04-25T00:18:23.732 INFO:tasks.workunit.client.0.smithi107.stdout:8/294: write d2/f16 [4189776,23654] 0 2024-04-25T00:18:23.743 INFO:tasks.workunit.client.0.smithi107.stdout:8/295: dread d2/f16 [4194304,4194304] 0 2024-04-25T00:18:23.744 INFO:tasks.workunit.client.0.smithi107.stdout:8/296: creat d2/d18/f68 x:0 0 0 2024-04-25T00:18:23.744 INFO:tasks.workunit.client.0.smithi107.stdout:8/297: stat d2/d18/d1a/d5e/c24 0 2024-04-25T00:18:23.744 INFO:tasks.workunit.client.0.smithi107.stdout:8/298: symlink d2/l69 0 2024-04-25T00:18:23.762 INFO:tasks.workunit.client.0.smithi107.stdout:8/299: dwrite d2/d18/d1a/d5e/d4f/f5f [0,4194304] 0 2024-04-25T00:18:23.771 INFO:tasks.workunit.client.0.smithi107.stdout:8/300: dread d2/d18/d1a/f3a [0,4194304] 0 2024-04-25T00:18:23.773 INFO:tasks.workunit.client.0.smithi107.stdout:8/301: creat d2/d18/d1a/d5e/d3e/f6a x:0 0 0 2024-04-25T00:18:23.780 INFO:tasks.workunit.client.0.smithi107.stdout:8/302: write d2/f16 [7392292,74017] 0 2024-04-25T00:18:23.782 INFO:tasks.workunit.client.0.smithi107.stdout:8/303: creat d2/d18/d1a/d5e/d3e/f6b x:0 0 0 2024-04-25T00:18:23.789 INFO:tasks.workunit.client.0.smithi107.stdout:8/304: dread d2/d18/d1a/f48 [0,4194304] 0 2024-04-25T00:18:23.806 INFO:tasks.workunit.client.0.smithi107.stdout:8/305: dwrite d2/f11 [0,4194304] 0 2024-04-25T00:18:23.809 INFO:tasks.workunit.client.0.smithi107.stdout:8/306: fsync d2/d18/d1a/f37 0 2024-04-25T00:18:23.811 INFO:tasks.workunit.client.0.smithi107.stdout:8/307: creat d2/d18/d1a/f6c x:0 0 0 2024-04-25T00:18:23.827 INFO:tasks.workunit.client.0.smithi107.stdout:8/308: dwrite d2/d18/d1a/f25 [0,4194304] 0 2024-04-25T00:18:23.829 INFO:tasks.workunit.client.0.smithi107.stdout:8/309: getdents d2/d18/d1a/d5e/d3d 0 2024-04-25T00:18:24.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:24 smithi195 ceph-mon[32833]: pgmap v61: 129 pgs: 129 active+clean; 333 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 40 MiB/s wr, 842 op/s 2024-04-25T00:18:24.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:24 smithi170 ceph-mon[32807]: pgmap v61: 129 pgs: 129 active+clean; 333 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 40 MiB/s wr, 842 op/s 2024-04-25T00:18:24.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:24 smithi107 ceph-mon[29153]: pgmap v61: 129 pgs: 129 active+clean; 333 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 40 MiB/s wr, 842 op/s 2024-04-25T00:18:26.203 INFO:tasks.workunit.client.0.smithi107.stdout:5/289: sync 2024-04-25T00:18:26.203 INFO:tasks.workunit.client.0.smithi107.stdout:1/199: sync 2024-04-25T00:18:26.203 INFO:tasks.workunit.client.0.smithi107.stdout:5/290: dread - d4/dc/f63 zero size 2024-04-25T00:18:26.203 INFO:tasks.workunit.client.0.smithi107.stdout:2/216: sync 2024-04-25T00:18:26.203 INFO:tasks.workunit.client.0.smithi107.stdout:0/129: sync 2024-04-25T00:18:26.203 INFO:tasks.workunit.client.0.smithi107.stdout:4/138: sync 2024-04-25T00:18:26.203 INFO:tasks.workunit.client.0.smithi107.stdout:3/152: sync 2024-04-25T00:18:26.203 INFO:tasks.workunit.client.0.smithi107.stdout:7/130: sync 2024-04-25T00:18:26.204 INFO:tasks.workunit.client.0.smithi107.stdout:3/153: fsync f7 0 2024-04-25T00:18:26.204 INFO:tasks.workunit.client.0.smithi107.stdout:7/131: stat d0/d8/ce 0 2024-04-25T00:18:26.204 INFO:tasks.workunit.client.0.smithi107.stdout:4/139: write d3/d10/f2d [252660,73516] 0 2024-04-25T00:18:26.204 INFO:tasks.workunit.client.0.smithi107.stdout:6/187: sync 2024-04-25T00:18:26.204 INFO:tasks.workunit.client.0.smithi107.stdout:9/173: sync 2024-04-25T00:18:26.204 INFO:tasks.workunit.client.0.smithi107.stdout:7/132: mknod d0/d21/c25 0 2024-04-25T00:18:26.204 INFO:tasks.workunit.client.0.smithi107.stdout:8/310: sync 2024-04-25T00:18:26.204 INFO:tasks.workunit.client.0.smithi107.stdout:4/140: mkdir d3/d5/d32 0 2024-04-25T00:18:26.206 INFO:tasks.workunit.client.0.smithi107.stdout:6/188: mkdir d1/d2/db/d3c/d43 0 2024-04-25T00:18:26.210 INFO:tasks.workunit.client.0.smithi107.stdout:4/141: rename d3/d5/c20 to d3/d24/d25/c33 0 2024-04-25T00:18:26.211 INFO:tasks.workunit.client.0.smithi107.stdout:4/142: write d3/f18 [83505,95192] 0 2024-04-25T00:18:26.211 INFO:tasks.workunit.client.0.smithi107.stdout:6/189: mknod d1/d2/db/d32/d36/d3a/c44 0 2024-04-25T00:18:26.213 INFO:tasks.workunit.client.0.smithi107.stdout:2/217: dread d6/ff [0,4194304] 0 2024-04-25T00:18:26.213 INFO:tasks.workunit.client.0.smithi107.stdout:2/218: write d6/fe [3917180,5773] 0 2024-04-25T00:18:26.213 INFO:tasks.workunit.client.0.smithi107.stdout:4/143: rename d3/d10/f11 to d3/d5/d32/f34 0 2024-04-25T00:18:26.213 INFO:tasks.workunit.client.0.smithi107.stdout:4/144: chown d3/d5/f22 1400 1 2024-04-25T00:18:26.215 INFO:tasks.workunit.client.0.smithi107.stdout:4/145: creat d3/f35 x:0 0 0 2024-04-25T00:18:26.215 INFO:tasks.workunit.client.0.smithi107.stdout:8/311: dread d2/d18/d1a/d33/f3f [0,4194304] 0 2024-04-25T00:18:26.223 INFO:tasks.workunit.client.0.smithi107.stdout:6/190: dread d1/d2/db/fd [0,4194304] 0 2024-04-25T00:18:26.224 INFO:tasks.workunit.client.0.smithi107.stdout:0/130: dwrite dc/f28 [0,4194304] 0 2024-04-25T00:18:26.226 INFO:tasks.workunit.client.0.smithi107.stdout:8/312: write d2/d18/d1a/d33/f34 [589997,93793] 0 2024-04-25T00:18:26.226 INFO:tasks.workunit.client.0.smithi107.stdout:8/313: getdents d2/d18/d1a/d5e/d3d 0 2024-04-25T00:18:26.230 INFO:tasks.workunit.client.0.smithi107.stdout:1/200: creat d2/dd/d1b/f42 x:0 0 0 2024-04-25T00:18:26.230 INFO:tasks.workunit.client.0.smithi107.stdout:1/201: read - d2/dd/d1b/f42 zero size 2024-04-25T00:18:26.230 INFO:tasks.workunit.client.0.smithi107.stdout:1/202: readlink d2/dd/d1b/l2b 0 2024-04-25T00:18:26.233 INFO:tasks.workunit.client.0.smithi107.stdout:6/191: dread d1/f1f [4194304,4194304] 0 2024-04-25T00:18:26.233 INFO:tasks.workunit.client.0.smithi107.stdout:9/174: mknod d0/c3a 0 2024-04-25T00:18:26.234 INFO:tasks.workunit.client.0.smithi107.stdout:6/192: symlink d1/d2/db/d32/l45 0 2024-04-25T00:18:26.235 INFO:tasks.workunit.client.0.smithi107.stdout:6/193: chown d1/d2/db/d32/d36/f3f 243635 1 2024-04-25T00:18:26.236 INFO:tasks.workunit.client.0.smithi107.stdout:6/194: creat d1/d2/db/d32/d36/f46 x:0 0 0 2024-04-25T00:18:26.239 INFO:tasks.workunit.client.0.smithi107.stdout:6/195: creat d1/d2/db/d32/d36/d3a/f47 x:0 0 0 2024-04-25T00:18:26.240 INFO:tasks.workunit.client.0.smithi107.stdout:6/196: mkdir d1/d2/db/d48 0 2024-04-25T00:18:26.243 INFO:tasks.workunit.client.0.smithi107.stdout:2/219: mkdir d6/d29/d4e 0 2024-04-25T00:18:26.243 INFO:tasks.workunit.client.0.smithi107.stdout:2/220: dread - d6/d18/d40/d42/d2b/d30/f3e zero size 2024-04-25T00:18:26.244 INFO:tasks.workunit.client.0.smithi107.stdout:0/131: dwrite f1 [0,4194304] 0 2024-04-25T00:18:26.246 INFO:tasks.workunit.client.0.smithi107.stdout:5/291: symlink d4/dc/l66 0 2024-04-25T00:18:26.252 INFO:tasks.workunit.client.0.smithi107.stdout:6/197: dread d1/d2/db/f34 [0,4194304] 0 2024-04-25T00:18:26.252 INFO:tasks.workunit.client.0.smithi107.stdout:2/221: dread d6/f17 [0,4194304] 0 2024-04-25T00:18:26.254 INFO:tasks.workunit.client.0.smithi107.stdout:8/314: mkdir d2/d18/d1a/d5e/d6d 0 2024-04-25T00:18:26.258 INFO:tasks.workunit.client.0.smithi107.stdout:1/203: truncate d2/dd/d1a/f1d 570701 0 2024-04-25T00:18:26.261 INFO:tasks.workunit.client.0.smithi107.stdout:9/175: symlink d0/d1b/l3b 0 2024-04-25T00:18:26.262 INFO:tasks.workunit.client.0.smithi107.stdout:2/222: write d6/d20/d22/f26 [4590286,113609] 0 2024-04-25T00:18:26.262 INFO:tasks.workunit.client.0.smithi107.stdout:2/223: truncate d6/f3d 601779 0 2024-04-25T00:18:26.267 INFO:tasks.workunit.client.0.smithi107.stdout:5/292: getdents d4/dc/d26/d29/d2e 0 2024-04-25T00:18:26.267 INFO:tasks.workunit.client.0.smithi107.stdout:5/293: write d4/dc/fd [1513739,95726] 0 2024-04-25T00:18:26.270 INFO:tasks.workunit.client.0.smithi107.stdout:6/198: mknod d1/d14/c49 0 2024-04-25T00:18:26.272 INFO:tasks.workunit.client.0.smithi107.stdout:8/315: mkdir d2/d18/d1a/d5e/d5c/d6e 0 2024-04-25T00:18:26.278 INFO:tasks.workunit.client.0.smithi107.stdout:1/204: creat d2/da/f43 x:0 0 0 2024-04-25T00:18:26.279 INFO:tasks.workunit.client.0.smithi107.stdout:2/224: dwrite d6/d18/d40/d42/d2b/d30/f3e [0,4194304] 0 2024-04-25T00:18:26.279 INFO:tasks.workunit.client.0.smithi107.stdout:2/225: truncate d6/f11 676979 0 2024-04-25T00:18:26.279 INFO:tasks.workunit.client.0.smithi107.stdout:7/133: creat d0/f26 x:0 0 0 2024-04-25T00:18:26.280 INFO:tasks.workunit.client.0.smithi107.stdout:9/176: creat d0/d8/de/d20/f3c x:0 0 0 2024-04-25T00:18:26.280 INFO:tasks.workunit.client.0.smithi107.stdout:8/316: dread d2/d18/f20 [0,4194304] 0 2024-04-25T00:18:26.280 INFO:tasks.workunit.client.0.smithi107.stdout:7/134: symlink d0/d8/l27 0 2024-04-25T00:18:26.281 INFO:tasks.workunit.client.0.smithi107.stdout:8/317: dread d2/d18/d1a/d5e/f47 [0,4194304] 0 2024-04-25T00:18:26.281 INFO:tasks.workunit.client.0.smithi107.stdout:8/318: dread - d2/d18/d1a/d33/f66 zero size 2024-04-25T00:18:26.283 INFO:tasks.workunit.client.0.smithi107.stdout:7/135: link d0/c17 d0/d8/db/c28 0 2024-04-25T00:18:26.283 INFO:tasks.workunit.client.0.smithi107.stdout:7/136: read - d0/f1c zero size 2024-04-25T00:18:26.286 INFO:tasks.workunit.client.0.smithi107.stdout:0/132: unlink dc/dd/f17 0 2024-04-25T00:18:26.288 INFO:tasks.workunit.client.0.smithi107.stdout:5/294: creat d4/dc/d26/d29/d51/f67 x:0 0 0 2024-04-25T00:18:26.290 INFO:tasks.workunit.client.0.smithi107.stdout:7/137: rename d0/d8/db/d18/d19/c1b to d0/d8/db/c29 0 2024-04-25T00:18:26.290 INFO:tasks.workunit.client.0.smithi107.stdout:6/199: symlink d1/d14/d18/l4a 0 2024-04-25T00:18:26.291 INFO:tasks.workunit.client.0.smithi107.stdout:7/138: creat d0/d8/db/f2a x:0 0 0 2024-04-25T00:18:26.292 INFO:tasks.workunit.client.0.smithi107.stdout:7/139: write d0/d8/db/fc [9120146,69367] 0 2024-04-25T00:18:26.292 INFO:tasks.workunit.client.0.smithi107.stdout:6/200: mknod d1/d2/d3/d26/c4b 0 2024-04-25T00:18:26.293 INFO:tasks.workunit.client.0.smithi107.stdout:1/205: link d2/da/c24 d2/dd/d1a/d1f/c44 0 2024-04-25T00:18:26.294 INFO:tasks.workunit.client.0.smithi107.stdout:7/140: mknod d0/d8/c2b 0 2024-04-25T00:18:26.299 INFO:tasks.workunit.client.0.smithi107.stdout:8/319: dwrite d2/d18/d1a/d5e/f56 [0,4194304] 0 2024-04-25T00:18:26.300 INFO:tasks.workunit.client.0.smithi107.stdout:8/320: truncate d2/d18/d1a/d5e/f47 1048045 0 2024-04-25T00:18:26.300 INFO:tasks.workunit.client.0.smithi107.stdout:8/321: write d2/f26 [6789633,81865] 0 2024-04-25T00:18:26.301 INFO:tasks.workunit.client.0.smithi107.stdout:3/154: stat d9/d15/d1a/c26 0 2024-04-25T00:18:26.302 INFO:tasks.workunit.client.0.smithi107.stdout:1/206: dread d2/dd/d1a/d1f/f39 [0,4194304] 0 2024-04-25T00:18:26.302 INFO:tasks.workunit.client.0.smithi107.stdout:2/226: symlink d6/d18/d40/d42/d2b/d30/d4c/l4f 0 2024-04-25T00:18:26.303 INFO:tasks.workunit.client.0.smithi107.stdout:6/201: write d1/d2/d3/f22 [2746526,94342] 0 2024-04-25T00:18:26.303 INFO:tasks.workunit.client.0.smithi107.stdout:6/202: readlink d1/d2/db/d32/l45 0 2024-04-25T00:18:26.305 INFO:tasks.workunit.client.0.smithi107.stdout:0/133: dwrite f1 [0,4194304] 0 2024-04-25T00:18:26.305 INFO:tasks.workunit.client.0.smithi107.stdout:0/134: write dc/f18 [613842,120982] 0 2024-04-25T00:18:26.307 INFO:tasks.workunit.client.0.smithi107.stdout:3/155: creat d9/d15/d1a/f30 x:0 0 0 2024-04-25T00:18:26.307 INFO:tasks.workunit.client.0.smithi107.stdout:5/295: creat d4/dc/d54/f68 x:0 0 0 2024-04-25T00:18:26.307 INFO:tasks.workunit.client.0.smithi107.stdout:5/296: truncate d4/dc/f13 1178435 0 2024-04-25T00:18:26.310 INFO:tasks.workunit.client.0.smithi107.stdout:5/297: dread d4/d35/f37 [0,4194304] 0 2024-04-25T00:18:26.314 INFO:tasks.workunit.client.0.smithi107.stdout:8/322: creat d2/d18/d1a/d5e/d6d/f6f x:0 0 0 2024-04-25T00:18:26.314 INFO:tasks.workunit.client.0.smithi107.stdout:7/141: dwrite d0/d8/db/f10 [0,4194304] 0 2024-04-25T00:18:26.317 INFO:tasks.workunit.client.0.smithi107.stdout:1/207: unlink d2/dd/d1a/l26 0 2024-04-25T00:18:26.318 INFO:tasks.workunit.client.0.smithi107.stdout:1/208: truncate d2/dd/d1a/d1f/f39 959692 0 2024-04-25T00:18:26.318 INFO:tasks.workunit.client.0.smithi107.stdout:7/142: getdents d0/d21 0 2024-04-25T00:18:26.319 INFO:tasks.workunit.client.0.smithi107.stdout:7/143: mkdir d0/d8/d2c 0 2024-04-25T00:18:26.319 INFO:tasks.workunit.client.0.smithi107.stdout:2/227: link d6/d18/l1f d6/d29/d4e/l50 0 2024-04-25T00:18:26.319 INFO:tasks.workunit.client.0.smithi107.stdout:5/298: dread d4/f48 [0,4194304] 0 2024-04-25T00:18:26.320 INFO:tasks.workunit.client.0.smithi107.stdout:7/144: mkdir d0/d8/db/d18/d19/d2d 0 2024-04-25T00:18:26.320 INFO:tasks.workunit.client.0.smithi107.stdout:6/203: mkdir d1/d4c 0 2024-04-25T00:18:26.321 INFO:tasks.workunit.client.0.smithi107.stdout:7/145: mkdir d0/d8/d2c/d2e 0 2024-04-25T00:18:26.330 INFO:tasks.workunit.client.0.smithi107.stdout:8/323: mknod d2/d18/d1a/d5e/c70 0 2024-04-25T00:18:26.332 INFO:tasks.workunit.client.0.smithi107.stdout:5/299: truncate d4/fa 2297558 0 2024-04-25T00:18:26.332 INFO:tasks.workunit.client.0.smithi107.stdout:5/300: write d4/dc/f11 [432709,99049] 0 2024-04-25T00:18:26.332 INFO:tasks.workunit.client.0.smithi107.stdout:5/301: stat d4/dc/l47 0 2024-04-25T00:18:26.335 INFO:tasks.workunit.client.0.smithi107.stdout:3/156: rename d9/d22/d1b/c1f to d9/d22/c31 0 2024-04-25T00:18:26.336 INFO:tasks.workunit.client.0.smithi107.stdout:6/204: rmdir d1/d14 39 2024-04-25T00:18:26.336 INFO:tasks.workunit.client.0.smithi107.stdout:6/205: truncate d1/d2/db/d32/d36/f46 407444 0 2024-04-25T00:18:26.336 INFO:tasks.workunit.client.0.smithi107.stdout:6/206: write d1/d2/d3/f41 [886349,81817] 0 2024-04-25T00:18:26.336 INFO:tasks.workunit.client.0.smithi107.stdout:6/207: dread - d1/d2/db/f38 zero size 2024-04-25T00:18:26.337 INFO:tasks.workunit.client.0.smithi107.stdout:1/209: mknod d2/da/d2f/c45 0 2024-04-25T00:18:26.338 INFO:tasks.workunit.client.0.smithi107.stdout:5/302: creat d4/d17/f69 x:0 0 0 2024-04-25T00:18:26.339 INFO:tasks.workunit.client.0.smithi107.stdout:8/324: write d2/d18/d1a/f25 [3288278,88602] 0 2024-04-25T00:18:26.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:26 smithi195 ceph-mon[32833]: pgmap v62: 129 pgs: 129 active+clean; 333 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 37 MiB/s wr, 442 op/s 2024-04-25T00:18:26.340 INFO:tasks.workunit.client.0.smithi107.stdout:3/157: creat d9/d15/d1a/f32 x:0 0 0 2024-04-25T00:18:26.340 INFO:tasks.workunit.client.0.smithi107.stdout:5/303: stat d4/dc/d26/c50 0 2024-04-25T00:18:26.341 INFO:tasks.workunit.client.0.smithi107.stdout:3/158: truncate d9/d15/d1a/f1c 2588 0 2024-04-25T00:18:26.341 INFO:tasks.workunit.client.0.smithi107.stdout:6/208: unlink d1/d2/d3/f2f 0 2024-04-25T00:18:26.343 INFO:tasks.workunit.client.0.smithi107.stdout:8/325: link d2/d18/d1a/d5e/d3d/c4a d2/d18/d1a/c71 0 2024-04-25T00:18:26.344 INFO:tasks.workunit.client.0.smithi107.stdout:5/304: dread d4/dc/f13 [0,4194304] 0 2024-04-25T00:18:26.345 INFO:tasks.workunit.client.0.smithi107.stdout:5/305: creat d4/d35/f6a x:0 0 0 2024-04-25T00:18:26.348 INFO:tasks.workunit.client.0.smithi107.stdout:7/146: dwrite d0/f1d [0,4194304] 0 2024-04-25T00:18:26.348 INFO:tasks.workunit.client.0.smithi107.stdout:2/228: dwrite d6/f10 [4194304,4194304] 0 2024-04-25T00:18:26.348 INFO:tasks.workunit.client.0.smithi107.stdout:2/229: stat d6/d18/d40/d42/d2b/f32 0 2024-04-25T00:18:26.357 INFO:tasks.workunit.client.0.smithi107.stdout:0/135: creat dc/dd/d16/f2a x:0 0 0 2024-04-25T00:18:26.358 INFO:tasks.workunit.client.0.smithi107.stdout:0/136: truncate dc/dd/d16/f22 766068 0 2024-04-25T00:18:26.358 INFO:tasks.workunit.client.0.smithi107.stdout:0/137: truncate f1 4893256 0 2024-04-25T00:18:26.359 INFO:tasks.workunit.client.0.smithi107.stdout:7/147: write d0/d8/db/fc [5980448,17114] 0 2024-04-25T00:18:26.360 INFO:tasks.workunit.client.0.smithi107.stdout:1/210: dwrite d2/dd/d1b/f3a [0,4194304] 0 2024-04-25T00:18:26.360 INFO:tasks.workunit.client.0.smithi107.stdout:1/211: readlink d2/dd/d25/l2c 0 2024-04-25T00:18:26.361 INFO:tasks.workunit.client.0.smithi107.stdout:7/148: creat d0/d8/db/d18/d19/f2f x:0 0 0 2024-04-25T00:18:26.364 INFO:tasks.workunit.client.0.smithi107.stdout:7/149: creat d0/d8/d2c/d2e/f30 x:0 0 0 2024-04-25T00:18:26.372 INFO:tasks.workunit.client.0.smithi107.stdout:0/138: rmdir dc/dd/d20 0 2024-04-25T00:18:26.375 INFO:tasks.workunit.client.0.smithi107.stdout:0/139: creat dc/d24/d25/f2b x:0 0 0 2024-04-25T00:18:26.375 INFO:tasks.workunit.client.0.smithi107.stdout:6/209: read d1/d2/db/f34 [4098167,122616] 0 2024-04-25T00:18:26.377 INFO:tasks.workunit.client.0.smithi107.stdout:0/140: rmdir dc/d24 39 2024-04-25T00:18:26.378 INFO:tasks.workunit.client.0.smithi107.stdout:0/141: mknod dc/c2c 0 2024-04-25T00:18:26.383 INFO:tasks.workunit.client.0.smithi107.stdout:7/150: dwrite d0/f11 [0,4194304] 0 2024-04-25T00:18:26.384 INFO:tasks.workunit.client.0.smithi107.stdout:3/159: dwrite d9/d15/d1a/f30 [0,4194304] 0 2024-04-25T00:18:26.385 INFO:tasks.workunit.client.0.smithi107.stdout:2/230: dread d6/fe [0,4194304] 0 2024-04-25T00:18:26.385 INFO:tasks.workunit.client.0.smithi107.stdout:2/231: truncate d6/d18/d40/d42/f3b 42689 0 2024-04-25T00:18:26.385 INFO:tasks.workunit.client.0.smithi107.stdout:2/232: stat d6/f7 0 2024-04-25T00:18:26.385 INFO:tasks.workunit.client.0.smithi107.stdout:2/233: write d6/d18/d40/d42/f3f [171429,50567] 0 2024-04-25T00:18:26.385 INFO:tasks.workunit.client.0.smithi107.stdout:2/234: write d6/d18/d40/d42/f3b [823103,61873] 0 2024-04-25T00:18:26.387 INFO:tasks.workunit.client.0.smithi107.stdout:7/151: creat d0/d8/db/d18/d19/d2d/f31 x:0 0 0 2024-04-25T00:18:26.388 INFO:tasks.workunit.client.0.smithi107.stdout:2/235: rename d6/d18/d40/d42/d2b/d30/f3e to d6/f51 0 2024-04-25T00:18:26.388 INFO:tasks.workunit.client.0.smithi107.stdout:2/236: chown d6/d20 3 1 2024-04-25T00:18:26.388 INFO:tasks.workunit.client.0.smithi107.stdout:2/237: read f4 [1294678,25399] 0 2024-04-25T00:18:26.389 INFO:tasks.workunit.client.0.smithi107.stdout:7/152: creat d0/d21/f32 x:0 0 0 2024-04-25T00:18:26.389 INFO:tasks.workunit.client.0.smithi107.stdout:7/153: chown d0/f13 9375 1 2024-04-25T00:18:26.389 INFO:tasks.workunit.client.0.smithi107.stdout:7/154: write d0/f26 [827384,59872] 0 2024-04-25T00:18:26.389 INFO:tasks.workunit.client.0.smithi107.stdout:7/155: write d0/d8/d2c/d2e/f30 [942221,83746] 0 2024-04-25T00:18:26.389 INFO:tasks.workunit.client.0.smithi107.stdout:8/326: dwrite d2/d18/d1a/f25 [4194304,4194304] 0 2024-04-25T00:18:26.389 INFO:tasks.workunit.client.0.smithi107.stdout:1/212: dread d2/da/f15 [0,4194304] 0 2024-04-25T00:18:26.390 INFO:tasks.workunit.client.0.smithi107.stdout:7/156: mknod d0/d8/db/d18/d19/d2d/c33 0 2024-04-25T00:18:26.391 INFO:tasks.workunit.client.0.smithi107.stdout:7/157: read - d0/f16 zero size 2024-04-25T00:18:26.391 INFO:tasks.workunit.client.0.smithi107.stdout:8/327: stat d2/d18/d1a/d5e/d3d/c4a 0 2024-04-25T00:18:26.391 INFO:tasks.workunit.client.0.smithi107.stdout:8/328: truncate d2/f45 4621403 0 2024-04-25T00:18:26.393 INFO:tasks.workunit.client.0.smithi107.stdout:7/158: symlink d0/d8/db/d18/d19/d2d/l34 0 2024-04-25T00:18:26.393 INFO:tasks.workunit.client.0.smithi107.stdout:8/329: mknod d2/d18/d1a/d5e/d6d/c72 0 2024-04-25T00:18:26.396 INFO:tasks.workunit.client.0.smithi107.stdout:1/213: link d2/da/lf d2/da/d2f/l46 0 2024-04-25T00:18:26.396 INFO:tasks.workunit.client.0.smithi107.stdout:7/159: mknod d0/d8/db/d18/d19/c35 0 2024-04-25T00:18:26.396 INFO:tasks.workunit.client.0.smithi107.stdout:5/306: dwrite d4/dc/f2f [0,4194304] 0 2024-04-25T00:18:26.396 INFO:tasks.workunit.client.0.smithi107.stdout:5/307: chown d4/d17/l19 292 1 2024-04-25T00:18:26.396 INFO:tasks.workunit.client.0.smithi107.stdout:6/210: dread d1/d2/db/d32/d36/f3f [0,4194304] 0 2024-04-25T00:18:26.397 INFO:tasks.workunit.client.0.smithi107.stdout:0/142: read f1 [3332330,29791] 0 2024-04-25T00:18:26.397 INFO:tasks.workunit.client.0.smithi107.stdout:0/143: dread - dc/d24/d25/f2b zero size 2024-04-25T00:18:26.398 INFO:tasks.workunit.client.0.smithi107.stdout:8/330: creat d2/d18/d1a/f73 x:0 0 0 2024-04-25T00:18:26.400 INFO:tasks.workunit.client.0.smithi107.stdout:7/160: symlink d0/d21/l36 0 2024-04-25T00:18:26.400 INFO:tasks.workunit.client.0.smithi107.stdout:7/161: chown d0/d8/db/f15 4903708 1 2024-04-25T00:18:26.403 INFO:tasks.workunit.client.0.smithi107.stdout:1/214: rename d2/dd/d1a/d1f to d2/d47 0 2024-04-25T00:18:26.403 INFO:tasks.workunit.client.0.smithi107.stdout:1/215: readlink d2/da/l14 0 2024-04-25T00:18:26.406 INFO:tasks.workunit.client.0.smithi107.stdout:3/160: read d9/d15/d1a/f30 [2937614,128761] 0 2024-04-25T00:18:26.406 INFO:tasks.workunit.client.0.smithi107.stdout:3/161: fdatasync d9/f29 0 2024-04-25T00:18:26.416 INFO:tasks.workunit.client.0.smithi107.stdout:5/308: rmdir d4/d17 39 2024-04-25T00:18:26.417 INFO:tasks.workunit.client.0.smithi107.stdout:8/331: mkdir d2/d18/d1a/d5e/d3e/d74 0 2024-04-25T00:18:26.417 INFO:tasks.workunit.client.0.smithi107.stdout:8/332: chown d2/d18/d1a/d5e/f56 14915 1 2024-04-25T00:18:26.417 INFO:tasks.workunit.client.0.smithi107.stdout:2/238: dwrite d6/d20/d22/f1a [0,4194304] 0 2024-04-25T00:18:26.417 INFO:tasks.workunit.client.0.smithi107.stdout:2/239: fdatasync d6/d18/d40/d42/d2b/f32 0 2024-04-25T00:18:26.418 INFO:tasks.workunit.client.0.smithi107.stdout:1/216: rename d2/da/d2f/l46 to d2/dd/l48 0 2024-04-25T00:18:26.419 INFO:tasks.workunit.client.0.smithi107.stdout:8/333: mknod d2/d18/d1a/d5e/d4f/c75 0 2024-04-25T00:18:26.419 INFO:tasks.workunit.client.0.smithi107.stdout:8/334: chown d2/d18/d1a/d5e/d3d/l4b 3512 1 2024-04-25T00:18:26.420 INFO:tasks.workunit.client.0.smithi107.stdout:3/162: dread d9/d22/f1e [0,4194304] 0 2024-04-25T00:18:26.420 INFO:tasks.workunit.client.0.smithi107.stdout:0/144: dwrite dc/dd/d16/f1b [0,4194304] 0 2024-04-25T00:18:26.421 INFO:tasks.workunit.client.0.smithi107.stdout:1/217: dread d2/d47/f39 [0,4194304] 0 2024-04-25T00:18:26.422 INFO:tasks.workunit.client.0.smithi107.stdout:8/335: read d2/f16 [8366688,103880] 0 2024-04-25T00:18:26.422 INFO:tasks.workunit.client.0.smithi107.stdout:8/336: stat d2/f41 0 2024-04-25T00:18:26.422 INFO:tasks.workunit.client.0.smithi107.stdout:3/163: mknod d9/d15/c33 0 2024-04-25T00:18:26.422 INFO:tasks.workunit.client.0.smithi107.stdout:3/164: read d9/d15/d1a/f30 [2426656,100511] 0 2024-04-25T00:18:26.425 INFO:tasks.workunit.client.0.smithi107.stdout:1/218: creat d2/da/d40/f49 x:0 0 0 2024-04-25T00:18:26.425 INFO:tasks.workunit.client.0.smithi107.stdout:4/146: sync 2024-04-25T00:18:26.425 INFO:tasks.workunit.client.0.smithi107.stdout:9/177: sync 2024-04-25T00:18:26.425 INFO:tasks.workunit.client.0.smithi107.stdout:9/178: write d0/fc [2579070,117189] 0 2024-04-25T00:18:26.426 INFO:tasks.workunit.client.0.smithi107.stdout:9/179: chown d0/d1b/l3b 3064011 1 2024-04-25T00:18:26.426 INFO:tasks.workunit.client.0.smithi107.stdout:9/180: chown d0/d8/d34/l39 60463572 1 2024-04-25T00:18:26.426 INFO:tasks.workunit.client.0.smithi107.stdout:6/211: dread d1/d2/d3/f41 [0,4194304] 0 2024-04-25T00:18:26.427 INFO:tasks.workunit.client.0.smithi107.stdout:3/165: link d9/d22/c31 d9/d22/c34 0 2024-04-25T00:18:26.427 INFO:tasks.workunit.client.0.smithi107.stdout:3/166: chown d9/d22/f18 3607982 1 2024-04-25T00:18:26.428 INFO:tasks.workunit.client.0.smithi107.stdout:6/212: rename d1/d2/db/f38 to d1/d2/db/f4d 0 2024-04-25T00:18:26.432 INFO:tasks.workunit.client.0.smithi107.stdout:9/181: mkdir d0/d8/d34/d36/d3d 0 2024-04-25T00:18:26.432 INFO:tasks.workunit.client.0.smithi107.stdout:9/182: dread - d0/d8/d32/f38 zero size 2024-04-25T00:18:26.433 INFO:tasks.workunit.client.0.smithi107.stdout:3/167: creat d9/d15/f35 x:0 0 0 2024-04-25T00:18:26.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:26 smithi170 ceph-mon[32807]: pgmap v62: 129 pgs: 129 active+clean; 333 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 37 MiB/s wr, 442 op/s 2024-04-25T00:18:26.435 INFO:tasks.workunit.client.0.smithi107.stdout:8/337: dread d2/d18/d1a/d33/f3f [0,4194304] 0 2024-04-25T00:18:26.436 INFO:tasks.workunit.client.0.smithi107.stdout:8/338: getdents d2/d18/d1a/d5e/d3e 0 2024-04-25T00:18:26.438 INFO:tasks.workunit.client.0.smithi107.stdout:8/339: rename d2/f2f to d2/d18/d1a/d5e/d4f/f76 0 2024-04-25T00:18:26.439 INFO:tasks.workunit.client.0.smithi107.stdout:3/168: dread d9/d22/f1e [0,4194304] 0 2024-04-25T00:18:26.439 INFO:tasks.workunit.client.0.smithi107.stdout:4/147: write d3/f2a [1951061,5165] 0 2024-04-25T00:18:26.442 INFO:tasks.workunit.client.0.smithi107.stdout:4/148: rename d3/d10/l1d to d3/d5/l36 0 2024-04-25T00:18:26.442 INFO:tasks.workunit.client.0.smithi107.stdout:4/149: fsync d3/f21 0 2024-04-25T00:18:26.442 INFO:tasks.workunit.client.0.smithi107.stdout:4/150: stat d3 0 2024-04-25T00:18:26.443 INFO:tasks.workunit.client.0.smithi107.stdout:2/240: dwrite d6/f11 [0,4194304] 0 2024-04-25T00:18:26.443 INFO:tasks.workunit.client.0.smithi107.stdout:4/151: symlink d3/d5/d32/l37 0 2024-04-25T00:18:26.445 INFO:tasks.workunit.client.0.smithi107.stdout:0/145: dwrite dc/dd/d16/f1b [0,4194304] 0 2024-04-25T00:18:26.445 INFO:tasks.workunit.client.0.smithi107.stdout:4/152: getdents d3 0 2024-04-25T00:18:26.446 INFO:tasks.workunit.client.0.smithi107.stdout:2/241: rmdir d6/d18/d40/d42/d2b 39 2024-04-25T00:18:26.448 INFO:tasks.workunit.client.0.smithi107.stdout:6/213: dread d1/d2/db/f34 [0,4194304] 0 2024-04-25T00:18:26.451 INFO:tasks.workunit.client.0.smithi107.stdout:0/146: creat dc/d24/f2d x:0 0 0 2024-04-25T00:18:26.451 INFO:tasks.workunit.client.0.smithi107.stdout:2/242: rename d6/d18/d40/d42/l44 to d6/d20/d22/l52 0 2024-04-25T00:18:26.451 INFO:tasks.workunit.client.0.smithi107.stdout:9/183: dread d0/f2 [0,4194304] 0 2024-04-25T00:18:26.452 INFO:tasks.workunit.client.0.smithi107.stdout:9/184: truncate d0/f30 178868 0 2024-04-25T00:18:26.455 INFO:tasks.workunit.client.0.smithi107.stdout:1/219: dwrite d2/d2a/f38 [0,4194304] 0 2024-04-25T00:18:26.455 INFO:tasks.workunit.client.0.smithi107.stdout:3/169: dread d9/d15/d1a/f30 [0,4194304] 0 2024-04-25T00:18:26.455 INFO:tasks.workunit.client.0.smithi107.stdout:3/170: read - d9/d22/f2e zero size 2024-04-25T00:18:26.456 INFO:tasks.workunit.client.0.smithi107.stdout:2/243: rename d6/d20/d22/l35 to d6/d20/l53 0 2024-04-25T00:18:26.456 INFO:tasks.workunit.client.0.smithi107.stdout:3/171: read d9/f14 [751452,118731] 0 2024-04-25T00:18:26.456 INFO:tasks.workunit.client.0.smithi107.stdout:0/147: link dc/dd/l27 dc/l2e 0 2024-04-25T00:18:26.457 INFO:tasks.workunit.client.0.smithi107.stdout:3/172: getdents d9/d15/d1a/d2c 0 2024-04-25T00:18:26.458 INFO:tasks.workunit.client.0.smithi107.stdout:4/153: dread d3/d10/f1b [0,4194304] 0 2024-04-25T00:18:26.459 INFO:tasks.workunit.client.0.smithi107.stdout:4/154: creat d3/d5/d32/f38 x:0 0 0 2024-04-25T00:18:26.463 INFO:tasks.workunit.client.0.smithi107.stdout:3/173: mknod d9/c36 0 2024-04-25T00:18:26.465 INFO:tasks.workunit.client.0.smithi107.stdout:3/174: rmdir d9/d15 39 2024-04-25T00:18:26.466 INFO:tasks.workunit.client.0.smithi107.stdout:4/155: read d3/d10/f1b [1928033,114098] 0 2024-04-25T00:18:26.468 INFO:tasks.workunit.client.0.smithi107.stdout:4/156: mknod d3/d10/c39 0 2024-04-25T00:18:26.468 INFO:tasks.workunit.client.0.smithi107.stdout:4/157: dread - d3/d24/d25/f28 zero size 2024-04-25T00:18:26.470 INFO:tasks.workunit.client.0.smithi107.stdout:4/158: symlink d3/d5/d32/l3a 0 2024-04-25T00:18:26.472 INFO:tasks.workunit.client.0.smithi107.stdout:2/244: write d6/f10 [4849253,26798] 0 2024-04-25T00:18:26.473 INFO:tasks.workunit.client.0.smithi107.stdout:4/159: creat d3/d24/f3b x:0 0 0 2024-04-25T00:18:26.473 INFO:tasks.workunit.client.0.smithi107.stdout:4/160: read - d3/d5/d32/f38 zero size 2024-04-25T00:18:26.473 INFO:tasks.workunit.client.0.smithi107.stdout:6/214: dwrite d1/d2/db/f19 [0,4194304] 0 2024-04-25T00:18:26.475 INFO:tasks.workunit.client.0.smithi107.stdout:2/245: symlink d6/d20/d22/d2c/l54 0 2024-04-25T00:18:26.477 INFO:tasks.workunit.client.0.smithi107.stdout:2/246: creat d6/d18/d40/f55 x:0 0 0 2024-04-25T00:18:26.479 INFO:tasks.workunit.client.0.smithi107.stdout:2/247: mkdir d6/d20/d22/d56 0 2024-04-25T00:18:26.479 INFO:tasks.workunit.client.0.smithi107.stdout:2/248: readlink d6/d18/d40/d42/d2b/l2f 0 2024-04-25T00:18:26.482 INFO:tasks.workunit.client.0.smithi107.stdout:1/220: dwrite d2/da/d40/f49 [0,4194304] 0 2024-04-25T00:18:26.482 INFO:tasks.workunit.client.0.smithi107.stdout:9/185: dwrite d0/d8/de/d20/f3 [0,4194304] 0 2024-04-25T00:18:26.482 INFO:tasks.workunit.client.0.smithi107.stdout:9/186: write d0/d8/de/d20/f4 [95865,79867] 0 2024-04-25T00:18:26.482 INFO:tasks.workunit.client.0.smithi107.stdout:9/187: write d0/d8/d32/f38 [937781,53459] 0 2024-04-25T00:18:26.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:26 smithi107 ceph-mon[29153]: pgmap v62: 129 pgs: 129 active+clean; 333 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 37 MiB/s wr, 442 op/s 2024-04-25T00:18:26.485 INFO:tasks.workunit.client.0.smithi107.stdout:1/221: link d2/da/f19 d2/d2a/f4a 0 2024-04-25T00:18:26.485 INFO:tasks.workunit.client.0.smithi107.stdout:0/148: dwrite dc/f1f [0,4194304] 0 2024-04-25T00:18:26.486 INFO:tasks.workunit.client.0.smithi107.stdout:3/175: dread d9/f29 [0,4194304] 0 2024-04-25T00:18:26.487 INFO:tasks.workunit.client.0.smithi107.stdout:3/176: creat d9/d15/d1a/d2c/f37 x:0 0 0 2024-04-25T00:18:26.487 INFO:tasks.workunit.client.0.smithi107.stdout:3/177: dread - d9/d15/d1a/f32 zero size 2024-04-25T00:18:26.491 INFO:tasks.workunit.client.0.smithi107.stdout:9/188: rename d0/d8/de/cf to d0/d8/d1f/c3e 0 2024-04-25T00:18:26.493 INFO:tasks.workunit.client.0.smithi107.stdout:4/161: dwrite d3/d5/d32/f38 [0,4194304] 0 2024-04-25T00:18:26.493 INFO:tasks.workunit.client.0.smithi107.stdout:4/162: write d3/f35 [77496,98523] 0 2024-04-25T00:18:26.494 INFO:tasks.workunit.client.0.smithi107.stdout:1/222: rename d2/da/d40/f49 to d2/dd/f4b 0 2024-04-25T00:18:26.496 INFO:tasks.workunit.client.0.smithi107.stdout:9/189: creat d0/d8/d34/f3f x:0 0 0 2024-04-25T00:18:26.500 INFO:tasks.workunit.client.0.smithi107.stdout:2/249: dwrite d6/d18/d40/d42/d2b/f41 [0,4194304] 0 2024-04-25T00:18:26.501 INFO:tasks.workunit.client.0.smithi107.stdout:4/163: dread d3/f35 [0,4194304] 0 2024-04-25T00:18:26.502 INFO:tasks.workunit.client.0.smithi107.stdout:9/190: write d0/d8/de/d20/f10 [891742,100235] 0 2024-04-25T00:18:26.502 INFO:tasks.workunit.client.0.smithi107.stdout:9/191: readlink d0/d8/de/l31 0 2024-04-25T00:18:26.503 INFO:tasks.workunit.client.0.smithi107.stdout:9/192: dread - d0/d1b/f2e zero size 2024-04-25T00:18:26.505 INFO:tasks.workunit.client.0.smithi107.stdout:2/250: unlink d6/d18/d40/d42/d2b/d30/d4c/l4f 0 2024-04-25T00:18:26.507 INFO:tasks.workunit.client.0.smithi107.stdout:4/164: truncate d3/d5/d32/f38 2556410 0 2024-04-25T00:18:26.508 INFO:tasks.workunit.client.0.smithi107.stdout:4/165: stat d3/d24/d25/c29 0 2024-04-25T00:18:26.508 INFO:tasks.workunit.client.0.smithi107.stdout:4/166: dread - d3/d24/f3b zero size 2024-04-25T00:18:26.508 INFO:tasks.workunit.client.0.smithi107.stdout:3/178: dwrite d9/ff [4194304,4194304] 0 2024-04-25T00:18:26.508 INFO:tasks.workunit.client.0.smithi107.stdout:3/179: read d9/f11 [3842421,29993] 0 2024-04-25T00:18:26.508 INFO:tasks.workunit.client.0.smithi107.stdout:3/180: write d9/d22/f2e [932217,40825] 0 2024-04-25T00:18:26.508 INFO:tasks.workunit.client.0.smithi107.stdout:3/181: write d9/d15/d1a/f1c [752502,21938] 0 2024-04-25T00:18:26.509 INFO:tasks.workunit.client.0.smithi107.stdout:2/251: rename d6/d20/d22/f1d to d6/d18/d40/d42/f57 0 2024-04-25T00:18:26.509 INFO:tasks.workunit.client.0.smithi107.stdout:2/252: fsync d6/d18/d40/d42/f3f 0 2024-04-25T00:18:26.509 INFO:tasks.workunit.client.0.smithi107.stdout:2/253: chown d6/d18/d40/d42/f47 46963 1 2024-04-25T00:18:26.511 INFO:tasks.workunit.client.0.smithi107.stdout:0/149: dwrite dc/f1f [0,4194304] 0 2024-04-25T00:18:26.511 INFO:tasks.workunit.client.0.smithi107.stdout:4/167: symlink d3/d5/l3c 0 2024-04-25T00:18:26.513 INFO:tasks.workunit.client.0.smithi107.stdout:4/168: creat d3/d5/f3d x:0 0 0 2024-04-25T00:18:26.513 INFO:tasks.workunit.client.0.smithi107.stdout:4/169: dread - d3/d24/d25/f30 zero size 2024-04-25T00:18:26.514 INFO:tasks.workunit.client.0.smithi107.stdout:4/170: creat d3/d24/f3e x:0 0 0 2024-04-25T00:18:26.515 INFO:tasks.workunit.client.0.smithi107.stdout:3/182: dread f0 [0,4194304] 0 2024-04-25T00:18:26.515 INFO:tasks.workunit.client.0.smithi107.stdout:3/183: stat d9/d22/d1b/l27 0 2024-04-25T00:18:26.515 INFO:tasks.workunit.client.0.smithi107.stdout:3/184: write d9/d15/d1a/f1c [1767562,79726] 0 2024-04-25T00:18:26.516 INFO:tasks.workunit.client.0.smithi107.stdout:4/171: symlink d3/d5/l3f 0 2024-04-25T00:18:26.518 INFO:tasks.workunit.client.0.smithi107.stdout:4/172: mknod d3/d24/c40 0 2024-04-25T00:18:26.519 INFO:tasks.workunit.client.0.smithi107.stdout:3/185: mknod d9/d22/d1b/c38 0 2024-04-25T00:18:26.519 INFO:tasks.workunit.client.0.smithi107.stdout:4/173: getdents d3/d5/d32 0 2024-04-25T00:18:26.520 INFO:tasks.workunit.client.0.smithi107.stdout:0/150: dread dc/f28 [0,4194304] 0 2024-04-25T00:18:26.524 INFO:tasks.workunit.client.0.smithi107.stdout:4/174: rename d3/d5/fc to d3/d5/f41 0 2024-04-25T00:18:26.524 INFO:tasks.workunit.client.0.smithi107.stdout:0/151: creat dc/f2f x:0 0 0 2024-04-25T00:18:26.524 INFO:tasks.workunit.client.0.smithi107.stdout:3/186: read f0 [556106,81894] 0 2024-04-25T00:18:26.524 INFO:tasks.workunit.client.0.smithi107.stdout:1/223: dwrite d2/dd/f10 [0,4194304] 0 2024-04-25T00:18:26.524 INFO:tasks.workunit.client.0.smithi107.stdout:3/187: write d9/d15/d1a/d2c/f37 [318418,33700] 0 2024-04-25T00:18:26.524 INFO:tasks.workunit.client.0.smithi107.stdout:2/254: dread d6/fe [0,4194304] 0 2024-04-25T00:18:26.524 INFO:tasks.workunit.client.0.smithi107.stdout:3/188: chown l3 891 1 2024-04-25T00:18:26.525 INFO:tasks.workunit.client.0.smithi107.stdout:0/152: getdents dc/d24/d25 0 2024-04-25T00:18:26.525 INFO:tasks.workunit.client.0.smithi107.stdout:4/175: symlink d3/d5/l42 0 2024-04-25T00:18:26.528 INFO:tasks.workunit.client.0.smithi107.stdout:2/255: rename d6/d18/d40/d42/d2b/l4a to d6/d20/d22/d2c/l58 0 2024-04-25T00:18:26.528 INFO:tasks.workunit.client.0.smithi107.stdout:4/176: creat d3/d24/d25/f43 x:0 0 0 2024-04-25T00:18:26.529 INFO:tasks.workunit.client.0.smithi107.stdout:4/177: mkdir d3/d24/d25/d44 0 2024-04-25T00:18:26.531 INFO:tasks.workunit.client.0.smithi107.stdout:4/178: dread d3/f35 [0,4194304] 0 2024-04-25T00:18:26.532 INFO:tasks.workunit.client.0.smithi107.stdout:9/193: dwrite d0/d8/de/d20/f10 [0,4194304] 0 2024-04-25T00:18:26.534 INFO:tasks.workunit.client.0.smithi107.stdout:4/179: creat d3/d5/d32/f45 x:0 0 0 2024-04-25T00:18:26.534 INFO:tasks.workunit.client.0.smithi107.stdout:4/180: dread - d3/f21 zero size 2024-04-25T00:18:26.534 INFO:tasks.workunit.client.0.smithi107.stdout:9/194: link d0/d8/d32/f38 d0/f40 0 2024-04-25T00:18:26.535 INFO:tasks.workunit.client.0.smithi107.stdout:4/181: symlink d3/d5/d32/l46 0 2024-04-25T00:18:26.535 INFO:tasks.workunit.client.0.smithi107.stdout:3/189: dread d9/f11 [0,4194304] 0 2024-04-25T00:18:26.545 INFO:tasks.workunit.client.0.smithi107.stdout:2/256: write d6/f8 [3987004,2280] 0 2024-04-25T00:18:26.547 INFO:tasks.workunit.client.0.smithi107.stdout:4/182: write d3/d5/d32/f38 [115516,32758] 0 2024-04-25T00:18:26.548 INFO:tasks.workunit.client.0.smithi107.stdout:1/224: dwrite d2/dd/f10 [0,4194304] 0 2024-04-25T00:18:26.548 INFO:tasks.workunit.client.0.smithi107.stdout:1/225: fdatasync d2/dd/d25/f33 0 2024-04-25T00:18:26.548 INFO:tasks.workunit.client.0.smithi107.stdout:1/226: truncate d2/dd/d1a/f1d 1199721 0 2024-04-25T00:18:26.548 INFO:tasks.workunit.client.0.smithi107.stdout:1/227: readlink d2/dd/d25/l2c 0 2024-04-25T00:18:26.548 INFO:tasks.workunit.client.0.smithi107.stdout:1/228: write d2/f30 [1108762,94037] 0 2024-04-25T00:18:26.549 INFO:tasks.workunit.client.0.smithi107.stdout:0/153: dwrite dc/f18 [0,4194304] 0 2024-04-25T00:18:26.549 INFO:tasks.workunit.client.0.smithi107.stdout:4/183: symlink d3/d5/d32/l47 0 2024-04-25T00:18:26.549 INFO:tasks.workunit.client.0.smithi107.stdout:4/184: fsync d3/d24/f3b 0 2024-04-25T00:18:26.549 INFO:tasks.workunit.client.0.smithi107.stdout:4/185: stat d3/d5/f12 0 2024-04-25T00:18:26.554 INFO:tasks.workunit.client.0.smithi107.stdout:3/190: dwrite d9/f10 [0,4194304] 0 2024-04-25T00:18:26.555 INFO:tasks.workunit.client.0.smithi107.stdout:9/195: dread d0/d8/de/d20/fd [0,4194304] 0 2024-04-25T00:18:26.555 INFO:tasks.workunit.client.0.smithi107.stdout:9/196: dread - d0/d1b/f2e zero size 2024-04-25T00:18:26.558 INFO:tasks.workunit.client.0.smithi107.stdout:2/257: dread d6/f1e [0,4194304] 0 2024-04-25T00:18:26.561 INFO:tasks.workunit.client.0.smithi107.stdout:3/191: mkdir d9/d15/d39 0 2024-04-25T00:18:26.562 INFO:tasks.workunit.client.0.smithi107.stdout:9/197: rmdir d0/d8 39 2024-04-25T00:18:26.563 INFO:tasks.workunit.client.0.smithi107.stdout:2/258: creat d6/d18/d4d/f59 x:0 0 0 2024-04-25T00:18:26.564 INFO:tasks.workunit.client.0.smithi107.stdout:2/259: write d6/d18/d40/d42/f3b [1198794,6274] 0 2024-04-25T00:18:26.564 INFO:tasks.workunit.client.0.smithi107.stdout:3/192: mkdir d9/d15/d1a/d3a 0 2024-04-25T00:18:26.565 INFO:tasks.workunit.client.0.smithi107.stdout:9/198: rmdir d0/d8/d32 39 2024-04-25T00:18:26.565 INFO:tasks.workunit.client.0.smithi107.stdout:9/199: chown d0/d8/de/f33 225 1 2024-04-25T00:18:26.566 INFO:tasks.workunit.client.0.smithi107.stdout:2/260: creat d6/d20/f5a x:0 0 0 2024-04-25T00:18:26.567 INFO:tasks.workunit.client.0.smithi107.stdout:1/229: dwrite d2/dd/d25/f33 [0,4194304] 0 2024-04-25T00:18:26.567 INFO:tasks.workunit.client.0.smithi107.stdout:1/230: dread - d2/da/f43 zero size 2024-04-25T00:18:26.567 INFO:tasks.workunit.client.0.smithi107.stdout:9/200: symlink d0/d8/d1f/l41 0 2024-04-25T00:18:26.570 INFO:tasks.workunit.client.0.smithi107.stdout:2/261: rename d6/d20/d22/f26 to d6/d18/d40/d42/d2b/f5b 0 2024-04-25T00:18:26.571 INFO:tasks.workunit.client.0.smithi107.stdout:0/154: dwrite dc/d24/f2d [0,4194304] 0 2024-04-25T00:18:26.572 INFO:tasks.workunit.client.0.smithi107.stdout:1/231: rmdir d2/dd/d1b 39 2024-04-25T00:18:26.572 INFO:tasks.workunit.client.0.smithi107.stdout:1/232: write d2/da/f43 [109131,103274] 0 2024-04-25T00:18:26.573 INFO:tasks.workunit.client.0.smithi107.stdout:9/201: link d0/d8/f26 d0/d8/d34/f42 0 2024-04-25T00:18:26.575 INFO:tasks.workunit.client.0.smithi107.stdout:0/155: mknod dc/d24/c30 0 2024-04-25T00:18:26.577 INFO:tasks.workunit.client.0.smithi107.stdout:1/233: truncate d2/da/f19 1984133 0 2024-04-25T00:18:26.577 INFO:tasks.workunit.client.0.smithi107.stdout:9/202: getdents d0 0 2024-04-25T00:18:26.581 INFO:tasks.workunit.client.0.smithi107.stdout:4/186: dwrite d3/d5/d32/f38 [0,4194304] 0 2024-04-25T00:18:26.581 INFO:tasks.workunit.client.0.smithi107.stdout:4/187: dread - d3/d24/f2c zero size 2024-04-25T00:18:26.582 INFO:tasks.workunit.client.0.smithi107.stdout:4/188: stat d3/d5/d32/l37 0 2024-04-25T00:18:26.582 INFO:tasks.workunit.client.0.smithi107.stdout:2/262: fsync d6/d18/d40/d42/d2b/f5b 0 2024-04-25T00:18:26.582 INFO:tasks.workunit.client.0.smithi107.stdout:2/263: stat d6/d18/d40 0 2024-04-25T00:18:26.582 INFO:tasks.workunit.client.0.smithi107.stdout:2/264: write d6/f43 [6975253,98853] 0 2024-04-25T00:18:26.582 INFO:tasks.workunit.client.0.smithi107.stdout:4/189: symlink d3/d24/d25/l48 0 2024-04-25T00:18:26.584 INFO:tasks.workunit.client.0.smithi107.stdout:2/265: truncate d6/d18/d40/d42/d2b/f5b 7039715 0 2024-04-25T00:18:26.584 INFO:tasks.workunit.client.0.smithi107.stdout:2/266: read d6/ff [5394040,37610] 0 2024-04-25T00:18:26.585 INFO:tasks.workunit.client.0.smithi107.stdout:2/267: symlink d6/d20/d22/l5c 0 2024-04-25T00:18:26.585 INFO:tasks.workunit.client.0.smithi107.stdout:3/193: dwrite d9/f20 [0,4194304] 0 2024-04-25T00:18:26.586 INFO:tasks.workunit.client.0.smithi107.stdout:4/190: read d3/fd [249949,45009] 0 2024-04-25T00:18:26.586 INFO:tasks.workunit.client.0.smithi107.stdout:4/191: readlink d3/d5/d32/l3a 0 2024-04-25T00:18:26.589 INFO:tasks.workunit.client.0.smithi107.stdout:2/268: rename d6/f43 to d6/d18/f5d 0 2024-04-25T00:18:26.589 INFO:tasks.workunit.client.0.smithi107.stdout:2/269: write d6/f3d [1052466,22987] 0 2024-04-25T00:18:26.593 INFO:tasks.workunit.client.0.smithi107.stdout:4/192: unlink d3/d24/f3b 0 2024-04-25T00:18:26.596 INFO:tasks.workunit.client.0.smithi107.stdout:2/270: rename d6/d18/d4d/f59 to d6/d18/d40/d42/f5e 0 2024-04-25T00:18:26.596 INFO:tasks.workunit.client.0.smithi107.stdout:4/193: creat d3/d24/d25/f49 x:0 0 0 2024-04-25T00:18:26.597 INFO:tasks.workunit.client.0.smithi107.stdout:4/194: mknod d3/d5/c4a 0 2024-04-25T00:18:26.597 INFO:tasks.workunit.client.0.smithi107.stdout:4/195: read - d3/d24/f3e zero size 2024-04-25T00:18:26.598 INFO:tasks.workunit.client.0.smithi107.stdout:4/196: chown d3/f2a 1 1 2024-04-25T00:18:26.603 INFO:tasks.workunit.client.0.smithi107.stdout:4/197: dread d3/d5/f7 [0,4194304] 0 2024-04-25T00:18:26.603 INFO:tasks.workunit.client.0.smithi107.stdout:2/271: creat d6/d18/d40/d42/d2b/d30/d45/f5f x:0 0 0 2024-04-25T00:18:26.604 INFO:tasks.workunit.client.0.smithi107.stdout:4/198: rmdir d3/d24/d25/d44 0 2024-04-25T00:18:26.605 INFO:tasks.workunit.client.0.smithi107.stdout:9/203: dwrite d0/d8/de/f21 [0,4194304] 0 2024-04-25T00:18:26.605 INFO:tasks.workunit.client.0.smithi107.stdout:9/204: fsync d0/d8/de/d20/f7 0 2024-04-25T00:18:26.607 INFO:tasks.workunit.client.0.smithi107.stdout:4/199: rename d3/d10/c39 to d3/c4b 0 2024-04-25T00:18:26.607 INFO:tasks.workunit.client.0.smithi107.stdout:4/200: dread - d3/d24/f3e zero size 2024-04-25T00:18:26.607 INFO:tasks.workunit.client.0.smithi107.stdout:0/156: dread dc/dd/f14 [0,4194304] 0 2024-04-25T00:18:26.607 INFO:tasks.workunit.client.0.smithi107.stdout:1/234: dwrite d2/dd/d1a/f27 [0,4194304] 0 2024-04-25T00:18:26.607 INFO:tasks.workunit.client.0.smithi107.stdout:4/201: write d3/d24/d25/f26 [863276,114545] 0 2024-04-25T00:18:26.607 INFO:tasks.workunit.client.0.smithi107.stdout:2/272: dread d6/f17 [0,4194304] 0 2024-04-25T00:18:26.608 INFO:tasks.workunit.client.0.smithi107.stdout:0/157: mkdir dc/d24/d25/d31 0 2024-04-25T00:18:26.609 INFO:tasks.workunit.client.0.smithi107.stdout:0/158: read dc/dd/d16/d1a/f1e [140568,15630] 0 2024-04-25T00:18:26.609 INFO:tasks.workunit.client.0.smithi107.stdout:0/159: write dc/d24/d25/f2b [726737,55148] 0 2024-04-25T00:18:26.612 INFO:tasks.workunit.client.0.smithi107.stdout:2/273: symlink d6/d18/d40/d42/d2b/d30/l60 0 2024-04-25T00:18:26.614 INFO:tasks.workunit.client.0.smithi107.stdout:1/235: truncate d2/dd/d1a/f27 4188556 0 2024-04-25T00:18:26.616 INFO:tasks.workunit.client.0.smithi107.stdout:1/236: symlink d2/d47/l4c 0 2024-04-25T00:18:26.618 INFO:tasks.workunit.client.0.smithi107.stdout:0/160: dread dc/dd/d16/f1b [0,4194304] 0 2024-04-25T00:18:26.620 INFO:tasks.workunit.client.0.smithi107.stdout:0/161: symlink dc/d24/d25/l32 0 2024-04-25T00:18:26.622 INFO:tasks.workunit.client.0.smithi107.stdout:0/162: mkdir dc/dd/d16/d33 0 2024-04-25T00:18:26.629 INFO:tasks.workunit.client.0.smithi107.stdout:1/237: dread d2/fc [0,4194304] 0 2024-04-25T00:18:26.631 INFO:tasks.workunit.client.0.smithi107.stdout:1/238: unlink d2/f13 0 2024-04-25T00:18:26.637 INFO:tasks.workunit.client.0.smithi107.stdout:1/239: dread d2/dd/f21 [0,4194304] 0 2024-04-25T00:18:26.637 INFO:tasks.workunit.client.0.smithi107.stdout:1/240: readlink d2/d47/l4c 0 2024-04-25T00:18:26.638 INFO:tasks.workunit.client.0.smithi107.stdout:1/241: mknod d2/da/c4d 0 2024-04-25T00:18:26.640 INFO:tasks.workunit.client.0.smithi107.stdout:1/242: creat d2/dd/f4e x:0 0 0 2024-04-25T00:18:26.640 INFO:tasks.workunit.client.0.smithi107.stdout:0/163: dwrite dc/f18 [0,4194304] 0 2024-04-25T00:18:26.641 INFO:tasks.workunit.client.0.smithi107.stdout:1/243: unlink d2/dd/f10 0 2024-04-25T00:18:26.641 INFO:tasks.workunit.client.0.smithi107.stdout:1/244: rename d2/dd to d2/dd/d4f 22 2024-04-25T00:18:26.641 INFO:tasks.workunit.client.0.smithi107.stdout:1/245: stat d2/d2a/f38 0 2024-04-25T00:18:26.642 INFO:tasks.workunit.client.0.smithi107.stdout:0/164: dread dc/dd/d16/f22 [0,4194304] 0 2024-04-25T00:18:26.643 INFO:tasks.workunit.client.0.smithi107.stdout:1/246: creat d2/dd/d25/f50 x:0 0 0 2024-04-25T00:18:26.643 INFO:tasks.workunit.client.0.smithi107.stdout:0/165: rename dc/c1c to dc/c34 0 2024-04-25T00:18:26.644 INFO:tasks.workunit.client.0.smithi107.stdout:1/247: creat d2/f51 x:0 0 0 2024-04-25T00:18:26.645 INFO:tasks.workunit.client.0.smithi107.stdout:1/248: getdents d2/dd/d1a 0 2024-04-25T00:18:26.652 INFO:tasks.workunit.client.0.smithi107.stdout:0/166: write dc/f18 [4165773,58280] 0 2024-04-25T00:18:26.652 INFO:tasks.workunit.client.0.smithi107.stdout:0/167: write dc/dd/d16/f2a [259167,21050] 0 2024-04-25T00:18:26.652 INFO:tasks.workunit.client.0.smithi107.stdout:0/168: write dc/dd/d16/f2a [995720,21421] 0 2024-04-25T00:18:26.653 INFO:tasks.workunit.client.0.smithi107.stdout:0/169: mkdir dc/dd/d16/d33/d35 0 2024-04-25T00:18:26.653 INFO:tasks.workunit.client.0.smithi107.stdout:0/170: chown dc/dd/l27 463453543 1 2024-04-25T00:18:26.653 INFO:tasks.workunit.client.0.smithi107.stdout:0/171: rmdir dc/d24 39 2024-04-25T00:18:26.760 INFO:tasks.workunit.client.0.smithi107.stdout:4/202: dread d3/d10/f1c [0,4194304] 0 2024-04-25T00:18:26.760 INFO:tasks.workunit.client.0.smithi107.stdout:4/203: chown d3/d24/d25/f49 92993455 1 2024-04-25T00:18:26.761 INFO:tasks.workunit.client.0.smithi107.stdout:4/204: chown d3/d5/ca 839853 1 2024-04-25T00:18:26.761 INFO:tasks.workunit.client.0.smithi107.stdout:4/205: symlink d3/d5/l4c 0 2024-04-25T00:18:26.764 INFO:tasks.workunit.client.0.smithi107.stdout:4/206: dread d3/d5/f12 [0,4194304] 0 2024-04-25T00:18:26.765 INFO:tasks.workunit.client.0.smithi107.stdout:4/207: creat d3/d5/f4d x:0 0 0 2024-04-25T00:18:26.767 INFO:tasks.workunit.client.0.smithi107.stdout:4/208: rename d3/d5/l42 to d3/d5/d32/l4e 0 2024-04-25T00:18:26.767 INFO:tasks.workunit.client.0.smithi107.stdout:4/209: readlink d3/d5/d32/l3a 0 2024-04-25T00:18:26.768 INFO:tasks.workunit.client.0.smithi107.stdout:4/210: mknod d3/d10/c4f 0 2024-04-25T00:18:26.769 INFO:tasks.workunit.client.0.smithi107.stdout:4/211: symlink d3/d10/l50 0 2024-04-25T00:18:26.770 INFO:tasks.workunit.client.0.smithi107.stdout:7/162: sync 2024-04-25T00:18:26.770 INFO:tasks.workunit.client.0.smithi107.stdout:7/163: write d0/f1d [5226064,95339] 0 2024-04-25T00:18:26.771 INFO:tasks.workunit.client.0.smithi107.stdout:7/164: mknod d0/d8/db/d18/d19/c37 0 2024-04-25T00:18:26.772 INFO:tasks.workunit.client.0.smithi107.stdout:7/165: mknod d0/c38 0 2024-04-25T00:18:26.774 INFO:tasks.workunit.client.0.smithi107.stdout:7/166: mkdir d0/d8/db/d18/d19/d39 0 2024-04-25T00:18:26.774 INFO:tasks.workunit.client.0.smithi107.stdout:7/167: mknod d0/d8/db/d18/d19/d39/c3a 0 2024-04-25T00:18:26.776 INFO:tasks.workunit.client.0.smithi107.stdout:7/168: creat d0/f3b x:0 0 0 2024-04-25T00:18:26.776 INFO:tasks.workunit.client.0.smithi107.stdout:7/169: chown d0/d8/d2c/d2e/f30 0 1 2024-04-25T00:18:26.776 INFO:tasks.workunit.client.0.smithi107.stdout:7/170: dread - d0/d8/db/d18/d19/f23 zero size 2024-04-25T00:18:26.776 INFO:tasks.workunit.client.0.smithi107.stdout:7/171: read - d0/d8/db/d18/d19/f2f zero size 2024-04-25T00:18:26.776 INFO:tasks.workunit.client.0.smithi107.stdout:7/172: creat d0/d8/db/d18/d19/f3c x:0 0 0 2024-04-25T00:18:26.778 INFO:tasks.workunit.client.0.smithi107.stdout:7/173: rename d0/d21/f24 to d0/d8/d2c/f3d 0 2024-04-25T00:18:26.785 INFO:tasks.workunit.client.0.smithi107.stdout:4/212: dwrite d3/d10/f1c [4194304,4194304] 0 2024-04-25T00:18:26.785 INFO:tasks.workunit.client.0.smithi107.stdout:4/213: chown d3/d24/d25/c2f 228805819 1 2024-04-25T00:18:26.785 INFO:tasks.workunit.client.0.smithi107.stdout:4/214: readlink d3/d24/d25/l48 0 2024-04-25T00:18:26.786 INFO:tasks.workunit.client.0.smithi107.stdout:4/215: mknod d3/d10/c51 0 2024-04-25T00:18:26.790 INFO:tasks.workunit.client.0.smithi107.stdout:7/174: dread d0/d8/fd [0,4194304] 0 2024-04-25T00:18:26.791 INFO:tasks.workunit.client.0.smithi107.stdout:7/175: creat d0/d8/db/d18/d19/d39/f3e x:0 0 0 2024-04-25T00:18:26.792 INFO:tasks.workunit.client.0.smithi107.stdout:7/176: dread - d0/d8/db/d18/d19/d39/f3e zero size 2024-04-25T00:18:26.804 INFO:tasks.workunit.client.0.smithi107.stdout:4/216: dwrite d3/f18 [0,4194304] 0 2024-04-25T00:18:26.804 INFO:tasks.workunit.client.0.smithi107.stdout:4/217: write d3/d24/d25/f28 [862441,56891] 0 2024-04-25T00:18:26.805 INFO:tasks.workunit.client.0.smithi107.stdout:4/218: dread d3/d5/f23 [0,4194304] 0 2024-04-25T00:18:26.805 INFO:tasks.workunit.client.0.smithi107.stdout:4/219: write d3/f35 [949560,50379] 0 2024-04-25T00:18:26.822 INFO:tasks.workunit.client.0.smithi107.stdout:4/220: dwrite d3/d5/d32/f34 [0,4194304] 0 2024-04-25T00:18:26.824 INFO:tasks.workunit.client.0.smithi107.stdout:4/221: symlink d3/d10/l52 0 2024-04-25T00:18:26.825 INFO:tasks.workunit.client.0.smithi107.stdout:4/222: mkdir d3/d5/d32/d53 0 2024-04-25T00:18:26.825 INFO:tasks.workunit.client.0.smithi107.stdout:4/223: read d3/f35 [111285,7755] 0 2024-04-25T00:18:26.826 INFO:tasks.workunit.client.0.smithi107.stdout:4/224: creat d3/d5/d32/f54 x:0 0 0 2024-04-25T00:18:26.828 INFO:tasks.workunit.client.0.smithi107.stdout:4/225: dread d3/d5/f7 [0,4194304] 0 2024-04-25T00:18:26.829 INFO:tasks.workunit.client.0.smithi107.stdout:4/226: chown d3/d24/c40 83905 1 2024-04-25T00:18:26.829 INFO:tasks.workunit.client.0.smithi107.stdout:4/227: mknod d3/d24/d25/c55 0 2024-04-25T00:18:26.830 INFO:tasks.workunit.client.0.smithi107.stdout:4/228: truncate d3/d24/d25/f43 371987 0 2024-04-25T00:18:26.830 INFO:tasks.workunit.client.0.smithi107.stdout:4/229: write d3/d24/f3e [949253,77834] 0 2024-04-25T00:18:26.832 INFO:tasks.workunit.client.0.smithi107.stdout:5/309: sync 2024-04-25T00:18:26.833 INFO:tasks.workunit.client.0.smithi107.stdout:8/340: sync 2024-04-25T00:18:26.833 INFO:tasks.workunit.client.0.smithi107.stdout:5/310: write d4/dc/f11 [4082975,23657] 0 2024-04-25T00:18:26.834 INFO:tasks.workunit.client.0.smithi107.stdout:8/341: mknod d2/d18/d1a/d5e/d3d/c77 0 2024-04-25T00:18:26.845 INFO:tasks.workunit.client.0.smithi107.stdout:5/311: dread d4/fa [0,4194304] 0 2024-04-25T00:18:26.848 INFO:tasks.workunit.client.0.smithi107.stdout:4/230: dwrite d3/d10/f2d [0,4194304] 0 2024-04-25T00:18:26.848 INFO:tasks.workunit.client.0.smithi107.stdout:4/231: write d3/d24/f3e [1775420,29227] 0 2024-04-25T00:18:26.851 INFO:tasks.workunit.client.0.smithi107.stdout:4/232: mknod d3/d24/d25/c56 0 2024-04-25T00:18:26.853 INFO:tasks.workunit.client.0.smithi107.stdout:8/342: dwrite d2/d18/d1a/f31 [0,4194304] 0 2024-04-25T00:18:26.863 INFO:tasks.workunit.client.0.smithi107.stdout:4/233: dread f0 [4194304,4194304] 0 2024-04-25T00:18:26.868 INFO:tasks.workunit.client.0.smithi107.stdout:5/312: dwrite d4/d1c/f28 [0,4194304] 0 2024-04-25T00:18:26.868 INFO:tasks.workunit.client.0.smithi107.stdout:5/313: write d4/dc/fd [2250907,72496] 0 2024-04-25T00:18:26.878 INFO:tasks.workunit.client.0.smithi107.stdout:5/314: dread d4/dc/fe [0,4194304] 0 2024-04-25T00:18:26.878 INFO:tasks.workunit.client.0.smithi107.stdout:5/315: chown d4/dc/d26/c46 73 1 2024-04-25T00:18:26.878 INFO:tasks.workunit.client.0.smithi107.stdout:4/234: dwrite d3/d24/d25/f43 [0,4194304] 0 2024-04-25T00:18:26.880 INFO:tasks.workunit.client.0.smithi107.stdout:5/316: write d4/dc/f20 [802609,98161] 0 2024-04-25T00:18:26.880 INFO:tasks.workunit.client.0.smithi107.stdout:4/235: mknod d3/c57 0 2024-04-25T00:18:26.881 INFO:tasks.workunit.client.0.smithi107.stdout:5/317: mknod d4/dc/d26/d29/c6b 0 2024-04-25T00:18:26.907 INFO:tasks.workunit.client.0.smithi107.stdout:4/236: dwrite d3/d5/f3d [0,4194304] 0 2024-04-25T00:18:26.908 INFO:tasks.workunit.client.0.smithi107.stdout:5/318: dread d4/d1c/f28 [0,4194304] 0 2024-04-25T00:18:26.908 INFO:tasks.workunit.client.0.smithi107.stdout:4/237: fdatasync d3/d5/f22 0 2024-04-25T00:18:26.908 INFO:tasks.workunit.client.0.smithi107.stdout:4/238: chown d3/d5/lf 23 1 2024-04-25T00:18:26.908 INFO:tasks.workunit.client.0.smithi107.stdout:5/319: write d4/fb [720073,61199] 0 2024-04-25T00:18:26.908 INFO:tasks.workunit.client.0.smithi107.stdout:5/320: chown d4/dc/d54/f68 24 1 2024-04-25T00:18:26.908 INFO:tasks.workunit.client.0.smithi107.stdout:5/321: truncate d4/f48 2262330 0 2024-04-25T00:18:26.910 INFO:tasks.workunit.client.0.smithi107.stdout:4/239: creat d3/f58 x:0 0 0 2024-04-25T00:18:26.910 INFO:tasks.workunit.client.0.smithi107.stdout:4/240: write d3/f58 [94910,54149] 0 2024-04-25T00:18:26.914 INFO:tasks.workunit.client.0.smithi107.stdout:4/241: unlink d3/d10/f2d 0 2024-04-25T00:18:26.914 INFO:tasks.workunit.client.0.smithi107.stdout:4/242: dread - d3/f21 zero size 2024-04-25T00:18:26.914 INFO:tasks.workunit.client.0.smithi107.stdout:4/243: write d3/d5/f7 [1478597,92784] 0 2024-04-25T00:18:26.916 INFO:tasks.workunit.client.0.smithi107.stdout:4/244: rmdir d3 39 2024-04-25T00:18:26.916 INFO:tasks.workunit.client.0.smithi107.stdout:4/245: dread - d3/d5/f22 zero size 2024-04-25T00:18:26.917 INFO:tasks.workunit.client.0.smithi107.stdout:4/246: write d3/d24/d25/f2b [874306,12214] 0 2024-04-25T00:18:26.930 INFO:tasks.workunit.client.0.smithi107.stdout:5/322: dwrite d4/dc/d26/d29/d51/f57 [4194304,4194304] 0 2024-04-25T00:18:26.930 INFO:tasks.workunit.client.0.smithi107.stdout:4/247: dread d3/d5/d32/f34 [0,4194304] 0 2024-04-25T00:18:26.945 INFO:tasks.workunit.client.0.smithi107.stdout:4/248: dwrite d3/d5/f41 [0,4194304] 0 2024-04-25T00:18:26.947 INFO:tasks.workunit.client.0.smithi107.stdout:4/249: link d3/d10/l52 d3/l59 0 2024-04-25T00:18:26.948 INFO:tasks.workunit.client.0.smithi107.stdout:4/250: creat d3/d5/f5a x:0 0 0 2024-04-25T00:18:26.949 INFO:tasks.workunit.client.0.smithi107.stdout:4/251: readlink d3/d5/d32/l4e 0 2024-04-25T00:18:26.950 INFO:tasks.workunit.client.0.smithi107.stdout:5/323: dwrite d4/dc/f2f [0,4194304] 0 2024-04-25T00:18:26.954 INFO:tasks.workunit.client.0.smithi107.stdout:6/215: sync 2024-04-25T00:18:26.973 INFO:tasks.workunit.client.0.smithi107.stdout:6/216: dwrite d1/d2/db/d32/d36/d3a/f47 [0,4194304] 0 2024-04-25T00:18:26.974 INFO:tasks.workunit.client.0.smithi107.stdout:4/252: dwrite d3/d5/f5a [0,4194304] 0 2024-04-25T00:18:26.974 INFO:tasks.workunit.client.0.smithi107.stdout:4/253: fdatasync d3/d5/f22 0 2024-04-25T00:18:26.974 INFO:tasks.workunit.client.0.smithi107.stdout:5/324: dwrite d4/dc/f14 [0,4194304] 0 2024-04-25T00:18:26.978 INFO:tasks.workunit.client.0.smithi107.stdout:4/254: truncate d3/d5/d32/f34 1111023 0 2024-04-25T00:18:26.980 INFO:tasks.workunit.client.0.smithi107.stdout:4/255: creat d3/d5/d32/d53/f5b x:0 0 0 2024-04-25T00:18:26.990 INFO:tasks.workunit.client.0.smithi107.stdout:4/256: dread d3/f18 [0,4194304] 0 2024-04-25T00:18:26.992 INFO:tasks.workunit.client.0.smithi107.stdout:4/257: mkdir d3/d24/d5c 0 2024-04-25T00:18:26.992 INFO:tasks.workunit.client.0.smithi107.stdout:4/258: write d3/d5/d32/f34 [1702199,111137] 0 2024-04-25T00:18:26.992 INFO:tasks.workunit.client.0.smithi107.stdout:4/259: dread - d3/d24/f2c zero size 2024-04-25T00:18:26.992 INFO:tasks.workunit.client.0.smithi107.stdout:4/260: dread - d3/d5/f4d zero size 2024-04-25T00:18:26.993 INFO:tasks.workunit.client.0.smithi107.stdout:4/261: write d3/d5/d32/d53/f5b [764719,94400] 0 2024-04-25T00:18:26.993 INFO:tasks.workunit.client.0.smithi107.stdout:4/262: chown d3 8862359 1 2024-04-25T00:18:26.994 INFO:tasks.workunit.client.0.smithi107.stdout:4/263: symlink d3/d5/d32/d53/l5d 0 2024-04-25T00:18:26.994 INFO:tasks.workunit.client.0.smithi107.stdout:4/264: write f0 [906250,69500] 0 2024-04-25T00:18:26.995 INFO:tasks.workunit.client.0.smithi107.stdout:5/325: dwrite d4/dc/f11 [0,4194304] 0 2024-04-25T00:18:26.995 INFO:tasks.workunit.client.0.smithi107.stdout:5/326: stat d4/dc/d26/d29/d2e/c3f 0 2024-04-25T00:18:26.995 INFO:tasks.workunit.client.0.smithi107.stdout:6/217: dwrite d1/f1f [0,4194304] 0 2024-04-25T00:18:26.995 INFO:tasks.workunit.client.0.smithi107.stdout:4/265: rmdir d3/d10 39 2024-04-25T00:18:26.995 INFO:tasks.workunit.client.0.smithi107.stdout:4/266: chown d3/d24/f2c 13 1 2024-04-25T00:18:26.996 INFO:tasks.workunit.client.0.smithi107.stdout:5/327: mknod d4/d17/c6c 0 2024-04-25T00:18:26.996 INFO:tasks.workunit.client.0.smithi107.stdout:6/218: creat d1/d2/db/d3c/f4e x:0 0 0 2024-04-25T00:18:26.998 INFO:tasks.workunit.client.0.smithi107.stdout:5/328: creat d4/dc/d26/d29/d2e/f6d x:0 0 0 2024-04-25T00:18:26.999 INFO:tasks.workunit.client.0.smithi107.stdout:6/219: mkdir d1/d2/db/d32/d33/d4f 0 2024-04-25T00:18:26.999 INFO:tasks.workunit.client.0.smithi107.stdout:5/329: symlink d4/d1c/l6e 0 2024-04-25T00:18:27.000 INFO:tasks.workunit.client.0.smithi107.stdout:4/267: unlink d3/d24/f3e 0 2024-04-25T00:18:27.000 INFO:tasks.workunit.client.0.smithi107.stdout:5/330: creat d4/d1c/d5c/f6f x:0 0 0 2024-04-25T00:18:27.001 INFO:tasks.workunit.client.0.smithi107.stdout:5/331: write d4/dc/d26/d29/d2e/f6d [198162,120815] 0 2024-04-25T00:18:27.001 INFO:tasks.workunit.client.0.smithi107.stdout:6/220: creat d1/d2/db/d3c/f50 x:0 0 0 2024-04-25T00:18:27.002 INFO:tasks.workunit.client.0.smithi107.stdout:5/332: mknod d4/dc/d26/d62/c70 0 2024-04-25T00:18:27.002 INFO:tasks.workunit.client.0.smithi107.stdout:4/268: mknod d3/d5/c5e 0 2024-04-25T00:18:27.002 INFO:tasks.workunit.client.0.smithi107.stdout:4/269: write d3/d5/d32/d53/f5b [1159840,67932] 0 2024-04-25T00:18:27.003 INFO:tasks.workunit.client.0.smithi107.stdout:6/221: creat d1/d2/d3/d26/f51 x:0 0 0 2024-04-25T00:18:27.003 INFO:tasks.workunit.client.0.smithi107.stdout:4/270: rmdir d3/d24/d25 39 2024-04-25T00:18:27.005 INFO:tasks.workunit.client.0.smithi107.stdout:6/222: mkdir d1/d2/d3/d28/d52 0 2024-04-25T00:18:27.006 INFO:tasks.workunit.client.0.smithi107.stdout:6/223: unlink d1/l2e 0 2024-04-25T00:18:27.007 INFO:tasks.workunit.client.0.smithi107.stdout:6/224: creat d1/d2/d3/d28/f53 x:0 0 0 2024-04-25T00:18:27.010 INFO:tasks.workunit.client.0.smithi107.stdout:5/333: dread d4/d1c/f28 [0,4194304] 0 2024-04-25T00:18:27.012 INFO:tasks.workunit.client.0.smithi107.stdout:5/334: rename d4/dc/f11 to d4/f71 0 2024-04-25T00:18:27.014 INFO:tasks.workunit.client.0.smithi107.stdout:5/335: rmdir d4 39 2024-04-25T00:18:27.019 INFO:tasks.workunit.client.0.smithi107.stdout:6/225: write d1/f1f [397032,113264] 0 2024-04-25T00:18:27.019 INFO:tasks.workunit.client.0.smithi107.stdout:6/226: write d1/d14/d18/f2c [4917074,535] 0 2024-04-25T00:18:27.026 INFO:tasks.workunit.client.0.smithi107.stdout:4/271: dwrite f0 [4194304,4194304] 0 2024-04-25T00:18:27.027 INFO:tasks.workunit.client.0.smithi107.stdout:4/272: mknod d3/d24/d5c/c5f 0 2024-04-25T00:18:27.030 INFO:tasks.workunit.client.0.smithi107.stdout:4/273: dread d3/d24/d25/f28 [0,4194304] 0 2024-04-25T00:18:27.030 INFO:tasks.workunit.client.0.smithi107.stdout:5/336: dwrite d4/dc/f20 [0,4194304] 0 2024-04-25T00:18:27.037 INFO:tasks.workunit.client.0.smithi107.stdout:4/274: mknod d3/d24/d5c/c60 0 2024-04-25T00:18:27.037 INFO:tasks.workunit.client.0.smithi107.stdout:4/275: stat d3/c1a 0 2024-04-25T00:18:27.039 INFO:tasks.workunit.client.0.smithi107.stdout:4/276: mkdir d3/d24/d61 0 2024-04-25T00:18:27.039 INFO:tasks.workunit.client.0.smithi107.stdout:5/337: dread d4/d17/f21 [0,4194304] 0 2024-04-25T00:18:27.040 INFO:tasks.workunit.client.0.smithi107.stdout:4/277: truncate d3/d5/f3d 1985836 0 2024-04-25T00:18:27.041 INFO:tasks.workunit.client.0.smithi107.stdout:5/338: unlink d4/l3b 0 2024-04-25T00:18:27.041 INFO:tasks.workunit.client.0.smithi107.stdout:5/339: chown d4/dc/l1d 0 1 2024-04-25T00:18:27.042 INFO:tasks.workunit.client.0.smithi107.stdout:5/340: symlink d4/dc/d54/l72 0 2024-04-25T00:18:27.044 INFO:tasks.workunit.client.0.smithi107.stdout:4/278: link d3/fd d3/f62 0 2024-04-25T00:18:27.046 INFO:tasks.workunit.client.0.smithi107.stdout:6/227: rename d1/d2/db/c1c to d1/d14/d18/c54 0 2024-04-25T00:18:27.048 INFO:tasks.workunit.client.0.smithi107.stdout:9/205: sync 2024-04-25T00:18:27.048 INFO:tasks.workunit.client.0.smithi107.stdout:7/177: sync 2024-04-25T00:18:27.049 INFO:tasks.workunit.client.0.smithi107.stdout:3/194: sync 2024-04-25T00:18:27.049 INFO:tasks.workunit.client.0.smithi107.stdout:7/178: write d0/f1c [266412,30925] 0 2024-04-25T00:18:27.049 INFO:tasks.workunit.client.0.smithi107.stdout:7/179: fsync d0/f11 0 2024-04-25T00:18:27.049 INFO:tasks.workunit.client.0.smithi107.stdout:1/249: sync 2024-04-25T00:18:27.049 INFO:tasks.workunit.client.0.smithi107.stdout:1/250: stat d2/fc 0 2024-04-25T00:18:27.049 INFO:tasks.workunit.client.0.smithi107.stdout:1/251: dread - d2/dd/d25/f50 zero size 2024-04-25T00:18:27.049 INFO:tasks.workunit.client.0.smithi107.stdout:1/252: getdents d2/d2a/d37 0 2024-04-25T00:18:27.049 INFO:tasks.workunit.client.0.smithi107.stdout:8/343: sync 2024-04-25T00:18:27.049 INFO:tasks.workunit.client.0.smithi107.stdout:1/253: read - d2/dd/d1b/f42 zero size 2024-04-25T00:18:27.049 INFO:tasks.workunit.client.0.smithi107.stdout:8/344: chown d2/d18/d1a/d5e/d3d/c4a 474620688 1 2024-04-25T00:18:27.049 INFO:tasks.workunit.client.0.smithi107.stdout:0/172: sync 2024-04-25T00:18:27.049 INFO:tasks.workunit.client.0.smithi107.stdout:2/274: sync 2024-04-25T00:18:27.050 INFO:tasks.workunit.client.0.smithi107.stdout:1/254: creat d2/dd/d1b/f52 x:0 0 0 2024-04-25T00:18:27.051 INFO:tasks.workunit.client.0.smithi107.stdout:8/345: getdents d2 0 2024-04-25T00:18:27.051 INFO:tasks.workunit.client.0.smithi107.stdout:6/228: symlink d1/d2/l55 0 2024-04-25T00:18:27.051 INFO:tasks.workunit.client.0.smithi107.stdout:7/180: mknod d0/d8/db/d18/d19/d39/c3f 0 2024-04-25T00:18:27.053 INFO:tasks.workunit.client.0.smithi107.stdout:2/275: rename d6/d18/d40/d42/f5e to d6/d18/d40/d42/d2b/d30/d45/f61 0 2024-04-25T00:18:27.054 INFO:tasks.workunit.client.0.smithi107.stdout:7/181: creat d0/d21/f40 x:0 0 0 2024-04-25T00:18:27.054 INFO:tasks.workunit.client.0.smithi107.stdout:2/276: readlink d6/l12 0 2024-04-25T00:18:27.054 INFO:tasks.workunit.client.0.smithi107.stdout:2/277: fdatasync d6/f17 0 2024-04-25T00:18:27.054 INFO:tasks.workunit.client.0.smithi107.stdout:2/278: truncate d6/f17 648640 0 2024-04-25T00:18:27.054 INFO:tasks.workunit.client.0.smithi107.stdout:2/279: fsync d6/d20/d22/f21 0 2024-04-25T00:18:27.055 INFO:tasks.workunit.client.0.smithi107.stdout:1/255: link d2/dd/d1a/f27 d2/da/f53 0 2024-04-25T00:18:27.055 INFO:tasks.workunit.client.0.smithi107.stdout:9/206: write d0/d8/d32/f38 [762744,107777] 0 2024-04-25T00:18:27.056 INFO:tasks.workunit.client.0.smithi107.stdout:8/346: rename d2/l69 to d2/d18/d1a/d33/l78 0 2024-04-25T00:18:27.061 INFO:tasks.workunit.client.0.smithi107.stdout:1/256: rename d2/d47 to d2/da/d54 0 2024-04-25T00:18:27.063 INFO:tasks.workunit.client.0.smithi107.stdout:7/182: write d0/d8/fd [2578464,43504] 0 2024-04-25T00:18:27.063 INFO:tasks.workunit.client.0.smithi107.stdout:7/183: chown d0 0 1 2024-04-25T00:18:27.063 INFO:tasks.workunit.client.0.smithi107.stdout:5/341: dwrite d4/dc/d26/d42/f5e [0,4194304] 0 2024-04-25T00:18:27.063 INFO:tasks.workunit.client.0.smithi107.stdout:5/342: write d4/dc/f63 [660003,32948] 0 2024-04-25T00:18:27.065 INFO:tasks.workunit.client.0.smithi107.stdout:7/184: creat d0/d8/db/d18/f41 x:0 0 0 2024-04-25T00:18:27.065 INFO:tasks.workunit.client.0.smithi107.stdout:7/185: dread - d0/d8/db/d18/d19/d2d/f31 zero size 2024-04-25T00:18:27.066 INFO:tasks.workunit.client.0.smithi107.stdout:5/343: rename d4/dc/c3e to d4/dc/d26/d62/c73 0 2024-04-25T00:18:27.066 INFO:tasks.workunit.client.0.smithi107.stdout:7/186: mkdir d0/d8/d2c/d2e/d42 0 2024-04-25T00:18:27.069 INFO:tasks.workunit.client.0.smithi107.stdout:7/187: unlink d0/d8/db/f10 0 2024-04-25T00:18:27.071 INFO:tasks.workunit.client.0.smithi107.stdout:7/188: symlink d0/d8/l43 0 2024-04-25T00:18:27.073 INFO:tasks.workunit.client.0.smithi107.stdout:1/257: dread d2/d2a/f38 [0,4194304] 0 2024-04-25T00:18:27.074 INFO:tasks.workunit.client.0.smithi107.stdout:4/279: dwrite d3/d10/f1c [0,4194304] 0 2024-04-25T00:18:27.087 INFO:tasks.workunit.client.0.smithi107.stdout:9/207: read d0/d8/de/d20/f10 [3680406,92335] 0 2024-04-25T00:18:27.088 INFO:tasks.workunit.client.0.smithi107.stdout:3/195: dwrite d9/d22/f2e [0,4194304] 0 2024-04-25T00:18:27.089 INFO:tasks.workunit.client.0.smithi107.stdout:0/173: dwrite f1 [0,4194304] 0 2024-04-25T00:18:27.092 INFO:tasks.workunit.client.0.smithi107.stdout:7/189: rename d0/d8/d2c/d2e to d0/d21/d44 0 2024-04-25T00:18:27.092 INFO:tasks.workunit.client.0.smithi107.stdout:3/196: unlink d9/d15/c33 0 2024-04-25T00:18:27.092 INFO:tasks.workunit.client.0.smithi107.stdout:3/197: readlink d9/d22/d1b/l2b 0 2024-04-25T00:18:27.092 INFO:tasks.workunit.client.0.smithi107.stdout:3/198: write d9/d15/d1a/f32 [877534,54520] 0 2024-04-25T00:18:27.093 INFO:tasks.workunit.client.0.smithi107.stdout:6/229: dwrite d1/d2/db/d32/d36/f3f [0,4194304] 0 2024-04-25T00:18:27.093 INFO:tasks.workunit.client.0.smithi107.stdout:6/230: write d1/d2/db/f34 [744667,98668] 0 2024-04-25T00:18:27.093 INFO:tasks.workunit.client.0.smithi107.stdout:6/231: readlink d1/d14/d18/l2d 0 2024-04-25T00:18:27.094 INFO:tasks.workunit.client.0.smithi107.stdout:7/190: mknod d0/d8/db/d18/d19/d39/c45 0 2024-04-25T00:18:27.094 INFO:tasks.workunit.client.0.smithi107.stdout:8/347: dwrite d2/d18/f20 [0,4194304] 0 2024-04-25T00:18:27.094 INFO:tasks.workunit.client.0.smithi107.stdout:0/174: creat dc/d24/d25/d31/f36 x:0 0 0 2024-04-25T00:18:27.097 INFO:tasks.workunit.client.0.smithi107.stdout:5/344: dwrite d4/d1c/f28 [0,4194304] 0 2024-04-25T00:18:27.098 INFO:tasks.workunit.client.0.smithi107.stdout:4/280: dwrite d3/f35 [0,4194304] 0 2024-04-25T00:18:27.098 INFO:tasks.workunit.client.0.smithi107.stdout:4/281: write d3/f21 [560986,24603] 0 2024-04-25T00:18:27.099 INFO:tasks.workunit.client.0.smithi107.stdout:9/208: rename d0/d8/de/d20 to d0/d43 0 2024-04-25T00:18:27.099 INFO:tasks.workunit.client.0.smithi107.stdout:1/258: fsync d2/dd/d1a/f27 0 2024-04-25T00:18:27.099 INFO:tasks.workunit.client.0.smithi107.stdout:1/259: write d2/dd/d25/f2e [1825860,82405] 0 2024-04-25T00:18:27.101 INFO:tasks.workunit.client.0.smithi107.stdout:0/175: getdents dc 0 2024-04-25T00:18:27.101 INFO:tasks.workunit.client.0.smithi107.stdout:7/191: rename d0/d21/d44 to d0/d21/d46 0 2024-04-25T00:18:27.104 INFO:tasks.workunit.client.0.smithi107.stdout:5/345: creat d4/d35/d61/f74 x:0 0 0 2024-04-25T00:18:27.104 INFO:tasks.workunit.client.0.smithi107.stdout:3/199: dread d9/ff [0,4194304] 0 2024-04-25T00:18:27.104 INFO:tasks.workunit.client.0.smithi107.stdout:4/282: stat d3/fd 0 2024-04-25T00:18:27.105 INFO:tasks.workunit.client.0.smithi107.stdout:8/348: rename d2/d18/d1a/d5e/c28 to d2/d18/d1a/d5e/d6d/c79 0 2024-04-25T00:18:27.105 INFO:tasks.workunit.client.0.smithi107.stdout:8/349: chown d2/d18/d1a/d5e/d3e/f6a 5152824 1 2024-04-25T00:18:27.107 INFO:tasks.workunit.client.0.smithi107.stdout:0/176: creat dc/dd/d16/d33/f37 x:0 0 0 2024-04-25T00:18:27.107 INFO:tasks.workunit.client.0.smithi107.stdout:0/177: chown dc/dd/d16/d33/f37 2217359 1 2024-04-25T00:18:27.107 INFO:tasks.workunit.client.0.smithi107.stdout:4/283: write d3/d24/d25/f28 [448387,118860] 0 2024-04-25T00:18:27.108 INFO:tasks.workunit.client.0.smithi107.stdout:5/346: creat d4/dc/d26/d29/d2e/d55/f75 x:0 0 0 2024-04-25T00:18:27.108 INFO:tasks.workunit.client.0.smithi107.stdout:4/284: symlink d3/d5/d32/d53/l63 0 2024-04-25T00:18:27.110 INFO:tasks.workunit.client.0.smithi107.stdout:9/209: rename d0/d1b/l3b to d0/d8/d1f/l44 0 2024-04-25T00:18:27.112 INFO:tasks.workunit.client.0.smithi107.stdout:0/178: mkdir dc/d24/d38 0 2024-04-25T00:18:27.113 INFO:tasks.workunit.client.0.smithi107.stdout:4/285: rename d3/c4b to d3/d10/c64 0 2024-04-25T00:18:27.113 INFO:tasks.workunit.client.0.smithi107.stdout:5/347: creat d4/dc/d26/d42/f76 x:0 0 0 2024-04-25T00:18:27.114 INFO:tasks.workunit.client.0.smithi107.stdout:8/350: dread d2/d18/f53 [0,4194304] 0 2024-04-25T00:18:27.114 INFO:tasks.workunit.client.0.smithi107.stdout:6/232: dwrite d1/d2/d3/d26/f51 [0,4194304] 0 2024-04-25T00:18:27.115 INFO:tasks.workunit.client.0.smithi107.stdout:4/286: creat d3/d5/f65 x:0 0 0 2024-04-25T00:18:27.115 INFO:tasks.workunit.client.0.smithi107.stdout:9/210: unlink d0/l18 0 2024-04-25T00:18:27.115 INFO:tasks.workunit.client.0.smithi107.stdout:9/211: readlink d0/d8/d1f/l41 0 2024-04-25T00:18:27.118 INFO:tasks.workunit.client.0.smithi107.stdout:5/348: truncate d4/dc/f13 288883 0 2024-04-25T00:18:27.118 INFO:tasks.workunit.client.0.smithi107.stdout:5/349: write d4/dc/f1f [1303666,13705] 0 2024-04-25T00:18:27.118 INFO:tasks.workunit.client.0.smithi107.stdout:6/233: truncate d1/d2/db/f19 3035685 0 2024-04-25T00:18:27.119 INFO:tasks.workunit.client.0.smithi107.stdout:8/351: creat d2/d18/d1a/d5e/d4f/f7a x:0 0 0 2024-04-25T00:18:27.119 INFO:tasks.workunit.client.0.smithi107.stdout:8/352: write d2/d18/d1a/f6c [497995,71709] 0 2024-04-25T00:18:27.119 INFO:tasks.workunit.client.0.smithi107.stdout:8/353: stat d2/cd 0 2024-04-25T00:18:27.119 INFO:tasks.workunit.client.0.smithi107.stdout:0/179: dread dc/dd/d16/f2a [0,4194304] 0 2024-04-25T00:18:27.119 INFO:tasks.workunit.client.0.smithi107.stdout:5/350: truncate d4/dc/fd 1609494 0 2024-04-25T00:18:27.122 INFO:tasks.workunit.client.0.smithi107.stdout:5/351: creat d4/dc/d26/d29/d51/f77 x:0 0 0 2024-04-25T00:18:27.122 INFO:tasks.workunit.client.0.smithi107.stdout:5/352: readlink d4/dc/l66 0 2024-04-25T00:18:27.124 INFO:tasks.workunit.client.0.smithi107.stdout:5/353: mkdir d4/dc/d26/d29/d51/d78 0 2024-04-25T00:18:27.126 INFO:tasks.workunit.client.0.smithi107.stdout:6/234: unlink d1/d2/c7 0 2024-04-25T00:18:27.127 INFO:tasks.workunit.client.0.smithi107.stdout:5/354: rename d4/d35/f6a to d4/d17/f79 0 2024-04-25T00:18:27.127 INFO:tasks.workunit.client.0.smithi107.stdout:5/355: dread - d4/d35/f3c zero size 2024-04-25T00:18:27.130 INFO:tasks.workunit.client.0.smithi107.stdout:7/192: dwrite d0/d8/db/f20 [0,4194304] 0 2024-04-25T00:18:27.132 INFO:tasks.workunit.client.0.smithi107.stdout:3/200: dwrite d9/d15/d1a/f1c [0,4194304] 0 2024-04-25T00:18:27.132 INFO:tasks.workunit.client.0.smithi107.stdout:3/201: dread - d9/d15/f35 zero size 2024-04-25T00:18:27.136 INFO:tasks.workunit.client.0.smithi107.stdout:1/260: dwrite d2/da/f43 [0,4194304] 0 2024-04-25T00:18:27.136 INFO:tasks.workunit.client.0.smithi107.stdout:1/261: chown d2/da/d54/c44 254966 1 2024-04-25T00:18:27.138 INFO:tasks.workunit.client.0.smithi107.stdout:3/202: unlink d9/d22/c19 0 2024-04-25T00:18:27.138 INFO:tasks.workunit.client.0.smithi107.stdout:9/212: dread d0/d43/f10 [0,4194304] 0 2024-04-25T00:18:27.138 INFO:tasks.workunit.client.0.smithi107.stdout:9/213: write d0/d8/d34/f42 [300126,23290] 0 2024-04-25T00:18:27.138 INFO:tasks.workunit.client.0.smithi107.stdout:7/193: rmdir d0/d21/d46/d42 0 2024-04-25T00:18:27.141 INFO:tasks.workunit.client.0.smithi107.stdout:4/287: dwrite d3/d24/d25/f49 [0,4194304] 0 2024-04-25T00:18:27.141 INFO:tasks.workunit.client.0.smithi107.stdout:4/288: truncate d3/d24/d25/f26 1119017 0 2024-04-25T00:18:27.141 INFO:tasks.workunit.client.0.smithi107.stdout:4/289: write d3/f2a [1812764,51893] 0 2024-04-25T00:18:27.147 INFO:tasks.workunit.client.0.smithi107.stdout:4/290: link d3/d5/d32/l37 d3/d5/l66 0 2024-04-25T00:18:27.148 INFO:tasks.workunit.client.0.smithi107.stdout:6/235: dwrite d1/d2/db/f4d [0,4194304] 0 2024-04-25T00:18:27.148 INFO:tasks.workunit.client.0.smithi107.stdout:6/236: fdatasync d1/d2/db/d3c/f50 0 2024-04-25T00:18:27.151 INFO:tasks.workunit.client.0.smithi107.stdout:7/194: rename d0/d8/db/d18/d19/d2d to d0/d8/db/d18/d19/d47 0 2024-04-25T00:18:27.152 INFO:tasks.workunit.client.0.smithi107.stdout:3/203: mkdir d9/d2f/d3b 0 2024-04-25T00:18:27.152 INFO:tasks.workunit.client.0.smithi107.stdout:3/204: write d9/d15/d1a/f32 [1501545,26938] 0 2024-04-25T00:18:27.152 INFO:tasks.workunit.client.0.smithi107.stdout:3/205: chown d9/d15/d1a/c26 1107 1 2024-04-25T00:18:27.154 INFO:tasks.workunit.client.0.smithi107.stdout:4/291: dread d3/d10/f1b [0,4194304] 0 2024-04-25T00:18:27.154 INFO:tasks.workunit.client.0.smithi107.stdout:6/237: truncate d1/d2/db/d32/d36/f3f 671328 0 2024-04-25T00:18:27.159 INFO:tasks.workunit.client.0.smithi107.stdout:1/262: write d2/da/d2f/f3e [3036005,37424] 0 2024-04-25T00:18:27.159 INFO:tasks.workunit.client.0.smithi107.stdout:4/292: dread d3/f58 [0,4194304] 0 2024-04-25T00:18:27.160 INFO:tasks.workunit.client.0.smithi107.stdout:4/293: mkdir d3/d24/d25/d67 0 2024-04-25T00:18:27.160 INFO:tasks.workunit.client.0.smithi107.stdout:4/294: write d3/d5/d32/f45 [651461,55372] 0 2024-04-25T00:18:27.162 INFO:tasks.workunit.client.0.smithi107.stdout:5/356: dwrite d4/dc/d26/f64 [0,4194304] 0 2024-04-25T00:18:27.162 INFO:tasks.workunit.client.0.smithi107.stdout:7/195: symlink d0/l48 0 2024-04-25T00:18:27.164 INFO:tasks.workunit.client.0.smithi107.stdout:9/214: dwrite d0/d8/d32/f38 [0,4194304] 0 2024-04-25T00:18:27.165 INFO:tasks.workunit.client.0.smithi107.stdout:5/357: unlink d4/d1c/d5c/f6f 0 2024-04-25T00:18:27.165 INFO:tasks.workunit.client.0.smithi107.stdout:5/358: write d4/f1b [913302,55350] 0 2024-04-25T00:18:27.169 INFO:tasks.workunit.client.0.smithi107.stdout:9/215: unlink d0/d1b/f2e 0 2024-04-25T00:18:27.172 INFO:tasks.workunit.client.0.smithi107.stdout:9/216: creat d0/d8/f45 x:0 0 0 2024-04-25T00:18:27.172 INFO:tasks.workunit.client.0.smithi107.stdout:3/206: dread d9/f20 [0,4194304] 0 2024-04-25T00:18:27.174 INFO:tasks.workunit.client.0.smithi107.stdout:6/238: dread d1/d2/db/d32/d36/d3a/f47 [0,4194304] 0 2024-04-25T00:18:27.174 INFO:tasks.workunit.client.0.smithi107.stdout:6/239: stat d1/d2/db/d3c/d43 0 2024-04-25T00:18:27.174 INFO:tasks.workunit.client.0.smithi107.stdout:6/240: fsync d1/d2/d3/f41 0 2024-04-25T00:18:27.175 INFO:tasks.workunit.client.0.smithi107.stdout:6/241: chown d1/d2/db/d32/d33 217920007 1 2024-04-25T00:18:27.175 INFO:tasks.workunit.client.0.smithi107.stdout:6/242: write d1/d2/d3/d28/f53 [136101,42012] 0 2024-04-25T00:18:27.175 INFO:tasks.workunit.client.0.smithi107.stdout:6/243: getdents d1/d2/db/d32/d36 0 2024-04-25T00:18:27.176 INFO:tasks.workunit.client.0.smithi107.stdout:5/359: read d4/f48 [401046,65909] 0 2024-04-25T00:18:27.176 INFO:tasks.workunit.client.0.smithi107.stdout:6/244: mkdir d1/d2/db/d48/d56 0 2024-04-25T00:18:27.176 INFO:tasks.workunit.client.0.smithi107.stdout:6/245: getdents d1/d2/d3/d26 0 2024-04-25T00:18:27.177 INFO:tasks.workunit.client.0.smithi107.stdout:5/360: creat d4/dc/d26/d62/f7a x:0 0 0 2024-04-25T00:18:27.178 INFO:tasks.workunit.client.0.smithi107.stdout:0/180: link dc/dd/ce dc/dd/d16/d1a/c39 0 2024-04-25T00:18:27.179 INFO:tasks.workunit.client.0.smithi107.stdout:5/361: rename d4/c52 to d4/dc/d26/d29/d51/c7b 0 2024-04-25T00:18:27.179 INFO:tasks.workunit.client.0.smithi107.stdout:5/362: chown d4/dc/d26/d29/d51/f77 3318 1 2024-04-25T00:18:27.180 INFO:tasks.workunit.client.0.smithi107.stdout:6/246: dread d1/d14/d18/f2c [4194304,4194304] 0 2024-04-25T00:18:27.180 INFO:tasks.workunit.client.0.smithi107.stdout:6/247: chown d1/d2/d2b/l1a 122055362 1 2024-04-25T00:18:27.180 INFO:tasks.workunit.client.0.smithi107.stdout:6/248: stat d1 0 2024-04-25T00:18:27.180 INFO:tasks.workunit.client.0.smithi107.stdout:6/249: chown d1/d2/db/d32/d36/f46 2028767554 1 2024-04-25T00:18:27.180 INFO:tasks.workunit.client.0.smithi107.stdout:6/250: readlink d1/d2/d2b/l1a 0 2024-04-25T00:18:27.181 INFO:tasks.workunit.client.0.smithi107.stdout:6/251: stat d1/f1f 0 2024-04-25T00:18:27.181 INFO:tasks.workunit.client.0.smithi107.stdout:6/252: mknod d1/d2/db/d32/d36/d3a/c57 0 2024-04-25T00:18:27.185 INFO:tasks.workunit.client.0.smithi107.stdout:0/181: dread dc/f1f [0,4194304] 0 2024-04-25T00:18:27.185 INFO:tasks.workunit.client.0.smithi107.stdout:0/182: chown dc/c2c 650222422 1 2024-04-25T00:18:27.191 INFO:tasks.workunit.client.0.smithi107.stdout:0/183: dread dc/f1f [0,4194304] 0 2024-04-25T00:18:27.191 INFO:tasks.workunit.client.0.smithi107.stdout:0/184: fdatasync dc/d24/d25/d31/f36 0 2024-04-25T00:18:27.194 INFO:tasks.workunit.client.0.smithi107.stdout:7/196: dwrite d0/d8/db/d18/d19/f3c [0,4194304] 0 2024-04-25T00:18:27.194 INFO:tasks.workunit.client.0.smithi107.stdout:7/197: chown d0/d8/db/d18/d19/d47 100 1 2024-04-25T00:18:27.197 INFO:tasks.workunit.client.0.smithi107.stdout:7/198: creat d0/d8/db/f49 x:0 0 0 2024-04-25T00:18:27.198 INFO:tasks.workunit.client.0.smithi107.stdout:9/217: dread d0/d8/de/f21 [0,4194304] 0 2024-04-25T00:18:27.199 INFO:tasks.workunit.client.0.smithi107.stdout:3/207: dread d9/d22/f2e [0,4194304] 0 2024-04-25T00:18:27.200 INFO:tasks.workunit.client.0.smithi107.stdout:3/208: creat d9/d2f/f3c x:0 0 0 2024-04-25T00:18:27.201 INFO:tasks.workunit.client.0.smithi107.stdout:4/295: dwrite d3/fd [0,4194304] 0 2024-04-25T00:18:27.202 INFO:tasks.workunit.client.0.smithi107.stdout:3/209: creat d9/d15/d39/f3d x:0 0 0 2024-04-25T00:18:27.202 INFO:tasks.workunit.client.0.smithi107.stdout:3/210: rename d9 to d9/d15/d39/d3e 22 2024-04-25T00:18:27.203 INFO:tasks.workunit.client.0.smithi107.stdout:5/363: dwrite d4/dc/d26/d42/f4f [0,4194304] 0 2024-04-25T00:18:27.205 INFO:tasks.workunit.client.0.smithi107.stdout:5/364: mkdir d4/d7c 0 2024-04-25T00:18:27.206 INFO:tasks.workunit.client.0.smithi107.stdout:5/365: mknod d4/dc/d26/c7d 0 2024-04-25T00:18:27.208 INFO:tasks.workunit.client.0.smithi107.stdout:5/366: mknod d4/dc/c7e 0 2024-04-25T00:18:27.208 INFO:tasks.workunit.client.0.smithi107.stdout:5/367: write d4/d17/f38 [982395,69947] 0 2024-04-25T00:18:27.209 INFO:tasks.workunit.client.0.smithi107.stdout:5/368: creat d4/dc/f7f x:0 0 0 2024-04-25T00:18:27.211 INFO:tasks.workunit.client.0.smithi107.stdout:4/296: truncate d3/f62 322542 0 2024-04-25T00:18:27.211 INFO:tasks.workunit.client.0.smithi107.stdout:4/297: dread - d3/d5/f4d zero size 2024-04-25T00:18:27.214 INFO:tasks.workunit.client.0.smithi107.stdout:9/218: read d0/d8/de/f21 [527835,85200] 0 2024-04-25T00:18:27.220 INFO:tasks.workunit.client.0.smithi107.stdout:6/253: dread d1/d2/db/f4d [0,4194304] 0 2024-04-25T00:18:27.222 INFO:tasks.workunit.client.0.smithi107.stdout:6/254: creat d1/d2/db/d32/d33/d4f/f58 x:0 0 0 2024-04-25T00:18:27.230 INFO:tasks.workunit.client.0.smithi107.stdout:5/369: write d4/dc/f2f [3781735,128684] 0 2024-04-25T00:18:27.230 INFO:tasks.workunit.client.0.smithi107.stdout:4/298: dwrite d3/f35 [0,4194304] 0 2024-04-25T00:18:27.231 INFO:tasks.workunit.client.0.smithi107.stdout:3/211: dwrite d9/d2f/f3c [0,4194304] 0 2024-04-25T00:18:27.232 INFO:tasks.workunit.client.0.smithi107.stdout:5/370: mkdir d4/dc/d26/d29/d80 0 2024-04-25T00:18:27.232 INFO:tasks.workunit.client.0.smithi107.stdout:4/299: mknod d3/d24/d61/c68 0 2024-04-25T00:18:27.233 INFO:tasks.workunit.client.0.smithi107.stdout:0/185: dwrite dc/dd/d16/d33/f37 [0,4194304] 0 2024-04-25T00:18:27.233 INFO:tasks.workunit.client.0.smithi107.stdout:9/219: dread d0/f40 [0,4194304] 0 2024-04-25T00:18:27.235 INFO:tasks.workunit.client.0.smithi107.stdout:0/186: getdents dc/d24 0 2024-04-25T00:18:27.235 INFO:tasks.workunit.client.0.smithi107.stdout:0/187: chown dc/c34 239390 1 2024-04-25T00:18:27.236 INFO:tasks.workunit.client.0.smithi107.stdout:4/300: link f0 d3/d24/f69 0 2024-04-25T00:18:27.236 INFO:tasks.workunit.client.0.smithi107.stdout:6/255: dread d1/d14/d18/f2c [0,4194304] 0 2024-04-25T00:18:27.236 INFO:tasks.workunit.client.0.smithi107.stdout:9/220: creat d0/d8/d32/f46 x:0 0 0 2024-04-25T00:18:27.236 INFO:tasks.workunit.client.0.smithi107.stdout:9/221: truncate d0/d8/de/f33 726335 0 2024-04-25T00:18:27.236 INFO:tasks.workunit.client.0.smithi107.stdout:9/222: write d0/f2 [2459571,16103] 0 2024-04-25T00:18:27.237 INFO:tasks.workunit.client.0.smithi107.stdout:3/212: fsync d9/d2f/f3c 0 2024-04-25T00:18:27.237 INFO:tasks.workunit.client.0.smithi107.stdout:9/223: read d0/d1b/f2f [342223,82139] 0 2024-04-25T00:18:27.237 INFO:tasks.workunit.client.0.smithi107.stdout:4/301: rename d3/d10/f1c to d3/d24/d5c/f6a 0 2024-04-25T00:18:27.239 INFO:tasks.workunit.client.0.smithi107.stdout:9/224: mknod d0/d8/d34/d36/c47 0 2024-04-25T00:18:27.241 INFO:tasks.workunit.client.0.smithi107.stdout:9/225: mknod d0/d8/c48 0 2024-04-25T00:18:27.241 INFO:tasks.workunit.client.0.smithi107.stdout:9/226: fdatasync d0/f2 0 2024-04-25T00:18:27.243 INFO:tasks.workunit.client.0.smithi107.stdout:3/213: write d9/d22/f2e [3038780,104838] 0 2024-04-25T00:18:27.244 INFO:tasks.workunit.client.0.smithi107.stdout:4/302: unlink d3/f2a 0 2024-04-25T00:18:27.244 INFO:tasks.workunit.client.0.smithi107.stdout:4/303: chown d3/d5/d32/l3a 101 1 2024-04-25T00:18:27.244 INFO:tasks.workunit.client.0.smithi107.stdout:9/227: mkdir d0/d43/d49 0 2024-04-25T00:18:27.244 INFO:tasks.workunit.client.0.smithi107.stdout:4/304: mknod d3/d24/d61/c6b 0 2024-04-25T00:18:27.246 INFO:tasks.workunit.client.0.smithi107.stdout:3/214: symlink d9/d2f/l3f 0 2024-04-25T00:18:27.246 INFO:tasks.workunit.client.0.smithi107.stdout:3/215: stat d9/f20 0 2024-04-25T00:18:27.246 INFO:tasks.workunit.client.0.smithi107.stdout:9/228: creat d0/d8/d34/d36/f4a x:0 0 0 2024-04-25T00:18:27.246 INFO:tasks.workunit.client.0.smithi107.stdout:9/229: write d0/d8/de/f21 [2235071,46451] 0 2024-04-25T00:18:27.248 INFO:tasks.workunit.client.0.smithi107.stdout:3/216: creat d9/d2f/f40 x:0 0 0 2024-04-25T00:18:27.248 INFO:tasks.workunit.client.0.smithi107.stdout:9/230: creat d0/f4b x:0 0 0 2024-04-25T00:18:27.249 INFO:tasks.workunit.client.0.smithi107.stdout:9/231: mknod d0/d8/c4c 0 2024-04-25T00:18:27.249 INFO:tasks.workunit.client.0.smithi107.stdout:9/232: chown d0/f40 1 1 2024-04-25T00:18:27.250 INFO:tasks.workunit.client.0.smithi107.stdout:9/233: creat d0/d8/d32/f4d x:0 0 0 2024-04-25T00:18:27.251 INFO:tasks.workunit.client.0.smithi107.stdout:9/234: symlink d0/d8/de/l4e 0 2024-04-25T00:18:27.252 INFO:tasks.workunit.client.0.smithi107.stdout:9/235: creat d0/d8/d34/f4f x:0 0 0 2024-04-25T00:18:27.252 INFO:tasks.workunit.client.0.smithi107.stdout:9/236: fdatasync d0/d43/f1c 0 2024-04-25T00:18:27.253 INFO:tasks.workunit.client.0.smithi107.stdout:9/237: readlink d0/d8/d1f/l41 0 2024-04-25T00:18:27.253 INFO:tasks.workunit.client.0.smithi107.stdout:3/217: dread d9/f14 [0,4194304] 0 2024-04-25T00:18:27.253 INFO:tasks.workunit.client.0.smithi107.stdout:0/188: dwrite dc/dd/f14 [4194304,4194304] 0 2024-04-25T00:18:27.253 INFO:tasks.workunit.client.0.smithi107.stdout:9/238: mkdir d0/d8/d32/d50 0 2024-04-25T00:18:27.254 INFO:tasks.workunit.client.0.smithi107.stdout:3/218: mkdir d9/d2f/d3b/d41 0 2024-04-25T00:18:27.256 INFO:tasks.workunit.client.0.smithi107.stdout:0/189: creat dc/d24/f3a x:0 0 0 2024-04-25T00:18:27.257 INFO:tasks.workunit.client.0.smithi107.stdout:9/239: truncate d0/d43/fd 673671 0 2024-04-25T00:18:27.258 INFO:tasks.workunit.client.0.smithi107.stdout:3/219: link d9/f14 d9/f42 0 2024-04-25T00:18:27.260 INFO:tasks.workunit.client.0.smithi107.stdout:0/190: mknod dc/d24/d25/d31/c3b 0 2024-04-25T00:18:27.261 INFO:tasks.workunit.client.0.smithi107.stdout:9/240: creat d0/d8/de/f51 x:0 0 0 2024-04-25T00:18:27.261 INFO:tasks.workunit.client.0.smithi107.stdout:9/241: truncate d0/d8/d34/f3f 25761 0 2024-04-25T00:18:27.261 INFO:tasks.workunit.client.0.smithi107.stdout:9/242: write d0/f2 [3504843,85856] 0 2024-04-25T00:18:27.262 INFO:tasks.workunit.client.0.smithi107.stdout:4/305: dwrite d3/f35 [0,4194304] 0 2024-04-25T00:18:27.262 INFO:tasks.workunit.client.0.smithi107.stdout:4/306: stat d3/d24/d25/l48 0 2024-04-25T00:18:27.262 INFO:tasks.workunit.client.0.smithi107.stdout:4/307: fsync d3/d5/f65 0 2024-04-25T00:18:27.263 INFO:tasks.workunit.client.0.smithi107.stdout:0/191: creat dc/dd/f3c x:0 0 0 2024-04-25T00:18:27.264 INFO:tasks.workunit.client.0.smithi107.stdout:4/308: rename d3/d24/d25/c33 to d3/d10/c6c 0 2024-04-25T00:18:27.265 INFO:tasks.workunit.client.0.smithi107.stdout:9/243: getdents d0/d43 0 2024-04-25T00:18:27.273 INFO:tasks.workunit.client.0.smithi107.stdout:4/309: getdents d3/d24/d25 0 2024-04-25T00:18:27.273 INFO:tasks.workunit.client.0.smithi107.stdout:4/310: chown d3/c17 1 1 2024-04-25T00:18:27.275 INFO:tasks.workunit.client.0.smithi107.stdout:6/256: link d1/f1f d1/d2/db/d32/d36/f59 0 2024-04-25T00:18:27.279 INFO:tasks.workunit.client.0.smithi107.stdout:4/311: dread d3/d5/d32/d53/f5b [0,4194304] 0 2024-04-25T00:18:27.281 INFO:tasks.workunit.client.0.smithi107.stdout:4/312: symlink d3/l6d 0 2024-04-25T00:18:27.281 INFO:tasks.workunit.client.0.smithi107.stdout:4/313: fsync d3/f62 0 2024-04-25T00:18:27.282 INFO:tasks.workunit.client.0.smithi107.stdout:3/220: dwrite f7 [0,4194304] 0 2024-04-25T00:18:27.282 INFO:tasks.workunit.client.0.smithi107.stdout:3/221: read d9/d15/d1a/f30 [203328,76210] 0 2024-04-25T00:18:27.283 INFO:tasks.workunit.client.0.smithi107.stdout:6/257: dread d1/d2/db/d32/d36/d3a/f47 [0,4194304] 0 2024-04-25T00:18:27.283 INFO:tasks.workunit.client.0.smithi107.stdout:6/258: fsync d1/d2/d3/d26/f51 0 2024-04-25T00:18:27.283 INFO:tasks.workunit.client.0.smithi107.stdout:6/259: stat d1/d14/d18/c54 0 2024-04-25T00:18:27.283 INFO:tasks.workunit.client.0.smithi107.stdout:6/260: fsync d1/d2/d3/f41 0 2024-04-25T00:18:27.286 INFO:tasks.workunit.client.0.smithi107.stdout:6/261: rename d1/d2/db/c15 to d1/d2/db/d3c/d43/c5a 0 2024-04-25T00:18:27.286 INFO:tasks.workunit.client.0.smithi107.stdout:3/222: symlink d9/d15/l43 0 2024-04-25T00:18:27.286 INFO:tasks.workunit.client.0.smithi107.stdout:3/223: chown d9/d2f/f40 455253 1 2024-04-25T00:18:27.288 INFO:tasks.workunit.client.0.smithi107.stdout:0/192: dwrite dc/f18 [0,4194304] 0 2024-04-25T00:18:27.288 INFO:tasks.workunit.client.0.smithi107.stdout:0/193: dread - dc/d24/d25/d31/f36 zero size 2024-04-25T00:18:27.288 INFO:tasks.workunit.client.0.smithi107.stdout:9/244: dwrite d0/d8/d32/f4d [0,4194304] 0 2024-04-25T00:18:27.289 INFO:tasks.workunit.client.0.smithi107.stdout:4/314: dread d3/d24/d5c/f6a [4194304,4194304] 0 2024-04-25T00:18:27.290 INFO:tasks.workunit.client.0.smithi107.stdout:4/315: truncate d3/d5/d32/f54 696317 0 2024-04-25T00:18:27.290 INFO:tasks.workunit.client.0.smithi107.stdout:4/316: fdatasync d3/f18 0 2024-04-25T00:18:27.291 INFO:tasks.workunit.client.0.smithi107.stdout:4/317: dread d3/d5/f23 [0,4194304] 0 2024-04-25T00:18:27.292 INFO:tasks.workunit.client.0.smithi107.stdout:4/318: chown d3/d5/f22 3 1 2024-04-25T00:18:27.292 INFO:tasks.workunit.client.0.smithi107.stdout:3/224: mkdir d9/d2f/d44 0 2024-04-25T00:18:27.293 INFO:tasks.workunit.client.0.smithi107.stdout:3/225: fsync d9/d2f/f3c 0 2024-04-25T00:18:27.293 INFO:tasks.workunit.client.0.smithi107.stdout:0/194: write dc/dd/f19 [1031088,58632] 0 2024-04-25T00:18:27.294 INFO:tasks.workunit.client.0.smithi107.stdout:0/195: stat dc/dd/d16/f1b 0 2024-04-25T00:18:27.294 INFO:tasks.workunit.client.0.smithi107.stdout:0/196: creat dc/dd/d16/d33/f3d x:0 0 0 2024-04-25T00:18:27.296 INFO:tasks.workunit.client.0.smithi107.stdout:9/245: unlink d0/d8/de/f21 0 2024-04-25T00:18:27.297 INFO:tasks.workunit.client.0.smithi107.stdout:3/226: symlink d9/d2f/d3b/l45 0 2024-04-25T00:18:27.298 INFO:tasks.workunit.client.0.smithi107.stdout:6/262: dread d1/d2/db/d32/d36/f59 [4194304,4194304] 0 2024-04-25T00:18:27.298 INFO:tasks.workunit.client.0.smithi107.stdout:6/263: truncate d1/d2/d3/d28/f53 872779 0 2024-04-25T00:18:27.298 INFO:tasks.workunit.client.0.smithi107.stdout:0/197: symlink dc/l3e 0 2024-04-25T00:18:27.298 INFO:tasks.workunit.client.0.smithi107.stdout:6/264: mkdir d1/d2/d2b/d3e/d5b 0 2024-04-25T00:18:27.300 INFO:tasks.workunit.client.0.smithi107.stdout:9/246: mknod d0/d8/d32/c52 0 2024-04-25T00:18:27.300 INFO:tasks.workunit.client.0.smithi107.stdout:4/319: dread d3/d5/f41 [0,4194304] 0 2024-04-25T00:18:27.301 INFO:tasks.workunit.client.0.smithi107.stdout:4/320: dread d3/d5/d32/f54 [0,4194304] 0 2024-04-25T00:18:27.301 INFO:tasks.workunit.client.0.smithi107.stdout:4/321: stat d3/d10/l31 0 2024-04-25T00:18:27.302 INFO:tasks.workunit.client.0.smithi107.stdout:4/322: creat d3/d5/d32/d53/f6e x:0 0 0 2024-04-25T00:18:27.304 INFO:tasks.workunit.client.0.smithi107.stdout:3/227: creat d9/d2f/f46 x:0 0 0 2024-04-25T00:18:27.304 INFO:tasks.workunit.client.0.smithi107.stdout:9/247: creat d0/d8/d32/d50/f53 x:0 0 0 2024-04-25T00:18:27.305 INFO:tasks.workunit.client.0.smithi107.stdout:0/198: write dc/dd/f15 [1506823,113294] 0 2024-04-25T00:18:27.305 INFO:tasks.workunit.client.0.smithi107.stdout:9/248: write d0/d8/d34/f42 [3740393,16104] 0 2024-04-25T00:18:27.305 INFO:tasks.workunit.client.0.smithi107.stdout:3/228: creat d9/d2f/d3b/f47 x:0 0 0 2024-04-25T00:18:27.307 INFO:tasks.workunit.client.0.smithi107.stdout:9/249: rename d0/d8/d32/d50/f53 to d0/d8/d32/d50/f54 0 2024-04-25T00:18:27.307 INFO:tasks.workunit.client.0.smithi107.stdout:9/250: read d0/f2 [1838557,93762] 0 2024-04-25T00:18:27.308 INFO:tasks.workunit.client.0.smithi107.stdout:4/323: write d3/d5/f3d [7246,83411] 0 2024-04-25T00:18:27.308 INFO:tasks.workunit.client.0.smithi107.stdout:3/229: rename d9/d22/f2e to d9/d22/d1b/f48 0 2024-04-25T00:18:27.310 INFO:tasks.workunit.client.0.smithi107.stdout:9/251: rename d0/f2 to d0/d8/f55 0 2024-04-25T00:18:27.312 INFO:tasks.workunit.client.0.smithi107.stdout:4/324: unlink d3/d5/l36 0 2024-04-25T00:18:27.312 INFO:tasks.workunit.client.0.smithi107.stdout:4/325: chown d3/d5/d32/l46 103 1 2024-04-25T00:18:27.312 INFO:tasks.workunit.client.0.smithi107.stdout:4/326: truncate d3/d5/f23 524628 0 2024-04-25T00:18:27.312 INFO:tasks.workunit.client.0.smithi107.stdout:0/199: dread dc/dd/f19 [0,4194304] 0 2024-04-25T00:18:27.313 INFO:tasks.workunit.client.0.smithi107.stdout:3/230: rename d9/d22/d1b/c38 to d9/d15/d39/c49 0 2024-04-25T00:18:27.313 INFO:tasks.workunit.client.0.smithi107.stdout:9/252: mkdir d0/d8/d56 0 2024-04-25T00:18:27.316 INFO:tasks.workunit.client.0.smithi107.stdout:0/200: unlink dc/d24/f2d 0 2024-04-25T00:18:27.316 INFO:tasks.workunit.client.0.smithi107.stdout:0/201: write dc/d24/f3a [584384,36382] 0 2024-04-25T00:18:27.316 INFO:tasks.workunit.client.0.smithi107.stdout:0/202: fsync dc/d24/d25/f2b 0 2024-04-25T00:18:27.318 INFO:tasks.workunit.client.0.smithi107.stdout:9/253: unlink d0/d8/de/c37 0 2024-04-25T00:18:27.318 INFO:tasks.workunit.client.0.smithi107.stdout:9/254: read - d0/d8/d32/f46 zero size 2024-04-25T00:18:27.318 INFO:tasks.workunit.client.0.smithi107.stdout:9/255: truncate d0/d43/f10 4580548 0 2024-04-25T00:18:27.318 INFO:tasks.workunit.client.0.smithi107.stdout:9/256: truncate d0/d8/de/f51 613149 0 2024-04-25T00:18:27.318 INFO:tasks.workunit.client.0.smithi107.stdout:6/265: dwrite d1/d2/d3/f22 [0,4194304] 0 2024-04-25T00:18:27.319 INFO:tasks.workunit.client.0.smithi107.stdout:4/327: rename d3/f62 to d3/d24/d5c/f6f 0 2024-04-25T00:18:27.320 INFO:tasks.workunit.client.0.smithi107.stdout:9/257: link d0/d8/f26 d0/d8/d56/f57 0 2024-04-25T00:18:27.320 INFO:tasks.workunit.client.0.smithi107.stdout:9/258: fdatasync d0/d43/f28 0 2024-04-25T00:18:27.320 INFO:tasks.workunit.client.0.smithi107.stdout:6/266: mknod d1/d2/d2b/d3e/c5c 0 2024-04-25T00:18:27.321 INFO:tasks.workunit.client.0.smithi107.stdout:6/267: getdents d1/d2/db/d48/d56 0 2024-04-25T00:18:27.322 INFO:tasks.workunit.client.0.smithi107.stdout:4/328: dread d3/f21 [0,4194304] 0 2024-04-25T00:18:27.322 INFO:tasks.workunit.client.0.smithi107.stdout:4/329: fdatasync d3/f35 0 2024-04-25T00:18:27.322 INFO:tasks.workunit.client.0.smithi107.stdout:4/330: write d3/d5/f23 [536546,15131] 0 2024-04-25T00:18:27.323 INFO:tasks.workunit.client.0.smithi107.stdout:4/331: creat d3/d24/d61/f70 x:0 0 0 2024-04-25T00:18:27.323 INFO:tasks.workunit.client.0.smithi107.stdout:4/332: chown d3/d24/d25/d67 3 1 2024-04-25T00:18:27.323 INFO:tasks.workunit.client.0.smithi107.stdout:6/268: write d1/d2/db/d32/d36/f3f [260935,75941] 0 2024-04-25T00:18:27.324 INFO:tasks.workunit.client.0.smithi107.stdout:6/269: read d1/d2/db/d32/d36/f46 [64618,14722] 0 2024-04-25T00:18:27.324 INFO:tasks.workunit.client.0.smithi107.stdout:6/270: chown d1/d2/db/d48/d56 93 1 2024-04-25T00:18:27.325 INFO:tasks.workunit.client.0.smithi107.stdout:6/271: dread d1/d2/db/d32/d36/f46 [0,4194304] 0 2024-04-25T00:18:27.326 INFO:tasks.workunit.client.0.smithi107.stdout:6/272: rmdir d1/d2/db/d32 39 2024-04-25T00:18:27.337 INFO:tasks.workunit.client.0.smithi107.stdout:0/203: dwrite f1 [0,4194304] 0 2024-04-25T00:18:27.340 INFO:tasks.workunit.client.0.smithi107.stdout:9/259: write d0/d8/d32/f4d [1439097,11029] 0 2024-04-25T00:18:27.340 INFO:tasks.workunit.client.0.smithi107.stdout:3/231: dwrite f0 [0,4194304] 0 2024-04-25T00:18:27.340 INFO:tasks.workunit.client.0.smithi107.stdout:0/204: dread dc/dd/d16/f22 [0,4194304] 0 2024-04-25T00:18:27.340 INFO:tasks.workunit.client.0.smithi107.stdout:0/205: write dc/d24/d25/f2b [1165910,80436] 0 2024-04-25T00:18:27.342 INFO:tasks.workunit.client.0.smithi107.stdout:9/260: creat d0/d8/d34/d36/d3d/f58 x:0 0 0 2024-04-25T00:18:27.343 INFO:tasks.workunit.client.0.smithi107.stdout:3/232: symlink d9/d15/l4a 0 2024-04-25T00:18:27.344 INFO:tasks.workunit.client.0.smithi107.stdout:4/333: dwrite d3/f18 [0,4194304] 0 2024-04-25T00:18:27.344 INFO:tasks.workunit.client.0.smithi107.stdout:4/334: fsync d3/f21 0 2024-04-25T00:18:27.344 INFO:tasks.workunit.client.0.smithi107.stdout:9/261: creat d0/d8/d56/f59 x:0 0 0 2024-04-25T00:18:27.344 INFO:tasks.workunit.client.0.smithi107.stdout:9/262: truncate d0/d43/fd 942624 0 2024-04-25T00:18:27.345 INFO:tasks.workunit.client.0.smithi107.stdout:3/233: mkdir d9/d2f/d44/d4b 0 2024-04-25T00:18:27.347 INFO:tasks.workunit.client.0.smithi107.stdout:4/335: mkdir d3/d24/d25/d71 0 2024-04-25T00:18:27.347 INFO:tasks.workunit.client.0.smithi107.stdout:9/263: creat d0/d8/f5a x:0 0 0 2024-04-25T00:18:27.349 INFO:tasks.workunit.client.0.smithi107.stdout:4/336: symlink d3/d5/d32/d53/l72 0 2024-04-25T00:18:27.349 INFO:tasks.workunit.client.0.smithi107.stdout:9/264: creat d0/d8/d32/f5b x:0 0 0 2024-04-25T00:18:27.352 INFO:tasks.workunit.client.0.smithi107.stdout:6/273: dwrite d1/d2/db/d32/d36/f59 [0,4194304] 0 2024-04-25T00:18:27.355 INFO:tasks.workunit.client.0.smithi107.stdout:6/274: mknod d1/d2/db/c5d 0 2024-04-25T00:18:27.355 INFO:tasks.workunit.client.0.smithi107.stdout:6/275: write d1/d2/db/d32/d36/f3f [1208360,57244] 0 2024-04-25T00:18:27.360 INFO:tasks.workunit.client.0.smithi107.stdout:4/337: write d3/d5/f12 [135913,66192] 0 2024-04-25T00:18:27.363 INFO:tasks.workunit.client.0.smithi107.stdout:0/206: dwrite dc/d24/d25/f2b [0,4194304] 0 2024-04-25T00:18:27.363 INFO:tasks.workunit.client.0.smithi107.stdout:9/265: dread d0/fc [0,4194304] 0 2024-04-25T00:18:27.367 INFO:tasks.workunit.client.0.smithi107.stdout:0/207: mkdir dc/dd/d16/d1a/d3f 0 2024-04-25T00:18:27.368 INFO:tasks.workunit.client.0.smithi107.stdout:0/208: dread - dc/d24/d25/d31/f36 zero size 2024-04-25T00:18:27.368 INFO:tasks.workunit.client.0.smithi107.stdout:0/209: write dc/dd/f3c [406736,9319] 0 2024-04-25T00:18:27.368 INFO:tasks.workunit.client.0.smithi107.stdout:0/210: mknod dc/dd/d16/d33/d35/c40 0 2024-04-25T00:18:27.379 INFO:tasks.workunit.client.0.smithi107.stdout:9/266: mkdir d0/d8/de/d5c 0 2024-04-25T00:18:27.379 INFO:tasks.workunit.client.0.smithi107.stdout:0/211: getdents dc/dd/d16/d1a 0 2024-04-25T00:18:27.383 INFO:tasks.workunit.client.0.smithi107.stdout:9/267: symlink d0/l5d 0 2024-04-25T00:18:27.383 INFO:tasks.workunit.client.0.smithi107.stdout:9/268: truncate d0/f30 918042 0 2024-04-25T00:18:27.387 INFO:tasks.workunit.client.0.smithi107.stdout:9/269: unlink d0/cb 0 2024-04-25T00:18:27.388 INFO:tasks.workunit.client.0.smithi107.stdout:0/212: mknod dc/dd/c41 0 2024-04-25T00:18:27.388 INFO:tasks.workunit.client.0.smithi107.stdout:0/213: truncate dc/d24/d25/d31/f36 731980 0 2024-04-25T00:18:27.391 INFO:tasks.workunit.client.0.smithi107.stdout:9/270: mknod d0/c5e 0 2024-04-25T00:18:27.392 INFO:tasks.workunit.client.0.smithi107.stdout:0/214: dread dc/dd/d16/f2a [0,4194304] 0 2024-04-25T00:18:27.392 INFO:tasks.workunit.client.0.smithi107.stdout:0/215: readlink dc/l12 0 2024-04-25T00:18:27.392 INFO:tasks.workunit.client.0.smithi107.stdout:9/271: mknod d0/d8/c5f 0 2024-04-25T00:18:27.394 INFO:tasks.workunit.client.0.smithi107.stdout:0/216: symlink dc/d24/l42 0 2024-04-25T00:18:27.398 INFO:tasks.workunit.client.0.smithi107.stdout:9/272: unlink d0/d8/d34/d36/d3d/f58 0 2024-04-25T00:18:27.398 INFO:tasks.workunit.client.0.smithi107.stdout:1/263: sync 2024-04-25T00:18:27.398 INFO:tasks.workunit.client.0.smithi107.stdout:9/273: chown d0/d8/d34/f42 19113 1 2024-04-25T00:18:27.398 INFO:tasks.workunit.client.0.smithi107.stdout:2/280: sync 2024-04-25T00:18:27.400 INFO:tasks.workunit.client.0.smithi107.stdout:9/274: link d0/d43/f1c d0/d43/f60 0 2024-04-25T00:18:27.404 INFO:tasks.workunit.client.0.smithi107.stdout:1/264: mkdir d2/dd/d1a/d55 0 2024-04-25T00:18:27.404 INFO:tasks.workunit.client.0.smithi107.stdout:6/276: dwrite d1/d2/db/d32/d36/f59 [4194304,4194304] 0 2024-04-25T00:18:27.406 INFO:tasks.workunit.client.0.smithi107.stdout:2/281: getdents d6/d20/d22 0 2024-04-25T00:18:27.409 INFO:tasks.workunit.client.0.smithi107.stdout:1/265: dread d2/dd/d25/f2e [0,4194304] 0 2024-04-25T00:18:27.409 INFO:tasks.workunit.client.0.smithi107.stdout:9/275: mknod d0/d8/de/d5c/c61 0 2024-04-25T00:18:27.411 INFO:tasks.workunit.client.0.smithi107.stdout:2/282: rename d6/fe to d6/d18/d40/d42/f62 0 2024-04-25T00:18:27.411 INFO:tasks.workunit.client.0.smithi107.stdout:2/283: fdatasync d6/d18/d40/d42/d2b/f32 0 2024-04-25T00:18:27.412 INFO:tasks.workunit.client.0.smithi107.stdout:0/217: dwrite dc/f1f [0,4194304] 0 2024-04-25T00:18:27.416 INFO:tasks.workunit.client.0.smithi107.stdout:1/266: rmdir d2/dd/d1a/d55 0 2024-04-25T00:18:27.416 INFO:tasks.workunit.client.0.smithi107.stdout:1/267: truncate d2/f51 475243 0 2024-04-25T00:18:27.420 INFO:tasks.workunit.client.0.smithi107.stdout:9/276: link d0/d8/d34/d36/f4a d0/d8/f62 0 2024-04-25T00:18:27.423 INFO:tasks.workunit.client.0.smithi107.stdout:0/218: dread dc/dd/f15 [0,4194304] 0 2024-04-25T00:18:27.423 INFO:tasks.workunit.client.0.smithi107.stdout:1/268: unlink d2/dd/d1b/f36 0 2024-04-25T00:18:27.423 INFO:tasks.workunit.client.0.smithi107.stdout:1/269: dread - d2/dd/d1b/f52 zero size 2024-04-25T00:18:27.424 INFO:tasks.workunit.client.0.smithi107.stdout:9/277: rmdir d0/d43/d49 0 2024-04-25T00:18:27.429 INFO:tasks.workunit.client.0.smithi107.stdout:6/277: dwrite d1/d2/d3/f41 [0,4194304] 0 2024-04-25T00:18:27.429 INFO:tasks.workunit.client.0.smithi107.stdout:6/278: chown d1/d2/db/d32/d33/c37 95 1 2024-04-25T00:18:27.433 INFO:tasks.workunit.client.0.smithi107.stdout:2/284: dwrite d6/f3d [0,4194304] 0 2024-04-25T00:18:27.433 INFO:tasks.workunit.client.0.smithi107.stdout:2/285: fsync d6/d20/d22/f3a 0 2024-04-25T00:18:27.433 INFO:tasks.workunit.client.0.smithi107.stdout:0/219: dread dc/f18 [0,4194304] 0 2024-04-25T00:18:27.434 INFO:tasks.workunit.client.0.smithi107.stdout:1/270: dread d2/dd/d1a/f27 [0,4194304] 0 2024-04-25T00:18:27.438 INFO:tasks.workunit.client.0.smithi107.stdout:0/220: creat dc/dd/d16/d1a/f43 x:0 0 0 2024-04-25T00:18:27.438 INFO:tasks.workunit.client.0.smithi107.stdout:0/221: rename dc/dd/d16 to dc/dd/d16/d33/d44 22 2024-04-25T00:18:27.444 INFO:tasks.workunit.client.0.smithi107.stdout:9/278: dwrite d0/d8/d34/f4f [0,4194304] 0 2024-04-25T00:18:27.444 INFO:tasks.workunit.client.0.smithi107.stdout:6/279: rmdir d1/d14 39 2024-04-25T00:18:27.446 INFO:tasks.workunit.client.0.smithi107.stdout:6/280: creat d1/d2/db/d32/d36/d3d/f5e x:0 0 0 2024-04-25T00:18:27.449 INFO:tasks.workunit.client.0.smithi107.stdout:6/281: creat d1/d2/f5f x:0 0 0 2024-04-25T00:18:27.449 INFO:tasks.workunit.client.0.smithi107.stdout:2/286: dwrite d6/d18/d40/d42/d2b/f32 [0,4194304] 0 2024-04-25T00:18:27.450 INFO:tasks.workunit.client.0.smithi107.stdout:6/282: creat d1/d2/db/d32/d36/d3d/f60 x:0 0 0 2024-04-25T00:18:27.452 INFO:tasks.workunit.client.0.smithi107.stdout:6/283: symlink d1/d14/l61 0 2024-04-25T00:18:27.456 INFO:tasks.workunit.client.0.smithi107.stdout:0/222: dwrite dc/dd/d16/d1a/f43 [0,4194304] 0 2024-04-25T00:18:27.458 INFO:tasks.workunit.client.0.smithi107.stdout:0/223: link dc/d24/l42 dc/d24/d38/l45 0 2024-04-25T00:18:27.458 INFO:tasks.workunit.client.0.smithi107.stdout:0/224: write dc/dd/d16/d1a/f1e [11792,708] 0 2024-04-25T00:18:27.458 INFO:tasks.workunit.client.0.smithi107.stdout:0/225: chown dc/d24/d25/d31/f36 229411316 1 2024-04-25T00:18:27.460 INFO:tasks.workunit.client.0.smithi107.stdout:2/287: write d6/f51 [594703,106548] 0 2024-04-25T00:18:27.462 INFO:tasks.workunit.client.0.smithi107.stdout:2/288: mknod d6/d29/d4e/c63 0 2024-04-25T00:18:27.464 INFO:tasks.workunit.client.0.smithi107.stdout:2/289: mknod d6/d18/d40/d42/c64 0 2024-04-25T00:18:27.464 INFO:tasks.workunit.client.0.smithi107.stdout:2/290: write d6/d20/d22/f1c [751929,97909] 0 2024-04-25T00:18:27.464 INFO:tasks.workunit.client.0.smithi107.stdout:2/291: stat d6/d18/d40/d42 0 2024-04-25T00:18:27.466 INFO:tasks.workunit.client.0.smithi107.stdout:2/292: creat d6/d18/f65 x:0 0 0 2024-04-25T00:18:27.466 INFO:tasks.workunit.client.0.smithi107.stdout:2/293: chown d6/d20/d22 1 1 2024-04-25T00:18:27.466 INFO:tasks.workunit.client.0.smithi107.stdout:2/294: dread - d6/d18/d40/d42/f47 zero size 2024-04-25T00:18:27.466 INFO:tasks.workunit.client.0.smithi107.stdout:2/295: fdatasync d6/d18/d40/d42/f57 0 2024-04-25T00:18:27.467 INFO:tasks.workunit.client.0.smithi107.stdout:2/296: unlink d6/d20/l53 0 2024-04-25T00:18:27.468 INFO:tasks.workunit.client.0.smithi107.stdout:0/226: dread f1 [0,4194304] 0 2024-04-25T00:18:27.468 INFO:tasks.workunit.client.0.smithi107.stdout:0/227: readlink dc/l26 0 2024-04-25T00:18:27.469 INFO:tasks.workunit.client.0.smithi107.stdout:2/297: symlink d6/d29/l66 0 2024-04-25T00:18:27.470 INFO:tasks.workunit.client.0.smithi107.stdout:0/228: creat dc/dd/d16/d1a/f46 x:0 0 0 2024-04-25T00:18:27.472 INFO:tasks.workunit.client.0.smithi107.stdout:0/229: truncate dc/dd/d16/d33/f37 1858628 0 2024-04-25T00:18:27.473 INFO:tasks.workunit.client.0.smithi107.stdout:6/284: dwrite d1/d2/db/f34 [0,4194304] 0 2024-04-25T00:18:27.473 INFO:tasks.workunit.client.0.smithi107.stdout:6/285: chown d1/d2/d2b/d3e/c5c 225 1 2024-04-25T00:18:27.475 INFO:tasks.workunit.client.0.smithi107.stdout:2/298: rename d6/d20/d22/d2c to d6/d18/d4d/d67 0 2024-04-25T00:18:27.475 INFO:tasks.workunit.client.0.smithi107.stdout:2/299: chown d6/d18/d40/f55 4342226 1 2024-04-25T00:18:27.476 INFO:tasks.workunit.client.0.smithi107.stdout:6/286: symlink d1/d2/db/d32/l62 0 2024-04-25T00:18:27.482 INFO:tasks.workunit.client.0.smithi107.stdout:2/300: unlink d6/d20/d22/f21 0 2024-04-25T00:18:27.492 INFO:tasks.workunit.client.0.smithi107.stdout:8/354: sync 2024-04-25T00:18:27.492 INFO:tasks.workunit.client.0.smithi107.stdout:7/199: sync 2024-04-25T00:18:27.492 INFO:tasks.workunit.client.0.smithi107.stdout:5/371: sync 2024-04-25T00:18:27.492 INFO:tasks.workunit.client.0.smithi107.stdout:8/355: readlink d2/d18/d1a/d5e/d5c/l67 0 2024-04-25T00:18:27.492 INFO:tasks.workunit.client.0.smithi107.stdout:8/356: fsync d2/d18/d1a/f6c 0 2024-04-25T00:18:27.492 INFO:tasks.workunit.client.0.smithi107.stdout:8/357: chown d2/d18/d1a/d5e/d4f/f76 10055 1 2024-04-25T00:18:27.492 INFO:tasks.workunit.client.0.smithi107.stdout:8/358: dread - d2/d18/d1a/d5e/d6d/f6f zero size 2024-04-25T00:18:27.492 INFO:tasks.workunit.client.0.smithi107.stdout:8/359: write d2/d18/f68 [774053,125564] 0 2024-04-25T00:18:27.492 INFO:tasks.workunit.client.0.smithi107.stdout:8/360: dread - d2/d18/d1a/d33/f64 zero size 2024-04-25T00:18:27.493 INFO:tasks.workunit.client.0.smithi107.stdout:0/230: dwrite dc/dd/f15 [0,4194304] 0 2024-04-25T00:18:27.493 INFO:tasks.workunit.client.0.smithi107.stdout:7/200: creat d0/d8/f4a x:0 0 0 2024-04-25T00:18:27.494 INFO:tasks.workunit.client.0.smithi107.stdout:8/361: getdents d2/d18 0 2024-04-25T00:18:27.495 INFO:tasks.workunit.client.0.smithi107.stdout:7/201: symlink d0/d8/db/d18/d19/d39/l4b 0 2024-04-25T00:18:27.495 INFO:tasks.workunit.client.0.smithi107.stdout:7/202: truncate d0/d8/db/d18/d19/d39/f3e 361804 0 2024-04-25T00:18:27.495 INFO:tasks.workunit.client.0.smithi107.stdout:7/203: chown d0/d8/db/d18/f41 11246 1 2024-04-25T00:18:27.502 INFO:tasks.workunit.client.0.smithi107.stdout:2/301: dread d6/d20/d22/f1a [0,4194304] 0 2024-04-25T00:18:27.502 INFO:tasks.workunit.client.0.smithi107.stdout:2/302: write d6/d18/d40/d42/d2b/d30/d45/f5f [348981,38275] 0 2024-04-25T00:18:27.503 INFO:tasks.workunit.client.0.smithi107.stdout:2/303: write d6/f10 [4627394,85604] 0 2024-04-25T00:18:27.505 INFO:tasks.workunit.client.0.smithi107.stdout:2/304: rename d6/d18/d40/d42/d2b/f5b to d6/d20/d22/d56/f68 0 2024-04-25T00:18:27.505 INFO:tasks.workunit.client.0.smithi107.stdout:2/305: truncate d6/d20/f5a 946997 0 2024-04-25T00:18:27.506 INFO:tasks.workunit.client.0.smithi107.stdout:2/306: read d6/f8 [1590663,115466] 0 2024-04-25T00:18:27.508 INFO:tasks.workunit.client.0.smithi107.stdout:5/372: write d4/f71 [1271612,106002] 0 2024-04-25T00:18:27.508 INFO:tasks.workunit.client.0.smithi107.stdout:5/373: chown d4/dc/fd 622711 1 2024-04-25T00:18:27.508 INFO:tasks.workunit.client.0.smithi107.stdout:5/374: stat d4/dc/d26 0 2024-04-25T00:18:27.509 INFO:tasks.workunit.client.0.smithi107.stdout:8/362: dread d2/f16 [0,4194304] 0 2024-04-25T00:18:27.509 INFO:tasks.workunit.client.0.smithi107.stdout:8/363: chown d2/d18/d1a/d5e/d5c 798222 1 2024-04-25T00:18:27.509 INFO:tasks.workunit.client.0.smithi107.stdout:6/287: dwrite d1/d2/db/d3c/f4e [0,4194304] 0 2024-04-25T00:18:27.511 INFO:tasks.workunit.client.0.smithi107.stdout:8/364: symlink d2/d18/l7b 0 2024-04-25T00:18:27.511 INFO:tasks.workunit.client.0.smithi107.stdout:8/365: fdatasync d2/d18/d1a/d5e/d3e/f6a 0 2024-04-25T00:18:27.513 INFO:tasks.workunit.client.0.smithi107.stdout:6/288: creat d1/d2/db/d32/d33/f63 x:0 0 0 2024-04-25T00:18:27.513 INFO:tasks.workunit.client.0.smithi107.stdout:8/366: dread d2/d18/d1a/d5e/f47 [0,4194304] 0 2024-04-25T00:18:27.513 INFO:tasks.workunit.client.0.smithi107.stdout:8/367: write d2/d18/d1a/d5e/f47 [490521,12643] 0 2024-04-25T00:18:27.513 INFO:tasks.workunit.client.0.smithi107.stdout:6/289: getdents d1/d14 0 2024-04-25T00:18:27.514 INFO:tasks.workunit.client.0.smithi107.stdout:6/290: mknod d1/d14/d18/c64 0 2024-04-25T00:18:27.515 INFO:tasks.workunit.client.0.smithi107.stdout:5/375: fdatasync d4/dc/d26/d42/f59 0 2024-04-25T00:18:27.515 INFO:tasks.workunit.client.0.smithi107.stdout:5/376: rename d4/dc/d26 to d4/dc/d26/d29/d2e/d81 22 2024-04-25T00:18:27.516 INFO:tasks.workunit.client.0.smithi107.stdout:5/377: rename d4/dc/d54 to d4/dc/d26/d29/d2e/d82 0 2024-04-25T00:18:27.518 INFO:tasks.workunit.client.0.smithi107.stdout:6/291: unlink d1/d2/d3/f22 0 2024-04-25T00:18:27.519 INFO:tasks.workunit.client.0.smithi107.stdout:5/378: mkdir d4/dc/d83 0 2024-04-25T00:18:27.519 INFO:tasks.workunit.client.0.smithi107.stdout:0/231: dwrite dc/dd/d16/f1b [0,4194304] 0 2024-04-25T00:18:27.519 INFO:tasks.workunit.client.0.smithi107.stdout:2/307: dread d6/d20/d22/d56/f68 [0,4194304] 0 2024-04-25T00:18:27.520 INFO:tasks.workunit.client.0.smithi107.stdout:5/379: truncate d4/dc/fd 85105 0 2024-04-25T00:18:27.520 INFO:tasks.workunit.client.0.smithi107.stdout:5/380: readlink d4/l9 0 2024-04-25T00:18:27.521 INFO:tasks.workunit.client.0.smithi107.stdout:7/204: dwrite d0/d8/fd [0,4194304] 0 2024-04-25T00:18:27.521 INFO:tasks.workunit.client.0.smithi107.stdout:2/308: mkdir d6/d18/d40/d42/d69 0 2024-04-25T00:18:27.521 INFO:tasks.workunit.client.0.smithi107.stdout:2/309: chown d6/d18/d40/d42/f3f 340660 1 2024-04-25T00:18:27.522 INFO:tasks.workunit.client.0.smithi107.stdout:7/205: mknod d0/d21/d46/c4c 0 2024-04-25T00:18:27.522 INFO:tasks.workunit.client.0.smithi107.stdout:7/206: dread - d0/d8/db/d18/d19/f2f zero size 2024-04-25T00:18:27.524 INFO:tasks.workunit.client.0.smithi107.stdout:2/310: creat d6/d18/d40/d42/f6a x:0 0 0 2024-04-25T00:18:27.524 INFO:tasks.workunit.client.0.smithi107.stdout:7/207: mkdir d0/d8/db/d18/d19/d47/d4d 0 2024-04-25T00:18:27.524 INFO:tasks.workunit.client.0.smithi107.stdout:7/208: chown d0/d8/db/f2a 1077 1 2024-04-25T00:18:27.524 INFO:tasks.workunit.client.0.smithi107.stdout:7/209: chown d0/d8/db/f20 501399151 1 2024-04-25T00:18:27.525 INFO:tasks.workunit.client.0.smithi107.stdout:7/210: write d0/d8/db/f15 [386995,14544] 0 2024-04-25T00:18:27.532 INFO:tasks.workunit.client.0.smithi107.stdout:6/292: rmdir d1/d2/d3 39 2024-04-25T00:18:27.532 INFO:tasks.workunit.client.0.smithi107.stdout:6/293: chown d1/d2/db/d32/l62 168117757 1 2024-04-25T00:18:27.533 INFO:tasks.workunit.client.0.smithi107.stdout:5/381: write d4/fa [3084529,78136] 0 2024-04-25T00:18:27.533 INFO:tasks.workunit.client.0.smithi107.stdout:5/382: chown d4/d17/f69 40540275 1 2024-04-25T00:18:27.538 INFO:tasks.workunit.client.0.smithi107.stdout:8/368: dwrite d2/f27 [0,4194304] 0 2024-04-25T00:18:27.538 INFO:tasks.workunit.client.0.smithi107.stdout:5/383: mkdir d4/d35/d61/d84 0 2024-04-25T00:18:27.538 INFO:tasks.workunit.client.0.smithi107.stdout:5/384: chown d4/d17/c27 17003702 1 2024-04-25T00:18:27.538 INFO:tasks.workunit.client.0.smithi107.stdout:7/211: rename d0/d21 to d0/d8/d4e 0 2024-04-25T00:18:27.538 INFO:tasks.workunit.client.0.smithi107.stdout:7/212: getdents d0/d8/db/d18/d19/d47/d4d 0 2024-04-25T00:18:27.538 INFO:tasks.workunit.client.0.smithi107.stdout:7/213: chown d0/f26 842952 1 2024-04-25T00:18:27.538 INFO:tasks.workunit.client.0.smithi107.stdout:7/214: chown d0/f3b 181903643 1 2024-04-25T00:18:27.540 INFO:tasks.workunit.client.0.smithi107.stdout:0/232: dwrite f1 [0,4194304] 0 2024-04-25T00:18:27.541 INFO:tasks.workunit.client.0.smithi107.stdout:7/215: rename d0/d8/db/d18/d19/c37 to d0/d8/db/d18/d19/d47/d4d/c4f 0 2024-04-25T00:18:27.541 INFO:tasks.workunit.client.0.smithi107.stdout:8/369: creat d2/d18/d1a/f7c x:0 0 0 2024-04-25T00:18:27.542 INFO:tasks.workunit.client.0.smithi107.stdout:6/294: mknod d1/d2/c65 0 2024-04-25T00:18:27.542 INFO:tasks.workunit.client.0.smithi107.stdout:5/385: creat d4/d17/f85 x:0 0 0 2024-04-25T00:18:27.543 INFO:tasks.workunit.client.0.smithi107.stdout:7/216: creat d0/d8/d2c/f50 x:0 0 0 2024-04-25T00:18:27.543 INFO:tasks.workunit.client.0.smithi107.stdout:2/311: dwrite d6/d18/d40/d42/f62 [4194304,4194304] 0 2024-04-25T00:18:27.543 INFO:tasks.workunit.client.0.smithi107.stdout:6/295: mknod d1/d2/db/d48/d56/c66 0 2024-04-25T00:18:27.544 INFO:tasks.workunit.client.0.smithi107.stdout:5/386: mknod d4/d1c/d5c/c86 0 2024-04-25T00:18:27.544 INFO:tasks.workunit.client.0.smithi107.stdout:7/217: creat d0/d8/db/f51 x:0 0 0 2024-04-25T00:18:27.546 INFO:tasks.workunit.client.0.smithi107.stdout:2/312: mknod d6/d18/d40/d42/d2b/d30/d4c/c6b 0 2024-04-25T00:18:27.547 INFO:tasks.workunit.client.0.smithi107.stdout:5/387: symlink d4/dc/d83/l87 0 2024-04-25T00:18:27.547 INFO:tasks.workunit.client.0.smithi107.stdout:5/388: fsync d4/f48 0 2024-04-25T00:18:27.551 INFO:tasks.workunit.client.0.smithi107.stdout:5/389: mkdir d4/d7c/d88 0 2024-04-25T00:18:27.551 INFO:tasks.workunit.client.0.smithi107.stdout:5/390: dread - d4/d17/f79 zero size 2024-04-25T00:18:27.551 INFO:tasks.workunit.client.0.smithi107.stdout:5/391: fsync d4/dc/fd 0 2024-04-25T00:18:27.551 INFO:tasks.workunit.client.0.smithi107.stdout:5/392: chown d4/fa 60750456 1 2024-04-25T00:18:27.551 INFO:tasks.workunit.client.0.smithi107.stdout:2/313: dread d6/d18/d40/d42/f3b [0,4194304] 0 2024-04-25T00:18:27.551 INFO:tasks.workunit.client.0.smithi107.stdout:2/314: chown d6/d29 61996988 1 2024-04-25T00:18:27.552 INFO:tasks.workunit.client.0.smithi107.stdout:5/393: rmdir d4/d1c 39 2024-04-25T00:18:27.553 INFO:tasks.workunit.client.0.smithi107.stdout:2/315: link d6/d20/d22/l52 d6/d29/d4e/l6c 0 2024-04-25T00:18:27.554 INFO:tasks.workunit.client.0.smithi107.stdout:2/316: symlink d6/d29/d4e/l6d 0 2024-04-25T00:18:27.555 INFO:tasks.workunit.client.0.smithi107.stdout:6/296: truncate d1/f1f 7587577 0 2024-04-25T00:18:27.555 INFO:tasks.workunit.client.0.smithi107.stdout:6/297: dread - d1/d2/db/d32/d36/d3d/f5e zero size 2024-04-25T00:18:27.556 INFO:tasks.workunit.client.0.smithi107.stdout:2/317: mkdir d6/d18/d40/d42/d69/d6e 0 2024-04-25T00:18:27.557 INFO:tasks.workunit.client.0.smithi107.stdout:6/298: mknod d1/d2/db/d32/d36/d3a/c67 0 2024-04-25T00:18:27.558 INFO:tasks.workunit.client.0.smithi107.stdout:2/318: mkdir d6/d29/d4e/d6f 0 2024-04-25T00:18:27.560 INFO:tasks.workunit.client.0.smithi107.stdout:2/319: write d6/d20/d22/f1c [374023,70145] 0 2024-04-25T00:18:27.560 INFO:tasks.workunit.client.0.smithi107.stdout:2/320: readlink d6/l15 0 2024-04-25T00:18:27.564 INFO:tasks.workunit.client.0.smithi107.stdout:2/321: mkdir d6/d18/d40/d42/d70 0 2024-04-25T00:18:27.566 INFO:tasks.workunit.client.0.smithi107.stdout:7/218: dwrite d0/d8/db/d18/d19/f3c [0,4194304] 0 2024-04-25T00:18:27.566 INFO:tasks.workunit.client.0.smithi107.stdout:7/219: dread - d0/d8/db/f49 zero size 2024-04-25T00:18:27.567 INFO:tasks.workunit.client.0.smithi107.stdout:7/220: write d0/d8/d2c/f3d [345648,83601] 0 2024-04-25T00:18:27.569 INFO:tasks.workunit.client.0.smithi107.stdout:2/322: mknod d6/d18/d40/d42/d69/d6e/c71 0 2024-04-25T00:18:27.574 INFO:tasks.workunit.client.0.smithi107.stdout:5/394: dwrite d4/dc/d26/d62/f7a [0,4194304] 0 2024-04-25T00:18:27.576 INFO:tasks.workunit.client.0.smithi107.stdout:2/323: dread d6/f17 [0,4194304] 0 2024-04-25T00:18:27.578 INFO:tasks.workunit.client.0.smithi107.stdout:6/299: dwrite d1/d2/db/d3c/f4e [4194304,4194304] 0 2024-04-25T00:18:27.580 INFO:tasks.workunit.client.0.smithi107.stdout:6/300: mkdir d1/d2/db/d32/d33/d68 0 2024-04-25T00:18:27.581 INFO:tasks.workunit.client.0.smithi107.stdout:6/301: mknod d1/d2/db/d32/d36/d3d/c69 0 2024-04-25T00:18:27.586 INFO:tasks.workunit.client.0.smithi107.stdout:0/233: rename dc/d24/c30 to dc/dd/d16/d33/d35/c47 0 2024-04-25T00:18:27.587 INFO:tasks.workunit.client.0.smithi107.stdout:0/234: write dc/dd/d16/f22 [1380513,91933] 0 2024-04-25T00:18:27.587 INFO:tasks.workunit.client.0.smithi107.stdout:0/235: chown dc/dd/f15 148602344 1 2024-04-25T00:18:27.587 INFO:tasks.workunit.client.0.smithi107.stdout:0/236: dread - dc/dd/d16/d33/f3d zero size 2024-04-25T00:18:27.588 INFO:tasks.workunit.client.0.smithi107.stdout:5/395: write d4/dc/d26/d42/f5e [2688599,29355] 0 2024-04-25T00:18:27.588 INFO:tasks.workunit.client.0.smithi107.stdout:5/396: dread - d4/dc/d26/f4b zero size 2024-04-25T00:18:27.588 INFO:tasks.workunit.client.0.smithi107.stdout:5/397: write d4/dc/f20 [923176,63383] 0 2024-04-25T00:18:27.588 INFO:tasks.workunit.client.0.smithi107.stdout:5/398: fsync d4/dc/fe 0 2024-04-25T00:18:27.588 INFO:tasks.workunit.client.0.smithi107.stdout:8/370: rename d2/d18/d1a/d5e/d4f/f5f to d2/d18/d1a/f7d 0 2024-04-25T00:18:27.589 INFO:tasks.workunit.client.0.smithi107.stdout:2/324: read d6/d18/d40/d42/f57 [1634305,79478] 0 2024-04-25T00:18:27.589 INFO:tasks.workunit.client.0.smithi107.stdout:2/325: write d6/d20/d22/f1c [1853017,60212] 0 2024-04-25T00:18:27.589 INFO:tasks.workunit.client.0.smithi107.stdout:5/399: mknod d4/dc/d26/d29/d51/c89 0 2024-04-25T00:18:27.589 INFO:tasks.workunit.client.0.smithi107.stdout:5/400: write d4/dc/d26/d29/d2e/f4c [791870,48872] 0 2024-04-25T00:18:27.589 INFO:tasks.workunit.client.0.smithi107.stdout:5/401: truncate d4/dc/f7f 683197 0 2024-04-25T00:18:27.590 INFO:tasks.workunit.client.0.smithi107.stdout:2/326: creat d6/f72 x:0 0 0 2024-04-25T00:18:27.590 INFO:tasks.workunit.client.0.smithi107.stdout:2/327: readlink d6/d20/d22/l5c 0 2024-04-25T00:18:27.591 INFO:tasks.workunit.client.0.smithi107.stdout:2/328: creat d6/d18/d40/d42/d2b/d30/d4c/f73 x:0 0 0 2024-04-25T00:18:27.592 INFO:tasks.workunit.client.0.smithi107.stdout:2/329: mknod d6/d18/d40/d42/d2b/c74 0 2024-04-25T00:18:27.600 INFO:tasks.workunit.client.0.smithi107.stdout:5/402: write d4/dc/d26/f3d [2782302,74439] 0 2024-04-25T00:18:27.602 INFO:tasks.workunit.client.0.smithi107.stdout:5/403: symlink d4/dc/d26/d29/d2e/l8a 0 2024-04-25T00:18:27.607 INFO:tasks.workunit.client.0.smithi107.stdout:0/237: dwrite dc/f1f [0,4194304] 0 2024-04-25T00:18:27.607 INFO:tasks.workunit.client.0.smithi107.stdout:0/238: fdatasync dc/f1f 0 2024-04-25T00:18:27.607 INFO:tasks.workunit.client.0.smithi107.stdout:0/239: readlink dc/d24/d38/l45 0 2024-04-25T00:18:27.608 INFO:tasks.workunit.client.0.smithi107.stdout:6/302: rename d1/d14/c49 to d1/d2/db/d32/d36/d3a/c6a 0 2024-04-25T00:18:27.610 INFO:tasks.workunit.client.0.smithi107.stdout:5/404: truncate d4/dc/d26/d42/f59 2496790 0 2024-04-25T00:18:27.612 INFO:tasks.workunit.client.0.smithi107.stdout:0/240: mkdir dc/dd/d16/d48 0 2024-04-25T00:18:27.614 INFO:tasks.workunit.client.0.smithi107.stdout:5/405: mkdir d4/dc/d26/d42/d8b 0 2024-04-25T00:18:27.615 INFO:tasks.workunit.client.0.smithi107.stdout:8/371: dwrite d2/f11 [0,4194304] 0 2024-04-25T00:18:27.617 INFO:tasks.workunit.client.0.smithi107.stdout:5/406: rename d4/dc/d26/f3d to d4/dc/d26/f8c 0 2024-04-25T00:18:27.620 INFO:tasks.workunit.client.0.smithi107.stdout:2/330: dwrite f4 [0,4194304] 0 2024-04-25T00:18:27.621 INFO:tasks.workunit.client.0.smithi107.stdout:8/372: creat d2/d18/d1a/d5e/d4f/f7e x:0 0 0 2024-04-25T00:18:27.623 INFO:tasks.workunit.client.0.smithi107.stdout:5/407: rmdir d4/dc/d26/d62 39 2024-04-25T00:18:27.625 INFO:tasks.workunit.client.0.smithi107.stdout:2/331: stat d6/d20/d22/l52 0 2024-04-25T00:18:27.625 INFO:tasks.workunit.client.0.smithi107.stdout:2/332: readlink d6/d29/d4e/l6d 0 2024-04-25T00:18:27.625 INFO:tasks.workunit.client.0.smithi107.stdout:8/373: symlink d2/d18/d1a/d5e/d5c/l7f 0 2024-04-25T00:18:27.627 INFO:tasks.workunit.client.0.smithi107.stdout:6/303: dwrite d1/d14/d18/f2c [0,4194304] 0 2024-04-25T00:18:27.628 INFO:tasks.workunit.client.0.smithi107.stdout:6/304: mkdir d1/d2/db/d48/d56/d6b 0 2024-04-25T00:18:27.628 INFO:tasks.workunit.client.0.smithi107.stdout:5/408: write d4/f1b [70726,53460] 0 2024-04-25T00:18:27.629 INFO:tasks.workunit.client.0.smithi107.stdout:6/305: mknod d1/d2/d2b/d3e/d5b/c6c 0 2024-04-25T00:18:27.630 INFO:tasks.workunit.client.0.smithi107.stdout:8/374: creat d2/d18/d1a/f80 x:0 0 0 2024-04-25T00:18:27.630 INFO:tasks.workunit.client.0.smithi107.stdout:6/306: fdatasync d1/d2/d3/d28/f53 0 2024-04-25T00:18:27.631 INFO:tasks.workunit.client.0.smithi107.stdout:8/375: creat d2/d18/d1a/d5e/d4f/f81 x:0 0 0 2024-04-25T00:18:27.634 INFO:tasks.workunit.client.0.smithi107.stdout:8/376: mkdir d2/d18/d1a/d82 0 2024-04-25T00:18:27.636 INFO:tasks.workunit.client.0.smithi107.stdout:8/377: creat d2/f83 x:0 0 0 2024-04-25T00:18:27.639 INFO:tasks.workunit.client.0.smithi107.stdout:6/307: write d1/d14/d18/f2c [3951440,21622] 0 2024-04-25T00:18:27.645 INFO:tasks.workunit.client.0.smithi107.stdout:0/241: rename dc/dd/d16/d1a/f46 to dc/f49 0 2024-04-25T00:18:27.649 INFO:tasks.workunit.client.0.smithi107.stdout:2/333: rename d6/d20/c27 to d6/d18/d40/c75 0 2024-04-25T00:18:27.649 INFO:tasks.workunit.client.0.smithi107.stdout:2/334: write d6/d18/d4d/d67/f49 [847003,53129] 0 2024-04-25T00:18:27.649 INFO:tasks.workunit.client.0.smithi107.stdout:2/335: chown d6/d18/d40/c75 591 1 2024-04-25T00:18:27.650 INFO:tasks.workunit.client.0.smithi107.stdout:6/308: dread d1/d2/d2b/f27 [0,4194304] 0 2024-04-25T00:18:27.650 INFO:tasks.workunit.client.0.smithi107.stdout:0/242: unlink dc/dd/d16/d1a/f43 0 2024-04-25T00:18:27.651 INFO:tasks.workunit.client.0.smithi107.stdout:2/336: rename d6/f11 to d6/d18/d40/d42/d70/f76 0 2024-04-25T00:18:27.651 INFO:tasks.workunit.client.0.smithi107.stdout:6/309: creat d1/d2/db/d32/d33/f6d x:0 0 0 2024-04-25T00:18:27.652 INFO:tasks.workunit.client.0.smithi107.stdout:2/337: mkdir d6/d20/d77 0 2024-04-25T00:18:27.655 INFO:tasks.workunit.client.0.smithi107.stdout:8/378: dwrite d2/f16 [0,4194304] 0 2024-04-25T00:18:27.662 INFO:tasks.workunit.client.0.smithi107.stdout:8/379: truncate d2/d18/d1a/d33/f3f 1187943 0 2024-04-25T00:18:27.663 INFO:tasks.workunit.client.0.smithi107.stdout:6/310: write d1/d2/d3/f41 [2769577,38320] 0 2024-04-25T00:18:27.672 INFO:tasks.workunit.client.0.smithi107.stdout:2/338: dwrite d6/d20/d22/f1c [0,4194304] 0 2024-04-25T00:18:27.681 INFO:tasks.workunit.client.0.smithi107.stdout:0/243: rename l2 to dc/dd/d16/l4a 0 2024-04-25T00:18:27.683 INFO:tasks.workunit.client.0.smithi107.stdout:8/380: rename d2/d18/c40 to d2/d18/d1a/d5e/d3d/c84 0 2024-04-25T00:18:27.684 INFO:tasks.workunit.client.0.smithi107.stdout:0/244: symlink dc/d24/l4b 0 2024-04-25T00:18:27.684 INFO:tasks.workunit.client.0.smithi107.stdout:0/245: write dc/dd/f19 [1055129,57365] 0 2024-04-25T00:18:27.685 INFO:tasks.workunit.client.0.smithi107.stdout:0/246: creat dc/dd/f4c x:0 0 0 2024-04-25T00:18:27.697 INFO:tasks.workunit.client.0.smithi107.stdout:6/311: rename d1/d14/d18/l4a to d1/d2/db/d48/d56/l6e 0 2024-04-25T00:18:27.697 INFO:tasks.workunit.client.0.smithi107.stdout:6/312: stat d1/d2/d2b/d3e/d5b/c6c 0 2024-04-25T00:18:27.698 INFO:tasks.workunit.client.0.smithi107.stdout:6/313: readlink d1/d2/db/d32/l45 0 2024-04-25T00:18:27.698 INFO:tasks.workunit.client.0.smithi107.stdout:6/314: chown d1/d2/d3/d26 1009 1 2024-04-25T00:18:27.700 INFO:tasks.workunit.client.0.smithi107.stdout:8/381: dwrite d2/d18/d1a/f62 [0,4194304] 0 2024-04-25T00:18:27.700 INFO:tasks.workunit.client.0.smithi107.stdout:0/247: dwrite dc/f1f [4194304,4194304] 0 2024-04-25T00:18:27.704 INFO:tasks.workunit.client.0.smithi107.stdout:8/382: mkdir d2/d18/d1a/d5e/d3d/d85 0 2024-04-25T00:18:27.704 INFO:tasks.workunit.client.0.smithi107.stdout:8/383: truncate d2/d18/d1a/d5e/d4f/f52 1066895 0 2024-04-25T00:18:27.711 INFO:tasks.workunit.client.0.smithi107.stdout:6/315: write d1/d2/db/f34 [2036114,26154] 0 2024-04-25T00:18:27.714 INFO:tasks.workunit.client.0.smithi107.stdout:6/316: creat d1/d2/db/d48/d56/d6b/f6f x:0 0 0 2024-04-25T00:18:27.715 INFO:tasks.workunit.client.0.smithi107.stdout:0/248: dread dc/dd/f14 [0,4194304] 0 2024-04-25T00:18:27.721 INFO:tasks.workunit.client.0.smithi107.stdout:8/384: dwrite d2/d18/f68 [0,4194304] 0 2024-04-25T00:18:27.721 INFO:tasks.workunit.client.0.smithi107.stdout:8/385: dread - d2/d18/d1a/f73 zero size 2024-04-25T00:18:27.723 INFO:tasks.workunit.client.0.smithi107.stdout:8/386: creat d2/d18/d1a/d5e/d3e/f86 x:0 0 0 2024-04-25T00:18:27.724 INFO:tasks.workunit.client.0.smithi107.stdout:8/387: truncate d2/d18/d1a/f7d 3599359 0 2024-04-25T00:18:27.726 INFO:tasks.workunit.client.0.smithi107.stdout:8/388: symlink d2/d18/d1a/d5e/d5c/d6e/l87 0 2024-04-25T00:18:27.730 INFO:tasks.workunit.client.0.smithi107.stdout:3/234: sync 2024-04-25T00:18:27.736 INFO:tasks.workunit.client.0.smithi107.stdout:8/389: dread d2/f11 [0,4194304] 0 2024-04-25T00:18:27.737 INFO:tasks.workunit.client.0.smithi107.stdout:8/390: read d2/d18/d1a/f6c [496911,93436] 0 2024-04-25T00:18:27.737 INFO:tasks.workunit.client.0.smithi107.stdout:8/391: fsync d2/d18/d1a/d5e/d3e/f6b 0 2024-04-25T00:18:27.737 INFO:tasks.workunit.client.0.smithi107.stdout:8/392: truncate d2/d18/d1a/d33/f34 2066297 0 2024-04-25T00:18:27.737 INFO:tasks.workunit.client.0.smithi107.stdout:3/235: dread d9/d15/d1a/d2c/f37 [0,4194304] 0 2024-04-25T00:18:27.738 INFO:tasks.workunit.client.0.smithi107.stdout:0/249: dwrite dc/dd/f14 [4194304,4194304] 0 2024-04-25T00:18:27.738 INFO:tasks.workunit.client.0.smithi107.stdout:0/250: write dc/dd/d16/f22 [1736833,78318] 0 2024-04-25T00:18:27.738 INFO:tasks.workunit.client.0.smithi107.stdout:0/251: write dc/d24/f3a [639713,91880] 0 2024-04-25T00:18:27.738 INFO:tasks.workunit.client.0.smithi107.stdout:8/393: unlink d2/f57 0 2024-04-25T00:18:27.738 INFO:tasks.workunit.client.0.smithi107.stdout:6/317: dwrite d1/d2/db/d48/d56/d6b/f6f [0,4194304] 0 2024-04-25T00:18:27.740 INFO:tasks.workunit.client.0.smithi107.stdout:3/236: link d9/d22/c34 d9/d2f/d44/d4b/c4c 0 2024-04-25T00:18:27.740 INFO:tasks.workunit.client.0.smithi107.stdout:3/237: chown d9/l12 352037601 1 2024-04-25T00:18:27.740 INFO:tasks.workunit.client.0.smithi107.stdout:3/238: chown d9/d15/d1a/c26 939004 1 2024-04-25T00:18:27.741 INFO:tasks.workunit.client.0.smithi107.stdout:8/394: creat d2/d18/d1a/d5e/d5c/f88 x:0 0 0 2024-04-25T00:18:27.742 INFO:tasks.workunit.client.0.smithi107.stdout:0/252: symlink dc/dd/d16/l4d 0 2024-04-25T00:18:27.745 INFO:tasks.workunit.client.0.smithi107.stdout:6/318: unlink d1/d2/db/d32/d36/d3d/c69 0 2024-04-25T00:18:27.748 INFO:tasks.workunit.client.0.smithi107.stdout:3/239: getdents d9 0 2024-04-25T00:18:27.748 INFO:tasks.workunit.client.0.smithi107.stdout:3/240: chown d9/d22/l28 1 1 2024-04-25T00:18:27.749 INFO:tasks.workunit.client.0.smithi107.stdout:8/395: rename d2/d18/c1d to d2/d18/d1a/d5e/d4f/c89 0 2024-04-25T00:18:27.751 INFO:tasks.workunit.client.0.smithi107.stdout:0/253: truncate dc/f18 2049456 0 2024-04-25T00:18:27.752 INFO:tasks.workunit.client.0.smithi107.stdout:6/319: creat d1/d4c/f70 x:0 0 0 2024-04-25T00:18:27.755 INFO:tasks.workunit.client.0.smithi107.stdout:8/396: rename d2/d18/d1a/f37 to d2/d18/d1a/d5e/d3e/d74/f8a 0 2024-04-25T00:18:27.756 INFO:tasks.workunit.client.0.smithi107.stdout:0/254: symlink dc/d24/d25/d31/l4e 0 2024-04-25T00:18:27.756 INFO:tasks.workunit.client.0.smithi107.stdout:0/255: chown dc/dd/d16/f1b 528 1 2024-04-25T00:18:27.756 INFO:tasks.workunit.client.0.smithi107.stdout:0/256: stat dc/f1f 0 2024-04-25T00:18:27.767 INFO:tasks.workunit.client.0.smithi107.stdout:3/241: dwrite d9/d2f/f40 [0,4194304] 0 2024-04-25T00:18:27.767 INFO:tasks.workunit.client.0.smithi107.stdout:3/242: write d9/d15/d1a/f30 [3505244,123906] 0 2024-04-25T00:18:27.769 INFO:tasks.workunit.client.0.smithi107.stdout:3/243: mknod d9/d2f/d3b/c4d 0 2024-04-25T00:18:27.771 INFO:tasks.workunit.client.0.smithi107.stdout:3/244: mkdir d9/d15/d1a/d4e 0 2024-04-25T00:18:27.772 INFO:tasks.workunit.client.0.smithi107.stdout:3/245: rmdir d9/d22/d1b 39 2024-04-25T00:18:27.774 INFO:tasks.workunit.client.0.smithi107.stdout:3/246: symlink d9/d22/d1b/l4f 0 2024-04-25T00:18:27.774 INFO:tasks.workunit.client.0.smithi107.stdout:3/247: write f0 [4407202,79872] 0 2024-04-25T00:18:27.774 INFO:tasks.workunit.client.0.smithi107.stdout:3/248: read d9/d22/d1b/f48 [3976386,58236] 0 2024-04-25T00:18:27.774 INFO:tasks.workunit.client.0.smithi107.stdout:3/249: getdents d9/d2f 0 2024-04-25T00:18:27.775 INFO:tasks.workunit.client.0.smithi107.stdout:3/250: creat d9/d22/f50 x:0 0 0 2024-04-25T00:18:27.776 INFO:tasks.workunit.client.0.smithi107.stdout:3/251: symlink d9/d15/l51 0 2024-04-25T00:18:27.777 INFO:tasks.workunit.client.0.smithi107.stdout:8/397: dwrite d2/d18/d1a/f7c [0,4194304] 0 2024-04-25T00:18:27.777 INFO:tasks.workunit.client.0.smithi107.stdout:8/398: chown d2/d18/d1a/d5e/d5c/d6e/l87 152 1 2024-04-25T00:18:27.778 INFO:tasks.workunit.client.0.smithi107.stdout:3/252: symlink d9/d2f/d3b/d41/l52 0 2024-04-25T00:18:27.779 INFO:tasks.workunit.client.0.smithi107.stdout:3/253: symlink d9/d15/d1a/d3a/l53 0 2024-04-25T00:18:27.779 INFO:tasks.workunit.client.0.smithi107.stdout:3/254: write d9/d15/f35 [403275,35755] 0 2024-04-25T00:18:27.779 INFO:tasks.workunit.client.0.smithi107.stdout:3/255: dread - d9/d22/f50 zero size 2024-04-25T00:18:27.779 INFO:tasks.workunit.client.0.smithi107.stdout:8/399: creat d2/d18/d1a/d82/f8b x:0 0 0 2024-04-25T00:18:27.780 INFO:tasks.workunit.client.0.smithi107.stdout:3/256: mkdir d9/d54 0 2024-04-25T00:18:27.791 INFO:tasks.workunit.client.0.smithi107.stdout:3/257: dread d9/f10 [0,4194304] 0 2024-04-25T00:18:27.792 INFO:tasks.workunit.client.0.smithi107.stdout:1/271: sync 2024-04-25T00:18:27.792 INFO:tasks.workunit.client.0.smithi107.stdout:4/338: sync 2024-04-25T00:18:27.792 INFO:tasks.workunit.client.0.smithi107.stdout:1/272: write f1 [4322110,33177] 0 2024-04-25T00:18:27.793 INFO:tasks.workunit.client.0.smithi107.stdout:4/339: truncate d3/d24/d25/f49 3227058 0 2024-04-25T00:18:27.797 INFO:tasks.workunit.client.0.smithi107.stdout:4/340: rmdir d3/d24/d5c 39 2024-04-25T00:18:27.800 INFO:tasks.workunit.client.0.smithi107.stdout:4/341: unlink d3/d24/d25/c29 0 2024-04-25T00:18:27.801 INFO:tasks.workunit.client.0.smithi107.stdout:3/258: dread d9/d15/d1a/f1c [0,4194304] 0 2024-04-25T00:18:27.808 INFO:tasks.workunit.client.0.smithi107.stdout:4/342: dread d3/d24/d25/f49 [0,4194304] 0 2024-04-25T00:18:27.810 INFO:tasks.workunit.client.0.smithi107.stdout:4/343: symlink d3/l73 0 2024-04-25T00:18:27.810 INFO:tasks.workunit.client.0.smithi107.stdout:4/344: write d3/d24/f2c [254678,65965] 0 2024-04-25T00:18:27.811 INFO:tasks.workunit.client.0.smithi107.stdout:4/345: mkdir d3/d24/d5c/d74 0 2024-04-25T00:18:27.813 INFO:tasks.workunit.client.0.smithi107.stdout:1/273: dwrite d2/dd/d25/f2e [0,4194304] 0 2024-04-25T00:18:27.816 INFO:tasks.workunit.client.0.smithi107.stdout:3/259: dwrite d9/f20 [4194304,4194304] 0 2024-04-25T00:18:27.821 INFO:tasks.workunit.client.0.smithi107.stdout:3/260: truncate d9/f20 5730386 0 2024-04-25T00:18:27.833 INFO:tasks.workunit.client.0.smithi107.stdout:1/274: dwrite d2/da/d2f/f3e [0,4194304] 0 2024-04-25T00:18:27.835 INFO:tasks.workunit.client.0.smithi107.stdout:4/346: dwrite d3/d5/f7 [0,4194304] 0 2024-04-25T00:18:27.836 INFO:tasks.workunit.client.0.smithi107.stdout:4/347: write d3/f58 [642741,57944] 0 2024-04-25T00:18:27.836 INFO:tasks.workunit.client.0.smithi107.stdout:1/275: rename d2/dd/d1b/f42 to d2/d2a/d37/f56 0 2024-04-25T00:18:27.837 INFO:tasks.workunit.client.0.smithi107.stdout:4/348: dread d3/d5/d32/f54 [0,4194304] 0 2024-04-25T00:18:27.838 INFO:tasks.workunit.client.0.smithi107.stdout:1/276: mknod d2/dd/c57 0 2024-04-25T00:18:27.839 INFO:tasks.workunit.client.0.smithi107.stdout:4/349: creat d3/d24/d25/d71/f75 x:0 0 0 2024-04-25T00:18:27.839 INFO:tasks.workunit.client.0.smithi107.stdout:1/277: creat d2/da/d54/f58 x:0 0 0 2024-04-25T00:18:27.839 INFO:tasks.workunit.client.0.smithi107.stdout:1/278: stat d2/da/l14 0 2024-04-25T00:18:27.839 INFO:tasks.workunit.client.0.smithi107.stdout:1/279: stat d2/dd/l48 0 2024-04-25T00:18:27.840 INFO:tasks.workunit.client.0.smithi107.stdout:4/350: symlink d3/d5/l76 0 2024-04-25T00:18:27.840 INFO:tasks.workunit.client.0.smithi107.stdout:1/280: symlink d2/da/l59 0 2024-04-25T00:18:27.841 INFO:tasks.workunit.client.0.smithi107.stdout:3/261: dwrite d9/d22/d1b/f48 [0,4194304] 0 2024-04-25T00:18:27.842 INFO:tasks.workunit.client.0.smithi107.stdout:3/262: mkdir d9/d22/d1b/d55 0 2024-04-25T00:18:27.843 INFO:tasks.workunit.client.0.smithi107.stdout:3/263: rename d9/d22/f50 to d9/f56 0 2024-04-25T00:18:27.845 INFO:tasks.workunit.client.0.smithi107.stdout:3/264: creat d9/d2f/d44/d4b/f57 x:0 0 0 2024-04-25T00:18:27.845 INFO:tasks.workunit.client.0.smithi107.stdout:4/351: dread d3/d10/f1b [0,4194304] 0 2024-04-25T00:18:27.845 INFO:tasks.workunit.client.0.smithi107.stdout:1/281: dread d2/dd/d1a/f1d [0,4194304] 0 2024-04-25T00:18:27.845 INFO:tasks.workunit.client.0.smithi107.stdout:1/282: mknod d2/da/c5a 0 2024-04-25T00:18:27.846 INFO:tasks.workunit.client.0.smithi107.stdout:4/352: getdents d3/d24 0 2024-04-25T00:18:27.847 INFO:tasks.workunit.client.0.smithi107.stdout:4/353: mknod d3/d24/d25/c77 0 2024-04-25T00:18:27.848 INFO:tasks.workunit.client.0.smithi107.stdout:4/354: symlink d3/d24/d25/d67/l78 0 2024-04-25T00:18:27.848 INFO:tasks.workunit.client.0.smithi107.stdout:4/355: write d3/f58 [1440431,129638] 0 2024-04-25T00:18:27.850 INFO:tasks.workunit.client.0.smithi107.stdout:4/356: link d3/d5/f7 d3/d5/d32/f79 0 2024-04-25T00:18:27.869 INFO:tasks.workunit.client.0.smithi107.stdout:1/283: write d2/dd/d1b/f3a [3199638,76613] 0 2024-04-25T00:18:27.870 INFO:tasks.workunit.client.0.smithi107.stdout:3/265: dwrite d9/d15/d1a/f30 [0,4194304] 0 2024-04-25T00:18:27.870 INFO:tasks.workunit.client.0.smithi107.stdout:1/284: fsync d2/d2a/f4a 0 2024-04-25T00:18:27.870 INFO:tasks.workunit.client.0.smithi107.stdout:1/285: chown d2/dd/d1b/l2b 405102506 1 2024-04-25T00:18:27.871 INFO:tasks.workunit.client.0.smithi107.stdout:4/357: dread d3/d5/f7 [0,4194304] 0 2024-04-25T00:18:27.871 INFO:tasks.workunit.client.0.smithi107.stdout:4/358: fdatasync d3/d5/d32/d53/f6e 0 2024-04-25T00:18:27.871 INFO:tasks.workunit.client.0.smithi107.stdout:4/359: chown d3/d24/f69 1155569 1 2024-04-25T00:18:27.871 INFO:tasks.workunit.client.0.smithi107.stdout:4/360: chown d3/l1e 13463 1 2024-04-25T00:18:27.871 INFO:tasks.workunit.client.0.smithi107.stdout:1/286: mknod d2/dd/d1b/c5b 0 2024-04-25T00:18:27.871 INFO:tasks.workunit.client.0.smithi107.stdout:3/266: dread d9/f42 [0,4194304] 0 2024-04-25T00:18:27.872 INFO:tasks.workunit.client.0.smithi107.stdout:4/361: mkdir d3/d24/d5c/d7a 0 2024-04-25T00:18:27.873 INFO:tasks.workunit.client.0.smithi107.stdout:4/362: mkdir d3/d10/d7b 0 2024-04-25T00:18:27.873 INFO:tasks.workunit.client.0.smithi107.stdout:4/363: write d3/d24/d25/f2b [1182859,12833] 0 2024-04-25T00:18:27.873 INFO:tasks.workunit.client.0.smithi107.stdout:4/364: chown d3/d5/lf 2 1 2024-04-25T00:18:27.873 INFO:tasks.workunit.client.0.smithi107.stdout:4/365: chown d3/d10/c6c 2698920 1 2024-04-25T00:18:27.875 INFO:tasks.workunit.client.0.smithi107.stdout:4/366: fdatasync d3/d24/d25/f2b 0 2024-04-25T00:18:27.880 INFO:tasks.workunit.client.0.smithi107.stdout:1/287: write d2/dd/d25/f33 [2454589,1464] 0 2024-04-25T00:18:27.880 INFO:tasks.workunit.client.0.smithi107.stdout:1/288: chown d2/d2a/c31 149 1 2024-04-25T00:18:27.890 INFO:tasks.workunit.client.0.smithi107.stdout:4/367: dread d3/d24/d25/f43 [0,4194304] 0 2024-04-25T00:18:27.890 INFO:tasks.workunit.client.0.smithi107.stdout:4/368: read d3/d5/f23 [541759,122802] 0 2024-04-25T00:18:27.890 INFO:tasks.workunit.client.0.smithi107.stdout:4/369: readlink d3/d5/l66 0 2024-04-25T00:18:27.891 INFO:tasks.workunit.client.0.smithi107.stdout:3/267: dwrite f7 [0,4194304] 0 2024-04-25T00:18:27.906 INFO:tasks.workunit.client.0.smithi107.stdout:3/268: read f0 [1582557,61031] 0 2024-04-25T00:18:27.906 INFO:tasks.workunit.client.0.smithi107.stdout:4/370: getdents d3/d24/d5c 0 2024-04-25T00:18:27.908 INFO:tasks.workunit.client.0.smithi107.stdout:4/371: getdents d3/d5 0 2024-04-25T00:18:27.909 INFO:tasks.workunit.client.0.smithi107.stdout:1/289: dwrite d2/f30 [0,4194304] 0 2024-04-25T00:18:27.915 INFO:tasks.workunit.client.0.smithi107.stdout:1/290: dread d2/da/f19 [0,4194304] 0 2024-04-25T00:18:27.915 INFO:tasks.workunit.client.0.smithi107.stdout:1/291: dread - d2/dd/d25/f50 zero size 2024-04-25T00:18:27.915 INFO:tasks.workunit.client.0.smithi107.stdout:1/292: write d2/dd/f4e [394453,47481] 0 2024-04-25T00:18:27.925 INFO:tasks.workunit.client.0.smithi107.stdout:4/372: dwrite d3/d24/d25/f28 [0,4194304] 0 2024-04-25T00:18:27.926 INFO:tasks.workunit.client.0.smithi107.stdout:3/269: dwrite d9/f10 [0,4194304] 0 2024-04-25T00:18:27.927 INFO:tasks.workunit.client.0.smithi107.stdout:3/270: mknod d9/d15/d1a/d4e/c58 0 2024-04-25T00:18:27.934 INFO:tasks.workunit.client.0.smithi107.stdout:1/293: dread d2/f30 [0,4194304] 0 2024-04-25T00:18:27.934 INFO:tasks.workunit.client.0.smithi107.stdout:1/294: getdents d2/da/d54 0 2024-04-25T00:18:27.935 INFO:tasks.workunit.client.0.smithi107.stdout:1/295: mkdir d2/d2a/d5c 0 2024-04-25T00:18:27.936 INFO:tasks.workunit.client.0.smithi107.stdout:1/296: symlink d2/d2a/d5c/l5d 0 2024-04-25T00:18:27.937 INFO:tasks.workunit.client.0.smithi107.stdout:1/297: fsync d2/da/d2f/f3e 0 2024-04-25T00:18:27.939 INFO:tasks.workunit.client.0.smithi107.stdout:1/298: rename d2/da/d54 to d2/dd/d25/d5e 0 2024-04-25T00:18:27.940 INFO:tasks.workunit.client.0.smithi107.stdout:1/299: creat d2/dd/d25/f5f x:0 0 0 2024-04-25T00:18:27.942 INFO:tasks.workunit.client.0.smithi107.stdout:9/279: sync 2024-04-25T00:18:27.942 INFO:tasks.workunit.client.0.smithi107.stdout:9/280: fsync d0/d43/f1c 0 2024-04-25T00:18:27.942 INFO:tasks.workunit.client.0.smithi107.stdout:9/281: chown d0/d8/d32/f38 3027 1 2024-04-25T00:18:27.943 INFO:tasks.workunit.client.0.smithi107.stdout:4/373: dwrite d3/f21 [0,4194304] 0 2024-04-25T00:18:27.943 INFO:tasks.workunit.client.0.smithi107.stdout:4/374: chown d3/fd 75782237 1 2024-04-25T00:18:27.943 INFO:tasks.workunit.client.0.smithi107.stdout:4/375: readlink d3/l6d 0 2024-04-25T00:18:27.943 INFO:tasks.workunit.client.0.smithi107.stdout:9/282: creat d0/d1b/f63 x:0 0 0 2024-04-25T00:18:27.943 INFO:tasks.workunit.client.0.smithi107.stdout:9/283: chown d0/c5e 20886 1 2024-04-25T00:18:27.944 INFO:tasks.workunit.client.0.smithi107.stdout:9/284: creat d0/d8/d56/f64 x:0 0 0 2024-04-25T00:18:27.946 INFO:tasks.workunit.client.0.smithi107.stdout:9/285: unlink d0/d8/d56/f57 0 2024-04-25T00:18:27.946 INFO:tasks.workunit.client.0.smithi107.stdout:9/286: write d0/d8/d32/f5b [968894,107193] 0 2024-04-25T00:18:27.947 INFO:tasks.workunit.client.0.smithi107.stdout:9/287: mknod d0/d1b/c65 0 2024-04-25T00:18:27.952 INFO:tasks.workunit.client.0.smithi107.stdout:4/376: write d3/d5/f7 [2369169,3945] 0 2024-04-25T00:18:27.954 INFO:tasks.workunit.client.0.smithi107.stdout:4/377: getdents d3 0 2024-04-25T00:18:27.959 INFO:tasks.workunit.client.0.smithi107.stdout:1/300: dwrite d2/dd/f21 [0,4194304] 0 2024-04-25T00:18:27.970 INFO:tasks.workunit.client.0.smithi107.stdout:4/378: dwrite d3/d24/d25/f43 [0,4194304] 0 2024-04-25T00:18:27.971 INFO:tasks.workunit.client.0.smithi107.stdout:1/301: write d2/d2a/f4a [191545,832] 0 2024-04-25T00:18:27.971 INFO:tasks.workunit.client.0.smithi107.stdout:1/302: chown d2/c29 1390 1 2024-04-25T00:18:27.971 INFO:tasks.workunit.client.0.smithi107.stdout:1/303: write d2/f30 [4499362,119287] 0 2024-04-25T00:18:27.971 INFO:tasks.workunit.client.0.smithi107.stdout:4/379: creat d3/d24/d25/d71/f7c x:0 0 0 2024-04-25T00:18:27.971 INFO:tasks.workunit.client.0.smithi107.stdout:4/380: stat d3/d24/f2c 0 2024-04-25T00:18:27.971 INFO:tasks.workunit.client.0.smithi107.stdout:4/381: truncate d3/d5/d32/d53/f6e 544285 0 2024-04-25T00:18:27.972 INFO:tasks.workunit.client.0.smithi107.stdout:1/304: symlink d2/d2a/l60 0 2024-04-25T00:18:27.972 INFO:tasks.workunit.client.0.smithi107.stdout:4/382: creat d3/d24/d5c/d74/f7d x:0 0 0 2024-04-25T00:18:27.974 INFO:tasks.workunit.client.0.smithi107.stdout:1/305: rename d2/fc to d2/dd/d1a/f61 0 2024-04-25T00:18:27.980 INFO:tasks.workunit.client.0.smithi107.stdout:4/383: fsync d3/d24/d5c/d74/f7d 0 2024-04-25T00:18:27.980 INFO:tasks.workunit.client.0.smithi107.stdout:4/384: chown d3/l1e 131035419 1 2024-04-25T00:18:27.988 INFO:tasks.workunit.client.0.smithi107.stdout:1/306: dwrite d2/dd/d25/f5f [0,4194304] 0 2024-04-25T00:18:27.992 INFO:tasks.workunit.client.0.smithi107.stdout:1/307: rmdir d2/da/d40 0 2024-04-25T00:18:27.992 INFO:tasks.workunit.client.0.smithi107.stdout:1/308: stat d2/da/c24 0 2024-04-25T00:18:27.992 INFO:tasks.workunit.client.0.smithi107.stdout:4/385: write d3/d5/f5a [2779286,110953] 0 2024-04-25T00:18:27.992 INFO:tasks.workunit.client.0.smithi107.stdout:4/386: dread - d3/d24/d61/f70 zero size 2024-04-25T00:18:27.993 INFO:tasks.workunit.client.0.smithi107.stdout:4/387: getdents d3/d5/d32/d53 0 2024-04-25T00:18:27.994 INFO:tasks.workunit.client.0.smithi107.stdout:1/309: mkdir d2/d62 0 2024-04-25T00:18:27.994 INFO:tasks.workunit.client.0.smithi107.stdout:1/310: chown d2/dd/c57 10997 1 2024-04-25T00:18:27.996 INFO:tasks.workunit.client.0.smithi107.stdout:1/311: creat d2/dd/d25/f63 x:0 0 0 2024-04-25T00:18:27.997 INFO:tasks.workunit.client.0.smithi107.stdout:1/312: creat d2/dd/d1a/f64 x:0 0 0 2024-04-25T00:18:28.004 INFO:tasks.workunit.client.0.smithi107.stdout:1/313: truncate d2/dd/f21 2608612 0 2024-04-25T00:18:28.007 INFO:tasks.workunit.client.0.smithi107.stdout:4/388: dwrite d3/d10/f1b [0,4194304] 0 2024-04-25T00:18:28.007 INFO:tasks.workunit.client.0.smithi107.stdout:4/389: chown d3/d5/d32/f54 968 1 2024-04-25T00:18:28.008 INFO:tasks.workunit.client.0.smithi107.stdout:4/390: mkdir d3/d5/d32/d53/d7e 0 2024-04-25T00:18:28.010 INFO:tasks.workunit.client.0.smithi107.stdout:4/391: mkdir d3/d24/d25/d71/d7f 0 2024-04-25T00:18:28.013 INFO:tasks.workunit.client.0.smithi107.stdout:1/314: dread d2/d2a/f4a [0,4194304] 0 2024-04-25T00:18:28.019 INFO:tasks.workunit.client.0.smithi107.stdout:4/392: write d3/f35 [2375329,2425] 0 2024-04-25T00:18:28.020 INFO:tasks.workunit.client.0.smithi107.stdout:4/393: link d3/d5/c14 d3/d24/c80 0 2024-04-25T00:18:28.021 INFO:tasks.workunit.client.0.smithi107.stdout:4/394: creat d3/d24/f81 x:0 0 0 2024-04-25T00:18:28.022 INFO:tasks.workunit.client.0.smithi107.stdout:4/395: mkdir d3/d5/d32/d53/d82 0 2024-04-25T00:18:28.023 INFO:tasks.workunit.client.0.smithi107.stdout:1/315: dread f1 [0,4194304] 0 2024-04-25T00:18:28.025 INFO:tasks.workunit.client.0.smithi107.stdout:1/316: symlink d2/l65 0 2024-04-25T00:18:28.025 INFO:tasks.workunit.client.0.smithi107.stdout:1/317: getdents d2/d2a/d37 0 2024-04-25T00:18:28.027 INFO:tasks.workunit.client.0.smithi107.stdout:1/318: mkdir d2/d62/d66 0 2024-04-25T00:18:28.027 INFO:tasks.workunit.client.0.smithi107.stdout:1/319: write d2/dd/d25/d5e/f58 [649341,36442] 0 2024-04-25T00:18:28.028 INFO:tasks.workunit.client.0.smithi107.stdout:1/320: mknod d2/da/d2f/c67 0 2024-04-25T00:18:28.028 INFO:tasks.workunit.client.0.smithi107.stdout:1/321: read d2/dd/d25/f33 [2857386,10660] 0 2024-04-25T00:18:28.041 INFO:tasks.workunit.client.0.smithi107.stdout:1/322: dread d2/dd/d1a/f27 [0,4194304] 0 2024-04-25T00:18:28.041 INFO:tasks.workunit.client.0.smithi107.stdout:4/396: dwrite d3/d5/f3d [0,4194304] 0 2024-04-25T00:18:28.041 INFO:tasks.workunit.client.0.smithi107.stdout:4/397: fsync d3/d5/f3d 0 2024-04-25T00:18:28.042 INFO:tasks.workunit.client.0.smithi107.stdout:1/323: creat d2/f68 x:0 0 0 2024-04-25T00:18:28.043 INFO:tasks.workunit.client.0.smithi107.stdout:1/324: creat d2/f69 x:0 0 0 2024-04-25T00:18:28.044 INFO:tasks.workunit.client.0.smithi107.stdout:1/325: creat d2/dd/d1b/f6a x:0 0 0 2024-04-25T00:18:28.050 INFO:tasks.workunit.client.0.smithi107.stdout:4/398: write d3/f18 [428628,123884] 0 2024-04-25T00:18:28.054 INFO:tasks.workunit.client.0.smithi107.stdout:1/326: write d2/dd/d1a/f1d [2765308,5524] 0 2024-04-25T00:18:28.054 INFO:tasks.workunit.client.0.smithi107.stdout:1/327: readlink d2/d2a/l60 0 2024-04-25T00:18:28.057 INFO:tasks.workunit.client.0.smithi107.stdout:1/328: truncate f1 2802244 0 2024-04-25T00:18:28.058 INFO:tasks.workunit.client.0.smithi107.stdout:1/329: write d2/f51 [240386,87076] 0 2024-04-25T00:18:28.058 INFO:tasks.workunit.client.0.smithi107.stdout:1/330: truncate d2/dd/d25/f2e 4690087 0 2024-04-25T00:18:28.058 INFO:tasks.workunit.client.0.smithi107.stdout:1/331: chown d2/dd/d25/d5e/c44 744147444 1 2024-04-25T00:18:28.063 INFO:tasks.workunit.client.0.smithi107.stdout:7/221: sync 2024-04-25T00:18:28.064 INFO:tasks.workunit.client.0.smithi107.stdout:4/399: write d3/d24/d25/f43 [2224220,18265] 0 2024-04-25T00:18:28.067 INFO:tasks.workunit.client.0.smithi107.stdout:4/400: rename d3/d5/d32/d53/d82 to d3/d24/d25/d71/d83 0 2024-04-25T00:18:28.067 INFO:tasks.workunit.client.0.smithi107.stdout:4/401: chown d3/c1a 83 1 2024-04-25T00:18:28.067 INFO:tasks.workunit.client.0.smithi107.stdout:4/402: fdatasync d3/d5/d32/f45 0 2024-04-25T00:18:28.075 INFO:tasks.workunit.client.0.smithi107.stdout:4/403: write d3/f21 [2026994,117114] 0 2024-04-25T00:18:28.075 INFO:tasks.workunit.client.0.smithi107.stdout:4/404: fdatasync d3/d5/f5a 0 2024-04-25T00:18:28.080 INFO:tasks.workunit.client.0.smithi107.stdout:4/405: write d3/d24/d25/f26 [686555,11601] 0 2024-04-25T00:18:28.081 INFO:tasks.workunit.client.0.smithi107.stdout:4/406: mknod d3/d5/c84 0 2024-04-25T00:18:28.082 INFO:tasks.workunit.client.0.smithi107.stdout:4/407: truncate d3/d5/f65 672277 0 2024-04-25T00:18:28.082 INFO:tasks.workunit.client.0.smithi107.stdout:4/408: symlink d3/d24/l85 0 2024-04-25T00:18:28.082 INFO:tasks.workunit.client.0.smithi107.stdout:4/409: rename d3/f21 to d3/d5/d32/d53/d7e/f86 0 2024-04-25T00:18:28.083 INFO:tasks.workunit.client.0.smithi107.stdout:1/332: dwrite d2/dd/d1a/f1d [4194304,4194304] 0 2024-04-25T00:18:28.084 INFO:tasks.workunit.client.0.smithi107.stdout:1/333: unlink d2/dd/d1b/f6a 0 2024-04-25T00:18:28.084 INFO:tasks.workunit.client.0.smithi107.stdout:1/334: mknod d2/dd/d25/d5e/c6b 0 2024-04-25T00:18:28.085 INFO:tasks.workunit.client.0.smithi107.stdout:1/335: symlink d2/dd/l6c 0 2024-04-25T00:18:28.086 INFO:tasks.workunit.client.0.smithi107.stdout:1/336: creat d2/dd/d1a/f6d x:0 0 0 2024-04-25T00:18:28.086 INFO:tasks.workunit.client.0.smithi107.stdout:1/337: chown d2/f51 518103002 1 2024-04-25T00:18:28.087 INFO:tasks.workunit.client.0.smithi107.stdout:1/338: creat d2/d62/d66/f6e x:0 0 0 2024-04-25T00:18:28.091 INFO:tasks.workunit.client.0.smithi107.stdout:4/410: dread d3/d24/f69 [0,4194304] 0 2024-04-25T00:18:28.092 INFO:tasks.workunit.client.0.smithi107.stdout:4/411: chown d3/d24/d25/d71/f75 2713 1 2024-04-25T00:18:28.092 INFO:tasks.workunit.client.0.smithi107.stdout:4/412: chown f0 69 1 2024-04-25T00:18:28.092 INFO:tasks.workunit.client.0.smithi107.stdout:4/413: truncate d3/d5/d32/d53/d7e/f86 94943 0 2024-04-25T00:18:28.093 INFO:tasks.workunit.client.0.smithi107.stdout:4/414: symlink d3/d5/l87 0 2024-04-25T00:18:28.099 INFO:tasks.workunit.client.0.smithi107.stdout:4/415: dread d3/f35 [0,4194304] 0 2024-04-25T00:18:28.099 INFO:tasks.workunit.client.0.smithi107.stdout:4/416: write d3/d5/f23 [686212,5178] 0 2024-04-25T00:18:28.114 INFO:tasks.workunit.client.0.smithi107.stdout:4/417: dwrite d3/d5/d32/f45 [0,4194304] 0 2024-04-25T00:18:28.116 INFO:tasks.workunit.client.0.smithi107.stdout:4/418: getdents d3/d24 0 2024-04-25T00:18:28.116 INFO:tasks.workunit.client.0.smithi107.stdout:4/419: chown d3/d24/d5c 1 1 2024-04-25T00:18:28.117 INFO:tasks.workunit.client.0.smithi107.stdout:4/420: creat d3/d5/d32/f88 x:0 0 0 2024-04-25T00:18:28.117 INFO:tasks.workunit.client.0.smithi107.stdout:4/421: dread - d3/d5/f4d zero size 2024-04-25T00:18:28.123 INFO:tasks.workunit.client.0.smithi107.stdout:4/422: dread d3/d24/d25/f49 [0,4194304] 0 2024-04-25T00:18:28.124 INFO:tasks.workunit.client.0.smithi107.stdout:4/423: mknod d3/d24/d5c/d74/c89 0 2024-04-25T00:18:28.139 INFO:tasks.workunit.client.0.smithi107.stdout:4/424: dwrite d3/d5/d32/f54 [0,4194304] 0 2024-04-25T00:18:28.139 INFO:tasks.workunit.client.0.smithi107.stdout:4/425: fsync d3/d24/d61/f70 0 2024-04-25T00:18:28.146 INFO:tasks.workunit.client.0.smithi107.stdout:4/426: write d3/d24/d5c/f6f [1148278,17159] 0 2024-04-25T00:18:28.147 INFO:tasks.workunit.client.0.smithi107.stdout:4/427: symlink d3/d24/d25/l8a 0 2024-04-25T00:18:28.148 INFO:tasks.workunit.client.0.smithi107.stdout:4/428: creat d3/d10/f8b x:0 0 0 2024-04-25T00:18:28.149 INFO:tasks.workunit.client.0.smithi107.stdout:4/429: link d3/d5/l3f d3/d24/d25/l8c 0 2024-04-25T00:18:28.169 INFO:tasks.workunit.client.0.smithi107.stdout:4/430: dwrite d3/d24/d5c/f6f [0,4194304] 0 2024-04-25T00:18:28.184 INFO:tasks.workunit.client.0.smithi107.stdout:4/431: dwrite f0 [4194304,4194304] 0 2024-04-25T00:18:28.188 INFO:tasks.workunit.client.0.smithi107.stdout:4/432: creat d3/d24/d5c/d74/f8d x:0 0 0 2024-04-25T00:18:28.189 INFO:tasks.workunit.client.0.smithi107.stdout:4/433: symlink d3/d24/d25/d67/l8e 0 2024-04-25T00:18:28.205 INFO:tasks.workunit.client.0.smithi107.stdout:4/434: dwrite d3/d5/d32/f45 [0,4194304] 0 2024-04-25T00:18:28.205 INFO:tasks.workunit.client.0.smithi107.stdout:4/435: readlink d3/d5/d32/d53/l63 0 2024-04-25T00:18:28.206 INFO:tasks.workunit.client.0.smithi107.stdout:4/436: creat d3/d5/f8f x:0 0 0 2024-04-25T00:18:28.209 INFO:tasks.workunit.client.0.smithi107.stdout:8/400: sync 2024-04-25T00:18:28.209 INFO:tasks.workunit.client.0.smithi107.stdout:2/339: sync 2024-04-25T00:18:28.209 INFO:tasks.workunit.client.0.smithi107.stdout:8/401: truncate d2/f26 1274671 0 2024-04-25T00:18:28.209 INFO:tasks.workunit.client.0.smithi107.stdout:6/320: sync 2024-04-25T00:18:28.209 INFO:tasks.workunit.client.0.smithi107.stdout:0/257: sync 2024-04-25T00:18:28.209 INFO:tasks.workunit.client.0.smithi107.stdout:5/409: sync 2024-04-25T00:18:28.209 INFO:tasks.workunit.client.0.smithi107.stdout:5/410: read - d4/dc/d26/d29/d51/f77 zero size 2024-04-25T00:18:28.209 INFO:tasks.workunit.client.0.smithi107.stdout:6/321: write d1/d2/db/d32/d36/d3d/f60 [935990,11872] 0 2024-04-25T00:18:28.209 INFO:tasks.workunit.client.0.smithi107.stdout:0/258: write dc/f49 [768380,25685] 0 2024-04-25T00:18:28.209 INFO:tasks.workunit.client.0.smithi107.stdout:8/402: write d2/d18/d1a/d33/f34 [1292404,57128] 0 2024-04-25T00:18:28.211 INFO:tasks.workunit.client.0.smithi107.stdout:5/411: mknod d4/d35/d61/d84/c8d 0 2024-04-25T00:18:28.211 INFO:tasks.workunit.client.0.smithi107.stdout:5/412: write d4/dc/d26/f4b [780912,99763] 0 2024-04-25T00:18:28.211 INFO:tasks.workunit.client.0.smithi107.stdout:0/259: symlink dc/dd/d16/d1a/l4f 0 2024-04-25T00:18:28.214 INFO:tasks.workunit.client.0.smithi107.stdout:6/322: mkdir d1/d2/d3/d71 0 2024-04-25T00:18:28.214 INFO:tasks.workunit.client.0.smithi107.stdout:0/260: rmdir dc/dd/d16/d33/d35 39 2024-04-25T00:18:28.216 INFO:tasks.workunit.client.0.smithi107.stdout:0/261: creat dc/f50 x:0 0 0 2024-04-25T00:18:28.216 INFO:tasks.workunit.client.0.smithi107.stdout:0/262: fsync f1 0 2024-04-25T00:18:28.218 INFO:tasks.workunit.client.0.smithi107.stdout:0/263: mknod dc/dd/d16/d48/c51 0 2024-04-25T00:18:28.226 INFO:tasks.workunit.client.0.smithi107.stdout:8/403: write d2/f45 [354872,78041] 0 2024-04-25T00:18:28.226 INFO:tasks.workunit.client.0.smithi107.stdout:8/404: read - d2/d18/d1a/d82/f8b zero size 2024-04-25T00:18:28.228 INFO:tasks.workunit.client.0.smithi107.stdout:4/437: dwrite d3/d24/f81 [0,4194304] 0 2024-04-25T00:18:28.228 INFO:tasks.workunit.client.0.smithi107.stdout:8/405: creat d2/d18/d1a/d5e/f8c x:0 0 0 2024-04-25T00:18:28.228 INFO:tasks.workunit.client.0.smithi107.stdout:8/406: chown d2/d18/d1a/d5e/d3d/d85 49172733 1 2024-04-25T00:18:28.230 INFO:tasks.workunit.client.0.smithi107.stdout:4/438: symlink d3/d5/l90 0 2024-04-25T00:18:28.231 INFO:tasks.workunit.client.0.smithi107.stdout:4/439: mknod d3/d24/d5c/d7a/c91 0 2024-04-25T00:18:28.234 INFO:tasks.workunit.client.0.smithi107.stdout:4/440: creat d3/d5/f92 x:0 0 0 2024-04-25T00:18:28.234 INFO:tasks.workunit.client.0.smithi107.stdout:4/441: fsync d3/d24/d5c/f6f 0 2024-04-25T00:18:28.234 INFO:tasks.workunit.client.0.smithi107.stdout:2/340: dwrite d6/d18/d4d/d67/f49 [0,4194304] 0 2024-04-25T00:18:28.235 INFO:tasks.workunit.client.0.smithi107.stdout:4/442: mknod d3/c93 0 2024-04-25T00:18:28.237 INFO:tasks.workunit.client.0.smithi107.stdout:2/341: unlink d6/d29/d4e/l6c 0 2024-04-25T00:18:28.237 INFO:tasks.workunit.client.0.smithi107.stdout:2/342: chown d6/f8 0 1 2024-04-25T00:18:28.237 INFO:tasks.workunit.client.0.smithi107.stdout:2/343: dread - d6/d18/d40/f55 zero size 2024-04-25T00:18:28.238 INFO:tasks.workunit.client.0.smithi107.stdout:5/413: dwrite d4/dc/d26/f4b [0,4194304] 0 2024-04-25T00:18:28.239 INFO:tasks.workunit.client.0.smithi107.stdout:5/414: chown d4/dc/d26/d42/f76 599 1 2024-04-25T00:18:28.239 INFO:tasks.workunit.client.0.smithi107.stdout:5/415: write d4/dc/f7f [1034116,111809] 0 2024-04-25T00:18:28.239 INFO:tasks.workunit.client.0.smithi107.stdout:2/344: creat d6/d18/f78 x:0 0 0 2024-04-25T00:18:28.240 INFO:tasks.workunit.client.0.smithi107.stdout:2/345: write d6/d18/d40/d42/f62 [168324,122097] 0 2024-04-25T00:18:28.241 INFO:tasks.workunit.client.0.smithi107.stdout:8/407: write d2/f8 [4267,112267] 0 2024-04-25T00:18:28.241 INFO:tasks.workunit.client.0.smithi107.stdout:2/346: symlink d6/d18/d40/d42/d69/d6e/l79 0 2024-04-25T00:18:28.242 INFO:tasks.workunit.client.0.smithi107.stdout:2/347: readlink d6/l12 0 2024-04-25T00:18:28.243 INFO:tasks.workunit.client.0.smithi107.stdout:2/348: creat d6/d18/d40/d42/d2b/f7a x:0 0 0 2024-04-25T00:18:28.243 INFO:tasks.workunit.client.0.smithi107.stdout:2/349: readlink d6/d20/d22/l5c 0 2024-04-25T00:18:28.244 INFO:tasks.workunit.client.0.smithi107.stdout:0/264: dwrite dc/dd/d16/d33/f37 [0,4194304] 0 2024-04-25T00:18:28.245 INFO:tasks.workunit.client.0.smithi107.stdout:2/350: mknod d6/d29/c7b 0 2024-04-25T00:18:28.260 INFO:tasks.workunit.client.0.smithi107.stdout:0/265: dwrite dc/dd/f15 [0,4194304] 0 2024-04-25T00:18:28.261 INFO:tasks.workunit.client.0.smithi107.stdout:0/266: rmdir dc 39 2024-04-25T00:18:28.262 INFO:tasks.workunit.client.0.smithi107.stdout:4/443: rename d3/d24/d25/l8c to d3/d24/d5c/d74/l94 0 2024-04-25T00:18:28.262 INFO:tasks.workunit.client.0.smithi107.stdout:4/444: fsync d3/d24/d5c/f6f 0 2024-04-25T00:18:28.263 INFO:tasks.workunit.client.0.smithi107.stdout:4/445: creat d3/d24/f95 x:0 0 0 2024-04-25T00:18:28.264 INFO:tasks.workunit.client.0.smithi107.stdout:5/416: rename d4/dc/d26/d29/d80 to d4/d7c/d88/d8e 0 2024-04-25T00:18:28.264 INFO:tasks.workunit.client.0.smithi107.stdout:4/446: mkdir d3/d24/d25/d96 0 2024-04-25T00:18:28.265 INFO:tasks.workunit.client.0.smithi107.stdout:4/447: mkdir d3/d5/d32/d53/d97 0 2024-04-25T00:18:28.265 INFO:tasks.workunit.client.0.smithi107.stdout:4/448: write d3/d24/d61/f70 [440528,95044] 0 2024-04-25T00:18:28.265 INFO:tasks.workunit.client.0.smithi107.stdout:4/449: chown d3/d5/f3d 3864787 1 2024-04-25T00:18:28.266 INFO:tasks.workunit.client.0.smithi107.stdout:8/408: rename d2/d18/d1a/d5e/d3e/l51 to d2/d18/d1a/l8d 0 2024-04-25T00:18:28.267 INFO:tasks.workunit.client.0.smithi107.stdout:4/450: rmdir d3/d24/d25/d71/d83 0 2024-04-25T00:18:28.267 INFO:tasks.workunit.client.0.smithi107.stdout:4/451: chown d3/d24/d25/d71 36 1 2024-04-25T00:18:28.267 INFO:tasks.workunit.client.0.smithi107.stdout:4/452: chown d3/d24/d25/d71 49281326 1 2024-04-25T00:18:28.268 INFO:tasks.workunit.client.0.smithi107.stdout:8/409: creat d2/d18/d1a/f8e x:0 0 0 2024-04-25T00:18:28.268 INFO:tasks.workunit.client.0.smithi107.stdout:8/410: readlink d2/l17 0 2024-04-25T00:18:28.269 INFO:tasks.workunit.client.0.smithi107.stdout:2/351: rename d6/d18/d40/f55 to d6/d18/d40/d42/d2b/d30/f7c 0 2024-04-25T00:18:28.270 INFO:tasks.workunit.client.0.smithi107.stdout:8/411: getdents d2/d18/d1a/d82 0 2024-04-25T00:18:28.270 INFO:tasks.workunit.client.0.smithi107.stdout:8/412: chown d2/d18/d1a/d5e/d5c/f88 55 1 2024-04-25T00:18:28.272 INFO:tasks.workunit.client.0.smithi107.stdout:0/267: rename l9 to dc/dd/l52 0 2024-04-25T00:18:28.273 INFO:tasks.workunit.client.0.smithi107.stdout:4/453: creat d3/d24/d5c/f98 x:0 0 0 2024-04-25T00:18:28.273 INFO:tasks.workunit.client.0.smithi107.stdout:4/454: stat d3/d5/l76 0 2024-04-25T00:18:28.274 INFO:tasks.workunit.client.0.smithi107.stdout:5/417: rename d4/dc/d26/d29/d2e/d82/l72 to d4/dc/d26/d29/d51/l8f 0 2024-04-25T00:18:28.275 INFO:tasks.workunit.client.0.smithi107.stdout:2/352: rename d6 to d6/d18/d40/d42/d69/d7d 22 2024-04-25T00:18:28.275 INFO:tasks.workunit.client.0.smithi107.stdout:4/455: getdents d3 0 2024-04-25T00:18:28.275 INFO:tasks.workunit.client.0.smithi107.stdout:4/456: chown d3/d24/d25/c2e 0 1 2024-04-25T00:18:28.275 INFO:tasks.workunit.client.0.smithi107.stdout:4/457: getdents d3/d5/d32/d53/d97 0 2024-04-25T00:18:28.275 INFO:tasks.workunit.client.0.smithi107.stdout:2/353: creat d6/d20/d22/d56/f7e x:0 0 0 2024-04-25T00:18:28.277 INFO:tasks.workunit.client.0.smithi107.stdout:2/354: creat d6/d29/d4e/f7f x:0 0 0 2024-04-25T00:18:28.278 INFO:tasks.workunit.client.0.smithi107.stdout:2/355: mkdir d6/d18/d40/d42/d69/d6e/d80 0 2024-04-25T00:18:28.279 INFO:tasks.workunit.client.0.smithi107.stdout:4/458: dread d3/d5/d32/f34 [0,4194304] 0 2024-04-25T00:18:28.279 INFO:tasks.workunit.client.0.smithi107.stdout:4/459: write d3/d24/d25/f43 [3841714,22068] 0 2024-04-25T00:18:28.279 INFO:tasks.workunit.client.0.smithi107.stdout:2/356: mknod d6/d29/d4e/d6f/c81 0 2024-04-25T00:18:28.281 INFO:tasks.workunit.client.0.smithi107.stdout:2/357: rmdir d6/d29/d4e 39 2024-04-25T00:18:28.281 INFO:tasks.workunit.client.0.smithi107.stdout:4/460: creat d3/d5/d32/d53/d97/f99 x:0 0 0 2024-04-25T00:18:28.282 INFO:tasks.workunit.client.0.smithi107.stdout:4/461: truncate d3/d5/d32/f45 2828299 0 2024-04-25T00:18:28.283 INFO:tasks.workunit.client.0.smithi107.stdout:2/358: mkdir d6/d18/d40/d42/d69/d82 0 2024-04-25T00:18:28.284 INFO:tasks.workunit.client.0.smithi107.stdout:4/462: rmdir d3/d24/d5c/d7a 39 2024-04-25T00:18:28.284 INFO:tasks.workunit.client.0.smithi107.stdout:4/463: write d3/d5/f5a [2337929,31808] 0 2024-04-25T00:18:28.288 INFO:tasks.workunit.client.0.smithi107.stdout:8/413: dwrite d2/d18/d1a/f31 [0,4194304] 0 2024-04-25T00:18:28.290 INFO:tasks.workunit.client.0.smithi107.stdout:8/414: rmdir d2/d18/d1a/d5e/d3e 39 2024-04-25T00:18:28.290 INFO:tasks.workunit.client.0.smithi107.stdout:8/415: truncate d2/d18/d1a/f3a 4629980 0 2024-04-25T00:18:28.290 INFO:tasks.workunit.client.0.smithi107.stdout:8/416: write d2/d18/d1a/d5e/d3e/f6a [549948,130833] 0 2024-04-25T00:18:28.290 INFO:tasks.workunit.client.0.smithi107.stdout:2/359: dread d6/d18/d40/d42/d2b/d30/d45/f5f [0,4194304] 0 2024-04-25T00:18:28.291 INFO:tasks.workunit.client.0.smithi107.stdout:8/417: mknod d2/d18/d1a/d5e/d3e/c8f 0 2024-04-25T00:18:28.291 INFO:tasks.workunit.client.0.smithi107.stdout:2/360: mknod d6/d18/d40/d42/d69/d6e/d80/c83 0 2024-04-25T00:18:28.292 INFO:tasks.workunit.client.0.smithi107.stdout:2/361: stat d6/d18/f2e 0 2024-04-25T00:18:28.292 INFO:tasks.workunit.client.0.smithi107.stdout:2/362: chown d6/d20/d22/d56/f68 202 1 2024-04-25T00:18:28.292 INFO:tasks.workunit.client.0.smithi107.stdout:8/418: rename d2/d18/c44 to d2/d18/d1a/d5e/d3e/c90 0 2024-04-25T00:18:28.293 INFO:tasks.workunit.client.0.smithi107.stdout:2/363: symlink d6/d18/d40/l84 0 2024-04-25T00:18:28.294 INFO:tasks.workunit.client.0.smithi107.stdout:8/419: mkdir d2/d18/d1a/d5e/d5c/d91 0 2024-04-25T00:18:28.295 INFO:tasks.workunit.client.0.smithi107.stdout:2/364: creat d6/d18/d4d/f85 x:0 0 0 2024-04-25T00:18:28.296 INFO:tasks.workunit.client.0.smithi107.stdout:8/420: mknod d2/d18/d1a/d5e/d3d/c92 0 2024-04-25T00:18:28.296 INFO:tasks.workunit.client.0.smithi107.stdout:2/365: creat d6/d18/d40/d42/d69/d6e/d80/f86 x:0 0 0 2024-04-25T00:18:28.297 INFO:tasks.workunit.client.0.smithi107.stdout:2/366: dread - d6/f72 zero size 2024-04-25T00:18:28.298 INFO:tasks.workunit.client.0.smithi107.stdout:8/421: unlink d2/d18/d1a/d33/f34 0 2024-04-25T00:18:28.298 INFO:tasks.workunit.client.0.smithi107.stdout:8/422: chown d2/d18/d1a/d5e/d5c/c65 1800653 1 2024-04-25T00:18:28.298 INFO:tasks.workunit.client.0.smithi107.stdout:3/271: sync 2024-04-25T00:18:28.298 INFO:tasks.workunit.client.0.smithi107.stdout:9/288: sync 2024-04-25T00:18:28.300 INFO:tasks.workunit.client.0.smithi107.stdout:3/272: mknod d9/d15/d1a/d4e/c59 0 2024-04-25T00:18:28.300 INFO:tasks.workunit.client.0.smithi107.stdout:3/273: write d9/d2f/f3c [4961075,71650] 0 2024-04-25T00:18:28.300 INFO:tasks.workunit.client.0.smithi107.stdout:3/274: readlink d9/d22/d1b/l27 0 2024-04-25T00:18:28.301 INFO:tasks.workunit.client.0.smithi107.stdout:2/367: rename d6/d18/d40/d42/d69/d6e/d80/c83 to d6/d18/c87 0 2024-04-25T00:18:28.301 INFO:tasks.workunit.client.0.smithi107.stdout:2/368: write d6/f51 [3241435,80291] 0 2024-04-25T00:18:28.301 INFO:tasks.workunit.client.0.smithi107.stdout:2/369: chown d6/d18/d40/d42/d69/d82 8 1 2024-04-25T00:18:28.301 INFO:tasks.workunit.client.0.smithi107.stdout:2/370: chown d6/d18/d40/d42/d2b/c74 598 1 2024-04-25T00:18:28.301 INFO:tasks.workunit.client.0.smithi107.stdout:3/275: truncate d9/d15/d1a/f32 1445306 0 2024-04-25T00:18:28.302 INFO:tasks.workunit.client.0.smithi107.stdout:4/464: dwrite d3/d5/f41 [0,4194304] 0 2024-04-25T00:18:28.303 INFO:tasks.workunit.client.0.smithi107.stdout:2/371: creat d6/d20/f88 x:0 0 0 2024-04-25T00:18:28.304 INFO:tasks.workunit.client.0.smithi107.stdout:2/372: fdatasync f4 0 2024-04-25T00:18:28.304 INFO:tasks.workunit.client.0.smithi107.stdout:4/465: symlink d3/d5/l9a 0 2024-04-25T00:18:28.304 INFO:tasks.workunit.client.0.smithi107.stdout:4/466: write d3/d5/f8f [122324,120987] 0 2024-04-25T00:18:28.304 INFO:tasks.workunit.client.0.smithi107.stdout:4/467: fsync f0 0 2024-04-25T00:18:28.304 INFO:tasks.workunit.client.0.smithi107.stdout:4/468: stat d3/f35 0 2024-04-25T00:18:28.304 INFO:tasks.workunit.client.0.smithi107.stdout:2/373: getdents d6/d18/d4d/d67 0 2024-04-25T00:18:28.304 INFO:tasks.workunit.client.0.smithi107.stdout:2/374: stat d6/d20/d22/l5c 0 2024-04-25T00:18:28.306 INFO:tasks.workunit.client.0.smithi107.stdout:4/469: rename d3/d5/f5a to d3/d5/d32/f9b 0 2024-04-25T00:18:28.307 INFO:tasks.workunit.client.0.smithi107.stdout:3/276: mknod d9/d15/d1a/d4e/c5a 0 2024-04-25T00:18:28.309 INFO:tasks.workunit.client.0.smithi107.stdout:5/418: dread d4/dc/f2f [0,4194304] 0 2024-04-25T00:18:28.309 INFO:tasks.workunit.client.0.smithi107.stdout:0/268: dwrite dc/f50 [0,4194304] 0 2024-04-25T00:18:28.310 INFO:tasks.workunit.client.0.smithi107.stdout:0/269: write dc/dd/d16/f2a [1011579,116003] 0 2024-04-25T00:18:28.313 INFO:tasks.workunit.client.0.smithi107.stdout:5/419: mknod d4/dc/d26/d29/c90 0 2024-04-25T00:18:28.317 INFO:tasks.workunit.client.0.smithi107.stdout:5/420: mkdir d4/d91 0 2024-04-25T00:18:28.320 INFO:tasks.workunit.client.0.smithi107.stdout:0/270: write dc/dd/d16/f1b [3626921,35295] 0 2024-04-25T00:18:28.321 INFO:tasks.workunit.client.0.smithi107.stdout:0/271: write dc/d24/f3a [1585991,65249] 0 2024-04-25T00:18:28.322 INFO:tasks.workunit.client.0.smithi107.stdout:0/272: symlink dc/d24/d25/d31/l53 0 2024-04-25T00:18:28.322 INFO:tasks.workunit.client.0.smithi107.stdout:0/273: write dc/d24/d25/d31/f36 [793319,125503] 0 2024-04-25T00:18:28.323 INFO:tasks.workunit.client.0.smithi107.stdout:0/274: creat dc/dd/d16/d48/f54 x:0 0 0 2024-04-25T00:18:28.326 INFO:tasks.workunit.client.0.smithi107.stdout:4/470: dread d3/d5/f7 [0,4194304] 0 2024-04-25T00:18:28.326 INFO:tasks.workunit.client.0.smithi107.stdout:4/471: dread - d3/d5/d32/f88 zero size 2024-04-25T00:18:28.326 INFO:tasks.workunit.client.0.smithi107.stdout:4/472: truncate d3/d5/d32/f88 5020 0 2024-04-25T00:18:28.328 INFO:tasks.workunit.client.0.smithi107.stdout:4/473: link d3/d5/f23 d3/d24/d25/d71/d7f/f9c 0 2024-04-25T00:18:28.329 INFO:tasks.workunit.client.0.smithi107.stdout:4/474: getdents d3/d5/d32 0 2024-04-25T00:18:28.329 INFO:tasks.workunit.client.0.smithi107.stdout:4/475: dread - d3/d24/d25/d71/f7c zero size 2024-04-25T00:18:28.329 INFO:tasks.workunit.client.0.smithi107.stdout:4/476: truncate d3/d5/f4d 1034788 0 2024-04-25T00:18:28.332 INFO:tasks.workunit.client.0.smithi107.stdout:4/477: unlink d3/d5/l87 0 2024-04-25T00:18:28.334 INFO:tasks.workunit.client.0.smithi107.stdout:4/478: mkdir d3/d5/d32/d53/d7e/d9d 0 2024-04-25T00:18:28.336 INFO:tasks.workunit.client.0.smithi107.stdout:4/479: mkdir d3/d5/d32/d9e 0 2024-04-25T00:18:28.336 INFO:tasks.workunit.client.0.smithi107.stdout:3/277: dread f0 [0,4194304] 0 2024-04-25T00:18:28.336 INFO:tasks.workunit.client.0.smithi107.stdout:3/278: truncate f0 4715432 0 2024-04-25T00:18:28.337 INFO:tasks.workunit.client.0.smithi107.stdout:3/279: fdatasync d9/d22/f18 0 2024-04-25T00:18:28.337 INFO:tasks.workunit.client.0.smithi107.stdout:3/280: truncate d9/d15/d39/f3d 889743 0 2024-04-25T00:18:28.337 INFO:tasks.workunit.client.0.smithi107.stdout:8/423: dwrite d2/f45 [0,4194304] 0 2024-04-25T00:18:28.337 INFO:tasks.workunit.client.0.smithi107.stdout:8/424: stat d2/c15 0 2024-04-25T00:18:28.338 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:28 smithi195 ceph-mon[32833]: pgmap v63: 129 pgs: 129 active+clean; 333 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 37 MiB/s wr, 442 op/s 2024-04-25T00:18:28.342 INFO:tasks.workunit.client.0.smithi107.stdout:9/289: dwrite d0/d8/d56/f64 [0,4194304] 0 2024-04-25T00:18:28.343 INFO:tasks.workunit.client.0.smithi107.stdout:2/375: dwrite d6/f10 [0,4194304] 0 2024-04-25T00:18:28.343 INFO:tasks.workunit.client.0.smithi107.stdout:5/421: dwrite d4/dc/d26/f4b [4194304,4194304] 0 2024-04-25T00:18:28.343 INFO:tasks.workunit.client.0.smithi107.stdout:5/422: stat d4/dc/d26/d29/d51 0 2024-04-25T00:18:28.343 INFO:tasks.workunit.client.0.smithi107.stdout:5/423: dread - d4/d17/f79 zero size 2024-04-25T00:18:28.344 INFO:tasks.workunit.client.0.smithi107.stdout:5/424: write d4/d35/f3c [348241,101150] 0 2024-04-25T00:18:28.345 INFO:tasks.workunit.client.0.smithi107.stdout:9/290: write d0/d8/d32/f5b [621506,19560] 0 2024-04-25T00:18:28.345 INFO:tasks.workunit.client.0.smithi107.stdout:2/376: creat d6/d20/f89 x:0 0 0 2024-04-25T00:18:28.346 INFO:tasks.workunit.client.0.smithi107.stdout:0/275: dwrite dc/dd/f4c [0,4194304] 0 2024-04-25T00:18:28.347 INFO:tasks.workunit.client.0.smithi107.stdout:5/425: creat d4/d7c/d88/f92 x:0 0 0 2024-04-25T00:18:28.348 INFO:tasks.workunit.client.0.smithi107.stdout:9/291: rename d0/d43/l1a to d0/d8/d34/d36/l66 0 2024-04-25T00:18:28.350 INFO:tasks.workunit.client.0.smithi107.stdout:8/425: write d2/d18/d1a/f31 [1180728,71764] 0 2024-04-25T00:18:28.356 INFO:tasks.workunit.client.0.smithi107.stdout:2/377: dread d6/d20/d22/d56/f68 [0,4194304] 0 2024-04-25T00:18:28.356 INFO:tasks.workunit.client.0.smithi107.stdout:2/378: dread - d6/f72 zero size 2024-04-25T00:18:28.356 INFO:tasks.workunit.client.0.smithi107.stdout:5/426: mkdir d4/d35/d61/d84/d93 0 2024-04-25T00:18:28.357 INFO:tasks.workunit.client.0.smithi107.stdout:0/276: dread dc/f49 [0,4194304] 0 2024-04-25T00:18:28.357 INFO:tasks.workunit.client.0.smithi107.stdout:8/426: mknod d2/d18/d1a/d5e/d6d/c93 0 2024-04-25T00:18:28.360 INFO:tasks.workunit.client.0.smithi107.stdout:0/277: symlink dc/l55 0 2024-04-25T00:18:28.360 INFO:tasks.workunit.client.0.smithi107.stdout:0/278: readlink dc/d24/d25/d31/l53 0 2024-04-25T00:18:28.360 INFO:tasks.workunit.client.0.smithi107.stdout:0/279: readlink dc/dd/d16/l4d 0 2024-04-25T00:18:28.362 INFO:tasks.workunit.client.0.smithi107.stdout:8/427: rename d2/d18/d1a/d5e/f39 to d2/d18/d1a/d5e/d3e/d74/f94 0 2024-04-25T00:18:28.363 INFO:tasks.workunit.client.0.smithi107.stdout:3/281: dread d9/d2f/f40 [0,4194304] 0 2024-04-25T00:18:28.363 INFO:tasks.workunit.client.0.smithi107.stdout:3/282: read - d9/d2f/f46 zero size 2024-04-25T00:18:28.363 INFO:tasks.workunit.client.0.smithi107.stdout:4/480: dwrite f0 [8388608,4194304] 0 2024-04-25T00:18:28.363 INFO:tasks.workunit.client.0.smithi107.stdout:4/481: write d3/f18 [941427,45847] 0 2024-04-25T00:18:28.365 INFO:tasks.workunit.client.0.smithi107.stdout:0/280: mknod dc/d24/d25/d31/c56 0 2024-04-25T00:18:28.369 INFO:tasks.workunit.client.0.smithi107.stdout:1/339: sync 2024-04-25T00:18:28.369 INFO:tasks.workunit.client.0.smithi107.stdout:7/222: sync 2024-04-25T00:18:28.371 INFO:tasks.workunit.client.0.smithi107.stdout:7/223: mkdir d0/d8/d52 0 2024-04-25T00:18:28.371 INFO:tasks.workunit.client.0.smithi107.stdout:7/224: fsync d0/d8/db/f49 0 2024-04-25T00:18:28.372 INFO:tasks.workunit.client.0.smithi107.stdout:7/225: creat d0/d8/db/d18/d19/d47/f53 x:0 0 0 2024-04-25T00:18:28.373 INFO:tasks.workunit.client.0.smithi107.stdout:6/323: sync 2024-04-25T00:18:28.375 INFO:tasks.workunit.client.0.smithi107.stdout:0/281: dread dc/f49 [0,4194304] 0 2024-04-25T00:18:28.375 INFO:tasks.workunit.client.0.smithi107.stdout:0/282: readlink dc/l2e 0 2024-04-25T00:18:28.375 INFO:tasks.workunit.client.0.smithi107.stdout:0/283: truncate dc/f49 1285662 0 2024-04-25T00:18:28.375 INFO:tasks.workunit.client.0.smithi107.stdout:0/284: readlink lb 0 2024-04-25T00:18:28.375 INFO:tasks.workunit.client.0.smithi107.stdout:6/324: rename d1/d2/d2b/f27 to d1/d2/db/d3c/f72 0 2024-04-25T00:18:28.375 INFO:tasks.workunit.client.0.smithi107.stdout:6/325: write d1/d14/d18/f2c [2908790,77106] 0 2024-04-25T00:18:28.375 INFO:tasks.workunit.client.0.smithi107.stdout:6/326: write d1/d2/db/d3c/f50 [218784,101070] 0 2024-04-25T00:18:28.380 INFO:tasks.workunit.client.0.smithi107.stdout:4/482: dread d3/f18 [0,4194304] 0 2024-04-25T00:18:28.382 INFO:tasks.workunit.client.0.smithi107.stdout:4/483: creat d3/d5/d32/f9f x:0 0 0 2024-04-25T00:18:28.382 INFO:tasks.workunit.client.0.smithi107.stdout:1/340: dread d2/da/f53 [0,4194304] 0 2024-04-25T00:18:28.383 INFO:tasks.workunit.client.0.smithi107.stdout:6/327: dread d1/d2/d3/f31 [0,4194304] 0 2024-04-25T00:18:28.384 INFO:tasks.workunit.client.0.smithi107.stdout:1/341: creat d2/dd/f6f x:0 0 0 2024-04-25T00:18:28.384 INFO:tasks.workunit.client.0.smithi107.stdout:4/484: rename d3/d10/c51 to d3/d5/d32/d53/d7e/d9d/ca0 0 2024-04-25T00:18:28.384 INFO:tasks.workunit.client.0.smithi107.stdout:4/485: dread - d3/d5/d32/d53/d97/f99 zero size 2024-04-25T00:18:28.385 INFO:tasks.workunit.client.0.smithi107.stdout:6/328: creat d1/d2/db/d32/f73 x:0 0 0 2024-04-25T00:18:28.386 INFO:tasks.workunit.client.0.smithi107.stdout:6/329: symlink d1/d2/db/d32/l74 0 2024-04-25T00:18:28.387 INFO:tasks.workunit.client.0.smithi107.stdout:6/330: truncate d1/d2/db/d3c/f4e 7264201 0 2024-04-25T00:18:28.387 INFO:tasks.workunit.client.0.smithi107.stdout:6/331: dread - d1/d2/db/d32/d36/d3d/f5e zero size 2024-04-25T00:18:28.390 INFO:tasks.workunit.client.0.smithi107.stdout:6/332: mkdir d1/d2/db/d32/d33/d68/d75 0 2024-04-25T00:18:28.394 INFO:tasks.workunit.client.0.smithi107.stdout:4/486: dread d3/d24/f69 [8388608,4194304] 0 2024-04-25T00:18:28.395 INFO:tasks.workunit.client.0.smithi107.stdout:1/342: read f1 [878563,7616] 0 2024-04-25T00:18:28.396 INFO:tasks.workunit.client.0.smithi107.stdout:1/343: mkdir d2/d62/d66/d70 0 2024-04-25T00:18:28.397 INFO:tasks.workunit.client.0.smithi107.stdout:4/487: getdents d3/d24/d25/d71/d7f 0 2024-04-25T00:18:28.408 INFO:tasks.workunit.client.0.smithi107.stdout:8/428: dwrite d2/f83 [0,4194304] 0 2024-04-25T00:18:28.411 INFO:tasks.workunit.client.0.smithi107.stdout:5/427: dwrite d4/f48 [0,4194304] 0 2024-04-25T00:18:28.411 INFO:tasks.workunit.client.0.smithi107.stdout:5/428: write d4/dc/fd [132349,10550] 0 2024-04-25T00:18:28.411 INFO:tasks.workunit.client.0.smithi107.stdout:8/429: creat d2/d18/d1a/d5e/d5c/f95 x:0 0 0 2024-04-25T00:18:28.412 INFO:tasks.workunit.client.0.smithi107.stdout:6/333: getdents d1/d2/db/d32/d33/d68 0 2024-04-25T00:18:28.412 INFO:tasks.workunit.client.0.smithi107.stdout:1/344: dread d2/dd/d1a/f61 [0,4194304] 0 2024-04-25T00:18:28.413 INFO:tasks.workunit.client.0.smithi107.stdout:7/226: dwrite d0/d8/d2c/f3d [0,4194304] 0 2024-04-25T00:18:28.413 INFO:tasks.workunit.client.0.smithi107.stdout:7/227: chown d0/d8/db/d18/f41 967 1 2024-04-25T00:18:28.413 INFO:tasks.workunit.client.0.smithi107.stdout:1/345: write d2/dd/d25/f2e [5392910,113897] 0 2024-04-25T00:18:28.413 INFO:tasks.workunit.client.0.smithi107.stdout:5/429: mknod d4/d35/d61/c94 0 2024-04-25T00:18:28.413 INFO:tasks.workunit.client.0.smithi107.stdout:5/430: truncate d4/d17/f69 852664 0 2024-04-25T00:18:28.413 INFO:tasks.workunit.client.0.smithi107.stdout:5/431: write d4/dc/d26/d29/d2e/d82/f68 [598894,82118] 0 2024-04-25T00:18:28.414 INFO:tasks.workunit.client.0.smithi107.stdout:8/430: getdents d2/d18/d1a/d5e 0 2024-04-25T00:18:28.415 INFO:tasks.workunit.client.0.smithi107.stdout:4/488: dread d3/d10/f1b [0,4194304] 0 2024-04-25T00:18:28.416 INFO:tasks.workunit.client.0.smithi107.stdout:6/334: rmdir d1/d4c 39 2024-04-25T00:18:28.416 INFO:tasks.workunit.client.0.smithi107.stdout:6/335: fsync d1/d2/db/d48/d56/d6b/f6f 0 2024-04-25T00:18:28.416 INFO:tasks.workunit.client.0.smithi107.stdout:1/346: creat d2/d2a/d5c/f71 x:0 0 0 2024-04-25T00:18:28.416 INFO:tasks.workunit.client.0.smithi107.stdout:6/336: readlink d1/d2/d2b/l1a 0 2024-04-25T00:18:28.416 INFO:tasks.workunit.client.0.smithi107.stdout:1/347: chown d2/dd/d25/f50 17904 1 2024-04-25T00:18:28.421 INFO:tasks.workunit.client.0.smithi107.stdout:5/432: mknod d4/dc/d26/d29/d51/d78/c95 0 2024-04-25T00:18:28.422 INFO:tasks.workunit.client.0.smithi107.stdout:8/431: mknod d2/d18/d1a/d5e/d3d/d85/c96 0 2024-04-25T00:18:28.422 INFO:tasks.workunit.client.0.smithi107.stdout:8/432: stat d2/d18/d1a/l4e 0 2024-04-25T00:18:28.423 INFO:tasks.workunit.client.0.smithi107.stdout:1/348: mknod d2/c72 0 2024-04-25T00:18:28.423 INFO:tasks.workunit.client.0.smithi107.stdout:1/349: write d2/da/f3b [972625,109733] 0 2024-04-25T00:18:28.424 INFO:tasks.workunit.client.0.smithi107.stdout:1/350: chown d2/d2a/f38 1176148404 1 2024-04-25T00:18:28.424 INFO:tasks.workunit.client.0.smithi107.stdout:1/351: dread - d2/d2a/d5c/f71 zero size 2024-04-25T00:18:28.424 INFO:tasks.workunit.client.0.smithi107.stdout:1/352: read - d2/dd/d1b/f52 zero size 2024-04-25T00:18:28.424 INFO:tasks.workunit.client.0.smithi107.stdout:1/353: write d2/f30 [2573919,45466] 0 2024-04-25T00:18:28.424 INFO:tasks.workunit.client.0.smithi107.stdout:5/433: symlink d4/d17/l96 0 2024-04-25T00:18:28.426 INFO:tasks.workunit.client.0.smithi107.stdout:8/433: creat d2/d18/d1a/d5e/d3e/d74/f97 x:0 0 0 2024-04-25T00:18:28.426 INFO:tasks.workunit.client.0.smithi107.stdout:8/434: chown d2/d18/d1a/d33/f66 7327697 1 2024-04-25T00:18:28.430 INFO:tasks.workunit.client.0.smithi107.stdout:8/435: rename d2/d18/d1a/d5e/d5c/l67 to d2/d18/d1a/d5e/d3d/l98 0 2024-04-25T00:18:28.432 INFO:tasks.workunit.client.0.smithi107.stdout:4/489: dread d3/fd [0,4194304] 0 2024-04-25T00:18:28.433 INFO:tasks.workunit.client.0.smithi107.stdout:8/436: link d2/d18/d1a/f3a d2/d18/d1a/d5e/d3e/f99 0 2024-04-25T00:18:28.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:28 smithi170 ceph-mon[32807]: pgmap v63: 129 pgs: 129 active+clean; 333 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 37 MiB/s wr, 442 op/s 2024-04-25T00:18:28.435 INFO:tasks.workunit.client.0.smithi107.stdout:4/490: mknod d3/d24/d5c/d74/ca1 0 2024-04-25T00:18:28.435 INFO:tasks.workunit.client.0.smithi107.stdout:8/437: dread d2/fb [0,4194304] 0 2024-04-25T00:18:28.436 INFO:tasks.workunit.client.0.smithi107.stdout:8/438: rmdir d2/d18/d1a/d5e/d3d 39 2024-04-25T00:18:28.436 INFO:tasks.workunit.client.0.smithi107.stdout:8/439: chown d2/d18/d1a/d5e/d3d/c42 23446612 1 2024-04-25T00:18:28.436 INFO:tasks.workunit.client.0.smithi107.stdout:7/228: dwrite d0/f16 [0,4194304] 0 2024-04-25T00:18:28.437 INFO:tasks.workunit.client.0.smithi107.stdout:8/440: mknod d2/d18/c9a 0 2024-04-25T00:18:28.442 INFO:tasks.workunit.client.0.smithi107.stdout:4/491: dread d3/f18 [0,4194304] 0 2024-04-25T00:18:28.443 INFO:tasks.workunit.client.0.smithi107.stdout:5/434: read d4/dc/fe [3952186,44646] 0 2024-04-25T00:18:28.443 INFO:tasks.workunit.client.0.smithi107.stdout:5/435: write d4/fa [3097347,106184] 0 2024-04-25T00:18:28.443 INFO:tasks.workunit.client.0.smithi107.stdout:4/492: mkdir d3/d5/d32/d9e/da2 0 2024-04-25T00:18:28.444 INFO:tasks.workunit.client.0.smithi107.stdout:5/436: unlink d4/dc/c53 0 2024-04-25T00:18:28.452 INFO:tasks.workunit.client.0.smithi107.stdout:4/493: write d3/d24/f81 [1082472,60536] 0 2024-04-25T00:18:28.454 INFO:tasks.workunit.client.0.smithi107.stdout:4/494: rmdir d3/d24/d25/d96 0 2024-04-25T00:18:28.454 INFO:tasks.workunit.client.0.smithi107.stdout:8/441: dwrite d2/d18/d1a/d5e/d4f/f81 [0,4194304] 0 2024-04-25T00:18:28.457 INFO:tasks.workunit.client.0.smithi107.stdout:4/495: mkdir d3/d24/d5c/d74/da3 0 2024-04-25T00:18:28.458 INFO:tasks.workunit.client.0.smithi107.stdout:4/496: unlink d3/le 0 2024-04-25T00:18:28.459 INFO:tasks.workunit.client.0.smithi107.stdout:4/497: creat d3/d5/d32/d9e/da2/fa4 x:0 0 0 2024-04-25T00:18:28.459 INFO:tasks.workunit.client.0.smithi107.stdout:4/498: stat d3/d24/d5c/d74/f8d 0 2024-04-25T00:18:28.459 INFO:tasks.workunit.client.0.smithi107.stdout:4/499: mkdir d3/d10/d7b/da5 0 2024-04-25T00:18:28.461 INFO:tasks.workunit.client.0.smithi107.stdout:4/500: creat d3/d24/d5c/d74/da3/fa6 x:0 0 0 2024-04-25T00:18:28.462 INFO:tasks.workunit.client.0.smithi107.stdout:4/501: symlink d3/d24/d25/d71/la7 0 2024-04-25T00:18:28.462 INFO:tasks.workunit.client.0.smithi107.stdout:4/502: chown d3/d24/d5c/d74 11353700 1 2024-04-25T00:18:28.462 INFO:tasks.workunit.client.0.smithi107.stdout:4/503: stat d3/d5/d32/f38 0 2024-04-25T00:18:28.462 INFO:tasks.workunit.client.0.smithi107.stdout:4/504: write d3/d5/d32/d9e/da2/fa4 [714303,26358] 0 2024-04-25T00:18:28.471 INFO:tasks.workunit.client.0.smithi107.stdout:5/437: dwrite d4/dc/d26/d29/d51/f57 [0,4194304] 0 2024-04-25T00:18:28.471 INFO:tasks.workunit.client.0.smithi107.stdout:4/505: dread d3/d24/d25/f28 [0,4194304] 0 2024-04-25T00:18:28.472 INFO:tasks.workunit.client.0.smithi107.stdout:5/438: symlink d4/d1c/l97 0 2024-04-25T00:18:28.472 INFO:tasks.workunit.client.0.smithi107.stdout:4/506: mkdir d3/da8 0 2024-04-25T00:18:28.474 INFO:tasks.workunit.client.0.smithi107.stdout:8/442: dwrite d2/d18/d1a/d5e/d3e/f6b [0,4194304] 0 2024-04-25T00:18:28.474 INFO:tasks.workunit.client.0.smithi107.stdout:8/443: fsync d2/d18/d1a/d5e/d3e/f86 0 2024-04-25T00:18:28.474 INFO:tasks.workunit.client.0.smithi107.stdout:8/444: dread - d2/d18/d1a/d5e/d3e/f86 zero size 2024-04-25T00:18:28.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:28 smithi107 ceph-mon[29153]: pgmap v63: 129 pgs: 129 active+clean; 333 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 37 MiB/s wr, 442 op/s 2024-04-25T00:18:28.490 INFO:tasks.workunit.client.0.smithi107.stdout:5/439: dwrite d4/dc/d26/d62/f7a [0,4194304] 0 2024-04-25T00:18:28.491 INFO:tasks.workunit.client.0.smithi107.stdout:4/507: rename d3/d5/f23 to d3/d24/d5c/d74/fa9 0 2024-04-25T00:18:28.493 INFO:tasks.workunit.client.0.smithi107.stdout:4/508: mknod d3/d24/d25/d71/caa 0 2024-04-25T00:18:28.493 INFO:tasks.workunit.client.0.smithi107.stdout:4/509: dread - d3/d24/f95 zero size 2024-04-25T00:18:28.493 INFO:tasks.workunit.client.0.smithi107.stdout:4/510: fdatasync d3/d24/d5c/f98 0 2024-04-25T00:18:28.494 INFO:tasks.workunit.client.0.smithi107.stdout:4/511: unlink d3/l1e 0 2024-04-25T00:18:28.494 INFO:tasks.workunit.client.0.smithi107.stdout:5/440: dread d4/dc/f7f [0,4194304] 0 2024-04-25T00:18:28.494 INFO:tasks.workunit.client.0.smithi107.stdout:4/512: read - d3/d24/f95 zero size 2024-04-25T00:18:28.495 INFO:tasks.workunit.client.0.smithi107.stdout:5/441: readlink d4/dc/d26/d29/l65 0 2024-04-25T00:18:28.495 INFO:tasks.workunit.client.0.smithi107.stdout:5/442: dread - d4/d35/d61/f74 zero size 2024-04-25T00:18:28.495 INFO:tasks.workunit.client.0.smithi107.stdout:5/443: fsync d4/d17/f79 0 2024-04-25T00:18:28.495 INFO:tasks.workunit.client.0.smithi107.stdout:4/513: write d3/d5/d32/f9b [3897073,50959] 0 2024-04-25T00:18:28.495 INFO:tasks.workunit.client.0.smithi107.stdout:4/514: write d3/d10/f8b [479869,47544] 0 2024-04-25T00:18:28.495 INFO:tasks.workunit.client.0.smithi107.stdout:5/444: unlink d4/dc/c39 0 2024-04-25T00:18:28.497 INFO:tasks.workunit.client.0.smithi107.stdout:8/445: dwrite d2/d18/d1a/d5e/d3e/f4c [0,4194304] 0 2024-04-25T00:18:28.497 INFO:tasks.workunit.client.0.smithi107.stdout:4/515: fdatasync d3/d10/f8b 0 2024-04-25T00:18:28.497 INFO:tasks.workunit.client.0.smithi107.stdout:5/445: creat d4/d35/d61/d84/d93/f98 x:0 0 0 2024-04-25T00:18:28.498 INFO:tasks.workunit.client.0.smithi107.stdout:5/446: creat d4/dc/d26/d42/d8b/f99 x:0 0 0 2024-04-25T00:18:28.499 INFO:tasks.workunit.client.0.smithi107.stdout:8/446: mknod d2/d18/d1a/d5e/d5c/c9b 0 2024-04-25T00:18:28.499 INFO:tasks.workunit.client.0.smithi107.stdout:8/447: write d2/d18/d1a/d5e/d4f/f76 [1260384,98309] 0 2024-04-25T00:18:28.499 INFO:tasks.workunit.client.0.smithi107.stdout:4/516: unlink d3/d24/c27 0 2024-04-25T00:18:28.499 INFO:tasks.workunit.client.0.smithi107.stdout:8/448: mknod d2/d18/d1a/d5e/c9c 0 2024-04-25T00:18:28.503 INFO:tasks.workunit.client.0.smithi107.stdout:4/517: rename d3/d24/d5c/d74 to d3/d24/d5c/dab 0 2024-04-25T00:18:28.510 INFO:tasks.workunit.client.0.smithi107.stdout:4/518: write d3/fd [1411209,120420] 0 2024-04-25T00:18:28.515 INFO:tasks.workunit.client.0.smithi107.stdout:4/519: truncate d3/d24/d5c/dab/f7d 271381 0 2024-04-25T00:18:28.515 INFO:tasks.workunit.client.0.smithi107.stdout:4/520: truncate d3/d5/f92 703182 0 2024-04-25T00:18:28.515 INFO:tasks.workunit.client.0.smithi107.stdout:4/521: chown d3/d10/l31 824286897 1 2024-04-25T00:18:28.516 INFO:tasks.workunit.client.0.smithi107.stdout:5/447: dwrite d4/dc/f13 [0,4194304] 0 2024-04-25T00:18:28.522 INFO:tasks.workunit.client.0.smithi107.stdout:4/522: dread d3/d24/d25/f26 [0,4194304] 0 2024-04-25T00:18:28.522 INFO:tasks.workunit.client.0.smithi107.stdout:4/523: stat d3/f18 0 2024-04-25T00:18:28.523 INFO:tasks.workunit.client.0.smithi107.stdout:4/524: mknod d3/d24/d5c/d7a/cac 0 2024-04-25T00:18:28.523 INFO:tasks.workunit.client.0.smithi107.stdout:4/525: chown d3/d5/d32/f88 14126 1 2024-04-25T00:18:28.529 INFO:tasks.workunit.client.0.smithi107.stdout:8/449: dwrite d2/d18/d1a/f7c [0,4194304] 0 2024-04-25T00:18:28.529 INFO:tasks.workunit.client.0.smithi107.stdout:8/450: chown d2/d18/d1a/d5e/d3d/l4b 12 1 2024-04-25T00:18:28.529 INFO:tasks.workunit.client.0.smithi107.stdout:8/451: readlink d2/d18/l49 0 2024-04-25T00:18:28.530 INFO:tasks.workunit.client.0.smithi107.stdout:8/452: creat d2/d18/d1a/d5e/d3d/f9d x:0 0 0 2024-04-25T00:18:28.532 INFO:tasks.workunit.client.0.smithi107.stdout:8/453: unlink d2/d18/d1a/d5e/d3e/d74/f94 0 2024-04-25T00:18:28.537 INFO:tasks.workunit.client.0.smithi107.stdout:5/448: dwrite d4/f16 [0,4194304] 0 2024-04-25T00:18:28.539 INFO:tasks.workunit.client.0.smithi107.stdout:4/526: dwrite f0 [4194304,4194304] 0 2024-04-25T00:18:28.539 INFO:tasks.workunit.client.0.smithi107.stdout:4/527: chown d3/d24/d25/c55 2712455 1 2024-04-25T00:18:28.539 INFO:tasks.workunit.client.0.smithi107.stdout:5/449: creat d4/dc/d26/d29/d2e/d55/f9a x:0 0 0 2024-04-25T00:18:28.542 INFO:tasks.workunit.client.0.smithi107.stdout:8/454: dread d2/f8 [0,4194304] 0 2024-04-25T00:18:28.543 INFO:tasks.workunit.client.0.smithi107.stdout:8/455: truncate d2/d18/d1a/d33/f3f 371810 0 2024-04-25T00:18:28.544 INFO:tasks.workunit.client.0.smithi107.stdout:8/456: creat d2/d18/d1a/d5e/d3d/f9e x:0 0 0 2024-04-25T00:18:28.547 INFO:tasks.workunit.client.0.smithi107.stdout:8/457: rename d2/f11 to d2/d18/d1a/d5e/d3e/d74/f9f 0 2024-04-25T00:18:28.560 INFO:tasks.workunit.client.0.smithi107.stdout:5/450: dwrite d4/dc/f7f [0,4194304] 0 2024-04-25T00:18:28.560 INFO:tasks.workunit.client.0.smithi107.stdout:8/458: dread d2/d18/d1a/d5e/d3e/f4c [0,4194304] 0 2024-04-25T00:18:28.560 INFO:tasks.workunit.client.0.smithi107.stdout:8/459: chown d2/d18/d1a/d5e/d4f/f7e 1117297 1 2024-04-25T00:18:28.560 INFO:tasks.workunit.client.0.smithi107.stdout:8/460: read d2/f8 [3515274,61177] 0 2024-04-25T00:18:28.562 INFO:tasks.workunit.client.0.smithi107.stdout:8/461: mkdir d2/d18/d1a/d5e/d4f/da0 0 2024-04-25T00:18:28.562 INFO:tasks.workunit.client.0.smithi107.stdout:4/528: dwrite d3/d24/d5c/f6a [0,4194304] 0 2024-04-25T00:18:28.563 INFO:tasks.workunit.client.0.smithi107.stdout:8/462: mkdir d2/d18/d1a/d5e/d4f/da0/da1 0 2024-04-25T00:18:28.563 INFO:tasks.workunit.client.0.smithi107.stdout:4/529: write d3/d5/f4d [522714,108037] 0 2024-04-25T00:18:28.565 INFO:tasks.workunit.client.0.smithi107.stdout:5/451: write d4/dc/f1f [340508,121190] 0 2024-04-25T00:18:28.565 INFO:tasks.workunit.client.0.smithi107.stdout:8/463: mkdir d2/d18/da2 0 2024-04-25T00:18:28.565 INFO:tasks.workunit.client.0.smithi107.stdout:4/530: link d3/d5/d32/d53/d7e/d9d/ca0 d3/d5/cad 0 2024-04-25T00:18:28.567 INFO:tasks.workunit.client.0.smithi107.stdout:5/452: dread d4/d35/f3c [0,4194304] 0 2024-04-25T00:18:28.567 INFO:tasks.workunit.client.0.smithi107.stdout:4/531: rename d3/d24/d61/c68 to d3/d5/d32/d9e/cae 0 2024-04-25T00:18:28.567 INFO:tasks.workunit.client.0.smithi107.stdout:4/532: write d3/d24/f95 [833514,3330] 0 2024-04-25T00:18:28.569 INFO:tasks.workunit.client.0.smithi107.stdout:4/533: mkdir d3/d5/d32/d9e/da2/daf 0 2024-04-25T00:18:28.569 INFO:tasks.workunit.client.0.smithi107.stdout:4/534: chown d3/d24/d25/f26 151103 1 2024-04-25T00:18:28.569 INFO:tasks.workunit.client.0.smithi107.stdout:4/535: stat d3/l6d 0 2024-04-25T00:18:28.569 INFO:tasks.workunit.client.0.smithi107.stdout:5/453: creat d4/dc/d26/d29/d51/d78/f9b x:0 0 0 2024-04-25T00:18:28.570 INFO:tasks.workunit.client.0.smithi107.stdout:4/536: symlink d3/d24/d5c/dab/da3/lb0 0 2024-04-25T00:18:28.570 INFO:tasks.workunit.client.0.smithi107.stdout:4/537: fsync d3/d5/f8f 0 2024-04-25T00:18:28.571 INFO:tasks.workunit.client.0.smithi107.stdout:5/454: unlink d4/dc/d26/c50 0 2024-04-25T00:18:28.571 INFO:tasks.workunit.client.0.smithi107.stdout:5/455: truncate d4/d17/f79 751954 0 2024-04-25T00:18:28.571 INFO:tasks.workunit.client.0.smithi107.stdout:5/456: write d4/d17/f85 [15602,45741] 0 2024-04-25T00:18:28.571 INFO:tasks.workunit.client.0.smithi107.stdout:5/457: chown d4/d1c/l97 0 1 2024-04-25T00:18:28.572 INFO:tasks.workunit.client.0.smithi107.stdout:5/458: unlink d4/d35/d61/f74 0 2024-04-25T00:18:28.575 INFO:tasks.workunit.client.0.smithi107.stdout:5/459: mknod d4/dc/d26/d29/d2e/c9c 0 2024-04-25T00:18:28.576 INFO:tasks.workunit.client.0.smithi107.stdout:5/460: mkdir d4/d7c/d88/d9d 0 2024-04-25T00:18:28.577 INFO:tasks.workunit.client.0.smithi107.stdout:5/461: rmdir d4/dc/d26 39 2024-04-25T00:18:28.579 INFO:tasks.workunit.client.0.smithi107.stdout:5/462: creat d4/d1c/f9e x:0 0 0 2024-04-25T00:18:28.579 INFO:tasks.workunit.client.0.smithi107.stdout:5/463: truncate d4/dc/d26/f30 1133528 0 2024-04-25T00:18:28.579 INFO:tasks.workunit.client.0.smithi107.stdout:8/464: dwrite d2/d18/d1a/d33/f3f [0,4194304] 0 2024-04-25T00:18:28.579 INFO:tasks.workunit.client.0.smithi107.stdout:8/465: stat d2/d18/d1a/d5e/d3d/c92 0 2024-04-25T00:18:28.580 INFO:tasks.workunit.client.0.smithi107.stdout:5/464: symlink d4/d7c/d88/l9f 0 2024-04-25T00:18:28.588 INFO:tasks.workunit.client.0.smithi107.stdout:8/466: write d2/d18/d1a/d5e/d3e/f6b [992346,31654] 0 2024-04-25T00:18:28.594 INFO:tasks.workunit.client.0.smithi107.stdout:4/538: dwrite d3/d5/f4d [0,4194304] 0 2024-04-25T00:18:28.604 INFO:tasks.workunit.client.0.smithi107.stdout:5/465: dwrite d4/dc/f1f [0,4194304] 0 2024-04-25T00:18:28.615 INFO:tasks.workunit.client.0.smithi107.stdout:5/466: write d4/dc/fe [5224518,9906] 0 2024-04-25T00:18:28.619 INFO:tasks.workunit.client.0.smithi107.stdout:5/467: creat d4/dc/d26/d29/d2e/d55/fa0 x:0 0 0 2024-04-25T00:18:28.619 INFO:tasks.workunit.client.0.smithi107.stdout:5/468: getdents d4/d7c/d88/d8e 0 2024-04-25T00:18:28.619 INFO:tasks.workunit.client.0.smithi107.stdout:5/469: write d4/d7c/d88/f92 [128425,36675] 0 2024-04-25T00:18:28.620 INFO:tasks.workunit.client.0.smithi107.stdout:4/539: write d3/d24/d5c/f6f [1341021,6545] 0 2024-04-25T00:18:28.620 INFO:tasks.workunit.client.0.smithi107.stdout:5/470: mknod d4/d7c/d88/d9d/ca1 0 2024-04-25T00:18:28.620 INFO:tasks.workunit.client.0.smithi107.stdout:5/471: chown d4/d1c/f28 23134849 1 2024-04-25T00:18:28.621 INFO:tasks.workunit.client.0.smithi107.stdout:4/540: mkdir d3/d24/db1 0 2024-04-25T00:18:28.621 INFO:tasks.workunit.client.0.smithi107.stdout:5/472: mknod d4/dc/d26/d29/d2e/d55/ca2 0 2024-04-25T00:18:28.622 INFO:tasks.workunit.client.0.smithi107.stdout:4/541: mknod d3/d24/d25/cb2 0 2024-04-25T00:18:28.622 INFO:tasks.workunit.client.0.smithi107.stdout:4/542: write d3/d24/d5c/dab/da3/fa6 [961264,93577] 0 2024-04-25T00:18:28.622 INFO:tasks.workunit.client.0.smithi107.stdout:4/543: chown d3/d5/l3c 744366099 1 2024-04-25T00:18:28.622 INFO:tasks.workunit.client.0.smithi107.stdout:4/544: chown d3/d24/d25/f2b 870426 1 2024-04-25T00:18:28.623 INFO:tasks.workunit.client.0.smithi107.stdout:4/545: creat d3/d5/d32/d53/d7e/fb3 x:0 0 0 2024-04-25T00:18:28.623 INFO:tasks.workunit.client.0.smithi107.stdout:4/546: write d3/f35 [2989069,95861] 0 2024-04-25T00:18:28.625 INFO:tasks.workunit.client.0.smithi107.stdout:4/547: symlink d3/da8/lb4 0 2024-04-25T00:18:28.625 INFO:tasks.workunit.client.0.smithi107.stdout:4/548: read d3/d5/f12 [436885,74441] 0 2024-04-25T00:18:28.630 INFO:tasks.workunit.client.0.smithi107.stdout:5/473: dread d4/d17/f21 [0,4194304] 0 2024-04-25T00:18:28.631 INFO:tasks.workunit.client.0.smithi107.stdout:5/474: rmdir d4/dc/d26/d29 39 2024-04-25T00:18:28.631 INFO:tasks.workunit.client.0.smithi107.stdout:5/475: truncate d4/dc/f1a 535156 0 2024-04-25T00:18:28.632 INFO:tasks.workunit.client.0.smithi107.stdout:5/476: symlink d4/la3 0 2024-04-25T00:18:28.633 INFO:tasks.workunit.client.0.smithi107.stdout:5/477: readlink d4/dc/d26/d42/l60 0 2024-04-25T00:18:28.636 INFO:tasks.workunit.client.0.smithi107.stdout:5/478: truncate d4/dc/d26/d29/d51/f57 7752540 0 2024-04-25T00:18:28.636 INFO:tasks.workunit.client.0.smithi107.stdout:5/479: chown d4/dc/d26/d42/f76 30 1 2024-04-25T00:18:28.636 INFO:tasks.workunit.client.0.smithi107.stdout:5/480: chown d4/dc/d26/d29/d2e/l25 706 1 2024-04-25T00:18:28.636 INFO:tasks.workunit.client.0.smithi107.stdout:5/481: truncate d4/dc/d26/d29/d2e/d55/f75 733221 0 2024-04-25T00:18:28.642 INFO:tasks.workunit.client.0.smithi107.stdout:4/549: dwrite d3/d24/f2c [0,4194304] 0 2024-04-25T00:18:28.645 INFO:tasks.workunit.client.0.smithi107.stdout:4/550: link d3/d5/d32/d53/l72 d3/d24/d25/lb5 0 2024-04-25T00:18:28.649 INFO:tasks.workunit.client.0.smithi107.stdout:4/551: dread d3/d24/d61/f70 [0,4194304] 0 2024-04-25T00:18:28.651 INFO:tasks.workunit.client.0.smithi107.stdout:4/552: unlink d3/d24/d5c/dab/fa9 0 2024-04-25T00:18:28.652 INFO:tasks.workunit.client.0.smithi107.stdout:5/482: dwrite d4/dc/f20 [4194304,4194304] 0 2024-04-25T00:18:28.652 INFO:tasks.workunit.client.0.smithi107.stdout:5/483: chown d4/dc/c34 1 1 2024-04-25T00:18:28.653 INFO:tasks.workunit.client.0.smithi107.stdout:4/553: rename d3/d24/d25/lb5 to d3/d5/d32/d53/d97/lb6 0 2024-04-25T00:18:28.653 INFO:tasks.workunit.client.0.smithi107.stdout:5/484: mkdir d4/d35/d61/d84/d93/da4 0 2024-04-25T00:18:28.655 INFO:tasks.workunit.client.0.smithi107.stdout:4/554: rename f0 to d3/da8/fb7 0 2024-04-25T00:18:28.656 INFO:tasks.workunit.client.0.smithi107.stdout:4/555: mknod d3/da8/cb8 0 2024-04-25T00:18:28.657 INFO:tasks.workunit.client.0.smithi107.stdout:4/556: write d3/d5/d32/d53/d97/f99 [769293,46790] 0 2024-04-25T00:18:28.658 INFO:tasks.workunit.client.0.smithi107.stdout:9/292: sync 2024-04-25T00:18:28.658 INFO:tasks.workunit.client.0.smithi107.stdout:2/379: sync 2024-04-25T00:18:28.658 INFO:tasks.workunit.client.0.smithi107.stdout:3/283: sync 2024-04-25T00:18:28.659 INFO:tasks.workunit.client.0.smithi107.stdout:2/380: getdents d6/d18/d4d 0 2024-04-25T00:18:28.659 INFO:tasks.workunit.client.0.smithi107.stdout:2/381: truncate d6/d18/d40/d42/d2b/d30/d45/f5f 727806 0 2024-04-25T00:18:28.660 INFO:tasks.workunit.client.0.smithi107.stdout:3/284: creat d9/d22/d1b/d55/f5b x:0 0 0 2024-04-25T00:18:28.661 INFO:tasks.workunit.client.0.smithi107.stdout:2/382: mknod d6/d20/c8a 0 2024-04-25T00:18:28.662 INFO:tasks.workunit.client.0.smithi107.stdout:2/383: creat d6/d18/d40/d42/d69/d6e/d80/f8b x:0 0 0 2024-04-25T00:18:28.662 INFO:tasks.workunit.client.0.smithi107.stdout:2/384: readlink d6/d18/d40/d42/d69/d6e/l79 0 2024-04-25T00:18:28.669 INFO:tasks.workunit.client.0.smithi107.stdout:3/285: getdents d9/d15/d1a 0 2024-04-25T00:18:28.671 INFO:tasks.workunit.client.0.smithi107.stdout:3/286: mkdir d9/d54/d5c 0 2024-04-25T00:18:28.673 INFO:tasks.workunit.client.0.smithi107.stdout:3/287: creat d9/d22/d1b/d55/f5d x:0 0 0 2024-04-25T00:18:28.680 INFO:tasks.workunit.client.0.smithi107.stdout:4/557: dwrite d3/d5/d32/f9f [0,4194304] 0 2024-04-25T00:18:28.681 INFO:tasks.workunit.client.0.smithi107.stdout:9/293: dwrite d0/d8/d32/f4d [0,4194304] 0 2024-04-25T00:18:28.681 INFO:tasks.workunit.client.0.smithi107.stdout:9/294: chown d0/d8/de/d5c 2826 1 2024-04-25T00:18:28.682 INFO:tasks.workunit.client.0.smithi107.stdout:9/295: symlink d0/d8/d56/l67 0 2024-04-25T00:18:28.683 INFO:tasks.workunit.client.0.smithi107.stdout:4/558: creat d3/da8/fb9 x:0 0 0 2024-04-25T00:18:28.683 INFO:tasks.workunit.client.0.smithi107.stdout:2/385: dwrite d6/f17 [0,4194304] 0 2024-04-25T00:18:28.683 INFO:tasks.workunit.client.0.smithi107.stdout:2/386: chown d6/d18/d40/d42/d69/d6e/d80 201 1 2024-04-25T00:18:28.684 INFO:tasks.workunit.client.0.smithi107.stdout:2/387: mkdir d6/d29/d4e/d8c 0 2024-04-25T00:18:28.684 INFO:tasks.workunit.client.0.smithi107.stdout:4/559: creat d3/d5/d32/d9e/fba x:0 0 0 2024-04-25T00:18:28.686 INFO:tasks.workunit.client.0.smithi107.stdout:2/388: symlink d6/d18/d40/d42/d69/d6e/d80/l8d 0 2024-04-25T00:18:28.687 INFO:tasks.workunit.client.0.smithi107.stdout:2/389: write d6/d18/d40/d42/d2b/d30/f4b [711210,20413] 0 2024-04-25T00:18:28.687 INFO:tasks.workunit.client.0.smithi107.stdout:2/390: write d6/d20/f88 [388320,120732] 0 2024-04-25T00:18:28.688 INFO:tasks.workunit.client.0.smithi107.stdout:3/288: read d9/d2f/f3c [3877236,73980] 0 2024-04-25T00:18:28.690 INFO:tasks.workunit.client.0.smithi107.stdout:4/560: dread d3/d24/d25/f2b [0,4194304] 0 2024-04-25T00:18:28.690 INFO:tasks.workunit.client.0.smithi107.stdout:4/561: stat d3/d24/f95 0 2024-04-25T00:18:28.691 INFO:tasks.workunit.client.0.smithi107.stdout:3/289: link d9/d22/f1e d9/f5e 0 2024-04-25T00:18:28.691 INFO:tasks.workunit.client.0.smithi107.stdout:4/562: mknod d3/cbb 0 2024-04-25T00:18:28.692 INFO:tasks.workunit.client.0.smithi107.stdout:4/563: symlink d3/d10/d7b/da5/lbc 0 2024-04-25T00:18:28.692 INFO:tasks.workunit.client.0.smithi107.stdout:3/290: mknod d9/d54/d5c/c5f 0 2024-04-25T00:18:28.694 INFO:tasks.workunit.client.0.smithi107.stdout:4/564: creat d3/da8/fbd x:0 0 0 2024-04-25T00:18:28.695 INFO:tasks.workunit.client.0.smithi107.stdout:3/291: rename d9/d2f/l3f to d9/d22/d1b/l60 0 2024-04-25T00:18:28.695 INFO:tasks.workunit.client.0.smithi107.stdout:4/565: rename d3/d24 to d3/d24/db1/dbe 22 2024-04-25T00:18:28.695 INFO:tasks.workunit.client.0.smithi107.stdout:3/292: chown f7 28331 1 2024-04-25T00:18:28.695 INFO:tasks.workunit.client.0.smithi107.stdout:4/566: readlink d3/d24/d25/d67/l78 0 2024-04-25T00:18:28.695 INFO:tasks.workunit.client.0.smithi107.stdout:3/293: creat d9/f61 x:0 0 0 2024-04-25T00:18:28.697 INFO:tasks.workunit.client.0.smithi107.stdout:3/294: link d9/ca d9/d2f/d3b/d41/c62 0 2024-04-25T00:18:28.697 INFO:tasks.workunit.client.0.smithi107.stdout:3/295: fdatasync d9/f10 0 2024-04-25T00:18:28.697 INFO:tasks.workunit.client.0.smithi107.stdout:3/296: stat d9/f11 0 2024-04-25T00:18:28.706 INFO:tasks.workunit.client.0.smithi107.stdout:9/296: dwrite d0/d8/d56/f64 [0,4194304] 0 2024-04-25T00:18:28.707 INFO:tasks.workunit.client.0.smithi107.stdout:9/297: truncate d0/d8/d34/f42 5139737 0 2024-04-25T00:18:28.707 INFO:tasks.workunit.client.0.smithi107.stdout:9/298: readlink d0/d43/l11 0 2024-04-25T00:18:28.707 INFO:tasks.workunit.client.0.smithi107.stdout:9/299: write d0/d8/d56/f59 [453164,101064] 0 2024-04-25T00:18:28.708 INFO:tasks.workunit.client.0.smithi107.stdout:9/300: symlink d0/d8/de/l68 0 2024-04-25T00:18:28.708 INFO:tasks.workunit.client.0.smithi107.stdout:9/301: truncate d0/f30 1490689 0 2024-04-25T00:18:28.709 INFO:tasks.workunit.client.0.smithi107.stdout:9/302: getdents d0/d43 0 2024-04-25T00:18:28.709 INFO:tasks.workunit.client.0.smithi107.stdout:9/303: stat d0/d43 0 2024-04-25T00:18:28.722 INFO:tasks.workunit.client.0.smithi107.stdout:9/304: truncate d0/d8/f26 2275382 0 2024-04-25T00:18:28.723 INFO:tasks.workunit.client.0.smithi107.stdout:9/305: mkdir d0/d8/d32/d69 0 2024-04-25T00:18:28.724 INFO:tasks.workunit.client.0.smithi107.stdout:4/567: dwrite d3/d24/d25/d71/d7f/f9c [0,4194304] 0 2024-04-25T00:18:28.724 INFO:tasks.workunit.client.0.smithi107.stdout:4/568: write d3/d24/f95 [1437553,78303] 0 2024-04-25T00:18:28.724 INFO:tasks.workunit.client.0.smithi107.stdout:3/297: dwrite d9/ff [0,4194304] 0 2024-04-25T00:18:28.726 INFO:tasks.workunit.client.0.smithi107.stdout:9/306: creat d0/d8/de/d5c/f6a x:0 0 0 2024-04-25T00:18:28.740 INFO:tasks.workunit.client.0.smithi107.stdout:3/298: unlink d9/d2f/d3b/d41/l52 0 2024-04-25T00:18:28.740 INFO:tasks.workunit.client.0.smithi107.stdout:3/299: truncate d9/d15/f35 1272877 0 2024-04-25T00:18:28.741 INFO:tasks.workunit.client.0.smithi107.stdout:4/569: dread d3/d24/d5c/f6f [0,4194304] 0 2024-04-25T00:18:28.742 INFO:tasks.workunit.client.0.smithi107.stdout:3/300: unlink d9/d15/d1a/f1c 0 2024-04-25T00:18:28.749 INFO:tasks.workunit.client.0.smithi107.stdout:9/307: dwrite d0/d8/d34/d36/f4a [0,4194304] 0 2024-04-25T00:18:28.751 INFO:tasks.workunit.client.0.smithi107.stdout:9/308: getdents d0/d8/d34/d36/d3d 0 2024-04-25T00:18:28.753 INFO:tasks.workunit.client.0.smithi107.stdout:9/309: rmdir d0/d8/d32/d50 39 2024-04-25T00:18:28.753 INFO:tasks.workunit.client.0.smithi107.stdout:9/310: dread - d0/d8/d32/f46 zero size 2024-04-25T00:18:28.753 INFO:tasks.workunit.client.0.smithi107.stdout:9/311: chown d0/d43/f10 0 1 2024-04-25T00:18:28.759 INFO:tasks.workunit.client.0.smithi107.stdout:3/301: dwrite d9/d22/d1b/d55/f5b [0,4194304] 0 2024-04-25T00:18:28.771 INFO:tasks.workunit.client.0.smithi107.stdout:9/312: dwrite d0/d8/f5a [0,4194304] 0 2024-04-25T00:18:28.774 INFO:tasks.workunit.client.0.smithi107.stdout:9/313: mknod d0/d8/de/c6b 0 2024-04-25T00:18:28.774 INFO:tasks.workunit.client.0.smithi107.stdout:9/314: dread - d0/d8/d32/d50/f54 zero size 2024-04-25T00:18:28.774 INFO:tasks.workunit.client.0.smithi107.stdout:9/315: chown d0/d8/c48 3810 1 2024-04-25T00:18:28.775 INFO:tasks.workunit.client.0.smithi107.stdout:9/316: truncate d0/d8/d34/f4f 1434349 0 2024-04-25T00:18:28.776 INFO:tasks.workunit.client.0.smithi107.stdout:9/317: mkdir d0/d8/d56/d6c 0 2024-04-25T00:18:28.777 INFO:tasks.workunit.client.0.smithi107.stdout:9/318: creat d0/d8/d56/d6c/f6d x:0 0 0 2024-04-25T00:18:28.778 INFO:tasks.workunit.client.0.smithi107.stdout:9/319: dread d0/d1b/f2f [0,4194304] 0 2024-04-25T00:18:28.778 INFO:tasks.workunit.client.0.smithi107.stdout:9/320: write d0/d43/f28 [274058,6470] 0 2024-04-25T00:18:28.779 INFO:tasks.workunit.client.0.smithi107.stdout:9/321: chown d0/d8/de/f33 983 1 2024-04-25T00:18:28.779 INFO:tasks.workunit.client.0.smithi107.stdout:9/322: chown d0/d8/de/f51 1091947803 1 2024-04-25T00:18:28.779 INFO:tasks.workunit.client.0.smithi107.stdout:3/302: dwrite d9/d2f/f3c [0,4194304] 0 2024-04-25T00:18:28.779 INFO:tasks.workunit.client.0.smithi107.stdout:3/303: fsync d9/d22/f18 0 2024-04-25T00:18:28.779 INFO:tasks.workunit.client.0.smithi107.stdout:3/304: fdatasync d9/d2f/d44/d4b/f57 0 2024-04-25T00:18:28.780 INFO:tasks.workunit.client.0.smithi107.stdout:3/305: symlink d9/d2f/d3b/d41/l63 0 2024-04-25T00:18:28.780 INFO:tasks.workunit.client.0.smithi107.stdout:3/306: read - d9/d2f/d3b/f47 zero size 2024-04-25T00:18:28.781 INFO:tasks.workunit.client.0.smithi107.stdout:9/323: unlink d0/d43/l11 0 2024-04-25T00:18:28.781 INFO:tasks.workunit.client.0.smithi107.stdout:9/324: read d0/d8/f26 [1722184,13279] 0 2024-04-25T00:18:28.783 INFO:tasks.workunit.client.0.smithi107.stdout:3/307: creat d9/d2f/d44/f64 x:0 0 0 2024-04-25T00:18:28.784 INFO:tasks.workunit.client.0.smithi107.stdout:9/325: symlink d0/d8/de/d5c/l6e 0 2024-04-25T00:18:28.785 INFO:tasks.workunit.client.0.smithi107.stdout:3/308: truncate d9/d22/d1b/d55/f5b 3260430 0 2024-04-25T00:18:28.787 INFO:tasks.workunit.client.0.smithi107.stdout:9/326: mkdir d0/d8/d6f 0 2024-04-25T00:18:28.789 INFO:tasks.workunit.client.0.smithi107.stdout:9/327: creat d0/d8/d34/f70 x:0 0 0 2024-04-25T00:18:28.790 INFO:tasks.workunit.client.0.smithi107.stdout:0/285: sync 2024-04-25T00:18:28.790 INFO:tasks.workunit.client.0.smithi107.stdout:0/286: readlink dc/dd/l27 0 2024-04-25T00:18:28.790 INFO:tasks.workunit.client.0.smithi107.stdout:0/287: dread - dc/f2f zero size 2024-04-25T00:18:28.792 INFO:tasks.workunit.client.0.smithi107.stdout:0/288: creat dc/dd/d16/d1a/f57 x:0 0 0 2024-04-25T00:18:28.801 INFO:tasks.workunit.client.0.smithi107.stdout:3/309: dwrite d9/f10 [4194304,4194304] 0 2024-04-25T00:18:28.809 INFO:tasks.workunit.client.0.smithi107.stdout:0/289: dwrite dc/f28 [8388608,4194304] 0 2024-04-25T00:18:28.809 INFO:tasks.workunit.client.0.smithi107.stdout:0/290: chown dc/dd 33470 1 2024-04-25T00:18:28.809 INFO:tasks.workunit.client.0.smithi107.stdout:0/291: truncate dc/f49 1315429 0 2024-04-25T00:18:28.818 INFO:tasks.workunit.client.0.smithi107.stdout:0/292: write dc/dd/d16/d33/f37 [275531,34005] 0 2024-04-25T00:18:28.820 INFO:tasks.workunit.client.0.smithi107.stdout:0/293: creat dc/d24/d25/f58 x:0 0 0 2024-04-25T00:18:28.820 INFO:tasks.workunit.client.0.smithi107.stdout:0/294: write dc/f49 [2256461,81733] 0 2024-04-25T00:18:28.820 INFO:tasks.workunit.client.0.smithi107.stdout:0/295: chown dc/dd/f14 726 1 2024-04-25T00:18:28.820 INFO:tasks.workunit.client.0.smithi107.stdout:0/296: fsync dc/dd/d16/d48/f54 0 2024-04-25T00:18:28.821 INFO:tasks.workunit.client.0.smithi107.stdout:0/297: unlink dc/d24/d25/d31/c3b 0 2024-04-25T00:18:28.822 INFO:tasks.workunit.client.0.smithi107.stdout:0/298: mkdir dc/dd/d16/d1a/d3f/d59 0 2024-04-25T00:18:28.822 INFO:tasks.workunit.client.0.smithi107.stdout:0/299: stat dc/d24 0 2024-04-25T00:18:28.823 INFO:tasks.workunit.client.0.smithi107.stdout:0/300: rename dc/dd/d16/d1a/f1e to dc/dd/d16/d48/f5a 0 2024-04-25T00:18:28.827 INFO:tasks.workunit.client.0.smithi107.stdout:0/301: dread dc/d24/f3a [0,4194304] 0 2024-04-25T00:18:28.827 INFO:tasks.workunit.client.0.smithi107.stdout:0/302: write dc/dd/d16/d48/f5a [1176504,39553] 0 2024-04-25T00:18:28.834 INFO:tasks.workunit.client.0.smithi107.stdout:0/303: truncate dc/f28 3066764 0 2024-04-25T00:18:28.834 INFO:tasks.workunit.client.0.smithi107.stdout:0/304: write dc/dd/d16/d1a/f57 [897962,17541] 0 2024-04-25T00:18:28.837 INFO:tasks.workunit.client.0.smithi107.stdout:0/305: write dc/f49 [591564,25586] 0 2024-04-25T00:18:28.839 INFO:tasks.workunit.client.0.smithi107.stdout:0/306: unlink dc/dd/l27 0 2024-04-25T00:18:28.843 INFO:tasks.workunit.client.0.smithi107.stdout:0/307: creat dc/d24/d25/d31/f5b x:0 0 0 2024-04-25T00:18:28.883 INFO:tasks.workunit.client.0.smithi107.stdout:6/337: sync 2024-04-25T00:18:28.886 INFO:tasks.workunit.client.0.smithi107.stdout:6/338: symlink d1/d2/l76 0 2024-04-25T00:18:28.887 INFO:tasks.workunit.client.0.smithi107.stdout:6/339: rename d1/d2/d3/c29 to d1/d2/db/d32/d33/d68/c77 0 2024-04-25T00:18:28.887 INFO:tasks.workunit.client.0.smithi107.stdout:6/340: fsync d1/d2/db/d32/d36/f46 0 2024-04-25T00:18:28.887 INFO:tasks.workunit.client.0.smithi107.stdout:6/341: stat d1/d2/db/d48/d56/d6b 0 2024-04-25T00:18:28.888 INFO:tasks.workunit.client.0.smithi107.stdout:6/342: creat d1/d2/db/d32/d33/d68/d75/f78 x:0 0 0 2024-04-25T00:18:28.892 INFO:tasks.workunit.client.0.smithi107.stdout:6/343: rename d1/d2/db/d48/d56 to d1/d2/d3/d28/d79 0 2024-04-25T00:18:28.893 INFO:tasks.workunit.client.0.smithi107.stdout:6/344: mkdir d1/d2/d2b/d3e/d5b/d7a 0 2024-04-25T00:18:28.909 INFO:tasks.workunit.client.0.smithi107.stdout:6/345: dwrite d1/d2/d3/d26/f51 [0,4194304] 0 2024-04-25T00:18:28.911 INFO:tasks.workunit.client.0.smithi107.stdout:6/346: symlink d1/d2/db/d3c/l7b 0 2024-04-25T00:18:28.914 INFO:tasks.workunit.client.0.smithi107.stdout:6/347: unlink d1/d2/db/d3c/f4e 0 2024-04-25T00:18:28.915 INFO:tasks.workunit.client.0.smithi107.stdout:6/348: unlink d1/d2/db/d32/d36/d3a/f47 0 2024-04-25T00:18:28.919 INFO:tasks.workunit.client.0.smithi107.stdout:6/349: rmdir d1/d2/d3 39 2024-04-25T00:18:28.922 INFO:tasks.workunit.client.0.smithi107.stdout:6/350: mknod d1/d2/db/d32/d33/d68/d75/c7c 0 2024-04-25T00:18:28.923 INFO:tasks.workunit.client.0.smithi107.stdout:6/351: mkdir d1/d2/db/d32/d36/d7d 0 2024-04-25T00:18:28.925 INFO:tasks.workunit.client.0.smithi107.stdout:6/352: mknod d1/d2/d3/c7e 0 2024-04-25T00:18:28.941 INFO:tasks.workunit.client.0.smithi107.stdout:6/353: dwrite d1/d2/db/fd [4194304,4194304] 0 2024-04-25T00:18:28.941 INFO:tasks.workunit.client.0.smithi107.stdout:6/354: dread - d1/d4c/f70 zero size 2024-04-25T00:18:28.942 INFO:tasks.workunit.client.0.smithi107.stdout:6/355: write d1/d2/f5f [783328,1600] 0 2024-04-25T00:18:28.946 INFO:tasks.workunit.client.0.smithi107.stdout:7/229: sync 2024-04-25T00:18:28.947 INFO:tasks.workunit.client.0.smithi107.stdout:1/354: sync 2024-04-25T00:18:28.948 INFO:tasks.workunit.client.0.smithi107.stdout:1/355: rename d2/c72 to d2/dd/d1b/c73 0 2024-04-25T00:18:28.948 INFO:tasks.workunit.client.0.smithi107.stdout:1/356: symlink d2/dd/d1a/l74 0 2024-04-25T00:18:28.949 INFO:tasks.workunit.client.0.smithi107.stdout:1/357: rmdir d2/d62/d66 39 2024-04-25T00:18:28.949 INFO:tasks.workunit.client.0.smithi107.stdout:1/358: write d2/dd/d1a/f1d [3934387,70772] 0 2024-04-25T00:18:28.952 INFO:tasks.workunit.client.0.smithi107.stdout:6/356: dread d1/d2/db/d32/d36/f59 [4194304,4194304] 0 2024-04-25T00:18:28.953 INFO:tasks.workunit.client.0.smithi107.stdout:6/357: chown d1/d2/d2b/d3e/d5b/c6c 3095089 1 2024-04-25T00:18:28.969 INFO:tasks.workunit.client.0.smithi107.stdout:7/230: dwrite d0/d8/db/d18/d19/d47/f31 [0,4194304] 0 2024-04-25T00:18:28.969 INFO:tasks.workunit.client.0.smithi107.stdout:7/231: chown d0/d8/d4e 1363 1 2024-04-25T00:18:28.972 INFO:tasks.workunit.client.0.smithi107.stdout:7/232: mknod d0/d8/d4e/d46/c54 0 2024-04-25T00:18:28.974 INFO:tasks.workunit.client.0.smithi107.stdout:7/233: creat d0/d8/d2c/f55 x:0 0 0 2024-04-25T00:18:28.974 INFO:tasks.workunit.client.0.smithi107.stdout:1/359: dwrite d2/dd/d1a/f6d [0,4194304] 0 2024-04-25T00:18:28.976 INFO:tasks.workunit.client.0.smithi107.stdout:9/328: sync 2024-04-25T00:18:28.976 INFO:tasks.workunit.client.0.smithi107.stdout:3/310: sync 2024-04-25T00:18:28.976 INFO:tasks.workunit.client.0.smithi107.stdout:4/570: sync 2024-04-25T00:18:28.976 INFO:tasks.workunit.client.0.smithi107.stdout:2/391: sync 2024-04-25T00:18:28.977 INFO:tasks.workunit.client.0.smithi107.stdout:5/485: sync 2024-04-25T00:18:28.977 INFO:tasks.workunit.client.0.smithi107.stdout:8/467: sync 2024-04-25T00:18:28.977 INFO:tasks.workunit.client.0.smithi107.stdout:8/468: fsync d2/d18/d1a/f25 0 2024-04-25T00:18:28.978 INFO:tasks.workunit.client.0.smithi107.stdout:4/571: creat d3/d5/d32/d9e/da2/daf/fbf x:0 0 0 2024-04-25T00:18:28.978 INFO:tasks.workunit.client.0.smithi107.stdout:9/329: symlink d0/d8/d32/d50/l71 0 2024-04-25T00:18:28.979 INFO:tasks.workunit.client.0.smithi107.stdout:2/392: mknod d6/d18/d40/c8e 0 2024-04-25T00:18:28.980 INFO:tasks.workunit.client.0.smithi107.stdout:8/469: rmdir d2/d18/d1a/d5e/d4f/da0 39 2024-04-25T00:18:28.981 INFO:tasks.workunit.client.0.smithi107.stdout:9/330: creat d0/d8/de/d5c/f72 x:0 0 0 2024-04-25T00:18:28.982 INFO:tasks.workunit.client.0.smithi107.stdout:9/331: stat d0/d8/d32/f4d 0 2024-04-25T00:18:28.982 INFO:tasks.workunit.client.0.smithi107.stdout:6/358: dwrite d1/d2/db/d3c/f50 [0,4194304] 0 2024-04-25T00:18:28.982 INFO:tasks.workunit.client.0.smithi107.stdout:6/359: write d1/d2/db/d32/d33/f6d [303842,109178] 0 2024-04-25T00:18:28.982 INFO:tasks.workunit.client.0.smithi107.stdout:6/360: write d1/d2/f5f [1263252,105163] 0 2024-04-25T00:18:28.982 INFO:tasks.workunit.client.0.smithi107.stdout:6/361: write d1/d2/d3/f31 [36170,101634] 0 2024-04-25T00:18:28.983 INFO:tasks.workunit.client.0.smithi107.stdout:2/393: creat d6/d18/d40/f8f x:0 0 0 2024-04-25T00:18:28.983 INFO:tasks.workunit.client.0.smithi107.stdout:7/234: write d0/f16 [3728672,86330] 0 2024-04-25T00:18:28.983 INFO:tasks.workunit.client.0.smithi107.stdout:7/235: dread - d0/d8/db/f51 zero size 2024-04-25T00:18:28.985 INFO:tasks.workunit.client.0.smithi107.stdout:9/332: creat d0/d8/d34/d36/f73 x:0 0 0 2024-04-25T00:18:28.986 INFO:tasks.workunit.client.0.smithi107.stdout:7/236: mkdir d0/d8/db/d56 0 2024-04-25T00:18:28.988 INFO:tasks.workunit.client.0.smithi107.stdout:6/362: rmdir d1/d2/db 39 2024-04-25T00:18:28.988 INFO:tasks.workunit.client.0.smithi107.stdout:0/308: sync 2024-04-25T00:18:28.990 INFO:tasks.workunit.client.0.smithi107.stdout:7/237: mknod d0/d8/db/d18/d19/c57 0 2024-04-25T00:18:28.992 INFO:tasks.workunit.client.0.smithi107.stdout:0/309: mkdir dc/dd/d16/d1a/d5c 0 2024-04-25T00:18:28.993 INFO:tasks.workunit.client.0.smithi107.stdout:7/238: mkdir d0/d8/d2c/d58 0 2024-04-25T00:18:28.997 INFO:tasks.workunit.client.0.smithi107.stdout:8/470: read d2/d18/f68 [4061131,69404] 0 2024-04-25T00:18:28.999 INFO:tasks.workunit.client.0.smithi107.stdout:8/471: mkdir d2/d18/d1a/d5e/d5c/d91/da3 0 2024-04-25T00:18:29.002 INFO:tasks.workunit.client.0.smithi107.stdout:3/311: write d9/d2f/f40 [1233088,14528] 0 2024-04-25T00:18:29.002 INFO:tasks.workunit.client.0.smithi107.stdout:3/312: chown d9/f56 168768 1 2024-04-25T00:18:29.002 INFO:tasks.workunit.client.0.smithi107.stdout:8/472: link d2/d18/d1a/d5e/d6d/c72 d2/d18/d1a/d5e/d4f/da0/da1/ca4 0 2024-04-25T00:18:29.002 INFO:tasks.workunit.client.0.smithi107.stdout:4/572: dwrite d3/d24/f69 [8388608,4194304] 0 2024-04-25T00:18:29.002 INFO:tasks.workunit.client.0.smithi107.stdout:0/310: write dc/f18 [127708,81859] 0 2024-04-25T00:18:29.003 INFO:tasks.workunit.client.0.smithi107.stdout:0/311: fsync dc/d24/d25/d31/f5b 0 2024-04-25T00:18:29.005 INFO:tasks.workunit.client.0.smithi107.stdout:4/573: mkdir d3/d5/d32/d53/dc0 0 2024-04-25T00:18:29.005 INFO:tasks.workunit.client.0.smithi107.stdout:4/574: readlink d3/d24/d5c/dab/l94 0 2024-04-25T00:18:29.005 INFO:tasks.workunit.client.0.smithi107.stdout:3/313: unlink d9/d22/f18 0 2024-04-25T00:18:29.005 INFO:tasks.workunit.client.0.smithi107.stdout:3/314: chown d9/f5e 1 1 2024-04-25T00:18:29.006 INFO:tasks.workunit.client.0.smithi107.stdout:3/315: read - d9/d2f/f46 zero size 2024-04-25T00:18:29.006 INFO:tasks.workunit.client.0.smithi107.stdout:3/316: fsync d9/d22/f1e 0 2024-04-25T00:18:29.007 INFO:tasks.workunit.client.0.smithi107.stdout:2/394: write d6/d18/d40/d42/d2b/f39 [4076889,27423] 0 2024-04-25T00:18:29.007 INFO:tasks.workunit.client.0.smithi107.stdout:2/395: readlink d6/d18/d40/d42/d69/d6e/d80/l8d 0 2024-04-25T00:18:29.007 INFO:tasks.workunit.client.0.smithi107.stdout:2/396: chown d6/d29/d4e/f7f 1 1 2024-04-25T00:18:29.007 INFO:tasks.workunit.client.0.smithi107.stdout:2/397: dread - d6/f72 zero size 2024-04-25T00:18:29.007 INFO:tasks.workunit.client.0.smithi107.stdout:8/473: unlink d2/d18/d1a/d5e/d3e/c8f 0 2024-04-25T00:18:29.007 INFO:tasks.workunit.client.0.smithi107.stdout:6/363: dread d1/d2/d3/f13 [0,4194304] 0 2024-04-25T00:18:29.007 INFO:tasks.workunit.client.0.smithi107.stdout:1/360: dwrite d2/dd/d1a/f27 [0,4194304] 0 2024-04-25T00:18:29.007 INFO:tasks.workunit.client.0.smithi107.stdout:3/317: truncate d9/d15/d1a/f30 2415182 0 2024-04-25T00:18:29.009 INFO:tasks.workunit.client.0.smithi107.stdout:2/398: creat d6/d29/d4e/d6f/f90 x:0 0 0 2024-04-25T00:18:29.013 INFO:tasks.workunit.client.0.smithi107.stdout:2/399: link d6/d29/d4e/l6d d6/d18/d4d/l91 0 2024-04-25T00:18:29.013 INFO:tasks.workunit.client.0.smithi107.stdout:2/400: write d6/f8 [1608082,26974] 0 2024-04-25T00:18:29.013 INFO:tasks.workunit.client.0.smithi107.stdout:2/401: dread - d6/d18/d40/d42/d2b/d30/d45/f61 zero size 2024-04-25T00:18:29.013 INFO:tasks.workunit.client.0.smithi107.stdout:2/402: fdatasync d6/f17 0 2024-04-25T00:18:29.013 INFO:tasks.workunit.client.0.smithi107.stdout:2/403: readlink d6/d18/d40/l84 0 2024-04-25T00:18:29.017 INFO:tasks.workunit.client.0.smithi107.stdout:2/404: link d6/d18/d40/d42/d2b/d30/d45/c46 d6/d29/c92 0 2024-04-25T00:18:29.018 INFO:tasks.workunit.client.0.smithi107.stdout:2/405: readlink d6/d18/l1f 0 2024-04-25T00:18:29.018 INFO:tasks.workunit.client.0.smithi107.stdout:5/486: dwrite d4/dc/fe [0,4194304] 0 2024-04-25T00:18:29.018 INFO:tasks.workunit.client.0.smithi107.stdout:5/487: fdatasync d4/dc/f63 0 2024-04-25T00:18:29.018 INFO:tasks.workunit.client.0.smithi107.stdout:5/488: dread - d4/dc/d26/d42/d8b/f99 zero size 2024-04-25T00:18:29.018 INFO:tasks.workunit.client.0.smithi107.stdout:5/489: chown d4/dc/f1a 6 1 2024-04-25T00:18:29.020 INFO:tasks.workunit.client.0.smithi107.stdout:2/406: truncate d6/d18/f5d 1491826 0 2024-04-25T00:18:29.020 INFO:tasks.workunit.client.0.smithi107.stdout:2/407: write d6/d18/d40/d42/d2b/f7a [144447,67407] 0 2024-04-25T00:18:29.020 INFO:tasks.workunit.client.0.smithi107.stdout:2/408: chown d6/d18/d40/d42/d69/d6e/d80 977181 1 2024-04-25T00:18:29.021 INFO:tasks.workunit.client.0.smithi107.stdout:5/490: symlink d4/d17/la5 0 2024-04-25T00:18:29.022 INFO:tasks.workunit.client.0.smithi107.stdout:2/409: truncate d6/d20/d22/d56/f68 5874705 0 2024-04-25T00:18:29.023 INFO:tasks.workunit.client.0.smithi107.stdout:5/491: rmdir d4/d35/d61/d84 39 2024-04-25T00:18:29.024 INFO:tasks.workunit.client.0.smithi107.stdout:7/239: dwrite d0/d8/db/d18/d19/d47/f53 [0,4194304] 0 2024-04-25T00:18:29.026 INFO:tasks.workunit.client.0.smithi107.stdout:2/410: truncate d6/d20/f5a 688254 0 2024-04-25T00:18:29.027 INFO:tasks.workunit.client.0.smithi107.stdout:5/492: fsync d4/dc/d26/d29/d51/f77 0 2024-04-25T00:18:29.027 INFO:tasks.workunit.client.0.smithi107.stdout:5/493: chown d4/dc/f14 35 1 2024-04-25T00:18:29.034 INFO:tasks.workunit.client.0.smithi107.stdout:2/411: write d6/ff [6122210,101600] 0 2024-04-25T00:18:29.034 INFO:tasks.workunit.client.0.smithi107.stdout:2/412: write d6/d18/d40/d42/f6a [617592,94346] 0 2024-04-25T00:18:29.034 INFO:tasks.workunit.client.0.smithi107.stdout:2/413: readlink d6/d29/l66 0 2024-04-25T00:18:29.036 INFO:tasks.workunit.client.0.smithi107.stdout:7/240: mkdir d0/d59 0 2024-04-25T00:18:29.037 INFO:tasks.workunit.client.0.smithi107.stdout:7/241: symlink d0/d8/db/d18/d19/d39/l5a 0 2024-04-25T00:18:29.038 INFO:tasks.workunit.client.0.smithi107.stdout:7/242: truncate d0/d8/d2c/f55 97910 0 2024-04-25T00:18:29.043 INFO:tasks.workunit.client.0.smithi107.stdout:5/494: dread d4/dc/f7f [0,4194304] 0 2024-04-25T00:18:29.043 INFO:tasks.workunit.client.0.smithi107.stdout:8/474: dwrite d2/d18/d1a/f8e [0,4194304] 0 2024-04-25T00:18:29.044 INFO:tasks.workunit.client.0.smithi107.stdout:1/361: dwrite d2/d62/d66/f6e [0,4194304] 0 2024-04-25T00:18:29.045 INFO:tasks.workunit.client.0.smithi107.stdout:5/495: creat d4/dc/d26/d29/d2e/d82/fa6 x:0 0 0 2024-04-25T00:18:29.045 INFO:tasks.workunit.client.0.smithi107.stdout:5/496: chown d4/dc/fe 2575 1 2024-04-25T00:18:29.048 INFO:tasks.workunit.client.0.smithi107.stdout:3/318: dwrite d9/d2f/d44/d4b/f57 [0,4194304] 0 2024-04-25T00:18:29.048 INFO:tasks.workunit.client.0.smithi107.stdout:3/319: write d9/f11 [933199,1963] 0 2024-04-25T00:18:29.048 INFO:tasks.workunit.client.0.smithi107.stdout:3/320: chown d9/d22/d1b/d55 1458282 1 2024-04-25T00:18:29.048 INFO:tasks.workunit.client.0.smithi107.stdout:3/321: dread - d9/d22/d1b/d55/f5d zero size 2024-04-25T00:18:29.048 INFO:tasks.workunit.client.0.smithi107.stdout:1/362: rmdir d2/dd/d25/d5e 39 2024-04-25T00:18:29.048 INFO:tasks.workunit.client.0.smithi107.stdout:1/363: dread - d2/dd/f6f zero size 2024-04-25T00:18:29.049 INFO:tasks.workunit.client.0.smithi107.stdout:5/497: rmdir d4/d17 39 2024-04-25T00:18:29.049 INFO:tasks.workunit.client.0.smithi107.stdout:5/498: chown d4/dc/l1d 68 1 2024-04-25T00:18:29.049 INFO:tasks.workunit.client.0.smithi107.stdout:5/499: write d4/dc/d26/d29/d2e/d82/f68 [1248357,118457] 0 2024-04-25T00:18:29.052 INFO:tasks.workunit.client.0.smithi107.stdout:1/364: mknod d2/d2a/d5c/c75 0 2024-04-25T00:18:29.055 INFO:tasks.workunit.client.0.smithi107.stdout:5/500: read d4/dc/d26/d29/d2e/f6d [28351,116340] 0 2024-04-25T00:18:29.055 INFO:tasks.workunit.client.0.smithi107.stdout:1/365: truncate d2/dd/d25/d5e/f58 429557 0 2024-04-25T00:18:29.056 INFO:tasks.workunit.client.0.smithi107.stdout:2/414: dwrite d6/d20/f88 [0,4194304] 0 2024-04-25T00:18:29.057 INFO:tasks.workunit.client.0.smithi107.stdout:2/415: fsync d6/d18/f78 0 2024-04-25T00:18:29.058 INFO:tasks.workunit.client.0.smithi107.stdout:7/243: dwrite d0/f16 [0,4194304] 0 2024-04-25T00:18:29.058 INFO:tasks.workunit.client.0.smithi107.stdout:7/244: chown d0/d8/db/d18/d19/f2f 901408038 1 2024-04-25T00:18:29.058 INFO:tasks.workunit.client.0.smithi107.stdout:7/245: readlink d0/d8/d4e/l36 0 2024-04-25T00:18:29.058 INFO:tasks.workunit.client.0.smithi107.stdout:7/246: chown d0/d8/d2c/f55 14328 1 2024-04-25T00:18:29.061 INFO:tasks.workunit.client.0.smithi107.stdout:2/416: mknod d6/d18/d40/d42/d2b/d30/d45/c93 0 2024-04-25T00:18:29.064 INFO:tasks.workunit.client.0.smithi107.stdout:3/322: getdents d9/d15/d1a 0 2024-04-25T00:18:29.064 INFO:tasks.workunit.client.0.smithi107.stdout:3/323: chown d9/f42 3 1 2024-04-25T00:18:29.065 INFO:tasks.workunit.client.0.smithi107.stdout:3/324: write d9/f56 [261247,93350] 0 2024-04-25T00:18:29.065 INFO:tasks.workunit.client.0.smithi107.stdout:0/312: rename dc/dd/d16/d1a to dc/dd/d5d 0 2024-04-25T00:18:29.068 INFO:tasks.workunit.client.0.smithi107.stdout:3/325: creat d9/d2f/d44/f65 x:0 0 0 2024-04-25T00:18:29.069 INFO:tasks.workunit.client.0.smithi107.stdout:4/575: rename d3/l73 to d3/d24/d61/lc1 0 2024-04-25T00:18:29.069 INFO:tasks.workunit.client.0.smithi107.stdout:4/576: chown d3/d10/d7b 0 1 2024-04-25T00:18:29.070 INFO:tasks.workunit.client.0.smithi107.stdout:4/577: readlink d3/d10/l31 0 2024-04-25T00:18:29.071 INFO:tasks.workunit.client.0.smithi107.stdout:4/578: readlink d3/da8/lb4 0 2024-04-25T00:18:29.071 INFO:tasks.workunit.client.0.smithi107.stdout:4/579: write d3/d5/d32/d53/d7e/fb3 [247457,75327] 0 2024-04-25T00:18:29.072 INFO:tasks.workunit.client.0.smithi107.stdout:2/417: write d6/d18/d40/d42/d70/f76 [716431,49884] 0 2024-04-25T00:18:29.072 INFO:tasks.workunit.client.0.smithi107.stdout:2/418: chown d6/d29/d4e/d6f 0 1 2024-04-25T00:18:29.073 INFO:tasks.workunit.client.0.smithi107.stdout:8/475: rename d2/d18/l49 to d2/d18/d1a/d5e/la5 0 2024-04-25T00:18:29.073 INFO:tasks.workunit.client.0.smithi107.stdout:4/580: mkdir d3/d24/d25/d71/dc2 0 2024-04-25T00:18:29.076 INFO:tasks.workunit.client.0.smithi107.stdout:5/501: dread d4/d1c/f28 [0,4194304] 0 2024-04-25T00:18:29.077 INFO:tasks.workunit.client.0.smithi107.stdout:8/476: rename d2/d18/d1a/d33/f50 to d2/d18/d1a/d5e/d5c/d6e/fa6 0 2024-04-25T00:18:29.082 INFO:tasks.workunit.client.0.smithi107.stdout:8/477: rename d2/d18/d1a/f48 to d2/d18/d1a/d5e/fa7 0 2024-04-25T00:18:29.082 INFO:tasks.workunit.client.0.smithi107.stdout:3/326: dread d9/d15/d1a/f30 [0,4194304] 0 2024-04-25T00:18:29.086 INFO:tasks.workunit.client.0.smithi107.stdout:0/313: read dc/d24/d25/f2b [2725828,60784] 0 2024-04-25T00:18:29.086 INFO:tasks.workunit.client.0.smithi107.stdout:0/314: chown dc/l2e 654389748 1 2024-04-25T00:18:29.089 INFO:tasks.workunit.client.0.smithi107.stdout:0/315: dread dc/d24/d25/d31/f36 [0,4194304] 0 2024-04-25T00:18:29.089 INFO:tasks.workunit.client.0.smithi107.stdout:0/316: readlink dc/l2e 0 2024-04-25T00:18:29.090 INFO:tasks.workunit.client.0.smithi107.stdout:1/366: dwrite d2/da/f53 [0,4194304] 0 2024-04-25T00:18:29.090 INFO:tasks.workunit.client.0.smithi107.stdout:1/367: readlink d2/dd/d1a/l74 0 2024-04-25T00:18:29.090 INFO:tasks.workunit.client.0.smithi107.stdout:7/247: dwrite d0/d8/f1e [0,4194304] 0 2024-04-25T00:18:29.090 INFO:tasks.workunit.client.0.smithi107.stdout:7/248: chown d0/d8/d4e/f40 1 1 2024-04-25T00:18:29.090 INFO:tasks.workunit.client.0.smithi107.stdout:7/249: chown d0/d8/db/d18/d19/d39/c45 237295 1 2024-04-25T00:18:29.092 INFO:tasks.workunit.client.0.smithi107.stdout:7/250: creat d0/d8/db/d18/d19/d47/f5b x:0 0 0 2024-04-25T00:18:29.093 INFO:tasks.workunit.client.0.smithi107.stdout:7/251: mkdir d0/d8/db/d18/d5c 0 2024-04-25T00:18:29.094 INFO:tasks.workunit.client.0.smithi107.stdout:0/317: mkdir dc/dd/d16/d5e 0 2024-04-25T00:18:29.094 INFO:tasks.workunit.client.0.smithi107.stdout:0/318: write dc/dd/d5d/f57 [1642173,33070] 0 2024-04-25T00:18:29.094 INFO:tasks.workunit.client.0.smithi107.stdout:0/319: chown dc/dd/d5d/c23 239 1 2024-04-25T00:18:29.095 INFO:tasks.workunit.client.0.smithi107.stdout:7/252: mkdir d0/d8/db/d18/d5c/d5d 0 2024-04-25T00:18:29.096 INFO:tasks.workunit.client.0.smithi107.stdout:3/327: dread d9/d2f/f40 [0,4194304] 0 2024-04-25T00:18:29.096 INFO:tasks.workunit.client.0.smithi107.stdout:7/253: creat d0/d8/d2c/d58/f5e x:0 0 0 2024-04-25T00:18:29.097 INFO:tasks.workunit.client.0.smithi107.stdout:7/254: write d0/d8/d2c/f50 [384517,70254] 0 2024-04-25T00:18:29.098 INFO:tasks.workunit.client.0.smithi107.stdout:7/255: creat d0/d8/d2c/d58/f5f x:0 0 0 2024-04-25T00:18:29.098 INFO:tasks.workunit.client.0.smithi107.stdout:0/320: read dc/dd/d16/f22 [1040639,124467] 0 2024-04-25T00:18:29.102 INFO:tasks.workunit.client.0.smithi107.stdout:8/478: dwrite d2/d18/d1a/f6c [0,4194304] 0 2024-04-25T00:18:29.103 INFO:tasks.workunit.client.0.smithi107.stdout:4/581: rename d3/da8/cb8 to d3/d24/d5c/cc3 0 2024-04-25T00:18:29.106 INFO:tasks.workunit.client.0.smithi107.stdout:5/502: rename d4/d17/c6c to d4/d35/d61/ca7 0 2024-04-25T00:18:29.107 INFO:tasks.workunit.client.0.smithi107.stdout:0/321: dread dc/f28 [0,4194304] 0 2024-04-25T00:18:29.109 INFO:tasks.workunit.client.0.smithi107.stdout:8/479: unlink d2/d18/d1a/d5e/c70 0 2024-04-25T00:18:29.109 INFO:tasks.workunit.client.0.smithi107.stdout:3/328: rename d9/d22/d1b/l21 to d9/l66 0 2024-04-25T00:18:29.110 INFO:tasks.workunit.client.0.smithi107.stdout:0/322: mkdir dc/dd/d16/d33/d35/d5f 0 2024-04-25T00:18:29.110 INFO:tasks.workunit.client.0.smithi107.stdout:0/323: chown dc/dd/f15 1850005062 1 2024-04-25T00:18:29.112 INFO:tasks.workunit.client.0.smithi107.stdout:3/329: dread d9/f14 [0,4194304] 0 2024-04-25T00:18:29.112 INFO:tasks.workunit.client.0.smithi107.stdout:3/330: read - d9/d15/f24 zero size 2024-04-25T00:18:29.112 INFO:tasks.workunit.client.0.smithi107.stdout:7/256: rename d0/d8/db/f20 to d0/d8/db/d18/d19/d47/f60 0 2024-04-25T00:18:29.115 INFO:tasks.workunit.client.0.smithi107.stdout:7/257: rmdir d0/d8/db/d18/d5c/d5d 0 2024-04-25T00:18:29.115 INFO:tasks.workunit.client.0.smithi107.stdout:7/258: dread - d0/d8/db/d18/d19/f23 zero size 2024-04-25T00:18:29.124 INFO:tasks.workunit.client.0.smithi107.stdout:7/259: rename d0/d8/db/d18/d19/d47/f60 to d0/d8/db/d18/f61 0 2024-04-25T00:18:29.124 INFO:tasks.workunit.client.0.smithi107.stdout:7/260: write d0/f13 [967187,12729] 0 2024-04-25T00:18:29.125 INFO:tasks.workunit.client.0.smithi107.stdout:7/261: creat d0/d8/db/d18/d19/d47/f62 x:0 0 0 2024-04-25T00:18:29.125 INFO:tasks.workunit.client.0.smithi107.stdout:7/262: chown d0/d59 245 1 2024-04-25T00:18:29.125 INFO:tasks.workunit.client.0.smithi107.stdout:7/263: chown d0/d8/db/d18 13518 1 2024-04-25T00:18:29.125 INFO:tasks.workunit.client.0.smithi107.stdout:3/331: dread f0 [0,4194304] 0 2024-04-25T00:18:29.128 INFO:tasks.workunit.client.0.smithi107.stdout:4/582: dwrite d3/d24/f69 [0,4194304] 0 2024-04-25T00:18:29.129 INFO:tasks.workunit.client.0.smithi107.stdout:5/503: dwrite d4/dc/d26/d29/d2e/f4c [0,4194304] 0 2024-04-25T00:18:29.130 INFO:tasks.workunit.client.0.smithi107.stdout:4/583: mknod d3/d10/d7b/cc4 0 2024-04-25T00:18:29.130 INFO:tasks.workunit.client.0.smithi107.stdout:4/584: write d3/d24/d61/f70 [956824,46411] 0 2024-04-25T00:18:29.130 INFO:tasks.workunit.client.0.smithi107.stdout:4/585: fsync d3/d5/d32/d53/f6e 0 2024-04-25T00:18:29.131 INFO:tasks.workunit.client.0.smithi107.stdout:0/324: dread dc/dd/f4c [0,4194304] 0 2024-04-25T00:18:29.135 INFO:tasks.workunit.client.0.smithi107.stdout:8/480: dwrite d2/d18/d1a/d5e/f8c [0,4194304] 0 2024-04-25T00:18:29.143 INFO:tasks.workunit.client.0.smithi107.stdout:8/481: truncate d2/d18/f53 1573184 0 2024-04-25T00:18:29.143 INFO:tasks.workunit.client.0.smithi107.stdout:3/332: write d9/f20 [3462548,40030] 0 2024-04-25T00:18:29.144 INFO:tasks.workunit.client.0.smithi107.stdout:3/333: truncate d9/f14 2506302 0 2024-04-25T00:18:29.145 INFO:tasks.workunit.client.0.smithi107.stdout:8/482: symlink d2/d18/d1a/d5e/d4f/la8 0 2024-04-25T00:18:29.156 INFO:tasks.workunit.client.0.smithi107.stdout:8/483: write d2/d18/d1a/d5e/d3e/f99 [4615255,46832] 0 2024-04-25T00:18:29.158 INFO:tasks.workunit.client.0.smithi107.stdout:4/586: dwrite d3/d5/d32/f9b [0,4194304] 0 2024-04-25T00:18:29.158 INFO:tasks.workunit.client.0.smithi107.stdout:7/264: dwrite d0/d8/d2c/f50 [0,4194304] 0 2024-04-25T00:18:29.159 INFO:tasks.workunit.client.0.smithi107.stdout:0/325: dread dc/dd/f15 [0,4194304] 0 2024-04-25T00:18:29.159 INFO:tasks.workunit.client.0.smithi107.stdout:0/326: chown dc/d24/d25/d31/l53 448924702 1 2024-04-25T00:18:29.159 INFO:tasks.workunit.client.0.smithi107.stdout:0/327: readlink dc/d24/d25/d31/l53 0 2024-04-25T00:18:29.161 INFO:tasks.workunit.client.0.smithi107.stdout:4/587: creat d3/d24/d25/d71/d7f/fc5 x:0 0 0 2024-04-25T00:18:29.162 INFO:tasks.workunit.client.0.smithi107.stdout:3/334: dwrite d9/d22/f1e [0,4194304] 0 2024-04-25T00:18:29.162 INFO:tasks.workunit.client.0.smithi107.stdout:7/265: mknod d0/d8/db/d18/d19/c63 0 2024-04-25T00:18:29.162 INFO:tasks.workunit.client.0.smithi107.stdout:7/266: write d0/d8/db/d18/d19/f2f [463724,122481] 0 2024-04-25T00:18:29.165 INFO:tasks.workunit.client.0.smithi107.stdout:0/328: creat dc/dd/d16/f60 x:0 0 0 2024-04-25T00:18:29.165 INFO:tasks.workunit.client.0.smithi107.stdout:0/329: stat dc/dd/d16/d33/f3d 0 2024-04-25T00:18:29.165 INFO:tasks.workunit.client.0.smithi107.stdout:0/330: truncate dc/d24/d25/f58 715035 0 2024-04-25T00:18:29.165 INFO:tasks.workunit.client.0.smithi107.stdout:0/331: truncate dc/dd/d16/d33/f3d 731700 0 2024-04-25T00:18:29.166 INFO:tasks.workunit.client.0.smithi107.stdout:0/332: creat dc/dd/d5d/d3f/f61 x:0 0 0 2024-04-25T00:18:29.166 INFO:tasks.workunit.client.0.smithi107.stdout:0/333: chown dc/l2e 1215233598 1 2024-04-25T00:18:29.166 INFO:tasks.workunit.client.0.smithi107.stdout:0/334: chown dc/dd/f4c 4 1 2024-04-25T00:18:29.167 INFO:tasks.workunit.client.0.smithi107.stdout:7/267: unlink d0/c17 0 2024-04-25T00:18:29.168 INFO:tasks.workunit.client.0.smithi107.stdout:0/335: getdents dc/dd/d16/d33 0 2024-04-25T00:18:29.170 INFO:tasks.workunit.client.0.smithi107.stdout:0/336: write dc/dd/f19 [976789,53404] 0 2024-04-25T00:18:29.170 INFO:tasks.workunit.client.0.smithi107.stdout:7/268: rename d0/d8/l27 to d0/l64 0 2024-04-25T00:18:29.171 INFO:tasks.workunit.client.0.smithi107.stdout:3/335: write d9/d22/d1b/d55/f5b [301916,54972] 0 2024-04-25T00:18:29.172 INFO:tasks.workunit.client.0.smithi107.stdout:7/269: creat d0/d8/f65 x:0 0 0 2024-04-25T00:18:29.172 INFO:tasks.workunit.client.0.smithi107.stdout:7/270: write d0/d8/db/d18/d19/f1f [4662412,60003] 0 2024-04-25T00:18:29.173 INFO:tasks.workunit.client.0.smithi107.stdout:3/336: creat d9/d15/d1a/d2c/f67 x:0 0 0 2024-04-25T00:18:29.174 INFO:tasks.workunit.client.0.smithi107.stdout:7/271: mknod d0/d8/c66 0 2024-04-25T00:18:29.178 INFO:tasks.workunit.client.0.smithi107.stdout:3/337: rename d9/d2f to d9/d54/d5c/d68 0 2024-04-25T00:18:29.178 INFO:tasks.workunit.client.0.smithi107.stdout:3/338: write d9/d54/d5c/d68/d3b/f47 [490575,20303] 0 2024-04-25T00:18:29.179 INFO:tasks.workunit.client.0.smithi107.stdout:3/339: mknod d9/c69 0 2024-04-25T00:18:29.180 INFO:tasks.workunit.client.0.smithi107.stdout:3/340: write d9/d15/d1a/d2c/f37 [416733,36391] 0 2024-04-25T00:18:29.180 INFO:tasks.workunit.client.0.smithi107.stdout:3/341: write d9/d15/d1a/f32 [2265168,90583] 0 2024-04-25T00:18:29.180 INFO:tasks.workunit.client.0.smithi107.stdout:4/588: dwrite d3/d24/d25/f28 [0,4194304] 0 2024-04-25T00:18:29.183 INFO:tasks.workunit.client.0.smithi107.stdout:4/589: mknod d3/d24/d61/cc6 0 2024-04-25T00:18:29.183 INFO:tasks.workunit.client.0.smithi107.stdout:4/590: chown d3/d10/d7b/cc4 510844 1 2024-04-25T00:18:29.183 INFO:tasks.workunit.client.0.smithi107.stdout:3/342: creat d9/d54/d5c/d68/d3b/f6a x:0 0 0 2024-04-25T00:18:29.186 INFO:tasks.workunit.client.0.smithi107.stdout:3/343: unlink d9/ff 0 2024-04-25T00:18:29.188 INFO:tasks.workunit.client.0.smithi107.stdout:3/344: mknod d9/d54/c6b 0 2024-04-25T00:18:29.190 INFO:tasks.workunit.client.0.smithi107.stdout:3/345: mknod d9/d22/d1b/d55/c6c 0 2024-04-25T00:18:29.191 INFO:tasks.workunit.client.0.smithi107.stdout:7/272: dread d0/d8/d2c/f50 [0,4194304] 0 2024-04-25T00:18:29.192 INFO:tasks.workunit.client.0.smithi107.stdout:0/337: dwrite dc/f49 [0,4194304] 0 2024-04-25T00:18:29.194 INFO:tasks.workunit.client.0.smithi107.stdout:0/338: creat dc/dd/d16/d33/d35/d5f/f62 x:0 0 0 2024-04-25T00:18:29.194 INFO:tasks.workunit.client.0.smithi107.stdout:0/339: chown dc/dd/d5d/d3f/d59 46509194 1 2024-04-25T00:18:29.197 INFO:tasks.workunit.client.0.smithi107.stdout:0/340: rename dc/dd/d16/l4d to dc/dd/d16/d5e/l63 0 2024-04-25T00:18:29.197 INFO:tasks.workunit.client.0.smithi107.stdout:6/364: sync 2024-04-25T00:18:29.197 INFO:tasks.workunit.client.0.smithi107.stdout:9/333: sync 2024-04-25T00:18:29.199 INFO:tasks.workunit.client.0.smithi107.stdout:0/341: mkdir dc/d24/d38/d64 0 2024-04-25T00:18:29.200 INFO:tasks.workunit.client.0.smithi107.stdout:0/342: getdents dc/d24/d25 0 2024-04-25T00:18:29.200 INFO:tasks.workunit.client.0.smithi107.stdout:0/343: fdatasync dc/dd/f3c 0 2024-04-25T00:18:29.202 INFO:tasks.workunit.client.0.smithi107.stdout:9/334: dread d0/d8/d32/f5b [0,4194304] 0 2024-04-25T00:18:29.202 INFO:tasks.workunit.client.0.smithi107.stdout:4/591: dwrite d3/d24/d25/f30 [0,4194304] 0 2024-04-25T00:18:29.203 INFO:tasks.workunit.client.0.smithi107.stdout:4/592: dread d3/d5/f92 [0,4194304] 0 2024-04-25T00:18:29.205 INFO:tasks.workunit.client.0.smithi107.stdout:9/335: unlink d0/d43/f7 0 2024-04-25T00:18:29.210 INFO:tasks.workunit.client.0.smithi107.stdout:9/336: rmdir d0/d8/d34/d36/d3d 0 2024-04-25T00:18:29.211 INFO:tasks.workunit.client.0.smithi107.stdout:9/337: rename d0/d8/de/l31 to d0/d8/d56/l74 0 2024-04-25T00:18:29.211 INFO:tasks.workunit.client.0.smithi107.stdout:9/338: fsync d0/d8/d32/f46 0 2024-04-25T00:18:29.212 INFO:tasks.workunit.client.0.smithi107.stdout:4/593: dread d3/d24/f81 [0,4194304] 0 2024-04-25T00:18:29.212 INFO:tasks.workunit.client.0.smithi107.stdout:3/346: dwrite f7 [0,4194304] 0 2024-04-25T00:18:29.213 INFO:tasks.workunit.client.0.smithi107.stdout:9/339: symlink d0/d8/d56/l75 0 2024-04-25T00:18:29.214 INFO:tasks.workunit.client.0.smithi107.stdout:3/347: mknod d9/d22/c6d 0 2024-04-25T00:18:29.216 INFO:tasks.workunit.client.0.smithi107.stdout:6/365: dread d1/f1f [0,4194304] 0 2024-04-25T00:18:29.216 INFO:tasks.workunit.client.0.smithi107.stdout:3/348: symlink d9/d15/d1a/l6e 0 2024-04-25T00:18:29.217 INFO:tasks.workunit.client.0.smithi107.stdout:3/349: symlink d9/l6f 0 2024-04-25T00:18:29.217 INFO:tasks.workunit.client.0.smithi107.stdout:3/350: stat d9/d54/d5c 0 2024-04-25T00:18:29.217 INFO:tasks.workunit.client.0.smithi107.stdout:6/366: symlink d1/d2/db/d32/d36/d7d/l7f 0 2024-04-25T00:18:29.218 INFO:tasks.workunit.client.0.smithi107.stdout:3/351: creat d9/d15/d1a/d4e/f70 x:0 0 0 2024-04-25T00:18:29.219 INFO:tasks.workunit.client.0.smithi107.stdout:6/367: truncate d1/d14/d18/f2c 1882387 0 2024-04-25T00:18:29.220 INFO:tasks.workunit.client.0.smithi107.stdout:3/352: mknod d9/d54/d5c/d68/d44/d4b/c71 0 2024-04-25T00:18:29.221 INFO:tasks.workunit.client.0.smithi107.stdout:3/353: creat d9/d15/d1a/d4e/f72 x:0 0 0 2024-04-25T00:18:29.222 INFO:tasks.workunit.client.0.smithi107.stdout:3/354: read d9/f42 [1797617,40850] 0 2024-04-25T00:18:29.222 INFO:tasks.workunit.client.0.smithi107.stdout:3/355: write d9/d22/d1b/d55/f5b [953515,43068] 0 2024-04-25T00:18:29.225 INFO:tasks.workunit.client.0.smithi107.stdout:3/356: symlink d9/d54/d5c/d68/l73 0 2024-04-25T00:18:29.225 INFO:tasks.workunit.client.0.smithi107.stdout:3/357: stat d9/f56 0 2024-04-25T00:18:29.225 INFO:tasks.workunit.client.0.smithi107.stdout:3/358: fdatasync d9/d54/d5c/d68/d44/d4b/f57 0 2024-04-25T00:18:29.226 INFO:tasks.workunit.client.0.smithi107.stdout:0/344: dwrite dc/dd/d16/f2a [0,4194304] 0 2024-04-25T00:18:29.226 INFO:tasks.workunit.client.0.smithi107.stdout:0/345: dread - dc/d24/d25/d31/f5b zero size 2024-04-25T00:18:29.231 INFO:tasks.workunit.client.0.smithi107.stdout:4/594: dread d3/d24/f2c [0,4194304] 0 2024-04-25T00:18:29.232 INFO:tasks.workunit.client.0.smithi107.stdout:9/340: write d0/d8/d34/d36/f4a [724068,71913] 0 2024-04-25T00:18:29.232 INFO:tasks.workunit.client.0.smithi107.stdout:4/595: creat d3/d24/d25/d71/dc2/fc7 x:0 0 0 2024-04-25T00:18:29.234 INFO:tasks.workunit.client.0.smithi107.stdout:3/359: dread d9/f11 [0,4194304] 0 2024-04-25T00:18:29.234 INFO:tasks.workunit.client.0.smithi107.stdout:9/341: unlink d0/d8/d34/d36/c47 0 2024-04-25T00:18:29.235 INFO:tasks.workunit.client.0.smithi107.stdout:4/596: link d3/d24/d61/f70 d3/d5/d32/d9e/da2/fc8 0 2024-04-25T00:18:29.235 INFO:tasks.workunit.client.0.smithi107.stdout:4/597: stat d3/d24/d25/c77 0 2024-04-25T00:18:29.238 INFO:tasks.workunit.client.0.smithi107.stdout:9/342: rename d0/d8/d32 to d0/d8/d56/d76 0 2024-04-25T00:18:29.238 INFO:tasks.workunit.client.0.smithi107.stdout:9/343: stat d0/d8/de/c2b 0 2024-04-25T00:18:29.238 INFO:tasks.workunit.client.0.smithi107.stdout:9/344: read - d0/d8/d34/f70 zero size 2024-04-25T00:18:29.239 INFO:tasks.workunit.client.0.smithi107.stdout:9/345: mkdir d0/d8/d34/d36/d77 0 2024-04-25T00:18:29.241 INFO:tasks.workunit.client.0.smithi107.stdout:9/346: creat d0/d8/d56/d76/f78 x:0 0 0 2024-04-25T00:18:29.243 INFO:tasks.workunit.client.0.smithi107.stdout:9/347: link d0/d8/c48 d0/d8/d6f/c79 0 2024-04-25T00:18:29.244 INFO:tasks.workunit.client.0.smithi107.stdout:6/368: dwrite d1/d2/f5f [0,4194304] 0 2024-04-25T00:18:29.246 INFO:tasks.workunit.client.0.smithi107.stdout:9/348: unlink d0/d8/d34/f3f 0 2024-04-25T00:18:29.246 INFO:tasks.workunit.client.0.smithi107.stdout:9/349: chown d0/d8/f45 1 1 2024-04-25T00:18:29.247 INFO:tasks.workunit.client.0.smithi107.stdout:6/369: symlink d1/d14/d18/l80 0 2024-04-25T00:18:29.255 INFO:tasks.workunit.client.0.smithi107.stdout:4/598: dwrite d3/d5/d32/d53/d7e/f86 [0,4194304] 0 2024-04-25T00:18:29.256 INFO:tasks.workunit.client.0.smithi107.stdout:0/346: dwrite dc/f50 [0,4194304] 0 2024-04-25T00:18:29.256 INFO:tasks.workunit.client.0.smithi107.stdout:3/360: dwrite d9/f5e [0,4194304] 0 2024-04-25T00:18:29.257 INFO:tasks.workunit.client.0.smithi107.stdout:4/599: creat d3/d24/db1/fc9 x:0 0 0 2024-04-25T00:18:29.257 INFO:tasks.workunit.client.0.smithi107.stdout:0/347: symlink dc/l65 0 2024-04-25T00:18:29.258 INFO:tasks.workunit.client.0.smithi107.stdout:0/348: chown dc/dd/d16/l4a 28121 1 2024-04-25T00:18:29.264 INFO:tasks.workunit.client.0.smithi107.stdout:3/361: mknod d9/d54/d5c/d68/d44/c74 0 2024-04-25T00:18:29.264 INFO:tasks.workunit.client.0.smithi107.stdout:3/362: truncate d9/d22/f1e 4910703 0 2024-04-25T00:18:29.265 INFO:tasks.workunit.client.0.smithi107.stdout:3/363: creat d9/d15/d1a/d2c/f75 x:0 0 0 2024-04-25T00:18:29.269 INFO:tasks.workunit.client.0.smithi107.stdout:3/364: unlink f0 0 2024-04-25T00:18:29.279 INFO:tasks.workunit.client.0.smithi107.stdout:0/349: dread dc/dd/f14 [0,4194304] 0 2024-04-25T00:18:29.279 INFO:tasks.workunit.client.0.smithi107.stdout:4/600: dwrite d3/d10/f8b [0,4194304] 0 2024-04-25T00:18:29.279 INFO:tasks.workunit.client.0.smithi107.stdout:0/350: creat dc/d24/d38/f66 x:0 0 0 2024-04-25T00:18:29.279 INFO:tasks.workunit.client.0.smithi107.stdout:3/365: write d9/f5e [1312587,35037] 0 2024-04-25T00:18:29.279 INFO:tasks.workunit.client.0.smithi107.stdout:4/601: creat d3/d5/d32/d53/dc0/fca x:0 0 0 2024-04-25T00:18:29.279 INFO:tasks.workunit.client.0.smithi107.stdout:0/351: truncate dc/f18 64466 0 2024-04-25T00:18:29.286 INFO:tasks.workunit.client.0.smithi107.stdout:0/352: dread dc/dd/d16/d48/f5a [0,4194304] 0 2024-04-25T00:18:29.291 INFO:tasks.workunit.client.0.smithi107.stdout:4/602: dread d3/d24/f69 [4194304,4194304] 0 2024-04-25T00:18:29.291 INFO:tasks.workunit.client.0.smithi107.stdout:0/353: dread dc/dd/d16/d33/f3d [0,4194304] 0 2024-04-25T00:18:29.292 INFO:tasks.workunit.client.0.smithi107.stdout:3/366: dread d9/d54/d5c/d68/d44/d4b/f57 [0,4194304] 0 2024-04-25T00:18:29.292 INFO:tasks.workunit.client.0.smithi107.stdout:4/603: mkdir d3/d24/d25/d67/dcb 0 2024-04-25T00:18:29.293 INFO:tasks.workunit.client.0.smithi107.stdout:0/354: creat dc/d24/d25/f67 x:0 0 0 2024-04-25T00:18:29.293 INFO:tasks.workunit.client.0.smithi107.stdout:4/604: truncate d3/d5/d32/f38 163436 0 2024-04-25T00:18:29.294 INFO:tasks.workunit.client.0.smithi107.stdout:4/605: chown d3/d5/d32/d9e 4001054 1 2024-04-25T00:18:29.294 INFO:tasks.workunit.client.0.smithi107.stdout:3/367: write d9/f56 [53151,82842] 0 2024-04-25T00:18:29.294 INFO:tasks.workunit.client.0.smithi107.stdout:0/355: creat dc/d24/d38/d64/f68 x:0 0 0 2024-04-25T00:18:29.295 INFO:tasks.workunit.client.0.smithi107.stdout:3/368: rename d9/d54/d5c/d68/d3b/d41 to d9/d15/d39/d76 0 2024-04-25T00:18:29.295 INFO:tasks.workunit.client.0.smithi107.stdout:3/369: fsync d9/f42 0 2024-04-25T00:18:29.295 INFO:tasks.workunit.client.0.smithi107.stdout:4/606: creat d3/d24/d25/d67/fcc x:0 0 0 2024-04-25T00:18:29.295 INFO:tasks.workunit.client.0.smithi107.stdout:4/607: dread - d3/d24/d25/d71/f75 zero size 2024-04-25T00:18:29.296 INFO:tasks.workunit.client.0.smithi107.stdout:4/608: write d3/d5/f8f [503193,15942] 0 2024-04-25T00:18:29.299 INFO:tasks.workunit.client.0.smithi107.stdout:0/356: dread dc/dd/d16/f1b [0,4194304] 0 2024-04-25T00:18:29.301 INFO:tasks.workunit.client.0.smithi107.stdout:0/357: write dc/dd/f3c [117882,71240] 0 2024-04-25T00:18:29.314 INFO:tasks.workunit.client.0.smithi107.stdout:4/609: dwrite d3/d5/d32/d9e/da2/fa4 [0,4194304] 0 2024-04-25T00:18:29.317 INFO:tasks.workunit.client.0.smithi107.stdout:4/610: dread d3/d24/d5c/dab/da3/fa6 [0,4194304] 0 2024-04-25T00:18:29.318 INFO:tasks.workunit.client.0.smithi107.stdout:3/370: dwrite d9/d54/d5c/d68/d3b/f47 [0,4194304] 0 2024-04-25T00:18:29.319 INFO:tasks.workunit.client.0.smithi107.stdout:4/611: mknod d3/d24/d61/ccd 0 2024-04-25T00:18:29.319 INFO:tasks.workunit.client.0.smithi107.stdout:4/612: write d3/d24/db1/fc9 [289790,9171] 0 2024-04-25T00:18:29.321 INFO:tasks.workunit.client.0.smithi107.stdout:4/613: symlink d3/d10/lce 0 2024-04-25T00:18:29.322 INFO:tasks.workunit.client.0.smithi107.stdout:4/614: symlink d3/d24/d25/d67/lcf 0 2024-04-25T00:18:29.322 INFO:tasks.workunit.client.0.smithi107.stdout:4/615: fsync d3/d5/f3d 0 2024-04-25T00:18:29.324 INFO:tasks.workunit.client.0.smithi107.stdout:4/616: unlink d3/d5/d32/d9e/da2/fc8 0 2024-04-25T00:18:29.324 INFO:tasks.workunit.client.0.smithi107.stdout:4/617: stat d3/da8/fb7 0 2024-04-25T00:18:29.336 INFO:tasks.workunit.client.0.smithi107.stdout:3/371: dwrite d9/f5e [0,4194304] 0 2024-04-25T00:18:29.379 INFO:tasks.workunit.client.0.smithi107.stdout:2/419: sync 2024-04-25T00:18:29.395 INFO:tasks.workunit.client.0.smithi107.stdout:2/420: dwrite d6/d18/d40/d42/f6a [0,4194304] 0 2024-04-25T00:18:29.397 INFO:tasks.workunit.client.0.smithi107.stdout:2/421: mkdir d6/d20/d22/d56/d94 0 2024-04-25T00:18:29.397 INFO:tasks.workunit.client.0.smithi107.stdout:2/422: read d6/d18/d40/d42/d2b/f39 [2141000,92834] 0 2024-04-25T00:18:29.398 INFO:tasks.workunit.client.0.smithi107.stdout:2/423: creat d6/d18/d40/d42/d69/d6e/d80/f95 x:0 0 0 2024-04-25T00:18:29.400 INFO:tasks.workunit.client.0.smithi107.stdout:2/424: rename d6/d18/d40/d42/f6a to d6/d20/d22/d56/f96 0 2024-04-25T00:18:29.401 INFO:tasks.workunit.client.0.smithi107.stdout:2/425: mknod d6/d29/d4e/d6f/c97 0 2024-04-25T00:18:29.415 INFO:tasks.workunit.client.0.smithi107.stdout:2/426: dwrite d6/d18/d40/d42/f3c [0,4194304] 0 2024-04-25T00:18:29.416 INFO:tasks.workunit.client.0.smithi107.stdout:2/427: dread - d6/d20/d22/f3a zero size 2024-04-25T00:18:29.418 INFO:tasks.workunit.client.0.smithi107.stdout:2/428: creat d6/d18/d4d/f98 x:0 0 0 2024-04-25T00:18:29.418 INFO:tasks.workunit.client.0.smithi107.stdout:2/429: write d6/f3d [4805117,85398] 0 2024-04-25T00:18:29.420 INFO:tasks.workunit.client.0.smithi107.stdout:2/430: rename d6/d20/d22/d56/f7e to d6/d18/d40/f99 0 2024-04-25T00:18:29.421 INFO:tasks.workunit.client.0.smithi107.stdout:2/431: creat d6/d18/d40/d42/d69/d6e/d80/f9a x:0 0 0 2024-04-25T00:18:29.423 INFO:tasks.workunit.client.0.smithi107.stdout:2/432: rename d6/f10 to d6/d29/d4e/d8c/f9b 0 2024-04-25T00:18:29.425 INFO:tasks.workunit.client.0.smithi107.stdout:2/433: symlink d6/d18/d40/d42/d69/d6e/l9c 0 2024-04-25T00:18:29.426 INFO:tasks.workunit.client.0.smithi107.stdout:2/434: symlink d6/d29/l9d 0 2024-04-25T00:18:29.426 INFO:tasks.workunit.client.0.smithi107.stdout:2/435: dread - d6/d18/d40/d42/f47 zero size 2024-04-25T00:18:29.427 INFO:tasks.workunit.client.0.smithi107.stdout:2/436: creat d6/d18/d40/d42/d2b/f9e x:0 0 0 2024-04-25T00:18:29.440 INFO:tasks.workunit.client.0.smithi107.stdout:2/437: write d6/d18/d40/d42/f3c [1199439,12957] 0 2024-04-25T00:18:29.440 INFO:tasks.workunit.client.0.smithi107.stdout:2/438: getdents d6/d20/d77 0 2024-04-25T00:18:29.441 INFO:tasks.workunit.client.0.smithi107.stdout:2/439: getdents d6/d18/d40 0 2024-04-25T00:18:29.442 INFO:tasks.workunit.client.0.smithi107.stdout:2/440: mkdir d6/d20/d77/d9f 0 2024-04-25T00:18:29.442 INFO:tasks.workunit.client.0.smithi107.stdout:2/441: write d6/d18/d40/d42/f47 [44107,33003] 0 2024-04-25T00:18:29.443 INFO:tasks.workunit.client.0.smithi107.stdout:2/442: creat d6/d20/d22/fa0 x:0 0 0 2024-04-25T00:18:29.448 INFO:tasks.workunit.client.0.smithi107.stdout:2/443: write d6/d18/f2e [2470061,40376] 0 2024-04-25T00:18:29.450 INFO:tasks.workunit.client.0.smithi107.stdout:2/444: creat d6/d20/d77/d9f/fa1 x:0 0 0 2024-04-25T00:18:29.451 INFO:tasks.workunit.client.0.smithi107.stdout:2/445: mkdir d6/d18/d40/d42/d69/d6e/da2 0 2024-04-25T00:18:29.451 INFO:tasks.workunit.client.0.smithi107.stdout:2/446: creat d6/d18/d40/d42/d2b/d30/d45/fa3 x:0 0 0 2024-04-25T00:18:29.458 INFO:tasks.workunit.client.0.smithi107.stdout:2/447: dread d6/d18/d40/d42/d70/f76 [0,4194304] 0 2024-04-25T00:18:29.458 INFO:tasks.workunit.client.0.smithi107.stdout:2/448: chown d6/d20 451322 1 2024-04-25T00:18:29.458 INFO:tasks.workunit.client.0.smithi107.stdout:2/449: fsync d6/d18/d4d/d67/f49 0 2024-04-25T00:18:29.460 INFO:tasks.workunit.client.0.smithi107.stdout:2/450: link d6/d18/d40/d42/d69/d6e/d80/l8d d6/d18/d40/d42/d2b/d30/d4c/la4 0 2024-04-25T00:18:29.474 INFO:tasks.workunit.client.0.smithi107.stdout:2/451: dwrite d6/d20/d22/f3a [0,4194304] 0 2024-04-25T00:18:29.476 INFO:tasks.workunit.client.0.smithi107.stdout:2/452: mknod d6/d18/d40/ca5 0 2024-04-25T00:18:29.479 INFO:tasks.workunit.client.0.smithi107.stdout:1/368: sync 2024-04-25T00:18:29.480 INFO:tasks.workunit.client.0.smithi107.stdout:1/369: getdents d2/dd/d1b 0 2024-04-25T00:18:29.482 INFO:tasks.workunit.client.0.smithi107.stdout:1/370: mknod d2/c76 0 2024-04-25T00:18:29.483 INFO:tasks.workunit.client.0.smithi107.stdout:1/371: creat d2/d62/d66/d70/f77 x:0 0 0 2024-04-25T00:18:29.483 INFO:tasks.workunit.client.0.smithi107.stdout:1/372: write d2/d62/d66/d70/f77 [240373,50763] 0 2024-04-25T00:18:29.484 INFO:tasks.workunit.client.0.smithi107.stdout:2/453: dread d6/ff [0,4194304] 0 2024-04-25T00:18:29.485 INFO:tasks.workunit.client.0.smithi107.stdout:1/373: rename d2/dd/f21 to d2/d62/d66/f78 0 2024-04-25T00:18:29.486 INFO:tasks.workunit.client.0.smithi107.stdout:1/374: truncate d2/dd/d25/f2e 4263369 0 2024-04-25T00:18:29.496 INFO:tasks.workunit.client.0.smithi107.stdout:1/375: dread d2/dd/d1a/f27 [0,4194304] 0 2024-04-25T00:18:29.496 INFO:tasks.workunit.client.0.smithi107.stdout:1/376: stat d2/da/f15 0 2024-04-25T00:18:29.506 INFO:tasks.workunit.client.0.smithi107.stdout:1/377: write d2/da/f43 [2307427,103403] 0 2024-04-25T00:18:29.509 INFO:tasks.workunit.client.0.smithi107.stdout:1/378: rename d2/dd/d1b/c73 to d2/c79 0 2024-04-25T00:18:29.520 INFO:tasks.workunit.client.0.smithi107.stdout:1/379: dread d2/f30 [4194304,4194304] 0 2024-04-25T00:18:29.522 INFO:tasks.workunit.client.0.smithi107.stdout:1/380: creat d2/d62/d66/d70/f7a x:0 0 0 2024-04-25T00:18:29.524 INFO:tasks.workunit.client.0.smithi107.stdout:1/381: rename d2/d2a/f4a to d2/da/f7b 0 2024-04-25T00:18:29.524 INFO:tasks.workunit.client.0.smithi107.stdout:1/382: chown d2/dd/f4b 5500 1 2024-04-25T00:18:29.525 INFO:tasks.workunit.client.0.smithi107.stdout:5/504: sync 2024-04-25T00:18:29.526 INFO:tasks.workunit.client.0.smithi107.stdout:1/383: write d2/dd/d25/d5e/f39 [150499,57929] 0 2024-04-25T00:18:29.526 INFO:tasks.workunit.client.0.smithi107.stdout:1/384: write d2/d2a/f38 [4461507,39757] 0 2024-04-25T00:18:29.526 INFO:tasks.workunit.client.0.smithi107.stdout:1/385: write d2/dd/d25/f33 [2028640,70226] 0 2024-04-25T00:18:29.526 INFO:tasks.workunit.client.0.smithi107.stdout:1/386: fsync d2/dd/d25/f5f 0 2024-04-25T00:18:29.530 INFO:tasks.workunit.client.0.smithi107.stdout:1/387: rename d2/dd/c1e to d2/d2a/d37/c7c 0 2024-04-25T00:18:29.533 INFO:tasks.workunit.client.0.smithi107.stdout:1/388: creat d2/dd/d25/f7d x:0 0 0 2024-04-25T00:18:29.533 INFO:tasks.workunit.client.0.smithi107.stdout:5/505: dread d4/dc/f2f [0,4194304] 0 2024-04-25T00:18:29.534 INFO:tasks.workunit.client.0.smithi107.stdout:1/389: creat d2/f7e x:0 0 0 2024-04-25T00:18:29.535 INFO:tasks.workunit.client.0.smithi107.stdout:5/506: chown d4/dc/d26/d29/d51/d78/c95 11171 1 2024-04-25T00:18:29.537 INFO:tasks.workunit.client.0.smithi107.stdout:1/390: creat d2/dd/d1b/f7f x:0 0 0 2024-04-25T00:18:29.537 INFO:tasks.workunit.client.0.smithi107.stdout:1/391: write d2/d2a/d5c/f71 [745897,26472] 0 2024-04-25T00:18:29.537 INFO:tasks.workunit.client.0.smithi107.stdout:5/507: creat d4/d1c/fa8 x:0 0 0 2024-04-25T00:18:29.548 INFO:tasks.workunit.client.0.smithi107.stdout:5/508: dread d4/dc/f13 [0,4194304] 0 2024-04-25T00:18:29.567 INFO:tasks.workunit.client.0.smithi107.stdout:5/509: dwrite d4/d17/f85 [0,4194304] 0 2024-04-25T00:18:29.577 INFO:tasks.workunit.client.0.smithi107.stdout:5/510: write d4/d17/f85 [3111943,59257] 0 2024-04-25T00:18:29.585 INFO:tasks.workunit.client.0.smithi107.stdout:5/511: dread d4/dc/d26/d42/f5e [0,4194304] 0 2024-04-25T00:18:29.586 INFO:tasks.workunit.client.0.smithi107.stdout:5/512: creat d4/dc/d26/d42/d8b/fa9 x:0 0 0 2024-04-25T00:18:29.587 INFO:tasks.workunit.client.0.smithi107.stdout:5/513: truncate d4/dc/d26/d29/d2e/d82/fa6 388936 0 2024-04-25T00:18:29.588 INFO:tasks.workunit.client.0.smithi107.stdout:5/514: symlink d4/d1c/d5c/laa 0 2024-04-25T00:18:29.588 INFO:tasks.workunit.client.0.smithi107.stdout:5/515: readlink d4/dc/d26/d29/d2e/l25 0 2024-04-25T00:18:29.602 INFO:tasks.workunit.client.0.smithi107.stdout:5/516: dwrite d4/dc/d26/d29/d2e/f6d [0,4194304] 0 2024-04-25T00:18:29.606 INFO:tasks.workunit.client.0.smithi107.stdout:5/517: rename d4/dc/d26/d29/d2e/d82/c5a to d4/d35/d61/d84/d93/cab 0 2024-04-25T00:18:29.608 INFO:tasks.workunit.client.0.smithi107.stdout:5/518: symlink d4/dc/d26/d29/d2e/lac 0 2024-04-25T00:18:29.609 INFO:tasks.workunit.client.0.smithi107.stdout:5/519: link d4/d17/l19 d4/d7c/d88/d8e/lad 0 2024-04-25T00:18:29.609 INFO:tasks.workunit.client.0.smithi107.stdout:5/520: readlink d4/dc/d26/d29/l65 0 2024-04-25T00:18:29.609 INFO:tasks.workunit.client.0.smithi107.stdout:5/521: stat d4/dc/d26/d29/d51/f57 0 2024-04-25T00:18:29.610 INFO:tasks.workunit.client.0.smithi107.stdout:0/358: sync 2024-04-25T00:18:29.610 INFO:tasks.workunit.client.0.smithi107.stdout:8/484: sync 2024-04-25T00:18:29.610 INFO:tasks.workunit.client.0.smithi107.stdout:7/273: sync 2024-04-25T00:18:29.610 INFO:tasks.workunit.client.0.smithi107.stdout:4/618: sync 2024-04-25T00:18:29.611 INFO:tasks.workunit.client.0.smithi107.stdout:6/370: sync 2024-04-25T00:18:29.611 INFO:tasks.workunit.client.0.smithi107.stdout:0/359: fdatasync dc/d24/f3a 0 2024-04-25T00:18:29.611 INFO:tasks.workunit.client.0.smithi107.stdout:6/371: chown d1/d2/db/d32/d36/d3d/f5e 309416 1 2024-04-25T00:18:29.611 INFO:tasks.workunit.client.0.smithi107.stdout:6/372: dread - d1/d2/db/d32/f73 zero size 2024-04-25T00:18:29.611 INFO:tasks.workunit.client.0.smithi107.stdout:6/373: chown d1/d2/db/d3c 114976986 1 2024-04-25T00:18:29.612 INFO:tasks.workunit.client.0.smithi107.stdout:8/485: symlink d2/d18/d1a/d5e/d3d/la9 0 2024-04-25T00:18:29.612 INFO:tasks.workunit.client.0.smithi107.stdout:6/374: mkdir d1/d2/d3/d71/d81 0 2024-04-25T00:18:29.612 INFO:tasks.workunit.client.0.smithi107.stdout:3/372: sync 2024-04-25T00:18:29.612 INFO:tasks.workunit.client.0.smithi107.stdout:9/350: sync 2024-04-25T00:18:29.612 INFO:tasks.workunit.client.0.smithi107.stdout:3/373: chown d9/l66 452804 1 2024-04-25T00:18:29.612 INFO:tasks.workunit.client.0.smithi107.stdout:3/374: chown d9/d15/l43 1 1 2024-04-25T00:18:29.612 INFO:tasks.workunit.client.0.smithi107.stdout:3/375: truncate d9/d15/d1a/d2c/f37 520131 0 2024-04-25T00:18:29.612 INFO:tasks.workunit.client.0.smithi107.stdout:5/522: mknod d4/dc/cae 0 2024-04-25T00:18:29.614 INFO:tasks.workunit.client.0.smithi107.stdout:1/392: sync 2024-04-25T00:18:29.614 INFO:tasks.workunit.client.0.smithi107.stdout:2/454: sync 2024-04-25T00:18:29.614 INFO:tasks.workunit.client.0.smithi107.stdout:9/351: dread d0/d43/fd [0,4194304] 0 2024-04-25T00:18:29.615 INFO:tasks.workunit.client.0.smithi107.stdout:3/376: creat d9/d54/d5c/d68/d44/f77 x:0 0 0 2024-04-25T00:18:29.615 INFO:tasks.workunit.client.0.smithi107.stdout:3/377: write d9/d15/d1a/d2c/f75 [563291,22966] 0 2024-04-25T00:18:29.615 INFO:tasks.workunit.client.0.smithi107.stdout:0/360: symlink dc/dd/d16/l69 0 2024-04-25T00:18:29.617 INFO:tasks.workunit.client.0.smithi107.stdout:8/486: unlink d2/f27 0 2024-04-25T00:18:29.618 INFO:tasks.workunit.client.0.smithi107.stdout:9/352: creat d0/d8/d56/d76/d50/f7a x:0 0 0 2024-04-25T00:18:29.618 INFO:tasks.workunit.client.0.smithi107.stdout:3/378: symlink d9/d54/d5c/d68/d44/l78 0 2024-04-25T00:18:29.619 INFO:tasks.workunit.client.0.smithi107.stdout:3/379: fdatasync d9/f5e 0 2024-04-25T00:18:29.620 INFO:tasks.workunit.client.0.smithi107.stdout:5/523: rename d4/dc/l66 to d4/dc/d26/d29/d51/laf 0 2024-04-25T00:18:29.620 INFO:tasks.workunit.client.0.smithi107.stdout:9/353: dread d0/d1b/f2f [0,4194304] 0 2024-04-25T00:18:29.620 INFO:tasks.workunit.client.0.smithi107.stdout:3/380: creat d9/d54/d5c/d68/d44/f79 x:0 0 0 2024-04-25T00:18:29.623 INFO:tasks.workunit.client.0.smithi107.stdout:7/274: dread d0/f11 [0,4194304] 0 2024-04-25T00:18:29.623 INFO:tasks.workunit.client.0.smithi107.stdout:1/393: rename d2/d2a/c31 to d2/da/d2f/c80 0 2024-04-25T00:18:29.625 INFO:tasks.workunit.client.0.smithi107.stdout:3/381: dread d9/f56 [0,4194304] 0 2024-04-25T00:18:29.626 INFO:tasks.workunit.client.0.smithi107.stdout:4/619: dread d3/d10/f8b [0,4194304] 0 2024-04-25T00:18:29.627 INFO:tasks.workunit.client.0.smithi107.stdout:0/361: rename dc/dd/d16/d33/d35/d5f/f62 to dc/dd/d16/d48/f6a 0 2024-04-25T00:18:29.627 INFO:tasks.workunit.client.0.smithi107.stdout:0/362: chown dc/dd/d16/d33 2101453 1 2024-04-25T00:18:29.629 INFO:tasks.workunit.client.0.smithi107.stdout:4/620: creat d3/d5/d32/d53/d7e/fd0 x:0 0 0 2024-04-25T00:18:29.629 INFO:tasks.workunit.client.0.smithi107.stdout:2/455: read d6/d18/d4d/d67/f49 [2218928,82107] 0 2024-04-25T00:18:29.629 INFO:tasks.workunit.client.0.smithi107.stdout:2/456: stat d6/d18/d40/d42/f47 0 2024-04-25T00:18:29.629 INFO:tasks.workunit.client.0.smithi107.stdout:2/457: chown d6/d18/d40/d42/d69/d82 28680183 1 2024-04-25T00:18:29.630 INFO:tasks.workunit.client.0.smithi107.stdout:2/458: stat d6/d18/d40/d42/f57 0 2024-04-25T00:18:29.630 INFO:tasks.workunit.client.0.smithi107.stdout:2/459: readlink d6/d18/d40/d42/d69/d6e/l79 0 2024-04-25T00:18:29.630 INFO:tasks.workunit.client.0.smithi107.stdout:2/460: chown d6/d20/f89 1 1 2024-04-25T00:18:29.630 INFO:tasks.workunit.client.0.smithi107.stdout:2/461: readlink d6/d18/d40/d42/d69/d6e/l9c 0 2024-04-25T00:18:29.630 INFO:tasks.workunit.client.0.smithi107.stdout:2/462: dread - d6/d18/f65 zero size 2024-04-25T00:18:29.630 INFO:tasks.workunit.client.0.smithi107.stdout:2/463: write d6/d20/d22/d56/f68 [2835378,105468] 0 2024-04-25T00:18:29.630 INFO:tasks.workunit.client.0.smithi107.stdout:6/375: dwrite d1/d2/d3/d28/d79/d6b/f6f [0,4194304] 0 2024-04-25T00:18:29.631 INFO:tasks.workunit.client.0.smithi107.stdout:7/275: mkdir d0/d8/db/d18/d19/d67 0 2024-04-25T00:18:29.631 INFO:tasks.workunit.client.0.smithi107.stdout:8/487: rename d2/d18/d1a/d5e/d4f/f52 to d2/d18/d1a/d5e/d3e/faa 0 2024-04-25T00:18:29.631 INFO:tasks.workunit.client.0.smithi107.stdout:8/488: fdatasync d2/d18/d1a/d5e/d6d/f6f 0 2024-04-25T00:18:29.631 INFO:tasks.workunit.client.0.smithi107.stdout:8/489: chown d2/c5b 261931257 1 2024-04-25T00:18:29.632 INFO:tasks.workunit.client.0.smithi107.stdout:3/382: write d9/d15/d1a/f30 [992093,111674] 0 2024-04-25T00:18:29.632 INFO:tasks.workunit.client.0.smithi107.stdout:6/376: unlink d1/d2/db/d32/d36/d7d/l7f 0 2024-04-25T00:18:29.633 INFO:tasks.workunit.client.0.smithi107.stdout:7/276: chown d0/d8/db/d56 90244 1 2024-04-25T00:18:29.633 INFO:tasks.workunit.client.0.smithi107.stdout:7/277: stat d0/d8/db/d18/d5c 0 2024-04-25T00:18:29.633 INFO:tasks.workunit.client.0.smithi107.stdout:7/278: rmdir d0/d8/d4e 39 2024-04-25T00:18:29.634 INFO:tasks.workunit.client.0.smithi107.stdout:5/524: rename d4/d7c/d88/f92 to d4/d35/d61/d84/fb0 0 2024-04-25T00:18:29.634 INFO:tasks.workunit.client.0.smithi107.stdout:5/525: fsync d4/dc/d26/d29/d51/f57 0 2024-04-25T00:18:29.635 INFO:tasks.workunit.client.0.smithi107.stdout:7/279: mknod d0/d8/d2c/d58/c68 0 2024-04-25T00:18:29.637 INFO:tasks.workunit.client.0.smithi107.stdout:8/490: creat d2/d18/d1a/d33/fab x:0 0 0 2024-04-25T00:18:29.637 INFO:tasks.workunit.client.0.smithi107.stdout:8/491: read d2/d18/d1a/f25 [265489,8770] 0 2024-04-25T00:18:29.639 INFO:tasks.workunit.client.0.smithi107.stdout:9/354: rename d0/d43/f3c to d0/d8/de/d5c/f7b 0 2024-04-25T00:18:29.639 INFO:tasks.workunit.client.0.smithi107.stdout:9/355: chown d0/d8/d56/d76/f38 15580 1 2024-04-25T00:18:29.640 INFO:tasks.workunit.client.0.smithi107.stdout:1/394: read d2/da/d2f/f3e [1744647,65742] 0 2024-04-25T00:18:29.641 INFO:tasks.workunit.client.0.smithi107.stdout:7/280: dread d0/d8/db/d18/d19/d39/f3e [0,4194304] 0 2024-04-25T00:18:29.641 INFO:tasks.workunit.client.0.smithi107.stdout:6/377: dread d1/d2/db/d32/d36/d3d/f60 [0,4194304] 0 2024-04-25T00:18:29.641 INFO:tasks.workunit.client.0.smithi107.stdout:7/281: chown d0/d8/db/d18/d19/d39/c45 34 1 2024-04-25T00:18:29.641 INFO:tasks.workunit.client.0.smithi107.stdout:8/492: creat d2/d18/d1a/d5e/d4f/da0/fac x:0 0 0 2024-04-25T00:18:29.641 INFO:tasks.workunit.client.0.smithi107.stdout:4/621: write d3/d5/f41 [1739362,7160] 0 2024-04-25T00:18:29.642 INFO:tasks.workunit.client.0.smithi107.stdout:6/378: symlink d1/d2/db/l82 0 2024-04-25T00:18:29.643 INFO:tasks.workunit.client.0.smithi107.stdout:9/356: creat d0/d8/de/f7c x:0 0 0 2024-04-25T00:18:29.645 INFO:tasks.workunit.client.0.smithi107.stdout:0/363: rename dc/dd/d16/f22 to dc/dd/d16/d48/f6b 0 2024-04-25T00:18:29.645 INFO:tasks.workunit.client.0.smithi107.stdout:1/395: rmdir d2/d62/d66/d70 39 2024-04-25T00:18:29.647 INFO:tasks.workunit.client.0.smithi107.stdout:6/379: creat d1/d2/f83 x:0 0 0 2024-04-25T00:18:29.649 INFO:tasks.workunit.client.0.smithi107.stdout:8/493: creat d2/d18/d1a/d5e/d4f/da0/da1/fad x:0 0 0 2024-04-25T00:18:29.649 INFO:tasks.workunit.client.0.smithi107.stdout:8/494: stat d2/d18/d1a/d5e/d3e/faa 0 2024-04-25T00:18:29.649 INFO:tasks.workunit.client.0.smithi107.stdout:8/495: chown d2/d18/d1a/d5e/d4f/f7e 415105502 1 2024-04-25T00:18:29.649 INFO:tasks.workunit.client.0.smithi107.stdout:8/496: write d2/d18/f68 [2804235,50128] 0 2024-04-25T00:18:29.652 INFO:tasks.workunit.client.0.smithi107.stdout:6/380: symlink d1/d2/db/d32/d36/d3d/l84 0 2024-04-25T00:18:29.652 INFO:tasks.workunit.client.0.smithi107.stdout:6/381: dread - d1/d2/db/d32/f73 zero size 2024-04-25T00:18:29.653 INFO:tasks.workunit.client.0.smithi107.stdout:5/526: dwrite d4/dc/d26/d42/f5e [0,4194304] 0 2024-04-25T00:18:29.654 INFO:tasks.workunit.client.0.smithi107.stdout:2/464: rename d6/d18/d40/d42/f3c to d6/d29/fa6 0 2024-04-25T00:18:29.654 INFO:tasks.workunit.client.0.smithi107.stdout:2/465: chown d6/d18/d40/d42/d69/d6e/d80/f9a 60836 1 2024-04-25T00:18:29.655 INFO:tasks.workunit.client.0.smithi107.stdout:0/364: symlink dc/d24/d38/l6c 0 2024-04-25T00:18:29.656 INFO:tasks.workunit.client.0.smithi107.stdout:4/622: creat d3/d24/d5c/fd1 x:0 0 0 2024-04-25T00:18:29.656 INFO:tasks.workunit.client.0.smithi107.stdout:6/382: symlink d1/d14/d18/l85 0 2024-04-25T00:18:29.657 INFO:tasks.workunit.client.0.smithi107.stdout:5/527: getdents d4/d35/d61 0 2024-04-25T00:18:29.657 INFO:tasks.workunit.client.0.smithi107.stdout:5/528: stat d4/d1c/l97 0 2024-04-25T00:18:29.657 INFO:tasks.workunit.client.0.smithi107.stdout:6/383: truncate d1/d2/d3/f31 117958 0 2024-04-25T00:18:29.658 INFO:tasks.workunit.client.0.smithi107.stdout:5/529: creat d4/fb1 x:0 0 0 2024-04-25T00:18:29.658 INFO:tasks.workunit.client.0.smithi107.stdout:7/282: rename d0/d8/d4e/f40 to d0/f69 0 2024-04-25T00:18:29.659 INFO:tasks.workunit.client.0.smithi107.stdout:4/623: rmdir d3 39 2024-04-25T00:18:29.661 INFO:tasks.workunit.client.0.smithi107.stdout:9/357: rename d0/d8/d56/d76/d69 to d0/d8/d56/d76/d50/d7d 0 2024-04-25T00:18:29.661 INFO:tasks.workunit.client.0.smithi107.stdout:7/283: creat d0/d8/db/d18/d19/d47/d4d/f6a x:0 0 0 2024-04-25T00:18:29.661 INFO:tasks.workunit.client.0.smithi107.stdout:4/624: fsync d3/d5/d32/d53/d7e/fb3 0 2024-04-25T00:18:29.662 INFO:tasks.workunit.client.0.smithi107.stdout:9/358: creat d0/f7e x:0 0 0 2024-04-25T00:18:29.662 INFO:tasks.workunit.client.0.smithi107.stdout:9/359: dread - d0/d8/d56/d76/f78 zero size 2024-04-25T00:18:29.662 INFO:tasks.workunit.client.0.smithi107.stdout:9/360: chown d0/d1b/c65 4911735 1 2024-04-25T00:18:29.664 INFO:tasks.workunit.client.0.smithi107.stdout:7/284: creat d0/d8/d4e/d46/f6b x:0 0 0 2024-04-25T00:18:29.664 INFO:tasks.workunit.client.0.smithi107.stdout:9/361: mknod d0/d43/c7f 0 2024-04-25T00:18:29.664 INFO:tasks.workunit.client.0.smithi107.stdout:9/362: write d0/d8/d56/f59 [1038512,38251] 0 2024-04-25T00:18:29.665 INFO:tasks.workunit.client.0.smithi107.stdout:7/285: creat d0/d8/db/d18/d19/d39/f6c x:0 0 0 2024-04-25T00:18:29.666 INFO:tasks.workunit.client.0.smithi107.stdout:4/625: dread d3/d24/d5c/dab/da3/fa6 [0,4194304] 0 2024-04-25T00:18:29.666 INFO:tasks.workunit.client.0.smithi107.stdout:9/363: mkdir d0/d8/d56/d6c/d80 0 2024-04-25T00:18:29.667 INFO:tasks.workunit.client.0.smithi107.stdout:4/626: creat d3/d10/d7b/fd2 x:0 0 0 2024-04-25T00:18:29.674 INFO:tasks.workunit.client.0.smithi107.stdout:8/497: dwrite d2/d18/d1a/d5e/d6d/f6f [0,4194304] 0 2024-04-25T00:18:29.676 INFO:tasks.workunit.client.0.smithi107.stdout:4/627: write d3/d5/d32/f45 [86854,98931] 0 2024-04-25T00:18:29.676 INFO:tasks.workunit.client.0.smithi107.stdout:4/628: chown l1 2002894654 1 2024-04-25T00:18:29.676 INFO:tasks.workunit.client.0.smithi107.stdout:4/629: dread - d3/d24/d25/d71/d7f/fc5 zero size 2024-04-25T00:18:29.678 INFO:tasks.workunit.client.0.smithi107.stdout:4/630: symlink d3/da8/ld3 0 2024-04-25T00:18:29.678 INFO:tasks.workunit.client.0.smithi107.stdout:4/631: chown d3/d24/d25/c55 2655 1 2024-04-25T00:18:29.679 INFO:tasks.workunit.client.0.smithi107.stdout:4/632: write d3/d5/f92 [1739178,34832] 0 2024-04-25T00:18:29.681 INFO:tasks.workunit.client.0.smithi107.stdout:2/466: dwrite d6/d18/d40/d42/d2b/d30/d45/f61 [0,4194304] 0 2024-04-25T00:18:29.687 INFO:tasks.workunit.client.0.smithi107.stdout:2/467: symlink d6/d18/d40/d42/d2b/d30/d4c/la7 0 2024-04-25T00:18:29.688 INFO:tasks.workunit.client.0.smithi107.stdout:2/468: chown d6/d29 5060993 1 2024-04-25T00:18:29.688 INFO:tasks.workunit.client.0.smithi107.stdout:9/364: write d0/d8/d56/d76/f38 [3667474,14174] 0 2024-04-25T00:18:29.690 INFO:tasks.workunit.client.0.smithi107.stdout:8/498: dread d2/d18/d1a/d5e/d3e/f4c [0,4194304] 0 2024-04-25T00:18:29.691 INFO:tasks.workunit.client.0.smithi107.stdout:0/365: dwrite dc/dd/d16/d33/f3d [0,4194304] 0 2024-04-25T00:18:29.691 INFO:tasks.workunit.client.0.smithi107.stdout:9/365: mknod d0/d8/d1f/c81 0 2024-04-25T00:18:29.692 INFO:tasks.workunit.client.0.smithi107.stdout:5/530: dwrite d4/dc/f20 [4194304,4194304] 0 2024-04-25T00:18:29.692 INFO:tasks.workunit.client.0.smithi107.stdout:9/366: creat d0/d8/d6f/f82 x:0 0 0 2024-04-25T00:18:29.692 INFO:tasks.workunit.client.0.smithi107.stdout:9/367: fsync d0/d43/f10 0 2024-04-25T00:18:29.693 INFO:tasks.workunit.client.0.smithi107.stdout:7/286: dwrite d0/d8/db/d18/d19/f2f [0,4194304] 0 2024-04-25T00:18:29.694 INFO:tasks.workunit.client.0.smithi107.stdout:5/531: creat d4/d17/fb2 x:0 0 0 2024-04-25T00:18:29.695 INFO:tasks.workunit.client.0.smithi107.stdout:9/368: creat d0/d8/de/f83 x:0 0 0 2024-04-25T00:18:29.696 INFO:tasks.workunit.client.0.smithi107.stdout:4/633: write d3/d24/d25/d71/d7f/f9c [3767231,20062] 0 2024-04-25T00:18:29.696 INFO:tasks.workunit.client.0.smithi107.stdout:4/634: stat d3/d5/l4c 0 2024-04-25T00:18:29.698 INFO:tasks.workunit.client.0.smithi107.stdout:5/532: truncate d4/dc/d26/f8c 832041 0 2024-04-25T00:18:29.698 INFO:tasks.workunit.client.0.smithi107.stdout:7/287: creat d0/d8/f6d x:0 0 0 2024-04-25T00:18:29.698 INFO:tasks.workunit.client.0.smithi107.stdout:0/366: rmdir dc/dd/d16/d48 39 2024-04-25T00:18:29.698 INFO:tasks.workunit.client.0.smithi107.stdout:0/367: chown dc/d24/d38/d64 39828932 1 2024-04-25T00:18:29.699 INFO:tasks.workunit.client.0.smithi107.stdout:4/635: creat d3/d5/d32/d53/d7e/fd4 x:0 0 0 2024-04-25T00:18:29.699 INFO:tasks.workunit.client.0.smithi107.stdout:9/369: creat d0/d8/d34/f84 x:0 0 0 2024-04-25T00:18:29.699 INFO:tasks.workunit.client.0.smithi107.stdout:9/370: stat d0/d8/de/c6b 0 2024-04-25T00:18:29.700 INFO:tasks.workunit.client.0.smithi107.stdout:7/288: mknod d0/d8/db/d18/d19/d47/c6e 0 2024-04-25T00:18:29.700 INFO:tasks.workunit.client.0.smithi107.stdout:9/371: creat d0/d8/d56/d76/d50/d7d/f85 x:0 0 0 2024-04-25T00:18:29.701 INFO:tasks.workunit.client.0.smithi107.stdout:7/289: dread - d0/f69 zero size 2024-04-25T00:18:29.701 INFO:tasks.workunit.client.0.smithi107.stdout:7/290: truncate d0/d8/d4e/f32 204587 0 2024-04-25T00:18:29.702 INFO:tasks.workunit.client.0.smithi107.stdout:9/372: creat d0/d8/d1f/f86 x:0 0 0 2024-04-25T00:18:29.703 INFO:tasks.workunit.client.0.smithi107.stdout:7/291: rename d0/d8/d2c/f50 to d0/d8/db/d18/d19/d39/f6f 0 2024-04-25T00:18:29.703 INFO:tasks.workunit.client.0.smithi107.stdout:9/373: rename d0/d1b/c65 to d0/d1b/c87 0 2024-04-25T00:18:29.704 INFO:tasks.workunit.client.0.smithi107.stdout:2/469: write f4 [4152790,129486] 0 2024-04-25T00:18:29.704 INFO:tasks.workunit.client.0.smithi107.stdout:0/368: write dc/d24/d25/d31/f36 [104044,31923] 0 2024-04-25T00:18:29.705 INFO:tasks.workunit.client.0.smithi107.stdout:7/292: rmdir d0/d59 0 2024-04-25T00:18:29.706 INFO:tasks.workunit.client.0.smithi107.stdout:9/374: mknod d0/d8/d34/d36/d77/c88 0 2024-04-25T00:18:29.707 INFO:tasks.workunit.client.0.smithi107.stdout:5/533: read d4/dc/d26/f8c [301982,83408] 0 2024-04-25T00:18:29.709 INFO:tasks.workunit.client.0.smithi107.stdout:2/470: mkdir d6/d18/d4d/d67/da8 0 2024-04-25T00:18:29.710 INFO:tasks.workunit.client.0.smithi107.stdout:7/293: rename d0/f16 to d0/d8/d4e/f70 0 2024-04-25T00:18:29.712 INFO:tasks.workunit.client.0.smithi107.stdout:9/375: symlink d0/d8/d1f/l89 0 2024-04-25T00:18:29.712 INFO:tasks.workunit.client.0.smithi107.stdout:9/376: chown d0/d8/c5f 52 1 2024-04-25T00:18:29.712 INFO:tasks.workunit.client.0.smithi107.stdout:8/499: dwrite d2/d18/d1a/d5e/d3d/f9d [0,4194304] 0 2024-04-25T00:18:29.713 INFO:tasks.workunit.client.0.smithi107.stdout:8/500: chown d2/d18/d1a/d5e/d3e/l4d 10196114 1 2024-04-25T00:18:29.713 INFO:tasks.workunit.client.0.smithi107.stdout:2/471: mkdir d6/d18/d40/da9 0 2024-04-25T00:18:29.716 INFO:tasks.workunit.client.0.smithi107.stdout:5/534: rename d4/d35/d61 to d4/d17/db3 0 2024-04-25T00:18:29.716 INFO:tasks.workunit.client.0.smithi107.stdout:5/535: fdatasync d4/dc/d26/d29/d2e/d55/fa0 0 2024-04-25T00:18:29.717 INFO:tasks.workunit.client.0.smithi107.stdout:4/636: dwrite d3/d5/f12 [0,4194304] 0 2024-04-25T00:18:29.718 INFO:tasks.workunit.client.0.smithi107.stdout:9/377: mkdir d0/d8a 0 2024-04-25T00:18:29.721 INFO:tasks.workunit.client.0.smithi107.stdout:5/536: creat d4/dc/d26/d42/fb4 x:0 0 0 2024-04-25T00:18:29.723 INFO:tasks.workunit.client.0.smithi107.stdout:5/537: fsync d4/dc/d26/d42/fb4 0 2024-04-25T00:18:29.723 INFO:tasks.workunit.client.0.smithi107.stdout:2/472: mknod d6/d18/d40/d42/d69/d6e/da2/caa 0 2024-04-25T00:18:29.726 INFO:tasks.workunit.client.0.smithi107.stdout:7/294: unlink d0/d8/db/fc 0 2024-04-25T00:18:29.726 INFO:tasks.workunit.client.0.smithi107.stdout:7/295: dread - d0/d8/db/d18/d19/d39/f6c zero size 2024-04-25T00:18:29.727 INFO:tasks.workunit.client.0.smithi107.stdout:7/296: symlink d0/d8/d4e/l71 0 2024-04-25T00:18:29.728 INFO:tasks.workunit.client.0.smithi107.stdout:9/378: link d0/d43/f4 d0/d8/d1f/f8b 0 2024-04-25T00:18:29.728 INFO:tasks.workunit.client.0.smithi107.stdout:9/379: read - d0/d8/f45 zero size 2024-04-25T00:18:29.728 INFO:tasks.workunit.client.0.smithi107.stdout:8/501: dread d2/d18/d1a/d5e/d3e/d74/f8a [0,4194304] 0 2024-04-25T00:18:29.728 INFO:tasks.workunit.client.0.smithi107.stdout:7/297: creat d0/d8/db/d18/d19/d67/f72 x:0 0 0 2024-04-25T00:18:29.728 INFO:tasks.workunit.client.0.smithi107.stdout:7/298: readlink d0/d8/db/d18/d19/d39/l4b 0 2024-04-25T00:18:29.729 INFO:tasks.workunit.client.0.smithi107.stdout:5/538: rmdir d4/d1c 39 2024-04-25T00:18:29.729 INFO:tasks.workunit.client.0.smithi107.stdout:4/637: write d3/d5/d32/d53/d7e/f86 [183865,86188] 0 2024-04-25T00:18:29.731 INFO:tasks.workunit.client.0.smithi107.stdout:5/539: rename d4/d17/c43 to d4/d91/cb5 0 2024-04-25T00:18:29.732 INFO:tasks.workunit.client.0.smithi107.stdout:4/638: getdents d3/d24/d5c/d7a 0 2024-04-25T00:18:29.735 INFO:tasks.workunit.client.0.smithi107.stdout:4/639: link d3/d24/f69 d3/d24/d25/d71/fd5 0 2024-04-25T00:18:29.735 INFO:tasks.workunit.client.0.smithi107.stdout:7/299: dread d0/d8/d4e/d46/f30 [0,4194304] 0 2024-04-25T00:18:29.735 INFO:tasks.workunit.client.0.smithi107.stdout:7/300: chown d0/d8/db/d18/d19/d39/l5a 392492 1 2024-04-25T00:18:29.735 INFO:tasks.workunit.client.0.smithi107.stdout:7/301: chown d0/d8/d2c/d58/c68 2064318 1 2024-04-25T00:18:29.735 INFO:tasks.workunit.client.0.smithi107.stdout:7/302: chown d0/d8/d2c/d58/f5f 870700 1 2024-04-25T00:18:29.735 INFO:tasks.workunit.client.0.smithi107.stdout:0/369: getdents dc/dd/d16 0 2024-04-25T00:18:29.736 INFO:tasks.workunit.client.0.smithi107.stdout:4/640: symlink d3/d5/d32/d53/d7e/d9d/ld6 0 2024-04-25T00:18:29.737 INFO:tasks.workunit.client.0.smithi107.stdout:4/641: truncate d3/d5/d32/f45 2892592 0 2024-04-25T00:18:29.737 INFO:tasks.workunit.client.0.smithi107.stdout:4/642: chown d3/d5/d32/d53/d97/f99 80646 1 2024-04-25T00:18:29.737 INFO:tasks.workunit.client.0.smithi107.stdout:7/303: rename d0/d8/d4e/f70 to d0/d8/f73 0 2024-04-25T00:18:29.738 INFO:tasks.workunit.client.0.smithi107.stdout:4/643: creat d3/d24/d5c/dab/fd7 x:0 0 0 2024-04-25T00:18:29.738 INFO:tasks.workunit.client.0.smithi107.stdout:9/380: dread d0/d8/d56/f59 [0,4194304] 0 2024-04-25T00:18:29.738 INFO:tasks.workunit.client.0.smithi107.stdout:9/381: read - d0/d8/d6f/f82 zero size 2024-04-25T00:18:29.740 INFO:tasks.workunit.client.0.smithi107.stdout:9/382: creat d0/d8/d34/d36/f8c x:0 0 0 2024-04-25T00:18:29.740 INFO:tasks.workunit.client.0.smithi107.stdout:9/383: write d0/f7e [257317,62116] 0 2024-04-25T00:18:29.740 INFO:tasks.workunit.client.0.smithi107.stdout:9/384: write d0/d8/f55 [3886508,56961] 0 2024-04-25T00:18:29.740 INFO:tasks.workunit.client.0.smithi107.stdout:9/385: write d0/d8/d56/d76/f38 [4867751,104839] 0 2024-04-25T00:18:29.740 INFO:tasks.workunit.client.0.smithi107.stdout:9/386: readlink d0/d8/d1f/l44 0 2024-04-25T00:18:29.740 INFO:tasks.workunit.client.0.smithi107.stdout:9/387: truncate d0/d8/d56/d76/d50/d7d/f85 627975 0 2024-04-25T00:18:29.740 INFO:tasks.workunit.client.0.smithi107.stdout:9/388: write d0/d8/d56/d76/f46 [283123,123208] 0 2024-04-25T00:18:29.740 INFO:tasks.workunit.client.0.smithi107.stdout:9/389: chown d0/d43/l12 58220 1 2024-04-25T00:18:29.740 INFO:tasks.workunit.client.0.smithi107.stdout:4/644: mkdir d3/dd8 0 2024-04-25T00:18:29.741 INFO:tasks.workunit.client.0.smithi107.stdout:4/645: chown d3/d24/d25/c56 92704 1 2024-04-25T00:18:29.742 INFO:tasks.workunit.client.0.smithi107.stdout:7/304: mknod d0/d8/db/d18/d19/c74 0 2024-04-25T00:18:29.744 INFO:tasks.workunit.client.0.smithi107.stdout:4/646: link d3/d10/d7b/da5/lbc d3/da8/ld9 0 2024-04-25T00:18:29.744 INFO:tasks.workunit.client.0.smithi107.stdout:2/473: dwrite d6/d18/d40/d42/d70/f76 [0,4194304] 0 2024-04-25T00:18:29.745 INFO:tasks.workunit.client.0.smithi107.stdout:4/647: symlink d3/lda 0 2024-04-25T00:18:29.747 INFO:tasks.workunit.client.0.smithi107.stdout:8/502: dwrite d2/d18/d1a/d5e/f8c [0,4194304] 0 2024-04-25T00:18:29.749 INFO:tasks.workunit.client.0.smithi107.stdout:8/503: mkdir d2/d18/d1a/d5e/d3d/d85/dae 0 2024-04-25T00:18:29.751 INFO:tasks.workunit.client.0.smithi107.stdout:5/540: dwrite d4/dc/d26/f8c [0,4194304] 0 2024-04-25T00:18:29.752 INFO:tasks.workunit.client.0.smithi107.stdout:8/504: getdents d2/d18/d1a/d5e 0 2024-04-25T00:18:29.756 INFO:tasks.workunit.client.0.smithi107.stdout:2/474: dread d6/d18/d4d/d67/f49 [0,4194304] 0 2024-04-25T00:18:29.756 INFO:tasks.workunit.client.0.smithi107.stdout:5/541: dread d4/d17/f69 [0,4194304] 0 2024-04-25T00:18:29.759 INFO:tasks.workunit.client.0.smithi107.stdout:2/475: creat d6/d29/d4e/fab x:0 0 0 2024-04-25T00:18:29.760 INFO:tasks.workunit.client.0.smithi107.stdout:5/542: creat d4/d7c/d88/d8e/fb6 x:0 0 0 2024-04-25T00:18:29.764 INFO:tasks.workunit.client.0.smithi107.stdout:8/505: write d2/f83 [3909112,96960] 0 2024-04-25T00:18:29.773 INFO:tasks.workunit.client.0.smithi107.stdout:5/543: read d4/dc/d26/f30 [584961,79233] 0 2024-04-25T00:18:29.779 INFO:tasks.workunit.client.0.smithi107.stdout:3/383: sync 2024-04-25T00:18:29.779 INFO:tasks.workunit.client.0.smithi107.stdout:0/370: rename f1 to dc/dd/d5d/f6d 0 2024-04-25T00:18:29.779 INFO:tasks.workunit.client.0.smithi107.stdout:3/384: chown d9/d22/d1b/l60 400906 1 2024-04-25T00:18:29.779 INFO:tasks.workunit.client.0.smithi107.stdout:0/371: fsync dc/dd/d16/f60 0 2024-04-25T00:18:29.780 INFO:tasks.workunit.client.0.smithi107.stdout:0/372: chown dc/d24 3111 1 2024-04-25T00:18:29.780 INFO:tasks.workunit.client.0.smithi107.stdout:3/385: write d9/d54/d5c/d68/d44/f64 [520240,119473] 0 2024-04-25T00:18:29.780 INFO:tasks.workunit.client.0.smithi107.stdout:3/386: write d9/d15/d1a/f30 [1321205,51028] 0 2024-04-25T00:18:29.781 INFO:tasks.workunit.client.0.smithi107.stdout:9/390: rename d0/d8/d56/l74 to d0/d8/d6f/l8d 0 2024-04-25T00:18:29.781 INFO:tasks.workunit.client.0.smithi107.stdout:0/373: mkdir dc/dd/d5d/d5c/d6e 0 2024-04-25T00:18:29.781 INFO:tasks.workunit.client.0.smithi107.stdout:0/374: chown dc/f2f 337522 1 2024-04-25T00:18:29.781 INFO:tasks.workunit.client.0.smithi107.stdout:0/375: chown dc/dd/d16/d5e/l63 8 1 2024-04-25T00:18:29.782 INFO:tasks.workunit.client.0.smithi107.stdout:3/387: creat d9/d22/f7a x:0 0 0 2024-04-25T00:18:29.783 INFO:tasks.workunit.client.0.smithi107.stdout:7/305: rename d0/d8/f6d to d0/d8/db/d18/d5c/f75 0 2024-04-25T00:18:29.788 INFO:tasks.workunit.client.0.smithi107.stdout:9/391: symlink d0/d8a/l8e 0 2024-04-25T00:18:29.789 INFO:tasks.workunit.client.0.smithi107.stdout:9/392: rename d0/d43/f4 to d0/f8f 0 2024-04-25T00:18:29.791 INFO:tasks.workunit.client.0.smithi107.stdout:9/393: creat d0/d8/de/f90 x:0 0 0 2024-04-25T00:18:29.791 INFO:tasks.workunit.client.0.smithi107.stdout:9/394: fdatasync d0/d8/de/d5c/f72 0 2024-04-25T00:18:29.796 INFO:tasks.workunit.client.0.smithi107.stdout:5/544: dread d4/fa [0,4194304] 0 2024-04-25T00:18:29.796 INFO:tasks.workunit.client.0.smithi107.stdout:8/506: dwrite d2/d18/d1a/f31 [0,4194304] 0 2024-04-25T00:18:29.797 INFO:tasks.workunit.client.0.smithi107.stdout:8/507: rename d2/d18/d1a/d5e/d3e/c54 to d2/d18/d1a/d5e/d6d/caf 0 2024-04-25T00:18:29.798 INFO:tasks.workunit.client.0.smithi107.stdout:8/508: getdents d2/d18/d1a/d5e/d3e/d74 0 2024-04-25T00:18:29.799 INFO:tasks.workunit.client.0.smithi107.stdout:8/509: truncate d2/ff 1513882 0 2024-04-25T00:18:29.799 INFO:tasks.workunit.client.0.smithi107.stdout:8/510: chown d2/d18/d1a/d5e/d6d/caf 791 1 2024-04-25T00:18:29.800 INFO:tasks.workunit.client.0.smithi107.stdout:8/511: rmdir d2/d18/d1a/d82 39 2024-04-25T00:18:29.801 INFO:tasks.workunit.client.0.smithi107.stdout:8/512: symlink d2/d18/d1a/d5e/d3e/lb0 0 2024-04-25T00:18:29.802 INFO:tasks.workunit.client.0.smithi107.stdout:8/513: mkdir d2/d18/d1a/d5e/d3e/db1 0 2024-04-25T00:18:29.802 INFO:tasks.workunit.client.0.smithi107.stdout:8/514: fsync d2/d18/f43 0 2024-04-25T00:18:29.803 INFO:tasks.workunit.client.0.smithi107.stdout:8/515: truncate d2/d18/f53 417247 0 2024-04-25T00:18:29.804 INFO:tasks.workunit.client.0.smithi107.stdout:8/516: chown d2/d18/c46 101507 1 2024-04-25T00:18:29.806 INFO:tasks.workunit.client.0.smithi107.stdout:3/388: dwrite d9/d15/d1a/d2c/f37 [0,4194304] 0 2024-04-25T00:18:29.807 INFO:tasks.workunit.client.0.smithi107.stdout:0/376: dwrite dc/dd/f19 [0,4194304] 0 2024-04-25T00:18:29.810 INFO:tasks.workunit.client.0.smithi107.stdout:0/377: truncate dc/dd/d16/d33/f3d 2571614 0 2024-04-25T00:18:29.811 INFO:tasks.workunit.client.0.smithi107.stdout:7/306: dwrite d0/f13 [0,4194304] 0 2024-04-25T00:18:29.811 INFO:tasks.workunit.client.0.smithi107.stdout:0/378: creat dc/d24/d25/d31/f6f x:0 0 0 2024-04-25T00:18:29.813 INFO:tasks.workunit.client.0.smithi107.stdout:0/379: truncate dc/dd/d5d/f57 2131718 0 2024-04-25T00:18:29.813 INFO:tasks.workunit.client.0.smithi107.stdout:0/380: write dc/d24/d25/d31/f36 [1618727,2294] 0 2024-04-25T00:18:29.813 INFO:tasks.workunit.client.0.smithi107.stdout:0/381: chown dc/d24/f3a 56998 1 2024-04-25T00:18:29.814 INFO:tasks.workunit.client.0.smithi107.stdout:8/517: read d2/f26 [2952480,7038] 0 2024-04-25T00:18:29.814 INFO:tasks.workunit.client.0.smithi107.stdout:8/518: truncate d2/d18/d1a/d5e/d4f/f7e 135270 0 2024-04-25T00:18:29.814 INFO:tasks.workunit.client.0.smithi107.stdout:7/307: write d0/d8/db/d18/d19/d39/f3e [331450,13110] 0 2024-04-25T00:18:29.814 INFO:tasks.workunit.client.0.smithi107.stdout:5/545: dwrite d4/f1b [0,4194304] 0 2024-04-25T00:18:29.817 INFO:tasks.workunit.client.0.smithi107.stdout:8/519: mkdir d2/d18/da2/db2 0 2024-04-25T00:18:29.819 INFO:tasks.workunit.client.0.smithi107.stdout:7/308: symlink d0/d8/db/d18/d19/d39/l76 0 2024-04-25T00:18:29.824 INFO:tasks.workunit.client.0.smithi107.stdout:8/520: creat d2/d18/d1a/d5e/d3d/fb3 x:0 0 0 2024-04-25T00:18:29.824 INFO:tasks.workunit.client.0.smithi107.stdout:8/521: dread - d2/d18/d1a/f73 zero size 2024-04-25T00:18:29.829 INFO:tasks.workunit.client.0.smithi107.stdout:8/522: dread d2/d18/d1a/d5e/f47 [0,4194304] 0 2024-04-25T00:18:29.829 INFO:tasks.workunit.client.0.smithi107.stdout:8/523: chown d2/d18/d1a/d33/l78 7660227 1 2024-04-25T00:18:29.841 INFO:tasks.workunit.client.0.smithi107.stdout:3/389: rename d9/d15/d39/f3d to d9/d15/d1a/d4e/f7b 0 2024-04-25T00:18:29.841 INFO:tasks.workunit.client.0.smithi107.stdout:3/390: readlink l3 0 2024-04-25T00:18:29.842 INFO:tasks.workunit.client.0.smithi107.stdout:3/391: write d9/f61 [52542,7725] 0 2024-04-25T00:18:29.842 INFO:tasks.workunit.client.0.smithi107.stdout:5/546: dwrite d4/d17/f21 [0,4194304] 0 2024-04-25T00:18:29.846 INFO:tasks.workunit.client.0.smithi107.stdout:3/392: unlink d9/d22/d1b/l4f 0 2024-04-25T00:18:29.846 INFO:tasks.workunit.client.0.smithi107.stdout:3/393: fdatasync d9/d22/d1b/d55/f5d 0 2024-04-25T00:18:29.846 INFO:tasks.workunit.client.0.smithi107.stdout:0/382: dwrite dc/d24/d25/d31/f5b [0,4194304] 0 2024-04-25T00:18:29.847 INFO:tasks.workunit.client.0.smithi107.stdout:7/309: dwrite d0/d8/f65 [0,4194304] 0 2024-04-25T00:18:29.849 INFO:tasks.workunit.client.0.smithi107.stdout:3/394: symlink d9/d54/d5c/d68/d3b/l7c 0 2024-04-25T00:18:29.850 INFO:tasks.workunit.client.0.smithi107.stdout:3/395: rmdir d9/d22/d1b 39 2024-04-25T00:18:29.851 INFO:tasks.workunit.client.0.smithi107.stdout:3/396: mknod d9/d54/d5c/d68/d44/d4b/c7d 0 2024-04-25T00:18:29.853 INFO:tasks.workunit.client.0.smithi107.stdout:8/524: dwrite d2/d18/d1a/d5e/d4f/f7a [0,4194304] 0 2024-04-25T00:18:29.861 INFO:tasks.workunit.client.0.smithi107.stdout:3/397: getdents d9/d15/d1a 0 2024-04-25T00:18:29.861 INFO:tasks.workunit.client.0.smithi107.stdout:3/398: chown d9/l66 261800 1 2024-04-25T00:18:29.861 INFO:tasks.workunit.client.0.smithi107.stdout:3/399: readlink d9/d54/d5c/d68/d3b/l45 0 2024-04-25T00:18:29.861 INFO:tasks.workunit.client.0.smithi107.stdout:8/525: dread d2/f8 [0,4194304] 0 2024-04-25T00:18:29.864 INFO:tasks.workunit.client.0.smithi107.stdout:3/400: rename d9/d54/d5c/d68/l73 to d9/d22/d1b/l7e 0 2024-04-25T00:18:29.865 INFO:tasks.workunit.client.0.smithi107.stdout:5/547: dwrite d4/d35/f3c [0,4194304] 0 2024-04-25T00:18:29.865 INFO:tasks.workunit.client.0.smithi107.stdout:5/548: readlink d4/d17/l19 0 2024-04-25T00:18:29.867 INFO:tasks.workunit.client.0.smithi107.stdout:3/401: dread d9/f42 [0,4194304] 0 2024-04-25T00:18:29.868 INFO:tasks.workunit.client.0.smithi107.stdout:3/402: dread - d9/d22/f7a zero size 2024-04-25T00:18:29.869 INFO:tasks.workunit.client.0.smithi107.stdout:5/549: getdents d4 0 2024-04-25T00:18:29.869 INFO:tasks.workunit.client.0.smithi107.stdout:3/403: creat d9/d22/d1b/d55/f7f x:0 0 0 2024-04-25T00:18:29.870 INFO:tasks.workunit.client.0.smithi107.stdout:5/550: mknod d4/dc/d26/d62/cb7 0 2024-04-25T00:18:29.874 INFO:tasks.workunit.client.0.smithi107.stdout:3/404: getdents d9/d54/d5c/d68/d3b 0 2024-04-25T00:18:29.874 INFO:tasks.workunit.client.0.smithi107.stdout:3/405: fsync d9/d54/d5c/d68/f40 0 2024-04-25T00:18:29.874 INFO:tasks.workunit.client.0.smithi107.stdout:3/406: stat l3 0 2024-04-25T00:18:29.874 INFO:tasks.workunit.client.0.smithi107.stdout:3/407: fsync d9/d54/d5c/d68/f40 0 2024-04-25T00:18:29.875 INFO:tasks.workunit.client.0.smithi107.stdout:0/383: dwrite dc/d24/d25/f58 [0,4194304] 0 2024-04-25T00:18:29.876 INFO:tasks.workunit.client.0.smithi107.stdout:7/310: dwrite d0/d8/d2c/f55 [0,4194304] 0 2024-04-25T00:18:29.878 INFO:tasks.workunit.client.0.smithi107.stdout:5/551: rename d4/d7c/d88/d8e/fb6 to d4/dc/d26/d29/d51/fb8 0 2024-04-25T00:18:29.880 INFO:tasks.workunit.client.0.smithi107.stdout:5/552: creat d4/dc/d26/fb9 x:0 0 0 2024-04-25T00:18:29.880 INFO:tasks.workunit.client.0.smithi107.stdout:5/553: fsync d4/dc/f20 0 2024-04-25T00:18:29.880 INFO:tasks.workunit.client.0.smithi107.stdout:5/554: chown d4/dc/d26/d29/d2e/l25 256 1 2024-04-25T00:18:29.880 INFO:tasks.workunit.client.0.smithi107.stdout:8/526: dwrite d2/f26 [0,4194304] 0 2024-04-25T00:18:29.881 INFO:tasks.workunit.client.0.smithi107.stdout:5/555: mknod d4/dc/d26/d29/d2e/cba 0 2024-04-25T00:18:29.913 INFO:tasks.workunit.client.0.smithi107.stdout:3/408: dwrite d9/f14 [0,4194304] 0 2024-04-25T00:18:29.914 INFO:tasks.workunit.client.0.smithi107.stdout:5/556: dwrite d4/d17/db3/d84/d93/f98 [0,4194304] 0 2024-04-25T00:18:29.915 INFO:tasks.workunit.client.0.smithi107.stdout:0/384: dwrite dc/dd/d5d/f57 [0,4194304] 0 2024-04-25T00:18:29.915 INFO:tasks.workunit.client.0.smithi107.stdout:7/311: dwrite d0/d8/db/d18/d19/f3c [4194304,4194304] 0 2024-04-25T00:18:29.915 INFO:tasks.workunit.client.0.smithi107.stdout:7/312: stat d0/d8/d4e/d46/f30 0 2024-04-25T00:18:29.915 INFO:tasks.workunit.client.0.smithi107.stdout:5/557: write d4/d17/f79 [615666,47780] 0 2024-04-25T00:18:29.916 INFO:tasks.workunit.client.0.smithi107.stdout:8/527: dwrite d2/d18/f43 [0,4194304] 0 2024-04-25T00:18:29.916 INFO:tasks.workunit.client.0.smithi107.stdout:8/528: dread - d2/d18/d1a/f73 zero size 2024-04-25T00:18:29.916 INFO:tasks.workunit.client.0.smithi107.stdout:7/313: symlink d0/d8/d4e/d46/l77 0 2024-04-25T00:18:29.918 INFO:tasks.workunit.client.0.smithi107.stdout:7/314: creat d0/d8/f78 x:0 0 0 2024-04-25T00:18:29.919 INFO:tasks.workunit.client.0.smithi107.stdout:7/315: creat d0/d8/d2c/d58/f79 x:0 0 0 2024-04-25T00:18:29.920 INFO:tasks.workunit.client.0.smithi107.stdout:5/558: unlink d4/dc/l10 0 2024-04-25T00:18:29.920 INFO:tasks.workunit.client.0.smithi107.stdout:7/316: chown d0/d8/db/c29 52279 1 2024-04-25T00:18:29.921 INFO:tasks.workunit.client.0.smithi107.stdout:5/559: rmdir d4/dc/d26/d62 39 2024-04-25T00:18:29.925 INFO:tasks.workunit.client.0.smithi107.stdout:1/396: sync 2024-04-25T00:18:29.928 INFO:tasks.workunit.client.0.smithi107.stdout:0/385: dread dc/d24/d25/f2b [0,4194304] 0 2024-04-25T00:18:29.929 INFO:tasks.workunit.client.0.smithi107.stdout:3/409: write d9/f29 [1744071,38561] 0 2024-04-25T00:18:29.929 INFO:tasks.workunit.client.0.smithi107.stdout:0/386: creat dc/dd/d5d/f70 x:0 0 0 2024-04-25T00:18:29.932 INFO:tasks.workunit.client.0.smithi107.stdout:8/529: read d2/d18/d1a/d5e/d4f/f7a [3367272,109938] 0 2024-04-25T00:18:29.933 INFO:tasks.workunit.client.0.smithi107.stdout:8/530: link d2/d18/d1a/l4e d2/d18/d1a/d33/lb4 0 2024-04-25T00:18:29.934 INFO:tasks.workunit.client.0.smithi107.stdout:8/531: chown d2/d18/d1a/f62 3377373 1 2024-04-25T00:18:29.935 INFO:tasks.workunit.client.0.smithi107.stdout:8/532: creat d2/d18/d1a/d5e/d3d/d85/fb5 x:0 0 0 2024-04-25T00:18:29.935 INFO:tasks.workunit.client.0.smithi107.stdout:8/533: chown d2/d18/d1a/f8e 28797192 1 2024-04-25T00:18:29.936 INFO:tasks.workunit.client.0.smithi107.stdout:8/534: creat d2/d18/d1a/d82/fb6 x:0 0 0 2024-04-25T00:18:29.937 INFO:tasks.workunit.client.0.smithi107.stdout:8/535: mkdir d2/d18/d1a/db7 0 2024-04-25T00:18:29.937 INFO:tasks.workunit.client.0.smithi107.stdout:8/536: stat d2/c2e 0 2024-04-25T00:18:29.938 INFO:tasks.workunit.client.0.smithi107.stdout:8/537: fdatasync d2/d18/d1a/d5e/d5c/d6e/fa6 0 2024-04-25T00:18:29.939 INFO:tasks.workunit.client.0.smithi107.stdout:3/410: read d9/f10 [1979854,8063] 0 2024-04-25T00:18:29.939 INFO:tasks.workunit.client.0.smithi107.stdout:8/538: creat d2/d18/d1a/d5e/d5c/d91/da3/fb8 x:0 0 0 2024-04-25T00:18:29.942 INFO:tasks.workunit.client.0.smithi107.stdout:3/411: mkdir d9/d15/d1a/d80 0 2024-04-25T00:18:29.943 INFO:tasks.workunit.client.0.smithi107.stdout:3/412: truncate d9/d15/d1a/d4e/f7b 493855 0 2024-04-25T00:18:29.945 INFO:tasks.workunit.client.0.smithi107.stdout:3/413: creat d9/d22/d1b/f81 x:0 0 0 2024-04-25T00:18:29.946 INFO:tasks.workunit.client.0.smithi107.stdout:3/414: mknod d9/d15/d39/d76/c82 0 2024-04-25T00:18:29.947 INFO:tasks.workunit.client.0.smithi107.stdout:5/560: dwrite d4/dc/d26/d29/d2e/d55/fa0 [0,4194304] 0 2024-04-25T00:18:29.949 INFO:tasks.workunit.client.0.smithi107.stdout:1/397: dwrite d2/f7e [0,4194304] 0 2024-04-25T00:18:29.949 INFO:tasks.workunit.client.0.smithi107.stdout:8/539: dread d2/d18/d1a/d5e/d3e/f4c [0,4194304] 0 2024-04-25T00:18:29.949 INFO:tasks.workunit.client.0.smithi107.stdout:8/540: chown d2/d18/d1a/d5e/d5c/f88 3 1 2024-04-25T00:18:29.949 INFO:tasks.workunit.client.0.smithi107.stdout:8/541: chown d2/d18/d1a/d5e/d4f/da0 1564642 1 2024-04-25T00:18:29.961 INFO:tasks.workunit.client.0.smithi107.stdout:8/542: dread d2/d18/d1a/d33/f59 [0,4194304] 0 2024-04-25T00:18:29.961 INFO:tasks.workunit.client.0.smithi107.stdout:8/543: dread - d2/d18/d1a/d5e/d3d/fb3 zero size 2024-04-25T00:18:29.961 INFO:tasks.workunit.client.0.smithi107.stdout:7/317: dwrite d0/f1c [0,4194304] 0 2024-04-25T00:18:29.962 INFO:tasks.workunit.client.0.smithi107.stdout:0/387: dread dc/dd/d16/d33/f37 [0,4194304] 0 2024-04-25T00:18:29.963 INFO:tasks.workunit.client.0.smithi107.stdout:8/544: mkdir d2/d18/d1a/d5e/d4f/da0/da1/db9 0 2024-04-25T00:18:29.966 INFO:tasks.workunit.client.0.smithi107.stdout:3/415: dwrite d9/d22/d1b/d55/f5d [0,4194304] 0 2024-04-25T00:18:29.969 INFO:tasks.workunit.client.0.smithi107.stdout:3/416: creat d9/d54/f83 x:0 0 0 2024-04-25T00:18:29.973 INFO:tasks.workunit.client.0.smithi107.stdout:3/417: unlink d9/f11 0 2024-04-25T00:18:29.973 INFO:tasks.workunit.client.0.smithi107.stdout:3/418: chown d9/d54/d5c/d68/d44/d4b/c4c 326846 1 2024-04-25T00:18:29.974 INFO:tasks.workunit.client.0.smithi107.stdout:3/419: mknod d9/d15/d1a/d2c/c84 0 2024-04-25T00:18:29.976 INFO:tasks.workunit.client.0.smithi107.stdout:7/318: getdents d0 0 2024-04-25T00:18:29.976 INFO:tasks.workunit.client.0.smithi107.stdout:1/398: dwrite d2/dd/f6f [0,4194304] 0 2024-04-25T00:18:29.976 INFO:tasks.workunit.client.0.smithi107.stdout:3/420: mknod d9/d54/d5c/d68/c85 0 2024-04-25T00:18:29.976 INFO:tasks.workunit.client.0.smithi107.stdout:3/421: chown d9/f42 3726384 1 2024-04-25T00:18:29.977 INFO:tasks.workunit.client.0.smithi107.stdout:5/561: dwrite d4/dc/d26/d42/d8b/f99 [0,4194304] 0 2024-04-25T00:18:29.984 INFO:tasks.workunit.client.0.smithi107.stdout:7/319: rmdir d0/d8 39 2024-04-25T00:18:29.988 INFO:tasks.workunit.client.0.smithi107.stdout:3/422: dread d9/d54/d5c/d68/f3c [0,4194304] 0 2024-04-25T00:18:29.988 INFO:tasks.workunit.client.0.smithi107.stdout:8/545: dwrite d2/fb [0,4194304] 0 2024-04-25T00:18:29.989 INFO:tasks.workunit.client.0.smithi107.stdout:7/320: unlink d0/d8/d4e/l36 0 2024-04-25T00:18:29.989 INFO:tasks.workunit.client.0.smithi107.stdout:7/321: chown d0/d8/d2c/f55 341187550 1 2024-04-25T00:18:29.990 INFO:tasks.workunit.client.0.smithi107.stdout:7/322: truncate d0/d8/db/d18/d19/f2f 2921382 0 2024-04-25T00:18:29.992 INFO:tasks.workunit.client.0.smithi107.stdout:7/323: rename d0/d8/db/d56 to d0/d8/db/d18/d19/d67/d7a 0 2024-04-25T00:18:29.993 INFO:tasks.workunit.client.0.smithi107.stdout:7/324: getdents d0/d8/db/d18/d19/d47/d4d 0 2024-04-25T00:18:29.995 INFO:tasks.workunit.client.0.smithi107.stdout:5/562: dwrite d4/dc/f63 [0,4194304] 0 2024-04-25T00:18:29.996 INFO:tasks.workunit.client.0.smithi107.stdout:7/325: mknod d0/d8/db/c7b 0 2024-04-25T00:18:29.998 INFO:tasks.workunit.client.0.smithi107.stdout:5/563: rename d4/dc/f2f to d4/d17/db3/fbb 0 2024-04-25T00:18:30.001 INFO:tasks.workunit.client.0.smithi107.stdout:5/564: link d4/dc/d26/d29/d2e/l45 d4/d17/db3/d84/d93/lbc 0 2024-04-25T00:18:30.001 INFO:tasks.workunit.client.0.smithi107.stdout:5/565: chown d4/d1c/d5c/c86 1 1 2024-04-25T00:18:30.006 INFO:tasks.workunit.client.0.smithi107.stdout:7/326: write d0/f1c [2573161,112738] 0 2024-04-25T00:18:30.008 INFO:tasks.workunit.client.0.smithi107.stdout:7/327: link d0/d8/db/d18/d19/d47/d4d/c4f d0/d8/db/d18/d5c/c7c 0 2024-04-25T00:18:30.008 INFO:tasks.workunit.client.0.smithi107.stdout:7/328: truncate d0/d8/db/d18/d19/d47/f62 475853 0 2024-04-25T00:18:30.009 INFO:tasks.workunit.client.0.smithi107.stdout:7/329: symlink d0/d8/db/l7d 0 2024-04-25T00:18:30.010 INFO:tasks.workunit.client.0.smithi107.stdout:7/330: creat d0/d8/d2c/d58/f7e x:0 0 0 2024-04-25T00:18:30.010 INFO:tasks.workunit.client.0.smithi107.stdout:7/331: chown d0/d8/cf 16396396 1 2024-04-25T00:18:30.010 INFO:tasks.workunit.client.0.smithi107.stdout:7/332: dread - d0/d8/db/d18/d19/d47/d4d/f6a zero size 2024-04-25T00:18:30.010 INFO:tasks.workunit.client.0.smithi107.stdout:7/333: readlink d0/d8/d4e/l71 0 2024-04-25T00:18:30.011 INFO:tasks.workunit.client.0.smithi107.stdout:7/334: readlink d0/d8/la 0 2024-04-25T00:18:30.011 INFO:tasks.workunit.client.0.smithi107.stdout:7/335: symlink d0/d8/d2c/l7f 0 2024-04-25T00:18:30.017 INFO:tasks.workunit.client.0.smithi107.stdout:5/566: write d4/dc/f63 [744351,13628] 0 2024-04-25T00:18:30.017 INFO:tasks.workunit.client.0.smithi107.stdout:3/423: dwrite d9/f42 [0,4194304] 0 2024-04-25T00:18:30.018 INFO:tasks.workunit.client.0.smithi107.stdout:5/567: mkdir d4/d17/db3/dbd 0 2024-04-25T00:18:30.019 INFO:tasks.workunit.client.0.smithi107.stdout:5/568: fsync d4/dc/d26/f4b 0 2024-04-25T00:18:30.021 INFO:tasks.workunit.client.0.smithi107.stdout:5/569: symlink d4/dc/lbe 0 2024-04-25T00:18:30.027 INFO:tasks.workunit.client.0.smithi107.stdout:7/336: dwrite d0/d8/db/d18/d19/d47/f31 [4194304,4194304] 0 2024-04-25T00:18:30.037 INFO:tasks.workunit.client.0.smithi107.stdout:7/337: write d0/d8/d2c/f55 [1034694,23607] 0 2024-04-25T00:18:30.038 INFO:tasks.workunit.client.0.smithi107.stdout:7/338: stat d0/d8/f78 0 2024-04-25T00:18:30.046 INFO:tasks.workunit.client.0.smithi107.stdout:7/339: dread d0/f1c [0,4194304] 0 2024-04-25T00:18:30.057 INFO:tasks.workunit.client.0.smithi107.stdout:7/340: write d0/d8/db/d18/d19/d47/f53 [2711522,48619] 0 2024-04-25T00:18:30.059 INFO:tasks.workunit.client.0.smithi107.stdout:7/341: unlink d0/d8/db/d18/d19/d39/c3f 0 2024-04-25T00:18:30.060 INFO:tasks.workunit.client.0.smithi107.stdout:7/342: mkdir d0/d80 0 2024-04-25T00:18:30.060 INFO:tasks.workunit.client.0.smithi107.stdout:7/343: fsync d0/f1d 0 2024-04-25T00:18:30.060 INFO:tasks.workunit.client.0.smithi107.stdout:7/344: write d0/d8/d2c/d58/f7e [790929,111126] 0 2024-04-25T00:18:30.061 INFO:tasks.workunit.client.0.smithi107.stdout:7/345: stat d0/d8/d2c/f3d 0 2024-04-25T00:18:30.061 INFO:tasks.workunit.client.0.smithi107.stdout:7/346: fsync d0/d8/db/d18/d19/f23 0 2024-04-25T00:18:30.070 INFO:tasks.workunit.client.0.smithi107.stdout:7/347: dread d0/d8/db/d18/f61 [0,4194304] 0 2024-04-25T00:18:30.070 INFO:tasks.workunit.client.0.smithi107.stdout:7/348: dread - d0/f69 zero size 2024-04-25T00:18:30.071 INFO:tasks.workunit.client.0.smithi107.stdout:7/349: symlink d0/d8/db/d18/d19/d67/l81 0 2024-04-25T00:18:30.072 INFO:tasks.workunit.client.0.smithi107.stdout:7/350: truncate d0/f1c 379108 0 2024-04-25T00:18:30.073 INFO:tasks.workunit.client.0.smithi107.stdout:7/351: rmdir d0/d8/d52 0 2024-04-25T00:18:30.074 INFO:tasks.workunit.client.0.smithi107.stdout:7/352: readlink d0/d8/db/d18/d19/d39/l76 0 2024-04-25T00:18:30.074 INFO:tasks.workunit.client.0.smithi107.stdout:7/353: mkdir d0/d8/db/d18/d19/d67/d7a/d82 0 2024-04-25T00:18:30.074 INFO:tasks.workunit.client.0.smithi107.stdout:7/354: readlink d0/d8/l22 0 2024-04-25T00:18:30.076 INFO:tasks.workunit.client.0.smithi107.stdout:7/355: truncate d0/d8/f1e 544824 0 2024-04-25T00:18:30.078 INFO:tasks.workunit.client.0.smithi107.stdout:7/356: unlink d0/d8/l22 0 2024-04-25T00:18:30.097 INFO:tasks.workunit.client.0.smithi107.stdout:7/357: dwrite d0/d8/d4e/d46/f6b [0,4194304] 0 2024-04-25T00:18:30.100 INFO:tasks.workunit.client.0.smithi107.stdout:7/358: creat d0/d8/d2c/f83 x:0 0 0 2024-04-25T00:18:30.101 INFO:tasks.workunit.client.0.smithi107.stdout:7/359: creat d0/d8/d2c/d58/f84 x:0 0 0 2024-04-25T00:18:30.101 INFO:tasks.workunit.client.0.smithi107.stdout:7/360: stat d0/d8/d4e/l71 0 2024-04-25T00:18:30.103 INFO:tasks.workunit.client.0.smithi107.stdout:7/361: rename d0/d8/d4e/d46/f30 to d0/d8/db/d18/d5c/f85 0 2024-04-25T00:18:30.103 INFO:tasks.workunit.client.0.smithi107.stdout:7/362: truncate d0/d8/db/d18/f41 124224 0 2024-04-25T00:18:30.103 INFO:tasks.workunit.client.0.smithi107.stdout:7/363: rmdir d0/d8/d2c 39 2024-04-25T00:18:30.105 INFO:tasks.workunit.client.0.smithi107.stdout:7/364: link d0/d8/d4e/d46/l77 d0/d8/db/d18/d19/d47/d4d/l86 0 2024-04-25T00:18:30.106 INFO:tasks.workunit.client.0.smithi107.stdout:7/365: creat d0/d8/d4e/f87 x:0 0 0 2024-04-25T00:18:30.107 INFO:tasks.workunit.client.0.smithi107.stdout:7/366: creat d0/d80/f88 x:0 0 0 2024-04-25T00:18:30.121 INFO:tasks.workunit.client.0.smithi107.stdout:7/367: dwrite d0/d8/db/d18/d19/d39/f6c [0,4194304] 0 2024-04-25T00:18:30.121 INFO:tasks.workunit.client.0.smithi107.stdout:7/368: write d0/d8/d4e/f87 [872906,34866] 0 2024-04-25T00:18:30.123 INFO:tasks.workunit.client.0.smithi107.stdout:7/369: symlink d0/d8/d4e/d46/l89 0 2024-04-25T00:18:30.124 INFO:tasks.workunit.client.0.smithi107.stdout:7/370: mkdir d0/d8/db/d18/d19/d47/d4d/d8a 0 2024-04-25T00:18:30.125 INFO:tasks.workunit.client.0.smithi107.stdout:7/371: truncate d0/d8/f73 320007 0 2024-04-25T00:18:30.126 INFO:tasks.workunit.client.0.smithi107.stdout:7/372: link d0/d8/d4e/d46/c4c d0/d8/db/d18/c8b 0 2024-04-25T00:18:30.129 INFO:tasks.workunit.client.0.smithi107.stdout:7/373: unlink d0/d8/c14 0 2024-04-25T00:18:30.129 INFO:tasks.workunit.client.0.smithi107.stdout:7/374: fdatasync d0/d8/db/f2a 0 2024-04-25T00:18:30.129 INFO:tasks.workunit.client.0.smithi107.stdout:7/375: chown d0/d8/db/c7b 145 1 2024-04-25T00:18:30.131 INFO:tasks.workunit.client.0.smithi107.stdout:7/376: mknod d0/d8/db/c8c 0 2024-04-25T00:18:30.141 INFO:tasks.workunit.client.0.smithi107.stdout:7/377: dread d0/d8/db/d18/d19/f3c [0,4194304] 0 2024-04-25T00:18:30.141 INFO:tasks.workunit.client.0.smithi107.stdout:7/378: truncate d0/d8/d4e/f32 462618 0 2024-04-25T00:18:30.141 INFO:tasks.workunit.client.0.smithi107.stdout:7/379: fsync d0/d8/db/d18/d19/d67/f72 0 2024-04-25T00:18:30.149 INFO:tasks.workunit.client.0.smithi107.stdout:7/380: dread d0/d8/db/d18/d19/d39/f6f [0,4194304] 0 2024-04-25T00:18:30.149 INFO:tasks.workunit.client.0.smithi107.stdout:7/381: dread - d0/d80/f88 zero size 2024-04-25T00:18:30.149 INFO:tasks.workunit.client.0.smithi107.stdout:7/382: read - d0/d8/db/d18/d5c/f75 zero size 2024-04-25T00:18:30.150 INFO:tasks.workunit.client.0.smithi107.stdout:7/383: creat d0/d8/db/d18/f8d x:0 0 0 2024-04-25T00:18:30.152 INFO:tasks.workunit.client.0.smithi107.stdout:7/384: creat d0/d8/db/d18/d19/f8e x:0 0 0 2024-04-25T00:18:30.153 INFO:tasks.workunit.client.0.smithi107.stdout:7/385: chown d0/d8/db/f15 232 1 2024-04-25T00:18:30.153 INFO:tasks.workunit.client.0.smithi107.stdout:7/386: chown d0/d8/d4e/d46/l89 5 1 2024-04-25T00:18:30.153 INFO:tasks.workunit.client.0.smithi107.stdout:7/387: write d0/d8/db/d18/d19/f23 [379380,126998] 0 2024-04-25T00:18:30.154 INFO:tasks.workunit.client.0.smithi107.stdout:7/388: mknod d0/d8/db/d18/d19/d67/d7a/d82/c8f 0 2024-04-25T00:18:30.157 INFO:tasks.workunit.client.0.smithi107.stdout:7/389: rename d0/f1c to d0/d8/d4e/f90 0 2024-04-25T00:18:30.158 INFO:tasks.workunit.client.0.smithi107.stdout:7/390: mknod d0/d8/db/d18/d19/d67/c91 0 2024-04-25T00:18:30.160 INFO:tasks.workunit.client.0.smithi107.stdout:7/391: mkdir d0/d8/db/d92 0 2024-04-25T00:18:30.162 INFO:tasks.workunit.client.0.smithi107.stdout:7/392: mkdir d0/d8/d4e/d46/d93 0 2024-04-25T00:18:30.162 INFO:tasks.workunit.client.0.smithi107.stdout:7/393: stat d0/d8/f78 0 2024-04-25T00:18:30.163 INFO:tasks.workunit.client.0.smithi107.stdout:7/394: rename d0/d8/d2c/l7f to d0/d8/db/d18/l94 0 2024-04-25T00:18:30.167 INFO:tasks.workunit.client.0.smithi107.stdout:7/395: link d0/d8/db/d18/d19/c57 d0/c95 0 2024-04-25T00:18:30.182 INFO:tasks.workunit.client.0.smithi107.stdout:7/396: dwrite d0/d8/d2c/d58/f5e [0,4194304] 0 2024-04-25T00:18:30.182 INFO:tasks.workunit.client.0.smithi107.stdout:7/397: getdents d0/d8/d4e/d46/d93 0 2024-04-25T00:18:30.182 INFO:tasks.workunit.client.0.smithi107.stdout:7/398: fsync d0/d8/f65 0 2024-04-25T00:18:30.184 INFO:tasks.workunit.client.0.smithi107.stdout:7/399: creat d0/d8/db/d18/d19/d39/f96 x:0 0 0 2024-04-25T00:18:30.191 INFO:tasks.workunit.client.0.smithi107.stdout:7/400: write d0/d8/db/f15 [2466925,45699] 0 2024-04-25T00:18:30.192 INFO:tasks.workunit.client.0.smithi107.stdout:7/401: readlink d0/d8/l43 0 2024-04-25T00:18:30.192 INFO:tasks.workunit.client.0.smithi107.stdout:7/402: write d0/d8/db/d18/f41 [522710,22714] 0 2024-04-25T00:18:30.193 INFO:tasks.workunit.client.0.smithi107.stdout:7/403: unlink d0/d8/db/d18/d19/d47/d4d/f6a 0 2024-04-25T00:18:30.194 INFO:tasks.workunit.client.0.smithi107.stdout:7/404: rmdir d0/d8/db/d18 39 2024-04-25T00:18:30.194 INFO:tasks.workunit.client.0.smithi107.stdout:7/405: dread - d0/d8/d2c/f83 zero size 2024-04-25T00:18:30.194 INFO:tasks.workunit.client.0.smithi107.stdout:7/406: fdatasync d0/d8/d2c/f83 0 2024-04-25T00:18:30.196 INFO:tasks.workunit.client.0.smithi107.stdout:7/407: dread d0/d8/f73 [0,4194304] 0 2024-04-25T00:18:30.196 INFO:tasks.workunit.client.0.smithi107.stdout:7/408: chown d0/d8/f65 123475309 1 2024-04-25T00:18:30.196 INFO:tasks.workunit.client.0.smithi107.stdout:7/409: rename d0 to d0/d8/d97 22 2024-04-25T00:18:30.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:30 smithi195 ceph-mon[32833]: pgmap v64: 129 pgs: 129 active+clean; 706 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s rd, 93 MiB/s wr, 526 op/s 2024-04-25T00:18:30.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:30 smithi170 ceph-mon[32807]: pgmap v64: 129 pgs: 129 active+clean; 706 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s rd, 93 MiB/s wr, 526 op/s 2024-04-25T00:18:30.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:30 smithi107 ceph-mon[29153]: pgmap v64: 129 pgs: 129 active+clean; 706 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s rd, 93 MiB/s wr, 526 op/s 2024-04-25T00:18:31.270 INFO:tasks.workunit.client.0.smithi107.stdout:7/410: sync 2024-04-25T00:18:31.270 INFO:tasks.workunit.client.0.smithi107.stdout:8/546: sync 2024-04-25T00:18:31.270 INFO:tasks.workunit.client.0.smithi107.stdout:3/424: sync 2024-04-25T00:18:31.270 INFO:tasks.workunit.client.0.smithi107.stdout:7/411: fsync d0/d8/fd 0 2024-04-25T00:18:31.270 INFO:tasks.workunit.client.0.smithi107.stdout:0/388: sync 2024-04-25T00:18:31.270 INFO:tasks.workunit.client.0.smithi107.stdout:6/384: sync 2024-04-25T00:18:31.270 INFO:tasks.workunit.client.0.smithi107.stdout:5/570: sync 2024-04-25T00:18:31.270 INFO:tasks.workunit.client.0.smithi107.stdout:1/399: sync 2024-04-25T00:18:31.270 INFO:tasks.workunit.client.0.smithi107.stdout:9/395: sync 2024-04-25T00:18:31.270 INFO:tasks.workunit.client.0.smithi107.stdout:1/400: fsync d2/dd/d25/f63 0 2024-04-25T00:18:31.270 INFO:tasks.workunit.client.0.smithi107.stdout:0/389: write dc/d24/f3a [2382938,42836] 0 2024-04-25T00:18:31.271 INFO:tasks.workunit.client.0.smithi107.stdout:0/390: chown dc/d24/d25/f67 1611793 1 2024-04-25T00:18:31.271 INFO:tasks.workunit.client.0.smithi107.stdout:4/648: sync 2024-04-25T00:18:31.271 INFO:tasks.workunit.client.0.smithi107.stdout:2/476: sync 2024-04-25T00:18:31.271 INFO:tasks.workunit.client.0.smithi107.stdout:6/385: truncate d1/d2/d3/f13 2983383 0 2024-04-25T00:18:31.271 INFO:tasks.workunit.client.0.smithi107.stdout:6/386: stat d1/d2/db/d32/d36/d3d/f5e 0 2024-04-25T00:18:31.272 INFO:tasks.workunit.client.0.smithi107.stdout:0/391: creat dc/dd/d5d/d5c/f71 x:0 0 0 2024-04-25T00:18:31.272 INFO:tasks.workunit.client.0.smithi107.stdout:3/425: symlink d9/d54/d5c/d68/d44/d4b/l86 0 2024-04-25T00:18:31.273 INFO:tasks.workunit.client.0.smithi107.stdout:6/387: creat d1/d2/d2b/f86 x:0 0 0 2024-04-25T00:18:31.275 INFO:tasks.workunit.client.0.smithi107.stdout:6/388: symlink d1/d2/db/d3c/d43/l87 0 2024-04-25T00:18:31.276 INFO:tasks.workunit.client.0.smithi107.stdout:1/401: rename d2/da/f43 to d2/dd/d25/d5e/f81 0 2024-04-25T00:18:31.277 INFO:tasks.workunit.client.0.smithi107.stdout:2/477: mknod d6/d18/d40/d42/d70/cac 0 2024-04-25T00:18:31.279 INFO:tasks.workunit.client.0.smithi107.stdout:5/571: link d4/f48 d4/fbf 0 2024-04-25T00:18:31.284 INFO:tasks.workunit.client.0.smithi107.stdout:5/572: creat d4/fc0 x:0 0 0 2024-04-25T00:18:31.284 INFO:tasks.workunit.client.0.smithi107.stdout:9/396: read d0/d8/d56/d76/f4d [3699125,85972] 0 2024-04-25T00:18:31.286 INFO:tasks.workunit.client.0.smithi107.stdout:5/573: truncate d4/dc/f14 2641965 0 2024-04-25T00:18:31.288 INFO:tasks.workunit.client.0.smithi107.stdout:9/397: creat d0/d8/f91 x:0 0 0 2024-04-25T00:18:31.290 INFO:tasks.workunit.client.0.smithi107.stdout:5/574: creat d4/dc/d26/d29/fc1 x:0 0 0 2024-04-25T00:18:31.290 INFO:tasks.workunit.client.0.smithi107.stdout:5/575: readlink l0 0 2024-04-25T00:18:31.291 INFO:tasks.workunit.client.0.smithi107.stdout:1/402: dread d2/d62/d66/f6e [0,4194304] 0 2024-04-25T00:18:31.291 INFO:tasks.workunit.client.0.smithi107.stdout:9/398: truncate d0/d8/f5a 3865035 0 2024-04-25T00:18:31.291 INFO:tasks.workunit.client.0.smithi107.stdout:9/399: read d0/d8/de/f33 [413157,127411] 0 2024-04-25T00:18:31.292 INFO:tasks.workunit.client.0.smithi107.stdout:5/576: rename d4/dc/d26/d29/d2e/l4d to d4/dc/d26/d29/d2e/lc2 0 2024-04-25T00:18:31.293 INFO:tasks.workunit.client.0.smithi107.stdout:8/547: dwrite d2/f63 [0,4194304] 0 2024-04-25T00:18:31.293 INFO:tasks.workunit.client.0.smithi107.stdout:8/548: fdatasync d2/d18/d1a/d5e/d3e/faa 0 2024-04-25T00:18:31.293 INFO:tasks.workunit.client.0.smithi107.stdout:8/549: getdents d2/d18/d1a/d5e/d3d/d85/dae 0 2024-04-25T00:18:31.294 INFO:tasks.workunit.client.0.smithi107.stdout:0/392: dwrite dc/dd/d16/d48/f6b [0,4194304] 0 2024-04-25T00:18:31.295 INFO:tasks.workunit.client.0.smithi107.stdout:0/393: chown dc/f50 25907 1 2024-04-25T00:18:31.297 INFO:tasks.workunit.client.0.smithi107.stdout:5/577: dread d4/d17/f79 [0,4194304] 0 2024-04-25T00:18:31.299 INFO:tasks.workunit.client.0.smithi107.stdout:6/389: dread d1/d2/db/d32/d36/f59 [4194304,4194304] 0 2024-04-25T00:18:31.301 INFO:tasks.workunit.client.0.smithi107.stdout:1/403: write d2/d62/d66/f6e [3994479,12478] 0 2024-04-25T00:18:31.301 INFO:tasks.workunit.client.0.smithi107.stdout:6/390: symlink d1/d2/d3/d28/d79/d6b/l88 0 2024-04-25T00:18:31.302 INFO:tasks.workunit.client.0.smithi107.stdout:5/578: creat d4/d7c/d88/fc3 x:0 0 0 2024-04-25T00:18:31.303 INFO:tasks.workunit.client.0.smithi107.stdout:9/400: dread d0/d43/f3 [0,4194304] 0 2024-04-25T00:18:31.304 INFO:tasks.workunit.client.0.smithi107.stdout:1/404: truncate d2/dd/f6f 2274211 0 2024-04-25T00:18:31.304 INFO:tasks.workunit.client.0.smithi107.stdout:5/579: mkdir d4/d17/db3/d84/d93/dc4 0 2024-04-25T00:18:31.304 INFO:tasks.workunit.client.0.smithi107.stdout:1/405: write d2/dd/d25/f63 [105132,84751] 0 2024-04-25T00:18:31.304 INFO:tasks.workunit.client.0.smithi107.stdout:1/406: stat d2/dd 0 2024-04-25T00:18:31.304 INFO:tasks.workunit.client.0.smithi107.stdout:1/407: write d2/da/f7b [1864662,84016] 0 2024-04-25T00:18:31.307 INFO:tasks.workunit.client.0.smithi107.stdout:6/391: creat d1/d14/f89 x:0 0 0 2024-04-25T00:18:31.308 INFO:tasks.workunit.client.0.smithi107.stdout:9/401: link d0/d43/c25 d0/d8/d6f/c92 0 2024-04-25T00:18:31.308 INFO:tasks.workunit.client.0.smithi107.stdout:9/402: write d0/fc [543575,88257] 0 2024-04-25T00:18:31.309 INFO:tasks.workunit.client.0.smithi107.stdout:5/580: creat d4/dc/d26/d29/d51/fc5 x:0 0 0 2024-04-25T00:18:31.310 INFO:tasks.workunit.client.0.smithi107.stdout:6/392: mkdir d1/d14/d8a 0 2024-04-25T00:18:31.319 INFO:tasks.workunit.client.0.smithi107.stdout:6/393: write d1/d2/db/f19 [2422431,69843] 0 2024-04-25T00:18:31.327 INFO:tasks.workunit.client.0.smithi107.stdout:1/408: dwrite d2/dd/d1a/f27 [4194304,4194304] 0 2024-04-25T00:18:31.327 INFO:tasks.workunit.client.0.smithi107.stdout:3/426: rename d9/d54/d5c/d68/d44/d4b/l86 to d9/d15/d1a/d2c/l87 0 2024-04-25T00:18:31.334 INFO:tasks.workunit.client.0.smithi107.stdout:1/409: dread d2/dd/f6f [0,4194304] 0 2024-04-25T00:18:31.335 INFO:tasks.workunit.client.0.smithi107.stdout:1/410: mkdir d2/dd/d82 0 2024-04-25T00:18:31.336 INFO:tasks.workunit.client.0.smithi107.stdout:3/427: rename d9/d15/d1a to d9/d88 0 2024-04-25T00:18:31.338 INFO:tasks.workunit.client.0.smithi107.stdout:3/428: mknod d9/d88/d4e/c89 0 2024-04-25T00:18:31.338 INFO:tasks.workunit.client.0.smithi107.stdout:3/429: write d9/d88/d4e/f72 [176075,113450] 0 2024-04-25T00:18:31.338 INFO:tasks.workunit.client.0.smithi107.stdout:3/430: chown d9/d88/d2c/l87 7306431 1 2024-04-25T00:18:31.338 INFO:tasks.workunit.client.0.smithi107.stdout:3/431: chown d9/d54/d5c/d68/d44/f65 0 1 2024-04-25T00:18:31.339 INFO:tasks.workunit.client.0.smithi107.stdout:3/432: symlink d9/d15/d39/d76/l8a 0 2024-04-25T00:18:31.349 INFO:tasks.workunit.client.0.smithi107.stdout:1/411: dwrite d2/dd/d1b/f52 [0,4194304] 0 2024-04-25T00:18:31.349 INFO:tasks.workunit.client.0.smithi107.stdout:1/412: chown d2/dd/d1b/f52 6 1 2024-04-25T00:18:31.349 INFO:tasks.workunit.client.0.smithi107.stdout:1/413: write d2/d62/d66/d70/f7a [946699,108579] 0 2024-04-25T00:18:31.351 INFO:tasks.workunit.client.0.smithi107.stdout:2/478: rename d6/d18/d4d to d6/dad 0 2024-04-25T00:18:31.352 INFO:tasks.workunit.client.0.smithi107.stdout:2/479: fdatasync d6/d29/d4e/fab 0 2024-04-25T00:18:31.352 INFO:tasks.workunit.client.0.smithi107.stdout:2/480: chown d6/d20/d22/d56/f68 61433 1 2024-04-25T00:18:31.352 INFO:tasks.workunit.client.0.smithi107.stdout:2/481: write d6/d29/d4e/d6f/f90 [312915,40335] 0 2024-04-25T00:18:31.352 INFO:tasks.workunit.client.0.smithi107.stdout:2/482: readlink d6/d18/d40/d42/d2b/d30/d4c/la4 0 2024-04-25T00:18:31.352 INFO:tasks.workunit.client.0.smithi107.stdout:2/483: fdatasync d6/d18/d40/d42/d69/d6e/d80/f95 0 2024-04-25T00:18:31.352 INFO:tasks.workunit.client.0.smithi107.stdout:2/484: write d6/d18/f65 [106996,48145] 0 2024-04-25T00:18:31.353 INFO:tasks.workunit.client.0.smithi107.stdout:8/550: rename d2/d18/d1a/d5e/c38 to d2/d18/d1a/d5e/d6d/cba 0 2024-04-25T00:18:31.353 INFO:tasks.workunit.client.0.smithi107.stdout:8/551: truncate d2/d18/d1a/f80 914473 0 2024-04-25T00:18:31.354 INFO:tasks.workunit.client.0.smithi107.stdout:2/485: rmdir d6/d18/d40/d42/d69 39 2024-04-25T00:18:31.354 INFO:tasks.workunit.client.0.smithi107.stdout:2/486: fsync d6/d18/d40/d42/d2b/d30/d45/f61 0 2024-04-25T00:18:31.355 INFO:tasks.workunit.client.0.smithi107.stdout:9/403: rename d0/d8 to d0/d8a/d93 0 2024-04-25T00:18:31.356 INFO:tasks.workunit.client.0.smithi107.stdout:9/404: truncate d0/d8a/d93/d34/d36/f73 829042 0 2024-04-25T00:18:31.356 INFO:tasks.workunit.client.0.smithi107.stdout:9/405: chown d0/d8a/d93/d34/d36/d77/c88 74 1 2024-04-25T00:18:31.356 INFO:tasks.workunit.client.0.smithi107.stdout:9/406: dread - d0/d8a/d93/d34/f70 zero size 2024-04-25T00:18:31.356 INFO:tasks.workunit.client.0.smithi107.stdout:2/487: rename d6/d18/l1f to d6/d18/d40/d42/d2b/d30/d4c/lae 0 2024-04-25T00:18:31.365 INFO:tasks.workunit.client.0.smithi107.stdout:8/552: write d2/f45 [2031560,73773] 0 2024-04-25T00:18:31.365 INFO:tasks.workunit.client.0.smithi107.stdout:8/553: fsync d2/f41 0 2024-04-25T00:18:31.375 INFO:tasks.workunit.client.0.smithi107.stdout:1/414: write d2/f30 [8028933,50628] 0 2024-04-25T00:18:31.375 INFO:tasks.workunit.client.0.smithi107.stdout:3/433: dwrite d9/d88/d4e/f72 [0,4194304] 0 2024-04-25T00:18:31.377 INFO:tasks.workunit.client.0.smithi107.stdout:8/554: dread d2/d18/f43 [0,4194304] 0 2024-04-25T00:18:31.378 INFO:tasks.workunit.client.0.smithi107.stdout:8/555: write d2/d18/d1a/f73 [24630,79537] 0 2024-04-25T00:18:31.378 INFO:tasks.workunit.client.0.smithi107.stdout:8/556: chown d2/d18/d1a/c71 1 1 2024-04-25T00:18:31.378 INFO:tasks.workunit.client.0.smithi107.stdout:3/434: truncate d9/d22/d1b/d55/f5d 2470868 0 2024-04-25T00:18:31.379 INFO:tasks.workunit.client.0.smithi107.stdout:1/415: write d2/dd/d25/d5e/f58 [112186,57143] 0 2024-04-25T00:18:31.380 INFO:tasks.workunit.client.0.smithi107.stdout:1/416: dread - d2/dd/d1b/f7f zero size 2024-04-25T00:18:31.380 INFO:tasks.workunit.client.0.smithi107.stdout:1/417: chown d2/c3 0 1 2024-04-25T00:18:31.380 INFO:tasks.workunit.client.0.smithi107.stdout:1/418: fsync d2/dd/f6f 0 2024-04-25T00:18:31.380 INFO:tasks.workunit.client.0.smithi107.stdout:9/407: dwrite d0/d8a/d93/d56/f64 [0,4194304] 0 2024-04-25T00:18:31.380 INFO:tasks.workunit.client.0.smithi107.stdout:1/419: rmdir d2 39 2024-04-25T00:18:31.381 INFO:tasks.workunit.client.0.smithi107.stdout:9/408: creat d0/f94 x:0 0 0 2024-04-25T00:18:31.382 INFO:tasks.workunit.client.0.smithi107.stdout:1/420: symlink d2/da/l83 0 2024-04-25T00:18:31.385 INFO:tasks.workunit.client.0.smithi107.stdout:2/488: dwrite d6/d18/d40/d42/d2b/d30/f7c [0,4194304] 0 2024-04-25T00:18:31.388 INFO:tasks.workunit.client.0.smithi107.stdout:2/489: rename d6/d20/l28 to d6/d20/d22/laf 0 2024-04-25T00:18:31.391 INFO:tasks.workunit.client.0.smithi107.stdout:1/421: read d2/dd/f4b [994156,96875] 0 2024-04-25T00:18:31.392 INFO:tasks.workunit.client.0.smithi107.stdout:1/422: symlink d2/dd/d1b/l84 0 2024-04-25T00:18:31.393 INFO:tasks.workunit.client.0.smithi107.stdout:1/423: getdents d2/d62/d66 0 2024-04-25T00:18:31.394 INFO:tasks.workunit.client.0.smithi107.stdout:1/424: mknod d2/da/d2f/c85 0 2024-04-25T00:18:31.395 INFO:tasks.workunit.client.0.smithi107.stdout:1/425: symlink d2/d2a/d37/l86 0 2024-04-25T00:18:31.397 INFO:tasks.workunit.client.0.smithi107.stdout:1/426: link d2/c3 d2/d62/d66/c87 0 2024-04-25T00:18:31.399 INFO:tasks.workunit.client.0.smithi107.stdout:9/409: dwrite d0/d8a/d93/de/d5c/f72 [0,4194304] 0 2024-04-25T00:18:31.402 INFO:tasks.workunit.client.0.smithi107.stdout:9/410: rename d0/d1b to d0/d8a/d93/de/d5c/d95 0 2024-04-25T00:18:31.405 INFO:tasks.workunit.client.0.smithi107.stdout:2/490: dwrite d6/d29/fa6 [0,4194304] 0 2024-04-25T00:18:31.407 INFO:tasks.workunit.client.0.smithi107.stdout:2/491: unlink d6/d18/d40/d42/d69/d6e/l79 0 2024-04-25T00:18:31.408 INFO:tasks.workunit.client.0.smithi107.stdout:2/492: dread - d6/d18/d40/d42/d69/d6e/d80/f95 zero size 2024-04-25T00:18:31.410 INFO:tasks.workunit.client.0.smithi107.stdout:2/493: mkdir d6/dad/db0 0 2024-04-25T00:18:31.410 INFO:tasks.workunit.client.0.smithi107.stdout:2/494: write d6/f72 [654068,5064] 0 2024-04-25T00:18:31.414 INFO:tasks.workunit.client.0.smithi107.stdout:9/411: dread d0/f40 [0,4194304] 0 2024-04-25T00:18:31.415 INFO:tasks.workunit.client.0.smithi107.stdout:9/412: write d0/d8a/d93/f91 [286876,55683] 0 2024-04-25T00:18:31.416 INFO:tasks.workunit.client.0.smithi107.stdout:9/413: rename d0/d43/f3 to d0/d8a/d93/de/d5c/f96 0 2024-04-25T00:18:31.418 INFO:tasks.workunit.client.0.smithi107.stdout:1/427: dwrite d2/dd/d25/d5e/f39 [0,4194304] 0 2024-04-25T00:18:31.419 INFO:tasks.workunit.client.0.smithi107.stdout:2/495: write d6/d18/d40/d42/d2b/d30/d45/f61 [2621417,56810] 0 2024-04-25T00:18:31.421 INFO:tasks.workunit.client.0.smithi107.stdout:2/496: creat d6/d18/fb1 x:0 0 0 2024-04-25T00:18:31.438 INFO:tasks.workunit.client.0.smithi107.stdout:2/497: dwrite d6/d18/fb1 [0,4194304] 0 2024-04-25T00:18:31.438 INFO:tasks.workunit.client.0.smithi107.stdout:2/498: stat d6/d18/d40/d42/d2b/d30/d45/fa3 0 2024-04-25T00:18:31.439 INFO:tasks.workunit.client.0.smithi107.stdout:1/428: dwrite d2/f7e [0,4194304] 0 2024-04-25T00:18:31.439 INFO:tasks.workunit.client.0.smithi107.stdout:1/429: chown d2/d2a/d5c/f71 187 1 2024-04-25T00:18:31.447 INFO:tasks.workunit.client.0.smithi107.stdout:9/414: dwrite d0/f30 [0,4194304] 0 2024-04-25T00:18:31.447 INFO:tasks.workunit.client.0.smithi107.stdout:1/430: dread d2/f51 [0,4194304] 0 2024-04-25T00:18:31.447 INFO:tasks.workunit.client.0.smithi107.stdout:9/415: write d0/f8f [1029455,92701] 0 2024-04-25T00:18:31.447 INFO:tasks.workunit.client.0.smithi107.stdout:9/416: fdatasync d0/d8a/d93/de/f51 0 2024-04-25T00:18:31.448 INFO:tasks.workunit.client.0.smithi107.stdout:1/431: mkdir d2/d2a/d37/d88 0 2024-04-25T00:18:31.450 INFO:tasks.workunit.client.0.smithi107.stdout:1/432: mkdir d2/d2a/d5c/d89 0 2024-04-25T00:18:31.450 INFO:tasks.workunit.client.0.smithi107.stdout:1/433: chown d2/da/l14 1884393 1 2024-04-25T00:18:31.450 INFO:tasks.workunit.client.0.smithi107.stdout:1/434: chown d2/dd 65 1 2024-04-25T00:18:31.452 INFO:tasks.workunit.client.0.smithi107.stdout:9/417: link d0/d8a/d93/de/d5c/f72 d0/d8a/d93/d56/d6c/d80/f97 0 2024-04-25T00:18:31.452 INFO:tasks.workunit.client.0.smithi107.stdout:9/418: truncate d0/d8a/d93/d56/d6c/f6d 871621 0 2024-04-25T00:18:31.452 INFO:tasks.workunit.client.0.smithi107.stdout:1/435: creat d2/f8a x:0 0 0 2024-04-25T00:18:31.452 INFO:tasks.workunit.client.0.smithi107.stdout:1/436: write f1 [1190964,44052] 0 2024-04-25T00:18:31.453 INFO:tasks.workunit.client.0.smithi107.stdout:2/499: dread d6/f17 [0,4194304] 0 2024-04-25T00:18:31.453 INFO:tasks.workunit.client.0.smithi107.stdout:9/419: creat d0/d8a/d93/d56/f98 x:0 0 0 2024-04-25T00:18:31.454 INFO:tasks.workunit.client.0.smithi107.stdout:1/437: creat d2/dd/f8b x:0 0 0 2024-04-25T00:18:31.454 INFO:tasks.workunit.client.0.smithi107.stdout:2/500: rmdir d6/d18/d40/d42/d69/d6e/d80 39 2024-04-25T00:18:31.455 INFO:tasks.workunit.client.0.smithi107.stdout:9/420: rmdir d0/d8a/d93/d56/d6c 39 2024-04-25T00:18:31.458 INFO:tasks.workunit.client.0.smithi107.stdout:1/438: dread d2/dd/f6f [0,4194304] 0 2024-04-25T00:18:31.459 INFO:tasks.workunit.client.0.smithi107.stdout:1/439: creat d2/d62/d66/d70/f8c x:0 0 0 2024-04-25T00:18:31.460 INFO:tasks.workunit.client.0.smithi107.stdout:1/440: creat d2/da/d2f/f8d x:0 0 0 2024-04-25T00:18:31.462 INFO:tasks.workunit.client.0.smithi107.stdout:1/441: fsync d2/dd/d25/d5e/f58 0 2024-04-25T00:18:31.464 INFO:tasks.workunit.client.0.smithi107.stdout:1/442: creat d2/da/f8e x:0 0 0 2024-04-25T00:18:31.467 INFO:tasks.workunit.client.0.smithi107.stdout:9/421: truncate d0/f8f 2274073 0 2024-04-25T00:18:31.468 INFO:tasks.workunit.client.0.smithi107.stdout:9/422: rmdir d0/d8a/d93/de 39 2024-04-25T00:18:31.471 INFO:tasks.workunit.client.0.smithi107.stdout:9/423: creat d0/d8a/d93/f99 x:0 0 0 2024-04-25T00:18:31.471 INFO:tasks.workunit.client.0.smithi107.stdout:9/424: truncate d0/d8a/d93/d56/f59 1380094 0 2024-04-25T00:18:31.471 INFO:tasks.workunit.client.0.smithi107.stdout:9/425: fdatasync d0/d8a/d93/f5a 0 2024-04-25T00:18:31.471 INFO:tasks.workunit.client.0.smithi107.stdout:9/426: stat d0/d8a/d93/d56/d76/f5b 0 2024-04-25T00:18:31.471 INFO:tasks.workunit.client.0.smithi107.stdout:9/427: fdatasync d0/d8a/d93/d1f/f86 0 2024-04-25T00:18:31.471 INFO:tasks.workunit.client.0.smithi107.stdout:9/428: read d0/d8a/d93/f62 [623901,81476] 0 2024-04-25T00:18:31.471 INFO:tasks.workunit.client.0.smithi107.stdout:9/429: chown d0/l29 31443 1 2024-04-25T00:18:31.472 INFO:tasks.workunit.client.0.smithi107.stdout:9/430: truncate d0/d43/f60 2029538 0 2024-04-25T00:18:31.478 INFO:tasks.workunit.client.0.smithi107.stdout:2/501: dwrite d6/d18/d40/d42/f3b [0,4194304] 0 2024-04-25T00:18:31.483 INFO:tasks.workunit.client.0.smithi107.stdout:1/443: dwrite d2/dd/d25/f7d [0,4194304] 0 2024-04-25T00:18:31.486 INFO:tasks.workunit.client.0.smithi107.stdout:2/502: rename d6/d18/d40/d42/d2b/d30/d4c/f73 to d6/d18/d40/d42/d69/fb2 0 2024-04-25T00:18:31.489 INFO:tasks.workunit.client.0.smithi107.stdout:2/503: getdents d6/d18/d40/d42/d69/d6e/d80 0 2024-04-25T00:18:31.489 INFO:tasks.workunit.client.0.smithi107.stdout:2/504: write d6/d18/d40/d42/d69/fb2 [934422,47920] 0 2024-04-25T00:18:31.489 INFO:tasks.workunit.client.0.smithi107.stdout:2/505: readlink d6/d18/d40/d42/d2b/d30/d4c/la4 0 2024-04-25T00:18:31.489 INFO:tasks.workunit.client.0.smithi107.stdout:2/506: fdatasync d6/ff 0 2024-04-25T00:18:31.489 INFO:tasks.workunit.client.0.smithi107.stdout:2/507: write d6/d18/d40/d42/d69/fb2 [925376,2354] 0 2024-04-25T00:18:31.496 INFO:tasks.workunit.client.0.smithi107.stdout:1/444: dread d2/dd/d1b/f52 [0,4194304] 0 2024-04-25T00:18:31.497 INFO:tasks.workunit.client.0.smithi107.stdout:1/445: unlink d2/d62/d66/c87 0 2024-04-25T00:18:31.497 INFO:tasks.workunit.client.0.smithi107.stdout:1/446: stat d2/d2a 0 2024-04-25T00:18:31.497 INFO:tasks.workunit.client.0.smithi107.stdout:1/447: stat d2/da/f15 0 2024-04-25T00:18:31.499 INFO:tasks.workunit.client.0.smithi107.stdout:9/431: dread d0/d43/f60 [0,4194304] 0 2024-04-25T00:18:31.499 INFO:tasks.workunit.client.0.smithi107.stdout:9/432: read d0/d8a/d93/f55 [324876,129456] 0 2024-04-25T00:18:31.508 INFO:tasks.workunit.client.0.smithi107.stdout:1/448: dread d2/da/f15 [0,4194304] 0 2024-04-25T00:18:31.508 INFO:tasks.workunit.client.0.smithi107.stdout:2/508: dwrite d6/d18/d40/d42/f3f [0,4194304] 0 2024-04-25T00:18:31.508 INFO:tasks.workunit.client.0.smithi107.stdout:2/509: dread - d6/d18/d40/d42/d2b/d30/d45/fa3 zero size 2024-04-25T00:18:31.512 INFO:tasks.workunit.client.0.smithi107.stdout:2/510: link d6/d20/f5a d6/d18/d40/d42/d69/fb3 0 2024-04-25T00:18:31.512 INFO:tasks.workunit.client.0.smithi107.stdout:2/511: chown d6/d20/d22/f1a 80513 1 2024-04-25T00:18:31.512 INFO:tasks.workunit.client.0.smithi107.stdout:9/433: read d0/f30 [743910,76041] 0 2024-04-25T00:18:31.514 INFO:tasks.workunit.client.0.smithi107.stdout:2/512: rename d6/d29/d4e/d6f/c81 to d6/d18/d40/d42/cb4 0 2024-04-25T00:18:31.524 INFO:tasks.workunit.client.0.smithi107.stdout:2/513: read d6/d18/d40/d42/d2b/f41 [2641919,39699] 0 2024-04-25T00:18:31.524 INFO:tasks.workunit.client.0.smithi107.stdout:2/514: fsync d6/d18/d40/d42/d69/d6e/d80/f9a 0 2024-04-25T00:18:31.527 INFO:tasks.workunit.client.0.smithi107.stdout:1/449: dwrite d2/dd/d25/f7d [0,4194304] 0 2024-04-25T00:18:31.529 INFO:tasks.workunit.client.0.smithi107.stdout:1/450: mknod d2/dd/d82/c8f 0 2024-04-25T00:18:31.529 INFO:tasks.workunit.client.0.smithi107.stdout:1/451: stat d2/dd/d25/d5e/f81 0 2024-04-25T00:18:31.534 INFO:tasks.workunit.client.0.smithi107.stdout:1/452: link d2/d62/d66/d70/f77 d2/dd/d82/f90 0 2024-04-25T00:18:31.534 INFO:tasks.workunit.client.0.smithi107.stdout:9/434: dread d0/d8a/d93/f55 [0,4194304] 0 2024-04-25T00:18:31.535 INFO:tasks.workunit.client.0.smithi107.stdout:1/453: write d2/dd/f8b [703825,54422] 0 2024-04-25T00:18:31.535 INFO:tasks.workunit.client.0.smithi107.stdout:1/454: write d2/dd/f6f [2691231,92305] 0 2024-04-25T00:18:31.541 INFO:tasks.workunit.client.0.smithi107.stdout:2/515: dwrite d6/d18/d40/d42/f47 [0,4194304] 0 2024-04-25T00:18:31.541 INFO:tasks.workunit.client.0.smithi107.stdout:1/455: dread d2/dd/f6f [0,4194304] 0 2024-04-25T00:18:31.541 INFO:tasks.workunit.client.0.smithi107.stdout:1/456: dread - d2/dd/d25/f50 zero size 2024-04-25T00:18:31.541 INFO:tasks.workunit.client.0.smithi107.stdout:1/457: stat d2/d62/d66 0 2024-04-25T00:18:31.545 INFO:tasks.workunit.client.0.smithi107.stdout:2/516: truncate d6/f17 2911088 0 2024-04-25T00:18:31.545 INFO:tasks.workunit.client.0.smithi107.stdout:2/517: write d6/d18/f5d [1990042,129593] 0 2024-04-25T00:18:31.545 INFO:tasks.workunit.client.0.smithi107.stdout:2/518: read d6/d18/d40/d42/d2b/f7a [185380,91983] 0 2024-04-25T00:18:31.547 INFO:tasks.workunit.client.0.smithi107.stdout:2/519: dread d6/d18/d40/d42/d2b/d30/f4b [0,4194304] 0 2024-04-25T00:18:31.551 INFO:tasks.workunit.client.0.smithi107.stdout:9/435: dwrite d0/d8a/d93/d1f/f86 [0,4194304] 0 2024-04-25T00:18:31.551 INFO:tasks.workunit.client.0.smithi107.stdout:9/436: write d0/d8a/d93/f91 [1310622,93860] 0 2024-04-25T00:18:31.551 INFO:tasks.workunit.client.0.smithi107.stdout:9/437: chown d0/d8a/d93/d56/d76/d50/f54 557808 1 2024-04-25T00:18:31.552 INFO:tasks.workunit.client.0.smithi107.stdout:9/438: chown d0/d8a/d93/d56/d6c 373541070 1 2024-04-25T00:18:31.553 INFO:tasks.workunit.client.0.smithi107.stdout:1/458: dread d2/f30 [4194304,4194304] 0 2024-04-25T00:18:31.554 INFO:tasks.workunit.client.0.smithi107.stdout:9/439: mknod d0/c9a 0 2024-04-25T00:18:31.554 INFO:tasks.workunit.client.0.smithi107.stdout:9/440: write d0/d8a/d93/d6f/f82 [902526,82870] 0 2024-04-25T00:18:31.555 INFO:tasks.workunit.client.0.smithi107.stdout:1/459: mknod d2/d62/d66/c91 0 2024-04-25T00:18:31.557 INFO:tasks.workunit.client.0.smithi107.stdout:2/520: dread d6/d20/d22/f1c [0,4194304] 0 2024-04-25T00:18:31.559 INFO:tasks.workunit.client.0.smithi107.stdout:9/441: mkdir d0/d8a/d93/d56/d6c/d80/d9b 0 2024-04-25T00:18:31.560 INFO:tasks.workunit.client.0.smithi107.stdout:1/460: unlink d2/cb 0 2024-04-25T00:18:31.562 INFO:tasks.workunit.client.0.smithi107.stdout:9/442: creat d0/d8a/d93/d56/d76/f9c x:0 0 0 2024-04-25T00:18:31.562 INFO:tasks.workunit.client.0.smithi107.stdout:9/443: fdatasync d0/f94 0 2024-04-25T00:18:31.562 INFO:tasks.workunit.client.0.smithi107.stdout:9/444: readlink d0/d43/l12 0 2024-04-25T00:18:31.562 INFO:tasks.workunit.client.0.smithi107.stdout:9/445: write d0/d8a/d93/f55 [3556045,91954] 0 2024-04-25T00:18:31.563 INFO:tasks.workunit.client.0.smithi107.stdout:9/446: mknod d0/d8a/d93/d56/d76/c9d 0 2024-04-25T00:18:31.563 INFO:tasks.workunit.client.0.smithi107.stdout:9/447: fdatasync d0/d8a/d93/d56/d6c/f6d 0 2024-04-25T00:18:31.563 INFO:tasks.workunit.client.0.smithi107.stdout:9/448: rename d0/d8a/d93 to d0/d8a/d93/d56/d6c/d9e 22 2024-04-25T00:18:31.565 INFO:tasks.workunit.client.0.smithi107.stdout:9/449: rename d0/d8a/d93/d56/d76/f4d to d0/d43/f9f 0 2024-04-25T00:18:31.566 INFO:tasks.workunit.client.0.smithi107.stdout:9/450: write d0/d43/f10 [1633295,33865] 0 2024-04-25T00:18:31.566 INFO:tasks.workunit.client.0.smithi107.stdout:2/521: write d6/d18/d40/d42/f62 [6313005,44292] 0 2024-04-25T00:18:31.568 INFO:tasks.workunit.client.0.smithi107.stdout:2/522: creat d6/d18/fb5 x:0 0 0 2024-04-25T00:18:31.568 INFO:tasks.workunit.client.0.smithi107.stdout:1/461: dread d2/dd/d25/f33 [0,4194304] 0 2024-04-25T00:18:31.569 INFO:tasks.workunit.client.0.smithi107.stdout:1/462: creat d2/dd/d82/f92 x:0 0 0 2024-04-25T00:18:31.570 INFO:tasks.workunit.client.0.smithi107.stdout:1/463: creat d2/f93 x:0 0 0 2024-04-25T00:18:31.572 INFO:tasks.workunit.client.0.smithi107.stdout:7/412: sync 2024-04-25T00:18:31.572 INFO:tasks.workunit.client.0.smithi107.stdout:7/413: read - d0/d8/d2c/d58/f79 zero size 2024-04-25T00:18:31.572 INFO:tasks.workunit.client.0.smithi107.stdout:7/414: fdatasync d0/d8/db/d18/d19/d39/f6c 0 2024-04-25T00:18:31.572 INFO:tasks.workunit.client.0.smithi107.stdout:7/415: fsync d0/d8/db/d18/f8d 0 2024-04-25T00:18:31.580 INFO:tasks.workunit.client.0.smithi107.stdout:7/416: rename d0/d8/d2c/d58/f7e to d0/d8/db/f98 0 2024-04-25T00:18:31.581 INFO:tasks.workunit.client.0.smithi107.stdout:9/451: dwrite d0/d8a/d93/d56/d76/f78 [0,4194304] 0 2024-04-25T00:18:31.581 INFO:tasks.workunit.client.0.smithi107.stdout:9/452: dread - d0/d8a/d93/d56/d76/d50/f7a zero size 2024-04-25T00:18:31.583 INFO:tasks.workunit.client.0.smithi107.stdout:7/417: read d0/d8/d4e/f87 [182537,52968] 0 2024-04-25T00:18:31.585 INFO:tasks.workunit.client.0.smithi107.stdout:9/453: dread d0/f7e [0,4194304] 0 2024-04-25T00:18:31.586 INFO:tasks.workunit.client.0.smithi107.stdout:7/418: mknod d0/d8/db/c99 0 2024-04-25T00:18:31.587 INFO:tasks.workunit.client.0.smithi107.stdout:7/419: getdents d0/d8/d4e/d46 0 2024-04-25T00:18:31.587 INFO:tasks.workunit.client.0.smithi107.stdout:9/454: unlink d0/fc 0 2024-04-25T00:18:31.588 INFO:tasks.workunit.client.0.smithi107.stdout:7/420: creat d0/d8/d2c/d58/f9a x:0 0 0 2024-04-25T00:18:31.589 INFO:tasks.workunit.client.0.smithi107.stdout:7/421: symlink d0/d8/db/d18/l9b 0 2024-04-25T00:18:31.592 INFO:tasks.workunit.client.0.smithi107.stdout:2/523: dwrite d6/f3d [0,4194304] 0 2024-04-25T00:18:31.592 INFO:tasks.workunit.client.0.smithi107.stdout:2/524: chown d6/d18/d40/d42/f3f 45742 1 2024-04-25T00:18:31.596 INFO:tasks.workunit.client.0.smithi107.stdout:2/525: mkdir d6/d18/d40/d42/d69/d6e/db6 0 2024-04-25T00:18:31.597 INFO:tasks.workunit.client.0.smithi107.stdout:7/422: dread d0/d8/db/f15 [0,4194304] 0 2024-04-25T00:18:31.597 INFO:tasks.workunit.client.0.smithi107.stdout:1/464: dwrite d2/dd/d1b/f52 [0,4194304] 0 2024-04-25T00:18:31.597 INFO:tasks.workunit.client.0.smithi107.stdout:1/465: stat d2/dd/d82 0 2024-04-25T00:18:31.600 INFO:tasks.workunit.client.0.smithi107.stdout:2/526: mknod d6/d20/d77/d9f/cb7 0 2024-04-25T00:18:31.600 INFO:tasks.workunit.client.0.smithi107.stdout:2/527: chown d6/d18/d40/d42/d2b/d30 14 1 2024-04-25T00:18:31.600 INFO:tasks.workunit.client.0.smithi107.stdout:2/528: write d6/d18/d40/d42/d2b/f41 [86687,69544] 0 2024-04-25T00:18:31.600 INFO:tasks.workunit.client.0.smithi107.stdout:2/529: chown d6/d18/d40/d42/d2b/d30/d4c/lae 12979459 1 2024-04-25T00:18:31.603 INFO:tasks.workunit.client.0.smithi107.stdout:2/530: rmdir d6/d20/d77/d9f 39 2024-04-25T00:18:31.603 INFO:tasks.workunit.client.0.smithi107.stdout:2/531: creat d6/d18/d40/d42/d69/d82/fb8 x:0 0 0 2024-04-25T00:18:31.618 INFO:tasks.workunit.client.0.smithi107.stdout:2/532: dwrite d6/f17 [0,4194304] 0 2024-04-25T00:18:31.621 INFO:tasks.workunit.client.0.smithi107.stdout:2/533: dread d6/d29/d4e/d6f/f90 [0,4194304] 0 2024-04-25T00:18:31.621 INFO:tasks.workunit.client.0.smithi107.stdout:2/534: stat d6/d18/d40/d42/d2b/l2f 0 2024-04-25T00:18:31.635 INFO:tasks.workunit.client.0.smithi107.stdout:0/394: sync 2024-04-25T00:18:31.635 INFO:tasks.workunit.client.0.smithi107.stdout:6/394: sync 2024-04-25T00:18:31.635 INFO:tasks.workunit.client.0.smithi107.stdout:4/649: sync 2024-04-25T00:18:31.635 INFO:tasks.workunit.client.0.smithi107.stdout:5/581: sync 2024-04-25T00:18:31.635 INFO:tasks.workunit.client.0.smithi107.stdout:8/557: sync 2024-04-25T00:18:31.635 INFO:tasks.workunit.client.0.smithi107.stdout:6/395: chown d1/d2/db/d32/d33/c37 71041 1 2024-04-25T00:18:31.635 INFO:tasks.workunit.client.0.smithi107.stdout:4/650: write d3/d5/d32/d53/f5b [2023177,31107] 0 2024-04-25T00:18:31.636 INFO:tasks.workunit.client.0.smithi107.stdout:8/558: mknod d2/d18/cbb 0 2024-04-25T00:18:31.636 INFO:tasks.workunit.client.0.smithi107.stdout:5/582: creat d4/dc/d26/d42/d8b/fc6 x:0 0 0 2024-04-25T00:18:31.636 INFO:tasks.workunit.client.0.smithi107.stdout:5/583: truncate d4/dc/d26/d29/d51/f67 785993 0 2024-04-25T00:18:31.638 INFO:tasks.workunit.client.0.smithi107.stdout:6/396: creat d1/d2/db/d32/d33/d68/d75/f8b x:0 0 0 2024-04-25T00:18:31.638 INFO:tasks.workunit.client.0.smithi107.stdout:6/397: write d1/d2/d3/d26/f51 [5201597,23524] 0 2024-04-25T00:18:31.638 INFO:tasks.workunit.client.0.smithi107.stdout:0/395: symlink dc/dd/d5d/d3f/d59/l72 0 2024-04-25T00:18:31.639 INFO:tasks.workunit.client.0.smithi107.stdout:5/584: truncate d4/d17/db3/d84/fb0 644408 0 2024-04-25T00:18:31.640 INFO:tasks.workunit.client.0.smithi107.stdout:0/396: truncate dc/dd/f19 2891825 0 2024-04-25T00:18:31.641 INFO:tasks.workunit.client.0.smithi107.stdout:0/397: mknod dc/d24/d38/d64/c73 0 2024-04-25T00:18:31.642 INFO:tasks.workunit.client.0.smithi107.stdout:0/398: fsync dc/f50 0 2024-04-25T00:18:31.642 INFO:tasks.workunit.client.0.smithi107.stdout:0/399: chown dc/d24/l4b 5 1 2024-04-25T00:18:31.642 INFO:tasks.workunit.client.0.smithi107.stdout:0/400: mkdir dc/d74 0 2024-04-25T00:18:31.644 INFO:tasks.workunit.client.0.smithi107.stdout:0/401: rename dc/dd/d16/d48/f6b to dc/dd/d16/d33/f75 0 2024-04-25T00:18:31.644 INFO:tasks.workunit.client.0.smithi107.stdout:0/402: write dc/d24/d25/f67 [840529,105079] 0 2024-04-25T00:18:31.651 INFO:tasks.workunit.client.0.smithi107.stdout:0/403: dread dc/d24/f3a [0,4194304] 0 2024-04-25T00:18:31.653 INFO:tasks.workunit.client.0.smithi107.stdout:4/651: dwrite d3/d5/d32/d9e/da2/fa4 [0,4194304] 0 2024-04-25T00:18:31.655 INFO:tasks.workunit.client.0.smithi107.stdout:0/404: unlink dc/dd/cf 0 2024-04-25T00:18:31.655 INFO:tasks.workunit.client.0.smithi107.stdout:0/405: write dc/d24/d38/f66 [703655,47599] 0 2024-04-25T00:18:31.659 INFO:tasks.workunit.client.0.smithi107.stdout:0/406: rename dc/f49 to dc/d24/d38/f76 0 2024-04-25T00:18:31.660 INFO:tasks.workunit.client.0.smithi107.stdout:0/407: creat dc/d24/f77 x:0 0 0 2024-04-25T00:18:31.678 INFO:tasks.workunit.client.0.smithi107.stdout:0/408: dwrite dc/d24/d38/f66 [0,4194304] 0 2024-04-25T00:18:31.680 INFO:tasks.workunit.client.0.smithi107.stdout:0/409: rmdir dc/d24 39 2024-04-25T00:18:31.681 INFO:tasks.workunit.client.0.smithi107.stdout:0/410: rmdir dc/d74 0 2024-04-25T00:18:31.684 INFO:tasks.workunit.client.0.smithi107.stdout:0/411: dread dc/dd/f3c [0,4194304] 0 2024-04-25T00:18:31.684 INFO:tasks.workunit.client.0.smithi107.stdout:0/412: write dc/dd/f3c [344845,88265] 0 2024-04-25T00:18:31.684 INFO:tasks.workunit.client.0.smithi107.stdout:0/413: mkdir dc/d24/d38/d64/d78 0 2024-04-25T00:18:31.688 INFO:tasks.workunit.client.0.smithi107.stdout:0/414: dread dc/dd/f3c [0,4194304] 0 2024-04-25T00:18:31.690 INFO:tasks.workunit.client.0.smithi107.stdout:0/415: write dc/d24/f3a [1084799,680] 0 2024-04-25T00:18:31.691 INFO:tasks.workunit.client.0.smithi107.stdout:0/416: symlink dc/dd/d16/d33/d35/d5f/l79 0 2024-04-25T00:18:31.707 INFO:tasks.workunit.client.0.smithi107.stdout:0/417: rename dc/f1f to dc/dd/d5d/d3f/f7a 0 2024-04-25T00:18:31.717 INFO:tasks.workunit.client.0.smithi107.stdout:0/418: dread dc/dd/f4c [0,4194304] 0 2024-04-25T00:18:31.718 INFO:tasks.workunit.client.0.smithi107.stdout:0/419: fsync dc/dd/d16/d33/f37 0 2024-04-25T00:18:31.718 INFO:tasks.workunit.client.0.smithi107.stdout:0/420: fsync dc/d24/d25/f2b 0 2024-04-25T00:18:31.752 INFO:tasks.workunit.client.0.smithi107.stdout:0/421: dwrite dc/dd/f3c [0,4194304] 0 2024-04-25T00:18:31.753 INFO:tasks.workunit.client.0.smithi107.stdout:0/422: mkdir dc/d24/d38/d64/d7b 0 2024-04-25T00:18:31.754 INFO:tasks.workunit.client.0.smithi107.stdout:0/423: readlink dc/l3e 0 2024-04-25T00:18:31.754 INFO:tasks.workunit.client.0.smithi107.stdout:0/424: dread - dc/f2f zero size 2024-04-25T00:18:31.754 INFO:tasks.workunit.client.0.smithi107.stdout:0/425: link dc/d24/d38/d64/c73 dc/d24/d38/d64/d7b/c7c 0 2024-04-25T00:18:31.757 INFO:tasks.workunit.client.0.smithi107.stdout:0/426: dread dc/d24/d25/f67 [0,4194304] 0 2024-04-25T00:18:31.758 INFO:tasks.workunit.client.0.smithi107.stdout:0/427: chown l6 578396 1 2024-04-25T00:18:31.765 INFO:tasks.workunit.client.0.smithi107.stdout:0/428: read dc/dd/f14 [5994073,20159] 0 2024-04-25T00:18:31.765 INFO:tasks.workunit.client.0.smithi107.stdout:0/429: stat dc/d24/d38/l45 0 2024-04-25T00:18:31.766 INFO:tasks.workunit.client.0.smithi107.stdout:0/430: write dc/dd/d16/f1b [5178825,51435] 0 2024-04-25T00:18:31.767 INFO:tasks.workunit.client.0.smithi107.stdout:0/431: unlink dc/c34 0 2024-04-25T00:18:31.783 INFO:tasks.workunit.client.0.smithi107.stdout:0/432: dwrite dc/dd/f4c [0,4194304] 0 2024-04-25T00:18:31.783 INFO:tasks.workunit.client.0.smithi107.stdout:0/433: write dc/dd/f14 [8359212,6294] 0 2024-04-25T00:18:31.785 INFO:tasks.workunit.client.0.smithi107.stdout:0/434: creat dc/dd/d5d/f7d x:0 0 0 2024-04-25T00:18:31.786 INFO:tasks.workunit.client.0.smithi107.stdout:0/435: mkdir dc/dd/d5d/d7e 0 2024-04-25T00:18:31.788 INFO:tasks.workunit.client.0.smithi107.stdout:0/436: unlink dc/dd/d16/d5e/l63 0 2024-04-25T00:18:31.788 INFO:tasks.workunit.client.0.smithi107.stdout:0/437: chown dc/dd/d5d/c39 56095 1 2024-04-25T00:18:31.804 INFO:tasks.workunit.client.0.smithi107.stdout:0/438: dwrite dc/dd/d16/d48/f54 [0,4194304] 0 2024-04-25T00:18:31.807 INFO:tasks.workunit.client.0.smithi107.stdout:0/439: creat dc/dd/d5d/d5c/f7f x:0 0 0 2024-04-25T00:18:31.807 INFO:tasks.workunit.client.0.smithi107.stdout:0/440: read dc/d24/d25/f2b [1060144,74543] 0 2024-04-25T00:18:31.811 INFO:tasks.workunit.client.0.smithi107.stdout:0/441: truncate dc/dd/f4c 2216590 0 2024-04-25T00:18:31.811 INFO:tasks.workunit.client.0.smithi107.stdout:0/442: chown dc/d24 5 1 2024-04-25T00:18:31.812 INFO:tasks.workunit.client.0.smithi107.stdout:0/443: link dc/d24/d38/f66 dc/d24/d25/f80 0 2024-04-25T00:18:31.812 INFO:tasks.workunit.client.0.smithi107.stdout:0/444: write dc/dd/d5d/d5c/f7f [754660,85597] 0 2024-04-25T00:18:31.813 INFO:tasks.workunit.client.0.smithi107.stdout:0/445: write dc/dd/d16/d48/f5a [1921393,100577] 0 2024-04-25T00:18:31.824 INFO:tasks.workunit.client.0.smithi107.stdout:0/446: rename dc/f50 to dc/dd/d5d/d3f/d59/f81 0 2024-04-25T00:18:31.825 INFO:tasks.workunit.client.0.smithi107.stdout:0/447: truncate dc/d24/d25/f58 1485357 0 2024-04-25T00:18:31.836 INFO:tasks.workunit.client.0.smithi107.stdout:0/448: rename dc/d24/d38/f76 to dc/dd/d16/d33/d35/d5f/f82 0 2024-04-25T00:18:31.837 INFO:tasks.workunit.client.0.smithi107.stdout:0/449: mkdir dc/dd/d16/d48/d83 0 2024-04-25T00:18:31.838 INFO:tasks.workunit.client.0.smithi107.stdout:0/450: rename dc/l12 to dc/d24/d38/d64/d78/l84 0 2024-04-25T00:18:31.842 INFO:tasks.workunit.client.0.smithi107.stdout:0/451: fsync dc/dd/d5d/d5c/f7f 0 2024-04-25T00:18:31.845 INFO:tasks.workunit.client.0.smithi107.stdout:0/452: unlink dc/dd/d5d/f6d 0 2024-04-25T00:18:31.869 INFO:tasks.workunit.client.0.smithi107.stdout:3/435: sync 2024-04-25T00:18:31.871 INFO:tasks.workunit.client.0.smithi107.stdout:3/436: creat d9/d22/f8b x:0 0 0 2024-04-25T00:18:31.887 INFO:tasks.workunit.client.0.smithi107.stdout:3/437: dwrite d9/d22/d1b/f48 [0,4194304] 0 2024-04-25T00:18:31.887 INFO:tasks.workunit.client.0.smithi107.stdout:3/438: chown d9/d88/d2c 6 1 2024-04-25T00:18:31.887 INFO:tasks.workunit.client.0.smithi107.stdout:3/439: read d9/d15/f35 [1242301,26098] 0 2024-04-25T00:18:31.888 INFO:tasks.workunit.client.0.smithi107.stdout:3/440: symlink d9/d22/d1b/d55/l8c 0 2024-04-25T00:18:31.888 INFO:tasks.workunit.client.0.smithi107.stdout:3/441: chown d9/d54/d5c/d68/d3b 1822465 1 2024-04-25T00:18:31.902 INFO:tasks.workunit.client.0.smithi107.stdout:3/442: dwrite d9/d15/f35 [0,4194304] 0 2024-04-25T00:18:31.902 INFO:tasks.workunit.client.0.smithi107.stdout:3/443: dread - d9/d22/d1b/f81 zero size 2024-04-25T00:18:31.905 INFO:tasks.workunit.client.0.smithi107.stdout:3/444: mknod d9/d54/d5c/d68/d44/c8d 0 2024-04-25T00:18:31.905 INFO:tasks.workunit.client.0.smithi107.stdout:3/445: fsync d9/d88/f32 0 2024-04-25T00:18:31.905 INFO:tasks.workunit.client.0.smithi107.stdout:3/446: write d9/d88/d4e/f72 [4628601,1031] 0 2024-04-25T00:18:31.905 INFO:tasks.workunit.client.0.smithi107.stdout:3/447: write d9/f56 [862111,96765] 0 2024-04-25T00:18:31.915 INFO:tasks.workunit.client.0.smithi107.stdout:3/448: write d9/d54/d5c/d68/d44/d4b/f57 [1025325,81206] 0 2024-04-25T00:18:31.915 INFO:tasks.workunit.client.0.smithi107.stdout:3/449: rename d9 to d9/d15/d8e 22 2024-04-25T00:18:31.916 INFO:tasks.workunit.client.0.smithi107.stdout:3/450: chown d9/d22 1277627784 1 2024-04-25T00:18:31.916 INFO:tasks.workunit.client.0.smithi107.stdout:3/451: write d9/d22/d1b/f81 [182362,1661] 0 2024-04-25T00:18:31.916 INFO:tasks.workunit.client.0.smithi107.stdout:3/452: chown d9/d15/d39/c49 55262780 1 2024-04-25T00:18:31.916 INFO:tasks.workunit.client.0.smithi107.stdout:3/453: write d9/d54/d5c/d68/d44/f77 [752807,34881] 0 2024-04-25T00:18:31.916 INFO:tasks.workunit.client.0.smithi107.stdout:3/454: chown d9/d54/d5c/d68/d44/d4b 11 1 2024-04-25T00:18:31.917 INFO:tasks.workunit.client.0.smithi107.stdout:3/455: dread d9/d22/f1e [4194304,4194304] 0 2024-04-25T00:18:31.935 INFO:tasks.workunit.client.0.smithi107.stdout:3/456: dwrite d9/f56 [0,4194304] 0 2024-04-25T00:18:31.935 INFO:tasks.workunit.client.0.smithi107.stdout:3/457: write d9/d22/d1b/f81 [599495,106978] 0 2024-04-25T00:18:31.944 INFO:tasks.workunit.client.0.smithi107.stdout:1/466: sync 2024-04-25T00:18:31.945 INFO:tasks.workunit.client.0.smithi107.stdout:9/455: sync 2024-04-25T00:18:31.945 INFO:tasks.workunit.client.0.smithi107.stdout:1/467: fsync d2/d62/d66/d70/f8c 0 2024-04-25T00:18:31.945 INFO:tasks.workunit.client.0.smithi107.stdout:9/456: write d0/d8a/d93/d34/d36/f8c [279104,84431] 0 2024-04-25T00:18:31.945 INFO:tasks.workunit.client.0.smithi107.stdout:9/457: creat d0/d8a/d93/d34/fa0 x:0 0 0 2024-04-25T00:18:31.945 INFO:tasks.workunit.client.0.smithi107.stdout:9/458: chown d0/d8a/d93/de 175699 1 2024-04-25T00:18:31.946 INFO:tasks.workunit.client.0.smithi107.stdout:9/459: mkdir d0/d8a/d93/d6f/da1 0 2024-04-25T00:18:31.948 INFO:tasks.workunit.client.0.smithi107.stdout:9/460: rename d0/d8a/d93/d1f/c3e to d0/d43/ca2 0 2024-04-25T00:18:31.949 INFO:tasks.workunit.client.0.smithi107.stdout:9/461: rename d0/d8a/d93/d56/d6c/f6d to d0/d8a/d93/d1f/fa3 0 2024-04-25T00:18:31.951 INFO:tasks.workunit.client.0.smithi107.stdout:9/462: dread d0/d8a/d93/d34/d36/f73 [0,4194304] 0 2024-04-25T00:18:31.951 INFO:tasks.workunit.client.0.smithi107.stdout:9/463: write d0/d8a/d93/de/d5c/d95/f2f [53797,70854] 0 2024-04-25T00:18:31.952 INFO:tasks.workunit.client.0.smithi107.stdout:9/464: creat d0/d8a/d93/d56/d6c/fa4 x:0 0 0 2024-04-25T00:18:31.952 INFO:tasks.workunit.client.0.smithi107.stdout:9/465: read d0/d8a/d93/d56/d76/f5b [738883,56359] 0 2024-04-25T00:18:31.952 INFO:tasks.workunit.client.0.smithi107.stdout:9/466: symlink d0/d8a/d93/d34/d36/d77/la5 0 2024-04-25T00:18:31.954 INFO:tasks.workunit.client.0.smithi107.stdout:9/467: write d0/d8a/d93/d56/f59 [40376,86859] 0 2024-04-25T00:18:31.954 INFO:tasks.workunit.client.0.smithi107.stdout:9/468: chown d0/d8a/d93/de/d5c/c61 3 1 2024-04-25T00:18:31.969 INFO:tasks.workunit.client.0.smithi107.stdout:9/469: dwrite d0/d8a/d93/f26 [0,4194304] 0 2024-04-25T00:18:31.971 INFO:tasks.workunit.client.0.smithi107.stdout:9/470: unlink d0/d8a/d93/de/d5c/l6e 0 2024-04-25T00:18:31.971 INFO:tasks.workunit.client.0.smithi107.stdout:9/471: symlink d0/d8a/la6 0 2024-04-25T00:18:31.986 INFO:tasks.workunit.client.0.smithi107.stdout:9/472: dwrite d0/d8a/d93/d56/d76/f46 [0,4194304] 0 2024-04-25T00:18:31.988 INFO:tasks.workunit.client.0.smithi107.stdout:9/473: mknod d0/d8a/d93/d34/d36/ca7 0 2024-04-25T00:18:31.988 INFO:tasks.workunit.client.0.smithi107.stdout:9/474: write d0/d8a/d93/de/f90 [1031175,113991] 0 2024-04-25T00:18:31.988 INFO:tasks.workunit.client.0.smithi107.stdout:9/475: fdatasync d0/d8a/d93/d56/f64 0 2024-04-25T00:18:31.989 INFO:tasks.workunit.client.0.smithi107.stdout:9/476: getdents d0/d8a/d93/d56/d76/d50 0 2024-04-25T00:18:31.991 INFO:tasks.workunit.client.0.smithi107.stdout:9/477: getdents d0/d8a/d93/d34/d36/d77 0 2024-04-25T00:18:31.991 INFO:tasks.workunit.client.0.smithi107.stdout:9/478: fsync d0/d8a/d93/d1f/f86 0 2024-04-25T00:18:31.991 INFO:tasks.workunit.client.0.smithi107.stdout:9/479: truncate d0/d8a/d93/d56/d76/d50/f7a 332258 0 2024-04-25T00:18:31.991 INFO:tasks.workunit.client.0.smithi107.stdout:9/480: chown d0/d8a/d93/c48 86593 1 2024-04-25T00:18:31.993 INFO:tasks.workunit.client.0.smithi107.stdout:9/481: symlink d0/d8a/d93/d56/d6c/d80/la8 0 2024-04-25T00:18:32.017 INFO:tasks.workunit.client.0.smithi107.stdout:6/398: sync 2024-04-25T00:18:32.017 INFO:tasks.workunit.client.0.smithi107.stdout:2/535: sync 2024-04-25T00:18:32.017 INFO:tasks.workunit.client.0.smithi107.stdout:7/423: sync 2024-04-25T00:18:32.017 INFO:tasks.workunit.client.0.smithi107.stdout:0/453: sync 2024-04-25T00:18:32.018 INFO:tasks.workunit.client.0.smithi107.stdout:5/585: sync 2024-04-25T00:18:32.018 INFO:tasks.workunit.client.0.smithi107.stdout:8/559: sync 2024-04-25T00:18:32.018 INFO:tasks.workunit.client.0.smithi107.stdout:4/652: sync 2024-04-25T00:18:32.018 INFO:tasks.workunit.client.0.smithi107.stdout:0/454: read - dc/d24/d25/d31/f6f zero size 2024-04-25T00:18:32.018 INFO:tasks.workunit.client.0.smithi107.stdout:7/424: dread - d0/d8/db/d18/d19/f8e zero size 2024-04-25T00:18:32.018 INFO:tasks.workunit.client.0.smithi107.stdout:5/586: chown d4/dc/d26/d29/d2e/l8a 13 1 2024-04-25T00:18:32.018 INFO:tasks.workunit.client.0.smithi107.stdout:5/587: fdatasync d4/dc/d26/d42/d8b/fa9 0 2024-04-25T00:18:32.018 INFO:tasks.workunit.client.0.smithi107.stdout:2/536: write d6/d18/d40/d42/d2b/f41 [2686252,62693] 0 2024-04-25T00:18:32.018 INFO:tasks.workunit.client.0.smithi107.stdout:4/653: write d3/d24/d5c/fd1 [812088,38131] 0 2024-04-25T00:18:32.018 INFO:tasks.workunit.client.0.smithi107.stdout:0/455: write dc/d24/d25/f67 [953467,56133] 0 2024-04-25T00:18:32.018 INFO:tasks.workunit.client.0.smithi107.stdout:2/537: dread - d6/d18/fb5 zero size 2024-04-25T00:18:32.018 INFO:tasks.workunit.client.0.smithi107.stdout:0/456: readlink l3 0 2024-04-25T00:18:32.018 INFO:tasks.workunit.client.0.smithi107.stdout:0/457: write dc/d24/f77 [596880,7736] 0 2024-04-25T00:18:32.019 INFO:tasks.workunit.client.0.smithi107.stdout:2/538: read d6/d18/d40/d42/d2b/f39 [2079177,51658] 0 2024-04-25T00:18:32.019 INFO:tasks.workunit.client.0.smithi107.stdout:0/458: write dc/d24/d25/f80 [4369419,73795] 0 2024-04-25T00:18:32.019 INFO:tasks.workunit.client.0.smithi107.stdout:8/560: symlink d2/d18/da2/db2/lbc 0 2024-04-25T00:18:32.020 INFO:tasks.workunit.client.0.smithi107.stdout:7/425: mkdir d0/d8/db/d18/d5c/d9c 0 2024-04-25T00:18:32.022 INFO:tasks.workunit.client.0.smithi107.stdout:2/539: creat d6/d18/d40/d42/d2b/d30/d45/fb9 x:0 0 0 2024-04-25T00:18:32.022 INFO:tasks.workunit.client.0.smithi107.stdout:2/540: truncate d6/d18/d40/f99 210834 0 2024-04-25T00:18:32.023 INFO:tasks.workunit.client.0.smithi107.stdout:6/399: unlink d1/d2/d3/d28/d79/c66 0 2024-04-25T00:18:32.023 INFO:tasks.workunit.client.0.smithi107.stdout:6/400: readlink d1/d2/db/d3c/d43/l87 0 2024-04-25T00:18:32.023 INFO:tasks.workunit.client.0.smithi107.stdout:6/401: fdatasync d1/d2/db/d32/f73 0 2024-04-25T00:18:32.023 INFO:tasks.workunit.client.0.smithi107.stdout:5/588: unlink d4/dc/d26/d29/d51/f57 0 2024-04-25T00:18:32.023 INFO:tasks.workunit.client.0.smithi107.stdout:7/426: truncate d0/d8/db/d18/d19/f3c 7118042 0 2024-04-25T00:18:32.025 INFO:tasks.workunit.client.0.smithi107.stdout:2/541: creat d6/d18/d40/d42/d69/d6e/da2/fba x:0 0 0 2024-04-25T00:18:32.025 INFO:tasks.workunit.client.0.smithi107.stdout:2/542: write d6/dad/f85 [638759,76786] 0 2024-04-25T00:18:32.034 INFO:tasks.workunit.client.0.smithi107.stdout:4/654: write d3/d10/f1b [713848,122806] 0 2024-04-25T00:18:32.034 INFO:tasks.workunit.client.0.smithi107.stdout:7/427: symlink d0/d8/db/l9d 0 2024-04-25T00:18:32.034 INFO:tasks.workunit.client.0.smithi107.stdout:4/655: fdatasync d3/d5/d32/f34 0 2024-04-25T00:18:32.035 INFO:tasks.workunit.client.0.smithi107.stdout:7/428: stat d0/d8/db/f49 0 2024-04-25T00:18:32.036 INFO:tasks.workunit.client.0.smithi107.stdout:4/656: truncate d3/d5/f4d 1816734 0 2024-04-25T00:18:32.038 INFO:tasks.workunit.client.0.smithi107.stdout:7/429: symlink d0/d8/db/d18/d19/d67/d7a/d82/l9e 0 2024-04-25T00:18:32.038 INFO:tasks.workunit.client.0.smithi107.stdout:7/430: truncate d0/d8/db/d18/d19/d47/f62 1141354 0 2024-04-25T00:18:32.040 INFO:tasks.workunit.client.0.smithi107.stdout:4/657: mkdir d3/d24/d25/d71/ddb 0 2024-04-25T00:18:32.040 INFO:tasks.workunit.client.0.smithi107.stdout:0/459: dread dc/dd/d16/f1b [0,4194304] 0 2024-04-25T00:18:32.042 INFO:tasks.workunit.client.0.smithi107.stdout:7/431: symlink d0/d8/db/d18/d19/d47/d4d/l9f 0 2024-04-25T00:18:32.045 INFO:tasks.workunit.client.0.smithi107.stdout:7/432: creat d0/d8/d4e/fa0 x:0 0 0 2024-04-25T00:18:32.050 INFO:tasks.workunit.client.0.smithi107.stdout:7/433: mkdir d0/d8/db/d18/d19/d47/da1 0 2024-04-25T00:18:32.050 INFO:tasks.workunit.client.0.smithi107.stdout:7/434: dread - d0/d8/db/d18/d5c/f75 zero size 2024-04-25T00:18:32.052 INFO:tasks.workunit.client.0.smithi107.stdout:8/561: dwrite d2/d18/d1a/f3a [4194304,4194304] 0 2024-04-25T00:18:32.052 INFO:tasks.workunit.client.0.smithi107.stdout:6/402: dwrite d1/d2/d3/f13 [0,4194304] 0 2024-04-25T00:18:32.052 INFO:tasks.workunit.client.0.smithi107.stdout:6/403: readlink d1/d2/d3/d28/d79/l6e 0 2024-04-25T00:18:32.056 INFO:tasks.workunit.client.0.smithi107.stdout:7/435: getdents d0/d8/db 0 2024-04-25T00:18:32.056 INFO:tasks.workunit.client.0.smithi107.stdout:8/562: unlink d2/d18/d1a/d5e/d3e/f6a 0 2024-04-25T00:18:32.057 INFO:tasks.workunit.client.0.smithi107.stdout:6/404: dread d1/d2/db/d32/d36/d3d/f60 [0,4194304] 0 2024-04-25T00:18:32.059 INFO:tasks.workunit.client.0.smithi107.stdout:2/543: dwrite f4 [4194304,4194304] 0 2024-04-25T00:18:32.060 INFO:tasks.workunit.client.0.smithi107.stdout:4/658: dwrite d3/d5/d32/d53/dc0/fca [0,4194304] 0 2024-04-25T00:18:32.060 INFO:tasks.workunit.client.0.smithi107.stdout:6/405: symlink d1/d2/d3/d28/d79/l8c 0 2024-04-25T00:18:32.064 INFO:tasks.workunit.client.0.smithi107.stdout:0/460: dwrite dc/dd/d5d/f57 [0,4194304] 0 2024-04-25T00:18:32.065 INFO:tasks.workunit.client.0.smithi107.stdout:5/589: dwrite d4/dc/d26/d42/f4f [0,4194304] 0 2024-04-25T00:18:32.066 INFO:tasks.workunit.client.0.smithi107.stdout:4/659: mknod d3/d5/cdc 0 2024-04-25T00:18:32.067 INFO:tasks.workunit.client.0.smithi107.stdout:5/590: rmdir d4/dc/d26 39 2024-04-25T00:18:32.067 INFO:tasks.workunit.client.0.smithi107.stdout:5/591: write d4/d17/fb2 [894190,21483] 0 2024-04-25T00:18:32.081 INFO:tasks.workunit.client.0.smithi107.stdout:8/563: dread d2/f83 [0,4194304] 0 2024-04-25T00:18:32.085 INFO:tasks.workunit.client.0.smithi107.stdout:7/436: dwrite d0/d8/db/d18/d5c/f85 [0,4194304] 0 2024-04-25T00:18:32.087 INFO:tasks.workunit.client.0.smithi107.stdout:5/592: read d4/dc/d26/d42/d8b/f99 [2358421,13175] 0 2024-04-25T00:18:32.087 INFO:tasks.workunit.client.0.smithi107.stdout:5/593: fsync d4/dc/d26/d29/d51/f77 0 2024-04-25T00:18:32.087 INFO:tasks.workunit.client.0.smithi107.stdout:5/594: dread - d4/dc/d26/d42/fb4 zero size 2024-04-25T00:18:32.092 INFO:tasks.workunit.client.0.smithi107.stdout:7/437: truncate d0/d8/db/d18/d19/f3c 3082873 0 2024-04-25T00:18:32.093 INFO:tasks.workunit.client.0.smithi107.stdout:4/660: dwrite d3/d24/f69 [0,4194304] 0 2024-04-25T00:18:32.095 INFO:tasks.workunit.client.0.smithi107.stdout:7/438: mkdir d0/d80/da2 0 2024-04-25T00:18:32.095 INFO:tasks.workunit.client.0.smithi107.stdout:7/439: chown d0/f26 108 1 2024-04-25T00:18:32.095 INFO:tasks.workunit.client.0.smithi107.stdout:0/461: dwrite dc/d24/d38/f66 [0,4194304] 0 2024-04-25T00:18:32.095 INFO:tasks.workunit.client.0.smithi107.stdout:0/462: readlink dc/d24/l42 0 2024-04-25T00:18:32.095 INFO:tasks.workunit.client.0.smithi107.stdout:7/440: write d0/d8/db/d18/d19/d67/f72 [752056,123079] 0 2024-04-25T00:18:32.095 INFO:tasks.workunit.client.0.smithi107.stdout:2/544: dwrite d6/d20/d22/f1a [0,4194304] 0 2024-04-25T00:18:32.095 INFO:tasks.workunit.client.0.smithi107.stdout:2/545: write d6/d18/d40/d42/f62 [628258,92568] 0 2024-04-25T00:18:32.096 INFO:tasks.workunit.client.0.smithi107.stdout:4/661: link d3/d5/f41 d3/d5/d32/d9e/da2/daf/fdd 0 2024-04-25T00:18:32.097 INFO:tasks.workunit.client.0.smithi107.stdout:2/546: truncate d6/d18/fb1 1911886 0 2024-04-25T00:18:32.098 INFO:tasks.workunit.client.0.smithi107.stdout:6/406: dwrite d1/d14/f89 [0,4194304] 0 2024-04-25T00:18:32.098 INFO:tasks.workunit.client.0.smithi107.stdout:7/441: rename d0/d8/cf to d0/d8/db/ca3 0 2024-04-25T00:18:32.099 INFO:tasks.workunit.client.0.smithi107.stdout:0/463: dread dc/d24/d25/f58 [0,4194304] 0 2024-04-25T00:18:32.100 INFO:tasks.workunit.client.0.smithi107.stdout:2/547: mknod d6/dad/d67/da8/cbb 0 2024-04-25T00:18:32.103 INFO:tasks.workunit.client.0.smithi107.stdout:8/564: dwrite d2/d18/d1a/d5e/d3e/d74/f8a [0,4194304] 0 2024-04-25T00:18:32.105 INFO:tasks.workunit.client.0.smithi107.stdout:6/407: creat d1/d2/db/d3c/f8d x:0 0 0 2024-04-25T00:18:32.106 INFO:tasks.workunit.client.0.smithi107.stdout:6/408: write d1/d2/db/d32/d36/d3d/f5e [98568,128289] 0 2024-04-25T00:18:32.106 INFO:tasks.workunit.client.0.smithi107.stdout:6/409: fdatasync d1/d2/db/d3c/f72 0 2024-04-25T00:18:32.106 INFO:tasks.workunit.client.0.smithi107.stdout:7/442: getdents d0/d8/db/d18/d19/d67 0 2024-04-25T00:18:32.108 INFO:tasks.workunit.client.0.smithi107.stdout:0/464: creat dc/dd/d5d/d7e/f85 x:0 0 0 2024-04-25T00:18:32.109 INFO:tasks.workunit.client.0.smithi107.stdout:0/465: truncate dc/f28 623466 0 2024-04-25T00:18:32.110 INFO:tasks.workunit.client.0.smithi107.stdout:5/595: dwrite d4/d17/db3/d84/d93/f98 [0,4194304] 0 2024-04-25T00:18:32.112 INFO:tasks.workunit.client.0.smithi107.stdout:5/596: creat d4/d17/db3/d84/d93/da4/fc7 x:0 0 0 2024-04-25T00:18:32.112 INFO:tasks.workunit.client.0.smithi107.stdout:5/597: readlink d4/d1c/l6e 0 2024-04-25T00:18:32.112 INFO:tasks.workunit.client.0.smithi107.stdout:8/565: unlink d2/d18/d1a/d5e/d3e/l5d 0 2024-04-25T00:18:32.115 INFO:tasks.workunit.client.0.smithi107.stdout:5/598: link d4/d17/fb2 d4/dc/d26/d62/fc8 0 2024-04-25T00:18:32.116 INFO:tasks.workunit.client.0.smithi107.stdout:4/662: dwrite d3/d5/d32/d53/d7e/f86 [4194304,4194304] 0 2024-04-25T00:18:32.118 INFO:tasks.workunit.client.0.smithi107.stdout:4/663: mknod d3/d24/d25/d67/cde 0 2024-04-25T00:18:32.121 INFO:tasks.workunit.client.0.smithi107.stdout:1/468: sync 2024-04-25T00:18:32.121 INFO:tasks.workunit.client.0.smithi107.stdout:9/482: sync 2024-04-25T00:18:32.121 INFO:tasks.workunit.client.0.smithi107.stdout:3/458: sync 2024-04-25T00:18:32.121 INFO:tasks.workunit.client.0.smithi107.stdout:3/459: truncate d9/d54/d5c/d68/d44/f79 845466 0 2024-04-25T00:18:32.121 INFO:tasks.workunit.client.0.smithi107.stdout:3/460: truncate d9/d88/d2c/f67 846673 0 2024-04-25T00:18:32.121 INFO:tasks.workunit.client.0.smithi107.stdout:3/461: write d9/f29 [2123006,51553] 0 2024-04-25T00:18:32.121 INFO:tasks.workunit.client.0.smithi107.stdout:3/462: symlink d9/d88/d80/l8f 0 2024-04-25T00:18:32.124 INFO:tasks.workunit.client.0.smithi107.stdout:1/469: unlink d2/dd/d1b/l2b 0 2024-04-25T00:18:32.125 INFO:tasks.workunit.client.0.smithi107.stdout:1/470: getdents d2/dd/d82 0 2024-04-25T00:18:32.126 INFO:tasks.workunit.client.0.smithi107.stdout:1/471: write d2/d2a/d37/f56 [131889,2291] 0 2024-04-25T00:18:32.128 INFO:tasks.workunit.client.0.smithi107.stdout:4/664: dread d3/f18 [0,4194304] 0 2024-04-25T00:18:32.128 INFO:tasks.workunit.client.0.smithi107.stdout:7/443: write d0/d8/d2c/f3d [1404191,10272] 0 2024-04-25T00:18:32.131 INFO:tasks.workunit.client.0.smithi107.stdout:6/410: write d1/d14/f89 [437509,68885] 0 2024-04-25T00:18:32.132 INFO:tasks.workunit.client.0.smithi107.stdout:7/444: creat d0/d8/db/d18/d19/d47/d4d/fa4 x:0 0 0 2024-04-25T00:18:32.132 INFO:tasks.workunit.client.0.smithi107.stdout:7/445: chown d0/d8/d4e/d46/f6b 20172 1 2024-04-25T00:18:32.133 INFO:tasks.workunit.client.0.smithi107.stdout:4/665: rmdir d3/da8 39 2024-04-25T00:18:32.134 INFO:tasks.workunit.client.0.smithi107.stdout:9/483: write d0/d8a/d93/f5a [3656122,40795] 0 2024-04-25T00:18:32.135 INFO:tasks.workunit.client.0.smithi107.stdout:6/411: creat d1/d2/db/d32/f8e x:0 0 0 2024-04-25T00:18:32.135 INFO:tasks.workunit.client.0.smithi107.stdout:6/412: chown d1/d2/db/d32/d36/f59 190015862 1 2024-04-25T00:18:32.137 INFO:tasks.workunit.client.0.smithi107.stdout:9/484: creat d0/d8a/d93/d34/fa9 x:0 0 0 2024-04-25T00:18:32.137 INFO:tasks.workunit.client.0.smithi107.stdout:4/666: creat d3/dd8/fdf x:0 0 0 2024-04-25T00:18:32.137 INFO:tasks.workunit.client.0.smithi107.stdout:9/485: truncate d0/d8a/d93/de/f7c 580453 0 2024-04-25T00:18:32.137 INFO:tasks.workunit.client.0.smithi107.stdout:9/486: fdatasync d0/d8a/d93/f26 0 2024-04-25T00:18:32.137 INFO:tasks.workunit.client.0.smithi107.stdout:6/413: creat d1/d2/d2b/d3e/f8f x:0 0 0 2024-04-25T00:18:32.137 INFO:tasks.workunit.client.0.smithi107.stdout:6/414: readlink d1/d2/db/d32/l62 0 2024-04-25T00:18:32.139 INFO:tasks.workunit.client.0.smithi107.stdout:4/667: link d3/d5/d32/d53/l72 d3/da8/le0 0 2024-04-25T00:18:32.141 INFO:tasks.workunit.client.0.smithi107.stdout:2/548: rename d6/d18/d40/da9 to d6/dad/db0/dbc 0 2024-04-25T00:18:32.146 INFO:tasks.workunit.client.0.smithi107.stdout:5/599: dwrite d4/dc/d26/d42/d8b/f99 [0,4194304] 0 2024-04-25T00:18:32.146 INFO:tasks.workunit.client.0.smithi107.stdout:7/446: dread d0/d8/db/d18/d19/d47/f31 [0,4194304] 0 2024-04-25T00:18:32.147 INFO:tasks.workunit.client.0.smithi107.stdout:7/447: chown d0/d8/d4e/f32 2571 1 2024-04-25T00:18:32.147 INFO:tasks.workunit.client.0.smithi107.stdout:0/466: rename dc/l55 to dc/dd/l86 0 2024-04-25T00:18:32.150 INFO:tasks.workunit.client.0.smithi107.stdout:0/467: write dc/d24/d25/f67 [352717,61881] 0 2024-04-25T00:18:32.150 INFO:tasks.workunit.client.0.smithi107.stdout:5/600: mknod d4/dc/d26/d42/d8b/cc9 0 2024-04-25T00:18:32.150 INFO:tasks.workunit.client.0.smithi107.stdout:5/601: chown d4/dc/d26/d62/c73 4 1 2024-04-25T00:18:32.150 INFO:tasks.workunit.client.0.smithi107.stdout:0/468: write dc/f18 [413652,125995] 0 2024-04-25T00:18:32.150 INFO:tasks.workunit.client.0.smithi107.stdout:0/469: dread - dc/dd/d5d/d5c/f71 zero size 2024-04-25T00:18:32.151 INFO:tasks.workunit.client.0.smithi107.stdout:5/602: symlink d4/d17/db3/d84/d93/da4/lca 0 2024-04-25T00:18:32.153 INFO:tasks.workunit.client.0.smithi107.stdout:3/463: dwrite d9/d54/d5c/d68/d3b/f47 [0,4194304] 0 2024-04-25T00:18:32.155 INFO:tasks.workunit.client.0.smithi107.stdout:3/464: unlink d9/d88/d4e/c59 0 2024-04-25T00:18:32.155 INFO:tasks.workunit.client.0.smithi107.stdout:1/472: dwrite d2/d62/d66/f6e [0,4194304] 0 2024-04-25T00:18:32.156 INFO:tasks.workunit.client.0.smithi107.stdout:0/470: dread dc/dd/d16/f1b [0,4194304] 0 2024-04-25T00:18:32.156 INFO:tasks.workunit.client.0.smithi107.stdout:0/471: dread - dc/dd/d5d/d7e/f85 zero size 2024-04-25T00:18:32.159 INFO:tasks.workunit.client.0.smithi107.stdout:3/465: mknod d9/d54/d5c/d68/d3b/c90 0 2024-04-25T00:18:32.159 INFO:tasks.workunit.client.0.smithi107.stdout:3/466: fsync d9/d88/d2c/f67 0 2024-04-25T00:18:32.160 INFO:tasks.workunit.client.0.smithi107.stdout:9/487: dread d0/d43/f1c [0,4194304] 0 2024-04-25T00:18:32.160 INFO:tasks.workunit.client.0.smithi107.stdout:9/488: dread - d0/d8a/d93/d34/fa9 zero size 2024-04-25T00:18:32.160 INFO:tasks.workunit.client.0.smithi107.stdout:9/489: fsync d0/d8a/d93/f91 0 2024-04-25T00:18:32.161 INFO:tasks.workunit.client.0.smithi107.stdout:9/490: mkdir d0/d8a/d93/d56/d6c/daa 0 2024-04-25T00:18:32.162 INFO:tasks.workunit.client.0.smithi107.stdout:3/467: mkdir d9/d88/d3a/d91 0 2024-04-25T00:18:32.178 INFO:tasks.workunit.client.0.smithi107.stdout:1/473: dread d2/dd/d1a/f61 [0,4194304] 0 2024-04-25T00:18:32.179 INFO:tasks.workunit.client.0.smithi107.stdout:7/448: dwrite d0/d8/db/d18/d19/d39/f6f [0,4194304] 0 2024-04-25T00:18:32.179 INFO:tasks.workunit.client.0.smithi107.stdout:1/474: fsync d2/dd/d25/f2e 0 2024-04-25T00:18:32.181 INFO:tasks.workunit.client.0.smithi107.stdout:8/566: rename d2/d18/d1a/l8d to d2/d18/d1a/d5e/d4f/da0/da1/lbd 0 2024-04-25T00:18:32.182 INFO:tasks.workunit.client.0.smithi107.stdout:4/668: dwrite d3/d5/d32/d9e/da2/daf/fbf [0,4194304] 0 2024-04-25T00:18:32.182 INFO:tasks.workunit.client.0.smithi107.stdout:8/567: creat d2/d18/d1a/d5e/fbe x:0 0 0 2024-04-25T00:18:32.182 INFO:tasks.workunit.client.0.smithi107.stdout:9/491: dread d0/d8a/d93/d56/d6c/d80/f97 [0,4194304] 0 2024-04-25T00:18:32.183 INFO:tasks.workunit.client.0.smithi107.stdout:1/475: dread d2/d62/d66/d70/f7a [0,4194304] 0 2024-04-25T00:18:32.184 INFO:tasks.workunit.client.0.smithi107.stdout:4/669: truncate d3/d24/f81 1032984 0 2024-04-25T00:18:32.189 INFO:tasks.workunit.client.0.smithi107.stdout:1/476: unlink d2/dd/f4b 0 2024-04-25T00:18:32.189 INFO:tasks.workunit.client.0.smithi107.stdout:8/568: unlink d2/d18/l7b 0 2024-04-25T00:18:32.189 INFO:tasks.workunit.client.0.smithi107.stdout:1/477: truncate d2/d62/d66/d70/f77 688846 0 2024-04-25T00:18:32.192 INFO:tasks.workunit.client.0.smithi107.stdout:0/472: dwrite dc/dd/d16/d48/f5a [0,4194304] 0 2024-04-25T00:18:32.192 INFO:tasks.workunit.client.0.smithi107.stdout:0/473: truncate dc/d24/f77 673761 0 2024-04-25T00:18:32.192 INFO:tasks.workunit.client.0.smithi107.stdout:0/474: dread - dc/dd/d5d/f70 zero size 2024-04-25T00:18:32.196 INFO:tasks.workunit.client.0.smithi107.stdout:5/603: dwrite d4/dc/f63 [4194304,4194304] 0 2024-04-25T00:18:32.196 INFO:tasks.workunit.client.0.smithi107.stdout:1/478: dread f1 [0,4194304] 0 2024-04-25T00:18:32.198 INFO:tasks.workunit.client.0.smithi107.stdout:5/604: creat d4/dc/d26/d29/d2e/d55/fcb x:0 0 0 2024-04-25T00:18:32.203 INFO:tasks.workunit.client.0.smithi107.stdout:7/449: dread d0/d8/d4e/d46/f6b [0,4194304] 0 2024-04-25T00:18:32.210 INFO:tasks.workunit.client.0.smithi107.stdout:9/492: dwrite d0/f94 [0,4194304] 0 2024-04-25T00:18:32.215 INFO:tasks.workunit.client.0.smithi107.stdout:9/493: mknod d0/d8a/d93/d56/d6c/daa/cab 0 2024-04-25T00:18:32.215 INFO:tasks.workunit.client.0.smithi107.stdout:9/494: read d0/d8a/d93/d34/d36/f4a [4180945,34548] 0 2024-04-25T00:18:32.215 INFO:tasks.workunit.client.0.smithi107.stdout:8/569: dwrite d2/d18/d1a/d33/f64 [0,4194304] 0 2024-04-25T00:18:32.217 INFO:tasks.workunit.client.0.smithi107.stdout:3/468: link d9/d22/f8b d9/d88/d4e/f92 0 2024-04-25T00:18:32.217 INFO:tasks.workunit.client.0.smithi107.stdout:3/469: write d9/d88/d2c/f75 [891533,36029] 0 2024-04-25T00:18:32.217 INFO:tasks.workunit.client.0.smithi107.stdout:3/470: chown d9/d88/d4e/c5a 25585716 1 2024-04-25T00:18:32.217 INFO:tasks.workunit.client.0.smithi107.stdout:9/495: mkdir d0/d8a/d93/d34/d36/d77/dac 0 2024-04-25T00:18:32.217 INFO:tasks.workunit.client.0.smithi107.stdout:3/471: mkdir d9/d88/d4e/d93 0 2024-04-25T00:18:32.221 INFO:tasks.workunit.client.0.smithi107.stdout:1/479: dwrite d2/dd/d1b/f7f [0,4194304] 0 2024-04-25T00:18:32.222 INFO:tasks.workunit.client.0.smithi107.stdout:5/605: dwrite d4/dc/d26/d29/d2e/d55/fa0 [0,4194304] 0 2024-04-25T00:18:32.222 INFO:tasks.workunit.client.0.smithi107.stdout:9/496: creat d0/d8a/d93/d56/d6c/d80/fad x:0 0 0 2024-04-25T00:18:32.225 INFO:tasks.workunit.client.0.smithi107.stdout:0/475: link dc/d24/d38/f66 dc/dd/d5d/f87 0 2024-04-25T00:18:32.226 INFO:tasks.workunit.client.0.smithi107.stdout:9/497: dread d0/d8a/d93/d56/d76/f5b [0,4194304] 0 2024-04-25T00:18:32.226 INFO:tasks.workunit.client.0.smithi107.stdout:9/498: read d0/d8a/d93/d1f/fa3 [600695,2903] 0 2024-04-25T00:18:32.227 INFO:tasks.workunit.client.0.smithi107.stdout:0/476: creat dc/d24/d38/d64/f88 x:0 0 0 2024-04-25T00:18:32.229 INFO:tasks.workunit.client.0.smithi107.stdout:1/480: rename d2/da/f19 to d2/dd/d25/d5e/f94 0 2024-04-25T00:18:32.231 INFO:tasks.workunit.client.0.smithi107.stdout:7/450: rmdir d0 39 2024-04-25T00:18:32.233 INFO:tasks.workunit.client.0.smithi107.stdout:7/451: rmdir d0/d8/d4e/d46 39 2024-04-25T00:18:32.233 INFO:tasks.workunit.client.0.smithi107.stdout:9/499: getdents d0 0 2024-04-25T00:18:32.235 INFO:tasks.workunit.client.0.smithi107.stdout:3/472: dwrite d9/d15/f24 [0,4194304] 0 2024-04-25T00:18:32.235 INFO:tasks.workunit.client.0.smithi107.stdout:3/473: chown d9/l66 1 1 2024-04-25T00:18:32.236 INFO:tasks.workunit.client.0.smithi107.stdout:5/606: rename d4/fb1 to d4/dc/d83/fcc 0 2024-04-25T00:18:32.236 INFO:tasks.workunit.client.0.smithi107.stdout:5/607: fdatasync d4/d17/fb2 0 2024-04-25T00:18:32.236 INFO:tasks.workunit.client.0.smithi107.stdout:1/481: write d2/da/f8e [754623,46411] 0 2024-04-25T00:18:32.236 INFO:tasks.workunit.client.0.smithi107.stdout:1/482: chown d2/f30 1134 1 2024-04-25T00:18:32.238 INFO:tasks.workunit.client.0.smithi107.stdout:0/477: write dc/d24/d25/d31/f5b [1011224,115821] 0 2024-04-25T00:18:32.239 INFO:tasks.workunit.client.0.smithi107.stdout:0/478: creat dc/dd/d16/d33/f89 x:0 0 0 2024-04-25T00:18:32.239 INFO:tasks.workunit.client.0.smithi107.stdout:0/479: readlink dc/l65 0 2024-04-25T00:18:32.241 INFO:tasks.workunit.client.0.smithi107.stdout:3/474: mkdir d9/d54/d94 0 2024-04-25T00:18:32.242 INFO:tasks.workunit.client.0.smithi107.stdout:3/475: getdents d9/d88/d3a/d91 0 2024-04-25T00:18:32.242 INFO:tasks.workunit.client.0.smithi107.stdout:3/476: rename d9/d54/d5c to d9/d54/d5c/d68/d44/d4b/d95 22 2024-04-25T00:18:32.242 INFO:tasks.workunit.client.0.smithi107.stdout:3/477: dread - d9/d54/d5c/d68/f46 zero size 2024-04-25T00:18:32.243 INFO:tasks.workunit.client.0.smithi107.stdout:1/483: mkdir d2/d62/d66/d70/d95 0 2024-04-25T00:18:32.243 INFO:tasks.workunit.client.0.smithi107.stdout:1/484: truncate d2/f7e 4954535 0 2024-04-25T00:18:32.243 INFO:tasks.workunit.client.0.smithi107.stdout:3/478: symlink d9/d88/d4e/d93/l96 0 2024-04-25T00:18:32.243 INFO:tasks.workunit.client.0.smithi107.stdout:0/480: symlink dc/d24/d38/d64/l8a 0 2024-04-25T00:18:32.243 INFO:tasks.workunit.client.0.smithi107.stdout:0/481: chown dc/d24/d25/f80 902755 1 2024-04-25T00:18:32.243 INFO:tasks.workunit.client.0.smithi107.stdout:9/500: dread d0/d8a/d93/d56/d6c/d80/f97 [0,4194304] 0 2024-04-25T00:18:32.243 INFO:tasks.workunit.client.0.smithi107.stdout:0/482: getdents dc/dd/d5d/d5c/d6e 0 2024-04-25T00:18:32.244 INFO:tasks.workunit.client.0.smithi107.stdout:3/479: symlink d9/d88/d2c/l97 0 2024-04-25T00:18:32.244 INFO:tasks.workunit.client.0.smithi107.stdout:0/483: mkdir dc/d24/d38/d8b 0 2024-04-25T00:18:32.247 INFO:tasks.workunit.client.0.smithi107.stdout:9/501: creat d0/d8a/d93/d56/d76/d50/d7d/fae x:0 0 0 2024-04-25T00:18:32.247 INFO:tasks.workunit.client.0.smithi107.stdout:9/502: dread - d0/d8a/d93/d56/d76/f9c zero size 2024-04-25T00:18:32.247 INFO:tasks.workunit.client.0.smithi107.stdout:0/484: fsync dc/dd/d16/d33/f89 0 2024-04-25T00:18:32.248 INFO:tasks.workunit.client.0.smithi107.stdout:9/503: creat d0/d8a/d93/d6f/faf x:0 0 0 2024-04-25T00:18:32.249 INFO:tasks.workunit.client.0.smithi107.stdout:1/485: symlink d2/da/d2f/l96 0 2024-04-25T00:18:32.249 INFO:tasks.workunit.client.0.smithi107.stdout:1/486: write d2/d62/d66/d70/f77 [943183,129096] 0 2024-04-25T00:18:32.250 INFO:tasks.workunit.client.0.smithi107.stdout:1/487: getdents d2/d2a/d5c/d89 0 2024-04-25T00:18:32.250 INFO:tasks.workunit.client.0.smithi107.stdout:1/488: dread - d2/f69 zero size 2024-04-25T00:18:32.250 INFO:tasks.workunit.client.0.smithi107.stdout:9/504: mknod d0/d8a/d93/d56/d6c/daa/cb0 0 2024-04-25T00:18:32.252 INFO:tasks.workunit.client.0.smithi107.stdout:9/505: creat d0/fb1 x:0 0 0 2024-04-25T00:18:32.252 INFO:tasks.workunit.client.0.smithi107.stdout:9/506: write d0/d8a/d93/f55 [2051191,125621] 0 2024-04-25T00:18:32.253 INFO:tasks.workunit.client.0.smithi107.stdout:1/489: link d2/dd/d25/f50 d2/da/d2f/f97 0 2024-04-25T00:18:32.254 INFO:tasks.workunit.client.0.smithi107.stdout:9/507: mkdir d0/d8a/d93/d56/d6c/d80/d9b/db2 0 2024-04-25T00:18:32.255 INFO:tasks.workunit.client.0.smithi107.stdout:1/490: mknod d2/d62/d66/d70/d95/c98 0 2024-04-25T00:18:32.256 INFO:tasks.workunit.client.0.smithi107.stdout:9/508: truncate d0/d8a/d93/d1f/f86 997874 0 2024-04-25T00:18:32.256 INFO:tasks.workunit.client.0.smithi107.stdout:9/509: truncate d0/d8a/d93/d34/d36/f8c 430684 0 2024-04-25T00:18:32.257 INFO:tasks.workunit.client.0.smithi107.stdout:1/491: creat d2/dd/d82/f99 x:0 0 0 2024-04-25T00:18:32.258 INFO:tasks.workunit.client.0.smithi107.stdout:1/492: truncate d2/dd/d1a/f61 2321849 0 2024-04-25T00:18:32.258 INFO:tasks.workunit.client.0.smithi107.stdout:3/480: dwrite d9/d54/d5c/d68/d44/f65 [0,4194304] 0 2024-04-25T00:18:32.262 INFO:tasks.workunit.client.0.smithi107.stdout:5/608: dwrite d4/dc/d26/d42/d8b/f99 [0,4194304] 0 2024-04-25T00:18:32.262 INFO:tasks.workunit.client.0.smithi107.stdout:1/493: symlink d2/d62/d66/d70/l9a 0 2024-04-25T00:18:32.262 INFO:tasks.workunit.client.0.smithi107.stdout:1/494: chown d2/d62/d66 15332 1 2024-04-25T00:18:32.264 INFO:tasks.workunit.client.0.smithi107.stdout:1/495: creat d2/da/d2f/f9b x:0 0 0 2024-04-25T00:18:32.264 INFO:tasks.workunit.client.0.smithi107.stdout:1/496: fdatasync d2/dd/d25/d5e/f94 0 2024-04-25T00:18:32.266 INFO:tasks.workunit.client.0.smithi107.stdout:1/497: mknod d2/d2a/d37/d88/c9c 0 2024-04-25T00:18:32.269 INFO:tasks.workunit.client.0.smithi107.stdout:0/485: dwrite dc/dd/d5d/f87 [0,4194304] 0 2024-04-25T00:18:32.271 INFO:tasks.workunit.client.0.smithi107.stdout:0/486: symlink dc/dd/d5d/d3f/l8c 0 2024-04-25T00:18:32.272 INFO:tasks.workunit.client.0.smithi107.stdout:5/609: write d4/dc/f14 [1163650,126572] 0 2024-04-25T00:18:32.276 INFO:tasks.workunit.client.0.smithi107.stdout:5/610: creat d4/dc/d26/d62/fcd x:0 0 0 2024-04-25T00:18:32.279 INFO:tasks.workunit.client.0.smithi107.stdout:5/611: rmdir d4/d17/db3/dbd 0 2024-04-25T00:18:32.280 INFO:tasks.workunit.client.0.smithi107.stdout:9/510: dwrite d0/d8a/d93/d56/d76/d50/f54 [0,4194304] 0 2024-04-25T00:18:32.280 INFO:tasks.workunit.client.0.smithi107.stdout:3/481: dwrite d9/f56 [0,4194304] 0 2024-04-25T00:18:32.286 INFO:tasks.workunit.client.0.smithi107.stdout:5/612: dread d4/dc/d26/d29/d2e/d82/f68 [0,4194304] 0 2024-04-25T00:18:32.286 INFO:tasks.workunit.client.0.smithi107.stdout:5/613: dread - d4/dc/d26/d62/fcd zero size 2024-04-25T00:18:32.287 INFO:tasks.workunit.client.0.smithi107.stdout:9/511: truncate d0/d8a/d93/d56/d76/f38 1838433 0 2024-04-25T00:18:32.289 INFO:tasks.workunit.client.0.smithi107.stdout:5/614: creat d4/d35/fce x:0 0 0 2024-04-25T00:18:32.289 INFO:tasks.workunit.client.0.smithi107.stdout:5/615: read - d4/dc/d26/d29/d51/fc5 zero size 2024-04-25T00:18:32.289 INFO:tasks.workunit.client.0.smithi107.stdout:5/616: dread - d4/dc/d26/fb9 zero size 2024-04-25T00:18:32.289 INFO:tasks.workunit.client.0.smithi107.stdout:5/617: chown d4/d1c/d5c/c86 28 1 2024-04-25T00:18:32.289 INFO:tasks.workunit.client.0.smithi107.stdout:5/618: dread - d4/fc0 zero size 2024-04-25T00:18:32.290 INFO:tasks.workunit.client.0.smithi107.stdout:9/512: mknod d0/d8a/cb3 0 2024-04-25T00:18:32.293 INFO:tasks.workunit.client.0.smithi107.stdout:5/619: mkdir d4/dc/d26/d62/dcf 0 2024-04-25T00:18:32.301 INFO:tasks.workunit.client.0.smithi107.stdout:3/482: dwrite d9/d22/d1b/d55/f5b [0,4194304] 0 2024-04-25T00:18:32.301 INFO:tasks.workunit.client.0.smithi107.stdout:1/498: dwrite d2/da/d2f/f3e [0,4194304] 0 2024-04-25T00:18:32.305 INFO:tasks.workunit.client.0.smithi107.stdout:1/499: write d2/da/f7b [1295908,15839] 0 2024-04-25T00:18:32.309 INFO:tasks.workunit.client.0.smithi107.stdout:5/620: dwrite d4/dc/d83/fcc [0,4194304] 0 2024-04-25T00:18:32.313 INFO:tasks.workunit.client.0.smithi107.stdout:9/513: dwrite d0/fb1 [0,4194304] 0 2024-04-25T00:18:32.315 INFO:tasks.workunit.client.0.smithi107.stdout:3/483: stat d9/d88 0 2024-04-25T00:18:32.315 INFO:tasks.workunit.client.0.smithi107.stdout:1/500: dread d2/dd/d25/f5f [0,4194304] 0 2024-04-25T00:18:32.316 INFO:tasks.workunit.client.0.smithi107.stdout:3/484: read d9/d88/d4e/f7b [248522,85912] 0 2024-04-25T00:18:32.316 INFO:tasks.workunit.client.0.smithi107.stdout:9/514: write d0/d43/fd [800135,23842] 0 2024-04-25T00:18:32.316 INFO:tasks.workunit.client.0.smithi107.stdout:1/501: creat d2/dd/d25/d5e/f9d x:0 0 0 2024-04-25T00:18:32.317 INFO:tasks.workunit.client.0.smithi107.stdout:3/485: truncate d9/f42 716032 0 2024-04-25T00:18:32.319 INFO:tasks.workunit.client.0.smithi107.stdout:9/515: mknod d0/d8a/d93/d56/d6c/d80/d9b/db2/cb4 0 2024-04-25T00:18:32.321 INFO:tasks.workunit.client.0.smithi107.stdout:3/486: rename d9/d22/c6d to d9/d54/d5c/d68/d44/d4b/c98 0 2024-04-25T00:18:32.322 INFO:tasks.workunit.client.0.smithi107.stdout:3/487: readlink d9/d54/d5c/d68/d3b/l45 0 2024-04-25T00:18:32.323 INFO:tasks.workunit.client.0.smithi107.stdout:9/516: creat d0/d8a/d93/d6f/fb5 x:0 0 0 2024-04-25T00:18:32.325 INFO:tasks.workunit.client.0.smithi107.stdout:5/621: link d4/fa d4/d17/db3/d84/d93/da4/fd0 0 2024-04-25T00:18:32.326 INFO:tasks.workunit.client.0.smithi107.stdout:3/488: creat d9/f99 x:0 0 0 2024-04-25T00:18:32.328 INFO:tasks.workunit.client.0.smithi107.stdout:9/517: creat d0/d8a/d93/d1f/fb6 x:0 0 0 2024-04-25T00:18:32.328 INFO:tasks.workunit.client.0.smithi107.stdout:9/518: write d0/d43/f9f [3420672,99008] 0 2024-04-25T00:18:32.329 INFO:tasks.workunit.client.0.smithi107.stdout:5/622: creat d4/d17/db3/fd1 x:0 0 0 2024-04-25T00:18:32.329 INFO:tasks.workunit.client.0.smithi107.stdout:5/623: dread - d4/dc/d26/d29/d51/f77 zero size 2024-04-25T00:18:32.331 INFO:tasks.workunit.client.0.smithi107.stdout:5/624: unlink d4/dc/c18 0 2024-04-25T00:18:32.333 INFO:tasks.workunit.client.0.smithi107.stdout:5/625: creat d4/dc/d26/d29/d2e/d82/fd2 x:0 0 0 2024-04-25T00:18:32.334 INFO:tasks.workunit.client.0.smithi107.stdout:5/626: mknod d4/cd3 0 2024-04-25T00:18:32.334 INFO:tasks.workunit.client.0.smithi107.stdout:5/627: read - d4/dc/d26/d42/f76 zero size 2024-04-25T00:18:32.338 INFO:tasks.workunit.client.0.smithi107.stdout:9/519: dread d0/d8a/d93/d34/d36/f4a [0,4194304] 0 2024-04-25T00:18:32.339 INFO:tasks.workunit.client.0.smithi107.stdout:3/489: getdents d9 0 2024-04-25T00:18:32.339 INFO:tasks.workunit.client.0.smithi107.stdout:9/520: fdatasync d0/d8a/d93/f91 0 2024-04-25T00:18:32.339 INFO:tasks.workunit.client.0.smithi107.stdout:3/490: chown d9/d54/d5c/d68/d44/f65 156 1 2024-04-25T00:18:32.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:32 smithi195 ceph-mon[32833]: pgmap v65: 129 pgs: 129 active+clean; 706 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 79 MiB/s rd, 89 MiB/s wr, 187 op/s 2024-04-25T00:18:32.339 INFO:tasks.workunit.client.0.smithi107.stdout:9/521: symlink d0/d8a/d93/d56/d6c/daa/lb7 0 2024-04-25T00:18:32.340 INFO:tasks.workunit.client.0.smithi107.stdout:3/491: creat d9/d88/d3a/d91/f9a x:0 0 0 2024-04-25T00:18:32.341 INFO:tasks.workunit.client.0.smithi107.stdout:9/522: creat d0/d8a/d93/d56/d76/fb8 x:0 0 0 2024-04-25T00:18:32.342 INFO:tasks.workunit.client.0.smithi107.stdout:9/523: mkdir d0/d8a/d93/db9 0 2024-04-25T00:18:32.343 INFO:tasks.workunit.client.0.smithi107.stdout:9/524: creat d0/d8a/d93/d56/fba x:0 0 0 2024-04-25T00:18:32.345 INFO:tasks.workunit.client.0.smithi107.stdout:9/525: unlink d0/d8a/d93/d6f/faf 0 2024-04-25T00:18:32.347 INFO:tasks.workunit.client.0.smithi107.stdout:9/526: link d0/l29 d0/d8a/d93/d56/d6c/lbb 0 2024-04-25T00:18:32.348 INFO:tasks.workunit.client.0.smithi107.stdout:9/527: creat d0/d8a/d93/d56/d76/d50/d7d/fbc x:0 0 0 2024-04-25T00:18:32.348 INFO:tasks.workunit.client.0.smithi107.stdout:9/528: write d0/d8a/d93/d34/fa0 [198880,128543] 0 2024-04-25T00:18:32.349 INFO:tasks.workunit.client.0.smithi107.stdout:5/628: dwrite d4/dc/d26/d29/d2e/d55/f75 [0,4194304] 0 2024-04-25T00:18:32.350 INFO:tasks.workunit.client.0.smithi107.stdout:3/492: dread d9/d54/d5c/d68/d44/d4b/f57 [0,4194304] 0 2024-04-25T00:18:32.352 INFO:tasks.workunit.client.0.smithi107.stdout:9/529: mknod d0/d43/cbd 0 2024-04-25T00:18:32.358 INFO:tasks.workunit.client.0.smithi107.stdout:5/629: dread d4/dc/f13 [0,4194304] 0 2024-04-25T00:18:32.363 INFO:tasks.workunit.client.0.smithi107.stdout:9/530: dread d0/d8a/d93/de/d5c/f72 [0,4194304] 0 2024-04-25T00:18:32.363 INFO:tasks.workunit.client.0.smithi107.stdout:9/531: chown d0/d8a/d93/d56/d76/f46 238 1 2024-04-25T00:18:32.365 INFO:tasks.workunit.client.0.smithi107.stdout:9/532: symlink d0/d8a/d93/d34/d36/lbe 0 2024-04-25T00:18:32.366 INFO:tasks.workunit.client.0.smithi107.stdout:3/493: dwrite d9/d15/f35 [0,4194304] 0 2024-04-25T00:18:32.366 INFO:tasks.workunit.client.0.smithi107.stdout:9/533: mknod d0/d8a/d93/d6f/da1/cbf 0 2024-04-25T00:18:32.366 INFO:tasks.workunit.client.0.smithi107.stdout:3/494: fdatasync d9/d54/d5c/d68/f40 0 2024-04-25T00:18:32.368 INFO:tasks.workunit.client.0.smithi107.stdout:3/495: getdents d9/d22/d1b/d55 0 2024-04-25T00:18:32.369 INFO:tasks.workunit.client.0.smithi107.stdout:3/496: creat d9/f9b x:0 0 0 2024-04-25T00:18:32.374 INFO:tasks.workunit.client.0.smithi107.stdout:9/534: read d0/d8a/d93/d56/d76/f46 [3175846,71617] 0 2024-04-25T00:18:32.376 INFO:tasks.workunit.client.0.smithi107.stdout:9/535: mknod d0/d8a/d93/d56/d6c/cc0 0 2024-04-25T00:18:32.376 INFO:tasks.workunit.client.0.smithi107.stdout:9/536: fsync d0/d8a/d93/de/d5c/d95/f63 0 2024-04-25T00:18:32.376 INFO:tasks.workunit.client.0.smithi107.stdout:9/537: truncate d0/d8a/d93/d1f/f86 1122257 0 2024-04-25T00:18:32.376 INFO:tasks.workunit.client.0.smithi107.stdout:9/538: rmdir d0/d8a/d93/d1f 39 2024-04-25T00:18:32.376 INFO:tasks.workunit.client.0.smithi107.stdout:9/539: read d0/d8a/d93/d1f/f8b [723186,41732] 0 2024-04-25T00:18:32.379 INFO:tasks.workunit.client.0.smithi107.stdout:9/540: dread d0/d8a/d93/de/d5c/d95/f2f [0,4194304] 0 2024-04-25T00:18:32.379 INFO:tasks.workunit.client.0.smithi107.stdout:9/541: rename d0/d8a/d93 to d0/d8a/d93/de/dc1 22 2024-04-25T00:18:32.387 INFO:tasks.workunit.client.0.smithi107.stdout:3/497: dwrite d9/d88/d4e/f7b [0,4194304] 0 2024-04-25T00:18:32.392 INFO:tasks.workunit.client.0.smithi107.stdout:9/542: write d0/d8a/d93/d56/f64 [3557394,86576] 0 2024-04-25T00:18:32.392 INFO:tasks.workunit.client.0.smithi107.stdout:9/543: stat d0/f4b 0 2024-04-25T00:18:32.392 INFO:tasks.workunit.client.0.smithi107.stdout:9/544: truncate d0/d8a/d93/de/f83 424621 0 2024-04-25T00:18:32.405 INFO:tasks.workunit.client.0.smithi107.stdout:3/498: rename d9/l66 to d9/d88/d2c/l9c 0 2024-04-25T00:18:32.410 INFO:tasks.workunit.client.0.smithi107.stdout:9/545: dwrite d0/d8a/d93/d56/f98 [0,4194304] 0 2024-04-25T00:18:32.414 INFO:tasks.workunit.client.0.smithi107.stdout:9/546: mkdir d0/d8a/d93/dc2 0 2024-04-25T00:18:32.415 INFO:tasks.workunit.client.0.smithi107.stdout:3/499: read d9/d88/d4e/f72 [664970,83236] 0 2024-04-25T00:18:32.419 INFO:tasks.workunit.client.0.smithi107.stdout:3/500: rmdir d9/d88/d80 39 2024-04-25T00:18:32.420 INFO:tasks.workunit.client.0.smithi107.stdout:6/415: sync 2024-04-25T00:18:32.433 INFO:tasks.workunit.client.0.smithi107.stdout:9/547: dwrite d0/d8a/d93/f45 [0,4194304] 0 2024-04-25T00:18:32.433 INFO:tasks.workunit.client.0.smithi107.stdout:9/548: chown d0/d8a/d93/de/d5c/d95/c2d 90 1 2024-04-25T00:18:32.433 INFO:tasks.workunit.client.0.smithi107.stdout:9/549: chown d0/f7e 1717382873 1 2024-04-25T00:18:32.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:32 smithi170 ceph-mon[32807]: pgmap v65: 129 pgs: 129 active+clean; 706 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 79 MiB/s rd, 89 MiB/s wr, 187 op/s 2024-04-25T00:18:32.436 INFO:tasks.workunit.client.0.smithi107.stdout:9/550: mknod d0/d8a/d93/de/d5c/d95/cc3 0 2024-04-25T00:18:32.437 INFO:tasks.workunit.client.0.smithi107.stdout:3/501: rename d9/d15/f24 to d9/d88/d4e/d93/f9d 0 2024-04-25T00:18:32.437 INFO:tasks.workunit.client.0.smithi107.stdout:3/502: read - d9/d88/d3a/d91/f9a zero size 2024-04-25T00:18:32.437 INFO:tasks.workunit.client.0.smithi107.stdout:9/551: creat d0/d8a/d93/d6f/fc4 x:0 0 0 2024-04-25T00:18:32.438 INFO:tasks.workunit.client.0.smithi107.stdout:9/552: chown d0/d43/cbd 139650 1 2024-04-25T00:18:32.439 INFO:tasks.workunit.client.0.smithi107.stdout:3/503: rmdir d9/d54/d5c 39 2024-04-25T00:18:32.451 INFO:tasks.workunit.client.0.smithi107.stdout:3/504: dread d9/d22/d1b/f48 [0,4194304] 0 2024-04-25T00:18:32.452 INFO:tasks.workunit.client.0.smithi107.stdout:3/505: rmdir d9/d22/d1b/d55 39 2024-04-25T00:18:32.453 INFO:tasks.workunit.client.0.smithi107.stdout:3/506: mknod d9/d22/d1b/d55/c9e 0 2024-04-25T00:18:32.456 INFO:tasks.workunit.client.0.smithi107.stdout:6/416: rename d1/d2/db/d3c/d43/c5a to d1/d14/c90 0 2024-04-25T00:18:32.456 INFO:tasks.workunit.client.0.smithi107.stdout:6/417: readlink d1/d2/l55 0 2024-04-25T00:18:32.456 INFO:tasks.workunit.client.0.smithi107.stdout:6/418: fsync d1/d2/db/d32/f73 0 2024-04-25T00:18:32.458 INFO:tasks.workunit.client.0.smithi107.stdout:3/507: dread d9/f61 [0,4194304] 0 2024-04-25T00:18:32.458 INFO:tasks.workunit.client.0.smithi107.stdout:9/553: dwrite d0/d8a/d93/d56/d6c/d80/f97 [4194304,4194304] 0 2024-04-25T00:18:32.458 INFO:tasks.workunit.client.0.smithi107.stdout:9/554: readlink d0/d8a/d93/d6f/l8d 0 2024-04-25T00:18:32.460 INFO:tasks.workunit.client.0.smithi107.stdout:3/508: creat d9/d15/d39/f9f x:0 0 0 2024-04-25T00:18:32.460 INFO:tasks.workunit.client.0.smithi107.stdout:3/509: fsync d9/f61 0 2024-04-25T00:18:32.463 INFO:tasks.workunit.client.0.smithi107.stdout:3/510: write d9/d22/d1b/f81 [495156,14658] 0 2024-04-25T00:18:32.463 INFO:tasks.workunit.client.0.smithi107.stdout:3/511: stat d9/d22/d1b/d55/f5d 0 2024-04-25T00:18:32.463 INFO:tasks.workunit.client.0.smithi107.stdout:3/512: write d9/d15/d39/f9f [96846,123441] 0 2024-04-25T00:18:32.463 INFO:tasks.workunit.client.0.smithi107.stdout:3/513: chown d9/d22/d1b/d55/f5b 7 1 2024-04-25T00:18:32.464 INFO:tasks.workunit.client.0.smithi107.stdout:3/514: creat d9/d22/fa0 x:0 0 0 2024-04-25T00:18:32.472 INFO:tasks.workunit.client.0.smithi107.stdout:3/515: rename d9/d54/d5c/d68/d44 to d9/d15/da1 0 2024-04-25T00:18:32.474 INFO:tasks.workunit.client.0.smithi107.stdout:3/516: truncate d9/f56 3559066 0 2024-04-25T00:18:32.475 INFO:tasks.workunit.client.0.smithi107.stdout:3/517: truncate d9/d54/d5c/d68/f46 671293 0 2024-04-25T00:18:32.482 INFO:tasks.workunit.client.0.smithi107.stdout:3/518: getdents d9/d22 0 2024-04-25T00:18:32.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:32 smithi107 ceph-mon[29153]: pgmap v65: 129 pgs: 129 active+clean; 706 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 79 MiB/s rd, 89 MiB/s wr, 187 op/s 2024-04-25T00:18:32.485 INFO:tasks.workunit.client.0.smithi107.stdout:6/419: dwrite d1/d2/db/d3c/f50 [0,4194304] 0 2024-04-25T00:18:32.485 INFO:tasks.workunit.client.0.smithi107.stdout:9/555: dwrite d0/d43/f60 [0,4194304] 0 2024-04-25T00:18:32.485 INFO:tasks.workunit.client.0.smithi107.stdout:9/556: chown d0/d8a/d93/d6f/c92 462594217 1 2024-04-25T00:18:32.485 INFO:tasks.workunit.client.0.smithi107.stdout:9/557: stat d0/d8a/d93/de/d5c/f72 0 2024-04-25T00:18:32.485 INFO:tasks.workunit.client.0.smithi107.stdout:9/558: write d0/d8a/d93/f5a [2349333,128726] 0 2024-04-25T00:18:32.486 INFO:tasks.workunit.client.0.smithi107.stdout:9/559: fsync d0/d8a/d93/de/f83 0 2024-04-25T00:18:32.486 INFO:tasks.workunit.client.0.smithi107.stdout:3/519: symlink d9/d22/d1b/la2 0 2024-04-25T00:18:32.486 INFO:tasks.workunit.client.0.smithi107.stdout:9/560: getdents d0/d8a 0 2024-04-25T00:18:32.488 INFO:tasks.workunit.client.0.smithi107.stdout:3/520: unlink d9/d15/da1/f77 0 2024-04-25T00:18:32.488 INFO:tasks.workunit.client.0.smithi107.stdout:3/521: stat l8 0 2024-04-25T00:18:32.488 INFO:tasks.workunit.client.0.smithi107.stdout:9/561: stat d0/d8a/d93/d6f/l8d 0 2024-04-25T00:18:32.489 INFO:tasks.workunit.client.0.smithi107.stdout:3/522: symlink d9/d88/d4e/la3 0 2024-04-25T00:18:32.489 INFO:tasks.workunit.client.0.smithi107.stdout:9/562: truncate d0/d8a/d93/d56/d76/d50/f54 3636692 0 2024-04-25T00:18:32.489 INFO:tasks.workunit.client.0.smithi107.stdout:6/420: mknod d1/d4c/c91 0 2024-04-25T00:18:32.490 INFO:tasks.workunit.client.0.smithi107.stdout:3/523: getdents d9/d88/d4e/d93 0 2024-04-25T00:18:32.491 INFO:tasks.workunit.client.0.smithi107.stdout:6/421: link d1/d2/db/d32/l62 d1/d2/d3/d28/d79/d6b/l92 0 2024-04-25T00:18:32.491 INFO:tasks.workunit.client.0.smithi107.stdout:6/422: fdatasync d1/d2/d3/f41 0 2024-04-25T00:18:32.491 INFO:tasks.workunit.client.0.smithi107.stdout:6/423: write d1/d2/db/d32/d33/f63 [939877,64353] 0 2024-04-25T00:18:32.491 INFO:tasks.workunit.client.0.smithi107.stdout:9/563: mknod d0/d8a/d93/d1f/cc5 0 2024-04-25T00:18:32.492 INFO:tasks.workunit.client.0.smithi107.stdout:6/424: creat d1/d2/db/d32/d33/d4f/f93 x:0 0 0 2024-04-25T00:18:32.492 INFO:tasks.workunit.client.0.smithi107.stdout:3/524: mkdir d9/d15/da1/d4b/da4 0 2024-04-25T00:18:32.493 INFO:tasks.workunit.client.0.smithi107.stdout:9/564: truncate d0/d8a/d93/f45 3132085 0 2024-04-25T00:18:32.493 INFO:tasks.workunit.client.0.smithi107.stdout:9/565: dread - d0/d8a/d93/d56/d76/d50/d7d/fbc zero size 2024-04-25T00:18:32.493 INFO:tasks.workunit.client.0.smithi107.stdout:6/425: mkdir d1/d2/d3/d71/d94 0 2024-04-25T00:18:32.494 INFO:tasks.workunit.client.0.smithi107.stdout:6/426: truncate d1/d2/db/d32/d36/d3d/f60 758212 0 2024-04-25T00:18:32.494 INFO:tasks.workunit.client.0.smithi107.stdout:6/427: dread - d1/d2/d2b/d3e/f8f zero size 2024-04-25T00:18:32.494 INFO:tasks.workunit.client.0.smithi107.stdout:6/428: truncate d1/d2/db/f19 3547441 0 2024-04-25T00:18:32.494 INFO:tasks.workunit.client.0.smithi107.stdout:3/525: mknod d9/d15/da1/d4b/ca5 0 2024-04-25T00:18:32.494 INFO:tasks.workunit.client.0.smithi107.stdout:9/566: mkdir d0/d8a/d93/dc2/dc6 0 2024-04-25T00:18:32.496 INFO:tasks.workunit.client.0.smithi107.stdout:3/526: unlink l3 0 2024-04-25T00:18:32.499 INFO:tasks.workunit.client.0.smithi107.stdout:3/527: unlink d9/d22/fa0 0 2024-04-25T00:18:32.501 INFO:tasks.workunit.client.0.smithi107.stdout:6/429: rename d1/d2/db/d32/d36/d3d/f60 to d1/d2/d2b/d3e/d5b/d7a/f95 0 2024-04-25T00:18:32.502 INFO:tasks.workunit.client.0.smithi107.stdout:6/430: mkdir d1/d2/db/d3c/d43/d96 0 2024-04-25T00:18:32.502 INFO:tasks.workunit.client.0.smithi107.stdout:6/431: chown d1/d2/db/d32/f8e 874 1 2024-04-25T00:18:32.503 INFO:tasks.workunit.client.0.smithi107.stdout:9/567: rename d0/d8a/d93/d56/f98 to d0/d8a/d93/dc2/fc7 0 2024-04-25T00:18:32.503 INFO:tasks.workunit.client.0.smithi107.stdout:6/432: truncate d1/d2/d3/f13 3438846 0 2024-04-25T00:18:32.507 INFO:tasks.workunit.client.0.smithi107.stdout:6/433: rename d1/d2/d3/d28/d79/l8c to d1/d2/db/d32/d33/l97 0 2024-04-25T00:18:32.507 INFO:tasks.workunit.client.0.smithi107.stdout:6/434: fsync d1/d2/db/d32/d33/d68/d75/f8b 0 2024-04-25T00:18:32.513 INFO:tasks.workunit.client.0.smithi107.stdout:3/528: write d9/d22/d1b/d55/f5b [3491480,59891] 0 2024-04-25T00:18:32.514 INFO:tasks.workunit.client.0.smithi107.stdout:3/529: link d9/d88/d2c/l97 d9/d88/d3a/la6 0 2024-04-25T00:18:32.516 INFO:tasks.workunit.client.0.smithi107.stdout:3/530: dread d9/d15/d39/f9f [0,4194304] 0 2024-04-25T00:18:32.516 INFO:tasks.workunit.client.0.smithi107.stdout:6/435: write d1/f1f [2919997,22427] 0 2024-04-25T00:18:32.518 INFO:tasks.workunit.client.0.smithi107.stdout:6/436: symlink d1/d2/d3/d28/d79/l98 0 2024-04-25T00:18:32.518 INFO:tasks.workunit.client.0.smithi107.stdout:6/437: truncate d1/d2/d2b/d3e/d5b/d7a/f95 1361908 0 2024-04-25T00:18:32.519 INFO:tasks.workunit.client.0.smithi107.stdout:3/531: unlink d9/d15/f35 0 2024-04-25T00:18:32.519 INFO:tasks.workunit.client.0.smithi107.stdout:3/532: fsync d9/d88/d4e/d93/f9d 0 2024-04-25T00:18:32.519 INFO:tasks.workunit.client.0.smithi107.stdout:3/533: write d9/d88/d3a/d91/f9a [992207,55131] 0 2024-04-25T00:18:32.519 INFO:tasks.workunit.client.0.smithi107.stdout:3/534: readlink d9/d88/d4e/la3 0 2024-04-25T00:18:32.519 INFO:tasks.workunit.client.0.smithi107.stdout:6/438: creat d1/d2/db/d32/d36/f99 x:0 0 0 2024-04-25T00:18:32.519 INFO:tasks.workunit.client.0.smithi107.stdout:6/439: truncate d1/d2/db/d32/f8e 167239 0 2024-04-25T00:18:32.531 INFO:tasks.workunit.client.0.smithi107.stdout:6/440: dread d1/d2/db/f34 [0,4194304] 0 2024-04-25T00:18:32.550 INFO:tasks.workunit.client.0.smithi107.stdout:6/441: dwrite d1/d14/f89 [0,4194304] 0 2024-04-25T00:18:32.554 INFO:tasks.workunit.client.0.smithi107.stdout:6/442: write d1/f1f [8006170,77504] 0 2024-04-25T00:18:32.556 INFO:tasks.workunit.client.0.smithi107.stdout:6/443: creat d1/d2/db/d32/d36/d3a/f9a x:0 0 0 2024-04-25T00:18:32.556 INFO:tasks.workunit.client.0.smithi107.stdout:6/444: write d1/d14/d18/f2c [2688035,95232] 0 2024-04-25T00:18:32.556 INFO:tasks.workunit.client.0.smithi107.stdout:6/445: readlink d1/d2/d2b/l1a 0 2024-04-25T00:18:32.633 INFO:tasks.workunit.client.0.smithi107.stdout:2/549: sync 2024-04-25T00:18:32.650 INFO:tasks.workunit.client.0.smithi107.stdout:2/550: dwrite d6/d18/d40/d42/d2b/f7a [0,4194304] 0 2024-04-25T00:18:32.652 INFO:tasks.workunit.client.0.smithi107.stdout:2/551: mknod d6/d18/d40/cbd 0 2024-04-25T00:18:32.654 INFO:tasks.workunit.client.0.smithi107.stdout:2/552: symlink d6/d20/d77/d9f/lbe 0 2024-04-25T00:18:32.655 INFO:tasks.workunit.client.0.smithi107.stdout:2/553: truncate d6/d18/d40/d42/d69/d82/fb8 876609 0 2024-04-25T00:18:32.655 INFO:tasks.workunit.client.0.smithi107.stdout:2/554: fdatasync d6/d18/d40/d42/d2b/d30/f7c 0 2024-04-25T00:18:32.656 INFO:tasks.workunit.client.0.smithi107.stdout:2/555: creat d6/d18/d40/d42/d2b/d30/fbf x:0 0 0 2024-04-25T00:18:32.657 INFO:tasks.workunit.client.0.smithi107.stdout:2/556: mkdir d6/d29/d4e/dc0 0 2024-04-25T00:18:32.682 INFO:tasks.workunit.client.0.smithi107.stdout:2/557: dwrite d6/d20/d22/d56/f96 [4194304,4194304] 0 2024-04-25T00:18:32.682 INFO:tasks.workunit.client.0.smithi107.stdout:2/558: fdatasync d6/d18/d40/d42/d2b/f41 0 2024-04-25T00:18:32.682 INFO:tasks.workunit.client.0.smithi107.stdout:2/559: read d6/ff [5032724,105692] 0 2024-04-25T00:18:32.682 INFO:tasks.workunit.client.0.smithi107.stdout:2/560: fsync d6/d18/d40/d42/d69/fb3 0 2024-04-25T00:18:32.683 INFO:tasks.workunit.client.0.smithi107.stdout:2/561: mknod d6/d20/cc1 0 2024-04-25T00:18:32.685 INFO:tasks.workunit.client.0.smithi107.stdout:2/562: mkdir d6/d18/d40/d42/d69/d6e/dc2 0 2024-04-25T00:18:32.686 INFO:tasks.workunit.client.0.smithi107.stdout:2/563: rmdir d6/d18/d40/d42/d69/d6e 39 2024-04-25T00:18:32.687 INFO:tasks.workunit.client.0.smithi107.stdout:2/564: symlink d6/d18/d40/d42/d2b/d30/d4c/lc3 0 2024-04-25T00:18:32.689 INFO:tasks.workunit.client.0.smithi107.stdout:2/565: creat d6/d18/d40/d42/d69/d6e/db6/fc4 x:0 0 0 2024-04-25T00:18:32.704 INFO:tasks.workunit.client.0.smithi107.stdout:2/566: dwrite d6/d18/d40/d42/d69/fb3 [0,4194304] 0 2024-04-25T00:18:32.705 INFO:tasks.workunit.client.0.smithi107.stdout:2/567: chown d6/d18/d40/d42/d69/d82/fb8 55895 1 2024-04-25T00:18:32.707 INFO:tasks.workunit.client.0.smithi107.stdout:2/568: truncate d6/f1e 2374228 0 2024-04-25T00:18:32.707 INFO:tasks.workunit.client.0.smithi107.stdout:2/569: chown d6/d18/d40/cbd 32 1 2024-04-25T00:18:32.707 INFO:tasks.workunit.client.0.smithi107.stdout:2/570: write d6/d18/fb5 [317836,71279] 0 2024-04-25T00:18:32.707 INFO:tasks.workunit.client.0.smithi107.stdout:2/571: dread - d6/dad/f98 zero size 2024-04-25T00:18:32.709 INFO:tasks.workunit.client.0.smithi107.stdout:2/572: mkdir d6/d18/d40/d42/d69/d6e/da2/dc5 0 2024-04-25T00:18:32.709 INFO:tasks.workunit.client.0.smithi107.stdout:2/573: chown d6/d18/d40/d42/d2b/d30/l60 30 1 2024-04-25T00:18:32.710 INFO:tasks.workunit.client.0.smithi107.stdout:2/574: truncate d6/d18/d40/d42/d2b/f41 801055 0 2024-04-25T00:18:32.721 INFO:tasks.workunit.client.0.smithi107.stdout:2/575: write d6/d18/d40/d42/f47 [3976241,85995] 0 2024-04-25T00:18:32.722 INFO:tasks.workunit.client.0.smithi107.stdout:2/576: creat d6/dad/d67/da8/fc6 x:0 0 0 2024-04-25T00:18:32.722 INFO:tasks.workunit.client.0.smithi107.stdout:2/577: write d6/d18/d40/d42/d69/d6e/d80/f86 [976555,76785] 0 2024-04-25T00:18:32.722 INFO:tasks.workunit.client.0.smithi107.stdout:2/578: chown f4 10991 1 2024-04-25T00:18:32.724 INFO:tasks.workunit.client.0.smithi107.stdout:2/579: mknod d6/dad/db0/dbc/cc7 0 2024-04-25T00:18:32.725 INFO:tasks.workunit.client.0.smithi107.stdout:2/580: mkdir d6/d18/d40/dc8 0 2024-04-25T00:18:32.725 INFO:tasks.workunit.client.0.smithi107.stdout:2/581: write d6/d29/d4e/f7f [542316,40284] 0 2024-04-25T00:18:32.733 INFO:tasks.workunit.client.0.smithi107.stdout:2/582: dread d6/d18/d40/d42/d70/f76 [0,4194304] 0 2024-04-25T00:18:32.733 INFO:tasks.workunit.client.0.smithi107.stdout:2/583: fdatasync d6/d20/d22/fa0 0 2024-04-25T00:18:32.734 INFO:tasks.workunit.client.0.smithi107.stdout:2/584: truncate d6/d18/d40/f8f 911748 0 2024-04-25T00:18:32.734 INFO:tasks.workunit.client.0.smithi107.stdout:2/585: mkdir d6/d18/d40/d42/d2b/d30/d4c/dc9 0 2024-04-25T00:18:32.749 INFO:tasks.workunit.client.0.smithi107.stdout:2/586: dwrite d6/d18/fb1 [0,4194304] 0 2024-04-25T00:18:32.750 INFO:tasks.workunit.client.0.smithi107.stdout:2/587: write d6/d18/d40/d42/d2b/d30/d45/fb9 [851639,129933] 0 2024-04-25T00:18:32.751 INFO:tasks.workunit.client.0.smithi107.stdout:2/588: mkdir d6/d18/d40/d42/d2b/d30/d45/dca 0 2024-04-25T00:18:32.753 INFO:tasks.workunit.client.0.smithi107.stdout:2/589: rename d6/d18/d40/d42/d69/d6e/d80/f8b to d6/d18/d40/d42/d69/d6e/d80/fcb 0 2024-04-25T00:18:32.754 INFO:tasks.workunit.client.0.smithi107.stdout:2/590: read d6/d29/d4e/f7f [114467,78091] 0 2024-04-25T00:18:32.759 INFO:tasks.workunit.client.0.smithi107.stdout:4/670: sync 2024-04-25T00:18:32.759 INFO:tasks.workunit.client.0.smithi107.stdout:8/570: sync 2024-04-25T00:18:32.760 INFO:tasks.workunit.client.0.smithi107.stdout:8/571: symlink d2/d18/d1a/d5e/d5c/lbf 0 2024-04-25T00:18:32.761 INFO:tasks.workunit.client.0.smithi107.stdout:8/572: rename d2/d18/d1a/d5e/d3d/c77 to d2/d18/d1a/d5e/d5c/d91/cc0 0 2024-04-25T00:18:32.763 INFO:tasks.workunit.client.0.smithi107.stdout:8/573: link d2/d18/d1a/d33/fab d2/d18/d1a/d5e/d6d/fc1 0 2024-04-25T00:18:32.763 INFO:tasks.workunit.client.0.smithi107.stdout:8/574: readlink d2/d18/d1a/d5e/d3d/la9 0 2024-04-25T00:18:32.763 INFO:tasks.workunit.client.0.smithi107.stdout:4/671: unlink d3/d24/d25/d71/la7 0 2024-04-25T00:18:32.763 INFO:tasks.workunit.client.0.smithi107.stdout:4/672: dread - d3/d5/d32/d53/d7e/fd4 zero size 2024-04-25T00:18:32.763 INFO:tasks.workunit.client.0.smithi107.stdout:8/575: rmdir d2/d18/d1a/d5e/d5c/d91/da3 39 2024-04-25T00:18:32.764 INFO:tasks.workunit.client.0.smithi107.stdout:4/673: creat d3/d24/d5c/fe1 x:0 0 0 2024-04-25T00:18:32.771 INFO:tasks.workunit.client.0.smithi107.stdout:4/674: dread d3/d5/d32/d53/d7e/f86 [0,4194304] 0 2024-04-25T00:18:32.771 INFO:tasks.workunit.client.0.smithi107.stdout:4/675: dread - d3/d5/f22 zero size 2024-04-25T00:18:32.773 INFO:tasks.workunit.client.0.smithi107.stdout:4/676: mknod d3/dd8/ce2 0 2024-04-25T00:18:32.773 INFO:tasks.workunit.client.0.smithi107.stdout:8/576: read d2/d18/d1a/f7d [1288528,28584] 0 2024-04-25T00:18:32.773 INFO:tasks.workunit.client.0.smithi107.stdout:2/591: dwrite d6/d18/d40/d42/d2b/d30/d45/f61 [0,4194304] 0 2024-04-25T00:18:32.773 INFO:tasks.workunit.client.0.smithi107.stdout:4/677: rmdir d3/d24/d61 39 2024-04-25T00:18:32.776 INFO:tasks.workunit.client.0.smithi107.stdout:4/678: mknod d3/d5/d32/d53/d7e/d9d/ce3 0 2024-04-25T00:18:32.778 INFO:tasks.workunit.client.0.smithi107.stdout:4/679: creat d3/d24/d25/d71/d7f/fe4 x:0 0 0 2024-04-25T00:18:32.781 INFO:tasks.workunit.client.0.smithi107.stdout:4/680: write d3/d24/f95 [542899,66238] 0 2024-04-25T00:18:32.800 INFO:tasks.workunit.client.0.smithi107.stdout:2/592: dwrite d6/d20/d77/d9f/fa1 [0,4194304] 0 2024-04-25T00:18:32.800 INFO:tasks.workunit.client.0.smithi107.stdout:8/577: dwrite d2/d18/f53 [0,4194304] 0 2024-04-25T00:18:32.802 INFO:tasks.workunit.client.0.smithi107.stdout:2/593: mknod d6/d18/d40/d42/d69/d6e/da2/ccc 0 2024-04-25T00:18:32.803 INFO:tasks.workunit.client.0.smithi107.stdout:2/594: mknod d6/d20/d77/d9f/ccd 0 2024-04-25T00:18:32.804 INFO:tasks.workunit.client.0.smithi107.stdout:4/681: dread d3/d5/d32/f45 [0,4194304] 0 2024-04-25T00:18:32.806 INFO:tasks.workunit.client.0.smithi107.stdout:4/682: creat d3/d24/fe5 x:0 0 0 2024-04-25T00:18:32.806 INFO:tasks.workunit.client.0.smithi107.stdout:4/683: fsync d3/d5/d32/f79 0 2024-04-25T00:18:32.806 INFO:tasks.workunit.client.0.smithi107.stdout:4/684: truncate d3/d5/d32/d9e/fba 565328 0 2024-04-25T00:18:32.807 INFO:tasks.workunit.client.0.smithi107.stdout:4/685: creat d3/d5/fe6 x:0 0 0 2024-04-25T00:18:32.807 INFO:tasks.workunit.client.0.smithi107.stdout:4/686: truncate d3/d5/d32/f34 2751462 0 2024-04-25T00:18:32.807 INFO:tasks.workunit.client.0.smithi107.stdout:4/687: getdents d3/d10/d7b 0 2024-04-25T00:18:32.808 INFO:tasks.workunit.client.0.smithi107.stdout:2/595: dread d6/f72 [0,4194304] 0 2024-04-25T00:18:32.809 INFO:tasks.workunit.client.0.smithi107.stdout:4/688: unlink d3/d24/c80 0 2024-04-25T00:18:32.809 INFO:tasks.workunit.client.0.smithi107.stdout:2/596: getdents d6/d18/d40/d42/d2b 0 2024-04-25T00:18:32.809 INFO:tasks.workunit.client.0.smithi107.stdout:2/597: stat d6/d20 0 2024-04-25T00:18:32.811 INFO:tasks.workunit.client.0.smithi107.stdout:8/578: write d2/d18/d1a/d33/f64 [4076442,6937] 0 2024-04-25T00:18:32.812 INFO:tasks.workunit.client.0.smithi107.stdout:8/579: truncate d2/d18/d1a/d5e/d3d/f9d 3184370 0 2024-04-25T00:18:32.812 INFO:tasks.workunit.client.0.smithi107.stdout:8/580: write d2/d18/d1a/d5e/d4f/da0/fac [136140,14313] 0 2024-04-25T00:18:32.813 INFO:tasks.workunit.client.0.smithi107.stdout:8/581: mkdir d2/d18/da2/dc2 0 2024-04-25T00:18:32.814 INFO:tasks.workunit.client.0.smithi107.stdout:8/582: truncate d2/d18/d1a/d5e/fbe 31783 0 2024-04-25T00:18:32.815 INFO:tasks.workunit.client.0.smithi107.stdout:8/583: creat d2/d18/d1a/d5e/d4f/fc3 x:0 0 0 2024-04-25T00:18:32.819 INFO:tasks.workunit.client.0.smithi107.stdout:4/689: dread d3/d24/d25/f49 [0,4194304] 0 2024-04-25T00:18:32.820 INFO:tasks.workunit.client.0.smithi107.stdout:4/690: creat d3/d24/d5c/d7a/fe7 x:0 0 0 2024-04-25T00:18:32.834 INFO:tasks.workunit.client.0.smithi107.stdout:0/487: sync 2024-04-25T00:18:32.834 INFO:tasks.workunit.client.0.smithi107.stdout:7/452: sync 2024-04-25T00:18:32.835 INFO:tasks.workunit.client.0.smithi107.stdout:7/453: dread d0/d8/db/d18/d19/d47/f62 [0,4194304] 0 2024-04-25T00:18:32.836 INFO:tasks.workunit.client.0.smithi107.stdout:0/488: rmdir dc/dd/d5d 39 2024-04-25T00:18:32.837 INFO:tasks.workunit.client.0.smithi107.stdout:0/489: creat dc/dd/d16/d33/d35/f8d x:0 0 0 2024-04-25T00:18:32.838 INFO:tasks.workunit.client.0.smithi107.stdout:0/490: mknod dc/d24/c8e 0 2024-04-25T00:18:32.840 INFO:tasks.workunit.client.0.smithi107.stdout:2/598: read d6/d18/fb1 [750817,38143] 0 2024-04-25T00:18:32.840 INFO:tasks.workunit.client.0.smithi107.stdout:8/584: dwrite d2/d18/d1a/d5e/d6d/fc1 [0,4194304] 0 2024-04-25T00:18:32.845 INFO:tasks.workunit.client.0.smithi107.stdout:7/454: read d0/d8/fd [1947155,6275] 0 2024-04-25T00:18:32.845 INFO:tasks.workunit.client.0.smithi107.stdout:7/455: fdatasync d0/d8/d4e/f87 0 2024-04-25T00:18:32.850 INFO:tasks.workunit.client.0.smithi107.stdout:2/599: read d6/d29/d4e/d6f/f90 [110891,122510] 0 2024-04-25T00:18:32.859 INFO:tasks.workunit.client.0.smithi107.stdout:8/585: dread d2/f26 [0,4194304] 0 2024-04-25T00:18:32.859 INFO:tasks.workunit.client.0.smithi107.stdout:0/491: dwrite dc/d24/d25/d31/f5b [4194304,4194304] 0 2024-04-25T00:18:32.860 INFO:tasks.workunit.client.0.smithi107.stdout:0/492: creat dc/dd/d16/d5e/f8f x:0 0 0 2024-04-25T00:18:32.862 INFO:tasks.workunit.client.0.smithi107.stdout:0/493: mknod dc/dd/d5d/d3f/c90 0 2024-04-25T00:18:32.864 INFO:tasks.workunit.client.0.smithi107.stdout:0/494: creat dc/f91 x:0 0 0 2024-04-25T00:18:32.864 INFO:tasks.workunit.client.0.smithi107.stdout:0/495: rmdir dc/d24/d38/d64/d78 39 2024-04-25T00:18:32.865 INFO:tasks.workunit.client.0.smithi107.stdout:0/496: write dc/d24/d25/d31/f6f [743239,84587] 0 2024-04-25T00:18:32.871 INFO:tasks.workunit.client.0.smithi107.stdout:2/600: dwrite d6/d20/d22/fa0 [0,4194304] 0 2024-04-25T00:18:32.871 INFO:tasks.workunit.client.0.smithi107.stdout:7/456: dwrite d0/f11 [0,4194304] 0 2024-04-25T00:18:32.874 INFO:tasks.workunit.client.0.smithi107.stdout:2/601: creat d6/d18/d40/d42/d69/fce x:0 0 0 2024-04-25T00:18:32.874 INFO:tasks.workunit.client.0.smithi107.stdout:7/457: rmdir d0/d8/db 39 2024-04-25T00:18:32.875 INFO:tasks.workunit.client.0.smithi107.stdout:7/458: fsync d0/d8/db/d18/f8d 0 2024-04-25T00:18:32.875 INFO:tasks.workunit.client.0.smithi107.stdout:7/459: symlink d0/d8/db/d18/d19/d47/d4d/la5 0 2024-04-25T00:18:32.877 INFO:tasks.workunit.client.0.smithi107.stdout:7/460: creat d0/d8/db/d18/d19/d47/d4d/d8a/fa6 x:0 0 0 2024-04-25T00:18:32.878 INFO:tasks.workunit.client.0.smithi107.stdout:7/461: rename d0/d8/db/d18/d19/d47/d4d/c4f to d0/d8/db/ca7 0 2024-04-25T00:18:32.880 INFO:tasks.workunit.client.0.smithi107.stdout:7/462: dread d0/d8/d4e/f90 [0,4194304] 0 2024-04-25T00:18:32.881 INFO:tasks.workunit.client.0.smithi107.stdout:7/463: write d0/d8/db/f15 [5144262,106313] 0 2024-04-25T00:18:32.882 INFO:tasks.workunit.client.0.smithi107.stdout:7/464: mkdir d0/d80/da8 0 2024-04-25T00:18:32.892 INFO:tasks.workunit.client.0.smithi107.stdout:7/465: write d0/d8/db/d18/d19/f2f [1014723,120312] 0 2024-04-25T00:18:32.894 INFO:tasks.workunit.client.0.smithi107.stdout:2/602: dwrite d6/d18/f78 [0,4194304] 0 2024-04-25T00:18:32.897 INFO:tasks.workunit.client.0.smithi107.stdout:2/603: symlink d6/d18/d40/lcf 0 2024-04-25T00:18:32.897 INFO:tasks.workunit.client.0.smithi107.stdout:7/466: unlink d0/c38 0 2024-04-25T00:18:32.899 INFO:tasks.workunit.client.0.smithi107.stdout:7/467: symlink d0/d8/d2c/d58/la9 0 2024-04-25T00:18:32.899 INFO:tasks.workunit.client.0.smithi107.stdout:7/468: fdatasync d0/d8/db/d18/d19/d47/f62 0 2024-04-25T00:18:32.901 INFO:tasks.workunit.client.0.smithi107.stdout:2/604: mknod d6/d18/d40/d42/d2b/d30/d45/cd0 0 2024-04-25T00:18:32.902 INFO:tasks.workunit.client.0.smithi107.stdout:7/469: fdatasync d0/d8/db/d18/d19/f2f 0 2024-04-25T00:18:32.902 INFO:tasks.workunit.client.0.smithi107.stdout:7/470: dread - d0/d8/db/d18/d19/d47/f5b zero size 2024-04-25T00:18:32.905 INFO:tasks.workunit.client.0.smithi107.stdout:7/471: creat d0/d8/d2c/faa x:0 0 0 2024-04-25T00:18:32.905 INFO:tasks.workunit.client.0.smithi107.stdout:7/472: read d0/f13 [3294445,80305] 0 2024-04-25T00:18:32.905 INFO:tasks.workunit.client.0.smithi107.stdout:7/473: write d0/d8/d4e/f32 [1450377,96422] 0 2024-04-25T00:18:32.905 INFO:tasks.workunit.client.0.smithi107.stdout:7/474: dread - d0/d8/db/f49 zero size 2024-04-25T00:18:32.905 INFO:tasks.workunit.client.0.smithi107.stdout:7/475: write d0/d8/db/d18/d19/d47/f53 [1913988,42768] 0 2024-04-25T00:18:32.905 INFO:tasks.workunit.client.0.smithi107.stdout:7/476: rename d0/d8 to d0/d8/db/d18/d19/d39/dab 22 2024-04-25T00:18:32.905 INFO:tasks.workunit.client.0.smithi107.stdout:7/477: getdents d0/d8/db/d92 0 2024-04-25T00:18:32.905 INFO:tasks.workunit.client.0.smithi107.stdout:2/605: unlink d6/d20/d22/f1a 0 2024-04-25T00:18:32.911 INFO:tasks.workunit.client.0.smithi107.stdout:6/446: sync 2024-04-25T00:18:32.911 INFO:tasks.workunit.client.0.smithi107.stdout:1/502: sync 2024-04-25T00:18:32.911 INFO:tasks.workunit.client.0.smithi107.stdout:9/568: sync 2024-04-25T00:18:32.912 INFO:tasks.workunit.client.0.smithi107.stdout:3/535: sync 2024-04-25T00:18:32.912 INFO:tasks.workunit.client.0.smithi107.stdout:5/630: sync 2024-04-25T00:18:32.912 INFO:tasks.workunit.client.0.smithi107.stdout:3/536: truncate d9/d22/f7a 273129 0 2024-04-25T00:18:32.915 INFO:tasks.workunit.client.0.smithi107.stdout:1/503: creat d2/d2a/f9e x:0 0 0 2024-04-25T00:18:32.915 INFO:tasks.workunit.client.0.smithi107.stdout:1/504: fdatasync d2/dd/d25/f2e 0 2024-04-25T00:18:32.917 INFO:tasks.workunit.client.0.smithi107.stdout:1/505: getdents d2/d2a/d37/d88 0 2024-04-25T00:18:32.917 INFO:tasks.workunit.client.0.smithi107.stdout:1/506: chown d2/da/d2f/c45 236418052 1 2024-04-25T00:18:32.917 INFO:tasks.workunit.client.0.smithi107.stdout:1/507: readlink d2/d62/d66/d70/l9a 0 2024-04-25T00:18:32.917 INFO:tasks.workunit.client.0.smithi107.stdout:1/508: fdatasync d2/dd/d25/f63 0 2024-04-25T00:18:32.918 INFO:tasks.workunit.client.0.smithi107.stdout:2/606: dread d6/d18/fb1 [0,4194304] 0 2024-04-25T00:18:32.918 INFO:tasks.workunit.client.0.smithi107.stdout:2/607: chown d6/d18/d40/lcf 431739941 1 2024-04-25T00:18:32.918 INFO:tasks.workunit.client.0.smithi107.stdout:2/608: dread - d6/dad/f98 zero size 2024-04-25T00:18:32.918 INFO:tasks.workunit.client.0.smithi107.stdout:2/609: write d6/d18/d40/d42/d69/fce [893195,15495] 0 2024-04-25T00:18:32.919 INFO:tasks.workunit.client.0.smithi107.stdout:1/509: link d2/d2a/d37/l86 d2/d2a/d5c/l9f 0 2024-04-25T00:18:32.920 INFO:tasks.workunit.client.0.smithi107.stdout:2/610: unlink d6/d18/d40/d42/d2b/f32 0 2024-04-25T00:18:32.920 INFO:tasks.workunit.client.0.smithi107.stdout:2/611: truncate d6/d18/f5d 3145695 0 2024-04-25T00:18:32.921 INFO:tasks.workunit.client.0.smithi107.stdout:7/478: write d0/d8/db/d18/d5c/f85 [832072,75176] 0 2024-04-25T00:18:32.921 INFO:tasks.workunit.client.0.smithi107.stdout:7/479: chown d0 0 1 2024-04-25T00:18:32.922 INFO:tasks.workunit.client.0.smithi107.stdout:7/480: fdatasync d0/d8/f73 0 2024-04-25T00:18:32.922 INFO:tasks.workunit.client.0.smithi107.stdout:7/481: chown d0/d8/d2c/d58 2 1 2024-04-25T00:18:32.928 INFO:tasks.workunit.client.0.smithi107.stdout:9/569: dread d0/f94 [0,4194304] 0 2024-04-25T00:18:32.930 INFO:tasks.workunit.client.0.smithi107.stdout:9/570: creat d0/d8a/d93/d6f/da1/fc8 x:0 0 0 2024-04-25T00:18:32.938 INFO:tasks.workunit.client.0.smithi107.stdout:3/537: dwrite d9/d22/f1e [4194304,4194304] 0 2024-04-25T00:18:32.938 INFO:tasks.workunit.client.0.smithi107.stdout:6/447: dread d1/d2/db/fd [4194304,4194304] 0 2024-04-25T00:18:32.943 INFO:tasks.workunit.client.0.smithi107.stdout:7/482: dwrite d0/d8/f73 [0,4194304] 0 2024-04-25T00:18:32.943 INFO:tasks.workunit.client.0.smithi107.stdout:7/483: readlink d0/d8/db/d18/d19/d39/l4b 0 2024-04-25T00:18:32.945 INFO:tasks.workunit.client.0.smithi107.stdout:3/538: dread d9/d54/d5c/d68/f46 [0,4194304] 0 2024-04-25T00:18:32.946 INFO:tasks.workunit.client.0.smithi107.stdout:6/448: write d1/f1f [4495608,65629] 0 2024-04-25T00:18:32.946 INFO:tasks.workunit.client.0.smithi107.stdout:3/539: truncate d9/d88/d4e/f7b 2211139 0 2024-04-25T00:18:32.947 INFO:tasks.workunit.client.0.smithi107.stdout:3/540: dread d9/d22/f7a [0,4194304] 0 2024-04-25T00:18:32.947 INFO:tasks.workunit.client.0.smithi107.stdout:7/484: unlink d0/d8/d4e/l71 0 2024-04-25T00:18:32.949 INFO:tasks.workunit.client.0.smithi107.stdout:3/541: creat d9/d15/fa7 x:0 0 0 2024-04-25T00:18:32.951 INFO:tasks.workunit.client.0.smithi107.stdout:1/510: dwrite d2/dd/f4e [0,4194304] 0 2024-04-25T00:18:32.951 INFO:tasks.workunit.client.0.smithi107.stdout:1/511: fdatasync d2/d62/d66/d70/f7a 0 2024-04-25T00:18:32.951 INFO:tasks.workunit.client.0.smithi107.stdout:1/512: truncate d2/dd/d25/d5e/f81 4808217 0 2024-04-25T00:18:32.951 INFO:tasks.workunit.client.0.smithi107.stdout:1/513: chown d2/da/d2f/f9b 59867991 1 2024-04-25T00:18:32.975 INFO:tasks.workunit.client.0.smithi107.stdout:9/571: dwrite d0/f30 [0,4194304] 0 2024-04-25T00:18:32.975 INFO:tasks.workunit.client.0.smithi107.stdout:6/449: dread d1/d2/d3/f13 [0,4194304] 0 2024-04-25T00:18:32.975 INFO:tasks.workunit.client.0.smithi107.stdout:7/485: dwrite d0/f3b [0,4194304] 0 2024-04-25T00:18:32.976 INFO:tasks.workunit.client.0.smithi107.stdout:6/450: truncate d1/d2/d3/f13 1081423 0 2024-04-25T00:18:32.976 INFO:tasks.workunit.client.0.smithi107.stdout:6/451: chown d1/d14/c90 38053351 1 2024-04-25T00:18:32.976 INFO:tasks.workunit.client.0.smithi107.stdout:6/452: chown d1/d2/db/d32/d36/d7d 27056589 1 2024-04-25T00:18:32.976 INFO:tasks.workunit.client.0.smithi107.stdout:6/453: dread - d1/d2/d3/d26/f35 zero size 2024-04-25T00:18:32.976 INFO:tasks.workunit.client.0.smithi107.stdout:6/454: chown d1/d2/db/d32/d36/f46 3098 1 2024-04-25T00:18:32.983 INFO:tasks.workunit.client.0.smithi107.stdout:3/542: dwrite d9/d88/f30 [0,4194304] 0 2024-04-25T00:18:32.983 INFO:tasks.workunit.client.0.smithi107.stdout:3/543: dread - d9/d54/f83 zero size 2024-04-25T00:18:32.983 INFO:tasks.workunit.client.0.smithi107.stdout:1/514: dwrite f1 [0,4194304] 0 2024-04-25T00:18:32.986 INFO:tasks.workunit.client.0.smithi107.stdout:1/515: creat d2/dd/d82/fa0 x:0 0 0 2024-04-25T00:18:32.986 INFO:tasks.workunit.client.0.smithi107.stdout:1/516: chown d2/dd/d25/c32 2480 1 2024-04-25T00:18:32.986 INFO:tasks.workunit.client.0.smithi107.stdout:3/544: getdents d9/d88/d4e 0 2024-04-25T00:18:32.986 INFO:tasks.workunit.client.0.smithi107.stdout:3/545: chown d9/d88/d3a 4076 1 2024-04-25T00:18:32.987 INFO:tasks.workunit.client.0.smithi107.stdout:6/455: dread d1/d14/d18/f2c [0,4194304] 0 2024-04-25T00:18:32.989 INFO:tasks.workunit.client.0.smithi107.stdout:6/456: creat d1/d2/d3/d71/d94/f9b x:0 0 0 2024-04-25T00:18:32.989 INFO:tasks.workunit.client.0.smithi107.stdout:3/546: link d9/f9b d9/d15/da1/fa8 0 2024-04-25T00:18:32.989 INFO:tasks.workunit.client.0.smithi107.stdout:3/547: readlink d9/d22/d1b/l2b 0 2024-04-25T00:18:32.989 INFO:tasks.workunit.client.0.smithi107.stdout:1/517: dread d2/f51 [0,4194304] 0 2024-04-25T00:18:32.990 INFO:tasks.workunit.client.0.smithi107.stdout:6/457: mkdir d1/d2/db/d3c/d9c 0 2024-04-25T00:18:32.990 INFO:tasks.workunit.client.0.smithi107.stdout:1/518: symlink d2/d2a/la1 0 2024-04-25T00:18:32.996 INFO:tasks.workunit.client.0.smithi107.stdout:6/458: rmdir d1/d14/d18 39 2024-04-25T00:18:32.997 INFO:tasks.workunit.client.0.smithi107.stdout:6/459: fdatasync d1/d2/db/d3c/f50 0 2024-04-25T00:18:32.997 INFO:tasks.workunit.client.0.smithi107.stdout:6/460: write d1/d2/db/d32/d36/d3a/f9a [343771,55181] 0 2024-04-25T00:18:32.997 INFO:tasks.workunit.client.0.smithi107.stdout:6/461: write d1/d2/db/f19 [1476070,2247] 0 2024-04-25T00:18:32.997 INFO:tasks.workunit.client.0.smithi107.stdout:9/572: dwrite d0/d8a/d93/d6f/fc4 [0,4194304] 0 2024-04-25T00:18:32.998 INFO:tasks.workunit.client.0.smithi107.stdout:9/573: stat d0/d8a/d93/d56/d6c/lbb 0 2024-04-25T00:18:33.004 INFO:tasks.workunit.client.0.smithi107.stdout:6/462: write d1/d2/db/d3c/f72 [247314,39102] 0 2024-04-25T00:18:33.004 INFO:tasks.workunit.client.0.smithi107.stdout:6/463: readlink d1/d2/l55 0 2024-04-25T00:18:33.004 INFO:tasks.workunit.client.0.smithi107.stdout:9/574: dread d0/d8a/d93/de/f90 [0,4194304] 0 2024-04-25T00:18:33.004 INFO:tasks.workunit.client.0.smithi107.stdout:6/464: write d1/d2/d3/d28/f53 [1275605,49148] 0 2024-04-25T00:18:33.006 INFO:tasks.workunit.client.0.smithi107.stdout:9/575: mknod d0/d8a/d93/d6f/cc9 0 2024-04-25T00:18:33.006 INFO:tasks.workunit.client.0.smithi107.stdout:9/576: fdatasync d0/d8a/d93/de/f33 0 2024-04-25T00:18:33.007 INFO:tasks.workunit.client.0.smithi107.stdout:9/577: link d0/d8a/d93/d34/d36/ca7 d0/d8a/d93/d34/d36/cca 0 2024-04-25T00:18:33.007 INFO:tasks.workunit.client.0.smithi107.stdout:9/578: stat d0/d8a/d93/d1f/f86 0 2024-04-25T00:18:33.009 INFO:tasks.workunit.client.0.smithi107.stdout:9/579: symlink d0/d8a/d93/d56/d6c/daa/lcb 0 2024-04-25T00:18:33.011 INFO:tasks.workunit.client.0.smithi107.stdout:9/580: truncate d0/d43/f60 1804565 0 2024-04-25T00:18:33.011 INFO:tasks.workunit.client.0.smithi107.stdout:9/581: dread - d0/f4b zero size 2024-04-25T00:18:33.011 INFO:tasks.workunit.client.0.smithi107.stdout:9/582: chown d0/d8a/d93/d56/d6c 98691 1 2024-04-25T00:18:33.011 INFO:tasks.workunit.client.0.smithi107.stdout:9/583: write d0/d8a/d93/d1f/fb6 [687997,98636] 0 2024-04-25T00:18:33.012 INFO:tasks.workunit.client.0.smithi107.stdout:1/519: dwrite d2/da/d2f/f3e [4194304,4194304] 0 2024-04-25T00:18:33.012 INFO:tasks.workunit.client.0.smithi107.stdout:9/584: creat d0/d8a/d93/de/fcc x:0 0 0 2024-04-25T00:18:33.014 INFO:tasks.workunit.client.0.smithi107.stdout:1/520: chown d2/da/l59 0 1 2024-04-25T00:18:33.016 INFO:tasks.workunit.client.0.smithi107.stdout:9/585: symlink d0/d8a/d93/db9/lcd 0 2024-04-25T00:18:33.018 INFO:tasks.workunit.client.0.smithi107.stdout:1/521: symlink d2/dd/la2 0 2024-04-25T00:18:33.020 INFO:tasks.workunit.client.0.smithi107.stdout:1/522: truncate d2/f30 8387257 0 2024-04-25T00:18:33.022 INFO:tasks.workunit.client.0.smithi107.stdout:6/465: dwrite d1/d2/d2b/d3e/f8f [0,4194304] 0 2024-04-25T00:18:33.045 INFO:tasks.workunit.client.0.smithi107.stdout:9/586: dwrite d0/d8a/d93/d56/d76/d50/d7d/fae [0,4194304] 0 2024-04-25T00:18:33.048 INFO:tasks.workunit.client.0.smithi107.stdout:9/587: creat d0/d8a/d93/d6f/da1/fce x:0 0 0 2024-04-25T00:18:33.049 INFO:tasks.workunit.client.0.smithi107.stdout:1/523: dwrite d2/da/f53 [4194304,4194304] 0 2024-04-25T00:18:33.049 INFO:tasks.workunit.client.0.smithi107.stdout:9/588: link d0/d8a/d93/d56/d76/c9d d0/d8a/d93/d34/d36/d77/ccf 0 2024-04-25T00:18:33.049 INFO:tasks.workunit.client.0.smithi107.stdout:9/589: fsync d0/d8a/d93/d56/d76/d50/d7d/fae 0 2024-04-25T00:18:33.050 INFO:tasks.workunit.client.0.smithi107.stdout:9/590: symlink d0/d8a/d93/ld0 0 2024-04-25T00:18:33.051 INFO:tasks.workunit.client.0.smithi107.stdout:9/591: rmdir d0/d8a/d93/dc2 39 2024-04-25T00:18:33.051 INFO:tasks.workunit.client.0.smithi107.stdout:9/592: dread - d0/d8a/d93/d56/fba zero size 2024-04-25T00:18:33.054 INFO:tasks.workunit.client.0.smithi107.stdout:9/593: dread d0/d43/fd [0,4194304] 0 2024-04-25T00:18:33.068 INFO:tasks.workunit.client.0.smithi107.stdout:1/524: dwrite d2/dd/d25/d5e/f58 [0,4194304] 0 2024-04-25T00:18:33.071 INFO:tasks.workunit.client.0.smithi107.stdout:1/525: unlink d2/dd/d82/f92 0 2024-04-25T00:18:33.071 INFO:tasks.workunit.client.0.smithi107.stdout:9/594: dwrite d0/d8a/d93/d56/f59 [0,4194304] 0 2024-04-25T00:18:33.073 INFO:tasks.workunit.client.0.smithi107.stdout:9/595: mknod d0/d8a/d93/d56/d6c/cd1 0 2024-04-25T00:18:33.094 INFO:tasks.workunit.client.0.smithi107.stdout:9/596: dwrite d0/d8a/d93/d56/f64 [0,4194304] 0 2024-04-25T00:18:33.094 INFO:tasks.workunit.client.0.smithi107.stdout:9/597: write d0/f7e [553965,11408] 0 2024-04-25T00:18:33.094 INFO:tasks.workunit.client.0.smithi107.stdout:9/598: chown d0/d8a/d93/de/d5c/d95/l23 29621300 1 2024-04-25T00:18:33.094 INFO:tasks.workunit.client.0.smithi107.stdout:9/599: truncate d0/d8a/d93/d56/d76/f78 4550001 0 2024-04-25T00:18:33.099 INFO:tasks.workunit.client.0.smithi107.stdout:9/600: dread d0/d8a/d93/d1f/fb6 [0,4194304] 0 2024-04-25T00:18:33.099 INFO:tasks.workunit.client.0.smithi107.stdout:9/601: dread - d0/d8a/d93/d34/f70 zero size 2024-04-25T00:18:33.099 INFO:tasks.workunit.client.0.smithi107.stdout:9/602: chown d0/d8a/d93/d56/d6c/d80 7 1 2024-04-25T00:18:33.101 INFO:tasks.workunit.client.0.smithi107.stdout:9/603: link d0/d43/c25 d0/d8a/d93/d34/d36/cd2 0 2024-04-25T00:18:33.101 INFO:tasks.workunit.client.0.smithi107.stdout:9/604: chown d0/d8a/d93/d56/d76/f46 271690 1 2024-04-25T00:18:33.114 INFO:tasks.workunit.client.0.smithi107.stdout:9/605: write d0/f30 [766736,97758] 0 2024-04-25T00:18:33.114 INFO:tasks.workunit.client.0.smithi107.stdout:9/606: truncate d0/d8a/d93/d56/d76/d50/d7d/fbc 261813 0 2024-04-25T00:18:33.131 INFO:tasks.workunit.client.0.smithi107.stdout:9/607: dwrite d0/d8a/d93/d34/f70 [0,4194304] 0 2024-04-25T00:18:33.132 INFO:tasks.workunit.client.0.smithi107.stdout:9/608: mknod d0/d8a/d93/d56/d76/d50/cd3 0 2024-04-25T00:18:33.133 INFO:tasks.workunit.client.0.smithi107.stdout:9/609: read d0/d8a/d93/d56/d76/d50/f7a [220058,25943] 0 2024-04-25T00:18:33.134 INFO:tasks.workunit.client.0.smithi107.stdout:9/610: symlink d0/d43/ld4 0 2024-04-25T00:18:33.134 INFO:tasks.workunit.client.0.smithi107.stdout:9/611: dread - d0/d8a/d93/d56/d6c/d80/fad zero size 2024-04-25T00:18:33.135 INFO:tasks.workunit.client.0.smithi107.stdout:9/612: readlink d0/d8a/d93/d56/d6c/lbb 0 2024-04-25T00:18:33.137 INFO:tasks.workunit.client.0.smithi107.stdout:9/613: dread d0/d8a/d93/d56/d76/f5b [0,4194304] 0 2024-04-25T00:18:33.138 INFO:tasks.workunit.client.0.smithi107.stdout:9/614: write d0/d8a/d93/d56/d76/d50/f54 [4346210,73604] 0 2024-04-25T00:18:33.139 INFO:tasks.workunit.client.0.smithi107.stdout:9/615: write d0/d8a/d93/de/f90 [1009263,545] 0 2024-04-25T00:18:33.140 INFO:tasks.workunit.client.0.smithi107.stdout:9/616: chown d0/d8a/d93/d56/d76/c52 0 1 2024-04-25T00:18:33.140 INFO:tasks.workunit.client.0.smithi107.stdout:9/617: chown d0/d8a/d93/dc2/fc7 261 1 2024-04-25T00:18:33.157 INFO:tasks.workunit.client.0.smithi107.stdout:9/618: dwrite d0/d8a/d93/de/d5c/f72 [0,4194304] 0 2024-04-25T00:18:33.157 INFO:tasks.workunit.client.0.smithi107.stdout:9/619: chown d0/d8a/d93/d6f/cc9 5 1 2024-04-25T00:18:33.157 INFO:tasks.workunit.client.0.smithi107.stdout:9/620: write d0/d8a/d93/de/d5c/d95/f63 [517031,21883] 0 2024-04-25T00:18:33.158 INFO:tasks.workunit.client.0.smithi107.stdout:9/621: creat d0/d8a/d93/d56/d6c/daa/fd5 x:0 0 0 2024-04-25T00:18:33.158 INFO:tasks.workunit.client.0.smithi107.stdout:9/622: symlink d0/d43/ld6 0 2024-04-25T00:18:33.159 INFO:tasks.workunit.client.0.smithi107.stdout:9/623: mknod d0/d8a/d93/d56/d6c/cd7 0 2024-04-25T00:18:33.160 INFO:tasks.workunit.client.0.smithi107.stdout:9/624: unlink d0/d8a/d93/d56/d6c/d80/d9b/db2/cb4 0 2024-04-25T00:18:33.162 INFO:tasks.workunit.client.0.smithi107.stdout:9/625: unlink d0/f30 0 2024-04-25T00:18:33.168 INFO:tasks.workunit.client.0.smithi107.stdout:7/486: sync 2024-04-25T00:18:33.168 INFO:tasks.workunit.client.0.smithi107.stdout:5/631: sync 2024-04-25T00:18:33.168 INFO:tasks.workunit.client.0.smithi107.stdout:8/586: sync 2024-04-25T00:18:33.168 INFO:tasks.workunit.client.0.smithi107.stdout:2/612: sync 2024-04-25T00:18:33.168 INFO:tasks.workunit.client.0.smithi107.stdout:4/691: sync 2024-04-25T00:18:33.168 INFO:tasks.workunit.client.0.smithi107.stdout:0/497: sync 2024-04-25T00:18:33.169 INFO:tasks.workunit.client.0.smithi107.stdout:8/587: creat d2/d18/d1a/d5e/d3d/fc4 x:0 0 0 2024-04-25T00:18:33.170 INFO:tasks.workunit.client.0.smithi107.stdout:8/588: chown d2/d18/d1a/d5e/d4f/da0/da1/fad 380 1 2024-04-25T00:18:33.171 INFO:tasks.workunit.client.0.smithi107.stdout:8/589: write d2/d18/d1a/f1e [4843247,50984] 0 2024-04-25T00:18:33.172 INFO:tasks.workunit.client.0.smithi107.stdout:0/498: unlink dc/d24/d38/l6c 0 2024-04-25T00:18:33.174 INFO:tasks.workunit.client.0.smithi107.stdout:2/613: write d6/d18/d40/d42/d69/fce [12481,107549] 0 2024-04-25T00:18:33.174 INFO:tasks.workunit.client.0.smithi107.stdout:2/614: write d6/f72 [829639,34050] 0 2024-04-25T00:18:33.174 INFO:tasks.workunit.client.0.smithi107.stdout:2/615: fdatasync d6/d20/d22/d56/f68 0 2024-04-25T00:18:33.174 INFO:tasks.workunit.client.0.smithi107.stdout:9/626: write d0/fb1 [2483193,36141] 0 2024-04-25T00:18:33.174 INFO:tasks.workunit.client.0.smithi107.stdout:9/627: write d0/d8a/d93/d34/fa0 [1053589,112884] 0 2024-04-25T00:18:33.180 INFO:tasks.workunit.client.0.smithi107.stdout:0/499: dread dc/dd/f4c [0,4194304] 0 2024-04-25T00:18:33.182 INFO:tasks.workunit.client.0.smithi107.stdout:7/487: dread d0/f3b [0,4194304] 0 2024-04-25T00:18:33.183 INFO:tasks.workunit.client.0.smithi107.stdout:0/500: creat dc/dd/d5d/d3f/f92 x:0 0 0 2024-04-25T00:18:33.183 INFO:tasks.workunit.client.0.smithi107.stdout:0/501: fsync dc/dd/d5d/d5c/f71 0 2024-04-25T00:18:33.183 INFO:tasks.workunit.client.0.smithi107.stdout:0/502: fdatasync dc/dd/d16/d33/f3d 0 2024-04-25T00:18:33.183 INFO:tasks.workunit.client.0.smithi107.stdout:0/503: write dc/dd/d16/d5e/f8f [690177,100502] 0 2024-04-25T00:18:33.185 INFO:tasks.workunit.client.0.smithi107.stdout:7/488: creat d0/d8/d4e/d46/d93/fac x:0 0 0 2024-04-25T00:18:33.185 INFO:tasks.workunit.client.0.smithi107.stdout:9/628: dread d0/d8a/d93/d56/d76/f78 [0,4194304] 0 2024-04-25T00:18:33.185 INFO:tasks.workunit.client.0.smithi107.stdout:9/629: write d0/d8a/d93/de/d5c/f6a [974787,107848] 0 2024-04-25T00:18:33.186 INFO:tasks.workunit.client.0.smithi107.stdout:2/616: dread d6/d18/fb1 [0,4194304] 0 2024-04-25T00:18:33.186 INFO:tasks.workunit.client.0.smithi107.stdout:7/489: creat d0/d80/da2/fad x:0 0 0 2024-04-25T00:18:33.193 INFO:tasks.workunit.client.0.smithi107.stdout:9/630: dread d0/f94 [0,4194304] 0 2024-04-25T00:18:33.193 INFO:tasks.workunit.client.0.smithi107.stdout:9/631: write d0/d43/fd [1142220,57993] 0 2024-04-25T00:18:33.193 INFO:tasks.workunit.client.0.smithi107.stdout:8/590: write d2/d18/d1a/d5e/fa7 [1243958,88179] 0 2024-04-25T00:18:33.193 INFO:tasks.workunit.client.0.smithi107.stdout:9/632: write d0/d8a/d93/f45 [3315719,47478] 0 2024-04-25T00:18:33.195 INFO:tasks.workunit.client.0.smithi107.stdout:8/591: mknod d2/d18/da2/db2/cc5 0 2024-04-25T00:18:33.195 INFO:tasks.workunit.client.0.smithi107.stdout:8/592: write d2/d18/d1a/d5e/d3e/f86 [906691,3927] 0 2024-04-25T00:18:33.195 INFO:tasks.workunit.client.0.smithi107.stdout:8/593: chown d2/f16 0 1 2024-04-25T00:18:33.196 INFO:tasks.workunit.client.0.smithi107.stdout:9/633: mkdir d0/d8a/d93/d56/d76/d50/dd8 0 2024-04-25T00:18:33.196 INFO:tasks.workunit.client.0.smithi107.stdout:9/634: fdatasync d0/d8a/d93/d56/f59 0 2024-04-25T00:18:33.197 INFO:tasks.workunit.client.0.smithi107.stdout:0/504: write dc/dd/f3c [2174259,109065] 0 2024-04-25T00:18:33.197 INFO:tasks.workunit.client.0.smithi107.stdout:0/505: readlink dc/dd/d16/l69 0 2024-04-25T00:18:33.197 INFO:tasks.workunit.client.0.smithi107.stdout:8/594: link d2/d18/d1a/f7d d2/fc6 0 2024-04-25T00:18:33.198 INFO:tasks.workunit.client.0.smithi107.stdout:9/635: link d0/d8a/d93/c48 d0/d8a/d93/db9/cd9 0 2024-04-25T00:18:33.198 INFO:tasks.workunit.client.0.smithi107.stdout:9/636: write d0/d8a/d93/f99 [209071,88997] 0 2024-04-25T00:18:33.199 INFO:tasks.workunit.client.0.smithi107.stdout:5/632: dwrite d4/d35/f37 [0,4194304] 0 2024-04-25T00:18:33.199 INFO:tasks.workunit.client.0.smithi107.stdout:7/490: dread d0/d8/db/d18/f61 [0,4194304] 0 2024-04-25T00:18:33.199 INFO:tasks.workunit.client.0.smithi107.stdout:7/491: chown d0/d8/db/d18/d5c/f85 1 1 2024-04-25T00:18:33.199 INFO:tasks.workunit.client.0.smithi107.stdout:7/492: truncate d0/d80/da2/fad 68679 0 2024-04-25T00:18:33.199 INFO:tasks.workunit.client.0.smithi107.stdout:7/493: chown d0/d8/db/d18/d19/d47/d4d/l9f 7 1 2024-04-25T00:18:33.200 INFO:tasks.workunit.client.0.smithi107.stdout:7/494: getdents d0/d8/db/d18/d5c 0 2024-04-25T00:18:33.201 INFO:tasks.workunit.client.0.smithi107.stdout:8/595: rename d2/d18/d1a/d5e/d6d/cba to d2/d18/d1a/d5e/d3d/cc7 0 2024-04-25T00:18:33.201 INFO:tasks.workunit.client.0.smithi107.stdout:8/596: stat d2/d18/d1a/d5e/d4f/f7e 0 2024-04-25T00:18:33.201 INFO:tasks.workunit.client.0.smithi107.stdout:8/597: fsync d2/d18/d1a/d5e/d3d/fc4 0 2024-04-25T00:18:33.201 INFO:tasks.workunit.client.0.smithi107.stdout:2/617: write d6/d29/fa6 [2246627,48655] 0 2024-04-25T00:18:33.202 INFO:tasks.workunit.client.0.smithi107.stdout:9/637: getdents d0/d8a/d93/de 0 2024-04-25T00:18:33.204 INFO:tasks.workunit.client.0.smithi107.stdout:7/495: dread d0/d8/f1e [0,4194304] 0 2024-04-25T00:18:33.204 INFO:tasks.workunit.client.0.smithi107.stdout:7/496: read - d0/d8/db/d18/d19/d39/f96 zero size 2024-04-25T00:18:33.205 INFO:tasks.workunit.client.0.smithi107.stdout:7/497: creat d0/d8/db/d18/d19/d47/da1/fae x:0 0 0 2024-04-25T00:18:33.206 INFO:tasks.workunit.client.0.smithi107.stdout:8/598: symlink d2/d18/d1a/d5e/d3e/d74/lc8 0 2024-04-25T00:18:33.206 INFO:tasks.workunit.client.0.smithi107.stdout:8/599: chown d2/d18/d1a/d5e/d5c/d91/da3/fb8 162304 1 2024-04-25T00:18:33.206 INFO:tasks.workunit.client.0.smithi107.stdout:8/600: chown d2/d18/d1a/d5e/d5c/d6e 704097062 1 2024-04-25T00:18:33.206 INFO:tasks.workunit.client.0.smithi107.stdout:9/638: dread d0/d8a/d93/d34/f4f [0,4194304] 0 2024-04-25T00:18:33.207 INFO:tasks.workunit.client.0.smithi107.stdout:2/618: mkdir d6/d29/d4e/dc0/dd1 0 2024-04-25T00:18:33.207 INFO:tasks.workunit.client.0.smithi107.stdout:2/619: readlink d6/d18/d40/lcf 0 2024-04-25T00:18:33.207 INFO:tasks.workunit.client.0.smithi107.stdout:2/620: write d6/f17 [4594689,123940] 0 2024-04-25T00:18:33.207 INFO:tasks.workunit.client.0.smithi107.stdout:2/621: chown d6/dad/db0/dbc/cc7 19600389 1 2024-04-25T00:18:33.218 INFO:tasks.workunit.client.0.smithi107.stdout:5/633: dread d4/dc/d26/f4b [4194304,4194304] 0 2024-04-25T00:18:33.218 INFO:tasks.workunit.client.0.smithi107.stdout:5/634: write d4/dc/d26/d42/f76 [296338,27090] 0 2024-04-25T00:18:33.218 INFO:tasks.workunit.client.0.smithi107.stdout:5/635: read d4/d17/f79 [89841,126235] 0 2024-04-25T00:18:33.218 INFO:tasks.workunit.client.0.smithi107.stdout:5/636: chown d4/d17/f21 6885805 1 2024-04-25T00:18:33.218 INFO:tasks.workunit.client.0.smithi107.stdout:5/637: stat d4/d1c/fa8 0 2024-04-25T00:18:33.221 INFO:tasks.workunit.client.0.smithi107.stdout:8/601: getdents d2/d18/d1a/d5e/d5c/d6e 0 2024-04-25T00:18:33.222 INFO:tasks.workunit.client.0.smithi107.stdout:2/622: mkdir d6/d18/d40/d42/d69/d6e/da2/dc5/dd2 0 2024-04-25T00:18:33.225 INFO:tasks.workunit.client.0.smithi107.stdout:0/506: link dc/dd/d16/d33/d35/c47 dc/dd/d16/c93 0 2024-04-25T00:18:33.225 INFO:tasks.workunit.client.0.smithi107.stdout:0/507: read dc/d24/d25/f67 [86986,75035] 0 2024-04-25T00:18:33.225 INFO:tasks.workunit.client.0.smithi107.stdout:0/508: stat dc/f18 0 2024-04-25T00:18:33.225 INFO:tasks.workunit.client.0.smithi107.stdout:9/639: dread d0/d8a/d93/de/d5c/f6a [0,4194304] 0 2024-04-25T00:18:33.226 INFO:tasks.workunit.client.0.smithi107.stdout:0/509: symlink dc/dd/d16/d33/d35/d5f/l94 0 2024-04-25T00:18:33.226 INFO:tasks.workunit.client.0.smithi107.stdout:0/510: read - dc/dd/d5d/d7e/f85 zero size 2024-04-25T00:18:33.227 INFO:tasks.workunit.client.0.smithi107.stdout:0/511: getdents dc/dd/d16/d5e 0 2024-04-25T00:18:33.227 INFO:tasks.workunit.client.0.smithi107.stdout:8/602: creat d2/d18/d1a/d5e/d3e/fc9 x:0 0 0 2024-04-25T00:18:33.229 INFO:tasks.workunit.client.0.smithi107.stdout:9/640: mknod d0/d8a/d93/d56/d76/d50/dd8/cda 0 2024-04-25T00:18:33.231 INFO:tasks.workunit.client.0.smithi107.stdout:7/498: dwrite d0/d8/db/d18/d19/f2f [0,4194304] 0 2024-04-25T00:18:33.231 INFO:tasks.workunit.client.0.smithi107.stdout:0/512: unlink dc/dd/d16/d33/d35/d5f/l94 0 2024-04-25T00:18:33.231 INFO:tasks.workunit.client.0.smithi107.stdout:0/513: chown dc/d24/d38/d64/f88 56605134 1 2024-04-25T00:18:33.235 INFO:tasks.workunit.client.0.smithi107.stdout:0/514: write dc/d24/d25/d31/f6f [143880,107735] 0 2024-04-25T00:18:33.236 INFO:tasks.workunit.client.0.smithi107.stdout:7/499: dread d0/d8/f1e [0,4194304] 0 2024-04-25T00:18:33.236 INFO:tasks.workunit.client.0.smithi107.stdout:7/500: stat d0/d8/d2c/d58/f5e 0 2024-04-25T00:18:33.238 INFO:tasks.workunit.client.0.smithi107.stdout:0/515: creat dc/dd/d5d/f95 x:0 0 0 2024-04-25T00:18:33.238 INFO:tasks.workunit.client.0.smithi107.stdout:7/501: symlink d0/d8/db/d92/laf 0 2024-04-25T00:18:33.239 INFO:tasks.workunit.client.0.smithi107.stdout:8/603: dread d2/d18/d1a/f7d [0,4194304] 0 2024-04-25T00:18:33.239 INFO:tasks.workunit.client.0.smithi107.stdout:7/502: creat d0/d8/d4e/d46/d93/fb0 x:0 0 0 2024-04-25T00:18:33.240 INFO:tasks.workunit.client.0.smithi107.stdout:0/516: mknod dc/dd/d16/c96 0 2024-04-25T00:18:33.246 INFO:tasks.workunit.client.0.smithi107.stdout:9/641: dread d0/d8a/d93/d56/f59 [0,4194304] 0 2024-04-25T00:18:33.246 INFO:tasks.workunit.client.0.smithi107.stdout:9/642: dread - d0/d8a/d93/de/fcc zero size 2024-04-25T00:18:33.248 INFO:tasks.workunit.client.0.smithi107.stdout:5/638: dwrite d4/d1c/f28 [0,4194304] 0 2024-04-25T00:18:33.248 INFO:tasks.workunit.client.0.smithi107.stdout:5/639: write d4/d17/f2b [1613539,30181] 0 2024-04-25T00:18:33.250 INFO:tasks.workunit.client.0.smithi107.stdout:5/640: mkdir d4/d1c/d5c/dd4 0 2024-04-25T00:18:33.250 INFO:tasks.workunit.client.0.smithi107.stdout:5/641: write d4/fb [713847,73683] 0 2024-04-25T00:18:33.252 INFO:tasks.workunit.client.0.smithi107.stdout:5/642: mkdir d4/dc/d26/d62/dd5 0 2024-04-25T00:18:33.252 INFO:tasks.workunit.client.0.smithi107.stdout:0/517: dread dc/dd/f3c [0,4194304] 0 2024-04-25T00:18:33.256 INFO:tasks.workunit.client.0.smithi107.stdout:0/518: creat dc/d24/d38/d8b/f97 x:0 0 0 2024-04-25T00:18:33.256 INFO:tasks.workunit.client.0.smithi107.stdout:2/623: rename d6/dad/d67/l2d to d6/d18/d40/d42/d69/d6e/ld3 0 2024-04-25T00:18:33.257 INFO:tasks.workunit.client.0.smithi107.stdout:2/624: truncate d6/d18/d40/d42/d69/d6e/d80/fcb 741053 0 2024-04-25T00:18:33.258 INFO:tasks.workunit.client.0.smithi107.stdout:0/519: symlink dc/d24/d25/d31/l98 0 2024-04-25T00:18:33.258 INFO:tasks.workunit.client.0.smithi107.stdout:8/604: dread d2/d18/d1a/f7d [0,4194304] 0 2024-04-25T00:18:33.258 INFO:tasks.workunit.client.0.smithi107.stdout:8/605: stat d2/d18/d1a/d5e/d3d/l5a 0 2024-04-25T00:18:33.259 INFO:tasks.workunit.client.0.smithi107.stdout:2/625: mkdir d6/d18/d40/d42/d2b/d30/d45/dca/dd4 0 2024-04-25T00:18:33.259 INFO:tasks.workunit.client.0.smithi107.stdout:2/626: chown d6/dad/db0/dbc 2939888 1 2024-04-25T00:18:33.260 INFO:tasks.workunit.client.0.smithi107.stdout:2/627: dread d6/d18/d40/d42/d69/d82/fb8 [0,4194304] 0 2024-04-25T00:18:33.260 INFO:tasks.workunit.client.0.smithi107.stdout:0/520: creat dc/dd/d5d/d5c/d6e/f99 x:0 0 0 2024-04-25T00:18:33.261 INFO:tasks.workunit.client.0.smithi107.stdout:8/606: mkdir d2/d18/d1a/dca 0 2024-04-25T00:18:33.262 INFO:tasks.workunit.client.0.smithi107.stdout:0/521: creat dc/dd/d16/d33/f9a x:0 0 0 2024-04-25T00:18:33.263 INFO:tasks.workunit.client.0.smithi107.stdout:2/628: creat d6/dad/fd5 x:0 0 0 2024-04-25T00:18:33.263 INFO:tasks.workunit.client.0.smithi107.stdout:2/629: stat d6/d29/d4e 0 2024-04-25T00:18:33.264 INFO:tasks.workunit.client.0.smithi107.stdout:8/607: link d2/f8 d2/d18/d1a/dca/fcb 0 2024-04-25T00:18:33.264 INFO:tasks.workunit.client.0.smithi107.stdout:8/608: write d2/d18/d1a/d5e/d3d/fc4 [63971,24023] 0 2024-04-25T00:18:33.264 INFO:tasks.workunit.client.0.smithi107.stdout:8/609: truncate d2/d18/d1a/f73 489482 0 2024-04-25T00:18:33.264 INFO:tasks.workunit.client.0.smithi107.stdout:7/503: dwrite d0/d8/db/d18/f61 [0,4194304] 0 2024-04-25T00:18:33.264 INFO:tasks.workunit.client.0.smithi107.stdout:9/643: write d0/d8a/d93/d56/f64 [823567,111399] 0 2024-04-25T00:18:33.264 INFO:tasks.workunit.client.0.smithi107.stdout:0/522: mknod dc/d24/c9b 0 2024-04-25T00:18:33.266 INFO:tasks.workunit.client.0.smithi107.stdout:2/630: creat d6/d18/d40/fd6 x:0 0 0 2024-04-25T00:18:33.267 INFO:tasks.workunit.client.0.smithi107.stdout:0/523: dread dc/f18 [0,4194304] 0 2024-04-25T00:18:33.267 INFO:tasks.workunit.client.0.smithi107.stdout:0/524: chown dc/dd/f19 115 1 2024-04-25T00:18:33.268 INFO:tasks.workunit.client.0.smithi107.stdout:7/504: dread d0/d8/db/f98 [0,4194304] 0 2024-04-25T00:18:33.268 INFO:tasks.workunit.client.0.smithi107.stdout:8/610: creat d2/d18/d1a/d5e/d3e/fcc x:0 0 0 2024-04-25T00:18:33.271 INFO:tasks.workunit.client.0.smithi107.stdout:3/548: sync 2024-04-25T00:18:33.271 INFO:tasks.workunit.client.0.smithi107.stdout:3/549: truncate d9/d88/d4e/f70 724399 0 2024-04-25T00:18:33.276 INFO:tasks.workunit.client.0.smithi107.stdout:2/631: mknod d6/d29/d4e/dc0/dd1/cd7 0 2024-04-25T00:18:33.276 INFO:tasks.workunit.client.0.smithi107.stdout:9/644: creat d0/d8a/d93/d56/d76/d50/fdb x:0 0 0 2024-04-25T00:18:33.277 INFO:tasks.workunit.client.0.smithi107.stdout:2/632: chown d6/dad/db0/dbc/cc7 1 1 2024-04-25T00:18:33.278 INFO:tasks.workunit.client.0.smithi107.stdout:2/633: creat d6/d18/d40/d42/d69/d6e/da2/dc5/fd8 x:0 0 0 2024-04-25T00:18:33.279 INFO:tasks.workunit.client.0.smithi107.stdout:5/643: dwrite d4/dc/d26/d29/d2e/d82/fa6 [0,4194304] 0 2024-04-25T00:18:33.290 INFO:tasks.workunit.client.0.smithi107.stdout:2/634: write d6/d18/d40/d42/f3f [4072306,96318] 0 2024-04-25T00:18:33.291 INFO:tasks.workunit.client.0.smithi107.stdout:1/526: sync 2024-04-25T00:18:33.291 INFO:tasks.workunit.client.0.smithi107.stdout:6/466: sync 2024-04-25T00:18:33.292 INFO:tasks.workunit.client.0.smithi107.stdout:2/635: link d6/d29/d4e/l6d d6/d29/d4e/d6f/ld9 0 2024-04-25T00:18:33.293 INFO:tasks.workunit.client.0.smithi107.stdout:9/645: dwrite d0/d8a/d93/d56/d76/d50/fdb [0,4194304] 0 2024-04-25T00:18:33.293 INFO:tasks.workunit.client.0.smithi107.stdout:9/646: chown d0/d43/ca2 13 1 2024-04-25T00:18:33.293 INFO:tasks.workunit.client.0.smithi107.stdout:9/647: chown d0/d8a/d93/d1f/l89 0 1 2024-04-25T00:18:33.294 INFO:tasks.workunit.client.0.smithi107.stdout:1/527: rename d2/dd/d1b/f3a to d2/dd/d1b/fa3 0 2024-04-25T00:18:33.295 INFO:tasks.workunit.client.0.smithi107.stdout:4/692: sync 2024-04-25T00:18:33.295 INFO:tasks.workunit.client.0.smithi107.stdout:2/636: link d6/d18/d40/cbd d6/d18/d40/d42/d70/cda 0 2024-04-25T00:18:33.296 INFO:tasks.workunit.client.0.smithi107.stdout:4/693: write d3/d5/f65 [35647,37453] 0 2024-04-25T00:18:33.296 INFO:tasks.workunit.client.0.smithi107.stdout:6/467: mknod d1/d4c/c9d 0 2024-04-25T00:18:33.297 INFO:tasks.workunit.client.0.smithi107.stdout:4/694: rmdir d3/d5/d32/d53/d7e/d9d 39 2024-04-25T00:18:33.297 INFO:tasks.workunit.client.0.smithi107.stdout:9/648: creat d0/d8a/d93/de/fdc x:0 0 0 2024-04-25T00:18:33.298 INFO:tasks.workunit.client.0.smithi107.stdout:4/695: mknod d3/d5/d32/d9e/ce8 0 2024-04-25T00:18:33.298 INFO:tasks.workunit.client.0.smithi107.stdout:2/637: symlink d6/d18/d40/d42/d69/d6e/da2/dc5/dd2/ldb 0 2024-04-25T00:18:33.299 INFO:tasks.workunit.client.0.smithi107.stdout:6/468: mknod d1/d14/d18/c9e 0 2024-04-25T00:18:33.299 INFO:tasks.workunit.client.0.smithi107.stdout:5/644: dread d4/dc/d26/d29/d2e/d82/fa6 [0,4194304] 0 2024-04-25T00:18:33.300 INFO:tasks.workunit.client.0.smithi107.stdout:5/645: chown d4/dc/d26/d29/d2e/d82/fa6 29792177 1 2024-04-25T00:18:33.300 INFO:tasks.workunit.client.0.smithi107.stdout:9/649: rename d0/d8a/d93/c48 to d0/d8a/d93/dc2/cdd 0 2024-04-25T00:18:33.301 INFO:tasks.workunit.client.0.smithi107.stdout:6/469: creat d1/f9f x:0 0 0 2024-04-25T00:18:33.302 INFO:tasks.workunit.client.0.smithi107.stdout:4/696: rename d3/d5/lf to d3/d10/d7b/le9 0 2024-04-25T00:18:33.302 INFO:tasks.workunit.client.0.smithi107.stdout:5/646: getdents d4/dc/d26 0 2024-04-25T00:18:33.302 INFO:tasks.workunit.client.0.smithi107.stdout:5/647: fsync d4/dc/d26/d29/d2e/f6d 0 2024-04-25T00:18:33.302 INFO:tasks.workunit.client.0.smithi107.stdout:5/648: stat d4/d17/db3/d84/fb0 0 2024-04-25T00:18:33.303 INFO:tasks.workunit.client.0.smithi107.stdout:6/470: creat d1/d2/db/d32/d33/d68/fa0 x:0 0 0 2024-04-25T00:18:33.303 INFO:tasks.workunit.client.0.smithi107.stdout:6/471: chown d1/d14/d18/l2d 55 1 2024-04-25T00:18:33.305 INFO:tasks.workunit.client.0.smithi107.stdout:5/649: truncate d4/dc/d26/d62/fc8 785125 0 2024-04-25T00:18:33.305 INFO:tasks.workunit.client.0.smithi107.stdout:1/528: dread d2/dd/d82/f90 [0,4194304] 0 2024-04-25T00:18:33.305 INFO:tasks.workunit.client.0.smithi107.stdout:9/650: dread d0/d8a/d93/d1f/fb6 [0,4194304] 0 2024-04-25T00:18:33.305 INFO:tasks.workunit.client.0.smithi107.stdout:9/651: readlink d0/d8a/d93/d56/l75 0 2024-04-25T00:18:33.305 INFO:tasks.workunit.client.0.smithi107.stdout:6/472: dread d1/d2/db/d32/d36/f46 [0,4194304] 0 2024-04-25T00:18:33.305 INFO:tasks.workunit.client.0.smithi107.stdout:9/652: truncate d0/d8a/d93/d56/d76/d50/d7d/f85 1557536 0 2024-04-25T00:18:33.305 INFO:tasks.workunit.client.0.smithi107.stdout:9/653: read d0/f8f [723506,95907] 0 2024-04-25T00:18:33.305 INFO:tasks.workunit.client.0.smithi107.stdout:7/505: dwrite d0/d80/f88 [0,4194304] 0 2024-04-25T00:18:33.306 INFO:tasks.workunit.client.0.smithi107.stdout:3/550: dwrite d9/d88/f30 [4194304,4194304] 0 2024-04-25T00:18:33.306 INFO:tasks.workunit.client.0.smithi107.stdout:7/506: dread - d0/f69 zero size 2024-04-25T00:18:33.306 INFO:tasks.workunit.client.0.smithi107.stdout:3/551: chown d9/d54/d5c/d68/f3c 94027 1 2024-04-25T00:18:33.306 INFO:tasks.workunit.client.0.smithi107.stdout:0/525: dwrite dc/dd/d5d/d5c/f71 [0,4194304] 0 2024-04-25T00:18:33.309 INFO:tasks.workunit.client.0.smithi107.stdout:6/473: link d1/d2/db/d32/d36/f99 d1/d2/db/d32/d36/d3d/fa1 0 2024-04-25T00:18:33.322 INFO:tasks.workunit.client.0.smithi107.stdout:2/638: dwrite d6/d18/d40/d42/f47 [0,4194304] 0 2024-04-25T00:18:33.322 INFO:tasks.workunit.client.0.smithi107.stdout:2/639: fdatasync d6/d20/f89 0 2024-04-25T00:18:33.325 INFO:tasks.workunit.client.0.smithi107.stdout:2/640: dread d6/d18/d40/d42/d2b/d30/f4b [0,4194304] 0 2024-04-25T00:18:33.328 INFO:tasks.workunit.client.0.smithi107.stdout:4/697: dread d3/d24/f2c [0,4194304] 0 2024-04-25T00:18:33.331 INFO:tasks.workunit.client.0.smithi107.stdout:4/698: mkdir d3/d24/dea 0 2024-04-25T00:18:33.331 INFO:tasks.workunit.client.0.smithi107.stdout:4/699: readlink d3/d5/l4c 0 2024-04-25T00:18:33.331 INFO:tasks.workunit.client.0.smithi107.stdout:1/529: dread d2/dd/d25/d5e/f39 [0,4194304] 0 2024-04-25T00:18:33.331 INFO:tasks.workunit.client.0.smithi107.stdout:1/530: dread - d2/dd/d1a/f64 zero size 2024-04-25T00:18:33.331 INFO:tasks.workunit.client.0.smithi107.stdout:1/531: stat d2/dd/d1a/f27 0 2024-04-25T00:18:33.333 INFO:tasks.workunit.client.0.smithi107.stdout:4/700: unlink d3/d24/d5c/dab/fd7 0 2024-04-25T00:18:33.335 INFO:tasks.workunit.client.0.smithi107.stdout:3/552: dwrite d9/d15/da1/f79 [0,4194304] 0 2024-04-25T00:18:33.336 INFO:tasks.workunit.client.0.smithi107.stdout:2/641: dread d6/f1e [0,4194304] 0 2024-04-25T00:18:33.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:33 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:18:33.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:33 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:18:33.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:33 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:18:33.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:33 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:18:33.348 INFO:tasks.workunit.client.0.smithi107.stdout:7/507: dwrite d0/d8/db/f49 [0,4194304] 0 2024-04-25T00:18:33.350 INFO:tasks.workunit.client.0.smithi107.stdout:7/508: symlink d0/d8/db/d18/d19/d67/d7a/lb1 0 2024-04-25T00:18:33.352 INFO:tasks.workunit.client.0.smithi107.stdout:7/509: link d0/d8/d2c/d58/c68 d0/d80/da2/cb2 0 2024-04-25T00:18:33.352 INFO:tasks.workunit.client.0.smithi107.stdout:7/510: stat d0/c1a 0 2024-04-25T00:18:33.352 INFO:tasks.workunit.client.0.smithi107.stdout:7/511: truncate d0/d8/d2c/faa 397808 0 2024-04-25T00:18:33.352 INFO:tasks.workunit.client.0.smithi107.stdout:7/512: dread - d0/d8/d2c/d58/f5f zero size 2024-04-25T00:18:33.352 INFO:tasks.workunit.client.0.smithi107.stdout:9/654: dwrite d0/d8a/d93/d56/d76/fb8 [0,4194304] 0 2024-04-25T00:18:33.353 INFO:tasks.workunit.client.0.smithi107.stdout:4/701: dread d3/fd [0,4194304] 0 2024-04-25T00:18:33.353 INFO:tasks.workunit.client.0.smithi107.stdout:4/702: read - d3/d5/d32/d53/d7e/fd4 zero size 2024-04-25T00:18:33.353 INFO:tasks.workunit.client.0.smithi107.stdout:3/553: write d9/f20 [4243491,116751] 0 2024-04-25T00:18:33.353 INFO:tasks.workunit.client.0.smithi107.stdout:7/513: mknod d0/d8/d4e/cb3 0 2024-04-25T00:18:33.353 INFO:tasks.workunit.client.0.smithi107.stdout:0/526: dwrite dc/d24/f77 [0,4194304] 0 2024-04-25T00:18:33.354 INFO:tasks.workunit.client.0.smithi107.stdout:5/650: dwrite d4/dc/d26/f30 [0,4194304] 0 2024-04-25T00:18:33.356 INFO:tasks.workunit.client.0.smithi107.stdout:3/554: unlink d9/d15/da1/c8d 0 2024-04-25T00:18:33.358 INFO:tasks.workunit.client.0.smithi107.stdout:0/527: creat dc/dd/d5d/d5c/d6e/f9c x:0 0 0 2024-04-25T00:18:33.358 INFO:tasks.workunit.client.0.smithi107.stdout:0/528: stat dc/dd/d16/d33/d35/d5f/l79 0 2024-04-25T00:18:33.358 INFO:tasks.workunit.client.0.smithi107.stdout:2/642: dread d6/f1e [0,4194304] 0 2024-04-25T00:18:33.360 INFO:tasks.workunit.client.0.smithi107.stdout:7/514: truncate d0/d8/db/d18/f61 47699 0 2024-04-25T00:18:33.364 INFO:tasks.workunit.client.0.smithi107.stdout:7/515: link d0/d8/d4e/fa0 d0/d8/d2c/d58/fb4 0 2024-04-25T00:18:33.366 INFO:tasks.workunit.client.0.smithi107.stdout:7/516: mkdir d0/d8/d2c/d58/db5 0 2024-04-25T00:18:33.368 INFO:tasks.workunit.client.0.smithi107.stdout:7/517: truncate d0/d8/db/d18/d5c/f85 1327338 0 2024-04-25T00:18:33.368 INFO:tasks.workunit.client.0.smithi107.stdout:5/651: write d4/f16 [472051,114903] 0 2024-04-25T00:18:33.369 INFO:tasks.workunit.client.0.smithi107.stdout:1/532: dwrite d2/dd/d25/f7d [4194304,4194304] 0 2024-04-25T00:18:33.369 INFO:tasks.workunit.client.0.smithi107.stdout:0/529: getdents dc/dd/d5d/d3f/d59 0 2024-04-25T00:18:33.370 INFO:tasks.workunit.client.0.smithi107.stdout:7/518: creat d0/d8/db/d18/d19/d67/d7a/d82/fb6 x:0 0 0 2024-04-25T00:18:33.370 INFO:tasks.workunit.client.0.smithi107.stdout:7/519: fdatasync d0/d8/f1e 0 2024-04-25T00:18:33.372 INFO:tasks.workunit.client.0.smithi107.stdout:2/643: dread d6/ff [0,4194304] 0 2024-04-25T00:18:33.372 INFO:tasks.workunit.client.0.smithi107.stdout:0/530: symlink dc/dd/d16/d48/l9d 0 2024-04-25T00:18:33.373 INFO:tasks.workunit.client.0.smithi107.stdout:5/652: creat d4/d17/fd6 x:0 0 0 2024-04-25T00:18:33.373 INFO:tasks.workunit.client.0.smithi107.stdout:5/653: dread - d4/dc/d26/d29/d2e/d55/f9a zero size 2024-04-25T00:18:33.373 INFO:tasks.workunit.client.0.smithi107.stdout:1/533: read d2/dd/d25/f63 [100161,59277] 0 2024-04-25T00:18:33.374 INFO:tasks.workunit.client.0.smithi107.stdout:0/531: truncate dc/dd/d16/d33/d35/d5f/f82 4033690 0 2024-04-25T00:18:33.376 INFO:tasks.workunit.client.0.smithi107.stdout:0/532: write dc/f28 [508401,51145] 0 2024-04-25T00:18:33.378 INFO:tasks.workunit.client.0.smithi107.stdout:2/644: unlink d6/d18/d40/d42/d2b/d30/d4c/lae 0 2024-04-25T00:18:33.378 INFO:tasks.workunit.client.0.smithi107.stdout:4/703: dwrite d3/d5/d32/d53/d7e/fb3 [0,4194304] 0 2024-04-25T00:18:33.380 INFO:tasks.workunit.client.0.smithi107.stdout:2/645: creat d6/dad/db0/fdc x:0 0 0 2024-04-25T00:18:33.380 INFO:tasks.workunit.client.0.smithi107.stdout:2/646: fsync d6/f7 0 2024-04-25T00:18:33.385 INFO:tasks.workunit.client.0.smithi107.stdout:5/654: creat d4/fd7 x:0 0 0 2024-04-25T00:18:33.385 INFO:tasks.workunit.client.0.smithi107.stdout:1/534: mkdir d2/da4 0 2024-04-25T00:18:33.385 INFO:tasks.workunit.client.0.smithi107.stdout:7/520: dread d0/d8/db/d18/d19/d39/f6f [0,4194304] 0 2024-04-25T00:18:33.385 INFO:tasks.workunit.client.0.smithi107.stdout:7/521: dread - d0/d8/d2c/d58/f5f zero size 2024-04-25T00:18:33.385 INFO:tasks.workunit.client.0.smithi107.stdout:7/522: chown d0/d8/d2c/f3d 43 1 2024-04-25T00:18:33.387 INFO:tasks.workunit.client.0.smithi107.stdout:3/555: dwrite d9/f29 [0,4194304] 0 2024-04-25T00:18:33.387 INFO:tasks.workunit.client.0.smithi107.stdout:3/556: dread - d9/f99 zero size 2024-04-25T00:18:33.387 INFO:tasks.workunit.client.0.smithi107.stdout:3/557: chown d9/d22/d1b/f81 4 1 2024-04-25T00:18:33.387 INFO:tasks.workunit.client.0.smithi107.stdout:7/523: rename d0/d8/db/d18/d19/f1f to d0/d8/db/d18/d19/fb7 0 2024-04-25T00:18:33.388 INFO:tasks.workunit.client.0.smithi107.stdout:7/524: dread - d0/d8/d2c/d58/f79 zero size 2024-04-25T00:18:33.388 INFO:tasks.workunit.client.0.smithi107.stdout:7/525: truncate d0/d8/f4a 133913 0 2024-04-25T00:18:33.388 INFO:tasks.workunit.client.0.smithi107.stdout:7/526: chown d0/d8/db/d18/d5c/c7c 1351414 1 2024-04-25T00:18:33.389 INFO:tasks.workunit.client.0.smithi107.stdout:3/558: dread d9/d15/d39/f9f [0,4194304] 0 2024-04-25T00:18:33.389 INFO:tasks.workunit.client.0.smithi107.stdout:9/655: dwrite d0/d8a/d93/f45 [0,4194304] 0 2024-04-25T00:18:33.391 INFO:tasks.workunit.client.0.smithi107.stdout:5/655: mkdir d4/d35/dd8 0 2024-04-25T00:18:33.391 INFO:tasks.workunit.client.0.smithi107.stdout:0/533: rmdir dc 39 2024-04-25T00:18:33.391 INFO:tasks.workunit.client.0.smithi107.stdout:2/647: creat d6/d18/d40/d42/d69/d6e/db6/fdd x:0 0 0 2024-04-25T00:18:33.391 INFO:tasks.workunit.client.0.smithi107.stdout:0/534: readlink dc/dd/d16/l4a 0 2024-04-25T00:18:33.391 INFO:tasks.workunit.client.0.smithi107.stdout:0/535: write dc/d24/d25/f2b [4078356,74972] 0 2024-04-25T00:18:33.394 INFO:tasks.workunit.client.0.smithi107.stdout:0/536: creat dc/dd/d16/d33/d35/f9e x:0 0 0 2024-04-25T00:18:33.394 INFO:tasks.workunit.client.0.smithi107.stdout:0/537: write dc/dd/d5d/d3f/f61 [76637,20789] 0 2024-04-25T00:18:33.394 INFO:tasks.workunit.client.0.smithi107.stdout:0/538: chown dc/dd/d5d/d3f/l8c 1540408914 1 2024-04-25T00:18:33.401 INFO:tasks.workunit.client.0.smithi107.stdout:7/527: dread d0/d8/db/d18/d19/f2f [0,4194304] 0 2024-04-25T00:18:33.405 INFO:tasks.workunit.client.0.smithi107.stdout:4/704: dwrite d3/da8/fb7 [8388608,4194304] 0 2024-04-25T00:18:33.407 INFO:tasks.workunit.client.0.smithi107.stdout:5/656: write d4/fa [4519546,56153] 0 2024-04-25T00:18:33.407 INFO:tasks.workunit.client.0.smithi107.stdout:5/657: read - d4/dc/d26/d42/d8b/fa9 zero size 2024-04-25T00:18:33.407 INFO:tasks.workunit.client.0.smithi107.stdout:5/658: chown d4/dc/d26/d62/fcd 28495 1 2024-04-25T00:18:33.407 INFO:tasks.workunit.client.0.smithi107.stdout:4/705: truncate d3/d24/d25/d71/d7f/f9c 3012147 0 2024-04-25T00:18:33.411 INFO:tasks.workunit.client.0.smithi107.stdout:5/659: rename d4/d17/fb2 to d4/d7c/d88/d9d/fd9 0 2024-04-25T00:18:33.411 INFO:tasks.workunit.client.0.smithi107.stdout:9/656: dwrite d0/d8a/d93/d56/d76/f38 [0,4194304] 0 2024-04-25T00:18:33.411 INFO:tasks.workunit.client.0.smithi107.stdout:9/657: fdatasync d0/d8a/d93/de/f7c 0 2024-04-25T00:18:33.413 INFO:tasks.workunit.client.0.smithi107.stdout:9/658: truncate d0/d8a/d93/d56/f59 1227891 0 2024-04-25T00:18:33.415 INFO:tasks.workunit.client.0.smithi107.stdout:9/659: creat d0/d8a/d93/de/fde x:0 0 0 2024-04-25T00:18:33.415 INFO:tasks.workunit.client.0.smithi107.stdout:9/660: write d0/d8a/d93/d34/f84 [877630,59641] 0 2024-04-25T00:18:33.415 INFO:tasks.workunit.client.0.smithi107.stdout:9/661: chown d0/c3a 13393818 1 2024-04-25T00:18:33.415 INFO:tasks.workunit.client.0.smithi107.stdout:9/662: fsync d0/d8a/d93/d6f/f82 0 2024-04-25T00:18:33.416 INFO:tasks.workunit.client.0.smithi107.stdout:7/528: write d0/f3b [2227388,94433] 0 2024-04-25T00:18:33.416 INFO:tasks.workunit.client.0.smithi107.stdout:7/529: write d0/d8/db/d18/f61 [1083115,96281] 0 2024-04-25T00:18:33.416 INFO:tasks.workunit.client.0.smithi107.stdout:1/535: dwrite d2/d2a/f9e [0,4194304] 0 2024-04-25T00:18:33.416 INFO:tasks.workunit.client.0.smithi107.stdout:9/663: rmdir d0/d8a/d93/d34/d36/d77 39 2024-04-25T00:18:33.417 INFO:tasks.workunit.client.0.smithi107.stdout:2/648: read d6/d18/f78 [2591894,45050] 0 2024-04-25T00:18:33.418 INFO:tasks.workunit.client.0.smithi107.stdout:7/530: creat d0/d8/db/d18/d19/d47/d4d/d8a/fb8 x:0 0 0 2024-04-25T00:18:33.419 INFO:tasks.workunit.client.0.smithi107.stdout:2/649: symlink d6/dad/d67/lde 0 2024-04-25T00:18:33.419 INFO:tasks.workunit.client.0.smithi107.stdout:2/650: stat d6/d18/d40/d42/d2b/d30/d45/f5f 0 2024-04-25T00:18:33.420 INFO:tasks.workunit.client.0.smithi107.stdout:0/539: write dc/dd/f15 [741603,87060] 0 2024-04-25T00:18:33.420 INFO:tasks.workunit.client.0.smithi107.stdout:7/531: dread d0/d8/f4a [0,4194304] 0 2024-04-25T00:18:33.421 INFO:tasks.workunit.client.0.smithi107.stdout:1/536: symlink d2/la5 0 2024-04-25T00:18:33.421 INFO:tasks.workunit.client.0.smithi107.stdout:0/540: mkdir dc/dd/d16/d48/d83/d9f 0 2024-04-25T00:18:33.424 INFO:tasks.workunit.client.0.smithi107.stdout:7/532: creat d0/d80/da8/fb9 x:0 0 0 2024-04-25T00:18:33.426 INFO:tasks.workunit.client.0.smithi107.stdout:9/664: creat d0/d8a/d93/d34/d36/d77/dac/fdf x:0 0 0 2024-04-25T00:18:33.426 INFO:tasks.workunit.client.0.smithi107.stdout:7/533: creat d0/d8/db/d18/d19/fba x:0 0 0 2024-04-25T00:18:33.426 INFO:tasks.workunit.client.0.smithi107.stdout:7/534: write d0/f13 [514766,64547] 0 2024-04-25T00:18:33.429 INFO:tasks.workunit.client.0.smithi107.stdout:7/535: mknod d0/d8/db/d18/d19/d67/cbb 0 2024-04-25T00:18:33.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:33 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:18:33.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:33 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:18:33.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:33 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:18:33.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:33 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:18:33.437 INFO:tasks.workunit.client.0.smithi107.stdout:2/651: dwrite d6/d18/d40/d42/d2b/f7a [0,4194304] 0 2024-04-25T00:18:33.437 INFO:tasks.workunit.client.0.smithi107.stdout:2/652: fdatasync d6/d29/fa6 0 2024-04-25T00:18:33.437 INFO:tasks.workunit.client.0.smithi107.stdout:2/653: dread - d6/d18/d40/d42/d69/d6e/da2/fba zero size 2024-04-25T00:18:33.438 INFO:tasks.workunit.client.0.smithi107.stdout:0/541: creat dc/dd/d5d/d3f/d59/fa0 x:0 0 0 2024-04-25T00:18:33.441 INFO:tasks.workunit.client.0.smithi107.stdout:9/665: mkdir d0/d8a/d93/de/d5c/de0 0 2024-04-25T00:18:33.441 INFO:tasks.workunit.client.0.smithi107.stdout:5/660: dwrite d4/dc/d26/d29/d2e/f4c [0,4194304] 0 2024-04-25T00:18:33.441 INFO:tasks.workunit.client.0.smithi107.stdout:7/536: dread d0/d8/db/d18/d19/d39/f6f [0,4194304] 0 2024-04-25T00:18:33.442 INFO:tasks.workunit.client.0.smithi107.stdout:2/654: creat d6/d20/d22/d56/d94/fdf x:0 0 0 2024-04-25T00:18:33.442 INFO:tasks.workunit.client.0.smithi107.stdout:2/655: write d6/d18/d40/d42/d2b/d30/d45/fa3 [809788,95824] 0 2024-04-25T00:18:33.443 INFO:tasks.workunit.client.0.smithi107.stdout:2/656: write d6/d18/d40/d42/d69/d6e/d80/fcb [140627,37105] 0 2024-04-25T00:18:33.444 INFO:tasks.workunit.client.0.smithi107.stdout:0/542: creat dc/d24/d38/d64/d78/fa1 x:0 0 0 2024-04-25T00:18:33.447 INFO:tasks.workunit.client.0.smithi107.stdout:0/543: unlink dc/d24/d25/d31/l53 0 2024-04-25T00:18:33.458 INFO:tasks.workunit.client.0.smithi107.stdout:7/537: dwrite d0/d8/db/f2a [0,4194304] 0 2024-04-25T00:18:33.458 INFO:tasks.workunit.client.0.smithi107.stdout:7/538: dread - d0/d8/db/d18/d19/d67/d7a/d82/fb6 zero size 2024-04-25T00:18:33.466 INFO:tasks.workunit.client.0.smithi107.stdout:7/539: dread d0/d8/db/d18/d19/fb7 [0,4194304] 0 2024-04-25T00:18:33.466 INFO:tasks.workunit.client.0.smithi107.stdout:7/540: read - d0/d8/d2c/d58/fb4 zero size 2024-04-25T00:18:33.466 INFO:tasks.workunit.client.0.smithi107.stdout:7/541: fsync d0/d8/db/f49 0 2024-04-25T00:18:33.468 INFO:tasks.workunit.client.0.smithi107.stdout:2/657: dwrite d6/d20/f5a [0,4194304] 0 2024-04-25T00:18:33.468 INFO:tasks.workunit.client.0.smithi107.stdout:0/544: dread dc/dd/f15 [0,4194304] 0 2024-04-25T00:18:33.469 INFO:tasks.workunit.client.0.smithi107.stdout:7/542: rename d0/d8/db/d18/d5c to d0/d8/db/d18/d19/d47/d4d/dbc 0 2024-04-25T00:18:33.469 INFO:tasks.workunit.client.0.smithi107.stdout:0/545: mknod dc/d24/d38/d64/d78/ca2 0 2024-04-25T00:18:33.469 INFO:tasks.workunit.client.0.smithi107.stdout:2/658: symlink d6/d18/d40/d42/le0 0 2024-04-25T00:18:33.471 INFO:tasks.workunit.client.0.smithi107.stdout:2/659: link d6/d18/d40/d42/f57 d6/d20/d77/d9f/fe1 0 2024-04-25T00:18:33.471 INFO:tasks.workunit.client.0.smithi107.stdout:2/660: chown d6/d20/d22 57274922 1 2024-04-25T00:18:33.472 INFO:tasks.workunit.client.0.smithi107.stdout:0/546: creat dc/dd/d16/d33/d35/fa3 x:0 0 0 2024-04-25T00:18:33.473 INFO:tasks.workunit.client.0.smithi107.stdout:2/661: creat d6/d20/d77/d9f/fe2 x:0 0 0 2024-04-25T00:18:33.474 INFO:tasks.workunit.client.0.smithi107.stdout:0/547: symlink dc/d24/d38/d64/d78/la4 0 2024-04-25T00:18:33.474 INFO:tasks.workunit.client.0.smithi107.stdout:0/548: fdatasync dc/d24/d38/d64/d78/fa1 0 2024-04-25T00:18:33.475 INFO:tasks.workunit.client.0.smithi107.stdout:2/662: creat d6/d18/d40/d42/d69/d6e/fe3 x:0 0 0 2024-04-25T00:18:33.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:33 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:18:33.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:33 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:18:33.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:33 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:18:33.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:33 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:18:33.486 INFO:tasks.workunit.client.0.smithi107.stdout:7/543: dwrite d0/d8/db/d18/d19/f2f [4194304,4194304] 0 2024-04-25T00:18:33.492 INFO:tasks.workunit.client.0.smithi107.stdout:1/537: dwrite d2/d62/d66/d70/f77 [0,4194304] 0 2024-04-25T00:18:33.493 INFO:tasks.workunit.client.0.smithi107.stdout:0/549: dwrite dc/dd/f4c [0,4194304] 0 2024-04-25T00:18:33.502 INFO:tasks.workunit.client.0.smithi107.stdout:1/538: link d2/dd/d1a/l74 d2/dd/d25/la6 0 2024-04-25T00:18:33.503 INFO:tasks.workunit.client.0.smithi107.stdout:7/544: dwrite d0/d8/db/f98 [0,4194304] 0 2024-04-25T00:18:33.503 INFO:tasks.workunit.client.0.smithi107.stdout:7/545: fsync d0/f11 0 2024-04-25T00:18:33.504 INFO:tasks.workunit.client.0.smithi107.stdout:1/539: link d2/d62/d66/d70/f8c d2/dd/d82/fa7 0 2024-04-25T00:18:33.505 INFO:tasks.workunit.client.0.smithi107.stdout:1/540: truncate d2/dd/d82/fa7 916901 0 2024-04-25T00:18:33.506 INFO:tasks.workunit.client.0.smithi107.stdout:7/546: link d0/d8/db/d18/d19/d47/d4d/d8a/fa6 d0/d8/d4e/d46/d93/fbd 0 2024-04-25T00:18:33.507 INFO:tasks.workunit.client.0.smithi107.stdout:1/541: unlink d2/d62/d66/c91 0 2024-04-25T00:18:33.507 INFO:tasks.workunit.client.0.smithi107.stdout:1/542: read - d2/da/d2f/f97 zero size 2024-04-25T00:18:33.508 INFO:tasks.workunit.client.0.smithi107.stdout:7/547: mkdir d0/d8/d4e/dbe 0 2024-04-25T00:18:33.509 INFO:tasks.workunit.client.0.smithi107.stdout:1/543: rmdir d2/dd/d1b 39 2024-04-25T00:18:33.509 INFO:tasks.workunit.client.0.smithi107.stdout:1/544: chown d2/dd/d1b/f52 494 1 2024-04-25T00:18:33.509 INFO:tasks.workunit.client.0.smithi107.stdout:1/545: fsync d2/dd/d1a/f27 0 2024-04-25T00:18:33.509 INFO:tasks.workunit.client.0.smithi107.stdout:1/546: readlink d2/dd/l6c 0 2024-04-25T00:18:33.509 INFO:tasks.workunit.client.0.smithi107.stdout:1/547: chown d2/dd/d25/d5e/l28 52746712 1 2024-04-25T00:18:33.510 INFO:tasks.workunit.client.0.smithi107.stdout:1/548: write d2/f7e [5941727,100138] 0 2024-04-25T00:18:33.510 INFO:tasks.workunit.client.0.smithi107.stdout:1/549: dread - d2/dd/d82/f99 zero size 2024-04-25T00:18:33.510 INFO:tasks.workunit.client.0.smithi107.stdout:1/550: fsync d2/f51 0 2024-04-25T00:18:33.512 INFO:tasks.workunit.client.0.smithi107.stdout:7/548: rmdir d0/d8/d4e/dbe 0 2024-04-25T00:18:33.515 INFO:tasks.workunit.client.0.smithi107.stdout:8/611: sync 2024-04-25T00:18:33.515 INFO:tasks.workunit.client.0.smithi107.stdout:8/612: chown d2/d18/d1a/d5e/d3d/c42 0 1 2024-04-25T00:18:33.515 INFO:tasks.workunit.client.0.smithi107.stdout:8/613: write d2/f55 [479448,47239] 0 2024-04-25T00:18:33.515 INFO:tasks.workunit.client.0.smithi107.stdout:0/550: dwrite dc/f18 [0,4194304] 0 2024-04-25T00:18:33.516 INFO:tasks.workunit.client.0.smithi107.stdout:8/614: truncate d2/d18/d1a/f3a 6565428 0 2024-04-25T00:18:33.516 INFO:tasks.workunit.client.0.smithi107.stdout:8/615: fdatasync d2/d18/d1a/f8e 0 2024-04-25T00:18:33.516 INFO:tasks.workunit.client.0.smithi107.stdout:8/616: read - d2/d18/d1a/d82/fb6 zero size 2024-04-25T00:18:33.517 INFO:tasks.workunit.client.0.smithi107.stdout:0/551: mkdir dc/dd/d5d/d3f/da5 0 2024-04-25T00:18:33.517 INFO:tasks.workunit.client.0.smithi107.stdout:0/552: stat dc/dd/d5d/f70 0 2024-04-25T00:18:33.517 INFO:tasks.workunit.client.0.smithi107.stdout:1/551: getdents d2/dd/d25 0 2024-04-25T00:18:33.518 INFO:tasks.workunit.client.0.smithi107.stdout:0/553: mkdir dc/da6 0 2024-04-25T00:18:33.519 INFO:tasks.workunit.client.0.smithi107.stdout:1/552: rmdir d2/d62/d66 39 2024-04-25T00:18:33.521 INFO:tasks.workunit.client.0.smithi107.stdout:0/554: mknod dc/d24/d38/d64/d78/ca7 0 2024-04-25T00:18:33.521 INFO:tasks.workunit.client.0.smithi107.stdout:1/553: rmdir d2/da 39 2024-04-25T00:18:33.524 INFO:tasks.workunit.client.0.smithi107.stdout:1/554: symlink d2/d2a/d5c/d89/la8 0 2024-04-25T00:18:33.524 INFO:tasks.workunit.client.0.smithi107.stdout:1/555: write d2/dd/d25/d5e/f9d [948714,118599] 0 2024-04-25T00:18:33.529 INFO:tasks.workunit.client.0.smithi107.stdout:1/556: read d2/dd/f8b [437387,94531] 0 2024-04-25T00:18:33.529 INFO:tasks.workunit.client.0.smithi107.stdout:1/557: stat d2/d62/d66/d70/l9a 0 2024-04-25T00:18:33.530 INFO:tasks.workunit.client.0.smithi107.stdout:1/558: chown d2/d62/d66/f78 1421815644 1 2024-04-25T00:18:33.531 INFO:tasks.workunit.client.0.smithi107.stdout:1/559: mkdir d2/dd/d25/da9 0 2024-04-25T00:18:33.531 INFO:tasks.workunit.client.0.smithi107.stdout:7/549: dwrite d0/d8/d4e/d46/d93/fbd [0,4194304] 0 2024-04-25T00:18:33.534 INFO:tasks.workunit.client.0.smithi107.stdout:7/550: mkdir d0/d8/d4e/d46/d93/dbf 0 2024-04-25T00:18:33.535 INFO:tasks.workunit.client.0.smithi107.stdout:7/551: write d0/d8/db/d18/d19/d47/d4d/dbc/f75 [171094,29732] 0 2024-04-25T00:18:33.535 INFO:tasks.workunit.client.0.smithi107.stdout:7/552: fdatasync d0/d8/d2c/d58/f9a 0 2024-04-25T00:18:33.536 INFO:tasks.workunit.client.0.smithi107.stdout:8/617: dwrite d2/d18/f68 [0,4194304] 0 2024-04-25T00:18:33.539 INFO:tasks.workunit.client.0.smithi107.stdout:8/618: symlink d2/d18/d1a/d5e/d4f/da0/da1/db9/lcd 0 2024-04-25T00:18:33.539 INFO:tasks.workunit.client.0.smithi107.stdout:8/619: dread - d2/d18/d1a/d5e/d5c/f88 zero size 2024-04-25T00:18:33.540 INFO:tasks.workunit.client.0.smithi107.stdout:8/620: symlink d2/d18/d1a/d5e/d5c/lce 0 2024-04-25T00:18:33.547 INFO:tasks.workunit.client.0.smithi107.stdout:0/555: rename dc/dd/d5d/c23 to dc/dd/d16/ca8 0 2024-04-25T00:18:33.550 INFO:tasks.workunit.client.0.smithi107.stdout:1/560: rename d2/d62/d66/d70/l9a to d2/da4/laa 0 2024-04-25T00:18:33.550 INFO:tasks.workunit.client.0.smithi107.stdout:1/561: write d2/dd/d25/f50 [806868,101447] 0 2024-04-25T00:18:33.552 INFO:tasks.workunit.client.0.smithi107.stdout:7/553: dwrite d0/d8/db/d18/d19/d39/f96 [0,4194304] 0 2024-04-25T00:18:33.552 INFO:tasks.workunit.client.0.smithi107.stdout:7/554: read d0/d8/d4e/f87 [648287,116815] 0 2024-04-25T00:18:33.553 INFO:tasks.workunit.client.0.smithi107.stdout:7/555: fdatasync d0/d80/f88 0 2024-04-25T00:18:33.553 INFO:tasks.workunit.client.0.smithi107.stdout:7/556: chown d0/d8/db/d18/d19/d47/f5b 5379 1 2024-04-25T00:18:33.556 INFO:tasks.workunit.client.0.smithi107.stdout:7/557: write d0/d8/db/f15 [4693149,9466] 0 2024-04-25T00:18:33.557 INFO:tasks.workunit.client.0.smithi107.stdout:7/558: truncate d0/d8/db/d18/d19/f3c 933178 0 2024-04-25T00:18:33.558 INFO:tasks.workunit.client.0.smithi107.stdout:7/559: mknod d0/d8/db/cc0 0 2024-04-25T00:18:33.559 INFO:tasks.workunit.client.0.smithi107.stdout:7/560: rmdir d0/d8/db/d18/d19/d47 39 2024-04-25T00:18:33.560 INFO:tasks.workunit.client.0.smithi107.stdout:1/562: dread d2/dd/d1b/f52 [0,4194304] 0 2024-04-25T00:18:33.560 INFO:tasks.workunit.client.0.smithi107.stdout:1/563: dread - d2/f68 zero size 2024-04-25T00:18:33.560 INFO:tasks.workunit.client.0.smithi107.stdout:1/564: readlink d2/d2a/d5c/l5d 0 2024-04-25T00:18:33.561 INFO:tasks.workunit.client.0.smithi107.stdout:7/561: mkdir d0/dc1 0 2024-04-25T00:18:33.562 INFO:tasks.workunit.client.0.smithi107.stdout:1/565: write d2/dd/d25/d5e/f9d [580540,33037] 0 2024-04-25T00:18:33.567 INFO:tasks.workunit.client.0.smithi107.stdout:1/566: truncate d2/dd/d25/d5e/f9d 627449 0 2024-04-25T00:18:33.570 INFO:tasks.workunit.client.0.smithi107.stdout:7/562: dread d0/d8/db/d18/d19/d47/d4d/d8a/fa6 [0,4194304] 0 2024-04-25T00:18:33.570 INFO:tasks.workunit.client.0.smithi107.stdout:7/563: write d0/d8/f4a [592019,37120] 0 2024-04-25T00:18:33.574 INFO:tasks.workunit.client.0.smithi107.stdout:0/556: rename dc/dd/d16/d5e/f8f to dc/dd/d5d/d3f/da5/fa9 0 2024-04-25T00:18:33.574 INFO:tasks.workunit.client.0.smithi107.stdout:7/564: rename d0/d8/d4e/d46/d93/dbf to d0/d8/db/d18/d19/d67/d7a/dc2 0 2024-04-25T00:18:33.576 INFO:tasks.workunit.client.0.smithi107.stdout:0/557: link dc/dd/d5d/f87 dc/d24/d38/d64/faa 0 2024-04-25T00:18:33.591 INFO:tasks.workunit.client.0.smithi107.stdout:1/567: dwrite d2/f8a [0,4194304] 0 2024-04-25T00:18:33.593 INFO:tasks.workunit.client.0.smithi107.stdout:7/565: dwrite d0/d8/f65 [0,4194304] 0 2024-04-25T00:18:33.599 INFO:tasks.workunit.client.0.smithi107.stdout:7/566: dread d0/f3b [0,4194304] 0 2024-04-25T00:18:33.603 INFO:tasks.workunit.client.0.smithi107.stdout:7/567: rename d0/d8/d2c/d58/c68 to d0/cc3 0 2024-04-25T00:18:33.604 INFO:tasks.workunit.client.0.smithi107.stdout:7/568: symlink d0/d8/d2c/d58/lc4 0 2024-04-25T00:18:33.607 INFO:tasks.workunit.client.0.smithi107.stdout:1/568: dwrite d2/da/f53 [0,4194304] 0 2024-04-25T00:18:33.610 INFO:tasks.workunit.client.0.smithi107.stdout:1/569: dread d2/d2a/d37/f56 [0,4194304] 0 2024-04-25T00:18:33.610 INFO:tasks.workunit.client.0.smithi107.stdout:1/570: readlink d2/dd/la2 0 2024-04-25T00:18:33.631 INFO:tasks.workunit.client.0.smithi107.stdout:7/569: dwrite d0/d8/db/f49 [0,4194304] 0 2024-04-25T00:18:33.631 INFO:tasks.workunit.client.0.smithi107.stdout:1/571: dread d2/d62/d66/f6e [0,4194304] 0 2024-04-25T00:18:33.633 INFO:tasks.workunit.client.0.smithi107.stdout:7/570: mknod d0/d8/db/d92/cc5 0 2024-04-25T00:18:33.633 INFO:tasks.workunit.client.0.smithi107.stdout:7/571: stat d0/d80/f88 0 2024-04-25T00:18:33.633 INFO:tasks.workunit.client.0.smithi107.stdout:1/572: mknod d2/dd/d1b/cab 0 2024-04-25T00:18:33.633 INFO:tasks.workunit.client.0.smithi107.stdout:1/573: write d2/da/f8e [1840712,10734] 0 2024-04-25T00:18:33.641 INFO:tasks.workunit.client.0.smithi107.stdout:7/572: write d0/d8/db/d18/d19/fb7 [2095589,35398] 0 2024-04-25T00:18:33.641 INFO:tasks.workunit.client.0.smithi107.stdout:7/573: read - d0/d8/d2c/d58/f79 zero size 2024-04-25T00:18:33.644 INFO:tasks.workunit.client.0.smithi107.stdout:7/574: creat d0/d8/db/d18/d19/d47/d4d/dbc/fc6 x:0 0 0 2024-04-25T00:18:33.644 INFO:tasks.workunit.client.0.smithi107.stdout:7/575: readlink d0/d8/db/d18/d19/d39/l4b 0 2024-04-25T00:18:33.644 INFO:tasks.workunit.client.0.smithi107.stdout:1/574: dread d2/da/d2f/f3e [0,4194304] 0 2024-04-25T00:18:33.645 INFO:tasks.workunit.client.0.smithi107.stdout:1/575: chown d2/da/f15 260696 1 2024-04-25T00:18:33.645 INFO:tasks.workunit.client.0.smithi107.stdout:1/576: write d2/dd/d1a/f1d [9134313,118587] 0 2024-04-25T00:18:33.646 INFO:tasks.workunit.client.0.smithi107.stdout:1/577: symlink d2/d62/d66/lac 0 2024-04-25T00:18:33.648 INFO:tasks.workunit.client.0.smithi107.stdout:1/578: creat d2/da/d2f/fad x:0 0 0 2024-04-25T00:18:33.649 INFO:tasks.workunit.client.0.smithi107.stdout:1/579: mknod d2/da/d2f/cae 0 2024-04-25T00:18:33.649 INFO:tasks.workunit.client.0.smithi107.stdout:1/580: getdents d2/d2a/d37/d88 0 2024-04-25T00:18:33.650 INFO:tasks.workunit.client.0.smithi107.stdout:1/581: symlink d2/d62/d66/laf 0 2024-04-25T00:18:33.651 INFO:tasks.workunit.client.0.smithi107.stdout:1/582: symlink d2/d2a/d37/d88/lb0 0 2024-04-25T00:18:33.668 INFO:tasks.workunit.client.0.smithi107.stdout:1/583: dwrite d2/da/d2f/f9b [0,4194304] 0 2024-04-25T00:18:33.668 INFO:tasks.workunit.client.0.smithi107.stdout:1/584: dread - d2/f69 zero size 2024-04-25T00:18:33.676 INFO:tasks.workunit.client.0.smithi107.stdout:1/585: read d2/da/f15 [2323799,61749] 0 2024-04-25T00:18:33.677 INFO:tasks.workunit.client.0.smithi107.stdout:1/586: symlink d2/d2a/lb1 0 2024-04-25T00:18:33.679 INFO:tasks.workunit.client.0.smithi107.stdout:1/587: dread d2/f51 [0,4194304] 0 2024-04-25T00:18:33.680 INFO:tasks.workunit.client.0.smithi107.stdout:1/588: creat d2/d2a/d37/d88/fb2 x:0 0 0 2024-04-25T00:18:33.681 INFO:tasks.workunit.client.0.smithi107.stdout:1/589: mknod d2/dd/d1b/cb3 0 2024-04-25T00:18:33.685 INFO:tasks.workunit.client.0.smithi107.stdout:6/474: sync 2024-04-25T00:18:33.685 INFO:tasks.workunit.client.0.smithi107.stdout:6/475: write d1/d2/db/d32/d33/f6d [1402308,50807] 0 2024-04-25T00:18:33.686 INFO:tasks.workunit.client.0.smithi107.stdout:6/476: rmdir d1/d14/d18 39 2024-04-25T00:18:33.688 INFO:tasks.workunit.client.0.smithi107.stdout:6/477: rename d1/d2/db/d3c/f72 to d1/d2/d3/d26/fa2 0 2024-04-25T00:18:33.688 INFO:tasks.workunit.client.0.smithi107.stdout:6/478: chown d1/d2/db/d32/d33/d4f 66294943 1 2024-04-25T00:18:33.705 INFO:tasks.workunit.client.0.smithi107.stdout:1/590: dwrite d2/d2a/d37/f56 [0,4194304] 0 2024-04-25T00:18:33.705 INFO:tasks.workunit.client.0.smithi107.stdout:6/479: dwrite d1/d2/d3/d71/d94/f9b [0,4194304] 0 2024-04-25T00:18:33.705 INFO:tasks.workunit.client.0.smithi107.stdout:6/480: chown d1/d2/db/d32/d33/d4f 239239 1 2024-04-25T00:18:33.706 INFO:tasks.workunit.client.0.smithi107.stdout:6/481: chown d1/d2/db/d32/d36/f46 0 1 2024-04-25T00:18:33.706 INFO:tasks.workunit.client.0.smithi107.stdout:6/482: read d1/d2/db/f34 [3318170,103351] 0 2024-04-25T00:18:33.708 INFO:tasks.workunit.client.0.smithi107.stdout:6/483: symlink d1/la3 0 2024-04-25T00:18:33.708 INFO:tasks.workunit.client.0.smithi107.stdout:6/484: readlink d1/d2/d3/d28/d79/l98 0 2024-04-25T00:18:33.711 INFO:tasks.workunit.client.0.smithi107.stdout:1/591: dread d2/dd/d25/d5e/f94 [0,4194304] 0 2024-04-25T00:18:33.713 INFO:tasks.workunit.client.0.smithi107.stdout:1/592: rename d2/da/c24 to d2/d2a/d5c/cb4 0 2024-04-25T00:18:33.729 INFO:tasks.workunit.client.0.smithi107.stdout:6/485: dwrite d1/d2/db/d32/d36/d3d/f5e [0,4194304] 0 2024-04-25T00:18:33.732 INFO:tasks.workunit.client.0.smithi107.stdout:6/486: creat d1/d2/d2b/d3e/fa4 x:0 0 0 2024-04-25T00:18:33.733 INFO:tasks.workunit.client.0.smithi107.stdout:6/487: mknod d1/ca5 0 2024-04-25T00:18:33.734 INFO:tasks.workunit.client.0.smithi107.stdout:6/488: write d1/d2/db/d32/d33/f6d [1223681,31193] 0 2024-04-25T00:18:33.735 INFO:tasks.workunit.client.0.smithi107.stdout:6/489: creat d1/d4c/fa6 x:0 0 0 2024-04-25T00:18:33.737 INFO:tasks.workunit.client.0.smithi107.stdout:6/490: creat d1/d2/db/d32/d36/d3a/fa7 x:0 0 0 2024-04-25T00:18:33.737 INFO:tasks.workunit.client.0.smithi107.stdout:6/491: chown d1/d2/d2b/d3e/f8f 6815 1 2024-04-25T00:18:33.738 INFO:tasks.workunit.client.0.smithi107.stdout:6/492: chown d1/d2/db/d32/d33/d68/d75/f78 88041 1 2024-04-25T00:18:33.738 INFO:tasks.workunit.client.0.smithi107.stdout:1/593: dwrite d2/dd/d25/d5e/f94 [0,4194304] 0 2024-04-25T00:18:33.738 INFO:tasks.workunit.client.0.smithi107.stdout:1/594: chown d2/d2a/d5c/d89/la8 8 1 2024-04-25T00:18:33.748 INFO:tasks.workunit.client.0.smithi107.stdout:6/493: dread d1/d2/d3/d71/d94/f9b [0,4194304] 0 2024-04-25T00:18:33.749 INFO:tasks.workunit.client.0.smithi107.stdout:6/494: mknod d1/d2/db/d32/d36/d3d/ca8 0 2024-04-25T00:18:33.751 INFO:tasks.workunit.client.0.smithi107.stdout:6/495: unlink d1/d2/db/d32/f8e 0 2024-04-25T00:18:33.752 INFO:tasks.workunit.client.0.smithi107.stdout:6/496: mknod d1/d2/d3/d28/d79/d6b/ca9 0 2024-04-25T00:18:33.754 INFO:tasks.workunit.client.0.smithi107.stdout:6/497: rmdir d1/d2/d3/d71/d81 0 2024-04-25T00:18:33.758 INFO:tasks.workunit.client.0.smithi107.stdout:1/595: dwrite d2/d62/d66/d70/f8c [0,4194304] 0 2024-04-25T00:18:33.760 INFO:tasks.workunit.client.0.smithi107.stdout:1/596: creat d2/dd/d25/da9/fb5 x:0 0 0 2024-04-25T00:18:33.765 INFO:tasks.workunit.client.0.smithi107.stdout:6/498: write d1/d2/d3/d26/f51 [2668225,75463] 0 2024-04-25T00:18:33.768 INFO:tasks.workunit.client.0.smithi107.stdout:3/559: sync 2024-04-25T00:18:33.769 INFO:tasks.workunit.client.0.smithi107.stdout:1/597: write d2/dd/d25/f7d [841690,121690] 0 2024-04-25T00:18:33.769 INFO:tasks.workunit.client.0.smithi107.stdout:1/598: write d2/dd/d25/f33 [3510099,127191] 0 2024-04-25T00:18:33.769 INFO:tasks.workunit.client.0.smithi107.stdout:3/560: creat d9/d22/d1b/fa9 x:0 0 0 2024-04-25T00:18:33.770 INFO:tasks.workunit.client.0.smithi107.stdout:3/561: truncate d9/d88/d4e/f92 501456 0 2024-04-25T00:18:33.770 INFO:tasks.workunit.client.0.smithi107.stdout:1/599: creat d2/da4/fb6 x:0 0 0 2024-04-25T00:18:33.771 INFO:tasks.workunit.client.0.smithi107.stdout:1/600: mkdir d2/da/db7 0 2024-04-25T00:18:33.772 INFO:tasks.workunit.client.0.smithi107.stdout:3/562: mkdir d9/d88/daa 0 2024-04-25T00:18:33.772 INFO:tasks.workunit.client.0.smithi107.stdout:1/601: mknod d2/dd/d25/d5e/cb8 0 2024-04-25T00:18:33.773 INFO:tasks.workunit.client.0.smithi107.stdout:3/563: mkdir d9/d54/d94/dab 0 2024-04-25T00:18:33.774 INFO:tasks.workunit.client.0.smithi107.stdout:3/564: getdents d9/d54/d5c 0 2024-04-25T00:18:33.774 INFO:tasks.workunit.client.0.smithi107.stdout:1/602: write d2/d2a/f38 [4269203,115256] 0 2024-04-25T00:18:33.775 INFO:tasks.workunit.client.0.smithi107.stdout:3/565: mkdir d9/d15/d39/d76/dac 0 2024-04-25T00:18:33.775 INFO:tasks.workunit.client.0.smithi107.stdout:6/499: write d1/d2/d2b/d3e/f8f [231209,33312] 0 2024-04-25T00:18:33.776 INFO:tasks.workunit.client.0.smithi107.stdout:1/603: getdents d2/dd/d1b 0 2024-04-25T00:18:33.776 INFO:tasks.workunit.client.0.smithi107.stdout:1/604: write d2/da/f8e [2479990,32633] 0 2024-04-25T00:18:33.776 INFO:tasks.workunit.client.0.smithi107.stdout:3/566: mkdir d9/d88/d3a/d91/dad 0 2024-04-25T00:18:33.777 INFO:tasks.workunit.client.0.smithi107.stdout:1/605: link d2/d62/d66/laf d2/d62/d66/d70/lb9 0 2024-04-25T00:18:33.777 INFO:tasks.workunit.client.0.smithi107.stdout:3/567: creat d9/d88/d80/fae x:0 0 0 2024-04-25T00:18:33.778 INFO:tasks.workunit.client.0.smithi107.stdout:1/606: creat d2/dd/d1b/fba x:0 0 0 2024-04-25T00:18:33.788 INFO:tasks.workunit.client.0.smithi107.stdout:1/607: write d2/dd/d82/fa7 [2075738,112632] 0 2024-04-25T00:18:33.793 INFO:tasks.workunit.client.0.smithi107.stdout:6/500: dwrite d1/d14/d18/f2c [0,4194304] 0 2024-04-25T00:18:33.795 INFO:tasks.workunit.client.0.smithi107.stdout:6/501: unlink d1/d2/db/d32/d33/d68/d75/c7c 0 2024-04-25T00:18:33.796 INFO:tasks.workunit.client.0.smithi107.stdout:6/502: unlink d1/d2/d2b/d3e/d5b/c6c 0 2024-04-25T00:18:33.796 INFO:tasks.workunit.client.0.smithi107.stdout:6/503: stat d1/d2/d3/d28/d79 0 2024-04-25T00:18:33.796 INFO:tasks.workunit.client.0.smithi107.stdout:3/568: rename d9/d15/da1/d4b/c71 to d9/d88/d3a/d91/caf 0 2024-04-25T00:18:33.799 INFO:tasks.workunit.client.0.smithi107.stdout:1/608: read d2/d62/d66/f78 [2491256,50433] 0 2024-04-25T00:18:33.801 INFO:tasks.workunit.client.0.smithi107.stdout:3/569: mkdir d9/d15/da1/db0 0 2024-04-25T00:18:33.803 INFO:tasks.workunit.client.0.smithi107.stdout:1/609: unlink d2/dd/d1a/l74 0 2024-04-25T00:18:33.804 INFO:tasks.workunit.client.0.smithi107.stdout:3/570: truncate d9/d54/d5c/d68/f3c 1892546 0 2024-04-25T00:18:33.814 INFO:tasks.workunit.client.0.smithi107.stdout:6/504: dwrite d1/d2/d3/f31 [0,4194304] 0 2024-04-25T00:18:33.814 INFO:tasks.workunit.client.0.smithi107.stdout:6/505: dread - d1/d2/d2b/f86 zero size 2024-04-25T00:18:33.818 INFO:tasks.workunit.client.0.smithi107.stdout:3/571: write d9/d88/d4e/d93/f9d [4174200,64318] 0 2024-04-25T00:18:33.818 INFO:tasks.workunit.client.0.smithi107.stdout:3/572: chown d9/d15/d39 131 1 2024-04-25T00:18:33.819 INFO:tasks.workunit.client.0.smithi107.stdout:1/610: write d2/dd/f4e [1141694,43694] 0 2024-04-25T00:18:33.819 INFO:tasks.workunit.client.0.smithi107.stdout:3/573: dread d9/d88/d4e/f70 [0,4194304] 0 2024-04-25T00:18:33.820 INFO:tasks.workunit.client.0.smithi107.stdout:6/506: rename d1/d2/c65 to d1/d2/db/d32/d36/d3d/caa 0 2024-04-25T00:18:33.820 INFO:tasks.workunit.client.0.smithi107.stdout:3/574: creat d9/d88/d3a/fb1 x:0 0 0 2024-04-25T00:18:33.822 INFO:tasks.workunit.client.0.smithi107.stdout:3/575: symlink d9/d54/d5c/lb2 0 2024-04-25T00:18:33.824 INFO:tasks.workunit.client.0.smithi107.stdout:3/576: mkdir d9/d54/d5c/d68/db3 0 2024-04-25T00:18:33.825 INFO:tasks.workunit.client.0.smithi107.stdout:3/577: dread d9/d22/f7a [0,4194304] 0 2024-04-25T00:18:33.826 INFO:tasks.workunit.client.0.smithi107.stdout:4/706: sync 2024-04-25T00:18:33.826 INFO:tasks.workunit.client.0.smithi107.stdout:5/661: sync 2024-04-25T00:18:33.827 INFO:tasks.workunit.client.0.smithi107.stdout:5/662: dread - d4/d1c/fa8 zero size 2024-04-25T00:18:33.827 INFO:tasks.workunit.client.0.smithi107.stdout:3/578: creat d9/d54/fb4 x:0 0 0 2024-04-25T00:18:33.827 INFO:tasks.workunit.client.0.smithi107.stdout:5/663: symlink d4/d1c/d5c/lda 0 2024-04-25T00:18:33.827 INFO:tasks.workunit.client.0.smithi107.stdout:5/664: write d4/d7c/d88/fc3 [927428,7693] 0 2024-04-25T00:18:33.829 INFO:tasks.workunit.client.0.smithi107.stdout:5/665: creat d4/dc/d26/d29/d51/d78/fdb x:0 0 0 2024-04-25T00:18:33.829 INFO:tasks.workunit.client.0.smithi107.stdout:5/666: write d4/dc/d26/d29/fc1 [93310,35682] 0 2024-04-25T00:18:33.829 INFO:tasks.workunit.client.0.smithi107.stdout:3/579: creat d9/d88/d3a/d91/dad/fb5 x:0 0 0 2024-04-25T00:18:33.831 INFO:tasks.workunit.client.0.smithi107.stdout:1/611: dread d2/dd/d25/f2e [0,4194304] 0 2024-04-25T00:18:33.831 INFO:tasks.workunit.client.0.smithi107.stdout:3/580: link d9/d15/da1/d4b/ca5 d9/d54/d5c/cb6 0 2024-04-25T00:18:33.831 INFO:tasks.workunit.client.0.smithi107.stdout:3/581: fdatasync d9/f99 0 2024-04-25T00:18:33.833 INFO:tasks.workunit.client.0.smithi107.stdout:5/667: rename d4/d91/cb5 to d4/dc/cdc 0 2024-04-25T00:18:33.834 INFO:tasks.workunit.client.0.smithi107.stdout:1/612: creat d2/d2a/d37/d88/fbb x:0 0 0 2024-04-25T00:18:33.836 INFO:tasks.workunit.client.0.smithi107.stdout:1/613: creat d2/da/d2f/fbc x:0 0 0 2024-04-25T00:18:33.838 INFO:tasks.workunit.client.0.smithi107.stdout:5/668: symlink d4/dc/d26/d29/d51/ldd 0 2024-04-25T00:18:33.838 INFO:tasks.workunit.client.0.smithi107.stdout:5/669: read - d4/d1c/f9e zero size 2024-04-25T00:18:33.839 INFO:tasks.workunit.client.0.smithi107.stdout:1/614: symlink d2/d2a/lbd 0 2024-04-25T00:18:33.839 INFO:tasks.workunit.client.0.smithi107.stdout:4/707: write d3/d24/f2c [1804801,103190] 0 2024-04-25T00:18:33.841 INFO:tasks.workunit.client.0.smithi107.stdout:4/708: rename d3/da8/fbd to d3/d10/d7b/feb 0 2024-04-25T00:18:33.843 INFO:tasks.workunit.client.0.smithi107.stdout:7/576: sync 2024-04-25T00:18:33.843 INFO:tasks.workunit.client.0.smithi107.stdout:9/666: sync 2024-04-25T00:18:33.843 INFO:tasks.workunit.client.0.smithi107.stdout:2/663: sync 2024-04-25T00:18:33.844 INFO:tasks.workunit.client.0.smithi107.stdout:8/621: sync 2024-04-25T00:18:33.844 INFO:tasks.workunit.client.0.smithi107.stdout:0/558: sync 2024-04-25T00:18:33.844 INFO:tasks.workunit.client.0.smithi107.stdout:1/615: rename d2/dd/d82/fa0 to d2/d62/d66/fbe 0 2024-04-25T00:18:33.844 INFO:tasks.workunit.client.0.smithi107.stdout:1/616: read d2/da/d2f/f97 [830760,26135] 0 2024-04-25T00:18:33.844 INFO:tasks.workunit.client.0.smithi107.stdout:1/617: dread - d2/f93 zero size 2024-04-25T00:18:33.845 INFO:tasks.workunit.client.0.smithi107.stdout:1/618: write d2/d62/d66/d70/f7a [1485398,103776] 0 2024-04-25T00:18:33.846 INFO:tasks.workunit.client.0.smithi107.stdout:9/667: truncate d0/d8a/d93/de/d5c/f72 45349 0 2024-04-25T00:18:33.847 INFO:tasks.workunit.client.0.smithi107.stdout:8/622: mkdir d2/d18/d1a/d5e/d3e/d74/dcf 0 2024-04-25T00:18:33.847 INFO:tasks.workunit.client.0.smithi107.stdout:8/623: dread - d2/d18/d1a/d5e/d5c/d91/da3/fb8 zero size 2024-04-25T00:18:33.847 INFO:tasks.workunit.client.0.smithi107.stdout:6/507: dwrite d1/d2/db/f4d [0,4194304] 0 2024-04-25T00:18:33.848 INFO:tasks.workunit.client.0.smithi107.stdout:3/582: dread d9/f29 [0,4194304] 0 2024-04-25T00:18:33.852 INFO:tasks.workunit.client.0.smithi107.stdout:3/583: rmdir d9/d15/d39/d76/dac 0 2024-04-25T00:18:33.854 INFO:tasks.workunit.client.0.smithi107.stdout:3/584: creat d9/d15/d39/fb7 x:0 0 0 2024-04-25T00:18:33.856 INFO:tasks.workunit.client.0.smithi107.stdout:8/624: write d2/d18/d1a/d5e/d3e/d74/f8a [4564032,61169] 0 2024-04-25T00:18:33.856 INFO:tasks.workunit.client.0.smithi107.stdout:0/559: dread dc/d24/d25/f80 [0,4194304] 0 2024-04-25T00:18:33.856 INFO:tasks.workunit.client.0.smithi107.stdout:8/625: fsync d2/d18/f43 0 2024-04-25T00:18:33.856 INFO:tasks.workunit.client.0.smithi107.stdout:0/560: chown dc/dd/d5d/d3f 0 1 2024-04-25T00:18:33.857 INFO:tasks.workunit.client.0.smithi107.stdout:3/585: creat d9/d54/d5c/d68/fb8 x:0 0 0 2024-04-25T00:18:33.857 INFO:tasks.workunit.client.0.smithi107.stdout:5/670: dread d4/dc/d26/d29/d2e/f6d [0,4194304] 0 2024-04-25T00:18:33.865 INFO:tasks.workunit.client.0.smithi107.stdout:0/561: rmdir dc/dd/d5d/d5c/d6e 39 2024-04-25T00:18:33.869 INFO:tasks.workunit.client.0.smithi107.stdout:3/586: dread d9/d88/f32 [0,4194304] 0 2024-04-25T00:18:33.870 INFO:tasks.workunit.client.0.smithi107.stdout:3/587: mkdir d9/d54/d5c/d68/db3/db9 0 2024-04-25T00:18:33.870 INFO:tasks.workunit.client.0.smithi107.stdout:3/588: chown d9/d15/da1/d4b/c7d 44 1 2024-04-25T00:18:33.872 INFO:tasks.workunit.client.0.smithi107.stdout:3/589: truncate d9/d88/d3a/d91/f9a 622464 0 2024-04-25T00:18:33.876 INFO:tasks.workunit.client.0.smithi107.stdout:9/668: dwrite d0/d8a/d93/d1f/f86 [0,4194304] 0 2024-04-25T00:18:33.887 INFO:tasks.workunit.client.0.smithi107.stdout:6/508: dwrite d1/d2/db/d32/d33/d68/d75/f8b [0,4194304] 0 2024-04-25T00:18:33.896 INFO:tasks.workunit.client.0.smithi107.stdout:1/619: dread d2/dd/d25/f33 [0,4194304] 0 2024-04-25T00:18:33.896 INFO:tasks.workunit.client.0.smithi107.stdout:2/664: dwrite d6/d29/d4e/fab [0,4194304] 0 2024-04-25T00:18:33.896 INFO:tasks.workunit.client.0.smithi107.stdout:1/620: truncate d2/d2a/f38 4746197 0 2024-04-25T00:18:33.898 INFO:tasks.workunit.client.0.smithi107.stdout:5/671: dwrite d4/dc/d26/f64 [0,4194304] 0 2024-04-25T00:18:33.898 INFO:tasks.workunit.client.0.smithi107.stdout:5/672: chown d4/dc/f20 47300 1 2024-04-25T00:18:33.898 INFO:tasks.workunit.client.0.smithi107.stdout:8/626: dwrite d2/d18/d1a/d5e/d3d/fc4 [0,4194304] 0 2024-04-25T00:18:33.901 INFO:tasks.workunit.client.0.smithi107.stdout:2/665: unlink d6/d29/c92 0 2024-04-25T00:18:33.901 INFO:tasks.workunit.client.0.smithi107.stdout:0/562: dwrite dc/dd/d5d/f87 [0,4194304] 0 2024-04-25T00:18:33.901 INFO:tasks.workunit.client.0.smithi107.stdout:2/666: write d6/d18/d40/d42/d69/d82/fb8 [712870,42838] 0 2024-04-25T00:18:33.903 INFO:tasks.workunit.client.0.smithi107.stdout:9/669: dwrite d0/d8a/d93/f91 [0,4194304] 0 2024-04-25T00:18:33.907 INFO:tasks.workunit.client.0.smithi107.stdout:8/627: link d2/d18/d1a/d5e/d3e/fc9 d2/d18/d1a/d5e/d3e/db1/fd0 0 2024-04-25T00:18:33.908 INFO:tasks.workunit.client.0.smithi107.stdout:6/509: dwrite d1/d4c/fa6 [0,4194304] 0 2024-04-25T00:18:33.909 INFO:tasks.workunit.client.0.smithi107.stdout:6/510: link d1/d2/db/c5d d1/d2/d3/d71/cab 0 2024-04-25T00:18:33.910 INFO:tasks.workunit.client.0.smithi107.stdout:0/563: mknod dc/d24/d25/cab 0 2024-04-25T00:18:33.911 INFO:tasks.workunit.client.0.smithi107.stdout:6/511: dread d1/d2/db/d32/d36/d3a/f9a [0,4194304] 0 2024-04-25T00:18:33.912 INFO:tasks.workunit.client.0.smithi107.stdout:2/667: link d6/d18/d40/c8e d6/ce4 0 2024-04-25T00:18:33.912 INFO:tasks.workunit.client.0.smithi107.stdout:2/668: write d6/d18/d40/d42/d2b/d30/d45/fa3 [1516294,53007] 0 2024-04-25T00:18:33.914 INFO:tasks.workunit.client.0.smithi107.stdout:0/564: mkdir dc/dd/d16/dac 0 2024-04-25T00:18:33.917 INFO:tasks.workunit.client.0.smithi107.stdout:9/670: symlink d0/d8a/d93/d56/d76/d50/dd8/le1 0 2024-04-25T00:18:33.921 INFO:tasks.workunit.client.0.smithi107.stdout:4/709: rename d3/d5/d32/f54 to d3/d24/d5c/dab/fec 0 2024-04-25T00:18:33.922 INFO:tasks.workunit.client.0.smithi107.stdout:8/628: mkdir d2/d18/d1a/d5e/d4f/dd1 0 2024-04-25T00:18:33.923 INFO:tasks.workunit.client.0.smithi107.stdout:6/512: dread d1/d2/db/d3c/f50 [0,4194304] 0 2024-04-25T00:18:33.923 INFO:tasks.workunit.client.0.smithi107.stdout:2/669: write d6/d18/d40/d42/d2b/f7a [3057483,57691] 0 2024-04-25T00:18:33.923 INFO:tasks.workunit.client.0.smithi107.stdout:7/577: rename d0/d8/db/d18/d19/d67/cbb to d0/d8/db/d18/d19/d47/d4d/cc7 0 2024-04-25T00:18:33.927 INFO:tasks.workunit.client.0.smithi107.stdout:8/629: mkdir d2/d18/da2/db2/dd2 0 2024-04-25T00:18:33.928 INFO:tasks.workunit.client.0.smithi107.stdout:7/578: creat d0/fc8 x:0 0 0 2024-04-25T00:18:33.929 INFO:tasks.workunit.client.0.smithi107.stdout:2/670: mkdir d6/d18/d40/dc8/de5 0 2024-04-25T00:18:33.930 INFO:tasks.workunit.client.0.smithi107.stdout:1/621: rename d2/d2a/la1 to d2/dd/d1a/lbf 0 2024-04-25T00:18:33.930 INFO:tasks.workunit.client.0.smithi107.stdout:7/579: creat d0/d8/db/d18/d19/d67/d7a/fc9 x:0 0 0 2024-04-25T00:18:33.930 INFO:tasks.workunit.client.0.smithi107.stdout:7/580: chown d0/d8/d4e/d46/d93/fac 167359 1 2024-04-25T00:18:33.932 INFO:tasks.workunit.client.0.smithi107.stdout:0/565: link dc/dd/d5d/c29 dc/d24/d38/cad 0 2024-04-25T00:18:33.932 INFO:tasks.workunit.client.0.smithi107.stdout:7/581: link d0/d8/db/f49 d0/d8/db/d18/d19/d47/d4d/d8a/fca 0 2024-04-25T00:18:33.933 INFO:tasks.workunit.client.0.smithi107.stdout:7/582: chown d0/d8/db/d18/d19/d67/d7a/fc9 13502 1 2024-04-25T00:18:33.933 INFO:tasks.workunit.client.0.smithi107.stdout:7/583: fsync d0/d80/da2/fad 0 2024-04-25T00:18:33.933 INFO:tasks.workunit.client.0.smithi107.stdout:7/584: dread - d0/d8/db/d18/d19/d47/d4d/d8a/fb8 zero size 2024-04-25T00:18:33.933 INFO:tasks.workunit.client.0.smithi107.stdout:9/671: link d0/f7e d0/d8a/d93/d56/d6c/d80/fe2 0 2024-04-25T00:18:33.934 INFO:tasks.workunit.client.0.smithi107.stdout:2/671: rmdir d6/d18/d40/d42/d69/d82 39 2024-04-25T00:18:33.935 INFO:tasks.workunit.client.0.smithi107.stdout:1/622: mkdir d2/d2a/d5c/d89/dc0 0 2024-04-25T00:18:33.935 INFO:tasks.workunit.client.0.smithi107.stdout:1/623: write d2/dd/d82/f99 [334235,106422] 0 2024-04-25T00:18:33.936 INFO:tasks.workunit.client.0.smithi107.stdout:0/566: creat dc/d24/d38/d64/d7b/fae x:0 0 0 2024-04-25T00:18:33.938 INFO:tasks.workunit.client.0.smithi107.stdout:9/672: mknod d0/d8a/d93/d34/d36/d77/dac/ce3 0 2024-04-25T00:18:33.938 INFO:tasks.workunit.client.0.smithi107.stdout:9/673: chown d0/d8a/d93/d56/d76/d50/d7d 64625703 1 2024-04-25T00:18:33.938 INFO:tasks.workunit.client.0.smithi107.stdout:0/567: creat dc/dd/d16/faf x:0 0 0 2024-04-25T00:18:33.939 INFO:tasks.workunit.client.0.smithi107.stdout:6/513: write d1/d2/db/fd [2199417,11316] 0 2024-04-25T00:18:33.939 INFO:tasks.workunit.client.0.smithi107.stdout:0/568: mkdir dc/dd/d16/d48/d83/d9f/db0 0 2024-04-25T00:18:33.940 INFO:tasks.workunit.client.0.smithi107.stdout:0/569: creat dc/dd/d16/d5e/fb1 x:0 0 0 2024-04-25T00:18:33.940 INFO:tasks.workunit.client.0.smithi107.stdout:0/570: fsync dc/dd/d5d/d3f/f92 0 2024-04-25T00:18:33.940 INFO:tasks.workunit.client.0.smithi107.stdout:0/571: dread - dc/dd/d16/d33/d35/f8d zero size 2024-04-25T00:18:33.943 INFO:tasks.workunit.client.0.smithi107.stdout:6/514: creat d1/d4c/fac x:0 0 0 2024-04-25T00:18:33.944 INFO:tasks.workunit.client.0.smithi107.stdout:6/515: chown d1/d2/db/d32 15 1 2024-04-25T00:18:33.944 INFO:tasks.workunit.client.0.smithi107.stdout:1/624: mkdir d2/dd/d82/dc1 0 2024-04-25T00:18:33.945 INFO:tasks.workunit.client.0.smithi107.stdout:2/672: rename d6/d18/d40/d42/d2b/d30/d4c/lc3 to d6/d29/d4e/d6f/le6 0 2024-04-25T00:18:33.947 INFO:tasks.workunit.client.0.smithi107.stdout:1/625: unlink d2/dd/d25/l2c 0 2024-04-25T00:18:33.948 INFO:tasks.workunit.client.0.smithi107.stdout:1/626: mkdir d2/dd/d1b/dc2 0 2024-04-25T00:18:33.951 INFO:tasks.workunit.client.0.smithi107.stdout:8/630: write d2/d18/f53 [3979582,111420] 0 2024-04-25T00:18:33.954 INFO:tasks.workunit.client.0.smithi107.stdout:8/631: mknod d2/d18/d1a/d5e/d4f/dd1/cd3 0 2024-04-25T00:18:33.960 INFO:tasks.workunit.client.0.smithi107.stdout:8/632: mknod d2/d18/d1a/d5e/d5c/cd4 0 2024-04-25T00:18:33.961 INFO:tasks.workunit.client.0.smithi107.stdout:2/673: dread d6/d18/d40/d42/f62 [0,4194304] 0 2024-04-25T00:18:33.963 INFO:tasks.workunit.client.0.smithi107.stdout:2/674: creat d6/d18/d40/d42/d69/d6e/dc2/fe7 x:0 0 0 2024-04-25T00:18:33.964 INFO:tasks.workunit.client.0.smithi107.stdout:5/673: dwrite d4/dc/d26/d42/f59 [0,4194304] 0 2024-04-25T00:18:33.964 INFO:tasks.workunit.client.0.smithi107.stdout:5/674: chown d4/dc/d26/d29/d51/fb8 962776 1 2024-04-25T00:18:33.973 INFO:tasks.workunit.client.0.smithi107.stdout:9/674: dwrite d0/d8a/d93/d34/f84 [0,4194304] 0 2024-04-25T00:18:33.975 INFO:tasks.workunit.client.0.smithi107.stdout:6/516: dread d1/d14/d18/f2c [0,4194304] 0 2024-04-25T00:18:33.975 INFO:tasks.workunit.client.0.smithi107.stdout:1/627: dwrite d2/da/d2f/f3e [0,4194304] 0 2024-04-25T00:18:33.975 INFO:tasks.workunit.client.0.smithi107.stdout:8/633: dread d2/f55 [0,4194304] 0 2024-04-25T00:18:33.975 INFO:tasks.workunit.client.0.smithi107.stdout:7/585: dwrite d0/d8/db/d18/d19/d39/f6c [4194304,4194304] 0 2024-04-25T00:18:33.976 INFO:tasks.workunit.client.0.smithi107.stdout:4/710: dwrite d3/d5/fe6 [0,4194304] 0 2024-04-25T00:18:33.978 INFO:tasks.workunit.client.0.smithi107.stdout:9/675: getdents d0/d8a/d93/de/d5c/d95 0 2024-04-25T00:18:33.978 INFO:tasks.workunit.client.0.smithi107.stdout:9/676: write d0/d8a/d93/de/f90 [397561,20752] 0 2024-04-25T00:18:33.978 INFO:tasks.workunit.client.0.smithi107.stdout:9/677: chown d0/d8a/d93/db9/lcd 835427085 1 2024-04-25T00:18:33.979 INFO:tasks.workunit.client.0.smithi107.stdout:5/675: write d4/dc/d83/fcc [3359159,59608] 0 2024-04-25T00:18:33.979 INFO:tasks.workunit.client.0.smithi107.stdout:5/676: stat d4/f16 0 2024-04-25T00:18:33.979 INFO:tasks.workunit.client.0.smithi107.stdout:5/677: fdatasync d4/dc/d26/d29/d51/d78/fdb 0 2024-04-25T00:18:33.979 INFO:tasks.workunit.client.0.smithi107.stdout:7/586: symlink d0/d8/db/d18/d19/d47/d4d/dbc/d9c/lcb 0 2024-04-25T00:18:33.982 INFO:tasks.workunit.client.0.smithi107.stdout:1/628: creat d2/d62/d66/fc3 x:0 0 0 2024-04-25T00:18:33.983 INFO:tasks.workunit.client.0.smithi107.stdout:8/634: getdents d2/d18/da2 0 2024-04-25T00:18:33.985 INFO:tasks.workunit.client.0.smithi107.stdout:6/517: mkdir d1/d2/dad 0 2024-04-25T00:18:33.986 INFO:tasks.workunit.client.0.smithi107.stdout:6/518: fsync d1/d2/db/d32/d33/d68/d75/f78 0 2024-04-25T00:18:33.988 INFO:tasks.workunit.client.0.smithi107.stdout:5/678: rmdir d4/dc/d26/d29/d2e/d55 39 2024-04-25T00:18:33.989 INFO:tasks.workunit.client.0.smithi107.stdout:1/629: rmdir d2/d2a/d37/d88 39 2024-04-25T00:18:33.991 INFO:tasks.workunit.client.0.smithi107.stdout:7/587: dread d0/d8/d2c/f3d [0,4194304] 0 2024-04-25T00:18:33.991 INFO:tasks.workunit.client.0.smithi107.stdout:2/675: dwrite d6/d20/d22/d56/d94/fdf [0,4194304] 0 2024-04-25T00:18:33.994 INFO:tasks.workunit.client.0.smithi107.stdout:7/588: dread d0/d8/db/d18/d19/f23 [0,4194304] 0 2024-04-25T00:18:33.994 INFO:tasks.workunit.client.0.smithi107.stdout:7/589: fsync d0/d8/d2c/d58/fb4 0 2024-04-25T00:18:33.996 INFO:tasks.workunit.client.0.smithi107.stdout:7/590: mknod d0/d8/d2c/d58/db5/ccc 0 2024-04-25T00:18:33.996 INFO:tasks.workunit.client.0.smithi107.stdout:7/591: write d0/d8/d4e/d46/d93/fac [135199,128634] 0 2024-04-25T00:18:33.997 INFO:tasks.workunit.client.0.smithi107.stdout:1/630: symlink d2/dd/d25/lc4 0 2024-04-25T00:18:34.004 INFO:tasks.workunit.client.0.smithi107.stdout:0/572: read dc/d24/d38/d64/faa [3916,88797] 0 2024-04-25T00:18:34.004 INFO:tasks.workunit.client.0.smithi107.stdout:0/573: fdatasync dc/d24/d38/d64/f88 0 2024-04-25T00:18:34.004 INFO:tasks.workunit.client.0.smithi107.stdout:0/574: fdatasync dc/dd/d16/faf 0 2024-04-25T00:18:34.004 INFO:tasks.workunit.client.0.smithi107.stdout:5/679: dread d4/d17/f21 [0,4194304] 0 2024-04-25T00:18:34.004 INFO:tasks.workunit.client.0.smithi107.stdout:5/680: dread - d4/dc/d26/d42/d8b/fc6 zero size 2024-04-25T00:18:34.005 INFO:tasks.workunit.client.0.smithi107.stdout:8/635: dwrite d2/d18/d1a/d5e/d4f/da0/da1/fad [0,4194304] 0 2024-04-25T00:18:34.005 INFO:tasks.workunit.client.0.smithi107.stdout:8/636: dread - d2/d18/d1a/d33/f66 zero size 2024-04-25T00:18:34.005 INFO:tasks.workunit.client.0.smithi107.stdout:5/681: mknod d4/dc/d26/d62/dd5/cde 0 2024-04-25T00:18:34.006 INFO:tasks.workunit.client.0.smithi107.stdout:5/682: fsync d4/dc/d26/d42/f4f 0 2024-04-25T00:18:34.012 INFO:tasks.workunit.client.0.smithi107.stdout:9/678: dwrite d0/d8a/d93/de/f90 [0,4194304] 0 2024-04-25T00:18:34.015 INFO:tasks.workunit.client.0.smithi107.stdout:8/637: dread d2/d18/d1a/d5e/d5c/d6e/fa6 [0,4194304] 0 2024-04-25T00:18:34.015 INFO:tasks.workunit.client.0.smithi107.stdout:8/638: readlink d2/d18/d1a/d5e/d3e/lb0 0 2024-04-25T00:18:34.015 INFO:tasks.workunit.client.0.smithi107.stdout:9/679: mkdir d0/d8a/d93/d56/d76/d50/de4 0 2024-04-25T00:18:34.016 INFO:tasks.workunit.client.0.smithi107.stdout:1/631: dwrite d2/da/d2f/f9b [0,4194304] 0 2024-04-25T00:18:34.016 INFO:tasks.workunit.client.0.smithi107.stdout:0/575: dread dc/dd/f14 [4194304,4194304] 0 2024-04-25T00:18:34.017 INFO:tasks.workunit.client.0.smithi107.stdout:1/632: symlink d2/dd/d82/lc5 0 2024-04-25T00:18:34.019 INFO:tasks.workunit.client.0.smithi107.stdout:0/576: mkdir dc/dd/d5d/d5c/db2 0 2024-04-25T00:18:34.020 INFO:tasks.workunit.client.0.smithi107.stdout:8/639: chown d2/d18/d1a/d5e/d4f/da0/da1/ca4 7 1 2024-04-25T00:18:34.022 INFO:tasks.workunit.client.0.smithi107.stdout:8/640: mkdir d2/d18/d1a/d5e/d3d/d85/dd5 0 2024-04-25T00:18:34.023 INFO:tasks.workunit.client.0.smithi107.stdout:4/711: rename d3/da8/lb4 to d3/d24/d5c/led 0 2024-04-25T00:18:34.024 INFO:tasks.workunit.client.0.smithi107.stdout:9/680: dread d0/d43/f60 [0,4194304] 0 2024-04-25T00:18:34.024 INFO:tasks.workunit.client.0.smithi107.stdout:9/681: chown d0/d8a/d93/d56/d6c/d80/f97 1702652 1 2024-04-25T00:18:34.025 INFO:tasks.workunit.client.0.smithi107.stdout:4/712: symlink d3/d24/d5c/d7a/lee 0 2024-04-25T00:18:34.025 INFO:tasks.workunit.client.0.smithi107.stdout:4/713: write d3/d24/d5c/dab/f7d [1091961,69282] 0 2024-04-25T00:18:34.026 INFO:tasks.workunit.client.0.smithi107.stdout:6/519: rename d1/d2/d3/d26/f51 to d1/d2/db/d48/fae 0 2024-04-25T00:18:34.026 INFO:tasks.workunit.client.0.smithi107.stdout:0/577: symlink dc/dd/d16/lb3 0 2024-04-25T00:18:34.027 INFO:tasks.workunit.client.0.smithi107.stdout:4/714: getdents d3/d24/db1 0 2024-04-25T00:18:34.027 INFO:tasks.workunit.client.0.smithi107.stdout:4/715: truncate d3/d24/d5c/f98 486083 0 2024-04-25T00:18:34.028 INFO:tasks.workunit.client.0.smithi107.stdout:9/682: creat d0/d8a/d93/fe5 x:0 0 0 2024-04-25T00:18:34.028 INFO:tasks.workunit.client.0.smithi107.stdout:3/590: sync 2024-04-25T00:18:34.028 INFO:tasks.workunit.client.0.smithi107.stdout:3/591: readlink d9/d88/d3a/l53 0 2024-04-25T00:18:34.028 INFO:tasks.workunit.client.0.smithi107.stdout:3/592: chown d9/d88/d4e/d93 187460 1 2024-04-25T00:18:34.029 INFO:tasks.workunit.client.0.smithi107.stdout:6/520: creat d1/d2/d2b/d3e/d5b/d7a/faf x:0 0 0 2024-04-25T00:18:34.030 INFO:tasks.workunit.client.0.smithi107.stdout:2/676: rename d6/d18/d40/d42/d69/d6e/d80/f86 to d6/d18/d40/fe8 0 2024-04-25T00:18:34.031 INFO:tasks.workunit.client.0.smithi107.stdout:5/683: dwrite d4/dc/d26/d29/d2e/d82/fa6 [0,4194304] 0 2024-04-25T00:18:34.031 INFO:tasks.workunit.client.0.smithi107.stdout:5/684: fdatasync d4/dc/d26/d29/d2e/f4c 0 2024-04-25T00:18:34.031 INFO:tasks.workunit.client.0.smithi107.stdout:5/685: write d4/dc/d26/d62/fc8 [1721540,79889] 0 2024-04-25T00:18:34.032 INFO:tasks.workunit.client.0.smithi107.stdout:4/716: link d3/d5/d32/d53/d7e/d9d/ca0 d3/d5/d32/cef 0 2024-04-25T00:18:34.032 INFO:tasks.workunit.client.0.smithi107.stdout:4/717: truncate d3/d5/d32/d53/f6e 654952 0 2024-04-25T00:18:34.032 INFO:tasks.workunit.client.0.smithi107.stdout:0/578: mknod dc/dd/d5d/d3f/da5/cb4 0 2024-04-25T00:18:34.033 INFO:tasks.workunit.client.0.smithi107.stdout:7/592: rename d0/d8/db/cc0 to d0/d8/d4e/ccd 0 2024-04-25T00:18:34.033 INFO:tasks.workunit.client.0.smithi107.stdout:9/683: creat d0/d8a/d93/db9/fe6 x:0 0 0 2024-04-25T00:18:34.034 INFO:tasks.workunit.client.0.smithi107.stdout:3/593: mkdir d9/d54/d94/dba 0 2024-04-25T00:18:34.034 INFO:tasks.workunit.client.0.smithi107.stdout:3/594: write d9/d22/d1b/d55/f5b [4451925,127218] 0 2024-04-25T00:18:34.036 INFO:tasks.workunit.client.0.smithi107.stdout:4/718: creat d3/d24/d25/d71/ff0 x:0 0 0 2024-04-25T00:18:34.036 INFO:tasks.workunit.client.0.smithi107.stdout:3/595: mknod d9/d88/d3a/d91/cbb 0 2024-04-25T00:18:34.037 INFO:tasks.workunit.client.0.smithi107.stdout:2/677: dread d6/d18/d40/d42/d2b/d30/d45/fa3 [0,4194304] 0 2024-04-25T00:18:34.037 INFO:tasks.workunit.client.0.smithi107.stdout:7/593: getdents d0/d8/db/d18/d19/d47/da1 0 2024-04-25T00:18:34.037 INFO:tasks.workunit.client.0.smithi107.stdout:1/633: rename d2/da/f15 to d2/d62/d66/fc6 0 2024-04-25T00:18:34.038 INFO:tasks.workunit.client.0.smithi107.stdout:9/684: mknod d0/d8a/d93/d34/d36/ce7 0 2024-04-25T00:18:34.039 INFO:tasks.workunit.client.0.smithi107.stdout:9/685: chown d0/d8a/d93/d6f/c92 91503 1 2024-04-25T00:18:34.039 INFO:tasks.workunit.client.0.smithi107.stdout:4/719: mknod d3/d5/cf1 0 2024-04-25T00:18:34.039 INFO:tasks.workunit.client.0.smithi107.stdout:4/720: fsync d3/d5/f3d 0 2024-04-25T00:18:34.039 INFO:tasks.workunit.client.0.smithi107.stdout:4/721: fsync d3/d24/d25/f26 0 2024-04-25T00:18:34.040 INFO:tasks.workunit.client.0.smithi107.stdout:7/594: creat d0/d8/d4e/fce x:0 0 0 2024-04-25T00:18:34.040 INFO:tasks.workunit.client.0.smithi107.stdout:2/678: creat d6/d20/d77/fe9 x:0 0 0 2024-04-25T00:18:34.041 INFO:tasks.workunit.client.0.smithi107.stdout:6/521: dread d1/d2/d3/d71/d94/f9b [0,4194304] 0 2024-04-25T00:18:34.042 INFO:tasks.workunit.client.0.smithi107.stdout:2/679: creat d6/dad/db0/fea x:0 0 0 2024-04-25T00:18:34.042 INFO:tasks.workunit.client.0.smithi107.stdout:3/596: rename d9/d22/d1b/l7e to d9/d54/d5c/d68/lbc 0 2024-04-25T00:18:34.042 INFO:tasks.workunit.client.0.smithi107.stdout:1/634: unlink d2/dd/d82/lc5 0 2024-04-25T00:18:34.044 INFO:tasks.workunit.client.0.smithi107.stdout:9/686: symlink d0/d8a/d93/d56/d6c/d80/d9b/le8 0 2024-04-25T00:18:34.044 INFO:tasks.workunit.client.0.smithi107.stdout:9/687: chown d0/d8a/d93/d34/d36/l66 540338 1 2024-04-25T00:18:34.045 INFO:tasks.workunit.client.0.smithi107.stdout:1/635: readlink d2/dd/d25/la6 0 2024-04-25T00:18:34.046 INFO:tasks.workunit.client.0.smithi107.stdout:6/522: rmdir d1/d2/d3/d28 39 2024-04-25T00:18:34.048 INFO:tasks.workunit.client.0.smithi107.stdout:4/722: dread d3/d5/f7 [0,4194304] 0 2024-04-25T00:18:34.048 INFO:tasks.workunit.client.0.smithi107.stdout:4/723: dread - d3/d24/d5c/d7a/fe7 zero size 2024-04-25T00:18:34.048 INFO:tasks.workunit.client.0.smithi107.stdout:4/724: write d3/d24/d5c/d7a/fe7 [15344,55984] 0 2024-04-25T00:18:34.050 INFO:tasks.workunit.client.0.smithi107.stdout:8/641: dwrite d2/d18/d1a/d5e/d3d/fb3 [0,4194304] 0 2024-04-25T00:18:34.051 INFO:tasks.workunit.client.0.smithi107.stdout:8/642: dread d2/d18/d1a/d5e/d4f/f7e [0,4194304] 0 2024-04-25T00:18:34.059 INFO:tasks.workunit.client.0.smithi107.stdout:1/636: write d2/dd/d1a/f6d [2024531,47360] 0 2024-04-25T00:18:34.059 INFO:tasks.workunit.client.0.smithi107.stdout:6/523: write d1/d2/d3/d28/d79/d6b/f6f [2355544,23122] 0 2024-04-25T00:18:34.060 INFO:tasks.workunit.client.0.smithi107.stdout:4/725: dread d3/d5/f7 [0,4194304] 0 2024-04-25T00:18:34.061 INFO:tasks.workunit.client.0.smithi107.stdout:4/726: getdents d3/d24/d25/d71/d7f 0 2024-04-25T00:18:34.061 INFO:tasks.workunit.client.0.smithi107.stdout:4/727: read d3/d24/d5c/d7a/fe7 [3728,45844] 0 2024-04-25T00:18:34.073 INFO:tasks.workunit.client.0.smithi107.stdout:3/597: mkdir d9/d22/d1b/dbd 0 2024-04-25T00:18:34.073 INFO:tasks.workunit.client.0.smithi107.stdout:3/598: write f7 [4949920,57495] 0 2024-04-25T00:18:34.074 INFO:tasks.workunit.client.0.smithi107.stdout:9/688: symlink d0/d8a/d93/d56/le9 0 2024-04-25T00:18:34.074 INFO:tasks.workunit.client.0.smithi107.stdout:9/689: write d0/d8a/d93/d6f/da1/fc8 [704952,114731] 0 2024-04-25T00:18:34.075 INFO:tasks.workunit.client.0.smithi107.stdout:7/595: dwrite d0/d8/db/d18/d19/d47/f62 [0,4194304] 0 2024-04-25T00:18:34.075 INFO:tasks.workunit.client.0.smithi107.stdout:7/596: write d0/fc8 [5171,71985] 0 2024-04-25T00:18:34.076 INFO:tasks.workunit.client.0.smithi107.stdout:2/680: dwrite d6/d18/d40/d42/d69/d6e/db6/fdd [0,4194304] 0 2024-04-25T00:18:34.077 INFO:tasks.workunit.client.0.smithi107.stdout:8/643: mkdir d2/d18/d1a/d5e/dd6 0 2024-04-25T00:18:34.077 INFO:tasks.workunit.client.0.smithi107.stdout:8/644: chown d2/d18/d1a/d5e/f56 2401563 1 2024-04-25T00:18:34.084 INFO:tasks.workunit.client.0.smithi107.stdout:4/728: dwrite d3/d5/d32/d9e/da2/fa4 [0,4194304] 0 2024-04-25T00:18:34.085 INFO:tasks.workunit.client.0.smithi107.stdout:1/637: truncate d2/d62/d66/f78 123623 0 2024-04-25T00:18:34.086 INFO:tasks.workunit.client.0.smithi107.stdout:9/690: stat d0/d8a/d93/d34/d36/l66 0 2024-04-25T00:18:34.087 INFO:tasks.workunit.client.0.smithi107.stdout:4/729: mkdir d3/d24/df2 0 2024-04-25T00:18:34.088 INFO:tasks.workunit.client.0.smithi107.stdout:2/681: mknod d6/d18/d40/d42/d69/d82/ceb 0 2024-04-25T00:18:34.088 INFO:tasks.workunit.client.0.smithi107.stdout:2/682: readlink d6/dad/d67/lde 0 2024-04-25T00:18:34.090 INFO:tasks.workunit.client.0.smithi107.stdout:8/645: link d2/d18/d1a/f80 d2/d18/da2/dc2/fd7 0 2024-04-25T00:18:34.092 INFO:tasks.workunit.client.0.smithi107.stdout:4/730: rmdir d3/d24/d25 39 2024-04-25T00:18:34.093 INFO:tasks.workunit.client.0.smithi107.stdout:1/638: truncate d2/f8a 1932336 0 2024-04-25T00:18:34.093 INFO:tasks.workunit.client.0.smithi107.stdout:1/639: write d2/da/d2f/f8d [48456,28003] 0 2024-04-25T00:18:34.095 INFO:tasks.workunit.client.0.smithi107.stdout:2/683: mkdir d6/d18/d40/d42/d2b/d30/d45/dec 0 2024-04-25T00:18:34.098 INFO:tasks.workunit.client.0.smithi107.stdout:8/646: creat d2/d18/d1a/dca/fd8 x:0 0 0 2024-04-25T00:18:34.099 INFO:tasks.workunit.client.0.smithi107.stdout:1/640: mknod d2/dd/d82/dc1/cc7 0 2024-04-25T00:18:34.101 INFO:tasks.workunit.client.0.smithi107.stdout:2/684: creat d6/d18/d40/d42/d69/d6e/da2/dc5/dd2/fed x:0 0 0 2024-04-25T00:18:34.102 INFO:tasks.workunit.client.0.smithi107.stdout:2/685: symlink d6/dad/d67/da8/lee 0 2024-04-25T00:18:34.103 INFO:tasks.workunit.client.0.smithi107.stdout:8/647: creat d2/d18/da2/fd9 x:0 0 0 2024-04-25T00:18:34.104 INFO:tasks.workunit.client.0.smithi107.stdout:8/648: truncate d2/d18/d1a/d5e/fbe 165366 0 2024-04-25T00:18:34.104 INFO:tasks.workunit.client.0.smithi107.stdout:8/649: fsync d2/d18/d1a/d5e/d4f/f76 0 2024-04-25T00:18:34.104 INFO:tasks.workunit.client.0.smithi107.stdout:3/599: dwrite d9/d54/fb4 [0,4194304] 0 2024-04-25T00:18:34.104 INFO:tasks.workunit.client.0.smithi107.stdout:9/691: dread d0/d8a/d93/d6f/fc4 [0,4194304] 0 2024-04-25T00:18:34.104 INFO:tasks.workunit.client.0.smithi107.stdout:1/641: creat d2/d62/fc8 x:0 0 0 2024-04-25T00:18:34.105 INFO:tasks.workunit.client.0.smithi107.stdout:2/686: mknod d6/d18/d40/d42/d2b/d30/d45/dca/dd4/cef 0 2024-04-25T00:18:34.106 INFO:tasks.workunit.client.0.smithi107.stdout:4/731: read d3/d24/d5c/dab/fec [3261842,92759] 0 2024-04-25T00:18:34.107 INFO:tasks.workunit.client.0.smithi107.stdout:4/732: mknod d3/d24/cf3 0 2024-04-25T00:18:34.108 INFO:tasks.workunit.client.0.smithi107.stdout:9/692: dread d0/d8a/d93/d6f/f82 [0,4194304] 0 2024-04-25T00:18:34.108 INFO:tasks.workunit.client.0.smithi107.stdout:4/733: creat d3/d5/d32/ff4 x:0 0 0 2024-04-25T00:18:34.108 INFO:tasks.workunit.client.0.smithi107.stdout:4/734: write d3/d5/d32/f45 [2946078,48555] 0 2024-04-25T00:18:34.109 INFO:tasks.workunit.client.0.smithi107.stdout:3/600: unlink d9/d22/d1b/d55/f7f 0 2024-04-25T00:18:34.110 INFO:tasks.workunit.client.0.smithi107.stdout:3/601: creat d9/d88/d2c/fbe x:0 0 0 2024-04-25T00:18:34.110 INFO:tasks.workunit.client.0.smithi107.stdout:3/602: fsync d9/d88/d3a/fb1 0 2024-04-25T00:18:34.111 INFO:tasks.workunit.client.0.smithi107.stdout:9/693: mkdir d0/d8a/d93/d56/d6c/d80/d9b/db2/dea 0 2024-04-25T00:18:34.112 INFO:tasks.workunit.client.0.smithi107.stdout:9/694: mknod d0/d8a/d93/d56/d6c/daa/ceb 0 2024-04-25T00:18:34.117 INFO:tasks.workunit.client.0.smithi107.stdout:1/642: dread d2/d62/d66/d70/f77 [0,4194304] 0 2024-04-25T00:18:34.122 INFO:tasks.workunit.client.0.smithi107.stdout:2/687: write d6/d29/d4e/fab [419800,79493] 0 2024-04-25T00:18:34.123 INFO:tasks.workunit.client.0.smithi107.stdout:1/643: mknod d2/d62/d66/d70/d95/cc9 0 2024-04-25T00:18:34.123 INFO:tasks.workunit.client.0.smithi107.stdout:1/644: write d2/dd/d82/fa7 [3726075,97187] 0 2024-04-25T00:18:34.125 INFO:tasks.workunit.client.0.smithi107.stdout:2/688: getdents d6/d18/d40/d42/d2b/d30/d45/dca/dd4 0 2024-04-25T00:18:34.127 INFO:tasks.workunit.client.0.smithi107.stdout:3/603: dread d9/d15/da1/d4b/f57 [0,4194304] 0 2024-04-25T00:18:34.129 INFO:tasks.workunit.client.0.smithi107.stdout:3/604: mkdir d9/d88/d2c/dbf 0 2024-04-25T00:18:34.129 INFO:tasks.workunit.client.0.smithi107.stdout:8/650: dwrite d2/f55 [0,4194304] 0 2024-04-25T00:18:34.131 INFO:tasks.workunit.client.0.smithi107.stdout:3/605: symlink d9/d54/d5c/d68/db3/lc0 0 2024-04-25T00:18:34.131 INFO:tasks.workunit.client.0.smithi107.stdout:4/735: write d3/d5/d32/f9b [1994394,9865] 0 2024-04-25T00:18:34.133 INFO:tasks.workunit.client.0.smithi107.stdout:8/651: symlink d2/d18/da2/lda 0 2024-04-25T00:18:34.133 INFO:tasks.workunit.client.0.smithi107.stdout:4/736: mknod d3/d24/d25/d71/cf5 0 2024-04-25T00:18:34.134 INFO:tasks.workunit.client.0.smithi107.stdout:4/737: creat d3/d24/d25/d71/dc2/ff6 x:0 0 0 2024-04-25T00:18:34.139 INFO:tasks.workunit.client.0.smithi107.stdout:3/606: creat d9/d15/da1/db0/fc1 x:0 0 0 2024-04-25T00:18:34.139 INFO:tasks.workunit.client.0.smithi107.stdout:3/607: fdatasync d9/d15/da1/f79 0 2024-04-25T00:18:34.139 INFO:tasks.workunit.client.0.smithi107.stdout:3/608: getdents d9/d88/d2c/dbf 0 2024-04-25T00:18:34.140 INFO:tasks.workunit.client.0.smithi107.stdout:8/652: symlink d2/d18/d1a/d5e/d3e/d74/ldb 0 2024-04-25T00:18:34.140 INFO:tasks.workunit.client.0.smithi107.stdout:8/653: write d2/d18/d1a/d5e/d3e/d74/f97 [572776,29455] 0 2024-04-25T00:18:34.140 INFO:tasks.workunit.client.0.smithi107.stdout:8/654: write d2/d18/d1a/d5e/d5c/f88 [990420,79742] 0 2024-04-25T00:18:34.142 INFO:tasks.workunit.client.0.smithi107.stdout:1/645: dwrite d2/dd/d1b/f52 [0,4194304] 0 2024-04-25T00:18:34.142 INFO:tasks.workunit.client.0.smithi107.stdout:6/524: rename d1/d2/db/d32/d36/f59 to d1/d2/d3/d71/d94/fb0 0 2024-04-25T00:18:34.142 INFO:tasks.workunit.client.0.smithi107.stdout:7/597: rename d0/d8/db/d18/d19/d47/d4d to d0/d8/db/d18/d19/d47/d4d/dcf 22 2024-04-25T00:18:34.142 INFO:tasks.workunit.client.0.smithi107.stdout:6/525: truncate d1/d2/db/d32/d36/d3d/fa1 940747 0 2024-04-25T00:18:34.142 INFO:tasks.workunit.client.0.smithi107.stdout:3/609: creat d9/d22/d1b/fc2 x:0 0 0 2024-04-25T00:18:34.143 INFO:tasks.workunit.client.0.smithi107.stdout:3/610: fsync d9/d88/d3a/d91/f9a 0 2024-04-25T00:18:34.143 INFO:tasks.workunit.client.0.smithi107.stdout:7/598: rmdir d0/d80/da2 39 2024-04-25T00:18:34.143 INFO:tasks.workunit.client.0.smithi107.stdout:7/599: fsync d0/d8/d2c/f55 0 2024-04-25T00:18:34.144 INFO:tasks.workunit.client.0.smithi107.stdout:8/655: creat d2/d18/d1a/fdc x:0 0 0 2024-04-25T00:18:34.144 INFO:tasks.workunit.client.0.smithi107.stdout:8/656: fdatasync d2/d18/d1a/d5e/d3e/d74/f8a 0 2024-04-25T00:18:34.145 INFO:tasks.workunit.client.0.smithi107.stdout:4/738: dread d3/fd [0,4194304] 0 2024-04-25T00:18:34.147 INFO:tasks.workunit.client.0.smithi107.stdout:4/739: link d3/d5/c4a d3/d24/d25/d71/d7f/cf7 0 2024-04-25T00:18:34.147 INFO:tasks.workunit.client.0.smithi107.stdout:4/740: chown d3/d5/d32/d53/d7e/fb3 0 1 2024-04-25T00:18:34.147 INFO:tasks.workunit.client.0.smithi107.stdout:2/689: dwrite d6/d20/d22/f3a [0,4194304] 0 2024-04-25T00:18:34.147 INFO:tasks.workunit.client.0.smithi107.stdout:4/741: write d3/d10/d7b/feb [847904,37696] 0 2024-04-25T00:18:34.149 INFO:tasks.workunit.client.0.smithi107.stdout:9/695: rename d0/d8a/d93/d56/l67 to d0/d8a/d93/d34/lec 0 2024-04-25T00:18:34.152 INFO:tasks.workunit.client.0.smithi107.stdout:4/742: rmdir d3/d5/d32/d9e 39 2024-04-25T00:18:34.152 INFO:tasks.workunit.client.0.smithi107.stdout:4/743: write d3/d5/d32/d53/d7e/fd4 [187551,32639] 0 2024-04-25T00:18:34.153 INFO:tasks.workunit.client.0.smithi107.stdout:8/657: unlink d2/d18/d1a/d5e/d3d/c61 0 2024-04-25T00:18:34.155 INFO:tasks.workunit.client.0.smithi107.stdout:7/600: rename d0/d8/db/d18/d19/d47/d4d/dbc/d9c to d0/d8/db/d18/d19/d67/d7a/dc2/dd0 0 2024-04-25T00:18:34.155 INFO:tasks.workunit.client.0.smithi107.stdout:7/601: write d0/d8/db/f15 [4036981,30072] 0 2024-04-25T00:18:34.155 INFO:tasks.workunit.client.0.smithi107.stdout:7/602: dread - d0/d80/da8/fb9 zero size 2024-04-25T00:18:34.156 INFO:tasks.workunit.client.0.smithi107.stdout:1/646: unlink d2/da/c5a 0 2024-04-25T00:18:34.156 INFO:tasks.workunit.client.0.smithi107.stdout:1/647: dread - d2/d62/d66/fc3 zero size 2024-04-25T00:18:34.159 INFO:tasks.workunit.client.0.smithi107.stdout:9/696: symlink d0/d8a/d93/db9/led 0 2024-04-25T00:18:34.166 INFO:tasks.workunit.client.0.smithi107.stdout:6/526: write d1/d2/d3/f31 [3905855,128818] 0 2024-04-25T00:18:34.166 INFO:tasks.workunit.client.0.smithi107.stdout:3/611: dwrite d9/d54/d5c/d68/f3c [0,4194304] 0 2024-04-25T00:18:34.168 INFO:tasks.workunit.client.0.smithi107.stdout:3/612: creat d9/fc3 x:0 0 0 2024-04-25T00:18:34.168 INFO:tasks.workunit.client.0.smithi107.stdout:6/527: dread d1/d2/db/d32/d36/d3d/fa1 [0,4194304] 0 2024-04-25T00:18:34.168 INFO:tasks.workunit.client.0.smithi107.stdout:3/613: creat d9/d54/fc4 x:0 0 0 2024-04-25T00:18:34.169 INFO:tasks.workunit.client.0.smithi107.stdout:8/658: write d2/f63 [3886003,4987] 0 2024-04-25T00:18:34.172 INFO:tasks.workunit.client.0.smithi107.stdout:2/690: dwrite d6/d18/d40/d42/d69/d6e/dc2/fe7 [0,4194304] 0 2024-04-25T00:18:34.177 INFO:tasks.workunit.client.0.smithi107.stdout:2/691: mknod d6/d18/d40/d42/d69/d6e/db6/cf0 0 2024-04-25T00:18:34.178 INFO:tasks.workunit.client.0.smithi107.stdout:4/744: dwrite d3/d24/d25/d71/ff0 [0,4194304] 0 2024-04-25T00:18:34.181 INFO:tasks.workunit.client.0.smithi107.stdout:4/745: getdents d3/d24/d5c/dab 0 2024-04-25T00:18:34.182 INFO:tasks.workunit.client.0.smithi107.stdout:8/659: unlink d2/d18/d1a/d5e/d4f/da0/da1/lbd 0 2024-04-25T00:18:34.182 INFO:tasks.workunit.client.0.smithi107.stdout:8/660: chown d2/d18/f68 23832 1 2024-04-25T00:18:34.182 INFO:tasks.workunit.client.0.smithi107.stdout:8/661: fdatasync d2/d18/d1a/d5e/d3e/fc9 0 2024-04-25T00:18:34.184 INFO:tasks.workunit.client.0.smithi107.stdout:7/603: dwrite d0/d8/db/d18/f41 [0,4194304] 0 2024-04-25T00:18:34.186 INFO:tasks.workunit.client.0.smithi107.stdout:1/648: dwrite d2/d2a/f9e [0,4194304] 0 2024-04-25T00:18:34.187 INFO:tasks.workunit.client.0.smithi107.stdout:4/746: rmdir d3/d24/d25/d67/dcb 0 2024-04-25T00:18:34.194 INFO:tasks.workunit.client.0.smithi107.stdout:6/528: dwrite d1/d2/db/d32/d36/d3d/fa1 [0,4194304] 0 2024-04-25T00:18:34.194 INFO:tasks.workunit.client.0.smithi107.stdout:1/649: dread d2/da/f8e [0,4194304] 0 2024-04-25T00:18:34.194 INFO:tasks.workunit.client.0.smithi107.stdout:6/529: write d1/d2/db/f34 [174900,66834] 0 2024-04-25T00:18:34.195 INFO:tasks.workunit.client.0.smithi107.stdout:8/662: dread d2/f63 [0,4194304] 0 2024-04-25T00:18:34.195 INFO:tasks.workunit.client.0.smithi107.stdout:1/650: write d2/dd/d25/f33 [3561863,127614] 0 2024-04-25T00:18:34.195 INFO:tasks.workunit.client.0.smithi107.stdout:1/651: chown d2/d2a/d5c/d89/dc0 6 1 2024-04-25T00:18:34.195 INFO:tasks.workunit.client.0.smithi107.stdout:1/652: chown d2/d62/d66/d70 2853 1 2024-04-25T00:18:34.196 INFO:tasks.workunit.client.0.smithi107.stdout:8/663: write d2/d18/d1a/f80 [226832,9163] 0 2024-04-25T00:18:34.196 INFO:tasks.workunit.client.0.smithi107.stdout:2/692: dwrite d6/d18/d40/d42/d69/fb2 [0,4194304] 0 2024-04-25T00:18:34.196 INFO:tasks.workunit.client.0.smithi107.stdout:2/693: write d6/d18/f78 [4355877,3685] 0 2024-04-25T00:18:34.196 INFO:tasks.workunit.client.0.smithi107.stdout:2/694: dread - d6/d20/f89 zero size 2024-04-25T00:18:34.196 INFO:tasks.workunit.client.0.smithi107.stdout:6/530: dread d1/d2/db/d32/d36/f46 [0,4194304] 0 2024-04-25T00:18:34.196 INFO:tasks.workunit.client.0.smithi107.stdout:6/531: write d1/d2/db/d32/d33/d4f/f93 [819756,110908] 0 2024-04-25T00:18:34.196 INFO:tasks.workunit.client.0.smithi107.stdout:1/653: truncate d2/f51 356857 0 2024-04-25T00:18:34.196 INFO:tasks.workunit.client.0.smithi107.stdout:1/654: write d2/da/f8e [3152937,28436] 0 2024-04-25T00:18:34.199 INFO:tasks.workunit.client.0.smithi107.stdout:6/532: read d1/d2/db/d32/d33/f63 [22120,57501] 0 2024-04-25T00:18:34.200 INFO:tasks.workunit.client.0.smithi107.stdout:2/695: mkdir d6/d29/d4e/d6f/df1 0 2024-04-25T00:18:34.202 INFO:tasks.workunit.client.0.smithi107.stdout:1/655: symlink d2/d2a/lca 0 2024-04-25T00:18:34.203 INFO:tasks.workunit.client.0.smithi107.stdout:1/656: readlink d2/da/l14 0 2024-04-25T00:18:34.203 INFO:tasks.workunit.client.0.smithi107.stdout:3/614: rename d9/d15/da1/db0/fc1 to d9/d88/d4e/d93/fc5 0 2024-04-25T00:18:34.204 INFO:tasks.workunit.client.0.smithi107.stdout:1/657: rmdir d2/da 39 2024-04-25T00:18:34.212 INFO:tasks.workunit.client.0.smithi107.stdout:7/604: dwrite d0/d8/db/d18/d19/d39/f6c [8388608,4194304] 0 2024-04-25T00:18:34.213 INFO:tasks.workunit.client.0.smithi107.stdout:7/605: stat d0/d8/d2c/d58/f84 0 2024-04-25T00:18:34.216 INFO:tasks.workunit.client.0.smithi107.stdout:7/606: unlink d0/d8/d2c/d58/f9a 0 2024-04-25T00:18:34.216 INFO:tasks.workunit.client.0.smithi107.stdout:7/607: rmdir d0/d80/da2 39 2024-04-25T00:18:34.218 INFO:tasks.workunit.client.0.smithi107.stdout:7/608: getdents d0/d8/db/d18 0 2024-04-25T00:18:34.219 INFO:tasks.workunit.client.0.smithi107.stdout:1/658: dread d2/dd/d25/f63 [0,4194304] 0 2024-04-25T00:18:34.220 INFO:tasks.workunit.client.0.smithi107.stdout:4/747: rename d3/d24/d25/d71/d7f/f9c to d3/ff8 0 2024-04-25T00:18:34.220 INFO:tasks.workunit.client.0.smithi107.stdout:4/748: truncate d3/f35 4311794 0 2024-04-25T00:18:34.221 INFO:tasks.workunit.client.0.smithi107.stdout:8/664: dwrite d2/d18/d1a/f31 [4194304,4194304] 0 2024-04-25T00:18:34.222 INFO:tasks.workunit.client.0.smithi107.stdout:1/659: creat d2/da4/fcb x:0 0 0 2024-04-25T00:18:34.223 INFO:tasks.workunit.client.0.smithi107.stdout:7/609: link d0/d8/db/d18/d19/d39/l4b d0/d8/d2c/ld1 0 2024-04-25T00:18:34.224 INFO:tasks.workunit.client.0.smithi107.stdout:6/533: dwrite d1/d2/db/d32/d36/d3a/fa7 [0,4194304] 0 2024-04-25T00:18:34.224 INFO:tasks.workunit.client.0.smithi107.stdout:6/534: chown d1/d2/dad 41727 1 2024-04-25T00:18:34.224 INFO:tasks.workunit.client.0.smithi107.stdout:6/535: read - d1/f9f zero size 2024-04-25T00:18:34.225 INFO:tasks.workunit.client.0.smithi107.stdout:6/536: chown d1/d2/db/d3c/d43/d96 340773 1 2024-04-25T00:18:34.225 INFO:tasks.workunit.client.0.smithi107.stdout:8/665: mkdir d2/d18/ddd 0 2024-04-25T00:18:34.225 INFO:tasks.workunit.client.0.smithi107.stdout:8/666: readlink d2/d18/d1a/d5e/d3e/d74/ldb 0 2024-04-25T00:18:34.226 INFO:tasks.workunit.client.0.smithi107.stdout:8/667: fdatasync d2/d18/d1a/d5e/f8c 0 2024-04-25T00:18:34.226 INFO:tasks.workunit.client.0.smithi107.stdout:7/610: truncate d0/d8/db/d18/d19/fb7 1817827 0 2024-04-25T00:18:34.226 INFO:tasks.workunit.client.0.smithi107.stdout:7/611: write d0/d8/d4e/f90 [1359763,39690] 0 2024-04-25T00:18:34.226 INFO:tasks.workunit.client.0.smithi107.stdout:7/612: chown d0/d8/d4e/d46/d93/fb0 17 1 2024-04-25T00:18:34.229 INFO:tasks.workunit.client.0.smithi107.stdout:7/613: creat d0/d80/fd2 x:0 0 0 2024-04-25T00:18:34.231 INFO:tasks.workunit.client.0.smithi107.stdout:3/615: dwrite d9/d54/d5c/d68/f3c [0,4194304] 0 2024-04-25T00:18:34.238 INFO:tasks.workunit.client.0.smithi107.stdout:3/616: dread d9/d88/d4e/d93/f9d [0,4194304] 0 2024-04-25T00:18:34.238 INFO:tasks.workunit.client.0.smithi107.stdout:3/617: chown d9/d54/d94/dab 2 1 2024-04-25T00:18:34.239 INFO:tasks.workunit.client.0.smithi107.stdout:3/618: stat d9/d88/c25 0 2024-04-25T00:18:34.239 INFO:tasks.workunit.client.0.smithi107.stdout:3/619: getdents d9/d88/d2c 0 2024-04-25T00:18:34.243 INFO:tasks.workunit.client.0.smithi107.stdout:6/537: dwrite d1/d2/d3/d71/d94/f9b [0,4194304] 0 2024-04-25T00:18:34.244 INFO:tasks.workunit.client.0.smithi107.stdout:6/538: creat d1/fb1 x:0 0 0 2024-04-25T00:18:34.251 INFO:tasks.workunit.client.0.smithi107.stdout:2/696: rename d6/dad/db0/fea to d6/d18/d40/d42/d69/d6e/da2/dc5/ff2 0 2024-04-25T00:18:34.253 INFO:tasks.workunit.client.0.smithi107.stdout:8/668: rename d2/d18/d1a/d5e/d3d/c84 to d2/d18/d1a/d5e/d5c/cde 0 2024-04-25T00:18:34.253 INFO:tasks.workunit.client.0.smithi107.stdout:8/669: truncate d2/d18/d1a/f80 1429990 0 2024-04-25T00:18:34.254 INFO:tasks.workunit.client.0.smithi107.stdout:2/697: creat d6/d18/d40/d42/d2b/d30/d45/dec/ff3 x:0 0 0 2024-04-25T00:18:34.255 INFO:tasks.workunit.client.0.smithi107.stdout:3/620: dwrite d9/d22/d1b/f81 [0,4194304] 0 2024-04-25T00:18:34.255 INFO:tasks.workunit.client.0.smithi107.stdout:7/614: rename d0/d8/db/d18/d19/d47/d4d/fa4 to d0/dc1/fd3 0 2024-04-25T00:18:34.255 INFO:tasks.workunit.client.0.smithi107.stdout:7/615: dread - d0/d8/db/d18/d19/d47/d4d/d8a/fb8 zero size 2024-04-25T00:18:34.256 INFO:tasks.workunit.client.0.smithi107.stdout:8/670: unlink d2/d18/f68 0 2024-04-25T00:18:34.257 INFO:tasks.workunit.client.0.smithi107.stdout:7/616: mkdir d0/d8/db/d18/d19/d47/da1/dd4 0 2024-04-25T00:18:34.259 INFO:tasks.workunit.client.0.smithi107.stdout:7/617: rmdir d0/d8/d4e/d46/d93 39 2024-04-25T00:18:34.259 INFO:tasks.workunit.client.0.smithi107.stdout:7/618: write d0/d8/d2c/d58/fb4 [192029,73099] 0 2024-04-25T00:18:34.259 INFO:tasks.workunit.client.0.smithi107.stdout:3/621: symlink d9/d54/d5c/lc6 0 2024-04-25T00:18:34.259 INFO:tasks.workunit.client.0.smithi107.stdout:3/622: readlink d9/d88/d2c/l87 0 2024-04-25T00:18:34.261 INFO:tasks.workunit.client.0.smithi107.stdout:8/671: creat d2/d18/ddd/fdf x:0 0 0 2024-04-25T00:18:34.261 INFO:tasks.workunit.client.0.smithi107.stdout:7/619: fdatasync d0/d8/d2c/d58/fb4 0 2024-04-25T00:18:34.261 INFO:tasks.workunit.client.0.smithi107.stdout:8/672: truncate d2/d18/d1a/d5e/d4f/da0/fac 219756 0 2024-04-25T00:18:34.261 INFO:tasks.workunit.client.0.smithi107.stdout:6/539: link d1/d14/d18/l80 d1/d2/d3/d26/lb2 0 2024-04-25T00:18:34.262 INFO:tasks.workunit.client.0.smithi107.stdout:3/623: mknod d9/d54/d5c/d68/db3/cc7 0 2024-04-25T00:18:34.262 INFO:tasks.workunit.client.0.smithi107.stdout:3/624: write d9/f10 [2881875,46787] 0 2024-04-25T00:18:34.262 INFO:tasks.workunit.client.0.smithi107.stdout:3/625: fsync d9/d88/d4e/d93/fc5 0 2024-04-25T00:18:34.263 INFO:tasks.workunit.client.0.smithi107.stdout:7/620: truncate d0/d8/db/f15 1960154 0 2024-04-25T00:18:34.263 INFO:tasks.workunit.client.0.smithi107.stdout:6/540: mknod d1/d2/db/d32/d33/d68/cb3 0 2024-04-25T00:18:34.263 INFO:tasks.workunit.client.0.smithi107.stdout:6/541: chown d1/d2/d2b/d3e/d5b/d7a/faf 72 1 2024-04-25T00:18:34.265 INFO:tasks.workunit.client.0.smithi107.stdout:3/626: symlink d9/d54/d94/dba/lc8 0 2024-04-25T00:18:34.265 INFO:tasks.workunit.client.0.smithi107.stdout:3/627: dread - d9/d22/d1b/fa9 zero size 2024-04-25T00:18:34.265 INFO:tasks.workunit.client.0.smithi107.stdout:7/621: mknod d0/d8/db/d18/d19/d47/da1/cd5 0 2024-04-25T00:18:34.269 INFO:tasks.workunit.client.0.smithi107.stdout:6/542: symlink d1/d14/d18/lb4 0 2024-04-25T00:18:34.270 INFO:tasks.workunit.client.0.smithi107.stdout:7/622: unlink d0/d8/db/d18/d19/d47/d4d/la5 0 2024-04-25T00:18:34.270 INFO:tasks.workunit.client.0.smithi107.stdout:7/623: write d0/d8/d2c/f3d [2685320,72698] 0 2024-04-25T00:18:34.275 INFO:tasks.workunit.client.0.smithi107.stdout:2/698: dwrite d6/d18/d40/d42/d69/d6e/da2/fba [0,4194304] 0 2024-04-25T00:18:34.275 INFO:tasks.workunit.client.0.smithi107.stdout:2/699: readlink d6/d20/d22/laf 0 2024-04-25T00:18:34.278 INFO:tasks.workunit.client.0.smithi107.stdout:2/700: mknod d6/d29/d4e/d8c/cf4 0 2024-04-25T00:18:34.280 INFO:tasks.workunit.client.0.smithi107.stdout:2/701: mknod d6/d18/d40/d42/d69/d6e/da2/dc5/dd2/cf5 0 2024-04-25T00:18:34.280 INFO:tasks.workunit.client.0.smithi107.stdout:2/702: fsync d6/d18/d40/d42/d69/d6e/da2/dc5/ff2 0 2024-04-25T00:18:34.280 INFO:tasks.workunit.client.0.smithi107.stdout:2/703: dread - d6/d18/d40/d42/d69/d6e/d80/f95 zero size 2024-04-25T00:18:34.282 INFO:tasks.workunit.client.0.smithi107.stdout:2/704: symlink d6/d29/d4e/d6f/lf6 0 2024-04-25T00:18:34.282 INFO:tasks.workunit.client.0.smithi107.stdout:8/673: dwrite d2/d18/d1a/dca/fcb [0,4194304] 0 2024-04-25T00:18:34.283 INFO:tasks.workunit.client.0.smithi107.stdout:2/705: rename d6/d18/d40/l84 to d6/d20/d77/lf7 0 2024-04-25T00:18:34.284 INFO:tasks.workunit.client.0.smithi107.stdout:8/674: symlink d2/d18/d1a/d5e/d3e/d74/le0 0 2024-04-25T00:18:34.285 INFO:tasks.workunit.client.0.smithi107.stdout:8/675: truncate d2/f55 1497498 0 2024-04-25T00:18:34.285 INFO:tasks.workunit.client.0.smithi107.stdout:6/543: write d1/d2/db/d32/d33/d68/d75/f8b [2892146,4040] 0 2024-04-25T00:18:34.287 INFO:tasks.workunit.client.0.smithi107.stdout:8/676: symlink d2/d18/d1a/d5e/d3e/le1 0 2024-04-25T00:18:34.287 INFO:tasks.workunit.client.0.smithi107.stdout:6/544: getdents d1/d2/d3/d28/d79/d6b 0 2024-04-25T00:18:34.288 INFO:tasks.workunit.client.0.smithi107.stdout:3/628: dwrite d9/d88/d2c/fbe [0,4194304] 0 2024-04-25T00:18:34.290 INFO:tasks.workunit.client.0.smithi107.stdout:3/629: getdents d9/d88/d3a/d91 0 2024-04-25T00:18:34.290 INFO:tasks.workunit.client.0.smithi107.stdout:3/630: stat d9/f9b 0 2024-04-25T00:18:34.292 INFO:tasks.workunit.client.0.smithi107.stdout:8/677: symlink d2/d18/ddd/le2 0 2024-04-25T00:18:34.293 INFO:tasks.workunit.client.0.smithi107.stdout:7/624: dwrite d0/d8/d2c/d58/f84 [0,4194304] 0 2024-04-25T00:18:34.294 INFO:tasks.workunit.client.0.smithi107.stdout:2/706: dread d6/d29/d4e/fab [0,4194304] 0 2024-04-25T00:18:34.297 INFO:tasks.workunit.client.0.smithi107.stdout:6/545: creat d1/d2/db/d3c/d43/d96/fb5 x:0 0 0 2024-04-25T00:18:34.297 INFO:tasks.workunit.client.0.smithi107.stdout:6/546: fdatasync d1/d2/db/d32/d36/f3f 0 2024-04-25T00:18:34.297 INFO:tasks.workunit.client.0.smithi107.stdout:8/678: mknod d2/d18/d1a/d5e/d3d/d85/dae/ce3 0 2024-04-25T00:18:34.297 INFO:tasks.workunit.client.0.smithi107.stdout:8/679: fsync d2/fb 0 2024-04-25T00:18:34.297 INFO:tasks.workunit.client.0.smithi107.stdout:8/680: chown d2/d18/d1a/c36 294139 1 2024-04-25T00:18:34.299 INFO:tasks.workunit.client.0.smithi107.stdout:8/681: mknod d2/ce4 0 2024-04-25T00:18:34.300 INFO:tasks.workunit.client.0.smithi107.stdout:7/625: mknod d0/d8/db/d92/cd6 0 2024-04-25T00:18:34.300 INFO:tasks.workunit.client.0.smithi107.stdout:7/626: chown d0/d8/db/d18/f41 14398835 1 2024-04-25T00:18:34.300 INFO:tasks.workunit.client.0.smithi107.stdout:7/627: getdents d0/d8/d4e/d46 0 2024-04-25T00:18:34.300 INFO:tasks.workunit.client.0.smithi107.stdout:6/547: mknod d1/d2/db/d32/d33/d68/d75/cb6 0 2024-04-25T00:18:34.301 INFO:tasks.workunit.client.0.smithi107.stdout:6/548: truncate d1/d2/f5f 4291368 0 2024-04-25T00:18:34.302 INFO:tasks.workunit.client.0.smithi107.stdout:8/682: mknod d2/d18/d1a/d5e/d3d/d85/ce5 0 2024-04-25T00:18:34.308 INFO:tasks.workunit.client.0.smithi107.stdout:6/549: rename d1/d2/db/d48/fae to d1/d2/db/d32/fb7 0 2024-04-25T00:18:34.309 INFO:tasks.workunit.client.0.smithi107.stdout:6/550: dread d1/d2/db/d32/d36/f46 [0,4194304] 0 2024-04-25T00:18:34.309 INFO:tasks.workunit.client.0.smithi107.stdout:6/551: read d1/d2/d3/d26/fa2 [4060008,32681] 0 2024-04-25T00:18:34.309 INFO:tasks.workunit.client.0.smithi107.stdout:6/552: write d1/d2/d3/d28/d79/d6b/f6f [2535544,121067] 0 2024-04-25T00:18:34.310 INFO:tasks.workunit.client.0.smithi107.stdout:7/628: dread d0/d8/db/d18/d19/d39/f6f [0,4194304] 0 2024-04-25T00:18:34.311 INFO:tasks.workunit.client.0.smithi107.stdout:3/631: dwrite d9/d54/fb4 [4194304,4194304] 0 2024-04-25T00:18:34.311 INFO:tasks.workunit.client.0.smithi107.stdout:6/553: rename d1/d2/db/f34 to d1/d2/d3/d28/d79/fb8 0 2024-04-25T00:18:34.311 INFO:tasks.workunit.client.0.smithi107.stdout:6/554: stat d1/c30 0 2024-04-25T00:18:34.313 INFO:tasks.workunit.client.0.smithi107.stdout:6/555: mknod d1/d2/db/d3c/d43/cb9 0 2024-04-25T00:18:34.322 INFO:tasks.workunit.client.0.smithi107.stdout:8/683: dwrite d2/d18/d1a/d5e/d5c/f88 [0,4194304] 0 2024-04-25T00:18:34.324 INFO:tasks.workunit.client.0.smithi107.stdout:3/632: dread d9/f20 [0,4194304] 0 2024-04-25T00:18:34.335 INFO:tasks.workunit.client.0.smithi107.stdout:7/629: dwrite d0/d8/d2c/d58/fb4 [0,4194304] 0 2024-04-25T00:18:34.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:34 smithi195 ceph-mon[32833]: pgmap v66: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 147 MiB/s rd, 176 MiB/s wr, 326 op/s 2024-04-25T00:18:34.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:34 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:18:34.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:34 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:18:34.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:34 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:34.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:34 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:18:34.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:34 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:18:34.339 INFO:tasks.workunit.client.0.smithi107.stdout:7/630: rename d0/d8/d2c/f3d to d0/d8/db/d18/d19/d47/fd7 0 2024-04-25T00:18:34.340 INFO:tasks.workunit.client.0.smithi107.stdout:7/631: readlink d0/d8/db/d18/d19/d67/d7a/dc2/dd0/lcb 0 2024-04-25T00:18:34.340 INFO:tasks.workunit.client.0.smithi107.stdout:6/556: dwrite d1/d2/db/f19 [4194304,4194304] 0 2024-04-25T00:18:34.348 INFO:tasks.workunit.client.0.smithi107.stdout:3/633: dwrite d9/d54/fc4 [0,4194304] 0 2024-04-25T00:18:34.349 INFO:tasks.workunit.client.0.smithi107.stdout:8/684: dwrite d2/d18/d1a/fdc [0,4194304] 0 2024-04-25T00:18:34.349 INFO:tasks.workunit.client.0.smithi107.stdout:8/685: chown d2/d18/d1a/d5e/d4f 13619 1 2024-04-25T00:18:34.349 INFO:tasks.workunit.client.0.smithi107.stdout:8/686: fsync d2/f41 0 2024-04-25T00:18:34.349 INFO:tasks.workunit.client.0.smithi107.stdout:7/632: dread d0/d8/d4e/fa0 [0,4194304] 0 2024-04-25T00:18:34.349 INFO:tasks.workunit.client.0.smithi107.stdout:8/687: dread - d2/d18/d1a/d5e/d3e/db1/fd0 zero size 2024-04-25T00:18:34.349 INFO:tasks.workunit.client.0.smithi107.stdout:8/688: chown d2/d18/d1a/d5e/d3e/f86 62686 1 2024-04-25T00:18:34.349 INFO:tasks.workunit.client.0.smithi107.stdout:7/633: write d0/d80/fd2 [577811,53853] 0 2024-04-25T00:18:34.349 INFO:tasks.workunit.client.0.smithi107.stdout:7/634: fdatasync d0/d8/db/d18/d19/d67/d7a/d82/fb6 0 2024-04-25T00:18:34.350 INFO:tasks.workunit.client.0.smithi107.stdout:7/635: getdents d0/d8/db/d18/d19/d47/d4d/d8a 0 2024-04-25T00:18:34.351 INFO:tasks.workunit.client.0.smithi107.stdout:3/634: mkdir d9/d54/d94/dc9 0 2024-04-25T00:18:34.351 INFO:tasks.workunit.client.0.smithi107.stdout:3/635: chown d9/d22/d1b/la2 46474 1 2024-04-25T00:18:34.352 INFO:tasks.workunit.client.0.smithi107.stdout:7/636: mkdir d0/d8/d4e/d46/d93/dd8 0 2024-04-25T00:18:34.352 INFO:tasks.workunit.client.0.smithi107.stdout:7/637: chown d0/f1d 1 1 2024-04-25T00:18:34.352 INFO:tasks.workunit.client.0.smithi107.stdout:3/636: creat d9/d88/d2c/fca x:0 0 0 2024-04-25T00:18:34.352 INFO:tasks.workunit.client.0.smithi107.stdout:3/637: chown d9/d15/da1/fa8 580021432 1 2024-04-25T00:18:34.353 INFO:tasks.workunit.client.0.smithi107.stdout:8/689: mkdir d2/d18/d1a/d5e/dd6/de6 0 2024-04-25T00:18:34.354 INFO:tasks.workunit.client.0.smithi107.stdout:8/690: write d2/d18/d1a/d5e/d6d/f6f [5042624,101856] 0 2024-04-25T00:18:34.355 INFO:tasks.workunit.client.0.smithi107.stdout:7/638: link d0/d8/db/d18/d19/d47/d4d/d8a/fb8 d0/d8/db/d18/d19/d67/d7a/d82/fd9 0 2024-04-25T00:18:34.355 INFO:tasks.workunit.client.0.smithi107.stdout:6/557: write d1/d2/f5f [1211278,41350] 0 2024-04-25T00:18:34.355 INFO:tasks.workunit.client.0.smithi107.stdout:3/638: mkdir d9/d54/d94/dc9/dcb 0 2024-04-25T00:18:34.356 INFO:tasks.workunit.client.0.smithi107.stdout:8/691: mkdir d2/d18/d1a/d5e/d5c/d91/de7 0 2024-04-25T00:18:34.356 INFO:tasks.workunit.client.0.smithi107.stdout:8/692: write d2/d18/da2/fd9 [592903,8535] 0 2024-04-25T00:18:34.357 INFO:tasks.workunit.client.0.smithi107.stdout:3/639: truncate d9/d22/f1e 3144793 0 2024-04-25T00:18:34.358 INFO:tasks.workunit.client.0.smithi107.stdout:3/640: mkdir d9/d54/d94/dba/dcc 0 2024-04-25T00:18:34.359 INFO:tasks.workunit.client.0.smithi107.stdout:7/639: dread d0/f13 [0,4194304] 0 2024-04-25T00:18:34.360 INFO:tasks.workunit.client.0.smithi107.stdout:3/641: creat d9/d15/da1/d4b/fcd x:0 0 0 2024-04-25T00:18:34.360 INFO:tasks.workunit.client.0.smithi107.stdout:3/642: chown d9/d88/d4e/d93/f9d 0 1 2024-04-25T00:18:34.364 INFO:tasks.workunit.client.0.smithi107.stdout:6/558: dread d1/d2/d3/d71/d94/f9b [0,4194304] 0 2024-04-25T00:18:34.366 INFO:tasks.workunit.client.0.smithi107.stdout:8/693: dread d2/f55 [0,4194304] 0 2024-04-25T00:18:34.370 INFO:tasks.workunit.client.0.smithi107.stdout:7/640: dread d0/d8/f65 [0,4194304] 0 2024-04-25T00:18:34.372 INFO:tasks.workunit.client.0.smithi107.stdout:8/694: rmdir d2/d18/d1a/d5e/d4f/da0/da1 39 2024-04-25T00:18:34.372 INFO:tasks.workunit.client.0.smithi107.stdout:7/641: link d0/d8/la d0/lda 0 2024-04-25T00:18:34.382 INFO:tasks.workunit.client.0.smithi107.stdout:3/643: dwrite d9/f20 [0,4194304] 0 2024-04-25T00:18:34.382 INFO:tasks.workunit.client.0.smithi107.stdout:8/695: dread d2/d18/d1a/d5e/d3e/f4c [0,4194304] 0 2024-04-25T00:18:34.382 INFO:tasks.workunit.client.0.smithi107.stdout:8/696: truncate d2/d18/d1a/d5e/f47 1694071 0 2024-04-25T00:18:34.382 INFO:tasks.workunit.client.0.smithi107.stdout:8/697: fdatasync d2/d18/d1a/d5e/d3d/fb3 0 2024-04-25T00:18:34.382 INFO:tasks.workunit.client.0.smithi107.stdout:8/698: read d2/fc6 [3431110,119560] 0 2024-04-25T00:18:34.383 INFO:tasks.workunit.client.0.smithi107.stdout:7/642: getdents d0 0 2024-04-25T00:18:34.383 INFO:tasks.workunit.client.0.smithi107.stdout:7/643: rename d0/d8/db/d18 to d0/d8/db/d18/d19/d67/ddb 22 2024-04-25T00:18:34.384 INFO:tasks.workunit.client.0.smithi107.stdout:6/559: dwrite d1/d4c/fac [0,4194304] 0 2024-04-25T00:18:34.384 INFO:tasks.workunit.client.0.smithi107.stdout:6/560: fsync d1/d2/d2b/d3e/f8f 0 2024-04-25T00:18:34.384 INFO:tasks.workunit.client.0.smithi107.stdout:6/561: fdatasync d1/d4c/fac 0 2024-04-25T00:18:34.384 INFO:tasks.workunit.client.0.smithi107.stdout:7/644: symlink d0/d8/db/d18/d19/d47/d4d/ldc 0 2024-04-25T00:18:34.386 INFO:tasks.workunit.client.0.smithi107.stdout:6/562: symlink d1/d2/d3/d28/d79/lba 0 2024-04-25T00:18:34.387 INFO:tasks.workunit.client.0.smithi107.stdout:8/699: mkdir d2/d18/d1a/d5e/d4f/da0/da1/de8 0 2024-04-25T00:18:34.388 INFO:tasks.workunit.client.0.smithi107.stdout:8/700: creat d2/d18/d1a/d5e/dd6/de6/fe9 x:0 0 0 2024-04-25T00:18:34.388 INFO:tasks.workunit.client.0.smithi107.stdout:8/701: truncate d2/d18/d1a/d82/fb6 931947 0 2024-04-25T00:18:34.388 INFO:tasks.workunit.client.0.smithi107.stdout:8/702: readlink d2/d18/da2/lda 0 2024-04-25T00:18:34.389 INFO:tasks.workunit.client.0.smithi107.stdout:8/703: symlink d2/d18/d1a/d5e/dd6/lea 0 2024-04-25T00:18:34.390 INFO:tasks.workunit.client.0.smithi107.stdout:0/579: sync 2024-04-25T00:18:34.390 INFO:tasks.workunit.client.0.smithi107.stdout:5/686: sync 2024-04-25T00:18:34.390 INFO:tasks.workunit.client.0.smithi107.stdout:8/704: getdents d2/d18/ddd 0 2024-04-25T00:18:34.390 INFO:tasks.workunit.client.0.smithi107.stdout:0/580: chown dc/dd/d16/d33/d35/d5f/l79 193883260 1 2024-04-25T00:18:34.391 INFO:tasks.workunit.client.0.smithi107.stdout:0/581: creat dc/d24/d25/d31/fb5 x:0 0 0 2024-04-25T00:18:34.395 INFO:tasks.workunit.client.0.smithi107.stdout:7/645: read d0/d8/db/d18/d19/d47/f62 [3670132,11141] 0 2024-04-25T00:18:34.397 INFO:tasks.workunit.client.0.smithi107.stdout:6/563: dread d1/d2/db/d3c/f50 [0,4194304] 0 2024-04-25T00:18:34.399 INFO:tasks.workunit.client.0.smithi107.stdout:6/564: creat d1/d2/db/d48/fbb x:0 0 0 2024-04-25T00:18:34.400 INFO:tasks.workunit.client.0.smithi107.stdout:6/565: creat d1/d2/d3/fbc x:0 0 0 2024-04-25T00:18:34.402 INFO:tasks.workunit.client.0.smithi107.stdout:6/566: mknod d1/d2/db/d3c/cbd 0 2024-04-25T00:18:34.402 INFO:tasks.workunit.client.0.smithi107.stdout:3/644: dwrite d9/d88/d2c/f75 [0,4194304] 0 2024-04-25T00:18:34.406 INFO:tasks.workunit.client.0.smithi107.stdout:3/645: creat d9/d54/d5c/fce x:0 0 0 2024-04-25T00:18:34.406 INFO:tasks.workunit.client.0.smithi107.stdout:3/646: write d9/f9b [848311,101407] 0 2024-04-25T00:18:34.410 INFO:tasks.workunit.client.0.smithi107.stdout:5/687: unlink d4/d17/db3/d84/d93/da4/fd0 0 2024-04-25T00:18:34.410 INFO:tasks.workunit.client.0.smithi107.stdout:5/688: write d4/dc/d26/d42/d8b/fa9 [537056,34143] 0 2024-04-25T00:18:34.411 INFO:tasks.workunit.client.0.smithi107.stdout:6/567: dread d1/d2/d2b/d3e/d5b/d7a/f95 [0,4194304] 0 2024-04-25T00:18:34.412 INFO:tasks.workunit.client.0.smithi107.stdout:5/689: truncate d4/f1b 1217026 0 2024-04-25T00:18:34.414 INFO:tasks.workunit.client.0.smithi107.stdout:0/582: dread dc/dd/f4c [0,4194304] 0 2024-04-25T00:18:34.414 INFO:tasks.workunit.client.0.smithi107.stdout:5/690: symlink d4/dc/d26/d29/d2e/ldf 0 2024-04-25T00:18:34.416 INFO:tasks.workunit.client.0.smithi107.stdout:8/705: dwrite d2/d18/d1a/f1e [0,4194304] 0 2024-04-25T00:18:34.417 INFO:tasks.workunit.client.0.smithi107.stdout:0/583: creat dc/dd/d16/d33/fb6 x:0 0 0 2024-04-25T00:18:34.417 INFO:tasks.workunit.client.0.smithi107.stdout:0/584: chown dc/d24/d25/l32 1806856 1 2024-04-25T00:18:34.418 INFO:tasks.workunit.client.0.smithi107.stdout:8/706: mknod d2/d18/d1a/d5e/dd6/de6/ceb 0 2024-04-25T00:18:34.419 INFO:tasks.workunit.client.0.smithi107.stdout:7/646: dwrite d0/d8/db/d18/d19/d67/d7a/d82/fb6 [0,4194304] 0 2024-04-25T00:18:34.419 INFO:tasks.workunit.client.0.smithi107.stdout:7/647: truncate d0/d8/db/d18/d19/d67/d7a/d82/fd9 31638 0 2024-04-25T00:18:34.421 INFO:tasks.workunit.client.0.smithi107.stdout:8/707: rename d2/d18/d1a/d33/f3f to d2/d18/ddd/fec 0 2024-04-25T00:18:34.421 INFO:tasks.workunit.client.0.smithi107.stdout:8/708: write d2/d18/d1a/d5e/d3e/d74/f97 [1422784,70533] 0 2024-04-25T00:18:34.423 INFO:tasks.workunit.client.0.smithi107.stdout:7/648: symlink d0/d8/db/d18/d19/d67/ldd 0 2024-04-25T00:18:34.429 INFO:tasks.workunit.client.0.smithi107.stdout:8/709: dread d2/f63 [0,4194304] 0 2024-04-25T00:18:34.430 INFO:tasks.workunit.client.0.smithi107.stdout:8/710: symlink d2/d18/d1a/d5e/d5c/d91/da3/led 0 2024-04-25T00:18:34.431 INFO:tasks.workunit.client.0.smithi107.stdout:8/711: creat d2/d18/d1a/d5e/d3d/d85/dae/fee x:0 0 0 2024-04-25T00:18:34.432 INFO:tasks.workunit.client.0.smithi107.stdout:0/585: dread dc/d24/d38/d64/faa [0,4194304] 0 2024-04-25T00:18:34.432 INFO:tasks.workunit.client.0.smithi107.stdout:0/586: dread - dc/dd/d16/d33/f9a zero size 2024-04-25T00:18:34.433 INFO:tasks.workunit.client.0.smithi107.stdout:0/587: rmdir dc/dd/d16/d48/d83 39 2024-04-25T00:18:34.434 INFO:tasks.workunit.client.0.smithi107.stdout:0/588: rmdir dc/d24/d38/d64/d78 39 2024-04-25T00:18:34.434 INFO:tasks.workunit.client.0.smithi107.stdout:0/589: write dc/dd/d5d/f70 [157261,36477] 0 2024-04-25T00:18:34.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:34 smithi170 ceph-mon[32807]: pgmap v66: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 147 MiB/s rd, 176 MiB/s wr, 326 op/s 2024-04-25T00:18:34.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:34 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:18:34.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:34 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:18:34.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:34 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:34.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:34 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:18:34.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:34 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:18:34.440 INFO:tasks.workunit.client.0.smithi107.stdout:6/568: dwrite d1/d2/d3/d26/fa2 [0,4194304] 0 2024-04-25T00:18:34.443 INFO:tasks.workunit.client.0.smithi107.stdout:3/647: dwrite d9/d88/d4e/f72 [0,4194304] 0 2024-04-25T00:18:34.443 INFO:tasks.workunit.client.0.smithi107.stdout:7/649: dwrite d0/d8/db/d18/f41 [0,4194304] 0 2024-04-25T00:18:34.443 INFO:tasks.workunit.client.0.smithi107.stdout:8/712: write d2/d18/d1a/f6c [3063220,114981] 0 2024-04-25T00:18:34.445 INFO:tasks.workunit.client.0.smithi107.stdout:3/648: link d9/d22/f7a d9/d22/d1b/d55/fcf 0 2024-04-25T00:18:34.445 INFO:tasks.workunit.client.0.smithi107.stdout:3/649: dread - d9/d88/d2c/fca zero size 2024-04-25T00:18:34.445 INFO:tasks.workunit.client.0.smithi107.stdout:8/713: symlink d2/d18/d1a/d5e/d5c/lef 0 2024-04-25T00:18:34.447 INFO:tasks.workunit.client.0.smithi107.stdout:3/650: rename d9/d54/d5c/d68/db3/db9 to d9/d15/da1/d4b/dd0 0 2024-04-25T00:18:34.448 INFO:tasks.workunit.client.0.smithi107.stdout:8/714: mkdir d2/d18/d1a/d5e/d3e/db1/df0 0 2024-04-25T00:18:34.448 INFO:tasks.workunit.client.0.smithi107.stdout:8/715: chown d2/l19 0 1 2024-04-25T00:18:34.448 INFO:tasks.workunit.client.0.smithi107.stdout:3/651: getdents d9/d54/d94/dc9 0 2024-04-25T00:18:34.451 INFO:tasks.workunit.client.0.smithi107.stdout:3/652: write d9/f14 [140556,129329] 0 2024-04-25T00:18:34.455 INFO:tasks.workunit.client.0.smithi107.stdout:6/569: dread d1/d2/db/fd [4194304,4194304] 0 2024-04-25T00:18:34.457 INFO:tasks.workunit.client.0.smithi107.stdout:7/650: dread d0/d8/d2c/d58/fb4 [0,4194304] 0 2024-04-25T00:18:34.457 INFO:tasks.workunit.client.0.smithi107.stdout:8/716: dread d2/d18/d1a/d33/f64 [0,4194304] 0 2024-04-25T00:18:34.457 INFO:tasks.workunit.client.0.smithi107.stdout:7/651: fdatasync d0/d8/db/d18/f61 0 2024-04-25T00:18:34.458 INFO:tasks.workunit.client.0.smithi107.stdout:7/652: mknod d0/d8/db/d18/d19/d67/d7a/dc2/cde 0 2024-04-25T00:18:34.458 INFO:tasks.workunit.client.0.smithi107.stdout:7/653: write d0/d8/d4e/f87 [528517,19451] 0 2024-04-25T00:18:34.459 INFO:tasks.workunit.client.0.smithi107.stdout:7/654: unlink d0/d8/db/d18/d19/d47/c6e 0 2024-04-25T00:18:34.465 INFO:tasks.workunit.client.0.smithi107.stdout:0/590: dread dc/dd/d5d/d5c/f71 [0,4194304] 0 2024-04-25T00:18:34.465 INFO:tasks.workunit.client.0.smithi107.stdout:0/591: chown l3 2 1 2024-04-25T00:18:34.465 INFO:tasks.workunit.client.0.smithi107.stdout:0/592: read dc/dd/d16/d33/f37 [3976681,123104] 0 2024-04-25T00:18:34.465 INFO:tasks.workunit.client.0.smithi107.stdout:0/593: truncate dc/dd/d16/f60 287375 0 2024-04-25T00:18:34.469 INFO:tasks.workunit.client.0.smithi107.stdout:6/570: dread d1/d4c/fa6 [0,4194304] 0 2024-04-25T00:18:34.469 INFO:tasks.workunit.client.0.smithi107.stdout:0/594: write dc/dd/d16/f1b [4187470,67185] 0 2024-04-25T00:18:34.469 INFO:tasks.workunit.client.0.smithi107.stdout:0/595: chown dc/dd/d16/d33/d35/d5f 28 1 2024-04-25T00:18:34.469 INFO:tasks.workunit.client.0.smithi107.stdout:0/596: dread - dc/d24/d38/d64/d7b/fae zero size 2024-04-25T00:18:34.469 INFO:tasks.workunit.client.0.smithi107.stdout:6/571: rmdir d1/d2/db/d32/d36/d3d 39 2024-04-25T00:18:34.470 INFO:tasks.workunit.client.0.smithi107.stdout:6/572: getdents d1/d2/db/d3c/d43/d96 0 2024-04-25T00:18:34.471 INFO:tasks.workunit.client.0.smithi107.stdout:3/653: dwrite d9/f5e [0,4194304] 0 2024-04-25T00:18:34.472 INFO:tasks.workunit.client.0.smithi107.stdout:3/654: symlink d9/d15/ld1 0 2024-04-25T00:18:34.474 INFO:tasks.workunit.client.0.smithi107.stdout:3/655: mknod d9/d88/d4e/d93/cd2 0 2024-04-25T00:18:34.477 INFO:tasks.workunit.client.0.smithi107.stdout:8/717: dwrite d2/d18/f53 [0,4194304] 0 2024-04-25T00:18:34.479 INFO:tasks.workunit.client.0.smithi107.stdout:8/718: creat d2/d18/d1a/d5e/d4f/da0/ff1 x:0 0 0 2024-04-25T00:18:34.481 INFO:tasks.workunit.client.0.smithi107.stdout:8/719: creat d2/d18/d1a/d5e/d3d/d85/dd5/ff2 x:0 0 0 2024-04-25T00:18:34.482 INFO:tasks.workunit.client.0.smithi107.stdout:7/655: dwrite d0/d8/db/d18/d19/d39/f6f [4194304,4194304] 0 2024-04-25T00:18:34.482 INFO:tasks.workunit.client.0.smithi107.stdout:6/573: getdents d1/d4c 0 2024-04-25T00:18:34.482 INFO:tasks.workunit.client.0.smithi107.stdout:7/656: write d0/d8/db/d18/d19/f23 [562283,47710] 0 2024-04-25T00:18:34.483 INFO:tasks.workunit.client.0.smithi107.stdout:7/657: truncate d0/d8/db/d18/d19/f8e 504922 0 2024-04-25T00:18:34.483 INFO:tasks.workunit.client.0.smithi107.stdout:8/720: symlink d2/d18/d1a/d5e/d3e/d74/lf3 0 2024-04-25T00:18:34.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:34 smithi107 ceph-mon[29153]: pgmap v66: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 147 MiB/s rd, 176 MiB/s wr, 326 op/s 2024-04-25T00:18:34.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:34 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:18:34.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:34 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:18:34.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:34 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:34.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:34 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:18:34.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:34 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:18:34.484 INFO:tasks.workunit.client.0.smithi107.stdout:7/658: symlink d0/d8/db/d18/d19/d67/d7a/d82/ldf 0 2024-04-25T00:18:34.485 INFO:tasks.workunit.client.0.smithi107.stdout:8/721: rmdir d2/d18/d1a/d5e/d3d/d85/dae 39 2024-04-25T00:18:34.491 INFO:tasks.workunit.client.0.smithi107.stdout:0/597: dwrite dc/d24/d25/f80 [0,4194304] 0 2024-04-25T00:18:34.494 INFO:tasks.workunit.client.0.smithi107.stdout:0/598: stat dc/dd/d5d/c29 0 2024-04-25T00:18:34.497 INFO:tasks.workunit.client.0.smithi107.stdout:0/599: creat dc/d24/d38/d64/d7b/fb7 x:0 0 0 2024-04-25T00:18:34.498 INFO:tasks.workunit.client.0.smithi107.stdout:3/656: dwrite d9/f29 [4194304,4194304] 0 2024-04-25T00:18:34.499 INFO:tasks.workunit.client.0.smithi107.stdout:3/657: fdatasync d9/d88/d4e/f70 0 2024-04-25T00:18:34.499 INFO:tasks.workunit.client.0.smithi107.stdout:3/658: mknod d9/d88/d2c/cd3 0 2024-04-25T00:18:34.500 INFO:tasks.workunit.client.0.smithi107.stdout:0/600: unlink dc/dd/d16/d33/d35/f8d 0 2024-04-25T00:18:34.503 INFO:tasks.workunit.client.0.smithi107.stdout:7/659: rename d0/d8/db/d18/d19/d67/d7a to d0/d8/db/d18/d19/d47/de0 0 2024-04-25T00:18:34.508 INFO:tasks.workunit.client.0.smithi107.stdout:6/574: dwrite d1/d2/db/d32/d33/d4f/f93 [0,4194304] 0 2024-04-25T00:18:34.509 INFO:tasks.workunit.client.0.smithi107.stdout:7/660: creat d0/d8/d4e/d46/d93/fe1 x:0 0 0 2024-04-25T00:18:34.510 INFO:tasks.workunit.client.0.smithi107.stdout:0/601: symlink dc/dd/d5d/lb8 0 2024-04-25T00:18:34.512 INFO:tasks.workunit.client.0.smithi107.stdout:7/661: symlink d0/d8/db/d18/le2 0 2024-04-25T00:18:34.512 INFO:tasks.workunit.client.0.smithi107.stdout:7/662: chown d0/l48 31235899 1 2024-04-25T00:18:34.512 INFO:tasks.workunit.client.0.smithi107.stdout:7/663: stat d0/d8/db/d18 0 2024-04-25T00:18:34.513 INFO:tasks.workunit.client.0.smithi107.stdout:6/575: truncate d1/d2/db/d32/d36/f46 395147 0 2024-04-25T00:18:34.513 INFO:tasks.workunit.client.0.smithi107.stdout:6/576: readlink d1/d14/d18/l2d 0 2024-04-25T00:18:34.516 INFO:tasks.workunit.client.0.smithi107.stdout:0/602: creat dc/dd/d16/d33/fb9 x:0 0 0 2024-04-25T00:18:34.516 INFO:tasks.workunit.client.0.smithi107.stdout:0/603: truncate dc/d24/d38/d64/d7b/fae 904397 0 2024-04-25T00:18:34.517 INFO:tasks.workunit.client.0.smithi107.stdout:7/664: link d0/d8/db/c7b d0/d80/da2/ce3 0 2024-04-25T00:18:34.519 INFO:tasks.workunit.client.0.smithi107.stdout:0/604: rmdir dc/d24/d38 39 2024-04-25T00:18:34.519 INFO:tasks.workunit.client.0.smithi107.stdout:0/605: chown dc/dd/d16/d33/d35/c47 502 1 2024-04-25T00:18:34.519 INFO:tasks.workunit.client.0.smithi107.stdout:0/606: fsync dc/dd/d16/d5e/fb1 0 2024-04-25T00:18:34.519 INFO:tasks.workunit.client.0.smithi107.stdout:8/722: dwrite d2/d18/d1a/d5e/d3e/f99 [0,4194304] 0 2024-04-25T00:18:34.520 INFO:tasks.workunit.client.0.smithi107.stdout:8/723: symlink d2/d18/da2/dc2/lf4 0 2024-04-25T00:18:34.521 INFO:tasks.workunit.client.0.smithi107.stdout:8/724: truncate d2/f16 547042 0 2024-04-25T00:18:34.523 INFO:tasks.workunit.client.0.smithi107.stdout:3/659: dwrite d9/f42 [0,4194304] 0 2024-04-25T00:18:34.524 INFO:tasks.workunit.client.0.smithi107.stdout:3/660: truncate d9/d88/d4e/f7b 137820 0 2024-04-25T00:18:34.524 INFO:tasks.workunit.client.0.smithi107.stdout:3/661: chown d9/d88/d4e/d93 5799825 1 2024-04-25T00:18:34.524 INFO:tasks.workunit.client.0.smithi107.stdout:8/725: mkdir d2/d18/d1a/d5e/d5c/df5 0 2024-04-25T00:18:34.524 INFO:tasks.workunit.client.0.smithi107.stdout:8/726: write d2/d18/d1a/d5e/d3e/db1/fd0 [793233,127975] 0 2024-04-25T00:18:34.526 INFO:tasks.workunit.client.0.smithi107.stdout:8/727: symlink d2/d18/d1a/dca/lf6 0 2024-04-25T00:18:34.530 INFO:tasks.workunit.client.0.smithi107.stdout:8/728: read d2/f55 [870379,83331] 0 2024-04-25T00:18:34.532 INFO:tasks.workunit.client.0.smithi107.stdout:8/729: rmdir d2/d18/da2/db2 39 2024-04-25T00:18:34.535 INFO:tasks.workunit.client.0.smithi107.stdout:8/730: link d2/d18/da2/db2/cc5 d2/d18/d1a/cf7 0 2024-04-25T00:18:34.535 INFO:tasks.workunit.client.0.smithi107.stdout:7/665: dwrite d0/d8/db/d18/d19/d47/de0/d82/fd9 [0,4194304] 0 2024-04-25T00:18:34.538 INFO:tasks.workunit.client.0.smithi107.stdout:0/607: dwrite dc/dd/d16/d33/f89 [0,4194304] 0 2024-04-25T00:18:34.538 INFO:tasks.workunit.client.0.smithi107.stdout:0/608: truncate dc/dd/d16/d33/fb6 783718 0 2024-04-25T00:18:34.538 INFO:tasks.workunit.client.0.smithi107.stdout:0/609: stat dc/d24/c8e 0 2024-04-25T00:18:34.538 INFO:tasks.workunit.client.0.smithi107.stdout:7/666: write d0/d8/f4a [246482,73219] 0 2024-04-25T00:18:34.539 INFO:tasks.workunit.client.0.smithi107.stdout:0/610: getdents dc/dd/d5d 0 2024-04-25T00:18:34.541 INFO:tasks.workunit.client.0.smithi107.stdout:7/667: getdents d0/d80 0 2024-04-25T00:18:34.542 INFO:tasks.workunit.client.0.smithi107.stdout:8/731: dread d2/d18/d1a/f7d [0,4194304] 0 2024-04-25T00:18:34.543 INFO:tasks.workunit.client.0.smithi107.stdout:6/577: rename d1/d14/d18/c54 to d1/d2/db/d32/d33/d4f/cbe 0 2024-04-25T00:18:34.544 INFO:tasks.workunit.client.0.smithi107.stdout:6/578: fsync d1/d2/f5f 0 2024-04-25T00:18:34.546 INFO:tasks.workunit.client.0.smithi107.stdout:3/662: rename d9/d88/d2c/l97 to d9/d88/d3a/ld4 0 2024-04-25T00:18:34.546 INFO:tasks.workunit.client.0.smithi107.stdout:3/663: truncate d9/d88/d3a/d91/f9a 885770 0 2024-04-25T00:18:34.555 INFO:tasks.workunit.client.0.smithi107.stdout:3/664: link d9/d88/d4e/la3 d9/d54/ld5 0 2024-04-25T00:18:34.562 INFO:tasks.workunit.client.0.smithi107.stdout:9/697: sync 2024-04-25T00:18:34.564 INFO:tasks.workunit.client.0.smithi107.stdout:9/698: mkdir d0/d8a/d93/d34/d36/d77/dac/dee 0 2024-04-25T00:18:34.566 INFO:tasks.workunit.client.0.smithi107.stdout:0/611: dwrite dc/f28 [0,4194304] 0 2024-04-25T00:18:34.566 INFO:tasks.workunit.client.0.smithi107.stdout:7/668: dwrite d0/f69 [0,4194304] 0 2024-04-25T00:18:34.568 INFO:tasks.workunit.client.0.smithi107.stdout:0/612: mkdir dc/dd/d16/d48/d83/d9f/dba 0 2024-04-25T00:18:34.571 INFO:tasks.workunit.client.0.smithi107.stdout:0/613: symlink dc/dd/d16/d48/d83/d9f/db0/lbb 0 2024-04-25T00:18:34.571 INFO:tasks.workunit.client.0.smithi107.stdout:0/614: dread - dc/dd/d16/d33/fb9 zero size 2024-04-25T00:18:34.571 INFO:tasks.workunit.client.0.smithi107.stdout:0/615: write dc/dd/d5d/d3f/f92 [477744,4513] 0 2024-04-25T00:18:34.571 INFO:tasks.workunit.client.0.smithi107.stdout:0/616: write dc/dd/d5d/f70 [692016,98761] 0 2024-04-25T00:18:34.571 INFO:tasks.workunit.client.0.smithi107.stdout:0/617: dread - dc/dd/d5d/f7d zero size 2024-04-25T00:18:34.574 INFO:tasks.workunit.client.0.smithi107.stdout:6/579: dwrite d1/d2/db/f19 [4194304,4194304] 0 2024-04-25T00:18:34.575 INFO:tasks.workunit.client.0.smithi107.stdout:8/732: dwrite d2/f83 [0,4194304] 0 2024-04-25T00:18:34.576 INFO:tasks.workunit.client.0.smithi107.stdout:9/699: dread d0/d8a/d93/d56/f64 [0,4194304] 0 2024-04-25T00:18:34.576 INFO:tasks.workunit.client.0.smithi107.stdout:0/618: dread dc/dd/d16/d33/fb6 [0,4194304] 0 2024-04-25T00:18:34.577 INFO:tasks.workunit.client.0.smithi107.stdout:8/733: link d2/d18/d1a/d5e/d3e/le1 d2/d18/d1a/d5e/lf8 0 2024-04-25T00:18:34.577 INFO:tasks.workunit.client.0.smithi107.stdout:8/734: fsync d2/d18/d1a/d33/f64 0 2024-04-25T00:18:34.580 INFO:tasks.workunit.client.0.smithi107.stdout:8/735: rename d2/d18/d1a/l4e to d2/d18/da2/lf9 0 2024-04-25T00:18:34.580 INFO:tasks.workunit.client.0.smithi107.stdout:7/669: dread d0/d8/d4e/d46/d93/fbd [0,4194304] 0 2024-04-25T00:18:34.580 INFO:tasks.workunit.client.0.smithi107.stdout:7/670: write d0/d8/db/d18/d19/d47/da1/fae [951026,7041] 0 2024-04-25T00:18:34.584 INFO:tasks.workunit.client.0.smithi107.stdout:8/736: creat d2/d18/d1a/d5e/d3e/db1/ffa x:0 0 0 2024-04-25T00:18:34.584 INFO:tasks.workunit.client.0.smithi107.stdout:8/737: dread - d2/d18/d1a/d5e/d5c/d91/da3/fb8 zero size 2024-04-25T00:18:34.584 INFO:tasks.workunit.client.0.smithi107.stdout:8/738: stat d2/d18/d1a/d5e/d3e/d74/f9f 0 2024-04-25T00:18:34.584 INFO:tasks.workunit.client.0.smithi107.stdout:7/671: dread d0/d8/db/d18/f61 [0,4194304] 0 2024-04-25T00:18:34.585 INFO:tasks.workunit.client.0.smithi107.stdout:7/672: symlink d0/d8/db/d18/d19/d47/d4d/dbc/le4 0 2024-04-25T00:18:34.586 INFO:tasks.workunit.client.0.smithi107.stdout:7/673: write d0/d8/d4e/d46/d93/fb0 [7280,94323] 0 2024-04-25T00:18:34.587 INFO:tasks.workunit.client.0.smithi107.stdout:7/674: mkdir d0/d8/d2c/de5 0 2024-04-25T00:18:34.589 INFO:tasks.workunit.client.0.smithi107.stdout:7/675: rename d0/d8/l43 to d0/d80/da8/le6 0 2024-04-25T00:18:34.589 INFO:tasks.workunit.client.0.smithi107.stdout:7/676: chown d0/d8/db/d18/d19/d39/c3a 14734 1 2024-04-25T00:18:34.595 INFO:tasks.workunit.client.0.smithi107.stdout:8/739: read d2/d18/d1a/f62 [3014869,22941] 0 2024-04-25T00:18:34.606 INFO:tasks.workunit.client.0.smithi107.stdout:0/619: dwrite dc/d24/d38/d64/d7b/fae [0,4194304] 0 2024-04-25T00:18:34.609 INFO:tasks.workunit.client.0.smithi107.stdout:9/700: dwrite d0/d8a/d93/d56/d6c/d80/fe2 [0,4194304] 0 2024-04-25T00:18:34.612 INFO:tasks.workunit.client.0.smithi107.stdout:9/701: creat d0/d8a/d93/de/d5c/d95/fef x:0 0 0 2024-04-25T00:18:34.612 INFO:tasks.workunit.client.0.smithi107.stdout:7/677: dwrite d0/d8/db/d18/d19/d47/d4d/dbc/f85 [0,4194304] 0 2024-04-25T00:18:34.612 INFO:tasks.workunit.client.0.smithi107.stdout:7/678: fsync d0/d8/d2c/f55 0 2024-04-25T00:18:34.613 INFO:tasks.workunit.client.0.smithi107.stdout:8/740: dwrite d2/d18/d1a/d82/fb6 [0,4194304] 0 2024-04-25T00:18:34.613 INFO:tasks.workunit.client.0.smithi107.stdout:9/702: creat d0/d8a/d93/d56/d6c/d80/d9b/ff0 x:0 0 0 2024-04-25T00:18:34.614 INFO:tasks.workunit.client.0.smithi107.stdout:7/679: mknod d0/d8/db/d18/d19/d47/ce7 0 2024-04-25T00:18:34.614 INFO:tasks.workunit.client.0.smithi107.stdout:7/680: chown d0/d8/db/d18/f61 15312 1 2024-04-25T00:18:34.614 INFO:tasks.workunit.client.0.smithi107.stdout:7/681: fsync d0/d8/d2c/f55 0 2024-04-25T00:18:34.617 INFO:tasks.workunit.client.0.smithi107.stdout:9/703: rmdir d0/d8a/d93/de/d5c 39 2024-04-25T00:18:34.617 INFO:tasks.workunit.client.0.smithi107.stdout:9/704: truncate d0/d8a/d93/d34/d36/f4a 4936712 0 2024-04-25T00:18:34.617 INFO:tasks.workunit.client.0.smithi107.stdout:9/705: rename d0 to d0/d8a/d93/dc2/df1 22 2024-04-25T00:18:34.617 INFO:tasks.workunit.client.0.smithi107.stdout:9/706: chown d0/l5d 0 1 2024-04-25T00:18:34.619 INFO:tasks.workunit.client.0.smithi107.stdout:9/707: mkdir d0/d8a/d93/d6f/da1/df2 0 2024-04-25T00:18:34.619 INFO:tasks.workunit.client.0.smithi107.stdout:9/708: truncate d0/d8a/d93/db9/fe6 240737 0 2024-04-25T00:18:34.620 INFO:tasks.workunit.client.0.smithi107.stdout:9/709: rename d0/d8a/d93/d1f/c81 to d0/d8a/d93/de/d5c/de0/cf3 0 2024-04-25T00:18:34.620 INFO:tasks.workunit.client.0.smithi107.stdout:9/710: chown d0/d8a/d93/c4c 25 1 2024-04-25T00:18:34.621 INFO:tasks.workunit.client.0.smithi107.stdout:9/711: fsync d0/d8a/d93/d6f/da1/fc8 0 2024-04-25T00:18:34.621 INFO:tasks.workunit.client.0.smithi107.stdout:0/620: write dc/d24/d38/d64/d7b/fae [4273882,93812] 0 2024-04-25T00:18:34.622 INFO:tasks.workunit.client.0.smithi107.stdout:9/712: mknod d0/d8a/d93/d56/d76/d50/de4/cf4 0 2024-04-25T00:18:34.623 INFO:tasks.workunit.client.0.smithi107.stdout:0/621: creat dc/fbc x:0 0 0 2024-04-25T00:18:34.624 INFO:tasks.workunit.client.0.smithi107.stdout:9/713: creat d0/d8a/d93/de/d5c/ff5 x:0 0 0 2024-04-25T00:18:34.626 INFO:tasks.workunit.client.0.smithi107.stdout:9/714: creat d0/d8a/d93/d6f/ff6 x:0 0 0 2024-04-25T00:18:34.627 INFO:tasks.workunit.client.0.smithi107.stdout:8/741: truncate d2/d18/d1a/d82/fb6 1870721 0 2024-04-25T00:18:34.627 INFO:tasks.workunit.client.0.smithi107.stdout:8/742: dread - d2/d18/d1a/dca/fd8 zero size 2024-04-25T00:18:34.628 INFO:tasks.workunit.client.0.smithi107.stdout:8/743: link d2/d18/d1a/d5e/d3e/d74/ldb d2/d18/d1a/dca/lfb 0 2024-04-25T00:18:34.629 INFO:tasks.workunit.client.0.smithi107.stdout:7/682: dwrite d0/d8/d2c/faa [0,4194304] 0 2024-04-25T00:18:34.629 INFO:tasks.workunit.client.0.smithi107.stdout:8/744: creat d2/d18/d1a/d5e/d5c/d91/da3/ffc x:0 0 0 2024-04-25T00:18:34.633 INFO:tasks.workunit.client.0.smithi107.stdout:8/745: write d2/d18/d1a/d82/fb6 [2053084,121456] 0 2024-04-25T00:18:34.636 INFO:tasks.workunit.client.0.smithi107.stdout:8/746: rename d2/d18/d1a/d5e/d5c/d6e/fa6 to d2/d18/ffd 0 2024-04-25T00:18:34.637 INFO:tasks.workunit.client.0.smithi107.stdout:8/747: creat d2/d18/d1a/d5e/d3d/d85/dae/ffe x:0 0 0 2024-04-25T00:18:34.638 INFO:tasks.workunit.client.0.smithi107.stdout:8/748: write d2/d18/d1a/d5e/d4f/da0/fac [1189464,120415] 0 2024-04-25T00:18:34.639 INFO:tasks.workunit.client.0.smithi107.stdout:8/749: creat d2/d18/d1a/d5e/d3d/d85/dd5/fff x:0 0 0 2024-04-25T00:18:34.639 INFO:tasks.workunit.client.0.smithi107.stdout:8/750: truncate d2/d18/d1a/d82/f8b 659801 0 2024-04-25T00:18:34.639 INFO:tasks.workunit.client.0.smithi107.stdout:9/715: dread d0/d8a/d93/d56/d76/fb8 [0,4194304] 0 2024-04-25T00:18:34.640 INFO:tasks.workunit.client.0.smithi107.stdout:9/716: dread - d0/d8a/d93/fe5 zero size 2024-04-25T00:18:34.641 INFO:tasks.workunit.client.0.smithi107.stdout:8/751: creat d2/d18/d1a/d5e/d3e/f100 x:0 0 0 2024-04-25T00:18:34.642 INFO:tasks.workunit.client.0.smithi107.stdout:9/717: creat d0/d8a/d93/d56/d6c/d80/d9b/db2/dea/ff7 x:0 0 0 2024-04-25T00:18:34.644 INFO:tasks.workunit.client.0.smithi107.stdout:8/752: truncate d2/d18/d1a/f62 3688383 0 2024-04-25T00:18:34.644 INFO:tasks.workunit.client.0.smithi107.stdout:8/753: stat d2/d18/d1a/dca 0 2024-04-25T00:18:34.646 INFO:tasks.workunit.client.0.smithi107.stdout:9/718: rename d0/d43/fd to d0/d8a/d93/d1f/ff8 0 2024-04-25T00:18:34.649 INFO:tasks.workunit.client.0.smithi107.stdout:7/683: dwrite d0/d8/db/f98 [0,4194304] 0 2024-04-25T00:18:34.649 INFO:tasks.workunit.client.0.smithi107.stdout:8/754: creat d2/d18/d1a/d5e/d3d/d85/f101 x:0 0 0 2024-04-25T00:18:34.658 INFO:tasks.workunit.client.0.smithi107.stdout:0/622: dwrite dc/dd/d5d/f57 [0,4194304] 0 2024-04-25T00:18:34.658 INFO:tasks.workunit.client.0.smithi107.stdout:9/719: dread d0/d8a/d93/d34/d36/f4a [0,4194304] 0 2024-04-25T00:18:34.658 INFO:tasks.workunit.client.0.smithi107.stdout:9/720: stat d0/d8a/d93/d56/d76/d50/d7d/f85 0 2024-04-25T00:18:34.659 INFO:tasks.workunit.client.0.smithi107.stdout:9/721: chown d0/d43/f28 6733104 1 2024-04-25T00:18:34.659 INFO:tasks.workunit.client.0.smithi107.stdout:0/623: creat dc/dd/d16/d48/d83/fbd x:0 0 0 2024-04-25T00:18:34.672 INFO:tasks.workunit.client.0.smithi107.stdout:7/684: dwrite d0/d8/db/d18/d19/f23 [0,4194304] 0 2024-04-25T00:18:34.677 INFO:tasks.workunit.client.0.smithi107.stdout:7/685: unlink d0/f69 0 2024-04-25T00:18:34.678 INFO:tasks.workunit.client.0.smithi107.stdout:7/686: rmdir d0/d8/db/d18/d19/d67 39 2024-04-25T00:18:34.678 INFO:tasks.workunit.client.0.smithi107.stdout:7/687: fsync d0/d8/fd 0 2024-04-25T00:18:34.680 INFO:tasks.workunit.client.0.smithi107.stdout:7/688: symlink d0/d8/d2c/d58/le8 0 2024-04-25T00:18:34.681 INFO:tasks.workunit.client.0.smithi107.stdout:8/755: dwrite d2/d18/d1a/d5e/d3e/d74/f9f [4194304,4194304] 0 2024-04-25T00:18:34.681 INFO:tasks.workunit.client.0.smithi107.stdout:8/756: read - d2/d18/d1a/d5e/d3e/fcc zero size 2024-04-25T00:18:34.682 INFO:tasks.workunit.client.0.smithi107.stdout:9/722: dwrite d0/d8a/d93/d34/fa0 [0,4194304] 0 2024-04-25T00:18:34.682 INFO:tasks.workunit.client.0.smithi107.stdout:9/723: dread - d0/d8a/d93/de/fde zero size 2024-04-25T00:18:34.682 INFO:tasks.workunit.client.0.smithi107.stdout:0/624: dwrite dc/d24/d38/d64/d78/fa1 [0,4194304] 0 2024-04-25T00:18:34.685 INFO:tasks.workunit.client.0.smithi107.stdout:0/625: mknod dc/dd/d5d/cbe 0 2024-04-25T00:18:34.686 INFO:tasks.workunit.client.0.smithi107.stdout:0/626: write dc/dd/f4c [4554427,36316] 0 2024-04-25T00:18:34.686 INFO:tasks.workunit.client.0.smithi107.stdout:0/627: chown dc/d24/d25/d31/l98 0 1 2024-04-25T00:18:34.686 INFO:tasks.workunit.client.0.smithi107.stdout:9/724: dread d0/d8a/d93/d6f/da1/fc8 [0,4194304] 0 2024-04-25T00:18:34.686 INFO:tasks.workunit.client.0.smithi107.stdout:9/725: readlink d0/d8a/d93/de/l17 0 2024-04-25T00:18:34.687 INFO:tasks.workunit.client.0.smithi107.stdout:0/628: getdents dc 0 2024-04-25T00:18:34.688 INFO:tasks.workunit.client.0.smithi107.stdout:9/726: creat d0/d8a/d93/d6f/ff9 x:0 0 0 2024-04-25T00:18:34.690 INFO:tasks.workunit.client.0.smithi107.stdout:0/629: rename dc/d24/d38/d64/d78/fa1 to dc/d24/fbf 0 2024-04-25T00:18:34.691 INFO:tasks.workunit.client.0.smithi107.stdout:0/630: mkdir dc/dd/d16/d33/d35/d5f/dc0 0 2024-04-25T00:18:34.692 INFO:tasks.workunit.client.0.smithi107.stdout:0/631: write dc/dd/d5d/f57 [4417824,13486] 0 2024-04-25T00:18:34.693 INFO:tasks.workunit.client.0.smithi107.stdout:9/727: getdents d0/d8a/d93/d56/d6c/d80 0 2024-04-25T00:18:34.693 INFO:tasks.workunit.client.0.smithi107.stdout:9/728: stat d0/d8a/d93/d34/d36/d77 0 2024-04-25T00:18:34.693 INFO:tasks.workunit.client.0.smithi107.stdout:9/729: chown d0/d8a/d93/de/d5c/d95/f63 3 1 2024-04-25T00:18:34.693 INFO:tasks.workunit.client.0.smithi107.stdout:9/730: chown d0/d8a/d93/d34/fa0 318206 1 2024-04-25T00:18:34.693 INFO:tasks.workunit.client.0.smithi107.stdout:9/731: chown d0/d8a/d93/d56/d6c/d80/f97 246971181 1 2024-04-25T00:18:34.695 INFO:tasks.workunit.client.0.smithi107.stdout:0/632: creat dc/dd/d5d/d5c/d6e/fc1 x:0 0 0 2024-04-25T00:18:34.697 INFO:tasks.workunit.client.0.smithi107.stdout:7/689: dread d0/d8/db/d18/d19/d47/d4d/d8a/fca [0,4194304] 0 2024-04-25T00:18:34.708 INFO:tasks.workunit.client.0.smithi107.stdout:9/732: dread d0/d8a/d93/d34/f84 [0,4194304] 0 2024-04-25T00:18:34.710 INFO:tasks.workunit.client.0.smithi107.stdout:9/733: read d0/d8a/d93/d56/d76/d50/d7d/f85 [1246368,22385] 0 2024-04-25T00:18:34.711 INFO:tasks.workunit.client.0.smithi107.stdout:8/757: dwrite d2/d18/d1a/d5e/d3e/d74/f97 [0,4194304] 0 2024-04-25T00:18:34.712 INFO:tasks.workunit.client.0.smithi107.stdout:9/734: write d0/d8a/d93/d56/d6c/d80/f97 [11491,57319] 0 2024-04-25T00:18:34.719 INFO:tasks.workunit.client.0.smithi107.stdout:7/690: dwrite d0/d8/f73 [0,4194304] 0 2024-04-25T00:18:34.719 INFO:tasks.workunit.client.0.smithi107.stdout:7/691: fsync d0/d8/db/d18/d19/d47/f31 0 2024-04-25T00:18:34.719 INFO:tasks.workunit.client.0.smithi107.stdout:0/633: dwrite dc/dd/d5d/d3f/d59/fa0 [0,4194304] 0 2024-04-25T00:18:34.722 INFO:tasks.workunit.client.0.smithi107.stdout:7/692: dread d0/f13 [0,4194304] 0 2024-04-25T00:18:34.723 INFO:tasks.workunit.client.0.smithi107.stdout:8/758: symlink d2/d18/d1a/d5e/d3d/d85/dd5/l102 0 2024-04-25T00:18:34.723 INFO:tasks.workunit.client.0.smithi107.stdout:8/759: truncate d2/d18/d1a/d5e/d4f/f76 2171477 0 2024-04-25T00:18:34.726 INFO:tasks.workunit.client.0.smithi107.stdout:9/735: unlink d0/d43/ld6 0 2024-04-25T00:18:34.726 INFO:tasks.workunit.client.0.smithi107.stdout:9/736: chown d0/d8a/d93/d56/d76/d50/d7d 106989 1 2024-04-25T00:18:34.729 INFO:tasks.workunit.client.0.smithi107.stdout:0/634: symlink dc/d24/d38/d64/d7b/lc2 0 2024-04-25T00:18:34.731 INFO:tasks.workunit.client.0.smithi107.stdout:8/760: unlink d2/d18/d1a/d5e/d3d/l5a 0 2024-04-25T00:18:34.732 INFO:tasks.workunit.client.0.smithi107.stdout:9/737: mknod d0/d8a/d93/d56/d6c/daa/cfa 0 2024-04-25T00:18:34.732 INFO:tasks.workunit.client.0.smithi107.stdout:9/738: truncate d0/d8a/d93/d6f/da1/fce 132776 0 2024-04-25T00:18:34.734 INFO:tasks.workunit.client.0.smithi107.stdout:8/761: rename d2/d18/d1a/d5e/d4f/f81 to d2/d18/d1a/d5e/d5c/d91/da3/f103 0 2024-04-25T00:18:34.736 INFO:tasks.workunit.client.0.smithi107.stdout:8/762: rmdir d2/d18/d1a/d5e/d5c/df5 0 2024-04-25T00:18:34.738 INFO:tasks.workunit.client.0.smithi107.stdout:8/763: truncate d2/d18/d1a/d5e/d3d/f9d 1752649 0 2024-04-25T00:18:34.738 INFO:tasks.workunit.client.0.smithi107.stdout:7/693: dwrite d0/d8/d2c/faa [0,4194304] 0 2024-04-25T00:18:34.738 INFO:tasks.workunit.client.0.smithi107.stdout:7/694: chown d0/d8/d4e/ccd 32622546 1 2024-04-25T00:18:34.740 INFO:tasks.workunit.client.0.smithi107.stdout:8/764: creat d2/d18/d1a/d5e/d5c/d91/de7/f104 x:0 0 0 2024-04-25T00:18:34.745 INFO:tasks.workunit.client.0.smithi107.stdout:9/739: dread d0/d8a/d93/f91 [0,4194304] 0 2024-04-25T00:18:34.745 INFO:tasks.workunit.client.0.smithi107.stdout:8/765: rename d2/f41 to d2/d18/d1a/d5e/d3d/d85/f105 0 2024-04-25T00:18:34.746 INFO:tasks.workunit.client.0.smithi107.stdout:9/740: mkdir d0/dfb 0 2024-04-25T00:18:34.748 INFO:tasks.workunit.client.0.smithi107.stdout:8/766: rename d2/d18/d1a/c71 to d2/d18/da2/c106 0 2024-04-25T00:18:34.752 INFO:tasks.workunit.client.0.smithi107.stdout:9/741: creat d0/d8a/d93/d6f/da1/ffc x:0 0 0 2024-04-25T00:18:34.755 INFO:tasks.workunit.client.0.smithi107.stdout:1/660: sync 2024-04-25T00:18:34.755 INFO:tasks.workunit.client.0.smithi107.stdout:4/749: sync 2024-04-25T00:18:34.755 INFO:tasks.workunit.client.0.smithi107.stdout:1/661: chown d2/da/f53 22665 1 2024-04-25T00:18:34.758 INFO:tasks.workunit.client.0.smithi107.stdout:9/742: link d0/d8a/d93/d6f/fb5 d0/ffd 0 2024-04-25T00:18:34.758 INFO:tasks.workunit.client.0.smithi107.stdout:9/743: chown d0/d8a/cb3 64 1 2024-04-25T00:18:34.758 INFO:tasks.workunit.client.0.smithi107.stdout:9/744: chown d0/d8a/d93/d34/lec 237 1 2024-04-25T00:18:34.758 INFO:tasks.workunit.client.0.smithi107.stdout:7/695: dwrite d0/d8/db/d18/d19/d47/d4d/dbc/f75 [0,4194304] 0 2024-04-25T00:18:34.760 INFO:tasks.workunit.client.0.smithi107.stdout:1/662: mkdir d2/d62/d66/d70/dcc 0 2024-04-25T00:18:34.760 INFO:tasks.workunit.client.0.smithi107.stdout:1/663: chown d2/d2a/lbd 2179 1 2024-04-25T00:18:34.761 INFO:tasks.workunit.client.0.smithi107.stdout:7/696: creat d0/d8/db/d18/d19/d47/de0/dc2/dd0/fe9 x:0 0 0 2024-04-25T00:18:34.761 INFO:tasks.workunit.client.0.smithi107.stdout:7/697: dread - d0/d8/d2c/d58/f5f zero size 2024-04-25T00:18:34.762 INFO:tasks.workunit.client.0.smithi107.stdout:7/698: write d0/d8/db/d18/d19/fb7 [1127570,23991] 0 2024-04-25T00:18:34.764 INFO:tasks.workunit.client.0.smithi107.stdout:9/745: creat d0/d8a/d93/d34/ffe x:0 0 0 2024-04-25T00:18:34.766 INFO:tasks.workunit.client.0.smithi107.stdout:8/767: rename d2/d18/da2 to d2/d18/d1a/d5e/d5c/d107 0 2024-04-25T00:18:34.767 INFO:tasks.workunit.client.0.smithi107.stdout:9/746: mknod d0/d8a/cff 0 2024-04-25T00:18:34.767 INFO:tasks.workunit.client.0.smithi107.stdout:9/747: truncate d0/d8a/d93/de/fdc 546179 0 2024-04-25T00:18:34.768 INFO:tasks.workunit.client.0.smithi107.stdout:7/699: rename d0/d8/db/c29 to d0/d8/db/d18/d19/d47/da1/dd4/cea 0 2024-04-25T00:18:34.768 INFO:tasks.workunit.client.0.smithi107.stdout:1/664: dread d2/dd/d1b/f52 [0,4194304] 0 2024-04-25T00:18:34.768 INFO:tasks.workunit.client.0.smithi107.stdout:1/665: fsync d2/d62/d66/f6e 0 2024-04-25T00:18:34.769 INFO:tasks.workunit.client.0.smithi107.stdout:8/768: mknod d2/d18/d1a/d5e/d3e/c108 0 2024-04-25T00:18:34.769 INFO:tasks.workunit.client.0.smithi107.stdout:8/769: write d2/d18/d1a/d5e/d4f/da0/fac [1351002,6678] 0 2024-04-25T00:18:34.769 INFO:tasks.workunit.client.0.smithi107.stdout:8/770: chown d2/d18/d1a/d5e/d5c/d91/da3/led 218777138 1 2024-04-25T00:18:34.769 INFO:tasks.workunit.client.0.smithi107.stdout:7/700: mknod d0/d8/d4e/d46/d93/ceb 0 2024-04-25T00:18:34.769 INFO:tasks.workunit.client.0.smithi107.stdout:9/748: dread d0/d8a/d93/de/d5c/d95/f2f [0,4194304] 0 2024-04-25T00:18:34.769 INFO:tasks.workunit.client.0.smithi107.stdout:9/749: fdatasync d0/d8a/d93/d56/d6c/daa/fd5 0 2024-04-25T00:18:34.769 INFO:tasks.workunit.client.0.smithi107.stdout:1/666: truncate d2/dd/d25/f2e 2588925 0 2024-04-25T00:18:34.772 INFO:tasks.workunit.client.0.smithi107.stdout:8/771: rmdir d2/d18/d1a/d5e/d5c/d107/db2/dd2 0 2024-04-25T00:18:34.773 INFO:tasks.workunit.client.0.smithi107.stdout:0/635: link dc/d24/d25/d31/l98 dc/dd/d16/d33/d35/d5f/lc3 0 2024-04-25T00:18:34.773 INFO:tasks.workunit.client.0.smithi107.stdout:9/750: creat d0/d8a/d93/d6f/da1/df2/f100 x:0 0 0 2024-04-25T00:18:34.775 INFO:tasks.workunit.client.0.smithi107.stdout:0/636: mkdir dc/dd/d16/d48/d83/d9f/dc4 0 2024-04-25T00:18:34.776 INFO:tasks.workunit.client.0.smithi107.stdout:0/637: mknod dc/dd/d16/d48/d83/d9f/db0/cc5 0 2024-04-25T00:18:34.782 INFO:tasks.workunit.client.0.smithi107.stdout:1/667: dread f1 [0,4194304] 0 2024-04-25T00:18:34.788 INFO:tasks.workunit.client.0.smithi107.stdout:4/750: link d3/d24/d5c/d7a/lee d3/d24/lf9 0 2024-04-25T00:18:34.789 INFO:tasks.workunit.client.0.smithi107.stdout:0/638: dread dc/d24/d38/f66 [0,4194304] 0 2024-04-25T00:18:34.789 INFO:tasks.workunit.client.0.smithi107.stdout:0/639: truncate dc/dd/d5d/d5c/f71 4273591 0 2024-04-25T00:18:34.789 INFO:tasks.workunit.client.0.smithi107.stdout:0/640: write dc/dd/d5d/d5c/d6e/f99 [575217,120605] 0 2024-04-25T00:18:34.789 INFO:tasks.workunit.client.0.smithi107.stdout:0/641: write dc/dd/f3c [3883635,14707] 0 2024-04-25T00:18:34.792 INFO:tasks.workunit.client.0.smithi107.stdout:0/642: symlink dc/dd/d16/d5e/lc6 0 2024-04-25T00:18:34.792 INFO:tasks.workunit.client.0.smithi107.stdout:1/668: dread d2/dd/d25/d5e/f81 [0,4194304] 0 2024-04-25T00:18:34.792 INFO:tasks.workunit.client.0.smithi107.stdout:1/669: readlink d2/d2a/lb1 0 2024-04-25T00:18:34.792 INFO:tasks.workunit.client.0.smithi107.stdout:1/670: chown d2/dd/d25/d5e 250 1 2024-04-25T00:18:34.793 INFO:tasks.workunit.client.0.smithi107.stdout:0/643: getdents dc/dd/d16/d48/d83 0 2024-04-25T00:18:34.793 INFO:tasks.workunit.client.0.smithi107.stdout:0/644: chown dc 3 1 2024-04-25T00:18:34.793 INFO:tasks.workunit.client.0.smithi107.stdout:1/671: mknod d2/dd/d82/dc1/ccd 0 2024-04-25T00:18:34.793 INFO:tasks.workunit.client.0.smithi107.stdout:1/672: fsync d2/dd/d1b/fa3 0 2024-04-25T00:18:34.793 INFO:tasks.workunit.client.0.smithi107.stdout:4/751: dread d3/d24/d5c/dab/f7d [0,4194304] 0 2024-04-25T00:18:34.794 INFO:tasks.workunit.client.0.smithi107.stdout:4/752: creat d3/d24/d25/d71/ffa x:0 0 0 2024-04-25T00:18:34.798 INFO:tasks.workunit.client.0.smithi107.stdout:4/753: fsync d3/d5/f22 0 2024-04-25T00:18:34.798 INFO:tasks.workunit.client.0.smithi107.stdout:1/673: link d2/f51 d2/d2a/d5c/d89/dc0/fce 0 2024-04-25T00:18:34.798 INFO:tasks.workunit.client.0.smithi107.stdout:1/674: chown d2/d62/fc8 25391034 1 2024-04-25T00:18:34.799 INFO:tasks.workunit.client.0.smithi107.stdout:9/751: dwrite d0/d8a/d93/f55 [4194304,4194304] 0 2024-04-25T00:18:34.799 INFO:tasks.workunit.client.0.smithi107.stdout:0/645: rmdir dc/dd/d5d/d3f 39 2024-04-25T00:18:34.799 INFO:tasks.workunit.client.0.smithi107.stdout:9/752: dread d0/d43/f10 [4194304,4194304] 0 2024-04-25T00:18:34.799 INFO:tasks.workunit.client.0.smithi107.stdout:4/754: creat d3/d24/d25/d71/dc2/ffb x:0 0 0 2024-04-25T00:18:34.799 INFO:tasks.workunit.client.0.smithi107.stdout:0/646: mkdir dc/dd/d5d/d3f/dc7 0 2024-04-25T00:18:34.799 INFO:tasks.workunit.client.0.smithi107.stdout:0/647: write dc/dd/f4c [4953391,43240] 0 2024-04-25T00:18:34.799 INFO:tasks.workunit.client.0.smithi107.stdout:0/648: creat dc/dd/d5d/d3f/fc8 x:0 0 0 2024-04-25T00:18:34.800 INFO:tasks.workunit.client.0.smithi107.stdout:9/753: write d0/d8a/d93/d56/f59 [1107662,123390] 0 2024-04-25T00:18:34.801 INFO:tasks.workunit.client.0.smithi107.stdout:0/649: creat dc/dd/fc9 x:0 0 0 2024-04-25T00:18:34.803 INFO:tasks.workunit.client.0.smithi107.stdout:9/754: dread d0/d8a/d93/d56/d76/f5b [0,4194304] 0 2024-04-25T00:18:34.805 INFO:tasks.workunit.client.0.smithi107.stdout:2/707: sync 2024-04-25T00:18:34.805 INFO:tasks.workunit.client.0.smithi107.stdout:5/691: sync 2024-04-25T00:18:34.806 INFO:tasks.workunit.client.0.smithi107.stdout:2/708: mknod d6/d18/d40/d42/d69/d6e/cf8 0 2024-04-25T00:18:34.806 INFO:tasks.workunit.client.0.smithi107.stdout:2/709: write d6/d18/d40/d42/d69/d6e/d80/f9a [89971,63072] 0 2024-04-25T00:18:34.806 INFO:tasks.workunit.client.0.smithi107.stdout:5/692: dread d4/d17/f69 [0,4194304] 0 2024-04-25T00:18:34.807 INFO:tasks.workunit.client.0.smithi107.stdout:9/755: dread d0/d8a/d93/de/d5c/f6a [0,4194304] 0 2024-04-25T00:18:34.808 INFO:tasks.workunit.client.0.smithi107.stdout:5/693: unlink d4/dc/d26/d29/d2e/d82/f68 0 2024-04-25T00:18:34.809 INFO:tasks.workunit.client.0.smithi107.stdout:9/756: mknod d0/d8a/d93/de/c101 0 2024-04-25T00:18:34.809 INFO:tasks.workunit.client.0.smithi107.stdout:9/757: readlink d0/l29 0 2024-04-25T00:18:34.809 INFO:tasks.workunit.client.0.smithi107.stdout:9/758: mknod d0/d8a/d93/d6f/c102 0 2024-04-25T00:18:34.811 INFO:tasks.workunit.client.0.smithi107.stdout:2/710: dread d6/d18/d40/d42/d2b/f7a [0,4194304] 0 2024-04-25T00:18:34.820 INFO:tasks.workunit.client.0.smithi107.stdout:5/694: dread d4/dc/d26/d42/f4f [0,4194304] 0 2024-04-25T00:18:34.820 INFO:tasks.workunit.client.0.smithi107.stdout:5/695: chown d4/d17/db3/d84/d93/da4/lca 32432 1 2024-04-25T00:18:34.820 INFO:tasks.workunit.client.0.smithi107.stdout:5/696: readlink d4/dc/d26/d29/l65 0 2024-04-25T00:18:34.822 INFO:tasks.workunit.client.0.smithi107.stdout:0/650: rename dc/l26 to dc/dd/d16/d48/d83/d9f/dc4/lca 0 2024-04-25T00:18:34.822 INFO:tasks.workunit.client.0.smithi107.stdout:0/651: fdatasync dc/d24/d25/d31/f6f 0 2024-04-25T00:18:34.823 INFO:tasks.workunit.client.0.smithi107.stdout:2/711: rename d6/d29/l66 to d6/d18/d40/dc8/lf9 0 2024-04-25T00:18:34.824 INFO:tasks.workunit.client.0.smithi107.stdout:2/712: mknod d6/cfa 0 2024-04-25T00:18:34.825 INFO:tasks.workunit.client.0.smithi107.stdout:2/713: write d6/dad/fd5 [260837,100476] 0 2024-04-25T00:18:34.827 INFO:tasks.workunit.client.0.smithi107.stdout:9/759: dread d0/d8a/d93/f55 [0,4194304] 0 2024-04-25T00:18:34.827 INFO:tasks.workunit.client.0.smithi107.stdout:2/714: dread d6/d29/d4e/d6f/f90 [0,4194304] 0 2024-04-25T00:18:34.827 INFO:tasks.workunit.client.0.smithi107.stdout:2/715: write d6/d18/d40/d42/d69/d6e/d80/fcb [73080,107062] 0 2024-04-25T00:18:34.828 INFO:tasks.workunit.client.0.smithi107.stdout:2/716: read - d6/dad/f98 zero size 2024-04-25T00:18:34.828 INFO:tasks.workunit.client.0.smithi107.stdout:2/717: chown d6/f72 4 1 2024-04-25T00:18:34.833 INFO:tasks.workunit.client.0.smithi107.stdout:5/697: link d4/d17/l19 d4/d17/db3/d84/le0 0 2024-04-25T00:18:34.834 INFO:tasks.workunit.client.0.smithi107.stdout:5/698: dread - d4/d17/db3/fd1 zero size 2024-04-25T00:18:34.836 INFO:tasks.workunit.client.0.smithi107.stdout:5/699: unlink l0 0 2024-04-25T00:18:34.838 INFO:tasks.workunit.client.0.smithi107.stdout:5/700: unlink d4/d7c/d88/d8e/lad 0 2024-04-25T00:18:34.840 INFO:tasks.workunit.client.0.smithi107.stdout:0/652: dwrite dc/d24/d25/f67 [0,4194304] 0 2024-04-25T00:18:34.844 INFO:tasks.workunit.client.0.smithi107.stdout:0/653: rename dc/dd/d5d/d3f/f92 to dc/fcb 0 2024-04-25T00:18:34.845 INFO:tasks.workunit.client.0.smithi107.stdout:0/654: readlink dc/dd/l86 0 2024-04-25T00:18:34.846 INFO:tasks.workunit.client.0.smithi107.stdout:5/701: dread d4/dc/d83/fcc [0,4194304] 0 2024-04-25T00:18:34.846 INFO:tasks.workunit.client.0.smithi107.stdout:5/702: chown d4/f16 0 1 2024-04-25T00:18:34.847 INFO:tasks.workunit.client.0.smithi107.stdout:9/760: dwrite d0/d8a/d93/d6f/ff6 [0,4194304] 0 2024-04-25T00:18:34.847 INFO:tasks.workunit.client.0.smithi107.stdout:2/718: dwrite d6/d18/d40/d42/d2b/f7a [4194304,4194304] 0 2024-04-25T00:18:34.847 INFO:tasks.workunit.client.0.smithi107.stdout:5/703: mkdir d4/d7c/d88/d9d/de1 0 2024-04-25T00:18:34.849 INFO:tasks.workunit.client.0.smithi107.stdout:5/704: creat d4/d17/db3/d84/d93/dc4/fe2 x:0 0 0 2024-04-25T00:18:34.851 INFO:tasks.workunit.client.0.smithi107.stdout:9/761: symlink d0/d8a/d93/dc2/dc6/l103 0 2024-04-25T00:18:34.852 INFO:tasks.workunit.client.0.smithi107.stdout:9/762: fdatasync d0/d8a/d93/de/d5c/d95/fef 0 2024-04-25T00:18:34.852 INFO:tasks.workunit.client.0.smithi107.stdout:9/763: chown d0/d8a/d93/de/d5c/de0/cf3 309 1 2024-04-25T00:18:34.852 INFO:tasks.workunit.client.0.smithi107.stdout:9/764: fsync d0/f4b 0 2024-04-25T00:18:34.852 INFO:tasks.workunit.client.0.smithi107.stdout:9/765: chown d0/d8a/d93/d56/d6c/daa/lcb 15 1 2024-04-25T00:18:34.852 INFO:tasks.workunit.client.0.smithi107.stdout:9/766: fdatasync d0/d8a/d93/d56/d6c/d80/f97 0 2024-04-25T00:18:34.852 INFO:tasks.workunit.client.0.smithi107.stdout:9/767: read d0/d8a/d93/d56/d76/d50/d7d/f85 [395940,37710] 0 2024-04-25T00:18:34.854 INFO:tasks.workunit.client.0.smithi107.stdout:2/719: creat d6/d18/d40/d42/d70/ffb x:0 0 0 2024-04-25T00:18:34.856 INFO:tasks.workunit.client.0.smithi107.stdout:9/768: link d0/f7e d0/d8a/d93/d56/d6c/d80/f104 0 2024-04-25T00:18:34.860 INFO:tasks.workunit.client.0.smithi107.stdout:2/720: unlink d6/d29/d4e/d6f/f90 0 2024-04-25T00:18:34.865 INFO:tasks.workunit.client.0.smithi107.stdout:5/705: dwrite d4/d35/f37 [4194304,4194304] 0 2024-04-25T00:18:34.869 INFO:tasks.workunit.client.0.smithi107.stdout:5/706: link d4/dc/d26/d62/fc8 d4/d1c/d5c/fe3 0 2024-04-25T00:18:34.873 INFO:tasks.workunit.client.0.smithi107.stdout:5/707: unlink d4/dc/d26/d29/d51/c89 0 2024-04-25T00:18:34.873 INFO:tasks.workunit.client.0.smithi107.stdout:0/655: link dc/dd/d5d/f87 dc/dd/d16/d5e/fcc 0 2024-04-25T00:18:34.875 INFO:tasks.workunit.client.0.smithi107.stdout:5/708: rename d4/dc/d26/d29/d2e/d55/f75 to d4/d7c/fe4 0 2024-04-25T00:18:34.876 INFO:tasks.workunit.client.0.smithi107.stdout:0/656: creat dc/dd/d5d/d5c/d6e/fcd x:0 0 0 2024-04-25T00:18:34.876 INFO:tasks.workunit.client.0.smithi107.stdout:5/709: creat d4/dc/d26/d29/d51/fe5 x:0 0 0 2024-04-25T00:18:34.877 INFO:tasks.workunit.client.0.smithi107.stdout:9/769: dwrite d0/d8a/d93/d56/d6c/daa/fd5 [0,4194304] 0 2024-04-25T00:18:34.877 INFO:tasks.workunit.client.0.smithi107.stdout:9/770: write d0/d8a/d93/d34/d36/f4a [1331518,105449] 0 2024-04-25T00:18:34.878 INFO:tasks.workunit.client.0.smithi107.stdout:5/710: symlink d4/dc/d26/d29/d2e/le6 0 2024-04-25T00:18:34.878 INFO:tasks.workunit.client.0.smithi107.stdout:5/711: stat d4/d7c/d88/d9d/de1 0 2024-04-25T00:18:34.878 INFO:tasks.workunit.client.0.smithi107.stdout:5/712: chown d4/dc/d26/d42/f59 132934001 1 2024-04-25T00:18:34.879 INFO:tasks.workunit.client.0.smithi107.stdout:5/713: stat d4/d35/fce 0 2024-04-25T00:18:34.879 INFO:tasks.workunit.client.0.smithi107.stdout:5/714: write d4/dc/d26/d29/d2e/d82/fd2 [731962,114778] 0 2024-04-25T00:18:34.881 INFO:tasks.workunit.client.0.smithi107.stdout:2/721: dwrite d6/d18/d40/d42/d2b/d30/fbf [0,4194304] 0 2024-04-25T00:18:34.888 INFO:tasks.workunit.client.0.smithi107.stdout:5/715: dread d4/dc/d26/d29/d2e/d82/fa6 [0,4194304] 0 2024-04-25T00:18:34.891 INFO:tasks.workunit.client.0.smithi107.stdout:2/722: symlink d6/d18/d40/d42/d69/d6e/da2/lfc 0 2024-04-25T00:18:34.893 INFO:tasks.workunit.client.0.smithi107.stdout:5/716: mknod d4/d7c/ce7 0 2024-04-25T00:18:34.893 INFO:tasks.workunit.client.0.smithi107.stdout:5/717: readlink d4/d17/l5f 0 2024-04-25T00:18:34.894 INFO:tasks.workunit.client.0.smithi107.stdout:9/771: dwrite d0/d8a/d93/d34/d36/f8c [0,4194304] 0 2024-04-25T00:18:34.895 INFO:tasks.workunit.client.0.smithi107.stdout:2/723: creat d6/d18/d40/d42/d69/d6e/db6/ffd x:0 0 0 2024-04-25T00:18:34.895 INFO:tasks.workunit.client.0.smithi107.stdout:2/724: chown d6/d18/d40/d42/d2b/d30 1038882897 1 2024-04-25T00:18:34.896 INFO:tasks.workunit.client.0.smithi107.stdout:5/718: creat d4/d7c/fe8 x:0 0 0 2024-04-25T00:18:34.900 INFO:tasks.workunit.client.0.smithi107.stdout:9/772: link d0/d8a/d93/d56/fba d0/d8a/d93/de/f105 0 2024-04-25T00:18:34.906 INFO:tasks.workunit.client.0.smithi107.stdout:5/719: creat d4/d35/fe9 x:0 0 0 2024-04-25T00:18:34.906 INFO:tasks.workunit.client.0.smithi107.stdout:2/725: mkdir d6/d20/d77/d9f/dfe 0 2024-04-25T00:18:34.906 INFO:tasks.workunit.client.0.smithi107.stdout:2/726: read - d6/dad/db0/fdc zero size 2024-04-25T00:18:34.906 INFO:tasks.workunit.client.0.smithi107.stdout:2/727: write d6/d18/d40/d42/d2b/f41 [967498,101296] 0 2024-04-25T00:18:34.906 INFO:tasks.workunit.client.0.smithi107.stdout:3/665: sync 2024-04-25T00:18:34.906 INFO:tasks.workunit.client.0.smithi107.stdout:6/580: sync 2024-04-25T00:18:34.906 INFO:tasks.workunit.client.0.smithi107.stdout:0/657: rename dc/dd/d16/d48/d83/d9f/dc4/lca to dc/d24/d25/d31/lce 0 2024-04-25T00:18:34.907 INFO:tasks.workunit.client.0.smithi107.stdout:3/666: dread d9/d22/d1b/d55/fcf [0,4194304] 0 2024-04-25T00:18:34.908 INFO:tasks.workunit.client.0.smithi107.stdout:3/667: dread - d9/d15/da1/d4b/fcd zero size 2024-04-25T00:18:34.908 INFO:tasks.workunit.client.0.smithi107.stdout:3/668: readlink d9/d88/l6e 0 2024-04-25T00:18:34.908 INFO:tasks.workunit.client.0.smithi107.stdout:5/720: creat d4/d1c/d5c/dd4/fea x:0 0 0 2024-04-25T00:18:34.911 INFO:tasks.workunit.client.0.smithi107.stdout:5/721: mkdir d4/d7c/d88/d9d/deb 0 2024-04-25T00:18:34.917 INFO:tasks.workunit.client.0.smithi107.stdout:6/581: dread d1/d2/d3/d71/d94/f9b [0,4194304] 0 2024-04-25T00:18:34.918 INFO:tasks.workunit.client.0.smithi107.stdout:9/773: dwrite d0/d8a/d93/de/d5c/f96 [0,4194304] 0 2024-04-25T00:18:34.919 INFO:tasks.workunit.client.0.smithi107.stdout:6/582: mknod d1/d2/db/d3c/d43/d96/cbf 0 2024-04-25T00:18:34.919 INFO:tasks.workunit.client.0.smithi107.stdout:6/583: fsync d1/d2/db/d32/d36/d3d/fa1 0 2024-04-25T00:18:34.919 INFO:tasks.workunit.client.0.smithi107.stdout:6/584: truncate d1/d2/db/d32/d36/d3a/f9a 1240296 0 2024-04-25T00:18:34.919 INFO:tasks.workunit.client.0.smithi107.stdout:6/585: write d1/d2/d2b/d3e/d5b/d7a/faf [147041,116587] 0 2024-04-25T00:18:34.919 INFO:tasks.workunit.client.0.smithi107.stdout:9/774: dread d0/d8a/d93/db9/fe6 [0,4194304] 0 2024-04-25T00:18:34.921 INFO:tasks.workunit.client.0.smithi107.stdout:6/586: symlink d1/d14/d18/lc0 0 2024-04-25T00:18:34.922 INFO:tasks.workunit.client.0.smithi107.stdout:3/669: write d9/d54/d5c/d68/f3c [133998,34504] 0 2024-04-25T00:18:34.922 INFO:tasks.workunit.client.0.smithi107.stdout:6/587: mknod d1/d2/db/d3c/cc1 0 2024-04-25T00:18:34.925 INFO:tasks.workunit.client.0.smithi107.stdout:6/588: rmdir d1/d14/d8a 0 2024-04-25T00:18:34.925 INFO:tasks.workunit.client.0.smithi107.stdout:6/589: dread - d1/d2/db/d32/d33/d4f/f58 zero size 2024-04-25T00:18:34.925 INFO:tasks.workunit.client.0.smithi107.stdout:5/722: read d4/dc/d26/f64 [14238,9439] 0 2024-04-25T00:18:34.925 INFO:tasks.workunit.client.0.smithi107.stdout:3/670: rmdir d9/d88/d4e 39 2024-04-25T00:18:34.927 INFO:tasks.workunit.client.0.smithi107.stdout:6/590: unlink d1/d2/d2b/d3e/c5c 0 2024-04-25T00:18:34.927 INFO:tasks.workunit.client.0.smithi107.stdout:5/723: symlink d4/dc/d26/d29/d2e/lec 0 2024-04-25T00:18:34.928 INFO:tasks.workunit.client.0.smithi107.stdout:3/671: creat d9/d54/d5c/d68/fd6 x:0 0 0 2024-04-25T00:18:34.929 INFO:tasks.workunit.client.0.smithi107.stdout:6/591: unlink d1/d2/d3/fbc 0 2024-04-25T00:18:34.929 INFO:tasks.workunit.client.0.smithi107.stdout:6/592: truncate d1/d2/d2b/d3e/d5b/d7a/faf 417782 0 2024-04-25T00:18:34.929 INFO:tasks.workunit.client.0.smithi107.stdout:6/593: write d1/d2/db/d32/d33/d68/d75/f8b [2958376,99226] 0 2024-04-25T00:18:34.930 INFO:tasks.workunit.client.0.smithi107.stdout:9/775: read d0/d8a/d93/d56/d76/d50/fdb [2481308,41861] 0 2024-04-25T00:18:34.930 INFO:tasks.workunit.client.0.smithi107.stdout:9/776: dread - d0/d8a/d93/d6f/ff9 zero size 2024-04-25T00:18:34.930 INFO:tasks.workunit.client.0.smithi107.stdout:5/724: creat d4/dc/d26/d29/d51/d78/fed x:0 0 0 2024-04-25T00:18:34.931 INFO:tasks.workunit.client.0.smithi107.stdout:3/672: creat d9/d88/d4e/fd7 x:0 0 0 2024-04-25T00:18:34.931 INFO:tasks.workunit.client.0.smithi107.stdout:3/673: fdatasync d9/d88/d4e/d93/f9d 0 2024-04-25T00:18:34.931 INFO:tasks.workunit.client.0.smithi107.stdout:9/777: mkdir d0/d8a/d93/de/d5c/d95/d106 0 2024-04-25T00:18:34.932 INFO:tasks.workunit.client.0.smithi107.stdout:9/778: fsync d0/d8a/d93/d56/d6c/d80/d9b/ff0 0 2024-04-25T00:18:34.932 INFO:tasks.workunit.client.0.smithi107.stdout:9/779: write d0/d8a/d93/de/d5c/d95/f2f [252907,16075] 0 2024-04-25T00:18:34.932 INFO:tasks.workunit.client.0.smithi107.stdout:9/780: write d0/d8a/d93/d6f/ff9 [761346,16646] 0 2024-04-25T00:18:34.937 INFO:tasks.workunit.client.0.smithi107.stdout:2/728: rename d6/dad/d67/da8/cbb to d6/d18/d40/d42/d69/d6e/da2/dc5/cff 0 2024-04-25T00:18:34.939 INFO:tasks.workunit.client.0.smithi107.stdout:3/674: read d9/d88/d2c/f37 [2870592,75061] 0 2024-04-25T00:18:34.940 INFO:tasks.workunit.client.0.smithi107.stdout:0/658: rename dc/dd/f15 to dc/dd/d5d/d3f/da5/fcf 0 2024-04-25T00:18:34.940 INFO:tasks.workunit.client.0.smithi107.stdout:0/659: stat dc/dd/ce 0 2024-04-25T00:18:34.940 INFO:tasks.workunit.client.0.smithi107.stdout:0/660: fsync dc/d24/d38/d64/f88 0 2024-04-25T00:18:34.942 INFO:tasks.workunit.client.0.smithi107.stdout:2/729: truncate d6/d20/d22/fa0 3257024 0 2024-04-25T00:18:34.942 INFO:tasks.workunit.client.0.smithi107.stdout:2/730: chown d6/d18/d40/d42/d69/d6e/da2/dc5 0 1 2024-04-25T00:18:34.942 INFO:tasks.workunit.client.0.smithi107.stdout:2/731: chown d6/d18/d40/d42/d2b/d30/d45/dca 9 1 2024-04-25T00:18:34.943 INFO:tasks.workunit.client.0.smithi107.stdout:3/675: mkdir d9/d15/da1/d4b/dd0/dd8 0 2024-04-25T00:18:34.944 INFO:tasks.workunit.client.0.smithi107.stdout:3/676: truncate d9/d88/d3a/fb1 923571 0 2024-04-25T00:18:34.944 INFO:tasks.workunit.client.0.smithi107.stdout:0/661: mknod dc/d24/d38/cd0 0 2024-04-25T00:18:34.946 INFO:tasks.workunit.client.0.smithi107.stdout:2/732: mkdir d6/d18/d40/d42/d69/d6e/da2/dc5/dd2/d100 0 2024-04-25T00:18:34.946 INFO:tasks.workunit.client.0.smithi107.stdout:0/662: mknod dc/dd/d16/d5e/cd1 0 2024-04-25T00:18:34.946 INFO:tasks.workunit.client.0.smithi107.stdout:0/663: write dc/d24/d25/d31/fb5 [157918,101892] 0 2024-04-25T00:18:34.948 INFO:tasks.workunit.client.0.smithi107.stdout:3/677: dread d9/f9b [0,4194304] 0 2024-04-25T00:18:34.953 INFO:tasks.workunit.client.0.smithi107.stdout:6/594: dwrite d1/d2/db/d32/d36/f99 [0,4194304] 0 2024-04-25T00:18:34.954 INFO:tasks.workunit.client.0.smithi107.stdout:6/595: creat d1/d2/db/d32/fc2 x:0 0 0 2024-04-25T00:18:34.955 INFO:tasks.workunit.client.0.smithi107.stdout:6/596: write d1/d4c/fa6 [4657410,83814] 0 2024-04-25T00:18:34.955 INFO:tasks.workunit.client.0.smithi107.stdout:6/597: chown d1/d2/d3/d28/l2a 75 1 2024-04-25T00:18:34.957 INFO:tasks.workunit.client.0.smithi107.stdout:9/781: dwrite d0/d8a/d93/d6f/da1/fce [0,4194304] 0 2024-04-25T00:18:34.957 INFO:tasks.workunit.client.0.smithi107.stdout:6/598: creat d1/d2/db/d32/d33/fc3 x:0 0 0 2024-04-25T00:18:34.959 INFO:tasks.workunit.client.0.smithi107.stdout:6/599: link d1/d2/d3/f31 d1/d2/d3/d71/d94/fc4 0 2024-04-25T00:18:34.960 INFO:tasks.workunit.client.0.smithi107.stdout:6/600: mknod d1/d2/db/d32/d36/cc5 0 2024-04-25T00:18:34.960 INFO:tasks.workunit.client.0.smithi107.stdout:6/601: write d1/d2/d3/d71/d94/fc4 [2313334,69239] 0 2024-04-25T00:18:34.961 INFO:tasks.workunit.client.0.smithi107.stdout:6/602: dread - d1/d2/db/d32/d33/fc3 zero size 2024-04-25T00:18:34.961 INFO:tasks.workunit.client.0.smithi107.stdout:0/664: dread dc/dd/d16/f2a [0,4194304] 0 2024-04-25T00:18:34.962 INFO:tasks.workunit.client.0.smithi107.stdout:6/603: unlink d1/fb1 0 2024-04-25T00:18:34.963 INFO:tasks.workunit.client.0.smithi107.stdout:0/665: symlink dc/dd/d16/d33/d35/d5f/ld2 0 2024-04-25T00:18:34.964 INFO:tasks.workunit.client.0.smithi107.stdout:6/604: creat d1/d2/db/d32/fc6 x:0 0 0 2024-04-25T00:18:34.965 INFO:tasks.workunit.client.0.smithi107.stdout:0/666: mknod dc/dd/d16/d48/d83/d9f/db0/cd3 0 2024-04-25T00:18:34.966 INFO:tasks.workunit.client.0.smithi107.stdout:6/605: mknod d1/d2/d3/d28/d79/cc7 0 2024-04-25T00:18:34.971 INFO:tasks.workunit.client.0.smithi107.stdout:5/725: link d4/d1c/l97 d4/d17/db3/d84/lee 0 2024-04-25T00:18:34.972 INFO:tasks.workunit.client.0.smithi107.stdout:5/726: mknod d4/d7c/d88/d9d/deb/cef 0 2024-04-25T00:18:34.973 INFO:tasks.workunit.client.0.smithi107.stdout:5/727: getdents d4/dc/d26/d29/d51 0 2024-04-25T00:18:34.973 INFO:tasks.workunit.client.0.smithi107.stdout:5/728: read - d4/dc/d26/d29/d2e/d55/fcb zero size 2024-04-25T00:18:34.973 INFO:tasks.workunit.client.0.smithi107.stdout:5/729: write d4/dc/d26/d29/d2e/d55/f9a [851715,33238] 0 2024-04-25T00:18:34.977 INFO:tasks.workunit.client.0.smithi107.stdout:3/678: rename d9/d54/d5c/d68/d3b/c90 to d9/d15/cd9 0 2024-04-25T00:18:34.979 INFO:tasks.workunit.client.0.smithi107.stdout:9/782: rename d0/d8a/d93/d34/l39 to d0/d8a/d93/d56/d76/d50/dd8/l107 0 2024-04-25T00:18:34.980 INFO:tasks.workunit.client.0.smithi107.stdout:9/783: creat d0/d8a/d93/d56/d6c/d80/d9b/db2/f108 x:0 0 0 2024-04-25T00:18:34.981 INFO:tasks.workunit.client.0.smithi107.stdout:9/784: creat d0/d8a/d93/d56/f109 x:0 0 0 2024-04-25T00:18:34.982 INFO:tasks.workunit.client.0.smithi107.stdout:9/785: mknod d0/d8a/d93/de/d5c/c10a 0 2024-04-25T00:18:34.983 INFO:tasks.workunit.client.0.smithi107.stdout:3/679: unlink d9/d88/d2c/l87 0 2024-04-25T00:18:34.983 INFO:tasks.workunit.client.0.smithi107.stdout:9/786: creat d0/d8a/d93/d34/d36/d77/dac/dee/f10b x:0 0 0 2024-04-25T00:18:34.983 INFO:tasks.workunit.client.0.smithi107.stdout:0/667: dwrite dc/dd/d16/d33/d35/f9e [0,4194304] 0 2024-04-25T00:18:34.988 INFO:tasks.workunit.client.0.smithi107.stdout:0/668: unlink dc/dd/d16/d5e/fb1 0 2024-04-25T00:18:34.989 INFO:tasks.workunit.client.0.smithi107.stdout:6/606: dwrite d1/d2/db/d32/fc6 [0,4194304] 0 2024-04-25T00:18:34.989 INFO:tasks.workunit.client.0.smithi107.stdout:6/607: fsync d1/d2/db/d32/d33/f63 0 2024-04-25T00:18:34.992 INFO:tasks.workunit.client.0.smithi107.stdout:6/608: symlink d1/d2/db/d32/d33/d68/lc8 0 2024-04-25T00:18:35.003 INFO:tasks.workunit.client.0.smithi107.stdout:3/680: dread d9/d88/d2c/f75 [0,4194304] 0 2024-04-25T00:18:35.003 INFO:tasks.workunit.client.0.smithi107.stdout:3/681: fsync d9/d22/d1b/d55/f5b 0 2024-04-25T00:18:35.003 INFO:tasks.workunit.client.0.smithi107.stdout:3/682: dread - d9/d54/d5c/d68/d3b/f6a zero size 2024-04-25T00:18:35.004 INFO:tasks.workunit.client.0.smithi107.stdout:3/683: mkdir d9/d88/d4e/dda 0 2024-04-25T00:18:35.005 INFO:tasks.workunit.client.0.smithi107.stdout:9/787: dwrite d0/d8a/d93/de/d5c/d95/fef [0,4194304] 0 2024-04-25T00:18:35.006 INFO:tasks.workunit.client.0.smithi107.stdout:5/730: rename d4/d35 to d4/d17/db3/d84/df0 0 2024-04-25T00:18:35.006 INFO:tasks.workunit.client.0.smithi107.stdout:3/684: creat d9/d88/d4e/d93/fdb x:0 0 0 2024-04-25T00:18:35.006 INFO:tasks.workunit.client.0.smithi107.stdout:3/685: chown d9/cd 0 1 2024-04-25T00:18:35.007 INFO:tasks.workunit.client.0.smithi107.stdout:3/686: fdatasync d9/d15/da1/d4b/fcd 0 2024-04-25T00:18:35.007 INFO:tasks.workunit.client.0.smithi107.stdout:3/687: chown d9/d15/d39/f9f 424 1 2024-04-25T00:18:35.007 INFO:tasks.workunit.client.0.smithi107.stdout:3/688: readlink d9/d54/d5c/lb2 0 2024-04-25T00:18:35.009 INFO:tasks.workunit.client.0.smithi107.stdout:5/731: mkdir d4/dc/d83/df1 0 2024-04-25T00:18:35.010 INFO:tasks.workunit.client.0.smithi107.stdout:3/689: mknod d9/d54/d94/dba/dcc/cdc 0 2024-04-25T00:18:35.013 INFO:tasks.workunit.client.0.smithi107.stdout:3/690: unlink d9/d54/d5c/cb6 0 2024-04-25T00:18:35.014 INFO:tasks.workunit.client.0.smithi107.stdout:3/691: unlink d9/d15/fa7 0 2024-04-25T00:18:35.014 INFO:tasks.workunit.client.0.smithi107.stdout:3/692: stat d9/d88/d2c/f75 0 2024-04-25T00:18:35.014 INFO:tasks.workunit.client.0.smithi107.stdout:3/693: chown d9/d15/d39/fb7 7 1 2024-04-25T00:18:35.017 INFO:tasks.workunit.client.0.smithi107.stdout:5/732: dread d4/d17/f2b [0,4194304] 0 2024-04-25T00:18:35.017 INFO:tasks.workunit.client.0.smithi107.stdout:6/609: dwrite d1/d2/db/d32/d36/d3a/fa7 [0,4194304] 0 2024-04-25T00:18:35.018 INFO:tasks.workunit.client.0.smithi107.stdout:6/610: mkdir d1/d2/db/d3c/d43/d96/dc9 0 2024-04-25T00:18:35.019 INFO:tasks.workunit.client.0.smithi107.stdout:6/611: mkdir d1/d2/db/d32/dca 0 2024-04-25T00:18:35.020 INFO:tasks.workunit.client.0.smithi107.stdout:6/612: read - d1/d2/d2b/d3e/fa4 zero size 2024-04-25T00:18:35.020 INFO:tasks.workunit.client.0.smithi107.stdout:6/613: chown d1/d2/db 472 1 2024-04-25T00:18:35.026 INFO:tasks.workunit.client.0.smithi107.stdout:3/694: dread d9/d22/d1b/f48 [0,4194304] 0 2024-04-25T00:18:35.030 INFO:tasks.workunit.client.0.smithi107.stdout:3/695: symlink d9/d54/d5c/d68/d3b/ldd 0 2024-04-25T00:18:35.030 INFO:tasks.workunit.client.0.smithi107.stdout:3/696: readlink d9/d15/ld1 0 2024-04-25T00:18:35.030 INFO:tasks.workunit.client.0.smithi107.stdout:0/669: rename dc/dd/d16/d33/f75 to dc/dd/d5d/d3f/dc7/fd4 0 2024-04-25T00:18:35.030 INFO:tasks.workunit.client.0.smithi107.stdout:9/788: dwrite d0/d8a/d93/d6f/da1/df2/f100 [0,4194304] 0 2024-04-25T00:18:35.032 INFO:tasks.workunit.client.0.smithi107.stdout:0/670: creat dc/dd/d16/d48/d83/d9f/dc4/fd5 x:0 0 0 2024-04-25T00:18:35.032 INFO:tasks.workunit.client.0.smithi107.stdout:6/614: dread d1/d2/d3/f41 [0,4194304] 0 2024-04-25T00:18:35.032 INFO:tasks.workunit.client.0.smithi107.stdout:6/615: chown d1/d14 2 1 2024-04-25T00:18:35.032 INFO:tasks.workunit.client.0.smithi107.stdout:0/671: readlink dc/d24/d25/l32 0 2024-04-25T00:18:35.032 INFO:tasks.workunit.client.0.smithi107.stdout:6/616: truncate d1/d2/db/d32/d36/d3d/f5e 4589445 0 2024-04-25T00:18:35.032 INFO:tasks.workunit.client.0.smithi107.stdout:6/617: truncate d1/d2/d3/d28/d79/fb8 4383335 0 2024-04-25T00:18:35.034 INFO:tasks.workunit.client.0.smithi107.stdout:0/672: link dc/d24/d38/d64/f88 dc/da6/fd6 0 2024-04-25T00:18:35.034 INFO:tasks.workunit.client.0.smithi107.stdout:0/673: fsync dc/d24/d38/d64/f68 0 2024-04-25T00:18:35.037 INFO:tasks.workunit.client.0.smithi107.stdout:9/789: dread d0/d8a/d93/d56/d76/d50/fdb [0,4194304] 0 2024-04-25T00:18:35.037 INFO:tasks.workunit.client.0.smithi107.stdout:3/697: mknod d9/d54/d5c/d68/d3b/cde 0 2024-04-25T00:18:35.039 INFO:tasks.workunit.client.0.smithi107.stdout:3/698: creat d9/d15/da1/d4b/dd0/fdf x:0 0 0 2024-04-25T00:18:35.052 INFO:tasks.workunit.client.0.smithi107.stdout:6/618: dwrite d1/d2/d3/d71/d94/fc4 [0,4194304] 0 2024-04-25T00:18:35.055 INFO:tasks.workunit.client.0.smithi107.stdout:6/619: truncate d1/d2/d3/d71/d94/f9b 3832938 0 2024-04-25T00:18:35.058 INFO:tasks.workunit.client.0.smithi107.stdout:3/699: dwrite d9/d22/d1b/f81 [4194304,4194304] 0 2024-04-25T00:18:35.058 INFO:tasks.workunit.client.0.smithi107.stdout:3/700: truncate d9/f20 6218103 0 2024-04-25T00:18:35.059 INFO:tasks.workunit.client.0.smithi107.stdout:0/674: dwrite dc/dd/d5d/f87 [0,4194304] 0 2024-04-25T00:18:35.064 INFO:tasks.workunit.client.0.smithi107.stdout:8/772: sync 2024-04-25T00:18:35.064 INFO:tasks.workunit.client.0.smithi107.stdout:1/675: sync 2024-04-25T00:18:35.064 INFO:tasks.workunit.client.0.smithi107.stdout:4/755: sync 2024-04-25T00:18:35.064 INFO:tasks.workunit.client.0.smithi107.stdout:7/701: sync 2024-04-25T00:18:35.064 INFO:tasks.workunit.client.0.smithi107.stdout:1/676: fdatasync d2/da/d2f/f9b 0 2024-04-25T00:18:35.064 INFO:tasks.workunit.client.0.smithi107.stdout:4/756: dread - d3/d5/d32/ff4 zero size 2024-04-25T00:18:35.064 INFO:tasks.workunit.client.0.smithi107.stdout:7/702: fsync d0/fc8 0 2024-04-25T00:18:35.064 INFO:tasks.workunit.client.0.smithi107.stdout:3/701: dread d9/d88/f32 [0,4194304] 0 2024-04-25T00:18:35.065 INFO:tasks.workunit.client.0.smithi107.stdout:8/773: rename d2/d18/d1a/d5e/d3e/fc9 to d2/d18/d1a/d5e/d3d/f109 0 2024-04-25T00:18:35.066 INFO:tasks.workunit.client.0.smithi107.stdout:4/757: write d3/d24/d61/f70 [202718,83254] 0 2024-04-25T00:18:35.066 INFO:tasks.workunit.client.0.smithi107.stdout:4/758: fsync d3/d5/f4d 0 2024-04-25T00:18:35.066 INFO:tasks.workunit.client.0.smithi107.stdout:1/677: rmdir d2/d2a/d5c/d89 39 2024-04-25T00:18:35.066 INFO:tasks.workunit.client.0.smithi107.stdout:1/678: write d2/d2a/d37/d88/fb2 [799028,83712] 0 2024-04-25T00:18:35.067 INFO:tasks.workunit.client.0.smithi107.stdout:4/759: rename d3/d24/d25/c2e to d3/d5/d32/cfc 0 2024-04-25T00:18:35.067 INFO:tasks.workunit.client.0.smithi107.stdout:3/702: symlink d9/d54/d94/dba/le0 0 2024-04-25T00:18:35.068 INFO:tasks.workunit.client.0.smithi107.stdout:6/620: link d1/d14/d18/f2c d1/d2/db/d32/d36/fcb 0 2024-04-25T00:18:35.068 INFO:tasks.workunit.client.0.smithi107.stdout:8/774: creat d2/d18/d1a/d5e/d6d/f10a x:0 0 0 2024-04-25T00:18:35.069 INFO:tasks.workunit.client.0.smithi107.stdout:6/621: mknod d1/d2/d2b/d3e/d5b/d7a/ccc 0 2024-04-25T00:18:35.070 INFO:tasks.workunit.client.0.smithi107.stdout:1/679: mknod d2/d2a/d5c/d89/dc0/ccf 0 2024-04-25T00:18:35.071 INFO:tasks.workunit.client.0.smithi107.stdout:8/775: creat d2/d18/d1a/d5e/d5c/d91/de7/f10b x:0 0 0 2024-04-25T00:18:35.072 INFO:tasks.workunit.client.0.smithi107.stdout:8/776: creat d2/d18/d1a/d5e/d5c/d107/dc2/f10c x:0 0 0 2024-04-25T00:18:35.074 INFO:tasks.workunit.client.0.smithi107.stdout:8/777: rmdir d2/d18/d1a/d5e/d5c/d107/dc2 39 2024-04-25T00:18:35.074 INFO:tasks.workunit.client.0.smithi107.stdout:8/778: dread - d2/d18/ddd/fdf zero size 2024-04-25T00:18:35.074 INFO:tasks.workunit.client.0.smithi107.stdout:8/779: truncate d2/d18/d1a/d5e/d5c/f88 5044432 0 2024-04-25T00:18:35.074 INFO:tasks.workunit.client.0.smithi107.stdout:8/780: fsync d2/d18/d1a/d5e/d3e/fcc 0 2024-04-25T00:18:35.074 INFO:tasks.workunit.client.0.smithi107.stdout:8/781: truncate d2/d18/d1a/d5e/d5c/d91/da3/fb8 999943 0 2024-04-25T00:18:35.074 INFO:tasks.workunit.client.0.smithi107.stdout:8/782: readlink d2/d18/d1a/d5e/d5c/l7f 0 2024-04-25T00:18:35.074 INFO:tasks.workunit.client.0.smithi107.stdout:6/622: symlink d1/d2/d3/d28/lcd 0 2024-04-25T00:18:35.074 INFO:tasks.workunit.client.0.smithi107.stdout:6/623: write d1/d2/db/d32/d33/fc3 [616528,38129] 0 2024-04-25T00:18:35.074 INFO:tasks.workunit.client.0.smithi107.stdout:6/624: truncate d1/d2/f5f 4680762 0 2024-04-25T00:18:35.074 INFO:tasks.workunit.client.0.smithi107.stdout:6/625: write d1/d2/db/d32/fb7 [1388883,5929] 0 2024-04-25T00:18:35.075 INFO:tasks.workunit.client.0.smithi107.stdout:8/783: rename d2/d18/d1a/d5e/d5c/cd4 to d2/d18/d1a/d5e/d5c/c10d 0 2024-04-25T00:18:35.075 INFO:tasks.workunit.client.0.smithi107.stdout:6/626: creat d1/d2/db/d32/dca/fce x:0 0 0 2024-04-25T00:18:35.076 INFO:tasks.workunit.client.0.smithi107.stdout:8/784: link d2/d18/d1a/d5e/dd6/de6/ceb d2/d18/d1a/d5e/d3d/d85/c10e 0 2024-04-25T00:18:35.076 INFO:tasks.workunit.client.0.smithi107.stdout:3/703: write d9/d88/d2c/f75 [71740,13488] 0 2024-04-25T00:18:35.079 INFO:tasks.workunit.client.0.smithi107.stdout:8/785: creat d2/d18/d1a/d33/f10f x:0 0 0 2024-04-25T00:18:35.079 INFO:tasks.workunit.client.0.smithi107.stdout:8/786: readlink d2/l19 0 2024-04-25T00:18:35.081 INFO:tasks.workunit.client.0.smithi107.stdout:8/787: mkdir d2/d18/d1a/d5e/d4f/da0/d110 0 2024-04-25T00:18:35.082 INFO:tasks.workunit.client.0.smithi107.stdout:8/788: mknod d2/d18/d1a/d5e/d3e/db1/c111 0 2024-04-25T00:18:35.083 INFO:tasks.workunit.client.0.smithi107.stdout:8/789: dread - d2/d18/d1a/d5e/d3d/d85/dae/fee zero size 2024-04-25T00:18:35.084 INFO:tasks.workunit.client.0.smithi107.stdout:6/627: write d1/d4c/fa6 [2243159,113082] 0 2024-04-25T00:18:35.084 INFO:tasks.workunit.client.0.smithi107.stdout:7/703: dread d0/d8/d4e/d46/d93/fb0 [0,4194304] 0 2024-04-25T00:18:35.085 INFO:tasks.workunit.client.0.smithi107.stdout:0/675: dwrite dc/d24/d38/d64/f68 [0,4194304] 0 2024-04-25T00:18:35.085 INFO:tasks.workunit.client.0.smithi107.stdout:0/676: fdatasync dc/da6/fd6 0 2024-04-25T00:18:35.085 INFO:tasks.workunit.client.0.smithi107.stdout:7/704: dread - d0/d8/db/d18/d19/d47/f5b zero size 2024-04-25T00:18:35.085 INFO:tasks.workunit.client.0.smithi107.stdout:7/705: dread - d0/dc1/fd3 zero size 2024-04-25T00:18:35.085 INFO:tasks.workunit.client.0.smithi107.stdout:0/677: chown dc/dd/d16/d48 41379 1 2024-04-25T00:18:35.085 INFO:tasks.workunit.client.0.smithi107.stdout:8/790: symlink d2/d18/d1a/d5e/d4f/da0/da1/de8/l112 0 2024-04-25T00:18:35.086 INFO:tasks.workunit.client.0.smithi107.stdout:8/791: read d2/d18/d1a/f80 [117698,64289] 0 2024-04-25T00:18:35.086 INFO:tasks.workunit.client.0.smithi107.stdout:8/792: write d2/d18/d1a/d5e/d3d/f9e [362126,32768] 0 2024-04-25T00:18:35.087 INFO:tasks.workunit.client.0.smithi107.stdout:7/706: mknod d0/d80/da2/cec 0 2024-04-25T00:18:35.087 INFO:tasks.workunit.client.0.smithi107.stdout:7/707: creat d0/d8/db/d18/d19/d47/de0/fed x:0 0 0 2024-04-25T00:18:35.087 INFO:tasks.workunit.client.0.smithi107.stdout:7/708: write d0/d8/db/d18/d19/f23 [4962545,68844] 0 2024-04-25T00:18:35.087 INFO:tasks.workunit.client.0.smithi107.stdout:1/680: dwrite d2/da/d2f/f9b [0,4194304] 0 2024-04-25T00:18:35.088 INFO:tasks.workunit.client.0.smithi107.stdout:0/678: mkdir dc/d24/d38/d8b/dd7 0 2024-04-25T00:18:35.088 INFO:tasks.workunit.client.0.smithi107.stdout:4/760: dwrite d3/d24/fe5 [0,4194304] 0 2024-04-25T00:18:35.089 INFO:tasks.workunit.client.0.smithi107.stdout:4/761: write d3/d24/d5c/fe1 [618802,60264] 0 2024-04-25T00:18:35.090 INFO:tasks.workunit.client.0.smithi107.stdout:4/762: stat d3/da8/fb7 0 2024-04-25T00:18:35.091 INFO:tasks.workunit.client.0.smithi107.stdout:4/763: write d3/d24/d25/d71/dc2/ffb [512722,118427] 0 2024-04-25T00:18:35.091 INFO:tasks.workunit.client.0.smithi107.stdout:4/764: chown d3/d5/d32/d53/d7e/fd4 24 1 2024-04-25T00:18:35.093 INFO:tasks.workunit.client.0.smithi107.stdout:1/681: unlink d2/dd/d25/d5e/c6b 0 2024-04-25T00:18:35.095 INFO:tasks.workunit.client.0.smithi107.stdout:8/793: rmdir d2/d18/d1a/d5e/d4f 39 2024-04-25T00:18:35.095 INFO:tasks.workunit.client.0.smithi107.stdout:8/794: chown d2/d18/d1a/c2c 109118 1 2024-04-25T00:18:35.096 INFO:tasks.workunit.client.0.smithi107.stdout:8/795: fsync d2/f55 0 2024-04-25T00:18:35.096 INFO:tasks.workunit.client.0.smithi107.stdout:4/765: creat d3/d10/d7b/ffd x:0 0 0 2024-04-25T00:18:35.097 INFO:tasks.workunit.client.0.smithi107.stdout:0/679: creat dc/dd/fd8 x:0 0 0 2024-04-25T00:18:35.098 INFO:tasks.workunit.client.0.smithi107.stdout:0/680: chown dc/dd/d16/d33/d35/c47 26 1 2024-04-25T00:18:35.099 INFO:tasks.workunit.client.0.smithi107.stdout:3/704: getdents d9/d88 0 2024-04-25T00:18:35.101 INFO:tasks.workunit.client.0.smithi107.stdout:6/628: dwrite d1/d2/d3/f13 [0,4194304] 0 2024-04-25T00:18:35.102 INFO:tasks.workunit.client.0.smithi107.stdout:6/629: mknod d1/d2/db/d3c/d43/d96/dc9/ccf 0 2024-04-25T00:18:35.103 INFO:tasks.workunit.client.0.smithi107.stdout:7/709: write d0/f1d [2481476,92222] 0 2024-04-25T00:18:35.107 INFO:tasks.workunit.client.0.smithi107.stdout:7/710: mknod d0/d8/db/d18/d19/d47/de0/cee 0 2024-04-25T00:18:35.109 INFO:tasks.workunit.client.0.smithi107.stdout:7/711: creat d0/d8/d4e/d46/fef x:0 0 0 2024-04-25T00:18:35.109 INFO:tasks.workunit.client.0.smithi107.stdout:7/712: truncate d0/d8/d2c/d58/f5f 368320 0 2024-04-25T00:18:35.109 INFO:tasks.workunit.client.0.smithi107.stdout:4/766: dread d3/da8/fb7 [4194304,4194304] 0 2024-04-25T00:18:35.109 INFO:tasks.workunit.client.0.smithi107.stdout:4/767: write d3/d5/f92 [1899596,11653] 0 2024-04-25T00:18:35.109 INFO:tasks.workunit.client.0.smithi107.stdout:7/713: creat d0/d8/db/d18/d19/d47/da1/dd4/ff0 x:0 0 0 2024-04-25T00:18:35.109 INFO:tasks.workunit.client.0.smithi107.stdout:4/768: creat d3/d5/d32/d53/d97/ffe x:0 0 0 2024-04-25T00:18:35.110 INFO:tasks.workunit.client.0.smithi107.stdout:7/714: symlink d0/d8/db/d18/d19/d67/lf1 0 2024-04-25T00:18:35.110 INFO:tasks.workunit.client.0.smithi107.stdout:7/715: chown d0/dc1 140 1 2024-04-25T00:18:35.115 INFO:tasks.workunit.client.0.smithi107.stdout:0/681: dread dc/dd/d16/d33/f37 [0,4194304] 0 2024-04-25T00:18:35.117 INFO:tasks.workunit.client.0.smithi107.stdout:8/796: dwrite d2/d18/d1a/fdc [4194304,4194304] 0 2024-04-25T00:18:35.117 INFO:tasks.workunit.client.0.smithi107.stdout:3/705: dread d9/d15/da1/f65 [0,4194304] 0 2024-04-25T00:18:35.117 INFO:tasks.workunit.client.0.smithi107.stdout:0/682: mknod dc/dd/d5d/d5c/cd9 0 2024-04-25T00:18:35.120 INFO:tasks.workunit.client.0.smithi107.stdout:0/683: getdents dc/d24/d38/d64/d78 0 2024-04-25T00:18:35.123 INFO:tasks.workunit.client.0.smithi107.stdout:8/797: symlink d2/d18/d1a/d5e/d4f/da0/da1/l113 0 2024-04-25T00:18:35.123 INFO:tasks.workunit.client.0.smithi107.stdout:8/798: write d2/f45 [2554895,120156] 0 2024-04-25T00:18:35.126 INFO:tasks.workunit.client.0.smithi107.stdout:8/799: symlink d2/d18/d1a/d5e/d3d/d85/dae/l114 0 2024-04-25T00:18:35.130 INFO:tasks.workunit.client.0.smithi107.stdout:6/630: dread d1/d2/db/f4d [0,4194304] 0 2024-04-25T00:18:35.132 INFO:tasks.workunit.client.0.smithi107.stdout:6/631: truncate d1/d2/db/d3c/f50 4056087 0 2024-04-25T00:18:35.133 INFO:tasks.workunit.client.0.smithi107.stdout:1/682: dwrite d2/dd/d25/f50 [0,4194304] 0 2024-04-25T00:18:35.133 INFO:tasks.workunit.client.0.smithi107.stdout:1/683: stat d2/d2a/d5c/d89 0 2024-04-25T00:18:35.133 INFO:tasks.workunit.client.0.smithi107.stdout:3/706: read d9/d54/fc4 [1476923,15828] 0 2024-04-25T00:18:35.134 INFO:tasks.workunit.client.0.smithi107.stdout:6/632: mkdir d1/d2/d3/d28/d52/dd0 0 2024-04-25T00:18:35.134 INFO:tasks.workunit.client.0.smithi107.stdout:6/633: chown d1/d2/db/d32/d33/d68 4 1 2024-04-25T00:18:35.134 INFO:tasks.workunit.client.0.smithi107.stdout:6/634: stat d1/d2/d3/d28/f53 0 2024-04-25T00:18:35.134 INFO:tasks.workunit.client.0.smithi107.stdout:6/635: truncate d1/d2/db/f4d 4887921 0 2024-04-25T00:18:35.134 INFO:tasks.workunit.client.0.smithi107.stdout:6/636: fdatasync d1/d2/db/d32/d36/f46 0 2024-04-25T00:18:35.137 INFO:tasks.workunit.client.0.smithi107.stdout:6/637: mknod d1/d2/d2b/d3e/d5b/d7a/cd1 0 2024-04-25T00:18:35.137 INFO:tasks.workunit.client.0.smithi107.stdout:6/638: chown d1/d2/d3/d28/d79/l98 693 1 2024-04-25T00:18:35.138 INFO:tasks.workunit.client.0.smithi107.stdout:7/716: dwrite d0/d8/db/d18/d19/f8e [0,4194304] 0 2024-04-25T00:18:35.139 INFO:tasks.workunit.client.0.smithi107.stdout:3/707: creat d9/d15/da1/db0/fe1 x:0 0 0 2024-04-25T00:18:35.140 INFO:tasks.workunit.client.0.smithi107.stdout:6/639: creat d1/d2/db/d48/fd2 x:0 0 0 2024-04-25T00:18:35.140 INFO:tasks.workunit.client.0.smithi107.stdout:6/640: fdatasync d1/d2/d3/d26/f35 0 2024-04-25T00:18:35.140 INFO:tasks.workunit.client.0.smithi107.stdout:6/641: stat d1/d2/db/d3c/d43/d96/cbf 0 2024-04-25T00:18:35.140 INFO:tasks.workunit.client.0.smithi107.stdout:6/642: read - d1/d2/db/d3c/f8d zero size 2024-04-25T00:18:35.144 INFO:tasks.workunit.client.0.smithi107.stdout:0/684: dwrite dc/d24/d25/d31/fb5 [0,4194304] 0 2024-04-25T00:18:35.152 INFO:tasks.workunit.client.0.smithi107.stdout:6/643: truncate d1/d2/db/fd 6369968 0 2024-04-25T00:18:35.156 INFO:tasks.workunit.client.0.smithi107.stdout:8/800: dread d2/d18/d1a/f8e [0,4194304] 0 2024-04-25T00:18:35.162 INFO:tasks.workunit.client.0.smithi107.stdout:1/684: dwrite d2/dd/d1a/f64 [0,4194304] 0 2024-04-25T00:18:35.162 INFO:tasks.workunit.client.0.smithi107.stdout:8/801: unlink d2/d18/d1a/d5e/d4f/c75 0 2024-04-25T00:18:35.162 INFO:tasks.workunit.client.0.smithi107.stdout:8/802: chown d2/d18/d1a/d5e/d3e/f86 35126237 1 2024-04-25T00:18:35.162 INFO:tasks.workunit.client.0.smithi107.stdout:7/717: write d0/d8/db/d18/d19/d47/f31 [6743485,99094] 0 2024-04-25T00:18:35.164 INFO:tasks.workunit.client.0.smithi107.stdout:1/685: creat d2/dd/d25/d5e/fd0 x:0 0 0 2024-04-25T00:18:35.164 INFO:tasks.workunit.client.0.smithi107.stdout:1/686: stat d2/d62/d66/lac 0 2024-04-25T00:18:35.165 INFO:tasks.workunit.client.0.smithi107.stdout:3/708: dwrite d9/d54/d5c/d68/f46 [0,4194304] 0 2024-04-25T00:18:35.165 INFO:tasks.workunit.client.0.smithi107.stdout:3/709: truncate d9/d15/da1/f64 834951 0 2024-04-25T00:18:35.167 INFO:tasks.workunit.client.0.smithi107.stdout:8/803: link d2/d18/d1a/d82/fb6 d2/d18/d1a/d5e/d4f/da0/f115 0 2024-04-25T00:18:35.170 INFO:tasks.workunit.client.0.smithi107.stdout:1/687: creat d2/fd1 x:0 0 0 2024-04-25T00:18:35.171 INFO:tasks.workunit.client.0.smithi107.stdout:0/685: dwrite dc/d24/fbf [0,4194304] 0 2024-04-25T00:18:35.171 INFO:tasks.workunit.client.0.smithi107.stdout:0/686: readlink dc/d24/d38/d64/d78/la4 0 2024-04-25T00:18:35.171 INFO:tasks.workunit.client.0.smithi107.stdout:0/687: dread - dc/dd/d5d/d3f/fc8 zero size 2024-04-25T00:18:35.171 INFO:tasks.workunit.client.0.smithi107.stdout:0/688: write dc/dd/d16/d33/f37 [4400763,107195] 0 2024-04-25T00:18:35.172 INFO:tasks.workunit.client.0.smithi107.stdout:3/710: creat d9/d54/d94/dc9/dcb/fe2 x:0 0 0 2024-04-25T00:18:35.175 INFO:tasks.workunit.client.0.smithi107.stdout:7/718: dread d0/d8/db/d18/d19/d39/f6f [0,4194304] 0 2024-04-25T00:18:35.176 INFO:tasks.workunit.client.0.smithi107.stdout:8/804: dread d2/d18/d1a/d5e/d3d/f109 [0,4194304] 0 2024-04-25T00:18:35.177 INFO:tasks.workunit.client.0.smithi107.stdout:8/805: truncate d2/d18/d1a/d5e/d6d/f10a 746805 0 2024-04-25T00:18:35.177 INFO:tasks.workunit.client.0.smithi107.stdout:7/719: truncate d0/d8/db/d18/d19/fb7 1638279 0 2024-04-25T00:18:35.177 INFO:tasks.workunit.client.0.smithi107.stdout:6/644: dwrite d1/d2/d3/d71/d94/f9b [4194304,4194304] 0 2024-04-25T00:18:35.179 INFO:tasks.workunit.client.0.smithi107.stdout:7/720: getdents d0/d8/db/d18/d19/d47/de0/dc2 0 2024-04-25T00:18:35.179 INFO:tasks.workunit.client.0.smithi107.stdout:8/806: rmdir d2/d18/d1a/d5e/d3e/d74/dcf 0 2024-04-25T00:18:35.179 INFO:tasks.workunit.client.0.smithi107.stdout:0/689: write dc/d24/d25/f67 [201804,94890] 0 2024-04-25T00:18:35.180 INFO:tasks.workunit.client.0.smithi107.stdout:7/721: creat d0/d8/db/ff2 x:0 0 0 2024-04-25T00:18:35.181 INFO:tasks.workunit.client.0.smithi107.stdout:8/807: getdents d2/d18/d1a/d5e/d5c/d91/de7 0 2024-04-25T00:18:35.181 INFO:tasks.workunit.client.0.smithi107.stdout:3/711: dread d9/d22/d1b/d55/f5d [0,4194304] 0 2024-04-25T00:18:35.182 INFO:tasks.workunit.client.0.smithi107.stdout:3/712: mknod d9/d88/d3a/d91/dad/ce3 0 2024-04-25T00:18:35.183 INFO:tasks.workunit.client.0.smithi107.stdout:3/713: chown d9/d15/da1/d4b/fcd 308342633 1 2024-04-25T00:18:35.183 INFO:tasks.workunit.client.0.smithi107.stdout:3/714: chown d9/c36 1869 1 2024-04-25T00:18:35.183 INFO:tasks.workunit.client.0.smithi107.stdout:3/715: mknod d9/d54/d94/dba/ce4 0 2024-04-25T00:18:35.183 INFO:tasks.workunit.client.0.smithi107.stdout:3/716: fdatasync d9/d22/d1b/fc2 0 2024-04-25T00:18:35.184 INFO:tasks.workunit.client.0.smithi107.stdout:3/717: symlink d9/d15/da1/d4b/le5 0 2024-04-25T00:18:35.185 INFO:tasks.workunit.client.0.smithi107.stdout:3/718: write d9/d88/d4e/d93/f9d [3130161,85299] 0 2024-04-25T00:18:35.187 INFO:tasks.workunit.client.0.smithi107.stdout:6/645: write d1/d2/d3/d71/d94/fb0 [6054228,30083] 0 2024-04-25T00:18:35.187 INFO:tasks.workunit.client.0.smithi107.stdout:3/719: link d9/d22/d1b/f48 d9/d22/d1b/fe6 0 2024-04-25T00:18:35.188 INFO:tasks.workunit.client.0.smithi107.stdout:3/720: creat d9/d15/da1/d4b/dd0/dd8/fe7 x:0 0 0 2024-04-25T00:18:35.200 INFO:tasks.workunit.client.0.smithi107.stdout:8/808: dwrite d2/d18/d1a/d5e/d3d/fc4 [0,4194304] 0 2024-04-25T00:18:35.201 INFO:tasks.workunit.client.0.smithi107.stdout:8/809: mknod d2/d18/d1a/d5e/d3e/d74/c116 0 2024-04-25T00:18:35.205 INFO:tasks.workunit.client.0.smithi107.stdout:3/721: rename d9/d54/d5c/d68/d3b/l7c to d9/le8 0 2024-04-25T00:18:35.205 INFO:tasks.workunit.client.0.smithi107.stdout:3/722: write d9/d88/d80/fae [272765,44662] 0 2024-04-25T00:18:35.205 INFO:tasks.workunit.client.0.smithi107.stdout:3/723: read - d9/d15/da1/d4b/dd0/fdf zero size 2024-04-25T00:18:35.206 INFO:tasks.workunit.client.0.smithi107.stdout:3/724: getdents d9/d15/d39/d76 0 2024-04-25T00:18:35.206 INFO:tasks.workunit.client.0.smithi107.stdout:3/725: chown d9/d54/d5c/d68 33762 1 2024-04-25T00:18:35.208 INFO:tasks.workunit.client.0.smithi107.stdout:3/726: dread d9/f9b [0,4194304] 0 2024-04-25T00:18:35.209 INFO:tasks.workunit.client.0.smithi107.stdout:3/727: creat d9/d54/d94/dc9/dcb/fe9 x:0 0 0 2024-04-25T00:18:35.210 INFO:tasks.workunit.client.0.smithi107.stdout:3/728: creat d9/d15/da1/fea x:0 0 0 2024-04-25T00:18:35.211 INFO:tasks.workunit.client.0.smithi107.stdout:3/729: write d9/d88/d4e/fd7 [955769,92108] 0 2024-04-25T00:18:35.211 INFO:tasks.workunit.client.0.smithi107.stdout:3/730: dread d9/d22/f7a [0,4194304] 0 2024-04-25T00:18:35.216 INFO:tasks.workunit.client.0.smithi107.stdout:8/810: dwrite d2/d18/d1a/d5e/f56 [0,4194304] 0 2024-04-25T00:18:35.218 INFO:tasks.workunit.client.0.smithi107.stdout:8/811: mknod d2/d18/d1a/d5e/dd6/de6/c117 0 2024-04-25T00:18:35.220 INFO:tasks.workunit.client.0.smithi107.stdout:8/812: rename d2/d18/d1a/f31 to d2/d18/d1a/db7/f118 0 2024-04-25T00:18:35.221 INFO:tasks.workunit.client.0.smithi107.stdout:8/813: truncate d2/d18/d1a/db7/f118 7560220 0 2024-04-25T00:18:35.222 INFO:tasks.workunit.client.0.smithi107.stdout:8/814: mkdir d2/d18/d1a/d5e/d4f/da0/da1/de8/d119 0 2024-04-25T00:18:35.222 INFO:tasks.workunit.client.0.smithi107.stdout:3/731: dread d9/d22/d1b/f81 [4194304,4194304] 0 2024-04-25T00:18:35.223 INFO:tasks.workunit.client.0.smithi107.stdout:8/815: creat d2/f11a x:0 0 0 2024-04-25T00:18:35.223 INFO:tasks.workunit.client.0.smithi107.stdout:3/732: mkdir d9/d88/d3a/d91/dad/deb 0 2024-04-25T00:18:35.225 INFO:tasks.workunit.client.0.smithi107.stdout:8/816: rename d2/d18/d1a/d5e/c24 to d2/d18/d1a/d5e/dd6/c11b 0 2024-04-25T00:18:35.225 INFO:tasks.workunit.client.0.smithi107.stdout:8/817: stat d2/d18/d1a/d5e/d4f/dd1/cd3 0 2024-04-25T00:18:35.227 INFO:tasks.workunit.client.0.smithi107.stdout:8/818: write d2/d18/d1a/d5e/d3e/f86 [695225,67945] 0 2024-04-25T00:18:35.227 INFO:tasks.workunit.client.0.smithi107.stdout:8/819: read - d2/d18/d1a/d5e/d5c/d107/dc2/f10c zero size 2024-04-25T00:18:35.234 INFO:tasks.workunit.client.0.smithi107.stdout:3/733: dread d9/d54/d5c/d68/d3b/f47 [0,4194304] 0 2024-04-25T00:18:35.235 INFO:tasks.workunit.client.0.smithi107.stdout:3/734: symlink d9/d54/d5c/d68/lec 0 2024-04-25T00:18:35.245 INFO:tasks.workunit.client.0.smithi107.stdout:8/820: dwrite d2/d18/d1a/d5e/d5c/f88 [0,4194304] 0 2024-04-25T00:18:35.245 INFO:tasks.workunit.client.0.smithi107.stdout:8/821: truncate d2/d18/d1a/d5e/f47 2227388 0 2024-04-25T00:18:35.245 INFO:tasks.workunit.client.0.smithi107.stdout:8/822: creat d2/d18/d1a/d5e/d3e/db1/f11c x:0 0 0 2024-04-25T00:18:35.246 INFO:tasks.workunit.client.0.smithi107.stdout:8/823: creat d2/d18/d1a/d5e/d4f/da0/da1/db9/f11d x:0 0 0 2024-04-25T00:18:35.251 INFO:tasks.workunit.client.0.smithi107.stdout:3/735: dwrite d9/d22/f8b [0,4194304] 0 2024-04-25T00:18:35.254 INFO:tasks.workunit.client.0.smithi107.stdout:3/736: mknod d9/d22/d1b/d55/ced 0 2024-04-25T00:18:35.254 INFO:tasks.workunit.client.0.smithi107.stdout:3/737: stat d9 0 2024-04-25T00:18:35.254 INFO:tasks.workunit.client.0.smithi107.stdout:3/738: write d9/d88/d80/fae [1230074,56128] 0 2024-04-25T00:18:35.256 INFO:tasks.workunit.client.0.smithi107.stdout:8/824: dread d2/d18/d1a/dca/fcb [0,4194304] 0 2024-04-25T00:18:35.256 INFO:tasks.workunit.client.0.smithi107.stdout:3/739: symlink d9/d54/d94/dab/lee 0 2024-04-25T00:18:35.257 INFO:tasks.workunit.client.0.smithi107.stdout:8/825: symlink d2/d18/d1a/d5e/d6d/l11e 0 2024-04-25T00:18:35.257 INFO:tasks.workunit.client.0.smithi107.stdout:8/826: readlink d2/d18/d1a/d5e/d3e/lb0 0 2024-04-25T00:18:35.257 INFO:tasks.workunit.client.0.smithi107.stdout:8/827: write d2/d18/d1a/d5e/d3e/faa [1117319,5145] 0 2024-04-25T00:18:35.257 INFO:tasks.workunit.client.0.smithi107.stdout:8/828: stat d2/d18/d1a/d5e/d4f/dd1/cd3 0 2024-04-25T00:18:35.258 INFO:tasks.workunit.client.0.smithi107.stdout:8/829: write d2/d18/d1a/d5e/d5c/d107/dc2/fd7 [835647,69855] 0 2024-04-25T00:18:35.258 INFO:tasks.workunit.client.0.smithi107.stdout:3/740: symlink d9/d15/da1/d4b/da4/lef 0 2024-04-25T00:18:35.259 INFO:tasks.workunit.client.0.smithi107.stdout:8/830: rmdir d2/d18/d1a/d5e/d3d 39 2024-04-25T00:18:35.259 INFO:tasks.workunit.client.0.smithi107.stdout:8/831: chown d2/d18/d1a/d5e/d5c/d91/da3/f103 6 1 2024-04-25T00:18:35.259 INFO:tasks.workunit.client.0.smithi107.stdout:8/832: truncate d2/d18/d1a/d5e/d3d/f9e 852064 0 2024-04-25T00:18:35.259 INFO:tasks.workunit.client.0.smithi107.stdout:8/833: chown d2/d18/d1a/d5e/d4f/f7e 20443760 1 2024-04-25T00:18:35.270 INFO:tasks.workunit.client.0.smithi107.stdout:2/733: sync 2024-04-25T00:18:35.280 INFO:tasks.workunit.client.0.smithi107.stdout:8/834: dwrite d2/d18/d1a/f80 [0,4194304] 0 2024-04-25T00:18:35.280 INFO:tasks.workunit.client.0.smithi107.stdout:2/734: dread d6/d18/d40/d42/d2b/d30/d45/f61 [0,4194304] 0 2024-04-25T00:18:35.282 INFO:tasks.workunit.client.0.smithi107.stdout:8/835: creat d2/d18/d1a/d5e/f11f x:0 0 0 2024-04-25T00:18:35.282 INFO:tasks.workunit.client.0.smithi107.stdout:3/741: rename d9/d88/d4e/f92 to d9/d54/d5c/d68/d3b/ff0 0 2024-04-25T00:18:35.284 INFO:tasks.workunit.client.0.smithi107.stdout:8/836: mknod d2/d18/d1a/c120 0 2024-04-25T00:18:35.285 INFO:tasks.workunit.client.0.smithi107.stdout:3/742: rename d9/f29 to d9/d54/ff1 0 2024-04-25T00:18:35.288 INFO:tasks.workunit.client.0.smithi107.stdout:8/837: rename d2/d18/d1a/d5e/d4f/da0/f115 to d2/d18/ddd/f121 0 2024-04-25T00:18:35.289 INFO:tasks.workunit.client.0.smithi107.stdout:3/743: unlink d9/d15/da1/d4b/dd0/fdf 0 2024-04-25T00:18:35.290 INFO:tasks.workunit.client.0.smithi107.stdout:8/838: mknod d2/d18/d1a/d5e/d5c/c122 0 2024-04-25T00:18:35.290 INFO:tasks.workunit.client.0.smithi107.stdout:3/744: getdents d9/d54/d94/dab 0 2024-04-25T00:18:35.290 INFO:tasks.workunit.client.0.smithi107.stdout:3/745: chown d9/d88/d3a/d91 225868 1 2024-04-25T00:18:35.291 INFO:tasks.workunit.client.0.smithi107.stdout:8/839: creat d2/d18/d1a/d5e/d4f/da0/f123 x:0 0 0 2024-04-25T00:18:35.291 INFO:tasks.workunit.client.0.smithi107.stdout:8/840: stat d2/d18/d1a/d5e/d4f/da0 0 2024-04-25T00:18:35.291 INFO:tasks.workunit.client.0.smithi107.stdout:8/841: chown d2/f63 7452 1 2024-04-25T00:18:35.292 INFO:tasks.workunit.client.0.smithi107.stdout:3/746: creat d9/d88/d3a/d91/ff2 x:0 0 0 2024-04-25T00:18:35.292 INFO:tasks.workunit.client.0.smithi107.stdout:3/747: mknod d9/d15/da1/d4b/dd0/dd8/cf3 0 2024-04-25T00:18:35.294 INFO:tasks.workunit.client.0.smithi107.stdout:3/748: creat d9/d88/d3a/d91/dad/deb/ff4 x:0 0 0 2024-04-25T00:18:35.294 INFO:tasks.workunit.client.0.smithi107.stdout:3/749: fsync d9/d54/d5c/fce 0 2024-04-25T00:18:35.294 INFO:tasks.workunit.client.0.smithi107.stdout:3/750: creat d9/d54/ff5 x:0 0 0 2024-04-25T00:18:35.300 INFO:tasks.workunit.client.0.smithi107.stdout:8/842: write d2/d18/d1a/f8e [2877670,7858] 0 2024-04-25T00:18:35.301 INFO:tasks.workunit.client.0.smithi107.stdout:8/843: creat d2/d18/d1a/d5e/d5c/d107/db2/f124 x:0 0 0 2024-04-25T00:18:35.302 INFO:tasks.workunit.client.0.smithi107.stdout:8/844: mkdir d2/d18/d1a/d5e/d5c/d107/db2/d125 0 2024-04-25T00:18:35.304 INFO:tasks.workunit.client.0.smithi107.stdout:3/751: dread d9/d22/d1b/f81 [4194304,4194304] 0 2024-04-25T00:18:35.304 INFO:tasks.workunit.client.0.smithi107.stdout:3/752: write d9/d22/d1b/d55/fcf [800753,31430] 0 2024-04-25T00:18:35.305 INFO:tasks.workunit.client.0.smithi107.stdout:3/753: link d9/d88/d3a/d91/f9a d9/d88/d4e/dda/ff6 0 2024-04-25T00:18:35.306 INFO:tasks.workunit.client.0.smithi107.stdout:3/754: chown d9/d15/da1/fea 6 1 2024-04-25T00:18:35.323 INFO:tasks.workunit.client.0.smithi107.stdout:3/755: rename d9/d88/d2c/dbf to d9/d22/d1b/d55/df7 0 2024-04-25T00:18:35.325 INFO:tasks.workunit.client.0.smithi107.stdout:3/756: rename d9/d54/fc4 to d9/d15/da1/d4b/dd0/ff8 0 2024-04-25T00:18:35.325 INFO:tasks.workunit.client.0.smithi107.stdout:3/757: write d9/f56 [3730406,38948] 0 2024-04-25T00:18:35.325 INFO:tasks.workunit.client.0.smithi107.stdout:3/758: getdents d9/d88/d3a/d91 0 2024-04-25T00:18:35.352 INFO:tasks.workunit.client.0.smithi107.stdout:3/759: dread d9/f42 [0,4194304] 0 2024-04-25T00:18:35.353 INFO:tasks.workunit.client.0.smithi107.stdout:3/760: creat d9/d88/d3a/d91/dad/deb/ff9 x:0 0 0 2024-04-25T00:18:35.354 INFO:tasks.workunit.client.0.smithi107.stdout:3/761: chown d9/d88/d4e/d93/fc5 115560347 1 2024-04-25T00:18:35.355 INFO:tasks.workunit.client.0.smithi107.stdout:3/762: creat d9/d22/d1b/d55/ffa x:0 0 0 2024-04-25T00:18:35.355 INFO:tasks.workunit.client.0.smithi107.stdout:3/763: fsync d9/d54/d94/dc9/dcb/fe2 0 2024-04-25T00:18:35.355 INFO:tasks.workunit.client.0.smithi107.stdout:3/764: dread - d9/d15/d39/fb7 zero size 2024-04-25T00:18:35.356 INFO:tasks.workunit.client.0.smithi107.stdout:3/765: creat d9/d22/d1b/dbd/ffb x:0 0 0 2024-04-25T00:18:35.356 INFO:tasks.workunit.client.0.smithi107.stdout:3/766: truncate f7 2050575 0 2024-04-25T00:18:35.358 INFO:tasks.workunit.client.0.smithi107.stdout:3/767: creat d9/d54/d94/dab/ffc x:0 0 0 2024-04-25T00:18:35.358 INFO:tasks.workunit.client.0.smithi107.stdout:3/768: readlink d9/d15/da1/l78 0 2024-04-25T00:18:35.358 INFO:tasks.workunit.client.0.smithi107.stdout:3/769: fsync d9/d22/d1b/d55/f5b 0 2024-04-25T00:18:35.359 INFO:tasks.workunit.client.0.smithi107.stdout:3/770: symlink d9/d22/d1b/lfd 0 2024-04-25T00:18:35.361 INFO:tasks.workunit.client.0.smithi107.stdout:3/771: getdents d9/d22 0 2024-04-25T00:18:35.361 INFO:tasks.workunit.client.0.smithi107.stdout:3/772: readlink d9/d54/d5c/d68/lec 0 2024-04-25T00:18:35.361 INFO:tasks.workunit.client.0.smithi107.stdout:3/773: readlink d9/d54/ld5 0 2024-04-25T00:18:35.363 INFO:tasks.workunit.client.0.smithi107.stdout:3/774: creat d9/d54/d94/dab/ffe x:0 0 0 2024-04-25T00:18:35.363 INFO:tasks.workunit.client.0.smithi107.stdout:3/775: dread - d9/d54/d5c/d68/fd6 zero size 2024-04-25T00:18:35.365 INFO:tasks.workunit.client.0.smithi107.stdout:3/776: creat d9/d54/d5c/d68/d3b/fff x:0 0 0 2024-04-25T00:18:35.366 INFO:tasks.workunit.client.0.smithi107.stdout:3/777: getdents d9/d22/d1b/d55 0 2024-04-25T00:18:35.374 INFO:tasks.workunit.client.0.smithi107.stdout:3/778: dread d9/d54/d5c/d68/d3b/f47 [0,4194304] 0 2024-04-25T00:18:35.374 INFO:tasks.workunit.client.0.smithi107.stdout:3/779: fdatasync d9/f20 0 2024-04-25T00:18:35.374 INFO:tasks.workunit.client.0.smithi107.stdout:3/780: write d9/d88/d4e/f7b [1112854,44580] 0 2024-04-25T00:18:35.374 INFO:tasks.workunit.client.0.smithi107.stdout:3/781: fdatasync d9/d88/d3a/d91/dad/fb5 0 2024-04-25T00:18:35.388 INFO:tasks.workunit.client.0.smithi107.stdout:3/782: dwrite d9/d88/d4e/d93/fdb [0,4194304] 0 2024-04-25T00:18:35.389 INFO:tasks.workunit.client.0.smithi107.stdout:3/783: symlink d9/d22/d1b/dbd/l100 0 2024-04-25T00:18:35.391 INFO:tasks.workunit.client.0.smithi107.stdout:3/784: symlink d9/d88/d3a/l101 0 2024-04-25T00:18:35.391 INFO:tasks.workunit.client.0.smithi107.stdout:3/785: chown d9/d54/d94/dab/lee 138334851 1 2024-04-25T00:18:35.409 INFO:tasks.workunit.client.0.smithi107.stdout:3/786: dwrite d9/d54/d5c/d68/d3b/f47 [0,4194304] 0 2024-04-25T00:18:35.409 INFO:tasks.workunit.client.0.smithi107.stdout:9/790: sync 2024-04-25T00:18:35.409 INFO:tasks.workunit.client.0.smithi107.stdout:4/769: sync 2024-04-25T00:18:35.409 INFO:tasks.workunit.client.0.smithi107.stdout:5/733: sync 2024-04-25T00:18:35.409 INFO:tasks.workunit.client.0.smithi107.stdout:3/787: write d9/d54/d5c/d68/fd6 [526469,122964] 0 2024-04-25T00:18:35.409 INFO:tasks.workunit.client.0.smithi107.stdout:9/791: write d0/d8a/d93/d34/fa9 [718893,125770] 0 2024-04-25T00:18:35.409 INFO:tasks.workunit.client.0.smithi107.stdout:4/770: readlink d3/d10/l50 0 2024-04-25T00:18:35.410 INFO:tasks.workunit.client.0.smithi107.stdout:4/771: chown d3/l59 168562044 1 2024-04-25T00:18:35.414 INFO:tasks.workunit.client.0.smithi107.stdout:3/788: creat d9/d88/d2c/f102 x:0 0 0 2024-04-25T00:18:35.414 INFO:tasks.workunit.client.0.smithi107.stdout:9/792: mkdir d0/d8a/d93/d56/d6c/d80/d9b/db2/dea/d10c 0 2024-04-25T00:18:35.414 INFO:tasks.workunit.client.0.smithi107.stdout:9/793: write d0/d8a/d93/d34/d36/d77/dac/dee/f10b [927186,35438] 0 2024-04-25T00:18:35.415 INFO:tasks.workunit.client.0.smithi107.stdout:9/794: chown d0/d8a/d93/d6f/da1 216 1 2024-04-25T00:18:35.415 INFO:tasks.workunit.client.0.smithi107.stdout:9/795: fdatasync d0/d8a/d93/d1f/f86 0 2024-04-25T00:18:35.415 INFO:tasks.workunit.client.0.smithi107.stdout:3/789: creat d9/d54/d94/f103 x:0 0 0 2024-04-25T00:18:35.415 INFO:tasks.workunit.client.0.smithi107.stdout:4/772: mknod d3/d24/df2/cff 0 2024-04-25T00:18:35.415 INFO:tasks.workunit.client.0.smithi107.stdout:3/790: creat d9/d22/d1b/dbd/f104 x:0 0 0 2024-04-25T00:18:35.417 INFO:tasks.workunit.client.0.smithi107.stdout:3/791: truncate d9/d88/d4e/f72 1540617 0 2024-04-25T00:18:35.420 INFO:tasks.workunit.client.0.smithi107.stdout:3/792: link d9/d54/d94/dc9/dcb/fe2 d9/d15/da1/f105 0 2024-04-25T00:18:35.425 INFO:tasks.workunit.client.0.smithi107.stdout:3/793: dread f7 [0,4194304] 0 2024-04-25T00:18:35.425 INFO:tasks.workunit.client.0.smithi107.stdout:3/794: stat d9/d54/fb4 0 2024-04-25T00:18:35.427 INFO:tasks.workunit.client.0.smithi107.stdout:3/795: creat d9/d15/da1/db0/f106 x:0 0 0 2024-04-25T00:18:35.427 INFO:tasks.workunit.client.0.smithi107.stdout:3/796: getdents d9/d54/d94/dc9 0 2024-04-25T00:18:35.427 INFO:tasks.workunit.client.0.smithi107.stdout:3/797: readlink d9/d88/d3a/l101 0 2024-04-25T00:18:35.427 INFO:tasks.workunit.client.0.smithi107.stdout:3/798: write d9/f99 [659410,22979] 0 2024-04-25T00:18:35.428 INFO:tasks.workunit.client.0.smithi107.stdout:3/799: creat d9/d22/d1b/dbd/f107 x:0 0 0 2024-04-25T00:18:35.431 INFO:tasks.workunit.client.0.smithi107.stdout:3/800: creat d9/d88/d3a/f108 x:0 0 0 2024-04-25T00:18:35.431 INFO:tasks.workunit.client.0.smithi107.stdout:3/801: write d9/d22/d1b/dbd/f104 [88736,27944] 0 2024-04-25T00:18:35.433 INFO:tasks.workunit.client.0.smithi107.stdout:3/802: dread d9/d15/d39/f9f [0,4194304] 0 2024-04-25T00:18:35.433 INFO:tasks.workunit.client.0.smithi107.stdout:3/803: fsync d9/f20 0 2024-04-25T00:18:35.434 INFO:tasks.workunit.client.0.smithi107.stdout:5/734: dwrite d4/dc/d26/d29/fc1 [0,4194304] 0 2024-04-25T00:18:35.434 INFO:tasks.workunit.client.0.smithi107.stdout:3/804: creat d9/d88/d3a/d91/dad/f109 x:0 0 0 2024-04-25T00:18:35.438 INFO:tasks.workunit.client.0.smithi107.stdout:4/773: dwrite d3/d24/d25/d71/dc2/fc7 [0,4194304] 0 2024-04-25T00:18:35.438 INFO:tasks.workunit.client.0.smithi107.stdout:4/774: fdatasync d3/d24/d25/f26 0 2024-04-25T00:18:35.441 INFO:tasks.workunit.client.0.smithi107.stdout:3/805: mknod d9/d15/da1/d4b/c10a 0 2024-04-25T00:18:35.441 INFO:tasks.workunit.client.0.smithi107.stdout:3/806: fdatasync d9/d88/d3a/fb1 0 2024-04-25T00:18:35.441 INFO:tasks.workunit.client.0.smithi107.stdout:3/807: chown d9/d22/d1b/d55 6750 1 2024-04-25T00:18:35.456 INFO:tasks.workunit.client.0.smithi107.stdout:5/735: dwrite d4/dc/d26/d42/d8b/f99 [4194304,4194304] 0 2024-04-25T00:18:35.464 INFO:tasks.workunit.client.0.smithi107.stdout:4/775: dwrite d3/d24/d25/f30 [0,4194304] 0 2024-04-25T00:18:35.464 INFO:tasks.workunit.client.0.smithi107.stdout:4/776: chown d3/d24/d25/d67 464 1 2024-04-25T00:18:35.464 INFO:tasks.workunit.client.0.smithi107.stdout:3/808: dwrite d9/d15/da1/fea [0,4194304] 0 2024-04-25T00:18:35.464 INFO:tasks.workunit.client.0.smithi107.stdout:3/809: readlink d9/le8 0 2024-04-25T00:18:35.464 INFO:tasks.workunit.client.0.smithi107.stdout:3/810: readlink d9/d88/d2c/l9c 0 2024-04-25T00:18:35.465 INFO:tasks.workunit.client.0.smithi107.stdout:4/777: creat d3/d24/d5c/dab/da3/f100 x:0 0 0 2024-04-25T00:18:35.468 INFO:tasks.workunit.client.0.smithi107.stdout:4/778: creat d3/d24/d5c/f101 x:0 0 0 2024-04-25T00:18:35.469 INFO:tasks.workunit.client.0.smithi107.stdout:4/779: symlink d3/d24/d5c/d7a/l102 0 2024-04-25T00:18:35.470 INFO:tasks.workunit.client.0.smithi107.stdout:5/736: rename d4/dc/d26/d42/f59 to d4/d1c/d5c/dd4/ff2 0 2024-04-25T00:18:35.470 INFO:tasks.workunit.client.0.smithi107.stdout:4/780: creat d3/d5/d32/d53/d97/f103 x:0 0 0 2024-04-25T00:18:35.470 INFO:tasks.workunit.client.0.smithi107.stdout:4/781: write d3/d24/d25/d67/fcc [1001501,14321] 0 2024-04-25T00:18:35.471 INFO:tasks.workunit.client.0.smithi107.stdout:5/737: stat d4/d1c/d5c/dd4/ff2 0 2024-04-25T00:18:35.471 INFO:tasks.workunit.client.0.smithi107.stdout:4/782: creat d3/d10/d7b/da5/f104 x:0 0 0 2024-04-25T00:18:35.472 INFO:tasks.workunit.client.0.smithi107.stdout:5/738: rename d4/c8 to d4/d7c/d88/d9d/de1/cf3 0 2024-04-25T00:18:35.473 INFO:tasks.workunit.client.0.smithi107.stdout:4/783: rmdir d3/d24/dea 0 2024-04-25T00:18:35.473 INFO:tasks.workunit.client.0.smithi107.stdout:4/784: write d3/d24/f81 [560025,89292] 0 2024-04-25T00:18:35.473 INFO:tasks.workunit.client.0.smithi107.stdout:4/785: write d3/d5/d32/d53/d97/ffe [783129,119271] 0 2024-04-25T00:18:35.473 INFO:tasks.workunit.client.0.smithi107.stdout:4/786: write d3/d24/d5c/f6a [8533997,76254] 0 2024-04-25T00:18:35.474 INFO:tasks.workunit.client.0.smithi107.stdout:4/787: read d3/d24/d5c/d7a/fe7 [22597,21311] 0 2024-04-25T00:18:35.474 INFO:tasks.workunit.client.0.smithi107.stdout:5/739: rename d4/l6 to d4/dc/d83/lf4 0 2024-04-25T00:18:35.474 INFO:tasks.workunit.client.0.smithi107.stdout:5/740: fsync d4/f1b 0 2024-04-25T00:18:35.474 INFO:tasks.workunit.client.0.smithi107.stdout:5/741: write d4/dc/d26/d62/fcd [1018498,80977] 0 2024-04-25T00:18:35.474 INFO:tasks.workunit.client.0.smithi107.stdout:5/742: write d4/d1c/d5c/dd4/fea [76174,71118] 0 2024-04-25T00:18:35.475 INFO:tasks.workunit.client.0.smithi107.stdout:5/743: write d4/dc/d26/d42/fb4 [1046936,57031] 0 2024-04-25T00:18:35.475 INFO:tasks.workunit.client.0.smithi107.stdout:5/744: symlink d4/dc/d83/lf5 0 2024-04-25T00:18:35.475 INFO:tasks.workunit.client.0.smithi107.stdout:5/745: creat d4/d1c/d5c/ff6 x:0 0 0 2024-04-25T00:18:35.476 INFO:tasks.workunit.client.0.smithi107.stdout:4/788: mknod d3/d5/d32/d53/dc0/c105 0 2024-04-25T00:18:35.476 INFO:tasks.workunit.client.0.smithi107.stdout:4/789: fsync d3/d24/d5c/dab/f8d 0 2024-04-25T00:18:35.477 INFO:tasks.workunit.client.0.smithi107.stdout:4/790: symlink d3/d24/db1/l106 0 2024-04-25T00:18:35.479 INFO:tasks.workunit.client.0.smithi107.stdout:4/791: rename d3/d24/d25/f49 to d3/d24/f107 0 2024-04-25T00:18:35.479 INFO:tasks.workunit.client.0.smithi107.stdout:4/792: rename d3/d5/d32/d53 to d3/d5/d32/d53/d97/d108 22 2024-04-25T00:18:35.479 INFO:tasks.workunit.client.0.smithi107.stdout:4/793: getdents d3/d24/d25/d71/ddb 0 2024-04-25T00:18:35.484 INFO:tasks.workunit.client.0.smithi107.stdout:5/746: dread d4/dc/f13 [0,4194304] 0 2024-04-25T00:18:35.484 INFO:tasks.workunit.client.0.smithi107.stdout:5/747: chown d4 1176172 1 2024-04-25T00:18:35.486 INFO:tasks.workunit.client.0.smithi107.stdout:3/811: dwrite d9/d22/d1b/d55/fcf [0,4194304] 0 2024-04-25T00:18:35.486 INFO:tasks.workunit.client.0.smithi107.stdout:3/812: fsync d9/d54/fb4 0 2024-04-25T00:18:35.486 INFO:tasks.workunit.client.0.smithi107.stdout:3/813: truncate d9/d88/d3a/d91/dad/deb/ff9 861897 0 2024-04-25T00:18:35.486 INFO:tasks.workunit.client.0.smithi107.stdout:3/814: dread - d9/d22/d1b/fc2 zero size 2024-04-25T00:18:35.487 INFO:tasks.workunit.client.0.smithi107.stdout:5/748: rename d4/dc/d26/d42/f4f to d4/dc/d26/d42/ff7 0 2024-04-25T00:18:35.487 INFO:tasks.workunit.client.0.smithi107.stdout:5/749: chown d4/dc/d26/d62/cb7 0 1 2024-04-25T00:18:35.487 INFO:tasks.workunit.client.0.smithi107.stdout:5/750: stat d4/dc/d26/d29/d2e/d55 0 2024-04-25T00:18:35.488 INFO:tasks.workunit.client.0.smithi107.stdout:3/815: mknod d9/d54/d94/dab/c10b 0 2024-04-25T00:18:35.489 INFO:tasks.workunit.client.0.smithi107.stdout:5/751: creat d4/dc/ff8 x:0 0 0 2024-04-25T00:18:35.489 INFO:tasks.workunit.client.0.smithi107.stdout:5/752: dread - d4/dc/d26/d29/d51/fb8 zero size 2024-04-25T00:18:35.489 INFO:tasks.workunit.client.0.smithi107.stdout:5/753: truncate d4/d7c/d88/d9d/fd9 2715699 0 2024-04-25T00:18:35.490 INFO:tasks.workunit.client.0.smithi107.stdout:4/794: dread d3/d5/d32/d9e/da2/fa4 [0,4194304] 0 2024-04-25T00:18:35.491 INFO:tasks.workunit.client.0.smithi107.stdout:5/754: link d4/dc/d26/d29/d51/laf d4/d1c/d5c/lf9 0 2024-04-25T00:18:35.492 INFO:tasks.workunit.client.0.smithi107.stdout:4/795: mkdir d3/d5/d32/d9e/da2/daf/d109 0 2024-04-25T00:18:35.495 INFO:tasks.workunit.client.0.smithi107.stdout:3/816: symlink d9/d88/l10c 0 2024-04-25T00:18:35.495 INFO:tasks.workunit.client.0.smithi107.stdout:3/817: truncate d9/f14 4398095 0 2024-04-25T00:18:35.495 INFO:tasks.workunit.client.0.smithi107.stdout:3/818: fdatasync d9/d54/d5c/fce 0 2024-04-25T00:18:35.495 INFO:tasks.workunit.client.0.smithi107.stdout:3/819: chown d9/d88/d2c/fbe 2813 1 2024-04-25T00:18:35.496 INFO:tasks.workunit.client.0.smithi107.stdout:3/820: mkdir d9/d54/d5c/d10d 0 2024-04-25T00:18:35.499 INFO:tasks.workunit.client.0.smithi107.stdout:3/821: read d9/d22/d1b/d55/f5b [4447052,67132] 0 2024-04-25T00:18:35.500 INFO:tasks.workunit.client.0.smithi107.stdout:3/822: creat d9/d88/d4e/dda/f10e x:0 0 0 2024-04-25T00:18:35.502 INFO:tasks.workunit.client.0.smithi107.stdout:3/823: mkdir d9/d54/d5c/d10f 0 2024-04-25T00:18:35.510 INFO:tasks.workunit.client.0.smithi107.stdout:5/755: dwrite d4/dc/d83/fcc [0,4194304] 0 2024-04-25T00:18:35.512 INFO:tasks.workunit.client.0.smithi107.stdout:4/796: dwrite d3/d5/d32/f38 [0,4194304] 0 2024-04-25T00:18:35.512 INFO:tasks.workunit.client.0.smithi107.stdout:4/797: stat d3/d24/c40 0 2024-04-25T00:18:35.516 INFO:tasks.workunit.client.0.smithi107.stdout:4/798: truncate d3/f58 600820 0 2024-04-25T00:18:35.522 INFO:tasks.workunit.client.0.smithi107.stdout:3/824: dwrite d9/d54/fb4 [0,4194304] 0 2024-04-25T00:18:35.526 INFO:tasks.workunit.client.0.smithi107.stdout:0/690: sync 2024-04-25T00:18:35.526 INFO:tasks.workunit.client.0.smithi107.stdout:8/845: sync 2024-04-25T00:18:35.526 INFO:tasks.workunit.client.0.smithi107.stdout:1/688: sync 2024-04-25T00:18:35.526 INFO:tasks.workunit.client.0.smithi107.stdout:6/646: sync 2024-04-25T00:18:35.526 INFO:tasks.workunit.client.0.smithi107.stdout:2/735: sync 2024-04-25T00:18:35.527 INFO:tasks.workunit.client.0.smithi107.stdout:7/722: sync 2024-04-25T00:18:35.527 INFO:tasks.workunit.client.0.smithi107.stdout:6/647: getdents d1/d2/d3/d71/d94 0 2024-04-25T00:18:35.528 INFO:tasks.workunit.client.0.smithi107.stdout:8/846: unlink d2/d18/d1a/d5e/d5c/d107/fd9 0 2024-04-25T00:18:35.529 INFO:tasks.workunit.client.0.smithi107.stdout:7/723: rmdir d0/d8/db/d18/d19/d47/de0/d82 39 2024-04-25T00:18:35.531 INFO:tasks.workunit.client.0.smithi107.stdout:7/724: write d0/d8/db/d18/d19/fb7 [2052071,95422] 0 2024-04-25T00:18:35.531 INFO:tasks.workunit.client.0.smithi107.stdout:7/725: chown d0/d8/ce 110769795 1 2024-04-25T00:18:35.531 INFO:tasks.workunit.client.0.smithi107.stdout:1/689: mknod d2/d2a/d37/d88/cd2 0 2024-04-25T00:18:35.537 INFO:tasks.workunit.client.0.smithi107.stdout:6/648: symlink d1/d2/db/d3c/d9c/ld3 0 2024-04-25T00:18:35.537 INFO:tasks.workunit.client.0.smithi107.stdout:6/649: read d1/d2/d3/d28/d79/d6b/f6f [2264047,76345] 0 2024-04-25T00:18:35.537 INFO:tasks.workunit.client.0.smithi107.stdout:8/847: rename d2/d18/d1a/d5e/d3e/d74/f9f to d2/d18/d1a/dca/f126 0 2024-04-25T00:18:35.537 INFO:tasks.workunit.client.0.smithi107.stdout:8/848: dread - d2/d18/d1a/d5e/d4f/da0/ff1 zero size 2024-04-25T00:18:35.537 INFO:tasks.workunit.client.0.smithi107.stdout:8/849: truncate d2/d18/d1a/d5e/d5c/d107/db2/f124 688748 0 2024-04-25T00:18:35.537 INFO:tasks.workunit.client.0.smithi107.stdout:8/850: symlink d2/d18/d1a/d5e/d5c/d91/l127 0 2024-04-25T00:18:35.537 INFO:tasks.workunit.client.0.smithi107.stdout:5/756: dwrite d4/dc/d26/d29/d51/d78/f9b [0,4194304] 0 2024-04-25T00:18:35.537 INFO:tasks.workunit.client.0.smithi107.stdout:5/757: fsync d4/dc/d26/d29/d51/fb8 0 2024-04-25T00:18:35.537 INFO:tasks.workunit.client.0.smithi107.stdout:8/851: mkdir d2/d18/d1a/d128 0 2024-04-25T00:18:35.542 INFO:tasks.workunit.client.0.smithi107.stdout:3/825: dread d9/f56 [0,4194304] 0 2024-04-25T00:18:35.542 INFO:tasks.workunit.client.0.smithi107.stdout:3/826: chown d9/d15/d39/d76 459337657 1 2024-04-25T00:18:35.542 INFO:tasks.workunit.client.0.smithi107.stdout:3/827: chown d9/d54/ff5 19 1 2024-04-25T00:18:35.542 INFO:tasks.workunit.client.0.smithi107.stdout:3/828: fsync d9/d15/da1/f64 0 2024-04-25T00:18:35.546 INFO:tasks.workunit.client.0.smithi107.stdout:9/796: sync 2024-04-25T00:18:35.549 INFO:tasks.workunit.client.0.smithi107.stdout:9/797: link d0/d8a/d93/de/f7c d0/d8a/d93/d34/f10d 0 2024-04-25T00:18:35.549 INFO:tasks.workunit.client.0.smithi107.stdout:9/798: write d0/d8a/d93/de/d5c/f72 [705941,10766] 0 2024-04-25T00:18:35.553 INFO:tasks.workunit.client.0.smithi107.stdout:9/799: unlink d0/d8a/d93/d6f/l8d 0 2024-04-25T00:18:35.555 INFO:tasks.workunit.client.0.smithi107.stdout:9/800: mknod d0/d8a/d93/d56/d76/d50/dd8/c10e 0 2024-04-25T00:18:35.555 INFO:tasks.workunit.client.0.smithi107.stdout:2/736: dwrite d6/d18/f5d [0,4194304] 0 2024-04-25T00:18:35.557 INFO:tasks.workunit.client.0.smithi107.stdout:4/799: dwrite d3/f35 [0,4194304] 0 2024-04-25T00:18:35.558 INFO:tasks.workunit.client.0.smithi107.stdout:7/726: dwrite d0/d8/d4e/d46/d93/fe1 [0,4194304] 0 2024-04-25T00:18:35.558 INFO:tasks.workunit.client.0.smithi107.stdout:9/801: getdents d0/d8a/d93/d56/d76 0 2024-04-25T00:18:35.560 INFO:tasks.workunit.client.0.smithi107.stdout:7/727: creat d0/d8/db/d18/d19/ff3 x:0 0 0 2024-04-25T00:18:35.562 INFO:tasks.workunit.client.0.smithi107.stdout:8/852: dwrite d2/d18/d1a/d5e/d4f/f7e [0,4194304] 0 2024-04-25T00:18:35.562 INFO:tasks.workunit.client.0.smithi107.stdout:1/690: dwrite d2/dd/d1b/fba [0,4194304] 0 2024-04-25T00:18:35.566 INFO:tasks.workunit.client.0.smithi107.stdout:3/829: link d9/d88/d2c/f102 d9/d54/d94/f110 0 2024-04-25T00:18:35.567 INFO:tasks.workunit.client.0.smithi107.stdout:2/737: dread d6/f7 [0,4194304] 0 2024-04-25T00:18:35.567 INFO:tasks.workunit.client.0.smithi107.stdout:2/738: fsync d6/d20/f89 0 2024-04-25T00:18:35.568 INFO:tasks.workunit.client.0.smithi107.stdout:8/853: symlink d2/d18/d1a/d82/l129 0 2024-04-25T00:18:35.572 INFO:tasks.workunit.client.0.smithi107.stdout:8/854: creat d2/d18/d1a/d128/f12a x:0 0 0 2024-04-25T00:18:35.572 INFO:tasks.workunit.client.0.smithi107.stdout:8/855: fsync d2/d18/ddd/fec 0 2024-04-25T00:18:35.574 INFO:tasks.workunit.client.0.smithi107.stdout:4/800: dread d3/d5/d32/f9b [0,4194304] 0 2024-04-25T00:18:35.574 INFO:tasks.workunit.client.0.smithi107.stdout:8/856: link d2/f16 d2/d18/d1a/d5e/d4f/da0/f12b 0 2024-04-25T00:18:35.578 INFO:tasks.workunit.client.0.smithi107.stdout:2/739: read d6/ff [3119770,57631] 0 2024-04-25T00:18:35.578 INFO:tasks.workunit.client.0.smithi107.stdout:2/740: chown d6/d20/d22/d56/d94 14657538 1 2024-04-25T00:18:35.578 INFO:tasks.workunit.client.0.smithi107.stdout:2/741: chown d6/d18/d40/d42/d69/d6e/da2/lfc 27 1 2024-04-25T00:18:35.579 INFO:tasks.workunit.client.0.smithi107.stdout:2/742: mknod d6/dad/db0/c101 0 2024-04-25T00:18:35.583 INFO:tasks.workunit.client.0.smithi107.stdout:2/743: rename d6/d20/d77 to d6/d18/d40/d42/d2b/d30/d4c/dc9/d102 0 2024-04-25T00:18:35.583 INFO:tasks.workunit.client.0.smithi107.stdout:2/744: chown d6/d18/d40/d42/d2b/d30/d45/fb9 3158 1 2024-04-25T00:18:35.584 INFO:tasks.workunit.client.0.smithi107.stdout:2/745: truncate d6/d20/d22/d56/d94/fdf 11420 0 2024-04-25T00:18:35.584 INFO:tasks.workunit.client.0.smithi107.stdout:2/746: truncate d6/ff 7119908 0 2024-04-25T00:18:35.588 INFO:tasks.workunit.client.0.smithi107.stdout:1/691: dwrite d2/da/f7b [0,4194304] 0 2024-04-25T00:18:35.590 INFO:tasks.workunit.client.0.smithi107.stdout:3/830: dwrite d9/d22/d1b/fa9 [0,4194304] 0 2024-04-25T00:18:35.590 INFO:tasks.workunit.client.0.smithi107.stdout:3/831: fsync d9/d88/d4e/f70 0 2024-04-25T00:18:35.590 INFO:tasks.workunit.client.0.smithi107.stdout:3/832: stat d9/d22/d1b/d55/ffa 0 2024-04-25T00:18:35.590 INFO:tasks.workunit.client.0.smithi107.stdout:3/833: readlink d9/d15/d39/d76/l63 0 2024-04-25T00:18:35.590 INFO:tasks.workunit.client.0.smithi107.stdout:3/834: stat d9/d88/d3a 0 2024-04-25T00:18:35.590 INFO:tasks.workunit.client.0.smithi107.stdout:1/692: creat d2/dd/d82/dc1/fd3 x:0 0 0 2024-04-25T00:18:35.592 INFO:tasks.workunit.client.0.smithi107.stdout:2/747: read d6/d20/d22/fa0 [2636204,44967] 0 2024-04-25T00:18:35.594 INFO:tasks.workunit.client.0.smithi107.stdout:4/801: write d3/d5/d32/d9e/da2/daf/fbf [3728374,8988] 0 2024-04-25T00:18:35.594 INFO:tasks.workunit.client.0.smithi107.stdout:3/835: mknod d9/d54/d94/dc9/c111 0 2024-04-25T00:18:35.595 INFO:tasks.workunit.client.0.smithi107.stdout:3/836: mknod d9/d88/d2c/c112 0 2024-04-25T00:18:35.600 INFO:tasks.workunit.client.0.smithi107.stdout:1/693: link d2/dd/d25/d5e/c44 d2/dd/d25/da9/cd4 0 2024-04-25T00:18:35.600 INFO:tasks.workunit.client.0.smithi107.stdout:4/802: mknod d3/c10a 0 2024-04-25T00:18:35.601 INFO:tasks.workunit.client.0.smithi107.stdout:8/857: dwrite d2/d18/d1a/d5e/fa7 [0,4194304] 0 2024-04-25T00:18:35.601 INFO:tasks.workunit.client.0.smithi107.stdout:8/858: readlink d2/l19 0 2024-04-25T00:18:35.603 INFO:tasks.workunit.client.0.smithi107.stdout:2/748: creat d6/d29/d4e/dc0/dd1/f103 x:0 0 0 2024-04-25T00:18:35.604 INFO:tasks.workunit.client.0.smithi107.stdout:1/694: symlink d2/d62/d66/d70/dcc/ld5 0 2024-04-25T00:18:35.606 INFO:tasks.workunit.client.0.smithi107.stdout:8/859: stat d2/d18/d1a/d5e/d3e/c90 0 2024-04-25T00:18:35.606 INFO:tasks.workunit.client.0.smithi107.stdout:8/860: write d2/fc6 [1365237,125600] 0 2024-04-25T00:18:35.608 INFO:tasks.workunit.client.0.smithi107.stdout:2/749: rename d6/d18/d40/c48 to d6/d18/d40/d42/d69/d6e/da2/dc5/c104 0 2024-04-25T00:18:35.608 INFO:tasks.workunit.client.0.smithi107.stdout:3/837: write d9/d54/d5c/d68/d3b/f47 [384270,126096] 0 2024-04-25T00:18:35.609 INFO:tasks.workunit.client.0.smithi107.stdout:1/695: creat d2/d2a/d37/fd6 x:0 0 0 2024-04-25T00:18:35.609 INFO:tasks.workunit.client.0.smithi107.stdout:3/838: mkdir d9/d88/d3a/d91/dad/deb/d113 0 2024-04-25T00:18:35.609 INFO:tasks.workunit.client.0.smithi107.stdout:3/839: chown d9/d88/l10c 7 1 2024-04-25T00:18:35.613 INFO:tasks.workunit.client.0.smithi107.stdout:8/861: truncate d2/d18/ddd/fec 297859 0 2024-04-25T00:18:35.614 INFO:tasks.workunit.client.0.smithi107.stdout:4/803: mknod d3/d5/d32/d53/dc0/c10b 0 2024-04-25T00:18:35.614 INFO:tasks.workunit.client.0.smithi107.stdout:4/804: chown d3/d5/d32/d9e/da2/daf/fbf 140515852 1 2024-04-25T00:18:35.618 INFO:tasks.workunit.client.0.smithi107.stdout:4/805: write d3/d24/d5c/dab/f8d [749546,37035] 0 2024-04-25T00:18:35.618 INFO:tasks.workunit.client.0.smithi107.stdout:4/806: dread - d3/d10/d7b/da5/f104 zero size 2024-04-25T00:18:35.618 INFO:tasks.workunit.client.0.smithi107.stdout:2/750: rename d6/d20/f89 to d6/d18/d40/dc8/de5/f105 0 2024-04-25T00:18:35.618 INFO:tasks.workunit.client.0.smithi107.stdout:1/696: creat d2/da4/fd7 x:0 0 0 2024-04-25T00:18:35.620 INFO:tasks.workunit.client.0.smithi107.stdout:1/697: link d2/d2a/d37/l86 d2/dd/d25/da9/ld8 0 2024-04-25T00:18:35.620 INFO:tasks.workunit.client.0.smithi107.stdout:1/698: readlink d2/d2a/l60 0 2024-04-25T00:18:35.622 INFO:tasks.workunit.client.0.smithi107.stdout:1/699: dread d2/d2a/d5c/d89/dc0/fce [0,4194304] 0 2024-04-25T00:18:35.622 INFO:tasks.workunit.client.0.smithi107.stdout:1/700: chown d2/da/d2f/f9b 117160 1 2024-04-25T00:18:35.623 INFO:tasks.workunit.client.0.smithi107.stdout:1/701: mknod d2/dd/d1a/cd9 0 2024-04-25T00:18:35.626 INFO:tasks.workunit.client.0.smithi107.stdout:4/807: write d3/d24/d25/d71/fd5 [2084497,74056] 0 2024-04-25T00:18:35.631 INFO:tasks.workunit.client.0.smithi107.stdout:8/862: dwrite d2/d18/d1a/d5e/d3d/d85/dd5/ff2 [0,4194304] 0 2024-04-25T00:18:35.633 INFO:tasks.workunit.client.0.smithi107.stdout:8/863: creat d2/d18/d1a/d128/f12c x:0 0 0 2024-04-25T00:18:35.634 INFO:tasks.workunit.client.0.smithi107.stdout:8/864: read - d2/d18/d1a/d5e/d4f/da0/ff1 zero size 2024-04-25T00:18:35.634 INFO:tasks.workunit.client.0.smithi107.stdout:8/865: fdatasync d2/d18/d1a/d5e/d4f/da0/ff1 0 2024-04-25T00:18:35.634 INFO:tasks.workunit.client.0.smithi107.stdout:8/866: creat d2/d18/d1a/dca/f12d x:0 0 0 2024-04-25T00:18:35.636 INFO:tasks.workunit.client.0.smithi107.stdout:8/867: creat d2/d18/d1a/d5e/d3e/db1/f12e x:0 0 0 2024-04-25T00:18:35.639 INFO:tasks.workunit.client.0.smithi107.stdout:3/840: rename d9/d88/d3a/d91/cbb to d9/d15/da1/d4b/dd0/c114 0 2024-04-25T00:18:35.639 INFO:tasks.workunit.client.0.smithi107.stdout:1/702: dread d2/d2a/d37/f56 [0,4194304] 0 2024-04-25T00:18:35.642 INFO:tasks.workunit.client.0.smithi107.stdout:2/751: rename d6/d29/d4e/d6f/c97 to d6/d18/d40/d42/d2b/d30/d4c/dc9/d102/c106 0 2024-04-25T00:18:35.644 INFO:tasks.workunit.client.0.smithi107.stdout:4/808: rename d3/d5/c4a to d3/d5/d32/d53/d7e/c10c 0 2024-04-25T00:18:35.647 INFO:tasks.workunit.client.0.smithi107.stdout:4/809: link d3/d10/d7b/feb d3/d10/f10d 0 2024-04-25T00:18:35.649 INFO:tasks.workunit.client.0.smithi107.stdout:4/810: link d3/d24/f81 d3/d5/d32/f10e 0 2024-04-25T00:18:35.651 INFO:tasks.workunit.client.0.smithi107.stdout:1/703: dread d2/da/f53 [0,4194304] 0 2024-04-25T00:18:35.653 INFO:tasks.workunit.client.0.smithi107.stdout:1/704: creat d2/da4/fda x:0 0 0 2024-04-25T00:18:35.654 INFO:tasks.workunit.client.0.smithi107.stdout:1/705: creat d2/d2a/d5c/d89/fdb x:0 0 0 2024-04-25T00:18:35.659 INFO:tasks.workunit.client.0.smithi107.stdout:2/752: dwrite d6/d18/d40/d42/d2b/f41 [0,4194304] 0 2024-04-25T00:18:35.665 INFO:tasks.workunit.client.0.smithi107.stdout:3/841: rename d9/d54/d94/dc9 to d9/d88/d80/d115 0 2024-04-25T00:18:35.667 INFO:tasks.workunit.client.0.smithi107.stdout:1/706: rename d2/dd/d1a/f6d to d2/dd/d1b/fdc 0 2024-04-25T00:18:35.667 INFO:tasks.workunit.client.0.smithi107.stdout:3/842: dread d9/d15/d39/f9f [0,4194304] 0 2024-04-25T00:18:35.669 INFO:tasks.workunit.client.0.smithi107.stdout:2/753: rename d6/d18/d40/d42/d69/fce to d6/d18/d40/d42/d69/d6e/da2/dc5/dd2/d100/f107 0 2024-04-25T00:18:35.670 INFO:tasks.workunit.client.0.smithi107.stdout:1/707: creat d2/dd/fdd x:0 0 0 2024-04-25T00:18:35.671 INFO:tasks.workunit.client.0.smithi107.stdout:4/811: dwrite d3/d5/f3d [0,4194304] 0 2024-04-25T00:18:35.671 INFO:tasks.workunit.client.0.smithi107.stdout:3/843: creat d9/d22/f116 x:0 0 0 2024-04-25T00:18:35.671 INFO:tasks.workunit.client.0.smithi107.stdout:1/708: truncate d2/f51 170519 0 2024-04-25T00:18:35.679 INFO:tasks.workunit.client.0.smithi107.stdout:3/844: dread d9/d22/d1b/d55/f5d [0,4194304] 0 2024-04-25T00:18:35.693 INFO:tasks.workunit.client.0.smithi107.stdout:2/754: write d6/d18/d40/d42/d2b/d30/fbf [3411773,85550] 0 2024-04-25T00:18:35.695 INFO:tasks.workunit.client.0.smithi107.stdout:2/755: rename d6/d18/d40/d42/d69/d6e/da2/caa to d6/d18/d40/dc8/de5/c108 0 2024-04-25T00:18:35.695 INFO:tasks.workunit.client.0.smithi107.stdout:1/709: dread d2/dd/d25/d5e/f94 [0,4194304] 0 2024-04-25T00:18:35.696 INFO:tasks.workunit.client.0.smithi107.stdout:2/756: mknod d6/d18/d40/d42/d70/c109 0 2024-04-25T00:18:35.696 INFO:tasks.workunit.client.0.smithi107.stdout:2/757: chown d6/d20/d22/d56/d94 245434 1 2024-04-25T00:18:35.696 INFO:tasks.workunit.client.0.smithi107.stdout:3/845: dread d9/d54/d5c/d68/d3b/ff0 [0,4194304] 0 2024-04-25T00:18:35.698 INFO:tasks.workunit.client.0.smithi107.stdout:3/846: rmdir d9/d15/da1/db0 39 2024-04-25T00:18:35.698 INFO:tasks.workunit.client.0.smithi107.stdout:3/847: read - d9/d88/d4e/dda/f10e zero size 2024-04-25T00:18:35.698 INFO:tasks.workunit.client.0.smithi107.stdout:3/848: chown d9/d15/da1/d4b/fcd 0 1 2024-04-25T00:18:35.698 INFO:tasks.workunit.client.0.smithi107.stdout:2/758: dread d6/d20/d22/d56/d94/fdf [0,4194304] 0 2024-04-25T00:18:35.698 INFO:tasks.workunit.client.0.smithi107.stdout:3/849: fdatasync d9/d22/d1b/d55/ffa 0 2024-04-25T00:18:35.698 INFO:tasks.workunit.client.0.smithi107.stdout:2/759: chown d6/d18/d40/f8f 290 1 2024-04-25T00:18:35.698 INFO:tasks.workunit.client.0.smithi107.stdout:2/760: read d6/d18/d40/d42/d69/d82/fb8 [622498,106327] 0 2024-04-25T00:18:35.699 INFO:tasks.workunit.client.0.smithi107.stdout:3/850: write d9/d88/d2c/f102 [408934,11285] 0 2024-04-25T00:18:35.699 INFO:tasks.workunit.client.0.smithi107.stdout:2/761: rmdir d6/d20/d22/d56 39 2024-04-25T00:18:35.700 INFO:tasks.workunit.client.0.smithi107.stdout:4/812: dwrite d3/d5/d32/d53/d7e/fd4 [0,4194304] 0 2024-04-25T00:18:35.701 INFO:tasks.workunit.client.0.smithi107.stdout:2/762: rename d6/d18/d40/d42/d69/d6e/l9c to d6/d29/d4e/d8c/l10a 0 2024-04-25T00:18:35.701 INFO:tasks.workunit.client.0.smithi107.stdout:3/851: fdatasync d9/d88/d2c/f102 0 2024-04-25T00:18:35.705 INFO:tasks.workunit.client.0.smithi107.stdout:1/710: dread d2/dd/d82/fa7 [0,4194304] 0 2024-04-25T00:18:35.705 INFO:tasks.workunit.client.0.smithi107.stdout:1/711: write d2/dd/d25/f63 [86119,35089] 0 2024-04-25T00:18:35.706 INFO:tasks.workunit.client.0.smithi107.stdout:1/712: symlink d2/dd/d1b/lde 0 2024-04-25T00:18:35.707 INFO:tasks.workunit.client.0.smithi107.stdout:1/713: truncate d2/dd/d1b/fdc 3346450 0 2024-04-25T00:18:35.710 INFO:tasks.workunit.client.0.smithi107.stdout:1/714: read d2/da/f8e [471914,10830] 0 2024-04-25T00:18:35.710 INFO:tasks.workunit.client.0.smithi107.stdout:1/715: write d2/dd/d82/dc1/fd3 [876563,97049] 0 2024-04-25T00:18:35.710 INFO:tasks.workunit.client.0.smithi107.stdout:1/716: write d2/d2a/d5c/d89/dc0/fce [517458,128385] 0 2024-04-25T00:18:35.711 INFO:tasks.workunit.client.0.smithi107.stdout:1/717: link d2/c3 d2/d2a/d5c/d89/cdf 0 2024-04-25T00:18:35.712 INFO:tasks.workunit.client.0.smithi107.stdout:1/718: creat d2/d2a/d37/d88/fe0 x:0 0 0 2024-04-25T00:18:35.716 INFO:tasks.workunit.client.0.smithi107.stdout:1/719: rename d2/dd/d25/d5e/cb8 to d2/d2a/d5c/d89/dc0/ce1 0 2024-04-25T00:18:35.716 INFO:tasks.workunit.client.0.smithi107.stdout:1/720: chown d2/d62/d66/d70/d95 0 1 2024-04-25T00:18:35.719 INFO:tasks.workunit.client.0.smithi107.stdout:3/852: link d9/d88/d4e/la3 d9/d88/d80/d115/dcb/l117 0 2024-04-25T00:18:35.720 INFO:tasks.workunit.client.0.smithi107.stdout:1/721: symlink d2/le2 0 2024-04-25T00:18:35.721 INFO:tasks.workunit.client.0.smithi107.stdout:4/813: dwrite d3/d5/d32/d53/d97/ffe [0,4194304] 0 2024-04-25T00:18:35.722 INFO:tasks.workunit.client.0.smithi107.stdout:2/763: dwrite d6/dad/d67/da8/fc6 [0,4194304] 0 2024-04-25T00:18:35.722 INFO:tasks.workunit.client.0.smithi107.stdout:2/764: chown d6/d20/d22/d56/f68 15441 1 2024-04-25T00:18:35.723 INFO:tasks.workunit.client.0.smithi107.stdout:1/722: dread d2/dd/d82/f99 [0,4194304] 0 2024-04-25T00:18:35.724 INFO:tasks.workunit.client.0.smithi107.stdout:1/723: chown d2/d62/d66/fc3 203743109 1 2024-04-25T00:18:35.724 INFO:tasks.workunit.client.0.smithi107.stdout:1/724: write d2/da/d2f/fbc [204512,112056] 0 2024-04-25T00:18:35.728 INFO:tasks.workunit.client.0.smithi107.stdout:2/765: mkdir d6/d20/d22/d56/d94/d10b 0 2024-04-25T00:18:35.732 INFO:tasks.workunit.client.0.smithi107.stdout:2/766: rmdir d6/d18/d40 39 2024-04-25T00:18:35.733 INFO:tasks.workunit.client.0.smithi107.stdout:1/725: dread d2/da/d2f/f9b [0,4194304] 0 2024-04-25T00:18:35.741 INFO:tasks.workunit.client.0.smithi107.stdout:3/853: dwrite d9/d22/d1b/fa9 [0,4194304] 0 2024-04-25T00:18:35.744 INFO:tasks.workunit.client.0.smithi107.stdout:4/814: dwrite d3/d5/d32/d9e/da2/fa4 [0,4194304] 0 2024-04-25T00:18:35.746 INFO:tasks.workunit.client.0.smithi107.stdout:2/767: dread d6/d18/d40/d42/d2b/d30/d4c/dc9/d102/d9f/fa1 [0,4194304] 0 2024-04-25T00:18:35.749 INFO:tasks.workunit.client.0.smithi107.stdout:2/768: rename d6/d18/d40/d42/d2b/d30/d4c/dc9 to d6/d18/d40/d42/d69/d6e/da2/d10c 0 2024-04-25T00:18:35.756 INFO:tasks.workunit.client.0.smithi107.stdout:3/854: link d9/f99 d9/d88/d4e/d93/f118 0 2024-04-25T00:18:35.756 INFO:tasks.workunit.client.0.smithi107.stdout:1/726: dwrite d2/da4/fda [0,4194304] 0 2024-04-25T00:18:35.758 INFO:tasks.workunit.client.0.smithi107.stdout:1/727: symlink d2/dd/d1b/le3 0 2024-04-25T00:18:35.758 INFO:tasks.workunit.client.0.smithi107.stdout:2/769: write d6/d20/f88 [3110969,24747] 0 2024-04-25T00:18:35.758 INFO:tasks.workunit.client.0.smithi107.stdout:2/770: chown d6/d18/d40/d42/d2b/l2f 256315 1 2024-04-25T00:18:35.759 INFO:tasks.workunit.client.0.smithi107.stdout:2/771: write d6/d18/d40/dc8/de5/f105 [346462,39411] 0 2024-04-25T00:18:35.759 INFO:tasks.workunit.client.0.smithi107.stdout:1/728: creat d2/d2a/fe4 x:0 0 0 2024-04-25T00:18:35.764 INFO:tasks.workunit.client.0.smithi107.stdout:2/772: rmdir d6/dad/db0/dbc 39 2024-04-25T00:18:35.764 INFO:tasks.workunit.client.0.smithi107.stdout:2/773: truncate d6/d18/d40/d42/d70/ffb 109147 0 2024-04-25T00:18:35.765 INFO:tasks.workunit.client.0.smithi107.stdout:2/774: rmdir d6/d18/d40/d42/d2b/d30/d45 39 2024-04-25T00:18:35.765 INFO:tasks.workunit.client.0.smithi107.stdout:1/729: dread d2/f8a [0,4194304] 0 2024-04-25T00:18:35.766 INFO:tasks.workunit.client.0.smithi107.stdout:1/730: mkdir d2/d62/d66/de5 0 2024-04-25T00:18:35.767 INFO:tasks.workunit.client.0.smithi107.stdout:2/775: creat d6/d18/d40/d42/d2b/d30/d45/f10d x:0 0 0 2024-04-25T00:18:35.769 INFO:tasks.workunit.client.0.smithi107.stdout:4/815: dwrite d3/d24/d5c/d7a/fe7 [0,4194304] 0 2024-04-25T00:18:35.769 INFO:tasks.workunit.client.0.smithi107.stdout:1/731: link d2/d62/d66/d70/f8c d2/d2a/d37/fe6 0 2024-04-25T00:18:35.769 INFO:tasks.workunit.client.0.smithi107.stdout:1/732: dread - d2/d62/d66/fc3 zero size 2024-04-25T00:18:35.771 INFO:tasks.workunit.client.0.smithi107.stdout:2/776: rename d6/d18/d40/d42/d69/d82 to d6/d18/d40/d42/d2b/d30/d4c/d10e 0 2024-04-25T00:18:35.773 INFO:tasks.workunit.client.0.smithi107.stdout:1/733: dread d2/dd/f8b [0,4194304] 0 2024-04-25T00:18:35.773 INFO:tasks.workunit.client.0.smithi107.stdout:4/816: unlink d3/d24/d5c/dab/da3/lb0 0 2024-04-25T00:18:35.774 INFO:tasks.workunit.client.0.smithi107.stdout:4/817: mkdir d3/d24/d25/d10f 0 2024-04-25T00:18:35.774 INFO:tasks.workunit.client.0.smithi107.stdout:4/818: dread - d3/d24/d25/d71/d7f/fc5 zero size 2024-04-25T00:18:35.775 INFO:tasks.workunit.client.0.smithi107.stdout:2/777: symlink d6/dad/l10f 0 2024-04-25T00:18:35.775 INFO:tasks.workunit.client.0.smithi107.stdout:2/778: chown d6/d18/d40/dc8/de5/c108 248 1 2024-04-25T00:18:35.775 INFO:tasks.workunit.client.0.smithi107.stdout:1/734: symlink d2/da/db7/le7 0 2024-04-25T00:18:35.775 INFO:tasks.workunit.client.0.smithi107.stdout:4/819: creat d3/d24/d61/f110 x:0 0 0 2024-04-25T00:18:35.776 INFO:tasks.workunit.client.0.smithi107.stdout:4/820: read - d3/d24/d25/d71/dc2/ff6 zero size 2024-04-25T00:18:35.776 INFO:tasks.workunit.client.0.smithi107.stdout:4/821: write d3/d5/d32/f10e [1275139,69768] 0 2024-04-25T00:18:35.776 INFO:tasks.workunit.client.0.smithi107.stdout:2/779: creat d6/dad/d67/f110 x:0 0 0 2024-04-25T00:18:35.777 INFO:tasks.workunit.client.0.smithi107.stdout:1/735: unlink d2/da/d2f/cae 0 2024-04-25T00:18:35.777 INFO:tasks.workunit.client.0.smithi107.stdout:1/736: write d2/dd/d25/f2e [3142418,40169] 0 2024-04-25T00:18:35.779 INFO:tasks.workunit.client.0.smithi107.stdout:1/737: mkdir d2/d2a/d37/de8 0 2024-04-25T00:18:35.779 INFO:tasks.workunit.client.0.smithi107.stdout:1/738: chown d2/dd/d1b/f7f 257 1 2024-04-25T00:18:35.780 INFO:tasks.workunit.client.0.smithi107.stdout:1/739: mknod d2/d62/ce9 0 2024-04-25T00:18:35.780 INFO:tasks.workunit.client.0.smithi107.stdout:1/740: fsync d2/dd/d1a/f64 0 2024-04-25T00:18:35.780 INFO:tasks.workunit.client.0.smithi107.stdout:4/822: rmdir d3/d24/d5c/dab 39 2024-04-25T00:18:35.781 INFO:tasks.workunit.client.0.smithi107.stdout:1/741: creat d2/d2a/d5c/fea x:0 0 0 2024-04-25T00:18:35.781 INFO:tasks.workunit.client.0.smithi107.stdout:1/742: write d2/dd/d25/f63 [866502,100926] 0 2024-04-25T00:18:35.782 INFO:tasks.workunit.client.0.smithi107.stdout:1/743: creat d2/d2a/d37/feb x:0 0 0 2024-04-25T00:18:35.796 INFO:tasks.workunit.client.0.smithi107.stdout:1/744: write d2/dd/f6f [1110525,20476] 0 2024-04-25T00:18:35.796 INFO:tasks.workunit.client.0.smithi107.stdout:2/780: dwrite d6/d18/d40/d42/d69/d6e/dc2/fe7 [0,4194304] 0 2024-04-25T00:18:35.797 INFO:tasks.workunit.client.0.smithi107.stdout:1/745: mkdir d2/dd/dec 0 2024-04-25T00:18:35.797 INFO:tasks.workunit.client.0.smithi107.stdout:2/781: truncate d6/d18/d40/d42/d2b/d30/d45/f5f 130267 0 2024-04-25T00:18:35.798 INFO:tasks.workunit.client.0.smithi107.stdout:2/782: truncate d6/d18/d40/d42/d69/d6e/da2/fba 4832064 0 2024-04-25T00:18:35.798 INFO:tasks.workunit.client.0.smithi107.stdout:2/783: stat d6/d18/d40/d42/d69/d6e/ld3 0 2024-04-25T00:18:35.798 INFO:tasks.workunit.client.0.smithi107.stdout:1/746: creat d2/d2a/d37/fed x:0 0 0 2024-04-25T00:18:35.799 INFO:tasks.workunit.client.0.smithi107.stdout:4/823: dwrite d3/d24/d25/f43 [0,4194304] 0 2024-04-25T00:18:35.799 INFO:tasks.workunit.client.0.smithi107.stdout:4/824: dread - d3/d24/d25/d71/dc2/ff6 zero size 2024-04-25T00:18:35.799 INFO:tasks.workunit.client.0.smithi107.stdout:2/784: creat d6/d18/d40/dc8/f111 x:0 0 0 2024-04-25T00:18:35.808 INFO:tasks.workunit.client.0.smithi107.stdout:2/785: dread d6/d18/d40/d42/d69/d6e/db6/fdd [0,4194304] 0 2024-04-25T00:18:35.809 INFO:tasks.workunit.client.0.smithi107.stdout:4/825: dread d3/d5/d32/d9e/da2/daf/fbf [0,4194304] 0 2024-04-25T00:18:35.809 INFO:tasks.workunit.client.0.smithi107.stdout:2/786: mknod d6/d18/d40/d42/d69/c112 0 2024-04-25T00:18:35.810 INFO:tasks.workunit.client.0.smithi107.stdout:4/826: rmdir d3/d5/d32/d9e/da2 39 2024-04-25T00:18:35.810 INFO:tasks.workunit.client.0.smithi107.stdout:4/827: readlink d3/d5/l90 0 2024-04-25T00:18:35.810 INFO:tasks.workunit.client.0.smithi107.stdout:2/787: truncate d6/f1e 3335747 0 2024-04-25T00:18:35.810 INFO:tasks.workunit.client.0.smithi107.stdout:2/788: readlink d6/d29/d4e/d8c/l10a 0 2024-04-25T00:18:35.812 INFO:tasks.workunit.client.0.smithi107.stdout:4/828: mkdir d3/d5/d32/d9e/d111 0 2024-04-25T00:18:35.813 INFO:tasks.workunit.client.0.smithi107.stdout:4/829: mknod d3/d5/d32/d9e/d111/c112 0 2024-04-25T00:18:35.815 INFO:tasks.workunit.client.0.smithi107.stdout:4/830: mknod d3/d24/d5c/dab/c113 0 2024-04-25T00:18:35.815 INFO:tasks.workunit.client.0.smithi107.stdout:1/747: dwrite d2/dd/d25/d5e/f9d [0,4194304] 0 2024-04-25T00:18:35.819 INFO:tasks.workunit.client.0.smithi107.stdout:1/748: symlink d2/dd/lee 0 2024-04-25T00:18:35.820 INFO:tasks.workunit.client.0.smithi107.stdout:1/749: symlink d2/d62/d66/d70/lef 0 2024-04-25T00:18:35.821 INFO:tasks.workunit.client.0.smithi107.stdout:1/750: creat d2/dd/d82/ff0 x:0 0 0 2024-04-25T00:18:35.821 INFO:tasks.workunit.client.0.smithi107.stdout:1/751: dread - d2/d2a/d5c/d89/fdb zero size 2024-04-25T00:18:35.822 INFO:tasks.workunit.client.0.smithi107.stdout:4/831: dread d3/da8/fb7 [8388608,4194304] 0 2024-04-25T00:18:35.822 INFO:tasks.workunit.client.0.smithi107.stdout:1/752: mknod d2/d2a/cf1 0 2024-04-25T00:18:35.822 INFO:tasks.workunit.client.0.smithi107.stdout:1/753: read d2/dd/d25/f33 [4181031,105012] 0 2024-04-25T00:18:35.826 INFO:tasks.workunit.client.0.smithi107.stdout:4/832: rmdir d3/d24/d25/d67 39 2024-04-25T00:18:35.826 INFO:tasks.workunit.client.0.smithi107.stdout:4/833: creat d3/d5/d32/d9e/da2/daf/f114 x:0 0 0 2024-04-25T00:18:35.826 INFO:tasks.workunit.client.0.smithi107.stdout:4/834: creat d3/d24/d5c/d7a/f115 x:0 0 0 2024-04-25T00:18:35.833 INFO:tasks.workunit.client.0.smithi107.stdout:1/754: write d2/da/d2f/f3e [8140100,120183] 0 2024-04-25T00:18:35.842 INFO:tasks.workunit.client.0.smithi107.stdout:2/789: write d6/f1e [3302561,20254] 0 2024-04-25T00:18:35.842 INFO:tasks.workunit.client.0.smithi107.stdout:2/790: chown d6/d18/d40/dc8/lf9 682267 1 2024-04-25T00:18:35.844 INFO:tasks.workunit.client.0.smithi107.stdout:2/791: mknod d6/d18/d40/d42/d2b/d30/d45/dca/c113 0 2024-04-25T00:18:35.852 INFO:tasks.workunit.client.0.smithi107.stdout:1/755: dwrite d2/d62/d66/d70/f77 [0,4194304] 0 2024-04-25T00:18:35.854 INFO:tasks.workunit.client.0.smithi107.stdout:1/756: rmdir d2/dd/d1a 39 2024-04-25T00:18:35.854 INFO:tasks.workunit.client.0.smithi107.stdout:1/757: chown d2/dd/fdd 2770 1 2024-04-25T00:18:35.854 INFO:tasks.workunit.client.0.smithi107.stdout:1/758: dread - d2/fd1 zero size 2024-04-25T00:18:35.855 INFO:tasks.workunit.client.0.smithi107.stdout:1/759: truncate d2/dd/d25/f50 1419348 0 2024-04-25T00:18:35.856 INFO:tasks.workunit.client.0.smithi107.stdout:1/760: link d2/dd/fdd d2/dd/d1b/ff2 0 2024-04-25T00:18:35.857 INFO:tasks.workunit.client.0.smithi107.stdout:1/761: truncate d2/f68 235154 0 2024-04-25T00:18:35.857 INFO:tasks.workunit.client.0.smithi107.stdout:1/762: chown d2/dd/d1a/f1d 323636489 1 2024-04-25T00:18:35.857 INFO:tasks.workunit.client.0.smithi107.stdout:1/763: chown d2/da4/fcb 949 1 2024-04-25T00:18:35.858 INFO:tasks.workunit.client.0.smithi107.stdout:1/764: dread d2/dd/d82/f99 [0,4194304] 0 2024-04-25T00:18:35.861 INFO:tasks.workunit.client.0.smithi107.stdout:0/691: sync 2024-04-25T00:18:35.861 INFO:tasks.workunit.client.0.smithi107.stdout:6/650: sync 2024-04-25T00:18:35.861 INFO:tasks.workunit.client.0.smithi107.stdout:2/792: link d6/dad/d67/da8/lee d6/d18/d40/d42/d69/d6e/dc2/l114 0 2024-04-25T00:18:35.863 INFO:tasks.workunit.client.0.smithi107.stdout:1/765: dread d2/d62/d66/d70/f7a [0,4194304] 0 2024-04-25T00:18:35.863 INFO:tasks.workunit.client.0.smithi107.stdout:6/651: link d1/d2/d2b/d3e/d5b/d7a/ccc d1/d2/d3/d28/d79/cd4 0 2024-04-25T00:18:35.867 INFO:tasks.workunit.client.0.smithi107.stdout:1/766: symlink d2/dd/d82/lf3 0 2024-04-25T00:18:35.867 INFO:tasks.workunit.client.0.smithi107.stdout:1/767: chown d2/f51 3 1 2024-04-25T00:18:35.870 INFO:tasks.workunit.client.0.smithi107.stdout:2/793: dread d6/d18/d40/d42/d70/f76 [0,4194304] 0 2024-04-25T00:18:35.871 INFO:tasks.workunit.client.0.smithi107.stdout:2/794: creat d6/d18/d40/d42/d69/f115 x:0 0 0 2024-04-25T00:18:35.872 INFO:tasks.workunit.client.0.smithi107.stdout:2/795: mkdir d6/dad/d67/da8/d116 0 2024-04-25T00:18:35.890 INFO:tasks.workunit.client.0.smithi107.stdout:6/652: dwrite d1/d2/d3/f13 [0,4194304] 0 2024-04-25T00:18:35.891 INFO:tasks.workunit.client.0.smithi107.stdout:2/796: dwrite d6/d18/d40/d42/d69/d6e/db6/fdd [0,4194304] 0 2024-04-25T00:18:35.891 INFO:tasks.workunit.client.0.smithi107.stdout:0/692: dwrite dc/dd/d5d/d5c/f7f [0,4194304] 0 2024-04-25T00:18:35.891 INFO:tasks.workunit.client.0.smithi107.stdout:0/693: chown dc/dd/d16/d48/d83 53 1 2024-04-25T00:18:35.891 INFO:tasks.workunit.client.0.smithi107.stdout:0/694: stat c5 0 2024-04-25T00:18:35.893 INFO:tasks.workunit.client.0.smithi107.stdout:2/797: link d6/d20/d22/l52 d6/d18/d40/d42/d69/d6e/da2/d10c/d102/d9f/l117 0 2024-04-25T00:18:35.893 INFO:tasks.workunit.client.0.smithi107.stdout:2/798: fdatasync d6/d18/d40/d42/d69/d6e/da2/d10c/d102/d9f/fa1 0 2024-04-25T00:18:35.902 INFO:tasks.workunit.client.0.smithi107.stdout:6/653: read d1/d2/db/d32/fc6 [2141250,102146] 0 2024-04-25T00:18:35.904 INFO:tasks.workunit.client.0.smithi107.stdout:6/654: link d1/d2/db/c1e d1/d2/db/d3c/d43/d96/dc9/cd5 0 2024-04-25T00:18:35.905 INFO:tasks.workunit.client.0.smithi107.stdout:6/655: symlink d1/d2/d2b/d3e/ld6 0 2024-04-25T00:18:35.915 INFO:tasks.workunit.client.0.smithi107.stdout:2/799: dwrite d6/d29/d4e/d8c/f9b [0,4194304] 0 2024-04-25T00:18:35.916 INFO:tasks.workunit.client.0.smithi107.stdout:0/695: dwrite dc/d24/d38/f66 [0,4194304] 0 2024-04-25T00:18:35.921 INFO:tasks.workunit.client.0.smithi107.stdout:0/696: dread dc/dd/d16/d33/fb6 [0,4194304] 0 2024-04-25T00:18:35.921 INFO:tasks.workunit.client.0.smithi107.stdout:6/656: write d1/d2/db/d32/d36/d3d/fa1 [642901,24078] 0 2024-04-25T00:18:35.923 INFO:tasks.workunit.client.0.smithi107.stdout:6/657: creat d1/d2/d3/fd7 x:0 0 0 2024-04-25T00:18:35.924 INFO:tasks.workunit.client.0.smithi107.stdout:6/658: getdents d1/d2/dad 0 2024-04-25T00:18:35.924 INFO:tasks.workunit.client.0.smithi107.stdout:0/697: symlink dc/dd/d5d/d5c/d6e/lda 0 2024-04-25T00:18:35.924 INFO:tasks.workunit.client.0.smithi107.stdout:0/698: fsync dc/dd/d5d/d3f/f7a 0 2024-04-25T00:18:35.924 INFO:tasks.workunit.client.0.smithi107.stdout:0/699: dread - dc/dd/d16/d33/fb9 zero size 2024-04-25T00:18:35.924 INFO:tasks.workunit.client.0.smithi107.stdout:0/700: chown dc/d24/c9b 467836435 1 2024-04-25T00:18:35.924 INFO:tasks.workunit.client.0.smithi107.stdout:0/701: write dc/da6/fd6 [189010,12570] 0 2024-04-25T00:18:35.924 INFO:tasks.workunit.client.0.smithi107.stdout:0/702: readlink dc/d24/d38/d64/d78/la4 0 2024-04-25T00:18:35.924 INFO:tasks.workunit.client.0.smithi107.stdout:0/703: dread - dc/fbc zero size 2024-04-25T00:18:35.925 INFO:tasks.workunit.client.0.smithi107.stdout:0/704: mkdir dc/dd/d16/d48/ddb 0 2024-04-25T00:18:35.925 INFO:tasks.workunit.client.0.smithi107.stdout:0/705: getdents dc/d24/d38/d8b 0 2024-04-25T00:18:35.925 INFO:tasks.workunit.client.0.smithi107.stdout:0/706: chown dc/d24/d38/d64/d78/l84 9454180 1 2024-04-25T00:18:35.927 INFO:tasks.workunit.client.0.smithi107.stdout:0/707: mkdir dc/d24/d38/d64/d78/ddc 0 2024-04-25T00:18:35.927 INFO:tasks.workunit.client.0.smithi107.stdout:0/708: truncate dc/dd/f3c 4476877 0 2024-04-25T00:18:35.927 INFO:tasks.workunit.client.0.smithi107.stdout:0/709: fdatasync dc/dd/d5d/f7d 0 2024-04-25T00:18:35.927 INFO:tasks.workunit.client.0.smithi107.stdout:0/710: readlink dc/d24/d25/d31/lce 0 2024-04-25T00:18:35.927 INFO:tasks.workunit.client.0.smithi107.stdout:0/711: read dc/d24/d38/d64/f88 [201440,116214] 0 2024-04-25T00:18:35.936 INFO:tasks.workunit.client.0.smithi107.stdout:2/800: dwrite d6/dad/d67/f110 [0,4194304] 0 2024-04-25T00:18:35.936 INFO:tasks.workunit.client.0.smithi107.stdout:2/801: mkdir d6/d20/d22/d56/d94/d10b/d118 0 2024-04-25T00:18:35.937 INFO:tasks.workunit.client.0.smithi107.stdout:2/802: truncate d6/d29/d4e/f7f 1077179 0 2024-04-25T00:18:35.942 INFO:tasks.workunit.client.0.smithi107.stdout:0/712: rename dc/dd/d16/d48/d83/d9f/db0 to dc/d24/d25/ddd 0 2024-04-25T00:18:35.946 INFO:tasks.workunit.client.0.smithi107.stdout:0/713: rmdir dc/d24/d25 39 2024-04-25T00:18:35.946 INFO:tasks.workunit.client.0.smithi107.stdout:0/714: fsync dc/dd/d16/d33/f3d 0 2024-04-25T00:18:35.946 INFO:tasks.workunit.client.0.smithi107.stdout:0/715: chown dc/d24/d38/d64/d7b/c7c 100341461 1 2024-04-25T00:18:35.948 INFO:tasks.workunit.client.0.smithi107.stdout:2/803: read d6/dad/d67/da8/fc6 [2420272,36623] 0 2024-04-25T00:18:35.950 INFO:tasks.workunit.client.0.smithi107.stdout:2/804: rename d6/dad/d67/da8/fc6 to d6/dad/d67/f119 0 2024-04-25T00:18:35.950 INFO:tasks.workunit.client.0.smithi107.stdout:2/805: write d6/d18/d40/d42/d69/f115 [790783,75610] 0 2024-04-25T00:18:35.950 INFO:tasks.workunit.client.0.smithi107.stdout:2/806: dread - d6/d18/d40/d42/d69/d6e/db6/ffd zero size 2024-04-25T00:18:35.950 INFO:tasks.workunit.client.0.smithi107.stdout:2/807: write d6/d18/d40/dc8/f111 [462465,3031] 0 2024-04-25T00:18:35.951 INFO:tasks.workunit.client.0.smithi107.stdout:2/808: stat d6 0 2024-04-25T00:18:35.951 INFO:tasks.workunit.client.0.smithi107.stdout:2/809: fdatasync d6/dad/d67/f119 0 2024-04-25T00:18:35.951 INFO:tasks.workunit.client.0.smithi107.stdout:2/810: write d6/d20/f88 [1771921,129478] 0 2024-04-25T00:18:35.951 INFO:tasks.workunit.client.0.smithi107.stdout:2/811: readlink d6/d29/d4e/d6f/lf6 0 2024-04-25T00:18:35.951 INFO:tasks.workunit.client.0.smithi107.stdout:2/812: mkdir d6/d18/d40/d42/d2b/d30/d45/dca/d11a 0 2024-04-25T00:18:35.951 INFO:tasks.workunit.client.0.smithi107.stdout:2/813: rmdir d6/d18/d40/d42/d69/d6e/da2/d10c 39 2024-04-25T00:18:35.953 INFO:tasks.workunit.client.0.smithi107.stdout:2/814: rmdir d6/dad/d67/da8 39 2024-04-25T00:18:35.956 INFO:tasks.workunit.client.0.smithi107.stdout:0/716: write dc/d24/d38/d64/f68 [964680,77766] 0 2024-04-25T00:18:35.958 INFO:tasks.workunit.client.0.smithi107.stdout:0/717: creat dc/dd/d5d/d3f/da5/fde x:0 0 0 2024-04-25T00:18:35.969 INFO:tasks.workunit.client.0.smithi107.stdout:2/815: rename d6/d18/d40/d42/d2b/c74 to d6/dad/d67/c11b 0 2024-04-25T00:18:35.969 INFO:tasks.workunit.client.0.smithi107.stdout:2/816: write d6/d18/d40/d42/d69/d6e/da2/dc5/dd2/fed [201448,5488] 0 2024-04-25T00:18:35.969 INFO:tasks.workunit.client.0.smithi107.stdout:2/817: fsync d6/d18/d40/d42/f62 0 2024-04-25T00:18:35.975 INFO:tasks.workunit.client.0.smithi107.stdout:0/718: dwrite dc/dd/d16/d5e/fcc [0,4194304] 0 2024-04-25T00:18:35.977 INFO:tasks.workunit.client.0.smithi107.stdout:0/719: creat dc/dd/d5d/fdf x:0 0 0 2024-04-25T00:18:35.977 INFO:tasks.workunit.client.0.smithi107.stdout:0/720: mknod dc/d24/d38/d64/d78/ddc/ce0 0 2024-04-25T00:18:35.979 INFO:tasks.workunit.client.0.smithi107.stdout:0/721: mkdir dc/d24/d38/d8b/dd7/de1 0 2024-04-25T00:18:35.979 INFO:tasks.workunit.client.0.smithi107.stdout:0/722: fsync dc/dd/d5d/f95 0 2024-04-25T00:18:35.979 INFO:tasks.workunit.client.0.smithi107.stdout:0/723: dread - dc/dd/fd8 zero size 2024-04-25T00:18:35.979 INFO:tasks.workunit.client.0.smithi107.stdout:0/724: dread - dc/dd/d5d/d5c/d6e/fcd zero size 2024-04-25T00:18:35.979 INFO:tasks.workunit.client.0.smithi107.stdout:0/725: stat dc/dd/d16/l69 0 2024-04-25T00:18:35.981 INFO:tasks.workunit.client.0.smithi107.stdout:0/726: read dc/d24/d38/d64/f88 [76127,109852] 0 2024-04-25T00:18:35.983 INFO:tasks.workunit.client.0.smithi107.stdout:0/727: rename dc/d24/d38/d64/faa to dc/d24/fe2 0 2024-04-25T00:18:35.984 INFO:tasks.workunit.client.0.smithi107.stdout:0/728: write dc/dd/d16/d33/fb6 [199976,100970] 0 2024-04-25T00:18:35.985 INFO:tasks.workunit.client.0.smithi107.stdout:0/729: mknod dc/da6/ce3 0 2024-04-25T00:18:35.985 INFO:tasks.workunit.client.0.smithi107.stdout:0/730: read dc/da6/fd6 [92784,86051] 0 2024-04-25T00:18:35.985 INFO:tasks.workunit.client.0.smithi107.stdout:0/731: write dc/dd/d5d/d3f/fc8 [532256,114799] 0 2024-04-25T00:18:35.985 INFO:tasks.workunit.client.0.smithi107.stdout:0/732: dread - dc/dd/d5d/d5c/d6e/fcd zero size 2024-04-25T00:18:35.985 INFO:tasks.workunit.client.0.smithi107.stdout:0/733: write dc/d24/d25/f2b [510856,86082] 0 2024-04-25T00:18:35.985 INFO:tasks.workunit.client.0.smithi107.stdout:0/734: chown dc/dd/d5d/d5c/d6e/lda 412 1 2024-04-25T00:18:35.985 INFO:tasks.workunit.client.0.smithi107.stdout:0/735: fsync dc/dd/d16/d33/d35/f9e 0 2024-04-25T00:18:35.986 INFO:tasks.workunit.client.0.smithi107.stdout:2/818: dwrite d6/d18/fb1 [4194304,4194304] 0 2024-04-25T00:18:35.987 INFO:tasks.workunit.client.0.smithi107.stdout:2/819: getdents d6/d18/d40/d42/d69/d6e/d80 0 2024-04-25T00:18:35.998 INFO:tasks.workunit.client.0.smithi107.stdout:0/736: rename dc/dd/d16/d48/f5a to dc/d24/d38/d64/d78/ddc/fe4 0 2024-04-25T00:18:35.999 INFO:tasks.workunit.client.0.smithi107.stdout:5/758: sync 2024-04-25T00:18:35.999 INFO:tasks.workunit.client.0.smithi107.stdout:8/868: sync 2024-04-25T00:18:35.999 INFO:tasks.workunit.client.0.smithi107.stdout:7/728: sync 2024-04-25T00:18:35.999 INFO:tasks.workunit.client.0.smithi107.stdout:9/802: sync 2024-04-25T00:18:35.999 INFO:tasks.workunit.client.0.smithi107.stdout:9/803: chown d0/d8a/d93/f99 864 1 2024-04-25T00:18:35.999 INFO:tasks.workunit.client.0.smithi107.stdout:9/804: chown d0/d8a/d93/f99 56 1 2024-04-25T00:18:35.999 INFO:tasks.workunit.client.0.smithi107.stdout:7/729: fsync d0/d8/db/d18/d19/ff3 0 2024-04-25T00:18:36.000 INFO:tasks.workunit.client.0.smithi107.stdout:5/759: symlink d4/d1c/lfa 0 2024-04-25T00:18:36.001 INFO:tasks.workunit.client.0.smithi107.stdout:8/869: creat d2/d18/d1a/d5e/d5c/d91/f12f x:0 0 0 2024-04-25T00:18:36.003 INFO:tasks.workunit.client.0.smithi107.stdout:5/760: truncate d4/dc/d26/d29/d2e/d55/fa0 2229521 0 2024-04-25T00:18:36.005 INFO:tasks.workunit.client.0.smithi107.stdout:8/870: mkdir d2/d18/d1a/d5e/d5c/d130 0 2024-04-25T00:18:36.005 INFO:tasks.workunit.client.0.smithi107.stdout:8/871: chown d2/d18/d1a/d5e/d3d/d85/dae/l114 2315 1 2024-04-25T00:18:36.007 INFO:tasks.workunit.client.0.smithi107.stdout:2/820: dwrite d6/d18/f5d [0,4194304] 0 2024-04-25T00:18:36.007 INFO:tasks.workunit.client.0.smithi107.stdout:7/730: dread d0/d8/d4e/fa0 [0,4194304] 0 2024-04-25T00:18:36.007 INFO:tasks.workunit.client.0.smithi107.stdout:2/821: stat d6/d18/d40/d42/d2b/d30/d45/dca/dd4/cef 0 2024-04-25T00:18:36.008 INFO:tasks.workunit.client.0.smithi107.stdout:8/872: symlink d2/d18/d1a/d5e/d5c/d6e/l131 0 2024-04-25T00:18:36.010 INFO:tasks.workunit.client.0.smithi107.stdout:5/761: mkdir d4/d17/db3/d84/df0/dfb 0 2024-04-25T00:18:36.011 INFO:tasks.workunit.client.0.smithi107.stdout:7/731: write d0/f26 [560877,35538] 0 2024-04-25T00:18:36.011 INFO:tasks.workunit.client.0.smithi107.stdout:7/732: truncate d0/d8/db/d18/d19/d47/de0/dc2/dd0/fe9 396257 0 2024-04-25T00:18:36.012 INFO:tasks.workunit.client.0.smithi107.stdout:7/733: mkdir d0/d8/d2c/df4 0 2024-04-25T00:18:36.012 INFO:tasks.workunit.client.0.smithi107.stdout:7/734: write d0/d8/db/d18/d19/d67/f72 [1184416,18484] 0 2024-04-25T00:18:36.012 INFO:tasks.workunit.client.0.smithi107.stdout:5/762: unlink d4/dc/c58 0 2024-04-25T00:18:36.012 INFO:tasks.workunit.client.0.smithi107.stdout:5/763: dread - d4/dc/d26/d29/d51/fb8 zero size 2024-04-25T00:18:36.013 INFO:tasks.workunit.client.0.smithi107.stdout:5/764: getdents d4/dc/d26/d29/d2e/d55 0 2024-04-25T00:18:36.014 INFO:tasks.workunit.client.0.smithi107.stdout:5/765: readlink d4/dc/d26/d29/d2e/ldf 0 2024-04-25T00:18:36.014 INFO:tasks.workunit.client.0.smithi107.stdout:5/766: stat d4/dc/d26/d42/d8b 0 2024-04-25T00:18:36.015 INFO:tasks.workunit.client.0.smithi107.stdout:5/767: mkdir d4/dc/d83/dfc 0 2024-04-25T00:18:36.017 INFO:tasks.workunit.client.0.smithi107.stdout:9/805: dread d0/d8a/d93/d56/f64 [0,4194304] 0 2024-04-25T00:18:36.017 INFO:tasks.workunit.client.0.smithi107.stdout:9/806: stat d0/d8a/d93/d6f/ff9 0 2024-04-25T00:18:36.017 INFO:tasks.workunit.client.0.smithi107.stdout:9/807: fsync d0/d8a/d93/d56/d76/d50/d7d/fae 0 2024-04-25T00:18:36.017 INFO:tasks.workunit.client.0.smithi107.stdout:9/808: write d0/d8a/d93/d34/fa9 [772148,12437] 0 2024-04-25T00:18:36.022 INFO:tasks.workunit.client.0.smithi107.stdout:9/809: mkdir d0/d8a/d93/d34/d36/d77/d10f 0 2024-04-25T00:18:36.024 INFO:tasks.workunit.client.0.smithi107.stdout:7/735: write d0/d8/db/d18/d19/d47/d4d/d8a/fb8 [1193519,24864] 0 2024-04-25T00:18:36.024 INFO:tasks.workunit.client.0.smithi107.stdout:8/873: write d2/d18/d1a/d5e/f8c [3735712,85467] 0 2024-04-25T00:18:36.024 INFO:tasks.workunit.client.0.smithi107.stdout:8/874: write d2/d18/d1a/d5e/d4f/f7a [1041912,123558] 0 2024-04-25T00:18:36.024 INFO:tasks.workunit.client.0.smithi107.stdout:9/810: mkdir d0/d8a/d93/d34/d36/d77/d110 0 2024-04-25T00:18:36.024 INFO:tasks.workunit.client.0.smithi107.stdout:7/736: symlink d0/d8/d4e/d46/d93/dd8/lf5 0 2024-04-25T00:18:36.026 INFO:tasks.workunit.client.0.smithi107.stdout:8/875: link d2/d18/d1a/d5e/c3c d2/d18/d1a/d5e/d4f/da0/d110/c132 0 2024-04-25T00:18:36.026 INFO:tasks.workunit.client.0.smithi107.stdout:8/876: dread - d2/d18/d1a/dca/fd8 zero size 2024-04-25T00:18:36.026 INFO:tasks.workunit.client.0.smithi107.stdout:7/737: creat d0/d8/db/d18/d19/d47/de0/d82/ff6 x:0 0 0 2024-04-25T00:18:36.027 INFO:tasks.workunit.client.0.smithi107.stdout:9/811: mkdir d0/d8a/d93/d56/d76/d111 0 2024-04-25T00:18:36.028 INFO:tasks.workunit.client.0.smithi107.stdout:2/822: dwrite d6/d18/d40/d42/d2b/d30/f7c [4194304,4194304] 0 2024-04-25T00:18:36.031 INFO:tasks.workunit.client.0.smithi107.stdout:9/812: mknod d0/d8a/d93/de/d5c/d95/c112 0 2024-04-25T00:18:36.031 INFO:tasks.workunit.client.0.smithi107.stdout:9/813: chown d0/d8a/d93/d34/d36/l66 217647 1 2024-04-25T00:18:36.031 INFO:tasks.workunit.client.0.smithi107.stdout:9/814: write d0/d8a/d93/de/d5c/f72 [1420685,35136] 0 2024-04-25T00:18:36.034 INFO:tasks.workunit.client.0.smithi107.stdout:2/823: truncate d6/d18/d40/d42/d69/d6e/da2/d10c/d102/d9f/fa1 3470832 0 2024-04-25T00:18:36.035 INFO:tasks.workunit.client.0.smithi107.stdout:5/768: dwrite d4/f1b [0,4194304] 0 2024-04-25T00:18:36.035 INFO:tasks.workunit.client.0.smithi107.stdout:5/769: readlink d4/dc/d26/d29/d51/ldd 0 2024-04-25T00:18:36.035 INFO:tasks.workunit.client.0.smithi107.stdout:5/770: truncate d4/f1b 5026404 0 2024-04-25T00:18:36.039 INFO:tasks.workunit.client.0.smithi107.stdout:5/771: link d4/d1c/d5c/lf9 d4/d17/db3/d84/df0/dfb/lfd 0 2024-04-25T00:18:36.041 INFO:tasks.workunit.client.0.smithi107.stdout:7/738: dread d0/d8/db/d18/d19/d39/f6f [0,4194304] 0 2024-04-25T00:18:36.041 INFO:tasks.workunit.client.0.smithi107.stdout:7/739: write d0/d8/db/f15 [107230,104563] 0 2024-04-25T00:18:36.041 INFO:tasks.workunit.client.0.smithi107.stdout:7/740: write d0/d8/f78 [930680,45073] 0 2024-04-25T00:18:36.041 INFO:tasks.workunit.client.0.smithi107.stdout:9/815: mkdir d0/d8a/d93/d56/d6c/d80/d113 0 2024-04-25T00:18:36.041 INFO:tasks.workunit.client.0.smithi107.stdout:9/816: chown d0/d8a/d93/d1f/fb6 452 1 2024-04-25T00:18:36.041 INFO:tasks.workunit.client.0.smithi107.stdout:9/817: readlink d0/d43/l12 0 2024-04-25T00:18:36.041 INFO:tasks.workunit.client.0.smithi107.stdout:9/818: write d0/d8a/d93/d34/f10d [70554,69104] 0 2024-04-25T00:18:36.041 INFO:tasks.workunit.client.0.smithi107.stdout:9/819: read d0/d8a/d93/f99 [260357,71471] 0 2024-04-25T00:18:36.042 INFO:tasks.workunit.client.0.smithi107.stdout:9/820: mknod d0/d8a/d93/d34/c114 0 2024-04-25T00:18:36.044 INFO:tasks.workunit.client.0.smithi107.stdout:9/821: mknod d0/d8a/d93/de/d5c/d95/d106/c115 0 2024-04-25T00:18:36.050 INFO:tasks.workunit.client.0.smithi107.stdout:7/741: write d0/d8/db/d18/d19/d39/f96 [693432,49876] 0 2024-04-25T00:18:36.051 INFO:tasks.workunit.client.0.smithi107.stdout:5/772: dread d4/dc/d26/f30 [0,4194304] 0 2024-04-25T00:18:36.052 INFO:tasks.workunit.client.0.smithi107.stdout:7/742: creat d0/d8/d2c/df4/ff7 x:0 0 0 2024-04-25T00:18:36.052 INFO:tasks.workunit.client.0.smithi107.stdout:5/773: mkdir d4/dc/d26/d29/d2e/d82/dfe 0 2024-04-25T00:18:36.053 INFO:tasks.workunit.client.0.smithi107.stdout:5/774: fsync d4/d17/fd6 0 2024-04-25T00:18:36.053 INFO:tasks.workunit.client.0.smithi107.stdout:2/824: dwrite d6/d18/d40/d42/d2b/d30/d4c/d10e/fb8 [0,4194304] 0 2024-04-25T00:18:36.054 INFO:tasks.workunit.client.0.smithi107.stdout:7/743: rename d0/lda to d0/d8/d4e/d46/d93/lf8 0 2024-04-25T00:18:36.056 INFO:tasks.workunit.client.0.smithi107.stdout:9/822: dread d0/d8a/d93/d56/d6c/d80/f104 [0,4194304] 0 2024-04-25T00:18:36.056 INFO:tasks.workunit.client.0.smithi107.stdout:7/744: symlink d0/d8/db/d18/d19/lf9 0 2024-04-25T00:18:36.057 INFO:tasks.workunit.client.0.smithi107.stdout:7/745: symlink d0/d8/db/d18/d19/d47/de0/dc2/lfa 0 2024-04-25T00:18:36.057 INFO:tasks.workunit.client.0.smithi107.stdout:7/746: chown d0/d8/f78 45 1 2024-04-25T00:18:36.058 INFO:tasks.workunit.client.0.smithi107.stdout:7/747: creat d0/d8/db/d18/d19/d47/d4d/d8a/ffb x:0 0 0 2024-04-25T00:18:36.059 INFO:tasks.workunit.client.0.smithi107.stdout:7/748: symlink d0/d8/d2c/lfc 0 2024-04-25T00:18:36.059 INFO:tasks.workunit.client.0.smithi107.stdout:7/749: dread - d0/d8/db/d18/d19/d47/d4d/d8a/ffb zero size 2024-04-25T00:18:36.061 INFO:tasks.workunit.client.0.smithi107.stdout:7/750: creat d0/d8/db/d18/d19/d47/de0/d82/ffd x:0 0 0 2024-04-25T00:18:36.063 INFO:tasks.workunit.client.0.smithi107.stdout:7/751: mkdir d0/d8/db/d18/d19/d39/dfe 0 2024-04-25T00:18:36.063 INFO:tasks.workunit.client.0.smithi107.stdout:7/752: truncate d0/d8/db/d18/d19/d47/de0/d82/ff6 937691 0 2024-04-25T00:18:36.065 INFO:tasks.workunit.client.0.smithi107.stdout:5/775: dread d4/dc/f1f [0,4194304] 0 2024-04-25T00:18:36.066 INFO:tasks.workunit.client.0.smithi107.stdout:5/776: symlink d4/dc/d83/dfc/lff 0 2024-04-25T00:18:36.069 INFO:tasks.workunit.client.0.smithi107.stdout:5/777: creat d4/dc/d83/f100 x:0 0 0 2024-04-25T00:18:36.070 INFO:tasks.workunit.client.0.smithi107.stdout:5/778: symlink d4/dc/d26/d42/d8b/l101 0 2024-04-25T00:18:36.071 INFO:tasks.workunit.client.0.smithi107.stdout:5/779: fdatasync d4/dc/f7f 0 2024-04-25T00:18:36.071 INFO:tasks.workunit.client.0.smithi107.stdout:5/780: fsync d4/d17/fd6 0 2024-04-25T00:18:36.073 INFO:tasks.workunit.client.0.smithi107.stdout:5/781: getdents d4/dc/d26/d62 0 2024-04-25T00:18:36.074 INFO:tasks.workunit.client.0.smithi107.stdout:7/753: getdents d0/d8/db/d18/d19 0 2024-04-25T00:18:36.075 INFO:tasks.workunit.client.0.smithi107.stdout:7/754: fsync d0/d8/d4e/fa0 0 2024-04-25T00:18:36.075 INFO:tasks.workunit.client.0.smithi107.stdout:7/755: readlink d0/l48 0 2024-04-25T00:18:36.075 INFO:tasks.workunit.client.0.smithi107.stdout:7/756: read d0/d8/d4e/d46/d93/fac [194889,115328] 0 2024-04-25T00:18:36.075 INFO:tasks.workunit.client.0.smithi107.stdout:5/782: stat d4/d17/db3/d84/c8d 0 2024-04-25T00:18:36.077 INFO:tasks.workunit.client.0.smithi107.stdout:2/825: dwrite d6/d18/d40/d42/d69/d6e/fe3 [0,4194304] 0 2024-04-25T00:18:36.077 INFO:tasks.workunit.client.0.smithi107.stdout:2/826: stat d6/dad/d67/l54 0 2024-04-25T00:18:36.077 INFO:tasks.workunit.client.0.smithi107.stdout:2/827: getdents d6/dad/d67/da8 0 2024-04-25T00:18:36.078 INFO:tasks.workunit.client.0.smithi107.stdout:9/823: dwrite d0/d8a/d93/f55 [0,4194304] 0 2024-04-25T00:18:36.078 INFO:tasks.workunit.client.0.smithi107.stdout:9/824: dread - d0/d8a/d93/d34/ffe zero size 2024-04-25T00:18:36.082 INFO:tasks.workunit.client.0.smithi107.stdout:2/828: truncate d6/d18/d40/d42/f62 2939559 0 2024-04-25T00:18:36.083 INFO:tasks.workunit.client.0.smithi107.stdout:9/825: symlink d0/d8a/l116 0 2024-04-25T00:18:36.084 INFO:tasks.workunit.client.0.smithi107.stdout:2/829: unlink d6/d18/d40/d42/d69/d6e/da2/dc5/dd2/fed 0 2024-04-25T00:18:36.085 INFO:tasks.workunit.client.0.smithi107.stdout:7/757: write d0/d8/db/d18/d19/d47/d4d/dbc/f85 [2557857,108650] 0 2024-04-25T00:18:36.086 INFO:tasks.workunit.client.0.smithi107.stdout:7/758: creat d0/d8/fff x:0 0 0 2024-04-25T00:18:36.086 INFO:tasks.workunit.client.0.smithi107.stdout:7/759: write d0/d8/d4e/d46/d93/fac [152221,62695] 0 2024-04-25T00:18:36.088 INFO:tasks.workunit.client.0.smithi107.stdout:7/760: unlink d0/d8/db/d18/d19/d47/d4d/dbc/le4 0 2024-04-25T00:18:36.088 INFO:tasks.workunit.client.0.smithi107.stdout:7/761: readlink d0/d8/db/d18/d19/d47/de0/lb1 0 2024-04-25T00:18:36.090 INFO:tasks.workunit.client.0.smithi107.stdout:9/826: creat d0/f117 x:0 0 0 2024-04-25T00:18:36.090 INFO:tasks.workunit.client.0.smithi107.stdout:7/762: unlink d0/d8/db/f49 0 2024-04-25T00:18:36.091 INFO:tasks.workunit.client.0.smithi107.stdout:2/830: creat d6/d18/d40/d42/d2b/f11c x:0 0 0 2024-04-25T00:18:36.092 INFO:tasks.workunit.client.0.smithi107.stdout:9/827: mknod d0/d8a/d93/d56/d76/d50/d7d/c118 0 2024-04-25T00:18:36.092 INFO:tasks.workunit.client.0.smithi107.stdout:9/828: chown d0/d8a/d93/d34/lec 0 1 2024-04-25T00:18:36.092 INFO:tasks.workunit.client.0.smithi107.stdout:7/763: unlink d0/d8/db/d18/d19/d47/de0/fed 0 2024-04-25T00:18:36.093 INFO:tasks.workunit.client.0.smithi107.stdout:7/764: symlink d0/d80/da8/l100 0 2024-04-25T00:18:36.094 INFO:tasks.workunit.client.0.smithi107.stdout:9/829: creat d0/d8a/d93/de/d5c/d95/d106/f119 x:0 0 0 2024-04-25T00:18:36.095 INFO:tasks.workunit.client.0.smithi107.stdout:9/830: link d0/d8a/d93/d56/d76/d50/dd8/c10e d0/d8a/d93/c11a 0 2024-04-25T00:18:36.108 INFO:tasks.workunit.client.0.smithi107.stdout:2/831: dwrite d6/d18/d40/d42/d69/d6e/dc2/fe7 [0,4194304] 0 2024-04-25T00:18:36.114 INFO:tasks.workunit.client.0.smithi107.stdout:7/765: dwrite d0/d8/db/d18/d19/d47/d4d/dbc/f85 [0,4194304] 0 2024-04-25T00:18:36.114 INFO:tasks.workunit.client.0.smithi107.stdout:7/766: stat d0/d8/db/d18/d19/d39/c3a 0 2024-04-25T00:18:36.114 INFO:tasks.workunit.client.0.smithi107.stdout:7/767: chown d0/d8/db/d18/d19/d39/c45 62 1 2024-04-25T00:18:36.121 INFO:tasks.workunit.client.0.smithi107.stdout:2/832: read d6/d20/f5a [222042,37291] 0 2024-04-25T00:18:36.123 INFO:tasks.workunit.client.0.smithi107.stdout:2/833: mkdir d6/d18/d40/d42/d2b/d11d 0 2024-04-25T00:18:36.123 INFO:tasks.workunit.client.0.smithi107.stdout:7/768: stat d0 0 2024-04-25T00:18:36.123 INFO:tasks.workunit.client.0.smithi107.stdout:7/769: fsync d0/d8/d2c/f55 0 2024-04-25T00:18:36.126 INFO:tasks.workunit.client.0.smithi107.stdout:2/834: dread d6/d18/d40/dc8/de5/f105 [0,4194304] 0 2024-04-25T00:18:36.126 INFO:tasks.workunit.client.0.smithi107.stdout:2/835: stat d6/d29/d4e/dc0 0 2024-04-25T00:18:36.127 INFO:tasks.workunit.client.0.smithi107.stdout:2/836: mknod d6/d18/d40/dc8/c11e 0 2024-04-25T00:18:36.128 INFO:tasks.workunit.client.0.smithi107.stdout:2/837: symlink d6/d18/d40/d42/d70/l11f 0 2024-04-25T00:18:36.131 INFO:tasks.workunit.client.0.smithi107.stdout:7/770: write d0/d8/d4e/fa0 [1313410,18446] 0 2024-04-25T00:18:36.131 INFO:tasks.workunit.client.0.smithi107.stdout:7/771: fdatasync d0/d8/db/d18/f61 0 2024-04-25T00:18:36.131 INFO:tasks.workunit.client.0.smithi107.stdout:7/772: rmdir d0/d8/db/d18 39 2024-04-25T00:18:36.131 INFO:tasks.workunit.client.0.smithi107.stdout:7/773: stat d0/d8/db/d18/f8d 0 2024-04-25T00:18:36.132 INFO:tasks.workunit.client.0.smithi107.stdout:7/774: chown d0/d8/db/d92/cc5 3041 1 2024-04-25T00:18:36.132 INFO:tasks.workunit.client.0.smithi107.stdout:7/775: stat d0/d8/d4e/d46/d93/fbd 0 2024-04-25T00:18:36.133 INFO:tasks.workunit.client.0.smithi107.stdout:5/783: rename d4/d17/db3/d84/fb0 to d4/dc/f102 0 2024-04-25T00:18:36.134 INFO:tasks.workunit.client.0.smithi107.stdout:5/784: mknod d4/dc/d26/d42/d8b/c103 0 2024-04-25T00:18:36.136 INFO:tasks.workunit.client.0.smithi107.stdout:9/831: rename d0/d8a/d93/de/d5c/d95/d106/c115 to d0/d8a/d93/d56/d76/d111/c11b 0 2024-04-25T00:18:36.136 INFO:tasks.workunit.client.0.smithi107.stdout:7/776: rename d0/d8/d4e/d46/d93 to d0/d8/d4e/d46/d93/dd8/d101 22 2024-04-25T00:18:36.136 INFO:tasks.workunit.client.0.smithi107.stdout:9/832: readlink d0/d8a/d93/d56/d76/d50/dd8/le1 0 2024-04-25T00:18:36.136 INFO:tasks.workunit.client.0.smithi107.stdout:9/833: stat d0/d8a/d93/db9/led 0 2024-04-25T00:18:36.136 INFO:tasks.workunit.client.0.smithi107.stdout:6/659: sync 2024-04-25T00:18:36.136 INFO:tasks.workunit.client.0.smithi107.stdout:4/835: sync 2024-04-25T00:18:36.136 INFO:tasks.workunit.client.0.smithi107.stdout:3/855: sync 2024-04-25T00:18:36.136 INFO:tasks.workunit.client.0.smithi107.stdout:1/768: sync 2024-04-25T00:18:36.136 INFO:tasks.workunit.client.0.smithi107.stdout:8/877: sync 2024-04-25T00:18:36.136 INFO:tasks.workunit.client.0.smithi107.stdout:0/737: sync 2024-04-25T00:18:36.137 INFO:tasks.workunit.client.0.smithi107.stdout:9/834: mkdir d0/d8a/d93/d34/d36/d77/dac/dee/d11c 0 2024-04-25T00:18:36.139 INFO:tasks.workunit.client.0.smithi107.stdout:0/738: truncate dc/d24/d38/d64/d78/ddc/fe4 687786 0 2024-04-25T00:18:36.139 INFO:tasks.workunit.client.0.smithi107.stdout:4/836: mknod d3/d5/d32/d53/c116 0 2024-04-25T00:18:36.139 INFO:tasks.workunit.client.0.smithi107.stdout:6/660: creat d1/d2/db/d32/d36/fd8 x:0 0 0 2024-04-25T00:18:36.139 INFO:tasks.workunit.client.0.smithi107.stdout:4/837: stat d3/d5/d32/d53/l63 0 2024-04-25T00:18:36.139 INFO:tasks.workunit.client.0.smithi107.stdout:4/838: readlink d3/d24/d25/l48 0 2024-04-25T00:18:36.140 INFO:tasks.workunit.client.0.smithi107.stdout:2/838: dread d6/d18/d40/d42/d69/d6e/da2/d10c/d102/d9f/fe1 [0,4194304] 0 2024-04-25T00:18:36.141 INFO:tasks.workunit.client.0.smithi107.stdout:2/839: stat d6/dad/d67/da8 0 2024-04-25T00:18:36.141 INFO:tasks.workunit.client.0.smithi107.stdout:0/739: creat dc/dd/d16/d5e/fe5 x:0 0 0 2024-04-25T00:18:36.141 INFO:tasks.workunit.client.0.smithi107.stdout:0/740: truncate dc/dd/d5d/f95 687587 0 2024-04-25T00:18:36.141 INFO:tasks.workunit.client.0.smithi107.stdout:0/741: fdatasync dc/dd/d5d/d5c/d6e/f9c 0 2024-04-25T00:18:36.141 INFO:tasks.workunit.client.0.smithi107.stdout:9/835: symlink d0/d8a/d93/d34/d36/d77/dac/l11d 0 2024-04-25T00:18:36.141 INFO:tasks.workunit.client.0.smithi107.stdout:4/839: mknod d3/d24/d25/d71/d7f/c117 0 2024-04-25T00:18:36.143 INFO:tasks.workunit.client.0.smithi107.stdout:2/840: unlink d6/d20/d22/f1c 0 2024-04-25T00:18:36.144 INFO:tasks.workunit.client.0.smithi107.stdout:0/742: mkdir dc/d24/d38/d64/d78/de6 0 2024-04-25T00:18:36.146 INFO:tasks.workunit.client.0.smithi107.stdout:7/777: read d0/d8/db/f2a [151282,117634] 0 2024-04-25T00:18:36.146 INFO:tasks.workunit.client.0.smithi107.stdout:7/778: chown d0/f1d 64329 1 2024-04-25T00:18:36.147 INFO:tasks.workunit.client.0.smithi107.stdout:9/836: rename d0/d8a/d93/d6f/da1/cbf to d0/d8a/c11e 0 2024-04-25T00:18:36.147 INFO:tasks.workunit.client.0.smithi107.stdout:9/837: fdatasync d0/d43/f1c 0 2024-04-25T00:18:36.148 INFO:tasks.workunit.client.0.smithi107.stdout:7/779: symlink d0/d8/d4e/d46/d93/l102 0 2024-04-25T00:18:36.148 INFO:tasks.workunit.client.0.smithi107.stdout:7/780: chown d0/d8/d4e/d46/d93/l102 2731 1 2024-04-25T00:18:36.149 INFO:tasks.workunit.client.0.smithi107.stdout:7/781: mknod d0/d8/db/d92/c103 0 2024-04-25T00:18:36.150 INFO:tasks.workunit.client.0.smithi107.stdout:7/782: mknod d0/d8/db/d18/d19/c104 0 2024-04-25T00:18:36.151 INFO:tasks.workunit.client.0.smithi107.stdout:7/783: symlink d0/d8/d4e/d46/l105 0 2024-04-25T00:18:36.162 INFO:tasks.workunit.client.0.smithi107.stdout:1/769: dwrite d2/da/d2f/f97 [0,4194304] 0 2024-04-25T00:18:36.164 INFO:tasks.workunit.client.0.smithi107.stdout:7/784: write d0/d8/db/d18/d19/d47/da1/fae [674656,128847] 0 2024-04-25T00:18:36.166 INFO:tasks.workunit.client.0.smithi107.stdout:1/770: creat d2/d2a/d5c/ff4 x:0 0 0 2024-04-25T00:18:36.167 INFO:tasks.workunit.client.0.smithi107.stdout:8/878: dwrite d2/ff [4194304,4194304] 0 2024-04-25T00:18:36.167 INFO:tasks.workunit.client.0.smithi107.stdout:7/785: symlink d0/d8/db/d18/d19/d47/d4d/l106 0 2024-04-25T00:18:36.168 INFO:tasks.workunit.client.0.smithi107.stdout:1/771: rename d2/dd/d1b/fdc to d2/d2a/d37/d88/ff5 0 2024-04-25T00:18:36.170 INFO:tasks.workunit.client.0.smithi107.stdout:8/879: link d2/d18/d1a/d5e/d5c/d91/da3/fb8 d2/d18/d1a/d5e/d3e/db1/df0/f133 0 2024-04-25T00:18:36.171 INFO:tasks.workunit.client.0.smithi107.stdout:1/772: creat d2/da/db7/ff6 x:0 0 0 2024-04-25T00:18:36.173 INFO:tasks.workunit.client.0.smithi107.stdout:8/880: rename d2/d18/d1a/d5e/d5c/d107/dc2/fd7 to d2/d18/d1a/d5e/d3e/db1/f134 0 2024-04-25T00:18:36.174 INFO:tasks.workunit.client.0.smithi107.stdout:1/773: stat d2/da/f3b 0 2024-04-25T00:18:36.175 INFO:tasks.workunit.client.0.smithi107.stdout:8/881: creat d2/d18/d1a/d5e/d4f/da0/da1/de8/f135 x:0 0 0 2024-04-25T00:18:36.176 INFO:tasks.workunit.client.0.smithi107.stdout:1/774: mknod d2/d62/cf7 0 2024-04-25T00:18:36.177 INFO:tasks.workunit.client.0.smithi107.stdout:9/838: dwrite d0/d8a/d93/de/d5c/f7b [0,4194304] 0 2024-04-25T00:18:36.177 INFO:tasks.workunit.client.0.smithi107.stdout:6/661: dwrite d1/f1f [4194304,4194304] 0 2024-04-25T00:18:36.177 INFO:tasks.workunit.client.0.smithi107.stdout:6/662: stat d1/d2/db/d3c/cbd 0 2024-04-25T00:18:36.178 INFO:tasks.workunit.client.0.smithi107.stdout:8/882: creat d2/d18/d1a/d5e/d3d/d85/dd5/f136 x:0 0 0 2024-04-25T00:18:36.178 INFO:tasks.workunit.client.0.smithi107.stdout:8/883: write d2/d18/d1a/dca/f12d [202694,94906] 0 2024-04-25T00:18:36.178 INFO:tasks.workunit.client.0.smithi107.stdout:6/663: symlink d1/d2/d3/d28/d79/d6b/ld9 0 2024-04-25T00:18:36.180 INFO:tasks.workunit.client.0.smithi107.stdout:4/840: link d3/d24/d25/d71/f7c d3/d5/d32/d53/dc0/f118 0 2024-04-25T00:18:36.181 INFO:tasks.workunit.client.0.smithi107.stdout:1/775: creat d2/dd/dec/ff8 x:0 0 0 2024-04-25T00:18:36.183 INFO:tasks.workunit.client.0.smithi107.stdout:9/839: rename d0/d8a/d93/de/d5c/d95/c87 to d0/dfb/c11f 0 2024-04-25T00:18:36.183 INFO:tasks.workunit.client.0.smithi107.stdout:9/840: chown d0/d8a/d93/d1f 3771 1 2024-04-25T00:18:36.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:35 smithi170 ceph-mon[32807]: pgmap v67: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 118 MiB/s rd, 143 MiB/s wr, 223 op/s 2024-04-25T00:18:36.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:35 smithi170 ceph-mon[32807]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:18:36.187 INFO:tasks.workunit.client.0.smithi107.stdout:2/841: dread d6/d29/d4e/fab [0,4194304] 0 2024-04-25T00:18:36.187 INFO:tasks.workunit.client.0.smithi107.stdout:5/785: dwrite d4/f48 [0,4194304] 0 2024-04-25T00:18:36.187 INFO:tasks.workunit.client.0.smithi107.stdout:5/786: chown d4/d17/db3/d84/df0/dfb 43 1 2024-04-25T00:18:36.187 INFO:tasks.workunit.client.0.smithi107.stdout:7/786: dwrite d0/d80/da2/fad [0,4194304] 0 2024-04-25T00:18:36.192 INFO:tasks.workunit.client.0.smithi107.stdout:7/787: creat d0/d8/db/d18/d19/f107 x:0 0 0 2024-04-25T00:18:36.193 INFO:tasks.workunit.client.0.smithi107.stdout:7/788: chown d0/d8/db/d18/d19/d47/da1/cd5 677343390 1 2024-04-25T00:18:36.203 INFO:tasks.workunit.client.0.smithi107.stdout:9/841: dwrite d0/d8a/d93/d34/d36/d77/dac/dee/f10b [0,4194304] 0 2024-04-25T00:18:36.203 INFO:tasks.workunit.client.0.smithi107.stdout:6/664: dwrite d1/d2/db/d32/d36/d3d/fa1 [0,4194304] 0 2024-04-25T00:18:36.204 INFO:tasks.workunit.client.0.smithi107.stdout:8/884: dwrite d2/d18/d1a/d5e/d5c/f88 [0,4194304] 0 2024-04-25T00:18:36.211 INFO:tasks.workunit.client.0.smithi107.stdout:2/842: symlink d6/d29/d4e/dc0/dd1/l120 0 2024-04-25T00:18:36.214 INFO:tasks.workunit.client.0.smithi107.stdout:9/842: write d0/f94 [2199925,22302] 0 2024-04-25T00:18:36.214 INFO:tasks.workunit.client.0.smithi107.stdout:9/843: write d0/d8a/d93/d56/d76/d50/fdb [1284473,93140] 0 2024-04-25T00:18:36.220 INFO:tasks.workunit.client.0.smithi107.stdout:1/776: rename d2/d2a/lbd to d2/d2a/d37/lf9 0 2024-04-25T00:18:36.225 INFO:tasks.workunit.client.0.smithi107.stdout:6/665: dwrite d1/d2/db/d32/d36/d3a/fa7 [0,4194304] 0 2024-04-25T00:18:36.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:35 smithi107 ceph-mon[29153]: pgmap v67: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 118 MiB/s rd, 143 MiB/s wr, 223 op/s 2024-04-25T00:18:36.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:35 smithi107 ceph-mon[29153]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:18:36.236 INFO:tasks.workunit.client.0.smithi107.stdout:9/844: dwrite d0/f8f [0,4194304] 0 2024-04-25T00:18:36.236 INFO:tasks.workunit.client.0.smithi107.stdout:9/845: chown d0/d8a/d93/d34/d36/lbe 85174894 1 2024-04-25T00:18:36.236 INFO:tasks.workunit.client.0.smithi107.stdout:2/843: dwrite d6/d18/d40/d42/f3f [0,4194304] 0 2024-04-25T00:18:36.237 INFO:tasks.workunit.client.0.smithi107.stdout:8/885: dwrite d2/d18/d1a/d5e/d6d/f6f [0,4194304] 0 2024-04-25T00:18:36.241 INFO:tasks.workunit.client.0.smithi107.stdout:6/666: dwrite d1/d2/d3/d28/f53 [0,4194304] 0 2024-04-25T00:18:36.245 INFO:tasks.workunit.client.0.smithi107.stdout:6/667: unlink d1/d2/d2b/d3e/ld6 0 2024-04-25T00:18:36.245 INFO:tasks.workunit.client.0.smithi107.stdout:6/668: write d1/d2/db/d32/d36/fd8 [933853,35977] 0 2024-04-25T00:18:36.245 INFO:tasks.workunit.client.0.smithi107.stdout:6/669: stat d1/d2/db/d32/d33/fc3 0 2024-04-25T00:18:36.245 INFO:tasks.workunit.client.0.smithi107.stdout:6/670: truncate d1/d4c/f70 1012229 0 2024-04-25T00:18:36.250 INFO:tasks.workunit.client.0.smithi107.stdout:1/777: mkdir d2/d62/dfa 0 2024-04-25T00:18:36.253 INFO:tasks.workunit.client.0.smithi107.stdout:9/846: mknod d0/d8a/d93/d56/d6c/d80/d113/c120 0 2024-04-25T00:18:36.253 INFO:tasks.workunit.client.0.smithi107.stdout:1/778: dread d2/da/d2f/f8d [0,4194304] 0 2024-04-25T00:18:36.258 INFO:tasks.workunit.client.0.smithi107.stdout:2/844: unlink d6/d18/d40/d42/d69/fb2 0 2024-04-25T00:18:36.259 INFO:tasks.workunit.client.0.smithi107.stdout:2/845: dread d6/d18/d40/f99 [0,4194304] 0 2024-04-25T00:18:36.262 INFO:tasks.workunit.client.0.smithi107.stdout:1/779: link d2/da/d2f/f3e d2/dd/d25/da9/ffb 0 2024-04-25T00:18:36.264 INFO:tasks.workunit.client.0.smithi107.stdout:6/671: dwrite d1/d2/d2b/d3e/fa4 [0,4194304] 0 2024-04-25T00:18:36.264 INFO:tasks.workunit.client.0.smithi107.stdout:8/886: dwrite d2/d18/d1a/d5e/d3e/d74/f97 [0,4194304] 0 2024-04-25T00:18:36.264 INFO:tasks.workunit.client.0.smithi107.stdout:6/672: chown d1/f1f 9354330 1 2024-04-25T00:18:36.266 INFO:tasks.workunit.client.0.smithi107.stdout:9/847: getdents d0/d8a/d93/de/d5c/d95/d106 0 2024-04-25T00:18:36.271 INFO:tasks.workunit.client.0.smithi107.stdout:6/673: truncate d1/d2/d3/f41 3859779 0 2024-04-25T00:18:36.272 INFO:tasks.workunit.client.0.smithi107.stdout:1/780: mknod d2/d2a/d5c/cfc 0 2024-04-25T00:18:36.273 INFO:tasks.workunit.client.0.smithi107.stdout:6/674: unlink d1/d2/db/d32/d36/d3d/ca8 0 2024-04-25T00:18:36.273 INFO:tasks.workunit.client.0.smithi107.stdout:6/675: chown d1/d2/dad 142064359 1 2024-04-25T00:18:36.287 INFO:tasks.workunit.client.0.smithi107.stdout:8/887: truncate d2/d18/d1a/d5e/d3e/f99 2428584 0 2024-04-25T00:18:36.289 INFO:tasks.workunit.client.0.smithi107.stdout:9/848: creat d0/d8a/d93/db9/f121 x:0 0 0 2024-04-25T00:18:36.291 INFO:tasks.workunit.client.0.smithi107.stdout:6/676: dwrite d1/d2/db/d48/fd2 [0,4194304] 0 2024-04-25T00:18:36.291 INFO:tasks.workunit.client.0.smithi107.stdout:6/677: fsync d1/d2/db/d32/d36/d3d/f5e 0 2024-04-25T00:18:36.294 INFO:tasks.workunit.client.0.smithi107.stdout:6/678: unlink d1/d2/db/d32/d36/d3a/l40 0 2024-04-25T00:18:36.294 INFO:tasks.workunit.client.0.smithi107.stdout:1/781: mknod d2/dd/d1b/cfd 0 2024-04-25T00:18:36.296 INFO:tasks.workunit.client.0.smithi107.stdout:6/679: creat d1/d2/db/d32/d33/d68/d75/fda x:0 0 0 2024-04-25T00:18:36.296 INFO:tasks.workunit.client.0.smithi107.stdout:6/680: stat d1/d2/db/d32/d33/c39 0 2024-04-25T00:18:36.297 INFO:tasks.workunit.client.0.smithi107.stdout:6/681: mknod d1/d2/d2b/cdb 0 2024-04-25T00:18:36.299 INFO:tasks.workunit.client.0.smithi107.stdout:6/682: mknod d1/d2/db/d3c/d43/d96/dc9/cdc 0 2024-04-25T00:18:36.299 INFO:tasks.workunit.client.0.smithi107.stdout:6/683: readlink d1/d2/db/d32/d36/d3d/l84 0 2024-04-25T00:18:36.301 INFO:tasks.workunit.client.0.smithi107.stdout:5/787: mkdir d4/d17/d104 0 2024-04-25T00:18:36.302 INFO:tasks.workunit.client.0.smithi107.stdout:9/849: symlink d0/d8a/d93/de/d5c/de0/l122 0 2024-04-25T00:18:36.303 INFO:tasks.workunit.client.0.smithi107.stdout:6/684: mkdir d1/d14/d18/ddd 0 2024-04-25T00:18:36.304 INFO:tasks.workunit.client.0.smithi107.stdout:9/850: creat d0/d8a/d93/dc2/dc6/f123 x:0 0 0 2024-04-25T00:18:36.306 INFO:tasks.workunit.client.0.smithi107.stdout:9/851: rename d0/d8a/d93/d56/d76/d50/de4 to d0/d8a/d93/de/d5c/d124 0 2024-04-25T00:18:36.309 INFO:tasks.workunit.client.0.smithi107.stdout:9/852: truncate d0/d8a/d93/d1f/f8b 3151356 0 2024-04-25T00:18:36.309 INFO:tasks.workunit.client.0.smithi107.stdout:9/853: chown d0/c3a 241525 1 2024-04-25T00:18:36.309 INFO:tasks.workunit.client.0.smithi107.stdout:9/854: stat d0/d8a/d93/c4c 0 2024-04-25T00:18:36.312 INFO:tasks.workunit.client.0.smithi107.stdout:1/782: dwrite d2/dd/d25/f50 [0,4194304] 0 2024-04-25T00:18:36.317 INFO:tasks.workunit.client.0.smithi107.stdout:1/783: mkdir d2/dfe 0 2024-04-25T00:18:36.317 INFO:tasks.workunit.client.0.smithi107.stdout:1/784: write d2/d2a/f9e [4420988,52244] 0 2024-04-25T00:18:36.317 INFO:tasks.workunit.client.0.smithi107.stdout:1/785: readlink d2/l3c 0 2024-04-25T00:18:36.320 INFO:tasks.workunit.client.0.smithi107.stdout:7/789: symlink d0/l108 0 2024-04-25T00:18:36.320 INFO:tasks.workunit.client.0.smithi107.stdout:7/790: dread - d0/d8/db/d18/d19/d47/de0/fc9 zero size 2024-04-25T00:18:36.321 INFO:tasks.workunit.client.0.smithi107.stdout:4/841: symlink d3/d24/d5c/l119 0 2024-04-25T00:18:36.321 INFO:tasks.workunit.client.0.smithi107.stdout:4/842: chown d3/d5/l66 19225 1 2024-04-25T00:18:36.322 INFO:tasks.workunit.client.0.smithi107.stdout:1/786: symlink d2/d2a/d5c/d89/dc0/lff 0 2024-04-25T00:18:36.323 INFO:tasks.workunit.client.0.smithi107.stdout:4/843: mknod d3/d24/d25/d71/c11a 0 2024-04-25T00:18:36.324 INFO:tasks.workunit.client.0.smithi107.stdout:6/685: dwrite d1/d2/db/d32/d36/f46 [0,4194304] 0 2024-04-25T00:18:36.325 INFO:tasks.workunit.client.0.smithi107.stdout:7/791: link d0/l64 d0/d8/db/d18/d19/d47/l109 0 2024-04-25T00:18:36.330 INFO:tasks.workunit.client.0.smithi107.stdout:9/855: dwrite d0/d8a/d93/de/f90 [0,4194304] 0 2024-04-25T00:18:36.330 INFO:tasks.workunit.client.0.smithi107.stdout:9/856: chown d0/d8a/d93/d34/d36/d77/ccf 18567349 1 2024-04-25T00:18:36.332 INFO:tasks.workunit.client.0.smithi107.stdout:6/686: creat d1/d2/db/d3c/d43/d96/dc9/fde x:0 0 0 2024-04-25T00:18:36.332 INFO:tasks.workunit.client.0.smithi107.stdout:6/687: write d1/d2/db/fd [6952036,16751] 0 2024-04-25T00:18:36.332 INFO:tasks.workunit.client.0.smithi107.stdout:6/688: truncate d1/d2/d2b/f86 706530 0 2024-04-25T00:18:36.332 INFO:tasks.workunit.client.0.smithi107.stdout:9/857: creat d0/d8a/d93/d56/d6c/d80/d113/f125 x:0 0 0 2024-04-25T00:18:36.334 INFO:tasks.workunit.client.0.smithi107.stdout:9/858: truncate d0/d8a/d93/d34/f84 3106497 0 2024-04-25T00:18:36.335 INFO:tasks.workunit.client.0.smithi107.stdout:9/859: creat d0/d8a/d93/d56/d76/d111/f126 x:0 0 0 2024-04-25T00:18:36.336 INFO:tasks.workunit.client.0.smithi107.stdout:9/860: readlink d0/d8a/d93/d34/d36/l66 0 2024-04-25T00:18:36.336 INFO:tasks.workunit.client.0.smithi107.stdout:9/861: readlink d0/d8a/d93/d34/lec 0 2024-04-25T00:18:36.337 INFO:tasks.workunit.client.0.smithi107.stdout:6/689: unlink d1/d2/db/d32/fb7 0 2024-04-25T00:18:36.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:35 smithi195 ceph-mon[32833]: pgmap v67: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 118 MiB/s rd, 143 MiB/s wr, 223 op/s 2024-04-25T00:18:36.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:35 smithi195 ceph-mon[32833]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:18:36.344 INFO:tasks.workunit.client.0.smithi107.stdout:2/846: getdents d6/dad 0 2024-04-25T00:18:36.347 INFO:tasks.workunit.client.0.smithi107.stdout:2/847: rename d6/f17 to d6/d18/d40/d42/d2b/d30/d45/dec/f121 0 2024-04-25T00:18:36.349 INFO:tasks.workunit.client.0.smithi107.stdout:2/848: mkdir d6/d20/d122 0 2024-04-25T00:18:36.349 INFO:tasks.workunit.client.0.smithi107.stdout:2/849: getdents d6/d18/d40/d42/d2b/d11d 0 2024-04-25T00:18:36.349 INFO:tasks.workunit.client.0.smithi107.stdout:2/850: dread - d6/dad/f98 zero size 2024-04-25T00:18:36.349 INFO:tasks.workunit.client.0.smithi107.stdout:2/851: readlink d6/dad/d67/da8/lee 0 2024-04-25T00:18:36.349 INFO:tasks.workunit.client.0.smithi107.stdout:7/792: dread d0/f1d [0,4194304] 0 2024-04-25T00:18:36.349 INFO:tasks.workunit.client.0.smithi107.stdout:7/793: chown d0/d8/db/d18/d19/c35 215 1 2024-04-25T00:18:36.351 INFO:tasks.workunit.client.0.smithi107.stdout:2/852: symlink d6/d29/d4e/dc0/dd1/l123 0 2024-04-25T00:18:36.352 INFO:tasks.workunit.client.0.smithi107.stdout:2/853: mknod d6/dad/db0/c124 0 2024-04-25T00:18:36.355 INFO:tasks.workunit.client.0.smithi107.stdout:6/690: write d1/d2/db/d48/fd2 [3540586,28455] 0 2024-04-25T00:18:36.356 INFO:tasks.workunit.client.0.smithi107.stdout:7/794: write d0/d8/db/d18/d19/d67/f72 [728536,110717] 0 2024-04-25T00:18:36.357 INFO:tasks.workunit.client.0.smithi107.stdout:9/862: dwrite d0/f94 [0,4194304] 0 2024-04-25T00:18:36.357 INFO:tasks.workunit.client.0.smithi107.stdout:9/863: chown d0/d8a/d93 2838939 1 2024-04-25T00:18:36.358 INFO:tasks.workunit.client.0.smithi107.stdout:1/787: dwrite d2/da4/fcb [0,4194304] 0 2024-04-25T00:18:36.358 INFO:tasks.workunit.client.0.smithi107.stdout:1/788: read d2/dd/f8b [478427,76301] 0 2024-04-25T00:18:36.358 INFO:tasks.workunit.client.0.smithi107.stdout:6/691: creat d1/d2/db/d3c/d43/d96/dc9/fdf x:0 0 0 2024-04-25T00:18:36.358 INFO:tasks.workunit.client.0.smithi107.stdout:4/844: dwrite d3/d24/d5c/dab/f8d [0,4194304] 0 2024-04-25T00:18:36.359 INFO:tasks.workunit.client.0.smithi107.stdout:7/795: dread d0/d8/db/d18/d19/d39/f3e [0,4194304] 0 2024-04-25T00:18:36.359 INFO:tasks.workunit.client.0.smithi107.stdout:7/796: write d0/d8/f78 [1066360,129585] 0 2024-04-25T00:18:36.361 INFO:tasks.workunit.client.0.smithi107.stdout:4/845: rmdir d3/d5/d32/d9e/da2/daf 39 2024-04-25T00:18:36.361 INFO:tasks.workunit.client.0.smithi107.stdout:4/846: readlink d3/d5/l3c 0 2024-04-25T00:18:36.361 INFO:tasks.workunit.client.0.smithi107.stdout:6/692: dread d1/d2/db/d32/d36/d3a/f9a [0,4194304] 0 2024-04-25T00:18:36.361 INFO:tasks.workunit.client.0.smithi107.stdout:9/864: creat d0/d8a/d93/de/d5c/de0/f127 x:0 0 0 2024-04-25T00:18:36.365 INFO:tasks.workunit.client.0.smithi107.stdout:4/847: creat d3/d24/d25/d71/d7f/f11b x:0 0 0 2024-04-25T00:18:36.366 INFO:tasks.workunit.client.0.smithi107.stdout:7/797: truncate d0/d8/db/d18/d19/d47/d4d/d8a/fca 1848388 0 2024-04-25T00:18:36.367 INFO:tasks.workunit.client.0.smithi107.stdout:9/865: creat d0/d8a/d93/de/d5c/de0/f128 x:0 0 0 2024-04-25T00:18:36.368 INFO:tasks.workunit.client.0.smithi107.stdout:7/798: symlink d0/d8/db/d18/d19/d47/d4d/d8a/l10a 0 2024-04-25T00:18:36.368 INFO:tasks.workunit.client.0.smithi107.stdout:7/799: chown d0/d8/db/d18/d19/d47/de0/d82/ffd 431292767 1 2024-04-25T00:18:36.369 INFO:tasks.workunit.client.0.smithi107.stdout:9/866: symlink d0/d8a/d93/d34/d36/d77/d110/l129 0 2024-04-25T00:18:36.372 INFO:tasks.workunit.client.0.smithi107.stdout:3/856: sync 2024-04-25T00:18:36.372 INFO:tasks.workunit.client.0.smithi107.stdout:0/743: sync 2024-04-25T00:18:36.372 INFO:tasks.workunit.client.0.smithi107.stdout:0/744: write dc/dd/d16/d48/f6a [451757,97778] 0 2024-04-25T00:18:36.379 INFO:tasks.workunit.client.0.smithi107.stdout:1/789: dread d2/da/f7b [0,4194304] 0 2024-04-25T00:18:36.379 INFO:tasks.workunit.client.0.smithi107.stdout:9/867: creat d0/d8a/d93/d1f/f12a x:0 0 0 2024-04-25T00:18:36.382 INFO:tasks.workunit.client.0.smithi107.stdout:3/857: creat d9/d15/da1/f119 x:0 0 0 2024-04-25T00:18:36.382 INFO:tasks.workunit.client.0.smithi107.stdout:3/858: fdatasync d9/d88/d3a/fb1 0 2024-04-25T00:18:36.385 INFO:tasks.workunit.client.0.smithi107.stdout:0/745: mkdir dc/d24/de7 0 2024-04-25T00:18:36.385 INFO:tasks.workunit.client.0.smithi107.stdout:6/693: dwrite d1/d2/d3/d28/d79/d6b/f6f [0,4194304] 0 2024-04-25T00:18:36.387 INFO:tasks.workunit.client.0.smithi107.stdout:6/694: mkdir d1/d2/d3/d28/d79/de0 0 2024-04-25T00:18:36.387 INFO:tasks.workunit.client.0.smithi107.stdout:1/790: dread f1 [0,4194304] 0 2024-04-25T00:18:36.388 INFO:tasks.workunit.client.0.smithi107.stdout:9/868: getdents d0/d8a/d93/d6f 0 2024-04-25T00:18:36.389 INFO:tasks.workunit.client.0.smithi107.stdout:6/695: creat d1/d2/d3/d28/d79/d6b/fe1 x:0 0 0 2024-04-25T00:18:36.390 INFO:tasks.workunit.client.0.smithi107.stdout:3/859: creat d9/d22/d1b/dbd/f11a x:0 0 0 2024-04-25T00:18:36.390 INFO:tasks.workunit.client.0.smithi107.stdout:7/800: dwrite d0/d80/da2/fad [4194304,4194304] 0 2024-04-25T00:18:36.391 INFO:tasks.workunit.client.0.smithi107.stdout:0/746: creat dc/da6/fe8 x:0 0 0 2024-04-25T00:18:36.392 INFO:tasks.workunit.client.0.smithi107.stdout:6/696: write d1/d2/d2b/d3e/d5b/d7a/f95 [1034610,9911] 0 2024-04-25T00:18:36.392 INFO:tasks.workunit.client.0.smithi107.stdout:7/801: mknod d0/d8/d4e/d46/c10b 0 2024-04-25T00:18:36.393 INFO:tasks.workunit.client.0.smithi107.stdout:1/791: creat d2/dd/d82/dc1/f100 x:0 0 0 2024-04-25T00:18:36.394 INFO:tasks.workunit.client.0.smithi107.stdout:9/869: creat d0/d8a/d93/d34/d36/f12b x:0 0 0 2024-04-25T00:18:36.394 INFO:tasks.workunit.client.0.smithi107.stdout:7/802: unlink d0/d8/db/d18/d19/d39/l76 0 2024-04-25T00:18:36.397 INFO:tasks.workunit.client.0.smithi107.stdout:7/803: getdents d0/dc1 0 2024-04-25T00:18:36.397 INFO:tasks.workunit.client.0.smithi107.stdout:1/792: creat d2/dd/d25/f101 x:0 0 0 2024-04-25T00:18:36.398 INFO:tasks.workunit.client.0.smithi107.stdout:1/793: creat d2/da/d2f/f102 x:0 0 0 2024-04-25T00:18:36.400 INFO:tasks.workunit.client.0.smithi107.stdout:9/870: dread d0/f8f [0,4194304] 0 2024-04-25T00:18:36.400 INFO:tasks.workunit.client.0.smithi107.stdout:7/804: creat d0/d8/f10c x:0 0 0 2024-04-25T00:18:36.401 INFO:tasks.workunit.client.0.smithi107.stdout:7/805: creat d0/d80/da2/f10d x:0 0 0 2024-04-25T00:18:36.402 INFO:tasks.workunit.client.0.smithi107.stdout:7/806: chown d0/d8/d2c/df4 17132 1 2024-04-25T00:18:36.402 INFO:tasks.workunit.client.0.smithi107.stdout:7/807: mkdir d0/d8/db/d10e 0 2024-04-25T00:18:36.403 INFO:tasks.workunit.client.0.smithi107.stdout:7/808: getdents d0/d8/db/d18/d19/d47 0 2024-04-25T00:18:36.403 INFO:tasks.workunit.client.0.smithi107.stdout:7/809: fdatasync d0/d8/db/f98 0 2024-04-25T00:18:36.404 INFO:tasks.workunit.client.0.smithi107.stdout:7/810: symlink d0/d8/d2c/d58/l10f 0 2024-04-25T00:18:36.421 INFO:tasks.workunit.client.0.smithi107.stdout:7/811: dread d0/d80/da2/fad [0,4194304] 0 2024-04-25T00:18:36.421 INFO:tasks.workunit.client.0.smithi107.stdout:7/812: chown d0/d8/d2c/d58/fb4 2098 1 2024-04-25T00:18:36.421 INFO:tasks.workunit.client.0.smithi107.stdout:7/813: write d0/d80/da2/f10d [506126,68918] 0 2024-04-25T00:18:36.421 INFO:tasks.workunit.client.0.smithi107.stdout:7/814: chown d0/d8/db/d18/d19/c63 1295 1 2024-04-25T00:18:36.421 INFO:tasks.workunit.client.0.smithi107.stdout:9/871: dwrite d0/d8a/d93/d56/d76/d111/f126 [0,4194304] 0 2024-04-25T00:18:36.421 INFO:tasks.workunit.client.0.smithi107.stdout:3/860: dwrite d9/d22/f8b [4194304,4194304] 0 2024-04-25T00:18:36.424 INFO:tasks.workunit.client.0.smithi107.stdout:9/872: unlink d0/d8a/d93/de/d5c/c61 0 2024-04-25T00:18:36.424 INFO:tasks.workunit.client.0.smithi107.stdout:7/815: creat d0/d8/db/d18/d19/d39/f110 x:0 0 0 2024-04-25T00:18:36.425 INFO:tasks.workunit.client.0.smithi107.stdout:4/848: rename d3/d5/d32/d53/dc0/c105 to d3/d24/c11c 0 2024-04-25T00:18:36.426 INFO:tasks.workunit.client.0.smithi107.stdout:3/861: unlink d9/d15/d39/d76/l8a 0 2024-04-25T00:18:36.430 INFO:tasks.workunit.client.0.smithi107.stdout:0/747: rename dc/dd/f14 to dc/d24/d38/d64/d78/de6/fe9 0 2024-04-25T00:18:36.432 INFO:tasks.workunit.client.0.smithi107.stdout:4/849: creat d3/d24/d5c/d7a/f11d x:0 0 0 2024-04-25T00:18:36.433 INFO:tasks.workunit.client.0.smithi107.stdout:4/850: truncate d3/d10/d7b/feb 624944 0 2024-04-25T00:18:36.433 INFO:tasks.workunit.client.0.smithi107.stdout:4/851: dread - d3/d24/d5c/d7a/f115 zero size 2024-04-25T00:18:36.434 INFO:tasks.workunit.client.0.smithi107.stdout:3/862: mknod d9/d54/d5c/d10f/c11b 0 2024-04-25T00:18:36.434 INFO:tasks.workunit.client.0.smithi107.stdout:4/852: symlink d3/d24/d25/d71/ddb/l11e 0 2024-04-25T00:18:36.436 INFO:tasks.workunit.client.0.smithi107.stdout:4/853: mkdir d3/d5/d11f 0 2024-04-25T00:18:36.436 INFO:tasks.workunit.client.0.smithi107.stdout:9/873: rmdir d0/d8a/d93/d56/d6c/d80/d9b/db2/dea 39 2024-04-25T00:18:36.436 INFO:tasks.workunit.client.0.smithi107.stdout:9/874: readlink d0/d8a/d93/d1f/l44 0 2024-04-25T00:18:36.436 INFO:tasks.workunit.client.0.smithi107.stdout:9/875: stat d0/d8a/d93/d56/d6c/daa/lb7 0 2024-04-25T00:18:36.438 INFO:tasks.workunit.client.0.smithi107.stdout:4/854: mkdir d3/d5/d32/d53/d7e/d120 0 2024-04-25T00:18:36.439 INFO:tasks.workunit.client.0.smithi107.stdout:1/794: rename d2/c29 to d2/d62/d66/d70/d95/c103 0 2024-04-25T00:18:36.441 INFO:tasks.workunit.client.0.smithi107.stdout:9/876: symlink d0/d8a/d93/de/d5c/de0/l12c 0 2024-04-25T00:18:36.441 INFO:tasks.workunit.client.0.smithi107.stdout:9/877: fsync d0/d8a/d93/f26 0 2024-04-25T00:18:36.442 INFO:tasks.workunit.client.0.smithi107.stdout:1/795: getdents d2/d2a/d5c/d89 0 2024-04-25T00:18:36.442 INFO:tasks.workunit.client.0.smithi107.stdout:1/796: chown d2/dd/d82/c8f 818 1 2024-04-25T00:18:36.442 INFO:tasks.workunit.client.0.smithi107.stdout:1/797: stat d2/dd/d25/d5e/f94 0 2024-04-25T00:18:36.442 INFO:tasks.workunit.client.0.smithi107.stdout:1/798: write d2/dd/d25/d5e/fd0 [392504,49804] 0 2024-04-25T00:18:36.442 INFO:tasks.workunit.client.0.smithi107.stdout:1/799: readlink d2/le2 0 2024-04-25T00:18:36.442 INFO:tasks.workunit.client.0.smithi107.stdout:7/816: rename d0/d8/db/d18/d19/d47/f62 to d0/d8/db/d18/d19/d47/da1/f111 0 2024-04-25T00:18:36.443 INFO:tasks.workunit.client.0.smithi107.stdout:9/878: mkdir d0/d8a/d93/de/d5c/d95/d106/d12d 0 2024-04-25T00:18:36.443 INFO:tasks.workunit.client.0.smithi107.stdout:7/817: link d0/d8/d4e/d46/d93/fac d0/d8/db/d18/d19/d39/dfe/f112 0 2024-04-25T00:18:36.444 INFO:tasks.workunit.client.0.smithi107.stdout:7/818: mknod d0/d8/db/d18/d19/d47/d4d/c113 0 2024-04-25T00:18:36.444 INFO:tasks.workunit.client.0.smithi107.stdout:7/819: stat d0/d8/db/d18/d19/d47/de0/cee 0 2024-04-25T00:18:36.444 INFO:tasks.workunit.client.0.smithi107.stdout:1/800: unlink d2/c16 0 2024-04-25T00:18:36.446 INFO:tasks.workunit.client.0.smithi107.stdout:1/801: symlink d2/d62/d66/de5/l104 0 2024-04-25T00:18:36.446 INFO:tasks.workunit.client.0.smithi107.stdout:1/802: write d2/d2a/d37/d88/fe0 [893117,72703] 0 2024-04-25T00:18:36.449 INFO:tasks.workunit.client.0.smithi107.stdout:4/855: dread d3/d24/d25/f28 [0,4194304] 0 2024-04-25T00:18:36.449 INFO:tasks.workunit.client.0.smithi107.stdout:4/856: dread - d3/d24/d25/d71/d7f/fc5 zero size 2024-04-25T00:18:36.450 INFO:tasks.workunit.client.0.smithi107.stdout:4/857: dread - d3/d10/d7b/da5/f104 zero size 2024-04-25T00:18:36.450 INFO:tasks.workunit.client.0.smithi107.stdout:0/748: dwrite dc/d24/d38/d64/f68 [0,4194304] 0 2024-04-25T00:18:36.451 INFO:tasks.workunit.client.0.smithi107.stdout:4/858: mkdir d3/d24/d61/d121 0 2024-04-25T00:18:36.455 INFO:tasks.workunit.client.0.smithi107.stdout:0/749: mknod dc/dd/d16/cea 0 2024-04-25T00:18:36.458 INFO:tasks.workunit.client.0.smithi107.stdout:4/859: read d3/d5/f4d [1687180,111071] 0 2024-04-25T00:18:36.458 INFO:tasks.workunit.client.0.smithi107.stdout:4/860: write d3/d24/f81 [1724255,75915] 0 2024-04-25T00:18:36.458 INFO:tasks.workunit.client.0.smithi107.stdout:4/861: chown d3/d24/d25/d71/ddb/l11e 57198 1 2024-04-25T00:18:36.460 INFO:tasks.workunit.client.0.smithi107.stdout:4/862: unlink d3/d5/d32/cfc 0 2024-04-25T00:18:36.463 INFO:tasks.workunit.client.0.smithi107.stdout:4/863: creat d3/d24/d61/f122 x:0 0 0 2024-04-25T00:18:36.463 INFO:tasks.workunit.client.0.smithi107.stdout:4/864: mknod d3/d24/df2/c123 0 2024-04-25T00:18:36.464 INFO:tasks.workunit.client.0.smithi107.stdout:4/865: symlink d3/d5/d32/d9e/da2/l124 0 2024-04-25T00:18:36.466 INFO:tasks.workunit.client.0.smithi107.stdout:0/750: dread dc/d24/fe2 [0,4194304] 0 2024-04-25T00:18:36.470 INFO:tasks.workunit.client.0.smithi107.stdout:7/820: dwrite d0/d8/d4e/d46/d93/fb0 [0,4194304] 0 2024-04-25T00:18:36.471 INFO:tasks.workunit.client.0.smithi107.stdout:1/803: dwrite d2/da/d2f/fad [0,4194304] 0 2024-04-25T00:18:36.471 INFO:tasks.workunit.client.0.smithi107.stdout:1/804: dread - d2/d2a/d5c/ff4 zero size 2024-04-25T00:18:36.474 INFO:tasks.workunit.client.0.smithi107.stdout:1/805: mkdir d2/d2a/d37/de8/d105 0 2024-04-25T00:18:36.474 INFO:tasks.workunit.client.0.smithi107.stdout:1/806: readlink d2/dd/d25/d5e/l4c 0 2024-04-25T00:18:36.486 INFO:tasks.workunit.client.0.smithi107.stdout:0/751: dwrite dc/dd/d5d/d3f/f7a [0,4194304] 0 2024-04-25T00:18:36.487 INFO:tasks.workunit.client.0.smithi107.stdout:1/807: write d2/da/f53 [2718303,78247] 0 2024-04-25T00:18:36.488 INFO:tasks.workunit.client.0.smithi107.stdout:0/752: symlink dc/dd/d16/d33/d35/d5f/leb 0 2024-04-25T00:18:36.488 INFO:tasks.workunit.client.0.smithi107.stdout:1/808: symlink d2/d62/l106 0 2024-04-25T00:18:36.489 INFO:tasks.workunit.client.0.smithi107.stdout:0/753: truncate dc/dd/d5d/d3f/d59/f81 4093901 0 2024-04-25T00:18:36.490 INFO:tasks.workunit.client.0.smithi107.stdout:1/809: creat d2/dfe/f107 x:0 0 0 2024-04-25T00:18:36.491 INFO:tasks.workunit.client.0.smithi107.stdout:0/754: creat dc/d24/d25/fec x:0 0 0 2024-04-25T00:18:36.491 INFO:tasks.workunit.client.0.smithi107.stdout:0/755: truncate dc/dd/d5d/d3f/dc7/fd4 4425675 0 2024-04-25T00:18:36.491 INFO:tasks.workunit.client.0.smithi107.stdout:7/821: dwrite d0/d8/db/d18/d19/d47/d4d/d8a/fa6 [0,4194304] 0 2024-04-25T00:18:36.492 INFO:tasks.workunit.client.0.smithi107.stdout:1/810: mkdir d2/d108 0 2024-04-25T00:18:36.492 INFO:tasks.workunit.client.0.smithi107.stdout:0/756: getdents dc/dd/d16/d48/d83/d9f 0 2024-04-25T00:18:36.494 INFO:tasks.workunit.client.0.smithi107.stdout:1/811: getdents d2/dd/d25/da9 0 2024-04-25T00:18:36.495 INFO:tasks.workunit.client.0.smithi107.stdout:1/812: dread - d2/d62/d66/fbe zero size 2024-04-25T00:18:36.495 INFO:tasks.workunit.client.0.smithi107.stdout:1/813: truncate d2/dd/d82/dc1/f100 375858 0 2024-04-25T00:18:36.495 INFO:tasks.workunit.client.0.smithi107.stdout:7/822: creat d0/d8/db/d18/d19/d47/d4d/f114 x:0 0 0 2024-04-25T00:18:36.499 INFO:tasks.workunit.client.0.smithi107.stdout:0/757: rename dc/dd/d5d/d5c/cd9 to dc/d24/d38/d64/d7b/ced 0 2024-04-25T00:18:36.499 INFO:tasks.workunit.client.0.smithi107.stdout:1/814: mknod d2/d62/d66/de5/c109 0 2024-04-25T00:18:36.499 INFO:tasks.workunit.client.0.smithi107.stdout:1/815: write d2/dd/f6f [2951325,32735] 0 2024-04-25T00:18:36.499 INFO:tasks.workunit.client.0.smithi107.stdout:1/816: chown d2/d62/d66/d70/dcc/ld5 9440485 1 2024-04-25T00:18:36.499 INFO:tasks.workunit.client.0.smithi107.stdout:1/817: getdents d2/d2a/d5c/d89 0 2024-04-25T00:18:36.500 INFO:tasks.workunit.client.0.smithi107.stdout:0/758: symlink dc/dd/d5d/d5c/d6e/lee 0 2024-04-25T00:18:36.503 INFO:tasks.workunit.client.0.smithi107.stdout:0/759: write dc/dd/d5d/f70 [249522,125349] 0 2024-04-25T00:18:36.506 INFO:tasks.workunit.client.0.smithi107.stdout:0/760: dread dc/da6/fd6 [0,4194304] 0 2024-04-25T00:18:36.507 INFO:tasks.workunit.client.0.smithi107.stdout:0/761: symlink dc/d24/de7/lef 0 2024-04-25T00:18:36.507 INFO:tasks.workunit.client.0.smithi107.stdout:0/762: read - dc/dd/d5d/d3f/da5/fde zero size 2024-04-25T00:18:36.508 INFO:tasks.workunit.client.0.smithi107.stdout:0/763: creat dc/d24/d38/ff0 x:0 0 0 2024-04-25T00:18:36.509 INFO:tasks.workunit.client.0.smithi107.stdout:0/764: rmdir dc/d24/de7 39 2024-04-25T00:18:36.510 INFO:tasks.workunit.client.0.smithi107.stdout:0/765: symlink dc/dd/d16/d48/ddb/lf1 0 2024-04-25T00:18:36.510 INFO:tasks.workunit.client.0.smithi107.stdout:0/766: truncate dc/dd/d16/d33/f3d 2520849 0 2024-04-25T00:18:36.512 INFO:tasks.workunit.client.0.smithi107.stdout:7/823: dwrite d0/d80/da8/fb9 [0,4194304] 0 2024-04-25T00:18:36.512 INFO:tasks.workunit.client.0.smithi107.stdout:7/824: write d0/d8/db/d18/d19/d39/f96 [481533,93702] 0 2024-04-25T00:18:36.514 INFO:tasks.workunit.client.0.smithi107.stdout:7/825: mkdir d0/d8/db/d18/d19/d47/de0/d115 0 2024-04-25T00:18:36.514 INFO:tasks.workunit.client.0.smithi107.stdout:7/826: chown d0/d8/db/d18/d19/d47/c33 10 1 2024-04-25T00:18:36.514 INFO:tasks.workunit.client.0.smithi107.stdout:7/827: readlink d0/d8/d2c/ld1 0 2024-04-25T00:18:36.515 INFO:tasks.workunit.client.0.smithi107.stdout:0/767: dread dc/d24/d25/f58 [0,4194304] 0 2024-04-25T00:18:36.519 INFO:tasks.workunit.client.0.smithi107.stdout:1/818: dwrite d2/d2a/d37/f56 [4194304,4194304] 0 2024-04-25T00:18:36.519 INFO:tasks.workunit.client.0.smithi107.stdout:0/768: write dc/d24/d25/f58 [410954,65353] 0 2024-04-25T00:18:36.519 INFO:tasks.workunit.client.0.smithi107.stdout:0/769: fsync dc/d24/f77 0 2024-04-25T00:18:36.522 INFO:tasks.workunit.client.0.smithi107.stdout:0/770: creat dc/ff2 x:0 0 0 2024-04-25T00:18:36.525 INFO:tasks.workunit.client.0.smithi107.stdout:0/771: rename dc/dd/d5d/d3f/d59/fa0 to dc/d24/de7/ff3 0 2024-04-25T00:18:36.533 INFO:tasks.workunit.client.0.smithi107.stdout:1/819: write d2/da/f7b [600392,86108] 0 2024-04-25T00:18:36.534 INFO:tasks.workunit.client.0.smithi107.stdout:1/820: mkdir d2/dd/dec/d10a 0 2024-04-25T00:18:36.535 INFO:tasks.workunit.client.0.smithi107.stdout:1/821: creat d2/da/f10b x:0 0 0 2024-04-25T00:18:36.535 INFO:tasks.workunit.client.0.smithi107.stdout:7/828: dwrite d0/d8/f1e [0,4194304] 0 2024-04-25T00:18:36.536 INFO:tasks.workunit.client.0.smithi107.stdout:1/822: truncate d2/d2a/d37/d88/ff5 3288888 0 2024-04-25T00:18:36.536 INFO:tasks.workunit.client.0.smithi107.stdout:7/829: link d0/d8/db/ff2 d0/d8/db/f116 0 2024-04-25T00:18:36.537 INFO:tasks.workunit.client.0.smithi107.stdout:7/830: write d0/d8/db/d18/d19/d47/d4d/d8a/fa6 [4484243,106881] 0 2024-04-25T00:18:36.537 INFO:tasks.workunit.client.0.smithi107.stdout:7/831: read d0/d8/db/d18/d19/d47/da1/f111 [3409483,54773] 0 2024-04-25T00:18:36.538 INFO:tasks.workunit.client.0.smithi107.stdout:7/832: rename d0/d8/d2c to d0/d8/db/d18/d19/d47/de0/d115/d117 0 2024-04-25T00:18:36.539 INFO:tasks.workunit.client.0.smithi107.stdout:7/833: creat d0/d8/db/d18/d19/d47/f118 x:0 0 0 2024-04-25T00:18:36.539 INFO:tasks.workunit.client.0.smithi107.stdout:7/834: write d0/d8/db/d18/d19/d39/f110 [975113,22290] 0 2024-04-25T00:18:36.541 INFO:tasks.workunit.client.0.smithi107.stdout:0/772: write dc/d24/fbf [2309643,50034] 0 2024-04-25T00:18:36.541 INFO:tasks.workunit.client.0.smithi107.stdout:0/773: write dc/dd/fc9 [727836,85316] 0 2024-04-25T00:18:36.545 INFO:tasks.workunit.client.0.smithi107.stdout:0/774: getdents dc/d24/d25/ddd 0 2024-04-25T00:18:36.558 INFO:tasks.workunit.client.0.smithi107.stdout:1/823: dwrite d2/d2a/d37/fe6 [0,4194304] 0 2024-04-25T00:18:36.559 INFO:tasks.workunit.client.0.smithi107.stdout:1/824: creat d2/d108/f10c x:0 0 0 2024-04-25T00:18:36.563 INFO:tasks.workunit.client.0.smithi107.stdout:7/835: dwrite d0/f26 [0,4194304] 0 2024-04-25T00:18:36.563 INFO:tasks.workunit.client.0.smithi107.stdout:7/836: write d0/d8/db/d18/d19/d47/de0/dc2/dd0/fe9 [547023,35377] 0 2024-04-25T00:18:36.578 INFO:tasks.workunit.client.0.smithi107.stdout:7/837: dwrite d0/d8/d4e/d46/f6b [0,4194304] 0 2024-04-25T00:18:36.593 INFO:tasks.workunit.client.0.smithi107.stdout:7/838: dwrite d0/d8/db/d18/d19/d47/d4d/d8a/fa6 [0,4194304] 0 2024-04-25T00:18:36.593 INFO:tasks.workunit.client.0.smithi107.stdout:7/839: readlink d0/d8/db/d18/d19/d47/d4d/l106 0 2024-04-25T00:18:36.593 INFO:tasks.workunit.client.0.smithi107.stdout:7/840: fsync d0/d8/db/d18/d19/d47/d4d/d8a/fb8 0 2024-04-25T00:18:36.594 INFO:tasks.workunit.client.0.smithi107.stdout:7/841: truncate d0/d8/d4e/f90 578024 0 2024-04-25T00:18:36.595 INFO:tasks.workunit.client.0.smithi107.stdout:7/842: chown d0/d8/d4e/d46/d93/dd8/lf5 505000220 1 2024-04-25T00:18:36.595 INFO:tasks.workunit.client.0.smithi107.stdout:7/843: write d0/d8/db/d18/d19/f107 [223907,27900] 0 2024-04-25T00:18:36.595 INFO:tasks.workunit.client.0.smithi107.stdout:7/844: chown d0/d8/d4e/d46/f6b 1243760 1 2024-04-25T00:18:36.596 INFO:tasks.workunit.client.0.smithi107.stdout:7/845: unlink d0/d8/db/d18/d19/d47/de0/d115/d117/d58/f5f 0 2024-04-25T00:18:36.597 INFO:tasks.workunit.client.0.smithi107.stdout:7/846: rename d0/d8/db/d18/d19/d47/de0/d115/d117/d58/lc4 to d0/d80/l119 0 2024-04-25T00:18:36.598 INFO:tasks.workunit.client.0.smithi107.stdout:7/847: rmdir d0/d8/db/d18/d19/d47/d4d/dbc 39 2024-04-25T00:18:36.598 INFO:tasks.workunit.client.0.smithi107.stdout:7/848: chown d0/d8/d4e/d46/d93/dd8/lf5 5646 1 2024-04-25T00:18:36.599 INFO:tasks.workunit.client.0.smithi107.stdout:7/849: mkdir d0/d8/db/d18/d19/d47/de0/d115/d117/d58/db5/d11a 0 2024-04-25T00:18:36.616 INFO:tasks.workunit.client.0.smithi107.stdout:7/850: dwrite d0/d8/db/d18/d19/d47/de0/d115/d117/df4/ff7 [0,4194304] 0 2024-04-25T00:18:36.616 INFO:tasks.workunit.client.0.smithi107.stdout:7/851: write d0/d8/db/d18/d19/d47/da1/f111 [653641,127683] 0 2024-04-25T00:18:36.620 INFO:tasks.workunit.client.0.smithi107.stdout:7/852: creat d0/d8/f11b x:0 0 0 2024-04-25T00:18:36.620 INFO:tasks.workunit.client.0.smithi107.stdout:7/853: chown d0/d8/db/ca7 206456171 1 2024-04-25T00:18:36.622 INFO:tasks.workunit.client.0.smithi107.stdout:7/854: rename d0/d8/db/d18/d19/fba to d0/d8/db/d18/d19/d47/da1/dd4/f11c 0 2024-04-25T00:18:36.622 INFO:tasks.workunit.client.0.smithi107.stdout:7/855: write d0/d8/db/d18/d19/d47/d4d/dbc/f85 [4570813,60315] 0 2024-04-25T00:18:36.628 INFO:tasks.workunit.client.0.smithi107.stdout:7/856: read d0/d8/d4e/d46/f6b [2418641,122793] 0 2024-04-25T00:18:36.630 INFO:tasks.workunit.client.0.smithi107.stdout:7/857: mkdir d0/d8/db/d18/d19/d47/da1/d11d 0 2024-04-25T00:18:36.630 INFO:tasks.workunit.client.0.smithi107.stdout:7/858: chown d0/d8/db/d18/d19/d47/de0/dc2/dd0 852 1 2024-04-25T00:18:36.630 INFO:tasks.workunit.client.0.smithi107.stdout:7/859: truncate d0/d8/db/d18/d19/d47/de0/d115/d117/d58/f79 935593 0 2024-04-25T00:18:36.630 INFO:tasks.workunit.client.0.smithi107.stdout:7/860: write d0/d8/db/d18/d19/d47/de0/d115/d117/d58/f5e [4589295,125587] 0 2024-04-25T00:18:36.631 INFO:tasks.workunit.client.0.smithi107.stdout:7/861: link d0/d8/d4e/d46/d93/dd8/lf5 d0/d8/db/d18/d19/d47/de0/d115/d117/d58/l11e 0 2024-04-25T00:18:36.631 INFO:tasks.workunit.client.0.smithi107.stdout:7/862: truncate d0/d8/db/d18/d19/d47/de0/fc9 1032650 0 2024-04-25T00:18:36.632 INFO:tasks.workunit.client.0.smithi107.stdout:7/863: mkdir d0/d8/d4e/d11f 0 2024-04-25T00:18:36.633 INFO:tasks.workunit.client.0.smithi107.stdout:7/864: creat d0/d8/db/d18/d19/d47/de0/d115/d117/d58/db5/f120 x:0 0 0 2024-04-25T00:18:36.634 INFO:tasks.workunit.client.0.smithi107.stdout:7/865: mknod d0/d8/d4e/d46/d93/dd8/c121 0 2024-04-25T00:18:36.635 INFO:tasks.workunit.client.0.smithi107.stdout:7/866: rmdir d0/d8/db/d18/d19 39 2024-04-25T00:18:36.635 INFO:tasks.workunit.client.0.smithi107.stdout:7/867: chown d0/d8/db/d18/d19/d39 14149649 1 2024-04-25T00:18:36.641 INFO:tasks.workunit.client.0.smithi107.stdout:7/868: link d0/d8/db/d18/d19/d47/da1/dd4/f11c d0/d80/da2/f122 0 2024-04-25T00:18:36.660 INFO:tasks.workunit.client.0.smithi107.stdout:7/869: dwrite d0/d8/f11b [0,4194304] 0 2024-04-25T00:18:36.660 INFO:tasks.workunit.client.0.smithi107.stdout:7/870: dread - d0/d8/f10c zero size 2024-04-25T00:18:36.662 INFO:tasks.workunit.client.0.smithi107.stdout:7/871: truncate d0/d8/db/d18/d19/d39/f6c 5606393 0 2024-04-25T00:18:36.663 INFO:tasks.workunit.client.0.smithi107.stdout:7/872: mkdir d0/d80/da8/d123 0 2024-04-25T00:18:36.681 INFO:tasks.workunit.client.0.smithi107.stdout:7/873: dwrite d0/dc1/fd3 [0,4194304] 0 2024-04-25T00:18:36.683 INFO:tasks.workunit.client.0.smithi107.stdout:7/874: rmdir d0/d8/db/d18/d19/d47/da1/dd4 39 2024-04-25T00:18:36.683 INFO:tasks.workunit.client.0.smithi107.stdout:7/875: chown d0/d8/db/d18/d19/d47/de0/dc2/dd0 65173611 1 2024-04-25T00:18:36.684 INFO:tasks.workunit.client.0.smithi107.stdout:7/876: creat d0/d8/db/d18/d19/d47/f124 x:0 0 0 2024-04-25T00:18:36.698 INFO:tasks.workunit.client.0.smithi107.stdout:7/877: dwrite d0/d8/db/d18/d19/d47/da1/dd4/ff0 [0,4194304] 0 2024-04-25T00:18:36.701 INFO:tasks.workunit.client.0.smithi107.stdout:7/878: rename d0/d8/db/ca7 to d0/d8/d4e/d11f/c125 0 2024-04-25T00:18:36.703 INFO:tasks.workunit.client.0.smithi107.stdout:7/879: symlink d0/d8/db/d18/d19/d47/de0/d115/d117/de5/l126 0 2024-04-25T00:18:36.704 INFO:tasks.workunit.client.0.smithi107.stdout:7/880: link d0/d8/f11b d0/d8/d4e/d46/d93/dd8/f127 0 2024-04-25T00:18:36.719 INFO:tasks.workunit.client.0.smithi107.stdout:7/881: dwrite d0/d8/db/d18/f8d [0,4194304] 0 2024-04-25T00:18:36.719 INFO:tasks.workunit.client.0.smithi107.stdout:7/882: getdents d0/dc1 0 2024-04-25T00:18:36.733 INFO:tasks.workunit.client.0.smithi107.stdout:7/883: dwrite d0/d8/d4e/d46/f6b [0,4194304] 0 2024-04-25T00:18:36.735 INFO:tasks.workunit.client.0.smithi107.stdout:7/884: mkdir d0/d8/db/d18/d19/d47/da1/dd4/d128 0 2024-04-25T00:18:36.735 INFO:tasks.workunit.client.0.smithi107.stdout:7/885: readlink d0/d8/db/d18/d19/d47/de0/d115/d117/d58/le8 0 2024-04-25T00:18:36.737 INFO:tasks.workunit.client.0.smithi107.stdout:2/854: sync 2024-04-25T00:18:36.737 INFO:tasks.workunit.client.0.smithi107.stdout:5/788: sync 2024-04-25T00:18:36.737 INFO:tasks.workunit.client.0.smithi107.stdout:8/888: sync 2024-04-25T00:18:36.737 INFO:tasks.workunit.client.0.smithi107.stdout:2/855: fdatasync d6/d18/d40/d42/f47 0 2024-04-25T00:18:36.737 INFO:tasks.workunit.client.0.smithi107.stdout:2/856: fsync d6/d18/d40/d42/d69/d6e/d80/f95 0 2024-04-25T00:18:36.738 INFO:tasks.workunit.client.0.smithi107.stdout:7/886: dread d0/d8/db/d18/d19/d47/da1/fae [0,4194304] 0 2024-04-25T00:18:36.738 INFO:tasks.workunit.client.0.smithi107.stdout:7/887: read - d0/d8/f10c zero size 2024-04-25T00:18:36.739 INFO:tasks.workunit.client.0.smithi107.stdout:7/888: chown d0/d8/d4e/d46/d93/fbd 20788037 1 2024-04-25T00:18:36.739 INFO:tasks.workunit.client.0.smithi107.stdout:7/889: readlink d0/d8/db/d18/d19/d47/d4d/l106 0 2024-04-25T00:18:36.739 INFO:tasks.workunit.client.0.smithi107.stdout:2/857: mknod d6/d29/d4e/c125 0 2024-04-25T00:18:36.739 INFO:tasks.workunit.client.0.smithi107.stdout:8/889: mkdir d2/d18/d1a/d5e/d4f/da0/da1/de8/d119/d137 0 2024-04-25T00:18:36.740 INFO:tasks.workunit.client.0.smithi107.stdout:8/890: truncate d2/d18/d1a/d5e/d3e/db1/fd0 475826 0 2024-04-25T00:18:36.740 INFO:tasks.workunit.client.0.smithi107.stdout:8/891: read - d2/d18/d1a/d5e/d5c/d91/de7/f104 zero size 2024-04-25T00:18:36.741 INFO:tasks.workunit.client.0.smithi107.stdout:8/892: rmdir d2/d18/d1a 39 2024-04-25T00:18:36.741 INFO:tasks.workunit.client.0.smithi107.stdout:8/893: fsync d2/d18/d1a/d5e/fbe 0 2024-04-25T00:18:36.742 INFO:tasks.workunit.client.0.smithi107.stdout:8/894: creat d2/d18/d1a/d5e/d4f/f138 x:0 0 0 2024-04-25T00:18:36.743 INFO:tasks.workunit.client.0.smithi107.stdout:8/895: rmdir d2/d18/d1a/d5e/d5c/d91 39 2024-04-25T00:18:36.747 INFO:tasks.workunit.client.0.smithi107.stdout:2/858: dread d6/d18/d40/d42/d70/f76 [0,4194304] 0 2024-04-25T00:18:36.748 INFO:tasks.workunit.client.0.smithi107.stdout:2/859: mknod d6/d29/d4e/d8c/c126 0 2024-04-25T00:18:36.749 INFO:tasks.workunit.client.0.smithi107.stdout:7/890: dread d0/d8/db/d18/d19/d47/fd7 [0,4194304] 0 2024-04-25T00:18:36.751 INFO:tasks.workunit.client.0.smithi107.stdout:7/891: creat d0/d8/db/d18/d19/d47/d4d/f129 x:0 0 0 2024-04-25T00:18:36.751 INFO:tasks.workunit.client.0.smithi107.stdout:2/860: mkdir d6/d18/d40/d42/d2b/d30/d127 0 2024-04-25T00:18:36.752 INFO:tasks.workunit.client.0.smithi107.stdout:2/861: fdatasync d6/d18/d40/f8f 0 2024-04-25T00:18:36.752 INFO:tasks.workunit.client.0.smithi107.stdout:7/892: symlink d0/d8/db/d18/d19/d39/dfe/l12a 0 2024-04-25T00:18:36.752 INFO:tasks.workunit.client.0.smithi107.stdout:7/893: stat d0/d8/f11b 0 2024-04-25T00:18:36.754 INFO:tasks.workunit.client.0.smithi107.stdout:7/894: mknod d0/d8/db/d18/d19/d47/de0/c12b 0 2024-04-25T00:18:36.756 INFO:tasks.workunit.client.0.smithi107.stdout:7/895: link d0/cc3 d0/d8/db/c12c 0 2024-04-25T00:18:36.757 INFO:tasks.workunit.client.0.smithi107.stdout:7/896: truncate d0/d80/f88 3363011 0 2024-04-25T00:18:36.759 INFO:tasks.workunit.client.0.smithi107.stdout:8/896: dread d2/d18/d1a/d5e/f56 [0,4194304] 0 2024-04-25T00:18:36.759 INFO:tasks.workunit.client.0.smithi107.stdout:7/897: creat d0/d8/db/d18/d19/d47/de0/f12d x:0 0 0 2024-04-25T00:18:36.759 INFO:tasks.workunit.client.0.smithi107.stdout:7/898: fdatasync d0/d8/db/d18/d19/d47/de0/dc2/dd0/fe9 0 2024-04-25T00:18:36.760 INFO:tasks.workunit.client.0.smithi107.stdout:5/789: dwrite d4/d17/f69 [0,4194304] 0 2024-04-25T00:18:36.760 INFO:tasks.workunit.client.0.smithi107.stdout:7/899: symlink d0/d8/db/d18/d19/d47/de0/d115/d117/df4/l12e 0 2024-04-25T00:18:36.763 INFO:tasks.workunit.client.0.smithi107.stdout:2/862: write d6/d20/d22/f3a [1038117,97769] 0 2024-04-25T00:18:36.763 INFO:tasks.workunit.client.0.smithi107.stdout:2/863: chown d6/d20/d22/d56/d94/fdf 22361 1 2024-04-25T00:18:36.764 INFO:tasks.workunit.client.0.smithi107.stdout:5/790: rename d4/dc/ff8 to d4/d17/db3/f105 0 2024-04-25T00:18:36.764 INFO:tasks.workunit.client.0.smithi107.stdout:5/791: write d4/d17/db3/d84/d93/da4/fc7 [648823,8746] 0 2024-04-25T00:18:36.764 INFO:tasks.workunit.client.0.smithi107.stdout:5/792: chown d4/dc/d26/d29/d2e/le6 0 1 2024-04-25T00:18:36.764 INFO:tasks.workunit.client.0.smithi107.stdout:5/793: fsync d4/dc/d26/f30 0 2024-04-25T00:18:36.769 INFO:tasks.workunit.client.0.smithi107.stdout:8/897: write d2/fb [1498182,83040] 0 2024-04-25T00:18:36.771 INFO:tasks.workunit.client.0.smithi107.stdout:7/900: write d0/d8/d4e/d46/f6b [2269117,30934] 0 2024-04-25T00:18:36.773 INFO:tasks.workunit.client.0.smithi107.stdout:7/901: symlink d0/d8/db/d18/d19/d47/de0/dc2/l12f 0 2024-04-25T00:18:36.774 INFO:tasks.workunit.client.0.smithi107.stdout:7/902: mknod d0/dc1/c130 0 2024-04-25T00:18:36.774 INFO:tasks.workunit.client.0.smithi107.stdout:7/903: write d0/f1d [2097474,88641] 0 2024-04-25T00:18:36.776 INFO:tasks.workunit.client.0.smithi107.stdout:7/904: creat d0/d8/db/d10e/f131 x:0 0 0 2024-04-25T00:18:36.777 INFO:tasks.workunit.client.0.smithi107.stdout:7/905: mkdir d0/d8/d132 0 2024-04-25T00:18:36.780 INFO:tasks.workunit.client.0.smithi107.stdout:7/906: unlink d0/d8/db/d18/d19/d47/de0/d115/d117/lfc 0 2024-04-25T00:18:36.787 INFO:tasks.workunit.client.0.smithi107.stdout:2/864: dwrite d6/dad/d67/f49 [0,4194304] 0 2024-04-25T00:18:36.789 INFO:tasks.workunit.client.0.smithi107.stdout:2/865: getdents d6/d18/d40 0 2024-04-25T00:18:36.794 INFO:tasks.workunit.client.0.smithi107.stdout:7/907: read d0/d8/f1e [2287351,29114] 0 2024-04-25T00:18:36.794 INFO:tasks.workunit.client.0.smithi107.stdout:7/908: dread - d0/d8/db/d18/d19/d47/de0/f12d zero size 2024-04-25T00:18:36.794 INFO:tasks.workunit.client.0.smithi107.stdout:5/794: dwrite d4/dc/d26/f4b [8388608,4194304] 0 2024-04-25T00:18:36.795 INFO:tasks.workunit.client.0.smithi107.stdout:7/909: mkdir d0/d8/db/d18/d19/d47/d4d/dbc/d133 0 2024-04-25T00:18:36.795 INFO:tasks.workunit.client.0.smithi107.stdout:7/910: fdatasync d0/d80/f88 0 2024-04-25T00:18:36.795 INFO:tasks.workunit.client.0.smithi107.stdout:7/911: truncate d0/d8/db/d18/d19/d67/f72 1797873 0 2024-04-25T00:18:36.797 INFO:tasks.workunit.client.0.smithi107.stdout:7/912: rename d0/d8/db/d18/d19/d47/de0/dc2/dd0/lcb to d0/l134 0 2024-04-25T00:18:36.800 INFO:tasks.workunit.client.0.smithi107.stdout:2/866: dread d6/d29/d4e/fab [0,4194304] 0 2024-04-25T00:18:36.800 INFO:tasks.workunit.client.0.smithi107.stdout:8/898: dwrite d2/d18/d1a/d5e/d3e/f99 [0,4194304] 0 2024-04-25T00:18:36.801 INFO:tasks.workunit.client.0.smithi107.stdout:2/867: truncate d6/d18/d40/d42/f62 1545711 0 2024-04-25T00:18:36.803 INFO:tasks.workunit.client.0.smithi107.stdout:8/899: symlink d2/d18/d1a/d5e/d3e/d74/l139 0 2024-04-25T00:18:36.803 INFO:tasks.workunit.client.0.smithi107.stdout:8/900: fsync d2/d18/d1a/d128/f12c 0 2024-04-25T00:18:36.806 INFO:tasks.workunit.client.0.smithi107.stdout:2/868: rename d6/d29/d4e/d8c to d6/d20/d22/d56/d94/d10b/d128 0 2024-04-25T00:18:36.808 INFO:tasks.workunit.client.0.smithi107.stdout:8/901: creat d2/d18/d1a/d5e/d5c/d130/f13a x:0 0 0 2024-04-25T00:18:36.809 INFO:tasks.workunit.client.0.smithi107.stdout:5/795: dread d4/d17/db3/fbb [0,4194304] 0 2024-04-25T00:18:36.810 INFO:tasks.workunit.client.0.smithi107.stdout:6/697: sync 2024-04-25T00:18:36.810 INFO:tasks.workunit.client.0.smithi107.stdout:3/863: sync 2024-04-25T00:18:36.811 INFO:tasks.workunit.client.0.smithi107.stdout:9/879: sync 2024-04-25T00:18:36.811 INFO:tasks.workunit.client.0.smithi107.stdout:7/913: dread d0/d8/db/d18/d19/d47/f31 [4194304,4194304] 0 2024-04-25T00:18:36.811 INFO:tasks.workunit.client.0.smithi107.stdout:7/914: readlink d0/d8/d4e/d46/d93/l102 0 2024-04-25T00:18:36.811 INFO:tasks.workunit.client.0.smithi107.stdout:5/796: mkdir d4/d17/db3/d84/df0/d106 0 2024-04-25T00:18:36.812 INFO:tasks.workunit.client.0.smithi107.stdout:3/864: mkdir d9/d22/d11c 0 2024-04-25T00:18:36.812 INFO:tasks.workunit.client.0.smithi107.stdout:5/797: mkdir d4/d1c/d5c/dd4/d107 0 2024-04-25T00:18:36.813 INFO:tasks.workunit.client.0.smithi107.stdout:5/798: truncate d4/dc/d26/d42/d8b/fc6 701825 0 2024-04-25T00:18:36.813 INFO:tasks.workunit.client.0.smithi107.stdout:3/865: creat d9/d88/d3a/d91/dad/deb/f11d x:0 0 0 2024-04-25T00:18:36.814 INFO:tasks.workunit.client.0.smithi107.stdout:3/866: write d9/d88/d2c/f37 [3294107,120509] 0 2024-04-25T00:18:36.814 INFO:tasks.workunit.client.0.smithi107.stdout:7/915: dread d0/d80/da2/f10d [0,4194304] 0 2024-04-25T00:18:36.815 INFO:tasks.workunit.client.0.smithi107.stdout:5/799: rename d4/dc/d26/d29/d2e/f4c to d4/dc/d83/dfc/f108 0 2024-04-25T00:18:36.815 INFO:tasks.workunit.client.0.smithi107.stdout:7/916: truncate d0/d8/db/d18/d19/d47/d4d/d8a/ffb 314932 0 2024-04-25T00:18:36.815 INFO:tasks.workunit.client.0.smithi107.stdout:7/917: chown d0/d8/db/d18/d19/d47/de0/d82/ldf 2097 1 2024-04-25T00:18:36.815 INFO:tasks.workunit.client.0.smithi107.stdout:7/918: read - d0/d8/d4e/fce zero size 2024-04-25T00:18:36.815 INFO:tasks.workunit.client.0.smithi107.stdout:7/919: readlink d0/d80/l119 0 2024-04-25T00:18:36.815 INFO:tasks.workunit.client.0.smithi107.stdout:5/800: write d4/dc/d26/d29/d2e/d55/fcb [669401,108990] 0 2024-04-25T00:18:36.817 INFO:tasks.workunit.client.0.smithi107.stdout:3/867: mkdir d9/d54/d5c/d10d/d11e 0 2024-04-25T00:18:36.818 INFO:tasks.workunit.client.0.smithi107.stdout:8/902: dread d2/ff [0,4194304] 0 2024-04-25T00:18:36.818 INFO:tasks.workunit.client.0.smithi107.stdout:5/801: creat d4/dc/d83/df1/f109 x:0 0 0 2024-04-25T00:18:36.818 INFO:tasks.workunit.client.0.smithi107.stdout:5/802: chown d4/d7c/d88/d9d/de1 27 1 2024-04-25T00:18:36.818 INFO:tasks.workunit.client.0.smithi107.stdout:5/803: truncate d4/d17/db3/d84/d93/dc4/fe2 335066 0 2024-04-25T00:18:36.819 INFO:tasks.workunit.client.0.smithi107.stdout:5/804: mknod d4/dc/d26/d42/d8b/c10a 0 2024-04-25T00:18:36.819 INFO:tasks.workunit.client.0.smithi107.stdout:5/805: readlink d4/dc/d26/d29/d51/laf 0 2024-04-25T00:18:36.823 INFO:tasks.workunit.client.0.smithi107.stdout:2/869: write d6/d18/d40/d42/f3b [3923133,2453] 0 2024-04-25T00:18:36.833 INFO:tasks.workunit.client.0.smithi107.stdout:3/868: link d9/d54/d94/f103 d9/d88/daa/f11f 0 2024-04-25T00:18:36.833 INFO:tasks.workunit.client.0.smithi107.stdout:3/869: read d9/d15/da1/d4b/dd0/ff8 [2499794,59086] 0 2024-04-25T00:18:36.833 INFO:tasks.workunit.client.0.smithi107.stdout:3/870: fdatasync d9/d15/da1/d4b/dd0/dd8/fe7 0 2024-04-25T00:18:36.833 INFO:tasks.workunit.client.0.smithi107.stdout:5/806: link d4/d1c/d5c/laa d4/d7c/d88/d9d/de1/l10b 0 2024-04-25T00:18:36.834 INFO:tasks.workunit.client.0.smithi107.stdout:3/871: mknod d9/d88/d3a/d91/dad/c120 0 2024-04-25T00:18:36.834 INFO:tasks.workunit.client.0.smithi107.stdout:3/872: write d9/d54/d5c/d68/f40 [219213,7710] 0 2024-04-25T00:18:36.837 INFO:tasks.workunit.client.0.smithi107.stdout:3/873: unlink d9/d15/da1/f64 0 2024-04-25T00:18:36.838 INFO:tasks.workunit.client.0.smithi107.stdout:3/874: mknod d9/d88/d80/c121 0 2024-04-25T00:18:36.842 INFO:tasks.workunit.client.0.smithi107.stdout:3/875: read f7 [1527414,118477] 0 2024-04-25T00:18:36.843 INFO:tasks.workunit.client.0.smithi107.stdout:3/876: write d9/d54/d5c/d68/d3b/f47 [144393,78525] 0 2024-04-25T00:18:36.844 INFO:tasks.workunit.client.0.smithi107.stdout:9/880: dwrite d0/d8a/d93/d56/d76/d50/fdb [0,4194304] 0 2024-04-25T00:18:36.847 INFO:tasks.workunit.client.0.smithi107.stdout:9/881: creat d0/d8a/d93/de/d5c/d95/f12e x:0 0 0 2024-04-25T00:18:36.848 INFO:tasks.workunit.client.0.smithi107.stdout:9/882: creat d0/d8a/d93/d56/f12f x:0 0 0 2024-04-25T00:18:36.850 INFO:tasks.workunit.client.0.smithi107.stdout:9/883: creat d0/d8a/d93/d56/d6c/d80/d9b/db2/dea/d10c/f130 x:0 0 0 2024-04-25T00:18:36.852 INFO:tasks.workunit.client.0.smithi107.stdout:9/884: symlink d0/d8a/d93/de/l131 0 2024-04-25T00:18:36.855 INFO:tasks.workunit.client.0.smithi107.stdout:5/807: rename d4/dc/d26/d42/d8b/f99 to d4/dc/d83/df1/f10c 0 2024-04-25T00:18:36.857 INFO:tasks.workunit.client.0.smithi107.stdout:5/808: dread d4/d17/f79 [0,4194304] 0 2024-04-25T00:18:36.858 INFO:tasks.workunit.client.0.smithi107.stdout:8/903: dwrite d2/d18/d1a/d5e/d4f/da0/da1/db9/f11d [0,4194304] 0 2024-04-25T00:18:36.858 INFO:tasks.workunit.client.0.smithi107.stdout:8/904: write d2/d18/d1a/d5e/d3e/fcc [896612,114193] 0 2024-04-25T00:18:36.858 INFO:tasks.workunit.client.0.smithi107.stdout:5/809: mkdir d4/d1c/d5c/dd4/d107/d10d 0 2024-04-25T00:18:36.859 INFO:tasks.workunit.client.0.smithi107.stdout:5/810: read d4/d17/f38 [109148,45485] 0 2024-04-25T00:18:36.859 INFO:tasks.workunit.client.0.smithi107.stdout:5/811: symlink d4/d91/l10e 0 2024-04-25T00:18:36.860 INFO:tasks.workunit.client.0.smithi107.stdout:7/920: dwrite d0/d8/db/d18/f8d [0,4194304] 0 2024-04-25T00:18:36.860 INFO:tasks.workunit.client.0.smithi107.stdout:2/870: dwrite d6/d20/d22/fa0 [0,4194304] 0 2024-04-25T00:18:36.861 INFO:tasks.workunit.client.0.smithi107.stdout:5/812: creat d4/dc/d26/d29/d2e/d55/f10f x:0 0 0 2024-04-25T00:18:36.862 INFO:tasks.workunit.client.0.smithi107.stdout:3/877: dwrite d9/d54/d94/dab/ffe [0,4194304] 0 2024-04-25T00:18:36.862 INFO:tasks.workunit.client.0.smithi107.stdout:2/871: truncate d6/d20/d22/d56/f68 797303 0 2024-04-25T00:18:36.863 INFO:tasks.workunit.client.0.smithi107.stdout:6/698: dwrite d1/d2/db/d32/d33/d4f/f58 [0,4194304] 0 2024-04-25T00:18:36.863 INFO:tasks.workunit.client.0.smithi107.stdout:2/872: write d6/d18/d40/d42/d2b/d30/f4b [1331901,42582] 0 2024-04-25T00:18:36.869 INFO:tasks.workunit.client.0.smithi107.stdout:6/699: mkdir d1/d2/d3/d26/de2 0 2024-04-25T00:18:36.869 INFO:tasks.workunit.client.0.smithi107.stdout:2/873: mkdir d6/d20/d122/d129 0 2024-04-25T00:18:36.871 INFO:tasks.workunit.client.0.smithi107.stdout:6/700: creat d1/d2/db/d32/d33/d68/fe3 x:0 0 0 2024-04-25T00:18:36.871 INFO:tasks.workunit.client.0.smithi107.stdout:6/701: read d1/d2/db/d32/d33/d68/d75/f8b [1873191,35350] 0 2024-04-25T00:18:36.871 INFO:tasks.workunit.client.0.smithi107.stdout:5/813: dread d4/dc/f13 [0,4194304] 0 2024-04-25T00:18:36.878 INFO:tasks.workunit.client.0.smithi107.stdout:9/885: dwrite d0/d43/f28 [0,4194304] 0 2024-04-25T00:18:36.878 INFO:tasks.workunit.client.0.smithi107.stdout:9/886: truncate d0/d8a/d93/de/d5c/d95/d106/f119 360215 0 2024-04-25T00:18:36.879 INFO:tasks.workunit.client.0.smithi107.stdout:3/878: read d9/d15/da1/fa8 [749197,22174] 0 2024-04-25T00:18:36.879 INFO:tasks.workunit.client.0.smithi107.stdout:3/879: write d9/d88/d2c/f37 [3686205,78175] 0 2024-04-25T00:18:36.881 INFO:tasks.workunit.client.0.smithi107.stdout:3/880: creat d9/d88/d3a/d91/f122 x:0 0 0 2024-04-25T00:18:36.883 INFO:tasks.workunit.client.0.smithi107.stdout:7/921: dwrite d0/d8/db/d18/d19/f3c [0,4194304] 0 2024-04-25T00:18:36.889 INFO:tasks.workunit.client.0.smithi107.stdout:2/874: dread d6/d18/d40/d42/d2b/d30/f7c [4194304,4194304] 0 2024-04-25T00:18:36.889 INFO:tasks.workunit.client.0.smithi107.stdout:8/905: rename d2/d18/c9a to d2/d18/d1a/d5e/d4f/da0/da1/de8/d119/d137/c13b 0 2024-04-25T00:18:36.889 INFO:tasks.workunit.client.0.smithi107.stdout:8/906: write d2/d18/d1a/d5e/d4f/da0/fac [1938434,36294] 0 2024-04-25T00:18:36.890 INFO:tasks.workunit.client.0.smithi107.stdout:6/702: write d1/d2/db/d3c/f50 [3890493,76782] 0 2024-04-25T00:18:36.890 INFO:tasks.workunit.client.0.smithi107.stdout:9/887: mkdir d0/d8a/d93/d34/d132 0 2024-04-25T00:18:36.890 INFO:tasks.workunit.client.0.smithi107.stdout:6/703: creat d1/d2/dad/fe4 x:0 0 0 2024-04-25T00:18:36.892 INFO:tasks.workunit.client.0.smithi107.stdout:4/866: sync 2024-04-25T00:18:36.893 INFO:tasks.workunit.client.0.smithi107.stdout:3/881: dread d9/d22/f7a [0,4194304] 0 2024-04-25T00:18:36.893 INFO:tasks.workunit.client.0.smithi107.stdout:3/882: chown d9/d88/d3a/d91/ff2 235 1 2024-04-25T00:18:36.894 INFO:tasks.workunit.client.0.smithi107.stdout:9/888: rmdir d0/d8a/d93/d34/d36/d77/dac/dee/d11c 0 2024-04-25T00:18:36.896 INFO:tasks.workunit.client.0.smithi107.stdout:6/704: rename d1/c30 to d1/d14/d18/ddd/ce5 0 2024-04-25T00:18:36.900 INFO:tasks.workunit.client.0.smithi107.stdout:2/875: dread d6/d18/d40/d42/d2b/f7a [4194304,4194304] 0 2024-04-25T00:18:36.900 INFO:tasks.workunit.client.0.smithi107.stdout:2/876: chown d6/d18/d40/d42/d69/d6e/da2/dc5/fd8 0 1 2024-04-25T00:18:36.900 INFO:tasks.workunit.client.0.smithi107.stdout:2/877: fsync d6/d18/d40/d42/d69/d6e/da2/dc5/ff2 0 2024-04-25T00:18:36.901 INFO:tasks.workunit.client.0.smithi107.stdout:7/922: dwrite d0/d8/db/d18/d19/f107 [0,4194304] 0 2024-04-25T00:18:36.903 INFO:tasks.workunit.client.0.smithi107.stdout:4/867: unlink d3/d24/d5c/d7a/l102 0 2024-04-25T00:18:36.904 INFO:tasks.workunit.client.0.smithi107.stdout:1/825: sync 2024-04-25T00:18:36.904 INFO:tasks.workunit.client.0.smithi107.stdout:0/775: sync 2024-04-25T00:18:36.904 INFO:tasks.workunit.client.0.smithi107.stdout:3/883: mkdir d9/d15/d123 0 2024-04-25T00:18:36.904 INFO:tasks.workunit.client.0.smithi107.stdout:8/907: dread d2/d18/f20 [0,4194304] 0 2024-04-25T00:18:36.904 INFO:tasks.workunit.client.0.smithi107.stdout:7/923: mknod d0/d8/d4e/d46/c135 0 2024-04-25T00:18:36.904 INFO:tasks.workunit.client.0.smithi107.stdout:2/878: truncate d6/d18/f65 1795 0 2024-04-25T00:18:36.905 INFO:tasks.workunit.client.0.smithi107.stdout:1/826: unlink d2/dd/f6f 0 2024-04-25T00:18:36.907 INFO:tasks.workunit.client.0.smithi107.stdout:0/776: creat dc/dd/d5d/d5c/ff4 x:0 0 0 2024-04-25T00:18:36.910 INFO:tasks.workunit.client.0.smithi107.stdout:2/879: unlink d6/d18/d40/d42/d69/d6e/d80/f95 0 2024-04-25T00:18:36.910 INFO:tasks.workunit.client.0.smithi107.stdout:2/880: write d6/d18/f2e [1438521,114260] 0 2024-04-25T00:18:36.910 INFO:tasks.workunit.client.0.smithi107.stdout:2/881: read d6/d18/f2e [1203235,8022] 0 2024-04-25T00:18:36.910 INFO:tasks.workunit.client.0.smithi107.stdout:2/882: chown d6/d18/d40/d42/d69/d6e/da2 111 1 2024-04-25T00:18:36.910 INFO:tasks.workunit.client.0.smithi107.stdout:1/827: getdents d2/d62/d66/d70/d95 0 2024-04-25T00:18:36.911 INFO:tasks.workunit.client.0.smithi107.stdout:7/924: creat d0/d8/db/d18/d19/d47/da1/dd4/f136 x:0 0 0 2024-04-25T00:18:36.911 INFO:tasks.workunit.client.0.smithi107.stdout:4/868: creat d3/d5/d32/d53/d7e/d9d/f125 x:0 0 0 2024-04-25T00:18:36.911 INFO:tasks.workunit.client.0.smithi107.stdout:9/889: creat d0/d8a/d93/d6f/da1/df2/f133 x:0 0 0 2024-04-25T00:18:36.911 INFO:tasks.workunit.client.0.smithi107.stdout:7/925: stat d0/d8/db/d18/d19/d67/c91 0 2024-04-25T00:18:36.913 INFO:tasks.workunit.client.0.smithi107.stdout:9/890: rmdir d0/d8a/d93/d34/d36/d77/dac/dee 39 2024-04-25T00:18:36.913 INFO:tasks.workunit.client.0.smithi107.stdout:9/891: write d0/d8a/d93/d34/fa9 [840765,61694] 0 2024-04-25T00:18:36.915 INFO:tasks.workunit.client.0.smithi107.stdout:9/892: symlink d0/d8a/d93/dc2/l134 0 2024-04-25T00:18:36.918 INFO:tasks.workunit.client.0.smithi107.stdout:9/893: dread d0/d8a/d93/de/f33 [0,4194304] 0 2024-04-25T00:18:36.919 INFO:tasks.workunit.client.0.smithi107.stdout:9/894: creat d0/d8a/f135 x:0 0 0 2024-04-25T00:18:36.920 INFO:tasks.workunit.client.0.smithi107.stdout:9/895: readlink d0/d8a/d93/d56/d76/d50/l71 0 2024-04-25T00:18:36.920 INFO:tasks.workunit.client.0.smithi107.stdout:9/896: chown d0/f40 37 1 2024-04-25T00:18:36.920 INFO:tasks.workunit.client.0.smithi107.stdout:9/897: readlink d0/d8a/d93/d34/d36/d77/d110/l129 0 2024-04-25T00:18:36.920 INFO:tasks.workunit.client.0.smithi107.stdout:9/898: write d0/d8a/d93/f5a [3717970,130607] 0 2024-04-25T00:18:36.921 INFO:tasks.workunit.client.0.smithi107.stdout:6/705: dwrite d1/d2/d3/d28/d79/d6b/f6f [0,4194304] 0 2024-04-25T00:18:36.927 INFO:tasks.workunit.client.0.smithi107.stdout:4/869: dread d3/d5/fe6 [0,4194304] 0 2024-04-25T00:18:36.927 INFO:tasks.workunit.client.0.smithi107.stdout:4/870: truncate d3/d24/d25/d71/f75 131928 0 2024-04-25T00:18:36.927 INFO:tasks.workunit.client.0.smithi107.stdout:4/871: chown d3/d5/d32/d53/l63 207842 1 2024-04-25T00:18:36.927 INFO:tasks.workunit.client.0.smithi107.stdout:4/872: write d3/d5/f7 [627634,5220] 0 2024-04-25T00:18:36.928 INFO:tasks.workunit.client.0.smithi107.stdout:1/828: write d2/d2a/f9e [3038707,92553] 0 2024-04-25T00:18:36.928 INFO:tasks.workunit.client.0.smithi107.stdout:2/883: dread d6/d18/d40/d42/d69/d6e/da2/d10c/d102/d9f/fe1 [0,4194304] 0 2024-04-25T00:18:36.929 INFO:tasks.workunit.client.0.smithi107.stdout:8/908: dwrite d2/d18/d1a/d5e/d3e/db1/df0/f133 [0,4194304] 0 2024-04-25T00:18:36.929 INFO:tasks.workunit.client.0.smithi107.stdout:3/884: dwrite d9/d15/da1/f79 [0,4194304] 0 2024-04-25T00:18:36.930 INFO:tasks.workunit.client.0.smithi107.stdout:2/884: mknod d6/dad/d67/da8/c12a 0 2024-04-25T00:18:36.930 INFO:tasks.workunit.client.0.smithi107.stdout:1/829: mknod d2/d62/d66/d70/c10d 0 2024-04-25T00:18:36.930 INFO:tasks.workunit.client.0.smithi107.stdout:1/830: dread - d2/d2a/d5c/fea zero size 2024-04-25T00:18:36.930 INFO:tasks.workunit.client.0.smithi107.stdout:2/885: write d6/d18/d40/d42/d70/ffb [231903,20060] 0 2024-04-25T00:18:36.934 INFO:tasks.workunit.client.0.smithi107.stdout:2/886: mkdir d6/d18/d40/d42/d12b 0 2024-04-25T00:18:36.935 INFO:tasks.workunit.client.0.smithi107.stdout:3/885: rmdir d9/d88/d80 39 2024-04-25T00:18:36.935 INFO:tasks.workunit.client.0.smithi107.stdout:3/886: chown d9/d15/da1/d4b/c10a 4692486 1 2024-04-25T00:18:36.935 INFO:tasks.workunit.client.0.smithi107.stdout:3/887: chown d9/d88/d2c/f102 118505 1 2024-04-25T00:18:36.937 INFO:tasks.workunit.client.0.smithi107.stdout:4/873: write d3/d24/d25/d71/fd5 [8598298,106313] 0 2024-04-25T00:18:36.938 INFO:tasks.workunit.client.0.smithi107.stdout:9/899: dread d0/d8a/d93/f26 [0,4194304] 0 2024-04-25T00:18:36.939 INFO:tasks.workunit.client.0.smithi107.stdout:9/900: readlink d0/d8a/d93/d34/lec 0 2024-04-25T00:18:36.939 INFO:tasks.workunit.client.0.smithi107.stdout:9/901: write d0/d8a/d93/de/d5c/f6a [1731524,109613] 0 2024-04-25T00:18:36.940 INFO:tasks.workunit.client.0.smithi107.stdout:2/887: dread d6/d20/d22/d56/d94/fdf [0,4194304] 0 2024-04-25T00:18:36.940 INFO:tasks.workunit.client.0.smithi107.stdout:2/888: write d6/dad/f85 [1529857,107691] 0 2024-04-25T00:18:36.941 INFO:tasks.workunit.client.0.smithi107.stdout:1/831: dread d2/da/d2f/f9b [0,4194304] 0 2024-04-25T00:18:36.943 INFO:tasks.workunit.client.0.smithi107.stdout:9/902: creat d0/d8a/d93/d56/d6c/d80/f136 x:0 0 0 2024-04-25T00:18:36.943 INFO:tasks.workunit.client.0.smithi107.stdout:9/903: fdatasync d0/f8f 0 2024-04-25T00:18:36.947 INFO:tasks.workunit.client.0.smithi107.stdout:3/888: write d9/d22/d1b/f48 [2739360,27521] 0 2024-04-25T00:18:36.947 INFO:tasks.workunit.client.0.smithi107.stdout:3/889: chown d9 471 1 2024-04-25T00:18:36.947 INFO:tasks.workunit.client.0.smithi107.stdout:3/890: dread - d9/d88/d3a/d91/dad/deb/ff4 zero size 2024-04-25T00:18:36.947 INFO:tasks.workunit.client.0.smithi107.stdout:3/891: readlink d9/d15/l43 0 2024-04-25T00:18:36.950 INFO:tasks.workunit.client.0.smithi107.stdout:1/832: link d2/dd/c57 d2/dd/d1b/dc2/c10e 0 2024-04-25T00:18:36.950 INFO:tasks.workunit.client.0.smithi107.stdout:8/909: dread d2/d18/d1a/fdc [0,4194304] 0 2024-04-25T00:18:36.950 INFO:tasks.workunit.client.0.smithi107.stdout:8/910: chown d2/d18/d1a/d5e/d3d/d85/f101 31325984 1 2024-04-25T00:18:36.952 INFO:tasks.workunit.client.0.smithi107.stdout:9/904: mkdir d0/d8a/d93/d56/d6c/d80/d9b/db2/dea/d10c/d137 0 2024-04-25T00:18:36.952 INFO:tasks.workunit.client.0.smithi107.stdout:9/905: fdatasync d0/d8a/d93/d56/d6c/d80/fad 0 2024-04-25T00:18:36.954 INFO:tasks.workunit.client.0.smithi107.stdout:3/892: write d9/d15/da1/fea [3606187,78704] 0 2024-04-25T00:18:36.955 INFO:tasks.workunit.client.0.smithi107.stdout:1/833: link d2/d2a/d37/d88/cd2 d2/d2a/d37/de8/d105/c10f 0 2024-04-25T00:18:36.955 INFO:tasks.workunit.client.0.smithi107.stdout:1/834: fsync d2/dd/f8b 0 2024-04-25T00:18:36.956 INFO:tasks.workunit.client.0.smithi107.stdout:8/911: truncate d2/d18/d1a/fdc 1423772 0 2024-04-25T00:18:36.959 INFO:tasks.workunit.client.0.smithi107.stdout:9/906: mkdir d0/d8a/d93/d56/d6c/d138 0 2024-04-25T00:18:36.960 INFO:tasks.workunit.client.0.smithi107.stdout:9/907: write d0/d8a/d93/de/d5c/d95/f12e [997750,58776] 0 2024-04-25T00:18:36.960 INFO:tasks.workunit.client.0.smithi107.stdout:9/908: write d0/d8a/d93/d34/d36/f12b [329074,119330] 0 2024-04-25T00:18:36.964 INFO:tasks.workunit.client.0.smithi107.stdout:1/835: truncate d2/da4/fda 1958845 0 2024-04-25T00:18:36.964 INFO:tasks.workunit.client.0.smithi107.stdout:2/889: dwrite d6/d29/d4e/f7f [0,4194304] 0 2024-04-25T00:18:36.965 INFO:tasks.workunit.client.0.smithi107.stdout:4/874: dwrite d3/d5/f4d [0,4194304] 0 2024-04-25T00:18:36.965 INFO:tasks.workunit.client.0.smithi107.stdout:4/875: chown d3/d5/d32/d53/c116 100916367 1 2024-04-25T00:18:36.966 INFO:tasks.workunit.client.0.smithi107.stdout:1/836: mknod d2/dd/d25/da9/c110 0 2024-04-25T00:18:36.973 INFO:tasks.workunit.client.0.smithi107.stdout:2/890: creat d6/d18/d40/d42/d2b/d11d/f12c x:0 0 0 2024-04-25T00:18:36.973 INFO:tasks.workunit.client.0.smithi107.stdout:2/891: write d6/d18/d40/d42/d2b/d30/d45/fb9 [1797553,17300] 0 2024-04-25T00:18:36.973 INFO:tasks.workunit.client.0.smithi107.stdout:2/892: write d6/d18/d40/d42/d2b/d30/d45/dec/ff3 [98120,101766] 0 2024-04-25T00:18:36.973 INFO:tasks.workunit.client.0.smithi107.stdout:2/893: chown d6/d20/d22/d56/d94 6144 1 2024-04-25T00:18:36.976 INFO:tasks.workunit.client.0.smithi107.stdout:1/837: creat d2/da/db7/f111 x:0 0 0 2024-04-25T00:18:36.978 INFO:tasks.workunit.client.0.smithi107.stdout:0/777: rename dc/dd/d5d/d3f/d59/f81 to dc/dd/d16/d5e/ff5 0 2024-04-25T00:18:36.979 INFO:tasks.workunit.client.0.smithi107.stdout:1/838: creat d2/d62/d66/d70/dcc/f112 x:0 0 0 2024-04-25T00:18:36.980 INFO:tasks.workunit.client.0.smithi107.stdout:7/926: rename d0/d8/db/d18/d19/d67/l81 to d0/d8/db/d92/l137 0 2024-04-25T00:18:36.982 INFO:tasks.workunit.client.0.smithi107.stdout:4/876: rmdir d3/d24/d5c/d7a 39 2024-04-25T00:18:36.982 INFO:tasks.workunit.client.0.smithi107.stdout:9/909: dwrite d0/fb1 [0,4194304] 0 2024-04-25T00:18:36.983 INFO:tasks.workunit.client.0.smithi107.stdout:9/910: write d0/d8a/d93/dc2/dc6/f123 [463545,2003] 0 2024-04-25T00:18:36.984 INFO:tasks.workunit.client.0.smithi107.stdout:7/927: symlink d0/l138 0 2024-04-25T00:18:36.987 INFO:tasks.workunit.client.0.smithi107.stdout:9/911: creat d0/d8a/d93/d6f/da1/df2/f139 x:0 0 0 2024-04-25T00:18:36.989 INFO:tasks.workunit.client.0.smithi107.stdout:9/912: symlink d0/d8a/d93/de/d5c/d124/l13a 0 2024-04-25T00:18:36.995 INFO:tasks.workunit.client.0.smithi107.stdout:1/839: dread d2/dd/d82/dc1/fd3 [0,4194304] 0 2024-04-25T00:18:36.998 INFO:tasks.workunit.client.0.smithi107.stdout:1/840: mknod d2/d2a/d37/de8/d105/c113 0 2024-04-25T00:18:36.998 INFO:tasks.workunit.client.0.smithi107.stdout:1/841: chown d2/dd/d1b/cab 52546 1 2024-04-25T00:18:36.998 INFO:tasks.workunit.client.0.smithi107.stdout:1/842: readlink d2/d2a/d37/d88/lb0 0 2024-04-25T00:18:37.000 INFO:tasks.workunit.client.0.smithi107.stdout:1/843: creat d2/da/d2f/f114 x:0 0 0 2024-04-25T00:18:37.002 INFO:tasks.workunit.client.0.smithi107.stdout:8/912: dread d2/d18/d1a/dca/fcb [0,4194304] 0 2024-04-25T00:18:37.002 INFO:tasks.workunit.client.0.smithi107.stdout:8/913: write d2/f26 [568599,109937] 0 2024-04-25T00:18:37.004 INFO:tasks.workunit.client.0.smithi107.stdout:2/894: dwrite d6/d18/d40/d42/f3b [0,4194304] 0 2024-04-25T00:18:37.004 INFO:tasks.workunit.client.0.smithi107.stdout:2/895: dread - d6/d18/d40/d42/d69/d6e/da2/dc5/fd8 zero size 2024-04-25T00:18:37.004 INFO:tasks.workunit.client.0.smithi107.stdout:2/896: write d6/f7 [842071,89133] 0 2024-04-25T00:18:37.004 INFO:tasks.workunit.client.0.smithi107.stdout:1/844: mknod d2/da/d2f/c115 0 2024-04-25T00:18:37.004 INFO:tasks.workunit.client.0.smithi107.stdout:1/845: read - d2/da/db7/ff6 zero size 2024-04-25T00:18:37.004 INFO:tasks.workunit.client.0.smithi107.stdout:1/846: stat d2/dd/fdd 0 2024-04-25T00:18:37.005 INFO:tasks.workunit.client.0.smithi107.stdout:0/778: dwrite dc/dd/fd8 [0,4194304] 0 2024-04-25T00:18:37.005 INFO:tasks.workunit.client.0.smithi107.stdout:8/914: mkdir d2/d18/d1a/db7/d13c 0 2024-04-25T00:18:37.008 INFO:tasks.workunit.client.0.smithi107.stdout:6/706: rename d1/d2 to d1/d14/de6 0 2024-04-25T00:18:37.008 INFO:tasks.workunit.client.0.smithi107.stdout:6/707: chown d1/d14/de6/db/d32/d33/d68/d75/f78 1819470 1 2024-04-25T00:18:37.008 INFO:tasks.workunit.client.0.smithi107.stdout:6/708: truncate d1/d14/de6/db/d3c/d43/d96/dc9/fdf 778085 0 2024-04-25T00:18:37.009 INFO:tasks.workunit.client.0.smithi107.stdout:7/928: dwrite d0/d8/d4e/d46/d93/fbd [0,4194304] 0 2024-04-25T00:18:37.009 INFO:tasks.workunit.client.0.smithi107.stdout:6/709: symlink d1/d14/de6/d3/d28/d79/de0/le7 0 2024-04-25T00:18:37.010 INFO:tasks.workunit.client.0.smithi107.stdout:6/710: write d1/d14/de6/db/d3c/f50 [1852180,58539] 0 2024-04-25T00:18:37.010 INFO:tasks.workunit.client.0.smithi107.stdout:2/897: creat d6/d18/d40/d42/d69/d6e/db6/f12d x:0 0 0 2024-04-25T00:18:37.010 INFO:tasks.workunit.client.0.smithi107.stdout:2/898: write d6/d18/d40/f8f [1834250,69698] 0 2024-04-25T00:18:37.012 INFO:tasks.workunit.client.0.smithi107.stdout:4/877: dwrite d3/d24/f2c [0,4194304] 0 2024-04-25T00:18:37.012 INFO:tasks.workunit.client.0.smithi107.stdout:7/929: truncate d0/d8/db/d18/d19/d47/f53 1136820 0 2024-04-25T00:18:37.013 INFO:tasks.workunit.client.0.smithi107.stdout:2/899: mkdir d6/d18/d12e 0 2024-04-25T00:18:37.013 INFO:tasks.workunit.client.0.smithi107.stdout:2/900: chown d6/d18/d40/dc8/de5/c108 3838 1 2024-04-25T00:18:37.013 INFO:tasks.workunit.client.0.smithi107.stdout:4/878: mkdir d3/d24/db1/d126 0 2024-04-25T00:18:37.013 INFO:tasks.workunit.client.0.smithi107.stdout:2/901: chown d6/d18/d40/d42/d69/d6e/da2/dc5/dd2/cf5 242 1 2024-04-25T00:18:37.014 INFO:tasks.workunit.client.0.smithi107.stdout:2/902: write d6/f51 [5123701,105686] 0 2024-04-25T00:18:37.014 INFO:tasks.workunit.client.0.smithi107.stdout:2/903: write d6/d18/d40/d42/d2b/d30/f4b [2397470,109449] 0 2024-04-25T00:18:37.014 INFO:tasks.workunit.client.0.smithi107.stdout:4/879: mknod d3/d24/d61/c127 0 2024-04-25T00:18:37.014 INFO:tasks.workunit.client.0.smithi107.stdout:4/880: truncate d3/d10/d7b/feb 1291585 0 2024-04-25T00:18:37.017 INFO:tasks.workunit.client.0.smithi107.stdout:1/847: dread d2/dd/d82/fa7 [0,4194304] 0 2024-04-25T00:18:37.017 INFO:tasks.workunit.client.0.smithi107.stdout:1/848: write d2/da/f8e [3458956,92925] 0 2024-04-25T00:18:37.017 INFO:tasks.workunit.client.0.smithi107.stdout:1/849: fdatasync d2/dd/d25/d5e/f94 0 2024-04-25T00:18:37.019 INFO:tasks.workunit.client.0.smithi107.stdout:1/850: creat d2/d62/d66/d70/dcc/f116 x:0 0 0 2024-04-25T00:18:37.020 INFO:tasks.workunit.client.0.smithi107.stdout:4/881: creat d3/d5/d32/d53/dc0/f128 x:0 0 0 2024-04-25T00:18:37.023 INFO:tasks.workunit.client.0.smithi107.stdout:4/882: mknod d3/c129 0 2024-04-25T00:18:37.023 INFO:tasks.workunit.client.0.smithi107.stdout:1/851: unlink d2/d2a/d37/d88/c9c 0 2024-04-25T00:18:37.025 INFO:tasks.workunit.client.0.smithi107.stdout:9/913: dwrite d0/d8a/d93/d6f/fc4 [0,4194304] 0 2024-04-25T00:18:37.028 INFO:tasks.workunit.client.0.smithi107.stdout:2/904: write d6/d18/d40/d42/d2b/d30/d45/f5f [74491,70147] 0 2024-04-25T00:18:37.028 INFO:tasks.workunit.client.0.smithi107.stdout:8/915: dwrite d2/d18/d1a/d5e/d3d/d85/f101 [0,4194304] 0 2024-04-25T00:18:37.028 INFO:tasks.workunit.client.0.smithi107.stdout:8/916: chown d2/d18/d1a/d5e/d3d/f109 96 1 2024-04-25T00:18:37.028 INFO:tasks.workunit.client.0.smithi107.stdout:8/917: stat d2/d18/d1a/d82 0 2024-04-25T00:18:37.030 INFO:tasks.workunit.client.0.smithi107.stdout:7/930: dread d0/f3b [0,4194304] 0 2024-04-25T00:18:37.031 INFO:tasks.workunit.client.0.smithi107.stdout:6/711: dwrite d1/d14/de6/db/d32/d33/fc3 [0,4194304] 0 2024-04-25T00:18:37.031 INFO:tasks.workunit.client.0.smithi107.stdout:4/883: symlink d3/d24/d5c/l12a 0 2024-04-25T00:18:37.037 INFO:tasks.workunit.client.0.smithi107.stdout:8/918: creat d2/d18/d1a/d5e/d4f/da0/da1/de8/d119/f13d x:0 0 0 2024-04-25T00:18:37.037 INFO:tasks.workunit.client.0.smithi107.stdout:7/931: creat d0/d8/db/d18/d19/d47/da1/dd4/d128/f139 x:0 0 0 2024-04-25T00:18:37.037 INFO:tasks.workunit.client.0.smithi107.stdout:8/919: readlink d2/d18/d1a/d5e/d3e/d74/lc8 0 2024-04-25T00:18:37.038 INFO:tasks.workunit.client.0.smithi107.stdout:1/852: dread d2/d2a/f9e [0,4194304] 0 2024-04-25T00:18:37.038 INFO:tasks.workunit.client.0.smithi107.stdout:6/712: mknod d1/d14/de6/db/d3c/d9c/ce8 0 2024-04-25T00:18:37.039 INFO:tasks.workunit.client.0.smithi107.stdout:6/713: truncate d1/d14/de6/d2b/d3e/f8f 4935284 0 2024-04-25T00:18:37.039 INFO:tasks.workunit.client.0.smithi107.stdout:6/714: write d1/d14/de6/dad/fe4 [992980,128944] 0 2024-04-25T00:18:37.039 INFO:tasks.workunit.client.0.smithi107.stdout:4/884: creat d3/f12b x:0 0 0 2024-04-25T00:18:37.042 INFO:tasks.workunit.client.0.smithi107.stdout:7/932: creat d0/d8/db/d18/d19/d67/f13a x:0 0 0 2024-04-25T00:18:37.042 INFO:tasks.workunit.client.0.smithi107.stdout:9/914: mknod d0/d8a/d93/d56/d6c/daa/c13b 0 2024-04-25T00:18:37.042 INFO:tasks.workunit.client.0.smithi107.stdout:7/933: chown d0/c95 3 1 2024-04-25T00:18:37.042 INFO:tasks.workunit.client.0.smithi107.stdout:7/934: chown d0/d8/db/d18/d19/d47/d4d/d8a/fa6 24288 1 2024-04-25T00:18:37.044 INFO:tasks.workunit.client.0.smithi107.stdout:4/885: creat d3/d5/d32/d9e/d111/f12c x:0 0 0 2024-04-25T00:18:37.044 INFO:tasks.workunit.client.0.smithi107.stdout:4/886: dread - d3/d5/d32/ff4 zero size 2024-04-25T00:18:37.044 INFO:tasks.workunit.client.0.smithi107.stdout:8/920: creat d2/d18/d1a/d5e/d4f/da0/d110/f13e x:0 0 0 2024-04-25T00:18:37.044 INFO:tasks.workunit.client.0.smithi107.stdout:8/921: truncate d2/d18/d1a/d5e/d4f/f138 333077 0 2024-04-25T00:18:37.047 INFO:tasks.workunit.client.0.smithi107.stdout:1/853: unlink d2/da/d2f/fad 0 2024-04-25T00:18:37.047 INFO:tasks.workunit.client.0.smithi107.stdout:7/935: mknod d0/d8/db/d92/c13b 0 2024-04-25T00:18:37.050 INFO:tasks.workunit.client.0.smithi107.stdout:4/887: symlink d3/d24/d25/l12d 0 2024-04-25T00:18:37.050 INFO:tasks.workunit.client.0.smithi107.stdout:9/915: creat d0/d8a/d93/db9/f13c x:0 0 0 2024-04-25T00:18:37.050 INFO:tasks.workunit.client.0.smithi107.stdout:9/916: chown d0/d8a/d93/d6f/da1/df2 9016 1 2024-04-25T00:18:37.052 INFO:tasks.workunit.client.0.smithi107.stdout:8/922: creat d2/d18/d1a/d128/f13f x:0 0 0 2024-04-25T00:18:37.053 INFO:tasks.workunit.client.0.smithi107.stdout:4/888: link d3/d5/d32/f79 d3/d5/d32/d53/d7e/f12e 0 2024-04-25T00:18:37.054 INFO:tasks.workunit.client.0.smithi107.stdout:9/917: mkdir d0/d8a/d93/d56/d13d 0 2024-04-25T00:18:37.056 INFO:tasks.workunit.client.0.smithi107.stdout:7/936: dread d0/d8/db/d18/d19/d39/f110 [0,4194304] 0 2024-04-25T00:18:37.056 INFO:tasks.workunit.client.0.smithi107.stdout:4/889: creat d3/d5/f12f x:0 0 0 2024-04-25T00:18:37.056 INFO:tasks.workunit.client.0.smithi107.stdout:9/918: mkdir d0/d8a/d93/d56/d6c/d80/d9b/db2/dea/d10c/d137/d13e 0 2024-04-25T00:18:37.058 INFO:tasks.workunit.client.0.smithi107.stdout:8/923: dread d2/d18/d1a/d5e/d3e/faa [0,4194304] 0 2024-04-25T00:18:37.060 INFO:tasks.workunit.client.0.smithi107.stdout:2/905: dwrite d6/f7 [0,4194304] 0 2024-04-25T00:18:37.060 INFO:tasks.workunit.client.0.smithi107.stdout:7/937: write d0/d8/d4e/f32 [540132,98070] 0 2024-04-25T00:18:37.060 INFO:tasks.workunit.client.0.smithi107.stdout:7/938: dread - d0/d8/db/ff2 zero size 2024-04-25T00:18:37.062 INFO:tasks.workunit.client.0.smithi107.stdout:2/906: symlink d6/d18/d40/d42/d69/d6e/d80/l12f 0 2024-04-25T00:18:37.065 INFO:tasks.workunit.client.0.smithi107.stdout:2/907: mknod d6/d18/d40/d42/d69/d6e/da2/dc5/c130 0 2024-04-25T00:18:37.072 INFO:tasks.workunit.client.0.smithi107.stdout:9/919: read d0/d8a/d93/de/f90 [2429107,98531] 0 2024-04-25T00:18:37.073 INFO:tasks.workunit.client.0.smithi107.stdout:5/814: sync 2024-04-25T00:18:37.074 INFO:tasks.workunit.client.0.smithi107.stdout:3/893: rename d9/d22/f1e to d9/d88/d2c/f124 0 2024-04-25T00:18:37.074 INFO:tasks.workunit.client.0.smithi107.stdout:1/854: dwrite d2/dd/d25/f63 [0,4194304] 0 2024-04-25T00:18:37.075 INFO:tasks.workunit.client.0.smithi107.stdout:5/815: symlink d4/d17/d104/l110 0 2024-04-25T00:18:37.078 INFO:tasks.workunit.client.0.smithi107.stdout:0/779: rename dc/d24/f3a to dc/d24/d38/d64/d78/de6/ff6 0 2024-04-25T00:18:37.081 INFO:tasks.workunit.client.0.smithi107.stdout:4/890: rename d3/d5/d32/d53/d7e/fd4 to d3/d5/d32/d9e/f130 0 2024-04-25T00:18:37.081 INFO:tasks.workunit.client.0.smithi107.stdout:7/939: dwrite d0/d8/db/d18/d19/d47/d4d/d8a/fb8 [0,4194304] 0 2024-04-25T00:18:37.081 INFO:tasks.workunit.client.0.smithi107.stdout:9/920: dread d0/d8a/d93/de/d5c/d95/f12e [0,4194304] 0 2024-04-25T00:18:37.083 INFO:tasks.workunit.client.0.smithi107.stdout:4/891: rmdir d3/d5/d11f 0 2024-04-25T00:18:37.087 INFO:tasks.workunit.client.0.smithi107.stdout:1/855: write d2/d62/d66/d70/f8c [3163967,36756] 0 2024-04-25T00:18:37.087 INFO:tasks.workunit.client.0.smithi107.stdout:1/856: dread - d2/da/d2f/f102 zero size 2024-04-25T00:18:37.088 INFO:tasks.workunit.client.0.smithi107.stdout:9/921: rmdir d0/d8a/d93/d56/d6c/d80/d9b/db2/dea/d10c/d137/d13e 0 2024-04-25T00:18:37.088 INFO:tasks.workunit.client.0.smithi107.stdout:9/922: chown d0/d43/l12 495 1 2024-04-25T00:18:37.088 INFO:tasks.workunit.client.0.smithi107.stdout:0/780: rmdir dc/d24/d25/d31 39 2024-04-25T00:18:37.088 INFO:tasks.workunit.client.0.smithi107.stdout:8/924: link d2/d18/d1a/d5e/d3d/d85/c10e d2/d18/d1a/c140 0 2024-04-25T00:18:37.088 INFO:tasks.workunit.client.0.smithi107.stdout:2/908: dwrite d6/d18/d40/d42/d69/d6e/da2/dc5/dd2/d100/f107 [0,4194304] 0 2024-04-25T00:18:37.088 INFO:tasks.workunit.client.0.smithi107.stdout:1/857: mknod d2/dd/d25/d5e/c117 0 2024-04-25T00:18:37.090 INFO:tasks.workunit.client.0.smithi107.stdout:0/781: mkdir dc/dd/d16/dac/df7 0 2024-04-25T00:18:37.090 INFO:tasks.workunit.client.0.smithi107.stdout:2/909: mkdir d6/dad/d67/d131 0 2024-04-25T00:18:37.091 INFO:tasks.workunit.client.0.smithi107.stdout:2/910: write d6/d18/d40/d42/d69/d6e/da2/d10c/d102/d9f/fe1 [2610467,52004] 0 2024-04-25T00:18:37.091 INFO:tasks.workunit.client.0.smithi107.stdout:2/911: fdatasync d6/d18/d40/fe8 0 2024-04-25T00:18:37.091 INFO:tasks.workunit.client.0.smithi107.stdout:8/925: truncate d2/d18/d1a/d5e/fa7 6721477 0 2024-04-25T00:18:37.093 INFO:tasks.workunit.client.0.smithi107.stdout:9/923: symlink d0/d8a/d93/d56/d6c/d80/l13f 0 2024-04-25T00:18:37.093 INFO:tasks.workunit.client.0.smithi107.stdout:8/926: creat d2/d18/d1a/d5e/d4f/da0/da1/de8/d119/d137/f141 x:0 0 0 2024-04-25T00:18:37.093 INFO:tasks.workunit.client.0.smithi107.stdout:8/927: chown d2 245610 1 2024-04-25T00:18:37.094 INFO:tasks.workunit.client.0.smithi107.stdout:8/928: mknod d2/d18/d1a/d5e/d3e/c142 0 2024-04-25T00:18:37.095 INFO:tasks.workunit.client.0.smithi107.stdout:3/894: dwrite d9/f99 [0,4194304] 0 2024-04-25T00:18:37.095 INFO:tasks.workunit.client.0.smithi107.stdout:3/895: fsync d9/d22/d1b/dbd/f11a 0 2024-04-25T00:18:37.095 INFO:tasks.workunit.client.0.smithi107.stdout:3/896: dread - d9/d54/d5c/d68/d3b/f6a zero size 2024-04-25T00:18:37.097 INFO:tasks.workunit.client.0.smithi107.stdout:0/782: creat dc/d24/d25/ddd/ff8 x:0 0 0 2024-04-25T00:18:37.097 INFO:tasks.workunit.client.0.smithi107.stdout:4/892: dread d3/d5/d32/d53/f5b [0,4194304] 0 2024-04-25T00:18:37.097 INFO:tasks.workunit.client.0.smithi107.stdout:0/783: dread - dc/f2f zero size 2024-04-25T00:18:37.097 INFO:tasks.workunit.client.0.smithi107.stdout:0/784: getdents dc/dd/d16/d48/d83 0 2024-04-25T00:18:37.098 INFO:tasks.workunit.client.0.smithi107.stdout:0/785: rmdir dc/dd/d16/dac/df7 0 2024-04-25T00:18:37.098 INFO:tasks.workunit.client.0.smithi107.stdout:2/912: dread d6/d20/f88 [0,4194304] 0 2024-04-25T00:18:37.099 INFO:tasks.workunit.client.0.smithi107.stdout:2/913: chown d6/d18/d40/d42/d2b/d30/d45/dca/d11a 0 1 2024-04-25T00:18:37.104 INFO:tasks.workunit.client.0.smithi107.stdout:0/786: creat dc/d24/d38/d64/d78/ddc/ff9 x:0 0 0 2024-04-25T00:18:37.106 INFO:tasks.workunit.client.0.smithi107.stdout:7/940: dwrite d0/f13 [0,4194304] 0 2024-04-25T00:18:37.106 INFO:tasks.workunit.client.0.smithi107.stdout:2/914: link d6/d18/d40/d42/d2b/d30/d45/dec/f121 d6/d20/d122/f132 0 2024-04-25T00:18:37.106 INFO:tasks.workunit.client.0.smithi107.stdout:2/915: dread - d6/dad/db0/fdc zero size 2024-04-25T00:18:37.111 INFO:tasks.workunit.client.0.smithi107.stdout:0/787: link dc/d24/d38/d64/f68 dc/d24/d38/d64/d7b/ffa 0 2024-04-25T00:18:37.111 INFO:tasks.workunit.client.0.smithi107.stdout:0/788: truncate dc/f18 5020294 0 2024-04-25T00:18:37.112 INFO:tasks.workunit.client.0.smithi107.stdout:3/897: mknod d9/d54/d5c/d68/d3b/c125 0 2024-04-25T00:18:37.112 INFO:tasks.workunit.client.0.smithi107.stdout:3/898: truncate d9/d15/d39/fb7 917509 0 2024-04-25T00:18:37.112 INFO:tasks.workunit.client.0.smithi107.stdout:3/899: write d9/d15/da1/d4b/f57 [146657,59631] 0 2024-04-25T00:18:37.112 INFO:tasks.workunit.client.0.smithi107.stdout:2/916: link d6/d18/d40/d42/d2b/d30/d45/dca/dd4/cef d6/d18/d40/d42/d2b/d30/d45/dca/dd4/c133 0 2024-04-25T00:18:37.114 INFO:tasks.workunit.client.0.smithi107.stdout:4/893: read d3/f35 [2546651,94186] 0 2024-04-25T00:18:37.114 INFO:tasks.workunit.client.0.smithi107.stdout:0/789: dread - dc/dd/d5d/d5c/d6e/fcd zero size 2024-04-25T00:18:37.114 INFO:tasks.workunit.client.0.smithi107.stdout:0/790: read - dc/fbc zero size 2024-04-25T00:18:37.115 INFO:tasks.workunit.client.0.smithi107.stdout:2/917: mkdir d6/d18/d40/d42/d69/d6e/dc2/d134 0 2024-04-25T00:18:37.115 INFO:tasks.workunit.client.0.smithi107.stdout:2/918: fsync d6/d29/d4e/fab 0 2024-04-25T00:18:37.118 INFO:tasks.workunit.client.0.smithi107.stdout:4/894: mknod d3/dd8/c131 0 2024-04-25T00:18:37.118 INFO:tasks.workunit.client.0.smithi107.stdout:4/895: read d3/d5/d32/d9e/da2/daf/fdd [1676953,103877] 0 2024-04-25T00:18:37.118 INFO:tasks.workunit.client.0.smithi107.stdout:4/896: chown d3/d24/d25/f30 323277 1 2024-04-25T00:18:37.119 INFO:tasks.workunit.client.0.smithi107.stdout:4/897: chown d3/f35 43 1 2024-04-25T00:18:37.119 INFO:tasks.workunit.client.0.smithi107.stdout:4/898: read d3/d24/d25/d71/f75 [31148,49883] 0 2024-04-25T00:18:37.120 INFO:tasks.workunit.client.0.smithi107.stdout:8/929: dwrite d2/d18/d1a/f1e [0,4194304] 0 2024-04-25T00:18:37.120 INFO:tasks.workunit.client.0.smithi107.stdout:8/930: stat d2/d18/d1a/d5e/d3e/d74/le0 0 2024-04-25T00:18:37.120 INFO:tasks.workunit.client.0.smithi107.stdout:8/931: fdatasync d2/d18/d1a/d5e/d4f/f7e 0 2024-04-25T00:18:37.122 INFO:tasks.workunit.client.0.smithi107.stdout:1/858: dwrite d2/dd/f4e [0,4194304] 0 2024-04-25T00:18:37.122 INFO:tasks.workunit.client.0.smithi107.stdout:1/859: truncate d2/d2a/d37/d88/fbb 887629 0 2024-04-25T00:18:37.123 INFO:tasks.workunit.client.0.smithi107.stdout:2/919: creat d6/d18/d40/d42/d2b/d30/d4c/f135 x:0 0 0 2024-04-25T00:18:37.125 INFO:tasks.workunit.client.0.smithi107.stdout:1/860: write d2/d62/d66/d70/f7a [718515,23283] 0 2024-04-25T00:18:37.125 INFO:tasks.workunit.client.0.smithi107.stdout:0/791: dread dc/d24/d25/f2b [0,4194304] 0 2024-04-25T00:18:37.127 INFO:tasks.workunit.client.0.smithi107.stdout:2/920: unlink d6/d18/d40/lcf 0 2024-04-25T00:18:37.128 INFO:tasks.workunit.client.0.smithi107.stdout:0/792: mkdir dc/dd/d16/d48/dfb 0 2024-04-25T00:18:37.130 INFO:tasks.workunit.client.0.smithi107.stdout:9/924: dwrite d0/d43/f1c [0,4194304] 0 2024-04-25T00:18:37.130 INFO:tasks.workunit.client.0.smithi107.stdout:7/941: dwrite d0/d8/db/d18/d19/d47/d4d/f129 [0,4194304] 0 2024-04-25T00:18:37.131 INFO:tasks.workunit.client.0.smithi107.stdout:9/925: readlink d0/d8a/d93/d56/d6c/d80/la8 0 2024-04-25T00:18:37.131 INFO:tasks.workunit.client.0.smithi107.stdout:9/926: write d0/d8a/d93/d56/d76/f46 [5052160,17842] 0 2024-04-25T00:18:37.131 INFO:tasks.workunit.client.0.smithi107.stdout:0/793: creat dc/dd/d16/d33/d35/ffc x:0 0 0 2024-04-25T00:18:37.132 INFO:tasks.workunit.client.0.smithi107.stdout:9/927: mkdir d0/d8a/d93/d56/d6c/d80/d9b/d140 0 2024-04-25T00:18:37.135 INFO:tasks.workunit.client.0.smithi107.stdout:7/942: creat d0/d8/d4e/d46/f13c x:0 0 0 2024-04-25T00:18:37.136 INFO:tasks.workunit.client.0.smithi107.stdout:9/928: truncate d0/d8a/d93/d6f/fc4 2805728 0 2024-04-25T00:18:37.136 INFO:tasks.workunit.client.0.smithi107.stdout:7/943: creat d0/d80/da8/d123/f13d x:0 0 0 2024-04-25T00:18:37.138 INFO:tasks.workunit.client.0.smithi107.stdout:7/944: getdents d0/d8/db/d18 0 2024-04-25T00:18:37.138 INFO:tasks.workunit.client.0.smithi107.stdout:7/945: fdatasync d0/d8/db/d18/d19/d47/f53 0 2024-04-25T00:18:37.138 INFO:tasks.workunit.client.0.smithi107.stdout:2/921: dread d6/f51 [0,4194304] 0 2024-04-25T00:18:37.139 INFO:tasks.workunit.client.0.smithi107.stdout:4/899: dwrite d3/d24/f81 [0,4194304] 0 2024-04-25T00:18:37.139 INFO:tasks.workunit.client.0.smithi107.stdout:9/929: mknod d0/d8a/d93/d56/d76/d50/dd8/c141 0 2024-04-25T00:18:37.142 INFO:tasks.workunit.client.0.smithi107.stdout:9/930: creat d0/d8a/d93/d34/d132/f142 x:0 0 0 2024-04-25T00:18:37.142 INFO:tasks.workunit.client.0.smithi107.stdout:9/931: stat d0/d8a/d93/db9/fe6 0 2024-04-25T00:18:37.142 INFO:tasks.workunit.client.0.smithi107.stdout:3/900: link d9/d15/l51 d9/d88/d2c/l126 0 2024-04-25T00:18:37.142 INFO:tasks.workunit.client.0.smithi107.stdout:8/932: dwrite d2/d18/d1a/d5e/d3d/fc4 [4194304,4194304] 0 2024-04-25T00:18:37.142 INFO:tasks.workunit.client.0.smithi107.stdout:7/946: rename d0/d8/db/d18/d19/c104 to d0/d8/db/d18/d19/d47/da1/d11d/c13e 0 2024-04-25T00:18:37.143 INFO:tasks.workunit.client.0.smithi107.stdout:9/932: mknod d0/d8a/d93/d56/d6c/d80/d9b/db2/dea/d10c/d137/c143 0 2024-04-25T00:18:37.145 INFO:tasks.workunit.client.0.smithi107.stdout:3/901: creat d9/d54/d5c/d10d/f127 x:0 0 0 2024-04-25T00:18:37.146 INFO:tasks.workunit.client.0.smithi107.stdout:3/902: mkdir d9/d88/d3a/d91/dad/d128 0 2024-04-25T00:18:37.147 INFO:tasks.workunit.client.0.smithi107.stdout:8/933: getdents d2/d18/d1a/d5e/d3d/d85/dae 0 2024-04-25T00:18:37.148 INFO:tasks.workunit.client.0.smithi107.stdout:0/794: write dc/dd/d5d/f87 [538680,66379] 0 2024-04-25T00:18:37.149 INFO:tasks.workunit.client.0.smithi107.stdout:9/933: creat d0/d8a/d93/d34/d132/f144 x:0 0 0 2024-04-25T00:18:37.150 INFO:tasks.workunit.client.0.smithi107.stdout:3/903: mkdir d9/d15/d39/d76/d129 0 2024-04-25T00:18:37.151 INFO:tasks.workunit.client.0.smithi107.stdout:9/934: symlink d0/d8a/d93/d56/d6c/daa/l145 0 2024-04-25T00:18:37.152 INFO:tasks.workunit.client.0.smithi107.stdout:3/904: mkdir d9/d88/d80/d115/d12a 0 2024-04-25T00:18:37.153 INFO:tasks.workunit.client.0.smithi107.stdout:9/935: creat d0/d8a/d93/dc2/dc6/f146 x:0 0 0 2024-04-25T00:18:37.162 INFO:tasks.workunit.client.0.smithi107.stdout:2/922: dwrite d6/d18/d40/d42/d70/ffb [0,4194304] 0 2024-04-25T00:18:37.165 INFO:tasks.workunit.client.0.smithi107.stdout:2/923: truncate d6/d20/d22/d56/d94/d10b/d128/f9b 2181797 0 2024-04-25T00:18:37.165 INFO:tasks.workunit.client.0.smithi107.stdout:2/924: truncate d6/d18/d40/d42/d69/d6e/db6/f12d 831092 0 2024-04-25T00:18:37.165 INFO:tasks.workunit.client.0.smithi107.stdout:2/925: stat d6/d29/d4e/l50 0 2024-04-25T00:18:37.165 INFO:tasks.workunit.client.0.smithi107.stdout:2/926: truncate d6/d18/d40/d42/d69/d6e/db6/ffd 765724 0 2024-04-25T00:18:37.166 INFO:tasks.workunit.client.0.smithi107.stdout:2/927: mknod d6/d29/c136 0 2024-04-25T00:18:37.167 INFO:tasks.workunit.client.0.smithi107.stdout:2/928: mkdir d6/d18/d40/d42/d69/d6e/da2/dc5/dd2/d137 0 2024-04-25T00:18:37.167 INFO:tasks.workunit.client.0.smithi107.stdout:9/936: dread d0/d8a/d93/d56/d6c/d80/f104 [0,4194304] 0 2024-04-25T00:18:37.170 INFO:tasks.workunit.client.0.smithi107.stdout:2/929: rename d6/d18/d40/d42/d2b/d30/d45/c46 to d6/d18/d40/d42/d69/d6e/da2/d10c/d102/d9f/c138 0 2024-04-25T00:18:37.171 INFO:tasks.workunit.client.0.smithi107.stdout:2/930: truncate d6/dad/fd5 402336 0 2024-04-25T00:18:37.172 INFO:tasks.workunit.client.0.smithi107.stdout:2/931: unlink d6/d18/d40/d42/d2b/d30/d4c/c6b 0 2024-04-25T00:18:37.173 INFO:tasks.workunit.client.0.smithi107.stdout:8/934: link d2/d18/d1a/d5e/d6d/c79 d2/d18/d1a/d5e/d4f/da0/da1/db9/c143 0 2024-04-25T00:18:37.173 INFO:tasks.workunit.client.0.smithi107.stdout:8/935: chown d2/d18/d1a/d5e/d3e/db1/fd0 297138 1 2024-04-25T00:18:37.173 INFO:tasks.workunit.client.0.smithi107.stdout:2/932: mkdir d6/d18/d139 0 2024-04-25T00:18:37.173 INFO:tasks.workunit.client.0.smithi107.stdout:2/933: dread - d6/d18/d40/d42/d69/d6e/da2/dc5/fd8 zero size 2024-04-25T00:18:37.173 INFO:tasks.workunit.client.0.smithi107.stdout:2/934: truncate d6/d18/d40/fe8 1066781 0 2024-04-25T00:18:37.174 INFO:tasks.workunit.client.0.smithi107.stdout:2/935: write d6/d18/f78 [3332454,109376] 0 2024-04-25T00:18:37.174 INFO:tasks.workunit.client.0.smithi107.stdout:8/936: rename d2/d18/d1a/d5e/d4f/da0/fac to d2/d18/d1a/d82/f144 0 2024-04-25T00:18:37.175 INFO:tasks.workunit.client.0.smithi107.stdout:2/936: mknod d6/d20/d22/d56/d94/c13a 0 2024-04-25T00:18:37.176 INFO:tasks.workunit.client.0.smithi107.stdout:8/937: mknod d2/d18/d1a/d5e/d5c/d130/c145 0 2024-04-25T00:18:37.177 INFO:tasks.workunit.client.0.smithi107.stdout:8/938: mkdir d2/d18/d1a/d5e/d4f/da0/da1/db9/d146 0 2024-04-25T00:18:37.177 INFO:tasks.workunit.client.0.smithi107.stdout:8/939: chown d2/c2e 29785324 1 2024-04-25T00:18:37.185 INFO:tasks.workunit.client.0.smithi107.stdout:0/795: dread dc/d24/d38/d64/d7b/ffa [0,4194304] 0 2024-04-25T00:18:37.188 INFO:tasks.workunit.client.0.smithi107.stdout:4/900: dwrite d3/d5/f12f [0,4194304] 0 2024-04-25T00:18:37.189 INFO:tasks.workunit.client.0.smithi107.stdout:3/905: write d9/f20 [606621,84686] 0 2024-04-25T00:18:37.190 INFO:tasks.workunit.client.0.smithi107.stdout:9/937: dwrite d0/d8a/d93/f45 [0,4194304] 0 2024-04-25T00:18:37.190 INFO:tasks.workunit.client.0.smithi107.stdout:9/938: chown d0/d8a/d93/de/fcc 397253959 1 2024-04-25T00:18:37.192 INFO:tasks.workunit.client.0.smithi107.stdout:4/901: rmdir d3/da8 39 2024-04-25T00:18:37.196 INFO:tasks.workunit.client.0.smithi107.stdout:2/937: dwrite d6/dad/f85 [0,4194304] 0 2024-04-25T00:18:37.197 INFO:tasks.workunit.client.0.smithi107.stdout:2/938: creat d6/d18/d40/d42/d2b/f13b x:0 0 0 2024-04-25T00:18:37.198 INFO:tasks.workunit.client.0.smithi107.stdout:8/940: rename d2/d18/d1a/d82/f8b to d2/d18/d1a/d5e/d4f/da0/da1/db9/d146/f147 0 2024-04-25T00:18:37.200 INFO:tasks.workunit.client.0.smithi107.stdout:8/941: creat d2/d18/d1a/d5e/d3d/d85/f148 x:0 0 0 2024-04-25T00:18:37.204 INFO:tasks.workunit.client.0.smithi107.stdout:0/796: rename dc/d24/d25/d31/fb5 to dc/d24/d38/d8b/dd7/ffd 0 2024-04-25T00:18:37.205 INFO:tasks.workunit.client.0.smithi107.stdout:9/939: rename d0/d8a/d93/d56 to d0/d8a/d93/d56/d6c/d80/d9b/db2/d147 22 2024-04-25T00:18:37.206 INFO:tasks.workunit.client.0.smithi107.stdout:0/797: truncate dc/d24/d25/d31/f36 1489442 0 2024-04-25T00:18:37.207 INFO:tasks.workunit.client.0.smithi107.stdout:4/902: rename d3/d5/d32/d53/d7e/d9d to d3/d24/db1/d126/d132 0 2024-04-25T00:18:37.207 INFO:tasks.workunit.client.0.smithi107.stdout:4/903: truncate d3/d10/d7b/ffd 603517 0 2024-04-25T00:18:37.207 INFO:tasks.workunit.client.0.smithi107.stdout:9/940: rmdir d0/d8a/d93/d34/d132 39 2024-04-25T00:18:37.207 INFO:tasks.workunit.client.0.smithi107.stdout:9/941: chown d0/d8a/d93/d56/d6c/d80/f136 862 1 2024-04-25T00:18:37.207 INFO:tasks.workunit.client.0.smithi107.stdout:4/904: fsync d3/ff8 0 2024-04-25T00:18:37.208 INFO:tasks.workunit.client.0.smithi107.stdout:3/906: dwrite d9/d15/da1/d4b/fcd [0,4194304] 0 2024-04-25T00:18:37.208 INFO:tasks.workunit.client.0.smithi107.stdout:3/907: dread - d9/d15/da1/f105 zero size 2024-04-25T00:18:37.208 INFO:tasks.workunit.client.0.smithi107.stdout:3/908: dread - d9/d88/d3a/f108 zero size 2024-04-25T00:18:37.208 INFO:tasks.workunit.client.0.smithi107.stdout:3/909: chown d9/d54/d5c/d10d 21861879 1 2024-04-25T00:18:37.208 INFO:tasks.workunit.client.0.smithi107.stdout:3/910: truncate d9/d88/d2c/fca 107150 0 2024-04-25T00:18:37.209 INFO:tasks.workunit.client.0.smithi107.stdout:0/798: mkdir dc/d24/d38/d64/d7b/dfe 0 2024-04-25T00:18:37.209 INFO:tasks.workunit.client.0.smithi107.stdout:9/942: creat d0/d8a/d93/d56/d76/d111/f148 x:0 0 0 2024-04-25T00:18:37.212 INFO:tasks.workunit.client.0.smithi107.stdout:3/911: mknod d9/d54/d5c/d68/db3/c12b 0 2024-04-25T00:18:37.213 INFO:tasks.workunit.client.0.smithi107.stdout:0/799: mknod dc/d24/d38/d64/cff 0 2024-04-25T00:18:37.214 INFO:tasks.workunit.client.0.smithi107.stdout:3/912: creat d9/d22/d1b/dbd/f12c x:0 0 0 2024-04-25T00:18:37.215 INFO:tasks.workunit.client.0.smithi107.stdout:8/942: write d2/d18/d1a/d5e/d3d/d85/f101 [3308279,79630] 0 2024-04-25T00:18:37.216 INFO:tasks.workunit.client.0.smithi107.stdout:0/800: mknod dc/d24/d25/c100 0 2024-04-25T00:18:37.216 INFO:tasks.workunit.client.0.smithi107.stdout:0/801: read - dc/d24/d38/d64/d78/ddc/ff9 zero size 2024-04-25T00:18:37.220 INFO:tasks.workunit.client.0.smithi107.stdout:8/943: symlink d2/d18/d1a/dca/l149 0 2024-04-25T00:18:37.221 INFO:tasks.workunit.client.0.smithi107.stdout:2/939: dwrite d6/f1e [0,4194304] 0 2024-04-25T00:18:37.222 INFO:tasks.workunit.client.0.smithi107.stdout:4/905: dread d3/d5/f7 [0,4194304] 0 2024-04-25T00:18:37.223 INFO:tasks.workunit.client.0.smithi107.stdout:4/906: truncate d3/d24/d25/d67/fcc 1983645 0 2024-04-25T00:18:37.223 INFO:tasks.workunit.client.0.smithi107.stdout:2/940: truncate d6/d20/d22/d56/d94/d10b/d128/f9b 1605914 0 2024-04-25T00:18:37.223 INFO:tasks.workunit.client.0.smithi107.stdout:2/941: truncate d6/d18/d40/d42/d69/fb3 4245946 0 2024-04-25T00:18:37.225 INFO:tasks.workunit.client.0.smithi107.stdout:4/907: creat d3/d24/d25/d71/ddb/f133 x:0 0 0 2024-04-25T00:18:37.227 INFO:tasks.workunit.client.0.smithi107.stdout:2/942: rename d6/d18/d40/d42/d2b/f7a to d6/d18/d40/d42/d69/d6e/dc2/d134/f13c 0 2024-04-25T00:18:37.227 INFO:tasks.workunit.client.0.smithi107.stdout:4/908: readlink d3/d5/l76 0 2024-04-25T00:18:37.231 INFO:tasks.workunit.client.0.smithi107.stdout:2/943: rename d6/d18/d40/d42/d2b/d30/d4c/f135 to d6/d20/d22/d56/d94/d10b/d118/f13d 0 2024-04-25T00:18:37.232 INFO:tasks.workunit.client.0.smithi107.stdout:4/909: creat d3/f134 x:0 0 0 2024-04-25T00:18:37.235 INFO:tasks.workunit.client.0.smithi107.stdout:2/944: write d6/d18/d40/d42/d2b/d30/f4b [1186062,37134] 0 2024-04-25T00:18:37.236 INFO:tasks.workunit.client.0.smithi107.stdout:3/913: dread d9/d88/d2c/fbe [0,4194304] 0 2024-04-25T00:18:37.236 INFO:tasks.workunit.client.0.smithi107.stdout:3/914: truncate d9/d22/d1b/d55/f5d 3034037 0 2024-04-25T00:18:37.236 INFO:tasks.workunit.client.0.smithi107.stdout:4/910: unlink d3/d24/d25/d71/cf5 0 2024-04-25T00:18:37.237 INFO:tasks.workunit.client.0.smithi107.stdout:3/915: truncate d9/d88/d2c/f37 3967754 0 2024-04-25T00:18:37.238 INFO:tasks.workunit.client.0.smithi107.stdout:2/945: unlink d6/d20/d22/f3a 0 2024-04-25T00:18:37.238 INFO:tasks.workunit.client.0.smithi107.stdout:2/946: chown d6/dad/d67/l54 241898960 1 2024-04-25T00:18:37.239 INFO:tasks.workunit.client.0.smithi107.stdout:3/916: mknod d9/d88/d80/d115/d12a/c12d 0 2024-04-25T00:18:37.240 INFO:tasks.workunit.client.0.smithi107.stdout:9/943: dwrite d0/d8a/d93/d56/d76/d50/d7d/fbc [0,4194304] 0 2024-04-25T00:18:37.241 INFO:tasks.workunit.client.0.smithi107.stdout:8/944: dread d2/d18/f20 [0,4194304] 0 2024-04-25T00:18:37.243 INFO:tasks.workunit.client.0.smithi107.stdout:2/947: rename d6/d18/d40/c8e to d6/d18/d40/d42/d69/d6e/da2/dc5/dd2/d100/c13e 0 2024-04-25T00:18:37.245 INFO:tasks.workunit.client.0.smithi107.stdout:3/917: dread d9/d88/d4e/fd7 [0,4194304] 0 2024-04-25T00:18:37.245 INFO:tasks.workunit.client.0.smithi107.stdout:3/918: truncate d9/d88/d3a/f108 367080 0 2024-04-25T00:18:37.246 INFO:tasks.workunit.client.0.smithi107.stdout:0/802: dwrite dc/d24/d38/d64/d7b/fae [0,4194304] 0 2024-04-25T00:18:37.260 INFO:tasks.workunit.client.0.smithi107.stdout:3/919: mkdir d9/d54/d5c/d10d/d12e 0 2024-04-25T00:18:37.261 INFO:tasks.workunit.client.0.smithi107.stdout:3/920: mknod d9/d88/daa/c12f 0 2024-04-25T00:18:37.261 INFO:tasks.workunit.client.0.smithi107.stdout:3/921: readlink d9/d54/d5c/d68/d3b/l45 0 2024-04-25T00:18:37.262 INFO:tasks.workunit.client.0.smithi107.stdout:0/803: creat dc/d24/d38/d64/d7b/f101 x:0 0 0 2024-04-25T00:18:37.264 INFO:tasks.workunit.client.0.smithi107.stdout:3/922: symlink d9/d22/d1b/dbd/l130 0 2024-04-25T00:18:37.264 INFO:tasks.workunit.client.0.smithi107.stdout:2/948: dwrite d6/d18/fb5 [0,4194304] 0 2024-04-25T00:18:37.264 INFO:tasks.workunit.client.0.smithi107.stdout:3/923: fsync d9/d88/d4e/f70 0 2024-04-25T00:18:37.266 INFO:tasks.workunit.client.0.smithi107.stdout:3/924: mkdir d9/d88/d80/d115/d131 0 2024-04-25T00:18:37.266 INFO:tasks.workunit.client.0.smithi107.stdout:3/925: readlink d9/d88/d3a/la6 0 2024-04-25T00:18:37.266 INFO:tasks.workunit.client.0.smithi107.stdout:2/949: mkdir d6/d20/d13f 0 2024-04-25T00:18:37.267 INFO:tasks.workunit.client.0.smithi107.stdout:2/950: link d6/d20/c8a d6/d18/d40/d42/d69/c140 0 2024-04-25T00:18:37.268 INFO:tasks.workunit.client.0.smithi107.stdout:3/926: mknod d9/d88/d80/d115/c132 0 2024-04-25T00:18:37.271 INFO:tasks.workunit.client.0.smithi107.stdout:3/927: unlink d9/d54/d5c/d68/d3b/cde 0 2024-04-25T00:18:37.272 INFO:tasks.workunit.client.0.smithi107.stdout:2/951: link d6/d18/d40/d42/d69/d6e/da2/dc5/ff2 d6/d20/d22/d56/d94/d10b/f141 0 2024-04-25T00:18:37.272 INFO:tasks.workunit.client.0.smithi107.stdout:3/928: getdents d9/d54/d94/dab 0 2024-04-25T00:18:37.272 INFO:tasks.workunit.client.0.smithi107.stdout:3/929: stat d9/d88/d3a/l101 0 2024-04-25T00:18:37.273 INFO:tasks.workunit.client.0.smithi107.stdout:9/944: dwrite d0/d8a/d93/d34/d36/d77/dac/fdf [0,4194304] 0 2024-04-25T00:18:37.274 INFO:tasks.workunit.client.0.smithi107.stdout:4/911: write d3/d24/d25/d71/f7c [723972,93950] 0 2024-04-25T00:18:37.274 INFO:tasks.workunit.client.0.smithi107.stdout:9/945: write d0/d8a/d93/d56/f64 [3512084,42458] 0 2024-04-25T00:18:37.274 INFO:tasks.workunit.client.0.smithi107.stdout:9/946: read d0/d8a/d93/d1f/fb6 [780839,41562] 0 2024-04-25T00:18:37.274 INFO:tasks.workunit.client.0.smithi107.stdout:8/945: dwrite d2/d18/d1a/d5e/d4f/f7e [0,4194304] 0 2024-04-25T00:18:37.275 INFO:tasks.workunit.client.0.smithi107.stdout:2/952: rename d6/d18/d40/d42/d69/d6e/da2/d10c/d102/d9f/dfe to d6/d18/d40/d42/d2b/d30/d127/d142 0 2024-04-25T00:18:37.275 INFO:tasks.workunit.client.0.smithi107.stdout:4/912: mknod d3/d5/d32/d53/d97/c135 0 2024-04-25T00:18:37.276 INFO:tasks.workunit.client.0.smithi107.stdout:3/930: creat d9/d15/d39/f133 x:0 0 0 2024-04-25T00:18:37.279 INFO:tasks.workunit.client.0.smithi107.stdout:9/947: unlink d0/d8a/d93/d56/d6c/daa/lb7 0 2024-04-25T00:18:37.280 INFO:tasks.workunit.client.0.smithi107.stdout:0/804: dwrite dc/dd/d16/d33/d35/ffc [0,4194304] 0 2024-04-25T00:18:37.280 INFO:tasks.workunit.client.0.smithi107.stdout:0/805: fsync dc/f91 0 2024-04-25T00:18:37.281 INFO:tasks.workunit.client.0.smithi107.stdout:8/946: creat d2/d18/d1a/d5e/d5c/d91/de7/f14a x:0 0 0 2024-04-25T00:18:37.282 INFO:tasks.workunit.client.0.smithi107.stdout:8/947: dread - d2/d18/d1a/d5e/d3e/db1/f11c zero size 2024-04-25T00:18:37.282 INFO:tasks.workunit.client.0.smithi107.stdout:8/948: write d2/d18/d1a/d5e/d4f/da0/ff1 [848840,97858] 0 2024-04-25T00:18:37.282 INFO:tasks.workunit.client.0.smithi107.stdout:8/949: readlink d2/d18/d1a/d5e/d4f/da0/da1/db9/lcd 0 2024-04-25T00:18:37.282 INFO:tasks.workunit.client.0.smithi107.stdout:8/950: dread - d2/d18/ddd/fdf zero size 2024-04-25T00:18:37.284 INFO:tasks.workunit.client.0.smithi107.stdout:3/931: dread d9/d22/d1b/f48 [0,4194304] 0 2024-04-25T00:18:37.285 INFO:tasks.workunit.client.0.smithi107.stdout:4/913: dread d3/d24/d25/d71/fd5 [0,4194304] 0 2024-04-25T00:18:37.287 INFO:tasks.workunit.client.0.smithi107.stdout:4/914: creat d3/d24/d25/d71/dc2/f136 x:0 0 0 2024-04-25T00:18:37.293 INFO:tasks.workunit.client.0.smithi107.stdout:0/806: creat dc/d24/d38/d8b/dd7/de1/f102 x:0 0 0 2024-04-25T00:18:37.296 INFO:tasks.workunit.client.0.smithi107.stdout:0/807: mkdir dc/dd/d5d/d103 0 2024-04-25T00:18:37.298 INFO:tasks.workunit.client.0.smithi107.stdout:8/951: dread d2/f83 [0,4194304] 0 2024-04-25T00:18:37.302 INFO:tasks.workunit.client.0.smithi107.stdout:3/932: write d9/d22/d1b/f81 [5338959,29835] 0 2024-04-25T00:18:37.303 INFO:tasks.workunit.client.0.smithi107.stdout:3/933: creat d9/d22/d1b/dbd/f134 x:0 0 0 2024-04-25T00:18:37.304 INFO:tasks.workunit.client.0.smithi107.stdout:9/948: dwrite d0/f40 [4194304,4194304] 0 2024-04-25T00:18:37.305 INFO:tasks.workunit.client.0.smithi107.stdout:3/934: creat d9/d15/d39/d76/d129/f135 x:0 0 0 2024-04-25T00:18:37.306 INFO:tasks.workunit.client.0.smithi107.stdout:3/935: read - d9/d15/da1/d4b/dd0/dd8/fe7 zero size 2024-04-25T00:18:37.311 INFO:tasks.workunit.client.0.smithi107.stdout:6/715: sync 2024-04-25T00:18:37.313 INFO:tasks.workunit.client.0.smithi107.stdout:4/915: rename d3/d24/d5c/l12a to d3/d5/d32/d9e/d111/l137 0 2024-04-25T00:18:37.315 INFO:tasks.workunit.client.0.smithi107.stdout:3/936: rename d9/d15/da1/d4b/fcd to d9/d22/d11c/f136 0 2024-04-25T00:18:37.319 INFO:tasks.workunit.client.0.smithi107.stdout:0/808: dwrite dc/dd/fc9 [0,4194304] 0 2024-04-25T00:18:37.319 INFO:tasks.workunit.client.0.smithi107.stdout:8/952: dwrite d2/d18/d1a/d5e/f8c [0,4194304] 0 2024-04-25T00:18:37.323 INFO:tasks.workunit.client.0.smithi107.stdout:0/809: mknod dc/d24/d38/d8b/dd7/de1/c104 0 2024-04-25T00:18:37.324 INFO:tasks.workunit.client.0.smithi107.stdout:0/810: symlink dc/dd/d16/dac/l105 0 2024-04-25T00:18:37.324 INFO:tasks.workunit.client.0.smithi107.stdout:4/916: write d3/d5/f12f [564848,29555] 0 2024-04-25T00:18:37.327 INFO:tasks.workunit.client.0.smithi107.stdout:4/917: truncate d3/d10/f10d 600323 0 2024-04-25T00:18:37.328 INFO:tasks.workunit.client.0.smithi107.stdout:4/918: mknod d3/d5/d32/d9e/d111/c138 0 2024-04-25T00:18:37.330 INFO:tasks.workunit.client.0.smithi107.stdout:6/716: link d1/d14/de6/db/d32/d36/fd8 d1/fe9 0 2024-04-25T00:18:37.332 INFO:tasks.workunit.client.0.smithi107.stdout:6/717: creat d1/d14/de6/db/d32/d33/d4f/fea x:0 0 0 2024-04-25T00:18:37.333 INFO:tasks.workunit.client.0.smithi107.stdout:4/919: dread d3/d24/d5c/fd1 [0,4194304] 0 2024-04-25T00:18:37.334 INFO:tasks.workunit.client.0.smithi107.stdout:6/718: symlink d1/d14/de6/d3/d28/leb 0 2024-04-25T00:18:37.335 INFO:tasks.workunit.client.0.smithi107.stdout:6/719: rmdir d1/d14/de6/db/d3c/d9c 39 2024-04-25T00:18:37.335 INFO:tasks.workunit.client.0.smithi107.stdout:8/953: dread d2/d18/d1a/d5e/d4f/da0/da1/fad [0,4194304] 0 2024-04-25T00:18:37.337 INFO:tasks.workunit.client.0.smithi107.stdout:8/954: creat d2/d18/d1a/d5e/d5c/d130/f14b x:0 0 0 2024-04-25T00:18:37.338 INFO:tasks.workunit.client.0.smithi107.stdout:8/955: mkdir d2/d14c 0 2024-04-25T00:18:37.340 INFO:tasks.workunit.client.0.smithi107.stdout:8/956: read d2/d18/d1a/dca/f12d [200079,16990] 0 2024-04-25T00:18:37.340 INFO:tasks.workunit.client.0.smithi107.stdout:8/957: dread - d2/d18/d1a/d5e/d3e/db1/ffa zero size 2024-04-25T00:18:37.341 INFO:tasks.workunit.client.0.smithi107.stdout:3/937: dwrite d9/d54/d94/f103 [0,4194304] 0 2024-04-25T00:18:37.341 INFO:tasks.workunit.client.0.smithi107.stdout:8/958: write d2/d18/d1a/dca/f126 [8761247,129452] 0 2024-04-25T00:18:37.341 INFO:tasks.workunit.client.0.smithi107.stdout:8/959: chown d2/d18/d1a/c120 0 1 2024-04-25T00:18:37.342 INFO:tasks.workunit.client.0.smithi107.stdout:0/811: dread dc/dd/f4c [0,4194304] 0 2024-04-25T00:18:37.342 INFO:tasks.workunit.client.0.smithi107.stdout:8/960: stat d2/d18/d1a/c140 0 2024-04-25T00:18:37.346 INFO:tasks.workunit.client.0.smithi107.stdout:0/812: link dc/dd/d16/d33/d35/d5f/ld2 dc/dd/d16/d48/d83/d9f/dc4/l106 0 2024-04-25T00:18:37.349 INFO:tasks.workunit.client.0.smithi107.stdout:0/813: mknod dc/dd/d5d/d3f/d59/c107 0 2024-04-25T00:18:37.350 INFO:tasks.workunit.client.0.smithi107.stdout:0/814: symlink dc/dd/d5d/l108 0 2024-04-25T00:18:37.351 INFO:tasks.workunit.client.0.smithi107.stdout:8/961: dread d2/d18/d1a/d5e/d4f/da0/da1/db9/f11d [0,4194304] 0 2024-04-25T00:18:37.352 INFO:tasks.workunit.client.0.smithi107.stdout:8/962: symlink d2/d18/d1a/d5e/d4f/l14d 0 2024-04-25T00:18:37.352 INFO:tasks.workunit.client.0.smithi107.stdout:8/963: readlink d2/d18/d1a/d5e/d3d/l98 0 2024-04-25T00:18:37.353 INFO:tasks.workunit.client.0.smithi107.stdout:8/964: write d2/d18/d1a/d5e/d4f/da0/f123 [468970,31099] 0 2024-04-25T00:18:37.357 INFO:tasks.workunit.client.0.smithi107.stdout:6/720: dwrite d1/d14/de6/d3/d26/fa2 [4194304,4194304] 0 2024-04-25T00:18:37.358 INFO:tasks.workunit.client.0.smithi107.stdout:4/920: dwrite d3/d10/f8b [0,4194304] 0 2024-04-25T00:18:37.365 INFO:tasks.workunit.client.0.smithi107.stdout:8/965: dread d2/d18/d1a/f62 [0,4194304] 0 2024-04-25T00:18:37.365 INFO:tasks.workunit.client.0.smithi107.stdout:8/966: write d2/d18/d1a/d5e/d5c/d91/de7/f10b [11285,83887] 0 2024-04-25T00:18:37.365 INFO:tasks.workunit.client.0.smithi107.stdout:8/967: fdatasync d2/d18/d1a/f3a 0 2024-04-25T00:18:37.365 INFO:tasks.workunit.client.0.smithi107.stdout:3/938: rename d9/d88/d3a/fb1 to d9/d54/d94/dba/dcc/f137 0 2024-04-25T00:18:37.366 INFO:tasks.workunit.client.0.smithi107.stdout:3/939: creat d9/d88/daa/f138 x:0 0 0 2024-04-25T00:18:37.367 INFO:tasks.workunit.client.0.smithi107.stdout:3/940: creat d9/d88/d80/d115/f139 x:0 0 0 2024-04-25T00:18:37.368 INFO:tasks.workunit.client.0.smithi107.stdout:3/941: dread d9/d54/d94/dba/dcc/f137 [0,4194304] 0 2024-04-25T00:18:37.370 INFO:tasks.workunit.client.0.smithi107.stdout:4/921: link d3/d24/d5c/fd1 d3/d24/d25/d71/f139 0 2024-04-25T00:18:37.371 INFO:tasks.workunit.client.0.smithi107.stdout:6/721: write d1/d14/de6/d3/f13 [2917624,130187] 0 2024-04-25T00:18:37.371 INFO:tasks.workunit.client.0.smithi107.stdout:6/722: fdatasync d1/d14/de6/db/f4d 0 2024-04-25T00:18:37.372 INFO:tasks.workunit.client.0.smithi107.stdout:3/942: creat d9/d54/d94/dba/f13a x:0 0 0 2024-04-25T00:18:37.372 INFO:tasks.workunit.client.0.smithi107.stdout:4/922: creat d3/d5/d32/d9e/da2/f13a x:0 0 0 2024-04-25T00:18:37.373 INFO:tasks.workunit.client.0.smithi107.stdout:0/815: rename dc/d24/d38/d8b/dd7/ffd to dc/dd/d5d/d5c/f109 0 2024-04-25T00:18:37.376 INFO:tasks.workunit.client.0.smithi107.stdout:0/816: mkdir dc/dd/d16/d33/d35/d10a 0 2024-04-25T00:18:37.376 INFO:tasks.workunit.client.0.smithi107.stdout:0/817: chown dc/dd/d16/d48/d83/d9f/dc4 237 1 2024-04-25T00:18:37.381 INFO:tasks.workunit.client.0.smithi107.stdout:0/818: creat dc/d24/d25/ddd/f10b x:0 0 0 2024-04-25T00:18:37.382 INFO:tasks.workunit.client.0.smithi107.stdout:0/819: mkdir dc/d24/d25/d10c 0 2024-04-25T00:18:37.384 INFO:tasks.workunit.client.0.smithi107.stdout:0/820: truncate dc/dd/d5d/d5c/f7f 164359 0 2024-04-25T00:18:37.384 INFO:tasks.workunit.client.0.smithi107.stdout:0/821: dread - dc/f91 zero size 2024-04-25T00:18:37.384 INFO:tasks.workunit.client.0.smithi107.stdout:0/822: fsync dc/f28 0 2024-04-25T00:18:37.385 INFO:tasks.workunit.client.0.smithi107.stdout:8/968: dwrite d2/d18/d1a/d5e/d5c/d107/db2/f124 [0,4194304] 0 2024-04-25T00:18:37.385 INFO:tasks.workunit.client.0.smithi107.stdout:8/969: chown d2/d18/d1a/d5e/d6d/c93 0 1 2024-04-25T00:18:37.385 INFO:tasks.workunit.client.0.smithi107.stdout:8/970: getdents d2/d18/d1a/d5e/d5c/d107/db2/d125 0 2024-04-25T00:18:37.385 INFO:tasks.workunit.client.0.smithi107.stdout:4/923: read d3/d5/d32/f38 [1081654,112085] 0 2024-04-25T00:18:37.385 INFO:tasks.workunit.client.0.smithi107.stdout:4/924: write d3/d5/d32/f79 [1263489,56336] 0 2024-04-25T00:18:37.385 INFO:tasks.workunit.client.0.smithi107.stdout:4/925: readlink d3/da8/ld9 0 2024-04-25T00:18:37.387 INFO:tasks.workunit.client.0.smithi107.stdout:4/926: symlink d3/d24/d25/d71/l13b 0 2024-04-25T00:18:37.387 INFO:tasks.workunit.client.0.smithi107.stdout:4/927: readlink d3/da8/ld3 0 2024-04-25T00:18:37.387 INFO:tasks.workunit.client.0.smithi107.stdout:4/928: chown d3/d24/d25/d67/lcf 462718 1 2024-04-25T00:18:37.387 INFO:tasks.workunit.client.0.smithi107.stdout:4/929: chown d3/d24/d5c/dab/f7d 2787456 1 2024-04-25T00:18:37.387 INFO:tasks.workunit.client.0.smithi107.stdout:4/930: write d3/d5/d32/f9b [5037007,66362] 0 2024-04-25T00:18:37.387 INFO:tasks.workunit.client.0.smithi107.stdout:4/931: fdatasync d3/d5/f65 0 2024-04-25T00:18:37.387 INFO:tasks.workunit.client.0.smithi107.stdout:0/823: rmdir dc/dd/d5d/d5c 39 2024-04-25T00:18:37.388 INFO:tasks.workunit.client.0.smithi107.stdout:0/824: truncate dc/dd/d5d/d7e/f85 844500 0 2024-04-25T00:18:37.390 INFO:tasks.workunit.client.0.smithi107.stdout:0/825: creat dc/dd/d5d/d5c/d6e/f10d x:0 0 0 2024-04-25T00:18:37.392 INFO:tasks.workunit.client.0.smithi107.stdout:3/943: dwrite d9/d54/d5c/fce [0,4194304] 0 2024-04-25T00:18:37.396 INFO:tasks.workunit.client.0.smithi107.stdout:8/971: dread d2/d18/d1a/d5e/d3d/fc4 [4194304,4194304] 0 2024-04-25T00:18:37.404 INFO:tasks.workunit.client.0.smithi107.stdout:3/944: dread d9/d54/d5c/d68/f3c [0,4194304] 0 2024-04-25T00:18:37.404 INFO:tasks.workunit.client.0.smithi107.stdout:3/945: chown d9/f9b 78 1 2024-04-25T00:18:37.406 INFO:tasks.workunit.client.0.smithi107.stdout:3/946: creat d9/d54/d94/dba/f13b x:0 0 0 2024-04-25T00:18:37.406 INFO:tasks.workunit.client.0.smithi107.stdout:4/932: dwrite d3/d5/d32/d53/d97/f99 [0,4194304] 0 2024-04-25T00:18:37.408 INFO:tasks.workunit.client.0.smithi107.stdout:8/972: chown d2/d18/d1a/d5e/dd6/de6 1249 1 2024-04-25T00:18:37.408 INFO:tasks.workunit.client.0.smithi107.stdout:8/973: write d2/d18/d1a/d128/f12c [718303,60719] 0 2024-04-25T00:18:37.408 INFO:tasks.workunit.client.0.smithi107.stdout:8/974: dread - d2/d18/d1a/d5e/d3e/db1/f12e zero size 2024-04-25T00:18:37.414 INFO:tasks.workunit.client.0.smithi107.stdout:0/826: dwrite dc/dd/d16/d48/f6a [0,4194304] 0 2024-04-25T00:18:37.414 INFO:tasks.workunit.client.0.smithi107.stdout:0/827: write dc/dd/d16/d33/fb9 [952193,37810] 0 2024-04-25T00:18:37.414 INFO:tasks.workunit.client.0.smithi107.stdout:0/828: readlink dc/dd/d16/d33/d35/d5f/ld2 0 2024-04-25T00:18:37.414 INFO:tasks.workunit.client.0.smithi107.stdout:0/829: fsync dc/da6/fd6 0 2024-04-25T00:18:37.429 INFO:tasks.workunit.client.0.smithi107.stdout:8/975: dwrite d2/d18/d1a/d33/f64 [0,4194304] 0 2024-04-25T00:18:37.430 INFO:tasks.workunit.client.0.smithi107.stdout:8/976: write d2/d18/d1a/d5e/d3e/faa [1856522,34177] 0 2024-04-25T00:18:37.430 INFO:tasks.workunit.client.0.smithi107.stdout:8/977: write d2/d18/d1a/d128/f12c [1572264,44865] 0 2024-04-25T00:18:37.430 INFO:tasks.workunit.client.0.smithi107.stdout:8/978: chown d2/d18/d1a/d5e/d5c/c9b 12264 1 2024-04-25T00:18:37.430 INFO:tasks.workunit.client.0.smithi107.stdout:0/830: dread dc/dd/fd8 [0,4194304] 0 2024-04-25T00:18:37.430 INFO:tasks.workunit.client.0.smithi107.stdout:0/831: chown dc/d24/d38/d64/d7b/fb7 41 1 2024-04-25T00:18:37.430 INFO:tasks.workunit.client.0.smithi107.stdout:4/933: dread d3/d24/d5c/d7a/fe7 [0,4194304] 0 2024-04-25T00:18:37.430 INFO:tasks.workunit.client.0.smithi107.stdout:3/947: dwrite d9/d88/d4e/f72 [0,4194304] 0 2024-04-25T00:18:37.431 INFO:tasks.workunit.client.0.smithi107.stdout:8/979: rmdir d2/d18/d1a/d5e/d3d/d85 39 2024-04-25T00:18:37.433 INFO:tasks.workunit.client.0.smithi107.stdout:4/934: mkdir d3/d24/d5c/dab/da3/d13c 0 2024-04-25T00:18:37.441 INFO:tasks.workunit.client.0.smithi107.stdout:4/935: write d3/d24/d5c/f98 [345308,107357] 0 2024-04-25T00:18:37.441 INFO:tasks.workunit.client.0.smithi107.stdout:4/936: dread - d3/d5/d32/ff4 zero size 2024-04-25T00:18:37.441 INFO:tasks.workunit.client.0.smithi107.stdout:4/937: write d3/d24/d5c/dab/da3/fa6 [1882957,7] 0 2024-04-25T00:18:37.449 INFO:tasks.workunit.client.0.smithi107.stdout:3/948: dwrite d9/d88/d2c/f102 [0,4194304] 0 2024-04-25T00:18:37.449 INFO:tasks.workunit.client.0.smithi107.stdout:3/949: chown d9/f42 2797 1 2024-04-25T00:18:37.455 INFO:tasks.workunit.client.0.smithi107.stdout:0/832: dread dc/d24/d25/f80 [0,4194304] 0 2024-04-25T00:18:37.457 INFO:tasks.workunit.client.0.smithi107.stdout:0/833: symlink dc/d24/d38/d64/l10e 0 2024-04-25T00:18:37.457 INFO:tasks.workunit.client.0.smithi107.stdout:0/834: chown dc/d24/d38 5572546 1 2024-04-25T00:18:37.466 INFO:tasks.workunit.client.0.smithi107.stdout:8/980: rename d2/d18/d1a/d5e/d3e/c108 to d2/d18/d1a/d5e/d4f/da0/da1/de8/d119/d137/c14e 0 2024-04-25T00:18:37.466 INFO:tasks.workunit.client.0.smithi107.stdout:8/981: read d2/d18/d1a/f25 [1278480,11316] 0 2024-04-25T00:18:37.467 INFO:tasks.workunit.client.0.smithi107.stdout:8/982: symlink d2/d18/d1a/l14f 0 2024-04-25T00:18:37.470 INFO:tasks.workunit.client.0.smithi107.stdout:3/950: dwrite d9/d15/da1/f119 [0,4194304] 0 2024-04-25T00:18:37.473 INFO:tasks.workunit.client.0.smithi107.stdout:3/951: truncate d9/f56 2383895 0 2024-04-25T00:18:37.473 INFO:tasks.workunit.client.0.smithi107.stdout:3/952: readlink d9/d88/d3a/ld4 0 2024-04-25T00:18:37.473 INFO:tasks.workunit.client.0.smithi107.stdout:0/835: dread dc/dd/d16/d33/d35/d5f/f82 [0,4194304] 0 2024-04-25T00:18:37.477 INFO:tasks.workunit.client.0.smithi107.stdout:5/816: sync 2024-04-25T00:18:37.477 INFO:tasks.workunit.client.0.smithi107.stdout:5/817: readlink d4/dc/d26/d29/d2e/lac 0 2024-04-25T00:18:37.479 INFO:tasks.workunit.client.0.smithi107.stdout:5/818: creat d4/d1c/d5c/dd4/f111 x:0 0 0 2024-04-25T00:18:37.479 INFO:tasks.workunit.client.0.smithi107.stdout:5/819: stat d4/fc0 0 2024-04-25T00:18:37.481 INFO:tasks.workunit.client.0.smithi107.stdout:3/953: getdents d9/d88 0 2024-04-25T00:18:37.484 INFO:tasks.workunit.client.0.smithi107.stdout:8/983: dwrite d2/d18/d1a/d5e/d3e/f100 [0,4194304] 0 2024-04-25T00:18:37.484 INFO:tasks.workunit.client.0.smithi107.stdout:8/984: chown d2/d18/d1a/f73 28296192 1 2024-04-25T00:18:37.484 INFO:tasks.workunit.client.0.smithi107.stdout:8/985: chown d2/cd 714752 1 2024-04-25T00:18:37.485 INFO:tasks.workunit.client.0.smithi107.stdout:0/836: write dc/d24/fe2 [238657,18578] 0 2024-04-25T00:18:37.485 INFO:tasks.workunit.client.0.smithi107.stdout:0/837: fsync dc/d24/d25/fec 0 2024-04-25T00:18:37.485 INFO:tasks.workunit.client.0.smithi107.stdout:0/838: chown dc/d24/d25/fec 29 1 2024-04-25T00:18:37.488 INFO:tasks.workunit.client.0.smithi107.stdout:8/986: creat d2/d18/d1a/d5e/d4f/da0/f150 x:0 0 0 2024-04-25T00:18:37.488 INFO:tasks.workunit.client.0.smithi107.stdout:8/987: write d2/d18/d1a/d5e/d5c/d91/de7/f104 [155591,13788] 0 2024-04-25T00:18:37.488 INFO:tasks.workunit.client.0.smithi107.stdout:8/988: chown d2/d18/d1a/d5e/d3e/faa 7 1 2024-04-25T00:18:37.492 INFO:tasks.workunit.client.0.smithi107.stdout:8/989: getdents d2/d18/d1a/d5e/d4f/da0 0 2024-04-25T00:18:37.493 INFO:tasks.workunit.client.0.smithi107.stdout:3/954: read d9/d15/da1/f119 [763169,86967] 0 2024-04-25T00:18:37.496 INFO:tasks.workunit.client.0.smithi107.stdout:5/820: write d4/dc/d83/dfc/f108 [1940623,97705] 0 2024-04-25T00:18:37.498 INFO:tasks.workunit.client.0.smithi107.stdout:5/821: creat d4/d17/db3/d84/d93/dc4/f112 x:0 0 0 2024-04-25T00:18:37.500 INFO:tasks.workunit.client.0.smithi107.stdout:5/822: rename d4/d17/db3/ca7 to d4/d7c/d88/d9d/deb/c113 0 2024-04-25T00:18:37.501 INFO:tasks.workunit.client.0.smithi107.stdout:5/823: creat d4/dc/f114 x:0 0 0 2024-04-25T00:18:37.501 INFO:tasks.workunit.client.0.smithi107.stdout:8/990: dread d2/d18/ddd/f121 [0,4194304] 0 2024-04-25T00:18:37.504 INFO:tasks.workunit.client.0.smithi107.stdout:5/824: rename d4/dc/d26/f8c to d4/d17/d104/f115 0 2024-04-25T00:18:37.515 INFO:tasks.workunit.client.0.smithi107.stdout:3/955: dwrite d9/d54/d5c/d68/fb8 [0,4194304] 0 2024-04-25T00:18:37.516 INFO:tasks.workunit.client.0.smithi107.stdout:0/839: dwrite dc/d24/d25/d31/f36 [0,4194304] 0 2024-04-25T00:18:37.520 INFO:tasks.workunit.client.0.smithi107.stdout:3/956: mkdir d9/d15/da1/d13c 0 2024-04-25T00:18:37.524 INFO:tasks.workunit.client.0.smithi107.stdout:5/825: dread d4/dc/d26/d29/d2e/f6d [0,4194304] 0 2024-04-25T00:18:37.526 INFO:tasks.workunit.client.0.smithi107.stdout:3/957: mknod d9/d54/d5c/c13d 0 2024-04-25T00:18:37.528 INFO:tasks.workunit.client.0.smithi107.stdout:3/958: mknod d9/d54/d5c/c13e 0 2024-04-25T00:18:37.528 INFO:tasks.workunit.client.0.smithi107.stdout:3/959: stat d9/d54/d94/dba/f13a 0 2024-04-25T00:18:37.532 INFO:tasks.workunit.client.0.smithi107.stdout:8/991: rename d2/d18/d1a/d5e/d5c/d91/da3 to d2/d18/d1a/d5e/d4f/da0/da1/d151 0 2024-04-25T00:18:37.532 INFO:tasks.workunit.client.0.smithi107.stdout:8/992: truncate d2/d18/d1a/d5e/d4f/da0/d110/f13e 465791 0 2024-04-25T00:18:37.534 INFO:tasks.workunit.client.0.smithi107.stdout:5/826: rename d4/dc/d26/d29 to d4/d17/db3/d116 0 2024-04-25T00:18:37.537 INFO:tasks.workunit.client.0.smithi107.stdout:3/960: write d9/d22/f7a [2974133,50161] 0 2024-04-25T00:18:37.538 INFO:tasks.workunit.client.0.smithi107.stdout:3/961: creat d9/d88/d3a/d91/dad/deb/f13f x:0 0 0 2024-04-25T00:18:37.539 INFO:tasks.workunit.client.0.smithi107.stdout:3/962: truncate d9/d15/da1/db0/fe1 539606 0 2024-04-25T00:18:37.539 INFO:tasks.workunit.client.0.smithi107.stdout:3/963: fsync d9/d15/d39/d76/d129/f135 0 2024-04-25T00:18:37.539 INFO:tasks.workunit.client.0.smithi107.stdout:3/964: mkdir d9/d22/d11c/d140 0 2024-04-25T00:18:37.541 INFO:tasks.workunit.client.0.smithi107.stdout:8/993: dread d2/d18/d1a/f8e [0,4194304] 0 2024-04-25T00:18:37.542 INFO:tasks.workunit.client.0.smithi107.stdout:5/827: dread d4/d17/db3/d116/d2e/d55/fcb [0,4194304] 0 2024-04-25T00:18:37.543 INFO:tasks.workunit.client.0.smithi107.stdout:5/828: mkdir d4/d17/db3/d116/d51/d117 0 2024-04-25T00:18:37.544 INFO:tasks.workunit.client.0.smithi107.stdout:5/829: dread d4/d17/db3/d84/d93/dc4/fe2 [0,4194304] 0 2024-04-25T00:18:37.544 INFO:tasks.workunit.client.0.smithi107.stdout:5/830: stat d4/d17/db3/c94 0 2024-04-25T00:18:37.546 INFO:tasks.workunit.client.0.smithi107.stdout:5/831: symlink d4/d7c/d88/d9d/de1/l118 0 2024-04-25T00:18:37.546 INFO:tasks.workunit.client.0.smithi107.stdout:5/832: read d4/d17/f79 [492876,30066] 0 2024-04-25T00:18:37.553 INFO:tasks.workunit.client.0.smithi107.stdout:8/994: write d2/d18/d1a/d5e/d3d/fc4 [80918,110074] 0 2024-04-25T00:18:37.553 INFO:tasks.workunit.client.0.smithi107.stdout:8/995: write d2/d18/d1a/d5e/fa7 [7646073,51698] 0 2024-04-25T00:18:37.553 INFO:tasks.workunit.client.0.smithi107.stdout:8/996: chown d2/d18/d1a/d5e/d3d/d85/dae/ffe 1259273 1 2024-04-25T00:18:37.553 INFO:tasks.workunit.client.0.smithi107.stdout:3/965: link d9/d54/d5c/d68/db3/lc0 d9/d88/d3a/l141 0 2024-04-25T00:18:37.555 INFO:tasks.workunit.client.0.smithi107.stdout:3/966: link d9/d15/d39/fb7 d9/d54/d94/dab/f142 0 2024-04-25T00:18:37.556 INFO:tasks.workunit.client.0.smithi107.stdout:3/967: creat d9/d88/d80/f143 x:0 0 0 2024-04-25T00:18:37.562 INFO:tasks.workunit.client.0.smithi107.stdout:8/997: dread d2/d18/f43 [0,4194304] 0 2024-04-25T00:18:37.568 INFO:tasks.workunit.client.0.smithi107.stdout:5/833: dwrite d4/fd7 [0,4194304] 0 2024-04-25T00:18:37.572 INFO:tasks.workunit.client.0.smithi107.stdout:5/834: symlink d4/dc/d26/d42/l119 0 2024-04-25T00:18:37.575 INFO:tasks.workunit.client.0.smithi107.stdout:3/968: dwrite d9/d88/d3a/d91/dad/deb/f11d [0,4194304] 0 2024-04-25T00:18:37.578 INFO:tasks.workunit.client.0.smithi107.stdout:3/969: mkdir d9/d54/d94/dab/d144 0 2024-04-25T00:18:37.578 INFO:tasks.workunit.client.0.smithi107.stdout:3/970: chown d9/d88/d80/d115/c132 12 1 2024-04-25T00:18:37.578 INFO:tasks.workunit.client.0.smithi107.stdout:3/971: fdatasync d9/d15/da1/f65 0 2024-04-25T00:18:37.580 INFO:tasks.workunit.client.0.smithi107.stdout:8/998: dwrite d2/d18/d1a/d5e/d4f/da0/da1/d151/ffc [0,4194304] 0 2024-04-25T00:18:37.582 INFO:tasks.workunit.client.0.smithi107.stdout:8/999: mkdir d2/d18/d1a/d5e/d4f/da0/da1/db9/d152 0 2024-04-25T00:18:37.586 INFO:tasks.workunit.client.0.smithi107.stdout:3/972: dread d9/d88/d4e/d93/fdb [0,4194304] 0 2024-04-25T00:18:37.590 INFO:tasks.workunit.client.0.smithi107.stdout:3/973: dread d9/f56 [0,4194304] 0 2024-04-25T00:18:38.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:37 smithi170 ceph-mon[32807]: pgmap v68: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 118 MiB/s rd, 143 MiB/s wr, 223 op/s 2024-04-25T00:18:38.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:37 smithi107 ceph-mon[29153]: pgmap v68: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 118 MiB/s rd, 143 MiB/s wr, 223 op/s 2024-04-25T00:18:38.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:37 smithi195 ceph-mon[32833]: pgmap v68: 129 pgs: 129 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 118 MiB/s rd, 143 MiB/s wr, 223 op/s 2024-04-25T00:18:38.938 INFO:tasks.workunit.client.0.smithi107.stdout:4/938: sync 2024-04-25T00:18:38.938 INFO:tasks.workunit.client.0.smithi107.stdout:0/840: sync 2024-04-25T00:18:38.939 INFO:tasks.workunit.client.0.smithi107.stdout:6/723: sync 2024-04-25T00:18:38.939 INFO:tasks.workunit.client.0.smithi107.stdout:7/947: sync 2024-04-25T00:18:38.939 INFO:tasks.workunit.client.0.smithi107.stdout:3/974: sync 2024-04-25T00:18:38.939 INFO:tasks.workunit.client.0.smithi107.stdout:0/841: chown dc/dd/d5d/d3f/f7a 0 1 2024-04-25T00:18:38.939 INFO:tasks.workunit.client.0.smithi107.stdout:1/861: sync 2024-04-25T00:18:38.939 INFO:tasks.workunit.client.0.smithi107.stdout:9/949: sync 2024-04-25T00:18:38.939 INFO:tasks.workunit.client.0.smithi107.stdout:1/862: read - d2/da/d2f/f102 zero size 2024-04-25T00:18:38.939 INFO:tasks.workunit.client.0.smithi107.stdout:5/835: sync 2024-04-25T00:18:38.939 INFO:tasks.workunit.client.0.smithi107.stdout:2/953: sync 2024-04-25T00:18:38.940 INFO:tasks.workunit.client.0.smithi107.stdout:3/975: truncate d9/f20 5873767 0 2024-04-25T00:18:38.940 INFO:tasks.workunit.client.0.smithi107.stdout:3/976: chown d9/d88/d3a/d91/dad/c120 0 1 2024-04-25T00:18:38.940 INFO:tasks.workunit.client.0.smithi107.stdout:3/977: write d9/d15/d39/f9f [1262464,84326] 0 2024-04-25T00:18:38.940 INFO:tasks.workunit.client.0.smithi107.stdout:3/978: chown d9/d22/c31 1553 1 2024-04-25T00:18:38.941 INFO:tasks.workunit.client.0.smithi107.stdout:6/724: creat d1/d14/de6/dad/fec x:0 0 0 2024-04-25T00:18:38.941 INFO:tasks.workunit.client.0.smithi107.stdout:6/725: chown d1/d14/de6/db/d32/d33/l97 1869228 1 2024-04-25T00:18:38.941 INFO:tasks.workunit.client.0.smithi107.stdout:9/950: mkdir d0/dfb/d149 0 2024-04-25T00:18:38.942 INFO:tasks.workunit.client.0.smithi107.stdout:6/726: mknod d1/d14/de6/db/d32/d33/ced 0 2024-04-25T00:18:38.943 INFO:tasks.workunit.client.0.smithi107.stdout:1/863: creat d2/d62/d66/de5/f118 x:0 0 0 2024-04-25T00:18:38.947 INFO:tasks.workunit.client.0.smithi107.stdout:9/951: rmdir d0/d8a/d93/d56/d6c/d80/d113 39 2024-04-25T00:18:38.947 INFO:tasks.workunit.client.0.smithi107.stdout:9/952: chown d0/d8a/d93/dc2/dc6/l103 2151787 1 2024-04-25T00:18:38.950 INFO:tasks.workunit.client.0.smithi107.stdout:9/953: mkdir d0/d8a/d93/d1f/d14a 0 2024-04-25T00:18:38.952 INFO:tasks.workunit.client.0.smithi107.stdout:5/836: mkdir d4/d7c/d88/d9d/de1/d11a 0 2024-04-25T00:18:38.952 INFO:tasks.workunit.client.0.smithi107.stdout:5/837: chown d4/d91/l10e 3750 1 2024-04-25T00:18:38.952 INFO:tasks.workunit.client.0.smithi107.stdout:9/954: symlink d0/d8a/d93/d34/d36/d77/dac/dee/l14b 0 2024-04-25T00:18:38.955 INFO:tasks.workunit.client.0.smithi107.stdout:3/979: mknod d9/d88/c145 0 2024-04-25T00:18:38.963 INFO:tasks.workunit.client.0.smithi107.stdout:6/727: dread d1/d4c/fac [0,4194304] 0 2024-04-25T00:18:38.963 INFO:tasks.workunit.client.0.smithi107.stdout:1/864: dread d2/da4/fcb [0,4194304] 0 2024-04-25T00:18:38.963 INFO:tasks.workunit.client.0.smithi107.stdout:3/980: rmdir d9/d88/daa 39 2024-04-25T00:18:38.964 INFO:tasks.workunit.client.0.smithi107.stdout:0/842: dwrite dc/dd/f19 [0,4194304] 0 2024-04-25T00:18:38.965 INFO:tasks.workunit.client.0.smithi107.stdout:1/865: mkdir d2/dd/dec/d119 0 2024-04-25T00:18:38.966 INFO:tasks.workunit.client.0.smithi107.stdout:0/843: symlink dc/d24/l10f 0 2024-04-25T00:18:38.968 INFO:tasks.workunit.client.0.smithi107.stdout:5/838: write d4/dc/f63 [4353334,69308] 0 2024-04-25T00:18:38.968 INFO:tasks.workunit.client.0.smithi107.stdout:0/844: creat dc/dd/d5d/d7e/f110 x:0 0 0 2024-04-25T00:18:38.969 INFO:tasks.workunit.client.0.smithi107.stdout:4/939: rename d3/d5/d32/d53/dc0/f118 to d3/d10/d7b/f13d 0 2024-04-25T00:18:38.973 INFO:tasks.workunit.client.0.smithi107.stdout:7/948: rename d0/d8/db/d18/d19/d47/de0/d115/d117/df4 to d0/d8/db/d18/d19/d47/de0/d115/d117/d58/db5/d13f 0 2024-04-25T00:18:38.973 INFO:tasks.workunit.client.0.smithi107.stdout:7/949: chown d0/d8/d4e/d46/d93/fe1 4144 1 2024-04-25T00:18:38.973 INFO:tasks.workunit.client.0.smithi107.stdout:7/950: chown d0/d8/db/d18/d19/d47/d4d 104616497 1 2024-04-25T00:18:38.973 INFO:tasks.workunit.client.0.smithi107.stdout:4/940: chown d3/ff8 458 1 2024-04-25T00:18:38.973 INFO:tasks.workunit.client.0.smithi107.stdout:4/941: write d3/d10/d7b/fd2 [112721,48316] 0 2024-04-25T00:18:38.973 INFO:tasks.workunit.client.0.smithi107.stdout:4/942: truncate d3/d5/d32/d53/d7e/fb3 4626426 0 2024-04-25T00:18:38.974 INFO:tasks.workunit.client.0.smithi107.stdout:6/728: mkdir d1/d14/de6/dee 0 2024-04-25T00:18:38.974 INFO:tasks.workunit.client.0.smithi107.stdout:6/729: stat d1/d14/de6/db/d3c 0 2024-04-25T00:18:38.976 INFO:tasks.workunit.client.0.smithi107.stdout:4/943: unlink d3/d5/d32/d9e/fba 0 2024-04-25T00:18:38.977 INFO:tasks.workunit.client.0.smithi107.stdout:0/845: mkdir dc/dd/d5d/d5c/d6e/d111 0 2024-04-25T00:18:38.978 INFO:tasks.workunit.client.0.smithi107.stdout:2/954: rename d6/dad/d67/lde to d6/dad/l143 0 2024-04-25T00:18:38.980 INFO:tasks.workunit.client.0.smithi107.stdout:4/944: link d3/l59 d3/d5/d32/d9e/d111/l13e 0 2024-04-25T00:18:38.982 INFO:tasks.workunit.client.0.smithi107.stdout:1/866: rename d2/d2a/d5c/d89/dc0 to d2/d2a/d37/de8/d11a 0 2024-04-25T00:18:38.983 INFO:tasks.workunit.client.0.smithi107.stdout:2/955: stat d6/d18/d40/d42/d2b/d30 0 2024-04-25T00:18:38.983 INFO:tasks.workunit.client.0.smithi107.stdout:4/945: mknod d3/d24/d61/d121/c13f 0 2024-04-25T00:18:38.983 INFO:tasks.workunit.client.0.smithi107.stdout:4/946: truncate d3/d24/d25/d71/d7f/fe4 324717 0 2024-04-25T00:18:38.985 INFO:tasks.workunit.client.0.smithi107.stdout:1/867: truncate d2/dd/d1b/fa3 2569522 0 2024-04-25T00:18:38.985 INFO:tasks.workunit.client.0.smithi107.stdout:7/951: rename d0/d8/db/d18/d19/d47/de0/d115/d117/d58/db5/f120 to d0/d8/d4e/d46/f140 0 2024-04-25T00:18:38.987 INFO:tasks.workunit.client.0.smithi107.stdout:4/947: creat d3/d24/d25/d71/dc2/f140 x:0 0 0 2024-04-25T00:18:38.987 INFO:tasks.workunit.client.0.smithi107.stdout:1/868: creat d2/d62/d66/d70/f11b x:0 0 0 2024-04-25T00:18:38.987 INFO:tasks.workunit.client.0.smithi107.stdout:3/981: dwrite d9/d88/d4e/dda/ff6 [0,4194304] 0 2024-04-25T00:18:38.988 INFO:tasks.workunit.client.0.smithi107.stdout:3/982: chown d9/d15/da1/db0 11047704 1 2024-04-25T00:18:38.988 INFO:tasks.workunit.client.0.smithi107.stdout:3/983: getdents d9/d22/d11c/d140 0 2024-04-25T00:18:38.988 INFO:tasks.workunit.client.0.smithi107.stdout:2/956: dread d6/d18/d40/d42/d2b/d30/d45/fa3 [0,4194304] 0 2024-04-25T00:18:38.988 INFO:tasks.workunit.client.0.smithi107.stdout:7/952: creat d0/f141 x:0 0 0 2024-04-25T00:18:38.988 INFO:tasks.workunit.client.0.smithi107.stdout:7/953: write d0/d8/fd [3741660,24956] 0 2024-04-25T00:18:38.989 INFO:tasks.workunit.client.0.smithi107.stdout:5/839: dread d4/d1c/d5c/dd4/ff2 [0,4194304] 0 2024-04-25T00:18:38.989 INFO:tasks.workunit.client.0.smithi107.stdout:1/869: rename d2/dd/d82/ff0 to d2/d108/f11c 0 2024-04-25T00:18:38.993 INFO:tasks.workunit.client.0.smithi107.stdout:0/846: read dc/dd/d16/d33/f89 [1132657,29367] 0 2024-04-25T00:18:38.993 INFO:tasks.workunit.client.0.smithi107.stdout:0/847: write dc/d24/d38/d8b/dd7/de1/f102 [908197,53333] 0 2024-04-25T00:18:38.993 INFO:tasks.workunit.client.0.smithi107.stdout:0/848: dread - dc/d24/d25/fec zero size 2024-04-25T00:18:38.993 INFO:tasks.workunit.client.0.smithi107.stdout:7/954: rename d0/d80/da8 to d0/d8/db/d18/d19/d142 0 2024-04-25T00:18:38.993 INFO:tasks.workunit.client.0.smithi107.stdout:7/955: chown d0/d8/db/d18/d19/d47/da1 14 1 2024-04-25T00:18:38.994 INFO:tasks.workunit.client.0.smithi107.stdout:7/956: write d0/d8/db/d18/d19/d47/f118 [320698,56235] 0 2024-04-25T00:18:38.994 INFO:tasks.workunit.client.0.smithi107.stdout:3/984: mkdir d9/d54/d94/d146 0 2024-04-25T00:18:38.994 INFO:tasks.workunit.client.0.smithi107.stdout:7/957: symlink d0/d8/l143 0 2024-04-25T00:18:38.996 INFO:tasks.workunit.client.0.smithi107.stdout:2/957: creat d6/d18/d40/dc8/f144 x:0 0 0 2024-04-25T00:18:38.997 INFO:tasks.workunit.client.0.smithi107.stdout:5/840: creat d4/dc/f11b x:0 0 0 2024-04-25T00:18:38.997 INFO:tasks.workunit.client.0.smithi107.stdout:5/841: write d4/dc/d83/f100 [318826,45766] 0 2024-04-25T00:18:38.998 INFO:tasks.workunit.client.0.smithi107.stdout:1/870: creat d2/da/db7/f11d x:0 0 0 2024-04-25T00:18:39.000 INFO:tasks.workunit.client.0.smithi107.stdout:2/958: truncate d6/d18/d40/d42/d69/d6e/da2/fba 3062666 0 2024-04-25T00:18:39.000 INFO:tasks.workunit.client.0.smithi107.stdout:2/959: fdatasync d6/d18/d40/d42/d69/fb3 0 2024-04-25T00:18:39.001 INFO:tasks.workunit.client.0.smithi107.stdout:5/842: mknod d4/d17/d104/c11c 0 2024-04-25T00:18:39.003 INFO:tasks.workunit.client.0.smithi107.stdout:1/871: symlink d2/d62/d66/l11e 0 2024-04-25T00:18:39.003 INFO:tasks.workunit.client.0.smithi107.stdout:1/872: readlink d2/d2a/d37/de8/d11a/lff 0 2024-04-25T00:18:39.004 INFO:tasks.workunit.client.0.smithi107.stdout:6/730: dwrite d1/d14/de6/db/d32/d36/d3d/f5e [4194304,4194304] 0 2024-04-25T00:18:39.005 INFO:tasks.workunit.client.0.smithi107.stdout:4/948: dread d3/d5/f7 [0,4194304] 0 2024-04-25T00:18:39.005 INFO:tasks.workunit.client.0.smithi107.stdout:4/949: chown d3/d24/d25/l8a 88760 1 2024-04-25T00:18:39.005 INFO:tasks.workunit.client.0.smithi107.stdout:4/950: chown d3/d24/d25/d71/dc2/ff6 4537922 1 2024-04-25T00:18:39.006 INFO:tasks.workunit.client.0.smithi107.stdout:2/960: link d6/d18/d40/d42/d2b/f41 d6/d29/d4e/dc0/dd1/f145 0 2024-04-25T00:18:39.007 INFO:tasks.workunit.client.0.smithi107.stdout:4/951: write d3/f12b [714989,117942] 0 2024-04-25T00:18:39.007 INFO:tasks.workunit.client.0.smithi107.stdout:0/849: read dc/dd/d5d/f57 [989931,49862] 0 2024-04-25T00:18:39.009 INFO:tasks.workunit.client.0.smithi107.stdout:5/843: unlink d4/d17/db3/d116/d2e/d55/ca2 0 2024-04-25T00:18:39.013 INFO:tasks.workunit.client.0.smithi107.stdout:2/961: mknod d6/d18/d40/d42/d2b/d30/d45/dca/c146 0 2024-04-25T00:18:39.013 INFO:tasks.workunit.client.0.smithi107.stdout:2/962: fsync d6/d18/d40/d42/f57 0 2024-04-25T00:18:39.013 INFO:tasks.workunit.client.0.smithi107.stdout:2/963: write d6/d29/d4e/fab [3011336,118791] 0 2024-04-25T00:18:39.013 INFO:tasks.workunit.client.0.smithi107.stdout:2/964: write d6/d18/d40/d42/d2b/f9e [201073,10209] 0 2024-04-25T00:18:39.015 INFO:tasks.workunit.client.0.smithi107.stdout:2/965: creat d6/d18/d40/dc8/de5/f147 x:0 0 0 2024-04-25T00:18:39.016 INFO:tasks.workunit.client.0.smithi107.stdout:1/873: write d2/dd/d1b/fba [929663,37268] 0 2024-04-25T00:18:39.018 INFO:tasks.workunit.client.0.smithi107.stdout:6/731: dread d1/d14/de6/d3/d26/fa2 [4194304,4194304] 0 2024-04-25T00:18:39.018 INFO:tasks.workunit.client.0.smithi107.stdout:6/732: chown d1/d14/de6/d3/d28/d79/d6b/ca9 461 1 2024-04-25T00:18:39.018 INFO:tasks.workunit.client.0.smithi107.stdout:1/874: rmdir d2/dd 39 2024-04-25T00:18:39.018 INFO:tasks.workunit.client.0.smithi107.stdout:7/958: dwrite d0/d8/db/d18/d19/d47/de0/d82/fd9 [0,4194304] 0 2024-04-25T00:18:39.019 INFO:tasks.workunit.client.0.smithi107.stdout:1/875: rmdir d2/d2a/d37/de8/d105 39 2024-04-25T00:18:39.020 INFO:tasks.workunit.client.0.smithi107.stdout:6/733: creat d1/d14/de6/d3/d26/de2/fef x:0 0 0 2024-04-25T00:18:39.020 INFO:tasks.workunit.client.0.smithi107.stdout:6/734: write d1/d14/de6/db/d32/d33/d68/d75/f8b [5071756,86580] 0 2024-04-25T00:18:39.022 INFO:tasks.workunit.client.0.smithi107.stdout:6/735: creat d1/d14/de6/db/d32/ff0 x:0 0 0 2024-04-25T00:18:39.024 INFO:tasks.workunit.client.0.smithi107.stdout:6/736: unlink d1/d14/de6/db/d3c/d43/d96/cbf 0 2024-04-25T00:18:39.026 INFO:tasks.workunit.client.0.smithi107.stdout:6/737: creat d1/d14/de6/d2b/d3e/d5b/ff1 x:0 0 0 2024-04-25T00:18:39.027 INFO:tasks.workunit.client.0.smithi107.stdout:4/952: dread d3/d5/d32/f9f [0,4194304] 0 2024-04-25T00:18:39.027 INFO:tasks.workunit.client.0.smithi107.stdout:4/953: write d3/f12b [976121,51887] 0 2024-04-25T00:18:39.027 INFO:tasks.workunit.client.0.smithi107.stdout:4/954: write d3/da8/fb7 [3353951,10734] 0 2024-04-25T00:18:39.028 INFO:tasks.workunit.client.0.smithi107.stdout:6/738: link d1/d14/de6/db/d32/d33/d68/lc8 d1/d14/de6/d3/d28/d79/de0/lf2 0 2024-04-25T00:18:39.029 INFO:tasks.workunit.client.0.smithi107.stdout:5/844: dwrite d4/d17/f69 [4194304,4194304] 0 2024-04-25T00:18:39.029 INFO:tasks.workunit.client.0.smithi107.stdout:4/955: truncate d3/d24/d5c/dab/f8d 390968 0 2024-04-25T00:18:39.029 INFO:tasks.workunit.client.0.smithi107.stdout:4/956: chown d3/d24/f69 3209 1 2024-04-25T00:18:39.029 INFO:tasks.workunit.client.0.smithi107.stdout:6/739: getdents d1/d14/de6/d3/d28 0 2024-04-25T00:18:39.030 INFO:tasks.workunit.client.0.smithi107.stdout:6/740: write d1/d14/de6/db/d32/dca/fce [858603,57394] 0 2024-04-25T00:18:39.030 INFO:tasks.workunit.client.0.smithi107.stdout:6/741: fsync d1/d14/de6/db/d32/d36/d3a/f9a 0 2024-04-25T00:18:39.031 INFO:tasks.workunit.client.0.smithi107.stdout:4/957: dread d3/d24/db1/fc9 [0,4194304] 0 2024-04-25T00:18:39.031 INFO:tasks.workunit.client.0.smithi107.stdout:5/845: mkdir d4/d17/db3/d116/d51/d117/d11d 0 2024-04-25T00:18:39.032 INFO:tasks.workunit.client.0.smithi107.stdout:7/959: write d0/d8/db/d18/f41 [3808645,89051] 0 2024-04-25T00:18:39.036 INFO:tasks.workunit.client.0.smithi107.stdout:3/985: rename d9/f5e to d9/d15/d123/f147 0 2024-04-25T00:18:39.036 INFO:tasks.workunit.client.0.smithi107.stdout:3/986: stat d9/d54/d5c/d10f 0 2024-04-25T00:18:39.039 INFO:tasks.workunit.client.0.smithi107.stdout:3/987: creat d9/d22/d1b/d55/df7/f148 x:0 0 0 2024-04-25T00:18:39.040 INFO:tasks.workunit.client.0.smithi107.stdout:6/742: write d1/d14/de6/d3/d71/d94/f9b [5961393,86735] 0 2024-04-25T00:18:39.050 INFO:tasks.workunit.client.0.smithi107.stdout:4/958: dwrite d3/d5/f65 [0,4194304] 0 2024-04-25T00:18:39.050 INFO:tasks.workunit.client.0.smithi107.stdout:4/959: fsync d3/d5/d32/f38 0 2024-04-25T00:18:39.051 INFO:tasks.workunit.client.0.smithi107.stdout:6/743: write d1/d14/de6/db/d32/d36/d3a/fa7 [4015270,27086] 0 2024-04-25T00:18:39.067 INFO:tasks.workunit.client.0.smithi107.stdout:6/744: link d1/d14/de6/db/d3c/d43/d96/dc9/cd5 d1/d14/de6/db/d3c/d43/cf3 0 2024-04-25T00:18:39.068 INFO:tasks.workunit.client.0.smithi107.stdout:4/960: truncate d3/d24/d5c/f6f 98762 0 2024-04-25T00:18:39.075 INFO:tasks.workunit.client.0.smithi107.stdout:4/961: dread d3/d24/d5c/fd1 [0,4194304] 0 2024-04-25T00:18:39.077 INFO:tasks.workunit.client.0.smithi107.stdout:4/962: truncate d3/d5/d32/d53/d97/f99 2250625 0 2024-04-25T00:18:39.080 INFO:tasks.workunit.client.0.smithi107.stdout:0/850: rename dc/d24/d38/d8b/dd7/de1/f102 to dc/dd/d16/d48/f112 0 2024-04-25T00:18:39.080 INFO:tasks.workunit.client.0.smithi107.stdout:0/851: stat dc/dd/d16/d48/d83/d9f/dc4 0 2024-04-25T00:18:39.082 INFO:tasks.workunit.client.0.smithi107.stdout:0/852: creat dc/dd/d5d/d3f/f113 x:0 0 0 2024-04-25T00:18:39.086 INFO:tasks.workunit.client.0.smithi107.stdout:9/955: sync 2024-04-25T00:18:39.086 INFO:tasks.workunit.client.0.smithi107.stdout:6/745: dwrite d1/d14/de6/d3/d28/d79/fb8 [0,4194304] 0 2024-04-25T00:18:39.086 INFO:tasks.workunit.client.0.smithi107.stdout:6/746: chown d1/d14/de6/d3/d71/cab 129315 1 2024-04-25T00:18:39.088 INFO:tasks.workunit.client.0.smithi107.stdout:9/956: mknod d0/d8a/d93/d34/d36/d77/dac/dee/c14c 0 2024-04-25T00:18:39.088 INFO:tasks.workunit.client.0.smithi107.stdout:6/747: mkdir d1/d14/de6/d2b/d3e/df4 0 2024-04-25T00:18:39.097 INFO:tasks.workunit.client.0.smithi107.stdout:0/853: dread dc/d24/d38/d64/d7b/fae [0,4194304] 0 2024-04-25T00:18:39.101 INFO:tasks.workunit.client.0.smithi107.stdout:2/966: rename d6/d18/d40/d42/d2b/d30/d45/dca to d6/dad/d67/da8/d148 0 2024-04-25T00:18:39.102 INFO:tasks.workunit.client.0.smithi107.stdout:2/967: chown d6/d18/d40/d42/d69 4 1 2024-04-25T00:18:39.103 INFO:tasks.workunit.client.0.smithi107.stdout:1/876: rename d2/da/l59 to d2/d62/l11f 0 2024-04-25T00:18:39.103 INFO:tasks.workunit.client.0.smithi107.stdout:1/877: chown d2/dd/d1b/dc2/c10e 110235 1 2024-04-25T00:18:39.104 INFO:tasks.workunit.client.0.smithi107.stdout:1/878: fdatasync d2/d62/d66/f78 0 2024-04-25T00:18:39.104 INFO:tasks.workunit.client.0.smithi107.stdout:1/879: fsync d2/d2a/d37/d88/fb2 0 2024-04-25T00:18:39.104 INFO:tasks.workunit.client.0.smithi107.stdout:1/880: truncate d2/dd/d1b/ff2 993150 0 2024-04-25T00:18:39.104 INFO:tasks.workunit.client.0.smithi107.stdout:1/881: write d2/d62/d66/d70/dcc/f116 [89642,54913] 0 2024-04-25T00:18:39.104 INFO:tasks.workunit.client.0.smithi107.stdout:2/968: creat d6/d18/d40/d42/d69/d6e/db6/f149 x:0 0 0 2024-04-25T00:18:39.106 INFO:tasks.workunit.client.0.smithi107.stdout:5/846: rename d4/d17/f85 to d4/d17/db3/f11e 0 2024-04-25T00:18:39.106 INFO:tasks.workunit.client.0.smithi107.stdout:5/847: readlink d4/dc/lbe 0 2024-04-25T00:18:39.106 INFO:tasks.workunit.client.0.smithi107.stdout:1/882: mkdir d2/d120 0 2024-04-25T00:18:39.108 INFO:tasks.workunit.client.0.smithi107.stdout:5/848: rmdir d4/d17/db3/d116 39 2024-04-25T00:18:39.109 INFO:tasks.workunit.client.0.smithi107.stdout:7/960: rename d0/d8/db/d18/le2 to d0/d8/db/d18/d19/d47/de0/dc2/dd0/l144 0 2024-04-25T00:18:39.109 INFO:tasks.workunit.client.0.smithi107.stdout:7/961: dread - d0/d8/db/ff2 zero size 2024-04-25T00:18:39.110 INFO:tasks.workunit.client.0.smithi107.stdout:5/849: symlink d4/d17/db3/d84/df0/dfb/l11f 0 2024-04-25T00:18:39.112 INFO:tasks.workunit.client.0.smithi107.stdout:7/962: symlink d0/d8/db/l145 0 2024-04-25T00:18:39.113 INFO:tasks.workunit.client.0.smithi107.stdout:6/748: dwrite d1/d14/de6/d3/f41 [0,4194304] 0 2024-04-25T00:18:39.115 INFO:tasks.workunit.client.0.smithi107.stdout:3/988: rename d9/d22/d1b/dbd/f12c to d9/d22/d11c/d140/f149 0 2024-04-25T00:18:39.115 INFO:tasks.workunit.client.0.smithi107.stdout:3/989: stat d9/d54/d5c/lc6 0 2024-04-25T00:18:39.115 INFO:tasks.workunit.client.0.smithi107.stdout:6/749: mkdir d1/d14/de6/dee/df5 0 2024-04-25T00:18:39.116 INFO:tasks.workunit.client.0.smithi107.stdout:6/750: dread - d1/d14/de6/d3/fd7 zero size 2024-04-25T00:18:39.117 INFO:tasks.workunit.client.0.smithi107.stdout:6/751: creat d1/d14/de6/dee/df5/ff6 x:0 0 0 2024-04-25T00:18:39.131 INFO:tasks.workunit.client.0.smithi107.stdout:6/752: write d1/d14/f89 [689108,103576] 0 2024-04-25T00:18:39.131 INFO:tasks.workunit.client.0.smithi107.stdout:6/753: write d1/d14/de6/d3/d28/d79/d6b/fe1 [440738,9943] 0 2024-04-25T00:18:39.131 INFO:tasks.workunit.client.0.smithi107.stdout:6/754: chown d1/d14/de6/d3/d28/d79/de0/lf2 10 1 2024-04-25T00:18:39.137 INFO:tasks.workunit.client.0.smithi107.stdout:5/850: dwrite d4/d17/fd6 [0,4194304] 0 2024-04-25T00:18:39.137 INFO:tasks.workunit.client.0.smithi107.stdout:5/851: chown d4/d7c/d88/d9d/ca1 5248005 1 2024-04-25T00:18:39.138 INFO:tasks.workunit.client.0.smithi107.stdout:5/852: write d4/d17/db3/f11e [3712919,21967] 0 2024-04-25T00:18:39.139 INFO:tasks.workunit.client.0.smithi107.stdout:5/853: creat d4/d17/db3/d84/d93/dc4/f120 x:0 0 0 2024-04-25T00:18:39.141 INFO:tasks.workunit.client.0.smithi107.stdout:3/990: dwrite d9/d15/da1/f119 [0,4194304] 0 2024-04-25T00:18:39.144 INFO:tasks.workunit.client.0.smithi107.stdout:5/854: creat d4/d7c/d88/d9d/f121 x:0 0 0 2024-04-25T00:18:39.145 INFO:tasks.workunit.client.0.smithi107.stdout:4/963: rename d3/d24/d25/d67/fcc to d3/d24/d5c/f141 0 2024-04-25T00:18:39.147 INFO:tasks.workunit.client.0.smithi107.stdout:4/964: creat d3/dd8/f142 x:0 0 0 2024-04-25T00:18:39.147 INFO:tasks.workunit.client.0.smithi107.stdout:4/965: read - d3/dd8/f142 zero size 2024-04-25T00:18:39.148 INFO:tasks.workunit.client.0.smithi107.stdout:6/755: dwrite d1/d14/de6/db/d48/fd2 [0,4194304] 0 2024-04-25T00:18:39.148 INFO:tasks.workunit.client.0.smithi107.stdout:4/966: getdents d3/d24/db1 0 2024-04-25T00:18:39.148 INFO:tasks.workunit.client.0.smithi107.stdout:4/967: write d3/d5/f8f [1218561,87151] 0 2024-04-25T00:18:39.148 INFO:tasks.workunit.client.0.smithi107.stdout:4/968: chown d3/d5/d32/d53/d7e/fb3 11973648 1 2024-04-25T00:18:39.149 INFO:tasks.workunit.client.0.smithi107.stdout:9/957: rename d0/d8a/d93/d56/d6c/d80/d9b/db2/dea/ff7 to d0/d8a/d93/d56/d6c/d80/d9b/db2/dea/d10c/d137/f14d 0 2024-04-25T00:18:39.151 INFO:tasks.workunit.client.0.smithi107.stdout:4/969: mknod d3/d5/d32/d9e/da2/daf/d109/c143 0 2024-04-25T00:18:39.151 INFO:tasks.workunit.client.0.smithi107.stdout:6/756: mkdir d1/d14/d18/ddd/df7 0 2024-04-25T00:18:39.152 INFO:tasks.workunit.client.0.smithi107.stdout:4/970: symlink d3/d24/d61/d121/l144 0 2024-04-25T00:18:39.154 INFO:tasks.workunit.client.0.smithi107.stdout:6/757: creat d1/d14/de6/d3/d28/ff8 x:0 0 0 2024-04-25T00:18:39.154 INFO:tasks.workunit.client.0.smithi107.stdout:6/758: readlink d1/d14/de6/d3/d28/d79/d6b/l88 0 2024-04-25T00:18:39.154 INFO:tasks.workunit.client.0.smithi107.stdout:6/759: truncate d1/d14/de6/f5f 5394662 0 2024-04-25T00:18:39.154 INFO:tasks.workunit.client.0.smithi107.stdout:6/760: chown d1/d14/d18/ddd/df7 51 1 2024-04-25T00:18:39.155 INFO:tasks.workunit.client.0.smithi107.stdout:3/991: write d9/d54/ff1 [2098499,33207] 0 2024-04-25T00:18:39.155 INFO:tasks.workunit.client.0.smithi107.stdout:3/992: truncate d9/d88/d3a/d91/dad/deb/f13f 1044 0 2024-04-25T00:18:39.155 INFO:tasks.workunit.client.0.smithi107.stdout:9/958: read d0/f8f [174932,113165] 0 2024-04-25T00:18:39.155 INFO:tasks.workunit.client.0.smithi107.stdout:9/959: chown d0/d8a/d93/d34/d36/d77/d110/l129 4481 1 2024-04-25T00:18:39.155 INFO:tasks.workunit.client.0.smithi107.stdout:4/971: creat d3/d5/d32/d9e/d111/f145 x:0 0 0 2024-04-25T00:18:39.156 INFO:tasks.workunit.client.0.smithi107.stdout:6/761: symlink d1/d14/de6/d2b/d3e/lf9 0 2024-04-25T00:18:39.157 INFO:tasks.workunit.client.0.smithi107.stdout:3/993: unlink d9/d88/d3a/ld4 0 2024-04-25T00:18:39.158 INFO:tasks.workunit.client.0.smithi107.stdout:4/972: creat d3/d10/d7b/da5/f146 x:0 0 0 2024-04-25T00:18:39.161 INFO:tasks.workunit.client.0.smithi107.stdout:0/854: rename dc/dd/d16/d33/d35/d5f/ld2 to dc/d24/d38/d8b/l114 0 2024-04-25T00:18:39.163 INFO:tasks.workunit.client.0.smithi107.stdout:0/855: unlink dc/d24/d25/c100 0 2024-04-25T00:18:39.164 INFO:tasks.workunit.client.0.smithi107.stdout:6/762: link d1/d14/de6/d3/d28/d79/l98 d1/d14/de6/d2b/d3e/d5b/d7a/lfa 0 2024-04-25T00:18:39.164 INFO:tasks.workunit.client.0.smithi107.stdout:4/973: mknod d3/d24/db1/d126/d132/c147 0 2024-04-25T00:18:39.166 INFO:tasks.workunit.client.0.smithi107.stdout:0/856: creat dc/d24/d38/d64/d78/f115 x:0 0 0 2024-04-25T00:18:39.167 INFO:tasks.workunit.client.0.smithi107.stdout:7/963: rename d0/d8/db/d18/d19/d47/de0/d115/d117/d58/f79 to d0/d8/d4e/d46/d93/f146 0 2024-04-25T00:18:39.168 INFO:tasks.workunit.client.0.smithi107.stdout:0/857: unlink dc/da6/fe8 0 2024-04-25T00:18:39.168 INFO:tasks.workunit.client.0.smithi107.stdout:0/858: write dc/ff2 [677395,115043] 0 2024-04-25T00:18:39.168 INFO:tasks.workunit.client.0.smithi107.stdout:7/964: mknod d0/d8/db/d18/d19/d47/d4d/dbc/c147 0 2024-04-25T00:18:39.168 INFO:tasks.workunit.client.0.smithi107.stdout:7/965: truncate d0/d8/db/d18/d19/d47/de0/d115/d117/d58/fb4 4502040 0 2024-04-25T00:18:39.170 INFO:tasks.workunit.client.0.smithi107.stdout:5/855: rename d4/d17/db3/d116/d2e/lec to d4/d17/db3/l122 0 2024-04-25T00:18:39.170 INFO:tasks.workunit.client.0.smithi107.stdout:4/974: dread d3/d24/f95 [0,4194304] 0 2024-04-25T00:18:39.170 INFO:tasks.workunit.client.0.smithi107.stdout:4/975: chown d3/d24/d61/d121/l144 61005785 1 2024-04-25T00:18:39.170 INFO:tasks.workunit.client.0.smithi107.stdout:7/966: creat d0/d8/db/d18/d19/d47/de0/d115/d117/d58/f148 x:0 0 0 2024-04-25T00:18:39.171 INFO:tasks.workunit.client.0.smithi107.stdout:0/859: creat dc/d24/d38/d64/d78/de6/f116 x:0 0 0 2024-04-25T00:18:39.172 INFO:tasks.workunit.client.0.smithi107.stdout:0/860: dread - dc/d24/d38/d64/d7b/f101 zero size 2024-04-25T00:18:39.172 INFO:tasks.workunit.client.0.smithi107.stdout:0/861: truncate dc/dd/d16/d48/f112 1858549 0 2024-04-25T00:18:39.172 INFO:tasks.workunit.client.0.smithi107.stdout:0/862: read dc/dd/d5d/d3f/da5/fcf [1962031,120053] 0 2024-04-25T00:18:39.172 INFO:tasks.workunit.client.0.smithi107.stdout:0/863: readlink l3 0 2024-04-25T00:18:39.172 INFO:tasks.workunit.client.0.smithi107.stdout:7/967: rmdir d0/d8/db/d10e 39 2024-04-25T00:18:39.172 INFO:tasks.workunit.client.0.smithi107.stdout:4/976: write d3/d10/d7b/feb [45639,115716] 0 2024-04-25T00:18:39.173 INFO:tasks.workunit.client.0.smithi107.stdout:9/960: rename d0/d8a/d93/d6f/c92 to d0/dfb/d149/c14e 0 2024-04-25T00:18:39.175 INFO:tasks.workunit.client.0.smithi107.stdout:7/968: getdents d0/d8/db/d18/d19/d47/de0/d115/d117/de5 0 2024-04-25T00:18:39.181 INFO:tasks.workunit.client.0.smithi107.stdout:4/977: read d3/d5/d32/f10e [775980,45389] 0 2024-04-25T00:18:39.181 INFO:tasks.workunit.client.0.smithi107.stdout:4/978: readlink d3/d24/d5c/l119 0 2024-04-25T00:18:39.192 INFO:tasks.workunit.client.0.smithi107.stdout:5/856: dwrite d4/d1c/fa8 [0,4194304] 0 2024-04-25T00:18:39.192 INFO:tasks.workunit.client.0.smithi107.stdout:5/857: chown d4/d17/db3/d116/d2e/c49 35 1 2024-04-25T00:18:39.192 INFO:tasks.workunit.client.0.smithi107.stdout:5/858: readlink d4/d17/db3/d116/d51/laf 0 2024-04-25T00:18:39.196 INFO:tasks.workunit.client.0.smithi107.stdout:9/961: dwrite d0/d8a/d93/d56/d76/d50/d7d/f85 [0,4194304] 0 2024-04-25T00:18:39.196 INFO:tasks.workunit.client.0.smithi107.stdout:9/962: dread - d0/d8a/d93/d56/f109 zero size 2024-04-25T00:18:39.196 INFO:tasks.workunit.client.0.smithi107.stdout:9/963: write d0/d8a/d93/d56/d76/d111/f148 [231602,94925] 0 2024-04-25T00:18:39.198 INFO:tasks.workunit.client.0.smithi107.stdout:4/979: read d3/d24/d25/f30 [2634388,80040] 0 2024-04-25T00:18:39.198 INFO:tasks.workunit.client.0.smithi107.stdout:4/980: write d3/d5/d32/f79 [2615413,128483] 0 2024-04-25T00:18:39.198 INFO:tasks.workunit.client.0.smithi107.stdout:4/981: readlink d3/d24/lf9 0 2024-04-25T00:18:39.199 INFO:tasks.workunit.client.0.smithi107.stdout:7/969: dwrite d0/d80/da2/f10d [0,4194304] 0 2024-04-25T00:18:39.201 INFO:tasks.workunit.client.0.smithi107.stdout:7/970: rmdir d0/d8/db/d18/d19/d47/da1/dd4/d128 39 2024-04-25T00:18:39.207 INFO:tasks.workunit.client.0.smithi107.stdout:4/982: dread d3/d5/f8f [0,4194304] 0 2024-04-25T00:18:39.207 INFO:tasks.workunit.client.0.smithi107.stdout:4/983: dread - d3/d24/d5c/dab/da3/f100 zero size 2024-04-25T00:18:39.208 INFO:tasks.workunit.client.0.smithi107.stdout:4/984: link d3/d5/d32/d53/l5d d3/d10/l148 0 2024-04-25T00:18:39.210 INFO:tasks.workunit.client.0.smithi107.stdout:0/864: dwrite dc/dd/d16/d5e/ff5 [0,4194304] 0 2024-04-25T00:18:39.212 INFO:tasks.workunit.client.0.smithi107.stdout:0/865: symlink dc/dd/d16/d48/dfb/l117 0 2024-04-25T00:18:39.212 INFO:tasks.workunit.client.0.smithi107.stdout:0/866: readlink dc/l65 0 2024-04-25T00:18:39.212 INFO:tasks.workunit.client.0.smithi107.stdout:0/867: write dc/d24/d38/d64/d78/de6/f116 [415523,124856] 0 2024-04-25T00:18:39.212 INFO:tasks.workunit.client.0.smithi107.stdout:0/868: fsync dc/d24/d38/d64/f68 0 2024-04-25T00:18:39.212 INFO:tasks.workunit.client.0.smithi107.stdout:0/869: readlink dc/dd/d16/l69 0 2024-04-25T00:18:39.212 INFO:tasks.workunit.client.0.smithi107.stdout:0/870: fsync dc/dd/f4c 0 2024-04-25T00:18:39.214 INFO:tasks.workunit.client.0.smithi107.stdout:9/964: write d0/f40 [979226,15325] 0 2024-04-25T00:18:39.216 INFO:tasks.workunit.client.0.smithi107.stdout:3/994: rename d9/d54/ld5 to d9/d88/daa/l14a 0 2024-04-25T00:18:39.220 INFO:tasks.workunit.client.0.smithi107.stdout:0/871: dread dc/d24/fbf [0,4194304] 0 2024-04-25T00:18:39.221 INFO:tasks.workunit.client.0.smithi107.stdout:6/763: rename d1/d14/de6/db/d32/d33/d4f to d1/d14/de6/dad/dfb 0 2024-04-25T00:18:39.224 INFO:tasks.workunit.client.0.smithi107.stdout:7/971: rename d0/d8/db/d18/d19/f3c to d0/d8/d4e/d46/f149 0 2024-04-25T00:18:39.224 INFO:tasks.workunit.client.0.smithi107.stdout:7/972: write d0/d8/db/d18/d19/d47/d4d/f114 [182309,106629] 0 2024-04-25T00:18:39.225 INFO:tasks.workunit.client.0.smithi107.stdout:6/764: dread d1/d14/de6/db/d32/d33/f63 [0,4194304] 0 2024-04-25T00:18:39.225 INFO:tasks.workunit.client.0.smithi107.stdout:6/765: fsync d1/d14/de6/db/d32/d36/d3a/f9a 0 2024-04-25T00:18:39.225 INFO:tasks.workunit.client.0.smithi107.stdout:9/965: dread d0/d43/f28 [0,4194304] 0 2024-04-25T00:18:39.226 INFO:tasks.workunit.client.0.smithi107.stdout:7/973: unlink d0/d80/fd2 0 2024-04-25T00:18:39.227 INFO:tasks.workunit.client.0.smithi107.stdout:9/966: creat d0/d8a/d93/d34/d36/f14f x:0 0 0 2024-04-25T00:18:39.227 INFO:tasks.workunit.client.0.smithi107.stdout:9/967: write d0/d8a/d93/d56/d6c/d80/f136 [69533,73037] 0 2024-04-25T00:18:39.227 INFO:tasks.workunit.client.0.smithi107.stdout:9/968: chown d0/d8a/d93/d34/fa9 42 1 2024-04-25T00:18:39.228 INFO:tasks.workunit.client.0.smithi107.stdout:6/766: link d1/d14/de6/db/d32/d33/c39 d1/d14/de6/dee/df5/cfc 0 2024-04-25T00:18:39.228 INFO:tasks.workunit.client.0.smithi107.stdout:9/969: creat d0/d8a/d93/d56/d6c/d80/d9b/d140/f150 x:0 0 0 2024-04-25T00:18:39.228 INFO:tasks.workunit.client.0.smithi107.stdout:9/970: fsync d0/d8a/d93/de/d5c/f96 0 2024-04-25T00:18:39.229 INFO:tasks.workunit.client.0.smithi107.stdout:6/767: mknod d1/d14/de6/d3/d26/cfd 0 2024-04-25T00:18:39.230 INFO:tasks.workunit.client.0.smithi107.stdout:9/971: rename d0/d8a/d93/de/d5c/d95/c112 to d0/d8a/d93/de/d5c/de0/c151 0 2024-04-25T00:18:39.230 INFO:tasks.workunit.client.0.smithi107.stdout:9/972: truncate d0/d8a/d93/dc2/dc6/f146 138133 0 2024-04-25T00:18:39.230 INFO:tasks.workunit.client.0.smithi107.stdout:9/973: dread - d0/d8a/d93/d56/d6c/d80/d9b/db2/dea/d10c/f130 zero size 2024-04-25T00:18:39.231 INFO:tasks.workunit.client.0.smithi107.stdout:9/974: link d0/f4b d0/d8a/d93/d34/d36/d77/d110/f152 0 2024-04-25T00:18:39.232 INFO:tasks.workunit.client.0.smithi107.stdout:9/975: mknod d0/d8a/d93/d34/d36/c153 0 2024-04-25T00:18:39.234 INFO:tasks.workunit.client.0.smithi107.stdout:0/872: write dc/d24/d38/d64/f68 [4047817,45816] 0 2024-04-25T00:18:39.234 INFO:tasks.workunit.client.0.smithi107.stdout:3/995: dwrite d9/d22/d1b/f81 [0,4194304] 0 2024-04-25T00:18:39.235 INFO:tasks.workunit.client.0.smithi107.stdout:0/873: creat dc/dd/d5d/d3f/dc7/f118 x:0 0 0 2024-04-25T00:18:39.236 INFO:tasks.workunit.client.0.smithi107.stdout:3/996: creat d9/d22/d1b/dbd/f14b x:0 0 0 2024-04-25T00:18:39.237 INFO:tasks.workunit.client.0.smithi107.stdout:0/874: mknod dc/dd/d16/d48/d83/c119 0 2024-04-25T00:18:39.237 INFO:tasks.workunit.client.0.smithi107.stdout:0/875: dread - dc/dd/d5d/d3f/da5/fde zero size 2024-04-25T00:18:39.238 INFO:tasks.workunit.client.0.smithi107.stdout:3/997: symlink d9/d88/d80/d115/d12a/l14c 0 2024-04-25T00:18:39.240 INFO:tasks.workunit.client.0.smithi107.stdout:3/998: mkdir d9/d54/d14d 0 2024-04-25T00:18:39.240 INFO:tasks.workunit.client.0.smithi107.stdout:3/999: chown d9/d22/d11c/d140 0 1 2024-04-25T00:18:39.241 INFO:tasks.workunit.client.0.smithi107.stdout:0/876: mkdir dc/dd/d5d/d5c/d6e/d11a 0 2024-04-25T00:18:39.244 INFO:tasks.workunit.client.0.smithi107.stdout:7/974: dwrite d0/d8/f4a [0,4194304] 0 2024-04-25T00:18:39.244 INFO:tasks.workunit.client.0.smithi107.stdout:7/975: dread - d0/d8/db/d18/d19/d47/f5b zero size 2024-04-25T00:18:39.244 INFO:tasks.workunit.client.0.smithi107.stdout:7/976: chown d0/d8/d4e/d11f 853480 1 2024-04-25T00:18:39.248 INFO:tasks.workunit.client.0.smithi107.stdout:7/977: mkdir d0/d8/d4e/d11f/d14a 0 2024-04-25T00:18:39.250 INFO:tasks.workunit.client.0.smithi107.stdout:7/978: mkdir d0/d8/db/d18/d19/d47/de0/dc2/d14b 0 2024-04-25T00:18:39.253 INFO:tasks.workunit.client.0.smithi107.stdout:7/979: mknod d0/d8/db/d18/d19/c14c 0 2024-04-25T00:18:39.253 INFO:tasks.workunit.client.0.smithi107.stdout:7/980: read - d0/d8/db/d18/d19/d142/d123/f13d zero size 2024-04-25T00:18:39.254 INFO:tasks.workunit.client.0.smithi107.stdout:6/768: dwrite d1/d14/de6/db/d32/d33/d68/fa0 [0,4194304] 0 2024-04-25T00:18:39.254 INFO:tasks.workunit.client.0.smithi107.stdout:6/769: chown d1/d4c/fac 6039053 1 2024-04-25T00:18:39.254 INFO:tasks.workunit.client.0.smithi107.stdout:6/770: dread - d1/d14/de6/d3/d28/ff8 zero size 2024-04-25T00:18:39.256 INFO:tasks.workunit.client.0.smithi107.stdout:9/976: rmdir d0/d8a/d93/d6f/da1/df2 39 2024-04-25T00:18:39.256 INFO:tasks.workunit.client.0.smithi107.stdout:7/981: rmdir d0/d8/db/d18/d19/d47/de0/dc2 39 2024-04-25T00:18:39.256 INFO:tasks.workunit.client.0.smithi107.stdout:7/982: dread - d0/d8/db/d18/d19/ff3 zero size 2024-04-25T00:18:39.260 INFO:tasks.workunit.client.0.smithi107.stdout:0/877: rename dc/dd/d16/d33 to dc/dd/d16/d48/ddb/d11b 0 2024-04-25T00:18:39.267 INFO:tasks.workunit.client.0.smithi107.stdout:0/878: write dc/dd/d16/d5e/fcc [1482601,119552] 0 2024-04-25T00:18:39.271 INFO:tasks.workunit.client.0.smithi107.stdout:9/977: link d0/d8a/d93/d6f/da1/df2/f133 d0/d8a/d93/d56/d6c/daa/f154 0 2024-04-25T00:18:39.272 INFO:tasks.workunit.client.0.smithi107.stdout:7/983: dread d0/d8/db/d18/d19/d47/d4d/dbc/f85 [0,4194304] 0 2024-04-25T00:18:39.272 INFO:tasks.workunit.client.0.smithi107.stdout:0/879: dread dc/f18 [4194304,4194304] 0 2024-04-25T00:18:39.273 INFO:tasks.workunit.client.0.smithi107.stdout:0/880: stat dc/dd/d5d/d3f/da5/fcf 0 2024-04-25T00:18:39.273 INFO:tasks.workunit.client.0.smithi107.stdout:0/881: write dc/dd/d5d/d3f/da5/fde [630145,109819] 0 2024-04-25T00:18:39.273 INFO:tasks.workunit.client.0.smithi107.stdout:9/978: link d0/d8a/d93/d56/d76/f46 d0/d8a/d93/d56/d6c/daa/f155 0 2024-04-25T00:18:39.273 INFO:tasks.workunit.client.0.smithi107.stdout:7/984: symlink d0/d8/d4e/d46/l14d 0 2024-04-25T00:18:39.274 INFO:tasks.workunit.client.0.smithi107.stdout:9/979: creat d0/d8a/d93/d56/d6c/d80/d9b/db2/dea/d10c/f156 x:0 0 0 2024-04-25T00:18:39.275 INFO:tasks.workunit.client.0.smithi107.stdout:7/985: creat d0/d8/db/d18/d19/d47/de0/d115/f14e x:0 0 0 2024-04-25T00:18:39.275 INFO:tasks.workunit.client.0.smithi107.stdout:6/771: dwrite d1/d14/de6/db/d3c/d43/d96/fb5 [0,4194304] 0 2024-04-25T00:18:39.275 INFO:tasks.workunit.client.0.smithi107.stdout:6/772: fsync d1/f1f 0 2024-04-25T00:18:39.275 INFO:tasks.workunit.client.0.smithi107.stdout:0/882: creat dc/d24/d38/d8b/f11c x:0 0 0 2024-04-25T00:18:39.276 INFO:tasks.workunit.client.0.smithi107.stdout:9/980: creat d0/d8a/d93/d56/d76/d50/d7d/f157 x:0 0 0 2024-04-25T00:18:39.276 INFO:tasks.workunit.client.0.smithi107.stdout:9/981: write d0/d8a/d93/d56/d6c/daa/fd5 [4283485,69760] 0 2024-04-25T00:18:39.277 INFO:tasks.workunit.client.0.smithi107.stdout:6/773: creat d1/d14/ffe x:0 0 0 2024-04-25T00:18:39.277 INFO:tasks.workunit.client.0.smithi107.stdout:6/774: chown d1/d14/de6/d2b 11552 1 2024-04-25T00:18:39.278 INFO:tasks.workunit.client.0.smithi107.stdout:7/986: rename d0/d8/db/d18/d19/d47/d4d/dbc/c7c to d0/d8/d4e/d46/d93/dd8/c14f 0 2024-04-25T00:18:39.279 INFO:tasks.workunit.client.0.smithi107.stdout:6/775: rmdir d1/d14/de6/d2b/d3e 39 2024-04-25T00:18:39.281 INFO:tasks.workunit.client.0.smithi107.stdout:6/776: rename d1/d14/de6/d3/d28/d79/l98 to d1/d14/de6/db/d3c/lff 0 2024-04-25T00:18:39.281 INFO:tasks.workunit.client.0.smithi107.stdout:0/883: dread dc/dd/d5d/d3f/da5/fa9 [0,4194304] 0 2024-04-25T00:18:39.284 INFO:tasks.workunit.client.0.smithi107.stdout:6/777: dread d1/d14/de6/db/d32/d36/d3a/f9a [0,4194304] 0 2024-04-25T00:18:39.284 INFO:tasks.workunit.client.0.smithi107.stdout:6/778: chown d1/d14/de6/db/d32/d33/d68/fa0 0 1 2024-04-25T00:18:39.284 INFO:tasks.workunit.client.0.smithi107.stdout:6/779: read - d1/d14/de6/db/d32/fc2 zero size 2024-04-25T00:18:39.291 INFO:tasks.workunit.client.0.smithi107.stdout:7/987: write d0/d8/d4e/d46/d93/fe1 [2388794,124275] 0 2024-04-25T00:18:39.292 INFO:tasks.workunit.client.0.smithi107.stdout:9/982: dwrite d0/d8a/d93/d6f/da1/ffc [0,4194304] 0 2024-04-25T00:18:39.292 INFO:tasks.workunit.client.0.smithi107.stdout:9/983: write d0/d8a/d93/d34/d36/d77/d110/f152 [117931,84836] 0 2024-04-25T00:18:39.294 INFO:tasks.workunit.client.0.smithi107.stdout:9/984: dread d0/d8a/d93/db9/fe6 [0,4194304] 0 2024-04-25T00:18:39.294 INFO:tasks.workunit.client.0.smithi107.stdout:9/985: readlink d0/d8a/d93/d34/d36/d77/d110/l129 0 2024-04-25T00:18:39.294 INFO:tasks.workunit.client.0.smithi107.stdout:7/988: link d0/d8/fff d0/d8/db/d18/d19/f150 0 2024-04-25T00:18:39.296 INFO:tasks.workunit.client.0.smithi107.stdout:9/986: rmdir d0/d8a/d93/d56/d6c/d80/d9b/db2 39 2024-04-25T00:18:39.296 INFO:tasks.workunit.client.0.smithi107.stdout:9/987: write d0/d8a/f135 [750999,15596] 0 2024-04-25T00:18:39.303 INFO:tasks.workunit.client.0.smithi107.stdout:0/884: dwrite dc/dd/d16/d48/ddb/d11b/d35/ffc [0,4194304] 0 2024-04-25T00:18:39.305 INFO:tasks.workunit.client.0.smithi107.stdout:0/885: creat dc/d24/d38/d64/d78/ddc/f11d x:0 0 0 2024-04-25T00:18:39.306 INFO:tasks.workunit.client.0.smithi107.stdout:0/886: creat dc/dd/d16/d48/ddb/d11b/d35/d10a/f11e x:0 0 0 2024-04-25T00:18:39.310 INFO:tasks.workunit.client.0.smithi107.stdout:6/780: dwrite d1/d14/ffe [0,4194304] 0 2024-04-25T00:18:39.322 INFO:tasks.workunit.client.0.smithi107.stdout:0/887: rmdir dc/d24/d38/d8b 39 2024-04-25T00:18:39.323 INFO:tasks.workunit.client.0.smithi107.stdout:7/989: rename d0/d8/db/d18/d19/c63 to d0/d8/db/d18/d19/d47/da1/dd4/c151 0 2024-04-25T00:18:39.323 INFO:tasks.workunit.client.0.smithi107.stdout:9/988: dwrite d0/d8a/d93/d56/d6c/d80/f97 [0,4194304] 0 2024-04-25T00:18:39.324 INFO:tasks.workunit.client.0.smithi107.stdout:6/781: dread d1/d14/de6/db/d48/fd2 [0,4194304] 0 2024-04-25T00:18:39.324 INFO:tasks.workunit.client.0.smithi107.stdout:9/989: fsync d0/d8a/d93/de/fdc 0 2024-04-25T00:18:39.324 INFO:tasks.workunit.client.0.smithi107.stdout:9/990: write d0/d8a/d93/d56/d76/f46 [1929752,9613] 0 2024-04-25T00:18:39.324 INFO:tasks.workunit.client.0.smithi107.stdout:7/990: rename d0/d8/d4e/d46/f6b to d0/d8/db/d18/d19/d47/de0/d115/d117/de5/f152 0 2024-04-25T00:18:39.326 INFO:tasks.workunit.client.0.smithi107.stdout:6/782: creat d1/d14/de6/d3/d71/f100 x:0 0 0 2024-04-25T00:18:39.326 INFO:tasks.workunit.client.0.smithi107.stdout:6/783: chown d1/d14/de6/db/d3c/d43 25872353 1 2024-04-25T00:18:39.328 INFO:tasks.workunit.client.0.smithi107.stdout:7/991: creat d0/d8/db/d18/d19/d47/de0/d115/d117/d58/f153 x:0 0 0 2024-04-25T00:18:39.328 INFO:tasks.workunit.client.0.smithi107.stdout:0/888: link dc/d24/d38/d64/d78/l84 dc/d24/d38/d64/d7b/dfe/l11f 0 2024-04-25T00:18:39.329 INFO:tasks.workunit.client.0.smithi107.stdout:9/991: mkdir d0/d8a/d93/d56/d76/d50/d7d/d158 0 2024-04-25T00:18:39.331 INFO:tasks.workunit.client.0.smithi107.stdout:9/992: mkdir d0/d8a/d93/d34/d36/d77/d159 0 2024-04-25T00:18:39.332 INFO:tasks.workunit.client.0.smithi107.stdout:0/889: rmdir dc/dd/d16/d48 39 2024-04-25T00:18:39.333 INFO:tasks.workunit.client.0.smithi107.stdout:0/890: chown dc/d24/d38/d64/d7b/f101 1148 1 2024-04-25T00:18:39.334 INFO:tasks.workunit.client.0.smithi107.stdout:0/891: creat dc/d24/d38/d64/d7b/f120 x:0 0 0 2024-04-25T00:18:39.337 INFO:tasks.workunit.client.0.smithi107.stdout:9/993: dread d0/d8a/d93/d34/f84 [0,4194304] 0 2024-04-25T00:18:39.341 INFO:tasks.workunit.client.0.smithi107.stdout:6/784: read d1/d14/de6/dad/dfb/f93 [4063416,60400] 0 2024-04-25T00:18:39.342 INFO:tasks.workunit.client.0.smithi107.stdout:7/992: dread d0/dc1/fd3 [0,4194304] 0 2024-04-25T00:18:39.343 INFO:tasks.workunit.client.0.smithi107.stdout:6/785: creat d1/d14/de6/db/d3c/d43/d96/dc9/f101 x:0 0 0 2024-04-25T00:18:39.343 INFO:tasks.workunit.client.0.smithi107.stdout:7/993: truncate d0/d8/db/d18/d19/d47/d4d/d8a/fa6 1497272 0 2024-04-25T00:18:39.345 INFO:tasks.workunit.client.0.smithi107.stdout:7/994: mknod d0/d8/db/d18/d19/d47/c154 0 2024-04-25T00:18:39.345 INFO:tasks.workunit.client.0.smithi107.stdout:6/786: mknod d1/d14/de6/d3/d26/de2/c102 0 2024-04-25T00:18:39.347 INFO:tasks.workunit.client.0.smithi107.stdout:7/995: mknod d0/d8/db/d18/d19/d47/de0/c155 0 2024-04-25T00:18:39.355 INFO:tasks.workunit.client.0.smithi107.stdout:0/892: rename dc/dd/d5d/c29 to dc/dd/d5d/d5c/d6e/c121 0 2024-04-25T00:18:39.359 INFO:tasks.workunit.client.0.smithi107.stdout:9/994: rename d0/d8a/d93/de/d5c/d95/fef to d0/d8a/d93/d56/d6c/d80/d113/f15a 0 2024-04-25T00:18:39.360 INFO:tasks.workunit.client.0.smithi107.stdout:9/995: creat d0/dfb/f15b x:0 0 0 2024-04-25T00:18:39.361 INFO:tasks.workunit.client.0.smithi107.stdout:9/996: getdents d0/d8a/d93/d6f 0 2024-04-25T00:18:39.365 INFO:tasks.workunit.client.0.smithi107.stdout:2/969: sync 2024-04-25T00:18:39.365 INFO:tasks.workunit.client.0.smithi107.stdout:1/883: sync 2024-04-25T00:18:39.365 INFO:tasks.workunit.client.0.smithi107.stdout:9/997: dread d0/d8a/d93/f99 [0,4194304] 0 2024-04-25T00:18:39.365 INFO:tasks.workunit.client.0.smithi107.stdout:9/998: chown d0/d8a/d93/d56/d76/d50/d7d/c118 2703 1 2024-04-25T00:18:39.366 INFO:tasks.workunit.client.0.smithi107.stdout:1/884: link d2/da/d2f/c45 d2/dd/d82/c121 0 2024-04-25T00:18:39.367 INFO:tasks.workunit.client.0.smithi107.stdout:9/999: symlink d0/dfb/d149/l15c 0 2024-04-25T00:18:39.367 INFO:tasks.workunit.client.0.smithi107.stdout:0/893: write dc/dd/d16/d5e/ff5 [3212785,20342] 0 2024-04-25T00:18:39.368 INFO:tasks.workunit.client.0.smithi107.stdout:1/885: unlink d2/da/d2f/c115 0 2024-04-25T00:18:39.368 INFO:tasks.workunit.client.0.smithi107.stdout:1/886: write d2/d62/d66/fc6 [4985830,29617] 0 2024-04-25T00:18:39.369 INFO:tasks.workunit.client.0.smithi107.stdout:1/887: creat d2/dd/dec/f122 x:0 0 0 2024-04-25T00:18:39.370 INFO:tasks.workunit.client.0.smithi107.stdout:6/787: dwrite d1/d14/de6/db/d32/d33/d68/d75/f8b [4194304,4194304] 0 2024-04-25T00:18:39.370 INFO:tasks.workunit.client.0.smithi107.stdout:1/888: creat d2/d2a/f123 x:0 0 0 2024-04-25T00:18:39.371 INFO:tasks.workunit.client.0.smithi107.stdout:7/996: dwrite d0/d8/db/d18/d19/d47/da1/dd4/ff0 [0,4194304] 0 2024-04-25T00:18:39.373 INFO:tasks.workunit.client.0.smithi107.stdout:1/889: unlink d2/d62/d66/fbe 0 2024-04-25T00:18:39.374 INFO:tasks.workunit.client.0.smithi107.stdout:7/997: creat d0/d8/d4e/d11f/d14a/f156 x:0 0 0 2024-04-25T00:18:39.375 INFO:tasks.workunit.client.0.smithi107.stdout:6/788: creat d1/d14/de6/f103 x:0 0 0 2024-04-25T00:18:39.376 INFO:tasks.workunit.client.0.smithi107.stdout:1/890: unlink d2/dd/f4e 0 2024-04-25T00:18:39.376 INFO:tasks.workunit.client.0.smithi107.stdout:2/970: write d6/d20/d22/d56/f68 [2324583,35882] 0 2024-04-25T00:18:39.377 INFO:tasks.workunit.client.0.smithi107.stdout:1/891: readlink d2/da/lf 0 2024-04-25T00:18:39.377 INFO:tasks.workunit.client.0.smithi107.stdout:1/892: write d2/dd/fdd [1997511,20677] 0 2024-04-25T00:18:39.380 INFO:tasks.workunit.client.0.smithi107.stdout:7/998: write d0/d8/d4e/d46/d93/fbd [752348,28465] 0 2024-04-25T00:18:39.381 INFO:tasks.workunit.client.0.smithi107.stdout:7/999: fdatasync d0/d8/db/d18/d19/d47/de0/d115/d117/d58/f84 0 2024-04-25T00:18:39.388 INFO:tasks.workunit.client.0.smithi107.stdout:5/859: sync 2024-04-25T00:18:39.388 INFO:tasks.workunit.client.0.smithi107.stdout:4/985: sync 2024-04-25T00:18:39.389 INFO:tasks.workunit.client.0.smithi107.stdout:0/894: link dc/dd/d16/d48/ddb/d11b/fb6 dc/d24/d38/d64/d78/de6/f122 0 2024-04-25T00:18:39.391 INFO:tasks.workunit.client.0.smithi107.stdout:5/860: mknod d4/d17/db3/d116/d51/d78/c123 0 2024-04-25T00:18:39.391 INFO:tasks.workunit.client.0.smithi107.stdout:5/861: chown d4/d17/db3/d84/d93/dc4/fe2 120020 1 2024-04-25T00:18:39.391 INFO:tasks.workunit.client.0.smithi107.stdout:4/986: rmdir d3/d24/d25/d71 39 2024-04-25T00:18:39.392 INFO:tasks.workunit.client.0.smithi107.stdout:0/895: rmdir dc/dd/d5d/d3f/dc7 39 2024-04-25T00:18:39.392 INFO:tasks.workunit.client.0.smithi107.stdout:0/896: getdents dc/dd/d16/d48/d83/d9f/dba 0 2024-04-25T00:18:39.394 INFO:tasks.workunit.client.0.smithi107.stdout:5/862: creat d4/d7c/d88/d9d/deb/f124 x:0 0 0 2024-04-25T00:18:39.395 INFO:tasks.workunit.client.0.smithi107.stdout:2/971: dread d6/d18/d40/d42/f3b [0,4194304] 0 2024-04-25T00:18:39.395 INFO:tasks.workunit.client.0.smithi107.stdout:2/972: stat d6/d20/d22/d56/d94/d10b/d128/f9b 0 2024-04-25T00:18:39.396 INFO:tasks.workunit.client.0.smithi107.stdout:5/863: mknod d4/d1c/d5c/dd4/d107/d10d/c125 0 2024-04-25T00:18:39.398 INFO:tasks.workunit.client.0.smithi107.stdout:2/973: rename d6/d18/d40/d42/d69/d6e/db6/fdd to d6/d18/d40/d42/d69/d6e/dc2/f14a 0 2024-04-25T00:18:39.400 INFO:tasks.workunit.client.0.smithi107.stdout:5/864: mkdir d4/d7c/d126 0 2024-04-25T00:18:39.401 INFO:tasks.workunit.client.0.smithi107.stdout:2/974: mknod d6/d18/d40/d42/d70/c14b 0 2024-04-25T00:18:39.401 INFO:tasks.workunit.client.0.smithi107.stdout:2/975: fdatasync d6/d18/d40/d42/d2b/f13b 0 2024-04-25T00:18:39.406 INFO:tasks.workunit.client.0.smithi107.stdout:1/893: dwrite d2/da/db7/f11d [0,4194304] 0 2024-04-25T00:18:39.406 INFO:tasks.workunit.client.0.smithi107.stdout:1/894: dread - d2/d62/d66/d70/dcc/f112 zero size 2024-04-25T00:18:39.412 INFO:tasks.workunit.client.0.smithi107.stdout:1/895: dread d2/d62/d66/d70/f8c [0,4194304] 0 2024-04-25T00:18:39.413 INFO:tasks.workunit.client.0.smithi107.stdout:4/987: dwrite d3/d24/d25/d71/ff0 [4194304,4194304] 0 2024-04-25T00:18:39.413 INFO:tasks.workunit.client.0.smithi107.stdout:1/896: rmdir d2/da/d2f 39 2024-04-25T00:18:39.414 INFO:tasks.workunit.client.0.smithi107.stdout:1/897: chown d2/da4/fcb 1049101600 1 2024-04-25T00:18:39.415 INFO:tasks.workunit.client.0.smithi107.stdout:4/988: unlink d3/d5/f12f 0 2024-04-25T00:18:39.419 INFO:tasks.workunit.client.0.smithi107.stdout:1/898: creat d2/f124 x:0 0 0 2024-04-25T00:18:39.419 INFO:tasks.workunit.client.0.smithi107.stdout:1/899: fdatasync d2/dd/d1a/f64 0 2024-04-25T00:18:39.429 INFO:tasks.workunit.client.0.smithi107.stdout:5/865: dwrite d4/d17/db3/d116/d2e/d55/fcb [0,4194304] 0 2024-04-25T00:18:39.431 INFO:tasks.workunit.client.0.smithi107.stdout:5/866: rmdir d4/d91 39 2024-04-25T00:18:39.431 INFO:tasks.workunit.client.0.smithi107.stdout:5/867: mkdir d4/d17/db3/d116/d2e/d55/d127 0 2024-04-25T00:18:39.432 INFO:tasks.workunit.client.0.smithi107.stdout:5/868: fsync d4/fd7 0 2024-04-25T00:18:39.432 INFO:tasks.workunit.client.0.smithi107.stdout:5/869: truncate d4/d7c/d88/d9d/fd9 1147776 0 2024-04-25T00:18:39.432 INFO:tasks.workunit.client.0.smithi107.stdout:5/870: chown d4/dc/d83/fcc 412 1 2024-04-25T00:18:39.433 INFO:tasks.workunit.client.0.smithi107.stdout:2/976: dwrite d6/d18/d40/d42/d2b/d30/f7c [0,4194304] 0 2024-04-25T00:18:39.434 INFO:tasks.workunit.client.0.smithi107.stdout:5/871: rename d4/d17/db3/d116/d51/d78/fdb to d4/dc/d26/d62/dd5/f128 0 2024-04-25T00:18:39.437 INFO:tasks.workunit.client.0.smithi107.stdout:2/977: unlink d6/d20/d22/laf 0 2024-04-25T00:18:39.438 INFO:tasks.workunit.client.0.smithi107.stdout:5/872: unlink d4/dc/f13 0 2024-04-25T00:18:39.439 INFO:tasks.workunit.client.0.smithi107.stdout:5/873: mknod d4/d1c/d5c/dd4/d107/c129 0 2024-04-25T00:18:39.440 INFO:tasks.workunit.client.0.smithi107.stdout:5/874: symlink d4/dc/d83/l12a 0 2024-04-25T00:18:39.443 INFO:tasks.workunit.client.0.smithi107.stdout:4/989: dwrite d3/d24/f2c [4194304,4194304] 0 2024-04-25T00:18:39.447 INFO:tasks.workunit.client.0.smithi107.stdout:1/900: dwrite d2/d2a/f123 [0,4194304] 0 2024-04-25T00:18:39.447 INFO:tasks.workunit.client.0.smithi107.stdout:4/990: rename d3/d5/f8f to d3/d24/d25/d71/d7f/f149 0 2024-04-25T00:18:39.447 INFO:tasks.workunit.client.0.smithi107.stdout:4/991: chown d3/d24/db1 138365389 1 2024-04-25T00:18:39.447 INFO:tasks.workunit.client.0.smithi107.stdout:4/992: fsync d3/d5/d32/d53/d97/f99 0 2024-04-25T00:18:39.449 INFO:tasks.workunit.client.0.smithi107.stdout:1/901: rename d2/le2 to d2/d2a/l125 0 2024-04-25T00:18:39.453 INFO:tasks.workunit.client.0.smithi107.stdout:5/875: write d4/d17/db3/d84/df0/f3c [57524,111373] 0 2024-04-25T00:18:39.455 INFO:tasks.workunit.client.0.smithi107.stdout:5/876: creat d4/d17/db3/d116/f12b x:0 0 0 2024-04-25T00:18:39.456 INFO:tasks.workunit.client.0.smithi107.stdout:2/978: dwrite d6/d18/d40/d42/d69/d6e/dc2/f14a [0,4194304] 0 2024-04-25T00:18:39.456 INFO:tasks.workunit.client.0.smithi107.stdout:2/979: stat d6/d29/d4e/d6f 0 2024-04-25T00:18:39.456 INFO:tasks.workunit.client.0.smithi107.stdout:2/980: dread - d6/d20/d22/d56/d94/d10b/f141 zero size 2024-04-25T00:18:39.459 INFO:tasks.workunit.client.0.smithi107.stdout:2/981: rename d6/d18/d40/dc8/de5/f147 to d6/d18/d40/d42/d2b/d30/d45/f14c 0 2024-04-25T00:18:39.461 INFO:tasks.workunit.client.0.smithi107.stdout:1/902: dread d2/dd/d25/f7d [0,4194304] 0 2024-04-25T00:18:39.463 INFO:tasks.workunit.client.0.smithi107.stdout:2/982: symlink d6/d20/l14d 0 2024-04-25T00:18:39.464 INFO:tasks.workunit.client.0.smithi107.stdout:1/903: mkdir d2/d62/dfa/d126 0 2024-04-25T00:18:39.465 INFO:tasks.workunit.client.0.smithi107.stdout:4/993: read d3/d10/f8b [2294959,48884] 0 2024-04-25T00:18:39.465 INFO:tasks.workunit.client.0.smithi107.stdout:4/994: read - d3/d24/d25/d71/dc2/f136 zero size 2024-04-25T00:18:39.465 INFO:tasks.workunit.client.0.smithi107.stdout:4/995: truncate d3/d24/db1/d126/d132/f125 773471 0 2024-04-25T00:18:39.466 INFO:tasks.workunit.client.0.smithi107.stdout:1/904: link d2/d62/d66/d70/c10d d2/d2a/d5c/c127 0 2024-04-25T00:18:39.466 INFO:tasks.workunit.client.0.smithi107.stdout:4/996: symlink d3/d24/l14a 0 2024-04-25T00:18:39.467 INFO:tasks.workunit.client.0.smithi107.stdout:4/997: creat d3/d24/d25/d71/f14b x:0 0 0 2024-04-25T00:18:39.468 INFO:tasks.workunit.client.0.smithi107.stdout:4/998: rename d3/d10/l50 to d3/d24/d25/d71/d7f/l14c 0 2024-04-25T00:18:39.469 INFO:tasks.workunit.client.0.smithi107.stdout:4/999: rmdir d3/d24/d5c/dab/da3/d13c 0 2024-04-25T00:18:39.472 INFO:tasks.workunit.client.0.smithi107.stdout:2/983: dread d6/d18/fb5 [0,4194304] 0 2024-04-25T00:18:39.473 INFO:tasks.workunit.client.0.smithi107.stdout:2/984: creat d6/d18/d40/d42/f14e x:0 0 0 2024-04-25T00:18:39.474 INFO:tasks.workunit.client.0.smithi107.stdout:2/985: unlink d6/d18/d40/d42/d2b/d30/d45/fb9 0 2024-04-25T00:18:39.474 INFO:tasks.workunit.client.0.smithi107.stdout:2/986: truncate d6/d18/d40/d42/d2b/d30/d45/f10d 337341 0 2024-04-25T00:18:39.476 INFO:tasks.workunit.client.0.smithi107.stdout:2/987: dread d6/d18/d40/d42/d2b/d30/d45/f5f [0,4194304] 0 2024-04-25T00:18:39.476 INFO:tasks.workunit.client.0.smithi107.stdout:2/988: stat d6/d29/d4e/dc0/dd1 0 2024-04-25T00:18:39.495 INFO:tasks.workunit.client.0.smithi107.stdout:5/877: dwrite d4/d17/db3/d116/d51/f67 [0,4194304] 0 2024-04-25T00:18:39.495 INFO:tasks.workunit.client.0.smithi107.stdout:1/905: dwrite d2/dd/d25/f101 [0,4194304] 0 2024-04-25T00:18:39.498 INFO:tasks.workunit.client.0.smithi107.stdout:2/989: dwrite d6/dad/d67/f49 [4194304,4194304] 0 2024-04-25T00:18:39.500 INFO:tasks.workunit.client.0.smithi107.stdout:1/906: rename d2/da/db7/ff6 to d2/da4/f128 0 2024-04-25T00:18:39.507 INFO:tasks.workunit.client.0.smithi107.stdout:1/907: dread d2/da/d2f/f9b [0,4194304] 0 2024-04-25T00:18:39.507 INFO:tasks.workunit.client.0.smithi107.stdout:1/908: stat d2/dd/l48 0 2024-04-25T00:18:39.507 INFO:tasks.workunit.client.0.smithi107.stdout:1/909: chown d2/da4/fcb 0 1 2024-04-25T00:18:39.507 INFO:tasks.workunit.client.0.smithi107.stdout:1/910: read d2/dd/d25/d5e/f81 [423936,61738] 0 2024-04-25T00:18:39.508 INFO:tasks.workunit.client.0.smithi107.stdout:1/911: creat d2/d62/dfa/d126/f129 x:0 0 0 2024-04-25T00:18:39.508 INFO:tasks.workunit.client.0.smithi107.stdout:1/912: dread - d2/d62/dfa/d126/f129 zero size 2024-04-25T00:18:39.519 INFO:tasks.workunit.client.0.smithi107.stdout:1/913: dread d2/da/f53 [4194304,4194304] 0 2024-04-25T00:18:39.520 INFO:tasks.workunit.client.0.smithi107.stdout:5/878: dwrite d4/d17/db3/d116/d2e/d82/fa6 [0,4194304] 0 2024-04-25T00:18:39.520 INFO:tasks.workunit.client.0.smithi107.stdout:5/879: dread - d4/d17/db3/d116/d51/fe5 zero size 2024-04-25T00:18:39.520 INFO:tasks.workunit.client.0.smithi107.stdout:1/914: write d2/dd/d25/da9/fb5 [859166,108994] 0 2024-04-25T00:18:39.521 INFO:tasks.workunit.client.0.smithi107.stdout:2/990: dwrite d6/d18/d40/d42/d2b/d30/d45/f5f [0,4194304] 0 2024-04-25T00:18:39.521 INFO:tasks.workunit.client.0.smithi107.stdout:2/991: stat d6/d18/d40/dc8/f111 0 2024-04-25T00:18:39.521 INFO:tasks.workunit.client.0.smithi107.stdout:2/992: dread - d6/d18/d40/d42/d69/d6e/da2/dc5/fd8 zero size 2024-04-25T00:18:39.521 INFO:tasks.workunit.client.0.smithi107.stdout:2/993: dread - d6/d18/d40/d42/d69/d6e/da2/d10c/d102/d9f/fe2 zero size 2024-04-25T00:18:39.521 INFO:tasks.workunit.client.0.smithi107.stdout:2/994: fsync d6/dad/d67/f49 0 2024-04-25T00:18:39.526 INFO:tasks.workunit.client.0.smithi107.stdout:2/995: mknod d6/d18/d40/d42/d2b/d30/d4c/c14f 0 2024-04-25T00:18:39.529 INFO:tasks.workunit.client.0.smithi107.stdout:1/915: rename d2/dd/d25/da9 to d2/dd/d25/d12a 0 2024-04-25T00:18:39.529 INFO:tasks.workunit.client.0.smithi107.stdout:1/916: chown d2/c3 6150 1 2024-04-25T00:18:39.534 INFO:tasks.workunit.client.0.smithi107.stdout:1/917: write d2/d2a/d37/d88/fb2 [659201,46582] 0 2024-04-25T00:18:39.534 INFO:tasks.workunit.client.0.smithi107.stdout:0/897: sync 2024-04-25T00:18:39.534 INFO:tasks.workunit.client.0.smithi107.stdout:6/789: sync 2024-04-25T00:18:39.535 INFO:tasks.workunit.client.0.smithi107.stdout:0/898: chown dc/fcb 14778 1 2024-04-25T00:18:39.536 INFO:tasks.workunit.client.0.smithi107.stdout:0/899: creat dc/dd/d5d/d7e/f123 x:0 0 0 2024-04-25T00:18:39.536 INFO:tasks.workunit.client.0.smithi107.stdout:0/900: write dc/d24/d38/d64/d78/ddc/f11d [39824,53944] 0 2024-04-25T00:18:39.537 INFO:tasks.workunit.client.0.smithi107.stdout:6/790: creat d1/d14/de6/d2b/d3e/d5b/d7a/f104 x:0 0 0 2024-04-25T00:18:39.538 INFO:tasks.workunit.client.0.smithi107.stdout:6/791: mknod d1/d14/de6/db/d32/d36/d7d/c105 0 2024-04-25T00:18:39.539 INFO:tasks.workunit.client.0.smithi107.stdout:5/880: dwrite d4/d17/db3/d84/d93/f98 [0,4194304] 0 2024-04-25T00:18:39.541 INFO:tasks.workunit.client.0.smithi107.stdout:6/792: unlink d1/d14/de6/db/d32/d36/d3d/l84 0 2024-04-25T00:18:39.543 INFO:tasks.workunit.client.0.smithi107.stdout:5/881: creat d4/dc/d26/d62/dcf/f12c x:0 0 0 2024-04-25T00:18:39.545 INFO:tasks.workunit.client.0.smithi107.stdout:1/918: write d2/dd/d25/f2e [1061782,27956] 0 2024-04-25T00:18:39.545 INFO:tasks.workunit.client.0.smithi107.stdout:5/882: rmdir d4/d1c/d5c 39 2024-04-25T00:18:39.550 INFO:tasks.workunit.client.0.smithi107.stdout:2/996: dwrite d6/d18/d40/d42/d2b/f11c [0,4194304] 0 2024-04-25T00:18:39.553 INFO:tasks.workunit.client.0.smithi107.stdout:5/883: write d4/dc/fe [2347313,69107] 0 2024-04-25T00:18:39.554 INFO:tasks.workunit.client.0.smithi107.stdout:2/997: rmdir d6/d29/d4e/d6f/df1 0 2024-04-25T00:18:39.554 INFO:tasks.workunit.client.0.smithi107.stdout:2/998: stat d6/d18/d40/d42/d69/d6e/da2/dc5/dd2/cf5 0 2024-04-25T00:18:39.554 INFO:tasks.workunit.client.0.smithi107.stdout:2/999: mknod d6/d18/d12e/c150 0 2024-04-25T00:18:39.555 INFO:tasks.workunit.client.0.smithi107.stdout:6/793: dread d1/d4c/fac [0,4194304] 0 2024-04-25T00:18:39.557 INFO:tasks.workunit.client.0.smithi107.stdout:6/794: creat d1/d14/de6/dee/df5/f106 x:0 0 0 2024-04-25T00:18:39.557 INFO:tasks.workunit.client.0.smithi107.stdout:6/795: write d1/d14/de6/db/d32/fc2 [388152,120558] 0 2024-04-25T00:18:39.558 INFO:tasks.workunit.client.0.smithi107.stdout:6/796: creat d1/d14/de6/db/d32/d36/f107 x:0 0 0 2024-04-25T00:18:39.560 INFO:tasks.workunit.client.0.smithi107.stdout:6/797: getdents d1/d14/de6/d2b/d3e/d5b 0 2024-04-25T00:18:39.560 INFO:tasks.workunit.client.0.smithi107.stdout:6/798: readlink d1/d14/de6/d3/l42 0 2024-04-25T00:18:39.561 INFO:tasks.workunit.client.0.smithi107.stdout:6/799: symlink d1/d14/de6/db/d32/dca/l108 0 2024-04-25T00:18:39.562 INFO:tasks.workunit.client.0.smithi107.stdout:1/919: dread d2/dd/d1a/f64 [0,4194304] 0 2024-04-25T00:18:39.562 INFO:tasks.workunit.client.0.smithi107.stdout:1/920: stat d2/dfe 0 2024-04-25T00:18:39.562 INFO:tasks.workunit.client.0.smithi107.stdout:0/901: dwrite dc/dd/d5d/d3f/da5/fa9 [0,4194304] 0 2024-04-25T00:18:39.563 INFO:tasks.workunit.client.0.smithi107.stdout:6/800: unlink d1/d14/de6/d3/l42 0 2024-04-25T00:18:39.564 INFO:tasks.workunit.client.0.smithi107.stdout:1/921: rename d2/d2a/d37/de8/d105 to d2/d2a/d37/d88/d12b 0 2024-04-25T00:18:39.564 INFO:tasks.workunit.client.0.smithi107.stdout:1/922: readlink d2/d2a/d37/l86 0 2024-04-25T00:18:39.564 INFO:tasks.workunit.client.0.smithi107.stdout:0/902: unlink dc/dd/d16/d48/ddb/d11b/d35/d5f/leb 0 2024-04-25T00:18:39.564 INFO:tasks.workunit.client.0.smithi107.stdout:0/903: chown dc/d24/d38/d64/d78/ddc/f11d 778154 1 2024-04-25T00:18:39.567 INFO:tasks.workunit.client.0.smithi107.stdout:1/923: mkdir d2/d120/d12c 0 2024-04-25T00:18:39.567 INFO:tasks.workunit.client.0.smithi107.stdout:1/924: dread - d2/da/d2f/f102 zero size 2024-04-25T00:18:39.574 INFO:tasks.workunit.client.0.smithi107.stdout:5/884: dwrite d4/d17/db3/d84/d93/dc4/f120 [0,4194304] 0 2024-04-25T00:18:39.579 INFO:tasks.workunit.client.0.smithi107.stdout:6/801: dread d1/d14/f89 [0,4194304] 0 2024-04-25T00:18:39.580 INFO:tasks.workunit.client.0.smithi107.stdout:6/802: symlink d1/d14/de6/db/d32/d36/d7d/l109 0 2024-04-25T00:18:39.583 INFO:tasks.workunit.client.0.smithi107.stdout:5/885: write d4/d1c/d5c/fe3 [618594,70266] 0 2024-04-25T00:18:39.583 INFO:tasks.workunit.client.0.smithi107.stdout:6/803: rmdir d1/d14/de6/db/d32/d36/d3d 39 2024-04-25T00:18:39.584 INFO:tasks.workunit.client.0.smithi107.stdout:0/904: dwrite dc/dd/d16/d48/ddb/d11b/d35/d5f/f82 [0,4194304] 0 2024-04-25T00:18:39.585 INFO:tasks.workunit.client.0.smithi107.stdout:5/886: getdents d4/d17/db3/d116/d51 0 2024-04-25T00:18:39.586 INFO:tasks.workunit.client.0.smithi107.stdout:6/804: creat d1/d14/de6/db/d48/f10a x:0 0 0 2024-04-25T00:18:39.588 INFO:tasks.workunit.client.0.smithi107.stdout:5/887: symlink d4/d17/db3/d84/d93/dc4/l12d 0 2024-04-25T00:18:39.590 INFO:tasks.workunit.client.0.smithi107.stdout:6/805: symlink d1/d14/de6/d2b/d3e/d5b/l10b 0 2024-04-25T00:18:39.593 INFO:tasks.workunit.client.0.smithi107.stdout:1/925: dwrite d2/d2a/d5c/d89/fdb [0,4194304] 0 2024-04-25T00:18:39.593 INFO:tasks.workunit.client.0.smithi107.stdout:1/926: dread - d2/da/f10b zero size 2024-04-25T00:18:39.593 INFO:tasks.workunit.client.0.smithi107.stdout:1/927: stat d2/dd/d82/dc1/fd3 0 2024-04-25T00:18:39.594 INFO:tasks.workunit.client.0.smithi107.stdout:1/928: write d2/d62/dfa/d126/f129 [776696,28365] 0 2024-04-25T00:18:39.594 INFO:tasks.workunit.client.0.smithi107.stdout:1/929: chown d2/dd/d82 14977127 1 2024-04-25T00:18:39.594 INFO:tasks.workunit.client.0.smithi107.stdout:1/930: chown d2/dd 116 1 2024-04-25T00:18:39.594 INFO:tasks.workunit.client.0.smithi107.stdout:6/806: rename d1/d14/de6/d3/d28/ff8 to d1/d14/de6/db/f10c 0 2024-04-25T00:18:39.595 INFO:tasks.workunit.client.0.smithi107.stdout:1/931: creat d2/d2a/d37/de8/d11a/f12d x:0 0 0 2024-04-25T00:18:39.597 INFO:tasks.workunit.client.0.smithi107.stdout:1/932: rename d2/da/d2f/f8d to d2/d62/d66/de5/f12e 0 2024-04-25T00:18:39.598 INFO:tasks.workunit.client.0.smithi107.stdout:1/933: unlink d2/d2a/f38 0 2024-04-25T00:18:39.602 INFO:tasks.workunit.client.0.smithi107.stdout:5/888: write d4/dc/d83/df1/f10c [3625888,118011] 0 2024-04-25T00:18:39.602 INFO:tasks.workunit.client.0.smithi107.stdout:5/889: chown d4/dc 8500291 1 2024-04-25T00:18:39.604 INFO:tasks.workunit.client.0.smithi107.stdout:5/890: creat d4/d7c/d88/d9d/de1/f12e x:0 0 0 2024-04-25T00:18:39.607 INFO:tasks.workunit.client.0.smithi107.stdout:5/891: mkdir d4/d17/db3/d84/d12f 0 2024-04-25T00:18:39.607 INFO:tasks.workunit.client.0.smithi107.stdout:5/892: chown d4/d17/db3/d84/df0/d106 15694143 1 2024-04-25T00:18:39.607 INFO:tasks.workunit.client.0.smithi107.stdout:5/893: chown d4/d17/db3/d116/d2e/le6 1443899 1 2024-04-25T00:18:39.609 INFO:tasks.workunit.client.0.smithi107.stdout:5/894: unlink d4/d17/db3/d116/d2e/lc2 0 2024-04-25T00:18:39.609 INFO:tasks.workunit.client.0.smithi107.stdout:5/895: chown d4/d1c/d5c 3411900 1 2024-04-25T00:18:39.609 INFO:tasks.workunit.client.0.smithi107.stdout:6/807: dwrite d1/d14/de6/db/fd [0,4194304] 0 2024-04-25T00:18:39.609 INFO:tasks.workunit.client.0.smithi107.stdout:6/808: write d1/d14/de6/db/d32/d33/f63 [586127,72590] 0 2024-04-25T00:18:39.610 INFO:tasks.workunit.client.0.smithi107.stdout:5/896: symlink d4/d17/db3/l130 0 2024-04-25T00:18:39.616 INFO:tasks.workunit.client.0.smithi107.stdout:1/934: dwrite d2/f124 [0,4194304] 0 2024-04-25T00:18:39.616 INFO:tasks.workunit.client.0.smithi107.stdout:1/935: chown d2/d62/cf7 101169437 1 2024-04-25T00:18:39.620 INFO:tasks.workunit.client.0.smithi107.stdout:1/936: mkdir d2/d2a/d5c/d12f 0 2024-04-25T00:18:39.620 INFO:tasks.workunit.client.0.smithi107.stdout:1/937: write d2/dd/d25/d5e/f94 [1577506,85082] 0 2024-04-25T00:18:39.622 INFO:tasks.workunit.client.0.smithi107.stdout:1/938: link d2/dd/d1b/c5b d2/d2a/d5c/d89/c130 0 2024-04-25T00:18:39.624 INFO:tasks.workunit.client.0.smithi107.stdout:1/939: mkdir d2/d2a/d37/de8/d131 0 2024-04-25T00:18:39.626 INFO:tasks.workunit.client.0.smithi107.stdout:1/940: mknod d2/d120/d12c/c132 0 2024-04-25T00:18:39.628 INFO:tasks.workunit.client.0.smithi107.stdout:1/941: link d2/da/l14 d2/da4/l133 0 2024-04-25T00:18:39.630 INFO:tasks.workunit.client.0.smithi107.stdout:1/942: creat d2/dd/d1a/f134 x:0 0 0 2024-04-25T00:18:39.630 INFO:tasks.workunit.client.0.smithi107.stdout:1/943: read - d2/d108/f10c zero size 2024-04-25T00:18:39.748 INFO:tasks.workunit.client.0.smithi107.stdout:5/897: sync 2024-04-25T00:18:39.748 INFO:tasks.workunit.client.0.smithi107.stdout:6/809: sync 2024-04-25T00:18:39.748 INFO:tasks.workunit.client.0.smithi107.stdout:5/898: dread - d4/d17/db3/d116/d2e/d55/f10f zero size 2024-04-25T00:18:39.749 INFO:tasks.workunit.client.0.smithi107.stdout:0/905: sync 2024-04-25T00:18:39.749 INFO:tasks.workunit.client.0.smithi107.stdout:6/810: chown d1/d14/de6/d3/d28/f53 7526244 1 2024-04-25T00:18:39.749 INFO:tasks.workunit.client.0.smithi107.stdout:5/899: truncate d4/d17/db3/d116/d51/fe5 946341 0 2024-04-25T00:18:39.750 INFO:tasks.workunit.client.0.smithi107.stdout:5/900: link d4/dc/d83/dfc/lff d4/d1c/d5c/dd4/d107/d10d/l131 0 2024-04-25T00:18:39.750 INFO:tasks.workunit.client.0.smithi107.stdout:6/811: unlink d1/d14/de6/db/d32/d36/d3a/f9a 0 2024-04-25T00:18:39.750 INFO:tasks.workunit.client.0.smithi107.stdout:0/906: mkdir dc/dd/d16/d48/ddb/d11b/d124 0 2024-04-25T00:18:39.750 INFO:tasks.workunit.client.0.smithi107.stdout:0/907: chown dc/d24/de7/ff3 42 1 2024-04-25T00:18:39.751 INFO:tasks.workunit.client.0.smithi107.stdout:5/901: mknod d4/d1c/d5c/c132 0 2024-04-25T00:18:39.751 INFO:tasks.workunit.client.0.smithi107.stdout:5/902: fdatasync d4/d17/db3/d116/fc1 0 2024-04-25T00:18:39.751 INFO:tasks.workunit.client.0.smithi107.stdout:0/908: mknod dc/dd/d16/d48/ddb/d11b/d35/c125 0 2024-04-25T00:18:39.751 INFO:tasks.workunit.client.0.smithi107.stdout:0/909: write dc/dd/d5d/d3f/dc7/f118 [469735,108369] 0 2024-04-25T00:18:39.752 INFO:tasks.workunit.client.0.smithi107.stdout:0/910: truncate dc/dd/d16/d48/ddb/d11b/d35/d10a/f11e 18007 0 2024-04-25T00:18:39.752 INFO:tasks.workunit.client.0.smithi107.stdout:0/911: symlink dc/dd/d5d/d3f/da5/l126 0 2024-04-25T00:18:39.753 INFO:tasks.workunit.client.0.smithi107.stdout:0/912: mknod dc/dd/d5d/d103/c127 0 2024-04-25T00:18:39.754 INFO:tasks.workunit.client.0.smithi107.stdout:0/913: write dc/d24/d38/d8b/f97 [831545,38214] 0 2024-04-25T00:18:39.754 INFO:tasks.workunit.client.0.smithi107.stdout:0/914: write dc/dd/d5d/d3f/dc7/f118 [872465,108015] 0 2024-04-25T00:18:39.754 INFO:tasks.workunit.client.0.smithi107.stdout:0/915: readlink dc/dd/d5d/l4f 0 2024-04-25T00:18:39.759 INFO:tasks.workunit.client.0.smithi107.stdout:6/812: read d1/d14/de6/dad/dfb/f58 [19717,72200] 0 2024-04-25T00:18:39.759 INFO:tasks.workunit.client.0.smithi107.stdout:6/813: chown d1/d14/d18/l2d 4 1 2024-04-25T00:18:39.761 INFO:tasks.workunit.client.0.smithi107.stdout:6/814: link d1/d14/de6/d3/d26/f35 d1/d14/de6/db/d32/f10d 0 2024-04-25T00:18:39.761 INFO:tasks.workunit.client.0.smithi107.stdout:5/903: dread d4/d17/db3/d116/fc1 [0,4194304] 0 2024-04-25T00:18:39.761 INFO:tasks.workunit.client.0.smithi107.stdout:6/815: mkdir d1/d14/de6/db/d32/d10e 0 2024-04-25T00:18:39.761 INFO:tasks.workunit.client.0.smithi107.stdout:6/816: stat d1/d14/de6/d3/d26/fa2 0 2024-04-25T00:18:39.762 INFO:tasks.workunit.client.0.smithi107.stdout:6/817: read d1/d14/de6/db/d3c/d43/d96/dc9/fdf [737558,60457] 0 2024-04-25T00:18:39.762 INFO:tasks.workunit.client.0.smithi107.stdout:6/818: fdatasync d1/d14/de6/db/d48/fbb 0 2024-04-25T00:18:39.762 INFO:tasks.workunit.client.0.smithi107.stdout:6/819: dread - d1/d14/de6/db/d48/fbb zero size 2024-04-25T00:18:39.763 INFO:tasks.workunit.client.0.smithi107.stdout:6/820: symlink d1/d14/de6/db/d3c/d43/d96/l10f 0 2024-04-25T00:18:39.764 INFO:tasks.workunit.client.0.smithi107.stdout:6/821: mknod d1/d14/de6/d3/d28/d52/dd0/c110 0 2024-04-25T00:18:39.764 INFO:tasks.workunit.client.0.smithi107.stdout:6/822: chown d1/d14/de6/d3/d28/d79 8089938 1 2024-04-25T00:18:39.765 INFO:tasks.workunit.client.0.smithi107.stdout:0/916: dread dc/d24/d38/f66 [0,4194304] 0 2024-04-25T00:18:39.766 INFO:tasks.workunit.client.0.smithi107.stdout:0/917: stat dc/d24/d38/d64/d78/ca7 0 2024-04-25T00:18:39.767 INFO:tasks.workunit.client.0.smithi107.stdout:0/918: truncate dc/dd/d5d/d5c/d6e/f99 513636 0 2024-04-25T00:18:39.775 INFO:tasks.workunit.client.0.smithi107.stdout:6/823: dread d1/d14/de6/db/f4d [0,4194304] 0 2024-04-25T00:18:39.780 INFO:tasks.workunit.client.0.smithi107.stdout:5/904: dread d4/dc/f1f [0,4194304] 0 2024-04-25T00:18:39.780 INFO:tasks.workunit.client.0.smithi107.stdout:6/824: write d1/d14/de6/d3/d71/d94/fb0 [4632894,44861] 0 2024-04-25T00:18:39.781 INFO:tasks.workunit.client.0.smithi107.stdout:5/905: creat d4/d17/db3/d84/df0/dfb/f133 x:0 0 0 2024-04-25T00:18:39.783 INFO:tasks.workunit.client.0.smithi107.stdout:5/906: creat d4/dc/d83/f134 x:0 0 0 2024-04-25T00:18:39.784 INFO:tasks.workunit.client.0.smithi107.stdout:5/907: creat d4/d1c/d5c/f135 x:0 0 0 2024-04-25T00:18:39.784 INFO:tasks.workunit.client.0.smithi107.stdout:5/908: fsync d4/dc/d26/d62/dd5/f128 0 2024-04-25T00:18:39.789 INFO:tasks.workunit.client.0.smithi107.stdout:5/909: rename d4/dc/l1d to d4/dc/l136 0 2024-04-25T00:18:39.794 INFO:tasks.workunit.client.0.smithi107.stdout:0/919: dwrite dc/d24/fe2 [0,4194304] 0 2024-04-25T00:18:39.794 INFO:tasks.workunit.client.0.smithi107.stdout:0/920: read dc/dd/f3c [1069132,62624] 0 2024-04-25T00:18:39.802 INFO:tasks.workunit.client.0.smithi107.stdout:6/825: dwrite d1/d14/de6/db/d32/d33/fc3 [0,4194304] 0 2024-04-25T00:18:39.804 INFO:tasks.workunit.client.0.smithi107.stdout:5/910: write d4/dc/d83/fcc [4009408,82056] 0 2024-04-25T00:18:39.804 INFO:tasks.workunit.client.0.smithi107.stdout:0/921: write dc/dd/fc9 [3616079,58758] 0 2024-04-25T00:18:39.805 INFO:tasks.workunit.client.0.smithi107.stdout:6/826: mknod d1/d14/de6/d2b/d3e/c111 0 2024-04-25T00:18:39.807 INFO:tasks.workunit.client.0.smithi107.stdout:6/827: rename d1/d14/de6/db/d32/d36/d3a/fa7 to d1/d14/de6/dee/f112 0 2024-04-25T00:18:39.809 INFO:tasks.workunit.client.0.smithi107.stdout:6/828: mknod d1/d14/de6/db/c113 0 2024-04-25T00:18:39.809 INFO:tasks.workunit.client.0.smithi107.stdout:0/922: dread dc/dd/d16/f1b [4194304,4194304] 0 2024-04-25T00:18:39.810 INFO:tasks.workunit.client.0.smithi107.stdout:0/923: mknod dc/dd/d5d/d3f/c128 0 2024-04-25T00:18:39.813 INFO:tasks.workunit.client.0.smithi107.stdout:0/924: mkdir dc/dd/d16/d48/ddb/d129 0 2024-04-25T00:18:39.813 INFO:tasks.workunit.client.0.smithi107.stdout:1/944: sync 2024-04-25T00:18:39.813 INFO:tasks.workunit.client.0.smithi107.stdout:1/945: readlink d2/dd/d82/lf3 0 2024-04-25T00:18:39.813 INFO:tasks.workunit.client.0.smithi107.stdout:1/946: readlink d2/d62/d66/d70/lb9 0 2024-04-25T00:18:39.815 INFO:tasks.workunit.client.0.smithi107.stdout:0/925: mkdir dc/d24/d25/ddd/d12a 0 2024-04-25T00:18:39.816 INFO:tasks.workunit.client.0.smithi107.stdout:1/947: read d2/dd/d25/d12a/fb5 [698412,66] 0 2024-04-25T00:18:39.816 INFO:tasks.workunit.client.0.smithi107.stdout:0/926: creat dc/dd/f12b x:0 0 0 2024-04-25T00:18:39.817 INFO:tasks.workunit.client.0.smithi107.stdout:1/948: symlink d2/d62/d66/d70/d95/l135 0 2024-04-25T00:18:39.819 INFO:tasks.workunit.client.0.smithi107.stdout:0/927: creat dc/dd/d5d/f12c x:0 0 0 2024-04-25T00:18:39.820 INFO:tasks.workunit.client.0.smithi107.stdout:0/928: mkdir dc/dd/d16/dac/d12d 0 2024-04-25T00:18:39.820 INFO:tasks.workunit.client.0.smithi107.stdout:0/929: fdatasync dc/dd/fd8 0 2024-04-25T00:18:39.822 INFO:tasks.workunit.client.0.smithi107.stdout:5/911: dwrite d4/d17/f79 [0,4194304] 0 2024-04-25T00:18:39.822 INFO:tasks.workunit.client.0.smithi107.stdout:5/912: stat d4/d17/db3/d84/d93/dc4/f120 0 2024-04-25T00:18:39.822 INFO:tasks.workunit.client.0.smithi107.stdout:5/913: fdatasync d4/d17/f2b 0 2024-04-25T00:18:39.822 INFO:tasks.workunit.client.0.smithi107.stdout:5/914: write d4/d17/db3/d116/d51/fe5 [1810002,54020] 0 2024-04-25T00:18:39.824 INFO:tasks.workunit.client.0.smithi107.stdout:5/915: creat d4/dc/d26/d62/dcf/f137 x:0 0 0 2024-04-25T00:18:39.824 INFO:tasks.workunit.client.0.smithi107.stdout:5/916: dread - d4/d7c/fe8 zero size 2024-04-25T00:18:39.830 INFO:tasks.workunit.client.0.smithi107.stdout:6/829: dwrite d1/d14/de6/db/d32/d36/fd8 [0,4194304] 0 2024-04-25T00:18:39.831 INFO:tasks.workunit.client.0.smithi107.stdout:1/949: dread d2/dd/d25/f5f [0,4194304] 0 2024-04-25T00:18:39.832 INFO:tasks.workunit.client.0.smithi107.stdout:1/950: getdents d2/d2a/d37/d88/d12b 0 2024-04-25T00:18:39.832 INFO:tasks.workunit.client.0.smithi107.stdout:1/951: write d2/dd/d82/dc1/f100 [350228,44262] 0 2024-04-25T00:18:39.833 INFO:tasks.workunit.client.0.smithi107.stdout:1/952: chown d2/d2a/d37/de8/d11a/lff 2565758 1 2024-04-25T00:18:39.840 INFO:tasks.workunit.client.0.smithi107.stdout:5/917: dread d4/d17/db3/d116/d51/fe5 [0,4194304] 0 2024-04-25T00:18:39.840 INFO:tasks.workunit.client.0.smithi107.stdout:0/930: dwrite dc/dd/d5d/d3f/f61 [0,4194304] 0 2024-04-25T00:18:39.841 INFO:tasks.workunit.client.0.smithi107.stdout:5/918: truncate d4/f71 2549004 0 2024-04-25T00:18:39.842 INFO:tasks.workunit.client.0.smithi107.stdout:5/919: write d4/d7c/d88/d9d/deb/f124 [103616,126468] 0 2024-04-25T00:18:39.844 INFO:tasks.workunit.client.0.smithi107.stdout:5/920: mknod d4/d7c/c138 0 2024-04-25T00:18:39.844 INFO:tasks.workunit.client.0.smithi107.stdout:5/921: write d4/d1c/f9e [794232,71209] 0 2024-04-25T00:18:39.844 INFO:tasks.workunit.client.0.smithi107.stdout:6/830: rename d1/ca5 to d1/d14/de6/db/d3c/d43/d96/dc9/c114 0 2024-04-25T00:18:39.844 INFO:tasks.workunit.client.0.smithi107.stdout:6/831: dread - d1/d14/de6/db/d32/d33/d68/d75/f78 zero size 2024-04-25T00:18:39.850 INFO:tasks.workunit.client.0.smithi107.stdout:1/953: write d2/d2a/d37/f56 [3290119,27336] 0 2024-04-25T00:18:39.855 INFO:tasks.workunit.client.0.smithi107.stdout:1/954: read d2/f30 [8536529,105928] 0 2024-04-25T00:18:39.862 INFO:tasks.workunit.client.0.smithi107.stdout:6/832: dwrite d1/d14/de6/db/d32/dca/fce [0,4194304] 0 2024-04-25T00:18:39.864 INFO:tasks.workunit.client.0.smithi107.stdout:1/955: write d2/da4/fda [1271430,74245] 0 2024-04-25T00:18:39.864 INFO:tasks.workunit.client.0.smithi107.stdout:1/956: chown d2/dd/d25/d5e/f58 0 1 2024-04-25T00:18:39.864 INFO:tasks.workunit.client.0.smithi107.stdout:0/931: dwrite dc/dd/d16/f2a [0,4194304] 0 2024-04-25T00:18:39.866 INFO:tasks.workunit.client.0.smithi107.stdout:6/833: rename d1/d14/de6/db/d3c/d43/d96/dc9/cd5 to d1/d14/de6/d3/d26/c115 0 2024-04-25T00:18:39.868 INFO:tasks.workunit.client.0.smithi107.stdout:5/922: dwrite d4/f16 [0,4194304] 0 2024-04-25T00:18:39.869 INFO:tasks.workunit.client.0.smithi107.stdout:1/957: creat d2/d2a/d37/f136 x:0 0 0 2024-04-25T00:18:39.872 INFO:tasks.workunit.client.0.smithi107.stdout:5/923: symlink d4/d1c/d5c/l139 0 2024-04-25T00:18:39.872 INFO:tasks.workunit.client.0.smithi107.stdout:5/924: stat d4/d17/db3/d116/d2e/d82/fa6 0 2024-04-25T00:18:39.872 INFO:tasks.workunit.client.0.smithi107.stdout:5/925: fdatasync d4/d17/db3/d84/d93/dc4/f120 0 2024-04-25T00:18:39.873 INFO:tasks.workunit.client.0.smithi107.stdout:1/958: symlink d2/dd/d25/d12a/l137 0 2024-04-25T00:18:39.881 INFO:tasks.workunit.client.0.smithi107.stdout:0/932: dwrite dc/d24/d25/ddd/ff8 [0,4194304] 0 2024-04-25T00:18:39.882 INFO:tasks.workunit.client.0.smithi107.stdout:0/933: write dc/dd/d5d/d3f/f113 [801957,124477] 0 2024-04-25T00:18:39.882 INFO:tasks.workunit.client.0.smithi107.stdout:6/834: write d1/d14/de6/db/d32/d36/fcb [653862,46995] 0 2024-04-25T00:18:39.896 INFO:tasks.workunit.client.0.smithi107.stdout:0/934: dread dc/dd/d5d/d3f/f113 [0,4194304] 0 2024-04-25T00:18:39.898 INFO:tasks.workunit.client.0.smithi107.stdout:0/935: creat dc/dd/f12e x:0 0 0 2024-04-25T00:18:39.898 INFO:tasks.workunit.client.0.smithi107.stdout:0/936: write dc/f2f [207906,34921] 0 2024-04-25T00:18:39.898 INFO:tasks.workunit.client.0.smithi107.stdout:5/926: dwrite d4/d17/db3/d116/d51/d78/fed [0,4194304] 0 2024-04-25T00:18:39.898 INFO:tasks.workunit.client.0.smithi107.stdout:1/959: dwrite d2/d2a/d37/de8/d11a/fce [0,4194304] 0 2024-04-25T00:18:39.901 INFO:tasks.workunit.client.0.smithi107.stdout:0/937: creat dc/dd/d5d/d3f/dc7/f12f x:0 0 0 2024-04-25T00:18:39.904 INFO:tasks.workunit.client.0.smithi107.stdout:1/960: stat d2/d2a/d37/de8/d11a/ce1 0 2024-04-25T00:18:39.904 INFO:tasks.workunit.client.0.smithi107.stdout:1/961: rename d2/d2a/d37/de8 to d2/d2a/d37/de8/d138 22 2024-04-25T00:18:39.905 INFO:tasks.workunit.client.0.smithi107.stdout:1/962: write d2/da4/f128 [87103,27664] 0 2024-04-25T00:18:39.905 INFO:tasks.workunit.client.0.smithi107.stdout:1/963: stat d2/d2a/d37/d88/ff5 0 2024-04-25T00:18:39.907 INFO:tasks.workunit.client.0.smithi107.stdout:5/927: dread d4/dc/fd [0,4194304] 0 2024-04-25T00:18:39.907 INFO:tasks.workunit.client.0.smithi107.stdout:5/928: chown d4/d17/db3/f105 0 1 2024-04-25T00:18:39.907 INFO:tasks.workunit.client.0.smithi107.stdout:1/964: mkdir d2/d2a/d37/de8/d11a/d139 0 2024-04-25T00:18:39.907 INFO:tasks.workunit.client.0.smithi107.stdout:1/965: dread - d2/da4/fb6 zero size 2024-04-25T00:18:39.914 INFO:tasks.workunit.client.0.smithi107.stdout:6/835: dwrite d1/d14/de6/dad/fe4 [0,4194304] 0 2024-04-25T00:18:39.915 INFO:tasks.workunit.client.0.smithi107.stdout:0/938: creat dc/d24/d38/d8b/dd7/f130 x:0 0 0 2024-04-25T00:18:39.916 INFO:tasks.workunit.client.0.smithi107.stdout:6/836: truncate d1/d14/f89 4142368 0 2024-04-25T00:18:39.917 INFO:tasks.workunit.client.0.smithi107.stdout:1/966: mknod d2/da/c13a 0 2024-04-25T00:18:39.923 INFO:tasks.workunit.client.0.smithi107.stdout:1/967: unlink d2/d62/cf7 0 2024-04-25T00:18:39.927 INFO:tasks.workunit.client.0.smithi107.stdout:6/837: mkdir d1/d4c/d116 0 2024-04-25T00:18:39.929 INFO:tasks.workunit.client.0.smithi107.stdout:1/968: getdents d2/da/d2f 0 2024-04-25T00:18:39.929 INFO:tasks.workunit.client.0.smithi107.stdout:6/838: unlink d1/d14/de6/db/d32/d36/d3d/fa1 0 2024-04-25T00:18:39.930 INFO:tasks.workunit.client.0.smithi107.stdout:6/839: getdents d1/d14/de6/d2b 0 2024-04-25T00:18:39.931 INFO:tasks.workunit.client.0.smithi107.stdout:0/939: dwrite dc/ff2 [0,4194304] 0 2024-04-25T00:18:39.931 INFO:tasks.workunit.client.0.smithi107.stdout:0/940: chown dc/d24/d25/ddd 2 1 2024-04-25T00:18:39.931 INFO:tasks.workunit.client.0.smithi107.stdout:0/941: stat dc/d24/d38 0 2024-04-25T00:18:39.931 INFO:tasks.workunit.client.0.smithi107.stdout:6/840: symlink d1/d14/de6/d2b/d3e/d5b/l117 0 2024-04-25T00:18:39.932 INFO:tasks.workunit.client.0.smithi107.stdout:6/841: chown d1/d14/de6/db/d32/d33 218289 1 2024-04-25T00:18:39.935 INFO:tasks.workunit.client.0.smithi107.stdout:6/842: mkdir d1/d118 0 2024-04-25T00:18:39.935 INFO:tasks.workunit.client.0.smithi107.stdout:6/843: write d1/d14/de6/d3/f13 [4144878,1249] 0 2024-04-25T00:18:39.935 INFO:tasks.workunit.client.0.smithi107.stdout:6/844: truncate d1/d14/de6/db/d32/ff0 648879 0 2024-04-25T00:18:39.941 INFO:tasks.workunit.client.0.smithi107.stdout:0/942: dread dc/dd/d16/d48/ddb/d11b/f89 [0,4194304] 0 2024-04-25T00:18:39.941 INFO:tasks.workunit.client.0.smithi107.stdout:0/943: fsync dc/dd/d16/d5e/fcc 0 2024-04-25T00:18:39.943 INFO:tasks.workunit.client.0.smithi107.stdout:0/944: truncate dc/dd/d16/f2a 3833955 0 2024-04-25T00:18:39.945 INFO:tasks.workunit.client.0.smithi107.stdout:1/969: dwrite d2/dd/f8b [0,4194304] 0 2024-04-25T00:18:39.946 INFO:tasks.workunit.client.0.smithi107.stdout:0/945: link dc/f2f dc/d24/d38/d8b/dd7/f131 0 2024-04-25T00:18:39.948 INFO:tasks.workunit.client.0.smithi107.stdout:1/970: creat d2/d2a/f13b x:0 0 0 2024-04-25T00:18:39.948 INFO:tasks.workunit.client.0.smithi107.stdout:6/845: dread d1/d14/de6/db/d32/d36/f99 [0,4194304] 0 2024-04-25T00:18:39.949 INFO:tasks.workunit.client.0.smithi107.stdout:6/846: creat d1/d14/d18/ddd/f119 x:0 0 0 2024-04-25T00:18:39.965 INFO:tasks.workunit.client.0.smithi107.stdout:1/971: dwrite d2/d2a/d5c/f71 [0,4194304] 0 2024-04-25T00:18:39.967 INFO:tasks.workunit.client.0.smithi107.stdout:0/946: dwrite dc/dd/d16/d48/ddb/d11b/d35/ffc [0,4194304] 0 2024-04-25T00:18:39.972 INFO:tasks.workunit.client.0.smithi107.stdout:6/847: dwrite d1/d14/de6/db/d3c/d43/d96/fb5 [4194304,4194304] 0 2024-04-25T00:18:39.972 INFO:tasks.workunit.client.0.smithi107.stdout:6/848: chown d1/d14/de6/d3/d28/d52/dd0 521662 1 2024-04-25T00:18:39.980 INFO:tasks.workunit.client.0.smithi107.stdout:1/972: write d2/dd/d25/d12a/ffb [630574,119544] 0 2024-04-25T00:18:39.981 INFO:tasks.workunit.client.0.smithi107.stdout:1/973: dread - d2/d108/f10c zero size 2024-04-25T00:18:39.981 INFO:tasks.workunit.client.0.smithi107.stdout:1/974: getdents d2/d2a/d5c/d89 0 2024-04-25T00:18:39.982 INFO:tasks.workunit.client.0.smithi107.stdout:1/975: mkdir d2/d2a/d5c/d13c 0 2024-04-25T00:18:39.982 INFO:tasks.workunit.client.0.smithi107.stdout:1/976: truncate d2/d108/f11c 524231 0 2024-04-25T00:18:39.983 INFO:tasks.workunit.client.0.smithi107.stdout:1/977: rmdir d2/da/d2f 39 2024-04-25T00:18:39.983 INFO:tasks.workunit.client.0.smithi107.stdout:1/978: chown d2/dd/d82/dc1/ccd 608500 1 2024-04-25T00:18:39.985 INFO:tasks.workunit.client.0.smithi107.stdout:0/947: dwrite dc/d24/d38/d64/d78/ddc/f11d [0,4194304] 0 2024-04-25T00:18:39.986 INFO:tasks.workunit.client.0.smithi107.stdout:0/948: getdents dc/dd/d16/d48/d83 0 2024-04-25T00:18:39.986 INFO:tasks.workunit.client.0.smithi107.stdout:0/949: fsync dc/dd/d5d/d3f/f7a 0 2024-04-25T00:18:39.988 INFO:tasks.workunit.client.0.smithi107.stdout:0/950: creat dc/f132 x:0 0 0 2024-04-25T00:18:39.988 INFO:tasks.workunit.client.0.smithi107.stdout:6/849: rename d1/d14/de6/db/d32/f10d to d1/d4c/f11a 0 2024-04-25T00:18:39.990 INFO:tasks.workunit.client.0.smithi107.stdout:0/951: truncate dc/dd/d5d/d5c/f71 1571849 0 2024-04-25T00:18:39.990 INFO:tasks.workunit.client.0.smithi107.stdout:6/850: creat d1/d14/de6/d3/d26/f11b x:0 0 0 2024-04-25T00:18:39.990 INFO:tasks.workunit.client.0.smithi107.stdout:6/851: truncate d1/d14/de6/dee/df5/ff6 6760 0 2024-04-25T00:18:39.992 INFO:tasks.workunit.client.0.smithi107.stdout:1/979: rename d2/d62/d66/d70/lb9 to d2/d2a/d5c/l13d 0 2024-04-25T00:18:39.993 INFO:tasks.workunit.client.0.smithi107.stdout:0/952: mknod dc/d24/d38/c133 0 2024-04-25T00:18:39.994 INFO:tasks.workunit.client.0.smithi107.stdout:0/953: chown dc/d24/d38/d64 239179691 1 2024-04-25T00:18:39.994 INFO:tasks.workunit.client.0.smithi107.stdout:0/954: read dc/d24/d38/d64/f68 [2089412,12524] 0 2024-04-25T00:18:39.994 INFO:tasks.workunit.client.0.smithi107.stdout:1/980: getdents d2/dd 0 2024-04-25T00:18:39.994 INFO:tasks.workunit.client.0.smithi107.stdout:1/981: readlink d2/d62/d66/de5/l104 0 2024-04-25T00:18:39.995 INFO:tasks.workunit.client.0.smithi107.stdout:0/955: creat dc/d24/d25/ddd/d12a/f134 x:0 0 0 2024-04-25T00:18:39.996 INFO:tasks.workunit.client.0.smithi107.stdout:1/982: creat d2/d120/f13e x:0 0 0 2024-04-25T00:18:40.001 INFO:tasks.workunit.client.0.smithi107.stdout:0/956: dread dc/d24/d25/f58 [0,4194304] 0 2024-04-25T00:18:40.001 INFO:tasks.workunit.client.0.smithi107.stdout:0/957: dread - dc/d24/d38/ff0 zero size 2024-04-25T00:18:40.001 INFO:tasks.workunit.client.0.smithi107.stdout:0/958: readlink dc/dd/d16/dac/l105 0 2024-04-25T00:18:40.017 INFO:tasks.workunit.client.0.smithi107.stdout:6/852: dwrite d1/d14/ffe [0,4194304] 0 2024-04-25T00:18:40.017 INFO:tasks.workunit.client.0.smithi107.stdout:1/983: dread d2/dd/d25/d5e/f94 [0,4194304] 0 2024-04-25T00:18:40.017 INFO:tasks.workunit.client.0.smithi107.stdout:0/959: rename dc/dd/f19 to dc/dd/d5d/d5c/d6e/d111/f135 0 2024-04-25T00:18:40.020 INFO:tasks.workunit.client.0.smithi107.stdout:1/984: mkdir d2/dd/d1b/d13f 0 2024-04-25T00:18:40.020 INFO:tasks.workunit.client.0.smithi107.stdout:0/960: mkdir dc/dd/d5d/d5c/d6e/d11a/d136 0 2024-04-25T00:18:40.020 INFO:tasks.workunit.client.0.smithi107.stdout:0/961: chown dc/dd/d16/l4a 1 1 2024-04-25T00:18:40.021 INFO:tasks.workunit.client.0.smithi107.stdout:0/962: creat dc/d24/d25/ddd/f137 x:0 0 0 2024-04-25T00:18:40.023 INFO:tasks.workunit.client.0.smithi107.stdout:5/929: sync 2024-04-25T00:18:40.025 INFO:tasks.workunit.client.0.smithi107.stdout:5/930: symlink d4/dc/l13a 0 2024-04-25T00:18:40.026 INFO:tasks.workunit.client.0.smithi107.stdout:5/931: creat d4/d7c/f13b x:0 0 0 2024-04-25T00:18:40.031 INFO:tasks.workunit.client.0.smithi107.stdout:1/985: write d2/d2a/d37/f56 [6434767,94839] 0 2024-04-25T00:18:40.032 INFO:tasks.workunit.client.0.smithi107.stdout:1/986: write d2/d62/d66/fc6 [968440,84889] 0 2024-04-25T00:18:40.033 INFO:tasks.workunit.client.0.smithi107.stdout:1/987: creat d2/dd/d1b/f140 x:0 0 0 2024-04-25T00:18:40.034 INFO:tasks.workunit.client.0.smithi107.stdout:1/988: mknod d2/c141 0 2024-04-25T00:18:40.034 INFO:tasks.workunit.client.0.smithi107.stdout:1/989: mkdir d2/d62/dfa/d126/d142 0 2024-04-25T00:18:40.035 INFO:tasks.workunit.client.0.smithi107.stdout:6/853: dwrite d1/d14/de6/db/d3c/d43/d96/dc9/fde [0,4194304] 0 2024-04-25T00:18:40.036 INFO:tasks.workunit.client.0.smithi107.stdout:1/990: mknod d2/d2a/d37/d88/d12b/c143 0 2024-04-25T00:18:40.036 INFO:tasks.workunit.client.0.smithi107.stdout:1/991: fdatasync d2/f93 0 2024-04-25T00:18:40.037 INFO:tasks.workunit.client.0.smithi107.stdout:1/992: creat d2/d2a/d37/d88/f144 x:0 0 0 2024-04-25T00:18:40.037 INFO:tasks.workunit.client.0.smithi107.stdout:0/963: link dc/d24/d38/d64/f88 dc/dd/d16/d48/d83/d9f/dc4/f138 0 2024-04-25T00:18:40.039 INFO:tasks.workunit.client.0.smithi107.stdout:6/854: mkdir d1/d14/de6/dee/df5/d11c 0 2024-04-25T00:18:40.039 INFO:tasks.workunit.client.0.smithi107.stdout:6/855: chown d1/d14/de6/d3/d28/lcd 285359 1 2024-04-25T00:18:40.039 INFO:tasks.workunit.client.0.smithi107.stdout:6/856: write d1/d14/de6/d3/d26/de2/fef [564832,100539] 0 2024-04-25T00:18:40.041 INFO:tasks.workunit.client.0.smithi107.stdout:6/857: unlink d1/d14/de6/d3/d28/d79/fb8 0 2024-04-25T00:18:40.049 INFO:tasks.workunit.client.0.smithi107.stdout:6/858: dread d1/d14/de6/d3/d71/d94/f9b [0,4194304] 0 2024-04-25T00:18:40.050 INFO:tasks.workunit.client.0.smithi107.stdout:5/932: rename d4/dc/d26/d62/c73 to d4/dc/d26/d42/c13c 0 2024-04-25T00:18:40.051 INFO:tasks.workunit.client.0.smithi107.stdout:6/859: symlink d1/d14/de6/d3/d71/d94/l11d 0 2024-04-25T00:18:40.052 INFO:tasks.workunit.client.0.smithi107.stdout:5/933: creat d4/d7c/f13d x:0 0 0 2024-04-25T00:18:40.053 INFO:tasks.workunit.client.0.smithi107.stdout:5/934: stat d4/d17/db3/d84/d93/lbc 0 2024-04-25T00:18:40.060 INFO:tasks.workunit.client.0.smithi107.stdout:1/993: dwrite d2/da/f10b [0,4194304] 0 2024-04-25T00:18:40.064 INFO:tasks.workunit.client.0.smithi107.stdout:1/994: link d2/d2a/d5c/d89/fdb d2/dd/d1b/dc2/f145 0 2024-04-25T00:18:40.064 INFO:tasks.workunit.client.0.smithi107.stdout:1/995: dread - d2/f69 zero size 2024-04-25T00:18:40.065 INFO:tasks.workunit.client.0.smithi107.stdout:1/996: write d2/dd/dec/ff8 [189014,101878] 0 2024-04-25T00:18:40.065 INFO:tasks.workunit.client.0.smithi107.stdout:1/997: write d2/dd/d1b/f140 [487191,76095] 0 2024-04-25T00:18:40.066 INFO:tasks.workunit.client.0.smithi107.stdout:1/998: creat d2/d62/d66/d70/d95/f146 x:0 0 0 2024-04-25T00:18:40.066 INFO:tasks.workunit.client.0.smithi107.stdout:1/999: getdents d2/d62/d66/d70 0 2024-04-25T00:18:40.069 INFO:tasks.workunit.client.0.smithi107.stdout:5/935: dwrite d4/dc/d83/f100 [0,4194304] 0 2024-04-25T00:18:40.077 INFO:tasks.workunit.client.0.smithi107.stdout:5/936: write d4/f1b [1695178,124229] 0 2024-04-25T00:18:40.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:39 smithi170 ceph-mon[32807]: pgmap v69: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 220 MiB/s rd, 265 MiB/s wr, 416 op/s 2024-04-25T00:18:40.202 INFO:tasks.workunit.client.0.smithi107.stdout:5/937: rename d4/dc/f14 to d4/d17/db3/d84/df0/f13e 0 2024-04-25T00:18:40.202 INFO:tasks.workunit.client.0.smithi107.stdout:5/938: fdatasync d4/d17/db3/d84/df0/fe9 0 2024-04-25T00:18:40.209 INFO:tasks.workunit.client.0.smithi107.stdout:0/964: sync 2024-04-25T00:18:40.209 INFO:tasks.workunit.client.0.smithi107.stdout:0/965: write dc/d24/d25/f67 [2339375,96015] 0 2024-04-25T00:18:40.215 INFO:tasks.workunit.client.0.smithi107.stdout:0/966: dread dc/dd/f3c [0,4194304] 0 2024-04-25T00:18:40.230 INFO:tasks.workunit.client.0.smithi107.stdout:0/967: dwrite dc/d24/d25/ddd/f137 [0,4194304] 0 2024-04-25T00:18:40.230 INFO:tasks.workunit.client.0.smithi107.stdout:0/968: truncate dc/d24/fbf 5075373 0 2024-04-25T00:18:40.233 INFO:tasks.workunit.client.0.smithi107.stdout:0/969: unlink dc/dd/l52 0 2024-04-25T00:18:40.233 INFO:tasks.workunit.client.0.smithi107.stdout:0/970: write dc/dd/d5d/d7e/f123 [520619,15312] 0 2024-04-25T00:18:40.233 INFO:tasks.workunit.client.0.smithi107.stdout:0/971: chown dc/dd/d5d/d3f/da5/fa9 7447813 1 2024-04-25T00:18:40.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:39 smithi107 ceph-mon[29153]: pgmap v69: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 220 MiB/s rd, 265 MiB/s wr, 416 op/s 2024-04-25T00:18:40.234 INFO:tasks.workunit.client.0.smithi107.stdout:0/972: read dc/f2f [159602,55417] 0 2024-04-25T00:18:40.235 INFO:tasks.workunit.client.0.smithi107.stdout:0/973: chown dc/dd/d16/d5e/fcc 193399646 1 2024-04-25T00:18:40.235 INFO:tasks.workunit.client.0.smithi107.stdout:0/974: getdents dc/d24/de7 0 2024-04-25T00:18:40.236 INFO:tasks.workunit.client.0.smithi107.stdout:0/975: rename dc/dd/d5d/d3f/da5/fcf to dc/d24/d38/d64/f139 0 2024-04-25T00:18:40.237 INFO:tasks.workunit.client.0.smithi107.stdout:0/976: readlink dc/dd/d16/d48/l9d 0 2024-04-25T00:18:40.237 INFO:tasks.workunit.client.0.smithi107.stdout:0/977: write dc/d24/d38/d64/d78/f115 [534910,11717] 0 2024-04-25T00:18:40.313 INFO:tasks.workunit.client.0.smithi107.stdout:6/860: sync 2024-04-25T00:18:40.313 INFO:tasks.workunit.client.0.smithi107.stdout:6/861: rmdir d1/d14/de6/dee/df5 39 2024-04-25T00:18:40.315 INFO:tasks.workunit.client.0.smithi107.stdout:6/862: unlink d1/d14/de6/db/d32/d36/d7d/c105 0 2024-04-25T00:18:40.316 INFO:tasks.workunit.client.0.smithi107.stdout:6/863: creat d1/d14/de6/db/d32/d36/d7d/f11e x:0 0 0 2024-04-25T00:18:40.317 INFO:tasks.workunit.client.0.smithi107.stdout:6/864: write d1/d14/de6/db/f19 [3249709,19767] 0 2024-04-25T00:18:40.318 INFO:tasks.workunit.client.0.smithi107.stdout:6/865: rmdir d1/d14 39 2024-04-25T00:18:40.318 INFO:tasks.workunit.client.0.smithi107.stdout:6/866: fsync d1/d14/de6/db/d32/d33/d68/d75/f78 0 2024-04-25T00:18:40.327 INFO:tasks.workunit.client.0.smithi107.stdout:6/867: write d1/d14/de6/db/fd [2153339,82730] 0 2024-04-25T00:18:40.328 INFO:tasks.workunit.client.0.smithi107.stdout:6/868: unlink d1/d14/de6/db/d3c/d43/cb9 0 2024-04-25T00:18:40.329 INFO:tasks.workunit.client.0.smithi107.stdout:6/869: mknod d1/d14/de6/d3/d28/d52/dd0/c11f 0 2024-04-25T00:18:40.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:39 smithi195 ceph-mon[32833]: pgmap v69: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 220 MiB/s rd, 265 MiB/s wr, 416 op/s 2024-04-25T00:18:40.339 INFO:tasks.workunit.client.0.smithi107.stdout:6/870: link d1/d14/d18/f2c d1/d14/de6/d3/d26/f120 0 2024-04-25T00:18:40.341 INFO:tasks.workunit.client.0.smithi107.stdout:5/939: sync 2024-04-25T00:18:40.341 INFO:tasks.workunit.client.0.smithi107.stdout:6/871: creat d1/d14/de6/db/d3c/f121 x:0 0 0 2024-04-25T00:18:40.341 INFO:tasks.workunit.client.0.smithi107.stdout:6/872: getdents d1/d14/de6/dad/dfb 0 2024-04-25T00:18:40.352 INFO:tasks.workunit.client.0.smithi107.stdout:5/940: rename d4/d17/db3/d84/c8d to d4/d17/d104/c13f 0 2024-04-25T00:18:40.354 INFO:tasks.workunit.client.0.smithi107.stdout:6/873: rename d1/d14/de6/db/d3c/d9c to d1/d14/de6/d3/d28/d79/de0/d122 0 2024-04-25T00:18:40.354 INFO:tasks.workunit.client.0.smithi107.stdout:6/874: fdatasync d1/d14/de6/db/d32/fc2 0 2024-04-25T00:18:40.355 INFO:tasks.workunit.client.0.smithi107.stdout:6/875: mknod d1/d14/de6/d3/d26/c123 0 2024-04-25T00:18:40.355 INFO:tasks.workunit.client.0.smithi107.stdout:6/876: write d1/d14/de6/d3/fd7 [13218,36032] 0 2024-04-25T00:18:40.355 INFO:tasks.workunit.client.0.smithi107.stdout:6/877: dread - d1/d14/de6/db/d32/d33/d68/fe3 zero size 2024-04-25T00:18:40.363 INFO:tasks.workunit.client.0.smithi107.stdout:5/941: dread d4/dc/f7f [0,4194304] 0 2024-04-25T00:18:40.363 INFO:tasks.workunit.client.0.smithi107.stdout:5/942: chown d4/dc/d26/f30 58860 1 2024-04-25T00:18:40.363 INFO:tasks.workunit.client.0.smithi107.stdout:5/943: write d4/d17/db3/d116/d51/fc5 [898141,92777] 0 2024-04-25T00:18:40.364 INFO:tasks.workunit.client.0.smithi107.stdout:6/878: write d1/d14/de6/db/d3c/d43/d96/dc9/fde [290261,75486] 0 2024-04-25T00:18:40.366 INFO:tasks.workunit.client.0.smithi107.stdout:6/879: mknod d1/d14/de6/d3/d28/c124 0 2024-04-25T00:18:40.374 INFO:tasks.workunit.client.0.smithi107.stdout:5/944: dread d4/d17/db3/f11e [0,4194304] 0 2024-04-25T00:18:40.376 INFO:tasks.workunit.client.0.smithi107.stdout:5/945: rename d4/dc/f7f to d4/dc/d83/f140 0 2024-04-25T00:18:40.379 INFO:tasks.workunit.client.0.smithi107.stdout:0/978: sync 2024-04-25T00:18:40.379 INFO:tasks.workunit.client.0.smithi107.stdout:0/979: stat dc/dd/d16/lb3 0 2024-04-25T00:18:40.379 INFO:tasks.workunit.client.0.smithi107.stdout:0/980: write dc/dd/d16/d48/ddb/d11b/d35/fa3 [1013199,30942] 0 2024-04-25T00:18:40.379 INFO:tasks.workunit.client.0.smithi107.stdout:0/981: write dc/d24/d25/f67 [4522603,32855] 0 2024-04-25T00:18:40.379 INFO:tasks.workunit.client.0.smithi107.stdout:0/982: readlink dc/d24/l42 0 2024-04-25T00:18:40.379 INFO:tasks.workunit.client.0.smithi107.stdout:0/983: write dc/f91 [903405,32149] 0 2024-04-25T00:18:40.393 INFO:tasks.workunit.client.0.smithi107.stdout:5/946: write d4/dc/f102 [57900,117813] 0 2024-04-25T00:18:40.393 INFO:tasks.workunit.client.0.smithi107.stdout:5/947: truncate d4/dc/fd 174883 0 2024-04-25T00:18:40.394 INFO:tasks.workunit.client.0.smithi107.stdout:5/948: getdents d4/d1c 0 2024-04-25T00:18:40.396 INFO:tasks.workunit.client.0.smithi107.stdout:5/949: creat d4/dc/d83/df1/f141 x:0 0 0 2024-04-25T00:18:40.396 INFO:tasks.workunit.client.0.smithi107.stdout:5/950: truncate d4/d17/db3/d116/d2e/d55/f9a 1777061 0 2024-04-25T00:18:40.399 INFO:tasks.workunit.client.0.smithi107.stdout:0/984: dwrite dc/dd/d5d/d5c/d6e/fcd [0,4194304] 0 2024-04-25T00:18:40.400 INFO:tasks.workunit.client.0.smithi107.stdout:0/985: chown dc/dd/d16/d48/f6a 2134502856 1 2024-04-25T00:18:40.400 INFO:tasks.workunit.client.0.smithi107.stdout:0/986: write dc/d24/d38/d64/f139 [4335717,98570] 0 2024-04-25T00:18:40.400 INFO:tasks.workunit.client.0.smithi107.stdout:0/987: write dc/dd/d5d/d3f/f7a [4585918,95516] 0 2024-04-25T00:18:40.404 INFO:tasks.workunit.client.0.smithi107.stdout:5/951: dread d4/d17/f21 [0,4194304] 0 2024-04-25T00:18:40.404 INFO:tasks.workunit.client.0.smithi107.stdout:5/952: chown d4/d1c/d5c/dd4/ff2 0 1 2024-04-25T00:18:40.415 INFO:tasks.workunit.client.0.smithi107.stdout:0/988: rename dc/dd/d16/d48/d83 to dc/dd/d5d/d103/d13a 0 2024-04-25T00:18:40.415 INFO:tasks.workunit.client.0.smithi107.stdout:5/953: dread d4/d17/db3/d84/df0/f37 [4194304,4194304] 0 2024-04-25T00:18:40.416 INFO:tasks.workunit.client.0.smithi107.stdout:0/989: mkdir dc/dd/d16/d48/ddb/d11b/d35/d13b 0 2024-04-25T00:18:40.417 INFO:tasks.workunit.client.0.smithi107.stdout:0/990: symlink dc/dd/d5d/d103/d13a/d9f/dba/l13c 0 2024-04-25T00:18:40.417 INFO:tasks.workunit.client.0.smithi107.stdout:0/991: chown dc/l3e 31371 1 2024-04-25T00:18:40.418 INFO:tasks.workunit.client.0.smithi107.stdout:0/992: chown dc/d24/d25/ddd/cd3 0 1 2024-04-25T00:18:40.418 INFO:tasks.workunit.client.0.smithi107.stdout:0/993: mkdir dc/d24/d38/d64/d13d 0 2024-04-25T00:18:40.420 INFO:tasks.workunit.client.0.smithi107.stdout:0/994: link dc/dd/d16/d48/ddb/d11b/f37 dc/dd/d5d/d5c/d6e/d11a/d136/f13e 0 2024-04-25T00:18:40.421 INFO:tasks.workunit.client.0.smithi107.stdout:0/995: creat dc/dd/d16/dac/d12d/f13f x:0 0 0 2024-04-25T00:18:40.422 INFO:tasks.workunit.client.0.smithi107.stdout:5/954: write d4/fa [1744432,43595] 0 2024-04-25T00:18:40.422 INFO:tasks.workunit.client.0.smithi107.stdout:0/996: mkdir dc/d24/d38/d64/d78/de6/d140 0 2024-04-25T00:18:40.422 INFO:tasks.workunit.client.0.smithi107.stdout:0/997: stat dc/dd/d16/f1b 0 2024-04-25T00:18:40.422 INFO:tasks.workunit.client.0.smithi107.stdout:0/998: truncate dc/dd/d5d/d5c/d6e/fc1 872564 0 2024-04-25T00:18:40.422 INFO:tasks.workunit.client.0.smithi107.stdout:0/999: truncate dc/dd/d5d/d103/d13a/fbd 799996 0 2024-04-25T00:18:40.430 INFO:tasks.workunit.client.0.smithi107.stdout:5/955: write d4/dc/d26/d42/f5e [2080545,93325] 0 2024-04-25T00:18:40.440 INFO:tasks.workunit.client.0.smithi107.stdout:5/956: dread d4/d17/db3/fbb [0,4194304] 0 2024-04-25T00:18:40.441 INFO:tasks.workunit.client.0.smithi107.stdout:5/957: mkdir d4/d17/db3/d84/df0/d106/d142 0 2024-04-25T00:18:40.447 INFO:tasks.workunit.client.0.smithi107.stdout:6/880: sync 2024-04-25T00:18:40.461 INFO:tasks.workunit.client.0.smithi107.stdout:5/958: dwrite d4/d17/db3/d84/df0/fce [0,4194304] 0 2024-04-25T00:18:40.461 INFO:tasks.workunit.client.0.smithi107.stdout:5/959: chown d4/d17/d104/c13f 44023 1 2024-04-25T00:18:40.463 INFO:tasks.workunit.client.0.smithi107.stdout:5/960: symlink d4/d1c/d5c/dd4/d107/d10d/l143 0 2024-04-25T00:18:40.463 INFO:tasks.workunit.client.0.smithi107.stdout:5/961: chown d4/d17/d104/l110 10539555 1 2024-04-25T00:18:40.464 INFO:tasks.workunit.client.0.smithi107.stdout:5/962: truncate d4/d7c/f13b 90767 0 2024-04-25T00:18:40.464 INFO:tasks.workunit.client.0.smithi107.stdout:6/881: dwrite d1/d14/de6/d2b/d3e/d5b/ff1 [0,4194304] 0 2024-04-25T00:18:40.464 INFO:tasks.workunit.client.0.smithi107.stdout:6/882: fsync d1/d14/de6/dad/dfb/f93 0 2024-04-25T00:18:40.464 INFO:tasks.workunit.client.0.smithi107.stdout:5/963: creat d4/d17/db3/d116/d2e/d82/dfe/f144 x:0 0 0 2024-04-25T00:18:40.467 INFO:tasks.workunit.client.0.smithi107.stdout:6/883: mkdir d1/d14/de6/db/d32/d36/d7d/d125 0 2024-04-25T00:18:40.468 INFO:tasks.workunit.client.0.smithi107.stdout:6/884: unlink d1/d14/de6/d3/d71/cab 0 2024-04-25T00:18:40.468 INFO:tasks.workunit.client.0.smithi107.stdout:5/964: dread d4/d17/db3/d116/d2e/d55/f9a [0,4194304] 0 2024-04-25T00:18:40.469 INFO:tasks.workunit.client.0.smithi107.stdout:5/965: mknod d4/c145 0 2024-04-25T00:18:40.469 INFO:tasks.workunit.client.0.smithi107.stdout:6/885: link d1/d14/de6/d2b/d3e/d5b/l117 d1/d14/de6/db/d32/d33/d68/l126 0 2024-04-25T00:18:40.470 INFO:tasks.workunit.client.0.smithi107.stdout:5/966: symlink d4/d1c/l146 0 2024-04-25T00:18:40.470 INFO:tasks.workunit.client.0.smithi107.stdout:5/967: truncate d4/d17/db3/d84/df0/fe9 419042 0 2024-04-25T00:18:40.470 INFO:tasks.workunit.client.0.smithi107.stdout:6/886: mkdir d1/d14/de6/db/d3c/d43/d96/dc9/d127 0 2024-04-25T00:18:40.471 INFO:tasks.workunit.client.0.smithi107.stdout:5/968: creat d4/d7c/d88/f147 x:0 0 0 2024-04-25T00:18:40.471 INFO:tasks.workunit.client.0.smithi107.stdout:5/969: chown d4/dc/d83/l87 14271 1 2024-04-25T00:18:40.471 INFO:tasks.workunit.client.0.smithi107.stdout:6/887: creat d1/d4c/d116/f128 x:0 0 0 2024-04-25T00:18:40.474 INFO:tasks.workunit.client.0.smithi107.stdout:5/970: write d4/d17/db3/d116/d51/fc5 [79848,15386] 0 2024-04-25T00:18:40.478 INFO:tasks.workunit.client.0.smithi107.stdout:5/971: dread d4/d17/db3/d116/d2e/d55/fa0 [0,4194304] 0 2024-04-25T00:18:40.479 INFO:tasks.workunit.client.0.smithi107.stdout:5/972: mknod d4/d1c/d5c/dd4/c148 0 2024-04-25T00:18:40.489 INFO:tasks.workunit.client.0.smithi107.stdout:6/888: dwrite d1/d14/de6/db/d32/f73 [0,4194304] 0 2024-04-25T00:18:40.491 INFO:tasks.workunit.client.0.smithi107.stdout:6/889: mkdir d1/d14/de6/d3/d71/d129 0 2024-04-25T00:18:40.493 INFO:tasks.workunit.client.0.smithi107.stdout:6/890: truncate d1/d14/de6/d2b/d3e/d5b/ff1 3497799 0 2024-04-25T00:18:40.502 INFO:tasks.workunit.client.0.smithi107.stdout:5/973: dwrite d4/f71 [0,4194304] 0 2024-04-25T00:18:40.503 INFO:tasks.workunit.client.0.smithi107.stdout:5/974: creat d4/d7c/d88/d9d/de1/d11a/f149 x:0 0 0 2024-04-25T00:18:40.505 INFO:tasks.workunit.client.0.smithi107.stdout:6/891: read d1/d14/de6/db/d32/d36/f99 [566116,65958] 0 2024-04-25T00:18:40.505 INFO:tasks.workunit.client.0.smithi107.stdout:6/892: chown d1/d14/de6/db/f19 316058885 1 2024-04-25T00:18:40.505 INFO:tasks.workunit.client.0.smithi107.stdout:6/893: write d1/d14/de6/d2b/f86 [849305,49869] 0 2024-04-25T00:18:40.505 INFO:tasks.workunit.client.0.smithi107.stdout:5/975: creat d4/d1c/d5c/dd4/d107/d10d/f14a x:0 0 0 2024-04-25T00:18:40.505 INFO:tasks.workunit.client.0.smithi107.stdout:6/894: truncate d1/d14/de6/d2b/d3e/d5b/d7a/faf 407479 0 2024-04-25T00:18:40.507 INFO:tasks.workunit.client.0.smithi107.stdout:5/976: unlink d4/dc/d26/d42/d8b/fa9 0 2024-04-25T00:18:40.507 INFO:tasks.workunit.client.0.smithi107.stdout:6/895: symlink d1/d14/de6/dad/dfb/l12a 0 2024-04-25T00:18:40.507 INFO:tasks.workunit.client.0.smithi107.stdout:6/896: fdatasync d1/d14/de6/db/f10c 0 2024-04-25T00:18:40.507 INFO:tasks.workunit.client.0.smithi107.stdout:6/897: chown d1/d14/de6/d3/d28/c124 1 1 2024-04-25T00:18:40.508 INFO:tasks.workunit.client.0.smithi107.stdout:6/898: link d1/d14/de6/db/d32/d36/f46 d1/d14/de6/db/d32/d10e/f12b 0 2024-04-25T00:18:40.509 INFO:tasks.workunit.client.0.smithi107.stdout:6/899: getdents d1/d14/de6/db/d32/d36 0 2024-04-25T00:18:40.509 INFO:tasks.workunit.client.0.smithi107.stdout:6/900: dread - d1/d14/de6/dad/fec zero size 2024-04-25T00:18:40.510 INFO:tasks.workunit.client.0.smithi107.stdout:6/901: creat d1/d14/de6/d2b/d3e/d5b/d7a/f12c x:0 0 0 2024-04-25T00:18:40.517 INFO:tasks.workunit.client.0.smithi107.stdout:5/977: dread d4/dc/d26/d42/ff7 [0,4194304] 0 2024-04-25T00:18:40.517 INFO:tasks.workunit.client.0.smithi107.stdout:5/978: write d4/d17/f2b [4567039,82066] 0 2024-04-25T00:18:40.519 INFO:tasks.workunit.client.0.smithi107.stdout:5/979: rmdir d4/dc/d83 39 2024-04-25T00:18:40.519 INFO:tasks.workunit.client.0.smithi107.stdout:5/980: chown d4/d17/db3/d84/d93/da4 20 1 2024-04-25T00:18:40.521 INFO:tasks.workunit.client.0.smithi107.stdout:5/981: link d4/d7c/d88/d9d/deb/c113 d4/dc/d26/c14b 0 2024-04-25T00:18:40.527 INFO:tasks.workunit.client.0.smithi107.stdout:6/902: link d1/d4c/f70 d1/d14/de6/f12d 0 2024-04-25T00:18:40.528 INFO:tasks.workunit.client.0.smithi107.stdout:6/903: chown d1/d14/de6/f103 36712 1 2024-04-25T00:18:40.528 INFO:tasks.workunit.client.0.smithi107.stdout:6/904: getdents d1/d14/de6/d3/d28/d79/d6b 0 2024-04-25T00:18:40.528 INFO:tasks.workunit.client.0.smithi107.stdout:6/905: chown d1/d14/de6/db/d32/d36/f99 218518 1 2024-04-25T00:18:40.534 INFO:tasks.workunit.client.0.smithi107.stdout:5/982: dread d4/d17/db3/d116/d2e/d55/fcb [0,4194304] 0 2024-04-25T00:18:40.534 INFO:tasks.workunit.client.0.smithi107.stdout:5/983: read d4/d17/db3/d84/df0/f3c [332745,47] 0 2024-04-25T00:18:40.534 INFO:tasks.workunit.client.0.smithi107.stdout:5/984: write d4/fd7 [4694328,119602] 0 2024-04-25T00:18:40.534 INFO:tasks.workunit.client.0.smithi107.stdout:5/985: truncate d4/dc/d26/d62/fcd 1785926 0 2024-04-25T00:18:40.540 INFO:tasks.workunit.client.0.smithi107.stdout:5/986: write d4/dc/fe [2465390,111956] 0 2024-04-25T00:18:40.540 INFO:tasks.workunit.client.0.smithi107.stdout:5/987: fsync d4/d17/f32 0 2024-04-25T00:18:40.544 INFO:tasks.workunit.client.0.smithi107.stdout:5/988: mknod d4/d17/db3/d116/d51/d117/d11d/c14c 0 2024-04-25T00:18:40.548 INFO:tasks.workunit.client.0.smithi107.stdout:5/989: link d4/d17/d104/c13f d4/d7c/c14d 0 2024-04-25T00:18:40.562 INFO:tasks.workunit.client.0.smithi107.stdout:5/990: rename d4/d7c/f13b to d4/d17/db3/d84/df0/d106/d142/f14e 0 2024-04-25T00:18:40.563 INFO:tasks.workunit.client.0.smithi107.stdout:5/991: creat d4/d7c/d88/d8e/f14f x:0 0 0 2024-04-25T00:18:40.577 INFO:tasks.workunit.client.0.smithi107.stdout:5/992: dwrite d4/d1c/d5c/dd4/d107/d10d/f14a [0,4194304] 0 2024-04-25T00:18:40.594 INFO:tasks.workunit.client.0.smithi107.stdout:5/993: dwrite d4/fbf [4194304,4194304] 0 2024-04-25T00:18:40.594 INFO:tasks.workunit.client.0.smithi107.stdout:5/994: fsync d4/d17/db3/f105 0 2024-04-25T00:18:40.608 INFO:tasks.workunit.client.0.smithi107.stdout:5/995: dwrite d4/d17/db3/d116/f12b [0,4194304] 0 2024-04-25T00:18:40.611 INFO:tasks.workunit.client.0.smithi107.stdout:5/996: creat d4/d7c/d88/f150 x:0 0 0 2024-04-25T00:18:40.624 INFO:tasks.workunit.client.0.smithi107.stdout:5/997: rename d4/dc/d26/d42/d8b to d4/d17/db3/d116/d2e/d55/d151 0 2024-04-25T00:18:40.625 INFO:tasks.workunit.client.0.smithi107.stdout:6/906: sync 2024-04-25T00:18:40.725 INFO:tasks.workunit.client.0.smithi107.stdout:5/998: link d4/c145 d4/d7c/d88/d9d/de1/d11a/c152 0 2024-04-25T00:18:40.726 INFO:tasks.workunit.client.0.smithi107.stdout:5/999: truncate d4/d17/db3/d116/d2e/d55/fcb 3483842 0 2024-04-25T00:18:40.733 INFO:tasks.workunit.client.0.smithi107.stdout:6/907: sync 2024-04-25T00:18:40.733 INFO:tasks.workunit.client.0.smithi107.stdout:6/908: chown d1/d14/d18/l80 2018 1 2024-04-25T00:18:40.751 INFO:tasks.workunit.client.0.smithi107.stdout:6/909: dwrite d1/d14/de6/f5f [0,4194304] 0 2024-04-25T00:18:40.753 INFO:tasks.workunit.client.0.smithi107.stdout:6/910: mknod d1/d14/de6/db/d32/d36/d3d/c12e 0 2024-04-25T00:18:40.753 INFO:tasks.workunit.client.0.smithi107.stdout:6/911: readlink d1/d14/l61 0 2024-04-25T00:18:40.753 INFO:tasks.workunit.client.0.smithi107.stdout:6/912: readlink d1/d14/de6/db/d32/d33/d68/l126 0 2024-04-25T00:18:40.754 INFO:tasks.workunit.client.0.smithi107.stdout:6/913: creat d1/d14/de6/dad/f12f x:0 0 0 2024-04-25T00:18:40.755 INFO:tasks.workunit.client.0.smithi107.stdout:6/914: mknod d1/d14/de6/db/d32/c130 0 2024-04-25T00:18:40.770 INFO:tasks.workunit.client.0.smithi107.stdout:6/915: dwrite d1/d14/de6/d3/d26/f11b [0,4194304] 0 2024-04-25T00:18:40.779 INFO:tasks.workunit.client.0.smithi107.stdout:6/916: read d1/fe9 [2261885,32205] 0 2024-04-25T00:18:40.797 INFO:tasks.workunit.client.0.smithi107.stdout:6/917: dwrite d1/d14/de6/db/d48/f10a [0,4194304] 0 2024-04-25T00:18:40.817 INFO:tasks.workunit.client.0.smithi107.stdout:6/918: dwrite d1/d14/de6/d3/f41 [4194304,4194304] 0 2024-04-25T00:18:40.826 INFO:tasks.workunit.client.0.smithi107.stdout:6/919: dread d1/d14/de6/dad/dfb/f58 [0,4194304] 0 2024-04-25T00:18:40.843 INFO:tasks.workunit.client.0.smithi107.stdout:6/920: dwrite d1/d14/de6/db/d32/fc2 [0,4194304] 0 2024-04-25T00:18:40.843 INFO:tasks.workunit.client.0.smithi107.stdout:6/921: write d1/d14/de6/db/d32/ff0 [283627,103348] 0 2024-04-25T00:18:40.843 INFO:tasks.workunit.client.0.smithi107.stdout:6/922: write d1/d14/de6/db/d32/d33/d68/d75/f78 [702319,67519] 0 2024-04-25T00:18:40.845 INFO:tasks.workunit.client.0.smithi107.stdout:6/923: dread d1/d14/de6/d3/d26/de2/fef [0,4194304] 0 2024-04-25T00:18:40.846 INFO:tasks.workunit.client.0.smithi107.stdout:6/924: truncate d1/d14/de6/db/d32/fc6 2056509 0 2024-04-25T00:18:40.846 INFO:tasks.workunit.client.0.smithi107.stdout:6/925: write d1/d14/de6/db/d32/dca/fce [4362384,28266] 0 2024-04-25T00:18:40.848 INFO:tasks.workunit.client.0.smithi107.stdout:6/926: rename d1/d14/de6/db/d32/d33/d68 to d1/d14/de6/d3/d28/d79/de0/d122/d131 0 2024-04-25T00:18:40.869 INFO:tasks.workunit.client.0.smithi107.stdout:6/927: dwrite d1/d14/de6/db/d32/d33/f6d [0,4194304] 0 2024-04-25T00:18:40.888 INFO:tasks.workunit.client.0.smithi107.stdout:6/928: dwrite d1/d14/de6/d3/fd7 [0,4194304] 0 2024-04-25T00:18:40.900 INFO:tasks.workunit.client.0.smithi107.stdout:6/929: rename d1/d14/de6/db/d32/l62 to d1/d14/l132 0 2024-04-25T00:18:40.900 INFO:tasks.workunit.client.0.smithi107.stdout:6/930: readlink d1/d14/de6/d3/d28/d79/de0/d122/d131/l126 0 2024-04-25T00:18:40.900 INFO:tasks.workunit.client.0.smithi107.stdout:6/931: chown d1/d14/de6/d2b/d3e/c111 199582338 1 2024-04-25T00:18:40.907 INFO:tasks.workunit.client.0.smithi107.stdout:6/932: dread d1/d14/de6/d3/f41 [4194304,4194304] 0 2024-04-25T00:18:40.920 INFO:tasks.workunit.client.0.smithi107.stdout:6/933: rename d1/d4c/fac to d1/d14/de6/d3/d28/d79/de0/f133 0 2024-04-25T00:18:40.920 INFO:tasks.workunit.client.0.smithi107.stdout:6/934: fdatasync d1/d4c/f11a 0 2024-04-25T00:18:40.943 INFO:tasks.workunit.client.0.smithi107.stdout:6/935: dwrite d1/d14/de6/f5f [4194304,4194304] 0 2024-04-25T00:18:40.944 INFO:tasks.workunit.client.0.smithi107.stdout:6/936: symlink d1/d14/de6/db/d3c/d43/l134 0 2024-04-25T00:18:40.945 INFO:tasks.workunit.client.0.smithi107.stdout:6/937: creat d1/d14/de6/dee/f135 x:0 0 0 2024-04-25T00:18:40.945 INFO:tasks.workunit.client.0.smithi107.stdout:6/938: dread - d1/d14/de6/db/d32/d36/d7d/f11e zero size 2024-04-25T00:18:40.950 INFO:tasks.workunit.client.0.smithi107.stdout:6/939: truncate d1/d14/de6/db/d32/dca/fce 3691317 0 2024-04-25T00:18:40.950 INFO:tasks.workunit.client.0.smithi107.stdout:6/940: chown d1/d14/de6/f5f 1967631 1 2024-04-25T00:18:40.951 INFO:tasks.workunit.client.0.smithi107.stdout:6/941: rename d1/d14/de6/d3/f13 to d1/d14/de6/d3/d28/d52/f136 0 2024-04-25T00:18:40.969 INFO:tasks.workunit.client.0.smithi107.stdout:6/942: dwrite d1/d14/de6/db/f19 [4194304,4194304] 0 2024-04-25T00:18:40.970 INFO:tasks.workunit.client.0.smithi107.stdout:6/943: creat d1/d14/de6/db/d32/d36/d3d/f137 x:0 0 0 2024-04-25T00:18:40.972 INFO:tasks.workunit.client.0.smithi107.stdout:6/944: link d1/d14/de6/d3/d28/d79/de0/d122/d131/d75/f8b d1/d14/de6/d3/d71/f138 0 2024-04-25T00:18:40.972 INFO:tasks.workunit.client.0.smithi107.stdout:6/945: chown d1/d14/de6/dad/f12f 0 1 2024-04-25T00:18:40.979 INFO:tasks.workunit.client.0.smithi107.stdout:6/946: rmdir d1/d4c 39 2024-04-25T00:18:40.988 INFO:tasks.workunit.client.0.smithi107.stdout:6/947: rename d1/d14/de6/d3/d71/d94/fb0 to d1/d14/de6/db/d3c/d43/d96/dc9/f139 0 2024-04-25T00:18:41.109 INFO:tasks.workunit.client.0.smithi107.stdout:6/948: sync 2024-04-25T00:18:41.110 INFO:tasks.workunit.client.0.smithi107.stdout:6/949: creat d1/d14/de6/d3/d28/d79/f13a x:0 0 0 2024-04-25T00:18:41.111 INFO:tasks.workunit.client.0.smithi107.stdout:6/950: truncate d1/d14/de6/db/d32/d36/f3f 315629 0 2024-04-25T00:18:41.112 INFO:tasks.workunit.client.0.smithi107.stdout:6/951: symlink d1/d14/de6/d3/d26/l13b 0 2024-04-25T00:18:41.133 INFO:tasks.workunit.client.0.smithi107.stdout:6/952: dwrite d1/d14/de6/d3/d28/d79/de0/d122/d131/fa0 [0,4194304] 0 2024-04-25T00:18:41.148 INFO:tasks.workunit.client.0.smithi107.stdout:6/953: dwrite d1/d14/de6/db/d32/ff0 [0,4194304] 0 2024-04-25T00:18:41.165 INFO:tasks.workunit.client.0.smithi107.stdout:6/954: dwrite d1/d14/de6/d3/d71/f138 [0,4194304] 0 2024-04-25T00:18:41.167 INFO:tasks.workunit.client.0.smithi107.stdout:6/955: rmdir d1/d14/de6/d2b/d3e 39 2024-04-25T00:18:41.170 INFO:tasks.workunit.client.0.smithi107.stdout:6/956: rmdir d1/d4c 39 2024-04-25T00:18:41.178 INFO:tasks.workunit.client.0.smithi107.stdout:6/957: dread d1/d14/de6/db/d48/fd2 [0,4194304] 0 2024-04-25T00:18:41.179 INFO:tasks.workunit.client.0.smithi107.stdout:6/958: symlink d1/d14/de6/d3/d28/l13c 0 2024-04-25T00:18:41.179 INFO:tasks.workunit.client.0.smithi107.stdout:6/959: chown d1/d14/de6/d3 5567 1 2024-04-25T00:18:41.179 INFO:tasks.workunit.client.0.smithi107.stdout:6/960: stat d1/d14/de6/d3/d26/l13b 0 2024-04-25T00:18:41.179 INFO:tasks.workunit.client.0.smithi107.stdout:6/961: fdatasync d1/d14/de6/d3/d28/d79/f13a 0 2024-04-25T00:18:41.179 INFO:tasks.workunit.client.0.smithi107.stdout:6/962: write d1/d14/de6/d3/d28/d79/d6b/fe1 [695988,81200] 0 2024-04-25T00:18:41.179 INFO:tasks.workunit.client.0.smithi107.stdout:6/963: truncate d1/d14/de6/d2b/d3e/d5b/d7a/f104 984525 0 2024-04-25T00:18:41.179 INFO:tasks.workunit.client.0.smithi107.stdout:6/964: rename d1/d14 to d1/d14/de6/d3/d28/d79/de0/d13d 22 2024-04-25T00:18:41.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:40 smithi170 ceph-mon[32807]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:18:41.190 INFO:tasks.workunit.client.0.smithi107.stdout:6/965: write d1/d14/de6/d3/d71/d94/fc4 [1250089,92437] 0 2024-04-25T00:18:41.190 INFO:tasks.workunit.client.0.smithi107.stdout:6/966: chown d1/d14/de6/dad/dfb/fea 26686 1 2024-04-25T00:18:41.190 INFO:tasks.workunit.client.0.smithi107.stdout:6/967: dread - d1/d14/de6/db/f10c zero size 2024-04-25T00:18:41.191 INFO:tasks.workunit.client.0.smithi107.stdout:6/968: mkdir d1/d14/de6/d3/d71/d94/d13e 0 2024-04-25T00:18:41.212 INFO:tasks.workunit.client.0.smithi107.stdout:6/969: dwrite d1/d14/de6/f5f [0,4194304] 0 2024-04-25T00:18:41.230 INFO:tasks.workunit.client.0.smithi107.stdout:6/970: dwrite d1/d14/de6/dad/dfb/fea [0,4194304] 0 2024-04-25T00:18:41.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:40 smithi107 ceph-mon[29153]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:18:41.274 INFO:tasks.workunit.client.0.smithi107.stdout:6/971: dwrite d1/f1f [0,4194304] 0 2024-04-25T00:18:41.275 INFO:tasks.workunit.client.0.smithi107.stdout:6/972: write d1/d14/d18/ddd/f119 [527649,53458] 0 2024-04-25T00:18:41.276 INFO:tasks.workunit.client.0.smithi107.stdout:6/973: creat d1/d14/de6/d3/f13f x:0 0 0 2024-04-25T00:18:41.277 INFO:tasks.workunit.client.0.smithi107.stdout:6/974: creat d1/d14/de6/d3/d71/d129/f140 x:0 0 0 2024-04-25T00:18:41.283 INFO:tasks.workunit.client.0.smithi107.stdout:6/975: write d1/d14/de6/db/d3c/d43/d96/dc9/f139 [2373976,122741] 0 2024-04-25T00:18:41.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:40 smithi195 ceph-mon[32833]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:18:41.401 INFO:tasks.workunit.client.0.smithi107.stdout:6/976: sync 2024-04-25T00:18:41.402 INFO:tasks.workunit.client.0.smithi107.stdout:6/977: getdents d1/d4c/d116 0 2024-04-25T00:18:41.402 INFO:tasks.workunit.client.0.smithi107.stdout:6/978: chown d1/d14/d18/lb4 1012128835 1 2024-04-25T00:18:41.410 INFO:tasks.workunit.client.0.smithi107.stdout:6/979: link d1/d14/de6/db/d32/d36/d7d/l109 d1/l141 0 2024-04-25T00:18:41.410 INFO:tasks.workunit.client.0.smithi107.stdout:6/980: chown d1/d14/de6/d3/d28/c124 167165173 1 2024-04-25T00:18:41.410 INFO:tasks.workunit.client.0.smithi107.stdout:6/981: chown d1/d14/de6/d3/d28/d79/de0/d122/ce8 3508 1 2024-04-25T00:18:41.411 INFO:tasks.workunit.client.0.smithi107.stdout:6/982: rmdir d1/d14/de6/dee 39 2024-04-25T00:18:41.412 INFO:tasks.workunit.client.0.smithi107.stdout:6/983: symlink d1/d14/de6/db/d32/d36/d7d/l142 0 2024-04-25T00:18:41.412 INFO:tasks.workunit.client.0.smithi107.stdout:6/984: write d1/d14/de6/db/d32/d36/fcb [1183142,108476] 0 2024-04-25T00:18:41.413 INFO:tasks.workunit.client.0.smithi107.stdout:6/985: creat d1/d14/de6/db/d32/d36/f143 x:0 0 0 2024-04-25T00:18:41.413 INFO:tasks.workunit.client.0.smithi107.stdout:6/986: fsync d1/d14/de6/d3/d28/d79/d6b/fe1 0 2024-04-25T00:18:41.414 INFO:tasks.workunit.client.0.smithi107.stdout:6/987: creat d1/d14/de6/db/d3c/d43/d96/dc9/d127/f144 x:0 0 0 2024-04-25T00:18:41.416 INFO:tasks.workunit.client.0.smithi107.stdout:6/988: link d1/d14/de6/d2b/d3e/d5b/ff1 d1/d14/de6/d2b/f145 0 2024-04-25T00:18:41.426 INFO:tasks.workunit.client.0.smithi107.stdout:6/989: dread d1/d14/de6/db/d32/d33/f6d [0,4194304] 0 2024-04-25T00:18:41.428 INFO:tasks.workunit.client.0.smithi107.stdout:6/990: link d1/d14/de6/db/d32/dca/l108 d1/d14/de6/db/d32/l146 0 2024-04-25T00:18:41.430 INFO:tasks.workunit.client.0.smithi107.stdout:6/991: dread d1/d14/de6/db/d32/d33/f63 [0,4194304] 0 2024-04-25T00:18:41.431 INFO:tasks.workunit.client.0.smithi107.stdout:6/992: mkdir d1/d14/de6/db/d147 0 2024-04-25T00:18:41.432 INFO:tasks.workunit.client.0.smithi107.stdout:6/993: creat d1/d14/de6/d3/d71/f148 x:0 0 0 2024-04-25T00:18:41.449 INFO:tasks.workunit.client.0.smithi107.stdout:6/994: dwrite d1/d14/de6/db/f19 [0,4194304] 0 2024-04-25T00:18:41.458 INFO:tasks.workunit.client.0.smithi107.stdout:6/995: dread d1/d14/de6/db/d32/d36/fcb [0,4194304] 0 2024-04-25T00:18:41.458 INFO:tasks.workunit.client.0.smithi107.stdout:6/996: chown d1/d14/de6/d3/d28/d52/dd0/c11f 30 1 2024-04-25T00:18:41.467 INFO:tasks.workunit.client.0.smithi107.stdout:6/997: dread d1/d14/de6/dad/dfb/fea [0,4194304] 0 2024-04-25T00:18:41.468 INFO:tasks.workunit.client.0.smithi107.stdout:6/998: creat d1/d14/de6/d3/d28/d79/f149 x:0 0 0 2024-04-25T00:18:41.468 INFO:tasks.workunit.client.0.smithi107.stdout:6/999: truncate d1/d14/de6/d3/d71/f148 759432 0 2024-04-25T00:18:41.471 INFO:tasks.workunit.client.0.smithi107.stderr:+ rm -rf -- ./tmp.Rw3HFxaOXA 2024-04-25T00:18:42.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:18:42.031 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:18:42.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.135+0000 7f81cb9a2640 1 -- 172.21.15.107:0/1563302789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c41029c0 msgr2=0x7f81c4102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.135+0000 7f81cb9a2640 1 --2- 172.21.15.107:0/1563302789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c41029c0 0x7f81c4102da0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f81c0009900 tx=0x7f81c002a510 comp rx=0 tx=0).stop 2024-04-25T00:18:42.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.135+0000 7f81cb9a2640 1 -- 172.21.15.107:0/1563302789 shutdown_connections 2024-04-25T00:18:42.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.135+0000 7f81cb9a2640 1 --2- 172.21.15.107:0/1563302789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81c410bd00 0x7f81c410e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.135+0000 7f81cb9a2640 1 --2- 172.21.15.107:0/1563302789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81c41032e0 0x7f81c410b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.135+0000 7f81cb9a2640 1 --2- 172.21.15.107:0/1563302789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c41029c0 0x7f81c4102da0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.135+0000 7f81cb9a2640 1 -- 172.21.15.107:0/1563302789 >> 172.21.15.107:0/1563302789 conn(0x7f81c4077740 msgr2=0x7f81c4077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:42.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.135+0000 7f81cb9a2640 1 -- 172.21.15.107:0/1563302789 shutdown_connections 2024-04-25T00:18:42.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.135+0000 7f81cb9a2640 1 -- 172.21.15.107:0/1563302789 wait complete. 2024-04-25T00:18:42.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.136+0000 7f81cb9a2640 1 Processor -- start 2024-04-25T00:18:42.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.136+0000 7f81cb9a2640 1 -- start start 2024-04-25T00:18:42.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.136+0000 7f81cb9a2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81c41029c0 0x7f81c4198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.136+0000 7f81cb9a2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81c41032e0 0x7f81c4198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.136+0000 7f81cb9a2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c410bd00 0x7f81c419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.136+0000 7f81cb9a2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f81c4111b00 con 0x7f81c410bd00 2024-04-25T00:18:42.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.136+0000 7f81cb9a2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f81c4111980 con 0x7f81c41032e0 2024-04-25T00:18:42.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.136+0000 7f81cb9a2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f81c4111c80 con 0x7f81c41029c0 2024-04-25T00:18:42.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.136+0000 7f81c9f3f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c410bd00 0x7f81c419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.136+0000 7f81c8f3d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81c41032e0 0x7f81c4198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.136+0000 7f81c9f3f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c410bd00 0x7f81c419c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41640/0 (socket says 172.21.15.107:41640) 2024-04-25T00:18:42.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.136+0000 7f81c9f3f640 1 -- 172.21.15.107:0/604520498 learned_addr learned my addr 172.21.15.107:0/604520498 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:42.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.136+0000 7f81c973e640 1 --2- 172.21.15.107:0/604520498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81c41029c0 0x7f81c4198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.136+0000 7f81c8f3d640 1 -- 172.21.15.107:0/604520498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81c41029c0 msgr2=0x7f81c4198000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.136+0000 7f81c8f3d640 1 --2- 172.21.15.107:0/604520498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81c41029c0 0x7f81c4198000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.136+0000 7f81c8f3d640 1 -- 172.21.15.107:0/604520498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c410bd00 msgr2=0x7f81c419c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.136+0000 7f81c8f3d640 1 --2- 172.21.15.107:0/604520498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c410bd00 0x7f81c419c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.136+0000 7f81c8f3d640 1 -- 172.21.15.107:0/604520498 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c419cff0 con 0x7f81c41032e0 2024-04-25T00:18:42.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.136+0000 7f81c9f3f640 1 --2- 172.21.15.107:0/604520498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c410bd00 0x7f81c419c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:42.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.136+0000 7f81c973e640 1 --2- 172.21.15.107:0/604520498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81c41029c0 0x7f81c4198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:42.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.137+0000 7f81c8f3d640 1 --2- 172.21.15.107:0/604520498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81c41032e0 0x7f81c4198540 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f81b400e9f0 tx=0x7f81b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:42.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.137+0000 7f81ba7fc640 1 -- 172.21.15.107:0/604520498 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b400ccc0 con 0x7f81c41032e0 2024-04-25T00:18:42.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.137+0000 7f81cb9a2640 1 -- 172.21.15.107:0/604520498 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c419d2e0 con 0x7f81c41032e0 2024-04-25T00:18:42.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.137+0000 7f81ba7fc640 1 -- 172.21.15.107:0/604520498 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81b4010910 con 0x7f81c41032e0 2024-04-25T00:18:42.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.137+0000 7f81ba7fc640 1 -- 172.21.15.107:0/604520498 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4018960 con 0x7f81c41032e0 2024-04-25T00:18:42.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.137+0000 7f81cb9a2640 1 -- 172.21.15.107:0/604520498 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c41a4b70 con 0x7f81c41032e0 2024-04-25T00:18:42.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.139+0000 7f81ce792740 1 Processor -- start 2024-04-25T00:18:42.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.139+0000 7f81ba7fc640 1 -- 172.21.15.107:0/604520498 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f81b4010ac0 con 0x7f81c41032e0 2024-04-25T00:18:42.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.139+0000 7f81ce792740 1 -- start start 2024-04-25T00:18:42.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.140+0000 7f81ce792740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f677f7630 0x560f677f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.140+0000 7f81ce792740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7f50 0x560f678020e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.140+0000 7f81ce792740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f67802620 0x560f67804a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.140+0000 7f81ce792740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560f6775c6b0 con 0x560f677f7f50 2024-04-25T00:18:42.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.140+0000 7f81ce792740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560f67704880 con 0x560f677f7630 2024-04-25T00:18:42.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.140+0000 7f81ce792740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560f67758670 con 0x560f67802620 2024-04-25T00:18:42.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.140+0000 7f81c973e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7f50 0x560f678020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.140+0000 7f81c9f3f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f677f7630 0x560f677f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.140+0000 7f81c9f3f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f677f7630 0x560f677f7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.107:56868/0 (socket says 172.21.15.107:56868) 2024-04-25T00:18:42.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.140+0000 7f81c9f3f640 1 -- 172.21.15.107:0/2687551812 learned_addr learned my addr 172.21.15.107:0/2687551812 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:42.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.140+0000 7f81c8f3d640 1 --2- 172.21.15.107:0/2687551812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f67802620 0x560f67804a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.140+0000 7f81ba7fc640 1 --2- 172.21.15.107:0/604520498 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f81b00a2d00 0x7f81b00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.140+0000 7f81c9f3f640 1 -- 172.21.15.107:0/2687551812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f67802620 msgr2=0x560f67804a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:42.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.140+0000 7f81c9f3f640 1 --2- 172.21.15.107:0/2687551812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f67802620 0x560f67804a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.140+0000 7f81c9f3f640 1 -- 172.21.15.107:0/2687551812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7f50 msgr2=0x560f678020e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:42.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.140+0000 7f81c9f3f640 1 --2- 172.21.15.107:0/2687551812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7f50 0x560f678020e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.140+0000 7f81c9f3f640 1 -- 172.21.15.107:0/2687551812 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f6753a7d0 con 0x560f677f7630 2024-04-25T00:18:42.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.140+0000 7f81c973e640 1 --2- 172.21.15.107:0/2687551812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7f50 0x560f678020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:42.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.140+0000 7f81ba7fc640 1 -- 172.21.15.107:0/604520498 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f81b40cfb30 con 0x7f81c41032e0 2024-04-25T00:18:42.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.140+0000 7f81c9f3f640 1 --2- 172.21.15.107:0/604520498 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f81b00a2d00 0x7f81b00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.140+0000 7f81c9f3f640 1 --2- 172.21.15.107:0/2687551812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f677f7630 0x560f677f7a10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0119d0 tx=0x7f81bc011e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d9424f87bbf91fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:42.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.140+0000 7f81af7fe640 1 -- 172.21.15.107:0/2687551812 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc017070 con 0x560f677f7630 2024-04-25T00:18:42.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.140+0000 7f81af7fe640 1 -- 172.21.15.107:0/2687551812 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81bc00fc40 con 0x560f677f7630 2024-04-25T00:18:42.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.140+0000 7f81af7fe640 1 -- 172.21.15.107:0/2687551812 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc00e3b0 con 0x560f677f7630 2024-04-25T00:18:42.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.141+0000 7f81ce792740 1 -- 172.21.15.107:0/2687551812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f677f7630 msgr2=0x560f677f7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.141+0000 7f81c9f3f640 1 --2- 172.21.15.107:0/604520498 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f81b00a2d00 0x7f81b00a51c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f81bc023000 tx=0x7f81bc024000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:42.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.141+0000 7f81ce792740 1 --2- 172.21.15.107:0/2687551812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f677f7630 0x560f677f7a10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f81bc0119d0 tx=0x7f81bc011e90 comp rx=0 tx=0).stop 2024-04-25T00:18:42.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.141+0000 7f81ce792740 1 -- 172.21.15.107:0/2687551812 shutdown_connections 2024-04-25T00:18:42.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.141+0000 7f81ce792740 1 --2- 172.21.15.107:0/2687551812 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f67802620 0x560f67804a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.141+0000 7f81ce792740 1 --2- 172.21.15.107:0/2687551812 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7f50 0x560f678020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.141+0000 7f81ce792740 1 --2- 172.21.15.107:0/2687551812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f677f7630 0x560f677f7a10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.141+0000 7f81ce792740 1 -- 172.21.15.107:0/2687551812 >> 172.21.15.107:0/2687551812 conn(0x560f677f3950 msgr2=0x560f677f3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:42.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.141+0000 7f81ce792740 1 -- 172.21.15.107:0/2687551812 shutdown_connections 2024-04-25T00:18:42.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.141+0000 7f81ce792740 1 -- 172.21.15.107:0/2687551812 wait complete. 2024-04-25T00:18:42.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.141+0000 7f81ce792740 1 -- start start 2024-04-25T00:18:42.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.142+0000 7f81ce792740 1 Processor -- start 2024-04-25T00:18:42.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.142+0000 7f81ce792740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f677f7630 0x560f678020e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.142+0000 7f81ce792740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7f50 0x560f677ff470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.142+0000 7f81ce792740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f67802620 0x560f677ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.142+0000 7f81ce792740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560f67805410 con 0x560f677f7f50 2024-04-25T00:18:42.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.142+0000 7f81ce792740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560f67717f50 con 0x560f67802620 2024-04-25T00:18:42.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.142+0000 7f81ce792740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560f67805590 con 0x560f677f7630 2024-04-25T00:18:42.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.142+0000 7f81ce792740 10 client.0 _ll_register_callbacks cb 0x560f67753000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:42.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.142+0000 7f81c8f3d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7f50 0x560f677ff470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.142+0000 7f81c8f3d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7f50 0x560f677ff470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41646/0 (socket says 172.21.15.107:41646) 2024-04-25T00:18:42.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.142+0000 7f81c8f3d640 1 -- 172.21.15.107:0/1144325567 learned_addr learned my addr 172.21.15.107:0/1144325567 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:42.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.142+0000 7f81c9f3f640 1 --2- 172.21.15.107:0/1144325567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f67802620 0x560f677ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.142+0000 7f81c973e640 1 --2- 172.21.15.107:0/1144325567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f677f7630 0x560f678020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.142+0000 7f81c8f3d640 1 -- 172.21.15.107:0/1144325567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f677f7630 msgr2=0x560f678020e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:42.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.142+0000 7f81c8f3d640 1 --2- 172.21.15.107:0/1144325567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f677f7630 0x560f678020e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.142+0000 7f81c8f3d640 1 -- 172.21.15.107:0/1144325567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f67802620 msgr2=0x560f677ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:42.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.142+0000 7f81c8f3d640 1 --2- 172.21.15.107:0/1144325567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f67802620 0x560f677ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.142+0000 7f81c8f3d640 1 -- 172.21.15.107:0/1144325567 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f674703f0 con 0x560f677f7f50 2024-04-25T00:18:42.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.143+0000 7f81c8f3d640 1 --2- 172.21.15.107:0/1144325567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7f50 0x560f677ff470 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f81b409ba90 tx=0x7f81b40d3260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557f809bf5ac5f64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:42.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.143+0000 7f81acff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:18:42.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.143+0000 7f81acff9640 1 -- 172.21.15.107:0/1144325567 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b409b6f0 con 0x560f677f7f50 2024-04-25T00:18:42.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.143+0000 7f81acff9640 1 -- 172.21.15.107:0/1144325567 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f81b409b3e0 con 0x560f677f7f50 2024-04-25T00:18:42.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.143+0000 7f81ce792740 1 -- 172.21.15.107:0/1144325567 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81bc00fc40 con 0x560f677f7f50 2024-04-25T00:18:42.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.143+0000 7f81acff9640 1 -- 172.21.15.107:0/1144325567 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f81b40cd3a0 con 0x560f677f7f50 2024-04-25T00:18:42.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.143+0000 7f81ce792740 10 client.16074 fetch_fsmap learned FSMap version 28 2024-04-25T00:18:42.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.143+0000 7f81ce792740 1 -- 172.21.15.107:0/1144325567 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f674300f0 con 0x560f677f7f50 2024-04-25T00:18:42.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.143+0000 7f81acff9640 1 -- 172.21.15.107:0/1144325567 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81bc00fc40 con 0x560f677f7f50 2024-04-25T00:18:42.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.143+0000 7f81acff9640 1 -- 172.21.15.107:0/1144325567 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f81b40b3050 con 0x560f677f7f50 2024-04-25T00:18:42.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.143+0000 7f81ce792740 10 client.16074 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:18:42.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.143+0000 7f81ce792740 10 client.16074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:18:42.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.143+0000 7f81ce792740 20 client.16074 populate_metadata read hostname 'smithi107' 2024-04-25T00:18:42.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.143+0000 7f81ce792740 1 --2- 172.21.15.107:0/1144325567 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x560f67812800 0x560f67814c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.144+0000 7f81ce792740 4 client.16074 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:18:42.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.144+0000 7f81ce792740 1 -- 172.21.15.107:0/1144325567 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f67769bf0 con 0x560f67812800 2024-04-25T00:18:42.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.144+0000 7f81c973e640 1 --2- 172.21.15.107:0/1144325567 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x560f67812800 0x560f67814c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.144+0000 7f81c973e640 1 --2- 172.21.15.107:0/1144325567 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x560f67812800 0x560f67814c40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c57e8057d2413adc server_cookie=cde439274d3ceb3 in_seq=0 out_seq=0 2024-04-25T00:18:42.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.144+0000 7f81acff9640 10 client.16074 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:42.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.145+0000 7f81acff9640 1 -- 172.21.15.107:0/1144325567 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560f67769bf0 con 0x560f67812800 2024-04-25T00:18:42.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.145+0000 7f81acff9640 10 client.16074 handle_command_reply: tid=0 2024-04-25T00:18:42.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.145+0000 7f81ce792740 1 client.16074 shutdown 2024-04-25T00:18:42.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.145+0000 7f81ce792740 1 -- 172.21.15.107:0/1144325567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7f50 msgr2=0x560f677ff470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.145+0000 7f81ce792740 1 --2- 172.21.15.107:0/1144325567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7f50 0x560f677ff470 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f81b409ba90 tx=0x7f81b40d3260 comp rx=0 tx=0).stop 2024-04-25T00:18:42.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.145+0000 7f81ce792740 1 -- 172.21.15.107:0/1144325567 shutdown_connections 2024-04-25T00:18:42.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.145+0000 7f81ce792740 1 --2- 172.21.15.107:0/1144325567 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x560f67812800 0x560f67814c40 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.145+0000 7f81ce792740 1 --2- 172.21.15.107:0/1144325567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f67802620 0x560f677ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.145+0000 7f81acff9640 0 client.16074 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:42.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.145+0000 7f81ce792740 1 --2- 172.21.15.107:0/1144325567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7f50 0x560f677ff470 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.145+0000 7f81ce792740 1 --2- 172.21.15.107:0/1144325567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f677f7630 0x560f678020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.145+0000 7f81ce792740 1 -- 172.21.15.107:0/1144325567 >> 172.21.15.107:0/1144325567 conn(0x560f677f3950 msgr2=0x560f677f9650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:42.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.145+0000 7f81ce792740 1 -- 172.21.15.107:0/1144325567 shutdown_connections 2024-04-25T00:18:42.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.145+0000 7f81ce792740 1 -- 172.21.15.107:0/1144325567 wait complete. 2024-04-25T00:18:42.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.145+0000 7f81ce792740 20 client.16074 trim_cache size 0 max 16384 2024-04-25T00:18:42.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.163+0000 7f81ce792740 1 Processor -- start 2024-04-25T00:18:42.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.164+0000 7f81ce792740 1 -- start start 2024-04-25T00:18:42.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.164+0000 7f81ce792740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7630 0x560f6780e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.164+0000 7f81ce792740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f677f7f50 0x560f67801ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.164+0000 7f81ce792740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f67802620 0x560f678020e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.164+0000 7f81ce792740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f81c41b1430 con 0x560f677f7630 2024-04-25T00:18:42.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.164+0000 7f81ce792740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f81c41aec40 con 0x560f67802620 2024-04-25T00:18:42.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.164+0000 7f81ce792740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f81c41aca90 con 0x560f677f7f50 2024-04-25T00:18:42.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.164+0000 7f81c973e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7630 0x560f6780e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.164+0000 7f81c973e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7630 0x560f6780e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41656/0 (socket says 172.21.15.107:41656) 2024-04-25T00:18:42.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.164+0000 7f81c973e640 1 -- 172.21.15.107:0/939289033 learned_addr learned my addr 172.21.15.107:0/939289033 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.164+0000 7f81c9f3f640 1 --2- 172.21.15.107:0/939289033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f67802620 0x560f678020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.164+0000 7f81c8f3d640 1 --2- 172.21.15.107:0/939289033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f677f7f50 0x560f67801ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.164+0000 7f81c973e640 1 -- 172.21.15.107:0/939289033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f677f7f50 msgr2=0x560f67801ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.164+0000 7f81c973e640 1 --2- 172.21.15.107:0/939289033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f677f7f50 0x560f67801ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.164+0000 7f81c973e640 1 -- 172.21.15.107:0/939289033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f67802620 msgr2=0x560f678020e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.164+0000 7f81c973e640 1 --2- 172.21.15.107:0/939289033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f67802620 0x560f678020e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.164+0000 7f81c973e640 1 -- 172.21.15.107:0/939289033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f677e5090 con 0x560f677f7630 2024-04-25T00:18:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.164+0000 7f81c973e640 1 --2- 172.21.15.107:0/939289033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7630 0x560f6780e2d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f81c002aa30 tx=0x7f81c0032910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c4cd5c9e00976 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.164+0000 7f81aeffd640 1 -- 172.21.15.107:0/939289033 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0035040 con 0x560f677f7630 2024-04-25T00:18:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.164+0000 7f81aeffd640 1 -- 172.21.15.107:0/939289033 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81c0032a00 con 0x560f677f7630 2024-04-25T00:18:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.164+0000 7f81aeffd640 1 -- 172.21.15.107:0/939289033 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0005280 con 0x560f677f7630 2024-04-25T00:18:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.165+0000 7f81ce792740 1 -- 172.21.15.107:0/939289033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7630 msgr2=0x560f6780e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.165+0000 7f81ce792740 1 --2- 172.21.15.107:0/939289033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7630 0x560f6780e2d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f81c002aa30 tx=0x7f81c0032910 comp rx=0 tx=0).stop 2024-04-25T00:18:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.165+0000 7f81ce792740 1 -- 172.21.15.107:0/939289033 shutdown_connections 2024-04-25T00:18:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.165+0000 7f81ce792740 1 --2- 172.21.15.107:0/939289033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f67802620 0x560f678020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.165+0000 7f81ce792740 1 --2- 172.21.15.107:0/939289033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f677f7f50 0x560f67801ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.165+0000 7f81ce792740 1 --2- 172.21.15.107:0/939289033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7630 0x560f6780e2d0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.165+0000 7f81ce792740 1 -- 172.21.15.107:0/939289033 >> 172.21.15.107:0/939289033 conn(0x560f677f3950 msgr2=0x560f677fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:42.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.165+0000 7f81ce792740 1 -- 172.21.15.107:0/939289033 shutdown_connections 2024-04-25T00:18:42.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.165+0000 7f81ce792740 1 -- 172.21.15.107:0/939289033 wait complete. 2024-04-25T00:18:42.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.165+0000 7f81ce792740 1 -- start start 2024-04-25T00:18:42.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.166+0000 7f81ce792740 1 Processor -- start 2024-04-25T00:18:42.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.166+0000 7f81ce792740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f677f7630 0x560f6780ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.166+0000 7f81ce792740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7f50 0x560f677fbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.166+0000 7f81ce792740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f67802620 0x560f677fc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.166+0000 7f81ce792740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f81c41ad100 con 0x560f677f7f50 2024-04-25T00:18:42.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.166+0000 7f81ce792740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f81c41ae570 con 0x560f677f7630 2024-04-25T00:18:42.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.166+0000 7f81ce792740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f81c41ad7d0 con 0x560f67802620 2024-04-25T00:18:42.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.166+0000 7f81ce792740 10 client.0 _ll_register_callbacks cb 0x7f81b40d3360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:42.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.166+0000 7f81c8f3d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7f50 0x560f677fbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.166+0000 7f81c8f3d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7f50 0x560f677fbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41668/0 (socket says 172.21.15.107:41668) 2024-04-25T00:18:42.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.166+0000 7f81c973e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f677f7630 0x560f6780ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.166+0000 7f81c8f3d640 1 -- 172.21.15.107:0/3391137954 learned_addr learned my addr 172.21.15.107:0/3391137954 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:42.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.166+0000 7f81c9f3f640 1 --2- 172.21.15.107:0/3391137954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f67802620 0x560f677fc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.166+0000 7f81c8f3d640 1 -- 172.21.15.107:0/3391137954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f67802620 msgr2=0x560f677fc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:42.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.166+0000 7f81c8f3d640 1 --2- 172.21.15.107:0/3391137954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f67802620 0x560f677fc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.166+0000 7f81c8f3d640 1 -- 172.21.15.107:0/3391137954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f677f7630 msgr2=0x560f6780ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:42.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.166+0000 7f81c8f3d640 1 --2- 172.21.15.107:0/3391137954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f677f7630 0x560f6780ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.166+0000 7f81c8f3d640 1 -- 172.21.15.107:0/3391137954 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f677fcd70 con 0x560f677f7f50 2024-04-25T00:18:42.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.166+0000 7f81c8f3d640 1 --2- 172.21.15.107:0/3391137954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7f50 0x560f677fbfc0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f81b40981f0 tx=0x7f81b400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed09c55d04d56929 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:42.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.166+0000 7f81acff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:18:42.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.167+0000 7f81acff9640 1 -- 172.21.15.107:0/3391137954 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b409b6f0 con 0x560f677f7f50 2024-04-25T00:18:42.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.167+0000 7f81acff9640 1 -- 172.21.15.107:0/3391137954 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f81b409ac50 con 0x560f677f7f50 2024-04-25T00:18:42.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.167+0000 7f81ce792740 1 -- 172.21.15.107:0/3391137954 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81c0032a00 con 0x560f677f7f50 2024-04-25T00:18:42.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.167+0000 7f81acff9640 1 -- 172.21.15.107:0/3391137954 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f81b40b9050 con 0x560f677f7f50 2024-04-25T00:18:42.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.167+0000 7f81ce792740 10 client.16080 fetch_fsmap learned FSMap version 28 2024-04-25T00:18:42.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.167+0000 7f81ce792740 1 -- 172.21.15.107:0/3391137954 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560f67878e70 con 0x560f677f7f50 2024-04-25T00:18:42.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.167+0000 7f81acff9640 1 -- 172.21.15.107:0/3391137954 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81c0032a00 con 0x560f677f7f50 2024-04-25T00:18:42.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.167+0000 7f81acff9640 1 -- 172.21.15.107:0/3391137954 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f81b40b3070 con 0x560f677f7f50 2024-04-25T00:18:42.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.167+0000 7f81ce792740 10 client.16080 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:18:42.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.167+0000 7f81ce792740 10 client.16080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:18:42.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.167+0000 7f81ce792740 20 client.16080 populate_metadata read hostname 'smithi107' 2024-04-25T00:18:42.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.167+0000 7f81ce792740 1 --2- 172.21.15.107:0/3391137954 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x560f67879670 0x560f6787ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.167+0000 7f81ce792740 4 client.16080 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:18:42.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.167+0000 7f81ce792740 1 -- 172.21.15.107:0/3391137954 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560f67763f40 con 0x560f67879670 2024-04-25T00:18:42.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.167+0000 7f81c973e640 1 --2- 172.21.15.107:0/3391137954 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x560f67879670 0x560f6787ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.168+0000 7f81c973e640 1 --2- 172.21.15.107:0/3391137954 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x560f67879670 0x560f6787ba90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5793682dbf246bd server_cookie=f59c3566ee7ef2d8 in_seq=0 out_seq=0 2024-04-25T00:18:42.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.168+0000 7f81acff9640 10 client.16080 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:42.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.168+0000 7f81acff9640 1 -- 172.21.15.107:0/3391137954 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560f67763f40 con 0x560f67879670 2024-04-25T00:18:42.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.168+0000 7f81acff9640 10 client.16080 handle_command_reply: tid=0 2024-04-25T00:18:42.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.168+0000 7f81ce792740 1 client.16080 shutdown 2024-04-25T00:18:42.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.168+0000 7f81ce792740 1 -- 172.21.15.107:0/3391137954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7f50 msgr2=0x560f677fbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.168+0000 7f81ce792740 1 --2- 172.21.15.107:0/3391137954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7f50 0x560f677fbfc0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f81b40981f0 tx=0x7f81b400c840 comp rx=0 tx=0).stop 2024-04-25T00:18:42.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.168+0000 7f81ce792740 1 -- 172.21.15.107:0/3391137954 shutdown_connections 2024-04-25T00:18:42.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.168+0000 7f81ce792740 1 --2- 172.21.15.107:0/3391137954 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x560f67879670 0x560f6787ba90 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.168+0000 7f81ce792740 1 --2- 172.21.15.107:0/3391137954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560f67802620 0x560f677fc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.168+0000 7f81acff9640 0 client.16080 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:42.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.168+0000 7f81ce792740 1 --2- 172.21.15.107:0/3391137954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f677f7f50 0x560f677fbfc0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.168+0000 7f81ce792740 1 --2- 172.21.15.107:0/3391137954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560f677f7630 0x560f6780ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.168+0000 7f81ce792740 1 -- 172.21.15.107:0/3391137954 >> 172.21.15.107:0/3391137954 conn(0x560f677f3950 msgr2=0x560f677f2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:42.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.168+0000 7f81ce792740 1 -- 172.21.15.107:0/3391137954 shutdown_connections 2024-04-25T00:18:42.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.168+0000 7f81ce792740 1 -- 172.21.15.107:0/3391137954 wait complete. 2024-04-25T00:18:42.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.168+0000 7f81ce792740 20 client.16080 trim_cache size 0 max 16384 2024-04-25T00:18:42.176 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:18:42.176 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-04-25T00:18:42.176 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-04-25T00:18:42.176 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:18:42.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.169+0000 7f81cb9a2640 1 -- 172.21.15.107:0/604520498 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f81b00a2d00 msgr2=0x7f81b00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.169+0000 7f81cb9a2640 1 --2- 172.21.15.107:0/604520498 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f81b00a2d00 0x7f81b00a51c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f81bc023000 tx=0x7f81bc024000 comp rx=0 tx=0).stop 2024-04-25T00:18:42.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.169+0000 7f81cb9a2640 1 -- 172.21.15.107:0/604520498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81c41032e0 msgr2=0x7f81c4198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.169+0000 7f81cb9a2640 1 --2- 172.21.15.107:0/604520498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81c41032e0 0x7f81c4198540 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f81b400e9f0 tx=0x7f81b400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.169+0000 7f81cb9a2640 1 -- 172.21.15.107:0/604520498 shutdown_connections 2024-04-25T00:18:42.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.169+0000 7f81cb9a2640 1 --2- 172.21.15.107:0/604520498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f81c410bd00 0x7f81c419c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.169+0000 7f81cb9a2640 1 --2- 172.21.15.107:0/604520498 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f81b00a2d00 0x7f81b00a51c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.169+0000 7f81cb9a2640 1 --2- 172.21.15.107:0/604520498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f81c41032e0 0x7f81c4198540 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.169+0000 7f81cb9a2640 1 --2- 172.21.15.107:0/604520498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f81c41029c0 0x7f81c4198000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.169+0000 7f81cb9a2640 1 -- 172.21.15.107:0/604520498 >> 172.21.15.107:0/604520498 conn(0x7f81c4077740 msgr2=0x7f81c40fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:42.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.169+0000 7f81cb9a2640 1 -- 172.21.15.107:0/604520498 shutdown_connections 2024-04-25T00:18:42.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.169+0000 7f81cb9a2640 1 -- 172.21.15.107:0/604520498 wait complete. 2024-04-25T00:18:42.179 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:41 smithi107 ceph-mon[29153]: pgmap v70: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 171 MiB/s rd, 208 MiB/s wr, 332 op/s 2024-04-25T00:18:42.180 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:18:42.180 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:18:42.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:41 smithi170 ceph-mon[32807]: pgmap v70: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 171 MiB/s rd, 208 MiB/s wr, 332 op/s 2024-04-25T00:18:42.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.285+0000 7faa32763640 1 -- 172.21.15.107:0/1563945525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa2c1029d0 msgr2=0x7faa2c102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.285+0000 7faa32763640 1 --2- 172.21.15.107:0/1563945525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa2c1029d0 0x7faa2c102db0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faa1c009860 tx=0x7faa1c02a460 comp rx=0 tx=0).stop 2024-04-25T00:18:42.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.286+0000 7faa32763640 1 -- 172.21.15.107:0/1563945525 shutdown_connections 2024-04-25T00:18:42.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.286+0000 7faa32763640 1 --2- 172.21.15.107:0/1563945525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa2c10bd10 0x7faa2c10e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.286+0000 7faa32763640 1 --2- 172.21.15.107:0/1563945525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa2c1032f0 0x7faa2c10b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.286+0000 7faa32763640 1 --2- 172.21.15.107:0/1563945525 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa2c1029d0 0x7faa2c102db0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.286+0000 7faa32763640 1 -- 172.21.15.107:0/1563945525 >> 172.21.15.107:0/1563945525 conn(0x7faa2c077740 msgr2=0x7faa2c077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:42.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.286+0000 7faa32763640 1 -- 172.21.15.107:0/1563945525 shutdown_connections 2024-04-25T00:18:42.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.286+0000 7faa32763640 1 -- 172.21.15.107:0/1563945525 wait complete. 2024-04-25T00:18:42.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.286+0000 7faa32763640 1 Processor -- start 2024-04-25T00:18:42.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.286+0000 7faa32763640 1 -- start start 2024-04-25T00:18:42.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.287+0000 7faa32763640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa2c1029d0 0x7faa2c198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.287+0000 7faa32763640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa2c1032f0 0x7faa2c198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.287+0000 7faa32763640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa2c10bd10 0x7faa2c19c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.287+0000 7faa32763640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c111b10 con 0x7faa2c1032f0 2024-04-25T00:18:42.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.287+0000 7faa32763640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c111990 con 0x7faa2c1029d0 2024-04-25T00:18:42.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.287+0000 7faa32763640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c111c90 con 0x7faa2c10bd10 2024-04-25T00:18:42.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.287+0000 7faa2b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa2c1032f0 0x7faa2c198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.287+0000 7faa2bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa2c1029d0 0x7faa2c198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.287+0000 7faa2b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa2c1032f0 0x7faa2c198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41696/0 (socket says 172.21.15.107:41696) 2024-04-25T00:18:42.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.287+0000 7faa2b7fe640 1 -- 172.21.15.107:0/1673906518 learned_addr learned my addr 172.21.15.107:0/1673906518 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:42.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.287+0000 7faa30d00640 1 --2- 172.21.15.107:0/1673906518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa2c10bd10 0x7faa2c19c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.287+0000 7faa2b7fe640 1 -- 172.21.15.107:0/1673906518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa2c10bd10 msgr2=0x7faa2c19c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.287+0000 7faa2b7fe640 1 --2- 172.21.15.107:0/1673906518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa2c10bd10 0x7faa2c19c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.287+0000 7faa2b7fe640 1 -- 172.21.15.107:0/1673906518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa2c1029d0 msgr2=0x7faa2c198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.287+0000 7faa2b7fe640 1 --2- 172.21.15.107:0/1673906518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa2c1029d0 0x7faa2c198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.287+0000 7faa2b7fe640 1 -- 172.21.15.107:0/1673906518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa2c19cfd0 con 0x7faa2c1032f0 2024-04-25T00:18:42.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.287+0000 7faa2b7fe640 1 --2- 172.21.15.107:0/1673906518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa2c1032f0 0x7faa2c198570 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7faa2000e9c0 tx=0x7faa2000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:42.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.287+0000 7faa297fa640 1 -- 172.21.15.107:0/1673906518 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2000cde0 con 0x7faa2c1032f0 2024-04-25T00:18:42.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.287+0000 7faa32763640 1 -- 172.21.15.107:0/1673906518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa2c19d2c0 con 0x7faa2c1032f0 2024-04-25T00:18:42.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.287+0000 7faa297fa640 1 -- 172.21.15.107:0/1673906518 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa200035f0 con 0x7faa2c1032f0 2024-04-25T00:18:42.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.287+0000 7faa297fa640 1 -- 172.21.15.107:0/1673906518 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa20010650 con 0x7faa2c1032f0 2024-04-25T00:18:42.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.287+0000 7faa32763640 1 -- 172.21.15.107:0/1673906518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa2c1a4b50 con 0x7faa2c1032f0 2024-04-25T00:18:42.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.288+0000 7faa32763640 1 -- 172.21.15.107:0/1673906518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa2c068560 con 0x7faa2c1032f0 2024-04-25T00:18:42.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.289+0000 7faa297fa640 1 -- 172.21.15.107:0/1673906518 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7faa200037a0 con 0x7faa2c1032f0 2024-04-25T00:18:42.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.290+0000 7faa297fa640 1 --2- 172.21.15.107:0/1673906518 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7faa140a2d00 0x7faa140a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.290+0000 7faa2bfff640 1 --2- 172.21.15.107:0/1673906518 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7faa140a2d00 0x7faa140a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.290+0000 7faa297fa640 1 -- 172.21.15.107:0/1673906518 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7faa200cfae0 con 0x7faa2c1032f0 2024-04-25T00:18:42.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.290+0000 7faa2bfff640 1 --2- 172.21.15.107:0/1673906518 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7faa140a2d00 0x7faa140a51c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faa1c009830 tx=0x7faa1c0094d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:42.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.294+0000 7faa297fa640 1 -- 172.21.15.107:0/1673906518 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faa2009c390 con 0x7faa2c1032f0 2024-04-25T00:18:42.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:41 smithi195 ceph-mon[32833]: pgmap v70: 129 pgs: 129 active+clean; 1.9 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 171 MiB/s rd, 208 MiB/s wr, 332 op/s 2024-04-25T00:18:42.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.545+0000 7faa32763640 1 -- 172.21.15.107:0/1673906518 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa2c106ef0 con 0x7faa2c1032f0 2024-04-25T00:18:42.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.545+0000 7faa297fa640 1 -- 172.21.15.107:0/1673906518 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11906 (secure 0 0 0) 0x7faa200b3ea0 con 0x7faa2c1032f0 2024-04-25T00:18:42.546 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:18:42.546 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:18:38.933541+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24493,24521]},"id":1}]} 2024-04-25T00:18:42.546 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 28 2024-04-25T00:18:42.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.547+0000 7faa32763640 1 -- 172.21.15.107:0/1673906518 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7faa140a2d00 msgr2=0x7faa140a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.547+0000 7faa32763640 1 --2- 172.21.15.107:0/1673906518 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7faa140a2d00 0x7faa140a51c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faa1c009830 tx=0x7faa1c0094d0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.547+0000 7faa32763640 1 -- 172.21.15.107:0/1673906518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa2c1032f0 msgr2=0x7faa2c198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.547+0000 7faa32763640 1 --2- 172.21.15.107:0/1673906518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa2c1032f0 0x7faa2c198570 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7faa2000e9c0 tx=0x7faa2000ee80 comp rx=0 tx=0).stop 2024-04-25T00:18:42.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.548+0000 7faa32763640 1 -- 172.21.15.107:0/1673906518 shutdown_connections 2024-04-25T00:18:42.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.548+0000 7faa32763640 1 --2- 172.21.15.107:0/1673906518 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faa2c10bd10 0x7faa2c19c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.548+0000 7faa32763640 1 --2- 172.21.15.107:0/1673906518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa2c1032f0 0x7faa2c198570 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.548+0000 7faa32763640 1 --2- 172.21.15.107:0/1673906518 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7faa140a2d00 0x7faa140a51c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.548+0000 7faa32763640 1 --2- 172.21.15.107:0/1673906518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faa2c1029d0 0x7faa2c198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.548+0000 7faa32763640 1 -- 172.21.15.107:0/1673906518 >> 172.21.15.107:0/1673906518 conn(0x7faa2c077740 msgr2=0x7faa2c0fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:42.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.548+0000 7faa32763640 1 -- 172.21.15.107:0/1673906518 shutdown_connections 2024-04-25T00:18:42.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.548+0000 7faa32763640 1 -- 172.21.15.107:0/1673906518 wait complete. 2024-04-25T00:18:42.559 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:18:42.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.665+0000 7f55fda5d640 1 -- 172.21.15.107:0/3079637441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55f8102150 msgr2=0x7f55f810e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.665+0000 7f55fda5d640 1 --2- 172.21.15.107:0/3079637441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55f8102150 0x7f55f810e660 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009cd0 tx=0x7f55f002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.665+0000 7f55fda5d640 1 -- 172.21.15.107:0/3079637441 shutdown_connections 2024-04-25T00:18:42.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.665+0000 7f55fda5d640 1 --2- 172.21.15.107:0/3079637441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55f8102150 0x7f55f810e660 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.665+0000 7f55fda5d640 1 --2- 172.21.15.107:0/3079637441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55f8101790 0x7f55f8101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.665+0000 7f55fda5d640 1 --2- 172.21.15.107:0/3079637441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55f81077a0 0x7f55f8107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.665+0000 7f55fda5d640 1 -- 172.21.15.107:0/3079637441 >> 172.21.15.107:0/3079637441 conn(0x7f55f80fd2a0 msgr2=0x7f55f80ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:42.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.665+0000 7f55fda5d640 1 -- 172.21.15.107:0/3079637441 shutdown_connections 2024-04-25T00:18:42.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.665+0000 7f55fda5d640 1 -- 172.21.15.107:0/3079637441 wait complete. 2024-04-25T00:18:42.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.665+0000 7f55fda5d640 1 Processor -- start 2024-04-25T00:18:42.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.666+0000 7f55fda5d640 1 -- start start 2024-04-25T00:18:42.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.666+0000 7f55fda5d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55f8101790 0x7f55f8198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.668 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.666+0000 7f55fda5d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55f8102150 0x7f55f81985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.666+0000 7f55fda5d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55f81077a0 0x7f55f819c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.666+0000 7f55fda5d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f55f8111bb0 con 0x7f55f8102150 2024-04-25T00:18:42.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.666+0000 7f55fda5d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f55f8111a30 con 0x7f55f8101790 2024-04-25T00:18:42.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.666+0000 7f55fda5d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f55f8111d30 con 0x7f55f81077a0 2024-04-25T00:18:42.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.666+0000 7f55f67fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55f8102150 0x7f55f81985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.666+0000 7f55f67fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55f8102150 0x7f55f81985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41720/0 (socket says 172.21.15.107:41720) 2024-04-25T00:18:42.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.666+0000 7f55f67fc640 1 -- 172.21.15.107:0/3305050548 learned_addr learned my addr 172.21.15.107:0/3305050548 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:42.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.666+0000 7f55f6ffd640 1 --2- 172.21.15.107:0/3305050548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55f8101790 0x7f55f8198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.666+0000 7f55f77fe640 1 --2- 172.21.15.107:0/3305050548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55f81077a0 0x7f55f819c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.666+0000 7f55f67fc640 1 -- 172.21.15.107:0/3305050548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55f81077a0 msgr2=0x7f55f819c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.666+0000 7f55f67fc640 1 --2- 172.21.15.107:0/3305050548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55f81077a0 0x7f55f819c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.666+0000 7f55f67fc640 1 -- 172.21.15.107:0/3305050548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55f8101790 msgr2=0x7f55f8198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.666+0000 7f55f67fc640 1 --2- 172.21.15.107:0/3305050548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55f8101790 0x7f55f8198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.669 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.666+0000 7f55f67fc640 1 -- 172.21.15.107:0/3305050548 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f819d080 con 0x7f55f8102150 2024-04-25T00:18:42.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.666+0000 7f55f77fe640 1 --2- 172.21.15.107:0/3305050548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55f81077a0 0x7f55f819c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:18:42.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.666+0000 7f55f67fc640 1 --2- 172.21.15.107:0/3305050548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55f8102150 0x7f55f81985d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00e9f0 tx=0x7f55ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:42.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.667+0000 7f55fca5b640 1 -- 172.21.15.107:0/3305050548 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec00cd80 con 0x7f55f8102150 2024-04-25T00:18:42.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.667+0000 7f55fda5d640 1 -- 172.21.15.107:0/3305050548 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f819d370 con 0x7f55f8102150 2024-04-25T00:18:42.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.667+0000 7f55fca5b640 1 -- 172.21.15.107:0/3305050548 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55ec005500 con 0x7f55f8102150 2024-04-25T00:18:42.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.667+0000 7f55fca5b640 1 -- 172.21.15.107:0/3305050548 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec010690 con 0x7f55f8102150 2024-04-25T00:18:42.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.667+0000 7f55fda5d640 1 -- 172.21.15.107:0/3305050548 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f81a4c00 con 0x7f55f8102150 2024-04-25T00:18:42.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.669+0000 7f55fca5b640 1 -- 172.21.15.107:0/3305050548 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f55ec0046c0 con 0x7f55f8102150 2024-04-25T00:18:42.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.669+0000 7f560076a740 1 Processor -- start 2024-04-25T00:18:42.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.669+0000 7f55fca5b640 1 --2- 172.21.15.107:0/3305050548 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f55e00a2d00 0x7f55e00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.669+0000 7f560076a740 1 -- start start 2024-04-25T00:18:42.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.669+0000 7f55f77fe640 1 --2- 172.21.15.107:0/3305050548 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f55e00a2d00 0x7f55e00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.669+0000 7f560076a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5581e7ddc580 0x5581e7ddc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.669+0000 7f560076a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7ddcea0 0x5581e7de7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.669+0000 7f55fca5b640 1 -- 172.21.15.107:0/3305050548 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f55ec0cfc90 con 0x7f55f8102150 2024-04-25T00:18:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.669+0000 7f560076a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581e7de7570 0x5581e7de9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.669+0000 7f560076a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5581e7d41670 con 0x5581e7ddcea0 2024-04-25T00:18:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.669+0000 7f560076a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5581e7ce9970 con 0x5581e7de7570 2024-04-25T00:18:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.669+0000 7f560076a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5581e7d3d630 con 0x5581e7ddc580 2024-04-25T00:18:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.670+0000 7f55f67fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7ddcea0 0x5581e7de7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.670+0000 7f55f67fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7ddcea0 0x5581e7de7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41724/0 (socket says 172.21.15.107:41724) 2024-04-25T00:18:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.670+0000 7f55f67fc640 1 -- 172.21.15.107:0/2817904784 learned_addr learned my addr 172.21.15.107:0/2817904784 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.670+0000 7f55f6ffd640 1 --2- 172.21.15.107:0/2817904784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5581e7ddc580 0x5581e7ddc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.670+0000 7f55f77fe640 1 --2- 172.21.15.107:0/2817904784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581e7de7570 0x5581e7de9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.670+0000 7f55f67fc640 1 -- 172.21.15.107:0/2817904784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5581e7ddc580 msgr2=0x5581e7ddc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.670+0000 7f55f67fc640 1 --2- 172.21.15.107:0/2817904784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5581e7ddc580 0x5581e7ddc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.670+0000 7f55f67fc640 1 -- 172.21.15.107:0/2817904784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581e7de7570 msgr2=0x5581e7de9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.670+0000 7f55f67fc640 1 --2- 172.21.15.107:0/2817904784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581e7de7570 0x5581e7de9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.670+0000 7f55f67fc640 1 -- 172.21.15.107:0/2817904784 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581e7d0d400 con 0x5581e7ddcea0 2024-04-25T00:18:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.670+0000 7f55f6ffd640 1 --2- 172.21.15.107:0/2817904784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5581e7ddc580 0x5581e7ddc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:18:42.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.670+0000 7f55f77fe640 1 --2- 172.21.15.107:0/2817904784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581e7de7570 0x5581e7de9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:18:42.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.670+0000 7f55f77fe640 1 --2- 172.21.15.107:0/3305050548 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f55e00a2d00 0x7f55e00a51c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009ca0 tx=0x7f55f00099e0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:42.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.670+0000 7f55f67fc640 1 --2- 172.21.15.107:0/2817904784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7ddcea0 0x5581e7de7030 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009380 tx=0x7f55ec09b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3df64c5af27ed77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:42.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.670+0000 7f55ddffb640 1 -- 172.21.15.107:0/2817904784 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec09e070 con 0x5581e7ddcea0 2024-04-25T00:18:42.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.670+0000 7f55ddffb640 1 -- 172.21.15.107:0/2817904784 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55ec0a01f0 con 0x5581e7ddcea0 2024-04-25T00:18:42.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.670+0000 7f55ddffb640 1 -- 172.21.15.107:0/2817904784 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec0a4880 con 0x5581e7ddcea0 2024-04-25T00:18:42.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.671+0000 7f560076a740 1 -- 172.21.15.107:0/2817904784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7ddcea0 msgr2=0x5581e7de7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.671+0000 7f560076a740 1 --2- 172.21.15.107:0/2817904784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7ddcea0 0x5581e7de7030 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009380 tx=0x7f55ec09b7a0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.671+0000 7f560076a740 1 -- 172.21.15.107:0/2817904784 shutdown_connections 2024-04-25T00:18:42.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.671+0000 7f560076a740 1 --2- 172.21.15.107:0/2817904784 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581e7de7570 0x5581e7de9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.671+0000 7f560076a740 1 --2- 172.21.15.107:0/2817904784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7ddcea0 0x5581e7de7030 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.671+0000 7f560076a740 1 --2- 172.21.15.107:0/2817904784 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5581e7ddc580 0x5581e7ddc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.671+0000 7f560076a740 1 -- 172.21.15.107:0/2817904784 >> 172.21.15.107:0/2817904784 conn(0x5581e7dd88a0 msgr2=0x5581e7dd8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.671+0000 7f560076a740 1 -- 172.21.15.107:0/2817904784 shutdown_connections 2024-04-25T00:18:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.671+0000 7f560076a740 1 -- 172.21.15.107:0/2817904784 wait complete. 2024-04-25T00:18:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.671+0000 7f560076a740 1 -- start start 2024-04-25T00:18:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.672+0000 7f560076a740 1 Processor -- start 2024-04-25T00:18:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.672+0000 7f560076a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5581e7ddc580 0x5581e7de6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.672+0000 7f560076a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581e7ddcea0 0x5581e7de42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.672+0000 7f560076a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7de7570 0x5581e7de49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.672+0000 7f560076a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5581e7dee770 con 0x5581e7de7570 2024-04-25T00:18:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.672+0000 7f560076a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5581e7cfd050 con 0x5581e7ddcea0 2024-04-25T00:18:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.672+0000 7f560076a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5581e7dee8f0 con 0x5581e7ddc580 2024-04-25T00:18:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.672+0000 7f560076a740 10 client.0 _ll_register_callbacks cb 0x5581e7a54be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.672+0000 7f55f77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7de7570 0x5581e7de49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.672+0000 7f55f67fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581e7ddcea0 0x5581e7de42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.672+0000 7f55f6ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5581e7ddc580 0x5581e7de6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.672+0000 7f55f77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7de7570 0x5581e7de49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41740/0 (socket says 172.21.15.107:41740) 2024-04-25T00:18:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.672+0000 7f55f67fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581e7ddcea0 0x5581e7de42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.107:56946/0 (socket says 172.21.15.107:56946) 2024-04-25T00:18:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.672+0000 7f55f77fe640 1 -- 172.21.15.107:0/1445669448 learned_addr learned my addr 172.21.15.107:0/1445669448 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.672+0000 7f55f77fe640 1 -- 172.21.15.107:0/1445669448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5581e7ddc580 msgr2=0x5581e7de6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.672+0000 7f55f77fe640 1 --2- 172.21.15.107:0/1445669448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5581e7ddc580 0x5581e7de6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.672+0000 7f55f77fe640 1 -- 172.21.15.107:0/1445669448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581e7ddcea0 msgr2=0x5581e7de42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.672+0000 7f55f77fe640 1 --2- 172.21.15.107:0/1445669448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581e7ddcea0 0x5581e7de42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.672+0000 7f55f77fe640 1 -- 172.21.15.107:0/1445669448 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581e7ccedc0 con 0x5581e7de7570 2024-04-25T00:18:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.672+0000 7f55f6ffd640 1 --2- 172.21.15.107:0/1445669448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5581e7ddc580 0x5581e7de6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.672+0000 7f55f77fe640 1 --2- 172.21.15.107:0/1445669448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7de7570 0x5581e7de49e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f55f0031750 tx=0x7f55f002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7ff9de6fdd80dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.672+0000 7f55d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:18:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.673+0000 7f55d6ffd640 1 -- 172.21.15.107:0/1445669448 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f00047d0 con 0x5581e7de7570 2024-04-25T00:18:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.673+0000 7f55d6ffd640 1 -- 172.21.15.107:0/1445669448 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f55f00305a0 con 0x5581e7de7570 2024-04-25T00:18:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.673+0000 7f560076a740 1 -- 172.21.15.107:0/1445669448 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55ec0a01f0 con 0x5581e7de7570 2024-04-25T00:18:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.673+0000 7f55d6ffd640 1 -- 172.21.15.107:0/1445669448 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f55f00307a0 con 0x5581e7de7570 2024-04-25T00:18:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.673+0000 7f560076a740 10 client.16098 fetch_fsmap learned FSMap version 28 2024-04-25T00:18:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.673+0000 7f560076a740 1 -- 172.21.15.107:0/1445669448 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5581e7a553f0 con 0x5581e7de7570 2024-04-25T00:18:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.673+0000 7f55d6ffd640 1 -- 172.21.15.107:0/1445669448 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55ec0a01f0 con 0x5581e7de7570 2024-04-25T00:18:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.673+0000 7f55d6ffd640 1 -- 172.21.15.107:0/1445669448 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f55f0030070 con 0x5581e7de7570 2024-04-25T00:18:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.673+0000 7f560076a740 10 client.16098 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:18:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.673+0000 7f560076a740 10 client.16098 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:18:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.673+0000 7f560076a740 20 client.16098 populate_metadata read hostname 'smithi107' 2024-04-25T00:18:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.673+0000 7f560076a740 1 --2- 172.21.15.107:0/1445669448 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5581e7dfbb20 0x5581e7dfdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.673+0000 7f560076a740 4 client.16098 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:18:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.673+0000 7f560076a740 1 -- 172.21.15.107:0/1445669448 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581e7d4ebb0 con 0x5581e7dfbb20 2024-04-25T00:18:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.673+0000 7f55f6ffd640 1 --2- 172.21.15.107:0/1445669448 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5581e7dfbb20 0x5581e7dfdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.674+0000 7f55d6ffd640 10 client.16098 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:18:42.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.674+0000 7f55f6ffd640 1 --2- 172.21.15.107:0/1445669448 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5581e7dfbb20 0x5581e7dfdf80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=755b5ae02beae705 server_cookie=475f90f008cc052f in_seq=0 out_seq=0 2024-04-25T00:18:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.677+0000 7f55d6ffd640 1 -- 172.21.15.107:0/1445669448 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5581e7d4ebb0 con 0x5581e7dfbb20 2024-04-25T00:18:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.677+0000 7f55d6ffd640 10 client.16098 handle_command_reply: tid=0 2024-04-25T00:18:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.677+0000 7f560076a740 1 client.16098 shutdown 2024-04-25T00:18:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.677+0000 7f560076a740 1 -- 172.21.15.107:0/1445669448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7de7570 msgr2=0x5581e7de49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.677+0000 7f560076a740 1 --2- 172.21.15.107:0/1445669448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7de7570 0x5581e7de49e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f55f0031750 tx=0x7f55f002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.677+0000 7f560076a740 1 -- 172.21.15.107:0/1445669448 shutdown_connections 2024-04-25T00:18:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.677+0000 7f560076a740 1 --2- 172.21.15.107:0/1445669448 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5581e7dfbb20 0x5581e7dfdf80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.677+0000 7f560076a740 1 --2- 172.21.15.107:0/1445669448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7de7570 0x5581e7de49e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.677+0000 7f560076a740 1 --2- 172.21.15.107:0/1445669448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581e7ddcea0 0x5581e7de42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.677+0000 7f560076a740 1 --2- 172.21.15.107:0/1445669448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5581e7ddc580 0x5581e7de6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.677+0000 7f55d6ffd640 0 client.16098 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:18:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.677+0000 7f560076a740 1 -- 172.21.15.107:0/1445669448 >> 172.21.15.107:0/1445669448 conn(0x5581e7dd88a0 msgr2=0x5581e7dde6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.677+0000 7f560076a740 1 -- 172.21.15.107:0/1445669448 shutdown_connections 2024-04-25T00:18:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.677+0000 7f560076a740 1 -- 172.21.15.107:0/1445669448 wait complete. 2024-04-25T00:18:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.678+0000 7f560076a740 20 client.16098 trim_cache size 0 max 16384 2024-04-25T00:18:42.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.700+0000 7f560076a740 1 Processor -- start 2024-04-25T00:18:42.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.700+0000 7f560076a740 1 -- start start 2024-04-25T00:18:42.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.700+0000 7f560076a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581e7ddc580 0x5581e7df7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.700+0000 7f560076a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5581e7ddcea0 0x5581e7dd79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.700+0000 7f560076a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7de7570 0x5581e7e03850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.700+0000 7f560076a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81af350 con 0x5581e7de7570 2024-04-25T00:18:42.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.700+0000 7f560076a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81acb60 con 0x5581e7ddc580 2024-04-25T00:18:42.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.700+0000 7f560076a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81aa9b0 con 0x5581e7ddcea0 2024-04-25T00:18:42.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.701+0000 7f55f77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7de7570 0x5581e7e03850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.701+0000 7f55f77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7de7570 0x5581e7e03850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41744/0 (socket says 172.21.15.107:41744) 2024-04-25T00:18:42.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.701+0000 7f55f77fe640 1 -- 172.21.15.107:0/998754542 learned_addr learned my addr 172.21.15.107:0/998754542 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:42.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.701+0000 7f55f6ffd640 1 --2- 172.21.15.107:0/998754542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581e7ddc580 0x5581e7df7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.701+0000 7f55f77fe640 1 -- 172.21.15.107:0/998754542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5581e7ddcea0 msgr2=0x5581e7dd79c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:18:42.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.701+0000 7f55f77fe640 1 --2- 172.21.15.107:0/998754542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5581e7ddcea0 0x5581e7dd79c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.701+0000 7f55f77fe640 1 -- 172.21.15.107:0/998754542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581e7ddc580 msgr2=0x5581e7df7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:42.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.701+0000 7f55f77fe640 1 --2- 172.21.15.107:0/998754542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581e7ddc580 0x5581e7df7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.701+0000 7f55f77fe640 1 -- 172.21.15.107:0/998754542 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581e7de9cd0 con 0x5581e7de7570 2024-04-25T00:18:42.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.702+0000 7f55f77fe640 1 --2- 172.21.15.107:0/998754542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7de7570 0x5581e7e03850 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009bf0 tx=0x7f55f00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57d1b345b182ff8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:42.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.703+0000 7f55dd7fa640 1 -- 172.21.15.107:0/998754542 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f00305a0 con 0x5581e7de7570 2024-04-25T00:18:42.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.703+0000 7f55dd7fa640 1 -- 172.21.15.107:0/998754542 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55f0036040 con 0x5581e7de7570 2024-04-25T00:18:42.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.703+0000 7f55dd7fa640 1 -- 172.21.15.107:0/998754542 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f003d650 con 0x5581e7de7570 2024-04-25T00:18:42.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.704+0000 7f560076a740 1 -- 172.21.15.107:0/998754542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7de7570 msgr2=0x5581e7e03850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.704+0000 7f560076a740 1 --2- 172.21.15.107:0/998754542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7de7570 0x5581e7e03850 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009bf0 tx=0x7f55f00095c0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.704+0000 7f560076a740 1 -- 172.21.15.107:0/998754542 shutdown_connections 2024-04-25T00:18:42.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.705+0000 7f560076a740 1 --2- 172.21.15.107:0/998754542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7de7570 0x5581e7e03850 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.705+0000 7f560076a740 1 --2- 172.21.15.107:0/998754542 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5581e7ddcea0 0x5581e7dd79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.705+0000 7f560076a740 1 --2- 172.21.15.107:0/998754542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581e7ddc580 0x5581e7df7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.705+0000 7f560076a740 1 -- 172.21.15.107:0/998754542 >> 172.21.15.107:0/998754542 conn(0x5581e7dd88a0 msgr2=0x5581e7dd72b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:42.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.705+0000 7f560076a740 1 -- 172.21.15.107:0/998754542 shutdown_connections 2024-04-25T00:18:42.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.705+0000 7f560076a740 1 -- 172.21.15.107:0/998754542 wait complete. 2024-04-25T00:18:42.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.705+0000 7f560076a740 1 -- start start 2024-04-25T00:18:42.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.706+0000 7f560076a740 1 Processor -- start 2024-04-25T00:18:42.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.706+0000 7f560076a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5581e7ddc580 0x5581e7decce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.706+0000 7f560076a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7ddcea0 0x5581e7ded220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.706+0000 7f560076a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581e7de7570 0x5581e7ded760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.706+0000 7f560076a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81ab020 con 0x5581e7ddcea0 2024-04-25T00:18:42.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.706+0000 7f560076a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81ac490 con 0x5581e7de7570 2024-04-25T00:18:42.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.706+0000 7f560076a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f55f81ab6f0 con 0x5581e7ddc580 2024-04-25T00:18:42.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.706+0000 7f560076a740 10 client.0 _ll_register_callbacks cb 0x7f55f0041050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:42.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.707+0000 7f55f77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581e7de7570 0x5581e7ded760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.707+0000 7f55f67fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7ddcea0 0x5581e7ded220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.707+0000 7f55f67fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7ddcea0 0x5581e7ded220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41752/0 (socket says 172.21.15.107:41752) 2024-04-25T00:18:42.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.707+0000 7f55f67fc640 1 -- 172.21.15.107:0/1800873718 learned_addr learned my addr 172.21.15.107:0/1800873718 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:42.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.708+0000 7f55f6ffd640 1 --2- 172.21.15.107:0/1800873718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5581e7ddc580 0x5581e7decce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.708+0000 7f55f67fc640 1 -- 172.21.15.107:0/1800873718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5581e7ddc580 msgr2=0x5581e7decce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:42.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.708+0000 7f55f67fc640 1 --2- 172.21.15.107:0/1800873718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5581e7ddc580 0x5581e7decce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.708+0000 7f55f67fc640 1 -- 172.21.15.107:0/1800873718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581e7de7570 msgr2=0x5581e7ded760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:42.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.708+0000 7f55f67fc640 1 --2- 172.21.15.107:0/1800873718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581e7de7570 0x5581e7ded760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.708+0000 7f55f67fc640 1 -- 172.21.15.107:0/1800873718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581e7b1f7d0 con 0x5581e7ddcea0 2024-04-25T00:18:42.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.709+0000 7f55f67fc640 1 --2- 172.21.15.107:0/1800873718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7ddcea0 0x5581e7ded220 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f55ec09bdd0 tx=0x7f55ec09b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e074ac1015c9a4ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:42.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.709+0000 7f55d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:18:42.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.709+0000 7f55d6ffd640 1 -- 172.21.15.107:0/1800873718 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ec09e040 con 0x5581e7ddcea0 2024-04-25T00:18:42.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.709+0000 7f560076a740 1 -- 172.21.15.107:0/1800873718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55f0036040 con 0x5581e7ddcea0 2024-04-25T00:18:42.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.709+0000 7f55d6ffd640 1 -- 172.21.15.107:0/1800873718 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f55ec0a4820 con 0x5581e7ddcea0 2024-04-25T00:18:42.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.710+0000 7f55d6ffd640 1 -- 172.21.15.107:0/1800873718 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f55ec0a01f0 con 0x5581e7ddcea0 2024-04-25T00:18:42.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.710+0000 7f560076a740 10 client.16104 fetch_fsmap learned FSMap version 28 2024-04-25T00:18:42.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.710+0000 7f560076a740 1 -- 172.21.15.107:0/1800873718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5581e7e6f160 con 0x5581e7ddcea0 2024-04-25T00:18:42.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.710+0000 7f55d6ffd640 1 -- 172.21.15.107:0/1800873718 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55f0036040 con 0x5581e7ddcea0 2024-04-25T00:18:42.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.711+0000 7f55d6ffd640 1 -- 172.21.15.107:0/1800873718 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f55ec0a1380 con 0x5581e7ddcea0 2024-04-25T00:18:42.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.711+0000 7f560076a740 10 client.16104 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:18:42.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.711+0000 7f560076a740 10 client.16104 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:18:42.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.711+0000 7f560076a740 20 client.16104 populate_metadata read hostname 'smithi107' 2024-04-25T00:18:42.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.711+0000 7f560076a740 1 --2- 172.21.15.107:0/1800873718 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5581e7e6f510 0x5581e7e71900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.711+0000 7f560076a740 4 client.16104 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:18:42.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.711+0000 7f560076a740 1 -- 172.21.15.107:0/1800873718 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581e7d48f00 con 0x5581e7e6f510 2024-04-25T00:18:42.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.711+0000 7f55f6ffd640 1 --2- 172.21.15.107:0/1800873718 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5581e7e6f510 0x5581e7e71900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.712+0000 7f55f6ffd640 1 --2- 172.21.15.107:0/1800873718 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5581e7e6f510 0x5581e7e71900 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cc7279a52d59fae5 server_cookie=c8508cd41f5c8f3e in_seq=0 out_seq=0 2024-04-25T00:18:42.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.712+0000 7f55d6ffd640 10 client.16104 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:18:42.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.713+0000 7f55d6ffd640 1 -- 172.21.15.107:0/1800873718 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581e7d48f00 con 0x5581e7e6f510 2024-04-25T00:18:42.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.713+0000 7f55d6ffd640 10 client.16104 handle_command_reply: tid=0 2024-04-25T00:18:42.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.713+0000 7f560076a740 1 client.16104 shutdown 2024-04-25T00:18:42.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.713+0000 7f560076a740 1 -- 172.21.15.107:0/1800873718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7ddcea0 msgr2=0x5581e7ded220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.713+0000 7f560076a740 1 --2- 172.21.15.107:0/1800873718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7ddcea0 0x5581e7ded220 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f55ec09bdd0 tx=0x7f55ec09b8d0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.714+0000 7f560076a740 1 -- 172.21.15.107:0/1800873718 shutdown_connections 2024-04-25T00:18:42.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.714+0000 7f560076a740 1 --2- 172.21.15.107:0/1800873718 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5581e7e6f510 0x5581e7e71900 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.714+0000 7f560076a740 1 --2- 172.21.15.107:0/1800873718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5581e7de7570 0x5581e7ded760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.714+0000 7f560076a740 1 --2- 172.21.15.107:0/1800873718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5581e7ddcea0 0x5581e7ded220 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.714+0000 7f560076a740 1 --2- 172.21.15.107:0/1800873718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5581e7ddc580 0x5581e7decce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.714+0000 7f560076a740 1 -- 172.21.15.107:0/1800873718 >> 172.21.15.107:0/1800873718 conn(0x5581e7dd88a0 msgr2=0x5581e7dd7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:42.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.714+0000 7f55d6ffd640 0 client.16104 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:18:42.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.714+0000 7f560076a740 1 -- 172.21.15.107:0/1800873718 shutdown_connections 2024-04-25T00:18:42.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.714+0000 7f560076a740 1 -- 172.21.15.107:0/1800873718 wait complete. 2024-04-25T00:18:42.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.715+0000 7f560076a740 20 client.16104 trim_cache size 0 max 16384 2024-04-25T00:18:42.716 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:18:42.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.718+0000 7f55fda5d640 1 -- 172.21.15.107:0/3305050548 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f55e00a2d00 msgr2=0x7f55e00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.718+0000 7f55fda5d640 1 --2- 172.21.15.107:0/3305050548 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f55e00a2d00 0x7f55e00a51c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f55f0009ca0 tx=0x7f55f00099e0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.718+0000 7f55fda5d640 1 -- 172.21.15.107:0/3305050548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55f8102150 msgr2=0x7f55f81985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.718+0000 7f55fda5d640 1 --2- 172.21.15.107:0/3305050548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55f8102150 0x7f55f81985d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00e9f0 tx=0x7f55ec00edf0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.718+0000 7f55fda5d640 1 -- 172.21.15.107:0/3305050548 shutdown_connections 2024-04-25T00:18:42.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.718+0000 7f55fda5d640 1 --2- 172.21.15.107:0/3305050548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55f81077a0 0x7f55f819c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.718+0000 7f55fda5d640 1 --2- 172.21.15.107:0/3305050548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55f8102150 0x7f55f81985d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.718+0000 7f55fda5d640 1 --2- 172.21.15.107:0/3305050548 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f55e00a2d00 0x7f55e00a51c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.718+0000 7f55fda5d640 1 --2- 172.21.15.107:0/3305050548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55f8101790 0x7f55f8198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.718+0000 7f55fda5d640 1 -- 172.21.15.107:0/3305050548 >> 172.21.15.107:0/3305050548 conn(0x7f55f80fd2a0 msgr2=0x7f55f80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:42.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.718+0000 7f55fda5d640 1 -- 172.21.15.107:0/3305050548 shutdown_connections 2024-04-25T00:18:42.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.718+0000 7f55fda5d640 1 -- 172.21.15.107:0/3305050548 wait complete. 2024-04-25T00:18:42.737 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:18:42.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.853+0000 7fa2afc53640 1 -- 172.21.15.107:0/2489854185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a81029c0 msgr2=0x7fa2a8102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.853+0000 7fa2afc53640 1 --2- 172.21.15.107:0/2489854185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a81029c0 0x7fa2a8102da0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4009ca0 tx=0x7fa2a402a9e0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.853+0000 7fa2afc53640 1 -- 172.21.15.107:0/2489854185 shutdown_connections 2024-04-25T00:18:42.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.853+0000 7fa2afc53640 1 --2- 172.21.15.107:0/2489854185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a810bd00 0x7fa2a810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.853+0000 7fa2afc53640 1 --2- 172.21.15.107:0/2489854185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2a81032e0 0x7fa2a810b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.853+0000 7fa2afc53640 1 --2- 172.21.15.107:0/2489854185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a81029c0 0x7fa2a8102da0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.853+0000 7fa2afc53640 1 -- 172.21.15.107:0/2489854185 >> 172.21.15.107:0/2489854185 conn(0x7fa2a8077740 msgr2=0x7fa2a8077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:42.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.853+0000 7fa2afc53640 1 -- 172.21.15.107:0/2489854185 shutdown_connections 2024-04-25T00:18:42.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.853+0000 7fa2afc53640 1 -- 172.21.15.107:0/2489854185 wait complete. 2024-04-25T00:18:42.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.853+0000 7fa2afc53640 1 Processor -- start 2024-04-25T00:18:42.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.853+0000 7fa2afc53640 1 -- start start 2024-04-25T00:18:42.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.854+0000 7fa2afc53640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a81029c0 0x7fa2a8197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.854+0000 7fa2afc53640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2a81032e0 0x7fa2a8198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.854+0000 7fa2afc53640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a810bd00 0x7fa2a819c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.854+0000 7fa2afc53640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8111b00 con 0x7fa2a81029c0 2024-04-25T00:18:42.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.854+0000 7fa2afc53640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8111980 con 0x7fa2a810bd00 2024-04-25T00:18:42.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.854+0000 7fa2afc53640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a8111c80 con 0x7fa2a81032e0 2024-04-25T00:18:42.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.854+0000 7fa2ad9ef640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a81029c0 0x7fa2a8197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.854+0000 7fa2ad9ef640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a81029c0 0x7fa2a8197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41774/0 (socket says 172.21.15.107:41774) 2024-04-25T00:18:42.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.854+0000 7fa2ad9ef640 1 -- 172.21.15.107:0/208525736 learned_addr learned my addr 172.21.15.107:0/208525736 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:42.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.854+0000 7fa2ad9ef640 1 -- 172.21.15.107:0/208525736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2a81032e0 msgr2=0x7fa2a8198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.854+0000 7fa2ad9ef640 1 --2- 172.21.15.107:0/208525736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2a81032e0 0x7fa2a8198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.854+0000 7fa2ad9ef640 1 -- 172.21.15.107:0/208525736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a810bd00 msgr2=0x7fa2a819c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.854+0000 7fa2ad9ef640 1 --2- 172.21.15.107:0/208525736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a810bd00 0x7fa2a819c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.854+0000 7fa2ad9ef640 1 -- 172.21.15.107:0/208525736 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a819cfd0 con 0x7fa2a81029c0 2024-04-25T00:18:42.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.854+0000 7fa2ad9ef640 1 --2- 172.21.15.107:0/208525736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a81029c0 0x7fa2a8197fe0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4004af0 tx=0x7fa2a4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:42.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.854+0000 7fa29effd640 1 -- 172.21.15.107:0/208525736 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4047070 con 0x7fa2a81029c0 2024-04-25T00:18:42.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.854+0000 7fa2afc53640 1 -- 172.21.15.107:0/208525736 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a819d260 con 0x7fa2a81029c0 2024-04-25T00:18:42.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.854+0000 7fa29effd640 1 -- 172.21.15.107:0/208525736 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a402fb20 con 0x7fa2a81029c0 2024-04-25T00:18:42.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.854+0000 7fa29effd640 1 -- 172.21.15.107:0/208525736 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4033050 con 0x7fa2a81029c0 2024-04-25T00:18:42.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.854+0000 7fa2afc53640 1 -- 172.21.15.107:0/208525736 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a81a4b50 con 0x7fa2a81029c0 2024-04-25T00:18:42.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.856+0000 7fa29effd640 1 -- 172.21.15.107:0/208525736 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa2a404a070 con 0x7fa2a81029c0 2024-04-25T00:18:42.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.857+0000 7fa29effd640 1 --2- 172.21.15.107:0/208525736 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa2940a2d00 0x7fa2940a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.857+0000 7fa2b2a33740 1 Processor -- start 2024-04-25T00:18:42.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.857+0000 7fa2ad1ee640 1 --2- 172.21.15.107:0/208525736 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa2940a2d00 0x7fa2940a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.857+0000 7fa29effd640 1 -- 172.21.15.107:0/208525736 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fa2a40f0400 con 0x7fa2a81029c0 2024-04-25T00:18:42.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.857+0000 7fa2b2a33740 1 -- start start 2024-04-25T00:18:42.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.858+0000 7fa2b2a33740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560496d6a580 0x560496d6a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.858+0000 7fa2b2a33740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d6aea0 0x560496d75030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.858+0000 7fa2b2a33740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560496d75570 0x560496d77960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.858+0000 7fa2b2a33740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560496ccf670 con 0x560496d6aea0 2024-04-25T00:18:42.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.858+0000 7fa2b2a33740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560496c77970 con 0x560496d75570 2024-04-25T00:18:42.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.858+0000 7fa2b2a33740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560496ccb630 con 0x560496d6a580 2024-04-25T00:18:42.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.858+0000 7fa2ad1ee640 1 --2- 172.21.15.107:0/208525736 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa2940a2d00 0x7fa2940a51c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa2980058c0 tx=0x7fa298005850 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:42.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.858+0000 7fa2ad1ee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d6aea0 0x560496d75030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.858+0000 7fa2ad9ef640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560496d6a580 0x560496d6a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.858+0000 7fa2ad1ee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d6aea0 0x560496d75030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41782/0 (socket says 172.21.15.107:41782) 2024-04-25T00:18:42.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.858+0000 7fa2ad1ee640 1 -- 172.21.15.107:0/3601141848 learned_addr learned my addr 172.21.15.107:0/3601141848 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:42.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.858+0000 7fa2ae1f0640 1 --2- 172.21.15.107:0/3601141848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560496d75570 0x560496d77960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.858+0000 7fa2ad1ee640 1 -- 172.21.15.107:0/3601141848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560496d6a580 msgr2=0x560496d6a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:42.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.858+0000 7fa2ad1ee640 1 --2- 172.21.15.107:0/3601141848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560496d6a580 0x560496d6a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.858+0000 7fa2ad1ee640 1 -- 172.21.15.107:0/3601141848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560496d75570 msgr2=0x560496d77960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:42.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.858+0000 7fa2ad1ee640 1 --2- 172.21.15.107:0/3601141848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560496d75570 0x560496d77960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.858+0000 7fa2ad1ee640 1 -- 172.21.15.107:0/3601141848 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560496c9b400 con 0x560496d6aea0 2024-04-25T00:18:42.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.858+0000 7fa2ad9ef640 1 --2- 172.21.15.107:0/3601141848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560496d6a580 0x560496d6a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:42.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.858+0000 7fa2ad1ee640 1 --2- 172.21.15.107:0/3601141848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d6aea0 0x560496d75030 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa29800ed50 tx=0x7fa298009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3c84842b4fe482 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:42.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.858+0000 7fa293fff640 1 -- 172.21.15.107:0/3601141848 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298020070 con 0x560496d6aea0 2024-04-25T00:18:42.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.858+0000 7fa293fff640 1 -- 172.21.15.107:0/3601141848 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa298025940 con 0x560496d6aea0 2024-04-25T00:18:42.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.858+0000 7fa293fff640 1 -- 172.21.15.107:0/3601141848 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29801cb00 con 0x560496d6aea0 2024-04-25T00:18:42.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.859+0000 7fa2b2a33740 1 -- 172.21.15.107:0/3601141848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d6aea0 msgr2=0x560496d75030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.859+0000 7fa2b2a33740 1 --2- 172.21.15.107:0/3601141848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d6aea0 0x560496d75030 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa29800ed50 tx=0x7fa298009f90 comp rx=0 tx=0).stop 2024-04-25T00:18:42.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.859+0000 7fa2b2a33740 1 -- 172.21.15.107:0/3601141848 shutdown_connections 2024-04-25T00:18:42.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.859+0000 7fa2b2a33740 1 --2- 172.21.15.107:0/3601141848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560496d75570 0x560496d77960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.859+0000 7fa2b2a33740 1 --2- 172.21.15.107:0/3601141848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d6aea0 0x560496d75030 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.859+0000 7fa2b2a33740 1 --2- 172.21.15.107:0/3601141848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560496d6a580 0x560496d6a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.859+0000 7fa2b2a33740 1 -- 172.21.15.107:0/3601141848 >> 172.21.15.107:0/3601141848 conn(0x560496d668a0 msgr2=0x560496d66c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:42.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.859+0000 7fa2b2a33740 1 -- 172.21.15.107:0/3601141848 shutdown_connections 2024-04-25T00:18:42.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.859+0000 7fa2b2a33740 1 -- 172.21.15.107:0/3601141848 wait complete. 2024-04-25T00:18:42.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.860+0000 7fa2b2a33740 1 -- start start 2024-04-25T00:18:42.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.860+0000 7fa2b2a33740 1 Processor -- start 2024-04-25T00:18:42.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.860+0000 7fa2b2a33740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d6a580 0x560496d74f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.860+0000 7fa2b2a33740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560496d6aea0 0x560496d72210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.860+0000 7fa2b2a33740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560496d75570 0x560496d727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.860+0000 7fa2b2a33740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560496d78360 con 0x560496d6a580 2024-04-25T00:18:42.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.860+0000 7fa2b2a33740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560496c8b050 con 0x560496d6aea0 2024-04-25T00:18:42.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.860+0000 7fa2b2a33740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560496d784e0 con 0x560496d75570 2024-04-25T00:18:42.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.860+0000 7fa2b2a33740 10 client.0 _ll_register_callbacks cb 0x5604969e2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:42.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.860+0000 7fa2ad9ef640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d6a580 0x560496d74f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.860+0000 7fa2ad9ef640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d6a580 0x560496d74f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41784/0 (socket says 172.21.15.107:41784) 2024-04-25T00:18:42.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.860+0000 7fa2ad9ef640 1 -- 172.21.15.107:0/2207343032 learned_addr learned my addr 172.21.15.107:0/2207343032 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:42.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.860+0000 7fa2ad1ee640 1 --2- 172.21.15.107:0/2207343032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560496d6aea0 0x560496d72210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.860+0000 7fa2ae1f0640 1 --2- 172.21.15.107:0/2207343032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560496d75570 0x560496d727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.860+0000 7fa2ad9ef640 1 -- 172.21.15.107:0/2207343032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560496d75570 msgr2=0x560496d727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:42.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.860+0000 7fa2ad9ef640 1 --2- 172.21.15.107:0/2207343032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560496d75570 0x560496d727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.860+0000 7fa2ad9ef640 1 -- 172.21.15.107:0/2207343032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560496d6aea0 msgr2=0x560496d72210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:42.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.860+0000 7fa2ad9ef640 1 --2- 172.21.15.107:0/2207343032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560496d6aea0 0x560496d72210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.860+0000 7fa2ad9ef640 1 -- 172.21.15.107:0/2207343032 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560496c5cdc0 con 0x560496d6a580 2024-04-25T00:18:42.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.860+0000 7fa2ad9ef640 1 --2- 172.21.15.107:0/2207343032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d6a580 0x560496d74f20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa2a40ed840 tx=0x7fa2a40bc200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f455389c3ca8817 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:42.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.860+0000 7fa2917fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:18:42.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.861+0000 7fa2917fa640 1 -- 172.21.15.107:0/2207343032 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a40c5220 con 0x560496d6a580 2024-04-25T00:18:42.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.861+0000 7fa2917fa640 1 -- 172.21.15.107:0/2207343032 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fa2a402e390 con 0x560496d6a580 2024-04-25T00:18:42.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.861+0000 7fa2b2a33740 1 -- 172.21.15.107:0/2207343032 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa298025940 con 0x560496d6a580 2024-04-25T00:18:42.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.861+0000 7fa2917fa640 1 -- 172.21.15.107:0/2207343032 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa2a40f00b0 con 0x560496d6a580 2024-04-25T00:18:42.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.861+0000 7fa2b2a33740 10 client.16116 fetch_fsmap learned FSMap version 28 2024-04-25T00:18:42.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.861+0000 7fa2b2a33740 1 -- 172.21.15.107:0/2207343032 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5604969e33f0 con 0x560496d6a580 2024-04-25T00:18:42.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.861+0000 7fa2917fa640 1 -- 172.21.15.107:0/2207343032 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa298025940 con 0x560496d6a580 2024-04-25T00:18:42.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.861+0000 7fa2917fa640 1 -- 172.21.15.107:0/2207343032 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fa2a40bbb20 con 0x560496d6a580 2024-04-25T00:18:42.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.861+0000 7fa2b2a33740 10 client.16116 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:18:42.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.861+0000 7fa2b2a33740 10 client.16116 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:18:42.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.861+0000 7fa2b2a33740 20 client.16116 populate_metadata read hostname 'smithi107' 2024-04-25T00:18:42.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.861+0000 7fa2b2a33740 1 --2- 172.21.15.107:0/2207343032 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x560496d85590 0x560496d879a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.861+0000 7fa2b2a33740 4 client.16116 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:18:42.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.861+0000 7fa2b2a33740 1 -- 172.21.15.107:0/2207343032 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560496cdcbb0 con 0x560496d85590 2024-04-25T00:18:42.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.861+0000 7fa2ad1ee640 1 --2- 172.21.15.107:0/2207343032 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x560496d85590 0x560496d879a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.862+0000 7fa2917fa640 10 client.16116 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:18:42.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.862+0000 7fa2ad1ee640 1 --2- 172.21.15.107:0/2207343032 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x560496d85590 0x560496d879a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ab871579b522f18 server_cookie=865eea92e97b5cde in_seq=0 out_seq=0 2024-04-25T00:18:42.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.862+0000 7fa2917fa640 1 -- 172.21.15.107:0/2207343032 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560496cdcbb0 con 0x560496d85590 2024-04-25T00:18:42.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.862+0000 7fa2917fa640 10 client.16116 handle_command_reply: tid=0 2024-04-25T00:18:42.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.862+0000 7fa2b2a33740 1 client.16116 shutdown 2024-04-25T00:18:42.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.863+0000 7fa2b2a33740 1 -- 172.21.15.107:0/2207343032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d6a580 msgr2=0x560496d74f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.863+0000 7fa2b2a33740 1 --2- 172.21.15.107:0/2207343032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d6a580 0x560496d74f20 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa2a40ed840 tx=0x7fa2a40bc200 comp rx=0 tx=0).stop 2024-04-25T00:18:42.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.863+0000 7fa2b2a33740 1 -- 172.21.15.107:0/2207343032 shutdown_connections 2024-04-25T00:18:42.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.863+0000 7fa2b2a33740 1 --2- 172.21.15.107:0/2207343032 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560496d75570 0x560496d727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.863+0000 7fa2b2a33740 1 --2- 172.21.15.107:0/2207343032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560496d6aea0 0x560496d72210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.863+0000 7fa2b2a33740 1 --2- 172.21.15.107:0/2207343032 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x560496d85590 0x560496d879a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.863+0000 7fa2b2a33740 1 --2- 172.21.15.107:0/2207343032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d6a580 0x560496d74f20 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.863+0000 7fa2917fa640 0 client.16116 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:18:42.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.863+0000 7fa2b2a33740 1 -- 172.21.15.107:0/2207343032 >> 172.21.15.107:0/2207343032 conn(0x560496d668a0 msgr2=0x560496d65b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:42.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.863+0000 7fa2b2a33740 1 -- 172.21.15.107:0/2207343032 shutdown_connections 2024-04-25T00:18:42.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.863+0000 7fa2b2a33740 1 -- 172.21.15.107:0/2207343032 wait complete. 2024-04-25T00:18:42.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.863+0000 7fa2b2a33740 20 client.16116 trim_cache size 0 max 16384 2024-04-25T00:18:42.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.877+0000 7fa2b2a33740 1 Processor -- start 2024-04-25T00:18:42.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.877+0000 7fa2b2a33740 1 -- start start 2024-04-25T00:18:42.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.877+0000 7fa2b2a33740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560496d6a580 0x560496d74a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.877+0000 7fa2b2a33740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d6aea0 0x560496d8d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.877+0000 7fa2b2a33740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560496d75570 0x560496d8d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.877+0000 7fa2b2a33740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81b1440 con 0x560496d6aea0 2024-04-25T00:18:42.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.877+0000 7fa2b2a33740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81aec50 con 0x560496d6a580 2024-04-25T00:18:42.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.877+0000 7fa2b2a33740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81acaa0 con 0x560496d75570 2024-04-25T00:18:42.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.877+0000 7fa2ad1ee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d6aea0 0x560496d8d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.877+0000 7fa2ad1ee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d6aea0 0x560496d8d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41790/0 (socket says 172.21.15.107:41790) 2024-04-25T00:18:42.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.877+0000 7fa2ad1ee640 1 -- 172.21.15.107:0/1240521848 learned_addr learned my addr 172.21.15.107:0/1240521848 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:42.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.877+0000 7fa2ad9ef640 1 --2- 172.21.15.107:0/1240521848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560496d6a580 0x560496d74a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.877+0000 7fa2ae1f0640 1 --2- 172.21.15.107:0/1240521848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560496d75570 0x560496d8d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.877+0000 7fa2ad1ee640 1 -- 172.21.15.107:0/1240521848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560496d75570 msgr2=0x560496d8d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:42.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.877+0000 7fa2ad1ee640 1 --2- 172.21.15.107:0/1240521848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560496d75570 0x560496d8d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.877+0000 7fa2ad1ee640 1 -- 172.21.15.107:0/1240521848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560496d6a580 msgr2=0x560496d74a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:42.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.877+0000 7fa2ad1ee640 1 --2- 172.21.15.107:0/1240521848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560496d6a580 0x560496d74a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.877+0000 7fa2ad1ee640 1 -- 172.21.15.107:0/1240521848 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560496aad7d0 con 0x560496d6aea0 2024-04-25T00:18:42.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.877+0000 7fa2ad1ee640 1 --2- 172.21.15.107:0/1240521848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d6aea0 0x560496d8d270 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa29801c630 tx=0x7fa29801c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de14e27de1f366b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:42.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.878+0000 7fa2937fe640 1 -- 172.21.15.107:0/1240521848 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298014040 con 0x560496d6aea0 2024-04-25T00:18:42.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.878+0000 7fa2937fe640 1 -- 172.21.15.107:0/1240521848 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa29800ed80 con 0x560496d6aea0 2024-04-25T00:18:42.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.878+0000 7fa2937fe640 1 -- 172.21.15.107:0/1240521848 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298020040 con 0x560496d6aea0 2024-04-25T00:18:42.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.878+0000 7fa2b2a33740 1 -- 172.21.15.107:0/1240521848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d6aea0 msgr2=0x560496d8d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.878+0000 7fa2b2a33740 1 --2- 172.21.15.107:0/1240521848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d6aea0 0x560496d8d270 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fa29801c630 tx=0x7fa29801c660 comp rx=0 tx=0).stop 2024-04-25T00:18:42.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.878+0000 7fa2b2a33740 1 -- 172.21.15.107:0/1240521848 shutdown_connections 2024-04-25T00:18:42.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.878+0000 7fa2b2a33740 1 --2- 172.21.15.107:0/1240521848 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560496d75570 0x560496d8d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.878+0000 7fa2b2a33740 1 --2- 172.21.15.107:0/1240521848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d6aea0 0x560496d8d270 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.878+0000 7fa2b2a33740 1 --2- 172.21.15.107:0/1240521848 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560496d6a580 0x560496d74a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.970 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.878+0000 7fa2b2a33740 1 -- 172.21.15.107:0/1240521848 >> 172.21.15.107:0/1240521848 conn(0x560496d668a0 msgr2=0x560496d6c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:42.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.878+0000 7fa2b2a33740 1 -- 172.21.15.107:0/1240521848 shutdown_connections 2024-04-25T00:18:42.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.878+0000 7fa2b2a33740 1 -- 172.21.15.107:0/1240521848 wait complete. 2024-04-25T00:18:42.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.879+0000 7fa2b2a33740 1 -- start start 2024-04-25T00:18:42.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.879+0000 7fa2b2a33740 1 Processor -- start 2024-04-25T00:18:42.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.879+0000 7fa2b2a33740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560496d6a580 0x560496d6f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.879+0000 7fa2b2a33740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560496d6aea0 0x560496d6fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.879+0000 7fa2b2a33740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d75570 0x560496deaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.879+0000 7fa2b2a33740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81ad110 con 0x560496d75570 2024-04-25T00:18:42.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.879+0000 7fa2b2a33740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81ae580 con 0x560496d6aea0 2024-04-25T00:18:42.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.879+0000 7fa2b2a33740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a81ad7e0 con 0x560496d6a580 2024-04-25T00:18:42.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.879+0000 7fa2b2a33740 10 client.0 _ll_register_callbacks cb 0x7fa2a40bbfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:42.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.879+0000 7fa2ae1f0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d75570 0x560496deaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.879+0000 7fa2ad9ef640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560496d6a580 0x560496d6f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.879+0000 7fa2ae1f0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d75570 0x560496deaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41794/0 (socket says 172.21.15.107:41794) 2024-04-25T00:18:42.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.879+0000 7fa2ad1ee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560496d6aea0 0x560496d6fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.879+0000 7fa2ae1f0640 1 -- 172.21.15.107:0/1771993669 learned_addr learned my addr 172.21.15.107:0/1771993669 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:42.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.879+0000 7fa2ae1f0640 1 -- 172.21.15.107:0/1771993669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560496d6a580 msgr2=0x560496d6f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:42.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.880+0000 7fa2ae1f0640 1 --2- 172.21.15.107:0/1771993669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560496d6a580 0x560496d6f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.880+0000 7fa2ae1f0640 1 -- 172.21.15.107:0/1771993669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560496d6aea0 msgr2=0x560496d6fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:42.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.880+0000 7fa2ae1f0640 1 --2- 172.21.15.107:0/1771993669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560496d6aea0 0x560496d6fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.880+0000 7fa2ae1f0640 1 -- 172.21.15.107:0/1771993669 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560496deb020 con 0x560496d75570 2024-04-25T00:18:42.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.880+0000 7fa2ae1f0640 1 --2- 172.21.15.107:0/1771993669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d75570 0x560496deaae0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0010770 tx=0x7fa2a0010c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1ab949968acce6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:42.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.880+0000 7fa2917fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:18:42.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.880+0000 7fa2917fa640 1 -- 172.21.15.107:0/1771993669 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0021060 con 0x560496d75570 2024-04-25T00:18:42.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.880+0000 7fa2917fa640 1 -- 172.21.15.107:0/1771993669 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fa2a0004460 con 0x560496d75570 2024-04-25T00:18:42.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.880+0000 7fa2b2a33740 1 -- 172.21.15.107:0/1771993669 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa29800ed80 con 0x560496d75570 2024-04-25T00:18:42.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.880+0000 7fa2917fa640 1 -- 172.21.15.107:0/1771993669 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa2a0028050 con 0x560496d75570 2024-04-25T00:18:42.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.880+0000 7fa2b2a33740 10 client.16122 fetch_fsmap learned FSMap version 28 2024-04-25T00:18:42.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.880+0000 7fa2b2a33740 1 -- 172.21.15.107:0/1771993669 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560496dec350 con 0x560496d75570 2024-04-25T00:18:42.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.880+0000 7fa2917fa640 1 -- 172.21.15.107:0/1771993669 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa29800ed80 con 0x560496d75570 2024-04-25T00:18:42.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.880+0000 7fa2917fa640 1 -- 172.21.15.107:0/1771993669 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fa2a001b030 con 0x560496d75570 2024-04-25T00:18:42.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.880+0000 7fa2b2a33740 10 client.16122 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:18:42.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.880+0000 7fa2b2a33740 10 client.16122 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:18:42.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.880+0000 7fa2b2a33740 20 client.16122 populate_metadata read hostname 'smithi107' 2024-04-25T00:18:42.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.880+0000 7fa2b2a33740 1 --2- 172.21.15.107:0/1771993669 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x560496decb60 0x560496deef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:42.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.880+0000 7fa2b2a33740 4 client.16122 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:18:42.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.880+0000 7fa2b2a33740 1 -- 172.21.15.107:0/1771993669 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560496cd6f00 con 0x560496decb60 2024-04-25T00:18:42.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.881+0000 7fa2ad9ef640 1 --2- 172.21.15.107:0/1771993669 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x560496decb60 0x560496deef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:42.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.881+0000 7fa2917fa640 10 client.16122 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:18:42.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.881+0000 7fa2ad9ef640 1 --2- 172.21.15.107:0/1771993669 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x560496decb60 0x560496deef80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=55c775ae5ce1ae87 server_cookie=732f78a796785550 in_seq=0 out_seq=0 2024-04-25T00:18:42.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.881+0000 7fa2917fa640 1 -- 172.21.15.107:0/1771993669 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560496cd6f00 con 0x560496decb60 2024-04-25T00:18:42.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.881+0000 7fa2917fa640 10 client.16122 handle_command_reply: tid=0 2024-04-25T00:18:42.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.881+0000 7fa2b2a33740 1 client.16122 shutdown 2024-04-25T00:18:42.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.881+0000 7fa2b2a33740 1 -- 172.21.15.107:0/1771993669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d75570 msgr2=0x560496deaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.881+0000 7fa2b2a33740 1 --2- 172.21.15.107:0/1771993669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d75570 0x560496deaae0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0010770 tx=0x7fa2a0010c30 comp rx=0 tx=0).stop 2024-04-25T00:18:42.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.881+0000 7fa2b2a33740 1 -- 172.21.15.107:0/1771993669 shutdown_connections 2024-04-25T00:18:42.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.881+0000 7fa2b2a33740 1 --2- 172.21.15.107:0/1771993669 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x560496decb60 0x560496deef80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.881+0000 7fa2b2a33740 1 --2- 172.21.15.107:0/1771993669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560496d75570 0x560496deaae0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.881+0000 7fa2b2a33740 1 --2- 172.21.15.107:0/1771993669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560496d6aea0 0x560496d6fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.881+0000 7fa2917fa640 0 client.16122 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:18:42.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.881+0000 7fa2b2a33740 1 --2- 172.21.15.107:0/1771993669 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560496d6a580 0x560496d6f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.881+0000 7fa2b2a33740 1 -- 172.21.15.107:0/1771993669 >> 172.21.15.107:0/1771993669 conn(0x560496d668a0 msgr2=0x560496d652b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:42.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.881+0000 7fa2b2a33740 1 -- 172.21.15.107:0/1771993669 shutdown_connections 2024-04-25T00:18:42.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.881+0000 7fa2b2a33740 1 -- 172.21.15.107:0/1771993669 wait complete. 2024-04-25T00:18:42.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.881+0000 7fa2b2a33740 20 client.16122 trim_cache size 0 max 16384 2024-04-25T00:18:42.978 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:18:42.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.882+0000 7fa2afc53640 1 -- 172.21.15.107:0/208525736 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa2940a2d00 msgr2=0x7fa2940a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.882+0000 7fa2afc53640 1 --2- 172.21.15.107:0/208525736 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa2940a2d00 0x7fa2940a51c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa2980058c0 tx=0x7fa298005850 comp rx=0 tx=0).stop 2024-04-25T00:18:42.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.882+0000 7fa2afc53640 1 -- 172.21.15.107:0/208525736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a81029c0 msgr2=0x7fa2a8197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:42.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.882+0000 7fa2afc53640 1 --2- 172.21.15.107:0/208525736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a81029c0 0x7fa2a8197fe0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4004af0 tx=0x7fa2a4003ba0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.882+0000 7fa2afc53640 1 -- 172.21.15.107:0/208525736 shutdown_connections 2024-04-25T00:18:42.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.882+0000 7fa2afc53640 1 --2- 172.21.15.107:0/208525736 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa2940a2d00 0x7fa2940a51c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.882+0000 7fa2afc53640 1 --2- 172.21.15.107:0/208525736 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a810bd00 0x7fa2a819c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.882+0000 7fa2afc53640 1 --2- 172.21.15.107:0/208525736 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa2a81032e0 0x7fa2a8198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.882+0000 7fa2afc53640 1 --2- 172.21.15.107:0/208525736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a81029c0 0x7fa2a8197fe0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:42.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.882+0000 7fa2afc53640 1 -- 172.21.15.107:0/208525736 >> 172.21.15.107:0/208525736 conn(0x7fa2a8077740 msgr2=0x7fa2a80fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:42.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.882+0000 7fa2afc53640 1 -- 172.21.15.107:0/208525736 shutdown_connections 2024-04-25T00:18:42.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:42.882+0000 7fa2afc53640 1 -- 172.21.15.107:0/208525736 wait complete. 2024-04-25T00:18:42.980 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:18:43.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.085+0000 7f4ac1676640 1 -- 172.21.15.107:0/4287251814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4abc100fd0 msgr2=0x7f4abc10d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:43.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.085+0000 7f4ac1676640 1 --2- 172.21.15.107:0/4287251814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4abc100fd0 0x7f4abc10d4e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4009930 tx=0x7f4ab402a420 comp rx=0 tx=0).stop 2024-04-25T00:18:43.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.085+0000 7f4ac1676640 1 -- 172.21.15.107:0/4287251814 shutdown_connections 2024-04-25T00:18:43.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.085+0000 7f4ac1676640 1 --2- 172.21.15.107:0/4287251814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4abc100fd0 0x7f4abc10d4e0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.085+0000 7f4ac1676640 1 --2- 172.21.15.107:0/4287251814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4abc100610 0x7f4abc100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.085+0000 7f4ac1676640 1 --2- 172.21.15.107:0/4287251814 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4abc1066b0 0x7f4abc106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.085+0000 7f4ac1676640 1 -- 172.21.15.107:0/4287251814 >> 172.21.15.107:0/4287251814 conn(0x7f4abc077740 msgr2=0x7f4abc077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:43.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.085+0000 7f4ac1676640 1 -- 172.21.15.107:0/4287251814 shutdown_connections 2024-04-25T00:18:43.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.085+0000 7f4ac1676640 1 -- 172.21.15.107:0/4287251814 wait complete. 2024-04-25T00:18:43.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.086+0000 7f4ac1676640 1 Processor -- start 2024-04-25T00:18:43.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.086+0000 7f4ac1676640 1 -- start start 2024-04-25T00:18:43.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.086+0000 7f4ac1676640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4abc100610 0x7f4abc197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.086+0000 7f4ac1676640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4abc100fd0 0x7f4abc198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.086+0000 7f4ac1676640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4abc1066b0 0x7f4abc19c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.086+0000 7f4ac1676640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc111b30 con 0x7f4abc100fd0 2024-04-25T00:18:43.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.086+0000 7f4ac1676640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc1119b0 con 0x7f4abc1066b0 2024-04-25T00:18:43.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.086+0000 7f4ac1676640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc111cb0 con 0x7f4abc100610 2024-04-25T00:18:43.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.086+0000 7f4aba7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4abc100fd0 0x7f4abc198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.086+0000 7f4aba7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4abc100fd0 0x7f4abc198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41802/0 (socket says 172.21.15.107:41802) 2024-04-25T00:18:43.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.086+0000 7f4aba7fc640 1 -- 172.21.15.107:0/389654047 learned_addr learned my addr 172.21.15.107:0/389654047 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:43.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.087+0000 7f4abaffd640 1 --2- 172.21.15.107:0/389654047 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4abc100610 0x7f4abc197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.087+0000 7f4abb7fe640 1 --2- 172.21.15.107:0/389654047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4abc1066b0 0x7f4abc19c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.087+0000 7f4aba7fc640 1 -- 172.21.15.107:0/389654047 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4abc100610 msgr2=0x7f4abc197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:43.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.087+0000 7f4aba7fc640 1 --2- 172.21.15.107:0/389654047 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4abc100610 0x7f4abc197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.087+0000 7f4aba7fc640 1 -- 172.21.15.107:0/389654047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4abc1066b0 msgr2=0x7f4abc19c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:43.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.087+0000 7f4aba7fc640 1 --2- 172.21.15.107:0/389654047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4abc1066b0 0x7f4abc19c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.087+0000 7f4aba7fc640 1 -- 172.21.15.107:0/389654047 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4abc19cee0 con 0x7f4abc100fd0 2024-04-25T00:18:43.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.087+0000 7f4abaffd640 1 --2- 172.21.15.107:0/389654047 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4abc100610 0x7f4abc197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:43.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.087+0000 7f4aba7fc640 1 --2- 172.21.15.107:0/389654047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4abc100fd0 0x7f4abc198430 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000e9f0 tx=0x7f4ab000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:43.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.087+0000 7f4aa3fff640 1 -- 172.21.15.107:0/389654047 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab000cd80 con 0x7f4abc100fd0 2024-04-25T00:18:43.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.087+0000 7f4ac1676640 1 -- 172.21.15.107:0/389654047 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4abc19d1d0 con 0x7f4abc100fd0 2024-04-25T00:18:43.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.087+0000 7f4aa3fff640 1 -- 172.21.15.107:0/389654047 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ab0010910 con 0x7f4abc100fd0 2024-04-25T00:18:43.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.087+0000 7f4aa3fff640 1 -- 172.21.15.107:0/389654047 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab00188f0 con 0x7f4abc100fd0 2024-04-25T00:18:43.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.087+0000 7f4ac1676640 1 -- 172.21.15.107:0/389654047 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4abc1a4a60 con 0x7f4abc100fd0 2024-04-25T00:18:43.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.089+0000 7f4aa3fff640 1 -- 172.21.15.107:0/389654047 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4ab0010ac0 con 0x7f4abc100fd0 2024-04-25T00:18:43.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.089+0000 7f4ac4386740 1 Processor -- start 2024-04-25T00:18:43.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.090+0000 7f4ac4386740 1 -- start start 2024-04-25T00:18:43.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.090+0000 7f4ac4386740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56239e453460 0x56239e453840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.090+0000 7f4ac4386740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e453d80 0x56239e462440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.090+0000 7f4aa3fff640 1 --2- 172.21.15.107:0/389654047 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4aa40a2d00 0x7f4aa40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.090+0000 7f4ac4386740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56239e462980 0x56239e464d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.090+0000 7f4ac4386740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56239e3b8670 con 0x56239e453d80 2024-04-25T00:18:43.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.090+0000 7f4ac4386740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56239e360970 con 0x56239e453460 2024-04-25T00:18:43.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.090+0000 7f4ac4386740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56239e3b4630 con 0x56239e462980 2024-04-25T00:18:43.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.090+0000 7f4aba7fc640 1 --2- 172.21.15.107:0/389654047 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4aa40a2d00 0x7f4aa40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.090+0000 7f4abaffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e453d80 0x56239e462440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.090+0000 7f4abaffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e453d80 0x56239e462440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41816/0 (socket says 172.21.15.107:41816) 2024-04-25T00:18:43.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.090+0000 7f4abaffd640 1 -- 172.21.15.107:0/1180156629 learned_addr learned my addr 172.21.15.107:0/1180156629 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:43.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.090+0000 7f4abb7fe640 1 --2- 172.21.15.107:0/1180156629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56239e453460 0x56239e453840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.090+0000 7f4abb7fe640 1 --2- 172.21.15.107:0/1180156629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56239e462980 0x56239e464d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.090+0000 7f4aa3fff640 1 -- 172.21.15.107:0/389654047 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f4ab00cfb80 con 0x7f4abc100fd0 2024-04-25T00:18:43.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.090+0000 7f4abaffd640 1 -- 172.21.15.107:0/1180156629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56239e462980 msgr2=0x56239e464d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:43.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.090+0000 7f4abaffd640 1 --2- 172.21.15.107:0/1180156629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56239e462980 0x56239e464d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.090+0000 7f4abaffd640 1 -- 172.21.15.107:0/1180156629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56239e453460 msgr2=0x56239e453840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:43.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.090+0000 7f4abaffd640 1 --2- 172.21.15.107:0/1180156629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56239e453460 0x56239e453840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.090+0000 7f4abaffd640 1 -- 172.21.15.107:0/1180156629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56239e384400 con 0x56239e453d80 2024-04-25T00:18:43.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.090+0000 7f4abb7fe640 1 --2- 172.21.15.107:0/1180156629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56239e462980 0x56239e464d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:18:43.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.090+0000 7f4abaffd640 1 --2- 172.21.15.107:0/1180156629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e453d80 0x56239e462440 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4aac011820 tx=0x7f4aac011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5f743c7813f249 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:43.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.090+0000 7f4aba7fc640 1 --2- 172.21.15.107:0/389654047 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4aa40a2d00 0x7f4aa40a51c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00987b0 tx=0x7f4ab0097a40 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:43.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.090+0000 7f4aa17fa640 1 -- 172.21.15.107:0/1180156629 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac011220 con 0x56239e453d80 2024-04-25T00:18:43.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.091+0000 7f4aa17fa640 1 -- 172.21.15.107:0/1180156629 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aac019430 con 0x56239e453d80 2024-04-25T00:18:43.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.091+0000 7f4aa17fa640 1 -- 172.21.15.107:0/1180156629 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac010670 con 0x56239e453d80 2024-04-25T00:18:43.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.091+0000 7f4ac4386740 1 -- 172.21.15.107:0/1180156629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e453d80 msgr2=0x56239e462440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:43.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.091+0000 7f4ac4386740 1 --2- 172.21.15.107:0/1180156629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e453d80 0x56239e462440 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4aac011820 tx=0x7f4aac011ce0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.091+0000 7f4ac4386740 1 -- 172.21.15.107:0/1180156629 shutdown_connections 2024-04-25T00:18:43.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.091+0000 7f4ac4386740 1 --2- 172.21.15.107:0/1180156629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56239e462980 0x56239e464d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.091+0000 7f4ac4386740 1 --2- 172.21.15.107:0/1180156629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e453d80 0x56239e462440 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.091+0000 7f4ac4386740 1 --2- 172.21.15.107:0/1180156629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56239e453460 0x56239e453840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.091+0000 7f4ac4386740 1 -- 172.21.15.107:0/1180156629 >> 172.21.15.107:0/1180156629 conn(0x56239e44f8a0 msgr2=0x56239e44fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:43.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.091+0000 7f4ac4386740 1 -- 172.21.15.107:0/1180156629 shutdown_connections 2024-04-25T00:18:43.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.091+0000 7f4ac4386740 1 -- 172.21.15.107:0/1180156629 wait complete. 2024-04-25T00:18:43.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.092+0000 7f4ac4386740 1 -- start start 2024-04-25T00:18:43.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.092+0000 7f4ac4386740 1 Processor -- start 2024-04-25T00:18:43.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.092+0000 7f4ac4386740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56239e453460 0x56239e462230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.092+0000 7f4ac4386740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56239e453d80 0x56239e45f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.092+0000 7f4ac4386740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e462980 0x56239e45fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.092+0000 7f4ac4386740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56239e4657c0 con 0x56239e462980 2024-04-25T00:18:43.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.092+0000 7f4ac4386740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56239e374050 con 0x56239e453460 2024-04-25T00:18:43.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.092+0000 7f4ac4386740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56239e465940 con 0x56239e453d80 2024-04-25T00:18:43.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.092+0000 7f4ac4386740 10 client.0 _ll_register_callbacks cb 0x56239e0cbbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:43.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.092+0000 7f4abb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e462980 0x56239e45fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.092+0000 7f4abb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e462980 0x56239e45fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41832/0 (socket says 172.21.15.107:41832) 2024-04-25T00:18:43.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.092+0000 7f4abb7fe640 1 -- 172.21.15.107:0/253666539 learned_addr learned my addr 172.21.15.107:0/253666539 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:43.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.092+0000 7f4abb7fe640 1 --2- 172.21.15.107:0/253666539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56239e453460 0x56239e462230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.092+0000 7f4abaffd640 1 --2- 172.21.15.107:0/253666539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56239e453d80 0x56239e45f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.092+0000 7f4abb7fe640 1 -- 172.21.15.107:0/253666539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56239e453d80 msgr2=0x56239e45f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:43.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.092+0000 7f4abb7fe640 1 --2- 172.21.15.107:0/253666539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56239e453d80 0x56239e45f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.092+0000 7f4abb7fe640 1 -- 172.21.15.107:0/253666539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56239e453460 msgr2=0x56239e462230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:43.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.092+0000 7f4abb7fe640 1 --2- 172.21.15.107:0/253666539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56239e453460 0x56239e462230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.092+0000 7f4abb7fe640 1 -- 172.21.15.107:0/253666539 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56239e345dc0 con 0x56239e462980 2024-04-25T00:18:43.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.092+0000 7f4abb7fe640 1 --2- 172.21.15.107:0/253666539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e462980 0x56239e45fa60 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4005ab0 tx=0x7f4ab40095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2619746209ea74b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:43.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.092+0000 7f4a9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:18:43.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.093+0000 7f4a9eff5640 1 -- 172.21.15.107:0/253666539 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4035040 con 0x56239e462980 2024-04-25T00:18:43.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.093+0000 7f4a9eff5640 1 -- 172.21.15.107:0/253666539 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f4ab4004320 con 0x56239e462980 2024-04-25T00:18:43.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.093+0000 7f4ac4386740 1 -- 172.21.15.107:0/253666539 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aac019430 con 0x56239e462980 2024-04-25T00:18:43.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.093+0000 7f4a9eff5640 1 -- 172.21.15.107:0/253666539 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4ab4054050 con 0x56239e462980 2024-04-25T00:18:43.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.093+0000 7f4ac4386740 10 client.16134 fetch_fsmap learned FSMap version 28 2024-04-25T00:18:43.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.093+0000 7f4ac4386740 1 -- 172.21.15.107:0/253666539 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56239e0cc3f0 con 0x56239e462980 2024-04-25T00:18:43.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.093+0000 7f4a9eff5640 1 -- 172.21.15.107:0/253666539 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aac019430 con 0x56239e462980 2024-04-25T00:18:43.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.093+0000 7f4a9eff5640 1 -- 172.21.15.107:0/253666539 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f4ab403ebb0 con 0x56239e462980 2024-04-25T00:18:43.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.093+0000 7f4ac4386740 10 client.16134 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:18:43.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.093+0000 7f4ac4386740 10 client.16134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:18:43.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.093+0000 7f4ac4386740 20 client.16134 populate_metadata read hostname 'smithi107' 2024-04-25T00:18:43.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.093+0000 7f4ac4386740 1 --2- 172.21.15.107:0/253666539 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56239e472970 0x56239e474dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.093+0000 7f4ac4386740 4 client.16134 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:18:43.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.093+0000 7f4ac4386740 1 -- 172.21.15.107:0/253666539 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56239e3c5bb0 con 0x56239e472970 2024-04-25T00:18:43.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.093+0000 7f4abaffd640 1 --2- 172.21.15.107:0/253666539 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56239e472970 0x56239e474dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.094+0000 7f4a9eff5640 10 client.16134 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:43.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.094+0000 7f4abaffd640 1 --2- 172.21.15.107:0/253666539 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56239e472970 0x56239e474dd0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49fa4ca8f795f655 server_cookie=ff734802552ff063 in_seq=0 out_seq=0 2024-04-25T00:18:43.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.095+0000 7f4a9eff5640 1 -- 172.21.15.107:0/253666539 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56239e3c5bb0 con 0x56239e472970 2024-04-25T00:18:43.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.095+0000 7f4a9eff5640 10 client.16134 handle_command_reply: tid=0 2024-04-25T00:18:43.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.095+0000 7f4ac4386740 1 client.16134 shutdown 2024-04-25T00:18:43.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.095+0000 7f4ac4386740 1 -- 172.21.15.107:0/253666539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e462980 msgr2=0x56239e45fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:43.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.095+0000 7f4ac4386740 1 --2- 172.21.15.107:0/253666539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e462980 0x56239e45fa60 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4005ab0 tx=0x7f4ab40095e0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.095+0000 7f4ac4386740 1 -- 172.21.15.107:0/253666539 shutdown_connections 2024-04-25T00:18:43.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.095+0000 7f4ac4386740 1 --2- 172.21.15.107:0/253666539 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56239e472970 0x56239e474dd0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.095+0000 7f4ac4386740 1 --2- 172.21.15.107:0/253666539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e462980 0x56239e45fa60 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.095+0000 7f4ac4386740 1 --2- 172.21.15.107:0/253666539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56239e453d80 0x56239e45f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.095+0000 7f4ac4386740 1 --2- 172.21.15.107:0/253666539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56239e453460 0x56239e462230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.095+0000 7f4a9eff5640 0 client.16134 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:43.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.095+0000 7f4ac4386740 1 -- 172.21.15.107:0/253666539 >> 172.21.15.107:0/253666539 conn(0x56239e44f8a0 msgr2=0x56239e455650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:43.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.095+0000 7f4ac4386740 1 -- 172.21.15.107:0/253666539 shutdown_connections 2024-04-25T00:18:43.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.095+0000 7f4ac4386740 1 -- 172.21.15.107:0/253666539 wait complete. 2024-04-25T00:18:43.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.096+0000 7f4ac4386740 20 client.16134 trim_cache size 0 max 16384 2024-04-25T00:18:43.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.110+0000 7f4ac4386740 1 Processor -- start 2024-04-25T00:18:43.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.110+0000 7f4ac4386740 1 -- start start 2024-04-25T00:18:43.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.110+0000 7f4ac4386740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e453460 0x56239e46e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.110+0000 7f4ac4386740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56239e453d80 0x56239e45ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.110+0000 7f4ac4386740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56239e462980 0x56239e45e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.110+0000 7f4ac4386740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc1af1e0 con 0x56239e453460 2024-04-25T00:18:43.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.110+0000 7f4ac4386740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc1ac9f0 con 0x56239e462980 2024-04-25T00:18:43.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.110+0000 7f4ac4386740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc1aa840 con 0x56239e453d80 2024-04-25T00:18:43.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.110+0000 7f4abb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e453460 0x56239e46e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.110+0000 7f4abb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e453460 0x56239e46e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41842/0 (socket says 172.21.15.107:41842) 2024-04-25T00:18:43.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.110+0000 7f4abb7fe640 1 -- 172.21.15.107:0/2428258692 learned_addr learned my addr 172.21.15.107:0/2428258692 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:43.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.110+0000 7f4abaffd640 1 --2- 172.21.15.107:0/2428258692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56239e453d80 0x56239e45ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.110+0000 7f4abb7fe640 1 --2- 172.21.15.107:0/2428258692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56239e462980 0x56239e45e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.110+0000 7f4abb7fe640 1 -- 172.21.15.107:0/2428258692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56239e453d80 msgr2=0x56239e45ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:43.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.110+0000 7f4abb7fe640 1 --2- 172.21.15.107:0/2428258692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56239e453d80 0x56239e45ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.110+0000 7f4abb7fe640 1 -- 172.21.15.107:0/2428258692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56239e462980 msgr2=0x56239e45e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:43.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.110+0000 7f4abb7fe640 1 --2- 172.21.15.107:0/2428258692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56239e462980 0x56239e45e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.110+0000 7f4abb7fe640 1 -- 172.21.15.107:0/2428258692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56239e453bb0 con 0x56239e453460 2024-04-25T00:18:43.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.111+0000 7f4abb7fe640 1 --2- 172.21.15.107:0/2428258692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e453460 0x56239e46e6f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4001eb0 tx=0x7f4ab402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a8d7179ce29095 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:43.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.111+0000 7f4aa0ff9640 1 -- 172.21.15.107:0/2428258692 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4035070 con 0x56239e453460 2024-04-25T00:18:43.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.111+0000 7f4aa0ff9640 1 -- 172.21.15.107:0/2428258692 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ab4043040 con 0x56239e453460 2024-04-25T00:18:43.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.111+0000 7f4aa0ff9640 1 -- 172.21.15.107:0/2428258692 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab403eba0 con 0x56239e453460 2024-04-25T00:18:43.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.111+0000 7f4ac4386740 1 -- 172.21.15.107:0/2428258692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e453460 msgr2=0x56239e46e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:43.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.111+0000 7f4ac4386740 1 --2- 172.21.15.107:0/2428258692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e453460 0x56239e46e6f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4001eb0 tx=0x7f4ab402a420 comp rx=0 tx=0).stop 2024-04-25T00:18:43.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.111+0000 7f4ac4386740 1 -- 172.21.15.107:0/2428258692 shutdown_connections 2024-04-25T00:18:43.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.111+0000 7f4ac4386740 1 --2- 172.21.15.107:0/2428258692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56239e462980 0x56239e45e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.111+0000 7f4ac4386740 1 --2- 172.21.15.107:0/2428258692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56239e453d80 0x56239e45ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.111+0000 7f4ac4386740 1 --2- 172.21.15.107:0/2428258692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e453460 0x56239e46e6f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.111+0000 7f4ac4386740 1 -- 172.21.15.107:0/2428258692 >> 172.21.15.107:0/2428258692 conn(0x56239e44f8a0 msgr2=0x56239e45ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:43.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.111+0000 7f4ac4386740 1 -- 172.21.15.107:0/2428258692 shutdown_connections 2024-04-25T00:18:43.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.111+0000 7f4ac4386740 1 -- 172.21.15.107:0/2428258692 wait complete. 2024-04-25T00:18:43.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.112+0000 7f4ac4386740 1 -- start start 2024-04-25T00:18:43.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.112+0000 7f4ac4386740 1 Processor -- start 2024-04-25T00:18:43.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.112+0000 7f4ac4386740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56239e453460 0x56239e46ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.112+0000 7f4ac4386740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56239e453d80 0x56239e46b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.112+0000 7f4ac4386740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e462980 0x56239e45b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.112+0000 7f4ac4386740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc1aaeb0 con 0x56239e462980 2024-04-25T00:18:43.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.112+0000 7f4ac4386740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc1ac320 con 0x56239e453d80 2024-04-25T00:18:43.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.112+0000 7f4ac4386740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4abc1ab580 con 0x56239e453460 2024-04-25T00:18:43.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.112+0000 7f4ac4386740 10 client.0 _ll_register_callbacks cb 0x7f4ab4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:43.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.112+0000 7f4abb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e462980 0x56239e45b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.112+0000 7f4abb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e462980 0x56239e45b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41852/0 (socket says 172.21.15.107:41852) 2024-04-25T00:18:43.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.112+0000 7f4abb7fe640 1 -- 172.21.15.107:0/16041963 learned_addr learned my addr 172.21.15.107:0/16041963 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:43.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.112+0000 7f4abaffd640 1 --2- 172.21.15.107:0/16041963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56239e453d80 0x56239e46b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.112+0000 7f4abb7fe640 1 --2- 172.21.15.107:0/16041963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56239e453460 0x56239e46ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.112+0000 7f4abb7fe640 1 -- 172.21.15.107:0/16041963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56239e453460 msgr2=0x56239e46ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:43.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.112+0000 7f4abb7fe640 1 --2- 172.21.15.107:0/16041963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56239e453460 0x56239e46ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.112+0000 7f4abb7fe640 1 -- 172.21.15.107:0/16041963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56239e453d80 msgr2=0x56239e46b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:43.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.112+0000 7f4abb7fe640 1 --2- 172.21.15.107:0/16041963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56239e453d80 0x56239e46b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.112+0000 7f4abb7fe640 1 -- 172.21.15.107:0/16041963 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56239e1967d0 con 0x56239e462980 2024-04-25T00:18:43.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.112+0000 7f4abb7fe640 1 --2- 172.21.15.107:0/16041963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e462980 0x56239e45b790 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x56239e45c910 tx=0x7f4ab40045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b92452391c2857 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:43.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.112+0000 7f4a9eff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:18:43.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.113+0000 7f4a9eff5640 1 -- 172.21.15.107:0/16041963 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4035070 con 0x56239e462980 2024-04-25T00:18:43.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.113+0000 7f4a9eff5640 1 -- 172.21.15.107:0/16041963 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f4ab403eb30 con 0x56239e462980 2024-04-25T00:18:43.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.113+0000 7f4ac4386740 1 -- 172.21.15.107:0/16041963 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ab4043040 con 0x56239e462980 2024-04-25T00:18:43.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.113+0000 7f4a9eff5640 1 -- 172.21.15.107:0/16041963 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4ab406c050 con 0x56239e462980 2024-04-25T00:18:43.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.113+0000 7f4ac4386740 10 client.16140 fetch_fsmap learned FSMap version 28 2024-04-25T00:18:43.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.113+0000 7f4ac4386740 1 -- 172.21.15.107:0/16041963 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56239e4d8510 con 0x56239e462980 2024-04-25T00:18:43.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.113+0000 7f4a9eff5640 1 -- 172.21.15.107:0/16041963 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ab4043040 con 0x56239e462980 2024-04-25T00:18:43.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.113+0000 7f4a9eff5640 1 -- 172.21.15.107:0/16041963 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f4ab4064020 con 0x56239e462980 2024-04-25T00:18:43.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.113+0000 7f4ac4386740 10 client.16140 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:18:43.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.113+0000 7f4ac4386740 10 client.16140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:18:43.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.113+0000 7f4ac4386740 20 client.16140 populate_metadata read hostname 'smithi107' 2024-04-25T00:18:43.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.113+0000 7f4ac4386740 1 --2- 172.21.15.107:0/16041963 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56239e4d8d70 0x56239e4db190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.113+0000 7f4ac4386740 4 client.16140 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:18:43.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.113+0000 7f4ac4386740 1 -- 172.21.15.107:0/16041963 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56239e3bff00 con 0x56239e4d8d70 2024-04-25T00:18:43.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.113+0000 7f4abaffd640 1 --2- 172.21.15.107:0/16041963 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56239e4d8d70 0x56239e4db190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.114+0000 7f4a9eff5640 10 client.16140 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:43.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.114+0000 7f4abaffd640 1 --2- 172.21.15.107:0/16041963 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56239e4d8d70 0x56239e4db190 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3833ee1651d912f4 server_cookie=fbda748c96cdac3c in_seq=0 out_seq=0 2024-04-25T00:18:43.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.116+0000 7f4a9eff5640 1 -- 172.21.15.107:0/16041963 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56239e3bff00 con 0x56239e4d8d70 2024-04-25T00:18:43.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.116+0000 7f4a9eff5640 10 client.16140 handle_command_reply: tid=0 2024-04-25T00:18:43.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.116+0000 7f4ac4386740 1 client.16140 shutdown 2024-04-25T00:18:43.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.117+0000 7f4ac4386740 1 -- 172.21.15.107:0/16041963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e462980 msgr2=0x56239e45b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:43.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.117+0000 7f4ac4386740 1 --2- 172.21.15.107:0/16041963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e462980 0x56239e45b790 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x56239e45c910 tx=0x7f4ab40045b0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.117+0000 7f4ac4386740 1 -- 172.21.15.107:0/16041963 shutdown_connections 2024-04-25T00:18:43.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.117+0000 7f4ac4386740 1 --2- 172.21.15.107:0/16041963 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56239e4d8d70 0x56239e4db190 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.117+0000 7f4ac4386740 1 --2- 172.21.15.107:0/16041963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56239e462980 0x56239e45b790 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.117+0000 7f4ac4386740 1 --2- 172.21.15.107:0/16041963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56239e453d80 0x56239e46b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.117+0000 7f4a9eff5640 0 client.16140 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:43.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.117+0000 7f4ac4386740 1 --2- 172.21.15.107:0/16041963 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56239e453460 0x56239e46ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.117+0000 7f4ac4386740 1 -- 172.21.15.107:0/16041963 >> 172.21.15.107:0/16041963 conn(0x56239e44f8a0 msgr2=0x56239e45e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:43.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.117+0000 7f4ac4386740 1 -- 172.21.15.107:0/16041963 shutdown_connections 2024-04-25T00:18:43.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.117+0000 7f4ac4386740 1 -- 172.21.15.107:0/16041963 wait complete. 2024-04-25T00:18:43.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.117+0000 7f4ac4386740 20 client.16140 trim_cache size 0 max 16384 2024-04-25T00:18:43.124 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:18:43.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.117+0000 7f4ac1676640 1 -- 172.21.15.107:0/389654047 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4aa40a2d00 msgr2=0x7f4aa40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:43.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.117+0000 7f4ac1676640 1 --2- 172.21.15.107:0/389654047 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4aa40a2d00 0x7f4aa40a51c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00987b0 tx=0x7f4ab0097a40 comp rx=0 tx=0).stop 2024-04-25T00:18:43.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.117+0000 7f4ac1676640 1 -- 172.21.15.107:0/389654047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4abc100fd0 msgr2=0x7f4abc198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:43.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.117+0000 7f4ac1676640 1 --2- 172.21.15.107:0/389654047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4abc100fd0 0x7f4abc198430 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000e9f0 tx=0x7f4ab000eeb0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.117+0000 7f4ac1676640 1 -- 172.21.15.107:0/389654047 shutdown_connections 2024-04-25T00:18:43.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.117+0000 7f4ac1676640 1 --2- 172.21.15.107:0/389654047 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4aa40a2d00 0x7f4aa40a51c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.117+0000 7f4ac1676640 1 --2- 172.21.15.107:0/389654047 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4abc1066b0 0x7f4abc19c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.117+0000 7f4ac1676640 1 --2- 172.21.15.107:0/389654047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4abc100fd0 0x7f4abc198430 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.117+0000 7f4ac1676640 1 --2- 172.21.15.107:0/389654047 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4abc100610 0x7f4abc197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.117+0000 7f4ac1676640 1 -- 172.21.15.107:0/389654047 >> 172.21.15.107:0/389654047 conn(0x7f4abc077740 msgr2=0x7f4abc0ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:43.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.117+0000 7f4ac1676640 1 -- 172.21.15.107:0/389654047 shutdown_connections 2024-04-25T00:18:43.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:43.117+0000 7f4ac1676640 1 -- 172.21.15.107:0/389654047 wait complete. 2024-04-25T00:18:43.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:42 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1673906518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:43.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:42 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1673906518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:43.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:42 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1673906518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:44.129 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:18:44.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:18:44.129 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:18:44.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:43 smithi170 ceph-mon[32807]: pgmap v71: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 208 MiB/s rd, 248 MiB/s wr, 426 op/s 2024-04-25T00:18:44.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:43 smithi107 ceph-mon[29153]: pgmap v71: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 208 MiB/s rd, 248 MiB/s wr, 426 op/s 2024-04-25T00:18:44.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.237+0000 7f292c1ee640 1 -- 172.21.15.107:0/2248342906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2924100fd0 msgr2=0x7f292410d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:44.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.237+0000 7f292c1ee640 1 --2- 172.21.15.107:0/2248342906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2924100fd0 0x7f292410d4e0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f291c009cd0 tx=0x7f291c02a8e0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.238+0000 7f292c1ee640 1 -- 172.21.15.107:0/2248342906 shutdown_connections 2024-04-25T00:18:44.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.238+0000 7f292c1ee640 1 --2- 172.21.15.107:0/2248342906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2924100fd0 0x7f292410d4e0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.238+0000 7f292c1ee640 1 --2- 172.21.15.107:0/2248342906 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2924100610 0x7f2924100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.238+0000 7f292c1ee640 1 --2- 172.21.15.107:0/2248342906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29241066b0 0x7f2924106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.238+0000 7f292c1ee640 1 -- 172.21.15.107:0/2248342906 >> 172.21.15.107:0/2248342906 conn(0x7f2924077740 msgr2=0x7f2924077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:44.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.238+0000 7f292c1ee640 1 -- 172.21.15.107:0/2248342906 shutdown_connections 2024-04-25T00:18:44.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.238+0000 7f292c1ee640 1 -- 172.21.15.107:0/2248342906 wait complete. 2024-04-25T00:18:44.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.238+0000 7f292c1ee640 1 Processor -- start 2024-04-25T00:18:44.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.238+0000 7f292c1ee640 1 -- start start 2024-04-25T00:18:44.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.238+0000 7f292c1ee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2924100610 0x7f2924198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.238+0000 7f292c1ee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2924100fd0 0x7f2924199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.238+0000 7f292c1ee640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29241066b0 0x7f29241a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.238+0000 7f292c1ee640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2924110a50 con 0x7f2924100fd0 2024-04-25T00:18:44.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.238+0000 7f292c1ee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f29241108d0 con 0x7f2924100610 2024-04-25T00:18:44.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.238+0000 7f292c1ee640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2924110bd0 con 0x7f29241066b0 2024-04-25T00:18:44.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.239+0000 7f2929789640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2924100fd0 0x7f2924199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.239+0000 7f2929789640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2924100fd0 0x7f2924199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41868/0 (socket says 172.21.15.107:41868) 2024-04-25T00:18:44.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.239+0000 7f2929789640 1 -- 172.21.15.107:0/1544757987 learned_addr learned my addr 172.21.15.107:0/1544757987 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:44.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.239+0000 7f2929f8a640 1 --2- 172.21.15.107:0/1544757987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2924100610 0x7f2924198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.239+0000 7f292a78b640 1 --2- 172.21.15.107:0/1544757987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29241066b0 0x7f29241a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.239+0000 7f2929789640 1 -- 172.21.15.107:0/1544757987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29241066b0 msgr2=0x7f29241a3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:44.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.239+0000 7f2929789640 1 --2- 172.21.15.107:0/1544757987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29241066b0 0x7f29241a3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.239+0000 7f2929789640 1 -- 172.21.15.107:0/1544757987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2924100610 msgr2=0x7f2924198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:44.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.239+0000 7f2929789640 1 --2- 172.21.15.107:0/1544757987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2924100610 0x7f2924198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.239+0000 7f2929789640 1 -- 172.21.15.107:0/1544757987 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29241a4410 con 0x7f2924100fd0 2024-04-25T00:18:44.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.239+0000 7f2929789640 1 --2- 172.21.15.107:0/1544757987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2924100fd0 0x7f2924199390 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f292000d950 tx=0x7f292000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:44.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.239+0000 7f291affd640 1 -- 172.21.15.107:0/1544757987 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920017070 con 0x7f2924100fd0 2024-04-25T00:18:44.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.239+0000 7f291affd640 1 -- 172.21.15.107:0/1544757987 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f292000de80 con 0x7f2924100fd0 2024-04-25T00:18:44.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.239+0000 7f292c1ee640 1 -- 172.21.15.107:0/1544757987 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29241a45e0 con 0x7f2924100fd0 2024-04-25T00:18:44.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.239+0000 7f291affd640 1 -- 172.21.15.107:0/1544757987 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292000f620 con 0x7f2924100fd0 2024-04-25T00:18:44.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.239+0000 7f292c1ee640 1 -- 172.21.15.107:0/1544757987 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29241a4aa0 con 0x7f2924100fd0 2024-04-25T00:18:44.243 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:43 smithi195 ceph-mon[32833]: pgmap v71: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 208 MiB/s rd, 248 MiB/s wr, 426 op/s 2024-04-25T00:18:44.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.241+0000 7f291affd640 1 -- 172.21.15.107:0/1544757987 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f2920009c30 con 0x7f2924100fd0 2024-04-25T00:18:44.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.241+0000 7f292efdc740 1 Processor -- start 2024-04-25T00:18:44.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.242+0000 7f292efdc740 1 -- start start 2024-04-25T00:18:44.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.242+0000 7f292efdc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55653cd0c980 0x55653cd0acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.242+0000 7f292efdc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd0b290 0x55653cd0b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.242+0000 7f292efdc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55653cd16010 0x55653cd18400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.242+0000 7f292efdc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55653cc6f2c0 con 0x55653cd0b290 2024-04-25T00:18:44.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.242+0000 7f292efdc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55653cc17910 con 0x55653cd0c980 2024-04-25T00:18:44.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.242+0000 7f292efdc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55653cc6b280 con 0x55653cd16010 2024-04-25T00:18:44.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.242+0000 7f2929f8a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd0b290 0x55653cd0b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.242+0000 7f291affd640 1 --2- 172.21.15.107:0/1544757987 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f29100a2d00 0x7f29100a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.242+0000 7f2929f8a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd0b290 0x55653cd0b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41870/0 (socket says 172.21.15.107:41870) 2024-04-25T00:18:44.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.242+0000 7f2929f8a640 1 -- 172.21.15.107:0/1940697702 learned_addr learned my addr 172.21.15.107:0/1940697702 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:44.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.242+0000 7f292a78b640 1 --2- 172.21.15.107:0/1940697702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55653cd0c980 0x55653cd0acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.242+0000 7f2929789640 1 --2- 172.21.15.107:0/1940697702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55653cd16010 0x55653cd18400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.242+0000 7f292a78b640 1 --2- 172.21.15.107:0/1544757987 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f29100a2d00 0x7f29100a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.242+0000 7f291affd640 1 -- 172.21.15.107:0/1544757987 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f29200cf3c0 con 0x7f2924100fd0 2024-04-25T00:18:44.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.242+0000 7f2929f8a640 1 -- 172.21.15.107:0/1940697702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55653cd16010 msgr2=0x55653cd18400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:44.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.242+0000 7f2929f8a640 1 --2- 172.21.15.107:0/1940697702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55653cd16010 0x55653cd18400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.242+0000 7f2929f8a640 1 -- 172.21.15.107:0/1940697702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55653cd0c980 msgr2=0x55653cd0acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:44.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.242+0000 7f2929f8a640 1 --2- 172.21.15.107:0/1940697702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55653cd0c980 0x55653cd0acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.242+0000 7f2929f8a640 1 -- 172.21.15.107:0/1940697702 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55653cc28fb0 con 0x55653cd0b290 2024-04-25T00:18:44.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.242+0000 7f2929789640 1 --2- 172.21.15.107:0/1940697702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55653cd16010 0x55653cd18400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:18:44.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.243+0000 7f2929f8a640 1 --2- 172.21.15.107:0/1940697702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd0b290 0x55653cd0b710 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2914009a40 tx=0x7f2914009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d22d2cd1dfcf7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:44.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.243+0000 7f292a78b640 1 --2- 172.21.15.107:0/1544757987 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f29100a2d00 0x7f29100a51c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f291c004ab0 tx=0x7f291c004a40 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:44.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.243+0000 7f290ffff640 1 -- 172.21.15.107:0/1940697702 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914017070 con 0x55653cd0b290 2024-04-25T00:18:44.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.243+0000 7f290ffff640 1 -- 172.21.15.107:0/1940697702 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2914013470 con 0x55653cd0b290 2024-04-25T00:18:44.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.243+0000 7f290ffff640 1 -- 172.21.15.107:0/1940697702 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914012630 con 0x55653cd0b290 2024-04-25T00:18:44.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.243+0000 7f292efdc740 1 -- 172.21.15.107:0/1940697702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd0b290 msgr2=0x55653cd0b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:44.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.243+0000 7f292efdc740 1 --2- 172.21.15.107:0/1940697702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd0b290 0x55653cd0b710 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2914009a40 tx=0x7f2914009f00 comp rx=0 tx=0).stop 2024-04-25T00:18:44.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.243+0000 7f292efdc740 1 -- 172.21.15.107:0/1940697702 shutdown_connections 2024-04-25T00:18:44.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.243+0000 7f292efdc740 1 --2- 172.21.15.107:0/1940697702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55653cd16010 0x55653cd18400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.243+0000 7f292efdc740 1 --2- 172.21.15.107:0/1940697702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd0b290 0x55653cd0b710 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.243+0000 7f292efdc740 1 --2- 172.21.15.107:0/1940697702 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55653cd0c980 0x55653cd0acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.243+0000 7f292efdc740 1 -- 172.21.15.107:0/1940697702 >> 172.21.15.107:0/1940697702 conn(0x55653ccd3950 msgr2=0x55653cd077f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:44.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.244+0000 7f292efdc740 1 -- 172.21.15.107:0/1940697702 shutdown_connections 2024-04-25T00:18:44.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.244+0000 7f292efdc740 1 -- 172.21.15.107:0/1940697702 wait complete. 2024-04-25T00:18:44.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.244+0000 7f292efdc740 1 -- start start 2024-04-25T00:18:44.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.244+0000 7f292efdc740 1 Processor -- start 2024-04-25T00:18:44.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.244+0000 7f292efdc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55653cd0b290 0x55653cd12f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.244+0000 7f292efdc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55653cd0c980 0x55653cd13450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.244+0000 7f292efdc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd16010 0x55653cd13990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.244+0000 7f292efdc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55653cd18ea0 con 0x55653cd16010 2024-04-25T00:18:44.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.244+0000 7f292efdc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55653cc2ae90 con 0x55653cd0b290 2024-04-25T00:18:44.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.244+0000 7f292efdc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55653cd19020 con 0x55653cd0c980 2024-04-25T00:18:44.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.244+0000 7f292efdc740 10 client.0 _ll_register_callbacks cb 0x55653cc66970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:44.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.244+0000 7f292a78b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd16010 0x55653cd13990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.244+0000 7f292a78b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd16010 0x55653cd13990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41878/0 (socket says 172.21.15.107:41878) 2024-04-25T00:18:44.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.244+0000 7f2929f8a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55653cd0b290 0x55653cd12f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.244+0000 7f292a78b640 1 -- 172.21.15.107:0/1914825141 learned_addr learned my addr 172.21.15.107:0/1914825141 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:44.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.245+0000 7f2929789640 1 --2- 172.21.15.107:0/1914825141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55653cd0c980 0x55653cd13450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.245+0000 7f292a78b640 1 -- 172.21.15.107:0/1914825141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55653cd0c980 msgr2=0x55653cd13450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:44.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.245+0000 7f292a78b640 1 --2- 172.21.15.107:0/1914825141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55653cd0c980 0x55653cd13450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.245+0000 7f292a78b640 1 -- 172.21.15.107:0/1914825141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55653cd0b290 msgr2=0x55653cd12f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:44.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.245+0000 7f292a78b640 1 --2- 172.21.15.107:0/1914825141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55653cd0b290 0x55653cd12f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.245+0000 7f292a78b640 1 -- 172.21.15.107:0/1914825141 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55653ca4b7e0 con 0x55653cd16010 2024-04-25T00:18:44.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.245+0000 7f2929789640 1 --2- 172.21.15.107:0/1914825141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55653cd0c980 0x55653cd13450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:18:44.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.245+0000 7f2929f8a640 1 --2- 172.21.15.107:0/1914825141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55653cd0b290 0x55653cd12f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:44.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.245+0000 7f292a78b640 1 --2- 172.21.15.107:0/1914825141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd16010 0x55653cd13990 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f291c003e70 tx=0x7f291c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed301c322e5981e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:44.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.245+0000 7f290d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:18:44.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.245+0000 7f290d7fa640 1 -- 172.21.15.107:0/1914825141 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291c0307a0 con 0x55653cd16010 2024-04-25T00:18:44.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.245+0000 7f292efdc740 1 -- 172.21.15.107:0/1914825141 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2914013470 con 0x55653cd16010 2024-04-25T00:18:44.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.245+0000 7f290d7fa640 1 -- 172.21.15.107:0/1914825141 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f291c030940 con 0x55653cd16010 2024-04-25T00:18:44.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.245+0000 7f290d7fa640 1 -- 172.21.15.107:0/1914825141 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f291c030b40 con 0x55653cd16010 2024-04-25T00:18:44.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.245+0000 7f292efdc740 10 client.16152 fetch_fsmap learned FSMap version 28 2024-04-25T00:18:44.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.245+0000 7f292efdc740 1 -- 172.21.15.107:0/1914825141 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55653caabf70 con 0x55653cd16010 2024-04-25T00:18:44.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.245+0000 7f290d7fa640 1 -- 172.21.15.107:0/1914825141 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2914013470 con 0x55653cd16010 2024-04-25T00:18:44.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.246+0000 7f290d7fa640 1 -- 172.21.15.107:0/1914825141 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f291c048040 con 0x55653cd16010 2024-04-25T00:18:44.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.246+0000 7f292efdc740 10 client.16152 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:18:44.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.246+0000 7f292efdc740 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:18:44.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.246+0000 7f292efdc740 20 client.16152 populate_metadata read hostname 'smithi107' 2024-04-25T00:18:44.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.246+0000 7f292efdc740 1 --2- 172.21.15.107:0/1914825141 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55653cd26120 0x55653cd28580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.246+0000 7f292efdc740 4 client.16152 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:18:44.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.246+0000 7f292efdc740 1 -- 172.21.15.107:0/1914825141 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55653cc435a0 con 0x55653cd26120 2024-04-25T00:18:44.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.246+0000 7f2929f8a640 1 --2- 172.21.15.107:0/1914825141 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55653cd26120 0x55653cd28580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.246+0000 7f290d7fa640 10 client.16152 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:44.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.246+0000 7f2929f8a640 1 --2- 172.21.15.107:0/1914825141 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55653cd26120 0x55653cd28580 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=215145f3b84a785c server_cookie=4585fc06a8b37b4a in_seq=0 out_seq=0 2024-04-25T00:18:44.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.248+0000 7f290d7fa640 1 -- 172.21.15.107:0/1914825141 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55653cc435a0 con 0x55653cd26120 2024-04-25T00:18:44.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.248+0000 7f290d7fa640 10 client.16152 handle_command_reply: tid=0 2024-04-25T00:18:44.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.248+0000 7f292efdc740 1 client.16152 shutdown 2024-04-25T00:18:44.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.248+0000 7f292efdc740 1 -- 172.21.15.107:0/1914825141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd16010 msgr2=0x55653cd13990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:44.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.248+0000 7f292efdc740 1 --2- 172.21.15.107:0/1914825141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd16010 0x55653cd13990 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f291c003e70 tx=0x7f291c0095c0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.248+0000 7f292efdc740 1 -- 172.21.15.107:0/1914825141 shutdown_connections 2024-04-25T00:18:44.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.248+0000 7f292efdc740 1 --2- 172.21.15.107:0/1914825141 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55653cd26120 0x55653cd28580 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.248+0000 7f292efdc740 1 --2- 172.21.15.107:0/1914825141 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd16010 0x55653cd13990 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.248+0000 7f292efdc740 1 --2- 172.21.15.107:0/1914825141 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55653cd0c980 0x55653cd13450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.248+0000 7f290d7fa640 0 client.16152 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:44.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.248+0000 7f292efdc740 1 --2- 172.21.15.107:0/1914825141 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55653cd0b290 0x55653cd12f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.248+0000 7f292efdc740 1 -- 172.21.15.107:0/1914825141 >> 172.21.15.107:0/1914825141 conn(0x55653ccd3950 msgr2=0x55653cd0d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:44.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.248+0000 7f292efdc740 1 -- 172.21.15.107:0/1914825141 shutdown_connections 2024-04-25T00:18:44.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.248+0000 7f292efdc740 1 -- 172.21.15.107:0/1914825141 wait complete. 2024-04-25T00:18:44.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.249+0000 7f292efdc740 20 client.16152 trim_cache size 0 max 16384 2024-04-25T00:18:44.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.267+0000 7f292efdc740 1 Processor -- start 2024-04-25T00:18:44.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.267+0000 7f292efdc740 1 -- start start 2024-04-25T00:18:44.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.267+0000 7f292efdc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd0b290 0x55653cd21da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.267+0000 7f292efdc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55653cd0c980 0x55653cd2de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.267+0000 7f292efdc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55653cd16010 0x55653cd2e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.267+0000 7f292efdc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f29241b13c0 con 0x55653cd0b290 2024-04-25T00:18:44.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.267+0000 7f292efdc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f29241aebd0 con 0x55653cd0c980 2024-04-25T00:18:44.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.267+0000 7f292efdc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f29241aca20 con 0x55653cd16010 2024-04-25T00:18:44.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.267+0000 7f2929f8a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd0b290 0x55653cd21da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.267+0000 7f2929f8a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd0b290 0x55653cd21da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41890/0 (socket says 172.21.15.107:41890) 2024-04-25T00:18:44.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.267+0000 7f2929f8a640 1 -- 172.21.15.107:0/2793438979 learned_addr learned my addr 172.21.15.107:0/2793438979 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:44.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.268+0000 7f2929789640 1 --2- 172.21.15.107:0/2793438979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55653cd0c980 0x55653cd2de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.268+0000 7f292a78b640 1 --2- 172.21.15.107:0/2793438979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55653cd16010 0x55653cd2e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.268+0000 7f2929f8a640 1 -- 172.21.15.107:0/2793438979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55653cd16010 msgr2=0x55653cd2e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:44.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.268+0000 7f2929f8a640 1 --2- 172.21.15.107:0/2793438979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55653cd16010 0x55653cd2e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.268+0000 7f2929f8a640 1 -- 172.21.15.107:0/2793438979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55653cd0c980 msgr2=0x55653cd2de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:44.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.268+0000 7f2929f8a640 1 --2- 172.21.15.107:0/2793438979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55653cd0c980 0x55653cd2de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.268+0000 7f2929f8a640 1 -- 172.21.15.107:0/2793438979 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55653c943170 con 0x55653cd0b290 2024-04-25T00:18:44.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.268+0000 7f2929f8a640 1 --2- 172.21.15.107:0/2793438979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd0b290 0x55653cd21da0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2914009960 tx=0x7f2914009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6176c0bff131d9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:44.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.268+0000 7f290f7fe640 1 -- 172.21.15.107:0/2793438979 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914017040 con 0x55653cd0b290 2024-04-25T00:18:44.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.268+0000 7f290f7fe640 1 -- 172.21.15.107:0/2793438979 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2914026890 con 0x55653cd0b290 2024-04-25T00:18:44.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.268+0000 7f290f7fe640 1 -- 172.21.15.107:0/2793438979 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291401c450 con 0x55653cd0b290 2024-04-25T00:18:44.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.269+0000 7f292efdc740 1 -- 172.21.15.107:0/2793438979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd0b290 msgr2=0x55653cd21da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:44.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.269+0000 7f292efdc740 1 --2- 172.21.15.107:0/2793438979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd0b290 0x55653cd21da0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2914009960 tx=0x7f2914009110 comp rx=0 tx=0).stop 2024-04-25T00:18:44.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.269+0000 7f292efdc740 1 -- 172.21.15.107:0/2793438979 shutdown_connections 2024-04-25T00:18:44.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.269+0000 7f292efdc740 1 --2- 172.21.15.107:0/2793438979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55653cd16010 0x55653cd2e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.269+0000 7f292efdc740 1 --2- 172.21.15.107:0/2793438979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55653cd0c980 0x55653cd2de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.269+0000 7f292efdc740 1 --2- 172.21.15.107:0/2793438979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd0b290 0x55653cd21da0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.269+0000 7f292efdc740 1 -- 172.21.15.107:0/2793438979 >> 172.21.15.107:0/2793438979 conn(0x55653ccd3950 msgr2=0x55653cd06950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:44.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.269+0000 7f292efdc740 1 -- 172.21.15.107:0/2793438979 shutdown_connections 2024-04-25T00:18:44.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.269+0000 7f292efdc740 1 -- 172.21.15.107:0/2793438979 wait complete. 2024-04-25T00:18:44.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.269+0000 7f292efdc740 1 -- start start 2024-04-25T00:18:44.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.269+0000 7f292efdc740 1 Processor -- start 2024-04-25T00:18:44.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.270+0000 7f292efdc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55653cd0b290 0x55653cd1b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.270+0000 7f292efdc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55653cd0c980 0x55653cd1b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.270+0000 7f292efdc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd16010 0x55653cd1bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.270+0000 7f292efdc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f29241ad090 con 0x55653cd16010 2024-04-25T00:18:44.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.270+0000 7f292efdc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f29241ae500 con 0x55653cd0b290 2024-04-25T00:18:44.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.270+0000 7f292efdc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f29241ad760 con 0x55653cd0c980 2024-04-25T00:18:44.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.270+0000 7f292efdc740 10 client.0 _ll_register_callbacks cb 0x7f291c003d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:44.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.270+0000 7f292a78b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd16010 0x55653cd1bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.270+0000 7f2929f8a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55653cd0b290 0x55653cd1b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.270+0000 7f292a78b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd16010 0x55653cd1bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41904/0 (socket says 172.21.15.107:41904) 2024-04-25T00:18:44.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.270+0000 7f292a78b640 1 -- 172.21.15.107:0/3060979877 learned_addr learned my addr 172.21.15.107:0/3060979877 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:44.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.270+0000 7f2929789640 1 --2- 172.21.15.107:0/3060979877 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55653cd0c980 0x55653cd1b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.270+0000 7f292a78b640 1 -- 172.21.15.107:0/3060979877 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55653cd0c980 msgr2=0x55653cd1b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:44.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.270+0000 7f292a78b640 1 --2- 172.21.15.107:0/3060979877 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55653cd0c980 0x55653cd1b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.270+0000 7f292a78b640 1 -- 172.21.15.107:0/3060979877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55653cd0b290 msgr2=0x55653cd1b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:44.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.270+0000 7f292a78b640 1 --2- 172.21.15.107:0/3060979877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55653cd0b290 0x55653cd1b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.270+0000 7f292a78b640 1 -- 172.21.15.107:0/3060979877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55653cd8c6d0 con 0x55653cd16010 2024-04-25T00:18:44.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.270+0000 7f2929f8a640 1 --2- 172.21.15.107:0/3060979877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55653cd0b290 0x55653cd1b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:44.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.270+0000 7f292a78b640 1 --2- 172.21.15.107:0/3060979877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd16010 0x55653cd1bd00 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f291c006010 tx=0x7f291c031610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cab6560be6cb292 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:44.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.270+0000 7f290d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:18:44.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.270+0000 7f290d7fa640 1 -- 172.21.15.107:0/3060979877 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291c067060 con 0x55653cd16010 2024-04-25T00:18:44.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.270+0000 7f290d7fa640 1 -- 172.21.15.107:0/3060979877 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f291c0307a0 con 0x55653cd16010 2024-04-25T00:18:44.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.270+0000 7f292efdc740 1 -- 172.21.15.107:0/3060979877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2914026890 con 0x55653cd16010 2024-04-25T00:18:44.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.271+0000 7f290d7fa640 1 -- 172.21.15.107:0/3060979877 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f291c0309a0 con 0x55653cd16010 2024-04-25T00:18:44.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.271+0000 7f292efdc740 10 client.16158 fetch_fsmap learned FSMap version 28 2024-04-25T00:18:44.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.271+0000 7f292efdc740 1 -- 172.21.15.107:0/3060979877 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55653cd8d850 con 0x55653cd16010 2024-04-25T00:18:44.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.271+0000 7f290d7fa640 1 -- 172.21.15.107:0/3060979877 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2914026890 con 0x55653cd16010 2024-04-25T00:18:44.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.271+0000 7f290d7fa640 1 -- 172.21.15.107:0/3060979877 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f291c048040 con 0x55653cd16010 2024-04-25T00:18:44.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.271+0000 7f292efdc740 10 client.16158 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:18:44.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.271+0000 7f292efdc740 10 client.16158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:18:44.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.271+0000 7f292efdc740 20 client.16158 populate_metadata read hostname 'smithi107' 2024-04-25T00:18:44.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.271+0000 7f292efdc740 1 --2- 172.21.15.107:0/3060979877 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55653cd8dde0 0x55653cd90200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.271+0000 7f292efdc740 4 client.16158 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:18:44.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.271+0000 7f292efdc740 1 -- 172.21.15.107:0/3060979877 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55653cc7c800 con 0x55653cd8dde0 2024-04-25T00:18:44.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.271+0000 7f2929f8a640 1 --2- 172.21.15.107:0/3060979877 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55653cd8dde0 0x55653cd90200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.271+0000 7f2929f8a640 1 --2- 172.21.15.107:0/3060979877 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55653cd8dde0 0x55653cd90200 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6e4cb51123094eb server_cookie=428419a9535230ad in_seq=0 out_seq=0 2024-04-25T00:18:44.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.271+0000 7f290d7fa640 10 client.16158 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:44.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.272+0000 7f290d7fa640 1 -- 172.21.15.107:0/3060979877 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55653cc7c800 con 0x55653cd8dde0 2024-04-25T00:18:44.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.272+0000 7f290d7fa640 10 client.16158 handle_command_reply: tid=0 2024-04-25T00:18:44.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.272+0000 7f292efdc740 1 client.16158 shutdown 2024-04-25T00:18:44.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.272+0000 7f292efdc740 1 -- 172.21.15.107:0/3060979877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd16010 msgr2=0x55653cd1bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:44.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.272+0000 7f292efdc740 1 --2- 172.21.15.107:0/3060979877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd16010 0x55653cd1bd00 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f291c006010 tx=0x7f291c031610 comp rx=0 tx=0).stop 2024-04-25T00:18:44.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.273+0000 7f292efdc740 1 -- 172.21.15.107:0/3060979877 shutdown_connections 2024-04-25T00:18:44.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.273+0000 7f292efdc740 1 --2- 172.21.15.107:0/3060979877 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55653cd8dde0 0x55653cd90200 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.273+0000 7f292efdc740 1 --2- 172.21.15.107:0/3060979877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55653cd16010 0x55653cd1bd00 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.273+0000 7f292efdc740 1 --2- 172.21.15.107:0/3060979877 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55653cd0c980 0x55653cd1b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.273+0000 7f290d7fa640 0 client.16158 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:44.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.273+0000 7f292efdc740 1 --2- 172.21.15.107:0/3060979877 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55653cd0b290 0x55653cd1b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.273+0000 7f292efdc740 1 -- 172.21.15.107:0/3060979877 >> 172.21.15.107:0/3060979877 conn(0x55653ccd3950 msgr2=0x55653cd12ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:44.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.273+0000 7f292efdc740 1 -- 172.21.15.107:0/3060979877 shutdown_connections 2024-04-25T00:18:44.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.273+0000 7f292efdc740 1 -- 172.21.15.107:0/3060979877 wait complete. 2024-04-25T00:18:44.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.273+0000 7f292efdc740 20 client.16158 trim_cache size 0 max 16384 2024-04-25T00:18:44.279 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:18:44.279 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-04-25T00:18:44.279 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "ebf9cfce-4d97-4f4c-9fc7-b3cc448508ff", 2024-04-25T00:18:44.279 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-04-25T00:18:44.279 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:18:44.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.273+0000 7f292c1ee640 1 -- 172.21.15.107:0/1544757987 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f29100a2d00 msgr2=0x7f29100a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:44.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.273+0000 7f292c1ee640 1 --2- 172.21.15.107:0/1544757987 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f29100a2d00 0x7f29100a51c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f291c004ab0 tx=0x7f291c004a40 comp rx=0 tx=0).stop 2024-04-25T00:18:44.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.273+0000 7f292c1ee640 1 -- 172.21.15.107:0/1544757987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2924100fd0 msgr2=0x7f2924199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:44.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.273+0000 7f292c1ee640 1 --2- 172.21.15.107:0/1544757987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2924100fd0 0x7f2924199390 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f292000d950 tx=0x7f292000dd50 comp rx=0 tx=0).stop 2024-04-25T00:18:44.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.273+0000 7f292c1ee640 1 -- 172.21.15.107:0/1544757987 shutdown_connections 2024-04-25T00:18:44.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.273+0000 7f292c1ee640 1 --2- 172.21.15.107:0/1544757987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f29241066b0 0x7f29241a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.273+0000 7f292c1ee640 1 --2- 172.21.15.107:0/1544757987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2924100fd0 0x7f2924199390 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.273+0000 7f292c1ee640 1 --2- 172.21.15.107:0/1544757987 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f29100a2d00 0x7f29100a51c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.273+0000 7f292c1ee640 1 --2- 172.21.15.107:0/1544757987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2924100610 0x7f2924198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.273+0000 7f292c1ee640 1 -- 172.21.15.107:0/1544757987 >> 172.21.15.107:0/1544757987 conn(0x7f2924077740 msgr2=0x7f29240ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:44.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.273+0000 7f292c1ee640 1 -- 172.21.15.107:0/1544757987 shutdown_connections 2024-04-25T00:18:44.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.273+0000 7f292c1ee640 1 -- 172.21.15.107:0/1544757987 wait complete. 2024-04-25T00:18:44.285 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:18:44.285 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:18:44.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.399+0000 7f95d7e38640 1 -- 172.21.15.107:0/2706992432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95d01032e0 msgr2=0x7f95d010b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:44.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.399+0000 7f95d7e38640 1 --2- 172.21.15.107:0/2706992432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95d01032e0 0x7f95d010b7c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009900 tx=0x7f95c002a510 comp rx=0 tx=0).stop 2024-04-25T00:18:44.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.399+0000 7f95d7e38640 1 -- 172.21.15.107:0/2706992432 shutdown_connections 2024-04-25T00:18:44.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.399+0000 7f95d7e38640 1 --2- 172.21.15.107:0/2706992432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95d010bd00 0x7f95d010e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.399+0000 7f95d7e38640 1 --2- 172.21.15.107:0/2706992432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95d01032e0 0x7f95d010b7c0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.399+0000 7f95d7e38640 1 --2- 172.21.15.107:0/2706992432 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95d01029c0 0x7f95d0102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.399+0000 7f95d7e38640 1 -- 172.21.15.107:0/2706992432 >> 172.21.15.107:0/2706992432 conn(0x7f95d0077740 msgr2=0x7f95d0077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:44.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.399+0000 7f95d7e38640 1 -- 172.21.15.107:0/2706992432 shutdown_connections 2024-04-25T00:18:44.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.399+0000 7f95d7e38640 1 -- 172.21.15.107:0/2706992432 wait complete. 2024-04-25T00:18:44.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.400+0000 7f95d7e38640 1 Processor -- start 2024-04-25T00:18:44.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.400+0000 7f95d7e38640 1 -- start start 2024-04-25T00:18:44.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.400+0000 7f95d7e38640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95d01029c0 0x7f95d0198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.400+0000 7f95d7e38640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95d01032e0 0x7f95d0198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.400+0000 7f95d7e38640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95d010bd00 0x7f95d019c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.400+0000 7f95d7e38640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f95d0111b50 con 0x7f95d01029c0 2024-04-25T00:18:44.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.400+0000 7f95d7e38640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f95d01119d0 con 0x7f95d010bd00 2024-04-25T00:18:44.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.400+0000 7f95d7e38640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f95d0111cd0 con 0x7f95d01032e0 2024-04-25T00:18:44.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.400+0000 7f95d5bd4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95d01029c0 0x7f95d0198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.400+0000 7f95d5bd4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95d01029c0 0x7f95d0198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41926/0 (socket says 172.21.15.107:41926) 2024-04-25T00:18:44.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.400+0000 7f95d53d3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95d01032e0 0x7f95d0198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.400+0000 7f95d5bd4640 1 -- 172.21.15.107:0/2635487808 learned_addr learned my addr 172.21.15.107:0/2635487808 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:44.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.400+0000 7f95d63d5640 1 --2- 172.21.15.107:0/2635487808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95d010bd00 0x7f95d019c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.401+0000 7f95d53d3640 1 -- 172.21.15.107:0/2635487808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95d010bd00 msgr2=0x7f95d019c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:44.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.401+0000 7f95d53d3640 1 --2- 172.21.15.107:0/2635487808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95d010bd00 0x7f95d019c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.401+0000 7f95d53d3640 1 -- 172.21.15.107:0/2635487808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95d01029c0 msgr2=0x7f95d0198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:44.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.401+0000 7f95d53d3640 1 --2- 172.21.15.107:0/2635487808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95d01029c0 0x7f95d0198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.401+0000 7f95d53d3640 1 -- 172.21.15.107:0/2635487808 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d019d000 con 0x7f95d01032e0 2024-04-25T00:18:44.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.401+0000 7f95d53d3640 1 --2- 172.21.15.107:0/2635487808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95d01032e0 0x7f95d0198550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f95c002aa30 tx=0x7f95c0003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:44.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.401+0000 7f95c6ffd640 1 -- 172.21.15.107:0/2635487808 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c003f070 con 0x7f95d01032e0 2024-04-25T00:18:44.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.401+0000 7f95d7e38640 1 -- 172.21.15.107:0/2635487808 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d019d290 con 0x7f95d01032e0 2024-04-25T00:18:44.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.401+0000 7f95c6ffd640 1 -- 172.21.15.107:0/2635487808 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95c0031c90 con 0x7f95d01032e0 2024-04-25T00:18:44.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.401+0000 7f95c6ffd640 1 -- 172.21.15.107:0/2635487808 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c0035030 con 0x7f95d01032e0 2024-04-25T00:18:44.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.401+0000 7f95d7e38640 1 -- 172.21.15.107:0/2635487808 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d0100540 con 0x7f95d01032e0 2024-04-25T00:18:44.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.404+0000 7f95dac22740 1 Processor -- start 2024-04-25T00:18:44.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.404+0000 7f95c6ffd640 1 -- 172.21.15.107:0/2635487808 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f95c0042070 con 0x7f95d01032e0 2024-04-25T00:18:44.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.405+0000 7f95dac22740 1 -- start start 2024-04-25T00:18:44.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.405+0000 7f95dac22740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55852b02b630 0x55852b02ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.405+0000 7f95dac22740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55852b02bf50 0x55852b0360e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.405+0000 7f95dac22740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b036620 0x55852b038a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.405+0000 7f95dac22740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55852af906b0 con 0x55852b036620 2024-04-25T00:18:44.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.405+0000 7f95dac22740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55852af38880 con 0x55852b02bf50 2024-04-25T00:18:44.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.405+0000 7f95dac22740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55852af8c670 con 0x55852b02b630 2024-04-25T00:18:44.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.405+0000 7f95d63d5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55852b02b630 0x55852b02ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.405+0000 7f95d63d5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55852b02b630 0x55852b02ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:47774/0 (socket says 172.21.15.107:47774) 2024-04-25T00:18:44.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.405+0000 7f95d63d5640 1 -- 172.21.15.107:0/3845219836 learned_addr learned my addr 172.21.15.107:0/3845219836 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:44.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.405+0000 7f95d53d3640 1 --2- 172.21.15.107:0/3845219836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b036620 0x55852b038a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.405+0000 7f95d5bd4640 1 --2- 172.21.15.107:0/3845219836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55852b02bf50 0x55852b0360e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.405+0000 7f95c6ffd640 1 --2- 172.21.15.107:0/2635487808 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f95bc0a2d80 0x7f95bc0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.406+0000 7f95d63d5640 1 -- 172.21.15.107:0/3845219836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55852b02bf50 msgr2=0x55852b0360e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:44.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.406+0000 7f95d63d5640 1 --2- 172.21.15.107:0/3845219836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55852b02bf50 0x55852b0360e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.406+0000 7f95d63d5640 1 -- 172.21.15.107:0/3845219836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b036620 msgr2=0x55852b038a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:44.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.406+0000 7f95d63d5640 1 --2- 172.21.15.107:0/3845219836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b036620 0x55852b038a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.406+0000 7f95d63d5640 1 -- 172.21.15.107:0/3845219836 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55852ad6e7d0 con 0x55852b02b630 2024-04-25T00:18:44.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.406+0000 7f95c6ffd640 1 -- 172.21.15.107:0/2635487808 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f95c00f0110 con 0x7f95d01032e0 2024-04-25T00:18:44.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.406+0000 7f95d63d5640 1 --2- 172.21.15.107:0/2635487808 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f95bc0a2d80 0x7f95bc0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.406+0000 7f95d63d5640 1 --2- 172.21.15.107:0/3845219836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55852b02b630 0x55852b02ba10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009f50 tx=0x7f95c8009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=880f5e9031d75cbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:44.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.406+0000 7f95bbfff640 1 -- 172.21.15.107:0/3845219836 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c8003dd0 con 0x55852b02b630 2024-04-25T00:18:44.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.406+0000 7f95bbfff640 1 -- 172.21.15.107:0/3845219836 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95c8004810 con 0x55852b02b630 2024-04-25T00:18:44.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.406+0000 7f95bbfff640 1 -- 172.21.15.107:0/3845219836 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c801b660 con 0x55852b02b630 2024-04-25T00:18:44.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.407+0000 7f95dac22740 1 -- 172.21.15.107:0/3845219836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55852b02b630 msgr2=0x55852b02ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:44.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.407+0000 7f95dac22740 1 --2- 172.21.15.107:0/3845219836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55852b02b630 0x55852b02ba10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009f50 tx=0x7f95c8009500 comp rx=0 tx=0).stop 2024-04-25T00:18:44.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.407+0000 7f95d63d5640 1 --2- 172.21.15.107:0/2635487808 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f95bc0a2d80 0x7f95bc0a5240 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009fd0 tx=0x7f95c801bdc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:44.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.407+0000 7f95dac22740 1 -- 172.21.15.107:0/3845219836 shutdown_connections 2024-04-25T00:18:44.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.407+0000 7f95dac22740 1 --2- 172.21.15.107:0/3845219836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b036620 0x55852b038a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.407+0000 7f95dac22740 1 --2- 172.21.15.107:0/3845219836 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55852b02bf50 0x55852b0360e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.407+0000 7f95dac22740 1 --2- 172.21.15.107:0/3845219836 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55852b02b630 0x55852b02ba10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.407+0000 7f95dac22740 1 -- 172.21.15.107:0/3845219836 >> 172.21.15.107:0/3845219836 conn(0x55852b027950 msgr2=0x55852b027d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:44.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.407+0000 7f95dac22740 1 -- 172.21.15.107:0/3845219836 shutdown_connections 2024-04-25T00:18:44.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.407+0000 7f95dac22740 1 -- 172.21.15.107:0/3845219836 wait complete. 2024-04-25T00:18:44.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.408+0000 7f95dac22740 1 -- start start 2024-04-25T00:18:44.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.408+0000 7f95dac22740 1 Processor -- start 2024-04-25T00:18:44.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.408+0000 7f95dac22740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55852b02b630 0x55852b0334f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.408+0000 7f95dac22740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b02bf50 0x55852b033a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.408+0000 7f95dac22740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55852b036620 0x55852b033f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.408+0000 7f95dac22740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55852b0393b0 con 0x55852b02bf50 2024-04-25T00:18:44.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.408+0000 7f95dac22740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55852af4bf50 con 0x55852b02b630 2024-04-25T00:18:44.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.408+0000 7f95dac22740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55852b039530 con 0x55852b036620 2024-04-25T00:18:44.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.408+0000 7f95dac22740 10 client.0 _ll_register_callbacks cb 0x55852af87000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:44.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.408+0000 7f95d53d3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b02bf50 0x55852b033a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.408+0000 7f95d53d3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b02bf50 0x55852b033a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41958/0 (socket says 172.21.15.107:41958) 2024-04-25T00:18:44.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.408+0000 7f95d53d3640 1 -- 172.21.15.107:0/2199557905 learned_addr learned my addr 172.21.15.107:0/2199557905 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:44.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.408+0000 7f95d5bd4640 1 --2- 172.21.15.107:0/2199557905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55852b02b630 0x55852b0334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.408+0000 7f95d63d5640 1 --2- 172.21.15.107:0/2199557905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55852b036620 0x55852b033f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.409+0000 7f95d53d3640 1 -- 172.21.15.107:0/2199557905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55852b036620 msgr2=0x55852b033f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:44.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.409+0000 7f95d53d3640 1 --2- 172.21.15.107:0/2199557905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55852b036620 0x55852b033f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.409+0000 7f95d53d3640 1 -- 172.21.15.107:0/2199557905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55852b02b630 msgr2=0x55852b0334f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:44.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.409+0000 7f95d53d3640 1 --2- 172.21.15.107:0/2199557905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55852b02b630 0x55852b0334f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.409+0000 7f95d53d3640 1 -- 172.21.15.107:0/2199557905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55852aca43f0 con 0x55852b02bf50 2024-04-25T00:18:44.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.409+0000 7f95d63d5640 1 --2- 172.21.15.107:0/2199557905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55852b036620 0x55852b033f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:18:44.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.409+0000 7f95d53d3640 1 --2- 172.21.15.107:0/2199557905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b02bf50 0x55852b033a30 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f95c0000c00 tx=0x7f95c00f3670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6388fa4addae2760 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:44.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.409+0000 7f95b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:18:44.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.409+0000 7f95b97fa640 1 -- 172.21.15.107:0/2199557905 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c0049070 con 0x55852b02bf50 2024-04-25T00:18:44.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.409+0000 7f95b97fa640 1 -- 172.21.15.107:0/2199557905 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f95c00b8980 con 0x55852b02bf50 2024-04-25T00:18:44.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.409+0000 7f95dac22740 1 -- 172.21.15.107:0/2199557905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95c8004810 con 0x55852b02bf50 2024-04-25T00:18:44.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.409+0000 7f95b97fa640 1 -- 172.21.15.107:0/2199557905 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f95c00bbe50 con 0x55852b02bf50 2024-04-25T00:18:44.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.409+0000 7f95dac22740 10 client.16170 fetch_fsmap learned FSMap version 28 2024-04-25T00:18:44.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.409+0000 7f95dac22740 1 -- 172.21.15.107:0/2199557905 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55852ac640f0 con 0x55852b02bf50 2024-04-25T00:18:44.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.410+0000 7f95b97fa640 1 -- 172.21.15.107:0/2199557905 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95c8004810 con 0x55852b02bf50 2024-04-25T00:18:44.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.410+0000 7f95b97fa640 1 -- 172.21.15.107:0/2199557905 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f95c00f3c30 con 0x55852b02bf50 2024-04-25T00:18:44.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.410+0000 7f95dac22740 10 client.16170 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:18:44.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.410+0000 7f95dac22740 10 client.16170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:18:44.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.410+0000 7f95dac22740 20 client.16170 populate_metadata read hostname 'smithi107' 2024-04-25T00:18:44.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.410+0000 7f95dac22740 1 --2- 172.21.15.107:0/2199557905 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55852b046800 0x55852b048c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.410+0000 7f95dac22740 4 client.16170 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:18:44.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.410+0000 7f95dac22740 1 -- 172.21.15.107:0/2199557905 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55852af9dbf0 con 0x55852b046800 2024-04-25T00:18:44.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.410+0000 7f95d5bd4640 1 --2- 172.21.15.107:0/2199557905 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55852b046800 0x55852b048c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.410+0000 7f95b97fa640 10 client.16170 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:44.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.410+0000 7f95d5bd4640 1 --2- 172.21.15.107:0/2199557905 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55852b046800 0x55852b048c60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c5158f2f009c698 server_cookie=ae994961e8df1c9d in_seq=0 out_seq=0 2024-04-25T00:18:44.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.414+0000 7f95b97fa640 1 -- 172.21.15.107:0/2199557905 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55852af9dbf0 con 0x55852b046800 2024-04-25T00:18:44.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.414+0000 7f95b97fa640 10 client.16170 handle_command_reply: tid=0 2024-04-25T00:18:44.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.414+0000 7f95dac22740 1 client.16170 shutdown 2024-04-25T00:18:44.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.414+0000 7f95dac22740 1 -- 172.21.15.107:0/2199557905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b02bf50 msgr2=0x55852b033a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:44.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.414+0000 7f95dac22740 1 --2- 172.21.15.107:0/2199557905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b02bf50 0x55852b033a30 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f95c0000c00 tx=0x7f95c00f3670 comp rx=0 tx=0).stop 2024-04-25T00:18:44.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.414+0000 7f95dac22740 1 -- 172.21.15.107:0/2199557905 shutdown_connections 2024-04-25T00:18:44.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.414+0000 7f95dac22740 1 --2- 172.21.15.107:0/2199557905 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55852b046800 0x55852b048c60 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.414+0000 7f95dac22740 1 --2- 172.21.15.107:0/2199557905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55852b036620 0x55852b033f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.414+0000 7f95dac22740 1 --2- 172.21.15.107:0/2199557905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b02bf50 0x55852b033a30 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.414+0000 7f95b97fa640 0 client.16170 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:44.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.414+0000 7f95dac22740 1 --2- 172.21.15.107:0/2199557905 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55852b02b630 0x55852b0334f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.414+0000 7f95dac22740 1 -- 172.21.15.107:0/2199557905 >> 172.21.15.107:0/2199557905 conn(0x55852b027950 msgr2=0x55852b02d750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:44.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.415+0000 7f95dac22740 1 -- 172.21.15.107:0/2199557905 shutdown_connections 2024-04-25T00:18:44.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.415+0000 7f95dac22740 1 -- 172.21.15.107:0/2199557905 wait complete. 2024-04-25T00:18:44.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.415+0000 7f95dac22740 20 client.16170 trim_cache size 0 max 16384 2024-04-25T00:18:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.434+0000 7f95dac22740 1 Processor -- start 2024-04-25T00:18:44.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.434+0000 7f95dac22740 1 -- start start 2024-04-25T00:18:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.434+0000 7f95dac22740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55852b02b630 0x55852b042270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.434+0000 7f95dac22740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55852b02bf50 0x55852b035bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.434+0000 7f95dac22740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b036620 0x55852b032470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.434+0000 7f95dac22740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f95d01af270 con 0x55852b036620 2024-04-25T00:18:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.434+0000 7f95dac22740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f95d01aca80 con 0x55852b02bf50 2024-04-25T00:18:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.434+0000 7f95dac22740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f95d01aa8d0 con 0x55852b02b630 2024-04-25T00:18:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.434+0000 7f95d63d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b036620 0x55852b032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.434+0000 7f95d63d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b036620 0x55852b032470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41968/0 (socket says 172.21.15.107:41968) 2024-04-25T00:18:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.434+0000 7f95d63d5640 1 -- 172.21.15.107:0/4113897437 learned_addr learned my addr 172.21.15.107:0/4113897437 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.434+0000 7f95d5bd4640 1 --2- 172.21.15.107:0/4113897437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55852b02b630 0x55852b042270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.434+0000 7f95d53d3640 1 --2- 172.21.15.107:0/4113897437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55852b02bf50 0x55852b035bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.434+0000 7f95d63d5640 1 -- 172.21.15.107:0/4113897437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55852b02b630 msgr2=0x55852b042270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.434+0000 7f95d63d5640 1 --2- 172.21.15.107:0/4113897437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55852b02b630 0x55852b042270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.434+0000 7f95d63d5640 1 -- 172.21.15.107:0/4113897437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55852b02bf50 msgr2=0x55852b035bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:44.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.434+0000 7f95d63d5640 1 --2- 172.21.15.107:0/4113897437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55852b02bf50 0x55852b035bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.434+0000 7f95d63d5640 1 -- 172.21.15.107:0/4113897437 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55852b0492e0 con 0x55852b036620 2024-04-25T00:18:44.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.435+0000 7f95d63d5640 1 --2- 172.21.15.107:0/4113897437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b036620 0x55852b032470 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009ef0 tx=0x7f95c802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99cb242ee83eebe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:44.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.435+0000 7f95bb7fe640 1 -- 172.21.15.107:0/4113897437 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c8003b50 con 0x55852b036620 2024-04-25T00:18:44.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.435+0000 7f95bb7fe640 1 -- 172.21.15.107:0/4113897437 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95c8037030 con 0x55852b036620 2024-04-25T00:18:44.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.435+0000 7f95bb7fe640 1 -- 172.21.15.107:0/4113897437 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c8026c10 con 0x55852b036620 2024-04-25T00:18:44.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.436+0000 7f95dac22740 1 -- 172.21.15.107:0/4113897437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b036620 msgr2=0x55852b032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:44.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.436+0000 7f95dac22740 1 --2- 172.21.15.107:0/4113897437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b036620 0x55852b032470 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009ef0 tx=0x7f95c802c000 comp rx=0 tx=0).stop 2024-04-25T00:18:44.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.436+0000 7f95dac22740 1 -- 172.21.15.107:0/4113897437 shutdown_connections 2024-04-25T00:18:44.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.436+0000 7f95dac22740 1 --2- 172.21.15.107:0/4113897437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b036620 0x55852b032470 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.436+0000 7f95dac22740 1 --2- 172.21.15.107:0/4113897437 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55852b02bf50 0x55852b035bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.436+0000 7f95dac22740 1 --2- 172.21.15.107:0/4113897437 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55852b02b630 0x55852b042270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.436+0000 7f95dac22740 1 -- 172.21.15.107:0/4113897437 >> 172.21.15.107:0/4113897437 conn(0x55852b027950 msgr2=0x55852b026af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:44.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.436+0000 7f95dac22740 1 -- 172.21.15.107:0/4113897437 shutdown_connections 2024-04-25T00:18:44.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.436+0000 7f95dac22740 1 -- 172.21.15.107:0/4113897437 wait complete. 2024-04-25T00:18:44.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.436+0000 7f95dac22740 1 -- start start 2024-04-25T00:18:44.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.437+0000 7f95dac22740 1 Processor -- start 2024-04-25T00:18:44.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.437+0000 7f95dac22740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55852b02b630 0x55852b03e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.437+0000 7f95dac22740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b02bf50 0x55852b02fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.437+0000 7f95dac22740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55852b036620 0x55852b030320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.437+0000 7f95dac22740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f95d01aaf40 con 0x55852b02bf50 2024-04-25T00:18:44.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.437+0000 7f95dac22740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f95d01ac3b0 con 0x55852b02b630 2024-04-25T00:18:44.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.437+0000 7f95dac22740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f95d01ab610 con 0x55852b036620 2024-04-25T00:18:44.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.437+0000 7f95dac22740 10 client.0 _ll_register_callbacks cb 0x7f95c00f3830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:44.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.437+0000 7f95d53d3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b02bf50 0x55852b02fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.437+0000 7f95d53d3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b02bf50 0x55852b02fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41972/0 (socket says 172.21.15.107:41972) 2024-04-25T00:18:44.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.437+0000 7f95d53d3640 1 -- 172.21.15.107:0/203775684 learned_addr learned my addr 172.21.15.107:0/203775684 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:18:44.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.437+0000 7f95d63d5640 1 --2- 172.21.15.107:0/203775684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55852b036620 0x55852b030320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.437+0000 7f95d5bd4640 1 --2- 172.21.15.107:0/203775684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55852b02b630 0x55852b03e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.437+0000 7f95d53d3640 1 -- 172.21.15.107:0/203775684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55852b036620 msgr2=0x55852b030320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:44.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.437+0000 7f95d53d3640 1 --2- 172.21.15.107:0/203775684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55852b036620 0x55852b030320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.437+0000 7f95d53d3640 1 -- 172.21.15.107:0/203775684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55852b02b630 msgr2=0x55852b03e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:44.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.437+0000 7f95d53d3640 1 --2- 172.21.15.107:0/203775684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55852b02b630 0x55852b03e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.437+0000 7f95d53d3640 1 -- 172.21.15.107:0/203775684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55852b0309c0 con 0x55852b02bf50 2024-04-25T00:18:44.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.438+0000 7f95d53d3640 1 --2- 172.21.15.107:0/203775684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b02bf50 0x55852b02fde0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f95c0006010 tx=0x7f95c00f3de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab81c8197ceba0fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:44.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.438+0000 7f95b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:18:44.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.438+0000 7f95b97fa640 1 -- 172.21.15.107:0/203775684 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c0049070 con 0x55852b02bf50 2024-04-25T00:18:44.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.438+0000 7f95b97fa640 1 -- 172.21.15.107:0/203775684 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f95c00edbe0 con 0x55852b02bf50 2024-04-25T00:18:44.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.438+0000 7f95dac22740 1 -- 172.21.15.107:0/203775684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95c8037030 con 0x55852b02bf50 2024-04-25T00:18:44.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.438+0000 7f95b97fa640 1 -- 172.21.15.107:0/203775684 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f95c00bbc70 con 0x55852b02bf50 2024-04-25T00:18:44.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.438+0000 7f95dac22740 10 client.16176 fetch_fsmap learned FSMap version 28 2024-04-25T00:18:44.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.438+0000 7f95dac22740 1 -- 172.21.15.107:0/203775684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55852b0acc80 con 0x55852b02bf50 2024-04-25T00:18:44.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.438+0000 7f95b97fa640 1 -- 172.21.15.107:0/203775684 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95c8037030 con 0x55852b02bf50 2024-04-25T00:18:44.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.438+0000 7f95b97fa640 1 -- 172.21.15.107:0/203775684 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f95c00dc0e0 con 0x55852b02bf50 2024-04-25T00:18:44.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.438+0000 7f95dac22740 10 client.16176 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:18:44.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.438+0000 7f95dac22740 10 client.16176 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:18:44.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.438+0000 7f95dac22740 20 client.16176 populate_metadata read hostname 'smithi107' 2024-04-25T00:18:44.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.439+0000 7f95dac22740 1 --2- 172.21.15.107:0/203775684 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55852b0ad210 0x55852b0af630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:44.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.439+0000 7f95dac22740 4 client.16176 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:18:44.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.439+0000 7f95dac22740 1 -- 172.21.15.107:0/203775684 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55852af97f40 con 0x55852b0ad210 2024-04-25T00:18:44.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.439+0000 7f95d5bd4640 1 --2- 172.21.15.107:0/203775684 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55852b0ad210 0x55852b0af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:44.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.439+0000 7f95b97fa640 10 client.16176 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:44.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.439+0000 7f95d5bd4640 1 --2- 172.21.15.107:0/203775684 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55852b0ad210 0x55852b0af630 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc3c4cebc4ced516 server_cookie=f9174168e73574fe in_seq=0 out_seq=0 2024-04-25T00:18:44.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.441+0000 7f95b97fa640 1 -- 172.21.15.107:0/203775684 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55852af97f40 con 0x55852b0ad210 2024-04-25T00:18:44.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.441+0000 7f95b97fa640 10 client.16176 handle_command_reply: tid=0 2024-04-25T00:18:44.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.441+0000 7f95dac22740 1 client.16176 shutdown 2024-04-25T00:18:44.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.441+0000 7f95dac22740 1 -- 172.21.15.107:0/203775684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b02bf50 msgr2=0x55852b02fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:44.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.441+0000 7f95dac22740 1 --2- 172.21.15.107:0/203775684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b02bf50 0x55852b02fde0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f95c0006010 tx=0x7f95c00f3de0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.441+0000 7f95dac22740 1 -- 172.21.15.107:0/203775684 shutdown_connections 2024-04-25T00:18:44.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.441+0000 7f95dac22740 1 --2- 172.21.15.107:0/203775684 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55852b0ad210 0x55852b0af630 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.441+0000 7f95dac22740 1 --2- 172.21.15.107:0/203775684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55852b036620 0x55852b030320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.441+0000 7f95dac22740 1 --2- 172.21.15.107:0/203775684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55852b02bf50 0x55852b02fde0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.441+0000 7f95b97fa640 0 client.16176 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:18:44.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.441+0000 7f95dac22740 1 --2- 172.21.15.107:0/203775684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55852b02b630 0x55852b03e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.441+0000 7f95dac22740 1 -- 172.21.15.107:0/203775684 >> 172.21.15.107:0/203775684 conn(0x55852b027950 msgr2=0x55852b02cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:44.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.441+0000 7f95dac22740 1 -- 172.21.15.107:0/203775684 shutdown_connections 2024-04-25T00:18:44.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.441+0000 7f95dac22740 1 -- 172.21.15.107:0/203775684 wait complete. 2024-04-25T00:18:44.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.441+0000 7f95dac22740 20 client.16176 trim_cache size 0 max 16384 2024-04-25T00:18:44.446 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:18:44.446 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "scrub active (133 inodes in the stack)", 2024-04-25T00:18:44.446 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-04-25T00:18:44.447 INFO:teuthology.orchestra.run.smithi107.stdout: "ebf9cfce-4d97-4f4c-9fc7-b3cc448508ff": { 2024-04-25T00:18:44.447 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-04-25T00:18:44.447 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "ebf9cfce-4d97-4f4c-9fc7-b3cc448508ff", 2024-04-25T00:18:44.447 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-04-25T00:18:44.447 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:18:44.447 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:18:44.447 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:18:44.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.442+0000 7f95d7e38640 1 -- 172.21.15.107:0/2635487808 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f95bc0a2d80 msgr2=0x7f95bc0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:44.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.442+0000 7f95d7e38640 1 --2- 172.21.15.107:0/2635487808 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f95bc0a2d80 0x7f95bc0a5240 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009fd0 tx=0x7f95c801bdc0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.442+0000 7f95d7e38640 1 -- 172.21.15.107:0/2635487808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95d01032e0 msgr2=0x7f95d0198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:44.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.442+0000 7f95d7e38640 1 --2- 172.21.15.107:0/2635487808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95d01032e0 0x7f95d0198550 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f95c002aa30 tx=0x7f95c0003b20 comp rx=0 tx=0).stop 2024-04-25T00:18:44.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.442+0000 7f95d7e38640 1 -- 172.21.15.107:0/2635487808 shutdown_connections 2024-04-25T00:18:44.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.442+0000 7f95d7e38640 1 --2- 172.21.15.107:0/2635487808 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f95bc0a2d80 0x7f95bc0a5240 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.442+0000 7f95d7e38640 1 --2- 172.21.15.107:0/2635487808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f95d010bd00 0x7f95d019c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.442+0000 7f95d7e38640 1 --2- 172.21.15.107:0/2635487808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95d01032e0 0x7f95d0198550 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.442+0000 7f95d7e38640 1 --2- 172.21.15.107:0/2635487808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95d01029c0 0x7f95d0198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:44.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.442+0000 7f95d7e38640 1 -- 172.21.15.107:0/2635487808 >> 172.21.15.107:0/2635487808 conn(0x7f95d0077740 msgr2=0x7f95d00fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:44.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.442+0000 7f95d7e38640 1 -- 172.21.15.107:0/2635487808 shutdown_connections 2024-04-25T00:18:44.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:18:44.442+0000 7f95d7e38640 1 -- 172.21.15.107:0/2635487808 wait complete. 2024-04-25T00:18:44.454 INFO:tasks.cephfs.filesystem:scrub status for tag:ebf9cfce-4d97-4f4c-9fc7-b3cc448508ff - {'path': '/', 'tag': 'ebf9cfce-4d97-4f4c-9fc7-b3cc448508ff', 'options': 'recursive,force'} 2024-04-25T00:18:45.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:44 smithi170 ceph-mon[32807]: scrub queued for path: / 2024-04-25T00:18:45.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:44 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:45.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:44 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:45.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:44 smithi107 ceph-mon[29153]: scrub queued for path: / 2024-04-25T00:18:45.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:44 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:45.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:44 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:45.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:44 smithi195 ceph-mon[32833]: scrub queued for path: / 2024-04-25T00:18:45.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:44 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:45.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:44 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: pgmap v72: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 140 MiB/s rd, 161 MiB/s wr, 287 op/s 2024-04-25T00:18:46.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.188 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.189 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.189 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.189 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.189 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.189 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.189 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.189 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.189 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.189 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.189 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.189 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.190 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.190 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.190 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.190 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.190 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.190 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.190 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.190 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.190 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.190 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.190 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.190 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.190 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.191 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.191 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.191 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.191 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.191 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.191 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.191 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.191 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.191 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.191 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.191 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.191 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.191 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.192 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.192 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.192 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.192 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.192 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.192 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.192 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.192 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:46.192 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:45 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: pgmap v72: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 140 MiB/s rd, 161 MiB/s wr, 287 op/s 2024-04-25T00:18:46.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.239 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.239 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.239 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.239 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.239 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.239 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.239 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.239 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.239 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.240 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.240 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.240 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.240 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.240 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.240 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.240 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.240 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.240 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.240 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.240 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.240 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.240 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.241 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.241 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.241 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.241 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.241 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.241 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.241 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.241 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.241 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.241 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.241 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.241 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.241 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.241 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.242 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.242 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.242 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.242 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:46.242 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:45 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: pgmap v72: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 140 MiB/s rd, 161 MiB/s wr, 287 op/s 2024-04-25T00:18:46.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.344 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.345 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.346 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.347 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.347 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.347 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.347 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.347 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.347 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.347 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:46.347 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:46.347 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:45 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:47.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:47.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:47.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:47.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:47.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:47.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:47.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:47.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:47.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:47.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:47.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:47.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:47.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:47.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:47.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:47.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:47.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:47.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:47.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:47.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:47.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:47.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:47.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:47.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:46 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:47.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:47.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:47.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:47.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:47.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:47.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:47.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:47.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:47.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:47.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:47.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:47.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:47.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:47.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:47.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:47.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:47.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:47.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:47.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:47.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:47.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:47.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:47.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:47.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:46 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:47.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:47.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:47.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:47.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:47.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:47.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:47.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:47.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:47.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:47.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:47.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:47.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:47.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:47.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:47.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:47.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:47.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:47.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:47.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:47.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:47.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:47.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:47.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:47.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:47.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:46 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:47 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:48.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:47 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:47 smithi107 ceph-mon[29153]: pgmap v73: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 140 MiB/s rd, 161 MiB/s wr, 287 op/s 2024-04-25T00:18:48.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:47 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:48.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:47 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:47 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:48.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:47 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:47 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:48.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:47 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:47 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:48.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:47 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:47 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:48.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:47 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:47 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:48.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:47 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:47 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:48.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:47 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:47 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:48.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:47 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:47 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:48.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:47 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:47 smithi195 ceph-mon[32833]: pgmap v73: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 140 MiB/s rd, 161 MiB/s wr, 287 op/s 2024-04-25T00:18:48.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:47 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:48.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:47 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:47 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:48.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:47 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:47 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:48.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:47 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:47 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:48.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:47 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:47 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:48.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:47 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:47 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:48.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:47 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:47 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:48.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:47 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:47 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:48.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:47 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:47 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:48.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:47 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:47 smithi170 ceph-mon[32807]: pgmap v73: 129 pgs: 129 active+clean; 2.1 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 140 MiB/s rd, 161 MiB/s wr, 287 op/s 2024-04-25T00:18:48.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:47 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:48.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:47 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:47 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:48.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:47 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:47 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:48.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:47 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:47 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:48.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:47 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:47 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:48.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:47 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:47 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:48.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:47 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:47 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:48.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:47 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:48.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:47 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:48.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:47 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:49.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:48 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:49.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:48 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:49.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:48 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:49.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:48 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:49.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:48 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:49.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:48 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:49.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:48 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:49.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:48 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:49.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:48 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:49.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:48 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:49.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:48 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:18:49.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:48 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:49.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:48 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:49.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:48 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:49.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:48 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:49.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:48 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:49.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:48 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:49.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:48 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:49.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:48 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:49.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:48 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:49.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:48 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:49.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:48 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:49.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:48 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:49.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:48 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:18:49.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:48 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:49.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:48 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:49.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:48 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:49.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:48 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:49.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:48 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:49.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:48 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:49.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:48 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:49.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:48 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:49.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:48 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:49.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:48 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:49.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:48 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:49.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:48 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:49.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:48 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:18:49.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:48 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:49.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:48 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:18:50.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:49 smithi107 ceph-mon[29153]: pgmap v74: 129 pgs: 129 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 140 MiB/s rd, 162 MiB/s wr, 375 op/s 2024-04-25T00:18:50.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:49 smithi195 ceph-mon[32833]: pgmap v74: 129 pgs: 129 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 140 MiB/s rd, 162 MiB/s wr, 375 op/s 2024-04-25T00:18:50.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:49 smithi170 ceph-mon[32807]: pgmap v74: 129 pgs: 129 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 140 MiB/s rd, 162 MiB/s wr, 375 op/s 2024-04-25T00:18:52.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:51 smithi107 ceph-mon[29153]: pgmap v75: 129 pgs: 129 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 40 MiB/s wr, 181 op/s 2024-04-25T00:18:52.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:51 smithi195 ceph-mon[32833]: pgmap v75: 129 pgs: 129 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 40 MiB/s wr, 181 op/s 2024-04-25T00:18:52.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:51 smithi170 ceph-mon[32807]: pgmap v75: 129 pgs: 129 active+clean; 1.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 MiB/s rd, 40 MiB/s wr, 181 op/s 2024-04-25T00:18:52.964 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T00:18:52.964 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T00:18:54.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:53 smithi107 ceph-mon[29153]: pgmap v76: 129 pgs: 129 active+clean; 409 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 MiB/s rd, 40 MiB/s wr, 270 op/s 2024-04-25T00:18:54.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:53 smithi195 ceph-mon[32833]: pgmap v76: 129 pgs: 129 active+clean; 409 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 MiB/s rd, 40 MiB/s wr, 270 op/s 2024-04-25T00:18:54.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:53 smithi170 ceph-mon[32807]: pgmap v76: 129 pgs: 129 active+clean; 409 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 MiB/s rd, 40 MiB/s wr, 270 op/s 2024-04-25T00:18:56.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:55 smithi107 ceph-mon[29153]: pgmap v77: 129 pgs: 129 active+clean; 409 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 1.0 MiB/s wr, 177 op/s 2024-04-25T00:18:56.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:55 smithi195 ceph-mon[32833]: pgmap v77: 129 pgs: 129 active+clean; 409 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 1.0 MiB/s wr, 177 op/s 2024-04-25T00:18:56.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:55 smithi170 ceph-mon[32807]: pgmap v77: 129 pgs: 129 active+clean; 409 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 1.0 MiB/s wr, 177 op/s 2024-04-25T00:18:58.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:57 smithi107 ceph-mon[29153]: pgmap v78: 129 pgs: 129 active+clean; 409 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 1.0 MiB/s wr, 177 op/s 2024-04-25T00:18:58.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:57 smithi195 ceph-mon[32833]: pgmap v78: 129 pgs: 129 active+clean; 409 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 1.0 MiB/s wr, 177 op/s 2024-04-25T00:18:58.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:57 smithi170 ceph-mon[32807]: pgmap v78: 129 pgs: 129 active+clean; 409 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 1.0 MiB/s wr, 177 op/s 2024-04-25T00:19:00.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:18:59 smithi107 ceph-mon[29153]: pgmap v79: 129 pgs: 129 active+clean; 177 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 285 op/s 2024-04-25T00:19:00.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:18:59 smithi195 ceph-mon[32833]: pgmap v79: 129 pgs: 129 active+clean; 177 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 285 op/s 2024-04-25T00:19:00.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:18:59 smithi170 ceph-mon[32807]: pgmap v79: 129 pgs: 129 active+clean; 177 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 285 op/s 2024-04-25T00:19:02.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:01 smithi107 ceph-mon[29153]: pgmap v80: 129 pgs: 129 active+clean; 177 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 197 op/s 2024-04-25T00:19:02.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:01 smithi195 ceph-mon[32833]: pgmap v80: 129 pgs: 129 active+clean; 177 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 197 op/s 2024-04-25T00:19:02.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:01 smithi170 ceph-mon[32807]: pgmap v80: 129 pgs: 129 active+clean; 177 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 197 op/s 2024-04-25T00:19:03.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:02 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:19:03.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:02 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:19:03.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:02 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:19:04.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:03 smithi107 ceph-mon[29153]: pgmap v81: 129 pgs: 129 active+clean; 185 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 353 op/s 2024-04-25T00:19:04.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:03 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:19:04.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:03 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:19:04.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:03 smithi195 ceph-mon[32833]: pgmap v81: 129 pgs: 129 active+clean; 185 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 353 op/s 2024-04-25T00:19:04.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:03 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:19:04.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:03 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:19:04.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:03 smithi170 ceph-mon[32807]: pgmap v81: 129 pgs: 129 active+clean; 185 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 353 op/s 2024-04-25T00:19:04.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:03 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:19:04.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:03 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:19:06.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:05 smithi107 ceph-mon[29153]: pgmap v82: 129 pgs: 129 active+clean; 185 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 264 op/s 2024-04-25T00:19:06.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:05 smithi195 ceph-mon[32833]: pgmap v82: 129 pgs: 129 active+clean; 185 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 264 op/s 2024-04-25T00:19:06.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:05 smithi170 ceph-mon[32807]: pgmap v82: 129 pgs: 129 active+clean; 185 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 264 op/s 2024-04-25T00:19:08.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:07 smithi107 ceph-mon[29153]: pgmap v83: 129 pgs: 129 active+clean; 185 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 264 op/s 2024-04-25T00:19:08.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:07 smithi195 ceph-mon[32833]: pgmap v83: 129 pgs: 129 active+clean; 185 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 264 op/s 2024-04-25T00:19:08.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:07 smithi170 ceph-mon[32807]: pgmap v83: 129 pgs: 129 active+clean; 185 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 264 op/s 2024-04-25T00:19:10.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:09 smithi107 ceph-mon[29153]: pgmap v84: 129 pgs: 129 active+clean; 195 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 396 op/s 2024-04-25T00:19:10.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:09 smithi195 ceph-mon[32833]: pgmap v84: 129 pgs: 129 active+clean; 195 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 396 op/s 2024-04-25T00:19:10.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:09 smithi170 ceph-mon[32807]: pgmap v84: 129 pgs: 129 active+clean; 195 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 396 op/s 2024-04-25T00:19:12.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:11 smithi195 ceph-mon[32833]: pgmap v85: 129 pgs: 129 active+clean; 195 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 288 op/s 2024-04-25T00:19:12.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:11 smithi170 ceph-mon[32807]: pgmap v85: 129 pgs: 129 active+clean; 195 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 288 op/s 2024-04-25T00:19:12.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:11 smithi107 ceph-mon[29153]: pgmap v85: 129 pgs: 129 active+clean; 195 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 288 op/s 2024-04-25T00:19:14.237 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:13 smithi170 ceph-mon[32807]: pgmap v86: 129 pgs: 129 active+clean; 206 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 435 op/s 2024-04-25T00:19:14.285 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:13 smithi107 ceph-mon[29153]: pgmap v86: 129 pgs: 129 active+clean; 206 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 435 op/s 2024-04-25T00:19:14.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:13 smithi195 ceph-mon[32833]: pgmap v86: 129 pgs: 129 active+clean; 206 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 435 op/s 2024-04-25T00:19:14.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:19:14.455 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:19:14.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.582+0000 7f7e87c29640 1 -- 172.21.15.107:0/3265111282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e80100000 msgr2=0x7f7e801003e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:14.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.582+0000 7f7e87c29640 1 --2- 172.21.15.107:0/3265111282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e80100000 0x7f7e801003e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c009900 tx=0x7f7e7c02a410 comp rx=0 tx=0).stop 2024-04-25T00:19:14.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.582+0000 7f7e87c29640 1 -- 172.21.15.107:0/3265111282 shutdown_connections 2024-04-25T00:19:14.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.582+0000 7f7e87c29640 1 --2- 172.21.15.107:0/3265111282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e801050d0 0x7f7e8010cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.582+0000 7f7e87c29640 1 --2- 172.21.15.107:0/3265111282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e801009b0 0x7f7e801049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.582+0000 7f7e87c29640 1 --2- 172.21.15.107:0/3265111282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e80100000 0x7f7e801003e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.582+0000 7f7e87c29640 1 -- 172.21.15.107:0/3265111282 >> 172.21.15.107:0/3265111282 conn(0x7f7e80076c00 msgr2=0x7f7e800fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:14.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.582+0000 7f7e87c29640 1 -- 172.21.15.107:0/3265111282 shutdown_connections 2024-04-25T00:19:14.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.582+0000 7f7e87c29640 1 -- 172.21.15.107:0/3265111282 wait complete. 2024-04-25T00:19:14.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.583+0000 7f7e87c29640 1 Processor -- start 2024-04-25T00:19:14.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.583+0000 7f7e87c29640 1 -- start start 2024-04-25T00:19:14.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.583+0000 7f7e87c29640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e80100000 0x7f7e80198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:14.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.583+0000 7f7e87c29640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e801050d0 0x7f7e80198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:14.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.583+0000 7f7e87c29640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e8019c9d0 0x7f7e8019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:14.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.583+0000 7f7e87c29640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8010f950 con 0x7f7e8019c9d0 2024-04-25T00:19:14.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.583+0000 7f7e87c29640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8010f7d0 con 0x7f7e801050d0 2024-04-25T00:19:14.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.583+0000 7f7e87c29640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8010fad0 con 0x7f7e80100000 2024-04-25T00:19:14.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.583+0000 7f7e861c6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e8019c9d0 0x7f7e8019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:14.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.583+0000 7f7e861c6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e8019c9d0 0x7f7e8019ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40016/0 (socket says 172.21.15.107:40016) 2024-04-25T00:19:14.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.583+0000 7f7e861c6640 1 -- 172.21.15.107:0/2086134405 learned_addr learned my addr 172.21.15.107:0/2086134405 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:14.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.583+0000 7f7e851c4640 1 --2- 172.21.15.107:0/2086134405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e801050d0 0x7f7e80198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:14.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.583+0000 7f7e859c5640 1 --2- 172.21.15.107:0/2086134405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e80100000 0x7f7e80198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:14.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.583+0000 7f7e861c6640 1 -- 172.21.15.107:0/2086134405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e80100000 msgr2=0x7f7e80198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:14.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.583+0000 7f7e861c6640 1 --2- 172.21.15.107:0/2086134405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e80100000 0x7f7e80198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.583+0000 7f7e861c6640 1 -- 172.21.15.107:0/2086134405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e801050d0 msgr2=0x7f7e80198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:14.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.583+0000 7f7e861c6640 1 --2- 172.21.15.107:0/2086134405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e801050d0 0x7f7e80198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.583+0000 7f7e861c6640 1 -- 172.21.15.107:0/2086134405 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8019d4c0 con 0x7f7e8019c9d0 2024-04-25T00:19:14.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.583+0000 7f7e861c6640 1 --2- 172.21.15.107:0/2086134405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e8019c9d0 0x7f7e8019ce80 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800de10 tx=0x7f7e7800b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:14.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.583+0000 7f7e76ffd640 1 -- 172.21.15.107:0/2086134405 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78003b70 con 0x7f7e8019c9d0 2024-04-25T00:19:14.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.584+0000 7f7e76ffd640 1 -- 172.21.15.107:0/2086134405 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e78003d10 con 0x7f7e8019c9d0 2024-04-25T00:19:14.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.584+0000 7f7e87c29640 1 -- 172.21.15.107:0/2086134405 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e8019d7b0 con 0x7f7e8019c9d0 2024-04-25T00:19:14.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.584+0000 7f7e76ffd640 1 -- 172.21.15.107:0/2086134405 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7800eba0 con 0x7f7e8019c9d0 2024-04-25T00:19:14.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.584+0000 7f7e87c29640 1 -- 172.21.15.107:0/2086134405 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e801a4fa0 con 0x7f7e8019c9d0 2024-04-25T00:19:14.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.586+0000 7f7e76ffd640 1 -- 172.21.15.107:0/2086134405 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7e78004000 con 0x7f7e8019c9d0 2024-04-25T00:19:14.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.586+0000 7f7e8aa0c740 1 Processor -- start 2024-04-25T00:19:14.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.586+0000 7f7e76ffd640 1 --2- 172.21.15.107:0/2086134405 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f7e6c0a2d80 0x7f7e6c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:14.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.587+0000 7f7e859c5640 1 --2- 172.21.15.107:0/2086134405 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f7e6c0a2d80 0x7f7e6c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:14.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.587+0000 7f7e76ffd640 1 -- 172.21.15.107:0/2086134405 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f7e780cf800 con 0x7f7e8019c9d0 2024-04-25T00:19:14.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.587+0000 7f7e8aa0c740 1 -- start start 2024-04-25T00:19:14.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.587+0000 7f7e8aa0c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc272630 0x557edc272a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:14.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.587+0000 7f7e8aa0c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557edc272f50 0x557edc27d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:14.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.587+0000 7f7e8aa0c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc27d620 0x557edc27fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:14.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.587+0000 7f7e8aa0c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557edc1d76b0 con 0x557edc272630 2024-04-25T00:19:14.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.587+0000 7f7e8aa0c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557edc17f880 con 0x557edc27d620 2024-04-25T00:19:14.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.587+0000 7f7e8aa0c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557edc1d3670 con 0x557edc272f50 2024-04-25T00:19:14.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.587+0000 7f7e859c5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc272630 0x557edc272a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:14.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.587+0000 7f7e861c6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc27d620 0x557edc27fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:14.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.587+0000 7f7e859c5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc272630 0x557edc272a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40020/0 (socket says 172.21.15.107:40020) 2024-04-25T00:19:14.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.587+0000 7f7e859c5640 1 -- 172.21.15.107:0/3435550595 learned_addr learned my addr 172.21.15.107:0/3435550595 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:14.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.587+0000 7f7e859c5640 1 --2- 172.21.15.107:0/2086134405 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f7e6c0a2d80 0x7f7e6c0a5240 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c000c00 tx=0x7f7e7c005b50 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:14.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.587+0000 7f7e851c4640 1 --2- 172.21.15.107:0/3435550595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557edc272f50 0x557edc27d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:14.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.587+0000 7f7e859c5640 1 -- 172.21.15.107:0/3435550595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557edc272f50 msgr2=0x557edc27d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:14.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.587+0000 7f7e859c5640 1 --2- 172.21.15.107:0/3435550595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557edc272f50 0x557edc27d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.587+0000 7f7e859c5640 1 -- 172.21.15.107:0/3435550595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc27d620 msgr2=0x557edc27fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:14.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.587+0000 7f7e859c5640 1 --2- 172.21.15.107:0/3435550595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc27d620 0x557edc27fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.587+0000 7f7e859c5640 1 -- 172.21.15.107:0/3435550595 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557edbfb57d0 con 0x557edc272630 2024-04-25T00:19:14.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.587+0000 7f7e861c6640 1 --2- 172.21.15.107:0/3435550595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc27d620 0x557edc27fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:14.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.588+0000 7f7e859c5640 1 --2- 172.21.15.107:0/3435550595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc272630 0x557edc272a10 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c009230 tx=0x7f7e7c03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28867594050965a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:14.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.588+0000 7f7e6bfff640 1 -- 172.21.15.107:0/3435550595 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c04d070 con 0x557edc272630 2024-04-25T00:19:14.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.588+0000 7f7e6bfff640 1 -- 172.21.15.107:0/3435550595 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e7c0046c0 con 0x557edc272630 2024-04-25T00:19:14.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.588+0000 7f7e6bfff640 1 -- 172.21.15.107:0/3435550595 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c035040 con 0x557edc272630 2024-04-25T00:19:14.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.588+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/3435550595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc272630 msgr2=0x557edc272a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:14.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.588+0000 7f7e8aa0c740 1 --2- 172.21.15.107:0/3435550595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc272630 0x557edc272a10 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c009230 tx=0x7f7e7c03d000 comp rx=0 tx=0).stop 2024-04-25T00:19:14.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.589+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/3435550595 shutdown_connections 2024-04-25T00:19:14.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.589+0000 7f7e8aa0c740 1 --2- 172.21.15.107:0/3435550595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc27d620 0x557edc27fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.589+0000 7f7e8aa0c740 1 --2- 172.21.15.107:0/3435550595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557edc272f50 0x557edc27d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.589+0000 7f7e8aa0c740 1 --2- 172.21.15.107:0/3435550595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc272630 0x557edc272a10 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.589+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/3435550595 >> 172.21.15.107:0/3435550595 conn(0x557edc26e950 msgr2=0x557edc26ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:14.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.589+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/3435550595 shutdown_connections 2024-04-25T00:19:14.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.589+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/3435550595 wait complete. 2024-04-25T00:19:14.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.589+0000 7f7e8aa0c740 1 -- start start 2024-04-25T00:19:14.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.589+0000 7f7e8aa0c740 1 Processor -- start 2024-04-25T00:19:14.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.589+0000 7f7e8aa0c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc272630 0x557edc27a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:14.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.589+0000 7f7e8aa0c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc272f50 0x557edc27aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:14.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.589+0000 7f7e8aa0c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557edc27d620 0x557edc27af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:14.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.589+0000 7f7e8aa0c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557edc2803b0 con 0x557edc272630 2024-04-25T00:19:14.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.589+0000 7f7e8aa0c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557edc192f50 con 0x557edc272f50 2024-04-25T00:19:14.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.589+0000 7f7e8aa0c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557edc280530 con 0x557edc27d620 2024-04-25T00:19:14.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.590+0000 7f7e8aa0c740 10 client.0 _ll_register_callbacks cb 0x557edc1ce000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:14.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.590+0000 7f7e859c5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc272630 0x557edc27a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:14.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.590+0000 7f7e851c4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc272f50 0x557edc27aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:14.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.590+0000 7f7e859c5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc272630 0x557edc27a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40030/0 (socket says 172.21.15.107:40030) 2024-04-25T00:19:14.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.590+0000 7f7e859c5640 1 -- 172.21.15.107:0/1447057379 learned_addr learned my addr 172.21.15.107:0/1447057379 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:14.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.590+0000 7f7e861c6640 1 --2- 172.21.15.107:0/1447057379 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557edc27d620 0x557edc27af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:14.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.590+0000 7f7e851c4640 1 -- 172.21.15.107:0/1447057379 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557edc27d620 msgr2=0x557edc27af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:14.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.590+0000 7f7e851c4640 1 --2- 172.21.15.107:0/1447057379 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557edc27d620 0x557edc27af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.590+0000 7f7e851c4640 1 -- 172.21.15.107:0/1447057379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc272630 msgr2=0x557edc27a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:14.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.590+0000 7f7e851c4640 1 --2- 172.21.15.107:0/1447057379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc272630 0x557edc27a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.590+0000 7f7e851c4640 1 -- 172.21.15.107:0/1447057379 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557edbeeb3f0 con 0x557edc272f50 2024-04-25T00:19:14.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.590+0000 7f7e859c5640 1 --2- 172.21.15.107:0/1447057379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc272630 0x557edc27a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:14.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.590+0000 7f7e851c4640 1 --2- 172.21.15.107:0/1447057379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc272f50 0x557edc27aa10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7e80104490 tx=0x7f7e70012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f23cbb6b1979b88a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:14.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.590+0000 7f7e697fa640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-04-25T00:19:14.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.590+0000 7f7e697fa640 1 -- 172.21.15.107:0/1447057379 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e70004050 con 0x557edc272f50 2024-04-25T00:19:14.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.590+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/1447057379 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e7c0046c0 con 0x557edc272f50 2024-04-25T00:19:14.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.590+0000 7f7e697fa640 1 -- 172.21.15.107:0/1447057379 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f7e700041f0 con 0x557edc272f50 2024-04-25T00:19:14.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.590+0000 7f7e697fa640 1 -- 172.21.15.107:0/1447057379 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7e70025050 con 0x557edc272f50 2024-04-25T00:19:14.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.590+0000 7f7e8aa0c740 10 client.25658 fetch_fsmap learned FSMap version 28 2024-04-25T00:19:14.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.590+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/1447057379 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557edbeab0f0 con 0x557edc272f50 2024-04-25T00:19:14.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.591+0000 7f7e697fa640 1 -- 172.21.15.107:0/1447057379 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e7c0046c0 con 0x557edc272f50 2024-04-25T00:19:14.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.591+0000 7f7e697fa640 1 -- 172.21.15.107:0/1447057379 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f7e70003a00 con 0x557edc272f50 2024-04-25T00:19:14.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.591+0000 7f7e8aa0c740 10 client.25658 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:19:14.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.591+0000 7f7e8aa0c740 10 client.25658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:14.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.591+0000 7f7e8aa0c740 20 client.25658 populate_metadata read hostname 'smithi107' 2024-04-25T00:19:14.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.591+0000 7f7e8aa0c740 1 --2- 172.21.15.107:0/1447057379 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x557edc28d800 0x557edc28fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:14.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.591+0000 7f7e8aa0c740 4 client.25658 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:19:14.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.591+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/1447057379 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557edc1e4bf0 con 0x557edc28d800 2024-04-25T00:19:14.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.591+0000 7f7e859c5640 1 --2- 172.21.15.107:0/1447057379 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x557edc28d800 0x557edc28fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:14.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.591+0000 7f7e859c5640 1 --2- 172.21.15.107:0/1447057379 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x557edc28d800 0x557edc28fc10 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4756c18c041fe63e server_cookie=1a31c7818b7b4c74 in_seq=0 out_seq=0 2024-04-25T00:19:14.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.591+0000 7f7e697fa640 10 client.25658 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:14.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.592+0000 7f7e697fa640 1 -- 172.21.15.107:0/1447057379 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557edc1e4bf0 con 0x557edc28d800 2024-04-25T00:19:14.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.592+0000 7f7e697fa640 10 client.25658 handle_command_reply: tid=0 2024-04-25T00:19:14.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.592+0000 7f7e8aa0c740 1 client.25658 shutdown 2024-04-25T00:19:14.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.592+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/1447057379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc272f50 msgr2=0x557edc27aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:14.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.592+0000 7f7e8aa0c740 1 --2- 172.21.15.107:0/1447057379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc272f50 0x557edc27aa10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7e80104490 tx=0x7f7e70012b60 comp rx=0 tx=0).stop 2024-04-25T00:19:14.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.592+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/1447057379 shutdown_connections 2024-04-25T00:19:14.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.592+0000 7f7e8aa0c740 1 --2- 172.21.15.107:0/1447057379 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x557edc28d800 0x557edc28fc10 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.592+0000 7f7e8aa0c740 1 --2- 172.21.15.107:0/1447057379 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557edc27d620 0x557edc27af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.592+0000 7f7e8aa0c740 1 --2- 172.21.15.107:0/1447057379 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc272f50 0x557edc27aa10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.592+0000 7f7e697fa640 0 client.25658 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:14.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.592+0000 7f7e8aa0c740 1 --2- 172.21.15.107:0/1447057379 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc272630 0x557edc27a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.592+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/1447057379 >> 172.21.15.107:0/1447057379 conn(0x557edc26e950 msgr2=0x557edc274650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:14.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.592+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/1447057379 shutdown_connections 2024-04-25T00:19:14.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.593+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/1447057379 wait complete. 2024-04-25T00:19:14.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.593+0000 7f7e8aa0c740 20 client.25658 trim_cache size 0 max 16384 2024-04-25T00:19:14.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.610+0000 7f7e8aa0c740 1 Processor -- start 2024-04-25T00:19:14.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.610+0000 7f7e8aa0c740 1 -- start start 2024-04-25T00:19:14.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.610+0000 7f7e8aa0c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557edc272630 0x557edc27cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:14.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.610+0000 7f7e8aa0c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc272f50 0x557edc2892a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:14.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.610+0000 7f7e8aa0c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc27d620 0x557edc279650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:14.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.610+0000 7f7e8aa0c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7e801b1530 con 0x557edc272f50 2024-04-25T00:19:14.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.610+0000 7f7e8aa0c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7e801aed40 con 0x557edc27d620 2024-04-25T00:19:14.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.610+0000 7f7e8aa0c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7e801acb90 con 0x557edc272630 2024-04-25T00:19:14.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.610+0000 7f7e851c4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc272f50 0x557edc2892a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:14.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.610+0000 7f7e851c4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc272f50 0x557edc2892a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40034/0 (socket says 172.21.15.107:40034) 2024-04-25T00:19:14.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.610+0000 7f7e851c4640 1 -- 172.21.15.107:0/1728204761 learned_addr learned my addr 172.21.15.107:0/1728204761 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:14.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.610+0000 7f7e861c6640 1 --2- 172.21.15.107:0/1728204761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc27d620 0x557edc279650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:14.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.611+0000 7f7e859c5640 1 --2- 172.21.15.107:0/1728204761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557edc272630 0x557edc27cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:14.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.611+0000 7f7e851c4640 1 -- 172.21.15.107:0/1728204761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557edc272630 msgr2=0x557edc27cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:14.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.611+0000 7f7e851c4640 1 --2- 172.21.15.107:0/1728204761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557edc272630 0x557edc27cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.611+0000 7f7e851c4640 1 -- 172.21.15.107:0/1728204761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc27d620 msgr2=0x557edc279650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:14.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.611+0000 7f7e851c4640 1 --2- 172.21.15.107:0/1728204761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc27d620 0x557edc279650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.611+0000 7f7e851c4640 1 -- 172.21.15.107:0/1728204761 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557edc272d80 con 0x557edc272f50 2024-04-25T00:19:14.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.611+0000 7f7e861c6640 1 --2- 172.21.15.107:0/1728204761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc27d620 0x557edc279650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:14.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.611+0000 7f7e851c4640 1 --2- 172.21.15.107:0/1728204761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc272f50 0x557edc2892a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f7e700058c0 tx=0x7f7e70010710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=debd7ec0daa6dbc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:14.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.611+0000 7f7e6b7fe640 1 -- 172.21.15.107:0/1728204761 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e700050d0 con 0x557edc272f50 2024-04-25T00:19:14.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.611+0000 7f7e6b7fe640 1 -- 172.21.15.107:0/1728204761 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e7002e070 con 0x557edc272f50 2024-04-25T00:19:14.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.611+0000 7f7e6b7fe640 1 -- 172.21.15.107:0/1728204761 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e700035f0 con 0x557edc272f50 2024-04-25T00:19:14.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.612+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/1728204761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc272f50 msgr2=0x557edc2892a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:14.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.612+0000 7f7e8aa0c740 1 --2- 172.21.15.107:0/1728204761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc272f50 0x557edc2892a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f7e700058c0 tx=0x7f7e70010710 comp rx=0 tx=0).stop 2024-04-25T00:19:14.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.612+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/1728204761 shutdown_connections 2024-04-25T00:19:14.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.612+0000 7f7e8aa0c740 1 --2- 172.21.15.107:0/1728204761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc27d620 0x557edc279650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.612+0000 7f7e8aa0c740 1 --2- 172.21.15.107:0/1728204761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc272f50 0x557edc2892a0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.612+0000 7f7e8aa0c740 1 --2- 172.21.15.107:0/1728204761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557edc272630 0x557edc27cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.612+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/1728204761 >> 172.21.15.107:0/1728204761 conn(0x557edc26e950 msgr2=0x557edc27a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:14.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.612+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/1728204761 shutdown_connections 2024-04-25T00:19:14.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.612+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/1728204761 wait complete. 2024-04-25T00:19:14.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.612+0000 7f7e8aa0c740 1 -- start start 2024-04-25T00:19:14.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.612+0000 7f7e8aa0c740 1 Processor -- start 2024-04-25T00:19:14.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.612+0000 7f7e8aa0c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc272630 0x557edc2772b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:14.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.612+0000 7f7e8aa0c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557edc272f50 0x557edc2777f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:14.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.612+0000 7f7e8aa0c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc27d620 0x557edc277d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:14.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.612+0000 7f7e8aa0c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7e801ad200 con 0x557edc27d620 2024-04-25T00:19:14.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.612+0000 7f7e8aa0c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7e801ae670 con 0x557edc272630 2024-04-25T00:19:14.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.612+0000 7f7e8aa0c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7e801ad8d0 con 0x557edc272f50 2024-04-25T00:19:14.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.612+0000 7f7e8aa0c740 10 client.0 _ll_register_callbacks cb 0x7f7e700097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:14.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.613+0000 7f7e861c6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc27d620 0x557edc277d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:14.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.613+0000 7f7e859c5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc272630 0x557edc2772b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:14.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.613+0000 7f7e851c4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557edc272f50 0x557edc2777f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:14.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.613+0000 7f7e859c5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc272630 0x557edc2772b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.107:53492/0 (socket says 172.21.15.107:53492) 2024-04-25T00:19:14.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.613+0000 7f7e859c5640 1 -- 172.21.15.107:0/1595977111 learned_addr learned my addr 172.21.15.107:0/1595977111 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:14.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.613+0000 7f7e861c6640 1 -- 172.21.15.107:0/1595977111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557edc272f50 msgr2=0x557edc2777f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:14.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.613+0000 7f7e861c6640 1 --2- 172.21.15.107:0/1595977111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557edc272f50 0x557edc2777f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.613+0000 7f7e861c6640 1 -- 172.21.15.107:0/1595977111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc272630 msgr2=0x557edc2772b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:14.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.613+0000 7f7e861c6640 1 --2- 172.21.15.107:0/1595977111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc272630 0x557edc2772b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.613+0000 7f7e861c6640 1 -- 172.21.15.107:0/1595977111 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557edc283ae0 con 0x557edc27d620 2024-04-25T00:19:14.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.613+0000 7f7e859c5640 1 --2- 172.21.15.107:0/1595977111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc272630 0x557edc2772b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:14.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.613+0000 7f7e861c6640 1 --2- 172.21.15.107:0/1595977111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc27d620 0x557edc277d30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7e780cccf0 tx=0x7f7e7809b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5736cb1abfc6e6dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:14.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.613+0000 7f7e697fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:19:14.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.613+0000 7f7e697fa640 1 -- 172.21.15.107:0/1595977111 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7809fe70 con 0x557edc27d620 2024-04-25T00:19:14.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.613+0000 7f7e697fa640 1 -- 172.21.15.107:0/1595977111 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f7e780981e0 con 0x557edc27d620 2024-04-25T00:19:14.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.613+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/1595977111 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e7002e070 con 0x557edc27d620 2024-04-25T00:19:14.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.613+0000 7f7e697fa640 1 -- 172.21.15.107:0/1595977111 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7e780a4730 con 0x557edc27d620 2024-04-25T00:19:14.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.613+0000 7f7e8aa0c740 10 client.16194 fetch_fsmap learned FSMap version 28 2024-04-25T00:19:14.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.613+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/1595977111 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557edc2f4180 con 0x557edc27d620 2024-04-25T00:19:14.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.614+0000 7f7e697fa640 1 -- 172.21.15.107:0/1595977111 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e7002e070 con 0x557edc27d620 2024-04-25T00:19:14.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.614+0000 7f7e697fa640 1 -- 172.21.15.107:0/1595977111 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f7e7800b040 con 0x557edc27d620 2024-04-25T00:19:14.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.614+0000 7f7e8aa0c740 10 client.16194 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:19:14.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.614+0000 7f7e8aa0c740 10 client.16194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:14.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.614+0000 7f7e8aa0c740 20 client.16194 populate_metadata read hostname 'smithi107' 2024-04-25T00:19:14.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.614+0000 7f7e8aa0c740 1 --2- 172.21.15.107:0/1595977111 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x557edc28d800 0x557edc2f6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:14.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.614+0000 7f7e8aa0c740 4 client.16194 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:19:14.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.614+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/1595977111 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557edc1def40 con 0x557edc28d800 2024-04-25T00:19:14.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.614+0000 7f7e859c5640 1 --2- 172.21.15.107:0/1595977111 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x557edc28d800 0x557edc2f6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:14.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.614+0000 7f7e859c5640 1 --2- 172.21.15.107:0/1595977111 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x557edc28d800 0x557edc2f6940 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd8cef3fe92ebeb7 server_cookie=97f66159810b57ca in_seq=0 out_seq=0 2024-04-25T00:19:14.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.614+0000 7f7e697fa640 10 client.16194 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:14.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.614+0000 7f7e697fa640 1 -- 172.21.15.107:0/1595977111 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557edc1def40 con 0x557edc28d800 2024-04-25T00:19:14.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.614+0000 7f7e697fa640 10 client.16194 handle_command_reply: tid=0 2024-04-25T00:19:14.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.614+0000 7f7e8aa0c740 1 client.16194 shutdown 2024-04-25T00:19:14.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.614+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/1595977111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc27d620 msgr2=0x557edc277d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:14.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.614+0000 7f7e8aa0c740 1 --2- 172.21.15.107:0/1595977111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc27d620 0x557edc277d30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7e780cccf0 tx=0x7f7e7809b590 comp rx=0 tx=0).stop 2024-04-25T00:19:14.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.615+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/1595977111 shutdown_connections 2024-04-25T00:19:14.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.615+0000 7f7e8aa0c740 1 --2- 172.21.15.107:0/1595977111 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x557edc28d800 0x557edc2f6940 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.615+0000 7f7e8aa0c740 1 --2- 172.21.15.107:0/1595977111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557edc27d620 0x557edc277d30 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.615+0000 7f7e8aa0c740 1 --2- 172.21.15.107:0/1595977111 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557edc272f50 0x557edc2777f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.615+0000 7f7e8aa0c740 1 --2- 172.21.15.107:0/1595977111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557edc272630 0x557edc2772b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.615+0000 7f7e697fa640 0 client.16194 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:14.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.615+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/1595977111 >> 172.21.15.107:0/1595977111 conn(0x557edc26e950 msgr2=0x557edc27ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:14.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.615+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/1595977111 shutdown_connections 2024-04-25T00:19:14.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.615+0000 7f7e8aa0c740 1 -- 172.21.15.107:0/1595977111 wait complete. 2024-04-25T00:19:14.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.615+0000 7f7e8aa0c740 20 client.16194 trim_cache size 0 max 16384 2024-04-25T00:19:14.622 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:19:14.622 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-04-25T00:19:14.622 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-04-25T00:19:14.622 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:19:14.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.615+0000 7f7e87c29640 1 -- 172.21.15.107:0/2086134405 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f7e6c0a2d80 msgr2=0x7f7e6c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:14.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.615+0000 7f7e87c29640 1 --2- 172.21.15.107:0/2086134405 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f7e6c0a2d80 0x7f7e6c0a5240 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c000c00 tx=0x7f7e7c005b50 comp rx=0 tx=0).stop 2024-04-25T00:19:14.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.615+0000 7f7e87c29640 1 -- 172.21.15.107:0/2086134405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e8019c9d0 msgr2=0x7f7e8019ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:14.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.615+0000 7f7e87c29640 1 --2- 172.21.15.107:0/2086134405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e8019c9d0 0x7f7e8019ce80 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7e7800de10 tx=0x7f7e7800b730 comp rx=0 tx=0).stop 2024-04-25T00:19:14.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.615+0000 7f7e87c29640 1 -- 172.21.15.107:0/2086134405 shutdown_connections 2024-04-25T00:19:14.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.615+0000 7f7e87c29640 1 --2- 172.21.15.107:0/2086134405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e8019c9d0 0x7f7e8019ce80 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.615+0000 7f7e87c29640 1 --2- 172.21.15.107:0/2086134405 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f7e6c0a2d80 0x7f7e6c0a5240 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.615+0000 7f7e87c29640 1 --2- 172.21.15.107:0/2086134405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7e801050d0 0x7f7e80198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.615+0000 7f7e87c29640 1 --2- 172.21.15.107:0/2086134405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7e80100000 0x7f7e80198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.615+0000 7f7e87c29640 1 -- 172.21.15.107:0/2086134405 >> 172.21.15.107:0/2086134405 conn(0x7f7e80076c00 msgr2=0x7f7e80103890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:14.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.615+0000 7f7e87c29640 1 -- 172.21.15.107:0/2086134405 shutdown_connections 2024-04-25T00:19:14.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.615+0000 7f7e87c29640 1 -- 172.21.15.107:0/2086134405 wait complete. 2024-04-25T00:19:14.626 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:19:14.626 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:19:14.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.728+0000 7fae8c04d640 1 -- 172.21.15.107:0/1077539977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae84106630 msgr2=0x7fae84106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:14.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.728+0000 7fae8c04d640 1 --2- 172.21.15.107:0/1077539977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae84106630 0x7fae84106a10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fae80009930 tx=0x7fae8002a420 comp rx=0 tx=0).stop 2024-04-25T00:19:14.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.728+0000 7fae8c04d640 1 -- 172.21.15.107:0/1077539977 shutdown_connections 2024-04-25T00:19:14.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.728+0000 7fae8c04d640 1 --2- 172.21.15.107:0/1077539977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae84100fe0 0x7fae8410d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.728+0000 7fae8c04d640 1 --2- 172.21.15.107:0/1077539977 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae84100620 0x7fae84100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.728+0000 7fae8c04d640 1 --2- 172.21.15.107:0/1077539977 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae84106630 0x7fae84106a10 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.728+0000 7fae8c04d640 1 -- 172.21.15.107:0/1077539977 >> 172.21.15.107:0/1077539977 conn(0x7fae84077740 msgr2=0x7fae84077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:14.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.728+0000 7fae8c04d640 1 -- 172.21.15.107:0/1077539977 shutdown_connections 2024-04-25T00:19:14.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.728+0000 7fae8c04d640 1 -- 172.21.15.107:0/1077539977 wait complete. 2024-04-25T00:19:14.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.728+0000 7fae8c04d640 1 Processor -- start 2024-04-25T00:19:14.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.728+0000 7fae8c04d640 1 -- start start 2024-04-25T00:19:14.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.729+0000 7fae8c04d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae84100620 0x7fae84197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:14.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.729+0000 7fae8c04d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae84100fe0 0x7fae841984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:14.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.729+0000 7fae8c04d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae84106630 0x7fae8419c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:14.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.729+0000 7fae8c04d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fae84110a00 con 0x7fae84100fe0 2024-04-25T00:19:14.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.729+0000 7fae8c04d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fae84110880 con 0x7fae84100620 2024-04-25T00:19:14.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.729+0000 7fae8c04d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fae84110b80 con 0x7fae84106630 2024-04-25T00:19:14.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.729+0000 7fae895e8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae84100fe0 0x7fae841984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:14.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.729+0000 7fae895e8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae84100fe0 0x7fae841984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40058/0 (socket says 172.21.15.107:40058) 2024-04-25T00:19:14.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.729+0000 7fae895e8640 1 -- 172.21.15.107:0/354407517 learned_addr learned my addr 172.21.15.107:0/354407517 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:14.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.729+0000 7fae89de9640 1 --2- 172.21.15.107:0/354407517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae84100620 0x7fae84197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:14.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.729+0000 7fae8a5ea640 1 --2- 172.21.15.107:0/354407517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae84106630 0x7fae8419c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:14.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.729+0000 7fae895e8640 1 -- 172.21.15.107:0/354407517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae84106630 msgr2=0x7fae8419c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:14.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.729+0000 7fae895e8640 1 --2- 172.21.15.107:0/354407517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae84106630 0x7fae8419c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.729+0000 7fae895e8640 1 -- 172.21.15.107:0/354407517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae84100620 msgr2=0x7fae84197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:14.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.729+0000 7fae895e8640 1 --2- 172.21.15.107:0/354407517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae84100620 0x7fae84197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:14.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.729+0000 7fae895e8640 1 -- 172.21.15.107:0/354407517 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8419cf90 con 0x7fae84100fe0 2024-04-25T00:19:14.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.729+0000 7fae89de9640 1 --2- 172.21.15.107:0/354407517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae84100620 0x7fae84197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:14.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.729+0000 7fae895e8640 1 --2- 172.21.15.107:0/354407517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae84100fe0 0x7fae841984e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fae7400d950 tx=0x7fae7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:14.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.729+0000 7fae7affd640 1 -- 172.21.15.107:0/354407517 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae74017070 con 0x7fae84100fe0 2024-04-25T00:19:14.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.729+0000 7fae8c04d640 1 -- 172.21.15.107:0/354407517 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae8419d280 con 0x7fae84100fe0 2024-04-25T00:19:14.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.729+0000 7fae7affd640 1 -- 172.21.15.107:0/354407517 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae74009c30 con 0x7fae84100fe0 2024-04-25T00:19:14.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.729+0000 7fae7affd640 1 -- 172.21.15.107:0/354407517 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7400f6b0 con 0x7fae84100fe0 2024-04-25T00:19:14.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.729+0000 7fae8c04d640 1 -- 172.21.15.107:0/354407517 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae841a4b10 con 0x7fae84100fe0 2024-04-25T00:19:14.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.730+0000 7fae8c04d640 1 -- 172.21.15.107:0/354407517 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae84068560 con 0x7fae84100fe0 2024-04-25T00:19:14.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.731+0000 7fae7affd640 1 -- 172.21.15.107:0/354407517 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fae740109d0 con 0x7fae84100fe0 2024-04-25T00:19:14.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.732+0000 7fae7affd640 1 --2- 172.21.15.107:0/354407517 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fae700a2d00 0x7fae700a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:14.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.732+0000 7fae89de9640 1 --2- 172.21.15.107:0/354407517 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fae700a2d00 0x7fae700a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:14.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.732+0000 7fae7affd640 1 -- 172.21.15.107:0/354407517 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fae740cf320 con 0x7fae84100fe0 2024-04-25T00:19:14.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.732+0000 7fae89de9640 1 --2- 172.21.15.107:0/354407517 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fae700a2d00 0x7fae700a51c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fae80009900 tx=0x7fae800058d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:14.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.737+0000 7fae7affd640 1 -- 172.21.15.107:0/354407517 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fae74009de0 con 0x7fae84100fe0 2024-04-25T00:19:14.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.995+0000 7fae8c04d640 1 -- 172.21.15.107:0/354407517 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae8402d170 con 0x7fae84100fe0 2024-04-25T00:19:14.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:14.996+0000 7fae7affd640 1 -- 172.21.15.107:0/354407517 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11906 (secure 0 0 0) 0x7fae7409b580 con 0x7fae84100fe0 2024-04-25T00:19:14.997 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:19:14.997 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:18:38.933541+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24493,24521]},"id":1}]} 2024-04-25T00:19:14.997 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 28 2024-04-25T00:19:15.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.000+0000 7fae8c04d640 1 -- 172.21.15.107:0/354407517 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fae700a2d00 msgr2=0x7fae700a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.000+0000 7fae8c04d640 1 --2- 172.21.15.107:0/354407517 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fae700a2d00 0x7fae700a51c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fae80009900 tx=0x7fae800058d0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.000+0000 7fae8c04d640 1 -- 172.21.15.107:0/354407517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae84100fe0 msgr2=0x7fae841984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.001+0000 7fae8c04d640 1 --2- 172.21.15.107:0/354407517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae84100fe0 0x7fae841984e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fae7400d950 tx=0x7fae7400de10 comp rx=0 tx=0).stop 2024-04-25T00:19:15.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.001+0000 7fae8c04d640 1 -- 172.21.15.107:0/354407517 shutdown_connections 2024-04-25T00:19:15.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.001+0000 7fae8c04d640 1 --2- 172.21.15.107:0/354407517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae84106630 0x7fae8419c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.001+0000 7fae8c04d640 1 --2- 172.21.15.107:0/354407517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae84100fe0 0x7fae841984e0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.001+0000 7fae8c04d640 1 --2- 172.21.15.107:0/354407517 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fae700a2d00 0x7fae700a51c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.001+0000 7fae8c04d640 1 --2- 172.21.15.107:0/354407517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae84100620 0x7fae84197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.001+0000 7fae8c04d640 1 -- 172.21.15.107:0/354407517 >> 172.21.15.107:0/354407517 conn(0x7fae84077740 msgr2=0x7fae84068c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.001+0000 7fae8c04d640 1 -- 172.21.15.107:0/354407517 shutdown_connections 2024-04-25T00:19:15.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.002+0000 7fae8c04d640 1 -- 172.21.15.107:0/354407517 wait complete. 2024-04-25T00:19:15.021 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:19:15.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.146+0000 7ff7a9a10640 1 -- 172.21.15.107:0/1212502569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a41032c0 msgr2=0x7ff7a410b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.146+0000 7ff7a9a10640 1 --2- 172.21.15.107:0/1212502569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a41032c0 0x7ff7a410b7a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009900 tx=0x7ff78c02a480 comp rx=0 tx=0).stop 2024-04-25T00:19:15.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.146+0000 7ff7a9a10640 1 -- 172.21.15.107:0/1212502569 shutdown_connections 2024-04-25T00:19:15.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.146+0000 7ff7a9a10640 1 --2- 172.21.15.107:0/1212502569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7a410bd70 0x7ff7a410e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.146+0000 7ff7a9a10640 1 --2- 172.21.15.107:0/1212502569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a41032c0 0x7ff7a410b7a0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.146+0000 7ff7a9a10640 1 --2- 172.21.15.107:0/1212502569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7a41029a0 0x7ff7a4102d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.146+0000 7ff7a9a10640 1 -- 172.21.15.107:0/1212502569 >> 172.21.15.107:0/1212502569 conn(0x7ff7a40769c0 msgr2=0x7ff7a4076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.146+0000 7ff7a9a10640 1 -- 172.21.15.107:0/1212502569 shutdown_connections 2024-04-25T00:19:15.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.146+0000 7ff7a9a10640 1 -- 172.21.15.107:0/1212502569 wait complete. 2024-04-25T00:19:15.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.146+0000 7ff7a9a10640 1 Processor -- start 2024-04-25T00:19:15.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.146+0000 7ff7a9a10640 1 -- start start 2024-04-25T00:19:15.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.146+0000 7ff7a9a10640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a41029a0 0x7ff7a41005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.146+0000 7ff7a9a10640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7a41032c0 0x7ff7a4100b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.146+0000 7ff7a9a10640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7a410bd70 0x7ff7a4101070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.146+0000 7ff7a9a10640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a4111b60 con 0x7ff7a41029a0 2024-04-25T00:19:15.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.146+0000 7ff7a9a10640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41119e0 con 0x7ff7a41032c0 2024-04-25T00:19:15.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.146+0000 7ff7a9a10640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a4111ce0 con 0x7ff7a410bd70 2024-04-25T00:19:15.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.147+0000 7ff7a2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a41029a0 0x7ff7a41005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.147+0000 7ff7a2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a41029a0 0x7ff7a41005f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40070/0 (socket says 172.21.15.107:40070) 2024-04-25T00:19:15.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.147+0000 7ff7a2ffd640 1 -- 172.21.15.107:0/2195401004 learned_addr learned my addr 172.21.15.107:0/2195401004 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.147+0000 7ff79bfff640 1 --2- 172.21.15.107:0/2195401004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7a41032c0 0x7ff7a4100b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.147+0000 7ff7a2ffd640 1 -- 172.21.15.107:0/2195401004 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7a410bd70 msgr2=0x7ff7a4101070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.147+0000 7ff7a2ffd640 1 --2- 172.21.15.107:0/2195401004 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7a410bd70 0x7ff7a4101070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.147+0000 7ff7a2ffd640 1 -- 172.21.15.107:0/2195401004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7a41032c0 msgr2=0x7ff7a4100b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.147+0000 7ff7a2ffd640 1 --2- 172.21.15.107:0/2195401004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7a41032c0 0x7ff7a4100b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.147+0000 7ff7a2ffd640 1 -- 172.21.15.107:0/2195401004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a41a6580 con 0x7ff7a41029a0 2024-04-25T00:19:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.147+0000 7ff7a2ffd640 1 --2- 172.21.15.107:0/2195401004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a41029a0 0x7ff7a41005f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff79400e9f0 tx=0x7ff79400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.147+0000 7ff7a8a0e640 1 -- 172.21.15.107:0/2195401004 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79400cd80 con 0x7ff7a41029a0 2024-04-25T00:19:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.147+0000 7ff7a9a10640 1 -- 172.21.15.107:0/2195401004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a41a6870 con 0x7ff7a41029a0 2024-04-25T00:19:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.147+0000 7ff7a8a0e640 1 -- 172.21.15.107:0/2195401004 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff794010910 con 0x7ff7a41029a0 2024-04-25T00:19:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.147+0000 7ff7a8a0e640 1 -- 172.21.15.107:0/2195401004 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7940188f0 con 0x7ff7a41029a0 2024-04-25T00:19:15.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.147+0000 7ff7a9a10640 1 -- 172.21.15.107:0/2195401004 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a41a6d90 con 0x7ff7a41029a0 2024-04-25T00:19:15.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.149+0000 7ff7a8a0e640 1 -- 172.21.15.107:0/2195401004 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff794010ac0 con 0x7ff7a41029a0 2024-04-25T00:19:15.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.150+0000 7ff7ac7f3740 1 Processor -- start 2024-04-25T00:19:15.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.150+0000 7ff7ac7f3740 1 -- start start 2024-04-25T00:19:15.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.150+0000 7ff7ac7f3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1703bd580 0x55f1703bd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.150+0000 7ff7ac7f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703bdea0 0x55f1703c8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.150+0000 7ff7ac7f3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1703c8570 0x55f1703ca960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.150+0000 7ff7ac7f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f170322670 con 0x55f1703bdea0 2024-04-25T00:19:15.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.150+0000 7ff7ac7f3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f1702ca970 con 0x55f1703bd580 2024-04-25T00:19:15.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.150+0000 7ff7ac7f3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f17031e630 con 0x55f1703c8570 2024-04-25T00:19:15.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.150+0000 7ff7a2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703bdea0 0x55f1703c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.150+0000 7ff7a37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1703bd580 0x55f1703bd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.150+0000 7ff7a37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1703bd580 0x55f1703bd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.107:53544/0 (socket says 172.21.15.107:53544) 2024-04-25T00:19:15.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.150+0000 7ff7a2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703bdea0 0x55f1703c8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40080/0 (socket says 172.21.15.107:40080) 2024-04-25T00:19:15.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.150+0000 7ff7a37fe640 1 -- 172.21.15.107:0/2106296037 learned_addr learned my addr 172.21.15.107:0/2106296037 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:15.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.150+0000 7ff7a8a0e640 1 --2- 172.21.15.107:0/2195401004 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff7900a2d30 0x7ff7900a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.150+0000 7ff79bfff640 1 --2- 172.21.15.107:0/2106296037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1703c8570 0x55f1703ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.150+0000 7ff7a37fe640 1 --2- 172.21.15.107:0/2195401004 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff7900a2d30 0x7ff7900a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.150+0000 7ff7a37fe640 1 -- 172.21.15.107:0/2106296037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1703c8570 msgr2=0x55f1703ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.150+0000 7ff7a37fe640 1 --2- 172.21.15.107:0/2106296037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1703c8570 0x55f1703ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.150+0000 7ff7a37fe640 1 -- 172.21.15.107:0/2106296037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703bdea0 msgr2=0x55f1703c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.150+0000 7ff7a8a0e640 1 -- 172.21.15.107:0/2195401004 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7ff7940cfc80 con 0x7ff7a41029a0 2024-04-25T00:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.150+0000 7ff7a37fe640 1 --2- 172.21.15.107:0/2106296037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703bdea0 0x55f1703c8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.150+0000 7ff7a37fe640 1 -- 172.21.15.107:0/2106296037 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1702ee400 con 0x55f1703bd580 2024-04-25T00:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.150+0000 7ff7a2ffd640 1 --2- 172.21.15.107:0/2106296037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703bdea0 0x55f1703c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.151+0000 7ff7a37fe640 1 --2- 172.21.15.107:0/2106296037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1703bd580 0x55f1703bd960 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0126b0 tx=0x7ff79c012b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9b508e8b1e74e91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.151+0000 7ff799ffb640 1 -- 172.21.15.107:0/2106296037 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c012050 con 0x55f1703bd580 2024-04-25T00:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.151+0000 7ff799ffb640 1 -- 172.21.15.107:0/2106296037 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff79c013040 con 0x55f1703bd580 2024-04-25T00:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.151+0000 7ff799ffb640 1 -- 172.21.15.107:0/2106296037 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c01b070 con 0x55f1703bd580 2024-04-25T00:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.151+0000 7ff7a37fe640 1 --2- 172.21.15.107:0/2195401004 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff7900a2d30 0x7ff7900a51f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff79c01d000 tx=0x7ff79c01b000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.151+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2106296037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1703bd580 msgr2=0x55f1703bd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.151+0000 7ff7ac7f3740 1 --2- 172.21.15.107:0/2106296037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1703bd580 0x55f1703bd960 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0126b0 tx=0x7ff79c012b70 comp rx=0 tx=0).stop 2024-04-25T00:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.151+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2106296037 shutdown_connections 2024-04-25T00:19:15.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.151+0000 7ff7ac7f3740 1 --2- 172.21.15.107:0/2106296037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1703c8570 0x55f1703ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.151+0000 7ff7ac7f3740 1 --2- 172.21.15.107:0/2106296037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703bdea0 0x55f1703c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.151+0000 7ff7ac7f3740 1 --2- 172.21.15.107:0/2106296037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1703bd580 0x55f1703bd960 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.151+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2106296037 >> 172.21.15.107:0/2106296037 conn(0x55f1703b98a0 msgr2=0x55f1703b9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.151+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2106296037 shutdown_connections 2024-04-25T00:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.151+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2106296037 wait complete. 2024-04-25T00:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.152+0000 7ff7ac7f3740 1 -- start start 2024-04-25T00:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.152+0000 7ff7ac7f3740 1 Processor -- start 2024-04-25T00:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.152+0000 7ff7ac7f3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1703bd580 0x55f1703c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.152+0000 7ff7ac7f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703bdea0 0x55f1703c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.152+0000 7ff7ac7f3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1703c8570 0x55f1703c5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.152+0000 7ff7ac7f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f1703cb360 con 0x55f1703bdea0 2024-04-25T00:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.152+0000 7ff7ac7f3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f1702de050 con 0x55f1703bd580 2024-04-25T00:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.152+0000 7ff7ac7f3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f1703cb4e0 con 0x55f1703c8570 2024-04-25T00:19:15.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.152+0000 7ff7ac7f3740 10 client.0 _ll_register_callbacks cb 0x55f170035be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:15.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.152+0000 7ff79bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703bdea0 0x55f1703c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.152+0000 7ff7a2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1703bd580 0x55f1703c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.152+0000 7ff79bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703bdea0 0x55f1703c51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40092/0 (socket says 172.21.15.107:40092) 2024-04-25T00:19:15.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.152+0000 7ff79bfff640 1 -- 172.21.15.107:0/2500306595 learned_addr learned my addr 172.21.15.107:0/2500306595 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:15.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.152+0000 7ff7a2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1703bd580 0x55f1703c7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.107:53556/0 (socket says 172.21.15.107:53556) 2024-04-25T00:19:15.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.153+0000 7ff7a37fe640 1 --2- 172.21.15.107:0/2500306595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1703c8570 0x55f1703c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.153+0000 7ff79bfff640 1 -- 172.21.15.107:0/2500306595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1703c8570 msgr2=0x55f1703c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.153+0000 7ff79bfff640 1 --2- 172.21.15.107:0/2500306595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1703c8570 0x55f1703c5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.153+0000 7ff79bfff640 1 -- 172.21.15.107:0/2500306595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1703bd580 msgr2=0x55f1703c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.153+0000 7ff79bfff640 1 --2- 172.21.15.107:0/2500306595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1703bd580 0x55f1703c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.153+0000 7ff79bfff640 1 -- 172.21.15.107:0/2500306595 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1702afdc0 con 0x55f1703bdea0 2024-04-25T00:19:15.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.153+0000 7ff79bfff640 1 --2- 172.21.15.107:0/2500306595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703bdea0 0x55f1703c51d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009900 tx=0x7ff78c031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238a4b9d8aac1bc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.153+0000 7ff782ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:19:15.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.153+0000 7ff782ffd640 1 -- 172.21.15.107:0/2500306595 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78c035040 con 0x55f1703bdea0 2024-04-25T00:19:15.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.153+0000 7ff782ffd640 1 -- 172.21.15.107:0/2500306595 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7ff78c004240 con 0x55f1703bdea0 2024-04-25T00:19:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.153+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2500306595 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff79c013040 con 0x55f1703bdea0 2024-04-25T00:19:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.153+0000 7ff782ffd640 1 -- 172.21.15.107:0/2500306595 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff78c04f050 con 0x55f1703bdea0 2024-04-25T00:19:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.153+0000 7ff7ac7f3740 10 client.16212 fetch_fsmap learned FSMap version 28 2024-04-25T00:19:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.153+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2500306595 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f1700363f0 con 0x55f1703bdea0 2024-04-25T00:19:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.153+0000 7ff782ffd640 1 -- 172.21.15.107:0/2500306595 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff79c013040 con 0x55f1703bdea0 2024-04-25T00:19:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.153+0000 7ff782ffd640 1 -- 172.21.15.107:0/2500306595 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7ff78c0466e0 con 0x55f1703bdea0 2024-04-25T00:19:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.153+0000 7ff7ac7f3740 10 client.16212 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:19:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.153+0000 7ff7ac7f3740 10 client.16212 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:19:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.153+0000 7ff7ac7f3740 20 client.16212 populate_metadata read hostname 'smithi107' 2024-04-25T00:19:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.154+0000 7ff7ac7f3740 1 --2- 172.21.15.107:0/2500306595 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55f1703d86b0 0x55f1703dab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.154+0000 7ff7ac7f3740 4 client.16212 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:19:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.154+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2500306595 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f17032fbb0 con 0x55f1703d86b0 2024-04-25T00:19:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.154+0000 7ff7a2ffd640 1 --2- 172.21.15.107:0/2500306595 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55f1703d86b0 0x55f1703dab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.154+0000 7ff7a2ffd640 1 --2- 172.21.15.107:0/2500306595 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55f1703d86b0 0x55f1703dab30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2196e32f6bdf5e56 server_cookie=11c0ae623aef23d6 in_seq=0 out_seq=0 2024-04-25T00:19:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.154+0000 7ff782ffd640 10 client.16212 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:19:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.155+0000 7ff782ffd640 1 -- 172.21.15.107:0/2500306595 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f17032fbb0 con 0x55f1703d86b0 2024-04-25T00:19:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.155+0000 7ff782ffd640 10 client.16212 handle_command_reply: tid=0 2024-04-25T00:19:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.155+0000 7ff7ac7f3740 1 client.16212 shutdown 2024-04-25T00:19:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.155+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2500306595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703bdea0 msgr2=0x55f1703c51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.155+0000 7ff7ac7f3740 1 --2- 172.21.15.107:0/2500306595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703bdea0 0x55f1703c51d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009900 tx=0x7ff78c031920 comp rx=0 tx=0).stop 2024-04-25T00:19:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.155+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2500306595 shutdown_connections 2024-04-25T00:19:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.155+0000 7ff7ac7f3740 1 --2- 172.21.15.107:0/2500306595 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55f1703d86b0 0x55f1703dab30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.155+0000 7ff7ac7f3740 1 --2- 172.21.15.107:0/2500306595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1703c8570 0x55f1703c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.155+0000 7ff7ac7f3740 1 --2- 172.21.15.107:0/2500306595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703bdea0 0x55f1703c51d0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.155+0000 7ff7ac7f3740 1 --2- 172.21.15.107:0/2500306595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1703bd580 0x55f1703c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.155+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2500306595 >> 172.21.15.107:0/2500306595 conn(0x55f1703b98a0 msgr2=0x55f1703bf6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.155+0000 7ff782ffd640 0 client.16212 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:19:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.155+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2500306595 shutdown_connections 2024-04-25T00:19:15.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.155+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2500306595 wait complete. 2024-04-25T00:19:15.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.155+0000 7ff7ac7f3740 20 client.16212 trim_cache size 0 max 16384 2024-04-25T00:19:15.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.169+0000 7ff7ac7f3740 1 Processor -- start 2024-04-25T00:19:15.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.169+0000 7ff7ac7f3740 1 -- start start 2024-04-25T00:19:15.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.169+0000 7ff7ac7f3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1703bd580 0x55f1703d4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.169+0000 7ff7ac7f3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1703bdea0 0x55f1703c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.169+0000 7ff7ac7f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703c8570 0x55f1703e0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.169+0000 7ff7ac7f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41b36d0 con 0x55f1703c8570 2024-04-25T00:19:15.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.169+0000 7ff7ac7f3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41b0ee0 con 0x55f1703bd580 2024-04-25T00:19:15.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.169+0000 7ff7ac7f3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41aed30 con 0x55f1703bdea0 2024-04-25T00:19:15.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.169+0000 7ff7a37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703c8570 0x55f1703e0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.169+0000 7ff7a37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703c8570 0x55f1703e0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40106/0 (socket says 172.21.15.107:40106) 2024-04-25T00:19:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.169+0000 7ff7a37fe640 1 -- 172.21.15.107:0/2125327868 learned_addr learned my addr 172.21.15.107:0/2125327868 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.169+0000 7ff7a2ffd640 1 --2- 172.21.15.107:0/2125327868 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1703bd580 0x55f1703d4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.169+0000 7ff79bfff640 1 --2- 172.21.15.107:0/2125327868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1703bdea0 0x55f1703c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.169+0000 7ff7a37fe640 1 -- 172.21.15.107:0/2125327868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1703bdea0 msgr2=0x55f1703c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.169+0000 7ff7a37fe640 1 --2- 172.21.15.107:0/2125327868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1703bdea0 0x55f1703c7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.169+0000 7ff7a37fe640 1 -- 172.21.15.107:0/2125327868 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1703bd580 msgr2=0x55f1703d4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.169+0000 7ff7a37fe640 1 --2- 172.21.15.107:0/2125327868 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1703bd580 0x55f1703d4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.169+0000 7ff7a37fe640 1 -- 172.21.15.107:0/2125327868 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1701007d0 con 0x55f1703c8570 2024-04-25T00:19:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.169+0000 7ff7a37fe640 1 --2- 172.21.15.107:0/2125327868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703c8570 0x55f1703e0440 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff79c000f80 tx=0x7ff79c012570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efdfed22ed173d4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.169+0000 7ff7997fa640 1 -- 172.21.15.107:0/2125327868 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c012040 con 0x55f1703c8570 2024-04-25T00:19:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.170+0000 7ff7997fa640 1 -- 172.21.15.107:0/2125327868 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff79c024070 con 0x55f1703c8570 2024-04-25T00:19:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.170+0000 7ff7997fa640 1 -- 172.21.15.107:0/2125327868 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c020450 con 0x55f1703c8570 2024-04-25T00:19:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.170+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2125327868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703c8570 msgr2=0x55f1703e0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.170+0000 7ff7ac7f3740 1 --2- 172.21.15.107:0/2125327868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703c8570 0x55f1703e0440 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff79c000f80 tx=0x7ff79c012570 comp rx=0 tx=0).stop 2024-04-25T00:19:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.170+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2125327868 shutdown_connections 2024-04-25T00:19:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.170+0000 7ff7ac7f3740 1 --2- 172.21.15.107:0/2125327868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703c8570 0x55f1703e0440 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.170+0000 7ff7ac7f3740 1 --2- 172.21.15.107:0/2125327868 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1703bdea0 0x55f1703c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.170+0000 7ff7ac7f3740 1 --2- 172.21.15.107:0/2125327868 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1703bd580 0x55f1703d4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.170+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2125327868 >> 172.21.15.107:0/2125327868 conn(0x55f1703b98a0 msgr2=0x55f1703c3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.170+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2125327868 shutdown_connections 2024-04-25T00:19:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.170+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2125327868 wait complete. 2024-04-25T00:19:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.170+0000 7ff7ac7f3740 1 -- start start 2024-04-25T00:19:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.171+0000 7ff7ac7f3740 1 Processor -- start 2024-04-25T00:19:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.171+0000 7ff7ac7f3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1703bd580 0x55f1703d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.171+0000 7ff7ac7f3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1703bdea0 0x55f1703cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.171+0000 7ff7ac7f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703c8570 0x55f1703ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.171+0000 7ff7ac7f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41af3a0 con 0x55f1703c8570 2024-04-25T00:19:15.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.171+0000 7ff7ac7f3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41b0810 con 0x55f1703bd580 2024-04-25T00:19:15.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.171+0000 7ff7ac7f3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41afa70 con 0x55f1703bdea0 2024-04-25T00:19:15.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.171+0000 7ff7ac7f3740 10 client.0 _ll_register_callbacks cb 0x7ff78c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:15.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.171+0000 7ff7a37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703c8570 0x55f1703ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.171+0000 7ff79bfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1703bdea0 0x55f1703cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.171+0000 7ff7a2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1703bd580 0x55f1703d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.171+0000 7ff79bfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1703bdea0 0x55f1703cdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:43750/0 (socket says 172.21.15.107:43750) 2024-04-25T00:19:15.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.171+0000 7ff79bfff640 1 -- 172.21.15.107:0/2584297036 learned_addr learned my addr 172.21.15.107:0/2584297036 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:15.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.171+0000 7ff7a37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703c8570 0x55f1703ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40112/0 (socket says 172.21.15.107:40112) 2024-04-25T00:19:15.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.171+0000 7ff79bfff640 1 -- 172.21.15.107:0/2584297036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1703bd580 msgr2=0x55f1703d0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.171+0000 7ff79bfff640 1 --2- 172.21.15.107:0/2584297036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1703bd580 0x55f1703d0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.171+0000 7ff79bfff640 1 -- 172.21.15.107:0/2584297036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703c8570 msgr2=0x55f1703ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.171+0000 7ff79bfff640 1 --2- 172.21.15.107:0/2584297036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703c8570 0x55f1703ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.171+0000 7ff79bfff640 1 -- 172.21.15.107:0/2584297036 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1703ce880 con 0x55f1703bdea0 2024-04-25T00:19:15.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.171+0000 7ff7a37fe640 1 --2- 172.21.15.107:0/2584297036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703c8570 0x55f1703ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:15.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.171+0000 7ff79bfff640 1 --2- 172.21.15.107:0/2584297036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1703bdea0 0x55f1703cdd00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009900 tx=0x7ff78c0467b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bef34961b0fac51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.171+0000 7ff782ffd640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-04-25T00:19:15.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.172+0000 7ff782ffd640 1 -- 172.21.15.107:0/2584297036 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78c035070 con 0x55f1703bdea0 2024-04-25T00:19:15.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.172+0000 7ff782ffd640 1 -- 172.21.15.107:0/2584297036 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7ff78c004450 con 0x55f1703bdea0 2024-04-25T00:19:15.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.172+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2584297036 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff79c024070 con 0x55f1703bdea0 2024-04-25T00:19:15.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.172+0000 7ff782ffd640 1 -- 172.21.15.107:0/2584297036 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff78c005070 con 0x55f1703bdea0 2024-04-25T00:19:15.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.172+0000 7ff7ac7f3740 10 client.25609 fetch_fsmap learned FSMap version 28 2024-04-25T00:19:15.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.172+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2584297036 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f17043eb40 con 0x55f1703bdea0 2024-04-25T00:19:15.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.172+0000 7ff782ffd640 1 -- 172.21.15.107:0/2584297036 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff79c024070 con 0x55f1703bdea0 2024-04-25T00:19:15.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.172+0000 7ff782ffd640 1 -- 172.21.15.107:0/2584297036 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7ff78c0048f0 con 0x55f1703bdea0 2024-04-25T00:19:15.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.172+0000 7ff7ac7f3740 10 client.25609 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:19:15.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.172+0000 7ff7ac7f3740 10 client.25609 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:19:15.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.172+0000 7ff7ac7f3740 20 client.25609 populate_metadata read hostname 'smithi107' 2024-04-25T00:19:15.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.172+0000 7ff7ac7f3740 1 --2- 172.21.15.107:0/2584297036 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55f17043f2f0 0x55f170441710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.172+0000 7ff7ac7f3740 4 client.25609 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:19:15.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.172+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2584297036 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f170329f00 con 0x55f17043f2f0 2024-04-25T00:19:15.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.172+0000 7ff7a2ffd640 1 --2- 172.21.15.107:0/2584297036 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55f17043f2f0 0x55f170441710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.173+0000 7ff782ffd640 10 client.25609 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:19:15.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.173+0000 7ff7a2ffd640 1 --2- 172.21.15.107:0/2584297036 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55f17043f2f0 0x55f170441710 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb7d156bd421d670 server_cookie=57ad522509240e3a in_seq=0 out_seq=0 2024-04-25T00:19:15.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.173+0000 7ff782ffd640 1 -- 172.21.15.107:0/2584297036 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f170329f00 con 0x55f17043f2f0 2024-04-25T00:19:15.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.173+0000 7ff782ffd640 10 client.25609 handle_command_reply: tid=0 2024-04-25T00:19:15.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.173+0000 7ff7ac7f3740 1 client.25609 shutdown 2024-04-25T00:19:15.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.173+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2584297036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1703bdea0 msgr2=0x55f1703cdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.173+0000 7ff7ac7f3740 1 --2- 172.21.15.107:0/2584297036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1703bdea0 0x55f1703cdd00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff78c009900 tx=0x7ff78c0467b0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.173+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2584297036 shutdown_connections 2024-04-25T00:19:15.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.173+0000 7ff7ac7f3740 1 --2- 172.21.15.107:0/2584297036 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55f17043f2f0 0x55f170441710 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.173+0000 7ff7ac7f3740 1 --2- 172.21.15.107:0/2584297036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f1703c8570 0x55f1703ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.173+0000 7ff7ac7f3740 1 --2- 172.21.15.107:0/2584297036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f1703bdea0 0x55f1703cdd00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.173+0000 7ff782ffd640 0 client.25609 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:19:15.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.173+0000 7ff7ac7f3740 1 --2- 172.21.15.107:0/2584297036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f1703bd580 0x55f1703d0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.173+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2584297036 >> 172.21.15.107:0/2584297036 conn(0x55f1703b98a0 msgr2=0x55f1703c6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.173+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2584297036 shutdown_connections 2024-04-25T00:19:15.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.173+0000 7ff7ac7f3740 1 -- 172.21.15.107:0/2584297036 wait complete. 2024-04-25T00:19:15.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.173+0000 7ff7ac7f3740 20 client.25609 trim_cache size 0 max 16384 2024-04-25T00:19:15.181 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:19:15.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.174+0000 7ff7a9a10640 1 -- 172.21.15.107:0/2195401004 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff7900a2d30 msgr2=0x7ff7900a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.174+0000 7ff7a9a10640 1 --2- 172.21.15.107:0/2195401004 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff7900a2d30 0x7ff7900a51f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff79c01d000 tx=0x7ff79c01b000 comp rx=0 tx=0).stop 2024-04-25T00:19:15.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.174+0000 7ff7a9a10640 1 -- 172.21.15.107:0/2195401004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a41029a0 msgr2=0x7ff7a41005f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.174+0000 7ff7a9a10640 1 --2- 172.21.15.107:0/2195401004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a41029a0 0x7ff7a41005f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff79400e9f0 tx=0x7ff79400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.174+0000 7ff7a9a10640 1 -- 172.21.15.107:0/2195401004 shutdown_connections 2024-04-25T00:19:15.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.174+0000 7ff7a9a10640 1 --2- 172.21.15.107:0/2195401004 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff7a410bd70 0x7ff7a4101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.174+0000 7ff7a9a10640 1 --2- 172.21.15.107:0/2195401004 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff7900a2d30 0x7ff7900a51f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.174+0000 7ff7a9a10640 1 --2- 172.21.15.107:0/2195401004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7a41032c0 0x7ff7a4100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.174+0000 7ff7a9a10640 1 --2- 172.21.15.107:0/2195401004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7a41029a0 0x7ff7a41005f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.174+0000 7ff7a9a10640 1 -- 172.21.15.107:0/2195401004 >> 172.21.15.107:0/2195401004 conn(0x7ff7a40769c0 msgr2=0x7ff7a40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.174+0000 7ff7a9a10640 1 -- 172.21.15.107:0/2195401004 shutdown_connections 2024-04-25T00:19:15.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.174+0000 7ff7a9a10640 1 -- 172.21.15.107:0/2195401004 wait complete. 2024-04-25T00:19:15.184 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:19:15.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.287+0000 7fb9f4f10640 1 -- 172.21.15.107:0/444776646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f0100610 msgr2=0x7fb9f0100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.287+0000 7fb9f4f10640 1 --2- 172.21.15.107:0/444776646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f0100610 0x7fb9f0100a90 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8009930 tx=0x7fb9d802a420 comp rx=0 tx=0).stop 2024-04-25T00:19:15.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.287+0000 7fb9f4f10640 1 -- 172.21.15.107:0/444776646 shutdown_connections 2024-04-25T00:19:15.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.287+0000 7fb9f4f10640 1 --2- 172.21.15.107:0/444776646 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9f0100fd0 0x7fb9f010d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.287+0000 7fb9f4f10640 1 --2- 172.21.15.107:0/444776646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f0100610 0x7fb9f0100a90 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.287+0000 7fb9f4f10640 1 --2- 172.21.15.107:0/444776646 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9f01066b0 0x7fb9f0106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.287+0000 7fb9f4f10640 1 -- 172.21.15.107:0/444776646 >> 172.21.15.107:0/444776646 conn(0x7fb9f0077740 msgr2=0x7fb9f0077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.287+0000 7fb9f4f10640 1 -- 172.21.15.107:0/444776646 shutdown_connections 2024-04-25T00:19:15.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.287+0000 7fb9f4f10640 1 -- 172.21.15.107:0/444776646 wait complete. 2024-04-25T00:19:15.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.288+0000 7fb9f4f10640 1 Processor -- start 2024-04-25T00:19:15.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.288+0000 7fb9f4f10640 1 -- start start 2024-04-25T00:19:15.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.288+0000 7fb9f4f10640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9f0100610 0x7fb9f0196ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.288+0000 7fb9f4f10640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f0100fd0 0x7fb9f0197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.288+0000 7fb9f4f10640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9f01066b0 0x7fb9f019b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.288+0000 7fb9f4f10640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f0110a80 con 0x7fb9f0100fd0 2024-04-25T00:19:15.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.288+0000 7fb9f4f10640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f0110900 con 0x7fb9f01066b0 2024-04-25T00:19:15.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.288+0000 7fb9f4f10640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f0110c00 con 0x7fb9f0100610 2024-04-25T00:19:15.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.288+0000 7fb9e7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f0100fd0 0x7fb9f0197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.288+0000 7fb9e7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f0100fd0 0x7fb9f0197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40118/0 (socket says 172.21.15.107:40118) 2024-04-25T00:19:15.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.288+0000 7fb9e7fff640 1 -- 172.21.15.107:0/624991110 learned_addr learned my addr 172.21.15.107:0/624991110 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:15.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.288+0000 7fb9eed9d640 1 --2- 172.21.15.107:0/624991110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9f01066b0 0x7fb9f019b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.288+0000 7fb9ee59c640 1 --2- 172.21.15.107:0/624991110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9f0100610 0x7fb9f0196ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.288+0000 7fb9e7fff640 1 -- 172.21.15.107:0/624991110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9f0100610 msgr2=0x7fb9f0196ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.288+0000 7fb9e7fff640 1 --2- 172.21.15.107:0/624991110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9f0100610 0x7fb9f0196ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.288+0000 7fb9e7fff640 1 -- 172.21.15.107:0/624991110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9f01066b0 msgr2=0x7fb9f019b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.288+0000 7fb9e7fff640 1 --2- 172.21.15.107:0/624991110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9f01066b0 0x7fb9f019b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.288+0000 7fb9e7fff640 1 -- 172.21.15.107:0/624991110 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f019bed0 con 0x7fb9f0100fd0 2024-04-25T00:19:15.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.289+0000 7fb9e7fff640 1 --2- 172.21.15.107:0/624991110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f0100fd0 0x7fb9f0197420 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb9d802a940 tx=0x7fb9d8003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.289+0000 7fb9e77fe640 1 -- 172.21.15.107:0/624991110 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d803f070 con 0x7fb9f0100fd0 2024-04-25T00:19:15.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.289+0000 7fb9e77fe640 1 -- 172.21.15.107:0/624991110 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9d8031c90 con 0x7fb9f0100fd0 2024-04-25T00:19:15.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.289+0000 7fb9f4f10640 1 -- 172.21.15.107:0/624991110 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9f019c160 con 0x7fb9f0100fd0 2024-04-25T00:19:15.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.289+0000 7fb9e77fe640 1 -- 172.21.15.107:0/624991110 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8035030 con 0x7fb9f0100fd0 2024-04-25T00:19:15.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.289+0000 7fb9f4f10640 1 -- 172.21.15.107:0/624991110 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9f01a3a50 con 0x7fb9f0100fd0 2024-04-25T00:19:15.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.292+0000 7fb9e77fe640 1 -- 172.21.15.107:0/624991110 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb9d8042070 con 0x7fb9f0100fd0 2024-04-25T00:19:15.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.292+0000 7fb9f7c1d740 1 Processor -- start 2024-04-25T00:19:15.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.292+0000 7fb9f7c1d740 1 -- start start 2024-04-25T00:19:15.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.292+0000 7fb9f7c1d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810250 0x563219810630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.292+0000 7fb9f7c1d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563219810b70 0x56321981d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.292+0000 7fb9f7c1d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56321981d780 0x56321981fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.292+0000 7fb9f7c1d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563219775670 con 0x563219810250 2024-04-25T00:19:15.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.292+0000 7fb9f7c1d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56321971d970 con 0x563219810b70 2024-04-25T00:19:15.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.292+0000 7fb9f7c1d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563219771630 con 0x56321981d780 2024-04-25T00:19:15.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.292+0000 7fb9eed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810250 0x563219810630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.292+0000 7fb9ee59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563219810b70 0x56321981d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.292+0000 7fb9eed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810250 0x563219810630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40124/0 (socket says 172.21.15.107:40124) 2024-04-25T00:19:15.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.292+0000 7fb9eed9d640 1 -- 172.21.15.107:0/2057202142 learned_addr learned my addr 172.21.15.107:0/2057202142 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:15.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.292+0000 7fb9e77fe640 1 --2- 172.21.15.107:0/624991110 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb9dc0a2d80 0x7fb9dc0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.292+0000 7fb9e7fff640 1 --2- 172.21.15.107:0/2057202142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56321981d780 0x56321981fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.292+0000 7fb9ee59c640 1 -- 172.21.15.107:0/2057202142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56321981d780 msgr2=0x56321981fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.292+0000 7fb9ee59c640 1 --2- 172.21.15.107:0/2057202142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56321981d780 0x56321981fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.292+0000 7fb9ee59c640 1 -- 172.21.15.107:0/2057202142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810250 msgr2=0x563219810630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.292+0000 7fb9ee59c640 1 --2- 172.21.15.107:0/2057202142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810250 0x563219810630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.292+0000 7fb9ee59c640 1 -- 172.21.15.107:0/2057202142 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563219741400 con 0x563219810b70 2024-04-25T00:19:15.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.292+0000 7fb9e7fff640 1 --2- 172.21.15.107:0/2057202142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56321981d780 0x56321981fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:15.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.292+0000 7fb9eed9d640 1 --2- 172.21.15.107:0/2057202142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810250 0x563219810630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:15.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.293+0000 7fb9eed9d640 1 --2- 172.21.15.107:0/624991110 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb9dc0a2d80 0x7fb9dc0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.293+0000 7fb9e77fe640 1 -- 172.21.15.107:0/624991110 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fb9d80efe00 con 0x7fb9f0100fd0 2024-04-25T00:19:15.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.293+0000 7fb9ee59c640 1 --2- 172.21.15.107:0/2057202142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563219810b70 0x56321981d240 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0009b60 tx=0x7fb9e0010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69bd32036149f438 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.293+0000 7fb9e4ff9640 1 -- 172.21.15.107:0/2057202142 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e0009e50 con 0x563219810b70 2024-04-25T00:19:15.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.293+0000 7fb9eed9d640 1 --2- 172.21.15.107:0/624991110 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb9dc0a2d80 0x7fb9dc0a5240 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8003df0 tx=0x7fb9e8004a40 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.293+0000 7fb9e4ff9640 1 -- 172.21.15.107:0/2057202142 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9e001b960 con 0x563219810b70 2024-04-25T00:19:15.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.293+0000 7fb9e4ff9640 1 -- 172.21.15.107:0/2057202142 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e0012a60 con 0x563219810b70 2024-04-25T00:19:15.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.294+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/2057202142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563219810b70 msgr2=0x56321981d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.294+0000 7fb9f7c1d740 1 --2- 172.21.15.107:0/2057202142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563219810b70 0x56321981d240 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0009b60 tx=0x7fb9e0010740 comp rx=0 tx=0).stop 2024-04-25T00:19:15.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.294+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/2057202142 shutdown_connections 2024-04-25T00:19:15.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.294+0000 7fb9f7c1d740 1 --2- 172.21.15.107:0/2057202142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56321981d780 0x56321981fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.294+0000 7fb9f7c1d740 1 --2- 172.21.15.107:0/2057202142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563219810b70 0x56321981d240 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.294+0000 7fb9f7c1d740 1 --2- 172.21.15.107:0/2057202142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810250 0x563219810630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.294+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/2057202142 >> 172.21.15.107:0/2057202142 conn(0x56321980c8a0 msgr2=0x56321980cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.294+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/2057202142 shutdown_connections 2024-04-25T00:19:15.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.294+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/2057202142 wait complete. 2024-04-25T00:19:15.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.294+0000 7fb9f7c1d740 1 -- start start 2024-04-25T00:19:15.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.294+0000 7fb9f7c1d740 1 Processor -- start 2024-04-25T00:19:15.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.294+0000 7fb9f7c1d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563219810250 0x56321981d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.294+0000 7fb9f7c1d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810b70 0x56321981a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.294+0000 7fb9f7c1d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56321981d780 0x56321981a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.294+0000 7fb9f7c1d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563219820540 con 0x563219810b70 2024-04-25T00:19:15.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.294+0000 7fb9f7c1d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563219731050 con 0x563219810250 2024-04-25T00:19:15.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.294+0000 7fb9f7c1d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5632198206c0 con 0x56321981d780 2024-04-25T00:19:15.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.294+0000 7fb9f7c1d740 10 client.0 _ll_register_callbacks cb 0x563219488be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:15.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.294+0000 7fb9e7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810b70 0x56321981a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.294+0000 7fb9ee59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563219810250 0x56321981d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.294+0000 7fb9e7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810b70 0x56321981a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40128/0 (socket says 172.21.15.107:40128) 2024-04-25T00:19:15.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.294+0000 7fb9e7fff640 1 -- 172.21.15.107:0/3801409585 learned_addr learned my addr 172.21.15.107:0/3801409585 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:15.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.294+0000 7fb9eed9d640 1 --2- 172.21.15.107:0/3801409585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56321981d780 0x56321981a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.295+0000 7fb9e7fff640 1 -- 172.21.15.107:0/3801409585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56321981d780 msgr2=0x56321981a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.295+0000 7fb9e7fff640 1 --2- 172.21.15.107:0/3801409585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56321981d780 0x56321981a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.295+0000 7fb9e7fff640 1 -- 172.21.15.107:0/3801409585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563219810250 msgr2=0x56321981d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.295+0000 7fb9e7fff640 1 --2- 172.21.15.107:0/3801409585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563219810250 0x56321981d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.295+0000 7fb9e7fff640 1 -- 172.21.15.107:0/3801409585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563219702dc0 con 0x563219810b70 2024-04-25T00:19:15.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.295+0000 7fb9e7fff640 1 --2- 172.21.15.107:0/3801409585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810b70 0x56321981a3d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80ed7e0 tx=0x7fb9d8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d914085f8016b943 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.295+0000 7fb9d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:19:15.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.295+0000 7fb9d5ffb640 1 -- 172.21.15.107:0/3801409585 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8049070 con 0x563219810b70 2024-04-25T00:19:15.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.295+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/3801409585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9e001b960 con 0x563219810b70 2024-04-25T00:19:15.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.295+0000 7fb9d5ffb640 1 -- 172.21.15.107:0/3801409585 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fb9d80b8620 con 0x563219810b70 2024-04-25T00:19:15.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.295+0000 7fb9d5ffb640 1 -- 172.21.15.107:0/3801409585 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb9d80bc6b0 con 0x563219810b70 2024-04-25T00:19:15.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.295+0000 7fb9f7c1d740 10 client.16230 fetch_fsmap learned FSMap version 28 2024-04-25T00:19:15.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.295+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/3801409585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5632194893f0 con 0x563219810b70 2024-04-25T00:19:15.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.295+0000 7fb9d5ffb640 1 -- 172.21.15.107:0/3801409585 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9e001b960 con 0x563219810b70 2024-04-25T00:19:15.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.295+0000 7fb9d5ffb640 1 -- 172.21.15.107:0/3801409585 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fb9d80c4790 con 0x563219810b70 2024-04-25T00:19:15.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.296+0000 7fb9f7c1d740 10 client.16230 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:19:15.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.296+0000 7fb9f7c1d740 10 client.16230 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:19:15.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.296+0000 7fb9f7c1d740 20 client.16230 populate_metadata read hostname 'smithi107' 2024-04-25T00:19:15.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.296+0000 7fb9f7c1d740 1 --2- 172.21.15.107:0/3801409585 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x56321982d770 0x56321982fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.296+0000 7fb9f7c1d740 4 client.16230 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:19:15.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.296+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/3801409585 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563219782bb0 con 0x56321982d770 2024-04-25T00:19:15.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.296+0000 7fb9ee59c640 1 --2- 172.21.15.107:0/3801409585 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x56321982d770 0x56321982fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.296+0000 7fb9d5ffb640 10 client.16230 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:19:15.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.296+0000 7fb9ee59c640 1 --2- 172.21.15.107:0/3801409585 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x56321982d770 0x56321982fbf0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=63b33d380a2508b3 server_cookie=eb44c1b6b10425b7 in_seq=0 out_seq=0 2024-04-25T00:19:15.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.297+0000 7fb9d5ffb640 1 -- 172.21.15.107:0/3801409585 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563219782bb0 con 0x56321982d770 2024-04-25T00:19:15.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.297+0000 7fb9d5ffb640 10 client.16230 handle_command_reply: tid=0 2024-04-25T00:19:15.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.297+0000 7fb9f7c1d740 1 client.16230 shutdown 2024-04-25T00:19:15.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.297+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/3801409585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810b70 msgr2=0x56321981a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.297+0000 7fb9f7c1d740 1 --2- 172.21.15.107:0/3801409585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810b70 0x56321981a3d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80ed7e0 tx=0x7fb9d8009260 comp rx=0 tx=0).stop 2024-04-25T00:19:15.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.297+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/3801409585 shutdown_connections 2024-04-25T00:19:15.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.297+0000 7fb9f7c1d740 1 --2- 172.21.15.107:0/3801409585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56321981d780 0x56321981a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.297+0000 7fb9f7c1d740 1 --2- 172.21.15.107:0/3801409585 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x56321982d770 0x56321982fbf0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.297+0000 7fb9f7c1d740 1 --2- 172.21.15.107:0/3801409585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810b70 0x56321981a3d0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.297+0000 7fb9d5ffb640 0 client.16230 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:19:15.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.297+0000 7fb9f7c1d740 1 --2- 172.21.15.107:0/3801409585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563219810250 0x56321981d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.297+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/3801409585 >> 172.21.15.107:0/3801409585 conn(0x56321980c8a0 msgr2=0x563219812710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.297+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/3801409585 shutdown_connections 2024-04-25T00:19:15.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.297+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/3801409585 wait complete. 2024-04-25T00:19:15.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.297+0000 7fb9f7c1d740 20 client.16230 trim_cache size 0 max 16384 2024-04-25T00:19:15.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.311+0000 7fb9f7c1d740 1 Processor -- start 2024-04-25T00:19:15.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.311+0000 7fb9f7c1d740 1 -- start start 2024-04-25T00:19:15.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.311+0000 7fb9f7c1d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563219810250 0x563219829470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.311+0000 7fb9f7c1d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810b70 0x56321981cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.311+0000 7fb9f7c1d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56321981d780 0x5632198354c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.311+0000 7fb9f7c1d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f01af220 con 0x563219810b70 2024-04-25T00:19:15.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.311+0000 7fb9f7c1d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f01aca30 con 0x563219810250 2024-04-25T00:19:15.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.311+0000 7fb9f7c1d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f01aa880 con 0x56321981d780 2024-04-25T00:19:15.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.311+0000 7fb9e7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810b70 0x56321981cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.311+0000 7fb9e7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810b70 0x56321981cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40136/0 (socket says 172.21.15.107:40136) 2024-04-25T00:19:15.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.311+0000 7fb9e7fff640 1 -- 172.21.15.107:0/2008182230 learned_addr learned my addr 172.21.15.107:0/2008182230 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:15.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.311+0000 7fb9ee59c640 1 --2- 172.21.15.107:0/2008182230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563219810250 0x563219829470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.311+0000 7fb9eed9d640 1 --2- 172.21.15.107:0/2008182230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56321981d780 0x5632198354c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.311+0000 7fb9e7fff640 1 -- 172.21.15.107:0/2008182230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56321981d780 msgr2=0x5632198354c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.311+0000 7fb9e7fff640 1 --2- 172.21.15.107:0/2008182230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56321981d780 0x5632198354c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.311+0000 7fb9e7fff640 1 -- 172.21.15.107:0/2008182230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563219810250 msgr2=0x563219829470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.311+0000 7fb9e7fff640 1 --2- 172.21.15.107:0/2008182230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563219810250 0x563219829470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.311+0000 7fb9e7fff640 1 -- 172.21.15.107:0/2008182230 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632195537d0 con 0x563219810b70 2024-04-25T00:19:15.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.311+0000 7fb9ee59c640 1 --2- 172.21.15.107:0/2008182230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563219810250 0x563219829470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:15.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.311+0000 7fb9e7fff640 1 --2- 172.21.15.107:0/2008182230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810b70 0x56321981cc10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80bbf40 tx=0x7fb9d8003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46af25518ebacd29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.311+0000 7fb9d7fff640 1 -- 172.21.15.107:0/2008182230 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d809e070 con 0x563219810b70 2024-04-25T00:19:15.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.312+0000 7fb9d7fff640 1 -- 172.21.15.107:0/2008182230 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9d80c0360 con 0x563219810b70 2024-04-25T00:19:15.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.312+0000 7fb9d7fff640 1 -- 172.21.15.107:0/2008182230 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8049030 con 0x563219810b70 2024-04-25T00:19:15.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.312+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/2008182230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810b70 msgr2=0x56321981cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.312+0000 7fb9f7c1d740 1 --2- 172.21.15.107:0/2008182230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810b70 0x56321981cc10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb9d80bbf40 tx=0x7fb9d8003f50 comp rx=0 tx=0).stop 2024-04-25T00:19:15.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.312+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/2008182230 shutdown_connections 2024-04-25T00:19:15.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.312+0000 7fb9f7c1d740 1 --2- 172.21.15.107:0/2008182230 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56321981d780 0x5632198354c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.312+0000 7fb9f7c1d740 1 --2- 172.21.15.107:0/2008182230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810b70 0x56321981cc10 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.312+0000 7fb9f7c1d740 1 --2- 172.21.15.107:0/2008182230 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563219810250 0x563219829470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.312+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/2008182230 >> 172.21.15.107:0/2008182230 conn(0x56321980c8a0 msgr2=0x563219815a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.312+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/2008182230 shutdown_connections 2024-04-25T00:19:15.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.312+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/2008182230 wait complete. 2024-04-25T00:19:15.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.312+0000 7fb9f7c1d740 1 -- start start 2024-04-25T00:19:15.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.312+0000 7fb9f7c1d740 1 Processor -- start 2024-04-25T00:19:15.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.313+0000 7fb9f7c1d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810250 0x5632198173f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.313+0000 7fb9f7c1d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563219810b70 0x563219817930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.313+0000 7fb9f7c1d740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56321981d780 0x563219892d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.313+0000 7fb9f7c1d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f01aaef0 con 0x563219810250 2024-04-25T00:19:15.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.313+0000 7fb9f7c1d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f01ac360 con 0x563219810b70 2024-04-25T00:19:15.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.313+0000 7fb9f7c1d740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f01ab5c0 con 0x56321981d780 2024-04-25T00:19:15.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.313+0000 7fb9f7c1d740 10 client.0 _ll_register_callbacks cb 0x7fb9d8005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:15.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.313+0000 7fb9ee59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810250 0x5632198173f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.313+0000 7fb9ee59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810250 0x5632198173f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40146/0 (socket says 172.21.15.107:40146) 2024-04-25T00:19:15.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.313+0000 7fb9e7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563219810b70 0x563219817930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.313+0000 7fb9ee59c640 1 -- 172.21.15.107:0/745210789 learned_addr learned my addr 172.21.15.107:0/745210789 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:15.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.313+0000 7fb9eed9d640 1 --2- 172.21.15.107:0/745210789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56321981d780 0x563219892d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.313+0000 7fb9ee59c640 1 -- 172.21.15.107:0/745210789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56321981d780 msgr2=0x563219892d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.313+0000 7fb9ee59c640 1 --2- 172.21.15.107:0/745210789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56321981d780 0x563219892d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.313+0000 7fb9ee59c640 1 -- 172.21.15.107:0/745210789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563219810b70 msgr2=0x563219817930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.313+0000 7fb9ee59c640 1 --2- 172.21.15.107:0/745210789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563219810b70 0x563219817930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.313+0000 7fb9ee59c640 1 -- 172.21.15.107:0/745210789 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563219893270 con 0x563219810250 2024-04-25T00:19:15.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.313+0000 7fb9ee59c640 1 --2- 172.21.15.107:0/745210789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810250 0x5632198173f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0009b60 tx=0x7fb9e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8a4dd2b1cf3def server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.313+0000 7fb9d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:19:15.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.313+0000 7fb9d5ffb640 1 -- 172.21.15.107:0/745210789 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e002f020 con 0x563219810250 2024-04-25T00:19:15.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.313+0000 7fb9d5ffb640 1 -- 172.21.15.107:0/745210789 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fb9e0004240 con 0x563219810250 2024-04-25T00:19:15.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.313+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/745210789 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9d80c0360 con 0x563219810250 2024-04-25T00:19:15.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.314+0000 7fb9d5ffb640 1 -- 172.21.15.107:0/745210789 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb9e003a050 con 0x563219810250 2024-04-25T00:19:15.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.314+0000 7fb9f7c1d740 10 client.16236 fetch_fsmap learned FSMap version 28 2024-04-25T00:19:15.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.314+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/745210789 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563219894370 con 0x563219810250 2024-04-25T00:19:15.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.314+0000 7fb9d5ffb640 1 -- 172.21.15.107:0/745210789 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9d80c0360 con 0x563219810250 2024-04-25T00:19:15.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.314+0000 7fb9d5ffb640 1 -- 172.21.15.107:0/745210789 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fb9e000f040 con 0x563219810250 2024-04-25T00:19:15.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.314+0000 7fb9f7c1d740 10 client.16236 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:19:15.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.314+0000 7fb9f7c1d740 10 client.16236 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:19:15.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.314+0000 7fb9f7c1d740 20 client.16236 populate_metadata read hostname 'smithi107' 2024-04-25T00:19:15.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.314+0000 7fb9f7c1d740 1 --2- 172.21.15.107:0/745210789 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5632198948e0 0x563219896d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.314+0000 7fb9f7c1d740 4 client.16236 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:19:15.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.314+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/745210789 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56321977cf00 con 0x5632198948e0 2024-04-25T00:19:15.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.314+0000 7fb9e7fff640 1 --2- 172.21.15.107:0/745210789 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5632198948e0 0x563219896d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.314+0000 7fb9e7fff640 1 --2- 172.21.15.107:0/745210789 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5632198948e0 0x563219896d00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=700d5d69baabb82d server_cookie=8e9bb3a5e5f23dcf in_seq=0 out_seq=0 2024-04-25T00:19:15.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.314+0000 7fb9d5ffb640 10 client.16236 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:19:15.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.315+0000 7fb9d5ffb640 1 -- 172.21.15.107:0/745210789 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56321977cf00 con 0x5632198948e0 2024-04-25T00:19:15.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.315+0000 7fb9d5ffb640 10 client.16236 handle_command_reply: tid=0 2024-04-25T00:19:15.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.315+0000 7fb9f7c1d740 1 client.16236 shutdown 2024-04-25T00:19:15.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.315+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/745210789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810250 msgr2=0x5632198173f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.315+0000 7fb9f7c1d740 1 --2- 172.21.15.107:0/745210789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810250 0x5632198173f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0009b60 tx=0x7fb9e00046c0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.315+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/745210789 shutdown_connections 2024-04-25T00:19:15.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.315+0000 7fb9f7c1d740 1 --2- 172.21.15.107:0/745210789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56321981d780 0x563219892d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.315+0000 7fb9f7c1d740 1 --2- 172.21.15.107:0/745210789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563219810b70 0x563219817930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.315+0000 7fb9f7c1d740 1 --2- 172.21.15.107:0/745210789 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5632198948e0 0x563219896d00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.315+0000 7fb9f7c1d740 1 --2- 172.21.15.107:0/745210789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563219810250 0x5632198173f0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.315+0000 7fb9d5ffb640 0 client.16236 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:19:15.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.315+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/745210789 >> 172.21.15.107:0/745210789 conn(0x56321980c8a0 msgr2=0x563219811e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.315+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/745210789 shutdown_connections 2024-04-25T00:19:15.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.316+0000 7fb9f7c1d740 1 -- 172.21.15.107:0/745210789 wait complete. 2024-04-25T00:19:15.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.316+0000 7fb9f7c1d740 20 client.16236 trim_cache size 0 max 16384 2024-04-25T00:19:15.323 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:19:15.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.316+0000 7fb9f4f10640 1 -- 172.21.15.107:0/624991110 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb9dc0a2d80 msgr2=0x7fb9dc0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.316+0000 7fb9f4f10640 1 --2- 172.21.15.107:0/624991110 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb9dc0a2d80 0x7fb9dc0a5240 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb9e8003df0 tx=0x7fb9e8004a40 comp rx=0 tx=0).stop 2024-04-25T00:19:15.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.316+0000 7fb9f4f10640 1 -- 172.21.15.107:0/624991110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f0100fd0 msgr2=0x7fb9f0197420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.316+0000 7fb9f4f10640 1 --2- 172.21.15.107:0/624991110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f0100fd0 0x7fb9f0197420 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb9d802a940 tx=0x7fb9d8003b20 comp rx=0 tx=0).stop 2024-04-25T00:19:15.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.316+0000 7fb9f4f10640 1 -- 172.21.15.107:0/624991110 shutdown_connections 2024-04-25T00:19:15.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.316+0000 7fb9f4f10640 1 --2- 172.21.15.107:0/624991110 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb9dc0a2d80 0x7fb9dc0a5240 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.316+0000 7fb9f4f10640 1 --2- 172.21.15.107:0/624991110 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb9f01066b0 0x7fb9f019b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.316+0000 7fb9f4f10640 1 --2- 172.21.15.107:0/624991110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9f0100fd0 0x7fb9f0197420 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.316+0000 7fb9f4f10640 1 --2- 172.21.15.107:0/624991110 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9f0100610 0x7fb9f0196ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.316+0000 7fb9f4f10640 1 -- 172.21.15.107:0/624991110 >> 172.21.15.107:0/624991110 conn(0x7fb9f0077740 msgr2=0x7fb9f00ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.316+0000 7fb9f4f10640 1 -- 172.21.15.107:0/624991110 shutdown_connections 2024-04-25T00:19:15.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.316+0000 7fb9f4f10640 1 -- 172.21.15.107:0/624991110 wait complete. 2024-04-25T00:19:15.327 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:19:15.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.430+0000 7fb61445d640 1 -- 172.21.15.107:0/893635458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb60c1077a0 msgr2=0x7fb60c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.430+0000 7fb61445d640 1 --2- 172.21.15.107:0/893635458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb60c1077a0 0x7fb60c107b80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc009930 tx=0x7fb5fc02a420 comp rx=0 tx=0).stop 2024-04-25T00:19:15.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.430+0000 7fb61445d640 1 -- 172.21.15.107:0/893635458 shutdown_connections 2024-04-25T00:19:15.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.430+0000 7fb61445d640 1 --2- 172.21.15.107:0/893635458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb60c102150 0x7fb60c10e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.430+0000 7fb61445d640 1 --2- 172.21.15.107:0/893635458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb60c101790 0x7fb60c101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.430+0000 7fb61445d640 1 --2- 172.21.15.107:0/893635458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb60c1077a0 0x7fb60c107b80 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.430+0000 7fb61445d640 1 -- 172.21.15.107:0/893635458 >> 172.21.15.107:0/893635458 conn(0x7fb60c0fd2a0 msgr2=0x7fb60c0ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.430+0000 7fb61445d640 1 -- 172.21.15.107:0/893635458 shutdown_connections 2024-04-25T00:19:15.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.431+0000 7fb61445d640 1 -- 172.21.15.107:0/893635458 wait complete. 2024-04-25T00:19:15.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.431+0000 7fb61445d640 1 Processor -- start 2024-04-25T00:19:15.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.431+0000 7fb61445d640 1 -- start start 2024-04-25T00:19:15.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.431+0000 7fb61445d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb60c101790 0x7fb60c073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.431+0000 7fb61445d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb60c102150 0x7fb60c073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.431+0000 7fb61445d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb60c1077a0 0x7fb60c0744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.431+0000 7fb61445d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c111bb0 con 0x7fb60c101790 2024-04-25T00:19:15.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.431+0000 7fb61445d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c111a30 con 0x7fb60c1077a0 2024-04-25T00:19:15.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.431+0000 7fb61445d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c111d30 con 0x7fb60c102150 2024-04-25T00:19:15.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.431+0000 7fb6121f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb60c101790 0x7fb60c073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.431+0000 7fb6121f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb60c101790 0x7fb60c073a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40164/0 (socket says 172.21.15.107:40164) 2024-04-25T00:19:15.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.431+0000 7fb6121f9640 1 -- 172.21.15.107:0/3984307028 learned_addr learned my addr 172.21.15.107:0/3984307028 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:15.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.431+0000 7fb6119f8640 1 --2- 172.21.15.107:0/3984307028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb60c102150 0x7fb60c073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.431+0000 7fb6129fa640 1 --2- 172.21.15.107:0/3984307028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb60c1077a0 0x7fb60c0744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.432+0000 7fb6121f9640 1 -- 172.21.15.107:0/3984307028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb60c102150 msgr2=0x7fb60c073f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.432+0000 7fb6121f9640 1 --2- 172.21.15.107:0/3984307028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb60c102150 0x7fb60c073f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.432+0000 7fb6121f9640 1 -- 172.21.15.107:0/3984307028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb60c1077a0 msgr2=0x7fb60c0744c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.432+0000 7fb6121f9640 1 --2- 172.21.15.107:0/3984307028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb60c1077a0 0x7fb60c0744c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.432+0000 7fb6121f9640 1 -- 172.21.15.107:0/3984307028 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60c1a6550 con 0x7fb60c101790 2024-04-25T00:19:15.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.432+0000 7fb6121f9640 1 --2- 172.21.15.107:0/3984307028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb60c101790 0x7fb60c073a40 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc009230 tx=0x7fb5fc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.432+0000 7fb6037fe640 1 -- 172.21.15.107:0/3984307028 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc03f030 con 0x7fb60c101790 2024-04-25T00:19:15.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.432+0000 7fb61445d640 1 -- 172.21.15.107:0/3984307028 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb60c1a67e0 con 0x7fb60c101790 2024-04-25T00:19:15.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.432+0000 7fb61445d640 1 -- 172.21.15.107:0/3984307028 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb60c1a6d20 con 0x7fb60c101790 2024-04-25T00:19:15.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.432+0000 7fb6037fe640 1 -- 172.21.15.107:0/3984307028 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5fc004440 con 0x7fb60c101790 2024-04-25T00:19:15.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.432+0000 7fb6037fe640 1 -- 172.21.15.107:0/3984307028 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc035070 con 0x7fb60c101790 2024-04-25T00:19:15.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.434+0000 7fb6037fe640 1 -- 172.21.15.107:0/3984307028 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb5fc042030 con 0x7fb60c101790 2024-04-25T00:19:15.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.435+0000 7fb61716c740 1 Processor -- start 2024-04-25T00:19:15.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.435+0000 7fb61716c740 1 -- start start 2024-04-25T00:19:15.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.435+0000 7fb61716c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d98d60e580 0x55d98d60e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.435+0000 7fb61716c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60eea0 0x55d98d619030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.435+0000 7fb61716c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d98d619570 0x55d98d61b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.435+0000 7fb61716c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d98d573670 con 0x55d98d60eea0 2024-04-25T00:19:15.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.435+0000 7fb61716c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d98d51b970 con 0x55d98d60e580 2024-04-25T00:19:15.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.435+0000 7fb61716c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d98d56f630 con 0x55d98d619570 2024-04-25T00:19:15.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.435+0000 7fb6037fe640 1 --2- 172.21.15.107:0/3984307028 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb5f80a2db0 0x7fb5f80a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.435+0000 7fb6121f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60eea0 0x55d98d619030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.435+0000 7fb6121f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60eea0 0x55d98d619030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40168/0 (socket says 172.21.15.107:40168) 2024-04-25T00:19:15.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.435+0000 7fb6121f9640 1 -- 172.21.15.107:0/3235552256 learned_addr learned my addr 172.21.15.107:0/3235552256 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:15.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.435+0000 7fb6129fa640 1 --2- 172.21.15.107:0/3235552256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d98d60e580 0x55d98d60e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.435+0000 7fb6129fa640 1 --2- 172.21.15.107:0/3984307028 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb5f80a2db0 0x7fb5f80a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.435+0000 7fb6119f8640 1 --2- 172.21.15.107:0/3235552256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d98d619570 0x55d98d61b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.435+0000 7fb6037fe640 1 -- 172.21.15.107:0/3984307028 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fb5fc0efa10 con 0x7fb60c101790 2024-04-25T00:19:15.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.435+0000 7fb6129fa640 1 -- 172.21.15.107:0/3235552256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d98d619570 msgr2=0x55d98d61b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.435+0000 7fb6129fa640 1 --2- 172.21.15.107:0/3235552256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d98d619570 0x55d98d61b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.436+0000 7fb6129fa640 1 -- 172.21.15.107:0/3235552256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60eea0 msgr2=0x55d98d619030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.436+0000 7fb6129fa640 1 --2- 172.21.15.107:0/3235552256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60eea0 0x55d98d619030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.436+0000 7fb6129fa640 1 -- 172.21.15.107:0/3235552256 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d98d53f400 con 0x55d98d60e580 2024-04-25T00:19:15.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.436+0000 7fb6121f9640 1 --2- 172.21.15.107:0/3235552256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60eea0 0x55d98d619030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:15.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.436+0000 7fb6129fa640 1 --2- 172.21.15.107:0/3235552256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d98d60e580 0x55d98d60e960 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb604011cc0 tx=0x7fb60400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d42ac5e737a7e12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.436+0000 7fb6129fa640 1 --2- 172.21.15.107:0/3984307028 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb5f80a2db0 0x7fb5f80a5270 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb60400fc90 tx=0x7fb604018000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.436+0000 7fb600bfe640 1 -- 172.21.15.107:0/3235552256 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb604004b20 con 0x55d98d60e580 2024-04-25T00:19:15.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.436+0000 7fb600bfe640 1 -- 172.21.15.107:0/3235552256 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb604009ce0 con 0x55d98d60e580 2024-04-25T00:19:15.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.436+0000 7fb600bfe640 1 -- 172.21.15.107:0/3235552256 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb604005490 con 0x55d98d60e580 2024-04-25T00:19:15.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.437+0000 7fb61716c740 1 -- 172.21.15.107:0/3235552256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d98d60e580 msgr2=0x55d98d60e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.437+0000 7fb61716c740 1 --2- 172.21.15.107:0/3235552256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d98d60e580 0x55d98d60e960 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb604011cc0 tx=0x7fb60400f790 comp rx=0 tx=0).stop 2024-04-25T00:19:15.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.437+0000 7fb61716c740 1 -- 172.21.15.107:0/3235552256 shutdown_connections 2024-04-25T00:19:15.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.437+0000 7fb61716c740 1 --2- 172.21.15.107:0/3235552256 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d98d619570 0x55d98d61b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.437+0000 7fb61716c740 1 --2- 172.21.15.107:0/3235552256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60eea0 0x55d98d619030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.437+0000 7fb61716c740 1 --2- 172.21.15.107:0/3235552256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d98d60e580 0x55d98d60e960 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.437+0000 7fb61716c740 1 -- 172.21.15.107:0/3235552256 >> 172.21.15.107:0/3235552256 conn(0x55d98d60a8a0 msgr2=0x55d98d60ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.437+0000 7fb61716c740 1 -- 172.21.15.107:0/3235552256 shutdown_connections 2024-04-25T00:19:15.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.437+0000 7fb61716c740 1 -- 172.21.15.107:0/3235552256 wait complete. 2024-04-25T00:19:15.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.437+0000 7fb61716c740 1 -- start start 2024-04-25T00:19:15.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.437+0000 7fb61716c740 1 Processor -- start 2024-04-25T00:19:15.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.437+0000 7fb61716c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60e580 0x55d98d618f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.437+0000 7fb61716c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d98d60eea0 0x55d98d616280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.437+0000 7fb61716c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d98d619570 0x55d98d616890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.437+0000 7fb61716c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d98d61c300 con 0x55d98d60e580 2024-04-25T00:19:15.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.437+0000 7fb61716c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d98d52f050 con 0x55d98d60eea0 2024-04-25T00:19:15.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.437+0000 7fb61716c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d98d61c480 con 0x55d98d619570 2024-04-25T00:19:15.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.437+0000 7fb61716c740 10 client.0 _ll_register_callbacks cb 0x55d98d286be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:15.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.438+0000 7fb6121f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60e580 0x55d98d618f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.438+0000 7fb6121f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60e580 0x55d98d618f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40180/0 (socket says 172.21.15.107:40180) 2024-04-25T00:19:15.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.438+0000 7fb6119f8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d98d60eea0 0x55d98d616280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.438+0000 7fb6121f9640 1 -- 172.21.15.107:0/2781575711 learned_addr learned my addr 172.21.15.107:0/2781575711 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:15.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.438+0000 7fb6129fa640 1 --2- 172.21.15.107:0/2781575711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d98d619570 0x55d98d616890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.438+0000 7fb6121f9640 1 -- 172.21.15.107:0/2781575711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d98d619570 msgr2=0x55d98d616890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.438+0000 7fb6121f9640 1 --2- 172.21.15.107:0/2781575711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d98d619570 0x55d98d616890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.438+0000 7fb6121f9640 1 -- 172.21.15.107:0/2781575711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d98d60eea0 msgr2=0x55d98d616280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.438+0000 7fb6121f9640 1 --2- 172.21.15.107:0/2781575711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d98d60eea0 0x55d98d616280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.438+0000 7fb6121f9640 1 -- 172.21.15.107:0/2781575711 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d98d500dc0 con 0x55d98d60e580 2024-04-25T00:19:15.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.438+0000 7fb6121f9640 1 --2- 172.21.15.107:0/2781575711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60e580 0x55d98d618f90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0bb970 tx=0x7fb5fc03b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8442052a03fa1ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.438+0000 7fb5f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:19:15.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.438+0000 7fb5f5ffb640 1 -- 172.21.15.107:0/2781575711 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc049040 con 0x55d98d60e580 2024-04-25T00:19:15.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.438+0000 7fb5f5ffb640 1 -- 172.21.15.107:0/2781575711 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fb5fc0ed070 con 0x55d98d60e580 2024-04-25T00:19:15.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.438+0000 7fb61716c740 1 -- 172.21.15.107:0/2781575711 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb604009ce0 con 0x55d98d60e580 2024-04-25T00:19:15.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.438+0000 7fb5f5ffb640 1 -- 172.21.15.107:0/2781575711 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb5fc0bbed0 con 0x55d98d60e580 2024-04-25T00:19:15.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.438+0000 7fb61716c740 10 client.16248 fetch_fsmap learned FSMap version 28 2024-04-25T00:19:15.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.438+0000 7fb61716c740 1 -- 172.21.15.107:0/2781575711 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d98d2873f0 con 0x55d98d60e580 2024-04-25T00:19:15.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.438+0000 7fb5f5ffb640 1 -- 172.21.15.107:0/2781575711 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb604009ce0 con 0x55d98d60e580 2024-04-25T00:19:15.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.439+0000 7fb5f5ffb640 1 -- 172.21.15.107:0/2781575711 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fb5fc0b82e0 con 0x55d98d60e580 2024-04-25T00:19:15.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.439+0000 7fb61716c740 10 client.16248 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:19:15.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.439+0000 7fb61716c740 10 client.16248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:15.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.439+0000 7fb61716c740 20 client.16248 populate_metadata read hostname 'smithi107' 2024-04-25T00:19:15.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.439+0000 7fb61716c740 1 --2- 172.21.15.107:0/2781575711 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d98d629550 0x55d98d62b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.439+0000 7fb61716c740 4 client.16248 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:19:15.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.439+0000 7fb61716c740 1 -- 172.21.15.107:0/2781575711 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d98d580bb0 con 0x55d98d629550 2024-04-25T00:19:15.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.439+0000 7fb6119f8640 1 --2- 172.21.15.107:0/2781575711 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d98d629550 0x55d98d62b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.439+0000 7fb5f5ffb640 10 client.16248 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:15.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.439+0000 7fb6119f8640 1 --2- 172.21.15.107:0/2781575711 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d98d629550 0x55d98d62b9d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8c3e27605859901 server_cookie=c590dc493c9c633e in_seq=0 out_seq=0 2024-04-25T00:19:15.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.440+0000 7fb5f5ffb640 1 -- 172.21.15.107:0/2781575711 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d98d580bb0 con 0x55d98d629550 2024-04-25T00:19:15.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.440+0000 7fb5f5ffb640 10 client.16248 handle_command_reply: tid=0 2024-04-25T00:19:15.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.440+0000 7fb61716c740 1 client.16248 shutdown 2024-04-25T00:19:15.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.440+0000 7fb61716c740 1 -- 172.21.15.107:0/2781575711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60e580 msgr2=0x55d98d618f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.440+0000 7fb61716c740 1 --2- 172.21.15.107:0/2781575711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60e580 0x55d98d618f90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0bb970 tx=0x7fb5fc03b560 comp rx=0 tx=0).stop 2024-04-25T00:19:15.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.440+0000 7fb61716c740 1 -- 172.21.15.107:0/2781575711 shutdown_connections 2024-04-25T00:19:15.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.440+0000 7fb61716c740 1 --2- 172.21.15.107:0/2781575711 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d98d629550 0x55d98d62b9d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.440+0000 7fb61716c740 1 --2- 172.21.15.107:0/2781575711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d98d619570 0x55d98d616890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.440+0000 7fb61716c740 1 --2- 172.21.15.107:0/2781575711 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d98d60eea0 0x55d98d616280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.440+0000 7fb5f5ffb640 0 client.16248 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:15.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.440+0000 7fb61716c740 1 --2- 172.21.15.107:0/2781575711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60e580 0x55d98d618f90 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.440+0000 7fb61716c740 1 -- 172.21.15.107:0/2781575711 >> 172.21.15.107:0/2781575711 conn(0x55d98d60a8a0 msgr2=0x55d98d609b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.440+0000 7fb61716c740 1 -- 172.21.15.107:0/2781575711 shutdown_connections 2024-04-25T00:19:15.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.440+0000 7fb61716c740 1 -- 172.21.15.107:0/2781575711 wait complete. 2024-04-25T00:19:15.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.441+0000 7fb61716c740 20 client.16248 trim_cache size 0 max 16384 2024-04-25T00:19:15.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.454+0000 7fb61716c740 1 Processor -- start 2024-04-25T00:19:15.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.454+0000 7fb61716c740 1 -- start start 2024-04-25T00:19:15.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.454+0000 7fb61716c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60e580 0x55d98d618ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.454+0000 7fb61716c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d98d60eea0 0x55d98d6312a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.454+0000 7fb61716c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d98d619570 0x55d98d6317e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.454+0000 7fb61716c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c1b36c0 con 0x55d98d60e580 2024-04-25T00:19:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.454+0000 7fb61716c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c1b0ed0 con 0x55d98d60eea0 2024-04-25T00:19:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.454+0000 7fb61716c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c1aed20 con 0x55d98d619570 2024-04-25T00:19:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.454+0000 7fb6121f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60e580 0x55d98d618ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.454+0000 7fb6121f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60e580 0x55d98d618ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40184/0 (socket says 172.21.15.107:40184) 2024-04-25T00:19:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.454+0000 7fb6121f9640 1 -- 172.21.15.107:0/494669596 learned_addr learned my addr 172.21.15.107:0/494669596 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.455+0000 7fb6119f8640 1 --2- 172.21.15.107:0/494669596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d98d60eea0 0x55d98d6312a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.455+0000 7fb6121f9640 1 -- 172.21.15.107:0/494669596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d98d619570 msgr2=0x55d98d6317e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.455+0000 7fb6121f9640 1 --2- 172.21.15.107:0/494669596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d98d619570 0x55d98d6317e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.455+0000 7fb6121f9640 1 -- 172.21.15.107:0/494669596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d98d60eea0 msgr2=0x55d98d6312a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.455+0000 7fb6121f9640 1 --2- 172.21.15.107:0/494669596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d98d60eea0 0x55d98d6312a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.455+0000 7fb6121f9640 1 -- 172.21.15.107:0/494669596 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d98d61bcd0 con 0x55d98d60e580 2024-04-25T00:19:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.455+0000 7fb6121f9640 1 --2- 172.21.15.107:0/494669596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60e580 0x55d98d618ac0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0bc2d0 tx=0x7fb5fc03b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6fb74e79187593 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.455+0000 7fb5f7fff640 1 -- 172.21.15.107:0/494669596 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc049030 con 0x55d98d60e580 2024-04-25T00:19:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.455+0000 7fb5f7fff640 1 -- 172.21.15.107:0/494669596 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5fc0c4830 con 0x55d98d60e580 2024-04-25T00:19:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.455+0000 7fb5f7fff640 1 -- 172.21.15.107:0/494669596 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc0ed190 con 0x55d98d60e580 2024-04-25T00:19:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.455+0000 7fb61716c740 1 -- 172.21.15.107:0/494669596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60e580 msgr2=0x55d98d618ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.455+0000 7fb61716c740 1 --2- 172.21.15.107:0/494669596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60e580 0x55d98d618ac0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0bc2d0 tx=0x7fb5fc03b460 comp rx=0 tx=0).stop 2024-04-25T00:19:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb61716c740 1 -- 172.21.15.107:0/494669596 shutdown_connections 2024-04-25T00:19:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb61716c740 1 --2- 172.21.15.107:0/494669596 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d98d619570 0x55d98d6317e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb61716c740 1 --2- 172.21.15.107:0/494669596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d98d60eea0 0x55d98d6312a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb61716c740 1 --2- 172.21.15.107:0/494669596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60e580 0x55d98d618ac0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb61716c740 1 -- 172.21.15.107:0/494669596 >> 172.21.15.107:0/494669596 conn(0x55d98d60a8a0 msgr2=0x55d98d60fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb61716c740 1 -- 172.21.15.107:0/494669596 shutdown_connections 2024-04-25T00:19:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb61716c740 1 -- 172.21.15.107:0/494669596 wait complete. 2024-04-25T00:19:15.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb61716c740 1 -- start start 2024-04-25T00:19:15.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb61716c740 1 Processor -- start 2024-04-25T00:19:15.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb61716c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60e580 0x55d98d6216d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb61716c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d98d60eea0 0x55d98d621c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb61716c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d98d619570 0x55d98d61e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb61716c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c1af390 con 0x55d98d60e580 2024-04-25T00:19:15.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb61716c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c1b0800 con 0x55d98d619570 2024-04-25T00:19:15.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb61716c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c1afa60 con 0x55d98d60eea0 2024-04-25T00:19:15.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb61716c740 10 client.0 _ll_register_callbacks cb 0x7fb5fc009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:15.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb6121f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60e580 0x55d98d6216d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb6121f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60e580 0x55d98d6216d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40186/0 (socket says 172.21.15.107:40186) 2024-04-25T00:19:15.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb6121f9640 1 -- 172.21.15.107:0/4052714841 learned_addr learned my addr 172.21.15.107:0/4052714841 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:15.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb6129fa640 1 --2- 172.21.15.107:0/4052714841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d98d619570 0x55d98d61e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb6119f8640 1 --2- 172.21.15.107:0/4052714841 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d98d60eea0 0x55d98d621c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb6121f9640 1 -- 172.21.15.107:0/4052714841 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d98d60eea0 msgr2=0x55d98d621c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb6121f9640 1 --2- 172.21.15.107:0/4052714841 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d98d60eea0 0x55d98d621c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb6121f9640 1 -- 172.21.15.107:0/4052714841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d98d619570 msgr2=0x55d98d61e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb6121f9640 1 --2- 172.21.15.107:0/4052714841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d98d619570 0x55d98d61e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.456+0000 7fb6121f9640 1 -- 172.21.15.107:0/4052714841 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d98d3517d0 con 0x55d98d60e580 2024-04-25T00:19:15.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.457+0000 7fb6121f9640 1 --2- 172.21.15.107:0/4052714841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60e580 0x55d98d6216d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0c4300 tx=0x7fb5fc03b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb88998b1c4e5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.457+0000 7fb5f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:19:15.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.457+0000 7fb5f5ffb640 1 -- 172.21.15.107:0/4052714841 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc09f060 con 0x55d98d60e580 2024-04-25T00:19:15.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.457+0000 7fb61716c740 1 -- 172.21.15.107:0/4052714841 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5fc0c4830 con 0x55d98d60e580 2024-04-25T00:19:15.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.457+0000 7fb5f5ffb640 1 -- 172.21.15.107:0/4052714841 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fb5fc0da2c0 con 0x55d98d60e580 2024-04-25T00:19:15.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.457+0000 7fb5f5ffb640 1 -- 172.21.15.107:0/4052714841 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb5fc049070 con 0x55d98d60e580 2024-04-25T00:19:15.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.457+0000 7fb61716c740 10 client.16254 fetch_fsmap learned FSMap version 28 2024-04-25T00:19:15.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.457+0000 7fb61716c740 1 -- 172.21.15.107:0/4052714841 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d98d3517d0 con 0x55d98d60e580 2024-04-25T00:19:15.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.457+0000 7fb5f5ffb640 1 -- 172.21.15.107:0/4052714841 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5fc0c4830 con 0x55d98d60e580 2024-04-25T00:19:15.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.457+0000 7fb5f5ffb640 1 -- 172.21.15.107:0/4052714841 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fb5fc0bab30 con 0x55d98d60e580 2024-04-25T00:19:15.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.457+0000 7fb61716c740 10 client.16254 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:19:15.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.457+0000 7fb61716c740 10 client.16254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:15.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.457+0000 7fb61716c740 20 client.16254 populate_metadata read hostname 'smithi107' 2024-04-25T00:19:15.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.457+0000 7fb61716c740 1 --2- 172.21.15.107:0/4052714841 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d98d6906d0 0x55d98d692ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.457+0000 7fb61716c740 4 client.16254 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:19:15.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.457+0000 7fb61716c740 1 -- 172.21.15.107:0/4052714841 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d98d57af00 con 0x55d98d6906d0 2024-04-25T00:19:15.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.457+0000 7fb6119f8640 1 --2- 172.21.15.107:0/4052714841 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d98d6906d0 0x55d98d692ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.458+0000 7fb6119f8640 1 --2- 172.21.15.107:0/4052714841 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d98d6906d0 0x55d98d692ac0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=383148cef681b5c6 server_cookie=dcd94b3788509791 in_seq=0 out_seq=0 2024-04-25T00:19:15.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.458+0000 7fb5f5ffb640 10 client.16254 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:15.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.459+0000 7fb5f5ffb640 1 -- 172.21.15.107:0/4052714841 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d98d57af00 con 0x55d98d6906d0 2024-04-25T00:19:15.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.459+0000 7fb5f5ffb640 10 client.16254 handle_command_reply: tid=0 2024-04-25T00:19:15.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.459+0000 7fb61716c740 1 client.16254 shutdown 2024-04-25T00:19:15.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.459+0000 7fb61716c740 1 -- 172.21.15.107:0/4052714841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60e580 msgr2=0x55d98d6216d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.459+0000 7fb61716c740 1 --2- 172.21.15.107:0/4052714841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60e580 0x55d98d6216d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc0c4300 tx=0x7fb5fc03b4d0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.459+0000 7fb61716c740 1 -- 172.21.15.107:0/4052714841 shutdown_connections 2024-04-25T00:19:15.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.459+0000 7fb61716c740 1 --2- 172.21.15.107:0/4052714841 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d98d6906d0 0x55d98d692ac0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.459+0000 7fb61716c740 1 --2- 172.21.15.107:0/4052714841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d98d619570 0x55d98d61e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.459+0000 7fb61716c740 1 --2- 172.21.15.107:0/4052714841 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d98d60eea0 0x55d98d621c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.459+0000 7fb5f5ffb640 0 client.16254 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:15.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.459+0000 7fb61716c740 1 --2- 172.21.15.107:0/4052714841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d98d60e580 0x55d98d6216d0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.459+0000 7fb61716c740 1 -- 172.21.15.107:0/4052714841 >> 172.21.15.107:0/4052714841 conn(0x55d98d60a8a0 msgr2=0x55d98d610560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.459+0000 7fb61716c740 1 -- 172.21.15.107:0/4052714841 shutdown_connections 2024-04-25T00:19:15.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.460+0000 7fb61716c740 1 -- 172.21.15.107:0/4052714841 wait complete. 2024-04-25T00:19:15.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.460+0000 7fb61716c740 20 client.16254 trim_cache size 0 max 16384 2024-04-25T00:19:15.466 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:19:15.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.460+0000 7fb61445d640 1 -- 172.21.15.107:0/3984307028 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb5f80a2db0 msgr2=0x7fb5f80a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.460+0000 7fb61445d640 1 --2- 172.21.15.107:0/3984307028 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb5f80a2db0 0x7fb5f80a5270 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb60400fc90 tx=0x7fb604018000 comp rx=0 tx=0).stop 2024-04-25T00:19:15.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.460+0000 7fb61445d640 1 -- 172.21.15.107:0/3984307028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb60c101790 msgr2=0x7fb60c073a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.460+0000 7fb61445d640 1 --2- 172.21.15.107:0/3984307028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb60c101790 0x7fb60c073a40 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc009230 tx=0x7fb5fc004040 comp rx=0 tx=0).stop 2024-04-25T00:19:15.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.460+0000 7fb61445d640 1 -- 172.21.15.107:0/3984307028 shutdown_connections 2024-04-25T00:19:15.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.460+0000 7fb61445d640 1 --2- 172.21.15.107:0/3984307028 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb5f80a2db0 0x7fb5f80a5270 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.460+0000 7fb61445d640 1 --2- 172.21.15.107:0/3984307028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb60c1077a0 0x7fb60c0744c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.460+0000 7fb61445d640 1 --2- 172.21.15.107:0/3984307028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb60c102150 0x7fb60c073f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.460+0000 7fb61445d640 1 --2- 172.21.15.107:0/3984307028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb60c101790 0x7fb60c073a40 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.460+0000 7fb61445d640 1 -- 172.21.15.107:0/3984307028 >> 172.21.15.107:0/3984307028 conn(0x7fb60c0fd2a0 msgr2=0x7fb60c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.460+0000 7fb61445d640 1 -- 172.21.15.107:0/3984307028 shutdown_connections 2024-04-25T00:19:15.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:15.460+0000 7fb61445d640 1 -- 172.21.15.107:0/3984307028 wait complete. 2024-04-25T00:19:16.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:15 smithi195 ceph-mon[32833]: pgmap v87: 129 pgs: 129 active+clean; 206 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 279 op/s 2024-04-25T00:19:16.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:15 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/354407517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:16.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:15 smithi170 ceph-mon[32807]: pgmap v87: 129 pgs: 129 active+clean; 206 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 279 op/s 2024-04-25T00:19:16.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:15 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/354407517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:16.471 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:19:16.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:19:16.471 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:19:16.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:15 smithi107 ceph-mon[29153]: pgmap v87: 129 pgs: 129 active+clean; 206 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 279 op/s 2024-04-25T00:19:16.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:15 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/354407517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:16.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.577+0000 7fd8b81b7640 1 -- 172.21.15.107:0/4064860540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8b0106aa0 msgr2=0x7fd8b010e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:16.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.577+0000 7fd8b81b7640 1 --2- 172.21.15.107:0/4064860540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8b0106aa0 0x7fd8b010e6b0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8009ca0 tx=0x7fd8a802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.577+0000 7fd8b81b7640 1 -- 172.21.15.107:0/4064860540 shutdown_connections 2024-04-25T00:19:16.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.577+0000 7fd8b81b7640 1 --2- 172.21.15.107:0/4064860540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8b0106aa0 0x7fd8b010e6b0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.577+0000 7fd8b81b7640 1 --2- 172.21.15.107:0/4064860540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8b0100c90 0x7fd8b01063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.577+0000 7fd8b81b7640 1 --2- 172.21.15.107:0/4064860540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8b01002e0 0x7fd8b01006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.577+0000 7fd8b81b7640 1 -- 172.21.15.107:0/4064860540 >> 172.21.15.107:0/4064860540 conn(0x7fd8b00758d0 msgr2=0x7fd8b0075ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:16.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.577+0000 7fd8b81b7640 1 -- 172.21.15.107:0/4064860540 shutdown_connections 2024-04-25T00:19:16.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.577+0000 7fd8b81b7640 1 -- 172.21.15.107:0/4064860540 wait complete. 2024-04-25T00:19:16.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.578+0000 7fd8b81b7640 1 Processor -- start 2024-04-25T00:19:16.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.578+0000 7fd8b81b7640 1 -- start start 2024-04-25T00:19:16.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.578+0000 7fd8b81b7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8b01002e0 0x7fd8b0198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.578+0000 7fd8b81b7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8b0106aa0 0x7fd8b01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.578+0000 7fd8b81b7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8b019c8e0 0x7fd8b019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.578+0000 7fd8b81b7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b0111b90 con 0x7fd8b0106aa0 2024-04-25T00:19:16.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.578+0000 7fd8b81b7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b0111a10 con 0x7fd8b01002e0 2024-04-25T00:19:16.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.578+0000 7fd8b81b7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b0111d10 con 0x7fd8b019c8e0 2024-04-25T00:19:16.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.578+0000 7fd8b5752640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8b0106aa0 0x7fd8b01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.578+0000 7fd8b5752640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8b0106aa0 0x7fd8b01985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40198/0 (socket says 172.21.15.107:40198) 2024-04-25T00:19:16.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.578+0000 7fd8b5752640 1 -- 172.21.15.107:0/1536179880 learned_addr learned my addr 172.21.15.107:0/1536179880 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:16.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.578+0000 7fd8b5f53640 1 --2- 172.21.15.107:0/1536179880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8b01002e0 0x7fd8b0198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.578+0000 7fd8b5752640 1 -- 172.21.15.107:0/1536179880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8b019c8e0 msgr2=0x7fd8b019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:16.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.578+0000 7fd8b5752640 1 --2- 172.21.15.107:0/1536179880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8b019c8e0 0x7fd8b019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.578+0000 7fd8b5752640 1 -- 172.21.15.107:0/1536179880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8b01002e0 msgr2=0x7fd8b0198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:16.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.578+0000 7fd8b5752640 1 --2- 172.21.15.107:0/1536179880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8b01002e0 0x7fd8b0198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.578+0000 7fd8b5752640 1 -- 172.21.15.107:0/1536179880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b019d3d0 con 0x7fd8b0106aa0 2024-04-25T00:19:16.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.579+0000 7fd8b5752640 1 --2- 172.21.15.107:0/1536179880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8b0106aa0 0x7fd8b01985b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000e9c0 tx=0x7fd8a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:16.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.579+0000 7fd8a6ffd640 1 -- 172.21.15.107:0/1536179880 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a000cd80 con 0x7fd8b0106aa0 2024-04-25T00:19:16.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.579+0000 7fd8a6ffd640 1 -- 172.21.15.107:0/1536179880 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8a0005500 con 0x7fd8b0106aa0 2024-04-25T00:19:16.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.579+0000 7fd8a6ffd640 1 -- 172.21.15.107:0/1536179880 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0010690 con 0x7fd8b0106aa0 2024-04-25T00:19:16.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.579+0000 7fd8b81b7640 1 -- 172.21.15.107:0/1536179880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b019d6c0 con 0x7fd8b0106aa0 2024-04-25T00:19:16.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.579+0000 7fd8b81b7640 1 -- 172.21.15.107:0/1536179880 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b01a4eb0 con 0x7fd8b0106aa0 2024-04-25T00:19:16.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.581+0000 7fd8a6ffd640 1 -- 172.21.15.107:0/1536179880 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fd8a00046c0 con 0x7fd8b0106aa0 2024-04-25T00:19:16.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.581+0000 7fd8bafa1740 1 Processor -- start 2024-04-25T00:19:16.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.582+0000 7fd8bafa1740 1 -- start start 2024-04-25T00:19:16.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.582+0000 7fd8bafa1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad715980 0x556bad713cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.582+0000 7fd8bafa1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556bad714290 0x556bad714710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.582+0000 7fd8bafa1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bad71f010 0x556bad721400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.582+0000 7fd8a6ffd640 1 --2- 172.21.15.107:0/1536179880 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd89c0a2d80 0x7fd89c0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.582+0000 7fd8bafa1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556bad6782c0 con 0x556bad715980 2024-04-25T00:19:16.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.582+0000 7fd8bafa1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556bad620910 con 0x556bad714290 2024-04-25T00:19:16.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.582+0000 7fd8bafa1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556bad674280 con 0x556bad71f010 2024-04-25T00:19:16.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.582+0000 7fd8b6754640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad715980 0x556bad713cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.582+0000 7fd8b5f53640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556bad714290 0x556bad714710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.582+0000 7fd8b6754640 1 --2- 172.21.15.107:0/1536179880 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd89c0a2d80 0x7fd89c0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.582+0000 7fd8b5f53640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556bad714290 0x556bad714710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.107:53704/0 (socket says 172.21.15.107:53704) 2024-04-25T00:19:16.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.582+0000 7fd8b5f53640 1 -- 172.21.15.107:0/3271346577 learned_addr learned my addr 172.21.15.107:0/3271346577 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:16.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.582+0000 7fd8a6ffd640 1 -- 172.21.15.107:0/1536179880 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fd8a00cfc00 con 0x7fd8b0106aa0 2024-04-25T00:19:16.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.582+0000 7fd8b5752640 1 --2- 172.21.15.107:0/3271346577 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bad71f010 0x556bad721400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.582+0000 7fd8b5f53640 1 -- 172.21.15.107:0/3271346577 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bad71f010 msgr2=0x556bad721400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:16.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.582+0000 7fd8b5f53640 1 --2- 172.21.15.107:0/3271346577 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bad71f010 0x556bad721400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.582+0000 7fd8b5f53640 1 -- 172.21.15.107:0/3271346577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad715980 msgr2=0x556bad713cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:16.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.582+0000 7fd8b5f53640 1 --2- 172.21.15.107:0/3271346577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad715980 0x556bad713cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.582+0000 7fd8b5f53640 1 -- 172.21.15.107:0/3271346577 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bad631fb0 con 0x556bad714290 2024-04-25T00:19:16.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.582+0000 7fd8b5752640 1 --2- 172.21.15.107:0/3271346577 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bad71f010 0x556bad721400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:16.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.583+0000 7fd8b5f53640 1 --2- 172.21.15.107:0/3271346577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556bad714290 0x556bad714710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00b760 tx=0x7fd8ac00bc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d057a02e904f7f23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:16.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.583+0000 7fd8b6754640 1 --2- 172.21.15.107:0/1536179880 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd89c0a2d80 0x7fd89c0a5240 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd8a802f540 tx=0x7fd8a8005d80 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:16.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.583+0000 7fd89bfff640 1 -- 172.21.15.107:0/3271346577 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac003dd0 con 0x556bad714290 2024-04-25T00:19:16.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.583+0000 7fd89bfff640 1 -- 172.21.15.107:0/3271346577 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8ac015070 con 0x556bad714290 2024-04-25T00:19:16.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.583+0000 7fd89bfff640 1 -- 172.21.15.107:0/3271346577 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac004de0 con 0x556bad714290 2024-04-25T00:19:16.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.583+0000 7fd8bafa1740 1 -- 172.21.15.107:0/3271346577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556bad714290 msgr2=0x556bad714710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:16.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.583+0000 7fd8bafa1740 1 --2- 172.21.15.107:0/3271346577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556bad714290 0x556bad714710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00b760 tx=0x7fd8ac00bc20 comp rx=0 tx=0).stop 2024-04-25T00:19:16.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.584+0000 7fd8bafa1740 1 -- 172.21.15.107:0/3271346577 shutdown_connections 2024-04-25T00:19:16.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.584+0000 7fd8bafa1740 1 --2- 172.21.15.107:0/3271346577 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bad71f010 0x556bad721400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.584+0000 7fd8bafa1740 1 --2- 172.21.15.107:0/3271346577 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556bad714290 0x556bad714710 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.584+0000 7fd8bafa1740 1 --2- 172.21.15.107:0/3271346577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad715980 0x556bad713cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.584+0000 7fd8bafa1740 1 -- 172.21.15.107:0/3271346577 >> 172.21.15.107:0/3271346577 conn(0x556bad6dc950 msgr2=0x556bad7107f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:16.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.584+0000 7fd8bafa1740 1 -- 172.21.15.107:0/3271346577 shutdown_connections 2024-04-25T00:19:16.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.584+0000 7fd8bafa1740 1 -- 172.21.15.107:0/3271346577 wait complete. 2024-04-25T00:19:16.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.584+0000 7fd8bafa1740 1 -- start start 2024-04-25T00:19:16.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.584+0000 7fd8bafa1740 1 Processor -- start 2024-04-25T00:19:16.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.584+0000 7fd8bafa1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556bad714290 0x556bad71bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.584+0000 7fd8bafa1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad715980 0x556bad71c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.584+0000 7fd8bafa1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bad71f010 0x556bad71c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.584+0000 7fd8bafa1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556bad721ea0 con 0x556bad715980 2024-04-25T00:19:16.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.584+0000 7fd8bafa1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556bad633e90 con 0x556bad714290 2024-04-25T00:19:16.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.584+0000 7fd8bafa1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x556bad722020 con 0x556bad71f010 2024-04-25T00:19:16.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.584+0000 7fd8bafa1740 10 client.0 _ll_register_callbacks cb 0x556bad66f970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:16.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.585+0000 7fd8b5752640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad715980 0x556bad71c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.585+0000 7fd8b5f53640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556bad714290 0x556bad71bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.585+0000 7fd8b5752640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad715980 0x556bad71c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40222/0 (socket says 172.21.15.107:40222) 2024-04-25T00:19:16.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.585+0000 7fd8b5752640 1 -- 172.21.15.107:0/126461253 learned_addr learned my addr 172.21.15.107:0/126461253 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:16.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.585+0000 7fd8b6754640 1 --2- 172.21.15.107:0/126461253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bad71f010 0x556bad71c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.585+0000 7fd8b5752640 1 -- 172.21.15.107:0/126461253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bad71f010 msgr2=0x556bad71c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:16.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.585+0000 7fd8b5752640 1 --2- 172.21.15.107:0/126461253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bad71f010 0x556bad71c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.585+0000 7fd8b5752640 1 -- 172.21.15.107:0/126461253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556bad714290 msgr2=0x556bad71bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:16.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.585+0000 7fd8b5752640 1 --2- 172.21.15.107:0/126461253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556bad714290 0x556bad71bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.585+0000 7fd8b5752640 1 -- 172.21.15.107:0/126461253 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bad4547e0 con 0x556bad715980 2024-04-25T00:19:16.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.585+0000 7fd8b5f53640 1 --2- 172.21.15.107:0/126461253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556bad714290 0x556bad71bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:16.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.585+0000 7fd8b5752640 1 --2- 172.21.15.107:0/126461253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad715980 0x556bad71c460 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0000c00 tx=0x7fd8a009b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58e7cf1c5236fe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:16.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.585+0000 7fd8997fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:19:16.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.585+0000 7fd8997fa640 1 -- 172.21.15.107:0/126461253 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a00a0100 con 0x556bad715980 2024-04-25T00:19:16.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.585+0000 7fd8997fa640 1 -- 172.21.15.107:0/126461253 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fd8a00a4530 con 0x556bad715980 2024-04-25T00:19:16.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.585+0000 7fd8bafa1740 1 -- 172.21.15.107:0/126461253 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8ac015070 con 0x556bad715980 2024-04-25T00:19:16.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.585+0000 7fd8997fa640 1 -- 172.21.15.107:0/126461253 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd8a00a49c0 con 0x556bad715980 2024-04-25T00:19:16.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.585+0000 7fd8bafa1740 10 client.16266 fetch_fsmap learned FSMap version 28 2024-04-25T00:19:16.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.585+0000 7fd8bafa1740 1 -- 172.21.15.107:0/126461253 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556bad4b4f70 con 0x556bad715980 2024-04-25T00:19:16.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.585+0000 7fd8997fa640 1 -- 172.21.15.107:0/126461253 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8ac015070 con 0x556bad715980 2024-04-25T00:19:16.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.586+0000 7fd8997fa640 1 -- 172.21.15.107:0/126461253 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fd8a009b210 con 0x556bad715980 2024-04-25T00:19:16.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.586+0000 7fd8bafa1740 10 client.16266 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:19:16.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.586+0000 7fd8bafa1740 10 client.16266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:16.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.586+0000 7fd8bafa1740 20 client.16266 populate_metadata read hostname 'smithi107' 2024-04-25T00:19:16.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.586+0000 7fd8bafa1740 1 --2- 172.21.15.107:0/126461253 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x556bad72efc0 0x556bad731460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.586+0000 7fd8bafa1740 4 client.16266 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:19:16.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.586+0000 7fd8bafa1740 1 -- 172.21.15.107:0/126461253 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bad64c5a0 con 0x556bad72efc0 2024-04-25T00:19:16.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.586+0000 7fd8b5f53640 1 --2- 172.21.15.107:0/126461253 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x556bad72efc0 0x556bad731460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.586+0000 7fd8997fa640 10 client.16266 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:16.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.586+0000 7fd8b5f53640 1 --2- 172.21.15.107:0/126461253 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x556bad72efc0 0x556bad731460 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3413ef4341336cbb server_cookie=d1dccd94a709d22e in_seq=0 out_seq=0 2024-04-25T00:19:16.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.587+0000 7fd8997fa640 1 -- 172.21.15.107:0/126461253 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556bad64c5a0 con 0x556bad72efc0 2024-04-25T00:19:16.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.587+0000 7fd8997fa640 10 client.16266 handle_command_reply: tid=0 2024-04-25T00:19:16.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.587+0000 7fd8bafa1740 1 client.16266 shutdown 2024-04-25T00:19:16.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.587+0000 7fd8bafa1740 1 -- 172.21.15.107:0/126461253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad715980 msgr2=0x556bad71c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:16.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.587+0000 7fd8bafa1740 1 --2- 172.21.15.107:0/126461253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad715980 0x556bad71c460 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0000c00 tx=0x7fd8a009b950 comp rx=0 tx=0).stop 2024-04-25T00:19:16.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.587+0000 7fd8bafa1740 1 -- 172.21.15.107:0/126461253 shutdown_connections 2024-04-25T00:19:16.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.587+0000 7fd8bafa1740 1 --2- 172.21.15.107:0/126461253 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x556bad72efc0 0x556bad731460 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.587+0000 7fd8bafa1740 1 --2- 172.21.15.107:0/126461253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bad71f010 0x556bad71c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.587+0000 7fd8bafa1740 1 --2- 172.21.15.107:0/126461253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad715980 0x556bad71c460 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.587+0000 7fd8997fa640 0 client.16266 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:16.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.587+0000 7fd8bafa1740 1 --2- 172.21.15.107:0/126461253 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556bad714290 0x556bad71bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.587+0000 7fd8bafa1740 1 -- 172.21.15.107:0/126461253 >> 172.21.15.107:0/126461253 conn(0x556bad6dc950 msgr2=0x556bad7165f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:16.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.588+0000 7fd8bafa1740 1 -- 172.21.15.107:0/126461253 shutdown_connections 2024-04-25T00:19:16.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.588+0000 7fd8bafa1740 1 -- 172.21.15.107:0/126461253 wait complete. 2024-04-25T00:19:16.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.588+0000 7fd8bafa1740 20 client.16266 trim_cache size 0 max 16384 2024-04-25T00:19:16.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.605+0000 7fd8bafa1740 1 Processor -- start 2024-04-25T00:19:16.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.605+0000 7fd8bafa1740 1 -- start start 2024-04-25T00:19:16.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.605+0000 7fd8bafa1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad714290 0x556bad72ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.605+0000 7fd8bafa1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556bad715980 0x556bad736d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.605+0000 7fd8bafa1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bad71f010 0x556bad737270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.605+0000 7fd8bafa1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b01b1350 con 0x556bad714290 2024-04-25T00:19:16.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.605+0000 7fd8bafa1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b01aeb60 con 0x556bad715980 2024-04-25T00:19:16.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.605+0000 7fd8bafa1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b01ac9b0 con 0x556bad71f010 2024-04-25T00:19:16.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.605+0000 7fd8b5f53640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad714290 0x556bad72ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.605+0000 7fd8b5f53640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad714290 0x556bad72ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40226/0 (socket says 172.21.15.107:40226) 2024-04-25T00:19:16.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.605+0000 7fd8b5f53640 1 -- 172.21.15.107:0/2233868334 learned_addr learned my addr 172.21.15.107:0/2233868334 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:16.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.606+0000 7fd8b5752640 1 --2- 172.21.15.107:0/2233868334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556bad715980 0x556bad736d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.606+0000 7fd8b5f53640 1 -- 172.21.15.107:0/2233868334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bad71f010 msgr2=0x556bad737270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:16.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.606+0000 7fd8b5f53640 1 --2- 172.21.15.107:0/2233868334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bad71f010 0x556bad737270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.606+0000 7fd8b5f53640 1 -- 172.21.15.107:0/2233868334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556bad715980 msgr2=0x556bad736d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:16.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.606+0000 7fd8b5f53640 1 --2- 172.21.15.107:0/2233868334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556bad715980 0x556bad736d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.606+0000 7fd8b5f53640 1 -- 172.21.15.107:0/2233868334 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bad34c170 con 0x556bad714290 2024-04-25T00:19:16.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.606+0000 7fd8b5f53640 1 --2- 172.21.15.107:0/2233868334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad714290 0x556bad72ada0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac004990 tx=0x7fd8ac004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5568ccd951829d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:16.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.606+0000 7fd89b7fe640 1 -- 172.21.15.107:0/2233868334 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac005e50 con 0x556bad714290 2024-04-25T00:19:16.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.606+0000 7fd89b7fe640 1 -- 172.21.15.107:0/2233868334 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8ac02d070 con 0x556bad714290 2024-04-25T00:19:16.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.606+0000 7fd89b7fe640 1 -- 172.21.15.107:0/2233868334 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac011840 con 0x556bad714290 2024-04-25T00:19:16.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.607+0000 7fd8bafa1740 1 -- 172.21.15.107:0/2233868334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad714290 msgr2=0x556bad72ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:16.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.607+0000 7fd8bafa1740 1 --2- 172.21.15.107:0/2233868334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad714290 0x556bad72ada0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac004990 tx=0x7fd8ac004920 comp rx=0 tx=0).stop 2024-04-25T00:19:16.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.607+0000 7fd8bafa1740 1 -- 172.21.15.107:0/2233868334 shutdown_connections 2024-04-25T00:19:16.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.607+0000 7fd8bafa1740 1 --2- 172.21.15.107:0/2233868334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bad71f010 0x556bad737270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.607+0000 7fd8bafa1740 1 --2- 172.21.15.107:0/2233868334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556bad715980 0x556bad736d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.607+0000 7fd8bafa1740 1 --2- 172.21.15.107:0/2233868334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad714290 0x556bad72ada0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.607+0000 7fd8bafa1740 1 -- 172.21.15.107:0/2233868334 >> 172.21.15.107:0/2233868334 conn(0x556bad6dc950 msgr2=0x556bad71dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:16.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.607+0000 7fd8bafa1740 1 -- 172.21.15.107:0/2233868334 shutdown_connections 2024-04-25T00:19:16.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.607+0000 7fd8bafa1740 1 -- 172.21.15.107:0/2233868334 wait complete. 2024-04-25T00:19:16.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.607+0000 7fd8bafa1740 1 -- start start 2024-04-25T00:19:16.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.607+0000 7fd8bafa1740 1 Processor -- start 2024-04-25T00:19:16.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.607+0000 7fd8bafa1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556bad714290 0x556bad7274a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.607+0000 7fd8bafa1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad715980 0x556bad724280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.607+0000 7fd8bafa1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bad71f010 0x556bad7247c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.607+0000 7fd8bafa1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b01ad020 con 0x556bad715980 2024-04-25T00:19:16.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.607+0000 7fd8bafa1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b01ae490 con 0x556bad714290 2024-04-25T00:19:16.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.607+0000 7fd8bafa1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b01ad6f0 con 0x556bad71f010 2024-04-25T00:19:16.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.607+0000 7fd8bafa1740 10 client.0 _ll_register_callbacks cb 0x7fd8a0009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:16.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.607+0000 7fd8b5752640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad715980 0x556bad724280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.608+0000 7fd8b5f53640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556bad714290 0x556bad7274a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.608+0000 7fd8b5752640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad715980 0x556bad724280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40230/0 (socket says 172.21.15.107:40230) 2024-04-25T00:19:16.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.608+0000 7fd8b5752640 1 -- 172.21.15.107:0/2070420865 learned_addr learned my addr 172.21.15.107:0/2070420865 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:16.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.608+0000 7fd8b6754640 1 --2- 172.21.15.107:0/2070420865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bad71f010 0x556bad7247c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.608+0000 7fd8b5752640 1 -- 172.21.15.107:0/2070420865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bad71f010 msgr2=0x556bad7247c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:16.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.608+0000 7fd8b5752640 1 --2- 172.21.15.107:0/2070420865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bad71f010 0x556bad7247c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.608+0000 7fd8b5752640 1 -- 172.21.15.107:0/2070420865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556bad714290 msgr2=0x556bad7274a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:16.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.608+0000 7fd8b5752640 1 --2- 172.21.15.107:0/2070420865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556bad714290 0x556bad7274a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.608+0000 7fd8b5752640 1 -- 172.21.15.107:0/2070420865 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bad724d00 con 0x556bad715980 2024-04-25T00:19:16.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.608+0000 7fd8b6754640 1 --2- 172.21.15.107:0/2070420865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bad71f010 0x556bad7247c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:16.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.608+0000 7fd8b5f53640 1 --2- 172.21.15.107:0/2070420865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556bad714290 0x556bad7274a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:16.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.608+0000 7fd8b5752640 1 --2- 172.21.15.107:0/2070420865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad715980 0x556bad724280 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000c880 tx=0x7fd8a0009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a0eadab0b4760d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:16.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.608+0000 7fd8997fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:19:16.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.608+0000 7fd8997fa640 1 -- 172.21.15.107:0/2070420865 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0097ff0 con 0x556bad715980 2024-04-25T00:19:16.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.608+0000 7fd8997fa640 1 -- 172.21.15.107:0/2070420865 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fd8a00a4530 con 0x556bad715980 2024-04-25T00:19:16.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.608+0000 7fd8bafa1740 1 -- 172.21.15.107:0/2070420865 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8ac02d070 con 0x556bad715980 2024-04-25T00:19:16.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.608+0000 7fd8997fa640 1 -- 172.21.15.107:0/2070420865 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd8a00a0100 con 0x556bad715980 2024-04-25T00:19:16.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.608+0000 7fd8bafa1740 10 client.16272 fetch_fsmap learned FSMap version 28 2024-04-25T00:19:16.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.608+0000 7fd8bafa1740 1 -- 172.21.15.107:0/2070420865 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556bad796340 con 0x556bad715980 2024-04-25T00:19:16.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.608+0000 7fd8997fa640 1 -- 172.21.15.107:0/2070420865 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8ac02d070 con 0x556bad715980 2024-04-25T00:19:16.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.609+0000 7fd8997fa640 1 -- 172.21.15.107:0/2070420865 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fd8a00a1450 con 0x556bad715980 2024-04-25T00:19:16.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.609+0000 7fd8bafa1740 10 client.16272 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:19:16.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.609+0000 7fd8bafa1740 10 client.16272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:16.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.609+0000 7fd8bafa1740 20 client.16272 populate_metadata read hostname 'smithi107' 2024-04-25T00:19:16.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.609+0000 7fd8bafa1740 1 --2- 172.21.15.107:0/2070420865 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x556bad796b20 0x556bad798f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.609+0000 7fd8bafa1740 4 client.16272 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:19:16.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.609+0000 7fd8bafa1740 1 -- 172.21.15.107:0/2070420865 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556bad685800 con 0x556bad796b20 2024-04-25T00:19:16.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.609+0000 7fd8b5f53640 1 --2- 172.21.15.107:0/2070420865 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x556bad796b20 0x556bad798f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.609+0000 7fd8997fa640 10 client.16272 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:16.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.609+0000 7fd8b5f53640 1 --2- 172.21.15.107:0/2070420865 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x556bad796b20 0x556bad798f40 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fed52f8ac0fc961 server_cookie=e59aa85d02dfff89 in_seq=0 out_seq=0 2024-04-25T00:19:16.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.624+0000 7fd8997fa640 1 -- 172.21.15.107:0/2070420865 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556bad685800 con 0x556bad796b20 2024-04-25T00:19:16.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.624+0000 7fd8997fa640 10 client.16272 handle_command_reply: tid=0 2024-04-25T00:19:16.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.624+0000 7fd8bafa1740 1 client.16272 shutdown 2024-04-25T00:19:16.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.624+0000 7fd8bafa1740 1 -- 172.21.15.107:0/2070420865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad715980 msgr2=0x556bad724280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:16.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.624+0000 7fd8bafa1740 1 --2- 172.21.15.107:0/2070420865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad715980 0x556bad724280 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000c880 tx=0x7fd8a0009700 comp rx=0 tx=0).stop 2024-04-25T00:19:16.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.624+0000 7fd8bafa1740 1 -- 172.21.15.107:0/2070420865 shutdown_connections 2024-04-25T00:19:16.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.624+0000 7fd8bafa1740 1 --2- 172.21.15.107:0/2070420865 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x556bad796b20 0x556bad798f40 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.624+0000 7fd8bafa1740 1 --2- 172.21.15.107:0/2070420865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x556bad71f010 0x556bad7247c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.624+0000 7fd8bafa1740 1 --2- 172.21.15.107:0/2070420865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556bad715980 0x556bad724280 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.624+0000 7fd8997fa640 0 client.16272 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:16.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.624+0000 7fd8bafa1740 1 --2- 172.21.15.107:0/2070420865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556bad714290 0x556bad7274a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.624+0000 7fd8bafa1740 1 -- 172.21.15.107:0/2070420865 >> 172.21.15.107:0/2070420865 conn(0x556bad6dc950 msgr2=0x556bad70f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:16.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.624+0000 7fd8bafa1740 1 -- 172.21.15.107:0/2070420865 shutdown_connections 2024-04-25T00:19:16.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.624+0000 7fd8bafa1740 1 -- 172.21.15.107:0/2070420865 wait complete. 2024-04-25T00:19:16.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.624+0000 7fd8bafa1740 20 client.16272 trim_cache size 0 max 16384 2024-04-25T00:19:16.626 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:19:16.626 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-04-25T00:19:16.626 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "e0594457-3687-44f4-a23d-854ab2963516", 2024-04-25T00:19:16.626 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-04-25T00:19:16.626 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:19:16.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.624+0000 7fd8b81b7640 1 -- 172.21.15.107:0/1536179880 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd89c0a2d80 msgr2=0x7fd89c0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:16.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.624+0000 7fd8b81b7640 1 --2- 172.21.15.107:0/1536179880 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd89c0a2d80 0x7fd89c0a5240 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd8a802f540 tx=0x7fd8a8005d80 comp rx=0 tx=0).stop 2024-04-25T00:19:16.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.624+0000 7fd8b81b7640 1 -- 172.21.15.107:0/1536179880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8b0106aa0 msgr2=0x7fd8b01985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:16.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.624+0000 7fd8b81b7640 1 --2- 172.21.15.107:0/1536179880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8b0106aa0 0x7fd8b01985b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000e9c0 tx=0x7fd8a000ee80 comp rx=0 tx=0).stop 2024-04-25T00:19:16.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.625+0000 7fd8b81b7640 1 -- 172.21.15.107:0/1536179880 shutdown_connections 2024-04-25T00:19:16.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.625+0000 7fd8b81b7640 1 --2- 172.21.15.107:0/1536179880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8b019c8e0 0x7fd8b019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.625+0000 7fd8b81b7640 1 --2- 172.21.15.107:0/1536179880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8b0106aa0 0x7fd8b01985b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.625+0000 7fd8b81b7640 1 --2- 172.21.15.107:0/1536179880 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd89c0a2d80 0x7fd89c0a5240 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.625+0000 7fd8b81b7640 1 --2- 172.21.15.107:0/1536179880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8b01002e0 0x7fd8b0198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.625+0000 7fd8b81b7640 1 -- 172.21.15.107:0/1536179880 >> 172.21.15.107:0/1536179880 conn(0x7fd8b00758d0 msgr2=0x7fd8b00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:16.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.625+0000 7fd8b81b7640 1 -- 172.21.15.107:0/1536179880 shutdown_connections 2024-04-25T00:19:16.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.625+0000 7fd8b81b7640 1 -- 172.21.15.107:0/1536179880 wait complete. 2024-04-25T00:19:16.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:19:16.636 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:19:16.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.785+0000 7f4b75ebd640 1 -- 172.21.15.107:0/1028819927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b701066b0 msgr2=0x7f4b70106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:16.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.785+0000 7f4b75ebd640 1 --2- 172.21.15.107:0/1028819927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b701066b0 0x7f4b70106a90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4b60009930 tx=0x7f4b6002a420 comp rx=0 tx=0).stop 2024-04-25T00:19:16.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.785+0000 7f4b75ebd640 1 -- 172.21.15.107:0/1028819927 shutdown_connections 2024-04-25T00:19:16.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.785+0000 7f4b75ebd640 1 --2- 172.21.15.107:0/1028819927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b70100fd0 0x7f4b7010d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.785+0000 7f4b75ebd640 1 --2- 172.21.15.107:0/1028819927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b70100610 0x7f4b70100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.785+0000 7f4b75ebd640 1 --2- 172.21.15.107:0/1028819927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b701066b0 0x7f4b70106a90 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.785+0000 7f4b75ebd640 1 -- 172.21.15.107:0/1028819927 >> 172.21.15.107:0/1028819927 conn(0x7f4b70077740 msgr2=0x7f4b70077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:16.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.785+0000 7f4b75ebd640 1 -- 172.21.15.107:0/1028819927 shutdown_connections 2024-04-25T00:19:16.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.785+0000 7f4b75ebd640 1 -- 172.21.15.107:0/1028819927 wait complete. 2024-04-25T00:19:16.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.785+0000 7f4b75ebd640 1 Processor -- start 2024-04-25T00:19:16.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.786+0000 7f4b75ebd640 1 -- start start 2024-04-25T00:19:16.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.786+0000 7f4b75ebd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b70100610 0x7f4b70197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.786+0000 7f4b75ebd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b70100fd0 0x7f4b70198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.786+0000 7f4b75ebd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b701066b0 0x7f4b7019c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.786+0000 7f4b75ebd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4b70110a30 con 0x7f4b701066b0 2024-04-25T00:19:16.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.786+0000 7f4b75ebd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4b701108b0 con 0x7f4b70100fd0 2024-04-25T00:19:16.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.786+0000 7f4b75ebd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4b70110bb0 con 0x7f4b70100610 2024-04-25T00:19:16.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.786+0000 7f4b6f7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b70100610 0x7f4b70197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.786+0000 7f4b6ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b701066b0 0x7f4b7019c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.786+0000 7f4b6f7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b70100610 0x7f4b70197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:43916/0 (socket says 172.21.15.107:43916) 2024-04-25T00:19:16.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.786+0000 7f4b6effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b70100fd0 0x7f4b70198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.786+0000 7f4b6f7fe640 1 -- 172.21.15.107:0/2955808349 learned_addr learned my addr 172.21.15.107:0/2955808349 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:16.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.786+0000 7f4b6ffff640 1 -- 172.21.15.107:0/2955808349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b70100610 msgr2=0x7f4b70197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:16.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.786+0000 7f4b6ffff640 1 --2- 172.21.15.107:0/2955808349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b70100610 0x7f4b70197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.786+0000 7f4b6ffff640 1 -- 172.21.15.107:0/2955808349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b70100fd0 msgr2=0x7f4b70198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:16.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.786+0000 7f4b6ffff640 1 --2- 172.21.15.107:0/2955808349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b70100fd0 0x7f4b70198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.786+0000 7f4b6ffff640 1 -- 172.21.15.107:0/2955808349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b7019cfb0 con 0x7f4b701066b0 2024-04-25T00:19:16.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.786+0000 7f4b6f7fe640 1 --2- 172.21.15.107:0/2955808349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b70100610 0x7f4b70197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:16.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.787+0000 7f4b6ffff640 1 --2- 172.21.15.107:0/2955808349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b701066b0 0x7f4b7019c880 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009fd0 tx=0x7f4b6800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:16.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.787+0000 7f4b6cff9640 1 -- 172.21.15.107:0/2955808349 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6800ce20 con 0x7f4b701066b0 2024-04-25T00:19:16.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.787+0000 7f4b75ebd640 1 -- 172.21.15.107:0/2955808349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b7019d2a0 con 0x7f4b701066b0 2024-04-25T00:19:16.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.787+0000 7f4b6cff9640 1 -- 172.21.15.107:0/2955808349 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b68005020 con 0x7f4b701066b0 2024-04-25T00:19:16.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.787+0000 7f4b6cff9640 1 -- 172.21.15.107:0/2955808349 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b68010720 con 0x7f4b701066b0 2024-04-25T00:19:16.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.787+0000 7f4b75ebd640 1 -- 172.21.15.107:0/2955808349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b701a4b30 con 0x7f4b701066b0 2024-04-25T00:19:16.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.789+0000 7f4b6cff9640 1 -- 172.21.15.107:0/2955808349 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4b68005540 con 0x7f4b701066b0 2024-04-25T00:19:16.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.789+0000 7f4b78bd3740 1 Processor -- start 2024-04-25T00:19:16.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.789+0000 7f4b78bd3740 1 -- start start 2024-04-25T00:19:16.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.790+0000 7f4b78bd3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564576700630 0x564576700a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.790+0000 7f4b78bd3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564576700f50 0x56457670b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.790+0000 7f4b78bd3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56457670b620 0x56457670da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.790+0000 7f4b78bd3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5645766656b0 con 0x564576700f50 2024-04-25T00:19:16.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.790+0000 7f4b78bd3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56457660d880 con 0x56457670b620 2024-04-25T00:19:16.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.790+0000 7f4b78bd3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564576661670 con 0x564576700630 2024-04-25T00:19:16.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.790+0000 7f4b6cff9640 1 --2- 172.21.15.107:0/2955808349 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4b580a2d00 0x7f4b580a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.790+0000 7f4b6f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564576700f50 0x56457670b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.790+0000 7f4b6f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564576700f50 0x56457670b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40268/0 (socket says 172.21.15.107:40268) 2024-04-25T00:19:16.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.790+0000 7f4b6f7fe640 1 -- 172.21.15.107:0/2082634005 learned_addr learned my addr 172.21.15.107:0/2082634005 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:16.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.790+0000 7f4b6effd640 1 --2- 172.21.15.107:0/2082634005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564576700630 0x564576700a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.790+0000 7f4b6effd640 1 --2- 172.21.15.107:0/2082634005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56457670b620 0x56457670da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.790+0000 7f4b6ffff640 1 --2- 172.21.15.107:0/2955808349 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4b580a2d00 0x7f4b580a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.790+0000 7f4b6f7fe640 1 -- 172.21.15.107:0/2082634005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564576700630 msgr2=0x564576700a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:16.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.790+0000 7f4b6f7fe640 1 --2- 172.21.15.107:0/2082634005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564576700630 0x564576700a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.790+0000 7f4b6f7fe640 1 -- 172.21.15.107:0/2082634005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56457670b620 msgr2=0x56457670da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.790+0000 7f4b6f7fe640 1 --2- 172.21.15.107:0/2082634005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56457670b620 0x56457670da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.790+0000 7f4b6f7fe640 1 -- 172.21.15.107:0/2082634005 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645764437d0 con 0x564576700f50 2024-04-25T00:19:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.790+0000 7f4b6cff9640 1 -- 172.21.15.107:0/2955808349 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f4b680cfe30 con 0x7f4b701066b0 2024-04-25T00:19:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.790+0000 7f4b6f7fe640 1 --2- 172.21.15.107:0/2082634005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564576700f50 0x56457670b0e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4b60006010 tx=0x7f4b60005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32eca83b527fcb3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.790+0000 7f4b6ffff640 1 --2- 172.21.15.107:0/2955808349 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4b580a2d00 0x7f4b580a51c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4b680a46c0 tx=0x7f4b680c6b10 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.790+0000 7f4b55ffb640 1 -- 172.21.15.107:0/2082634005 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b60035040 con 0x564576700f50 2024-04-25T00:19:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.790+0000 7f4b55ffb640 1 -- 172.21.15.107:0/2082634005 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b60031830 con 0x564576700f50 2024-04-25T00:19:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.790+0000 7f4b55ffb640 1 -- 172.21.15.107:0/2082634005 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b600049a0 con 0x564576700f50 2024-04-25T00:19:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.791+0000 7f4b78bd3740 1 -- 172.21.15.107:0/2082634005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564576700f50 msgr2=0x56457670b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.791+0000 7f4b78bd3740 1 --2- 172.21.15.107:0/2082634005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564576700f50 0x56457670b0e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4b60006010 tx=0x7f4b60005d50 comp rx=0 tx=0).stop 2024-04-25T00:19:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.791+0000 7f4b78bd3740 1 -- 172.21.15.107:0/2082634005 shutdown_connections 2024-04-25T00:19:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.791+0000 7f4b78bd3740 1 --2- 172.21.15.107:0/2082634005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56457670b620 0x56457670da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.791+0000 7f4b78bd3740 1 --2- 172.21.15.107:0/2082634005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564576700f50 0x56457670b0e0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.791+0000 7f4b78bd3740 1 --2- 172.21.15.107:0/2082634005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564576700630 0x564576700a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.791+0000 7f4b78bd3740 1 -- 172.21.15.107:0/2082634005 >> 172.21.15.107:0/2082634005 conn(0x5645766fc950 msgr2=0x5645766fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:16.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.791+0000 7f4b78bd3740 1 -- 172.21.15.107:0/2082634005 shutdown_connections 2024-04-25T00:19:16.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.791+0000 7f4b78bd3740 1 -- 172.21.15.107:0/2082634005 wait complete. 2024-04-25T00:19:16.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.791+0000 7f4b78bd3740 1 -- start start 2024-04-25T00:19:16.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.792+0000 7f4b78bd3740 1 Processor -- start 2024-04-25T00:19:16.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.792+0000 7f4b78bd3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564576700630 0x5645767084d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.792+0000 7f4b78bd3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564576700f50 0x564576708a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.792+0000 7f4b78bd3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56457670b620 0x564576708f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.792+0000 7f4b78bd3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56457670e3b0 con 0x564576700630 2024-04-25T00:19:16.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.792+0000 7f4b78bd3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564576620f50 con 0x56457670b620 2024-04-25T00:19:16.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.792+0000 7f4b78bd3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56457670e530 con 0x564576700f50 2024-04-25T00:19:16.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.792+0000 7f4b78bd3740 10 client.0 _ll_register_callbacks cb 0x56457665c000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:16.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.792+0000 7f4b6effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564576700630 0x5645767084d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.792+0000 7f4b6effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564576700630 0x5645767084d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40284/0 (socket says 172.21.15.107:40284) 2024-04-25T00:19:16.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.792+0000 7f4b6effd640 1 -- 172.21.15.107:0/684955684 learned_addr learned my addr 172.21.15.107:0/684955684 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:16.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.792+0000 7f4b6f7fe640 1 --2- 172.21.15.107:0/684955684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564576700f50 0x564576708a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.792+0000 7f4b6effd640 1 --2- 172.21.15.107:0/684955684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56457670b620 0x564576708f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.792+0000 7f4b6effd640 1 -- 172.21.15.107:0/684955684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564576700f50 msgr2=0x564576708a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:16.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.792+0000 7f4b6effd640 1 --2- 172.21.15.107:0/684955684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564576700f50 0x564576708a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.792+0000 7f4b6effd640 1 -- 172.21.15.107:0/684955684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56457670b620 msgr2=0x564576708f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:16.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.792+0000 7f4b6effd640 1 --2- 172.21.15.107:0/684955684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56457670b620 0x564576708f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.792+0000 7f4b6effd640 1 -- 172.21.15.107:0/684955684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645763793f0 con 0x564576700630 2024-04-25T00:19:16.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.793+0000 7f4b6effd640 1 --2- 172.21.15.107:0/684955684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564576700630 0x5645767084d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4b64004580 tx=0x7f4b640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62280883dbef6d70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:16.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.793+0000 7f4b4effd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:19:16.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.793+0000 7f4b4effd640 1 -- 172.21.15.107:0/684955684 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64003d00 con 0x564576700630 2024-04-25T00:19:16.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.793+0000 7f4b4effd640 1 -- 172.21.15.107:0/684955684 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f4b64005020 con 0x564576700630 2024-04-25T00:19:16.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.793+0000 7f4b78bd3740 1 -- 172.21.15.107:0/684955684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b60031830 con 0x564576700630 2024-04-25T00:19:16.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.793+0000 7f4b4effd640 1 -- 172.21.15.107:0/684955684 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4b64030050 con 0x564576700630 2024-04-25T00:19:16.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.793+0000 7f4b78bd3740 10 client.16284 fetch_fsmap learned FSMap version 28 2024-04-25T00:19:16.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.793+0000 7f4b78bd3740 1 -- 172.21.15.107:0/684955684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5645763390f0 con 0x564576700630 2024-04-25T00:19:16.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.793+0000 7f4b4effd640 1 -- 172.21.15.107:0/684955684 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b60031830 con 0x564576700630 2024-04-25T00:19:16.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.793+0000 7f4b4effd640 1 -- 172.21.15.107:0/684955684 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f4b64004be0 con 0x564576700630 2024-04-25T00:19:16.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.793+0000 7f4b78bd3740 10 client.16284 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:19:16.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.793+0000 7f4b78bd3740 10 client.16284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:16.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.793+0000 7f4b78bd3740 20 client.16284 populate_metadata read hostname 'smithi107' 2024-04-25T00:19:16.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.793+0000 7f4b78bd3740 1 --2- 172.21.15.107:0/684955684 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56457671b850 0x56457671dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.793+0000 7f4b78bd3740 4 client.16284 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:19:16.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.793+0000 7f4b78bd3740 1 -- 172.21.15.107:0/684955684 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564576672bf0 con 0x56457671b850 2024-04-25T00:19:16.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.793+0000 7f4b6f7fe640 1 --2- 172.21.15.107:0/684955684 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56457671b850 0x56457671dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.794+0000 7f4b6f7fe640 1 --2- 172.21.15.107:0/684955684 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56457671b850 0x56457671dcb0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eab8732cbc44d343 server_cookie=44dbc8cb5f4938e8 in_seq=0 out_seq=0 2024-04-25T00:19:16.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.794+0000 7f4b4effd640 10 client.16284 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:16.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.796+0000 7f4b4effd640 1 -- 172.21.15.107:0/684955684 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564576672bf0 con 0x56457671b850 2024-04-25T00:19:16.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.796+0000 7f4b4effd640 10 client.16284 handle_command_reply: tid=0 2024-04-25T00:19:16.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.796+0000 7f4b78bd3740 1 client.16284 shutdown 2024-04-25T00:19:16.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.796+0000 7f4b78bd3740 1 -- 172.21.15.107:0/684955684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564576700630 msgr2=0x5645767084d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:16.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.796+0000 7f4b78bd3740 1 --2- 172.21.15.107:0/684955684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564576700630 0x5645767084d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4b64004580 tx=0x7f4b640046c0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.796+0000 7f4b78bd3740 1 -- 172.21.15.107:0/684955684 shutdown_connections 2024-04-25T00:19:16.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.796+0000 7f4b78bd3740 1 --2- 172.21.15.107:0/684955684 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56457671b850 0x56457671dcb0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.796+0000 7f4b78bd3740 1 --2- 172.21.15.107:0/684955684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56457670b620 0x564576708f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.796+0000 7f4b78bd3740 1 --2- 172.21.15.107:0/684955684 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564576700f50 0x564576708a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.796+0000 7f4b4effd640 0 client.16284 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:16.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.796+0000 7f4b78bd3740 1 --2- 172.21.15.107:0/684955684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564576700630 0x5645767084d0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.796+0000 7f4b78bd3740 1 -- 172.21.15.107:0/684955684 >> 172.21.15.107:0/684955684 conn(0x5645766fc950 msgr2=0x5645766fbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:16.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.796+0000 7f4b78bd3740 1 -- 172.21.15.107:0/684955684 shutdown_connections 2024-04-25T00:19:16.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.796+0000 7f4b78bd3740 1 -- 172.21.15.107:0/684955684 wait complete. 2024-04-25T00:19:16.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.796+0000 7f4b78bd3740 20 client.16284 trim_cache size 0 max 16384 2024-04-25T00:19:16.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.815+0000 7f4b78bd3740 1 Processor -- start 2024-04-25T00:19:16.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.815+0000 7f4b78bd3740 1 -- start start 2024-04-25T00:19:16.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.815+0000 7f4b78bd3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564576700630 0x5645767172a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.815+0000 7f4b78bd3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564576700f50 0x564576702d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.815+0000 7f4b78bd3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56457670b620 0x564576708ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.815+0000 7f4b78bd3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4b701af2c0 con 0x564576700630 2024-04-25T00:19:16.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.815+0000 7f4b78bd3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4b701acad0 con 0x564576700f50 2024-04-25T00:19:16.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.815+0000 7f4b78bd3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4b701aa920 con 0x56457670b620 2024-04-25T00:19:16.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.815+0000 7f4b6effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564576700630 0x5645767172a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.815+0000 7f4b6effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564576700630 0x5645767172a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40294/0 (socket says 172.21.15.107:40294) 2024-04-25T00:19:16.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.815+0000 7f4b6effd640 1 -- 172.21.15.107:0/3111506847 learned_addr learned my addr 172.21.15.107:0/3111506847 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:16.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.816+0000 7f4b6f7fe640 1 --2- 172.21.15.107:0/3111506847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564576700f50 0x564576702d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.816+0000 7f4b6effd640 1 --2- 172.21.15.107:0/3111506847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56457670b620 0x564576708ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.816+0000 7f4b6effd640 1 -- 172.21.15.107:0/3111506847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56457670b620 msgr2=0x564576708ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:16.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.816+0000 7f4b6effd640 1 --2- 172.21.15.107:0/3111506847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56457670b620 0x564576708ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.816+0000 7f4b6effd640 1 -- 172.21.15.107:0/3111506847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564576700f50 msgr2=0x564576702d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:16.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.816+0000 7f4b6effd640 1 --2- 172.21.15.107:0/3111506847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564576700f50 0x564576702d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.816+0000 7f4b6effd640 1 -- 172.21.15.107:0/3111506847 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564576700d80 con 0x564576700630 2024-04-25T00:19:16.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.816+0000 7f4b6effd640 1 --2- 172.21.15.107:0/3111506847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564576700630 0x5645767172a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4b64001eb0 tx=0x7f4b64005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72ac82dd6d0305f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:16.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.816+0000 7f4b557fa640 1 -- 172.21.15.107:0/3111506847 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64004120 con 0x564576700630 2024-04-25T00:19:16.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.816+0000 7f4b557fa640 1 -- 172.21.15.107:0/3111506847 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b640054e0 con 0x564576700630 2024-04-25T00:19:16.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.816+0000 7f4b557fa640 1 -- 172.21.15.107:0/3111506847 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64004b60 con 0x564576700630 2024-04-25T00:19:16.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.817+0000 7f4b78bd3740 1 -- 172.21.15.107:0/3111506847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564576700630 msgr2=0x5645767172a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:16.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.817+0000 7f4b78bd3740 1 --2- 172.21.15.107:0/3111506847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564576700630 0x5645767172a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4b64001eb0 tx=0x7f4b64005f90 comp rx=0 tx=0).stop 2024-04-25T00:19:16.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.817+0000 7f4b78bd3740 1 -- 172.21.15.107:0/3111506847 shutdown_connections 2024-04-25T00:19:16.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.817+0000 7f4b78bd3740 1 --2- 172.21.15.107:0/3111506847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56457670b620 0x564576708ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.817+0000 7f4b78bd3740 1 --2- 172.21.15.107:0/3111506847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564576700f50 0x564576702d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.817+0000 7f4b78bd3740 1 --2- 172.21.15.107:0/3111506847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564576700630 0x5645767172a0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.817+0000 7f4b78bd3740 1 -- 172.21.15.107:0/3111506847 >> 172.21.15.107:0/3111506847 conn(0x5645766fc950 msgr2=0x564576708490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:16.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.817+0000 7f4b78bd3740 1 -- 172.21.15.107:0/3111506847 shutdown_connections 2024-04-25T00:19:16.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.817+0000 7f4b78bd3740 1 -- 172.21.15.107:0/3111506847 wait complete. 2024-04-25T00:19:16.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.817+0000 7f4b78bd3740 1 -- start start 2024-04-25T00:19:16.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.818+0000 7f4b78bd3740 1 Processor -- start 2024-04-25T00:19:16.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.818+0000 7f4b78bd3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564576700630 0x564576713960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.818+0000 7f4b78bd3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564576700f50 0x56457670ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.818+0000 7f4b78bd3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56457670b620 0x56457670f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.818+0000 7f4b78bd3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4b701aaf90 con 0x56457670b620 2024-04-25T00:19:16.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.818+0000 7f4b78bd3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4b701ac400 con 0x564576700f50 2024-04-25T00:19:16.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.818+0000 7f4b78bd3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4b701ab660 con 0x564576700630 2024-04-25T00:19:16.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.818+0000 7f4b78bd3740 10 client.0 _ll_register_callbacks cb 0x7f4b640043a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:16.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.818+0000 7f4b6effd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564576700630 0x564576713960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.818+0000 7f4b6f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564576700f50 0x56457670ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.818+0000 7f4b6effd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564576700630 0x564576713960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:43948/0 (socket says 172.21.15.107:43948) 2024-04-25T00:19:16.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.818+0000 7f4b6effd640 1 -- 172.21.15.107:0/275471282 learned_addr learned my addr 172.21.15.107:0/275471282 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:16.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.818+0000 7f4b6effd640 1 --2- 172.21.15.107:0/275471282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56457670b620 0x56457670f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.818+0000 7f4b6effd640 1 -- 172.21.15.107:0/275471282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564576700f50 msgr2=0x56457670ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:16.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.818+0000 7f4b6effd640 1 --2- 172.21.15.107:0/275471282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564576700f50 0x56457670ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.818+0000 7f4b6effd640 1 -- 172.21.15.107:0/275471282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56457670b620 msgr2=0x56457670f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:16.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.818+0000 7f4b6effd640 1 --2- 172.21.15.107:0/275471282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56457670b620 0x56457670f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.818+0000 7f4b6effd640 1 -- 172.21.15.107:0/275471282 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56457670f910 con 0x564576700630 2024-04-25T00:19:16.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.819+0000 7f4b6effd640 1 --2- 172.21.15.107:0/275471282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564576700630 0x564576713960 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4b64017000 tx=0x7f4b64037040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ad6506d6d1db285 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:16.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.819+0000 7f4b4effd640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-04-25T00:19:16.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.819+0000 7f4b4effd640 1 -- 172.21.15.107:0/275471282 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64005000 con 0x564576700630 2024-04-25T00:19:16.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.819+0000 7f4b78bd3740 1 -- 172.21.15.107:0/275471282 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b640054e0 con 0x564576700630 2024-04-25T00:19:16.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.819+0000 7f4b4effd640 1 -- 172.21.15.107:0/275471282 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f4b640051a0 con 0x564576700630 2024-04-25T00:19:16.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.819+0000 7f4b4effd640 1 -- 172.21.15.107:0/275471282 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4b64003d00 con 0x564576700630 2024-04-25T00:19:16.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.819+0000 7f4b78bd3740 10 client.25657 fetch_fsmap learned FSMap version 28 2024-04-25T00:19:16.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.819+0000 7f4b78bd3740 1 -- 172.21.15.107:0/275471282 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564576781c10 con 0x564576700630 2024-04-25T00:19:16.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.819+0000 7f4b4effd640 1 -- 172.21.15.107:0/275471282 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b640054e0 con 0x564576700630 2024-04-25T00:19:16.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.819+0000 7f4b4effd640 1 -- 172.21.15.107:0/275471282 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f4b6404fae0 con 0x564576700630 2024-04-25T00:19:16.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.819+0000 7f4b78bd3740 10 client.25657 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:19:16.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.819+0000 7f4b78bd3740 10 client.25657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:16.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.819+0000 7f4b78bd3740 20 client.25657 populate_metadata read hostname 'smithi107' 2024-04-25T00:19:16.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.819+0000 7f4b78bd3740 1 --2- 172.21.15.107:0/275471282 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x564576782290 0x5645767846b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:16.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.819+0000 7f4b78bd3740 4 client.25657 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:19:16.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.819+0000 7f4b78bd3740 1 -- 172.21.15.107:0/275471282 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56457666cf40 con 0x564576782290 2024-04-25T00:19:16.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.820+0000 7f4b6f7fe640 1 --2- 172.21.15.107:0/275471282 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x564576782290 0x5645767846b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:16.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.820+0000 7f4b6f7fe640 1 --2- 172.21.15.107:0/275471282 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x564576782290 0x5645767846b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f54324f84169173 server_cookie=67c9979128d6b59e in_seq=0 out_seq=0 2024-04-25T00:19:16.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.820+0000 7f4b4effd640 10 client.25657 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:16.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.822+0000 7f4b4effd640 1 -- 172.21.15.107:0/275471282 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x56457666cf40 con 0x564576782290 2024-04-25T00:19:16.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.822+0000 7f4b4effd640 10 client.25657 handle_command_reply: tid=0 2024-04-25T00:19:16.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.822+0000 7f4b78bd3740 1 client.25657 shutdown 2024-04-25T00:19:16.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.822+0000 7f4b78bd3740 1 -- 172.21.15.107:0/275471282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564576700630 msgr2=0x564576713960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:16.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.822+0000 7f4b78bd3740 1 --2- 172.21.15.107:0/275471282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564576700630 0x564576713960 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4b64017000 tx=0x7f4b64037040 comp rx=0 tx=0).stop 2024-04-25T00:19:16.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.822+0000 7f4b78bd3740 1 -- 172.21.15.107:0/275471282 shutdown_connections 2024-04-25T00:19:16.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.822+0000 7f4b78bd3740 1 --2- 172.21.15.107:0/275471282 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x564576782290 0x5645767846b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.822+0000 7f4b78bd3740 1 --2- 172.21.15.107:0/275471282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56457670b620 0x56457670f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.822+0000 7f4b78bd3740 1 --2- 172.21.15.107:0/275471282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564576700f50 0x56457670ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.822+0000 7f4b78bd3740 1 --2- 172.21.15.107:0/275471282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564576700630 0x564576713960 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.822+0000 7f4b4effd640 0 client.25657 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:16.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.822+0000 7f4b78bd3740 1 -- 172.21.15.107:0/275471282 >> 172.21.15.107:0/275471282 conn(0x5645766fc950 msgr2=0x564576708d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:16.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.822+0000 7f4b78bd3740 1 -- 172.21.15.107:0/275471282 shutdown_connections 2024-04-25T00:19:16.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.822+0000 7f4b78bd3740 1 -- 172.21.15.107:0/275471282 wait complete. 2024-04-25T00:19:16.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.822+0000 7f4b78bd3740 20 client.25657 trim_cache size 0 max 16384 2024-04-25T00:19:16.827 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:19:16.827 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "scrub active (266 inodes in the stack)", 2024-04-25T00:19:16.827 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-04-25T00:19:16.827 INFO:teuthology.orchestra.run.smithi107.stdout: "e0594457-3687-44f4-a23d-854ab2963516": { 2024-04-25T00:19:16.827 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-04-25T00:19:16.827 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "e0594457-3687-44f4-a23d-854ab2963516", 2024-04-25T00:19:16.827 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-04-25T00:19:16.827 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:19:16.827 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:19:16.827 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:19:16.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.822+0000 7f4b75ebd640 1 -- 172.21.15.107:0/2955808349 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4b580a2d00 msgr2=0x7f4b580a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:16.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.822+0000 7f4b75ebd640 1 --2- 172.21.15.107:0/2955808349 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4b580a2d00 0x7f4b580a51c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4b680a46c0 tx=0x7f4b680c6b10 comp rx=0 tx=0).stop 2024-04-25T00:19:16.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.822+0000 7f4b75ebd640 1 -- 172.21.15.107:0/2955808349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b701066b0 msgr2=0x7f4b7019c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:16.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.822+0000 7f4b75ebd640 1 --2- 172.21.15.107:0/2955808349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b701066b0 0x7f4b7019c880 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4b68009fd0 tx=0x7f4b6800ef10 comp rx=0 tx=0).stop 2024-04-25T00:19:16.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.822+0000 7f4b75ebd640 1 -- 172.21.15.107:0/2955808349 shutdown_connections 2024-04-25T00:19:16.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.823+0000 7f4b75ebd640 1 --2- 172.21.15.107:0/2955808349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b701066b0 0x7f4b7019c880 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.823+0000 7f4b75ebd640 1 --2- 172.21.15.107:0/2955808349 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4b580a2d00 0x7f4b580a51c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.823+0000 7f4b75ebd640 1 --2- 172.21.15.107:0/2955808349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b70100fd0 0x7f4b70198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.823+0000 7f4b75ebd640 1 --2- 172.21.15.107:0/2955808349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4b70100610 0x7f4b70197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:16.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.823+0000 7f4b75ebd640 1 -- 172.21.15.107:0/2955808349 >> 172.21.15.107:0/2955808349 conn(0x7f4b70077740 msgr2=0x7f4b700fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:16.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.823+0000 7f4b75ebd640 1 -- 172.21.15.107:0/2955808349 shutdown_connections 2024-04-25T00:19:16.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:16.823+0000 7f4b75ebd640 1 -- 172.21.15.107:0/2955808349 wait complete. 2024-04-25T00:19:16.834 INFO:tasks.cephfs.filesystem:scrub status for tag:e0594457-3687-44f4-a23d-854ab2963516 - {'path': '/', 'tag': 'e0594457-3687-44f4-a23d-854ab2963516', 'options': 'recursive,force'} 2024-04-25T00:19:17.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:17.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:17.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:17.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:17.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:17.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:17.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:17.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.235 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:17.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:17.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:17.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:17.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:17.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:17.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:17.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:17.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:17.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:17.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:17.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:17.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:17.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:17.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:17.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:17.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:16 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:17.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:17.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:17.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:17.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:17.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:17.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:17.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:17.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:17.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:17.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:17.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:17.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:17.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:17.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:17.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:17.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:17.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:17.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:17.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:17.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:17.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:17.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:17.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:16 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:17.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:17.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:17.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:17.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:17.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:17.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:17.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:17.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:17.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:17.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:17.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:17.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:17.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:17.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:17.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:17.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:17.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:17.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:17.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:17.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:17.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:17.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:17.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:17.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:16 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:17 smithi195 ceph-mon[32833]: pgmap v88: 129 pgs: 129 active+clean; 206 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 279 op/s 2024-04-25T00:19:18.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:17 smithi195 ceph-mon[32833]: scrub queued for path: / 2024-04-25T00:19:18.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:17 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:17 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:18.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:17 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:18.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:17 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:17 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:18.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:17 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:17 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:18.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:17 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:17 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:18.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:17 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:17 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:18.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:17 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:17 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:18.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:17 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:17 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:18.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:17 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:17 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:18.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:17 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:17 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:18.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:17 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:18 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:18.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:18 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:18 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:18.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:18 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:18 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:18.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:18 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.341 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:18 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:18.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:18 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:18 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:18.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:18 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:18 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:18.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:18 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:18 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:18.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:18 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:18 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:18.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:18 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:18 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:18.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:18 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:18 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:18.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:18 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.342 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:18 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:18.343 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:18 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:17 smithi170 ceph-mon[32807]: pgmap v88: 129 pgs: 129 active+clean; 206 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 279 op/s 2024-04-25T00:19:18.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:17 smithi170 ceph-mon[32807]: scrub queued for path: / 2024-04-25T00:19:18.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:17 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:17 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:18.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:17 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:18.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:17 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:18.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:18.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:18.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:18.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:18.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:18.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:18.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:18.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:18.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:18.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:18.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:18.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:18.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:18.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:18.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:18.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:18.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:18.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:18.439 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:18 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: pgmap v88: 129 pgs: 129 active+clean; 206 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 279 op/s 2024-04-25T00:19:18.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub queued for path: / 2024-04-25T00:19:18.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:18.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:18.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:18.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:18.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:18.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:18.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:18.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:18.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:18.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:18.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:18.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:18.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:18.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:18.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:18.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:18.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:18.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:18.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:17 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:18.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:18 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:18 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:18.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:18 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:18.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:18 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:18.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:18 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:19.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:19.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:19.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:19.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:19.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:19.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:19.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:19.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:19.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:19.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:19.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:19.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:19.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:19.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:19.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:20.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:20 smithi195 ceph-mon[32833]: pgmap v89: 129 pgs: 129 active+clean; 216 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 427 op/s 2024-04-25T00:19:20.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:20 smithi170 ceph-mon[32807]: pgmap v89: 129 pgs: 129 active+clean; 216 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 427 op/s 2024-04-25T00:19:20.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:20 smithi107 ceph-mon[29153]: pgmap v89: 129 pgs: 129 active+clean; 216 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 427 op/s 2024-04-25T00:19:22.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:22 smithi195 ceph-mon[32833]: pgmap v90: 129 pgs: 129 active+clean; 216 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 295 op/s 2024-04-25T00:19:22.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:22 smithi170 ceph-mon[32807]: pgmap v90: 129 pgs: 129 active+clean; 216 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 295 op/s 2024-04-25T00:19:22.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:22 smithi107 ceph-mon[29153]: pgmap v90: 129 pgs: 129 active+clean; 216 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 295 op/s 2024-04-25T00:19:24.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:24 smithi195 ceph-mon[32833]: pgmap v91: 129 pgs: 129 active+clean; 225 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 428 op/s 2024-04-25T00:19:24.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:24 smithi170 ceph-mon[32807]: pgmap v91: 129 pgs: 129 active+clean; 225 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 428 op/s 2024-04-25T00:19:24.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:24 smithi107 ceph-mon[29153]: pgmap v91: 129 pgs: 129 active+clean; 225 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 428 op/s 2024-04-25T00:19:26.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:26 smithi195 ceph-mon[32833]: pgmap v92: 129 pgs: 129 active+clean; 225 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 280 op/s 2024-04-25T00:19:26.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:26 smithi170 ceph-mon[32807]: pgmap v92: 129 pgs: 129 active+clean; 225 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 280 op/s 2024-04-25T00:19:26.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:26 smithi107 ceph-mon[29153]: pgmap v92: 129 pgs: 129 active+clean; 225 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 280 op/s 2024-04-25T00:19:28.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:28 smithi195 ceph-mon[32833]: pgmap v93: 129 pgs: 129 active+clean; 225 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 280 op/s 2024-04-25T00:19:28.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:28 smithi170 ceph-mon[32807]: pgmap v93: 129 pgs: 129 active+clean; 225 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 280 op/s 2024-04-25T00:19:28.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:28 smithi107 ceph-mon[29153]: pgmap v93: 129 pgs: 129 active+clean; 225 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 280 op/s 2024-04-25T00:19:30.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:30 smithi195 ceph-mon[32833]: pgmap v94: 129 pgs: 129 active+clean; 234 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 415 op/s 2024-04-25T00:19:30.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:30 smithi170 ceph-mon[32807]: pgmap v94: 129 pgs: 129 active+clean; 234 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 415 op/s 2024-04-25T00:19:30.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:30 smithi107 ceph-mon[29153]: pgmap v94: 129 pgs: 129 active+clean; 234 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 415 op/s 2024-04-25T00:19:32.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:32 smithi195 ceph-mon[32833]: pgmap v95: 129 pgs: 129 active+clean; 234 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 267 op/s 2024-04-25T00:19:32.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:32 smithi170 ceph-mon[32807]: pgmap v95: 129 pgs: 129 active+clean; 234 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 267 op/s 2024-04-25T00:19:32.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:32 smithi107 ceph-mon[29153]: pgmap v95: 129 pgs: 129 active+clean; 234 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 267 op/s 2024-04-25T00:19:33.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:33 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:19:33.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:33 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:19:33.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:33 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:19:33.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:33 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:19:33.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:33 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:19:33.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:33 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:19:34.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:34 smithi195 ceph-mon[32833]: pgmap v96: 129 pgs: 129 active+clean; 242 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 392 op/s 2024-04-25T00:19:34.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:34 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:19:34.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:34 smithi170 ceph-mon[32807]: pgmap v96: 129 pgs: 129 active+clean; 242 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 392 op/s 2024-04-25T00:19:34.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:34 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:19:34.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:34 smithi107 ceph-mon[29153]: pgmap v96: 129 pgs: 129 active+clean; 242 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 392 op/s 2024-04-25T00:19:34.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:34 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:19:35.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:35 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:19:35.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:35 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:35.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:35 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:19:35.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:35 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:19:35.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:35 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:19:35.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:35 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:35.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:35 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:19:35.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:35 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:19:35.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:35 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:19:35.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:35 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:35.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:35 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:19:35.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:35 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:19:36.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:36 smithi195 ceph-mon[32833]: pgmap v97: 129 pgs: 129 active+clean; 242 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 259 op/s 2024-04-25T00:19:36.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:36 smithi170 ceph-mon[32807]: pgmap v97: 129 pgs: 129 active+clean; 242 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 259 op/s 2024-04-25T00:19:36.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:36 smithi107 ceph-mon[29153]: pgmap v97: 129 pgs: 129 active+clean; 242 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 259 op/s 2024-04-25T00:19:38.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:38 smithi195 ceph-mon[32833]: pgmap v98: 129 pgs: 129 active+clean; 242 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 259 op/s 2024-04-25T00:19:38.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:38 smithi170 ceph-mon[32807]: pgmap v98: 129 pgs: 129 active+clean; 242 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 259 op/s 2024-04-25T00:19:38.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:38 smithi107 ceph-mon[29153]: pgmap v98: 129 pgs: 129 active+clean; 242 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 259 op/s 2024-04-25T00:19:38.607 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-04-25T00:19:38.608 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T00:19:38.897 DEBUG:teuthology.parallel:result is None 2024-04-25T00:19:38.897 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T00:19:38.933 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T00:19:38.933 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-25T00:19:38.946 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-25T00:19:38.946 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T00:19:38.946 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T00:19:38.946 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-25T00:19:38.998 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-25T00:19:40.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:40 smithi195 ceph-mon[32833]: pgmap v99: 129 pgs: 129 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 389 op/s 2024-04-25T00:19:40.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:40 smithi170 ceph-mon[32807]: pgmap v99: 129 pgs: 129 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 389 op/s 2024-04-25T00:19:40.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:40 smithi107 ceph-mon[29153]: pgmap v99: 129 pgs: 129 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 389 op/s 2024-04-25T00:19:42.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:42 smithi195 ceph-mon[32833]: pgmap v100: 129 pgs: 129 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 255 op/s 2024-04-25T00:19:42.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:42 smithi170 ceph-mon[32807]: pgmap v100: 129 pgs: 129 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 255 op/s 2024-04-25T00:19:42.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:42 smithi107 ceph-mon[29153]: pgmap v100: 129 pgs: 129 active+clean; 251 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 255 op/s 2024-04-25T00:19:44.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:44 smithi195 ceph-mon[32833]: pgmap v101: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 314 op/s 2024-04-25T00:19:44.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:44 smithi170 ceph-mon[32807]: pgmap v101: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 314 op/s 2024-04-25T00:19:44.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:44 smithi107 ceph-mon[29153]: pgmap v101: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 314 op/s 2024-04-25T00:19:46.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:46 smithi195 ceph-mon[32833]: pgmap v102: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.0 MiB/s wr, 188 op/s 2024-04-25T00:19:46.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:46 smithi170 ceph-mon[32807]: pgmap v102: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.0 MiB/s wr, 188 op/s 2024-04-25T00:19:46.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:46 smithi107 ceph-mon[29153]: pgmap v102: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.0 MiB/s wr, 188 op/s 2024-04-25T00:19:46.835 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:19:46.836 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:19:46.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.948+0000 7f96983ee640 1 -- 172.21.15.107:0/4148436139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9690106630 msgr2=0x7f9690106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:46.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.948+0000 7f96983ee640 1 --2- 172.21.15.107:0/4148436139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9690106630 0x7f9690106a10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9684009930 tx=0x7f968402a420 comp rx=0 tx=0).stop 2024-04-25T00:19:46.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.948+0000 7f96983ee640 1 -- 172.21.15.107:0/4148436139 shutdown_connections 2024-04-25T00:19:46.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.948+0000 7f96983ee640 1 --2- 172.21.15.107:0/4148436139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9690100fe0 0x7f969010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.948+0000 7f96983ee640 1 --2- 172.21.15.107:0/4148436139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9690100620 0x7f9690100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.948+0000 7f96983ee640 1 --2- 172.21.15.107:0/4148436139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9690106630 0x7f9690106a10 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.948+0000 7f96983ee640 1 -- 172.21.15.107:0/4148436139 >> 172.21.15.107:0/4148436139 conn(0x7f96900776a0 msgr2=0x7f9690077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:46.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.948+0000 7f96983ee640 1 -- 172.21.15.107:0/4148436139 shutdown_connections 2024-04-25T00:19:46.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.948+0000 7f96983ee640 1 -- 172.21.15.107:0/4148436139 wait complete. 2024-04-25T00:19:46.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.949+0000 7f96983ee640 1 Processor -- start 2024-04-25T00:19:46.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.949+0000 7f96983ee640 1 -- start start 2024-04-25T00:19:46.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.949+0000 7f96983ee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9690100620 0x7f9690198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.949+0000 7f96983ee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9690100fe0 0x7f9690198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.949+0000 7f96983ee640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9690106630 0x7f969019c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.949+0000 7f96983ee640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9690111b40 con 0x7f9690100620 2024-04-25T00:19:46.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.949+0000 7f96983ee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f96901119c0 con 0x7f9690100fe0 2024-04-25T00:19:46.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.949+0000 7f96983ee640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9690111cc0 con 0x7f9690106630 2024-04-25T00:19:46.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.949+0000 7f969618a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9690100620 0x7f9690198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.949+0000 7f969618a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9690100620 0x7f9690198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55340/0 (socket says 172.21.15.107:55340) 2024-04-25T00:19:46.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.949+0000 7f969618a640 1 -- 172.21.15.107:0/745930129 learned_addr learned my addr 172.21.15.107:0/745930129 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:46.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.949+0000 7f969698b640 1 --2- 172.21.15.107:0/745930129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9690106630 0x7f969019c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.949+0000 7f969618a640 1 -- 172.21.15.107:0/745930129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9690106630 msgr2=0x7f969019c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:46.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.949+0000 7f9695989640 1 --2- 172.21.15.107:0/745930129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9690100fe0 0x7f9690198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.949+0000 7f969618a640 1 --2- 172.21.15.107:0/745930129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9690106630 0x7f969019c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.949+0000 7f969618a640 1 -- 172.21.15.107:0/745930129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9690100fe0 msgr2=0x7f9690198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:46.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.949+0000 7f969618a640 1 --2- 172.21.15.107:0/745930129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9690100fe0 0x7f9690198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.949+0000 7f969618a640 1 -- 172.21.15.107:0/745930129 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f969019d110 con 0x7f9690100620 2024-04-25T00:19:46.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.949+0000 7f969698b640 1 --2- 172.21.15.107:0/745930129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9690106630 0x7f969019c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:46.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.949+0000 7f969618a640 1 --2- 172.21.15.107:0/745930129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9690100620 0x7f9690198120 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9684009230 tx=0x7f9684004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:46.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.950+0000 7f96837fe640 1 -- 172.21.15.107:0/745930129 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968403f030 con 0x7f9690100620 2024-04-25T00:19:46.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.950+0000 7f96983ee640 1 -- 172.21.15.107:0/745930129 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f969019d3a0 con 0x7f9690100620 2024-04-25T00:19:46.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.950+0000 7f96837fe640 1 -- 172.21.15.107:0/745930129 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9684003ad0 con 0x7f9690100620 2024-04-25T00:19:46.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.950+0000 7f96837fe640 1 -- 172.21.15.107:0/745930129 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684035070 con 0x7f9690100620 2024-04-25T00:19:46.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.950+0000 7f96983ee640 1 -- 172.21.15.107:0/745930129 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96901a4c90 con 0x7f9690100620 2024-04-25T00:19:46.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.952+0000 7f96837fe640 1 -- 172.21.15.107:0/745930129 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f9684042030 con 0x7f9690100620 2024-04-25T00:19:46.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.952+0000 7f969b1dd740 1 Processor -- start 2024-04-25T00:19:46.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.953+0000 7f969b1dd740 1 -- start start 2024-04-25T00:19:46.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.953+0000 7f96837fe640 1 --2- 172.21.15.107:0/745930129 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f967c0a2d00 0x7f967c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.953+0000 7f969b1dd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460d902630 0x56460d902a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.953+0000 7f969b1dd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d902f50 0x56460d90d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.953+0000 7f969b1dd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56460d90d620 0x56460d90fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.953+0000 7f969b1dd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56460d8676b0 con 0x56460d902f50 2024-04-25T00:19:46.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.953+0000 7f969b1dd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56460d80f880 con 0x56460d902630 2024-04-25T00:19:46.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.953+0000 7f969b1dd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56460d863670 con 0x56460d90d620 2024-04-25T00:19:46.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.953+0000 7f9695989640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d902f50 0x56460d90d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.953+0000 7f969698b640 1 --2- 172.21.15.107:0/745930129 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f967c0a2d00 0x7f967c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.953+0000 7f9695989640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d902f50 0x56460d90d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55344/0 (socket says 172.21.15.107:55344) 2024-04-25T00:19:46.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.953+0000 7f9695989640 1 -- 172.21.15.107:0/1903770724 learned_addr learned my addr 172.21.15.107:0/1903770724 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:46.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.953+0000 7f969618a640 1 --2- 172.21.15.107:0/1903770724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460d902630 0x56460d902a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.953+0000 7f96837fe640 1 -- 172.21.15.107:0/745930129 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f96840ef9d0 con 0x7f9690100620 2024-04-25T00:19:46.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.953+0000 7f9695989640 1 -- 172.21.15.107:0/1903770724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56460d90d620 msgr2=0x56460d90fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:46.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.953+0000 7f9695989640 1 --2- 172.21.15.107:0/1903770724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56460d90d620 0x56460d90fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.953+0000 7f9695989640 1 -- 172.21.15.107:0/1903770724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460d902630 msgr2=0x56460d902a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:46.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.953+0000 7f9695989640 1 --2- 172.21.15.107:0/1903770724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460d902630 0x56460d902a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.953+0000 7f9695989640 1 -- 172.21.15.107:0/1903770724 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56460d6457d0 con 0x56460d902f50 2024-04-25T00:19:46.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.953+0000 7f969698b640 1 --2- 172.21.15.107:0/745930129 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f967c0a2d00 0x7f967c0a51c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9688009270 tx=0x7f9688009880 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:46.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.953+0000 7f9695989640 1 --2- 172.21.15.107:0/1903770724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d902f50 0x56460d90d0e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f968c00a8e0 tx=0x7f968c00ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf973c3a6593271 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:46.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.954+0000 7f9680bfe640 1 -- 172.21.15.107:0/1903770724 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c015070 con 0x56460d902f50 2024-04-25T00:19:46.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.954+0000 7f9680bfe640 1 -- 172.21.15.107:0/1903770724 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f968c0113f0 con 0x56460d902f50 2024-04-25T00:19:46.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.954+0000 7f9680bfe640 1 -- 172.21.15.107:0/1903770724 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c010590 con 0x56460d902f50 2024-04-25T00:19:46.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.954+0000 7f969b1dd740 1 -- 172.21.15.107:0/1903770724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d902f50 msgr2=0x56460d90d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:46.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.954+0000 7f969b1dd740 1 --2- 172.21.15.107:0/1903770724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d902f50 0x56460d90d0e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f968c00a8e0 tx=0x7f968c00ada0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.954+0000 7f969b1dd740 1 -- 172.21.15.107:0/1903770724 shutdown_connections 2024-04-25T00:19:46.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.954+0000 7f969b1dd740 1 --2- 172.21.15.107:0/1903770724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56460d90d620 0x56460d90fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.954+0000 7f969b1dd740 1 --2- 172.21.15.107:0/1903770724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d902f50 0x56460d90d0e0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.954+0000 7f969b1dd740 1 --2- 172.21.15.107:0/1903770724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460d902630 0x56460d902a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.954+0000 7f969b1dd740 1 -- 172.21.15.107:0/1903770724 >> 172.21.15.107:0/1903770724 conn(0x56460d8fe950 msgr2=0x56460d8fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:46.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.955+0000 7f969b1dd740 1 -- 172.21.15.107:0/1903770724 shutdown_connections 2024-04-25T00:19:46.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.955+0000 7f969b1dd740 1 -- 172.21.15.107:0/1903770724 wait complete. 2024-04-25T00:19:46.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.955+0000 7f969b1dd740 1 -- start start 2024-04-25T00:19:46.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.955+0000 7f969b1dd740 1 Processor -- start 2024-04-25T00:19:46.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.955+0000 7f969b1dd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56460d902630 0x56460d90a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.955+0000 7f969b1dd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460d902f50 0x56460d90a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.955+0000 7f969b1dd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d90d620 0x56460d90af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.955+0000 7f969b1dd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56460d910410 con 0x56460d90d620 2024-04-25T00:19:46.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.955+0000 7f969b1dd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56460d822f50 con 0x56460d902f50 2024-04-25T00:19:46.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.955+0000 7f969b1dd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56460d910590 con 0x56460d902630 2024-04-25T00:19:46.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.955+0000 7f969b1dd740 10 client.0 _ll_register_callbacks cb 0x56460d85e000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:46.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.955+0000 7f969698b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d90d620 0x56460d90af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.955+0000 7f969698b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d90d620 0x56460d90af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55354/0 (socket says 172.21.15.107:55354) 2024-04-25T00:19:46.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.955+0000 7f969698b640 1 -- 172.21.15.107:0/3682971328 learned_addr learned my addr 172.21.15.107:0/3682971328 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:46.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.955+0000 7f9695989640 1 --2- 172.21.15.107:0/3682971328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460d902f50 0x56460d90a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.955+0000 7f969698b640 1 -- 172.21.15.107:0/3682971328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56460d902630 msgr2=0x56460d90a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:46.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.955+0000 7f969698b640 1 --2- 172.21.15.107:0/3682971328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56460d902630 0x56460d90a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.955+0000 7f969698b640 1 -- 172.21.15.107:0/3682971328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460d902f50 msgr2=0x56460d90a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:46.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.955+0000 7f969698b640 1 --2- 172.21.15.107:0/3682971328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460d902f50 0x56460d90a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.956+0000 7f969698b640 1 -- 172.21.15.107:0/3682971328 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56460d57b3f0 con 0x56460d90d620 2024-04-25T00:19:46.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.956+0000 7f9695989640 1 --2- 172.21.15.107:0/3682971328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460d902f50 0x56460d90a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:46.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.956+0000 7f969698b640 1 --2- 172.21.15.107:0/3682971328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d90d620 0x56460d90af30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f96880127f0 tx=0x7f9688004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c543be3eb0136a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:46.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.956+0000 7f9679ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:19:46.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.956+0000 7f9679ffb640 1 -- 172.21.15.107:0/3682971328 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9688023070 con 0x56460d90d620 2024-04-25T00:19:46.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.956+0000 7f9679ffb640 1 -- 172.21.15.107:0/3682971328 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f968800ece0 con 0x56460d90d620 2024-04-25T00:19:46.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.956+0000 7f969b1dd740 1 -- 172.21.15.107:0/3682971328 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f968c0113f0 con 0x56460d90d620 2024-04-25T00:19:46.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.956+0000 7f9679ffb640 1 -- 172.21.15.107:0/3682971328 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f968802f050 con 0x56460d90d620 2024-04-25T00:19:46.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.956+0000 7f969b1dd740 10 client.16302 fetch_fsmap learned FSMap version 28 2024-04-25T00:19:46.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.956+0000 7f969b1dd740 1 -- 172.21.15.107:0/3682971328 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56460d53b0f0 con 0x56460d90d620 2024-04-25T00:19:46.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.956+0000 7f9679ffb640 1 -- 172.21.15.107:0/3682971328 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f968c0113f0 con 0x56460d90d620 2024-04-25T00:19:46.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.956+0000 7f9679ffb640 1 -- 172.21.15.107:0/3682971328 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f96880273b0 con 0x56460d90d620 2024-04-25T00:19:46.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.956+0000 7f969b1dd740 10 client.16302 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:19:46.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.956+0000 7f969b1dd740 10 client.16302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:46.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.956+0000 7f969b1dd740 20 client.16302 populate_metadata read hostname 'smithi107' 2024-04-25T00:19:46.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.956+0000 7f969b1dd740 1 --2- 172.21.15.107:0/3682971328 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56460d91d830 0x56460d91fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.956+0000 7f969b1dd740 4 client.16302 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:19:46.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.956+0000 7f969b1dd740 1 -- 172.21.15.107:0/3682971328 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56460d874bf0 con 0x56460d91d830 2024-04-25T00:19:46.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.957+0000 7f969618a640 1 --2- 172.21.15.107:0/3682971328 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56460d91d830 0x56460d91fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.957+0000 7f9679ffb640 10 client.16302 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:46.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.957+0000 7f969618a640 1 --2- 172.21.15.107:0/3682971328 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56460d91d830 0x56460d91fcf0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb0bf2f28eca14d5 server_cookie=a495360eedce0359 in_seq=0 out_seq=0 2024-04-25T00:19:46.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.958+0000 7f9679ffb640 1 -- 172.21.15.107:0/3682971328 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56460d874bf0 con 0x56460d91d830 2024-04-25T00:19:46.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.958+0000 7f9679ffb640 10 client.16302 handle_command_reply: tid=0 2024-04-25T00:19:46.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.958+0000 7f969b1dd740 1 client.16302 shutdown 2024-04-25T00:19:46.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.958+0000 7f969b1dd740 1 -- 172.21.15.107:0/3682971328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d90d620 msgr2=0x56460d90af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:46.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.958+0000 7f969b1dd740 1 --2- 172.21.15.107:0/3682971328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d90d620 0x56460d90af30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f96880127f0 tx=0x7f9688004ab0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.958+0000 7f969b1dd740 1 -- 172.21.15.107:0/3682971328 shutdown_connections 2024-04-25T00:19:46.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.958+0000 7f969b1dd740 1 --2- 172.21.15.107:0/3682971328 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56460d91d830 0x56460d91fcf0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.958+0000 7f969b1dd740 1 --2- 172.21.15.107:0/3682971328 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d90d620 0x56460d90af30 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.958+0000 7f9679ffb640 0 client.16302 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:46.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.958+0000 7f969b1dd740 1 --2- 172.21.15.107:0/3682971328 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460d902f50 0x56460d90a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.958+0000 7f969b1dd740 1 --2- 172.21.15.107:0/3682971328 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56460d902630 0x56460d90a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.958+0000 7f969b1dd740 1 -- 172.21.15.107:0/3682971328 >> 172.21.15.107:0/3682971328 conn(0x56460d8fe950 msgr2=0x56460d904650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:46.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.958+0000 7f969b1dd740 1 -- 172.21.15.107:0/3682971328 shutdown_connections 2024-04-25T00:19:46.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.958+0000 7f969b1dd740 1 -- 172.21.15.107:0/3682971328 wait complete. 2024-04-25T00:19:46.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.958+0000 7f969b1dd740 20 client.16302 trim_cache size 0 max 16384 2024-04-25T00:19:46.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.975+0000 7f969b1dd740 1 Processor -- start 2024-04-25T00:19:46.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.975+0000 7f969b1dd740 1 -- start start 2024-04-25T00:19:46.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.976+0000 7f969b1dd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460d902630 0x56460d90cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.976+0000 7f969b1dd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d902f50 0x56460d919300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.976+0000 7f969b1dd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56460d90d620 0x56460d915bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.976+0000 7f969b1dd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f96901b1490 con 0x56460d902f50 2024-04-25T00:19:46.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.976+0000 7f969b1dd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f96901aeca0 con 0x56460d902630 2024-04-25T00:19:46.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.976+0000 7f969b1dd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f96901acaf0 con 0x56460d90d620 2024-04-25T00:19:46.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.976+0000 7f9695989640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d902f50 0x56460d919300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.976+0000 7f9695989640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d902f50 0x56460d919300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55366/0 (socket says 172.21.15.107:55366) 2024-04-25T00:19:46.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.976+0000 7f9695989640 1 -- 172.21.15.107:0/1324677138 learned_addr learned my addr 172.21.15.107:0/1324677138 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:46.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.976+0000 7f9695989640 1 -- 172.21.15.107:0/1324677138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56460d90d620 msgr2=0x56460d915bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:46.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.976+0000 7f969698b640 1 --2- 172.21.15.107:0/1324677138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56460d90d620 0x56460d915bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.976+0000 7f9695989640 1 --2- 172.21.15.107:0/1324677138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56460d90d620 0x56460d915bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.976+0000 7f9695989640 1 -- 172.21.15.107:0/1324677138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460d902630 msgr2=0x56460d90cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:46.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.976+0000 7f9695989640 1 --2- 172.21.15.107:0/1324677138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460d902630 0x56460d90cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.976+0000 7f9695989640 1 -- 172.21.15.107:0/1324677138 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56460d9052b0 con 0x56460d902f50 2024-04-25T00:19:46.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.976+0000 7f969698b640 1 --2- 172.21.15.107:0/1324677138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56460d90d620 0x56460d915bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:46.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.976+0000 7f9695989640 1 --2- 172.21.15.107:0/1324677138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d902f50 0x56460d919300 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x56460d903bf0 tx=0x7f968c00a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8debc063299aa2bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:46.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.976+0000 7f967bfff640 1 -- 172.21.15.107:0/1324677138 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c028070 con 0x56460d902f50 2024-04-25T00:19:46.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.976+0000 7f967bfff640 1 -- 172.21.15.107:0/1324677138 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f968c0113f0 con 0x56460d902f50 2024-04-25T00:19:46.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.976+0000 7f967bfff640 1 -- 172.21.15.107:0/1324677138 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968c015040 con 0x56460d902f50 2024-04-25T00:19:46.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.977+0000 7f969b1dd740 1 -- 172.21.15.107:0/1324677138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d902f50 msgr2=0x56460d919300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:46.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.977+0000 7f969b1dd740 1 --2- 172.21.15.107:0/1324677138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d902f50 0x56460d919300 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x56460d903bf0 tx=0x7f968c00a120 comp rx=0 tx=0).stop 2024-04-25T00:19:46.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.977+0000 7f969b1dd740 1 -- 172.21.15.107:0/1324677138 shutdown_connections 2024-04-25T00:19:46.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.977+0000 7f969b1dd740 1 --2- 172.21.15.107:0/1324677138 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56460d90d620 0x56460d915bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.977+0000 7f969b1dd740 1 --2- 172.21.15.107:0/1324677138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d902f50 0x56460d919300 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.977+0000 7f969b1dd740 1 --2- 172.21.15.107:0/1324677138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460d902630 0x56460d90cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.977+0000 7f969b1dd740 1 -- 172.21.15.107:0/1324677138 >> 172.21.15.107:0/1324677138 conn(0x56460d8fe950 msgr2=0x56460d8fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:46.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.977+0000 7f969b1dd740 1 -- 172.21.15.107:0/1324677138 shutdown_connections 2024-04-25T00:19:46.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.977+0000 7f969b1dd740 1 -- 172.21.15.107:0/1324677138 wait complete. 2024-04-25T00:19:46.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.977+0000 7f969b1dd740 1 -- start start 2024-04-25T00:19:46.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.977+0000 7f969b1dd740 1 Processor -- start 2024-04-25T00:19:46.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.977+0000 7f969b1dd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56460d902630 0x56460d980750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.977+0000 7f969b1dd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460d902f50 0x56460d980c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.978+0000 7f969b1dd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d90d620 0x56460d924e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.978+0000 7f969b1dd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f96901ad160 con 0x56460d90d620 2024-04-25T00:19:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.978+0000 7f969b1dd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f96901ae5d0 con 0x56460d902f50 2024-04-25T00:19:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.978+0000 7f969b1dd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f96901ad830 con 0x56460d902630 2024-04-25T00:19:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.978+0000 7f969b1dd740 10 client.0 _ll_register_callbacks cb 0x7f96880124c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.978+0000 7f969698b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d90d620 0x56460d924e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.978+0000 7f969698b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d90d620 0x56460d924e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55372/0 (socket says 172.21.15.107:55372) 2024-04-25T00:19:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.978+0000 7f969698b640 1 -- 172.21.15.107:0/3951402785 learned_addr learned my addr 172.21.15.107:0/3951402785 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.978+0000 7f969618a640 1 --2- 172.21.15.107:0/3951402785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56460d902630 0x56460d980750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.978+0000 7f969698b640 1 -- 172.21.15.107:0/3951402785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56460d902630 msgr2=0x56460d980750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.978+0000 7f969698b640 1 --2- 172.21.15.107:0/3951402785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56460d902630 0x56460d980750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.978+0000 7f969698b640 1 -- 172.21.15.107:0/3951402785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460d902f50 msgr2=0x56460d980c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:46.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.978+0000 7f9695989640 1 --2- 172.21.15.107:0/3951402785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460d902f50 0x56460d980c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.978+0000 7f969698b640 1 --2- 172.21.15.107:0/3951402785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460d902f50 0x56460d980c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.978+0000 7f969698b640 1 -- 172.21.15.107:0/3951402785 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56460d925510 con 0x56460d90d620 2024-04-25T00:19:46.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.978+0000 7f969698b640 1 --2- 172.21.15.107:0/3951402785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d90d620 0x56460d924e20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9688006010 tx=0x7f9688011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964e82959f86223 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:46.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.978+0000 7f9679ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:19:46.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.978+0000 7f9679ffb640 1 -- 172.21.15.107:0/3951402785 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9688023070 con 0x56460d90d620 2024-04-25T00:19:46.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.978+0000 7f9679ffb640 1 -- 172.21.15.107:0/3951402785 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f9688004b20 con 0x56460d90d620 2024-04-25T00:19:46.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.978+0000 7f969b1dd740 1 -- 172.21.15.107:0/3951402785 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f968c0113f0 con 0x56460d90d620 2024-04-25T00:19:46.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.978+0000 7f9679ffb640 1 -- 172.21.15.107:0/3951402785 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9688041050 con 0x56460d90d620 2024-04-25T00:19:46.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.978+0000 7f969b1dd740 10 client.16308 fetch_fsmap learned FSMap version 28 2024-04-25T00:19:46.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.978+0000 7f969b1dd740 1 -- 172.21.15.107:0/3951402785 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56460d987f00 con 0x56460d90d620 2024-04-25T00:19:46.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.979+0000 7f9679ffb640 1 -- 172.21.15.107:0/3951402785 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f968c0113f0 con 0x56460d90d620 2024-04-25T00:19:46.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.979+0000 7f9679ffb640 1 -- 172.21.15.107:0/3951402785 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f9688028740 con 0x56460d90d620 2024-04-25T00:19:46.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.979+0000 7f969b1dd740 10 client.16308 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:19:46.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.979+0000 7f969b1dd740 10 client.16308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:46.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.979+0000 7f969b1dd740 20 client.16308 populate_metadata read hostname 'smithi107' 2024-04-25T00:19:46.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.979+0000 7f969b1dd740 1 --2- 172.21.15.107:0/3951402785 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56460d9886d0 0x56460d98aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:46.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.979+0000 7f969b1dd740 4 client.16308 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:19:46.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.979+0000 7f969b1dd740 1 -- 172.21.15.107:0/3951402785 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56460d86ef40 con 0x56460d9886d0 2024-04-25T00:19:46.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.979+0000 7f969618a640 1 --2- 172.21.15.107:0/3951402785 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56460d9886d0 0x56460d98aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:46.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.979+0000 7f969618a640 1 --2- 172.21.15.107:0/3951402785 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56460d9886d0 0x56460d98aaf0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28ce67086a63cf1d server_cookie=7abf3c705736d02c in_seq=0 out_seq=0 2024-04-25T00:19:46.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.979+0000 7f9679ffb640 10 client.16308 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:46.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.979+0000 7f9679ffb640 1 -- 172.21.15.107:0/3951402785 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56460d86ef40 con 0x56460d9886d0 2024-04-25T00:19:46.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.979+0000 7f9679ffb640 10 client.16308 handle_command_reply: tid=0 2024-04-25T00:19:46.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.979+0000 7f969b1dd740 1 client.16308 shutdown 2024-04-25T00:19:46.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.980+0000 7f969b1dd740 1 -- 172.21.15.107:0/3951402785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d90d620 msgr2=0x56460d924e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:46.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.980+0000 7f969b1dd740 1 --2- 172.21.15.107:0/3951402785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d90d620 0x56460d924e20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9688006010 tx=0x7f9688011d70 comp rx=0 tx=0).stop 2024-04-25T00:19:46.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.980+0000 7f969b1dd740 1 -- 172.21.15.107:0/3951402785 shutdown_connections 2024-04-25T00:19:46.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.980+0000 7f969b1dd740 1 --2- 172.21.15.107:0/3951402785 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56460d9886d0 0x56460d98aaf0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.980+0000 7f969b1dd740 1 --2- 172.21.15.107:0/3951402785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56460d90d620 0x56460d924e20 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.980+0000 7f9679ffb640 0 client.16308 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:46.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.980+0000 7f969b1dd740 1 --2- 172.21.15.107:0/3951402785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56460d902f50 0x56460d980c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.980+0000 7f969b1dd740 1 --2- 172.21.15.107:0/3951402785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56460d902630 0x56460d980750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.980+0000 7f969b1dd740 1 -- 172.21.15.107:0/3951402785 >> 172.21.15.107:0/3951402785 conn(0x56460d8fe950 msgr2=0x56460d90a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:46.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.980+0000 7f969b1dd740 1 -- 172.21.15.107:0/3951402785 shutdown_connections 2024-04-25T00:19:46.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.980+0000 7f969b1dd740 1 -- 172.21.15.107:0/3951402785 wait complete. 2024-04-25T00:19:46.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.980+0000 7f969b1dd740 20 client.16308 trim_cache size 0 max 16384 2024-04-25T00:19:46.988 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:19:46.988 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-04-25T00:19:46.988 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-04-25T00:19:46.988 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:19:46.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.980+0000 7f96983ee640 1 -- 172.21.15.107:0/745930129 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f967c0a2d00 msgr2=0x7f967c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:46.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.980+0000 7f96983ee640 1 --2- 172.21.15.107:0/745930129 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f967c0a2d00 0x7f967c0a51c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9688009270 tx=0x7f9688009880 comp rx=0 tx=0).stop 2024-04-25T00:19:46.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.980+0000 7f96983ee640 1 -- 172.21.15.107:0/745930129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9690100620 msgr2=0x7f9690198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:46.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.980+0000 7f96983ee640 1 --2- 172.21.15.107:0/745930129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9690100620 0x7f9690198120 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f9684009230 tx=0x7f9684004270 comp rx=0 tx=0).stop 2024-04-25T00:19:46.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.980+0000 7f96983ee640 1 -- 172.21.15.107:0/745930129 shutdown_connections 2024-04-25T00:19:46.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.980+0000 7f96983ee640 1 --2- 172.21.15.107:0/745930129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9690106630 0x7f969019c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.980+0000 7f96983ee640 1 --2- 172.21.15.107:0/745930129 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f967c0a2d00 0x7f967c0a51c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.980+0000 7f96983ee640 1 --2- 172.21.15.107:0/745930129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9690100fe0 0x7f9690198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.980+0000 7f96983ee640 1 --2- 172.21.15.107:0/745930129 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9690100620 0x7f9690198120 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:46.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.980+0000 7f96983ee640 1 -- 172.21.15.107:0/745930129 >> 172.21.15.107:0/745930129 conn(0x7f96900776a0 msgr2=0x7f96901031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:46.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.980+0000 7f96983ee640 1 -- 172.21.15.107:0/745930129 shutdown_connections 2024-04-25T00:19:46.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:46.980+0000 7f96983ee640 1 -- 172.21.15.107:0/745930129 wait complete. 2024-04-25T00:19:46.991 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:19:46.992 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:19:47.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.090+0000 7f9872d63640 1 -- 172.21.15.107:0/4265291980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f986c107720 msgr2=0x7f986c107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.090+0000 7f9872d63640 1 --2- 172.21.15.107:0/4265291980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f986c107720 0x7f986c107b00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9860009930 tx=0x7f986002a420 comp rx=0 tx=0).stop 2024-04-25T00:19:47.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.090+0000 7f9872d63640 1 -- 172.21.15.107:0/4265291980 shutdown_connections 2024-04-25T00:19:47.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.090+0000 7f9872d63640 1 --2- 172.21.15.107:0/4265291980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f986c1020d0 0x7f986c10e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.090+0000 7f9872d63640 1 --2- 172.21.15.107:0/4265291980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f986c101710 0x7f986c101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.090+0000 7f9872d63640 1 --2- 172.21.15.107:0/4265291980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f986c107720 0x7f986c107b00 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.090+0000 7f9872d63640 1 -- 172.21.15.107:0/4265291980 >> 172.21.15.107:0/4265291980 conn(0x7f986c078b00 msgr2=0x7f986c078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.090+0000 7f9872d63640 1 -- 172.21.15.107:0/4265291980 shutdown_connections 2024-04-25T00:19:47.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.090+0000 7f9872d63640 1 -- 172.21.15.107:0/4265291980 wait complete. 2024-04-25T00:19:47.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.090+0000 7f9872d63640 1 Processor -- start 2024-04-25T00:19:47.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.090+0000 7f9872d63640 1 -- start start 2024-04-25T00:19:47.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.090+0000 7f9872d63640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f986c101710 0x7f986c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.090+0000 7f9872d63640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f986c1020d0 0x7f986c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.090+0000 7f9872d63640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f986c107720 0x7f986c19c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.090+0000 7f9872d63640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f986c111af0 con 0x7f986c101710 2024-04-25T00:19:47.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.090+0000 7f9872d63640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f986c111970 con 0x7f986c107720 2024-04-25T00:19:47.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.090+0000 7f9872d63640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f986c111c70 con 0x7f986c1020d0 2024-04-25T00:19:47.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.091+0000 7f9870aff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f986c101710 0x7f986c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.091+0000 7f9870aff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f986c101710 0x7f986c198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55388/0 (socket says 172.21.15.107:55388) 2024-04-25T00:19:47.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.091+0000 7f9870aff640 1 -- 172.21.15.107:0/172747286 learned_addr learned my addr 172.21.15.107:0/172747286 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:47.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.091+0000 7f986bfff640 1 --2- 172.21.15.107:0/172747286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f986c1020d0 0x7f986c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.091+0000 7f9870aff640 1 -- 172.21.15.107:0/172747286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f986c1020d0 msgr2=0x7f986c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.091+0000 7f9871300640 1 --2- 172.21.15.107:0/172747286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f986c107720 0x7f986c19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.091+0000 7f9870aff640 1 --2- 172.21.15.107:0/172747286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f986c1020d0 0x7f986c198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.091+0000 7f9870aff640 1 -- 172.21.15.107:0/172747286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f986c107720 msgr2=0x7f986c19c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.091+0000 7f9870aff640 1 --2- 172.21.15.107:0/172747286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f986c107720 0x7f986c19c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.091+0000 7f9870aff640 1 -- 172.21.15.107:0/172747286 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986c19d040 con 0x7f986c101710 2024-04-25T00:19:47.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.091+0000 7f986bfff640 1 --2- 172.21.15.107:0/172747286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f986c1020d0 0x7f986c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:47.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.091+0000 7f9870aff640 1 --2- 172.21.15.107:0/172747286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f986c101710 0x7f986c198050 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9860009230 tx=0x7f9860004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.091+0000 7f9869ffb640 1 -- 172.21.15.107:0/172747286 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986003f030 con 0x7f986c101710 2024-04-25T00:19:47.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.091+0000 7f9869ffb640 1 -- 172.21.15.107:0/172747286 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9860003ad0 con 0x7f986c101710 2024-04-25T00:19:47.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.091+0000 7f9872d63640 1 -- 172.21.15.107:0/172747286 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986c19d2d0 con 0x7f986c101710 2024-04-25T00:19:47.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.091+0000 7f9869ffb640 1 -- 172.21.15.107:0/172747286 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860035070 con 0x7f986c101710 2024-04-25T00:19:47.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.091+0000 7f9872d63640 1 -- 172.21.15.107:0/172747286 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f986c1a4bc0 con 0x7f986c101710 2024-04-25T00:19:47.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.092+0000 7f9872d63640 1 -- 172.21.15.107:0/172747286 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f986c0689b0 con 0x7f986c101710 2024-04-25T00:19:47.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.093+0000 7f9869ffb640 1 -- 172.21.15.107:0/172747286 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f9860042030 con 0x7f986c101710 2024-04-25T00:19:47.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.094+0000 7f9869ffb640 1 --2- 172.21.15.107:0/172747286 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f985c0a2d00 0x7f985c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.094+0000 7f986bfff640 1 --2- 172.21.15.107:0/172747286 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f985c0a2d00 0x7f985c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.094+0000 7f9869ffb640 1 -- 172.21.15.107:0/172747286 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f98600efd70 con 0x7f986c101710 2024-04-25T00:19:47.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.095+0000 7f986bfff640 1 --2- 172.21.15.107:0/172747286 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f985c0a2d00 0x7f985c0a51c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f986c103860 tx=0x7f9858005d90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.099+0000 7f9869ffb640 1 -- 172.21.15.107:0/172747286 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f98600bc620 con 0x7f986c101710 2024-04-25T00:19:47.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.312+0000 7f9872d63640 1 -- 172.21.15.107:0/172747286 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f986c1993a0 con 0x7f986c101710 2024-04-25T00:19:47.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.313+0000 7f9869ffb640 1 -- 172.21.15.107:0/172747286 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11906 (secure 0 0 0) 0x7f98600bbf20 con 0x7f986c101710 2024-04-25T00:19:47.313 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:19:47.313 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:18:38.933541+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24493,24521]},"id":1}]} 2024-04-25T00:19:47.314 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 28 2024-04-25T00:19:47.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.315+0000 7f9872d63640 1 -- 172.21.15.107:0/172747286 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f985c0a2d00 msgr2=0x7f985c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.315+0000 7f9872d63640 1 --2- 172.21.15.107:0/172747286 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f985c0a2d00 0x7f985c0a51c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f986c103860 tx=0x7f9858005d90 comp rx=0 tx=0).stop 2024-04-25T00:19:47.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.315+0000 7f9872d63640 1 -- 172.21.15.107:0/172747286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f986c101710 msgr2=0x7f986c198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.315+0000 7f9872d63640 1 --2- 172.21.15.107:0/172747286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f986c101710 0x7f986c198050 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9860009230 tx=0x7f9860004270 comp rx=0 tx=0).stop 2024-04-25T00:19:47.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.315+0000 7f9872d63640 1 -- 172.21.15.107:0/172747286 shutdown_connections 2024-04-25T00:19:47.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.315+0000 7f9872d63640 1 --2- 172.21.15.107:0/172747286 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f985c0a2d00 0x7f985c0a51c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.315+0000 7f9872d63640 1 --2- 172.21.15.107:0/172747286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f986c107720 0x7f986c19c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.315+0000 7f9872d63640 1 --2- 172.21.15.107:0/172747286 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f986c1020d0 0x7f986c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.315+0000 7f9872d63640 1 --2- 172.21.15.107:0/172747286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f986c101710 0x7f986c198050 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.315+0000 7f9872d63640 1 -- 172.21.15.107:0/172747286 >> 172.21.15.107:0/172747286 conn(0x7f986c078b00 msgr2=0x7f986c104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.315+0000 7f9872d63640 1 -- 172.21.15.107:0/172747286 shutdown_connections 2024-04-25T00:19:47.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.315+0000 7f9872d63640 1 -- 172.21.15.107:0/172747286 wait complete. 2024-04-25T00:19:47.326 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:19:47.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.425+0000 7f95a229a640 1 -- 172.21.15.107:0/3278482853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f959c1066b0 msgr2=0x7f959c106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.425+0000 7f95a229a640 1 --2- 172.21.15.107:0/3278482853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f959c1066b0 0x7f959c106a90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9590009930 tx=0x7f959002a420 comp rx=0 tx=0).stop 2024-04-25T00:19:47.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.426+0000 7f95a229a640 1 -- 172.21.15.107:0/3278482853 shutdown_connections 2024-04-25T00:19:47.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.426+0000 7f95a229a640 1 --2- 172.21.15.107:0/3278482853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f959c100fd0 0x7f959c10d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.426+0000 7f95a229a640 1 --2- 172.21.15.107:0/3278482853 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f959c100610 0x7f959c100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.426+0000 7f95a229a640 1 --2- 172.21.15.107:0/3278482853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f959c1066b0 0x7f959c106a90 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.426+0000 7f95a229a640 1 -- 172.21.15.107:0/3278482853 >> 172.21.15.107:0/3278482853 conn(0x7f959c077740 msgr2=0x7f959c077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.426+0000 7f95a229a640 1 -- 172.21.15.107:0/3278482853 shutdown_connections 2024-04-25T00:19:47.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.426+0000 7f95a229a640 1 -- 172.21.15.107:0/3278482853 wait complete. 2024-04-25T00:19:47.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.426+0000 7f95a229a640 1 Processor -- start 2024-04-25T00:19:47.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.426+0000 7f95a229a640 1 -- start start 2024-04-25T00:19:47.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.426+0000 7f95a229a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f959c100610 0x7f959c198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.426+0000 7f95a229a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f959c100fd0 0x7f959c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.426+0000 7f95a229a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f959c1066b0 0x7f959c19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.426+0000 7f95a229a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f959c111b80 con 0x7f959c100610 2024-04-25T00:19:47.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.426+0000 7f95a229a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f959c111a00 con 0x7f959c1066b0 2024-04-25T00:19:47.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.426+0000 7f95a229a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f959c111d00 con 0x7f959c100fd0 2024-04-25T00:19:47.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.427+0000 7f959bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f959c100610 0x7f959c198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.427+0000 7f959bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f959c100610 0x7f959c198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55402/0 (socket says 172.21.15.107:55402) 2024-04-25T00:19:47.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.427+0000 7f959bfff640 1 -- 172.21.15.107:0/1927742753 learned_addr learned my addr 172.21.15.107:0/1927742753 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:47.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.427+0000 7f959b7fe640 1 --2- 172.21.15.107:0/1927742753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f959c100fd0 0x7f959c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.427+0000 7f95a0837640 1 --2- 172.21.15.107:0/1927742753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f959c1066b0 0x7f959c19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.427+0000 7f959bfff640 1 -- 172.21.15.107:0/1927742753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f959c100fd0 msgr2=0x7f959c198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.427+0000 7f959bfff640 1 --2- 172.21.15.107:0/1927742753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f959c100fd0 0x7f959c198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.427+0000 7f959bfff640 1 -- 172.21.15.107:0/1927742753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f959c1066b0 msgr2=0x7f959c19c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.427+0000 7f959bfff640 1 --2- 172.21.15.107:0/1927742753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f959c1066b0 0x7f959c19c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.427+0000 7f959bfff640 1 -- 172.21.15.107:0/1927742753 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959c19d010 con 0x7f959c100610 2024-04-25T00:19:47.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.427+0000 7f959bfff640 1 --2- 172.21.15.107:0/1927742753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f959c100610 0x7f959c198020 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f959002a940 tx=0x7f9590003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.427+0000 7f95997fa640 1 -- 172.21.15.107:0/1927742753 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959003f070 con 0x7f959c100610 2024-04-25T00:19:47.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.427+0000 7f95a229a640 1 -- 172.21.15.107:0/1927742753 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959c19d2a0 con 0x7f959c100610 2024-04-25T00:19:47.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.427+0000 7f95997fa640 1 -- 172.21.15.107:0/1927742753 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f959003b460 con 0x7f959c100610 2024-04-25T00:19:47.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.427+0000 7f95997fa640 1 -- 172.21.15.107:0/1927742753 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590035030 con 0x7f959c100610 2024-04-25T00:19:47.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.427+0000 7f95a229a640 1 -- 172.21.15.107:0/1927742753 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f959c1a4b90 con 0x7f959c100610 2024-04-25T00:19:47.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.430+0000 7f95a4fa5740 1 Processor -- start 2024-04-25T00:19:47.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.430+0000 7f95a4fa5740 1 -- start start 2024-04-25T00:19:47.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.430+0000 7f95997fa640 1 -- 172.21.15.107:0/1927742753 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f9590042070 con 0x7f959c100610 2024-04-25T00:19:47.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.430+0000 7f95a4fa5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557aa8b21580 0x557aa8b21960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.430+0000 7f95a4fa5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557aa8b21ea0 0x557aa8b2c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.430+0000 7f95a4fa5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b2c570 0x557aa8b2e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.430+0000 7f95a4fa5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557aa8a86670 con 0x557aa8b2c570 2024-04-25T00:19:47.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.430+0000 7f95a4fa5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557aa8a2e970 con 0x557aa8b21580 2024-04-25T00:19:47.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.430+0000 7f95a4fa5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557aa8a82630 con 0x557aa8b21ea0 2024-04-25T00:19:47.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.430+0000 7f959b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b2c570 0x557aa8b2e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.430+0000 7f959b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b2c570 0x557aa8b2e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55410/0 (socket says 172.21.15.107:55410) 2024-04-25T00:19:47.438 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.430+0000 7f959b7fe640 1 -- 172.21.15.107:0/3830092990 learned_addr learned my addr 172.21.15.107:0/3830092990 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:47.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.430+0000 7f95a0837640 1 --2- 172.21.15.107:0/3830092990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557aa8b21580 0x557aa8b21960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.430+0000 7f959bfff640 1 --2- 172.21.15.107:0/3830092990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557aa8b21ea0 0x557aa8b2c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.430+0000 7f959b7fe640 1 -- 172.21.15.107:0/3830092990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557aa8b21ea0 msgr2=0x557aa8b2c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:47.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.430+0000 7f959b7fe640 1 --2- 172.21.15.107:0/3830092990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557aa8b21ea0 0x557aa8b2c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.430+0000 7f959b7fe640 1 -- 172.21.15.107:0/3830092990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557aa8b21580 msgr2=0x557aa8b21960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:47.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.430+0000 7f959b7fe640 1 --2- 172.21.15.107:0/3830092990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557aa8b21580 0x557aa8b21960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.430+0000 7f959b7fe640 1 -- 172.21.15.107:0/3830092990 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557aa8a52400 con 0x557aa8b2c570 2024-04-25T00:19:47.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.430+0000 7f95997fa640 1 --2- 172.21.15.107:0/1927742753 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f95840a2d00 0x7f95840a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.431+0000 7f959b7fe640 1 --2- 172.21.15.107:0/3830092990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b2c570 0x557aa8b2e960 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f958c00b7f0 tx=0x7f958c00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=454a83f6291254c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.431+0000 7f95a0837640 1 --2- 172.21.15.107:0/1927742753 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f95840a2d00 0x7f95840a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.431+0000 7f95997fa640 1 -- 172.21.15.107:0/1927742753 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f95900f0130 con 0x7f959c100610 2024-04-25T00:19:47.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.431+0000 7f95827fc640 1 -- 172.21.15.107:0/3830092990 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c00d950 con 0x557aa8b2c570 2024-04-25T00:19:47.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.431+0000 7f95827fc640 1 -- 172.21.15.107:0/3830092990 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f958c018070 con 0x557aa8b2c570 2024-04-25T00:19:47.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.431+0000 7f95827fc640 1 -- 172.21.15.107:0/3830092990 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c0123e0 con 0x557aa8b2c570 2024-04-25T00:19:47.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.431+0000 7f95a4fa5740 1 -- 172.21.15.107:0/3830092990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b2c570 msgr2=0x557aa8b2e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.431+0000 7f95a0837640 1 --2- 172.21.15.107:0/1927742753 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f95840a2d00 0x7f95840a51c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9594008e40 tx=0x7f95940085f0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.431+0000 7f95a4fa5740 1 --2- 172.21.15.107:0/3830092990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b2c570 0x557aa8b2e960 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f958c00b7f0 tx=0x7f958c00bcb0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.432+0000 7f95a4fa5740 1 -- 172.21.15.107:0/3830092990 shutdown_connections 2024-04-25T00:19:47.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.432+0000 7f95a4fa5740 1 --2- 172.21.15.107:0/3830092990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b2c570 0x557aa8b2e960 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.432+0000 7f95a4fa5740 1 --2- 172.21.15.107:0/3830092990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557aa8b21ea0 0x557aa8b2c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.432+0000 7f95a4fa5740 1 --2- 172.21.15.107:0/3830092990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557aa8b21580 0x557aa8b21960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.432+0000 7f95a4fa5740 1 -- 172.21.15.107:0/3830092990 >> 172.21.15.107:0/3830092990 conn(0x557aa8b1d8a0 msgr2=0x557aa8b1dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.432+0000 7f95a4fa5740 1 -- 172.21.15.107:0/3830092990 shutdown_connections 2024-04-25T00:19:47.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.432+0000 7f95a4fa5740 1 -- 172.21.15.107:0/3830092990 wait complete. 2024-04-25T00:19:47.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.432+0000 7f95a4fa5740 1 -- start start 2024-04-25T00:19:47.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.432+0000 7f95a4fa5740 1 Processor -- start 2024-04-25T00:19:47.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.432+0000 7f95a4fa5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557aa8b21580 0x557aa8b2be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.432+0000 7f95a4fa5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557aa8b21ea0 0x557aa8b29150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.432+0000 7f95a4fa5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b2c570 0x557aa8b29690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.432+0000 7f95a4fa5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557aa8b2f360 con 0x557aa8b2c570 2024-04-25T00:19:47.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.432+0000 7f95a4fa5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557aa8a42050 con 0x557aa8b21ea0 2024-04-25T00:19:47.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.432+0000 7f95a4fa5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557aa8b2f4e0 con 0x557aa8b21580 2024-04-25T00:19:47.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.432+0000 7f95a4fa5740 10 client.0 _ll_register_callbacks cb 0x557aa8799be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:47.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.432+0000 7f95a0837640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b2c570 0x557aa8b29690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.433+0000 7f95a0837640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b2c570 0x557aa8b29690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55422/0 (socket says 172.21.15.107:55422) 2024-04-25T00:19:47.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.433+0000 7f95a0837640 1 -- 172.21.15.107:0/3604818262 learned_addr learned my addr 172.21.15.107:0/3604818262 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:47.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.433+0000 7f959bfff640 1 --2- 172.21.15.107:0/3604818262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557aa8b21580 0x557aa8b2be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.433+0000 7f959b7fe640 1 --2- 172.21.15.107:0/3604818262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557aa8b21ea0 0x557aa8b29150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.433+0000 7f95a0837640 1 -- 172.21.15.107:0/3604818262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557aa8b21580 msgr2=0x557aa8b2be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:47.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.433+0000 7f95a0837640 1 --2- 172.21.15.107:0/3604818262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557aa8b21580 0x557aa8b2be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.433+0000 7f95a0837640 1 -- 172.21.15.107:0/3604818262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557aa8b21ea0 msgr2=0x557aa8b29150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:47.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.433+0000 7f95a0837640 1 --2- 172.21.15.107:0/3604818262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557aa8b21ea0 0x557aa8b29150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.433+0000 7f95a0837640 1 -- 172.21.15.107:0/3604818262 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557aa8a13dc0 con 0x557aa8b2c570 2024-04-25T00:19:47.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.433+0000 7f95a0837640 1 --2- 172.21.15.107:0/3604818262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b2c570 0x557aa8b29690 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f959400edb0 tx=0x7f9594004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8ac5d059593a73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.433+0000 7f957b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:19:47.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.433+0000 7f957b7fe640 1 -- 172.21.15.107:0/3604818262 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959401cd70 con 0x557aa8b2c570 2024-04-25T00:19:47.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.433+0000 7f957b7fe640 1 -- 172.21.15.107:0/3604818262 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f9594027430 con 0x557aa8b2c570 2024-04-25T00:19:47.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.433+0000 7f95a4fa5740 1 -- 172.21.15.107:0/3604818262 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f958c018070 con 0x557aa8b2c570 2024-04-25T00:19:47.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.433+0000 7f957b7fe640 1 -- 172.21.15.107:0/3604818262 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9594027630 con 0x557aa8b2c570 2024-04-25T00:19:47.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.433+0000 7f95a4fa5740 10 client.16326 fetch_fsmap learned FSMap version 28 2024-04-25T00:19:47.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.433+0000 7f95a4fa5740 1 -- 172.21.15.107:0/3604818262 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557aa879a3f0 con 0x557aa8b2c570 2024-04-25T00:19:47.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.433+0000 7f957b7fe640 1 -- 172.21.15.107:0/3604818262 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f958c018070 con 0x557aa8b2c570 2024-04-25T00:19:47.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.434+0000 7f957b7fe640 1 -- 172.21.15.107:0/3604818262 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f959401d9e0 con 0x557aa8b2c570 2024-04-25T00:19:47.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.434+0000 7f95a4fa5740 10 client.16326 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:19:47.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.434+0000 7f95a4fa5740 10 client.16326 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:19:47.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.434+0000 7f95a4fa5740 20 client.16326 populate_metadata read hostname 'smithi107' 2024-04-25T00:19:47.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.434+0000 7f95a4fa5740 1 --2- 172.21.15.107:0/3604818262 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x557aa8b3c610 0x557aa8b3ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.434+0000 7f95a4fa5740 4 client.16326 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:19:47.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.434+0000 7f95a4fa5740 1 -- 172.21.15.107:0/3604818262 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557aa8a93bb0 con 0x557aa8b3c610 2024-04-25T00:19:47.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.434+0000 7f959bfff640 1 --2- 172.21.15.107:0/3604818262 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x557aa8b3c610 0x557aa8b3ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.434+0000 7f957b7fe640 10 client.16326 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:19:47.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.434+0000 7f959bfff640 1 --2- 172.21.15.107:0/3604818262 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x557aa8b3c610 0x557aa8b3ea40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6ae0c083677ca663 server_cookie=6feaba7bf74916e3 in_seq=0 out_seq=0 2024-04-25T00:19:47.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.435+0000 7f957b7fe640 1 -- 172.21.15.107:0/3604818262 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557aa8a93bb0 con 0x557aa8b3c610 2024-04-25T00:19:47.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.435+0000 7f957b7fe640 10 client.16326 handle_command_reply: tid=0 2024-04-25T00:19:47.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.435+0000 7f95a4fa5740 1 client.16326 shutdown 2024-04-25T00:19:47.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.435+0000 7f95a4fa5740 1 -- 172.21.15.107:0/3604818262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b2c570 msgr2=0x557aa8b29690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.435+0000 7f95a4fa5740 1 --2- 172.21.15.107:0/3604818262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b2c570 0x557aa8b29690 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f959400edb0 tx=0x7f9594004a40 comp rx=0 tx=0).stop 2024-04-25T00:19:47.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.435+0000 7f95a4fa5740 1 -- 172.21.15.107:0/3604818262 shutdown_connections 2024-04-25T00:19:47.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.435+0000 7f95a4fa5740 1 --2- 172.21.15.107:0/3604818262 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x557aa8b3c610 0x557aa8b3ea40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.435+0000 7f95a4fa5740 1 --2- 172.21.15.107:0/3604818262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b2c570 0x557aa8b29690 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.435+0000 7f957b7fe640 0 client.16326 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:19:47.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.435+0000 7f95a4fa5740 1 --2- 172.21.15.107:0/3604818262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557aa8b21ea0 0x557aa8b29150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.435+0000 7f95a4fa5740 1 --2- 172.21.15.107:0/3604818262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557aa8b21580 0x557aa8b2be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.435+0000 7f95a4fa5740 1 -- 172.21.15.107:0/3604818262 >> 172.21.15.107:0/3604818262 conn(0x557aa8b1d8a0 msgr2=0x557aa8b1cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.435+0000 7f95a4fa5740 1 -- 172.21.15.107:0/3604818262 shutdown_connections 2024-04-25T00:19:47.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.435+0000 7f95a4fa5740 1 -- 172.21.15.107:0/3604818262 wait complete. 2024-04-25T00:19:47.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.435+0000 7f95a4fa5740 20 client.16326 trim_cache size 0 max 16384 2024-04-25T00:19:47.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.448+0000 7f95a4fa5740 1 Processor -- start 2024-04-25T00:19:47.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.448+0000 7f95a4fa5740 1 -- start start 2024-04-25T00:19:47.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.448+0000 7f95a4fa5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557aa8b1d8a0 0x557aa8b38220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.448+0000 7f95a4fa5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b21580 0x557aa8b44310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.448+0000 7f95a4fa5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557aa8b21ea0 0x557aa8b44850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.448+0000 7f95a4fa5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f959c1b1490 con 0x557aa8b21580 2024-04-25T00:19:47.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.448+0000 7f95a4fa5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f959c1aeca0 con 0x557aa8b21ea0 2024-04-25T00:19:47.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.448+0000 7f95a4fa5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f959c1acaf0 con 0x557aa8b1d8a0 2024-04-25T00:19:47.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.449+0000 7f959b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b21580 0x557aa8b44310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.449+0000 7f959b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b21580 0x557aa8b44310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55438/0 (socket says 172.21.15.107:55438) 2024-04-25T00:19:47.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.449+0000 7f959b7fe640 1 -- 172.21.15.107:0/51136847 learned_addr learned my addr 172.21.15.107:0/51136847 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:47.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.449+0000 7f959b7fe640 1 -- 172.21.15.107:0/51136847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557aa8b1d8a0 msgr2=0x557aa8b38220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:47.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.449+0000 7f959b7fe640 1 --2- 172.21.15.107:0/51136847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557aa8b1d8a0 0x557aa8b38220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.449+0000 7f959b7fe640 1 -- 172.21.15.107:0/51136847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557aa8b21ea0 msgr2=0x557aa8b44850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:47.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.449+0000 7f959b7fe640 1 --2- 172.21.15.107:0/51136847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557aa8b21ea0 0x557aa8b44850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.449+0000 7f959b7fe640 1 -- 172.21.15.107:0/51136847 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557aa88647d0 con 0x557aa8b21580 2024-04-25T00:19:47.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.449+0000 7f959b7fe640 1 --2- 172.21.15.107:0/51136847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b21580 0x557aa8b44310 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x557aa8b1ea30 tx=0x7f958c00b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551da2134482320b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.449+0000 7f9581ffb640 1 -- 172.21.15.107:0/51136847 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c013680 con 0x557aa8b21580 2024-04-25T00:19:47.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.449+0000 7f9581ffb640 1 -- 172.21.15.107:0/51136847 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f958c018040 con 0x557aa8b21580 2024-04-25T00:19:47.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.449+0000 7f9581ffb640 1 -- 172.21.15.107:0/51136847 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c026a20 con 0x557aa8b21580 2024-04-25T00:19:47.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.450+0000 7f95a4fa5740 1 -- 172.21.15.107:0/51136847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b21580 msgr2=0x557aa8b44310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.450+0000 7f95a4fa5740 1 --2- 172.21.15.107:0/51136847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b21580 0x557aa8b44310 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x557aa8b1ea30 tx=0x7f958c00b740 comp rx=0 tx=0).stop 2024-04-25T00:19:47.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.450+0000 7f95a4fa5740 1 -- 172.21.15.107:0/51136847 shutdown_connections 2024-04-25T00:19:47.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.450+0000 7f95a4fa5740 1 --2- 172.21.15.107:0/51136847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557aa8b21ea0 0x557aa8b44850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.450+0000 7f95a4fa5740 1 --2- 172.21.15.107:0/51136847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b21580 0x557aa8b44310 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.450+0000 7f95a4fa5740 1 --2- 172.21.15.107:0/51136847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557aa8b1d8a0 0x557aa8b38220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.450+0000 7f95a4fa5740 1 -- 172.21.15.107:0/51136847 >> 172.21.15.107:0/51136847 conn(0x557aa8b297f0 msgr2=0x557aa8b1cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.450+0000 7f95a4fa5740 1 -- 172.21.15.107:0/51136847 shutdown_connections 2024-04-25T00:19:47.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.450+0000 7f95a4fa5740 1 -- 172.21.15.107:0/51136847 wait complete. 2024-04-25T00:19:47.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.450+0000 7f95a4fa5740 1 -- start start 2024-04-25T00:19:47.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.450+0000 7f95a4fa5740 1 Processor -- start 2024-04-25T00:19:47.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.450+0000 7f95a4fa5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557aa8b1d8a0 0x557aa8b32960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.450+0000 7f95a4fa5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b21580 0x557aa8b32ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.450+0000 7f95a4fa5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557aa8b21ea0 0x557aa8ba1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.450+0000 7f95a4fa5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f959c1ad160 con 0x557aa8b21580 2024-04-25T00:19:47.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.450+0000 7f95a4fa5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f959c1ae5d0 con 0x557aa8b1d8a0 2024-04-25T00:19:47.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.450+0000 7f95a4fa5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f959c1ad830 con 0x557aa8b21ea0 2024-04-25T00:19:47.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.451+0000 7f95a4fa5740 10 client.0 _ll_register_callbacks cb 0x7f959400ea30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:47.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.451+0000 7f959b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b21580 0x557aa8b32ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.451+0000 7f959b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b21580 0x557aa8b32ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55454/0 (socket says 172.21.15.107:55454) 2024-04-25T00:19:47.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.451+0000 7f959b7fe640 1 -- 172.21.15.107:0/4242239086 learned_addr learned my addr 172.21.15.107:0/4242239086 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:47.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.451+0000 7f959bfff640 1 --2- 172.21.15.107:0/4242239086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557aa8b1d8a0 0x557aa8b32960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.451+0000 7f959b7fe640 1 -- 172.21.15.107:0/4242239086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557aa8b21ea0 msgr2=0x557aa8ba1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:47.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.451+0000 7f95a0837640 1 --2- 172.21.15.107:0/4242239086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557aa8b21ea0 0x557aa8ba1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.451+0000 7f959b7fe640 1 --2- 172.21.15.107:0/4242239086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557aa8b21ea0 0x557aa8ba1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.451+0000 7f959b7fe640 1 -- 172.21.15.107:0/4242239086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557aa8b1d8a0 msgr2=0x557aa8b32960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:47.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.451+0000 7f959b7fe640 1 --2- 172.21.15.107:0/4242239086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557aa8b1d8a0 0x557aa8b32960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.451+0000 7f959b7fe640 1 -- 172.21.15.107:0/4242239086 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557aa8ba21a0 con 0x557aa8b21580 2024-04-25T00:19:47.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.451+0000 7f959bfff640 1 --2- 172.21.15.107:0/4242239086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557aa8b1d8a0 0x557aa8b32960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:47.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.451+0000 7f959b7fe640 1 --2- 172.21.15.107:0/4242239086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b21580 0x557aa8b32ea0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f958c0266d0 tx=0x7f958c00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e73bf8728792b90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.451+0000 7f957b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:19:47.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.451+0000 7f957b7fe640 1 -- 172.21.15.107:0/4242239086 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f958c02c070 con 0x557aa8b21580 2024-04-25T00:19:47.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.451+0000 7f957b7fe640 1 -- 172.21.15.107:0/4242239086 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f958c00d950 con 0x557aa8b21580 2024-04-25T00:19:47.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.451+0000 7f95a4fa5740 1 -- 172.21.15.107:0/4242239086 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f958c018040 con 0x557aa8b21580 2024-04-25T00:19:47.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.451+0000 7f957b7fe640 1 -- 172.21.15.107:0/4242239086 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f958c00db90 con 0x557aa8b21580 2024-04-25T00:19:47.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.451+0000 7f95a4fa5740 10 client.16332 fetch_fsmap learned FSMap version 28 2024-04-25T00:19:47.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.451+0000 7f95a4fa5740 1 -- 172.21.15.107:0/4242239086 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557aa8ba34a0 con 0x557aa8b21580 2024-04-25T00:19:47.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.451+0000 7f957b7fe640 1 -- 172.21.15.107:0/4242239086 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f958c018040 con 0x557aa8b21580 2024-04-25T00:19:47.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.452+0000 7f957b7fe640 1 -- 172.21.15.107:0/4242239086 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f958c030550 con 0x557aa8b21580 2024-04-25T00:19:47.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.452+0000 7f95a4fa5740 10 client.16332 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:19:47.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.452+0000 7f95a4fa5740 10 client.16332 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:19:47.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.452+0000 7f95a4fa5740 20 client.16332 populate_metadata read hostname 'smithi107' 2024-04-25T00:19:47.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.452+0000 7f95a4fa5740 1 --2- 172.21.15.107:0/4242239086 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x557aa8b2c570 0x557aa8ba5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.452+0000 7f95a4fa5740 4 client.16332 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:19:47.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.452+0000 7f95a4fa5740 1 -- 172.21.15.107:0/4242239086 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557aa8a8df00 con 0x557aa8b2c570 2024-04-25T00:19:47.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.452+0000 7f959bfff640 1 --2- 172.21.15.107:0/4242239086 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x557aa8b2c570 0x557aa8ba5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.452+0000 7f959bfff640 1 --2- 172.21.15.107:0/4242239086 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x557aa8b2c570 0x557aa8ba5ab0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ca69279b51ad23ea server_cookie=e84337ca88fe806a in_seq=0 out_seq=0 2024-04-25T00:19:47.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.452+0000 7f957b7fe640 10 client.16332 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:19:47.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f957b7fe640 1 -- 172.21.15.107:0/4242239086 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557aa8a8df00 con 0x557aa8b2c570 2024-04-25T00:19:47.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f957b7fe640 10 client.16332 handle_command_reply: tid=0 2024-04-25T00:19:47.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f95a4fa5740 1 client.16332 shutdown 2024-04-25T00:19:47.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f95a4fa5740 1 -- 172.21.15.107:0/4242239086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b21580 msgr2=0x557aa8b32ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f95a4fa5740 1 --2- 172.21.15.107:0/4242239086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b21580 0x557aa8b32ea0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f958c0266d0 tx=0x7f958c00bcb0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f95a4fa5740 1 -- 172.21.15.107:0/4242239086 shutdown_connections 2024-04-25T00:19:47.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f95a4fa5740 1 --2- 172.21.15.107:0/4242239086 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x557aa8b2c570 0x557aa8ba5ab0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f95a4fa5740 1 --2- 172.21.15.107:0/4242239086 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557aa8b21ea0 0x557aa8ba1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f95a4fa5740 1 --2- 172.21.15.107:0/4242239086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557aa8b21580 0x557aa8b32ea0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f957b7fe640 0 client.16332 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:19:47.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f95a4fa5740 1 --2- 172.21.15.107:0/4242239086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557aa8b1d8a0 0x557aa8b32960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f95a4fa5740 1 -- 172.21.15.107:0/4242239086 >> 172.21.15.107:0/4242239086 conn(0x557aa8b297f0 msgr2=0x557aa8b37210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f95a4fa5740 1 -- 172.21.15.107:0/4242239086 shutdown_connections 2024-04-25T00:19:47.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f95a4fa5740 1 -- 172.21.15.107:0/4242239086 wait complete. 2024-04-25T00:19:47.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f95a4fa5740 20 client.16332 trim_cache size 0 max 16384 2024-04-25T00:19:47.460 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:19:47.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f95a229a640 1 -- 172.21.15.107:0/1927742753 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f95840a2d00 msgr2=0x7f95840a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f95a229a640 1 --2- 172.21.15.107:0/1927742753 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f95840a2d00 0x7f95840a51c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9594008e40 tx=0x7f95940085f0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f95a229a640 1 -- 172.21.15.107:0/1927742753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f959c100610 msgr2=0x7f959c198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f95a229a640 1 --2- 172.21.15.107:0/1927742753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f959c100610 0x7f959c198020 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f959002a940 tx=0x7f9590003c00 comp rx=0 tx=0).stop 2024-04-25T00:19:47.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f95a229a640 1 -- 172.21.15.107:0/1927742753 shutdown_connections 2024-04-25T00:19:47.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f95a229a640 1 --2- 172.21.15.107:0/1927742753 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f95840a2d00 0x7f95840a51c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f95a229a640 1 --2- 172.21.15.107:0/1927742753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f959c1066b0 0x7f959c19c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f95a229a640 1 --2- 172.21.15.107:0/1927742753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f959c100fd0 0x7f959c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f95a229a640 1 --2- 172.21.15.107:0/1927742753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f959c100610 0x7f959c198020 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.453+0000 7f95a229a640 1 -- 172.21.15.107:0/1927742753 >> 172.21.15.107:0/1927742753 conn(0x7f959c077740 msgr2=0x7f959c0ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.454+0000 7f95a229a640 1 -- 172.21.15.107:0/1927742753 shutdown_connections 2024-04-25T00:19:47.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.454+0000 7f95a229a640 1 -- 172.21.15.107:0/1927742753 wait complete. 2024-04-25T00:19:47.464 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:19:47.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.564+0000 7f8fd39dc640 1 -- 172.21.15.107:0/1326356297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fcc100540 msgr2=0x7f8fcc100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.564+0000 7f8fd39dc640 1 --2- 172.21.15.107:0/1326356297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fcc100540 0x7f8fcc100920 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009900 tx=0x7f8fbc02a410 comp rx=0 tx=0).stop 2024-04-25T00:19:47.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.564+0000 7f8fd39dc640 1 -- 172.21.15.107:0/1326356297 shutdown_connections 2024-04-25T00:19:47.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.564+0000 7f8fd39dc640 1 --2- 172.21.15.107:0/1326356297 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fcc10e0f0 0x7f8fcc1104e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.564+0000 7f8fd39dc640 1 --2- 172.21.15.107:0/1326356297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fcc100e60 0x7f8fcc10dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.564+0000 7f8fd39dc640 1 --2- 172.21.15.107:0/1326356297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fcc100540 0x7f8fcc100920 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.564+0000 7f8fd39dc640 1 -- 172.21.15.107:0/1326356297 >> 172.21.15.107:0/1326356297 conn(0x7f8fcc077740 msgr2=0x7f8fcc077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.564+0000 7f8fd39dc640 1 -- 172.21.15.107:0/1326356297 shutdown_connections 2024-04-25T00:19:47.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.564+0000 7f8fd39dc640 1 -- 172.21.15.107:0/1326356297 wait complete. 2024-04-25T00:19:47.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.564+0000 7f8fd39dc640 1 Processor -- start 2024-04-25T00:19:47.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.564+0000 7f8fd39dc640 1 -- start start 2024-04-25T00:19:47.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.564+0000 7f8fd39dc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fcc100540 0x7f8fcc19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.564+0000 7f8fd39dc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fcc100e60 0x7f8fcc19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.564+0000 7f8fd39dc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fcc10e0f0 0x7f8fcc19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.564+0000 7f8fd39dc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc113d20 con 0x7f8fcc100e60 2024-04-25T00:19:47.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.564+0000 7f8fd39dc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc113ba0 con 0x7f8fcc10e0f0 2024-04-25T00:19:47.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.564+0000 7f8fd39dc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc113ea0 con 0x7f8fcc100540 2024-04-25T00:19:47.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.565+0000 7f8fd0f77640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fcc100e60 0x7f8fcc19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.565+0000 7f8fd0f77640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fcc100e60 0x7f8fcc19a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55472/0 (socket says 172.21.15.107:55472) 2024-04-25T00:19:47.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.565+0000 7f8fd0f77640 1 -- 172.21.15.107:0/537649883 learned_addr learned my addr 172.21.15.107:0/537649883 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:47.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.565+0000 7f8fd1f79640 1 --2- 172.21.15.107:0/537649883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fcc10e0f0 0x7f8fcc19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.565+0000 7f8fd1778640 1 --2- 172.21.15.107:0/537649883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fcc100540 0x7f8fcc19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.565+0000 7f8fd0f77640 1 -- 172.21.15.107:0/537649883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fcc100540 msgr2=0x7f8fcc19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.565+0000 7f8fd0f77640 1 --2- 172.21.15.107:0/537649883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fcc100540 0x7f8fcc19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.565+0000 7f8fd0f77640 1 -- 172.21.15.107:0/537649883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fcc10e0f0 msgr2=0x7f8fcc19eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.565+0000 7f8fd0f77640 1 --2- 172.21.15.107:0/537649883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fcc10e0f0 0x7f8fcc19eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.565+0000 7f8fd0f77640 1 -- 172.21.15.107:0/537649883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fcc19f270 con 0x7f8fcc100e60 2024-04-25T00:19:47.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.565+0000 7f8fd1f79640 1 --2- 172.21.15.107:0/537649883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fcc10e0f0 0x7f8fcc19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:47.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.565+0000 7f8fd0f77640 1 --2- 172.21.15.107:0/537649883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fcc100e60 0x7f8fcc19a7c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800d950 tx=0x7f8fc800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.565+0000 7f8fc27fc640 1 -- 172.21.15.107:0/537649883 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8017070 con 0x7f8fcc100e60 2024-04-25T00:19:47.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.565+0000 7f8fc27fc640 1 -- 172.21.15.107:0/537649883 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc8009c30 con 0x7f8fcc100e60 2024-04-25T00:19:47.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.565+0000 7f8fd39dc640 1 -- 172.21.15.107:0/537649883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fcc19f560 con 0x7f8fcc100e60 2024-04-25T00:19:47.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.565+0000 7f8fc27fc640 1 -- 172.21.15.107:0/537649883 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc800f6b0 con 0x7f8fcc100e60 2024-04-25T00:19:47.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.565+0000 7f8fd39dc640 1 -- 172.21.15.107:0/537649883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fcc1a6df0 con 0x7f8fcc100e60 2024-04-25T00:19:47.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.567+0000 7f8fc27fc640 1 -- 172.21.15.107:0/537649883 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f8fc80109d0 con 0x7f8fcc100e60 2024-04-25T00:19:47.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.568+0000 7f8fd67c7740 1 Processor -- start 2024-04-25T00:19:47.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.568+0000 7f8fd67c7740 1 -- start start 2024-04-25T00:19:47.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.568+0000 7f8fd67c7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c56906a580 0x55c56906a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.568+0000 7f8fc27fc640 1 --2- 172.21.15.107:0/537649883 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8fb80a2d80 0x7f8fb80a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.568+0000 7f8fd67c7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c56906aea0 0x55c569075030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.568+0000 7f8fd67c7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c569075570 0x55c569077960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.568+0000 7f8fd67c7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c568fcf670 con 0x55c56906aea0 2024-04-25T00:19:47.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.568+0000 7f8fd67c7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c568f77970 con 0x55c569075570 2024-04-25T00:19:47.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.568+0000 7f8fd67c7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c568fcb630 con 0x55c56906a580 2024-04-25T00:19:47.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.568+0000 7f8fd0f77640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c56906aea0 0x55c569075030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.568+0000 7f8fd1778640 1 --2- 172.21.15.107:0/537649883 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8fb80a2d80 0x7f8fb80a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.568+0000 7f8fd0f77640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c56906aea0 0x55c569075030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55476/0 (socket says 172.21.15.107:55476) 2024-04-25T00:19:47.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.568+0000 7f8fd0f77640 1 -- 172.21.15.107:0/236219473 learned_addr learned my addr 172.21.15.107:0/236219473 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:47.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.568+0000 7f8fc27fc640 1 -- 172.21.15.107:0/537649883 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f8fc80cf400 con 0x7f8fcc100e60 2024-04-25T00:19:47.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.569+0000 7f8fd1f79640 1 --2- 172.21.15.107:0/236219473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c569075570 0x55c569077960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.569+0000 7f8fd1f79640 1 --2- 172.21.15.107:0/236219473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c56906a580 0x55c56906a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.569+0000 7f8fd0f77640 1 -- 172.21.15.107:0/236219473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c56906a580 msgr2=0x55c56906a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:47.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.569+0000 7f8fd0f77640 1 --2- 172.21.15.107:0/236219473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c56906a580 0x55c56906a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.569+0000 7f8fd0f77640 1 -- 172.21.15.107:0/236219473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c569075570 msgr2=0x55c569077960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:47.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.569+0000 7f8fd0f77640 1 --2- 172.21.15.107:0/236219473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c569075570 0x55c569077960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.569+0000 7f8fd0f77640 1 -- 172.21.15.107:0/236219473 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c568f9b400 con 0x55c56906aea0 2024-04-25T00:19:47.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.569+0000 7f8fd1778640 1 --2- 172.21.15.107:0/537649883 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8fb80a2d80 0x7f8fb80a5240 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc005ab0 tx=0x7f8fbc005a20 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.569+0000 7f8fd0f77640 1 --2- 172.21.15.107:0/236219473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c56906aea0 0x55c569075030 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8fc809b540 tx=0x7f8fc8090a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f97eb2f5f44b2e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.569+0000 7f8fb77fe640 1 -- 172.21.15.107:0/236219473 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8020070 con 0x55c56906aea0 2024-04-25T00:19:47.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.569+0000 7f8fb77fe640 1 -- 172.21.15.107:0/236219473 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc809bcb0 con 0x55c56906aea0 2024-04-25T00:19:47.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.569+0000 7f8fb77fe640 1 -- 172.21.15.107:0/236219473 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8009de0 con 0x55c56906aea0 2024-04-25T00:19:47.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.570+0000 7f8fd67c7740 1 -- 172.21.15.107:0/236219473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c56906aea0 msgr2=0x55c569075030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.570+0000 7f8fd67c7740 1 --2- 172.21.15.107:0/236219473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c56906aea0 0x55c569075030 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8fc809b540 tx=0x7f8fc8090a40 comp rx=0 tx=0).stop 2024-04-25T00:19:47.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.570+0000 7f8fd67c7740 1 -- 172.21.15.107:0/236219473 shutdown_connections 2024-04-25T00:19:47.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.570+0000 7f8fd67c7740 1 --2- 172.21.15.107:0/236219473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c569075570 0x55c569077960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.570+0000 7f8fd67c7740 1 --2- 172.21.15.107:0/236219473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c56906aea0 0x55c569075030 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.570+0000 7f8fd67c7740 1 --2- 172.21.15.107:0/236219473 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c56906a580 0x55c56906a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.570+0000 7f8fd67c7740 1 -- 172.21.15.107:0/236219473 >> 172.21.15.107:0/236219473 conn(0x55c5690668a0 msgr2=0x55c569066c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.570+0000 7f8fd67c7740 1 -- 172.21.15.107:0/236219473 shutdown_connections 2024-04-25T00:19:47.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.570+0000 7f8fd67c7740 1 -- 172.21.15.107:0/236219473 wait complete. 2024-04-25T00:19:47.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.570+0000 7f8fd67c7740 1 -- start start 2024-04-25T00:19:47.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.570+0000 7f8fd67c7740 1 Processor -- start 2024-04-25T00:19:47.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fd67c7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c56906a580 0x55c569074e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fd67c7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c56906aea0 0x55c569072110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fd67c7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c569075570 0x55c569072650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fd67c7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c5690782a0 con 0x55c569075570 2024-04-25T00:19:47.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fd67c7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c568f8b050 con 0x55c56906aea0 2024-04-25T00:19:47.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fd67c7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c569078420 con 0x55c56906a580 2024-04-25T00:19:47.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fd67c7740 10 client.0 _ll_register_callbacks cb 0x55c568ce2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:47.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fd1f79640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c569075570 0x55c569072650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fd1f79640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c569075570 0x55c569072650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55490/0 (socket says 172.21.15.107:55490) 2024-04-25T00:19:47.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fd1f79640 1 -- 172.21.15.107:0/439070334 learned_addr learned my addr 172.21.15.107:0/439070334 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:47.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fd1778640 1 --2- 172.21.15.107:0/439070334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c56906a580 0x55c569074e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fd0f77640 1 --2- 172.21.15.107:0/439070334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c56906aea0 0x55c569072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fd1f79640 1 -- 172.21.15.107:0/439070334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c56906a580 msgr2=0x55c569074e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:47.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fd1f79640 1 --2- 172.21.15.107:0/439070334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c56906a580 0x55c569074e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fd1f79640 1 -- 172.21.15.107:0/439070334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c56906aea0 msgr2=0x55c569072110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:47.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fd1f79640 1 --2- 172.21.15.107:0/439070334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c56906aea0 0x55c569072110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fd1f79640 1 -- 172.21.15.107:0/439070334 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c568f5cdc0 con 0x55c569075570 2024-04-25T00:19:47.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fd0f77640 1 --2- 172.21.15.107:0/439070334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c56906aea0 0x55c569072110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:47.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fd1f79640 1 --2- 172.21.15.107:0/439070334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c569075570 0x55c569072650 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4012b50 tx=0x7f8fc40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcc71002eed0273 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fb4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:19:47.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fb4ff9640 1 -- 172.21.15.107:0/439070334 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4012e50 con 0x55c569075570 2024-04-25T00:19:47.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fb4ff9640 1 -- 172.21.15.107:0/439070334 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f8fc4004b20 con 0x55c569075570 2024-04-25T00:19:47.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fd67c7740 1 -- 172.21.15.107:0/439070334 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fc809bcb0 con 0x55c569075570 2024-04-25T00:19:47.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fb4ff9640 1 -- 172.21.15.107:0/439070334 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8fc4004d20 con 0x55c569075570 2024-04-25T00:19:47.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fd67c7740 10 client.16344 fetch_fsmap learned FSMap version 28 2024-04-25T00:19:47.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.571+0000 7f8fd67c7740 1 -- 172.21.15.107:0/439070334 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c568ce33f0 con 0x55c569075570 2024-04-25T00:19:47.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.572+0000 7f8fb4ff9640 1 -- 172.21.15.107:0/439070334 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc809bcb0 con 0x55c569075570 2024-04-25T00:19:47.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.572+0000 7f8fb4ff9640 1 -- 172.21.15.107:0/439070334 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f8fc4010600 con 0x55c569075570 2024-04-25T00:19:47.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.572+0000 7f8fd67c7740 10 client.16344 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:19:47.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.572+0000 7f8fd67c7740 10 client.16344 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:19:47.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.572+0000 7f8fd67c7740 20 client.16344 populate_metadata read hostname 'smithi107' 2024-04-25T00:19:47.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.572+0000 7f8fd67c7740 1 --2- 172.21.15.107:0/439070334 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55c569085520 0x55c569087960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.572+0000 7f8fd67c7740 4 client.16344 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:19:47.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.572+0000 7f8fd67c7740 1 -- 172.21.15.107:0/439070334 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c568fdcbb0 con 0x55c569085520 2024-04-25T00:19:47.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.572+0000 7f8fd1778640 1 --2- 172.21.15.107:0/439070334 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55c569085520 0x55c569087960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.573+0000 7f8fb4ff9640 10 client.16344 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:19:47.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.573+0000 7f8fd1778640 1 --2- 172.21.15.107:0/439070334 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55c569085520 0x55c569087960 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8faa2f7a1d7f862c server_cookie=a89d16961ffa0381 in_seq=0 out_seq=0 2024-04-25T00:19:47.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.573+0000 7f8fb4ff9640 1 -- 172.21.15.107:0/439070334 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c568fdcbb0 con 0x55c569085520 2024-04-25T00:19:47.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.573+0000 7f8fb4ff9640 10 client.16344 handle_command_reply: tid=0 2024-04-25T00:19:47.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.573+0000 7f8fd67c7740 1 client.16344 shutdown 2024-04-25T00:19:47.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.573+0000 7f8fd67c7740 1 -- 172.21.15.107:0/439070334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c569075570 msgr2=0x55c569072650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.573+0000 7f8fd67c7740 1 --2- 172.21.15.107:0/439070334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c569075570 0x55c569072650 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4012b50 tx=0x7f8fc40117b0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.574+0000 7f8fd67c7740 1 -- 172.21.15.107:0/439070334 shutdown_connections 2024-04-25T00:19:47.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.574+0000 7f8fd67c7740 1 --2- 172.21.15.107:0/439070334 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55c569085520 0x55c569087960 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.574+0000 7f8fd67c7740 1 --2- 172.21.15.107:0/439070334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c569075570 0x55c569072650 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.574+0000 7f8fd67c7740 1 --2- 172.21.15.107:0/439070334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c56906aea0 0x55c569072110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.574+0000 7f8fd67c7740 1 --2- 172.21.15.107:0/439070334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c56906a580 0x55c569074e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.574+0000 7f8fd67c7740 1 -- 172.21.15.107:0/439070334 >> 172.21.15.107:0/439070334 conn(0x55c5690668a0 msgr2=0x55c569065b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.574+0000 7f8fb4ff9640 0 client.16344 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:19:47.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.574+0000 7f8fd67c7740 1 -- 172.21.15.107:0/439070334 shutdown_connections 2024-04-25T00:19:47.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.574+0000 7f8fd67c7740 1 -- 172.21.15.107:0/439070334 wait complete. 2024-04-25T00:19:47.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.574+0000 7f8fd67c7740 20 client.16344 trim_cache size 0 max 16384 2024-04-25T00:19:47.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.587+0000 7f8fd67c7740 1 Processor -- start 2024-04-25T00:19:47.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.587+0000 7f8fd67c7740 1 -- start start 2024-04-25T00:19:47.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.587+0000 7f8fd67c7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c56906a580 0x55c569081190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.587+0000 7f8fd67c7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c56906aea0 0x55c5690659c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.587+0000 7f8fd67c7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c569075570 0x55c56908d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.587+0000 7f8fd67c7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc1b3700 con 0x55c56906aea0 2024-04-25T00:19:47.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.587+0000 7f8fd67c7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc1b0f10 con 0x55c569075570 2024-04-25T00:19:47.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.587+0000 7f8fd67c7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc1aed60 con 0x55c56906a580 2024-04-25T00:19:47.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.587+0000 7f8fd0f77640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c56906aea0 0x55c5690659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.587+0000 7f8fd0f77640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c56906aea0 0x55c5690659c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55502/0 (socket says 172.21.15.107:55502) 2024-04-25T00:19:47.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.587+0000 7f8fd0f77640 1 -- 172.21.15.107:0/1357848831 learned_addr learned my addr 172.21.15.107:0/1357848831 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:47.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.587+0000 7f8fd0f77640 1 -- 172.21.15.107:0/1357848831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c56906a580 msgr2=0x55c569081190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.587+0000 7f8fd0f77640 1 --2- 172.21.15.107:0/1357848831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c56906a580 0x55c569081190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.587+0000 7f8fd0f77640 1 -- 172.21.15.107:0/1357848831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c569075570 msgr2=0x55c56908d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.587+0000 7f8fd0f77640 1 --2- 172.21.15.107:0/1357848831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c569075570 0x55c56908d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.587+0000 7f8fd0f77640 1 -- 172.21.15.107:0/1357848831 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c568dad7d0 con 0x55c56906aea0 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.587+0000 7f8fd0f77640 1 --2- 172.21.15.107:0/1357848831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c56906aea0 0x55c5690659c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8fc80a3cf0 tx=0x7f8fc8097980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f3fa1a7f85a48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.588+0000 7f8fb6ffd640 1 -- 172.21.15.107:0/1357848831 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8020040 con 0x55c56906aea0 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.588+0000 7f8fb6ffd640 1 -- 172.21.15.107:0/1357848831 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc8009de0 con 0x55c56906aea0 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.588+0000 7f8fb6ffd640 1 -- 172.21.15.107:0/1357848831 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc809f960 con 0x55c56906aea0 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.588+0000 7f8fd67c7740 1 -- 172.21.15.107:0/1357848831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c56906aea0 msgr2=0x55c5690659c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.588+0000 7f8fd67c7740 1 --2- 172.21.15.107:0/1357848831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c56906aea0 0x55c5690659c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8fc80a3cf0 tx=0x7f8fc8097980 comp rx=0 tx=0).stop 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.588+0000 7f8fd67c7740 1 -- 172.21.15.107:0/1357848831 shutdown_connections 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.588+0000 7f8fd67c7740 1 --2- 172.21.15.107:0/1357848831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c569075570 0x55c56908d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.588+0000 7f8fd67c7740 1 --2- 172.21.15.107:0/1357848831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c56906aea0 0x55c5690659c0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.588+0000 7f8fd67c7740 1 --2- 172.21.15.107:0/1357848831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c56906a580 0x55c569081190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.588+0000 7f8fd67c7740 1 -- 172.21.15.107:0/1357848831 >> 172.21.15.107:0/1357848831 conn(0x55c5690668a0 msgr2=0x55c5690652b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.588+0000 7f8fd67c7740 1 -- 172.21.15.107:0/1357848831 shutdown_connections 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.588+0000 7f8fd67c7740 1 -- 172.21.15.107:0/1357848831 wait complete. 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.588+0000 7f8fd67c7740 1 -- start start 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.589+0000 7f8fd67c7740 1 Processor -- start 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.589+0000 7f8fd67c7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c56906a580 0x55c56907d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.589+0000 7f8fd67c7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c56906aea0 0x55c56907aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.589+0000 7f8fd67c7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c569075570 0x55c56907afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.589+0000 7f8fd67c7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc1af3d0 con 0x55c56906a580 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.589+0000 7f8fd67c7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc1b0840 con 0x55c56906aea0 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.589+0000 7f8fd67c7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc1afaa0 con 0x55c569075570 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.589+0000 7f8fd67c7740 10 client.0 _ll_register_callbacks cb 0x7f8fc40095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:47.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.589+0000 7f8fd1778640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c56906a580 0x55c56907d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.589+0000 7f8fd1778640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c56906a580 0x55c56907d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55514/0 (socket says 172.21.15.107:55514) 2024-04-25T00:19:47.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.589+0000 7f8fd1778640 1 -- 172.21.15.107:0/2550225036 learned_addr learned my addr 172.21.15.107:0/2550225036 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:47.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.589+0000 7f8fd1f79640 1 --2- 172.21.15.107:0/2550225036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c569075570 0x55c56907afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.589+0000 7f8fd0f77640 1 --2- 172.21.15.107:0/2550225036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c56906aea0 0x55c56907aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.589+0000 7f8fd1778640 1 -- 172.21.15.107:0/2550225036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c569075570 msgr2=0x55c56907afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:47.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.589+0000 7f8fd1778640 1 --2- 172.21.15.107:0/2550225036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c569075570 0x55c56907afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.589+0000 7f8fd1778640 1 -- 172.21.15.107:0/2550225036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c56906aea0 msgr2=0x55c56907aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:47.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.589+0000 7f8fd1778640 1 --2- 172.21.15.107:0/2550225036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c56906aea0 0x55c56907aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.589+0000 7f8fd1778640 1 -- 172.21.15.107:0/2550225036 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c56907b520 con 0x55c56906a580 2024-04-25T00:19:47.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.589+0000 7f8fd1f79640 1 --2- 172.21.15.107:0/2550225036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c569075570 0x55c56907afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:47.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.590+0000 7f8fd1778640 1 --2- 172.21.15.107:0/2550225036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c56906a580 0x55c56907d710 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc02a930 tx=0x7f8fbc0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fcbe4f5071b3854 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.590+0000 7f8fb4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:19:47.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.590+0000 7f8fb4ff9640 1 -- 172.21.15.107:0/2550225036 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc056060 con 0x55c56906a580 2024-04-25T00:19:47.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.590+0000 7f8fb4ff9640 1 -- 172.21.15.107:0/2550225036 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f8fbc042040 con 0x55c56906a580 2024-04-25T00:19:47.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.590+0000 7f8fd67c7740 1 -- 172.21.15.107:0/2550225036 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fc8009de0 con 0x55c56906a580 2024-04-25T00:19:47.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.590+0000 7f8fb4ff9640 1 -- 172.21.15.107:0/2550225036 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8fbc05d050 con 0x55c56906a580 2024-04-25T00:19:47.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.590+0000 7f8fd67c7740 10 client.16350 fetch_fsmap learned FSMap version 28 2024-04-25T00:19:47.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.590+0000 7f8fd67c7740 1 -- 172.21.15.107:0/2550225036 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c5690eb7c0 con 0x55c56906a580 2024-04-25T00:19:47.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.590+0000 7f8fb4ff9640 1 -- 172.21.15.107:0/2550225036 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc8009de0 con 0x55c56906a580 2024-04-25T00:19:47.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.590+0000 7f8fb4ff9640 1 -- 172.21.15.107:0/2550225036 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f8fbc004750 con 0x55c56906a580 2024-04-25T00:19:47.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.590+0000 7f8fd67c7740 10 client.16350 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:19:47.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.590+0000 7f8fd67c7740 10 client.16350 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:19:47.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.590+0000 7f8fd67c7740 20 client.16350 populate_metadata read hostname 'smithi107' 2024-04-25T00:19:47.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.590+0000 7f8fd67c7740 1 --2- 172.21.15.107:0/2550225036 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55c5690ebf50 0x55c5690ee370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.590+0000 7f8fd67c7740 4 client.16350 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:19:47.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.590+0000 7f8fd67c7740 1 -- 172.21.15.107:0/2550225036 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c568fd6f00 con 0x55c5690ebf50 2024-04-25T00:19:47.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.590+0000 7f8fd0f77640 1 --2- 172.21.15.107:0/2550225036 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55c5690ebf50 0x55c5690ee370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.591+0000 7f8fd0f77640 1 --2- 172.21.15.107:0/2550225036 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55c5690ebf50 0x55c5690ee370 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ca431179495757ef server_cookie=dfd565bee6c39c4c in_seq=0 out_seq=0 2024-04-25T00:19:47.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.591+0000 7f8fb4ff9640 10 client.16350 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:19:47.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.591+0000 7f8fb4ff9640 1 -- 172.21.15.107:0/2550225036 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c568fd6f00 con 0x55c5690ebf50 2024-04-25T00:19:47.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.591+0000 7f8fb4ff9640 10 client.16350 handle_command_reply: tid=0 2024-04-25T00:19:47.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.591+0000 7f8fd67c7740 1 client.16350 shutdown 2024-04-25T00:19:47.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.591+0000 7f8fd67c7740 1 -- 172.21.15.107:0/2550225036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c56906a580 msgr2=0x55c56907d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.591+0000 7f8fd67c7740 1 --2- 172.21.15.107:0/2550225036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c56906a580 0x55c56907d710 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc02a930 tx=0x7f8fbc0058d0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.591+0000 7f8fd67c7740 1 -- 172.21.15.107:0/2550225036 shutdown_connections 2024-04-25T00:19:47.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.591+0000 7f8fd67c7740 1 --2- 172.21.15.107:0/2550225036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c569075570 0x55c56907afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.591+0000 7f8fd67c7740 1 --2- 172.21.15.107:0/2550225036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c56906aea0 0x55c56907aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.591+0000 7f8fd67c7740 1 --2- 172.21.15.107:0/2550225036 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55c5690ebf50 0x55c5690ee370 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.591+0000 7f8fd67c7740 1 --2- 172.21.15.107:0/2550225036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c56906a580 0x55c56907d710 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.591+0000 7f8fb4ff9640 0 client.16350 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:19:47.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.591+0000 7f8fd67c7740 1 -- 172.21.15.107:0/2550225036 >> 172.21.15.107:0/2550225036 conn(0x55c5690668a0 msgr2=0x55c569065b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.591+0000 7f8fd67c7740 1 -- 172.21.15.107:0/2550225036 shutdown_connections 2024-04-25T00:19:47.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.591+0000 7f8fd67c7740 1 -- 172.21.15.107:0/2550225036 wait complete. 2024-04-25T00:19:47.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.591+0000 7f8fd67c7740 20 client.16350 trim_cache size 0 max 16384 2024-04-25T00:19:47.600 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:19:47.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.592+0000 7f8fd39dc640 1 -- 172.21.15.107:0/537649883 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8fb80a2d80 msgr2=0x7f8fb80a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.592+0000 7f8fd39dc640 1 --2- 172.21.15.107:0/537649883 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8fb80a2d80 0x7f8fb80a5240 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc005ab0 tx=0x7f8fbc005a20 comp rx=0 tx=0).stop 2024-04-25T00:19:47.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.592+0000 7f8fd39dc640 1 -- 172.21.15.107:0/537649883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fcc100e60 msgr2=0x7f8fcc19a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.592+0000 7f8fd39dc640 1 --2- 172.21.15.107:0/537649883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fcc100e60 0x7f8fcc19a7c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8fc800d950 tx=0x7f8fc800de10 comp rx=0 tx=0).stop 2024-04-25T00:19:47.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.592+0000 7f8fd39dc640 1 -- 172.21.15.107:0/537649883 shutdown_connections 2024-04-25T00:19:47.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.592+0000 7f8fd39dc640 1 --2- 172.21.15.107:0/537649883 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8fb80a2d80 0x7f8fb80a5240 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.592+0000 7f8fd39dc640 1 --2- 172.21.15.107:0/537649883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8fcc10e0f0 0x7f8fcc19eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.592+0000 7f8fd39dc640 1 --2- 172.21.15.107:0/537649883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fcc100e60 0x7f8fcc19a7c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.592+0000 7f8fd39dc640 1 --2- 172.21.15.107:0/537649883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fcc100540 0x7f8fcc19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.592+0000 7f8fd39dc640 1 -- 172.21.15.107:0/537649883 >> 172.21.15.107:0/537649883 conn(0x7f8fcc077740 msgr2=0x7f8fcc10f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.592+0000 7f8fd39dc640 1 -- 172.21.15.107:0/537649883 shutdown_connections 2024-04-25T00:19:47.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.592+0000 7f8fd39dc640 1 -- 172.21.15.107:0/537649883 wait complete. 2024-04-25T00:19:47.603 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:19:47.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.702+0000 7f581ec83640 1 -- 172.21.15.107:0/3152411665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58181066b0 msgr2=0x7f5818106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.702+0000 7f581ec83640 1 --2- 172.21.15.107:0/3152411665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58181066b0 0x7f5818106a90 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f580c009930 tx=0x7f580c02a420 comp rx=0 tx=0).stop 2024-04-25T00:19:47.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.702+0000 7f581ec83640 1 -- 172.21.15.107:0/3152411665 shutdown_connections 2024-04-25T00:19:47.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.702+0000 7f581ec83640 1 --2- 172.21.15.107:0/3152411665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5818100fd0 0x7f581810d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.702+0000 7f581ec83640 1 --2- 172.21.15.107:0/3152411665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5818100610 0x7f5818100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.702+0000 7f581ec83640 1 --2- 172.21.15.107:0/3152411665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58181066b0 0x7f5818106a90 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.702+0000 7f581ec83640 1 -- 172.21.15.107:0/3152411665 >> 172.21.15.107:0/3152411665 conn(0x7f5818077740 msgr2=0x7f5818077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.702+0000 7f581ec83640 1 -- 172.21.15.107:0/3152411665 shutdown_connections 2024-04-25T00:19:47.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.702+0000 7f581ec83640 1 -- 172.21.15.107:0/3152411665 wait complete. 2024-04-25T00:19:47.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.703+0000 7f581ec83640 1 Processor -- start 2024-04-25T00:19:47.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.703+0000 7f581ec83640 1 -- start start 2024-04-25T00:19:47.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.703+0000 7f581ec83640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5818100610 0x7f5818198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.703+0000 7f581ec83640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5818100fd0 0x7f5818199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.703+0000 7f581ec83640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58181066b0 0x7f58181a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.703+0000 7f581ec83640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5818110a30 con 0x7f5818100fd0 2024-04-25T00:19:47.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.703+0000 7f581ec83640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f58181108b0 con 0x7f58181066b0 2024-04-25T00:19:47.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.703+0000 7f581ec83640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5818110bb0 con 0x7f5818100610 2024-04-25T00:19:47.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.703+0000 7f5817fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5818100fd0 0x7f5818199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.703+0000 7f5817fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5818100fd0 0x7f5818199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55534/0 (socket says 172.21.15.107:55534) 2024-04-25T00:19:47.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.703+0000 7f5817fff640 1 -- 172.21.15.107:0/622929644 learned_addr learned my addr 172.21.15.107:0/622929644 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:47.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.703+0000 7f581d220640 1 --2- 172.21.15.107:0/622929644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58181066b0 0x7f58181a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.703+0000 7f5817fff640 1 -- 172.21.15.107:0/622929644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5818100610 msgr2=0x7f5818198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.703+0000 7f581ca1f640 1 --2- 172.21.15.107:0/622929644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5818100610 0x7f5818198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.703+0000 7f5817fff640 1 --2- 172.21.15.107:0/622929644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5818100610 0x7f5818198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.703+0000 7f5817fff640 1 -- 172.21.15.107:0/622929644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58181066b0 msgr2=0x7f58181a3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.703+0000 7f5817fff640 1 --2- 172.21.15.107:0/622929644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58181066b0 0x7f58181a3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.703+0000 7f5817fff640 1 -- 172.21.15.107:0/622929644 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58181a4390 con 0x7f5818100fd0 2024-04-25T00:19:47.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.703+0000 7f581d220640 1 --2- 172.21.15.107:0/622929644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58181066b0 0x7f58181a3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:47.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.703+0000 7f581ca1f640 1 --2- 172.21.15.107:0/622929644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5818100610 0x7f5818198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:47.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.703+0000 7f5817fff640 1 --2- 172.21.15.107:0/622929644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5818100fd0 0x7f5818199310 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f580400e9f0 tx=0x7f580400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.704+0000 7f5815ffb640 1 -- 172.21.15.107:0/622929644 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580400cd80 con 0x7f5818100fd0 2024-04-25T00:19:47.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.704+0000 7f5815ffb640 1 -- 172.21.15.107:0/622929644 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5804010910 con 0x7f5818100fd0 2024-04-25T00:19:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.704+0000 7f581ec83640 1 -- 172.21.15.107:0/622929644 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58181a4560 con 0x7f5818100fd0 2024-04-25T00:19:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.704+0000 7f5815ffb640 1 -- 172.21.15.107:0/622929644 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5804018990 con 0x7f5818100fd0 2024-04-25T00:19:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.704+0000 7f581ec83640 1 -- 172.21.15.107:0/622929644 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58181a4a80 con 0x7f5818100fd0 2024-04-25T00:19:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.706+0000 7f5815ffb640 1 -- 172.21.15.107:0/622929644 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f5804010ac0 con 0x7f5818100fd0 2024-04-25T00:19:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.706+0000 7f582198b740 1 Processor -- start 2024-04-25T00:19:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.706+0000 7f582198b740 1 -- start start 2024-04-25T00:19:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.707+0000 7f582198b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d76120e580 0x55d76120e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.707+0000 7f582198b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d76120eea0 0x55d761219030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.707+0000 7f582198b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d761219570 0x55d76121b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.707+0000 7f582198b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d761173670 con 0x55d76120e580 2024-04-25T00:19:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.707+0000 7f582198b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d76111b970 con 0x55d761219570 2024-04-25T00:19:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.707+0000 7f582198b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d76116f630 con 0x55d76120eea0 2024-04-25T00:19:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.707+0000 7f5815ffb640 1 --2- 172.21.15.107:0/622929644 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f58080a2d00 0x7f58080a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.707+0000 7f581d220640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d76120e580 0x55d76120e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.707+0000 7f581d220640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d76120e580 0x55d76120e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55546/0 (socket says 172.21.15.107:55546) 2024-04-25T00:19:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.707+0000 7f581d220640 1 -- 172.21.15.107:0/3577909494 learned_addr learned my addr 172.21.15.107:0/3577909494 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.707+0000 7f581ca1f640 1 --2- 172.21.15.107:0/3577909494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d76120eea0 0x55d761219030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.707+0000 7f581d220640 1 --2- 172.21.15.107:0/622929644 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f58080a2d00 0x7f58080a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.707+0000 7f5815ffb640 1 -- 172.21.15.107:0/622929644 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f58040cfc10 con 0x7f5818100fd0 2024-04-25T00:19:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.707+0000 7f5817fff640 1 --2- 172.21.15.107:0/3577909494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d761219570 0x55d76121b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.707+0000 7f581d220640 1 -- 172.21.15.107:0/3577909494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d76120eea0 msgr2=0x55d761219030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.707+0000 7f581d220640 1 --2- 172.21.15.107:0/3577909494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d76120eea0 0x55d761219030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.707+0000 7f581d220640 1 -- 172.21.15.107:0/3577909494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d761219570 msgr2=0x55d76121b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.707+0000 7f581d220640 1 --2- 172.21.15.107:0/3577909494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d761219570 0x55d76121b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.707+0000 7f581d220640 1 -- 172.21.15.107:0/3577909494 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d76113f400 con 0x55d76120e580 2024-04-25T00:19:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.707+0000 7f581d220640 1 --2- 172.21.15.107:0/3577909494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d76120e580 0x55d76120e960 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f58100092f0 tx=0x7f5810011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff12812933756a46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.707+0000 7f581d220640 1 --2- 172.21.15.107:0/622929644 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f58080a2d00 0x7f58080a51c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5810011c50 tx=0x7f5810016000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.707+0000 7f5802ffd640 1 -- 172.21.15.107:0/3577909494 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810022070 con 0x55d76120e580 2024-04-25T00:19:47.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.707+0000 7f5802ffd640 1 -- 172.21.15.107:0/3577909494 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5810009bc0 con 0x55d76120e580 2024-04-25T00:19:47.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.708+0000 7f5802ffd640 1 -- 172.21.15.107:0/3577909494 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810005420 con 0x55d76120e580 2024-04-25T00:19:47.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.708+0000 7f582198b740 1 -- 172.21.15.107:0/3577909494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d76120e580 msgr2=0x55d76120e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.708+0000 7f582198b740 1 --2- 172.21.15.107:0/3577909494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d76120e580 0x55d76120e960 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f58100092f0 tx=0x7f5810011750 comp rx=0 tx=0).stop 2024-04-25T00:19:47.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.708+0000 7f582198b740 1 -- 172.21.15.107:0/3577909494 shutdown_connections 2024-04-25T00:19:47.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.708+0000 7f582198b740 1 --2- 172.21.15.107:0/3577909494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d761219570 0x55d76121b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.708+0000 7f582198b740 1 --2- 172.21.15.107:0/3577909494 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d76120eea0 0x55d761219030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.708+0000 7f582198b740 1 --2- 172.21.15.107:0/3577909494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d76120e580 0x55d76120e960 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.708+0000 7f582198b740 1 -- 172.21.15.107:0/3577909494 >> 172.21.15.107:0/3577909494 conn(0x55d76120a8a0 msgr2=0x55d76120ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.708+0000 7f582198b740 1 -- 172.21.15.107:0/3577909494 shutdown_connections 2024-04-25T00:19:47.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.708+0000 7f582198b740 1 -- 172.21.15.107:0/3577909494 wait complete. 2024-04-25T00:19:47.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.708+0000 7f582198b740 1 -- start start 2024-04-25T00:19:47.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.709+0000 7f582198b740 1 Processor -- start 2024-04-25T00:19:47.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.709+0000 7f582198b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d76120e580 0x55d761218f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.709+0000 7f582198b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d76120eea0 0x55d761216250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.709+0000 7f582198b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d761219570 0x55d761216860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.709+0000 7f582198b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d76121c300 con 0x55d761219570 2024-04-25T00:19:47.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.709+0000 7f582198b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d76112f050 con 0x55d76120e580 2024-04-25T00:19:47.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.709+0000 7f582198b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d76121c480 con 0x55d76120eea0 2024-04-25T00:19:47.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.709+0000 7f582198b740 10 client.0 _ll_register_callbacks cb 0x55d760e86be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:47.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.709+0000 7f581d220640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d761219570 0x55d761216860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.709+0000 7f581d220640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d761219570 0x55d761216860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55552/0 (socket says 172.21.15.107:55552) 2024-04-25T00:19:47.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.709+0000 7f581d220640 1 -- 172.21.15.107:0/3646606125 learned_addr learned my addr 172.21.15.107:0/3646606125 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:47.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.709+0000 7f581ca1f640 1 --2- 172.21.15.107:0/3646606125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d76120e580 0x55d761218f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.709+0000 7f5817fff640 1 --2- 172.21.15.107:0/3646606125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d76120eea0 0x55d761216250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.709+0000 7f581d220640 1 -- 172.21.15.107:0/3646606125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d76120eea0 msgr2=0x55d761216250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:47.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.709+0000 7f581d220640 1 --2- 172.21.15.107:0/3646606125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d76120eea0 0x55d761216250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.709+0000 7f581d220640 1 -- 172.21.15.107:0/3646606125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d76120e580 msgr2=0x55d761218f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:47.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.709+0000 7f581d220640 1 --2- 172.21.15.107:0/3646606125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d76120e580 0x55d761218f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.709+0000 7f581d220640 1 -- 172.21.15.107:0/3646606125 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d761100dc0 con 0x55d761219570 2024-04-25T00:19:47.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.709+0000 7f581ca1f640 1 --2- 172.21.15.107:0/3646606125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d76120e580 0x55d761218f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:47.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.709+0000 7f581d220640 1 --2- 172.21.15.107:0/3646606125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d761219570 0x55d761216860 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f581000cf90 tx=0x7f5810011fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11aad6d78783f12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.709+0000 7f57f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:19:47.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.710+0000 7f57f7fff640 1 -- 172.21.15.107:0/3646606125 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810033060 con 0x55d761219570 2024-04-25T00:19:47.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.710+0000 7f57f7fff640 1 -- 172.21.15.107:0/3646606125 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f58100049c0 con 0x55d761219570 2024-04-25T00:19:47.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.710+0000 7f582198b740 1 -- 172.21.15.107:0/3646606125 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5810009bc0 con 0x55d761219570 2024-04-25T00:19:47.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.710+0000 7f57f7fff640 1 -- 172.21.15.107:0/3646606125 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f581003a050 con 0x55d761219570 2024-04-25T00:19:47.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.710+0000 7f582198b740 10 client.16362 fetch_fsmap learned FSMap version 28 2024-04-25T00:19:47.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.710+0000 7f582198b740 1 -- 172.21.15.107:0/3646606125 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d760e873f0 con 0x55d761219570 2024-04-25T00:19:47.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.710+0000 7f57f7fff640 1 -- 172.21.15.107:0/3646606125 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5810009bc0 con 0x55d761219570 2024-04-25T00:19:47.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.710+0000 7f57f7fff640 1 -- 172.21.15.107:0/3646606125 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f581002e430 con 0x55d761219570 2024-04-25T00:19:47.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.710+0000 7f582198b740 10 client.16362 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:19:47.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.710+0000 7f582198b740 10 client.16362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:47.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.710+0000 7f582198b740 20 client.16362 populate_metadata read hostname 'smithi107' 2024-04-25T00:19:47.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.710+0000 7f582198b740 1 --2- 172.21.15.107:0/3646606125 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d7612295a0 0x55d76122b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.710+0000 7f582198b740 4 client.16362 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:19:47.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.710+0000 7f582198b740 1 -- 172.21.15.107:0/3646606125 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d761180bb0 con 0x55d7612295a0 2024-04-25T00:19:47.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.711+0000 7f581ca1f640 1 --2- 172.21.15.107:0/3646606125 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d7612295a0 0x55d76122b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.711+0000 7f57f7fff640 10 client.16362 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:47.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.711+0000 7f581ca1f640 1 --2- 172.21.15.107:0/3646606125 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d7612295a0 0x55d76122b990 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fb294ac4bb40400 server_cookie=fbac11f3ccf0e0d2 in_seq=0 out_seq=0 2024-04-25T00:19:47.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.712+0000 7f57f7fff640 1 -- 172.21.15.107:0/3646606125 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d761180bb0 con 0x55d7612295a0 2024-04-25T00:19:47.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.712+0000 7f57f7fff640 10 client.16362 handle_command_reply: tid=0 2024-04-25T00:19:47.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.712+0000 7f582198b740 1 client.16362 shutdown 2024-04-25T00:19:47.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.712+0000 7f582198b740 1 -- 172.21.15.107:0/3646606125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d761219570 msgr2=0x55d761216860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.712+0000 7f582198b740 1 --2- 172.21.15.107:0/3646606125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d761219570 0x55d761216860 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f581000cf90 tx=0x7f5810011fb0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.712+0000 7f582198b740 1 -- 172.21.15.107:0/3646606125 shutdown_connections 2024-04-25T00:19:47.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.712+0000 7f582198b740 1 --2- 172.21.15.107:0/3646606125 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d7612295a0 0x55d76122b990 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.712+0000 7f582198b740 1 --2- 172.21.15.107:0/3646606125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d761219570 0x55d761216860 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.712+0000 7f582198b740 1 --2- 172.21.15.107:0/3646606125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d76120eea0 0x55d761216250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.712+0000 7f57f7fff640 0 client.16362 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:47.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.712+0000 7f582198b740 1 --2- 172.21.15.107:0/3646606125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d76120e580 0x55d761218f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.712+0000 7f582198b740 1 -- 172.21.15.107:0/3646606125 >> 172.21.15.107:0/3646606125 conn(0x55d76120a8a0 msgr2=0x55d7612106a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.712+0000 7f582198b740 1 -- 172.21.15.107:0/3646606125 shutdown_connections 2024-04-25T00:19:47.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.712+0000 7f582198b740 1 -- 172.21.15.107:0/3646606125 wait complete. 2024-04-25T00:19:47.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.712+0000 7f582198b740 20 client.16362 trim_cache size 0 max 16384 2024-04-25T00:19:47.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.725+0000 7f582198b740 1 Processor -- start 2024-04-25T00:19:47.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.725+0000 7f582198b740 1 -- start start 2024-04-25T00:19:47.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.725+0000 7f582198b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d76120e580 0x55d7612251c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.725+0000 7f582198b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d76120eea0 0x55d7612099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.725+0000 7f582198b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d761219570 0x55d761218a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.725+0000 7f582198b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f58181b1510 con 0x55d76120e580 2024-04-25T00:19:47.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.725+0000 7f582198b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f58181aed20 con 0x55d761219570 2024-04-25T00:19:47.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.725+0000 7f582198b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f58181acb70 con 0x55d76120eea0 2024-04-25T00:19:47.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.725+0000 7f581ca1f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d76120e580 0x55d7612251c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.725+0000 7f581ca1f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d76120e580 0x55d7612251c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55564/0 (socket says 172.21.15.107:55564) 2024-04-25T00:19:47.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.725+0000 7f581ca1f640 1 -- 172.21.15.107:0/4100236607 learned_addr learned my addr 172.21.15.107:0/4100236607 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:47.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.725+0000 7f581d220640 1 --2- 172.21.15.107:0/4100236607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d761219570 0x55d761218a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.725+0000 7f581ca1f640 1 -- 172.21.15.107:0/4100236607 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d76120eea0 msgr2=0x55d7612099c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:47.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.725+0000 7f581ca1f640 1 --2- 172.21.15.107:0/4100236607 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d76120eea0 0x55d7612099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.725+0000 7f581ca1f640 1 -- 172.21.15.107:0/4100236607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d761219570 msgr2=0x55d761218a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:47.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.725+0000 7f581ca1f640 1 --2- 172.21.15.107:0/4100236607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d761219570 0x55d761218a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.725+0000 7f581ca1f640 1 -- 172.21.15.107:0/4100236607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d760f517d0 con 0x55d76120e580 2024-04-25T00:19:47.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.725+0000 7f581d220640 1 --2- 172.21.15.107:0/4100236607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d761219570 0x55d761218a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:47.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.726+0000 7f581ca1f640 1 --2- 172.21.15.107:0/4100236607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d76120e580 0x55d7612251c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5818101750 tx=0x7f580c032f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da236049a88323a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.726+0000 7f58027fc640 1 -- 172.21.15.107:0/4100236607 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c04a070 con 0x55d76120e580 2024-04-25T00:19:47.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.726+0000 7f58027fc640 1 -- 172.21.15.107:0/4100236607 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f580c005040 con 0x55d76120e580 2024-04-25T00:19:47.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.726+0000 7f58027fc640 1 -- 172.21.15.107:0/4100236607 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c035040 con 0x55d76120e580 2024-04-25T00:19:47.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.726+0000 7f582198b740 1 -- 172.21.15.107:0/4100236607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d76120e580 msgr2=0x55d7612251c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.726+0000 7f582198b740 1 --2- 172.21.15.107:0/4100236607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d76120e580 0x55d7612251c0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5818101750 tx=0x7f580c032f70 comp rx=0 tx=0).stop 2024-04-25T00:19:47.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.726+0000 7f582198b740 1 -- 172.21.15.107:0/4100236607 shutdown_connections 2024-04-25T00:19:47.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.726+0000 7f582198b740 1 --2- 172.21.15.107:0/4100236607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d761219570 0x55d761218a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.726+0000 7f582198b740 1 --2- 172.21.15.107:0/4100236607 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d76120eea0 0x55d7612099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.726+0000 7f582198b740 1 --2- 172.21.15.107:0/4100236607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d76120e580 0x55d7612251c0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.726+0000 7f582198b740 1 -- 172.21.15.107:0/4100236607 >> 172.21.15.107:0/4100236607 conn(0x55d76120a8a0 msgr2=0x55d7612092b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.726+0000 7f582198b740 1 -- 172.21.15.107:0/4100236607 shutdown_connections 2024-04-25T00:19:47.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.726+0000 7f582198b740 1 -- 172.21.15.107:0/4100236607 wait complete. 2024-04-25T00:19:47.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.727+0000 7f582198b740 1 -- start start 2024-04-25T00:19:47.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.727+0000 7f582198b740 1 Processor -- start 2024-04-25T00:19:47.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.727+0000 7f582198b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d76120e580 0x55d761212f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.727+0000 7f582198b740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d76120eea0 0x55d7612134a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.727+0000 7f582198b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d761219570 0x55d7612139e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.727+0000 7f582198b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f58181ad1e0 con 0x55d761219570 2024-04-25T00:19:47.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.727+0000 7f582198b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f58181ae650 con 0x55d76120e580 2024-04-25T00:19:47.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.727+0000 7f582198b740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f58181ad8b0 con 0x55d76120eea0 2024-04-25T00:19:47.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.727+0000 7f582198b740 10 client.0 _ll_register_callbacks cb 0x7f5804005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:47.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.727+0000 7f581d220640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d761219570 0x55d7612139e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.727+0000 7f581d220640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d761219570 0x55d7612139e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55574/0 (socket says 172.21.15.107:55574) 2024-04-25T00:19:47.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.727+0000 7f581d220640 1 -- 172.21.15.107:0/3190481120 learned_addr learned my addr 172.21.15.107:0/3190481120 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:47.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.727+0000 7f5817fff640 1 --2- 172.21.15.107:0/3190481120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d76120eea0 0x55d7612134a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.727+0000 7f581ca1f640 1 --2- 172.21.15.107:0/3190481120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d76120e580 0x55d761212f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.727+0000 7f581d220640 1 -- 172.21.15.107:0/3190481120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d76120eea0 msgr2=0x55d7612134a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:47.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.727+0000 7f581d220640 1 --2- 172.21.15.107:0/3190481120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d76120eea0 0x55d7612134a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.727+0000 7f581d220640 1 -- 172.21.15.107:0/3190481120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d76120e580 msgr2=0x55d761212f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:47.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.727+0000 7f581d220640 1 --2- 172.21.15.107:0/3190481120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d76120e580 0x55d761212f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.727+0000 7f581d220640 1 -- 172.21.15.107:0/3190481120 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d761292dd0 con 0x55d761219570 2024-04-25T00:19:47.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.727+0000 7f581ca1f640 1 --2- 172.21.15.107:0/3190481120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d76120e580 0x55d761212f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:47.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.728+0000 7f581d220640 1 --2- 172.21.15.107:0/3190481120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d761219570 0x55d7612139e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5810009240 tx=0x7f5810009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ec25008136477 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:47.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.728+0000 7f57f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:19:47.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.728+0000 7f57f7fff640 1 -- 172.21.15.107:0/3190481120 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5810033070 con 0x55d761219570 2024-04-25T00:19:47.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.728+0000 7f57f7fff640 1 -- 172.21.15.107:0/3190481120 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f581002e9d0 con 0x55d761219570 2024-04-25T00:19:47.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.728+0000 7f582198b740 1 -- 172.21.15.107:0/3190481120 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f580c005040 con 0x55d761219570 2024-04-25T00:19:47.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.728+0000 7f57f7fff640 1 -- 172.21.15.107:0/3190481120 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5810022040 con 0x55d761219570 2024-04-25T00:19:47.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.728+0000 7f582198b740 10 client.16368 fetch_fsmap learned FSMap version 28 2024-04-25T00:19:47.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.728+0000 7f582198b740 1 -- 172.21.15.107:0/3190481120 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d761294020 con 0x55d761219570 2024-04-25T00:19:47.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.728+0000 7f57f7fff640 1 -- 172.21.15.107:0/3190481120 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f580c005040 con 0x55d761219570 2024-04-25T00:19:47.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.728+0000 7f57f7fff640 1 -- 172.21.15.107:0/3190481120 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f5810005030 con 0x55d761219570 2024-04-25T00:19:47.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.728+0000 7f582198b740 10 client.16368 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:19:47.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.728+0000 7f582198b740 10 client.16368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:47.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.728+0000 7f582198b740 20 client.16368 populate_metadata read hostname 'smithi107' 2024-04-25T00:19:47.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.728+0000 7f582198b740 1 --2- 172.21.15.107:0/3190481120 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d761294880 0x55d761296ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:47.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.728+0000 7f582198b740 4 client.16368 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:19:47.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.728+0000 7f582198b740 1 -- 172.21.15.107:0/3190481120 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d76117af00 con 0x55d761294880 2024-04-25T00:19:47.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.728+0000 7f581ca1f640 1 --2- 172.21.15.107:0/3190481120 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d761294880 0x55d761296ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:47.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.729+0000 7f57f7fff640 10 client.16368 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:47.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.729+0000 7f581ca1f640 1 --2- 172.21.15.107:0/3190481120 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d761294880 0x55d761296ca0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9fe445200791563 server_cookie=d8738fceb401ed5a in_seq=0 out_seq=0 2024-04-25T00:19:47.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.729+0000 7f57f7fff640 1 -- 172.21.15.107:0/3190481120 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d76117af00 con 0x55d761294880 2024-04-25T00:19:47.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.729+0000 7f57f7fff640 10 client.16368 handle_command_reply: tid=0 2024-04-25T00:19:47.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.729+0000 7f582198b740 1 client.16368 shutdown 2024-04-25T00:19:47.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.729+0000 7f582198b740 1 -- 172.21.15.107:0/3190481120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d761219570 msgr2=0x55d7612139e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.729+0000 7f582198b740 1 --2- 172.21.15.107:0/3190481120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d761219570 0x55d7612139e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5810009240 tx=0x7f5810009720 comp rx=0 tx=0).stop 2024-04-25T00:19:47.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.729+0000 7f582198b740 1 -- 172.21.15.107:0/3190481120 shutdown_connections 2024-04-25T00:19:47.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.729+0000 7f582198b740 1 --2- 172.21.15.107:0/3190481120 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55d761294880 0x55d761296ca0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.729+0000 7f582198b740 1 --2- 172.21.15.107:0/3190481120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d761219570 0x55d7612139e0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.729+0000 7f582198b740 1 --2- 172.21.15.107:0/3190481120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d76120eea0 0x55d7612134a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.729+0000 7f57f7fff640 0 client.16368 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:47.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.729+0000 7f582198b740 1 --2- 172.21.15.107:0/3190481120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d76120e580 0x55d761212f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.729+0000 7f582198b740 1 -- 172.21.15.107:0/3190481120 >> 172.21.15.107:0/3190481120 conn(0x55d76120a8a0 msgr2=0x55d761209b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.729+0000 7f582198b740 1 -- 172.21.15.107:0/3190481120 shutdown_connections 2024-04-25T00:19:47.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.729+0000 7f582198b740 1 -- 172.21.15.107:0/3190481120 wait complete. 2024-04-25T00:19:47.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.729+0000 7f582198b740 20 client.16368 trim_cache size 0 max 16384 2024-04-25T00:19:47.740 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:19:47.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.730+0000 7f581ec83640 1 -- 172.21.15.107:0/622929644 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f58080a2d00 msgr2=0x7f58080a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.730+0000 7f581ec83640 1 --2- 172.21.15.107:0/622929644 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f58080a2d00 0x7f58080a51c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5810011c50 tx=0x7f5810016000 comp rx=0 tx=0).stop 2024-04-25T00:19:47.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.730+0000 7f581ec83640 1 -- 172.21.15.107:0/622929644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5818100fd0 msgr2=0x7f5818199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:47.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.730+0000 7f581ec83640 1 --2- 172.21.15.107:0/622929644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5818100fd0 0x7f5818199310 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f580400e9f0 tx=0x7f580400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.730+0000 7f581ec83640 1 -- 172.21.15.107:0/622929644 shutdown_connections 2024-04-25T00:19:47.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.730+0000 7f581ec83640 1 --2- 172.21.15.107:0/622929644 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f58080a2d00 0x7f58080a51c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.730+0000 7f581ec83640 1 --2- 172.21.15.107:0/622929644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58181066b0 0x7f58181a3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.730+0000 7f581ec83640 1 --2- 172.21.15.107:0/622929644 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5818100fd0 0x7f5818199310 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.730+0000 7f581ec83640 1 --2- 172.21.15.107:0/622929644 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5818100610 0x7f5818198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:47.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.730+0000 7f581ec83640 1 -- 172.21.15.107:0/622929644 >> 172.21.15.107:0/622929644 conn(0x7f5818077740 msgr2=0x7f58180ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:47.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.730+0000 7f581ec83640 1 -- 172.21.15.107:0/622929644 shutdown_connections 2024-04-25T00:19:47.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:47.730+0000 7f581ec83640 1 -- 172.21.15.107:0/622929644 wait complete. 2024-04-25T00:19:48.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:48 smithi195 ceph-mon[32833]: pgmap v103: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.0 MiB/s wr, 188 op/s 2024-04-25T00:19:48.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:48 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/172747286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:48.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:48 smithi170 ceph-mon[32807]: pgmap v103: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.0 MiB/s wr, 188 op/s 2024-04-25T00:19:48.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:48 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/172747286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:48.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:48 smithi107 ceph-mon[29153]: pgmap v103: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.0 MiB/s wr, 188 op/s 2024-04-25T00:19:48.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:48 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/172747286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:19:48.744 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:19:48.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:19:48.744 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:19:48.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.858+0000 7feefe5c8640 1 -- 172.21.15.107:0/1492982890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feef8100600 msgr2=0x7feef810e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:48.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.858+0000 7feefe5c8640 1 --2- 172.21.15.107:0/1492982890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feef8100600 0x7feef810e5e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7feef0009cd0 tx=0x7feef002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.858+0000 7feefe5c8640 1 -- 172.21.15.107:0/1492982890 shutdown_connections 2024-04-25T00:19:48.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.858+0000 7feefe5c8640 1 --2- 172.21.15.107:0/1492982890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feef8100600 0x7feef810e5e0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.858+0000 7feefe5c8640 1 --2- 172.21.15.107:0/1492982890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feef80ffc40 0x7feef81000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.858+0000 7feefe5c8640 1 --2- 172.21.15.107:0/1492982890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feef8107720 0x7feef8107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.858+0000 7feefe5c8640 1 -- 172.21.15.107:0/1492982890 >> 172.21.15.107:0/1492982890 conn(0x7feef8076c00 msgr2=0x7feef80fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:48.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.858+0000 7feefe5c8640 1 -- 172.21.15.107:0/1492982890 shutdown_connections 2024-04-25T00:19:48.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.858+0000 7feefe5c8640 1 -- 172.21.15.107:0/1492982890 wait complete. 2024-04-25T00:19:48.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.859+0000 7feefe5c8640 1 Processor -- start 2024-04-25T00:19:48.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.859+0000 7feefe5c8640 1 -- start start 2024-04-25T00:19:48.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.859+0000 7feefe5c8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feef80ffc40 0x7feef8198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.859+0000 7feefe5c8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feef8100600 0x7feef8198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.859+0000 7feefe5c8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feef8107720 0x7feef819c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.859+0000 7feefe5c8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feef8111b30 con 0x7feef8107720 2024-04-25T00:19:48.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.859+0000 7feefe5c8640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7feef81119b0 con 0x7feef80ffc40 2024-04-25T00:19:48.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.859+0000 7feefe5c8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feef8111cb0 con 0x7feef8100600 2024-04-25T00:19:48.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.859+0000 7feefcb65640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feef8107720 0x7feef819c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.860+0000 7feefcb65640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feef8107720 0x7feef819c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55592/0 (socket says 172.21.15.107:55592) 2024-04-25T00:19:48.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.860+0000 7feefcb65640 1 -- 172.21.15.107:0/159061173 learned_addr learned my addr 172.21.15.107:0/159061173 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:48.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.860+0000 7feef7fff640 1 --2- 172.21.15.107:0/159061173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feef80ffc40 0x7feef8198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.860+0000 7feefcb65640 1 -- 172.21.15.107:0/159061173 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feef8100600 msgr2=0x7feef8198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:48.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.860+0000 7feefcb65640 1 --2- 172.21.15.107:0/159061173 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feef8100600 0x7feef8198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.860+0000 7feefcb65640 1 -- 172.21.15.107:0/159061173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feef80ffc40 msgr2=0x7feef8198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:48.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.860+0000 7feefcb65640 1 --2- 172.21.15.107:0/159061173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feef80ffc40 0x7feef8198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.860+0000 7feefcb65640 1 -- 172.21.15.107:0/159061173 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef819cfd0 con 0x7feef8107720 2024-04-25T00:19:48.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.860+0000 7feefcb65640 1 --2- 172.21.15.107:0/159061173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feef8107720 0x7feef819c8a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7feef0005cc0 tx=0x7feef0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:48.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.860+0000 7feef57fa640 1 -- 172.21.15.107:0/159061173 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0004040 con 0x7feef8107720 2024-04-25T00:19:48.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.860+0000 7feefe5c8640 1 -- 172.21.15.107:0/159061173 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef819d260 con 0x7feef8107720 2024-04-25T00:19:48.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.860+0000 7feef57fa640 1 -- 172.21.15.107:0/159061173 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feef00046c0 con 0x7feef8107720 2024-04-25T00:19:48.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.860+0000 7feef57fa640 1 -- 172.21.15.107:0/159061173 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0046480 con 0x7feef8107720 2024-04-25T00:19:48.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.860+0000 7feefe5c8640 1 -- 172.21.15.107:0/159061173 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef81a4b50 con 0x7feef8107720 2024-04-25T00:19:48.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.862+0000 7feef57fa640 1 -- 172.21.15.107:0/159061173 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7feef00305a0 con 0x7feef8107720 2024-04-25T00:19:48.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.862+0000 7fef013b3740 1 Processor -- start 2024-04-25T00:19:48.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.863+0000 7fef013b3740 1 -- start start 2024-04-25T00:19:48.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.863+0000 7fef013b3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d6b980 0x55b9e2d69cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.863+0000 7fef013b3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9e2d6a290 0x55b9e2d6a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.863+0000 7fef013b3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b9e2d75010 0x55b9e2d77400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.863+0000 7fef013b3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b9e2cce2c0 con 0x55b9e2d6b980 2024-04-25T00:19:48.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.863+0000 7fef013b3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b9e2c76910 con 0x55b9e2d75010 2024-04-25T00:19:48.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.863+0000 7fef013b3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b9e2cca280 con 0x55b9e2d6a290 2024-04-25T00:19:48.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.863+0000 7feef57fa640 1 --2- 172.21.15.107:0/159061173 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7feee00a2d00 0x7feee00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.863+0000 7feef77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d6b980 0x55b9e2d69cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.863+0000 7feef77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d6b980 0x55b9e2d69cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55594/0 (socket says 172.21.15.107:55594) 2024-04-25T00:19:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.863+0000 7feef77fe640 1 -- 172.21.15.107:0/85759829 learned_addr learned my addr 172.21.15.107:0/85759829 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.863+0000 7feef7fff640 1 --2- 172.21.15.107:0/85759829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9e2d6a290 0x55b9e2d6a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.863+0000 7feef77fe640 1 --2- 172.21.15.107:0/85759829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b9e2d75010 0x55b9e2d77400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.863+0000 7feefcb65640 1 --2- 172.21.15.107:0/159061173 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7feee00a2d00 0x7feee00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.863+0000 7feef57fa640 1 -- 172.21.15.107:0/159061173 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7feef002e030 con 0x7feef8107720 2024-04-25T00:19:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.863+0000 7feef77fe640 1 -- 172.21.15.107:0/85759829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9e2d6a290 msgr2=0x55b9e2d6a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.863+0000 7feef77fe640 1 --2- 172.21.15.107:0/85759829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9e2d6a290 0x55b9e2d6a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.863+0000 7feef77fe640 1 -- 172.21.15.107:0/85759829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b9e2d75010 msgr2=0x55b9e2d77400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.863+0000 7feef77fe640 1 --2- 172.21.15.107:0/85759829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b9e2d75010 0x55b9e2d77400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.863+0000 7feef77fe640 1 -- 172.21.15.107:0/85759829 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9e2c87fb0 con 0x55b9e2d6b980 2024-04-25T00:19:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.863+0000 7feef77fe640 1 --2- 172.21.15.107:0/85759829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d6b980 0x55b9e2d69cc0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7feeec013770 tx=0x7feeec005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9476c50bb15dfb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:48.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.864+0000 7feefcb65640 1 --2- 172.21.15.107:0/159061173 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7feee00a2d00 0x7feee00a51c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7feef00ee560 tx=0x7feef00dd040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:48.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.864+0000 7feede7fc640 1 -- 172.21.15.107:0/85759829 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec004220 con 0x55b9e2d6b980 2024-04-25T00:19:48.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.864+0000 7feede7fc640 1 -- 172.21.15.107:0/85759829 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeec00ada0 con 0x55b9e2d6b980 2024-04-25T00:19:48.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.864+0000 7feede7fc640 1 -- 172.21.15.107:0/85759829 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec003d00 con 0x55b9e2d6b980 2024-04-25T00:19:48.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.864+0000 7fef013b3740 1 -- 172.21.15.107:0/85759829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d6b980 msgr2=0x55b9e2d69cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:48.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.864+0000 7fef013b3740 1 --2- 172.21.15.107:0/85759829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d6b980 0x55b9e2d69cc0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7feeec013770 tx=0x7feeec005f90 comp rx=0 tx=0).stop 2024-04-25T00:19:48.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.864+0000 7fef013b3740 1 -- 172.21.15.107:0/85759829 shutdown_connections 2024-04-25T00:19:48.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.864+0000 7fef013b3740 1 --2- 172.21.15.107:0/85759829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b9e2d75010 0x55b9e2d77400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.864+0000 7fef013b3740 1 --2- 172.21.15.107:0/85759829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9e2d6a290 0x55b9e2d6a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.864+0000 7fef013b3740 1 --2- 172.21.15.107:0/85759829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d6b980 0x55b9e2d69cc0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.864+0000 7fef013b3740 1 -- 172.21.15.107:0/85759829 >> 172.21.15.107:0/85759829 conn(0x55b9e2d32950 msgr2=0x55b9e2d667f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:48.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.864+0000 7fef013b3740 1 -- 172.21.15.107:0/85759829 shutdown_connections 2024-04-25T00:19:48.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.864+0000 7fef013b3740 1 -- 172.21.15.107:0/85759829 wait complete. 2024-04-25T00:19:48.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.865+0000 7fef013b3740 1 -- start start 2024-04-25T00:19:48.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.865+0000 7fef013b3740 1 Processor -- start 2024-04-25T00:19:48.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.865+0000 7fef013b3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d6a290 0x55b9e2d71ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.865+0000 7fef013b3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9e2d6b980 0x55b9e2d72430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.865+0000 7fef013b3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b9e2d75010 0x55b9e2d72970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.865+0000 7fef013b3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b9e2d77ea0 con 0x55b9e2d6a290 2024-04-25T00:19:48.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.865+0000 7fef013b3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b9e2c89e90 con 0x55b9e2d75010 2024-04-25T00:19:48.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.865+0000 7fef013b3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b9e2d78020 con 0x55b9e2d6b980 2024-04-25T00:19:48.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.865+0000 7fef013b3740 10 client.0 _ll_register_callbacks cb 0x55b9e2cc5970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:48.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.865+0000 7feef77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d6a290 0x55b9e2d71ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.865+0000 7feef77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d6a290 0x55b9e2d71ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55596/0 (socket says 172.21.15.107:55596) 2024-04-25T00:19:48.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.865+0000 7feef77fe640 1 -- 172.21.15.107:0/1383970418 learned_addr learned my addr 172.21.15.107:0/1383970418 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:48.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.866+0000 7feef77fe640 1 -- 172.21.15.107:0/1383970418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9e2d6b980 msgr2=0x55b9e2d72430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:48.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.866+0000 7feef77fe640 1 --2- 172.21.15.107:0/1383970418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9e2d6b980 0x55b9e2d72430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.866+0000 7feef77fe640 1 -- 172.21.15.107:0/1383970418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b9e2d75010 msgr2=0x55b9e2d72970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:48.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.866+0000 7feef77fe640 1 --2- 172.21.15.107:0/1383970418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b9e2d75010 0x55b9e2d72970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.866+0000 7feef77fe640 1 -- 172.21.15.107:0/1383970418 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9e2aaa7e0 con 0x55b9e2d6a290 2024-04-25T00:19:48.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.866+0000 7feef77fe640 1 --2- 172.21.15.107:0/1383970418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d6a290 0x55b9e2d71ef0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7feeec005c30 tx=0x7feeec00afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a220d8db2c7973cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:48.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.866+0000 7feed77fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:19:48.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.866+0000 7feed77fe640 1 -- 172.21.15.107:0/1383970418 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec013050 con 0x55b9e2d6a290 2024-04-25T00:19:48.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.866+0000 7feed77fe640 1 -- 172.21.15.107:0/1383970418 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7feeec0131f0 con 0x55b9e2d6a290 2024-04-25T00:19:48.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.866+0000 7fef013b3740 1 -- 172.21.15.107:0/1383970418 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeec00ada0 con 0x55b9e2d6a290 2024-04-25T00:19:48.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.866+0000 7feed77fe640 1 -- 172.21.15.107:0/1383970418 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feeec030050 con 0x55b9e2d6a290 2024-04-25T00:19:48.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.866+0000 7fef013b3740 10 client.16380 fetch_fsmap learned FSMap version 28 2024-04-25T00:19:48.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.866+0000 7fef013b3740 1 -- 172.21.15.107:0/1383970418 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b9e2b0af70 con 0x55b9e2d6a290 2024-04-25T00:19:48.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.866+0000 7feed77fe640 1 -- 172.21.15.107:0/1383970418 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeec00ada0 con 0x55b9e2d6a290 2024-04-25T00:19:48.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.866+0000 7feed77fe640 1 -- 172.21.15.107:0/1383970418 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7feeec025020 con 0x55b9e2d6a290 2024-04-25T00:19:48.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.866+0000 7fef013b3740 10 client.16380 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:19:48.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.866+0000 7fef013b3740 10 client.16380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:48.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.866+0000 7fef013b3740 20 client.16380 populate_metadata read hostname 'smithi107' 2024-04-25T00:19:48.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.866+0000 7fef013b3740 1 --2- 172.21.15.107:0/1383970418 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b9e2d84fd0 0x55b9e2d873c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.867+0000 7fef013b3740 4 client.16380 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:19:48.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.867+0000 7fef013b3740 1 -- 172.21.15.107:0/1383970418 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9e2ca25a0 con 0x55b9e2d84fd0 2024-04-25T00:19:48.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.867+0000 7feef7fff640 1 --2- 172.21.15.107:0/1383970418 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b9e2d84fd0 0x55b9e2d873c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.867+0000 7feed77fe640 10 client.16380 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:48.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.867+0000 7feef7fff640 1 --2- 172.21.15.107:0/1383970418 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b9e2d84fd0 0x55b9e2d873c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10463bc8a4e533cd server_cookie=12b0a01548f99b8a in_seq=0 out_seq=0 2024-04-25T00:19:48.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.868+0000 7feed77fe640 1 -- 172.21.15.107:0/1383970418 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b9e2ca25a0 con 0x55b9e2d84fd0 2024-04-25T00:19:48.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.868+0000 7feed77fe640 10 client.16380 handle_command_reply: tid=0 2024-04-25T00:19:48.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.868+0000 7fef013b3740 1 client.16380 shutdown 2024-04-25T00:19:48.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.868+0000 7fef013b3740 1 -- 172.21.15.107:0/1383970418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d6a290 msgr2=0x55b9e2d71ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:48.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.868+0000 7fef013b3740 1 --2- 172.21.15.107:0/1383970418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d6a290 0x55b9e2d71ef0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7feeec005c30 tx=0x7feeec00afc0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.868+0000 7fef013b3740 1 -- 172.21.15.107:0/1383970418 shutdown_connections 2024-04-25T00:19:48.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.868+0000 7fef013b3740 1 --2- 172.21.15.107:0/1383970418 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b9e2d84fd0 0x55b9e2d873c0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.868+0000 7fef013b3740 1 --2- 172.21.15.107:0/1383970418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b9e2d75010 0x55b9e2d72970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.868+0000 7fef013b3740 1 --2- 172.21.15.107:0/1383970418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9e2d6b980 0x55b9e2d72430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.868+0000 7feed77fe640 0 client.16380 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:48.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.868+0000 7fef013b3740 1 --2- 172.21.15.107:0/1383970418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d6a290 0x55b9e2d71ef0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.868+0000 7fef013b3740 1 -- 172.21.15.107:0/1383970418 >> 172.21.15.107:0/1383970418 conn(0x55b9e2d32950 msgr2=0x55b9e2d6c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:48.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.868+0000 7fef013b3740 1 -- 172.21.15.107:0/1383970418 shutdown_connections 2024-04-25T00:19:48.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.869+0000 7fef013b3740 1 -- 172.21.15.107:0/1383970418 wait complete. 2024-04-25T00:19:48.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.869+0000 7fef013b3740 20 client.16380 trim_cache size 0 max 16384 2024-04-25T00:19:48.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.886+0000 7fef013b3740 1 Processor -- start 2024-04-25T00:19:48.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.886+0000 7fef013b3740 1 -- start start 2024-04-25T00:19:48.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.886+0000 7fef013b3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9e2d6a290 0x55b9e2d70e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.886+0000 7fef013b3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b9e2d6b980 0x55b9e2d80dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.886+0000 7fef013b3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d75010 0x55b9e2d7d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.886+0000 7fef013b3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feef81b13e0 con 0x55b9e2d75010 2024-04-25T00:19:48.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.886+0000 7fef013b3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7feef81aebf0 con 0x55b9e2d6b980 2024-04-25T00:19:48.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.886+0000 7fef013b3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feef81aca40 con 0x55b9e2d6a290 2024-04-25T00:19:48.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.886+0000 7feef77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d75010 0x55b9e2d7d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.887+0000 7feef77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d75010 0x55b9e2d7d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55612/0 (socket says 172.21.15.107:55612) 2024-04-25T00:19:48.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.887+0000 7feef77fe640 1 -- 172.21.15.107:0/670117040 learned_addr learned my addr 172.21.15.107:0/670117040 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:48.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.887+0000 7feef77fe640 1 -- 172.21.15.107:0/670117040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9e2d6a290 msgr2=0x55b9e2d70e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:48.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.887+0000 7feef77fe640 1 --2- 172.21.15.107:0/670117040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9e2d6a290 0x55b9e2d70e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.887+0000 7feef77fe640 1 -- 172.21.15.107:0/670117040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b9e2d6b980 msgr2=0x55b9e2d80dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:48.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.887+0000 7feef7fff640 1 --2- 172.21.15.107:0/670117040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b9e2d6b980 0x55b9e2d80dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.887+0000 7feef77fe640 1 --2- 172.21.15.107:0/670117040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b9e2d6b980 0x55b9e2d80dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.887+0000 7feef77fe640 1 -- 172.21.15.107:0/670117040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9e29a2170 con 0x55b9e2d75010 2024-04-25T00:19:48.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.887+0000 7feef7fff640 1 --2- 172.21.15.107:0/670117040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b9e2d6b980 0x55b9e2d80dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:48.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.887+0000 7feef77fe640 1 --2- 172.21.15.107:0/670117040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d75010 0x55b9e2d7d6c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7feeec00ee60 tx=0x7feeec005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf11324a87b7bbdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:48.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.887+0000 7feeddffb640 1 -- 172.21.15.107:0/670117040 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec02d040 con 0x55b9e2d75010 2024-04-25T00:19:48.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.887+0000 7feeddffb640 1 -- 172.21.15.107:0/670117040 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeec027890 con 0x55b9e2d75010 2024-04-25T00:19:48.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.887+0000 7feeddffb640 1 -- 172.21.15.107:0/670117040 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec027bb0 con 0x55b9e2d75010 2024-04-25T00:19:48.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.888+0000 7fef013b3740 1 -- 172.21.15.107:0/670117040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d75010 msgr2=0x55b9e2d7d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:48.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.888+0000 7fef013b3740 1 --2- 172.21.15.107:0/670117040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d75010 0x55b9e2d7d6c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7feeec00ee60 tx=0x7feeec005bc0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.888+0000 7fef013b3740 1 -- 172.21.15.107:0/670117040 shutdown_connections 2024-04-25T00:19:48.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.888+0000 7fef013b3740 1 --2- 172.21.15.107:0/670117040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d75010 0x55b9e2d7d6c0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.888+0000 7fef013b3740 1 --2- 172.21.15.107:0/670117040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b9e2d6b980 0x55b9e2d80dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.888+0000 7fef013b3740 1 --2- 172.21.15.107:0/670117040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9e2d6a290 0x55b9e2d70e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.888+0000 7fef013b3740 1 -- 172.21.15.107:0/670117040 >> 172.21.15.107:0/670117040 conn(0x55b9e2d32950 msgr2=0x55b9e2d65950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:48.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.888+0000 7fef013b3740 1 -- 172.21.15.107:0/670117040 shutdown_connections 2024-04-25T00:19:48.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.888+0000 7fef013b3740 1 -- 172.21.15.107:0/670117040 wait complete. 2024-04-25T00:19:48.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.888+0000 7fef013b3740 1 -- start start 2024-04-25T00:19:48.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.888+0000 7fef013b3740 1 Processor -- start 2024-04-25T00:19:48.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.888+0000 7fef013b3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d6a290 0x55b9e2d6f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.888+0000 7fef013b3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b9e2d6b980 0x55b9e2d6e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.888+0000 7fef013b3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9e2d75010 0x55b9e2d6e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.888+0000 7fef013b3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feef81ad0b0 con 0x55b9e2d6a290 2024-04-25T00:19:48.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.888+0000 7fef013b3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7feef81ae520 con 0x55b9e2d6b980 2024-04-25T00:19:48.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.888+0000 7fef013b3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feef81ad780 con 0x55b9e2d75010 2024-04-25T00:19:48.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.888+0000 7fef013b3740 10 client.0 _ll_register_callbacks cb 0x7feeec00eca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:48.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.889+0000 7feef77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d6a290 0x55b9e2d6f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.889+0000 7feef77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d6a290 0x55b9e2d6f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55614/0 (socket says 172.21.15.107:55614) 2024-04-25T00:19:48.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.889+0000 7feef77fe640 1 -- 172.21.15.107:0/2350877808 learned_addr learned my addr 172.21.15.107:0/2350877808 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:48.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.889+0000 7feef7fff640 1 --2- 172.21.15.107:0/2350877808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b9e2d6b980 0x55b9e2d6e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.889+0000 7feef77fe640 1 --2- 172.21.15.107:0/2350877808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9e2d75010 0x55b9e2d6e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.889+0000 7feef77fe640 1 -- 172.21.15.107:0/2350877808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9e2d75010 msgr2=0x55b9e2d6e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:48.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.889+0000 7feef77fe640 1 --2- 172.21.15.107:0/2350877808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9e2d75010 0x55b9e2d6e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.889+0000 7feef77fe640 1 -- 172.21.15.107:0/2350877808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b9e2d6b980 msgr2=0x55b9e2d6e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:48.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.889+0000 7feef77fe640 1 --2- 172.21.15.107:0/2350877808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b9e2d6b980 0x55b9e2d6e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.889+0000 7feef77fe640 1 -- 172.21.15.107:0/2350877808 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9e2d6ec20 con 0x55b9e2d6a290 2024-04-25T00:19:48.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.889+0000 7feef77fe640 1 --2- 172.21.15.107:0/2350877808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d6a290 0x55b9e2d6f870 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7feeec013770 tx=0x7feeec027440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f69218a60a756a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:48.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.889+0000 7feed77fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:19:48.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.889+0000 7feed77fe640 1 -- 172.21.15.107:0/2350877808 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec027bb0 con 0x55b9e2d6a290 2024-04-25T00:19:48.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.889+0000 7feed77fe640 1 -- 172.21.15.107:0/2350877808 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7feeec027d50 con 0x55b9e2d6a290 2024-04-25T00:19:48.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.889+0000 7fef013b3740 1 -- 172.21.15.107:0/2350877808 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeec027890 con 0x55b9e2d6a290 2024-04-25T00:19:48.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.890+0000 7feed77fe640 1 -- 172.21.15.107:0/2350877808 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feeec00ab90 con 0x55b9e2d6a290 2024-04-25T00:19:48.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.890+0000 7fef013b3740 10 client.16386 fetch_fsmap learned FSMap version 28 2024-04-25T00:19:48.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.890+0000 7fef013b3740 1 -- 172.21.15.107:0/2350877808 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b9e2df0090 con 0x55b9e2d6a290 2024-04-25T00:19:48.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.890+0000 7feed77fe640 1 -- 172.21.15.107:0/2350877808 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeec00ad30 con 0x55b9e2d6a290 2024-04-25T00:19:48.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.890+0000 7feed77fe640 1 -- 172.21.15.107:0/2350877808 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7feeec025070 con 0x55b9e2d6a290 2024-04-25T00:19:48.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.890+0000 7fef013b3740 10 client.16386 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:19:48.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.890+0000 7fef013b3740 10 client.16386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:48.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.890+0000 7fef013b3740 20 client.16386 populate_metadata read hostname 'smithi107' 2024-04-25T00:19:48.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.890+0000 7fef013b3740 1 --2- 172.21.15.107:0/2350877808 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b9e2df0720 0x55b9e2df2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:48.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.890+0000 7fef013b3740 4 client.16386 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:19:48.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.890+0000 7fef013b3740 1 -- 172.21.15.107:0/2350877808 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7feeec027890 con 0x55b9e2df0720 2024-04-25T00:19:48.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.890+0000 7feef7fff640 1 --2- 172.21.15.107:0/2350877808 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b9e2df0720 0x55b9e2df2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:48.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.891+0000 7feed77fe640 10 client.16386 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:48.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.891+0000 7feef7fff640 1 --2- 172.21.15.107:0/2350877808 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b9e2df0720 0x55b9e2df2b40 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c0e1b34740171c2 server_cookie=411fc47614b87582 in_seq=0 out_seq=0 2024-04-25T00:19:48.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.891+0000 7feed77fe640 1 -- 172.21.15.107:0/2350877808 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7feeec027890 con 0x55b9e2df0720 2024-04-25T00:19:48.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.891+0000 7feed77fe640 10 client.16386 handle_command_reply: tid=0 2024-04-25T00:19:48.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.891+0000 7fef013b3740 1 client.16386 shutdown 2024-04-25T00:19:48.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.892+0000 7fef013b3740 1 -- 172.21.15.107:0/2350877808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d6a290 msgr2=0x55b9e2d6f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:48.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.892+0000 7fef013b3740 1 --2- 172.21.15.107:0/2350877808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d6a290 0x55b9e2d6f870 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7feeec013770 tx=0x7feeec027440 comp rx=0 tx=0).stop 2024-04-25T00:19:48.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.892+0000 7fef013b3740 1 -- 172.21.15.107:0/2350877808 shutdown_connections 2024-04-25T00:19:48.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.892+0000 7fef013b3740 1 --2- 172.21.15.107:0/2350877808 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b9e2df0720 0x55b9e2df2b40 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.892+0000 7fef013b3740 1 --2- 172.21.15.107:0/2350877808 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b9e2d75010 0x55b9e2d6e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.892+0000 7fef013b3740 1 --2- 172.21.15.107:0/2350877808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b9e2d6b980 0x55b9e2d6e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.892+0000 7fef013b3740 1 --2- 172.21.15.107:0/2350877808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b9e2d6a290 0x55b9e2d6f870 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.892+0000 7fef013b3740 1 -- 172.21.15.107:0/2350877808 >> 172.21.15.107:0/2350877808 conn(0x55b9e2d32950 msgr2=0x55b9e2d71e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:48.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.892+0000 7feed77fe640 0 client.16386 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:48.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.892+0000 7fef013b3740 1 -- 172.21.15.107:0/2350877808 shutdown_connections 2024-04-25T00:19:48.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.892+0000 7fef013b3740 1 -- 172.21.15.107:0/2350877808 wait complete. 2024-04-25T00:19:48.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.892+0000 7fef013b3740 20 client.16386 trim_cache size 0 max 16384 2024-04-25T00:19:48.899 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:19:48.899 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-04-25T00:19:48.899 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "5902815d-6463-4f78-8252-be90a6e0cb33", 2024-04-25T00:19:48.899 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-04-25T00:19:48.899 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:19:48.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.892+0000 7feefe5c8640 1 -- 172.21.15.107:0/159061173 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7feee00a2d00 msgr2=0x7feee00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:48.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.892+0000 7feefe5c8640 1 --2- 172.21.15.107:0/159061173 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7feee00a2d00 0x7feee00a51c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7feef00ee560 tx=0x7feef00dd040 comp rx=0 tx=0).stop 2024-04-25T00:19:48.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.892+0000 7feefe5c8640 1 -- 172.21.15.107:0/159061173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feef8107720 msgr2=0x7feef819c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:48.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.892+0000 7feefe5c8640 1 --2- 172.21.15.107:0/159061173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feef8107720 0x7feef819c8a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7feef0005cc0 tx=0x7feef0003970 comp rx=0 tx=0).stop 2024-04-25T00:19:48.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.893+0000 7feefe5c8640 1 -- 172.21.15.107:0/159061173 shutdown_connections 2024-04-25T00:19:48.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.893+0000 7feefe5c8640 1 --2- 172.21.15.107:0/159061173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feef8107720 0x7feef819c8a0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.893+0000 7feefe5c8640 1 --2- 172.21.15.107:0/159061173 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feef8100600 0x7feef8198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.893+0000 7feefe5c8640 1 --2- 172.21.15.107:0/159061173 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7feee00a2d00 0x7feee00a51c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.893+0000 7feefe5c8640 1 --2- 172.21.15.107:0/159061173 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feef80ffc40 0x7feef8198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:48.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.893+0000 7feefe5c8640 1 -- 172.21.15.107:0/159061173 >> 172.21.15.107:0/159061173 conn(0x7feef8076c00 msgr2=0x7feef80fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:48.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.893+0000 7feefe5c8640 1 -- 172.21.15.107:0/159061173 shutdown_connections 2024-04-25T00:19:48.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:48.893+0000 7feefe5c8640 1 -- 172.21.15.107:0/159061173 wait complete. 2024-04-25T00:19:48.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:19:48.904 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:19:49.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.004+0000 7f7ae071d640 1 -- 172.21.15.107:0/3987645051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ad8107720 msgr2=0x7f7ad8107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:49.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.004+0000 7f7ae071d640 1 --2- 172.21.15.107:0/3987645051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ad8107720 0x7f7ad8107b00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009930 tx=0x7f7ac802a420 comp rx=0 tx=0).stop 2024-04-25T00:19:49.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.004+0000 7f7ae071d640 1 -- 172.21.15.107:0/3987645051 shutdown_connections 2024-04-25T00:19:49.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.004+0000 7f7ae071d640 1 --2- 172.21.15.107:0/3987645051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ad8100600 0x7f7ad810e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.004+0000 7f7ae071d640 1 --2- 172.21.15.107:0/3987645051 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ad80ffc40 0x7f7ad81000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.004+0000 7f7ae071d640 1 --2- 172.21.15.107:0/3987645051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ad8107720 0x7f7ad8107b00 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.004+0000 7f7ae071d640 1 -- 172.21.15.107:0/3987645051 >> 172.21.15.107:0/3987645051 conn(0x7f7ad8076c00 msgr2=0x7f7ad80fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:49.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.004+0000 7f7ae071d640 1 -- 172.21.15.107:0/3987645051 shutdown_connections 2024-04-25T00:19:49.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.004+0000 7f7ae071d640 1 -- 172.21.15.107:0/3987645051 wait complete. 2024-04-25T00:19:49.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.004+0000 7f7ae071d640 1 Processor -- start 2024-04-25T00:19:49.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.005+0000 7f7ae071d640 1 -- start start 2024-04-25T00:19:49.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.005+0000 7f7ae071d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ad80ffc40 0x7f7ad8100e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:49.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.005+0000 7f7ae071d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ad8100600 0x7f7ad8101360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:49.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.005+0000 7f7ae071d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ad8107720 0x7f7ad81018a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:49.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.005+0000 7f7ae071d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad8111b30 con 0x7f7ad8100600 2024-04-25T00:19:49.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.005+0000 7f7ae071d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad81119b0 con 0x7f7ad80ffc40 2024-04-25T00:19:49.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.005+0000 7f7ae071d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad8111cb0 con 0x7f7ad8107720 2024-04-25T00:19:49.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.005+0000 7f7addcb8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ad8100600 0x7f7ad8101360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:49.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.005+0000 7f7addcb8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ad8100600 0x7f7ad8101360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55622/0 (socket says 172.21.15.107:55622) 2024-04-25T00:19:49.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.005+0000 7f7addcb8640 1 -- 172.21.15.107:0/9087055 learned_addr learned my addr 172.21.15.107:0/9087055 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:49.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.005+0000 7f7ade4b9640 1 --2- 172.21.15.107:0/9087055 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ad80ffc40 0x7f7ad8100e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:49.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.005+0000 7f7adecba640 1 --2- 172.21.15.107:0/9087055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ad8107720 0x7f7ad81018a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:49.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.005+0000 7f7addcb8640 1 -- 172.21.15.107:0/9087055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ad8107720 msgr2=0x7f7ad81018a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:49.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.005+0000 7f7addcb8640 1 --2- 172.21.15.107:0/9087055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ad8107720 0x7f7ad81018a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.005+0000 7f7addcb8640 1 -- 172.21.15.107:0/9087055 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ad80ffc40 msgr2=0x7f7ad8100e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:49.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.005+0000 7f7addcb8640 1 --2- 172.21.15.107:0/9087055 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ad80ffc40 0x7f7ad8100e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.005+0000 7f7addcb8640 1 -- 172.21.15.107:0/9087055 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad81a6550 con 0x7f7ad8100600 2024-04-25T00:19:49.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.005+0000 7f7adecba640 1 --2- 172.21.15.107:0/9087055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ad8107720 0x7f7ad81018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:49.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.005+0000 7f7addcb8640 1 --2- 172.21.15.107:0/9087055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ad8100600 0x7f7ad8101360 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400d950 tx=0x7f7ad400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:49.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.005+0000 7f7acf7fe640 1 -- 172.21.15.107:0/9087055 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4017070 con 0x7f7ad8100600 2024-04-25T00:19:49.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.005+0000 7f7ae071d640 1 -- 172.21.15.107:0/9087055 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad81a6840 con 0x7f7ad8100600 2024-04-25T00:19:49.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.005+0000 7f7acf7fe640 1 -- 172.21.15.107:0/9087055 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ad4009c30 con 0x7f7ad8100600 2024-04-25T00:19:49.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.005+0000 7f7acf7fe640 1 -- 172.21.15.107:0/9087055 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad400f6b0 con 0x7f7ad8100600 2024-04-25T00:19:49.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.005+0000 7f7ae071d640 1 -- 172.21.15.107:0/9087055 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad81a6d90 con 0x7f7ad8100600 2024-04-25T00:19:49.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.008+0000 7f7acf7fe640 1 -- 172.21.15.107:0/9087055 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7ad40109d0 con 0x7f7ad8100600 2024-04-25T00:19:49.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.008+0000 7f7ae3420740 1 Processor -- start 2024-04-25T00:19:49.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.009+0000 7f7ae3420740 1 -- start start 2024-04-25T00:19:49.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.009+0000 7f7ae3420740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bed1630 0x564f4bed1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:49.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.009+0000 7f7ae3420740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4bed1f50 0x564f4bedc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:49.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.009+0000 7f7ae3420740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f4bedc620 0x564f4bedea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:49.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.009+0000 7f7ae3420740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564f4be366b0 con 0x564f4bed1630 2024-04-25T00:19:49.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.009+0000 7f7ae3420740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564f4bdde880 con 0x564f4bed1f50 2024-04-25T00:19:49.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.009+0000 7f7ae3420740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564f4be32670 con 0x564f4bedc620 2024-04-25T00:19:49.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.009+0000 7f7adecba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bed1630 0x564f4bed1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:49.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.009+0000 7f7adecba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bed1630 0x564f4bed1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55624/0 (socket says 172.21.15.107:55624) 2024-04-25T00:19:49.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.009+0000 7f7adecba640 1 -- 172.21.15.107:0/2797930421 learned_addr learned my addr 172.21.15.107:0/2797930421 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:49.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.009+0000 7f7acf7fe640 1 --2- 172.21.15.107:0/9087055 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f7ac40a2d00 0x7f7ac40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:49.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.009+0000 7f7ade4b9640 1 --2- 172.21.15.107:0/2797930421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4bed1f50 0x564f4bedc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:49.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.009+0000 7f7adecba640 1 --2- 172.21.15.107:0/9087055 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f7ac40a2d00 0x7f7ac40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:49.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.009+0000 7f7addcb8640 1 --2- 172.21.15.107:0/2797930421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f4bedc620 0x564f4bedea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:49.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.009+0000 7f7acf7fe640 1 -- 172.21.15.107:0/9087055 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f7ad40cf430 con 0x7f7ad8100600 2024-04-25T00:19:49.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.009+0000 7f7adecba640 1 -- 172.21.15.107:0/2797930421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f4bedc620 msgr2=0x564f4bedea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:49.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.009+0000 7f7adecba640 1 --2- 172.21.15.107:0/2797930421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f4bedc620 0x564f4bedea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.009+0000 7f7adecba640 1 -- 172.21.15.107:0/2797930421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4bed1f50 msgr2=0x564f4bedc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:49.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.009+0000 7f7adecba640 1 --2- 172.21.15.107:0/2797930421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4bed1f50 0x564f4bedc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.009+0000 7f7adecba640 1 -- 172.21.15.107:0/2797930421 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f4bc147d0 con 0x564f4bed1630 2024-04-25T00:19:49.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.009+0000 7f7adecba640 1 --2- 172.21.15.107:0/2797930421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bed1630 0x564f4bed1a10 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0014ec0 tx=0x7f7ad0012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d781098f535844b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:49.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.010+0000 7f7adecba640 1 --2- 172.21.15.107:0/9087055 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f7ac40a2d00 0x7f7ac40a51c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0012c60 tx=0x7f7ad000c000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:49.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.010+0000 7f7accff9640 1 -- 172.21.15.107:0/2797930421 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0023070 con 0x564f4bed1630 2024-04-25T00:19:49.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.010+0000 7f7accff9640 1 -- 172.21.15.107:0/2797930421 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ad0014470 con 0x564f4bed1630 2024-04-25T00:19:49.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.010+0000 7f7accff9640 1 -- 172.21.15.107:0/2797930421 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad001fb50 con 0x564f4bed1630 2024-04-25T00:19:49.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.010+0000 7f7ae3420740 1 -- 172.21.15.107:0/2797930421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bed1630 msgr2=0x564f4bed1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:49.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.010+0000 7f7ae3420740 1 --2- 172.21.15.107:0/2797930421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bed1630 0x564f4bed1a10 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0014ec0 tx=0x7f7ad0012760 comp rx=0 tx=0).stop 2024-04-25T00:19:49.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.010+0000 7f7ae3420740 1 -- 172.21.15.107:0/2797930421 shutdown_connections 2024-04-25T00:19:49.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.010+0000 7f7ae3420740 1 --2- 172.21.15.107:0/2797930421 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f4bedc620 0x564f4bedea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.010+0000 7f7ae3420740 1 --2- 172.21.15.107:0/2797930421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4bed1f50 0x564f4bedc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.010+0000 7f7ae3420740 1 --2- 172.21.15.107:0/2797930421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bed1630 0x564f4bed1a10 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.010+0000 7f7ae3420740 1 -- 172.21.15.107:0/2797930421 >> 172.21.15.107:0/2797930421 conn(0x564f4becd950 msgr2=0x564f4becdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:49.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.010+0000 7f7ae3420740 1 -- 172.21.15.107:0/2797930421 shutdown_connections 2024-04-25T00:19:49.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.010+0000 7f7ae3420740 1 -- 172.21.15.107:0/2797930421 wait complete. 2024-04-25T00:19:49.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.011+0000 7f7ae3420740 1 -- start start 2024-04-25T00:19:49.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.011+0000 7f7ae3420740 1 Processor -- start 2024-04-25T00:19:49.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.011+0000 7f7ae3420740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4bed1630 0x564f4bed9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:49.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.011+0000 7f7ae3420740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f4bed1f50 0x564f4bed9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:49.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.011+0000 7f7ae3420740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bedc620 0x564f4beda0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:49.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.011+0000 7f7ae3420740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564f4bedf3b0 con 0x564f4bedc620 2024-04-25T00:19:49.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.011+0000 7f7ae3420740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564f4bdf1f50 con 0x564f4bed1630 2024-04-25T00:19:49.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.011+0000 7f7ae3420740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564f4bedf530 con 0x564f4bed1f50 2024-04-25T00:19:49.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.011+0000 7f7ae3420740 10 client.0 _ll_register_callbacks cb 0x564f4be2d000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:49.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.011+0000 7f7adecba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bedc620 0x564f4beda0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:49.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.011+0000 7f7adecba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bedc620 0x564f4beda0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55630/0 (socket says 172.21.15.107:55630) 2024-04-25T00:19:49.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.011+0000 7f7adecba640 1 -- 172.21.15.107:0/3710377831 learned_addr learned my addr 172.21.15.107:0/3710377831 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:49.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.011+0000 7f7ade4b9640 1 --2- 172.21.15.107:0/3710377831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4bed1630 0x564f4bed9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:49.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.011+0000 7f7adecba640 1 -- 172.21.15.107:0/3710377831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f4bed1f50 msgr2=0x564f4bed9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:49.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.011+0000 7f7adecba640 1 --2- 172.21.15.107:0/3710377831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f4bed1f50 0x564f4bed9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.011+0000 7f7adecba640 1 -- 172.21.15.107:0/3710377831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4bed1630 msgr2=0x564f4bed9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:49.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.011+0000 7f7adecba640 1 --2- 172.21.15.107:0/3710377831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4bed1630 0x564f4bed9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.011+0000 7f7adecba640 1 -- 172.21.15.107:0/3710377831 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f4bb4a3f0 con 0x564f4bedc620 2024-04-25T00:19:49.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.011+0000 7f7ade4b9640 1 --2- 172.21.15.107:0/3710377831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4bed1630 0x564f4bed9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:49.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.012+0000 7f7adecba640 1 --2- 172.21.15.107:0/3710377831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bedc620 0x564f4beda0d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0014ad0 tx=0x7f7ad0014b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a817f1a8d797cdd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:49.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.012+0000 7f7ac1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:19:49.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.012+0000 7f7ac1ffb640 1 -- 172.21.15.107:0/3710377831 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0023070 con 0x564f4bedc620 2024-04-25T00:19:49.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.012+0000 7f7ac1ffb640 1 -- 172.21.15.107:0/3710377831 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f7ad0004a00 con 0x564f4bedc620 2024-04-25T00:19:49.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.012+0000 7f7ae3420740 1 -- 172.21.15.107:0/3710377831 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ad0014470 con 0x564f4bedc620 2024-04-25T00:19:49.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.012+0000 7f7ac1ffb640 1 -- 172.21.15.107:0/3710377831 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7ad0004c20 con 0x564f4bedc620 2024-04-25T00:19:49.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.012+0000 7f7ae3420740 10 client.16398 fetch_fsmap learned FSMap version 28 2024-04-25T00:19:49.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.012+0000 7f7ae3420740 1 -- 172.21.15.107:0/3710377831 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564f4bb0a0f0 con 0x564f4bedc620 2024-04-25T00:19:49.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.012+0000 7f7ac1ffb640 1 -- 172.21.15.107:0/3710377831 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ad0014470 con 0x564f4bedc620 2024-04-25T00:19:49.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.012+0000 7f7ac1ffb640 1 -- 172.21.15.107:0/3710377831 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f7ad002e070 con 0x564f4bedc620 2024-04-25T00:19:49.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.012+0000 7f7ae3420740 10 client.16398 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:19:49.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.012+0000 7f7ae3420740 10 client.16398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:49.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.012+0000 7f7ae3420740 20 client.16398 populate_metadata read hostname 'smithi107' 2024-04-25T00:19:49.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.012+0000 7f7ae3420740 1 --2- 172.21.15.107:0/3710377831 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x564f4beec7d0 0x564f4beeec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:49.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.012+0000 7f7ae3420740 4 client.16398 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:19:49.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.012+0000 7f7ae3420740 1 -- 172.21.15.107:0/3710377831 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f4be43bf0 con 0x564f4beec7d0 2024-04-25T00:19:49.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.012+0000 7f7ade4b9640 1 --2- 172.21.15.107:0/3710377831 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x564f4beec7d0 0x564f4beeec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:49.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.013+0000 7f7ac1ffb640 10 client.16398 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:49.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.013+0000 7f7ade4b9640 1 --2- 172.21.15.107:0/3710377831 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x564f4beec7d0 0x564f4beeec00 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c777e8ec04fdc51 server_cookie=2eeb706d0a96f9ef in_seq=0 out_seq=0 2024-04-25T00:19:49.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.013+0000 7f7ac1ffb640 1 -- 172.21.15.107:0/3710377831 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f4be43bf0 con 0x564f4beec7d0 2024-04-25T00:19:49.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.014+0000 7f7ac1ffb640 10 client.16398 handle_command_reply: tid=0 2024-04-25T00:19:49.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.014+0000 7f7ae3420740 1 client.16398 shutdown 2024-04-25T00:19:49.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.014+0000 7f7ae3420740 1 -- 172.21.15.107:0/3710377831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bedc620 msgr2=0x564f4beda0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:49.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.014+0000 7f7ae3420740 1 --2- 172.21.15.107:0/3710377831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bedc620 0x564f4beda0d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0014ad0 tx=0x7f7ad0014b00 comp rx=0 tx=0).stop 2024-04-25T00:19:49.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.014+0000 7f7ae3420740 1 -- 172.21.15.107:0/3710377831 shutdown_connections 2024-04-25T00:19:49.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.014+0000 7f7ae3420740 1 --2- 172.21.15.107:0/3710377831 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x564f4beec7d0 0x564f4beeec00 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.014+0000 7f7ae3420740 1 --2- 172.21.15.107:0/3710377831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bedc620 0x564f4beda0d0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.014+0000 7f7ae3420740 1 --2- 172.21.15.107:0/3710377831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f4bed1f50 0x564f4bed9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.014+0000 7f7ae3420740 1 --2- 172.21.15.107:0/3710377831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4bed1630 0x564f4bed9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.014+0000 7f7ac1ffb640 0 client.16398 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:49.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.014+0000 7f7ae3420740 1 -- 172.21.15.107:0/3710377831 >> 172.21.15.107:0/3710377831 conn(0x564f4becd950 msgr2=0x564f4bed3750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:49.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.014+0000 7f7ae3420740 1 -- 172.21.15.107:0/3710377831 shutdown_connections 2024-04-25T00:19:49.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.014+0000 7f7ae3420740 1 -- 172.21.15.107:0/3710377831 wait complete. 2024-04-25T00:19:49.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.014+0000 7f7ae3420740 20 client.16398 trim_cache size 0 max 16384 2024-04-25T00:19:49.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.031+0000 7f7ae3420740 1 Processor -- start 2024-04-25T00:19:49.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.031+0000 7f7ae3420740 1 -- start start 2024-04-25T00:19:49.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.031+0000 7f7ae3420740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f4bed1630 0x564f4bedbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:49.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.031+0000 7f7ae3420740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4bed1f50 0x564f4bee8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:49.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.031+0000 7f7ae3420740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bedc620 0x564f4becc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:49.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.031+0000 7f7ae3420740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad81b36f0 con 0x564f4bedc620 2024-04-25T00:19:49.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.031+0000 7f7ae3420740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad81b0f00 con 0x564f4bed1f50 2024-04-25T00:19:49.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.031+0000 7f7ae3420740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad81aed50 con 0x564f4bed1630 2024-04-25T00:19:49.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.031+0000 7f7adecba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bedc620 0x564f4becc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:49.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.031+0000 7f7adecba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bedc620 0x564f4becc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55634/0 (socket says 172.21.15.107:55634) 2024-04-25T00:19:49.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.031+0000 7f7adecba640 1 -- 172.21.15.107:0/4180834778 learned_addr learned my addr 172.21.15.107:0/4180834778 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:49.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.031+0000 7f7adecba640 1 -- 172.21.15.107:0/4180834778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f4bed1630 msgr2=0x564f4bedbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:49.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.031+0000 7f7adecba640 1 --2- 172.21.15.107:0/4180834778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f4bed1630 0x564f4bedbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.031+0000 7f7addcb8640 1 --2- 172.21.15.107:0/4180834778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4bed1f50 0x564f4bee8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:49.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.031+0000 7f7adecba640 1 -- 172.21.15.107:0/4180834778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4bed1f50 msgr2=0x564f4bee8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:49.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.031+0000 7f7adecba640 1 --2- 172.21.15.107:0/4180834778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4bed1f50 0x564f4bee8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.031+0000 7f7adecba640 1 -- 172.21.15.107:0/4180834778 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f4bebf090 con 0x564f4bedc620 2024-04-25T00:19:49.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.031+0000 7f7addcb8640 1 --2- 172.21.15.107:0/4180834778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4bed1f50 0x564f4bee8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:49.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.031+0000 7f7adecba640 1 --2- 172.21.15.107:0/4180834778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bedc620 0x564f4becc970 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0014d60 tx=0x7f7ad0009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35af109c1a54497 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:49.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.032+0000 7f7ac3fff640 1 -- 172.21.15.107:0/4180834778 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0023070 con 0x564f4bedc620 2024-04-25T00:19:49.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.032+0000 7f7ac3fff640 1 -- 172.21.15.107:0/4180834778 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ad000ce20 con 0x564f4bedc620 2024-04-25T00:19:49.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.032+0000 7f7ac3fff640 1 -- 172.21.15.107:0/4180834778 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad001f8f0 con 0x564f4bedc620 2024-04-25T00:19:49.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.032+0000 7f7ae3420740 1 -- 172.21.15.107:0/4180834778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bedc620 msgr2=0x564f4becc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:49.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.032+0000 7f7ae3420740 1 --2- 172.21.15.107:0/4180834778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bedc620 0x564f4becc970 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0014d60 tx=0x7f7ad0009c60 comp rx=0 tx=0).stop 2024-04-25T00:19:49.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.032+0000 7f7ae3420740 1 -- 172.21.15.107:0/4180834778 shutdown_connections 2024-04-25T00:19:49.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.032+0000 7f7ae3420740 1 --2- 172.21.15.107:0/4180834778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bedc620 0x564f4becc970 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.032+0000 7f7ae3420740 1 --2- 172.21.15.107:0/4180834778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4bed1f50 0x564f4bee8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.032+0000 7f7ae3420740 1 --2- 172.21.15.107:0/4180834778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f4bed1630 0x564f4bedbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.032+0000 7f7ae3420740 1 -- 172.21.15.107:0/4180834778 >> 172.21.15.107:0/4180834778 conn(0x564f4becd950 msgr2=0x564f4becc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:49.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.032+0000 7f7ae3420740 1 -- 172.21.15.107:0/4180834778 shutdown_connections 2024-04-25T00:19:49.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.032+0000 7f7ae3420740 1 -- 172.21.15.107:0/4180834778 wait complete. 2024-04-25T00:19:49.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.032+0000 7f7ae3420740 1 -- start start 2024-04-25T00:19:49.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.033+0000 7f7ae3420740 1 Processor -- start 2024-04-25T00:19:49.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.033+0000 7f7ae3420740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4bed1630 0x564f4bedfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:49.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.033+0000 7f7ae3420740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bed1f50 0x564f4bee02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:49.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.033+0000 7f7ae3420740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f4bedc620 0x564f4bee4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:49.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.033+0000 7f7ae3420740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad81af3c0 con 0x564f4bed1f50 2024-04-25T00:19:49.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.033+0000 7f7ae3420740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad81b0830 con 0x564f4bed1630 2024-04-25T00:19:49.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.033+0000 7f7ae3420740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad81afa90 con 0x564f4bedc620 2024-04-25T00:19:49.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.033+0000 7f7ae3420740 10 client.0 _ll_register_callbacks cb 0x7f7ad0014c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:49.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.033+0000 7f7addcb8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bed1f50 0x564f4bee02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:49.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.033+0000 7f7addcb8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bed1f50 0x564f4bee02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55648/0 (socket says 172.21.15.107:55648) 2024-04-25T00:19:49.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.033+0000 7f7addcb8640 1 -- 172.21.15.107:0/875370928 learned_addr learned my addr 172.21.15.107:0/875370928 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:19:49.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.033+0000 7f7ade4b9640 1 --2- 172.21.15.107:0/875370928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4bed1630 0x564f4bedfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:49.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.033+0000 7f7adecba640 1 --2- 172.21.15.107:0/875370928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f4bedc620 0x564f4bee4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:49.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.033+0000 7f7addcb8640 1 -- 172.21.15.107:0/875370928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f4bedc620 msgr2=0x564f4bee4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:49.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.033+0000 7f7addcb8640 1 --2- 172.21.15.107:0/875370928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f4bedc620 0x564f4bee4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.033+0000 7f7addcb8640 1 -- 172.21.15.107:0/875370928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4bed1630 msgr2=0x564f4bedfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:49.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.033+0000 7f7addcb8640 1 --2- 172.21.15.107:0/875370928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4bed1630 0x564f4bedfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.033+0000 7f7addcb8640 1 -- 172.21.15.107:0/875370928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f4bee4870 con 0x564f4bed1f50 2024-04-25T00:19:49.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.033+0000 7f7adecba640 1 --2- 172.21.15.107:0/875370928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f4bedc620 0x564f4bee4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.033+0000 7f7addcb8640 1 --2- 172.21.15.107:0/875370928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bed1f50 0x564f4bee02a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4097fb0 tx=0x7f7ad4022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cda14976f94e022 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.033+0000 7f7ac1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.033+0000 7f7ac1ffb640 1 -- 172.21.15.107:0/875370928 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4020040 con 0x564f4bed1f50 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.033+0000 7f7ac1ffb640 1 -- 172.21.15.107:0/875370928 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f7ad4097c50 con 0x564f4bed1f50 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.033+0000 7f7ae3420740 1 -- 172.21.15.107:0/875370928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ad000ce20 con 0x564f4bed1f50 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.034+0000 7f7ac1ffb640 1 -- 172.21.15.107:0/875370928 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7ad4009de0 con 0x564f4bed1f50 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.034+0000 7f7ae3420740 10 client.16404 fetch_fsmap learned FSMap version 28 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.034+0000 7f7ae3420740 1 -- 172.21.15.107:0/875370928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564f4bf528a0 con 0x564f4bed1f50 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.034+0000 7f7ac1ffb640 1 -- 172.21.15.107:0/875370928 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ad000ce20 con 0x564f4bed1f50 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.034+0000 7f7ac1ffb640 1 -- 172.21.15.107:0/875370928 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f7ad409aa30 con 0x564f4bed1f50 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.034+0000 7f7ae3420740 10 client.16404 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.034+0000 7f7ae3420740 10 client.16404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.034+0000 7f7ae3420740 20 client.16404 populate_metadata read hostname 'smithi107' 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.034+0000 7f7ae3420740 1 --2- 172.21.15.107:0/875370928 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x564f4bf530a0 0x564f4bf554c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:49.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.034+0000 7f7ae3420740 4 client.16404 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:19:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.034+0000 7f7ae3420740 1 -- 172.21.15.107:0/875370928 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564f4be3df40 con 0x564f4bf530a0 2024-04-25T00:19:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.034+0000 7f7ade4b9640 1 --2- 172.21.15.107:0/875370928 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x564f4bf530a0 0x564f4bf554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.034+0000 7f7ade4b9640 1 --2- 172.21.15.107:0/875370928 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x564f4bf530a0 0x564f4bf554c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d50df6ebbb77758 server_cookie=b7321d821998d6ae in_seq=0 out_seq=0 2024-04-25T00:19:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.034+0000 7f7ac1ffb640 10 client.16404 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.035+0000 7f7ac1ffb640 1 -- 172.21.15.107:0/875370928 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564f4be3df40 con 0x564f4bf530a0 2024-04-25T00:19:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.035+0000 7f7ac1ffb640 10 client.16404 handle_command_reply: tid=0 2024-04-25T00:19:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.035+0000 7f7ae3420740 1 client.16404 shutdown 2024-04-25T00:19:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.035+0000 7f7ae3420740 1 -- 172.21.15.107:0/875370928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bed1f50 msgr2=0x564f4bee02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.035+0000 7f7ae3420740 1 --2- 172.21.15.107:0/875370928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bed1f50 0x564f4bee02a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4097fb0 tx=0x7f7ad4022000 comp rx=0 tx=0).stop 2024-04-25T00:19:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.035+0000 7f7ae3420740 1 -- 172.21.15.107:0/875370928 shutdown_connections 2024-04-25T00:19:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.035+0000 7f7ae3420740 1 --2- 172.21.15.107:0/875370928 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x564f4bf530a0 0x564f4bf554c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.035+0000 7f7ae3420740 1 --2- 172.21.15.107:0/875370928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564f4bedc620 0x564f4bee4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.035+0000 7f7ac1ffb640 0 client.16404 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:19:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.035+0000 7f7ae3420740 1 --2- 172.21.15.107:0/875370928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564f4bed1f50 0x564f4bee02a0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.035+0000 7f7ae3420740 1 --2- 172.21.15.107:0/875370928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564f4bed1630 0x564f4bedfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.035+0000 7f7ae3420740 1 -- 172.21.15.107:0/875370928 >> 172.21.15.107:0/875370928 conn(0x564f4becd950 msgr2=0x564f4beccaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:49.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.035+0000 7f7ae3420740 1 -- 172.21.15.107:0/875370928 shutdown_connections 2024-04-25T00:19:49.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.035+0000 7f7ae3420740 1 -- 172.21.15.107:0/875370928 wait complete. 2024-04-25T00:19:49.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.035+0000 7f7ae3420740 20 client.16404 trim_cache size 0 max 16384 2024-04-25T00:19:49.044 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:19:49.044 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:19:49.044 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-04-25T00:19:49.044 INFO:teuthology.orchestra.run.smithi107.stdout: "5902815d-6463-4f78-8252-be90a6e0cb33": { 2024-04-25T00:19:49.044 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-04-25T00:19:49.045 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "5902815d-6463-4f78-8252-be90a6e0cb33", 2024-04-25T00:19:49.045 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-04-25T00:19:49.045 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:19:49.045 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:19:49.045 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:19:49.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.035+0000 7f7ae071d640 1 -- 172.21.15.107:0/9087055 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f7ac40a2d00 msgr2=0x7f7ac40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:49.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.035+0000 7f7ae071d640 1 --2- 172.21.15.107:0/9087055 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f7ac40a2d00 0x7f7ac40a51c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0012c60 tx=0x7f7ad000c000 comp rx=0 tx=0).stop 2024-04-25T00:19:49.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.035+0000 7f7ae071d640 1 -- 172.21.15.107:0/9087055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ad8100600 msgr2=0x7f7ad8101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:49.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.035+0000 7f7ae071d640 1 --2- 172.21.15.107:0/9087055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ad8100600 0x7f7ad8101360 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400d950 tx=0x7f7ad400de10 comp rx=0 tx=0).stop 2024-04-25T00:19:49.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.036+0000 7f7ae071d640 1 -- 172.21.15.107:0/9087055 shutdown_connections 2024-04-25T00:19:49.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.036+0000 7f7ae071d640 1 --2- 172.21.15.107:0/9087055 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7ad8107720 0x7f7ad81018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.036+0000 7f7ae071d640 1 --2- 172.21.15.107:0/9087055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ad8100600 0x7f7ad8101360 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.036+0000 7f7ae071d640 1 --2- 172.21.15.107:0/9087055 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f7ac40a2d00 0x7f7ac40a51c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.036+0000 7f7ae071d640 1 --2- 172.21.15.107:0/9087055 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7ad80ffc40 0x7f7ad8100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:49.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.036+0000 7f7ae071d640 1 -- 172.21.15.107:0/9087055 >> 172.21.15.107:0/9087055 conn(0x7f7ad8076c00 msgr2=0x7f7ad80fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:49.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.036+0000 7f7ae071d640 1 -- 172.21.15.107:0/9087055 shutdown_connections 2024-04-25T00:19:49.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:19:49.036+0000 7f7ae071d640 1 -- 172.21.15.107:0/9087055 wait complete. 2024-04-25T00:19:49.047 INFO:tasks.cephfs.filesystem:scrub status for tag:5902815d-6463-4f78-8252-be90a6e0cb33 - {'path': '/', 'tag': '5902815d-6463-4f78-8252-be90a6e0cb33', 'options': 'recursive,force'} 2024-04-25T00:19:50.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:50 smithi195 ceph-mon[32833]: pgmap v104: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 193 op/s 2024-04-25T00:19:50.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:50 smithi195 ceph-mon[32833]: scrub queued for path: / 2024-04-25T00:19:50.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:50 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:50.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:50 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:19:50.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:50 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:50.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:50 smithi170 ceph-mon[32807]: pgmap v104: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 193 op/s 2024-04-25T00:19:50.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:50 smithi170 ceph-mon[32807]: scrub queued for path: / 2024-04-25T00:19:50.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:50 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:50.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:50 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:19:50.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:50 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:50.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:50 smithi107 ceph-mon[29153]: pgmap v104: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 193 op/s 2024-04-25T00:19:50.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:50 smithi107 ceph-mon[29153]: scrub queued for path: / 2024-04-25T00:19:50.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:50 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:50.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:50 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:19:50.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:50 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:19:52.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:52 smithi195 ceph-mon[32833]: pgmap v105: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 298 KiB/s wr, 63 op/s 2024-04-25T00:19:52.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:52 smithi170 ceph-mon[32807]: pgmap v105: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 298 KiB/s wr, 63 op/s 2024-04-25T00:19:52.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:52 smithi107 ceph-mon[29153]: pgmap v105: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 298 KiB/s wr, 63 op/s 2024-04-25T00:19:53.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:53 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:19:53.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:53 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:19:53.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:53 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:19:54.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:54 smithi195 ceph-mon[32833]: pgmap v106: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 300 KiB/s wr, 66 op/s 2024-04-25T00:19:54.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:54 smithi170 ceph-mon[32807]: pgmap v106: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 300 KiB/s wr, 66 op/s 2024-04-25T00:19:54.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:54 smithi107 ceph-mon[29153]: pgmap v106: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 300 KiB/s wr, 66 op/s 2024-04-25T00:19:56.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:56 smithi195 ceph-mon[32833]: pgmap v107: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.5 KiB/s wr, 7 op/s 2024-04-25T00:19:56.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:56 smithi170 ceph-mon[32807]: pgmap v107: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.5 KiB/s wr, 7 op/s 2024-04-25T00:19:56.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:56 smithi107 ceph-mon[29153]: pgmap v107: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.5 KiB/s wr, 7 op/s 2024-04-25T00:19:58.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:19:58 smithi195 ceph-mon[32833]: pgmap v108: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.5 KiB/s wr, 7 op/s 2024-04-25T00:19:58.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:19:58 smithi170 ceph-mon[32807]: pgmap v108: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.5 KiB/s wr, 7 op/s 2024-04-25T00:19:58.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:19:58 smithi107 ceph-mon[29153]: pgmap v108: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.5 KiB/s wr, 7 op/s 2024-04-25T00:20:00.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:00 smithi195 ceph-mon[32833]: pgmap v109: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.0 KiB/s wr, 9 op/s 2024-04-25T00:20:00.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:00 smithi195 ceph-mon[32833]: overall HEALTH_OK 2024-04-25T00:20:00.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:00 smithi170 ceph-mon[32807]: pgmap v109: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.0 KiB/s wr, 9 op/s 2024-04-25T00:20:00.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:00 smithi170 ceph-mon[32807]: overall HEALTH_OK 2024-04-25T00:20:00.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:00 smithi107 ceph-mon[29153]: pgmap v109: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.0 KiB/s wr, 9 op/s 2024-04-25T00:20:00.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:00 smithi107 ceph-mon[29153]: overall HEALTH_OK 2024-04-25T00:20:02.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:02 smithi195 ceph-mon[32833]: pgmap v110: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-25T00:20:02.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:02 smithi170 ceph-mon[32807]: pgmap v110: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-25T00:20:02.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:02 smithi107 ceph-mon[29153]: pgmap v110: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-25T00:20:04.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:04 smithi195 ceph-mon[32833]: pgmap v111: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-25T00:20:04.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:04 smithi170 ceph-mon[32807]: pgmap v111: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-25T00:20:04.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:04 smithi107 ceph-mon[29153]: pgmap v111: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-25T00:20:06.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:06 smithi195 ceph-mon[32833]: pgmap v112: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T00:20:06.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:06 smithi170 ceph-mon[32807]: pgmap v112: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T00:20:06.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:06 smithi107 ceph-mon[29153]: pgmap v112: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T00:20:08.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:08 smithi195 ceph-mon[32833]: pgmap v113: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T00:20:08.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:08 smithi170 ceph-mon[32807]: pgmap v113: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T00:20:08.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:08 smithi107 ceph-mon[29153]: pgmap v113: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-25T00:20:10.438 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:10 smithi170 ceph-mon[32807]: pgmap v114: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-25T00:20:10.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:10 smithi107 ceph-mon[29153]: pgmap v114: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-25T00:20:10.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:10 smithi195 ceph-mon[32833]: pgmap v114: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-25T00:20:12.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:12 smithi170 ceph-mon[32807]: pgmap v115: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T00:20:12.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:12 smithi107 ceph-mon[29153]: pgmap v115: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T00:20:12.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:12 smithi195 ceph-mon[32833]: pgmap v115: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T00:20:14.390 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:14 smithi195 ceph-mon[32833]: pgmap v116: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T00:20:14.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:14 smithi170 ceph-mon[32807]: pgmap v116: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T00:20:14.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:14 smithi107 ceph-mon[29153]: pgmap v116: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T00:20:16.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:16 smithi170 ceph-mon[32807]: pgmap v117: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:20:16.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:16 smithi107 ceph-mon[29153]: pgmap v117: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:20:16.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:16 smithi195 ceph-mon[32833]: pgmap v117: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:20:18.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:18 smithi170 ceph-mon[32807]: pgmap v118: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:20:18.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:18 smithi107 ceph-mon[29153]: pgmap v118: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:20:18.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:18 smithi195 ceph-mon[32833]: pgmap v118: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:20:19.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:20:19.049 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:20:19.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.164+0000 7f28d7ae1640 1 -- 172.21.15.107:0/2889686017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28d0100e60 msgr2=0x7f28d010dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.164+0000 7f28d7ae1640 1 --2- 172.21.15.107:0/2889686017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28d0100e60 0x7f28d010dbb0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009900 tx=0x7f28c002a410 comp rx=0 tx=0).stop 2024-04-25T00:20:19.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.164+0000 7f28d7ae1640 1 -- 172.21.15.107:0/2889686017 shutdown_connections 2024-04-25T00:20:19.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.164+0000 7f28d7ae1640 1 --2- 172.21.15.107:0/2889686017 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28d010e0f0 0x7f28d01104e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.164+0000 7f28d7ae1640 1 --2- 172.21.15.107:0/2889686017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28d0100e60 0x7f28d010dbb0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.164+0000 7f28d7ae1640 1 --2- 172.21.15.107:0/2889686017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28d0100540 0x7f28d0100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.164+0000 7f28d7ae1640 1 -- 172.21.15.107:0/2889686017 >> 172.21.15.107:0/2889686017 conn(0x7f28d0077740 msgr2=0x7f28d0077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.164+0000 7f28d7ae1640 1 -- 172.21.15.107:0/2889686017 shutdown_connections 2024-04-25T00:20:19.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.164+0000 7f28d7ae1640 1 -- 172.21.15.107:0/2889686017 wait complete. 2024-04-25T00:20:19.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.165+0000 7f28d7ae1640 1 Processor -- start 2024-04-25T00:20:19.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.165+0000 7f28d7ae1640 1 -- start start 2024-04-25T00:20:19.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.165+0000 7f28d7ae1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28d0100540 0x7f28d019a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.165+0000 7f28d7ae1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28d0100e60 0x7f28d019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.165+0000 7f28d7ae1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28d010e0f0 0x7f28d019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.165+0000 7f28d7ae1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f28d0113d20 con 0x7f28d010e0f0 2024-04-25T00:20:19.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.165+0000 7f28d7ae1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f28d0113ba0 con 0x7f28d0100540 2024-04-25T00:20:19.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.165+0000 7f28d7ae1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f28d0113ea0 con 0x7f28d0100e60 2024-04-25T00:20:19.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.165+0000 7f28d607e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28d010e0f0 0x7f28d019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.165+0000 7f28d607e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28d010e0f0 0x7f28d019eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38534/0 (socket says 172.21.15.107:38534) 2024-04-25T00:20:19.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.165+0000 7f28d607e640 1 -- 172.21.15.107:0/3497241569 learned_addr learned my addr 172.21.15.107:0/3497241569 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:19.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.165+0000 7f28d587d640 1 --2- 172.21.15.107:0/3497241569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28d0100540 0x7f28d019a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.165+0000 7f28d507c640 1 --2- 172.21.15.107:0/3497241569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28d0100e60 0x7f28d019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.165+0000 7f28d607e640 1 -- 172.21.15.107:0/3497241569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28d0100e60 msgr2=0x7f28d019a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.165+0000 7f28d607e640 1 --2- 172.21.15.107:0/3497241569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28d0100e60 0x7f28d019a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.165+0000 7f28d607e640 1 -- 172.21.15.107:0/3497241569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28d0100540 msgr2=0x7f28d019a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.165+0000 7f28d607e640 1 --2- 172.21.15.107:0/3497241569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28d0100540 0x7f28d019a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.165+0000 7f28d607e640 1 -- 172.21.15.107:0/3497241569 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d019f210 con 0x7f28d010e0f0 2024-04-25T00:20:19.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.165+0000 7f28d607e640 1 --2- 172.21.15.107:0/3497241569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28d010e0f0 0x7f28d019eae0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f28c800d720 tx=0x7f28c800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.166+0000 7f28c6ffd640 1 -- 172.21.15.107:0/3497241569 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c8017070 con 0x7f28d010e0f0 2024-04-25T00:20:19.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.166+0000 7f28d7ae1640 1 -- 172.21.15.107:0/3497241569 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d019f500 con 0x7f28d010e0f0 2024-04-25T00:20:19.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.166+0000 7f28c6ffd640 1 -- 172.21.15.107:0/3497241569 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28c800de30 con 0x7f28d010e0f0 2024-04-25T00:20:19.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.166+0000 7f28c6ffd640 1 -- 172.21.15.107:0/3497241569 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c8004d90 con 0x7f28d010e0f0 2024-04-25T00:20:19.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.166+0000 7f28d7ae1640 1 -- 172.21.15.107:0/3497241569 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d01a6d90 con 0x7f28d010e0f0 2024-04-25T00:20:19.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.168+0000 7f28c6ffd640 1 -- 172.21.15.107:0/3497241569 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f28c8004f30 con 0x7f28d010e0f0 2024-04-25T00:20:19.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.168+0000 7f28da7f3740 1 Processor -- start 2024-04-25T00:20:19.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.169+0000 7f28da7f3740 1 -- start start 2024-04-25T00:20:19.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.169+0000 7f28da7f3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec6518630 0x55cec6518a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.169+0000 7f28da7f3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cec6518f50 0x55cec65230e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.169+0000 7f28da7f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6523620 0x55cec6525a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.169+0000 7f28da7f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cec647d6b0 con 0x55cec6523620 2024-04-25T00:20:19.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.169+0000 7f28da7f3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cec6425880 con 0x55cec6518f50 2024-04-25T00:20:19.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.169+0000 7f28da7f3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cec6479670 con 0x55cec6518630 2024-04-25T00:20:19.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.169+0000 7f28c6ffd640 1 --2- 172.21.15.107:0/3497241569 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f28bc0a2d00 0x7f28bc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.169+0000 7f28d507c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6523620 0x55cec6525a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.169+0000 7f28d507c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6523620 0x55cec6525a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38548/0 (socket says 172.21.15.107:38548) 2024-04-25T00:20:19.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.169+0000 7f28d507c640 1 -- 172.21.15.107:0/3937311768 learned_addr learned my addr 172.21.15.107:0/3937311768 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:19.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.169+0000 7f28d587d640 1 --2- 172.21.15.107:0/3937311768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cec6518f50 0x55cec65230e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.169+0000 7f28d607e640 1 --2- 172.21.15.107:0/3497241569 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f28bc0a2d00 0x7f28bc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.169+0000 7f28d507c640 1 --2- 172.21.15.107:0/3937311768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec6518630 0x55cec6518a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.169+0000 7f28c6ffd640 1 -- 172.21.15.107:0/3497241569 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f28c80cf110 con 0x7f28d010e0f0 2024-04-25T00:20:19.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.169+0000 7f28d507c640 1 -- 172.21.15.107:0/3937311768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec6518630 msgr2=0x55cec6518a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.169+0000 7f28d507c640 1 --2- 172.21.15.107:0/3937311768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec6518630 0x55cec6518a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.169+0000 7f28d507c640 1 -- 172.21.15.107:0/3937311768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cec6518f50 msgr2=0x55cec65230e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.169+0000 7f28d507c640 1 --2- 172.21.15.107:0/3937311768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cec6518f50 0x55cec65230e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.169+0000 7f28d507c640 1 -- 172.21.15.107:0/3937311768 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cec625b7d0 con 0x55cec6523620 2024-04-25T00:20:19.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.169+0000 7f28d587d640 1 --2- 172.21.15.107:0/3937311768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cec6518f50 0x55cec65230e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:20:19.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.170+0000 7f28d507c640 1 --2- 172.21.15.107:0/3937311768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6523620 0x55cec6525a10 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f28c0031550 tx=0x7f28c0005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1a0dc0409392b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.170+0000 7f28d607e640 1 --2- 172.21.15.107:0/3497241569 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f28bc0a2d00 0x7f28bc0a51c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f28c80cc3a0 tx=0x7f28c806d220 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.170+0000 7f28bbfff640 1 -- 172.21.15.107:0/3937311768 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0035040 con 0x55cec6523620 2024-04-25T00:20:19.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.170+0000 7f28bbfff640 1 -- 172.21.15.107:0/3937311768 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28c0004030 con 0x55cec6523620 2024-04-25T00:20:19.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.170+0000 7f28bbfff640 1 -- 172.21.15.107:0/3937311768 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0003b10 con 0x55cec6523620 2024-04-25T00:20:19.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.171+0000 7f28da7f3740 1 -- 172.21.15.107:0/3937311768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6523620 msgr2=0x55cec6525a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.171+0000 7f28da7f3740 1 --2- 172.21.15.107:0/3937311768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6523620 0x55cec6525a10 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f28c0031550 tx=0x7f28c0005b50 comp rx=0 tx=0).stop 2024-04-25T00:20:19.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.171+0000 7f28da7f3740 1 -- 172.21.15.107:0/3937311768 shutdown_connections 2024-04-25T00:20:19.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.171+0000 7f28da7f3740 1 --2- 172.21.15.107:0/3937311768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6523620 0x55cec6525a10 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.171+0000 7f28da7f3740 1 --2- 172.21.15.107:0/3937311768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cec6518f50 0x55cec65230e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.171+0000 7f28da7f3740 1 --2- 172.21.15.107:0/3937311768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec6518630 0x55cec6518a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.171+0000 7f28da7f3740 1 -- 172.21.15.107:0/3937311768 >> 172.21.15.107:0/3937311768 conn(0x55cec6514950 msgr2=0x55cec6514d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.171+0000 7f28da7f3740 1 -- 172.21.15.107:0/3937311768 shutdown_connections 2024-04-25T00:20:19.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.171+0000 7f28da7f3740 1 -- 172.21.15.107:0/3937311768 wait complete. 2024-04-25T00:20:19.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.171+0000 7f28da7f3740 1 -- start start 2024-04-25T00:20:19.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.171+0000 7f28da7f3740 1 Processor -- start 2024-04-25T00:20:19.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.172+0000 7f28da7f3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cec6518630 0x55cec65205a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.172+0000 7f28da7f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6518f50 0x55cec6520ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.172+0000 7f28da7f3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec6523620 0x55cec6521020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.172+0000 7f28da7f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cec65263b0 con 0x55cec6518f50 2024-04-25T00:20:19.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.172+0000 7f28da7f3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cec6438f50 con 0x55cec6518630 2024-04-25T00:20:19.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.172+0000 7f28da7f3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cec6526530 con 0x55cec6523620 2024-04-25T00:20:19.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.172+0000 7f28da7f3740 10 client.0 _ll_register_callbacks cb 0x55cec6474000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:19.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.172+0000 7f28d587d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6518f50 0x55cec6520ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.172+0000 7f28d587d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6518f50 0x55cec6520ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38552/0 (socket says 172.21.15.107:38552) 2024-04-25T00:20:19.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.172+0000 7f28d587d640 1 -- 172.21.15.107:0/3667593026 learned_addr learned my addr 172.21.15.107:0/3667593026 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:19.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.172+0000 7f28d587d640 1 -- 172.21.15.107:0/3667593026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec6523620 msgr2=0x55cec6521020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.172+0000 7f28d587d640 1 --2- 172.21.15.107:0/3667593026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec6523620 0x55cec6521020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.172+0000 7f28d587d640 1 -- 172.21.15.107:0/3667593026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cec6518630 msgr2=0x55cec65205a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.172+0000 7f28d587d640 1 --2- 172.21.15.107:0/3667593026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cec6518630 0x55cec65205a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.172+0000 7f28d587d640 1 -- 172.21.15.107:0/3667593026 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cec61913f0 con 0x55cec6518f50 2024-04-25T00:20:19.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.172+0000 7f28d587d640 1 --2- 172.21.15.107:0/3667593026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6518f50 0x55cec6520ae0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f28cc004070 tx=0x7f28cc004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba78c4f9b3e9ede server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.172+0000 7f28b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:20:19.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.172+0000 7f28b97fa640 1 -- 172.21.15.107:0/3667593026 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc004d20 con 0x55cec6518f50 2024-04-25T00:20:19.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.172+0000 7f28b97fa640 1 -- 172.21.15.107:0/3667593026 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f28cc013a40 con 0x55cec6518f50 2024-04-25T00:20:19.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.172+0000 7f28da7f3740 1 -- 172.21.15.107:0/3667593026 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28c0004030 con 0x55cec6518f50 2024-04-25T00:20:19.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.172+0000 7f28b97fa640 1 -- 172.21.15.107:0/3667593026 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f28cc024050 con 0x55cec6518f50 2024-04-25T00:20:19.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.172+0000 7f28da7f3740 10 client.16416 fetch_fsmap learned FSMap version 28 2024-04-25T00:20:19.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.172+0000 7f28da7f3740 1 -- 172.21.15.107:0/3667593026 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cec61510f0 con 0x55cec6518f50 2024-04-25T00:20:19.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.173+0000 7f28b97fa640 1 -- 172.21.15.107:0/3667593026 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28c0004030 con 0x55cec6518f50 2024-04-25T00:20:19.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.173+0000 7f28b97fa640 1 -- 172.21.15.107:0/3667593026 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f28cc01a030 con 0x55cec6518f50 2024-04-25T00:20:19.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.173+0000 7f28da7f3740 10 client.16416 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:20:19.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.173+0000 7f28da7f3740 10 client.16416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:20:19.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.173+0000 7f28da7f3740 20 client.16416 populate_metadata read hostname 'smithi107' 2024-04-25T00:20:19.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.173+0000 7f28da7f3740 1 --2- 172.21.15.107:0/3667593026 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55cec65339b0 0x55cec6535e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.173+0000 7f28da7f3740 4 client.16416 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:20:19.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.173+0000 7f28da7f3740 1 -- 172.21.15.107:0/3667593026 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cec648abf0 con 0x55cec65339b0 2024-04-25T00:20:19.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.173+0000 7f28d507c640 1 --2- 172.21.15.107:0/3667593026 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55cec65339b0 0x55cec6535e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.173+0000 7f28b97fa640 10 client.16416 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:19.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.173+0000 7f28d507c640 1 --2- 172.21.15.107:0/3667593026 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55cec65339b0 0x55cec6535e20 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db10b9076187c3dd server_cookie=5ee58ad56466ca27 in_seq=0 out_seq=0 2024-04-25T00:20:19.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.174+0000 7f28b97fa640 1 -- 172.21.15.107:0/3667593026 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cec648abf0 con 0x55cec65339b0 2024-04-25T00:20:19.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.174+0000 7f28b97fa640 10 client.16416 handle_command_reply: tid=0 2024-04-25T00:20:19.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.174+0000 7f28da7f3740 1 client.16416 shutdown 2024-04-25T00:20:19.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.174+0000 7f28da7f3740 1 -- 172.21.15.107:0/3667593026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6518f50 msgr2=0x55cec6520ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.174+0000 7f28da7f3740 1 --2- 172.21.15.107:0/3667593026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6518f50 0x55cec6520ae0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f28cc004070 tx=0x7f28cc004530 comp rx=0 tx=0).stop 2024-04-25T00:20:19.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.174+0000 7f28da7f3740 1 -- 172.21.15.107:0/3667593026 shutdown_connections 2024-04-25T00:20:19.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.174+0000 7f28da7f3740 1 --2- 172.21.15.107:0/3667593026 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55cec65339b0 0x55cec6535e20 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.174+0000 7f28da7f3740 1 --2- 172.21.15.107:0/3667593026 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec6523620 0x55cec6521020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.174+0000 7f28da7f3740 1 --2- 172.21.15.107:0/3667593026 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6518f50 0x55cec6520ae0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.174+0000 7f28b97fa640 0 client.16416 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:19.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.174+0000 7f28da7f3740 1 --2- 172.21.15.107:0/3667593026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cec6518630 0x55cec65205a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.174+0000 7f28da7f3740 1 -- 172.21.15.107:0/3667593026 >> 172.21.15.107:0/3667593026 conn(0x55cec6514950 msgr2=0x55cec651a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.174+0000 7f28da7f3740 1 -- 172.21.15.107:0/3667593026 shutdown_connections 2024-04-25T00:20:19.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.174+0000 7f28da7f3740 1 -- 172.21.15.107:0/3667593026 wait complete. 2024-04-25T00:20:19.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.175+0000 7f28da7f3740 20 client.16416 trim_cache size 0 max 16384 2024-04-25T00:20:19.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.192+0000 7f28da7f3740 1 Processor -- start 2024-04-25T00:20:19.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.192+0000 7f28da7f3740 1 -- start start 2024-04-25T00:20:19.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.192+0000 7f28da7f3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cec6518630 0x55cec652f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.192+0000 7f28da7f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6518f50 0x55cec651ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.192+0000 7f28da7f3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec6523620 0x55cec653b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.192+0000 7f28da7f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f28d01b3580 con 0x55cec6518f50 2024-04-25T00:20:19.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.192+0000 7f28da7f3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f28d01b0d90 con 0x55cec6518630 2024-04-25T00:20:19.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.192+0000 7f28da7f3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f28d01aebe0 con 0x55cec6523620 2024-04-25T00:20:19.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.192+0000 7f28d587d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6518f50 0x55cec651ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.192+0000 7f28d587d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6518f50 0x55cec651ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38566/0 (socket says 172.21.15.107:38566) 2024-04-25T00:20:19.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.192+0000 7f28d587d640 1 -- 172.21.15.107:0/1625445500 learned_addr learned my addr 172.21.15.107:0/1625445500 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:19.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.192+0000 7f28d507c640 1 --2- 172.21.15.107:0/1625445500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec6523620 0x55cec653b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.192+0000 7f28d587d640 1 -- 172.21.15.107:0/1625445500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec6523620 msgr2=0x55cec653b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.192+0000 7f28d587d640 1 --2- 172.21.15.107:0/1625445500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec6523620 0x55cec653b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.192+0000 7f28d587d640 1 -- 172.21.15.107:0/1625445500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cec6518630 msgr2=0x55cec652f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.192+0000 7f28d587d640 1 --2- 172.21.15.107:0/1625445500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cec6518630 0x55cec652f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.192+0000 7f28d587d640 1 -- 172.21.15.107:0/1625445500 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cec6518d80 con 0x55cec6518f50 2024-04-25T00:20:19.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.192+0000 7f28d507c640 1 --2- 172.21.15.107:0/1625445500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec6523620 0x55cec653b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:20:19.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.193+0000 7f28d587d640 1 --2- 172.21.15.107:0/1625445500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6518f50 0x55cec651ae30 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f28cc011430 tx=0x7f28cc009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf825a11b005ace server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.193+0000 7f28bb7fe640 1 -- 172.21.15.107:0/1625445500 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc02e020 con 0x55cec6518f50 2024-04-25T00:20:19.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.193+0000 7f28bb7fe640 1 -- 172.21.15.107:0/1625445500 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28cc003dd0 con 0x55cec6518f50 2024-04-25T00:20:19.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.193+0000 7f28bb7fe640 1 -- 172.21.15.107:0/1625445500 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc01b650 con 0x55cec6518f50 2024-04-25T00:20:19.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.193+0000 7f28da7f3740 1 -- 172.21.15.107:0/1625445500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6518f50 msgr2=0x55cec651ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.193+0000 7f28da7f3740 1 --2- 172.21.15.107:0/1625445500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6518f50 0x55cec651ae30 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f28cc011430 tx=0x7f28cc009770 comp rx=0 tx=0).stop 2024-04-25T00:20:19.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.193+0000 7f28da7f3740 1 -- 172.21.15.107:0/1625445500 shutdown_connections 2024-04-25T00:20:19.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.193+0000 7f28da7f3740 1 --2- 172.21.15.107:0/1625445500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec6523620 0x55cec653b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.193+0000 7f28da7f3740 1 --2- 172.21.15.107:0/1625445500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6518f50 0x55cec651ae30 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.193+0000 7f28da7f3740 1 --2- 172.21.15.107:0/1625445500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cec6518630 0x55cec652f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.193+0000 7f28da7f3740 1 -- 172.21.15.107:0/1625445500 >> 172.21.15.107:0/1625445500 conn(0x55cec6514950 msgr2=0x55cec6520df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.193+0000 7f28da7f3740 1 -- 172.21.15.107:0/1625445500 shutdown_connections 2024-04-25T00:20:19.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.193+0000 7f28da7f3740 1 -- 172.21.15.107:0/1625445500 wait complete. 2024-04-25T00:20:19.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.194+0000 7f28da7f3740 1 -- start start 2024-04-25T00:20:19.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.194+0000 7f28da7f3740 1 Processor -- start 2024-04-25T00:20:19.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.194+0000 7f28da7f3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec6518630 0x55cec6527520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.194+0000 7f28da7f3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cec6518f50 0x55cec6527a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.194+0000 7f28da7f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6523620 0x55cec6598f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.194+0000 7f28da7f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f28d01af250 con 0x55cec6523620 2024-04-25T00:20:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.194+0000 7f28da7f3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f28d01b06c0 con 0x55cec6518f50 2024-04-25T00:20:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.194+0000 7f28da7f3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f28d01af920 con 0x55cec6518630 2024-04-25T00:20:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.194+0000 7f28da7f3740 10 client.0 _ll_register_callbacks cb 0x7f28cc009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.194+0000 7f28d507c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6523620 0x55cec6598f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.194+0000 7f28d507c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6523620 0x55cec6598f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38572/0 (socket says 172.21.15.107:38572) 2024-04-25T00:20:19.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.194+0000 7f28d507c640 1 -- 172.21.15.107:0/3159272540 learned_addr learned my addr 172.21.15.107:0/3159272540 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:19.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.195+0000 7f28d507c640 1 -- 172.21.15.107:0/3159272540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec6518630 msgr2=0x55cec6527520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.195+0000 7f28d507c640 1 --2- 172.21.15.107:0/3159272540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec6518630 0x55cec6527520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.195+0000 7f28d507c640 1 -- 172.21.15.107:0/3159272540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cec6518f50 msgr2=0x55cec6527a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.195+0000 7f28d507c640 1 --2- 172.21.15.107:0/3159272540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cec6518f50 0x55cec6527a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.195+0000 7f28d507c640 1 -- 172.21.15.107:0/3159272540 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cec6528330 con 0x55cec6523620 2024-04-25T00:20:19.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.195+0000 7f28d507c640 1 --2- 172.21.15.107:0/3159272540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6523620 0x55cec6598f60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f28c0003700 tx=0x7f28c0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e79a7202fd50e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.195+0000 7f28b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:20:19.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.195+0000 7f28b97fa640 1 -- 172.21.15.107:0/3159272540 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0035040 con 0x55cec6523620 2024-04-25T00:20:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.195+0000 7f28b97fa640 1 -- 172.21.15.107:0/3159272540 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f28c0050b50 con 0x55cec6523620 2024-04-25T00:20:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.195+0000 7f28da7f3740 1 -- 172.21.15.107:0/3159272540 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28cc003dd0 con 0x55cec6523620 2024-04-25T00:20:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.195+0000 7f28b97fa640 1 -- 172.21.15.107:0/3159272540 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f28c0050d50 con 0x55cec6523620 2024-04-25T00:20:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.195+0000 7f28da7f3740 10 client.16422 fetch_fsmap learned FSMap version 28 2024-04-25T00:20:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.195+0000 7f28da7f3740 1 -- 172.21.15.107:0/3159272540 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cec659a610 con 0x55cec6523620 2024-04-25T00:20:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.195+0000 7f28b97fa640 1 -- 172.21.15.107:0/3159272540 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28cc003dd0 con 0x55cec6523620 2024-04-25T00:20:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.195+0000 7f28b97fa640 1 -- 172.21.15.107:0/3159272540 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f28c003a040 con 0x55cec6523620 2024-04-25T00:20:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.195+0000 7f28da7f3740 10 client.16422 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:20:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.195+0000 7f28da7f3740 10 client.16422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:20:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.195+0000 7f28da7f3740 20 client.16422 populate_metadata read hostname 'smithi107' 2024-04-25T00:20:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.195+0000 7f28da7f3740 1 --2- 172.21.15.107:0/3159272540 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55cec659a9f0 0x55cec659ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.195+0000 7f28da7f3740 4 client.16422 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:20:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.195+0000 7f28da7f3740 1 -- 172.21.15.107:0/3159272540 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cec6484f40 con 0x55cec659a9f0 2024-04-25T00:20:19.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.196+0000 7f28d587d640 1 --2- 172.21.15.107:0/3159272540 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55cec659a9f0 0x55cec659ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.196+0000 7f28d587d640 1 --2- 172.21.15.107:0/3159272540 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55cec659a9f0 0x55cec659ce10 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f12a3b26527d9624 server_cookie=c19c4f1973abff4f in_seq=0 out_seq=0 2024-04-25T00:20:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.196+0000 7f28b97fa640 10 client.16422 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.196+0000 7f28b97fa640 1 -- 172.21.15.107:0/3159272540 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cec6484f40 con 0x55cec659a9f0 2024-04-25T00:20:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.196+0000 7f28b97fa640 10 client.16422 handle_command_reply: tid=0 2024-04-25T00:20:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.196+0000 7f28da7f3740 1 client.16422 shutdown 2024-04-25T00:20:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.196+0000 7f28da7f3740 1 -- 172.21.15.107:0/3159272540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6523620 msgr2=0x55cec6598f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.196+0000 7f28da7f3740 1 --2- 172.21.15.107:0/3159272540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6523620 0x55cec6598f60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f28c0003700 tx=0x7f28c0005fd0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.196+0000 7f28da7f3740 1 -- 172.21.15.107:0/3159272540 shutdown_connections 2024-04-25T00:20:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.196+0000 7f28da7f3740 1 --2- 172.21.15.107:0/3159272540 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55cec659a9f0 0x55cec659ce10 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.196+0000 7f28da7f3740 1 --2- 172.21.15.107:0/3159272540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cec6523620 0x55cec6598f60 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.196+0000 7f28da7f3740 1 --2- 172.21.15.107:0/3159272540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cec6518f50 0x55cec6527a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.196+0000 7f28b97fa640 0 client.16422 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:19.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.196+0000 7f28da7f3740 1 --2- 172.21.15.107:0/3159272540 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cec6518630 0x55cec6527520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.196+0000 7f28da7f3740 1 -- 172.21.15.107:0/3159272540 >> 172.21.15.107:0/3159272540 conn(0x55cec6514950 msgr2=0x55cec6513260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.197+0000 7f28da7f3740 1 -- 172.21.15.107:0/3159272540 shutdown_connections 2024-04-25T00:20:19.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.197+0000 7f28da7f3740 1 -- 172.21.15.107:0/3159272540 wait complete. 2024-04-25T00:20:19.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.197+0000 7f28da7f3740 20 client.16422 trim_cache size 0 max 16384 2024-04-25T00:20:19.207 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:20:19.207 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-04-25T00:20:19.207 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-04-25T00:20:19.207 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:20:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.197+0000 7f28d7ae1640 1 -- 172.21.15.107:0/3497241569 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f28bc0a2d00 msgr2=0x7f28bc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.197+0000 7f28d7ae1640 1 --2- 172.21.15.107:0/3497241569 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f28bc0a2d00 0x7f28bc0a51c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f28c80cc3a0 tx=0x7f28c806d220 comp rx=0 tx=0).stop 2024-04-25T00:20:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.197+0000 7f28d7ae1640 1 -- 172.21.15.107:0/3497241569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28d010e0f0 msgr2=0x7f28d019eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.197+0000 7f28d7ae1640 1 --2- 172.21.15.107:0/3497241569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28d010e0f0 0x7f28d019eae0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f28c800d720 tx=0x7f28c800dbe0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.197+0000 7f28d7ae1640 1 -- 172.21.15.107:0/3497241569 shutdown_connections 2024-04-25T00:20:19.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.197+0000 7f28d7ae1640 1 --2- 172.21.15.107:0/3497241569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28d010e0f0 0x7f28d019eae0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.197+0000 7f28d7ae1640 1 --2- 172.21.15.107:0/3497241569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28d0100e60 0x7f28d019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.197+0000 7f28d7ae1640 1 --2- 172.21.15.107:0/3497241569 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f28bc0a2d00 0x7f28bc0a51c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.197+0000 7f28d7ae1640 1 --2- 172.21.15.107:0/3497241569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28d0100540 0x7f28d019a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.197+0000 7f28d7ae1640 1 -- 172.21.15.107:0/3497241569 >> 172.21.15.107:0/3497241569 conn(0x7f28d0077740 msgr2=0x7f28d010f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.197+0000 7f28d7ae1640 1 -- 172.21.15.107:0/3497241569 shutdown_connections 2024-04-25T00:20:19.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.197+0000 7f28d7ae1640 1 -- 172.21.15.107:0/3497241569 wait complete. 2024-04-25T00:20:19.212 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:20:19.212 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:20:19.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.310+0000 7f7c14d02640 1 -- 172.21.15.107:0/3516751986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c101066b0 msgr2=0x7f7c10106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.310+0000 7f7c14d02640 1 --2- 172.21.15.107:0/3516751986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c101066b0 0x7f7c10106a90 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7c08009cd0 tx=0x7f7c0802a8e0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.310+0000 7f7c14d02640 1 -- 172.21.15.107:0/3516751986 shutdown_connections 2024-04-25T00:20:19.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.310+0000 7f7c14d02640 1 --2- 172.21.15.107:0/3516751986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c10100fd0 0x7f7c1010d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.310+0000 7f7c14d02640 1 --2- 172.21.15.107:0/3516751986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7c10100610 0x7f7c10100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.310+0000 7f7c14d02640 1 --2- 172.21.15.107:0/3516751986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c101066b0 0x7f7c10106a90 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.310+0000 7f7c14d02640 1 -- 172.21.15.107:0/3516751986 >> 172.21.15.107:0/3516751986 conn(0x7f7c100769c0 msgr2=0x7f7c10076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.310+0000 7f7c14d02640 1 -- 172.21.15.107:0/3516751986 shutdown_connections 2024-04-25T00:20:19.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.310+0000 7f7c14d02640 1 -- 172.21.15.107:0/3516751986 wait complete. 2024-04-25T00:20:19.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.310+0000 7f7c14d02640 1 Processor -- start 2024-04-25T00:20:19.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.311+0000 7f7c14d02640 1 -- start start 2024-04-25T00:20:19.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.311+0000 7f7c14d02640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7c10100610 0x7f7c10198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.311+0000 7f7c14d02640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c10100fd0 0x7f7c10199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.311+0000 7f7c14d02640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c101066b0 0x7f7c101a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.311+0000 7f7c14d02640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7c10110a30 con 0x7f7c101066b0 2024-04-25T00:20:19.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.311+0000 7f7c14d02640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7c101108b0 con 0x7f7c10100fd0 2024-04-25T00:20:19.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.311+0000 7f7c14d02640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7c10110bb0 con 0x7f7c10100610 2024-04-25T00:20:19.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.311+0000 7f7c0ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c101066b0 0x7f7c101a3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.311+0000 7f7c0ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c101066b0 0x7f7c101a3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38590/0 (socket says 172.21.15.107:38590) 2024-04-25T00:20:19.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.311+0000 7f7c0ed9d640 1 -- 172.21.15.107:0/418194563 learned_addr learned my addr 172.21.15.107:0/418194563 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:19.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.311+0000 7f7c0e59c640 1 --2- 172.21.15.107:0/418194563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7c10100610 0x7f7c10198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.311+0000 7f7c0dd9b640 1 --2- 172.21.15.107:0/418194563 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c10100fd0 0x7f7c10199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.311+0000 7f7c0ed9d640 1 -- 172.21.15.107:0/418194563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7c10100610 msgr2=0x7f7c10198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.311+0000 7f7c0ed9d640 1 --2- 172.21.15.107:0/418194563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7c10100610 0x7f7c10198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.311+0000 7f7c0ed9d640 1 -- 172.21.15.107:0/418194563 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c10100fd0 msgr2=0x7f7c10199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.311+0000 7f7c0ed9d640 1 --2- 172.21.15.107:0/418194563 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c10100fd0 0x7f7c10199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.311+0000 7f7c0ed9d640 1 -- 172.21.15.107:0/418194563 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c101a43d0 con 0x7f7c101066b0 2024-04-25T00:20:19.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.311+0000 7f7c0dd9b640 1 --2- 172.21.15.107:0/418194563 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c10100fd0 0x7f7c10199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:20:19.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.311+0000 7f7c0ed9d640 1 --2- 172.21.15.107:0/418194563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c101066b0 0x7f7c101a3e90 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000e970 tx=0x7f7c0000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.311+0000 7f7c077fe640 1 -- 172.21.15.107:0/418194563 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0000ccc0 con 0x7f7c101066b0 2024-04-25T00:20:19.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.311+0000 7f7c14d02640 1 -- 172.21.15.107:0/418194563 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c101a45a0 con 0x7f7c101066b0 2024-04-25T00:20:19.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.311+0000 7f7c077fe640 1 -- 172.21.15.107:0/418194563 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c0000ce60 con 0x7f7c101066b0 2024-04-25T00:20:19.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.311+0000 7f7c077fe640 1 -- 172.21.15.107:0/418194563 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00010490 con 0x7f7c101066b0 2024-04-25T00:20:19.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.312+0000 7f7c14d02640 1 -- 172.21.15.107:0/418194563 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c101a4af0 con 0x7f7c101066b0 2024-04-25T00:20:19.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.312+0000 7f7c14d02640 1 -- 172.21.15.107:0/418194563 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c10101d50 con 0x7f7c101066b0 2024-04-25T00:20:19.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.314+0000 7f7c077fe640 1 -- 172.21.15.107:0/418194563 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7c00005000 con 0x7f7c101066b0 2024-04-25T00:20:19.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.314+0000 7f7c077fe640 1 --2- 172.21.15.107:0/418194563 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f7bfc0a2d00 0x7f7bfc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.314+0000 7f7c0e59c640 1 --2- 172.21.15.107:0/418194563 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f7bfc0a2d00 0x7f7bfc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.314+0000 7f7c077fe640 1 -- 172.21.15.107:0/418194563 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f7c000cfef0 con 0x7f7c101066b0 2024-04-25T00:20:19.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.315+0000 7f7c0e59c640 1 --2- 172.21.15.107:0/418194563 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f7bfc0a2d00 0x7f7bfc0a51c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7c08006010 tx=0x7f7c080093f0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.318+0000 7f7c077fe640 1 -- 172.21.15.107:0/418194563 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7c00053740 con 0x7f7c101066b0 2024-04-25T00:20:19.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.532+0000 7f7c14d02640 1 -- 172.21.15.107:0/418194563 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c10108b80 con 0x7f7c101066b0 2024-04-25T00:20:19.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.533+0000 7f7c077fe640 1 -- 172.21.15.107:0/418194563 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11906 (secure 0 0 0) 0x7f7c0009c7a0 con 0x7f7c101066b0 2024-04-25T00:20:19.533 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:20:19.534 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:18:38.933541+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24493,24521]},"id":1}]} 2024-04-25T00:20:19.534 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 28 2024-04-25T00:20:19.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.535+0000 7f7c14d02640 1 -- 172.21.15.107:0/418194563 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f7bfc0a2d00 msgr2=0x7f7bfc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.535+0000 7f7c14d02640 1 --2- 172.21.15.107:0/418194563 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f7bfc0a2d00 0x7f7bfc0a51c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7c08006010 tx=0x7f7c080093f0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.535+0000 7f7c14d02640 1 -- 172.21.15.107:0/418194563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c101066b0 msgr2=0x7f7c101a3e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.535+0000 7f7c14d02640 1 --2- 172.21.15.107:0/418194563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c101066b0 0x7f7c101a3e90 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000e970 tx=0x7f7c0000ed70 comp rx=0 tx=0).stop 2024-04-25T00:20:19.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.535+0000 7f7c14d02640 1 -- 172.21.15.107:0/418194563 shutdown_connections 2024-04-25T00:20:19.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.535+0000 7f7c14d02640 1 --2- 172.21.15.107:0/418194563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c101066b0 0x7f7c101a3e90 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.535+0000 7f7c14d02640 1 --2- 172.21.15.107:0/418194563 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f7bfc0a2d00 0x7f7bfc0a51c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.535+0000 7f7c14d02640 1 --2- 172.21.15.107:0/418194563 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c10100fd0 0x7f7c10199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.535+0000 7f7c14d02640 1 --2- 172.21.15.107:0/418194563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7c10100610 0x7f7c10198e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.535+0000 7f7c14d02640 1 -- 172.21.15.107:0/418194563 >> 172.21.15.107:0/418194563 conn(0x7f7c100769c0 msgr2=0x7f7c100fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.535+0000 7f7c14d02640 1 -- 172.21.15.107:0/418194563 shutdown_connections 2024-04-25T00:20:19.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.535+0000 7f7c14d02640 1 -- 172.21.15.107:0/418194563 wait complete. 2024-04-25T00:20:19.546 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:20:19.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.646+0000 7feea049a640 1 -- 172.21.15.107:0/3305232552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee98100fe0 msgr2=0x7fee9810d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.646+0000 7feea049a640 1 --2- 172.21.15.107:0/3305232552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee98100fe0 0x7fee9810d4f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fee90009cd0 tx=0x7fee9002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.647+0000 7feea049a640 1 -- 172.21.15.107:0/3305232552 shutdown_connections 2024-04-25T00:20:19.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.647+0000 7feea049a640 1 --2- 172.21.15.107:0/3305232552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee98100fe0 0x7fee9810d4f0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.647+0000 7feea049a640 1 --2- 172.21.15.107:0/3305232552 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee98100620 0x7fee98100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.647+0000 7feea049a640 1 --2- 172.21.15.107:0/3305232552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee98106630 0x7fee98106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.647+0000 7feea049a640 1 -- 172.21.15.107:0/3305232552 >> 172.21.15.107:0/3305232552 conn(0x7fee980776a0 msgr2=0x7fee98077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.647+0000 7feea049a640 1 -- 172.21.15.107:0/3305232552 shutdown_connections 2024-04-25T00:20:19.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.647+0000 7feea049a640 1 -- 172.21.15.107:0/3305232552 wait complete. 2024-04-25T00:20:19.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.647+0000 7feea049a640 1 Processor -- start 2024-04-25T00:20:19.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.647+0000 7feea049a640 1 -- start start 2024-04-25T00:20:19.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.647+0000 7feea049a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee98100620 0x7fee98198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.647+0000 7feea049a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee98100fe0 0x7fee98199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.647+0000 7feea049a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee98106630 0x7fee981a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.647+0000 7feea049a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fee98110a40 con 0x7fee98106630 2024-04-25T00:20:19.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.647+0000 7feea049a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fee981108c0 con 0x7fee98100fe0 2024-04-25T00:20:19.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.647+0000 7feea049a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fee98110bc0 con 0x7fee98100620 2024-04-25T00:20:19.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.648+0000 7fee9ea37640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee98106630 0x7fee981a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.648+0000 7fee9ea37640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee98106630 0x7fee981a3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39846/0 (socket says 172.21.15.107:39846) 2024-04-25T00:20:19.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.648+0000 7fee9ea37640 1 -- 172.21.15.107:0/1825017466 learned_addr learned my addr 172.21.15.107:0/1825017466 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:19.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.648+0000 7fee9e236640 1 --2- 172.21.15.107:0/1825017466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee98100620 0x7fee98198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.648+0000 7fee9da35640 1 --2- 172.21.15.107:0/1825017466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee98100fe0 0x7fee98199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.648+0000 7fee9ea37640 1 -- 172.21.15.107:0/1825017466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee98100620 msgr2=0x7fee98198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.648+0000 7fee9ea37640 1 --2- 172.21.15.107:0/1825017466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee98100620 0x7fee98198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.648+0000 7fee9ea37640 1 -- 172.21.15.107:0/1825017466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee98100fe0 msgr2=0x7fee98199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.648+0000 7fee9ea37640 1 --2- 172.21.15.107:0/1825017466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee98100fe0 0x7fee98199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.648+0000 7fee9ea37640 1 -- 172.21.15.107:0/1825017466 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee981a43c0 con 0x7fee98106630 2024-04-25T00:20:19.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.648+0000 7fee9ea37640 1 --2- 172.21.15.107:0/1825017466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee98106630 0x7fee981a3e80 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fee90005cc0 tx=0x7fee90003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.648+0000 7fee8b7fe640 1 -- 172.21.15.107:0/1825017466 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee90004040 con 0x7fee98106630 2024-04-25T00:20:19.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.648+0000 7fee8b7fe640 1 -- 172.21.15.107:0/1825017466 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee900046c0 con 0x7fee98106630 2024-04-25T00:20:19.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.648+0000 7feea049a640 1 -- 172.21.15.107:0/1825017466 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee981a4590 con 0x7fee98106630 2024-04-25T00:20:19.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.648+0000 7fee8b7fe640 1 -- 172.21.15.107:0/1825017466 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee90046480 con 0x7fee98106630 2024-04-25T00:20:19.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.648+0000 7feea049a640 1 -- 172.21.15.107:0/1825017466 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee981a4a50 con 0x7fee98106630 2024-04-25T00:20:19.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.650+0000 7fee8b7fe640 1 -- 172.21.15.107:0/1825017466 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fee900305a0 con 0x7fee98106630 2024-04-25T00:20:19.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.651+0000 7feea31a1740 1 Processor -- start 2024-04-25T00:20:19.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.651+0000 7feea31a1740 1 -- start start 2024-04-25T00:20:19.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.651+0000 7feea31a1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a54135480 0x558a54135860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.651+0000 7feea31a1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54135da0 0x558a54144440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.651+0000 7feea31a1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a54144980 0x558a54146d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.651+0000 7feea31a1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558a5409a670 con 0x558a54135da0 2024-04-25T00:20:19.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.651+0000 7feea31a1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558a54042970 con 0x558a54144980 2024-04-25T00:20:19.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.651+0000 7feea31a1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558a54096630 con 0x558a54135480 2024-04-25T00:20:19.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.651+0000 7fee8b7fe640 1 --2- 172.21.15.107:0/1825017466 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fee840a2d00 0x7fee840a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.651+0000 7fee9e236640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54135da0 0x558a54144440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.651+0000 7fee9e236640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54135da0 0x558a54144440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39856/0 (socket says 172.21.15.107:39856) 2024-04-25T00:20:19.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.651+0000 7fee9e236640 1 -- 172.21.15.107:0/1684877795 learned_addr learned my addr 172.21.15.107:0/1684877795 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:19.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.651+0000 7fee9ea37640 1 --2- 172.21.15.107:0/1825017466 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fee840a2d00 0x7fee840a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.651+0000 7fee9da35640 1 --2- 172.21.15.107:0/1684877795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a54135480 0x558a54135860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.651+0000 7fee8b7fe640 1 -- 172.21.15.107:0/1825017466 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fee9002e030 con 0x7fee98106630 2024-04-25T00:20:19.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.651+0000 7fee9da35640 1 --2- 172.21.15.107:0/1684877795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a54144980 0x558a54146d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.651+0000 7fee9e236640 1 -- 172.21.15.107:0/1684877795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a54135480 msgr2=0x558a54135860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.651+0000 7fee9e236640 1 --2- 172.21.15.107:0/1684877795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a54135480 0x558a54135860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.651+0000 7fee9e236640 1 -- 172.21.15.107:0/1684877795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a54144980 msgr2=0x558a54146d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.651+0000 7fee9e236640 1 --2- 172.21.15.107:0/1684877795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a54144980 0x558a54146d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.651+0000 7fee9e236640 1 -- 172.21.15.107:0/1684877795 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a54066400 con 0x558a54135da0 2024-04-25T00:20:19.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.651+0000 7fee9da35640 1 --2- 172.21.15.107:0/1684877795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a54144980 0x558a54146d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:20:19.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.652+0000 7fee9ea37640 1 --2- 172.21.15.107:0/1825017466 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fee840a2d00 0x7fee840a51c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fee900ee470 tx=0x7fee90084040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.652+0000 7fee9e236640 1 --2- 172.21.15.107:0/1684877795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54135da0 0x558a54144440 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00aba0 tx=0x7fee8c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8c32c83c83534de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.652+0000 7fee88bfe640 1 -- 172.21.15.107:0/1684877795 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c01d070 con 0x558a54135da0 2024-04-25T00:20:19.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.652+0000 7fee88bfe640 1 -- 172.21.15.107:0/1684877795 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee8c018b20 con 0x558a54135da0 2024-04-25T00:20:19.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.652+0000 7fee88bfe640 1 -- 172.21.15.107:0/1684877795 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c010b60 con 0x558a54135da0 2024-04-25T00:20:19.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.653+0000 7feea31a1740 1 -- 172.21.15.107:0/1684877795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54135da0 msgr2=0x558a54144440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.653+0000 7feea31a1740 1 --2- 172.21.15.107:0/1684877795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54135da0 0x558a54144440 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00aba0 tx=0x7fee8c00f790 comp rx=0 tx=0).stop 2024-04-25T00:20:19.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.653+0000 7feea31a1740 1 -- 172.21.15.107:0/1684877795 shutdown_connections 2024-04-25T00:20:19.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.653+0000 7feea31a1740 1 --2- 172.21.15.107:0/1684877795 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a54144980 0x558a54146d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.653+0000 7feea31a1740 1 --2- 172.21.15.107:0/1684877795 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54135da0 0x558a54144440 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.653+0000 7feea31a1740 1 --2- 172.21.15.107:0/1684877795 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a54135480 0x558a54135860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.653+0000 7feea31a1740 1 -- 172.21.15.107:0/1684877795 >> 172.21.15.107:0/1684877795 conn(0x558a541318a0 msgr2=0x558a54131c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.653+0000 7feea31a1740 1 -- 172.21.15.107:0/1684877795 shutdown_connections 2024-04-25T00:20:19.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.653+0000 7feea31a1740 1 -- 172.21.15.107:0/1684877795 wait complete. 2024-04-25T00:20:19.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.653+0000 7feea31a1740 1 -- start start 2024-04-25T00:20:19.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.653+0000 7feea31a1740 1 Processor -- start 2024-04-25T00:20:19.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.653+0000 7feea31a1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a54135480 0x558a541442e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.653+0000 7feea31a1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a54135da0 0x558a541415d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.653+0000 7feea31a1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54144980 0x558a54141b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.653+0000 7feea31a1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558a54147740 con 0x558a54144980 2024-04-25T00:20:19.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.653+0000 7feea31a1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558a54056050 con 0x558a54135480 2024-04-25T00:20:19.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.653+0000 7feea31a1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558a541478c0 con 0x558a54135da0 2024-04-25T00:20:19.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.653+0000 7feea31a1740 10 client.0 _ll_register_callbacks cb 0x558a53dadbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:19.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.653+0000 7fee9da35640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54144980 0x558a54141b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.653+0000 7fee9da35640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54144980 0x558a54141b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39870/0 (socket says 172.21.15.107:39870) 2024-04-25T00:20:19.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.654+0000 7fee9da35640 1 -- 172.21.15.107:0/1333491728 learned_addr learned my addr 172.21.15.107:0/1333491728 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:19.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.654+0000 7fee9e236640 1 --2- 172.21.15.107:0/1333491728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a54135da0 0x558a541415d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.654+0000 7fee9da35640 1 --2- 172.21.15.107:0/1333491728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a54135480 0x558a541442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.654+0000 7fee9da35640 1 -- 172.21.15.107:0/1333491728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a54135da0 msgr2=0x558a541415d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.654+0000 7fee9da35640 1 --2- 172.21.15.107:0/1333491728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a54135da0 0x558a541415d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.654+0000 7fee9da35640 1 -- 172.21.15.107:0/1333491728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a54135480 msgr2=0x558a541442e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.654+0000 7fee9da35640 1 --2- 172.21.15.107:0/1333491728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a54135480 0x558a541442e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.654+0000 7fee9da35640 1 -- 172.21.15.107:0/1333491728 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a54027dc0 con 0x558a54144980 2024-04-25T00:20:19.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.654+0000 7fee9da35640 1 --2- 172.21.15.107:0/1333491728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54144980 0x558a54141b30 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fee940127b0 tx=0x7fee9400d600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=493e52aa90527683 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.654+0000 7fee81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:20:19.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.654+0000 7fee81ffb640 1 -- 172.21.15.107:0/1333491728 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9400e850 con 0x558a54144980 2024-04-25T00:20:19.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.654+0000 7fee81ffb640 1 -- 172.21.15.107:0/1333491728 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fee9400e9f0 con 0x558a54144980 2024-04-25T00:20:19.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.654+0000 7feea31a1740 1 -- 172.21.15.107:0/1333491728 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee8c018b20 con 0x558a54144980 2024-04-25T00:20:19.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.654+0000 7fee81ffb640 1 -- 172.21.15.107:0/1333491728 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fee94024050 con 0x558a54144980 2024-04-25T00:20:19.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.654+0000 7feea31a1740 10 client.16440 fetch_fsmap learned FSMap version 28 2024-04-25T00:20:19.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.654+0000 7feea31a1740 1 -- 172.21.15.107:0/1333491728 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558a53dae3f0 con 0x558a54144980 2024-04-25T00:20:19.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.654+0000 7fee81ffb640 1 -- 172.21.15.107:0/1333491728 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee8c018b20 con 0x558a54144980 2024-04-25T00:20:19.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.655+0000 7fee81ffb640 1 -- 172.21.15.107:0/1333491728 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fee9401d060 con 0x558a54144980 2024-04-25T00:20:19.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.655+0000 7feea31a1740 10 client.16440 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:20:19.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.655+0000 7feea31a1740 10 client.16440 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:20:19.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.655+0000 7feea31a1740 20 client.16440 populate_metadata read hostname 'smithi107' 2024-04-25T00:20:19.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.655+0000 7feea31a1740 1 --2- 172.21.15.107:0/1333491728 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x558a54154a50 0x558a54156e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.655+0000 7feea31a1740 4 client.16440 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:20:19.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.655+0000 7feea31a1740 1 -- 172.21.15.107:0/1333491728 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a540a7bb0 con 0x558a54154a50 2024-04-25T00:20:19.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.655+0000 7fee9e236640 1 --2- 172.21.15.107:0/1333491728 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x558a54154a50 0x558a54156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.655+0000 7fee81ffb640 10 client.16440 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:20:19.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.655+0000 7fee9e236640 1 --2- 172.21.15.107:0/1333491728 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x558a54154a50 0x558a54156e90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b38b891c4da28e8 server_cookie=dfa9c2e405d0b977 in_seq=0 out_seq=0 2024-04-25T00:20:19.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.656+0000 7fee81ffb640 1 -- 172.21.15.107:0/1333491728 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558a540a7bb0 con 0x558a54154a50 2024-04-25T00:20:19.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.656+0000 7fee81ffb640 10 client.16440 handle_command_reply: tid=0 2024-04-25T00:20:19.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.656+0000 7feea31a1740 1 client.16440 shutdown 2024-04-25T00:20:19.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.656+0000 7feea31a1740 1 -- 172.21.15.107:0/1333491728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54144980 msgr2=0x558a54141b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.656+0000 7feea31a1740 1 --2- 172.21.15.107:0/1333491728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54144980 0x558a54141b30 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fee940127b0 tx=0x7fee9400d600 comp rx=0 tx=0).stop 2024-04-25T00:20:19.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.656+0000 7feea31a1740 1 -- 172.21.15.107:0/1333491728 shutdown_connections 2024-04-25T00:20:19.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.656+0000 7feea31a1740 1 --2- 172.21.15.107:0/1333491728 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x558a54154a50 0x558a54156e90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.656+0000 7feea31a1740 1 --2- 172.21.15.107:0/1333491728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54144980 0x558a54141b30 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.656+0000 7fee81ffb640 0 client.16440 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:20:19.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.656+0000 7feea31a1740 1 --2- 172.21.15.107:0/1333491728 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a54135da0 0x558a541415d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.656+0000 7feea31a1740 1 --2- 172.21.15.107:0/1333491728 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a54135480 0x558a541442e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.656+0000 7feea31a1740 1 -- 172.21.15.107:0/1333491728 >> 172.21.15.107:0/1333491728 conn(0x558a541318a0 msgr2=0x558a541376d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.656+0000 7feea31a1740 1 -- 172.21.15.107:0/1333491728 shutdown_connections 2024-04-25T00:20:19.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.656+0000 7feea31a1740 1 -- 172.21.15.107:0/1333491728 wait complete. 2024-04-25T00:20:19.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.656+0000 7feea31a1740 20 client.16440 trim_cache size 0 max 16384 2024-04-25T00:20:19.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.669+0000 7feea31a1740 1 Processor -- start 2024-04-25T00:20:19.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.669+0000 7feea31a1740 1 -- start start 2024-04-25T00:20:19.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.669+0000 7feea31a1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a54135480 0x558a54150640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.669+0000 7feea31a1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54135da0 0x558a54140730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.669+0000 7feea31a1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a54144980 0x558a5413caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.669+0000 7feea31a1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fee981af1b0 con 0x558a54135da0 2024-04-25T00:20:19.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.669+0000 7feea31a1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fee981ac9c0 con 0x558a54144980 2024-04-25T00:20:19.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.669+0000 7feea31a1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fee981aa810 con 0x558a54135480 2024-04-25T00:20:19.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.670+0000 7fee9e236640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54135da0 0x558a54140730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.670+0000 7fee9e236640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54135da0 0x558a54140730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39874/0 (socket says 172.21.15.107:39874) 2024-04-25T00:20:19.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.670+0000 7fee9e236640 1 -- 172.21.15.107:0/480603395 learned_addr learned my addr 172.21.15.107:0/480603395 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:19.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.670+0000 7fee9da35640 1 --2- 172.21.15.107:0/480603395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a54135480 0x558a54150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.670+0000 7fee9e236640 1 -- 172.21.15.107:0/480603395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a54135480 msgr2=0x558a54150640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.670+0000 7fee9e236640 1 --2- 172.21.15.107:0/480603395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a54135480 0x558a54150640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.670+0000 7fee9e236640 1 -- 172.21.15.107:0/480603395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a54144980 msgr2=0x558a5413caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.670+0000 7fee9e236640 1 --2- 172.21.15.107:0/480603395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a54144980 0x558a5413caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.670+0000 7fee9e236640 1 -- 172.21.15.107:0/480603395 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a54135bd0 con 0x558a54135da0 2024-04-25T00:20:19.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.670+0000 7fee9da35640 1 --2- 172.21.15.107:0/480603395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a54135480 0x558a54150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:20:19.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.670+0000 7fee9e236640 1 --2- 172.21.15.107:0/480603395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54135da0 0x558a54140730 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x558a54137f80 tx=0x7fee8c00aad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf78f360dd846258 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.670+0000 7fee83fff640 1 -- 172.21.15.107:0/480603395 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c02d070 con 0x558a54135da0 2024-04-25T00:20:19.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.670+0000 7fee83fff640 1 -- 172.21.15.107:0/480603395 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee8c01e040 con 0x558a54135da0 2024-04-25T00:20:19.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.670+0000 7fee83fff640 1 -- 172.21.15.107:0/480603395 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c01d040 con 0x558a54135da0 2024-04-25T00:20:19.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.671+0000 7feea31a1740 1 -- 172.21.15.107:0/480603395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54135da0 msgr2=0x558a54140730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.671+0000 7feea31a1740 1 --2- 172.21.15.107:0/480603395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54135da0 0x558a54140730 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x558a54137f80 tx=0x7fee8c00aad0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.671+0000 7feea31a1740 1 -- 172.21.15.107:0/480603395 shutdown_connections 2024-04-25T00:20:19.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.671+0000 7feea31a1740 1 --2- 172.21.15.107:0/480603395 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a54144980 0x558a5413caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.671+0000 7feea31a1740 1 --2- 172.21.15.107:0/480603395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54135da0 0x558a54140730 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.671+0000 7feea31a1740 1 --2- 172.21.15.107:0/480603395 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a54135480 0x558a54150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.671+0000 7feea31a1740 1 -- 172.21.15.107:0/480603395 >> 172.21.15.107:0/480603395 conn(0x558a541318a0 msgr2=0x558a5413c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.671+0000 7feea31a1740 1 -- 172.21.15.107:0/480603395 shutdown_connections 2024-04-25T00:20:19.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.671+0000 7feea31a1740 1 -- 172.21.15.107:0/480603395 wait complete. 2024-04-25T00:20:19.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.671+0000 7feea31a1740 1 -- start start 2024-04-25T00:20:19.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.671+0000 7feea31a1740 1 Processor -- start 2024-04-25T00:20:19.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.671+0000 7feea31a1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a54135480 0x558a5414cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.671+0000 7feea31a1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54135da0 0x558a54148110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.671+0000 7feea31a1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a54144980 0x558a54148650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.671+0000 7feea31a1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fee981aae80 con 0x558a54135da0 2024-04-25T00:20:19.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.671+0000 7feea31a1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fee981ac2f0 con 0x558a54135480 2024-04-25T00:20:19.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.671+0000 7feea31a1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fee981ab550 con 0x558a54144980 2024-04-25T00:20:19.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.671+0000 7feea31a1740 10 client.0 _ll_register_callbacks cb 0x7fee9400d460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:19.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.671+0000 7fee9e236640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54135da0 0x558a54148110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.672+0000 7fee9e236640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54135da0 0x558a54148110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39888/0 (socket says 172.21.15.107:39888) 2024-04-25T00:20:19.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.672+0000 7fee9e236640 1 -- 172.21.15.107:0/1177953444 learned_addr learned my addr 172.21.15.107:0/1177953444 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:19.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.672+0000 7fee9da35640 1 --2- 172.21.15.107:0/1177953444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a54144980 0x558a54148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.672+0000 7fee9da35640 1 --2- 172.21.15.107:0/1177953444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a54135480 0x558a5414cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.672+0000 7fee9e236640 1 -- 172.21.15.107:0/1177953444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a54144980 msgr2=0x558a54148650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.672+0000 7fee9e236640 1 --2- 172.21.15.107:0/1177953444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a54144980 0x558a54148650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.672+0000 7fee9e236640 1 -- 172.21.15.107:0/1177953444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a54135480 msgr2=0x558a5414cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.672+0000 7fee9e236640 1 --2- 172.21.15.107:0/1177953444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a54135480 0x558a5414cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.672+0000 7fee9e236640 1 -- 172.21.15.107:0/1177953444 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a53e787d0 con 0x558a54135da0 2024-04-25T00:20:19.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.672+0000 7fee9e236640 1 --2- 172.21.15.107:0/1177953444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54135da0 0x558a54148110 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00fe70 tx=0x7fee8c0104b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df949531294eeaa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.672+0000 7fee81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:20:19.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.672+0000 7fee81ffb640 1 -- 172.21.15.107:0/1177953444 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c036030 con 0x558a54135da0 2024-04-25T00:20:19.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.672+0000 7fee81ffb640 1 -- 172.21.15.107:0/1177953444 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fee8c018440 con 0x558a54135da0 2024-04-25T00:20:19.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.672+0000 7feea31a1740 1 -- 172.21.15.107:0/1177953444 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee8c01e040 con 0x558a54135da0 2024-04-25T00:20:19.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.672+0000 7fee81ffb640 1 -- 172.21.15.107:0/1177953444 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fee8c043050 con 0x558a54135da0 2024-04-25T00:20:19.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.672+0000 7feea31a1740 10 client.16446 fetch_fsmap learned FSMap version 28 2024-04-25T00:20:19.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.672+0000 7feea31a1740 1 -- 172.21.15.107:0/1177953444 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558a541bab00 con 0x558a54135da0 2024-04-25T00:20:19.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.672+0000 7fee81ffb640 1 -- 172.21.15.107:0/1177953444 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee8c01e040 con 0x558a54135da0 2024-04-25T00:20:19.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.673+0000 7fee81ffb640 1 -- 172.21.15.107:0/1177953444 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fee8c029590 con 0x558a54135da0 2024-04-25T00:20:19.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.673+0000 7feea31a1740 10 client.16446 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:20:19.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.673+0000 7feea31a1740 10 client.16446 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:20:19.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.673+0000 7feea31a1740 20 client.16446 populate_metadata read hostname 'smithi107' 2024-04-25T00:20:19.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.673+0000 7feea31a1740 1 --2- 172.21.15.107:0/1177953444 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x558a541bb130 0x558a541bd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.673+0000 7feea31a1740 4 client.16446 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:20:19.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.673+0000 7feea31a1740 1 -- 172.21.15.107:0/1177953444 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a540a1f00 con 0x558a541bb130 2024-04-25T00:20:19.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.673+0000 7fee9da35640 1 --2- 172.21.15.107:0/1177953444 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x558a541bb130 0x558a541bd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.673+0000 7fee9da35640 1 --2- 172.21.15.107:0/1177953444 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x558a541bb130 0x558a541bd550 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=292c09f4efd3ed5a server_cookie=f3c5ded2383bec89 in_seq=0 out_seq=0 2024-04-25T00:20:19.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.673+0000 7fee81ffb640 10 client.16446 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:20:19.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.673+0000 7fee81ffb640 1 -- 172.21.15.107:0/1177953444 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a540a1f00 con 0x558a541bb130 2024-04-25T00:20:19.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.673+0000 7fee81ffb640 10 client.16446 handle_command_reply: tid=0 2024-04-25T00:20:19.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.673+0000 7feea31a1740 1 client.16446 shutdown 2024-04-25T00:20:19.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.673+0000 7feea31a1740 1 -- 172.21.15.107:0/1177953444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54135da0 msgr2=0x558a54148110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.673+0000 7feea31a1740 1 --2- 172.21.15.107:0/1177953444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54135da0 0x558a54148110 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00fe70 tx=0x7fee8c0104b0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.673+0000 7feea31a1740 1 -- 172.21.15.107:0/1177953444 shutdown_connections 2024-04-25T00:20:19.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.673+0000 7feea31a1740 1 --2- 172.21.15.107:0/1177953444 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x558a541bb130 0x558a541bd550 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.673+0000 7feea31a1740 1 --2- 172.21.15.107:0/1177953444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558a54144980 0x558a54148650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.673+0000 7feea31a1740 1 --2- 172.21.15.107:0/1177953444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558a54135da0 0x558a54148110 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.673+0000 7feea31a1740 1 --2- 172.21.15.107:0/1177953444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558a54135480 0x558a5414cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.673+0000 7fee81ffb640 0 client.16446 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:20:19.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.674+0000 7feea31a1740 1 -- 172.21.15.107:0/1177953444 >> 172.21.15.107:0/1177953444 conn(0x558a541318a0 msgr2=0x558a5413cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.674+0000 7feea31a1740 1 -- 172.21.15.107:0/1177953444 shutdown_connections 2024-04-25T00:20:19.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.674+0000 7feea31a1740 1 -- 172.21.15.107:0/1177953444 wait complete. 2024-04-25T00:20:19.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.674+0000 7feea31a1740 20 client.16446 trim_cache size 0 max 16384 2024-04-25T00:20:19.682 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:20:19.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.674+0000 7feea049a640 1 -- 172.21.15.107:0/1825017466 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fee840a2d00 msgr2=0x7fee840a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.674+0000 7feea049a640 1 --2- 172.21.15.107:0/1825017466 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fee840a2d00 0x7fee840a51c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fee900ee470 tx=0x7fee90084040 comp rx=0 tx=0).stop 2024-04-25T00:20:19.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.674+0000 7feea049a640 1 -- 172.21.15.107:0/1825017466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee98106630 msgr2=0x7fee981a3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.674+0000 7feea049a640 1 --2- 172.21.15.107:0/1825017466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee98106630 0x7fee981a3e80 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fee90005cc0 tx=0x7fee90003970 comp rx=0 tx=0).stop 2024-04-25T00:20:19.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.674+0000 7feea049a640 1 -- 172.21.15.107:0/1825017466 shutdown_connections 2024-04-25T00:20:19.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.674+0000 7feea049a640 1 --2- 172.21.15.107:0/1825017466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee98106630 0x7fee981a3e80 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.674+0000 7feea049a640 1 --2- 172.21.15.107:0/1825017466 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fee840a2d00 0x7fee840a51c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.674+0000 7feea049a640 1 --2- 172.21.15.107:0/1825017466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee98100fe0 0x7fee98199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.674+0000 7feea049a640 1 --2- 172.21.15.107:0/1825017466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee98100620 0x7fee98198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.674+0000 7feea049a640 1 -- 172.21.15.107:0/1825017466 >> 172.21.15.107:0/1825017466 conn(0x7fee980776a0 msgr2=0x7fee98103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.674+0000 7feea049a640 1 -- 172.21.15.107:0/1825017466 shutdown_connections 2024-04-25T00:20:19.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.674+0000 7feea049a640 1 -- 172.21.15.107:0/1825017466 wait complete. 2024-04-25T00:20:19.685 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:20:19.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.785+0000 7ff0b2e38640 1 -- 172.21.15.107:0/3587418543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0ac0ffc40 msgr2=0x7ff0ac1000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.785+0000 7ff0b2e38640 1 --2- 172.21.15.107:0/3587418543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0ac0ffc40 0x7ff0ac1000c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7ff098009930 tx=0x7ff09802a420 comp rx=0 tx=0).stop 2024-04-25T00:20:19.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.785+0000 7ff0b2e38640 1 -- 172.21.15.107:0/3587418543 shutdown_connections 2024-04-25T00:20:19.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.785+0000 7ff0b2e38640 1 --2- 172.21.15.107:0/3587418543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ac100600 0x7ff0ac10e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.785+0000 7ff0b2e38640 1 --2- 172.21.15.107:0/3587418543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0ac0ffc40 0x7ff0ac1000c0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.785+0000 7ff0b2e38640 1 --2- 172.21.15.107:0/3587418543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0ac107720 0x7ff0ac107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.785+0000 7ff0b2e38640 1 -- 172.21.15.107:0/3587418543 >> 172.21.15.107:0/3587418543 conn(0x7ff0ac076bd0 msgr2=0x7ff0ac0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.785+0000 7ff0b2e38640 1 -- 172.21.15.107:0/3587418543 shutdown_connections 2024-04-25T00:20:19.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.785+0000 7ff0b2e38640 1 -- 172.21.15.107:0/3587418543 wait complete. 2024-04-25T00:20:19.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.785+0000 7ff0b2e38640 1 Processor -- start 2024-04-25T00:20:19.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.785+0000 7ff0b2e38640 1 -- start start 2024-04-25T00:20:19.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.785+0000 7ff0b2e38640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ac0ffc40 0x7ff0ac198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.785+0000 7ff0b2e38640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0ac100600 0x7ff0ac1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.785+0000 7ff0b2e38640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0ac107720 0x7ff0ac19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.785+0000 7ff0b2e38640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac111b30 con 0x7ff0ac100600 2024-04-25T00:20:19.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.785+0000 7ff0b2e38640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac1119b0 con 0x7ff0ac0ffc40 2024-04-25T00:20:19.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.785+0000 7ff0b2e38640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac111cb0 con 0x7ff0ac107720 2024-04-25T00:20:19.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.786+0000 7ff0abfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0ac100600 0x7ff0ac1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.786+0000 7ff0abfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0ac100600 0x7ff0ac1985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39898/0 (socket says 172.21.15.107:39898) 2024-04-25T00:20:19.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.786+0000 7ff0abfff640 1 -- 172.21.15.107:0/715458445 learned_addr learned my addr 172.21.15.107:0/715458445 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:19.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.786+0000 7ff0b0bd4640 1 --2- 172.21.15.107:0/715458445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ac0ffc40 0x7ff0ac198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.786+0000 7ff0abfff640 1 -- 172.21.15.107:0/715458445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0ac107720 msgr2=0x7ff0ac19c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.786+0000 7ff0b13d5640 1 --2- 172.21.15.107:0/715458445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0ac107720 0x7ff0ac19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.786+0000 7ff0abfff640 1 --2- 172.21.15.107:0/715458445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0ac107720 0x7ff0ac19c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.786+0000 7ff0abfff640 1 -- 172.21.15.107:0/715458445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ac0ffc40 msgr2=0x7ff0ac198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.786+0000 7ff0abfff640 1 --2- 172.21.15.107:0/715458445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ac0ffc40 0x7ff0ac198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.786+0000 7ff0abfff640 1 -- 172.21.15.107:0/715458445 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ac19d030 con 0x7ff0ac100600 2024-04-25T00:20:19.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.786+0000 7ff0b0bd4640 1 --2- 172.21.15.107:0/715458445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ac0ffc40 0x7ff0ac198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:20:19.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.786+0000 7ff0abfff640 1 --2- 172.21.15.107:0/715458445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0ac100600 0x7ff0ac1985d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff098009230 tx=0x7ff098004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.786+0000 7ff0a9ffb640 1 -- 172.21.15.107:0/715458445 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09803f030 con 0x7ff0ac100600 2024-04-25T00:20:19.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.786+0000 7ff0b2e38640 1 -- 172.21.15.107:0/715458445 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ac19d2c0 con 0x7ff0ac100600 2024-04-25T00:20:19.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.786+0000 7ff0a9ffb640 1 -- 172.21.15.107:0/715458445 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff098003ad0 con 0x7ff0ac100600 2024-04-25T00:20:19.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.786+0000 7ff0a9ffb640 1 -- 172.21.15.107:0/715458445 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff098035070 con 0x7ff0ac100600 2024-04-25T00:20:19.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.786+0000 7ff0b2e38640 1 -- 172.21.15.107:0/715458445 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ac1a4bb0 con 0x7ff0ac100600 2024-04-25T00:20:19.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.789+0000 7ff0a9ffb640 1 -- 172.21.15.107:0/715458445 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff098042030 con 0x7ff0ac100600 2024-04-25T00:20:19.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.789+0000 7ff0b5c1e740 1 Processor -- start 2024-04-25T00:20:19.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.789+0000 7ff0b5c1e740 1 -- start start 2024-04-25T00:20:19.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.789+0000 7ff0b5c1e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a5389580 0x5579a5389960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.789+0000 7ff0b5c1e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5389ea0 0x5579a5394030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.789+0000 7ff0b5c1e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579a5394570 0x5579a5396960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.789+0000 7ff0a9ffb640 1 --2- 172.21.15.107:0/715458445 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff09c0a2d00 0x7ff09c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.789+0000 7ff0b5c1e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5579a52ee670 con 0x5579a5389ea0 2024-04-25T00:20:19.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.789+0000 7ff0b5c1e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5579a5296970 con 0x5579a5389580 2024-04-25T00:20:19.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.789+0000 7ff0b5c1e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5579a52ea630 con 0x5579a5394570 2024-04-25T00:20:19.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.789+0000 7ff0b0bd4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5389ea0 0x5579a5394030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.789+0000 7ff0b13d5640 1 --2- 172.21.15.107:0/715458445 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff09c0a2d00 0x7ff09c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.789+0000 7ff0b0bd4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5389ea0 0x5579a5394030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39900/0 (socket says 172.21.15.107:39900) 2024-04-25T00:20:19.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.789+0000 7ff0b0bd4640 1 -- 172.21.15.107:0/960457282 learned_addr learned my addr 172.21.15.107:0/960457282 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:19.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.789+0000 7ff0a9ffb640 1 -- 172.21.15.107:0/715458445 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7ff0980efd70 con 0x7ff0ac100600 2024-04-25T00:20:19.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.790+0000 7ff0b13d5640 1 --2- 172.21.15.107:0/960457282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a5389580 0x5579a5389960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.790+0000 7ff0abfff640 1 --2- 172.21.15.107:0/960457282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579a5394570 0x5579a5396960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.790+0000 7ff0b0bd4640 1 -- 172.21.15.107:0/960457282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579a5394570 msgr2=0x5579a5396960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.790+0000 7ff0b0bd4640 1 --2- 172.21.15.107:0/960457282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579a5394570 0x5579a5396960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.790+0000 7ff0b0bd4640 1 -- 172.21.15.107:0/960457282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a5389580 msgr2=0x5579a5389960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.790+0000 7ff0b0bd4640 1 --2- 172.21.15.107:0/960457282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a5389580 0x5579a5389960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.790+0000 7ff0b0bd4640 1 -- 172.21.15.107:0/960457282 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579a52ba400 con 0x5579a5389ea0 2024-04-25T00:20:19.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.790+0000 7ff0abfff640 1 --2- 172.21.15.107:0/960457282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579a5394570 0x5579a5396960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:20:19.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.790+0000 7ff0b13d5640 1 --2- 172.21.15.107:0/715458445 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff09c0a2d00 0x7ff09c0a51c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4005fd0 tx=0x7ff0a40129b0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.790+0000 7ff0b0bd4640 1 --2- 172.21.15.107:0/960457282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5389ea0 0x5579a5394030 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000baf0 tx=0x7ff0a000d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b7cf12581b696c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.790+0000 7ff096ffd640 1 -- 172.21.15.107:0/960457282 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0017420 con 0x5579a5389ea0 2024-04-25T00:20:19.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.790+0000 7ff096ffd640 1 -- 172.21.15.107:0/960457282 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a0017aa0 con 0x5579a5389ea0 2024-04-25T00:20:19.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.790+0000 7ff096ffd640 1 -- 172.21.15.107:0/960457282 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a000fad0 con 0x5579a5389ea0 2024-04-25T00:20:19.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.791+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/960457282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5389ea0 msgr2=0x5579a5394030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.791+0000 7ff0b5c1e740 1 --2- 172.21.15.107:0/960457282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5389ea0 0x5579a5394030 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000baf0 tx=0x7ff0a000d340 comp rx=0 tx=0).stop 2024-04-25T00:20:19.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.791+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/960457282 shutdown_connections 2024-04-25T00:20:19.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.791+0000 7ff0b5c1e740 1 --2- 172.21.15.107:0/960457282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579a5394570 0x5579a5396960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.791+0000 7ff0b5c1e740 1 --2- 172.21.15.107:0/960457282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5389ea0 0x5579a5394030 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.791+0000 7ff0b5c1e740 1 --2- 172.21.15.107:0/960457282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a5389580 0x5579a5389960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.791+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/960457282 >> 172.21.15.107:0/960457282 conn(0x5579a53858a0 msgr2=0x5579a5385c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.791+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/960457282 shutdown_connections 2024-04-25T00:20:19.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.791+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/960457282 wait complete. 2024-04-25T00:20:19.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.791+0000 7ff0b5c1e740 1 -- start start 2024-04-25T00:20:19.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.791+0000 7ff0b5c1e740 1 Processor -- start 2024-04-25T00:20:19.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.791+0000 7ff0b5c1e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579a5389580 0x5579a5393e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.791+0000 7ff0b5c1e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a5389ea0 0x5579a5391110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.791+0000 7ff0b5c1e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5394570 0x5579a5391650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.791+0000 7ff0b5c1e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5579a5397360 con 0x5579a5394570 2024-04-25T00:20:19.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.791+0000 7ff0b5c1e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5579a52aa050 con 0x5579a5389ea0 2024-04-25T00:20:19.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.791+0000 7ff0b5c1e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5579a53974e0 con 0x5579a5389580 2024-04-25T00:20:19.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.792+0000 7ff0b5c1e740 10 client.0 _ll_register_callbacks cb 0x5579a5001be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:19.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.792+0000 7ff0b13d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5394570 0x5579a5391650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.792+0000 7ff0b13d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5394570 0x5579a5391650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39906/0 (socket says 172.21.15.107:39906) 2024-04-25T00:20:19.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.792+0000 7ff0b13d5640 1 -- 172.21.15.107:0/2686015707 learned_addr learned my addr 172.21.15.107:0/2686015707 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:19.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.792+0000 7ff0b0bd4640 1 --2- 172.21.15.107:0/2686015707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579a5389580 0x5579a5393e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.792+0000 7ff0b13d5640 1 -- 172.21.15.107:0/2686015707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579a5389580 msgr2=0x5579a5393e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.792+0000 7ff0b13d5640 1 --2- 172.21.15.107:0/2686015707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579a5389580 0x5579a5393e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.792+0000 7ff0abfff640 1 --2- 172.21.15.107:0/2686015707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a5389ea0 0x5579a5391110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.792+0000 7ff0b13d5640 1 -- 172.21.15.107:0/2686015707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a5389ea0 msgr2=0x5579a5391110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.792+0000 7ff0b13d5640 1 --2- 172.21.15.107:0/2686015707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a5389ea0 0x5579a5391110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.792+0000 7ff0b13d5640 1 -- 172.21.15.107:0/2686015707 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579a527bdc0 con 0x5579a5394570 2024-04-25T00:20:19.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.792+0000 7ff0abfff640 1 --2- 172.21.15.107:0/2686015707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a5389ea0 0x5579a5391110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:20:19.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.792+0000 7ff0b13d5640 1 --2- 172.21.15.107:0/2686015707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5394570 0x5579a5391650 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40107c0 tx=0x7ff0a4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9de4a85ba678cb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.792+0000 7ff08bfff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:20:19.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.792+0000 7ff08bfff640 1 -- 172.21.15.107:0/2686015707 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a401caf0 con 0x5579a5394570 2024-04-25T00:20:19.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.792+0000 7ff08bfff640 1 -- 172.21.15.107:0/2686015707 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7ff0a401cc90 con 0x5579a5394570 2024-04-25T00:20:19.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.792+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/2686015707 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0a0017aa0 con 0x5579a5394570 2024-04-25T00:20:19.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.792+0000 7ff08bfff640 1 -- 172.21.15.107:0/2686015707 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff0a401e450 con 0x5579a5394570 2024-04-25T00:20:19.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.792+0000 7ff0b5c1e740 10 client.16458 fetch_fsmap learned FSMap version 28 2024-04-25T00:20:19.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.792+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/2686015707 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5579a50023f0 con 0x5579a5394570 2024-04-25T00:20:19.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.792+0000 7ff08bfff640 1 -- 172.21.15.107:0/2686015707 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a0017aa0 con 0x5579a5394570 2024-04-25T00:20:19.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.793+0000 7ff08bfff640 1 -- 172.21.15.107:0/2686015707 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7ff0a401b040 con 0x5579a5394570 2024-04-25T00:20:19.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.793+0000 7ff0b5c1e740 10 client.16458 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:20:19.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.793+0000 7ff0b5c1e740 10 client.16458 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:20:19.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.793+0000 7ff0b5c1e740 20 client.16458 populate_metadata read hostname 'smithi107' 2024-04-25T00:20:19.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.793+0000 7ff0b5c1e740 1 --2- 172.21.15.107:0/2686015707 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5579a53a4510 0x5579a53a6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.793+0000 7ff0b5c1e740 4 client.16458 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:20:19.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.793+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/2686015707 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579a52fbbb0 con 0x5579a53a4510 2024-04-25T00:20:19.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.793+0000 7ff0b0bd4640 1 --2- 172.21.15.107:0/2686015707 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5579a53a4510 0x5579a53a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.793+0000 7ff08bfff640 10 client.16458 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:20:19.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.793+0000 7ff0b0bd4640 1 --2- 172.21.15.107:0/2686015707 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5579a53a4510 0x5579a53a6970 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=89844387eecc75ea server_cookie=48030d1e6099753 in_seq=0 out_seq=0 2024-04-25T00:20:19.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.794+0000 7ff08bfff640 1 -- 172.21.15.107:0/2686015707 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5579a52fbbb0 con 0x5579a53a4510 2024-04-25T00:20:19.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.794+0000 7ff08bfff640 10 client.16458 handle_command_reply: tid=0 2024-04-25T00:20:19.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.794+0000 7ff0b5c1e740 1 client.16458 shutdown 2024-04-25T00:20:19.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.794+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/2686015707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5394570 msgr2=0x5579a5391650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.794+0000 7ff0b5c1e740 1 --2- 172.21.15.107:0/2686015707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5394570 0x5579a5391650 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40107c0 tx=0x7ff0a4018000 comp rx=0 tx=0).stop 2024-04-25T00:20:19.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.794+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/2686015707 shutdown_connections 2024-04-25T00:20:19.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.794+0000 7ff0b5c1e740 1 --2- 172.21.15.107:0/2686015707 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5579a53a4510 0x5579a53a6970 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.794+0000 7ff0b5c1e740 1 --2- 172.21.15.107:0/2686015707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5394570 0x5579a5391650 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.794+0000 7ff0b5c1e740 1 --2- 172.21.15.107:0/2686015707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a5389ea0 0x5579a5391110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.794+0000 7ff0b5c1e740 1 --2- 172.21.15.107:0/2686015707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579a5389580 0x5579a5393e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.794+0000 7ff08bfff640 0 client.16458 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:20:19.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.794+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/2686015707 >> 172.21.15.107:0/2686015707 conn(0x5579a53858a0 msgr2=0x5579a5384b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.794+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/2686015707 shutdown_connections 2024-04-25T00:20:19.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.794+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/2686015707 wait complete. 2024-04-25T00:20:19.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.795+0000 7ff0b5c1e740 20 client.16458 trim_cache size 0 max 16384 2024-04-25T00:20:19.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.808+0000 7ff0b5c1e740 1 Processor -- start 2024-04-25T00:20:19.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.808+0000 7ff0b5c1e740 1 -- start start 2024-04-25T00:20:19.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.808+0000 7ff0b5c1e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5389580 0x5579a53a0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.808+0000 7ff0b5c1e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579a5389ea0 0x5579a53ac240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.808+0000 7ff0b5c1e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a5394570 0x5579a53ac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.808+0000 7ff0b5c1e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac1b1440 con 0x5579a5389580 2024-04-25T00:20:19.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.808+0000 7ff0b5c1e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac1aec50 con 0x5579a5394570 2024-04-25T00:20:19.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.808+0000 7ff0b5c1e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac1acaa0 con 0x5579a5389ea0 2024-04-25T00:20:19.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.808+0000 7ff0b0bd4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5389580 0x5579a53a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.808+0000 7ff0b0bd4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5389580 0x5579a53a0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39922/0 (socket says 172.21.15.107:39922) 2024-04-25T00:20:19.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.808+0000 7ff0b0bd4640 1 -- 172.21.15.107:0/279560404 learned_addr learned my addr 172.21.15.107:0/279560404 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:19.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.808+0000 7ff0b0bd4640 1 -- 172.21.15.107:0/279560404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579a5389ea0 msgr2=0x5579a53ac240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.808+0000 7ff0b0bd4640 1 --2- 172.21.15.107:0/279560404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579a5389ea0 0x5579a53ac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.808+0000 7ff0b0bd4640 1 -- 172.21.15.107:0/279560404 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a5394570 msgr2=0x5579a53ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.808+0000 7ff0b0bd4640 1 --2- 172.21.15.107:0/279560404 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a5394570 0x5579a53ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.808+0000 7ff0b0bd4640 1 -- 172.21.15.107:0/279560404 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579a50cc7d0 con 0x5579a5389580 2024-04-25T00:20:19.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.808+0000 7ff0b0bd4640 1 --2- 172.21.15.107:0/279560404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5389580 0x5579a53a0220 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000d840 tx=0x7ff0a000d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63adbfb6b98a506 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.808+0000 7ff0967fc640 1 -- 172.21.15.107:0/279560404 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0028d50 con 0x5579a5389580 2024-04-25T00:20:19.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.808+0000 7ff0967fc640 1 -- 172.21.15.107:0/279560404 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a002d070 con 0x5579a5389580 2024-04-25T00:20:19.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.808+0000 7ff0967fc640 1 -- 172.21.15.107:0/279560404 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a000fa00 con 0x5579a5389580 2024-04-25T00:20:19.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.809+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/279560404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5389580 msgr2=0x5579a53a0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.809+0000 7ff0b5c1e740 1 --2- 172.21.15.107:0/279560404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5389580 0x5579a53a0220 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000d840 tx=0x7ff0a000d340 comp rx=0 tx=0).stop 2024-04-25T00:20:19.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.809+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/279560404 shutdown_connections 2024-04-25T00:20:19.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.809+0000 7ff0b5c1e740 1 --2- 172.21.15.107:0/279560404 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a5394570 0x5579a53ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.809+0000 7ff0b5c1e740 1 --2- 172.21.15.107:0/279560404 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579a5389ea0 0x5579a53ac240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.809+0000 7ff0b5c1e740 1 --2- 172.21.15.107:0/279560404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5389580 0x5579a53a0220 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.809+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/279560404 >> 172.21.15.107:0/279560404 conn(0x5579a53858a0 msgr2=0x5579a53842b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.809+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/279560404 shutdown_connections 2024-04-25T00:20:19.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.809+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/279560404 wait complete. 2024-04-25T00:20:19.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.809+0000 7ff0b5c1e740 1 -- start start 2024-04-25T00:20:19.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.809+0000 7ff0b5c1e740 1 Processor -- start 2024-04-25T00:20:19.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff0b5c1e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a5389580 0x5579a539c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff0b5c1e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5389ea0 0x5579a5399d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff0b5c1e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579a5394570 0x5579a539a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff0b5c1e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac1ad110 con 0x5579a5389ea0 2024-04-25T00:20:19.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff0b5c1e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac1ae580 con 0x5579a5389580 2024-04-25T00:20:19.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff0b5c1e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac1ad7e0 con 0x5579a5394570 2024-04-25T00:20:19.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff0b5c1e740 10 client.0 _ll_register_callbacks cb 0x7ff0a4009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:19.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff0abfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5389ea0 0x5579a5399d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff0abfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5389ea0 0x5579a5399d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39924/0 (socket says 172.21.15.107:39924) 2024-04-25T00:20:19.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff0abfff640 1 -- 172.21.15.107:0/3444052081 learned_addr learned my addr 172.21.15.107:0/3444052081 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:19.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff0b0bd4640 1 --2- 172.21.15.107:0/3444052081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a5389580 0x5579a539c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff0b13d5640 1 --2- 172.21.15.107:0/3444052081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579a5394570 0x5579a539a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff0abfff640 1 -- 172.21.15.107:0/3444052081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579a5394570 msgr2=0x5579a539a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff0abfff640 1 --2- 172.21.15.107:0/3444052081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579a5394570 0x5579a539a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff0abfff640 1 -- 172.21.15.107:0/3444052081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a5389580 msgr2=0x5579a539c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff0abfff640 1 --2- 172.21.15.107:0/3444052081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a5389580 0x5579a539c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff0abfff640 1 -- 172.21.15.107:0/3444052081 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579a539a9b0 con 0x5579a5389ea0 2024-04-25T00:20:19.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff0b13d5640 1 --2- 172.21.15.107:0/3444052081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579a5394570 0x5579a539a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:20:19.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff0abfff640 1 --2- 172.21.15.107:0/3444052081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5389ea0 0x5579a5399d80 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff0980b8380 tx=0x7ff0980bbb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34361f6c49ba53b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff08bfff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:20:19.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff08bfff640 1 -- 172.21.15.107:0/3444052081 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff098049040 con 0x5579a5389ea0 2024-04-25T00:20:19.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff08bfff640 1 -- 172.21.15.107:0/3444052081 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7ff0980b8590 con 0x5579a5389ea0 2024-04-25T00:20:19.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/3444052081 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0a002d070 con 0x5579a5389ea0 2024-04-25T00:20:19.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff08bfff640 1 -- 172.21.15.107:0/3444052081 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff0980c02d0 con 0x5579a5389ea0 2024-04-25T00:20:19.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.810+0000 7ff0b5c1e740 10 client.16464 fetch_fsmap learned FSMap version 28 2024-04-25T00:20:19.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.811+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/3444052081 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5579a540ee30 con 0x5579a5389ea0 2024-04-25T00:20:19.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.811+0000 7ff08bfff640 1 -- 172.21.15.107:0/3444052081 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a002d070 con 0x5579a5389ea0 2024-04-25T00:20:19.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.811+0000 7ff08bfff640 1 -- 172.21.15.107:0/3444052081 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7ff0980cbd90 con 0x5579a5389ea0 2024-04-25T00:20:19.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.811+0000 7ff0b5c1e740 10 client.16464 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:20:19.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.811+0000 7ff0b5c1e740 10 client.16464 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:20:19.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.811+0000 7ff0b5c1e740 20 client.16464 populate_metadata read hostname 'smithi107' 2024-04-25T00:20:19.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.811+0000 7ff0b5c1e740 1 --2- 172.21.15.107:0/3444052081 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5579a540f5e0 0x5579a5411a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.811+0000 7ff0b5c1e740 4 client.16464 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:20:19.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.811+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/3444052081 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5579a52f5f00 con 0x5579a540f5e0 2024-04-25T00:20:19.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.811+0000 7ff0b0bd4640 1 --2- 172.21.15.107:0/3444052081 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5579a540f5e0 0x5579a5411a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.811+0000 7ff08bfff640 10 client.16464 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:20:19.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.811+0000 7ff0b0bd4640 1 --2- 172.21.15.107:0/3444052081 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5579a540f5e0 0x5579a5411a00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3e270d95f33e7269 server_cookie=8e96c2155ce77885 in_seq=0 out_seq=0 2024-04-25T00:20:19.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.812+0000 7ff08bfff640 1 -- 172.21.15.107:0/3444052081 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5579a52f5f00 con 0x5579a540f5e0 2024-04-25T00:20:19.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.812+0000 7ff08bfff640 10 client.16464 handle_command_reply: tid=0 2024-04-25T00:20:19.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.812+0000 7ff0b5c1e740 1 client.16464 shutdown 2024-04-25T00:20:19.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.812+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/3444052081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5389ea0 msgr2=0x5579a5399d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.812+0000 7ff0b5c1e740 1 --2- 172.21.15.107:0/3444052081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5389ea0 0x5579a5399d80 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff0980b8380 tx=0x7ff0980bbb20 comp rx=0 tx=0).stop 2024-04-25T00:20:19.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.812+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/3444052081 shutdown_connections 2024-04-25T00:20:19.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.812+0000 7ff0b5c1e740 1 --2- 172.21.15.107:0/3444052081 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5579a5394570 0x5579a539a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.812+0000 7ff0b5c1e740 1 --2- 172.21.15.107:0/3444052081 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5579a540f5e0 0x5579a5411a00 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.812+0000 7ff0b5c1e740 1 --2- 172.21.15.107:0/3444052081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579a5389ea0 0x5579a5399d80 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.812+0000 7ff08bfff640 0 client.16464 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:20:19.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.812+0000 7ff0b5c1e740 1 --2- 172.21.15.107:0/3444052081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5579a5389580 0x5579a539c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.812+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/3444052081 >> 172.21.15.107:0/3444052081 conn(0x5579a53858a0 msgr2=0x5579a5384b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.812+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/3444052081 shutdown_connections 2024-04-25T00:20:19.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.812+0000 7ff0b5c1e740 1 -- 172.21.15.107:0/3444052081 wait complete. 2024-04-25T00:20:19.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.812+0000 7ff0b5c1e740 20 client.16464 trim_cache size 0 max 16384 2024-04-25T00:20:19.820 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:20:19.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.812+0000 7ff0b2e38640 1 -- 172.21.15.107:0/715458445 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff09c0a2d00 msgr2=0x7ff09c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.812+0000 7ff0b2e38640 1 --2- 172.21.15.107:0/715458445 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff09c0a2d00 0x7ff09c0a51c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4005fd0 tx=0x7ff0a40129b0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.812+0000 7ff0b2e38640 1 -- 172.21.15.107:0/715458445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0ac100600 msgr2=0x7ff0ac1985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.812+0000 7ff0b2e38640 1 --2- 172.21.15.107:0/715458445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0ac100600 0x7ff0ac1985d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff098009230 tx=0x7ff098004270 comp rx=0 tx=0).stop 2024-04-25T00:20:19.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.813+0000 7ff0b2e38640 1 -- 172.21.15.107:0/715458445 shutdown_connections 2024-04-25T00:20:19.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.813+0000 7ff0b2e38640 1 --2- 172.21.15.107:0/715458445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff0ac107720 0x7ff0ac19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.813+0000 7ff0b2e38640 1 --2- 172.21.15.107:0/715458445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0ac100600 0x7ff0ac1985d0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.813+0000 7ff0b2e38640 1 --2- 172.21.15.107:0/715458445 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff09c0a2d00 0x7ff09c0a51c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.813+0000 7ff0b2e38640 1 --2- 172.21.15.107:0/715458445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0ac0ffc40 0x7ff0ac198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.813+0000 7ff0b2e38640 1 -- 172.21.15.107:0/715458445 >> 172.21.15.107:0/715458445 conn(0x7ff0ac076bd0 msgr2=0x7ff0ac0fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.813+0000 7ff0b2e38640 1 -- 172.21.15.107:0/715458445 shutdown_connections 2024-04-25T00:20:19.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.813+0000 7ff0b2e38640 1 -- 172.21.15.107:0/715458445 wait complete. 2024-04-25T00:20:19.824 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:20:19.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.923+0000 7fb74489a640 1 -- 172.21.15.107:0/2995474160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb740100620 msgr2=0x7fb740100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.923+0000 7fb74489a640 1 --2- 172.21.15.107:0/2995474160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb740100620 0x7fb740100aa0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb730009930 tx=0x7fb73002a420 comp rx=0 tx=0).stop 2024-04-25T00:20:19.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.923+0000 7fb74489a640 1 -- 172.21.15.107:0/2995474160 shutdown_connections 2024-04-25T00:20:19.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.923+0000 7fb74489a640 1 --2- 172.21.15.107:0/2995474160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb740100fe0 0x7fb74010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.923+0000 7fb74489a640 1 --2- 172.21.15.107:0/2995474160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb740100620 0x7fb740100aa0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.923+0000 7fb74489a640 1 --2- 172.21.15.107:0/2995474160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb740106630 0x7fb740106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.923+0000 7fb74489a640 1 -- 172.21.15.107:0/2995474160 >> 172.21.15.107:0/2995474160 conn(0x7fb7400776a0 msgr2=0x7fb740077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.923+0000 7fb74489a640 1 -- 172.21.15.107:0/2995474160 shutdown_connections 2024-04-25T00:20:19.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.923+0000 7fb74489a640 1 -- 172.21.15.107:0/2995474160 wait complete. 2024-04-25T00:20:19.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.924+0000 7fb74489a640 1 Processor -- start 2024-04-25T00:20:19.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.924+0000 7fb74489a640 1 -- start start 2024-04-25T00:20:19.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.924+0000 7fb74489a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb740100620 0x7fb740198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.924+0000 7fb74489a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb740100fe0 0x7fb7401985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.924+0000 7fb74489a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb740106630 0x7fb74019c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.924+0000 7fb74489a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb740111b40 con 0x7fb740100fe0 2024-04-25T00:20:19.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.924+0000 7fb74489a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401119c0 con 0x7fb740100620 2024-04-25T00:20:19.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.924+0000 7fb74489a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb740111cc0 con 0x7fb740106630 2024-04-25T00:20:19.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.924+0000 7fb73dd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb740100fe0 0x7fb7401985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.924+0000 7fb73dd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb740100fe0 0x7fb7401985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39938/0 (socket says 172.21.15.107:39938) 2024-04-25T00:20:19.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.924+0000 7fb73dd9b640 1 -- 172.21.15.107:0/3803784581 learned_addr learned my addr 172.21.15.107:0/3803784581 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:19.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.924+0000 7fb73e59c640 1 --2- 172.21.15.107:0/3803784581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb740100620 0x7fb740198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.924+0000 7fb73ed9d640 1 --2- 172.21.15.107:0/3803784581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb740106630 0x7fb74019c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.924+0000 7fb73dd9b640 1 -- 172.21.15.107:0/3803784581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb740106630 msgr2=0x7fb74019c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.924+0000 7fb73dd9b640 1 --2- 172.21.15.107:0/3803784581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb740106630 0x7fb74019c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.924+0000 7fb73dd9b640 1 -- 172.21.15.107:0/3803784581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb740100620 msgr2=0x7fb740198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.924+0000 7fb73dd9b640 1 --2- 172.21.15.107:0/3803784581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb740100620 0x7fb740198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.924+0000 7fb73dd9b640 1 -- 172.21.15.107:0/3803784581 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb74019d050 con 0x7fb740100fe0 2024-04-25T00:20:19.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.925+0000 7fb73dd9b640 1 --2- 172.21.15.107:0/3803784581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb740100fe0 0x7fb7401985a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb730009230 tx=0x7fb730004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.925+0000 7fb7377fe640 1 -- 172.21.15.107:0/3803784581 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73003f030 con 0x7fb740100fe0 2024-04-25T00:20:19.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.925+0000 7fb74489a640 1 -- 172.21.15.107:0/3803784581 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb74019d2e0 con 0x7fb740100fe0 2024-04-25T00:20:19.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.925+0000 7fb7377fe640 1 -- 172.21.15.107:0/3803784581 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb730003ad0 con 0x7fb740100fe0 2024-04-25T00:20:19.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.925+0000 7fb7377fe640 1 -- 172.21.15.107:0/3803784581 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb730035070 con 0x7fb740100fe0 2024-04-25T00:20:19.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.925+0000 7fb74489a640 1 -- 172.21.15.107:0/3803784581 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7401a4bd0 con 0x7fb740100fe0 2024-04-25T00:20:19.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.927+0000 7fb7377fe640 1 -- 172.21.15.107:0/3803784581 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb730042030 con 0x7fb740100fe0 2024-04-25T00:20:19.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.927+0000 7fb7475a5740 1 Processor -- start 2024-04-25T00:20:19.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.927+0000 7fb7475a5740 1 -- start start 2024-04-25T00:20:19.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.928+0000 7fb7475a5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561110dcd580 0x561110dcd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.928+0000 7fb7475a5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcdea0 0x561110dd8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.928+0000 7fb7377fe640 1 --2- 172.21.15.107:0/3803784581 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb72c0a2d00 0x7fb72c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.928+0000 7fb7475a5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561110dd8570 0x561110dda960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.928+0000 7fb7475a5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561110d32670 con 0x561110dcdea0 2024-04-25T00:20:19.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.928+0000 7fb7475a5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561110cda970 con 0x561110dcd580 2024-04-25T00:20:19.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.928+0000 7fb7475a5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561110d2e630 con 0x561110dd8570 2024-04-25T00:20:19.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.928+0000 7fb73dd9b640 1 --2- 172.21.15.107:0/3803784581 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb72c0a2d00 0x7fb72c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.928+0000 7fb73e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcdea0 0x561110dd8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.928+0000 7fb73e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcdea0 0x561110dd8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39950/0 (socket says 172.21.15.107:39950) 2024-04-25T00:20:19.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.928+0000 7fb73e59c640 1 -- 172.21.15.107:0/3714313978 learned_addr learned my addr 172.21.15.107:0/3714313978 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:19.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.928+0000 7fb73ed9d640 1 --2- 172.21.15.107:0/3714313978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561110dcd580 0x561110dcd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.928+0000 7fb7377fe640 1 -- 172.21.15.107:0/3803784581 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fb7300efc70 con 0x7fb740100fe0 2024-04-25T00:20:19.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.928+0000 7fb73ed9d640 1 --2- 172.21.15.107:0/3714313978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561110dd8570 0x561110dda960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.928+0000 7fb73e59c640 1 -- 172.21.15.107:0/3714313978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561110dd8570 msgr2=0x561110dda960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.928+0000 7fb73e59c640 1 --2- 172.21.15.107:0/3714313978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561110dd8570 0x561110dda960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.928+0000 7fb73e59c640 1 -- 172.21.15.107:0/3714313978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561110dcd580 msgr2=0x561110dcd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.928+0000 7fb73e59c640 1 --2- 172.21.15.107:0/3714313978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561110dcd580 0x561110dcd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.928+0000 7fb73e59c640 1 -- 172.21.15.107:0/3714313978 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561110cfe400 con 0x561110dcdea0 2024-04-25T00:20:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.928+0000 7fb73dd9b640 1 --2- 172.21.15.107:0/3803784581 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb72c0a2d00 0x7fb72c0a51c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb7300bace0 tx=0x7fb73008dde0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.928+0000 7fb73e59c640 1 --2- 172.21.15.107:0/3714313978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcdea0 0x561110dd8030 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb72800b710 tx=0x7fb728004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6c927d1a59541c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.928+0000 7fb734ff9640 1 -- 172.21.15.107:0/3714313978 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728003b50 con 0x561110dcdea0 2024-04-25T00:20:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.929+0000 7fb734ff9640 1 -- 172.21.15.107:0/3714313978 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb728015070 con 0x561110dcdea0 2024-04-25T00:20:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.929+0000 7fb734ff9640 1 -- 172.21.15.107:0/3714313978 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728004d20 con 0x561110dcdea0 2024-04-25T00:20:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.929+0000 7fb7475a5740 1 -- 172.21.15.107:0/3714313978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcdea0 msgr2=0x561110dd8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.929+0000 7fb7475a5740 1 --2- 172.21.15.107:0/3714313978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcdea0 0x561110dd8030 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb72800b710 tx=0x7fb728004610 comp rx=0 tx=0).stop 2024-04-25T00:20:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.929+0000 7fb7475a5740 1 -- 172.21.15.107:0/3714313978 shutdown_connections 2024-04-25T00:20:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.929+0000 7fb7475a5740 1 --2- 172.21.15.107:0/3714313978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561110dd8570 0x561110dda960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.929+0000 7fb7475a5740 1 --2- 172.21.15.107:0/3714313978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcdea0 0x561110dd8030 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.929+0000 7fb7475a5740 1 --2- 172.21.15.107:0/3714313978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561110dcd580 0x561110dcd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.929+0000 7fb7475a5740 1 -- 172.21.15.107:0/3714313978 >> 172.21.15.107:0/3714313978 conn(0x561110dc98a0 msgr2=0x561110dc9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.929+0000 7fb7475a5740 1 -- 172.21.15.107:0/3714313978 shutdown_connections 2024-04-25T00:20:19.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.929+0000 7fb7475a5740 1 -- 172.21.15.107:0/3714313978 wait complete. 2024-04-25T00:20:19.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.930+0000 7fb7475a5740 1 -- start start 2024-04-25T00:20:19.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.930+0000 7fb7475a5740 1 Processor -- start 2024-04-25T00:20:19.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.930+0000 7fb7475a5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561110dcd580 0x561110dd7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.930+0000 7fb7475a5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcdea0 0x561110dd51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.930+0000 7fb7475a5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561110dd8570 0x561110dd5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.930+0000 7fb7475a5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561110ddb360 con 0x561110dcdea0 2024-04-25T00:20:19.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.930+0000 7fb7475a5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561110cee050 con 0x561110dd8570 2024-04-25T00:20:19.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.930+0000 7fb7475a5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561110ddb4e0 con 0x561110dcd580 2024-04-25T00:20:19.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.930+0000 7fb7475a5740 10 client.0 _ll_register_callbacks cb 0x561110a45be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:19.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.930+0000 7fb73e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcdea0 0x561110dd51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.930+0000 7fb73e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcdea0 0x561110dd51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39962/0 (socket says 172.21.15.107:39962) 2024-04-25T00:20:19.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.930+0000 7fb73e59c640 1 -- 172.21.15.107:0/3224942872 learned_addr learned my addr 172.21.15.107:0/3224942872 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:19.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.930+0000 7fb73ed9d640 1 --2- 172.21.15.107:0/3224942872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561110dd8570 0x561110dd5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.930+0000 7fb73ed9d640 1 --2- 172.21.15.107:0/3224942872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561110dcd580 0x561110dd7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.930+0000 7fb73e59c640 1 -- 172.21.15.107:0/3224942872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561110dcd580 msgr2=0x561110dd7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.930+0000 7fb73e59c640 1 --2- 172.21.15.107:0/3224942872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561110dcd580 0x561110dd7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.930+0000 7fb73e59c640 1 -- 172.21.15.107:0/3224942872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561110dd8570 msgr2=0x561110dd5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.930+0000 7fb73e59c640 1 --2- 172.21.15.107:0/3224942872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561110dd8570 0x561110dd5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.930+0000 7fb73e59c640 1 -- 172.21.15.107:0/3224942872 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561110cbfdc0 con 0x561110dcdea0 2024-04-25T00:20:19.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.930+0000 7fb73ed9d640 1 --2- 172.21.15.107:0/3224942872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561110dcd580 0x561110dd7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:20:19.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.930+0000 7fb73e59c640 1 --2- 172.21.15.107:0/3224942872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcdea0 0x561110dd51d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb7280039c0 tx=0x7fb72800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3443a6c5bb1664 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.930+0000 7fb725ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:20:19.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.931+0000 7fb725ffb640 1 -- 172.21.15.107:0/3224942872 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728028060 con 0x561110dcdea0 2024-04-25T00:20:19.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.931+0000 7fb725ffb640 1 -- 172.21.15.107:0/3224942872 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fb728011440 con 0x561110dcdea0 2024-04-25T00:20:19.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.931+0000 7fb7475a5740 1 -- 172.21.15.107:0/3224942872 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb728015070 con 0x561110dcdea0 2024-04-25T00:20:19.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.931+0000 7fb725ffb640 1 -- 172.21.15.107:0/3224942872 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb72802f050 con 0x561110dcdea0 2024-04-25T00:20:19.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.931+0000 7fb7475a5740 10 client.16476 fetch_fsmap learned FSMap version 28 2024-04-25T00:20:19.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.931+0000 7fb7475a5740 1 -- 172.21.15.107:0/3224942872 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561110a463f0 con 0x561110dcdea0 2024-04-25T00:20:19.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.931+0000 7fb725ffb640 1 -- 172.21.15.107:0/3224942872 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb728015070 con 0x561110dcdea0 2024-04-25T00:20:19.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.931+0000 7fb725ffb640 1 -- 172.21.15.107:0/3224942872 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fb728022070 con 0x561110dcdea0 2024-04-25T00:20:19.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.931+0000 7fb7475a5740 10 client.16476 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:20:19.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.931+0000 7fb7475a5740 10 client.16476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:20:19.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.931+0000 7fb7475a5740 20 client.16476 populate_metadata read hostname 'smithi107' 2024-04-25T00:20:19.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.931+0000 7fb7475a5740 1 --2- 172.21.15.107:0/3224942872 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x561110de8590 0x561110dea980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.931+0000 7fb7475a5740 4 client.16476 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:20:19.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.931+0000 7fb7475a5740 1 -- 172.21.15.107:0/3224942872 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561110d3fbb0 con 0x561110de8590 2024-04-25T00:20:19.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.931+0000 7fb73ed9d640 1 --2- 172.21.15.107:0/3224942872 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x561110de8590 0x561110dea980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.932+0000 7fb725ffb640 10 client.16476 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:19.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.932+0000 7fb73ed9d640 1 --2- 172.21.15.107:0/3224942872 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x561110de8590 0x561110dea980 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2912b88274095ca server_cookie=36446b0cb2905a6 in_seq=0 out_seq=0 2024-04-25T00:20:19.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.932+0000 7fb725ffb640 1 -- 172.21.15.107:0/3224942872 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561110d3fbb0 con 0x561110de8590 2024-04-25T00:20:19.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.932+0000 7fb725ffb640 10 client.16476 handle_command_reply: tid=0 2024-04-25T00:20:19.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.932+0000 7fb7475a5740 1 client.16476 shutdown 2024-04-25T00:20:19.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.932+0000 7fb7475a5740 1 -- 172.21.15.107:0/3224942872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcdea0 msgr2=0x561110dd51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.932+0000 7fb7475a5740 1 --2- 172.21.15.107:0/3224942872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcdea0 0x561110dd51d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb7280039c0 tx=0x7fb72800be20 comp rx=0 tx=0).stop 2024-04-25T00:20:19.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.932+0000 7fb7475a5740 1 -- 172.21.15.107:0/3224942872 shutdown_connections 2024-04-25T00:20:19.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.932+0000 7fb7475a5740 1 --2- 172.21.15.107:0/3224942872 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x561110de8590 0x561110dea980 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.932+0000 7fb7475a5740 1 --2- 172.21.15.107:0/3224942872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561110dd8570 0x561110dd5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.932+0000 7fb7475a5740 1 --2- 172.21.15.107:0/3224942872 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcdea0 0x561110dd51d0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.932+0000 7fb725ffb640 0 client.16476 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:19.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.932+0000 7fb7475a5740 1 --2- 172.21.15.107:0/3224942872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561110dcd580 0x561110dd7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.933+0000 7fb7475a5740 1 -- 172.21.15.107:0/3224942872 >> 172.21.15.107:0/3224942872 conn(0x561110dc98a0 msgr2=0x561110dc8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.933+0000 7fb7475a5740 1 -- 172.21.15.107:0/3224942872 shutdown_connections 2024-04-25T00:20:19.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.933+0000 7fb7475a5740 1 -- 172.21.15.107:0/3224942872 wait complete. 2024-04-25T00:20:19.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.933+0000 7fb7475a5740 20 client.16476 trim_cache size 0 max 16384 2024-04-25T00:20:19.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.946+0000 7fb7475a5740 1 Processor -- start 2024-04-25T00:20:19.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.946+0000 7fb7475a5740 1 -- start start 2024-04-25T00:20:19.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.946+0000 7fb7475a5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561110dcd580 0x561110dd0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.946+0000 7fb7475a5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcdea0 0x561110df0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.946+0000 7fb7475a5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561110dd8570 0x561110df0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.946+0000 7fb7475a5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401b1430 con 0x561110dcdea0 2024-04-25T00:20:19.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.946+0000 7fb7475a5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401aec40 con 0x561110dcd580 2024-04-25T00:20:19.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.946+0000 7fb7475a5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401aca90 con 0x561110dd8570 2024-04-25T00:20:19.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.946+0000 7fb73e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcdea0 0x561110df0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.946+0000 7fb73e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcdea0 0x561110df0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39970/0 (socket says 172.21.15.107:39970) 2024-04-25T00:20:19.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.946+0000 7fb73e59c640 1 -- 172.21.15.107:0/696866760 learned_addr learned my addr 172.21.15.107:0/696866760 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:19.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.946+0000 7fb73ed9d640 1 --2- 172.21.15.107:0/696866760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561110dcd580 0x561110dd0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.946+0000 7fb73e59c640 1 -- 172.21.15.107:0/696866760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561110dd8570 msgr2=0x561110df0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.946+0000 7fb73e59c640 1 --2- 172.21.15.107:0/696866760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561110dd8570 0x561110df0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.946+0000 7fb73e59c640 1 -- 172.21.15.107:0/696866760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561110dcd580 msgr2=0x561110dd0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.946+0000 7fb73e59c640 1 --2- 172.21.15.107:0/696866760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561110dcd580 0x561110dd0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.946+0000 7fb73e59c640 1 -- 172.21.15.107:0/696866760 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561110b107d0 con 0x561110dcdea0 2024-04-25T00:20:19.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.946+0000 7fb73e59c640 1 --2- 172.21.15.107:0/696866760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcdea0 0x561110df0010 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb728019a50 tx=0x7fb72800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b2069608d5f879 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.946+0000 7fb727fff640 1 -- 172.21.15.107:0/696866760 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728028070 con 0x561110dcdea0 2024-04-25T00:20:19.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.946+0000 7fb727fff640 1 -- 172.21.15.107:0/696866760 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb728015040 con 0x561110dcdea0 2024-04-25T00:20:19.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.946+0000 7fb727fff640 1 -- 172.21.15.107:0/696866760 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728011b80 con 0x561110dcdea0 2024-04-25T00:20:19.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.947+0000 7fb7475a5740 1 -- 172.21.15.107:0/696866760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcdea0 msgr2=0x561110df0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.947+0000 7fb7475a5740 1 --2- 172.21.15.107:0/696866760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcdea0 0x561110df0010 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb728019a50 tx=0x7fb72800b390 comp rx=0 tx=0).stop 2024-04-25T00:20:19.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.947+0000 7fb7475a5740 1 -- 172.21.15.107:0/696866760 shutdown_connections 2024-04-25T00:20:19.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.947+0000 7fb7475a5740 1 --2- 172.21.15.107:0/696866760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561110dd8570 0x561110df0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.947+0000 7fb7475a5740 1 --2- 172.21.15.107:0/696866760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcdea0 0x561110df0010 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.947+0000 7fb7475a5740 1 --2- 172.21.15.107:0/696866760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561110dcd580 0x561110dd0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.947+0000 7fb7475a5740 1 -- 172.21.15.107:0/696866760 >> 172.21.15.107:0/696866760 conn(0x561110dc98a0 msgr2=0x561110dcf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.947+0000 7fb7475a5740 1 -- 172.21.15.107:0/696866760 shutdown_connections 2024-04-25T00:20:19.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.947+0000 7fb7475a5740 1 -- 172.21.15.107:0/696866760 wait complete. 2024-04-25T00:20:19.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.947+0000 7fb7475a5740 1 -- start start 2024-04-25T00:20:19.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.947+0000 7fb7475a5740 1 Processor -- start 2024-04-25T00:20:19.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb7475a5740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcd580 0x561110dd2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb7475a5740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561110dcdea0 0x561110dd25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb7475a5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561110dd8570 0x561110dd2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb7475a5740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401ad100 con 0x561110dcd580 2024-04-25T00:20:19.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb7475a5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401ae570 con 0x561110dd8570 2024-04-25T00:20:19.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb7475a5740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb7401ad7d0 con 0x561110dcdea0 2024-04-25T00:20:19.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb7475a5740 10 client.0 _ll_register_callbacks cb 0x7fb738009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:19.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb73ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcd580 0x561110dd2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb73ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcd580 0x561110dd2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39984/0 (socket says 172.21.15.107:39984) 2024-04-25T00:20:19.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb73ed9d640 1 -- 172.21.15.107:0/4067204560 learned_addr learned my addr 172.21.15.107:0/4067204560 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:19.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb73ed9d640 1 --2- 172.21.15.107:0/4067204560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561110dd8570 0x561110dd2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb73e59c640 1 --2- 172.21.15.107:0/4067204560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561110dcdea0 0x561110dd25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb73ed9d640 1 -- 172.21.15.107:0/4067204560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561110dcdea0 msgr2=0x561110dd25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb73ed9d640 1 --2- 172.21.15.107:0/4067204560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561110dcdea0 0x561110dd25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb73ed9d640 1 -- 172.21.15.107:0/4067204560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561110dd8570 msgr2=0x561110dd2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:19.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb73ed9d640 1 --2- 172.21.15.107:0/4067204560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561110dd8570 0x561110dd2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb73ed9d640 1 -- 172.21.15.107:0/4067204560 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561110e4d930 con 0x561110dcd580 2024-04-25T00:20:19.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb73e59c640 1 --2- 172.21.15.107:0/4067204560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561110dcdea0 0x561110dd25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:20:19.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb73ed9d640 1 --2- 172.21.15.107:0/4067204560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcd580 0x561110dd2090 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x561110deafb0 tx=0x7fb738009cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe9205dc4f74dcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:19.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb725ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:20:19.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb725ffb640 1 -- 172.21.15.107:0/4067204560 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb738029060 con 0x561110dcd580 2024-04-25T00:20:19.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb725ffb640 1 -- 172.21.15.107:0/4067204560 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fb738013df0 con 0x561110dcd580 2024-04-25T00:20:19.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb7475a5740 1 -- 172.21.15.107:0/4067204560 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb728015040 con 0x561110dcd580 2024-04-25T00:20:19.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb725ffb640 1 -- 172.21.15.107:0/4067204560 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb738005510 con 0x561110dcd580 2024-04-25T00:20:19.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb7475a5740 10 client.16482 fetch_fsmap learned FSMap version 28 2024-04-25T00:20:19.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.948+0000 7fb7475a5740 1 -- 172.21.15.107:0/4067204560 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561110e4ec70 con 0x561110dcd580 2024-04-25T00:20:19.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.949+0000 7fb725ffb640 1 -- 172.21.15.107:0/4067204560 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb728015040 con 0x561110dcd580 2024-04-25T00:20:19.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.949+0000 7fb725ffb640 1 -- 172.21.15.107:0/4067204560 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fb73800f040 con 0x561110dcd580 2024-04-25T00:20:19.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.949+0000 7fb7475a5740 10 client.16482 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:20:19.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.949+0000 7fb7475a5740 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:20:19.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.949+0000 7fb7475a5740 20 client.16482 populate_metadata read hostname 'smithi107' 2024-04-25T00:20:19.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.949+0000 7fb7475a5740 1 --2- 172.21.15.107:0/4067204560 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x561110e4f430 0x561110e51850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:19.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.949+0000 7fb7475a5740 4 client.16482 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:20:19.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.949+0000 7fb7475a5740 1 -- 172.21.15.107:0/4067204560 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561110d39f00 con 0x561110e4f430 2024-04-25T00:20:19.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.949+0000 7fb73e59c640 1 --2- 172.21.15.107:0/4067204560 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x561110e4f430 0x561110e51850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:19.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.949+0000 7fb725ffb640 10 client.16482 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:19.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.949+0000 7fb73e59c640 1 --2- 172.21.15.107:0/4067204560 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x561110e4f430 0x561110e51850 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f568adbf812f05b8 server_cookie=6a95eafc0696966b in_seq=0 out_seq=0 2024-04-25T00:20:19.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.949+0000 7fb725ffb640 1 -- 172.21.15.107:0/4067204560 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561110d39f00 con 0x561110e4f430 2024-04-25T00:20:19.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.949+0000 7fb725ffb640 10 client.16482 handle_command_reply: tid=0 2024-04-25T00:20:19.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.949+0000 7fb7475a5740 1 client.16482 shutdown 2024-04-25T00:20:19.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.950+0000 7fb7475a5740 1 -- 172.21.15.107:0/4067204560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcd580 msgr2=0x561110dd2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.950+0000 7fb7475a5740 1 --2- 172.21.15.107:0/4067204560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcd580 0x561110dd2090 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x561110deafb0 tx=0x7fb738009cc0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.950+0000 7fb7475a5740 1 -- 172.21.15.107:0/4067204560 shutdown_connections 2024-04-25T00:20:19.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.950+0000 7fb7475a5740 1 --2- 172.21.15.107:0/4067204560 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x561110e4f430 0x561110e51850 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.950+0000 7fb7475a5740 1 --2- 172.21.15.107:0/4067204560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561110dd8570 0x561110dd2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.950+0000 7fb725ffb640 0 client.16482 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:19.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.950+0000 7fb7475a5740 1 --2- 172.21.15.107:0/4067204560 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561110dcdea0 0x561110dd25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.950+0000 7fb7475a5740 1 --2- 172.21.15.107:0/4067204560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561110dcd580 0x561110dd2090 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.950+0000 7fb7475a5740 1 -- 172.21.15.107:0/4067204560 >> 172.21.15.107:0/4067204560 conn(0x561110dc98a0 msgr2=0x561110dcea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.950+0000 7fb7475a5740 1 -- 172.21.15.107:0/4067204560 shutdown_connections 2024-04-25T00:20:19.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.950+0000 7fb7475a5740 1 -- 172.21.15.107:0/4067204560 wait complete. 2024-04-25T00:20:19.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.950+0000 7fb7475a5740 20 client.16482 trim_cache size 0 max 16384 2024-04-25T00:20:19.959 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:20:19.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.950+0000 7fb74489a640 1 -- 172.21.15.107:0/3803784581 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb72c0a2d00 msgr2=0x7fb72c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.950+0000 7fb74489a640 1 --2- 172.21.15.107:0/3803784581 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb72c0a2d00 0x7fb72c0a51c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb7300bace0 tx=0x7fb73008dde0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.950+0000 7fb74489a640 1 -- 172.21.15.107:0/3803784581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb740100fe0 msgr2=0x7fb7401985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:19.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.950+0000 7fb74489a640 1 --2- 172.21.15.107:0/3803784581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb740100fe0 0x7fb7401985a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb730009230 tx=0x7fb730004270 comp rx=0 tx=0).stop 2024-04-25T00:20:19.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.950+0000 7fb74489a640 1 -- 172.21.15.107:0/3803784581 shutdown_connections 2024-04-25T00:20:19.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.950+0000 7fb74489a640 1 --2- 172.21.15.107:0/3803784581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb740106630 0x7fb74019c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.950+0000 7fb74489a640 1 --2- 172.21.15.107:0/3803784581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb740100fe0 0x7fb7401985a0 secure :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb730009230 tx=0x7fb730004270 comp rx=0 tx=0).stop 2024-04-25T00:20:19.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.950+0000 7fb74489a640 1 --2- 172.21.15.107:0/3803784581 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb72c0a2d00 0x7fb72c0a51c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.950+0000 7fb74489a640 1 --2- 172.21.15.107:0/3803784581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb740100620 0x7fb740198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:19.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.950+0000 7fb74489a640 1 -- 172.21.15.107:0/3803784581 >> 172.21.15.107:0/3803784581 conn(0x7fb7400776a0 msgr2=0x7fb740103230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:19.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.950+0000 7fb74489a640 1 -- 172.21.15.107:0/3803784581 shutdown_connections 2024-04-25T00:20:19.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:19.950+0000 7fb74489a640 1 -- 172.21.15.107:0/3803784581 wait complete. 2024-04-25T00:20:20.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:20 smithi170 ceph-mon[32807]: pgmap v119: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T00:20:20.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:20 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/418194563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:20:20.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:20 smithi107 ceph-mon[29153]: pgmap v119: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T00:20:20.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:20 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/418194563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:20:20.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:20 smithi195 ceph-mon[32833]: pgmap v119: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T00:20:20.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:20 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/418194563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:20:20.962 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:20:20.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:20:20.962 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:20:21.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.074+0000 7f41668ee640 1 -- 172.21.15.107:0/2335284965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4160107720 msgr2=0x7f4160107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.074+0000 7f41668ee640 1 --2- 172.21.15.107:0/2335284965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4160107720 0x7f4160107b00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4150009930 tx=0x7f415002a420 comp rx=0 tx=0).stop 2024-04-25T00:20:21.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.074+0000 7f41668ee640 1 -- 172.21.15.107:0/2335284965 shutdown_connections 2024-04-25T00:20:21.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.074+0000 7f41668ee640 1 --2- 172.21.15.107:0/2335284965 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41601020d0 0x7f416010e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.074+0000 7f41668ee640 1 --2- 172.21.15.107:0/2335284965 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4160101710 0x7f4160101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.074+0000 7f41668ee640 1 --2- 172.21.15.107:0/2335284965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4160107720 0x7f4160107b00 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.074+0000 7f41668ee640 1 -- 172.21.15.107:0/2335284965 >> 172.21.15.107:0/2335284965 conn(0x7f4160078b90 msgr2=0x7f4160078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:21.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.074+0000 7f41668ee640 1 -- 172.21.15.107:0/2335284965 shutdown_connections 2024-04-25T00:20:21.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.074+0000 7f41668ee640 1 -- 172.21.15.107:0/2335284965 wait complete. 2024-04-25T00:20:21.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.075+0000 7f41668ee640 1 Processor -- start 2024-04-25T00:20:21.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.075+0000 7f41668ee640 1 -- start start 2024-04-25T00:20:21.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.075+0000 7f41668ee640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4160101710 0x7f4160073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.075+0000 7f41668ee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41601020d0 0x7f4160073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.075+0000 7f41668ee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4160107720 0x7f41600744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.075+0000 7f41668ee640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4160111b30 con 0x7f4160107720 2024-04-25T00:20:21.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.075+0000 7f41668ee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f41601119b0 con 0x7f41601020d0 2024-04-25T00:20:21.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.075+0000 7f41668ee640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4160111cb0 con 0x7f4160101710 2024-04-25T00:20:21.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.075+0000 7f4164e8b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4160107720 0x7f41600744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.075+0000 7f4164e8b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4160107720 0x7f41600744c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40004/0 (socket says 172.21.15.107:40004) 2024-04-25T00:20:21.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.075+0000 7f4164e8b640 1 -- 172.21.15.107:0/2280546172 learned_addr learned my addr 172.21.15.107:0/2280546172 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:21.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.076+0000 7f4164e8b640 1 -- 172.21.15.107:0/2280546172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4160101710 msgr2=0x7f4160073a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.076+0000 7f4164e8b640 1 --2- 172.21.15.107:0/2280546172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4160101710 0x7f4160073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.076+0000 7f4164e8b640 1 -- 172.21.15.107:0/2280546172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41601020d0 msgr2=0x7f4160073f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.076+0000 7f415f7fe640 1 --2- 172.21.15.107:0/2280546172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41601020d0 0x7f4160073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.076+0000 7f4164e8b640 1 --2- 172.21.15.107:0/2280546172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41601020d0 0x7f4160073f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.076+0000 7f4164e8b640 1 -- 172.21.15.107:0/2280546172 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41601a6530 con 0x7f4160107720 2024-04-25T00:20:21.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.076+0000 7f415f7fe640 1 --2- 172.21.15.107:0/2280546172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41601020d0 0x7f4160073f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:20:21.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.076+0000 7f4164e8b640 1 --2- 172.21.15.107:0/2280546172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4160107720 0x7f41600744c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f415800eaa0 tx=0x7f415800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:21.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.076+0000 7f415d7fa640 1 -- 172.21.15.107:0/2280546172 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415800cdb0 con 0x7f4160107720 2024-04-25T00:20:21.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.076+0000 7f41668ee640 1 -- 172.21.15.107:0/2280546172 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41601a6820 con 0x7f4160107720 2024-04-25T00:20:21.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.076+0000 7f415d7fa640 1 -- 172.21.15.107:0/2280546172 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4158005020 con 0x7f4160107720 2024-04-25T00:20:21.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.076+0000 7f41668ee640 1 -- 172.21.15.107:0/2280546172 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41601a6d70 con 0x7f4160107720 2024-04-25T00:20:21.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.076+0000 7f415d7fa640 1 -- 172.21.15.107:0/2280546172 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415801f060 con 0x7f4160107720 2024-04-25T00:20:21.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.078+0000 7f415d7fa640 1 -- 172.21.15.107:0/2280546172 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f41580054d0 con 0x7f4160107720 2024-04-25T00:20:21.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.079+0000 7f41695fc740 1 Processor -- start 2024-04-25T00:20:21.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.079+0000 7f41695fc740 1 -- start start 2024-04-25T00:20:21.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.079+0000 7f415d7fa640 1 --2- 172.21.15.107:0/2280546172 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f41480a2d80 0x7f41480a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.079+0000 7f41695fc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558895c5d980 0x558895c5bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.079+0000 7f41695fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c5c290 0x558895c5c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.079+0000 7f41695fc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558895c67010 0x558895c69400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.079+0000 7f41695fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558895bc02c0 con 0x558895c5c290 2024-04-25T00:20:21.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.079+0000 7f41695fc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558895b68910 con 0x558895c67010 2024-04-25T00:20:21.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.079+0000 7f41695fc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558895bbc280 con 0x558895c5d980 2024-04-25T00:20:21.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.079+0000 7f415f7fe640 1 --2- 172.21.15.107:0/2280546172 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f41480a2d80 0x7f41480a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.079+0000 7f415d7fa640 1 -- 172.21.15.107:0/2280546172 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f41580cfde0 con 0x7f4160107720 2024-04-25T00:20:21.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.079+0000 7f415f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c5c290 0x558895c5c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.079+0000 7f415f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c5c290 0x558895c5c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40012/0 (socket says 172.21.15.107:40012) 2024-04-25T00:20:21.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.079+0000 7f415f7fe640 1 -- 172.21.15.107:0/3832433835 learned_addr learned my addr 172.21.15.107:0/3832433835 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:21.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.079+0000 7f415ffff640 1 --2- 172.21.15.107:0/3832433835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558895c5d980 0x558895c5bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.079+0000 7f4164e8b640 1 --2- 172.21.15.107:0/3832433835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558895c67010 0x558895c69400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.079+0000 7f415f7fe640 1 -- 172.21.15.107:0/3832433835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558895c5d980 msgr2=0x558895c5bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:21.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.079+0000 7f415f7fe640 1 --2- 172.21.15.107:0/3832433835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558895c5d980 0x558895c5bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.079+0000 7f415f7fe640 1 -- 172.21.15.107:0/3832433835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558895c67010 msgr2=0x558895c69400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:21.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.079+0000 7f415f7fe640 1 --2- 172.21.15.107:0/3832433835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558895c67010 0x558895c69400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.079+0000 7f415f7fe640 1 -- 172.21.15.107:0/3832433835 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558895b79fb0 con 0x558895c5c290 2024-04-25T00:20:21.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.080+0000 7f415f7fe640 1 --2- 172.21.15.107:0/2280546172 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f41480a2d80 0x7f41480a5240 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f415400bfd0 tx=0x7f415400bb20 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:21.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.080+0000 7f415f7fe640 1 --2- 172.21.15.107:0/3832433835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c5c290 0x558895c5c710 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4154005d00 tx=0x7f415401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b308d2fa4485f0bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:21.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.080+0000 7f41467fc640 1 -- 172.21.15.107:0/3832433835 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154023070 con 0x558895c5c290 2024-04-25T00:20:21.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.080+0000 7f41467fc640 1 -- 172.21.15.107:0/3832433835 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4154010e40 con 0x558895c5c290 2024-04-25T00:20:21.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.080+0000 7f41467fc640 1 -- 172.21.15.107:0/3832433835 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154005260 con 0x558895c5c290 2024-04-25T00:20:21.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.080+0000 7f41695fc740 1 -- 172.21.15.107:0/3832433835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c5c290 msgr2=0x558895c5c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.080+0000 7f41695fc740 1 --2- 172.21.15.107:0/3832433835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c5c290 0x558895c5c710 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4154005d00 tx=0x7f415401a000 comp rx=0 tx=0).stop 2024-04-25T00:20:21.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.080+0000 7f41695fc740 1 -- 172.21.15.107:0/3832433835 shutdown_connections 2024-04-25T00:20:21.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.080+0000 7f41695fc740 1 --2- 172.21.15.107:0/3832433835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558895c67010 0x558895c69400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.080+0000 7f41695fc740 1 --2- 172.21.15.107:0/3832433835 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c5c290 0x558895c5c710 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.080+0000 7f41695fc740 1 --2- 172.21.15.107:0/3832433835 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558895c5d980 0x558895c5bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.080+0000 7f41695fc740 1 -- 172.21.15.107:0/3832433835 >> 172.21.15.107:0/3832433835 conn(0x558895c24950 msgr2=0x558895c587f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:21.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.081+0000 7f41695fc740 1 -- 172.21.15.107:0/3832433835 shutdown_connections 2024-04-25T00:20:21.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.081+0000 7f41695fc740 1 -- 172.21.15.107:0/3832433835 wait complete. 2024-04-25T00:20:21.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.081+0000 7f41695fc740 1 -- start start 2024-04-25T00:20:21.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.081+0000 7f41695fc740 1 Processor -- start 2024-04-25T00:20:21.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.081+0000 7f41695fc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558895c5c290 0x558895c63f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.081+0000 7f41695fc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558895c5d980 0x558895c644a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.081+0000 7f41695fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c67010 0x558895c649e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.081+0000 7f41695fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558895c69e40 con 0x558895c67010 2024-04-25T00:20:21.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.081+0000 7f41695fc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558895b7be90 con 0x558895c5d980 2024-04-25T00:20:21.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.081+0000 7f41695fc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558895c69fc0 con 0x558895c5c290 2024-04-25T00:20:21.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.081+0000 7f41695fc740 10 client.0 _ll_register_callbacks cb 0x558895bb7970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:21.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.081+0000 7f4164e8b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c67010 0x558895c649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.082+0000 7f415f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558895c5d980 0x558895c644a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.082+0000 7f4164e8b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c67010 0x558895c649e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40014/0 (socket says 172.21.15.107:40014) 2024-04-25T00:20:21.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.082+0000 7f415ffff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558895c5c290 0x558895c63f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.082+0000 7f4164e8b640 1 -- 172.21.15.107:0/1704785048 learned_addr learned my addr 172.21.15.107:0/1704785048 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:21.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.082+0000 7f4164e8b640 1 -- 172.21.15.107:0/1704785048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558895c5c290 msgr2=0x558895c63f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:21.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.082+0000 7f4164e8b640 1 --2- 172.21.15.107:0/1704785048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558895c5c290 0x558895c63f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.082+0000 7f4164e8b640 1 -- 172.21.15.107:0/1704785048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558895c5d980 msgr2=0x558895c644a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:21.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.082+0000 7f4164e8b640 1 --2- 172.21.15.107:0/1704785048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558895c5d980 0x558895c644a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.082+0000 7f4164e8b640 1 -- 172.21.15.107:0/1704785048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55889599c7e0 con 0x558895c67010 2024-04-25T00:20:21.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.082+0000 7f4164e8b640 1 --2- 172.21.15.107:0/1704785048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c67010 0x558895c649e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f415809bc30 tx=0x7f4158079430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c620d24c111c867 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:21.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.082+0000 7f413f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:20:21.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.082+0000 7f413f7fe640 1 -- 172.21.15.107:0/1704785048 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415809c630 con 0x558895c67010 2024-04-25T00:20:21.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.082+0000 7f413f7fe640 1 -- 172.21.15.107:0/1704785048 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f415807ad50 con 0x558895c67010 2024-04-25T00:20:21.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.082+0000 7f41695fc740 1 -- 172.21.15.107:0/1704785048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4154010e40 con 0x558895c67010 2024-04-25T00:20:21.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.082+0000 7f413f7fe640 1 -- 172.21.15.107:0/1704785048 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4158079140 con 0x558895c67010 2024-04-25T00:20:21.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.082+0000 7f41695fc740 10 client.16494 fetch_fsmap learned FSMap version 28 2024-04-25T00:20:21.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.082+0000 7f41695fc740 1 -- 172.21.15.107:0/1704785048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5588959fcf70 con 0x558895c67010 2024-04-25T00:20:21.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.082+0000 7f413f7fe640 1 -- 172.21.15.107:0/1704785048 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4154010e40 con 0x558895c67010 2024-04-25T00:20:21.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.083+0000 7f413f7fe640 1 -- 172.21.15.107:0/1704785048 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f41580ba030 con 0x558895c67010 2024-04-25T00:20:21.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.083+0000 7f41695fc740 10 client.16494 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:20:21.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.083+0000 7f41695fc740 10 client.16494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:20:21.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.083+0000 7f41695fc740 20 client.16494 populate_metadata read hostname 'smithi107' 2024-04-25T00:20:21.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.083+0000 7f41695fc740 1 --2- 172.21.15.107:0/1704785048 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x558895c77140 0x558895c79560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.083+0000 7f41695fc740 4 client.16494 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:20:21.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.083+0000 7f41695fc740 1 -- 172.21.15.107:0/1704785048 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558895b945a0 con 0x558895c77140 2024-04-25T00:20:21.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.083+0000 7f415ffff640 1 --2- 172.21.15.107:0/1704785048 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x558895c77140 0x558895c79560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.083+0000 7f415ffff640 1 --2- 172.21.15.107:0/1704785048 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x558895c77140 0x558895c79560 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=969b97aca799c50a server_cookie=189ccdc65aa069c7 in_seq=0 out_seq=0 2024-04-25T00:20:21.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.083+0000 7f413f7fe640 10 client.16494 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:21.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.084+0000 7f413f7fe640 1 -- 172.21.15.107:0/1704785048 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558895b945a0 con 0x558895c77140 2024-04-25T00:20:21.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.084+0000 7f413f7fe640 10 client.16494 handle_command_reply: tid=0 2024-04-25T00:20:21.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.084+0000 7f41695fc740 1 client.16494 shutdown 2024-04-25T00:20:21.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.084+0000 7f41695fc740 1 -- 172.21.15.107:0/1704785048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c67010 msgr2=0x558895c649e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.084+0000 7f41695fc740 1 --2- 172.21.15.107:0/1704785048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c67010 0x558895c649e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f415809bc30 tx=0x7f4158079430 comp rx=0 tx=0).stop 2024-04-25T00:20:21.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.084+0000 7f41695fc740 1 -- 172.21.15.107:0/1704785048 shutdown_connections 2024-04-25T00:20:21.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.084+0000 7f41695fc740 1 --2- 172.21.15.107:0/1704785048 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x558895c77140 0x558895c79560 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.084+0000 7f41695fc740 1 --2- 172.21.15.107:0/1704785048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c67010 0x558895c649e0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.084+0000 7f41695fc740 1 --2- 172.21.15.107:0/1704785048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558895c5d980 0x558895c644a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.084+0000 7f41695fc740 1 --2- 172.21.15.107:0/1704785048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558895c5c290 0x558895c63f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.084+0000 7f413f7fe640 0 client.16494 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:21.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.084+0000 7f41695fc740 1 -- 172.21.15.107:0/1704785048 >> 172.21.15.107:0/1704785048 conn(0x558895c24950 msgr2=0x558895c5e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:21.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.084+0000 7f41695fc740 1 -- 172.21.15.107:0/1704785048 shutdown_connections 2024-04-25T00:20:21.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.084+0000 7f41695fc740 1 -- 172.21.15.107:0/1704785048 wait complete. 2024-04-25T00:20:21.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.084+0000 7f41695fc740 20 client.16494 trim_cache size 0 max 16384 2024-04-25T00:20:21.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.101+0000 7f41695fc740 1 Processor -- start 2024-04-25T00:20:21.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.101+0000 7f41695fc740 1 -- start start 2024-04-25T00:20:21.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.102+0000 7f41695fc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558895c5c290 0x558895c72d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.102+0000 7f41695fc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558895c5d980 0x558895c7ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.102+0000 7f41695fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c67010 0x558895c7f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.102+0000 7f41695fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f41601b37a0 con 0x558895c67010 2024-04-25T00:20:21.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.102+0000 7f41695fc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f41601b0fb0 con 0x558895c5d980 2024-04-25T00:20:21.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.102+0000 7f41695fc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f41601aee00 con 0x558895c5c290 2024-04-25T00:20:21.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.102+0000 7f4164e8b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c67010 0x558895c7f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.102+0000 7f4164e8b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c67010 0x558895c7f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40026/0 (socket says 172.21.15.107:40026) 2024-04-25T00:20:21.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.102+0000 7f4164e8b640 1 -- 172.21.15.107:0/446523863 learned_addr learned my addr 172.21.15.107:0/446523863 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:21.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.102+0000 7f415ffff640 1 --2- 172.21.15.107:0/446523863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558895c5c290 0x558895c72d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.102+0000 7f4164e8b640 1 -- 172.21.15.107:0/446523863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558895c5c290 msgr2=0x558895c72d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:21.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.102+0000 7f4164e8b640 1 --2- 172.21.15.107:0/446523863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558895c5c290 0x558895c72d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.102+0000 7f4164e8b640 1 -- 172.21.15.107:0/446523863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558895c5d980 msgr2=0x558895c7ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:21.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.102+0000 7f4164e8b640 1 --2- 172.21.15.107:0/446523863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558895c5d980 0x558895c7ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.102+0000 7f4164e8b640 1 -- 172.21.15.107:0/446523863 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558895894170 con 0x558895c67010 2024-04-25T00:20:21.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.102+0000 7f415ffff640 1 --2- 172.21.15.107:0/446523863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558895c5c290 0x558895c72d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:20:21.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.102+0000 7f4164e8b640 1 --2- 172.21.15.107:0/446523863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c67010 0x558895c7f3b0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f41580107d0 tx=0x7f4158009270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf98e5d6a00c3b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:21.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.102+0000 7f4145ffb640 1 -- 172.21.15.107:0/446523863 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41580981d0 con 0x558895c67010 2024-04-25T00:20:21.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.102+0000 7f4145ffb640 1 -- 172.21.15.107:0/446523863 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f415809c630 con 0x558895c67010 2024-04-25T00:20:21.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.102+0000 7f4145ffb640 1 -- 172.21.15.107:0/446523863 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415807ad50 con 0x558895c67010 2024-04-25T00:20:21.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.103+0000 7f41695fc740 1 -- 172.21.15.107:0/446523863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c67010 msgr2=0x558895c7f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.103+0000 7f41695fc740 1 --2- 172.21.15.107:0/446523863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c67010 0x558895c7f3b0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f41580107d0 tx=0x7f4158009270 comp rx=0 tx=0).stop 2024-04-25T00:20:21.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.103+0000 7f41695fc740 1 -- 172.21.15.107:0/446523863 shutdown_connections 2024-04-25T00:20:21.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.103+0000 7f41695fc740 1 --2- 172.21.15.107:0/446523863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c67010 0x558895c7f3b0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.103+0000 7f41695fc740 1 --2- 172.21.15.107:0/446523863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558895c5d980 0x558895c7ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.103+0000 7f41695fc740 1 --2- 172.21.15.107:0/446523863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558895c5c290 0x558895c72d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.103+0000 7f41695fc740 1 -- 172.21.15.107:0/446523863 >> 172.21.15.107:0/446523863 conn(0x558895c24950 msgr2=0x558895c57950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:21.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.103+0000 7f41695fc740 1 -- 172.21.15.107:0/446523863 shutdown_connections 2024-04-25T00:20:21.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.103+0000 7f41695fc740 1 -- 172.21.15.107:0/446523863 wait complete. 2024-04-25T00:20:21.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.103+0000 7f41695fc740 1 -- start start 2024-04-25T00:20:21.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.104+0000 7f41695fc740 1 Processor -- start 2024-04-25T00:20:21.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.104+0000 7f41695fc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558895c5c290 0x558895c6c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.104+0000 7f41695fc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c5d980 0x558895c6c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.104+0000 7f41695fc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558895c67010 0x558895c6ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.104+0000 7f41695fc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f41601af470 con 0x558895c5d980 2024-04-25T00:20:21.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.104+0000 7f41695fc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f41601b08e0 con 0x558895c67010 2024-04-25T00:20:21.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.104+0000 7f41695fc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f41601afb40 con 0x558895c5c290 2024-04-25T00:20:21.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.104+0000 7f41695fc740 10 client.0 _ll_register_callbacks cb 0x7f415800cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:21.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.104+0000 7f415f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c5d980 0x558895c6c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.104+0000 7f415f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c5d980 0x558895c6c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40040/0 (socket says 172.21.15.107:40040) 2024-04-25T00:20:21.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.104+0000 7f415f7fe640 1 -- 172.21.15.107:0/3991474676 learned_addr learned my addr 172.21.15.107:0/3991474676 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:21.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.104+0000 7f4164e8b640 1 --2- 172.21.15.107:0/3991474676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558895c67010 0x558895c6ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.104+0000 7f415ffff640 1 --2- 172.21.15.107:0/3991474676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558895c5c290 0x558895c6c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.104+0000 7f415f7fe640 1 -- 172.21.15.107:0/3991474676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558895c5c290 msgr2=0x558895c6c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:21.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.104+0000 7f415f7fe640 1 --2- 172.21.15.107:0/3991474676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558895c5c290 0x558895c6c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.104+0000 7f415f7fe640 1 -- 172.21.15.107:0/3991474676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558895c67010 msgr2=0x558895c6ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:21.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.104+0000 7f415f7fe640 1 --2- 172.21.15.107:0/3991474676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558895c67010 0x558895c6ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.104+0000 7f415f7fe640 1 -- 172.21.15.107:0/3991474676 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558895cdd6f0 con 0x558895c5d980 2024-04-25T00:20:21.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.104+0000 7f415f7fe640 1 --2- 172.21.15.107:0/3991474676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c5d980 0x558895c6c7b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4154005d00 tx=0x7f4154010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3d84b350b40935 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:21.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.104+0000 7f413f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:20:21.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.106+0000 7f413f7fe640 1 -- 172.21.15.107:0/3991474676 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154023040 con 0x558895c5d980 2024-04-25T00:20:21.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.106+0000 7f413f7fe640 1 -- 172.21.15.107:0/3991474676 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f4154013240 con 0x558895c5d980 2024-04-25T00:20:21.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.106+0000 7f41695fc740 1 -- 172.21.15.107:0/3991474676 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f415809c630 con 0x558895c5d980 2024-04-25T00:20:21.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.106+0000 7f413f7fe640 1 -- 172.21.15.107:0/3991474676 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f415400ebc0 con 0x558895c5d980 2024-04-25T00:20:21.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.106+0000 7f41695fc740 10 client.16500 fetch_fsmap learned FSMap version 28 2024-04-25T00:20:21.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.106+0000 7f41695fc740 1 -- 172.21.15.107:0/3991474676 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558895cde8e0 con 0x558895c5d980 2024-04-25T00:20:21.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.106+0000 7f413f7fe640 1 -- 172.21.15.107:0/3991474676 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f415809c630 con 0x558895c5d980 2024-04-25T00:20:21.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.107+0000 7f413f7fe640 1 -- 172.21.15.107:0/3991474676 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f415402f070 con 0x558895c5d980 2024-04-25T00:20:21.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.107+0000 7f41695fc740 10 client.16500 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:20:21.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.107+0000 7f41695fc740 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:20:21.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.107+0000 7f41695fc740 20 client.16500 populate_metadata read hostname 'smithi107' 2024-04-25T00:20:21.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.107+0000 7f41695fc740 1 --2- 172.21.15.107:0/3991474676 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x558895cdeef0 0x558895ce1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.107+0000 7f41695fc740 4 client.16500 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:20:21.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.107+0000 7f41695fc740 1 -- 172.21.15.107:0/3991474676 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558895bcd800 con 0x558895cdeef0 2024-04-25T00:20:21.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.107+0000 7f415ffff640 1 --2- 172.21.15.107:0/3991474676 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x558895cdeef0 0x558895ce1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.107+0000 7f415ffff640 1 --2- 172.21.15.107:0/3991474676 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x558895cdeef0 0x558895ce1310 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5d7e8c2ec88cf44 server_cookie=c3060d68789fb23e in_seq=0 out_seq=0 2024-04-25T00:20:21.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.107+0000 7f413f7fe640 10 client.16500 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:21.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.108+0000 7f413f7fe640 1 -- 172.21.15.107:0/3991474676 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558895bcd800 con 0x558895cdeef0 2024-04-25T00:20:21.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.108+0000 7f413f7fe640 10 client.16500 handle_command_reply: tid=0 2024-04-25T00:20:21.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.108+0000 7f41695fc740 1 client.16500 shutdown 2024-04-25T00:20:21.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.109+0000 7f41695fc740 1 -- 172.21.15.107:0/3991474676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c5d980 msgr2=0x558895c6c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.109+0000 7f41695fc740 1 --2- 172.21.15.107:0/3991474676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c5d980 0x558895c6c7b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4154005d00 tx=0x7f4154010ef0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.109+0000 7f41695fc740 1 -- 172.21.15.107:0/3991474676 shutdown_connections 2024-04-25T00:20:21.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.109+0000 7f41695fc740 1 --2- 172.21.15.107:0/3991474676 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x558895cdeef0 0x558895ce1310 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.109+0000 7f41695fc740 1 --2- 172.21.15.107:0/3991474676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558895c67010 0x558895c6ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.109+0000 7f41695fc740 1 --2- 172.21.15.107:0/3991474676 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558895c5d980 0x558895c6c7b0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.109+0000 7f41695fc740 1 --2- 172.21.15.107:0/3991474676 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558895c5c290 0x558895c6c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.109+0000 7f41695fc740 1 -- 172.21.15.107:0/3991474676 >> 172.21.15.107:0/3991474676 conn(0x558895c24950 msgr2=0x558895c5dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:21.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.109+0000 7f413f7fe640 0 client.16500 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:21.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.109+0000 7f41695fc740 1 -- 172.21.15.107:0/3991474676 shutdown_connections 2024-04-25T00:20:21.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.109+0000 7f41695fc740 1 -- 172.21.15.107:0/3991474676 wait complete. 2024-04-25T00:20:21.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.109+0000 7f41695fc740 20 client.16500 trim_cache size 0 max 16384 2024-04-25T00:20:21.115 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:20:21.115 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-04-25T00:20:21.115 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "3b4a8480-b977-4bb8-99f7-b03f4a85c48f", 2024-04-25T00:20:21.116 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-04-25T00:20:21.116 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:20:21.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.109+0000 7f41668ee640 1 -- 172.21.15.107:0/2280546172 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f41480a2d80 msgr2=0x7f41480a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.109+0000 7f41668ee640 1 --2- 172.21.15.107:0/2280546172 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f41480a2d80 0x7f41480a5240 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f415400bfd0 tx=0x7f415400bb20 comp rx=0 tx=0).stop 2024-04-25T00:20:21.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.109+0000 7f41668ee640 1 -- 172.21.15.107:0/2280546172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4160107720 msgr2=0x7f41600744c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.109+0000 7f41668ee640 1 --2- 172.21.15.107:0/2280546172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4160107720 0x7f41600744c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f415800eaa0 tx=0x7f415800ef60 comp rx=0 tx=0).stop 2024-04-25T00:20:21.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.109+0000 7f41668ee640 1 -- 172.21.15.107:0/2280546172 shutdown_connections 2024-04-25T00:20:21.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.110+0000 7f41668ee640 1 --2- 172.21.15.107:0/2280546172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4160107720 0x7f41600744c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.110+0000 7f41668ee640 1 --2- 172.21.15.107:0/2280546172 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f41480a2d80 0x7f41480a5240 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.110+0000 7f41668ee640 1 --2- 172.21.15.107:0/2280546172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f41601020d0 0x7f4160073f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.110+0000 7f41668ee640 1 --2- 172.21.15.107:0/2280546172 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4160101710 0x7f4160073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.110+0000 7f41668ee640 1 -- 172.21.15.107:0/2280546172 >> 172.21.15.107:0/2280546172 conn(0x7f4160078b90 msgr2=0x7f41600ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:21.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.110+0000 7f41668ee640 1 -- 172.21.15.107:0/2280546172 shutdown_connections 2024-04-25T00:20:21.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.110+0000 7f41668ee640 1 -- 172.21.15.107:0/2280546172 wait complete. 2024-04-25T00:20:21.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:20:21.122 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:20:21.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.221+0000 7f15e2a64640 1 -- 172.21.15.107:0/451036036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15dc100990 msgr2=0x7f15dc100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.221+0000 7f15e2a64640 1 --2- 172.21.15.107:0/451036036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15dc100990 0x7f15dc100df0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f15c8009900 tx=0x7f15c802a410 comp rx=0 tx=0).stop 2024-04-25T00:20:21.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.221+0000 7f15e2a64640 1 -- 172.21.15.107:0/451036036 shutdown_connections 2024-04-25T00:20:21.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.221+0000 7f15e2a64640 1 --2- 172.21.15.107:0/451036036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15dc101330 0x7f15dc10d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.221+0000 7f15e2a64640 1 --2- 172.21.15.107:0/451036036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15dc100990 0x7f15dc100df0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.221+0000 7f15e2a64640 1 --2- 172.21.15.107:0/451036036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15dc106690 0x7f15dc106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.221+0000 7f15e2a64640 1 -- 172.21.15.107:0/451036036 >> 172.21.15.107:0/451036036 conn(0x7f15dc076440 msgr2=0x7f15dc076850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:21.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.221+0000 7f15e2a64640 1 -- 172.21.15.107:0/451036036 shutdown_connections 2024-04-25T00:20:21.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.221+0000 7f15e2a64640 1 -- 172.21.15.107:0/451036036 wait complete. 2024-04-25T00:20:21.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.222+0000 7f15e2a64640 1 Processor -- start 2024-04-25T00:20:21.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.222+0000 7f15e2a64640 1 -- start start 2024-04-25T00:20:21.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.222+0000 7f15e2a64640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15dc100990 0x7f15dc196e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.222+0000 7f15e2a64640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15dc101330 0x7f15dc1973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.222+0000 7f15e2a64640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15dc106690 0x7f15dc19b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.222+0000 7f15e2a64640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc1109e0 con 0x7f15dc100990 2024-04-25T00:20:21.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.222+0000 7f15e2a64640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc110860 con 0x7f15dc101330 2024-04-25T00:20:21.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.222+0000 7f15e2a64640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc110b60 con 0x7f15dc106690 2024-04-25T00:20:21.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.222+0000 7f15e0800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15dc100990 0x7f15dc196e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.222+0000 7f15e0800640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15dc100990 0x7f15dc196e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40044/0 (socket says 172.21.15.107:40044) 2024-04-25T00:20:21.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.222+0000 7f15e0800640 1 -- 172.21.15.107:0/2081493137 learned_addr learned my addr 172.21.15.107:0/2081493137 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:21.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.222+0000 7f15e1001640 1 --2- 172.21.15.107:0/2081493137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15dc106690 0x7f15dc19b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.222+0000 7f15dbfff640 1 --2- 172.21.15.107:0/2081493137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15dc101330 0x7f15dc1973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.222+0000 7f15e0800640 1 -- 172.21.15.107:0/2081493137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15dc106690 msgr2=0x7f15dc19b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.222+0000 7f15e0800640 1 --2- 172.21.15.107:0/2081493137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15dc106690 0x7f15dc19b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.222+0000 7f15e0800640 1 -- 172.21.15.107:0/2081493137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15dc101330 msgr2=0x7f15dc1973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.222+0000 7f15e0800640 1 --2- 172.21.15.107:0/2081493137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15dc101330 0x7f15dc1973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.222+0000 7f15e0800640 1 -- 172.21.15.107:0/2081493137 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15dc19be00 con 0x7f15dc100990 2024-04-25T00:20:21.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.222+0000 7f15dbfff640 1 --2- 172.21.15.107:0/2081493137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15dc101330 0x7f15dc1973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:20:21.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.223+0000 7f15e0800640 1 --2- 172.21.15.107:0/2081493137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15dc100990 0x7f15dc196e60 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f15d000c970 tx=0x7f15d000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:21.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.223+0000 7f15d9ffb640 1 -- 172.21.15.107:0/2081493137 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d0007bb0 con 0x7f15dc100990 2024-04-25T00:20:21.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.223+0000 7f15d9ffb640 1 -- 172.21.15.107:0/2081493137 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15d0007d50 con 0x7f15dc100990 2024-04-25T00:20:21.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.223+0000 7f15d9ffb640 1 -- 172.21.15.107:0/2081493137 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d000f450 con 0x7f15dc100990 2024-04-25T00:20:21.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.223+0000 7f15e2a64640 1 -- 172.21.15.107:0/2081493137 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15dc19c0f0 con 0x7f15dc100990 2024-04-25T00:20:21.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.223+0000 7f15e2a64640 1 -- 172.21.15.107:0/2081493137 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15dc1a3980 con 0x7f15dc100990 2024-04-25T00:20:21.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.225+0000 7f15d9ffb640 1 -- 172.21.15.107:0/2081493137 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f15d0005020 con 0x7f15dc100990 2024-04-25T00:20:21.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.225+0000 7f15e5774740 1 Processor -- start 2024-04-25T00:20:21.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.225+0000 7f15e5774740 1 -- start start 2024-04-25T00:20:21.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.225+0000 7f15e5774740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2b05c32e0 0x55e2b05c36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.225+0000 7f15e5774740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2b05c3c90 0x55e2b05d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.226+0000 7f15e5774740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05d0830 0x55e2b05d2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.226+0000 7f15d9ffb640 1 --2- 172.21.15.107:0/2081493137 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f15cc0a2d00 0x7f15cc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.226+0000 7f15e5774740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e2b05286b0 con 0x55e2b05d0830 2024-04-25T00:20:21.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.226+0000 7f15e5774740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e2b04d0880 con 0x55e2b05c3c90 2024-04-25T00:20:21.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.226+0000 7f15e5774740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e2b0524670 con 0x55e2b05c32e0 2024-04-25T00:20:21.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.226+0000 7f15dbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05d0830 0x55e2b05d2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.226+0000 7f15e1001640 1 --2- 172.21.15.107:0/2081493137 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f15cc0a2d00 0x7f15cc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.226+0000 7f15dbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05d0830 0x55e2b05d2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40056/0 (socket says 172.21.15.107:40056) 2024-04-25T00:20:21.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.226+0000 7f15e0800640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2b05c3c90 0x55e2b05d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.226+0000 7f15dbfff640 1 -- 172.21.15.107:0/2429307194 learned_addr learned my addr 172.21.15.107:0/2429307194 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:21.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.226+0000 7f15e1001640 1 --2- 172.21.15.107:0/2429307194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2b05c32e0 0x55e2b05c36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.226+0000 7f15d9ffb640 1 -- 172.21.15.107:0/2081493137 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f15d00cf5b0 con 0x7f15dc100990 2024-04-25T00:20:21.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.226+0000 7f15dbfff640 1 -- 172.21.15.107:0/2429307194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2b05c32e0 msgr2=0x55e2b05c36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:21.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.226+0000 7f15dbfff640 1 --2- 172.21.15.107:0/2429307194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2b05c32e0 0x55e2b05c36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.226+0000 7f15dbfff640 1 -- 172.21.15.107:0/2429307194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2b05c3c90 msgr2=0x55e2b05d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:21.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.226+0000 7f15dbfff640 1 --2- 172.21.15.107:0/2429307194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2b05c3c90 0x55e2b05d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.226+0000 7f15dbfff640 1 -- 172.21.15.107:0/2429307194 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2b03067d0 con 0x55e2b05d0830 2024-04-25T00:20:21.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.226+0000 7f15e1001640 1 --2- 172.21.15.107:0/2081493137 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f15cc0a2d00 0x7f15cc0a51c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f15d40077f0 tx=0x7f15d4007de0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:21.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.226+0000 7f15dbfff640 1 --2- 172.21.15.107:0/2429307194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05d0830 0x55e2b05d2c20 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f15c802a930 tx=0x7f15c80316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5264bdba5e36a286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:21.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.226+0000 7f15c6ffd640 1 -- 172.21.15.107:0/2429307194 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c803f070 con 0x55e2b05d0830 2024-04-25T00:20:21.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.226+0000 7f15c6ffd640 1 -- 172.21.15.107:0/2429307194 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15c80035f0 con 0x55e2b05d0830 2024-04-25T00:20:21.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.226+0000 7f15c6ffd640 1 -- 172.21.15.107:0/2429307194 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c8035070 con 0x55e2b05d0830 2024-04-25T00:20:21.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.227+0000 7f15e5774740 1 -- 172.21.15.107:0/2429307194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05d0830 msgr2=0x55e2b05d2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.227+0000 7f15e5774740 1 --2- 172.21.15.107:0/2429307194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05d0830 0x55e2b05d2c20 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f15c802a930 tx=0x7f15c80316e0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.227+0000 7f15e5774740 1 -- 172.21.15.107:0/2429307194 shutdown_connections 2024-04-25T00:20:21.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.227+0000 7f15e5774740 1 --2- 172.21.15.107:0/2429307194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05d0830 0x55e2b05d2c20 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.227+0000 7f15e5774740 1 --2- 172.21.15.107:0/2429307194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2b05c3c90 0x55e2b05d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.227+0000 7f15e5774740 1 --2- 172.21.15.107:0/2429307194 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2b05c32e0 0x55e2b05c36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.227+0000 7f15e5774740 1 -- 172.21.15.107:0/2429307194 >> 172.21.15.107:0/2429307194 conn(0x55e2b05bf950 msgr2=0x55e2b05bfd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:21.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.227+0000 7f15e5774740 1 -- 172.21.15.107:0/2429307194 shutdown_connections 2024-04-25T00:20:21.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.227+0000 7f15e5774740 1 -- 172.21.15.107:0/2429307194 wait complete. 2024-04-25T00:20:21.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.227+0000 7f15e5774740 1 -- start start 2024-04-25T00:20:21.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.227+0000 7f15e5774740 1 Processor -- start 2024-04-25T00:20:21.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.228+0000 7f15e5774740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2b05c32e0 0x55e2b05cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.228+0000 7f15e5774740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05c3c90 0x55e2b05cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.228+0000 7f15e5774740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2b05d0830 0x55e2b05ce200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.228+0000 7f15e5774740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e2b05d35c0 con 0x55e2b05c3c90 2024-04-25T00:20:21.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.228+0000 7f15e5774740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e2b04e3f50 con 0x55e2b05d0830 2024-04-25T00:20:21.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.228+0000 7f15e5774740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e2b05d3740 con 0x55e2b05c32e0 2024-04-25T00:20:21.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.228+0000 7f15e5774740 10 client.0 _ll_register_callbacks cb 0x55e2b051f000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:21.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.228+0000 7f15dbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05c3c90 0x55e2b05cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.228+0000 7f15dbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05c3c90 0x55e2b05cdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40072/0 (socket says 172.21.15.107:40072) 2024-04-25T00:20:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.228+0000 7f15dbfff640 1 -- 172.21.15.107:0/976262343 learned_addr learned my addr 172.21.15.107:0/976262343 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.228+0000 7f15e1001640 1 --2- 172.21.15.107:0/976262343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2b05d0830 0x55e2b05ce200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.228+0000 7f15e0800640 1 --2- 172.21.15.107:0/976262343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2b05c32e0 0x55e2b05cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.228+0000 7f15dbfff640 1 -- 172.21.15.107:0/976262343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2b05c32e0 msgr2=0x55e2b05cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.228+0000 7f15dbfff640 1 --2- 172.21.15.107:0/976262343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2b05c32e0 0x55e2b05cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.228+0000 7f15dbfff640 1 -- 172.21.15.107:0/976262343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2b05d0830 msgr2=0x55e2b05ce200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.228+0000 7f15dbfff640 1 --2- 172.21.15.107:0/976262343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2b05d0830 0x55e2b05ce200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.228+0000 7f15dbfff640 1 -- 172.21.15.107:0/976262343 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2b023c3f0 con 0x55e2b05c3c90 2024-04-25T00:20:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.228+0000 7f15e0800640 1 --2- 172.21.15.107:0/976262343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2b05c32e0 0x55e2b05cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:20:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.228+0000 7f15dbfff640 1 --2- 172.21.15.107:0/976262343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05c3c90 0x55e2b05cdcc0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f15c802aa00 tx=0x7f15c8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabc334be8aa186a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.228+0000 7f15bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:20:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.229+0000 7f15bbfff640 1 -- 172.21.15.107:0/976262343 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c8035070 con 0x55e2b05c3c90 2024-04-25T00:20:21.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.229+0000 7f15bbfff640 1 -- 172.21.15.107:0/976262343 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f15c80039b0 con 0x55e2b05c3c90 2024-04-25T00:20:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.229+0000 7f15e5774740 1 -- 172.21.15.107:0/976262343 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15c80035f0 con 0x55e2b05c3c90 2024-04-25T00:20:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.229+0000 7f15bbfff640 1 -- 172.21.15.107:0/976262343 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f15c8054050 con 0x55e2b05c3c90 2024-04-25T00:20:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.229+0000 7f15e5774740 10 client.16512 fetch_fsmap learned FSMap version 28 2024-04-25T00:20:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.229+0000 7f15e5774740 1 -- 172.21.15.107:0/976262343 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e2b01fc0f0 con 0x55e2b05c3c90 2024-04-25T00:20:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.229+0000 7f15bbfff640 1 -- 172.21.15.107:0/976262343 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15c80035f0 con 0x55e2b05c3c90 2024-04-25T00:20:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.229+0000 7f15bbfff640 1 -- 172.21.15.107:0/976262343 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f15c8043400 con 0x55e2b05c3c90 2024-04-25T00:20:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.229+0000 7f15e5774740 10 client.16512 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:20:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.229+0000 7f15e5774740 10 client.16512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:20:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.229+0000 7f15e5774740 20 client.16512 populate_metadata read hostname 'smithi107' 2024-04-25T00:20:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.229+0000 7f15e5774740 1 --2- 172.21.15.107:0/976262343 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e2b05e0940 0x55e2b05e2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.229+0000 7f15e5774740 4 client.16512 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:20:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.229+0000 7f15e5774740 1 -- 172.21.15.107:0/976262343 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2b0535bf0 con 0x55e2b05e0940 2024-04-25T00:20:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.229+0000 7f15e0800640 1 --2- 172.21.15.107:0/976262343 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e2b05e0940 0x55e2b05e2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.230+0000 7f15e0800640 1 --2- 172.21.15.107:0/976262343 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e2b05e0940 0x55e2b05e2d30 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=889752d06b20f96c server_cookie=9dc9d7e1e5d37ab9 in_seq=0 out_seq=0 2024-04-25T00:20:21.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.230+0000 7f15bbfff640 10 client.16512 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:21.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.230+0000 7f15bbfff640 1 -- 172.21.15.107:0/976262343 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e2b0535bf0 con 0x55e2b05e0940 2024-04-25T00:20:21.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.230+0000 7f15bbfff640 10 client.16512 handle_command_reply: tid=0 2024-04-25T00:20:21.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.230+0000 7f15e5774740 1 client.16512 shutdown 2024-04-25T00:20:21.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.230+0000 7f15e5774740 1 -- 172.21.15.107:0/976262343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05c3c90 msgr2=0x55e2b05cdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.231+0000 7f15e5774740 1 --2- 172.21.15.107:0/976262343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05c3c90 0x55e2b05cdcc0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f15c802aa00 tx=0x7f15c8004600 comp rx=0 tx=0).stop 2024-04-25T00:20:21.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.231+0000 7f15e5774740 1 -- 172.21.15.107:0/976262343 shutdown_connections 2024-04-25T00:20:21.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.231+0000 7f15e5774740 1 --2- 172.21.15.107:0/976262343 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e2b05e0940 0x55e2b05e2d30 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.231+0000 7f15e5774740 1 --2- 172.21.15.107:0/976262343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2b05d0830 0x55e2b05ce200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.231+0000 7f15e5774740 1 --2- 172.21.15.107:0/976262343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05c3c90 0x55e2b05cdcc0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.231+0000 7f15e5774740 1 --2- 172.21.15.107:0/976262343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2b05c32e0 0x55e2b05cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.231+0000 7f15bbfff640 0 client.16512 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:21.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.231+0000 7f15e5774740 1 -- 172.21.15.107:0/976262343 >> 172.21.15.107:0/976262343 conn(0x55e2b05bf950 msgr2=0x55e2b05c57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:21.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.231+0000 7f15e5774740 1 -- 172.21.15.107:0/976262343 shutdown_connections 2024-04-25T00:20:21.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.231+0000 7f15e5774740 1 -- 172.21.15.107:0/976262343 wait complete. 2024-04-25T00:20:21.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.231+0000 7f15e5774740 20 client.16512 trim_cache size 0 max 16384 2024-04-25T00:20:21.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.248+0000 7f15e5774740 1 Processor -- start 2024-04-25T00:20:21.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.248+0000 7f15e5774740 1 -- start start 2024-04-25T00:20:21.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.248+0000 7f15e5774740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2b05c32e0 0x55e2b05bf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.248+0000 7f15e5774740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05c3c90 0x55e2b05dc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.248+0000 7f15e5774740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2b05d0830 0x55e2b05cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.248+0000 7f15e5774740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc1af100 con 0x55e2b05c3c90 2024-04-25T00:20:21.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.248+0000 7f15e5774740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc1ac910 con 0x55e2b05d0830 2024-04-25T00:20:21.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.248+0000 7f15e5774740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc1aa760 con 0x55e2b05c32e0 2024-04-25T00:20:21.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.248+0000 7f15dbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05c3c90 0x55e2b05dc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.248+0000 7f15dbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05c3c90 0x55e2b05dc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40078/0 (socket says 172.21.15.107:40078) 2024-04-25T00:20:21.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.248+0000 7f15dbfff640 1 -- 172.21.15.107:0/3164935036 learned_addr learned my addr 172.21.15.107:0/3164935036 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:21.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.248+0000 7f15dbfff640 1 -- 172.21.15.107:0/3164935036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2b05c32e0 msgr2=0x55e2b05bf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:21.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.248+0000 7f15dbfff640 1 --2- 172.21.15.107:0/3164935036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2b05c32e0 0x55e2b05bf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.248+0000 7f15dbfff640 1 -- 172.21.15.107:0/3164935036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2b05d0830 msgr2=0x55e2b05cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:21.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.248+0000 7f15dbfff640 1 --2- 172.21.15.107:0/3164935036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2b05d0830 0x55e2b05cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.248+0000 7f15dbfff640 1 -- 172.21.15.107:0/3164935036 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2b05b1090 con 0x55e2b05c3c90 2024-04-25T00:20:21.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.248+0000 7f15dbfff640 1 --2- 172.21.15.107:0/3164935036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05c3c90 0x55e2b05dc4c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f15c80437d0 tx=0x7f15c8005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9efa5b7613ee2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:21.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.249+0000 7f15c67fc640 1 -- 172.21.15.107:0/3164935036 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c8035040 con 0x55e2b05c3c90 2024-04-25T00:20:21.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.249+0000 7f15c67fc640 1 -- 172.21.15.107:0/3164935036 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15c8043400 con 0x55e2b05c3c90 2024-04-25T00:20:21.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.249+0000 7f15c67fc640 1 -- 172.21.15.107:0/3164935036 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c803f040 con 0x55e2b05c3c90 2024-04-25T00:20:21.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.249+0000 7f15e5774740 1 -- 172.21.15.107:0/3164935036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05c3c90 msgr2=0x55e2b05dc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.249+0000 7f15e5774740 1 --2- 172.21.15.107:0/3164935036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05c3c90 0x55e2b05dc4c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f15c80437d0 tx=0x7f15c8005950 comp rx=0 tx=0).stop 2024-04-25T00:20:21.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.249+0000 7f15e5774740 1 -- 172.21.15.107:0/3164935036 shutdown_connections 2024-04-25T00:20:21.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.249+0000 7f15e5774740 1 --2- 172.21.15.107:0/3164935036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2b05d0830 0x55e2b05cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.249+0000 7f15e5774740 1 --2- 172.21.15.107:0/3164935036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05c3c90 0x55e2b05dc4c0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.249+0000 7f15e5774740 1 --2- 172.21.15.107:0/3164935036 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2b05c32e0 0x55e2b05bf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.249+0000 7f15e5774740 1 -- 172.21.15.107:0/3164935036 >> 172.21.15.107:0/3164935036 conn(0x55e2b05bf950 msgr2=0x55e2b05cd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:21.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.249+0000 7f15e5774740 1 -- 172.21.15.107:0/3164935036 shutdown_connections 2024-04-25T00:20:21.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.249+0000 7f15e5774740 1 -- 172.21.15.107:0/3164935036 wait complete. 2024-04-25T00:20:21.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.249+0000 7f15e5774740 1 -- start start 2024-04-25T00:20:21.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.250+0000 7f15e5774740 1 Processor -- start 2024-04-25T00:20:21.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.250+0000 7f15e5774740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2b05c32e0 0x55e2b05d89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.250+0000 7f15e5774740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2b05c3c90 0x55e2b05d8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.250+0000 7f15e5774740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05d0830 0x55e2b05c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.250+0000 7f15e5774740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc1aadd0 con 0x55e2b05d0830 2024-04-25T00:20:21.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.250+0000 7f15e5774740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc1ac240 con 0x55e2b05c3c90 2024-04-25T00:20:21.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.250+0000 7f15e5774740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f15dc1ab4a0 con 0x55e2b05c32e0 2024-04-25T00:20:21.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.250+0000 7f15e5774740 10 client.0 _ll_register_callbacks cb 0x7f15d0020ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:21.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.250+0000 7f15e1001640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05d0830 0x55e2b05c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.250+0000 7f15e1001640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05d0830 0x55e2b05c9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40084/0 (socket says 172.21.15.107:40084) 2024-04-25T00:20:21.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.250+0000 7f15e1001640 1 -- 172.21.15.107:0/2248578305 learned_addr learned my addr 172.21.15.107:0/2248578305 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:21.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.250+0000 7f15dbfff640 1 --2- 172.21.15.107:0/2248578305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2b05c3c90 0x55e2b05d8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.250+0000 7f15e0800640 1 --2- 172.21.15.107:0/2248578305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2b05c32e0 0x55e2b05d89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.250+0000 7f15e1001640 1 -- 172.21.15.107:0/2248578305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2b05c32e0 msgr2=0x55e2b05d89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:21.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.250+0000 7f15e1001640 1 --2- 172.21.15.107:0/2248578305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2b05c32e0 0x55e2b05d89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.250+0000 7f15e1001640 1 -- 172.21.15.107:0/2248578305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2b05c3c90 msgr2=0x55e2b05d8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:21.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.250+0000 7f15e1001640 1 --2- 172.21.15.107:0/2248578305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2b05c3c90 0x55e2b05d8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.250+0000 7f15e1001640 1 -- 172.21.15.107:0/2248578305 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2b05ca650 con 0x55e2b05d0830 2024-04-25T00:20:21.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.251+0000 7f15e1001640 1 --2- 172.21.15.107:0/2248578305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05d0830 0x55e2b05c9e20 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f15d4011da0 tx=0x7f15d4007520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84fbf4655ee6a825 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:21.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.251+0000 7f15bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:20:21.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.251+0000 7f15bbfff640 1 -- 172.21.15.107:0/2248578305 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4025070 con 0x55e2b05d0830 2024-04-25T00:20:21.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.251+0000 7f15bbfff640 1 -- 172.21.15.107:0/2248578305 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f15d4004d20 con 0x55e2b05d0830 2024-04-25T00:20:21.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.251+0000 7f15e5774740 1 -- 172.21.15.107:0/2248578305 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15c8043400 con 0x55e2b05d0830 2024-04-25T00:20:21.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.251+0000 7f15bbfff640 1 -- 172.21.15.107:0/2248578305 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f15d4032050 con 0x55e2b05d0830 2024-04-25T00:20:21.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.251+0000 7f15e5774740 10 client.16518 fetch_fsmap learned FSMap version 28 2024-04-25T00:20:21.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.251+0000 7f15e5774740 1 -- 172.21.15.107:0/2248578305 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e2b06466a0 con 0x55e2b05d0830 2024-04-25T00:20:21.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.251+0000 7f15bbfff640 1 -- 172.21.15.107:0/2248578305 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15c8043400 con 0x55e2b05d0830 2024-04-25T00:20:21.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.251+0000 7f15bbfff640 1 -- 172.21.15.107:0/2248578305 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f15d4029530 con 0x55e2b05d0830 2024-04-25T00:20:21.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.251+0000 7f15e5774740 10 client.16518 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:20:21.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.251+0000 7f15e5774740 10 client.16518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:20:21.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.251+0000 7f15e5774740 20 client.16518 populate_metadata read hostname 'smithi107' 2024-04-25T00:20:21.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.251+0000 7f15e5774740 1 --2- 172.21.15.107:0/2248578305 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e2b0646eb0 0x55e2b06492a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.251+0000 7f15e5774740 4 client.16518 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:20:21.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.251+0000 7f15e5774740 1 -- 172.21.15.107:0/2248578305 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e2b052ff40 con 0x55e2b0646eb0 2024-04-25T00:20:21.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.252+0000 7f15e0800640 1 --2- 172.21.15.107:0/2248578305 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e2b0646eb0 0x55e2b06492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.252+0000 7f15e0800640 1 --2- 172.21.15.107:0/2248578305 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e2b0646eb0 0x55e2b06492a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46bb762e5fab028 server_cookie=f96e811426029f6b in_seq=0 out_seq=0 2024-04-25T00:20:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.252+0000 7f15bbfff640 10 client.16518 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.252+0000 7f15bbfff640 1 -- 172.21.15.107:0/2248578305 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e2b052ff40 con 0x55e2b0646eb0 2024-04-25T00:20:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.252+0000 7f15bbfff640 10 client.16518 handle_command_reply: tid=0 2024-04-25T00:20:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.252+0000 7f15e5774740 1 client.16518 shutdown 2024-04-25T00:20:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.252+0000 7f15e5774740 1 -- 172.21.15.107:0/2248578305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05d0830 msgr2=0x55e2b05c9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.252+0000 7f15e5774740 1 --2- 172.21.15.107:0/2248578305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05d0830 0x55e2b05c9e20 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f15d4011da0 tx=0x7f15d4007520 comp rx=0 tx=0).stop 2024-04-25T00:20:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.252+0000 7f15e5774740 1 -- 172.21.15.107:0/2248578305 shutdown_connections 2024-04-25T00:20:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.252+0000 7f15e5774740 1 --2- 172.21.15.107:0/2248578305 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e2b0646eb0 0x55e2b06492a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.252+0000 7f15e5774740 1 --2- 172.21.15.107:0/2248578305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2b05d0830 0x55e2b05c9e20 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.252+0000 7f15e5774740 1 --2- 172.21.15.107:0/2248578305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2b05c3c90 0x55e2b05d8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.252+0000 7f15bbfff640 0 client.16518 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.252+0000 7f15e5774740 1 --2- 172.21.15.107:0/2248578305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2b05c32e0 0x55e2b05d89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.252+0000 7f15e5774740 1 -- 172.21.15.107:0/2248578305 >> 172.21.15.107:0/2248578305 conn(0x55e2b05bf950 msgr2=0x55e2b05cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.252+0000 7f15e5774740 1 -- 172.21.15.107:0/2248578305 shutdown_connections 2024-04-25T00:20:21.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.252+0000 7f15e5774740 1 -- 172.21.15.107:0/2248578305 wait complete. 2024-04-25T00:20:21.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.252+0000 7f15e5774740 20 client.16518 trim_cache size 0 max 16384 2024-04-25T00:20:21.261 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:20:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:20:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-04-25T00:20:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: "3b4a8480-b977-4bb8-99f7-b03f4a85c48f": { 2024-04-25T00:20:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-04-25T00:20:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "3b4a8480-b977-4bb8-99f7-b03f4a85c48f", 2024-04-25T00:20:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-04-25T00:20:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:20:21.261 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:20:21.261 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:20:21.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.253+0000 7f15e2a64640 1 -- 172.21.15.107:0/2081493137 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f15cc0a2d00 msgr2=0x7f15cc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.253+0000 7f15e2a64640 1 --2- 172.21.15.107:0/2081493137 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f15cc0a2d00 0x7f15cc0a51c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f15d40077f0 tx=0x7f15d4007de0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.253+0000 7f15e2a64640 1 -- 172.21.15.107:0/2081493137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15dc100990 msgr2=0x7f15dc196e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.253+0000 7f15e2a64640 1 --2- 172.21.15.107:0/2081493137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15dc100990 0x7f15dc196e60 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f15d000c970 tx=0x7f15d000ce30 comp rx=0 tx=0).stop 2024-04-25T00:20:21.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.253+0000 7f15e2a64640 1 -- 172.21.15.107:0/2081493137 shutdown_connections 2024-04-25T00:20:21.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.253+0000 7f15e2a64640 1 --2- 172.21.15.107:0/2081493137 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15dc106690 0x7f15dc19b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.253+0000 7f15e2a64640 1 --2- 172.21.15.107:0/2081493137 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f15cc0a2d00 0x7f15cc0a51c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.253+0000 7f15e2a64640 1 --2- 172.21.15.107:0/2081493137 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f15dc101330 0x7f15dc1973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.253+0000 7f15e2a64640 1 --2- 172.21.15.107:0/2081493137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15dc100990 0x7f15dc196e60 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.253+0000 7f15e2a64640 1 -- 172.21.15.107:0/2081493137 >> 172.21.15.107:0/2081493137 conn(0x7f15dc076440 msgr2=0x7f15dc0ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:21.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.253+0000 7f15e2a64640 1 -- 172.21.15.107:0/2081493137 shutdown_connections 2024-04-25T00:20:21.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:21.253+0000 7f15e2a64640 1 -- 172.21.15.107:0/2081493137 wait complete. 2024-04-25T00:20:21.265 INFO:tasks.cephfs.filesystem:scrub status for tag:3b4a8480-b977-4bb8-99f7-b03f4a85c48f - {'path': '/', 'tag': '3b4a8480-b977-4bb8-99f7-b03f4a85c48f', 'options': 'recursive,force'} 2024-04-25T00:20:22.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:22 smithi170 ceph-mon[32807]: pgmap v120: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T00:20:22.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:22 smithi170 ceph-mon[32807]: scrub queued for path: / 2024-04-25T00:20:22.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:22 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:20:22.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:22 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:20:22.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:22 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:20:22.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:22 smithi107 ceph-mon[29153]: pgmap v120: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T00:20:22.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:22 smithi107 ceph-mon[29153]: scrub queued for path: / 2024-04-25T00:20:22.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:22 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:20:22.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:22 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:20:22.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:22 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:20:22.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:22 smithi195 ceph-mon[32833]: pgmap v120: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T00:20:22.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:22 smithi195 ceph-mon[32833]: scrub queued for path: / 2024-04-25T00:20:22.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:22 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:20:22.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:22 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:20:22.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:22 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:20:23.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:23 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:20:23.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:23 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:20:23.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:23 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:20:24.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:24 smithi170 ceph-mon[32807]: pgmap v121: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T00:20:24.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:24 smithi107 ceph-mon[29153]: pgmap v121: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T00:20:24.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:24 smithi195 ceph-mon[32833]: pgmap v121: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T00:20:26.437 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:26 smithi170 ceph-mon[32807]: pgmap v122: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:20:26.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:26 smithi107 ceph-mon[29153]: pgmap v122: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:20:26.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:26 smithi195 ceph-mon[32833]: pgmap v122: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:20:28.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:28 smithi170 ceph-mon[32807]: pgmap v123: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:20:28.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:28 smithi107 ceph-mon[29153]: pgmap v123: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:20:28.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:28 smithi195 ceph-mon[32833]: pgmap v123: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:20:30.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:30 smithi170 ceph-mon[32807]: pgmap v124: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:20:30.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:30 smithi107 ceph-mon[29153]: pgmap v124: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:20:30.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:30 smithi195 ceph-mon[32833]: pgmap v124: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:20:32.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:32 smithi170 ceph-mon[32807]: pgmap v125: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:20:32.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:32 smithi107 ceph-mon[29153]: pgmap v125: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:20:32.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:32 smithi195 ceph-mon[32833]: pgmap v125: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:20:34.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:34 smithi170 ceph-mon[32807]: pgmap v126: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:20:34.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:34 smithi107 ceph-mon[29153]: pgmap v126: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:20:34.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:34 smithi195 ceph-mon[32833]: pgmap v126: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:20:35.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:35 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:20:35.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:35 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:20:35.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:35 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:20:36.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:36 smithi170 ceph-mon[32807]: pgmap v127: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:20:36.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:36 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:20:36.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:36 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:20:36.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:36 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:20:36.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:36 smithi107 ceph-mon[29153]: pgmap v127: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:20:36.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:36 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:20:36.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:36 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:20:36.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:36 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:20:36.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:36 smithi195 ceph-mon[32833]: pgmap v127: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:20:36.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:36 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:20:36.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:36 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:20:36.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:36 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:20:38.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:38 smithi170 ceph-mon[32807]: pgmap v128: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:20:38.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:38 smithi107 ceph-mon[29153]: pgmap v128: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:20:38.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:38 smithi195 ceph-mon[32833]: pgmap v128: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:20:40.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:40 smithi170 ceph-mon[32807]: pgmap v129: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:20:40.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:40 smithi107 ceph-mon[29153]: pgmap v129: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:20:40.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:40 smithi195 ceph-mon[32833]: pgmap v129: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:20:42.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:42 smithi170 ceph-mon[32807]: pgmap v130: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:20:42.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:42 smithi107 ceph-mon[29153]: pgmap v130: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:20:42.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:42 smithi195 ceph-mon[32833]: pgmap v130: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:20:44.409 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:44 smithi195 ceph-mon[32833]: pgmap v131: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:20:44.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:44 smithi170 ceph-mon[32807]: pgmap v131: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:20:44.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:44 smithi107 ceph-mon[29153]: pgmap v131: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:20:46.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:46 smithi170 ceph-mon[32807]: pgmap v132: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:20:46.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:46 smithi107 ceph-mon[29153]: pgmap v132: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:20:46.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:46 smithi195 ceph-mon[32833]: pgmap v132: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:20:48.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:48 smithi170 ceph-mon[32807]: pgmap v133: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:20:48.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:48 smithi107 ceph-mon[29153]: pgmap v133: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:20:48.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:48 smithi195 ceph-mon[32833]: pgmap v133: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:20:50.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:50 smithi170 ceph-mon[32807]: pgmap v134: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:20:50.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:50 smithi107 ceph-mon[29153]: pgmap v134: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:20:50.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:50 smithi195 ceph-mon[32833]: pgmap v134: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:20:51.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:20:51.267 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:20:51.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.379+0000 7f87913ce640 1 -- 172.21.15.107:0/2442489337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f878c100fd0 msgr2=0x7f878c10d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.379+0000 7f87913ce640 1 --2- 172.21.15.107:0/2442489337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f878c100fd0 0x7f878c10d4e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8784009cd0 tx=0x7f878402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.379+0000 7f87913ce640 1 -- 172.21.15.107:0/2442489337 shutdown_connections 2024-04-25T00:20:51.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.379+0000 7f87913ce640 1 --2- 172.21.15.107:0/2442489337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f878c100fd0 0x7f878c10d4e0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.379+0000 7f87913ce640 1 --2- 172.21.15.107:0/2442489337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f878c100610 0x7f878c100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.379+0000 7f87913ce640 1 --2- 172.21.15.107:0/2442489337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f878c1066b0 0x7f878c106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.379+0000 7f87913ce640 1 -- 172.21.15.107:0/2442489337 >> 172.21.15.107:0/2442489337 conn(0x7f878c0769c0 msgr2=0x7f878c076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:51.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.379+0000 7f87913ce640 1 -- 172.21.15.107:0/2442489337 shutdown_connections 2024-04-25T00:20:51.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.379+0000 7f87913ce640 1 -- 172.21.15.107:0/2442489337 wait complete. 2024-04-25T00:20:51.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.380+0000 7f87913ce640 1 Processor -- start 2024-04-25T00:20:51.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.380+0000 7f87913ce640 1 -- start start 2024-04-25T00:20:51.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.380+0000 7f87913ce640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f878c100610 0x7f878c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.380+0000 7f87913ce640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f878c100fd0 0x7f878c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.380+0000 7f87913ce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f878c1066b0 0x7f878c19c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.380+0000 7f87913ce640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f878c110a30 con 0x7f878c1066b0 2024-04-25T00:20:51.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.380+0000 7f87913ce640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f878c1108b0 con 0x7f878c100610 2024-04-25T00:20:51.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.380+0000 7f87913ce640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f878c110bb0 con 0x7f878c100fd0 2024-04-25T00:20:51.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.380+0000 7f878b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f878c1066b0 0x7f878c19c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.380+0000 7f878b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f878c1066b0 0x7f878c19c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43872/0 (socket says 172.21.15.107:43872) 2024-04-25T00:20:51.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.380+0000 7f878b7fe640 1 -- 172.21.15.107:0/4145947937 learned_addr learned my addr 172.21.15.107:0/4145947937 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:51.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.380+0000 7f878affd640 1 --2- 172.21.15.107:0/4145947937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f878c100610 0x7f878c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.380+0000 7f8783fff640 1 --2- 172.21.15.107:0/4145947937 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f878c100fd0 0x7f878c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.381+0000 7f878b7fe640 1 -- 172.21.15.107:0/4145947937 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f878c100fd0 msgr2=0x7f878c198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.381+0000 7f878b7fe640 1 --2- 172.21.15.107:0/4145947937 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f878c100fd0 0x7f878c198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.381+0000 7f878b7fe640 1 -- 172.21.15.107:0/4145947937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f878c100610 msgr2=0x7f878c197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.381+0000 7f878b7fe640 1 --2- 172.21.15.107:0/4145947937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f878c100610 0x7f878c197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.381+0000 7f878b7fe640 1 -- 172.21.15.107:0/4145947937 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f878c19cf80 con 0x7f878c1066b0 2024-04-25T00:20:51.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.381+0000 7f878b7fe640 1 --2- 172.21.15.107:0/4145947937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f878c1066b0 0x7f878c19c850 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8784005cc0 tx=0x7f8784003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:51.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.381+0000 7f8788ff9640 1 -- 172.21.15.107:0/4145947937 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8784004040 con 0x7f878c1066b0 2024-04-25T00:20:51.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.381+0000 7f8788ff9640 1 -- 172.21.15.107:0/4145947937 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87840046c0 con 0x7f878c1066b0 2024-04-25T00:20:51.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.381+0000 7f8788ff9640 1 -- 172.21.15.107:0/4145947937 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8784046480 con 0x7f878c1066b0 2024-04-25T00:20:51.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.381+0000 7f87913ce640 1 -- 172.21.15.107:0/4145947937 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f878c19d210 con 0x7f878c1066b0 2024-04-25T00:20:51.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.381+0000 7f87913ce640 1 -- 172.21.15.107:0/4145947937 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f878c1a4b00 con 0x7f878c1066b0 2024-04-25T00:20:51.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.383+0000 7f8788ff9640 1 -- 172.21.15.107:0/4145947937 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f87840305a0 con 0x7f878c1066b0 2024-04-25T00:20:51.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.384+0000 7f87941bb740 1 Processor -- start 2024-04-25T00:20:51.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.384+0000 7f87941bb740 1 -- start start 2024-04-25T00:20:51.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.384+0000 7f87941bb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8329b9630 0x55f8329b9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.384+0000 7f87941bb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329b9f50 0x55f8329c40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.384+0000 7f87941bb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8329c4620 0x55f8329c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.384+0000 7f87941bb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f83291e6b0 con 0x55f8329b9f50 2024-04-25T00:20:51.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.384+0000 7f87941bb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f8328c6880 con 0x55f8329b9630 2024-04-25T00:20:51.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.384+0000 7f87941bb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f83291a670 con 0x55f8329c4620 2024-04-25T00:20:51.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.384+0000 7f8788ff9640 1 --2- 172.21.15.107:0/4145947937 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f87780a2d00 0x7f87780a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.384+0000 7f878affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329b9f50 0x55f8329c40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.384+0000 7f878affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329b9f50 0x55f8329c40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43882/0 (socket says 172.21.15.107:43882) 2024-04-25T00:20:51.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.384+0000 7f878affd640 1 -- 172.21.15.107:0/754249778 learned_addr learned my addr 172.21.15.107:0/754249778 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:51.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.384+0000 7f878b7fe640 1 --2- 172.21.15.107:0/4145947937 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f87780a2d00 0x7f87780a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.384+0000 7f8783fff640 1 --2- 172.21.15.107:0/754249778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8329c4620 0x55f8329c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.384+0000 7f8783fff640 1 --2- 172.21.15.107:0/754249778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8329b9630 0x55f8329b9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.385+0000 7f878affd640 1 -- 172.21.15.107:0/754249778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8329c4620 msgr2=0x55f8329c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:51.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.385+0000 7f878affd640 1 --2- 172.21.15.107:0/754249778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8329c4620 0x55f8329c6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.385+0000 7f878affd640 1 -- 172.21.15.107:0/754249778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8329b9630 msgr2=0x55f8329b9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:51.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.385+0000 7f878affd640 1 --2- 172.21.15.107:0/754249778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8329b9630 0x55f8329b9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.385+0000 7f8788ff9640 1 -- 172.21.15.107:0/4145947937 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f878402e030 con 0x7f878c1066b0 2024-04-25T00:20:51.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.385+0000 7f878affd640 1 -- 172.21.15.107:0/754249778 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8326fc7d0 con 0x55f8329b9f50 2024-04-25T00:20:51.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.385+0000 7f8783fff640 1 --2- 172.21.15.107:0/754249778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8329b9630 0x55f8329b9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:20:51.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.385+0000 7f878affd640 1 --2- 172.21.15.107:0/754249778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329b9f50 0x55f8329c40e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f877c00bcb0 tx=0x7f877c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab6e82871ac8452 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:51.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.385+0000 7f878b7fe640 1 --2- 172.21.15.107:0/4145947937 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f87780a2d00 0x7f87780a51c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f87840b95f0 tx=0x7f87840e7d20 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:51.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.385+0000 7f87817fa640 1 -- 172.21.15.107:0/754249778 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c004020 con 0x55f8329b9f50 2024-04-25T00:20:51.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.385+0000 7f87817fa640 1 -- 172.21.15.107:0/754249778 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f877c005500 con 0x55f8329b9f50 2024-04-25T00:20:51.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.385+0000 7f87817fa640 1 -- 172.21.15.107:0/754249778 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c004d20 con 0x55f8329b9f50 2024-04-25T00:20:51.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.386+0000 7f87941bb740 1 -- 172.21.15.107:0/754249778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329b9f50 msgr2=0x55f8329c40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.386+0000 7f87941bb740 1 --2- 172.21.15.107:0/754249778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329b9f50 0x55f8329c40e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f877c00bcb0 tx=0x7f877c0107a0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.386+0000 7f87941bb740 1 -- 172.21.15.107:0/754249778 shutdown_connections 2024-04-25T00:20:51.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.386+0000 7f87941bb740 1 --2- 172.21.15.107:0/754249778 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8329c4620 0x55f8329c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.386+0000 7f87941bb740 1 --2- 172.21.15.107:0/754249778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329b9f50 0x55f8329c40e0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.386+0000 7f87941bb740 1 --2- 172.21.15.107:0/754249778 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8329b9630 0x55f8329b9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.386+0000 7f87941bb740 1 -- 172.21.15.107:0/754249778 >> 172.21.15.107:0/754249778 conn(0x55f8329b5950 msgr2=0x55f8329b5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:51.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.386+0000 7f87941bb740 1 -- 172.21.15.107:0/754249778 shutdown_connections 2024-04-25T00:20:51.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.386+0000 7f87941bb740 1 -- 172.21.15.107:0/754249778 wait complete. 2024-04-25T00:20:51.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.386+0000 7f87941bb740 1 -- start start 2024-04-25T00:20:51.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.386+0000 7f87941bb740 1 Processor -- start 2024-04-25T00:20:51.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.386+0000 7f87941bb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329b9630 0x55f8329c14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.386+0000 7f87941bb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8329b9f50 0x55f8329c19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.386+0000 7f87941bb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8329c4620 0x55f8329c1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.386+0000 7f87941bb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f8329c7410 con 0x55f8329b9630 2024-04-25T00:20:51.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.386+0000 7f87941bb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f8328d9f50 con 0x55f8329c4620 2024-04-25T00:20:51.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.386+0000 7f87941bb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f8329c7590 con 0x55f8329b9f50 2024-04-25T00:20:51.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.387+0000 7f87941bb740 10 client.0 _ll_register_callbacks cb 0x55f832915000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:51.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.387+0000 7f8783fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329b9630 0x55f8329c14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.387+0000 7f8783fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329b9630 0x55f8329c14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43894/0 (socket says 172.21.15.107:43894) 2024-04-25T00:20:51.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.387+0000 7f8783fff640 1 -- 172.21.15.107:0/1644149921 learned_addr learned my addr 172.21.15.107:0/1644149921 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:51.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.387+0000 7f878affd640 1 --2- 172.21.15.107:0/1644149921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8329b9f50 0x55f8329c19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.387+0000 7f8783fff640 1 -- 172.21.15.107:0/1644149921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8329b9f50 msgr2=0x55f8329c19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:51.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.387+0000 7f8783fff640 1 --2- 172.21.15.107:0/1644149921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8329b9f50 0x55f8329c19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.387+0000 7f8783fff640 1 -- 172.21.15.107:0/1644149921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8329c4620 msgr2=0x55f8329c1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:51.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.387+0000 7f8783fff640 1 --2- 172.21.15.107:0/1644149921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8329c4620 0x55f8329c1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.387+0000 7f8783fff640 1 -- 172.21.15.107:0/1644149921 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8326323f0 con 0x55f8329b9630 2024-04-25T00:20:51.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.387+0000 7f8783fff640 1 --2- 172.21.15.107:0/1644149921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329b9630 0x55f8329c14b0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f87740058c0 tx=0x7f8774011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b16c4007b0bd8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:51.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.387+0000 7f87727fc640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:20:51.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.387+0000 7f87727fc640 1 -- 172.21.15.107:0/1644149921 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877400f650 con 0x55f8329b9630 2024-04-25T00:20:51.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.387+0000 7f87727fc640 1 -- 172.21.15.107:0/1644149921 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f877400f7f0 con 0x55f8329b9630 2024-04-25T00:20:51.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.387+0000 7f87941bb740 1 -- 172.21.15.107:0/1644149921 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f877c005500 con 0x55f8329b9630 2024-04-25T00:20:51.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.387+0000 7f87727fc640 1 -- 172.21.15.107:0/1644149921 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8774025050 con 0x55f8329b9630 2024-04-25T00:20:51.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.387+0000 7f87941bb740 10 client.16530 fetch_fsmap learned FSMap version 28 2024-04-25T00:20:51.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.387+0000 7f87941bb740 1 -- 172.21.15.107:0/1644149921 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f8325f20f0 con 0x55f8329b9630 2024-04-25T00:20:51.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.388+0000 7f87727fc640 1 -- 172.21.15.107:0/1644149921 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f877c005500 con 0x55f8329b9630 2024-04-25T00:20:51.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.388+0000 7f87727fc640 1 -- 172.21.15.107:0/1644149921 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f8774010a50 con 0x55f8329b9630 2024-04-25T00:20:51.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.388+0000 7f87941bb740 10 client.16530 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:20:51.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.388+0000 7f87941bb740 10 client.16530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:20:51.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.388+0000 7f87941bb740 20 client.16530 populate_metadata read hostname 'smithi107' 2024-04-25T00:20:51.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.388+0000 7f87941bb740 1 --2- 172.21.15.107:0/1644149921 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55f8329d48b0 0x55f8329d6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.388+0000 7f87941bb740 4 client.16530 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:20:51.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.388+0000 7f87941bb740 1 -- 172.21.15.107:0/1644149921 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f83292bbf0 con 0x55f8329d48b0 2024-04-25T00:20:51.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.388+0000 7f878affd640 1 --2- 172.21.15.107:0/1644149921 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55f8329d48b0 0x55f8329d6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.389+0000 7f87727fc640 10 client.16530 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:51.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.389+0000 7f878affd640 1 --2- 172.21.15.107:0/1644149921 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55f8329d48b0 0x55f8329d6cf0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac2a139b4c353cf server_cookie=3377b4ada20fbabe in_seq=0 out_seq=0 2024-04-25T00:20:51.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.389+0000 7f87727fc640 1 -- 172.21.15.107:0/1644149921 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f83292bbf0 con 0x55f8329d48b0 2024-04-25T00:20:51.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.389+0000 7f87727fc640 10 client.16530 handle_command_reply: tid=0 2024-04-25T00:20:51.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.389+0000 7f87941bb740 1 client.16530 shutdown 2024-04-25T00:20:51.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.390+0000 7f87941bb740 1 -- 172.21.15.107:0/1644149921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329b9630 msgr2=0x55f8329c14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.390+0000 7f87941bb740 1 --2- 172.21.15.107:0/1644149921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329b9630 0x55f8329c14b0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f87740058c0 tx=0x7f8774011810 comp rx=0 tx=0).stop 2024-04-25T00:20:51.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.390+0000 7f87941bb740 1 -- 172.21.15.107:0/1644149921 shutdown_connections 2024-04-25T00:20:51.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.390+0000 7f87941bb740 1 --2- 172.21.15.107:0/1644149921 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55f8329d48b0 0x55f8329d6cf0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.390+0000 7f87941bb740 1 --2- 172.21.15.107:0/1644149921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8329c4620 0x55f8329c1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.390+0000 7f87941bb740 1 --2- 172.21.15.107:0/1644149921 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8329b9f50 0x55f8329c19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.390+0000 7f87727fc640 0 client.16530 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:51.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.390+0000 7f87941bb740 1 --2- 172.21.15.107:0/1644149921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329b9630 0x55f8329c14b0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.390+0000 7f87941bb740 1 -- 172.21.15.107:0/1644149921 >> 172.21.15.107:0/1644149921 conn(0x55f8329b5950 msgr2=0x55f8329bb650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:51.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.390+0000 7f87941bb740 1 -- 172.21.15.107:0/1644149921 shutdown_connections 2024-04-25T00:20:51.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.390+0000 7f87941bb740 1 -- 172.21.15.107:0/1644149921 wait complete. 2024-04-25T00:20:51.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.390+0000 7f87941bb740 20 client.16530 trim_cache size 0 max 16384 2024-04-25T00:20:51.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.407+0000 7f87941bb740 1 Processor -- start 2024-04-25T00:20:51.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.408+0000 7f87941bb740 1 -- start start 2024-04-25T00:20:51.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.408+0000 7f87941bb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8329b9630 0x55f8329c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.408+0000 7f87941bb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8329b9f50 0x55f8329d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.408+0000 7f87941bb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329c4620 0x55f8329bc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.408+0000 7f87941bb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f878c1b1380 con 0x55f8329c4620 2024-04-25T00:20:51.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.408+0000 7f87941bb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f878c1aeb90 con 0x55f8329b9f50 2024-04-25T00:20:51.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.408+0000 7f87941bb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f878c1ac9e0 con 0x55f8329b9630 2024-04-25T00:20:51.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.408+0000 7f8783fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329c4620 0x55f8329bc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.408+0000 7f8783fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329c4620 0x55f8329bc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43906/0 (socket says 172.21.15.107:43906) 2024-04-25T00:20:51.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.408+0000 7f8783fff640 1 -- 172.21.15.107:0/2815560889 learned_addr learned my addr 172.21.15.107:0/2815560889 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:51.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.408+0000 7f8783fff640 1 --2- 172.21.15.107:0/2815560889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8329b9630 0x55f8329c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.408+0000 7f878affd640 1 --2- 172.21.15.107:0/2815560889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8329b9f50 0x55f8329d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.408+0000 7f8783fff640 1 -- 172.21.15.107:0/2815560889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8329b9630 msgr2=0x55f8329c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:51.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.408+0000 7f8783fff640 1 --2- 172.21.15.107:0/2815560889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8329b9630 0x55f8329c3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.408+0000 7f8783fff640 1 -- 172.21.15.107:0/2815560889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8329b9f50 msgr2=0x55f8329d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:51.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.408+0000 7f8783fff640 1 --2- 172.21.15.107:0/2815560889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8329b9f50 0x55f8329d0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.408+0000 7f8783fff640 1 -- 172.21.15.107:0/2815560889 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8329a7090 con 0x55f8329c4620 2024-04-25T00:20:51.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.408+0000 7f8783fff640 1 --2- 172.21.15.107:0/2815560889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329c4620 0x55f8329bc7c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f87740128c0 tx=0x7f87740103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb23910b2b8a377 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:51.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.409+0000 7f8780ff9640 1 -- 172.21.15.107:0/2815560889 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877400e600 con 0x55f8329c4620 2024-04-25T00:20:51.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.409+0000 7f8780ff9640 1 -- 172.21.15.107:0/2815560889 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8774005500 con 0x55f8329c4620 2024-04-25T00:20:51.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.409+0000 7f8780ff9640 1 -- 172.21.15.107:0/2815560889 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8774010610 con 0x55f8329c4620 2024-04-25T00:20:51.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.409+0000 7f87941bb740 1 -- 172.21.15.107:0/2815560889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329c4620 msgr2=0x55f8329bc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.409+0000 7f87941bb740 1 --2- 172.21.15.107:0/2815560889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329c4620 0x55f8329bc7c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f87740128c0 tx=0x7f87740103d0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.409+0000 7f87941bb740 1 -- 172.21.15.107:0/2815560889 shutdown_connections 2024-04-25T00:20:51.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.409+0000 7f87941bb740 1 --2- 172.21.15.107:0/2815560889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329c4620 0x55f8329bc7c0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.409+0000 7f87941bb740 1 --2- 172.21.15.107:0/2815560889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8329b9f50 0x55f8329d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.409+0000 7f87941bb740 1 --2- 172.21.15.107:0/2815560889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8329b9630 0x55f8329c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.409+0000 7f87941bb740 1 -- 172.21.15.107:0/2815560889 >> 172.21.15.107:0/2815560889 conn(0x55f8329b5950 msgr2=0x55f8329b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:51.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.409+0000 7f87941bb740 1 -- 172.21.15.107:0/2815560889 shutdown_connections 2024-04-25T00:20:51.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.409+0000 7f87941bb740 1 -- 172.21.15.107:0/2815560889 wait complete. 2024-04-25T00:20:51.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.410+0000 7f87941bb740 1 -- start start 2024-04-25T00:20:51.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.410+0000 7f87941bb740 1 Processor -- start 2024-04-25T00:20:51.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.410+0000 7f87941bb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329b9630 0x55f832a37770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.410+0000 7f87941bb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8329b9f50 0x55f832a37cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.410+0000 7f87941bb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8329c4620 0x55f8329dbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.410+0000 7f87941bb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f878c1ad050 con 0x55f8329b9630 2024-04-25T00:20:51.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.410+0000 7f87941bb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f878c1ae4c0 con 0x55f8329b9f50 2024-04-25T00:20:51.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.410+0000 7f87941bb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f878c1ad720 con 0x55f8329c4620 2024-04-25T00:20:51.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.410+0000 7f87941bb740 10 client.0 _ll_register_callbacks cb 0x7f877400cd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:51.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.410+0000 7f8783fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329b9630 0x55f832a37770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.410+0000 7f8783fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329b9630 0x55f832a37770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43910/0 (socket says 172.21.15.107:43910) 2024-04-25T00:20:51.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.410+0000 7f8783fff640 1 -- 172.21.15.107:0/914319555 learned_addr learned my addr 172.21.15.107:0/914319555 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:51.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.410+0000 7f878affd640 1 --2- 172.21.15.107:0/914319555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8329b9f50 0x55f832a37cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.410+0000 7f8783fff640 1 -- 172.21.15.107:0/914319555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8329c4620 msgr2=0x55f8329dbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:51.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.410+0000 7f8783fff640 1 --2- 172.21.15.107:0/914319555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8329c4620 0x55f8329dbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.410+0000 7f8783fff640 1 -- 172.21.15.107:0/914319555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8329b9f50 msgr2=0x55f832a37cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:51.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.410+0000 7f8783fff640 1 --2- 172.21.15.107:0/914319555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8329b9f50 0x55f832a37cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.410+0000 7f8783fff640 1 -- 172.21.15.107:0/914319555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8329dc5e0 con 0x55f8329b9630 2024-04-25T00:20:51.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.411+0000 7f8783fff640 1 --2- 172.21.15.107:0/914319555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329b9630 0x55f832a37770 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f8774020040 tx=0x7f8774012a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77f9526b4395bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:51.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.411+0000 7f87727fc640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:20:51.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.411+0000 7f87727fc640 1 -- 172.21.15.107:0/914319555 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877403b060 con 0x55f8329b9630 2024-04-25T00:20:51.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.411+0000 7f87727fc640 1 -- 172.21.15.107:0/914319555 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f877400e400 con 0x55f8329b9630 2024-04-25T00:20:51.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.411+0000 7f87941bb740 1 -- 172.21.15.107:0/914319555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8774005500 con 0x55f8329b9630 2024-04-25T00:20:51.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.411+0000 7f87727fc640 1 -- 172.21.15.107:0/914319555 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f877400eea0 con 0x55f8329b9630 2024-04-25T00:20:51.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.411+0000 7f87941bb740 10 client.16536 fetch_fsmap learned FSMap version 28 2024-04-25T00:20:51.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.411+0000 7f87941bb740 1 -- 172.21.15.107:0/914319555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f832a3ef30 con 0x55f8329b9630 2024-04-25T00:20:51.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.411+0000 7f87727fc640 1 -- 172.21.15.107:0/914319555 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8774005500 con 0x55f8329b9630 2024-04-25T00:20:51.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.411+0000 7f87727fc640 1 -- 172.21.15.107:0/914319555 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f8774035070 con 0x55f8329b9630 2024-04-25T00:20:51.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.411+0000 7f87941bb740 10 client.16536 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:20:51.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.411+0000 7f87941bb740 10 client.16536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:20:51.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.411+0000 7f87941bb740 20 client.16536 populate_metadata read hostname 'smithi107' 2024-04-25T00:20:51.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.411+0000 7f87941bb740 1 --2- 172.21.15.107:0/914319555 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55f832a3f590 0x55f832a419b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.411+0000 7f87941bb740 4 client.16536 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:20:51.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.411+0000 7f87941bb740 1 -- 172.21.15.107:0/914319555 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f832925f40 con 0x55f832a3f590 2024-04-25T00:20:51.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.411+0000 7f878affd640 1 --2- 172.21.15.107:0/914319555 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55f832a3f590 0x55f832a419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.412+0000 7f87727fc640 10 client.16536 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:51.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.412+0000 7f878affd640 1 --2- 172.21.15.107:0/914319555 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55f832a3f590 0x55f832a419b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abe50bda6548fa2c server_cookie=d7e8811a634990d in_seq=0 out_seq=0 2024-04-25T00:20:51.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.412+0000 7f87727fc640 1 -- 172.21.15.107:0/914319555 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f832925f40 con 0x55f832a3f590 2024-04-25T00:20:51.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.412+0000 7f87727fc640 10 client.16536 handle_command_reply: tid=0 2024-04-25T00:20:51.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.412+0000 7f87941bb740 1 client.16536 shutdown 2024-04-25T00:20:51.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.412+0000 7f87941bb740 1 -- 172.21.15.107:0/914319555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329b9630 msgr2=0x55f832a37770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.412+0000 7f87941bb740 1 --2- 172.21.15.107:0/914319555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329b9630 0x55f832a37770 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f8774020040 tx=0x7f8774012a20 comp rx=0 tx=0).stop 2024-04-25T00:20:51.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.412+0000 7f87941bb740 1 -- 172.21.15.107:0/914319555 shutdown_connections 2024-04-25T00:20:51.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.412+0000 7f87941bb740 1 --2- 172.21.15.107:0/914319555 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55f832a3f590 0x55f832a419b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.412+0000 7f87941bb740 1 --2- 172.21.15.107:0/914319555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f8329c4620 0x55f8329dbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.412+0000 7f87941bb740 1 --2- 172.21.15.107:0/914319555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f8329b9f50 0x55f832a37cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.412+0000 7f87727fc640 0 client.16536 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:51.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.412+0000 7f87941bb740 1 --2- 172.21.15.107:0/914319555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f8329b9630 0x55f832a37770 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.412+0000 7f87941bb740 1 -- 172.21.15.107:0/914319555 >> 172.21.15.107:0/914319555 conn(0x55f8329b5950 msgr2=0x55f8329c1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:51.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.412+0000 7f87941bb740 1 -- 172.21.15.107:0/914319555 shutdown_connections 2024-04-25T00:20:51.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.412+0000 7f87941bb740 1 -- 172.21.15.107:0/914319555 wait complete. 2024-04-25T00:20:51.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.412+0000 7f87941bb740 20 client.16536 trim_cache size 0 max 16384 2024-04-25T00:20:51.420 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:20:51.420 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-04-25T00:20:51.420 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-04-25T00:20:51.420 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:20:51.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.413+0000 7f87913ce640 1 -- 172.21.15.107:0/4145947937 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f87780a2d00 msgr2=0x7f87780a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.413+0000 7f87913ce640 1 --2- 172.21.15.107:0/4145947937 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f87780a2d00 0x7f87780a51c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f87840b95f0 tx=0x7f87840e7d20 comp rx=0 tx=0).stop 2024-04-25T00:20:51.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.413+0000 7f87913ce640 1 -- 172.21.15.107:0/4145947937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f878c1066b0 msgr2=0x7f878c19c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.413+0000 7f87913ce640 1 --2- 172.21.15.107:0/4145947937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f878c1066b0 0x7f878c19c850 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8784005cc0 tx=0x7f8784003970 comp rx=0 tx=0).stop 2024-04-25T00:20:51.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.413+0000 7f87913ce640 1 -- 172.21.15.107:0/4145947937 shutdown_connections 2024-04-25T00:20:51.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.413+0000 7f87913ce640 1 --2- 172.21.15.107:0/4145947937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f878c1066b0 0x7f878c19c850 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.413+0000 7f87913ce640 1 --2- 172.21.15.107:0/4145947937 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f878c100fd0 0x7f878c198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.413+0000 7f87913ce640 1 --2- 172.21.15.107:0/4145947937 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f87780a2d00 0x7f87780a51c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.413+0000 7f87913ce640 1 --2- 172.21.15.107:0/4145947937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f878c100610 0x7f878c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.413+0000 7f87913ce640 1 -- 172.21.15.107:0/4145947937 >> 172.21.15.107:0/4145947937 conn(0x7f878c0769c0 msgr2=0x7f878c0ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:51.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.413+0000 7f87913ce640 1 -- 172.21.15.107:0/4145947937 shutdown_connections 2024-04-25T00:20:51.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.413+0000 7f87913ce640 1 -- 172.21.15.107:0/4145947937 wait complete. 2024-04-25T00:20:51.424 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:20:51.425 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:20:51.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.522+0000 7f0b22083640 1 -- 172.21.15.107:0/1405338334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b1c100fe0 msgr2=0x7f0b1c10d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.522+0000 7f0b22083640 1 --2- 172.21.15.107:0/1405338334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b1c100fe0 0x7f0b1c10d4f0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0b14009930 tx=0x7f0b1402a420 comp rx=0 tx=0).stop 2024-04-25T00:20:51.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.522+0000 7f0b22083640 1 -- 172.21.15.107:0/1405338334 shutdown_connections 2024-04-25T00:20:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.522+0000 7f0b22083640 1 --2- 172.21.15.107:0/1405338334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b1c100fe0 0x7f0b1c10d4f0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.522+0000 7f0b22083640 1 --2- 172.21.15.107:0/1405338334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b1c100620 0x7f0b1c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.522+0000 7f0b22083640 1 --2- 172.21.15.107:0/1405338334 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b1c106630 0x7f0b1c106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.522+0000 7f0b22083640 1 -- 172.21.15.107:0/1405338334 >> 172.21.15.107:0/1405338334 conn(0x7f0b1c077740 msgr2=0x7f0b1c077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.522+0000 7f0b22083640 1 -- 172.21.15.107:0/1405338334 shutdown_connections 2024-04-25T00:20:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.522+0000 7f0b22083640 1 -- 172.21.15.107:0/1405338334 wait complete. 2024-04-25T00:20:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.523+0000 7f0b22083640 1 Processor -- start 2024-04-25T00:20:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.523+0000 7f0b22083640 1 -- start start 2024-04-25T00:20:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.523+0000 7f0b22083640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b1c100620 0x7f0b1c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.523+0000 7f0b22083640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b1c100fe0 0x7f0b1c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.523+0000 7f0b22083640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b1c106630 0x7f0b1c19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.523+0000 7f0b22083640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c110a40 con 0x7f0b1c100620 2024-04-25T00:20:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.523+0000 7f0b22083640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c1108c0 con 0x7f0b1c100fe0 2024-04-25T00:20:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.523+0000 7f0b22083640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c110bc0 con 0x7f0b1c106630 2024-04-25T00:20:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.524+0000 7f0b1b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b1c100620 0x7f0b1c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.524+0000 7f0b1b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b1c100620 0x7f0b1c19a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43934/0 (socket says 172.21.15.107:43934) 2024-04-25T00:20:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.524+0000 7f0b1b7fe640 1 -- 172.21.15.107:0/2981319883 learned_addr learned my addr 172.21.15.107:0/2981319883 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.524+0000 7f0b1affd640 1 --2- 172.21.15.107:0/2981319883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b1c100fe0 0x7f0b1c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.524+0000 7f0b1bfff640 1 --2- 172.21.15.107:0/2981319883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b1c106630 0x7f0b1c19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.524+0000 7f0b1b7fe640 1 -- 172.21.15.107:0/2981319883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b1c106630 msgr2=0x7f0b1c19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.524+0000 7f0b1b7fe640 1 --2- 172.21.15.107:0/2981319883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b1c106630 0x7f0b1c19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.524+0000 7f0b1b7fe640 1 -- 172.21.15.107:0/2981319883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b1c100fe0 msgr2=0x7f0b1c19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.524+0000 7f0b1b7fe640 1 --2- 172.21.15.107:0/2981319883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b1c100fe0 0x7f0b1c19a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.524+0000 7f0b1b7fe640 1 -- 172.21.15.107:0/2981319883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1c19f220 con 0x7f0b1c100620 2024-04-25T00:20:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.524+0000 7f0b1bfff640 1 --2- 172.21.15.107:0/2981319883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b1c106630 0x7f0b1c19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:20:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.524+0000 7f0b1b7fe640 1 --2- 172.21.15.107:0/2981319883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b1c100620 0x7f0b1c19a230 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00eab0 tx=0x7f0b0c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.524+0000 7f0b18ff9640 1 -- 172.21.15.107:0/2981319883 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c00cd80 con 0x7f0b1c100620 2024-04-25T00:20:51.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.524+0000 7f0b22083640 1 -- 172.21.15.107:0/2981319883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b1c19f510 con 0x7f0b1c100620 2024-04-25T00:20:51.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.524+0000 7f0b18ff9640 1 -- 172.21.15.107:0/2981319883 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b0c011950 con 0x7f0b1c100620 2024-04-25T00:20:51.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.524+0000 7f0b18ff9640 1 -- 172.21.15.107:0/2981319883 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c010a10 con 0x7f0b1c100620 2024-04-25T00:20:51.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.524+0000 7f0b22083640 1 -- 172.21.15.107:0/2981319883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b1c1a6da0 con 0x7f0b1c100620 2024-04-25T00:20:51.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.525+0000 7f0b22083640 1 -- 172.21.15.107:0/2981319883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b1c068560 con 0x7f0b1c100620 2024-04-25T00:20:51.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.526+0000 7f0b18ff9640 1 -- 172.21.15.107:0/2981319883 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0b0c011b00 con 0x7f0b1c100620 2024-04-25T00:20:51.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.527+0000 7f0b18ff9640 1 --2- 172.21.15.107:0/2981319883 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f0b080a2d00 0x7f0b080a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.527+0000 7f0b1affd640 1 --2- 172.21.15.107:0/2981319883 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f0b080a2d00 0x7f0b080a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.527+0000 7f0b18ff9640 1 -- 172.21.15.107:0/2981319883 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f0b0c0cfdb0 con 0x7f0b1c100620 2024-04-25T00:20:51.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.527+0000 7f0b1affd640 1 --2- 172.21.15.107:0/2981319883 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f0b080a2d00 0x7f0b080a51c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c10d2f0 tx=0x7f0b04009110 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:51.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.531+0000 7f0b18ff9640 1 -- 172.21.15.107:0/2981319883 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f0b0c016030 con 0x7f0b1c100620 2024-04-25T00:20:51.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.747+0000 7f0b22083640 1 -- 172.21.15.107:0/2981319883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b1c1a7030 con 0x7f0b1c100620 2024-04-25T00:20:51.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.748+0000 7f0b18ff9640 1 -- 172.21.15.107:0/2981319883 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11906 (secure 0 0 0) 0x7f0b0c09c660 con 0x7f0b1c100620 2024-04-25T00:20:51.748 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:20:51.749 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:18:38.933541+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24493,24521]},"id":1}]} 2024-04-25T00:20:51.749 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 28 2024-04-25T00:20:51.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.750+0000 7f0b22083640 1 -- 172.21.15.107:0/2981319883 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f0b080a2d00 msgr2=0x7f0b080a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.750+0000 7f0b22083640 1 --2- 172.21.15.107:0/2981319883 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f0b080a2d00 0x7f0b080a51c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c10d2f0 tx=0x7f0b04009110 comp rx=0 tx=0).stop 2024-04-25T00:20:51.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.750+0000 7f0b22083640 1 -- 172.21.15.107:0/2981319883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b1c100620 msgr2=0x7f0b1c19a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.750+0000 7f0b22083640 1 --2- 172.21.15.107:0/2981319883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b1c100620 0x7f0b1c19a230 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c00eab0 tx=0x7f0b0c00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.750+0000 7f0b22083640 1 -- 172.21.15.107:0/2981319883 shutdown_connections 2024-04-25T00:20:51.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.750+0000 7f0b22083640 1 --2- 172.21.15.107:0/2981319883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b1c106630 0x7f0b1c19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.750+0000 7f0b22083640 1 --2- 172.21.15.107:0/2981319883 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f0b080a2d00 0x7f0b080a51c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.750+0000 7f0b22083640 1 --2- 172.21.15.107:0/2981319883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b1c100fe0 0x7f0b1c19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.750+0000 7f0b22083640 1 --2- 172.21.15.107:0/2981319883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b1c100620 0x7f0b1c19a230 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.750+0000 7f0b22083640 1 -- 172.21.15.107:0/2981319883 >> 172.21.15.107:0/2981319883 conn(0x7f0b1c077740 msgr2=0x7f0b1c0688d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:51.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.750+0000 7f0b22083640 1 -- 172.21.15.107:0/2981319883 shutdown_connections 2024-04-25T00:20:51.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.750+0000 7f0b22083640 1 -- 172.21.15.107:0/2981319883 wait complete. 2024-04-25T00:20:51.762 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:20:51.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.861+0000 7f46e06ca640 1 -- 172.21.15.107:0/407707249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d8100610 msgr2=0x7f46d8100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.861+0000 7f46e06ca640 1 --2- 172.21.15.107:0/407707249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d8100610 0x7f46d8100a90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009930 tx=0x7f46c802a420 comp rx=0 tx=0).stop 2024-04-25T00:20:51.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.862+0000 7f46e06ca640 1 -- 172.21.15.107:0/407707249 shutdown_connections 2024-04-25T00:20:51.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.862+0000 7f46e06ca640 1 --2- 172.21.15.107:0/407707249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46d8100fd0 0x7f46d810d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.862+0000 7f46e06ca640 1 --2- 172.21.15.107:0/407707249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d8100610 0x7f46d8100a90 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.862+0000 7f46e06ca640 1 --2- 172.21.15.107:0/407707249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46d81066b0 0x7f46d8106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.862+0000 7f46e06ca640 1 -- 172.21.15.107:0/407707249 >> 172.21.15.107:0/407707249 conn(0x7f46d8077740 msgr2=0x7f46d8077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:51.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.862+0000 7f46e06ca640 1 -- 172.21.15.107:0/407707249 shutdown_connections 2024-04-25T00:20:51.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.862+0000 7f46e06ca640 1 -- 172.21.15.107:0/407707249 wait complete. 2024-04-25T00:20:51.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.862+0000 7f46e06ca640 1 Processor -- start 2024-04-25T00:20:51.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.862+0000 7f46e06ca640 1 -- start start 2024-04-25T00:20:51.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.862+0000 7f46e06ca640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46d8100610 0x7f46d8197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.862+0000 7f46e06ca640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46d8100fd0 0x7f46d81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.862+0000 7f46e06ca640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d81066b0 0x7f46d819c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.862+0000 7f46e06ca640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f46d8111b30 con 0x7f46d81066b0 2024-04-25T00:20:51.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.862+0000 7f46e06ca640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f46d81119b0 con 0x7f46d8100610 2024-04-25T00:20:51.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.862+0000 7f46e06ca640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f46d8111cb0 con 0x7f46d8100fd0 2024-04-25T00:20:51.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.862+0000 7f46dec67640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d81066b0 0x7f46d819c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.862+0000 7f46dec67640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d81066b0 0x7f46d819c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43958/0 (socket says 172.21.15.107:43958) 2024-04-25T00:20:51.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.862+0000 7f46dec67640 1 -- 172.21.15.107:0/566729685 learned_addr learned my addr 172.21.15.107:0/566729685 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:51.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.863+0000 7f46de466640 1 --2- 172.21.15.107:0/566729685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46d8100610 0x7f46d8197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.863+0000 7f46dec67640 1 -- 172.21.15.107:0/566729685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46d8100fd0 msgr2=0x7f46d81984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.863+0000 7f46dec67640 1 --2- 172.21.15.107:0/566729685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46d8100fd0 0x7f46d81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.863+0000 7f46dec67640 1 -- 172.21.15.107:0/566729685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46d8100610 msgr2=0x7f46d8197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.863+0000 7f46dec67640 1 --2- 172.21.15.107:0/566729685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46d8100610 0x7f46d8197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.863+0000 7f46dec67640 1 -- 172.21.15.107:0/566729685 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d819cfa0 con 0x7f46d81066b0 2024-04-25T00:20:51.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.863+0000 7f46de466640 1 --2- 172.21.15.107:0/566729685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46d8100610 0x7f46d8197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:20:51.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.863+0000 7f46dec67640 1 --2- 172.21.15.107:0/566729685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d81066b0 0x7f46d819c870 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f46d000cad0 tx=0x7f46d000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:51.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.863+0000 7f46cf7fe640 1 -- 172.21.15.107:0/566729685 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d000ae20 con 0x7f46d81066b0 2024-04-25T00:20:51.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.863+0000 7f46e06ca640 1 -- 172.21.15.107:0/566729685 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d819d290 con 0x7f46d81066b0 2024-04-25T00:20:51.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.863+0000 7f46cf7fe640 1 -- 172.21.15.107:0/566729685 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46d0005020 con 0x7f46d81066b0 2024-04-25T00:20:51.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.863+0000 7f46cf7fe640 1 -- 172.21.15.107:0/566729685 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d000f6f0 con 0x7f46d81066b0 2024-04-25T00:20:51.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.863+0000 7f46e06ca640 1 -- 172.21.15.107:0/566729685 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d81a4b20 con 0x7f46d81066b0 2024-04-25T00:20:51.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.866+0000 7f46cf7fe640 1 -- 172.21.15.107:0/566729685 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f46d00051d0 con 0x7f46d81066b0 2024-04-25T00:20:51.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.866+0000 7f46e33d6740 1 Processor -- start 2024-04-25T00:20:51.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.866+0000 7f46e33d6740 1 -- start start 2024-04-25T00:20:51.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.866+0000 7f46e33d6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6db511580 0x55f6db511960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.866+0000 7f46e33d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511ea0 0x55f6db51c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.866+0000 7f46e33d6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6db51c570 0x55f6db51e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.866+0000 7f46e33d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f6db476670 con 0x55f6db511ea0 2024-04-25T00:20:51.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.866+0000 7f46e33d6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f6db41e970 con 0x55f6db511580 2024-04-25T00:20:51.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.866+0000 7f46e33d6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f6db472630 con 0x55f6db51c570 2024-04-25T00:20:51.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.866+0000 7f46de466640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511ea0 0x55f6db51c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.866+0000 7f46de466640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511ea0 0x55f6db51c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43968/0 (socket says 172.21.15.107:43968) 2024-04-25T00:20:51.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.866+0000 7f46de466640 1 -- 172.21.15.107:0/479209077 learned_addr learned my addr 172.21.15.107:0/479209077 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:51.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.867+0000 7f46ddc65640 1 --2- 172.21.15.107:0/479209077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6db511580 0x55f6db511960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.867+0000 7f46cf7fe640 1 --2- 172.21.15.107:0/566729685 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f46c40a2d00 0x7f46c40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.867+0000 7f46ddc65640 1 --2- 172.21.15.107:0/479209077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6db51c570 0x55f6db51e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.867+0000 7f46de466640 1 -- 172.21.15.107:0/479209077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6db51c570 msgr2=0x55f6db51e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:51.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.867+0000 7f46de466640 1 --2- 172.21.15.107:0/479209077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6db51c570 0x55f6db51e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.867+0000 7f46de466640 1 -- 172.21.15.107:0/479209077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6db511580 msgr2=0x55f6db511960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:51.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.867+0000 7f46de466640 1 --2- 172.21.15.107:0/479209077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6db511580 0x55f6db511960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.867+0000 7f46de466640 1 -- 172.21.15.107:0/479209077 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6db442400 con 0x55f6db511ea0 2024-04-25T00:20:51.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.867+0000 7f46dec67640 1 --2- 172.21.15.107:0/566729685 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f46c40a2d00 0x7f46c40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.867+0000 7f46cf7fe640 1 -- 172.21.15.107:0/566729685 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f46d00cf930 con 0x7f46d81066b0 2024-04-25T00:20:51.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.867+0000 7f46de466640 1 --2- 172.21.15.107:0/479209077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511ea0 0x55f6db51c030 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f46d4009c20 tx=0x7f46d400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297c091c7684e597 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:51.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.867+0000 7f46ccff9640 1 -- 172.21.15.107:0/479209077 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d4016070 con 0x55f6db511ea0 2024-04-25T00:20:51.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.867+0000 7f46ccff9640 1 -- 172.21.15.107:0/479209077 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46d4011950 con 0x55f6db511ea0 2024-04-25T00:20:51.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.867+0000 7f46ccff9640 1 -- 172.21.15.107:0/479209077 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d40107d0 con 0x55f6db511ea0 2024-04-25T00:20:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.868+0000 7f46e33d6740 1 -- 172.21.15.107:0/479209077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511ea0 msgr2=0x55f6db51c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.868+0000 7f46dec67640 1 --2- 172.21.15.107:0/566729685 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f46c40a2d00 0x7f46c40a51c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f46d00ccdc0 tx=0x7f46d009e040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.868+0000 7f46e33d6740 1 --2- 172.21.15.107:0/479209077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511ea0 0x55f6db51c030 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f46d4009c20 tx=0x7f46d400e6c0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.868+0000 7f46e33d6740 1 -- 172.21.15.107:0/479209077 shutdown_connections 2024-04-25T00:20:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.868+0000 7f46e33d6740 1 --2- 172.21.15.107:0/479209077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6db51c570 0x55f6db51e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.868+0000 7f46e33d6740 1 --2- 172.21.15.107:0/479209077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511ea0 0x55f6db51c030 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.868+0000 7f46e33d6740 1 --2- 172.21.15.107:0/479209077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6db511580 0x55f6db511960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.868+0000 7f46e33d6740 1 -- 172.21.15.107:0/479209077 >> 172.21.15.107:0/479209077 conn(0x55f6db50d8a0 msgr2=0x55f6db50dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.868+0000 7f46e33d6740 1 -- 172.21.15.107:0/479209077 shutdown_connections 2024-04-25T00:20:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.868+0000 7f46e33d6740 1 -- 172.21.15.107:0/479209077 wait complete. 2024-04-25T00:20:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.868+0000 7f46e33d6740 1 -- start start 2024-04-25T00:20:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.868+0000 7f46e33d6740 1 Processor -- start 2024-04-25T00:20:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.868+0000 7f46e33d6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6db511580 0x55f6db51bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46e33d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511ea0 0x55f6db5191b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46e33d6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6db51c570 0x55f6db519710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46e33d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f6db51f360 con 0x55f6db511ea0 2024-04-25T00:20:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46e33d6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f6db432050 con 0x55f6db511580 2024-04-25T00:20:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46e33d6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f6db51f4e0 con 0x55f6db51c570 2024-04-25T00:20:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46e33d6740 10 client.0 _ll_register_callbacks cb 0x55f6db189be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46de466640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511ea0 0x55f6db5191b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46de466640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511ea0 0x55f6db5191b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43972/0 (socket says 172.21.15.107:43972) 2024-04-25T00:20:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46de466640 1 -- 172.21.15.107:0/3797724139 learned_addr learned my addr 172.21.15.107:0/3797724139 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46ddc65640 1 --2- 172.21.15.107:0/3797724139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6db511580 0x55f6db51bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46ddc65640 1 --2- 172.21.15.107:0/3797724139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6db51c570 0x55f6db519710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46de466640 1 -- 172.21.15.107:0/3797724139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6db51c570 msgr2=0x55f6db519710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46de466640 1 --2- 172.21.15.107:0/3797724139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6db51c570 0x55f6db519710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46de466640 1 -- 172.21.15.107:0/3797724139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6db511580 msgr2=0x55f6db51bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46de466640 1 --2- 172.21.15.107:0/3797724139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6db511580 0x55f6db51bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46de466640 1 -- 172.21.15.107:0/3797724139 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6db403dc0 con 0x55f6db511ea0 2024-04-25T00:20:51.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46ddc65640 1 --2- 172.21.15.107:0/3797724139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6db51c570 0x55f6db519710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:20:51.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46de466640 1 --2- 172.21.15.107:0/3797724139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511ea0 0x55f6db5191b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f46d4010410 tx=0x7f46d4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827098bed2b7f3e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:51.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:20:51.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46c1ffb640 1 -- 172.21.15.107:0/3797724139 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d4016050 con 0x55f6db511ea0 2024-04-25T00:20:51.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46c1ffb640 1 -- 172.21.15.107:0/3797724139 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f46d4005040 con 0x55f6db511ea0 2024-04-25T00:20:51.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46e33d6740 1 -- 172.21.15.107:0/3797724139 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46d4011950 con 0x55f6db511ea0 2024-04-25T00:20:51.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46c1ffb640 1 -- 172.21.15.107:0/3797724139 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f46d4030050 con 0x55f6db511ea0 2024-04-25T00:20:51.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46e33d6740 10 client.16554 fetch_fsmap learned FSMap version 28 2024-04-25T00:20:51.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.869+0000 7f46e33d6740 1 -- 172.21.15.107:0/3797724139 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f6db18a3f0 con 0x55f6db511ea0 2024-04-25T00:20:51.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.870+0000 7f46c1ffb640 1 -- 172.21.15.107:0/3797724139 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46d4011950 con 0x55f6db511ea0 2024-04-25T00:20:51.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.870+0000 7f46c1ffb640 1 -- 172.21.15.107:0/3797724139 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f46d402b030 con 0x55f6db511ea0 2024-04-25T00:20:51.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.870+0000 7f46e33d6740 10 client.16554 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:20:51.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.870+0000 7f46e33d6740 10 client.16554 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:20:51.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.870+0000 7f46e33d6740 20 client.16554 populate_metadata read hostname 'smithi107' 2024-04-25T00:20:51.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.870+0000 7f46e33d6740 1 --2- 172.21.15.107:0/3797724139 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55f6db52c560 0x55f6db52e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.870+0000 7f46e33d6740 4 client.16554 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:20:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.870+0000 7f46e33d6740 1 -- 172.21.15.107:0/3797724139 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6db483bb0 con 0x55f6db52c560 2024-04-25T00:20:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.870+0000 7f46ddc65640 1 --2- 172.21.15.107:0/3797724139 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55f6db52c560 0x55f6db52e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.870+0000 7f46ddc65640 1 --2- 172.21.15.107:0/3797724139 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55f6db52c560 0x55f6db52e950 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=62656d24582ca594 server_cookie=263fb276a6bce449 in_seq=0 out_seq=0 2024-04-25T00:20:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.870+0000 7f46c1ffb640 10 client.16554 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:20:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.871+0000 7f46c1ffb640 1 -- 172.21.15.107:0/3797724139 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f6db483bb0 con 0x55f6db52c560 2024-04-25T00:20:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.871+0000 7f46c1ffb640 10 client.16554 handle_command_reply: tid=0 2024-04-25T00:20:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.871+0000 7f46e33d6740 1 client.16554 shutdown 2024-04-25T00:20:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.871+0000 7f46e33d6740 1 -- 172.21.15.107:0/3797724139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511ea0 msgr2=0x55f6db5191b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.871+0000 7f46e33d6740 1 --2- 172.21.15.107:0/3797724139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511ea0 0x55f6db5191b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f46d4010410 tx=0x7f46d4005850 comp rx=0 tx=0).stop 2024-04-25T00:20:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.871+0000 7f46e33d6740 1 -- 172.21.15.107:0/3797724139 shutdown_connections 2024-04-25T00:20:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.871+0000 7f46e33d6740 1 --2- 172.21.15.107:0/3797724139 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55f6db52c560 0x55f6db52e950 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.871+0000 7f46e33d6740 1 --2- 172.21.15.107:0/3797724139 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6db51c570 0x55f6db519710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.871+0000 7f46c1ffb640 0 client.16554 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:20:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.871+0000 7f46e33d6740 1 --2- 172.21.15.107:0/3797724139 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511ea0 0x55f6db5191b0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.871+0000 7f46e33d6740 1 --2- 172.21.15.107:0/3797724139 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6db511580 0x55f6db51bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.871+0000 7f46e33d6740 1 -- 172.21.15.107:0/3797724139 >> 172.21.15.107:0/3797724139 conn(0x55f6db50d8a0 msgr2=0x55f6db5136a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.871+0000 7f46e33d6740 1 -- 172.21.15.107:0/3797724139 shutdown_connections 2024-04-25T00:20:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.871+0000 7f46e33d6740 1 -- 172.21.15.107:0/3797724139 wait complete. 2024-04-25T00:20:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.871+0000 7f46e33d6740 20 client.16554 trim_cache size 0 max 16384 2024-04-25T00:20:51.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.884+0000 7f46e33d6740 1 Processor -- start 2024-04-25T00:20:51.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.884+0000 7f46e33d6740 1 -- start start 2024-04-25T00:20:51.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.884+0000 7f46e33d6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6db511580 0x55f6db528220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.884+0000 7f46e33d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511ea0 0x55f6db534070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.884+0000 7f46e33d6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6db51c570 0x55f6db5345b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.885+0000 7f46e33d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f46d81b1540 con 0x55f6db511ea0 2024-04-25T00:20:51.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.885+0000 7f46e33d6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f46d81aed50 con 0x55f6db51c570 2024-04-25T00:20:51.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.885+0000 7f46e33d6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f46d81acba0 con 0x55f6db511580 2024-04-25T00:20:51.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.885+0000 7f46de466640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511ea0 0x55f6db534070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.885+0000 7f46de466640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511ea0 0x55f6db534070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43978/0 (socket says 172.21.15.107:43978) 2024-04-25T00:20:51.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.885+0000 7f46de466640 1 -- 172.21.15.107:0/1266341668 learned_addr learned my addr 172.21.15.107:0/1266341668 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:51.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.885+0000 7f46ddc65640 1 --2- 172.21.15.107:0/1266341668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6db511580 0x55f6db528220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.885+0000 7f46de466640 1 -- 172.21.15.107:0/1266341668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6db511580 msgr2=0x55f6db528220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:51.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.885+0000 7f46de466640 1 --2- 172.21.15.107:0/1266341668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6db511580 0x55f6db528220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.885+0000 7f46de466640 1 -- 172.21.15.107:0/1266341668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6db51c570 msgr2=0x55f6db5345b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:51.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.885+0000 7f46de466640 1 --2- 172.21.15.107:0/1266341668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6db51c570 0x55f6db5345b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.885+0000 7f46de466640 1 -- 172.21.15.107:0/1266341668 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6db2547d0 con 0x55f6db511ea0 2024-04-25T00:20:51.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.885+0000 7f46de466640 1 --2- 172.21.15.107:0/1266341668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511ea0 0x55f6db534070 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f46d40105f0 tx=0x7f46d4009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33282933d6b23cce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:51.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.885+0000 7f46c3fff640 1 -- 172.21.15.107:0/1266341668 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d4039070 con 0x55f6db511ea0 2024-04-25T00:20:51.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.885+0000 7f46c3fff640 1 -- 172.21.15.107:0/1266341668 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46d4010670 con 0x55f6db511ea0 2024-04-25T00:20:51.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.885+0000 7f46c3fff640 1 -- 172.21.15.107:0/1266341668 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d4016070 con 0x55f6db511ea0 2024-04-25T00:20:51.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.886+0000 7f46e33d6740 1 -- 172.21.15.107:0/1266341668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511ea0 msgr2=0x55f6db534070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.886+0000 7f46e33d6740 1 --2- 172.21.15.107:0/1266341668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511ea0 0x55f6db534070 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f46d40105f0 tx=0x7f46d4009aa0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.886+0000 7f46e33d6740 1 -- 172.21.15.107:0/1266341668 shutdown_connections 2024-04-25T00:20:51.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.886+0000 7f46e33d6740 1 --2- 172.21.15.107:0/1266341668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6db51c570 0x55f6db5345b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.886+0000 7f46e33d6740 1 --2- 172.21.15.107:0/1266341668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511ea0 0x55f6db534070 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.886+0000 7f46e33d6740 1 --2- 172.21.15.107:0/1266341668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6db511580 0x55f6db528220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.886+0000 7f46e33d6740 1 -- 172.21.15.107:0/1266341668 >> 172.21.15.107:0/1266341668 conn(0x55f6db50d8a0 msgr2=0x55f6db50cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:51.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.886+0000 7f46e33d6740 1 -- 172.21.15.107:0/1266341668 shutdown_connections 2024-04-25T00:20:51.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.886+0000 7f46e33d6740 1 -- 172.21.15.107:0/1266341668 wait complete. 2024-04-25T00:20:51.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.886+0000 7f46e33d6740 1 -- start start 2024-04-25T00:20:51.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.886+0000 7f46e33d6740 1 Processor -- start 2024-04-25T00:20:51.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.886+0000 7f46e33d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511580 0x55f6db524940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.886+0000 7f46e33d6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6db511ea0 0x55f6db521da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.886+0000 7f46e33d6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6db51c570 0x55f6db5222e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.886+0000 7f46e33d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f46d81ad210 con 0x55f6db511580 2024-04-25T00:20:51.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.886+0000 7f46e33d6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f46d81ae680 con 0x55f6db51c570 2024-04-25T00:20:51.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.886+0000 7f46e33d6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f46d81ad8e0 con 0x55f6db511ea0 2024-04-25T00:20:51.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.886+0000 7f46e33d6740 10 client.0 _ll_register_callbacks cb 0x7f46c8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:51.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.886+0000 7f46ddc65640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511580 0x55f6db524940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.886+0000 7f46ddc65640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511580 0x55f6db524940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43988/0 (socket says 172.21.15.107:43988) 2024-04-25T00:20:51.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.886+0000 7f46ddc65640 1 -- 172.21.15.107:0/1603047699 learned_addr learned my addr 172.21.15.107:0/1603047699 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:51.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.887+0000 7f46de466640 1 --2- 172.21.15.107:0/1603047699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6db511ea0 0x55f6db521da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.887+0000 7f46ddc65640 1 --2- 172.21.15.107:0/1603047699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6db51c570 0x55f6db5222e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.887+0000 7f46ddc65640 1 -- 172.21.15.107:0/1603047699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6db511ea0 msgr2=0x55f6db521da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:51.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.887+0000 7f46ddc65640 1 --2- 172.21.15.107:0/1603047699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6db511ea0 0x55f6db521da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.887+0000 7f46ddc65640 1 -- 172.21.15.107:0/1603047699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6db51c570 msgr2=0x55f6db5222e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:51.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.887+0000 7f46ddc65640 1 --2- 172.21.15.107:0/1603047699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6db51c570 0x55f6db5222e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.887+0000 7f46ddc65640 1 -- 172.21.15.107:0/1603047699 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6db522920 con 0x55f6db511580 2024-04-25T00:20:51.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.887+0000 7f46ddc65640 1 --2- 172.21.15.107:0/1603047699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511580 0x55f6db524940 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f46c802a940 tx=0x7f46c8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a283e34877e52f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:51.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.887+0000 7f46c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:20:51.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.887+0000 7f46c1ffb640 1 -- 172.21.15.107:0/1603047699 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8035040 con 0x55f6db511580 2024-04-25T00:20:51.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.887+0000 7f46c1ffb640 1 -- 172.21.15.107:0/1603047699 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f46c80492d0 con 0x55f6db511580 2024-04-25T00:20:51.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.887+0000 7f46e33d6740 1 -- 172.21.15.107:0/1603047699 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46d4010670 con 0x55f6db511580 2024-04-25T00:20:51.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.887+0000 7f46c1ffb640 1 -- 172.21.15.107:0/1603047699 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f46c8059050 con 0x55f6db511580 2024-04-25T00:20:51.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.887+0000 7f46e33d6740 10 client.16560 fetch_fsmap learned FSMap version 28 2024-04-25T00:20:51.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.887+0000 7f46e33d6740 1 -- 172.21.15.107:0/1603047699 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f6db592740 con 0x55f6db511580 2024-04-25T00:20:51.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.887+0000 7f46c1ffb640 1 -- 172.21.15.107:0/1603047699 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46d4010670 con 0x55f6db511580 2024-04-25T00:20:51.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.887+0000 7f46c1ffb640 1 -- 172.21.15.107:0/1603047699 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f46c803a040 con 0x55f6db511580 2024-04-25T00:20:51.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.887+0000 7f46e33d6740 10 client.16560 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:20:51.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.888+0000 7f46e33d6740 10 client.16560 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:20:51.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.888+0000 7f46e33d6740 20 client.16560 populate_metadata read hostname 'smithi107' 2024-04-25T00:20:51.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.888+0000 7f46e33d6740 1 --2- 172.21.15.107:0/1603047699 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55f6db592f20 0x55f6db595340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:51.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.888+0000 7f46e33d6740 4 client.16560 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:20:51.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.888+0000 7f46e33d6740 1 -- 172.21.15.107:0/1603047699 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f6db47df00 con 0x55f6db592f20 2024-04-25T00:20:51.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.888+0000 7f46de466640 1 --2- 172.21.15.107:0/1603047699 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55f6db592f20 0x55f6db595340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:51.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.888+0000 7f46de466640 1 --2- 172.21.15.107:0/1603047699 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55f6db592f20 0x55f6db595340 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=14451153b3d554ba server_cookie=b839b9d74acbc8f4 in_seq=0 out_seq=0 2024-04-25T00:20:51.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.888+0000 7f46c1ffb640 10 client.16560 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:20:51.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.888+0000 7f46c1ffb640 1 -- 172.21.15.107:0/1603047699 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f6db47df00 con 0x55f6db592f20 2024-04-25T00:20:51.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.888+0000 7f46c1ffb640 10 client.16560 handle_command_reply: tid=0 2024-04-25T00:20:51.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.888+0000 7f46e33d6740 1 client.16560 shutdown 2024-04-25T00:20:51.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.888+0000 7f46e33d6740 1 -- 172.21.15.107:0/1603047699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511580 msgr2=0x55f6db524940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.888+0000 7f46e33d6740 1 --2- 172.21.15.107:0/1603047699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511580 0x55f6db524940 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f46c802a940 tx=0x7f46c8009260 comp rx=0 tx=0).stop 2024-04-25T00:20:51.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.888+0000 7f46e33d6740 1 -- 172.21.15.107:0/1603047699 shutdown_connections 2024-04-25T00:20:51.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.888+0000 7f46e33d6740 1 --2- 172.21.15.107:0/1603047699 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55f6db592f20 0x55f6db595340 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.888+0000 7f46e33d6740 1 --2- 172.21.15.107:0/1603047699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6db51c570 0x55f6db5222e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.888+0000 7f46c1ffb640 0 client.16560 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:20:51.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.888+0000 7f46e33d6740 1 --2- 172.21.15.107:0/1603047699 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f6db511ea0 0x55f6db521da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.888+0000 7f46e33d6740 1 --2- 172.21.15.107:0/1603047699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f6db511580 0x55f6db524940 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.888+0000 7f46e33d6740 1 -- 172.21.15.107:0/1603047699 >> 172.21.15.107:0/1603047699 conn(0x55f6db50d8a0 msgr2=0x55f6db527210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:51.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.889+0000 7f46e33d6740 1 -- 172.21.15.107:0/1603047699 shutdown_connections 2024-04-25T00:20:51.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.889+0000 7f46e33d6740 1 -- 172.21.15.107:0/1603047699 wait complete. 2024-04-25T00:20:51.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.889+0000 7f46e33d6740 20 client.16560 trim_cache size 0 max 16384 2024-04-25T00:20:51.897 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:20:51.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.889+0000 7f46e06ca640 1 -- 172.21.15.107:0/566729685 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f46c40a2d00 msgr2=0x7f46c40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.889+0000 7f46e06ca640 1 --2- 172.21.15.107:0/566729685 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f46c40a2d00 0x7f46c40a51c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f46d00ccdc0 tx=0x7f46d009e040 comp rx=0 tx=0).stop 2024-04-25T00:20:51.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.889+0000 7f46e06ca640 1 -- 172.21.15.107:0/566729685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d81066b0 msgr2=0x7f46d819c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:51.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.889+0000 7f46e06ca640 1 --2- 172.21.15.107:0/566729685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d81066b0 0x7f46d819c870 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f46d000cad0 tx=0x7f46d000cf90 comp rx=0 tx=0).stop 2024-04-25T00:20:51.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.889+0000 7f46e06ca640 1 -- 172.21.15.107:0/566729685 shutdown_connections 2024-04-25T00:20:51.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.889+0000 7f46e06ca640 1 --2- 172.21.15.107:0/566729685 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46d81066b0 0x7f46d819c870 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.889+0000 7f46e06ca640 1 --2- 172.21.15.107:0/566729685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f46d8100fd0 0x7f46d81984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.889+0000 7f46e06ca640 1 --2- 172.21.15.107:0/566729685 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f46c40a2d00 0x7f46c40a51c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.889+0000 7f46e06ca640 1 --2- 172.21.15.107:0/566729685 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f46d8100610 0x7f46d8197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:51.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.889+0000 7f46e06ca640 1 -- 172.21.15.107:0/566729685 >> 172.21.15.107:0/566729685 conn(0x7f46d8077740 msgr2=0x7f46d80ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:51.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.889+0000 7f46e06ca640 1 -- 172.21.15.107:0/566729685 shutdown_connections 2024-04-25T00:20:51.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:51.889+0000 7f46e06ca640 1 -- 172.21.15.107:0/566729685 wait complete. 2024-04-25T00:20:51.900 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:20:52.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.000+0000 7fe4eb891640 1 -- 172.21.15.107:0/2633341236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e4100990 msgr2=0x7fe4e4100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:52.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.000+0000 7fe4eb891640 1 --2- 172.21.15.107:0/2633341236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e4100990 0x7fe4e4100df0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0009900 tx=0x7fe4e002a410 comp rx=0 tx=0).stop 2024-04-25T00:20:52.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.000+0000 7fe4eb891640 1 -- 172.21.15.107:0/2633341236 shutdown_connections 2024-04-25T00:20:52.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.000+0000 7fe4eb891640 1 --2- 172.21.15.107:0/2633341236 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4e4101330 0x7fe4e410d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.000+0000 7fe4eb891640 1 --2- 172.21.15.107:0/2633341236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e4100990 0x7fe4e4100df0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.000+0000 7fe4eb891640 1 --2- 172.21.15.107:0/2633341236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4e4106690 0x7fe4e4106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.000+0000 7fe4eb891640 1 -- 172.21.15.107:0/2633341236 >> 172.21.15.107:0/2633341236 conn(0x7fe4e4076440 msgr2=0x7fe4e4076850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:52.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.000+0000 7fe4eb891640 1 -- 172.21.15.107:0/2633341236 shutdown_connections 2024-04-25T00:20:52.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.000+0000 7fe4eb891640 1 -- 172.21.15.107:0/2633341236 wait complete. 2024-04-25T00:20:52.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.000+0000 7fe4eb891640 1 Processor -- start 2024-04-25T00:20:52.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.001+0000 7fe4eb891640 1 -- start start 2024-04-25T00:20:52.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.001+0000 7fe4eb891640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4e4100990 0x7fe4e4196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.001+0000 7fe4eb891640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e4101330 0x7fe4e41973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.001+0000 7fe4eb891640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4e4106690 0x7fe4e419b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.001+0000 7fe4eb891640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e41109e0 con 0x7fe4e4101330 2024-04-25T00:20:52.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.001+0000 7fe4eb891640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e4110860 con 0x7fe4e4106690 2024-04-25T00:20:52.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.001+0000 7fe4eb891640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e4110b60 con 0x7fe4e4100990 2024-04-25T00:20:52.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.001+0000 7fe4e8e2c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e4101330 0x7fe4e41973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.001+0000 7fe4e8e2c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e4101330 0x7fe4e41973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44010/0 (socket says 172.21.15.107:44010) 2024-04-25T00:20:52.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.001+0000 7fe4e8e2c640 1 -- 172.21.15.107:0/1119255192 learned_addr learned my addr 172.21.15.107:0/1119255192 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:52.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.001+0000 7fe4e962d640 1 --2- 172.21.15.107:0/1119255192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4e4100990 0x7fe4e4196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.001+0000 7fe4e8e2c640 1 -- 172.21.15.107:0/1119255192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4e4100990 msgr2=0x7fe4e4196ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:52.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.001+0000 7fe4e8e2c640 1 --2- 172.21.15.107:0/1119255192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4e4100990 0x7fe4e4196ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.001+0000 7fe4e8e2c640 1 -- 172.21.15.107:0/1119255192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4e4106690 msgr2=0x7fe4e419b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:52.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.001+0000 7fe4e8e2c640 1 --2- 172.21.15.107:0/1119255192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4e4106690 0x7fe4e419b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.001+0000 7fe4e8e2c640 1 -- 172.21.15.107:0/1119255192 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e419be90 con 0x7fe4e4101330 2024-04-25T00:20:52.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.001+0000 7fe4e962d640 1 --2- 172.21.15.107:0/1119255192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4e4100990 0x7fe4e4196ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:20:52.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.001+0000 7fe4e8e2c640 1 --2- 172.21.15.107:0/1119255192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e4101330 0x7fe4e41973e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0006010 tx=0x7fe4e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:52.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.001+0000 7fe4d67fc640 1 -- 172.21.15.107:0/1119255192 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e003f030 con 0x7fe4e4101330 2024-04-25T00:20:52.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.001+0000 7fe4d67fc640 1 -- 172.21.15.107:0/1119255192 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e0003ce0 con 0x7fe4e4101330 2024-04-25T00:20:52.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.001+0000 7fe4d67fc640 1 -- 172.21.15.107:0/1119255192 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0035070 con 0x7fe4e4101330 2024-04-25T00:20:52.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.001+0000 7fe4eb891640 1 -- 172.21.15.107:0/1119255192 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e419c120 con 0x7fe4e4101330 2024-04-25T00:20:52.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.001+0000 7fe4eb891640 1 -- 172.21.15.107:0/1119255192 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e41a3a10 con 0x7fe4e4101330 2024-04-25T00:20:52.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.003+0000 7fe4d67fc640 1 -- 172.21.15.107:0/1119255192 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe4e0042030 con 0x7fe4e4101330 2024-04-25T00:20:52.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.004+0000 7fe4ee595740 1 Processor -- start 2024-04-25T00:20:52.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.004+0000 7fe4ee595740 1 -- start start 2024-04-25T00:20:52.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.004+0000 7fe4d67fc640 1 --2- 172.21.15.107:0/1119255192 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe4d00a2d00 0x7fe4d00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.004+0000 7fe4ee595740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4dd2250 0x563fc4dd2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.004+0000 7fe4ee595740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fc4dd2b70 0x563fc4ddf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.004+0000 7fe4ee595740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563fc4ddf780 0x563fc4de1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.004+0000 7fe4ee595740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563fc4d37670 con 0x563fc4dd2250 2024-04-25T00:20:52.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.004+0000 7fe4ee595740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563fc4cdf970 con 0x563fc4ddf780 2024-04-25T00:20:52.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.004+0000 7fe4ee595740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563fc4d33630 con 0x563fc4dd2b70 2024-04-25T00:20:52.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.004+0000 7fe4e962d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4dd2250 0x563fc4dd2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.004+0000 7fe4e9e2e640 1 --2- 172.21.15.107:0/1119255192 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe4d00a2d00 0x7fe4d00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.004+0000 7fe4e962d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4dd2250 0x563fc4dd2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44020/0 (socket says 172.21.15.107:44020) 2024-04-25T00:20:52.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.004+0000 7fe4e962d640 1 -- 172.21.15.107:0/1984422691 learned_addr learned my addr 172.21.15.107:0/1984422691 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:52.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.004+0000 7fe4d67fc640 1 -- 172.21.15.107:0/1119255192 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fe4e00efd80 con 0x7fe4e4101330 2024-04-25T00:20:52.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.005+0000 7fe4e8e2c640 1 --2- 172.21.15.107:0/1984422691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fc4dd2b70 0x563fc4ddf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.005+0000 7fe4e962d640 1 -- 172.21.15.107:0/1984422691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fc4dd2b70 msgr2=0x563fc4ddf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:52.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.005+0000 7fe4e962d640 1 --2- 172.21.15.107:0/1984422691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fc4dd2b70 0x563fc4ddf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.005+0000 7fe4e962d640 1 -- 172.21.15.107:0/1984422691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563fc4ddf780 msgr2=0x563fc4de1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:52.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.005+0000 7fe4e962d640 1 --2- 172.21.15.107:0/1984422691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563fc4ddf780 0x563fc4de1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.005+0000 7fe4e962d640 1 -- 172.21.15.107:0/1984422691 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc4d03400 con 0x563fc4dd2250 2024-04-25T00:20:52.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.005+0000 7fe4e9e2e640 1 --2- 172.21.15.107:0/1119255192 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe4d00a2d00 0x7fe4d00a51c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc009270 tx=0x7fe4dc009760 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:52.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.005+0000 7fe4e962d640 1 --2- 172.21.15.107:0/1984422691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4dd2250 0x563fc4dd2630 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009c50 tx=0x7fe4d800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd8386d9966c14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:52.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.005+0000 7fe4cf7fe640 1 -- 172.21.15.107:0/1984422691 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8016070 con 0x563fc4dd2250 2024-04-25T00:20:52.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.005+0000 7fe4cf7fe640 1 -- 172.21.15.107:0/1984422691 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4d800ee70 con 0x563fc4dd2250 2024-04-25T00:20:52.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.005+0000 7fe4cf7fe640 1 -- 172.21.15.107:0/1984422691 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8010840 con 0x563fc4dd2250 2024-04-25T00:20:52.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.006+0000 7fe4ee595740 1 -- 172.21.15.107:0/1984422691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4dd2250 msgr2=0x563fc4dd2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:52.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.006+0000 7fe4ee595740 1 --2- 172.21.15.107:0/1984422691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4dd2250 0x563fc4dd2630 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8009c50 tx=0x7fe4d800e6c0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.006+0000 7fe4ee595740 1 -- 172.21.15.107:0/1984422691 shutdown_connections 2024-04-25T00:20:52.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.006+0000 7fe4ee595740 1 --2- 172.21.15.107:0/1984422691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563fc4ddf780 0x563fc4de1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.006+0000 7fe4ee595740 1 --2- 172.21.15.107:0/1984422691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fc4dd2b70 0x563fc4ddf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.006+0000 7fe4ee595740 1 --2- 172.21.15.107:0/1984422691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4dd2250 0x563fc4dd2630 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.006+0000 7fe4ee595740 1 -- 172.21.15.107:0/1984422691 >> 172.21.15.107:0/1984422691 conn(0x563fc4dce8a0 msgr2=0x563fc4dcec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:52.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.006+0000 7fe4ee595740 1 -- 172.21.15.107:0/1984422691 shutdown_connections 2024-04-25T00:20:52.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.006+0000 7fe4ee595740 1 -- 172.21.15.107:0/1984422691 wait complete. 2024-04-25T00:20:52.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.006+0000 7fe4ee595740 1 -- start start 2024-04-25T00:20:52.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.006+0000 7fe4ee595740 1 Processor -- start 2024-04-25T00:20:52.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.006+0000 7fe4ee595740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563fc4dd2250 0x563fc4ddf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.006+0000 7fe4ee595740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fc4dd2b70 0x563fc4ddc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.006+0000 7fe4ee595740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4ddf780 0x563fc4ddc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.006+0000 7fe4ee595740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563fc4de2540 con 0x563fc4ddf780 2024-04-25T00:20:52.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.007+0000 7fe4ee595740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563fc4cf3050 con 0x563fc4dd2250 2024-04-25T00:20:52.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.007+0000 7fe4ee595740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563fc4de26c0 con 0x563fc4dd2b70 2024-04-25T00:20:52.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.007+0000 7fe4ee595740 10 client.0 _ll_register_callbacks cb 0x563fc4a4abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:52.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.007+0000 7fe4e9e2e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4ddf780 0x563fc4ddc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.007+0000 7fe4e9e2e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4ddf780 0x563fc4ddc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44034/0 (socket says 172.21.15.107:44034) 2024-04-25T00:20:52.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.007+0000 7fe4e9e2e640 1 -- 172.21.15.107:0/2155519468 learned_addr learned my addr 172.21.15.107:0/2155519468 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:52.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.007+0000 7fe4e8e2c640 1 --2- 172.21.15.107:0/2155519468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fc4dd2b70 0x563fc4ddc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.007+0000 7fe4e962d640 1 --2- 172.21.15.107:0/2155519468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563fc4dd2250 0x563fc4ddf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.007+0000 7fe4e9e2e640 1 -- 172.21.15.107:0/2155519468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fc4dd2b70 msgr2=0x563fc4ddc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:52.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.007+0000 7fe4e9e2e640 1 --2- 172.21.15.107:0/2155519468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fc4dd2b70 0x563fc4ddc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.007+0000 7fe4e9e2e640 1 -- 172.21.15.107:0/2155519468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563fc4dd2250 msgr2=0x563fc4ddf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:52.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.007+0000 7fe4e9e2e640 1 --2- 172.21.15.107:0/2155519468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563fc4dd2250 0x563fc4ddf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.007+0000 7fe4e9e2e640 1 -- 172.21.15.107:0/2155519468 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fc4cc4dc0 con 0x563fc4ddf780 2024-04-25T00:20:52.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.007+0000 7fe4e962d640 1 --2- 172.21.15.107:0/2155519468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563fc4dd2250 0x563fc4ddf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:20:52.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.007+0000 7fe4e9e2e640 1 --2- 172.21.15.107:0/2155519468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4ddf780 0x563fc4ddc890 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0096a0 tx=0x7fe4dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4446c853f4fa30ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:52.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.007+0000 7fe4ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:20:52.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.007+0000 7fe4ccff9640 1 -- 172.21.15.107:0/2155519468 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc023020 con 0x563fc4ddf780 2024-04-25T00:20:52.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.007+0000 7fe4ccff9640 1 -- 172.21.15.107:0/2155519468 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fe4dc0118d0 con 0x563fc4ddf780 2024-04-25T00:20:52.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.007+0000 7fe4ee595740 1 -- 172.21.15.107:0/2155519468 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4d800ee70 con 0x563fc4ddf780 2024-04-25T00:20:52.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.007+0000 7fe4ccff9640 1 -- 172.21.15.107:0/2155519468 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe4dc02e050 con 0x563fc4ddf780 2024-04-25T00:20:52.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.007+0000 7fe4ee595740 10 client.16572 fetch_fsmap learned FSMap version 28 2024-04-25T00:20:52.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.007+0000 7fe4ee595740 1 -- 172.21.15.107:0/2155519468 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563fc4a4b3f0 con 0x563fc4ddf780 2024-04-25T00:20:52.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.008+0000 7fe4ccff9640 1 -- 172.21.15.107:0/2155519468 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4d800ee70 con 0x563fc4ddf780 2024-04-25T00:20:52.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.008+0000 7fe4ccff9640 1 -- 172.21.15.107:0/2155519468 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fe4dc01e3b0 con 0x563fc4ddf780 2024-04-25T00:20:52.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.008+0000 7fe4ee595740 10 client.16572 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:20:52.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.008+0000 7fe4ee595740 10 client.16572 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:20:52.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.008+0000 7fe4ee595740 20 client.16572 populate_metadata read hostname 'smithi107' 2024-04-25T00:20:52.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.008+0000 7fe4ee595740 1 --2- 172.21.15.107:0/2155519468 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x563fc4def880 0x563fc4df1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.008+0000 7fe4ee595740 4 client.16572 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:20:52.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.008+0000 7fe4ee595740 1 -- 172.21.15.107:0/2155519468 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fc4d44bb0 con 0x563fc4def880 2024-04-25T00:20:52.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.008+0000 7fe4e962d640 1 --2- 172.21.15.107:0/2155519468 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x563fc4def880 0x563fc4df1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.009+0000 7fe4ccff9640 10 client.16572 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:20:52.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.009+0000 7fe4e962d640 1 --2- 172.21.15.107:0/2155519468 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x563fc4def880 0x563fc4df1ce0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c4351c84e4f5658c server_cookie=33d000d8ea90022c in_seq=0 out_seq=0 2024-04-25T00:20:52.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.009+0000 7fe4ccff9640 1 -- 172.21.15.107:0/2155519468 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563fc4d44bb0 con 0x563fc4def880 2024-04-25T00:20:52.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.009+0000 7fe4ccff9640 10 client.16572 handle_command_reply: tid=0 2024-04-25T00:20:52.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.009+0000 7fe4ee595740 1 client.16572 shutdown 2024-04-25T00:20:52.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.009+0000 7fe4ee595740 1 -- 172.21.15.107:0/2155519468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4ddf780 msgr2=0x563fc4ddc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:52.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.009+0000 7fe4ee595740 1 --2- 172.21.15.107:0/2155519468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4ddf780 0x563fc4ddc890 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0096a0 tx=0x7fe4dc004a40 comp rx=0 tx=0).stop 2024-04-25T00:20:52.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.010+0000 7fe4ee595740 1 -- 172.21.15.107:0/2155519468 shutdown_connections 2024-04-25T00:20:52.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.010+0000 7fe4ee595740 1 --2- 172.21.15.107:0/2155519468 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x563fc4def880 0x563fc4df1ce0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.010+0000 7fe4ee595740 1 --2- 172.21.15.107:0/2155519468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4ddf780 0x563fc4ddc890 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.010+0000 7fe4ee595740 1 --2- 172.21.15.107:0/2155519468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fc4dd2b70 0x563fc4ddc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.010+0000 7fe4ccff9640 0 client.16572 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:20:52.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.010+0000 7fe4ee595740 1 --2- 172.21.15.107:0/2155519468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563fc4dd2250 0x563fc4ddf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.010+0000 7fe4ee595740 1 -- 172.21.15.107:0/2155519468 >> 172.21.15.107:0/2155519468 conn(0x563fc4dce8a0 msgr2=0x563fc4dd4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:52.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.010+0000 7fe4ee595740 1 -- 172.21.15.107:0/2155519468 shutdown_connections 2024-04-25T00:20:52.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.010+0000 7fe4ee595740 1 -- 172.21.15.107:0/2155519468 wait complete. 2024-04-25T00:20:52.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.010+0000 7fe4ee595740 20 client.16572 trim_cache size 0 max 16384 2024-04-25T00:20:52.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.023+0000 7fe4ee595740 1 Processor -- start 2024-04-25T00:20:52.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.023+0000 7fe4ee595740 1 -- start start 2024-04-25T00:20:52.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.023+0000 7fe4ee595740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fc4dd2250 0x563fc4dd7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.023+0000 7fe4ee595740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563fc4dd2b70 0x563fc4dcd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.023+0000 7fe4ee595740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4ddf780 0x563fc4ddeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.023+0000 7fe4ee595740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e41af1b0 con 0x563fc4ddf780 2024-04-25T00:20:52.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.023+0000 7fe4ee595740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e41ac9c0 con 0x563fc4dd2b70 2024-04-25T00:20:52.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.023+0000 7fe4ee595740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e41aa810 con 0x563fc4dd2250 2024-04-25T00:20:52.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.023+0000 7fe4e9e2e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4ddf780 0x563fc4ddeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.023+0000 7fe4e9e2e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4ddf780 0x563fc4ddeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44046/0 (socket says 172.21.15.107:44046) 2024-04-25T00:20:52.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.023+0000 7fe4e9e2e640 1 -- 172.21.15.107:0/3826229134 learned_addr learned my addr 172.21.15.107:0/3826229134 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:52.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.023+0000 7fe4e8e2c640 1 --2- 172.21.15.107:0/3826229134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563fc4dd2b70 0x563fc4dcd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.023+0000 7fe4e9e2e640 1 -- 172.21.15.107:0/3826229134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fc4dd2250 msgr2=0x563fc4dd7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:52.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.023+0000 7fe4e9e2e640 1 --2- 172.21.15.107:0/3826229134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fc4dd2250 0x563fc4dd7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.023+0000 7fe4e9e2e640 1 -- 172.21.15.107:0/3826229134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563fc4dd2b70 msgr2=0x563fc4dcd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:52.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.023+0000 7fe4e9e2e640 1 --2- 172.21.15.107:0/3826229134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563fc4dd2b70 0x563fc4dcd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.023+0000 7fe4e9e2e640 1 -- 172.21.15.107:0/3826229134 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc4b157d0 con 0x563fc4ddf780 2024-04-25T00:20:52.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.024+0000 7fe4e9e2e640 1 --2- 172.21.15.107:0/3826229134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4ddf780 0x563fc4ddeb90 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc010b90 tx=0x7fe4dc005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86dce30304067dbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:52.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.024+0000 7fe4ceffd640 1 -- 172.21.15.107:0/3826229134 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc023070 con 0x563fc4ddf780 2024-04-25T00:20:52.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.024+0000 7fe4ceffd640 1 -- 172.21.15.107:0/3826229134 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4dc036030 con 0x563fc4ddf780 2024-04-25T00:20:52.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.024+0000 7fe4ceffd640 1 -- 172.21.15.107:0/3826229134 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc011930 con 0x563fc4ddf780 2024-04-25T00:20:52.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.024+0000 7fe4ee595740 1 -- 172.21.15.107:0/3826229134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4ddf780 msgr2=0x563fc4ddeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:52.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.024+0000 7fe4ee595740 1 --2- 172.21.15.107:0/3826229134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4ddf780 0x563fc4ddeb90 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc010b90 tx=0x7fe4dc005b70 comp rx=0 tx=0).stop 2024-04-25T00:20:52.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.024+0000 7fe4ee595740 1 -- 172.21.15.107:0/3826229134 shutdown_connections 2024-04-25T00:20:52.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.024+0000 7fe4ee595740 1 --2- 172.21.15.107:0/3826229134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4ddf780 0x563fc4ddeb90 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.024+0000 7fe4ee595740 1 --2- 172.21.15.107:0/3826229134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563fc4dd2b70 0x563fc4dcd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.024+0000 7fe4ee595740 1 --2- 172.21.15.107:0/3826229134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fc4dd2250 0x563fc4dd7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.024+0000 7fe4ee595740 1 -- 172.21.15.107:0/3826229134 >> 172.21.15.107:0/3826229134 conn(0x563fc4dce8a0 msgr2=0x563fc4dcd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:52.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.024+0000 7fe4ee595740 1 -- 172.21.15.107:0/3826229134 shutdown_connections 2024-04-25T00:20:52.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.024+0000 7fe4ee595740 1 -- 172.21.15.107:0/3826229134 wait complete. 2024-04-25T00:20:52.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.025+0000 7fe4ee595740 1 -- start start 2024-04-25T00:20:52.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.025+0000 7fe4ee595740 1 Processor -- start 2024-04-25T00:20:52.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.025+0000 7fe4ee595740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fc4dd2250 0x563fc4de7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.025+0000 7fe4ee595740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4dd2b70 0x563fc4dd8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.025+0000 7fe4ee595740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563fc4ddf780 0x563fc4dd9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.025+0000 7fe4ee595740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e41aae80 con 0x563fc4dd2b70 2024-04-25T00:20:52.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.025+0000 7fe4ee595740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e41ac2f0 con 0x563fc4ddf780 2024-04-25T00:20:52.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.025+0000 7fe4ee595740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e41ab550 con 0x563fc4dd2250 2024-04-25T00:20:52.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.025+0000 7fe4ee595740 10 client.0 _ll_register_callbacks cb 0x7fe4dc016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:52.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.025+0000 7fe4e8e2c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4dd2b70 0x563fc4dd8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.025+0000 7fe4e8e2c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4dd2b70 0x563fc4dd8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44062/0 (socket says 172.21.15.107:44062) 2024-04-25T00:20:52.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.025+0000 7fe4e8e2c640 1 -- 172.21.15.107:0/3314104979 learned_addr learned my addr 172.21.15.107:0/3314104979 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:52.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.025+0000 7fe4e8e2c640 1 -- 172.21.15.107:0/3314104979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fc4dd2250 msgr2=0x563fc4de7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:52.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.025+0000 7fe4e8e2c640 1 --2- 172.21.15.107:0/3314104979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fc4dd2250 0x563fc4de7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.025+0000 7fe4e8e2c640 1 -- 172.21.15.107:0/3314104979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563fc4ddf780 msgr2=0x563fc4dd9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:52.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.025+0000 7fe4e9e2e640 1 --2- 172.21.15.107:0/3314104979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563fc4ddf780 0x563fc4dd9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.025+0000 7fe4e8e2c640 1 --2- 172.21.15.107:0/3314104979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563fc4ddf780 0x563fc4dd9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.025+0000 7fe4e8e2c640 1 -- 172.21.15.107:0/3314104979 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fc4dd9bc0 con 0x563fc4dd2b70 2024-04-25T00:20:52.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.026+0000 7fe4e8e2c640 1 --2- 172.21.15.107:0/3314104979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4dd2b70 0x563fc4dd8d80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00ed1c0 tx=0x7fe4e00bbc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6bd87fc6f72857 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:52.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.026+0000 7fe4ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:20:52.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.026+0000 7fe4ccff9640 1 -- 172.21.15.107:0/3314104979 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0049040 con 0x563fc4dd2b70 2024-04-25T00:20:52.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.026+0000 7fe4ccff9640 1 -- 172.21.15.107:0/3314104979 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fe4e00d54f0 con 0x563fc4dd2b70 2024-04-25T00:20:52.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.026+0000 7fe4ee595740 1 -- 172.21.15.107:0/3314104979 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4dc036030 con 0x563fc4dd2b70 2024-04-25T00:20:52.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.026+0000 7fe4ccff9640 1 -- 172.21.15.107:0/3314104979 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe4e00d5130 con 0x563fc4dd2b70 2024-04-25T00:20:52.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.026+0000 7fe4ee595740 10 client.16578 fetch_fsmap learned FSMap version 28 2024-04-25T00:20:52.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.026+0000 7fe4ee595740 1 -- 172.21.15.107:0/3314104979 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563fc4e55ef0 con 0x563fc4dd2b70 2024-04-25T00:20:52.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.026+0000 7fe4ccff9640 1 -- 172.21.15.107:0/3314104979 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4dc036030 con 0x563fc4dd2b70 2024-04-25T00:20:52.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.026+0000 7fe4ccff9640 1 -- 172.21.15.107:0/3314104979 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fe4e00d8370 con 0x563fc4dd2b70 2024-04-25T00:20:52.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.026+0000 7fe4ee595740 10 client.16578 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:20:52.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.026+0000 7fe4ee595740 10 client.16578 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:20:52.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.026+0000 7fe4ee595740 20 client.16578 populate_metadata read hostname 'smithi107' 2024-04-25T00:20:52.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.026+0000 7fe4ee595740 1 --2- 172.21.15.107:0/3314104979 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x563fc4e566d0 0x563fc4e58af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.026+0000 7fe4ee595740 4 client.16578 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:20:52.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.026+0000 7fe4ee595740 1 -- 172.21.15.107:0/3314104979 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563fc4d3ef00 con 0x563fc4e566d0 2024-04-25T00:20:52.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.026+0000 7fe4e962d640 1 --2- 172.21.15.107:0/3314104979 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x563fc4e566d0 0x563fc4e58af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.027+0000 7fe4ccff9640 10 client.16578 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:20:52.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.027+0000 7fe4e962d640 1 --2- 172.21.15.107:0/3314104979 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x563fc4e566d0 0x563fc4e58af0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=abd74d80d8e9b1b server_cookie=2edf8ac4f0a19f53 in_seq=0 out_seq=0 2024-04-25T00:20:52.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.027+0000 7fe4ccff9640 1 -- 172.21.15.107:0/3314104979 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563fc4d3ef00 con 0x563fc4e566d0 2024-04-25T00:20:52.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.027+0000 7fe4ccff9640 10 client.16578 handle_command_reply: tid=0 2024-04-25T00:20:52.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.027+0000 7fe4ee595740 1 client.16578 shutdown 2024-04-25T00:20:52.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.027+0000 7fe4ee595740 1 -- 172.21.15.107:0/3314104979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4dd2b70 msgr2=0x563fc4dd8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:52.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.027+0000 7fe4ee595740 1 --2- 172.21.15.107:0/3314104979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4dd2b70 0x563fc4dd8d80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00ed1c0 tx=0x7fe4e00bbc60 comp rx=0 tx=0).stop 2024-04-25T00:20:52.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.027+0000 7fe4ee595740 1 -- 172.21.15.107:0/3314104979 shutdown_connections 2024-04-25T00:20:52.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.027+0000 7fe4ee595740 1 --2- 172.21.15.107:0/3314104979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563fc4ddf780 0x563fc4dd9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.027+0000 7fe4ee595740 1 --2- 172.21.15.107:0/3314104979 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x563fc4e566d0 0x563fc4e58af0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.027+0000 7fe4ee595740 1 --2- 172.21.15.107:0/3314104979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563fc4dd2b70 0x563fc4dd8d80 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.027+0000 7fe4ccff9640 0 client.16578 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:20:52.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.027+0000 7fe4ee595740 1 --2- 172.21.15.107:0/3314104979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563fc4dd2250 0x563fc4de7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.027+0000 7fe4ee595740 1 -- 172.21.15.107:0/3314104979 >> 172.21.15.107:0/3314104979 conn(0x563fc4dce8a0 msgr2=0x563fc4dcdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:52.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.027+0000 7fe4ee595740 1 -- 172.21.15.107:0/3314104979 shutdown_connections 2024-04-25T00:20:52.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.027+0000 7fe4ee595740 1 -- 172.21.15.107:0/3314104979 wait complete. 2024-04-25T00:20:52.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.028+0000 7fe4ee595740 20 client.16578 trim_cache size 0 max 16384 2024-04-25T00:20:52.035 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:20:52.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.028+0000 7fe4eb891640 1 -- 172.21.15.107:0/1119255192 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe4d00a2d00 msgr2=0x7fe4d00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:52.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.028+0000 7fe4eb891640 1 --2- 172.21.15.107:0/1119255192 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe4d00a2d00 0x7fe4d00a51c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc009270 tx=0x7fe4dc009760 comp rx=0 tx=0).stop 2024-04-25T00:20:52.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.028+0000 7fe4eb891640 1 -- 172.21.15.107:0/1119255192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e4101330 msgr2=0x7fe4e41973e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:52.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.028+0000 7fe4eb891640 1 --2- 172.21.15.107:0/1119255192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e4101330 0x7fe4e41973e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0006010 tx=0x7fe4e00042f0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.028+0000 7fe4eb891640 1 -- 172.21.15.107:0/1119255192 shutdown_connections 2024-04-25T00:20:52.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.028+0000 7fe4eb891640 1 --2- 172.21.15.107:0/1119255192 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe4d00a2d00 0x7fe4d00a51c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.028+0000 7fe4eb891640 1 --2- 172.21.15.107:0/1119255192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe4e4106690 0x7fe4e419b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.028+0000 7fe4eb891640 1 --2- 172.21.15.107:0/1119255192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4e4101330 0x7fe4e41973e0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.028+0000 7fe4eb891640 1 --2- 172.21.15.107:0/1119255192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe4e4100990 0x7fe4e4196ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.028+0000 7fe4eb891640 1 -- 172.21.15.107:0/1119255192 >> 172.21.15.107:0/1119255192 conn(0x7fe4e4076440 msgr2=0x7fe4e40ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:52.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.028+0000 7fe4eb891640 1 -- 172.21.15.107:0/1119255192 shutdown_connections 2024-04-25T00:20:52.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.028+0000 7fe4eb891640 1 -- 172.21.15.107:0/1119255192 wait complete. 2024-04-25T00:20:52.039 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:20:52.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.137+0000 7f99369b7640 1 -- 172.21.15.107:0/84988384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99301066b0 msgr2=0x7f9930106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:52.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.137+0000 7f99369b7640 1 --2- 172.21.15.107:0/84988384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99301066b0 0x7f9930106a90 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f9920009930 tx=0x7f992002a420 comp rx=0 tx=0).stop 2024-04-25T00:20:52.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.137+0000 7f99369b7640 1 -- 172.21.15.107:0/84988384 shutdown_connections 2024-04-25T00:20:52.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.137+0000 7f99369b7640 1 --2- 172.21.15.107:0/84988384 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9930100fd0 0x7f993010d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.137+0000 7f99369b7640 1 --2- 172.21.15.107:0/84988384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9930100610 0x7f9930100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.137+0000 7f99369b7640 1 --2- 172.21.15.107:0/84988384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99301066b0 0x7f9930106a90 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.137+0000 7f99369b7640 1 -- 172.21.15.107:0/84988384 >> 172.21.15.107:0/84988384 conn(0x7f9930077740 msgr2=0x7f9930077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:52.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.138+0000 7f99369b7640 1 -- 172.21.15.107:0/84988384 shutdown_connections 2024-04-25T00:20:52.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.138+0000 7f99369b7640 1 -- 172.21.15.107:0/84988384 wait complete. 2024-04-25T00:20:52.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.138+0000 7f99369b7640 1 Processor -- start 2024-04-25T00:20:52.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.138+0000 7f99369b7640 1 -- start start 2024-04-25T00:20:52.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.138+0000 7f99369b7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9930100610 0x7f9930198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.138+0000 7f99369b7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9930100fd0 0x7f9930199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.138+0000 7f99369b7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99301066b0 0x7f99301a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.138+0000 7f99369b7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9930110a30 con 0x7f9930100fd0 2024-04-25T00:20:52.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.138+0000 7f99369b7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f99301108b0 con 0x7f9930100610 2024-04-25T00:20:52.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.138+0000 7f99369b7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9930110bb0 con 0x7f99301066b0 2024-04-25T00:20:52.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.139+0000 7f992f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9930100fd0 0x7f9930199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.139+0000 7f992f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9930100fd0 0x7f9930199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44082/0 (socket says 172.21.15.107:44082) 2024-04-25T00:20:52.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.139+0000 7f992f7fe640 1 -- 172.21.15.107:0/3788042117 learned_addr learned my addr 172.21.15.107:0/3788042117 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:52.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.139+0000 7f9934f54640 1 --2- 172.21.15.107:0/3788042117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99301066b0 0x7f99301a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.139+0000 7f992ffff640 1 --2- 172.21.15.107:0/3788042117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9930100610 0x7f9930198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.139+0000 7f992f7fe640 1 -- 172.21.15.107:0/3788042117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99301066b0 msgr2=0x7f99301a3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:52.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.139+0000 7f992f7fe640 1 --2- 172.21.15.107:0/3788042117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99301066b0 0x7f99301a3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.139+0000 7f992f7fe640 1 -- 172.21.15.107:0/3788042117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9930100610 msgr2=0x7f9930198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:52.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.139+0000 7f992f7fe640 1 --2- 172.21.15.107:0/3788042117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9930100610 0x7f9930198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.139+0000 7f992f7fe640 1 -- 172.21.15.107:0/3788042117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99301a43f0 con 0x7f9930100fd0 2024-04-25T00:20:52.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.139+0000 7f9934f54640 1 --2- 172.21.15.107:0/3788042117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99301066b0 0x7f99301a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:20:52.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.139+0000 7f992ffff640 1 --2- 172.21.15.107:0/3788042117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9930100610 0x7f9930198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:20:52.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.139+0000 7f992f7fe640 1 --2- 172.21.15.107:0/3788042117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9930100fd0 0x7f9930199370 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f992400d950 tx=0x7f992400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:52.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.139+0000 7f992d7fa640 1 -- 172.21.15.107:0/3788042117 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9924017070 con 0x7f9930100fd0 2024-04-25T00:20:52.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.139+0000 7f992d7fa640 1 -- 172.21.15.107:0/3788042117 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9924009c30 con 0x7f9930100fd0 2024-04-25T00:20:52.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.139+0000 7f99369b7640 1 -- 172.21.15.107:0/3788042117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99301a45c0 con 0x7f9930100fd0 2024-04-25T00:20:52.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.139+0000 7f992d7fa640 1 -- 172.21.15.107:0/3788042117 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992400f6b0 con 0x7f9930100fd0 2024-04-25T00:20:52.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.139+0000 7f99369b7640 1 -- 172.21.15.107:0/3788042117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99301a4a80 con 0x7f9930100fd0 2024-04-25T00:20:52.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.141+0000 7f992d7fa640 1 -- 172.21.15.107:0/3788042117 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f99240109d0 con 0x7f9930100fd0 2024-04-25T00:20:52.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.142+0000 7f99397a3740 1 Processor -- start 2024-04-25T00:20:52.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.142+0000 7f99397a3740 1 -- start start 2024-04-25T00:20:52.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.142+0000 7f992d7fa640 1 --2- 172.21.15.107:0/3788042117 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f99180a2d00 0x7f99180a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.142+0000 7f99397a3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55993d75b580 0x55993d75b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.142+0000 7f99397a3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d75bea0 0x55993d766030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.142+0000 7f99397a3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993d766570 0x55993d768960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.142+0000 7f9934f54640 1 --2- 172.21.15.107:0/3788042117 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f99180a2d00 0x7f99180a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.142+0000 7f99397a3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55993d6c0670 con 0x55993d75bea0 2024-04-25T00:20:52.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.142+0000 7f99397a3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55993d668970 con 0x55993d766570 2024-04-25T00:20:52.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.142+0000 7f99397a3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55993d6bc630 con 0x55993d75b580 2024-04-25T00:20:52.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.142+0000 7f992d7fa640 1 -- 172.21.15.107:0/3788042117 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f99240cf500 con 0x7f9930100fd0 2024-04-25T00:20:52.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.142+0000 7f992f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d75bea0 0x55993d766030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.142+0000 7f992f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d75bea0 0x55993d766030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44092/0 (socket says 172.21.15.107:44092) 2024-04-25T00:20:52.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.142+0000 7f992f7fe640 1 -- 172.21.15.107:0/2196229594 learned_addr learned my addr 172.21.15.107:0/2196229594 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:52.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.142+0000 7f992ffff640 1 --2- 172.21.15.107:0/2196229594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55993d75b580 0x55993d75b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.142+0000 7f9934f54640 1 --2- 172.21.15.107:0/2196229594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993d766570 0x55993d768960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.142+0000 7f992f7fe640 1 -- 172.21.15.107:0/2196229594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55993d75b580 msgr2=0x55993d75b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:52.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.142+0000 7f992f7fe640 1 --2- 172.21.15.107:0/2196229594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55993d75b580 0x55993d75b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.142+0000 7f992f7fe640 1 -- 172.21.15.107:0/2196229594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993d766570 msgr2=0x55993d768960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:52.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.142+0000 7f9934f54640 1 --2- 172.21.15.107:0/3788042117 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f99180a2d00 0x7f99180a51c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9928005c40 tx=0x7f9928009ac0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:52.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.142+0000 7f992f7fe640 1 --2- 172.21.15.107:0/2196229594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993d766570 0x55993d768960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.142+0000 7f992f7fe640 1 -- 172.21.15.107:0/2196229594 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55993d68c400 con 0x55993d75bea0 2024-04-25T00:20:52.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.143+0000 7f992f7fe640 1 --2- 172.21.15.107:0/2196229594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d75bea0 0x55993d766030 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9924097b10 tx=0x7f9924005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330e806bade8d93e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:52.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.143+0000 7f99167fc640 1 -- 172.21.15.107:0/2196229594 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9924020040 con 0x55993d75bea0 2024-04-25T00:20:52.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.143+0000 7f99167fc640 1 -- 172.21.15.107:0/2196229594 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9924086070 con 0x55993d75bea0 2024-04-25T00:20:52.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.143+0000 7f99167fc640 1 -- 172.21.15.107:0/2196229594 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99240cf130 con 0x55993d75bea0 2024-04-25T00:20:52.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.143+0000 7f99397a3740 1 -- 172.21.15.107:0/2196229594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d75bea0 msgr2=0x55993d766030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:52.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.143+0000 7f99397a3740 1 --2- 172.21.15.107:0/2196229594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d75bea0 0x55993d766030 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9924097b10 tx=0x7f9924005f90 comp rx=0 tx=0).stop 2024-04-25T00:20:52.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.143+0000 7f99397a3740 1 -- 172.21.15.107:0/2196229594 shutdown_connections 2024-04-25T00:20:52.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.143+0000 7f99397a3740 1 --2- 172.21.15.107:0/2196229594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993d766570 0x55993d768960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.143+0000 7f99397a3740 1 --2- 172.21.15.107:0/2196229594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d75bea0 0x55993d766030 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.143+0000 7f99397a3740 1 --2- 172.21.15.107:0/2196229594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55993d75b580 0x55993d75b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.143+0000 7f99397a3740 1 -- 172.21.15.107:0/2196229594 >> 172.21.15.107:0/2196229594 conn(0x55993d7578a0 msgr2=0x55993d757c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:52.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.143+0000 7f99397a3740 1 -- 172.21.15.107:0/2196229594 shutdown_connections 2024-04-25T00:20:52.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.143+0000 7f99397a3740 1 -- 172.21.15.107:0/2196229594 wait complete. 2024-04-25T00:20:52.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.144+0000 7f99397a3740 1 -- start start 2024-04-25T00:20:52.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.144+0000 7f99397a3740 1 Processor -- start 2024-04-25T00:20:52.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.144+0000 7f99397a3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993d75b580 0x55993d765ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.144+0000 7f99397a3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55993d75bea0 0x55993d763250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.144+0000 7f99397a3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d766570 0x55993d763880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.144+0000 7f99397a3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55993d769300 con 0x55993d766570 2024-04-25T00:20:52.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.144+0000 7f99397a3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55993d67c050 con 0x55993d75b580 2024-04-25T00:20:52.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.144+0000 7f99397a3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55993d769480 con 0x55993d75bea0 2024-04-25T00:20:52.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.144+0000 7f99397a3740 10 client.0 _ll_register_callbacks cb 0x55993d3d3be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:52.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.144+0000 7f9934f54640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d766570 0x55993d763880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.144+0000 7f9934f54640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d766570 0x55993d763880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44100/0 (socket says 172.21.15.107:44100) 2024-04-25T00:20:52.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.144+0000 7f9934f54640 1 -- 172.21.15.107:0/3697582547 learned_addr learned my addr 172.21.15.107:0/3697582547 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:52.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.144+0000 7f992f7fe640 1 --2- 172.21.15.107:0/3697582547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55993d75bea0 0x55993d763250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.145+0000 7f9934f54640 1 -- 172.21.15.107:0/3697582547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55993d75bea0 msgr2=0x55993d763250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:52.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.145+0000 7f9934f54640 1 --2- 172.21.15.107:0/3697582547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55993d75bea0 0x55993d763250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.145+0000 7f9934f54640 1 -- 172.21.15.107:0/3697582547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993d75b580 msgr2=0x55993d765ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:52.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.145+0000 7f9934f54640 1 --2- 172.21.15.107:0/3697582547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993d75b580 0x55993d765ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.145+0000 7f9934f54640 1 -- 172.21.15.107:0/3697582547 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55993d64ddc0 con 0x55993d766570 2024-04-25T00:20:52.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.145+0000 7f9934f54640 1 --2- 172.21.15.107:0/3697582547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d766570 0x55993d763880 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f992801ca60 tx=0x7f9928004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e375e7151572b05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:52.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.145+0000 7f990f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:20:52.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.145+0000 7f990f7fe640 1 -- 172.21.15.107:0/3697582547 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992802a060 con 0x55993d766570 2024-04-25T00:20:52.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.145+0000 7f990f7fe640 1 -- 172.21.15.107:0/3697582547 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f992800eb20 con 0x55993d766570 2024-04-25T00:20:52.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.145+0000 7f99397a3740 1 -- 172.21.15.107:0/3697582547 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9924086070 con 0x55993d766570 2024-04-25T00:20:52.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.145+0000 7f990f7fe640 1 -- 172.21.15.107:0/3697582547 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9928031050 con 0x55993d766570 2024-04-25T00:20:52.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.145+0000 7f99397a3740 10 client.16590 fetch_fsmap learned FSMap version 28 2024-04-25T00:20:52.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.145+0000 7f99397a3740 1 -- 172.21.15.107:0/3697582547 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55993d3d43f0 con 0x55993d766570 2024-04-25T00:20:52.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.145+0000 7f990f7fe640 1 -- 172.21.15.107:0/3697582547 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9924086070 con 0x55993d766570 2024-04-25T00:20:52.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.145+0000 7f990f7fe640 1 -- 172.21.15.107:0/3697582547 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f9928024070 con 0x55993d766570 2024-04-25T00:20:52.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.145+0000 7f99397a3740 10 client.16590 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:20:52.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.145+0000 7f99397a3740 10 client.16590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:20:52.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.145+0000 7f99397a3740 20 client.16590 populate_metadata read hostname 'smithi107' 2024-04-25T00:20:52.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.145+0000 7f99397a3740 1 --2- 172.21.15.107:0/3697582547 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55993d7766d0 0x55993d778b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.145+0000 7f99397a3740 4 client.16590 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:20:52.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.145+0000 7f99397a3740 1 -- 172.21.15.107:0/3697582547 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55993d6cdbb0 con 0x55993d7766d0 2024-04-25T00:20:52.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.146+0000 7f992ffff640 1 --2- 172.21.15.107:0/3697582547 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55993d7766d0 0x55993d778b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.146+0000 7f990f7fe640 10 client.16590 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:52.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.146+0000 7f992ffff640 1 --2- 172.21.15.107:0/3697582547 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55993d7766d0 0x55993d778b50 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2cee3808e3b493f server_cookie=8159914fb941ffb4 in_seq=0 out_seq=0 2024-04-25T00:20:52.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.147+0000 7f990f7fe640 1 -- 172.21.15.107:0/3697582547 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55993d6cdbb0 con 0x55993d7766d0 2024-04-25T00:20:52.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.147+0000 7f990f7fe640 10 client.16590 handle_command_reply: tid=0 2024-04-25T00:20:52.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.147+0000 7f99397a3740 1 client.16590 shutdown 2024-04-25T00:20:52.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.147+0000 7f99397a3740 1 -- 172.21.15.107:0/3697582547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d766570 msgr2=0x55993d763880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:52.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.147+0000 7f99397a3740 1 --2- 172.21.15.107:0/3697582547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d766570 0x55993d763880 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f992801ca60 tx=0x7f9928004ab0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.147+0000 7f99397a3740 1 -- 172.21.15.107:0/3697582547 shutdown_connections 2024-04-25T00:20:52.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.147+0000 7f99397a3740 1 --2- 172.21.15.107:0/3697582547 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55993d7766d0 0x55993d778b50 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.147+0000 7f99397a3740 1 --2- 172.21.15.107:0/3697582547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d766570 0x55993d763880 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.147+0000 7f99397a3740 1 --2- 172.21.15.107:0/3697582547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55993d75bea0 0x55993d763250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.147+0000 7f99397a3740 1 --2- 172.21.15.107:0/3697582547 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993d75b580 0x55993d765ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.147+0000 7f990f7fe640 0 client.16590 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:52.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.147+0000 7f99397a3740 1 -- 172.21.15.107:0/3697582547 >> 172.21.15.107:0/3697582547 conn(0x55993d7578a0 msgr2=0x55993d756b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:52.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.147+0000 7f99397a3740 1 -- 172.21.15.107:0/3697582547 shutdown_connections 2024-04-25T00:20:52.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.147+0000 7f99397a3740 1 -- 172.21.15.107:0/3697582547 wait complete. 2024-04-25T00:20:52.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.147+0000 7f99397a3740 20 client.16590 trim_cache size 0 max 16384 2024-04-25T00:20:52.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.160+0000 7f99397a3740 1 Processor -- start 2024-04-25T00:20:52.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.161+0000 7f99397a3740 1 -- start start 2024-04-25T00:20:52.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.161+0000 7f99397a3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55993d75b580 0x55993d7721c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.161+0000 7f99397a3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d75bea0 0x55993d77e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.161+0000 7f99397a3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993d766570 0x55993d77e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.161+0000 7f99397a3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f99301b1500 con 0x55993d75bea0 2024-04-25T00:20:52.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.161+0000 7f99397a3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f99301aed10 con 0x55993d766570 2024-04-25T00:20:52.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.161+0000 7f99397a3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f99301acb60 con 0x55993d75b580 2024-04-25T00:20:52.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.161+0000 7f992f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d75bea0 0x55993d77e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.161+0000 7f992f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d75bea0 0x55993d77e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44104/0 (socket says 172.21.15.107:44104) 2024-04-25T00:20:52.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.161+0000 7f992f7fe640 1 -- 172.21.15.107:0/504733097 learned_addr learned my addr 172.21.15.107:0/504733097 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:52.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.161+0000 7f992ffff640 1 --2- 172.21.15.107:0/504733097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55993d75b580 0x55993d7721c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.161+0000 7f9934f54640 1 --2- 172.21.15.107:0/504733097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993d766570 0x55993d77e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.161+0000 7f992f7fe640 1 -- 172.21.15.107:0/504733097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55993d75b580 msgr2=0x55993d7721c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:52.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.161+0000 7f992f7fe640 1 --2- 172.21.15.107:0/504733097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55993d75b580 0x55993d7721c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.161+0000 7f992f7fe640 1 -- 172.21.15.107:0/504733097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993d766570 msgr2=0x55993d77e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:52.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.161+0000 7f992f7fe640 1 --2- 172.21.15.107:0/504733097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993d766570 0x55993d77e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.161+0000 7f992f7fe640 1 -- 172.21.15.107:0/504733097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55993d49e7d0 con 0x55993d75bea0 2024-04-25T00:20:52.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.161+0000 7f992f7fe640 1 --2- 172.21.15.107:0/504733097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d75bea0 0x55993d77e460 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f99240ccee0 tx=0x7f9924005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830bc68c68035a6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:52.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.162+0000 7f9915ffb640 1 -- 172.21.15.107:0/504733097 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99240b9070 con 0x55993d75bea0 2024-04-25T00:20:52.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.162+0000 7f9915ffb640 1 -- 172.21.15.107:0/504733097 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f992409fa60 con 0x55993d75bea0 2024-04-25T00:20:52.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.162+0000 7f9915ffb640 1 -- 172.21.15.107:0/504733097 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9924020040 con 0x55993d75bea0 2024-04-25T00:20:52.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.163+0000 7f99397a3740 1 -- 172.21.15.107:0/504733097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d75bea0 msgr2=0x55993d77e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.163+0000 7f99397a3740 1 --2- 172.21.15.107:0/504733097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d75bea0 0x55993d77e460 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f99240ccee0 tx=0x7f9924005880 comp rx=0 tx=0).stop 2024-04-25T00:20:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.163+0000 7f99397a3740 1 -- 172.21.15.107:0/504733097 shutdown_connections 2024-04-25T00:20:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.163+0000 7f99397a3740 1 --2- 172.21.15.107:0/504733097 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993d766570 0x55993d77e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.163+0000 7f99397a3740 1 --2- 172.21.15.107:0/504733097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d75bea0 0x55993d77e460 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.163+0000 7f99397a3740 1 --2- 172.21.15.107:0/504733097 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55993d75b580 0x55993d7721c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.163+0000 7f99397a3740 1 -- 172.21.15.107:0/504733097 >> 172.21.15.107:0/504733097 conn(0x55993d7578a0 msgr2=0x55993d762790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.163+0000 7f99397a3740 1 -- 172.21.15.107:0/504733097 shutdown_connections 2024-04-25T00:20:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.163+0000 7f99397a3740 1 -- 172.21.15.107:0/504733097 wait complete. 2024-04-25T00:20:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.163+0000 7f99397a3740 1 -- start start 2024-04-25T00:20:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.163+0000 7f99397a3740 1 Processor -- start 2024-04-25T00:20:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f99397a3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d75b580 0x55993d76a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f99397a3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993d75bea0 0x55993d76a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f99397a3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55993d766570 0x55993d76abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f99397a3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f99301ad1d0 con 0x55993d75b580 2024-04-25T00:20:52.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f99397a3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f99301ae640 con 0x55993d75bea0 2024-04-25T00:20:52.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f99397a3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f99301ad8a0 con 0x55993d766570 2024-04-25T00:20:52.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f99397a3740 10 client.0 _ll_register_callbacks cb 0x7f9928009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:52.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f992ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d75b580 0x55993d76a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f992ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d75b580 0x55993d76a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44120/0 (socket says 172.21.15.107:44120) 2024-04-25T00:20:52.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f992ffff640 1 -- 172.21.15.107:0/1198967551 learned_addr learned my addr 172.21.15.107:0/1198967551 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:52.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f992f7fe640 1 --2- 172.21.15.107:0/1198967551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993d75bea0 0x55993d76a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f9934f54640 1 --2- 172.21.15.107:0/1198967551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55993d766570 0x55993d76abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f992ffff640 1 -- 172.21.15.107:0/1198967551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55993d766570 msgr2=0x55993d76abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:52.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f992ffff640 1 --2- 172.21.15.107:0/1198967551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55993d766570 0x55993d76abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f992ffff640 1 -- 172.21.15.107:0/1198967551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993d75bea0 msgr2=0x55993d76a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:52.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f992ffff640 1 --2- 172.21.15.107:0/1198967551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993d75bea0 0x55993d76a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f992ffff640 1 -- 172.21.15.107:0/1198967551 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55993d7dbd20 con 0x55993d75b580 2024-04-25T00:20:52.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f992ffff640 1 --2- 172.21.15.107:0/1198967551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d75b580 0x55993d76a130 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f9920006010 tx=0x7f9920004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2716cd19fb93a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:52.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f990f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:20:52.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f990f7fe640 1 -- 172.21.15.107:0/1198967551 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9920035040 con 0x55993d75b580 2024-04-25T00:20:52.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f990f7fe640 1 -- 172.21.15.107:0/1198967551 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f9920004130 con 0x55993d75b580 2024-04-25T00:20:52.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f99397a3740 1 -- 172.21.15.107:0/1198967551 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f992409fa60 con 0x55993d75b580 2024-04-25T00:20:52.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f990f7fe640 1 -- 172.21.15.107:0/1198967551 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9920052050 con 0x55993d75b580 2024-04-25T00:20:52.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f99397a3740 10 client.16596 fetch_fsmap learned FSMap version 28 2024-04-25T00:20:52.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.164+0000 7f99397a3740 1 -- 172.21.15.107:0/1198967551 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55993d7dd060 con 0x55993d75b580 2024-04-25T00:20:52.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.165+0000 7f990f7fe640 1 -- 172.21.15.107:0/1198967551 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f992409fa60 con 0x55993d75b580 2024-04-25T00:20:52.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.165+0000 7f990f7fe640 1 -- 172.21.15.107:0/1198967551 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7f9920004a10 con 0x55993d75b580 2024-04-25T00:20:52.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.165+0000 7f99397a3740 10 client.16596 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:20:52.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.165+0000 7f99397a3740 10 client.16596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:20:52.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.165+0000 7f99397a3740 20 client.16596 populate_metadata read hostname 'smithi107' 2024-04-25T00:20:52.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.165+0000 7f99397a3740 1 --2- 172.21.15.107:0/1198967551 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55993d7dd890 0x55993d7dfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:52.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.165+0000 7f99397a3740 4 client.16596 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:20:52.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.165+0000 7f99397a3740 1 -- 172.21.15.107:0/1198967551 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55993d6c7f00 con 0x55993d7dd890 2024-04-25T00:20:52.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.165+0000 7f992f7fe640 1 --2- 172.21.15.107:0/1198967551 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55993d7dd890 0x55993d7dfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:52.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.165+0000 7f990f7fe640 10 client.16596 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:52.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.165+0000 7f992f7fe640 1 --2- 172.21.15.107:0/1198967551 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55993d7dd890 0x55993d7dfcb0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e57eb10c2b613e1 server_cookie=36a1bd120e81988a in_seq=0 out_seq=0 2024-04-25T00:20:52.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.166+0000 7f990f7fe640 1 -- 172.21.15.107:0/1198967551 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55993d6c7f00 con 0x55993d7dd890 2024-04-25T00:20:52.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.166+0000 7f990f7fe640 10 client.16596 handle_command_reply: tid=0 2024-04-25T00:20:52.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.166+0000 7f99397a3740 1 client.16596 shutdown 2024-04-25T00:20:52.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.166+0000 7f99397a3740 1 -- 172.21.15.107:0/1198967551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d75b580 msgr2=0x55993d76a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:52.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.166+0000 7f99397a3740 1 --2- 172.21.15.107:0/1198967551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d75b580 0x55993d76a130 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f9920006010 tx=0x7f9920004040 comp rx=0 tx=0).stop 2024-04-25T00:20:52.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.166+0000 7f99397a3740 1 -- 172.21.15.107:0/1198967551 shutdown_connections 2024-04-25T00:20:52.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.166+0000 7f99397a3740 1 --2- 172.21.15.107:0/1198967551 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55993d7dd890 0x55993d7dfcb0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.166+0000 7f99397a3740 1 --2- 172.21.15.107:0/1198967551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55993d766570 0x55993d76abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.166+0000 7f990f7fe640 0 client.16596 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:52.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.166+0000 7f99397a3740 1 --2- 172.21.15.107:0/1198967551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55993d75bea0 0x55993d76a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.166+0000 7f99397a3740 1 --2- 172.21.15.107:0/1198967551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55993d75b580 0x55993d76a130 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.166+0000 7f99397a3740 1 -- 172.21.15.107:0/1198967551 >> 172.21.15.107:0/1198967551 conn(0x55993d7578a0 msgr2=0x55993d7562b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:52.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.166+0000 7f99397a3740 1 -- 172.21.15.107:0/1198967551 shutdown_connections 2024-04-25T00:20:52.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.166+0000 7f99397a3740 1 -- 172.21.15.107:0/1198967551 wait complete. 2024-04-25T00:20:52.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.166+0000 7f99397a3740 20 client.16596 trim_cache size 0 max 16384 2024-04-25T00:20:52.173 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:20:52.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.167+0000 7f99369b7640 1 -- 172.21.15.107:0/3788042117 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f99180a2d00 msgr2=0x7f99180a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:52.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.167+0000 7f99369b7640 1 --2- 172.21.15.107:0/3788042117 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f99180a2d00 0x7f99180a51c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9928005c40 tx=0x7f9928009ac0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.167+0000 7f99369b7640 1 -- 172.21.15.107:0/3788042117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9930100fd0 msgr2=0x7f9930199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:52.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.167+0000 7f99369b7640 1 --2- 172.21.15.107:0/3788042117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9930100fd0 0x7f9930199370 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f992400d950 tx=0x7f992400de10 comp rx=0 tx=0).stop 2024-04-25T00:20:52.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.167+0000 7f99369b7640 1 -- 172.21.15.107:0/3788042117 shutdown_connections 2024-04-25T00:20:52.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.167+0000 7f99369b7640 1 --2- 172.21.15.107:0/3788042117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99301066b0 0x7f99301a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.167+0000 7f99369b7640 1 --2- 172.21.15.107:0/3788042117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9930100fd0 0x7f9930199370 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.167+0000 7f99369b7640 1 --2- 172.21.15.107:0/3788042117 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f99180a2d00 0x7f99180a51c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.167+0000 7f99369b7640 1 --2- 172.21.15.107:0/3788042117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9930100610 0x7f9930198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:52.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.167+0000 7f99369b7640 1 -- 172.21.15.107:0/3788042117 >> 172.21.15.107:0/3788042117 conn(0x7f9930077740 msgr2=0x7f99300ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:52.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.167+0000 7f99369b7640 1 -- 172.21.15.107:0/3788042117 shutdown_connections 2024-04-25T00:20:52.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:52.167+0000 7f99369b7640 1 -- 172.21.15.107:0/3788042117 wait complete. 2024-04-25T00:20:52.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:52 smithi170 ceph-mon[32807]: pgmap v135: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:20:52.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:52 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2981319883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:20:52.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:52 smithi107 ceph-mon[29153]: pgmap v135: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:20:52.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:52 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2981319883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:20:52.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:52 smithi195 ceph-mon[32833]: pgmap v135: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:20:52.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:52 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2981319883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:20:53.178 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:20:53.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:20:53.178 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:20:53.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.292+0000 7fdcebd9b640 1 -- 172.21.15.107:0/1285618249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdce41007b0 msgr2=0x7fdce4100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:53.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.292+0000 7fdcebd9b640 1 --2- 172.21.15.107:0/1285618249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdce41007b0 0x7fdce4100b90 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009900 tx=0x7fdce002a510 comp rx=0 tx=0).stop 2024-04-25T00:20:53.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.292+0000 7fdcebd9b640 1 -- 172.21.15.107:0/1285618249 shutdown_connections 2024-04-25T00:20:53.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.292+0000 7fdcebd9b640 1 --2- 172.21.15.107:0/1285618249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdce4105ad0 0x7fdce4105f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.292+0000 7fdcebd9b640 1 --2- 172.21.15.107:0/1285618249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdce41010d0 0x7fdce4105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.292+0000 7fdcebd9b640 1 --2- 172.21.15.107:0/1285618249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdce41007b0 0x7fdce4100b90 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.292+0000 7fdcebd9b640 1 -- 172.21.15.107:0/1285618249 >> 172.21.15.107:0/1285618249 conn(0x7fdce4077740 msgr2=0x7fdce4077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.292+0000 7fdcebd9b640 1 -- 172.21.15.107:0/1285618249 shutdown_connections 2024-04-25T00:20:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.292+0000 7fdcebd9b640 1 -- 172.21.15.107:0/1285618249 wait complete. 2024-04-25T00:20:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.293+0000 7fdcebd9b640 1 Processor -- start 2024-04-25T00:20:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.293+0000 7fdcebd9b640 1 -- start start 2024-04-25T00:20:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.293+0000 7fdcebd9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdce41007b0 0x7fdce419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.293+0000 7fdcebd9b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdce41010d0 0x7fdce419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.293+0000 7fdcebd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdce4105ad0 0x7fdce41a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.293+0000 7fdcebd9b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdce4118150 con 0x7fdce4105ad0 2024-04-25T00:20:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.293+0000 7fdcebd9b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdce4117fd0 con 0x7fdce41007b0 2024-04-25T00:20:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.293+0000 7fdcebd9b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41182d0 con 0x7fdce41010d0 2024-04-25T00:20:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.293+0000 7fdcea338640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdce4105ad0 0x7fdce41a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.293+0000 7fdcea338640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdce4105ad0 0x7fdce41a2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44140/0 (socket says 172.21.15.107:44140) 2024-04-25T00:20:53.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.293+0000 7fdcea338640 1 -- 172.21.15.107:0/55750123 learned_addr learned my addr 172.21.15.107:0/55750123 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:53.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.293+0000 7fdce9b37640 1 --2- 172.21.15.107:0/55750123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdce41007b0 0x7fdce419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.293+0000 7fdcea338640 1 -- 172.21.15.107:0/55750123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdce41010d0 msgr2=0x7fdce419eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:53.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.293+0000 7fdce9336640 1 --2- 172.21.15.107:0/55750123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdce41010d0 0x7fdce419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.294+0000 7fdcea338640 1 --2- 172.21.15.107:0/55750123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdce41010d0 0x7fdce419eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.294+0000 7fdcea338640 1 -- 172.21.15.107:0/55750123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdce41007b0 msgr2=0x7fdce419e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:53.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.294+0000 7fdcea338640 1 --2- 172.21.15.107:0/55750123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdce41007b0 0x7fdce419e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.294+0000 7fdcea338640 1 -- 172.21.15.107:0/55750123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce41a35f0 con 0x7fdce4105ad0 2024-04-25T00:20:53.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.294+0000 7fdce9b37640 1 --2- 172.21.15.107:0/55750123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdce41007b0 0x7fdce419e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:20:53.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.294+0000 7fdce9336640 1 --2- 172.21.15.107:0/55750123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdce41010d0 0x7fdce419eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:20:53.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.294+0000 7fdcea338640 1 --2- 172.21.15.107:0/55750123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdce4105ad0 0x7fdce41a2ec0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00cce0 tx=0x7fdcdc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:53.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.294+0000 7fdcdaffd640 1 -- 172.21.15.107:0/55750123 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc00f450 con 0x7fdce4105ad0 2024-04-25T00:20:53.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.294+0000 7fdcdaffd640 1 -- 172.21.15.107:0/55750123 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcdc00fad0 con 0x7fdce4105ad0 2024-04-25T00:20:53.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.294+0000 7fdcebd9b640 1 -- 172.21.15.107:0/55750123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce41a38e0 con 0x7fdce4105ad0 2024-04-25T00:20:53.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.294+0000 7fdcdaffd640 1 -- 172.21.15.107:0/55750123 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc018c10 con 0x7fdce4105ad0 2024-04-25T00:20:53.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.294+0000 7fdcebd9b640 1 -- 172.21.15.107:0/55750123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce41ab170 con 0x7fdce4105ad0 2024-04-25T00:20:53.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.296+0000 7fdcdaffd640 1 -- 172.21.15.107:0/55750123 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fdcdc00f5f0 con 0x7fdce4105ad0 2024-04-25T00:20:53.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.297+0000 7fdceeb83740 1 Processor -- start 2024-04-25T00:20:53.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.297+0000 7fdceeb83740 1 -- start start 2024-04-25T00:20:53.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.297+0000 7fdceeb83740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1473980 0x55c7f1471cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.297+0000 7fdceeb83740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1472290 0x55c7f1472710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.297+0000 7fdceeb83740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f147d010 0x55c7f147f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.297+0000 7fdceeb83740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c7f13d62c0 con 0x55c7f1472290 2024-04-25T00:20:53.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.297+0000 7fdceeb83740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c7f137e910 con 0x55c7f1473980 2024-04-25T00:20:53.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.297+0000 7fdceeb83740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c7f13d2280 con 0x55c7f147d010 2024-04-25T00:20:53.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.297+0000 7fdcdaffd640 1 --2- 172.21.15.107:0/55750123 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdcd00a2d80 0x7fdcd00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.297+0000 7fdce9b37640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1472290 0x55c7f1472710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.297+0000 7fdce9b37640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1472290 0x55c7f1472710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44146/0 (socket says 172.21.15.107:44146) 2024-04-25T00:20:53.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.297+0000 7fdce9b37640 1 -- 172.21.15.107:0/1624475314 learned_addr learned my addr 172.21.15.107:0/1624475314 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:53.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.297+0000 7fdcea338640 1 --2- 172.21.15.107:0/55750123 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdcd00a2d80 0x7fdcd00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.297+0000 7fdce9336640 1 --2- 172.21.15.107:0/1624475314 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f147d010 0x55c7f147f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.298+0000 7fdce9336640 1 --2- 172.21.15.107:0/1624475314 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1473980 0x55c7f1471cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.298+0000 7fdce9b37640 1 -- 172.21.15.107:0/1624475314 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f147d010 msgr2=0x55c7f147f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:53.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.298+0000 7fdce9b37640 1 --2- 172.21.15.107:0/1624475314 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f147d010 0x55c7f147f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.298+0000 7fdce9b37640 1 -- 172.21.15.107:0/1624475314 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1473980 msgr2=0x55c7f1471cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:53.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.298+0000 7fdce9b37640 1 --2- 172.21.15.107:0/1624475314 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1473980 0x55c7f1471cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.298+0000 7fdcdaffd640 1 -- 172.21.15.107:0/55750123 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fdcdc0cfeb0 con 0x7fdce4105ad0 2024-04-25T00:20:53.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.298+0000 7fdce9b37640 1 -- 172.21.15.107:0/1624475314 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7f138ffb0 con 0x55c7f1472290 2024-04-25T00:20:53.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.298+0000 7fdce9336640 1 --2- 172.21.15.107:0/1624475314 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1473980 0x55c7f1471cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:20:53.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.298+0000 7fdcea338640 1 --2- 172.21.15.107:0/55750123 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdcd00a2d80 0x7fdcd00a5240 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0984c0 tx=0x7fdcdc0c6bf0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:53.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.298+0000 7fdce9b37640 1 --2- 172.21.15.107:0/1624475314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1472290 0x55c7f1472710 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009900 tx=0x7fdce00316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7124de129275e9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:53.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.298+0000 7fdccffff640 1 -- 172.21.15.107:0/1624475314 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce003f070 con 0x55c7f1472290 2024-04-25T00:20:53.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.298+0000 7fdccffff640 1 -- 172.21.15.107:0/1624475314 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdce00035f0 con 0x55c7f1472290 2024-04-25T00:20:53.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.298+0000 7fdccffff640 1 -- 172.21.15.107:0/1624475314 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce0035070 con 0x55c7f1472290 2024-04-25T00:20:53.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.299+0000 7fdceeb83740 1 -- 172.21.15.107:0/1624475314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1472290 msgr2=0x55c7f1472710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:53.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.299+0000 7fdceeb83740 1 --2- 172.21.15.107:0/1624475314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1472290 0x55c7f1472710 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009900 tx=0x7fdce00316e0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.299+0000 7fdceeb83740 1 -- 172.21.15.107:0/1624475314 shutdown_connections 2024-04-25T00:20:53.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.299+0000 7fdceeb83740 1 --2- 172.21.15.107:0/1624475314 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f147d010 0x55c7f147f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.299+0000 7fdceeb83740 1 --2- 172.21.15.107:0/1624475314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1472290 0x55c7f1472710 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.299+0000 7fdceeb83740 1 --2- 172.21.15.107:0/1624475314 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1473980 0x55c7f1471cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.299+0000 7fdceeb83740 1 -- 172.21.15.107:0/1624475314 >> 172.21.15.107:0/1624475314 conn(0x55c7f143a950 msgr2=0x55c7f146e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:53.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.299+0000 7fdceeb83740 1 -- 172.21.15.107:0/1624475314 shutdown_connections 2024-04-25T00:20:53.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.299+0000 7fdceeb83740 1 -- 172.21.15.107:0/1624475314 wait complete. 2024-04-25T00:20:53.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.299+0000 7fdceeb83740 1 -- start start 2024-04-25T00:20:53.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.299+0000 7fdceeb83740 1 Processor -- start 2024-04-25T00:20:53.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.300+0000 7fdceeb83740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1472290 0x55c7f1479ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.300+0000 7fdceeb83740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1473980 0x55c7f147a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.300+0000 7fdceeb83740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f147d010 0x55c7f147a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.300+0000 7fdceeb83740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c7f147fe40 con 0x55c7f1473980 2024-04-25T00:20:53.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.300+0000 7fdceeb83740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c7f1391e90 con 0x55c7f1472290 2024-04-25T00:20:53.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.300+0000 7fdceeb83740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c7f147ffc0 con 0x55c7f147d010 2024-04-25T00:20:53.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.300+0000 7fdceeb83740 10 client.0 _ll_register_callbacks cb 0x55c7f13cd970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:53.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.300+0000 7fdce9b37640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1473980 0x55c7f147a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.300+0000 7fdce9b37640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1473980 0x55c7f147a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44156/0 (socket says 172.21.15.107:44156) 2024-04-25T00:20:53.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.300+0000 7fdce9b37640 1 -- 172.21.15.107:0/499074933 learned_addr learned my addr 172.21.15.107:0/499074933 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:53.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.300+0000 7fdce9b37640 1 -- 172.21.15.107:0/499074933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f147d010 msgr2=0x55c7f147a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:53.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.300+0000 7fdce9336640 1 --2- 172.21.15.107:0/499074933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f147d010 0x55c7f147a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.300+0000 7fdce9b37640 1 --2- 172.21.15.107:0/499074933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f147d010 0x55c7f147a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.300+0000 7fdce9b37640 1 -- 172.21.15.107:0/499074933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1472290 msgr2=0x55c7f1479ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:53.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.300+0000 7fdce9b37640 1 --2- 172.21.15.107:0/499074933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1472290 0x55c7f1479ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.300+0000 7fdce9b37640 1 -- 172.21.15.107:0/499074933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7f11b27e0 con 0x55c7f1473980 2024-04-25T00:20:53.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.300+0000 7fdce9336640 1 --2- 172.21.15.107:0/499074933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f147d010 0x55c7f147a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:20:53.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.300+0000 7fdce9b37640 1 --2- 172.21.15.107:0/499074933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1473980 0x55c7f147a410 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fdce0000c00 tx=0x7fdce0003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2094d05d11a4769a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:53.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.300+0000 7fdccd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:20:53.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.301+0000 7fdccd7fa640 1 -- 172.21.15.107:0/499074933 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce0035070 con 0x55c7f1473980 2024-04-25T00:20:53.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.301+0000 7fdccd7fa640 1 -- 172.21.15.107:0/499074933 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fdce0004460 con 0x55c7f1473980 2024-04-25T00:20:53.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.301+0000 7fdceeb83740 1 -- 172.21.15.107:0/499074933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdce00035f0 con 0x55c7f1473980 2024-04-25T00:20:53.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.301+0000 7fdccd7fa640 1 -- 172.21.15.107:0/499074933 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdce0055050 con 0x55c7f1473980 2024-04-25T00:20:53.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.301+0000 7fdceeb83740 10 client.16608 fetch_fsmap learned FSMap version 28 2024-04-25T00:20:53.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.301+0000 7fdceeb83740 1 -- 172.21.15.107:0/499074933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c7f1212f70 con 0x55c7f1473980 2024-04-25T00:20:53.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.301+0000 7fdccd7fa640 1 -- 172.21.15.107:0/499074933 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdce00035f0 con 0x55c7f1473980 2024-04-25T00:20:53.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.301+0000 7fdccd7fa640 1 -- 172.21.15.107:0/499074933 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fdce0043790 con 0x55c7f1473980 2024-04-25T00:20:53.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.301+0000 7fdceeb83740 10 client.16608 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:20:53.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.301+0000 7fdceeb83740 10 client.16608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:20:53.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.301+0000 7fdceeb83740 20 client.16608 populate_metadata read hostname 'smithi107' 2024-04-25T00:20:53.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.301+0000 7fdceeb83740 1 --2- 172.21.15.107:0/499074933 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c7f148cf20 0x55c7f148f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.301+0000 7fdceeb83740 4 client.16608 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:20:53.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.301+0000 7fdceeb83740 1 -- 172.21.15.107:0/499074933 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7f13aa5a0 con 0x55c7f148cf20 2024-04-25T00:20:53.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.302+0000 7fdce9336640 1 --2- 172.21.15.107:0/499074933 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c7f148cf20 0x55c7f148f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.302+0000 7fdccd7fa640 10 client.16608 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:53.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.302+0000 7fdce9336640 1 --2- 172.21.15.107:0/499074933 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c7f148cf20 0x55c7f148f310 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1423f31ab7cb6d1e server_cookie=6bfebbe891cc01ba in_seq=0 out_seq=0 2024-04-25T00:20:53.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.303+0000 7fdccd7fa640 1 -- 172.21.15.107:0/499074933 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c7f13aa5a0 con 0x55c7f148cf20 2024-04-25T00:20:53.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.303+0000 7fdccd7fa640 10 client.16608 handle_command_reply: tid=0 2024-04-25T00:20:53.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.303+0000 7fdceeb83740 1 client.16608 shutdown 2024-04-25T00:20:53.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.303+0000 7fdceeb83740 1 -- 172.21.15.107:0/499074933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1473980 msgr2=0x55c7f147a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:53.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.303+0000 7fdceeb83740 1 --2- 172.21.15.107:0/499074933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1473980 0x55c7f147a410 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fdce0000c00 tx=0x7fdce0003c90 comp rx=0 tx=0).stop 2024-04-25T00:20:53.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.303+0000 7fdceeb83740 1 -- 172.21.15.107:0/499074933 shutdown_connections 2024-04-25T00:20:53.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.303+0000 7fdceeb83740 1 --2- 172.21.15.107:0/499074933 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c7f148cf20 0x55c7f148f310 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.303+0000 7fdceeb83740 1 --2- 172.21.15.107:0/499074933 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f147d010 0x55c7f147a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.303+0000 7fdccd7fa640 0 client.16608 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:53.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.303+0000 7fdceeb83740 1 --2- 172.21.15.107:0/499074933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1473980 0x55c7f147a410 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.303+0000 7fdceeb83740 1 --2- 172.21.15.107:0/499074933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1472290 0x55c7f1479ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.303+0000 7fdceeb83740 1 -- 172.21.15.107:0/499074933 >> 172.21.15.107:0/499074933 conn(0x55c7f143a950 msgr2=0x55c7f14745f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:53.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.303+0000 7fdceeb83740 1 -- 172.21.15.107:0/499074933 shutdown_connections 2024-04-25T00:20:53.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.303+0000 7fdceeb83740 1 -- 172.21.15.107:0/499074933 wait complete. 2024-04-25T00:20:53.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.303+0000 7fdceeb83740 20 client.16608 trim_cache size 0 max 16384 2024-04-25T00:20:53.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.320+0000 7fdceeb83740 1 Processor -- start 2024-04-25T00:20:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.320+0000 7fdceeb83740 1 -- start start 2024-04-25T00:20:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.321+0000 7fdceeb83740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1472290 0x55c7f1488da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.321+0000 7fdceeb83740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1473980 0x55c7f14948b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.321+0000 7fdceeb83740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f147d010 0x55c7f1494df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.321+0000 7fdceeb83740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41b7970 con 0x55c7f1473980 2024-04-25T00:20:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.321+0000 7fdceeb83740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41b5180 con 0x55c7f147d010 2024-04-25T00:20:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.321+0000 7fdceeb83740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41b2fd0 con 0x55c7f1472290 2024-04-25T00:20:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.321+0000 7fdce9b37640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1473980 0x55c7f14948b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.321+0000 7fdce9b37640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1473980 0x55c7f14948b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44160/0 (socket says 172.21.15.107:44160) 2024-04-25T00:20:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.321+0000 7fdce9b37640 1 -- 172.21.15.107:0/4279669488 learned_addr learned my addr 172.21.15.107:0/4279669488 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.321+0000 7fdce9336640 1 --2- 172.21.15.107:0/4279669488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1472290 0x55c7f1488da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.321+0000 7fdce9b37640 1 -- 172.21.15.107:0/4279669488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1472290 msgr2=0x55c7f1488da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.321+0000 7fdce9b37640 1 --2- 172.21.15.107:0/4279669488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1472290 0x55c7f1488da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.321+0000 7fdce9b37640 1 -- 172.21.15.107:0/4279669488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f147d010 msgr2=0x55c7f1494df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.321+0000 7fdce9b37640 1 --2- 172.21.15.107:0/4279669488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f147d010 0x55c7f1494df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.321+0000 7fdce9b37640 1 -- 172.21.15.107:0/4279669488 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7f10aa170 con 0x55c7f1473980 2024-04-25T00:20:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.321+0000 7fdce9336640 1 --2- 172.21.15.107:0/4279669488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1472290 0x55c7f1488da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:20:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.321+0000 7fdce9b37640 1 --2- 172.21.15.107:0/4279669488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1473980 0x55c7f14948b0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fdce0043bb0 tx=0x7fdce0003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb03dbf9a8b88fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.321+0000 7fdccf7fe640 1 -- 172.21.15.107:0/4279669488 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce0035040 con 0x55c7f1473980 2024-04-25T00:20:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.321+0000 7fdccf7fe640 1 -- 172.21.15.107:0/4279669488 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdce004c070 con 0x55c7f1473980 2024-04-25T00:20:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.321+0000 7fdccf7fe640 1 -- 172.21.15.107:0/4279669488 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce003f040 con 0x55c7f1473980 2024-04-25T00:20:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.322+0000 7fdceeb83740 1 -- 172.21.15.107:0/4279669488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1473980 msgr2=0x55c7f14948b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.322+0000 7fdceeb83740 1 --2- 172.21.15.107:0/4279669488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1473980 0x55c7f14948b0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fdce0043bb0 tx=0x7fdce0003ab0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.322+0000 7fdceeb83740 1 -- 172.21.15.107:0/4279669488 shutdown_connections 2024-04-25T00:20:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.322+0000 7fdceeb83740 1 --2- 172.21.15.107:0/4279669488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f147d010 0x55c7f1494df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.322+0000 7fdceeb83740 1 --2- 172.21.15.107:0/4279669488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1473980 0x55c7f14948b0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.322+0000 7fdceeb83740 1 --2- 172.21.15.107:0/4279669488 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1472290 0x55c7f1488da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.322+0000 7fdceeb83740 1 -- 172.21.15.107:0/4279669488 >> 172.21.15.107:0/4279669488 conn(0x55c7f143a950 msgr2=0x55c7f146d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:53.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.322+0000 7fdceeb83740 1 -- 172.21.15.107:0/4279669488 shutdown_connections 2024-04-25T00:20:53.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.322+0000 7fdceeb83740 1 -- 172.21.15.107:0/4279669488 wait complete. 2024-04-25T00:20:53.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.322+0000 7fdceeb83740 1 -- start start 2024-04-25T00:20:53.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.322+0000 7fdceeb83740 1 Processor -- start 2024-04-25T00:20:53.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.323+0000 7fdceeb83740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1472290 0x55c7f14852b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.323+0000 7fdceeb83740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1473980 0x55c7f14857f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.323+0000 7fdceeb83740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f147d010 0x55c7f1482280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.323+0000 7fdceeb83740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41b3640 con 0x55c7f1472290 2024-04-25T00:20:53.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.323+0000 7fdceeb83740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41b4ab0 con 0x55c7f147d010 2024-04-25T00:20:53.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.323+0000 7fdceeb83740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41b3d10 con 0x55c7f1473980 2024-04-25T00:20:53.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.323+0000 7fdceeb83740 10 client.0 _ll_register_callbacks cb 0x7fdcd400c750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:53.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.323+0000 7fdce9336640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1472290 0x55c7f14852b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.323+0000 7fdce9336640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1472290 0x55c7f14852b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44170/0 (socket says 172.21.15.107:44170) 2024-04-25T00:20:53.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.323+0000 7fdce9336640 1 -- 172.21.15.107:0/640966071 learned_addr learned my addr 172.21.15.107:0/640966071 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:53.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.323+0000 7fdce9b37640 1 --2- 172.21.15.107:0/640966071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1473980 0x55c7f14857f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.323+0000 7fdce9336640 1 -- 172.21.15.107:0/640966071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1473980 msgr2=0x55c7f14857f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:53.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.323+0000 7fdce9336640 1 --2- 172.21.15.107:0/640966071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1473980 0x55c7f14857f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.323+0000 7fdce9336640 1 -- 172.21.15.107:0/640966071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f147d010 msgr2=0x55c7f1482280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:53.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.323+0000 7fdce9336640 1 --2- 172.21.15.107:0/640966071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f147d010 0x55c7f1482280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.323+0000 7fdce9336640 1 -- 172.21.15.107:0/640966071 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7f14827c0 con 0x55c7f1472290 2024-04-25T00:20:53.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.323+0000 7fdce9336640 1 --2- 172.21.15.107:0/640966071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1472290 0x55c7f14852b0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x55c7f1480110 tx=0x7fdcd4017810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522949c360afb7e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:53.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.323+0000 7fdccd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:20:53.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.323+0000 7fdccd7fa640 1 -- 172.21.15.107:0/640966071 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd400f040 con 0x55c7f1472290 2024-04-25T00:20:53.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.323+0000 7fdccd7fa640 1 -- 172.21.15.107:0/640966071 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fdcd40255c0 con 0x55c7f1472290 2024-04-25T00:20:53.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.323+0000 7fdceeb83740 1 -- 172.21.15.107:0/640966071 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdce004c070 con 0x55c7f1472290 2024-04-25T00:20:53.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.323+0000 7fdccd7fa640 1 -- 172.21.15.107:0/640966071 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdcd40257c0 con 0x55c7f1472290 2024-04-25T00:20:53.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.324+0000 7fdceeb83740 10 client.16614 fetch_fsmap learned FSMap version 28 2024-04-25T00:20:53.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.324+0000 7fdceeb83740 1 -- 172.21.15.107:0/640966071 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c7f14f38e0 con 0x55c7f1472290 2024-04-25T00:20:53.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.324+0000 7fdccd7fa640 1 -- 172.21.15.107:0/640966071 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdce004c070 con 0x55c7f1472290 2024-04-25T00:20:53.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.324+0000 7fdccd7fa640 1 -- 172.21.15.107:0/640966071 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fdcd4015040 con 0x55c7f1472290 2024-04-25T00:20:53.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.324+0000 7fdceeb83740 10 client.16614 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:20:53.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.324+0000 7fdceeb83740 10 client.16614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:20:53.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.324+0000 7fdceeb83740 20 client.16614 populate_metadata read hostname 'smithi107' 2024-04-25T00:20:53.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.324+0000 7fdceeb83740 1 --2- 172.21.15.107:0/640966071 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c7f14f40a0 0x55c7f14f64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.324+0000 7fdceeb83740 4 client.16614 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:20:53.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.324+0000 7fdceeb83740 1 -- 172.21.15.107:0/640966071 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c7f13e3800 con 0x55c7f14f40a0 2024-04-25T00:20:53.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.324+0000 7fdce9b37640 1 --2- 172.21.15.107:0/640966071 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c7f14f40a0 0x55c7f14f64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.325+0000 7fdce9b37640 1 --2- 172.21.15.107:0/640966071 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c7f14f40a0 0x55c7f14f64c0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf91e9c1aacefbd6 server_cookie=e398a73567b42fe9 in_seq=0 out_seq=0 2024-04-25T00:20:53.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.325+0000 7fdccd7fa640 10 client.16614 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:53.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.326+0000 7fdccd7fa640 1 -- 172.21.15.107:0/640966071 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c7f13e3800 con 0x55c7f14f40a0 2024-04-25T00:20:53.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.326+0000 7fdccd7fa640 10 client.16614 handle_command_reply: tid=0 2024-04-25T00:20:53.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.326+0000 7fdceeb83740 1 client.16614 shutdown 2024-04-25T00:20:53.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.326+0000 7fdceeb83740 1 -- 172.21.15.107:0/640966071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1472290 msgr2=0x55c7f14852b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:53.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.326+0000 7fdceeb83740 1 --2- 172.21.15.107:0/640966071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1472290 0x55c7f14852b0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x55c7f1480110 tx=0x7fdcd4017810 comp rx=0 tx=0).stop 2024-04-25T00:20:53.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.326+0000 7fdceeb83740 1 -- 172.21.15.107:0/640966071 shutdown_connections 2024-04-25T00:20:53.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.326+0000 7fdceeb83740 1 --2- 172.21.15.107:0/640966071 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c7f14f40a0 0x55c7f14f64c0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.326+0000 7fdceeb83740 1 --2- 172.21.15.107:0/640966071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f147d010 0x55c7f1482280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.326+0000 7fdceeb83740 1 --2- 172.21.15.107:0/640966071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1473980 0x55c7f14857f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.326+0000 7fdceeb83740 1 --2- 172.21.15.107:0/640966071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1472290 0x55c7f14852b0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.326+0000 7fdccd7fa640 0 client.16614 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:53.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.326+0000 7fdceeb83740 1 -- 172.21.15.107:0/640966071 >> 172.21.15.107:0/640966071 conn(0x55c7f143a950 msgr2=0x55c7f1479e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:53.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.326+0000 7fdceeb83740 1 -- 172.21.15.107:0/640966071 shutdown_connections 2024-04-25T00:20:53.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.326+0000 7fdceeb83740 1 -- 172.21.15.107:0/640966071 wait complete. 2024-04-25T00:20:53.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.326+0000 7fdceeb83740 20 client.16614 trim_cache size 0 max 16384 2024-04-25T00:20:53.333 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:20:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-04-25T00:20:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "78b9f7fc-100b-4003-b334-ea85d59807be", 2024-04-25T00:20:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-04-25T00:20:53.333 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:20:53.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.327+0000 7fdcebd9b640 1 -- 172.21.15.107:0/55750123 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdcd00a2d80 msgr2=0x7fdcd00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:53.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.327+0000 7fdcebd9b640 1 --2- 172.21.15.107:0/55750123 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdcd00a2d80 0x7fdcd00a5240 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0984c0 tx=0x7fdcdc0c6bf0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.327+0000 7fdcebd9b640 1 -- 172.21.15.107:0/55750123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdce4105ad0 msgr2=0x7fdce41a2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:53.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.327+0000 7fdcebd9b640 1 --2- 172.21.15.107:0/55750123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdce4105ad0 0x7fdce41a2ec0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00cce0 tx=0x7fdcdc00a620 comp rx=0 tx=0).stop 2024-04-25T00:20:53.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.327+0000 7fdcebd9b640 1 -- 172.21.15.107:0/55750123 shutdown_connections 2024-04-25T00:20:53.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.327+0000 7fdcebd9b640 1 --2- 172.21.15.107:0/55750123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdce4105ad0 0x7fdce41a2ec0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.327+0000 7fdcebd9b640 1 --2- 172.21.15.107:0/55750123 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdce41010d0 0x7fdce419eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.327+0000 7fdcebd9b640 1 --2- 172.21.15.107:0/55750123 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdcd00a2d80 0x7fdcd00a5240 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.327+0000 7fdcebd9b640 1 --2- 172.21.15.107:0/55750123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdce41007b0 0x7fdce419e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.327+0000 7fdcebd9b640 1 -- 172.21.15.107:0/55750123 >> 172.21.15.107:0/55750123 conn(0x7fdce4077740 msgr2=0x7fdce4103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:53.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.327+0000 7fdcebd9b640 1 -- 172.21.15.107:0/55750123 shutdown_connections 2024-04-25T00:20:53.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.327+0000 7fdcebd9b640 1 -- 172.21.15.107:0/55750123 wait complete. 2024-04-25T00:20:53.338 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:20:53.338 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:20:53.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.438+0000 7fe036bb6640 1 -- 172.21.15.107:0/3737005887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe030100620 msgr2=0x7fe030100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:53.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.438+0000 7fe036bb6640 1 --2- 172.21.15.107:0/3737005887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe030100620 0x7fe030100aa0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe01c009930 tx=0x7fe01c02a420 comp rx=0 tx=0).stop 2024-04-25T00:20:53.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.438+0000 7fe036bb6640 1 -- 172.21.15.107:0/3737005887 shutdown_connections 2024-04-25T00:20:53.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.438+0000 7fe036bb6640 1 --2- 172.21.15.107:0/3737005887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe030100fe0 0x7fe03010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.438+0000 7fe036bb6640 1 --2- 172.21.15.107:0/3737005887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe030100620 0x7fe030100aa0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.438+0000 7fe036bb6640 1 --2- 172.21.15.107:0/3737005887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe030106630 0x7fe030106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.438+0000 7fe036bb6640 1 -- 172.21.15.107:0/3737005887 >> 172.21.15.107:0/3737005887 conn(0x7fe0300776a0 msgr2=0x7fe030077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:53.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.438+0000 7fe036bb6640 1 -- 172.21.15.107:0/3737005887 shutdown_connections 2024-04-25T00:20:53.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.438+0000 7fe036bb6640 1 -- 172.21.15.107:0/3737005887 wait complete. 2024-04-25T00:20:53.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.439+0000 7fe036bb6640 1 Processor -- start 2024-04-25T00:20:53.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.439+0000 7fe036bb6640 1 -- start start 2024-04-25T00:20:53.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.439+0000 7fe036bb6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe030100620 0x7fe030198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.439+0000 7fe036bb6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe030100fe0 0x7fe030199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.439+0000 7fe036bb6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe030106630 0x7fe0301a3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.439+0000 7fe036bb6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe030110a40 con 0x7fe030100620 2024-04-25T00:20:53.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.439+0000 7fe036bb6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe0301108c0 con 0x7fe030100fe0 2024-04-25T00:20:53.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.439+0000 7fe036bb6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe030110bc0 con 0x7fe030106630 2024-04-25T00:20:53.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.439+0000 7fe034952640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe030100620 0x7fe030198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.439+0000 7fe034952640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe030100620 0x7fe030198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44184/0 (socket says 172.21.15.107:44184) 2024-04-25T00:20:53.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.439+0000 7fe034952640 1 -- 172.21.15.107:0/1530933410 learned_addr learned my addr 172.21.15.107:0/1530933410 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:53.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.439+0000 7fe035153640 1 --2- 172.21.15.107:0/1530933410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe030106630 0x7fe0301a3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.439+0000 7fe034952640 1 -- 172.21.15.107:0/1530933410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe030106630 msgr2=0x7fe0301a3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:53.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.439+0000 7fe034952640 1 --2- 172.21.15.107:0/1530933410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe030106630 0x7fe0301a3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.439+0000 7fe034952640 1 -- 172.21.15.107:0/1530933410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe030100fe0 msgr2=0x7fe030199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:53.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.439+0000 7fe034952640 1 --2- 172.21.15.107:0/1530933410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe030100fe0 0x7fe030199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.439+0000 7fe034952640 1 -- 172.21.15.107:0/1530933410 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0301a43e0 con 0x7fe030100620 2024-04-25T00:20:53.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.439+0000 7fe034952640 1 --2- 172.21.15.107:0/1530933410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe030100620 0x7fe030198e20 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe02400d770 tx=0x7fe02400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:53.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.439+0000 7fe02dffb640 1 -- 172.21.15.107:0/1530933410 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe024017070 con 0x7fe030100620 2024-04-25T00:20:53.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.440+0000 7fe02dffb640 1 -- 172.21.15.107:0/1530933410 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe024005500 con 0x7fe030100620 2024-04-25T00:20:53.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.440+0000 7fe02dffb640 1 -- 172.21.15.107:0/1530933410 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe024004e00 con 0x7fe030100620 2024-04-25T00:20:53.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.440+0000 7fe036bb6640 1 -- 172.21.15.107:0/1530933410 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0301a45b0 con 0x7fe030100620 2024-04-25T00:20:53.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.440+0000 7fe036bb6640 1 -- 172.21.15.107:0/1530933410 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0301a4a70 con 0x7fe030100620 2024-04-25T00:20:53.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.442+0000 7fe02dffb640 1 -- 172.21.15.107:0/1530933410 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe024004fa0 con 0x7fe030100620 2024-04-25T00:20:53.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.442+0000 7fe03999c740 1 Processor -- start 2024-04-25T00:20:53.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.442+0000 7fe02dffb640 1 --2- 172.21.15.107:0/1530933410 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe0200a2d80 0x7fe0200a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.442+0000 7fe03999c740 1 -- start start 2024-04-25T00:20:53.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.442+0000 7fe035153640 1 --2- 172.21.15.107:0/1530933410 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe0200a2d80 0x7fe0200a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.442+0000 7fe02dffb640 1 -- 172.21.15.107:0/1530933410 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fe0240cf210 con 0x7fe030100620 2024-04-25T00:20:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.442+0000 7fe03999c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561469380300 0x5614693806e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.443+0000 7fe03999c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469380c20 0x561469393cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.443+0000 7fe03999c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561469394200 0x5614693965f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.443+0000 7fe03999c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5614692e56b0 con 0x561469380c20 2024-04-25T00:20:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.443+0000 7fe03999c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56146928d880 con 0x561469394200 2024-04-25T00:20:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.443+0000 7fe03999c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5614692e1670 con 0x561469380300 2024-04-25T00:20:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.443+0000 7fe02ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469380c20 0x561469393cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.443+0000 7fe02ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469380c20 0x561469393cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44192/0 (socket says 172.21.15.107:44192) 2024-04-25T00:20:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.443+0000 7fe02ffff640 1 -- 172.21.15.107:0/1727422020 learned_addr learned my addr 172.21.15.107:0/1727422020 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.443+0000 7fe034952640 1 --2- 172.21.15.107:0/1727422020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561469380300 0x5614693806e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.443+0000 7fe035153640 1 --2- 172.21.15.107:0/1727422020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561469394200 0x5614693965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.443+0000 7fe02ffff640 1 -- 172.21.15.107:0/1727422020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561469380300 msgr2=0x5614693806e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.443+0000 7fe02ffff640 1 --2- 172.21.15.107:0/1727422020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561469380300 0x5614693806e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.443+0000 7fe02ffff640 1 -- 172.21.15.107:0/1727422020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561469394200 msgr2=0x5614693965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.443+0000 7fe02ffff640 1 --2- 172.21.15.107:0/1727422020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561469394200 0x5614693965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.443+0000 7fe02ffff640 1 -- 172.21.15.107:0/1727422020 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614690c37d0 con 0x561469380c20 2024-04-25T00:20:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.443+0000 7fe034952640 1 --2- 172.21.15.107:0/1727422020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561469380300 0x5614693806e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:20:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.443+0000 7fe035153640 1 --2- 172.21.15.107:0/1727422020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561469394200 0x5614693965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:20:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.443+0000 7fe035153640 1 --2- 172.21.15.107:0/1530933410 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe0200a2d80 0x7fe0200a5240 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe02800ad70 tx=0x7fe02800e720 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.443+0000 7fe02ffff640 1 --2- 172.21.15.107:0/1727422020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469380c20 0x561469393cc0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe01c031530 tx=0x7fe01c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cfba01ce3e48d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.443+0000 7fe01affd640 1 -- 172.21.15.107:0/1727422020 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c040070 con 0x561469380c20 2024-04-25T00:20:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.443+0000 7fe01affd640 1 -- 172.21.15.107:0/1727422020 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe01c004040 con 0x561469380c20 2024-04-25T00:20:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.443+0000 7fe01affd640 1 -- 172.21.15.107:0/1727422020 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe01c035040 con 0x561469380c20 2024-04-25T00:20:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.444+0000 7fe03999c740 1 -- 172.21.15.107:0/1727422020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469380c20 msgr2=0x561469393cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.444+0000 7fe03999c740 1 --2- 172.21.15.107:0/1727422020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469380c20 0x561469393cc0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe01c031530 tx=0x7fe01c005d50 comp rx=0 tx=0).stop 2024-04-25T00:20:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.444+0000 7fe03999c740 1 -- 172.21.15.107:0/1727422020 shutdown_connections 2024-04-25T00:20:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.444+0000 7fe03999c740 1 --2- 172.21.15.107:0/1727422020 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561469394200 0x5614693965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.444+0000 7fe03999c740 1 --2- 172.21.15.107:0/1727422020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469380c20 0x561469393cc0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.444+0000 7fe03999c740 1 --2- 172.21.15.107:0/1727422020 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561469380300 0x5614693806e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.444+0000 7fe03999c740 1 -- 172.21.15.107:0/1727422020 >> 172.21.15.107:0/1727422020 conn(0x56146937c950 msgr2=0x56146937cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.444+0000 7fe03999c740 1 -- 172.21.15.107:0/1727422020 shutdown_connections 2024-04-25T00:20:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.444+0000 7fe03999c740 1 -- 172.21.15.107:0/1727422020 wait complete. 2024-04-25T00:20:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.444+0000 7fe03999c740 1 -- start start 2024-04-25T00:20:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.444+0000 7fe03999c740 1 Processor -- start 2024-04-25T00:20:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.445+0000 7fe03999c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561469380300 0x561469391170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.445+0000 7fe03999c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561469380c20 0x5614693916b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.445+0000 7fe03999c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469394200 0x561469391bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.445+0000 7fe03999c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561469396bc0 con 0x561469394200 2024-04-25T00:20:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.445+0000 7fe03999c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5614692a0f50 con 0x561469380c20 2024-04-25T00:20:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.445+0000 7fe03999c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561469396d40 con 0x561469380300 2024-04-25T00:20:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.445+0000 7fe03999c740 10 client.0 _ll_register_callbacks cb 0x5614692dc000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:53.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.445+0000 7fe035153640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469394200 0x561469391bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.445+0000 7fe035153640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469394200 0x561469391bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44206/0 (socket says 172.21.15.107:44206) 2024-04-25T00:20:53.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.445+0000 7fe035153640 1 -- 172.21.15.107:0/3077004580 learned_addr learned my addr 172.21.15.107:0/3077004580 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:53.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.445+0000 7fe02ffff640 1 --2- 172.21.15.107:0/3077004580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561469380c20 0x5614693916b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.445+0000 7fe034952640 1 --2- 172.21.15.107:0/3077004580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561469380300 0x561469391170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.445+0000 7fe035153640 1 -- 172.21.15.107:0/3077004580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561469380300 msgr2=0x561469391170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:53.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.445+0000 7fe035153640 1 --2- 172.21.15.107:0/3077004580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561469380300 0x561469391170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.445+0000 7fe035153640 1 -- 172.21.15.107:0/3077004580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561469380c20 msgr2=0x5614693916b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:53.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.445+0000 7fe035153640 1 --2- 172.21.15.107:0/3077004580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561469380c20 0x5614693916b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.445+0000 7fe035153640 1 -- 172.21.15.107:0/3077004580 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561468ff93f0 con 0x561469394200 2024-04-25T00:20:53.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.445+0000 7fe035153640 1 --2- 172.21.15.107:0/3077004580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469394200 0x561469391bf0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fe028012e70 tx=0x7fe028004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37aed95311c10768 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:53.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.445+0000 7fe00ffff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:20:53.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.445+0000 7fe00ffff640 1 -- 172.21.15.107:0/3077004580 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe028004d20 con 0x561469394200 2024-04-25T00:20:53.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.446+0000 7fe00ffff640 1 -- 172.21.15.107:0/3077004580 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fe028005020 con 0x561469394200 2024-04-25T00:20:53.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.446+0000 7fe03999c740 1 -- 172.21.15.107:0/3077004580 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe01c004040 con 0x561469394200 2024-04-25T00:20:53.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.446+0000 7fe00ffff640 1 -- 172.21.15.107:0/3077004580 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe02802f050 con 0x561469394200 2024-04-25T00:20:53.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.446+0000 7fe03999c740 10 client.16626 fetch_fsmap learned FSMap version 28 2024-04-25T00:20:53.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.446+0000 7fe03999c740 1 -- 172.21.15.107:0/3077004580 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561468fb90f0 con 0x561469394200 2024-04-25T00:20:53.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.446+0000 7fe00ffff640 1 -- 172.21.15.107:0/3077004580 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe01c004040 con 0x561469394200 2024-04-25T00:20:53.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.446+0000 7fe00ffff640 1 -- 172.21.15.107:0/3077004580 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fe02801e670 con 0x561469394200 2024-04-25T00:20:53.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.446+0000 7fe03999c740 10 client.16626 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:20:53.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.446+0000 7fe03999c740 10 client.16626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:20:53.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.446+0000 7fe03999c740 20 client.16626 populate_metadata read hostname 'smithi107' 2024-04-25T00:20:53.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.446+0000 7fe03999c740 1 --2- 172.21.15.107:0/3077004580 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5614693a4170 0x5614693a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.446+0000 7fe03999c740 4 client.16626 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:20:53.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.446+0000 7fe03999c740 1 -- 172.21.15.107:0/3077004580 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614692f2bf0 con 0x5614693a4170 2024-04-25T00:20:53.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.446+0000 7fe034952640 1 --2- 172.21.15.107:0/3077004580 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5614693a4170 0x5614693a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.447+0000 7fe034952640 1 --2- 172.21.15.107:0/3077004580 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5614693a4170 0x5614693a65b0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd62aa81bbc9a5a0 server_cookie=e38ba392bb14fb63 in_seq=0 out_seq=0 2024-04-25T00:20:53.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.447+0000 7fe00ffff640 10 client.16626 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:53.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.447+0000 7fe00ffff640 1 -- 172.21.15.107:0/3077004580 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5614692f2bf0 con 0x5614693a4170 2024-04-25T00:20:53.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.447+0000 7fe00ffff640 10 client.16626 handle_command_reply: tid=0 2024-04-25T00:20:53.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.447+0000 7fe03999c740 1 client.16626 shutdown 2024-04-25T00:20:53.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.447+0000 7fe03999c740 1 -- 172.21.15.107:0/3077004580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469394200 msgr2=0x561469391bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:53.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.447+0000 7fe03999c740 1 --2- 172.21.15.107:0/3077004580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469394200 0x561469391bf0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fe028012e70 tx=0x7fe028004ab0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.447+0000 7fe03999c740 1 -- 172.21.15.107:0/3077004580 shutdown_connections 2024-04-25T00:20:53.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.447+0000 7fe03999c740 1 --2- 172.21.15.107:0/3077004580 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5614693a4170 0x5614693a65b0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.447+0000 7fe03999c740 1 --2- 172.21.15.107:0/3077004580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469394200 0x561469391bf0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.447+0000 7fe03999c740 1 --2- 172.21.15.107:0/3077004580 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561469380c20 0x5614693916b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.447+0000 7fe00ffff640 0 client.16626 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:53.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.447+0000 7fe03999c740 1 --2- 172.21.15.107:0/3077004580 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561469380300 0x561469391170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.447+0000 7fe03999c740 1 -- 172.21.15.107:0/3077004580 >> 172.21.15.107:0/3077004580 conn(0x56146937c950 msgr2=0x5614693827c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:53.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.448+0000 7fe03999c740 1 -- 172.21.15.107:0/3077004580 shutdown_connections 2024-04-25T00:20:53.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.448+0000 7fe03999c740 1 -- 172.21.15.107:0/3077004580 wait complete. 2024-04-25T00:20:53.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.448+0000 7fe03999c740 20 client.16626 trim_cache size 0 max 16384 2024-04-25T00:20:53.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.465+0000 7fe03999c740 1 Processor -- start 2024-04-25T00:20:53.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.465+0000 7fe03999c740 1 -- start start 2024-04-25T00:20:53.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.465+0000 7fe03999c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469380300 0x56146939fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.465+0000 7fe03999c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561469380c20 0x561469391a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.465+0000 7fe03999c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561469394200 0x5614693abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.465+0000 7fe03999c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe0301af0f0 con 0x561469380300 2024-04-25T00:20:53.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.465+0000 7fe03999c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe0301ac900 con 0x561469394200 2024-04-25T00:20:53.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.465+0000 7fe03999c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe0301aa750 con 0x561469380c20 2024-04-25T00:20:53.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.465+0000 7fe034952640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469380300 0x56146939fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.465+0000 7fe034952640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469380300 0x56146939fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44208/0 (socket says 172.21.15.107:44208) 2024-04-25T00:20:53.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.465+0000 7fe034952640 1 -- 172.21.15.107:0/1620994423 learned_addr learned my addr 172.21.15.107:0/1620994423 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.465+0000 7fe02ffff640 1 --2- 172.21.15.107:0/1620994423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561469380c20 0x561469391a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.465+0000 7fe035153640 1 --2- 172.21.15.107:0/1620994423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561469394200 0x5614693abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.465+0000 7fe034952640 1 -- 172.21.15.107:0/1620994423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561469380c20 msgr2=0x561469391a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.465+0000 7fe034952640 1 --2- 172.21.15.107:0/1620994423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561469380c20 0x561469391a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.465+0000 7fe034952640 1 -- 172.21.15.107:0/1620994423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561469394200 msgr2=0x5614693abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.465+0000 7fe034952640 1 --2- 172.21.15.107:0/1620994423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561469394200 0x5614693abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.465+0000 7fe034952640 1 -- 172.21.15.107:0/1620994423 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614693965f0 con 0x561469380300 2024-04-25T00:20:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.465+0000 7fe035153640 1 --2- 172.21.15.107:0/1620994423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561469394200 0x5614693abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:20:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.465+0000 7fe034952640 1 --2- 172.21.15.107:0/1620994423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469380300 0x56146939fac0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe0240cc6a0 tx=0x7fe024009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f4c3dd9afd4866 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.466+0000 7fe01a7fc640 1 -- 172.21.15.107:0/1620994423 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe024084070 con 0x561469380300 2024-04-25T00:20:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.466+0000 7fe01a7fc640 1 -- 172.21.15.107:0/1620994423 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe02409f770 con 0x561469380300 2024-04-25T00:20:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.466+0000 7fe01a7fc640 1 -- 172.21.15.107:0/1620994423 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe024020040 con 0x561469380300 2024-04-25T00:20:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.466+0000 7fe03999c740 1 -- 172.21.15.107:0/1620994423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469380300 msgr2=0x56146939fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.466+0000 7fe03999c740 1 --2- 172.21.15.107:0/1620994423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469380300 0x56146939fac0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe0240cc6a0 tx=0x7fe024009c30 comp rx=0 tx=0).stop 2024-04-25T00:20:53.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.466+0000 7fe03999c740 1 -- 172.21.15.107:0/1620994423 shutdown_connections 2024-04-25T00:20:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.466+0000 7fe03999c740 1 --2- 172.21.15.107:0/1620994423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561469394200 0x5614693abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.466+0000 7fe03999c740 1 --2- 172.21.15.107:0/1620994423 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561469380c20 0x561469391a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.466+0000 7fe03999c740 1 --2- 172.21.15.107:0/1620994423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469380300 0x56146939fac0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.466+0000 7fe03999c740 1 -- 172.21.15.107:0/1620994423 >> 172.21.15.107:0/1620994423 conn(0x56146937c950 msgr2=0x56146937baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.466+0000 7fe03999c740 1 -- 172.21.15.107:0/1620994423 shutdown_connections 2024-04-25T00:20:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.466+0000 7fe03999c740 1 -- 172.21.15.107:0/1620994423 wait complete. 2024-04-25T00:20:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.466+0000 7fe03999c740 1 -- start start 2024-04-25T00:20:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.467+0000 7fe03999c740 1 Processor -- start 2024-04-25T00:20:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.467+0000 7fe03999c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561469380300 0x561469397ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.467+0000 7fe03999c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561469380c20 0x561469397ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.467+0000 7fe03999c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469394200 0x561469398530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.467+0000 7fe03999c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe0301aadc0 con 0x561469394200 2024-04-25T00:20:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.467+0000 7fe03999c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe0301ac230 con 0x561469380c20 2024-04-25T00:20:53.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.467+0000 7fe03999c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe0301ab490 con 0x561469380300 2024-04-25T00:20:53.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.467+0000 7fe03999c740 10 client.0 _ll_register_callbacks cb 0x7fe028016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:53.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.467+0000 7fe035153640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469394200 0x561469398530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.467+0000 7fe035153640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469394200 0x561469398530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44214/0 (socket says 172.21.15.107:44214) 2024-04-25T00:20:53.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.467+0000 7fe035153640 1 -- 172.21.15.107:0/3300056994 learned_addr learned my addr 172.21.15.107:0/3300056994 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:20:53.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.467+0000 7fe02ffff640 1 --2- 172.21.15.107:0/3300056994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561469380c20 0x561469397ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.467+0000 7fe034952640 1 --2- 172.21.15.107:0/3300056994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561469380300 0x561469397ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.467+0000 7fe035153640 1 -- 172.21.15.107:0/3300056994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561469380300 msgr2=0x561469397ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:53.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.467+0000 7fe035153640 1 --2- 172.21.15.107:0/3300056994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561469380300 0x561469397ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.467+0000 7fe035153640 1 -- 172.21.15.107:0/3300056994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561469380c20 msgr2=0x561469397ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:53.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.467+0000 7fe035153640 1 --2- 172.21.15.107:0/3300056994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561469380c20 0x561469397ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.467+0000 7fe035153640 1 -- 172.21.15.107:0/3300056994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614694097c0 con 0x561469394200 2024-04-25T00:20:53.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.467+0000 7fe035153640 1 --2- 172.21.15.107:0/3300056994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469394200 0x561469398530 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe02801e740 tx=0x7fe02801e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb10b27efe2e58f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:53.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.467+0000 7fe00ffff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:20:53.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.468+0000 7fe00ffff640 1 -- 172.21.15.107:0/3300056994 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe028037030 con 0x561469394200 2024-04-25T00:20:53.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.468+0000 7fe00ffff640 1 -- 172.21.15.107:0/3300056994 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fe028026660 con 0x561469394200 2024-04-25T00:20:53.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.468+0000 7fe03999c740 1 -- 172.21.15.107:0/3300056994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe02409f770 con 0x561469394200 2024-04-25T00:20:53.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.468+0000 7fe00ffff640 1 -- 172.21.15.107:0/3300056994 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe028011c40 con 0x561469394200 2024-04-25T00:20:53.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.468+0000 7fe03999c740 10 client.16632 fetch_fsmap learned FSMap version 28 2024-04-25T00:20:53.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.468+0000 7fe03999c740 1 -- 172.21.15.107:0/3300056994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56146940aa20 con 0x561469394200 2024-04-25T00:20:53.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.468+0000 7fe00ffff640 1 -- 172.21.15.107:0/3300056994 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe02409f770 con 0x561469394200 2024-04-25T00:20:53.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.468+0000 7fe00ffff640 1 -- 172.21.15.107:0/3300056994 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 28) ==== 8370+0+0 (secure 0 0 0) 0x7fe028032040 con 0x561469394200 2024-04-25T00:20:53.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.468+0000 7fe03999c740 10 client.16632 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:20:53.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.468+0000 7fe03999c740 10 client.16632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:20:53.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.468+0000 7fe03999c740 20 client.16632 populate_metadata read hostname 'smithi107' 2024-04-25T00:20:53.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.468+0000 7fe03999c740 1 --2- 172.21.15.107:0/3300056994 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56146940b170 0x56146940d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:53.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.468+0000 7fe03999c740 4 client.16632 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:20:53.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.468+0000 7fe03999c740 1 -- 172.21.15.107:0/3300056994 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5614692ecf40 con 0x56146940b170 2024-04-25T00:20:53.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.468+0000 7fe034952640 1 --2- 172.21.15.107:0/3300056994 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56146940b170 0x56146940d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:53.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.469+0000 7fe00ffff640 10 client.16632 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:53.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.469+0000 7fe034952640 1 --2- 172.21.15.107:0/3300056994 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56146940b170 0x56146940d590 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bddfb363db78d6dc server_cookie=882d4bc28adec4f3 in_seq=0 out_seq=0 2024-04-25T00:20:53.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.469+0000 7fe00ffff640 1 -- 172.21.15.107:0/3300056994 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5614692ecf40 con 0x56146940b170 2024-04-25T00:20:53.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.469+0000 7fe00ffff640 10 client.16632 handle_command_reply: tid=0 2024-04-25T00:20:53.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.469+0000 7fe03999c740 1 client.16632 shutdown 2024-04-25T00:20:53.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.469+0000 7fe03999c740 1 -- 172.21.15.107:0/3300056994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469394200 msgr2=0x561469398530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:53.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.469+0000 7fe03999c740 1 --2- 172.21.15.107:0/3300056994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469394200 0x561469398530 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe02801e740 tx=0x7fe02801e770 comp rx=0 tx=0).stop 2024-04-25T00:20:53.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.469+0000 7fe03999c740 1 -- 172.21.15.107:0/3300056994 shutdown_connections 2024-04-25T00:20:53.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.469+0000 7fe03999c740 1 --2- 172.21.15.107:0/3300056994 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56146940b170 0x56146940d590 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.469+0000 7fe03999c740 1 --2- 172.21.15.107:0/3300056994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561469394200 0x561469398530 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.469+0000 7fe03999c740 1 --2- 172.21.15.107:0/3300056994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561469380c20 0x561469397ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.469+0000 7fe00ffff640 0 client.16632 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:20:53.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.469+0000 7fe03999c740 1 --2- 172.21.15.107:0/3300056994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561469380300 0x561469397ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.469+0000 7fe03999c740 1 -- 172.21.15.107:0/3300056994 >> 172.21.15.107:0/3300056994 conn(0x56146937c950 msgr2=0x561469381ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:53.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.469+0000 7fe03999c740 1 -- 172.21.15.107:0/3300056994 shutdown_connections 2024-04-25T00:20:53.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.469+0000 7fe03999c740 1 -- 172.21.15.107:0/3300056994 wait complete. 2024-04-25T00:20:53.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.469+0000 7fe03999c740 20 client.16632 trim_cache size 0 max 16384 2024-04-25T00:20:53.477 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:20:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:20:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-04-25T00:20:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "78b9f7fc-100b-4003-b334-ea85d59807be": { 2024-04-25T00:20:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-04-25T00:20:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "78b9f7fc-100b-4003-b334-ea85d59807be", 2024-04-25T00:20:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-04-25T00:20:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:20:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:20:53.478 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:20:53.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.470+0000 7fe036bb6640 1 -- 172.21.15.107:0/1530933410 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe0200a2d80 msgr2=0x7fe0200a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:53.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.470+0000 7fe036bb6640 1 --2- 172.21.15.107:0/1530933410 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe0200a2d80 0x7fe0200a5240 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe02800ad70 tx=0x7fe02800e720 comp rx=0 tx=0).stop 2024-04-25T00:20:53.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.470+0000 7fe036bb6640 1 -- 172.21.15.107:0/1530933410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe030100620 msgr2=0x7fe030198e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:53.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.470+0000 7fe036bb6640 1 --2- 172.21.15.107:0/1530933410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe030100620 0x7fe030198e20 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe02400d770 tx=0x7fe02400dc30 comp rx=0 tx=0).stop 2024-04-25T00:20:53.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.470+0000 7fe036bb6640 1 -- 172.21.15.107:0/1530933410 shutdown_connections 2024-04-25T00:20:53.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.470+0000 7fe036bb6640 1 --2- 172.21.15.107:0/1530933410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe030106630 0x7fe0301a3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.470+0000 7fe036bb6640 1 --2- 172.21.15.107:0/1530933410 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe0200a2d80 0x7fe0200a5240 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.470+0000 7fe036bb6640 1 --2- 172.21.15.107:0/1530933410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe030100fe0 0x7fe030199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.470+0000 7fe036bb6640 1 --2- 172.21.15.107:0/1530933410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe030100620 0x7fe030198e20 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:53.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.470+0000 7fe036bb6640 1 -- 172.21.15.107:0/1530933410 >> 172.21.15.107:0/1530933410 conn(0x7fe0300776a0 msgr2=0x7fe030103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:53.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.470+0000 7fe036bb6640 1 -- 172.21.15.107:0/1530933410 shutdown_connections 2024-04-25T00:20:53.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:20:53.470+0000 7fe036bb6640 1 -- 172.21.15.107:0/1530933410 wait complete. 2024-04-25T00:20:53.481 INFO:tasks.cephfs.filesystem:scrub status for tag:78b9f7fc-100b-4003-b334-ea85d59807be - {'path': '/', 'tag': '78b9f7fc-100b-4003-b334-ea85d59807be', 'options': 'recursive,force'} 2024-04-25T00:20:54.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:54 smithi170 ceph-mon[32807]: pgmap v136: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:20:54.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:54 smithi170 ceph-mon[32807]: scrub queued for path: / 2024-04-25T00:20:54.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:54 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:20:54.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:54 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:20:54.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:54 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:20:54.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:54 smithi107 ceph-mon[29153]: pgmap v136: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:20:54.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:54 smithi107 ceph-mon[29153]: scrub queued for path: / 2024-04-25T00:20:54.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:54 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:20:54.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:54 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:20:54.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:54 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:20:54.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:54 smithi195 ceph-mon[32833]: pgmap v136: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:20:54.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:54 smithi195 ceph-mon[32833]: scrub queued for path: / 2024-04-25T00:20:54.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:54 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:20:54.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:54 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:20:54.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:54 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:20:56.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:56 smithi170 ceph-mon[32807]: pgmap v137: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:20:56.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:56 smithi107 ceph-mon[29153]: pgmap v137: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:20:56.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:56 smithi195 ceph-mon[32833]: pgmap v137: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:20:58.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:58 smithi170 ceph-mon[32807]: pgmap v138: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:20:58.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:20:58 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:20:58.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:58 smithi107 ceph-mon[29153]: pgmap v138: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:20:58.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:20:58 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:20:58.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:58 smithi195 ceph-mon[32833]: pgmap v138: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:20:58.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:20:58 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:21:00.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:00 smithi170 ceph-mon[32807]: pgmap v139: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:21:00.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:00 smithi107 ceph-mon[29153]: pgmap v139: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:21:00.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:00 smithi195 ceph-mon[32833]: pgmap v139: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:21:02.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:02 smithi170 ceph-mon[32807]: pgmap v140: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:21:02.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:02 smithi107 ceph-mon[29153]: pgmap v140: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:21:02.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:02 smithi195 ceph-mon[32833]: pgmap v140: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:21:04.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:04 smithi170 ceph-mon[32807]: pgmap v141: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T00:21:04.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:04 smithi107 ceph-mon[29153]: pgmap v141: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T00:21:04.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:04 smithi195 ceph-mon[32833]: pgmap v141: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T00:21:06.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:06 smithi170 ceph-mon[32807]: pgmap v142: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T00:21:06.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:06 smithi107 ceph-mon[29153]: pgmap v142: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T00:21:06.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:06 smithi195 ceph-mon[32833]: pgmap v142: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T00:21:08.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:08 smithi170 ceph-mon[32807]: pgmap v143: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T00:21:08.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:08 smithi107 ceph-mon[29153]: pgmap v143: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T00:21:08.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:08 smithi195 ceph-mon[32833]: pgmap v143: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T00:21:10.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:10 smithi107 ceph-mon[29153]: pgmap v144: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T00:21:10.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:10 smithi195 ceph-mon[32833]: pgmap v144: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T00:21:10.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:10 smithi170 ceph-mon[32807]: pgmap v144: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T00:21:12.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:12 smithi107 ceph-mon[29153]: pgmap v145: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T00:21:12.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:12 smithi195 ceph-mon[32833]: pgmap v145: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T00:21:12.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:12 smithi170 ceph-mon[32807]: pgmap v145: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T00:21:14.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:14 smithi107 ceph-mon[29153]: pgmap v146: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T00:21:14.487 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:14 smithi170 ceph-mon[32807]: pgmap v146: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T00:21:14.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:14 smithi195 ceph-mon[32833]: pgmap v146: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T00:21:16.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:16 smithi107 ceph-mon[29153]: pgmap v147: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:21:16.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:16 smithi195 ceph-mon[32833]: pgmap v147: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:21:16.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:16 smithi170 ceph-mon[32807]: pgmap v147: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:21:18.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:18 smithi107 ceph-mon[29153]: pgmap v148: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:21:18.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:18 smithi195 ceph-mon[32833]: pgmap v148: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:21:18.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:18 smithi170 ceph-mon[32807]: pgmap v148: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:21:19.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:19 smithi195 ceph-mon[32833]: pgmap v149: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T00:21:19.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:19 smithi170 ceph-mon[32807]: pgmap v149: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T00:21:19.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:19 smithi107 ceph-mon[29153]: pgmap v149: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T00:21:21.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:21 smithi170 ceph-mon[32807]: pgmap v150: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T00:21:21.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:21 smithi107 ceph-mon[29153]: pgmap v150: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T00:21:21.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:21 smithi195 ceph-mon[32833]: pgmap v150: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T00:21:22.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:22 smithi170 ceph-mon[32807]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:21:22.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:22 smithi107 ceph-mon[29153]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:21:22.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:22 smithi195 ceph-mon[32833]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:21:23.482 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:21:23.482 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:21:23.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.595+0000 7fe7f38d4640 1 -- 172.21.15.107:0/376942536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7ec100fd0 msgr2=0x7fe7ec10d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:23.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.595+0000 7fe7f38d4640 1 --2- 172.21.15.107:0/376942536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7ec100fd0 0x7fe7ec10d4e0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009cd0 tx=0x7fe7e402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.595+0000 7fe7f38d4640 1 -- 172.21.15.107:0/376942536 shutdown_connections 2024-04-25T00:21:23.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.595+0000 7fe7f38d4640 1 --2- 172.21.15.107:0/376942536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7ec100fd0 0x7fe7ec10d4e0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.595+0000 7fe7f38d4640 1 --2- 172.21.15.107:0/376942536 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7ec100610 0x7fe7ec100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.595+0000 7fe7f38d4640 1 --2- 172.21.15.107:0/376942536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7ec1066b0 0x7fe7ec106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.595+0000 7fe7f38d4640 1 -- 172.21.15.107:0/376942536 >> 172.21.15.107:0/376942536 conn(0x7fe7ec077740 msgr2=0x7fe7ec077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:23.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.595+0000 7fe7f38d4640 1 -- 172.21.15.107:0/376942536 shutdown_connections 2024-04-25T00:21:23.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.595+0000 7fe7f38d4640 1 -- 172.21.15.107:0/376942536 wait complete. 2024-04-25T00:21:23.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.595+0000 7fe7f38d4640 1 Processor -- start 2024-04-25T00:21:23.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.595+0000 7fe7f38d4640 1 -- start start 2024-04-25T00:21:23.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.596+0000 7fe7f38d4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7ec100610 0x7fe7ec198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.596+0000 7fe7f38d4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7ec100fd0 0x7fe7ec199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.596+0000 7fe7f38d4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7ec1066b0 0x7fe7ec1a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.596+0000 7fe7f38d4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec110a30 con 0x7fe7ec100fd0 2024-04-25T00:21:23.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.596+0000 7fe7f38d4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1108b0 con 0x7fe7ec1066b0 2024-04-25T00:21:23.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.596+0000 7fe7f38d4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec110bb0 con 0x7fe7ec100610 2024-04-25T00:21:23.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.596+0000 7fe7f0e6f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7ec100fd0 0x7fe7ec199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.596+0000 7fe7f0e6f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7ec100fd0 0x7fe7ec199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48574/0 (socket says 172.21.15.107:48574) 2024-04-25T00:21:23.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.596+0000 7fe7f0e6f640 1 -- 172.21.15.107:0/1208700468 learned_addr learned my addr 172.21.15.107:0/1208700468 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:23.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.596+0000 7fe7f1670640 1 --2- 172.21.15.107:0/1208700468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7ec100610 0x7fe7ec198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.596+0000 7fe7f1e71640 1 --2- 172.21.15.107:0/1208700468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7ec1066b0 0x7fe7ec1a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.596+0000 7fe7f0e6f640 1 -- 172.21.15.107:0/1208700468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7ec100610 msgr2=0x7fe7ec198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:23.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.596+0000 7fe7f0e6f640 1 --2- 172.21.15.107:0/1208700468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7ec100610 0x7fe7ec198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.596+0000 7fe7f0e6f640 1 -- 172.21.15.107:0/1208700468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7ec1066b0 msgr2=0x7fe7ec1a3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:23.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.596+0000 7fe7f0e6f640 1 --2- 172.21.15.107:0/1208700468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7ec1066b0 0x7fe7ec1a3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.596+0000 7fe7f0e6f640 1 -- 172.21.15.107:0/1208700468 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ec1a4390 con 0x7fe7ec100fd0 2024-04-25T00:21:23.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.596+0000 7fe7f0e6f640 1 --2- 172.21.15.107:0/1208700468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7ec100fd0 0x7fe7ec199310 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00e9f0 tx=0x7fe7dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:23.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.596+0000 7fe7e27fc640 1 -- 172.21.15.107:0/1208700468 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc00cd80 con 0x7fe7ec100fd0 2024-04-25T00:21:23.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.596+0000 7fe7e27fc640 1 -- 172.21.15.107:0/1208700468 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7dc005500 con 0x7fe7ec100fd0 2024-04-25T00:21:23.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.596+0000 7fe7f38d4640 1 -- 172.21.15.107:0/1208700468 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ec1a4560 con 0x7fe7ec100fd0 2024-04-25T00:21:23.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.596+0000 7fe7e27fc640 1 -- 172.21.15.107:0/1208700468 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc010690 con 0x7fe7ec100fd0 2024-04-25T00:21:23.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.597+0000 7fe7f38d4640 1 -- 172.21.15.107:0/1208700468 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ec1a4a60 con 0x7fe7ec100fd0 2024-04-25T00:21:23.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.599+0000 7fe7e27fc640 1 -- 172.21.15.107:0/1208700468 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe7dc0046c0 con 0x7fe7ec100fd0 2024-04-25T00:21:23.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.599+0000 7fe7f65e4740 1 Processor -- start 2024-04-25T00:21:23.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.599+0000 7fe7f65e4740 1 -- start start 2024-04-25T00:21:23.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.600+0000 7fe7e27fc640 1 --2- 172.21.15.107:0/1208700468 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe7d80a2d00 0x7fe7d80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.600+0000 7fe7f1e71640 1 --2- 172.21.15.107:0/1208700468 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe7d80a2d00 0x7fe7d80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.600+0000 7fe7f65e4740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bec1ce4630 0x55bec1ce4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.600+0000 7fe7f65e4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bec1ce4f50 0x55bec1cef0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.600+0000 7fe7f65e4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1cef620 0x55bec1cf1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.600+0000 7fe7f65e4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bec1c496b0 con 0x55bec1cef620 2024-04-25T00:21:23.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.600+0000 7fe7f65e4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bec1bf1880 con 0x55bec1ce4f50 2024-04-25T00:21:23.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.600+0000 7fe7f65e4740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bec1c45670 con 0x55bec1ce4630 2024-04-25T00:21:23.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.600+0000 7fe7e27fc640 1 -- 172.21.15.107:0/1208700468 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fe7dc0cfd80 con 0x7fe7ec100fd0 2024-04-25T00:21:23.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.600+0000 7fe7f1e71640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1cef620 0x55bec1cf1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.600+0000 7fe7f1e71640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1cef620 0x55bec1cf1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48584/0 (socket says 172.21.15.107:48584) 2024-04-25T00:21:23.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.600+0000 7fe7f1e71640 1 -- 172.21.15.107:0/3251537099 learned_addr learned my addr 172.21.15.107:0/3251537099 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:23.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.600+0000 7fe7f1670640 1 --2- 172.21.15.107:0/3251537099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bec1ce4630 0x55bec1ce4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.600+0000 7fe7f0e6f640 1 --2- 172.21.15.107:0/3251537099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bec1ce4f50 0x55bec1cef0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.600+0000 7fe7f1e71640 1 --2- 172.21.15.107:0/1208700468 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe7d80a2d00 0x7fe7d80a51c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009cd0 tx=0x7fe7e40316c0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:23.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.600+0000 7fe7f1e71640 1 -- 172.21.15.107:0/3251537099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bec1ce4630 msgr2=0x55bec1ce4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:23.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.600+0000 7fe7f1e71640 1 --2- 172.21.15.107:0/3251537099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bec1ce4630 0x55bec1ce4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.600+0000 7fe7f1e71640 1 -- 172.21.15.107:0/3251537099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bec1ce4f50 msgr2=0x55bec1cef0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:23.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.600+0000 7fe7f1e71640 1 --2- 172.21.15.107:0/3251537099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bec1ce4f50 0x55bec1cef0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.600+0000 7fe7f1e71640 1 -- 172.21.15.107:0/3251537099 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bec1a277d0 con 0x55bec1cef620 2024-04-25T00:21:23.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.600+0000 7fe7f1e71640 1 --2- 172.21.15.107:0/3251537099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1cef620 0x55bec1cf1a10 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4035000 tx=0x7fe7e4049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e002daea3a2e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:23.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.601+0000 7fe7d77fe640 1 -- 172.21.15.107:0/3251537099 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e40044c0 con 0x55bec1cef620 2024-04-25T00:21:23.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.601+0000 7fe7d77fe640 1 -- 172.21.15.107:0/3251537099 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7e4050070 con 0x55bec1cef620 2024-04-25T00:21:23.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.601+0000 7fe7d77fe640 1 -- 172.21.15.107:0/3251537099 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e404a450 con 0x55bec1cef620 2024-04-25T00:21:23.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.601+0000 7fe7f65e4740 1 -- 172.21.15.107:0/3251537099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1cef620 msgr2=0x55bec1cf1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:23.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.601+0000 7fe7f65e4740 1 --2- 172.21.15.107:0/3251537099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1cef620 0x55bec1cf1a10 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4035000 tx=0x7fe7e4049000 comp rx=0 tx=0).stop 2024-04-25T00:21:23.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.601+0000 7fe7f65e4740 1 -- 172.21.15.107:0/3251537099 shutdown_connections 2024-04-25T00:21:23.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.601+0000 7fe7f65e4740 1 --2- 172.21.15.107:0/3251537099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1cef620 0x55bec1cf1a10 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.601+0000 7fe7f65e4740 1 --2- 172.21.15.107:0/3251537099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bec1ce4f50 0x55bec1cef0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.601+0000 7fe7f65e4740 1 --2- 172.21.15.107:0/3251537099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bec1ce4630 0x55bec1ce4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.601+0000 7fe7f65e4740 1 -- 172.21.15.107:0/3251537099 >> 172.21.15.107:0/3251537099 conn(0x55bec1ce0950 msgr2=0x55bec1ce0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:23.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.601+0000 7fe7f65e4740 1 -- 172.21.15.107:0/3251537099 shutdown_connections 2024-04-25T00:21:23.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.601+0000 7fe7f65e4740 1 -- 172.21.15.107:0/3251537099 wait complete. 2024-04-25T00:21:23.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.602+0000 7fe7f65e4740 1 -- start start 2024-04-25T00:21:23.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.602+0000 7fe7f65e4740 1 Processor -- start 2024-04-25T00:21:23.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.602+0000 7fe7f65e4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1ce4630 0x55bec1cec570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.602+0000 7fe7f65e4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bec1ce4f50 0x55bec1cecab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.602+0000 7fe7f65e4740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bec1cef620 0x55bec1cecff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.602+0000 7fe7f65e4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bec1cf23b0 con 0x55bec1ce4630 2024-04-25T00:21:23.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.602+0000 7fe7f65e4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bec1c04f50 con 0x55bec1ce4f50 2024-04-25T00:21:23.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.602+0000 7fe7f65e4740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55bec1cf2530 con 0x55bec1cef620 2024-04-25T00:21:23.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.602+0000 7fe7f65e4740 10 client.0 _ll_register_callbacks cb 0x55bec1c40000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:23.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.602+0000 7fe7f1670640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1ce4630 0x55bec1cec570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.602+0000 7fe7f1670640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1ce4630 0x55bec1cec570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48600/0 (socket says 172.21.15.107:48600) 2024-04-25T00:21:23.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.602+0000 7fe7f1670640 1 -- 172.21.15.107:0/2259545116 learned_addr learned my addr 172.21.15.107:0/2259545116 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:23.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.603+0000 7fe7f0e6f640 1 --2- 172.21.15.107:0/2259545116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bec1ce4f50 0x55bec1cecab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.603+0000 7fe7f1670640 1 -- 172.21.15.107:0/2259545116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bec1cef620 msgr2=0x55bec1cecff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:23.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.603+0000 7fe7f1670640 1 --2- 172.21.15.107:0/2259545116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bec1cef620 0x55bec1cecff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.603+0000 7fe7f1670640 1 -- 172.21.15.107:0/2259545116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bec1ce4f50 msgr2=0x55bec1cecab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:23.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.603+0000 7fe7f1670640 1 --2- 172.21.15.107:0/2259545116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bec1ce4f50 0x55bec1cecab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.603+0000 7fe7f1670640 1 -- 172.21.15.107:0/2259545116 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bec195d3f0 con 0x55bec1ce4630 2024-04-25T00:21:23.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.603+0000 7fe7f0e6f640 1 --2- 172.21.15.107:0/2259545116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bec1ce4f50 0x55bec1cecab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:23.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.603+0000 7fe7f1670640 1 --2- 172.21.15.107:0/2259545116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1ce4630 0x55bec1cec570 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800be80 tx=0x7fe7e8012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def58fa89b82770b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:23.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.603+0000 7fe7d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:21:23.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.603+0000 7fe7d4ff9640 1 -- 172.21.15.107:0/2259545116 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e801f060 con 0x55bec1ce4630 2024-04-25T00:21:23.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.603+0000 7fe7d4ff9640 1 -- 172.21.15.107:0/2259545116 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fe7e80108a0 con 0x55bec1ce4630 2024-04-25T00:21:23.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.603+0000 7fe7f65e4740 1 -- 172.21.15.107:0/2259545116 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7e4050070 con 0x55bec1ce4630 2024-04-25T00:21:23.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.603+0000 7fe7d4ff9640 1 -- 172.21.15.107:0/2259545116 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe7e8026050 con 0x55bec1ce4630 2024-04-25T00:21:23.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.603+0000 7fe7f65e4740 10 client.16644 fetch_fsmap learned FSMap version 29 2024-04-25T00:21:23.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.603+0000 7fe7f65e4740 1 -- 172.21.15.107:0/2259545116 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bec191d0f0 con 0x55bec1ce4630 2024-04-25T00:21:23.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.603+0000 7fe7d4ff9640 1 -- 172.21.15.107:0/2259545116 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7e4050070 con 0x55bec1ce4630 2024-04-25T00:21:23.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.603+0000 7fe7d4ff9640 1 -- 172.21.15.107:0/2259545116 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fe7e8019030 con 0x55bec1ce4630 2024-04-25T00:21:23.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.604+0000 7fe7f65e4740 10 client.16644 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:21:23.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.604+0000 7fe7f65e4740 10 client.16644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:23.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.604+0000 7fe7f65e4740 20 client.16644 populate_metadata read hostname 'smithi107' 2024-04-25T00:21:23.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.604+0000 7fe7f65e4740 1 --2- 172.21.15.107:0/2259545116 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55bec1cff910 0x55bec1d01d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.604+0000 7fe7f65e4740 4 client.16644 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:23.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.604+0000 7fe7f65e4740 1 -- 172.21.15.107:0/2259545116 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bec1c56bf0 con 0x55bec1cff910 2024-04-25T00:21:23.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.604+0000 7fe7f0e6f640 1 --2- 172.21.15.107:0/2259545116 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55bec1cff910 0x55bec1d01d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.604+0000 7fe7f0e6f640 1 --2- 172.21.15.107:0/2259545116 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55bec1cff910 0x55bec1d01d90 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46a4b257b2dfa2da server_cookie=a99df95a5e461ed6 in_seq=0 out_seq=0 2024-04-25T00:21:23.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.604+0000 7fe7d4ff9640 10 client.16644 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:23.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.605+0000 7fe7d4ff9640 1 -- 172.21.15.107:0/2259545116 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bec1c56bf0 con 0x55bec1cff910 2024-04-25T00:21:23.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.605+0000 7fe7d4ff9640 10 client.16644 handle_command_reply: tid=0 2024-04-25T00:21:23.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.605+0000 7fe7f65e4740 1 client.16644 shutdown 2024-04-25T00:21:23.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.605+0000 7fe7f65e4740 1 -- 172.21.15.107:0/2259545116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1ce4630 msgr2=0x55bec1cec570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:23.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.605+0000 7fe7f65e4740 1 --2- 172.21.15.107:0/2259545116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1ce4630 0x55bec1cec570 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800be80 tx=0x7fe7e8012820 comp rx=0 tx=0).stop 2024-04-25T00:21:23.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.605+0000 7fe7f65e4740 1 -- 172.21.15.107:0/2259545116 shutdown_connections 2024-04-25T00:21:23.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.605+0000 7fe7f65e4740 1 --2- 172.21.15.107:0/2259545116 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55bec1cff910 0x55bec1d01d90 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.605+0000 7fe7f65e4740 1 --2- 172.21.15.107:0/2259545116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bec1cef620 0x55bec1cecff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.605+0000 7fe7f65e4740 1 --2- 172.21.15.107:0/2259545116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bec1ce4f50 0x55bec1cecab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.605+0000 7fe7d4ff9640 0 client.16644 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:23.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.605+0000 7fe7f65e4740 1 --2- 172.21.15.107:0/2259545116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1ce4630 0x55bec1cec570 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.605+0000 7fe7f65e4740 1 -- 172.21.15.107:0/2259545116 >> 172.21.15.107:0/2259545116 conn(0x55bec1ce0950 msgr2=0x55bec1ce6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:23.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.605+0000 7fe7f65e4740 1 -- 172.21.15.107:0/2259545116 shutdown_connections 2024-04-25T00:21:23.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.605+0000 7fe7f65e4740 1 -- 172.21.15.107:0/2259545116 wait complete. 2024-04-25T00:21:23.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.605+0000 7fe7f65e4740 20 client.16644 trim_cache size 0 max 16384 2024-04-25T00:21:23.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.623+0000 7fe7f65e4740 1 Processor -- start 2024-04-25T00:21:23.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.623+0000 7fe7f65e4740 1 -- start start 2024-04-25T00:21:23.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.623+0000 7fe7f65e4740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bec1ce4630 0x55bec1cfb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.623+0000 7fe7f65e4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bec1ce4f50 0x55bec1ce7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.623+0000 7fe7f65e4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1cef620 0x55bec1ce6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.623+0000 7fe7f65e4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1b1470 con 0x55bec1cef620 2024-04-25T00:21:23.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.623+0000 7fe7f65e4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1aecb0 con 0x55bec1ce4f50 2024-04-25T00:21:23.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.623+0000 7fe7f65e4740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1acb00 con 0x55bec1ce4630 2024-04-25T00:21:23.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.623+0000 7fe7f1e71640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1cef620 0x55bec1ce6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.623+0000 7fe7f1e71640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1cef620 0x55bec1ce6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48610/0 (socket says 172.21.15.107:48610) 2024-04-25T00:21:23.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.623+0000 7fe7f1e71640 1 -- 172.21.15.107:0/3985218122 learned_addr learned my addr 172.21.15.107:0/3985218122 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:23.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.623+0000 7fe7f0e6f640 1 --2- 172.21.15.107:0/3985218122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bec1ce4f50 0x55bec1ce7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.623+0000 7fe7f1e71640 1 -- 172.21.15.107:0/3985218122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bec1ce4630 msgr2=0x55bec1cfb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:23.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.623+0000 7fe7f1670640 1 --2- 172.21.15.107:0/3985218122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bec1ce4630 0x55bec1cfb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.623+0000 7fe7f1e71640 1 --2- 172.21.15.107:0/3985218122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bec1ce4630 0x55bec1cfb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.623+0000 7fe7f1e71640 1 -- 172.21.15.107:0/3985218122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bec1ce4f50 msgr2=0x55bec1ce7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:23.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.623+0000 7fe7f1e71640 1 --2- 172.21.15.107:0/3985218122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bec1ce4f50 0x55bec1ce7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.623+0000 7fe7f1e71640 1 -- 172.21.15.107:0/3985218122 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bec1cd2090 con 0x55bec1cef620 2024-04-25T00:21:23.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.623+0000 7fe7f1e71640 1 --2- 172.21.15.107:0/3985218122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1cef620 0x55bec1ce6ea0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4031b50 tx=0x7fe7e4009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4faf4141bb527 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:23.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.624+0000 7fe7d6ffd640 1 -- 172.21.15.107:0/3985218122 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e402af20 con 0x55bec1cef620 2024-04-25T00:21:23.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.624+0000 7fe7d6ffd640 1 -- 172.21.15.107:0/3985218122 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7e4050040 con 0x55bec1cef620 2024-04-25T00:21:23.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.624+0000 7fe7d6ffd640 1 -- 172.21.15.107:0/3985218122 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e404a450 con 0x55bec1cef620 2024-04-25T00:21:23.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.624+0000 7fe7f65e4740 1 -- 172.21.15.107:0/3985218122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1cef620 msgr2=0x55bec1ce6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:23.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.624+0000 7fe7f65e4740 1 --2- 172.21.15.107:0/3985218122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1cef620 0x55bec1ce6ea0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4031b50 tx=0x7fe7e4009360 comp rx=0 tx=0).stop 2024-04-25T00:21:23.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.624+0000 7fe7f65e4740 1 -- 172.21.15.107:0/3985218122 shutdown_connections 2024-04-25T00:21:23.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.624+0000 7fe7f65e4740 1 --2- 172.21.15.107:0/3985218122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1cef620 0x55bec1ce6ea0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.624+0000 7fe7f65e4740 1 --2- 172.21.15.107:0/3985218122 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bec1ce4f50 0x55bec1ce7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.624+0000 7fe7f65e4740 1 --2- 172.21.15.107:0/3985218122 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bec1ce4630 0x55bec1cfb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.624+0000 7fe7f65e4740 1 -- 172.21.15.107:0/3985218122 >> 172.21.15.107:0/3985218122 conn(0x55bec1ce0950 msgr2=0x55bec1cdf260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:23.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.624+0000 7fe7f65e4740 1 -- 172.21.15.107:0/3985218122 shutdown_connections 2024-04-25T00:21:23.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.624+0000 7fe7f65e4740 1 -- 172.21.15.107:0/3985218122 wait complete. 2024-04-25T00:21:23.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.624+0000 7fe7f65e4740 1 -- start start 2024-04-25T00:21:23.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.625+0000 7fe7f65e4740 1 Processor -- start 2024-04-25T00:21:23.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.625+0000 7fe7f65e4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1ce4630 0x55bec1cf7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.625+0000 7fe7f65e4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bec1ce4f50 0x55bec1cf3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.625+0000 7fe7f65e4740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bec1cef620 0x55bec1cf3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.625+0000 7fe7f65e4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1ad170 con 0x55bec1ce4630 2024-04-25T00:21:23.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.625+0000 7fe7f65e4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1ae5e0 con 0x55bec1ce4f50 2024-04-25T00:21:23.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.625+0000 7fe7f65e4740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec1ad840 con 0x55bec1cef620 2024-04-25T00:21:23.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.625+0000 7fe7f65e4740 10 client.0 _ll_register_callbacks cb 0x7fe7e800b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:23.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.625+0000 7fe7f1670640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1ce4630 0x55bec1cf7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.625+0000 7fe7f1670640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1ce4630 0x55bec1cf7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48622/0 (socket says 172.21.15.107:48622) 2024-04-25T00:21:23.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.625+0000 7fe7f1670640 1 -- 172.21.15.107:0/1626107700 learned_addr learned my addr 172.21.15.107:0/1626107700 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:23.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.625+0000 7fe7f0e6f640 1 --2- 172.21.15.107:0/1626107700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bec1ce4f50 0x55bec1cf3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.625+0000 7fe7f1670640 1 -- 172.21.15.107:0/1626107700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bec1cef620 msgr2=0x55bec1cf3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:23.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.625+0000 7fe7f1670640 1 --2- 172.21.15.107:0/1626107700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bec1cef620 0x55bec1cf3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.625+0000 7fe7f1670640 1 -- 172.21.15.107:0/1626107700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bec1ce4f50 msgr2=0x55bec1cf3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:23.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.625+0000 7fe7f1670640 1 --2- 172.21.15.107:0/1626107700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bec1ce4f50 0x55bec1cf3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.625+0000 7fe7f1670640 1 -- 172.21.15.107:0/1626107700 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bec1d64ed0 con 0x55bec1ce4630 2024-04-25T00:21:23.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.625+0000 7fe7f0e6f640 1 --2- 172.21.15.107:0/1626107700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bec1ce4f50 0x55bec1cf3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:23.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.625+0000 7fe7f1670640 1 --2- 172.21.15.107:0/1626107700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1ce4630 0x55bec1cf7ce0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800b890 tx=0x7fe7e8005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31813f0772d9019 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:23.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.625+0000 7fe7d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:21:23.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.626+0000 7fe7d4ff9640 1 -- 172.21.15.107:0/1626107700 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e801f020 con 0x55bec1ce4630 2024-04-25T00:21:23.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.626+0000 7fe7d4ff9640 1 -- 172.21.15.107:0/1626107700 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fe7e800f790 con 0x55bec1ce4630 2024-04-25T00:21:23.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.626+0000 7fe7f65e4740 1 -- 172.21.15.107:0/1626107700 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7e4050040 con 0x55bec1ce4630 2024-04-25T00:21:23.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.626+0000 7fe7d4ff9640 1 -- 172.21.15.107:0/1626107700 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe7e8012d20 con 0x55bec1ce4630 2024-04-25T00:21:23.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.626+0000 7fe7f65e4740 10 client.16650 fetch_fsmap learned FSMap version 29 2024-04-25T00:21:23.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.626+0000 7fe7f65e4740 1 -- 172.21.15.107:0/1626107700 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bec1d66080 con 0x55bec1ce4630 2024-04-25T00:21:23.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.626+0000 7fe7d4ff9640 1 -- 172.21.15.107:0/1626107700 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7e4050040 con 0x55bec1ce4630 2024-04-25T00:21:23.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.626+0000 7fe7d4ff9640 1 -- 172.21.15.107:0/1626107700 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fe7e8019050 con 0x55bec1ce4630 2024-04-25T00:21:23.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.626+0000 7fe7f65e4740 10 client.16650 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:21:23.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.626+0000 7fe7f65e4740 10 client.16650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:23.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.626+0000 7fe7f65e4740 20 client.16650 populate_metadata read hostname 'smithi107' 2024-04-25T00:21:23.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.626+0000 7fe7f65e4740 1 --2- 172.21.15.107:0/1626107700 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55bec1d66820 0x55bec1d68c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.626+0000 7fe7f65e4740 4 client.16650 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:21:23.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.626+0000 7fe7f65e4740 1 -- 172.21.15.107:0/1626107700 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bec1c50f40 con 0x55bec1d66820 2024-04-25T00:21:23.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.626+0000 7fe7f0e6f640 1 --2- 172.21.15.107:0/1626107700 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55bec1d66820 0x55bec1d68c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.627+0000 7fe7f0e6f640 1 --2- 172.21.15.107:0/1626107700 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55bec1d66820 0x55bec1d68c60 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccab47c6cf43043 server_cookie=dd2b3f018fc1afef in_seq=0 out_seq=0 2024-04-25T00:21:23.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.627+0000 7fe7d4ff9640 10 client.16650 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:23.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.627+0000 7fe7d4ff9640 1 -- 172.21.15.107:0/1626107700 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bec1c50f40 con 0x55bec1d66820 2024-04-25T00:21:23.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.627+0000 7fe7d4ff9640 10 client.16650 handle_command_reply: tid=0 2024-04-25T00:21:23.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.627+0000 7fe7f65e4740 1 client.16650 shutdown 2024-04-25T00:21:23.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.627+0000 7fe7f65e4740 1 -- 172.21.15.107:0/1626107700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1ce4630 msgr2=0x55bec1cf7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:23.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.627+0000 7fe7f65e4740 1 --2- 172.21.15.107:0/1626107700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1ce4630 0x55bec1cf7ce0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe7e800b890 tx=0x7fe7e8005270 comp rx=0 tx=0).stop 2024-04-25T00:21:23.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.627+0000 7fe7f65e4740 1 -- 172.21.15.107:0/1626107700 shutdown_connections 2024-04-25T00:21:23.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.627+0000 7fe7f65e4740 1 --2- 172.21.15.107:0/1626107700 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55bec1d66820 0x55bec1d68c60 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.627+0000 7fe7f65e4740 1 --2- 172.21.15.107:0/1626107700 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55bec1cef620 0x55bec1cf3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.627+0000 7fe7f65e4740 1 --2- 172.21.15.107:0/1626107700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bec1ce4f50 0x55bec1cf3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.627+0000 7fe7d4ff9640 0 client.16650 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:23.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.627+0000 7fe7f65e4740 1 --2- 172.21.15.107:0/1626107700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bec1ce4630 0x55bec1cf7ce0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.627+0000 7fe7f65e4740 1 -- 172.21.15.107:0/1626107700 >> 172.21.15.107:0/1626107700 conn(0x55bec1ce0950 msgr2=0x55bec1cdfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:23.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.627+0000 7fe7f65e4740 1 -- 172.21.15.107:0/1626107700 shutdown_connections 2024-04-25T00:21:23.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.627+0000 7fe7f65e4740 1 -- 172.21.15.107:0/1626107700 wait complete. 2024-04-25T00:21:23.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.627+0000 7fe7f65e4740 20 client.16650 trim_cache size 0 max 16384 2024-04-25T00:21:23.636 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:21:23.637 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-04-25T00:21:23.637 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-04-25T00:21:23.637 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:21:23.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.628+0000 7fe7f38d4640 1 -- 172.21.15.107:0/1208700468 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe7d80a2d00 msgr2=0x7fe7d80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:23.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.628+0000 7fe7f38d4640 1 --2- 172.21.15.107:0/1208700468 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe7d80a2d00 0x7fe7d80a51c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4009cd0 tx=0x7fe7e40316c0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.628+0000 7fe7f38d4640 1 -- 172.21.15.107:0/1208700468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7ec100fd0 msgr2=0x7fe7ec199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:23.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.628+0000 7fe7f38d4640 1 --2- 172.21.15.107:0/1208700468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7ec100fd0 0x7fe7ec199310 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00e9f0 tx=0x7fe7dc00edf0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.628+0000 7fe7f38d4640 1 -- 172.21.15.107:0/1208700468 shutdown_connections 2024-04-25T00:21:23.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.628+0000 7fe7f38d4640 1 --2- 172.21.15.107:0/1208700468 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe7d80a2d00 0x7fe7d80a51c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.628+0000 7fe7f38d4640 1 --2- 172.21.15.107:0/1208700468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7ec1066b0 0x7fe7ec1a3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.628+0000 7fe7f38d4640 1 --2- 172.21.15.107:0/1208700468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7ec100fd0 0x7fe7ec199310 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.628+0000 7fe7f38d4640 1 --2- 172.21.15.107:0/1208700468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7ec100610 0x7fe7ec198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.628+0000 7fe7f38d4640 1 -- 172.21.15.107:0/1208700468 >> 172.21.15.107:0/1208700468 conn(0x7fe7ec077740 msgr2=0x7fe7ec0ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:23.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.628+0000 7fe7f38d4640 1 -- 172.21.15.107:0/1208700468 shutdown_connections 2024-04-25T00:21:23.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.628+0000 7fe7f38d4640 1 -- 172.21.15.107:0/1208700468 wait complete. 2024-04-25T00:21:23.640 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:21:23.640 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:21:23.660 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:23 smithi107 ceph-mon[29153]: pgmap v151: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T00:21:23.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:23 smithi170 ceph-mon[32807]: pgmap v151: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T00:21:23.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.736+0000 7f739c29a640 1 -- 172.21.15.107:0/1031326851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7394102a20 msgr2=0x7f7394102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:23.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.736+0000 7f739c29a640 1 --2- 172.21.15.107:0/1031326851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7394102a20 0x7f7394102e00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7390009900 tx=0x7f739002a480 comp rx=0 tx=0).stop 2024-04-25T00:21:23.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.736+0000 7f739c29a640 1 -- 172.21.15.107:0/1031326851 shutdown_connections 2024-04-25T00:21:23.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.736+0000 7f739c29a640 1 --2- 172.21.15.107:0/1031326851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f739410bd60 0x7f739410e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.736+0000 7f739c29a640 1 --2- 172.21.15.107:0/1031326851 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7394103340 0x7f739410b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.736+0000 7f739c29a640 1 --2- 172.21.15.107:0/1031326851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7394102a20 0x7f7394102e00 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.736+0000 7f739c29a640 1 -- 172.21.15.107:0/1031326851 >> 172.21.15.107:0/1031326851 conn(0x7f73940777a0 msgr2=0x7f7394077bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:23.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.736+0000 7f739c29a640 1 -- 172.21.15.107:0/1031326851 shutdown_connections 2024-04-25T00:21:23.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.736+0000 7f739c29a640 1 -- 172.21.15.107:0/1031326851 wait complete. 2024-04-25T00:21:23.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.737+0000 7f739c29a640 1 Processor -- start 2024-04-25T00:21:23.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.737+0000 7f739c29a640 1 -- start start 2024-04-25T00:21:23.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.737+0000 7f739c29a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7394102a20 0x7f7394198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.737+0000 7f739c29a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7394103340 0x7f7394198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.737+0000 7f739c29a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f739410bd60 0x7f739419c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.737+0000 7f739c29a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7394111b30 con 0x7f7394102a20 2024-04-25T00:21:23.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.737+0000 7f739c29a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f73941119b0 con 0x7f7394103340 2024-04-25T00:21:23.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.737+0000 7f739c29a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7394111cb0 con 0x7f739410bd60 2024-04-25T00:21:23.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.737+0000 7f739a036640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7394102a20 0x7f7394198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.737+0000 7f739a036640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7394102a20 0x7f7394198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48648/0 (socket says 172.21.15.107:48648) 2024-04-25T00:21:23.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.737+0000 7f739a036640 1 -- 172.21.15.107:0/4208573001 learned_addr learned my addr 172.21.15.107:0/4208573001 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:23.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.737+0000 7f7399835640 1 --2- 172.21.15.107:0/4208573001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7394103340 0x7f7394198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.737+0000 7f739a837640 1 --2- 172.21.15.107:0/4208573001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f739410bd60 0x7f739419c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.737+0000 7f739a036640 1 -- 172.21.15.107:0/4208573001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f739410bd60 msgr2=0x7f739419c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:23.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.737+0000 7f739a036640 1 --2- 172.21.15.107:0/4208573001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f739410bd60 0x7f739419c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.737+0000 7f739a036640 1 -- 172.21.15.107:0/4208573001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7394103340 msgr2=0x7f7394198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:23.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.737+0000 7f739a036640 1 --2- 172.21.15.107:0/4208573001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7394103340 0x7f7394198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.737+0000 7f739a036640 1 -- 172.21.15.107:0/4208573001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f739419d040 con 0x7f7394102a20 2024-04-25T00:21:23.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.737+0000 7f739a837640 1 --2- 172.21.15.107:0/4208573001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f739410bd60 0x7f739419c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:23.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.737+0000 7f739a036640 1 --2- 172.21.15.107:0/4208573001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7394102a20 0x7f7394198050 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7390006010 tx=0x7f73900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:23.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.738+0000 7f738b7fe640 1 -- 172.21.15.107:0/4208573001 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f739003f030 con 0x7f7394102a20 2024-04-25T00:21:23.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.738+0000 7f739c29a640 1 -- 172.21.15.107:0/4208573001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f739419d2d0 con 0x7f7394102a20 2024-04-25T00:21:23.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.738+0000 7f738b7fe640 1 -- 172.21.15.107:0/4208573001 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7390003ce0 con 0x7f7394102a20 2024-04-25T00:21:23.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.738+0000 7f738b7fe640 1 -- 172.21.15.107:0/4208573001 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7390035070 con 0x7f7394102a20 2024-04-25T00:21:23.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.738+0000 7f739c29a640 1 -- 172.21.15.107:0/4208573001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73941a4bc0 con 0x7f7394102a20 2024-04-25T00:21:23.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.738+0000 7f739c29a640 1 -- 172.21.15.107:0/4208573001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73940689b0 con 0x7f7394102a20 2024-04-25T00:21:23.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.740+0000 7f738b7fe640 1 -- 172.21.15.107:0/4208573001 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7390042030 con 0x7f7394102a20 2024-04-25T00:21:23.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.740+0000 7f738b7fe640 1 --2- 172.21.15.107:0/4208573001 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f73800a2d00 0x7f73800a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:23.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.740+0000 7f7399835640 1 --2- 172.21.15.107:0/4208573001 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f73800a2d00 0x7f73800a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:23.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.740+0000 7f738b7fe640 1 -- 172.21.15.107:0/4208573001 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f73900efcd0 con 0x7f7394102a20 2024-04-25T00:21:23.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.741+0000 7f7399835640 1 --2- 172.21.15.107:0/4208573001 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f73800a2d00 0x7f73800a51c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f739410da70 tx=0x7f738400b390 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:23.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.745+0000 7f738b7fe640 1 -- 172.21.15.107:0/4208573001 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f73900bc640 con 0x7f7394102a20 2024-04-25T00:21:23.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:23 smithi195 ceph-mon[32833]: pgmap v151: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T00:21:23.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.955+0000 7f739c29a640 1 -- 172.21.15.107:0/4208573001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7394106ee0 con 0x7f7394102a20 2024-04-25T00:21:23.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.956+0000 7f738b7fe640 1 -- 172.21.15.107:0/4208573001 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11904 (secure 0 0 0) 0x7f73900bbf40 con 0x7f7394102a20 2024-04-25T00:21:23.957 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:21:23.957 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:21:21.198322+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24521,24493]},"id":1}]} 2024-04-25T00:21:23.958 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 29 2024-04-25T00:21:23.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.958+0000 7f739c29a640 1 -- 172.21.15.107:0/4208573001 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f73800a2d00 msgr2=0x7f73800a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:23.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.958+0000 7f739c29a640 1 --2- 172.21.15.107:0/4208573001 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f73800a2d00 0x7f73800a51c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f739410da70 tx=0x7f738400b390 comp rx=0 tx=0).stop 2024-04-25T00:21:23.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.958+0000 7f739c29a640 1 -- 172.21.15.107:0/4208573001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7394102a20 msgr2=0x7f7394198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:23.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.958+0000 7f739c29a640 1 --2- 172.21.15.107:0/4208573001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7394102a20 0x7f7394198050 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7390006010 tx=0x7f73900042f0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.958+0000 7f739c29a640 1 -- 172.21.15.107:0/4208573001 shutdown_connections 2024-04-25T00:21:23.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.958+0000 7f739c29a640 1 --2- 172.21.15.107:0/4208573001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f739410bd60 0x7f739419c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.958+0000 7f739c29a640 1 --2- 172.21.15.107:0/4208573001 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f73800a2d00 0x7f73800a51c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.958+0000 7f739c29a640 1 --2- 172.21.15.107:0/4208573001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7394103340 0x7f7394198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.958+0000 7f739c29a640 1 --2- 172.21.15.107:0/4208573001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7394102a20 0x7f7394198050 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:23.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.958+0000 7f739c29a640 1 -- 172.21.15.107:0/4208573001 >> 172.21.15.107:0/4208573001 conn(0x7f73940777a0 msgr2=0x7f73940fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:23.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.958+0000 7f739c29a640 1 -- 172.21.15.107:0/4208573001 shutdown_connections 2024-04-25T00:21:23.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:23.958+0000 7f739c29a640 1 -- 172.21.15.107:0/4208573001 wait complete. 2024-04-25T00:21:23.971 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:21:24.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.110+0000 7ff632815640 1 -- 172.21.15.107:0/1806443575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff62c102120 msgr2=0x7ff62c10e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.110+0000 7ff632815640 1 --2- 172.21.15.107:0/1806443575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff62c102120 0x7ff62c10e630 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff624009cd0 tx=0x7ff62402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.110+0000 7ff632815640 1 -- 172.21.15.107:0/1806443575 shutdown_connections 2024-04-25T00:21:24.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.110+0000 7ff632815640 1 --2- 172.21.15.107:0/1806443575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff62c102120 0x7ff62c10e630 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.110+0000 7ff632815640 1 --2- 172.21.15.107:0/1806443575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff62c101760 0x7ff62c101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.110+0000 7ff632815640 1 --2- 172.21.15.107:0/1806443575 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff62c107770 0x7ff62c107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.110+0000 7ff632815640 1 -- 172.21.15.107:0/1806443575 >> 172.21.15.107:0/1806443575 conn(0x7ff62c0fd250 msgr2=0x7ff62c0ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:24.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.110+0000 7ff632815640 1 -- 172.21.15.107:0/1806443575 shutdown_connections 2024-04-25T00:21:24.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.110+0000 7ff632815640 1 -- 172.21.15.107:0/1806443575 wait complete. 2024-04-25T00:21:24.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.110+0000 7ff632815640 1 Processor -- start 2024-04-25T00:21:24.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.110+0000 7ff632815640 1 -- start start 2024-04-25T00:21:24.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.111+0000 7ff632815640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff62c101760 0x7ff62c073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.111+0000 7ff632815640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff62c102120 0x7ff62c073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.111+0000 7ff632815640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff62c107770 0x7ff62c0744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.111+0000 7ff632815640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c111c00 con 0x7ff62c107770 2024-04-25T00:21:24.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.111+0000 7ff632815640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c111a80 con 0x7ff62c101760 2024-04-25T00:21:24.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.111+0000 7ff632815640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c111d80 con 0x7ff62c102120 2024-04-25T00:21:24.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.111+0000 7ff630db2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff62c107770 0x7ff62c0744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.111+0000 7ff630db2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff62c107770 0x7ff62c0744c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48654/0 (socket says 172.21.15.107:48654) 2024-04-25T00:21:24.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.111+0000 7ff630db2640 1 -- 172.21.15.107:0/1993247607 learned_addr learned my addr 172.21.15.107:0/1993247607 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:24.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.111+0000 7ff62bfff640 1 --2- 172.21.15.107:0/1993247607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff62c101760 0x7ff62c073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.111+0000 7ff62b7fe640 1 --2- 172.21.15.107:0/1993247607 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff62c102120 0x7ff62c073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.111+0000 7ff630db2640 1 -- 172.21.15.107:0/1993247607 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff62c102120 msgr2=0x7ff62c073f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.111+0000 7ff630db2640 1 --2- 172.21.15.107:0/1993247607 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff62c102120 0x7ff62c073f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.111+0000 7ff630db2640 1 -- 172.21.15.107:0/1993247607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff62c101760 msgr2=0x7ff62c073a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.111+0000 7ff630db2640 1 --2- 172.21.15.107:0/1993247607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff62c101760 0x7ff62c073a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.111+0000 7ff630db2640 1 -- 172.21.15.107:0/1993247607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff62c1a6620 con 0x7ff62c107770 2024-04-25T00:21:24.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.111+0000 7ff630db2640 1 --2- 172.21.15.107:0/1993247607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff62c107770 0x7ff62c0744c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff624006010 tx=0x7ff624003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:24.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.112+0000 7ff6297fa640 1 -- 172.21.15.107:0/1993247607 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624004450 con 0x7ff62c107770 2024-04-25T00:21:24.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.112+0000 7ff632815640 1 -- 172.21.15.107:0/1993247607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff62c1a68b0 con 0x7ff62c107770 2024-04-25T00:21:24.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.112+0000 7ff6297fa640 1 -- 172.21.15.107:0/1993247607 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff624003e10 con 0x7ff62c107770 2024-04-25T00:21:24.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.112+0000 7ff6297fa640 1 -- 172.21.15.107:0/1993247607 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624046480 con 0x7ff62c107770 2024-04-25T00:21:24.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.112+0000 7ff632815640 1 -- 172.21.15.107:0/1993247607 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff62c1a6e00 con 0x7ff62c107770 2024-04-25T00:21:24.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.114+0000 7ff6297fa640 1 -- 172.21.15.107:0/1993247607 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff6240045f0 con 0x7ff62c107770 2024-04-25T00:21:24.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.114+0000 7ff635602740 1 Processor -- start 2024-04-25T00:21:24.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.114+0000 7ff6297fa640 1 --2- 172.21.15.107:0/1993247607 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff6140a2d00 0x7ff6140a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.114+0000 7ff635602740 1 -- start start 2024-04-25T00:21:24.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.114+0000 7ff6297fa640 1 -- 172.21.15.107:0/1993247607 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7ff62402e030 con 0x7ff62c107770 2024-04-25T00:21:24.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.114+0000 7ff62b7fe640 1 --2- 172.21.15.107:0/1993247607 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff6140a2d00 0x7ff6140a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.114+0000 7ff635602740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c5e580 0x557eb9c5e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.114+0000 7ff635602740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557eb9c5eea0 0x557eb9c69030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.114+0000 7ff635602740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557eb9c69570 0x557eb9c6b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.114+0000 7ff635602740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557eb9bc3670 con 0x557eb9c5e580 2024-04-25T00:21:24.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.114+0000 7ff635602740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557eb9b6b970 con 0x557eb9c5eea0 2024-04-25T00:21:24.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.114+0000 7ff635602740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557eb9bbf630 con 0x557eb9c69570 2024-04-25T00:21:24.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.115+0000 7ff62bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c5e580 0x557eb9c5e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.115+0000 7ff62bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c5e580 0x557eb9c5e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48668/0 (socket says 172.21.15.107:48668) 2024-04-25T00:21:24.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.115+0000 7ff62bfff640 1 -- 172.21.15.107:0/860334840 learned_addr learned my addr 172.21.15.107:0/860334840 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:24.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.115+0000 7ff630db2640 1 --2- 172.21.15.107:0/860334840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557eb9c69570 0x557eb9c6b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.115+0000 7ff62bfff640 1 -- 172.21.15.107:0/860334840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557eb9c69570 msgr2=0x557eb9c6b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:24.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.115+0000 7ff62bfff640 1 --2- 172.21.15.107:0/860334840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557eb9c69570 0x557eb9c6b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.115+0000 7ff62bfff640 1 -- 172.21.15.107:0/860334840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557eb9c5eea0 msgr2=0x557eb9c69030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:24.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.115+0000 7ff62bfff640 1 --2- 172.21.15.107:0/860334840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557eb9c5eea0 0x557eb9c69030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.115+0000 7ff62bfff640 1 -- 172.21.15.107:0/860334840 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eb9b8f400 con 0x557eb9c5e580 2024-04-25T00:21:24.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.115+0000 7ff62b7fe640 1 --2- 172.21.15.107:0/1993247607 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff6140a2d00 0x7ff6140a51c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff62c100780 tx=0x7ff620009c70 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:24.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.115+0000 7ff62bfff640 1 --2- 172.21.15.107:0/860334840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c5e580 0x557eb9c5e960 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff61c009a70 tx=0x7ff61c009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622028c1522837af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:24.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.115+0000 7ff6127fc640 1 -- 172.21.15.107:0/860334840 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c017070 con 0x557eb9c5e580 2024-04-25T00:21:24.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.115+0000 7ff6127fc640 1 -- 172.21.15.107:0/860334840 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff61c013470 con 0x557eb9c5e580 2024-04-25T00:21:24.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.115+0000 7ff6127fc640 1 -- 172.21.15.107:0/860334840 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c012630 con 0x557eb9c5e580 2024-04-25T00:21:24.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.116+0000 7ff635602740 1 -- 172.21.15.107:0/860334840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c5e580 msgr2=0x557eb9c5e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.116+0000 7ff635602740 1 --2- 172.21.15.107:0/860334840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c5e580 0x557eb9c5e960 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7ff61c009a70 tx=0x7ff61c009f30 comp rx=0 tx=0).stop 2024-04-25T00:21:24.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.116+0000 7ff635602740 1 -- 172.21.15.107:0/860334840 shutdown_connections 2024-04-25T00:21:24.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.116+0000 7ff635602740 1 --2- 172.21.15.107:0/860334840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557eb9c69570 0x557eb9c6b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.116+0000 7ff635602740 1 --2- 172.21.15.107:0/860334840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557eb9c5eea0 0x557eb9c69030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.116+0000 7ff635602740 1 --2- 172.21.15.107:0/860334840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c5e580 0x557eb9c5e960 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.116+0000 7ff635602740 1 -- 172.21.15.107:0/860334840 >> 172.21.15.107:0/860334840 conn(0x557eb9c5a8a0 msgr2=0x557eb9c5ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:24.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.116+0000 7ff635602740 1 -- 172.21.15.107:0/860334840 shutdown_connections 2024-04-25T00:21:24.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.116+0000 7ff635602740 1 -- 172.21.15.107:0/860334840 wait complete. 2024-04-25T00:21:24.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.116+0000 7ff635602740 1 -- start start 2024-04-25T00:21:24.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.116+0000 7ff635602740 1 Processor -- start 2024-04-25T00:21:24.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.117+0000 7ff635602740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557eb9c5e580 0x557eb9c68e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.117+0000 7ff635602740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557eb9c5eea0 0x557eb9c66160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.117+0000 7ff635602740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c69570 0x557eb9c666a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.117+0000 7ff635602740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557eb9c6c360 con 0x557eb9c69570 2024-04-25T00:21:24.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.117+0000 7ff635602740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557eb9b7f050 con 0x557eb9c5e580 2024-04-25T00:21:24.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.117+0000 7ff635602740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557eb9c6c4e0 con 0x557eb9c5eea0 2024-04-25T00:21:24.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.117+0000 7ff635602740 10 client.0 _ll_register_callbacks cb 0x557eb98d6be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:24.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.117+0000 7ff630db2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c69570 0x557eb9c666a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.117+0000 7ff630db2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c69570 0x557eb9c666a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48682/0 (socket says 172.21.15.107:48682) 2024-04-25T00:21:24.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.117+0000 7ff630db2640 1 -- 172.21.15.107:0/3569448142 learned_addr learned my addr 172.21.15.107:0/3569448142 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:24.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.117+0000 7ff62b7fe640 1 --2- 172.21.15.107:0/3569448142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557eb9c5eea0 0x557eb9c66160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.117+0000 7ff62bfff640 1 --2- 172.21.15.107:0/3569448142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557eb9c5e580 0x557eb9c68e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.117+0000 7ff630db2640 1 -- 172.21.15.107:0/3569448142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557eb9c5eea0 msgr2=0x557eb9c66160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:24.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.117+0000 7ff630db2640 1 --2- 172.21.15.107:0/3569448142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557eb9c5eea0 0x557eb9c66160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.117+0000 7ff630db2640 1 -- 172.21.15.107:0/3569448142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557eb9c5e580 msgr2=0x557eb9c68e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:24.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.117+0000 7ff630db2640 1 --2- 172.21.15.107:0/3569448142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557eb9c5e580 0x557eb9c68e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.117+0000 7ff630db2640 1 -- 172.21.15.107:0/3569448142 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eb9b50dc0 con 0x557eb9c69570 2024-04-25T00:21:24.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.117+0000 7ff630db2640 1 --2- 172.21.15.107:0/3569448142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c69570 0x557eb9c666a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff6240b9780 tx=0x7ff624009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980344ee1d3a1e44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:24.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.117+0000 7ff60b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:21:24.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.117+0000 7ff60b7fe640 1 -- 172.21.15.107:0/3569448142 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6240c16d0 con 0x557eb9c69570 2024-04-25T00:21:24.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.117+0000 7ff60b7fe640 1 -- 172.21.15.107:0/3569448142 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7ff6240eeb90 con 0x557eb9c69570 2024-04-25T00:21:24.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.117+0000 7ff635602740 1 -- 172.21.15.107:0/3569448142 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff61c013470 con 0x557eb9c69570 2024-04-25T00:21:24.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.117+0000 7ff60b7fe640 1 -- 172.21.15.107:0/3569448142 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff6240c5f90 con 0x557eb9c69570 2024-04-25T00:21:24.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.118+0000 7ff635602740 10 client.16668 fetch_fsmap learned FSMap version 29 2024-04-25T00:21:24.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.118+0000 7ff635602740 1 -- 172.21.15.107:0/3569448142 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557eb98d73f0 con 0x557eb9c69570 2024-04-25T00:21:24.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.118+0000 7ff60b7fe640 1 -- 172.21.15.107:0/3569448142 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff61c013470 con 0x557eb9c69570 2024-04-25T00:21:24.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.118+0000 7ff60b7fe640 1 -- 172.21.15.107:0/3569448142 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7ff62404e070 con 0x557eb9c69570 2024-04-25T00:21:24.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.118+0000 7ff635602740 10 client.16668 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:21:24.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.118+0000 7ff635602740 10 client.16668 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:21:24.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.118+0000 7ff635602740 20 client.16668 populate_metadata read hostname 'smithi107' 2024-04-25T00:21:24.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.118+0000 7ff635602740 1 --2- 172.21.15.107:0/3569448142 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x557eb9c79550 0x557eb9c7b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.118+0000 7ff635602740 4 client.16668 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:24.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.118+0000 7ff635602740 1 -- 172.21.15.107:0/3569448142 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557eb9bd0bb0 con 0x557eb9c79550 2024-04-25T00:21:24.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.118+0000 7ff62bfff640 1 --2- 172.21.15.107:0/3569448142 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x557eb9c79550 0x557eb9c7b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.118+0000 7ff60b7fe640 10 client.16668 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:21:24.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.118+0000 7ff62bfff640 1 --2- 172.21.15.107:0/3569448142 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x557eb9c79550 0x557eb9c7b9b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=37e79ddf2ebdbdc0 server_cookie=e8ea2bbba81557bf in_seq=0 out_seq=0 2024-04-25T00:21:24.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.119+0000 7ff60b7fe640 1 -- 172.21.15.107:0/3569448142 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557eb9bd0bb0 con 0x557eb9c79550 2024-04-25T00:21:24.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.119+0000 7ff60b7fe640 10 client.16668 handle_command_reply: tid=0 2024-04-25T00:21:24.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.119+0000 7ff635602740 1 client.16668 shutdown 2024-04-25T00:21:24.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.119+0000 7ff635602740 1 -- 172.21.15.107:0/3569448142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c69570 msgr2=0x557eb9c666a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.119+0000 7ff635602740 1 --2- 172.21.15.107:0/3569448142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c69570 0x557eb9c666a0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7ff6240b9780 tx=0x7ff624009990 comp rx=0 tx=0).stop 2024-04-25T00:21:24.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.119+0000 7ff635602740 1 -- 172.21.15.107:0/3569448142 shutdown_connections 2024-04-25T00:21:24.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.119+0000 7ff635602740 1 --2- 172.21.15.107:0/3569448142 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x557eb9c79550 0x557eb9c7b9b0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.119+0000 7ff635602740 1 --2- 172.21.15.107:0/3569448142 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c69570 0x557eb9c666a0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.120+0000 7ff635602740 1 --2- 172.21.15.107:0/3569448142 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557eb9c5eea0 0x557eb9c66160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.120+0000 7ff635602740 1 --2- 172.21.15.107:0/3569448142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557eb9c5e580 0x557eb9c68e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.120+0000 7ff60b7fe640 0 client.16668 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:21:24.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.120+0000 7ff635602740 1 -- 172.21.15.107:0/3569448142 >> 172.21.15.107:0/3569448142 conn(0x557eb9c5a8a0 msgr2=0x557eb9c59b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:24.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.120+0000 7ff635602740 1 -- 172.21.15.107:0/3569448142 shutdown_connections 2024-04-25T00:21:24.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.120+0000 7ff635602740 1 -- 172.21.15.107:0/3569448142 wait complete. 2024-04-25T00:21:24.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.120+0000 7ff635602740 20 client.16668 trim_cache size 0 max 16384 2024-04-25T00:21:24.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.133+0000 7ff635602740 1 Processor -- start 2024-04-25T00:21:24.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.133+0000 7ff635602740 1 -- start start 2024-04-25T00:21:24.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.133+0000 7ff635602740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557eb9c5e580 0x557eb9c75220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.133+0000 7ff635602740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557eb9c5eea0 0x557eb9c81270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.133+0000 7ff635602740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c69570 0x557eb9c817b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.133+0000 7ff635602740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c1b35b0 con 0x557eb9c69570 2024-04-25T00:21:24.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.133+0000 7ff635602740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c1b0df0 con 0x557eb9c5e580 2024-04-25T00:21:24.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.133+0000 7ff635602740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c1aec40 con 0x557eb9c5eea0 2024-04-25T00:21:24.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.133+0000 7ff630db2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c69570 0x557eb9c817b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.133+0000 7ff630db2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c69570 0x557eb9c817b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48694/0 (socket says 172.21.15.107:48694) 2024-04-25T00:21:24.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.133+0000 7ff630db2640 1 -- 172.21.15.107:0/273926039 learned_addr learned my addr 172.21.15.107:0/273926039 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:24.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.133+0000 7ff62bfff640 1 --2- 172.21.15.107:0/273926039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557eb9c5e580 0x557eb9c75220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.133+0000 7ff630db2640 1 -- 172.21.15.107:0/273926039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557eb9c5eea0 msgr2=0x557eb9c81270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:24.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.133+0000 7ff62b7fe640 1 --2- 172.21.15.107:0/273926039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557eb9c5eea0 0x557eb9c81270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.133+0000 7ff630db2640 1 --2- 172.21.15.107:0/273926039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557eb9c5eea0 0x557eb9c81270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.133+0000 7ff630db2640 1 -- 172.21.15.107:0/273926039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557eb9c5e580 msgr2=0x557eb9c75220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:24.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.133+0000 7ff630db2640 1 --2- 172.21.15.107:0/273926039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557eb9c5e580 0x557eb9c75220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.133+0000 7ff630db2640 1 -- 172.21.15.107:0/273926039 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eb99a17d0 con 0x557eb9c69570 2024-04-25T00:21:24.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.133+0000 7ff62bfff640 1 --2- 172.21.15.107:0/273926039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557eb9c5e580 0x557eb9c75220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:24.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.133+0000 7ff62b7fe640 1 --2- 172.21.15.107:0/273926039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557eb9c5eea0 0x557eb9c81270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:24.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.133+0000 7ff630db2640 1 --2- 172.21.15.107:0/273926039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c69570 0x557eb9c817b0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ff6240049b0 tx=0x7ff6240bb830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4355eb3e6e748aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:24.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.133+0000 7ff611ffb640 1 -- 172.21.15.107:0/273926039 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6240c16d0 con 0x557eb9c69570 2024-04-25T00:21:24.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.133+0000 7ff611ffb640 1 -- 172.21.15.107:0/273926039 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff624036040 con 0x557eb9c69570 2024-04-25T00:21:24.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.133+0000 7ff611ffb640 1 -- 172.21.15.107:0/273926039 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6240b9990 con 0x557eb9c69570 2024-04-25T00:21:24.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.134+0000 7ff635602740 1 -- 172.21.15.107:0/273926039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c69570 msgr2=0x557eb9c817b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.134+0000 7ff635602740 1 --2- 172.21.15.107:0/273926039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c69570 0x557eb9c817b0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ff6240049b0 tx=0x7ff6240bb830 comp rx=0 tx=0).stop 2024-04-25T00:21:24.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.134+0000 7ff635602740 1 -- 172.21.15.107:0/273926039 shutdown_connections 2024-04-25T00:21:24.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.134+0000 7ff635602740 1 --2- 172.21.15.107:0/273926039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c69570 0x557eb9c817b0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.134+0000 7ff635602740 1 --2- 172.21.15.107:0/273926039 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557eb9c5eea0 0x557eb9c81270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.134+0000 7ff635602740 1 --2- 172.21.15.107:0/273926039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557eb9c5e580 0x557eb9c75220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.134+0000 7ff635602740 1 -- 172.21.15.107:0/273926039 >> 172.21.15.107:0/273926039 conn(0x557eb9c5a8a0 msgr2=0x557eb9c59b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:24.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.134+0000 7ff635602740 1 -- 172.21.15.107:0/273926039 shutdown_connections 2024-04-25T00:21:24.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.134+0000 7ff635602740 1 -- 172.21.15.107:0/273926039 wait complete. 2024-04-25T00:21:24.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.134+0000 7ff635602740 1 -- start start 2024-04-25T00:21:24.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.134+0000 7ff635602740 1 Processor -- start 2024-04-25T00:21:24.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.135+0000 7ff635602740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c5e580 0x557eb9c6f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.135+0000 7ff635602740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557eb9c5eea0 0x557eb9c6f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.135+0000 7ff635602740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557eb9c69570 0x557eb9c6fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.135+0000 7ff635602740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c1af2b0 con 0x557eb9c5e580 2024-04-25T00:21:24.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.135+0000 7ff635602740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c1b0720 con 0x557eb9c5eea0 2024-04-25T00:21:24.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.135+0000 7ff635602740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff62c1af980 con 0x557eb9c69570 2024-04-25T00:21:24.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.135+0000 7ff635602740 10 client.0 _ll_register_callbacks cb 0x7ff624004900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:24.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.135+0000 7ff62bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c5e580 0x557eb9c6f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.135+0000 7ff62bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c5e580 0x557eb9c6f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48706/0 (socket says 172.21.15.107:48706) 2024-04-25T00:21:24.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.135+0000 7ff62bfff640 1 -- 172.21.15.107:0/2481967850 learned_addr learned my addr 172.21.15.107:0/2481967850 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:24.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.135+0000 7ff630db2640 1 --2- 172.21.15.107:0/2481967850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557eb9c69570 0x557eb9c6fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.135+0000 7ff62b7fe640 1 --2- 172.21.15.107:0/2481967850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557eb9c5eea0 0x557eb9c6f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.135+0000 7ff62bfff640 1 -- 172.21.15.107:0/2481967850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557eb9c69570 msgr2=0x557eb9c6fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:24.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.135+0000 7ff62bfff640 1 --2- 172.21.15.107:0/2481967850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557eb9c69570 0x557eb9c6fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.135+0000 7ff62bfff640 1 -- 172.21.15.107:0/2481967850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557eb9c5eea0 msgr2=0x557eb9c6f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:24.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.135+0000 7ff62bfff640 1 --2- 172.21.15.107:0/2481967850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557eb9c5eea0 0x557eb9c6f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.135+0000 7ff62bfff640 1 -- 172.21.15.107:0/2481967850 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eb9cdeb20 con 0x557eb9c5e580 2024-04-25T00:21:24.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.135+0000 7ff62bfff640 1 --2- 172.21.15.107:0/2481967850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c5e580 0x557eb9c6f190 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00eb60 tx=0x7ff61c013ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e52d4ea472553a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:24.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.135+0000 7ff60b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:21:24.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.135+0000 7ff60b7fe640 1 -- 172.21.15.107:0/2481967850 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c017040 con 0x557eb9c5e580 2024-04-25T00:21:24.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.135+0000 7ff60b7fe640 1 -- 172.21.15.107:0/2481967850 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7ff61c005dd0 con 0x557eb9c5e580 2024-04-25T00:21:24.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.135+0000 7ff635602740 1 -- 172.21.15.107:0/2481967850 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff624036040 con 0x557eb9c5e580 2024-04-25T00:21:24.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.135+0000 7ff60b7fe640 1 -- 172.21.15.107:0/2481967850 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff61c006b60 con 0x557eb9c5e580 2024-04-25T00:21:24.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.136+0000 7ff635602740 10 client.16674 fetch_fsmap learned FSMap version 29 2024-04-25T00:21:24.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.136+0000 7ff635602740 1 -- 172.21.15.107:0/2481967850 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557eb9cdfd70 con 0x557eb9c5e580 2024-04-25T00:21:24.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.136+0000 7ff60b7fe640 1 -- 172.21.15.107:0/2481967850 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff624036040 con 0x557eb9c5e580 2024-04-25T00:21:24.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.136+0000 7ff60b7fe640 1 -- 172.21.15.107:0/2481967850 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7ff61c006050 con 0x557eb9c5e580 2024-04-25T00:21:24.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.136+0000 7ff635602740 10 client.16674 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:21:24.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.136+0000 7ff635602740 10 client.16674 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:21:24.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.136+0000 7ff635602740 20 client.16674 populate_metadata read hostname 'smithi107' 2024-04-25T00:21:24.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.136+0000 7ff635602740 1 --2- 172.21.15.107:0/2481967850 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x557eb9ce0270 0x557eb9ce2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.136+0000 7ff635602740 4 client.16674 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:21:24.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.136+0000 7ff635602740 1 -- 172.21.15.107:0/2481967850 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557eb9bcaf00 con 0x557eb9ce0270 2024-04-25T00:21:24.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.136+0000 7ff62b7fe640 1 --2- 172.21.15.107:0/2481967850 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x557eb9ce0270 0x557eb9ce2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.136+0000 7ff60b7fe640 10 client.16674 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:21:24.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.136+0000 7ff62b7fe640 1 --2- 172.21.15.107:0/2481967850 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x557eb9ce0270 0x557eb9ce2690 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9580076cbad15167 server_cookie=5ad99cb81e1b1a9f in_seq=0 out_seq=0 2024-04-25T00:21:24.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.137+0000 7ff60b7fe640 1 -- 172.21.15.107:0/2481967850 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557eb9bcaf00 con 0x557eb9ce0270 2024-04-25T00:21:24.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.137+0000 7ff60b7fe640 10 client.16674 handle_command_reply: tid=0 2024-04-25T00:21:24.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.137+0000 7ff635602740 1 client.16674 shutdown 2024-04-25T00:21:24.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.137+0000 7ff635602740 1 -- 172.21.15.107:0/2481967850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c5e580 msgr2=0x557eb9c6f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.137+0000 7ff635602740 1 --2- 172.21.15.107:0/2481967850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c5e580 0x557eb9c6f190 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00eb60 tx=0x7ff61c013ed0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.137+0000 7ff635602740 1 -- 172.21.15.107:0/2481967850 shutdown_connections 2024-04-25T00:21:24.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.137+0000 7ff635602740 1 --2- 172.21.15.107:0/2481967850 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x557eb9ce0270 0x557eb9ce2690 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.137+0000 7ff635602740 1 --2- 172.21.15.107:0/2481967850 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557eb9c69570 0x557eb9c6fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.137+0000 7ff60b7fe640 0 client.16674 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:21:24.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.137+0000 7ff635602740 1 --2- 172.21.15.107:0/2481967850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557eb9c5eea0 0x557eb9c6f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.137+0000 7ff635602740 1 --2- 172.21.15.107:0/2481967850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557eb9c5e580 0x557eb9c6f190 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.137+0000 7ff635602740 1 -- 172.21.15.107:0/2481967850 >> 172.21.15.107:0/2481967850 conn(0x557eb9c5a8a0 msgr2=0x557eb9c74210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:24.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.137+0000 7ff635602740 1 -- 172.21.15.107:0/2481967850 shutdown_connections 2024-04-25T00:21:24.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.137+0000 7ff635602740 1 -- 172.21.15.107:0/2481967850 wait complete. 2024-04-25T00:21:24.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.137+0000 7ff635602740 20 client.16674 trim_cache size 0 max 16384 2024-04-25T00:21:24.147 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:21:24.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.137+0000 7ff632815640 1 -- 172.21.15.107:0/1993247607 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff6140a2d00 msgr2=0x7ff6140a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.137+0000 7ff632815640 1 --2- 172.21.15.107:0/1993247607 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff6140a2d00 0x7ff6140a51c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff62c100780 tx=0x7ff620009c70 comp rx=0 tx=0).stop 2024-04-25T00:21:24.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.137+0000 7ff632815640 1 -- 172.21.15.107:0/1993247607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff62c107770 msgr2=0x7ff62c0744c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.137+0000 7ff632815640 1 --2- 172.21.15.107:0/1993247607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff62c107770 0x7ff62c0744c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff624006010 tx=0x7ff624003c20 comp rx=0 tx=0).stop 2024-04-25T00:21:24.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.138+0000 7ff632815640 1 -- 172.21.15.107:0/1993247607 shutdown_connections 2024-04-25T00:21:24.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.138+0000 7ff632815640 1 --2- 172.21.15.107:0/1993247607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff62c107770 0x7ff62c0744c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.138+0000 7ff632815640 1 --2- 172.21.15.107:0/1993247607 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff62c102120 0x7ff62c073f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.138+0000 7ff632815640 1 --2- 172.21.15.107:0/1993247607 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff6140a2d00 0x7ff6140a51c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.138+0000 7ff632815640 1 --2- 172.21.15.107:0/1993247607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff62c101760 0x7ff62c073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.138+0000 7ff632815640 1 -- 172.21.15.107:0/1993247607 >> 172.21.15.107:0/1993247607 conn(0x7ff62c0fd250 msgr2=0x7ff62c0fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:24.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.138+0000 7ff632815640 1 -- 172.21.15.107:0/1993247607 shutdown_connections 2024-04-25T00:21:24.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.138+0000 7ff632815640 1 -- 172.21.15.107:0/1993247607 wait complete. 2024-04-25T00:21:24.150 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:21:24.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.248+0000 7f3f8646b640 1 -- 172.21.15.107:0/1889339775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f801055f0 msgr2=0x7f3f801059d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.248+0000 7f3f8646b640 1 --2- 172.21.15.107:0/1889339775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f801055f0 0x7f3f801059d0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009900 tx=0x7f3f7002a410 comp rx=0 tx=0).stop 2024-04-25T00:21:24.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.248+0000 7f3f8646b640 1 -- 172.21.15.107:0/1889339775 shutdown_connections 2024-04-25T00:21:24.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.248+0000 7f3f8646b640 1 --2- 172.21.15.107:0/1889339775 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f80101770 0x7f3f8010d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.248+0000 7f3f8646b640 1 --2- 172.21.15.107:0/1889339775 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f80100db0 0x7f3f80101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.248+0000 7f3f8646b640 1 --2- 172.21.15.107:0/1889339775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f801055f0 0x7f3f801059d0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.248+0000 7f3f8646b640 1 -- 172.21.15.107:0/1889339775 >> 172.21.15.107:0/1889339775 conn(0x7f3f80076c00 msgr2=0x7f3f800fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:24.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.248+0000 7f3f8646b640 1 -- 172.21.15.107:0/1889339775 shutdown_connections 2024-04-25T00:21:24.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.248+0000 7f3f8646b640 1 -- 172.21.15.107:0/1889339775 wait complete. 2024-04-25T00:21:24.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.248+0000 7f3f8646b640 1 Processor -- start 2024-04-25T00:21:24.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.248+0000 7f3f8646b640 1 -- start start 2024-04-25T00:21:24.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.249+0000 7f3f8646b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f80100db0 0x7f3f80198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.249+0000 7f3f8646b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f80101770 0x7f3f80198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.249+0000 7f3f8646b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f801055f0 0x7f3f8019c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.249+0000 7f3f8646b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3f80111b30 con 0x7f3f80100db0 2024-04-25T00:21:24.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.249+0000 7f3f8646b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801119b0 con 0x7f3f801055f0 2024-04-25T00:21:24.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.249+0000 7f3f8646b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3f80111cb0 con 0x7f3f80101770 2024-04-25T00:21:24.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.249+0000 7f3f7ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f80100db0 0x7f3f80198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.249+0000 7f3f7ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f80100db0 0x7f3f80198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48728/0 (socket says 172.21.15.107:48728) 2024-04-25T00:21:24.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.249+0000 7f3f7ffff640 1 -- 172.21.15.107:0/4109330089 learned_addr learned my addr 172.21.15.107:0/4109330089 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:24.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.249+0000 7f3f84a08640 1 --2- 172.21.15.107:0/4109330089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f801055f0 0x7f3f8019c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.249+0000 7f3f7f7fe640 1 --2- 172.21.15.107:0/4109330089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f80101770 0x7f3f80198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.249+0000 7f3f7ffff640 1 -- 172.21.15.107:0/4109330089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f80101770 msgr2=0x7f3f80198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.249+0000 7f3f7ffff640 1 --2- 172.21.15.107:0/4109330089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f80101770 0x7f3f80198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.249+0000 7f3f7ffff640 1 -- 172.21.15.107:0/4109330089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f801055f0 msgr2=0x7f3f8019c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.249+0000 7f3f7ffff640 1 --2- 172.21.15.107:0/4109330089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f801055f0 0x7f3f8019c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.249+0000 7f3f7ffff640 1 -- 172.21.15.107:0/4109330089 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8019d000 con 0x7f3f80100db0 2024-04-25T00:21:24.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.249+0000 7f3f7ffff640 1 --2- 172.21.15.107:0/4109330089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f80100db0 0x7f3f80198010 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f3f70006010 tx=0x7f3f70003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:24.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.249+0000 7f3f7d7fa640 1 -- 172.21.15.107:0/4109330089 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7003f070 con 0x7f3f80100db0 2024-04-25T00:21:24.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.250+0000 7f3f8646b640 1 -- 172.21.15.107:0/4109330089 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8019d290 con 0x7f3f80100db0 2024-04-25T00:21:24.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.250+0000 7f3f7d7fa640 1 -- 172.21.15.107:0/4109330089 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f70031c30 con 0x7f3f80100db0 2024-04-25T00:21:24.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.250+0000 7f3f7d7fa640 1 -- 172.21.15.107:0/4109330089 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70035030 con 0x7f3f80100db0 2024-04-25T00:21:24.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.250+0000 7f3f8646b640 1 -- 172.21.15.107:0/4109330089 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f801a4b80 con 0x7f3f80100db0 2024-04-25T00:21:24.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.252+0000 7f3f89179740 1 Processor -- start 2024-04-25T00:21:24.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.252+0000 7f3f7d7fa640 1 -- 172.21.15.107:0/4109330089 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f3f70042070 con 0x7f3f80100db0 2024-04-25T00:21:24.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.252+0000 7f3f89179740 1 -- start start 2024-04-25T00:21:24.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.253+0000 7f3f89179740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590bbe45580 0x5590bbe45960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.253+0000 7f3f89179740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590bbe45ea0 0x5590bbe50030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.253+0000 7f3f89179740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe50570 0x5590bbe52960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.253+0000 7f3f89179740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5590bbdaa670 con 0x5590bbe50570 2024-04-25T00:21:24.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.253+0000 7f3f89179740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5590bbd52970 con 0x5590bbe45ea0 2024-04-25T00:21:24.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.253+0000 7f3f89179740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5590bbda6630 con 0x5590bbe45580 2024-04-25T00:21:24.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.253+0000 7f3f7f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe50570 0x5590bbe52960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.253+0000 7f3f7f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe50570 0x5590bbe52960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48744/0 (socket says 172.21.15.107:48744) 2024-04-25T00:21:24.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.253+0000 7f3f7f7fe640 1 -- 172.21.15.107:0/2660389887 learned_addr learned my addr 172.21.15.107:0/2660389887 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:24.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.253+0000 7f3f84a08640 1 --2- 172.21.15.107:0/2660389887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590bbe45580 0x5590bbe45960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.253+0000 7f3f7ffff640 1 --2- 172.21.15.107:0/2660389887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590bbe45ea0 0x5590bbe50030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.253+0000 7f3f7f7fe640 1 -- 172.21.15.107:0/2660389887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590bbe45580 msgr2=0x5590bbe45960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:24.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.253+0000 7f3f7d7fa640 1 --2- 172.21.15.107:0/4109330089 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f3f680a2d00 0x7f3f680a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.253+0000 7f3f7f7fe640 1 --2- 172.21.15.107:0/2660389887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590bbe45580 0x5590bbe45960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.253+0000 7f3f7f7fe640 1 -- 172.21.15.107:0/2660389887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590bbe45ea0 msgr2=0x5590bbe50030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:24.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.253+0000 7f3f7f7fe640 1 --2- 172.21.15.107:0/2660389887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590bbe45ea0 0x5590bbe50030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.253+0000 7f3f7f7fe640 1 -- 172.21.15.107:0/2660389887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590bbd76400 con 0x5590bbe50570 2024-04-25T00:21:24.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.253+0000 7f3f7ffff640 1 --2- 172.21.15.107:0/2660389887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590bbe45ea0 0x5590bbe50030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:24.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.253+0000 7f3f84a08640 1 --2- 172.21.15.107:0/2660389887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590bbe45580 0x5590bbe45960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:24.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.253+0000 7f3f84a08640 1 --2- 172.21.15.107:0/4109330089 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f3f680a2d00 0x7f3f680a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.253+0000 7f3f7d7fa640 1 -- 172.21.15.107:0/4109330089 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f3f700eff00 con 0x7f3f80100db0 2024-04-25T00:21:24.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.253+0000 7f3f7f7fe640 1 --2- 172.21.15.107:0/2660389887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe50570 0x5590bbe52960 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009c20 tx=0x7f3f740117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f1cbcafb2f8f02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:24.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.253+0000 7f3f667fc640 1 -- 172.21.15.107:0/2660389887 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74012450 con 0x5590bbe50570 2024-04-25T00:21:24.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.254+0000 7f3f667fc640 1 -- 172.21.15.107:0/2660389887 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f74012ad0 con 0x5590bbe50570 2024-04-25T00:21:24.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.254+0000 7f3f667fc640 1 -- 172.21.15.107:0/2660389887 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7400fbc0 con 0x5590bbe50570 2024-04-25T00:21:24.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.254+0000 7f3f89179740 1 -- 172.21.15.107:0/2660389887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe50570 msgr2=0x5590bbe52960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.254+0000 7f3f84a08640 1 --2- 172.21.15.107:0/4109330089 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f3f680a2d00 0x7f3f680a51c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800a640 tx=0x7f3f78004b50 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:24.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.254+0000 7f3f89179740 1 --2- 172.21.15.107:0/2660389887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe50570 0x5590bbe52960 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009c20 tx=0x7f3f740117b0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.254+0000 7f3f89179740 1 -- 172.21.15.107:0/2660389887 shutdown_connections 2024-04-25T00:21:24.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.254+0000 7f3f89179740 1 --2- 172.21.15.107:0/2660389887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe50570 0x5590bbe52960 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.254+0000 7f3f89179740 1 --2- 172.21.15.107:0/2660389887 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590bbe45ea0 0x5590bbe50030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.254+0000 7f3f89179740 1 --2- 172.21.15.107:0/2660389887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590bbe45580 0x5590bbe45960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.254+0000 7f3f89179740 1 -- 172.21.15.107:0/2660389887 >> 172.21.15.107:0/2660389887 conn(0x5590bbe418a0 msgr2=0x5590bbe41c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:24.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.254+0000 7f3f89179740 1 -- 172.21.15.107:0/2660389887 shutdown_connections 2024-04-25T00:21:24.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.254+0000 7f3f89179740 1 -- 172.21.15.107:0/2660389887 wait complete. 2024-04-25T00:21:24.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.255+0000 7f3f89179740 1 -- start start 2024-04-25T00:21:24.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.255+0000 7f3f89179740 1 Processor -- start 2024-04-25T00:21:24.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.255+0000 7f3f89179740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590bbe45580 0x5590bbe4fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.255+0000 7f3f89179740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590bbe45ea0 0x5590bbe4d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.255+0000 7f3f89179740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe50570 0x5590bbe4d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.255+0000 7f3f89179740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5590bbe53360 con 0x5590bbe50570 2024-04-25T00:21:24.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.255+0000 7f3f89179740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5590bbd66050 con 0x5590bbe45ea0 2024-04-25T00:21:24.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.255+0000 7f3f89179740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5590bbe534e0 con 0x5590bbe45580 2024-04-25T00:21:24.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.255+0000 7f3f89179740 10 client.0 _ll_register_callbacks cb 0x5590bbabdbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:24.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.255+0000 7f3f84a08640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe50570 0x5590bbe4d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.255+0000 7f3f84a08640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe50570 0x5590bbe4d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48760/0 (socket says 172.21.15.107:48760) 2024-04-25T00:21:24.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.255+0000 7f3f84a08640 1 -- 172.21.15.107:0/3158365566 learned_addr learned my addr 172.21.15.107:0/3158365566 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:24.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.255+0000 7f3f7f7fe640 1 --2- 172.21.15.107:0/3158365566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590bbe45ea0 0x5590bbe4d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.255+0000 7f3f7ffff640 1 --2- 172.21.15.107:0/3158365566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590bbe45580 0x5590bbe4fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.255+0000 7f3f84a08640 1 -- 172.21.15.107:0/3158365566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590bbe45580 msgr2=0x5590bbe4fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:24.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.255+0000 7f3f84a08640 1 --2- 172.21.15.107:0/3158365566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590bbe45580 0x5590bbe4fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.255+0000 7f3f84a08640 1 -- 172.21.15.107:0/3158365566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590bbe45ea0 msgr2=0x5590bbe4d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:24.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.255+0000 7f3f84a08640 1 --2- 172.21.15.107:0/3158365566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590bbe45ea0 0x5590bbe4d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.255+0000 7f3f84a08640 1 -- 172.21.15.107:0/3158365566 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590bbd37dc0 con 0x5590bbe50570 2024-04-25T00:21:24.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.256+0000 7f3f84a08640 1 --2- 172.21.15.107:0/3158365566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe50570 0x5590bbe4d5f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3f78007a80 tx=0x7f3f7800d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6db70bc864776b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:24.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.256+0000 7f3f5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:21:24.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.256+0000 7f3f5f7fe640 1 -- 172.21.15.107:0/3158365566 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78017b90 con 0x5590bbe50570 2024-04-25T00:21:24.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.256+0000 7f3f5f7fe640 1 -- 172.21.15.107:0/3158365566 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f3f78017d30 con 0x5590bbe50570 2024-04-25T00:21:24.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.256+0000 7f3f89179740 1 -- 172.21.15.107:0/3158365566 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f74012ad0 con 0x5590bbe50570 2024-04-25T00:21:24.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.256+0000 7f3f5f7fe640 1 -- 172.21.15.107:0/3158365566 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3f78031050 con 0x5590bbe50570 2024-04-25T00:21:24.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.256+0000 7f3f89179740 10 client.16686 fetch_fsmap learned FSMap version 29 2024-04-25T00:21:24.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.256+0000 7f3f89179740 1 -- 172.21.15.107:0/3158365566 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5590bbabe3f0 con 0x5590bbe50570 2024-04-25T00:21:24.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.256+0000 7f3f5f7fe640 1 -- 172.21.15.107:0/3158365566 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f74012ad0 con 0x5590bbe50570 2024-04-25T00:21:24.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.256+0000 7f3f5f7fe640 1 -- 172.21.15.107:0/3158365566 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f3f7802c5f0 con 0x5590bbe50570 2024-04-25T00:21:24.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.256+0000 7f3f89179740 10 client.16686 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:21:24.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.256+0000 7f3f89179740 10 client.16686 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:21:24.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.256+0000 7f3f89179740 20 client.16686 populate_metadata read hostname 'smithi107' 2024-04-25T00:21:24.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.256+0000 7f3f89179740 1 --2- 172.21.15.107:0/3158365566 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5590bbe605a0 0x5590bbe62a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.256+0000 7f3f89179740 4 client.16686 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:24.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.256+0000 7f3f89179740 1 -- 172.21.15.107:0/3158365566 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590bbdb7bb0 con 0x5590bbe605a0 2024-04-25T00:21:24.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.257+0000 7f3f7ffff640 1 --2- 172.21.15.107:0/3158365566 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5590bbe605a0 0x5590bbe62a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.257+0000 7f3f7ffff640 1 --2- 172.21.15.107:0/3158365566 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5590bbe605a0 0x5590bbe62a00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c87dd2a0d72a6209 server_cookie=c3ae94c8e69cf4aa in_seq=0 out_seq=0 2024-04-25T00:21:24.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.257+0000 7f3f5f7fe640 10 client.16686 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:21:24.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.258+0000 7f3f5f7fe640 1 -- 172.21.15.107:0/3158365566 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5590bbdb7bb0 con 0x5590bbe605a0 2024-04-25T00:21:24.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.258+0000 7f3f5f7fe640 10 client.16686 handle_command_reply: tid=0 2024-04-25T00:21:24.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.258+0000 7f3f89179740 1 client.16686 shutdown 2024-04-25T00:21:24.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.258+0000 7f3f89179740 1 -- 172.21.15.107:0/3158365566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe50570 msgr2=0x5590bbe4d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.258+0000 7f3f89179740 1 --2- 172.21.15.107:0/3158365566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe50570 0x5590bbe4d5f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f3f78007a80 tx=0x7f3f7800d040 comp rx=0 tx=0).stop 2024-04-25T00:21:24.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.258+0000 7f3f89179740 1 -- 172.21.15.107:0/3158365566 shutdown_connections 2024-04-25T00:21:24.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.258+0000 7f3f89179740 1 --2- 172.21.15.107:0/3158365566 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5590bbe605a0 0x5590bbe62a00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.258+0000 7f3f89179740 1 --2- 172.21.15.107:0/3158365566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe50570 0x5590bbe4d5f0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.258+0000 7f3f5f7fe640 0 client.16686 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:21:24.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.258+0000 7f3f89179740 1 --2- 172.21.15.107:0/3158365566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590bbe45ea0 0x5590bbe4d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.258+0000 7f3f89179740 1 --2- 172.21.15.107:0/3158365566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590bbe45580 0x5590bbe4fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.258+0000 7f3f89179740 1 -- 172.21.15.107:0/3158365566 >> 172.21.15.107:0/3158365566 conn(0x5590bbe418a0 msgr2=0x5590bbe40b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:24.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.258+0000 7f3f89179740 1 -- 172.21.15.107:0/3158365566 shutdown_connections 2024-04-25T00:21:24.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.258+0000 7f3f89179740 1 -- 172.21.15.107:0/3158365566 wait complete. 2024-04-25T00:21:24.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.258+0000 7f3f89179740 20 client.16686 trim_cache size 0 max 16384 2024-04-25T00:21:24.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.271+0000 7f3f89179740 1 Processor -- start 2024-04-25T00:21:24.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.271+0000 7f3f89179740 1 -- start start 2024-04-25T00:21:24.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.271+0000 7f3f89179740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590bbe45580 0x5590bbe5c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.271+0000 7f3f89179740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe45ea0 0x5590bbe682d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.271+0000 7f3f89179740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590bbe50570 0x5590bbe68810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.271+0000 7f3f89179740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801b14d0 con 0x5590bbe45ea0 2024-04-25T00:21:24.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.271+0000 7f3f89179740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801aed10 con 0x5590bbe45580 2024-04-25T00:21:24.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.271+0000 7f3f89179740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801acb60 con 0x5590bbe50570 2024-04-25T00:21:24.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.271+0000 7f3f7f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe45ea0 0x5590bbe682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.271+0000 7f3f7f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe45ea0 0x5590bbe682d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48770/0 (socket says 172.21.15.107:48770) 2024-04-25T00:21:24.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.271+0000 7f3f7f7fe640 1 -- 172.21.15.107:0/2023649792 learned_addr learned my addr 172.21.15.107:0/2023649792 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:24.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.272+0000 7f3f7f7fe640 1 -- 172.21.15.107:0/2023649792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590bbe50570 msgr2=0x5590bbe68810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:24.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.272+0000 7f3f7f7fe640 1 --2- 172.21.15.107:0/2023649792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590bbe50570 0x5590bbe68810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.272+0000 7f3f7f7fe640 1 -- 172.21.15.107:0/2023649792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590bbe45580 msgr2=0x5590bbe5c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:24.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.272+0000 7f3f7f7fe640 1 --2- 172.21.15.107:0/2023649792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590bbe45580 0x5590bbe5c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.272+0000 7f3f7f7fe640 1 -- 172.21.15.107:0/2023649792 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590bbb887d0 con 0x5590bbe45ea0 2024-04-25T00:21:24.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.272+0000 7f3f7f7fe640 1 --2- 172.21.15.107:0/2023649792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe45ea0 0x5590bbe682d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x5590bbe42a30 tx=0x7f3f740058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f8a3e089bd9769 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:24.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.272+0000 7f3f65ffb640 1 -- 172.21.15.107:0/2023649792 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74005030 con 0x5590bbe45ea0 2024-04-25T00:21:24.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.272+0000 7f3f65ffb640 1 -- 172.21.15.107:0/2023649792 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f74004680 con 0x5590bbe45ea0 2024-04-25T00:21:24.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.272+0000 7f3f65ffb640 1 -- 172.21.15.107:0/2023649792 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f740266d0 con 0x5590bbe45ea0 2024-04-25T00:21:24.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.272+0000 7f3f89179740 1 -- 172.21.15.107:0/2023649792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe45ea0 msgr2=0x5590bbe682d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.272+0000 7f3f89179740 1 --2- 172.21.15.107:0/2023649792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe45ea0 0x5590bbe682d0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x5590bbe42a30 tx=0x7f3f740058f0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.273+0000 7f3f89179740 1 -- 172.21.15.107:0/2023649792 shutdown_connections 2024-04-25T00:21:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.273+0000 7f3f89179740 1 --2- 172.21.15.107:0/2023649792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590bbe50570 0x5590bbe68810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.273+0000 7f3f89179740 1 --2- 172.21.15.107:0/2023649792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe45ea0 0x5590bbe682d0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.273+0000 7f3f89179740 1 --2- 172.21.15.107:0/2023649792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590bbe45580 0x5590bbe5c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.273+0000 7f3f89179740 1 -- 172.21.15.107:0/2023649792 >> 172.21.15.107:0/2023649792 conn(0x5590bbe418a0 msgr2=0x5590bbe402b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.273+0000 7f3f89179740 1 -- 172.21.15.107:0/2023649792 shutdown_connections 2024-04-25T00:21:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.273+0000 7f3f89179740 1 -- 172.21.15.107:0/2023649792 wait complete. 2024-04-25T00:21:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.273+0000 7f3f89179740 1 -- start start 2024-04-25T00:21:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.273+0000 7f3f89179740 1 Processor -- start 2024-04-25T00:21:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.273+0000 7f3f89179740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe45580 0x5590bbe58c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.273+0000 7f3f89179740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590bbe45ea0 0x5590bbe56000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.273+0000 7f3f89179740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590bbe50570 0x5590bbe56740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.273+0000 7f3f89179740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801ad1d0 con 0x5590bbe45580 2024-04-25T00:21:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.273+0000 7f3f89179740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801ae640 con 0x5590bbe50570 2024-04-25T00:21:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.273+0000 7f3f89179740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801ad8a0 con 0x5590bbe45ea0 2024-04-25T00:21:24.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.273+0000 7f3f89179740 10 client.0 _ll_register_callbacks cb 0x7f3f78007780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.273+0000 7f3f7ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe45580 0x5590bbe58c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.273+0000 7f3f7ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe45580 0x5590bbe58c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48780/0 (socket says 172.21.15.107:48780) 2024-04-25T00:21:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.273+0000 7f3f7ffff640 1 -- 172.21.15.107:0/1251685180 learned_addr learned my addr 172.21.15.107:0/1251685180 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.274+0000 7f3f84a08640 1 --2- 172.21.15.107:0/1251685180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590bbe50570 0x5590bbe56740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.274+0000 7f3f7ffff640 1 -- 172.21.15.107:0/1251685180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590bbe45ea0 msgr2=0x5590bbe56000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.274+0000 7f3f7ffff640 1 --2- 172.21.15.107:0/1251685180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590bbe45ea0 0x5590bbe56000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.274+0000 7f3f7ffff640 1 -- 172.21.15.107:0/1251685180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590bbe50570 msgr2=0x5590bbe56740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.274+0000 7f3f7ffff640 1 --2- 172.21.15.107:0/1251685180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590bbe50570 0x5590bbe56740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.274+0000 7f3f7ffff640 1 -- 172.21.15.107:0/1251685180 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590bbec5b50 con 0x5590bbe45580 2024-04-25T00:21:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.274+0000 7f3f84a08640 1 --2- 172.21.15.107:0/1251685180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590bbe50570 0x5590bbe56740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.274+0000 7f3f7ffff640 1 --2- 172.21.15.107:0/1251685180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe45580 0x5590bbe58c70 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f3f700ed8e0 tx=0x7f3f70003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10ed38a248c7ce1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.274+0000 7f3f5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:21:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.274+0000 7f3f5f7fe640 1 -- 172.21.15.107:0/1251685180 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f70049040 con 0x5590bbe45580 2024-04-25T00:21:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.274+0000 7f3f5f7fe640 1 -- 172.21.15.107:0/1251685180 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f3f700dc0b0 con 0x5590bbe45580 2024-04-25T00:21:24.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.274+0000 7f3f89179740 1 -- 172.21.15.107:0/1251685180 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f74004680 con 0x5590bbe45580 2024-04-25T00:21:24.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.274+0000 7f3f5f7fe640 1 -- 172.21.15.107:0/1251685180 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3f700bc7b0 con 0x5590bbe45580 2024-04-25T00:21:24.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.274+0000 7f3f89179740 10 client.16692 fetch_fsmap learned FSMap version 29 2024-04-25T00:21:24.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.274+0000 7f3f89179740 1 -- 172.21.15.107:0/1251685180 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5590bbec6d50 con 0x5590bbe45580 2024-04-25T00:21:24.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.274+0000 7f3f5f7fe640 1 -- 172.21.15.107:0/1251685180 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f74004680 con 0x5590bbe45580 2024-04-25T00:21:24.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.274+0000 7f3f5f7fe640 1 -- 172.21.15.107:0/1251685180 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f3f700ed560 con 0x5590bbe45580 2024-04-25T00:21:24.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.274+0000 7f3f89179740 10 client.16692 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:21:24.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.274+0000 7f3f89179740 10 client.16692 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:21:24.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.274+0000 7f3f89179740 20 client.16692 populate_metadata read hostname 'smithi107' 2024-04-25T00:21:24.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.274+0000 7f3f89179740 1 --2- 172.21.15.107:0/1251685180 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5590bbec7500 0x5590bbec98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.274+0000 7f3f89179740 4 client.16692 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:21:24.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.274+0000 7f3f89179740 1 -- 172.21.15.107:0/1251685180 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5590bbdb1f00 con 0x5590bbec7500 2024-04-25T00:21:24.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.275+0000 7f3f7f7fe640 1 --2- 172.21.15.107:0/1251685180 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5590bbec7500 0x5590bbec98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.275+0000 7f3f7f7fe640 1 --2- 172.21.15.107:0/1251685180 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5590bbec7500 0x5590bbec98f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de99b43fffae4f52 server_cookie=adb86f9e98711eb3 in_seq=0 out_seq=0 2024-04-25T00:21:24.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.275+0000 7f3f5f7fe640 10 client.16692 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:21:24.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.275+0000 7f3f5f7fe640 1 -- 172.21.15.107:0/1251685180 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5590bbdb1f00 con 0x5590bbec7500 2024-04-25T00:21:24.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.275+0000 7f3f5f7fe640 10 client.16692 handle_command_reply: tid=0 2024-04-25T00:21:24.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.275+0000 7f3f89179740 1 client.16692 shutdown 2024-04-25T00:21:24.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.275+0000 7f3f89179740 1 -- 172.21.15.107:0/1251685180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe45580 msgr2=0x5590bbe58c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.275+0000 7f3f89179740 1 --2- 172.21.15.107:0/1251685180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe45580 0x5590bbe58c70 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f3f700ed8e0 tx=0x7f3f70003d80 comp rx=0 tx=0).stop 2024-04-25T00:21:24.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.275+0000 7f3f89179740 1 -- 172.21.15.107:0/1251685180 shutdown_connections 2024-04-25T00:21:24.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.275+0000 7f3f89179740 1 --2- 172.21.15.107:0/1251685180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5590bbe50570 0x5590bbe56740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.275+0000 7f3f89179740 1 --2- 172.21.15.107:0/1251685180 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5590bbe45ea0 0x5590bbe56000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.275+0000 7f3f89179740 1 --2- 172.21.15.107:0/1251685180 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5590bbec7500 0x5590bbec98f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.275+0000 7f3f89179740 1 --2- 172.21.15.107:0/1251685180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590bbe45580 0x5590bbe58c70 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.275+0000 7f3f5f7fe640 0 client.16692 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:21:24.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.275+0000 7f3f89179740 1 -- 172.21.15.107:0/1251685180 >> 172.21.15.107:0/1251685180 conn(0x5590bbe418a0 msgr2=0x5590bbe40b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:24.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.275+0000 7f3f89179740 1 -- 172.21.15.107:0/1251685180 shutdown_connections 2024-04-25T00:21:24.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.275+0000 7f3f89179740 1 -- 172.21.15.107:0/1251685180 wait complete. 2024-04-25T00:21:24.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.276+0000 7f3f89179740 20 client.16692 trim_cache size 0 max 16384 2024-04-25T00:21:24.284 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:21:24.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.276+0000 7f3f8646b640 1 -- 172.21.15.107:0/4109330089 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f3f680a2d00 msgr2=0x7f3f680a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.276+0000 7f3f8646b640 1 --2- 172.21.15.107:0/4109330089 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f3f680a2d00 0x7f3f680a51c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800a640 tx=0x7f3f78004b50 comp rx=0 tx=0).stop 2024-04-25T00:21:24.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.276+0000 7f3f8646b640 1 -- 172.21.15.107:0/4109330089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f80100db0 msgr2=0x7f3f80198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.276+0000 7f3f8646b640 1 --2- 172.21.15.107:0/4109330089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f80100db0 0x7f3f80198010 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f3f70006010 tx=0x7f3f70003c10 comp rx=0 tx=0).stop 2024-04-25T00:21:24.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.276+0000 7f3f8646b640 1 -- 172.21.15.107:0/4109330089 shutdown_connections 2024-04-25T00:21:24.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.276+0000 7f3f8646b640 1 --2- 172.21.15.107:0/4109330089 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f3f680a2d00 0x7f3f680a51c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.276+0000 7f3f8646b640 1 --2- 172.21.15.107:0/4109330089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3f801055f0 0x7f3f8019c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.276+0000 7f3f8646b640 1 --2- 172.21.15.107:0/4109330089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3f80101770 0x7f3f80198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.276+0000 7f3f8646b640 1 --2- 172.21.15.107:0/4109330089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f80100db0 0x7f3f80198010 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.276+0000 7f3f8646b640 1 -- 172.21.15.107:0/4109330089 >> 172.21.15.107:0/4109330089 conn(0x7f3f80076c00 msgr2=0x7f3f80103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:24.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.276+0000 7f3f8646b640 1 -- 172.21.15.107:0/4109330089 shutdown_connections 2024-04-25T00:21:24.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.276+0000 7f3f8646b640 1 -- 172.21.15.107:0/4109330089 wait complete. 2024-04-25T00:21:24.288 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:21:24.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.387+0000 7f61b11b0640 1 -- 172.21.15.107:0/3920084108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61ac101330 msgr2=0x7f61ac10d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.387+0000 7f61b11b0640 1 --2- 172.21.15.107:0/3920084108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61ac101330 0x7f61ac10d460 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f61a4009ca0 tx=0x7f61a402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.387+0000 7f61b11b0640 1 -- 172.21.15.107:0/3920084108 shutdown_connections 2024-04-25T00:21:24.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.387+0000 7f61b11b0640 1 --2- 172.21.15.107:0/3920084108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61ac101330 0x7f61ac10d460 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.387+0000 7f61b11b0640 1 --2- 172.21.15.107:0/3920084108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61ac100990 0x7f61ac100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.387+0000 7f61b11b0640 1 --2- 172.21.15.107:0/3920084108 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61ac106690 0x7f61ac106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.387+0000 7f61b11b0640 1 -- 172.21.15.107:0/3920084108 >> 172.21.15.107:0/3920084108 conn(0x7f61ac076440 msgr2=0x7f61ac076850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:24.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.387+0000 7f61b11b0640 1 -- 172.21.15.107:0/3920084108 shutdown_connections 2024-04-25T00:21:24.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.387+0000 7f61b11b0640 1 -- 172.21.15.107:0/3920084108 wait complete. 2024-04-25T00:21:24.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.387+0000 7f61b11b0640 1 Processor -- start 2024-04-25T00:21:24.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.388+0000 7f61b11b0640 1 -- start start 2024-04-25T00:21:24.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.388+0000 7f61b11b0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61ac100990 0x7f61ac196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.388+0000 7f61b11b0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61ac101330 0x7f61ac197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.388+0000 7f61b11b0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61ac106690 0x7f61ac19b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.388+0000 7f61b11b0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac110a00 con 0x7f61ac106690 2024-04-25T00:21:24.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.388+0000 7f61b11b0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac110880 con 0x7f61ac100990 2024-04-25T00:21:24.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.388+0000 7f61b11b0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac110b80 con 0x7f61ac101330 2024-04-25T00:21:24.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.388+0000 7f61ab59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61ac106690 0x7f61ac19b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.388+0000 7f61ab59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61ac106690 0x7f61ac19b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48798/0 (socket says 172.21.15.107:48798) 2024-04-25T00:21:24.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.388+0000 7f61ab59e640 1 -- 172.21.15.107:0/408351464 learned_addr learned my addr 172.21.15.107:0/408351464 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:24.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.388+0000 7f61aad9d640 1 --2- 172.21.15.107:0/408351464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61ac100990 0x7f61ac196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.388+0000 7f61aa59c640 1 --2- 172.21.15.107:0/408351464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61ac101330 0x7f61ac197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.388+0000 7f61ab59e640 1 -- 172.21.15.107:0/408351464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61ac101330 msgr2=0x7f61ac197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.388+0000 7f61ab59e640 1 --2- 172.21.15.107:0/408351464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61ac101330 0x7f61ac197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.388+0000 7f61ab59e640 1 -- 172.21.15.107:0/408351464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61ac100990 msgr2=0x7f61ac196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.388+0000 7f61ab59e640 1 --2- 172.21.15.107:0/408351464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61ac100990 0x7f61ac196e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.388+0000 7f61ab59e640 1 -- 172.21.15.107:0/408351464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ac19bde0 con 0x7f61ac106690 2024-04-25T00:21:24.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.388+0000 7f61ab59e640 1 --2- 172.21.15.107:0/408351464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61ac106690 0x7f61ac19b6b0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f61a4031a60 tx=0x7f61a4031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:24.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.388+0000 7f6193fff640 1 -- 172.21.15.107:0/408351464 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a4003b70 con 0x7f61ac106690 2024-04-25T00:21:24.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.388+0000 7f61b11b0640 1 -- 172.21.15.107:0/408351464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61ac19c070 con 0x7f61ac106690 2024-04-25T00:21:24.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.389+0000 7f6193fff640 1 -- 172.21.15.107:0/408351464 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61a4003d10 con 0x7f61ac106690 2024-04-25T00:21:24.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.389+0000 7f6193fff640 1 -- 172.21.15.107:0/408351464 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a4004770 con 0x7f61ac106690 2024-04-25T00:21:24.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.389+0000 7f61b11b0640 1 -- 172.21.15.107:0/408351464 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61ac1a3960 con 0x7f61ac106690 2024-04-25T00:21:24.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.391+0000 7f6193fff640 1 -- 172.21.15.107:0/408351464 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f61a402f790 con 0x7f61ac106690 2024-04-25T00:21:24.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.391+0000 7f61b3f9e740 1 Processor -- start 2024-04-25T00:21:24.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.391+0000 7f61b3f9e740 1 -- start start 2024-04-25T00:21:24.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.392+0000 7f61b3f9e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563208237580 0x563208237960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.392+0000 7f61b3f9e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563208237ea0 0x563208242030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.392+0000 7f61b3f9e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563208242570 0x563208244960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.392+0000 7f61b3f9e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56320819c670 con 0x563208242570 2024-04-25T00:21:24.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.392+0000 7f61b3f9e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563208144970 con 0x563208237ea0 2024-04-25T00:21:24.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.392+0000 7f61b3f9e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563208198630 con 0x563208237580 2024-04-25T00:21:24.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.392+0000 7f61aa59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563208242570 0x563208244960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.392+0000 7f6193fff640 1 --2- 172.21.15.107:0/408351464 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f61940a2d00 0x7f61940a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.392+0000 7f61aa59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563208242570 0x563208244960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48810/0 (socket says 172.21.15.107:48810) 2024-04-25T00:21:24.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.392+0000 7f61aa59c640 1 -- 172.21.15.107:0/3362832309 learned_addr learned my addr 172.21.15.107:0/3362832309 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:24.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.392+0000 7f61aa59c640 1 --2- 172.21.15.107:0/3362832309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563208237580 0x563208237960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.392+0000 7f61aad9d640 1 --2- 172.21.15.107:0/3362832309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563208237ea0 0x563208242030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.392+0000 7f61ab59e640 1 --2- 172.21.15.107:0/408351464 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f61940a2d00 0x7f61940a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.392+0000 7f61aa59c640 1 -- 172.21.15.107:0/3362832309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563208237580 msgr2=0x563208237960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:24.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.392+0000 7f61aa59c640 1 --2- 172.21.15.107:0/3362832309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563208237580 0x563208237960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.392+0000 7f61aa59c640 1 -- 172.21.15.107:0/3362832309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563208237ea0 msgr2=0x563208242030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:24.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.392+0000 7f61aa59c640 1 --2- 172.21.15.107:0/3362832309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563208237ea0 0x563208242030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.392+0000 7f6193fff640 1 -- 172.21.15.107:0/408351464 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f61a40f0a00 con 0x7f61ac106690 2024-04-25T00:21:24.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.392+0000 7f61aa59c640 1 -- 172.21.15.107:0/3362832309 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563208168400 con 0x563208242570 2024-04-25T00:21:24.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.392+0000 7f61aa59c640 1 --2- 172.21.15.107:0/3362832309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563208242570 0x563208244960 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f61a00199d0 tx=0x7f61a000b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d1175e6b66da65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:24.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.392+0000 7f61ab59e640 1 --2- 172.21.15.107:0/408351464 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f61940a2d00 0x7f61940a51c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f61a40b9120 tx=0x7f61a40b8840 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:24.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.394+0000 7f61917fa640 1 -- 172.21.15.107:0/3362832309 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a000eb60 con 0x563208242570 2024-04-25T00:21:24.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.394+0000 7f61917fa640 1 -- 172.21.15.107:0/3362832309 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61a000ed00 con 0x563208242570 2024-04-25T00:21:24.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.394+0000 7f61917fa640 1 -- 172.21.15.107:0/3362832309 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a0010690 con 0x563208242570 2024-04-25T00:21:24.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.395+0000 7f61b3f9e740 1 -- 172.21.15.107:0/3362832309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563208242570 msgr2=0x563208244960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.395+0000 7f61b3f9e740 1 --2- 172.21.15.107:0/3362832309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563208242570 0x563208244960 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f61a00199d0 tx=0x7f61a000b460 comp rx=0 tx=0).stop 2024-04-25T00:21:24.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.395+0000 7f61b3f9e740 1 -- 172.21.15.107:0/3362832309 shutdown_connections 2024-04-25T00:21:24.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.395+0000 7f61b3f9e740 1 --2- 172.21.15.107:0/3362832309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563208242570 0x563208244960 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.395+0000 7f61b3f9e740 1 --2- 172.21.15.107:0/3362832309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563208237ea0 0x563208242030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.395+0000 7f61b3f9e740 1 --2- 172.21.15.107:0/3362832309 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563208237580 0x563208237960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.395+0000 7f61b3f9e740 1 -- 172.21.15.107:0/3362832309 >> 172.21.15.107:0/3362832309 conn(0x5632082338a0 msgr2=0x563208233c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:24.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.395+0000 7f61b3f9e740 1 -- 172.21.15.107:0/3362832309 shutdown_connections 2024-04-25T00:21:24.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.395+0000 7f61b3f9e740 1 -- 172.21.15.107:0/3362832309 wait complete. 2024-04-25T00:21:24.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.395+0000 7f61b3f9e740 1 -- start start 2024-04-25T00:21:24.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.395+0000 7f61b3f9e740 1 Processor -- start 2024-04-25T00:21:24.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.395+0000 7f61b3f9e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563208237580 0x563208241e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.395+0000 7f61b3f9e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563208237ea0 0x56320823f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.395+0000 7f61b3f9e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563208242570 0x56320823f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.395+0000 7f61b3f9e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563208245360 con 0x563208237580 2024-04-25T00:21:24.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.395+0000 7f61b3f9e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563208158050 con 0x563208237ea0 2024-04-25T00:21:24.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.395+0000 7f61b3f9e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5632082454e0 con 0x563208242570 2024-04-25T00:21:24.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.396+0000 7f61b3f9e740 10 client.0 _ll_register_callbacks cb 0x563207eafbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:24.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.396+0000 7f61aa59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563208237580 0x563208241e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.396+0000 7f61aa59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563208237580 0x563208241e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48812/0 (socket says 172.21.15.107:48812) 2024-04-25T00:21:24.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.396+0000 7f61aa59c640 1 -- 172.21.15.107:0/2234618412 learned_addr learned my addr 172.21.15.107:0/2234618412 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:24.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.396+0000 7f61aad9d640 1 --2- 172.21.15.107:0/2234618412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563208237ea0 0x56320823f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.396+0000 7f61aa59c640 1 --2- 172.21.15.107:0/2234618412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563208242570 0x56320823f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.396+0000 7f61aa59c640 1 -- 172.21.15.107:0/2234618412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563208242570 msgr2=0x56320823f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:24.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.396+0000 7f61aa59c640 1 --2- 172.21.15.107:0/2234618412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563208242570 0x56320823f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.396+0000 7f61aa59c640 1 -- 172.21.15.107:0/2234618412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563208237ea0 msgr2=0x56320823f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:24.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.396+0000 7f61aa59c640 1 --2- 172.21.15.107:0/2234618412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563208237ea0 0x56320823f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.396+0000 7f61aa59c640 1 -- 172.21.15.107:0/2234618412 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563208129dc0 con 0x563208237580 2024-04-25T00:21:24.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.396+0000 7f61aad9d640 1 --2- 172.21.15.107:0/2234618412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563208237ea0 0x56320823f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:24.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.396+0000 7f61aa59c640 1 --2- 172.21.15.107:0/2234618412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563208237580 0x563208241e80 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f61a0001eb0 tx=0x7f61a0011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bef5da1501b75f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:24.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.396+0000 7f618eff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:21:24.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.400+0000 7f618eff5640 1 -- 172.21.15.107:0/2234618412 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a002d020 con 0x563208237580 2024-04-25T00:21:24.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.400+0000 7f618eff5640 1 -- 172.21.15.107:0/2234618412 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f61a0011650 con 0x563208237580 2024-04-25T00:21:24.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.400+0000 7f61b3f9e740 1 -- 172.21.15.107:0/2234618412 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61a000ed00 con 0x563208237580 2024-04-25T00:21:24.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.400+0000 7f618eff5640 1 -- 172.21.15.107:0/2234618412 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f61a0038050 con 0x563208237580 2024-04-25T00:21:24.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.400+0000 7f61b3f9e740 10 client.16704 fetch_fsmap learned FSMap version 29 2024-04-25T00:21:24.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.400+0000 7f61b3f9e740 1 -- 172.21.15.107:0/2234618412 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563207eb03f0 con 0x563208237580 2024-04-25T00:21:24.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.400+0000 7f618eff5640 1 -- 172.21.15.107:0/2234618412 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61a000ed00 con 0x563208237580 2024-04-25T00:21:24.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.400+0000 7f618eff5640 1 -- 172.21.15.107:0/2234618412 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f61a0004860 con 0x563208237580 2024-04-25T00:21:24.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.400+0000 7f61b3f9e740 10 client.16704 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:21:24.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.400+0000 7f61b3f9e740 10 client.16704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:24.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.400+0000 7f61b3f9e740 20 client.16704 populate_metadata read hostname 'smithi107' 2024-04-25T00:21:24.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.400+0000 7f61b3f9e740 1 --2- 172.21.15.107:0/2234618412 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5632082524e0 0x5632082548d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.400+0000 7f61b3f9e740 4 client.16704 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:24.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.400+0000 7f61b3f9e740 1 -- 172.21.15.107:0/2234618412 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632081a9bb0 con 0x5632082524e0 2024-04-25T00:21:24.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.401+0000 7f61aad9d640 1 --2- 172.21.15.107:0/2234618412 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5632082524e0 0x5632082548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.401+0000 7f61aad9d640 1 --2- 172.21.15.107:0/2234618412 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5632082524e0 0x5632082548d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=771675236a70c929 server_cookie=9c2fe3a8275973f6 in_seq=0 out_seq=0 2024-04-25T00:21:24.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.401+0000 7f618eff5640 10 client.16704 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:24.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.402+0000 7f618eff5640 1 -- 172.21.15.107:0/2234618412 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5632081a9bb0 con 0x5632082524e0 2024-04-25T00:21:24.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.402+0000 7f618eff5640 10 client.16704 handle_command_reply: tid=0 2024-04-25T00:21:24.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.402+0000 7f61b3f9e740 1 client.16704 shutdown 2024-04-25T00:21:24.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.402+0000 7f61b3f9e740 1 -- 172.21.15.107:0/2234618412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563208237580 msgr2=0x563208241e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.402+0000 7f61b3f9e740 1 --2- 172.21.15.107:0/2234618412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563208237580 0x563208241e80 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f61a0001eb0 tx=0x7f61a0011dd0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.402+0000 7f61b3f9e740 1 -- 172.21.15.107:0/2234618412 shutdown_connections 2024-04-25T00:21:24.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.402+0000 7f61b3f9e740 1 --2- 172.21.15.107:0/2234618412 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5632082524e0 0x5632082548d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.402+0000 7f61b3f9e740 1 --2- 172.21.15.107:0/2234618412 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563208242570 0x56320823f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.402+0000 7f61b3f9e740 1 --2- 172.21.15.107:0/2234618412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563208237ea0 0x56320823f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.402+0000 7f618eff5640 0 client.16704 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:24.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.402+0000 7f61b3f9e740 1 --2- 172.21.15.107:0/2234618412 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563208237580 0x563208241e80 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.402+0000 7f61b3f9e740 1 -- 172.21.15.107:0/2234618412 >> 172.21.15.107:0/2234618412 conn(0x5632082338a0 msgr2=0x5632082396a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:24.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.402+0000 7f61b3f9e740 1 -- 172.21.15.107:0/2234618412 shutdown_connections 2024-04-25T00:21:24.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.402+0000 7f61b3f9e740 1 -- 172.21.15.107:0/2234618412 wait complete. 2024-04-25T00:21:24.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.402+0000 7f61b3f9e740 20 client.16704 trim_cache size 0 max 16384 2024-04-25T00:21:24.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.415+0000 7f61b3f9e740 1 Processor -- start 2024-04-25T00:21:24.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.415+0000 7f61b3f9e740 1 -- start start 2024-04-25T00:21:24.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.415+0000 7f61b3f9e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632082338a0 0x56320824e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.415+0000 7f61b3f9e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563208237580 0x5632082329c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.415+0000 7f61b3f9e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563208237ea0 0x563208259fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.415+0000 7f61b3f9e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac1af1b0 con 0x5632082338a0 2024-04-25T00:21:24.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.415+0000 7f61b3f9e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac1ac9f0 con 0x563208237580 2024-04-25T00:21:24.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.415+0000 7f61b3f9e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac1aa840 con 0x563208237ea0 2024-04-25T00:21:24.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.415+0000 7f61aa59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632082338a0 0x56320824e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.415+0000 7f61aa59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632082338a0 0x56320824e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48824/0 (socket says 172.21.15.107:48824) 2024-04-25T00:21:24.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.415+0000 7f61aa59c640 1 -- 172.21.15.107:0/1296519626 learned_addr learned my addr 172.21.15.107:0/1296519626 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:24.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.415+0000 7f61aad9d640 1 --2- 172.21.15.107:0/1296519626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563208237580 0x5632082329c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.416+0000 7f61aa59c640 1 -- 172.21.15.107:0/1296519626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563208237ea0 msgr2=0x563208259fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:24.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.416+0000 7f61aa59c640 1 --2- 172.21.15.107:0/1296519626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563208237ea0 0x563208259fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.416+0000 7f61aa59c640 1 -- 172.21.15.107:0/1296519626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563208237580 msgr2=0x5632082329c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:24.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.416+0000 7f61aa59c640 1 --2- 172.21.15.107:0/1296519626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563208237580 0x5632082329c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.416+0000 7f61aa59c640 1 -- 172.21.15.107:0/1296519626 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563207f7a7d0 con 0x5632082338a0 2024-04-25T00:21:24.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.416+0000 7f61aa59c640 1 --2- 172.21.15.107:0/1296519626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632082338a0 0x56320824e250 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f61a0004c30 tx=0x7f61a0005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dac1a18451a8515 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:24.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.416+0000 7f6190ff9640 1 -- 172.21.15.107:0/1296519626 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a002d040 con 0x5632082338a0 2024-04-25T00:21:24.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.416+0000 7f6190ff9640 1 -- 172.21.15.107:0/1296519626 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61a0023440 con 0x5632082338a0 2024-04-25T00:21:24.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.416+0000 7f6190ff9640 1 -- 172.21.15.107:0/1296519626 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a0005040 con 0x5632082338a0 2024-04-25T00:21:24.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.416+0000 7f61b3f9e740 1 -- 172.21.15.107:0/1296519626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632082338a0 msgr2=0x56320824e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.416+0000 7f61b3f9e740 1 --2- 172.21.15.107:0/1296519626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632082338a0 0x56320824e250 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f61a0004c30 tx=0x7f61a0005b10 comp rx=0 tx=0).stop 2024-04-25T00:21:24.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.416+0000 7f61b3f9e740 1 -- 172.21.15.107:0/1296519626 shutdown_connections 2024-04-25T00:21:24.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.416+0000 7f61b3f9e740 1 --2- 172.21.15.107:0/1296519626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563208237ea0 0x563208259fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.416+0000 7f61b3f9e740 1 --2- 172.21.15.107:0/1296519626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563208237580 0x5632082329c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.416+0000 7f61b3f9e740 1 --2- 172.21.15.107:0/1296519626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632082338a0 0x56320824e250 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.416+0000 7f61b3f9e740 1 -- 172.21.15.107:0/1296519626 >> 172.21.15.107:0/1296519626 conn(0x56320823f810 msgr2=0x5632082322b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:24.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.417+0000 7f61b3f9e740 1 -- 172.21.15.107:0/1296519626 shutdown_connections 2024-04-25T00:21:24.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.417+0000 7f61b3f9e740 1 -- 172.21.15.107:0/1296519626 wait complete. 2024-04-25T00:21:24.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.417+0000 7f61b3f9e740 1 -- start start 2024-04-25T00:21:24.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.417+0000 7f61b3f9e740 1 Processor -- start 2024-04-25T00:21:24.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.417+0000 7f61b3f9e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632082338a0 0x56320824a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.417+0000 7f61b3f9e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563208237580 0x56320824ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.417+0000 7f61b3f9e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563208237ea0 0x563208247d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.417+0000 7f61b3f9e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac1aaeb0 con 0x5632082338a0 2024-04-25T00:21:24.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.417+0000 7f61b3f9e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac1ac320 con 0x563208237580 2024-04-25T00:21:24.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.417+0000 7f61b3f9e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac1ab580 con 0x563208237ea0 2024-04-25T00:21:24.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.417+0000 7f61b3f9e740 10 client.0 _ll_register_callbacks cb 0x7f61a0019f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:24.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.417+0000 7f61aa59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632082338a0 0x56320824a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.417+0000 7f61aa59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632082338a0 0x56320824a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48832/0 (socket says 172.21.15.107:48832) 2024-04-25T00:21:24.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.417+0000 7f61aa59c640 1 -- 172.21.15.107:0/2533731692 learned_addr learned my addr 172.21.15.107:0/2533731692 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:24.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.417+0000 7f61aad9d640 1 --2- 172.21.15.107:0/2533731692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563208237580 0x56320824ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.417+0000 7f61aa59c640 1 --2- 172.21.15.107:0/2533731692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563208237ea0 0x563208247d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.417+0000 7f61aa59c640 1 -- 172.21.15.107:0/2533731692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563208237ea0 msgr2=0x563208247d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:24.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.417+0000 7f61aa59c640 1 --2- 172.21.15.107:0/2533731692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563208237ea0 0x563208247d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.417+0000 7f61aa59c640 1 -- 172.21.15.107:0/2533731692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563208237580 msgr2=0x56320824ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:24.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.417+0000 7f61aa59c640 1 --2- 172.21.15.107:0/2533731692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563208237580 0x56320824ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.417+0000 7f61aa59c640 1 -- 172.21.15.107:0/2533731692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563208248530 con 0x5632082338a0 2024-04-25T00:21:24.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.417+0000 7f61aad9d640 1 --2- 172.21.15.107:0/2533731692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563208237580 0x56320824ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:24.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.418+0000 7f61aa59c640 1 --2- 172.21.15.107:0/2533731692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632082338a0 0x56320824a7a0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f61a0004da0 tx=0x7f61a0011760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df43c5583796ca96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:24.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.418+0000 7f618eff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:21:24.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.418+0000 7f618eff5640 1 -- 172.21.15.107:0/2533731692 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a002d050 con 0x5632082338a0 2024-04-25T00:21:24.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.418+0000 7f618eff5640 1 -- 172.21.15.107:0/2533731692 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f61a0019030 con 0x5632082338a0 2024-04-25T00:21:24.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.418+0000 7f61b3f9e740 1 -- 172.21.15.107:0/2533731692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61a0023440 con 0x5632082338a0 2024-04-25T00:21:24.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.418+0000 7f618eff5640 1 -- 172.21.15.107:0/2533731692 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f61a000be70 con 0x5632082338a0 2024-04-25T00:21:24.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.418+0000 7f61b3f9e740 10 client.16710 fetch_fsmap learned FSMap version 29 2024-04-25T00:21:24.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.418+0000 7f61b3f9e740 1 -- 172.21.15.107:0/2533731692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5632082b84d0 con 0x5632082338a0 2024-04-25T00:21:24.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.418+0000 7f618eff5640 1 -- 172.21.15.107:0/2533731692 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61a0023440 con 0x5632082338a0 2024-04-25T00:21:24.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.418+0000 7f618eff5640 1 -- 172.21.15.107:0/2533731692 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f61a00054c0 con 0x5632082338a0 2024-04-25T00:21:24.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.418+0000 7f61b3f9e740 10 client.16710 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:21:24.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.418+0000 7f61b3f9e740 10 client.16710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:24.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.418+0000 7f61b3f9e740 20 client.16710 populate_metadata read hostname 'smithi107' 2024-04-25T00:21:24.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.418+0000 7f61b3f9e740 1 --2- 172.21.15.107:0/2533731692 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x563208242570 0x5632082baa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:24.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.418+0000 7f61b3f9e740 4 client.16710 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:21:24.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.418+0000 7f61b3f9e740 1 -- 172.21.15.107:0/2533731692 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5632081a3f00 con 0x563208242570 2024-04-25T00:21:24.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.418+0000 7f61aad9d640 1 --2- 172.21.15.107:0/2533731692 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x563208242570 0x5632082baa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:24.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.419+0000 7f618eff5640 10 client.16710 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:24.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.419+0000 7f61aad9d640 1 --2- 172.21.15.107:0/2533731692 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x563208242570 0x5632082baa40 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60e48ffbf7c0fcd4 server_cookie=78796ee0f6db42c in_seq=0 out_seq=0 2024-04-25T00:21:24.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.419+0000 7f618eff5640 1 -- 172.21.15.107:0/2533731692 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5632081a3f00 con 0x563208242570 2024-04-25T00:21:24.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.419+0000 7f618eff5640 10 client.16710 handle_command_reply: tid=0 2024-04-25T00:21:24.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.419+0000 7f61b3f9e740 1 client.16710 shutdown 2024-04-25T00:21:24.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.419+0000 7f61b3f9e740 1 -- 172.21.15.107:0/2533731692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632082338a0 msgr2=0x56320824a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.419+0000 7f61b3f9e740 1 --2- 172.21.15.107:0/2533731692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632082338a0 0x56320824a7a0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f61a0004da0 tx=0x7f61a0011760 comp rx=0 tx=0).stop 2024-04-25T00:21:24.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.419+0000 7f61b3f9e740 1 -- 172.21.15.107:0/2533731692 shutdown_connections 2024-04-25T00:21:24.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.419+0000 7f61b3f9e740 1 --2- 172.21.15.107:0/2533731692 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x563208242570 0x5632082baa40 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.419+0000 7f61b3f9e740 1 --2- 172.21.15.107:0/2533731692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563208237ea0 0x563208247d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.419+0000 7f61b3f9e740 1 --2- 172.21.15.107:0/2533731692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563208237580 0x56320824ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.419+0000 7f618eff5640 0 client.16710 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:24.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.419+0000 7f61b3f9e740 1 --2- 172.21.15.107:0/2533731692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632082338a0 0x56320824a7a0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.419+0000 7f61b3f9e740 1 -- 172.21.15.107:0/2533731692 >> 172.21.15.107:0/2533731692 conn(0x56320823f810 msgr2=0x563208232b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:24.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.419+0000 7f61b3f9e740 1 -- 172.21.15.107:0/2533731692 shutdown_connections 2024-04-25T00:21:24.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.419+0000 7f61b3f9e740 1 -- 172.21.15.107:0/2533731692 wait complete. 2024-04-25T00:21:24.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.419+0000 7f61b3f9e740 20 client.16710 trim_cache size 0 max 16384 2024-04-25T00:21:24.430 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:21:24.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.420+0000 7f61b11b0640 1 -- 172.21.15.107:0/408351464 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f61940a2d00 msgr2=0x7f61940a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.420+0000 7f61b11b0640 1 --2- 172.21.15.107:0/408351464 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f61940a2d00 0x7f61940a51c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f61a40b9120 tx=0x7f61a40b8840 comp rx=0 tx=0).stop 2024-04-25T00:21:24.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.420+0000 7f61b11b0640 1 -- 172.21.15.107:0/408351464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61ac106690 msgr2=0x7f61ac19b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:24.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.420+0000 7f61b11b0640 1 --2- 172.21.15.107:0/408351464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61ac106690 0x7f61ac19b6b0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f61a4031a60 tx=0x7f61a4031a90 comp rx=0 tx=0).stop 2024-04-25T00:21:24.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.420+0000 7f61b11b0640 1 -- 172.21.15.107:0/408351464 shutdown_connections 2024-04-25T00:21:24.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.420+0000 7f61b11b0640 1 --2- 172.21.15.107:0/408351464 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f61ac106690 0x7f61ac19b6b0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.420+0000 7f61b11b0640 1 --2- 172.21.15.107:0/408351464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f61ac101330 0x7f61ac197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.420+0000 7f61b11b0640 1 --2- 172.21.15.107:0/408351464 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f61940a2d00 0x7f61940a51c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.420+0000 7f61b11b0640 1 --2- 172.21.15.107:0/408351464 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f61ac100990 0x7f61ac196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:24.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.420+0000 7f61b11b0640 1 -- 172.21.15.107:0/408351464 >> 172.21.15.107:0/408351464 conn(0x7f61ac076440 msgr2=0x7f61ac0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:24.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.420+0000 7f61b11b0640 1 -- 172.21.15.107:0/408351464 shutdown_connections 2024-04-25T00:21:24.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:24.420+0000 7f61b11b0640 1 -- 172.21.15.107:0/408351464 wait complete. 2024-04-25T00:21:24.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:24 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/4208573001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:21:24.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:24 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/4208573001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:21:24.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:24 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/4208573001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:21:25.433 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:21:25.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:21:25.434 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:21:25.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.547+0000 7f5654f34640 1 -- 172.21.15.107:0/916021853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56501032e0 msgr2=0x7f565010b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.547+0000 7f5654f34640 1 --2- 172.21.15.107:0/916021853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56501032e0 0x7f565010b7c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f5644009900 tx=0x7f564402a510 comp rx=0 tx=0).stop 2024-04-25T00:21:25.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.547+0000 7f5654f34640 1 -- 172.21.15.107:0/916021853 shutdown_connections 2024-04-25T00:21:25.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.547+0000 7f5654f34640 1 --2- 172.21.15.107:0/916021853 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f565010bd00 0x7f565010e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.547+0000 7f5654f34640 1 --2- 172.21.15.107:0/916021853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56501032e0 0x7f565010b7c0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.547+0000 7f5654f34640 1 --2- 172.21.15.107:0/916021853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56501029c0 0x7f5650102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.547+0000 7f5654f34640 1 -- 172.21.15.107:0/916021853 >> 172.21.15.107:0/916021853 conn(0x7f5650077740 msgr2=0x7f5650077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:25.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.547+0000 7f5654f34640 1 -- 172.21.15.107:0/916021853 shutdown_connections 2024-04-25T00:21:25.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.547+0000 7f5654f34640 1 -- 172.21.15.107:0/916021853 wait complete. 2024-04-25T00:21:25.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.547+0000 7f5654f34640 1 Processor -- start 2024-04-25T00:21:25.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.547+0000 7f5654f34640 1 -- start start 2024-04-25T00:21:25.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.548+0000 7f5654f34640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56501029c0 0x7f5650198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.548+0000 7f5654f34640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56501032e0 0x7f56501986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.548+0000 7f5654f34640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f565010bd00 0x7f565019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.548+0000 7f5654f34640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5650111bc0 con 0x7f565010bd00 2024-04-25T00:21:25.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.548+0000 7f5654f34640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5650111a40 con 0x7f56501032e0 2024-04-25T00:21:25.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.548+0000 7f5654f34640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f5650111d40 con 0x7f56501029c0 2024-04-25T00:21:25.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.548+0000 7f564ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f565010bd00 0x7f565019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.548+0000 7f564ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f565010bd00 0x7f565019ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48852/0 (socket says 172.21.15.107:48852) 2024-04-25T00:21:25.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.548+0000 7f564ed9d640 1 -- 172.21.15.107:0/1327469259 learned_addr learned my addr 172.21.15.107:0/1327469259 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:25.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.548+0000 7f564dd9b640 1 --2- 172.21.15.107:0/1327469259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56501032e0 0x7f56501986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.548+0000 7f564e59c640 1 --2- 172.21.15.107:0/1327469259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56501029c0 0x7f5650198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.548+0000 7f564ed9d640 1 -- 172.21.15.107:0/1327469259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56501029c0 msgr2=0x7f5650198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.548+0000 7f564ed9d640 1 --2- 172.21.15.107:0/1327469259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56501029c0 0x7f5650198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.548+0000 7f564ed9d640 1 -- 172.21.15.107:0/1327469259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56501032e0 msgr2=0x7f56501986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.548+0000 7f564ed9d640 1 --2- 172.21.15.107:0/1327469259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56501032e0 0x7f56501986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.548+0000 7f564ed9d640 1 -- 172.21.15.107:0/1327469259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565019d170 con 0x7f565010bd00 2024-04-25T00:21:25.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.548+0000 7f564e59c640 1 --2- 172.21.15.107:0/1327469259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56501029c0 0x7f5650198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:25.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.548+0000 7f564ed9d640 1 --2- 172.21.15.107:0/1327469259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f565010bd00 0x7f565019ca40 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5648009fd0 tx=0x7f564800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:25.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.548+0000 7f563f7fe640 1 -- 172.21.15.107:0/1327469259 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5648011830 con 0x7f565010bd00 2024-04-25T00:21:25.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.549+0000 7f563f7fe640 1 -- 172.21.15.107:0/1327469259 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5648016070 con 0x7f565010bd00 2024-04-25T00:21:25.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.549+0000 7f5654f34640 1 -- 172.21.15.107:0/1327469259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f565019d460 con 0x7f565010bd00 2024-04-25T00:21:25.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.549+0000 7f563f7fe640 1 -- 172.21.15.107:0/1327469259 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564800f2e0 con 0x7f565010bd00 2024-04-25T00:21:25.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.549+0000 7f5654f34640 1 -- 172.21.15.107:0/1327469259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56501a4cf0 con 0x7f565010bd00 2024-04-25T00:21:25.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.551+0000 7f563f7fe640 1 -- 172.21.15.107:0/1327469259 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f56480119d0 con 0x7f565010bd00 2024-04-25T00:21:25.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.551+0000 7f5657c40740 1 Processor -- start 2024-04-25T00:21:25.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.551+0000 7f5657c40740 1 -- start start 2024-04-25T00:21:25.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.552+0000 7f5657c40740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dfb7520980 0x55dfb751ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.552+0000 7f5657c40740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dfb751f290 0x55dfb751f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.552+0000 7f5657c40740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb752a010 0x55dfb752c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.552+0000 7f5657c40740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dfb74832c0 con 0x55dfb752a010 2024-04-25T00:21:25.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.552+0000 7f5657c40740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55dfb742b910 con 0x55dfb7520980 2024-04-25T00:21:25.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.552+0000 7f5657c40740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55dfb747f280 con 0x55dfb751f290 2024-04-25T00:21:25.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.552+0000 7f564dd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb752a010 0x55dfb752c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.552+0000 7f564e59c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dfb751f290 0x55dfb751f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.552+0000 7f564dd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb752a010 0x55dfb752c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48856/0 (socket says 172.21.15.107:48856) 2024-04-25T00:21:25.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.552+0000 7f564dd9b640 1 -- 172.21.15.107:0/1355158597 learned_addr learned my addr 172.21.15.107:0/1355158597 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:25.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.552+0000 7f563f7fe640 1 --2- 172.21.15.107:0/1327469259 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f56380a2d00 0x7f56380a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.552+0000 7f564dd9b640 1 --2- 172.21.15.107:0/1355158597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dfb7520980 0x55dfb751ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.552+0000 7f564dd9b640 1 -- 172.21.15.107:0/1355158597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dfb751f290 msgr2=0x55dfb751f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:25.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.552+0000 7f564dd9b640 1 --2- 172.21.15.107:0/1355158597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dfb751f290 0x55dfb751f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.552+0000 7f564dd9b640 1 -- 172.21.15.107:0/1355158597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dfb7520980 msgr2=0x55dfb751ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:25.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.552+0000 7f564dd9b640 1 --2- 172.21.15.107:0/1355158597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dfb7520980 0x55dfb751ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.552+0000 7f564dd9b640 1 -- 172.21.15.107:0/1355158597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfb743cfb0 con 0x55dfb752a010 2024-04-25T00:21:25.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.552+0000 7f564ed9d640 1 --2- 172.21.15.107:0/1327469259 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f56380a2d00 0x7f56380a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.552+0000 7f563f7fe640 1 -- 172.21.15.107:0/1327469259 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f56480cecd0 con 0x7f565010bd00 2024-04-25T00:21:25.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.552+0000 7f564dd9b640 1 --2- 172.21.15.107:0/1355158597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb752a010 0x55dfb752c400 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5644005ec0 tx=0x7f5644005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e0edbc84ae2cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:25.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.552+0000 7f563cff9640 1 -- 172.21.15.107:0/1355158597 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5644035070 con 0x55dfb752a010 2024-04-25T00:21:25.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.553+0000 7f563cff9640 1 -- 172.21.15.107:0/1355158597 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5644031800 con 0x55dfb752a010 2024-04-25T00:21:25.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.553+0000 7f563cff9640 1 -- 172.21.15.107:0/1355158597 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56440047a0 con 0x55dfb752a010 2024-04-25T00:21:25.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.553+0000 7f564ed9d640 1 --2- 172.21.15.107:0/1327469259 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f56380a2d00 0x7f56380a51c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f564809ac90 tx=0x7f564806cf50 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:25.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.553+0000 7f5657c40740 1 -- 172.21.15.107:0/1355158597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb752a010 msgr2=0x55dfb752c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.553+0000 7f5657c40740 1 --2- 172.21.15.107:0/1355158597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb752a010 0x55dfb752c400 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5644005ec0 tx=0x7f5644005d90 comp rx=0 tx=0).stop 2024-04-25T00:21:25.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.553+0000 7f5657c40740 1 -- 172.21.15.107:0/1355158597 shutdown_connections 2024-04-25T00:21:25.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.553+0000 7f5657c40740 1 --2- 172.21.15.107:0/1355158597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb752a010 0x55dfb752c400 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.553+0000 7f5657c40740 1 --2- 172.21.15.107:0/1355158597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dfb751f290 0x55dfb751f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.553+0000 7f5657c40740 1 --2- 172.21.15.107:0/1355158597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dfb7520980 0x55dfb751ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.553+0000 7f5657c40740 1 -- 172.21.15.107:0/1355158597 >> 172.21.15.107:0/1355158597 conn(0x55dfb74e7950 msgr2=0x55dfb751b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:25.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.553+0000 7f5657c40740 1 -- 172.21.15.107:0/1355158597 shutdown_connections 2024-04-25T00:21:25.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.553+0000 7f5657c40740 1 -- 172.21.15.107:0/1355158597 wait complete. 2024-04-25T00:21:25.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.554+0000 7f5657c40740 1 -- start start 2024-04-25T00:21:25.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.554+0000 7f5657c40740 1 Processor -- start 2024-04-25T00:21:25.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.554+0000 7f5657c40740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb751f290 0x55dfb7526f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.554+0000 7f5657c40740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dfb7520980 0x55dfb7527450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.554+0000 7f5657c40740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dfb752a010 0x55dfb7527990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.554+0000 7f5657c40740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55dfb752ce40 con 0x55dfb751f290 2024-04-25T00:21:25.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.554+0000 7f5657c40740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55dfb743ee90 con 0x55dfb7520980 2024-04-25T00:21:25.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.554+0000 7f5657c40740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55dfb752cfc0 con 0x55dfb752a010 2024-04-25T00:21:25.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.554+0000 7f5657c40740 10 client.0 _ll_register_callbacks cb 0x55dfb747a970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:25.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.554+0000 7f564dd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb751f290 0x55dfb7526f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.554+0000 7f564dd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb751f290 0x55dfb7526f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48858/0 (socket says 172.21.15.107:48858) 2024-04-25T00:21:25.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.554+0000 7f564dd9b640 1 -- 172.21.15.107:0/3818058973 learned_addr learned my addr 172.21.15.107:0/3818058973 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:25.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.554+0000 7f564e59c640 1 --2- 172.21.15.107:0/3818058973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dfb7520980 0x55dfb7527450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.554+0000 7f564dd9b640 1 --2- 172.21.15.107:0/3818058973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dfb752a010 0x55dfb7527990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.554+0000 7f564dd9b640 1 -- 172.21.15.107:0/3818058973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dfb752a010 msgr2=0x55dfb7527990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:25.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.554+0000 7f564dd9b640 1 --2- 172.21.15.107:0/3818058973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dfb752a010 0x55dfb7527990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.554+0000 7f564dd9b640 1 -- 172.21.15.107:0/3818058973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dfb7520980 msgr2=0x55dfb7527450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:25.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.554+0000 7f564dd9b640 1 --2- 172.21.15.107:0/3818058973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dfb7520980 0x55dfb7527450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.554+0000 7f564dd9b640 1 -- 172.21.15.107:0/3818058973 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfb725f7e0 con 0x55dfb751f290 2024-04-25T00:21:25.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.555+0000 7f564dd9b640 1 --2- 172.21.15.107:0/3818058973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb751f290 0x55dfb7526f10 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5644006010 tx=0x7f56440054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f9b1d73831728b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:25.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.555+0000 7f5635ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:21:25.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.555+0000 7f5635ffb640 1 -- 172.21.15.107:0/3818058973 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5644050020 con 0x55dfb751f290 2024-04-25T00:21:25.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.555+0000 7f5635ffb640 1 -- 172.21.15.107:0/3818058973 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f5644004250 con 0x55dfb751f290 2024-04-25T00:21:25.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.555+0000 7f5657c40740 1 -- 172.21.15.107:0/3818058973 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5644031800 con 0x55dfb751f290 2024-04-25T00:21:25.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.555+0000 7f5635ffb640 1 -- 172.21.15.107:0/3818058973 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f564405b050 con 0x55dfb751f290 2024-04-25T00:21:25.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.555+0000 7f5657c40740 10 client.16722 fetch_fsmap learned FSMap version 29 2024-04-25T00:21:25.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.555+0000 7f5657c40740 1 -- 172.21.15.107:0/3818058973 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dfb72bff70 con 0x55dfb751f290 2024-04-25T00:21:25.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.555+0000 7f5635ffb640 1 -- 172.21.15.107:0/3818058973 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5644031800 con 0x55dfb751f290 2024-04-25T00:21:25.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.555+0000 7f5635ffb640 1 -- 172.21.15.107:0/3818058973 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f5644003710 con 0x55dfb751f290 2024-04-25T00:21:25.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.555+0000 7f5657c40740 10 client.16722 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:21:25.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.555+0000 7f5657c40740 10 client.16722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:25.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.555+0000 7f5657c40740 20 client.16722 populate_metadata read hostname 'smithi107' 2024-04-25T00:21:25.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.556+0000 7f5657c40740 1 --2- 172.21.15.107:0/3818058973 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55dfb7539f40 0x55dfb753c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.556+0000 7f5657c40740 4 client.16722 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:25.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.556+0000 7f5657c40740 1 -- 172.21.15.107:0/3818058973 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfb74575a0 con 0x55dfb7539f40 2024-04-25T00:21:25.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.556+0000 7f564e59c640 1 --2- 172.21.15.107:0/3818058973 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55dfb7539f40 0x55dfb753c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.556+0000 7f564e59c640 1 --2- 172.21.15.107:0/3818058973 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55dfb7539f40 0x55dfb753c330 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c8538b2aa86be1f server_cookie=95852d5d5eb85a71 in_seq=0 out_seq=0 2024-04-25T00:21:25.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.556+0000 7f5635ffb640 10 client.16722 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:25.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.557+0000 7f5635ffb640 1 -- 172.21.15.107:0/3818058973 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dfb74575a0 con 0x55dfb7539f40 2024-04-25T00:21:25.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.557+0000 7f5635ffb640 10 client.16722 handle_command_reply: tid=0 2024-04-25T00:21:25.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.557+0000 7f5657c40740 1 client.16722 shutdown 2024-04-25T00:21:25.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.557+0000 7f5657c40740 1 -- 172.21.15.107:0/3818058973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb751f290 msgr2=0x55dfb7526f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.557+0000 7f5657c40740 1 --2- 172.21.15.107:0/3818058973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb751f290 0x55dfb7526f10 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5644006010 tx=0x7f56440054a0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.557+0000 7f5657c40740 1 -- 172.21.15.107:0/3818058973 shutdown_connections 2024-04-25T00:21:25.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.557+0000 7f5657c40740 1 --2- 172.21.15.107:0/3818058973 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55dfb7539f40 0x55dfb753c330 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.557+0000 7f5657c40740 1 --2- 172.21.15.107:0/3818058973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dfb752a010 0x55dfb7527990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.557+0000 7f5657c40740 1 --2- 172.21.15.107:0/3818058973 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dfb7520980 0x55dfb7527450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.557+0000 7f5635ffb640 0 client.16722 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:25.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.557+0000 7f5657c40740 1 --2- 172.21.15.107:0/3818058973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb751f290 0x55dfb7526f10 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.557+0000 7f5657c40740 1 -- 172.21.15.107:0/3818058973 >> 172.21.15.107:0/3818058973 conn(0x55dfb74e7950 msgr2=0x55dfb7521660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:25.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.557+0000 7f5657c40740 1 -- 172.21.15.107:0/3818058973 shutdown_connections 2024-04-25T00:21:25.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.557+0000 7f5657c40740 1 -- 172.21.15.107:0/3818058973 wait complete. 2024-04-25T00:21:25.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.557+0000 7f5657c40740 20 client.16722 trim_cache size 0 max 16384 2024-04-25T00:21:25.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.575+0000 7f5657c40740 1 Processor -- start 2024-04-25T00:21:25.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.575+0000 7f5657c40740 1 -- start start 2024-04-25T00:21:25.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.575+0000 7f5657c40740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dfb751f290 0x55dfb7535d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.575+0000 7f5657c40740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dfb7520980 0x55dfb7525de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.575+0000 7f5657c40740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb752a010 0x55dfb7541930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.575+0000 7f5657c40740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f56501b1560 con 0x55dfb752a010 2024-04-25T00:21:25.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.575+0000 7f5657c40740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f56501aeda0 con 0x55dfb751f290 2024-04-25T00:21:25.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.575+0000 7f5657c40740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f56501acbf0 con 0x55dfb7520980 2024-04-25T00:21:25.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.575+0000 7f564dd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb752a010 0x55dfb7541930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.575+0000 7f564dd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb752a010 0x55dfb7541930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48860/0 (socket says 172.21.15.107:48860) 2024-04-25T00:21:25.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.575+0000 7f564dd9b640 1 -- 172.21.15.107:0/409922517 learned_addr learned my addr 172.21.15.107:0/409922517 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:25.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.575+0000 7f564e59c640 1 --2- 172.21.15.107:0/409922517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dfb7520980 0x55dfb7525de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.575+0000 7f564dd9b640 1 -- 172.21.15.107:0/409922517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dfb7520980 msgr2=0x55dfb7525de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:25.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.575+0000 7f564dd9b640 1 --2- 172.21.15.107:0/409922517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dfb7520980 0x55dfb7525de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.575+0000 7f564dd9b640 1 -- 172.21.15.107:0/409922517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dfb751f290 msgr2=0x55dfb7535d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:25.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.575+0000 7f564dd9b640 1 --2- 172.21.15.107:0/409922517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dfb751f290 0x55dfb7535d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.575+0000 7f564dd9b640 1 -- 172.21.15.107:0/409922517 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfb7157170 con 0x55dfb752a010 2024-04-25T00:21:25.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.575+0000 7f564dd9b640 1 --2- 172.21.15.107:0/409922517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb752a010 0x55dfb7541930 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f564402a550 tx=0x7f5644005510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a50549a1696ca7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:25.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.575+0000 7f5637fff640 1 -- 172.21.15.107:0/409922517 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5644050070 con 0x55dfb752a010 2024-04-25T00:21:25.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.575+0000 7f5637fff640 1 -- 172.21.15.107:0/409922517 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5644004c80 con 0x55dfb752a010 2024-04-25T00:21:25.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.575+0000 7f5637fff640 1 -- 172.21.15.107:0/409922517 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5644035040 con 0x55dfb752a010 2024-04-25T00:21:25.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.576+0000 7f5657c40740 1 -- 172.21.15.107:0/409922517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb752a010 msgr2=0x55dfb7541930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.576+0000 7f5657c40740 1 --2- 172.21.15.107:0/409922517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb752a010 0x55dfb7541930 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f564402a550 tx=0x7f5644005510 comp rx=0 tx=0).stop 2024-04-25T00:21:25.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.576+0000 7f5657c40740 1 -- 172.21.15.107:0/409922517 shutdown_connections 2024-04-25T00:21:25.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.576+0000 7f5657c40740 1 --2- 172.21.15.107:0/409922517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb752a010 0x55dfb7541930 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.576+0000 7f5657c40740 1 --2- 172.21.15.107:0/409922517 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dfb7520980 0x55dfb7525de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.576+0000 7f5657c40740 1 --2- 172.21.15.107:0/409922517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dfb751f290 0x55dfb7535d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.576+0000 7f5657c40740 1 -- 172.21.15.107:0/409922517 >> 172.21.15.107:0/409922517 conn(0x55dfb74e7950 msgr2=0x55dfb75256d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:25.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.576+0000 7f5657c40740 1 -- 172.21.15.107:0/409922517 shutdown_connections 2024-04-25T00:21:25.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.576+0000 7f5657c40740 1 -- 172.21.15.107:0/409922517 wait complete. 2024-04-25T00:21:25.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.576+0000 7f5657c40740 1 -- start start 2024-04-25T00:21:25.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.576+0000 7f5657c40740 1 Processor -- start 2024-04-25T00:21:25.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.577+0000 7f5657c40740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dfb751f290 0x55dfb75242f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.577+0000 7f5657c40740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dfb7520980 0x55dfb7524830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.577+0000 7f5657c40740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb752a010 0x55dfb759f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.577+0000 7f5657c40740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f56501ad260 con 0x55dfb752a010 2024-04-25T00:21:25.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.577+0000 7f5657c40740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f56501ae6d0 con 0x55dfb7520980 2024-04-25T00:21:25.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.577+0000 7f5657c40740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f56501ad930 con 0x55dfb751f290 2024-04-25T00:21:25.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.577+0000 7f5657c40740 10 client.0 _ll_register_callbacks cb 0x7f5640009770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:25.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.577+0000 7f564dd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb752a010 0x55dfb759f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.577+0000 7f564dd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb752a010 0x55dfb759f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48862/0 (socket says 172.21.15.107:48862) 2024-04-25T00:21:25.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.577+0000 7f564dd9b640 1 -- 172.21.15.107:0/2097513353 learned_addr learned my addr 172.21.15.107:0/2097513353 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:25.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.577+0000 7f564e59c640 1 --2- 172.21.15.107:0/2097513353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dfb7520980 0x55dfb7524830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.577+0000 7f564dd9b640 1 --2- 172.21.15.107:0/2097513353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dfb751f290 0x55dfb75242f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.577+0000 7f564dd9b640 1 -- 172.21.15.107:0/2097513353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dfb751f290 msgr2=0x55dfb75242f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:25.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.577+0000 7f564dd9b640 1 --2- 172.21.15.107:0/2097513353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dfb751f290 0x55dfb75242f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.577+0000 7f564dd9b640 1 -- 172.21.15.107:0/2097513353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dfb7520980 msgr2=0x55dfb7524830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:25.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.577+0000 7f564dd9b640 1 --2- 172.21.15.107:0/2097513353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dfb7520980 0x55dfb7524830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.577+0000 7f564dd9b640 1 -- 172.21.15.107:0/2097513353 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfb759f6e0 con 0x55dfb752a010 2024-04-25T00:21:25.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.577+0000 7f564dd9b640 1 --2- 172.21.15.107:0/2097513353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb752a010 0x55dfb759f1a0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f5644031460 tx=0x7f564404c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f02063b3fc703a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:25.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.577+0000 7f5635ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:21:25.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.577+0000 7f5635ffb640 1 -- 172.21.15.107:0/2097513353 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5644035060 con 0x55dfb752a010 2024-04-25T00:21:25.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.577+0000 7f5635ffb640 1 -- 172.21.15.107:0/2097513353 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f56440049b0 con 0x55dfb752a010 2024-04-25T00:21:25.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.577+0000 7f5657c40740 1 -- 172.21.15.107:0/2097513353 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5644004c80 con 0x55dfb752a010 2024-04-25T00:21:25.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.577+0000 7f5635ffb640 1 -- 172.21.15.107:0/2097513353 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5644050040 con 0x55dfb752a010 2024-04-25T00:21:25.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.577+0000 7f5657c40740 10 client.16728 fetch_fsmap learned FSMap version 29 2024-04-25T00:21:25.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.578+0000 7f5657c40740 1 -- 172.21.15.107:0/2097513353 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dfb75a09c0 con 0x55dfb752a010 2024-04-25T00:21:25.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.578+0000 7f5635ffb640 1 -- 172.21.15.107:0/2097513353 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5644004c80 con 0x55dfb752a010 2024-04-25T00:21:25.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.578+0000 7f5635ffb640 1 -- 172.21.15.107:0/2097513353 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f564403d640 con 0x55dfb752a010 2024-04-25T00:21:25.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.578+0000 7f5657c40740 10 client.16728 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:21:25.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.578+0000 7f5657c40740 10 client.16728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:25.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.578+0000 7f5657c40740 20 client.16728 populate_metadata read hostname 'smithi107' 2024-04-25T00:21:25.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.578+0000 7f5657c40740 1 --2- 172.21.15.107:0/2097513353 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55dfb75a1010 0x55dfb75a3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.578+0000 7f5657c40740 4 client.16728 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:21:25.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.578+0000 7f5657c40740 1 -- 172.21.15.107:0/2097513353 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dfb7490800 con 0x55dfb75a1010 2024-04-25T00:21:25.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.578+0000 7f564e59c640 1 --2- 172.21.15.107:0/2097513353 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55dfb75a1010 0x55dfb75a3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.578+0000 7f5635ffb640 10 client.16728 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:25.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.578+0000 7f564e59c640 1 --2- 172.21.15.107:0/2097513353 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55dfb75a1010 0x55dfb75a3430 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd5f6b6e91943b1 server_cookie=39b213904d7d57c0 in_seq=0 out_seq=0 2024-04-25T00:21:25.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.579+0000 7f5635ffb640 1 -- 172.21.15.107:0/2097513353 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dfb7490800 con 0x55dfb75a1010 2024-04-25T00:21:25.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.579+0000 7f5635ffb640 10 client.16728 handle_command_reply: tid=0 2024-04-25T00:21:25.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.579+0000 7f5657c40740 1 client.16728 shutdown 2024-04-25T00:21:25.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.579+0000 7f5657c40740 1 -- 172.21.15.107:0/2097513353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb752a010 msgr2=0x55dfb759f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.579+0000 7f5657c40740 1 --2- 172.21.15.107:0/2097513353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb752a010 0x55dfb759f1a0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f5644031460 tx=0x7f564404c040 comp rx=0 tx=0).stop 2024-04-25T00:21:25.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.580+0000 7f5657c40740 1 -- 172.21.15.107:0/2097513353 shutdown_connections 2024-04-25T00:21:25.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.580+0000 7f5657c40740 1 --2- 172.21.15.107:0/2097513353 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55dfb75a1010 0x55dfb75a3430 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.580+0000 7f5657c40740 1 --2- 172.21.15.107:0/2097513353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55dfb752a010 0x55dfb759f1a0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.580+0000 7f5657c40740 1 --2- 172.21.15.107:0/2097513353 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dfb7520980 0x55dfb7524830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.580+0000 7f5657c40740 1 --2- 172.21.15.107:0/2097513353 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55dfb751f290 0x55dfb75242f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.580+0000 7f5657c40740 1 -- 172.21.15.107:0/2097513353 >> 172.21.15.107:0/2097513353 conn(0x55dfb74e7950 msgr2=0x55dfb7525f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:25.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.580+0000 7f5635ffb640 0 client.16728 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:25.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.580+0000 7f5657c40740 1 -- 172.21.15.107:0/2097513353 shutdown_connections 2024-04-25T00:21:25.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.580+0000 7f5657c40740 1 -- 172.21.15.107:0/2097513353 wait complete. 2024-04-25T00:21:25.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.580+0000 7f5657c40740 20 client.16728 trim_cache size 0 max 16384 2024-04-25T00:21:25.588 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:21:25.588 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-04-25T00:21:25.588 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "96f3e171-d5be-4730-9a05-fe2b6791e1ff", 2024-04-25T00:21:25.588 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-04-25T00:21:25.588 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:21:25.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.580+0000 7f5654f34640 1 -- 172.21.15.107:0/1327469259 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f56380a2d00 msgr2=0x7f56380a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.580+0000 7f5654f34640 1 --2- 172.21.15.107:0/1327469259 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f56380a2d00 0x7f56380a51c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f564809ac90 tx=0x7f564806cf50 comp rx=0 tx=0).stop 2024-04-25T00:21:25.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.580+0000 7f5654f34640 1 -- 172.21.15.107:0/1327469259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f565010bd00 msgr2=0x7f565019ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.580+0000 7f5654f34640 1 --2- 172.21.15.107:0/1327469259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f565010bd00 0x7f565019ca40 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5648009fd0 tx=0x7f564800b8e0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.580+0000 7f5654f34640 1 -- 172.21.15.107:0/1327469259 shutdown_connections 2024-04-25T00:21:25.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.580+0000 7f5654f34640 1 --2- 172.21.15.107:0/1327469259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f565010bd00 0x7f565019ca40 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.580+0000 7f5654f34640 1 --2- 172.21.15.107:0/1327469259 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f56380a2d00 0x7f56380a51c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.580+0000 7f5654f34640 1 --2- 172.21.15.107:0/1327469259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56501032e0 0x7f56501986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.580+0000 7f5654f34640 1 --2- 172.21.15.107:0/1327469259 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56501029c0 0x7f5650198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.580+0000 7f5654f34640 1 -- 172.21.15.107:0/1327469259 >> 172.21.15.107:0/1327469259 conn(0x7f5650077740 msgr2=0x7f56500fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:25.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.580+0000 7f5654f34640 1 -- 172.21.15.107:0/1327469259 shutdown_connections 2024-04-25T00:21:25.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.580+0000 7f5654f34640 1 -- 172.21.15.107:0/1327469259 wait complete. 2024-04-25T00:21:25.592 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:21:25.593 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:21:25.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:25 smithi170 ceph-mon[32807]: pgmap v152: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:21:25.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.692+0000 7fb4da22c640 1 -- 172.21.15.107:0/138392668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4d4100610 msgr2=0x7fb4d4100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.692+0000 7fb4da22c640 1 --2- 172.21.15.107:0/138392668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4d4100610 0x7fb4d4100a90 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009930 tx=0x7fb4bc02a420 comp rx=0 tx=0).stop 2024-04-25T00:21:25.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.692+0000 7fb4da22c640 1 -- 172.21.15.107:0/138392668 shutdown_connections 2024-04-25T00:21:25.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.692+0000 7fb4da22c640 1 --2- 172.21.15.107:0/138392668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4d4100fd0 0x7fb4d410d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.692+0000 7fb4da22c640 1 --2- 172.21.15.107:0/138392668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4d4100610 0x7fb4d4100a90 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.692+0000 7fb4da22c640 1 --2- 172.21.15.107:0/138392668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d41066b0 0x7fb4d4106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.692+0000 7fb4da22c640 1 -- 172.21.15.107:0/138392668 >> 172.21.15.107:0/138392668 conn(0x7fb4d4077740 msgr2=0x7fb4d4077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:25.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.692+0000 7fb4da22c640 1 -- 172.21.15.107:0/138392668 shutdown_connections 2024-04-25T00:21:25.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.692+0000 7fb4da22c640 1 -- 172.21.15.107:0/138392668 wait complete. 2024-04-25T00:21:25.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.692+0000 7fb4da22c640 1 Processor -- start 2024-04-25T00:21:25.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.692+0000 7fb4da22c640 1 -- start start 2024-04-25T00:21:25.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.693+0000 7fb4da22c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4d4100610 0x7fb4d4197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.693+0000 7fb4da22c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4d4100fd0 0x7fb4d41984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.693+0000 7fb4da22c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d41066b0 0x7fb4d419c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.693+0000 7fb4da22c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d41109f0 con 0x7fb4d4100fd0 2024-04-25T00:21:25.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.693+0000 7fb4da22c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d4110870 con 0x7fb4d41066b0 2024-04-25T00:21:25.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.693+0000 7fb4da22c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d4110b70 con 0x7fb4d4100610 2024-04-25T00:21:25.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.693+0000 7fb4cbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4d4100fd0 0x7fb4d41984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.693+0000 7fb4cbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4d4100fd0 0x7fb4d41984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48866/0 (socket says 172.21.15.107:48866) 2024-04-25T00:21:25.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.693+0000 7fb4cbfff640 1 -- 172.21.15.107:0/4112636830 learned_addr learned my addr 172.21.15.107:0/4112636830 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:25.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.693+0000 7fb4d3fff640 1 --2- 172.21.15.107:0/4112636830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d41066b0 0x7fb4d419c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.693+0000 7fb4d37fe640 1 --2- 172.21.15.107:0/4112636830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4d4100610 0x7fb4d4197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.693+0000 7fb4cbfff640 1 -- 172.21.15.107:0/4112636830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4d4100610 msgr2=0x7fb4d4197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.693+0000 7fb4cbfff640 1 --2- 172.21.15.107:0/4112636830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4d4100610 0x7fb4d4197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.693+0000 7fb4cbfff640 1 -- 172.21.15.107:0/4112636830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d41066b0 msgr2=0x7fb4d419c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.693+0000 7fb4cbfff640 1 --2- 172.21.15.107:0/4112636830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d41066b0 0x7fb4d419c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.693+0000 7fb4cbfff640 1 -- 172.21.15.107:0/4112636830 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d419cf70 con 0x7fb4d4100fd0 2024-04-25T00:21:25.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.693+0000 7fb4d3fff640 1 --2- 172.21.15.107:0/4112636830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d41066b0 0x7fb4d419c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:25.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.693+0000 7fb4cbfff640 1 --2- 172.21.15.107:0/4112636830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4d4100fd0 0x7fb4d41984c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009230 tx=0x7fb4bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:25.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.693+0000 7fb4d17fa640 1 -- 172.21.15.107:0/4112636830 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc03f030 con 0x7fb4d4100fd0 2024-04-25T00:21:25.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.693+0000 7fb4d17fa640 1 -- 172.21.15.107:0/4112636830 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4bc003ad0 con 0x7fb4d4100fd0 2024-04-25T00:21:25.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.693+0000 7fb4d17fa640 1 -- 172.21.15.107:0/4112636830 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc035070 con 0x7fb4d4100fd0 2024-04-25T00:21:25.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.693+0000 7fb4da22c640 1 -- 172.21.15.107:0/4112636830 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d419d200 con 0x7fb4d4100fd0 2024-04-25T00:21:25.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.693+0000 7fb4da22c640 1 -- 172.21.15.107:0/4112636830 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d41a4af0 con 0x7fb4d4100fd0 2024-04-25T00:21:25.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.695+0000 7fb4d17fa640 1 -- 172.21.15.107:0/4112636830 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb4bc042030 con 0x7fb4d4100fd0 2024-04-25T00:21:25.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.696+0000 7fb4dd010740 1 Processor -- start 2024-04-25T00:21:25.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.696+0000 7fb4dd010740 1 -- start start 2024-04-25T00:21:25.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.696+0000 7fb4d17fa640 1 --2- 172.21.15.107:0/4112636830 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb4c00a2d00 0x7fb4c00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.696+0000 7fb4d3fff640 1 --2- 172.21.15.107:0/4112636830 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb4c00a2d00 0x7fb4c00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.696+0000 7fb4dd010740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0630 0x562ce1da0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.696+0000 7fb4dd010740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ce1da0f50 0x562ce1dab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.696+0000 7fb4dd010740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ce1dab620 0x562ce1dada10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.696+0000 7fb4dd010740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562ce1d056b0 con 0x562ce1da0630 2024-04-25T00:21:25.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.696+0000 7fb4dd010740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562ce1cad880 con 0x562ce1da0f50 2024-04-25T00:21:25.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.696+0000 7fb4dd010740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562ce1d01670 con 0x562ce1dab620 2024-04-25T00:21:25.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.696+0000 7fb4d17fa640 1 -- 172.21.15.107:0/4112636830 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fb4bc0efeb0 con 0x7fb4d4100fd0 2024-04-25T00:21:25.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.696+0000 7fb4d37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0630 0x562ce1da0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.696+0000 7fb4d37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0630 0x562ce1da0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48878/0 (socket says 172.21.15.107:48878) 2024-04-25T00:21:25.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.696+0000 7fb4d37fe640 1 -- 172.21.15.107:0/3705753089 learned_addr learned my addr 172.21.15.107:0/3705753089 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:25.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.697+0000 7fb4d37fe640 1 -- 172.21.15.107:0/3705753089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ce1dab620 msgr2=0x562ce1dada10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:25.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.697+0000 7fb4d37fe640 1 --2- 172.21.15.107:0/3705753089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ce1dab620 0x562ce1dada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.697+0000 7fb4d37fe640 1 -- 172.21.15.107:0/3705753089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ce1da0f50 msgr2=0x562ce1dab0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:25.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.697+0000 7fb4d37fe640 1 --2- 172.21.15.107:0/3705753089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ce1da0f50 0x562ce1dab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.697+0000 7fb4d37fe640 1 -- 172.21.15.107:0/3705753089 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ce1ae37d0 con 0x562ce1da0630 2024-04-25T00:21:25.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.697+0000 7fb4d3fff640 1 --2- 172.21.15.107:0/4112636830 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb4c00a2d00 0x7fb4c00a51c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc005fd0 tx=0x7fb4cc00f730 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:25.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.697+0000 7fb4d37fe640 1 --2- 172.21.15.107:0/3705753089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0630 0x562ce1da0a10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4009b20 tx=0x7fb4c40113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c679729104beacfd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:25.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.697+0000 7fb4c9ffb640 1 -- 172.21.15.107:0/3705753089 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4011e10 con 0x562ce1da0630 2024-04-25T00:21:25.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.697+0000 7fb4c9ffb640 1 -- 172.21.15.107:0/3705753089 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4c40035f0 con 0x562ce1da0630 2024-04-25T00:21:25.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.697+0000 7fb4c9ffb640 1 -- 172.21.15.107:0/3705753089 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4012690 con 0x562ce1da0630 2024-04-25T00:21:25.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.697+0000 7fb4dd010740 1 -- 172.21.15.107:0/3705753089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0630 msgr2=0x562ce1da0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.697+0000 7fb4dd010740 1 --2- 172.21.15.107:0/3705753089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0630 0x562ce1da0a10 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb4c4009b20 tx=0x7fb4c40113e0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.697+0000 7fb4dd010740 1 -- 172.21.15.107:0/3705753089 shutdown_connections 2024-04-25T00:21:25.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.698+0000 7fb4dd010740 1 --2- 172.21.15.107:0/3705753089 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ce1dab620 0x562ce1dada10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.698+0000 7fb4dd010740 1 --2- 172.21.15.107:0/3705753089 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ce1da0f50 0x562ce1dab0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.698+0000 7fb4dd010740 1 --2- 172.21.15.107:0/3705753089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0630 0x562ce1da0a10 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.698+0000 7fb4dd010740 1 -- 172.21.15.107:0/3705753089 >> 172.21.15.107:0/3705753089 conn(0x562ce1d9c950 msgr2=0x562ce1d9cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:25.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.698+0000 7fb4dd010740 1 -- 172.21.15.107:0/3705753089 shutdown_connections 2024-04-25T00:21:25.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.698+0000 7fb4dd010740 1 -- 172.21.15.107:0/3705753089 wait complete. 2024-04-25T00:21:25.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.698+0000 7fb4dd010740 1 -- start start 2024-04-25T00:21:25.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.698+0000 7fb4dd010740 1 Processor -- start 2024-04-25T00:21:25.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.698+0000 7fb4dd010740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ce1da0630 0x562ce1da8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.698+0000 7fb4dd010740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0f50 0x562ce1da8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.698+0000 7fb4dd010740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ce1dab620 0x562ce1da90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.698+0000 7fb4dd010740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562ce1dae410 con 0x562ce1da0f50 2024-04-25T00:21:25.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.698+0000 7fb4dd010740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562ce1cc0f50 con 0x562ce1da0630 2024-04-25T00:21:25.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.698+0000 7fb4dd010740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562ce1dae590 con 0x562ce1dab620 2024-04-25T00:21:25.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.698+0000 7fb4dd010740 10 client.0 _ll_register_callbacks cb 0x562ce1cfc000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:25.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.698+0000 7fb4cbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0f50 0x562ce1da8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.698+0000 7fb4cbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0f50 0x562ce1da8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48894/0 (socket says 172.21.15.107:48894) 2024-04-25T00:21:25.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.698+0000 7fb4cbfff640 1 -- 172.21.15.107:0/3745731324 learned_addr learned my addr 172.21.15.107:0/3745731324 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:25.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.699+0000 7fb4d37fe640 1 --2- 172.21.15.107:0/3745731324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ce1da0630 0x562ce1da8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.699+0000 7fb4cbfff640 1 -- 172.21.15.107:0/3745731324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ce1dab620 msgr2=0x562ce1da90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:25.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.699+0000 7fb4cbfff640 1 --2- 172.21.15.107:0/3745731324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ce1dab620 0x562ce1da90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.699+0000 7fb4cbfff640 1 -- 172.21.15.107:0/3745731324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ce1da0630 msgr2=0x562ce1da8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:25.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.699+0000 7fb4cbfff640 1 --2- 172.21.15.107:0/3745731324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ce1da0630 0x562ce1da8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.699+0000 7fb4cbfff640 1 -- 172.21.15.107:0/3745731324 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ce1a193f0 con 0x562ce1da0f50 2024-04-25T00:21:25.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.699+0000 7fb4d37fe640 1 --2- 172.21.15.107:0/3745731324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ce1da0630 0x562ce1da8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:25.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.699+0000 7fb4cbfff640 1 --2- 172.21.15.107:0/3745731324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0f50 0x562ce1da8b70 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0ed2f0 tx=0x7fb4bc0bba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5f41ec5e488459 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:25.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.699+0000 7fb4b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:21:25.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.699+0000 7fb4b2ffd640 1 -- 172.21.15.107:0/3745731324 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc049040 con 0x562ce1da0f50 2024-04-25T00:21:25.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.699+0000 7fb4b2ffd640 1 -- 172.21.15.107:0/3745731324 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fb4bc03bd60 con 0x562ce1da0f50 2024-04-25T00:21:25.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.699+0000 7fb4dd010740 1 -- 172.21.15.107:0/3745731324 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4c40035f0 con 0x562ce1da0f50 2024-04-25T00:21:25.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.699+0000 7fb4b2ffd640 1 -- 172.21.15.107:0/3745731324 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb4bc0c4cd0 con 0x562ce1da0f50 2024-04-25T00:21:25.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.699+0000 7fb4dd010740 10 client.16740 fetch_fsmap learned FSMap version 29 2024-04-25T00:21:25.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.699+0000 7fb4dd010740 1 -- 172.21.15.107:0/3745731324 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562ce19d90f0 con 0x562ce1da0f50 2024-04-25T00:21:25.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.699+0000 7fb4b2ffd640 1 -- 172.21.15.107:0/3745731324 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4c40035f0 con 0x562ce1da0f50 2024-04-25T00:21:25.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.699+0000 7fb4b2ffd640 1 -- 172.21.15.107:0/3745731324 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fb4bc0b86d0 con 0x562ce1da0f50 2024-04-25T00:21:25.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.699+0000 7fb4dd010740 10 client.16740 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:21:25.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.699+0000 7fb4dd010740 10 client.16740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:25.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.699+0000 7fb4dd010740 20 client.16740 populate_metadata read hostname 'smithi107' 2024-04-25T00:21:25.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.700+0000 7fb4dd010740 1 --2- 172.21.15.107:0/3745731324 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562ce1dbb9a0 0x562ce1dbddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.700+0000 7fb4dd010740 4 client.16740 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:25.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.700+0000 7fb4dd010740 1 -- 172.21.15.107:0/3745731324 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ce1d12bf0 con 0x562ce1dbb9a0 2024-04-25T00:21:25.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.700+0000 7fb4d37fe640 1 --2- 172.21.15.107:0/3745731324 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562ce1dbb9a0 0x562ce1dbddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.700+0000 7fb4d37fe640 1 --2- 172.21.15.107:0/3745731324 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562ce1dbb9a0 0x562ce1dbddc0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9520ccb8643ffb58 server_cookie=85b3317fd822e20a in_seq=0 out_seq=0 2024-04-25T00:21:25.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.700+0000 7fb4b2ffd640 10 client.16740 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:25.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.701+0000 7fb4b2ffd640 1 -- 172.21.15.107:0/3745731324 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562ce1d12bf0 con 0x562ce1dbb9a0 2024-04-25T00:21:25.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.701+0000 7fb4b2ffd640 10 client.16740 handle_command_reply: tid=0 2024-04-25T00:21:25.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.701+0000 7fb4dd010740 1 client.16740 shutdown 2024-04-25T00:21:25.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.701+0000 7fb4dd010740 1 -- 172.21.15.107:0/3745731324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0f50 msgr2=0x562ce1da8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.701+0000 7fb4dd010740 1 --2- 172.21.15.107:0/3745731324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0f50 0x562ce1da8b70 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0ed2f0 tx=0x7fb4bc0bba80 comp rx=0 tx=0).stop 2024-04-25T00:21:25.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.701+0000 7fb4dd010740 1 -- 172.21.15.107:0/3745731324 shutdown_connections 2024-04-25T00:21:25.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.701+0000 7fb4dd010740 1 --2- 172.21.15.107:0/3745731324 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562ce1dbb9a0 0x562ce1dbddc0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.701+0000 7fb4dd010740 1 --2- 172.21.15.107:0/3745731324 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ce1dab620 0x562ce1da90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.701+0000 7fb4dd010740 1 --2- 172.21.15.107:0/3745731324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0f50 0x562ce1da8b70 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.701+0000 7fb4b2ffd640 0 client.16740 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:25.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.701+0000 7fb4dd010740 1 --2- 172.21.15.107:0/3745731324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ce1da0630 0x562ce1da8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.701+0000 7fb4dd010740 1 -- 172.21.15.107:0/3745731324 >> 172.21.15.107:0/3745731324 conn(0x562ce1d9c950 msgr2=0x562ce1da2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:25.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.701+0000 7fb4dd010740 1 -- 172.21.15.107:0/3745731324 shutdown_connections 2024-04-25T00:21:25.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.701+0000 7fb4dd010740 1 -- 172.21.15.107:0/3745731324 wait complete. 2024-04-25T00:21:25.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.701+0000 7fb4dd010740 20 client.16740 trim_cache size 0 max 16384 2024-04-25T00:21:25.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.718+0000 7fb4dd010740 1 Processor -- start 2024-04-25T00:21:25.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.718+0000 7fb4dd010740 1 -- start start 2024-04-25T00:21:25.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.718+0000 7fb4dd010740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ce1da0630 0x562ce1daad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.718+0000 7fb4dd010740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0f50 0x562ce1db7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.718+0000 7fb4dd010740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ce1dab620 0x562ce1da8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.718+0000 7fb4dd010740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d41b1330 con 0x562ce1da0f50 2024-04-25T00:21:25.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.718+0000 7fb4dd010740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d41aeb70 con 0x562ce1da0630 2024-04-25T00:21:25.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.718+0000 7fb4dd010740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d41ac9c0 con 0x562ce1dab620 2024-04-25T00:21:25.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.718+0000 7fb4cbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0f50 0x562ce1db7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.719+0000 7fb4cbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0f50 0x562ce1db7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48908/0 (socket says 172.21.15.107:48908) 2024-04-25T00:21:25.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.719+0000 7fb4cbfff640 1 -- 172.21.15.107:0/748556729 learned_addr learned my addr 172.21.15.107:0/748556729 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:25.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.719+0000 7fb4d37fe640 1 --2- 172.21.15.107:0/748556729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ce1da0630 0x562ce1daad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.719+0000 7fb4cbfff640 1 -- 172.21.15.107:0/748556729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ce1dab620 msgr2=0x562ce1da8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:25.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.719+0000 7fb4cbfff640 1 --2- 172.21.15.107:0/748556729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ce1dab620 0x562ce1da8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.719+0000 7fb4cbfff640 1 -- 172.21.15.107:0/748556729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ce1da0630 msgr2=0x562ce1daad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:25.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.719+0000 7fb4cbfff640 1 --2- 172.21.15.107:0/748556729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ce1da0630 0x562ce1daad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.719+0000 7fb4cbfff640 1 -- 172.21.15.107:0/748556729 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ce1d9c270 con 0x562ce1da0f50 2024-04-25T00:21:25.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.719+0000 7fb4d37fe640 1 --2- 172.21.15.107:0/748556729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ce1da0630 0x562ce1daad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:25.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.719+0000 7fb4cbfff640 1 --2- 172.21.15.107:0/748556729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0f50 0x562ce1db7300 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0ed890 tx=0x7fb4bc0ba570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761b2ae10acd496e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:25.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.719+0000 7fb4c97fa640 1 -- 172.21.15.107:0/748556729 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0bbe30 con 0x562ce1da0f50 2024-04-25T00:21:25.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.719+0000 7fb4c97fa640 1 -- 172.21.15.107:0/748556729 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4bc03bd60 con 0x562ce1da0f50 2024-04-25T00:21:25.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.719+0000 7fb4c97fa640 1 -- 172.21.15.107:0/748556729 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc0c4840 con 0x562ce1da0f50 2024-04-25T00:21:25.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.720+0000 7fb4dd010740 1 -- 172.21.15.107:0/748556729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0f50 msgr2=0x562ce1db7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.720+0000 7fb4dd010740 1 --2- 172.21.15.107:0/748556729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0f50 0x562ce1db7300 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0ed890 tx=0x7fb4bc0ba570 comp rx=0 tx=0).stop 2024-04-25T00:21:25.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.720+0000 7fb4dd010740 1 -- 172.21.15.107:0/748556729 shutdown_connections 2024-04-25T00:21:25.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.720+0000 7fb4dd010740 1 --2- 172.21.15.107:0/748556729 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ce1dab620 0x562ce1da8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.720+0000 7fb4dd010740 1 --2- 172.21.15.107:0/748556729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0f50 0x562ce1db7300 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.720+0000 7fb4dd010740 1 --2- 172.21.15.107:0/748556729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ce1da0630 0x562ce1daad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.720+0000 7fb4dd010740 1 -- 172.21.15.107:0/748556729 >> 172.21.15.107:0/748556729 conn(0x562ce1d9c950 msgr2=0x562ce1da85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:25.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.720+0000 7fb4dd010740 1 -- 172.21.15.107:0/748556729 shutdown_connections 2024-04-25T00:21:25.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.720+0000 7fb4dd010740 1 -- 172.21.15.107:0/748556729 wait complete. 2024-04-25T00:21:25.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.720+0000 7fb4dd010740 1 -- start start 2024-04-25T00:21:25.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.720+0000 7fb4dd010740 1 Processor -- start 2024-04-25T00:21:25.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.720+0000 7fb4dd010740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ce1da0630 0x562ce1daf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.720+0000 7fb4dd010740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0f50 0x562ce1daf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.720+0000 7fb4dd010740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ce1dab620 0x562ce1dafe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.720+0000 7fb4dd010740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d41ad030 con 0x562ce1da0f50 2024-04-25T00:21:25.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.720+0000 7fb4dd010740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d41ae4a0 con 0x562ce1da0630 2024-04-25T00:21:25.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.720+0000 7fb4dd010740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d41ad700 con 0x562ce1dab620 2024-04-25T00:21:25.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.720+0000 7fb4dd010740 10 client.0 _ll_register_callbacks cb 0x7fb4bc009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:25.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.720+0000 7fb4cbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0f50 0x562ce1daf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.720+0000 7fb4cbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0f50 0x562ce1daf8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48924/0 (socket says 172.21.15.107:48924) 2024-04-25T00:21:25.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.721+0000 7fb4cbfff640 1 -- 172.21.15.107:0/489513418 learned_addr learned my addr 172.21.15.107:0/489513418 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:25.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.721+0000 7fb4d37fe640 1 --2- 172.21.15.107:0/489513418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ce1da0630 0x562ce1daf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.721+0000 7fb4d3fff640 1 --2- 172.21.15.107:0/489513418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ce1dab620 0x562ce1dafe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.721+0000 7fb4cbfff640 1 -- 172.21.15.107:0/489513418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ce1dab620 msgr2=0x562ce1dafe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:25.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.721+0000 7fb4cbfff640 1 --2- 172.21.15.107:0/489513418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ce1dab620 0x562ce1dafe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.721+0000 7fb4cbfff640 1 -- 172.21.15.107:0/489513418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ce1da0630 msgr2=0x562ce1daf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:25.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.721+0000 7fb4cbfff640 1 --2- 172.21.15.107:0/489513418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ce1da0630 0x562ce1daf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.721+0000 7fb4cbfff640 1 -- 172.21.15.107:0/489513418 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ce1e210b0 con 0x562ce1da0f50 2024-04-25T00:21:25.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.721+0000 7fb4cbfff640 1 --2- 172.21.15.107:0/489513418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0f50 0x562ce1daf8e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0f01d0 tx=0x7fb4bc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7a0d11e5f961a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:25.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.721+0000 7fb4b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:21:25.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.721+0000 7fb4b2ffd640 1 -- 172.21.15.107:0/489513418 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc049040 con 0x562ce1da0f50 2024-04-25T00:21:25.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.721+0000 7fb4b2ffd640 1 -- 172.21.15.107:0/489513418 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fb4bc031bd0 con 0x562ce1da0f50 2024-04-25T00:21:25.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.721+0000 7fb4dd010740 1 -- 172.21.15.107:0/489513418 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4bc03bd60 con 0x562ce1da0f50 2024-04-25T00:21:25.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.721+0000 7fb4b2ffd640 1 -- 172.21.15.107:0/489513418 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb4bc0bbe30 con 0x562ce1da0f50 2024-04-25T00:21:25.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.721+0000 7fb4dd010740 10 client.16746 fetch_fsmap learned FSMap version 29 2024-04-25T00:21:25.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.721+0000 7fb4dd010740 1 -- 172.21.15.107:0/489513418 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562ce1e210b0 con 0x562ce1da0f50 2024-04-25T00:21:25.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.721+0000 7fb4b2ffd640 1 -- 172.21.15.107:0/489513418 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4bc03bd60 con 0x562ce1da0f50 2024-04-25T00:21:25.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.721+0000 7fb4b2ffd640 1 -- 172.21.15.107:0/489513418 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fb4bc0bb4b0 con 0x562ce1da0f50 2024-04-25T00:21:25.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.721+0000 7fb4dd010740 10 client.16746 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:21:25.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.721+0000 7fb4dd010740 10 client.16746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:25.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.721+0000 7fb4dd010740 20 client.16746 populate_metadata read hostname 'smithi107' 2024-04-25T00:21:25.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.721+0000 7fb4dd010740 1 --2- 172.21.15.107:0/489513418 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562ce1e22650 0x562ce1e24a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:25.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.721+0000 7fb4dd010740 4 client.16746 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:21:25.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.721+0000 7fb4dd010740 1 -- 172.21.15.107:0/489513418 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562ce1d0cf40 con 0x562ce1e22650 2024-04-25T00:21:25.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.722+0000 7fb4d37fe640 1 --2- 172.21.15.107:0/489513418 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562ce1e22650 0x562ce1e24a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:25.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.722+0000 7fb4b2ffd640 10 client.16746 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:25.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.722+0000 7fb4d37fe640 1 --2- 172.21.15.107:0/489513418 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562ce1e22650 0x562ce1e24a70 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad073412b3bd4385 server_cookie=a3d43be4f602346f in_seq=0 out_seq=0 2024-04-25T00:21:25.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.722+0000 7fb4b2ffd640 1 -- 172.21.15.107:0/489513418 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562ce1d0cf40 con 0x562ce1e22650 2024-04-25T00:21:25.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.722+0000 7fb4b2ffd640 10 client.16746 handle_command_reply: tid=0 2024-04-25T00:21:25.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.722+0000 7fb4dd010740 1 client.16746 shutdown 2024-04-25T00:21:25.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.722+0000 7fb4dd010740 1 -- 172.21.15.107:0/489513418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0f50 msgr2=0x562ce1daf8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.722+0000 7fb4dd010740 1 --2- 172.21.15.107:0/489513418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0f50 0x562ce1daf8e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0f01d0 tx=0x7fb4bc009260 comp rx=0 tx=0).stop 2024-04-25T00:21:25.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.722+0000 7fb4dd010740 1 -- 172.21.15.107:0/489513418 shutdown_connections 2024-04-25T00:21:25.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.722+0000 7fb4dd010740 1 --2- 172.21.15.107:0/489513418 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562ce1e22650 0x562ce1e24a70 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.722+0000 7fb4dd010740 1 --2- 172.21.15.107:0/489513418 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562ce1dab620 0x562ce1dafe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.722+0000 7fb4dd010740 1 --2- 172.21.15.107:0/489513418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ce1da0f50 0x562ce1daf8e0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.722+0000 7fb4b2ffd640 0 client.16746 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:25.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.722+0000 7fb4dd010740 1 --2- 172.21.15.107:0/489513418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562ce1da0630 0x562ce1daf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.722+0000 7fb4dd010740 1 -- 172.21.15.107:0/489513418 >> 172.21.15.107:0/489513418 conn(0x562ce1d9c950 msgr2=0x562ce1d9baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:25.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.722+0000 7fb4dd010740 1 -- 172.21.15.107:0/489513418 shutdown_connections 2024-04-25T00:21:25.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.722+0000 7fb4dd010740 1 -- 172.21.15.107:0/489513418 wait complete. 2024-04-25T00:21:25.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.723+0000 7fb4dd010740 20 client.16746 trim_cache size 0 max 16384 2024-04-25T00:21:25.732 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:21:25.732 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:21:25.732 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-04-25T00:21:25.732 INFO:teuthology.orchestra.run.smithi107.stdout: "96f3e171-d5be-4730-9a05-fe2b6791e1ff": { 2024-04-25T00:21:25.732 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-04-25T00:21:25.732 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "96f3e171-d5be-4730-9a05-fe2b6791e1ff", 2024-04-25T00:21:25.732 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-04-25T00:21:25.732 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:21:25.732 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:21:25.732 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:21:25.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.723+0000 7fb4da22c640 1 -- 172.21.15.107:0/4112636830 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb4c00a2d00 msgr2=0x7fb4c00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.723+0000 7fb4da22c640 1 --2- 172.21.15.107:0/4112636830 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb4c00a2d00 0x7fb4c00a51c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc005fd0 tx=0x7fb4cc00f730 comp rx=0 tx=0).stop 2024-04-25T00:21:25.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.723+0000 7fb4da22c640 1 -- 172.21.15.107:0/4112636830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4d4100fd0 msgr2=0x7fb4d41984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:25.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.723+0000 7fb4da22c640 1 --2- 172.21.15.107:0/4112636830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4d4100fd0 0x7fb4d41984c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc009230 tx=0x7fb4bc004270 comp rx=0 tx=0).stop 2024-04-25T00:21:25.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.723+0000 7fb4da22c640 1 -- 172.21.15.107:0/4112636830 shutdown_connections 2024-04-25T00:21:25.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.723+0000 7fb4da22c640 1 --2- 172.21.15.107:0/4112636830 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb4c00a2d00 0x7fb4c00a51c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.723+0000 7fb4da22c640 1 --2- 172.21.15.107:0/4112636830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d41066b0 0x7fb4d419c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.723+0000 7fb4da22c640 1 --2- 172.21.15.107:0/4112636830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4d4100fd0 0x7fb4d41984c0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.723+0000 7fb4da22c640 1 --2- 172.21.15.107:0/4112636830 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb4d4100610 0x7fb4d4197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:25.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.723+0000 7fb4da22c640 1 -- 172.21.15.107:0/4112636830 >> 172.21.15.107:0/4112636830 conn(0x7fb4d4077740 msgr2=0x7fb4d41047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:25.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.723+0000 7fb4da22c640 1 -- 172.21.15.107:0/4112636830 shutdown_connections 2024-04-25T00:21:25.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:25.723+0000 7fb4da22c640 1 -- 172.21.15.107:0/4112636830 wait complete. 2024-04-25T00:21:25.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:25 smithi107 ceph-mon[29153]: pgmap v152: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:21:25.735 INFO:tasks.cephfs.filesystem:scrub status for tag:96f3e171-d5be-4730-9a05-fe2b6791e1ff - {'path': '/', 'tag': '96f3e171-d5be-4730-9a05-fe2b6791e1ff', 'options': 'recursive,force'} 2024-04-25T00:21:25.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:25 smithi195 ceph-mon[32833]: pgmap v152: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:21:27.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:27 smithi170 ceph-mon[32807]: scrub queued for path: / 2024-04-25T00:21:27.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:27 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:21:27.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:27 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:21:27.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:27 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:21:27.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:27 smithi170 ceph-mon[32807]: pgmap v153: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:21:27.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:27 smithi107 ceph-mon[29153]: scrub queued for path: / 2024-04-25T00:21:27.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:27 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:21:27.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:27 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:21:27.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:27 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:21:27.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:27 smithi107 ceph-mon[29153]: pgmap v153: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:21:27.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:27 smithi195 ceph-mon[32833]: scrub queued for path: / 2024-04-25T00:21:27.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:27 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:21:27.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:27 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:21:27.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:27 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:21:27.840 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:27 smithi195 ceph-mon[32833]: pgmap v153: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:21:28.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:28 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:21:28.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:28 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:21:28.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:28 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:21:29.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:29 smithi170 ceph-mon[32807]: pgmap v154: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:21:29.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:29 smithi107 ceph-mon[29153]: pgmap v154: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:21:29.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:29 smithi195 ceph-mon[32833]: pgmap v154: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:21:31.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:31 smithi170 ceph-mon[32807]: pgmap v155: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:21:31.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:31 smithi107 ceph-mon[29153]: pgmap v155: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:21:31.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:31 smithi195 ceph-mon[32833]: pgmap v155: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:21:33.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:33 smithi170 ceph-mon[32807]: pgmap v156: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:21:33.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:33 smithi107 ceph-mon[29153]: pgmap v156: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:21:33.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:33 smithi195 ceph-mon[32833]: pgmap v156: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:21:35.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:35 smithi170 ceph-mon[32807]: pgmap v157: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:21:35.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:35 smithi107 ceph-mon[29153]: pgmap v157: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:21:35.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:35 smithi195 ceph-mon[32833]: pgmap v157: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:21:36.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:36 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:21:36.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:36 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:21:36.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:36 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:21:37.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:37 smithi170 ceph-mon[32807]: pgmap v158: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:21:37.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:37 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:21:37.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:37 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:21:37.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:37 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:21:37.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:37 smithi107 ceph-mon[29153]: pgmap v158: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:21:37.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:37 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:21:37.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:37 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:21:37.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:37 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:21:37.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:37 smithi195 ceph-mon[32833]: pgmap v158: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:21:37.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:37 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:21:37.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:37 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:21:37.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:37 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:21:38.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:38 smithi107 ceph-mon[29153]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:21:38.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:38 smithi195 ceph-mon[32833]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:21:38.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:38 smithi170 ceph-mon[32807]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:21:39.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:39 smithi107 ceph-mon[29153]: pgmap v159: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:21:39.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:39 smithi195 ceph-mon[32833]: pgmap v159: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:21:39.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:39 smithi170 ceph-mon[32807]: pgmap v159: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:21:41.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:41 smithi107 ceph-mon[29153]: pgmap v160: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:21:41.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:41 smithi195 ceph-mon[32833]: pgmap v160: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:21:41.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:41 smithi170 ceph-mon[32807]: pgmap v160: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:21:43.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:43 smithi107 ceph-mon[29153]: pgmap v161: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:21:43.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:43 smithi195 ceph-mon[32833]: pgmap v161: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:21:43.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:43 smithi170 ceph-mon[32807]: pgmap v161: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:21:45.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:45 smithi107 ceph-mon[29153]: pgmap v162: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:21:45.737 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:45 smithi170 ceph-mon[32807]: pgmap v162: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:21:45.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:45 smithi195 ceph-mon[32833]: pgmap v162: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:21:47.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:47 smithi107 ceph-mon[29153]: pgmap v163: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:21:47.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:47 smithi195 ceph-mon[32833]: pgmap v163: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:21:47.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:47 smithi170 ceph-mon[32807]: pgmap v163: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:21:49.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:49 smithi107 ceph-mon[29153]: pgmap v164: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:21:49.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:49 smithi195 ceph-mon[32833]: pgmap v164: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:21:49.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:49 smithi170 ceph-mon[32807]: pgmap v164: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:21:51.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:51 smithi107 ceph-mon[29153]: pgmap v165: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:21:51.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:51 smithi195 ceph-mon[32833]: pgmap v165: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:21:51.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:51 smithi170 ceph-mon[32807]: pgmap v165: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:21:53.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:53 smithi107 ceph-mon[29153]: pgmap v166: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:21:53.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:53 smithi195 ceph-mon[32833]: pgmap v166: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:21:53.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:53 smithi170 ceph-mon[32807]: pgmap v166: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:21:55.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:55 smithi107 ceph-mon[29153]: pgmap v167: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:21:55.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:21:55.737 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:21:55.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:55 smithi195 ceph-mon[32833]: pgmap v167: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:21:55.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.848+0000 7f85cbb5a640 1 -- 172.21.15.107:0/1613353262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85c406a300 msgr2=0x7f85c406a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:55.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.848+0000 7f85cbb5a640 1 --2- 172.21.15.107:0/1613353262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85c406a300 0x7f85c406a760 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009900 tx=0x7f85b402a410 comp rx=0 tx=0).stop 2024-04-25T00:21:55.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.848+0000 7f85cbb5a640 1 -- 172.21.15.107:0/1613353262 shutdown_connections 2024-04-25T00:21:55.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.848+0000 7f85cbb5a640 1 --2- 172.21.15.107:0/1613353262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85c4104170 0x7f85c41045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.848+0000 7f85cbb5a640 1 --2- 172.21.15.107:0/1613353262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85c406a300 0x7f85c406a760 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.848+0000 7f85cbb5a640 1 --2- 172.21.15.107:0/1613353262 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85c4106670 0x7f85c4069dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.848+0000 7f85cbb5a640 1 -- 172.21.15.107:0/1613353262 >> 172.21.15.107:0/1613353262 conn(0x7f85c40776a0 msgr2=0x7f85c4077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:55.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.848+0000 7f85cbb5a640 1 -- 172.21.15.107:0/1613353262 shutdown_connections 2024-04-25T00:21:55.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.848+0000 7f85cbb5a640 1 -- 172.21.15.107:0/1613353262 wait complete. 2024-04-25T00:21:55.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.849+0000 7f85cbb5a640 1 Processor -- start 2024-04-25T00:21:55.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.849+0000 7f85cbb5a640 1 -- start start 2024-04-25T00:21:55.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.849+0000 7f85cbb5a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85c406a300 0x7f85c419e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.849+0000 7f85cbb5a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85c4104170 0x7f85c419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.849+0000 7f85cbb5a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85c4106670 0x7f85c41a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.849+0000 7f85cbb5a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41180f0 con 0x7f85c406a300 2024-04-25T00:21:55.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.849+0000 7f85cbb5a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f85c4117f70 con 0x7f85c4104170 2024-04-25T00:21:55.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.849+0000 7f85cbb5a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f85c4118270 con 0x7f85c4106670 2024-04-25T00:21:55.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.849+0000 7f85c98f6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85c406a300 0x7f85c419e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.849+0000 7f85c98f6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85c406a300 0x7f85c419e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55516/0 (socket says 172.21.15.107:55516) 2024-04-25T00:21:55.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.849+0000 7f85c98f6640 1 -- 172.21.15.107:0/3944298707 learned_addr learned my addr 172.21.15.107:0/3944298707 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:55.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.849+0000 7f85c90f5640 1 --2- 172.21.15.107:0/3944298707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85c4104170 0x7f85c419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.849+0000 7f85ca0f7640 1 --2- 172.21.15.107:0/3944298707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85c4106670 0x7f85c41a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.849+0000 7f85c98f6640 1 -- 172.21.15.107:0/3944298707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85c4106670 msgr2=0x7f85c41a2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:55.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.849+0000 7f85c98f6640 1 --2- 172.21.15.107:0/3944298707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85c4106670 0x7f85c41a2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.849+0000 7f85c98f6640 1 -- 172.21.15.107:0/3944298707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85c4104170 msgr2=0x7f85c419eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:55.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.849+0000 7f85c98f6640 1 --2- 172.21.15.107:0/3944298707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85c4104170 0x7f85c419eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.849+0000 7f85c98f6640 1 -- 172.21.15.107:0/3944298707 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c41a3570 con 0x7f85c406a300 2024-04-25T00:21:55.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.849+0000 7f85ca0f7640 1 --2- 172.21.15.107:0/3944298707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85c4106670 0x7f85c41a2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:55.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.849+0000 7f85c90f5640 1 --2- 172.21.15.107:0/3944298707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85c4104170 0x7f85c419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:55.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.850+0000 7f85c98f6640 1 --2- 172.21.15.107:0/3944298707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85c406a300 0x7f85c419e5d0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f85c000c970 tx=0x7f85c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:55.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.850+0000 7f85baffd640 1 -- 172.21.15.107:0/3944298707 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c0007bb0 con 0x7f85c406a300 2024-04-25T00:21:55.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.850+0000 7f85cbb5a640 1 -- 172.21.15.107:0/3944298707 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85c41a3860 con 0x7f85c406a300 2024-04-25T00:21:55.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.850+0000 7f85baffd640 1 -- 172.21.15.107:0/3944298707 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c0007d50 con 0x7f85c406a300 2024-04-25T00:21:55.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.850+0000 7f85baffd640 1 -- 172.21.15.107:0/3944298707 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c000f450 con 0x7f85c406a300 2024-04-25T00:21:55.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.850+0000 7f85cbb5a640 1 -- 172.21.15.107:0/3944298707 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85c41ab0f0 con 0x7f85c406a300 2024-04-25T00:21:55.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.852+0000 7f85baffd640 1 -- 172.21.15.107:0/3944298707 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f85c0005020 con 0x7f85c406a300 2024-04-25T00:21:55.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.853+0000 7f85ce866740 1 Processor -- start 2024-04-25T00:21:55.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.853+0000 7f85ce866740 1 -- start start 2024-04-25T00:21:55.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.853+0000 7f85baffd640 1 --2- 172.21.15.107:0/3944298707 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f85b00a2d00 0x7f85b00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.853+0000 7f85ce866740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f89630 0x55cfc9f89a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.853+0000 7f85ce866740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cfc9f89f50 0x55cfc9f940e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.853+0000 7f85ce866740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfc9f94620 0x55cfc9f96a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.853+0000 7f85ce866740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cfc9eee6b0 con 0x55cfc9f89630 2024-04-25T00:21:55.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.853+0000 7f85ce866740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cfc9e96880 con 0x55cfc9f89f50 2024-04-25T00:21:55.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.853+0000 7f85ce866740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cfc9eea670 con 0x55cfc9f94620 2024-04-25T00:21:55.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.854+0000 7f85ca0f7640 1 --2- 172.21.15.107:0/3944298707 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f85b00a2d00 0x7f85b00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.854+0000 7f85c98f6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f89630 0x55cfc9f89a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.854+0000 7f85c98f6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f89630 0x55cfc9f89a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55520/0 (socket says 172.21.15.107:55520) 2024-04-25T00:21:55.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.854+0000 7f85c98f6640 1 -- 172.21.15.107:0/305094153 learned_addr learned my addr 172.21.15.107:0/305094153 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:55.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.854+0000 7f85baffd640 1 -- 172.21.15.107:0/3944298707 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f85c00cf660 con 0x7f85c406a300 2024-04-25T00:21:55.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.854+0000 7f85ca0f7640 1 --2- 172.21.15.107:0/305094153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfc9f94620 0x55cfc9f96a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.854+0000 7f85c90f5640 1 --2- 172.21.15.107:0/305094153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cfc9f89f50 0x55cfc9f940e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.854+0000 7f85c98f6640 1 -- 172.21.15.107:0/305094153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfc9f94620 msgr2=0x55cfc9f96a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:55.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.854+0000 7f85c98f6640 1 --2- 172.21.15.107:0/305094153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfc9f94620 0x55cfc9f96a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.854+0000 7f85c98f6640 1 -- 172.21.15.107:0/305094153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cfc9f89f50 msgr2=0x55cfc9f940e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:55.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.854+0000 7f85c98f6640 1 --2- 172.21.15.107:0/305094153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cfc9f89f50 0x55cfc9f940e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.854+0000 7f85c98f6640 1 -- 172.21.15.107:0/305094153 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfc9ccc7d0 con 0x55cfc9f89630 2024-04-25T00:21:55.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.854+0000 7f85ca0f7640 1 --2- 172.21.15.107:0/3944298707 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f85b00a2d00 0x7f85b00a51c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0146e0 tx=0x7f85bc014ae0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:55.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.854+0000 7f85c98f6640 1 --2- 172.21.15.107:0/305094153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f89630 0x55cfc9f89a10 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f85c00cc920 tx=0x7f85c0050e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b455d8b58eb84b22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:55.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.854+0000 7f85affff640 1 -- 172.21.15.107:0/305094153 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c001d070 con 0x55cfc9f89630 2024-04-25T00:21:55.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.854+0000 7f85affff640 1 -- 172.21.15.107:0/305094153 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c009beb0 con 0x55cfc9f89630 2024-04-25T00:21:55.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.854+0000 7f85affff640 1 -- 172.21.15.107:0/305094153 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c00a4420 con 0x55cfc9f89630 2024-04-25T00:21:55.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.855+0000 7f85ce866740 1 -- 172.21.15.107:0/305094153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f89630 msgr2=0x55cfc9f89a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:55.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.855+0000 7f85ce866740 1 --2- 172.21.15.107:0/305094153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f89630 0x55cfc9f89a10 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f85c00cc920 tx=0x7f85c0050e70 comp rx=0 tx=0).stop 2024-04-25T00:21:55.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.855+0000 7f85ce866740 1 -- 172.21.15.107:0/305094153 shutdown_connections 2024-04-25T00:21:55.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.855+0000 7f85ce866740 1 --2- 172.21.15.107:0/305094153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfc9f94620 0x55cfc9f96a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.855+0000 7f85ce866740 1 --2- 172.21.15.107:0/305094153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cfc9f89f50 0x55cfc9f940e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.855+0000 7f85ce866740 1 --2- 172.21.15.107:0/305094153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f89630 0x55cfc9f89a10 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.855+0000 7f85ce866740 1 -- 172.21.15.107:0/305094153 >> 172.21.15.107:0/305094153 conn(0x55cfc9f85950 msgr2=0x55cfc9f85d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:55.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.855+0000 7f85ce866740 1 -- 172.21.15.107:0/305094153 shutdown_connections 2024-04-25T00:21:55.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.855+0000 7f85ce866740 1 -- 172.21.15.107:0/305094153 wait complete. 2024-04-25T00:21:55.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.855+0000 7f85ce866740 1 -- start start 2024-04-25T00:21:55.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.856+0000 7f85ce866740 1 Processor -- start 2024-04-25T00:21:55.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.856+0000 7f85ce866740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfc9f89630 0x55cfc9f91580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.856+0000 7f85ce866740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cfc9f89f50 0x55cfc9f91ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.856+0000 7f85ce866740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f94620 0x55cfc9f92000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.856+0000 7f85ce866740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cfc9f973b0 con 0x55cfc9f94620 2024-04-25T00:21:55.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.856+0000 7f85ce866740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cfc9ea9f50 con 0x55cfc9f89f50 2024-04-25T00:21:55.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.856+0000 7f85ce866740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cfc9f97530 con 0x55cfc9f89630 2024-04-25T00:21:55.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.856+0000 7f85ce866740 10 client.0 _ll_register_callbacks cb 0x55cfc9ee5000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:55.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.856+0000 7f85ca0f7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f94620 0x55cfc9f92000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.856+0000 7f85c98f6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfc9f89630 0x55cfc9f91580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.856+0000 7f85ca0f7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f94620 0x55cfc9f92000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55530/0 (socket says 172.21.15.107:55530) 2024-04-25T00:21:55.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.856+0000 7f85c98f6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfc9f89630 0x55cfc9f91580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:42464/0 (socket says 172.21.15.107:42464) 2024-04-25T00:21:55.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.856+0000 7f85ca0f7640 1 -- 172.21.15.107:0/2146107440 learned_addr learned my addr 172.21.15.107:0/2146107440 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:55.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.856+0000 7f85c90f5640 1 --2- 172.21.15.107:0/2146107440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cfc9f89f50 0x55cfc9f91ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.856+0000 7f85ca0f7640 1 -- 172.21.15.107:0/2146107440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfc9f89630 msgr2=0x55cfc9f91580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:55.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.856+0000 7f85ca0f7640 1 --2- 172.21.15.107:0/2146107440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfc9f89630 0x55cfc9f91580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.856+0000 7f85ca0f7640 1 -- 172.21.15.107:0/2146107440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cfc9f89f50 msgr2=0x55cfc9f91ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:55.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.857+0000 7f85ca0f7640 1 --2- 172.21.15.107:0/2146107440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cfc9f89f50 0x55cfc9f91ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.857+0000 7f85ca0f7640 1 -- 172.21.15.107:0/2146107440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfc9c023f0 con 0x55cfc9f94620 2024-04-25T00:21:55.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.857+0000 7f85c98f6640 1 --2- 172.21.15.107:0/2146107440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfc9f89630 0x55cfc9f91580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:55.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.857+0000 7f85ca0f7640 1 --2- 172.21.15.107:0/2146107440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f94620 0x55cfc9f92000 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f85bc011dc0 tx=0x7f85bc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c85f02528a29b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:55.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.857+0000 7f85ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:21:55.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.857+0000 7f85ad7fa640 1 -- 172.21.15.107:0/2146107440 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc01f630 con 0x55cfc9f94620 2024-04-25T00:21:55.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.857+0000 7f85ad7fa640 1 -- 172.21.15.107:0/2146107440 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f85bc01f7d0 con 0x55cfc9f94620 2024-04-25T00:21:55.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.857+0000 7f85ce866740 1 -- 172.21.15.107:0/2146107440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85c009beb0 con 0x55cfc9f94620 2024-04-25T00:21:55.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.857+0000 7f85ad7fa640 1 -- 172.21.15.107:0/2146107440 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f85bc01f9d0 con 0x55cfc9f94620 2024-04-25T00:21:55.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.857+0000 7f85ce866740 10 client.16758 fetch_fsmap learned FSMap version 30 2024-04-25T00:21:55.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.857+0000 7f85ce866740 1 -- 172.21.15.107:0/2146107440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cfc9bc20f0 con 0x55cfc9f94620 2024-04-25T00:21:55.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.857+0000 7f85ad7fa640 1 -- 172.21.15.107:0/2146107440 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85c009beb0 con 0x55cfc9f94620 2024-04-25T00:21:55.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.857+0000 7f85ad7fa640 1 -- 172.21.15.107:0/2146107440 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f85bc01e3c0 con 0x55cfc9f94620 2024-04-25T00:21:55.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.857+0000 7f85ce866740 10 client.16758 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:21:55.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.857+0000 7f85ce866740 10 client.16758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:55.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.857+0000 7f85ce866740 20 client.16758 populate_metadata read hostname 'smithi107' 2024-04-25T00:21:55.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.857+0000 7f85ce866740 1 --2- 172.21.15.107:0/2146107440 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55cfc9fa4870 0x55cfc9fa6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.857+0000 7f85ce866740 4 client.16758 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:55.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.857+0000 7f85ce866740 1 -- 172.21.15.107:0/2146107440 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfc9efbbf0 con 0x55cfc9fa4870 2024-04-25T00:21:55.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.858+0000 7f85c98f6640 1 --2- 172.21.15.107:0/2146107440 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55cfc9fa4870 0x55cfc9fa6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.858+0000 7f85ad7fa640 10 client.16758 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:55.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.858+0000 7f85c98f6640 1 --2- 172.21.15.107:0/2146107440 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55cfc9fa4870 0x55cfc9fa6cd0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4374b794b9af5a37 server_cookie=26809817ef00e9bb in_seq=0 out_seq=0 2024-04-25T00:21:55.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.859+0000 7f85ad7fa640 1 -- 172.21.15.107:0/2146107440 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cfc9efbbf0 con 0x55cfc9fa4870 2024-04-25T00:21:55.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.859+0000 7f85ad7fa640 10 client.16758 handle_command_reply: tid=0 2024-04-25T00:21:55.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.859+0000 7f85ce866740 1 client.16758 shutdown 2024-04-25T00:21:55.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.859+0000 7f85ce866740 1 -- 172.21.15.107:0/2146107440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f94620 msgr2=0x55cfc9f92000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:55.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.859+0000 7f85ce866740 1 --2- 172.21.15.107:0/2146107440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f94620 0x55cfc9f92000 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f85bc011dc0 tx=0x7f85bc004ab0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.859+0000 7f85ce866740 1 -- 172.21.15.107:0/2146107440 shutdown_connections 2024-04-25T00:21:55.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.859+0000 7f85ce866740 1 --2- 172.21.15.107:0/2146107440 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55cfc9fa4870 0x55cfc9fa6cd0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.859+0000 7f85ce866740 1 --2- 172.21.15.107:0/2146107440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f94620 0x55cfc9f92000 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.859+0000 7f85ce866740 1 --2- 172.21.15.107:0/2146107440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cfc9f89f50 0x55cfc9f91ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.859+0000 7f85ce866740 1 --2- 172.21.15.107:0/2146107440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfc9f89630 0x55cfc9f91580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.859+0000 7f85ad7fa640 0 client.16758 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:55.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.859+0000 7f85ce866740 1 -- 172.21.15.107:0/2146107440 >> 172.21.15.107:0/2146107440 conn(0x55cfc9f85950 msgr2=0x55cfc9f8b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:55.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.859+0000 7f85ce866740 1 -- 172.21.15.107:0/2146107440 shutdown_connections 2024-04-25T00:21:55.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.859+0000 7f85ce866740 1 -- 172.21.15.107:0/2146107440 wait complete. 2024-04-25T00:21:55.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.859+0000 7f85ce866740 20 client.16758 trim_cache size 0 max 16384 2024-04-25T00:21:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.877+0000 7f85ce866740 1 Processor -- start 2024-04-25T00:21:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.877+0000 7f85ce866740 1 -- start start 2024-04-25T00:21:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.877+0000 7f85ce866740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cfc9f89630 0x55cfc9fa0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.877+0000 7f85ce866740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfc9f89f50 0x55cfc9f8bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.877+0000 7f85ce866740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f94620 0x55cfc9f84970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.877+0000 7f85ce866740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41b79b0 con 0x55cfc9f94620 2024-04-25T00:21:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.877+0000 7f85ce866740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41b51f0 con 0x55cfc9f89630 2024-04-25T00:21:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.877+0000 7f85ce866740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41b3040 con 0x55cfc9f89f50 2024-04-25T00:21:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.877+0000 7f85ca0f7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f94620 0x55cfc9f84970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.877+0000 7f85ca0f7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f94620 0x55cfc9f84970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55536/0 (socket says 172.21.15.107:55536) 2024-04-25T00:21:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.877+0000 7f85ca0f7640 1 -- 172.21.15.107:0/858009856 learned_addr learned my addr 172.21.15.107:0/858009856 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.877+0000 7f85ca0f7640 1 -- 172.21.15.107:0/858009856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfc9f89f50 msgr2=0x55cfc9f8bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:55.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.877+0000 7f85ca0f7640 1 --2- 172.21.15.107:0/858009856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfc9f89f50 0x55cfc9f8bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.877+0000 7f85ca0f7640 1 -- 172.21.15.107:0/858009856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cfc9f89630 msgr2=0x55cfc9fa0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.877+0000 7f85ca0f7640 1 --2- 172.21.15.107:0/858009856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cfc9f89630 0x55cfc9fa0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.877+0000 7f85ca0f7640 1 -- 172.21.15.107:0/858009856 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfc9f8a450 con 0x55cfc9f94620 2024-04-25T00:21:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.877+0000 7f85ca0f7640 1 --2- 172.21.15.107:0/858009856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f94620 0x55cfc9f84970 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f85bc011450 tx=0x7f85bc011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1472cc39a7b0cbbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.878+0000 7f85af7fe640 1 -- 172.21.15.107:0/858009856 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc012cb0 con 0x55cfc9f94620 2024-04-25T00:21:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.878+0000 7f85af7fe640 1 -- 172.21.15.107:0/858009856 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85bc03b070 con 0x55cfc9f94620 2024-04-25T00:21:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.878+0000 7f85af7fe640 1 -- 172.21.15.107:0/858009856 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc01ec30 con 0x55cfc9f94620 2024-04-25T00:21:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.878+0000 7f85ce866740 1 -- 172.21.15.107:0/858009856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f94620 msgr2=0x55cfc9f84970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.878+0000 7f85ce866740 1 --2- 172.21.15.107:0/858009856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f94620 0x55cfc9f84970 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f85bc011450 tx=0x7f85bc011a50 comp rx=0 tx=0).stop 2024-04-25T00:21:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.878+0000 7f85ce866740 1 -- 172.21.15.107:0/858009856 shutdown_connections 2024-04-25T00:21:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.878+0000 7f85ce866740 1 --2- 172.21.15.107:0/858009856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f94620 0x55cfc9f84970 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.878+0000 7f85ce866740 1 --2- 172.21.15.107:0/858009856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfc9f89f50 0x55cfc9f8bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.878+0000 7f85ce866740 1 --2- 172.21.15.107:0/858009856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cfc9f89630 0x55cfc9fa0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.878+0000 7f85ce866740 1 -- 172.21.15.107:0/858009856 >> 172.21.15.107:0/858009856 conn(0x55cfc9f85950 msgr2=0x55cfc9f84260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.878+0000 7f85ce866740 1 -- 172.21.15.107:0/858009856 shutdown_connections 2024-04-25T00:21:55.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.878+0000 7f85ce866740 1 -- 172.21.15.107:0/858009856 wait complete. 2024-04-25T00:21:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.878+0000 7f85ce866740 1 -- start start 2024-04-25T00:21:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.879+0000 7f85ce866740 1 Processor -- start 2024-04-25T00:21:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.879+0000 7f85ce866740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f89630 0x55cfc9f9c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.879+0000 7f85ce866740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfc9f89f50 0x55cfc9f8ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.879+0000 7f85ce866740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cfc9f94620 0x55cfc9f8e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.879+0000 7f85ce866740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41b36b0 con 0x55cfc9f89630 2024-04-25T00:21:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.879+0000 7f85ce866740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41b4b20 con 0x55cfc9f94620 2024-04-25T00:21:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.879+0000 7f85ce866740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f85c41b3d80 con 0x55cfc9f89f50 2024-04-25T00:21:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.879+0000 7f85ce866740 10 client.0 _ll_register_callbacks cb 0x7f85bc008300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.879+0000 7f85c98f6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f89630 0x55cfc9f9c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.879+0000 7f85c98f6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f89630 0x55cfc9f9c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55552/0 (socket says 172.21.15.107:55552) 2024-04-25T00:21:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.879+0000 7f85c98f6640 1 -- 172.21.15.107:0/382645993 learned_addr learned my addr 172.21.15.107:0/382645993 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.879+0000 7f85ca0f7640 1 --2- 172.21.15.107:0/382645993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cfc9f94620 0x55cfc9f8e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.879+0000 7f85c98f6640 1 -- 172.21.15.107:0/382645993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfc9f89f50 msgr2=0x55cfc9f8ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:55.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.879+0000 7f85c98f6640 1 --2- 172.21.15.107:0/382645993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfc9f89f50 0x55cfc9f8ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.879+0000 7f85c98f6640 1 -- 172.21.15.107:0/382645993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cfc9f94620 msgr2=0x55cfc9f8e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.879+0000 7f85c98f6640 1 --2- 172.21.15.107:0/382645993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cfc9f94620 0x55cfc9f8e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.879+0000 7f85c98f6640 1 -- 172.21.15.107:0/382645993 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfc9f8e8d0 con 0x55cfc9f89630 2024-04-25T00:21:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.879+0000 7f85c98f6640 1 --2- 172.21.15.107:0/382645993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f89630 0x55cfc9f9c970 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f85c0098180 tx=0x7f85c000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d1bb7a6993a2e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.879+0000 7f85ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:21:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.880+0000 7f85ad7fa640 1 -- 172.21.15.107:0/382645993 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c001d040 con 0x55cfc9f89630 2024-04-25T00:21:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.880+0000 7f85ad7fa640 1 -- 172.21.15.107:0/382645993 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f85c0099850 con 0x55cfc9f89630 2024-04-25T00:21:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.880+0000 7f85ce866740 1 -- 172.21.15.107:0/382645993 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85bc03b070 con 0x55cfc9f89630 2024-04-25T00:21:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.880+0000 7f85ad7fa640 1 -- 172.21.15.107:0/382645993 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f85c009beb0 con 0x55cfc9f89630 2024-04-25T00:21:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.880+0000 7f85ce866740 10 client.16764 fetch_fsmap learned FSMap version 30 2024-04-25T00:21:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.880+0000 7f85ce866740 1 -- 172.21.15.107:0/382645993 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cfca00abd0 con 0x55cfc9f89630 2024-04-25T00:21:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.880+0000 7f85ad7fa640 1 -- 172.21.15.107:0/382645993 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85bc03b070 con 0x55cfc9f89630 2024-04-25T00:21:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.880+0000 7f85ad7fa640 1 -- 172.21.15.107:0/382645993 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f85c0097e20 con 0x55cfc9f89630 2024-04-25T00:21:55.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.880+0000 7f85ce866740 10 client.16764 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:21:55.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.880+0000 7f85ce866740 10 client.16764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:55.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.880+0000 7f85ce866740 20 client.16764 populate_metadata read hostname 'smithi107' 2024-04-25T00:21:55.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.880+0000 7f85ce866740 1 --2- 172.21.15.107:0/382645993 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55cfca00af10 0x55cfca00d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.880+0000 7f85ce866740 4 client.16764 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:21:55.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.880+0000 7f85ce866740 1 -- 172.21.15.107:0/382645993 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cfc9ef5f40 con 0x55cfca00af10 2024-04-25T00:21:55.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.880+0000 7f85c90f5640 1 --2- 172.21.15.107:0/382645993 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55cfca00af10 0x55cfca00d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.881+0000 7f85ad7fa640 10 client.16764 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:55.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.881+0000 7f85c90f5640 1 --2- 172.21.15.107:0/382645993 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55cfca00af10 0x55cfca00d300 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c35da4fd64828104 server_cookie=80286b173a7e08a6 in_seq=0 out_seq=0 2024-04-25T00:21:55.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.881+0000 7f85ad7fa640 1 -- 172.21.15.107:0/382645993 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cfc9ef5f40 con 0x55cfca00af10 2024-04-25T00:21:55.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.881+0000 7f85ad7fa640 10 client.16764 handle_command_reply: tid=0 2024-04-25T00:21:55.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.881+0000 7f85ce866740 1 client.16764 shutdown 2024-04-25T00:21:55.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.881+0000 7f85ce866740 1 -- 172.21.15.107:0/382645993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f89630 msgr2=0x55cfc9f9c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:55.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.881+0000 7f85ce866740 1 --2- 172.21.15.107:0/382645993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f89630 0x55cfc9f9c970 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f85c0098180 tx=0x7f85c000cf60 comp rx=0 tx=0).stop 2024-04-25T00:21:55.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.881+0000 7f85ce866740 1 -- 172.21.15.107:0/382645993 shutdown_connections 2024-04-25T00:21:55.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.881+0000 7f85ce866740 1 --2- 172.21.15.107:0/382645993 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55cfca00af10 0x55cfca00d300 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.881+0000 7f85ce866740 1 --2- 172.21.15.107:0/382645993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cfc9f94620 0x55cfc9f8e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.881+0000 7f85ce866740 1 --2- 172.21.15.107:0/382645993 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cfc9f89f50 0x55cfc9f8ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.881+0000 7f85ad7fa640 0 client.16764 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:55.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.881+0000 7f85ce866740 1 --2- 172.21.15.107:0/382645993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfc9f89630 0x55cfc9f9c970 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.881+0000 7f85ce866740 1 -- 172.21.15.107:0/382645993 >> 172.21.15.107:0/382645993 conn(0x55cfc9f85950 msgr2=0x55cfc9f91dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:55.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.881+0000 7f85ce866740 1 -- 172.21.15.107:0/382645993 shutdown_connections 2024-04-25T00:21:55.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.881+0000 7f85ce866740 1 -- 172.21.15.107:0/382645993 wait complete. 2024-04-25T00:21:55.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.881+0000 7f85ce866740 20 client.16764 trim_cache size 0 max 16384 2024-04-25T00:21:55.888 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:21:55.889 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-04-25T00:21:55.889 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-04-25T00:21:55.889 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:21:55.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.882+0000 7f85cbb5a640 1 -- 172.21.15.107:0/3944298707 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f85b00a2d00 msgr2=0x7f85b00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:55.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.882+0000 7f85cbb5a640 1 --2- 172.21.15.107:0/3944298707 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f85b00a2d00 0x7f85b00a51c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0146e0 tx=0x7f85bc014ae0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.882+0000 7f85cbb5a640 1 -- 172.21.15.107:0/3944298707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85c406a300 msgr2=0x7f85c419e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:55.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.882+0000 7f85cbb5a640 1 --2- 172.21.15.107:0/3944298707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85c406a300 0x7f85c419e5d0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f85c000c970 tx=0x7f85c000ce30 comp rx=0 tx=0).stop 2024-04-25T00:21:55.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.882+0000 7f85cbb5a640 1 -- 172.21.15.107:0/3944298707 shutdown_connections 2024-04-25T00:21:55.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.882+0000 7f85cbb5a640 1 --2- 172.21.15.107:0/3944298707 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f85c4106670 0x7f85c41a2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.882+0000 7f85cbb5a640 1 --2- 172.21.15.107:0/3944298707 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f85b00a2d00 0x7f85b00a51c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.882+0000 7f85cbb5a640 1 --2- 172.21.15.107:0/3944298707 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85c4104170 0x7f85c419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.882+0000 7f85cbb5a640 1 --2- 172.21.15.107:0/3944298707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85c406a300 0x7f85c419e5d0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.882+0000 7f85cbb5a640 1 -- 172.21.15.107:0/3944298707 >> 172.21.15.107:0/3944298707 conn(0x7f85c40776a0 msgr2=0x7f85c4103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:55.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.882+0000 7f85cbb5a640 1 -- 172.21.15.107:0/3944298707 shutdown_connections 2024-04-25T00:21:55.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.882+0000 7f85cbb5a640 1 -- 172.21.15.107:0/3944298707 wait complete. 2024-04-25T00:21:55.893 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:21:55.893 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:21:55.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:55 smithi170 ceph-mon[32807]: pgmap v167: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:21:55.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.991+0000 7f9c551d5640 1 -- 172.21.15.107:0/2885571191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c50101050 msgr2=0x7f9c5010d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:55.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.991+0000 7f9c551d5640 1 --2- 172.21.15.107:0/2885571191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c50101050 0x7f9c5010d510 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f9c48009cd0 tx=0x7f9c4802a8e0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.991+0000 7f9c551d5640 1 -- 172.21.15.107:0/2885571191 shutdown_connections 2024-04-25T00:21:55.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.991+0000 7f9c551d5640 1 --2- 172.21.15.107:0/2885571191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c50101050 0x7f9c5010d510 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.991+0000 7f9c551d5640 1 --2- 172.21.15.107:0/2885571191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c50100690 0x7f9c50100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.991+0000 7f9c551d5640 1 --2- 172.21.15.107:0/2885571191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c50106650 0x7f9c50106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.991+0000 7f9c551d5640 1 -- 172.21.15.107:0/2885571191 >> 172.21.15.107:0/2885571191 conn(0x7f9c500777a0 msgr2=0x7f9c50077bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:55.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.991+0000 7f9c551d5640 1 -- 172.21.15.107:0/2885571191 shutdown_connections 2024-04-25T00:21:55.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.991+0000 7f9c551d5640 1 -- 172.21.15.107:0/2885571191 wait complete. 2024-04-25T00:21:55.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.991+0000 7f9c551d5640 1 Processor -- start 2024-04-25T00:21:55.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.992+0000 7f9c551d5640 1 -- start start 2024-04-25T00:21:55.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.992+0000 7f9c551d5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c50100690 0x7f9c50199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.992+0000 7f9c551d5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c50101050 0x7f9c50199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.992+0000 7f9c551d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c50106650 0x7f9c501a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.992+0000 7f9c551d5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9c50110ae0 con 0x7f9c50106650 2024-04-25T00:21:55.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.992+0000 7f9c551d5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9c50110960 con 0x7f9c50100690 2024-04-25T00:21:55.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.992+0000 7f9c551d5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9c50110c60 con 0x7f9c50101050 2024-04-25T00:21:55.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.992+0000 7f9c4f59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c50106650 0x7f9c501a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.992+0000 7f9c4f59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c50106650 0x7f9c501a40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55568/0 (socket says 172.21.15.107:55568) 2024-04-25T00:21:55.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.992+0000 7f9c4f59e640 1 -- 172.21.15.107:0/4109824692 learned_addr learned my addr 172.21.15.107:0/4109824692 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:55.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.992+0000 7f9c4ed9d640 1 --2- 172.21.15.107:0/4109824692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c50100690 0x7f9c50199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.992+0000 7f9c4e59c640 1 --2- 172.21.15.107:0/4109824692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c50101050 0x7f9c50199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.992+0000 7f9c4f59e640 1 -- 172.21.15.107:0/4109824692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c50101050 msgr2=0x7f9c50199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:55.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.992+0000 7f9c4f59e640 1 --2- 172.21.15.107:0/4109824692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c50101050 0x7f9c50199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.992+0000 7f9c4f59e640 1 -- 172.21.15.107:0/4109824692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c50100690 msgr2=0x7f9c50199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:55.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.992+0000 7f9c4f59e640 1 --2- 172.21.15.107:0/4109824692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c50100690 0x7f9c50199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:55.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.992+0000 7f9c4f59e640 1 -- 172.21.15.107:0/4109824692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c501a45e0 con 0x7f9c50106650 2024-04-25T00:21:55.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.992+0000 7f9c4f59e640 1 --2- 172.21.15.107:0/4109824692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c50106650 0x7f9c501a40a0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f9c48006010 tx=0x7f9c480039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:55.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.993+0000 7f9c37fff640 1 -- 172.21.15.107:0/4109824692 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c48004040 con 0x7f9c50106650 2024-04-25T00:21:55.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.993+0000 7f9c551d5640 1 -- 172.21.15.107:0/4109824692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c501a47b0 con 0x7f9c50106650 2024-04-25T00:21:55.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.993+0000 7f9c37fff640 1 -- 172.21.15.107:0/4109824692 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c480046c0 con 0x7f9c50106650 2024-04-25T00:21:55.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.993+0000 7f9c37fff640 1 -- 172.21.15.107:0/4109824692 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c48046480 con 0x7f9c50106650 2024-04-25T00:21:55.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.993+0000 7f9c551d5640 1 -- 172.21.15.107:0/4109824692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c501a4ca0 con 0x7f9c50106650 2024-04-25T00:21:55.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.994+0000 7f9c551d5640 1 -- 172.21.15.107:0/4109824692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c500689b0 con 0x7f9c50106650 2024-04-25T00:21:55.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.995+0000 7f9c37fff640 1 -- 172.21.15.107:0/4109824692 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f9c480306a0 con 0x7f9c50106650 2024-04-25T00:21:55.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.995+0000 7f9c37fff640 1 --2- 172.21.15.107:0/4109824692 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f9c380a2d00 0x7f9c380a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:55.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.995+0000 7f9c4ed9d640 1 --2- 172.21.15.107:0/4109824692 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f9c380a2d00 0x7f9c380a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:55.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.996+0000 7f9c37fff640 1 -- 172.21.15.107:0/4109824692 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f9c4802e030 con 0x7f9c50106650 2024-04-25T00:21:55.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:55.996+0000 7f9c4ed9d640 1 --2- 172.21.15.107:0/4109824692 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f9c380a2d00 0x7f9c380a51c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9c40009990 tx=0x7f9c40009110 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:56.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.000+0000 7f9c37fff640 1 -- 172.21.15.107:0/4109824692 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9c4804b060 con 0x7f9c50106650 2024-04-25T00:21:56.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.213+0000 7f9c551d5640 1 -- 172.21.15.107:0/4109824692 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c50108bd0 con 0x7f9c50106650 2024-04-25T00:21:56.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.214+0000 7f9c37fff640 1 -- 172.21.15.107:0/4109824692 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11902 (secure 0 0 0) 0x7f9c48039380 con 0x7f9c50106650 2024-04-25T00:21:56.215 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:21:56.215 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:21:37.401503+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24493,24521]},"id":1}]} 2024-04-25T00:21:56.215 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 30 2024-04-25T00:21:56.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.216+0000 7f9c551d5640 1 -- 172.21.15.107:0/4109824692 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f9c380a2d00 msgr2=0x7f9c380a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.216+0000 7f9c551d5640 1 --2- 172.21.15.107:0/4109824692 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f9c380a2d00 0x7f9c380a51c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9c40009990 tx=0x7f9c40009110 comp rx=0 tx=0).stop 2024-04-25T00:21:56.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.216+0000 7f9c551d5640 1 -- 172.21.15.107:0/4109824692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c50106650 msgr2=0x7f9c501a40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.216+0000 7f9c551d5640 1 --2- 172.21.15.107:0/4109824692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c50106650 0x7f9c501a40a0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f9c48006010 tx=0x7f9c480039f0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.216+0000 7f9c551d5640 1 -- 172.21.15.107:0/4109824692 shutdown_connections 2024-04-25T00:21:56.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.216+0000 7f9c551d5640 1 --2- 172.21.15.107:0/4109824692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c50106650 0x7f9c501a40a0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.216+0000 7f9c551d5640 1 --2- 172.21.15.107:0/4109824692 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c50101050 0x7f9c50199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.216+0000 7f9c551d5640 1 --2- 172.21.15.107:0/4109824692 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f9c380a2d00 0x7f9c380a51c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.216+0000 7f9c551d5640 1 --2- 172.21.15.107:0/4109824692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c50100690 0x7f9c50199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.216+0000 7f9c551d5640 1 -- 172.21.15.107:0/4109824692 >> 172.21.15.107:0/4109824692 conn(0x7f9c500777a0 msgr2=0x7f9c50103250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.216+0000 7f9c551d5640 1 -- 172.21.15.107:0/4109824692 shutdown_connections 2024-04-25T00:21:56.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.216+0000 7f9c551d5640 1 -- 172.21.15.107:0/4109824692 wait complete. 2024-04-25T00:21:56.227 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:21:56.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.326+0000 7f8b13de7640 1 -- 172.21.15.107:0/2647636891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b0c074490 msgr2=0x7f8b0c10da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.326+0000 7f8b13de7640 1 --2- 172.21.15.107:0/2647636891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b0c074490 0x7f8b0c10da10 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009900 tx=0x7f8afc02a480 comp rx=0 tx=0).stop 2024-04-25T00:21:56.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.326+0000 7f8b13de7640 1 -- 172.21.15.107:0/2647636891 shutdown_connections 2024-04-25T00:21:56.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.326+0000 7f8b13de7640 1 --2- 172.21.15.107:0/2647636891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b0c10df50 0x7f8b0c1103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.326+0000 7f8b13de7640 1 --2- 172.21.15.107:0/2647636891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b0c074490 0x7f8b0c10da10 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.326+0000 7f8b13de7640 1 --2- 172.21.15.107:0/2647636891 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b0c073b70 0x7f8b0c073f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.326+0000 7f8b13de7640 1 -- 172.21.15.107:0/2647636891 >> 172.21.15.107:0/2647636891 conn(0x7f8b0c0fd180 msgr2=0x7f8b0c0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.326+0000 7f8b13de7640 1 -- 172.21.15.107:0/2647636891 shutdown_connections 2024-04-25T00:21:56.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.326+0000 7f8b13de7640 1 -- 172.21.15.107:0/2647636891 wait complete. 2024-04-25T00:21:56.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.327+0000 7f8b13de7640 1 Processor -- start 2024-04-25T00:21:56.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.327+0000 7f8b13de7640 1 -- start start 2024-04-25T00:21:56.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.327+0000 7f8b13de7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b0c073b70 0x7f8b0c19a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.327+0000 7f8b13de7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b0c074490 0x7f8b0c19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.327+0000 7f8b13de7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b0c10df50 0x7f8b0c19eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.327+0000 7f8b13de7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c113cf0 con 0x7f8b0c10df50 2024-04-25T00:21:56.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.327+0000 7f8b13de7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c113b70 con 0x7f8b0c074490 2024-04-25T00:21:56.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.327+0000 7f8b13de7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c113e70 con 0x7f8b0c073b70 2024-04-25T00:21:56.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.327+0000 7f8b12384640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b0c10df50 0x7f8b0c19eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.327+0000 7f8b12384640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b0c10df50 0x7f8b0c19eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55592/0 (socket says 172.21.15.107:55592) 2024-04-25T00:21:56.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.327+0000 7f8b12384640 1 -- 172.21.15.107:0/1697100099 learned_addr learned my addr 172.21.15.107:0/1697100099 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:56.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.327+0000 7f8b11b83640 1 --2- 172.21.15.107:0/1697100099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b0c073b70 0x7f8b0c19a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.327+0000 7f8b11382640 1 --2- 172.21.15.107:0/1697100099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b0c074490 0x7f8b0c19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.327+0000 7f8b12384640 1 -- 172.21.15.107:0/1697100099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b0c073b70 msgr2=0x7f8b0c19a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.327+0000 7f8b12384640 1 --2- 172.21.15.107:0/1697100099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b0c073b70 0x7f8b0c19a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.327+0000 7f8b12384640 1 -- 172.21.15.107:0/1697100099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b0c074490 msgr2=0x7f8b0c19a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.327+0000 7f8b12384640 1 --2- 172.21.15.107:0/1697100099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b0c074490 0x7f8b0c19a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.327+0000 7f8b12384640 1 -- 172.21.15.107:0/1697100099 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b0c19f240 con 0x7f8b0c10df50 2024-04-25T00:21:56.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.327+0000 7f8b12384640 1 --2- 172.21.15.107:0/1697100099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b0c10df50 0x7f8b0c19eb10 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009fd0 tx=0x7f8b0400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:56.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.327+0000 7f8b02ffd640 1 -- 172.21.15.107:0/1697100099 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b04017070 con 0x7f8b0c10df50 2024-04-25T00:21:56.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.328+0000 7f8b13de7640 1 -- 172.21.15.107:0/1697100099 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b0c19f530 con 0x7f8b0c10df50 2024-04-25T00:21:56.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.328+0000 7f8b02ffd640 1 -- 172.21.15.107:0/1697100099 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b04003e50 con 0x7f8b0c10df50 2024-04-25T00:21:56.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.328+0000 7f8b02ffd640 1 -- 172.21.15.107:0/1697100099 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b040106c0 con 0x7f8b0c10df50 2024-04-25T00:21:56.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.328+0000 7f8b13de7640 1 -- 172.21.15.107:0/1697100099 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b0c1a6dc0 con 0x7f8b0c10df50 2024-04-25T00:21:56.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.330+0000 7f8b02ffd640 1 -- 172.21.15.107:0/1697100099 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f8b04003970 con 0x7f8b0c10df50 2024-04-25T00:21:56.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.331+0000 7f8b16bd1740 1 Processor -- start 2024-04-25T00:21:56.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.331+0000 7f8b16bd1740 1 -- start start 2024-04-25T00:21:56.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.331+0000 7f8b16bd1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612583b8580 0x5612583b8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.331+0000 7f8b16bd1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612583b8ea0 0x5612583c3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.331+0000 7f8b16bd1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583c3570 0x5612583c5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.331+0000 7f8b16bd1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56125831d670 con 0x5612583c3570 2024-04-25T00:21:56.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.331+0000 7f8b16bd1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5612582c5970 con 0x5612583b8580 2024-04-25T00:21:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.331+0000 7f8b16bd1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561258319630 con 0x5612583b8ea0 2024-04-25T00:21:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.331+0000 7f8b11382640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583c3570 0x5612583c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.331+0000 7f8b11382640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583c3570 0x5612583c5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55604/0 (socket says 172.21.15.107:55604) 2024-04-25T00:21:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.331+0000 7f8b11382640 1 -- 172.21.15.107:0/1920377739 learned_addr learned my addr 172.21.15.107:0/1920377739 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.331+0000 7f8b02ffd640 1 --2- 172.21.15.107:0/1697100099 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8af80a2d00 0x7f8af80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.331+0000 7f8b11382640 1 --2- 172.21.15.107:0/1920377739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612583b8580 0x5612583b8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.331+0000 7f8b11b83640 1 --2- 172.21.15.107:0/1920377739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612583b8ea0 0x5612583c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.331+0000 7f8b12384640 1 --2- 172.21.15.107:0/1697100099 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8af80a2d00 0x7f8af80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.332+0000 7f8b11382640 1 -- 172.21.15.107:0/1920377739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612583b8ea0 msgr2=0x5612583c3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.332+0000 7f8b11382640 1 --2- 172.21.15.107:0/1920377739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612583b8ea0 0x5612583c3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.332+0000 7f8b11382640 1 -- 172.21.15.107:0/1920377739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612583b8580 msgr2=0x5612583b8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.332+0000 7f8b11382640 1 --2- 172.21.15.107:0/1920377739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612583b8580 0x5612583b8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.332+0000 7f8b11382640 1 -- 172.21.15.107:0/1920377739 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612582e9400 con 0x5612583c3570 2024-04-25T00:21:56.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.332+0000 7f8b02ffd640 1 -- 172.21.15.107:0/1697100099 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f8b040cf450 con 0x7f8b0c10df50 2024-04-25T00:21:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.332+0000 7f8b11382640 1 --2- 172.21.15.107:0/1920377739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583c3570 0x5612583c5960 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009900 tx=0x7f8afc005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfdfb6d0f13a9d6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.332+0000 7f8b12384640 1 --2- 172.21.15.107:0/1697100099 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8af80a2d00 0x7f8af80a51c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8b040cc980 tx=0x7f8b0406e710 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.332+0000 7f8af7fff640 1 -- 172.21.15.107:0/1920377739 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc035040 con 0x5612583c3570 2024-04-25T00:21:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.332+0000 7f8af7fff640 1 -- 172.21.15.107:0/1920377739 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8afc0035f0 con 0x5612583c3570 2024-04-25T00:21:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.332+0000 7f8af7fff640 1 -- 172.21.15.107:0/1920377739 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc03d6e0 con 0x5612583c3570 2024-04-25T00:21:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.333+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1920377739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583c3570 msgr2=0x5612583c5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.333+0000 7f8b16bd1740 1 --2- 172.21.15.107:0/1920377739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583c3570 0x5612583c5960 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8afc009900 tx=0x7f8afc005b50 comp rx=0 tx=0).stop 2024-04-25T00:21:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.333+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1920377739 shutdown_connections 2024-04-25T00:21:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.333+0000 7f8b16bd1740 1 --2- 172.21.15.107:0/1920377739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583c3570 0x5612583c5960 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.333+0000 7f8b16bd1740 1 --2- 172.21.15.107:0/1920377739 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612583b8ea0 0x5612583c3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.333+0000 7f8b16bd1740 1 --2- 172.21.15.107:0/1920377739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612583b8580 0x5612583b8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.333+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1920377739 >> 172.21.15.107:0/1920377739 conn(0x5612583b48a0 msgr2=0x5612583b4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.333+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1920377739 shutdown_connections 2024-04-25T00:21:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.333+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1920377739 wait complete. 2024-04-25T00:21:56.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.333+0000 7f8b16bd1740 1 -- start start 2024-04-25T00:21:56.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.333+0000 7f8b16bd1740 1 Processor -- start 2024-04-25T00:21:56.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.333+0000 7f8b16bd1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583b8580 0x5612583c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8b16bd1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612583b8ea0 0x5612583c0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8b16bd1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612583c3570 0x5612583c06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8b16bd1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5612583c6300 con 0x5612583b8580 2024-04-25T00:21:56.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8b16bd1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5612582d9050 con 0x5612583b8ea0 2024-04-25T00:21:56.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8b16bd1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5612583c6480 con 0x5612583c3570 2024-04-25T00:21:56.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8b16bd1740 10 client.0 _ll_register_callbacks cb 0x561258030be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:56.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8b11382640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583b8580 0x5612583c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8b11382640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583b8580 0x5612583c2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55610/0 (socket says 172.21.15.107:55610) 2024-04-25T00:21:56.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8b11382640 1 -- 172.21.15.107:0/1984333248 learned_addr learned my addr 172.21.15.107:0/1984333248 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:56.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8b11382640 1 --2- 172.21.15.107:0/1984333248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612583c3570 0x5612583c06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8b11b83640 1 --2- 172.21.15.107:0/1984333248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612583b8ea0 0x5612583c0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8b11382640 1 -- 172.21.15.107:0/1984333248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612583c3570 msgr2=0x5612583c06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:56.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8b11382640 1 --2- 172.21.15.107:0/1984333248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612583c3570 0x5612583c06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8b11382640 1 -- 172.21.15.107:0/1984333248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612583b8ea0 msgr2=0x5612583c0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:56.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8b11382640 1 --2- 172.21.15.107:0/1984333248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612583b8ea0 0x5612583c0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8b11382640 1 -- 172.21.15.107:0/1984333248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612582aadc0 con 0x5612583b8580 2024-04-25T00:21:56.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8b11b83640 1 --2- 172.21.15.107:0/1984333248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612583b8ea0 0x5612583c0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:56.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8b11382640 1 --2- 172.21.15.107:0/1984333248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583b8580 0x5612583c2e80 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f8afc001eb0 tx=0x7f8afc032e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67796ed50cd2baf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:56.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8af57fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:21:56.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8af57fa640 1 -- 172.21.15.107:0/1984333248 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc035070 con 0x5612583b8580 2024-04-25T00:21:56.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8af57fa640 1 -- 172.21.15.107:0/1984333248 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f8afc004200 con 0x5612583b8580 2024-04-25T00:21:56.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1984333248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8afc0035f0 con 0x5612583b8580 2024-04-25T00:21:56.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8af57fa640 1 -- 172.21.15.107:0/1984333248 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8afc004400 con 0x5612583b8580 2024-04-25T00:21:56.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8b16bd1740 10 client.16782 fetch_fsmap learned FSMap version 30 2024-04-25T00:21:56.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.334+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1984333248 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5612580313f0 con 0x5612583b8580 2024-04-25T00:21:56.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.335+0000 7f8af57fa640 1 -- 172.21.15.107:0/1984333248 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8afc0035f0 con 0x5612583b8580 2024-04-25T00:21:56.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.335+0000 7f8af57fa640 1 -- 172.21.15.107:0/1984333248 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8afc03a040 con 0x5612583b8580 2024-04-25T00:21:56.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.335+0000 7f8b16bd1740 10 client.16782 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:21:56.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.335+0000 7f8b16bd1740 10 client.16782 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:21:56.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.335+0000 7f8b16bd1740 20 client.16782 populate_metadata read hostname 'smithi107' 2024-04-25T00:21:56.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.335+0000 7f8b16bd1740 1 --2- 172.21.15.107:0/1984333248 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5612583d33d0 0x5612583d57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.335+0000 7f8b16bd1740 4 client.16782 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:56.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.335+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1984333248 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56125832abb0 con 0x5612583d33d0 2024-04-25T00:21:56.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.335+0000 7f8b11b83640 1 --2- 172.21.15.107:0/1984333248 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5612583d33d0 0x5612583d57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.336+0000 7f8b11b83640 1 --2- 172.21.15.107:0/1984333248 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5612583d33d0 0x5612583d57c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6bf756e946fa1157 server_cookie=35ba3b6dd43f3235 in_seq=0 out_seq=0 2024-04-25T00:21:56.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.336+0000 7f8af57fa640 10 client.16782 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:21:56.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.336+0000 7f8af57fa640 1 -- 172.21.15.107:0/1984333248 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56125832abb0 con 0x5612583d33d0 2024-04-25T00:21:56.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.336+0000 7f8af57fa640 10 client.16782 handle_command_reply: tid=0 2024-04-25T00:21:56.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.336+0000 7f8b16bd1740 1 client.16782 shutdown 2024-04-25T00:21:56.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.336+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1984333248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583b8580 msgr2=0x5612583c2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.336+0000 7f8b16bd1740 1 --2- 172.21.15.107:0/1984333248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583b8580 0x5612583c2e80 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f8afc001eb0 tx=0x7f8afc032e30 comp rx=0 tx=0).stop 2024-04-25T00:21:56.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.337+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1984333248 shutdown_connections 2024-04-25T00:21:56.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.337+0000 7f8b16bd1740 1 --2- 172.21.15.107:0/1984333248 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5612583d33d0 0x5612583d57c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.337+0000 7f8b16bd1740 1 --2- 172.21.15.107:0/1984333248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612583c3570 0x5612583c06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.337+0000 7f8b16bd1740 1 --2- 172.21.15.107:0/1984333248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612583b8ea0 0x5612583c0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.337+0000 7f8af57fa640 0 client.16782 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:21:56.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.337+0000 7f8b16bd1740 1 --2- 172.21.15.107:0/1984333248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583b8580 0x5612583c2e80 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.337+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1984333248 >> 172.21.15.107:0/1984333248 conn(0x5612583b48a0 msgr2=0x5612583ba6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.337+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1984333248 shutdown_connections 2024-04-25T00:21:56.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.337+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1984333248 wait complete. 2024-04-25T00:21:56.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.337+0000 7f8b16bd1740 20 client.16782 trim_cache size 0 max 16384 2024-04-25T00:21:56.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.350+0000 7f8b16bd1740 1 Processor -- start 2024-04-25T00:21:56.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.350+0000 7f8b16bd1740 1 -- start start 2024-04-25T00:21:56.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.350+0000 7f8b16bd1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583b8580 0x5612583b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.350+0000 7f8b16bd1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612583b8ea0 0x5612583d87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.350+0000 7f8b16bd1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612583c3570 0x5612583d8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.350+0000 7f8b16bd1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1b35f0 con 0x5612583b8580 2024-04-25T00:21:56.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.350+0000 7f8b16bd1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1b0e30 con 0x5612583c3570 2024-04-25T00:21:56.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.350+0000 7f8b16bd1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1aec80 con 0x5612583b8ea0 2024-04-25T00:21:56.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.350+0000 7f8b11382640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583b8580 0x5612583b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.350+0000 7f8b11382640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583b8580 0x5612583b39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55612/0 (socket says 172.21.15.107:55612) 2024-04-25T00:21:56.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.350+0000 7f8b11382640 1 -- 172.21.15.107:0/1377188903 learned_addr learned my addr 172.21.15.107:0/1377188903 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:56.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.351+0000 7f8b11382640 1 --2- 172.21.15.107:0/1377188903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612583c3570 0x5612583d8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.351+0000 7f8b11382640 1 -- 172.21.15.107:0/1377188903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612583b8ea0 msgr2=0x5612583d87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:56.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.351+0000 7f8b11382640 1 --2- 172.21.15.107:0/1377188903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612583b8ea0 0x5612583d87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.351+0000 7f8b11382640 1 -- 172.21.15.107:0/1377188903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612583c3570 msgr2=0x5612583d8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:56.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.351+0000 7f8b11382640 1 --2- 172.21.15.107:0/1377188903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612583c3570 0x5612583d8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.351+0000 7f8b11382640 1 -- 172.21.15.107:0/1377188903 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612580fb7d0 con 0x5612583b8580 2024-04-25T00:21:56.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.351+0000 7f8b11382640 1 --2- 172.21.15.107:0/1377188903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583b8580 0x5612583b39c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8afc000c00 tx=0x7f8afc03afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73628c82ca3d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:56.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.351+0000 7f8af77fe640 1 -- 172.21.15.107:0/1377188903 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc035040 con 0x5612583b8580 2024-04-25T00:21:56.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.351+0000 7f8af77fe640 1 -- 172.21.15.107:0/1377188903 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8afc0035f0 con 0x5612583b8580 2024-04-25T00:21:56.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.351+0000 7f8af77fe640 1 -- 172.21.15.107:0/1377188903 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc005040 con 0x5612583b8580 2024-04-25T00:21:56.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.352+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1377188903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583b8580 msgr2=0x5612583b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.352+0000 7f8b16bd1740 1 --2- 172.21.15.107:0/1377188903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583b8580 0x5612583b39c0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8afc000c00 tx=0x7f8afc03afd0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.352+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1377188903 shutdown_connections 2024-04-25T00:21:56.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.352+0000 7f8b16bd1740 1 --2- 172.21.15.107:0/1377188903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612583c3570 0x5612583d8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.352+0000 7f8b16bd1740 1 --2- 172.21.15.107:0/1377188903 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612583b8ea0 0x5612583d87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.352+0000 7f8b16bd1740 1 --2- 172.21.15.107:0/1377188903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583b8580 0x5612583b39c0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.352+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1377188903 >> 172.21.15.107:0/1377188903 conn(0x5612583b48a0 msgr2=0x5612583b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.352+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1377188903 shutdown_connections 2024-04-25T00:21:56.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.352+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1377188903 wait complete. 2024-04-25T00:21:56.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.352+0000 7f8b16bd1740 1 -- start start 2024-04-25T00:21:56.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.352+0000 7f8b16bd1740 1 Processor -- start 2024-04-25T00:21:56.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.352+0000 7f8b16bd1740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612583b8580 0x561258431d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.352+0000 7f8b16bd1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583b8ea0 0x5612584322a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.352+0000 7f8b16bd1740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612583c3570 0x5612583d7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.352+0000 7f8b16bd1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1af2f0 con 0x5612583b8ea0 2024-04-25T00:21:56.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.352+0000 7f8b16bd1740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1b0760 con 0x5612583b8580 2024-04-25T00:21:56.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.352+0000 7f8b16bd1740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0c1af9c0 con 0x5612583c3570 2024-04-25T00:21:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.352+0000 7f8b16bd1740 10 client.0 _ll_register_callbacks cb 0x7f8afc032c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.352+0000 7f8b11b83640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583b8ea0 0x5612584322a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.352+0000 7f8b11b83640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583b8ea0 0x5612584322a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55628/0 (socket says 172.21.15.107:55628) 2024-04-25T00:21:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.353+0000 7f8b11b83640 1 -- 172.21.15.107:0/1690637780 learned_addr learned my addr 172.21.15.107:0/1690637780 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.353+0000 7f8b11382640 1 --2- 172.21.15.107:0/1690637780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612583b8580 0x561258431d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.353+0000 7f8b11b83640 1 -- 172.21.15.107:0/1690637780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612583c3570 msgr2=0x5612583d7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.353+0000 7f8b11b83640 1 --2- 172.21.15.107:0/1690637780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612583c3570 0x5612583d7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.353+0000 7f8b11b83640 1 -- 172.21.15.107:0/1690637780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612583b8580 msgr2=0x561258431d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.353+0000 7f8b11b83640 1 --2- 172.21.15.107:0/1690637780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612583b8580 0x561258431d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.353+0000 7f8b11b83640 1 -- 172.21.15.107:0/1690637780 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612583d8590 con 0x5612583b8ea0 2024-04-25T00:21:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.353+0000 7f8b11b83640 1 --2- 172.21.15.107:0/1690637780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583b8ea0 0x5612584322a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x5612583bf9d0 tx=0x7f8b0800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c69561e9d86275a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.353+0000 7f8af57fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:21:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.353+0000 7f8af57fa640 1 -- 172.21.15.107:0/1690637780 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b08013cf0 con 0x5612583b8ea0 2024-04-25T00:21:56.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.353+0000 7f8af57fa640 1 -- 172.21.15.107:0/1690637780 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f8b0801d430 con 0x5612583b8ea0 2024-04-25T00:21:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.353+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1690637780 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8afc0035f0 con 0x5612583b8ea0 2024-04-25T00:21:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.353+0000 7f8af57fa640 1 -- 172.21.15.107:0/1690637780 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8b0802c050 con 0x5612583b8ea0 2024-04-25T00:21:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.353+0000 7f8b16bd1740 10 client.16788 fetch_fsmap learned FSMap version 30 2024-04-25T00:21:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.353+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1690637780 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5612584395c0 con 0x5612583b8ea0 2024-04-25T00:21:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.353+0000 7f8af57fa640 1 -- 172.21.15.107:0/1690637780 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8afc0035f0 con 0x5612583b8ea0 2024-04-25T00:21:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.353+0000 7f8af57fa640 1 -- 172.21.15.107:0/1690637780 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8b08011bf0 con 0x5612583b8ea0 2024-04-25T00:21:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.353+0000 7f8b16bd1740 10 client.16788 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:21:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.353+0000 7f8b16bd1740 10 client.16788 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:21:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.354+0000 7f8b16bd1740 20 client.16788 populate_metadata read hostname 'smithi107' 2024-04-25T00:21:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.354+0000 7f8b16bd1740 1 --2- 172.21.15.107:0/1690637780 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x561258439d90 0x56125843c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.354+0000 7f8b16bd1740 4 client.16788 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:21:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.354+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1690637780 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5612583c0a40 con 0x561258439d90 2024-04-25T00:21:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.354+0000 7f8b11382640 1 --2- 172.21.15.107:0/1690637780 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x561258439d90 0x56125843c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.354+0000 7f8b11382640 1 --2- 172.21.15.107:0/1690637780 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x561258439d90 0x56125843c1b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1fa401ca125616b server_cookie=efbcba715ff9c15b in_seq=0 out_seq=0 2024-04-25T00:21:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.354+0000 7f8af57fa640 10 client.16788 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:21:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.354+0000 7f8af57fa640 1 -- 172.21.15.107:0/1690637780 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5612583c0a40 con 0x561258439d90 2024-04-25T00:21:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.354+0000 7f8af57fa640 10 client.16788 handle_command_reply: tid=0 2024-04-25T00:21:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.354+0000 7f8b16bd1740 1 client.16788 shutdown 2024-04-25T00:21:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.354+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1690637780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583b8ea0 msgr2=0x5612584322a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.354+0000 7f8b16bd1740 1 --2- 172.21.15.107:0/1690637780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583b8ea0 0x5612584322a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x5612583bf9d0 tx=0x7f8b0800bf10 comp rx=0 tx=0).stop 2024-04-25T00:21:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.355+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1690637780 shutdown_connections 2024-04-25T00:21:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.355+0000 7f8b16bd1740 1 --2- 172.21.15.107:0/1690637780 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x561258439d90 0x56125843c1b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.355+0000 7f8b16bd1740 1 --2- 172.21.15.107:0/1690637780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5612583c3570 0x5612583d7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.355+0000 7f8b16bd1740 1 --2- 172.21.15.107:0/1690637780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5612583b8ea0 0x5612584322a0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.355+0000 7f8af57fa640 0 client.16788 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:21:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.355+0000 7f8b16bd1740 1 --2- 172.21.15.107:0/1690637780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5612583b8580 0x561258431d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.355+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1690637780 >> 172.21.15.107:0/1690637780 conn(0x5612583b48a0 msgr2=0x5612583cef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.355+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1690637780 shutdown_connections 2024-04-25T00:21:56.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.355+0000 7f8b16bd1740 1 -- 172.21.15.107:0/1690637780 wait complete. 2024-04-25T00:21:56.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.355+0000 7f8b16bd1740 20 client.16788 trim_cache size 0 max 16384 2024-04-25T00:21:56.362 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:21:56.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.355+0000 7f8b13de7640 1 -- 172.21.15.107:0/1697100099 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8af80a2d00 msgr2=0x7f8af80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.355+0000 7f8b13de7640 1 --2- 172.21.15.107:0/1697100099 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8af80a2d00 0x7f8af80a51c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8b040cc980 tx=0x7f8b0406e710 comp rx=0 tx=0).stop 2024-04-25T00:21:56.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.355+0000 7f8b13de7640 1 -- 172.21.15.107:0/1697100099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b0c10df50 msgr2=0x7f8b0c19eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.355+0000 7f8b13de7640 1 --2- 172.21.15.107:0/1697100099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b0c10df50 0x7f8b0c19eb10 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009fd0 tx=0x7f8b0400b260 comp rx=0 tx=0).stop 2024-04-25T00:21:56.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.355+0000 7f8b13de7640 1 -- 172.21.15.107:0/1697100099 shutdown_connections 2024-04-25T00:21:56.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.355+0000 7f8b13de7640 1 --2- 172.21.15.107:0/1697100099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b0c10df50 0x7f8b0c19eb10 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.355+0000 7f8b13de7640 1 --2- 172.21.15.107:0/1697100099 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8af80a2d00 0x7f8af80a51c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.355+0000 7f8b13de7640 1 --2- 172.21.15.107:0/1697100099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b0c074490 0x7f8b0c19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.355+0000 7f8b13de7640 1 --2- 172.21.15.107:0/1697100099 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b0c073b70 0x7f8b0c19a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.355+0000 7f8b13de7640 1 -- 172.21.15.107:0/1697100099 >> 172.21.15.107:0/1697100099 conn(0x7f8b0c0fd180 msgr2=0x7f8b0c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.355+0000 7f8b13de7640 1 -- 172.21.15.107:0/1697100099 shutdown_connections 2024-04-25T00:21:56.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.355+0000 7f8b13de7640 1 -- 172.21.15.107:0/1697100099 wait complete. 2024-04-25T00:21:56.366 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:21:56.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.466+0000 7fcf1406b640 1 -- 172.21.15.107:0/165783980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf0c101710 msgr2=0x7fcf0c101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.466+0000 7fcf1406b640 1 --2- 172.21.15.107:0/165783980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf0c101710 0x7fcf0c101b90 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009930 tx=0x7fcefc02a420 comp rx=0 tx=0).stop 2024-04-25T00:21:56.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.466+0000 7fcf1406b640 1 -- 172.21.15.107:0/165783980 shutdown_connections 2024-04-25T00:21:56.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.466+0000 7fcf1406b640 1 --2- 172.21.15.107:0/165783980 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf0c1020d0 0x7fcf0c10e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.466+0000 7fcf1406b640 1 --2- 172.21.15.107:0/165783980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf0c101710 0x7fcf0c101b90 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.466+0000 7fcf1406b640 1 --2- 172.21.15.107:0/165783980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf0c107720 0x7fcf0c107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.466+0000 7fcf1406b640 1 -- 172.21.15.107:0/165783980 >> 172.21.15.107:0/165783980 conn(0x7fcf0c0fd180 msgr2=0x7fcf0c0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.466+0000 7fcf1406b640 1 -- 172.21.15.107:0/165783980 shutdown_connections 2024-04-25T00:21:56.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.466+0000 7fcf1406b640 1 -- 172.21.15.107:0/165783980 wait complete. 2024-04-25T00:21:56.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.467+0000 7fcf1406b640 1 Processor -- start 2024-04-25T00:21:56.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.467+0000 7fcf1406b640 1 -- start start 2024-04-25T00:21:56.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.467+0000 7fcf1406b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf0c101710 0x7fcf0c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.467+0000 7fcf1406b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf0c1020d0 0x7fcf0c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.467+0000 7fcf1406b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf0c107720 0x7fcf0c19c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.467+0000 7fcf1406b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c111b30 con 0x7fcf0c101710 2024-04-25T00:21:56.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.467+0000 7fcf1406b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c1119b0 con 0x7fcf0c107720 2024-04-25T00:21:56.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.467+0000 7fcf1406b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c111cb0 con 0x7fcf0c1020d0 2024-04-25T00:21:56.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.467+0000 7fcf11e07640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf0c101710 0x7fcf0c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.467+0000 7fcf11e07640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf0c101710 0x7fcf0c1980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55642/0 (socket says 172.21.15.107:55642) 2024-04-25T00:21:56.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.467+0000 7fcf11e07640 1 -- 172.21.15.107:0/2290552721 learned_addr learned my addr 172.21.15.107:0/2290552721 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:56.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.467+0000 7fcf12608640 1 --2- 172.21.15.107:0/2290552721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf0c107720 0x7fcf0c19c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.467+0000 7fcf11606640 1 --2- 172.21.15.107:0/2290552721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf0c1020d0 0x7fcf0c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.467+0000 7fcf11e07640 1 -- 172.21.15.107:0/2290552721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf0c1020d0 msgr2=0x7fcf0c198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.467+0000 7fcf11e07640 1 --2- 172.21.15.107:0/2290552721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf0c1020d0 0x7fcf0c198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.467+0000 7fcf11e07640 1 -- 172.21.15.107:0/2290552721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf0c107720 msgr2=0x7fcf0c19c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.467+0000 7fcf11e07640 1 --2- 172.21.15.107:0/2290552721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf0c107720 0x7fcf0c19c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.467+0000 7fcf11e07640 1 -- 172.21.15.107:0/2290552721 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0c19d0c0 con 0x7fcf0c101710 2024-04-25T00:21:56.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.467+0000 7fcf11606640 1 --2- 172.21.15.107:0/2290552721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf0c1020d0 0x7fcf0c198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:56.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.467+0000 7fcf11e07640 1 --2- 172.21.15.107:0/2290552721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf0c101710 0x7fcf0c1980d0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400eb90 tx=0x7fcf0400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:56.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.467+0000 7fcf02ffd640 1 -- 172.21.15.107:0/2290552721 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf04004d20 con 0x7fcf0c101710 2024-04-25T00:21:56.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.468+0000 7fcf1406b640 1 -- 172.21.15.107:0/2290552721 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf0c19d3b0 con 0x7fcf0c101710 2024-04-25T00:21:56.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.468+0000 7fcf02ffd640 1 -- 172.21.15.107:0/2290552721 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf0400ee90 con 0x7fcf0c101710 2024-04-25T00:21:56.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.468+0000 7fcf02ffd640 1 -- 172.21.15.107:0/2290552721 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf04010830 con 0x7fcf0c101710 2024-04-25T00:21:56.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.468+0000 7fcf1406b640 1 -- 172.21.15.107:0/2290552721 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf0c1a4c40 con 0x7fcf0c101710 2024-04-25T00:21:56.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.470+0000 7fcf02ffd640 1 -- 172.21.15.107:0/2290552721 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fcf04005020 con 0x7fcf0c101710 2024-04-25T00:21:56.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.470+0000 7fcf16d78740 1 Processor -- start 2024-04-25T00:21:56.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.470+0000 7fcf16d78740 1 -- start start 2024-04-25T00:21:56.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.471+0000 7fcf16d78740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b77258a580 0x55b77258a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.471+0000 7fcf16d78740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77258aea0 0x55b772595030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.471+0000 7fcf16d78740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b772595570 0x55b772597960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.471+0000 7fcf16d78740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b7724ef670 con 0x55b77258aea0 2024-04-25T00:21:56.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.471+0000 7fcf16d78740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b772497970 con 0x55b77258a580 2024-04-25T00:21:56.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.471+0000 7fcf16d78740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b7724eb630 con 0x55b772595570 2024-04-25T00:21:56.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.471+0000 7fcf02ffd640 1 --2- 172.21.15.107:0/2290552721 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fcef80a2d00 0x7fcef80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.471+0000 7fcf11e07640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77258aea0 0x55b772595030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.471+0000 7fcf11e07640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77258aea0 0x55b772595030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55654/0 (socket says 172.21.15.107:55654) 2024-04-25T00:21:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.471+0000 7fcf11e07640 1 -- 172.21.15.107:0/1917442899 learned_addr learned my addr 172.21.15.107:0/1917442899 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.471+0000 7fcf12608640 1 --2- 172.21.15.107:0/2290552721 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fcef80a2d00 0x7fcef80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.471+0000 7fcf11606640 1 --2- 172.21.15.107:0/1917442899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b772595570 0x55b772597960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.471+0000 7fcf12608640 1 --2- 172.21.15.107:0/1917442899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b77258a580 0x55b77258a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.471+0000 7fcf02ffd640 1 -- 172.21.15.107:0/2290552721 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fcf040cfe90 con 0x7fcf0c101710 2024-04-25T00:21:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.471+0000 7fcf11e07640 1 -- 172.21.15.107:0/1917442899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b772595570 msgr2=0x55b772597960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.471+0000 7fcf11e07640 1 --2- 172.21.15.107:0/1917442899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b772595570 0x55b772597960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.471+0000 7fcf11e07640 1 -- 172.21.15.107:0/1917442899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b77258a580 msgr2=0x55b77258a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.471+0000 7fcf11e07640 1 --2- 172.21.15.107:0/1917442899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b77258a580 0x55b77258a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.471+0000 7fcf11e07640 1 -- 172.21.15.107:0/1917442899 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7724bb400 con 0x55b77258aea0 2024-04-25T00:21:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.471+0000 7fcf11e07640 1 --2- 172.21.15.107:0/1917442899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77258aea0 0x55b772595030 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fcf040989b0 tx=0x7fcf0409bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d751fcf63228b01c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.471+0000 7fcf12608640 1 --2- 172.21.15.107:0/2290552721 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fcef80a2d00 0x7fcef80a51c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcf08005fd0 tx=0x7fcf0800ba80 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:56.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.471+0000 7fcef7fff640 1 -- 172.21.15.107:0/1917442899 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf040a2b80 con 0x55b77258aea0 2024-04-25T00:21:56.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.471+0000 7fcef7fff640 1 -- 172.21.15.107:0/1917442899 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf0409c6e0 con 0x55b77258aea0 2024-04-25T00:21:56.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.472+0000 7fcef7fff640 1 -- 172.21.15.107:0/1917442899 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0409a2e0 con 0x55b77258aea0 2024-04-25T00:21:56.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.472+0000 7fcf16d78740 1 -- 172.21.15.107:0/1917442899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77258aea0 msgr2=0x55b772595030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.472+0000 7fcf16d78740 1 --2- 172.21.15.107:0/1917442899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77258aea0 0x55b772595030 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fcf040989b0 tx=0x7fcf0409bc50 comp rx=0 tx=0).stop 2024-04-25T00:21:56.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.472+0000 7fcf16d78740 1 -- 172.21.15.107:0/1917442899 shutdown_connections 2024-04-25T00:21:56.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.472+0000 7fcf16d78740 1 --2- 172.21.15.107:0/1917442899 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b772595570 0x55b772597960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.472+0000 7fcf16d78740 1 --2- 172.21.15.107:0/1917442899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77258aea0 0x55b772595030 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.472+0000 7fcf16d78740 1 --2- 172.21.15.107:0/1917442899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b77258a580 0x55b77258a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.472+0000 7fcf16d78740 1 -- 172.21.15.107:0/1917442899 >> 172.21.15.107:0/1917442899 conn(0x55b7725868a0 msgr2=0x55b772586c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.472+0000 7fcf16d78740 1 -- 172.21.15.107:0/1917442899 shutdown_connections 2024-04-25T00:21:56.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.472+0000 7fcf16d78740 1 -- 172.21.15.107:0/1917442899 wait complete. 2024-04-25T00:21:56.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.473+0000 7fcf16d78740 1 -- start start 2024-04-25T00:21:56.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.473+0000 7fcf16d78740 1 Processor -- start 2024-04-25T00:21:56.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.473+0000 7fcf16d78740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b77258a580 0x55b772594f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.473+0000 7fcf16d78740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b77258aea0 0x55b772592270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.473+0000 7fcf16d78740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b772595570 0x55b772592880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.473+0000 7fcf16d78740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b772598360 con 0x55b772595570 2024-04-25T00:21:56.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.473+0000 7fcf16d78740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b7724ab050 con 0x55b77258aea0 2024-04-25T00:21:56.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.473+0000 7fcf16d78740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b7725984e0 con 0x55b77258a580 2024-04-25T00:21:56.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.473+0000 7fcf16d78740 10 client.0 _ll_register_callbacks cb 0x55b772202be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:56.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.473+0000 7fcf12608640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b772595570 0x55b772592880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.473+0000 7fcf12608640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b772595570 0x55b772592880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55670/0 (socket says 172.21.15.107:55670) 2024-04-25T00:21:56.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.473+0000 7fcf12608640 1 -- 172.21.15.107:0/2481189125 learned_addr learned my addr 172.21.15.107:0/2481189125 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:56.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.473+0000 7fcf11606640 1 --2- 172.21.15.107:0/2481189125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b77258aea0 0x55b772592270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.473+0000 7fcf11e07640 1 --2- 172.21.15.107:0/2481189125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b77258a580 0x55b772594f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.474+0000 7fcf12608640 1 -- 172.21.15.107:0/2481189125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b77258a580 msgr2=0x55b772594f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:56.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.474+0000 7fcf12608640 1 --2- 172.21.15.107:0/2481189125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b77258a580 0x55b772594f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.474+0000 7fcf12608640 1 -- 172.21.15.107:0/2481189125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b77258aea0 msgr2=0x55b772592270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:56.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.474+0000 7fcf12608640 1 --2- 172.21.15.107:0/2481189125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b77258aea0 0x55b772592270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.474+0000 7fcf12608640 1 -- 172.21.15.107:0/2481189125 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b77247cdc0 con 0x55b772595570 2024-04-25T00:21:56.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.474+0000 7fcf12608640 1 --2- 172.21.15.107:0/2481189125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b772595570 0x55b772592880 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800b7c0 tx=0x7fcf08005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e35c2affe2c2094 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:56.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.474+0000 7fcef57fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:21:56.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.474+0000 7fcef57fa640 1 -- 172.21.15.107:0/2481189125 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf080038c0 con 0x55b772595570 2024-04-25T00:21:56.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.474+0000 7fcf16d78740 1 -- 172.21.15.107:0/2481189125 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf0409c6e0 con 0x55b772595570 2024-04-25T00:21:56.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.474+0000 7fcef57fa640 1 -- 172.21.15.107:0/2481189125 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fcf08003a60 con 0x55b772595570 2024-04-25T00:21:56.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.474+0000 7fcef57fa640 1 -- 172.21.15.107:0/2481189125 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcf08003c60 con 0x55b772595570 2024-04-25T00:21:56.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.474+0000 7fcf16d78740 10 client.16800 fetch_fsmap learned FSMap version 30 2024-04-25T00:21:56.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.474+0000 7fcf16d78740 1 -- 172.21.15.107:0/2481189125 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b7722033f0 con 0x55b772595570 2024-04-25T00:21:56.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.477+0000 7fcef57fa640 1 -- 172.21.15.107:0/2481189125 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf0409c6e0 con 0x55b772595570 2024-04-25T00:21:56.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.477+0000 7fcef57fa640 1 -- 172.21.15.107:0/2481189125 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcf0801f070 con 0x55b772595570 2024-04-25T00:21:56.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.477+0000 7fcf16d78740 10 client.16800 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:21:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.477+0000 7fcf16d78740 10 client.16800 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:21:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.477+0000 7fcf16d78740 20 client.16800 populate_metadata read hostname 'smithi107' 2024-04-25T00:21:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.477+0000 7fcf16d78740 1 --2- 172.21.15.107:0/2481189125 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55b7725a5680 0x55b7725a7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.477+0000 7fcf16d78740 4 client.16800 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.477+0000 7fcf16d78740 1 -- 172.21.15.107:0/2481189125 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7724fcbb0 con 0x55b7725a5680 2024-04-25T00:21:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.477+0000 7fcf11e07640 1 --2- 172.21.15.107:0/2481189125 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55b7725a5680 0x55b7725a7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.478+0000 7fcf11e07640 1 --2- 172.21.15.107:0/2481189125 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55b7725a5680 0x55b7725a7ae0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a88646adecfe325a server_cookie=acc4eaba350bc62c in_seq=0 out_seq=0 2024-04-25T00:21:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.478+0000 7fcef57fa640 10 client.16800 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:21:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.479+0000 7fcef57fa640 1 -- 172.21.15.107:0/2481189125 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b7724fcbb0 con 0x55b7725a5680 2024-04-25T00:21:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.479+0000 7fcef57fa640 10 client.16800 handle_command_reply: tid=0 2024-04-25T00:21:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.479+0000 7fcf16d78740 1 client.16800 shutdown 2024-04-25T00:21:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.479+0000 7fcf16d78740 1 -- 172.21.15.107:0/2481189125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b772595570 msgr2=0x55b772592880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.479+0000 7fcf16d78740 1 --2- 172.21.15.107:0/2481189125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b772595570 0x55b772592880 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800b7c0 tx=0x7fcf08005cc0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.479+0000 7fcf16d78740 1 -- 172.21.15.107:0/2481189125 shutdown_connections 2024-04-25T00:21:56.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.479+0000 7fcf16d78740 1 --2- 172.21.15.107:0/2481189125 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55b7725a5680 0x55b7725a7ae0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.479+0000 7fcf16d78740 1 --2- 172.21.15.107:0/2481189125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b772595570 0x55b772592880 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.479+0000 7fcf16d78740 1 --2- 172.21.15.107:0/2481189125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b77258aea0 0x55b772592270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.479+0000 7fcf16d78740 1 --2- 172.21.15.107:0/2481189125 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b77258a580 0x55b772594f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.479+0000 7fcef57fa640 0 client.16800 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:21:56.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.479+0000 7fcf16d78740 1 -- 172.21.15.107:0/2481189125 >> 172.21.15.107:0/2481189125 conn(0x55b7725868a0 msgr2=0x55b772585b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.479+0000 7fcf16d78740 1 -- 172.21.15.107:0/2481189125 shutdown_connections 2024-04-25T00:21:56.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.479+0000 7fcf16d78740 1 -- 172.21.15.107:0/2481189125 wait complete. 2024-04-25T00:21:56.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.479+0000 7fcf16d78740 20 client.16800 trim_cache size 0 max 16384 2024-04-25T00:21:56.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.494+0000 7fcf16d78740 1 Processor -- start 2024-04-25T00:21:56.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.494+0000 7fcf16d78740 1 -- start start 2024-04-25T00:21:56.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.494+0000 7fcf16d78740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b77258a580 0x55b7725a1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.494+0000 7fcf16d78740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77258aea0 0x55b772586250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.494+0000 7fcf16d78740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b772595570 0x55b7725ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.494+0000 7fcf16d78740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c1af250 con 0x55b77258aea0 2024-04-25T00:21:56.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.494+0000 7fcf16d78740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c1aca90 con 0x55b77258a580 2024-04-25T00:21:56.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.494+0000 7fcf16d78740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c1aa8e0 con 0x55b772595570 2024-04-25T00:21:56.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.494+0000 7fcf11606640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77258aea0 0x55b772586250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.494+0000 7fcf11606640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77258aea0 0x55b772586250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55686/0 (socket says 172.21.15.107:55686) 2024-04-25T00:21:56.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.494+0000 7fcf11606640 1 -- 172.21.15.107:0/1197937114 learned_addr learned my addr 172.21.15.107:0/1197937114 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:56.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.494+0000 7fcf11e07640 1 --2- 172.21.15.107:0/1197937114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b77258a580 0x55b7725a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.494+0000 7fcf11606640 1 -- 172.21.15.107:0/1197937114 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b772595570 msgr2=0x55b7725ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:56.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.494+0000 7fcf11606640 1 --2- 172.21.15.107:0/1197937114 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b772595570 0x55b7725ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.494+0000 7fcf11606640 1 -- 172.21.15.107:0/1197937114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b77258a580 msgr2=0x55b7725a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:56.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.494+0000 7fcf11606640 1 --2- 172.21.15.107:0/1197937114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b77258a580 0x55b7725a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.494+0000 7fcf11606640 1 -- 172.21.15.107:0/1197937114 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7722cd7d0 con 0x55b77258aea0 2024-04-25T00:21:56.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.495+0000 7fcf11606640 1 --2- 172.21.15.107:0/1197937114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77258aea0 0x55b772586250 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009930 tx=0x7fcefc032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8eaf1498d82522 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:56.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.495+0000 7fcef77fe640 1 -- 172.21.15.107:0/1197937114 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc035040 con 0x55b77258aea0 2024-04-25T00:21:56.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.495+0000 7fcef77fe640 1 -- 172.21.15.107:0/1197937114 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcefc0317b0 con 0x55b77258aea0 2024-04-25T00:21:56.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.495+0000 7fcef77fe640 1 -- 172.21.15.107:0/1197937114 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc0046c0 con 0x55b77258aea0 2024-04-25T00:21:56.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.495+0000 7fcf16d78740 1 -- 172.21.15.107:0/1197937114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77258aea0 msgr2=0x55b772586250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.495+0000 7fcf16d78740 1 --2- 172.21.15.107:0/1197937114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77258aea0 0x55b772586250 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009930 tx=0x7fcefc032ce0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.495+0000 7fcf16d78740 1 -- 172.21.15.107:0/1197937114 shutdown_connections 2024-04-25T00:21:56.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.495+0000 7fcf16d78740 1 --2- 172.21.15.107:0/1197937114 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b772595570 0x55b7725ad3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.496+0000 7fcf16d78740 1 --2- 172.21.15.107:0/1197937114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b77258aea0 0x55b772586250 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.496+0000 7fcf16d78740 1 --2- 172.21.15.107:0/1197937114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b77258a580 0x55b7725a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.496+0000 7fcf16d78740 1 -- 172.21.15.107:0/1197937114 >> 172.21.15.107:0/1197937114 conn(0x55b7725868a0 msgr2=0x55b772585b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.496+0000 7fcf16d78740 1 -- 172.21.15.107:0/1197937114 shutdown_connections 2024-04-25T00:21:56.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.496+0000 7fcf16d78740 1 -- 172.21.15.107:0/1197937114 wait complete. 2024-04-25T00:21:56.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.496+0000 7fcf16d78740 1 -- start start 2024-04-25T00:21:56.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.496+0000 7fcf16d78740 1 Processor -- start 2024-04-25T00:21:56.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.496+0000 7fcf16d78740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b77258a580 0x55b77259ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.496+0000 7fcf16d78740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b77258aea0 0x55b77259bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.496+0000 7fcf16d78740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b772595570 0x55b772604c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.496+0000 7fcf16d78740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c1aaf50 con 0x55b772595570 2024-04-25T00:21:56.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.496+0000 7fcf16d78740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c1ac3c0 con 0x55b77258aea0 2024-04-25T00:21:56.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.496+0000 7fcf16d78740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c1ab620 con 0x55b77258a580 2024-04-25T00:21:56.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.496+0000 7fcf16d78740 10 client.0 _ll_register_callbacks cb 0x7fcf0800bc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:56.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.496+0000 7fcf12608640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b772595570 0x55b772604c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.496+0000 7fcf12608640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b772595570 0x55b772604c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55690/0 (socket says 172.21.15.107:55690) 2024-04-25T00:21:56.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.496+0000 7fcf12608640 1 -- 172.21.15.107:0/1035404103 learned_addr learned my addr 172.21.15.107:0/1035404103 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:56.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.496+0000 7fcf11e07640 1 --2- 172.21.15.107:0/1035404103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b77258a580 0x55b77259ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.497+0000 7fcf11606640 1 --2- 172.21.15.107:0/1035404103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b77258aea0 0x55b77259bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.497+0000 7fcf12608640 1 -- 172.21.15.107:0/1035404103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b77258a580 msgr2=0x55b77259ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:56.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.497+0000 7fcf12608640 1 --2- 172.21.15.107:0/1035404103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b77258a580 0x55b77259ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.497+0000 7fcf12608640 1 -- 172.21.15.107:0/1035404103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b77258aea0 msgr2=0x55b77259bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:56.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.497+0000 7fcf12608640 1 --2- 172.21.15.107:0/1035404103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b77258aea0 0x55b77259bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.497+0000 7fcf12608640 1 -- 172.21.15.107:0/1035404103 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7726052b0 con 0x55b772595570 2024-04-25T00:21:56.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.497+0000 7fcf12608640 1 --2- 172.21.15.107:0/1035404103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b772595570 0x55b772604c70 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fcf080205c0 tx=0x7fcf08003bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5389e87137d0a199 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:56.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.497+0000 7fcef57fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:21:56.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.497+0000 7fcef57fa640 1 -- 172.21.15.107:0/1035404103 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0800be10 con 0x55b772595570 2024-04-25T00:21:56.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.497+0000 7fcef57fa640 1 -- 172.21.15.107:0/1035404103 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fcf08003e00 con 0x55b772595570 2024-04-25T00:21:56.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.497+0000 7fcf16d78740 1 -- 172.21.15.107:0/1035404103 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcefc0317b0 con 0x55b772595570 2024-04-25T00:21:56.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.497+0000 7fcef57fa640 1 -- 172.21.15.107:0/1035404103 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcf08045050 con 0x55b772595570 2024-04-25T00:21:56.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.497+0000 7fcf16d78740 10 client.16806 fetch_fsmap learned FSMap version 30 2024-04-25T00:21:56.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.497+0000 7fcf16d78740 1 -- 172.21.15.107:0/1035404103 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b77260c660 con 0x55b772595570 2024-04-25T00:21:56.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.497+0000 7fcef57fa640 1 -- 172.21.15.107:0/1035404103 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcefc0317b0 con 0x55b772595570 2024-04-25T00:21:56.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.497+0000 7fcef57fa640 1 -- 172.21.15.107:0/1035404103 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fcf08037780 con 0x55b772595570 2024-04-25T00:21:56.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.497+0000 7fcf16d78740 10 client.16806 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:21:56.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.497+0000 7fcf16d78740 10 client.16806 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:21:56.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.497+0000 7fcf16d78740 20 client.16806 populate_metadata read hostname 'smithi107' 2024-04-25T00:21:56.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.497+0000 7fcf16d78740 1 --2- 172.21.15.107:0/1035404103 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55b77260ce00 0x55b77260f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.497+0000 7fcf16d78740 4 client.16806 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:21:56.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.498+0000 7fcf16d78740 1 -- 172.21.15.107:0/1035404103 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b7724f6f00 con 0x55b77260ce00 2024-04-25T00:21:56.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.498+0000 7fcf11e07640 1 --2- 172.21.15.107:0/1035404103 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55b77260ce00 0x55b77260f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.498+0000 7fcf11e07640 1 --2- 172.21.15.107:0/1035404103 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55b77260ce00 0x55b77260f220 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=29582b7d91be787c server_cookie=c236be3e27300fa0 in_seq=0 out_seq=0 2024-04-25T00:21:56.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.498+0000 7fcef57fa640 10 client.16806 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:21:56.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.498+0000 7fcef57fa640 1 -- 172.21.15.107:0/1035404103 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b7724f6f00 con 0x55b77260ce00 2024-04-25T00:21:56.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.498+0000 7fcef57fa640 10 client.16806 handle_command_reply: tid=0 2024-04-25T00:21:56.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.498+0000 7fcf16d78740 1 client.16806 shutdown 2024-04-25T00:21:56.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.498+0000 7fcf16d78740 1 -- 172.21.15.107:0/1035404103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b772595570 msgr2=0x55b772604c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.498+0000 7fcf16d78740 1 --2- 172.21.15.107:0/1035404103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b772595570 0x55b772604c70 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fcf080205c0 tx=0x7fcf08003bf0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.498+0000 7fcf16d78740 1 -- 172.21.15.107:0/1035404103 shutdown_connections 2024-04-25T00:21:56.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.498+0000 7fcf16d78740 1 --2- 172.21.15.107:0/1035404103 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55b77260ce00 0x55b77260f220 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.498+0000 7fcf16d78740 1 --2- 172.21.15.107:0/1035404103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b772595570 0x55b772604c70 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.498+0000 7fcf16d78740 1 --2- 172.21.15.107:0/1035404103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b77258aea0 0x55b77259bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.498+0000 7fcf16d78740 1 --2- 172.21.15.107:0/1035404103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b77258a580 0x55b77259ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.498+0000 7fcef57fa640 0 client.16806 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:21:56.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.498+0000 7fcf16d78740 1 -- 172.21.15.107:0/1035404103 >> 172.21.15.107:0/1035404103 conn(0x55b7725868a0 msgr2=0x55b7725a0210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.499+0000 7fcf16d78740 1 -- 172.21.15.107:0/1035404103 shutdown_connections 2024-04-25T00:21:56.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.499+0000 7fcf16d78740 1 -- 172.21.15.107:0/1035404103 wait complete. 2024-04-25T00:21:56.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.499+0000 7fcf16d78740 20 client.16806 trim_cache size 0 max 16384 2024-04-25T00:21:56.506 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:21:56.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.499+0000 7fcf1406b640 1 -- 172.21.15.107:0/2290552721 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fcef80a2d00 msgr2=0x7fcef80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.499+0000 7fcf1406b640 1 --2- 172.21.15.107:0/2290552721 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fcef80a2d00 0x7fcef80a51c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcf08005fd0 tx=0x7fcf0800ba80 comp rx=0 tx=0).stop 2024-04-25T00:21:56.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.499+0000 7fcf1406b640 1 -- 172.21.15.107:0/2290552721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf0c101710 msgr2=0x7fcf0c1980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.499+0000 7fcf1406b640 1 --2- 172.21.15.107:0/2290552721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf0c101710 0x7fcf0c1980d0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400eb90 tx=0x7fcf0400c6a0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.499+0000 7fcf1406b640 1 -- 172.21.15.107:0/2290552721 shutdown_connections 2024-04-25T00:21:56.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.499+0000 7fcf1406b640 1 --2- 172.21.15.107:0/2290552721 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fcef80a2d00 0x7fcef80a51c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.499+0000 7fcf1406b640 1 --2- 172.21.15.107:0/2290552721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf0c107720 0x7fcf0c19c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.499+0000 7fcf1406b640 1 --2- 172.21.15.107:0/2290552721 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf0c1020d0 0x7fcf0c198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.499+0000 7fcf1406b640 1 --2- 172.21.15.107:0/2290552721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf0c101710 0x7fcf0c1980d0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.499+0000 7fcf1406b640 1 -- 172.21.15.107:0/2290552721 >> 172.21.15.107:0/2290552721 conn(0x7fcf0c0fd180 msgr2=0x7fcf0c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.499+0000 7fcf1406b640 1 -- 172.21.15.107:0/2290552721 shutdown_connections 2024-04-25T00:21:56.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.499+0000 7fcf1406b640 1 -- 172.21.15.107:0/2290552721 wait complete. 2024-04-25T00:21:56.510 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:21:56.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.611+0000 7f4ce93d5640 1 -- 172.21.15.107:0/192154842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ce41029c0 msgr2=0x7f4ce4102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.611+0000 7f4ce93d5640 1 --2- 172.21.15.107:0/192154842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ce41029c0 0x7f4ce4102da0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009900 tx=0x7f4cd402a510 comp rx=0 tx=0).stop 2024-04-25T00:21:56.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.611+0000 7f4ce93d5640 1 -- 172.21.15.107:0/192154842 shutdown_connections 2024-04-25T00:21:56.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.611+0000 7f4ce93d5640 1 --2- 172.21.15.107:0/192154842 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ce410bd00 0x7f4ce410e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.611+0000 7f4ce93d5640 1 --2- 172.21.15.107:0/192154842 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ce41032e0 0x7f4ce410b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.611+0000 7f4ce93d5640 1 --2- 172.21.15.107:0/192154842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ce41029c0 0x7f4ce4102da0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.611+0000 7f4ce93d5640 1 -- 172.21.15.107:0/192154842 >> 172.21.15.107:0/192154842 conn(0x7f4ce4077740 msgr2=0x7f4ce4077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.611+0000 7f4ce93d5640 1 -- 172.21.15.107:0/192154842 shutdown_connections 2024-04-25T00:21:56.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.611+0000 7f4ce93d5640 1 -- 172.21.15.107:0/192154842 wait complete. 2024-04-25T00:21:56.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.612+0000 7f4ce93d5640 1 Processor -- start 2024-04-25T00:21:56.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.612+0000 7f4ce93d5640 1 -- start start 2024-04-25T00:21:56.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.612+0000 7f4ce93d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ce41029c0 0x7f4ce4197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.612+0000 7f4ce93d5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ce41032e0 0x7f4ce4198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.612+0000 7f4ce93d5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ce410bd00 0x7f4ce419c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.612+0000 7f4ce93d5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce4111b00 con 0x7f4ce41029c0 2024-04-25T00:21:56.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.612+0000 7f4ce93d5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce4111980 con 0x7f4ce41032e0 2024-04-25T00:21:56.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.612+0000 7f4ce93d5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce4111c80 con 0x7f4ce410bd00 2024-04-25T00:21:56.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.612+0000 7f4ce2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ce41029c0 0x7f4ce4197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.612+0000 7f4ce2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ce41029c0 0x7f4ce4197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55700/0 (socket says 172.21.15.107:55700) 2024-04-25T00:21:56.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.612+0000 7f4ce2ffd640 1 -- 172.21.15.107:0/2147568461 learned_addr learned my addr 172.21.15.107:0/2147568461 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:56.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.612+0000 7f4cdbfff640 1 --2- 172.21.15.107:0/2147568461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ce41032e0 0x7f4ce4198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.612+0000 7f4ce2ffd640 1 -- 172.21.15.107:0/2147568461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ce410bd00 msgr2=0x7f4ce419c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.612+0000 7f4ce2ffd640 1 --2- 172.21.15.107:0/2147568461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ce410bd00 0x7f4ce419c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.612+0000 7f4ce2ffd640 1 -- 172.21.15.107:0/2147568461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ce41032e0 msgr2=0x7f4ce4198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.612+0000 7f4ce2ffd640 1 --2- 172.21.15.107:0/2147568461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ce41032e0 0x7f4ce4198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.612+0000 7f4ce2ffd640 1 -- 172.21.15.107:0/2147568461 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce419cfd0 con 0x7f4ce41029c0 2024-04-25T00:21:56.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.612+0000 7f4cdbfff640 1 --2- 172.21.15.107:0/2147568461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ce41032e0 0x7f4ce4198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:56.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.612+0000 7f4ce2ffd640 1 --2- 172.21.15.107:0/2147568461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ce41029c0 0x7f4ce4197fe0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4006010 tx=0x7f4cd40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:56.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.613+0000 7f4ce0ff9640 1 -- 172.21.15.107:0/2147568461 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd403f030 con 0x7f4ce41029c0 2024-04-25T00:21:56.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.613+0000 7f4ce0ff9640 1 -- 172.21.15.107:0/2147568461 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cd4003ce0 con 0x7f4ce41029c0 2024-04-25T00:21:56.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.613+0000 7f4ce93d5640 1 -- 172.21.15.107:0/2147568461 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce419d260 con 0x7f4ce41029c0 2024-04-25T00:21:56.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.613+0000 7f4ce0ff9640 1 -- 172.21.15.107:0/2147568461 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4035070 con 0x7f4ce41029c0 2024-04-25T00:21:56.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.613+0000 7f4ce93d5640 1 -- 172.21.15.107:0/2147568461 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce41a4b50 con 0x7f4ce41029c0 2024-04-25T00:21:56.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.615+0000 7f4ce0ff9640 1 -- 172.21.15.107:0/2147568461 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4cd4042030 con 0x7f4ce41029c0 2024-04-25T00:21:56.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.616+0000 7f4cec1bf740 1 Processor -- start 2024-04-25T00:21:56.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.616+0000 7f4cec1bf740 1 -- start start 2024-04-25T00:21:56.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.616+0000 7f4cec1bf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2ba580 0x56122a2ba960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.616+0000 7f4cec1bf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56122a2baea0 0x56122a2c5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.616+0000 7f4cec1bf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56122a2c5570 0x56122a2c7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.616+0000 7f4cec1bf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56122a21f670 con 0x56122a2ba580 2024-04-25T00:21:56.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.616+0000 7f4cec1bf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56122a1c7970 con 0x56122a2baea0 2024-04-25T00:21:56.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.616+0000 7f4cec1bf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56122a21b630 con 0x56122a2c5570 2024-04-25T00:21:56.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.616+0000 7f4ce37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2ba580 0x56122a2ba960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.616+0000 7f4ce37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2ba580 0x56122a2ba960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55716/0 (socket says 172.21.15.107:55716) 2024-04-25T00:21:56.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.616+0000 7f4ce37fe640 1 -- 172.21.15.107:0/2159245959 learned_addr learned my addr 172.21.15.107:0/2159245959 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:56.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.616+0000 7f4ce0ff9640 1 --2- 172.21.15.107:0/2147568461 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4cd00a2d00 0x7f4cd00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.616+0000 7f4cdbfff640 1 --2- 172.21.15.107:0/2159245959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56122a2c5570 0x56122a2c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.616+0000 7f4ce37fe640 1 --2- 172.21.15.107:0/2147568461 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4cd00a2d00 0x7f4cd00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.616+0000 7f4ce0ff9640 1 -- 172.21.15.107:0/2147568461 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f4cd40efeb0 con 0x7f4ce41029c0 2024-04-25T00:21:56.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.616+0000 7f4ce2ffd640 1 --2- 172.21.15.107:0/2159245959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56122a2baea0 0x56122a2c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.616+0000 7f4ce37fe640 1 -- 172.21.15.107:0/2159245959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56122a2c5570 msgr2=0x56122a2c7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:56.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.616+0000 7f4ce37fe640 1 --2- 172.21.15.107:0/2159245959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56122a2c5570 0x56122a2c7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.616+0000 7f4ce37fe640 1 -- 172.21.15.107:0/2159245959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56122a2baea0 msgr2=0x56122a2c5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:56.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.616+0000 7f4ce37fe640 1 --2- 172.21.15.107:0/2159245959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56122a2baea0 0x56122a2c5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.616+0000 7f4ce37fe640 1 -- 172.21.15.107:0/2159245959 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56122a1eb400 con 0x56122a2ba580 2024-04-25T00:21:56.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.617+0000 7f4ce37fe640 1 --2- 172.21.15.107:0/2159245959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2ba580 0x56122a2ba960 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc015890 tx=0x7f4cdc015d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59cde1be503a1a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:56.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.617+0000 7f4cd97fa640 1 -- 172.21.15.107:0/2159245959 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc022070 con 0x56122a2ba580 2024-04-25T00:21:56.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.617+0000 7f4cd97fa640 1 -- 172.21.15.107:0/2159245959 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cdc009e40 con 0x56122a2ba580 2024-04-25T00:21:56.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.617+0000 7f4cd97fa640 1 -- 172.21.15.107:0/2159245959 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc010d90 con 0x56122a2ba580 2024-04-25T00:21:56.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.617+0000 7f4cec1bf740 1 -- 172.21.15.107:0/2159245959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2ba580 msgr2=0x56122a2ba960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.618+0000 7f4cec1bf740 1 --2- 172.21.15.107:0/2159245959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2ba580 0x56122a2ba960 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc015890 tx=0x7f4cdc015d50 comp rx=0 tx=0).stop 2024-04-25T00:21:56.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.617+0000 7f4ce37fe640 1 --2- 172.21.15.107:0/2147568461 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4cd00a2d00 0x7f4cd00a51c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc01f000 tx=0x7f4cdc01b000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:56.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.618+0000 7f4cec1bf740 1 -- 172.21.15.107:0/2159245959 shutdown_connections 2024-04-25T00:21:56.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.618+0000 7f4cec1bf740 1 --2- 172.21.15.107:0/2159245959 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56122a2c5570 0x56122a2c7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.618+0000 7f4cec1bf740 1 --2- 172.21.15.107:0/2159245959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56122a2baea0 0x56122a2c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.618+0000 7f4cec1bf740 1 --2- 172.21.15.107:0/2159245959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2ba580 0x56122a2ba960 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.618+0000 7f4cec1bf740 1 -- 172.21.15.107:0/2159245959 >> 172.21.15.107:0/2159245959 conn(0x56122a2b68a0 msgr2=0x56122a2b6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.618+0000 7f4cec1bf740 1 -- 172.21.15.107:0/2159245959 shutdown_connections 2024-04-25T00:21:56.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.618+0000 7f4cec1bf740 1 -- 172.21.15.107:0/2159245959 wait complete. 2024-04-25T00:21:56.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.618+0000 7f4cec1bf740 1 -- start start 2024-04-25T00:21:56.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.618+0000 7f4cec1bf740 1 Processor -- start 2024-04-25T00:21:56.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.618+0000 7f4cec1bf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56122a2ba580 0x56122a2c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.618+0000 7f4cec1bf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2baea0 0x56122a2c2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.618+0000 7f4cec1bf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56122a2c5570 0x56122a2c26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.618+0000 7f4cec1bf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56122a2c8360 con 0x56122a2baea0 2024-04-25T00:21:56.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.618+0000 7f4cec1bf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56122a1db050 con 0x56122a2ba580 2024-04-25T00:21:56.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.618+0000 7f4cec1bf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56122a2c84e0 con 0x56122a2c5570 2024-04-25T00:21:56.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.618+0000 7f4cec1bf740 10 client.0 _ll_register_callbacks cb 0x561229f32be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:56.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.618+0000 7f4cdbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2baea0 0x56122a2c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.618+0000 7f4cdbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2baea0 0x56122a2c2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55728/0 (socket says 172.21.15.107:55728) 2024-04-25T00:21:56.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.618+0000 7f4cdbfff640 1 -- 172.21.15.107:0/586728144 learned_addr learned my addr 172.21.15.107:0/586728144 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:56.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.618+0000 7f4ce37fe640 1 --2- 172.21.15.107:0/586728144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56122a2c5570 0x56122a2c26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.619+0000 7f4ce2ffd640 1 --2- 172.21.15.107:0/586728144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56122a2ba580 0x56122a2c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.619+0000 7f4cdbfff640 1 -- 172.21.15.107:0/586728144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56122a2c5570 msgr2=0x56122a2c26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:56.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.619+0000 7f4cdbfff640 1 --2- 172.21.15.107:0/586728144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56122a2c5570 0x56122a2c26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.619+0000 7f4cdbfff640 1 -- 172.21.15.107:0/586728144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56122a2ba580 msgr2=0x56122a2c4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:56.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.619+0000 7f4cdbfff640 1 --2- 172.21.15.107:0/586728144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56122a2ba580 0x56122a2c4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.619+0000 7f4cdbfff640 1 -- 172.21.15.107:0/586728144 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56122a1acdc0 con 0x56122a2baea0 2024-04-25T00:21:56.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.619+0000 7f4ce2ffd640 1 --2- 172.21.15.107:0/586728144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56122a2ba580 0x56122a2c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:56.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.619+0000 7f4cdbfff640 1 --2- 172.21.15.107:0/586728144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2baea0 0x56122a2c2180 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc0136e0 tx=0x7f4ccc013ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb767af9b162bc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:56.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.619+0000 7f4cca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:21:56.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.619+0000 7f4cca7fc640 1 -- 172.21.15.107:0/586728144 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc011dd0 con 0x56122a2baea0 2024-04-25T00:21:56.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.619+0000 7f4cca7fc640 1 -- 172.21.15.107:0/586728144 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f4ccc004030 con 0x56122a2baea0 2024-04-25T00:21:56.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.619+0000 7f4cec1bf740 1 -- 172.21.15.107:0/586728144 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56122a22cbb0 con 0x56122a2baea0 2024-04-25T00:21:56.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.619+0000 7f4cca7fc640 1 -- 172.21.15.107:0/586728144 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4ccc025050 con 0x56122a2baea0 2024-04-25T00:21:56.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.619+0000 7f4cec1bf740 10 client.16818 fetch_fsmap learned FSMap version 30 2024-04-25T00:21:56.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.619+0000 7f4cec1bf740 1 -- 172.21.15.107:0/586728144 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561229f333f0 con 0x56122a2baea0 2024-04-25T00:21:56.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.619+0000 7f4cca7fc640 1 -- 172.21.15.107:0/586728144 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56122a22cbb0 con 0x56122a2baea0 2024-04-25T00:21:56.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.619+0000 7f4cca7fc640 1 -- 172.21.15.107:0/586728144 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4ccc00f040 con 0x56122a2baea0 2024-04-25T00:21:56.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.619+0000 7f4cec1bf740 10 client.16818 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:21:56.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.619+0000 7f4cec1bf740 10 client.16818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:56.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.619+0000 7f4cec1bf740 20 client.16818 populate_metadata read hostname 'smithi107' 2024-04-25T00:21:56.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.620+0000 7f4cec1bf740 1 --2- 172.21.15.107:0/586728144 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56122a2d56a0 0x56122a2d7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.620+0000 7f4cec1bf740 4 client.16818 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:56.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.620+0000 7f4cec1bf740 1 -- 172.21.15.107:0/586728144 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56122a226f00 con 0x56122a2d56a0 2024-04-25T00:21:56.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.620+0000 7f4ce2ffd640 1 --2- 172.21.15.107:0/586728144 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56122a2d56a0 0x56122a2d7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.620+0000 7f4cca7fc640 10 client.16818 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:56.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.620+0000 7f4ce2ffd640 1 --2- 172.21.15.107:0/586728144 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56122a2d56a0 0x56122a2d7b40 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa9399f5d6761f45 server_cookie=6b004ea0965491e5 in_seq=0 out_seq=0 2024-04-25T00:21:56.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.621+0000 7f4cca7fc640 1 -- 172.21.15.107:0/586728144 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56122a226f00 con 0x56122a2d56a0 2024-04-25T00:21:56.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.621+0000 7f4cca7fc640 10 client.16818 handle_command_reply: tid=0 2024-04-25T00:21:56.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.621+0000 7f4cec1bf740 1 client.16818 shutdown 2024-04-25T00:21:56.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.621+0000 7f4cec1bf740 1 -- 172.21.15.107:0/586728144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2baea0 msgr2=0x56122a2c2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.621+0000 7f4cec1bf740 1 --2- 172.21.15.107:0/586728144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2baea0 0x56122a2c2180 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc0136e0 tx=0x7f4ccc013ba0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.621+0000 7f4cec1bf740 1 -- 172.21.15.107:0/586728144 shutdown_connections 2024-04-25T00:21:56.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.621+0000 7f4cec1bf740 1 --2- 172.21.15.107:0/586728144 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56122a2d56a0 0x56122a2d7b40 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.621+0000 7f4cec1bf740 1 --2- 172.21.15.107:0/586728144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56122a2c5570 0x56122a2c26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.621+0000 7f4cec1bf740 1 --2- 172.21.15.107:0/586728144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2baea0 0x56122a2c2180 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.621+0000 7f4cca7fc640 0 client.16818 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:56.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.621+0000 7f4cec1bf740 1 --2- 172.21.15.107:0/586728144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56122a2ba580 0x56122a2c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.621+0000 7f4cec1bf740 1 -- 172.21.15.107:0/586728144 >> 172.21.15.107:0/586728144 conn(0x56122a2b68a0 msgr2=0x56122a2b5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.621+0000 7f4cec1bf740 1 -- 172.21.15.107:0/586728144 shutdown_connections 2024-04-25T00:21:56.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.621+0000 7f4cec1bf740 1 -- 172.21.15.107:0/586728144 wait complete. 2024-04-25T00:21:56.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.621+0000 7f4cec1bf740 20 client.16818 trim_cache size 0 max 16384 2024-04-25T00:21:56.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.634+0000 7f4cec1bf740 1 Processor -- start 2024-04-25T00:21:56.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.634+0000 7f4cec1bf740 1 -- start start 2024-04-25T00:21:56.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.634+0000 7f4cec1bf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56122a2ba580 0x56122a2d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.634+0000 7f4cec1bf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2baea0 0x56122a2c1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.634+0000 7f4cec1bf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56122a2c5570 0x56122a2c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.634+0000 7f4cec1bf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce41b1410 con 0x56122a2baea0 2024-04-25T00:21:56.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.634+0000 7f4cec1bf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce41aec50 con 0x56122a2ba580 2024-04-25T00:21:56.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.634+0000 7f4cec1bf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce41acaa0 con 0x56122a2c5570 2024-04-25T00:21:56.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.635+0000 7f4cdbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2baea0 0x56122a2c1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.635+0000 7f4cdbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2baea0 0x56122a2c1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55730/0 (socket says 172.21.15.107:55730) 2024-04-25T00:21:56.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.635+0000 7f4cdbfff640 1 -- 172.21.15.107:0/1143338624 learned_addr learned my addr 172.21.15.107:0/1143338624 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:56.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.635+0000 7f4ce2ffd640 1 --2- 172.21.15.107:0/1143338624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56122a2ba580 0x56122a2d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.635+0000 7f4cdbfff640 1 -- 172.21.15.107:0/1143338624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56122a2c5570 msgr2=0x56122a2c49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:56.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.635+0000 7f4ce37fe640 1 --2- 172.21.15.107:0/1143338624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56122a2c5570 0x56122a2c49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.635+0000 7f4cdbfff640 1 --2- 172.21.15.107:0/1143338624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56122a2c5570 0x56122a2c49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.635+0000 7f4cdbfff640 1 -- 172.21.15.107:0/1143338624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56122a2ba580 msgr2=0x56122a2d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:56.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.635+0000 7f4cdbfff640 1 --2- 172.21.15.107:0/1143338624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56122a2ba580 0x56122a2d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.635+0000 7f4cdbfff640 1 -- 172.21.15.107:0/1143338624 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561229ffd7d0 con 0x56122a2baea0 2024-04-25T00:21:56.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.635+0000 7f4ce37fe640 1 --2- 172.21.15.107:0/1143338624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56122a2c5570 0x56122a2c49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:56.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.635+0000 7f4cdbfff640 1 --2- 172.21.15.107:0/1143338624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2baea0 0x56122a2c1160 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009e10 tx=0x7f4ccc009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c069db81c9f00dca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:56.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.635+0000 7f4cd8ff9640 1 -- 172.21.15.107:0/1143338624 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc011bd0 con 0x56122a2baea0 2024-04-25T00:21:56.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.635+0000 7f4cd8ff9640 1 -- 172.21.15.107:0/1143338624 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ccc0055f0 con 0x56122a2baea0 2024-04-25T00:21:56.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.635+0000 7f4cd8ff9640 1 -- 172.21.15.107:0/1143338624 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc0035f0 con 0x56122a2baea0 2024-04-25T00:21:56.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.636+0000 7f4cec1bf740 1 -- 172.21.15.107:0/1143338624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2baea0 msgr2=0x56122a2c1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.636+0000 7f4cec1bf740 1 --2- 172.21.15.107:0/1143338624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2baea0 0x56122a2c1160 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009e10 tx=0x7f4ccc009fd0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.636+0000 7f4cec1bf740 1 -- 172.21.15.107:0/1143338624 shutdown_connections 2024-04-25T00:21:56.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.636+0000 7f4cec1bf740 1 --2- 172.21.15.107:0/1143338624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56122a2c5570 0x56122a2c49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.636+0000 7f4cec1bf740 1 --2- 172.21.15.107:0/1143338624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2baea0 0x56122a2c1160 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.636+0000 7f4cec1bf740 1 --2- 172.21.15.107:0/1143338624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56122a2ba580 0x56122a2d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.636+0000 7f4cec1bf740 1 -- 172.21.15.107:0/1143338624 >> 172.21.15.107:0/1143338624 conn(0x56122a2b68a0 msgr2=0x56122a2c0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.636+0000 7f4cec1bf740 1 -- 172.21.15.107:0/1143338624 shutdown_connections 2024-04-25T00:21:56.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.636+0000 7f4cec1bf740 1 -- 172.21.15.107:0/1143338624 wait complete. 2024-04-25T00:21:56.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.636+0000 7f4cec1bf740 1 -- start start 2024-04-25T00:21:56.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.636+0000 7f4cec1bf740 1 Processor -- start 2024-04-25T00:21:56.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.636+0000 7f4cec1bf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56122a2ba580 0x56122a2be3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.636+0000 7f4cec1bf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56122a2baea0 0x56122a2be930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.636+0000 7f4cec1bf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2c5570 0x56122a2bee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.636+0000 7f4cec1bf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce41ad110 con 0x56122a2c5570 2024-04-25T00:21:56.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.636+0000 7f4cec1bf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce41ae580 con 0x56122a2ba580 2024-04-25T00:21:56.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.636+0000 7f4cec1bf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce41ad7e0 con 0x56122a2baea0 2024-04-25T00:21:56.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.637+0000 7f4cec1bf740 10 client.0 _ll_register_callbacks cb 0x7f4ccc005870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:56.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.637+0000 7f4ce37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2c5570 0x56122a2bee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.637+0000 7f4ce37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2c5570 0x56122a2bee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55744/0 (socket says 172.21.15.107:55744) 2024-04-25T00:21:56.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.637+0000 7f4ce37fe640 1 -- 172.21.15.107:0/1452492558 learned_addr learned my addr 172.21.15.107:0/1452492558 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:56.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.637+0000 7f4cdbfff640 1 --2- 172.21.15.107:0/1452492558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56122a2baea0 0x56122a2be930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.637+0000 7f4ce2ffd640 1 --2- 172.21.15.107:0/1452492558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56122a2ba580 0x56122a2be3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.637+0000 7f4ce37fe640 1 -- 172.21.15.107:0/1452492558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56122a2baea0 msgr2=0x56122a2be930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:56.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.637+0000 7f4ce37fe640 1 --2- 172.21.15.107:0/1452492558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56122a2baea0 0x56122a2be930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.637+0000 7f4ce37fe640 1 -- 172.21.15.107:0/1452492558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56122a2ba580 msgr2=0x56122a2be3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:56.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.637+0000 7f4ce37fe640 1 --2- 172.21.15.107:0/1452492558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56122a2ba580 0x56122a2be3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.637+0000 7f4ce37fe640 1 -- 172.21.15.107:0/1452492558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56122a2bf680 con 0x56122a2c5570 2024-04-25T00:21:56.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.637+0000 7f4cdbfff640 1 --2- 172.21.15.107:0/1452492558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56122a2baea0 0x56122a2be930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:56.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.637+0000 7f4ce2ffd640 1 --2- 172.21.15.107:0/1452492558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56122a2ba580 0x56122a2be3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:56.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.637+0000 7f4ce37fe640 1 --2- 172.21.15.107:0/1452492558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2c5570 0x56122a2bee70 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc015890 tx=0x7f4cdc010610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141b1edf30a3a162 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:56.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.637+0000 7f4cca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:21:56.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.637+0000 7f4cca7fc640 1 -- 172.21.15.107:0/1452492558 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc022040 con 0x56122a2c5570 2024-04-25T00:21:56.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.637+0000 7f4cca7fc640 1 -- 172.21.15.107:0/1452492558 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f4cdc015250 con 0x56122a2c5570 2024-04-25T00:21:56.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.637+0000 7f4cec1bf740 1 -- 172.21.15.107:0/1452492558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56122a2c2a50 con 0x56122a2c5570 2024-04-25T00:21:56.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.637+0000 7f4cca7fc640 1 -- 172.21.15.107:0/1452492558 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4cdc03f050 con 0x56122a2c5570 2024-04-25T00:21:56.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.637+0000 7f4cec1bf740 10 client.16824 fetch_fsmap learned FSMap version 30 2024-04-25T00:21:56.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.637+0000 7f4cec1bf740 1 -- 172.21.15.107:0/1452492558 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56122a340140 con 0x56122a2c5570 2024-04-25T00:21:56.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.638+0000 7f4cca7fc640 1 -- 172.21.15.107:0/1452492558 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56122a2c2a50 con 0x56122a2c5570 2024-04-25T00:21:56.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.638+0000 7f4cca7fc640 1 -- 172.21.15.107:0/1452492558 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f4cdc030050 con 0x56122a2c5570 2024-04-25T00:21:56.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.638+0000 7f4cec1bf740 10 client.16824 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:21:56.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.638+0000 7f4cec1bf740 10 client.16824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:56.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.638+0000 7f4cec1bf740 20 client.16824 populate_metadata read hostname 'smithi107' 2024-04-25T00:21:56.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.638+0000 7f4cec1bf740 1 --2- 172.21.15.107:0/1452492558 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56122a3407a0 0x56122a342bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:56.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.638+0000 7f4cec1bf740 4 client.16824 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:21:56.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.638+0000 7f4cec1bf740 1 -- 172.21.15.107:0/1452492558 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56122a1a4820 con 0x56122a3407a0 2024-04-25T00:21:56.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.638+0000 7f4ce2ffd640 1 --2- 172.21.15.107:0/1452492558 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56122a3407a0 0x56122a342bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:56.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.638+0000 7f4cca7fc640 10 client.16824 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:56.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.638+0000 7f4ce2ffd640 1 --2- 172.21.15.107:0/1452492558 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56122a3407a0 0x56122a342bc0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71daea43ef2a40b0 server_cookie=f76f213564245a7 in_seq=0 out_seq=0 2024-04-25T00:21:56.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4cca7fc640 1 -- 172.21.15.107:0/1452492558 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56122a1a4820 con 0x56122a3407a0 2024-04-25T00:21:56.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4cca7fc640 10 client.16824 handle_command_reply: tid=0 2024-04-25T00:21:56.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4cec1bf740 1 client.16824 shutdown 2024-04-25T00:21:56.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4cec1bf740 1 -- 172.21.15.107:0/1452492558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2c5570 msgr2=0x56122a2bee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4cec1bf740 1 --2- 172.21.15.107:0/1452492558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2c5570 0x56122a2bee70 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc015890 tx=0x7f4cdc010610 comp rx=0 tx=0).stop 2024-04-25T00:21:56.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4cec1bf740 1 -- 172.21.15.107:0/1452492558 shutdown_connections 2024-04-25T00:21:56.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4cec1bf740 1 --2- 172.21.15.107:0/1452492558 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56122a3407a0 0x56122a342bc0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4cec1bf740 1 --2- 172.21.15.107:0/1452492558 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56122a2c5570 0x56122a2bee70 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4cec1bf740 1 --2- 172.21.15.107:0/1452492558 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56122a2baea0 0x56122a2be930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4cca7fc640 0 client.16824 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:56.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4cec1bf740 1 --2- 172.21.15.107:0/1452492558 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56122a2ba580 0x56122a2be3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4cec1bf740 1 -- 172.21.15.107:0/1452492558 >> 172.21.15.107:0/1452492558 conn(0x56122a2b68a0 msgr2=0x56122a2c12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4cec1bf740 1 -- 172.21.15.107:0/1452492558 shutdown_connections 2024-04-25T00:21:56.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4cec1bf740 1 -- 172.21.15.107:0/1452492558 wait complete. 2024-04-25T00:21:56.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4cec1bf740 20 client.16824 trim_cache size 0 max 16384 2024-04-25T00:21:56.647 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:21:56.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4ce93d5640 1 -- 172.21.15.107:0/2147568461 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4cd00a2d00 msgr2=0x7f4cd00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4ce93d5640 1 --2- 172.21.15.107:0/2147568461 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4cd00a2d00 0x7f4cd00a51c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc01f000 tx=0x7f4cdc01b000 comp rx=0 tx=0).stop 2024-04-25T00:21:56.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4ce93d5640 1 -- 172.21.15.107:0/2147568461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ce41029c0 msgr2=0x7f4ce4197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:56.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4ce93d5640 1 --2- 172.21.15.107:0/2147568461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ce41029c0 0x7f4ce4197fe0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4006010 tx=0x7f4cd40042f0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4ce93d5640 1 -- 172.21.15.107:0/2147568461 shutdown_connections 2024-04-25T00:21:56.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4ce93d5640 1 --2- 172.21.15.107:0/2147568461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4ce410bd00 0x7f4ce419c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4ce93d5640 1 --2- 172.21.15.107:0/2147568461 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4cd00a2d00 0x7f4cd00a51c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4ce93d5640 1 --2- 172.21.15.107:0/2147568461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ce41032e0 0x7f4ce4198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4ce93d5640 1 --2- 172.21.15.107:0/2147568461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ce41029c0 0x7f4ce4197fe0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:56.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.639+0000 7f4ce93d5640 1 -- 172.21.15.107:0/2147568461 >> 172.21.15.107:0/2147568461 conn(0x7f4ce4077740 msgr2=0x7f4ce40fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:56.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.640+0000 7f4ce93d5640 1 -- 172.21.15.107:0/2147568461 shutdown_connections 2024-04-25T00:21:56.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:56.640+0000 7f4ce93d5640 1 -- 172.21.15.107:0/2147568461 wait complete. 2024-04-25T00:21:56.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:56 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/4109824692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:21:56.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:56 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/4109824692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:21:56.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:56 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/4109824692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:21:57.651 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:21:57.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:21:57.652 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:21:57.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.765+0000 7f1783a93640 1 -- 172.21.15.107:0/3908422837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f177c105ae0 msgr2=0x7f177c105f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.765+0000 7f1783a93640 1 --2- 172.21.15.107:0/3908422837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f177c105ae0 0x7f177c105f70 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f1774009ca0 tx=0x7f177402a9e0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.765+0000 7f1783a93640 1 -- 172.21.15.107:0/3908422837 shutdown_connections 2024-04-25T00:21:57.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.765+0000 7f1783a93640 1 --2- 172.21.15.107:0/3908422837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f177c105ae0 0x7f177c105f70 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.765+0000 7f1783a93640 1 --2- 172.21.15.107:0/3908422837 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f177c1010e0 0x7f177c1055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.765+0000 7f1783a93640 1 --2- 172.21.15.107:0/3908422837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f177c1007c0 0x7f177c100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.765+0000 7f1783a93640 1 -- 172.21.15.107:0/3908422837 >> 172.21.15.107:0/3908422837 conn(0x7f177c0769c0 msgr2=0x7f177c076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:57.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.765+0000 7f1783a93640 1 -- 172.21.15.107:0/3908422837 shutdown_connections 2024-04-25T00:21:57.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.766+0000 7f1783a93640 1 -- 172.21.15.107:0/3908422837 wait complete. 2024-04-25T00:21:57.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.766+0000 7f1783a93640 1 Processor -- start 2024-04-25T00:21:57.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.766+0000 7f1783a93640 1 -- start start 2024-04-25T00:21:57.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.766+0000 7f1783a93640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f177c1007c0 0x7f177c19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.766+0000 7f1783a93640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f177c1010e0 0x7f177c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.766+0000 7f1783a93640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f177c105ae0 0x7f177c1a2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.766+0000 7f1783a93640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f177c118180 con 0x7f177c1010e0 2024-04-25T00:21:57.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.766+0000 7f1783a93640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f177c118000 con 0x7f177c105ae0 2024-04-25T00:21:57.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.766+0000 7f1783a93640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f177c118300 con 0x7f177c1007c0 2024-04-25T00:21:57.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.766+0000 7f178102e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f177c1010e0 0x7f177c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.766+0000 7f178102e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f177c1010e0 0x7f177c19eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55764/0 (socket says 172.21.15.107:55764) 2024-04-25T00:21:57.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.766+0000 7f178102e640 1 -- 172.21.15.107:0/2010314591 learned_addr learned my addr 172.21.15.107:0/2010314591 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:57.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.767+0000 7f178182f640 1 --2- 172.21.15.107:0/2010314591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f177c1007c0 0x7f177c19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.767+0000 7f1782030640 1 --2- 172.21.15.107:0/2010314591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f177c105ae0 0x7f177c1a2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.767+0000 7f178102e640 1 -- 172.21.15.107:0/2010314591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f177c1007c0 msgr2=0x7f177c19e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.767+0000 7f178102e640 1 --2- 172.21.15.107:0/2010314591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f177c1007c0 0x7f177c19e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.767+0000 7f178102e640 1 -- 172.21.15.107:0/2010314591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f177c105ae0 msgr2=0x7f177c1a2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.767+0000 7f178102e640 1 --2- 172.21.15.107:0/2010314591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f177c105ae0 0x7f177c1a2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.767+0000 7f178102e640 1 -- 172.21.15.107:0/2010314591 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177c1a3610 con 0x7f177c1010e0 2024-04-25T00:21:57.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.767+0000 7f178182f640 1 --2- 172.21.15.107:0/2010314591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f177c1007c0 0x7f177c19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:57.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.767+0000 7f1782030640 1 --2- 172.21.15.107:0/2010314591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f177c105ae0 0x7f177c1a2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:57.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.767+0000 7f178102e640 1 --2- 172.21.15.107:0/2010314591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f177c1010e0 0x7f177c19eb60 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f176c00d770 tx=0x7f176c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:57.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.767+0000 7f1772ffd640 1 -- 172.21.15.107:0/2010314591 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c017070 con 0x7f177c1010e0 2024-04-25T00:21:57.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.767+0000 7f1772ffd640 1 -- 172.21.15.107:0/2010314591 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f176c005500 con 0x7f177c1010e0 2024-04-25T00:21:57.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.767+0000 7f1783a93640 1 -- 172.21.15.107:0/2010314591 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f177c1a3900 con 0x7f177c1010e0 2024-04-25T00:21:57.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.767+0000 7f1772ffd640 1 -- 172.21.15.107:0/2010314591 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c004e00 con 0x7f177c1010e0 2024-04-25T00:21:57.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.767+0000 7f1783a93640 1 -- 172.21.15.107:0/2010314591 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f177c1ab190 con 0x7f177c1010e0 2024-04-25T00:21:57.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.769+0000 7f1772ffd640 1 -- 172.21.15.107:0/2010314591 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f176c004fa0 con 0x7f177c1010e0 2024-04-25T00:21:57.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.770+0000 7f178679f740 1 Processor -- start 2024-04-25T00:21:57.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.770+0000 7f178679f740 1 -- start start 2024-04-25T00:21:57.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.770+0000 7f178679f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabbc980 0x5600eabbacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.770+0000 7f178679f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600eabbb290 0x5600eabbb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.770+0000 7f178679f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600eabc6010 0x5600eabc8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.770+0000 7f178679f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5600eab1f2c0 con 0x5600eabbc980 2024-04-25T00:21:57.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.770+0000 7f178679f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5600eaac7910 con 0x5600eabc6010 2024-04-25T00:21:57.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.770+0000 7f178679f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5600eab1b280 con 0x5600eabbb290 2024-04-25T00:21:57.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.770+0000 7f1782030640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabbc980 0x5600eabbacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.770+0000 7f1772ffd640 1 --2- 172.21.15.107:0/2010314591 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f17680a2d00 0x7f17680a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.770+0000 7f1782030640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabbc980 0x5600eabbacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55766/0 (socket says 172.21.15.107:55766) 2024-04-25T00:21:57.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.770+0000 7f1782030640 1 -- 172.21.15.107:0/700489380 learned_addr learned my addr 172.21.15.107:0/700489380 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:57.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.770+0000 7f1782030640 1 --2- 172.21.15.107:0/2010314591 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f17680a2d00 0x7f17680a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.770+0000 7f178102e640 1 --2- 172.21.15.107:0/700489380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600eabc6010 0x5600eabc8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.770+0000 7f178182f640 1 --2- 172.21.15.107:0/700489380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600eabbb290 0x5600eabbb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.770+0000 7f1772ffd640 1 -- 172.21.15.107:0/2010314591 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f176c0cf2f0 con 0x7f177c1010e0 2024-04-25T00:21:57.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.771+0000 7f1782030640 1 -- 172.21.15.107:0/700489380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600eabbb290 msgr2=0x5600eabbb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:57.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.771+0000 7f1782030640 1 --2- 172.21.15.107:0/700489380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600eabbb290 0x5600eabbb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.771+0000 7f1782030640 1 -- 172.21.15.107:0/700489380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600eabc6010 msgr2=0x5600eabc8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:57.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.771+0000 7f1782030640 1 --2- 172.21.15.107:0/700489380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600eabc6010 0x5600eabc8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.771+0000 7f1782030640 1 -- 172.21.15.107:0/700489380 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600eaad8fb0 con 0x5600eabbc980 2024-04-25T00:21:57.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.771+0000 7f1782030640 1 --2- 172.21.15.107:0/700489380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabbc980 0x5600eabbacc0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f1774003b20 tx=0x7f1774003b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82ccc8ea9a5eaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:57.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.771+0000 7f1782030640 1 --2- 172.21.15.107:0/2010314591 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f17680a2d00 0x7f17680a51c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1774003bc0 tx=0x7f1774047000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:57.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.771+0000 7f1767fff640 1 -- 172.21.15.107:0/700489380 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774033050 con 0x5600eabbc980 2024-04-25T00:21:57.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.771+0000 7f1767fff640 1 -- 172.21.15.107:0/700489380 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1774003e30 con 0x5600eabbc980 2024-04-25T00:21:57.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.771+0000 7f1767fff640 1 -- 172.21.15.107:0/700489380 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774004040 con 0x5600eabbc980 2024-04-25T00:21:57.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.772+0000 7f178679f740 1 -- 172.21.15.107:0/700489380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabbc980 msgr2=0x5600eabbacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.772+0000 7f178679f740 1 --2- 172.21.15.107:0/700489380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabbc980 0x5600eabbacc0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f1774003b20 tx=0x7f1774003b50 comp rx=0 tx=0).stop 2024-04-25T00:21:57.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.772+0000 7f178679f740 1 -- 172.21.15.107:0/700489380 shutdown_connections 2024-04-25T00:21:57.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.772+0000 7f178679f740 1 --2- 172.21.15.107:0/700489380 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600eabc6010 0x5600eabc8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.772+0000 7f178679f740 1 --2- 172.21.15.107:0/700489380 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600eabbb290 0x5600eabbb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.772+0000 7f178679f740 1 --2- 172.21.15.107:0/700489380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabbc980 0x5600eabbacc0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.772+0000 7f178679f740 1 -- 172.21.15.107:0/700489380 >> 172.21.15.107:0/700489380 conn(0x5600eab83950 msgr2=0x5600eabb77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:57.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.772+0000 7f178679f740 1 -- 172.21.15.107:0/700489380 shutdown_connections 2024-04-25T00:21:57.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.772+0000 7f178679f740 1 -- 172.21.15.107:0/700489380 wait complete. 2024-04-25T00:21:57.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.772+0000 7f178679f740 1 -- start start 2024-04-25T00:21:57.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.772+0000 7f178679f740 1 Processor -- start 2024-04-25T00:21:57.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.773+0000 7f178679f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600eabbb290 0x5600eabc2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.773+0000 7f178679f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabbc980 0x5600eabc3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.773+0000 7f178679f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600eabc6010 0x5600eabc3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.773+0000 7f178679f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5600eabc8e40 con 0x5600eabbc980 2024-04-25T00:21:57.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.773+0000 7f178679f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5600eaadae90 con 0x5600eabc6010 2024-04-25T00:21:57.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.773+0000 7f178679f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5600eabc8fc0 con 0x5600eabbb290 2024-04-25T00:21:57.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.773+0000 7f178679f740 10 client.0 _ll_register_callbacks cb 0x5600eab16970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:57.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.773+0000 7f178102e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabbc980 0x5600eabc3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.773+0000 7f178102e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabbc980 0x5600eabc3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55772/0 (socket says 172.21.15.107:55772) 2024-04-25T00:21:57.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.773+0000 7f178102e640 1 -- 172.21.15.107:0/2486546453 learned_addr learned my addr 172.21.15.107:0/2486546453 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:57.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.773+0000 7f1782030640 1 --2- 172.21.15.107:0/2486546453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600eabc6010 0x5600eabc3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.773+0000 7f178182f640 1 --2- 172.21.15.107:0/2486546453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600eabbb290 0x5600eabc2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.773+0000 7f178102e640 1 -- 172.21.15.107:0/2486546453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600eabbb290 msgr2=0x5600eabc2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:57.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.773+0000 7f178102e640 1 --2- 172.21.15.107:0/2486546453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600eabbb290 0x5600eabc2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.773+0000 7f178102e640 1 -- 172.21.15.107:0/2486546453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600eabc6010 msgr2=0x5600eabc3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:57.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.773+0000 7f178102e640 1 --2- 172.21.15.107:0/2486546453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600eabc6010 0x5600eabc3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.773+0000 7f178102e640 1 -- 172.21.15.107:0/2486546453 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600ea8fb7e0 con 0x5600eabbc980 2024-04-25T00:21:57.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.773+0000 7f178102e640 1 --2- 172.21.15.107:0/2486546453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabbc980 0x5600eabc3420 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f176c09b1a0 tx=0x7f176c0209d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a8b28f00247be2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:57.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.773+0000 7f17657fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:21:57.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.773+0000 7f17657fa640 1 -- 172.21.15.107:0/2486546453 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c020040 con 0x5600eabbc980 2024-04-25T00:21:57.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.773+0000 7f17657fa640 1 -- 172.21.15.107:0/2486546453 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f176c018d60 con 0x5600eabbc980 2024-04-25T00:21:57.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.773+0000 7f178679f740 1 -- 172.21.15.107:0/2486546453 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1774003e30 con 0x5600eabbc980 2024-04-25T00:21:57.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.774+0000 7f17657fa640 1 -- 172.21.15.107:0/2486546453 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f176c09bba0 con 0x5600eabbc980 2024-04-25T00:21:57.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.774+0000 7f178679f740 10 client.16836 fetch_fsmap learned FSMap version 30 2024-04-25T00:21:57.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.774+0000 7f178679f740 1 -- 172.21.15.107:0/2486546453 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5600ea95bf70 con 0x5600eabbc980 2024-04-25T00:21:57.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.774+0000 7f17657fa640 1 -- 172.21.15.107:0/2486546453 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1774003e30 con 0x5600eabbc980 2024-04-25T00:21:57.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.774+0000 7f17657fa640 1 -- 172.21.15.107:0/2486546453 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f176c0cc950 con 0x5600eabbc980 2024-04-25T00:21:57.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.774+0000 7f178679f740 10 client.16836 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:21:57.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.774+0000 7f178679f740 10 client.16836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:57.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.774+0000 7f178679f740 20 client.16836 populate_metadata read hostname 'smithi107' 2024-04-25T00:21:57.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.774+0000 7f178679f740 1 --2- 172.21.15.107:0/2486546453 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5600eabd5fc0 0x5600eabd83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.774+0000 7f178679f740 4 client.16836 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:57.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.774+0000 7f178679f740 1 -- 172.21.15.107:0/2486546453 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600eaaf35a0 con 0x5600eabd5fc0 2024-04-25T00:21:57.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.774+0000 7f178182f640 1 --2- 172.21.15.107:0/2486546453 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5600eabd5fc0 0x5600eabd83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.775+0000 7f17657fa640 10 client.16836 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:57.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.775+0000 7f178182f640 1 --2- 172.21.15.107:0/2486546453 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5600eabd5fc0 0x5600eabd83b0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83b6d89123af1f90 server_cookie=9e74096e2e60b385 in_seq=0 out_seq=0 2024-04-25T00:21:57.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.775+0000 7f17657fa640 1 -- 172.21.15.107:0/2486546453 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5600eaaf35a0 con 0x5600eabd5fc0 2024-04-25T00:21:57.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.775+0000 7f17657fa640 10 client.16836 handle_command_reply: tid=0 2024-04-25T00:21:57.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.775+0000 7f178679f740 1 client.16836 shutdown 2024-04-25T00:21:57.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.776+0000 7f178679f740 1 -- 172.21.15.107:0/2486546453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabbc980 msgr2=0x5600eabc3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.776+0000 7f178679f740 1 --2- 172.21.15.107:0/2486546453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabbc980 0x5600eabc3420 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f176c09b1a0 tx=0x7f176c0209d0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.776+0000 7f178679f740 1 -- 172.21.15.107:0/2486546453 shutdown_connections 2024-04-25T00:21:57.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.776+0000 7f178679f740 1 --2- 172.21.15.107:0/2486546453 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5600eabd5fc0 0x5600eabd83b0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.776+0000 7f178679f740 1 --2- 172.21.15.107:0/2486546453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600eabc6010 0x5600eabc3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.776+0000 7f178679f740 1 --2- 172.21.15.107:0/2486546453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabbc980 0x5600eabc3420 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.776+0000 7f178679f740 1 --2- 172.21.15.107:0/2486546453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600eabbb290 0x5600eabc2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.776+0000 7f17657fa640 0 client.16836 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:57.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.776+0000 7f178679f740 1 -- 172.21.15.107:0/2486546453 >> 172.21.15.107:0/2486546453 conn(0x5600eab83950 msgr2=0x5600eabbd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:57.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.776+0000 7f178679f740 1 -- 172.21.15.107:0/2486546453 shutdown_connections 2024-04-25T00:21:57.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.776+0000 7f178679f740 1 -- 172.21.15.107:0/2486546453 wait complete. 2024-04-25T00:21:57.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.776+0000 7f178679f740 20 client.16836 trim_cache size 0 max 16384 2024-04-25T00:21:57.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.793+0000 7f178679f740 1 Processor -- start 2024-04-25T00:21:57.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.793+0000 7f178679f740 1 -- start start 2024-04-25T00:21:57.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.793+0000 7f178679f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600eabbb290 0x5600eabd1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.793+0000 7f178679f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600eabbc980 0x5600eabc27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.793+0000 7f178679f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabc6010 0x5600eabbd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.793+0000 7f178679f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f177c1b79e0 con 0x5600eabc6010 2024-04-25T00:21:57.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.793+0000 7f178679f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f177c1b5220 con 0x5600eabbc980 2024-04-25T00:21:57.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.793+0000 7f178679f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f177c1b3070 con 0x5600eabbb290 2024-04-25T00:21:57.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.794+0000 7f1782030640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabc6010 0x5600eabbd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.794+0000 7f1782030640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabc6010 0x5600eabbd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55780/0 (socket says 172.21.15.107:55780) 2024-04-25T00:21:57.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.794+0000 7f1782030640 1 -- 172.21.15.107:0/2659336626 learned_addr learned my addr 172.21.15.107:0/2659336626 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:57.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.794+0000 7f178102e640 1 --2- 172.21.15.107:0/2659336626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600eabbc980 0x5600eabc27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.794+0000 7f1782030640 1 -- 172.21.15.107:0/2659336626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600eabbb290 msgr2=0x5600eabd1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:57.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.794+0000 7f178182f640 1 --2- 172.21.15.107:0/2659336626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600eabbb290 0x5600eabd1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.794+0000 7f1782030640 1 --2- 172.21.15.107:0/2659336626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600eabbb290 0x5600eabd1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.794+0000 7f1782030640 1 -- 172.21.15.107:0/2659336626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600eabbc980 msgr2=0x5600eabc27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:57.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.794+0000 7f1782030640 1 --2- 172.21.15.107:0/2659336626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600eabbc980 0x5600eabc27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.794+0000 7f1782030640 1 -- 172.21.15.107:0/2659336626 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600ea7f3170 con 0x5600eabc6010 2024-04-25T00:21:57.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.794+0000 7f178102e640 1 --2- 172.21.15.107:0/2659336626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600eabbc980 0x5600eabc27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:57.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.794+0000 7f1782030640 1 --2- 172.21.15.107:0/2659336626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabc6010 0x5600eabbd470 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f1774005e30 tx=0x7f1774004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a270d778b6aec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:57.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.794+0000 7f17677fe640 1 -- 172.21.15.107:0/2659336626 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177402b060 con 0x5600eabc6010 2024-04-25T00:21:57.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.794+0000 7f17677fe640 1 -- 172.21.15.107:0/2659336626 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1774054070 con 0x5600eabc6010 2024-04-25T00:21:57.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.794+0000 7f17677fe640 1 -- 172.21.15.107:0/2659336626 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17740046e0 con 0x5600eabc6010 2024-04-25T00:21:57.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.795+0000 7f178679f740 1 -- 172.21.15.107:0/2659336626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabc6010 msgr2=0x5600eabbd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.795+0000 7f178679f740 1 --2- 172.21.15.107:0/2659336626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabc6010 0x5600eabbd470 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f1774005e30 tx=0x7f1774004be0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.795+0000 7f178679f740 1 -- 172.21.15.107:0/2659336626 shutdown_connections 2024-04-25T00:21:57.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.795+0000 7f178679f740 1 --2- 172.21.15.107:0/2659336626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabc6010 0x5600eabbd470 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.795+0000 7f178679f740 1 --2- 172.21.15.107:0/2659336626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600eabbc980 0x5600eabc27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.795+0000 7f178679f740 1 --2- 172.21.15.107:0/2659336626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600eabbb290 0x5600eabd1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.795+0000 7f178679f740 1 -- 172.21.15.107:0/2659336626 >> 172.21.15.107:0/2659336626 conn(0x5600eab83950 msgr2=0x5600eabbcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:57.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.795+0000 7f178679f740 1 -- 172.21.15.107:0/2659336626 shutdown_connections 2024-04-25T00:21:57.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.795+0000 7f178679f740 1 -- 172.21.15.107:0/2659336626 wait complete. 2024-04-25T00:21:57.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.795+0000 7f178679f740 1 -- start start 2024-04-25T00:21:57.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.795+0000 7f178679f740 1 Processor -- start 2024-04-25T00:21:57.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.796+0000 7f178679f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600eabbb290 0x5600eabc0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.796+0000 7f178679f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600eabbc980 0x5600eabc07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.796+0000 7f178679f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabc6010 0x5600eac3b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.796+0000 7f178679f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f177c1b36e0 con 0x5600eabc6010 2024-04-25T00:21:57.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.796+0000 7f178679f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f177c1b4b50 con 0x5600eabbc980 2024-04-25T00:21:57.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.796+0000 7f178679f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f177c1b3db0 con 0x5600eabbb290 2024-04-25T00:21:57.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.796+0000 7f178679f740 10 client.0 _ll_register_callbacks cb 0x7f176c00d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:57.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.796+0000 7f1782030640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabc6010 0x5600eac3b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.796+0000 7f1782030640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabc6010 0x5600eac3b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55788/0 (socket says 172.21.15.107:55788) 2024-04-25T00:21:57.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.796+0000 7f1782030640 1 -- 172.21.15.107:0/1722548249 learned_addr learned my addr 172.21.15.107:0/1722548249 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:57.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.796+0000 7f178102e640 1 --2- 172.21.15.107:0/1722548249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600eabbc980 0x5600eabc07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.796+0000 7f178182f640 1 --2- 172.21.15.107:0/1722548249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600eabbb290 0x5600eabc0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.796+0000 7f1782030640 1 -- 172.21.15.107:0/1722548249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600eabbb290 msgr2=0x5600eabc0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:57.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.796+0000 7f1782030640 1 --2- 172.21.15.107:0/1722548249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600eabbb290 0x5600eabc0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.796+0000 7f1782030640 1 -- 172.21.15.107:0/1722548249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600eabbc980 msgr2=0x5600eabc07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:57.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.796+0000 7f1782030640 1 --2- 172.21.15.107:0/1722548249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600eabbc980 0x5600eabc07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.796+0000 7f1782030640 1 -- 172.21.15.107:0/1722548249 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600eac3ba30 con 0x5600eabc6010 2024-04-25T00:21:57.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.796+0000 7f1782030640 1 --2- 172.21.15.107:0/1722548249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabc6010 0x5600eac3b4f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1774005fd0 tx=0x7f177402e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11ad7f34e4677af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:57.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.796+0000 7f17657fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:21:57.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.796+0000 7f17657fa640 1 -- 172.21.15.107:0/1722548249 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774033040 con 0x5600eabc6010 2024-04-25T00:21:57.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.796+0000 7f17657fa640 1 -- 172.21.15.107:0/1722548249 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f1774004520 con 0x5600eabc6010 2024-04-25T00:21:57.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.796+0000 7f178679f740 1 -- 172.21.15.107:0/1722548249 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1774054070 con 0x5600eabc6010 2024-04-25T00:21:57.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.797+0000 7f17657fa640 1 -- 172.21.15.107:0/1722548249 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f177402b060 con 0x5600eabc6010 2024-04-25T00:21:57.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.797+0000 7f178679f740 10 client.16842 fetch_fsmap learned FSMap version 30 2024-04-25T00:21:57.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.797+0000 7f178679f740 1 -- 172.21.15.107:0/1722548249 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5600eac3cc50 con 0x5600eabc6010 2024-04-25T00:21:57.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.797+0000 7f17657fa640 1 -- 172.21.15.107:0/1722548249 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1774054070 con 0x5600eabc6010 2024-04-25T00:21:57.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.797+0000 7f17657fa640 1 -- 172.21.15.107:0/1722548249 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1774036480 con 0x5600eabc6010 2024-04-25T00:21:57.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.797+0000 7f178679f740 10 client.16842 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:21:57.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.797+0000 7f178679f740 10 client.16842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:57.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.797+0000 7f178679f740 20 client.16842 populate_metadata read hostname 'smithi107' 2024-04-25T00:21:57.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.797+0000 7f178679f740 1 --2- 172.21.15.107:0/1722548249 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5600eac3cf40 0x5600eac3f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.797+0000 7f178679f740 4 client.16842 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:21:57.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.797+0000 7f178679f740 1 -- 172.21.15.107:0/1722548249 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5600eab2c800 con 0x5600eac3cf40 2024-04-25T00:21:57.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.797+0000 7f178182f640 1 --2- 172.21.15.107:0/1722548249 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5600eac3cf40 0x5600eac3f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.798+0000 7f178182f640 1 --2- 172.21.15.107:0/1722548249 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5600eac3cf40 0x5600eac3f330 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=652eff944c604cd0 server_cookie=7eed0092f7cf2dee in_seq=0 out_seq=0 2024-04-25T00:21:57.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.798+0000 7f17657fa640 10 client.16842 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:57.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.799+0000 7f17657fa640 1 -- 172.21.15.107:0/1722548249 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5600eab2c800 con 0x5600eac3cf40 2024-04-25T00:21:57.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.799+0000 7f17657fa640 10 client.16842 handle_command_reply: tid=0 2024-04-25T00:21:57.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.799+0000 7f178679f740 1 client.16842 shutdown 2024-04-25T00:21:57.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.799+0000 7f178679f740 1 -- 172.21.15.107:0/1722548249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabc6010 msgr2=0x5600eac3b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.799+0000 7f178679f740 1 --2- 172.21.15.107:0/1722548249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabc6010 0x5600eac3b4f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f1774005fd0 tx=0x7f177402e820 comp rx=0 tx=0).stop 2024-04-25T00:21:57.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.799+0000 7f178679f740 1 -- 172.21.15.107:0/1722548249 shutdown_connections 2024-04-25T00:21:57.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.799+0000 7f178679f740 1 --2- 172.21.15.107:0/1722548249 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5600eac3cf40 0x5600eac3f330 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.799+0000 7f178679f740 1 --2- 172.21.15.107:0/1722548249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5600eabc6010 0x5600eac3b4f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.799+0000 7f178679f740 1 --2- 172.21.15.107:0/1722548249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5600eabbc980 0x5600eabc07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.799+0000 7f17657fa640 0 client.16842 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:57.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.799+0000 7f178679f740 1 --2- 172.21.15.107:0/1722548249 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5600eabbb290 0x5600eabc0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.799+0000 7f178679f740 1 -- 172.21.15.107:0/1722548249 >> 172.21.15.107:0/1722548249 conn(0x5600eab83950 msgr2=0x5600eabc4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:57.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.799+0000 7f178679f740 1 -- 172.21.15.107:0/1722548249 shutdown_connections 2024-04-25T00:21:57.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.799+0000 7f178679f740 1 -- 172.21.15.107:0/1722548249 wait complete. 2024-04-25T00:21:57.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.799+0000 7f178679f740 20 client.16842 trim_cache size 0 max 16384 2024-04-25T00:21:57.806 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:21:57.806 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-04-25T00:21:57.806 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "d27ef40b-74ef-49b1-aa62-7b5ae77bbdb1", 2024-04-25T00:21:57.806 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-04-25T00:21:57.806 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:21:57.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.800+0000 7f1783a93640 1 -- 172.21.15.107:0/2010314591 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f17680a2d00 msgr2=0x7f17680a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.800+0000 7f1783a93640 1 --2- 172.21.15.107:0/2010314591 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f17680a2d00 0x7f17680a51c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1774003bc0 tx=0x7f1774047000 comp rx=0 tx=0).stop 2024-04-25T00:21:57.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.800+0000 7f1783a93640 1 -- 172.21.15.107:0/2010314591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f177c1010e0 msgr2=0x7f177c19eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.800+0000 7f1783a93640 1 --2- 172.21.15.107:0/2010314591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f177c1010e0 0x7f177c19eb60 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f176c00d770 tx=0x7f176c00dc30 comp rx=0 tx=0).stop 2024-04-25T00:21:57.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.800+0000 7f1783a93640 1 -- 172.21.15.107:0/2010314591 shutdown_connections 2024-04-25T00:21:57.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.800+0000 7f1783a93640 1 --2- 172.21.15.107:0/2010314591 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f17680a2d00 0x7f17680a51c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.800+0000 7f1783a93640 1 --2- 172.21.15.107:0/2010314591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f177c105ae0 0x7f177c1a2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.800+0000 7f1783a93640 1 --2- 172.21.15.107:0/2010314591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f177c1010e0 0x7f177c19eb60 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.800+0000 7f1783a93640 1 --2- 172.21.15.107:0/2010314591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f177c1007c0 0x7f177c19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.800+0000 7f1783a93640 1 -- 172.21.15.107:0/2010314591 >> 172.21.15.107:0/2010314591 conn(0x7f177c0769c0 msgr2=0x7f177c103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:57.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.800+0000 7f1783a93640 1 -- 172.21.15.107:0/2010314591 shutdown_connections 2024-04-25T00:21:57.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.800+0000 7f1783a93640 1 -- 172.21.15.107:0/2010314591 wait complete. 2024-04-25T00:21:57.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:21:57.811 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:21:57.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.910+0000 7f1ef00b3640 1 -- 172.21.15.107:0/3545922291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ee810bd00 msgr2=0x7f1ee810e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.910+0000 7f1ef00b3640 1 --2- 172.21.15.107:0/3545922291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ee810bd00 0x7f1ee810e1a0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0009ca0 tx=0x7f1ee002a9e0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.911+0000 7f1ef00b3640 1 -- 172.21.15.107:0/3545922291 shutdown_connections 2024-04-25T00:21:57.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.911+0000 7f1ef00b3640 1 --2- 172.21.15.107:0/3545922291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ee810bd00 0x7f1ee810e1a0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.911+0000 7f1ef00b3640 1 --2- 172.21.15.107:0/3545922291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ee81032e0 0x7f1ee810b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.911+0000 7f1ef00b3640 1 --2- 172.21.15.107:0/3545922291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ee81029c0 0x7f1ee8102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.911+0000 7f1ef00b3640 1 -- 172.21.15.107:0/3545922291 >> 172.21.15.107:0/3545922291 conn(0x7f1ee8077740 msgr2=0x7f1ee8077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:57.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.911+0000 7f1ef00b3640 1 -- 172.21.15.107:0/3545922291 shutdown_connections 2024-04-25T00:21:57.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.911+0000 7f1ef00b3640 1 -- 172.21.15.107:0/3545922291 wait complete. 2024-04-25T00:21:57.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.911+0000 7f1ef00b3640 1 Processor -- start 2024-04-25T00:21:57.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.911+0000 7f1ef00b3640 1 -- start start 2024-04-25T00:21:57.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.911+0000 7f1ef00b3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ee81029c0 0x7f1ee8198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.911+0000 7f1ef00b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ee81032e0 0x7f1ee81985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.911+0000 7f1ef00b3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ee810bd00 0x7f1ee819c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.911+0000 7f1ef00b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee8111b40 con 0x7f1ee81032e0 2024-04-25T00:21:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.911+0000 7f1ef00b3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee81119c0 con 0x7f1ee810bd00 2024-04-25T00:21:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.911+0000 7f1ef00b3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee8111cc0 con 0x7f1ee81029c0 2024-04-25T00:21:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.911+0000 7f1eed64e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ee81032e0 0x7f1ee81985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.912+0000 7f1eed64e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ee81032e0 0x7f1ee81985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55806/0 (socket says 172.21.15.107:55806) 2024-04-25T00:21:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.912+0000 7f1eed64e640 1 -- 172.21.15.107:0/3989910441 learned_addr learned my addr 172.21.15.107:0/3989910441 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.912+0000 7f1eede4f640 1 --2- 172.21.15.107:0/3989910441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ee81029c0 0x7f1ee8198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.912+0000 7f1eee650640 1 --2- 172.21.15.107:0/3989910441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ee810bd00 0x7f1ee819c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.912+0000 7f1eed64e640 1 -- 172.21.15.107:0/3989910441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ee81029c0 msgr2=0x7f1ee8198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.912+0000 7f1eed64e640 1 --2- 172.21.15.107:0/3989910441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ee81029c0 0x7f1ee8198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.912+0000 7f1eed64e640 1 -- 172.21.15.107:0/3989910441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ee810bd00 msgr2=0x7f1ee819c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.912+0000 7f1eed64e640 1 --2- 172.21.15.107:0/3989910441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ee810bd00 0x7f1ee819c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.912+0000 7f1eed64e640 1 -- 172.21.15.107:0/3989910441 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee819d020 con 0x7f1ee81032e0 2024-04-25T00:21:57.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.912+0000 7f1eed64e640 1 --2- 172.21.15.107:0/3989910441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ee81032e0 0x7f1ee81985c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1ed800b4a0 tx=0x7f1ed800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:57.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.912+0000 7f1edeffd640 1 -- 172.21.15.107:0/3989910441 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8014070 con 0x7f1ee81032e0 2024-04-25T00:21:57.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.912+0000 7f1ef00b3640 1 -- 172.21.15.107:0/3989910441 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee819d310 con 0x7f1ee81032e0 2024-04-25T00:21:57.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.912+0000 7f1edeffd640 1 -- 172.21.15.107:0/3989910441 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ed8010af0 con 0x7f1ee81032e0 2024-04-25T00:21:57.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.912+0000 7f1edeffd640 1 -- 172.21.15.107:0/3989910441 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8017d10 con 0x7f1ee81032e0 2024-04-25T00:21:57.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.912+0000 7f1ef00b3640 1 -- 172.21.15.107:0/3989910441 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee81a4ba0 con 0x7f1ee81032e0 2024-04-25T00:21:57.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.915+0000 7f1edeffd640 1 -- 172.21.15.107:0/3989910441 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f1ed8017070 con 0x7f1ee81032e0 2024-04-25T00:21:57.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.915+0000 7f1ef2dbe740 1 Processor -- start 2024-04-25T00:21:57.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.915+0000 7f1ef2dbe740 1 -- start start 2024-04-25T00:21:57.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.915+0000 7f1ef2dbe740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5639ba70d630 0x5639ba70da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.915+0000 7f1ef2dbe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba70df50 0x5639ba7180e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.915+0000 7f1ef2dbe740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639ba718620 0x5639ba71aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.915+0000 7f1ef2dbe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5639ba6726b0 con 0x5639ba70df50 2024-04-25T00:21:57.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.915+0000 7f1ef2dbe740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5639ba61a880 con 0x5639ba718620 2024-04-25T00:21:57.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.915+0000 7f1ef2dbe740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5639ba66e670 con 0x5639ba70d630 2024-04-25T00:21:57.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.916+0000 7f1eede4f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba70df50 0x5639ba7180e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.916+0000 7f1eede4f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba70df50 0x5639ba7180e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55808/0 (socket says 172.21.15.107:55808) 2024-04-25T00:21:57.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.916+0000 7f1eede4f640 1 -- 172.21.15.107:0/3525595809 learned_addr learned my addr 172.21.15.107:0/3525595809 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:57.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.916+0000 7f1edeffd640 1 --2- 172.21.15.107:0/3989910441 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f1ed40a2d00 0x7f1ed40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.916+0000 7f1eee650640 1 --2- 172.21.15.107:0/3525595809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5639ba70d630 0x5639ba70da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.916+0000 7f1eed64e640 1 --2- 172.21.15.107:0/3525595809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639ba718620 0x5639ba71aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.916+0000 7f1eede4f640 1 -- 172.21.15.107:0/3525595809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5639ba70d630 msgr2=0x5639ba70da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:57.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.916+0000 7f1eede4f640 1 --2- 172.21.15.107:0/3525595809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5639ba70d630 0x5639ba70da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.916+0000 7f1eee650640 1 --2- 172.21.15.107:0/3989910441 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f1ed40a2d00 0x7f1ed40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.916+0000 7f1eede4f640 1 -- 172.21.15.107:0/3525595809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639ba718620 msgr2=0x5639ba71aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:57.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.916+0000 7f1eede4f640 1 --2- 172.21.15.107:0/3525595809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639ba718620 0x5639ba71aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.916+0000 7f1eede4f640 1 -- 172.21.15.107:0/3525595809 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639ba4507d0 con 0x5639ba70df50 2024-04-25T00:21:57.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.916+0000 7f1eed64e640 1 --2- 172.21.15.107:0/3525595809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639ba718620 0x5639ba71aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:57.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.916+0000 7f1edeffd640 1 -- 172.21.15.107:0/3989910441 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f1ed80cea80 con 0x7f1ee81032e0 2024-04-25T00:21:57.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.916+0000 7f1eede4f640 1 --2- 172.21.15.107:0/3525595809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba70df50 0x5639ba7180e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1ee400ac20 tx=0x7f1ee400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5154bd83dfd4f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:57.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.916+0000 7f1ed3fff640 1 -- 172.21.15.107:0/3525595809 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee401d070 con 0x5639ba70df50 2024-04-25T00:21:57.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.916+0000 7f1eee650640 1 --2- 172.21.15.107:0/3989910441 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f1ed40a2d00 0x7f1ed40a51c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0006010 tx=0x7f1ee00095c0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:57.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.916+0000 7f1ed3fff640 1 -- 172.21.15.107:0/3525595809 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ee4018b20 con 0x5639ba70df50 2024-04-25T00:21:57.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.916+0000 7f1ed3fff640 1 -- 172.21.15.107:0/3525595809 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4010b50 con 0x5639ba70df50 2024-04-25T00:21:57.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.917+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/3525595809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba70df50 msgr2=0x5639ba7180e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.917+0000 7f1ef2dbe740 1 --2- 172.21.15.107:0/3525595809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba70df50 0x5639ba7180e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1ee400ac20 tx=0x7f1ee400f790 comp rx=0 tx=0).stop 2024-04-25T00:21:57.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.917+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/3525595809 shutdown_connections 2024-04-25T00:21:57.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.917+0000 7f1ef2dbe740 1 --2- 172.21.15.107:0/3525595809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639ba718620 0x5639ba71aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.917+0000 7f1ef2dbe740 1 --2- 172.21.15.107:0/3525595809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba70df50 0x5639ba7180e0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.917+0000 7f1ef2dbe740 1 --2- 172.21.15.107:0/3525595809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5639ba70d630 0x5639ba70da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.917+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/3525595809 >> 172.21.15.107:0/3525595809 conn(0x5639ba709950 msgr2=0x5639ba709d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:57.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.917+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/3525595809 shutdown_connections 2024-04-25T00:21:57.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.917+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/3525595809 wait complete. 2024-04-25T00:21:57.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.917+0000 7f1ef2dbe740 1 -- start start 2024-04-25T00:21:57.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.917+0000 7f1ef2dbe740 1 Processor -- start 2024-04-25T00:21:57.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.918+0000 7f1ef2dbe740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639ba70d630 0x5639ba7154d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.918+0000 7f1ef2dbe740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5639ba70df50 0x5639ba715a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.918+0000 7f1ef2dbe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba718620 0x5639ba715f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.918+0000 7f1ef2dbe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5639ba71b410 con 0x5639ba718620 2024-04-25T00:21:57.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.918+0000 7f1ef2dbe740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5639ba62df50 con 0x5639ba70d630 2024-04-25T00:21:57.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.918+0000 7f1ef2dbe740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5639ba71b590 con 0x5639ba70df50 2024-04-25T00:21:57.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.918+0000 7f1ef2dbe740 10 client.0 _ll_register_callbacks cb 0x5639ba669000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:57.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.918+0000 7f1eee650640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba718620 0x5639ba715f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.918+0000 7f1eede4f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639ba70d630 0x5639ba7154d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.918+0000 7f1eee650640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba718620 0x5639ba715f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55818/0 (socket says 172.21.15.107:55818) 2024-04-25T00:21:57.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.918+0000 7f1eee650640 1 -- 172.21.15.107:0/3853766994 learned_addr learned my addr 172.21.15.107:0/3853766994 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:57.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.918+0000 7f1eed64e640 1 --2- 172.21.15.107:0/3853766994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5639ba70df50 0x5639ba715a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.918+0000 7f1eee650640 1 -- 172.21.15.107:0/3853766994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5639ba70df50 msgr2=0x5639ba715a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:57.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.918+0000 7f1eee650640 1 --2- 172.21.15.107:0/3853766994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5639ba70df50 0x5639ba715a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.918+0000 7f1eee650640 1 -- 172.21.15.107:0/3853766994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639ba70d630 msgr2=0x5639ba7154d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:57.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.918+0000 7f1eee650640 1 --2- 172.21.15.107:0/3853766994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639ba70d630 0x5639ba7154d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.918+0000 7f1eee650640 1 -- 172.21.15.107:0/3853766994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639ba3863f0 con 0x5639ba718620 2024-04-25T00:21:57.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.918+0000 7f1eee650640 1 --2- 172.21.15.107:0/3853766994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba718620 0x5639ba715f50 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0003ae0 tx=0x7f1ee002a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6dce931e759c1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:57.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.918+0000 7f1ed17fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:21:57.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.919+0000 7f1ed17fa640 1 -- 172.21.15.107:0/3853766994 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0033040 con 0x5639ba718620 2024-04-25T00:21:57.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.919+0000 7f1ed17fa640 1 -- 172.21.15.107:0/3853766994 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f1ee002fa50 con 0x5639ba718620 2024-04-25T00:21:57.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.919+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/3853766994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ee4018b20 con 0x5639ba718620 2024-04-25T00:21:57.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.919+0000 7f1ed17fa640 1 -- 172.21.15.107:0/3853766994 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1ee0041040 con 0x5639ba718620 2024-04-25T00:21:57.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.919+0000 7f1ef2dbe740 10 client.16854 fetch_fsmap learned FSMap version 30 2024-04-25T00:21:57.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.919+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/3853766994 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5639ba3460f0 con 0x5639ba718620 2024-04-25T00:21:57.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.919+0000 7f1ed17fa640 1 -- 172.21.15.107:0/3853766994 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ee4018b20 con 0x5639ba718620 2024-04-25T00:21:57.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.919+0000 7f1ed17fa640 1 -- 172.21.15.107:0/3853766994 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1ee004f070 con 0x5639ba718620 2024-04-25T00:21:57.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.919+0000 7f1ef2dbe740 10 client.16854 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:21:57.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.919+0000 7f1ef2dbe740 10 client.16854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:57.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.919+0000 7f1ef2dbe740 20 client.16854 populate_metadata read hostname 'smithi107' 2024-04-25T00:21:57.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.919+0000 7f1ef2dbe740 1 --2- 172.21.15.107:0/3853766994 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5639ba728820 0x5639ba72ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.919+0000 7f1ef2dbe740 4 client.16854 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:57.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.919+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/3853766994 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639ba67fbf0 con 0x5639ba728820 2024-04-25T00:21:57.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.919+0000 7f1eede4f640 1 --2- 172.21.15.107:0/3853766994 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5639ba728820 0x5639ba72ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.920+0000 7f1ed17fa640 10 client.16854 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:57.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.920+0000 7f1eede4f640 1 --2- 172.21.15.107:0/3853766994 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5639ba728820 0x5639ba72ac70 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=242341797cb2c1fd server_cookie=6e105363ab65be67 in_seq=0 out_seq=0 2024-04-25T00:21:57.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.920+0000 7f1ed17fa640 1 -- 172.21.15.107:0/3853766994 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5639ba67fbf0 con 0x5639ba728820 2024-04-25T00:21:57.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.920+0000 7f1ed17fa640 10 client.16854 handle_command_reply: tid=0 2024-04-25T00:21:57.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.921+0000 7f1ef2dbe740 1 client.16854 shutdown 2024-04-25T00:21:57.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.921+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/3853766994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba718620 msgr2=0x5639ba715f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.921+0000 7f1ef2dbe740 1 --2- 172.21.15.107:0/3853766994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba718620 0x5639ba715f50 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0003ae0 tx=0x7f1ee002a9e0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.921+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/3853766994 shutdown_connections 2024-04-25T00:21:57.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.921+0000 7f1ef2dbe740 1 --2- 172.21.15.107:0/3853766994 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5639ba728820 0x5639ba72ac70 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.921+0000 7f1ef2dbe740 1 --2- 172.21.15.107:0/3853766994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba718620 0x5639ba715f50 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.921+0000 7f1ef2dbe740 1 --2- 172.21.15.107:0/3853766994 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5639ba70df50 0x5639ba715a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.921+0000 7f1ed17fa640 0 client.16854 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:57.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.921+0000 7f1ef2dbe740 1 --2- 172.21.15.107:0/3853766994 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639ba70d630 0x5639ba7154d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.921+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/3853766994 >> 172.21.15.107:0/3853766994 conn(0x5639ba709950 msgr2=0x5639ba708af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:57.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.921+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/3853766994 shutdown_connections 2024-04-25T00:21:57.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.921+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/3853766994 wait complete. 2024-04-25T00:21:57.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.921+0000 7f1ef2dbe740 20 client.16854 trim_cache size 0 max 16384 2024-04-25T00:21:57.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:57 smithi170 ceph-mon[32807]: pgmap v168: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:21:57.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.938+0000 7f1ef2dbe740 1 Processor -- start 2024-04-25T00:21:57.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.938+0000 7f1ef2dbe740 1 -- start start 2024-04-25T00:21:57.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.938+0000 7f1ef2dbe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba70d630 0x5639ba724300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.938+0000 7f1ef2dbe740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5639ba70df50 0x5639ba709270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.938+0000 7f1ef2dbe740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639ba718620 0x5639ba70f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.938+0000 7f1ef2dbe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee81b14d0 con 0x5639ba70d630 2024-04-25T00:21:57.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.938+0000 7f1ef2dbe740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee81aed10 con 0x5639ba718620 2024-04-25T00:21:57.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.938+0000 7f1ef2dbe740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee81acb60 con 0x5639ba70df50 2024-04-25T00:21:57.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.938+0000 7f1eede4f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba70d630 0x5639ba724300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.938+0000 7f1eede4f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba70d630 0x5639ba724300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55826/0 (socket says 172.21.15.107:55826) 2024-04-25T00:21:57.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.938+0000 7f1eede4f640 1 -- 172.21.15.107:0/1775918069 learned_addr learned my addr 172.21.15.107:0/1775918069 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:57.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.939+0000 7f1eede4f640 1 -- 172.21.15.107:0/1775918069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5639ba70df50 msgr2=0x5639ba709270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:57.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.939+0000 7f1eede4f640 1 --2- 172.21.15.107:0/1775918069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5639ba70df50 0x5639ba709270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.939+0000 7f1eede4f640 1 -- 172.21.15.107:0/1775918069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639ba718620 msgr2=0x5639ba70f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:57.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.939+0000 7f1eede4f640 1 --2- 172.21.15.107:0/1775918069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639ba718620 0x5639ba70f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.939+0000 7f1eede4f640 1 -- 172.21.15.107:0/1775918069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639ba70dd80 con 0x5639ba70d630 2024-04-25T00:21:57.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.939+0000 7f1eede4f640 1 --2- 172.21.15.107:0/1775918069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba70d630 0x5639ba724300 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4000c00 tx=0x7f1ee4005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7161b117c3de357c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:57.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.939+0000 7f1ed37fe640 1 -- 172.21.15.107:0/1775918069 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee401d040 con 0x5639ba70d630 2024-04-25T00:21:57.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.939+0000 7f1ed37fe640 1 -- 172.21.15.107:0/1775918069 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ee401e040 con 0x5639ba70d630 2024-04-25T00:21:57.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.939+0000 7f1ed37fe640 1 -- 172.21.15.107:0/1775918069 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee40293b0 con 0x5639ba70d630 2024-04-25T00:21:57.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.939+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/1775918069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba70d630 msgr2=0x5639ba724300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.939+0000 7f1ef2dbe740 1 --2- 172.21.15.107:0/1775918069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba70d630 0x5639ba724300 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4000c00 tx=0x7f1ee4005dc0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.939+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/1775918069 shutdown_connections 2024-04-25T00:21:57.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.939+0000 7f1ef2dbe740 1 --2- 172.21.15.107:0/1775918069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639ba718620 0x5639ba70f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.939+0000 7f1ef2dbe740 1 --2- 172.21.15.107:0/1775918069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5639ba70df50 0x5639ba709270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.939+0000 7f1ef2dbe740 1 --2- 172.21.15.107:0/1775918069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba70d630 0x5639ba724300 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.939+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/1775918069 >> 172.21.15.107:0/1775918069 conn(0x5639ba709950 msgr2=0x5639ba70ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:57.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.939+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/1775918069 shutdown_connections 2024-04-25T00:21:57.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.939+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/1775918069 wait complete. 2024-04-25T00:21:57.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.940+0000 7f1ef2dbe740 1 -- start start 2024-04-25T00:21:57.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.940+0000 7f1ef2dbe740 1 Processor -- start 2024-04-25T00:21:57.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.940+0000 7f1ef2dbe740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5639ba70d630 0x5639ba720850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.940+0000 7f1ef2dbe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba70df50 0x5639ba720d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.940+0000 7f1ef2dbe740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639ba718620 0x5639ba71bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.940+0000 7f1ef2dbe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee81ad1d0 con 0x5639ba70df50 2024-04-25T00:21:57.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.940+0000 7f1ef2dbe740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee81ae640 con 0x5639ba718620 2024-04-25T00:21:57.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.940+0000 7f1ef2dbe740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee81ad8a0 con 0x5639ba70d630 2024-04-25T00:21:57.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.940+0000 7f1ef2dbe740 10 client.0 _ll_register_callbacks cb 0x7f1ee0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:57.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.940+0000 7f1eed64e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba70df50 0x5639ba720d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.940+0000 7f1eed64e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba70df50 0x5639ba720d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55838/0 (socket says 172.21.15.107:55838) 2024-04-25T00:21:57.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.940+0000 7f1eed64e640 1 -- 172.21.15.107:0/2544970673 learned_addr learned my addr 172.21.15.107:0/2544970673 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:21:57.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.940+0000 7f1eee650640 1 --2- 172.21.15.107:0/2544970673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639ba718620 0x5639ba71bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.941+0000 7f1eede4f640 1 --2- 172.21.15.107:0/2544970673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5639ba70d630 0x5639ba720850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.941+0000 7f1eed64e640 1 -- 172.21.15.107:0/2544970673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5639ba70d630 msgr2=0x5639ba720850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:57.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.941+0000 7f1eed64e640 1 --2- 172.21.15.107:0/2544970673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5639ba70d630 0x5639ba720850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.941+0000 7f1eed64e640 1 -- 172.21.15.107:0/2544970673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639ba718620 msgr2=0x5639ba71bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:57.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.941+0000 7f1eed64e640 1 --2- 172.21.15.107:0/2544970673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639ba718620 0x5639ba71bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.941+0000 7f1eed64e640 1 -- 172.21.15.107:0/2544970673 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639ba71c5c0 con 0x5639ba70df50 2024-04-25T00:21:57.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.941+0000 7f1eee650640 1 --2- 172.21.15.107:0/2544970673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639ba718620 0x5639ba71bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:21:57.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.941+0000 7f1eed64e640 1 --2- 172.21.15.107:0/2544970673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba70df50 0x5639ba720d90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f1ed809a9e0 tx=0x7f1ed8065db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b56d14bb471852 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:57.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.941+0000 7f1ed17fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:21:57.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.941+0000 7f1ed17fa640 1 -- 172.21.15.107:0/2544970673 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed801e040 con 0x5639ba70df50 2024-04-25T00:21:57.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.941+0000 7f1ed17fa640 1 -- 172.21.15.107:0/2544970673 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f1ed8097350 con 0x5639ba70df50 2024-04-25T00:21:57.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.941+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/2544970673 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ee401e040 con 0x5639ba70df50 2024-04-25T00:21:57.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.941+0000 7f1ed17fa640 1 -- 172.21.15.107:0/2544970673 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1ed809f090 con 0x5639ba70df50 2024-04-25T00:21:57.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.941+0000 7f1ef2dbe740 10 client.16860 fetch_fsmap learned FSMap version 30 2024-04-25T00:21:57.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.941+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/2544970673 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5639ba78eb00 con 0x5639ba70df50 2024-04-25T00:21:57.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.941+0000 7f1ed17fa640 1 -- 172.21.15.107:0/2544970673 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ee401e040 con 0x5639ba70df50 2024-04-25T00:21:57.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.941+0000 7f1ed17fa640 1 -- 172.21.15.107:0/2544970673 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1ed806abb0 con 0x5639ba70df50 2024-04-25T00:21:57.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.941+0000 7f1ef2dbe740 10 client.16860 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:21:57.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.941+0000 7f1ef2dbe740 10 client.16860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:57.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.941+0000 7f1ef2dbe740 20 client.16860 populate_metadata read hostname 'smithi107' 2024-04-25T00:21:57.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.941+0000 7f1ef2dbe740 1 --2- 172.21.15.107:0/2544970673 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5639ba78f260 0x5639ba791680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:57.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.941+0000 7f1ef2dbe740 4 client.16860 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:21:57.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.941+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/2544970673 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5639ba679f40 con 0x5639ba78f260 2024-04-25T00:21:57.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.942+0000 7f1eede4f640 1 --2- 172.21.15.107:0/2544970673 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5639ba78f260 0x5639ba791680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:57.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.942+0000 7f1ed17fa640 10 client.16860 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:57.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.942+0000 7f1eede4f640 1 --2- 172.21.15.107:0/2544970673 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5639ba78f260 0x5639ba791680 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e6a650cbc794acc server_cookie=5911b022cc403514 in_seq=0 out_seq=0 2024-04-25T00:21:57.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.942+0000 7f1ed17fa640 1 -- 172.21.15.107:0/2544970673 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5639ba679f40 con 0x5639ba78f260 2024-04-25T00:21:57.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.942+0000 7f1ed17fa640 10 client.16860 handle_command_reply: tid=0 2024-04-25T00:21:57.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.942+0000 7f1ef2dbe740 1 client.16860 shutdown 2024-04-25T00:21:57.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.942+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/2544970673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba70df50 msgr2=0x5639ba720d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.942+0000 7f1ef2dbe740 1 --2- 172.21.15.107:0/2544970673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba70df50 0x5639ba720d90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f1ed809a9e0 tx=0x7f1ed8065db0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.942+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/2544970673 shutdown_connections 2024-04-25T00:21:57.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.942+0000 7f1ef2dbe740 1 --2- 172.21.15.107:0/2544970673 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5639ba78f260 0x5639ba791680 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.942+0000 7f1ef2dbe740 1 --2- 172.21.15.107:0/2544970673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5639ba718620 0x5639ba71bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.942+0000 7f1ed17fa640 0 client.16860 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:21:57.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.942+0000 7f1ef2dbe740 1 --2- 172.21.15.107:0/2544970673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5639ba70df50 0x5639ba720d90 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.942+0000 7f1ef2dbe740 1 --2- 172.21.15.107:0/2544970673 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5639ba70d630 0x5639ba720850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.942+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/2544970673 >> 172.21.15.107:0/2544970673 conn(0x5639ba709950 msgr2=0x5639ba70f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:57.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.942+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/2544970673 shutdown_connections 2024-04-25T00:21:57.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.942+0000 7f1ef2dbe740 1 -- 172.21.15.107:0/2544970673 wait complete. 2024-04-25T00:21:57.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.943+0000 7f1ef2dbe740 20 client.16860 trim_cache size 0 max 16384 2024-04-25T00:21:57.950 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:21:57.950 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:21:57.950 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-04-25T00:21:57.950 INFO:teuthology.orchestra.run.smithi107.stdout: "d27ef40b-74ef-49b1-aa62-7b5ae77bbdb1": { 2024-04-25T00:21:57.950 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-04-25T00:21:57.951 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "d27ef40b-74ef-49b1-aa62-7b5ae77bbdb1", 2024-04-25T00:21:57.951 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-04-25T00:21:57.951 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:21:57.951 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:21:57.951 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:21:57.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.943+0000 7f1ef00b3640 1 -- 172.21.15.107:0/3989910441 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f1ed40a2d00 msgr2=0x7f1ed40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.943+0000 7f1ef00b3640 1 --2- 172.21.15.107:0/3989910441 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f1ed40a2d00 0x7f1ed40a51c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1ee0006010 tx=0x7f1ee00095c0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.943+0000 7f1ef00b3640 1 -- 172.21.15.107:0/3989910441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ee81032e0 msgr2=0x7f1ee81985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:57.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.943+0000 7f1ef00b3640 1 --2- 172.21.15.107:0/3989910441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ee81032e0 0x7f1ee81985c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1ed800b4a0 tx=0x7f1ed800b960 comp rx=0 tx=0).stop 2024-04-25T00:21:57.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.943+0000 7f1ef00b3640 1 -- 172.21.15.107:0/3989910441 shutdown_connections 2024-04-25T00:21:57.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.943+0000 7f1ef00b3640 1 --2- 172.21.15.107:0/3989910441 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f1ed40a2d00 0x7f1ed40a51c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.943+0000 7f1ef00b3640 1 --2- 172.21.15.107:0/3989910441 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ee810bd00 0x7f1ee819c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.943+0000 7f1ef00b3640 1 --2- 172.21.15.107:0/3989910441 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ee81032e0 0x7f1ee81985c0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.943+0000 7f1ef00b3640 1 --2- 172.21.15.107:0/3989910441 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ee81029c0 0x7f1ee8198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:57.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.943+0000 7f1ef00b3640 1 -- 172.21.15.107:0/3989910441 >> 172.21.15.107:0/3989910441 conn(0x7f1ee8077740 msgr2=0x7f1ee80fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:57.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.943+0000 7f1ef00b3640 1 -- 172.21.15.107:0/3989910441 shutdown_connections 2024-04-25T00:21:57.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:21:57.943+0000 7f1ef00b3640 1 -- 172.21.15.107:0/3989910441 wait complete. 2024-04-25T00:21:57.953 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:57 smithi107 ceph-mon[29153]: pgmap v168: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:21:57.954 INFO:tasks.cephfs.filesystem:scrub status for tag:d27ef40b-74ef-49b1-aa62-7b5ae77bbdb1 - {'path': '/', 'tag': 'd27ef40b-74ef-49b1-aa62-7b5ae77bbdb1', 'options': 'recursive,force'} 2024-04-25T00:21:58.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:57 smithi195 ceph-mon[32833]: pgmap v168: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:21:58.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:58 smithi170 ceph-mon[32807]: scrub queued for path: / 2024-04-25T00:21:58.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:58 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:21:58.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:58 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:21:58.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:58 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:21:58.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:58 smithi107 ceph-mon[29153]: scrub queued for path: / 2024-04-25T00:21:58.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:58 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:21:58.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:58 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:21:58.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:58 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:21:59.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:58 smithi195 ceph-mon[32833]: scrub queued for path: / 2024-04-25T00:21:59.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:58 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:21:59.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:58 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:21:59.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:58 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:21:59.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:21:59 smithi170 ceph-mon[32807]: pgmap v169: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:21:59.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:21:59 smithi107 ceph-mon[29153]: pgmap v169: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:22:00.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:21:59 smithi195 ceph-mon[32833]: pgmap v169: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:22:00.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:00 smithi170 ceph-mon[32807]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:22:00.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:00 smithi107 ceph-mon[29153]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:22:01.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:00 smithi195 ceph-mon[32833]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:22:01.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:01 smithi170 ceph-mon[32807]: pgmap v170: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:01.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:01 smithi107 ceph-mon[29153]: pgmap v170: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:02.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:01 smithi195 ceph-mon[32833]: pgmap v170: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:02.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:02 smithi170 ceph-mon[32807]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:22:02.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:02 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:22:02.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:02 smithi107 ceph-mon[29153]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:22:02.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:02 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:22:03.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:02 smithi195 ceph-mon[32833]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:22:03.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:02 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:22:03.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:03 smithi170 ceph-mon[32807]: pgmap v171: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T00:22:03.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:03 smithi107 ceph-mon[29153]: pgmap v171: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T00:22:04.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:03 smithi195 ceph-mon[32833]: pgmap v171: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T00:22:05.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:05 smithi170 ceph-mon[32807]: pgmap v172: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T00:22:05.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:05 smithi107 ceph-mon[29153]: pgmap v172: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T00:22:06.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:05 smithi195 ceph-mon[32833]: pgmap v172: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T00:22:07.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:07 smithi170 ceph-mon[32807]: pgmap v173: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T00:22:07.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:07 smithi107 ceph-mon[29153]: pgmap v173: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T00:22:08.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:07 smithi195 ceph-mon[32833]: pgmap v173: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T00:22:09.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:09 smithi170 ceph-mon[32807]: pgmap v174: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T00:22:09.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:09 smithi107 ceph-mon[29153]: pgmap v174: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T00:22:10.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:09 smithi195 ceph-mon[32833]: pgmap v174: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T00:22:11.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:11 smithi170 ceph-mon[32807]: pgmap v175: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T00:22:11.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:11 smithi107 ceph-mon[29153]: pgmap v175: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T00:22:12.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:11 smithi195 ceph-mon[32833]: pgmap v175: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T00:22:13.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:13 smithi170 ceph-mon[32807]: pgmap v176: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T00:22:13.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:13 smithi107 ceph-mon[29153]: pgmap v176: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T00:22:14.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:13 smithi195 ceph-mon[32833]: pgmap v176: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T00:22:15.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:15 smithi170 ceph-mon[32807]: pgmap v177: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:22:15.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:15 smithi107 ceph-mon[29153]: pgmap v177: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:22:16.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:15 smithi195 ceph-mon[32833]: pgmap v177: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:22:17.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:17 smithi170 ceph-mon[32807]: pgmap v178: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:22:17.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:17 smithi107 ceph-mon[29153]: pgmap v178: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:22:18.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:17 smithi195 ceph-mon[32833]: pgmap v178: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:22:19.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:19 smithi170 ceph-mon[32807]: pgmap v179: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T00:22:19.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:19 smithi107 ceph-mon[29153]: pgmap v179: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T00:22:20.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:19 smithi195 ceph-mon[32833]: pgmap v179: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T00:22:21.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:21 smithi170 ceph-mon[32807]: pgmap v180: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T00:22:21.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:21 smithi107 ceph-mon[29153]: pgmap v180: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T00:22:22.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:21 smithi195 ceph-mon[32833]: pgmap v180: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T00:22:23.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:23 smithi170 ceph-mon[32807]: pgmap v181: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T00:22:23.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:23 smithi107 ceph-mon[29153]: pgmap v181: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T00:22:24.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:23 smithi195 ceph-mon[32833]: pgmap v181: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T00:22:25.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:25 smithi170 ceph-mon[32807]: pgmap v182: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:25.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:25 smithi170 ceph-mon[32807]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:22:25.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:25 smithi107 ceph-mon[29153]: pgmap v182: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:25.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:25 smithi107 ceph-mon[29153]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:22:26.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:25 smithi195 ceph-mon[32833]: pgmap v182: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:26.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:25 smithi195 ceph-mon[32833]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:22:27.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:27 smithi170 ceph-mon[32807]: pgmap v183: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:27.956 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:22:27.956 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:22:27.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:27 smithi107 ceph-mon[29153]: pgmap v183: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:28.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.068+0000 7f74a04f5640 1 -- 172.21.15.107:0/3752920726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7498100540 msgr2=0x7f7498100920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.068+0000 7f74a04f5640 1 --2- 172.21.15.107:0/3752920726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7498100540 0x7f7498100920 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7494009ca0 tx=0x7f749402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.068+0000 7f74a04f5640 1 -- 172.21.15.107:0/3752920726 shutdown_connections 2024-04-25T00:22:28.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.068+0000 7f74a04f5640 1 --2- 172.21.15.107:0/3752920726 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f749810e0f0 0x7f74981104e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.068+0000 7f74a04f5640 1 --2- 172.21.15.107:0/3752920726 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7498100e60 0x7f749810dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.068+0000 7f74a04f5640 1 --2- 172.21.15.107:0/3752920726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7498100540 0x7f7498100920 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.068+0000 7f74a04f5640 1 -- 172.21.15.107:0/3752920726 >> 172.21.15.107:0/3752920726 conn(0x7f7498077740 msgr2=0x7f7498077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.068+0000 7f74a04f5640 1 -- 172.21.15.107:0/3752920726 shutdown_connections 2024-04-25T00:22:28.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.068+0000 7f74a04f5640 1 -- 172.21.15.107:0/3752920726 wait complete. 2024-04-25T00:22:28.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.068+0000 7f74a04f5640 1 Processor -- start 2024-04-25T00:22:28.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.068+0000 7f74a04f5640 1 -- start start 2024-04-25T00:22:28.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.068+0000 7f74a04f5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7498100540 0x7f749819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.068+0000 7f74a04f5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7498100e60 0x7f749819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.068+0000 7f74a04f5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f749810e0f0 0x7f749819eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.069+0000 7f74a04f5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7498113d80 con 0x7f7498100540 2024-04-25T00:22:28.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.069+0000 7f74a04f5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7498113c00 con 0x7f7498100e60 2024-04-25T00:22:28.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.069+0000 7f74a04f5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7498113f00 con 0x7f749810e0f0 2024-04-25T00:22:28.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.069+0000 7f749e291640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7498100540 0x7f749819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.069+0000 7f749e291640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7498100540 0x7f749819a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49850/0 (socket says 172.21.15.107:49850) 2024-04-25T00:22:28.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.069+0000 7f749e291640 1 -- 172.21.15.107:0/541086117 learned_addr learned my addr 172.21.15.107:0/541086117 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:28.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.069+0000 7f749da90640 1 --2- 172.21.15.107:0/541086117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7498100e60 0x7f749819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.069+0000 7f749ea92640 1 --2- 172.21.15.107:0/541086117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f749810e0f0 0x7f749819eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.069+0000 7f749e291640 1 -- 172.21.15.107:0/541086117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f749810e0f0 msgr2=0x7f749819eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.069+0000 7f749e291640 1 --2- 172.21.15.107:0/541086117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f749810e0f0 0x7f749819eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.069+0000 7f749e291640 1 -- 172.21.15.107:0/541086117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7498100e60 msgr2=0x7f749819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.069+0000 7f749e291640 1 --2- 172.21.15.107:0/541086117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7498100e60 0x7f749819a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.069+0000 7f749e291640 1 -- 172.21.15.107:0/541086117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f749819f210 con 0x7f7498100540 2024-04-25T00:22:28.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.069+0000 7f749ea92640 1 --2- 172.21.15.107:0/541086117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f749810e0f0 0x7f749819eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:28.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.069+0000 7f749da90640 1 --2- 172.21.15.107:0/541086117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7498100e60 0x7f749819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:28.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.069+0000 7f749e291640 1 --2- 172.21.15.107:0/541086117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7498100540 0x7f749819a220 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f749402ade0 tx=0x7f74940319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.069+0000 7f748f7fe640 1 -- 172.21.15.107:0/541086117 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749402fa60 con 0x7f7498100540 2024-04-25T00:22:28.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.069+0000 7f74a04f5640 1 -- 172.21.15.107:0/541086117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f749819f4a0 con 0x7f7498100540 2024-04-25T00:22:28.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.069+0000 7f748f7fe640 1 -- 172.21.15.107:0/541086117 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f749402fc00 con 0x7f7498100540 2024-04-25T00:22:28.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.069+0000 7f748f7fe640 1 -- 172.21.15.107:0/541086117 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7494046800 con 0x7f7498100540 2024-04-25T00:22:28.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.069+0000 7f74a04f5640 1 -- 172.21.15.107:0/541086117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74981a6d90 con 0x7f7498100540 2024-04-25T00:22:28.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.071+0000 7f748f7fe640 1 -- 172.21.15.107:0/541086117 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f74940469a0 con 0x7f7498100540 2024-04-25T00:22:28.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.072+0000 7f74a31ff740 1 Processor -- start 2024-04-25T00:22:28.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.072+0000 7f74a31ff740 1 -- start start 2024-04-25T00:22:28.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.072+0000 7f748f7fe640 1 --2- 172.21.15.107:0/541086117 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f74840a2d00 0x7f74840a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.072+0000 7f74a31ff740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f1042f630 0x563f1042fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.072+0000 7f74a31ff740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1042ff50 0x563f1043a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.072+0000 7f74a31ff740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f1043a620 0x563f1043ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.072+0000 7f74a31ff740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563f103946b0 con 0x563f1042ff50 2024-04-25T00:22:28.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.072+0000 7f74a31ff740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563f1033c880 con 0x563f1042f630 2024-04-25T00:22:28.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.072+0000 7f74a31ff740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563f10390670 con 0x563f1043a620 2024-04-25T00:22:28.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.072+0000 7f749ea92640 1 --2- 172.21.15.107:0/541086117 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f74840a2d00 0x7f74840a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.072+0000 7f748f7fe640 1 -- 172.21.15.107:0/541086117 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f74940f0c60 con 0x7f7498100540 2024-04-25T00:22:28.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.073+0000 7f749da90640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1042ff50 0x563f1043a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.073+0000 7f749da90640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1042ff50 0x563f1043a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49854/0 (socket says 172.21.15.107:49854) 2024-04-25T00:22:28.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.073+0000 7f749da90640 1 -- 172.21.15.107:0/356991594 learned_addr learned my addr 172.21.15.107:0/356991594 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:28.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.073+0000 7f749e291640 1 --2- 172.21.15.107:0/356991594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f1042f630 0x563f1042fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.073+0000 7f749ea92640 1 --2- 172.21.15.107:0/356991594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f1043a620 0x563f1043ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.073+0000 7f749da90640 1 -- 172.21.15.107:0/356991594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f1043a620 msgr2=0x563f1043ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.073+0000 7f749da90640 1 --2- 172.21.15.107:0/356991594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f1043a620 0x563f1043ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.073+0000 7f749da90640 1 -- 172.21.15.107:0/356991594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f1042f630 msgr2=0x563f1042fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.073+0000 7f749da90640 1 --2- 172.21.15.107:0/356991594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f1042f630 0x563f1042fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.073+0000 7f749da90640 1 -- 172.21.15.107:0/356991594 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f101727d0 con 0x563f1042ff50 2024-04-25T00:22:28.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.073+0000 7f749ea92640 1 --2- 172.21.15.107:0/541086117 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f74840a2d00 0x7f74840a51c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7490006fd0 tx=0x7f7490013650 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.073+0000 7f749da90640 1 --2- 172.21.15.107:0/356991594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1042ff50 0x563f1043a0e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7488009c20 tx=0x7f748800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11321c0f9d0479d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.073+0000 7f748cbfe640 1 -- 172.21.15.107:0/356991594 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7488016070 con 0x563f1042ff50 2024-04-25T00:22:28.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.073+0000 7f748cbfe640 1 -- 172.21.15.107:0/356991594 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f748800ee70 con 0x563f1042ff50 2024-04-25T00:22:28.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.073+0000 7f748cbfe640 1 -- 172.21.15.107:0/356991594 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7488010800 con 0x563f1042ff50 2024-04-25T00:22:28.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.074+0000 7f74a31ff740 1 -- 172.21.15.107:0/356991594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1042ff50 msgr2=0x563f1043a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.074+0000 7f74a31ff740 1 --2- 172.21.15.107:0/356991594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1042ff50 0x563f1043a0e0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7488009c20 tx=0x7f748800e6c0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.074+0000 7f74a31ff740 1 -- 172.21.15.107:0/356991594 shutdown_connections 2024-04-25T00:22:28.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.074+0000 7f74a31ff740 1 --2- 172.21.15.107:0/356991594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f1043a620 0x563f1043ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.074+0000 7f74a31ff740 1 --2- 172.21.15.107:0/356991594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1042ff50 0x563f1043a0e0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.074+0000 7f74a31ff740 1 --2- 172.21.15.107:0/356991594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f1042f630 0x563f1042fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.074+0000 7f74a31ff740 1 -- 172.21.15.107:0/356991594 >> 172.21.15.107:0/356991594 conn(0x563f1042b950 msgr2=0x563f1042bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.074+0000 7f74a31ff740 1 -- 172.21.15.107:0/356991594 shutdown_connections 2024-04-25T00:22:28.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.074+0000 7f74a31ff740 1 -- 172.21.15.107:0/356991594 wait complete. 2024-04-25T00:22:28.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.074+0000 7f74a31ff740 1 -- start start 2024-04-25T00:22:28.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.074+0000 7f74a31ff740 1 Processor -- start 2024-04-25T00:22:28.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.075+0000 7f74a31ff740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f1042f630 0x563f10437560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.075+0000 7f74a31ff740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f1042ff50 0x563f10437aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.075+0000 7f74a31ff740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1043a620 0x563f10437fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.075+0000 7f74a31ff740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563f1043d410 con 0x563f1043a620 2024-04-25T00:22:28.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.075+0000 7f74a31ff740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563f1034ff50 con 0x563f1042ff50 2024-04-25T00:22:28.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.075+0000 7f74a31ff740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563f1043d590 con 0x563f1042f630 2024-04-25T00:22:28.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.075+0000 7f74a31ff740 10 client.0 _ll_register_callbacks cb 0x563f1038b000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:28.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.075+0000 7f749ea92640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1043a620 0x563f10437fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.075+0000 7f749ea92640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1043a620 0x563f10437fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49856/0 (socket says 172.21.15.107:49856) 2024-04-25T00:22:28.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.075+0000 7f749ea92640 1 -- 172.21.15.107:0/1871868209 learned_addr learned my addr 172.21.15.107:0/1871868209 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:28.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.075+0000 7f749da90640 1 --2- 172.21.15.107:0/1871868209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f1042ff50 0x563f10437aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.075+0000 7f749e291640 1 --2- 172.21.15.107:0/1871868209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f1042f630 0x563f10437560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.075+0000 7f749ea92640 1 -- 172.21.15.107:0/1871868209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f1042f630 msgr2=0x563f10437560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.075+0000 7f749ea92640 1 --2- 172.21.15.107:0/1871868209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f1042f630 0x563f10437560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.075+0000 7f749ea92640 1 -- 172.21.15.107:0/1871868209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f1042ff50 msgr2=0x563f10437aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.075+0000 7f749ea92640 1 --2- 172.21.15.107:0/1871868209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f1042ff50 0x563f10437aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.075+0000 7f749ea92640 1 -- 172.21.15.107:0/1871868209 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f100a83f0 con 0x563f1043a620 2024-04-25T00:22:28.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.075+0000 7f749ea92640 1 --2- 172.21.15.107:0/1871868209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1043a620 0x563f10437fe0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f7490018b70 tx=0x7f74900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600a04b3634e9f7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.075+0000 7f7481ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:22:28.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.075+0000 7f7481ffb640 1 -- 172.21.15.107:0/1871868209 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7490027060 con 0x563f1043a620 2024-04-25T00:22:28.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.075+0000 7f7481ffb640 1 -- 172.21.15.107:0/1871868209 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f7490018070 con 0x563f1043a620 2024-04-25T00:22:28.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.075+0000 7f74a31ff740 1 -- 172.21.15.107:0/1871868209 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f748800ee70 con 0x563f1043a620 2024-04-25T00:22:28.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.076+0000 7f7481ffb640 1 -- 172.21.15.107:0/1871868209 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f749002e050 con 0x563f1043a620 2024-04-25T00:22:28.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.076+0000 7f74a31ff740 10 client.16872 fetch_fsmap learned FSMap version 34 2024-04-25T00:22:28.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.076+0000 7f74a31ff740 1 -- 172.21.15.107:0/1871868209 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563f100680f0 con 0x563f1043a620 2024-04-25T00:22:28.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.076+0000 7f7481ffb640 1 -- 172.21.15.107:0/1871868209 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f748800ee70 con 0x563f1043a620 2024-04-25T00:22:28.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.076+0000 7f7481ffb640 1 -- 172.21.15.107:0/1871868209 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f7490015040 con 0x563f1043a620 2024-04-25T00:22:28.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.076+0000 7f74a31ff740 10 client.16872 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:22:28.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.076+0000 7f74a31ff740 10 client.16872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:28.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.076+0000 7f74a31ff740 20 client.16872 populate_metadata read hostname 'smithi107' 2024-04-25T00:22:28.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.076+0000 7f74a31ff740 1 --2- 172.21.15.107:0/1871868209 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x563f1044a940 0x563f1044cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.076+0000 7f74a31ff740 4 client.16872 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:22:28.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.076+0000 7f74a31ff740 1 -- 172.21.15.107:0/1871868209 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f103a1bf0 con 0x563f1044a940 2024-04-25T00:22:28.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.076+0000 7f749e291640 1 --2- 172.21.15.107:0/1871868209 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x563f1044a940 0x563f1044cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.077+0000 7f749e291640 1 --2- 172.21.15.107:0/1871868209 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x563f1044a940 0x563f1044cd90 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb6bd0eed918861e server_cookie=1a6ee3d462a3f5b1 in_seq=0 out_seq=0 2024-04-25T00:22:28.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.077+0000 7f7481ffb640 10 client.16872 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:22:28.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.078+0000 7f7481ffb640 1 -- 172.21.15.107:0/1871868209 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f103a1bf0 con 0x563f1044a940 2024-04-25T00:22:28.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.078+0000 7f7481ffb640 10 client.16872 handle_command_reply: tid=0 2024-04-25T00:22:28.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.078+0000 7f74a31ff740 1 client.16872 shutdown 2024-04-25T00:22:28.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.078+0000 7f74a31ff740 1 -- 172.21.15.107:0/1871868209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1043a620 msgr2=0x563f10437fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.078+0000 7f74a31ff740 1 --2- 172.21.15.107:0/1871868209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1043a620 0x563f10437fe0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f7490018b70 tx=0x7f74900046c0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.078+0000 7f74a31ff740 1 -- 172.21.15.107:0/1871868209 shutdown_connections 2024-04-25T00:22:28.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.078+0000 7f74a31ff740 1 --2- 172.21.15.107:0/1871868209 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x563f1044a940 0x563f1044cd90 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.078+0000 7f74a31ff740 1 --2- 172.21.15.107:0/1871868209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1043a620 0x563f10437fe0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.078+0000 7f74a31ff740 1 --2- 172.21.15.107:0/1871868209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f1042ff50 0x563f10437aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.078+0000 7f7481ffb640 0 client.16872 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:22:28.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.078+0000 7f74a31ff740 1 --2- 172.21.15.107:0/1871868209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f1042f630 0x563f10437560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.078+0000 7f74a31ff740 1 -- 172.21.15.107:0/1871868209 >> 172.21.15.107:0/1871868209 conn(0x563f1042b950 msgr2=0x563f10431650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.078+0000 7f74a31ff740 1 -- 172.21.15.107:0/1871868209 shutdown_connections 2024-04-25T00:22:28.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.078+0000 7f74a31ff740 1 -- 172.21.15.107:0/1871868209 wait complete. 2024-04-25T00:22:28.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.078+0000 7f74a31ff740 20 client.16872 trim_cache size 0 max 16384 2024-04-25T00:22:28.092 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:27 smithi195 ceph-mon[32833]: pgmap v183: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:28.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.095+0000 7f74a31ff740 1 Processor -- start 2024-04-25T00:22:28.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.095+0000 7f74a31ff740 1 -- start start 2024-04-25T00:22:28.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.095+0000 7f74a31ff740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f1042f630 0x563f10446300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.095+0000 7f74a31ff740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1042ff50 0x563f1042bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.095+0000 7f74a31ff740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f1043a620 0x563f10431d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.095+0000 7f74a31ff740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f74981b3580 con 0x563f1042ff50 2024-04-25T00:22:28.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.095+0000 7f74a31ff740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f74981b0e50 con 0x563f1043a620 2024-04-25T00:22:28.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.095+0000 7f74a31ff740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f74981aeca0 con 0x563f1042f630 2024-04-25T00:22:28.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.095+0000 7f749da90640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1042ff50 0x563f1042bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.095+0000 7f749da90640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1042ff50 0x563f1042bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49858/0 (socket says 172.21.15.107:49858) 2024-04-25T00:22:28.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.095+0000 7f749da90640 1 -- 172.21.15.107:0/1362477153 learned_addr learned my addr 172.21.15.107:0/1362477153 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:28.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.096+0000 7f749da90640 1 -- 172.21.15.107:0/1362477153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f1042f630 msgr2=0x563f10446300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.096+0000 7f749da90640 1 --2- 172.21.15.107:0/1362477153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f1042f630 0x563f10446300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.096+0000 7f749da90640 1 -- 172.21.15.107:0/1362477153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f1043a620 msgr2=0x563f10431d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.096+0000 7f749da90640 1 --2- 172.21.15.107:0/1362477153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f1043a620 0x563f10431d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.096+0000 7f749da90640 1 -- 172.21.15.107:0/1362477153 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f1041d090 con 0x563f1042ff50 2024-04-25T00:22:28.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.096+0000 7f749da90640 1 --2- 172.21.15.107:0/1362477153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1042ff50 0x563f1042bd30 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f7488009b70 tx=0x7f74880104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3f9a419455d291 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.096+0000 7f7483fff640 1 -- 172.21.15.107:0/1362477153 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7488028020 con 0x563f1042ff50 2024-04-25T00:22:28.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.096+0000 7f7483fff640 1 -- 172.21.15.107:0/1362477153 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7488009d90 con 0x563f1042ff50 2024-04-25T00:22:28.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.096+0000 7f7483fff640 1 -- 172.21.15.107:0/1362477153 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7488016040 con 0x563f1042ff50 2024-04-25T00:22:28.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.097+0000 7f74a31ff740 1 -- 172.21.15.107:0/1362477153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1042ff50 msgr2=0x563f1042bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.097+0000 7f74a31ff740 1 --2- 172.21.15.107:0/1362477153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1042ff50 0x563f1042bd30 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f7488009b70 tx=0x7f74880104a0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.097+0000 7f74a31ff740 1 -- 172.21.15.107:0/1362477153 shutdown_connections 2024-04-25T00:22:28.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.097+0000 7f74a31ff740 1 --2- 172.21.15.107:0/1362477153 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f1043a620 0x563f10431d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.097+0000 7f74a31ff740 1 --2- 172.21.15.107:0/1362477153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1042ff50 0x563f1042bd30 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.097+0000 7f74a31ff740 1 --2- 172.21.15.107:0/1362477153 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f1042f630 0x563f10446300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.097+0000 7f74a31ff740 1 -- 172.21.15.107:0/1362477153 >> 172.21.15.107:0/1362477153 conn(0x563f1042b950 msgr2=0x563f1042a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.097+0000 7f74a31ff740 1 -- 172.21.15.107:0/1362477153 shutdown_connections 2024-04-25T00:22:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.097+0000 7f74a31ff740 1 -- 172.21.15.107:0/1362477153 wait complete. 2024-04-25T00:22:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.097+0000 7f74a31ff740 1 -- start start 2024-04-25T00:22:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.097+0000 7f74a31ff740 1 Processor -- start 2024-04-25T00:22:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.097+0000 7f74a31ff740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f1042f630 0x563f10442cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.097+0000 7f74a31ff740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1042ff50 0x563f10433fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.097+0000 7f74a31ff740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f1043a620 0x563f10434580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.097+0000 7f74a31ff740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f74981af310 con 0x563f1042ff50 2024-04-25T00:22:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.097+0000 7f74a31ff740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f74981b0780 con 0x563f1042f630 2024-04-25T00:22:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.097+0000 7f74a31ff740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f74981af9e0 con 0x563f1043a620 2024-04-25T00:22:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.097+0000 7f74a31ff740 10 client.0 _ll_register_callbacks cb 0x7f7490018820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.097+0000 7f749da90640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1042ff50 0x563f10433fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.097+0000 7f749da90640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1042ff50 0x563f10433fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49872/0 (socket says 172.21.15.107:49872) 2024-04-25T00:22:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.097+0000 7f749da90640 1 -- 172.21.15.107:0/1379338733 learned_addr learned my addr 172.21.15.107:0/1379338733 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.098+0000 7f749e291640 1 --2- 172.21.15.107:0/1379338733 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f1042f630 0x563f10442cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.098+0000 7f749ea92640 1 --2- 172.21.15.107:0/1379338733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f1043a620 0x563f10434580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.098+0000 7f749da90640 1 -- 172.21.15.107:0/1379338733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f1043a620 msgr2=0x563f10434580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.098+0000 7f749da90640 1 --2- 172.21.15.107:0/1379338733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f1043a620 0x563f10434580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.098+0000 7f749da90640 1 -- 172.21.15.107:0/1379338733 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f1042f630 msgr2=0x563f10442cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.098+0000 7f749da90640 1 --2- 172.21.15.107:0/1379338733 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f1042f630 0x563f10442cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.098+0000 7f749da90640 1 -- 172.21.15.107:0/1379338733 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f10434db0 con 0x563f1042ff50 2024-04-25T00:22:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.098+0000 7f749ea92640 1 --2- 172.21.15.107:0/1379338733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f1043a620 0x563f10434580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.098+0000 7f749da90640 1 --2- 172.21.15.107:0/1379338733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1042ff50 0x563f10433fe0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f7488010410 tx=0x7f7488009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59cdbc123cf0673 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.098+0000 7f7481ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:22:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.098+0000 7f7481ffb640 1 -- 172.21.15.107:0/1379338733 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7488028070 con 0x563f1042ff50 2024-04-25T00:22:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.098+0000 7f7481ffb640 1 -- 172.21.15.107:0/1379338733 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f7488003a00 con 0x563f1042ff50 2024-04-25T00:22:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.098+0000 7f74a31ff740 1 -- 172.21.15.107:0/1379338733 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7488009d90 con 0x563f1042ff50 2024-04-25T00:22:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.098+0000 7f7481ffb640 1 -- 172.21.15.107:0/1379338733 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f748803a050 con 0x563f1042ff50 2024-04-25T00:22:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.098+0000 7f74a31ff740 10 client.16878 fetch_fsmap learned FSMap version 34 2024-04-25T00:22:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.098+0000 7f74a31ff740 1 -- 172.21.15.107:0/1379338733 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563f104b0f50 con 0x563f1042ff50 2024-04-25T00:22:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.098+0000 7f7481ffb640 1 -- 172.21.15.107:0/1379338733 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7488009d90 con 0x563f1042ff50 2024-04-25T00:22:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.098+0000 7f7481ffb640 1 -- 172.21.15.107:0/1379338733 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f7488035030 con 0x563f1042ff50 2024-04-25T00:22:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.098+0000 7f74a31ff740 10 client.16878 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:22:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.098+0000 7f74a31ff740 10 client.16878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.098+0000 7f74a31ff740 20 client.16878 populate_metadata read hostname 'smithi107' 2024-04-25T00:22:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.099+0000 7f74a31ff740 1 --2- 172.21.15.107:0/1379338733 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x563f104b1500 0x563f104b3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.099+0000 7f74a31ff740 4 client.16878 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:22:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.099+0000 7f74a31ff740 1 -- 172.21.15.107:0/1379338733 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563f1039bf40 con 0x563f104b1500 2024-04-25T00:22:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.099+0000 7f749e291640 1 --2- 172.21.15.107:0/1379338733 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x563f104b1500 0x563f104b3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.099+0000 7f7481ffb640 10 client.16878 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:22:28.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.099+0000 7f749e291640 1 --2- 172.21.15.107:0/1379338733 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x563f104b1500 0x563f104b3920 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5de525fd66db77e server_cookie=3a4e54a6eff6ac61 in_seq=0 out_seq=0 2024-04-25T00:22:28.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.099+0000 7f7481ffb640 1 -- 172.21.15.107:0/1379338733 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563f1039bf40 con 0x563f104b1500 2024-04-25T00:22:28.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f7481ffb640 10 client.16878 handle_command_reply: tid=0 2024-04-25T00:22:28.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f74a31ff740 1 client.16878 shutdown 2024-04-25T00:22:28.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f74a31ff740 1 -- 172.21.15.107:0/1379338733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1042ff50 msgr2=0x563f10433fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f74a31ff740 1 --2- 172.21.15.107:0/1379338733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1042ff50 0x563f10433fe0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f7488010410 tx=0x7f7488009910 comp rx=0 tx=0).stop 2024-04-25T00:22:28.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f74a31ff740 1 -- 172.21.15.107:0/1379338733 shutdown_connections 2024-04-25T00:22:28.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f74a31ff740 1 --2- 172.21.15.107:0/1379338733 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x563f104b1500 0x563f104b3920 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f74a31ff740 1 --2- 172.21.15.107:0/1379338733 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563f1043a620 0x563f10434580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f7481ffb640 0 client.16878 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:22:28.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f74a31ff740 1 --2- 172.21.15.107:0/1379338733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563f1042ff50 0x563f10433fe0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f74a31ff740 1 --2- 172.21.15.107:0/1379338733 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563f1042f630 0x563f10442cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f74a31ff740 1 -- 172.21.15.107:0/1379338733 >> 172.21.15.107:0/1379338733 conn(0x563f1042b950 msgr2=0x563f1042aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f74a31ff740 1 -- 172.21.15.107:0/1379338733 shutdown_connections 2024-04-25T00:22:28.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f74a31ff740 1 -- 172.21.15.107:0/1379338733 wait complete. 2024-04-25T00:22:28.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f74a31ff740 20 client.16878 trim_cache size 0 max 16384 2024-04-25T00:22:28.109 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:22:28.109 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-04-25T00:22:28.109 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-04-25T00:22:28.109 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:22:28.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f74a04f5640 1 -- 172.21.15.107:0/541086117 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f74840a2d00 msgr2=0x7f74840a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f74a04f5640 1 --2- 172.21.15.107:0/541086117 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f74840a2d00 0x7f74840a51c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7490006fd0 tx=0x7f7490013650 comp rx=0 tx=0).stop 2024-04-25T00:22:28.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f74a04f5640 1 -- 172.21.15.107:0/541086117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7498100540 msgr2=0x7f749819a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f74a04f5640 1 --2- 172.21.15.107:0/541086117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7498100540 0x7f749819a220 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f749402ade0 tx=0x7f74940319f0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f74a04f5640 1 -- 172.21.15.107:0/541086117 shutdown_connections 2024-04-25T00:22:28.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f74a04f5640 1 --2- 172.21.15.107:0/541086117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f749810e0f0 0x7f749819eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f74a04f5640 1 --2- 172.21.15.107:0/541086117 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f74840a2d00 0x7f74840a51c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f74a04f5640 1 --2- 172.21.15.107:0/541086117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7498100e60 0x7f749819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f74a04f5640 1 --2- 172.21.15.107:0/541086117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7498100540 0x7f749819a220 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f74a04f5640 1 -- 172.21.15.107:0/541086117 >> 172.21.15.107:0/541086117 conn(0x7f7498077740 msgr2=0x7f749810f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f74a04f5640 1 -- 172.21.15.107:0/541086117 shutdown_connections 2024-04-25T00:22:28.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.100+0000 7f74a04f5640 1 -- 172.21.15.107:0/541086117 wait complete. 2024-04-25T00:22:28.113 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:22:28.113 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:22:28.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.249+0000 7f34b4dd5640 1 -- 172.21.15.107:0/197917347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34b0100620 msgr2=0x7f34b0100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.249+0000 7f34b4dd5640 1 --2- 172.21.15.107:0/197917347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34b0100620 0x7f34b0100aa0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009930 tx=0x7f34a002a420 comp rx=0 tx=0).stop 2024-04-25T00:22:28.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.249+0000 7f34b4dd5640 1 -- 172.21.15.107:0/197917347 shutdown_connections 2024-04-25T00:22:28.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.249+0000 7f34b4dd5640 1 --2- 172.21.15.107:0/197917347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34b0100fe0 0x7f34b010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.249+0000 7f34b4dd5640 1 --2- 172.21.15.107:0/197917347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34b0100620 0x7f34b0100aa0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.249+0000 7f34b4dd5640 1 --2- 172.21.15.107:0/197917347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34b0106630 0x7f34b0106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.249+0000 7f34b4dd5640 1 -- 172.21.15.107:0/197917347 >> 172.21.15.107:0/197917347 conn(0x7f34b0077740 msgr2=0x7f34b0077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.249+0000 7f34b4dd5640 1 -- 172.21.15.107:0/197917347 shutdown_connections 2024-04-25T00:22:28.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.249+0000 7f34b4dd5640 1 -- 172.21.15.107:0/197917347 wait complete. 2024-04-25T00:22:28.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.250+0000 7f34b4dd5640 1 Processor -- start 2024-04-25T00:22:28.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.250+0000 7f34b4dd5640 1 -- start start 2024-04-25T00:22:28.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.250+0000 7f34b4dd5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34b0100620 0x7f34b0196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.250+0000 7f34b4dd5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34b0100fe0 0x7f34b0197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.250+0000 7f34b4dd5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34b0106630 0x7f34b019b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.250+0000 7f34b4dd5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f34b0110a40 con 0x7f34b0100fe0 2024-04-25T00:22:28.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.250+0000 7f34b4dd5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f34b01108c0 con 0x7f34b0100620 2024-04-25T00:22:28.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.250+0000 7f34b4dd5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f34b0110bc0 con 0x7f34b0106630 2024-04-25T00:22:28.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.250+0000 7f34add9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34b0100fe0 0x7f34b0197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.250+0000 7f34add9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34b0100fe0 0x7f34b0197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49894/0 (socket says 172.21.15.107:49894) 2024-04-25T00:22:28.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.250+0000 7f34add9b640 1 -- 172.21.15.107:0/363883986 learned_addr learned my addr 172.21.15.107:0/363883986 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:28.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.251+0000 7f34aed9d640 1 --2- 172.21.15.107:0/363883986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34b0106630 0x7f34b019b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.251+0000 7f34ae59c640 1 --2- 172.21.15.107:0/363883986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34b0100620 0x7f34b0196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.251+0000 7f34add9b640 1 -- 172.21.15.107:0/363883986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34b0106630 msgr2=0x7f34b019b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.251+0000 7f34add9b640 1 --2- 172.21.15.107:0/363883986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34b0106630 0x7f34b019b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.251+0000 7f34add9b640 1 -- 172.21.15.107:0/363883986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34b0100620 msgr2=0x7f34b0196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.251+0000 7f34add9b640 1 --2- 172.21.15.107:0/363883986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34b0100620 0x7f34b0196fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.251+0000 7f34add9b640 1 -- 172.21.15.107:0/363883986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b019bfb0 con 0x7f34b0100fe0 2024-04-25T00:22:28.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.251+0000 7f34aed9d640 1 --2- 172.21.15.107:0/363883986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34b0106630 0x7f34b019b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:28.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.251+0000 7f34ae59c640 1 --2- 172.21.15.107:0/363883986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34b0100620 0x7f34b0196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:28.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.251+0000 7f34add9b640 1 --2- 172.21.15.107:0/363883986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34b0100fe0 0x7f34b0197500 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009230 tx=0x7f34a0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.251+0000 7f349f7fe640 1 -- 172.21.15.107:0/363883986 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a003f030 con 0x7f34b0100fe0 2024-04-25T00:22:28.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.251+0000 7f349f7fe640 1 -- 172.21.15.107:0/363883986 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34a0003ad0 con 0x7f34b0100fe0 2024-04-25T00:22:28.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.251+0000 7f34b4dd5640 1 -- 172.21.15.107:0/363883986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b019c240 con 0x7f34b0100fe0 2024-04-25T00:22:28.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.251+0000 7f349f7fe640 1 -- 172.21.15.107:0/363883986 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0035070 con 0x7f34b0100fe0 2024-04-25T00:22:28.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.251+0000 7f34b4dd5640 1 -- 172.21.15.107:0/363883986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b01a3b30 con 0x7f34b0100fe0 2024-04-25T00:22:28.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.252+0000 7f34b4dd5640 1 -- 172.21.15.107:0/363883986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b0068560 con 0x7f34b0100fe0 2024-04-25T00:22:28.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.253+0000 7f349f7fe640 1 -- 172.21.15.107:0/363883986 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f34a0042030 con 0x7f34b0100fe0 2024-04-25T00:22:28.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.254+0000 7f349f7fe640 1 --2- 172.21.15.107:0/363883986 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f34980a2d00 0x7f34980a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.254+0000 7f34ae59c640 1 --2- 172.21.15.107:0/363883986 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f34980a2d00 0x7f34980a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.254+0000 7f349f7fe640 1 -- 172.21.15.107:0/363883986 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f34a00efa00 con 0x7f34b0100fe0 2024-04-25T00:22:28.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.254+0000 7f34ae59c640 1 --2- 172.21.15.107:0/363883986 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f34980a2d00 0x7f34980a51c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f34a4006fd0 tx=0x7f34a4008040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.258+0000 7f349f7fe640 1 -- 172.21.15.107:0/363883986 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f34a00bc2b0 con 0x7f34b0100fe0 2024-04-25T00:22:28.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.472+0000 7f34b4dd5640 1 -- 172.21.15.107:0/363883986 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34b0198310 con 0x7f34b0100fe0 2024-04-25T00:22:28.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.473+0000 7f349f7fe640 1 -- 172.21.15.107:0/363883986 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11897 (secure 0 0 0) 0x7f34a00bbbb0 con 0x7f34b0100fe0 2024-04-25T00:22:28.475 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:22:28.475 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:22:27.402680+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24493,24521]},"id":1}]} 2024-04-25T00:22:28.475 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 34 2024-04-25T00:22:28.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.475+0000 7f34b4dd5640 1 -- 172.21.15.107:0/363883986 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f34980a2d00 msgr2=0x7f34980a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.475+0000 7f34b4dd5640 1 --2- 172.21.15.107:0/363883986 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f34980a2d00 0x7f34980a51c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f34a4006fd0 tx=0x7f34a4008040 comp rx=0 tx=0).stop 2024-04-25T00:22:28.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.475+0000 7f34b4dd5640 1 -- 172.21.15.107:0/363883986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34b0100fe0 msgr2=0x7f34b0197500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.475+0000 7f34b4dd5640 1 --2- 172.21.15.107:0/363883986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34b0100fe0 0x7f34b0197500 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009230 tx=0x7f34a0004270 comp rx=0 tx=0).stop 2024-04-25T00:22:28.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.475+0000 7f34b4dd5640 1 -- 172.21.15.107:0/363883986 shutdown_connections 2024-04-25T00:22:28.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.475+0000 7f34b4dd5640 1 --2- 172.21.15.107:0/363883986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f34b0106630 0x7f34b019b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.475+0000 7f34b4dd5640 1 --2- 172.21.15.107:0/363883986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34b0100fe0 0x7f34b0197500 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.475+0000 7f34b4dd5640 1 --2- 172.21.15.107:0/363883986 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f34980a2d00 0x7f34980a51c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.475+0000 7f34b4dd5640 1 --2- 172.21.15.107:0/363883986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f34b0100620 0x7f34b0196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.475+0000 7f34b4dd5640 1 -- 172.21.15.107:0/363883986 >> 172.21.15.107:0/363883986 conn(0x7f34b0077740 msgr2=0x7f34b0102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.475+0000 7f34b4dd5640 1 -- 172.21.15.107:0/363883986 shutdown_connections 2024-04-25T00:22:28.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.475+0000 7f34b4dd5640 1 -- 172.21.15.107:0/363883986 wait complete. 2024-04-25T00:22:28.488 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:22:28.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.586+0000 7f8b69c88640 1 -- 172.21.15.107:0/3881814793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b64101710 msgr2=0x7f8b6410d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.586+0000 7f8b69c88640 1 --2- 172.21.15.107:0/3881814793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b64101710 0x7f8b6410d700 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009cd0 tx=0x7f8b5c02a8e0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.586+0000 7f8b69c88640 1 -- 172.21.15.107:0/3881814793 shutdown_connections 2024-04-25T00:22:28.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.586+0000 7f8b69c88640 1 --2- 172.21.15.107:0/3881814793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b64101710 0x7f8b6410d700 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.586+0000 7f8b69c88640 1 --2- 172.21.15.107:0/3881814793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b64100d50 0x7f8b641011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.590 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.586+0000 7f8b69c88640 1 --2- 172.21.15.107:0/3881814793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b64106840 0x7f8b64106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.586+0000 7f8b69c88640 1 -- 172.21.15.107:0/3881814793 >> 172.21.15.107:0/3881814793 conn(0x7f8b64076bd0 msgr2=0x7f8b640fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.586+0000 7f8b69c88640 1 -- 172.21.15.107:0/3881814793 shutdown_connections 2024-04-25T00:22:28.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.586+0000 7f8b69c88640 1 -- 172.21.15.107:0/3881814793 wait complete. 2024-04-25T00:22:28.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.587+0000 7f8b69c88640 1 Processor -- start 2024-04-25T00:22:28.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.587+0000 7f8b69c88640 1 -- start start 2024-04-25T00:22:28.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.587+0000 7f8b69c88640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b64100d50 0x7f8b64199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.587+0000 7f8b69c88640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b64101710 0x7f8b64199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.587+0000 7f8b69c88640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b64106840 0x7f8b641a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.587+0000 7f8b69c88640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8b64110c50 con 0x7f8b64101710 2024-04-25T00:22:28.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.587+0000 7f8b69c88640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8b64110ad0 con 0x7f8b64106840 2024-04-25T00:22:28.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.587+0000 7f8b69c88640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8b64110dd0 con 0x7f8b64100d50 2024-04-25T00:22:28.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.587+0000 7f8b62ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b64101710 0x7f8b64199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.587+0000 7f8b62ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b64101710 0x7f8b64199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49902/0 (socket says 172.21.15.107:49902) 2024-04-25T00:22:28.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.587+0000 7f8b62ffd640 1 -- 172.21.15.107:0/2360933315 learned_addr learned my addr 172.21.15.107:0/2360933315 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:28.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.587+0000 7f8b637fe640 1 --2- 172.21.15.107:0/2360933315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b64100d50 0x7f8b64199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.588+0000 7f8b63fff640 1 --2- 172.21.15.107:0/2360933315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b64106840 0x7f8b641a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.588+0000 7f8b62ffd640 1 -- 172.21.15.107:0/2360933315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b64100d50 msgr2=0x7f8b64199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.588+0000 7f8b62ffd640 1 --2- 172.21.15.107:0/2360933315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b64100d50 0x7f8b64199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.588+0000 7f8b62ffd640 1 -- 172.21.15.107:0/2360933315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b64106840 msgr2=0x7f8b641a4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.588+0000 7f8b62ffd640 1 --2- 172.21.15.107:0/2360933315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b64106840 0x7f8b641a4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.588+0000 7f8b62ffd640 1 -- 172.21.15.107:0/2360933315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b641a45d0 con 0x7f8b64101710 2024-04-25T00:22:28.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.588+0000 7f8b62ffd640 1 --2- 172.21.15.107:0/2360933315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b64101710 0x7f8b64199550 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800c920 tx=0x7f8b5800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.588+0000 7f8b60ff9640 1 -- 172.21.15.107:0/2360933315 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58007c70 con 0x7f8b64101710 2024-04-25T00:22:28.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.588+0000 7f8b69c88640 1 -- 172.21.15.107:0/2360933315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b641a47a0 con 0x7f8b64101710 2024-04-25T00:22:28.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.588+0000 7f8b60ff9640 1 -- 172.21.15.107:0/2360933315 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b58007e10 con 0x7f8b64101710 2024-04-25T00:22:28.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.588+0000 7f8b60ff9640 1 -- 172.21.15.107:0/2360933315 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58018890 con 0x7f8b64101710 2024-04-25T00:22:28.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.588+0000 7f8b69c88640 1 -- 172.21.15.107:0/2360933315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b641a4d10 con 0x7f8b64101710 2024-04-25T00:22:28.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.591+0000 7f8b60ff9640 1 -- 172.21.15.107:0/2360933315 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f8b58018a70 con 0x7f8b64101710 2024-04-25T00:22:28.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.591+0000 7f8b6c993740 1 Processor -- start 2024-04-25T00:22:28.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.591+0000 7f8b6c993740 1 -- start start 2024-04-25T00:22:28.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.591+0000 7f8b6c993740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b321e71580 0x55b321e71960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.591+0000 7f8b6c993740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e71ea0 0x55b321e7c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.591+0000 7f8b6c993740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b321e7c570 0x55b321e7e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.591+0000 7f8b6c993740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b321dd6670 con 0x55b321e71ea0 2024-04-25T00:22:28.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.591+0000 7f8b6c993740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b321d7e970 con 0x55b321e7c570 2024-04-25T00:22:28.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.591+0000 7f8b6c993740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b321dd2630 con 0x55b321e71580 2024-04-25T00:22:28.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.591+0000 7f8b637fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e71ea0 0x55b321e7c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.591+0000 7f8b637fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e71ea0 0x55b321e7c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49906/0 (socket says 172.21.15.107:49906) 2024-04-25T00:22:28.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.591+0000 7f8b637fe640 1 -- 172.21.15.107:0/2602175727 learned_addr learned my addr 172.21.15.107:0/2602175727 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:28.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.591+0000 7f8b63fff640 1 --2- 172.21.15.107:0/2602175727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b321e71580 0x55b321e71960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.591+0000 7f8b60ff9640 1 --2- 172.21.15.107:0/2360933315 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8b4c0a2d00 0x7f8b4c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.591+0000 7f8b637fe640 1 -- 172.21.15.107:0/2602175727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b321e71580 msgr2=0x55b321e71960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.591+0000 7f8b637fe640 1 --2- 172.21.15.107:0/2602175727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b321e71580 0x55b321e71960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.591+0000 7f8b637fe640 1 -- 172.21.15.107:0/2602175727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b321e7c570 msgr2=0x55b321e7e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.591+0000 7f8b637fe640 1 --2- 172.21.15.107:0/2602175727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b321e7c570 0x55b321e7e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.591+0000 7f8b637fe640 1 -- 172.21.15.107:0/2602175727 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b321da2400 con 0x55b321e71ea0 2024-04-25T00:22:28.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.592+0000 7f8b63fff640 1 --2- 172.21.15.107:0/2360933315 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8b4c0a2d00 0x7f8b4c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.592+0000 7f8b60ff9640 1 -- 172.21.15.107:0/2360933315 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f8b580cf860 con 0x7f8b64101710 2024-04-25T00:22:28.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.592+0000 7f8b637fe640 1 --2- 172.21.15.107:0/2602175727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e71ea0 0x55b321e7c030 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f8b54009c70 tx=0x7f8b54011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10af133b69735477 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.592+0000 7f8b49ffb640 1 -- 172.21.15.107:0/2602175727 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b54004230 con 0x55b321e71ea0 2024-04-25T00:22:28.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.592+0000 7f8b49ffb640 1 -- 172.21.15.107:0/2602175727 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b540043d0 con 0x55b321e71ea0 2024-04-25T00:22:28.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.592+0000 7f8b49ffb640 1 -- 172.21.15.107:0/2602175727 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b54019740 con 0x55b321e71ea0 2024-04-25T00:22:28.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.592+0000 7f8b63fff640 1 --2- 172.21.15.107:0/2360933315 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8b4c0a2d00 0x7f8b4c0a51c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009a60 tx=0x7f8b5c03e040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.592+0000 7f8b6c993740 1 -- 172.21.15.107:0/2602175727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e71ea0 msgr2=0x55b321e7c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.592+0000 7f8b6c993740 1 --2- 172.21.15.107:0/2602175727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e71ea0 0x55b321e7c030 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f8b54009c70 tx=0x7f8b54011810 comp rx=0 tx=0).stop 2024-04-25T00:22:28.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.593+0000 7f8b6c993740 1 -- 172.21.15.107:0/2602175727 shutdown_connections 2024-04-25T00:22:28.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.593+0000 7f8b6c993740 1 --2- 172.21.15.107:0/2602175727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b321e7c570 0x55b321e7e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.593+0000 7f8b6c993740 1 --2- 172.21.15.107:0/2602175727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e71ea0 0x55b321e7c030 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.593+0000 7f8b6c993740 1 --2- 172.21.15.107:0/2602175727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b321e71580 0x55b321e71960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.593+0000 7f8b6c993740 1 -- 172.21.15.107:0/2602175727 >> 172.21.15.107:0/2602175727 conn(0x55b321e6d8a0 msgr2=0x55b321e6dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.593+0000 7f8b6c993740 1 -- 172.21.15.107:0/2602175727 shutdown_connections 2024-04-25T00:22:28.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.593+0000 7f8b6c993740 1 -- 172.21.15.107:0/2602175727 wait complete. 2024-04-25T00:22:28.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.593+0000 7f8b6c993740 1 -- start start 2024-04-25T00:22:28.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.593+0000 7f8b6c993740 1 Processor -- start 2024-04-25T00:22:28.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.593+0000 7f8b6c993740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b321e71580 0x55b321e7bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.593+0000 7f8b6c993740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b321e71ea0 0x55b321e79250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.593+0000 7f8b6c993740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e7c570 0x55b321e79860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.593+0000 7f8b6c993740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b321e7f360 con 0x55b321e7c570 2024-04-25T00:22:28.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.593+0000 7f8b6c993740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b321d92050 con 0x55b321e71580 2024-04-25T00:22:28.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.593+0000 7f8b6c993740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b321e7f4e0 con 0x55b321e71ea0 2024-04-25T00:22:28.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.593+0000 7f8b6c993740 10 client.0 _ll_register_callbacks cb 0x55b321ae9be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:28.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.593+0000 7f8b63fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e7c570 0x55b321e79860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.594+0000 7f8b63fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e7c570 0x55b321e79860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49918/0 (socket says 172.21.15.107:49918) 2024-04-25T00:22:28.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.594+0000 7f8b63fff640 1 -- 172.21.15.107:0/207702862 learned_addr learned my addr 172.21.15.107:0/207702862 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:28.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.594+0000 7f8b62ffd640 1 --2- 172.21.15.107:0/207702862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b321e71ea0 0x55b321e79250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.594+0000 7f8b637fe640 1 --2- 172.21.15.107:0/207702862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b321e71580 0x55b321e7bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.594+0000 7f8b63fff640 1 -- 172.21.15.107:0/207702862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b321e71ea0 msgr2=0x55b321e79250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.594+0000 7f8b63fff640 1 --2- 172.21.15.107:0/207702862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b321e71ea0 0x55b321e79250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.594+0000 7f8b63fff640 1 -- 172.21.15.107:0/207702862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b321e71580 msgr2=0x55b321e7bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.594+0000 7f8b63fff640 1 --2- 172.21.15.107:0/207702862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b321e71580 0x55b321e7bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.594+0000 7f8b63fff640 1 -- 172.21.15.107:0/207702862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b321d63dc0 con 0x55b321e7c570 2024-04-25T00:22:28.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.594+0000 7f8b63fff640 1 --2- 172.21.15.107:0/207702862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e7c570 0x55b321e79860 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c035000 tx=0x7f8b5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ed44f302dff4f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.594+0000 7f8b42ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:22:28.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.594+0000 7f8b42ffd640 1 -- 172.21.15.107:0/207702862 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c0316a0 con 0x55b321e7c570 2024-04-25T00:22:28.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.594+0000 7f8b42ffd640 1 -- 172.21.15.107:0/207702862 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f8b5c0305a0 con 0x55b321e7c570 2024-04-25T00:22:28.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.594+0000 7f8b6c993740 1 -- 172.21.15.107:0/207702862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b540043d0 con 0x55b321e7c570 2024-04-25T00:22:28.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.594+0000 7f8b42ffd640 1 -- 172.21.15.107:0/207702862 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8b5c0307a0 con 0x55b321e7c570 2024-04-25T00:22:28.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.594+0000 7f8b6c993740 10 client.16896 fetch_fsmap learned FSMap version 34 2024-04-25T00:22:28.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.594+0000 7f8b6c993740 1 -- 172.21.15.107:0/207702862 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b321aea3f0 con 0x55b321e7c570 2024-04-25T00:22:28.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.594+0000 7f8b42ffd640 1 -- 172.21.15.107:0/207702862 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b540043d0 con 0x55b321e7c570 2024-04-25T00:22:28.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.595+0000 7f8b42ffd640 1 -- 172.21.15.107:0/207702862 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f8b5c030070 con 0x55b321e7c570 2024-04-25T00:22:28.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.595+0000 7f8b6c993740 10 client.16896 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:22:28.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.595+0000 7f8b6c993740 10 client.16896 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:22:28.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.595+0000 7f8b6c993740 20 client.16896 populate_metadata read hostname 'smithi107' 2024-04-25T00:22:28.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.595+0000 7f8b6c993740 1 --2- 172.21.15.107:0/207702862 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55b321e8c640 0x55b321e8ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.595+0000 7f8b6c993740 4 client.16896 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:22:28.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.595+0000 7f8b6c993740 1 -- 172.21.15.107:0/207702862 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b321de3bb0 con 0x55b321e8c640 2024-04-25T00:22:28.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.595+0000 7f8b637fe640 1 --2- 172.21.15.107:0/207702862 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55b321e8c640 0x55b321e8ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.595+0000 7f8b42ffd640 10 client.16896 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:22:28.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.595+0000 7f8b637fe640 1 --2- 172.21.15.107:0/207702862 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55b321e8c640 0x55b321e8ea30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c707259a30520928 server_cookie=a47892433db54643 in_seq=0 out_seq=0 2024-04-25T00:22:28.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.596+0000 7f8b42ffd640 1 -- 172.21.15.107:0/207702862 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b321de3bb0 con 0x55b321e8c640 2024-04-25T00:22:28.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.596+0000 7f8b42ffd640 10 client.16896 handle_command_reply: tid=0 2024-04-25T00:22:28.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.596+0000 7f8b6c993740 1 client.16896 shutdown 2024-04-25T00:22:28.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.596+0000 7f8b6c993740 1 -- 172.21.15.107:0/207702862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e7c570 msgr2=0x55b321e79860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.596+0000 7f8b6c993740 1 --2- 172.21.15.107:0/207702862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e7c570 0x55b321e79860 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c035000 tx=0x7f8b5c004a40 comp rx=0 tx=0).stop 2024-04-25T00:22:28.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.596+0000 7f8b6c993740 1 -- 172.21.15.107:0/207702862 shutdown_connections 2024-04-25T00:22:28.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.596+0000 7f8b6c993740 1 --2- 172.21.15.107:0/207702862 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55b321e8c640 0x55b321e8ea30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.596+0000 7f8b6c993740 1 --2- 172.21.15.107:0/207702862 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e7c570 0x55b321e79860 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.596+0000 7f8b6c993740 1 --2- 172.21.15.107:0/207702862 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b321e71ea0 0x55b321e79250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.596+0000 7f8b42ffd640 0 client.16896 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:22:28.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.596+0000 7f8b6c993740 1 --2- 172.21.15.107:0/207702862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b321e71580 0x55b321e7bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.596+0000 7f8b6c993740 1 -- 172.21.15.107:0/207702862 >> 172.21.15.107:0/207702862 conn(0x55b321e6d8a0 msgr2=0x55b321e736a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.596+0000 7f8b6c993740 1 -- 172.21.15.107:0/207702862 shutdown_connections 2024-04-25T00:22:28.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.596+0000 7f8b6c993740 1 -- 172.21.15.107:0/207702862 wait complete. 2024-04-25T00:22:28.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.596+0000 7f8b6c993740 20 client.16896 trim_cache size 0 max 16384 2024-04-25T00:22:28.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.609+0000 7f8b6c993740 1 Processor -- start 2024-04-25T00:22:28.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.609+0000 7f8b6c993740 1 -- start start 2024-04-25T00:22:28.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.609+0000 7f8b6c993740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e71580 0x55b321e88220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.609+0000 7f8b6c993740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b321e71ea0 0x55b321e94300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.609+0000 7f8b6c993740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b321e7c570 0x55b321e94840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.609+0000 7f8b6c993740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8b641b1340 con 0x55b321e71580 2024-04-25T00:22:28.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.609+0000 7f8b6c993740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8b641aec10 con 0x55b321e71ea0 2024-04-25T00:22:28.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.609+0000 7f8b6c993740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8b641aca60 con 0x55b321e7c570 2024-04-25T00:22:28.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.610+0000 7f8b637fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e71580 0x55b321e88220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.610+0000 7f8b637fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e71580 0x55b321e88220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49932/0 (socket says 172.21.15.107:49932) 2024-04-25T00:22:28.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.610+0000 7f8b637fe640 1 -- 172.21.15.107:0/920644696 learned_addr learned my addr 172.21.15.107:0/920644696 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:28.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.610+0000 7f8b63fff640 1 --2- 172.21.15.107:0/920644696 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b321e7c570 0x55b321e94840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.610+0000 7f8b637fe640 1 -- 172.21.15.107:0/920644696 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b321e7c570 msgr2=0x55b321e94840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.610+0000 7f8b62ffd640 1 --2- 172.21.15.107:0/920644696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b321e71ea0 0x55b321e94300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.610+0000 7f8b637fe640 1 --2- 172.21.15.107:0/920644696 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b321e7c570 0x55b321e94840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.610+0000 7f8b637fe640 1 -- 172.21.15.107:0/920644696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b321e71ea0 msgr2=0x55b321e94300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.610+0000 7f8b637fe640 1 --2- 172.21.15.107:0/920644696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b321e71ea0 0x55b321e94300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.610+0000 7f8b637fe640 1 -- 172.21.15.107:0/920644696 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b321e71cd0 con 0x55b321e71580 2024-04-25T00:22:28.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.610+0000 7f8b637fe640 1 --2- 172.21.15.107:0/920644696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e71580 0x55b321e88220 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f8b54009bc0 tx=0x7f8b54003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a079a2e8a3540b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.610+0000 7f8b497fa640 1 -- 172.21.15.107:0/920644696 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5402f070 con 0x55b321e71580 2024-04-25T00:22:28.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.610+0000 7f8b497fa640 1 -- 172.21.15.107:0/920644696 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b54005520 con 0x55b321e71580 2024-04-25T00:22:28.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.610+0000 7f8b497fa640 1 -- 172.21.15.107:0/920644696 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5402ae60 con 0x55b321e71580 2024-04-25T00:22:28.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.611+0000 7f8b6c993740 1 -- 172.21.15.107:0/920644696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e71580 msgr2=0x55b321e88220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.611+0000 7f8b6c993740 1 --2- 172.21.15.107:0/920644696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e71580 0x55b321e88220 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f8b54009bc0 tx=0x7f8b54003ed0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.611+0000 7f8b6c993740 1 -- 172.21.15.107:0/920644696 shutdown_connections 2024-04-25T00:22:28.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.611+0000 7f8b6c993740 1 --2- 172.21.15.107:0/920644696 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b321e7c570 0x55b321e94840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.611+0000 7f8b6c993740 1 --2- 172.21.15.107:0/920644696 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b321e71ea0 0x55b321e94300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.611+0000 7f8b6c993740 1 --2- 172.21.15.107:0/920644696 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e71580 0x55b321e88220 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.611+0000 7f8b6c993740 1 -- 172.21.15.107:0/920644696 >> 172.21.15.107:0/920644696 conn(0x55b321e6d8a0 msgr2=0x55b321e6cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.611+0000 7f8b6c993740 1 -- 172.21.15.107:0/920644696 shutdown_connections 2024-04-25T00:22:28.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.611+0000 7f8b6c993740 1 -- 172.21.15.107:0/920644696 wait complete. 2024-04-25T00:22:28.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.611+0000 7f8b6c993740 1 -- start start 2024-04-25T00:22:28.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.611+0000 7f8b6c993740 1 Processor -- start 2024-04-25T00:22:28.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.611+0000 7f8b6c993740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b321e71580 0x55b321e82460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b6c993740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b321e71ea0 0x55b321e829a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b6c993740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e7c570 0x55b321e82ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b6c993740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8b641ad0d0 con 0x55b321e7c570 2024-04-25T00:22:28.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b6c993740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8b641ae540 con 0x55b321e71580 2024-04-25T00:22:28.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b6c993740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8b641ad7a0 con 0x55b321e71ea0 2024-04-25T00:22:28.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b6c993740 10 client.0 _ll_register_callbacks cb 0x7f8b5c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:28.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b63fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e7c570 0x55b321e82ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b63fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e7c570 0x55b321e82ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49938/0 (socket says 172.21.15.107:49938) 2024-04-25T00:22:28.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b63fff640 1 -- 172.21.15.107:0/746852987 learned_addr learned my addr 172.21.15.107:0/746852987 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:28.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b62ffd640 1 --2- 172.21.15.107:0/746852987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b321e71ea0 0x55b321e829a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b637fe640 1 --2- 172.21.15.107:0/746852987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b321e71580 0x55b321e82460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b63fff640 1 -- 172.21.15.107:0/746852987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b321e71ea0 msgr2=0x55b321e829a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b63fff640 1 --2- 172.21.15.107:0/746852987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b321e71ea0 0x55b321e829a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b63fff640 1 -- 172.21.15.107:0/746852987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b321e71580 msgr2=0x55b321e82460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b63fff640 1 --2- 172.21.15.107:0/746852987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b321e71580 0x55b321e82460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b63fff640 1 -- 172.21.15.107:0/746852987 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b321e8eda0 con 0x55b321e7c570 2024-04-25T00:22:28.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b63fff640 1 --2- 172.21.15.107:0/746852987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e7c570 0x55b321e82ee0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c005e30 tx=0x7f8b5c0093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea9afb999f9009a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b42ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:22:28.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b42ffd640 1 -- 172.21.15.107:0/746852987 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c031c60 con 0x55b321e7c570 2024-04-25T00:22:28.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b42ffd640 1 -- 172.21.15.107:0/746852987 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f8b5c030940 con 0x55b321e7c570 2024-04-25T00:22:28.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b6c993740 1 -- 172.21.15.107:0/746852987 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b54005520 con 0x55b321e7c570 2024-04-25T00:22:28.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b42ffd640 1 -- 172.21.15.107:0/746852987 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8b5c068050 con 0x55b321e7c570 2024-04-25T00:22:28.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b6c993740 10 client.16902 fetch_fsmap learned FSMap version 34 2024-04-25T00:22:28.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b6c993740 1 -- 172.21.15.107:0/746852987 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b321bb47d0 con 0x55b321e7c570 2024-04-25T00:22:28.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.612+0000 7f8b42ffd640 1 -- 172.21.15.107:0/746852987 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b54005520 con 0x55b321e7c570 2024-04-25T00:22:28.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.613+0000 7f8b42ffd640 1 -- 172.21.15.107:0/746852987 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f8b5c045a10 con 0x55b321e7c570 2024-04-25T00:22:28.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.613+0000 7f8b6c993740 10 client.16902 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:22:28.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.613+0000 7f8b6c993740 10 client.16902 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:22:28.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.613+0000 7f8b6c993740 20 client.16902 populate_metadata read hostname 'smithi107' 2024-04-25T00:22:28.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.613+0000 7f8b6c993740 1 --2- 172.21.15.107:0/746852987 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55b321ef7880 0x55b321ef9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.613+0000 7f8b6c993740 4 client.16902 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:22:28.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.613+0000 7f8b6c993740 1 -- 172.21.15.107:0/746852987 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b321dddf00 con 0x55b321ef7880 2024-04-25T00:22:28.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.613+0000 7f8b637fe640 1 --2- 172.21.15.107:0/746852987 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55b321ef7880 0x55b321ef9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.613+0000 7f8b637fe640 1 --2- 172.21.15.107:0/746852987 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55b321ef7880 0x55b321ef9ca0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=beb3641f6c19e068 server_cookie=e901f8b05cd7f5aa in_seq=0 out_seq=0 2024-04-25T00:22:28.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.613+0000 7f8b42ffd640 10 client.16902 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:22:28.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.613+0000 7f8b42ffd640 1 -- 172.21.15.107:0/746852987 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b321dddf00 con 0x55b321ef7880 2024-04-25T00:22:28.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.613+0000 7f8b42ffd640 10 client.16902 handle_command_reply: tid=0 2024-04-25T00:22:28.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.613+0000 7f8b6c993740 1 client.16902 shutdown 2024-04-25T00:22:28.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.614+0000 7f8b6c993740 1 -- 172.21.15.107:0/746852987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e7c570 msgr2=0x55b321e82ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.614+0000 7f8b6c993740 1 --2- 172.21.15.107:0/746852987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e7c570 0x55b321e82ee0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c005e30 tx=0x7f8b5c0093e0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.614+0000 7f8b6c993740 1 -- 172.21.15.107:0/746852987 shutdown_connections 2024-04-25T00:22:28.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.614+0000 7f8b6c993740 1 --2- 172.21.15.107:0/746852987 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55b321ef7880 0x55b321ef9ca0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.614+0000 7f8b6c993740 1 --2- 172.21.15.107:0/746852987 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b321e7c570 0x55b321e82ee0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.614+0000 7f8b42ffd640 0 client.16902 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:22:28.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.614+0000 7f8b6c993740 1 --2- 172.21.15.107:0/746852987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b321e71ea0 0x55b321e829a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.614+0000 7f8b6c993740 1 --2- 172.21.15.107:0/746852987 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b321e71580 0x55b321e82460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.614+0000 7f8b6c993740 1 -- 172.21.15.107:0/746852987 >> 172.21.15.107:0/746852987 conn(0x55b321e6d8a0 msgr2=0x55b321e87210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.614+0000 7f8b6c993740 1 -- 172.21.15.107:0/746852987 shutdown_connections 2024-04-25T00:22:28.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.614+0000 7f8b6c993740 1 -- 172.21.15.107:0/746852987 wait complete. 2024-04-25T00:22:28.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.614+0000 7f8b6c993740 20 client.16902 trim_cache size 0 max 16384 2024-04-25T00:22:28.624 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:22:28.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.614+0000 7f8b69c88640 1 -- 172.21.15.107:0/2360933315 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8b4c0a2d00 msgr2=0x7f8b4c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.614+0000 7f8b69c88640 1 --2- 172.21.15.107:0/2360933315 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8b4c0a2d00 0x7f8b4c0a51c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009a60 tx=0x7f8b5c03e040 comp rx=0 tx=0).stop 2024-04-25T00:22:28.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.614+0000 7f8b69c88640 1 -- 172.21.15.107:0/2360933315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b64101710 msgr2=0x7f8b64199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.614+0000 7f8b69c88640 1 --2- 172.21.15.107:0/2360933315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b64101710 0x7f8b64199550 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800c920 tx=0x7f8b5800cde0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.614+0000 7f8b69c88640 1 -- 172.21.15.107:0/2360933315 shutdown_connections 2024-04-25T00:22:28.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.614+0000 7f8b69c88640 1 --2- 172.21.15.107:0/2360933315 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8b4c0a2d00 0x7f8b4c0a51c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.614+0000 7f8b69c88640 1 --2- 172.21.15.107:0/2360933315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b64106840 0x7f8b641a4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.614+0000 7f8b69c88640 1 --2- 172.21.15.107:0/2360933315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8b64101710 0x7f8b64199550 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.614+0000 7f8b69c88640 1 --2- 172.21.15.107:0/2360933315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8b64100d50 0x7f8b64199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.614+0000 7f8b69c88640 1 -- 172.21.15.107:0/2360933315 >> 172.21.15.107:0/2360933315 conn(0x7f8b64076bd0 msgr2=0x7f8b64103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.614+0000 7f8b69c88640 1 -- 172.21.15.107:0/2360933315 shutdown_connections 2024-04-25T00:22:28.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.614+0000 7f8b69c88640 1 -- 172.21.15.107:0/2360933315 wait complete. 2024-04-25T00:22:28.626 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:22:28.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.726+0000 7fdb7df16640 1 -- 172.21.15.107:0/1360272462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb78101330 msgr2=0x7fdb7810d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.726+0000 7fdb7df16640 1 --2- 172.21.15.107:0/1360272462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb78101330 0x7fdb7810d460 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009ca0 tx=0x7fdb7002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.727+0000 7fdb7df16640 1 -- 172.21.15.107:0/1360272462 shutdown_connections 2024-04-25T00:22:28.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.727+0000 7fdb7df16640 1 --2- 172.21.15.107:0/1360272462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb78101330 0x7fdb7810d460 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.727+0000 7fdb7df16640 1 --2- 172.21.15.107:0/1360272462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb78100990 0x7fdb78100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.727+0000 7fdb7df16640 1 --2- 172.21.15.107:0/1360272462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb78106690 0x7fdb78106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.727+0000 7fdb7df16640 1 -- 172.21.15.107:0/1360272462 >> 172.21.15.107:0/1360272462 conn(0x7fdb78076440 msgr2=0x7fdb78076850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.727+0000 7fdb7df16640 1 -- 172.21.15.107:0/1360272462 shutdown_connections 2024-04-25T00:22:28.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.727+0000 7fdb7df16640 1 -- 172.21.15.107:0/1360272462 wait complete. 2024-04-25T00:22:28.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.727+0000 7fdb7df16640 1 Processor -- start 2024-04-25T00:22:28.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.727+0000 7fdb7df16640 1 -- start start 2024-04-25T00:22:28.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.727+0000 7fdb7df16640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb78100990 0x7fdb78198d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.727+0000 7fdb7df16640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb78101330 0x7fdb781992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.728+0000 7fdb7df16640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb78106690 0x7fdb781a3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.728+0000 7fdb7df16640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb781109e0 con 0x7fdb78101330 2024-04-25T00:22:28.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.728+0000 7fdb7df16640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb78110860 con 0x7fdb78100990 2024-04-25T00:22:28.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.728+0000 7fdb7df16640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdb78110b60 con 0x7fdb78106690 2024-04-25T00:22:28.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.728+0000 7fdb76ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb78101330 0x7fdb781992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.728+0000 7fdb76ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb78101330 0x7fdb781992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49966/0 (socket says 172.21.15.107:49966) 2024-04-25T00:22:28.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.728+0000 7fdb76ffd640 1 -- 172.21.15.107:0/3030809071 learned_addr learned my addr 172.21.15.107:0/3030809071 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:28.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.728+0000 7fdb777fe640 1 --2- 172.21.15.107:0/3030809071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb78100990 0x7fdb78198d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.728+0000 7fdb77fff640 1 --2- 172.21.15.107:0/3030809071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb78106690 0x7fdb781a3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.728+0000 7fdb76ffd640 1 -- 172.21.15.107:0/3030809071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb78106690 msgr2=0x7fdb781a3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.728+0000 7fdb76ffd640 1 --2- 172.21.15.107:0/3030809071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb78106690 0x7fdb781a3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.728+0000 7fdb76ffd640 1 -- 172.21.15.107:0/3030809071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb78100990 msgr2=0x7fdb78198d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.728+0000 7fdb76ffd640 1 --2- 172.21.15.107:0/3030809071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb78100990 0x7fdb78198d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.728+0000 7fdb76ffd640 1 -- 172.21.15.107:0/3030809071 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb781a4340 con 0x7fdb78101330 2024-04-25T00:22:28.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.728+0000 7fdb76ffd640 1 --2- 172.21.15.107:0/3030809071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb78101330 0x7fdb781992c0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fdb6800d950 tx=0x7fdb6800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.728+0000 7fdb74ff9640 1 -- 172.21.15.107:0/3030809071 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68017070 con 0x7fdb78101330 2024-04-25T00:22:28.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.728+0000 7fdb74ff9640 1 -- 172.21.15.107:0/3030809071 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb68009c30 con 0x7fdb78101330 2024-04-25T00:22:28.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.728+0000 7fdb7df16640 1 -- 172.21.15.107:0/3030809071 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb781a4510 con 0x7fdb78101330 2024-04-25T00:22:28.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.728+0000 7fdb74ff9640 1 -- 172.21.15.107:0/3030809071 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6800f6b0 con 0x7fdb78101330 2024-04-25T00:22:28.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.728+0000 7fdb7df16640 1 -- 172.21.15.107:0/3030809071 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb781a4a30 con 0x7fdb78101330 2024-04-25T00:22:28.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.730+0000 7fdb74ff9640 1 -- 172.21.15.107:0/3030809071 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fdb680109d0 con 0x7fdb78101330 2024-04-25T00:22:28.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.731+0000 7fdb80c19740 1 Processor -- start 2024-04-25T00:22:28.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.731+0000 7fdb80c19740 1 -- start start 2024-04-25T00:22:28.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.731+0000 7fdb80c19740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2382c5580 0x55e2382c5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.731+0000 7fdb80c19740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382c5ea0 0x55e2382d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.731+0000 7fdb80c19740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2382d0570 0x55e2382d2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.731+0000 7fdb80c19740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e23822a670 con 0x55e2382c5ea0 2024-04-25T00:22:28.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.731+0000 7fdb80c19740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e2381d2970 con 0x55e2382d0570 2024-04-25T00:22:28.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.731+0000 7fdb80c19740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e238226630 con 0x55e2382c5580 2024-04-25T00:22:28.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.731+0000 7fdb74ff9640 1 --2- 172.21.15.107:0/3030809071 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdb600a2d00 0x7fdb600a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.731+0000 7fdb777fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382c5ea0 0x55e2382d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.731+0000 7fdb777fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382c5ea0 0x55e2382d0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49972/0 (socket says 172.21.15.107:49972) 2024-04-25T00:22:28.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.731+0000 7fdb777fe640 1 -- 172.21.15.107:0/3754612999 learned_addr learned my addr 172.21.15.107:0/3754612999 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:28.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.731+0000 7fdb76ffd640 1 --2- 172.21.15.107:0/3754612999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2382d0570 0x55e2382d2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.731+0000 7fdb77fff640 1 --2- 172.21.15.107:0/3030809071 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdb600a2d00 0x7fdb600a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.731+0000 7fdb77fff640 1 --2- 172.21.15.107:0/3754612999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2382c5580 0x55e2382c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.731+0000 7fdb777fe640 1 -- 172.21.15.107:0/3754612999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2382c5580 msgr2=0x55e2382c5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.731+0000 7fdb777fe640 1 --2- 172.21.15.107:0/3754612999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2382c5580 0x55e2382c5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.731+0000 7fdb777fe640 1 -- 172.21.15.107:0/3754612999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2382d0570 msgr2=0x55e2382d2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.731+0000 7fdb777fe640 1 --2- 172.21.15.107:0/3754612999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2382d0570 0x55e2382d2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.731+0000 7fdb777fe640 1 -- 172.21.15.107:0/3754612999 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2381f6400 con 0x55e2382c5ea0 2024-04-25T00:22:28.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.731+0000 7fdb74ff9640 1 -- 172.21.15.107:0/3030809071 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fdb680cf360 con 0x7fdb78101330 2024-04-25T00:22:28.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.731+0000 7fdb76ffd640 1 --2- 172.21.15.107:0/3754612999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2382d0570 0x55e2382d2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:28.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.731+0000 7fdb77fff640 1 --2- 172.21.15.107:0/3754612999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2382c5580 0x55e2382c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:28.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.732+0000 7fdb777fe640 1 --2- 172.21.15.107:0/3754612999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382c5ea0 0x55e2382d0030 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00b710 tx=0x7fdb6c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9305039c124a43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.732+0000 7fdb77fff640 1 --2- 172.21.15.107:0/3030809071 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdb600a2d00 0x7fdb600a51c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdb70004af0 tx=0x7fdb7002f310 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.732+0000 7fdb5dffb640 1 -- 172.21.15.107:0/3754612999 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c003bd0 con 0x55e2382c5ea0 2024-04-25T00:22:28.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.732+0000 7fdb5dffb640 1 -- 172.21.15.107:0/3754612999 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb6c015070 con 0x55e2382c5ea0 2024-04-25T00:22:28.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.732+0000 7fdb5dffb640 1 -- 172.21.15.107:0/3754612999 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c004e40 con 0x55e2382c5ea0 2024-04-25T00:22:28.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.732+0000 7fdb80c19740 1 -- 172.21.15.107:0/3754612999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382c5ea0 msgr2=0x55e2382d0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.732+0000 7fdb80c19740 1 --2- 172.21.15.107:0/3754612999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382c5ea0 0x55e2382d0030 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c00b710 tx=0x7fdb6c004610 comp rx=0 tx=0).stop 2024-04-25T00:22:28.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.732+0000 7fdb80c19740 1 -- 172.21.15.107:0/3754612999 shutdown_connections 2024-04-25T00:22:28.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.732+0000 7fdb80c19740 1 --2- 172.21.15.107:0/3754612999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2382d0570 0x55e2382d2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.732+0000 7fdb80c19740 1 --2- 172.21.15.107:0/3754612999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382c5ea0 0x55e2382d0030 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.732+0000 7fdb80c19740 1 --2- 172.21.15.107:0/3754612999 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2382c5580 0x55e2382c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.732+0000 7fdb80c19740 1 -- 172.21.15.107:0/3754612999 >> 172.21.15.107:0/3754612999 conn(0x55e2382c18a0 msgr2=0x55e2382c1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.732+0000 7fdb80c19740 1 -- 172.21.15.107:0/3754612999 shutdown_connections 2024-04-25T00:22:28.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.732+0000 7fdb80c19740 1 -- 172.21.15.107:0/3754612999 wait complete. 2024-04-25T00:22:28.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.733+0000 7fdb80c19740 1 -- start start 2024-04-25T00:22:28.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.733+0000 7fdb80c19740 1 Processor -- start 2024-04-25T00:22:28.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.733+0000 7fdb80c19740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382c5580 0x55e2382cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.733+0000 7fdb80c19740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2382c5ea0 0x55e2382cd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.733+0000 7fdb80c19740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2382d0570 0x55e2382cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.733+0000 7fdb80c19740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e2382d3360 con 0x55e2382c5580 2024-04-25T00:22:28.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.733+0000 7fdb80c19740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e2381e6050 con 0x55e2382d0570 2024-04-25T00:22:28.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.733+0000 7fdb80c19740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e2382d34e0 con 0x55e2382c5ea0 2024-04-25T00:22:28.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.733+0000 7fdb80c19740 10 client.0 _ll_register_callbacks cb 0x55e237f3dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:28.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.733+0000 7fdb777fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382c5580 0x55e2382cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.733+0000 7fdb777fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382c5580 0x55e2382cfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49974/0 (socket says 172.21.15.107:49974) 2024-04-25T00:22:28.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.733+0000 7fdb777fe640 1 -- 172.21.15.107:0/1583489617 learned_addr learned my addr 172.21.15.107:0/1583489617 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:28.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.733+0000 7fdb76ffd640 1 --2- 172.21.15.107:0/1583489617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2382c5ea0 0x55e2382cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.733+0000 7fdb77fff640 1 --2- 172.21.15.107:0/1583489617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2382d0570 0x55e2382cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.734+0000 7fdb777fe640 1 -- 172.21.15.107:0/1583489617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2382c5ea0 msgr2=0x55e2382cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.734+0000 7fdb777fe640 1 --2- 172.21.15.107:0/1583489617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2382c5ea0 0x55e2382cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.734+0000 7fdb777fe640 1 -- 172.21.15.107:0/1583489617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2382d0570 msgr2=0x55e2382cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.734+0000 7fdb777fe640 1 --2- 172.21.15.107:0/1583489617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2382d0570 0x55e2382cd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.734+0000 7fdb777fe640 1 -- 172.21.15.107:0/1583489617 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2381b7dc0 con 0x55e2382c5580 2024-04-25T00:22:28.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.734+0000 7fdb777fe640 1 --2- 172.21.15.107:0/1583489617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382c5580 0x55e2382cfea0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c0039c0 tx=0x7fdb6c004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d39741d3907bd31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.734+0000 7fdb56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:22:28.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.734+0000 7fdb56ffd640 1 -- 172.21.15.107:0/1583489617 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c028060 con 0x55e2382c5580 2024-04-25T00:22:28.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.734+0000 7fdb56ffd640 1 -- 172.21.15.107:0/1583489617 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fdb6c011640 con 0x55e2382c5580 2024-04-25T00:22:28.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.734+0000 7fdb80c19740 1 -- 172.21.15.107:0/1583489617 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb6c015070 con 0x55e2382c5580 2024-04-25T00:22:28.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.734+0000 7fdb56ffd640 1 -- 172.21.15.107:0/1583489617 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdb6c02f050 con 0x55e2382c5580 2024-04-25T00:22:28.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.734+0000 7fdb80c19740 10 client.16914 fetch_fsmap learned FSMap version 34 2024-04-25T00:22:28.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.734+0000 7fdb80c19740 1 -- 172.21.15.107:0/1583489617 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e237f3e3f0 con 0x55e2382c5580 2024-04-25T00:22:28.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.734+0000 7fdb56ffd640 1 -- 172.21.15.107:0/1583489617 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb6c015070 con 0x55e2382c5580 2024-04-25T00:22:28.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.734+0000 7fdb56ffd640 1 -- 172.21.15.107:0/1583489617 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fdb6c022070 con 0x55e2382c5580 2024-04-25T00:22:28.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.734+0000 7fdb80c19740 10 client.16914 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:22:28.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.734+0000 7fdb80c19740 10 client.16914 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:22:28.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.734+0000 7fdb80c19740 20 client.16914 populate_metadata read hostname 'smithi107' 2024-04-25T00:22:28.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.734+0000 7fdb80c19740 1 --2- 172.21.15.107:0/1583489617 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55e2382e05c0 0x55e2382e29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.734+0000 7fdb80c19740 4 client.16914 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:22:28.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.734+0000 7fdb80c19740 1 -- 172.21.15.107:0/1583489617 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e238237bb0 con 0x55e2382e05c0 2024-04-25T00:22:28.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.735+0000 7fdb76ffd640 1 --2- 172.21.15.107:0/1583489617 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55e2382e05c0 0x55e2382e29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.735+0000 7fdb56ffd640 10 client.16914 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:22:28.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.735+0000 7fdb76ffd640 1 --2- 172.21.15.107:0/1583489617 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55e2382e05c0 0x55e2382e29b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ba6e31c747564697 server_cookie=5ce10a02812d9217 in_seq=0 out_seq=0 2024-04-25T00:22:28.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.736+0000 7fdb56ffd640 1 -- 172.21.15.107:0/1583489617 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e238237bb0 con 0x55e2382e05c0 2024-04-25T00:22:28.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.736+0000 7fdb56ffd640 10 client.16914 handle_command_reply: tid=0 2024-04-25T00:22:28.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.736+0000 7fdb80c19740 1 client.16914 shutdown 2024-04-25T00:22:28.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.736+0000 7fdb80c19740 1 -- 172.21.15.107:0/1583489617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382c5580 msgr2=0x55e2382cfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.736+0000 7fdb80c19740 1 --2- 172.21.15.107:0/1583489617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382c5580 0x55e2382cfea0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c0039c0 tx=0x7fdb6c004800 comp rx=0 tx=0).stop 2024-04-25T00:22:28.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.736+0000 7fdb80c19740 1 -- 172.21.15.107:0/1583489617 shutdown_connections 2024-04-25T00:22:28.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.736+0000 7fdb80c19740 1 --2- 172.21.15.107:0/1583489617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2382d0570 0x55e2382cd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.736+0000 7fdb80c19740 1 --2- 172.21.15.107:0/1583489617 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2382c5ea0 0x55e2382cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.736+0000 7fdb80c19740 1 --2- 172.21.15.107:0/1583489617 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55e2382e05c0 0x55e2382e29b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.736+0000 7fdb80c19740 1 --2- 172.21.15.107:0/1583489617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382c5580 0x55e2382cfea0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.736+0000 7fdb56ffd640 0 client.16914 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:22:28.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.736+0000 7fdb80c19740 1 -- 172.21.15.107:0/1583489617 >> 172.21.15.107:0/1583489617 conn(0x55e2382c18a0 msgr2=0x55e2382c0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.736+0000 7fdb80c19740 1 -- 172.21.15.107:0/1583489617 shutdown_connections 2024-04-25T00:22:28.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.736+0000 7fdb80c19740 1 -- 172.21.15.107:0/1583489617 wait complete. 2024-04-25T00:22:28.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.736+0000 7fdb80c19740 20 client.16914 trim_cache size 0 max 16384 2024-04-25T00:22:28.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.749+0000 7fdb80c19740 1 Processor -- start 2024-04-25T00:22:28.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.749+0000 7fdb80c19740 1 -- start start 2024-04-25T00:22:28.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.749+0000 7fdb80c19740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382c5580 0x55e2382dc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.750+0000 7fdb80c19740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2382c5ea0 0x55e2382cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.750+0000 7fdb80c19740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2382d0570 0x55e2382e7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.750+0000 7fdb80c19740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb781b1320 con 0x55e2382c5580 2024-04-25T00:22:28.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.750+0000 7fdb80c19740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb781aebf0 con 0x55e2382d0570 2024-04-25T00:22:28.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.750+0000 7fdb80c19740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdb781aca40 con 0x55e2382c5ea0 2024-04-25T00:22:28.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.750+0000 7fdb777fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382c5580 0x55e2382dc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.750+0000 7fdb777fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382c5580 0x55e2382dc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49990/0 (socket says 172.21.15.107:49990) 2024-04-25T00:22:28.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.750+0000 7fdb777fe640 1 -- 172.21.15.107:0/3724948093 learned_addr learned my addr 172.21.15.107:0/3724948093 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:28.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.750+0000 7fdb76ffd640 1 --2- 172.21.15.107:0/3724948093 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2382c5ea0 0x55e2382cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.750+0000 7fdb777fe640 1 -- 172.21.15.107:0/3724948093 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2382c5ea0 msgr2=0x55e2382cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.750+0000 7fdb777fe640 1 --2- 172.21.15.107:0/3724948093 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2382c5ea0 0x55e2382cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.750+0000 7fdb777fe640 1 -- 172.21.15.107:0/3724948093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2382d0570 msgr2=0x55e2382e7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.750+0000 7fdb77fff640 1 --2- 172.21.15.107:0/3724948093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2382d0570 0x55e2382e7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.750+0000 7fdb777fe640 1 --2- 172.21.15.107:0/3724948093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2382d0570 0x55e2382e7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.750+0000 7fdb777fe640 1 -- 172.21.15.107:0/3724948093 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2380087d0 con 0x55e2382c5580 2024-04-25T00:22:28.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.750+0000 7fdb77fff640 1 --2- 172.21.15.107:0/3724948093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2382d0570 0x55e2382e7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:28.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.750+0000 7fdb777fe640 1 --2- 172.21.15.107:0/3724948093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382c5580 0x55e2382dc250 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c0198c0 tx=0x7fdb6c004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef67d24754a126e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.750+0000 7fdb5d7fa640 1 -- 172.21.15.107:0/3724948093 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c038070 con 0x55e2382c5580 2024-04-25T00:22:28.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.750+0000 7fdb5d7fa640 1 -- 172.21.15.107:0/3724948093 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb6c015070 con 0x55e2382c5580 2024-04-25T00:22:28.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.750+0000 7fdb5d7fa640 1 -- 172.21.15.107:0/3724948093 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6c028050 con 0x55e2382c5580 2024-04-25T00:22:28.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.751+0000 7fdb80c19740 1 -- 172.21.15.107:0/3724948093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382c5580 msgr2=0x55e2382dc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.751+0000 7fdb80c19740 1 --2- 172.21.15.107:0/3724948093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382c5580 0x55e2382dc250 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c0198c0 tx=0x7fdb6c004970 comp rx=0 tx=0).stop 2024-04-25T00:22:28.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.751+0000 7fdb80c19740 1 -- 172.21.15.107:0/3724948093 shutdown_connections 2024-04-25T00:22:28.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.751+0000 7fdb80c19740 1 --2- 172.21.15.107:0/3724948093 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2382d0570 0x55e2382e7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.751+0000 7fdb80c19740 1 --2- 172.21.15.107:0/3724948093 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2382c5ea0 0x55e2382cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.751+0000 7fdb80c19740 1 --2- 172.21.15.107:0/3724948093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382c5580 0x55e2382dc250 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.751+0000 7fdb80c19740 1 -- 172.21.15.107:0/3724948093 >> 172.21.15.107:0/3724948093 conn(0x55e2382c18a0 msgr2=0x55e2382c75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.751+0000 7fdb80c19740 1 -- 172.21.15.107:0/3724948093 shutdown_connections 2024-04-25T00:22:28.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.751+0000 7fdb80c19740 1 -- 172.21.15.107:0/3724948093 wait complete. 2024-04-25T00:22:28.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.751+0000 7fdb80c19740 1 -- start start 2024-04-25T00:22:28.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.751+0000 7fdb80c19740 1 Processor -- start 2024-04-25T00:22:28.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb80c19740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2382c5580 0x55e2382d89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb80c19740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2382c5ea0 0x55e2382c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb80c19740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382d0570 0x55e2382c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb80c19740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb781ad0b0 con 0x55e2382d0570 2024-04-25T00:22:28.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb80c19740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb781ae520 con 0x55e2382c5ea0 2024-04-25T00:22:28.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb80c19740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdb781ad780 con 0x55e2382c5580 2024-04-25T00:22:28.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb80c19740 10 client.0 _ll_register_callbacks cb 0x7fdb6c004510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:28.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb77fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382d0570 0x55e2382c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb77fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382d0570 0x55e2382c9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50006/0 (socket says 172.21.15.107:50006) 2024-04-25T00:22:28.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb77fff640 1 -- 172.21.15.107:0/1416010024 learned_addr learned my addr 172.21.15.107:0/1416010024 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:28.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb76ffd640 1 --2- 172.21.15.107:0/1416010024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2382c5ea0 0x55e2382c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb777fe640 1 --2- 172.21.15.107:0/1416010024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2382c5580 0x55e2382d89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb77fff640 1 -- 172.21.15.107:0/1416010024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2382c5580 msgr2=0x55e2382d89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb77fff640 1 --2- 172.21.15.107:0/1416010024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2382c5580 0x55e2382d89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb77fff640 1 -- 172.21.15.107:0/1416010024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2382c5ea0 msgr2=0x55e2382c99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb77fff640 1 --2- 172.21.15.107:0/1416010024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2382c5ea0 0x55e2382c99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb77fff640 1 -- 172.21.15.107:0/1416010024 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2382ca540 con 0x55e2382d0570 2024-04-25T00:22:28.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb77fff640 1 --2- 172.21.15.107:0/1416010024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382d0570 0x55e2382c9f00 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009ca0 tx=0x7fdb700095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d05d93f0201a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:22:28.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb56ffd640 1 -- 172.21.15.107:0/1416010024 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7002f610 con 0x55e2382d0570 2024-04-25T00:22:28.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb56ffd640 1 -- 172.21.15.107:0/1416010024 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fdb7002f7b0 con 0x55e2382d0570 2024-04-25T00:22:28.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb80c19740 1 -- 172.21.15.107:0/1416010024 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb6c015070 con 0x55e2382d0570 2024-04-25T00:22:28.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb56ffd640 1 -- 172.21.15.107:0/1416010024 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdb7002f9b0 con 0x55e2382d0570 2024-04-25T00:22:28.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb80c19740 10 client.16920 fetch_fsmap learned FSMap version 34 2024-04-25T00:22:28.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.752+0000 7fdb80c19740 1 -- 172.21.15.107:0/1416010024 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e2383465b0 con 0x55e2382d0570 2024-04-25T00:22:28.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.753+0000 7fdb56ffd640 1 -- 172.21.15.107:0/1416010024 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb6c015070 con 0x55e2382d0570 2024-04-25T00:22:28.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.753+0000 7fdb56ffd640 1 -- 172.21.15.107:0/1416010024 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fdb70003e90 con 0x55e2382d0570 2024-04-25T00:22:28.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.753+0000 7fdb80c19740 10 client.16920 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:22:28.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.753+0000 7fdb80c19740 10 client.16920 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:22:28.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.753+0000 7fdb80c19740 20 client.16920 populate_metadata read hostname 'smithi107' 2024-04-25T00:22:28.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.753+0000 7fdb80c19740 1 --2- 172.21.15.107:0/1416010024 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55e238346d60 0x55e238349180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.753+0000 7fdb80c19740 4 client.16920 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:22:28.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.753+0000 7fdb80c19740 1 -- 172.21.15.107:0/1416010024 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e238231f00 con 0x55e238346d60 2024-04-25T00:22:28.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.753+0000 7fdb777fe640 1 --2- 172.21.15.107:0/1416010024 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55e238346d60 0x55e238349180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.753+0000 7fdb56ffd640 10 client.16920 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:22:28.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.753+0000 7fdb777fe640 1 --2- 172.21.15.107:0/1416010024 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55e238346d60 0x55e238349180 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ece8120f3c58761 server_cookie=8ce21277e8cf1abe in_seq=0 out_seq=0 2024-04-25T00:22:28.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb56ffd640 1 -- 172.21.15.107:0/1416010024 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e238231f00 con 0x55e238346d60 2024-04-25T00:22:28.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb56ffd640 10 client.16920 handle_command_reply: tid=0 2024-04-25T00:22:28.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb80c19740 1 client.16920 shutdown 2024-04-25T00:22:28.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb80c19740 1 -- 172.21.15.107:0/1416010024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382d0570 msgr2=0x55e2382c9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb80c19740 1 --2- 172.21.15.107:0/1416010024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382d0570 0x55e2382c9f00 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009ca0 tx=0x7fdb700095c0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb80c19740 1 -- 172.21.15.107:0/1416010024 shutdown_connections 2024-04-25T00:22:28.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb80c19740 1 --2- 172.21.15.107:0/1416010024 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55e238346d60 0x55e238349180 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb80c19740 1 --2- 172.21.15.107:0/1416010024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2382d0570 0x55e2382c9f00 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb80c19740 1 --2- 172.21.15.107:0/1416010024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e2382c5ea0 0x55e2382c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb56ffd640 0 client.16920 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:22:28.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb80c19740 1 --2- 172.21.15.107:0/1416010024 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e2382c5580 0x55e2382d89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb80c19740 1 -- 172.21.15.107:0/1416010024 >> 172.21.15.107:0/1416010024 conn(0x55e2382c18a0 msgr2=0x55e2382c02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb80c19740 1 -- 172.21.15.107:0/1416010024 shutdown_connections 2024-04-25T00:22:28.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb80c19740 1 -- 172.21.15.107:0/1416010024 wait complete. 2024-04-25T00:22:28.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb80c19740 20 client.16920 trim_cache size 0 max 16384 2024-04-25T00:22:28.763 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:22:28.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb7df16640 1 -- 172.21.15.107:0/3030809071 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdb600a2d00 msgr2=0x7fdb600a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb7df16640 1 --2- 172.21.15.107:0/3030809071 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdb600a2d00 0x7fdb600a51c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdb70004af0 tx=0x7fdb7002f310 comp rx=0 tx=0).stop 2024-04-25T00:22:28.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb7df16640 1 -- 172.21.15.107:0/3030809071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb78101330 msgr2=0x7fdb781992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb7df16640 1 --2- 172.21.15.107:0/3030809071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb78101330 0x7fdb781992c0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fdb6800d950 tx=0x7fdb6800de10 comp rx=0 tx=0).stop 2024-04-25T00:22:28.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb7df16640 1 -- 172.21.15.107:0/3030809071 shutdown_connections 2024-04-25T00:22:28.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb7df16640 1 --2- 172.21.15.107:0/3030809071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb78106690 0x7fdb781a3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb7df16640 1 --2- 172.21.15.107:0/3030809071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb78101330 0x7fdb781992c0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb7df16640 1 --2- 172.21.15.107:0/3030809071 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdb600a2d00 0x7fdb600a51c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb7df16640 1 --2- 172.21.15.107:0/3030809071 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb78100990 0x7fdb78198d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.754+0000 7fdb7df16640 1 -- 172.21.15.107:0/3030809071 >> 172.21.15.107:0/3030809071 conn(0x7fdb78076440 msgr2=0x7fdb780ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.755+0000 7fdb7df16640 1 -- 172.21.15.107:0/3030809071 shutdown_connections 2024-04-25T00:22:28.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.755+0000 7fdb7df16640 1 -- 172.21.15.107:0/3030809071 wait complete. 2024-04-25T00:22:28.765 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:28 smithi107 ceph-mon[29153]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:22:28.765 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:28 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/363883986' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:22:28.767 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:22:28.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.865+0000 7f3a0bf63640 1 -- 172.21.15.107:0/1223003635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a04101710 msgr2=0x7f3a04101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.865+0000 7f3a0bf63640 1 --2- 172.21.15.107:0/1223003635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a04101710 0x7f3a04101b90 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009930 tx=0x7f39f402a420 comp rx=0 tx=0).stop 2024-04-25T00:22:28.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.865+0000 7f3a0bf63640 1 -- 172.21.15.107:0/1223003635 shutdown_connections 2024-04-25T00:22:28.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.865+0000 7f3a0bf63640 1 --2- 172.21.15.107:0/1223003635 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a041020d0 0x7f3a0410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.865+0000 7f3a0bf63640 1 --2- 172.21.15.107:0/1223003635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a04101710 0x7f3a04101b90 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.865+0000 7f3a0bf63640 1 --2- 172.21.15.107:0/1223003635 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a04107720 0x7f3a04107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.865+0000 7f3a0bf63640 1 -- 172.21.15.107:0/1223003635 >> 172.21.15.107:0/1223003635 conn(0x7f3a04078b90 msgr2=0x7f3a04078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.865+0000 7f3a0bf63640 1 -- 172.21.15.107:0/1223003635 shutdown_connections 2024-04-25T00:22:28.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.865+0000 7f3a0bf63640 1 -- 172.21.15.107:0/1223003635 wait complete. 2024-04-25T00:22:28.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.866+0000 7f3a0bf63640 1 Processor -- start 2024-04-25T00:22:28.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.866+0000 7f3a0bf63640 1 -- start start 2024-04-25T00:22:28.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.866+0000 7f3a0bf63640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a04101710 0x7f3a04197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.866+0000 7f3a0bf63640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a041020d0 0x7f3a04198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.866+0000 7f3a0bf63640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a04107720 0x7f3a0419c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.866+0000 7f3a0bf63640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3a04111b30 con 0x7f3a041020d0 2024-04-25T00:22:28.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.866+0000 7f3a0bf63640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3a041119b0 con 0x7f3a04101710 2024-04-25T00:22:28.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.866+0000 7f3a0bf63640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3a04111cb0 con 0x7f3a04107720 2024-04-25T00:22:28.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.866+0000 7f3a094fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a041020d0 0x7f3a04198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.866+0000 7f3a094fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a041020d0 0x7f3a04198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50026/0 (socket says 172.21.15.107:50026) 2024-04-25T00:22:28.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.866+0000 7f3a094fe640 1 -- 172.21.15.107:0/1929420598 learned_addr learned my addr 172.21.15.107:0/1929420598 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:28.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.866+0000 7f3a0a500640 1 --2- 172.21.15.107:0/1929420598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a04107720 0x7f3a0419c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.866+0000 7f3a094fe640 1 -- 172.21.15.107:0/1929420598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a04107720 msgr2=0x7f3a0419c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.866+0000 7f3a094fe640 1 --2- 172.21.15.107:0/1929420598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a04107720 0x7f3a0419c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.866+0000 7f3a094fe640 1 -- 172.21.15.107:0/1929420598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a04101710 msgr2=0x7f3a04197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.866+0000 7f3a094fe640 1 --2- 172.21.15.107:0/1929420598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a04101710 0x7f3a04197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.866+0000 7f3a094fe640 1 -- 172.21.15.107:0/1929420598 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a0419cfe0 con 0x7f3a041020d0 2024-04-25T00:22:28.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.867+0000 7f3a094fe640 1 --2- 172.21.15.107:0/1929420598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a041020d0 0x7f3a04198530 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009230 tx=0x7f39f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.867+0000 7f39faffd640 1 -- 172.21.15.107:0/1929420598 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f403f030 con 0x7f3a041020d0 2024-04-25T00:22:28.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.867+0000 7f39faffd640 1 -- 172.21.15.107:0/1929420598 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39f4003ad0 con 0x7f3a041020d0 2024-04-25T00:22:28.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.867+0000 7f3a0bf63640 1 -- 172.21.15.107:0/1929420598 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a0419d270 con 0x7f3a041020d0 2024-04-25T00:22:28.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.867+0000 7f39faffd640 1 -- 172.21.15.107:0/1929420598 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f4035070 con 0x7f3a041020d0 2024-04-25T00:22:28.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.867+0000 7f3a0bf63640 1 -- 172.21.15.107:0/1929420598 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a041a4b60 con 0x7f3a041020d0 2024-04-25T00:22:28.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.869+0000 7f39faffd640 1 -- 172.21.15.107:0/1929420598 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f39f4042030 con 0x7f3a041020d0 2024-04-25T00:22:28.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.869+0000 7f3a0ec77740 1 Processor -- start 2024-04-25T00:22:28.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.870+0000 7f3a0ec77740 1 -- start start 2024-04-25T00:22:28.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.870+0000 7f3a0ec77740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aefb6cc580 0x55aefb6cc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.870+0000 7f3a0ec77740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aefb6ccea0 0x55aefb6d7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.870+0000 7f3a0ec77740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6d7570 0x55aefb6d9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.870+0000 7f3a0ec77740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55aefb631670 con 0x55aefb6d7570 2024-04-25T00:22:28.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.870+0000 7f3a0ec77740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55aefb5d9970 con 0x55aefb6cc580 2024-04-25T00:22:28.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.870+0000 7f3a0ec77740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55aefb62d630 con 0x55aefb6ccea0 2024-04-25T00:22:28.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.870+0000 7f3a094fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6d7570 0x55aefb6d9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.870+0000 7f3a094fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6d7570 0x55aefb6d9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50030/0 (socket says 172.21.15.107:50030) 2024-04-25T00:22:28.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.870+0000 7f39faffd640 1 --2- 172.21.15.107:0/1929420598 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f39f00a2d00 0x7f39f00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.870+0000 7f3a094fe640 1 -- 172.21.15.107:0/1202947272 learned_addr learned my addr 172.21.15.107:0/1202947272 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:28.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.870+0000 7f3a0a500640 1 --2- 172.21.15.107:0/1202947272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aefb6cc580 0x55aefb6cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.870+0000 7f3a09cff640 1 --2- 172.21.15.107:0/1202947272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aefb6ccea0 0x55aefb6d7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.870+0000 7f3a0a500640 1 --2- 172.21.15.107:0/1929420598 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f39f00a2d00 0x7f39f00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.870+0000 7f39faffd640 1 -- 172.21.15.107:0/1929420598 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f39f40eff10 con 0x7f3a041020d0 2024-04-25T00:22:28.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.870+0000 7f3a094fe640 1 -- 172.21.15.107:0/1202947272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aefb6ccea0 msgr2=0x55aefb6d7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.870+0000 7f3a094fe640 1 --2- 172.21.15.107:0/1202947272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aefb6ccea0 0x55aefb6d7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.870+0000 7f3a094fe640 1 -- 172.21.15.107:0/1202947272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aefb6cc580 msgr2=0x55aefb6cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.870+0000 7f3a094fe640 1 --2- 172.21.15.107:0/1202947272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aefb6cc580 0x55aefb6cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.870+0000 7f3a094fe640 1 -- 172.21.15.107:0/1202947272 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aefb5fd400 con 0x55aefb6d7570 2024-04-25T00:22:28.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.870+0000 7f3a09cff640 1 --2- 172.21.15.107:0/1202947272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aefb6ccea0 0x55aefb6d7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:28.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.870+0000 7f3a0a500640 1 --2- 172.21.15.107:0/1202947272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aefb6cc580 0x55aefb6cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:28.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.871+0000 7f3a094fe640 1 --2- 172.21.15.107:0/1202947272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6d7570 0x55aefb6d9960 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f39f40bbd60 tx=0x7f39f4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51ffe99ba8b2d7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.871+0000 7f3a0a500640 1 --2- 172.21.15.107:0/1929420598 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f39f00a2d00 0x7f39f00a51c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f39fc005c40 tx=0x7f39fc009500 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.871+0000 7f39effff640 1 -- 172.21.15.107:0/1202947272 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f4049070 con 0x55aefb6d7570 2024-04-25T00:22:28.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.871+0000 7f39effff640 1 -- 172.21.15.107:0/1202947272 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39f40bc760 con 0x55aefb6d7570 2024-04-25T00:22:28.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.871+0000 7f39effff640 1 -- 172.21.15.107:0/1202947272 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f40b86f0 con 0x55aefb6d7570 2024-04-25T00:22:28.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.871+0000 7f3a0ec77740 1 -- 172.21.15.107:0/1202947272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6d7570 msgr2=0x55aefb6d9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.871+0000 7f3a0ec77740 1 --2- 172.21.15.107:0/1202947272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6d7570 0x55aefb6d9960 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f39f40bbd60 tx=0x7f39f4009260 comp rx=0 tx=0).stop 2024-04-25T00:22:28.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.871+0000 7f3a0ec77740 1 -- 172.21.15.107:0/1202947272 shutdown_connections 2024-04-25T00:22:28.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.871+0000 7f3a0ec77740 1 --2- 172.21.15.107:0/1202947272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6d7570 0x55aefb6d9960 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.871+0000 7f3a0ec77740 1 --2- 172.21.15.107:0/1202947272 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aefb6ccea0 0x55aefb6d7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.871+0000 7f3a0ec77740 1 --2- 172.21.15.107:0/1202947272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aefb6cc580 0x55aefb6cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.871+0000 7f3a0ec77740 1 -- 172.21.15.107:0/1202947272 >> 172.21.15.107:0/1202947272 conn(0x55aefb6c88a0 msgr2=0x55aefb6c8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.872+0000 7f3a0ec77740 1 -- 172.21.15.107:0/1202947272 shutdown_connections 2024-04-25T00:22:28.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.872+0000 7f3a0ec77740 1 -- 172.21.15.107:0/1202947272 wait complete. 2024-04-25T00:22:28.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.872+0000 7f3a0ec77740 1 -- start start 2024-04-25T00:22:28.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.872+0000 7f3a0ec77740 1 Processor -- start 2024-04-25T00:22:28.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.872+0000 7f3a0ec77740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aefb6cc580 0x55aefb6d6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.872+0000 7f3a0ec77740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6ccea0 0x55aefb6d41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.872+0000 7f3a0ec77740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aefb6d7570 0x55aefb6d4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.872+0000 7f3a0ec77740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55aefb631670 con 0x55aefb6ccea0 2024-04-25T00:22:28.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.872+0000 7f3a0ec77740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55aefb5ed050 con 0x55aefb6d7570 2024-04-25T00:22:28.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.872+0000 7f3a0ec77740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55aefb6da120 con 0x55aefb6cc580 2024-04-25T00:22:28.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.872+0000 7f3a0ec77740 10 client.0 _ll_register_callbacks cb 0x55aefb344be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:28.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.872+0000 7f3a094fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6ccea0 0x55aefb6d41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.872+0000 7f3a094fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6ccea0 0x55aefb6d41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50034/0 (socket says 172.21.15.107:50034) 2024-04-25T00:22:28.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.872+0000 7f3a094fe640 1 -- 172.21.15.107:0/131082761 learned_addr learned my addr 172.21.15.107:0/131082761 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:28.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.873+0000 7f3a0a500640 1 --2- 172.21.15.107:0/131082761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aefb6d7570 0x55aefb6d4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.873+0000 7f3a09cff640 1 --2- 172.21.15.107:0/131082761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aefb6cc580 0x55aefb6d6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.873+0000 7f3a094fe640 1 -- 172.21.15.107:0/131082761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aefb6cc580 msgr2=0x55aefb6d6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.873+0000 7f3a094fe640 1 --2- 172.21.15.107:0/131082761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aefb6cc580 0x55aefb6d6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.873+0000 7f3a094fe640 1 -- 172.21.15.107:0/131082761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aefb6d7570 msgr2=0x55aefb6d4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.873+0000 7f3a094fe640 1 --2- 172.21.15.107:0/131082761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aefb6d7570 0x55aefb6d4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.873+0000 7f3a094fe640 1 -- 172.21.15.107:0/131082761 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aefb5bedc0 con 0x55aefb6ccea0 2024-04-25T00:22:28.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.873+0000 7f3a09cff640 1 --2- 172.21.15.107:0/131082761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aefb6cc580 0x55aefb6d6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:28.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.873+0000 7f3a094fe640 1 --2- 172.21.15.107:0/131082761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6ccea0 0x55aefb6d41f0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f39f40dc0c0 tx=0x7f39f403bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72e2a30f2f3046f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.873+0000 7f39ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:22:28.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.873+0000 7f39ed7fa640 1 -- 172.21.15.107:0/131082761 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f4049070 con 0x55aefb6ccea0 2024-04-25T00:22:28.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.873+0000 7f39ed7fa640 1 -- 172.21.15.107:0/131082761 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f39f4003c80 con 0x55aefb6ccea0 2024-04-25T00:22:28.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.873+0000 7f3a0ec77740 1 -- 172.21.15.107:0/131082761 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39f40bc760 con 0x55aefb6ccea0 2024-04-25T00:22:28.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.873+0000 7f39ed7fa640 1 -- 172.21.15.107:0/131082761 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f39f40ba100 con 0x55aefb6ccea0 2024-04-25T00:22:28.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.873+0000 7f3a0ec77740 10 client.16932 fetch_fsmap learned FSMap version 34 2024-04-25T00:22:28.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.873+0000 7f3a0ec77740 1 -- 172.21.15.107:0/131082761 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55aefb3453f0 con 0x55aefb6ccea0 2024-04-25T00:22:28.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.873+0000 7f39ed7fa640 1 -- 172.21.15.107:0/131082761 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39f40bc760 con 0x55aefb6ccea0 2024-04-25T00:22:28.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.873+0000 7f39ed7fa640 1 -- 172.21.15.107:0/131082761 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f39f40b7db0 con 0x55aefb6ccea0 2024-04-25T00:22:28.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.873+0000 7f3a0ec77740 10 client.16932 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:22:28.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.873+0000 7f3a0ec77740 10 client.16932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:28.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.874+0000 7f3a0ec77740 20 client.16932 populate_metadata read hostname 'smithi107' 2024-04-25T00:22:28.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.874+0000 7f3a0ec77740 1 --2- 172.21.15.107:0/131082761 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55aefb6e70e0 0x55aefb6e94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.874+0000 7f3a0ec77740 4 client.16932 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:22:28.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.874+0000 7f3a0ec77740 1 -- 172.21.15.107:0/131082761 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aefb63ebb0 con 0x55aefb6e70e0 2024-04-25T00:22:28.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.874+0000 7f3a09cff640 1 --2- 172.21.15.107:0/131082761 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55aefb6e70e0 0x55aefb6e94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.874+0000 7f3a09cff640 1 --2- 172.21.15.107:0/131082761 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55aefb6e70e0 0x55aefb6e94d0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bffe4e65a1a3ad9d server_cookie=fd6d0c978efb4207 in_seq=0 out_seq=0 2024-04-25T00:22:28.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.874+0000 7f39ed7fa640 10 client.16932 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:22:28.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.875+0000 7f39ed7fa640 1 -- 172.21.15.107:0/131082761 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aefb63ebb0 con 0x55aefb6e70e0 2024-04-25T00:22:28.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.875+0000 7f39ed7fa640 10 client.16932 handle_command_reply: tid=0 2024-04-25T00:22:28.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.875+0000 7f3a0ec77740 1 client.16932 shutdown 2024-04-25T00:22:28.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.875+0000 7f3a0ec77740 1 -- 172.21.15.107:0/131082761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6ccea0 msgr2=0x55aefb6d41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.875+0000 7f3a0ec77740 1 --2- 172.21.15.107:0/131082761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6ccea0 0x55aefb6d41f0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f39f40dc0c0 tx=0x7f39f403bf90 comp rx=0 tx=0).stop 2024-04-25T00:22:28.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.875+0000 7f3a0ec77740 1 -- 172.21.15.107:0/131082761 shutdown_connections 2024-04-25T00:22:28.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.875+0000 7f3a0ec77740 1 --2- 172.21.15.107:0/131082761 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55aefb6e70e0 0x55aefb6e94d0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.875+0000 7f3a0ec77740 1 --2- 172.21.15.107:0/131082761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aefb6d7570 0x55aefb6d4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.875+0000 7f3a0ec77740 1 --2- 172.21.15.107:0/131082761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6ccea0 0x55aefb6d41f0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.875+0000 7f3a0ec77740 1 --2- 172.21.15.107:0/131082761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aefb6cc580 0x55aefb6d6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.875+0000 7f39ed7fa640 0 client.16932 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:22:28.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.875+0000 7f3a0ec77740 1 -- 172.21.15.107:0/131082761 >> 172.21.15.107:0/131082761 conn(0x55aefb6c88a0 msgr2=0x55aefb6c7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.875+0000 7f3a0ec77740 1 -- 172.21.15.107:0/131082761 shutdown_connections 2024-04-25T00:22:28.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.875+0000 7f3a0ec77740 1 -- 172.21.15.107:0/131082761 wait complete. 2024-04-25T00:22:28.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.875+0000 7f3a0ec77740 20 client.16932 trim_cache size 0 max 16384 2024-04-25T00:22:28.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.888+0000 7f3a0ec77740 1 Processor -- start 2024-04-25T00:22:28.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.888+0000 7f3a0ec77740 1 -- start start 2024-04-25T00:22:28.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.888+0000 7f3a0ec77740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6cc580 0x55aefb6e2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.888+0000 7f3a0ec77740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aefb6ccea0 0x55aefb6df780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.888+0000 7f3a0ec77740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aefb6d7570 0x55aefb6d6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.888+0000 7f3a0ec77740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3a041b13c0 con 0x55aefb6cc580 2024-04-25T00:22:28.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.888+0000 7f3a0ec77740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3a041aec90 con 0x55aefb6d7570 2024-04-25T00:22:28.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.888+0000 7f3a0ec77740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3a041acae0 con 0x55aefb6ccea0 2024-04-25T00:22:28.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.889+0000 7f3a09cff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6cc580 0x55aefb6e2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.889+0000 7f3a09cff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6cc580 0x55aefb6e2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50040/0 (socket says 172.21.15.107:50040) 2024-04-25T00:22:28.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.889+0000 7f3a09cff640 1 -- 172.21.15.107:0/4117933904 learned_addr learned my addr 172.21.15.107:0/4117933904 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:28.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.889+0000 7f3a09cff640 1 -- 172.21.15.107:0/4117933904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aefb6ccea0 msgr2=0x55aefb6df780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.889+0000 7f3a094fe640 1 --2- 172.21.15.107:0/4117933904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aefb6ccea0 0x55aefb6df780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.889+0000 7f3a09cff640 1 --2- 172.21.15.107:0/4117933904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aefb6ccea0 0x55aefb6df780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.889+0000 7f3a09cff640 1 -- 172.21.15.107:0/4117933904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aefb6d7570 msgr2=0x55aefb6d6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.889+0000 7f3a0a500640 1 --2- 172.21.15.107:0/4117933904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aefb6d7570 0x55aefb6d6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.889+0000 7f3a09cff640 1 --2- 172.21.15.107:0/4117933904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aefb6d7570 0x55aefb6d6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.889+0000 7f3a09cff640 1 -- 172.21.15.107:0/4117933904 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aefb6cccd0 con 0x55aefb6cc580 2024-04-25T00:22:28.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.889+0000 7f3a094fe640 1 --2- 172.21.15.107:0/4117933904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aefb6ccea0 0x55aefb6df780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:28.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.889+0000 7f3a09cff640 1 --2- 172.21.15.107:0/4117933904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6cc580 0x55aefb6e2e90 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x55aefb6e9c30 tx=0x7f3a00005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f970071e70ae76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.889+0000 7f39ef7fe640 1 -- 172.21.15.107:0/4117933904 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00004020 con 0x55aefb6cc580 2024-04-25T00:22:28.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.889+0000 7f39ef7fe640 1 -- 172.21.15.107:0/4117933904 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a000041c0 con 0x55aefb6cc580 2024-04-25T00:22:28.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.889+0000 7f39ef7fe640 1 -- 172.21.15.107:0/4117933904 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a000047a0 con 0x55aefb6cc580 2024-04-25T00:22:28.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.890+0000 7f3a0ec77740 1 -- 172.21.15.107:0/4117933904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6cc580 msgr2=0x55aefb6e2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.890+0000 7f3a0ec77740 1 --2- 172.21.15.107:0/4117933904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6cc580 0x55aefb6e2e90 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x55aefb6e9c30 tx=0x7f3a00005e90 comp rx=0 tx=0).stop 2024-04-25T00:22:28.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.890+0000 7f3a0ec77740 1 -- 172.21.15.107:0/4117933904 shutdown_connections 2024-04-25T00:22:28.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.890+0000 7f3a0ec77740 1 --2- 172.21.15.107:0/4117933904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aefb6d7570 0x55aefb6d6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.890+0000 7f3a0ec77740 1 --2- 172.21.15.107:0/4117933904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aefb6ccea0 0x55aefb6df780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.890+0000 7f3a0ec77740 1 --2- 172.21.15.107:0/4117933904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6cc580 0x55aefb6e2e90 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.890+0000 7f3a0ec77740 1 -- 172.21.15.107:0/4117933904 >> 172.21.15.107:0/4117933904 conn(0x55aefb6c88a0 msgr2=0x55aefb6cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.890+0000 7f3a0ec77740 1 -- 172.21.15.107:0/4117933904 shutdown_connections 2024-04-25T00:22:28.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.890+0000 7f3a0ec77740 1 -- 172.21.15.107:0/4117933904 wait complete. 2024-04-25T00:22:28.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.890+0000 7f3a0ec77740 1 -- start start 2024-04-25T00:22:28.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.890+0000 7f3a0ec77740 1 Processor -- start 2024-04-25T00:22:28.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f3a0ec77740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aefb6cc580 0x55aefb74a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f3a0ec77740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6ccea0 0x55aefb74acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f3a0ec77740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aefb6d7570 0x55aefb749500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f3a0ec77740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3a041ad150 con 0x55aefb6ccea0 2024-04-25T00:22:28.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f3a0ec77740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3a041ae5c0 con 0x55aefb6d7570 2024-04-25T00:22:28.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f3a0ec77740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3a041ad820 con 0x55aefb6cc580 2024-04-25T00:22:28.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f3a0ec77740 10 client.0 _ll_register_callbacks cb 0x7f39f40b7cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:28.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f3a094fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6ccea0 0x55aefb74acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f3a094fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6ccea0 0x55aefb74acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50044/0 (socket says 172.21.15.107:50044) 2024-04-25T00:22:28.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f3a094fe640 1 -- 172.21.15.107:0/3766080315 learned_addr learned my addr 172.21.15.107:0/3766080315 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:28.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f3a09cff640 1 --2- 172.21.15.107:0/3766080315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aefb6cc580 0x55aefb74a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f3a0a500640 1 --2- 172.21.15.107:0/3766080315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aefb6d7570 0x55aefb749500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f3a094fe640 1 -- 172.21.15.107:0/3766080315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aefb6cc580 msgr2=0x55aefb74a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f3a094fe640 1 --2- 172.21.15.107:0/3766080315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aefb6cc580 0x55aefb74a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f3a094fe640 1 -- 172.21.15.107:0/3766080315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aefb6d7570 msgr2=0x55aefb749500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:28.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f3a094fe640 1 --2- 172.21.15.107:0/3766080315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aefb6d7570 0x55aefb749500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f3a094fe640 1 -- 172.21.15.107:0/3766080315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aefb40f7d0 con 0x55aefb6ccea0 2024-04-25T00:22:28.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f3a09cff640 1 --2- 172.21.15.107:0/3766080315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aefb6cc580 0x55aefb74a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:28.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f3a0a500640 1 --2- 172.21.15.107:0/3766080315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aefb6d7570 0x55aefb749500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:28.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f3a094fe640 1 --2- 172.21.15.107:0/3766080315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6ccea0 0x55aefb74acb0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f39f40b8a30 tx=0x7f39f40bbc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e8de1756f9a119 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:28.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f39ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:22:28.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f39ed7fa640 1 -- 172.21.15.107:0/3766080315 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f4049070 con 0x55aefb6ccea0 2024-04-25T00:22:28.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f39ed7fa640 1 -- 172.21.15.107:0/3766080315 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f39f40b6bd0 con 0x55aefb6ccea0 2024-04-25T00:22:28.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f3a0ec77740 1 -- 172.21.15.107:0/3766080315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a000041c0 con 0x55aefb6ccea0 2024-04-25T00:22:28.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f39ed7fa640 1 -- 172.21.15.107:0/3766080315 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f39f40b8300 con 0x55aefb6ccea0 2024-04-25T00:22:28.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f3a0ec77740 10 client.16938 fetch_fsmap learned FSMap version 34 2024-04-25T00:22:28.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.891+0000 7f3a0ec77740 1 -- 172.21.15.107:0/3766080315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55aefb40f7d0 con 0x55aefb6ccea0 2024-04-25T00:22:28.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.892+0000 7f39ed7fa640 1 -- 172.21.15.107:0/3766080315 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a000041c0 con 0x55aefb6ccea0 2024-04-25T00:22:28.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.892+0000 7f39ed7fa640 1 -- 172.21.15.107:0/3766080315 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f39f40c4840 con 0x55aefb6ccea0 2024-04-25T00:22:28.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.892+0000 7f3a0ec77740 10 client.16938 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:22:28.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.892+0000 7f3a0ec77740 10 client.16938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:28.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.892+0000 7f3a0ec77740 20 client.16938 populate_metadata read hostname 'smithi107' 2024-04-25T00:22:28.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.892+0000 7f3a0ec77740 1 --2- 172.21.15.107:0/3766080315 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55aefb752120 0x55aefb754540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:28.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.892+0000 7f3a0ec77740 4 client.16938 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:22:28.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.892+0000 7f3a0ec77740 1 -- 172.21.15.107:0/3766080315 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aefb638f00 con 0x55aefb752120 2024-04-25T00:22:28.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.892+0000 7f3a09cff640 1 --2- 172.21.15.107:0/3766080315 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55aefb752120 0x55aefb754540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:28.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.892+0000 7f3a09cff640 1 --2- 172.21.15.107:0/3766080315 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55aefb752120 0x55aefb754540 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcfd9c136830edc7 server_cookie=69fd7516562fcf52 in_seq=0 out_seq=0 2024-04-25T00:22:28.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.892+0000 7f39ed7fa640 10 client.16938 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:22:28.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f39ed7fa640 1 -- 172.21.15.107:0/3766080315 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aefb638f00 con 0x55aefb752120 2024-04-25T00:22:28.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f39ed7fa640 10 client.16938 handle_command_reply: tid=0 2024-04-25T00:22:28.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f3a0ec77740 1 client.16938 shutdown 2024-04-25T00:22:28.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f3a0ec77740 1 -- 172.21.15.107:0/3766080315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6ccea0 msgr2=0x55aefb74acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f3a0ec77740 1 --2- 172.21.15.107:0/3766080315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6ccea0 0x55aefb74acb0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f39f40b8a30 tx=0x7f39f40bbc60 comp rx=0 tx=0).stop 2024-04-25T00:22:28.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f3a0ec77740 1 -- 172.21.15.107:0/3766080315 shutdown_connections 2024-04-25T00:22:28.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f3a0ec77740 1 --2- 172.21.15.107:0/3766080315 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55aefb752120 0x55aefb754540 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f3a0ec77740 1 --2- 172.21.15.107:0/3766080315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aefb6d7570 0x55aefb749500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f3a0ec77740 1 --2- 172.21.15.107:0/3766080315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55aefb6ccea0 0x55aefb74acb0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f39ed7fa640 0 client.16938 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:22:28.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f3a0ec77740 1 --2- 172.21.15.107:0/3766080315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55aefb6cc580 0x55aefb74a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f3a0ec77740 1 -- 172.21.15.107:0/3766080315 >> 172.21.15.107:0/3766080315 conn(0x55aefb6c88a0 msgr2=0x55aefb6e1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f3a0ec77740 1 -- 172.21.15.107:0/3766080315 shutdown_connections 2024-04-25T00:22:28.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f3a0ec77740 1 -- 172.21.15.107:0/3766080315 wait complete. 2024-04-25T00:22:28.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f3a0ec77740 20 client.16938 trim_cache size 0 max 16384 2024-04-25T00:22:28.911 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:22:28.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f3a0bf63640 1 -- 172.21.15.107:0/1929420598 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f39f00a2d00 msgr2=0x7f39f00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f3a0bf63640 1 --2- 172.21.15.107:0/1929420598 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f39f00a2d00 0x7f39f00a51c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f39fc005c40 tx=0x7f39fc009500 comp rx=0 tx=0).stop 2024-04-25T00:22:28.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f3a0bf63640 1 -- 172.21.15.107:0/1929420598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a041020d0 msgr2=0x7f3a04198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:28.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f3a0bf63640 1 --2- 172.21.15.107:0/1929420598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a041020d0 0x7f3a04198530 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009230 tx=0x7f39f4004270 comp rx=0 tx=0).stop 2024-04-25T00:22:28.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f3a0bf63640 1 -- 172.21.15.107:0/1929420598 shutdown_connections 2024-04-25T00:22:28.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f3a0bf63640 1 --2- 172.21.15.107:0/1929420598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a04107720 0x7f3a0419c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f3a0bf63640 1 --2- 172.21.15.107:0/1929420598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a041020d0 0x7f3a04198530 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f3a0bf63640 1 --2- 172.21.15.107:0/1929420598 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f39f00a2d00 0x7f39f00a51c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f3a0bf63640 1 --2- 172.21.15.107:0/1929420598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a04101710 0x7f3a04197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:28.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f3a0bf63640 1 -- 172.21.15.107:0/1929420598 >> 172.21.15.107:0/1929420598 conn(0x7f3a04078b90 msgr2=0x7f3a040ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:28.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f3a0bf63640 1 -- 172.21.15.107:0/1929420598 shutdown_connections 2024-04-25T00:22:28.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:28.893+0000 7f3a0bf63640 1 -- 172.21.15.107:0/1929420598 wait complete. 2024-04-25T00:22:28.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:28 smithi170 ceph-mon[32807]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:22:28.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:28 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/363883986' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:22:29.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:28 smithi195 ceph-mon[32833]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:22:29.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:28 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/363883986' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:22:29.917 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:22:29.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:22:29.917 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:22:29.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:29 smithi107 ceph-mon[29153]: pgmap v184: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:22:30.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.024+0000 7f28a85ce640 1 -- 172.21.15.107:0/2908978214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a006a280 msgr2=0x7f28a006a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:30.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.024+0000 7f28a85ce640 1 --2- 172.21.15.107:0/2908978214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a006a280 0x7f28a006a6e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f2890009900 tx=0x7f289002a410 comp rx=0 tx=0).stop 2024-04-25T00:22:30.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.024+0000 7f28a85ce640 1 -- 172.21.15.107:0/2908978214 shutdown_connections 2024-04-25T00:22:30.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.024+0000 7f28a85ce640 1 --2- 172.21.15.107:0/2908978214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28a010e000 0x7f28a01103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.024+0000 7f28a85ce640 1 --2- 172.21.15.107:0/2908978214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a006a280 0x7f28a006a6e0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.024+0000 7f28a85ce640 1 --2- 172.21.15.107:0/2908978214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28a0102610 0x7f28a0069d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.024+0000 7f28a85ce640 1 -- 172.21.15.107:0/2908978214 >> 172.21.15.107:0/2908978214 conn(0x7f28a0076440 msgr2=0x7f28a0076850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:30.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.024+0000 7f28a85ce640 1 -- 172.21.15.107:0/2908978214 shutdown_connections 2024-04-25T00:22:30.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.024+0000 7f28a85ce640 1 -- 172.21.15.107:0/2908978214 wait complete. 2024-04-25T00:22:30.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.024+0000 7f28a85ce640 1 Processor -- start 2024-04-25T00:22:30.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.025+0000 7f28a85ce640 1 -- start start 2024-04-25T00:22:30.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.025+0000 7f28a85ce640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28a006a280 0x7f28a019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.025+0000 7f28a85ce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a0102610 0x7f28a019a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.025+0000 7f28a85ce640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28a010e000 0x7f28a019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.025+0000 7f28a85ce640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f28a0113de0 con 0x7f28a0102610 2024-04-25T00:22:30.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.025+0000 7f28a85ce640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f28a0113c60 con 0x7f28a010e000 2024-04-25T00:22:30.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.025+0000 7f28a85ce640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f28a0113f60 con 0x7f28a006a280 2024-04-25T00:22:30.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.025+0000 7f28a5b69640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a0102610 0x7f28a019a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.025+0000 7f28a5b69640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a0102610 0x7f28a019a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52432/0 (socket says 172.21.15.107:52432) 2024-04-25T00:22:30.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.025+0000 7f28a5b69640 1 -- 172.21.15.107:0/3442329831 learned_addr learned my addr 172.21.15.107:0/3442329831 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:30.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.025+0000 7f28a6b6b640 1 --2- 172.21.15.107:0/3442329831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28a010e000 0x7f28a019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.025+0000 7f28a636a640 1 --2- 172.21.15.107:0/3442329831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28a006a280 0x7f28a019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.025+0000 7f28a5b69640 1 -- 172.21.15.107:0/3442329831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28a006a280 msgr2=0x7f28a019a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:30.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.025+0000 7f28a5b69640 1 --2- 172.21.15.107:0/3442329831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28a006a280 0x7f28a019a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.025+0000 7f28a5b69640 1 -- 172.21.15.107:0/3442329831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28a010e000 msgr2=0x7f28a019ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:30.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.025+0000 7f28a5b69640 1 --2- 172.21.15.107:0/3442329831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28a010e000 0x7f28a019ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.025+0000 7f28a5b69640 1 -- 172.21.15.107:0/3442329831 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a019f390 con 0x7f28a0102610 2024-04-25T00:22:30.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.025+0000 7f28a5b69640 1 --2- 172.21.15.107:0/3442329831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a0102610 0x7f28a019a8e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f2890005ab0 tx=0x7f28900042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:30.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.026+0000 7f28977fe640 1 -- 172.21.15.107:0/3442329831 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289003f030 con 0x7f28a0102610 2024-04-25T00:22:30.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.026+0000 7f28a85ce640 1 -- 172.21.15.107:0/3442329831 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28a019f620 con 0x7f28a0102610 2024-04-25T00:22:30.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.026+0000 7f28977fe640 1 -- 172.21.15.107:0/3442329831 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2890003ce0 con 0x7f28a0102610 2024-04-25T00:22:30.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.026+0000 7f28977fe640 1 -- 172.21.15.107:0/3442329831 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2890035070 con 0x7f28a0102610 2024-04-25T00:22:30.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.026+0000 7f28a85ce640 1 -- 172.21.15.107:0/3442329831 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28a01a6f10 con 0x7f28a0102610 2024-04-25T00:22:30.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.028+0000 7f28977fe640 1 -- 172.21.15.107:0/3442329831 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f2890042030 con 0x7f28a0102610 2024-04-25T00:22:30.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.028+0000 7f28ab3bc740 1 Processor -- start 2024-04-25T00:22:30.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.029+0000 7f28ab3bc740 1 -- start start 2024-04-25T00:22:30.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.029+0000 7f28ab3bc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c079251980 0x55c07924fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.029+0000 7f28ab3bc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c079250290 0x55c079250710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.029+0000 7f28ab3bc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c07925b010 0x55c07925d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.029+0000 7f28ab3bc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c0791b42c0 con 0x55c079251980 2024-04-25T00:22:30.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.029+0000 7f28ab3bc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c07915c910 con 0x55c07925b010 2024-04-25T00:22:30.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.029+0000 7f28ab3bc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c0791b0280 con 0x55c079250290 2024-04-25T00:22:30.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.029+0000 7f28977fe640 1 --2- 172.21.15.107:0/3442329831 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f288c0a2d00 0x7f288c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.029+0000 7f28a6b6b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c079251980 0x55c07924fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.029+0000 7f28a636a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c079250290 0x55c079250710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.029+0000 7f28a6b6b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c079251980 0x55c07924fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52436/0 (socket says 172.21.15.107:52436) 2024-04-25T00:22:30.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.029+0000 7f28a6b6b640 1 -- 172.21.15.107:0/3036625208 learned_addr learned my addr 172.21.15.107:0/3036625208 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:30.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.029+0000 7f28a5b69640 1 --2- 172.21.15.107:0/3036625208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c07925b010 0x55c07925d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.029+0000 7f28a6b6b640 1 --2- 172.21.15.107:0/3442329831 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f288c0a2d00 0x7f288c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.029+0000 7f28977fe640 1 -- 172.21.15.107:0/3442329831 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f28900f01e0 con 0x7f28a0102610 2024-04-25T00:22:30.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.029+0000 7f28a6b6b640 1 -- 172.21.15.107:0/3036625208 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c079250290 msgr2=0x55c079250710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:30.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.029+0000 7f28a6b6b640 1 --2- 172.21.15.107:0/3036625208 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c079250290 0x55c079250710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.029+0000 7f28a6b6b640 1 -- 172.21.15.107:0/3036625208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c07925b010 msgr2=0x55c07925d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:30.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.029+0000 7f28a6b6b640 1 --2- 172.21.15.107:0/3036625208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c07925b010 0x55c07925d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.029+0000 7f28a6b6b640 1 -- 172.21.15.107:0/3036625208 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c07916dfb0 con 0x55c079251980 2024-04-25T00:22:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.029+0000 7f28a636a640 1 --2- 172.21.15.107:0/3036625208 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c079250290 0x55c079250710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:22:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.030+0000 7f28a6b6b640 1 --2- 172.21.15.107:0/3036625208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c079251980 0x55c07924fcc0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f289800aef0 tx=0x7f2898016640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585123168c6a0931 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.030+0000 7f28a6b6b640 1 --2- 172.21.15.107:0/3442329831 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f288c0a2d00 0x7f288c0a51c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f28980070b0 tx=0x7f289801a000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.030+0000 7f2894bfe640 1 -- 172.21.15.107:0/3036625208 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2898026070 con 0x55c079251980 2024-04-25T00:22:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.030+0000 7f2894bfe640 1 -- 172.21.15.107:0/3036625208 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2898009c50 con 0x55c079251980 2024-04-25T00:22:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.030+0000 7f2894bfe640 1 -- 172.21.15.107:0/3036625208 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2898013040 con 0x55c079251980 2024-04-25T00:22:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.030+0000 7f28ab3bc740 1 -- 172.21.15.107:0/3036625208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c079251980 msgr2=0x55c07924fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.030+0000 7f28ab3bc740 1 --2- 172.21.15.107:0/3036625208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c079251980 0x55c07924fcc0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f289800aef0 tx=0x7f2898016640 comp rx=0 tx=0).stop 2024-04-25T00:22:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.030+0000 7f28ab3bc740 1 -- 172.21.15.107:0/3036625208 shutdown_connections 2024-04-25T00:22:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.030+0000 7f28ab3bc740 1 --2- 172.21.15.107:0/3036625208 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c07925b010 0x55c07925d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.030+0000 7f28ab3bc740 1 --2- 172.21.15.107:0/3036625208 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c079250290 0x55c079250710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.030+0000 7f28ab3bc740 1 --2- 172.21.15.107:0/3036625208 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c079251980 0x55c07924fcc0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.030+0000 7f28ab3bc740 1 -- 172.21.15.107:0/3036625208 >> 172.21.15.107:0/3036625208 conn(0x55c079218950 msgr2=0x55c07924c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.031+0000 7f28ab3bc740 1 -- 172.21.15.107:0/3036625208 shutdown_connections 2024-04-25T00:22:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.031+0000 7f28ab3bc740 1 -- 172.21.15.107:0/3036625208 wait complete. 2024-04-25T00:22:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.031+0000 7f28ab3bc740 1 -- start start 2024-04-25T00:22:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.031+0000 7f28ab3bc740 1 Processor -- start 2024-04-25T00:22:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.031+0000 7f28ab3bc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c079250290 0x55c079257f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.031+0000 7f28ab3bc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c079251980 0x55c0792584d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.031+0000 7f28ab3bc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c07925b010 0x55c079258a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.031+0000 7f28ab3bc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c07925de40 con 0x55c079250290 2024-04-25T00:22:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.031+0000 7f28ab3bc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c07916fe90 con 0x55c07925b010 2024-04-25T00:22:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.031+0000 7f28ab3bc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c07925dfc0 con 0x55c079251980 2024-04-25T00:22:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.031+0000 7f28ab3bc740 10 client.0 _ll_register_callbacks cb 0x55c0791ab970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.031+0000 7f28a636a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c079250290 0x55c079257f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.031+0000 7f28a636a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c079250290 0x55c079257f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52446/0 (socket says 172.21.15.107:52446) 2024-04-25T00:22:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.031+0000 7f28a636a640 1 -- 172.21.15.107:0/306591643 learned_addr learned my addr 172.21.15.107:0/306591643 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:30.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.031+0000 7f28a5b69640 1 --2- 172.21.15.107:0/306591643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c079251980 0x55c0792584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.031+0000 7f28a6b6b640 1 --2- 172.21.15.107:0/306591643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c07925b010 0x55c079258a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.031+0000 7f28a636a640 1 -- 172.21.15.107:0/306591643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c079251980 msgr2=0x55c0792584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:30.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.032+0000 7f28a636a640 1 --2- 172.21.15.107:0/306591643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c079251980 0x55c0792584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.032+0000 7f28a636a640 1 -- 172.21.15.107:0/306591643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c07925b010 msgr2=0x55c079258a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:30.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.032+0000 7f28a636a640 1 --2- 172.21.15.107:0/306591643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c07925b010 0x55c079258a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.032+0000 7f28a636a640 1 -- 172.21.15.107:0/306591643 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c078f907e0 con 0x55c079250290 2024-04-25T00:22:30.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.032+0000 7f28a6b6b640 1 --2- 172.21.15.107:0/306591643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c07925b010 0x55c079258a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:30.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.032+0000 7f28a636a640 1 --2- 172.21.15.107:0/306591643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c079250290 0x55c079257f90 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f289c00d470 tx=0x7f289c00d930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72bf719ba27b2765 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:30.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.032+0000 7f2889ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:22:30.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.032+0000 7f2889ffb640 1 -- 172.21.15.107:0/306591643 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c01c060 con 0x55c079250290 2024-04-25T00:22:30.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.032+0000 7f2889ffb640 1 -- 172.21.15.107:0/306591643 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f289c015750 con 0x55c079250290 2024-04-25T00:22:30.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.032+0000 7f28ab3bc740 1 -- 172.21.15.107:0/306591643 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2898009c50 con 0x55c079250290 2024-04-25T00:22:30.041 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.032+0000 7f2889ffb640 1 -- 172.21.15.107:0/306591643 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f289c023050 con 0x55c079250290 2024-04-25T00:22:30.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.032+0000 7f28ab3bc740 10 client.16950 fetch_fsmap learned FSMap version 34 2024-04-25T00:22:30.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.032+0000 7f28ab3bc740 1 -- 172.21.15.107:0/306591643 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c078ff0f70 con 0x55c079250290 2024-04-25T00:22:30.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.032+0000 7f2889ffb640 1 -- 172.21.15.107:0/306591643 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2898009c50 con 0x55c079250290 2024-04-25T00:22:30.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.032+0000 7f2889ffb640 1 -- 172.21.15.107:0/306591643 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f289c016bd0 con 0x55c079250290 2024-04-25T00:22:30.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.032+0000 7f28ab3bc740 10 client.16950 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:22:30.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.032+0000 7f28ab3bc740 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:30.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.032+0000 7f28ab3bc740 20 client.16950 populate_metadata read hostname 'smithi107' 2024-04-25T00:22:30.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.032+0000 7f28ab3bc740 1 --2- 172.21.15.107:0/306591643 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c07926b170 0x55c07926d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.032+0000 7f28ab3bc740 4 client.16950 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:22:30.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.032+0000 7f28ab3bc740 1 -- 172.21.15.107:0/306591643 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0791885a0 con 0x55c07926b170 2024-04-25T00:22:30.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.033+0000 7f28a5b69640 1 --2- 172.21.15.107:0/306591643 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c07926b170 0x55c07926d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.033+0000 7f28a5b69640 1 --2- 172.21.15.107:0/306591643 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c07926b170 0x55c07926d5e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6d438827246ec8b server_cookie=d71e670e3d89a2d7 in_seq=0 out_seq=0 2024-04-25T00:22:30.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.033+0000 7f2889ffb640 10 client.16950 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:22:30.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.034+0000 7f2889ffb640 1 -- 172.21.15.107:0/306591643 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c0791885a0 con 0x55c07926b170 2024-04-25T00:22:30.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.034+0000 7f2889ffb640 10 client.16950 handle_command_reply: tid=0 2024-04-25T00:22:30.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.034+0000 7f28ab3bc740 1 client.16950 shutdown 2024-04-25T00:22:30.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.034+0000 7f28ab3bc740 1 -- 172.21.15.107:0/306591643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c079250290 msgr2=0x55c079257f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:30.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.034+0000 7f28ab3bc740 1 --2- 172.21.15.107:0/306591643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c079250290 0x55c079257f90 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f289c00d470 tx=0x7f289c00d930 comp rx=0 tx=0).stop 2024-04-25T00:22:30.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.034+0000 7f28ab3bc740 1 -- 172.21.15.107:0/306591643 shutdown_connections 2024-04-25T00:22:30.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.034+0000 7f28ab3bc740 1 --2- 172.21.15.107:0/306591643 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c07926b170 0x55c07926d5e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.034+0000 7f28ab3bc740 1 --2- 172.21.15.107:0/306591643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c07925b010 0x55c079258a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.034+0000 7f28ab3bc740 1 --2- 172.21.15.107:0/306591643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c079251980 0x55c0792584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.034+0000 7f2889ffb640 0 client.16950 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:22:30.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.034+0000 7f28ab3bc740 1 --2- 172.21.15.107:0/306591643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c079250290 0x55c079257f90 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.034+0000 7f28ab3bc740 1 -- 172.21.15.107:0/306591643 >> 172.21.15.107:0/306591643 conn(0x55c079218950 msgr2=0x55c0792525f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:30.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.034+0000 7f28ab3bc740 1 -- 172.21.15.107:0/306591643 shutdown_connections 2024-04-25T00:22:30.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.034+0000 7f28ab3bc740 1 -- 172.21.15.107:0/306591643 wait complete. 2024-04-25T00:22:30.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.034+0000 7f28ab3bc740 20 client.16950 trim_cache size 0 max 16384 2024-04-25T00:22:30.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.051+0000 7f28ab3bc740 1 Processor -- start 2024-04-25T00:22:30.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.051+0000 7f28ab3bc740 1 -- start start 2024-04-25T00:22:30.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.051+0000 7f28ab3bc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c079250290 0x55c079266d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.051+0000 7f28ab3bc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c079251980 0x55c07924b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.052+0000 7f28ab3bc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c07925b010 0x55c079272eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.052+0000 7f28ab3bc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f28a01b35a0 con 0x55c07925b010 2024-04-25T00:22:30.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.052+0000 7f28ab3bc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f28a01b0e70 con 0x55c079250290 2024-04-25T00:22:30.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.052+0000 7f28ab3bc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f28a01aecc0 con 0x55c079251980 2024-04-25T00:22:30.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.052+0000 7f28a6b6b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c07925b010 0x55c079272eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.052+0000 7f28a6b6b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c07925b010 0x55c079272eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52458/0 (socket says 172.21.15.107:52458) 2024-04-25T00:22:30.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.052+0000 7f28a6b6b640 1 -- 172.21.15.107:0/2128009937 learned_addr learned my addr 172.21.15.107:0/2128009937 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:30.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.052+0000 7f28a5b69640 1 --2- 172.21.15.107:0/2128009937 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c079251980 0x55c07924b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.052+0000 7f28a6b6b640 1 -- 172.21.15.107:0/2128009937 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c079251980 msgr2=0x55c07924b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:30.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.052+0000 7f28a6b6b640 1 --2- 172.21.15.107:0/2128009937 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c079251980 0x55c07924b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.052+0000 7f28a6b6b640 1 -- 172.21.15.107:0/2128009937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c079250290 msgr2=0x55c079266d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:30.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.052+0000 7f28a636a640 1 --2- 172.21.15.107:0/2128009937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c079250290 0x55c079266d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.052+0000 7f28a6b6b640 1 --2- 172.21.15.107:0/2128009937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c079250290 0x55c079266d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.052+0000 7f28a6b6b640 1 -- 172.21.15.107:0/2128009937 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c078e88170 con 0x55c07925b010 2024-04-25T00:22:30.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.052+0000 7f28a636a640 1 --2- 172.21.15.107:0/2128009937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c079250290 0x55c079266d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:30.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.052+0000 7f28a6b6b640 1 --2- 172.21.15.107:0/2128009937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c07925b010 0x55c079272eb0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f2898017010 tx=0x7f289800ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2fc291ec5fa3cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:30.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.052+0000 7f288bfff640 1 -- 172.21.15.107:0/2128009937 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2898030070 con 0x55c07925b010 2024-04-25T00:22:30.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.052+0000 7f288bfff640 1 -- 172.21.15.107:0/2128009937 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2898019c50 con 0x55c07925b010 2024-04-25T00:22:30.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.052+0000 7f288bfff640 1 -- 172.21.15.107:0/2128009937 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2898026040 con 0x55c07925b010 2024-04-25T00:22:30.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.053+0000 7f28ab3bc740 1 -- 172.21.15.107:0/2128009937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c07925b010 msgr2=0x55c079272eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:30.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.053+0000 7f28ab3bc740 1 --2- 172.21.15.107:0/2128009937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c07925b010 0x55c079272eb0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f2898017010 tx=0x7f289800ad70 comp rx=0 tx=0).stop 2024-04-25T00:22:30.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.053+0000 7f28ab3bc740 1 -- 172.21.15.107:0/2128009937 shutdown_connections 2024-04-25T00:22:30.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.053+0000 7f28ab3bc740 1 --2- 172.21.15.107:0/2128009937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c07925b010 0x55c079272eb0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.053+0000 7f28ab3bc740 1 --2- 172.21.15.107:0/2128009937 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c079251980 0x55c07924b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.053+0000 7f28ab3bc740 1 --2- 172.21.15.107:0/2128009937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c079250290 0x55c079266d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.053+0000 7f28ab3bc740 1 -- 172.21.15.107:0/2128009937 >> 172.21.15.107:0/2128009937 conn(0x55c079218950 msgr2=0x55c07924b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:30.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.053+0000 7f28ab3bc740 1 -- 172.21.15.107:0/2128009937 shutdown_connections 2024-04-25T00:22:30.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.053+0000 7f28ab3bc740 1 -- 172.21.15.107:0/2128009937 wait complete. 2024-04-25T00:22:30.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.053+0000 7f28ab3bc740 1 -- start start 2024-04-25T00:22:30.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.053+0000 7f28ab3bc740 1 Processor -- start 2024-04-25T00:22:30.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.053+0000 7f28ab3bc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c079250290 0x55c079263250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f28ab3bc740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c079251980 0x55c079263790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f28ab3bc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c07925b010 0x55c079260220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f28ab3bc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f28a01af330 con 0x55c079250290 2024-04-25T00:22:30.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f28ab3bc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f28a01b07a0 con 0x55c07925b010 2024-04-25T00:22:30.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f28ab3bc740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f28a01afa00 con 0x55c079251980 2024-04-25T00:22:30.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f28ab3bc740 10 client.0 _ll_register_callbacks cb 0x7f289c00b500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:30.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f28a636a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c079250290 0x55c079263250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f28a636a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c079250290 0x55c079263250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52468/0 (socket says 172.21.15.107:52468) 2024-04-25T00:22:30.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f28a636a640 1 -- 172.21.15.107:0/1127566655 learned_addr learned my addr 172.21.15.107:0/1127566655 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:30.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f28a5b69640 1 --2- 172.21.15.107:0/1127566655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c079251980 0x55c079263790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f28a6b6b640 1 --2- 172.21.15.107:0/1127566655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c07925b010 0x55c079260220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f28a636a640 1 -- 172.21.15.107:0/1127566655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c079251980 msgr2=0x55c079263790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:30.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f28a636a640 1 --2- 172.21.15.107:0/1127566655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c079251980 0x55c079263790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f28a636a640 1 -- 172.21.15.107:0/1127566655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c07925b010 msgr2=0x55c079260220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:30.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f28a636a640 1 --2- 172.21.15.107:0/1127566655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c07925b010 0x55c079260220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f28a636a640 1 -- 172.21.15.107:0/1127566655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c079260a50 con 0x55c079250290 2024-04-25T00:22:30.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f28a636a640 1 --2- 172.21.15.107:0/1127566655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c079250290 0x55c079263250 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f289c000c00 tx=0x7f289c0257a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772555620b7d96c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:30.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f2889ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:22:30.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f2889ffb640 1 -- 172.21.15.107:0/1127566655 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c02b020 con 0x55c079250290 2024-04-25T00:22:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f2889ffb640 1 -- 172.21.15.107:0/1127566655 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f289c025cd0 con 0x55c079250290 2024-04-25T00:22:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f28ab3bc740 1 -- 172.21.15.107:0/1127566655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2898019c50 con 0x55c079250290 2024-04-25T00:22:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f2889ffb640 1 -- 172.21.15.107:0/1127566655 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f289c036050 con 0x55c079250290 2024-04-25T00:22:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f28ab3bc740 10 client.16956 fetch_fsmap learned FSMap version 34 2024-04-25T00:22:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f28ab3bc740 1 -- 172.21.15.107:0/1127566655 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c0792d21a0 con 0x55c079250290 2024-04-25T00:22:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.054+0000 7f2889ffb640 1 -- 172.21.15.107:0/1127566655 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2898019c50 con 0x55c079250290 2024-04-25T00:22:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.055+0000 7f2889ffb640 1 -- 172.21.15.107:0/1127566655 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f289c006670 con 0x55c079250290 2024-04-25T00:22:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.055+0000 7f28ab3bc740 10 client.16956 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:22:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.055+0000 7f28ab3bc740 10 client.16956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.055+0000 7f28ab3bc740 20 client.16956 populate_metadata read hostname 'smithi107' 2024-04-25T00:22:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.055+0000 7f28ab3bc740 1 --2- 172.21.15.107:0/1127566655 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c0792d2970 0x55c0792d4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.055+0000 7f28ab3bc740 4 client.16956 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:22:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.055+0000 7f28ab3bc740 1 -- 172.21.15.107:0/1127566655 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c0791c1800 con 0x55c0792d2970 2024-04-25T00:22:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.055+0000 7f28a5b69640 1 --2- 172.21.15.107:0/1127566655 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c0792d2970 0x55c0792d4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.055+0000 7f2889ffb640 10 client.16956 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:22:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.055+0000 7f28a5b69640 1 --2- 172.21.15.107:0/1127566655 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c0792d2970 0x55c0792d4d90 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10af4b9a865c3ced server_cookie=a78276c72e80e389 in_seq=0 out_seq=0 2024-04-25T00:22:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.056+0000 7f2889ffb640 1 -- 172.21.15.107:0/1127566655 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c0791c1800 con 0x55c0792d2970 2024-04-25T00:22:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.056+0000 7f2889ffb640 10 client.16956 handle_command_reply: tid=0 2024-04-25T00:22:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.056+0000 7f28ab3bc740 1 client.16956 shutdown 2024-04-25T00:22:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.057+0000 7f28ab3bc740 1 -- 172.21.15.107:0/1127566655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c079250290 msgr2=0x55c079263250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.057+0000 7f28ab3bc740 1 --2- 172.21.15.107:0/1127566655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c079250290 0x55c079263250 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f289c000c00 tx=0x7f289c0257a0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.057+0000 7f28ab3bc740 1 -- 172.21.15.107:0/1127566655 shutdown_connections 2024-04-25T00:22:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.057+0000 7f28ab3bc740 1 --2- 172.21.15.107:0/1127566655 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c0792d2970 0x55c0792d4d90 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.057+0000 7f28ab3bc740 1 --2- 172.21.15.107:0/1127566655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c07925b010 0x55c079260220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.057+0000 7f28ab3bc740 1 --2- 172.21.15.107:0/1127566655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c079251980 0x55c079263790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.057+0000 7f28ab3bc740 1 --2- 172.21.15.107:0/1127566655 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c079250290 0x55c079263250 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.057+0000 7f2889ffb640 0 client.16956 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:22:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.057+0000 7f28ab3bc740 1 -- 172.21.15.107:0/1127566655 >> 172.21.15.107:0/1127566655 conn(0x55c079218950 msgr2=0x55c07924b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.057+0000 7f28ab3bc740 1 -- 172.21.15.107:0/1127566655 shutdown_connections 2024-04-25T00:22:30.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.057+0000 7f28ab3bc740 1 -- 172.21.15.107:0/1127566655 wait complete. 2024-04-25T00:22:30.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.057+0000 7f28ab3bc740 20 client.16956 trim_cache size 0 max 16384 2024-04-25T00:22:30.065 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:22:30.065 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-04-25T00:22:30.065 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "dd0cc5c1-c09e-48d0-97f8-f0897b1bfe0f", 2024-04-25T00:22:30.065 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-04-25T00:22:30.065 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:22:30.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.057+0000 7f28a85ce640 1 -- 172.21.15.107:0/3442329831 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f288c0a2d00 msgr2=0x7f288c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:30.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.057+0000 7f28a85ce640 1 --2- 172.21.15.107:0/3442329831 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f288c0a2d00 0x7f288c0a51c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f28980070b0 tx=0x7f289801a000 comp rx=0 tx=0).stop 2024-04-25T00:22:30.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.057+0000 7f28a85ce640 1 -- 172.21.15.107:0/3442329831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a0102610 msgr2=0x7f28a019a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:30.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.057+0000 7f28a85ce640 1 --2- 172.21.15.107:0/3442329831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a0102610 0x7f28a019a8e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f2890005ab0 tx=0x7f28900042f0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.057+0000 7f28a85ce640 1 -- 172.21.15.107:0/3442329831 shutdown_connections 2024-04-25T00:22:30.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.057+0000 7f28a85ce640 1 --2- 172.21.15.107:0/3442329831 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f288c0a2d00 0x7f288c0a51c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.057+0000 7f28a85ce640 1 --2- 172.21.15.107:0/3442329831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f28a010e000 0x7f28a019ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.057+0000 7f28a85ce640 1 --2- 172.21.15.107:0/3442329831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28a0102610 0x7f28a019a8e0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.057+0000 7f28a85ce640 1 --2- 172.21.15.107:0/3442329831 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f28a006a280 0x7f28a019a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.057+0000 7f28a85ce640 1 -- 172.21.15.107:0/3442329831 >> 172.21.15.107:0/3442329831 conn(0x7f28a0076440 msgr2=0x7f28a010eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:30.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.057+0000 7f28a85ce640 1 -- 172.21.15.107:0/3442329831 shutdown_connections 2024-04-25T00:22:30.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.057+0000 7f28a85ce640 1 -- 172.21.15.107:0/3442329831 wait complete. 2024-04-25T00:22:30.069 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:22:30.069 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:22:30.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:29 smithi195 ceph-mon[32833]: pgmap v184: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:22:30.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:29 smithi170 ceph-mon[32807]: pgmap v184: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:22:30.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.207+0000 7fdb21f29640 1 -- 172.21.15.107:0/3864180265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb1c101710 msgr2=0x7fdb1c10d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:30.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.207+0000 7fdb21f29640 1 --2- 172.21.15.107:0/3864180265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb1c101710 0x7fdb1c10d700 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fdb14009cd0 tx=0x7fdb1402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.207+0000 7fdb21f29640 1 -- 172.21.15.107:0/3864180265 shutdown_connections 2024-04-25T00:22:30.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.207+0000 7fdb21f29640 1 --2- 172.21.15.107:0/3864180265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb1c101710 0x7fdb1c10d700 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.207+0000 7fdb21f29640 1 --2- 172.21.15.107:0/3864180265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb1c100d50 0x7fdb1c1011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.207+0000 7fdb21f29640 1 --2- 172.21.15.107:0/3864180265 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb1c106840 0x7fdb1c106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.207+0000 7fdb21f29640 1 -- 172.21.15.107:0/3864180265 >> 172.21.15.107:0/3864180265 conn(0x7fdb1c076c00 msgr2=0x7fdb1c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:30.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.207+0000 7fdb21f29640 1 -- 172.21.15.107:0/3864180265 shutdown_connections 2024-04-25T00:22:30.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.207+0000 7fdb21f29640 1 -- 172.21.15.107:0/3864180265 wait complete. 2024-04-25T00:22:30.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.208+0000 7fdb21f29640 1 Processor -- start 2024-04-25T00:22:30.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.208+0000 7fdb21f29640 1 -- start start 2024-04-25T00:22:30.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.208+0000 7fdb21f29640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb1c100d50 0x7fdb1c198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.208+0000 7fdb21f29640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb1c101710 0x7fdb1c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.208+0000 7fdb21f29640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb1c106840 0x7fdb1c19cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.208+0000 7fdb21f29640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c111d50 con 0x7fdb1c106840 2024-04-25T00:22:30.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.208+0000 7fdb21f29640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c111bd0 con 0x7fdb1c100d50 2024-04-25T00:22:30.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.208+0000 7fdb21f29640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c111ed0 con 0x7fdb1c101710 2024-04-25T00:22:30.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.208+0000 7fdb1bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb1c106840 0x7fdb1c19cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.208+0000 7fdb1bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb1c106840 0x7fdb1c19cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52484/0 (socket says 172.21.15.107:52484) 2024-04-25T00:22:30.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.208+0000 7fdb1bfff640 1 -- 172.21.15.107:0/1762942450 learned_addr learned my addr 172.21.15.107:0/1762942450 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:30.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.208+0000 7fdb1b7fe640 1 --2- 172.21.15.107:0/1762942450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb1c100d50 0x7fdb1c198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.209+0000 7fdb1bfff640 1 -- 172.21.15.107:0/1762942450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb1c101710 msgr2=0x7fdb1c198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:30.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.209+0000 7fdb1bfff640 1 --2- 172.21.15.107:0/1762942450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb1c101710 0x7fdb1c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.209+0000 7fdb1bfff640 1 -- 172.21.15.107:0/1762942450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb1c100d50 msgr2=0x7fdb1c198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:30.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.209+0000 7fdb1bfff640 1 --2- 172.21.15.107:0/1762942450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb1c100d50 0x7fdb1c198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.209+0000 7fdb1bfff640 1 -- 172.21.15.107:0/1762942450 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1c19d200 con 0x7fdb1c106840 2024-04-25T00:22:30.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.209+0000 7fdb1b7fe640 1 --2- 172.21.15.107:0/1762942450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb1c100d50 0x7fdb1c198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:30.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.209+0000 7fdb1bfff640 1 --2- 172.21.15.107:0/1762942450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb1c106840 0x7fdb1c19cad0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fdb14005cc0 tx=0x7fdb14003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:30.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.209+0000 7fdb18ff9640 1 -- 172.21.15.107:0/1762942450 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14004040 con 0x7fdb1c106840 2024-04-25T00:22:30.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.209+0000 7fdb18ff9640 1 -- 172.21.15.107:0/1762942450 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb140046c0 con 0x7fdb1c106840 2024-04-25T00:22:30.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.209+0000 7fdb18ff9640 1 -- 172.21.15.107:0/1762942450 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb14046480 con 0x7fdb1c106840 2024-04-25T00:22:30.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.209+0000 7fdb21f29640 1 -- 172.21.15.107:0/1762942450 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1c19d490 con 0x7fdb1c106840 2024-04-25T00:22:30.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.209+0000 7fdb21f29640 1 -- 172.21.15.107:0/1762942450 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb1c1a4d80 con 0x7fdb1c106840 2024-04-25T00:22:30.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.211+0000 7fdb18ff9640 1 -- 172.21.15.107:0/1762942450 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fdb140305a0 con 0x7fdb1c106840 2024-04-25T00:22:30.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.211+0000 7fdb24c39740 1 Processor -- start 2024-04-25T00:22:30.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.211+0000 7fdb24c39740 1 -- start start 2024-04-25T00:22:30.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.211+0000 7fdb24c39740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca4d630 0x55da1ca4da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.211+0000 7fdb24c39740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da1ca4df50 0x55da1ca580e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.211+0000 7fdb24c39740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da1ca58620 0x55da1ca5aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.211+0000 7fdb24c39740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55da1c9b26b0 con 0x55da1ca4d630 2024-04-25T00:22:30.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.211+0000 7fdb18ff9640 1 --2- 172.21.15.107:0/1762942450 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdb040a2d00 0x7fdb040a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.211+0000 7fdb24c39740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55da1c95a880 con 0x55da1ca58620 2024-04-25T00:22:30.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.211+0000 7fdb24c39740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55da1c9ae670 con 0x55da1ca4df50 2024-04-25T00:22:30.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.212+0000 7fdb1affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca4d630 0x55da1ca4da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.212+0000 7fdb1affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca4d630 0x55da1ca4da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52500/0 (socket says 172.21.15.107:52500) 2024-04-25T00:22:30.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.212+0000 7fdb1affd640 1 -- 172.21.15.107:0/3768751659 learned_addr learned my addr 172.21.15.107:0/3768751659 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:30.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.212+0000 7fdb1bfff640 1 --2- 172.21.15.107:0/1762942450 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdb040a2d00 0x7fdb040a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.212+0000 7fdb18ff9640 1 -- 172.21.15.107:0/1762942450 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fdb1402e030 con 0x7fdb1c106840 2024-04-25T00:22:30.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.212+0000 7fdb1b7fe640 1 --2- 172.21.15.107:0/3768751659 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da1ca4df50 0x55da1ca580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.212+0000 7fdb1affd640 1 --2- 172.21.15.107:0/3768751659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da1ca58620 0x55da1ca5aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.212+0000 7fdb1affd640 1 -- 172.21.15.107:0/3768751659 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da1ca4df50 msgr2=0x55da1ca580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:30.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.212+0000 7fdb1affd640 1 --2- 172.21.15.107:0/3768751659 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da1ca4df50 0x55da1ca580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.212+0000 7fdb1affd640 1 -- 172.21.15.107:0/3768751659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da1ca58620 msgr2=0x55da1ca5aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:30.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.212+0000 7fdb1affd640 1 --2- 172.21.15.107:0/3768751659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da1ca58620 0x55da1ca5aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.212+0000 7fdb1affd640 1 -- 172.21.15.107:0/3768751659 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da1c7907d0 con 0x55da1ca4d630 2024-04-25T00:22:30.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.212+0000 7fdb1bfff640 1 --2- 172.21.15.107:0/1762942450 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdb040a2d00 0x7fdb040a51c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdb140ee9a0 tx=0x7fdb140dd040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:30.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.212+0000 7fdb1affd640 1 --2- 172.21.15.107:0/3768751659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca4d630 0x55da1ca4da10 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fdb10014490 tx=0x7fdb1000b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8da0830613bb58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:30.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.212+0000 7fdb01ffb640 1 -- 172.21.15.107:0/3768751659 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb10005020 con 0x55da1ca4d630 2024-04-25T00:22:30.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.212+0000 7fdb01ffb640 1 -- 172.21.15.107:0/3768751659 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb100047a0 con 0x55da1ca4d630 2024-04-25T00:22:30.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.212+0000 7fdb01ffb640 1 -- 172.21.15.107:0/3768751659 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb10012630 con 0x55da1ca4d630 2024-04-25T00:22:30.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.213+0000 7fdb24c39740 1 -- 172.21.15.107:0/3768751659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca4d630 msgr2=0x55da1ca4da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:30.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.213+0000 7fdb24c39740 1 --2- 172.21.15.107:0/3768751659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca4d630 0x55da1ca4da10 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fdb10014490 tx=0x7fdb1000b4d0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.213+0000 7fdb24c39740 1 -- 172.21.15.107:0/3768751659 shutdown_connections 2024-04-25T00:22:30.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.213+0000 7fdb24c39740 1 --2- 172.21.15.107:0/3768751659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da1ca58620 0x55da1ca5aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.213+0000 7fdb24c39740 1 --2- 172.21.15.107:0/3768751659 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da1ca4df50 0x55da1ca580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.213+0000 7fdb24c39740 1 --2- 172.21.15.107:0/3768751659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca4d630 0x55da1ca4da10 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.213+0000 7fdb24c39740 1 -- 172.21.15.107:0/3768751659 >> 172.21.15.107:0/3768751659 conn(0x55da1ca49950 msgr2=0x55da1ca49d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:30.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.213+0000 7fdb24c39740 1 -- 172.21.15.107:0/3768751659 shutdown_connections 2024-04-25T00:22:30.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.213+0000 7fdb24c39740 1 -- 172.21.15.107:0/3768751659 wait complete. 2024-04-25T00:22:30.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.213+0000 7fdb24c39740 1 -- start start 2024-04-25T00:22:30.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.214+0000 7fdb24c39740 1 Processor -- start 2024-04-25T00:22:30.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.214+0000 7fdb24c39740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da1ca4d630 0x55da1ca55560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.214+0000 7fdb24c39740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca4df50 0x55da1ca55aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.214+0000 7fdb24c39740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da1ca58620 0x55da1ca55fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.214+0000 7fdb24c39740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55da1ca5b410 con 0x55da1ca4df50 2024-04-25T00:22:30.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.214+0000 7fdb24c39740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55da1c96df50 con 0x55da1ca4d630 2024-04-25T00:22:30.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.214+0000 7fdb24c39740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55da1ca5b590 con 0x55da1ca58620 2024-04-25T00:22:30.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.214+0000 7fdb24c39740 10 client.0 _ll_register_callbacks cb 0x55da1c9a9000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:30.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.214+0000 7fdb1b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca4df50 0x55da1ca55aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.214+0000 7fdb1b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca4df50 0x55da1ca55aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52508/0 (socket says 172.21.15.107:52508) 2024-04-25T00:22:30.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.214+0000 7fdb1b7fe640 1 -- 172.21.15.107:0/606715116 learned_addr learned my addr 172.21.15.107:0/606715116 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:30.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.214+0000 7fdb1affd640 1 --2- 172.21.15.107:0/606715116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da1ca4d630 0x55da1ca55560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.214+0000 7fdb1affd640 1 --2- 172.21.15.107:0/606715116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da1ca58620 0x55da1ca55fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.214+0000 7fdb1b7fe640 1 -- 172.21.15.107:0/606715116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da1ca58620 msgr2=0x55da1ca55fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:30.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.214+0000 7fdb1b7fe640 1 --2- 172.21.15.107:0/606715116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da1ca58620 0x55da1ca55fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.214+0000 7fdb1b7fe640 1 -- 172.21.15.107:0/606715116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da1ca4d630 msgr2=0x55da1ca55560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:30.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.214+0000 7fdb1b7fe640 1 --2- 172.21.15.107:0/606715116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da1ca4d630 0x55da1ca55560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.214+0000 7fdb1b7fe640 1 -- 172.21.15.107:0/606715116 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da1c6c63f0 con 0x55da1ca4df50 2024-04-25T00:22:30.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.214+0000 7fdb1affd640 1 --2- 172.21.15.107:0/606715116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da1ca58620 0x55da1ca55fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:30.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.214+0000 7fdb1b7fe640 1 --2- 172.21.15.107:0/606715116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca4df50 0x55da1ca55aa0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c005940 tx=0x7fdb0c00fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9063e700dc18a8f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:30.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.214+0000 7fdafaffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:22:30.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.215+0000 7fdafaffd640 1 -- 172.21.15.107:0/606715116 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c016070 con 0x55da1ca4df50 2024-04-25T00:22:30.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.215+0000 7fdafaffd640 1 -- 172.21.15.107:0/606715116 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fdb0c00baa0 con 0x55da1ca4df50 2024-04-25T00:22:30.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.215+0000 7fdb24c39740 1 -- 172.21.15.107:0/606715116 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb100047a0 con 0x55da1ca4df50 2024-04-25T00:22:30.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.215+0000 7fdafaffd640 1 -- 172.21.15.107:0/606715116 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdb0c00bca0 con 0x55da1ca4df50 2024-04-25T00:22:30.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.215+0000 7fdb24c39740 10 client.16968 fetch_fsmap learned FSMap version 34 2024-04-25T00:22:30.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.215+0000 7fdb24c39740 1 -- 172.21.15.107:0/606715116 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55da1c6860f0 con 0x55da1ca4df50 2024-04-25T00:22:30.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.215+0000 7fdafaffd640 1 -- 172.21.15.107:0/606715116 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb100047a0 con 0x55da1ca4df50 2024-04-25T00:22:30.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.215+0000 7fdafaffd640 1 -- 172.21.15.107:0/606715116 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fdb0c01a550 con 0x55da1ca4df50 2024-04-25T00:22:30.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.215+0000 7fdb24c39740 10 client.16968 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:22:30.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.215+0000 7fdb24c39740 10 client.16968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:30.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.215+0000 7fdb24c39740 20 client.16968 populate_metadata read hostname 'smithi107' 2024-04-25T00:22:30.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.215+0000 7fdb24c39740 1 --2- 172.21.15.107:0/606715116 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55da1ca689a0 0x55da1ca6add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.215+0000 7fdb24c39740 4 client.16968 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:22:30.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.215+0000 7fdb24c39740 1 -- 172.21.15.107:0/606715116 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da1c9bfbf0 con 0x55da1ca689a0 2024-04-25T00:22:30.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.215+0000 7fdb1affd640 1 --2- 172.21.15.107:0/606715116 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55da1ca689a0 0x55da1ca6add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.216+0000 7fdafaffd640 10 client.16968 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:22:30.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.216+0000 7fdb1affd640 1 --2- 172.21.15.107:0/606715116 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55da1ca689a0 0x55da1ca6add0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efdf468e97a918e0 server_cookie=6f164de2064ee385 in_seq=0 out_seq=0 2024-04-25T00:22:30.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.216+0000 7fdafaffd640 1 -- 172.21.15.107:0/606715116 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da1c9bfbf0 con 0x55da1ca689a0 2024-04-25T00:22:30.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.216+0000 7fdafaffd640 10 client.16968 handle_command_reply: tid=0 2024-04-25T00:22:30.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.216+0000 7fdb24c39740 1 client.16968 shutdown 2024-04-25T00:22:30.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.217+0000 7fdb24c39740 1 -- 172.21.15.107:0/606715116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca4df50 msgr2=0x55da1ca55aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:30.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.217+0000 7fdb24c39740 1 --2- 172.21.15.107:0/606715116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca4df50 0x55da1ca55aa0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c005940 tx=0x7fdb0c00fbe0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.217+0000 7fdb24c39740 1 -- 172.21.15.107:0/606715116 shutdown_connections 2024-04-25T00:22:30.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.217+0000 7fdb24c39740 1 --2- 172.21.15.107:0/606715116 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55da1ca689a0 0x55da1ca6add0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.217+0000 7fdb24c39740 1 --2- 172.21.15.107:0/606715116 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da1ca58620 0x55da1ca55fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.217+0000 7fdb24c39740 1 --2- 172.21.15.107:0/606715116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca4df50 0x55da1ca55aa0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.217+0000 7fdafaffd640 0 client.16968 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:22:30.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.217+0000 7fdb24c39740 1 --2- 172.21.15.107:0/606715116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da1ca4d630 0x55da1ca55560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.217+0000 7fdb24c39740 1 -- 172.21.15.107:0/606715116 >> 172.21.15.107:0/606715116 conn(0x55da1ca49950 msgr2=0x55da1ca4f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:30.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.217+0000 7fdb24c39740 1 -- 172.21.15.107:0/606715116 shutdown_connections 2024-04-25T00:22:30.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.217+0000 7fdb24c39740 1 -- 172.21.15.107:0/606715116 wait complete. 2024-04-25T00:22:30.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.217+0000 7fdb24c39740 20 client.16968 trim_cache size 0 max 16384 2024-04-25T00:22:30.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.234+0000 7fdb24c39740 1 Processor -- start 2024-04-25T00:22:30.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.234+0000 7fdb24c39740 1 -- start start 2024-04-25T00:22:30.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.234+0000 7fdb24c39740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da1ca4d630 0x55da1ca64300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.234+0000 7fdb24c39740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca4df50 0x55da1ca4fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.234+0000 7fdb24c39740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da1ca58620 0x55da1ca55c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.234+0000 7fdb24c39740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c1b1390 con 0x55da1ca4df50 2024-04-25T00:22:30.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.234+0000 7fdb24c39740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c1aec60 con 0x55da1ca4d630 2024-04-25T00:22:30.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.234+0000 7fdb24c39740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c1acab0 con 0x55da1ca58620 2024-04-25T00:22:30.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.234+0000 7fdb1b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca4df50 0x55da1ca4fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.234+0000 7fdb1b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca4df50 0x55da1ca4fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52524/0 (socket says 172.21.15.107:52524) 2024-04-25T00:22:30.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.234+0000 7fdb1b7fe640 1 -- 172.21.15.107:0/390819076 learned_addr learned my addr 172.21.15.107:0/390819076 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:30.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.234+0000 7fdb1affd640 1 --2- 172.21.15.107:0/390819076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da1ca4d630 0x55da1ca64300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.234+0000 7fdb1b7fe640 1 -- 172.21.15.107:0/390819076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da1ca58620 msgr2=0x55da1ca55c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:30.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.235+0000 7fdb1b7fe640 1 --2- 172.21.15.107:0/390819076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da1ca58620 0x55da1ca55c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.235+0000 7fdb1b7fe640 1 -- 172.21.15.107:0/390819076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da1ca4d630 msgr2=0x55da1ca64300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:30.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.235+0000 7fdb1b7fe640 1 --2- 172.21.15.107:0/390819076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da1ca4d630 0x55da1ca64300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.235+0000 7fdb1b7fe640 1 -- 172.21.15.107:0/390819076 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da1ca6b450 con 0x55da1ca4df50 2024-04-25T00:22:30.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.235+0000 7fdb1affd640 1 --2- 172.21.15.107:0/390819076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da1ca4d630 0x55da1ca64300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:30.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.235+0000 7fdb1b7fe640 1 --2- 172.21.15.107:0/390819076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca4df50 0x55da1ca4fd90 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00d4d0 tx=0x7fdb0c00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924a5c2ad6ab0708 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:30.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.235+0000 7fdb017fa640 1 -- 172.21.15.107:0/390819076 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c016040 con 0x55da1ca4df50 2024-04-25T00:22:30.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.235+0000 7fdb017fa640 1 -- 172.21.15.107:0/390819076 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb0c01a610 con 0x55da1ca4df50 2024-04-25T00:22:30.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.235+0000 7fdb017fa640 1 -- 172.21.15.107:0/390819076 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c028cb0 con 0x55da1ca4df50 2024-04-25T00:22:30.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.235+0000 7fdb24c39740 1 -- 172.21.15.107:0/390819076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca4df50 msgr2=0x55da1ca4fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:30.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.235+0000 7fdb24c39740 1 --2- 172.21.15.107:0/390819076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca4df50 0x55da1ca4fd90 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00d4d0 tx=0x7fdb0c00bfd0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.235+0000 7fdb24c39740 1 -- 172.21.15.107:0/390819076 shutdown_connections 2024-04-25T00:22:30.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.235+0000 7fdb24c39740 1 --2- 172.21.15.107:0/390819076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da1ca58620 0x55da1ca55c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.235+0000 7fdb24c39740 1 --2- 172.21.15.107:0/390819076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca4df50 0x55da1ca4fd90 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.235+0000 7fdb24c39740 1 --2- 172.21.15.107:0/390819076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da1ca4d630 0x55da1ca64300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.235+0000 7fdb24c39740 1 -- 172.21.15.107:0/390819076 >> 172.21.15.107:0/390819076 conn(0x55da1ca49950 msgr2=0x55da1ca55520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:30.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.235+0000 7fdb24c39740 1 -- 172.21.15.107:0/390819076 shutdown_connections 2024-04-25T00:22:30.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.235+0000 7fdb24c39740 1 -- 172.21.15.107:0/390819076 wait complete. 2024-04-25T00:22:30.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.236+0000 7fdb24c39740 1 -- start start 2024-04-25T00:22:30.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.236+0000 7fdb24c39740 1 Processor -- start 2024-04-25T00:22:30.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.236+0000 7fdb24c39740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da1ca4d630 0x55da1ca52860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.236+0000 7fdb24c39740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da1ca4df50 0x55da1ca52da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.236+0000 7fdb24c39740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca58620 0x55da1cacdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.236+0000 7fdb24c39740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c1ad120 con 0x55da1ca58620 2024-04-25T00:22:30.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.236+0000 7fdb24c39740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c1ae590 con 0x55da1ca4df50 2024-04-25T00:22:30.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.236+0000 7fdb24c39740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdb1c1ad7f0 con 0x55da1ca4d630 2024-04-25T00:22:30.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.236+0000 7fdb24c39740 10 client.0 _ll_register_callbacks cb 0x7fdb0c00d380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:30.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.236+0000 7fdb1affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca58620 0x55da1cacdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.236+0000 7fdb1affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca58620 0x55da1cacdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52538/0 (socket says 172.21.15.107:52538) 2024-04-25T00:22:30.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.236+0000 7fdb1affd640 1 -- 172.21.15.107:0/183265954 learned_addr learned my addr 172.21.15.107:0/183265954 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:22:30.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.236+0000 7fdb1b7fe640 1 --2- 172.21.15.107:0/183265954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da1ca4df50 0x55da1ca52da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.237+0000 7fdb1affd640 1 -- 172.21.15.107:0/183265954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da1ca4d630 msgr2=0x55da1ca52860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:30.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.237+0000 7fdb1affd640 1 --2- 172.21.15.107:0/183265954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da1ca4d630 0x55da1ca52860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.237+0000 7fdb1affd640 1 -- 172.21.15.107:0/183265954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da1ca4df50 msgr2=0x55da1ca52da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:30.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.237+0000 7fdb1affd640 1 --2- 172.21.15.107:0/183265954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da1ca4df50 0x55da1ca52da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.237+0000 7fdb1affd640 1 -- 172.21.15.107:0/183265954 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da1cace490 con 0x55da1ca58620 2024-04-25T00:22:30.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.237+0000 7fdb1b7fe640 1 --2- 172.21.15.107:0/183265954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da1ca4df50 0x55da1ca52da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:30.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.237+0000 7fdb1affd640 1 --2- 172.21.15.107:0/183265954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca58620 0x55da1cacdf50 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fdb10014de0 tx=0x7fdb100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963e766de4aaeb74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:30.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.237+0000 7fdafaffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:22:30.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.237+0000 7fdafaffd640 1 -- 172.21.15.107:0/183265954 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb10005220 con 0x55da1ca58620 2024-04-25T00:22:30.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.237+0000 7fdafaffd640 1 -- 172.21.15.107:0/183265954 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fdb100053c0 con 0x55da1ca58620 2024-04-25T00:22:30.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.237+0000 7fdb24c39740 1 -- 172.21.15.107:0/183265954 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb0c01a610 con 0x55da1ca58620 2024-04-25T00:22:30.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.237+0000 7fdafaffd640 1 -- 172.21.15.107:0/183265954 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdb100055c0 con 0x55da1ca58620 2024-04-25T00:22:30.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.237+0000 7fdb24c39740 10 client.16974 fetch_fsmap learned FSMap version 34 2024-04-25T00:22:30.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.237+0000 7fdb24c39740 1 -- 172.21.15.107:0/183265954 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55da1cacf670 con 0x55da1ca58620 2024-04-25T00:22:30.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.237+0000 7fdafaffd640 1 -- 172.21.15.107:0/183265954 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb0c01a610 con 0x55da1ca58620 2024-04-25T00:22:30.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.237+0000 7fdafaffd640 1 -- 172.21.15.107:0/183265954 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fdb10011600 con 0x55da1ca58620 2024-04-25T00:22:30.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.237+0000 7fdb24c39740 10 client.16974 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:22:30.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.237+0000 7fdb24c39740 10 client.16974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:30.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.237+0000 7fdb24c39740 20 client.16974 populate_metadata read hostname 'smithi107' 2024-04-25T00:22:30.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.237+0000 7fdb24c39740 1 --2- 172.21.15.107:0/183265954 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55da1cacfb40 0x55da1cad1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:30.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.237+0000 7fdb24c39740 4 client.16974 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:22:30.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.237+0000 7fdb24c39740 1 -- 172.21.15.107:0/183265954 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55da1c9b9f40 con 0x55da1cacfb40 2024-04-25T00:22:30.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.238+0000 7fdb1b7fe640 1 --2- 172.21.15.107:0/183265954 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55da1cacfb40 0x55da1cad1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:30.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.238+0000 7fdafaffd640 10 client.16974 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:22:30.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.238+0000 7fdb1b7fe640 1 --2- 172.21.15.107:0/183265954 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55da1cacfb40 0x55da1cad1f60 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=babe926480028b4b server_cookie=58852eff3d8db5c1 in_seq=0 out_seq=0 2024-04-25T00:22:30.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.238+0000 7fdafaffd640 1 -- 172.21.15.107:0/183265954 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55da1c9b9f40 con 0x55da1cacfb40 2024-04-25T00:22:30.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.238+0000 7fdafaffd640 10 client.16974 handle_command_reply: tid=0 2024-04-25T00:22:30.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.238+0000 7fdb24c39740 1 client.16974 shutdown 2024-04-25T00:22:30.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.238+0000 7fdb24c39740 1 -- 172.21.15.107:0/183265954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca58620 msgr2=0x55da1cacdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:30.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.238+0000 7fdb24c39740 1 --2- 172.21.15.107:0/183265954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca58620 0x55da1cacdf50 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fdb10014de0 tx=0x7fdb100046c0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.238+0000 7fdb24c39740 1 -- 172.21.15.107:0/183265954 shutdown_connections 2024-04-25T00:22:30.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.238+0000 7fdb24c39740 1 --2- 172.21.15.107:0/183265954 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55da1cacfb40 0x55da1cad1f60 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.238+0000 7fdb24c39740 1 --2- 172.21.15.107:0/183265954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55da1ca58620 0x55da1cacdf50 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.238+0000 7fdb24c39740 1 --2- 172.21.15.107:0/183265954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55da1ca4df50 0x55da1ca52da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.238+0000 7fdb24c39740 1 --2- 172.21.15.107:0/183265954 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da1ca4d630 0x55da1ca52860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.238+0000 7fdafaffd640 0 client.16974 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:22:30.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.238+0000 7fdb24c39740 1 -- 172.21.15.107:0/183265954 >> 172.21.15.107:0/183265954 conn(0x55da1ca49950 msgr2=0x55da1ca48af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:30.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.238+0000 7fdb24c39740 1 -- 172.21.15.107:0/183265954 shutdown_connections 2024-04-25T00:22:30.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.238+0000 7fdb24c39740 1 -- 172.21.15.107:0/183265954 wait complete. 2024-04-25T00:22:30.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.238+0000 7fdb24c39740 20 client.16974 trim_cache size 0 max 16384 2024-04-25T00:22:30.250 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:22:30.250 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:22:30.250 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-04-25T00:22:30.250 INFO:teuthology.orchestra.run.smithi107.stdout: "dd0cc5c1-c09e-48d0-97f8-f0897b1bfe0f": { 2024-04-25T00:22:30.250 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-04-25T00:22:30.250 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "dd0cc5c1-c09e-48d0-97f8-f0897b1bfe0f", 2024-04-25T00:22:30.250 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-04-25T00:22:30.250 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:22:30.250 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:22:30.250 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:22:30.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.239+0000 7fdb21f29640 1 -- 172.21.15.107:0/1762942450 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdb040a2d00 msgr2=0x7fdb040a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:30.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.239+0000 7fdb21f29640 1 --2- 172.21.15.107:0/1762942450 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdb040a2d00 0x7fdb040a51c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdb140ee9a0 tx=0x7fdb140dd040 comp rx=0 tx=0).stop 2024-04-25T00:22:30.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.239+0000 7fdb21f29640 1 -- 172.21.15.107:0/1762942450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb1c106840 msgr2=0x7fdb1c19cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:30.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.239+0000 7fdb21f29640 1 --2- 172.21.15.107:0/1762942450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb1c106840 0x7fdb1c19cad0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fdb14005cc0 tx=0x7fdb14003970 comp rx=0 tx=0).stop 2024-04-25T00:22:30.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.239+0000 7fdb21f29640 1 -- 172.21.15.107:0/1762942450 shutdown_connections 2024-04-25T00:22:30.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.239+0000 7fdb21f29640 1 --2- 172.21.15.107:0/1762942450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb1c106840 0x7fdb1c19cad0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.239+0000 7fdb21f29640 1 --2- 172.21.15.107:0/1762942450 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb1c101710 0x7fdb1c198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.239+0000 7fdb21f29640 1 --2- 172.21.15.107:0/1762942450 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdb040a2d00 0x7fdb040a51c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.239+0000 7fdb21f29640 1 --2- 172.21.15.107:0/1762942450 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb1c100d50 0x7fdb1c198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:30.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.239+0000 7fdb21f29640 1 -- 172.21.15.107:0/1762942450 >> 172.21.15.107:0/1762942450 conn(0x7fdb1c076c00 msgr2=0x7fdb1c103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:30.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.239+0000 7fdb21f29640 1 -- 172.21.15.107:0/1762942450 shutdown_connections 2024-04-25T00:22:30.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:22:30.239+0000 7fdb21f29640 1 -- 172.21.15.107:0/1762942450 wait complete. 2024-04-25T00:22:30.253 INFO:tasks.cephfs.filesystem:scrub status for tag:dd0cc5c1-c09e-48d0-97f8-f0897b1bfe0f - {'path': '/', 'tag': 'dd0cc5c1-c09e-48d0-97f8-f0897b1bfe0f', 'options': 'recursive,force'} 2024-04-25T00:22:30.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:30 smithi107 ceph-mon[29153]: scrub queued for path: / 2024-04-25T00:22:30.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:30 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:22:30.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:30 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:22:30.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:30 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:22:31.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:30 smithi195 ceph-mon[32833]: scrub queued for path: / 2024-04-25T00:22:31.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:30 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:22:31.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:30 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:22:31.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:30 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:22:31.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:30 smithi170 ceph-mon[32807]: scrub queued for path: / 2024-04-25T00:22:31.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:30 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:22:31.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:30 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:22:31.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:30 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:22:31.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:31 smithi107 ceph-mon[29153]: pgmap v185: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:32.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:31 smithi195 ceph-mon[32833]: pgmap v185: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:32.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:31 smithi170 ceph-mon[32807]: pgmap v185: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:32.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:32 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:22:33.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:32 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:22:33.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:32 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:22:33.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:33 smithi107 ceph-mon[29153]: pgmap v186: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:22:34.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:33 smithi195 ceph-mon[32833]: pgmap v186: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:22:34.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:33 smithi170 ceph-mon[32807]: pgmap v186: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:22:35.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:35 smithi107 ceph-mon[29153]: pgmap v187: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:22:36.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:35 smithi195 ceph-mon[32833]: pgmap v187: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:22:36.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:35 smithi170 ceph-mon[32807]: pgmap v187: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:22:36.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:36 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:37.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:36 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:37.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:36 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:37.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:37 smithi107 ceph-mon[29153]: pgmap v188: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:22:37.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:37 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:37.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:37 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:37.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:37 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:22:38.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:37 smithi195 ceph-mon[32833]: pgmap v188: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:22:38.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:37 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:38.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:37 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:38.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:37 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:22:38.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:37 smithi170 ceph-mon[32807]: pgmap v188: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:22:38.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:37 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:38.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:37 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:38.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:37 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:22:39.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:39 smithi107 ceph-mon[29153]: pgmap v189: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:22:40.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:39 smithi195 ceph-mon[32833]: pgmap v189: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:22:40.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:39 smithi170 ceph-mon[32807]: pgmap v189: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:22:41.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:41 smithi107 ceph-mon[29153]: pgmap v190: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:22:42.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:41 smithi195 ceph-mon[32833]: pgmap v190: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:22:42.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:41 smithi170 ceph-mon[32807]: pgmap v190: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:22:43.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:43 smithi107 ceph-mon[29153]: pgmap v191: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:22:44.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:43 smithi195 ceph-mon[32833]: pgmap v191: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:22:44.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:43 smithi170 ceph-mon[32807]: pgmap v191: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:22:45.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:45 smithi107 ceph-mon[29153]: pgmap v192: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:45.987 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:45 smithi170 ceph-mon[32807]: pgmap v192: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:46.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:45 smithi195 ceph-mon[32833]: pgmap v192: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:47.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:47 smithi107 ceph-mon[29153]: pgmap v193: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:48.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:47 smithi195 ceph-mon[32833]: pgmap v193: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:48.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:47 smithi170 ceph-mon[32807]: pgmap v193: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:49.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:49 smithi107 ceph-mon[29153]: pgmap v194: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:22:50.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:49 smithi195 ceph-mon[32833]: pgmap v194: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:22:50.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:49 smithi170 ceph-mon[32807]: pgmap v194: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:22:52.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:51 smithi195 ceph-mon[32833]: pgmap v195: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:52.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:51 smithi170 ceph-mon[32807]: pgmap v195: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:52.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:51 smithi107 ceph-mon[29153]: pgmap v195: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:54.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:53 smithi195 ceph-mon[32833]: pgmap v196: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:22:54.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:53 smithi170 ceph-mon[32807]: pgmap v196: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:22:54.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:53 smithi107 ceph-mon[29153]: pgmap v196: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:22:56.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:55 smithi195 ceph-mon[32833]: pgmap v197: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:56.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:55 smithi170 ceph-mon[32807]: pgmap v197: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:56.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:55 smithi107 ceph-mon[29153]: pgmap v197: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:58.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:57 smithi195 ceph-mon[32833]: pgmap v198: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:58.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:57 smithi170 ceph-mon[32807]: pgmap v198: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:22:58.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:57 smithi107 ceph-mon[29153]: pgmap v198: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:23:00.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:22:59 smithi195 ceph-mon[32833]: pgmap v199: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:23:00.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:22:59 smithi170 ceph-mon[32807]: pgmap v199: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:23:00.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:22:59 smithi107 ceph-mon[29153]: pgmap v199: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:23:00.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:23:00.254 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:23:00.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.366+0000 7efd4b963640 1 -- 172.21.15.107:0/878262186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd44101710 msgr2=0x7efd44101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.366+0000 7efd4b963640 1 --2- 172.21.15.107:0/878262186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd44101710 0x7efd44101b90 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7efd34009930 tx=0x7efd3402a420 comp rx=0 tx=0).stop 2024-04-25T00:23:00.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.366+0000 7efd4b963640 1 -- 172.21.15.107:0/878262186 shutdown_connections 2024-04-25T00:23:00.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.366+0000 7efd4b963640 1 --2- 172.21.15.107:0/878262186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd441020d0 0x7efd4410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.366+0000 7efd4b963640 1 --2- 172.21.15.107:0/878262186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd44101710 0x7efd44101b90 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.366+0000 7efd4b963640 1 --2- 172.21.15.107:0/878262186 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd44107720 0x7efd44107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.366+0000 7efd4b963640 1 -- 172.21.15.107:0/878262186 >> 172.21.15.107:0/878262186 conn(0x7efd44078b90 msgr2=0x7efd44078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:00.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.367+0000 7efd4b963640 1 -- 172.21.15.107:0/878262186 shutdown_connections 2024-04-25T00:23:00.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.367+0000 7efd4b963640 1 -- 172.21.15.107:0/878262186 wait complete. 2024-04-25T00:23:00.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.367+0000 7efd4b963640 1 Processor -- start 2024-04-25T00:23:00.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.367+0000 7efd4b963640 1 -- start start 2024-04-25T00:23:00.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.367+0000 7efd4b963640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd44101710 0x7efd4419a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.367+0000 7efd4b963640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd441020d0 0x7efd4419a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.367+0000 7efd4b963640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd44107720 0x7efd4419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.367+0000 7efd4b963640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efd44111b30 con 0x7efd44101710 2024-04-25T00:23:00.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.367+0000 7efd4b963640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efd441119b0 con 0x7efd441020d0 2024-04-25T00:23:00.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.367+0000 7efd4b963640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efd44111cb0 con 0x7efd44107720 2024-04-25T00:23:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.367+0000 7efd496ff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd44101710 0x7efd4419a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.367+0000 7efd496ff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd44101710 0x7efd4419a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48372/0 (socket says 172.21.15.107:48372) 2024-04-25T00:23:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.367+0000 7efd496ff640 1 -- 172.21.15.107:0/1101731179 learned_addr learned my addr 172.21.15.107:0/1101731179 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.367+0000 7efd48efe640 1 --2- 172.21.15.107:0/1101731179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd441020d0 0x7efd4419a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.368+0000 7efd49f00640 1 --2- 172.21.15.107:0/1101731179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd44107720 0x7efd4419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.368+0000 7efd496ff640 1 -- 172.21.15.107:0/1101731179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd44107720 msgr2=0x7efd4419eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.368+0000 7efd496ff640 1 --2- 172.21.15.107:0/1101731179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd44107720 0x7efd4419eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.368+0000 7efd496ff640 1 -- 172.21.15.107:0/1101731179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd441020d0 msgr2=0x7efd4419a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.368+0000 7efd496ff640 1 --2- 172.21.15.107:0/1101731179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd441020d0 0x7efd4419a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.368+0000 7efd496ff640 1 -- 172.21.15.107:0/1101731179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd4419f240 con 0x7efd44101710 2024-04-25T00:23:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.368+0000 7efd496ff640 1 --2- 172.21.15.107:0/1101731179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd44101710 0x7efd4419a250 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7efd4000b4a0 tx=0x7efd4000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.368+0000 7efd3a7fc640 1 -- 172.21.15.107:0/1101731179 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40014070 con 0x7efd44101710 2024-04-25T00:23:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.368+0000 7efd3a7fc640 1 -- 172.21.15.107:0/1101731179 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd40010af0 con 0x7efd44101710 2024-04-25T00:23:00.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.368+0000 7efd3a7fc640 1 -- 172.21.15.107:0/1101731179 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40017d10 con 0x7efd44101710 2024-04-25T00:23:00.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.368+0000 7efd4b963640 1 -- 172.21.15.107:0/1101731179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd4419f530 con 0x7efd44101710 2024-04-25T00:23:00.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.368+0000 7efd4b963640 1 -- 172.21.15.107:0/1101731179 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd441a6dc0 con 0x7efd44101710 2024-04-25T00:23:00.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.370+0000 7efd3a7fc640 1 -- 172.21.15.107:0/1101731179 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7efd40017070 con 0x7efd44101710 2024-04-25T00:23:00.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.371+0000 7efd4e679740 1 Processor -- start 2024-04-25T00:23:00.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.371+0000 7efd4e679740 1 -- start start 2024-04-25T00:23:00.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.371+0000 7efd4e679740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715e630 0x560a0715ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.371+0000 7efd4e679740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a0715ef50 0x560a071690e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.371+0000 7efd4e679740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a07169620 0x560a0716ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.371+0000 7efd4e679740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560a070c36b0 con 0x560a0715e630 2024-04-25T00:23:00.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.371+0000 7efd3a7fc640 1 --2- 172.21.15.107:0/1101731179 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7efd300a2d00 0x7efd300a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.371+0000 7efd4e679740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560a0706b880 con 0x560a0715ef50 2024-04-25T00:23:00.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.371+0000 7efd4e679740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560a070bf670 con 0x560a07169620 2024-04-25T00:23:00.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.371+0000 7efd49f00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715e630 0x560a0715ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.371+0000 7efd49f00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715e630 0x560a0715ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48386/0 (socket says 172.21.15.107:48386) 2024-04-25T00:23:00.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.371+0000 7efd49f00640 1 -- 172.21.15.107:0/65217220 learned_addr learned my addr 172.21.15.107:0/65217220 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:00.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.371+0000 7efd49f00640 1 --2- 172.21.15.107:0/1101731179 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7efd300a2d00 0x7efd300a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.372+0000 7efd3a7fc640 1 -- 172.21.15.107:0/1101731179 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7efd400ceae0 con 0x7efd44101710 2024-04-25T00:23:00.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.372+0000 7efd48efe640 1 --2- 172.21.15.107:0/65217220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a07169620 0x560a0716ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.372+0000 7efd496ff640 1 --2- 172.21.15.107:0/65217220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a0715ef50 0x560a071690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.372+0000 7efd49f00640 1 -- 172.21.15.107:0/65217220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a07169620 msgr2=0x560a0716ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:00.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.372+0000 7efd49f00640 1 --2- 172.21.15.107:0/65217220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a07169620 0x560a0716ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.372+0000 7efd49f00640 1 -- 172.21.15.107:0/65217220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a0715ef50 msgr2=0x560a071690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:00.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.372+0000 7efd49f00640 1 --2- 172.21.15.107:0/65217220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a0715ef50 0x560a071690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.372+0000 7efd49f00640 1 -- 172.21.15.107:0/65217220 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a06ea17d0 con 0x560a0715e630 2024-04-25T00:23:00.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.372+0000 7efd48efe640 1 --2- 172.21.15.107:0/65217220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a07169620 0x560a0716ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:00.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.372+0000 7efd496ff640 1 --2- 172.21.15.107:0/65217220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a0715ef50 0x560a071690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:00.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.372+0000 7efd49f00640 1 --2- 172.21.15.107:0/65217220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715e630 0x560a0715ea10 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7efd3c013e60 tx=0x7efd3c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db7ca1a438d724d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:00.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.372+0000 7efd49f00640 1 --2- 172.21.15.107:0/1101731179 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7efd300a2d00 0x7efd300a51c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efd3c010ca0 tx=0x7efd3c015000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:00.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.372+0000 7efd2f7fe640 1 -- 172.21.15.107:0/65217220 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c00ae10 con 0x560a0715e630 2024-04-25T00:23:00.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.372+0000 7efd2f7fe640 1 -- 172.21.15.107:0/65217220 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd3c007be0 con 0x560a0715e630 2024-04-25T00:23:00.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.372+0000 7efd2f7fe640 1 -- 172.21.15.107:0/65217220 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c005900 con 0x560a0715e630 2024-04-25T00:23:00.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.373+0000 7efd4e679740 1 -- 172.21.15.107:0/65217220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715e630 msgr2=0x560a0715ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.373+0000 7efd4e679740 1 --2- 172.21.15.107:0/65217220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715e630 0x560a0715ea10 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7efd3c013e60 tx=0x7efd3c0107a0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.373+0000 7efd4e679740 1 -- 172.21.15.107:0/65217220 shutdown_connections 2024-04-25T00:23:00.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.373+0000 7efd4e679740 1 --2- 172.21.15.107:0/65217220 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a07169620 0x560a0716ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.373+0000 7efd4e679740 1 --2- 172.21.15.107:0/65217220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a0715ef50 0x560a071690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.373+0000 7efd4e679740 1 --2- 172.21.15.107:0/65217220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715e630 0x560a0715ea10 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.373+0000 7efd4e679740 1 -- 172.21.15.107:0/65217220 >> 172.21.15.107:0/65217220 conn(0x560a0715a950 msgr2=0x560a0715ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:00.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.373+0000 7efd4e679740 1 -- 172.21.15.107:0/65217220 shutdown_connections 2024-04-25T00:23:00.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.373+0000 7efd4e679740 1 -- 172.21.15.107:0/65217220 wait complete. 2024-04-25T00:23:00.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.373+0000 7efd4e679740 1 -- start start 2024-04-25T00:23:00.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.373+0000 7efd4e679740 1 Processor -- start 2024-04-25T00:23:00.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.374+0000 7efd4e679740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a0715e630 0x560a071665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.374+0000 7efd4e679740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715ef50 0x560a07166ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.374+0000 7efd4e679740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a07169620 0x560a07167020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.374+0000 7efd4e679740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560a0716c3b0 con 0x560a0715ef50 2024-04-25T00:23:00.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.374+0000 7efd4e679740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560a0707ef50 con 0x560a0715e630 2024-04-25T00:23:00.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.374+0000 7efd4e679740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560a0716c530 con 0x560a07169620 2024-04-25T00:23:00.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.374+0000 7efd4e679740 10 client.0 _ll_register_callbacks cb 0x560a070ba000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:00.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.374+0000 7efd48efe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715ef50 0x560a07166ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.374+0000 7efd48efe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715ef50 0x560a07166ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48392/0 (socket says 172.21.15.107:48392) 2024-04-25T00:23:00.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.374+0000 7efd48efe640 1 -- 172.21.15.107:0/4192510539 learned_addr learned my addr 172.21.15.107:0/4192510539 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:00.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.374+0000 7efd496ff640 1 --2- 172.21.15.107:0/4192510539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a0715e630 0x560a071665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.374+0000 7efd48efe640 1 -- 172.21.15.107:0/4192510539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a07169620 msgr2=0x560a07167020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:00.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.374+0000 7efd48efe640 1 --2- 172.21.15.107:0/4192510539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a07169620 0x560a07167020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.374+0000 7efd48efe640 1 -- 172.21.15.107:0/4192510539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a0715e630 msgr2=0x560a071665a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:00.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.374+0000 7efd48efe640 1 --2- 172.21.15.107:0/4192510539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a0715e630 0x560a071665a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.374+0000 7efd48efe640 1 -- 172.21.15.107:0/4192510539 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a06dd73f0 con 0x560a0715ef50 2024-04-25T00:23:00.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.374+0000 7efd48efe640 1 --2- 172.21.15.107:0/4192510539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715ef50 0x560a07166ae0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7efd34000c00 tx=0x7efd340319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe67feafc16785f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:00.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.374+0000 7efd2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:00.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.375+0000 7efd2cff9640 1 -- 172.21.15.107:0/4192510539 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd34035040 con 0x560a0715ef50 2024-04-25T00:23:00.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.375+0000 7efd2cff9640 1 -- 172.21.15.107:0/4192510539 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7efd34004300 con 0x560a0715ef50 2024-04-25T00:23:00.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.375+0000 7efd4e679740 1 -- 172.21.15.107:0/4192510539 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd3c007be0 con 0x560a0715ef50 2024-04-25T00:23:00.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.375+0000 7efd2cff9640 1 -- 172.21.15.107:0/4192510539 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efd3404e050 con 0x560a0715ef50 2024-04-25T00:23:00.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.375+0000 7efd4e679740 10 client.16986 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:00.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.375+0000 7efd4e679740 1 -- 172.21.15.107:0/4192510539 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560a06d970f0 con 0x560a0715ef50 2024-04-25T00:23:00.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.375+0000 7efd2cff9640 1 -- 172.21.15.107:0/4192510539 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd3c007be0 con 0x560a0715ef50 2024-04-25T00:23:00.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.375+0000 7efd2cff9640 1 -- 172.21.15.107:0/4192510539 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7efd34045720 con 0x560a0715ef50 2024-04-25T00:23:00.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.375+0000 7efd4e679740 10 client.16986 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:00.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.375+0000 7efd4e679740 10 client.16986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:00.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.375+0000 7efd4e679740 20 client.16986 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:00.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.375+0000 7efd4e679740 1 --2- 172.21.15.107:0/4192510539 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x560a07179880 0x560a0717bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.375+0000 7efd4e679740 4 client.16986 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:00.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.375+0000 7efd4e679740 1 -- 172.21.15.107:0/4192510539 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a070d0bf0 con 0x560a07179880 2024-04-25T00:23:00.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.375+0000 7efd496ff640 1 --2- 172.21.15.107:0/4192510539 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x560a07179880 0x560a0717bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.376+0000 7efd2cff9640 10 client.16986 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:00.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.376+0000 7efd496ff640 1 --2- 172.21.15.107:0/4192510539 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x560a07179880 0x560a0717bd20 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f991d199c2729fe server_cookie=f16f437239b09ebd in_seq=0 out_seq=0 2024-04-25T00:23:00.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.376+0000 7efd2cff9640 1 -- 172.21.15.107:0/4192510539 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a070d0bf0 con 0x560a07179880 2024-04-25T00:23:00.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.376+0000 7efd2cff9640 10 client.16986 handle_command_reply: tid=0 2024-04-25T00:23:00.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.377+0000 7efd4e679740 1 client.16986 shutdown 2024-04-25T00:23:00.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.377+0000 7efd4e679740 1 -- 172.21.15.107:0/4192510539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715ef50 msgr2=0x560a07166ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.377+0000 7efd4e679740 1 --2- 172.21.15.107:0/4192510539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715ef50 0x560a07166ae0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7efd34000c00 tx=0x7efd340319a0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.377+0000 7efd4e679740 1 -- 172.21.15.107:0/4192510539 shutdown_connections 2024-04-25T00:23:00.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.377+0000 7efd4e679740 1 --2- 172.21.15.107:0/4192510539 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x560a07179880 0x560a0717bd20 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.377+0000 7efd4e679740 1 --2- 172.21.15.107:0/4192510539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a07169620 0x560a07167020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.377+0000 7efd4e679740 1 --2- 172.21.15.107:0/4192510539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715ef50 0x560a07166ae0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.377+0000 7efd2cff9640 0 client.16986 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:00.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.377+0000 7efd4e679740 1 --2- 172.21.15.107:0/4192510539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a0715e630 0x560a071665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.377+0000 7efd4e679740 1 -- 172.21.15.107:0/4192510539 >> 172.21.15.107:0/4192510539 conn(0x560a0715a950 msgr2=0x560a07160650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:00.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.377+0000 7efd4e679740 1 -- 172.21.15.107:0/4192510539 shutdown_connections 2024-04-25T00:23:00.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.377+0000 7efd4e679740 1 -- 172.21.15.107:0/4192510539 wait complete. 2024-04-25T00:23:00.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.377+0000 7efd4e679740 20 client.16986 trim_cache size 0 max 16384 2024-04-25T00:23:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.394+0000 7efd4e679740 1 Processor -- start 2024-04-25T00:23:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.394+0000 7efd4e679740 1 -- start start 2024-04-25T00:23:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.394+0000 7efd4e679740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715a950 0x560a0715a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.394+0000 7efd4e679740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a0715e630 0x560a07175270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.394+0000 7efd4e679740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a0715ef50 0x560a071815f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.394+0000 7efd4e679740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efd441b3620 con 0x560a0715a950 2024-04-25T00:23:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.394+0000 7efd4e679740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efd441b0ef0 con 0x560a0715ef50 2024-04-25T00:23:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.394+0000 7efd4e679740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efd441aed40 con 0x560a0715e630 2024-04-25T00:23:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.394+0000 7efd496ff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715a950 0x560a0715a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.395+0000 7efd496ff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715a950 0x560a0715a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48396/0 (socket says 172.21.15.107:48396) 2024-04-25T00:23:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.395+0000 7efd496ff640 1 -- 172.21.15.107:0/2921332504 learned_addr learned my addr 172.21.15.107:0/2921332504 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.395+0000 7efd49f00640 1 --2- 172.21.15.107:0/2921332504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a0715ef50 0x560a071815f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.395+0000 7efd48efe640 1 --2- 172.21.15.107:0/2921332504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a0715e630 0x560a07175270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.395+0000 7efd496ff640 1 -- 172.21.15.107:0/2921332504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a0715e630 msgr2=0x560a07175270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.395+0000 7efd496ff640 1 --2- 172.21.15.107:0/2921332504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a0715e630 0x560a07175270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.395+0000 7efd496ff640 1 -- 172.21.15.107:0/2921332504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a0715ef50 msgr2=0x560a071815f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.395+0000 7efd496ff640 1 --2- 172.21.15.107:0/2921332504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a0715ef50 0x560a071815f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.395+0000 7efd496ff640 1 -- 172.21.15.107:0/2921332504 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a0714c090 con 0x560a0715a950 2024-04-25T00:23:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.395+0000 7efd48efe640 1 --2- 172.21.15.107:0/2921332504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a0715e630 0x560a07175270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.395+0000 7efd496ff640 1 --2- 172.21.15.107:0/2921332504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715a950 0x560a0715a270 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7efd400971a0 tx=0x7efd400ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71725896ce649d43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.395+0000 7efd2effd640 1 -- 172.21.15.107:0/2921332504 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4001e040 con 0x560a0715a950 2024-04-25T00:23:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.395+0000 7efd2effd640 1 -- 172.21.15.107:0/2921332504 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd400a39b0 con 0x560a0715a950 2024-04-25T00:23:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.395+0000 7efd2effd640 1 -- 172.21.15.107:0/2921332504 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd40099e10 con 0x560a0715a950 2024-04-25T00:23:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.396+0000 7efd4e679740 1 -- 172.21.15.107:0/2921332504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715a950 msgr2=0x560a0715a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.396+0000 7efd4e679740 1 --2- 172.21.15.107:0/2921332504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715a950 0x560a0715a270 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7efd400971a0 tx=0x7efd400ce040 comp rx=0 tx=0).stop 2024-04-25T00:23:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.396+0000 7efd4e679740 1 -- 172.21.15.107:0/2921332504 shutdown_connections 2024-04-25T00:23:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.396+0000 7efd4e679740 1 --2- 172.21.15.107:0/2921332504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a0715ef50 0x560a071815f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.396+0000 7efd4e679740 1 --2- 172.21.15.107:0/2921332504 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a0715e630 0x560a07175270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.396+0000 7efd4e679740 1 --2- 172.21.15.107:0/2921332504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715a950 0x560a0715a270 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.396+0000 7efd4e679740 1 -- 172.21.15.107:0/2921332504 >> 172.21.15.107:0/2921332504 conn(0x560a07165200 msgr2=0x560a07166df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:00.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.396+0000 7efd4e679740 1 -- 172.21.15.107:0/2921332504 shutdown_connections 2024-04-25T00:23:00.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.396+0000 7efd4e679740 1 -- 172.21.15.107:0/2921332504 wait complete. 2024-04-25T00:23:00.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.396+0000 7efd4e679740 1 -- start start 2024-04-25T00:23:00.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.396+0000 7efd4e679740 1 Processor -- start 2024-04-25T00:23:00.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.396+0000 7efd4e679740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715a950 0x560a07171780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.396+0000 7efd4e679740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a0715e630 0x560a07171cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.396+0000 7efd4e679740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a0715ef50 0x560a0716e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.396+0000 7efd4e679740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efd441af3b0 con 0x560a0715a950 2024-04-25T00:23:00.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.396+0000 7efd4e679740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efd441b0820 con 0x560a0715ef50 2024-04-25T00:23:00.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.396+0000 7efd4e679740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7efd441afa80 con 0x560a0715e630 2024-04-25T00:23:00.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.396+0000 7efd4e679740 10 client.0 _ll_register_callbacks cb 0x7efd34005d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:00.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.397+0000 7efd496ff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715a950 0x560a07171780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.397+0000 7efd496ff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715a950 0x560a07171780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48402/0 (socket says 172.21.15.107:48402) 2024-04-25T00:23:00.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.397+0000 7efd496ff640 1 -- 172.21.15.107:0/3510074789 learned_addr learned my addr 172.21.15.107:0/3510074789 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:00.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.397+0000 7efd48efe640 1 --2- 172.21.15.107:0/3510074789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a0715e630 0x560a07171cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.397+0000 7efd49f00640 1 --2- 172.21.15.107:0/3510074789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a0715ef50 0x560a0716e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.397+0000 7efd496ff640 1 -- 172.21.15.107:0/3510074789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a0715e630 msgr2=0x560a07171cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:00.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.397+0000 7efd496ff640 1 --2- 172.21.15.107:0/3510074789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a0715e630 0x560a07171cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.397+0000 7efd496ff640 1 -- 172.21.15.107:0/3510074789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a0715ef50 msgr2=0x560a0716e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:00.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.397+0000 7efd496ff640 1 --2- 172.21.15.107:0/3510074789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a0715ef50 0x560a0716e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.397+0000 7efd496ff640 1 -- 172.21.15.107:0/3510074789 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a0716ee40 con 0x560a0715a950 2024-04-25T00:23:00.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.397+0000 7efd496ff640 1 --2- 172.21.15.107:0/3510074789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715a950 0x560a07171780 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7efd40012010 tx=0x7efd400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7738058ea10b4939 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:00.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.397+0000 7efd2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:00.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.397+0000 7efd2cff9640 1 -- 172.21.15.107:0/3510074789 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd4007d070 con 0x560a0715a950 2024-04-25T00:23:00.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.397+0000 7efd2cff9640 1 -- 172.21.15.107:0/3510074789 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7efd40099040 con 0x560a0715a950 2024-04-25T00:23:00.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.397+0000 7efd4e679740 1 -- 172.21.15.107:0/3510074789 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd400a39b0 con 0x560a0715a950 2024-04-25T00:23:00.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.397+0000 7efd2cff9640 1 -- 172.21.15.107:0/3510074789 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efd400b7030 con 0x560a0715a950 2024-04-25T00:23:00.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.397+0000 7efd4e679740 10 client.16992 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:00.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.397+0000 7efd4e679740 1 -- 172.21.15.107:0/3510074789 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560a071e07d0 con 0x560a0715a950 2024-04-25T00:23:00.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.397+0000 7efd2cff9640 1 -- 172.21.15.107:0/3510074789 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd400a39b0 con 0x560a0715a950 2024-04-25T00:23:00.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.398+0000 7efd2cff9640 1 -- 172.21.15.107:0/3510074789 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7efd40099e10 con 0x560a0715a950 2024-04-25T00:23:00.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.398+0000 7efd4e679740 10 client.16992 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:00.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.398+0000 7efd4e679740 10 client.16992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:00.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.398+0000 7efd4e679740 20 client.16992 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:00.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.398+0000 7efd4e679740 1 --2- 172.21.15.107:0/3510074789 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x560a07169620 0x560a071e2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.398+0000 7efd4e679740 4 client.16992 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:23:00.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.398+0000 7efd4e679740 1 -- 172.21.15.107:0/3510074789 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560a070caf40 con 0x560a07169620 2024-04-25T00:23:00.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.398+0000 7efd48efe640 1 --2- 172.21.15.107:0/3510074789 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x560a07169620 0x560a071e2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.398+0000 7efd2cff9640 10 client.16992 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:00.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.398+0000 7efd48efe640 1 --2- 172.21.15.107:0/3510074789 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x560a07169620 0x560a071e2ea0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9133b29d6d5a9a83 server_cookie=c3b11c1432932fc in_seq=0 out_seq=0 2024-04-25T00:23:00.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.398+0000 7efd2cff9640 1 -- 172.21.15.107:0/3510074789 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560a070caf40 con 0x560a07169620 2024-04-25T00:23:00.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.398+0000 7efd2cff9640 10 client.16992 handle_command_reply: tid=0 2024-04-25T00:23:00.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.398+0000 7efd4e679740 1 client.16992 shutdown 2024-04-25T00:23:00.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.399+0000 7efd4e679740 1 -- 172.21.15.107:0/3510074789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715a950 msgr2=0x560a07171780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.399+0000 7efd4e679740 1 --2- 172.21.15.107:0/3510074789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715a950 0x560a07171780 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7efd40012010 tx=0x7efd400046c0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.399+0000 7efd4e679740 1 -- 172.21.15.107:0/3510074789 shutdown_connections 2024-04-25T00:23:00.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.399+0000 7efd4e679740 1 --2- 172.21.15.107:0/3510074789 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x560a07169620 0x560a071e2ea0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.399+0000 7efd4e679740 1 --2- 172.21.15.107:0/3510074789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560a0715ef50 0x560a0716e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.399+0000 7efd4e679740 1 --2- 172.21.15.107:0/3510074789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560a0715e630 0x560a07171cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.399+0000 7efd2cff9640 0 client.16992 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:00.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.399+0000 7efd4e679740 1 --2- 172.21.15.107:0/3510074789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560a0715a950 0x560a07171780 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.399+0000 7efd4e679740 1 -- 172.21.15.107:0/3510074789 >> 172.21.15.107:0/3510074789 conn(0x560a07165200 msgr2=0x560a0715fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:00.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.399+0000 7efd4e679740 1 -- 172.21.15.107:0/3510074789 shutdown_connections 2024-04-25T00:23:00.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.399+0000 7efd4e679740 1 -- 172.21.15.107:0/3510074789 wait complete. 2024-04-25T00:23:00.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.399+0000 7efd4e679740 20 client.16992 trim_cache size 0 max 16384 2024-04-25T00:23:00.407 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:23:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-04-25T00:23:00.407 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-04-25T00:23:00.407 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:23:00.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.399+0000 7efd4b963640 1 -- 172.21.15.107:0/1101731179 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7efd300a2d00 msgr2=0x7efd300a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.399+0000 7efd4b963640 1 --2- 172.21.15.107:0/1101731179 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7efd300a2d00 0x7efd300a51c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efd3c010ca0 tx=0x7efd3c015000 comp rx=0 tx=0).stop 2024-04-25T00:23:00.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.399+0000 7efd4b963640 1 -- 172.21.15.107:0/1101731179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd44101710 msgr2=0x7efd4419a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.399+0000 7efd4b963640 1 --2- 172.21.15.107:0/1101731179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd44101710 0x7efd4419a250 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7efd4000b4a0 tx=0x7efd4000b960 comp rx=0 tx=0).stop 2024-04-25T00:23:00.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.399+0000 7efd4b963640 1 -- 172.21.15.107:0/1101731179 shutdown_connections 2024-04-25T00:23:00.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.399+0000 7efd4b963640 1 --2- 172.21.15.107:0/1101731179 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efd44107720 0x7efd4419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.399+0000 7efd4b963640 1 --2- 172.21.15.107:0/1101731179 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7efd300a2d00 0x7efd300a51c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.399+0000 7efd4b963640 1 --2- 172.21.15.107:0/1101731179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd441020d0 0x7efd4419a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.399+0000 7efd4b963640 1 --2- 172.21.15.107:0/1101731179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd44101710 0x7efd4419a250 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.399+0000 7efd4b963640 1 -- 172.21.15.107:0/1101731179 >> 172.21.15.107:0/1101731179 conn(0x7efd44078b90 msgr2=0x7efd4410ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:00.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.399+0000 7efd4b963640 1 -- 172.21.15.107:0/1101731179 shutdown_connections 2024-04-25T00:23:00.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.399+0000 7efd4b963640 1 -- 172.21.15.107:0/1101731179 wait complete. 2024-04-25T00:23:00.410 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:23:00.411 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:23:00.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.521+0000 7fa91bb5c640 1 -- 172.21.15.107:0/4092678861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa914106630 msgr2=0x7fa914106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.521+0000 7fa91bb5c640 1 --2- 172.21.15.107:0/4092678861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa914106630 0x7fa914106a10 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fa910009cd0 tx=0x7fa91002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.521+0000 7fa91bb5c640 1 -- 172.21.15.107:0/4092678861 shutdown_connections 2024-04-25T00:23:00.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.521+0000 7fa91bb5c640 1 --2- 172.21.15.107:0/4092678861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa914100fe0 0x7fa91410d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.521+0000 7fa91bb5c640 1 --2- 172.21.15.107:0/4092678861 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa914100620 0x7fa914100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.521+0000 7fa91bb5c640 1 --2- 172.21.15.107:0/4092678861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa914106630 0x7fa914106a10 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.521+0000 7fa91bb5c640 1 -- 172.21.15.107:0/4092678861 >> 172.21.15.107:0/4092678861 conn(0x7fa914077740 msgr2=0x7fa914077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:00.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.521+0000 7fa91bb5c640 1 -- 172.21.15.107:0/4092678861 shutdown_connections 2024-04-25T00:23:00.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.521+0000 7fa91bb5c640 1 -- 172.21.15.107:0/4092678861 wait complete. 2024-04-25T00:23:00.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.522+0000 7fa91bb5c640 1 Processor -- start 2024-04-25T00:23:00.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.522+0000 7fa91bb5c640 1 -- start start 2024-04-25T00:23:00.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.522+0000 7fa91bb5c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa914100620 0x7fa914195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.522+0000 7fa91bb5c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa914100fe0 0x7fa914196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.522+0000 7fa91bb5c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa914106630 0x7fa91419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.522+0000 7fa91bb5c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa91410f930 con 0x7fa914106630 2024-04-25T00:23:00.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.522+0000 7fa91bb5c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa91410f7b0 con 0x7fa914100620 2024-04-25T00:23:00.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.522+0000 7fa91bb5c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa91410fab0 con 0x7fa914100fe0 2024-04-25T00:23:00.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.522+0000 7fa91a0f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa914106630 0x7fa91419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.522+0000 7fa9190f7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa914100fe0 0x7fa914196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.522+0000 7fa91a0f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa914106630 0x7fa91419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48432/0 (socket says 172.21.15.107:48432) 2024-04-25T00:23:00.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.522+0000 7fa91a0f9640 1 -- 172.21.15.107:0/2828282045 learned_addr learned my addr 172.21.15.107:0/2828282045 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:00.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.522+0000 7fa9198f8640 1 --2- 172.21.15.107:0/2828282045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa914100620 0x7fa914195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.522+0000 7fa91a0f9640 1 -- 172.21.15.107:0/2828282045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa914100fe0 msgr2=0x7fa914196330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.522+0000 7fa91a0f9640 1 --2- 172.21.15.107:0/2828282045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa914100fe0 0x7fa914196330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.522+0000 7fa91a0f9640 1 -- 172.21.15.107:0/2828282045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa914100620 msgr2=0x7fa914195df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.522+0000 7fa91a0f9640 1 --2- 172.21.15.107:0/2828282045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa914100620 0x7fa914195df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.522+0000 7fa91a0f9640 1 -- 172.21.15.107:0/2828282045 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91419ade0 con 0x7fa914106630 2024-04-25T00:23:00.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.522+0000 7fa91a0f9640 1 --2- 172.21.15.107:0/2828282045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa914106630 0x7fa91419a6b0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00dd50 tx=0x7fa90c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:00.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.522+0000 7fa90affd640 1 -- 172.21.15.107:0/2828282045 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c005020 con 0x7fa914106630 2024-04-25T00:23:00.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.523+0000 7fa90affd640 1 -- 172.21.15.107:0/2828282045 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa90c0056a0 con 0x7fa914106630 2024-04-25T00:23:00.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.523+0000 7fa91bb5c640 1 -- 172.21.15.107:0/2828282045 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91419b0d0 con 0x7fa914106630 2024-04-25T00:23:00.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.523+0000 7fa90affd640 1 -- 172.21.15.107:0/2828282045 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c009970 con 0x7fa914106630 2024-04-25T00:23:00.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.523+0000 7fa91bb5c640 1 -- 172.21.15.107:0/2828282045 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9141a2960 con 0x7fa914106630 2024-04-25T00:23:00.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.523+0000 7fa91bb5c640 1 -- 172.21.15.107:0/2828282045 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa914068560 con 0x7fa914106630 2024-04-25T00:23:00.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.524+0000 7fa90affd640 1 -- 172.21.15.107:0/2828282045 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa90c01c020 con 0x7fa914106630 2024-04-25T00:23:00.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.525+0000 7fa90affd640 1 --2- 172.21.15.107:0/2828282045 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa9000a2d00 0x7fa9000a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.525+0000 7fa9198f8640 1 --2- 172.21.15.107:0/2828282045 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa9000a2d00 0x7fa9000a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.525+0000 7fa90affd640 1 -- 172.21.15.107:0/2828282045 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fa90c0ced50 con 0x7fa914106630 2024-04-25T00:23:00.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.525+0000 7fa9198f8640 1 --2- 172.21.15.107:0/2828282045 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa9000a2d00 0x7fa9000a51c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa910006010 tx=0x7fa910009360 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:00.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.529+0000 7fa90affd640 1 -- 172.21.15.107:0/2828282045 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa90c0d2030 con 0x7fa914106630 2024-04-25T00:23:00.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.742+0000 7fa91bb5c640 1 -- 172.21.15.107:0/2828282045 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa914108600 con 0x7fa914106630 2024-04-25T00:23:00.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.743+0000 7fa90affd640 1 -- 172.21.15.107:0/2828282045 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11897 (secure 0 0 0) 0x7fa90c09b600 con 0x7fa914106630 2024-04-25T00:23:00.744 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:23:00.744 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:22:27.402680+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24493,24521]},"id":1}]} 2024-04-25T00:23:00.744 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 34 2024-04-25T00:23:00.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.745+0000 7fa91bb5c640 1 -- 172.21.15.107:0/2828282045 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa9000a2d00 msgr2=0x7fa9000a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.745+0000 7fa91bb5c640 1 --2- 172.21.15.107:0/2828282045 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa9000a2d00 0x7fa9000a51c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa910006010 tx=0x7fa910009360 comp rx=0 tx=0).stop 2024-04-25T00:23:00.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.745+0000 7fa91bb5c640 1 -- 172.21.15.107:0/2828282045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa914106630 msgr2=0x7fa91419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.745+0000 7fa91bb5c640 1 --2- 172.21.15.107:0/2828282045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa914106630 0x7fa91419a6b0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00dd50 tx=0x7fa90c0039e0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.745+0000 7fa91bb5c640 1 -- 172.21.15.107:0/2828282045 shutdown_connections 2024-04-25T00:23:00.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.745+0000 7fa91bb5c640 1 --2- 172.21.15.107:0/2828282045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa914106630 0x7fa91419a6b0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.745+0000 7fa91bb5c640 1 --2- 172.21.15.107:0/2828282045 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa914100fe0 0x7fa914196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.745+0000 7fa91bb5c640 1 --2- 172.21.15.107:0/2828282045 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa9000a2d00 0x7fa9000a51c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.745+0000 7fa91bb5c640 1 --2- 172.21.15.107:0/2828282045 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa914100620 0x7fa914195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.745+0000 7fa91bb5c640 1 -- 172.21.15.107:0/2828282045 >> 172.21.15.107:0/2828282045 conn(0x7fa914077740 msgr2=0x7fa914102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:00.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.746+0000 7fa91bb5c640 1 -- 172.21.15.107:0/2828282045 shutdown_connections 2024-04-25T00:23:00.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.746+0000 7fa91bb5c640 1 -- 172.21.15.107:0/2828282045 wait complete. 2024-04-25T00:23:00.757 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:23:00.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.856+0000 7f0c32916640 1 -- 172.21.15.107:0/1350378145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c2c1029c0 msgr2=0x7f0c2c102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.856+0000 7f0c32916640 1 --2- 172.21.15.107:0/1350378145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c2c1029c0 0x7f0c2c102da0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009900 tx=0x7f0c1c02a510 comp rx=0 tx=0).stop 2024-04-25T00:23:00.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.856+0000 7f0c32916640 1 -- 172.21.15.107:0/1350378145 shutdown_connections 2024-04-25T00:23:00.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.856+0000 7f0c32916640 1 --2- 172.21.15.107:0/1350378145 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c2c10bd00 0x7f0c2c10e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.856+0000 7f0c32916640 1 --2- 172.21.15.107:0/1350378145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c2c1032e0 0x7f0c2c10b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.856+0000 7f0c32916640 1 --2- 172.21.15.107:0/1350378145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c2c1029c0 0x7f0c2c102da0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.856+0000 7f0c32916640 1 -- 172.21.15.107:0/1350378145 >> 172.21.15.107:0/1350378145 conn(0x7f0c2c077740 msgr2=0x7f0c2c077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:00.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.856+0000 7f0c32916640 1 -- 172.21.15.107:0/1350378145 shutdown_connections 2024-04-25T00:23:00.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.856+0000 7f0c32916640 1 -- 172.21.15.107:0/1350378145 wait complete. 2024-04-25T00:23:00.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.857+0000 7f0c32916640 1 Processor -- start 2024-04-25T00:23:00.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.857+0000 7f0c32916640 1 -- start start 2024-04-25T00:23:00.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.857+0000 7f0c32916640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c2c1029c0 0x7f0c2c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.857+0000 7f0c32916640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c2c1032e0 0x7f0c2c1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.857+0000 7f0c32916640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c2c10bd00 0x7f0c2c19c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.857+0000 7f0c32916640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c111b90 con 0x7f0c2c1029c0 2024-04-25T00:23:00.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.857+0000 7f0c32916640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c111a10 con 0x7f0c2c1032e0 2024-04-25T00:23:00.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.857+0000 7f0c32916640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c111d10 con 0x7f0c2c10bd00 2024-04-25T00:23:00.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.858+0000 7f0c2bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c2c1029c0 0x7f0c2c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.858+0000 7f0c2bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c2c1029c0 0x7f0c2c198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48438/0 (socket says 172.21.15.107:48438) 2024-04-25T00:23:00.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.858+0000 7f0c2bfff640 1 -- 172.21.15.107:0/675766614 learned_addr learned my addr 172.21.15.107:0/675766614 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:00.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.858+0000 7f0c30eb3640 1 --2- 172.21.15.107:0/675766614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c2c10bd00 0x7f0c2c19c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.858+0000 7f0c2b7fe640 1 --2- 172.21.15.107:0/675766614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c2c1032e0 0x7f0c2c1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.858+0000 7f0c2bfff640 1 -- 172.21.15.107:0/675766614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c2c10bd00 msgr2=0x7f0c2c19c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.858+0000 7f0c2bfff640 1 --2- 172.21.15.107:0/675766614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c2c10bd00 0x7f0c2c19c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.858+0000 7f0c2bfff640 1 -- 172.21.15.107:0/675766614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c2c1032e0 msgr2=0x7f0c2c1985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.858+0000 7f0c2bfff640 1 --2- 172.21.15.107:0/675766614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c2c1032e0 0x7f0c2c1985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.858+0000 7f0c2bfff640 1 -- 172.21.15.107:0/675766614 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2c19d080 con 0x7f0c2c1029c0 2024-04-25T00:23:00.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.858+0000 7f0c2bfff640 1 --2- 172.21.15.107:0/675766614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c2c1029c0 0x7f0c2c198090 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c004770 tx=0x7f0c1c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:00.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.858+0000 7f0c297fa640 1 -- 172.21.15.107:0/675766614 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c03f070 con 0x7f0c2c1029c0 2024-04-25T00:23:00.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.858+0000 7f0c297fa640 1 -- 172.21.15.107:0/675766614 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c1c031ca0 con 0x7f0c2c1029c0 2024-04-25T00:23:00.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.858+0000 7f0c32916640 1 -- 172.21.15.107:0/675766614 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c2c19d310 con 0x7f0c2c1029c0 2024-04-25T00:23:00.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.858+0000 7f0c297fa640 1 -- 172.21.15.107:0/675766614 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c035030 con 0x7f0c2c1029c0 2024-04-25T00:23:00.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.858+0000 7f0c32916640 1 -- 172.21.15.107:0/675766614 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c2c1a4c00 con 0x7f0c2c1029c0 2024-04-25T00:23:00.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.860+0000 7f0c297fa640 1 -- 172.21.15.107:0/675766614 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0c1c042070 con 0x7f0c2c1029c0 2024-04-25T00:23:00.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.861+0000 7f0c35619740 1 Processor -- start 2024-04-25T00:23:00.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.861+0000 7f0c35619740 1 -- start start 2024-04-25T00:23:00.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.861+0000 7f0c35619740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa54cd1580 0x55fa54cd1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.861+0000 7f0c35619740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa54cd1ea0 0x55fa54cdc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.861+0000 7f0c35619740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cdc570 0x55fa54cde960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.861+0000 7f0c35619740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fa54c36670 con 0x55fa54cdc570 2024-04-25T00:23:00.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.861+0000 7f0c35619740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55fa54bde970 con 0x55fa54cd1ea0 2024-04-25T00:23:00.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.861+0000 7f0c35619740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fa54c32630 con 0x55fa54cd1580 2024-04-25T00:23:00.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.861+0000 7f0c297fa640 1 --2- 172.21.15.107:0/675766614 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f0c140a2d00 0x7f0c140a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.861+0000 7f0c2b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cdc570 0x55fa54cde960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.861+0000 7f0c2b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cdc570 0x55fa54cde960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48444/0 (socket says 172.21.15.107:48444) 2024-04-25T00:23:00.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.861+0000 7f0c2b7fe640 1 -- 172.21.15.107:0/2046224585 learned_addr learned my addr 172.21.15.107:0/2046224585 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:00.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.861+0000 7f0c30eb3640 1 --2- 172.21.15.107:0/675766614 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f0c140a2d00 0x7f0c140a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.861+0000 7f0c297fa640 1 -- 172.21.15.107:0/675766614 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f0c1c0eff90 con 0x7f0c2c1029c0 2024-04-25T00:23:00.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.861+0000 7f0c2bfff640 1 --2- 172.21.15.107:0/2046224585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa54cd1ea0 0x55fa54cdc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.861+0000 7f0c30eb3640 1 --2- 172.21.15.107:0/2046224585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa54cd1580 0x55fa54cd1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.861+0000 7f0c2b7fe640 1 -- 172.21.15.107:0/2046224585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa54cd1580 msgr2=0x55fa54cd1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:00.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.861+0000 7f0c2b7fe640 1 --2- 172.21.15.107:0/2046224585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa54cd1580 0x55fa54cd1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.861+0000 7f0c2b7fe640 1 -- 172.21.15.107:0/2046224585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa54cd1ea0 msgr2=0x55fa54cdc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:00.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.861+0000 7f0c2b7fe640 1 --2- 172.21.15.107:0/2046224585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa54cd1ea0 0x55fa54cdc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.861+0000 7f0c2b7fe640 1 -- 172.21.15.107:0/2046224585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa54c02400 con 0x55fa54cdc570 2024-04-25T00:23:00.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.861+0000 7f0c2bfff640 1 --2- 172.21.15.107:0/2046224585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa54cd1ea0 0x55fa54cdc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:00.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.862+0000 7f0c30eb3640 1 --2- 172.21.15.107:0/675766614 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f0c140a2d00 0x7f0c140a51c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0c24009440 tx=0x7f0c2400ba10 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:00.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.862+0000 7f0c2b7fe640 1 --2- 172.21.15.107:0/2046224585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cdc570 0x55fa54cde960 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000bc80 tx=0x7f0c2000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87423060f7e5f553 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:00.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.862+0000 7f0c127fc640 1 -- 172.21.15.107:0/2046224585 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20015070 con 0x55fa54cdc570 2024-04-25T00:23:00.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.862+0000 7f0c127fc640 1 -- 172.21.15.107:0/2046224585 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c2001a940 con 0x55fa54cdc570 2024-04-25T00:23:00.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.862+0000 7f0c127fc640 1 -- 172.21.15.107:0/2046224585 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c200118a0 con 0x55fa54cdc570 2024-04-25T00:23:00.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.862+0000 7f0c35619740 1 -- 172.21.15.107:0/2046224585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cdc570 msgr2=0x55fa54cde960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.862+0000 7f0c35619740 1 --2- 172.21.15.107:0/2046224585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cdc570 0x55fa54cde960 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000bc80 tx=0x7f0c2000f730 comp rx=0 tx=0).stop 2024-04-25T00:23:00.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.862+0000 7f0c35619740 1 -- 172.21.15.107:0/2046224585 shutdown_connections 2024-04-25T00:23:00.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.862+0000 7f0c35619740 1 --2- 172.21.15.107:0/2046224585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cdc570 0x55fa54cde960 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.862+0000 7f0c35619740 1 --2- 172.21.15.107:0/2046224585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa54cd1ea0 0x55fa54cdc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.862+0000 7f0c35619740 1 --2- 172.21.15.107:0/2046224585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa54cd1580 0x55fa54cd1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.862+0000 7f0c35619740 1 -- 172.21.15.107:0/2046224585 >> 172.21.15.107:0/2046224585 conn(0x55fa54ccd8a0 msgr2=0x55fa54ccdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:00.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.862+0000 7f0c35619740 1 -- 172.21.15.107:0/2046224585 shutdown_connections 2024-04-25T00:23:00.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.862+0000 7f0c35619740 1 -- 172.21.15.107:0/2046224585 wait complete. 2024-04-25T00:23:00.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.863+0000 7f0c35619740 1 -- start start 2024-04-25T00:23:00.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.863+0000 7f0c35619740 1 Processor -- start 2024-04-25T00:23:00.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.863+0000 7f0c35619740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa54cd1580 0x55fa54cdbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.863+0000 7f0c35619740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cd1ea0 0x55fa54cd9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.863+0000 7f0c35619740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa54cdc570 0x55fa54cd9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.863+0000 7f0c35619740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fa54cdf360 con 0x55fa54cd1ea0 2024-04-25T00:23:00.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.863+0000 7f0c35619740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55fa54bf2050 con 0x55fa54cdc570 2024-04-25T00:23:00.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.863+0000 7f0c35619740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fa54cdf4e0 con 0x55fa54cd1580 2024-04-25T00:23:00.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.863+0000 7f0c35619740 10 client.0 _ll_register_callbacks cb 0x55fa54949be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:00.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.863+0000 7f0c2b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cd1ea0 0x55fa54cd9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.863+0000 7f0c2b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cd1ea0 0x55fa54cd9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48450/0 (socket says 172.21.15.107:48450) 2024-04-25T00:23:00.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.863+0000 7f0c2b7fe640 1 -- 172.21.15.107:0/1214417444 learned_addr learned my addr 172.21.15.107:0/1214417444 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:00.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.864+0000 7f0c2bfff640 1 --2- 172.21.15.107:0/1214417444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa54cd1580 0x55fa54cdbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.864+0000 7f0c2b7fe640 1 -- 172.21.15.107:0/1214417444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa54cd1580 msgr2=0x55fa54cdbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:00.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.864+0000 7f0c30eb3640 1 --2- 172.21.15.107:0/1214417444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa54cdc570 0x55fa54cd9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.864+0000 7f0c2b7fe640 1 --2- 172.21.15.107:0/1214417444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa54cd1580 0x55fa54cdbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.864+0000 7f0c2b7fe640 1 -- 172.21.15.107:0/1214417444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa54cdc570 msgr2=0x55fa54cd9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:00.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.864+0000 7f0c2b7fe640 1 --2- 172.21.15.107:0/1214417444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa54cdc570 0x55fa54cd9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.864+0000 7f0c2b7fe640 1 -- 172.21.15.107:0/1214417444 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa54bc3dc0 con 0x55fa54cd1ea0 2024-04-25T00:23:00.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.864+0000 7f0c2b7fe640 1 --2- 172.21.15.107:0/1214417444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cd1ea0 0x55fa54cd9130 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000b870 tx=0x7f0c2001aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0188bad7695f111 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:00.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.864+0000 7f0c0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:00.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.864+0000 7f0c0b7fe640 1 -- 172.21.15.107:0/1214417444 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20028060 con 0x55fa54cd1ea0 2024-04-25T00:23:00.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.864+0000 7f0c0b7fe640 1 -- 172.21.15.107:0/1214417444 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f0c2001a460 con 0x55fa54cd1ea0 2024-04-25T00:23:00.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.864+0000 7f0c35619740 1 -- 172.21.15.107:0/1214417444 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c2001a940 con 0x55fa54cd1ea0 2024-04-25T00:23:00.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.864+0000 7f0c0b7fe640 1 -- 172.21.15.107:0/1214417444 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0c2002f050 con 0x55fa54cd1ea0 2024-04-25T00:23:00.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.864+0000 7f0c35619740 10 client.17010 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:00.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.864+0000 7f0c35619740 1 -- 172.21.15.107:0/1214417444 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fa5494a3f0 con 0x55fa54cd1ea0 2024-04-25T00:23:00.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.864+0000 7f0c0b7fe640 1 -- 172.21.15.107:0/1214417444 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c2001a940 con 0x55fa54cd1ea0 2024-04-25T00:23:00.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.864+0000 7f0c0b7fe640 1 -- 172.21.15.107:0/1214417444 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f0c200049f0 con 0x55fa54cd1ea0 2024-04-25T00:23:00.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.864+0000 7f0c35619740 10 client.17010 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:00.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.864+0000 7f0c35619740 10 client.17010 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:23:00.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.864+0000 7f0c35619740 20 client.17010 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:00.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.864+0000 7f0c35619740 1 --2- 172.21.15.107:0/1214417444 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55fa54cec4d0 0x55fa54cee8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.864+0000 7f0c35619740 4 client.17010 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:00.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.864+0000 7f0c35619740 1 -- 172.21.15.107:0/1214417444 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa54c43bb0 con 0x55fa54cec4d0 2024-04-25T00:23:00.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.865+0000 7f0c2bfff640 1 --2- 172.21.15.107:0/1214417444 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55fa54cec4d0 0x55fa54cee8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.865+0000 7f0c2bfff640 1 --2- 172.21.15.107:0/1214417444 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55fa54cec4d0 0x55fa54cee8c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e40e4d6b3dc3ddfe server_cookie=6ac3f359222c987c in_seq=0 out_seq=0 2024-04-25T00:23:00.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.865+0000 7f0c0b7fe640 10 client.17010 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:23:00.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.866+0000 7f0c0b7fe640 1 -- 172.21.15.107:0/1214417444 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa54c43bb0 con 0x55fa54cec4d0 2024-04-25T00:23:00.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.866+0000 7f0c0b7fe640 10 client.17010 handle_command_reply: tid=0 2024-04-25T00:23:00.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.866+0000 7f0c35619740 1 client.17010 shutdown 2024-04-25T00:23:00.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.866+0000 7f0c35619740 1 -- 172.21.15.107:0/1214417444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cd1ea0 msgr2=0x55fa54cd9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.866+0000 7f0c35619740 1 --2- 172.21.15.107:0/1214417444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cd1ea0 0x55fa54cd9130 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000b870 tx=0x7f0c2001aaf0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.866+0000 7f0c35619740 1 -- 172.21.15.107:0/1214417444 shutdown_connections 2024-04-25T00:23:00.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.866+0000 7f0c35619740 1 --2- 172.21.15.107:0/1214417444 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55fa54cec4d0 0x55fa54cee8c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.866+0000 7f0c35619740 1 --2- 172.21.15.107:0/1214417444 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa54cdc570 0x55fa54cd9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.866+0000 7f0c0b7fe640 0 client.17010 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:23:00.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.866+0000 7f0c35619740 1 --2- 172.21.15.107:0/1214417444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cd1ea0 0x55fa54cd9130 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.866+0000 7f0c35619740 1 --2- 172.21.15.107:0/1214417444 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa54cd1580 0x55fa54cdbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.866+0000 7f0c35619740 1 -- 172.21.15.107:0/1214417444 >> 172.21.15.107:0/1214417444 conn(0x55fa54ccd8a0 msgr2=0x55fa54cccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:00.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.866+0000 7f0c35619740 1 -- 172.21.15.107:0/1214417444 shutdown_connections 2024-04-25T00:23:00.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.866+0000 7f0c35619740 1 -- 172.21.15.107:0/1214417444 wait complete. 2024-04-25T00:23:00.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.866+0000 7f0c35619740 20 client.17010 trim_cache size 0 max 16384 2024-04-25T00:23:00.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.879+0000 7f0c35619740 1 Processor -- start 2024-04-25T00:23:00.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.879+0000 7f0c35619740 1 -- start start 2024-04-25T00:23:00.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.880+0000 7f0c35619740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa54cd1580 0x55fa54ce8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.880+0000 7f0c35619740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cd1ea0 0x55fa54cccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.880+0000 7f0c35619740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa54cdc570 0x55fa54cf3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.880+0000 7f0c35619740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c1b1440 con 0x55fa54cd1ea0 2024-04-25T00:23:00.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.880+0000 7f0c35619740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c1aed10 con 0x55fa54cdc570 2024-04-25T00:23:00.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.880+0000 7f0c35619740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c1acb60 con 0x55fa54cd1580 2024-04-25T00:23:00.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.880+0000 7f0c2b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cd1ea0 0x55fa54cccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.880+0000 7f0c2b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cd1ea0 0x55fa54cccb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48458/0 (socket says 172.21.15.107:48458) 2024-04-25T00:23:00.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.880+0000 7f0c2b7fe640 1 -- 172.21.15.107:0/3262579120 learned_addr learned my addr 172.21.15.107:0/3262579120 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:00.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.880+0000 7f0c2bfff640 1 --2- 172.21.15.107:0/3262579120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa54cd1580 0x55fa54ce8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.880+0000 7f0c2b7fe640 1 -- 172.21.15.107:0/3262579120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa54cd1580 msgr2=0x55fa54ce8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:00.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.880+0000 7f0c2b7fe640 1 --2- 172.21.15.107:0/3262579120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa54cd1580 0x55fa54ce8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.880+0000 7f0c2b7fe640 1 -- 172.21.15.107:0/3262579120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa54cdc570 msgr2=0x55fa54cf3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:00.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.880+0000 7f0c2b7fe640 1 --2- 172.21.15.107:0/3262579120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa54cdc570 0x55fa54cf3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.880+0000 7f0c2b7fe640 1 -- 172.21.15.107:0/3262579120 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa54cd80e0 con 0x55fa54cd1ea0 2024-04-25T00:23:00.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.880+0000 7f0c2bfff640 1 --2- 172.21.15.107:0/3262579120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa54cd1580 0x55fa54ce8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:00.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.880+0000 7f0c2b7fe640 1 --2- 172.21.15.107:0/3262579120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cd1ea0 0x55fa54cccb40 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f0c20004f00 tx=0x7f0c2000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7cbeeb05c493a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:00.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.880+0000 7f0c11ffb640 1 -- 172.21.15.107:0/3262579120 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20028070 con 0x55fa54cd1ea0 2024-04-25T00:23:00.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.880+0000 7f0c11ffb640 1 -- 172.21.15.107:0/3262579120 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c20011660 con 0x55fa54cd1ea0 2024-04-25T00:23:00.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.880+0000 7f0c11ffb640 1 -- 172.21.15.107:0/3262579120 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20015070 con 0x55fa54cd1ea0 2024-04-25T00:23:00.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.881+0000 7f0c35619740 1 -- 172.21.15.107:0/3262579120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cd1ea0 msgr2=0x55fa54cccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.881+0000 7f0c35619740 1 --2- 172.21.15.107:0/3262579120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cd1ea0 0x55fa54cccb40 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f0c20004f00 tx=0x7f0c2000bd70 comp rx=0 tx=0).stop 2024-04-25T00:23:00.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.881+0000 7f0c35619740 1 -- 172.21.15.107:0/3262579120 shutdown_connections 2024-04-25T00:23:00.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.881+0000 7f0c35619740 1 --2- 172.21.15.107:0/3262579120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa54cdc570 0x55fa54cf3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.881+0000 7f0c35619740 1 --2- 172.21.15.107:0/3262579120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cd1ea0 0x55fa54cccb40 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.881+0000 7f0c35619740 1 --2- 172.21.15.107:0/3262579120 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa54cd1580 0x55fa54ce8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.881+0000 7f0c35619740 1 -- 172.21.15.107:0/3262579120 >> 172.21.15.107:0/3262579120 conn(0x55fa54ccd8a0 msgr2=0x55fa54ccdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:00.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.881+0000 7f0c35619740 1 -- 172.21.15.107:0/3262579120 shutdown_connections 2024-04-25T00:23:00.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.881+0000 7f0c35619740 1 -- 172.21.15.107:0/3262579120 wait complete. 2024-04-25T00:23:00.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.881+0000 7f0c35619740 1 -- start start 2024-04-25T00:23:00.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.881+0000 7f0c35619740 1 Processor -- start 2024-04-25T00:23:00.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.881+0000 7f0c35619740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cd1580 0x55fa54ce2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.881+0000 7f0c35619740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa54cd1ea0 0x55fa54ce2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.881+0000 7f0c35619740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa54cdc570 0x55fa54ce2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.882+0000 7f0c35619740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c1ad1d0 con 0x55fa54cd1580 2024-04-25T00:23:00.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.882+0000 7f0c35619740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c1ae640 con 0x55fa54cd1ea0 2024-04-25T00:23:00.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.882+0000 7f0c35619740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c1ad8a0 con 0x55fa54cdc570 2024-04-25T00:23:00.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.882+0000 7f0c35619740 10 client.0 _ll_register_callbacks cb 0x7f0c2000bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:00.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.882+0000 7f0c2bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cd1580 0x55fa54ce2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.882+0000 7f0c2bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cd1580 0x55fa54ce2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48468/0 (socket says 172.21.15.107:48468) 2024-04-25T00:23:00.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.882+0000 7f0c2bfff640 1 -- 172.21.15.107:0/187097612 learned_addr learned my addr 172.21.15.107:0/187097612 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:00.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.882+0000 7f0c2b7fe640 1 --2- 172.21.15.107:0/187097612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa54cd1ea0 0x55fa54ce2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.882+0000 7f0c2bfff640 1 -- 172.21.15.107:0/187097612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa54cdc570 msgr2=0x55fa54ce2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:00.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.882+0000 7f0c2bfff640 1 --2- 172.21.15.107:0/187097612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa54cdc570 0x55fa54ce2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.882+0000 7f0c2bfff640 1 -- 172.21.15.107:0/187097612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa54cd1ea0 msgr2=0x55fa54ce2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:00.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.882+0000 7f0c2bfff640 1 --2- 172.21.15.107:0/187097612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa54cd1ea0 0x55fa54ce2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.882+0000 7f0c2bfff640 1 -- 172.21.15.107:0/187097612 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa54a147d0 con 0x55fa54cd1580 2024-04-25T00:23:00.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.882+0000 7f0c2bfff640 1 --2- 172.21.15.107:0/187097612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cd1580 0x55fa54ce2110 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c0b85a0 tx=0x7f0c1c0dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325df98aa5dcb87f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:00.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.882+0000 7f0c0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:00.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.882+0000 7f0c0b7fe640 1 -- 172.21.15.107:0/187097612 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c049040 con 0x55fa54cd1580 2024-04-25T00:23:00.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.882+0000 7f0c0b7fe640 1 -- 172.21.15.107:0/187097612 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f0c1c0dc0b0 con 0x55fa54cd1580 2024-04-25T00:23:00.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.882+0000 7f0c35619740 1 -- 172.21.15.107:0/187097612 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c20011660 con 0x55fa54cd1580 2024-04-25T00:23:00.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.882+0000 7f0c0b7fe640 1 -- 172.21.15.107:0/187097612 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0c1c0bc140 con 0x55fa54cd1580 2024-04-25T00:23:00.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.882+0000 7f0c35619740 10 client.17016 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:00.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.882+0000 7f0c35619740 1 -- 172.21.15.107:0/187097612 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fa54d52840 con 0x55fa54cd1580 2024-04-25T00:23:00.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.882+0000 7f0c0b7fe640 1 -- 172.21.15.107:0/187097612 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c20011660 con 0x55fa54cd1580 2024-04-25T00:23:00.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.883+0000 7f0c0b7fe640 1 -- 172.21.15.107:0/187097612 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f0c1c0c0160 con 0x55fa54cd1580 2024-04-25T00:23:00.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.883+0000 7f0c35619740 10 client.17016 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:00.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.883+0000 7f0c35619740 10 client.17016 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:23:00.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.883+0000 7f0c35619740 20 client.17016 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:00.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.883+0000 7f0c35619740 1 --2- 172.21.15.107:0/187097612 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55fa54d52f90 0x55fa54d553f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.883+0000 7f0c35619740 4 client.17016 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:23:00.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.883+0000 7f0c35619740 1 -- 172.21.15.107:0/187097612 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa54c3df00 con 0x55fa54d52f90 2024-04-25T00:23:00.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.883+0000 7f0c2b7fe640 1 --2- 172.21.15.107:0/187097612 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55fa54d52f90 0x55fa54d553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.883+0000 7f0c0b7fe640 10 client.17016 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:23:00.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.883+0000 7f0c2b7fe640 1 --2- 172.21.15.107:0/187097612 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55fa54d52f90 0x55fa54d553f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=781411f2fafe4ca8 server_cookie=beaf6d6163b15378 in_seq=0 out_seq=0 2024-04-25T00:23:00.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.883+0000 7f0c0b7fe640 1 -- 172.21.15.107:0/187097612 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa54c3df00 con 0x55fa54d52f90 2024-04-25T00:23:00.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.883+0000 7f0c0b7fe640 10 client.17016 handle_command_reply: tid=0 2024-04-25T00:23:00.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.883+0000 7f0c35619740 1 client.17016 shutdown 2024-04-25T00:23:00.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.884+0000 7f0c35619740 1 -- 172.21.15.107:0/187097612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cd1580 msgr2=0x55fa54ce2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.884+0000 7f0c35619740 1 --2- 172.21.15.107:0/187097612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cd1580 0x55fa54ce2110 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c0b85a0 tx=0x7f0c1c0dc040 comp rx=0 tx=0).stop 2024-04-25T00:23:00.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.884+0000 7f0c35619740 1 -- 172.21.15.107:0/187097612 shutdown_connections 2024-04-25T00:23:00.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.884+0000 7f0c35619740 1 --2- 172.21.15.107:0/187097612 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55fa54d52f90 0x55fa54d553f0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.884+0000 7f0c35619740 1 --2- 172.21.15.107:0/187097612 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fa54cdc570 0x55fa54ce2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.884+0000 7f0c35619740 1 --2- 172.21.15.107:0/187097612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fa54cd1ea0 0x55fa54ce2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.884+0000 7f0c0b7fe640 0 client.17016 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:23:00.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.884+0000 7f0c35619740 1 --2- 172.21.15.107:0/187097612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa54cd1580 0x55fa54ce2110 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.884+0000 7f0c35619740 1 -- 172.21.15.107:0/187097612 >> 172.21.15.107:0/187097612 conn(0x55fa54ccd8a0 msgr2=0x55fa54cccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:00.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.884+0000 7f0c35619740 1 -- 172.21.15.107:0/187097612 shutdown_connections 2024-04-25T00:23:00.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.884+0000 7f0c35619740 1 -- 172.21.15.107:0/187097612 wait complete. 2024-04-25T00:23:00.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.884+0000 7f0c35619740 20 client.17016 trim_cache size 0 max 16384 2024-04-25T00:23:00.896 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:23:00.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.884+0000 7f0c32916640 1 -- 172.21.15.107:0/675766614 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f0c140a2d00 msgr2=0x7f0c140a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.884+0000 7f0c32916640 1 --2- 172.21.15.107:0/675766614 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f0c140a2d00 0x7f0c140a51c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0c24009440 tx=0x7f0c2400ba10 comp rx=0 tx=0).stop 2024-04-25T00:23:00.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.884+0000 7f0c32916640 1 -- 172.21.15.107:0/675766614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c2c1029c0 msgr2=0x7f0c2c198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.884+0000 7f0c32916640 1 --2- 172.21.15.107:0/675766614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c2c1029c0 0x7f0c2c198090 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c004770 tx=0x7f0c1c003c10 comp rx=0 tx=0).stop 2024-04-25T00:23:00.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.884+0000 7f0c32916640 1 -- 172.21.15.107:0/675766614 shutdown_connections 2024-04-25T00:23:00.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.884+0000 7f0c32916640 1 --2- 172.21.15.107:0/675766614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0c2c10bd00 0x7f0c2c19c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.884+0000 7f0c32916640 1 --2- 172.21.15.107:0/675766614 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f0c140a2d00 0x7f0c140a51c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.884+0000 7f0c32916640 1 --2- 172.21.15.107:0/675766614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0c2c1032e0 0x7f0c2c1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.884+0000 7f0c32916640 1 --2- 172.21.15.107:0/675766614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c2c1029c0 0x7f0c2c198090 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.884+0000 7f0c32916640 1 -- 172.21.15.107:0/675766614 >> 172.21.15.107:0/675766614 conn(0x7f0c2c077740 msgr2=0x7f0c2c0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:00.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.884+0000 7f0c32916640 1 -- 172.21.15.107:0/675766614 shutdown_connections 2024-04-25T00:23:00.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.884+0000 7f0c32916640 1 -- 172.21.15.107:0/675766614 wait complete. 2024-04-25T00:23:00.899 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:23:01.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.998+0000 7f0948d29640 1 -- 172.21.15.107:0/1587676140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0944101710 msgr2=0x7f0944101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:01.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.998+0000 7f0948d29640 1 --2- 172.21.15.107:0/1587676140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0944101710 0x7f0944101b90 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f092c009930 tx=0x7f092c02a420 comp rx=0 tx=0).stop 2024-04-25T00:23:01.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.998+0000 7f0948d29640 1 -- 172.21.15.107:0/1587676140 shutdown_connections 2024-04-25T00:23:01.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.998+0000 7f0948d29640 1 --2- 172.21.15.107:0/1587676140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09441020d0 0x7f094410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.998+0000 7f0948d29640 1 --2- 172.21.15.107:0/1587676140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0944101710 0x7f0944101b90 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.998+0000 7f0948d29640 1 --2- 172.21.15.107:0/1587676140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0944107720 0x7f0944107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.998+0000 7f0948d29640 1 -- 172.21.15.107:0/1587676140 >> 172.21.15.107:0/1587676140 conn(0x7f0944078b90 msgr2=0x7f0944078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:01.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.998+0000 7f0948d29640 1 -- 172.21.15.107:0/1587676140 shutdown_connections 2024-04-25T00:23:01.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.998+0000 7f0948d29640 1 -- 172.21.15.107:0/1587676140 wait complete. 2024-04-25T00:23:01.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.999+0000 7f0948d29640 1 Processor -- start 2024-04-25T00:23:01.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.999+0000 7f0948d29640 1 -- start start 2024-04-25T00:23:01.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.999+0000 7f0948d29640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0944101710 0x7f0944198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.999+0000 7f0948d29640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09441020d0 0x7f09441985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.999+0000 7f0948d29640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0944107720 0x7f094419c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.999+0000 7f0948d29640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0944111b30 con 0x7f0944101710 2024-04-25T00:23:01.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.999+0000 7f0948d29640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f09441119b0 con 0x7f0944107720 2024-04-25T00:23:01.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.999+0000 7f0948d29640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0944111cb0 con 0x7f09441020d0 2024-04-25T00:23:01.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.999+0000 7f094259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0944101710 0x7f0944198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.999+0000 7f094259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0944101710 0x7f0944198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48480/0 (socket says 172.21.15.107:48480) 2024-04-25T00:23:01.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.999+0000 7f094259c640 1 -- 172.21.15.107:0/3185142368 learned_addr learned my addr 172.21.15.107:0/3185142368 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:01.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.999+0000 7f0941d9b640 1 --2- 172.21.15.107:0/3185142368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09441020d0 0x7f09441985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.999+0000 7f0942d9d640 1 --2- 172.21.15.107:0/3185142368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0944107720 0x7f094419c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.999+0000 7f094259c640 1 -- 172.21.15.107:0/3185142368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09441020d0 msgr2=0x7f09441985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:01.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.999+0000 7f094259c640 1 --2- 172.21.15.107:0/3185142368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09441020d0 0x7f09441985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.999+0000 7f094259c640 1 -- 172.21.15.107:0/3185142368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0944107720 msgr2=0x7f094419c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:01.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:00.999+0000 7f094259c640 1 --2- 172.21.15.107:0/3185142368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0944107720 0x7f094419c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.000+0000 7f094259c640 1 -- 172.21.15.107:0/3185142368 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f094419d080 con 0x7f0944101710 2024-04-25T00:23:01.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.000+0000 7f094259c640 1 --2- 172.21.15.107:0/3185142368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0944101710 0x7f0944198090 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f093400d950 tx=0x7f093400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:01.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.000+0000 7f093b7fe640 1 -- 172.21.15.107:0/3185142368 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934017070 con 0x7f0944101710 2024-04-25T00:23:01.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.000+0000 7f0948d29640 1 -- 172.21.15.107:0/3185142368 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f094419d370 con 0x7f0944101710 2024-04-25T00:23:01.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.000+0000 7f093b7fe640 1 -- 172.21.15.107:0/3185142368 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0934009c30 con 0x7f0944101710 2024-04-25T00:23:01.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.000+0000 7f093b7fe640 1 -- 172.21.15.107:0/3185142368 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093400f6b0 con 0x7f0944101710 2024-04-25T00:23:01.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.000+0000 7f0948d29640 1 -- 172.21.15.107:0/3185142368 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09441a4c00 con 0x7f0944101710 2024-04-25T00:23:01.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.002+0000 7f093b7fe640 1 -- 172.21.15.107:0/3185142368 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f09340109d0 con 0x7f0944101710 2024-04-25T00:23:01.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.002+0000 7f094ba38740 1 Processor -- start 2024-04-25T00:23:01.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.002+0000 7f094ba38740 1 -- start start 2024-04-25T00:23:01.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.003+0000 7f093b7fe640 1 --2- 172.21.15.107:0/3185142368 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f09300a2d00 0x7f09300a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.003+0000 7f094ba38740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c1f580 0x55e5d8c1f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.003+0000 7f094ba38740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5d8c1fea0 0x55e5d8c2a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.003+0000 7f094ba38740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d8c2a570 0x55e5d8c2c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.003+0000 7f094ba38740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e5d8b84670 con 0x55e5d8c1f580 2024-04-25T00:23:01.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.003+0000 7f094ba38740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e5d8b2c970 con 0x55e5d8c1fea0 2024-04-25T00:23:01.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.003+0000 7f094ba38740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e5d8b80630 con 0x55e5d8c2a570 2024-04-25T00:23:01.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.003+0000 7f094259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c1f580 0x55e5d8c1f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.003+0000 7f0942d9d640 1 --2- 172.21.15.107:0/3185142368 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f09300a2d00 0x7f09300a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.003+0000 7f094259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c1f580 0x55e5d8c1f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48492/0 (socket says 172.21.15.107:48492) 2024-04-25T00:23:01.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.003+0000 7f094259c640 1 -- 172.21.15.107:0/941032486 learned_addr learned my addr 172.21.15.107:0/941032486 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:01.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.003+0000 7f093b7fe640 1 -- 172.21.15.107:0/3185142368 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f09340cf400 con 0x7f0944101710 2024-04-25T00:23:01.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.003+0000 7f0941d9b640 1 --2- 172.21.15.107:0/941032486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5d8c1fea0 0x55e5d8c2a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.003+0000 7f094259c640 1 -- 172.21.15.107:0/941032486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d8c2a570 msgr2=0x55e5d8c2c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:01.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.003+0000 7f094259c640 1 --2- 172.21.15.107:0/941032486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d8c2a570 0x55e5d8c2c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.003+0000 7f094259c640 1 -- 172.21.15.107:0/941032486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5d8c1fea0 msgr2=0x55e5d8c2a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:01.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.003+0000 7f094259c640 1 --2- 172.21.15.107:0/941032486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5d8c1fea0 0x55e5d8c2a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.003+0000 7f094259c640 1 -- 172.21.15.107:0/941032486 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5d8b50400 con 0x55e5d8c1f580 2024-04-25T00:23:01.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.003+0000 7f094259c640 1 --2- 172.21.15.107:0/941032486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c1f580 0x55e5d8c1f960 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f0934097a10 tx=0x7f093400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100785ecfbb54f9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:01.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.003+0000 7f0942d9d640 1 --2- 172.21.15.107:0/3185142368 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f09300a2d00 0x7f09300a51c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f093c009370 tx=0x7f093c009960 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:01.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.003+0000 7f0938ff9640 1 -- 172.21.15.107:0/941032486 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0934020070 con 0x55e5d8c1f580 2024-04-25T00:23:01.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.003+0000 7f0938ff9640 1 -- 172.21.15.107:0/941032486 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f093409bcb0 con 0x55e5d8c1f580 2024-04-25T00:23:01.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.003+0000 7f0938ff9640 1 -- 172.21.15.107:0/941032486 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093409b5b0 con 0x55e5d8c1f580 2024-04-25T00:23:01.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.004+0000 7f094ba38740 1 -- 172.21.15.107:0/941032486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c1f580 msgr2=0x55e5d8c1f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:01.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.004+0000 7f094ba38740 1 --2- 172.21.15.107:0/941032486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c1f580 0x55e5d8c1f960 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f0934097a10 tx=0x7f093400bf60 comp rx=0 tx=0).stop 2024-04-25T00:23:01.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.004+0000 7f094ba38740 1 -- 172.21.15.107:0/941032486 shutdown_connections 2024-04-25T00:23:01.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.004+0000 7f094ba38740 1 --2- 172.21.15.107:0/941032486 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d8c2a570 0x55e5d8c2c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.004+0000 7f094ba38740 1 --2- 172.21.15.107:0/941032486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5d8c1fea0 0x55e5d8c2a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.004+0000 7f094ba38740 1 --2- 172.21.15.107:0/941032486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c1f580 0x55e5d8c1f960 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.004+0000 7f094ba38740 1 -- 172.21.15.107:0/941032486 >> 172.21.15.107:0/941032486 conn(0x55e5d8c1b8a0 msgr2=0x55e5d8c1bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:01.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.004+0000 7f094ba38740 1 -- 172.21.15.107:0/941032486 shutdown_connections 2024-04-25T00:23:01.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.004+0000 7f094ba38740 1 -- 172.21.15.107:0/941032486 wait complete. 2024-04-25T00:23:01.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.004+0000 7f094ba38740 1 -- start start 2024-04-25T00:23:01.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.005+0000 7f094ba38740 1 Processor -- start 2024-04-25T00:23:01.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.005+0000 7f094ba38740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5d8c1f580 0x55e5d8c29ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.005+0000 7f094ba38740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c1fea0 0x55e5d8c271b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.005+0000 7f094ba38740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d8c2a570 0x55e5d8c27710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.005+0000 7f094ba38740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e5d8c2d2a0 con 0x55e5d8c1fea0 2024-04-25T00:23:01.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.005+0000 7f094ba38740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e5d8b40050 con 0x55e5d8c1f580 2024-04-25T00:23:01.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.005+0000 7f094ba38740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e5d8c2d420 con 0x55e5d8c2a570 2024-04-25T00:23:01.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.005+0000 7f094ba38740 10 client.0 _ll_register_callbacks cb 0x55e5d8897be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:01.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.005+0000 7f0941d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c1fea0 0x55e5d8c271b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.005+0000 7f0941d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c1fea0 0x55e5d8c271b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48494/0 (socket says 172.21.15.107:48494) 2024-04-25T00:23:01.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.005+0000 7f0941d9b640 1 -- 172.21.15.107:0/480971654 learned_addr learned my addr 172.21.15.107:0/480971654 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:01.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.005+0000 7f094259c640 1 --2- 172.21.15.107:0/480971654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5d8c1f580 0x55e5d8c29ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.005+0000 7f0941d9b640 1 -- 172.21.15.107:0/480971654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d8c2a570 msgr2=0x55e5d8c27710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:01.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.005+0000 7f0941d9b640 1 --2- 172.21.15.107:0/480971654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d8c2a570 0x55e5d8c27710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.005+0000 7f0941d9b640 1 -- 172.21.15.107:0/480971654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5d8c1f580 msgr2=0x55e5d8c29ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:01.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.005+0000 7f0941d9b640 1 --2- 172.21.15.107:0/480971654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5d8c1f580 0x55e5d8c29ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.005+0000 7f0941d9b640 1 -- 172.21.15.107:0/480971654 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5d8b11dc0 con 0x55e5d8c1fea0 2024-04-25T00:23:01.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.005+0000 7f0941d9b640 1 --2- 172.21.15.107:0/480971654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c1fea0 0x55e5d8c271b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f092c009930 tx=0x7f092c0317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce64a578a9c7e287 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:01.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.005+0000 7f0929ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:01.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.005+0000 7f0929ffb640 1 -- 172.21.15.107:0/480971654 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c035070 con 0x55e5d8c1fea0 2024-04-25T00:23:01.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.005+0000 7f0929ffb640 1 -- 172.21.15.107:0/480971654 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f092c004040 con 0x55e5d8c1fea0 2024-04-25T00:23:01.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.006+0000 7f094ba38740 1 -- 172.21.15.107:0/480971654 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f093409bcb0 con 0x55e5d8c1fea0 2024-04-25T00:23:01.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.006+0000 7f0929ffb640 1 -- 172.21.15.107:0/480971654 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f092c004240 con 0x55e5d8c1fea0 2024-04-25T00:23:01.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.006+0000 7f094ba38740 10 client.17028 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:01.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.006+0000 7f094ba38740 1 -- 172.21.15.107:0/480971654 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e5d88983f0 con 0x55e5d8c1fea0 2024-04-25T00:23:01.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.006+0000 7f0929ffb640 1 -- 172.21.15.107:0/480971654 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f093409bcb0 con 0x55e5d8c1fea0 2024-04-25T00:23:01.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.006+0000 7f0929ffb640 1 -- 172.21.15.107:0/480971654 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f092c004a90 con 0x55e5d8c1fea0 2024-04-25T00:23:01.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.006+0000 7f094ba38740 10 client.17028 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:01.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.006+0000 7f094ba38740 10 client.17028 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:23:01.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.006+0000 7f094ba38740 20 client.17028 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:01.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.006+0000 7f094ba38740 1 --2- 172.21.15.107:0/480971654 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55e5d8c3a550 0x55e5d8c3c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.006+0000 7f094ba38740 4 client.17028 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:01.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.006+0000 7f094ba38740 1 -- 172.21.15.107:0/480971654 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5d8b91bb0 con 0x55e5d8c3a550 2024-04-25T00:23:01.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.006+0000 7f094259c640 1 --2- 172.21.15.107:0/480971654 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55e5d8c3a550 0x55e5d8c3c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.007+0000 7f0929ffb640 10 client.17028 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:23:01.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.007+0000 7f094259c640 1 --2- 172.21.15.107:0/480971654 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55e5d8c3a550 0x55e5d8c3c9d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f006197880d0d55 server_cookie=40c8dbff82a49f68 in_seq=0 out_seq=0 2024-04-25T00:23:01.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.007+0000 7f0929ffb640 1 -- 172.21.15.107:0/480971654 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e5d8b91bb0 con 0x55e5d8c3a550 2024-04-25T00:23:01.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.007+0000 7f0929ffb640 10 client.17028 handle_command_reply: tid=0 2024-04-25T00:23:01.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.007+0000 7f094ba38740 1 client.17028 shutdown 2024-04-25T00:23:01.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.008+0000 7f094ba38740 1 -- 172.21.15.107:0/480971654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c1fea0 msgr2=0x55e5d8c271b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:01.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.008+0000 7f094ba38740 1 --2- 172.21.15.107:0/480971654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c1fea0 0x55e5d8c271b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f092c009930 tx=0x7f092c0317e0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.008+0000 7f094ba38740 1 -- 172.21.15.107:0/480971654 shutdown_connections 2024-04-25T00:23:01.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.008+0000 7f094ba38740 1 --2- 172.21.15.107:0/480971654 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d8c2a570 0x55e5d8c27710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.008+0000 7f094ba38740 1 --2- 172.21.15.107:0/480971654 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55e5d8c3a550 0x55e5d8c3c9d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.008+0000 7f094ba38740 1 --2- 172.21.15.107:0/480971654 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c1fea0 0x55e5d8c271b0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.008+0000 7f0929ffb640 0 client.17028 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:23:01.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.008+0000 7f094ba38740 1 --2- 172.21.15.107:0/480971654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5d8c1f580 0x55e5d8c29ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.008+0000 7f094ba38740 1 -- 172.21.15.107:0/480971654 >> 172.21.15.107:0/480971654 conn(0x55e5d8c1b8a0 msgr2=0x55e5d8c216a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:01.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.008+0000 7f094ba38740 1 -- 172.21.15.107:0/480971654 shutdown_connections 2024-04-25T00:23:01.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.008+0000 7f094ba38740 1 -- 172.21.15.107:0/480971654 wait complete. 2024-04-25T00:23:01.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.008+0000 7f094ba38740 20 client.17028 trim_cache size 0 max 16384 2024-04-25T00:23:01.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.021+0000 7f094ba38740 1 Processor -- start 2024-04-25T00:23:01.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.021+0000 7f094ba38740 1 -- start start 2024-04-25T00:23:01.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.021+0000 7f094ba38740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d8c1f580 0x55e5d8c36160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.021+0000 7f094ba38740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5d8c1fea0 0x55e5d8c422a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.021+0000 7f094ba38740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c2a570 0x55e5d8c427e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.021+0000 7f094ba38740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f09441b1490 con 0x55e5d8c2a570 2024-04-25T00:23:01.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.021+0000 7f094ba38740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f09441aed60 con 0x55e5d8c1fea0 2024-04-25T00:23:01.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.021+0000 7f094ba38740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f09441acbb0 con 0x55e5d8c1f580 2024-04-25T00:23:01.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.021+0000 7f0942d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c2a570 0x55e5d8c427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.021+0000 7f0942d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c2a570 0x55e5d8c427e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48506/0 (socket says 172.21.15.107:48506) 2024-04-25T00:23:01.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.021+0000 7f0942d9d640 1 -- 172.21.15.107:0/2691372995 learned_addr learned my addr 172.21.15.107:0/2691372995 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:01.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.021+0000 7f0942d9d640 1 -- 172.21.15.107:0/2691372995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d8c1f580 msgr2=0x55e5d8c36160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:01.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.021+0000 7f0942d9d640 1 --2- 172.21.15.107:0/2691372995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d8c1f580 0x55e5d8c36160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.021+0000 7f0942d9d640 1 -- 172.21.15.107:0/2691372995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5d8c1fea0 msgr2=0x55e5d8c422a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:01.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.021+0000 7f0942d9d640 1 --2- 172.21.15.107:0/2691372995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5d8c1fea0 0x55e5d8c422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.021+0000 7f0942d9d640 1 -- 172.21.15.107:0/2691372995 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5d89627d0 con 0x55e5d8c2a570 2024-04-25T00:23:01.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.021+0000 7f0942d9d640 1 --2- 172.21.15.107:0/2691372995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c2a570 0x55e5d8c427e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f093c0126c0 tx=0x7f093c014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470254aeb8031560 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:01.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.022+0000 7f092bfff640 1 -- 172.21.15.107:0/2691372995 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c023070 con 0x55e5d8c2a570 2024-04-25T00:23:01.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.022+0000 7f092bfff640 1 -- 172.21.15.107:0/2691372995 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f093c016070 con 0x55e5d8c2a570 2024-04-25T00:23:01.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.022+0000 7f092bfff640 1 -- 172.21.15.107:0/2691372995 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c01eb70 con 0x55e5d8c2a570 2024-04-25T00:23:01.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.022+0000 7f094ba38740 1 -- 172.21.15.107:0/2691372995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c2a570 msgr2=0x55e5d8c427e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:01.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.022+0000 7f094ba38740 1 --2- 172.21.15.107:0/2691372995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c2a570 0x55e5d8c427e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f093c0126c0 tx=0x7f093c014070 comp rx=0 tx=0).stop 2024-04-25T00:23:01.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.022+0000 7f094ba38740 1 -- 172.21.15.107:0/2691372995 shutdown_connections 2024-04-25T00:23:01.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.022+0000 7f094ba38740 1 --2- 172.21.15.107:0/2691372995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c2a570 0x55e5d8c427e0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.022+0000 7f094ba38740 1 --2- 172.21.15.107:0/2691372995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5d8c1fea0 0x55e5d8c422a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.022+0000 7f094ba38740 1 --2- 172.21.15.107:0/2691372995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d8c1f580 0x55e5d8c36160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.022+0000 7f094ba38740 1 -- 172.21.15.107:0/2691372995 >> 172.21.15.107:0/2691372995 conn(0x55e5d8c1b8a0 msgr2=0x55e5d8c25aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:01.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.022+0000 7f094ba38740 1 -- 172.21.15.107:0/2691372995 shutdown_connections 2024-04-25T00:23:01.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.022+0000 7f094ba38740 1 -- 172.21.15.107:0/2691372995 wait complete. 2024-04-25T00:23:01.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.022+0000 7f094ba38740 1 -- start start 2024-04-25T00:23:01.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.023+0000 7f094ba38740 1 Processor -- start 2024-04-25T00:23:01.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.023+0000 7f094ba38740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d8c1f580 0x55e5d8c30200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.023+0000 7f094ba38740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c1fea0 0x55e5d8c30740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.023+0000 7f094ba38740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5d8c2a570 0x55e5d8c30c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.023+0000 7f094ba38740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f09441ad220 con 0x55e5d8c1fea0 2024-04-25T00:23:01.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.023+0000 7f094ba38740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f09441ae690 con 0x55e5d8c2a570 2024-04-25T00:23:01.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.023+0000 7f094ba38740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f09441ad8f0 con 0x55e5d8c1f580 2024-04-25T00:23:01.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.023+0000 7f094ba38740 10 client.0 _ll_register_callbacks cb 0x7f092c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:01.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.023+0000 7f0941d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c1fea0 0x55e5d8c30740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.023+0000 7f0941d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c1fea0 0x55e5d8c30740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48520/0 (socket says 172.21.15.107:48520) 2024-04-25T00:23:01.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.023+0000 7f0941d9b640 1 -- 172.21.15.107:0/3698671245 learned_addr learned my addr 172.21.15.107:0/3698671245 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:01.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.023+0000 7f0942d9d640 1 --2- 172.21.15.107:0/3698671245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5d8c2a570 0x55e5d8c30c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.023+0000 7f094259c640 1 --2- 172.21.15.107:0/3698671245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d8c1f580 0x55e5d8c30200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.023+0000 7f0941d9b640 1 -- 172.21.15.107:0/3698671245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d8c1f580 msgr2=0x55e5d8c30200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:01.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.023+0000 7f0941d9b640 1 --2- 172.21.15.107:0/3698671245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d8c1f580 0x55e5d8c30200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.023+0000 7f0941d9b640 1 -- 172.21.15.107:0/3698671245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5d8c2a570 msgr2=0x55e5d8c30c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:01.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.023+0000 7f0941d9b640 1 --2- 172.21.15.107:0/3698671245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5d8c2a570 0x55e5d8c30c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.023+0000 7f0941d9b640 1 -- 172.21.15.107:0/3698671245 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5d8c9fc20 con 0x55e5d8c1fea0 2024-04-25T00:23:01.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.023+0000 7f0942d9d640 1 --2- 172.21.15.107:0/3698671245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5d8c2a570 0x55e5d8c30c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:01.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.023+0000 7f0941d9b640 1 --2- 172.21.15.107:0/3698671245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c1fea0 0x55e5d8c30740 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f092c004ee0 tx=0x7f092c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb03ab5eeffeeea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:01.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.023+0000 7f0929ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:01.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.023+0000 7f0929ffb640 1 -- 172.21.15.107:0/3698671245 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092c056060 con 0x55e5d8c1fea0 2024-04-25T00:23:01.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.023+0000 7f0929ffb640 1 -- 172.21.15.107:0/3698671245 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f092c003f70 con 0x55e5d8c1fea0 2024-04-25T00:23:01.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.023+0000 7f094ba38740 1 -- 172.21.15.107:0/3698671245 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f093c016070 con 0x55e5d8c1fea0 2024-04-25T00:23:01.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.024+0000 7f0929ffb640 1 -- 172.21.15.107:0/3698671245 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f092c05d050 con 0x55e5d8c1fea0 2024-04-25T00:23:01.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.024+0000 7f094ba38740 10 client.17034 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:01.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.024+0000 7f094ba38740 1 -- 172.21.15.107:0/3698671245 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e5d8ca0f20 con 0x55e5d8c1fea0 2024-04-25T00:23:01.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.024+0000 7f0929ffb640 1 -- 172.21.15.107:0/3698671245 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f093c016070 con 0x55e5d8c1fea0 2024-04-25T00:23:01.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.024+0000 7f0929ffb640 1 -- 172.21.15.107:0/3698671245 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f092c0053c0 con 0x55e5d8c1fea0 2024-04-25T00:23:01.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.024+0000 7f094ba38740 10 client.17034 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:01.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.024+0000 7f094ba38740 10 client.17034 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:23:01.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.024+0000 7f094ba38740 20 client.17034 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:01.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.024+0000 7f094ba38740 1 --2- 172.21.15.107:0/3698671245 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55e5d8ca16a0 0x55e5d8ca3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.024+0000 7f094ba38740 4 client.17034 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:23:01.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.024+0000 7f094ba38740 1 -- 172.21.15.107:0/3698671245 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5d8b8bf00 con 0x55e5d8ca16a0 2024-04-25T00:23:01.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.024+0000 7f094259c640 1 --2- 172.21.15.107:0/3698671245 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55e5d8ca16a0 0x55e5d8ca3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.025+0000 7f0929ffb640 10 client.17034 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:23:01.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.025+0000 7f094259c640 1 --2- 172.21.15.107:0/3698671245 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55e5d8ca16a0 0x55e5d8ca3ac0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e2a9d7446e66e3 server_cookie=24199aec09ae348c in_seq=0 out_seq=0 2024-04-25T00:23:01.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.025+0000 7f0929ffb640 1 -- 172.21.15.107:0/3698671245 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5d8b8bf00 con 0x55e5d8ca16a0 2024-04-25T00:23:01.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.025+0000 7f0929ffb640 10 client.17034 handle_command_reply: tid=0 2024-04-25T00:23:01.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.025+0000 7f094ba38740 1 client.17034 shutdown 2024-04-25T00:23:01.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.025+0000 7f094ba38740 1 -- 172.21.15.107:0/3698671245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c1fea0 msgr2=0x55e5d8c30740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:01.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.025+0000 7f094ba38740 1 --2- 172.21.15.107:0/3698671245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c1fea0 0x55e5d8c30740 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f092c004ee0 tx=0x7f092c004610 comp rx=0 tx=0).stop 2024-04-25T00:23:01.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.025+0000 7f094ba38740 1 -- 172.21.15.107:0/3698671245 shutdown_connections 2024-04-25T00:23:01.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.025+0000 7f094ba38740 1 --2- 172.21.15.107:0/3698671245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e5d8c2a570 0x55e5d8c30c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.025+0000 7f094ba38740 1 --2- 172.21.15.107:0/3698671245 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55e5d8ca16a0 0x55e5d8ca3ac0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.025+0000 7f094ba38740 1 --2- 172.21.15.107:0/3698671245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5d8c1fea0 0x55e5d8c30740 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.025+0000 7f094ba38740 1 --2- 172.21.15.107:0/3698671245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e5d8c1f580 0x55e5d8c30200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.025+0000 7f0929ffb640 0 client.17034 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:23:01.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.025+0000 7f094ba38740 1 -- 172.21.15.107:0/3698671245 >> 172.21.15.107:0/3698671245 conn(0x55e5d8c1b8a0 msgr2=0x55e5d8c35150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:01.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.025+0000 7f094ba38740 1 -- 172.21.15.107:0/3698671245 shutdown_connections 2024-04-25T00:23:01.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.025+0000 7f094ba38740 1 -- 172.21.15.107:0/3698671245 wait complete. 2024-04-25T00:23:01.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.025+0000 7f094ba38740 20 client.17034 trim_cache size 0 max 16384 2024-04-25T00:23:01.033 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:23:01.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.025+0000 7f0948d29640 1 -- 172.21.15.107:0/3185142368 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f09300a2d00 msgr2=0x7f09300a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:01.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.025+0000 7f0948d29640 1 --2- 172.21.15.107:0/3185142368 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f09300a2d00 0x7f09300a51c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f093c009370 tx=0x7f093c009960 comp rx=0 tx=0).stop 2024-04-25T00:23:01.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.025+0000 7f0948d29640 1 -- 172.21.15.107:0/3185142368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0944101710 msgr2=0x7f0944198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:01.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.025+0000 7f0948d29640 1 --2- 172.21.15.107:0/3185142368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0944101710 0x7f0944198090 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f093400d950 tx=0x7f093400de10 comp rx=0 tx=0).stop 2024-04-25T00:23:01.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.026+0000 7f0948d29640 1 -- 172.21.15.107:0/3185142368 shutdown_connections 2024-04-25T00:23:01.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.026+0000 7f0948d29640 1 --2- 172.21.15.107:0/3185142368 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f09300a2d00 0x7f09300a51c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.026+0000 7f0948d29640 1 --2- 172.21.15.107:0/3185142368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0944107720 0x7f094419c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.026+0000 7f0948d29640 1 --2- 172.21.15.107:0/3185142368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f09441020d0 0x7f09441985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.026+0000 7f0948d29640 1 --2- 172.21.15.107:0/3185142368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0944101710 0x7f0944198090 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.026+0000 7f0948d29640 1 -- 172.21.15.107:0/3185142368 >> 172.21.15.107:0/3185142368 conn(0x7f0944078b90 msgr2=0x7f09440ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:01.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.026+0000 7f0948d29640 1 -- 172.21.15.107:0/3185142368 shutdown_connections 2024-04-25T00:23:01.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.026+0000 7f0948d29640 1 -- 172.21.15.107:0/3185142368 wait complete. 2024-04-25T00:23:01.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:00 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2828282045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:23:01.036 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:23:01.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:00 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2828282045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:23:01.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.135+0000 7f6905984640 1 -- 172.21.15.107:0/1130212291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6900100610 msgr2=0x7f6900100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:01.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.135+0000 7f6905984640 1 --2- 172.21.15.107:0/1130212291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6900100610 0x7f6900100a90 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009930 tx=0x7f68f402a420 comp rx=0 tx=0).stop 2024-04-25T00:23:01.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.135+0000 7f6905984640 1 -- 172.21.15.107:0/1130212291 shutdown_connections 2024-04-25T00:23:01.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.135+0000 7f6905984640 1 --2- 172.21.15.107:0/1130212291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6900100fd0 0x7f690010d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.135+0000 7f6905984640 1 --2- 172.21.15.107:0/1130212291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6900100610 0x7f6900100a90 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.135+0000 7f6905984640 1 --2- 172.21.15.107:0/1130212291 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69001066b0 0x7f6900106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.135+0000 7f6905984640 1 -- 172.21.15.107:0/1130212291 >> 172.21.15.107:0/1130212291 conn(0x7f6900077740 msgr2=0x7f6900077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:01.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.136+0000 7f6905984640 1 -- 172.21.15.107:0/1130212291 shutdown_connections 2024-04-25T00:23:01.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.136+0000 7f6905984640 1 -- 172.21.15.107:0/1130212291 wait complete. 2024-04-25T00:23:01.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.136+0000 7f6905984640 1 Processor -- start 2024-04-25T00:23:01.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.136+0000 7f6905984640 1 -- start start 2024-04-25T00:23:01.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.136+0000 7f6905984640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6900100610 0x7f6900198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.136+0000 7f6905984640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6900100fd0 0x7f6900198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.136+0000 7f6905984640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69001066b0 0x7f690019c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.136+0000 7f6905984640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6900111b30 con 0x7f6900100fd0 2024-04-25T00:23:01.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.137+0000 7f6905984640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f69001119b0 con 0x7f6900100610 2024-04-25T00:23:01.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.137+0000 7f6905984640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6900111cb0 con 0x7f69001066b0 2024-04-25T00:23:01.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.137+0000 7f68fe7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6900100fd0 0x7f6900198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.137+0000 7f68fe7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6900100fd0 0x7f6900198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48532/0 (socket says 172.21.15.107:48532) 2024-04-25T00:23:01.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.137+0000 7f68fe7fc640 1 -- 172.21.15.107:0/3723282306 learned_addr learned my addr 172.21.15.107:0/3723282306 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:01.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.137+0000 7f68ff7fe640 1 --2- 172.21.15.107:0/3723282306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69001066b0 0x7f690019c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.137+0000 7f68fe7fc640 1 -- 172.21.15.107:0/3723282306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69001066b0 msgr2=0x7f690019c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:01.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.137+0000 7f68fe7fc640 1 --2- 172.21.15.107:0/3723282306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69001066b0 0x7f690019c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.137+0000 7f68feffd640 1 --2- 172.21.15.107:0/3723282306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6900100610 0x7f6900198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.137+0000 7f68fe7fc640 1 -- 172.21.15.107:0/3723282306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6900100610 msgr2=0x7f6900198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:01.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.137+0000 7f68fe7fc640 1 --2- 172.21.15.107:0/3723282306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6900100610 0x7f6900198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.137+0000 7f68fe7fc640 1 -- 172.21.15.107:0/3723282306 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690019d020 con 0x7f6900100fd0 2024-04-25T00:23:01.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.137+0000 7f68ff7fe640 1 --2- 172.21.15.107:0/3723282306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69001066b0 0x7f690019c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:01.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.137+0000 7f68fe7fc640 1 --2- 172.21.15.107:0/3723282306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6900100fd0 0x7f6900198570 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009230 tx=0x7f68f4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:01.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.137+0000 7f6904982640 1 -- 172.21.15.107:0/3723282306 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f403f030 con 0x7f6900100fd0 2024-04-25T00:23:01.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.137+0000 7f6904982640 1 -- 172.21.15.107:0/3723282306 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68f4003ad0 con 0x7f6900100fd0 2024-04-25T00:23:01.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.137+0000 7f6905984640 1 -- 172.21.15.107:0/3723282306 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f690019d2b0 con 0x7f6900100fd0 2024-04-25T00:23:01.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.137+0000 7f6904982640 1 -- 172.21.15.107:0/3723282306 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4035070 con 0x7f6900100fd0 2024-04-25T00:23:01.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.137+0000 7f6905984640 1 -- 172.21.15.107:0/3723282306 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69001a4ba0 con 0x7f6900100fd0 2024-04-25T00:23:01.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.140+0000 7f6904982640 1 -- 172.21.15.107:0/3723282306 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f68f4042030 con 0x7f6900100fd0 2024-04-25T00:23:01.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.140+0000 7f6908768740 1 Processor -- start 2024-04-25T00:23:01.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.140+0000 7f6908768740 1 -- start start 2024-04-25T00:23:01.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.140+0000 7f6908768740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5e9580 0x555bee5e9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.140+0000 7f6908768740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bee5e9ea0 0x555bee5f4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.140+0000 7f6908768740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555bee5f4570 0x555bee5f6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.140+0000 7f6908768740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555bee54e670 con 0x555bee5e9580 2024-04-25T00:23:01.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.140+0000 7f6908768740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555bee4f6970 con 0x555bee5e9ea0 2024-04-25T00:23:01.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.140+0000 7f6908768740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555bee54a630 con 0x555bee5f4570 2024-04-25T00:23:01.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.140+0000 7f6904982640 1 --2- 172.21.15.107:0/3723282306 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f68e80a2d00 0x7f68e80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.140+0000 7f68ff7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5e9580 0x555bee5e9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.140+0000 7f68ff7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5e9580 0x555bee5e9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48538/0 (socket says 172.21.15.107:48538) 2024-04-25T00:23:01.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.140+0000 7f68ff7fe640 1 -- 172.21.15.107:0/3828066813 learned_addr learned my addr 172.21.15.107:0/3828066813 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:01.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.140+0000 7f68ff7fe640 1 --2- 172.21.15.107:0/3723282306 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f68e80a2d00 0x7f68e80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.141+0000 7f68feffd640 1 --2- 172.21.15.107:0/3828066813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bee5e9ea0 0x555bee5f4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.141+0000 7f6904982640 1 -- 172.21.15.107:0/3723282306 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f68f40efdd0 con 0x7f6900100fd0 2024-04-25T00:23:01.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.141+0000 7f68fe7fc640 1 --2- 172.21.15.107:0/3828066813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555bee5f4570 0x555bee5f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.141+0000 7f68ff7fe640 1 -- 172.21.15.107:0/3828066813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555bee5f4570 msgr2=0x555bee5f6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:01.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.141+0000 7f68ff7fe640 1 --2- 172.21.15.107:0/3828066813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555bee5f4570 0x555bee5f6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.141+0000 7f68ff7fe640 1 -- 172.21.15.107:0/3828066813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bee5e9ea0 msgr2=0x555bee5f4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:01.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.141+0000 7f68ff7fe640 1 --2- 172.21.15.107:0/3828066813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bee5e9ea0 0x555bee5f4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.141+0000 7f68ff7fe640 1 -- 172.21.15.107:0/3828066813 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bee51a400 con 0x555bee5e9580 2024-04-25T00:23:01.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.141+0000 7f68feffd640 1 --2- 172.21.15.107:0/3828066813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bee5e9ea0 0x555bee5f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:01.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.141+0000 7f68ff7fe640 1 --2- 172.21.15.107:0/3828066813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5e9580 0x555bee5e9960 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f68f8014cb0 tx=0x7f68f8011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bee4a7c82ff9bca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:01.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.141+0000 7f68ff7fe640 1 --2- 172.21.15.107:0/3723282306 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f68e80a2d00 0x7f68e80a51c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f68f80058c0 tx=0x7f68f801c000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:01.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.141+0000 7f68e5ffb640 1 -- 172.21.15.107:0/3828066813 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8022070 con 0x555bee5e9580 2024-04-25T00:23:01.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.141+0000 7f68e5ffb640 1 -- 172.21.15.107:0/3828066813 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68f800ba80 con 0x555bee5e9580 2024-04-25T00:23:01.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.141+0000 7f68e5ffb640 1 -- 172.21.15.107:0/3828066813 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8005580 con 0x555bee5e9580 2024-04-25T00:23:01.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.142+0000 7f6908768740 1 -- 172.21.15.107:0/3828066813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5e9580 msgr2=0x555bee5e9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:01.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.142+0000 7f6908768740 1 --2- 172.21.15.107:0/3828066813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5e9580 0x555bee5e9960 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f68f8014cb0 tx=0x7f68f8011750 comp rx=0 tx=0).stop 2024-04-25T00:23:01.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.142+0000 7f6908768740 1 -- 172.21.15.107:0/3828066813 shutdown_connections 2024-04-25T00:23:01.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.142+0000 7f6908768740 1 --2- 172.21.15.107:0/3828066813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555bee5f4570 0x555bee5f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.142+0000 7f6908768740 1 --2- 172.21.15.107:0/3828066813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bee5e9ea0 0x555bee5f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.142+0000 7f6908768740 1 --2- 172.21.15.107:0/3828066813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5e9580 0x555bee5e9960 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.142+0000 7f6908768740 1 -- 172.21.15.107:0/3828066813 >> 172.21.15.107:0/3828066813 conn(0x555bee5e58a0 msgr2=0x555bee5e5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:01.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.142+0000 7f6908768740 1 -- 172.21.15.107:0/3828066813 shutdown_connections 2024-04-25T00:23:01.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.142+0000 7f6908768740 1 -- 172.21.15.107:0/3828066813 wait complete. 2024-04-25T00:23:01.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.142+0000 7f6908768740 1 -- start start 2024-04-25T00:23:01.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.142+0000 7f6908768740 1 Processor -- start 2024-04-25T00:23:01.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f6908768740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5e9580 0x555bee5f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f6908768740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555bee5e9ea0 0x555bee5f11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f6908768740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bee5f4570 0x555bee5f1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f6908768740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555bee5f7300 con 0x555bee5e9580 2024-04-25T00:23:01.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f6908768740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555bee50a050 con 0x555bee5f4570 2024-04-25T00:23:01.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f6908768740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555bee5f7480 con 0x555bee5e9ea0 2024-04-25T00:23:01.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f6908768740 10 client.0 _ll_register_callbacks cb 0x555bee261be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:01.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f68feffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5e9580 0x555bee5f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f68feffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5e9580 0x555bee5f3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48550/0 (socket says 172.21.15.107:48550) 2024-04-25T00:23:01.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f68feffd640 1 -- 172.21.15.107:0/3135644058 learned_addr learned my addr 172.21.15.107:0/3135644058 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:01.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f68fe7fc640 1 --2- 172.21.15.107:0/3135644058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555bee5e9ea0 0x555bee5f11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f68ff7fe640 1 --2- 172.21.15.107:0/3135644058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bee5f4570 0x555bee5f1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f68feffd640 1 -- 172.21.15.107:0/3135644058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555bee5e9ea0 msgr2=0x555bee5f11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:01.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f68feffd640 1 --2- 172.21.15.107:0/3135644058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555bee5e9ea0 0x555bee5f11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f68feffd640 1 -- 172.21.15.107:0/3135644058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bee5f4570 msgr2=0x555bee5f1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:01.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f68feffd640 1 --2- 172.21.15.107:0/3135644058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bee5f4570 0x555bee5f1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f68feffd640 1 -- 172.21.15.107:0/3135644058 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bee4dbdc0 con 0x555bee5e9580 2024-04-25T00:23:01.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f68feffd640 1 --2- 172.21.15.107:0/3135644058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5e9580 0x555bee5f3ed0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f68f0009fa0 tx=0x7f68f0011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff49c3dc8ac49b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:01.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f68deffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:01.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f68deffd640 1 -- 172.21.15.107:0/3135644058 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f0012710 con 0x555bee5e9580 2024-04-25T00:23:01.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f68deffd640 1 -- 172.21.15.107:0/3135644058 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f68f00128b0 con 0x555bee5e9580 2024-04-25T00:23:01.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f6908768740 1 -- 172.21.15.107:0/3135644058 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68f800ba80 con 0x555bee5e9580 2024-04-25T00:23:01.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f68deffd640 1 -- 172.21.15.107:0/3135644058 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f68f0012ab0 con 0x555bee5e9580 2024-04-25T00:23:01.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f6908768740 10 client.17046 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:01.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.143+0000 7f6908768740 1 -- 172.21.15.107:0/3135644058 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555bee2623f0 con 0x555bee5e9580 2024-04-25T00:23:01.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.144+0000 7f68deffd640 1 -- 172.21.15.107:0/3135644058 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68f800ba80 con 0x555bee5e9580 2024-04-25T00:23:01.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.144+0000 7f68deffd640 1 -- 172.21.15.107:0/3135644058 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f68f001a3d0 con 0x555bee5e9580 2024-04-25T00:23:01.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.144+0000 7f6908768740 10 client.17046 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:01.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.144+0000 7f6908768740 10 client.17046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:01.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.144+0000 7f6908768740 20 client.17046 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:01.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.144+0000 7f6908768740 1 --2- 172.21.15.107:0/3135644058 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x555bee6045c0 0x555bee6069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.144+0000 7f6908768740 4 client.17046 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:01.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.144+0000 7f6908768740 1 -- 172.21.15.107:0/3135644058 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bee55bbb0 con 0x555bee6045c0 2024-04-25T00:23:01.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.144+0000 7f68fe7fc640 1 --2- 172.21.15.107:0/3135644058 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x555bee6045c0 0x555bee6069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.144+0000 7f68deffd640 10 client.17046 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:01.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.144+0000 7f68fe7fc640 1 --2- 172.21.15.107:0/3135644058 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x555bee6045c0 0x555bee6069f0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c525db635c48d7e server_cookie=5890694b7fa173c4 in_seq=0 out_seq=0 2024-04-25T00:23:01.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.145+0000 7f68deffd640 1 -- 172.21.15.107:0/3135644058 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555bee55bbb0 con 0x555bee6045c0 2024-04-25T00:23:01.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.145+0000 7f68deffd640 10 client.17046 handle_command_reply: tid=0 2024-04-25T00:23:01.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.145+0000 7f6908768740 1 client.17046 shutdown 2024-04-25T00:23:01.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.145+0000 7f6908768740 1 -- 172.21.15.107:0/3135644058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5e9580 msgr2=0x555bee5f3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:01.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.145+0000 7f6908768740 1 --2- 172.21.15.107:0/3135644058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5e9580 0x555bee5f3ed0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f68f0009fa0 tx=0x7f68f0011870 comp rx=0 tx=0).stop 2024-04-25T00:23:01.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.145+0000 7f6908768740 1 -- 172.21.15.107:0/3135644058 shutdown_connections 2024-04-25T00:23:01.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.145+0000 7f6908768740 1 --2- 172.21.15.107:0/3135644058 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x555bee6045c0 0x555bee6069f0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.145+0000 7f6908768740 1 --2- 172.21.15.107:0/3135644058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bee5f4570 0x555bee5f1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.145+0000 7f6908768740 1 --2- 172.21.15.107:0/3135644058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555bee5e9ea0 0x555bee5f11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.145+0000 7f6908768740 1 --2- 172.21.15.107:0/3135644058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5e9580 0x555bee5f3ed0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.145+0000 7f68deffd640 0 client.17046 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:01.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.145+0000 7f6908768740 1 -- 172.21.15.107:0/3135644058 >> 172.21.15.107:0/3135644058 conn(0x555bee5e58a0 msgr2=0x555bee5e4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:01.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.145+0000 7f6908768740 1 -- 172.21.15.107:0/3135644058 shutdown_connections 2024-04-25T00:23:01.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.145+0000 7f6908768740 1 -- 172.21.15.107:0/3135644058 wait complete. 2024-04-25T00:23:01.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.145+0000 7f6908768740 20 client.17046 trim_cache size 0 max 16384 2024-04-25T00:23:01.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.159+0000 7f6908768740 1 Processor -- start 2024-04-25T00:23:01.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.159+0000 7f6908768740 1 -- start start 2024-04-25T00:23:01.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.159+0000 7f6908768740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bee5e9580 0x555bee6001c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.159+0000 7f6908768740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5e9ea0 0x555bee5ec5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.159+0000 7f6908768740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555bee5f4570 0x555bee5f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.159+0000 7f6908768740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f69001b1400 con 0x555bee5e9ea0 2024-04-25T00:23:01.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.159+0000 7f6908768740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f69001aecd0 con 0x555bee5e9580 2024-04-25T00:23:01.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.159+0000 7f6908768740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f69001acb20 con 0x555bee5f4570 2024-04-25T00:23:01.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.159+0000 7f68fe7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5e9ea0 0x555bee5ec5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.159+0000 7f68fe7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5e9ea0 0x555bee5ec5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48558/0 (socket says 172.21.15.107:48558) 2024-04-25T00:23:01.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.159+0000 7f68fe7fc640 1 -- 172.21.15.107:0/955188282 learned_addr learned my addr 172.21.15.107:0/955188282 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:01.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.159+0000 7f68feffd640 1 --2- 172.21.15.107:0/955188282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bee5e9580 0x555bee6001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.159+0000 7f68fe7fc640 1 -- 172.21.15.107:0/955188282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555bee5f4570 msgr2=0x555bee5f3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:01.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.159+0000 7f68ff7fe640 1 --2- 172.21.15.107:0/955188282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555bee5f4570 0x555bee5f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.159+0000 7f68fe7fc640 1 --2- 172.21.15.107:0/955188282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555bee5f4570 0x555bee5f3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.159+0000 7f68fe7fc640 1 -- 172.21.15.107:0/955188282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bee5e9580 msgr2=0x555bee6001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:01.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.159+0000 7f68fe7fc640 1 --2- 172.21.15.107:0/955188282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bee5e9580 0x555bee6001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.159+0000 7f68fe7fc640 1 -- 172.21.15.107:0/955188282 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bee5f1a90 con 0x555bee5e9ea0 2024-04-25T00:23:01.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.159+0000 7f68ff7fe640 1 --2- 172.21.15.107:0/955188282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555bee5f4570 0x555bee5f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:01.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.159+0000 7f68fe7fc640 1 --2- 172.21.15.107:0/955188282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5e9ea0 0x555bee5ec5c0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009900 tx=0x7f68f408dee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72e028b5e9c95da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:01.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.159+0000 7f68e57fa640 1 -- 172.21.15.107:0/955188282 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4049040 con 0x555bee5e9ea0 2024-04-25T00:23:01.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.160+0000 7f68e57fa640 1 -- 172.21.15.107:0/955188282 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68f40bbf20 con 0x555bee5e9ea0 2024-04-25T00:23:01.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.160+0000 7f68e57fa640 1 -- 172.21.15.107:0/955188282 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f40b8590 con 0x555bee5e9ea0 2024-04-25T00:23:01.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.160+0000 7f6908768740 1 -- 172.21.15.107:0/955188282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5e9ea0 msgr2=0x555bee5ec5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:01.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.160+0000 7f6908768740 1 --2- 172.21.15.107:0/955188282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5e9ea0 0x555bee5ec5c0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009900 tx=0x7f68f408dee0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.160+0000 7f6908768740 1 -- 172.21.15.107:0/955188282 shutdown_connections 2024-04-25T00:23:01.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.160+0000 7f6908768740 1 --2- 172.21.15.107:0/955188282 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555bee5f4570 0x555bee5f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.160+0000 7f6908768740 1 --2- 172.21.15.107:0/955188282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5e9ea0 0x555bee5ec5c0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.160+0000 7f6908768740 1 --2- 172.21.15.107:0/955188282 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bee5e9580 0x555bee6001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.160+0000 7f6908768740 1 -- 172.21.15.107:0/955188282 >> 172.21.15.107:0/955188282 conn(0x555bee5e58a0 msgr2=0x555bee5ebeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:01.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.160+0000 7f6908768740 1 -- 172.21.15.107:0/955188282 shutdown_connections 2024-04-25T00:23:01.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.160+0000 7f6908768740 1 -- 172.21.15.107:0/955188282 wait complete. 2024-04-25T00:23:01.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.160+0000 7f6908768740 1 -- start start 2024-04-25T00:23:01.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.161+0000 7f6908768740 1 Processor -- start 2024-04-25T00:23:01.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.161+0000 7f6908768740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555bee5e9580 0x555bee5ee2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.161+0000 7f6908768740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bee5e9ea0 0x555bee5ee7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.161+0000 7f6908768740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5f4570 0x555bee66df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.161+0000 7f6908768740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f69001ad190 con 0x555bee5f4570 2024-04-25T00:23:01.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.161+0000 7f6908768740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f69001ae600 con 0x555bee5e9ea0 2024-04-25T00:23:01.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.161+0000 7f6908768740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f69001ad860 con 0x555bee5e9580 2024-04-25T00:23:01.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.161+0000 7f6908768740 10 client.0 _ll_register_callbacks cb 0x7f68f00094c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:01.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.161+0000 7f68ff7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5f4570 0x555bee66df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.161+0000 7f68ff7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5f4570 0x555bee66df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48562/0 (socket says 172.21.15.107:48562) 2024-04-25T00:23:01.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.161+0000 7f68ff7fe640 1 -- 172.21.15.107:0/46121587 learned_addr learned my addr 172.21.15.107:0/46121587 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:01.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.161+0000 7f68fe7fc640 1 --2- 172.21.15.107:0/46121587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bee5e9ea0 0x555bee5ee7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.161+0000 7f68feffd640 1 --2- 172.21.15.107:0/46121587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555bee5e9580 0x555bee5ee2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.161+0000 7f68ff7fe640 1 -- 172.21.15.107:0/46121587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555bee5e9580 msgr2=0x555bee5ee2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:01.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.161+0000 7f68ff7fe640 1 --2- 172.21.15.107:0/46121587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555bee5e9580 0x555bee5ee2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.161+0000 7f68ff7fe640 1 -- 172.21.15.107:0/46121587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bee5e9ea0 msgr2=0x555bee5ee7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:01.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.161+0000 7f68ff7fe640 1 --2- 172.21.15.107:0/46121587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bee5e9ea0 0x555bee5ee7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.161+0000 7f68ff7fe640 1 -- 172.21.15.107:0/46121587 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bee32c7d0 con 0x555bee5f4570 2024-04-25T00:23:01.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.161+0000 7f68ff7fe640 1 --2- 172.21.15.107:0/46121587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5f4570 0x555bee66df90 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f68f80051d0 tx=0x7f68f8014830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14aa9c73c760ec62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:01.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.161+0000 7f68deffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:01.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.162+0000 7f68deffd640 1 -- 172.21.15.107:0/46121587 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8022040 con 0x555bee5f4570 2024-04-25T00:23:01.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.162+0000 7f68deffd640 1 -- 172.21.15.107:0/46121587 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f68f801bcc0 con 0x555bee5f4570 2024-04-25T00:23:01.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.162+0000 7f6908768740 1 -- 172.21.15.107:0/46121587 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68f40bbf20 con 0x555bee5f4570 2024-04-25T00:23:01.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.162+0000 7f68deffd640 1 -- 172.21.15.107:0/46121587 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f68f8014020 con 0x555bee5f4570 2024-04-25T00:23:01.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.162+0000 7f6908768740 10 client.17052 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:01.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.162+0000 7f6908768740 1 -- 172.21.15.107:0/46121587 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555bee66f3f0 con 0x555bee5f4570 2024-04-25T00:23:01.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.162+0000 7f68deffd640 1 -- 172.21.15.107:0/46121587 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68f40bbf20 con 0x555bee5f4570 2024-04-25T00:23:01.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.162+0000 7f68deffd640 1 -- 172.21.15.107:0/46121587 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f68f801d070 con 0x555bee5f4570 2024-04-25T00:23:01.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.162+0000 7f6908768740 10 client.17052 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:01.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.162+0000 7f6908768740 10 client.17052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:01.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.162+0000 7f6908768740 20 client.17052 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:01.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.162+0000 7f6908768740 1 --2- 172.21.15.107:0/46121587 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x555bee66f890 0x555bee671cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:01.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.162+0000 7f6908768740 4 client.17052 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:23:01.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.162+0000 7f6908768740 1 -- 172.21.15.107:0/46121587 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555bee555f00 con 0x555bee66f890 2024-04-25T00:23:01.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.162+0000 7f68feffd640 1 --2- 172.21.15.107:0/46121587 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x555bee66f890 0x555bee671cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:01.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.163+0000 7f68deffd640 10 client.17052 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:01.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.163+0000 7f68feffd640 1 --2- 172.21.15.107:0/46121587 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x555bee66f890 0x555bee671cb0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3891a2aee05950e9 server_cookie=327b39b9a6295333 in_seq=0 out_seq=0 2024-04-25T00:23:01.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.163+0000 7f68deffd640 1 -- 172.21.15.107:0/46121587 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555bee555f00 con 0x555bee66f890 2024-04-25T00:23:01.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.163+0000 7f68deffd640 10 client.17052 handle_command_reply: tid=0 2024-04-25T00:23:01.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.163+0000 7f6908768740 1 client.17052 shutdown 2024-04-25T00:23:01.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.163+0000 7f6908768740 1 -- 172.21.15.107:0/46121587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5f4570 msgr2=0x555bee66df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:01.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.163+0000 7f6908768740 1 --2- 172.21.15.107:0/46121587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5f4570 0x555bee66df90 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f68f80051d0 tx=0x7f68f8014830 comp rx=0 tx=0).stop 2024-04-25T00:23:01.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.163+0000 7f6908768740 1 -- 172.21.15.107:0/46121587 shutdown_connections 2024-04-25T00:23:01.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.163+0000 7f6908768740 1 --2- 172.21.15.107:0/46121587 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x555bee66f890 0x555bee671cb0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.163+0000 7f6908768740 1 --2- 172.21.15.107:0/46121587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bee5f4570 0x555bee66df90 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.163+0000 7f68deffd640 0 client.17052 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:01.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.163+0000 7f6908768740 1 --2- 172.21.15.107:0/46121587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bee5e9ea0 0x555bee5ee7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.163+0000 7f6908768740 1 --2- 172.21.15.107:0/46121587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555bee5e9580 0x555bee5ee2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.163+0000 7f6908768740 1 -- 172.21.15.107:0/46121587 >> 172.21.15.107:0/46121587 conn(0x555bee5e58a0 msgr2=0x555bee5eb560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:01.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.163+0000 7f6908768740 1 -- 172.21.15.107:0/46121587 shutdown_connections 2024-04-25T00:23:01.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.163+0000 7f6908768740 1 -- 172.21.15.107:0/46121587 wait complete. 2024-04-25T00:23:01.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.163+0000 7f6908768740 20 client.17052 trim_cache size 0 max 16384 2024-04-25T00:23:01.176 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:23:01.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.164+0000 7f6905984640 1 -- 172.21.15.107:0/3723282306 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f68e80a2d00 msgr2=0x7f68e80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:01.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.164+0000 7f6905984640 1 --2- 172.21.15.107:0/3723282306 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f68e80a2d00 0x7f68e80a51c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f68f80058c0 tx=0x7f68f801c000 comp rx=0 tx=0).stop 2024-04-25T00:23:01.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.164+0000 7f6905984640 1 -- 172.21.15.107:0/3723282306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6900100fd0 msgr2=0x7f6900198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:01.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.164+0000 7f6905984640 1 --2- 172.21.15.107:0/3723282306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6900100fd0 0x7f6900198570 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009230 tx=0x7f68f4004270 comp rx=0 tx=0).stop 2024-04-25T00:23:01.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.164+0000 7f6905984640 1 -- 172.21.15.107:0/3723282306 shutdown_connections 2024-04-25T00:23:01.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.164+0000 7f6905984640 1 --2- 172.21.15.107:0/3723282306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69001066b0 0x7f690019c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.164+0000 7f6905984640 1 --2- 172.21.15.107:0/3723282306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6900100fd0 0x7f6900198570 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.164+0000 7f6905984640 1 --2- 172.21.15.107:0/3723282306 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f68e80a2d00 0x7f68e80a51c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.164+0000 7f6905984640 1 --2- 172.21.15.107:0/3723282306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6900100610 0x7f6900198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:01.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.164+0000 7f6905984640 1 -- 172.21.15.107:0/3723282306 >> 172.21.15.107:0/3723282306 conn(0x7f6900077740 msgr2=0x7f69000ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:01.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.164+0000 7f6905984640 1 -- 172.21.15.107:0/3723282306 shutdown_connections 2024-04-25T00:23:01.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:01.164+0000 7f6905984640 1 -- 172.21.15.107:0/3723282306 wait complete. 2024-04-25T00:23:01.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:00 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2828282045' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:23:02.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:01 smithi195 ceph-mon[32833]: pgmap v200: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:23:02.179 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:23:02.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:23:02.179 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:23:02.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:01 smithi170 ceph-mon[32807]: pgmap v200: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:23:02.205 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:01 smithi107 ceph-mon[29153]: pgmap v200: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:23:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.294+0000 7fb43b790640 1 -- 172.21.15.107:0/3234587809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4341055e0 msgr2=0x7fb4341059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.294+0000 7fb43b790640 1 --2- 172.21.15.107:0/3234587809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4341055e0 0x7fb4341059c0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fb430009900 tx=0x7fb43002a410 comp rx=0 tx=0).stop 2024-04-25T00:23:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.294+0000 7fb43b790640 1 -- 172.21.15.107:0/3234587809 shutdown_connections 2024-04-25T00:23:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.294+0000 7fb43b790640 1 --2- 172.21.15.107:0/3234587809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb434101760 0x7fb43410d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.294+0000 7fb43b790640 1 --2- 172.21.15.107:0/3234587809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb434100da0 0x7fb434101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.294+0000 7fb43b790640 1 --2- 172.21.15.107:0/3234587809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4341055e0 0x7fb4341059c0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.294+0000 7fb43b790640 1 -- 172.21.15.107:0/3234587809 >> 172.21.15.107:0/3234587809 conn(0x7fb434076be0 msgr2=0x7fb4340fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.294+0000 7fb43b790640 1 -- 172.21.15.107:0/3234587809 shutdown_connections 2024-04-25T00:23:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.294+0000 7fb43b790640 1 -- 172.21.15.107:0/3234587809 wait complete. 2024-04-25T00:23:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.295+0000 7fb43b790640 1 Processor -- start 2024-04-25T00:23:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.295+0000 7fb43b790640 1 -- start start 2024-04-25T00:23:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.295+0000 7fb43b790640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb434100da0 0x7fb434197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.295+0000 7fb43b790640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb434101760 0x7fb4341984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.295+0000 7fb43b790640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4341055e0 0x7fb43419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.295+0000 7fb43b790640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb434111b10 con 0x7fb4341055e0 2024-04-25T00:23:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.295+0000 7fb43b790640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb434111990 con 0x7fb434100da0 2024-04-25T00:23:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.295+0000 7fb43b790640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb434111c90 con 0x7fb434101760 2024-04-25T00:23:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.295+0000 7fb439d2d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4341055e0 0x7fb43419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.295+0000 7fb439d2d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4341055e0 0x7fb43419c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48576/0 (socket says 172.21.15.107:48576) 2024-04-25T00:23:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.295+0000 7fb439d2d640 1 -- 172.21.15.107:0/2691853080 learned_addr learned my addr 172.21.15.107:0/2691853080 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.295+0000 7fb43952c640 1 --2- 172.21.15.107:0/2691853080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb434100da0 0x7fb434197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.295+0000 7fb438d2b640 1 --2- 172.21.15.107:0/2691853080 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb434101760 0x7fb4341984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.295+0000 7fb439d2d640 1 -- 172.21.15.107:0/2691853080 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb434101760 msgr2=0x7fb4341984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.295+0000 7fb439d2d640 1 --2- 172.21.15.107:0/2691853080 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb434101760 0x7fb4341984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.295+0000 7fb439d2d640 1 -- 172.21.15.107:0/2691853080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb434100da0 msgr2=0x7fb434197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.295+0000 7fb439d2d640 1 --2- 172.21.15.107:0/2691853080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb434100da0 0x7fb434197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.295+0000 7fb439d2d640 1 -- 172.21.15.107:0/2691853080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb43419cf50 con 0x7fb4341055e0 2024-04-25T00:23:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.296+0000 7fb439d2d640 1 --2- 172.21.15.107:0/2691853080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4341055e0 0x7fb43419c820 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fb42c00eb10 tx=0x7fb42c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.296+0000 7fb42a7fc640 1 -- 172.21.15.107:0/2691853080 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c004b20 con 0x7fb4341055e0 2024-04-25T00:23:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.296+0000 7fb43b790640 1 -- 172.21.15.107:0/2691853080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb43419d240 con 0x7fb4341055e0 2024-04-25T00:23:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.296+0000 7fb42a7fc640 1 -- 172.21.15.107:0/2691853080 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb42c004cc0 con 0x7fb4341055e0 2024-04-25T00:23:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.296+0000 7fb42a7fc640 1 -- 172.21.15.107:0/2691853080 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c010760 con 0x7fb4341055e0 2024-04-25T00:23:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.296+0000 7fb43b790640 1 -- 172.21.15.107:0/2691853080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4341a4ad0 con 0x7fb4341055e0 2024-04-25T00:23:02.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.299+0000 7fb42a7fc640 1 -- 172.21.15.107:0/2691853080 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb42c005020 con 0x7fb4341055e0 2024-04-25T00:23:02.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.299+0000 7fb43e579740 1 Processor -- start 2024-04-25T00:23:02.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.299+0000 7fb43e579740 1 -- start start 2024-04-25T00:23:02.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.299+0000 7fb43e579740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576a962a980 0x5576a9628cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.299+0000 7fb42a7fc640 1 --2- 172.21.15.107:0/2691853080 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb4200a2d00 0x7fb4200a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.299+0000 7fb43e579740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576a9629290 0x5576a9629710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.299+0000 7fb43e579740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a9634010 0x5576a9636400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.299+0000 7fb43e579740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5576a958d2c0 con 0x5576a9634010 2024-04-25T00:23:02.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.299+0000 7fb43e579740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5576a9535910 con 0x5576a962a980 2024-04-25T00:23:02.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.299+0000 7fb43e579740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5576a9589280 con 0x5576a9629290 2024-04-25T00:23:02.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.300+0000 7fb439d2d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a9634010 0x5576a9636400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.300+0000 7fb43952c640 1 --2- 172.21.15.107:0/2691853080 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb4200a2d00 0x7fb4200a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.300+0000 7fb439d2d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a9634010 0x5576a9636400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48580/0 (socket says 172.21.15.107:48580) 2024-04-25T00:23:02.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.300+0000 7fb439d2d640 1 -- 172.21.15.107:0/843346131 learned_addr learned my addr 172.21.15.107:0/843346131 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:02.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.300+0000 7fb438d2b640 1 --2- 172.21.15.107:0/843346131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576a962a980 0x5576a9628cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.300+0000 7fb42a7fc640 1 -- 172.21.15.107:0/2691853080 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fb42c0cfe60 con 0x7fb4341055e0 2024-04-25T00:23:02.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.300+0000 7fb438d2b640 1 --2- 172.21.15.107:0/843346131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576a9629290 0x5576a9629710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.300+0000 7fb439d2d640 1 -- 172.21.15.107:0/843346131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576a9629290 msgr2=0x5576a9629710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:02.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.300+0000 7fb439d2d640 1 --2- 172.21.15.107:0/843346131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576a9629290 0x5576a9629710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.300+0000 7fb439d2d640 1 -- 172.21.15.107:0/843346131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576a962a980 msgr2=0x5576a9628cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:02.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.300+0000 7fb439d2d640 1 --2- 172.21.15.107:0/843346131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576a962a980 0x5576a9628cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.300+0000 7fb439d2d640 1 -- 172.21.15.107:0/843346131 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576a9546fb0 con 0x5576a9634010 2024-04-25T00:23:02.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.300+0000 7fb438d2b640 1 --2- 172.21.15.107:0/843346131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576a9629290 0x5576a9629710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:02.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.300+0000 7fb43952c640 1 --2- 172.21.15.107:0/2691853080 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb4200a2d00 0x7fb4200a51c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb430009230 tx=0x7fb4300058d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:02.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.300+0000 7fb439d2d640 1 --2- 172.21.15.107:0/843346131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a9634010 0x5576a9636400 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0a4750 tx=0x7fb42c00efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf2fac12a91551 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:02.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.300+0000 7fb41f7fe640 1 -- 172.21.15.107:0/843346131 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c09bb50 con 0x5576a9634010 2024-04-25T00:23:02.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.300+0000 7fb41f7fe640 1 -- 172.21.15.107:0/843346131 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb42c0bc0d0 con 0x5576a9634010 2024-04-25T00:23:02.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.300+0000 7fb41f7fe640 1 -- 172.21.15.107:0/843346131 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c09c710 con 0x5576a9634010 2024-04-25T00:23:02.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.301+0000 7fb43e579740 1 -- 172.21.15.107:0/843346131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a9634010 msgr2=0x5576a9636400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.301+0000 7fb43e579740 1 --2- 172.21.15.107:0/843346131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a9634010 0x5576a9636400 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fb42c0a4750 tx=0x7fb42c00efd0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.301+0000 7fb43e579740 1 -- 172.21.15.107:0/843346131 shutdown_connections 2024-04-25T00:23:02.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.301+0000 7fb43e579740 1 --2- 172.21.15.107:0/843346131 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a9634010 0x5576a9636400 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.301+0000 7fb43e579740 1 --2- 172.21.15.107:0/843346131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576a9629290 0x5576a9629710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.301+0000 7fb43e579740 1 --2- 172.21.15.107:0/843346131 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576a962a980 0x5576a9628cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.301+0000 7fb43e579740 1 -- 172.21.15.107:0/843346131 >> 172.21.15.107:0/843346131 conn(0x5576a95f1950 msgr2=0x5576a96257f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:02.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.301+0000 7fb43e579740 1 -- 172.21.15.107:0/843346131 shutdown_connections 2024-04-25T00:23:02.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.301+0000 7fb43e579740 1 -- 172.21.15.107:0/843346131 wait complete. 2024-04-25T00:23:02.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.301+0000 7fb43e579740 1 -- start start 2024-04-25T00:23:02.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.301+0000 7fb43e579740 1 Processor -- start 2024-04-25T00:23:02.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.302+0000 7fb43e579740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576a9629290 0x5576a9630fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.302+0000 7fb43e579740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a962a980 0x5576a9631500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.302+0000 7fb43e579740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576a9634010 0x5576a9631a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.302+0000 7fb43e579740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5576a9636ea0 con 0x5576a962a980 2024-04-25T00:23:02.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.302+0000 7fb43e579740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5576a9548e90 con 0x5576a9634010 2024-04-25T00:23:02.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.302+0000 7fb43e579740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5576a9637020 con 0x5576a9629290 2024-04-25T00:23:02.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.302+0000 7fb43e579740 10 client.0 _ll_register_callbacks cb 0x5576a9584970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:02.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.302+0000 7fb438d2b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a962a980 0x5576a9631500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.302+0000 7fb438d2b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a962a980 0x5576a9631500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48588/0 (socket says 172.21.15.107:48588) 2024-04-25T00:23:02.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.302+0000 7fb438d2b640 1 -- 172.21.15.107:0/1555918818 learned_addr learned my addr 172.21.15.107:0/1555918818 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:02.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.302+0000 7fb439d2d640 1 --2- 172.21.15.107:0/1555918818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576a9634010 0x5576a9631a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.302+0000 7fb43952c640 1 --2- 172.21.15.107:0/1555918818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576a9629290 0x5576a9630fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.302+0000 7fb438d2b640 1 -- 172.21.15.107:0/1555918818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576a9629290 msgr2=0x5576a9630fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:02.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.302+0000 7fb438d2b640 1 --2- 172.21.15.107:0/1555918818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576a9629290 0x5576a9630fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.302+0000 7fb438d2b640 1 -- 172.21.15.107:0/1555918818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576a9634010 msgr2=0x5576a9631a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:02.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.302+0000 7fb438d2b640 1 --2- 172.21.15.107:0/1555918818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576a9634010 0x5576a9631a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.302+0000 7fb438d2b640 1 -- 172.21.15.107:0/1555918818 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576a93697e0 con 0x5576a962a980 2024-04-25T00:23:02.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.302+0000 7fb43952c640 1 --2- 172.21.15.107:0/1555918818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576a9629290 0x5576a9630fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:02.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.302+0000 7fb438d2b640 1 --2- 172.21.15.107:0/1555918818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a962a980 0x5576a9631500 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fb4240126b0 tx=0x7fb424012b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d13415da20bbe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:02.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.302+0000 7fb41cff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:02.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.303+0000 7fb41cff9640 1 -- 172.21.15.107:0/1555918818 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424011d70 con 0x5576a962a980 2024-04-25T00:23:02.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.303+0000 7fb41cff9640 1 -- 172.21.15.107:0/1555918818 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fb424004030 con 0x5576a962a980 2024-04-25T00:23:02.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.303+0000 7fb43e579740 1 -- 172.21.15.107:0/1555918818 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb42c0bc0d0 con 0x5576a962a980 2024-04-25T00:23:02.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.303+0000 7fb41cff9640 1 -- 172.21.15.107:0/1555918818 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb424012e20 con 0x5576a962a980 2024-04-25T00:23:02.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.303+0000 7fb43e579740 10 client.17064 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:02.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.303+0000 7fb43e579740 1 -- 172.21.15.107:0/1555918818 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5576a93c9f70 con 0x5576a962a980 2024-04-25T00:23:02.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.303+0000 7fb41cff9640 1 -- 172.21.15.107:0/1555918818 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb42c0bc0d0 con 0x5576a962a980 2024-04-25T00:23:02.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.303+0000 7fb41cff9640 1 -- 172.21.15.107:0/1555918818 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fb42401c070 con 0x5576a962a980 2024-04-25T00:23:02.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.303+0000 7fb43e579740 10 client.17064 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:02.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.303+0000 7fb43e579740 10 client.17064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:02.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.303+0000 7fb43e579740 20 client.17064 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:02.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.303+0000 7fb43e579740 1 --2- 172.21.15.107:0/1555918818 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5576a9644220 0x5576a9646690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.303+0000 7fb43e579740 4 client.17064 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:02.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.303+0000 7fb43e579740 1 -- 172.21.15.107:0/1555918818 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576a95615a0 con 0x5576a9644220 2024-04-25T00:23:02.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.303+0000 7fb43952c640 1 --2- 172.21.15.107:0/1555918818 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5576a9644220 0x5576a9646690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.304+0000 7fb41cff9640 10 client.17064 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:02.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.304+0000 7fb43952c640 1 --2- 172.21.15.107:0/1555918818 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5576a9644220 0x5576a9646690 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e8e05b201f9230a server_cookie=afb398b5311184bc in_seq=0 out_seq=0 2024-04-25T00:23:02.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.304+0000 7fb41cff9640 1 -- 172.21.15.107:0/1555918818 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5576a95615a0 con 0x5576a9644220 2024-04-25T00:23:02.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.304+0000 7fb41cff9640 10 client.17064 handle_command_reply: tid=0 2024-04-25T00:23:02.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.305+0000 7fb43e579740 1 client.17064 shutdown 2024-04-25T00:23:02.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.305+0000 7fb43e579740 1 -- 172.21.15.107:0/1555918818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a962a980 msgr2=0x5576a9631500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.305+0000 7fb43e579740 1 --2- 172.21.15.107:0/1555918818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a962a980 0x5576a9631500 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fb4240126b0 tx=0x7fb424012b70 comp rx=0 tx=0).stop 2024-04-25T00:23:02.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.305+0000 7fb43e579740 1 -- 172.21.15.107:0/1555918818 shutdown_connections 2024-04-25T00:23:02.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.305+0000 7fb43e579740 1 --2- 172.21.15.107:0/1555918818 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5576a9644220 0x5576a9646690 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.305+0000 7fb43e579740 1 --2- 172.21.15.107:0/1555918818 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576a9634010 0x5576a9631a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.305+0000 7fb43e579740 1 --2- 172.21.15.107:0/1555918818 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a962a980 0x5576a9631500 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.305+0000 7fb41cff9640 0 client.17064 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:02.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.305+0000 7fb43e579740 1 --2- 172.21.15.107:0/1555918818 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576a9629290 0x5576a9630fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.305+0000 7fb43e579740 1 -- 172.21.15.107:0/1555918818 >> 172.21.15.107:0/1555918818 conn(0x5576a95f1950 msgr2=0x5576a962b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:02.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.305+0000 7fb43e579740 1 -- 172.21.15.107:0/1555918818 shutdown_connections 2024-04-25T00:23:02.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.305+0000 7fb43e579740 1 -- 172.21.15.107:0/1555918818 wait complete. 2024-04-25T00:23:02.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.305+0000 7fb43e579740 20 client.17064 trim_cache size 0 max 16384 2024-04-25T00:23:02.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.322+0000 7fb43e579740 1 Processor -- start 2024-04-25T00:23:02.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.322+0000 7fb43e579740 1 -- start start 2024-04-25T00:23:02.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.322+0000 7fb43e579740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576a9629290 0x5576a963fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.322+0000 7fb43e579740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576a962a980 0x5576a96247d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.322+0000 7fb43e579740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a9634010 0x5576a964bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.322+0000 7fb43e579740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb4341b13f0 con 0x5576a9634010 2024-04-25T00:23:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.322+0000 7fb43e579740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb4341aecc0 con 0x5576a9629290 2024-04-25T00:23:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.322+0000 7fb43e579740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb4341acb10 con 0x5576a962a980 2024-04-25T00:23:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.322+0000 7fb439d2d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a9634010 0x5576a964bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.322+0000 7fb439d2d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a9634010 0x5576a964bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48592/0 (socket says 172.21.15.107:48592) 2024-04-25T00:23:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.322+0000 7fb439d2d640 1 -- 172.21.15.107:0/489952782 learned_addr learned my addr 172.21.15.107:0/489952782 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.322+0000 7fb43952c640 1 --2- 172.21.15.107:0/489952782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576a9629290 0x5576a963fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.322+0000 7fb438d2b640 1 --2- 172.21.15.107:0/489952782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576a962a980 0x5576a96247d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.323+0000 7fb439d2d640 1 -- 172.21.15.107:0/489952782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576a962a980 msgr2=0x5576a96247d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.323+0000 7fb439d2d640 1 --2- 172.21.15.107:0/489952782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576a962a980 0x5576a96247d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.323+0000 7fb439d2d640 1 -- 172.21.15.107:0/489952782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576a9629290 msgr2=0x5576a963fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:02.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.323+0000 7fb439d2d640 1 --2- 172.21.15.107:0/489952782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576a9629290 0x5576a963fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.323+0000 7fb439d2d640 1 -- 172.21.15.107:0/489952782 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576a962ff70 con 0x5576a9634010 2024-04-25T00:23:02.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.323+0000 7fb439d2d640 1 --2- 172.21.15.107:0/489952782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a9634010 0x5576a964bfa0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fb42c004af0 tx=0x7fb42c009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1e48c0f3fd9466 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:02.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.323+0000 7fb41effd640 1 -- 172.21.15.107:0/489952782 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c09d140 con 0x5576a9634010 2024-04-25T00:23:02.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.323+0000 7fb41effd640 1 -- 172.21.15.107:0/489952782 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb42c09c710 con 0x5576a9634010 2024-04-25T00:23:02.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.323+0000 7fb41effd640 1 -- 172.21.15.107:0/489952782 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c09b970 con 0x5576a9634010 2024-04-25T00:23:02.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.323+0000 7fb43e579740 1 -- 172.21.15.107:0/489952782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a9634010 msgr2=0x5576a964bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.323+0000 7fb43e579740 1 --2- 172.21.15.107:0/489952782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a9634010 0x5576a964bfa0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fb42c004af0 tx=0x7fb42c009200 comp rx=0 tx=0).stop 2024-04-25T00:23:02.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.323+0000 7fb43e579740 1 -- 172.21.15.107:0/489952782 shutdown_connections 2024-04-25T00:23:02.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.323+0000 7fb43e579740 1 --2- 172.21.15.107:0/489952782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a9634010 0x5576a964bfa0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.323+0000 7fb43e579740 1 --2- 172.21.15.107:0/489952782 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576a962a980 0x5576a96247d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.323+0000 7fb43e579740 1 --2- 172.21.15.107:0/489952782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576a9629290 0x5576a963fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.323+0000 7fb43e579740 1 -- 172.21.15.107:0/489952782 >> 172.21.15.107:0/489952782 conn(0x5576a95f1950 msgr2=0x5576a96240c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:02.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.324+0000 7fb43e579740 1 -- 172.21.15.107:0/489952782 shutdown_connections 2024-04-25T00:23:02.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.324+0000 7fb43e579740 1 -- 172.21.15.107:0/489952782 wait complete. 2024-04-25T00:23:02.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.324+0000 7fb43e579740 1 -- start start 2024-04-25T00:23:02.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.324+0000 7fb43e579740 1 Processor -- start 2024-04-25T00:23:02.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.324+0000 7fb43e579740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a9629290 0x5576a9638900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.324+0000 7fb43e579740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576a962a980 0x5576a9638e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.324+0000 7fb43e579740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576a9634010 0x5576a96a27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.324+0000 7fb43e579740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb4341ad180 con 0x5576a9629290 2024-04-25T00:23:02.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.324+0000 7fb43e579740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb4341ae5f0 con 0x5576a962a980 2024-04-25T00:23:02.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.324+0000 7fb43e579740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb4341ad850 con 0x5576a9634010 2024-04-25T00:23:02.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.324+0000 7fb43e579740 10 client.0 _ll_register_callbacks cb 0x7fb42400bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:02.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.324+0000 7fb43952c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a9629290 0x5576a9638900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.324+0000 7fb43952c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a9629290 0x5576a9638900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48604/0 (socket says 172.21.15.107:48604) 2024-04-25T00:23:02.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.324+0000 7fb43952c640 1 -- 172.21.15.107:0/412858538 learned_addr learned my addr 172.21.15.107:0/412858538 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:02.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.325+0000 7fb438d2b640 1 --2- 172.21.15.107:0/412858538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576a962a980 0x5576a9638e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.325+0000 7fb439d2d640 1 --2- 172.21.15.107:0/412858538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576a9634010 0x5576a96a27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.325+0000 7fb43952c640 1 -- 172.21.15.107:0/412858538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576a9634010 msgr2=0x5576a96a27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:02.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.325+0000 7fb43952c640 1 --2- 172.21.15.107:0/412858538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576a9634010 0x5576a96a27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.325+0000 7fb43952c640 1 -- 172.21.15.107:0/412858538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576a962a980 msgr2=0x5576a9638e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:02.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.325+0000 7fb43952c640 1 --2- 172.21.15.107:0/412858538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576a962a980 0x5576a9638e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.325+0000 7fb43952c640 1 -- 172.21.15.107:0/412858538 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576a9261170 con 0x5576a9629290 2024-04-25T00:23:02.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.325+0000 7fb438d2b640 1 --2- 172.21.15.107:0/412858538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576a962a980 0x5576a9638e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:02.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.325+0000 7fb43952c640 1 --2- 172.21.15.107:0/412858538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a9629290 0x5576a9638900 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fb43002a930 tx=0x7fb43003e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44e9a7457c50951 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:02.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.325+0000 7fb41cff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:02.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.325+0000 7fb41cff9640 1 -- 172.21.15.107:0/412858538 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb430035050 con 0x5576a9629290 2024-04-25T00:23:02.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.325+0000 7fb41cff9640 1 -- 172.21.15.107:0/412858538 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fb430003c10 con 0x5576a9629290 2024-04-25T00:23:02.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.325+0000 7fb43e579740 1 -- 172.21.15.107:0/412858538 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb42c09c710 con 0x5576a9629290 2024-04-25T00:23:02.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.325+0000 7fb41cff9640 1 -- 172.21.15.107:0/412858538 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb430062050 con 0x5576a9629290 2024-04-25T00:23:02.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.325+0000 7fb43e579740 10 client.17070 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:02.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.325+0000 7fb43e579740 1 -- 172.21.15.107:0/412858538 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5576a96ab1f0 con 0x5576a9629290 2024-04-25T00:23:02.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.325+0000 7fb41cff9640 1 -- 172.21.15.107:0/412858538 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb42c09c710 con 0x5576a9629290 2024-04-25T00:23:02.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.325+0000 7fb41cff9640 1 -- 172.21.15.107:0/412858538 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fb430031700 con 0x5576a9629290 2024-04-25T00:23:02.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.325+0000 7fb43e579740 10 client.17070 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:02.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.325+0000 7fb43e579740 10 client.17070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:02.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.325+0000 7fb43e579740 20 client.17070 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:02.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.325+0000 7fb43e579740 1 --2- 172.21.15.107:0/412858538 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5576a96ab8c0 0x5576a96add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.325+0000 7fb43e579740 4 client.17070 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:23:02.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.325+0000 7fb43e579740 1 -- 172.21.15.107:0/412858538 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5576a959a800 con 0x5576a96ab8c0 2024-04-25T00:23:02.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.326+0000 7fb438d2b640 1 --2- 172.21.15.107:0/412858538 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5576a96ab8c0 0x5576a96add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.326+0000 7fb41cff9640 10 client.17070 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:02.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.326+0000 7fb438d2b640 1 --2- 172.21.15.107:0/412858538 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5576a96ab8c0 0x5576a96add00 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33413e58b3c4489a server_cookie=eada935d3c9af2eb in_seq=0 out_seq=0 2024-04-25T00:23:02.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.327+0000 7fb41cff9640 1 -- 172.21.15.107:0/412858538 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5576a959a800 con 0x5576a96ab8c0 2024-04-25T00:23:02.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.327+0000 7fb41cff9640 10 client.17070 handle_command_reply: tid=0 2024-04-25T00:23:02.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.327+0000 7fb43e579740 1 client.17070 shutdown 2024-04-25T00:23:02.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.327+0000 7fb43e579740 1 -- 172.21.15.107:0/412858538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a9629290 msgr2=0x5576a9638900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.327+0000 7fb43e579740 1 --2- 172.21.15.107:0/412858538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a9629290 0x5576a9638900 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fb43002a930 tx=0x7fb43003e070 comp rx=0 tx=0).stop 2024-04-25T00:23:02.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.327+0000 7fb43e579740 1 -- 172.21.15.107:0/412858538 shutdown_connections 2024-04-25T00:23:02.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.327+0000 7fb43e579740 1 --2- 172.21.15.107:0/412858538 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5576a96ab8c0 0x5576a96add00 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.327+0000 7fb43e579740 1 --2- 172.21.15.107:0/412858538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5576a9634010 0x5576a96a27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.327+0000 7fb43e579740 1 --2- 172.21.15.107:0/412858538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5576a962a980 0x5576a9638e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.327+0000 7fb43e579740 1 --2- 172.21.15.107:0/412858538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5576a9629290 0x5576a9638900 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.327+0000 7fb41cff9640 0 client.17070 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:02.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.327+0000 7fb43e579740 1 -- 172.21.15.107:0/412858538 >> 172.21.15.107:0/412858538 conn(0x5576a95f1950 msgr2=0x5576a9624950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:02.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.327+0000 7fb43e579740 1 -- 172.21.15.107:0/412858538 shutdown_connections 2024-04-25T00:23:02.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.327+0000 7fb43e579740 1 -- 172.21.15.107:0/412858538 wait complete. 2024-04-25T00:23:02.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.328+0000 7fb43e579740 20 client.17070 trim_cache size 0 max 16384 2024-04-25T00:23:02.335 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:23:02.335 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-04-25T00:23:02.335 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "6ee536ba-f385-4845-b2d0-60b65df96a9d", 2024-04-25T00:23:02.335 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-04-25T00:23:02.335 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:23:02.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.328+0000 7fb43b790640 1 -- 172.21.15.107:0/2691853080 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb4200a2d00 msgr2=0x7fb4200a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.328+0000 7fb43b790640 1 --2- 172.21.15.107:0/2691853080 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb4200a2d00 0x7fb4200a51c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb430009230 tx=0x7fb4300058d0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.328+0000 7fb43b790640 1 -- 172.21.15.107:0/2691853080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4341055e0 msgr2=0x7fb43419c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.328+0000 7fb43b790640 1 --2- 172.21.15.107:0/2691853080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4341055e0 0x7fb43419c820 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fb42c00eb10 tx=0x7fb42c00c2d0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.328+0000 7fb43b790640 1 -- 172.21.15.107:0/2691853080 shutdown_connections 2024-04-25T00:23:02.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.328+0000 7fb43b790640 1 --2- 172.21.15.107:0/2691853080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4341055e0 0x7fb43419c820 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.328+0000 7fb43b790640 1 --2- 172.21.15.107:0/2691853080 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb434101760 0x7fb4341984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.328+0000 7fb43b790640 1 --2- 172.21.15.107:0/2691853080 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb4200a2d00 0x7fb4200a51c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.328+0000 7fb43b790640 1 --2- 172.21.15.107:0/2691853080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb434100da0 0x7fb434197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.328+0000 7fb43b790640 1 -- 172.21.15.107:0/2691853080 >> 172.21.15.107:0/2691853080 conn(0x7fb434076be0 msgr2=0x7fb434103290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:02.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.328+0000 7fb43b790640 1 -- 172.21.15.107:0/2691853080 shutdown_connections 2024-04-25T00:23:02.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.328+0000 7fb43b790640 1 -- 172.21.15.107:0/2691853080 wait complete. 2024-04-25T00:23:02.339 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:23:02.339 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:23:02.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.479+0000 7f18d6784640 1 -- 172.21.15.107:0/1216480497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18d0100600 msgr2=0x7f18d010e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.479+0000 7f18d6784640 1 --2- 172.21.15.107:0/1216480497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18d0100600 0x7f18d010e5e0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f18c8009cd0 tx=0x7f18c802a8e0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.479+0000 7f18d6784640 1 -- 172.21.15.107:0/1216480497 shutdown_connections 2024-04-25T00:23:02.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.479+0000 7f18d6784640 1 --2- 172.21.15.107:0/1216480497 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18d0100600 0x7f18d010e5e0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.479+0000 7f18d6784640 1 --2- 172.21.15.107:0/1216480497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18d00ffc40 0x7f18d01000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.479+0000 7f18d6784640 1 --2- 172.21.15.107:0/1216480497 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18d0107720 0x7f18d0107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.479+0000 7f18d6784640 1 -- 172.21.15.107:0/1216480497 >> 172.21.15.107:0/1216480497 conn(0x7f18d0076bd0 msgr2=0x7f18d00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:02.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.479+0000 7f18d6784640 1 -- 172.21.15.107:0/1216480497 shutdown_connections 2024-04-25T00:23:02.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.479+0000 7f18d6784640 1 -- 172.21.15.107:0/1216480497 wait complete. 2024-04-25T00:23:02.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.480+0000 7f18d6784640 1 Processor -- start 2024-04-25T00:23:02.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.480+0000 7f18d6784640 1 -- start start 2024-04-25T00:23:02.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.480+0000 7f18d6784640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18d00ffc40 0x7f18d0198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.480+0000 7f18d6784640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18d0100600 0x7f18d0198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.480+0000 7f18d6784640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18d0107720 0x7f18d019c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.480+0000 7f18d6784640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0111b30 con 0x7f18d0107720 2024-04-25T00:23:02.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.480+0000 7f18d6784640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01119b0 con 0x7f18d0100600 2024-04-25T00:23:02.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.480+0000 7f18d6784640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f18d0111cb0 con 0x7f18d00ffc40 2024-04-25T00:23:02.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.480+0000 7f18d4d21640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18d0107720 0x7f18d019c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.480+0000 7f18d4d21640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18d0107720 0x7f18d019c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48632/0 (socket says 172.21.15.107:48632) 2024-04-25T00:23:02.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.480+0000 7f18d4d21640 1 -- 172.21.15.107:0/743087801 learned_addr learned my addr 172.21.15.107:0/743087801 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:02.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.480+0000 7f18d4d21640 1 -- 172.21.15.107:0/743087801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18d00ffc40 msgr2=0x7f18d0198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.480+0000 7f18d4d21640 1 --2- 172.21.15.107:0/743087801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18d00ffc40 0x7f18d0198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.480+0000 7f18d4d21640 1 -- 172.21.15.107:0/743087801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18d0100600 msgr2=0x7f18d0198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.480+0000 7f18cf7fe640 1 --2- 172.21.15.107:0/743087801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18d0100600 0x7f18d0198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.480+0000 7f18d4d21640 1 --2- 172.21.15.107:0/743087801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18d0100600 0x7f18d0198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.480+0000 7f18d4d21640 1 -- 172.21.15.107:0/743087801 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d019d020 con 0x7f18d0107720 2024-04-25T00:23:02.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.481+0000 7f18d4d21640 1 --2- 172.21.15.107:0/743087801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18d0107720 0x7f18d019c8f0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f18c8006010 tx=0x7f18c8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:02.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.481+0000 7f18cd7fa640 1 -- 172.21.15.107:0/743087801 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c8004450 con 0x7f18d0107720 2024-04-25T00:23:02.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.481+0000 7f18d6784640 1 -- 172.21.15.107:0/743087801 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d019d2b0 con 0x7f18d0107720 2024-04-25T00:23:02.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.481+0000 7f18cd7fa640 1 -- 172.21.15.107:0/743087801 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18c8003e10 con 0x7f18d0107720 2024-04-25T00:23:02.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.481+0000 7f18cd7fa640 1 -- 172.21.15.107:0/743087801 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c80466e0 con 0x7f18d0107720 2024-04-25T00:23:02.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.481+0000 7f18d6784640 1 -- 172.21.15.107:0/743087801 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d01a4ba0 con 0x7f18d0107720 2024-04-25T00:23:02.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.483+0000 7f18cd7fa640 1 -- 172.21.15.107:0/743087801 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f18c80305a0 con 0x7f18d0107720 2024-04-25T00:23:02.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.483+0000 7f18d9568740 1 Processor -- start 2024-04-25T00:23:02.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.483+0000 7f18d9568740 1 -- start start 2024-04-25T00:23:02.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.484+0000 7f18cd7fa640 1 --2- 172.21.15.107:0/743087801 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f18b80a2d00 0x7f18b80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.484+0000 7f18d9568740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562cb04dd630 0x562cb04dda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.484+0000 7f18d9568740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04ddf50 0x562cb04e80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.484+0000 7f18d9568740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cb04e8620 0x562cb04eaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.484+0000 7f18d9568740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562cb04426b0 con 0x562cb04ddf50 2024-04-25T00:23:02.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.484+0000 7f18d9568740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562cb03ea880 con 0x562cb04e8620 2024-04-25T00:23:02.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.484+0000 7f18d9568740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562cb043e670 con 0x562cb04dd630 2024-04-25T00:23:02.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.484+0000 7f18cf7fe640 1 --2- 172.21.15.107:0/743087801 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f18b80a2d00 0x7f18b80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.484+0000 7f18cf7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04ddf50 0x562cb04e80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.484+0000 7f18cf7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04ddf50 0x562cb04e80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48636/0 (socket says 172.21.15.107:48636) 2024-04-25T00:23:02.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.484+0000 7f18cf7fe640 1 -- 172.21.15.107:0/1603361396 learned_addr learned my addr 172.21.15.107:0/1603361396 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:02.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.484+0000 7f18cffff640 1 --2- 172.21.15.107:0/1603361396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562cb04dd630 0x562cb04dda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.484+0000 7f18cd7fa640 1 -- 172.21.15.107:0/743087801 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f18c802e030 con 0x7f18d0107720 2024-04-25T00:23:02.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.484+0000 7f18d4d21640 1 --2- 172.21.15.107:0/1603361396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cb04e8620 0x562cb04eaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.484+0000 7f18cf7fe640 1 -- 172.21.15.107:0/1603361396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562cb04dd630 msgr2=0x562cb04dda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:02.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.484+0000 7f18cf7fe640 1 --2- 172.21.15.107:0/1603361396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562cb04dd630 0x562cb04dda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.484+0000 7f18cf7fe640 1 -- 172.21.15.107:0/1603361396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cb04e8620 msgr2=0x562cb04eaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:02.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.484+0000 7f18cf7fe640 1 --2- 172.21.15.107:0/1603361396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cb04e8620 0x562cb04eaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.484+0000 7f18cf7fe640 1 -- 172.21.15.107:0/1603361396 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cb02207d0 con 0x562cb04ddf50 2024-04-25T00:23:02.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.484+0000 7f18cffff640 1 --2- 172.21.15.107:0/1603361396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562cb04dd630 0x562cb04dda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:02.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.484+0000 7f18d4d21640 1 --2- 172.21.15.107:0/1603361396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cb04e8620 0x562cb04eaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:02.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.484+0000 7f18cf7fe640 1 --2- 172.21.15.107:0/743087801 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f18b80a2d00 0x7f18b80a51c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f18c40126a0 tx=0x7f18c400d970 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:02.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.484+0000 7f18cf7fe640 1 --2- 172.21.15.107:0/1603361396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04ddf50 0x562cb04e80e0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f18c400f840 tx=0x7f18c4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d583ca24a88c878d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:02.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.485+0000 7f18b67fc640 1 -- 172.21.15.107:0/1603361396 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4022070 con 0x562cb04ddf50 2024-04-25T00:23:02.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.485+0000 7f18b67fc640 1 -- 172.21.15.107:0/1603361396 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18c401cdc0 con 0x562cb04ddf50 2024-04-25T00:23:02.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.485+0000 7f18b67fc640 1 -- 172.21.15.107:0/1603361396 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c401ecb0 con 0x562cb04ddf50 2024-04-25T00:23:02.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.485+0000 7f18d9568740 1 -- 172.21.15.107:0/1603361396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04ddf50 msgr2=0x562cb04e80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.485+0000 7f18d9568740 1 --2- 172.21.15.107:0/1603361396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04ddf50 0x562cb04e80e0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f18c400f840 tx=0x7f18c4019000 comp rx=0 tx=0).stop 2024-04-25T00:23:02.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.485+0000 7f18d9568740 1 -- 172.21.15.107:0/1603361396 shutdown_connections 2024-04-25T00:23:02.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.485+0000 7f18d9568740 1 --2- 172.21.15.107:0/1603361396 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cb04e8620 0x562cb04eaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.485+0000 7f18d9568740 1 --2- 172.21.15.107:0/1603361396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04ddf50 0x562cb04e80e0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.485+0000 7f18d9568740 1 --2- 172.21.15.107:0/1603361396 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562cb04dd630 0x562cb04dda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.485+0000 7f18d9568740 1 -- 172.21.15.107:0/1603361396 >> 172.21.15.107:0/1603361396 conn(0x562cb04d9950 msgr2=0x562cb04d9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:02.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.485+0000 7f18d9568740 1 -- 172.21.15.107:0/1603361396 shutdown_connections 2024-04-25T00:23:02.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.485+0000 7f18d9568740 1 -- 172.21.15.107:0/1603361396 wait complete. 2024-04-25T00:23:02.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.486+0000 7f18d9568740 1 -- start start 2024-04-25T00:23:02.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.486+0000 7f18d9568740 1 Processor -- start 2024-04-25T00:23:02.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.486+0000 7f18d9568740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04dd630 0x562cb04e5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.486+0000 7f18d9568740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562cb04ddf50 0x562cb04e5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.486+0000 7f18d9568740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cb04e8620 0x562cb04e5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.486+0000 7f18d9568740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562cb04ef7c0 con 0x562cb04dd630 2024-04-25T00:23:02.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.486+0000 7f18d9568740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562cb03fdf50 con 0x562cb04e8620 2024-04-25T00:23:02.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.486+0000 7f18d9568740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562cb04ef940 con 0x562cb04ddf50 2024-04-25T00:23:02.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.486+0000 7f18d9568740 10 client.0 _ll_register_callbacks cb 0x562cb0439000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:02.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.486+0000 7f18cffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04dd630 0x562cb04e5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.486+0000 7f18cffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04dd630 0x562cb04e5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48640/0 (socket says 172.21.15.107:48640) 2024-04-25T00:23:02.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.486+0000 7f18cffff640 1 -- 172.21.15.107:0/4049248843 learned_addr learned my addr 172.21.15.107:0/4049248843 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:02.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.486+0000 7f18d4d21640 1 --2- 172.21.15.107:0/4049248843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cb04e8620 0x562cb04e5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.486+0000 7f18cffff640 1 -- 172.21.15.107:0/4049248843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562cb04ddf50 msgr2=0x562cb04e5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:02.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.486+0000 7f18cffff640 1 --2- 172.21.15.107:0/4049248843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562cb04ddf50 0x562cb04e5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.486+0000 7f18cffff640 1 -- 172.21.15.107:0/4049248843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cb04e8620 msgr2=0x562cb04e5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:02.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.486+0000 7f18cffff640 1 --2- 172.21.15.107:0/4049248843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cb04e8620 0x562cb04e5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.486+0000 7f18cffff640 1 -- 172.21.15.107:0/4049248843 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cb01563f0 con 0x562cb04dd630 2024-04-25T00:23:02.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.486+0000 7f18d4d21640 1 --2- 172.21.15.107:0/4049248843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cb04e8620 0x562cb04e5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:02.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.487+0000 7f18cffff640 1 --2- 172.21.15.107:0/4049248843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04dd630 0x562cb04e5530 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f18c000e280 tx=0x7f18c000e740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04e86d7a5b602d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:02.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.487+0000 7f18af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:02.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.487+0000 7f18af7fe640 1 -- 172.21.15.107:0/4049248843 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c0017070 con 0x562cb04dd630 2024-04-25T00:23:02.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.487+0000 7f18af7fe640 1 -- 172.21.15.107:0/4049248843 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f18c0013680 con 0x562cb04dd630 2024-04-25T00:23:02.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.487+0000 7f18d9568740 1 -- 172.21.15.107:0/4049248843 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18c401cdc0 con 0x562cb04dd630 2024-04-25T00:23:02.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.487+0000 7f18af7fe640 1 -- 172.21.15.107:0/4049248843 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f18c0023050 con 0x562cb04dd630 2024-04-25T00:23:02.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.487+0000 7f18d9568740 10 client.17082 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:02.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.487+0000 7f18d9568740 1 -- 172.21.15.107:0/4049248843 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562cb01160f0 con 0x562cb04dd630 2024-04-25T00:23:02.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.487+0000 7f18af7fe640 1 -- 172.21.15.107:0/4049248843 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18c401cdc0 con 0x562cb04dd630 2024-04-25T00:23:02.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.487+0000 7f18af7fe640 1 -- 172.21.15.107:0/4049248843 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f18c0012a80 con 0x562cb04dd630 2024-04-25T00:23:02.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.487+0000 7f18d9568740 10 client.17082 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:02.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.487+0000 7f18d9568740 10 client.17082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:02.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.487+0000 7f18d9568740 20 client.17082 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:02.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.487+0000 7f18d9568740 1 --2- 172.21.15.107:0/4049248843 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562cb04fcd30 0x562cb04ff190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.487+0000 7f18d9568740 4 client.17082 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:02.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.487+0000 7f18d9568740 1 -- 172.21.15.107:0/4049248843 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cb044fbf0 con 0x562cb04fcd30 2024-04-25T00:23:02.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.488+0000 7f18cf7fe640 1 --2- 172.21.15.107:0/4049248843 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562cb04fcd30 0x562cb04ff190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.488+0000 7f18cf7fe640 1 --2- 172.21.15.107:0/4049248843 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562cb04fcd30 0x562cb04ff190 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d89f9886494c3f9 server_cookie=6b2b8ca94392da06 in_seq=0 out_seq=0 2024-04-25T00:23:02.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.488+0000 7f18af7fe640 10 client.17082 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:02.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.489+0000 7f18af7fe640 1 -- 172.21.15.107:0/4049248843 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562cb044fbf0 con 0x562cb04fcd30 2024-04-25T00:23:02.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.489+0000 7f18af7fe640 10 client.17082 handle_command_reply: tid=0 2024-04-25T00:23:02.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.489+0000 7f18d9568740 1 client.17082 shutdown 2024-04-25T00:23:02.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.489+0000 7f18d9568740 1 -- 172.21.15.107:0/4049248843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04dd630 msgr2=0x562cb04e5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.489+0000 7f18d9568740 1 --2- 172.21.15.107:0/4049248843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04dd630 0x562cb04e5530 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f18c000e280 tx=0x7f18c000e740 comp rx=0 tx=0).stop 2024-04-25T00:23:02.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.489+0000 7f18d9568740 1 -- 172.21.15.107:0/4049248843 shutdown_connections 2024-04-25T00:23:02.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.489+0000 7f18d9568740 1 --2- 172.21.15.107:0/4049248843 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562cb04fcd30 0x562cb04ff190 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.489+0000 7f18d9568740 1 --2- 172.21.15.107:0/4049248843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cb04e8620 0x562cb04e5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.489+0000 7f18d9568740 1 --2- 172.21.15.107:0/4049248843 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562cb04ddf50 0x562cb04e5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.489+0000 7f18af7fe640 0 client.17082 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:02.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.489+0000 7f18d9568740 1 --2- 172.21.15.107:0/4049248843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04dd630 0x562cb04e5530 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.489+0000 7f18d9568740 1 -- 172.21.15.107:0/4049248843 >> 172.21.15.107:0/4049248843 conn(0x562cb04d9950 msgr2=0x562cb04df750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:02.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.489+0000 7f18d9568740 1 -- 172.21.15.107:0/4049248843 shutdown_connections 2024-04-25T00:23:02.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.489+0000 7f18d9568740 1 -- 172.21.15.107:0/4049248843 wait complete. 2024-04-25T00:23:02.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.489+0000 7f18d9568740 20 client.17082 trim_cache size 0 max 16384 2024-04-25T00:23:02.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.506+0000 7f18d9568740 1 Processor -- start 2024-04-25T00:23:02.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.506+0000 7f18d9568740 1 -- start start 2024-04-25T00:23:02.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.506+0000 7f18d9568740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562cb04dd630 0x562cb04f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.506+0000 7f18d9568740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cb04ddf50 0x562cb04dfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.506+0000 7f18d9568740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04e8620 0x562cb04d8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.506+0000 7f18d9568740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01af090 con 0x562cb04e8620 2024-04-25T00:23:02.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.506+0000 7f18d9568740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01ac960 con 0x562cb04ddf50 2024-04-25T00:23:02.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.506+0000 7f18d9568740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01aa7b0 con 0x562cb04dd630 2024-04-25T00:23:02.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.507+0000 7f18d4d21640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04e8620 0x562cb04d8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.507+0000 7f18d4d21640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04e8620 0x562cb04d8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48642/0 (socket says 172.21.15.107:48642) 2024-04-25T00:23:02.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.507+0000 7f18d4d21640 1 -- 172.21.15.107:0/802392801 learned_addr learned my addr 172.21.15.107:0/802392801 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:02.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.507+0000 7f18cf7fe640 1 --2- 172.21.15.107:0/802392801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cb04ddf50 0x562cb04dfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.507+0000 7f18cffff640 1 --2- 172.21.15.107:0/802392801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562cb04dd630 0x562cb04f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.507+0000 7f18d4d21640 1 -- 172.21.15.107:0/802392801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562cb04dd630 msgr2=0x562cb04f86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:02.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.507+0000 7f18d4d21640 1 --2- 172.21.15.107:0/802392801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562cb04dd630 0x562cb04f86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.507+0000 7f18d4d21640 1 -- 172.21.15.107:0/802392801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cb04ddf50 msgr2=0x562cb04dfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:02.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.507+0000 7f18d4d21640 1 --2- 172.21.15.107:0/802392801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cb04ddf50 0x562cb04dfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.507+0000 7f18d4d21640 1 -- 172.21.15.107:0/802392801 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cb04d9780 con 0x562cb04e8620 2024-04-25T00:23:02.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.507+0000 7f18d4d21640 1 --2- 172.21.15.107:0/802392801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04e8620 0x562cb04d8970 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f18c80bd350 tx=0x7f18c80b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d12a6ec72cfa556 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:02.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.507+0000 7f18b5ffb640 1 -- 172.21.15.107:0/802392801 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c80a0e60 con 0x562cb04e8620 2024-04-25T00:23:02.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.507+0000 7f18b5ffb640 1 -- 172.21.15.107:0/802392801 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18c8036050 con 0x562cb04e8620 2024-04-25T00:23:02.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.507+0000 7f18b5ffb640 1 -- 172.21.15.107:0/802392801 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c80c6030 con 0x562cb04e8620 2024-04-25T00:23:02.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.508+0000 7f18d9568740 1 -- 172.21.15.107:0/802392801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04e8620 msgr2=0x562cb04d8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.508+0000 7f18d9568740 1 --2- 172.21.15.107:0/802392801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04e8620 0x562cb04d8970 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f18c80bd350 tx=0x7f18c80b9040 comp rx=0 tx=0).stop 2024-04-25T00:23:02.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.508+0000 7f18d9568740 1 -- 172.21.15.107:0/802392801 shutdown_connections 2024-04-25T00:23:02.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.508+0000 7f18d9568740 1 --2- 172.21.15.107:0/802392801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04e8620 0x562cb04d8970 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.508+0000 7f18d9568740 1 --2- 172.21.15.107:0/802392801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cb04ddf50 0x562cb04dfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.508+0000 7f18d9568740 1 --2- 172.21.15.107:0/802392801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562cb04dd630 0x562cb04f86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.508+0000 7f18d9568740 1 -- 172.21.15.107:0/802392801 >> 172.21.15.107:0/802392801 conn(0x562cb04d9950 msgr2=0x562cb04d8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:02.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.508+0000 7f18d9568740 1 -- 172.21.15.107:0/802392801 shutdown_connections 2024-04-25T00:23:02.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.508+0000 7f18d9568740 1 -- 172.21.15.107:0/802392801 wait complete. 2024-04-25T00:23:02.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.508+0000 7f18d9568740 1 -- start start 2024-04-25T00:23:02.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.508+0000 7f18d9568740 1 Processor -- start 2024-04-25T00:23:02.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.508+0000 7f18d9568740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04dd630 0x562cb04f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.508+0000 7f18d9568740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562cb04ddf50 0x562cb04e1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.508+0000 7f18d9568740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cb04e8620 0x562cb04e22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.508+0000 7f18d9568740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01aae20 con 0x562cb04dd630 2024-04-25T00:23:02.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.508+0000 7f18d9568740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01ac290 con 0x562cb04e8620 2024-04-25T00:23:02.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.508+0000 7f18d9568740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f18d01ab4f0 con 0x562cb04ddf50 2024-04-25T00:23:02.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.508+0000 7f18d9568740 10 client.0 _ll_register_callbacks cb 0x7f18c0009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:02.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.509+0000 7f18cffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04dd630 0x562cb04f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.509+0000 7f18cffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04dd630 0x562cb04f4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48656/0 (socket says 172.21.15.107:48656) 2024-04-25T00:23:02.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.509+0000 7f18cffff640 1 -- 172.21.15.107:0/795700584 learned_addr learned my addr 172.21.15.107:0/795700584 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:02.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.509+0000 7f18cf7fe640 1 --2- 172.21.15.107:0/795700584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562cb04ddf50 0x562cb04e1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.509+0000 7f18d4d21640 1 --2- 172.21.15.107:0/795700584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cb04e8620 0x562cb04e22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.509+0000 7f18cffff640 1 -- 172.21.15.107:0/795700584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562cb04ddf50 msgr2=0x562cb04e1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:02.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.509+0000 7f18cffff640 1 --2- 172.21.15.107:0/795700584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562cb04ddf50 0x562cb04e1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.509+0000 7f18cffff640 1 -- 172.21.15.107:0/795700584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cb04e8620 msgr2=0x562cb04e22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:02.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.509+0000 7f18cffff640 1 --2- 172.21.15.107:0/795700584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cb04e8620 0x562cb04e22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.509+0000 7f18cffff640 1 -- 172.21.15.107:0/795700584 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cb04e2920 con 0x562cb04dd630 2024-04-25T00:23:02.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.509+0000 7f18cffff640 1 --2- 172.21.15.107:0/795700584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04dd630 0x562cb04f4e40 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f18c0012e50 tx=0x7f18c001b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adefe4fe89892cac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:02.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.509+0000 7f18af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:02.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.509+0000 7f18af7fe640 1 -- 172.21.15.107:0/795700584 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c0017050 con 0x562cb04dd630 2024-04-25T00:23:02.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.509+0000 7f18af7fe640 1 -- 172.21.15.107:0/795700584 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f18c001b920 con 0x562cb04dd630 2024-04-25T00:23:02.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.509+0000 7f18d9568740 1 -- 172.21.15.107:0/795700584 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18c8036050 con 0x562cb04dd630 2024-04-25T00:23:02.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.509+0000 7f18af7fe640 1 -- 172.21.15.107:0/795700584 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f18c0035050 con 0x562cb04dd630 2024-04-25T00:23:02.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.509+0000 7f18d9568740 10 client.17088 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:02.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.509+0000 7f18d9568740 1 -- 172.21.15.107:0/795700584 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562cb0563130 con 0x562cb04dd630 2024-04-25T00:23:02.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.509+0000 7f18af7fe640 1 -- 172.21.15.107:0/795700584 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18c8036050 con 0x562cb04dd630 2024-04-25T00:23:02.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.510+0000 7f18af7fe640 1 -- 172.21.15.107:0/795700584 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f18c0013c20 con 0x562cb04dd630 2024-04-25T00:23:02.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.510+0000 7f18d9568740 10 client.17088 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.510+0000 7f18d9568740 10 client.17088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.510+0000 7f18d9568740 20 client.17088 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.510+0000 7f18d9568740 1 --2- 172.21.15.107:0/795700584 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562cb0563900 0x562cb0565d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.510+0000 7f18d9568740 4 client.17088 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:23:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.510+0000 7f18d9568740 1 -- 172.21.15.107:0/795700584 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562cb0449f40 con 0x562cb0563900 2024-04-25T00:23:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.510+0000 7f18cf7fe640 1 --2- 172.21.15.107:0/795700584 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562cb0563900 0x562cb0565d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.510+0000 7f18af7fe640 10 client.17088 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.510+0000 7f18cf7fe640 1 --2- 172.21.15.107:0/795700584 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562cb0563900 0x562cb0565d20 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b0a3a35ccc03c10 server_cookie=8a0069d79a4a37e4 in_seq=0 out_seq=0 2024-04-25T00:23:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.510+0000 7f18af7fe640 1 -- 172.21.15.107:0/795700584 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562cb0449f40 con 0x562cb0563900 2024-04-25T00:23:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.510+0000 7f18af7fe640 10 client.17088 handle_command_reply: tid=0 2024-04-25T00:23:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.510+0000 7f18d9568740 1 client.17088 shutdown 2024-04-25T00:23:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.511+0000 7f18d9568740 1 -- 172.21.15.107:0/795700584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04dd630 msgr2=0x562cb04f4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.511+0000 7f18d9568740 1 --2- 172.21.15.107:0/795700584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04dd630 0x562cb04f4e40 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f18c0012e50 tx=0x7f18c001b690 comp rx=0 tx=0).stop 2024-04-25T00:23:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.511+0000 7f18d9568740 1 -- 172.21.15.107:0/795700584 shutdown_connections 2024-04-25T00:23:02.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.511+0000 7f18d9568740 1 --2- 172.21.15.107:0/795700584 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562cb0563900 0x562cb0565d20 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.511+0000 7f18d9568740 1 --2- 172.21.15.107:0/795700584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562cb04e8620 0x562cb04e22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.511+0000 7f18d9568740 1 --2- 172.21.15.107:0/795700584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562cb04ddf50 0x562cb04e1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.511+0000 7f18d9568740 1 --2- 172.21.15.107:0/795700584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562cb04dd630 0x562cb04f4e40 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.511+0000 7f18af7fe640 0 client.17088 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:02.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.511+0000 7f18d9568740 1 -- 172.21.15.107:0/795700584 >> 172.21.15.107:0/795700584 conn(0x562cb04d9950 msgr2=0x562cb04d8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:02.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.511+0000 7f18d9568740 1 -- 172.21.15.107:0/795700584 shutdown_connections 2024-04-25T00:23:02.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.511+0000 7f18d9568740 1 -- 172.21.15.107:0/795700584 wait complete. 2024-04-25T00:23:02.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.511+0000 7f18d9568740 20 client.17088 trim_cache size 0 max 16384 2024-04-25T00:23:02.518 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:23:02.518 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-04-25T00:23:02.519 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-04-25T00:23:02.519 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:23:02.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.511+0000 7f18d6784640 1 -- 172.21.15.107:0/743087801 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f18b80a2d00 msgr2=0x7f18b80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.511+0000 7f18d6784640 1 --2- 172.21.15.107:0/743087801 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f18b80a2d00 0x7f18b80a51c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f18c40126a0 tx=0x7f18c400d970 comp rx=0 tx=0).stop 2024-04-25T00:23:02.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.511+0000 7f18d6784640 1 -- 172.21.15.107:0/743087801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18d0107720 msgr2=0x7f18d019c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.511+0000 7f18d6784640 1 --2- 172.21.15.107:0/743087801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18d0107720 0x7f18d019c8f0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f18c8006010 tx=0x7f18c8003c20 comp rx=0 tx=0).stop 2024-04-25T00:23:02.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.511+0000 7f18d6784640 1 -- 172.21.15.107:0/743087801 shutdown_connections 2024-04-25T00:23:02.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.511+0000 7f18d6784640 1 --2- 172.21.15.107:0/743087801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18d0107720 0x7f18d019c8f0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.511+0000 7f18d6784640 1 --2- 172.21.15.107:0/743087801 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f18b80a2d00 0x7f18b80a51c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.511+0000 7f18d6784640 1 --2- 172.21.15.107:0/743087801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f18d0100600 0x7f18d0198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.511+0000 7f18d6784640 1 --2- 172.21.15.107:0/743087801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f18d00ffc40 0x7f18d0198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.511+0000 7f18d6784640 1 -- 172.21.15.107:0/743087801 >> 172.21.15.107:0/743087801 conn(0x7f18d0076bd0 msgr2=0x7f18d00fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:02.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.511+0000 7f18d6784640 1 -- 172.21.15.107:0/743087801 shutdown_connections 2024-04-25T00:23:02.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.511+0000 7f18d6784640 1 -- 172.21.15.107:0/743087801 wait complete. 2024-04-25T00:23:02.522 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:23:02.522 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:23:02.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.620+0000 7f24d3f42640 1 -- 172.21.15.107:0/1944556817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24cc100fd0 msgr2=0x7f24cc10d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.620+0000 7f24d3f42640 1 --2- 172.21.15.107:0/1944556817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24cc100fd0 0x7f24cc10d4e0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f24c4009cd0 tx=0x7f24c402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.620+0000 7f24d3f42640 1 -- 172.21.15.107:0/1944556817 shutdown_connections 2024-04-25T00:23:02.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.620+0000 7f24d3f42640 1 --2- 172.21.15.107:0/1944556817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24cc100fd0 0x7f24cc10d4e0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.620+0000 7f24d3f42640 1 --2- 172.21.15.107:0/1944556817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24cc100610 0x7f24cc100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.620+0000 7f24d3f42640 1 --2- 172.21.15.107:0/1944556817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24cc1066b0 0x7f24cc106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.620+0000 7f24d3f42640 1 -- 172.21.15.107:0/1944556817 >> 172.21.15.107:0/1944556817 conn(0x7f24cc0769c0 msgr2=0x7f24cc076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:02.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.620+0000 7f24d3f42640 1 -- 172.21.15.107:0/1944556817 shutdown_connections 2024-04-25T00:23:02.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.620+0000 7f24d3f42640 1 -- 172.21.15.107:0/1944556817 wait complete. 2024-04-25T00:23:02.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.621+0000 7f24d3f42640 1 Processor -- start 2024-04-25T00:23:02.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.621+0000 7f24d3f42640 1 -- start start 2024-04-25T00:23:02.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.621+0000 7f24d3f42640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24cc100610 0x7f24cc197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.621+0000 7f24d3f42640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24cc100fd0 0x7f24cc1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.621+0000 7f24d3f42640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24cc1066b0 0x7f24cc19c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.621+0000 7f24d3f42640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc1109f0 con 0x7f24cc100fd0 2024-04-25T00:23:02.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.621+0000 7f24d3f42640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc110870 con 0x7f24cc1066b0 2024-04-25T00:23:02.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.621+0000 7f24d3f42640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f24cc110b70 con 0x7f24cc100610 2024-04-25T00:23:02.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.621+0000 7f24d14dd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24cc100fd0 0x7f24cc1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.621+0000 7f24d14dd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24cc100fd0 0x7f24cc1984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48676/0 (socket says 172.21.15.107:48676) 2024-04-25T00:23:02.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.621+0000 7f24d14dd640 1 -- 172.21.15.107:0/1459816529 learned_addr learned my addr 172.21.15.107:0/1459816529 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:02.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.621+0000 7f24d24df640 1 --2- 172.21.15.107:0/1459816529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24cc1066b0 0x7f24cc19c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.621+0000 7f24d1cde640 1 --2- 172.21.15.107:0/1459816529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24cc100610 0x7f24cc197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.621+0000 7f24d14dd640 1 -- 172.21.15.107:0/1459816529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24cc100610 msgr2=0x7f24cc197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.621+0000 7f24d14dd640 1 --2- 172.21.15.107:0/1459816529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24cc100610 0x7f24cc197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.621+0000 7f24d14dd640 1 -- 172.21.15.107:0/1459816529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24cc1066b0 msgr2=0x7f24cc19c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.621+0000 7f24d14dd640 1 --2- 172.21.15.107:0/1459816529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24cc1066b0 0x7f24cc19c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.621+0000 7f24d14dd640 1 -- 172.21.15.107:0/1459816529 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24cc19cf50 con 0x7f24cc100fd0 2024-04-25T00:23:02.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.622+0000 7f24d14dd640 1 --2- 172.21.15.107:0/1459816529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24cc100fd0 0x7f24cc1984f0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f24bc00c920 tx=0x7f24bc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:02.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.622+0000 7f24c2ffd640 1 -- 172.21.15.107:0/1459816529 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc007c70 con 0x7f24cc100fd0 2024-04-25T00:23:02.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.622+0000 7f24c2ffd640 1 -- 172.21.15.107:0/1459816529 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24bc007e10 con 0x7f24cc100fd0 2024-04-25T00:23:02.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.622+0000 7f24d3f42640 1 -- 172.21.15.107:0/1459816529 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24cc19d240 con 0x7f24cc100fd0 2024-04-25T00:23:02.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.622+0000 7f24c2ffd640 1 -- 172.21.15.107:0/1459816529 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc018890 con 0x7f24cc100fd0 2024-04-25T00:23:02.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.622+0000 7f24d3f42640 1 -- 172.21.15.107:0/1459816529 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24cc1a4ad0 con 0x7f24cc100fd0 2024-04-25T00:23:02.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.623+0000 7f24d3f42640 1 -- 172.21.15.107:0/1459816529 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24cc101d50 con 0x7f24cc100fd0 2024-04-25T00:23:02.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.624+0000 7f24c2ffd640 1 -- 172.21.15.107:0/1459816529 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f24bc018a70 con 0x7f24cc100fd0 2024-04-25T00:23:02.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.625+0000 7f24c2ffd640 1 --2- 172.21.15.107:0/1459816529 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f24b80a2d00 0x7f24b80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.625+0000 7f24d1cde640 1 --2- 172.21.15.107:0/1459816529 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f24b80a2d00 0x7f24b80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.625+0000 7f24c2ffd640 1 -- 172.21.15.107:0/1459816529 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f24bc0cf860 con 0x7f24cc100fd0 2024-04-25T00:23:02.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.625+0000 7f24d1cde640 1 --2- 172.21.15.107:0/1459816529 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f24b80a2d00 0x7f24b80a51c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f24c80058c0 tx=0x7f24c800a2d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:02.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.629+0000 7f24c2ffd640 1 -- 172.21.15.107:0/1459816529 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f24bc014030 con 0x7f24cc100fd0 2024-04-25T00:23:02.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.845+0000 7f24d3f42640 1 -- 172.21.15.107:0/1459816529 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24cc1992b0 con 0x7f24cc100fd0 2024-04-25T00:23:02.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.846+0000 7f24c2ffd640 1 -- 172.21.15.107:0/1459816529 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11897 (secure 0 0 0) 0x7f24bc09c110 con 0x7f24cc100fd0 2024-04-25T00:23:02.846 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:23:02.846 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:22:27.402680+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24493,24521]},"id":1}]} 2024-04-25T00:23:02.847 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 34 2024-04-25T00:23:02.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.847+0000 7f24d3f42640 1 -- 172.21.15.107:0/1459816529 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f24b80a2d00 msgr2=0x7f24b80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.847+0000 7f24d3f42640 1 --2- 172.21.15.107:0/1459816529 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f24b80a2d00 0x7f24b80a51c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f24c80058c0 tx=0x7f24c800a2d0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.847+0000 7f24d3f42640 1 -- 172.21.15.107:0/1459816529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24cc100fd0 msgr2=0x7f24cc1984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.847+0000 7f24d3f42640 1 --2- 172.21.15.107:0/1459816529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24cc100fd0 0x7f24cc1984f0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f24bc00c920 tx=0x7f24bc00cde0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.847+0000 7f24d3f42640 1 -- 172.21.15.107:0/1459816529 shutdown_connections 2024-04-25T00:23:02.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.848+0000 7f24d3f42640 1 --2- 172.21.15.107:0/1459816529 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f24b80a2d00 0x7f24b80a51c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.848+0000 7f24d3f42640 1 --2- 172.21.15.107:0/1459816529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24cc1066b0 0x7f24cc19c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.848+0000 7f24d3f42640 1 --2- 172.21.15.107:0/1459816529 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24cc100fd0 0x7f24cc1984f0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.848+0000 7f24d3f42640 1 --2- 172.21.15.107:0/1459816529 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24cc100610 0x7f24cc197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.848+0000 7f24d3f42640 1 -- 172.21.15.107:0/1459816529 >> 172.21.15.107:0/1459816529 conn(0x7f24cc0769c0 msgr2=0x7f24cc1046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:02.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.848+0000 7f24d3f42640 1 -- 172.21.15.107:0/1459816529 shutdown_connections 2024-04-25T00:23:02.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.848+0000 7f24d3f42640 1 -- 172.21.15.107:0/1459816529 wait complete. 2024-04-25T00:23:02.858 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:02 smithi107 ceph-mon[29153]: scrub queued for path: / 2024-04-25T00:23:02.858 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:02 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:02.858 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:02 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:23:02.859 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:02 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:02.859 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:02 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:23:02.861 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:23:02.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.962+0000 7fc279258640 1 -- 172.21.15.107:0/544842986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc274101330 msgr2=0x7fc27410d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.962+0000 7fc279258640 1 --2- 172.21.15.107:0/544842986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc274101330 0x7fc27410d460 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009ca0 tx=0x7fc26c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.962+0000 7fc279258640 1 -- 172.21.15.107:0/544842986 shutdown_connections 2024-04-25T00:23:02.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.962+0000 7fc279258640 1 --2- 172.21.15.107:0/544842986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc274101330 0x7fc27410d460 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.962+0000 7fc279258640 1 --2- 172.21.15.107:0/544842986 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc274100990 0x7fc274100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.962+0000 7fc279258640 1 --2- 172.21.15.107:0/544842986 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc274106690 0x7fc274106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.962+0000 7fc279258640 1 -- 172.21.15.107:0/544842986 >> 172.21.15.107:0/544842986 conn(0x7fc274076440 msgr2=0x7fc274076850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:02.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.962+0000 7fc279258640 1 -- 172.21.15.107:0/544842986 shutdown_connections 2024-04-25T00:23:02.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.962+0000 7fc279258640 1 -- 172.21.15.107:0/544842986 wait complete. 2024-04-25T00:23:02.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.963+0000 7fc279258640 1 Processor -- start 2024-04-25T00:23:02.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.963+0000 7fc279258640 1 -- start start 2024-04-25T00:23:02.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.963+0000 7fc279258640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc274100990 0x7fc274198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.963+0000 7fc279258640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc274101330 0x7fc274198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.963+0000 7fc279258640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc274106690 0x7fc27419c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.963+0000 7fc279258640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc274110a20 con 0x7fc274101330 2024-04-25T00:23:02.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.963+0000 7fc279258640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc2741108a0 con 0x7fc274100990 2024-04-25T00:23:02.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.963+0000 7fc279258640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc274110ba0 con 0x7fc274106690 2024-04-25T00:23:02.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.963+0000 7fc27259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc274101330 0x7fc274198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.963+0000 7fc27259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc274101330 0x7fc274198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48682/0 (socket says 172.21.15.107:48682) 2024-04-25T00:23:02.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.963+0000 7fc27259c640 1 -- 172.21.15.107:0/1440049789 learned_addr learned my addr 172.21.15.107:0/1440049789 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:02.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.963+0000 7fc272d9d640 1 --2- 172.21.15.107:0/1440049789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc274100990 0x7fc274198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.963+0000 7fc27359e640 1 --2- 172.21.15.107:0/1440049789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc274106690 0x7fc27419c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.963+0000 7fc27259c640 1 -- 172.21.15.107:0/1440049789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc274106690 msgr2=0x7fc27419c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.963+0000 7fc27259c640 1 --2- 172.21.15.107:0/1440049789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc274106690 0x7fc27419c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.963+0000 7fc27259c640 1 -- 172.21.15.107:0/1440049789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc274100990 msgr2=0x7fc274198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.963+0000 7fc27259c640 1 --2- 172.21.15.107:0/1440049789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc274100990 0x7fc274198010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.963+0000 7fc27259c640 1 -- 172.21.15.107:0/1440049789 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27419d000 con 0x7fc274101330 2024-04-25T00:23:02.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.963+0000 7fc27359e640 1 --2- 172.21.15.107:0/1440049789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc274106690 0x7fc27419c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:02.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.964+0000 7fc27259c640 1 --2- 172.21.15.107:0/1440049789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc274101330 0x7fc274198550 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fc26400d770 tx=0x7fc26400db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:02.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.964+0000 7fc25bfff640 1 -- 172.21.15.107:0/1440049789 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264017070 con 0x7fc274101330 2024-04-25T00:23:02.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.964+0000 7fc279258640 1 -- 172.21.15.107:0/1440049789 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc27419d2f0 con 0x7fc274101330 2024-04-25T00:23:02.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.964+0000 7fc25bfff640 1 -- 172.21.15.107:0/1440049789 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc264009820 con 0x7fc274101330 2024-04-25T00:23:02.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.964+0000 7fc25bfff640 1 -- 172.21.15.107:0/1440049789 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264004cd0 con 0x7fc274101330 2024-04-25T00:23:02.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.964+0000 7fc279258640 1 -- 172.21.15.107:0/1440049789 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2741a4b80 con 0x7fc274101330 2024-04-25T00:23:02.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.966+0000 7fc25bfff640 1 -- 172.21.15.107:0/1440049789 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc2640099d0 con 0x7fc274101330 2024-04-25T00:23:02.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.966+0000 7fc27bf64740 1 Processor -- start 2024-04-25T00:23:02.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.966+0000 7fc27bf64740 1 -- start start 2024-04-25T00:23:02.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.966+0000 7fc27bf64740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562cd1a5460 0x5562cd1a5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.966+0000 7fc27bf64740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1a5d80 0x5562cd1b4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.966+0000 7fc27bf64740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562cd1b4980 0x5562cd1b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.966+0000 7fc27bf64740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5562cd10a670 con 0x5562cd1a5d80 2024-04-25T00:23:02.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.966+0000 7fc27bf64740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5562cd0b2970 con 0x5562cd1b4980 2024-04-25T00:23:02.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.966+0000 7fc27bf64740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5562cd106630 con 0x5562cd1a5460 2024-04-25T00:23:02.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.966+0000 7fc272d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1a5d80 0x5562cd1b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.966+0000 7fc272d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1a5d80 0x5562cd1b4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48694/0 (socket says 172.21.15.107:48694) 2024-04-25T00:23:02.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.966+0000 7fc272d9d640 1 -- 172.21.15.107:0/2605505459 learned_addr learned my addr 172.21.15.107:0/2605505459 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:02.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.967+0000 7fc25bfff640 1 --2- 172.21.15.107:0/1440049789 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc25c0a2d00 0x7fc25c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.967+0000 7fc27359e640 1 --2- 172.21.15.107:0/2605505459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562cd1a5460 0x5562cd1a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.967+0000 7fc27259c640 1 --2- 172.21.15.107:0/2605505459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562cd1b4980 0x5562cd1b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.967+0000 7fc27359e640 1 --2- 172.21.15.107:0/1440049789 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc25c0a2d00 0x7fc25c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.967+0000 7fc272d9d640 1 -- 172.21.15.107:0/2605505459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562cd1a5460 msgr2=0x5562cd1a5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:02.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.967+0000 7fc272d9d640 1 --2- 172.21.15.107:0/2605505459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562cd1a5460 0x5562cd1a5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.967+0000 7fc272d9d640 1 -- 172.21.15.107:0/2605505459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562cd1b4980 msgr2=0x5562cd1b6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:02.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.967+0000 7fc25bfff640 1 -- 172.21.15.107:0/1440049789 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fc2640cf200 con 0x7fc274101330 2024-04-25T00:23:02.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.967+0000 7fc272d9d640 1 --2- 172.21.15.107:0/2605505459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562cd1b4980 0x5562cd1b6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.967+0000 7fc272d9d640 1 -- 172.21.15.107:0/2605505459 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562cd0d6400 con 0x5562cd1a5d80 2024-04-25T00:23:02.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.967+0000 7fc27259c640 1 --2- 172.21.15.107:0/2605505459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562cd1b4980 0x5562cd1b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:02.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.967+0000 7fc272d9d640 1 --2- 172.21.15.107:0/2605505459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1a5d80 0x5562cd1b4440 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fc268009a70 tx=0x7fc268009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584ac2bf4745ab05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:02.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.967+0000 7fc27359e640 1 --2- 172.21.15.107:0/1440049789 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc25c0a2d00 0x7fc25c0a51c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc26c005e30 tx=0x7fc26c0095c0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:02.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.967+0000 7fc2597fa640 1 -- 172.21.15.107:0/2605505459 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268017070 con 0x5562cd1a5d80 2024-04-25T00:23:02.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.967+0000 7fc2597fa640 1 -- 172.21.15.107:0/2605505459 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc268013470 con 0x5562cd1a5d80 2024-04-25T00:23:02.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.967+0000 7fc2597fa640 1 -- 172.21.15.107:0/2605505459 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268012630 con 0x5562cd1a5d80 2024-04-25T00:23:02.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.968+0000 7fc27bf64740 1 -- 172.21.15.107:0/2605505459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1a5d80 msgr2=0x5562cd1b4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.968+0000 7fc27bf64740 1 --2- 172.21.15.107:0/2605505459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1a5d80 0x5562cd1b4440 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fc268009a70 tx=0x7fc268009f30 comp rx=0 tx=0).stop 2024-04-25T00:23:02.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.968+0000 7fc27bf64740 1 -- 172.21.15.107:0/2605505459 shutdown_connections 2024-04-25T00:23:02.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.968+0000 7fc27bf64740 1 --2- 172.21.15.107:0/2605505459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562cd1b4980 0x5562cd1b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.968+0000 7fc27bf64740 1 --2- 172.21.15.107:0/2605505459 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1a5d80 0x5562cd1b4440 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.968+0000 7fc27bf64740 1 --2- 172.21.15.107:0/2605505459 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562cd1a5460 0x5562cd1a5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.968+0000 7fc27bf64740 1 -- 172.21.15.107:0/2605505459 >> 172.21.15.107:0/2605505459 conn(0x5562cd1a18a0 msgr2=0x5562cd1a1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:02.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.968+0000 7fc27bf64740 1 -- 172.21.15.107:0/2605505459 shutdown_connections 2024-04-25T00:23:02.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.968+0000 7fc27bf64740 1 -- 172.21.15.107:0/2605505459 wait complete. 2024-04-25T00:23:02.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.968+0000 7fc27bf64740 1 -- start start 2024-04-25T00:23:02.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.968+0000 7fc27bf64740 1 Processor -- start 2024-04-25T00:23:02.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.968+0000 7fc27bf64740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562cd1a5460 0x5562cd1b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.968+0000 7fc27bf64740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1a5d80 0x5562cd1b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc27bf64740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562cd1b4980 0x5562cd1b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc27bf64740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5562cd1b77c0 con 0x5562cd1a5d80 2024-04-25T00:23:02.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc27bf64740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5562cd0c6050 con 0x5562cd1a5460 2024-04-25T00:23:02.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc27bf64740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5562cd1b7940 con 0x5562cd1b4980 2024-04-25T00:23:02.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc27bf64740 10 client.0 _ll_register_callbacks cb 0x5562cce1dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:02.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc27259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1a5d80 0x5562cd1b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc27259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1a5d80 0x5562cd1b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48700/0 (socket says 172.21.15.107:48700) 2024-04-25T00:23:02.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc27259c640 1 -- 172.21.15.107:0/3353933456 learned_addr learned my addr 172.21.15.107:0/3353933456 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:02.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc27359e640 1 --2- 172.21.15.107:0/3353933456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562cd1b4980 0x5562cd1b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc272d9d640 1 --2- 172.21.15.107:0/3353933456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562cd1a5460 0x5562cd1b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc27259c640 1 -- 172.21.15.107:0/3353933456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562cd1b4980 msgr2=0x5562cd1b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:02.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc27259c640 1 --2- 172.21.15.107:0/3353933456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562cd1b4980 0x5562cd1b1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc27259c640 1 -- 172.21.15.107:0/3353933456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562cd1a5460 msgr2=0x5562cd1b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:02.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc27259c640 1 --2- 172.21.15.107:0/3353933456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562cd1a5460 0x5562cd1b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc27259c640 1 -- 172.21.15.107:0/3353933456 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562cd097dc0 con 0x5562cd1a5d80 2024-04-25T00:23:02.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc27359e640 1 --2- 172.21.15.107:0/3353933456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562cd1b4980 0x5562cd1b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:02.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc27259c640 1 --2- 172.21.15.107:0/3353933456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1a5d80 0x5562cd1b1510 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fc2640ccbe0 tx=0x7fc264018a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fde0b45162cc07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:02.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc256ff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:02.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc256ff5640 1 -- 172.21.15.107:0/3353933456 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264020040 con 0x5562cd1a5d80 2024-04-25T00:23:02.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc256ff5640 1 -- 172.21.15.107:0/3353933456 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fc2640cc860 con 0x5562cd1a5d80 2024-04-25T00:23:02.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc27bf64740 1 -- 172.21.15.107:0/3353933456 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc268013470 con 0x5562cd1a5d80 2024-04-25T00:23:02.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc256ff5640 1 -- 172.21.15.107:0/3353933456 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc26409f760 con 0x5562cd1a5d80 2024-04-25T00:23:02.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc27bf64740 10 client.17106 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:02.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc27bf64740 1 -- 172.21.15.107:0/3353933456 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5562cce1e3f0 con 0x5562cd1a5d80 2024-04-25T00:23:02.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.969+0000 7fc256ff5640 1 -- 172.21.15.107:0/3353933456 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc268013470 con 0x5562cd1a5d80 2024-04-25T00:23:02.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.970+0000 7fc256ff5640 1 -- 172.21.15.107:0/3353933456 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fc2640b41b0 con 0x5562cd1a5d80 2024-04-25T00:23:02.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.970+0000 7fc27bf64740 10 client.17106 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:02.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.970+0000 7fc27bf64740 10 client.17106 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:23:02.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.970+0000 7fc27bf64740 20 client.17106 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:02.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.970+0000 7fc27bf64740 1 --2- 172.21.15.107:0/3353933456 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5562cd1c4840 0x5562cd1c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.970+0000 7fc27bf64740 4 client.17106 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:02.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.970+0000 7fc27bf64740 1 -- 172.21.15.107:0/3353933456 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562cd117bb0 con 0x5562cd1c4840 2024-04-25T00:23:02.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.970+0000 7fc272d9d640 1 --2- 172.21.15.107:0/3353933456 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5562cd1c4840 0x5562cd1c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.970+0000 7fc256ff5640 10 client.17106 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:23:02.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.970+0000 7fc272d9d640 1 --2- 172.21.15.107:0/3353933456 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5562cd1c4840 0x5562cd1c6c50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e342bb66573318bb server_cookie=93d79f5f92e4534c in_seq=0 out_seq=0 2024-04-25T00:23:02.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.971+0000 7fc256ff5640 1 -- 172.21.15.107:0/3353933456 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5562cd117bb0 con 0x5562cd1c4840 2024-04-25T00:23:02.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.971+0000 7fc256ff5640 10 client.17106 handle_command_reply: tid=0 2024-04-25T00:23:02.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.971+0000 7fc27bf64740 1 client.17106 shutdown 2024-04-25T00:23:02.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.971+0000 7fc27bf64740 1 -- 172.21.15.107:0/3353933456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1a5d80 msgr2=0x5562cd1b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.971+0000 7fc27bf64740 1 --2- 172.21.15.107:0/3353933456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1a5d80 0x5562cd1b1510 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fc2640ccbe0 tx=0x7fc264018a40 comp rx=0 tx=0).stop 2024-04-25T00:23:02.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.971+0000 7fc27bf64740 1 -- 172.21.15.107:0/3353933456 shutdown_connections 2024-04-25T00:23:02.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.971+0000 7fc27bf64740 1 --2- 172.21.15.107:0/3353933456 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5562cd1c4840 0x5562cd1c6c50 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.971+0000 7fc27bf64740 1 --2- 172.21.15.107:0/3353933456 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562cd1b4980 0x5562cd1b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.971+0000 7fc27bf64740 1 --2- 172.21.15.107:0/3353933456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1a5d80 0x5562cd1b1510 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.971+0000 7fc256ff5640 0 client.17106 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:23:02.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.971+0000 7fc27bf64740 1 --2- 172.21.15.107:0/3353933456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562cd1a5460 0x5562cd1b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.971+0000 7fc27bf64740 1 -- 172.21.15.107:0/3353933456 >> 172.21.15.107:0/3353933456 conn(0x5562cd1a18a0 msgr2=0x5562cd1a7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:02.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.971+0000 7fc27bf64740 1 -- 172.21.15.107:0/3353933456 shutdown_connections 2024-04-25T00:23:02.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.971+0000 7fc27bf64740 1 -- 172.21.15.107:0/3353933456 wait complete. 2024-04-25T00:23:02.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.971+0000 7fc27bf64740 20 client.17106 trim_cache size 0 max 16384 2024-04-25T00:23:02.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.985+0000 7fc27bf64740 1 Processor -- start 2024-04-25T00:23:02.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.985+0000 7fc27bf64740 1 -- start start 2024-04-25T00:23:02.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.985+0000 7fc27bf64740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562cd1a5460 0x5562cd1c0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.985+0000 7fc27bf64740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1a5d80 0x5562cd1a74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.985+0000 7fc27bf64740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562cd1b4980 0x5562cd1b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.985+0000 7fc27bf64740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc2741af0d0 con 0x5562cd1a5d80 2024-04-25T00:23:02.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.985+0000 7fc27bf64740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc2741ac9a0 con 0x5562cd1a5460 2024-04-25T00:23:02.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.985+0000 7fc27bf64740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc2741aa7f0 con 0x5562cd1b4980 2024-04-25T00:23:02.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.986+0000 7fc27259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1a5d80 0x5562cd1a74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.986+0000 7fc27259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1a5d80 0x5562cd1a74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48702/0 (socket says 172.21.15.107:48702) 2024-04-25T00:23:02.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.986+0000 7fc27259c640 1 -- 172.21.15.107:0/3477119643 learned_addr learned my addr 172.21.15.107:0/3477119643 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:02.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.986+0000 7fc27359e640 1 --2- 172.21.15.107:0/3477119643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562cd1b4980 0x5562cd1b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.986+0000 7fc27259c640 1 -- 172.21.15.107:0/3477119643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562cd1b4980 msgr2=0x5562cd1b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:02.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.986+0000 7fc27259c640 1 --2- 172.21.15.107:0/3477119643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562cd1b4980 0x5562cd1b3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.986+0000 7fc27259c640 1 -- 172.21.15.107:0/3477119643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562cd1a5460 msgr2=0x5562cd1c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:02.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.986+0000 7fc27259c640 1 --2- 172.21.15.107:0/3477119643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562cd1a5460 0x5562cd1c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.986+0000 7fc27259c640 1 -- 172.21.15.107:0/3477119643 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562ccee87d0 con 0x5562cd1a5d80 2024-04-25T00:23:02.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.986+0000 7fc27259c640 1 --2- 172.21.15.107:0/3477119643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1a5d80 0x5562cd1a74d0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fc264097d80 tx=0x7fc264009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=114fa9a9c4d95c8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:02.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.986+0000 7fc258ff9640 1 -- 172.21.15.107:0/3477119643 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264095030 con 0x5562cd1a5d80 2024-04-25T00:23:02.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.986+0000 7fc258ff9640 1 -- 172.21.15.107:0/3477119643 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2640cc860 con 0x5562cd1a5d80 2024-04-25T00:23:02.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.986+0000 7fc258ff9640 1 -- 172.21.15.107:0/3477119643 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264020070 con 0x5562cd1a5d80 2024-04-25T00:23:02.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.987+0000 7fc27bf64740 1 -- 172.21.15.107:0/3477119643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1a5d80 msgr2=0x5562cd1a74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:02.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.987+0000 7fc27bf64740 1 --2- 172.21.15.107:0/3477119643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1a5d80 0x5562cd1a74d0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fc264097d80 tx=0x7fc264009f90 comp rx=0 tx=0).stop 2024-04-25T00:23:02.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.987+0000 7fc27bf64740 1 -- 172.21.15.107:0/3477119643 shutdown_connections 2024-04-25T00:23:02.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.987+0000 7fc27bf64740 1 --2- 172.21.15.107:0/3477119643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562cd1b4980 0x5562cd1b3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.987+0000 7fc27bf64740 1 --2- 172.21.15.107:0/3477119643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1a5d80 0x5562cd1a74d0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.987+0000 7fc27bf64740 1 --2- 172.21.15.107:0/3477119643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562cd1a5460 0x5562cd1c0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.987+0000 7fc27bf64740 1 -- 172.21.15.107:0/3477119643 >> 172.21.15.107:0/3477119643 conn(0x5562cd1a18a0 msgr2=0x5562cd1a6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:02.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.987+0000 7fc27bf64740 1 -- 172.21.15.107:0/3477119643 shutdown_connections 2024-04-25T00:23:02.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.987+0000 7fc27bf64740 1 -- 172.21.15.107:0/3477119643 wait complete. 2024-04-25T00:23:02.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.987+0000 7fc27bf64740 1 -- start start 2024-04-25T00:23:02.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.987+0000 7fc27bf64740 1 Processor -- start 2024-04-25T00:23:02.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc27bf64740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562cd1a5460 0x5562cd1bcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc27bf64740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562cd1a5d80 0x5562cd1b81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc27bf64740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1b4980 0x5562cd1b8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:02.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc27bf64740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc2741aae60 con 0x5562cd1b4980 2024-04-25T00:23:02.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc27bf64740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc2741ac2d0 con 0x5562cd1a5460 2024-04-25T00:23:02.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc27bf64740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc2741ab530 con 0x5562cd1a5d80 2024-04-25T00:23:02.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc27bf64740 10 client.0 _ll_register_callbacks cb 0x7fc2640046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:02.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc27359e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1b4980 0x5562cd1b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc27359e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1b4980 0x5562cd1b8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48718/0 (socket says 172.21.15.107:48718) 2024-04-25T00:23:02.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc27359e640 1 -- 172.21.15.107:0/1392607214 learned_addr learned my addr 172.21.15.107:0/1392607214 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:02.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc27259c640 1 --2- 172.21.15.107:0/1392607214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562cd1a5d80 0x5562cd1b81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc27359e640 1 -- 172.21.15.107:0/1392607214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562cd1a5d80 msgr2=0x5562cd1b81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:02.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc27359e640 1 --2- 172.21.15.107:0/1392607214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562cd1a5d80 0x5562cd1b81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc272d9d640 1 --2- 172.21.15.107:0/1392607214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562cd1a5460 0x5562cd1bcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:02.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc27359e640 1 -- 172.21.15.107:0/1392607214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562cd1a5460 msgr2=0x5562cd1bcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:02.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc27359e640 1 --2- 172.21.15.107:0/1392607214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562cd1a5460 0x5562cd1bcd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:02.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc27359e640 1 -- 172.21.15.107:0/1392607214 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562cd1b8c60 con 0x5562cd1b4980 2024-04-25T00:23:02.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc27359e640 1 --2- 172.21.15.107:0/1392607214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1b4980 0x5562cd1b8720 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009ca0 tx=0x7fc26c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7029a8f83d837afb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:02.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc256ff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:02.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc256ff5640 1 -- 172.21.15.107:0/1392607214 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c042040 con 0x5562cd1b4980 2024-04-25T00:23:02.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc256ff5640 1 -- 172.21.15.107:0/1392607214 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fc26c004460 con 0x5562cd1b4980 2024-04-25T00:23:02.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc27bf64740 1 -- 172.21.15.107:0/1392607214 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2640cc860 con 0x5562cd1b4980 2024-04-25T00:23:02.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc256ff5640 1 -- 172.21.15.107:0/1392607214 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc26c05d050 con 0x5562cd1b4980 2024-04-25T00:23:02.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc27bf64740 10 client.17112 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:02.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.988+0000 7fc27bf64740 1 -- 172.21.15.107:0/1392607214 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5562cd22aa70 con 0x5562cd1b4980 2024-04-25T00:23:02.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.989+0000 7fc256ff5640 1 -- 172.21.15.107:0/1392607214 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2640cc860 con 0x5562cd1b4980 2024-04-25T00:23:02.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.989+0000 7fc256ff5640 1 -- 172.21.15.107:0/1392607214 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fc26c048070 con 0x5562cd1b4980 2024-04-25T00:23:02.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.989+0000 7fc27bf64740 10 client.17112 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:02.999 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.989+0000 7fc27bf64740 10 client.17112 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:23:03.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.989+0000 7fc27bf64740 20 client.17112 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:03.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.989+0000 7fc27bf64740 1 --2- 172.21.15.107:0/1392607214 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5562cd22b220 0x5562cd22d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.989+0000 7fc27bf64740 4 client.17112 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:23:03.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.989+0000 7fc27bf64740 1 -- 172.21.15.107:0/1392607214 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5562cd1ad110 con 0x5562cd22b220 2024-04-25T00:23:03.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.989+0000 7fc272d9d640 1 --2- 172.21.15.107:0/1392607214 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5562cd22b220 0x5562cd22d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.989+0000 7fc272d9d640 1 --2- 172.21.15.107:0/1392607214 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5562cd22b220 0x5562cd22d640 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f0dc0f923baed22 server_cookie=30577c5f3def4558 in_seq=0 out_seq=0 2024-04-25T00:23:03.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.989+0000 7fc256ff5640 10 client.17112 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:23:03.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc256ff5640 1 -- 172.21.15.107:0/1392607214 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5562cd1ad110 con 0x5562cd22b220 2024-04-25T00:23:03.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc256ff5640 10 client.17112 handle_command_reply: tid=0 2024-04-25T00:23:03.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc27bf64740 1 client.17112 shutdown 2024-04-25T00:23:03.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc27bf64740 1 -- 172.21.15.107:0/1392607214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1b4980 msgr2=0x5562cd1b8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:03.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc27bf64740 1 --2- 172.21.15.107:0/1392607214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1b4980 0x5562cd1b8720 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009ca0 tx=0x7fc26c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc27bf64740 1 -- 172.21.15.107:0/1392607214 shutdown_connections 2024-04-25T00:23:03.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc27bf64740 1 --2- 172.21.15.107:0/1392607214 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x5562cd22b220 0x5562cd22d640 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc27bf64740 1 --2- 172.21.15.107:0/1392607214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562cd1b4980 0x5562cd1b8720 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc27bf64740 1 --2- 172.21.15.107:0/1392607214 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5562cd1a5d80 0x5562cd1b81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc256ff5640 0 client.17112 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:23:03.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc27bf64740 1 --2- 172.21.15.107:0/1392607214 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562cd1a5460 0x5562cd1bcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc27bf64740 1 -- 172.21.15.107:0/1392607214 >> 172.21.15.107:0/1392607214 conn(0x5562cd1a18a0 msgr2=0x5562cd1a02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:03.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc27bf64740 1 -- 172.21.15.107:0/1392607214 shutdown_connections 2024-04-25T00:23:03.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc27bf64740 1 -- 172.21.15.107:0/1392607214 wait complete. 2024-04-25T00:23:03.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc27bf64740 20 client.17112 trim_cache size 0 max 16384 2024-04-25T00:23:03.001 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:23:03.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc279258640 1 -- 172.21.15.107:0/1440049789 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc25c0a2d00 msgr2=0x7fc25c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:03.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc279258640 1 --2- 172.21.15.107:0/1440049789 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc25c0a2d00 0x7fc25c0a51c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc26c005e30 tx=0x7fc26c0095c0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc279258640 1 -- 172.21.15.107:0/1440049789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc274101330 msgr2=0x7fc274198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:03.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc279258640 1 --2- 172.21.15.107:0/1440049789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc274101330 0x7fc274198550 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fc26400d770 tx=0x7fc26400db70 comp rx=0 tx=0).stop 2024-04-25T00:23:03.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc279258640 1 -- 172.21.15.107:0/1440049789 shutdown_connections 2024-04-25T00:23:03.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc279258640 1 --2- 172.21.15.107:0/1440049789 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc274106690 0x7fc27419c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.003 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc279258640 1 --2- 172.21.15.107:0/1440049789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc274101330 0x7fc274198550 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc279258640 1 --2- 172.21.15.107:0/1440049789 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc25c0a2d00 0x7fc25c0a51c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc279258640 1 --2- 172.21.15.107:0/1440049789 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc274100990 0x7fc274198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc279258640 1 -- 172.21.15.107:0/1440049789 >> 172.21.15.107:0/1440049789 conn(0x7fc274076440 msgr2=0x7fc274069fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:03.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc279258640 1 -- 172.21.15.107:0/1440049789 shutdown_connections 2024-04-25T00:23:03.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:02.990+0000 7fc279258640 1 -- 172.21.15.107:0/1440049789 wait complete. 2024-04-25T00:23:03.004 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:23:03.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:02 smithi195 ceph-mon[32833]: scrub queued for path: / 2024-04-25T00:23:03.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:02 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:03.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:02 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:23:03.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:02 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:03.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:02 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:23:03.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.104+0000 7fbc4de31640 1 -- 172.21.15.107:0/2948092995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc48105160 msgr2=0x7fbc4810cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:03.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.104+0000 7fbc4de31640 1 --2- 172.21.15.107:0/2948092995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc48105160 0x7fbc4810cce0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fbc4000adb0 tx=0x7fbc4002b7a0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.104+0000 7fbc4de31640 1 -- 172.21.15.107:0/2948092995 shutdown_connections 2024-04-25T00:23:03.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.104+0000 7fbc4de31640 1 --2- 172.21.15.107:0/2948092995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc48105160 0x7fbc4810cce0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.104+0000 7fbc4de31640 1 --2- 172.21.15.107:0/2948092995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc48100c90 0x7fbc48104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.104+0000 7fbc4de31640 1 --2- 172.21.15.107:0/2948092995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc481002e0 0x7fbc481006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.104+0000 7fbc4de31640 1 -- 172.21.15.107:0/2948092995 >> 172.21.15.107:0/2948092995 conn(0x7fbc480758d0 msgr2=0x7fbc48075ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:03.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.104+0000 7fbc4de31640 1 -- 172.21.15.107:0/2948092995 shutdown_connections 2024-04-25T00:23:03.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.104+0000 7fbc4de31640 1 -- 172.21.15.107:0/2948092995 wait complete. 2024-04-25T00:23:03.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.105+0000 7fbc4de31640 1 Processor -- start 2024-04-25T00:23:03.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.105+0000 7fbc4de31640 1 -- start start 2024-04-25T00:23:03.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.105+0000 7fbc4de31640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc481002e0 0x7fbc48197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.105+0000 7fbc4de31640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc48105160 0x7fbc48198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.105+0000 7fbc4de31640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc4819c7f0 0x7fbc4819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.105+0000 7fbc4de31640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4810f940 con 0x7fbc48105160 2024-04-25T00:23:03.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.105+0000 7fbc4de31640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4810f7c0 con 0x7fbc4819c7f0 2024-04-25T00:23:03.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.105+0000 7fbc4de31640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbc4810fac0 con 0x7fbc481002e0 2024-04-25T00:23:03.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.105+0000 7fbc46ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc48105160 0x7fbc48198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.105+0000 7fbc46ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc48105160 0x7fbc48198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48732/0 (socket says 172.21.15.107:48732) 2024-04-25T00:23:03.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.105+0000 7fbc46ffd640 1 -- 172.21.15.107:0/3944250662 learned_addr learned my addr 172.21.15.107:0/3944250662 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:03.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.105+0000 7fbc477fe640 1 --2- 172.21.15.107:0/3944250662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc481002e0 0x7fbc48197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.105+0000 7fbc46ffd640 1 -- 172.21.15.107:0/3944250662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc481002e0 msgr2=0x7fbc48197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.105+0000 7fbc46ffd640 1 --2- 172.21.15.107:0/3944250662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc481002e0 0x7fbc48197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.105+0000 7fbc46ffd640 1 -- 172.21.15.107:0/3944250662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc4819c7f0 msgr2=0x7fbc4819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.105+0000 7fbc46ffd640 1 --2- 172.21.15.107:0/3944250662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc4819c7f0 0x7fbc4819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.105+0000 7fbc46ffd640 1 -- 172.21.15.107:0/3944250662 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4819d2e0 con 0x7fbc48105160 2024-04-25T00:23:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.105+0000 7fbc477fe640 1 --2- 172.21.15.107:0/3944250662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc481002e0 0x7fbc48197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.106+0000 7fbc46ffd640 1 --2- 172.21.15.107:0/3944250662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc48105160 0x7fbc48198470 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c005fd0 tx=0x7fbc3c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.106+0000 7fbc44ff9640 1 -- 172.21.15.107:0/3944250662 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c017070 con 0x7fbc48105160 2024-04-25T00:23:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.106+0000 7fbc4de31640 1 -- 172.21.15.107:0/3944250662 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc4819d5d0 con 0x7fbc48105160 2024-04-25T00:23:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.106+0000 7fbc44ff9640 1 -- 172.21.15.107:0/3944250662 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc3c009c30 con 0x7fbc48105160 2024-04-25T00:23:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.106+0000 7fbc44ff9640 1 -- 172.21.15.107:0/3944250662 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c003d70 con 0x7fbc48105160 2024-04-25T00:23:03.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.106+0000 7fbc4de31640 1 -- 172.21.15.107:0/3944250662 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc481a4dc0 con 0x7fbc48105160 2024-04-25T00:23:03.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.108+0000 7fbc44ff9640 1 -- 172.21.15.107:0/3944250662 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fbc3c003f10 con 0x7fbc48105160 2024-04-25T00:23:03.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.108+0000 7fbc50c1c740 1 Processor -- start 2024-04-25T00:23:03.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.108+0000 7fbc50c1c740 1 -- start start 2024-04-25T00:23:03.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.108+0000 7fbc44ff9640 1 --2- 172.21.15.107:0/3944250662 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbc300a2d00 0x7fbc300a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.108+0000 7fbc50c1c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ccc87c460 0x557ccc87c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.108+0000 7fbc50c1c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ccc87cd80 0x557ccc88b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.108+0000 7fbc50c1c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc88b980 0x557ccc88dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.108+0000 7fbc50c1c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557ccc7e1670 con 0x557ccc88b980 2024-04-25T00:23:03.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.108+0000 7fbc50c1c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557ccc789970 con 0x557ccc87c460 2024-04-25T00:23:03.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.108+0000 7fbc50c1c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557ccc7dd630 con 0x557ccc87cd80 2024-04-25T00:23:03.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.108+0000 7fbc47fff640 1 --2- 172.21.15.107:0/3944250662 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbc300a2d00 0x7fbc300a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.109+0000 7fbc47fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc88b980 0x557ccc88dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.109+0000 7fbc46ffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ccc87cd80 0x557ccc88b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.109+0000 7fbc47fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc88b980 0x557ccc88dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48740/0 (socket says 172.21.15.107:48740) 2024-04-25T00:23:03.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.109+0000 7fbc47fff640 1 -- 172.21.15.107:0/609289015 learned_addr learned my addr 172.21.15.107:0/609289015 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:03.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.109+0000 7fbc44ff9640 1 -- 172.21.15.107:0/3944250662 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fbc3c0cf180 con 0x7fbc48105160 2024-04-25T00:23:03.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.109+0000 7fbc477fe640 1 --2- 172.21.15.107:0/609289015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ccc87c460 0x557ccc87c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.109+0000 7fbc47fff640 1 -- 172.21.15.107:0/609289015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ccc87cd80 msgr2=0x557ccc88b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:03.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.109+0000 7fbc47fff640 1 --2- 172.21.15.107:0/609289015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ccc87cd80 0x557ccc88b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.109+0000 7fbc47fff640 1 -- 172.21.15.107:0/609289015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ccc87c460 msgr2=0x557ccc87c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:03.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.109+0000 7fbc47fff640 1 --2- 172.21.15.107:0/609289015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ccc87c460 0x557ccc87c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.109+0000 7fbc47fff640 1 -- 172.21.15.107:0/609289015 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ccc7ad400 con 0x557ccc88b980 2024-04-25T00:23:03.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.109+0000 7fbc47fff640 1 --2- 172.21.15.107:0/609289015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc88b980 0x557ccc88dd70 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fbc40032700 tx=0x7fbc40032730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99cce06b4c124ae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:03.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.109+0000 7fbc47fff640 1 --2- 172.21.15.107:0/3944250662 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbc300a2d00 0x7fbc300a51c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbc40006010 tx=0x7fbc40043000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:03.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.109+0000 7fbc2dffb640 1 -- 172.21.15.107:0/609289015 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc40032a10 con 0x557ccc88b980 2024-04-25T00:23:03.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.109+0000 7fbc2dffb640 1 -- 172.21.15.107:0/609289015 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc40030510 con 0x557ccc88b980 2024-04-25T00:23:03.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.109+0000 7fbc2dffb640 1 -- 172.21.15.107:0/609289015 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4004b470 con 0x557ccc88b980 2024-04-25T00:23:03.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.110+0000 7fbc50c1c740 1 -- 172.21.15.107:0/609289015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc88b980 msgr2=0x557ccc88dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:03.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.110+0000 7fbc50c1c740 1 --2- 172.21.15.107:0/609289015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc88b980 0x557ccc88dd70 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fbc40032700 tx=0x7fbc40032730 comp rx=0 tx=0).stop 2024-04-25T00:23:03.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.110+0000 7fbc50c1c740 1 -- 172.21.15.107:0/609289015 shutdown_connections 2024-04-25T00:23:03.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.110+0000 7fbc50c1c740 1 --2- 172.21.15.107:0/609289015 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc88b980 0x557ccc88dd70 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.110+0000 7fbc50c1c740 1 --2- 172.21.15.107:0/609289015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ccc87cd80 0x557ccc88b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.110+0000 7fbc50c1c740 1 --2- 172.21.15.107:0/609289015 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ccc87c460 0x557ccc87c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.110+0000 7fbc50c1c740 1 -- 172.21.15.107:0/609289015 >> 172.21.15.107:0/609289015 conn(0x557ccc8788a0 msgr2=0x557ccc878c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:03.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.110+0000 7fbc50c1c740 1 -- 172.21.15.107:0/609289015 shutdown_connections 2024-04-25T00:23:03.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.110+0000 7fbc50c1c740 1 -- 172.21.15.107:0/609289015 wait complete. 2024-04-25T00:23:03.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.110+0000 7fbc50c1c740 1 -- start start 2024-04-25T00:23:03.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.110+0000 7fbc50c1c740 1 Processor -- start 2024-04-25T00:23:03.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.111+0000 7fbc50c1c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ccc87c460 0x557ccc88b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.111+0000 7fbc50c1c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ccc87cd80 0x557ccc888620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.111+0000 7fbc50c1c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc88b980 0x557ccc888bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.111+0000 7fbc50c1c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557ccc88e760 con 0x557ccc88b980 2024-04-25T00:23:03.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.111+0000 7fbc50c1c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557ccc79d050 con 0x557ccc87cd80 2024-04-25T00:23:03.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.111+0000 7fbc50c1c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x557ccc88e8e0 con 0x557ccc87c460 2024-04-25T00:23:03.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.111+0000 7fbc50c1c740 10 client.0 _ll_register_callbacks cb 0x557ccc4f4be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:03.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.111+0000 7fbc47fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc88b980 0x557ccc888bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.111+0000 7fbc47fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc88b980 0x557ccc888bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48756/0 (socket says 172.21.15.107:48756) 2024-04-25T00:23:03.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.111+0000 7fbc47fff640 1 -- 172.21.15.107:0/1478588735 learned_addr learned my addr 172.21.15.107:0/1478588735 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:03.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.111+0000 7fbc477fe640 1 --2- 172.21.15.107:0/1478588735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ccc87c460 0x557ccc88b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.111+0000 7fbc46ffd640 1 --2- 172.21.15.107:0/1478588735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ccc87cd80 0x557ccc888620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.111+0000 7fbc47fff640 1 -- 172.21.15.107:0/1478588735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ccc87c460 msgr2=0x557ccc88b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:03.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.111+0000 7fbc47fff640 1 --2- 172.21.15.107:0/1478588735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ccc87c460 0x557ccc88b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.111+0000 7fbc47fff640 1 -- 172.21.15.107:0/1478588735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ccc87cd80 msgr2=0x557ccc888620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:03.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.111+0000 7fbc47fff640 1 --2- 172.21.15.107:0/1478588735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ccc87cd80 0x557ccc888620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.111+0000 7fbc47fff640 1 -- 172.21.15.107:0/1478588735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ccc76edc0 con 0x557ccc88b980 2024-04-25T00:23:03.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.111+0000 7fbc47fff640 1 --2- 172.21.15.107:0/1478588735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc88b980 0x557ccc888bc0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fbc4000ab70 tx=0x7fbc40005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6774ea493bb0c677 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:03.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.111+0000 7fbc26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:03.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.111+0000 7fbc26ffd640 1 -- 172.21.15.107:0/1478588735 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4004b470 con 0x557ccc88b980 2024-04-25T00:23:03.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.111+0000 7fbc26ffd640 1 -- 172.21.15.107:0/1478588735 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fbc40047070 con 0x557ccc88b980 2024-04-25T00:23:03.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.111+0000 7fbc50c1c740 1 -- 172.21.15.107:0/1478588735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc40030510 con 0x557ccc88b980 2024-04-25T00:23:03.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.112+0000 7fbc26ffd640 1 -- 172.21.15.107:0/1478588735 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbc40061050 con 0x557ccc88b980 2024-04-25T00:23:03.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.112+0000 7fbc50c1c740 10 client.17124 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:03.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.112+0000 7fbc50c1c740 1 -- 172.21.15.107:0/1478588735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557ccc4f53f0 con 0x557ccc88b980 2024-04-25T00:23:03.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.112+0000 7fbc26ffd640 1 -- 172.21.15.107:0/1478588735 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc40030510 con 0x557ccc88b980 2024-04-25T00:23:03.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.112+0000 7fbc26ffd640 1 -- 172.21.15.107:0/1478588735 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fbc4004b660 con 0x557ccc88b980 2024-04-25T00:23:03.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.112+0000 7fbc50c1c740 10 client.17124 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:03.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.112+0000 7fbc50c1c740 10 client.17124 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:23:03.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.112+0000 7fbc50c1c740 20 client.17124 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:03.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.112+0000 7fbc50c1c740 1 --2- 172.21.15.107:0/1478588735 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x557ccc89b930 0x557ccc89dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.112+0000 7fbc50c1c740 4 client.17124 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:03.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.112+0000 7fbc50c1c740 1 -- 172.21.15.107:0/1478588735 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ccc7eebb0 con 0x557ccc89b930 2024-04-25T00:23:03.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.112+0000 7fbc477fe640 1 --2- 172.21.15.107:0/1478588735 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x557ccc89b930 0x557ccc89dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.113+0000 7fbc26ffd640 10 client.17124 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:23:03.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.113+0000 7fbc477fe640 1 --2- 172.21.15.107:0/1478588735 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x557ccc89b930 0x557ccc89dd20 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9398de646400197e server_cookie=887d906db5361fb3 in_seq=0 out_seq=0 2024-04-25T00:23:03.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.113+0000 7fbc26ffd640 1 -- 172.21.15.107:0/1478588735 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557ccc7eebb0 con 0x557ccc89b930 2024-04-25T00:23:03.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.113+0000 7fbc26ffd640 10 client.17124 handle_command_reply: tid=0 2024-04-25T00:23:03.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.113+0000 7fbc50c1c740 1 client.17124 shutdown 2024-04-25T00:23:03.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.114+0000 7fbc50c1c740 1 -- 172.21.15.107:0/1478588735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc88b980 msgr2=0x557ccc888bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:03.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.114+0000 7fbc50c1c740 1 --2- 172.21.15.107:0/1478588735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc88b980 0x557ccc888bc0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fbc4000ab70 tx=0x7fbc40005d80 comp rx=0 tx=0).stop 2024-04-25T00:23:03.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.114+0000 7fbc50c1c740 1 -- 172.21.15.107:0/1478588735 shutdown_connections 2024-04-25T00:23:03.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.114+0000 7fbc50c1c740 1 --2- 172.21.15.107:0/1478588735 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x557ccc89b930 0x557ccc89dd20 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.114+0000 7fbc50c1c740 1 --2- 172.21.15.107:0/1478588735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc88b980 0x557ccc888bc0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.114+0000 7fbc50c1c740 1 --2- 172.21.15.107:0/1478588735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ccc87cd80 0x557ccc888620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.114+0000 7fbc50c1c740 1 --2- 172.21.15.107:0/1478588735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ccc87c460 0x557ccc88b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.114+0000 7fbc26ffd640 0 client.17124 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:23:03.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.114+0000 7fbc50c1c740 1 -- 172.21.15.107:0/1478588735 >> 172.21.15.107:0/1478588735 conn(0x557ccc8788a0 msgr2=0x557ccc87e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:03.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.114+0000 7fbc50c1c740 1 -- 172.21.15.107:0/1478588735 shutdown_connections 2024-04-25T00:23:03.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.114+0000 7fbc50c1c740 1 -- 172.21.15.107:0/1478588735 wait complete. 2024-04-25T00:23:03.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.114+0000 7fbc50c1c740 20 client.17124 trim_cache size 0 max 16384 2024-04-25T00:23:03.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.127+0000 7fbc50c1c740 1 Processor -- start 2024-04-25T00:23:03.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.127+0000 7fbc50c1c740 1 -- start start 2024-04-25T00:23:03.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.127+0000 7fbc50c1c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc87c460 0x557ccc883eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.127+0000 7fbc50c1c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ccc87cd80 0x557ccc8976c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.127+0000 7fbc50c1c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ccc88b980 0x557ccc88ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.127+0000 7fbc50c1c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbc481af0d0 con 0x557ccc87c460 2024-04-25T00:23:03.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.127+0000 7fbc50c1c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbc481ac9a0 con 0x557ccc88b980 2024-04-25T00:23:03.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.127+0000 7fbc50c1c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbc481aa7f0 con 0x557ccc87cd80 2024-04-25T00:23:03.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.127+0000 7fbc477fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc87c460 0x557ccc883eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.127+0000 7fbc477fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc87c460 0x557ccc883eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48762/0 (socket says 172.21.15.107:48762) 2024-04-25T00:23:03.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.127+0000 7fbc477fe640 1 -- 172.21.15.107:0/4278669958 learned_addr learned my addr 172.21.15.107:0/4278669958 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:03.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.127+0000 7fbc477fe640 1 -- 172.21.15.107:0/4278669958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ccc87cd80 msgr2=0x557ccc8976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:03.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.127+0000 7fbc477fe640 1 --2- 172.21.15.107:0/4278669958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ccc87cd80 0x557ccc8976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.127+0000 7fbc477fe640 1 -- 172.21.15.107:0/4278669958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ccc88b980 msgr2=0x557ccc88ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:03.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.127+0000 7fbc477fe640 1 --2- 172.21.15.107:0/4278669958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ccc88b980 0x557ccc88ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.127+0000 7fbc477fe640 1 -- 172.21.15.107:0/4278669958 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ccc5bf7d0 con 0x557ccc87c460 2024-04-25T00:23:03.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.128+0000 7fbc477fe640 1 --2- 172.21.15.107:0/4278669958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc87c460 0x557ccc883eb0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fbc38004700 tx=0x7fbc3800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b9239d62c04c3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:03.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.128+0000 7fbc2d7fa640 1 -- 172.21.15.107:0/4278669958 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38022070 con 0x557ccc87c460 2024-04-25T00:23:03.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.128+0000 7fbc2d7fa640 1 -- 172.21.15.107:0/4278669958 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc3801c890 con 0x557ccc87c460 2024-04-25T00:23:03.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.128+0000 7fbc2d7fa640 1 -- 172.21.15.107:0/4278669958 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38003b20 con 0x557ccc87c460 2024-04-25T00:23:03.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.128+0000 7fbc50c1c740 1 -- 172.21.15.107:0/4278669958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc87c460 msgr2=0x557ccc883eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:03.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.128+0000 7fbc50c1c740 1 --2- 172.21.15.107:0/4278669958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc87c460 0x557ccc883eb0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fbc38004700 tx=0x7fbc3800f7f0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.128+0000 7fbc50c1c740 1 -- 172.21.15.107:0/4278669958 shutdown_connections 2024-04-25T00:23:03.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.128+0000 7fbc50c1c740 1 --2- 172.21.15.107:0/4278669958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ccc88b980 0x557ccc88ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.128+0000 7fbc50c1c740 1 --2- 172.21.15.107:0/4278669958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ccc87cd80 0x557ccc8976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.128+0000 7fbc50c1c740 1 --2- 172.21.15.107:0/4278669958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc87c460 0x557ccc883eb0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.128+0000 7fbc50c1c740 1 -- 172.21.15.107:0/4278669958 >> 172.21.15.107:0/4278669958 conn(0x557ccc8788a0 msgr2=0x557ccc877b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:03.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.128+0000 7fbc50c1c740 1 -- 172.21.15.107:0/4278669958 shutdown_connections 2024-04-25T00:23:03.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.128+0000 7fbc50c1c740 1 -- 172.21.15.107:0/4278669958 wait complete. 2024-04-25T00:23:03.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.128+0000 7fbc50c1c740 1 -- start start 2024-04-25T00:23:03.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.129+0000 7fbc50c1c740 1 Processor -- start 2024-04-25T00:23:03.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.129+0000 7fbc50c1c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc87c460 0x557ccc8940b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.129+0000 7fbc50c1c740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ccc87cd80 0x557ccc885060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.129+0000 7fbc50c1c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ccc88b980 0x557ccc885600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.129+0000 7fbc50c1c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbc481aae60 con 0x557ccc87c460 2024-04-25T00:23:03.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.129+0000 7fbc50c1c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbc481ac2d0 con 0x557ccc88b980 2024-04-25T00:23:03.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.129+0000 7fbc50c1c740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbc481ab530 con 0x557ccc87cd80 2024-04-25T00:23:03.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.129+0000 7fbc50c1c740 10 client.0 _ll_register_callbacks cb 0x7fbc3800b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:03.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.129+0000 7fbc477fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc87c460 0x557ccc8940b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.129+0000 7fbc477fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc87c460 0x557ccc8940b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48778/0 (socket says 172.21.15.107:48778) 2024-04-25T00:23:03.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.129+0000 7fbc477fe640 1 -- 172.21.15.107:0/1914614896 learned_addr learned my addr 172.21.15.107:0/1914614896 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:03.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.129+0000 7fbc47fff640 1 --2- 172.21.15.107:0/1914614896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ccc88b980 0x557ccc885600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.129+0000 7fbc477fe640 1 -- 172.21.15.107:0/1914614896 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ccc87cd80 msgr2=0x557ccc885060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:03.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.129+0000 7fbc477fe640 1 --2- 172.21.15.107:0/1914614896 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ccc87cd80 0x557ccc885060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.129+0000 7fbc477fe640 1 -- 172.21.15.107:0/1914614896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ccc88b980 msgr2=0x557ccc885600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:03.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.129+0000 7fbc477fe640 1 --2- 172.21.15.107:0/1914614896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ccc88b980 0x557ccc885600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.129+0000 7fbc477fe640 1 -- 172.21.15.107:0/1914614896 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ccc885e30 con 0x557ccc87c460 2024-04-25T00:23:03.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.129+0000 7fbc477fe640 1 --2- 172.21.15.107:0/1914614896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc87c460 0x557ccc8940b0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800b850 tx=0x7fbc3801c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c319d4869d4e536 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:03.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.129+0000 7fbc26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:03.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.130+0000 7fbc26ffd640 1 -- 172.21.15.107:0/1914614896 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3802a020 con 0x557ccc87c460 2024-04-25T00:23:03.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.130+0000 7fbc26ffd640 1 -- 172.21.15.107:0/1914614896 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7fbc3801b660 con 0x557ccc87c460 2024-04-25T00:23:03.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.130+0000 7fbc50c1c740 1 -- 172.21.15.107:0/1914614896 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc3801c890 con 0x557ccc87c460 2024-04-25T00:23:03.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.130+0000 7fbc26ffd640 1 -- 172.21.15.107:0/1914614896 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbc38035050 con 0x557ccc87c460 2024-04-25T00:23:03.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.130+0000 7fbc50c1c740 10 client.17130 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:03.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.130+0000 7fbc50c1c740 1 -- 172.21.15.107:0/1914614896 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557ccc901c30 con 0x557ccc87c460 2024-04-25T00:23:03.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.130+0000 7fbc26ffd640 1 -- 172.21.15.107:0/1914614896 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc3801c890 con 0x557ccc87c460 2024-04-25T00:23:03.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.130+0000 7fbc26ffd640 1 -- 172.21.15.107:0/1914614896 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fbc38025070 con 0x557ccc87c460 2024-04-25T00:23:03.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.130+0000 7fbc50c1c740 10 client.17130 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:03.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.130+0000 7fbc50c1c740 10 client.17130 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:23:03.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.130+0000 7fbc50c1c740 20 client.17130 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:03.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.130+0000 7fbc50c1c740 1 --2- 172.21.15.107:0/1914614896 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x557ccc902340 0x557ccc904760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.130+0000 7fbc50c1c740 4 client.17130 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:23:03.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.130+0000 7fbc50c1c740 1 -- 172.21.15.107:0/1914614896 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557ccc7e8f00 con 0x557ccc902340 2024-04-25T00:23:03.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.130+0000 7fbc46ffd640 1 --2- 172.21.15.107:0/1914614896 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x557ccc902340 0x557ccc904760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.131+0000 7fbc26ffd640 10 client.17130 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:23:03.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.131+0000 7fbc46ffd640 1 --2- 172.21.15.107:0/1914614896 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x557ccc902340 0x557ccc904760 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=50d76e0b9b970b58 server_cookie=576463f0a22a5542 in_seq=0 out_seq=0 2024-04-25T00:23:03.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.131+0000 7fbc26ffd640 1 -- 172.21.15.107:0/1914614896 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557ccc7e8f00 con 0x557ccc902340 2024-04-25T00:23:03.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.131+0000 7fbc26ffd640 10 client.17130 handle_command_reply: tid=0 2024-04-25T00:23:03.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.131+0000 7fbc50c1c740 1 client.17130 shutdown 2024-04-25T00:23:03.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.131+0000 7fbc50c1c740 1 -- 172.21.15.107:0/1914614896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc87c460 msgr2=0x557ccc8940b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:03.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.131+0000 7fbc50c1c740 1 --2- 172.21.15.107:0/1914614896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc87c460 0x557ccc8940b0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800b850 tx=0x7fbc3801c660 comp rx=0 tx=0).stop 2024-04-25T00:23:03.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.131+0000 7fbc50c1c740 1 -- 172.21.15.107:0/1914614896 shutdown_connections 2024-04-25T00:23:03.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.131+0000 7fbc50c1c740 1 --2- 172.21.15.107:0/1914614896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557ccc88b980 0x557ccc885600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.131+0000 7fbc50c1c740 1 --2- 172.21.15.107:0/1914614896 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x557ccc87cd80 0x557ccc885060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.131+0000 7fbc50c1c740 1 --2- 172.21.15.107:0/1914614896 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x557ccc902340 0x557ccc904760 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.131+0000 7fbc50c1c740 1 --2- 172.21.15.107:0/1914614896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557ccc87c460 0x557ccc8940b0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.131+0000 7fbc26ffd640 0 client.17130 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:23:03.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.131+0000 7fbc50c1c740 1 -- 172.21.15.107:0/1914614896 >> 172.21.15.107:0/1914614896 conn(0x557ccc8788a0 msgr2=0x557ccc87dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:03.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.131+0000 7fbc50c1c740 1 -- 172.21.15.107:0/1914614896 shutdown_connections 2024-04-25T00:23:03.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.131+0000 7fbc50c1c740 1 -- 172.21.15.107:0/1914614896 wait complete. 2024-04-25T00:23:03.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.131+0000 7fbc50c1c740 20 client.17130 trim_cache size 0 max 16384 2024-04-25T00:23:03.139 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:23:03.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.132+0000 7fbc4de31640 1 -- 172.21.15.107:0/3944250662 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbc300a2d00 msgr2=0x7fbc300a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:03.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.132+0000 7fbc4de31640 1 --2- 172.21.15.107:0/3944250662 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbc300a2d00 0x7fbc300a51c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbc40006010 tx=0x7fbc40043000 comp rx=0 tx=0).stop 2024-04-25T00:23:03.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.132+0000 7fbc4de31640 1 -- 172.21.15.107:0/3944250662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc48105160 msgr2=0x7fbc48198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:03.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.132+0000 7fbc4de31640 1 --2- 172.21.15.107:0/3944250662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc48105160 0x7fbc48198470 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c005fd0 tx=0x7fbc3c00dde0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.132+0000 7fbc4de31640 1 -- 172.21.15.107:0/3944250662 shutdown_connections 2024-04-25T00:23:03.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.132+0000 7fbc4de31640 1 --2- 172.21.15.107:0/3944250662 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbc300a2d00 0x7fbc300a51c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.132+0000 7fbc4de31640 1 --2- 172.21.15.107:0/3944250662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc4819c7f0 0x7fbc4819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.132+0000 7fbc4de31640 1 --2- 172.21.15.107:0/3944250662 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc48105160 0x7fbc48198470 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.132+0000 7fbc4de31640 1 --2- 172.21.15.107:0/3944250662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc481002e0 0x7fbc48197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.132+0000 7fbc4de31640 1 -- 172.21.15.107:0/3944250662 >> 172.21.15.107:0/3944250662 conn(0x7fbc480758d0 msgr2=0x7fbc48103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:03.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.132+0000 7fbc4de31640 1 -- 172.21.15.107:0/3944250662 shutdown_connections 2024-04-25T00:23:03.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.132+0000 7fbc4de31640 1 -- 172.21.15.107:0/3944250662 wait complete. 2024-04-25T00:23:03.143 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:23:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:02 smithi170 ceph-mon[32807]: scrub queued for path: / 2024-04-25T00:23:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:02 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:02 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:23:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:02 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:03.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:02 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:23:03.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.243+0000 7f36ad89e640 1 -- 172.21.15.107:0/2672134958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36a81020d0 msgr2=0x7f36a810e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:03.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.243+0000 7f36ad89e640 1 --2- 172.21.15.107:0/2672134958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36a81020d0 0x7f36a810e5e0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009cd0 tx=0x7f36a002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.243+0000 7f36ad89e640 1 -- 172.21.15.107:0/2672134958 shutdown_connections 2024-04-25T00:23:03.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.243+0000 7f36ad89e640 1 --2- 172.21.15.107:0/2672134958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36a81020d0 0x7f36a810e5e0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.243+0000 7f36ad89e640 1 --2- 172.21.15.107:0/2672134958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36a8101710 0x7f36a8101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.243+0000 7f36ad89e640 1 --2- 172.21.15.107:0/2672134958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36a8107720 0x7f36a8107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.243+0000 7f36ad89e640 1 -- 172.21.15.107:0/2672134958 >> 172.21.15.107:0/2672134958 conn(0x7f36a8078b90 msgr2=0x7f36a8078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:03.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.243+0000 7f36ad89e640 1 -- 172.21.15.107:0/2672134958 shutdown_connections 2024-04-25T00:23:03.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.243+0000 7f36ad89e640 1 -- 172.21.15.107:0/2672134958 wait complete. 2024-04-25T00:23:03.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.243+0000 7f36ad89e640 1 Processor -- start 2024-04-25T00:23:03.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.243+0000 7f36ad89e640 1 -- start start 2024-04-25T00:23:03.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.243+0000 7f36ad89e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36a8101710 0x7f36a8197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.244+0000 7f36ad89e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36a81020d0 0x7f36a81984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.244+0000 7f36ad89e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36a8107720 0x7f36a819c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.244+0000 7f36ad89e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f36a8111b30 con 0x7f36a8101710 2024-04-25T00:23:03.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.244+0000 7f36ad89e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f36a81119b0 con 0x7f36a8107720 2024-04-25T00:23:03.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.244+0000 7f36ad89e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f36a8111cb0 con 0x7f36a81020d0 2024-04-25T00:23:03.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.244+0000 7f36a6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36a8101710 0x7f36a8197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.244+0000 7f36a6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36a8101710 0x7f36a8197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48798/0 (socket says 172.21.15.107:48798) 2024-04-25T00:23:03.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.244+0000 7f36a6ffd640 1 -- 172.21.15.107:0/809581883 learned_addr learned my addr 172.21.15.107:0/809581883 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:03.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.244+0000 7f369ffff640 1 --2- 172.21.15.107:0/809581883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36a81020d0 0x7f36a81984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.244+0000 7f36a6ffd640 1 -- 172.21.15.107:0/809581883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36a81020d0 msgr2=0x7f36a81984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:03.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.244+0000 7f36a6ffd640 1 --2- 172.21.15.107:0/809581883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36a81020d0 0x7f36a81984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.244+0000 7f36a6ffd640 1 -- 172.21.15.107:0/809581883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36a8107720 msgr2=0x7f36a819c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:03.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.244+0000 7f36a6ffd640 1 --2- 172.21.15.107:0/809581883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36a8107720 0x7f36a819c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.244+0000 7f36a6ffd640 1 -- 172.21.15.107:0/809581883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a819cfa0 con 0x7f36a8101710 2024-04-25T00:23:03.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.244+0000 7f36a6ffd640 1 --2- 172.21.15.107:0/809581883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36a8101710 0x7f36a8197fb0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f369800c920 tx=0x7f369800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:03.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.244+0000 7f36ac89c640 1 -- 172.21.15.107:0/809581883 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698007c70 con 0x7f36a8101710 2024-04-25T00:23:03.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.244+0000 7f36ad89e640 1 -- 172.21.15.107:0/809581883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a819d290 con 0x7f36a8101710 2024-04-25T00:23:03.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.244+0000 7f36ac89c640 1 -- 172.21.15.107:0/809581883 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3698007e10 con 0x7f36a8101710 2024-04-25T00:23:03.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.244+0000 7f36ac89c640 1 -- 172.21.15.107:0/809581883 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698018890 con 0x7f36a8101710 2024-04-25T00:23:03.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.244+0000 7f36ad89e640 1 -- 172.21.15.107:0/809581883 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a81a4b20 con 0x7f36a8101710 2024-04-25T00:23:03.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.246+0000 7f36ac89c640 1 -- 172.21.15.107:0/809581883 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f3698018a70 con 0x7f36a8101710 2024-04-25T00:23:03.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.247+0000 7f36b05a9740 1 Processor -- start 2024-04-25T00:23:03.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.247+0000 7f36b05a9740 1 -- start start 2024-04-25T00:23:03.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.247+0000 7f36ac89c640 1 --2- 172.21.15.107:0/809581883 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f36940a2d00 0x7f36940a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.247+0000 7f36b05a9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e675be1580 0x55e675be1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.247+0000 7f36a77fe640 1 --2- 172.21.15.107:0/809581883 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f36940a2d00 0x7f36940a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.247+0000 7f36b05a9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1ea0 0x55e675bec030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.247+0000 7f36b05a9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e675bec570 0x55e675bee960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.247+0000 7f36b05a9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e675b46670 con 0x55e675be1ea0 2024-04-25T00:23:03.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.247+0000 7f36b05a9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e675aee970 con 0x55e675be1580 2024-04-25T00:23:03.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.247+0000 7f36ac89c640 1 -- 172.21.15.107:0/809581883 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f36980cf890 con 0x7f36a8101710 2024-04-25T00:23:03.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.247+0000 7f36b05a9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e675b42630 con 0x55e675bec570 2024-04-25T00:23:03.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.247+0000 7f369ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1ea0 0x55e675bec030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.247+0000 7f369ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1ea0 0x55e675bec030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48804/0 (socket says 172.21.15.107:48804) 2024-04-25T00:23:03.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.247+0000 7f369ffff640 1 -- 172.21.15.107:0/2123667369 learned_addr learned my addr 172.21.15.107:0/2123667369 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:03.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.247+0000 7f36a77fe640 1 --2- 172.21.15.107:0/2123667369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e675bec570 0x55e675bee960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.247+0000 7f36a6ffd640 1 --2- 172.21.15.107:0/2123667369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e675be1580 0x55e675be1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.247+0000 7f369ffff640 1 -- 172.21.15.107:0/2123667369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e675bec570 msgr2=0x55e675bee960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:03.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.247+0000 7f369ffff640 1 --2- 172.21.15.107:0/2123667369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e675bec570 0x55e675bee960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.247+0000 7f369ffff640 1 -- 172.21.15.107:0/2123667369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e675be1580 msgr2=0x55e675be1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:03.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.247+0000 7f369ffff640 1 --2- 172.21.15.107:0/2123667369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e675be1580 0x55e675be1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.247+0000 7f369ffff640 1 -- 172.21.15.107:0/2123667369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e675b12400 con 0x55e675be1ea0 2024-04-25T00:23:03.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.247+0000 7f36a77fe640 1 --2- 172.21.15.107:0/809581883 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f36940a2d00 0x7f36940a51c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f36a002ae00 tx=0x7f36a0031510 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:03.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.248+0000 7f369ffff640 1 --2- 172.21.15.107:0/2123667369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1ea0 0x55e675bec030 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f369000b690 tx=0x7f369000bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ca5d5944463f00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:03.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.248+0000 7f369dffb640 1 -- 172.21.15.107:0/2123667369 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690003b50 con 0x55e675be1ea0 2024-04-25T00:23:03.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.248+0000 7f369dffb640 1 -- 172.21.15.107:0/2123667369 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3690015070 con 0x55e675be1ea0 2024-04-25T00:23:03.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.248+0000 7f369dffb640 1 -- 172.21.15.107:0/2123667369 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690004d50 con 0x55e675be1ea0 2024-04-25T00:23:03.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.248+0000 7f36b05a9740 1 -- 172.21.15.107:0/2123667369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1ea0 msgr2=0x55e675bec030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:03.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.248+0000 7f36b05a9740 1 --2- 172.21.15.107:0/2123667369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1ea0 0x55e675bec030 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f369000b690 tx=0x7f369000bb50 comp rx=0 tx=0).stop 2024-04-25T00:23:03.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.248+0000 7f36b05a9740 1 -- 172.21.15.107:0/2123667369 shutdown_connections 2024-04-25T00:23:03.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.248+0000 7f36b05a9740 1 --2- 172.21.15.107:0/2123667369 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e675bec570 0x55e675bee960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.248+0000 7f36b05a9740 1 --2- 172.21.15.107:0/2123667369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1ea0 0x55e675bec030 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.248+0000 7f36b05a9740 1 --2- 172.21.15.107:0/2123667369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e675be1580 0x55e675be1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.248+0000 7f36b05a9740 1 -- 172.21.15.107:0/2123667369 >> 172.21.15.107:0/2123667369 conn(0x55e675bdd8a0 msgr2=0x55e675bddc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:03.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.248+0000 7f36b05a9740 1 -- 172.21.15.107:0/2123667369 shutdown_connections 2024-04-25T00:23:03.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.248+0000 7f36b05a9740 1 -- 172.21.15.107:0/2123667369 wait complete. 2024-04-25T00:23:03.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.249+0000 7f36b05a9740 1 -- start start 2024-04-25T00:23:03.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.249+0000 7f36b05a9740 1 Processor -- start 2024-04-25T00:23:03.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.249+0000 7f36b05a9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e675be1580 0x55e675bebed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.249+0000 7f36b05a9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1ea0 0x55e675be91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.249+0000 7f36b05a9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e675bec570 0x55e675be9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.249+0000 7f36b05a9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e675bef360 con 0x55e675be1ea0 2024-04-25T00:23:03.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.249+0000 7f36b05a9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e675b02050 con 0x55e675bec570 2024-04-25T00:23:03.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.249+0000 7f36b05a9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e675bef4e0 con 0x55e675be1580 2024-04-25T00:23:03.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.249+0000 7f36b05a9740 10 client.0 _ll_register_callbacks cb 0x55e675859be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:03.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.249+0000 7f369ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1ea0 0x55e675be91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.249+0000 7f369ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1ea0 0x55e675be91c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48812/0 (socket says 172.21.15.107:48812) 2024-04-25T00:23:03.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.249+0000 7f369ffff640 1 -- 172.21.15.107:0/642023100 learned_addr learned my addr 172.21.15.107:0/642023100 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:03.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.249+0000 7f36a6ffd640 1 --2- 172.21.15.107:0/642023100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e675be1580 0x55e675bebed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.249+0000 7f36a77fe640 1 --2- 172.21.15.107:0/642023100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e675bec570 0x55e675be9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.249+0000 7f369ffff640 1 -- 172.21.15.107:0/642023100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e675be1580 msgr2=0x55e675bebed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:03.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.249+0000 7f369ffff640 1 --2- 172.21.15.107:0/642023100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e675be1580 0x55e675bebed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.249+0000 7f369ffff640 1 -- 172.21.15.107:0/642023100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e675bec570 msgr2=0x55e675be9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:03.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.249+0000 7f369ffff640 1 --2- 172.21.15.107:0/642023100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e675bec570 0x55e675be9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.249+0000 7f369ffff640 1 -- 172.21.15.107:0/642023100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e675ad3dc0 con 0x55e675be1ea0 2024-04-25T00:23:03.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.250+0000 7f36a77fe640 1 --2- 172.21.15.107:0/642023100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e675bec570 0x55e675be9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:03.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.250+0000 7f369ffff640 1 --2- 172.21.15.107:0/642023100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1ea0 0x55e675be91c0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f3690004470 tx=0x7f369000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224d55a4fd2ed72e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:03.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.250+0000 7f3686ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:03.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.250+0000 7f3686ffd640 1 -- 172.21.15.107:0/642023100 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690028060 con 0x55e675be1ea0 2024-04-25T00:23:03.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.250+0000 7f3686ffd640 1 -- 172.21.15.107:0/642023100 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f3690011640 con 0x55e675be1ea0 2024-04-25T00:23:03.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.250+0000 7f36b05a9740 1 -- 172.21.15.107:0/642023100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3690015070 con 0x55e675be1ea0 2024-04-25T00:23:03.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.250+0000 7f3686ffd640 1 -- 172.21.15.107:0/642023100 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f369002f050 con 0x55e675be1ea0 2024-04-25T00:23:03.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.250+0000 7f36b05a9740 10 client.17142 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:03.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.250+0000 7f36b05a9740 1 -- 172.21.15.107:0/642023100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e67585a3f0 con 0x55e675be1ea0 2024-04-25T00:23:03.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.250+0000 7f3686ffd640 1 -- 172.21.15.107:0/642023100 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3690015070 con 0x55e675be1ea0 2024-04-25T00:23:03.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.250+0000 7f3686ffd640 1 -- 172.21.15.107:0/642023100 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f3690022070 con 0x55e675be1ea0 2024-04-25T00:23:03.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.250+0000 7f36b05a9740 10 client.17142 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:03.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.250+0000 7f36b05a9740 10 client.17142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:03.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.250+0000 7f36b05a9740 20 client.17142 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:03.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.250+0000 7f36b05a9740 1 --2- 172.21.15.107:0/642023100 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e675bfc4e0 0x55e675bfe8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.250+0000 7f36b05a9740 4 client.17142 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:03.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.250+0000 7f36b05a9740 1 -- 172.21.15.107:0/642023100 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e675b53bb0 con 0x55e675bfc4e0 2024-04-25T00:23:03.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.251+0000 7f36a6ffd640 1 --2- 172.21.15.107:0/642023100 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e675bfc4e0 0x55e675bfe8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.251+0000 7f3686ffd640 10 client.17142 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:03.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.251+0000 7f36a6ffd640 1 --2- 172.21.15.107:0/642023100 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e675bfc4e0 0x55e675bfe8d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e0e1254e375393d server_cookie=354c34f86723334d in_seq=0 out_seq=0 2024-04-25T00:23:03.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.252+0000 7f3686ffd640 1 -- 172.21.15.107:0/642023100 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e675b53bb0 con 0x55e675bfc4e0 2024-04-25T00:23:03.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.252+0000 7f3686ffd640 10 client.17142 handle_command_reply: tid=0 2024-04-25T00:23:03.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.252+0000 7f36b05a9740 1 client.17142 shutdown 2024-04-25T00:23:03.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.252+0000 7f36b05a9740 1 -- 172.21.15.107:0/642023100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1ea0 msgr2=0x55e675be91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:03.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.252+0000 7f36b05a9740 1 --2- 172.21.15.107:0/642023100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1ea0 0x55e675be91c0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f3690004470 tx=0x7f369000bfd0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.252+0000 7f36b05a9740 1 -- 172.21.15.107:0/642023100 shutdown_connections 2024-04-25T00:23:03.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.252+0000 7f36b05a9740 1 --2- 172.21.15.107:0/642023100 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e675bfc4e0 0x55e675bfe8d0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.252+0000 7f36b05a9740 1 --2- 172.21.15.107:0/642023100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e675bec570 0x55e675be9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.252+0000 7f36b05a9740 1 --2- 172.21.15.107:0/642023100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1ea0 0x55e675be91c0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.252+0000 7f36b05a9740 1 --2- 172.21.15.107:0/642023100 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e675be1580 0x55e675bebed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.252+0000 7f3686ffd640 0 client.17142 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:03.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.252+0000 7f36b05a9740 1 -- 172.21.15.107:0/642023100 >> 172.21.15.107:0/642023100 conn(0x55e675bdd8a0 msgr2=0x55e675bdcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:03.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.252+0000 7f36b05a9740 1 -- 172.21.15.107:0/642023100 shutdown_connections 2024-04-25T00:23:03.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.252+0000 7f36b05a9740 1 -- 172.21.15.107:0/642023100 wait complete. 2024-04-25T00:23:03.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.252+0000 7f36b05a9740 20 client.17142 trim_cache size 0 max 16384 2024-04-25T00:23:03.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.265+0000 7f36b05a9740 1 Processor -- start 2024-04-25T00:23:03.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.265+0000 7f36b05a9740 1 -- start start 2024-04-25T00:23:03.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.265+0000 7f36b05a9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1580 0x55e675bf8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.265+0000 7f36b05a9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e675be1ea0 0x55e675beba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.265+0000 7f36b05a9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e675bec570 0x55e675c04040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.265+0000 7f36b05a9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f36a81b13f0 con 0x55e675be1580 2024-04-25T00:23:03.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.265+0000 7f36b05a9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f36a81aecc0 con 0x55e675be1ea0 2024-04-25T00:23:03.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.265+0000 7f36b05a9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f36a81acb10 con 0x55e675bec570 2024-04-25T00:23:03.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.265+0000 7f36a6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1580 0x55e675bf8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.265+0000 7f36a6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1580 0x55e675bf8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48820/0 (socket says 172.21.15.107:48820) 2024-04-25T00:23:03.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.265+0000 7f36a6ffd640 1 -- 172.21.15.107:0/593268971 learned_addr learned my addr 172.21.15.107:0/593268971 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:03.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.265+0000 7f36a77fe640 1 --2- 172.21.15.107:0/593268971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e675bec570 0x55e675c04040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.265+0000 7f36a6ffd640 1 -- 172.21.15.107:0/593268971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e675bec570 msgr2=0x55e675c04040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:03.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.265+0000 7f36a6ffd640 1 --2- 172.21.15.107:0/593268971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e675bec570 0x55e675c04040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.266+0000 7f36a6ffd640 1 -- 172.21.15.107:0/593268971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e675be1ea0 msgr2=0x55e675beba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:03.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.266+0000 7f36a6ffd640 1 --2- 172.21.15.107:0/593268971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e675be1ea0 0x55e675beba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.266+0000 7f36a6ffd640 1 -- 172.21.15.107:0/593268971 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6759247d0 con 0x55e675be1580 2024-04-25T00:23:03.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.266+0000 7f36a6ffd640 1 --2- 172.21.15.107:0/593268971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1580 0x55e675bf8250 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f3698097e50 tx=0x7f3698086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54c3cefeb3c3b20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:03.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.266+0000 7f369d7fa640 1 -- 172.21.15.107:0/593268971 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36980b9070 con 0x55e675be1580 2024-04-25T00:23:03.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.266+0000 7f369d7fa640 1 -- 172.21.15.107:0/593268971 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3698099d40 con 0x55e675be1580 2024-04-25T00:23:03.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.266+0000 7f369d7fa640 1 -- 172.21.15.107:0/593268971 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36980980b0 con 0x55e675be1580 2024-04-25T00:23:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.266+0000 7f36b05a9740 1 -- 172.21.15.107:0/593268971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1580 msgr2=0x55e675bf8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.266+0000 7f36b05a9740 1 --2- 172.21.15.107:0/593268971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1580 0x55e675bf8250 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f3698097e50 tx=0x7f3698086040 comp rx=0 tx=0).stop 2024-04-25T00:23:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.266+0000 7f36b05a9740 1 -- 172.21.15.107:0/593268971 shutdown_connections 2024-04-25T00:23:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.266+0000 7f36b05a9740 1 --2- 172.21.15.107:0/593268971 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e675bec570 0x55e675c04040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.266+0000 7f36b05a9740 1 --2- 172.21.15.107:0/593268971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e675be1ea0 0x55e675beba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.266+0000 7f36b05a9740 1 --2- 172.21.15.107:0/593268971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1580 0x55e675bf8250 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.266+0000 7f36b05a9740 1 -- 172.21.15.107:0/593268971 >> 172.21.15.107:0/593268971 conn(0x55e675bdd8a0 msgr2=0x55e675be4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.266+0000 7f36b05a9740 1 -- 172.21.15.107:0/593268971 shutdown_connections 2024-04-25T00:23:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.266+0000 7f36b05a9740 1 -- 172.21.15.107:0/593268971 wait complete. 2024-04-25T00:23:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.267+0000 7f36b05a9740 1 -- start start 2024-04-25T00:23:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.267+0000 7f36b05a9740 1 Processor -- start 2024-04-25T00:23:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.267+0000 7f36b05a9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1580 0x55e675be5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.267+0000 7f36b05a9740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e675be1ea0 0x55e675be62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.267+0000 7f36b05a9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e675bec570 0x55e675be6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.267+0000 7f36b05a9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f36a81ad180 con 0x55e675be1580 2024-04-25T00:23:03.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.267+0000 7f36b05a9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f36a81ae5f0 con 0x55e675bec570 2024-04-25T00:23:03.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.267+0000 7f36b05a9740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f36a81ad850 con 0x55e675be1ea0 2024-04-25T00:23:03.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.267+0000 7f36b05a9740 10 client.0 _ll_register_callbacks cb 0x7f3690004530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:03.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.267+0000 7f36a6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1580 0x55e675be5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.267+0000 7f36a6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1580 0x55e675be5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48824/0 (socket says 172.21.15.107:48824) 2024-04-25T00:23:03.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.267+0000 7f36a6ffd640 1 -- 172.21.15.107:0/1388758337 learned_addr learned my addr 172.21.15.107:0/1388758337 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:03.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.267+0000 7f369ffff640 1 --2- 172.21.15.107:0/1388758337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e675be1ea0 0x55e675be62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.268+0000 7f36a77fe640 1 --2- 172.21.15.107:0/1388758337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e675bec570 0x55e675be6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.268+0000 7f36a6ffd640 1 -- 172.21.15.107:0/1388758337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e675be1ea0 msgr2=0x55e675be62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:03.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.268+0000 7f36a6ffd640 1 --2- 172.21.15.107:0/1388758337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e675be1ea0 0x55e675be62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.268+0000 7f36a6ffd640 1 -- 172.21.15.107:0/1388758337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e675bec570 msgr2=0x55e675be6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:03.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.268+0000 7f36a6ffd640 1 --2- 172.21.15.107:0/1388758337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e675bec570 0x55e675be6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.268+0000 7f36a6ffd640 1 -- 172.21.15.107:0/1388758337 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e675c65cc0 con 0x55e675be1580 2024-04-25T00:23:03.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.268+0000 7f36a77fe640 1 --2- 172.21.15.107:0/1388758337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e675bec570 0x55e675be6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:03.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.268+0000 7f36a6ffd640 1 --2- 172.21.15.107:0/1388758337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1580 0x55e675be5d90 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f36980ccd20 tx=0x7f3698097dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ef5e6adc856b8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:03.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.268+0000 7f3686ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:03.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.268+0000 7f3686ffd640 1 -- 172.21.15.107:0/1388758337 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698065060 con 0x55e675be1580 2024-04-25T00:23:03.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.268+0000 7f3686ffd640 1 -- 172.21.15.107:0/1388758337 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f36980980b0 con 0x55e675be1580 2024-04-25T00:23:03.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.268+0000 7f36b05a9740 1 -- 172.21.15.107:0/1388758337 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3698099d40 con 0x55e675be1580 2024-04-25T00:23:03.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.268+0000 7f3686ffd640 1 -- 172.21.15.107:0/1388758337 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f36980b9050 con 0x55e675be1580 2024-04-25T00:23:03.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.268+0000 7f36b05a9740 10 client.17148 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:03.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.268+0000 7f36b05a9740 1 -- 172.21.15.107:0/1388758337 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e675c66ea0 con 0x55e675be1580 2024-04-25T00:23:03.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.268+0000 7f3686ffd640 1 -- 172.21.15.107:0/1388758337 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3698099d40 con 0x55e675be1580 2024-04-25T00:23:03.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.268+0000 7f3686ffd640 1 -- 172.21.15.107:0/1388758337 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f369809af20 con 0x55e675be1580 2024-04-25T00:23:03.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.268+0000 7f36b05a9740 10 client.17148 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:03.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.268+0000 7f36b05a9740 10 client.17148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:03.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.268+0000 7f36b05a9740 20 client.17148 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:03.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.268+0000 7f36b05a9740 1 --2- 172.21.15.107:0/1388758337 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e675c674e0 0x55e675c69900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:03.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.268+0000 7f36b05a9740 4 client.17148 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:23:03.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.268+0000 7f36b05a9740 1 -- 172.21.15.107:0/1388758337 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e675b4df00 con 0x55e675c674e0 2024-04-25T00:23:03.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.269+0000 7f369ffff640 1 --2- 172.21.15.107:0/1388758337 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e675c674e0 0x55e675c69900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:03.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.269+0000 7f3686ffd640 10 client.17148 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:03.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.269+0000 7f369ffff640 1 --2- 172.21.15.107:0/1388758337 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e675c674e0 0x55e675c69900 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18c106acee36d00d server_cookie=77d0ac21ea68eaf9 in_seq=0 out_seq=0 2024-04-25T00:23:03.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.269+0000 7f3686ffd640 1 -- 172.21.15.107:0/1388758337 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e675b4df00 con 0x55e675c674e0 2024-04-25T00:23:03.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.269+0000 7f3686ffd640 10 client.17148 handle_command_reply: tid=0 2024-04-25T00:23:03.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.269+0000 7f36b05a9740 1 client.17148 shutdown 2024-04-25T00:23:03.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.269+0000 7f36b05a9740 1 -- 172.21.15.107:0/1388758337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1580 msgr2=0x55e675be5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:03.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.269+0000 7f36b05a9740 1 --2- 172.21.15.107:0/1388758337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1580 0x55e675be5d90 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f36980ccd20 tx=0x7f3698097dd0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.269+0000 7f36b05a9740 1 -- 172.21.15.107:0/1388758337 shutdown_connections 2024-04-25T00:23:03.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.269+0000 7f36b05a9740 1 --2- 172.21.15.107:0/1388758337 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e675c674e0 0x55e675c69900 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.269+0000 7f36b05a9740 1 --2- 172.21.15.107:0/1388758337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e675bec570 0x55e675be6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.269+0000 7f36b05a9740 1 --2- 172.21.15.107:0/1388758337 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e675be1ea0 0x55e675be62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.269+0000 7f36b05a9740 1 --2- 172.21.15.107:0/1388758337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e675be1580 0x55e675be5d90 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.269+0000 7f3686ffd640 0 client.17148 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:03.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.269+0000 7f36b05a9740 1 -- 172.21.15.107:0/1388758337 >> 172.21.15.107:0/1388758337 conn(0x55e675bdd8a0 msgr2=0x55e675be2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:03.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.269+0000 7f36b05a9740 1 -- 172.21.15.107:0/1388758337 shutdown_connections 2024-04-25T00:23:03.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.269+0000 7f36b05a9740 1 -- 172.21.15.107:0/1388758337 wait complete. 2024-04-25T00:23:03.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.270+0000 7f36b05a9740 20 client.17148 trim_cache size 0 max 16384 2024-04-25T00:23:03.277 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:23:03.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.270+0000 7f36ad89e640 1 -- 172.21.15.107:0/809581883 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f36940a2d00 msgr2=0x7f36940a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:03.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.270+0000 7f36ad89e640 1 --2- 172.21.15.107:0/809581883 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f36940a2d00 0x7f36940a51c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f36a002ae00 tx=0x7f36a0031510 comp rx=0 tx=0).stop 2024-04-25T00:23:03.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.270+0000 7f36ad89e640 1 -- 172.21.15.107:0/809581883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36a8101710 msgr2=0x7f36a8197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:03.278 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.270+0000 7f36ad89e640 1 --2- 172.21.15.107:0/809581883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36a8101710 0x7f36a8197fb0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f369800c920 tx=0x7f369800cde0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.270+0000 7f36ad89e640 1 -- 172.21.15.107:0/809581883 shutdown_connections 2024-04-25T00:23:03.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.270+0000 7f36ad89e640 1 --2- 172.21.15.107:0/809581883 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f36940a2d00 0x7f36940a51c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.270+0000 7f36ad89e640 1 --2- 172.21.15.107:0/809581883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36a8107720 0x7f36a819c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.270+0000 7f36ad89e640 1 --2- 172.21.15.107:0/809581883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36a81020d0 0x7f36a81984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.270+0000 7f36ad89e640 1 --2- 172.21.15.107:0/809581883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36a8101710 0x7f36a8197fb0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:03.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.270+0000 7f36ad89e640 1 -- 172.21.15.107:0/809581883 >> 172.21.15.107:0/809581883 conn(0x7f36a8078b90 msgr2=0x7f36a80ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:03.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.270+0000 7f36ad89e640 1 -- 172.21.15.107:0/809581883 shutdown_connections 2024-04-25T00:23:03.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:03.270+0000 7f36ad89e640 1 -- 172.21.15.107:0/809581883 wait complete. 2024-04-25T00:23:04.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:03 smithi195 ceph-mon[32833]: pgmap v201: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T00:23:04.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:03 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1459816529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:23:04.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:03 smithi170 ceph-mon[32807]: pgmap v201: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T00:23:04.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:03 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1459816529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:23:04.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:03 smithi107 ceph-mon[29153]: pgmap v201: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T00:23:04.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:03 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1459816529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:23:04.282 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:23:04.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:23:04.282 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:23:04.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.396+0000 7f890b0df640 1 -- 172.21.15.107:0/8050567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8904106630 msgr2=0x7f8904106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:04.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.396+0000 7f890b0df640 1 --2- 172.21.15.107:0/8050567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8904106630 0x7f8904106a10 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009930 tx=0x7f88f402a420 comp rx=0 tx=0).stop 2024-04-25T00:23:04.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.396+0000 7f890b0df640 1 -- 172.21.15.107:0/8050567 shutdown_connections 2024-04-25T00:23:04.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.396+0000 7f890b0df640 1 --2- 172.21.15.107:0/8050567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8904100fe0 0x7f890410d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.396+0000 7f890b0df640 1 --2- 172.21.15.107:0/8050567 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8904100620 0x7f8904100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.396+0000 7f890b0df640 1 --2- 172.21.15.107:0/8050567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8904106630 0x7f8904106a10 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.396+0000 7f890b0df640 1 -- 172.21.15.107:0/8050567 >> 172.21.15.107:0/8050567 conn(0x7f89040776a0 msgr2=0x7f8904077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:04.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.396+0000 7f890b0df640 1 -- 172.21.15.107:0/8050567 shutdown_connections 2024-04-25T00:23:04.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.396+0000 7f890b0df640 1 -- 172.21.15.107:0/8050567 wait complete. 2024-04-25T00:23:04.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.397+0000 7f890b0df640 1 Processor -- start 2024-04-25T00:23:04.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.397+0000 7f890b0df640 1 -- start start 2024-04-25T00:23:04.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.397+0000 7f890b0df640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8904100620 0x7f8904198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.397+0000 7f890b0df640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8904100fe0 0x7f8904198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.397+0000 7f890b0df640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8904106630 0x7f890419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.397+0000 7f890b0df640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8904111b40 con 0x7f8904100fe0 2024-04-25T00:23:04.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.397+0000 7f890b0df640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f89041119c0 con 0x7f8904100620 2024-04-25T00:23:04.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.397+0000 7f890b0df640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8904111cc0 con 0x7f8904106630 2024-04-25T00:23:04.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.397+0000 7f8903fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8904100fe0 0x7f8904198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.397+0000 7f8903fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8904100fe0 0x7f8904198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48840/0 (socket says 172.21.15.107:48840) 2024-04-25T00:23:04.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.397+0000 7f8903fff640 1 -- 172.21.15.107:0/3732875849 learned_addr learned my addr 172.21.15.107:0/3732875849 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:04.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.397+0000 7f8908e7b640 1 --2- 172.21.15.107:0/3732875849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8904100620 0x7f8904198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.397+0000 7f890967c640 1 --2- 172.21.15.107:0/3732875849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8904106630 0x7f890419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.397+0000 7f8903fff640 1 -- 172.21.15.107:0/3732875849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8904106630 msgr2=0x7f890419c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:04.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.397+0000 7f8903fff640 1 --2- 172.21.15.107:0/3732875849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8904106630 0x7f890419c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.397+0000 7f8903fff640 1 -- 172.21.15.107:0/3732875849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8904100620 msgr2=0x7f8904198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:04.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.397+0000 7f8903fff640 1 --2- 172.21.15.107:0/3732875849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8904100620 0x7f8904198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.397+0000 7f8903fff640 1 -- 172.21.15.107:0/3732875849 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890419d010 con 0x7f8904100fe0 2024-04-25T00:23:04.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.397+0000 7f890967c640 1 --2- 172.21.15.107:0/3732875849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8904106630 0x7f890419c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:04.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.398+0000 7f8903fff640 1 --2- 172.21.15.107:0/3732875849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8904100fe0 0x7f8904198560 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00d770 tx=0x7f88ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:04.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.398+0000 7f8901ffb640 1 -- 172.21.15.107:0/3732875849 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec017070 con 0x7f8904100fe0 2024-04-25T00:23:04.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.398+0000 7f890b0df640 1 -- 172.21.15.107:0/3732875849 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890419d300 con 0x7f8904100fe0 2024-04-25T00:23:04.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.398+0000 7f8901ffb640 1 -- 172.21.15.107:0/3732875849 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88ec005500 con 0x7f8904100fe0 2024-04-25T00:23:04.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.398+0000 7f8901ffb640 1 -- 172.21.15.107:0/3732875849 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ec004e00 con 0x7f8904100fe0 2024-04-25T00:23:04.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.398+0000 7f890b0df640 1 -- 172.21.15.107:0/3732875849 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89041a4b90 con 0x7f8904100fe0 2024-04-25T00:23:04.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.400+0000 7f8901ffb640 1 -- 172.21.15.107:0/3732875849 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f88ec004fa0 con 0x7f8904100fe0 2024-04-25T00:23:04.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.401+0000 7f890dde0740 1 Processor -- start 2024-04-25T00:23:04.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.401+0000 7f8901ffb640 1 --2- 172.21.15.107:0/3732875849 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f88f00a2d00 0x7f88f00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.401+0000 7f890dde0740 1 -- start start 2024-04-25T00:23:04.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.401+0000 7f890967c640 1 --2- 172.21.15.107:0/3732875849 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f88f00a2d00 0x7f88f00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.401+0000 7f8901ffb640 1 -- 172.21.15.107:0/3732875849 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f88ec0cf2a0 con 0x7f8904100fe0 2024-04-25T00:23:04.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.401+0000 7f890dde0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6e0e39980 0x55e6e0e37cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.401+0000 7f890dde0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6e0e38290 0x55e6e0e38710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.401+0000 7f890dde0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 0x55e6e0e45400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.401+0000 7f890dde0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e6e0d9c2c0 con 0x55e6e0e43010 2024-04-25T00:23:04.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.401+0000 7f890dde0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e6e0d44910 con 0x55e6e0e39980 2024-04-25T00:23:04.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.401+0000 7f890dde0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e6e0d98280 con 0x55e6e0e38290 2024-04-25T00:23:04.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.401+0000 7f890967c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 0x55e6e0e45400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.401+0000 7f890967c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 0x55e6e0e45400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48850/0 (socket says 172.21.15.107:48850) 2024-04-25T00:23:04.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.401+0000 7f890967c640 1 -- 172.21.15.107:0/4292546881 learned_addr learned my addr 172.21.15.107:0/4292546881 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:04.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.401+0000 7f890967c640 1 --2- 172.21.15.107:0/3732875849 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f88f00a2d00 0x7f88f00a51c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f88fc005fd0 tx=0x7f88fc00f730 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:04.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.402+0000 7f8908e7b640 1 --2- 172.21.15.107:0/4292546881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6e0e39980 0x55e6e0e37cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.402+0000 7f8903fff640 1 --2- 172.21.15.107:0/4292546881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6e0e38290 0x55e6e0e38710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.402+0000 7f890967c640 1 -- 172.21.15.107:0/4292546881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6e0e38290 msgr2=0x55e6e0e38710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:04.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.402+0000 7f890967c640 1 --2- 172.21.15.107:0/4292546881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6e0e38290 0x55e6e0e38710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.402+0000 7f890967c640 1 -- 172.21.15.107:0/4292546881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6e0e39980 msgr2=0x55e6e0e37cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:04.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.402+0000 7f890967c640 1 --2- 172.21.15.107:0/4292546881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6e0e39980 0x55e6e0e37cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.402+0000 7f890967c640 1 -- 172.21.15.107:0/4292546881 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6e0d55fb0 con 0x55e6e0e43010 2024-04-25T00:23:04.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.402+0000 7f890967c640 1 --2- 172.21.15.107:0/4292546881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 0x55e6e0e45400 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0126c0 tx=0x7f88fc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3d288b55707cd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:04.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.402+0000 7f88faffd640 1 -- 172.21.15.107:0/4292546881 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc00d710 con 0x55e6e0e43010 2024-04-25T00:23:04.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.402+0000 7f88faffd640 1 -- 172.21.15.107:0/4292546881 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88fc00dd90 con 0x55e6e0e43010 2024-04-25T00:23:04.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.402+0000 7f88faffd640 1 -- 172.21.15.107:0/4292546881 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc0103d0 con 0x55e6e0e43010 2024-04-25T00:23:04.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.403+0000 7f890dde0740 1 -- 172.21.15.107:0/4292546881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 msgr2=0x55e6e0e45400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:04.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.403+0000 7f890dde0740 1 --2- 172.21.15.107:0/4292546881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 0x55e6e0e45400 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0126c0 tx=0x7f88fc01a000 comp rx=0 tx=0).stop 2024-04-25T00:23:04.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.403+0000 7f890dde0740 1 -- 172.21.15.107:0/4292546881 shutdown_connections 2024-04-25T00:23:04.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.403+0000 7f890dde0740 1 --2- 172.21.15.107:0/4292546881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 0x55e6e0e45400 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.403+0000 7f890dde0740 1 --2- 172.21.15.107:0/4292546881 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6e0e38290 0x55e6e0e38710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.403+0000 7f890dde0740 1 --2- 172.21.15.107:0/4292546881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6e0e39980 0x55e6e0e37cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.403+0000 7f890dde0740 1 -- 172.21.15.107:0/4292546881 >> 172.21.15.107:0/4292546881 conn(0x55e6e0e00950 msgr2=0x55e6e0e347f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:04.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.403+0000 7f890dde0740 1 -- 172.21.15.107:0/4292546881 shutdown_connections 2024-04-25T00:23:04.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.403+0000 7f890dde0740 1 -- 172.21.15.107:0/4292546881 wait complete. 2024-04-25T00:23:04.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.403+0000 7f890dde0740 1 -- start start 2024-04-25T00:23:04.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.403+0000 7f890dde0740 1 Processor -- start 2024-04-25T00:23:04.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.403+0000 7f890dde0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6e0e38290 0x55e6e0e3fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.403+0000 7f890dde0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6e0e39980 0x55e6e0e40410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.404+0000 7f890dde0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 0x55e6e0e40950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.404+0000 7f890dde0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e6e0e45ea0 con 0x55e6e0e43010 2024-04-25T00:23:04.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.404+0000 7f890dde0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e6e0d57e90 con 0x55e6e0e38290 2024-04-25T00:23:04.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.404+0000 7f890dde0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e6e0e46020 con 0x55e6e0e39980 2024-04-25T00:23:04.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.404+0000 7f890dde0740 10 client.0 _ll_register_callbacks cb 0x55e6e0d93970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:04.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.404+0000 7f890967c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 0x55e6e0e40950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.404+0000 7f890967c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 0x55e6e0e40950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48854/0 (socket says 172.21.15.107:48854) 2024-04-25T00:23:04.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.404+0000 7f890967c640 1 -- 172.21.15.107:0/2626324233 learned_addr learned my addr 172.21.15.107:0/2626324233 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:04.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.404+0000 7f8908e7b640 1 --2- 172.21.15.107:0/2626324233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6e0e38290 0x55e6e0e3fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.404+0000 7f8903fff640 1 --2- 172.21.15.107:0/2626324233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6e0e39980 0x55e6e0e40410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.404+0000 7f890967c640 1 -- 172.21.15.107:0/2626324233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6e0e39980 msgr2=0x55e6e0e40410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:04.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.404+0000 7f890967c640 1 --2- 172.21.15.107:0/2626324233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6e0e39980 0x55e6e0e40410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.404+0000 7f890967c640 1 -- 172.21.15.107:0/2626324233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6e0e38290 msgr2=0x55e6e0e3fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:04.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.404+0000 7f890967c640 1 --2- 172.21.15.107:0/2626324233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6e0e38290 0x55e6e0e3fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.404+0000 7f890967c640 1 -- 172.21.15.107:0/2626324233 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6e0b787e0 con 0x55e6e0e43010 2024-04-25T00:23:04.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.404+0000 7f890967c640 1 --2- 172.21.15.107:0/2626324233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 0x55e6e0e40950 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f88fc01eff0 tx=0x7f88fc009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=126e0eb8a558ff63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:04.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.404+0000 7f88e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:04.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.404+0000 7f88e3fff640 1 -- 172.21.15.107:0/2626324233 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc033060 con 0x55e6e0e43010 2024-04-25T00:23:04.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.404+0000 7f88e3fff640 1 -- 172.21.15.107:0/2626324233 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f88fc012050 con 0x55e6e0e43010 2024-04-25T00:23:04.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.404+0000 7f890dde0740 1 -- 172.21.15.107:0/2626324233 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88fc00dd90 con 0x55e6e0e43010 2024-04-25T00:23:04.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.404+0000 7f88e3fff640 1 -- 172.21.15.107:0/2626324233 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f88fc03a050 con 0x55e6e0e43010 2024-04-25T00:23:04.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.404+0000 7f890dde0740 10 client.17160 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:04.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.404+0000 7f890dde0740 1 -- 172.21.15.107:0/2626324233 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e6e0bd8f70 con 0x55e6e0e43010 2024-04-25T00:23:04.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.405+0000 7f88e3fff640 1 -- 172.21.15.107:0/2626324233 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88fc00dd90 con 0x55e6e0e43010 2024-04-25T00:23:04.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.405+0000 7f88e3fff640 1 -- 172.21.15.107:0/2626324233 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f88fc0115f0 con 0x55e6e0e43010 2024-04-25T00:23:04.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.405+0000 7f890dde0740 10 client.17160 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:04.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.405+0000 7f890dde0740 10 client.17160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:04.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.405+0000 7f890dde0740 20 client.17160 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:04.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.405+0000 7f890dde0740 1 --2- 172.21.15.107:0/2626324233 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e6e0e53030 0x55e6e0e55420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.405+0000 7f890dde0740 4 client.17160 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:04.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.405+0000 7f890dde0740 1 -- 172.21.15.107:0/2626324233 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6e0d705a0 con 0x55e6e0e53030 2024-04-25T00:23:04.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.405+0000 7f8908e7b640 1 --2- 172.21.15.107:0/2626324233 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e6e0e53030 0x55e6e0e55420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.405+0000 7f88e3fff640 10 client.17160 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:04.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.405+0000 7f8908e7b640 1 --2- 172.21.15.107:0/2626324233 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e6e0e53030 0x55e6e0e55420 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7218e859ba44c20 server_cookie=5c2a602b53777436 in_seq=0 out_seq=0 2024-04-25T00:23:04.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.406+0000 7f88e3fff640 1 -- 172.21.15.107:0/2626324233 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e6e0d705a0 con 0x55e6e0e53030 2024-04-25T00:23:04.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.406+0000 7f88e3fff640 10 client.17160 handle_command_reply: tid=0 2024-04-25T00:23:04.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.406+0000 7f890dde0740 1 client.17160 shutdown 2024-04-25T00:23:04.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.406+0000 7f890dde0740 1 -- 172.21.15.107:0/2626324233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 msgr2=0x55e6e0e40950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:04.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.406+0000 7f890dde0740 1 --2- 172.21.15.107:0/2626324233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 0x55e6e0e40950 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f88fc01eff0 tx=0x7f88fc009500 comp rx=0 tx=0).stop 2024-04-25T00:23:04.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.406+0000 7f890dde0740 1 -- 172.21.15.107:0/2626324233 shutdown_connections 2024-04-25T00:23:04.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.406+0000 7f890dde0740 1 --2- 172.21.15.107:0/2626324233 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e6e0e53030 0x55e6e0e55420 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.406+0000 7f890dde0740 1 --2- 172.21.15.107:0/2626324233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 0x55e6e0e40950 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.406+0000 7f890dde0740 1 --2- 172.21.15.107:0/2626324233 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6e0e39980 0x55e6e0e40410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.406+0000 7f88e3fff640 0 client.17160 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:04.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.406+0000 7f890dde0740 1 --2- 172.21.15.107:0/2626324233 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6e0e38290 0x55e6e0e3fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.406+0000 7f890dde0740 1 -- 172.21.15.107:0/2626324233 >> 172.21.15.107:0/2626324233 conn(0x55e6e0e00950 msgr2=0x55e6e0e3a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:04.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.407+0000 7f890dde0740 1 -- 172.21.15.107:0/2626324233 shutdown_connections 2024-04-25T00:23:04.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.407+0000 7f890dde0740 1 -- 172.21.15.107:0/2626324233 wait complete. 2024-04-25T00:23:04.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.407+0000 7f890dde0740 20 client.17160 trim_cache size 0 max 16384 2024-04-25T00:23:04.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.424+0000 7f890dde0740 1 Processor -- start 2024-04-25T00:23:04.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.424+0000 7f890dde0740 1 -- start start 2024-04-25T00:23:04.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.424+0000 7f890dde0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6e0e38290 0x55e6e0e4eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.424+0000 7f890dde0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6e0e39980 0x55e6e0e41e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.424+0000 7f890dde0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 0x55e6e0e5aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.424+0000 7f890dde0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f89041b1350 con 0x55e6e0e43010 2024-04-25T00:23:04.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.424+0000 7f890dde0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f89041aec20 con 0x55e6e0e38290 2024-04-25T00:23:04.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.424+0000 7f890dde0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f89041aca70 con 0x55e6e0e39980 2024-04-25T00:23:04.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.424+0000 7f890967c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 0x55e6e0e5aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.424+0000 7f890967c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 0x55e6e0e5aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48864/0 (socket says 172.21.15.107:48864) 2024-04-25T00:23:04.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.424+0000 7f890967c640 1 -- 172.21.15.107:0/1473418614 learned_addr learned my addr 172.21.15.107:0/1473418614 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:04.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.425+0000 7f8903fff640 1 --2- 172.21.15.107:0/1473418614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6e0e39980 0x55e6e0e41e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.425+0000 7f890967c640 1 -- 172.21.15.107:0/1473418614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6e0e39980 msgr2=0x55e6e0e41e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:04.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.425+0000 7f890967c640 1 --2- 172.21.15.107:0/1473418614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6e0e39980 0x55e6e0e41e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.425+0000 7f890967c640 1 -- 172.21.15.107:0/1473418614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6e0e38290 msgr2=0x55e6e0e4eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:04.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.425+0000 7f890967c640 1 --2- 172.21.15.107:0/1473418614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6e0e38290 0x55e6e0e4eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.425+0000 7f890967c640 1 -- 172.21.15.107:0/1473418614 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6e0a70170 con 0x55e6e0e43010 2024-04-25T00:23:04.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.425+0000 7f8903fff640 1 --2- 172.21.15.107:0/1473418614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6e0e39980 0x55e6e0e41e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:04.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.425+0000 7f890967c640 1 --2- 172.21.15.107:0/1473418614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 0x55e6e0e5aa50 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f88fc01f040 tx=0x7f88fc01d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e41a68177395aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:04.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.425+0000 7f88fa7fc640 1 -- 172.21.15.107:0/1473418614 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc033070 con 0x55e6e0e43010 2024-04-25T00:23:04.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.425+0000 7f88fa7fc640 1 -- 172.21.15.107:0/1473418614 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88fc01d7f0 con 0x55e6e0e43010 2024-04-25T00:23:04.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.425+0000 7f88fa7fc640 1 -- 172.21.15.107:0/1473418614 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc011fb0 con 0x55e6e0e43010 2024-04-25T00:23:04.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.425+0000 7f890dde0740 1 -- 172.21.15.107:0/1473418614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 msgr2=0x55e6e0e5aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:04.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.425+0000 7f890dde0740 1 --2- 172.21.15.107:0/1473418614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 0x55e6e0e5aa50 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f88fc01f040 tx=0x7f88fc01d4a0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.425+0000 7f890dde0740 1 -- 172.21.15.107:0/1473418614 shutdown_connections 2024-04-25T00:23:04.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.426+0000 7f890dde0740 1 --2- 172.21.15.107:0/1473418614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 0x55e6e0e5aa50 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.426+0000 7f890dde0740 1 --2- 172.21.15.107:0/1473418614 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6e0e39980 0x55e6e0e41e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.426+0000 7f890dde0740 1 --2- 172.21.15.107:0/1473418614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6e0e38290 0x55e6e0e4eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.426+0000 7f890dde0740 1 -- 172.21.15.107:0/1473418614 >> 172.21.15.107:0/1473418614 conn(0x55e6e0e00950 msgr2=0x55e6e0e41700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:04.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.426+0000 7f890dde0740 1 -- 172.21.15.107:0/1473418614 shutdown_connections 2024-04-25T00:23:04.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.426+0000 7f890dde0740 1 -- 172.21.15.107:0/1473418614 wait complete. 2024-04-25T00:23:04.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.426+0000 7f890dde0740 1 -- start start 2024-04-25T00:23:04.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.426+0000 7f890dde0740 1 Processor -- start 2024-04-25T00:23:04.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.426+0000 7f890dde0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6e0e38290 0x55e6e0e4b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.426+0000 7f890dde0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6e0e39980 0x55e6e0e4b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.426+0000 7f890dde0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 0x55e6e0e48040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.426+0000 7f890dde0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f89041ad0e0 con 0x55e6e0e43010 2024-04-25T00:23:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.426+0000 7f890dde0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f89041ae550 con 0x55e6e0e38290 2024-04-25T00:23:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.426+0000 7f890dde0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f89041ad7b0 con 0x55e6e0e39980 2024-04-25T00:23:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.426+0000 7f890dde0740 10 client.0 _ll_register_callbacks cb 0x7f88fc012610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.426+0000 7f890967c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 0x55e6e0e48040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.426+0000 7f890967c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 0x55e6e0e48040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48880/0 (socket says 172.21.15.107:48880) 2024-04-25T00:23:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.426+0000 7f890967c640 1 -- 172.21.15.107:0/2868702927 learned_addr learned my addr 172.21.15.107:0/2868702927 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.427+0000 7f8903fff640 1 --2- 172.21.15.107:0/2868702927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6e0e39980 0x55e6e0e4b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.427+0000 7f890967c640 1 -- 172.21.15.107:0/2868702927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6e0e39980 msgr2=0x55e6e0e4b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.427+0000 7f890967c640 1 --2- 172.21.15.107:0/2868702927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6e0e39980 0x55e6e0e4b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.434 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.427+0000 7f8908e7b640 1 --2- 172.21.15.107:0/2868702927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6e0e38290 0x55e6e0e4b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.427+0000 7f890967c640 1 -- 172.21.15.107:0/2868702927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6e0e38290 msgr2=0x55e6e0e4b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:04.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.427+0000 7f890967c640 1 --2- 172.21.15.107:0/2868702927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6e0e38290 0x55e6e0e4b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.427+0000 7f890967c640 1 -- 172.21.15.107:0/2868702927 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6e0e48580 con 0x55e6e0e43010 2024-04-25T00:23:04.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.427+0000 7f890967c640 1 --2- 172.21.15.107:0/2868702927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 0x55e6e0e48040 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00fcd0 tx=0x7f88fc011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46705f4add3ff9e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:04.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.427+0000 7f88e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:04.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.427+0000 7f88e3fff640 1 -- 172.21.15.107:0/2868702927 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc033040 con 0x55e6e0e43010 2024-04-25T00:23:04.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.427+0000 7f88e3fff640 1 -- 172.21.15.107:0/2868702927 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f88fc011430 con 0x55e6e0e43010 2024-04-25T00:23:04.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.427+0000 7f890dde0740 1 -- 172.21.15.107:0/2868702927 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88fc01d7f0 con 0x55e6e0e43010 2024-04-25T00:23:04.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.427+0000 7f88e3fff640 1 -- 172.21.15.107:0/2868702927 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f88fc051050 con 0x55e6e0e43010 2024-04-25T00:23:04.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.427+0000 7f890dde0740 10 client.17166 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:04.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.427+0000 7f890dde0740 1 -- 172.21.15.107:0/2868702927 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e6e0eb9970 con 0x55e6e0e43010 2024-04-25T00:23:04.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.427+0000 7f88e3fff640 1 -- 172.21.15.107:0/2868702927 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88fc01d7f0 con 0x55e6e0e43010 2024-04-25T00:23:04.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.427+0000 7f88e3fff640 1 -- 172.21.15.107:0/2868702927 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f88fc00d630 con 0x55e6e0e43010 2024-04-25T00:23:04.435 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.427+0000 7f890dde0740 10 client.17166 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:04.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.427+0000 7f890dde0740 10 client.17166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:04.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.427+0000 7f890dde0740 20 client.17166 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:04.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.427+0000 7f890dde0740 1 --2- 172.21.15.107:0/2868702927 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e6e0eba010 0x55e6e0ebc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.427+0000 7f890dde0740 4 client.17166 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:23:04.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.427+0000 7f890dde0740 1 -- 172.21.15.107:0/2868702927 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e6e0da9800 con 0x55e6e0eba010 2024-04-25T00:23:04.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.428+0000 7f8908e7b640 1 --2- 172.21.15.107:0/2868702927 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e6e0eba010 0x55e6e0ebc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.428+0000 7f88e3fff640 10 client.17166 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:04.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.428+0000 7f8908e7b640 1 --2- 172.21.15.107:0/2868702927 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e6e0eba010 0x55e6e0ebc430 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45e0a8b8a9487d1b server_cookie=eeb4a1b76efb6c88 in_seq=0 out_seq=0 2024-04-25T00:23:04.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.429+0000 7f88e3fff640 1 -- 172.21.15.107:0/2868702927 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e6e0da9800 con 0x55e6e0eba010 2024-04-25T00:23:04.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.429+0000 7f88e3fff640 10 client.17166 handle_command_reply: tid=0 2024-04-25T00:23:04.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.429+0000 7f890dde0740 1 client.17166 shutdown 2024-04-25T00:23:04.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.429+0000 7f890dde0740 1 -- 172.21.15.107:0/2868702927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 msgr2=0x55e6e0e48040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:04.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.429+0000 7f890dde0740 1 --2- 172.21.15.107:0/2868702927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 0x55e6e0e48040 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00fcd0 tx=0x7f88fc011bb0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.436 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.429+0000 7f890dde0740 1 -- 172.21.15.107:0/2868702927 shutdown_connections 2024-04-25T00:23:04.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.429+0000 7f890dde0740 1 --2- 172.21.15.107:0/2868702927 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e6e0eba010 0x55e6e0ebc430 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.429+0000 7f890dde0740 1 --2- 172.21.15.107:0/2868702927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e6e0e43010 0x55e6e0e48040 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.429+0000 7f890dde0740 1 --2- 172.21.15.107:0/2868702927 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e6e0e39980 0x55e6e0e4b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.429+0000 7f88e3fff640 0 client.17166 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:04.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.429+0000 7f890dde0740 1 --2- 172.21.15.107:0/2868702927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e6e0e38290 0x55e6e0e4b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.429+0000 7f890dde0740 1 -- 172.21.15.107:0/2868702927 >> 172.21.15.107:0/2868702927 conn(0x55e6e0e00950 msgr2=0x55e6e0e41f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:04.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.430+0000 7f890dde0740 1 -- 172.21.15.107:0/2868702927 shutdown_connections 2024-04-25T00:23:04.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.430+0000 7f890dde0740 1 -- 172.21.15.107:0/2868702927 wait complete. 2024-04-25T00:23:04.437 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.430+0000 7f890dde0740 20 client.17166 trim_cache size 0 max 16384 2024-04-25T00:23:04.437 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:23:04.438 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-04-25T00:23:04.438 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "8e40b219-dcdf-4c56-8b5a-70dcc44e2969", 2024-04-25T00:23:04.438 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-04-25T00:23:04.438 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:23:04.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.430+0000 7f890b0df640 1 -- 172.21.15.107:0/3732875849 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f88f00a2d00 msgr2=0x7f88f00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:04.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.430+0000 7f890b0df640 1 --2- 172.21.15.107:0/3732875849 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f88f00a2d00 0x7f88f00a51c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f88fc005fd0 tx=0x7f88fc00f730 comp rx=0 tx=0).stop 2024-04-25T00:23:04.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.430+0000 7f890b0df640 1 -- 172.21.15.107:0/3732875849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8904100fe0 msgr2=0x7f8904198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:04.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.430+0000 7f890b0df640 1 --2- 172.21.15.107:0/3732875849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8904100fe0 0x7f8904198560 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00d770 tx=0x7f88ec00dc30 comp rx=0 tx=0).stop 2024-04-25T00:23:04.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.430+0000 7f890b0df640 1 -- 172.21.15.107:0/3732875849 shutdown_connections 2024-04-25T00:23:04.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.430+0000 7f890b0df640 1 --2- 172.21.15.107:0/3732875849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8904106630 0x7f890419c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.430+0000 7f890b0df640 1 --2- 172.21.15.107:0/3732875849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8904100fe0 0x7f8904198560 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.430+0000 7f890b0df640 1 --2- 172.21.15.107:0/3732875849 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f88f00a2d00 0x7f88f00a51c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.430+0000 7f890b0df640 1 --2- 172.21.15.107:0/3732875849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8904100620 0x7f8904198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.430+0000 7f890b0df640 1 -- 172.21.15.107:0/3732875849 >> 172.21.15.107:0/3732875849 conn(0x7f89040776a0 msgr2=0x7f8904103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:04.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.430+0000 7f890b0df640 1 -- 172.21.15.107:0/3732875849 shutdown_connections 2024-04-25T00:23:04.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.430+0000 7f890b0df640 1 -- 172.21.15.107:0/3732875849 wait complete. 2024-04-25T00:23:04.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:23:04.442 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:23:04.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.541+0000 7f8f8ddf9640 1 -- 172.21.15.107:0/3316311915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f8806a780 msgr2=0x7f8f8810e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:04.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.541+0000 7f8f8ddf9640 1 --2- 172.21.15.107:0/3316311915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f8806a780 0x7f8f8810e5f0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009cd0 tx=0x7f8f8002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.541+0000 7f8f8ddf9640 1 -- 172.21.15.107:0/3316311915 shutdown_connections 2024-04-25T00:23:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.541+0000 7f8f8ddf9640 1 --2- 172.21.15.107:0/3316311915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f8806a780 0x7f8f8810e5f0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.541+0000 7f8f8ddf9640 1 --2- 172.21.15.107:0/3316311915 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f88069dc0 0x7f8f8806a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.541+0000 7f8f8ddf9640 1 --2- 172.21.15.107:0/3316311915 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f88107730 0x7f8f88107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.541+0000 7f8f8ddf9640 1 -- 172.21.15.107:0/3316311915 >> 172.21.15.107:0/3316311915 conn(0x7f8f880776a0 msgr2=0x7f8f88077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.541+0000 7f8f8ddf9640 1 -- 172.21.15.107:0/3316311915 shutdown_connections 2024-04-25T00:23:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.541+0000 7f8f8ddf9640 1 -- 172.21.15.107:0/3316311915 wait complete. 2024-04-25T00:23:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.542+0000 7f8f8ddf9640 1 Processor -- start 2024-04-25T00:23:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.542+0000 7f8f8ddf9640 1 -- start start 2024-04-25T00:23:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.542+0000 7f8f8ddf9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f88069dc0 0x7f8f88197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.542+0000 7f8f8ddf9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f8806a780 0x7f8f881984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.542+0000 7f8f8ddf9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f88107730 0x7f8f8819c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.542+0000 7f8f8ddf9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8f88111b40 con 0x7f8f88069dc0 2024-04-25T00:23:04.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.542+0000 7f8f8ddf9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8f881119c0 con 0x7f8f88107730 2024-04-25T00:23:04.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.542+0000 7f8f8ddf9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8f88111cc0 con 0x7f8f8806a780 2024-04-25T00:23:04.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.542+0000 7f8f877fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f88069dc0 0x7f8f88197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.542+0000 7f8f877fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f88069dc0 0x7f8f88197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48898/0 (socket says 172.21.15.107:48898) 2024-04-25T00:23:04.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.542+0000 7f8f877fe640 1 -- 172.21.15.107:0/22293798 learned_addr learned my addr 172.21.15.107:0/22293798 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:04.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.542+0000 7f8f86ffd640 1 --2- 172.21.15.107:0/22293798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f8806a780 0x7f8f881984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.543+0000 7f8f87fff640 1 --2- 172.21.15.107:0/22293798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f88107730 0x7f8f8819c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.543+0000 7f8f877fe640 1 -- 172.21.15.107:0/22293798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f8806a780 msgr2=0x7f8f881984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:04.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.543+0000 7f8f877fe640 1 --2- 172.21.15.107:0/22293798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f8806a780 0x7f8f881984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.543+0000 7f8f877fe640 1 -- 172.21.15.107:0/22293798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f88107730 msgr2=0x7f8f8819c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:04.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.543+0000 7f8f877fe640 1 --2- 172.21.15.107:0/22293798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f88107730 0x7f8f8819c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.543+0000 7f8f877fe640 1 -- 172.21.15.107:0/22293798 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8819cf90 con 0x7f8f88069dc0 2024-04-25T00:23:04.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.543+0000 7f8f87fff640 1 --2- 172.21.15.107:0/22293798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f88107730 0x7f8f8819c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:04.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.543+0000 7f8f877fe640 1 --2- 172.21.15.107:0/22293798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f88069dc0 0x7f8f88197fa0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800c890 tx=0x7f8f7800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:04.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.543+0000 7f8f84ff9640 1 -- 172.21.15.107:0/22293798 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f78004220 con 0x7f8f88069dc0 2024-04-25T00:23:04.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.543+0000 7f8f8ddf9640 1 -- 172.21.15.107:0/22293798 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8819d280 con 0x7f8f88069dc0 2024-04-25T00:23:04.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.543+0000 7f8f84ff9640 1 -- 172.21.15.107:0/22293798 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f780043c0 con 0x7f8f88069dc0 2024-04-25T00:23:04.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.543+0000 7f8f84ff9640 1 -- 172.21.15.107:0/22293798 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f78004a00 con 0x7f8f88069dc0 2024-04-25T00:23:04.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.543+0000 7f8f8ddf9640 1 -- 172.21.15.107:0/22293798 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f881a4b10 con 0x7f8f88069dc0 2024-04-25T00:23:04.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.545+0000 7f8f84ff9640 1 -- 172.21.15.107:0/22293798 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f8f780079b0 con 0x7f8f88069dc0 2024-04-25T00:23:04.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.546+0000 7f8f90be7740 1 Processor -- start 2024-04-25T00:23:04.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.546+0000 7f8f90be7740 1 -- start start 2024-04-25T00:23:04.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.546+0000 7f8f90be7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3a630 0x55e240e3aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.546+0000 7f8f90be7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e240e3af50 0x55e240e450e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.546+0000 7f8f84ff9640 1 --2- 172.21.15.107:0/22293798 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8f700a2d00 0x7f8f700a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.546+0000 7f8f90be7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e240e45620 0x55e240e47a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.546+0000 7f8f90be7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e240d9f6b0 con 0x55e240e3a630 2024-04-25T00:23:04.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.546+0000 7f8f90be7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e240d47880 con 0x55e240e45620 2024-04-25T00:23:04.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.546+0000 7f8f90be7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e240d9b670 con 0x55e240e3af50 2024-04-25T00:23:04.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.546+0000 7f8f87fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3a630 0x55e240e3aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.546+0000 7f8f877fe640 1 --2- 172.21.15.107:0/22293798 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8f700a2d00 0x7f8f700a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.546+0000 7f8f87fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3a630 0x55e240e3aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48902/0 (socket says 172.21.15.107:48902) 2024-04-25T00:23:04.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.546+0000 7f8f87fff640 1 -- 172.21.15.107:0/1480060417 learned_addr learned my addr 172.21.15.107:0/1480060417 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:04.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.546+0000 7f8f84ff9640 1 -- 172.21.15.107:0/22293798 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f8f780cf4b0 con 0x7f8f88069dc0 2024-04-25T00:23:04.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.546+0000 7f8f86ffd640 1 --2- 172.21.15.107:0/1480060417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e240e3af50 0x55e240e450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.546+0000 7f8f87fff640 1 --2- 172.21.15.107:0/1480060417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e240e45620 0x55e240e47a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.546+0000 7f8f87fff640 1 -- 172.21.15.107:0/1480060417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e240e3af50 msgr2=0x55e240e450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:04.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.546+0000 7f8f87fff640 1 --2- 172.21.15.107:0/1480060417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e240e3af50 0x55e240e450e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.546+0000 7f8f87fff640 1 -- 172.21.15.107:0/1480060417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e240e45620 msgr2=0x55e240e47a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:04.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.546+0000 7f8f87fff640 1 --2- 172.21.15.107:0/1480060417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e240e45620 0x55e240e47a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.546+0000 7f8f87fff640 1 -- 172.21.15.107:0/1480060417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e240b7d7d0 con 0x55e240e3a630 2024-04-25T00:23:04.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.547+0000 7f8f877fe640 1 --2- 172.21.15.107:0/22293798 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8f700a2d00 0x7f8f700a51c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8f7809b300 tx=0x7f8f7806d5c0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:04.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.547+0000 7f8f87fff640 1 --2- 172.21.15.107:0/1480060417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3a630 0x55e240e3aa10 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f8f80005e30 tx=0x7f8f80004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce31e7a223f83ed2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:04.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.547+0000 7f8f6dffb640 1 -- 172.21.15.107:0/1480060417 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80031c40 con 0x55e240e3a630 2024-04-25T00:23:04.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.547+0000 7f8f6dffb640 1 -- 172.21.15.107:0/1480060417 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f80046070 con 0x55e240e3a630 2024-04-25T00:23:04.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.547+0000 7f8f6dffb640 1 -- 172.21.15.107:0/1480060417 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80004820 con 0x55e240e3a630 2024-04-25T00:23:04.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.547+0000 7f8f90be7740 1 -- 172.21.15.107:0/1480060417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3a630 msgr2=0x55e240e3aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:04.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.547+0000 7f8f90be7740 1 --2- 172.21.15.107:0/1480060417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3a630 0x55e240e3aa10 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f8f80005e30 tx=0x7f8f80004a60 comp rx=0 tx=0).stop 2024-04-25T00:23:04.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.547+0000 7f8f90be7740 1 -- 172.21.15.107:0/1480060417 shutdown_connections 2024-04-25T00:23:04.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.547+0000 7f8f90be7740 1 --2- 172.21.15.107:0/1480060417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e240e45620 0x55e240e47a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.547+0000 7f8f90be7740 1 --2- 172.21.15.107:0/1480060417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e240e3af50 0x55e240e450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.547+0000 7f8f90be7740 1 --2- 172.21.15.107:0/1480060417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3a630 0x55e240e3aa10 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.547+0000 7f8f90be7740 1 -- 172.21.15.107:0/1480060417 >> 172.21.15.107:0/1480060417 conn(0x55e240e36950 msgr2=0x55e240e36d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:04.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.547+0000 7f8f90be7740 1 -- 172.21.15.107:0/1480060417 shutdown_connections 2024-04-25T00:23:04.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.547+0000 7f8f90be7740 1 -- 172.21.15.107:0/1480060417 wait complete. 2024-04-25T00:23:04.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.548+0000 7f8f90be7740 1 -- start start 2024-04-25T00:23:04.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.548+0000 7f8f90be7740 1 Processor -- start 2024-04-25T00:23:04.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.548+0000 7f8f90be7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e240e3a630 0x55e240e425e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.548+0000 7f8f90be7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3af50 0x55e240e42b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.548+0000 7f8f90be7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e240e45620 0x55e240e43060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.548+0000 7f8f90be7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e240e483b0 con 0x55e240e3af50 2024-04-25T00:23:04.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.548+0000 7f8f90be7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e240d5af50 con 0x55e240e3a630 2024-04-25T00:23:04.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.548+0000 7f8f90be7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e240e48530 con 0x55e240e45620 2024-04-25T00:23:04.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.548+0000 7f8f90be7740 10 client.0 _ll_register_callbacks cb 0x55e240d96000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:04.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.548+0000 7f8f86ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3af50 0x55e240e42b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.548+0000 7f8f86ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3af50 0x55e240e42b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48910/0 (socket says 172.21.15.107:48910) 2024-04-25T00:23:04.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.548+0000 7f8f86ffd640 1 -- 172.21.15.107:0/3649319928 learned_addr learned my addr 172.21.15.107:0/3649319928 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:04.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.548+0000 7f8f87fff640 1 --2- 172.21.15.107:0/3649319928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e240e3a630 0x55e240e425e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.548+0000 7f8f87fff640 1 --2- 172.21.15.107:0/3649319928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e240e45620 0x55e240e43060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.549+0000 7f8f86ffd640 1 -- 172.21.15.107:0/3649319928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e240e45620 msgr2=0x55e240e43060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:04.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.549+0000 7f8f86ffd640 1 --2- 172.21.15.107:0/3649319928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e240e45620 0x55e240e43060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.549+0000 7f8f86ffd640 1 -- 172.21.15.107:0/3649319928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e240e3a630 msgr2=0x55e240e425e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:04.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.549+0000 7f8f86ffd640 1 --2- 172.21.15.107:0/3649319928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e240e3a630 0x55e240e425e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.549+0000 7f8f86ffd640 1 -- 172.21.15.107:0/3649319928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e240ab33f0 con 0x55e240e3af50 2024-04-25T00:23:04.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.549+0000 7f8f86ffd640 1 --2- 172.21.15.107:0/3649319928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3af50 0x55e240e42b20 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00ba90 tx=0x7f8f7c00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0cdcd52cb91437b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:04.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.549+0000 7f8f66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:04.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.549+0000 7f8f66ffd640 1 -- 172.21.15.107:0/3649319928 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c003e80 con 0x55e240e3af50 2024-04-25T00:23:04.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.549+0000 7f8f90be7740 1 -- 172.21.15.107:0/3649319928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f80046070 con 0x55e240e3af50 2024-04-25T00:23:04.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.549+0000 7f8f66ffd640 1 -- 172.21.15.107:0/3649319928 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f8f7c005020 con 0x55e240e3af50 2024-04-25T00:23:04.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.549+0000 7f8f66ffd640 1 -- 172.21.15.107:0/3649319928 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8f7c005220 con 0x55e240e3af50 2024-04-25T00:23:04.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.549+0000 7f8f90be7740 10 client.17178 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:04.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.549+0000 7f8f90be7740 1 -- 172.21.15.107:0/3649319928 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e240a730f0 con 0x55e240e3af50 2024-04-25T00:23:04.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.549+0000 7f8f66ffd640 1 -- 172.21.15.107:0/3649319928 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f80046070 con 0x55e240e3af50 2024-04-25T00:23:04.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.549+0000 7f8f66ffd640 1 -- 172.21.15.107:0/3649319928 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f8f7c01b590 con 0x55e240e3af50 2024-04-25T00:23:04.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.549+0000 7f8f90be7740 10 client.17178 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:04.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.549+0000 7f8f90be7740 10 client.17178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:04.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.549+0000 7f8f90be7740 20 client.17178 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:04.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.549+0000 7f8f90be7740 1 --2- 172.21.15.107:0/3649319928 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e240e55d00 0x55e240e58180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.549+0000 7f8f90be7740 4 client.17178 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:04.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.549+0000 7f8f90be7740 1 -- 172.21.15.107:0/3649319928 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e240dacbf0 con 0x55e240e55d00 2024-04-25T00:23:04.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.550+0000 7f8f87fff640 1 --2- 172.21.15.107:0/3649319928 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e240e55d00 0x55e240e58180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.550+0000 7f8f66ffd640 10 client.17178 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:04.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.550+0000 7f8f87fff640 1 --2- 172.21.15.107:0/3649319928 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e240e55d00 0x55e240e58180 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d9a3436b74d05fb server_cookie=26411719dda36468 in_seq=0 out_seq=0 2024-04-25T00:23:04.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.551+0000 7f8f66ffd640 1 -- 172.21.15.107:0/3649319928 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e240dacbf0 con 0x55e240e55d00 2024-04-25T00:23:04.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.551+0000 7f8f66ffd640 10 client.17178 handle_command_reply: tid=0 2024-04-25T00:23:04.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.551+0000 7f8f90be7740 1 client.17178 shutdown 2024-04-25T00:23:04.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.551+0000 7f8f90be7740 1 -- 172.21.15.107:0/3649319928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3af50 msgr2=0x55e240e42b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:04.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.551+0000 7f8f90be7740 1 --2- 172.21.15.107:0/3649319928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3af50 0x55e240e42b20 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00ba90 tx=0x7f8f7c00bf50 comp rx=0 tx=0).stop 2024-04-25T00:23:04.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.551+0000 7f8f90be7740 1 -- 172.21.15.107:0/3649319928 shutdown_connections 2024-04-25T00:23:04.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.551+0000 7f8f90be7740 1 --2- 172.21.15.107:0/3649319928 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e240e55d00 0x55e240e58180 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.551+0000 7f8f90be7740 1 --2- 172.21.15.107:0/3649319928 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e240e45620 0x55e240e43060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.551+0000 7f8f66ffd640 0 client.17178 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:04.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.551+0000 7f8f90be7740 1 --2- 172.21.15.107:0/3649319928 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3af50 0x55e240e42b20 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.551+0000 7f8f90be7740 1 --2- 172.21.15.107:0/3649319928 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e240e3a630 0x55e240e425e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.551+0000 7f8f90be7740 1 -- 172.21.15.107:0/3649319928 >> 172.21.15.107:0/3649319928 conn(0x55e240e36950 msgr2=0x55e240e3c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:04.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.551+0000 7f8f90be7740 1 -- 172.21.15.107:0/3649319928 shutdown_connections 2024-04-25T00:23:04.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.551+0000 7f8f90be7740 1 -- 172.21.15.107:0/3649319928 wait complete. 2024-04-25T00:23:04.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.551+0000 7f8f90be7740 20 client.17178 trim_cache size 0 max 16384 2024-04-25T00:23:04.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.568+0000 7f8f90be7740 1 Processor -- start 2024-04-25T00:23:04.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.568+0000 7f8f90be7740 1 -- start start 2024-04-25T00:23:04.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.568+0000 7f8f90be7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3a630 0x55e240e51270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.568+0000 7f8f90be7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e240e3af50 0x55e240e41520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.568+0000 7f8f90be7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e240e45620 0x55e240e3cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.568+0000 7f8f90be7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8f881b1400 con 0x55e240e3a630 2024-04-25T00:23:04.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.568+0000 7f8f90be7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8f881aecd0 con 0x55e240e3af50 2024-04-25T00:23:04.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.568+0000 7f8f90be7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8f881acb20 con 0x55e240e45620 2024-04-25T00:23:04.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.568+0000 7f8f87fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3a630 0x55e240e51270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.569+0000 7f8f87fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3a630 0x55e240e51270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48912/0 (socket says 172.21.15.107:48912) 2024-04-25T00:23:04.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.569+0000 7f8f87fff640 1 -- 172.21.15.107:0/313669873 learned_addr learned my addr 172.21.15.107:0/313669873 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:04.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.569+0000 7f8f87fff640 1 -- 172.21.15.107:0/313669873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e240e45620 msgr2=0x55e240e3cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:04.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.569+0000 7f8f87fff640 1 --2- 172.21.15.107:0/313669873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e240e45620 0x55e240e3cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.569+0000 7f8f87fff640 1 -- 172.21.15.107:0/313669873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e240e3af50 msgr2=0x55e240e41520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:04.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.569+0000 7f8f87fff640 1 --2- 172.21.15.107:0/313669873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e240e3af50 0x55e240e41520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.569+0000 7f8f87fff640 1 -- 172.21.15.107:0/313669873 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e240e28090 con 0x55e240e3a630 2024-04-25T00:23:04.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.569+0000 7f8f87fff640 1 --2- 172.21.15.107:0/313669873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3a630 0x55e240e51270 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f8f8002c010 tx=0x7f8f80005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e889b14cad8b2646 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:04.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.569+0000 7f8f6d7fa640 1 -- 172.21.15.107:0/313669873 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80042030 con 0x55e240e3a630 2024-04-25T00:23:04.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.569+0000 7f8f6d7fa640 1 -- 172.21.15.107:0/313669873 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f80046040 con 0x55e240e3a630 2024-04-25T00:23:04.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.569+0000 7f8f6d7fa640 1 -- 172.21.15.107:0/313669873 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f800041b0 con 0x55e240e3a630 2024-04-25T00:23:04.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.569+0000 7f8f90be7740 1 -- 172.21.15.107:0/313669873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3a630 msgr2=0x55e240e51270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:04.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.569+0000 7f8f90be7740 1 --2- 172.21.15.107:0/313669873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3a630 0x55e240e51270 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f8f8002c010 tx=0x7f8f80005dc0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.570+0000 7f8f90be7740 1 -- 172.21.15.107:0/313669873 shutdown_connections 2024-04-25T00:23:04.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.570+0000 7f8f90be7740 1 --2- 172.21.15.107:0/313669873 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e240e45620 0x55e240e3cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.570+0000 7f8f90be7740 1 --2- 172.21.15.107:0/313669873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e240e3af50 0x55e240e41520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.570+0000 7f8f90be7740 1 --2- 172.21.15.107:0/313669873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3a630 0x55e240e51270 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.570+0000 7f8f90be7740 1 -- 172.21.15.107:0/313669873 >> 172.21.15.107:0/313669873 conn(0x55e240e36950 msgr2=0x55e240e42e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:04.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.570+0000 7f8f90be7740 1 -- 172.21.15.107:0/313669873 shutdown_connections 2024-04-25T00:23:04.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.570+0000 7f8f90be7740 1 -- 172.21.15.107:0/313669873 wait complete. 2024-04-25T00:23:04.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.570+0000 7f8f90be7740 1 -- start start 2024-04-25T00:23:04.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.570+0000 7f8f90be7740 1 Processor -- start 2024-04-25T00:23:04.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.570+0000 7f8f90be7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e240e3a630 0x55e240e4d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.570+0000 7f8f90be7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3af50 0x55e240e4dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.570+0000 7f8f90be7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e240e45620 0x55e240e48c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.570+0000 7f8f90be7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8f881ad190 con 0x55e240e3af50 2024-04-25T00:23:04.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.570+0000 7f8f90be7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8f881ae600 con 0x55e240e3a630 2024-04-25T00:23:04.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.570+0000 7f8f90be7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8f881ad860 con 0x55e240e45620 2024-04-25T00:23:04.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.570+0000 7f8f90be7740 10 client.0 _ll_register_callbacks cb 0x7f8f7c0042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:04.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.570+0000 7f8f86ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3af50 0x55e240e4dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.570+0000 7f8f86ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3af50 0x55e240e4dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48920/0 (socket says 172.21.15.107:48920) 2024-04-25T00:23:04.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.570+0000 7f8f86ffd640 1 -- 172.21.15.107:0/2941974417 learned_addr learned my addr 172.21.15.107:0/2941974417 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:04.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.571+0000 7f8f86ffd640 1 -- 172.21.15.107:0/2941974417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e240e45620 msgr2=0x55e240e48c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:04.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.571+0000 7f8f86ffd640 1 --2- 172.21.15.107:0/2941974417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e240e45620 0x55e240e48c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.571+0000 7f8f87fff640 1 --2- 172.21.15.107:0/2941974417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e240e3a630 0x55e240e4d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.571+0000 7f8f86ffd640 1 -- 172.21.15.107:0/2941974417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e240e3a630 msgr2=0x55e240e4d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:04.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.571+0000 7f8f86ffd640 1 --2- 172.21.15.107:0/2941974417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e240e3a630 0x55e240e4d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.571+0000 7f8f86ffd640 1 -- 172.21.15.107:0/2941974417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e240e49450 con 0x55e240e3af50 2024-04-25T00:23:04.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.571+0000 7f8f87fff640 1 --2- 172.21.15.107:0/2941974417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e240e3a630 0x55e240e4d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:04.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.571+0000 7f8f86ffd640 1 --2- 172.21.15.107:0/2941974417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3af50 0x55e240e4dc60 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c001eb0 tx=0x7f8f7c003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae0b5fa2215e243 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:04.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.571+0000 7f8f66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:04.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.571+0000 7f8f66ffd640 1 -- 172.21.15.107:0/2941974417 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c02e030 con 0x55e240e3af50 2024-04-25T00:23:04.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.571+0000 7f8f66ffd640 1 -- 172.21.15.107:0/2941974417 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9395+0+0 (secure 0 0 0) 0x7f8f7c017070 con 0x55e240e3af50 2024-04-25T00:23:04.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.571+0000 7f8f90be7740 1 -- 172.21.15.107:0/2941974417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f80046040 con 0x55e240e3af50 2024-04-25T00:23:04.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.571+0000 7f8f66ffd640 1 -- 172.21.15.107:0/2941974417 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8f7c038050 con 0x55e240e3af50 2024-04-25T00:23:04.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.571+0000 7f8f90be7740 10 client.17184 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:04.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.571+0000 7f8f90be7740 1 -- 172.21.15.107:0/2941974417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e240ebbf10 con 0x55e240e3af50 2024-04-25T00:23:04.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.571+0000 7f8f66ffd640 1 -- 172.21.15.107:0/2941974417 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f80046040 con 0x55e240e3af50 2024-04-25T00:23:04.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.571+0000 7f8f66ffd640 1 -- 172.21.15.107:0/2941974417 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f8f7c011970 con 0x55e240e3af50 2024-04-25T00:23:04.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.571+0000 7f8f90be7740 10 client.17184 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:04.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.571+0000 7f8f90be7740 10 client.17184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:04.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.571+0000 7f8f90be7740 20 client.17184 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:04.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.571+0000 7f8f90be7740 1 --2- 172.21.15.107:0/2941974417 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e240e55d00 0x55e240ebe6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:04.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.571+0000 7f8f90be7740 4 client.17184 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:23:04.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.571+0000 7f8f90be7740 1 -- 172.21.15.107:0/2941974417 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e240da6f40 con 0x55e240e55d00 2024-04-25T00:23:04.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.572+0000 7f8f87fff640 1 --2- 172.21.15.107:0/2941974417 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e240e55d00 0x55e240ebe6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:04.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.572+0000 7f8f87fff640 1 --2- 172.21.15.107:0/2941974417 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e240e55d00 0x55e240ebe6c0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=512d5a5d2c5b0fbf server_cookie=360d67ab7bed74d1 in_seq=0 out_seq=0 2024-04-25T00:23:04.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.572+0000 7f8f66ffd640 10 client.17184 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:04.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.572+0000 7f8f66ffd640 1 -- 172.21.15.107:0/2941974417 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e240da6f40 con 0x55e240e55d00 2024-04-25T00:23:04.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.572+0000 7f8f66ffd640 10 client.17184 handle_command_reply: tid=0 2024-04-25T00:23:04.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.572+0000 7f8f90be7740 1 client.17184 shutdown 2024-04-25T00:23:04.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.573+0000 7f8f90be7740 1 -- 172.21.15.107:0/2941974417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3af50 msgr2=0x55e240e4dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:04.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.573+0000 7f8f90be7740 1 --2- 172.21.15.107:0/2941974417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3af50 0x55e240e4dc60 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c001eb0 tx=0x7f8f7c003a80 comp rx=0 tx=0).stop 2024-04-25T00:23:04.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.573+0000 7f8f90be7740 1 -- 172.21.15.107:0/2941974417 shutdown_connections 2024-04-25T00:23:04.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.573+0000 7f8f90be7740 1 --2- 172.21.15.107:0/2941974417 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e240e55d00 0x55e240ebe6c0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.573+0000 7f8f90be7740 1 --2- 172.21.15.107:0/2941974417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e240e45620 0x55e240e48c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.573+0000 7f8f90be7740 1 --2- 172.21.15.107:0/2941974417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e240e3af50 0x55e240e4dc60 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.573+0000 7f8f66ffd640 0 client.17184 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:04.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.573+0000 7f8f90be7740 1 --2- 172.21.15.107:0/2941974417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e240e3a630 0x55e240e4d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.573+0000 7f8f90be7740 1 -- 172.21.15.107:0/2941974417 >> 172.21.15.107:0/2941974417 conn(0x55e240e36950 msgr2=0x55e240e35260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:04.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.573+0000 7f8f90be7740 1 -- 172.21.15.107:0/2941974417 shutdown_connections 2024-04-25T00:23:04.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.573+0000 7f8f90be7740 1 -- 172.21.15.107:0/2941974417 wait complete. 2024-04-25T00:23:04.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.573+0000 7f8f90be7740 20 client.17184 trim_cache size 0 max 16384 2024-04-25T00:23:04.580 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:23:04.580 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:23:04.580 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-04-25T00:23:04.580 INFO:teuthology.orchestra.run.smithi107.stdout: "8e40b219-dcdf-4c56-8b5a-70dcc44e2969": { 2024-04-25T00:23:04.580 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-04-25T00:23:04.580 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "8e40b219-dcdf-4c56-8b5a-70dcc44e2969", 2024-04-25T00:23:04.580 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-04-25T00:23:04.580 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:23:04.581 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:23:04.581 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:23:04.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.573+0000 7f8f8ddf9640 1 -- 172.21.15.107:0/22293798 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8f700a2d00 msgr2=0x7f8f700a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:04.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.573+0000 7f8f8ddf9640 1 --2- 172.21.15.107:0/22293798 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8f700a2d00 0x7f8f700a51c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8f7809b300 tx=0x7f8f7806d5c0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.573+0000 7f8f8ddf9640 1 -- 172.21.15.107:0/22293798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f88069dc0 msgr2=0x7f8f88197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:04.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.573+0000 7f8f8ddf9640 1 --2- 172.21.15.107:0/22293798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f88069dc0 0x7f8f88197fa0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800c890 tx=0x7f8f7800cd50 comp rx=0 tx=0).stop 2024-04-25T00:23:04.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.573+0000 7f8f8ddf9640 1 -- 172.21.15.107:0/22293798 shutdown_connections 2024-04-25T00:23:04.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.573+0000 7f8f8ddf9640 1 --2- 172.21.15.107:0/22293798 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8f700a2d00 0x7f8f700a51c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.573+0000 7f8f8ddf9640 1 --2- 172.21.15.107:0/22293798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8f88107730 0x7f8f8819c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.573+0000 7f8f8ddf9640 1 --2- 172.21.15.107:0/22293798 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f8806a780 0x7f8f881984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.573+0000 7f8f8ddf9640 1 --2- 172.21.15.107:0/22293798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f88069dc0 0x7f8f88197fa0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:04.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.573+0000 7f8f8ddf9640 1 -- 172.21.15.107:0/22293798 >> 172.21.15.107:0/22293798 conn(0x7f8f880776a0 msgr2=0x7f8f880fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:04.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.573+0000 7f8f8ddf9640 1 -- 172.21.15.107:0/22293798 shutdown_connections 2024-04-25T00:23:04.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:04.573+0000 7f8f8ddf9640 1 -- 172.21.15.107:0/22293798 wait complete. 2024-04-25T00:23:04.585 INFO:tasks.cephfs.filesystem:scrub status for tag:8e40b219-dcdf-4c56-8b5a-70dcc44e2969 - {'path': '/', 'tag': '8e40b219-dcdf-4c56-8b5a-70dcc44e2969', 'options': 'recursive,force'} 2024-04-25T00:23:04.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:04 smithi107 ceph-mon[29153]: scrub queued for path: / 2024-04-25T00:23:04.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:04 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:04.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:04 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:23:04.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:04 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:05.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:04 smithi195 ceph-mon[32833]: scrub queued for path: / 2024-04-25T00:23:05.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:04 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:05.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:04 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:23:05.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:04 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:05.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:04 smithi170 ceph-mon[32807]: scrub queued for path: / 2024-04-25T00:23:05.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:04 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:05.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:04 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:23:05.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:04 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:06.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:05 smithi195 ceph-mon[32833]: pgmap v202: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:23:06.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:05 smithi170 ceph-mon[32807]: pgmap v202: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:23:06.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:05 smithi107 ceph-mon[29153]: pgmap v202: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:23:08.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:07 smithi195 ceph-mon[32833]: pgmap v203: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:23:08.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:07 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:23:08.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:07 smithi170 ceph-mon[32807]: pgmap v203: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:23:08.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:07 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:23:08.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:07 smithi107 ceph-mon[29153]: pgmap v203: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:23:08.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:07 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:23:09.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:08 smithi195 ceph-mon[32833]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T00:23:09.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:08 smithi170 ceph-mon[32807]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T00:23:09.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:08 smithi107 ceph-mon[29153]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T00:23:10.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:09 smithi195 ceph-mon[32833]: pgmap v205: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-25T00:23:10.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:09 smithi170 ceph-mon[32807]: pgmap v205: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-25T00:23:10.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:09 smithi107 ceph-mon[29153]: pgmap v205: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-25T00:23:12.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:11 smithi195 ceph-mon[32833]: pgmap v206: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-25T00:23:12.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:11 smithi170 ceph-mon[32807]: pgmap v206: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-25T00:23:12.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:11 smithi107 ceph-mon[29153]: pgmap v206: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 27 op/s 2024-04-25T00:23:14.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:13 smithi195 ceph-mon[32833]: pgmap v207: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:23:14.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:13 smithi170 ceph-mon[32807]: pgmap v207: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:23:14.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:13 smithi107 ceph-mon[29153]: pgmap v207: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:23:16.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:15 smithi195 ceph-mon[32833]: pgmap v208: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:23:16.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:15 smithi170 ceph-mon[32807]: pgmap v208: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:23:16.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:15 smithi107 ceph-mon[29153]: pgmap v208: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:23:17.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:17 smithi170 ceph-mon[32807]: pgmap v209: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:23:17.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:17 smithi170 ceph-mon[32807]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T00:23:17.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:17 smithi107 ceph-mon[29153]: pgmap v209: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:23:17.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:17 smithi107 ceph-mon[29153]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T00:23:18.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:17 smithi195 ceph-mon[32833]: pgmap v209: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:23:18.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:17 smithi195 ceph-mon[32833]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T00:23:19.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:19 smithi170 ceph-mon[32807]: pgmap v211: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:23:19.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:19 smithi107 ceph-mon[29153]: pgmap v211: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:23:20.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:19 smithi195 ceph-mon[32833]: pgmap v211: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:23:21.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:21 smithi170 ceph-mon[32807]: pgmap v212: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:23:21.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:21 smithi107 ceph-mon[29153]: pgmap v212: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:23:22.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:21 smithi195 ceph-mon[32833]: pgmap v212: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:23:23.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:23 smithi170 ceph-mon[32807]: pgmap v213: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:23:23.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:23 smithi107 ceph-mon[29153]: pgmap v213: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:23:24.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:23 smithi195 ceph-mon[32833]: pgmap v213: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:23:25.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:25 smithi170 ceph-mon[32807]: pgmap v214: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:23:25.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:25 smithi107 ceph-mon[29153]: pgmap v214: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:23:26.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:25 smithi195 ceph-mon[32833]: pgmap v214: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:23:27.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:27 smithi170 ceph-mon[32807]: pgmap v215: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:23:27.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:27 smithi107 ceph-mon[29153]: pgmap v215: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:23:28.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:27 smithi195 ceph-mon[32833]: pgmap v215: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:23:29.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:29 smithi170 ceph-mon[32807]: pgmap v216: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2024-04-25T00:23:29.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:29 smithi107 ceph-mon[29153]: pgmap v216: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2024-04-25T00:23:30.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:29 smithi195 ceph-mon[32833]: pgmap v216: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2024-04-25T00:23:31.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:31 smithi170 ceph-mon[32807]: pgmap v217: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:23:31.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:31 smithi107 ceph-mon[29153]: pgmap v217: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:23:32.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:31 smithi195 ceph-mon[32833]: pgmap v217: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:23:33.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:33 smithi170 ceph-mon[32807]: pgmap v218: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:23:33.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:33 smithi107 ceph-mon[29153]: pgmap v218: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:23:34.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:33 smithi195 ceph-mon[32833]: pgmap v218: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:23:34.586 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:23:34.587 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:23:34.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.699+0000 7f6f403dc640 1 -- 172.21.15.107:0/3470805059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f38100c90 msgr2=0x7f6f381063c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:34.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.699+0000 7f6f403dc640 1 --2- 172.21.15.107:0/3470805059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f38100c90 0x7f6f381063c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f6f28009900 tx=0x7f6f2802a410 comp rx=0 tx=0).stop 2024-04-25T00:23:34.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.699+0000 7f6f403dc640 1 -- 172.21.15.107:0/3470805059 shutdown_connections 2024-04-25T00:23:34.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.699+0000 7f6f403dc640 1 --2- 172.21.15.107:0/3470805059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f38106aa0 0x7f6f3810e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.699+0000 7f6f403dc640 1 --2- 172.21.15.107:0/3470805059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f38100c90 0x7f6f381063c0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.699+0000 7f6f403dc640 1 --2- 172.21.15.107:0/3470805059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f381002e0 0x7f6f381006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.699+0000 7f6f403dc640 1 -- 172.21.15.107:0/3470805059 >> 172.21.15.107:0/3470805059 conn(0x7f6f380758d0 msgr2=0x7f6f38075ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:34.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.699+0000 7f6f403dc640 1 -- 172.21.15.107:0/3470805059 shutdown_connections 2024-04-25T00:23:34.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.699+0000 7f6f403dc640 1 -- 172.21.15.107:0/3470805059 wait complete. 2024-04-25T00:23:34.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.700+0000 7f6f403dc640 1 Processor -- start 2024-04-25T00:23:34.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.700+0000 7f6f403dc640 1 -- start start 2024-04-25T00:23:34.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.700+0000 7f6f403dc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f381002e0 0x7f6f381980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:34.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.700+0000 7f6f403dc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f38106aa0 0x7f6f381985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:34.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.700+0000 7f6f403dc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f3819c970 0x7f6f3819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:34.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.700+0000 7f6f403dc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6f38111b90 con 0x7f6f3819c970 2024-04-25T00:23:34.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.700+0000 7f6f403dc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6f38111a10 con 0x7f6f381002e0 2024-04-25T00:23:34.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.700+0000 7f6f403dc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6f38111d10 con 0x7f6f38106aa0 2024-04-25T00:23:34.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.700+0000 7f6f3e979640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f3819c970 0x7f6f3819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:34.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.700+0000 7f6f3e979640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f3819c970 0x7f6f3819ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57308/0 (socket says 172.21.15.107:57308) 2024-04-25T00:23:34.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.700+0000 7f6f3e979640 1 -- 172.21.15.107:0/1320767610 learned_addr learned my addr 172.21.15.107:0/1320767610 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:34.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.700+0000 7f6f3e178640 1 --2- 172.21.15.107:0/1320767610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f381002e0 0x7f6f381980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:34.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.701+0000 7f6f3d977640 1 --2- 172.21.15.107:0/1320767610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f38106aa0 0x7f6f381985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:34.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.701+0000 7f6f3e979640 1 -- 172.21.15.107:0/1320767610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f38106aa0 msgr2=0x7f6f381985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:34.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.701+0000 7f6f3e979640 1 --2- 172.21.15.107:0/1320767610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f38106aa0 0x7f6f381985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.701+0000 7f6f3e979640 1 -- 172.21.15.107:0/1320767610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f381002e0 msgr2=0x7f6f381980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:34.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.701+0000 7f6f3e979640 1 --2- 172.21.15.107:0/1320767610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f381002e0 0x7f6f381980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.701+0000 7f6f3e979640 1 -- 172.21.15.107:0/1320767610 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3819d460 con 0x7f6f3819c970 2024-04-25T00:23:34.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.701+0000 7f6f3d977640 1 --2- 172.21.15.107:0/1320767610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f38106aa0 0x7f6f381985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:34.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.701+0000 7f6f3e979640 1 --2- 172.21.15.107:0/1320767610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f3819c970 0x7f6f3819ce20 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000cad0 tx=0x7f6f3000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:34.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.701+0000 7f6f2f7fe640 1 -- 172.21.15.107:0/1320767610 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3000ae20 con 0x7f6f3819c970 2024-04-25T00:23:34.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.701+0000 7f6f403dc640 1 -- 172.21.15.107:0/1320767610 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f3819d750 con 0x7f6f3819c970 2024-04-25T00:23:34.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.701+0000 7f6f2f7fe640 1 -- 172.21.15.107:0/1320767610 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f30005020 con 0x7f6f3819c970 2024-04-25T00:23:34.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.701+0000 7f6f2f7fe640 1 -- 172.21.15.107:0/1320767610 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3000f6f0 con 0x7f6f3819c970 2024-04-25T00:23:34.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.701+0000 7f6f403dc640 1 -- 172.21.15.107:0/1320767610 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f381a4ef0 con 0x7f6f3819c970 2024-04-25T00:23:34.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.704+0000 7f6f2f7fe640 1 -- 172.21.15.107:0/1320767610 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f6f300051d0 con 0x7f6f3819c970 2024-04-25T00:23:34.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.704+0000 7f6f431c8740 1 Processor -- start 2024-04-25T00:23:34.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.705+0000 7f6f431c8740 1 -- start start 2024-04-25T00:23:34.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.705+0000 7f6f431c8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8e9c530 0x55e9b8e9c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:34.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.705+0000 7f6f431c8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b8e9ce50 0x55e9b8eab4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:34.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.705+0000 7f6f431c8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9b8eaba30 0x55e9b8eade20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:34.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.705+0000 7f6f431c8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e9b8e016b0 con 0x55e9b8e9c530 2024-04-25T00:23:34.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.705+0000 7f6f431c8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e9b8da9880 con 0x55e9b8eaba30 2024-04-25T00:23:34.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.705+0000 7f6f431c8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e9b8dfd670 con 0x55e9b8e9ce50 2024-04-25T00:23:34.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.705+0000 7f6f3d977640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8e9c530 0x55e9b8e9c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:34.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.705+0000 7f6f3d977640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8e9c530 0x55e9b8e9c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57320/0 (socket says 172.21.15.107:57320) 2024-04-25T00:23:34.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.705+0000 7f6f3d977640 1 -- 172.21.15.107:0/1788474255 learned_addr learned my addr 172.21.15.107:0/1788474255 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:34.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.705+0000 7f6f2f7fe640 1 --2- 172.21.15.107:0/1320767610 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f6f240a2d00 0x7f6f240a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:34.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.705+0000 7f6f3e178640 1 --2- 172.21.15.107:0/1788474255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b8e9ce50 0x55e9b8eab4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:34.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.705+0000 7f6f3d977640 1 --2- 172.21.15.107:0/1788474255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9b8eaba30 0x55e9b8eade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:34.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.705+0000 7f6f3e979640 1 --2- 172.21.15.107:0/1320767610 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f6f240a2d00 0x7f6f240a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:34.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.705+0000 7f6f3d977640 1 -- 172.21.15.107:0/1788474255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b8e9ce50 msgr2=0x55e9b8eab4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:34.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.705+0000 7f6f3d977640 1 --2- 172.21.15.107:0/1788474255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b8e9ce50 0x55e9b8eab4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.705+0000 7f6f3d977640 1 -- 172.21.15.107:0/1788474255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9b8eaba30 msgr2=0x55e9b8eade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:34.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.705+0000 7f6f3d977640 1 --2- 172.21.15.107:0/1788474255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9b8eaba30 0x55e9b8eade20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.705+0000 7f6f3d977640 1 -- 172.21.15.107:0/1788474255 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9b8bdf7d0 con 0x55e9b8e9c530 2024-04-25T00:23:34.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.705+0000 7f6f2f7fe640 1 -- 172.21.15.107:0/1320767610 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f6f300cf840 con 0x7f6f3819c970 2024-04-25T00:23:34.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.706+0000 7f6f3d977640 1 --2- 172.21.15.107:0/1788474255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8e9c530 0x55e9b8e9c910 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f6f28031550 tx=0x7f6f28000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73e5fc8230eb6bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:34.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.706+0000 7f6f3e979640 1 --2- 172.21.15.107:0/1320767610 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f6f240a2d00 0x7f6f240a51c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6f300ccc80 tx=0x7f6f30005960 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:34.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.706+0000 7f6f2cbfe640 1 -- 172.21.15.107:0/1788474255 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28035070 con 0x55e9b8e9c530 2024-04-25T00:23:34.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.706+0000 7f6f2cbfe640 1 -- 172.21.15.107:0/1788474255 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f280319f0 con 0x55e9b8e9c530 2024-04-25T00:23:34.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.706+0000 7f6f2cbfe640 1 -- 172.21.15.107:0/1788474255 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28004920 con 0x55e9b8e9c530 2024-04-25T00:23:34.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.706+0000 7f6f431c8740 1 -- 172.21.15.107:0/1788474255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8e9c530 msgr2=0x55e9b8e9c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:34.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.706+0000 7f6f431c8740 1 --2- 172.21.15.107:0/1788474255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8e9c530 0x55e9b8e9c910 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f6f28031550 tx=0x7f6f28000bc0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.707+0000 7f6f431c8740 1 -- 172.21.15.107:0/1788474255 shutdown_connections 2024-04-25T00:23:34.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.707+0000 7f6f431c8740 1 --2- 172.21.15.107:0/1788474255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9b8eaba30 0x55e9b8eade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.707+0000 7f6f431c8740 1 --2- 172.21.15.107:0/1788474255 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b8e9ce50 0x55e9b8eab4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.707+0000 7f6f431c8740 1 --2- 172.21.15.107:0/1788474255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8e9c530 0x55e9b8e9c910 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.707+0000 7f6f431c8740 1 -- 172.21.15.107:0/1788474255 >> 172.21.15.107:0/1788474255 conn(0x55e9b8e98950 msgr2=0x55e9b8e98d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:34.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.707+0000 7f6f431c8740 1 -- 172.21.15.107:0/1788474255 shutdown_connections 2024-04-25T00:23:34.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.707+0000 7f6f431c8740 1 -- 172.21.15.107:0/1788474255 wait complete. 2024-04-25T00:23:34.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.707+0000 7f6f431c8740 1 -- start start 2024-04-25T00:23:34.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.707+0000 7f6f431c8740 1 Processor -- start 2024-04-25T00:23:34.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.707+0000 7f6f431c8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8e9c530 0x55e9b8ea8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:34.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.707+0000 7f6f431c8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b8e9ce50 0x55e9b8ea8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:34.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.707+0000 7f6f431c8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9b8eaba30 0x55e9b8ea93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:34.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.707+0000 7f6f431c8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e9b8eae790 con 0x55e9b8e9c530 2024-04-25T00:23:34.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.707+0000 7f6f431c8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e9b8dbcf50 con 0x55e9b8eaba30 2024-04-25T00:23:34.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.707+0000 7f6f431c8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e9b8eae910 con 0x55e9b8e9ce50 2024-04-25T00:23:34.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.707+0000 7f6f431c8740 10 client.0 _ll_register_callbacks cb 0x55e9b8df8000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:34.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.707+0000 7f6f3d977640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8e9c530 0x55e9b8ea8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:34.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.707+0000 7f6f3d977640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8e9c530 0x55e9b8ea8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57322/0 (socket says 172.21.15.107:57322) 2024-04-25T00:23:34.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.707+0000 7f6f3d977640 1 -- 172.21.15.107:0/1766378427 learned_addr learned my addr 172.21.15.107:0/1766378427 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:34.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.708+0000 7f6f3d977640 1 -- 172.21.15.107:0/1766378427 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b8e9ce50 msgr2=0x55e9b8ea8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:34.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.708+0000 7f6f3d977640 1 --2- 172.21.15.107:0/1766378427 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b8e9ce50 0x55e9b8ea8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.708+0000 7f6f3d977640 1 -- 172.21.15.107:0/1766378427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9b8eaba30 msgr2=0x55e9b8ea93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:34.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.708+0000 7f6f3d977640 1 --2- 172.21.15.107:0/1766378427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9b8eaba30 0x55e9b8ea93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.708+0000 7f6f3d977640 1 -- 172.21.15.107:0/1766378427 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9b8b153f0 con 0x55e9b8e9c530 2024-04-25T00:23:34.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.708+0000 7f6f3d977640 1 --2- 172.21.15.107:0/1766378427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8e9c530 0x55e9b8ea8950 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f6f28032fd0 tx=0x7f6f28009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e42035326c921bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:34.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.708+0000 7f6f21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:34.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.708+0000 7f6f21ffb640 1 -- 172.21.15.107:0/1766378427 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28035040 con 0x55e9b8e9c530 2024-04-25T00:23:34.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.708+0000 7f6f21ffb640 1 -- 172.21.15.107:0/1766378427 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f6f280048d0 con 0x55e9b8e9c530 2024-04-25T00:23:34.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.708+0000 7f6f431c8740 1 -- 172.21.15.107:0/1766378427 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f280319f0 con 0x55e9b8e9c530 2024-04-25T00:23:34.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.708+0000 7f6f21ffb640 1 -- 172.21.15.107:0/1766378427 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6f28057050 con 0x55e9b8e9c530 2024-04-25T00:23:34.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.708+0000 7f6f431c8740 10 client.17196 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.708+0000 7f6f431c8740 1 -- 172.21.15.107:0/1766378427 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e9b8ad50f0 con 0x55e9b8e9c530 2024-04-25T00:23:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.708+0000 7f6f21ffb640 1 -- 172.21.15.107:0/1766378427 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f280319f0 con 0x55e9b8e9c530 2024-04-25T00:23:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.708+0000 7f6f21ffb640 1 -- 172.21.15.107:0/1766378427 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f6f28052030 con 0x55e9b8e9c530 2024-04-25T00:23:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.708+0000 7f6f431c8740 10 client.17196 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.709+0000 7f6f431c8740 10 client.17196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.709+0000 7f6f431c8740 20 client.17196 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.709+0000 7f6f431c8740 1 --2- 172.21.15.107:0/1766378427 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e9b8ebbb10 0x55e9b8ebdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.709+0000 7f6f431c8740 4 client.17196 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.709+0000 7f6f431c8740 1 -- 172.21.15.107:0/1766378427 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9b8e0ebf0 con 0x55e9b8ebbb10 2024-04-25T00:23:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.709+0000 7f6f3e178640 1 --2- 172.21.15.107:0/1766378427 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e9b8ebbb10 0x55e9b8ebdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.709+0000 7f6f3e178640 1 --2- 172.21.15.107:0/1766378427 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e9b8ebbb10 0x55e9b8ebdf00 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba8bf3858a1481a1 server_cookie=882be51dd2f0aa3f in_seq=0 out_seq=0 2024-04-25T00:23:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.709+0000 7f6f21ffb640 10 client.17196 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.710+0000 7f6f21ffb640 1 -- 172.21.15.107:0/1766378427 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e9b8e0ebf0 con 0x55e9b8ebbb10 2024-04-25T00:23:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.710+0000 7f6f21ffb640 10 client.17196 handle_command_reply: tid=0 2024-04-25T00:23:34.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.710+0000 7f6f431c8740 1 client.17196 shutdown 2024-04-25T00:23:34.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.710+0000 7f6f431c8740 1 -- 172.21.15.107:0/1766378427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8e9c530 msgr2=0x55e9b8ea8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:34.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.710+0000 7f6f431c8740 1 --2- 172.21.15.107:0/1766378427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8e9c530 0x55e9b8ea8950 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f6f28032fd0 tx=0x7f6f28009290 comp rx=0 tx=0).stop 2024-04-25T00:23:34.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.710+0000 7f6f431c8740 1 -- 172.21.15.107:0/1766378427 shutdown_connections 2024-04-25T00:23:34.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.710+0000 7f6f431c8740 1 --2- 172.21.15.107:0/1766378427 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e9b8ebbb10 0x55e9b8ebdf00 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.710+0000 7f6f431c8740 1 --2- 172.21.15.107:0/1766378427 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9b8eaba30 0x55e9b8ea93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.710+0000 7f6f431c8740 1 --2- 172.21.15.107:0/1766378427 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b8e9ce50 0x55e9b8ea8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.710+0000 7f6f431c8740 1 --2- 172.21.15.107:0/1766378427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8e9c530 0x55e9b8ea8950 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.710+0000 7f6f21ffb640 0 client.17196 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:34.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.710+0000 7f6f431c8740 1 -- 172.21.15.107:0/1766378427 >> 172.21.15.107:0/1766378427 conn(0x55e9b8e98950 msgr2=0x55e9b8e9e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:34.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.710+0000 7f6f431c8740 1 -- 172.21.15.107:0/1766378427 shutdown_connections 2024-04-25T00:23:34.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.710+0000 7f6f431c8740 1 -- 172.21.15.107:0/1766378427 wait complete. 2024-04-25T00:23:34.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.710+0000 7f6f431c8740 20 client.17196 trim_cache size 0 max 16384 2024-04-25T00:23:34.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.728+0000 7f6f431c8740 1 Processor -- start 2024-04-25T00:23:34.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.728+0000 7f6f431c8740 1 -- start start 2024-04-25T00:23:34.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.728+0000 7f6f431c8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8e9c530 0x55e9b8eab080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:34.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.728+0000 7f6f431c8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b8e9ce50 0x55e9b8e98d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:34.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.728+0000 7f6f431c8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9b8eaba30 0x55e9b8e9e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:34.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.728+0000 7f6f431c8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6f381af250 con 0x55e9b8e9c530 2024-04-25T00:23:34.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.728+0000 7f6f431c8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6f381acb20 con 0x55e9b8eaba30 2024-04-25T00:23:34.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.728+0000 7f6f431c8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6f381aa990 con 0x55e9b8e9ce50 2024-04-25T00:23:34.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.728+0000 7f6f3d977640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8e9c530 0x55e9b8eab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:34.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.728+0000 7f6f3d977640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8e9c530 0x55e9b8eab080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57324/0 (socket says 172.21.15.107:57324) 2024-04-25T00:23:34.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.728+0000 7f6f3d977640 1 -- 172.21.15.107:0/3594166865 learned_addr learned my addr 172.21.15.107:0/3594166865 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:34.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.728+0000 7f6f3e178640 1 --2- 172.21.15.107:0/3594166865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b8e9ce50 0x55e9b8e98d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:34.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.728+0000 7f6f3d977640 1 --2- 172.21.15.107:0/3594166865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9b8eaba30 0x55e9b8e9e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:34.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.728+0000 7f6f3d977640 1 -- 172.21.15.107:0/3594166865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b8e9ce50 msgr2=0x55e9b8e98d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:34.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.728+0000 7f6f3d977640 1 --2- 172.21.15.107:0/3594166865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b8e9ce50 0x55e9b8e98d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.728+0000 7f6f3d977640 1 -- 172.21.15.107:0/3594166865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9b8eaba30 msgr2=0x55e9b8e9e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:34.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.728+0000 7f6f3d977640 1 --2- 172.21.15.107:0/3594166865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9b8eaba30 0x55e9b8e9e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.728+0000 7f6f3d977640 1 -- 172.21.15.107:0/3594166865 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9b8e9cc80 con 0x55e9b8e9c530 2024-04-25T00:23:34.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.728+0000 7f6f3d977640 1 --2- 172.21.15.107:0/3594166865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8e9c530 0x55e9b8eab080 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f6f28006010 tx=0x7f6f28000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9286144cf758a20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:34.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.728+0000 7f6f23fff640 1 -- 172.21.15.107:0/3594166865 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28035070 con 0x55e9b8e9c530 2024-04-25T00:23:34.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.729+0000 7f6f23fff640 1 -- 172.21.15.107:0/3594166865 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f280316e0 con 0x55e9b8e9c530 2024-04-25T00:23:34.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.729+0000 7f6f23fff640 1 -- 172.21.15.107:0/3594166865 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28004b30 con 0x55e9b8e9c530 2024-04-25T00:23:34.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.729+0000 7f6f431c8740 1 -- 172.21.15.107:0/3594166865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8e9c530 msgr2=0x55e9b8eab080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:34.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.729+0000 7f6f431c8740 1 --2- 172.21.15.107:0/3594166865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8e9c530 0x55e9b8eab080 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f6f28006010 tx=0x7f6f28000bc0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.729+0000 7f6f431c8740 1 -- 172.21.15.107:0/3594166865 shutdown_connections 2024-04-25T00:23:34.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.729+0000 7f6f431c8740 1 --2- 172.21.15.107:0/3594166865 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9b8eaba30 0x55e9b8e9e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.729+0000 7f6f431c8740 1 --2- 172.21.15.107:0/3594166865 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b8e9ce50 0x55e9b8e98d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.729+0000 7f6f431c8740 1 --2- 172.21.15.107:0/3594166865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8e9c530 0x55e9b8eab080 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.729+0000 7f6f431c8740 1 -- 172.21.15.107:0/3594166865 >> 172.21.15.107:0/3594166865 conn(0x55e9b8e98950 msgr2=0x55e9b8eb6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:34.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.729+0000 7f6f431c8740 1 -- 172.21.15.107:0/3594166865 shutdown_connections 2024-04-25T00:23:34.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.729+0000 7f6f431c8740 1 -- 172.21.15.107:0/3594166865 wait complete. 2024-04-25T00:23:34.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.729+0000 7f6f431c8740 1 -- start start 2024-04-25T00:23:34.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.729+0000 7f6f431c8740 1 Processor -- start 2024-04-25T00:23:34.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.730+0000 7f6f431c8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b8e9c530 0x55e9b8f1a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:34.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.730+0000 7f6f431c8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9b8e9ce50 0x55e9b8f1a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:34.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.730+0000 7f6f431c8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8eaba30 0x55e9b8ebe190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:34.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.730+0000 7f6f431c8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6f381aafe0 con 0x55e9b8eaba30 2024-04-25T00:23:34.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.730+0000 7f6f431c8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6f381ac450 con 0x55e9b8e9ce50 2024-04-25T00:23:34.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.730+0000 7f6f431c8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6f381ab6b0 con 0x55e9b8e9c530 2024-04-25T00:23:34.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.730+0000 7f6f431c8740 10 client.0 _ll_register_callbacks cb 0x7f6f28041070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:34.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.730+0000 7f6f3d977640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8eaba30 0x55e9b8ebe190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:34.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.730+0000 7f6f3d977640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8eaba30 0x55e9b8ebe190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57334/0 (socket says 172.21.15.107:57334) 2024-04-25T00:23:34.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.730+0000 7f6f3d977640 1 -- 172.21.15.107:0/61452839 learned_addr learned my addr 172.21.15.107:0/61452839 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:34.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.730+0000 7f6f3e178640 1 --2- 172.21.15.107:0/61452839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9b8e9ce50 0x55e9b8f1a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:34.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.730+0000 7f6f3d977640 1 --2- 172.21.15.107:0/61452839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b8e9c530 0x55e9b8f1a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:34.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.730+0000 7f6f3d977640 1 -- 172.21.15.107:0/61452839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b8e9c530 msgr2=0x55e9b8f1a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:34.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.730+0000 7f6f3d977640 1 --2- 172.21.15.107:0/61452839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b8e9c530 0x55e9b8f1a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.730+0000 7f6f3d977640 1 -- 172.21.15.107:0/61452839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9b8e9ce50 msgr2=0x55e9b8f1a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:34.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.730+0000 7f6f3d977640 1 --2- 172.21.15.107:0/61452839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9b8e9ce50 0x55e9b8f1a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.730+0000 7f6f3d977640 1 -- 172.21.15.107:0/61452839 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9b8ebe880 con 0x55e9b8eaba30 2024-04-25T00:23:34.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.730+0000 7f6f3d977640 1 --2- 172.21.15.107:0/61452839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8eaba30 0x55e9b8ebe190 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x55e9b8ec19d0 tx=0x7f6f280058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482933697139299d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:34.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.730+0000 7f6f21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:34.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.730+0000 7f6f21ffb640 1 -- 172.21.15.107:0/61452839 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f28035070 con 0x55e9b8eaba30 2024-04-25T00:23:34.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.730+0000 7f6f21ffb640 1 -- 172.21.15.107:0/61452839 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f6f28004ae0 con 0x55e9b8eaba30 2024-04-25T00:23:34.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.730+0000 7f6f431c8740 1 -- 172.21.15.107:0/61452839 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f280316e0 con 0x55e9b8eaba30 2024-04-25T00:23:34.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.731+0000 7f6f21ffb640 1 -- 172.21.15.107:0/61452839 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6f280319f0 con 0x55e9b8eaba30 2024-04-25T00:23:34.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.731+0000 7f6f431c8740 10 client.17202 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:34.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.731+0000 7f6f431c8740 1 -- 172.21.15.107:0/61452839 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e9b8f21810 con 0x55e9b8eaba30 2024-04-25T00:23:34.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.731+0000 7f6f21ffb640 1 -- 172.21.15.107:0/61452839 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f280316e0 con 0x55e9b8eaba30 2024-04-25T00:23:34.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.731+0000 7f6f21ffb640 1 -- 172.21.15.107:0/61452839 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f6f28052040 con 0x55e9b8eaba30 2024-04-25T00:23:34.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.731+0000 7f6f431c8740 10 client.17202 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:34.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.731+0000 7f6f431c8740 10 client.17202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:34.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.731+0000 7f6f431c8740 20 client.17202 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:34.739 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.731+0000 7f6f431c8740 1 --2- 172.21.15.107:0/61452839 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e9b8f21ba0 0x55e9b8f23f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:34.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.731+0000 7f6f431c8740 4 client.17202 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:23:34.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.731+0000 7f6f431c8740 1 -- 172.21.15.107:0/61452839 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e9b8e08f40 con 0x55e9b8f21ba0 2024-04-25T00:23:34.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.731+0000 7f6f3e178640 1 --2- 172.21.15.107:0/61452839 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e9b8f21ba0 0x55e9b8f23f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:34.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.732+0000 7f6f21ffb640 10 client.17202 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:34.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.732+0000 7f6f3e178640 1 --2- 172.21.15.107:0/61452839 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e9b8f21ba0 0x55e9b8f23f90 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52caf518ca17ffc0 server_cookie=7ca2b7724aea66b0 in_seq=0 out_seq=0 2024-04-25T00:23:34.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.732+0000 7f6f21ffb640 1 -- 172.21.15.107:0/61452839 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e9b8e08f40 con 0x55e9b8f21ba0 2024-04-25T00:23:34.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.732+0000 7f6f21ffb640 10 client.17202 handle_command_reply: tid=0 2024-04-25T00:23:34.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.732+0000 7f6f431c8740 1 client.17202 shutdown 2024-04-25T00:23:34.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.732+0000 7f6f431c8740 1 -- 172.21.15.107:0/61452839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8eaba30 msgr2=0x55e9b8ebe190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:34.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.732+0000 7f6f431c8740 1 --2- 172.21.15.107:0/61452839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8eaba30 0x55e9b8ebe190 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x55e9b8ec19d0 tx=0x7f6f280058b0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.732+0000 7f6f431c8740 1 -- 172.21.15.107:0/61452839 shutdown_connections 2024-04-25T00:23:34.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.732+0000 7f6f431c8740 1 --2- 172.21.15.107:0/61452839 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55e9b8f21ba0 0x55e9b8f23f90 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.732+0000 7f6f431c8740 1 --2- 172.21.15.107:0/61452839 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9b8eaba30 0x55e9b8ebe190 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.732+0000 7f6f431c8740 1 --2- 172.21.15.107:0/61452839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e9b8e9ce50 0x55e9b8f1a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.732+0000 7f6f21ffb640 0 client.17202 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:34.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.732+0000 7f6f431c8740 1 --2- 172.21.15.107:0/61452839 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e9b8e9c530 0x55e9b8f1a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.732+0000 7f6f431c8740 1 -- 172.21.15.107:0/61452839 >> 172.21.15.107:0/61452839 conn(0x55e9b8e98950 msgr2=0x55e9b8eb7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:34.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.732+0000 7f6f431c8740 1 -- 172.21.15.107:0/61452839 shutdown_connections 2024-04-25T00:23:34.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.732+0000 7f6f431c8740 1 -- 172.21.15.107:0/61452839 wait complete. 2024-04-25T00:23:34.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.732+0000 7f6f431c8740 20 client.17202 trim_cache size 0 max 16384 2024-04-25T00:23:34.741 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:23:34.741 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-04-25T00:23:34.741 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-04-25T00:23:34.741 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:23:34.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.732+0000 7f6f403dc640 1 -- 172.21.15.107:0/1320767610 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f6f240a2d00 msgr2=0x7f6f240a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:34.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.732+0000 7f6f403dc640 1 --2- 172.21.15.107:0/1320767610 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f6f240a2d00 0x7f6f240a51c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6f300ccc80 tx=0x7f6f30005960 comp rx=0 tx=0).stop 2024-04-25T00:23:34.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.732+0000 7f6f403dc640 1 -- 172.21.15.107:0/1320767610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f3819c970 msgr2=0x7f6f3819ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:34.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.732+0000 7f6f403dc640 1 --2- 172.21.15.107:0/1320767610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f3819c970 0x7f6f3819ce20 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000cad0 tx=0x7f6f3000cf90 comp rx=0 tx=0).stop 2024-04-25T00:23:34.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.733+0000 7f6f403dc640 1 -- 172.21.15.107:0/1320767610 shutdown_connections 2024-04-25T00:23:34.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.733+0000 7f6f403dc640 1 --2- 172.21.15.107:0/1320767610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6f3819c970 0x7f6f3819ce20 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.733+0000 7f6f403dc640 1 --2- 172.21.15.107:0/1320767610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f38106aa0 0x7f6f381985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.733+0000 7f6f403dc640 1 --2- 172.21.15.107:0/1320767610 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f6f240a2d00 0x7f6f240a51c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.733+0000 7f6f403dc640 1 --2- 172.21.15.107:0/1320767610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6f381002e0 0x7f6f381980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.733+0000 7f6f403dc640 1 -- 172.21.15.107:0/1320767610 >> 172.21.15.107:0/1320767610 conn(0x7f6f380758d0 msgr2=0x7f6f380fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:34.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.733+0000 7f6f403dc640 1 -- 172.21.15.107:0/1320767610 shutdown_connections 2024-04-25T00:23:34.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.733+0000 7f6f403dc640 1 -- 172.21.15.107:0/1320767610 wait complete. 2024-04-25T00:23:34.744 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:23:34.745 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:23:34.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.842+0000 7f6970b09640 1 -- 172.21.15.107:0/3796158477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f696c106630 msgr2=0x7f696c106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:34.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.842+0000 7f6970b09640 1 --2- 172.21.15.107:0/3796158477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f696c106630 0x7f696c106a10 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f6964009cd0 tx=0x7f696402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.842+0000 7f6970b09640 1 -- 172.21.15.107:0/3796158477 shutdown_connections 2024-04-25T00:23:34.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.842+0000 7f6970b09640 1 --2- 172.21.15.107:0/3796158477 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f696c100fe0 0x7f696c10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.842+0000 7f6970b09640 1 --2- 172.21.15.107:0/3796158477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f696c100620 0x7f696c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.842+0000 7f6970b09640 1 --2- 172.21.15.107:0/3796158477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f696c106630 0x7f696c106a10 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.842+0000 7f6970b09640 1 -- 172.21.15.107:0/3796158477 >> 172.21.15.107:0/3796158477 conn(0x7f696c077740 msgr2=0x7f696c077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:34.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.842+0000 7f6970b09640 1 -- 172.21.15.107:0/3796158477 shutdown_connections 2024-04-25T00:23:34.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.842+0000 7f6970b09640 1 -- 172.21.15.107:0/3796158477 wait complete. 2024-04-25T00:23:34.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.842+0000 7f6970b09640 1 Processor -- start 2024-04-25T00:23:34.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.842+0000 7f6970b09640 1 -- start start 2024-04-25T00:23:34.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.843+0000 7f6970b09640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f696c100620 0x7f696c196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:34.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.843+0000 7f6970b09640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f696c100fe0 0x7f696c1974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:34.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.843+0000 7f6970b09640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f696c106630 0x7f696c19b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:34.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.843+0000 7f6970b09640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f696c110a40 con 0x7f696c100fe0 2024-04-25T00:23:34.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.843+0000 7f6970b09640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f696c1108c0 con 0x7f696c106630 2024-04-25T00:23:34.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.843+0000 7f6970b09640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f696c110bc0 con 0x7f696c100620 2024-04-25T00:23:34.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.843+0000 7f6969d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f696c100fe0 0x7f696c1974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:34.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.843+0000 7f6969d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f696c100fe0 0x7f696c1974c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57354/0 (socket says 172.21.15.107:57354) 2024-04-25T00:23:34.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.843+0000 7f6969d9b640 1 -- 172.21.15.107:0/1252755428 learned_addr learned my addr 172.21.15.107:0/1252755428 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:34.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.843+0000 7f696a59c640 1 --2- 172.21.15.107:0/1252755428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f696c100620 0x7f696c196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:34.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.843+0000 7f696ad9d640 1 --2- 172.21.15.107:0/1252755428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f696c106630 0x7f696c19b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:34.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.843+0000 7f6969d9b640 1 -- 172.21.15.107:0/1252755428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f696c100620 msgr2=0x7f696c196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:34.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.843+0000 7f6969d9b640 1 --2- 172.21.15.107:0/1252755428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f696c100620 0x7f696c196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.843+0000 7f6969d9b640 1 -- 172.21.15.107:0/1252755428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f696c106630 msgr2=0x7f696c19b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:34.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.843+0000 7f6969d9b640 1 --2- 172.21.15.107:0/1252755428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f696c106630 0x7f696c19b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:34.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.843+0000 7f6969d9b640 1 -- 172.21.15.107:0/1252755428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696c19bf20 con 0x7f696c100fe0 2024-04-25T00:23:34.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.843+0000 7f696ad9d640 1 --2- 172.21.15.107:0/1252755428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f696c106630 0x7f696c19b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:34.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.843+0000 7f6969d9b640 1 --2- 172.21.15.107:0/1252755428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f696c100fe0 0x7f696c1974c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f696000b4a0 tx=0x7f696000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:34.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.843+0000 7f695b7fe640 1 -- 172.21.15.107:0/1252755428 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6960014070 con 0x7f696c100fe0 2024-04-25T00:23:34.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.843+0000 7f695b7fe640 1 -- 172.21.15.107:0/1252755428 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6960010af0 con 0x7f696c100fe0 2024-04-25T00:23:34.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.843+0000 7f6970b09640 1 -- 172.21.15.107:0/1252755428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f696c19c210 con 0x7f696c100fe0 2024-04-25T00:23:34.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.843+0000 7f695b7fe640 1 -- 172.21.15.107:0/1252755428 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6960017d10 con 0x7f696c100fe0 2024-04-25T00:23:34.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.843+0000 7f6970b09640 1 -- 172.21.15.107:0/1252755428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f696c1a3aa0 con 0x7f696c100fe0 2024-04-25T00:23:34.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.844+0000 7f6970b09640 1 -- 172.21.15.107:0/1252755428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f696c068560 con 0x7f696c100fe0 2024-04-25T00:23:34.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.845+0000 7f695b7fe640 1 -- 172.21.15.107:0/1252755428 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f6960017070 con 0x7f696c100fe0 2024-04-25T00:23:34.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.846+0000 7f695b7fe640 1 --2- 172.21.15.107:0/1252755428 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f69540a2d00 0x7f69540a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:34.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.846+0000 7f696a59c640 1 --2- 172.21.15.107:0/1252755428 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f69540a2d00 0x7f69540a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:34.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.846+0000 7f695b7fe640 1 -- 172.21.15.107:0/1252755428 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f69600ceb20 con 0x7f696c100fe0 2024-04-25T00:23:34.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.846+0000 7f696a59c640 1 --2- 172.21.15.107:0/1252755428 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f69540a2d00 0x7f69540a51c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6964006010 tx=0x7f6964009800 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:34.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:34.851+0000 7f695b7fe640 1 -- 172.21.15.107:0/1252755428 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f696009b490 con 0x7f696c100fe0 2024-04-25T00:23:35.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.066+0000 7f6970b09640 1 -- 172.21.15.107:0/1252755428 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f696c198280 con 0x7f696c100fe0 2024-04-25T00:23:35.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.067+0000 7f695b7fe640 1 -- 172.21.15.107:0/1252755428 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11897 (secure 0 0 0) 0x7f696c198280 con 0x7f696c100fe0 2024-04-25T00:23:35.068 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:23:35.068 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:22:27.402680+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24493,24521]},"id":1}]} 2024-04-25T00:23:35.068 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 34 2024-04-25T00:23:35.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.069+0000 7f6970b09640 1 -- 172.21.15.107:0/1252755428 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f69540a2d00 msgr2=0x7f69540a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.069+0000 7f6970b09640 1 --2- 172.21.15.107:0/1252755428 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f69540a2d00 0x7f69540a51c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6964006010 tx=0x7f6964009800 comp rx=0 tx=0).stop 2024-04-25T00:23:35.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.069+0000 7f6970b09640 1 -- 172.21.15.107:0/1252755428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f696c100fe0 msgr2=0x7f696c1974c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.069+0000 7f6970b09640 1 --2- 172.21.15.107:0/1252755428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f696c100fe0 0x7f696c1974c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f696000b4a0 tx=0x7f696000b960 comp rx=0 tx=0).stop 2024-04-25T00:23:35.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.069+0000 7f6970b09640 1 -- 172.21.15.107:0/1252755428 shutdown_connections 2024-04-25T00:23:35.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.069+0000 7f6970b09640 1 --2- 172.21.15.107:0/1252755428 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f69540a2d00 0x7f69540a51c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.069+0000 7f6970b09640 1 --2- 172.21.15.107:0/1252755428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f696c106630 0x7f696c19b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.069+0000 7f6970b09640 1 --2- 172.21.15.107:0/1252755428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f696c100fe0 0x7f696c1974c0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.069+0000 7f6970b09640 1 --2- 172.21.15.107:0/1252755428 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f696c100620 0x7f696c196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.069+0000 7f6970b09640 1 -- 172.21.15.107:0/1252755428 >> 172.21.15.107:0/1252755428 conn(0x7f696c077740 msgr2=0x7f696c102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:35.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.069+0000 7f6970b09640 1 -- 172.21.15.107:0/1252755428 shutdown_connections 2024-04-25T00:23:35.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.069+0000 7f6970b09640 1 -- 172.21.15.107:0/1252755428 wait complete. 2024-04-25T00:23:35.080 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:23:35.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.179+0000 7fa0201ed640 1 -- 172.21.15.107:0/1925883240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa018107720 msgr2=0x7fa018107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.180+0000 7fa0201ed640 1 --2- 172.21.15.107:0/1925883240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa018107720 0x7fa018107b00 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa014009930 tx=0x7fa01402a420 comp rx=0 tx=0).stop 2024-04-25T00:23:35.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.180+0000 7fa0201ed640 1 -- 172.21.15.107:0/1925883240 shutdown_connections 2024-04-25T00:23:35.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.180+0000 7fa0201ed640 1 --2- 172.21.15.107:0/1925883240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0181020d0 0x7fa01810e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.180+0000 7fa0201ed640 1 --2- 172.21.15.107:0/1925883240 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa018101710 0x7fa018101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.180+0000 7fa0201ed640 1 --2- 172.21.15.107:0/1925883240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa018107720 0x7fa018107b00 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.180+0000 7fa0201ed640 1 -- 172.21.15.107:0/1925883240 >> 172.21.15.107:0/1925883240 conn(0x7fa0180fd180 msgr2=0x7fa0180ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:35.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.180+0000 7fa0201ed640 1 -- 172.21.15.107:0/1925883240 shutdown_connections 2024-04-25T00:23:35.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.180+0000 7fa0201ed640 1 -- 172.21.15.107:0/1925883240 wait complete. 2024-04-25T00:23:35.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.180+0000 7fa0201ed640 1 Processor -- start 2024-04-25T00:23:35.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.180+0000 7fa0201ed640 1 -- start start 2024-04-25T00:23:35.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.180+0000 7fa0201ed640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa018101710 0x7fa018198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.180+0000 7fa0201ed640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0181020d0 0x7fa0181985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.180+0000 7fa0201ed640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa018107720 0x7fa01819c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.180+0000 7fa0201ed640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa018111b30 con 0x7fa018101710 2024-04-25T00:23:35.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.180+0000 7fa0201ed640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa0181119b0 con 0x7fa0181020d0 2024-04-25T00:23:35.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.180+0000 7fa0201ed640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa018111cb0 con 0x7fa018107720 2024-04-25T00:23:35.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.180+0000 7fa01df89640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa018101710 0x7fa018198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.181+0000 7fa01df89640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa018101710 0x7fa018198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57366/0 (socket says 172.21.15.107:57366) 2024-04-25T00:23:35.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.181+0000 7fa01df89640 1 -- 172.21.15.107:0/3347935296 learned_addr learned my addr 172.21.15.107:0/3347935296 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:35.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.181+0000 7fa01e78a640 1 --2- 172.21.15.107:0/3347935296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa018107720 0x7fa01819c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.181+0000 7fa01df89640 1 -- 172.21.15.107:0/3347935296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa018107720 msgr2=0x7fa01819c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.181+0000 7fa01df89640 1 --2- 172.21.15.107:0/3347935296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa018107720 0x7fa01819c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.181+0000 7fa01df89640 1 -- 172.21.15.107:0/3347935296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0181020d0 msgr2=0x7fa0181985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.181+0000 7fa01df89640 1 --2- 172.21.15.107:0/3347935296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0181020d0 0x7fa0181985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.181+0000 7fa01df89640 1 -- 172.21.15.107:0/3347935296 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa01819d060 con 0x7fa018101710 2024-04-25T00:23:35.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.181+0000 7fa01e78a640 1 --2- 172.21.15.107:0/3347935296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa018107720 0x7fa01819c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:35.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.181+0000 7fa01df89640 1 --2- 172.21.15.107:0/3347935296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa018101710 0x7fa018198070 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa014009230 tx=0x7fa014004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:35.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.181+0000 7fa00effd640 1 -- 172.21.15.107:0/3347935296 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01403f030 con 0x7fa018101710 2024-04-25T00:23:35.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.181+0000 7fa00effd640 1 -- 172.21.15.107:0/3347935296 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa014003ad0 con 0x7fa018101710 2024-04-25T00:23:35.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.181+0000 7fa0201ed640 1 -- 172.21.15.107:0/3347935296 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa01819d2f0 con 0x7fa018101710 2024-04-25T00:23:35.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.181+0000 7fa00effd640 1 -- 172.21.15.107:0/3347935296 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa014035070 con 0x7fa018101710 2024-04-25T00:23:35.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.181+0000 7fa0201ed640 1 -- 172.21.15.107:0/3347935296 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0181a4be0 con 0x7fa018101710 2024-04-25T00:23:35.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.184+0000 7fa00effd640 1 -- 172.21.15.107:0/3347935296 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa014042030 con 0x7fa018101710 2024-04-25T00:23:35.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.184+0000 7fa022fd0740 1 Processor -- start 2024-04-25T00:23:35.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.184+0000 7fa022fd0740 1 -- start start 2024-04-25T00:23:35.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.184+0000 7fa022fd0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56202e1a7580 0x56202e1a7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.184+0000 7fa022fd0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56202e1a7ea0 0x56202e1b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.184+0000 7fa022fd0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1b2570 0x56202e1b4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.184+0000 7fa022fd0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56202e10c670 con 0x56202e1b2570 2024-04-25T00:23:35.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.184+0000 7fa022fd0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56202e0b4970 con 0x56202e1a7580 2024-04-25T00:23:35.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.184+0000 7fa022fd0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56202e108630 con 0x56202e1a7ea0 2024-04-25T00:23:35.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.184+0000 7fa01d788640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1b2570 0x56202e1b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.185+0000 7fa01d788640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1b2570 0x56202e1b4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57378/0 (socket says 172.21.15.107:57378) 2024-04-25T00:23:35.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.185+0000 7fa01d788640 1 -- 172.21.15.107:0/1275430900 learned_addr learned my addr 172.21.15.107:0/1275430900 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:35.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.185+0000 7fa00effd640 1 --2- 172.21.15.107:0/3347935296 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa0040a2d00 0x7fa0040a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.185+0000 7fa01e78a640 1 --2- 172.21.15.107:0/1275430900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56202e1a7580 0x56202e1a7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.185+0000 7fa01df89640 1 --2- 172.21.15.107:0/1275430900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56202e1a7ea0 0x56202e1b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.185+0000 7fa01e78a640 1 --2- 172.21.15.107:0/3347935296 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa0040a2d00 0x7fa0040a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.185+0000 7fa01d788640 1 -- 172.21.15.107:0/1275430900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56202e1a7ea0 msgr2=0x56202e1b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:35.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.185+0000 7fa01d788640 1 --2- 172.21.15.107:0/1275430900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56202e1a7ea0 0x56202e1b2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.185+0000 7fa01d788640 1 -- 172.21.15.107:0/1275430900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56202e1a7580 msgr2=0x56202e1a7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:35.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.185+0000 7fa01d788640 1 --2- 172.21.15.107:0/1275430900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56202e1a7580 0x56202e1a7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.185+0000 7fa01d788640 1 -- 172.21.15.107:0/1275430900 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56202e0d8400 con 0x56202e1b2570 2024-04-25T00:23:35.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.185+0000 7fa01e78a640 1 --2- 172.21.15.107:0/1275430900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56202e1a7580 0x56202e1a7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:35.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.185+0000 7fa01df89640 1 --2- 172.21.15.107:0/1275430900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56202e1a7ea0 0x56202e1b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:35.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.185+0000 7fa00effd640 1 -- 172.21.15.107:0/3347935296 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fa0140efdc0 con 0x7fa018101710 2024-04-25T00:23:35.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.185+0000 7fa01d788640 1 --2- 172.21.15.107:0/1275430900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1b2570 0x56202e1b4960 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fa00800bae0 tx=0x7fa00800d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27adbddb60609e4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:35.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.185+0000 7fa003fff640 1 -- 172.21.15.107:0/1275430900 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa008016070 con 0x56202e1b2570 2024-04-25T00:23:35.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.185+0000 7fa003fff640 1 -- 172.21.15.107:0/1275430900 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa008005500 con 0x56202e1b2570 2024-04-25T00:23:35.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.185+0000 7fa003fff640 1 -- 172.21.15.107:0/1275430900 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa008010460 con 0x56202e1b2570 2024-04-25T00:23:35.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.186+0000 7fa022fd0740 1 -- 172.21.15.107:0/1275430900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1b2570 msgr2=0x56202e1b4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.186+0000 7fa01e78a640 1 --2- 172.21.15.107:0/3347935296 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa0040a2d00 0x7fa0040a51c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa010005c40 tx=0x7fa010009280 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:35.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.186+0000 7fa022fd0740 1 --2- 172.21.15.107:0/1275430900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1b2570 0x56202e1b4960 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fa00800bae0 tx=0x7fa00800d2e0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.186+0000 7fa022fd0740 1 -- 172.21.15.107:0/1275430900 shutdown_connections 2024-04-25T00:23:35.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.186+0000 7fa022fd0740 1 --2- 172.21.15.107:0/1275430900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1b2570 0x56202e1b4960 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.186+0000 7fa022fd0740 1 --2- 172.21.15.107:0/1275430900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56202e1a7ea0 0x56202e1b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.186+0000 7fa022fd0740 1 --2- 172.21.15.107:0/1275430900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56202e1a7580 0x56202e1a7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.186+0000 7fa022fd0740 1 -- 172.21.15.107:0/1275430900 >> 172.21.15.107:0/1275430900 conn(0x56202e1a38a0 msgr2=0x56202e1a3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:35.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.186+0000 7fa022fd0740 1 -- 172.21.15.107:0/1275430900 shutdown_connections 2024-04-25T00:23:35.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.186+0000 7fa022fd0740 1 -- 172.21.15.107:0/1275430900 wait complete. 2024-04-25T00:23:35.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.186+0000 7fa022fd0740 1 -- start start 2024-04-25T00:23:35.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.186+0000 7fa022fd0740 1 Processor -- start 2024-04-25T00:23:35.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.187+0000 7fa022fd0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56202e1a7580 0x56202e1b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.187+0000 7fa022fd0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56202e1a7ea0 0x56202e1af170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.187+0000 7fa022fd0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1b2570 0x56202e1af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.187+0000 7fa022fd0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56202e1b5360 con 0x56202e1b2570 2024-04-25T00:23:35.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.187+0000 7fa022fd0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56202e0c8050 con 0x56202e1a7580 2024-04-25T00:23:35.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.187+0000 7fa022fd0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56202e1b54e0 con 0x56202e1a7ea0 2024-04-25T00:23:35.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.187+0000 7fa022fd0740 10 client.0 _ll_register_callbacks cb 0x56202de1fbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:35.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.187+0000 7fa01e78a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1b2570 0x56202e1af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.187+0000 7fa01e78a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1b2570 0x56202e1af6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57392/0 (socket says 172.21.15.107:57392) 2024-04-25T00:23:35.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.187+0000 7fa01e78a640 1 -- 172.21.15.107:0/1284988226 learned_addr learned my addr 172.21.15.107:0/1284988226 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:35.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.187+0000 7fa01df89640 1 --2- 172.21.15.107:0/1284988226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56202e1a7580 0x56202e1b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.187+0000 7fa01d788640 1 --2- 172.21.15.107:0/1284988226 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56202e1a7ea0 0x56202e1af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.187+0000 7fa01e78a640 1 -- 172.21.15.107:0/1284988226 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56202e1a7ea0 msgr2=0x56202e1af170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:35.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.187+0000 7fa01e78a640 1 --2- 172.21.15.107:0/1284988226 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56202e1a7ea0 0x56202e1af170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.187+0000 7fa01e78a640 1 -- 172.21.15.107:0/1284988226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56202e1a7580 msgr2=0x56202e1b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:35.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.187+0000 7fa01e78a640 1 --2- 172.21.15.107:0/1284988226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56202e1a7580 0x56202e1b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.187+0000 7fa01e78a640 1 -- 172.21.15.107:0/1284988226 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56202e099dc0 con 0x56202e1b2570 2024-04-25T00:23:35.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.187+0000 7fa01d788640 1 --2- 172.21.15.107:0/1284988226 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56202e1a7ea0 0x56202e1af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:35.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.187+0000 7fa01e78a640 1 --2- 172.21.15.107:0/1284988226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1b2570 0x56202e1af6b0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fa010010970 tx=0x7fa010004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c63abafd5ab61b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:35.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.187+0000 7fa0017fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:35.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.187+0000 7fa0017fa640 1 -- 172.21.15.107:0/1284988226 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01002a060 con 0x56202e1b2570 2024-04-25T00:23:35.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.187+0000 7fa0017fa640 1 -- 172.21.15.107:0/1284988226 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fa01001db90 con 0x56202e1b2570 2024-04-25T00:23:35.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.187+0000 7fa022fd0740 1 -- 172.21.15.107:0/1284988226 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa008005500 con 0x56202e1b2570 2024-04-25T00:23:35.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.188+0000 7fa0017fa640 1 -- 172.21.15.107:0/1284988226 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa010031050 con 0x56202e1b2570 2024-04-25T00:23:35.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.188+0000 7fa022fd0740 10 client.17220 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:35.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.188+0000 7fa022fd0740 1 -- 172.21.15.107:0/1284988226 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56202de203f0 con 0x56202e1b2570 2024-04-25T00:23:35.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.188+0000 7fa0017fa640 1 -- 172.21.15.107:0/1284988226 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa008005500 con 0x56202e1b2570 2024-04-25T00:23:35.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.188+0000 7fa0017fa640 1 -- 172.21.15.107:0/1284988226 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fa010019040 con 0x56202e1b2570 2024-04-25T00:23:35.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.188+0000 7fa022fd0740 10 client.17220 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:35.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.188+0000 7fa022fd0740 10 client.17220 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:23:35.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.188+0000 7fa022fd0740 20 client.17220 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:35.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.188+0000 7fa022fd0740 1 --2- 172.21.15.107:0/1284988226 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x56202e1c25c0 0x56202e1c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.188+0000 7fa022fd0740 4 client.17220 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:35.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.188+0000 7fa022fd0740 1 -- 172.21.15.107:0/1284988226 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56202e119bb0 con 0x56202e1c25c0 2024-04-25T00:23:35.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.188+0000 7fa01df89640 1 --2- 172.21.15.107:0/1284988226 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x56202e1c25c0 0x56202e1c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.189+0000 7fa01df89640 1 --2- 172.21.15.107:0/1284988226 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x56202e1c25c0 0x56202e1c4a20 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9f4cf54c6dfdda1 server_cookie=4edfe900e0f634cb in_seq=0 out_seq=0 2024-04-25T00:23:35.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.189+0000 7fa0017fa640 10 client.17220 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:23:35.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.189+0000 7fa0017fa640 1 -- 172.21.15.107:0/1284988226 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56202e119bb0 con 0x56202e1c25c0 2024-04-25T00:23:35.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.189+0000 7fa0017fa640 10 client.17220 handle_command_reply: tid=0 2024-04-25T00:23:35.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.189+0000 7fa022fd0740 1 client.17220 shutdown 2024-04-25T00:23:35.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.189+0000 7fa022fd0740 1 -- 172.21.15.107:0/1284988226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1b2570 msgr2=0x56202e1af6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.189+0000 7fa022fd0740 1 --2- 172.21.15.107:0/1284988226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1b2570 0x56202e1af6b0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fa010010970 tx=0x7fa010004ab0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.190+0000 7fa022fd0740 1 -- 172.21.15.107:0/1284988226 shutdown_connections 2024-04-25T00:23:35.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.190+0000 7fa022fd0740 1 --2- 172.21.15.107:0/1284988226 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x56202e1c25c0 0x56202e1c4a20 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.190+0000 7fa022fd0740 1 --2- 172.21.15.107:0/1284988226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1b2570 0x56202e1af6b0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.190+0000 7fa022fd0740 1 --2- 172.21.15.107:0/1284988226 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56202e1a7ea0 0x56202e1af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.190+0000 7fa0017fa640 0 client.17220 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:23:35.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.190+0000 7fa022fd0740 1 --2- 172.21.15.107:0/1284988226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56202e1a7580 0x56202e1b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.190+0000 7fa022fd0740 1 -- 172.21.15.107:0/1284988226 >> 172.21.15.107:0/1284988226 conn(0x56202e1a38a0 msgr2=0x56202e1a2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:35.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.190+0000 7fa022fd0740 1 -- 172.21.15.107:0/1284988226 shutdown_connections 2024-04-25T00:23:35.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.190+0000 7fa022fd0740 1 -- 172.21.15.107:0/1284988226 wait complete. 2024-04-25T00:23:35.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.190+0000 7fa022fd0740 20 client.17220 trim_cache size 0 max 16384 2024-04-25T00:23:35.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.203+0000 7fa022fd0740 1 Processor -- start 2024-04-25T00:23:35.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.203+0000 7fa022fd0740 1 -- start start 2024-04-25T00:23:35.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.203+0000 7fa022fd0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56202e1a7580 0x56202e1be250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.203+0000 7fa022fd0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56202e1a7ea0 0x56202e1ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.203+0000 7fa022fd0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1b2570 0x56202e1ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.203+0000 7fa022fd0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa0181b1590 con 0x56202e1b2570 2024-04-25T00:23:35.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.203+0000 7fa022fd0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa0181aee60 con 0x56202e1a7ea0 2024-04-25T00:23:35.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.203+0000 7fa022fd0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa0181accd0 con 0x56202e1a7580 2024-04-25T00:23:35.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.203+0000 7fa01e78a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1b2570 0x56202e1ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.203+0000 7fa01e78a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1b2570 0x56202e1ca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57394/0 (socket says 172.21.15.107:57394) 2024-04-25T00:23:35.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.203+0000 7fa01e78a640 1 -- 172.21.15.107:0/3796641257 learned_addr learned my addr 172.21.15.107:0/3796641257 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:35.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.203+0000 7fa01d788640 1 --2- 172.21.15.107:0/3796641257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56202e1a7ea0 0x56202e1ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.203+0000 7fa01df89640 1 --2- 172.21.15.107:0/3796641257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56202e1a7580 0x56202e1be250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.203+0000 7fa01e78a640 1 -- 172.21.15.107:0/3796641257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56202e1a7580 msgr2=0x56202e1be250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:35.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.203+0000 7fa01e78a640 1 --2- 172.21.15.107:0/3796641257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56202e1a7580 0x56202e1be250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.203+0000 7fa01e78a640 1 -- 172.21.15.107:0/3796641257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56202e1a7ea0 msgr2=0x56202e1ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:35.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.203+0000 7fa01e78a640 1 --2- 172.21.15.107:0/3796641257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56202e1a7ea0 0x56202e1ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.203+0000 7fa01e78a640 1 -- 172.21.15.107:0/3796641257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56202deea7d0 con 0x56202e1b2570 2024-04-25T00:23:35.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.203+0000 7fa01e78a640 1 --2- 172.21.15.107:0/3796641257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1b2570 0x56202e1ca830 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fa010017040 tx=0x7fa010012d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6812daf63b80e01b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:35.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.204+0000 7fa0037fe640 1 -- 172.21.15.107:0/3796641257 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01003a070 con 0x56202e1b2570 2024-04-25T00:23:35.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.204+0000 7fa0037fe640 1 -- 172.21.15.107:0/3796641257 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa01001d7d0 con 0x56202e1b2570 2024-04-25T00:23:35.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.204+0000 7fa0037fe640 1 -- 172.21.15.107:0/3796641257 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01001b940 con 0x56202e1b2570 2024-04-25T00:23:35.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.204+0000 7fa022fd0740 1 -- 172.21.15.107:0/3796641257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1b2570 msgr2=0x56202e1ca830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.204+0000 7fa022fd0740 1 --2- 172.21.15.107:0/3796641257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1b2570 0x56202e1ca830 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fa010017040 tx=0x7fa010012d60 comp rx=0 tx=0).stop 2024-04-25T00:23:35.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.204+0000 7fa022fd0740 1 -- 172.21.15.107:0/3796641257 shutdown_connections 2024-04-25T00:23:35.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.204+0000 7fa022fd0740 1 --2- 172.21.15.107:0/3796641257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1b2570 0x56202e1ca830 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.204+0000 7fa022fd0740 1 --2- 172.21.15.107:0/3796641257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56202e1a7ea0 0x56202e1ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.204+0000 7fa022fd0740 1 --2- 172.21.15.107:0/3796641257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56202e1a7580 0x56202e1be250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.204+0000 7fa022fd0740 1 -- 172.21.15.107:0/3796641257 >> 172.21.15.107:0/3796641257 conn(0x56202e1a38a0 msgr2=0x56202e1a22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:35.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.204+0000 7fa022fd0740 1 -- 172.21.15.107:0/3796641257 shutdown_connections 2024-04-25T00:23:35.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.204+0000 7fa022fd0740 1 -- 172.21.15.107:0/3796641257 wait complete. 2024-04-25T00:23:35.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.205+0000 7fa022fd0740 1 -- start start 2024-04-25T00:23:35.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.205+0000 7fa022fd0740 1 Processor -- start 2024-04-25T00:23:35.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.205+0000 7fa022fd0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1a7580 0x56202e1baaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.205+0000 7fa022fd0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56202e1a7ea0 0x56202e1b7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.205+0000 7fa022fd0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56202e1b2570 0x56202e1b8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.205+0000 7fa022fd0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa0181ad320 con 0x56202e1a7580 2024-04-25T00:23:35.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.205+0000 7fa022fd0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa0181ae790 con 0x56202e1b2570 2024-04-25T00:23:35.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.205+0000 7fa022fd0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa0181ad9f0 con 0x56202e1a7ea0 2024-04-25T00:23:35.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.205+0000 7fa022fd0740 10 client.0 _ll_register_callbacks cb 0x7fa0100098b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:35.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.205+0000 7fa01df89640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1a7580 0x56202e1baaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.205+0000 7fa01df89640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1a7580 0x56202e1baaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57404/0 (socket says 172.21.15.107:57404) 2024-04-25T00:23:35.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.205+0000 7fa01df89640 1 -- 172.21.15.107:0/590802373 learned_addr learned my addr 172.21.15.107:0/590802373 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:35.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.205+0000 7fa01e78a640 1 --2- 172.21.15.107:0/590802373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56202e1b2570 0x56202e1b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.205+0000 7fa01df89640 1 -- 172.21.15.107:0/590802373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56202e1a7ea0 msgr2=0x56202e1b7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:35.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.205+0000 7fa01df89640 1 --2- 172.21.15.107:0/590802373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56202e1a7ea0 0x56202e1b7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.205+0000 7fa01df89640 1 -- 172.21.15.107:0/590802373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56202e1b2570 msgr2=0x56202e1b8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:35.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.205+0000 7fa01df89640 1 --2- 172.21.15.107:0/590802373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56202e1b2570 0x56202e1b8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.205+0000 7fa01df89640 1 -- 172.21.15.107:0/590802373 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56202e1b8b20 con 0x56202e1a7580 2024-04-25T00:23:35.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.206+0000 7fa01df89640 1 --2- 172.21.15.107:0/590802373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1a7580 0x56202e1baaa0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fa014032c00 tx=0x7fa014036450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a86197209f33d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:35.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.206+0000 7fa0017fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:35.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.206+0000 7fa0017fa640 1 -- 172.21.15.107:0/590802373 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0140d7060 con 0x56202e1a7580 2024-04-25T00:23:35.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.206+0000 7fa0017fa640 1 -- 172.21.15.107:0/590802373 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fa01403b460 con 0x56202e1a7580 2024-04-25T00:23:35.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.206+0000 7fa022fd0740 1 -- 172.21.15.107:0/590802373 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa01001d7d0 con 0x56202e1a7580 2024-04-25T00:23:35.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.206+0000 7fa0017fa640 1 -- 172.21.15.107:0/590802373 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa0140d4070 con 0x56202e1a7580 2024-04-25T00:23:35.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.206+0000 7fa022fd0740 10 client.17226 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:35.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.206+0000 7fa022fd0740 1 -- 172.21.15.107:0/590802373 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56202e228b50 con 0x56202e1a7580 2024-04-25T00:23:35.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.206+0000 7fa0017fa640 1 -- 172.21.15.107:0/590802373 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa01001d7d0 con 0x56202e1a7580 2024-04-25T00:23:35.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.206+0000 7fa0017fa640 1 -- 172.21.15.107:0/590802373 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fa0140bf070 con 0x56202e1a7580 2024-04-25T00:23:35.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.206+0000 7fa022fd0740 10 client.17226 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:35.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.206+0000 7fa022fd0740 10 client.17226 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:23:35.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.206+0000 7fa022fd0740 20 client.17226 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:35.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.206+0000 7fa022fd0740 1 --2- 172.21.15.107:0/590802373 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x56202e229220 0x56202e22b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.206+0000 7fa022fd0740 4 client.17226 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:23:35.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.206+0000 7fa022fd0740 1 -- 172.21.15.107:0/590802373 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56202e113f00 con 0x56202e229220 2024-04-25T00:23:35.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.207+0000 7fa01d788640 1 --2- 172.21.15.107:0/590802373 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x56202e229220 0x56202e22b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.207+0000 7fa01d788640 1 --2- 172.21.15.107:0/590802373 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x56202e229220 0x56202e22b660 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=303fe99e27061676 server_cookie=2cde0f1a19eb5cd6 in_seq=0 out_seq=0 2024-04-25T00:23:35.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.207+0000 7fa0017fa640 10 client.17226 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:23:35.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.207+0000 7fa0017fa640 1 -- 172.21.15.107:0/590802373 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56202e113f00 con 0x56202e229220 2024-04-25T00:23:35.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.207+0000 7fa0017fa640 10 client.17226 handle_command_reply: tid=0 2024-04-25T00:23:35.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.207+0000 7fa022fd0740 1 client.17226 shutdown 2024-04-25T00:23:35.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.207+0000 7fa022fd0740 1 -- 172.21.15.107:0/590802373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1a7580 msgr2=0x56202e1baaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.207+0000 7fa022fd0740 1 --2- 172.21.15.107:0/590802373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1a7580 0x56202e1baaa0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fa014032c00 tx=0x7fa014036450 comp rx=0 tx=0).stop 2024-04-25T00:23:35.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.207+0000 7fa022fd0740 1 -- 172.21.15.107:0/590802373 shutdown_connections 2024-04-25T00:23:35.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.207+0000 7fa022fd0740 1 --2- 172.21.15.107:0/590802373 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x56202e229220 0x56202e22b660 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.207+0000 7fa022fd0740 1 --2- 172.21.15.107:0/590802373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56202e1b2570 0x56202e1b8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.207+0000 7fa022fd0740 1 --2- 172.21.15.107:0/590802373 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56202e1a7ea0 0x56202e1b7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.207+0000 7fa0017fa640 0 client.17226 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:23:35.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.207+0000 7fa022fd0740 1 --2- 172.21.15.107:0/590802373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56202e1a7580 0x56202e1baaa0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.207+0000 7fa022fd0740 1 -- 172.21.15.107:0/590802373 >> 172.21.15.107:0/590802373 conn(0x56202e1a38a0 msgr2=0x56202e1a2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:35.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.207+0000 7fa022fd0740 1 -- 172.21.15.107:0/590802373 shutdown_connections 2024-04-25T00:23:35.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.207+0000 7fa022fd0740 1 -- 172.21.15.107:0/590802373 wait complete. 2024-04-25T00:23:35.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.207+0000 7fa022fd0740 20 client.17226 trim_cache size 0 max 16384 2024-04-25T00:23:35.215 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:23:35.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.208+0000 7fa0201ed640 1 -- 172.21.15.107:0/3347935296 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa0040a2d00 msgr2=0x7fa0040a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.208+0000 7fa0201ed640 1 --2- 172.21.15.107:0/3347935296 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa0040a2d00 0x7fa0040a51c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fa010005c40 tx=0x7fa010009280 comp rx=0 tx=0).stop 2024-04-25T00:23:35.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.208+0000 7fa0201ed640 1 -- 172.21.15.107:0/3347935296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa018101710 msgr2=0x7fa018198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.208+0000 7fa0201ed640 1 --2- 172.21.15.107:0/3347935296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa018101710 0x7fa018198070 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa014009230 tx=0x7fa014004270 comp rx=0 tx=0).stop 2024-04-25T00:23:35.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.208+0000 7fa0201ed640 1 -- 172.21.15.107:0/3347935296 shutdown_connections 2024-04-25T00:23:35.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.208+0000 7fa0201ed640 1 --2- 172.21.15.107:0/3347935296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa018107720 0x7fa01819c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.208+0000 7fa0201ed640 1 --2- 172.21.15.107:0/3347935296 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa0040a2d00 0x7fa0040a51c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.208+0000 7fa0201ed640 1 --2- 172.21.15.107:0/3347935296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa0181020d0 0x7fa0181985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.208+0000 7fa0201ed640 1 --2- 172.21.15.107:0/3347935296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa018101710 0x7fa018198070 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.208+0000 7fa0201ed640 1 -- 172.21.15.107:0/3347935296 >> 172.21.15.107:0/3347935296 conn(0x7fa0180fd180 msgr2=0x7fa0180fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:35.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.208+0000 7fa0201ed640 1 -- 172.21.15.107:0/3347935296 shutdown_connections 2024-04-25T00:23:35.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.208+0000 7fa0201ed640 1 -- 172.21.15.107:0/3347935296 wait complete. 2024-04-25T00:23:35.219 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:23:35.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.319+0000 7f8edaa58640 1 -- 172.21.15.107:0/3640592038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ed4100b50 msgr2=0x7f8ed410cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.319+0000 7f8edaa58640 1 --2- 172.21.15.107:0/3640592038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ed4100b50 0x7f8ed410cca0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009ca0 tx=0x7f8ecc02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.319+0000 7f8edaa58640 1 -- 172.21.15.107:0/3640592038 shutdown_connections 2024-04-25T00:23:35.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.319+0000 7f8edaa58640 1 --2- 172.21.15.107:0/3640592038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ed4100b50 0x7f8ed410cca0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.319+0000 7f8edaa58640 1 --2- 172.21.15.107:0/3640592038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ed41001b0 0x7f8ed4100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.319+0000 7f8edaa58640 1 --2- 172.21.15.107:0/3640592038 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ed4105e90 0x7f8ed4106270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.319+0000 7f8edaa58640 1 -- 172.21.15.107:0/3640592038 >> 172.21.15.107:0/3640592038 conn(0x7f8ed40767a0 msgr2=0x7f8ed40fdb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:35.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.319+0000 7f8edaa58640 1 -- 172.21.15.107:0/3640592038 shutdown_connections 2024-04-25T00:23:35.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.319+0000 7f8edaa58640 1 -- 172.21.15.107:0/3640592038 wait complete. 2024-04-25T00:23:35.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.320+0000 7f8edaa58640 1 Processor -- start 2024-04-25T00:23:35.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.320+0000 7f8edaa58640 1 -- start start 2024-04-25T00:23:35.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.320+0000 7f8edaa58640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ed41001b0 0x7f8ed4198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.320+0000 7f8edaa58640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ed4100b50 0x7f8ed4198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.320+0000 7f8edaa58640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ed4105e90 0x7f8ed419ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.320+0000 7f8edaa58640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed4111bb0 con 0x7f8ed4105e90 2024-04-25T00:23:35.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.320+0000 7f8edaa58640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed4111a30 con 0x7f8ed41001b0 2024-04-25T00:23:35.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.320+0000 7f8edaa58640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed4111d30 con 0x7f8ed4100b50 2024-04-25T00:23:35.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.320+0000 7f8ed8ff5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ed4105e90 0x7f8ed419ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.320+0000 7f8ed8ff5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ed4105e90 0x7f8ed419ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57422/0 (socket says 172.21.15.107:57422) 2024-04-25T00:23:35.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.320+0000 7f8ed8ff5640 1 -- 172.21.15.107:0/1640629192 learned_addr learned my addr 172.21.15.107:0/1640629192 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:35.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.320+0000 7f8ed3fff640 1 --2- 172.21.15.107:0/1640629192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ed41001b0 0x7f8ed4198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.320+0000 7f8ed37fe640 1 --2- 172.21.15.107:0/1640629192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ed4100b50 0x7f8ed4198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.321+0000 7f8ed8ff5640 1 -- 172.21.15.107:0/1640629192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ed4100b50 msgr2=0x7f8ed4198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.321+0000 7f8ed8ff5640 1 --2- 172.21.15.107:0/1640629192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ed4100b50 0x7f8ed4198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.321+0000 7f8ed8ff5640 1 -- 172.21.15.107:0/1640629192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ed41001b0 msgr2=0x7f8ed4198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.321+0000 7f8ed8ff5640 1 --2- 172.21.15.107:0/1640629192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ed41001b0 0x7f8ed4198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.321+0000 7f8ed8ff5640 1 -- 172.21.15.107:0/1640629192 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed419d140 con 0x7f8ed4105e90 2024-04-25T00:23:35.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.321+0000 7f8ed37fe640 1 --2- 172.21.15.107:0/1640629192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ed4100b50 0x7f8ed4198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:35.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.321+0000 7f8ed8ff5640 1 --2- 172.21.15.107:0/1640629192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ed4105e90 0x7f8ed419ca10 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc031970 tx=0x7f8ecc0319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:35.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.321+0000 7f8ed17fa640 1 -- 172.21.15.107:0/1640629192 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc003b70 con 0x7f8ed4105e90 2024-04-25T00:23:35.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.321+0000 7f8ed17fa640 1 -- 172.21.15.107:0/1640629192 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ecc003d10 con 0x7f8ed4105e90 2024-04-25T00:23:35.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.321+0000 7f8edaa58640 1 -- 172.21.15.107:0/1640629192 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed419d3d0 con 0x7f8ed4105e90 2024-04-25T00:23:35.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.321+0000 7f8ed17fa640 1 -- 172.21.15.107:0/1640629192 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc02fc00 con 0x7f8ed4105e90 2024-04-25T00:23:35.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.321+0000 7f8edaa58640 1 -- 172.21.15.107:0/1640629192 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed41a4cc0 con 0x7f8ed4105e90 2024-04-25T00:23:35.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.323+0000 7f8ed17fa640 1 -- 172.21.15.107:0/1640629192 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f8ecc0046a0 con 0x7f8ed4105e90 2024-04-25T00:23:35.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.323+0000 7f8edd763740 1 Processor -- start 2024-04-25T00:23:35.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.324+0000 7f8edd763740 1 -- start start 2024-04-25T00:23:35.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.324+0000 7f8edd763740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6459cb580 0x55d6459cb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.324+0000 7f8edd763740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459cbea0 0x55d6459d6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.324+0000 7f8edd763740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6459d6570 0x55d6459d8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.324+0000 7f8edd763740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d645930670 con 0x55d6459cbea0 2024-04-25T00:23:35.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.324+0000 7f8edd763740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d6458d8970 con 0x55d6459cb580 2024-04-25T00:23:35.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.324+0000 7f8edd763740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d64592c630 con 0x55d6459d6570 2024-04-25T00:23:35.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.324+0000 7f8ed17fa640 1 --2- 172.21.15.107:0/1640629192 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8ebc0a2d00 0x7f8ebc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.324+0000 7f8ed3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459cbea0 0x55d6459d6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.324+0000 7f8ed3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459cbea0 0x55d6459d6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57430/0 (socket says 172.21.15.107:57430) 2024-04-25T00:23:35.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.324+0000 7f8ed3fff640 1 -- 172.21.15.107:0/1229297636 learned_addr learned my addr 172.21.15.107:0/1229297636 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:35.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.324+0000 7f8ed8ff5640 1 --2- 172.21.15.107:0/1640629192 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8ebc0a2d00 0x7f8ebc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.324+0000 7f8ed37fe640 1 --2- 172.21.15.107:0/1229297636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6459cb580 0x55d6459cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.324+0000 7f8ed37fe640 1 --2- 172.21.15.107:0/1229297636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6459d6570 0x55d6459d8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.324+0000 7f8ed17fa640 1 -- 172.21.15.107:0/1640629192 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f8ecc0f0750 con 0x7f8ed4105e90 2024-04-25T00:23:35.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.324+0000 7f8ed3fff640 1 -- 172.21.15.107:0/1229297636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6459d6570 msgr2=0x55d6459d8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:35.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.324+0000 7f8ed3fff640 1 --2- 172.21.15.107:0/1229297636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6459d6570 0x55d6459d8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.324+0000 7f8ed3fff640 1 -- 172.21.15.107:0/1229297636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6459cb580 msgr2=0x55d6459cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:35.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.324+0000 7f8ed3fff640 1 --2- 172.21.15.107:0/1229297636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6459cb580 0x55d6459cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.324+0000 7f8ed3fff640 1 -- 172.21.15.107:0/1229297636 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6458fc400 con 0x55d6459cbea0 2024-04-25T00:23:35.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.324+0000 7f8ed3fff640 1 --2- 172.21.15.107:0/1229297636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459cbea0 0x55d6459d6030 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009ca0 tx=0x7f8ec4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf039c6410414d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:35.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.324+0000 7f8ed8ff5640 1 --2- 172.21.15.107:0/1640629192 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8ebc0a2d00 0x7f8ebc0a51c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc0ee0e0 tx=0x7f8ecc08e980 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:35.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.325+0000 7f8eba7fc640 1 -- 172.21.15.107:0/1229297636 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec401e070 con 0x55d6459cbea0 2024-04-25T00:23:35.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.325+0000 7f8eba7fc640 1 -- 172.21.15.107:0/1229297636 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec4009e50 con 0x55d6459cbea0 2024-04-25T00:23:35.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.325+0000 7f8eba7fc640 1 -- 172.21.15.107:0/1229297636 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec40187e0 con 0x55d6459cbea0 2024-04-25T00:23:35.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.325+0000 7f8edd763740 1 -- 172.21.15.107:0/1229297636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459cbea0 msgr2=0x55d6459d6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.325+0000 7f8edd763740 1 --2- 172.21.15.107:0/1229297636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459cbea0 0x55d6459d6030 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009ca0 tx=0x7f8ec4011810 comp rx=0 tx=0).stop 2024-04-25T00:23:35.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.325+0000 7f8edd763740 1 -- 172.21.15.107:0/1229297636 shutdown_connections 2024-04-25T00:23:35.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.325+0000 7f8edd763740 1 --2- 172.21.15.107:0/1229297636 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6459d6570 0x55d6459d8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.325+0000 7f8edd763740 1 --2- 172.21.15.107:0/1229297636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459cbea0 0x55d6459d6030 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.325+0000 7f8edd763740 1 --2- 172.21.15.107:0/1229297636 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6459cb580 0x55d6459cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.325+0000 7f8edd763740 1 -- 172.21.15.107:0/1229297636 >> 172.21.15.107:0/1229297636 conn(0x55d6459c78a0 msgr2=0x55d6459c7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:35.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.325+0000 7f8edd763740 1 -- 172.21.15.107:0/1229297636 shutdown_connections 2024-04-25T00:23:35.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.325+0000 7f8edd763740 1 -- 172.21.15.107:0/1229297636 wait complete. 2024-04-25T00:23:35.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.325+0000 7f8edd763740 1 -- start start 2024-04-25T00:23:35.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.326+0000 7f8edd763740 1 Processor -- start 2024-04-25T00:23:35.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.326+0000 7f8edd763740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6459cb580 0x55d6459d5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.326+0000 7f8edd763740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6459cbea0 0x55d6459d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.326+0000 7f8edd763740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459d6570 0x55d6459d3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.326+0000 7f8edd763740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d6459d9360 con 0x55d6459d6570 2024-04-25T00:23:35.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.326+0000 7f8edd763740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d6458ec050 con 0x55d6459cbea0 2024-04-25T00:23:35.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.326+0000 7f8edd763740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55d6459d94e0 con 0x55d6459cb580 2024-04-25T00:23:35.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.326+0000 7f8edd763740 10 client.0 _ll_register_callbacks cb 0x55d645643be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:35.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.326+0000 7f8ed37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459d6570 0x55d6459d3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.326+0000 7f8ed37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459d6570 0x55d6459d3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57432/0 (socket says 172.21.15.107:57432) 2024-04-25T00:23:35.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.326+0000 7f8ed37fe640 1 -- 172.21.15.107:0/3366754764 learned_addr learned my addr 172.21.15.107:0/3366754764 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:35.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.326+0000 7f8ed3fff640 1 --2- 172.21.15.107:0/3366754764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6459cbea0 0x55d6459d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.326+0000 7f8ed37fe640 1 -- 172.21.15.107:0/3366754764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6459cb580 msgr2=0x55d6459d5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:35.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.326+0000 7f8ed37fe640 1 --2- 172.21.15.107:0/3366754764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6459cb580 0x55d6459d5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.326+0000 7f8ed37fe640 1 -- 172.21.15.107:0/3366754764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6459cbea0 msgr2=0x55d6459d31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:35.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.326+0000 7f8ed37fe640 1 --2- 172.21.15.107:0/3366754764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6459cbea0 0x55d6459d31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.326+0000 7f8ed37fe640 1 -- 172.21.15.107:0/3366754764 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6458bddc0 con 0x55d6459d6570 2024-04-25T00:23:35.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.327+0000 7f8ed37fe640 1 --2- 172.21.15.107:0/3366754764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459d6570 0x55d6459d3730 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8012c40 tx=0x7f8ec8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205303af52e2420a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:35.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.327+0000 7f8eb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:35.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.327+0000 7f8eb37fe640 1 -- 172.21.15.107:0/3366754764 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8004230 con 0x55d6459d6570 2024-04-25T00:23:35.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.327+0000 7f8eb37fe640 1 -- 172.21.15.107:0/3366754764 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f8ec80043d0 con 0x55d6459d6570 2024-04-25T00:23:35.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.327+0000 7f8edd763740 1 -- 172.21.15.107:0/3366754764 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec4009e50 con 0x55d6459d6570 2024-04-25T00:23:35.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.327+0000 7f8eb37fe640 1 -- 172.21.15.107:0/3366754764 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8ec8027050 con 0x55d6459d6570 2024-04-25T00:23:35.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.327+0000 7f8edd763740 10 client.17238 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:35.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.327+0000 7f8edd763740 1 -- 172.21.15.107:0/3366754764 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d6456443f0 con 0x55d6459d6570 2024-04-25T00:23:35.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.327+0000 7f8eb37fe640 1 -- 172.21.15.107:0/3366754764 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec4009e50 con 0x55d6459d6570 2024-04-25T00:23:35.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.327+0000 7f8eb37fe640 1 -- 172.21.15.107:0/3366754764 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f8ec801e570 con 0x55d6459d6570 2024-04-25T00:23:35.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.327+0000 7f8edd763740 10 client.17238 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:35.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.327+0000 7f8edd763740 10 client.17238 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:23:35.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.327+0000 7f8edd763740 20 client.17238 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:35.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.327+0000 7f8edd763740 1 --2- 172.21.15.107:0/3366754764 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55d6459e6650 0x55d6459e8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.327+0000 7f8edd763740 4 client.17238 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:35.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.327+0000 7f8edd763740 1 -- 172.21.15.107:0/3366754764 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d64593dbb0 con 0x55d6459e6650 2024-04-25T00:23:35.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.327+0000 7f8ed3fff640 1 --2- 172.21.15.107:0/3366754764 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55d6459e6650 0x55d6459e8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.328+0000 7f8ed3fff640 1 --2- 172.21.15.107:0/3366754764 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55d6459e6650 0x55d6459e8a80 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c33865354a1ab90e server_cookie=2ba1d90d6edb5675 in_seq=0 out_seq=0 2024-04-25T00:23:35.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.328+0000 7f8eb37fe640 10 client.17238 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:23:35.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.329+0000 7f8eb37fe640 1 -- 172.21.15.107:0/3366754764 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d64593dbb0 con 0x55d6459e6650 2024-04-25T00:23:35.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.329+0000 7f8eb37fe640 10 client.17238 handle_command_reply: tid=0 2024-04-25T00:23:35.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.329+0000 7f8edd763740 1 client.17238 shutdown 2024-04-25T00:23:35.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.329+0000 7f8edd763740 1 -- 172.21.15.107:0/3366754764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459d6570 msgr2=0x55d6459d3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.329+0000 7f8edd763740 1 --2- 172.21.15.107:0/3366754764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459d6570 0x55d6459d3730 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8012c40 tx=0x7f8ec8011810 comp rx=0 tx=0).stop 2024-04-25T00:23:35.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.329+0000 7f8edd763740 1 -- 172.21.15.107:0/3366754764 shutdown_connections 2024-04-25T00:23:35.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.329+0000 7f8edd763740 1 --2- 172.21.15.107:0/3366754764 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55d6459e6650 0x55d6459e8a80 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.329+0000 7f8edd763740 1 --2- 172.21.15.107:0/3366754764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459d6570 0x55d6459d3730 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.329+0000 7f8edd763740 1 --2- 172.21.15.107:0/3366754764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6459cbea0 0x55d6459d31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.329+0000 7f8eb37fe640 0 client.17238 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:23:35.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.329+0000 7f8edd763740 1 --2- 172.21.15.107:0/3366754764 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6459cb580 0x55d6459d5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.329+0000 7f8edd763740 1 -- 172.21.15.107:0/3366754764 >> 172.21.15.107:0/3366754764 conn(0x55d6459c78a0 msgr2=0x55d6459c6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:35.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.329+0000 7f8edd763740 1 -- 172.21.15.107:0/3366754764 shutdown_connections 2024-04-25T00:23:35.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.329+0000 7f8edd763740 1 -- 172.21.15.107:0/3366754764 wait complete. 2024-04-25T00:23:35.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.329+0000 7f8edd763740 20 client.17238 trim_cache size 0 max 16384 2024-04-25T00:23:35.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.342+0000 7f8edd763740 1 Processor -- start 2024-04-25T00:23:35.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.342+0000 7f8edd763740 1 -- start start 2024-04-25T00:23:35.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.342+0000 7f8edd763740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6459cb580 0x55d6459e2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.342+0000 7f8edd763740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6459cbea0 0x55d6459ee350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.342+0000 7f8edd763740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459d6570 0x55d6459ee890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.342+0000 7f8edd763740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41b15a0 con 0x55d6459d6570 2024-04-25T00:23:35.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.342+0000 7f8edd763740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41aee70 con 0x55d6459cbea0 2024-04-25T00:23:35.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.342+0000 7f8edd763740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41acce0 con 0x55d6459cb580 2024-04-25T00:23:35.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.342+0000 7f8ed37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459d6570 0x55d6459ee890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.342+0000 7f8ed37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459d6570 0x55d6459ee890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57446/0 (socket says 172.21.15.107:57446) 2024-04-25T00:23:35.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.342+0000 7f8ed37fe640 1 -- 172.21.15.107:0/1964186907 learned_addr learned my addr 172.21.15.107:0/1964186907 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:35.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.342+0000 7f8ed37fe640 1 --2- 172.21.15.107:0/1964186907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6459cb580 0x55d6459e2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.343+0000 7f8ed3fff640 1 --2- 172.21.15.107:0/1964186907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6459cbea0 0x55d6459ee350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.343+0000 7f8ed37fe640 1 -- 172.21.15.107:0/1964186907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6459cb580 msgr2=0x55d6459e2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:35.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.343+0000 7f8ed37fe640 1 --2- 172.21.15.107:0/1964186907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6459cb580 0x55d6459e2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.343+0000 7f8ed37fe640 1 -- 172.21.15.107:0/1964186907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6459cbea0 msgr2=0x55d6459ee350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:35.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.343+0000 7f8ed37fe640 1 --2- 172.21.15.107:0/1964186907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6459cbea0 0x55d6459ee350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.343+0000 7f8ed37fe640 1 -- 172.21.15.107:0/1964186907 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d64570e7d0 con 0x55d6459d6570 2024-04-25T00:23:35.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.343+0000 7f8ed37fe640 1 --2- 172.21.15.107:0/1964186907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459d6570 0x55d6459ee890 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f8ec801eae0 tx=0x7f8ec8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32a571cf4d7a32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:35.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.343+0000 7f8eb9ffb640 1 -- 172.21.15.107:0/1964186907 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8011e80 con 0x55d6459d6570 2024-04-25T00:23:35.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.343+0000 7f8eb9ffb640 1 -- 172.21.15.107:0/1964186907 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec8017070 con 0x55d6459d6570 2024-04-25T00:23:35.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.343+0000 7f8eb9ffb640 1 -- 172.21.15.107:0/1964186907 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec801e690 con 0x55d6459d6570 2024-04-25T00:23:35.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.344+0000 7f8edd763740 1 -- 172.21.15.107:0/1964186907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459d6570 msgr2=0x55d6459ee890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.344+0000 7f8edd763740 1 --2- 172.21.15.107:0/1964186907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459d6570 0x55d6459ee890 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f8ec801eae0 tx=0x7f8ec8005740 comp rx=0 tx=0).stop 2024-04-25T00:23:35.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.344+0000 7f8edd763740 1 -- 172.21.15.107:0/1964186907 shutdown_connections 2024-04-25T00:23:35.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.344+0000 7f8edd763740 1 --2- 172.21.15.107:0/1964186907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459d6570 0x55d6459ee890 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.344+0000 7f8edd763740 1 --2- 172.21.15.107:0/1964186907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6459cbea0 0x55d6459ee350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.344+0000 7f8edd763740 1 --2- 172.21.15.107:0/1964186907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6459cb580 0x55d6459e2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.344+0000 7f8edd763740 1 -- 172.21.15.107:0/1964186907 >> 172.21.15.107:0/1964186907 conn(0x55d6459c78a0 msgr2=0x55d6459cd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:35.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.344+0000 7f8edd763740 1 -- 172.21.15.107:0/1964186907 shutdown_connections 2024-04-25T00:23:35.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.344+0000 7f8edd763740 1 -- 172.21.15.107:0/1964186907 wait complete. 2024-04-25T00:23:35.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.344+0000 7f8edd763740 1 -- start start 2024-04-25T00:23:35.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.344+0000 7f8edd763740 1 Processor -- start 2024-04-25T00:23:35.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8edd763740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459cb580 0x55d6459de990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8edd763740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6459cbea0 0x55d6459cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8edd763740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6459d6570 0x55d6459cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8edd763740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41ad330 con 0x55d6459cb580 2024-04-25T00:23:35.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8edd763740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41ae7a0 con 0x55d6459cbea0 2024-04-25T00:23:35.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8edd763740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41ada00 con 0x55d6459d6570 2024-04-25T00:23:35.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8edd763740 10 client.0 _ll_register_callbacks cb 0x7f8ec8009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:35.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8ed37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459cb580 0x55d6459de990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8ed3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6459cbea0 0x55d6459cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8ed37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459cb580 0x55d6459de990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57456/0 (socket says 172.21.15.107:57456) 2024-04-25T00:23:35.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8ed37fe640 1 -- 172.21.15.107:0/1880918598 learned_addr learned my addr 172.21.15.107:0/1880918598 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:35.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8ed37fe640 1 --2- 172.21.15.107:0/1880918598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6459d6570 0x55d6459cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8ed37fe640 1 -- 172.21.15.107:0/1880918598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6459d6570 msgr2=0x55d6459cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:35.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8ed37fe640 1 --2- 172.21.15.107:0/1880918598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6459d6570 0x55d6459cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8ed37fe640 1 -- 172.21.15.107:0/1880918598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6459cbea0 msgr2=0x55d6459cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:35.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8ed37fe640 1 --2- 172.21.15.107:0/1880918598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6459cbea0 0x55d6459cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8ed37fe640 1 -- 172.21.15.107:0/1880918598 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6459d0600 con 0x55d6459cb580 2024-04-25T00:23:35.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8ed37fe640 1 --2- 172.21.15.107:0/1880918598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459cb580 0x55d6459de990 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8005bf0 tx=0x7f8ec8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ccee70cb40bd68e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:35.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8eb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:35.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8eb37fe640 1 -- 172.21.15.107:0/1880918598 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8024040 con 0x55d6459cb580 2024-04-25T00:23:35.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8eb37fe640 1 -- 172.21.15.107:0/1880918598 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f8ec80035f0 con 0x55d6459cb580 2024-04-25T00:23:35.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8edd763740 1 -- 172.21.15.107:0/1880918598 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec8017070 con 0x55d6459cb580 2024-04-25T00:23:35.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8eb37fe640 1 -- 172.21.15.107:0/1880918598 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8ec80037f0 con 0x55d6459cb580 2024-04-25T00:23:35.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8edd763740 10 client.17244 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:35.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.345+0000 7f8edd763740 1 -- 172.21.15.107:0/1880918598 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d645a4ca10 con 0x55d6459cb580 2024-04-25T00:23:35.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.346+0000 7f8eb37fe640 1 -- 172.21.15.107:0/1880918598 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ec8017070 con 0x55d6459cb580 2024-04-25T00:23:35.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.346+0000 7f8eb37fe640 1 -- 172.21.15.107:0/1880918598 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f8ec803c840 con 0x55d6459cb580 2024-04-25T00:23:35.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.346+0000 7f8edd763740 10 client.17244 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:35.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.346+0000 7f8edd763740 10 client.17244 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:23:35.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.346+0000 7f8edd763740 20 client.17244 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:35.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.346+0000 7f8edd763740 1 --2- 172.21.15.107:0/1880918598 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55d645a4d080 0x55d645a4f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.346+0000 7f8edd763740 4 client.17244 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:23:35.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.346+0000 7f8edd763740 1 -- 172.21.15.107:0/1880918598 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d645937f00 con 0x55d645a4d080 2024-04-25T00:23:35.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.346+0000 7f8ed3fff640 1 --2- 172.21.15.107:0/1880918598 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55d645a4d080 0x55d645a4f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.346+0000 7f8eb37fe640 10 client.17244 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:23:35.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.346+0000 7f8ed3fff640 1 --2- 172.21.15.107:0/1880918598 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55d645a4d080 0x55d645a4f4a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ac5d43c18ca4586d server_cookie=b45c3a5fd48ac5f2 in_seq=0 out_seq=0 2024-04-25T00:23:35.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.347+0000 7f8eb37fe640 1 -- 172.21.15.107:0/1880918598 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d645937f00 con 0x55d645a4d080 2024-04-25T00:23:35.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.347+0000 7f8eb37fe640 10 client.17244 handle_command_reply: tid=0 2024-04-25T00:23:35.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.347+0000 7f8edd763740 1 client.17244 shutdown 2024-04-25T00:23:35.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.347+0000 7f8edd763740 1 -- 172.21.15.107:0/1880918598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459cb580 msgr2=0x55d6459de990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.347+0000 7f8edd763740 1 --2- 172.21.15.107:0/1880918598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459cb580 0x55d6459de990 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8005bf0 tx=0x7f8ec8005850 comp rx=0 tx=0).stop 2024-04-25T00:23:35.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.347+0000 7f8edd763740 1 -- 172.21.15.107:0/1880918598 shutdown_connections 2024-04-25T00:23:35.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.347+0000 7f8edd763740 1 --2- 172.21.15.107:0/1880918598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55d6459d6570 0x55d6459cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.347+0000 7f8edd763740 1 --2- 172.21.15.107:0/1880918598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d6459cbea0 0x55d6459cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.347+0000 7f8edd763740 1 --2- 172.21.15.107:0/1880918598 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55d645a4d080 0x55d645a4f4a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.347+0000 7f8edd763740 1 --2- 172.21.15.107:0/1880918598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d6459cb580 0x55d6459de990 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.347+0000 7f8eb37fe640 0 client.17244 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:23:35.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.347+0000 7f8edd763740 1 -- 172.21.15.107:0/1880918598 >> 172.21.15.107:0/1880918598 conn(0x55d6459c78a0 msgr2=0x55d6459cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:35.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.347+0000 7f8edd763740 1 -- 172.21.15.107:0/1880918598 shutdown_connections 2024-04-25T00:23:35.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.347+0000 7f8edd763740 1 -- 172.21.15.107:0/1880918598 wait complete. 2024-04-25T00:23:35.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.347+0000 7f8edd763740 20 client.17244 trim_cache size 0 max 16384 2024-04-25T00:23:35.355 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:23:35.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.347+0000 7f8edaa58640 1 -- 172.21.15.107:0/1640629192 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8ebc0a2d00 msgr2=0x7f8ebc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.347+0000 7f8edaa58640 1 --2- 172.21.15.107:0/1640629192 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8ebc0a2d00 0x7f8ebc0a51c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc0ee0e0 tx=0x7f8ecc08e980 comp rx=0 tx=0).stop 2024-04-25T00:23:35.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.347+0000 7f8edaa58640 1 -- 172.21.15.107:0/1640629192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ed4105e90 msgr2=0x7f8ed419ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.347+0000 7f8edaa58640 1 --2- 172.21.15.107:0/1640629192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ed4105e90 0x7f8ed419ca10 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc031970 tx=0x7f8ecc0319a0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.348+0000 7f8edaa58640 1 -- 172.21.15.107:0/1640629192 shutdown_connections 2024-04-25T00:23:35.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.348+0000 7f8edaa58640 1 --2- 172.21.15.107:0/1640629192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ed4105e90 0x7f8ed419ca10 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.348+0000 7f8edaa58640 1 --2- 172.21.15.107:0/1640629192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ed4100b50 0x7f8ed4198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.348+0000 7f8edaa58640 1 --2- 172.21.15.107:0/1640629192 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f8ebc0a2d00 0x7f8ebc0a51c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.348+0000 7f8edaa58640 1 --2- 172.21.15.107:0/1640629192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ed41001b0 0x7f8ed4198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.348+0000 7f8edaa58640 1 -- 172.21.15.107:0/1640629192 >> 172.21.15.107:0/1640629192 conn(0x7f8ed40767a0 msgr2=0x7f8ed40fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:35.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.348+0000 7f8edaa58640 1 -- 172.21.15.107:0/1640629192 shutdown_connections 2024-04-25T00:23:35.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.348+0000 7f8edaa58640 1 -- 172.21.15.107:0/1640629192 wait complete. 2024-04-25T00:23:35.359 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:23:35.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.458+0000 7fc9d73dc640 1 -- 172.21.15.107:0/2587296041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d0101710 msgr2=0x7fc9d010d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.458+0000 7fc9d73dc640 1 --2- 172.21.15.107:0/2587296041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d0101710 0x7fc9d010d700 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009cd0 tx=0x7fc9c802a8f0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.458+0000 7fc9d73dc640 1 -- 172.21.15.107:0/2587296041 shutdown_connections 2024-04-25T00:23:35.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.458+0000 7fc9d73dc640 1 --2- 172.21.15.107:0/2587296041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d0101710 0x7fc9d010d700 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.458+0000 7fc9d73dc640 1 --2- 172.21.15.107:0/2587296041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d0100d50 0x7fc9d01011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.458+0000 7fc9d73dc640 1 --2- 172.21.15.107:0/2587296041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9d0106840 0x7fc9d0106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.458+0000 7fc9d73dc640 1 -- 172.21.15.107:0/2587296041 >> 172.21.15.107:0/2587296041 conn(0x7fc9d0076c00 msgr2=0x7fc9d00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:35.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.458+0000 7fc9d73dc640 1 -- 172.21.15.107:0/2587296041 shutdown_connections 2024-04-25T00:23:35.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.458+0000 7fc9d73dc640 1 -- 172.21.15.107:0/2587296041 wait complete. 2024-04-25T00:23:35.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.459+0000 7fc9d73dc640 1 Processor -- start 2024-04-25T00:23:35.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.459+0000 7fc9d73dc640 1 -- start start 2024-04-25T00:23:35.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.459+0000 7fc9d73dc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d0100d50 0x7fc9d01970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.459+0000 7fc9d73dc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d0101710 0x7fc9d01975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.459+0000 7fc9d73dc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9d0106840 0x7fc9d019b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.459+0000 7fc9d73dc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0110c50 con 0x7fc9d0101710 2024-04-25T00:23:35.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.459+0000 7fc9d73dc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0110ad0 con 0x7fc9d0106840 2024-04-25T00:23:35.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.459+0000 7fc9d73dc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d0110dd0 con 0x7fc9d0100d50 2024-04-25T00:23:35.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.459+0000 7fc9d4977640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d0101710 0x7fc9d01975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.459+0000 7fc9d4977640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d0101710 0x7fc9d01975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57470/0 (socket says 172.21.15.107:57470) 2024-04-25T00:23:35.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.459+0000 7fc9d4977640 1 -- 172.21.15.107:0/3291471376 learned_addr learned my addr 172.21.15.107:0/3291471376 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:35.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.459+0000 7fc9d5178640 1 --2- 172.21.15.107:0/3291471376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d0100d50 0x7fc9d01970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.459+0000 7fc9d4977640 1 -- 172.21.15.107:0/3291471376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d0100d50 msgr2=0x7fc9d01970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.459+0000 7fc9d5979640 1 --2- 172.21.15.107:0/3291471376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9d0106840 0x7fc9d019b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.459+0000 7fc9d4977640 1 --2- 172.21.15.107:0/3291471376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d0100d50 0x7fc9d01970b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.459+0000 7fc9d4977640 1 -- 172.21.15.107:0/3291471376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9d0106840 msgr2=0x7fc9d019b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.459+0000 7fc9d4977640 1 --2- 172.21.15.107:0/3291471376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9d0106840 0x7fc9d019b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.459+0000 7fc9d4977640 1 -- 172.21.15.107:0/3291471376 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9d019c050 con 0x7fc9d0101710 2024-04-25T00:23:35.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.460+0000 7fc9d4977640 1 --2- 172.21.15.107:0/3291471376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d0101710 0x7fc9d01975f0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000c920 tx=0x7fc9c000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:35.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.460+0000 7fc9c67fc640 1 -- 172.21.15.107:0/3291471376 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0007c70 con 0x7fc9d0101710 2024-04-25T00:23:35.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.460+0000 7fc9c67fc640 1 -- 172.21.15.107:0/3291471376 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9c0007e10 con 0x7fc9d0101710 2024-04-25T00:23:35.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.460+0000 7fc9d73dc640 1 -- 172.21.15.107:0/3291471376 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9d019c340 con 0x7fc9d0101710 2024-04-25T00:23:35.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.460+0000 7fc9c67fc640 1 -- 172.21.15.107:0/3291471376 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0018890 con 0x7fc9d0101710 2024-04-25T00:23:35.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.460+0000 7fc9d73dc640 1 -- 172.21.15.107:0/3291471376 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9d01a3bd0 con 0x7fc9d0101710 2024-04-25T00:23:35.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.462+0000 7fc9c67fc640 1 -- 172.21.15.107:0/3291471376 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc9c0018a70 con 0x7fc9d0101710 2024-04-25T00:23:35.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.462+0000 7fc9da1c8740 1 Processor -- start 2024-04-25T00:23:35.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.462+0000 7fc9da1c8740 1 -- start start 2024-04-25T00:23:35.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.462+0000 7fc9da1c8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8580 0x55863dde8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.462+0000 7fc9da1c8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863dde8ea0 0x55863ddf3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.462+0000 7fc9da1c8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55863ddf3570 0x55863ddf5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.462+0000 7fc9da1c8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55863dd4d670 con 0x55863dde8580 2024-04-25T00:23:35.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.463+0000 7fc9da1c8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55863dcf5970 con 0x55863ddf3570 2024-04-25T00:23:35.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.463+0000 7fc9da1c8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55863dd49630 con 0x55863dde8ea0 2024-04-25T00:23:35.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.463+0000 7fc9c67fc640 1 --2- 172.21.15.107:0/3291471376 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc9bc0a2d00 0x7fc9bc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.463+0000 7fc9d5979640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8580 0x55863dde8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.463+0000 7fc9d5979640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8580 0x55863dde8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57472/0 (socket says 172.21.15.107:57472) 2024-04-25T00:23:35.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.463+0000 7fc9d5979640 1 -- 172.21.15.107:0/1961242276 learned_addr learned my addr 172.21.15.107:0/1961242276 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:35.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.463+0000 7fc9d5178640 1 --2- 172.21.15.107:0/1961242276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863dde8ea0 0x55863ddf3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.463+0000 7fc9d5979640 1 --2- 172.21.15.107:0/3291471376 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc9bc0a2d00 0x7fc9bc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.463+0000 7fc9c67fc640 1 -- 172.21.15.107:0/3291471376 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fc9c00cfa10 con 0x7fc9d0101710 2024-04-25T00:23:35.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.463+0000 7fc9d4977640 1 --2- 172.21.15.107:0/1961242276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55863ddf3570 0x55863ddf5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.463+0000 7fc9d5979640 1 -- 172.21.15.107:0/1961242276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863dde8ea0 msgr2=0x55863ddf3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:35.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.463+0000 7fc9d5979640 1 --2- 172.21.15.107:0/1961242276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863dde8ea0 0x55863ddf3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.463+0000 7fc9d5979640 1 -- 172.21.15.107:0/1961242276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55863ddf3570 msgr2=0x55863ddf5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:35.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.463+0000 7fc9d5979640 1 --2- 172.21.15.107:0/1961242276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55863ddf3570 0x55863ddf5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.463+0000 7fc9d5979640 1 -- 172.21.15.107:0/1961242276 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55863dd19400 con 0x55863dde8580 2024-04-25T00:23:35.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.463+0000 7fc9d5979640 1 --2- 172.21.15.107:0/1961242276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8580 0x55863dde8960 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8030ae0 tx=0x7fc9c8030b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b906b3ade9f9493 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:35.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.463+0000 7fc9d5979640 1 --2- 172.21.15.107:0/3291471376 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc9bc0a2d00 0x7fc9bc0a51c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc9c802ae10 tx=0x7fc9c8042000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:35.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.463+0000 7fc9bb7fe640 1 -- 172.21.15.107:0/1961242276 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c804d070 con 0x55863dde8580 2024-04-25T00:23:35.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.463+0000 7fc9bb7fe640 1 -- 172.21.15.107:0/1961242276 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9c8036050 con 0x55863dde8580 2024-04-25T00:23:35.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.463+0000 7fc9bb7fe640 1 -- 172.21.15.107:0/1961242276 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80043a0 con 0x55863dde8580 2024-04-25T00:23:35.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.464+0000 7fc9da1c8740 1 -- 172.21.15.107:0/1961242276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8580 msgr2=0x55863dde8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.464+0000 7fc9da1c8740 1 --2- 172.21.15.107:0/1961242276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8580 0x55863dde8960 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8030ae0 tx=0x7fc9c8030b10 comp rx=0 tx=0).stop 2024-04-25T00:23:35.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.464+0000 7fc9da1c8740 1 -- 172.21.15.107:0/1961242276 shutdown_connections 2024-04-25T00:23:35.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.464+0000 7fc9da1c8740 1 --2- 172.21.15.107:0/1961242276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55863ddf3570 0x55863ddf5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.464+0000 7fc9da1c8740 1 --2- 172.21.15.107:0/1961242276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863dde8ea0 0x55863ddf3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.464+0000 7fc9da1c8740 1 --2- 172.21.15.107:0/1961242276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8580 0x55863dde8960 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.464+0000 7fc9da1c8740 1 -- 172.21.15.107:0/1961242276 >> 172.21.15.107:0/1961242276 conn(0x55863dde48a0 msgr2=0x55863dde4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:35.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.464+0000 7fc9da1c8740 1 -- 172.21.15.107:0/1961242276 shutdown_connections 2024-04-25T00:23:35.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.464+0000 7fc9da1c8740 1 -- 172.21.15.107:0/1961242276 wait complete. 2024-04-25T00:23:35.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.464+0000 7fc9da1c8740 1 -- start start 2024-04-25T00:23:35.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.464+0000 7fc9da1c8740 1 Processor -- start 2024-04-25T00:23:35.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.465+0000 7fc9da1c8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55863dde8580 0x55863ddfb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.465+0000 7fc9da1c8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8ea0 0x55863ddfb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.465+0000 7fc9da1c8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ddf3570 0x55863ddf24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.465+0000 7fc9da1c8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55863ddf6300 con 0x55863dde8ea0 2024-04-25T00:23:35.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.465+0000 7fc9da1c8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55863dd09050 con 0x55863dde8580 2024-04-25T00:23:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.465+0000 7fc9da1c8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55863ddf6480 con 0x55863ddf3570 2024-04-25T00:23:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.465+0000 7fc9da1c8740 10 client.0 _ll_register_callbacks cb 0x55863da60be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.465+0000 7fc9d4977640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8ea0 0x55863ddfb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.465+0000 7fc9d4977640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8ea0 0x55863ddfb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57474/0 (socket says 172.21.15.107:57474) 2024-04-25T00:23:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.465+0000 7fc9d4977640 1 -- 172.21.15.107:0/1224630539 learned_addr learned my addr 172.21.15.107:0/1224630539 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.465+0000 7fc9d5178640 1 --2- 172.21.15.107:0/1224630539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55863dde8580 0x55863ddfb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.465+0000 7fc9d5979640 1 --2- 172.21.15.107:0/1224630539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ddf3570 0x55863ddf24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.465+0000 7fc9d4977640 1 -- 172.21.15.107:0/1224630539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ddf3570 msgr2=0x55863ddf24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.465+0000 7fc9d4977640 1 --2- 172.21.15.107:0/1224630539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ddf3570 0x55863ddf24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.465+0000 7fc9d4977640 1 -- 172.21.15.107:0/1224630539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55863dde8580 msgr2=0x55863ddfb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.465+0000 7fc9d4977640 1 --2- 172.21.15.107:0/1224630539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55863dde8580 0x55863ddfb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.465+0000 7fc9d4977640 1 -- 172.21.15.107:0/1224630539 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55863dcdadc0 con 0x55863dde8ea0 2024-04-25T00:23:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.465+0000 7fc9d5178640 1 --2- 172.21.15.107:0/1224630539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55863dde8580 0x55863ddfb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:35.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.465+0000 7fc9d4977640 1 --2- 172.21.15.107:0/1224630539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8ea0 0x55863ddfb820 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0097fd0 tx=0x7fc9c009b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a9d1728b44420cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.465+0000 7fc9b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.466+0000 7fc9b8ff9640 1 -- 172.21.15.107:0/1224630539 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c000fb30 con 0x55863dde8ea0 2024-04-25T00:23:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.466+0000 7fc9b8ff9640 1 -- 172.21.15.107:0/1224630539 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fc9c001dd40 con 0x55863dde8ea0 2024-04-25T00:23:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.466+0000 7fc9da1c8740 1 -- 172.21.15.107:0/1224630539 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9c8036050 con 0x55863dde8ea0 2024-04-25T00:23:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.466+0000 7fc9b8ff9640 1 -- 172.21.15.107:0/1224630539 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc9c0064050 con 0x55863dde8ea0 2024-04-25T00:23:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.466+0000 7fc9da1c8740 10 client.17256 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.466+0000 7fc9da1c8740 1 -- 172.21.15.107:0/1224630539 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55863da613f0 con 0x55863dde8ea0 2024-04-25T00:23:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.466+0000 7fc9b8ff9640 1 -- 172.21.15.107:0/1224630539 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9c8036050 con 0x55863dde8ea0 2024-04-25T00:23:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.466+0000 7fc9b8ff9640 1 -- 172.21.15.107:0/1224630539 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fc9c00981e0 con 0x55863dde8ea0 2024-04-25T00:23:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.466+0000 7fc9da1c8740 10 client.17256 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.466+0000 7fc9da1c8740 10 client.17256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.466+0000 7fc9da1c8740 20 client.17256 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.466+0000 7fc9da1c8740 1 --2- 172.21.15.107:0/1224630539 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55863de05870 0x55863de07c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.466+0000 7fc9da1c8740 4 client.17256 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:35.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.466+0000 7fc9da1c8740 1 -- 172.21.15.107:0/1224630539 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55863dd5abb0 con 0x55863de05870 2024-04-25T00:23:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.466+0000 7fc9d5178640 1 --2- 172.21.15.107:0/1224630539 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55863de05870 0x55863de07c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.467+0000 7fc9b8ff9640 10 client.17256 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.467+0000 7fc9d5178640 1 --2- 172.21.15.107:0/1224630539 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55863de05870 0x55863de07c60 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84a9b925b2d008f6 server_cookie=dc0000ae985d31be in_seq=0 out_seq=0 2024-04-25T00:23:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.468+0000 7fc9b8ff9640 1 -- 172.21.15.107:0/1224630539 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55863dd5abb0 con 0x55863de05870 2024-04-25T00:23:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.468+0000 7fc9b8ff9640 10 client.17256 handle_command_reply: tid=0 2024-04-25T00:23:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.468+0000 7fc9da1c8740 1 client.17256 shutdown 2024-04-25T00:23:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.468+0000 7fc9da1c8740 1 -- 172.21.15.107:0/1224630539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8ea0 msgr2=0x55863ddfb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.468+0000 7fc9da1c8740 1 --2- 172.21.15.107:0/1224630539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8ea0 0x55863ddfb820 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0097fd0 tx=0x7fc9c009b590 comp rx=0 tx=0).stop 2024-04-25T00:23:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.468+0000 7fc9da1c8740 1 -- 172.21.15.107:0/1224630539 shutdown_connections 2024-04-25T00:23:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.468+0000 7fc9da1c8740 1 --2- 172.21.15.107:0/1224630539 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55863de05870 0x55863de07c60 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.468+0000 7fc9da1c8740 1 --2- 172.21.15.107:0/1224630539 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ddf3570 0x55863ddf24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.468+0000 7fc9da1c8740 1 --2- 172.21.15.107:0/1224630539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8ea0 0x55863ddfb820 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.468+0000 7fc9b8ff9640 0 client.17256 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:35.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.468+0000 7fc9da1c8740 1 --2- 172.21.15.107:0/1224630539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55863dde8580 0x55863ddfb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.468+0000 7fc9da1c8740 1 -- 172.21.15.107:0/1224630539 >> 172.21.15.107:0/1224630539 conn(0x55863dde48a0 msgr2=0x55863ddea6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:35.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.468+0000 7fc9da1c8740 1 -- 172.21.15.107:0/1224630539 shutdown_connections 2024-04-25T00:23:35.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.468+0000 7fc9da1c8740 1 -- 172.21.15.107:0/1224630539 wait complete. 2024-04-25T00:23:35.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.468+0000 7fc9da1c8740 20 client.17256 trim_cache size 0 max 16384 2024-04-25T00:23:35.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.481+0000 7fc9da1c8740 1 Processor -- start 2024-04-25T00:23:35.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.481+0000 7fc9da1c8740 1 -- start start 2024-04-25T00:23:35.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.481+0000 7fc9da1c8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8580 0x55863de0d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.481+0000 7fc9da1c8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55863dde8ea0 0x55863de0d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.481+0000 7fc9da1c8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ddf3570 0x55863de014f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.481+0000 7fc9da1c8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01af250 con 0x55863dde8580 2024-04-25T00:23:35.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.481+0000 7fc9da1c8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01acb20 con 0x55863dde8ea0 2024-04-25T00:23:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.481+0000 7fc9da1c8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01aa990 con 0x55863ddf3570 2024-04-25T00:23:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.481+0000 7fc9d5178640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8580 0x55863de0d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.481+0000 7fc9d5178640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8580 0x55863de0d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57482/0 (socket says 172.21.15.107:57482) 2024-04-25T00:23:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.481+0000 7fc9d5178640 1 -- 172.21.15.107:0/3033315701 learned_addr learned my addr 172.21.15.107:0/3033315701 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.482+0000 7fc9d4977640 1 --2- 172.21.15.107:0/3033315701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55863dde8ea0 0x55863de0d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.482+0000 7fc9d5178640 1 -- 172.21.15.107:0/3033315701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ddf3570 msgr2=0x55863de014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.482+0000 7fc9d5979640 1 --2- 172.21.15.107:0/3033315701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ddf3570 0x55863de014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.482+0000 7fc9d5178640 1 --2- 172.21.15.107:0/3033315701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ddf3570 0x55863de014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.482+0000 7fc9d5178640 1 -- 172.21.15.107:0/3033315701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55863dde8ea0 msgr2=0x55863de0d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.482+0000 7fc9d5178640 1 --2- 172.21.15.107:0/3033315701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55863dde8ea0 0x55863de0d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.482+0000 7fc9d5178640 1 -- 172.21.15.107:0/3033315701 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55863db2b7d0 con 0x55863dde8580 2024-04-25T00:23:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.482+0000 7fc9d5178640 1 --2- 172.21.15.107:0/3033315701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8580 0x55863de0d470 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x55863de083e0 tx=0x7fc9cc014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c47cd3b6ac82af4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.482+0000 7fc9baffd640 1 -- 172.21.15.107:0/3033315701 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc0142d0 con 0x55863dde8580 2024-04-25T00:23:35.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.482+0000 7fc9baffd640 1 -- 172.21.15.107:0/3033315701 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9cc01e530 con 0x55863dde8580 2024-04-25T00:23:35.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.482+0000 7fc9baffd640 1 -- 172.21.15.107:0/3033315701 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc012a30 con 0x55863dde8580 2024-04-25T00:23:35.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.482+0000 7fc9da1c8740 1 -- 172.21.15.107:0/3033315701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8580 msgr2=0x55863de0d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.482+0000 7fc9da1c8740 1 --2- 172.21.15.107:0/3033315701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8580 0x55863de0d470 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x55863de083e0 tx=0x7fc9cc014840 comp rx=0 tx=0).stop 2024-04-25T00:23:35.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.482+0000 7fc9da1c8740 1 -- 172.21.15.107:0/3033315701 shutdown_connections 2024-04-25T00:23:35.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.482+0000 7fc9da1c8740 1 --2- 172.21.15.107:0/3033315701 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ddf3570 0x55863de014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.482+0000 7fc9da1c8740 1 --2- 172.21.15.107:0/3033315701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55863dde8ea0 0x55863de0d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.482+0000 7fc9da1c8740 1 --2- 172.21.15.107:0/3033315701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8580 0x55863de0d470 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.482+0000 7fc9da1c8740 1 -- 172.21.15.107:0/3033315701 >> 172.21.15.107:0/3033315701 conn(0x55863dde48a0 msgr2=0x55863ddeeb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:35.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.483+0000 7fc9da1c8740 1 -- 172.21.15.107:0/3033315701 shutdown_connections 2024-04-25T00:23:35.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.483+0000 7fc9da1c8740 1 -- 172.21.15.107:0/3033315701 wait complete. 2024-04-25T00:23:35.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.483+0000 7fc9da1c8740 1 -- start start 2024-04-25T00:23:35.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.483+0000 7fc9da1c8740 1 Processor -- start 2024-04-25T00:23:35.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.483+0000 7fc9da1c8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8580 0x55863ddf1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.483+0000 7fc9da1c8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55863dde8ea0 0x55863ddf19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.483+0000 7fc9da1c8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ddf3570 0x55863de65290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.483+0000 7fc9da1c8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01aafe0 con 0x55863dde8580 2024-04-25T00:23:35.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.483+0000 7fc9da1c8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01ac450 con 0x55863dde8ea0 2024-04-25T00:23:35.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.483+0000 7fc9da1c8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc9d01ab6b0 con 0x55863ddf3570 2024-04-25T00:23:35.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.483+0000 7fc9da1c8740 10 client.0 _ll_register_callbacks cb 0x7fc9c000fd50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:35.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.483+0000 7fc9d5178640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8580 0x55863ddf1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.483+0000 7fc9d5178640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8580 0x55863ddf1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57492/0 (socket says 172.21.15.107:57492) 2024-04-25T00:23:35.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.483+0000 7fc9d5178640 1 -- 172.21.15.107:0/2805777809 learned_addr learned my addr 172.21.15.107:0/2805777809 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:35.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.483+0000 7fc9d4977640 1 --2- 172.21.15.107:0/2805777809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55863dde8ea0 0x55863ddf19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.483+0000 7fc9d5979640 1 --2- 172.21.15.107:0/2805777809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ddf3570 0x55863de65290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.484+0000 7fc9d5178640 1 -- 172.21.15.107:0/2805777809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ddf3570 msgr2=0x55863de65290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:35.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.484+0000 7fc9d5178640 1 --2- 172.21.15.107:0/2805777809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ddf3570 0x55863de65290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.484+0000 7fc9d5178640 1 -- 172.21.15.107:0/2805777809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55863dde8ea0 msgr2=0x55863ddf19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:35.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.484+0000 7fc9d5178640 1 --2- 172.21.15.107:0/2805777809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55863dde8ea0 0x55863ddf19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.484+0000 7fc9d5178640 1 -- 172.21.15.107:0/2805777809 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55863de65900 con 0x55863dde8580 2024-04-25T00:23:35.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.484+0000 7fc9d5178640 1 --2- 172.21.15.107:0/2805777809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8580 0x55863ddf1460 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc005d40 tx=0x7fc9cc0099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a845d08e5dedc74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:35.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.484+0000 7fc9b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:35.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.484+0000 7fc9b8ff9640 1 -- 172.21.15.107:0/2805777809 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc035060 con 0x55863dde8580 2024-04-25T00:23:35.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.484+0000 7fc9b8ff9640 1 -- 172.21.15.107:0/2805777809 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fc9cc0140d0 con 0x55863dde8580 2024-04-25T00:23:35.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.484+0000 7fc9da1c8740 1 -- 172.21.15.107:0/2805777809 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9cc01e530 con 0x55863dde8580 2024-04-25T00:23:35.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.484+0000 7fc9b8ff9640 1 -- 172.21.15.107:0/2805777809 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc9cc0142d0 con 0x55863dde8580 2024-04-25T00:23:35.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.484+0000 7fc9da1c8740 10 client.17262 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:35.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.484+0000 7fc9da1c8740 1 -- 172.21.15.107:0/2805777809 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55863de6dea0 con 0x55863dde8580 2024-04-25T00:23:35.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.484+0000 7fc9b8ff9640 1 -- 172.21.15.107:0/2805777809 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9cc01e530 con 0x55863dde8580 2024-04-25T00:23:35.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.484+0000 7fc9b8ff9640 1 -- 172.21.15.107:0/2805777809 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fc9cc00f040 con 0x55863dde8580 2024-04-25T00:23:35.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.484+0000 7fc9da1c8740 10 client.17262 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:35.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.484+0000 7fc9da1c8740 10 client.17262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:35.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.484+0000 7fc9da1c8740 20 client.17262 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.484+0000 7fc9da1c8740 1 --2- 172.21.15.107:0/2805777809 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55863de6e5c0 0x55863de709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.484+0000 7fc9da1c8740 4 client.17262 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:23:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.484+0000 7fc9da1c8740 1 -- 172.21.15.107:0/2805777809 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55863dd54f00 con 0x55863de6e5c0 2024-04-25T00:23:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.485+0000 7fc9d4977640 1 --2- 172.21.15.107:0/2805777809 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55863de6e5c0 0x55863de709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.485+0000 7fc9d4977640 1 --2- 172.21.15.107:0/2805777809 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55863de6e5c0 0x55863de709e0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26ceca3f89459ff6 server_cookie=63613a2efae28cc5 in_seq=0 out_seq=0 2024-04-25T00:23:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.485+0000 7fc9b8ff9640 10 client.17262 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.485+0000 7fc9b8ff9640 1 -- 172.21.15.107:0/2805777809 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55863dd54f00 con 0x55863de6e5c0 2024-04-25T00:23:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.485+0000 7fc9b8ff9640 10 client.17262 handle_command_reply: tid=0 2024-04-25T00:23:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.485+0000 7fc9da1c8740 1 client.17262 shutdown 2024-04-25T00:23:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.485+0000 7fc9da1c8740 1 -- 172.21.15.107:0/2805777809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8580 msgr2=0x55863ddf1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.485+0000 7fc9da1c8740 1 --2- 172.21.15.107:0/2805777809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8580 0x55863ddf1460 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc005d40 tx=0x7fc9cc0099f0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.485+0000 7fc9da1c8740 1 -- 172.21.15.107:0/2805777809 shutdown_connections 2024-04-25T00:23:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.485+0000 7fc9da1c8740 1 --2- 172.21.15.107:0/2805777809 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55863de6e5c0 0x55863de709e0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.485+0000 7fc9da1c8740 1 --2- 172.21.15.107:0/2805777809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55863ddf3570 0x55863de65290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.485+0000 7fc9da1c8740 1 --2- 172.21.15.107:0/2805777809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55863dde8ea0 0x55863ddf19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.485+0000 7fc9b8ff9640 0 client.17262 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:35.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.485+0000 7fc9da1c8740 1 --2- 172.21.15.107:0/2805777809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55863dde8580 0x55863ddf1460 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.485+0000 7fc9da1c8740 1 -- 172.21.15.107:0/2805777809 >> 172.21.15.107:0/2805777809 conn(0x55863dde48a0 msgr2=0x55863ddfaea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:35.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.486+0000 7fc9da1c8740 1 -- 172.21.15.107:0/2805777809 shutdown_connections 2024-04-25T00:23:35.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.486+0000 7fc9da1c8740 1 -- 172.21.15.107:0/2805777809 wait complete. 2024-04-25T00:23:35.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.486+0000 7fc9da1c8740 20 client.17262 trim_cache size 0 max 16384 2024-04-25T00:23:35.494 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:23:35.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.486+0000 7fc9d73dc640 1 -- 172.21.15.107:0/3291471376 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc9bc0a2d00 msgr2=0x7fc9bc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.486+0000 7fc9d73dc640 1 --2- 172.21.15.107:0/3291471376 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc9bc0a2d00 0x7fc9bc0a51c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc9c802ae10 tx=0x7fc9c8042000 comp rx=0 tx=0).stop 2024-04-25T00:23:35.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.486+0000 7fc9d73dc640 1 -- 172.21.15.107:0/3291471376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d0101710 msgr2=0x7fc9d01975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:35.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.486+0000 7fc9d73dc640 1 --2- 172.21.15.107:0/3291471376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d0101710 0x7fc9d01975f0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000c920 tx=0x7fc9c000cde0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.486+0000 7fc9d73dc640 1 -- 172.21.15.107:0/3291471376 shutdown_connections 2024-04-25T00:23:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.486+0000 7fc9d73dc640 1 --2- 172.21.15.107:0/3291471376 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc9bc0a2d00 0x7fc9bc0a51c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.486+0000 7fc9d73dc640 1 --2- 172.21.15.107:0/3291471376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc9d0106840 0x7fc9d019b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.486+0000 7fc9d73dc640 1 --2- 172.21.15.107:0/3291471376 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9d0101710 0x7fc9d01975f0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.486+0000 7fc9d73dc640 1 --2- 172.21.15.107:0/3291471376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d0100d50 0x7fc9d01970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.486+0000 7fc9d73dc640 1 -- 172.21.15.107:0/3291471376 >> 172.21.15.107:0/3291471376 conn(0x7fc9d0076c00 msgr2=0x7fc9d0103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.486+0000 7fc9d73dc640 1 -- 172.21.15.107:0/3291471376 shutdown_connections 2024-04-25T00:23:35.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:35.486+0000 7fc9d73dc640 1 -- 172.21.15.107:0/3291471376 wait complete. 2024-04-25T00:23:35.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:35 smithi170 ceph-mon[32807]: pgmap v219: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:23:35.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:35 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1252755428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:23:35.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:35 smithi107 ceph-mon[29153]: pgmap v219: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:23:35.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:35 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1252755428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:23:36.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:35 smithi195 ceph-mon[32833]: pgmap v219: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:23:36.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:35 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1252755428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:23:36.498 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:23:36.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:23:36.498 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:23:36.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.613+0000 7f6674d16640 1 -- 172.21.15.107:0/360789823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6670101760 msgr2=0x7f6670101be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:36.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.613+0000 7f6674d16640 1 --2- 172.21.15.107:0/360789823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6670101760 0x7f6670101be0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f6664009930 tx=0x7f666402a420 comp rx=0 tx=0).stop 2024-04-25T00:23:36.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.613+0000 7f6674d16640 1 -- 172.21.15.107:0/360789823 shutdown_connections 2024-04-25T00:23:36.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.613+0000 7f6674d16640 1 --2- 172.21.15.107:0/360789823 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6670102120 0x7f667010e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.613+0000 7f6674d16640 1 --2- 172.21.15.107:0/360789823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6670101760 0x7f6670101be0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.613+0000 7f6674d16640 1 --2- 172.21.15.107:0/360789823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6670107770 0x7f6670107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.613+0000 7f6674d16640 1 -- 172.21.15.107:0/360789823 >> 172.21.15.107:0/360789823 conn(0x7f66700fd250 msgr2=0x7f66700ff670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:36.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.613+0000 7f6674d16640 1 -- 172.21.15.107:0/360789823 shutdown_connections 2024-04-25T00:23:36.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.613+0000 7f6674d16640 1 -- 172.21.15.107:0/360789823 wait complete. 2024-04-25T00:23:36.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.613+0000 7f6674d16640 1 Processor -- start 2024-04-25T00:23:36.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.614+0000 7f6674d16640 1 -- start start 2024-04-25T00:23:36.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.614+0000 7f6674d16640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6670101760 0x7f6670073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.614+0000 7f6674d16640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6670102120 0x7f6670073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.614+0000 7f6674d16640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6670107770 0x7f66700744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.614+0000 7f6674d16640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6670111b80 con 0x7f6670107770 2024-04-25T00:23:36.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.614+0000 7f6674d16640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6670111a00 con 0x7f6670101760 2024-04-25T00:23:36.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.614+0000 7f6674d16640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6670111d00 con 0x7f6670102120 2024-04-25T00:23:36.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.614+0000 7f666ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6670107770 0x7f66700744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.614+0000 7f666ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6670107770 0x7f66700744c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57508/0 (socket says 172.21.15.107:57508) 2024-04-25T00:23:36.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.614+0000 7f666ed9d640 1 -- 172.21.15.107:0/2339752349 learned_addr learned my addr 172.21.15.107:0/2339752349 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:36.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.614+0000 7f666dd9b640 1 --2- 172.21.15.107:0/2339752349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6670102120 0x7f6670073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.614+0000 7f666e59c640 1 --2- 172.21.15.107:0/2339752349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6670101760 0x7f6670073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.614+0000 7f666ed9d640 1 -- 172.21.15.107:0/2339752349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6670102120 msgr2=0x7f6670073f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:36.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.614+0000 7f666ed9d640 1 --2- 172.21.15.107:0/2339752349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6670102120 0x7f6670073f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.614+0000 7f666ed9d640 1 -- 172.21.15.107:0/2339752349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6670101760 msgr2=0x7f6670073a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:36.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.614+0000 7f666ed9d640 1 --2- 172.21.15.107:0/2339752349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6670101760 0x7f6670073a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.614+0000 7f666ed9d640 1 -- 172.21.15.107:0/2339752349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66701a6600 con 0x7f6670107770 2024-04-25T00:23:36.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.614+0000 7f666e59c640 1 --2- 172.21.15.107:0/2339752349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6670101760 0x7f6670073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:36.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.614+0000 7f666ed9d640 1 --2- 172.21.15.107:0/2339752349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6670107770 0x7f66700744c0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f666800b970 tx=0x7f666800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:36.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.615+0000 7f665f7fe640 1 -- 172.21.15.107:0/2339752349 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668019070 con 0x7f6670107770 2024-04-25T00:23:36.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.615+0000 7f6674d16640 1 -- 172.21.15.107:0/2339752349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66701a68f0 con 0x7f6670107770 2024-04-25T00:23:36.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.615+0000 7f665f7fe640 1 -- 172.21.15.107:0/2339752349 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f666800cb40 con 0x7f6670107770 2024-04-25T00:23:36.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.615+0000 7f665f7fe640 1 -- 172.21.15.107:0/2339752349 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668007650 con 0x7f6670107770 2024-04-25T00:23:36.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.615+0000 7f6674d16640 1 -- 172.21.15.107:0/2339752349 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66701a6e10 con 0x7f6670107770 2024-04-25T00:23:36.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.617+0000 7f665f7fe640 1 -- 172.21.15.107:0/2339752349 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f666801c070 con 0x7f6670107770 2024-04-25T00:23:36.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.617+0000 7f6677a1a740 1 Processor -- start 2024-04-25T00:23:36.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.617+0000 7f6677a1a740 1 -- start start 2024-04-25T00:23:36.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.617+0000 7f6677a1a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56046c3ef980 0x56046c3edcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.617+0000 7f6677a1a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56046c3ee290 0x56046c3ee710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.618+0000 7f6677a1a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3f9010 0x56046c3fb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.618+0000 7f665f7fe640 1 --2- 172.21.15.107:0/2339752349 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f66580a2d00 0x7f66580a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.618+0000 7f6677a1a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56046c3522c0 con 0x56046c3f9010 2024-04-25T00:23:36.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.618+0000 7f6677a1a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56046c2fa910 con 0x56046c3ef980 2024-04-25T00:23:36.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.618+0000 7f6677a1a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56046c34e280 con 0x56046c3ee290 2024-04-25T00:23:36.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.618+0000 7f666dd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3f9010 0x56046c3fb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.618+0000 7f666dd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3f9010 0x56046c3fb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57518/0 (socket says 172.21.15.107:57518) 2024-04-25T00:23:36.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.618+0000 7f666ed9d640 1 --2- 172.21.15.107:0/2339752349 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f66580a2d00 0x7f66580a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.618+0000 7f666dd9b640 1 -- 172.21.15.107:0/2926504274 learned_addr learned my addr 172.21.15.107:0/2926504274 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:36.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.618+0000 7f666e59c640 1 --2- 172.21.15.107:0/2926504274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56046c3ee290 0x56046c3ee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.618+0000 7f666dd9b640 1 --2- 172.21.15.107:0/2926504274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56046c3ef980 0x56046c3edcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.618+0000 7f665f7fe640 1 -- 172.21.15.107:0/2339752349 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f66680ced70 con 0x7f6670107770 2024-04-25T00:23:36.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.618+0000 7f666dd9b640 1 -- 172.21.15.107:0/2926504274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56046c3ee290 msgr2=0x56046c3ee710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:36.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.618+0000 7f666dd9b640 1 --2- 172.21.15.107:0/2926504274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56046c3ee290 0x56046c3ee710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.618+0000 7f666dd9b640 1 -- 172.21.15.107:0/2926504274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56046c3ef980 msgr2=0x56046c3edcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:36.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.618+0000 7f666dd9b640 1 --2- 172.21.15.107:0/2926504274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56046c3ef980 0x56046c3edcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.618+0000 7f666dd9b640 1 -- 172.21.15.107:0/2926504274 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56046c30bfb0 con 0x56046c3f9010 2024-04-25T00:23:36.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.618+0000 7f666dd9b640 1 --2- 172.21.15.107:0/2926504274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3f9010 0x56046c3fb400 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f6664005940 tx=0x7f66640058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26ac7af170ef960 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:36.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.618+0000 7f666ed9d640 1 --2- 172.21.15.107:0/2339752349 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f66580a2d00 0x7f66580a51c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f66680979b0 tx=0x7f66680c5b70 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:36.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.618+0000 7f665cff9640 1 -- 172.21.15.107:0/2926504274 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664043030 con 0x56046c3f9010 2024-04-25T00:23:36.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.618+0000 7f665cff9640 1 -- 172.21.15.107:0/2926504274 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6664031750 con 0x56046c3f9010 2024-04-25T00:23:36.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.618+0000 7f665cff9640 1 -- 172.21.15.107:0/2926504274 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664035040 con 0x56046c3f9010 2024-04-25T00:23:36.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.619+0000 7f6677a1a740 1 -- 172.21.15.107:0/2926504274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3f9010 msgr2=0x56046c3fb400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:36.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.619+0000 7f6677a1a740 1 --2- 172.21.15.107:0/2926504274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3f9010 0x56046c3fb400 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f6664005940 tx=0x7f66640058d0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.619+0000 7f6677a1a740 1 -- 172.21.15.107:0/2926504274 shutdown_connections 2024-04-25T00:23:36.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.619+0000 7f6677a1a740 1 --2- 172.21.15.107:0/2926504274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3f9010 0x56046c3fb400 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.619+0000 7f6677a1a740 1 --2- 172.21.15.107:0/2926504274 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56046c3ee290 0x56046c3ee710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.619+0000 7f6677a1a740 1 --2- 172.21.15.107:0/2926504274 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56046c3ef980 0x56046c3edcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.619+0000 7f6677a1a740 1 -- 172.21.15.107:0/2926504274 >> 172.21.15.107:0/2926504274 conn(0x56046c3b6950 msgr2=0x56046c3ea7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:36.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.619+0000 7f6677a1a740 1 -- 172.21.15.107:0/2926504274 shutdown_connections 2024-04-25T00:23:36.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.619+0000 7f6677a1a740 1 -- 172.21.15.107:0/2926504274 wait complete. 2024-04-25T00:23:36.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.619+0000 7f6677a1a740 1 -- start start 2024-04-25T00:23:36.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.620+0000 7f6677a1a740 1 Processor -- start 2024-04-25T00:23:36.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.620+0000 7f6677a1a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3ee290 0x56046c3f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.620+0000 7f6677a1a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56046c3ef980 0x56046c3f6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.620+0000 7f6677a1a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56046c3f9010 0x56046c3f6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.620+0000 7f6677a1a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56046c3fbe40 con 0x56046c3ee290 2024-04-25T00:23:36.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.620+0000 7f6677a1a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56046c30de90 con 0x56046c3ef980 2024-04-25T00:23:36.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.620+0000 7f6677a1a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56046c3fbfc0 con 0x56046c3f9010 2024-04-25T00:23:36.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.620+0000 7f6677a1a740 10 client.0 _ll_register_callbacks cb 0x56046c349970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:36.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.620+0000 7f666dd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3ee290 0x56046c3f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.620+0000 7f666dd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3ee290 0x56046c3f5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57534/0 (socket says 172.21.15.107:57534) 2024-04-25T00:23:36.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.620+0000 7f666dd9b640 1 -- 172.21.15.107:0/606529472 learned_addr learned my addr 172.21.15.107:0/606529472 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:36.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.620+0000 7f666e59c640 1 --2- 172.21.15.107:0/606529472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56046c3ef980 0x56046c3f6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.620+0000 7f666dd9b640 1 --2- 172.21.15.107:0/606529472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56046c3f9010 0x56046c3f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.620+0000 7f666dd9b640 1 -- 172.21.15.107:0/606529472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56046c3f9010 msgr2=0x56046c3f6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:36.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.620+0000 7f666dd9b640 1 --2- 172.21.15.107:0/606529472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56046c3f9010 0x56046c3f6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.620+0000 7f666dd9b640 1 -- 172.21.15.107:0/606529472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56046c3ef980 msgr2=0x56046c3f6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:36.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.620+0000 7f666dd9b640 1 --2- 172.21.15.107:0/606529472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56046c3ef980 0x56046c3f6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.620+0000 7f666dd9b640 1 -- 172.21.15.107:0/606529472 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56046c12e7e0 con 0x56046c3ee290 2024-04-25T00:23:36.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.620+0000 7f666dd9b640 1 --2- 172.21.15.107:0/606529472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3ee290 0x56046c3f5f10 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f6664031550 tx=0x7f6664005360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73c801ced0faf1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:36.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.620+0000 7f6655ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:36.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.620+0000 7f6655ffb640 1 -- 172.21.15.107:0/606529472 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664035070 con 0x56046c3ee290 2024-04-25T00:23:36.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.621+0000 7f6655ffb640 1 -- 172.21.15.107:0/606529472 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f6664004bb0 con 0x56046c3ee290 2024-04-25T00:23:36.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.621+0000 7f6677a1a740 1 -- 172.21.15.107:0/606529472 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6664031750 con 0x56046c3ee290 2024-04-25T00:23:36.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.621+0000 7f6655ffb640 1 -- 172.21.15.107:0/606529472 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6664059050 con 0x56046c3ee290 2024-04-25T00:23:36.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.621+0000 7f6677a1a740 10 client.17274 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:36.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.621+0000 7f6677a1a740 1 -- 172.21.15.107:0/606529472 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56046c18ef70 con 0x56046c3ee290 2024-04-25T00:23:36.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.621+0000 7f6655ffb640 1 -- 172.21.15.107:0/606529472 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6664031750 con 0x56046c3ee290 2024-04-25T00:23:36.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.621+0000 7f6655ffb640 1 -- 172.21.15.107:0/606529472 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f666403a040 con 0x56046c3ee290 2024-04-25T00:23:36.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.621+0000 7f6677a1a740 10 client.17274 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:36.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.621+0000 7f6677a1a740 10 client.17274 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:36.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.621+0000 7f6677a1a740 20 client.17274 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:36.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.621+0000 7f6677a1a740 1 --2- 172.21.15.107:0/606529472 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56046c408f40 0x56046c40b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.621+0000 7f6677a1a740 4 client.17274 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:36.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.621+0000 7f6677a1a740 1 -- 172.21.15.107:0/606529472 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56046c3265a0 con 0x56046c408f40 2024-04-25T00:23:36.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.622+0000 7f666e59c640 1 --2- 172.21.15.107:0/606529472 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56046c408f40 0x56046c40b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.622+0000 7f6655ffb640 10 client.17274 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:36.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.622+0000 7f666e59c640 1 --2- 172.21.15.107:0/606529472 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56046c408f40 0x56046c40b330 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43ee275a80fbfbf0 server_cookie=84fd9ccb388d87a3 in_seq=0 out_seq=0 2024-04-25T00:23:36.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.623+0000 7f6655ffb640 1 -- 172.21.15.107:0/606529472 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56046c3265a0 con 0x56046c408f40 2024-04-25T00:23:36.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.623+0000 7f6655ffb640 10 client.17274 handle_command_reply: tid=0 2024-04-25T00:23:36.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.623+0000 7f6677a1a740 1 client.17274 shutdown 2024-04-25T00:23:36.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.623+0000 7f6677a1a740 1 -- 172.21.15.107:0/606529472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3ee290 msgr2=0x56046c3f5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:36.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.623+0000 7f6677a1a740 1 --2- 172.21.15.107:0/606529472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3ee290 0x56046c3f5f10 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f6664031550 tx=0x7f6664005360 comp rx=0 tx=0).stop 2024-04-25T00:23:36.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.623+0000 7f6677a1a740 1 -- 172.21.15.107:0/606529472 shutdown_connections 2024-04-25T00:23:36.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.623+0000 7f6677a1a740 1 --2- 172.21.15.107:0/606529472 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56046c408f40 0x56046c40b330 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.623+0000 7f6677a1a740 1 --2- 172.21.15.107:0/606529472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56046c3f9010 0x56046c3f6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.623+0000 7f6677a1a740 1 --2- 172.21.15.107:0/606529472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56046c3ef980 0x56046c3f6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.623+0000 7f6677a1a740 1 --2- 172.21.15.107:0/606529472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3ee290 0x56046c3f5f10 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.623+0000 7f6655ffb640 0 client.17274 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:36.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.623+0000 7f6677a1a740 1 -- 172.21.15.107:0/606529472 >> 172.21.15.107:0/606529472 conn(0x56046c3b6950 msgr2=0x56046c3f0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:36.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.623+0000 7f6677a1a740 1 -- 172.21.15.107:0/606529472 shutdown_connections 2024-04-25T00:23:36.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.623+0000 7f6677a1a740 1 -- 172.21.15.107:0/606529472 wait complete. 2024-04-25T00:23:36.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.623+0000 7f6677a1a740 20 client.17274 trim_cache size 0 max 16384 2024-04-25T00:23:36.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.640+0000 7f6677a1a740 1 Processor -- start 2024-04-25T00:23:36.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.640+0000 7f6677a1a740 1 -- start start 2024-04-25T00:23:36.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.640+0000 7f6677a1a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56046c3ee290 0x56046c3e97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.640+0000 7f6677a1a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56046c3ef980 0x56046c40e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.640+0000 7f6677a1a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3f9010 0x56046c40e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.641+0000 7f6677a1a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f66701b3660 con 0x56046c3f9010 2024-04-25T00:23:36.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.641+0000 7f6677a1a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f66701b0f30 con 0x56046c3ee290 2024-04-25T00:23:36.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.641+0000 7f6677a1a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f66701aeda0 con 0x56046c3ef980 2024-04-25T00:23:36.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.641+0000 7f666dd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3f9010 0x56046c40e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.641+0000 7f666dd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3f9010 0x56046c40e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57550/0 (socket says 172.21.15.107:57550) 2024-04-25T00:23:36.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.641+0000 7f666dd9b640 1 -- 172.21.15.107:0/3339960192 learned_addr learned my addr 172.21.15.107:0/3339960192 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:36.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.641+0000 7f666dd9b640 1 --2- 172.21.15.107:0/3339960192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56046c3ee290 0x56046c3e97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.641+0000 7f666dd9b640 1 -- 172.21.15.107:0/3339960192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56046c3ef980 msgr2=0x56046c40e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.641+0000 7f666dd9b640 1 --2- 172.21.15.107:0/3339960192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56046c3ef980 0x56046c40e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.641+0000 7f666dd9b640 1 -- 172.21.15.107:0/3339960192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56046c3ee290 msgr2=0x56046c3e97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.641+0000 7f666dd9b640 1 --2- 172.21.15.107:0/3339960192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56046c3ee290 0x56046c3e97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.641+0000 7f666dd9b640 1 -- 172.21.15.107:0/3339960192 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56046c026170 con 0x56046c3f9010 2024-04-25T00:23:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.641+0000 7f666dd9b640 1 --2- 172.21.15.107:0/3339960192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3f9010 0x56046c40e890 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6664049780 tx=0x7f666403df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2952f565e58ecf2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.641+0000 7f6657fff640 1 -- 172.21.15.107:0/3339960192 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664043070 con 0x56046c3f9010 2024-04-25T00:23:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.641+0000 7f6657fff640 1 -- 172.21.15.107:0/3339960192 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6664050070 con 0x56046c3f9010 2024-04-25T00:23:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.641+0000 7f6657fff640 1 -- 172.21.15.107:0/3339960192 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664035070 con 0x56046c3f9010 2024-04-25T00:23:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.642+0000 7f6677a1a740 1 -- 172.21.15.107:0/3339960192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3f9010 msgr2=0x56046c40e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.642+0000 7f6677a1a740 1 --2- 172.21.15.107:0/3339960192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3f9010 0x56046c40e890 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6664049780 tx=0x7f666403df90 comp rx=0 tx=0).stop 2024-04-25T00:23:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.642+0000 7f6677a1a740 1 -- 172.21.15.107:0/3339960192 shutdown_connections 2024-04-25T00:23:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.642+0000 7f6677a1a740 1 --2- 172.21.15.107:0/3339960192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3f9010 0x56046c40e890 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.642+0000 7f6677a1a740 1 --2- 172.21.15.107:0/3339960192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56046c3ef980 0x56046c40e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.642+0000 7f6677a1a740 1 --2- 172.21.15.107:0/3339960192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56046c3ee290 0x56046c3e97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.642+0000 7f6677a1a740 1 -- 172.21.15.107:0/3339960192 >> 172.21.15.107:0/3339960192 conn(0x56046c3b6950 msgr2=0x56046c3e90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:36.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.642+0000 7f6677a1a740 1 -- 172.21.15.107:0/3339960192 shutdown_connections 2024-04-25T00:23:36.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.642+0000 7f6677a1a740 1 -- 172.21.15.107:0/3339960192 wait complete. 2024-04-25T00:23:36.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.642+0000 7f6677a1a740 1 -- start start 2024-04-25T00:23:36.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.642+0000 7f6677a1a740 1 Processor -- start 2024-04-25T00:23:36.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.642+0000 7f6677a1a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56046c3ee290 0x56046c40c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.642+0000 7f6677a1a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3ef980 0x56046c40c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.642+0000 7f6677a1a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56046c3f9010 0x56046c40caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.642+0000 7f6677a1a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f66701af3f0 con 0x56046c3ef980 2024-04-25T00:23:36.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.642+0000 7f6677a1a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f66701b0860 con 0x56046c3f9010 2024-04-25T00:23:36.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.642+0000 7f6677a1a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f66701afac0 con 0x56046c3ee290 2024-04-25T00:23:36.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.642+0000 7f6677a1a740 10 client.0 _ll_register_callbacks cb 0x7f666000c390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:36.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.642+0000 7f666e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3ef980 0x56046c40c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.643+0000 7f666e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3ef980 0x56046c40c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57562/0 (socket says 172.21.15.107:57562) 2024-04-25T00:23:36.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.643+0000 7f666e59c640 1 -- 172.21.15.107:0/3185865854 learned_addr learned my addr 172.21.15.107:0/3185865854 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:36.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.643+0000 7f666dd9b640 1 --2- 172.21.15.107:0/3185865854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56046c3f9010 0x56046c40caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.643+0000 7f666dd9b640 1 --2- 172.21.15.107:0/3185865854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56046c3ee290 0x56046c40c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.643+0000 7f666e59c640 1 -- 172.21.15.107:0/3185865854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56046c3ee290 msgr2=0x56046c40c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:36.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.643+0000 7f666e59c640 1 --2- 172.21.15.107:0/3185865854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56046c3ee290 0x56046c40c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.643+0000 7f666e59c640 1 -- 172.21.15.107:0/3185865854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56046c3f9010 msgr2=0x56046c40caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:36.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.643+0000 7f666e59c640 1 --2- 172.21.15.107:0/3185865854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56046c3f9010 0x56046c40caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.643+0000 7f666e59c640 1 -- 172.21.15.107:0/3185865854 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56046c46e410 con 0x56046c3ef980 2024-04-25T00:23:36.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.643+0000 7f666dd9b640 1 --2- 172.21.15.107:0/3185865854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56046c3ee290 0x56046c40c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:36.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.643+0000 7f666e59c640 1 --2- 172.21.15.107:0/3185865854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3ef980 0x56046c40c570 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x56046c3f5770 tx=0x7f6660010af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc895e13cadc3c5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:36.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.643+0000 7f6655ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:36.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.643+0000 7f6655ffb640 1 -- 172.21.15.107:0/3185865854 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6660020030 con 0x56046c3ef980 2024-04-25T00:23:36.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.643+0000 7f6655ffb640 1 -- 172.21.15.107:0/3185865854 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f666001c6f0 con 0x56046c3ef980 2024-04-25T00:23:36.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.643+0000 7f6677a1a740 1 -- 172.21.15.107:0/3185865854 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6664050070 con 0x56046c3ef980 2024-04-25T00:23:36.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.643+0000 7f6655ffb640 1 -- 172.21.15.107:0/3185865854 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f666002a050 con 0x56046c3ef980 2024-04-25T00:23:36.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.643+0000 7f6677a1a740 10 client.17280 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:36.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.643+0000 7f6677a1a740 1 -- 172.21.15.107:0/3185865854 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56046c46f7b0 con 0x56046c3ef980 2024-04-25T00:23:36.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.643+0000 7f6655ffb640 1 -- 172.21.15.107:0/3185865854 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6664050070 con 0x56046c3ef980 2024-04-25T00:23:36.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.644+0000 7f6655ffb640 1 -- 172.21.15.107:0/3185865854 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f6660025760 con 0x56046c3ef980 2024-04-25T00:23:36.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.644+0000 7f6677a1a740 10 client.17280 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:36.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.644+0000 7f6677a1a740 10 client.17280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:36.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.644+0000 7f6677a1a740 20 client.17280 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:36.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.644+0000 7f6677a1a740 1 --2- 172.21.15.107:0/3185865854 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56046c46ff60 0x56046c472380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.644+0000 7f6677a1a740 4 client.17280 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:23:36.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.644+0000 7f6677a1a740 1 -- 172.21.15.107:0/3185865854 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56046c35f800 con 0x56046c46ff60 2024-04-25T00:23:36.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.644+0000 7f666dd9b640 1 --2- 172.21.15.107:0/3185865854 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56046c46ff60 0x56046c472380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.644+0000 7f6655ffb640 10 client.17280 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:36.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.644+0000 7f666dd9b640 1 --2- 172.21.15.107:0/3185865854 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56046c46ff60 0x56046c472380 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=300dbcc36b914f9a server_cookie=784221679a376204 in_seq=0 out_seq=0 2024-04-25T00:23:36.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.645+0000 7f6655ffb640 1 -- 172.21.15.107:0/3185865854 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56046c35f800 con 0x56046c46ff60 2024-04-25T00:23:36.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.645+0000 7f6655ffb640 10 client.17280 handle_command_reply: tid=0 2024-04-25T00:23:36.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.645+0000 7f6677a1a740 1 client.17280 shutdown 2024-04-25T00:23:36.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.645+0000 7f6677a1a740 1 -- 172.21.15.107:0/3185865854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3ef980 msgr2=0x56046c40c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:36.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.645+0000 7f6677a1a740 1 --2- 172.21.15.107:0/3185865854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3ef980 0x56046c40c570 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x56046c3f5770 tx=0x7f6660010af0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.646+0000 7f6677a1a740 1 -- 172.21.15.107:0/3185865854 shutdown_connections 2024-04-25T00:23:36.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.646+0000 7f6677a1a740 1 --2- 172.21.15.107:0/3185865854 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56046c46ff60 0x56046c472380 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.646+0000 7f6677a1a740 1 --2- 172.21.15.107:0/3185865854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56046c3f9010 0x56046c40caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.646+0000 7f6677a1a740 1 --2- 172.21.15.107:0/3185865854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56046c3ef980 0x56046c40c570 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.646+0000 7f6655ffb640 0 client.17280 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:36.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.646+0000 7f6677a1a740 1 --2- 172.21.15.107:0/3185865854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56046c3ee290 0x56046c40c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.646+0000 7f6677a1a740 1 -- 172.21.15.107:0/3185865854 >> 172.21.15.107:0/3185865854 conn(0x56046c3b6950 msgr2=0x56046c3e9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:36.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.646+0000 7f6677a1a740 1 -- 172.21.15.107:0/3185865854 shutdown_connections 2024-04-25T00:23:36.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.646+0000 7f6677a1a740 1 -- 172.21.15.107:0/3185865854 wait complete. 2024-04-25T00:23:36.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.646+0000 7f6677a1a740 20 client.17280 trim_cache size 0 max 16384 2024-04-25T00:23:36.653 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:23:36.653 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-04-25T00:23:36.653 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "1b0c85e9-b80a-46bb-8a44-4a927206476b", 2024-04-25T00:23:36.653 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-04-25T00:23:36.653 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:23:36.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.646+0000 7f6674d16640 1 -- 172.21.15.107:0/2339752349 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f66580a2d00 msgr2=0x7f66580a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:36.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.646+0000 7f6674d16640 1 --2- 172.21.15.107:0/2339752349 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f66580a2d00 0x7f66580a51c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f66680979b0 tx=0x7f66680c5b70 comp rx=0 tx=0).stop 2024-04-25T00:23:36.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.646+0000 7f6674d16640 1 -- 172.21.15.107:0/2339752349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6670107770 msgr2=0x7f66700744c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:36.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.646+0000 7f6674d16640 1 --2- 172.21.15.107:0/2339752349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6670107770 0x7f66700744c0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f666800b970 tx=0x7f666800be30 comp rx=0 tx=0).stop 2024-04-25T00:23:36.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.646+0000 7f6674d16640 1 -- 172.21.15.107:0/2339752349 shutdown_connections 2024-04-25T00:23:36.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.646+0000 7f6674d16640 1 --2- 172.21.15.107:0/2339752349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6670107770 0x7f66700744c0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.646+0000 7f6674d16640 1 --2- 172.21.15.107:0/2339752349 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6670102120 0x7f6670073f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.646+0000 7f6674d16640 1 --2- 172.21.15.107:0/2339752349 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f66580a2d00 0x7f66580a51c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.646+0000 7f6674d16640 1 --2- 172.21.15.107:0/2339752349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6670101760 0x7f6670073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.646+0000 7f6674d16640 1 -- 172.21.15.107:0/2339752349 >> 172.21.15.107:0/2339752349 conn(0x7f66700fd250 msgr2=0x7f66700fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:36.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.646+0000 7f6674d16640 1 -- 172.21.15.107:0/2339752349 shutdown_connections 2024-04-25T00:23:36.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.646+0000 7f6674d16640 1 -- 172.21.15.107:0/2339752349 wait complete. 2024-04-25T00:23:36.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:23:36.658 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:23:36.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.757+0000 7f6352c12640 1 -- 172.21.15.107:0/1173109436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c107720 msgr2=0x7f634c107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:36.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.757+0000 7f6352c12640 1 --2- 172.21.15.107:0/1173109436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c107720 0x7f634c107b00 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f6344009cd0 tx=0x7f634402a8f0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.757+0000 7f6352c12640 1 -- 172.21.15.107:0/1173109436 shutdown_connections 2024-04-25T00:23:36.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.757+0000 7f6352c12640 1 --2- 172.21.15.107:0/1173109436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f634c1020d0 0x7f634c10e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.757+0000 7f6352c12640 1 --2- 172.21.15.107:0/1173109436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f634c101710 0x7f634c101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.757+0000 7f6352c12640 1 --2- 172.21.15.107:0/1173109436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c107720 0x7f634c107b00 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.757+0000 7f6352c12640 1 -- 172.21.15.107:0/1173109436 >> 172.21.15.107:0/1173109436 conn(0x7f634c0fd180 msgr2=0x7f634c0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:36.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.757+0000 7f6352c12640 1 -- 172.21.15.107:0/1173109436 shutdown_connections 2024-04-25T00:23:36.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.757+0000 7f6352c12640 1 -- 172.21.15.107:0/1173109436 wait complete. 2024-04-25T00:23:36.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.758+0000 7f6352c12640 1 Processor -- start 2024-04-25T00:23:36.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.758+0000 7f6352c12640 1 -- start start 2024-04-25T00:23:36.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.758+0000 7f6352c12640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f634c101710 0x7f634c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.758+0000 7f6352c12640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f634c1020d0 0x7f634c1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.758+0000 7f6352c12640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c107720 0x7f634c19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.758+0000 7f6352c12640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f634c111b30 con 0x7f634c107720 2024-04-25T00:23:36.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.758+0000 7f6352c12640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1119b0 con 0x7f634c1020d0 2024-04-25T00:23:36.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.758+0000 7f6352c12640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f634c111cb0 con 0x7f634c101710 2024-04-25T00:23:36.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.758+0000 7f63511af640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c107720 0x7f634c19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.758+0000 7f63511af640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c107720 0x7f634c19c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57570/0 (socket says 172.21.15.107:57570) 2024-04-25T00:23:36.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.758+0000 7f63511af640 1 -- 172.21.15.107:0/1199081451 learned_addr learned my addr 172.21.15.107:0/1199081451 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:36.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.758+0000 7f63509ae640 1 --2- 172.21.15.107:0/1199081451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f634c101710 0x7f634c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.758+0000 7f634bfff640 1 --2- 172.21.15.107:0/1199081451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f634c1020d0 0x7f634c1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.758+0000 7f63511af640 1 -- 172.21.15.107:0/1199081451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f634c101710 msgr2=0x7f634c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:36.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.758+0000 7f63511af640 1 --2- 172.21.15.107:0/1199081451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f634c101710 0x7f634c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.758+0000 7f63511af640 1 -- 172.21.15.107:0/1199081451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f634c1020d0 msgr2=0x7f634c1985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:36.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.758+0000 7f63511af640 1 --2- 172.21.15.107:0/1199081451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f634c1020d0 0x7f634c1985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.758+0000 7f63511af640 1 -- 172.21.15.107:0/1199081451 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634c19d030 con 0x7f634c107720 2024-04-25T00:23:36.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.758+0000 7f634bfff640 1 --2- 172.21.15.107:0/1199081451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f634c1020d0 0x7f634c1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:36.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.759+0000 7f63511af640 1 --2- 172.21.15.107:0/1199081451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c107720 0x7f634c19c900 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f633c00c970 tx=0x7f633c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:36.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.759+0000 7f6349ffb640 1 -- 172.21.15.107:0/1199081451 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c007bb0 con 0x7f634c107720 2024-04-25T00:23:36.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.759+0000 7f6352c12640 1 -- 172.21.15.107:0/1199081451 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634c19d320 con 0x7f634c107720 2024-04-25T00:23:36.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.759+0000 7f6349ffb640 1 -- 172.21.15.107:0/1199081451 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f633c007d50 con 0x7f634c107720 2024-04-25T00:23:36.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.759+0000 7f6349ffb640 1 -- 172.21.15.107:0/1199081451 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c018800 con 0x7f634c107720 2024-04-25T00:23:36.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.759+0000 7f6352c12640 1 -- 172.21.15.107:0/1199081451 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f634c1a4bb0 con 0x7f634c107720 2024-04-25T00:23:36.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.761+0000 7f6349ffb640 1 -- 172.21.15.107:0/1199081451 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f633c00fa50 con 0x7f634c107720 2024-04-25T00:23:36.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.761+0000 7f63559f8740 1 Processor -- start 2024-04-25T00:23:36.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.761+0000 7f63559f8740 1 -- start start 2024-04-25T00:23:36.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.761+0000 7f6349ffb640 1 --2- 172.21.15.107:0/1199081451 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f63340a2d00 0x7f63340a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.761+0000 7f63559f8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55920125a630 0x55920125aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.761+0000 7f63559f8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55920125af50 0x5592012650e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.761+0000 7f63559f8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559201265620 0x559201267a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.761+0000 7f63559f8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5592011bf6b0 con 0x55920125a630 2024-04-25T00:23:36.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.761+0000 7f63559f8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559201167880 con 0x559201265620 2024-04-25T00:23:36.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.761+0000 7f63559f8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5592011bb670 con 0x55920125af50 2024-04-25T00:23:36.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.762+0000 7f63509ae640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55920125a630 0x55920125aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.762+0000 7f634bfff640 1 --2- 172.21.15.107:0/1199081451 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f63340a2d00 0x7f63340a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.762+0000 7f63509ae640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55920125a630 0x55920125aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57584/0 (socket says 172.21.15.107:57584) 2024-04-25T00:23:36.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.762+0000 7f63509ae640 1 -- 172.21.15.107:0/4263269483 learned_addr learned my addr 172.21.15.107:0/4263269483 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:36.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.762+0000 7f6349ffb640 1 -- 172.21.15.107:0/1199081451 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f633c0cf810 con 0x7f634c107720 2024-04-25T00:23:36.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.762+0000 7f634bfff640 1 --2- 172.21.15.107:0/4263269483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55920125af50 0x5592012650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.762+0000 7f63511af640 1 --2- 172.21.15.107:0/4263269483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559201265620 0x559201267a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.762+0000 7f63509ae640 1 -- 172.21.15.107:0/4263269483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55920125af50 msgr2=0x5592012650e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:36.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.762+0000 7f63509ae640 1 --2- 172.21.15.107:0/4263269483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55920125af50 0x5592012650e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.762+0000 7f63509ae640 1 -- 172.21.15.107:0/4263269483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559201265620 msgr2=0x559201267a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:36.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.762+0000 7f63509ae640 1 --2- 172.21.15.107:0/4263269483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559201265620 0x559201267a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.762+0000 7f63509ae640 1 -- 172.21.15.107:0/4263269483 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559200f9d7d0 con 0x55920125a630 2024-04-25T00:23:36.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.762+0000 7f634bfff640 1 --2- 172.21.15.107:0/1199081451 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f63340a2d00 0x7f63340a51c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f634c106d60 tx=0x7f63400138e0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:36.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.762+0000 7f63509ae640 1 --2- 172.21.15.107:0/4263269483 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55920125a630 0x55920125aa10 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f6344009cd0 tx=0x7f6344030450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d3ddabcdaf7f00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:36.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.762+0000 7f6332ffd640 1 -- 172.21.15.107:0/4263269483 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344030930 con 0x55920125a630 2024-04-25T00:23:36.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.762+0000 7f6332ffd640 1 -- 172.21.15.107:0/4263269483 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6344036030 con 0x55920125a630 2024-04-25T00:23:36.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.762+0000 7f6332ffd640 1 -- 172.21.15.107:0/4263269483 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63440044d0 con 0x55920125a630 2024-04-25T00:23:36.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.763+0000 7f63559f8740 1 -- 172.21.15.107:0/4263269483 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55920125a630 msgr2=0x55920125aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:36.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.763+0000 7f63559f8740 1 --2- 172.21.15.107:0/4263269483 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55920125a630 0x55920125aa10 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f6344009cd0 tx=0x7f6344030450 comp rx=0 tx=0).stop 2024-04-25T00:23:36.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.763+0000 7f63559f8740 1 -- 172.21.15.107:0/4263269483 shutdown_connections 2024-04-25T00:23:36.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.763+0000 7f63559f8740 1 --2- 172.21.15.107:0/4263269483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559201265620 0x559201267a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.763+0000 7f63559f8740 1 --2- 172.21.15.107:0/4263269483 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55920125af50 0x5592012650e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.763+0000 7f63559f8740 1 --2- 172.21.15.107:0/4263269483 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55920125a630 0x55920125aa10 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.763+0000 7f63559f8740 1 -- 172.21.15.107:0/4263269483 >> 172.21.15.107:0/4263269483 conn(0x559201256950 msgr2=0x559201256d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:36.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.763+0000 7f63559f8740 1 -- 172.21.15.107:0/4263269483 shutdown_connections 2024-04-25T00:23:36.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.763+0000 7f63559f8740 1 -- 172.21.15.107:0/4263269483 wait complete. 2024-04-25T00:23:36.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.763+0000 7f63559f8740 1 -- start start 2024-04-25T00:23:36.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.763+0000 7f63559f8740 1 Processor -- start 2024-04-25T00:23:36.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.764+0000 7f63559f8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55920125a630 0x559201262550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.764+0000 7f63559f8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55920125af50 0x559201262a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.764+0000 7f63559f8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559201265620 0x559201262fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.764+0000 7f63559f8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5592012683b0 con 0x55920125af50 2024-04-25T00:23:36.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.764+0000 7f63559f8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55920117af50 con 0x559201265620 2024-04-25T00:23:36.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.764+0000 7f63559f8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559201268530 con 0x55920125a630 2024-04-25T00:23:36.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.764+0000 7f63559f8740 10 client.0 _ll_register_callbacks cb 0x5592011b6000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:36.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.764+0000 7f634bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55920125af50 0x559201262a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.764+0000 7f634bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55920125af50 0x559201262a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57588/0 (socket says 172.21.15.107:57588) 2024-04-25T00:23:36.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.764+0000 7f634bfff640 1 -- 172.21.15.107:0/609985106 learned_addr learned my addr 172.21.15.107:0/609985106 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:36.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.764+0000 7f63511af640 1 --2- 172.21.15.107:0/609985106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559201265620 0x559201262fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.764+0000 7f63509ae640 1 --2- 172.21.15.107:0/609985106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55920125a630 0x559201262550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.764+0000 7f634bfff640 1 -- 172.21.15.107:0/609985106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55920125a630 msgr2=0x559201262550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:36.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.764+0000 7f634bfff640 1 --2- 172.21.15.107:0/609985106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55920125a630 0x559201262550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.764+0000 7f634bfff640 1 -- 172.21.15.107:0/609985106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559201265620 msgr2=0x559201262fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:36.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.764+0000 7f634bfff640 1 --2- 172.21.15.107:0/609985106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559201265620 0x559201262fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.764+0000 7f634bfff640 1 -- 172.21.15.107:0/609985106 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559200ed33f0 con 0x55920125af50 2024-04-25T00:23:36.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.764+0000 7f63509ae640 1 --2- 172.21.15.107:0/609985106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55920125a630 0x559201262550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:36.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.764+0000 7f634bfff640 1 --2- 172.21.15.107:0/609985106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55920125af50 0x559201262a90 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f6340011760 tx=0x7f6340004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113a0168a97d3507 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:36.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.764+0000 7f632bfff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:36.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.764+0000 7f632bfff640 1 -- 172.21.15.107:0/609985106 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6340024030 con 0x55920125af50 2024-04-25T00:23:36.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.764+0000 7f632bfff640 1 -- 172.21.15.107:0/609985106 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f634000e810 con 0x55920125af50 2024-04-25T00:23:36.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.764+0000 7f63559f8740 1 -- 172.21.15.107:0/609985106 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6344036030 con 0x55920125af50 2024-04-25T00:23:36.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.765+0000 7f632bfff640 1 -- 172.21.15.107:0/609985106 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6340031050 con 0x55920125af50 2024-04-25T00:23:36.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.765+0000 7f63559f8740 10 client.17292 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:36.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.765+0000 7f63559f8740 1 -- 172.21.15.107:0/609985106 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559200e930f0 con 0x55920125af50 2024-04-25T00:23:36.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.765+0000 7f632bfff640 1 -- 172.21.15.107:0/609985106 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6344036030 con 0x55920125af50 2024-04-25T00:23:36.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.765+0000 7f632bfff640 1 -- 172.21.15.107:0/609985106 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f6340020b70 con 0x55920125af50 2024-04-25T00:23:36.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.765+0000 7f63559f8740 10 client.17292 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:36.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.765+0000 7f63559f8740 10 client.17292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:36.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.765+0000 7f63559f8740 20 client.17292 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:36.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.765+0000 7f63559f8740 1 --2- 172.21.15.107:0/609985106 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x559201275700 0x559201277bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.765+0000 7f63559f8740 4 client.17292 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:36.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.765+0000 7f63559f8740 1 -- 172.21.15.107:0/609985106 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592011ccbf0 con 0x559201275700 2024-04-25T00:23:36.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.765+0000 7f63509ae640 1 --2- 172.21.15.107:0/609985106 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x559201275700 0x559201277bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.766+0000 7f63509ae640 1 --2- 172.21.15.107:0/609985106 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x559201275700 0x559201277bc0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b617f49ee552b6 server_cookie=5c25b4d7ee72338e in_seq=0 out_seq=0 2024-04-25T00:23:36.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.766+0000 7f632bfff640 10 client.17292 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:36.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.766+0000 7f632bfff640 1 -- 172.21.15.107:0/609985106 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5592011ccbf0 con 0x559201275700 2024-04-25T00:23:36.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.766+0000 7f632bfff640 10 client.17292 handle_command_reply: tid=0 2024-04-25T00:23:36.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.766+0000 7f63559f8740 1 client.17292 shutdown 2024-04-25T00:23:36.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.767+0000 7f63559f8740 1 -- 172.21.15.107:0/609985106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55920125af50 msgr2=0x559201262a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:36.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.767+0000 7f63559f8740 1 --2- 172.21.15.107:0/609985106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55920125af50 0x559201262a90 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f6340011760 tx=0x7f6340004730 comp rx=0 tx=0).stop 2024-04-25T00:23:36.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.767+0000 7f63559f8740 1 -- 172.21.15.107:0/609985106 shutdown_connections 2024-04-25T00:23:36.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.767+0000 7f63559f8740 1 --2- 172.21.15.107:0/609985106 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x559201275700 0x559201277bc0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.767+0000 7f63559f8740 1 --2- 172.21.15.107:0/609985106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559201265620 0x559201262fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.767+0000 7f63559f8740 1 --2- 172.21.15.107:0/609985106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55920125af50 0x559201262a90 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.767+0000 7f63559f8740 1 --2- 172.21.15.107:0/609985106 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55920125a630 0x559201262550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.767+0000 7f632bfff640 0 client.17292 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:36.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.767+0000 7f63559f8740 1 -- 172.21.15.107:0/609985106 >> 172.21.15.107:0/609985106 conn(0x559201256950 msgr2=0x55920125c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:36.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.767+0000 7f63559f8740 1 -- 172.21.15.107:0/609985106 shutdown_connections 2024-04-25T00:23:36.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.767+0000 7f63559f8740 1 -- 172.21.15.107:0/609985106 wait complete. 2024-04-25T00:23:36.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.767+0000 7f63559f8740 20 client.17292 trim_cache size 0 max 16384 2024-04-25T00:23:36.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.784+0000 7f63559f8740 1 Processor -- start 2024-04-25T00:23:36.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.784+0000 7f63559f8740 1 -- start start 2024-04-25T00:23:36.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.784+0000 7f63559f8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55920125a630 0x559201271270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.784+0000 7f63559f8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55920125af50 0x559201256d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.784+0000 7f63559f8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559201265620 0x55920125cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.784+0000 7f63559f8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1b1390 con 0x559201265620 2024-04-25T00:23:36.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.784+0000 7f63559f8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1aec60 con 0x55920125af50 2024-04-25T00:23:36.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.784+0000 7f63559f8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1acad0 con 0x55920125a630 2024-04-25T00:23:36.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.784+0000 7f63511af640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559201265620 0x55920125cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.784+0000 7f63511af640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559201265620 0x55920125cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57602/0 (socket says 172.21.15.107:57602) 2024-04-25T00:23:36.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.784+0000 7f63511af640 1 -- 172.21.15.107:0/3854047351 learned_addr learned my addr 172.21.15.107:0/3854047351 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:36.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.785+0000 7f63511af640 1 -- 172.21.15.107:0/3854047351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55920125a630 msgr2=0x559201271270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:36.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.785+0000 7f63511af640 1 --2- 172.21.15.107:0/3854047351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55920125a630 0x559201271270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.785+0000 7f63511af640 1 -- 172.21.15.107:0/3854047351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55920125af50 msgr2=0x559201256d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:36.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.785+0000 7f63511af640 1 --2- 172.21.15.107:0/3854047351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55920125af50 0x559201256d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.785+0000 7f63511af640 1 -- 172.21.15.107:0/3854047351 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559201260aa0 con 0x559201265620 2024-04-25T00:23:36.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.785+0000 7f63511af640 1 --2- 172.21.15.107:0/3854047351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559201265620 0x55920125cdd0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f633c09b900 tx=0x7f633c09b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302ea5604c1ce41e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:36.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.785+0000 7f63327fc640 1 -- 172.21.15.107:0/3854047351 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c09b970 con 0x559201265620 2024-04-25T00:23:36.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.785+0000 7f63327fc640 1 -- 172.21.15.107:0/3854047351 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f633c09fd20 con 0x559201265620 2024-04-25T00:23:36.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.785+0000 7f63327fc640 1 -- 172.21.15.107:0/3854047351 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c07f1a0 con 0x559201265620 2024-04-25T00:23:36.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.785+0000 7f63559f8740 1 -- 172.21.15.107:0/3854047351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559201265620 msgr2=0x55920125cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:36.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.785+0000 7f63559f8740 1 --2- 172.21.15.107:0/3854047351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559201265620 0x55920125cdd0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f633c09b900 tx=0x7f633c09b3e0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.785+0000 7f63559f8740 1 -- 172.21.15.107:0/3854047351 shutdown_connections 2024-04-25T00:23:36.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.785+0000 7f63559f8740 1 --2- 172.21.15.107:0/3854047351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559201265620 0x55920125cdd0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.785+0000 7f63559f8740 1 --2- 172.21.15.107:0/3854047351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55920125af50 0x559201256d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.785+0000 7f63559f8740 1 --2- 172.21.15.107:0/3854047351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55920125a630 0x559201271270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.785+0000 7f63559f8740 1 -- 172.21.15.107:0/3854047351 >> 172.21.15.107:0/3854047351 conn(0x559201256950 msgr2=0x559201262510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:36.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.785+0000 7f63559f8740 1 -- 172.21.15.107:0/3854047351 shutdown_connections 2024-04-25T00:23:36.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.785+0000 7f63559f8740 1 -- 172.21.15.107:0/3854047351 wait complete. 2024-04-25T00:23:36.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.786+0000 7f63559f8740 1 -- start start 2024-04-25T00:23:36.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.786+0000 7f63559f8740 1 Processor -- start 2024-04-25T00:23:36.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.786+0000 7f63559f8740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55920125a630 0x55920126dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.786+0000 7f63559f8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55920125af50 0x559201268f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.786+0000 7f63559f8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559201265620 0x559201269490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.786+0000 7f63559f8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1ad120 con 0x559201265620 2024-04-25T00:23:36.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.786+0000 7f63559f8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1ae590 con 0x55920125af50 2024-04-25T00:23:36.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.786+0000 7f63559f8740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1ad7f0 con 0x55920125a630 2024-04-25T00:23:36.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.786+0000 7f63559f8740 10 client.0 _ll_register_callbacks cb 0x7f6340011670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:36.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.786+0000 7f63511af640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559201265620 0x559201269490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.786+0000 7f63511af640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559201265620 0x559201269490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57618/0 (socket says 172.21.15.107:57618) 2024-04-25T00:23:36.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.786+0000 7f63511af640 1 -- 172.21.15.107:0/2352806224 learned_addr learned my addr 172.21.15.107:0/2352806224 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:23:36.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.786+0000 7f63509ae640 1 --2- 172.21.15.107:0/2352806224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55920125a630 0x55920126dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.787+0000 7f63511af640 1 -- 172.21.15.107:0/2352806224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55920125a630 msgr2=0x55920126dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:36.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.787+0000 7f63511af640 1 --2- 172.21.15.107:0/2352806224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55920125a630 0x55920126dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.787+0000 7f63511af640 1 -- 172.21.15.107:0/2352806224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55920125af50 msgr2=0x559201268f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:36.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.787+0000 7f63511af640 1 --2- 172.21.15.107:0/2352806224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55920125af50 0x559201268f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.787+0000 7f63511af640 1 -- 172.21.15.107:0/2352806224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559201269cc0 con 0x559201265620 2024-04-25T00:23:36.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.787+0000 7f63509ae640 1 --2- 172.21.15.107:0/2352806224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55920125a630 0x55920126dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:36.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.787+0000 7f63511af640 1 --2- 172.21.15.107:0/2352806224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559201265620 0x559201269490 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f633c097dd0 tx=0x7f633c09bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4405928f039cd6b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:36.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.787+0000 7f632bfff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:23:36.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.787+0000 7f632bfff640 1 -- 172.21.15.107:0/2352806224 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633c0b4060 con 0x559201265620 2024-04-25T00:23:36.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.787+0000 7f632bfff640 1 -- 172.21.15.107:0/2352806224 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f633c097fe0 con 0x559201265620 2024-04-25T00:23:36.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.787+0000 7f63559f8740 1 -- 172.21.15.107:0/2352806224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f633c09fd20 con 0x559201265620 2024-04-25T00:23:36.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.787+0000 7f632bfff640 1 -- 172.21.15.107:0/2352806224 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f633c07e070 con 0x559201265620 2024-04-25T00:23:36.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.787+0000 7f63559f8740 10 client.17298 fetch_fsmap learned FSMap version 34 2024-04-25T00:23:36.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.787+0000 7f63559f8740 1 -- 172.21.15.107:0/2352806224 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5592012dbda0 con 0x559201265620 2024-04-25T00:23:36.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.787+0000 7f632bfff640 1 -- 172.21.15.107:0/2352806224 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f633c09fd20 con 0x559201265620 2024-04-25T00:23:36.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.787+0000 7f632bfff640 1 -- 172.21.15.107:0/2352806224 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f633c0ce070 con 0x559201265620 2024-04-25T00:23:36.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.787+0000 7f63559f8740 10 client.17298 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:23:36.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.787+0000 7f63559f8740 10 client.17298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:36.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.787+0000 7f63559f8740 20 client.17298 populate_metadata read hostname 'smithi107' 2024-04-25T00:23:36.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.787+0000 7f63559f8740 1 --2- 172.21.15.107:0/2352806224 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5592012dc360 0x5592012de780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:36.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.787+0000 7f63559f8740 4 client.17298 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:23:36.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.787+0000 7f63559f8740 1 -- 172.21.15.107:0/2352806224 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5592011c6f40 con 0x5592012dc360 2024-04-25T00:23:36.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.788+0000 7f63509ae640 1 --2- 172.21.15.107:0/2352806224 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5592012dc360 0x5592012de780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:36.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.788+0000 7f63509ae640 1 --2- 172.21.15.107:0/2352806224 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5592012dc360 0x5592012de780 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cda81e0479f85e3e server_cookie=beef46bfa261848b in_seq=0 out_seq=0 2024-04-25T00:23:36.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.788+0000 7f632bfff640 10 client.17298 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:36.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.788+0000 7f632bfff640 1 -- 172.21.15.107:0/2352806224 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5592011c6f40 con 0x5592012dc360 2024-04-25T00:23:36.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.788+0000 7f632bfff640 10 client.17298 handle_command_reply: tid=0 2024-04-25T00:23:36.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.788+0000 7f63559f8740 1 client.17298 shutdown 2024-04-25T00:23:36.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.788+0000 7f63559f8740 1 -- 172.21.15.107:0/2352806224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559201265620 msgr2=0x559201269490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:36.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.788+0000 7f63559f8740 1 --2- 172.21.15.107:0/2352806224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559201265620 0x559201269490 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f633c097dd0 tx=0x7f633c09bab0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.788+0000 7f63559f8740 1 -- 172.21.15.107:0/2352806224 shutdown_connections 2024-04-25T00:23:36.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.788+0000 7f63559f8740 1 --2- 172.21.15.107:0/2352806224 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5592012dc360 0x5592012de780 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.788+0000 7f63559f8740 1 --2- 172.21.15.107:0/2352806224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559201265620 0x559201269490 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.788+0000 7f63559f8740 1 --2- 172.21.15.107:0/2352806224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55920125af50 0x559201268f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.788+0000 7f63559f8740 1 --2- 172.21.15.107:0/2352806224 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55920125a630 0x55920126dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.788+0000 7f63559f8740 1 -- 172.21.15.107:0/2352806224 >> 172.21.15.107:0/2352806224 conn(0x559201256950 msgr2=0x559201255af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:36.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.788+0000 7f632bfff640 0 client.17298 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:23:36.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.789+0000 7f63559f8740 1 -- 172.21.15.107:0/2352806224 shutdown_connections 2024-04-25T00:23:36.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.789+0000 7f63559f8740 1 -- 172.21.15.107:0/2352806224 wait complete. 2024-04-25T00:23:36.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.789+0000 7f63559f8740 20 client.17298 trim_cache size 0 max 16384 2024-04-25T00:23:36.797 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:23:36.797 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:23:36.797 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-04-25T00:23:36.797 INFO:teuthology.orchestra.run.smithi107.stdout: "1b0c85e9-b80a-46bb-8a44-4a927206476b": { 2024-04-25T00:23:36.797 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-04-25T00:23:36.797 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "1b0c85e9-b80a-46bb-8a44-4a927206476b", 2024-04-25T00:23:36.797 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-04-25T00:23:36.797 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:23:36.797 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:23:36.797 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:23:36.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.789+0000 7f6352c12640 1 -- 172.21.15.107:0/1199081451 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f63340a2d00 msgr2=0x7f63340a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:36.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.789+0000 7f6352c12640 1 --2- 172.21.15.107:0/1199081451 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f63340a2d00 0x7f63340a51c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f634c106d60 tx=0x7f63400138e0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.789+0000 7f6352c12640 1 -- 172.21.15.107:0/1199081451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c107720 msgr2=0x7f634c19c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:36.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.789+0000 7f6352c12640 1 --2- 172.21.15.107:0/1199081451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c107720 0x7f634c19c900 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f633c00c970 tx=0x7f633c00ce30 comp rx=0 tx=0).stop 2024-04-25T00:23:36.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.789+0000 7f6352c12640 1 -- 172.21.15.107:0/1199081451 shutdown_connections 2024-04-25T00:23:36.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.789+0000 7f6352c12640 1 --2- 172.21.15.107:0/1199081451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f634c107720 0x7f634c19c900 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.789+0000 7f6352c12640 1 --2- 172.21.15.107:0/1199081451 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f63340a2d00 0x7f63340a51c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.789+0000 7f6352c12640 1 --2- 172.21.15.107:0/1199081451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f634c1020d0 0x7f634c1985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.789+0000 7f6352c12640 1 --2- 172.21.15.107:0/1199081451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f634c101710 0x7f634c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:36.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.789+0000 7f6352c12640 1 -- 172.21.15.107:0/1199081451 >> 172.21.15.107:0/1199081451 conn(0x7f634c0fd180 msgr2=0x7f634c0fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:36.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.789+0000 7f6352c12640 1 -- 172.21.15.107:0/1199081451 shutdown_connections 2024-04-25T00:23:36.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:23:36.789+0000 7f6352c12640 1 -- 172.21.15.107:0/1199081451 wait complete. 2024-04-25T00:23:36.800 INFO:tasks.cephfs.filesystem:scrub status for tag:1b0c85e9-b80a-46bb-8a44-4a927206476b - {'path': '/', 'tag': '1b0c85e9-b80a-46bb-8a44-4a927206476b', 'options': 'recursive,force'} 2024-04-25T00:23:37.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:37 smithi170 ceph-mon[32807]: pgmap v220: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:23:37.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:37 smithi170 ceph-mon[32807]: scrub queued for path: / 2024-04-25T00:23:37.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:37 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:37.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:37 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:23:37.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:37 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:37.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:37 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:23:37.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:37 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:23:37.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:37 smithi107 ceph-mon[29153]: pgmap v220: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:23:37.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:37 smithi107 ceph-mon[29153]: scrub queued for path: / 2024-04-25T00:23:37.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:37 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:37.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:37 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:23:37.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:37 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:37.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:37 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:23:37.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:37 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:23:38.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:37 smithi195 ceph-mon[32833]: pgmap v220: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:23:38.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:37 smithi195 ceph-mon[32833]: scrub queued for path: / 2024-04-25T00:23:38.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:37 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:38.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:37 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:23:38.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:37 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:23:38.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:37 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:23:38.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:37 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:23:38.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:38 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:38.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:38 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:23:38.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:38 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:23:38.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:38 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:38.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:38 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:23:38.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:38 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:23:39.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:38 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:39.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:38 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:23:39.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:38 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:23:40.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:40 smithi195 ceph-mon[32833]: pgmap v221: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:23:40.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:40 smithi170 ceph-mon[32807]: pgmap v221: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:23:40.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:40 smithi107 ceph-mon[29153]: pgmap v221: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:23:42.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:42 smithi195 ceph-mon[32833]: pgmap v222: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:23:42.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:42 smithi170 ceph-mon[32807]: pgmap v222: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:23:42.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:42 smithi107 ceph-mon[29153]: pgmap v222: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:23:44.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:44 smithi195 ceph-mon[32833]: pgmap v223: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:23:44.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:44 smithi170 ceph-mon[32807]: pgmap v223: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:23:44.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:44 smithi107 ceph-mon[29153]: pgmap v223: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:23:46.286 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:46 smithi107 ceph-mon[29153]: pgmap v224: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:23:46.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:46 smithi195 ceph-mon[32833]: pgmap v224: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:23:46.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:46 smithi170 ceph-mon[32807]: pgmap v224: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:23:48.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:48 smithi195 ceph-mon[32833]: pgmap v225: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:23:48.436 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:48 smithi170 ceph-mon[32807]: pgmap v225: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:23:48.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:48 smithi107 ceph-mon[29153]: pgmap v225: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:23:50.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:50 smithi195 ceph-mon[32833]: pgmap v226: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:23:50.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:50 smithi170 ceph-mon[32807]: pgmap v226: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:23:50.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:50 smithi107 ceph-mon[29153]: pgmap v226: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:23:52.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:52 smithi195 ceph-mon[32833]: pgmap v227: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:23:52.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:52 smithi170 ceph-mon[32807]: pgmap v227: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:23:52.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:52 smithi107 ceph-mon[29153]: pgmap v227: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:23:54.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:54 smithi195 ceph-mon[32833]: pgmap v228: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:23:54.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:54 smithi170 ceph-mon[32807]: pgmap v228: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:23:54.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:54 smithi107 ceph-mon[29153]: pgmap v228: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:23:56.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:56 smithi195 ceph-mon[32833]: pgmap v229: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:23:56.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:56 smithi170 ceph-mon[32807]: pgmap v229: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:23:56.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:56 smithi107 ceph-mon[29153]: pgmap v229: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:23:58.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:23:58 smithi195 ceph-mon[32833]: pgmap v230: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:23:58.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:23:58 smithi170 ceph-mon[32807]: pgmap v230: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:23:58.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:23:58 smithi107 ceph-mon[29153]: pgmap v230: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:24:00.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:00 smithi195 ceph-mon[32833]: pgmap v231: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:24:00.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:00 smithi170 ceph-mon[32807]: pgmap v231: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:24:00.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:00 smithi107 ceph-mon[29153]: pgmap v231: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:24:02.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:02 smithi195 ceph-mon[32833]: pgmap v232: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T00:24:02.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:02 smithi170 ceph-mon[32807]: pgmap v232: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T00:24:02.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:02 smithi107 ceph-mon[29153]: pgmap v232: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T00:24:04.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:04 smithi195 ceph-mon[32833]: pgmap v233: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T00:24:04.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:04 smithi170 ceph-mon[32807]: pgmap v233: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T00:24:04.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:04 smithi107 ceph-mon[29153]: pgmap v233: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T00:24:06.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:06 smithi195 ceph-mon[32833]: pgmap v234: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:24:06.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:06 smithi170 ceph-mon[32807]: pgmap v234: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:24:06.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:06 smithi107 ceph-mon[29153]: pgmap v234: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:24:06.801 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:24:06.801 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:24:06.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.916+0000 7f3959e8c640 1 -- 172.21.15.107:0/2617950958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39541020d0 msgr2=0x7f395410e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:06.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.916+0000 7f3959e8c640 1 --2- 172.21.15.107:0/2617950958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39541020d0 0x7f395410e5e0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f394c009cd0 tx=0x7f394c02a8e0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.916+0000 7f3959e8c640 1 -- 172.21.15.107:0/2617950958 shutdown_connections 2024-04-25T00:24:06.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.916+0000 7f3959e8c640 1 --2- 172.21.15.107:0/2617950958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39541020d0 0x7f395410e5e0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.916+0000 7f3959e8c640 1 --2- 172.21.15.107:0/2617950958 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3954101710 0x7f3954101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.916+0000 7f3959e8c640 1 --2- 172.21.15.107:0/2617950958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3954107720 0x7f3954107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.916+0000 7f3959e8c640 1 -- 172.21.15.107:0/2617950958 >> 172.21.15.107:0/2617950958 conn(0x7f3954078b90 msgr2=0x7f3954078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:06.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.916+0000 7f3959e8c640 1 -- 172.21.15.107:0/2617950958 shutdown_connections 2024-04-25T00:24:06.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.916+0000 7f3959e8c640 1 -- 172.21.15.107:0/2617950958 wait complete. 2024-04-25T00:24:06.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.916+0000 7f3959e8c640 1 Processor -- start 2024-04-25T00:24:06.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.916+0000 7f3959e8c640 1 -- start start 2024-04-25T00:24:06.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.917+0000 7f3959e8c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3954101710 0x7f395419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:06.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.917+0000 7f3959e8c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39541020d0 0x7f395419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:06.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.917+0000 7f3959e8c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3954107720 0x7f395419eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:06.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.917+0000 7f3959e8c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3954111b30 con 0x7f3954107720 2024-04-25T00:24:06.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.917+0000 7f3959e8c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f39541119b0 con 0x7f39541020d0 2024-04-25T00:24:06.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.917+0000 7f3959e8c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3954111cb0 con 0x7f3954101710 2024-04-25T00:24:06.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.917+0000 7f3953fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3954107720 0x7f395419eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:06.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.917+0000 7f3953fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3954107720 0x7f395419eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42550/0 (socket says 172.21.15.107:42550) 2024-04-25T00:24:06.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.917+0000 7f3953fff640 1 -- 172.21.15.107:0/1667104587 learned_addr learned my addr 172.21.15.107:0/1667104587 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:06.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.917+0000 7f39537fe640 1 --2- 172.21.15.107:0/1667104587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3954101710 0x7f395419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:06.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.917+0000 7f394bfff640 1 --2- 172.21.15.107:0/1667104587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39541020d0 0x7f395419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:06.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.917+0000 7f3953fff640 1 -- 172.21.15.107:0/1667104587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3954101710 msgr2=0x7f395419a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:06.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.917+0000 7f3953fff640 1 --2- 172.21.15.107:0/1667104587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3954101710 0x7f395419a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.917+0000 7f3953fff640 1 -- 172.21.15.107:0/1667104587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39541020d0 msgr2=0x7f395419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:06.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.917+0000 7f3953fff640 1 --2- 172.21.15.107:0/1667104587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39541020d0 0x7f395419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.917+0000 7f3953fff640 1 -- 172.21.15.107:0/1667104587 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f395419f1e0 con 0x7f3954107720 2024-04-25T00:24:06.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.917+0000 7f3953fff640 1 --2- 172.21.15.107:0/1667104587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3954107720 0x7f395419eab0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f394c005cc0 tx=0x7f394c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:06.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.917+0000 7f39517fa640 1 -- 172.21.15.107:0/1667104587 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c004040 con 0x7f3954107720 2024-04-25T00:24:06.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.917+0000 7f39517fa640 1 -- 172.21.15.107:0/1667104587 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f394c0046c0 con 0x7f3954107720 2024-04-25T00:24:06.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.917+0000 7f3959e8c640 1 -- 172.21.15.107:0/1667104587 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f395419f470 con 0x7f3954107720 2024-04-25T00:24:06.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.917+0000 7f39517fa640 1 -- 172.21.15.107:0/1667104587 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c046480 con 0x7f3954107720 2024-04-25T00:24:06.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.917+0000 7f3959e8c640 1 -- 172.21.15.107:0/1667104587 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39541a6d60 con 0x7f3954107720 2024-04-25T00:24:06.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.920+0000 7f39517fa640 1 -- 172.21.15.107:0/1667104587 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f394c0305a0 con 0x7f3954107720 2024-04-25T00:24:06.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.920+0000 7f395cb96740 1 Processor -- start 2024-04-25T00:24:06.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.920+0000 7f395cb96740 1 -- start start 2024-04-25T00:24:06.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.920+0000 7f39517fa640 1 --2- 172.21.15.107:0/1667104587 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f39400a2d00 0x7f39400a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:06.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.920+0000 7f395cb96740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0d2f4a630 0x55b0d2f4aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:06.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.920+0000 7f395cb96740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0d2f4af50 0x55b0d2f550e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:06.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.920+0000 7f395cb96740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f55620 0x55b0d2f57a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:06.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.920+0000 7f395cb96740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b0d2eaf6b0 con 0x55b0d2f55620 2024-04-25T00:24:06.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.920+0000 7f395cb96740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b0d2e57880 con 0x55b0d2f4af50 2024-04-25T00:24:06.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.920+0000 7f395cb96740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b0d2eab670 con 0x55b0d2f4a630 2024-04-25T00:24:06.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.920+0000 7f394bfff640 1 --2- 172.21.15.107:0/1667104587 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f39400a2d00 0x7f39400a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:06.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.920+0000 7f3953fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f55620 0x55b0d2f57a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:06.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.920+0000 7f3953fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f55620 0x55b0d2f57a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42562/0 (socket says 172.21.15.107:42562) 2024-04-25T00:24:06.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.920+0000 7f3953fff640 1 -- 172.21.15.107:0/1245215264 learned_addr learned my addr 172.21.15.107:0/1245215264 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:06.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.920+0000 7f39517fa640 1 -- 172.21.15.107:0/1667104587 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f394c02e030 con 0x7f3954107720 2024-04-25T00:24:06.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.921+0000 7f39537fe640 1 --2- 172.21.15.107:0/1245215264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0d2f4a630 0x55b0d2f4aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:06.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.921+0000 7f394bfff640 1 --2- 172.21.15.107:0/1245215264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0d2f4af50 0x55b0d2f550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:06.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.921+0000 7f3953fff640 1 -- 172.21.15.107:0/1245215264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0d2f4a630 msgr2=0x55b0d2f4aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:06.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.921+0000 7f3953fff640 1 --2- 172.21.15.107:0/1245215264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0d2f4a630 0x55b0d2f4aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.921+0000 7f3953fff640 1 -- 172.21.15.107:0/1245215264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0d2f4af50 msgr2=0x55b0d2f550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:06.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.921+0000 7f3953fff640 1 --2- 172.21.15.107:0/1245215264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0d2f4af50 0x55b0d2f550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.921+0000 7f3953fff640 1 -- 172.21.15.107:0/1245215264 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0d2c8d7d0 con 0x55b0d2f55620 2024-04-25T00:24:06.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.921+0000 7f394bfff640 1 --2- 172.21.15.107:0/1667104587 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f39400a2d00 0x7f39400a51c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f39540febf0 tx=0x7f393c0138e0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:06.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.921+0000 7f3953fff640 1 --2- 172.21.15.107:0/1245215264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f55620 0x55b0d2f57a10 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f394c0c59c0 tx=0x7f394c049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469bf978b9c0155b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:06.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.921+0000 7f3949ffb640 1 -- 172.21.15.107:0/1245215264 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c0bcdc0 con 0x55b0d2f55620 2024-04-25T00:24:06.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.921+0000 7f3949ffb640 1 -- 172.21.15.107:0/1245215264 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f394c036040 con 0x55b0d2f55620 2024-04-25T00:24:06.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.921+0000 7f3949ffb640 1 -- 172.21.15.107:0/1245215264 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c04a1b0 con 0x55b0d2f55620 2024-04-25T00:24:06.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.922+0000 7f395cb96740 1 -- 172.21.15.107:0/1245215264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f55620 msgr2=0x55b0d2f57a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:06.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.922+0000 7f395cb96740 1 --2- 172.21.15.107:0/1245215264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f55620 0x55b0d2f57a10 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f394c0c59c0 tx=0x7f394c049000 comp rx=0 tx=0).stop 2024-04-25T00:24:06.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.922+0000 7f395cb96740 1 -- 172.21.15.107:0/1245215264 shutdown_connections 2024-04-25T00:24:06.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.922+0000 7f395cb96740 1 --2- 172.21.15.107:0/1245215264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f55620 0x55b0d2f57a10 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.922+0000 7f395cb96740 1 --2- 172.21.15.107:0/1245215264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0d2f4af50 0x55b0d2f550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.922+0000 7f395cb96740 1 --2- 172.21.15.107:0/1245215264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0d2f4a630 0x55b0d2f4aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.922+0000 7f395cb96740 1 -- 172.21.15.107:0/1245215264 >> 172.21.15.107:0/1245215264 conn(0x55b0d2f46950 msgr2=0x55b0d2f46d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:06.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.922+0000 7f395cb96740 1 -- 172.21.15.107:0/1245215264 shutdown_connections 2024-04-25T00:24:06.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.922+0000 7f395cb96740 1 -- 172.21.15.107:0/1245215264 wait complete. 2024-04-25T00:24:06.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.922+0000 7f395cb96740 1 -- start start 2024-04-25T00:24:06.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.922+0000 7f395cb96740 1 Processor -- start 2024-04-25T00:24:06.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.922+0000 7f395cb96740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0d2f4a630 0x55b0d2f524e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:06.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f395cb96740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0d2f4af50 0x55b0d2f52a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:06.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f395cb96740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f55620 0x55b0d2f52f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:06.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f395cb96740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b0d2f581d0 con 0x55b0d2f55620 2024-04-25T00:24:06.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f395cb96740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b0d2e6af50 con 0x55b0d2f4a630 2024-04-25T00:24:06.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f395cb96740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b0d2f58350 con 0x55b0d2f4af50 2024-04-25T00:24:06.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f395cb96740 10 client.0 _ll_register_callbacks cb 0x55b0d2ea6000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:06.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f3953fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f55620 0x55b0d2f52f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:06.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f3953fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f55620 0x55b0d2f52f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42578/0 (socket says 172.21.15.107:42578) 2024-04-25T00:24:06.931 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f3953fff640 1 -- 172.21.15.107:0/3266126760 learned_addr learned my addr 172.21.15.107:0/3266126760 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:06.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f39537fe640 1 --2- 172.21.15.107:0/3266126760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0d2f4a630 0x55b0d2f524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:06.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f394bfff640 1 --2- 172.21.15.107:0/3266126760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0d2f4af50 0x55b0d2f52a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:06.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f3953fff640 1 -- 172.21.15.107:0/3266126760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0d2f4af50 msgr2=0x55b0d2f52a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:06.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f3953fff640 1 --2- 172.21.15.107:0/3266126760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0d2f4af50 0x55b0d2f52a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f3953fff640 1 -- 172.21.15.107:0/3266126760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0d2f4a630 msgr2=0x55b0d2f524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:06.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f3953fff640 1 --2- 172.21.15.107:0/3266126760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0d2f4a630 0x55b0d2f524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f3953fff640 1 -- 172.21.15.107:0/3266126760 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0d2bc33f0 con 0x55b0d2f55620 2024-04-25T00:24:06.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f39537fe640 1 --2- 172.21.15.107:0/3266126760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0d2f4a630 0x55b0d2f524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:06.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f3953fff640 1 --2- 172.21.15.107:0/3266126760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f55620 0x55b0d2f52f60 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f394c0b96e0 tx=0x7f394c0bd330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa21f4608de28cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:06.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f3932ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:24:06.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f3932ffd640 1 -- 172.21.15.107:0/3266126760 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c085fb0 con 0x55b0d2f55620 2024-04-25T00:24:06.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f3932ffd640 1 -- 172.21.15.107:0/3266126760 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f394c0b98f0 con 0x55b0d2f55620 2024-04-25T00:24:06.932 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f395cb96740 1 -- 172.21.15.107:0/3266126760 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f394c036040 con 0x55b0d2f55620 2024-04-25T00:24:06.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f3932ffd640 1 -- 172.21.15.107:0/3266126760 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f394c0bcbe0 con 0x55b0d2f55620 2024-04-25T00:24:06.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f395cb96740 10 client.17310 fetch_fsmap learned FSMap version 34 2024-04-25T00:24:06.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.923+0000 7f395cb96740 1 -- 172.21.15.107:0/3266126760 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b0d2b830f0 con 0x55b0d2f55620 2024-04-25T00:24:06.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.924+0000 7f3932ffd640 1 -- 172.21.15.107:0/3266126760 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f394c036040 con 0x55b0d2f55620 2024-04-25T00:24:06.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.924+0000 7f3932ffd640 1 -- 172.21.15.107:0/3266126760 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f394c04e070 con 0x55b0d2f55620 2024-04-25T00:24:06.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.924+0000 7f395cb96740 10 client.17310 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:24:06.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.924+0000 7f395cb96740 10 client.17310 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:06.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.924+0000 7f395cb96740 20 client.17310 populate_metadata read hostname 'smithi107' 2024-04-25T00:24:06.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.924+0000 7f395cb96740 1 --2- 172.21.15.107:0/3266126760 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b0d2f65440 0x55b0d2f67830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:06.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.924+0000 7f395cb96740 4 client.17310 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:06.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.924+0000 7f395cb96740 1 -- 172.21.15.107:0/3266126760 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0d2ebcbf0 con 0x55b0d2f65440 2024-04-25T00:24:06.933 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.924+0000 7f39537fe640 1 --2- 172.21.15.107:0/3266126760 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b0d2f65440 0x55b0d2f67830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:06.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.924+0000 7f3932ffd640 10 client.17310 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:06.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.924+0000 7f39537fe640 1 --2- 172.21.15.107:0/3266126760 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b0d2f65440 0x55b0d2f67830 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e7091aa0715d2ab server_cookie=42555273d5dc3a86 in_seq=0 out_seq=0 2024-04-25T00:24:06.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.925+0000 7f3932ffd640 1 -- 172.21.15.107:0/3266126760 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b0d2ebcbf0 con 0x55b0d2f65440 2024-04-25T00:24:06.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.925+0000 7f3932ffd640 10 client.17310 handle_command_reply: tid=0 2024-04-25T00:24:06.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.925+0000 7f395cb96740 1 client.17310 shutdown 2024-04-25T00:24:06.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.926+0000 7f395cb96740 1 -- 172.21.15.107:0/3266126760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f55620 msgr2=0x55b0d2f52f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:06.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.926+0000 7f395cb96740 1 --2- 172.21.15.107:0/3266126760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f55620 0x55b0d2f52f60 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f394c0b96e0 tx=0x7f394c0bd330 comp rx=0 tx=0).stop 2024-04-25T00:24:06.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.926+0000 7f395cb96740 1 -- 172.21.15.107:0/3266126760 shutdown_connections 2024-04-25T00:24:06.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.926+0000 7f395cb96740 1 --2- 172.21.15.107:0/3266126760 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b0d2f65440 0x55b0d2f67830 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.926+0000 7f395cb96740 1 --2- 172.21.15.107:0/3266126760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f55620 0x55b0d2f52f60 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.926+0000 7f3932ffd640 0 client.17310 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:06.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.926+0000 7f395cb96740 1 --2- 172.21.15.107:0/3266126760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0d2f4af50 0x55b0d2f52a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.926+0000 7f395cb96740 1 --2- 172.21.15.107:0/3266126760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0d2f4a630 0x55b0d2f524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.926+0000 7f395cb96740 1 -- 172.21.15.107:0/3266126760 >> 172.21.15.107:0/3266126760 conn(0x55b0d2f46950 msgr2=0x55b0d2f4c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:06.934 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.926+0000 7f395cb96740 1 -- 172.21.15.107:0/3266126760 shutdown_connections 2024-04-25T00:24:06.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.926+0000 7f395cb96740 1 -- 172.21.15.107:0/3266126760 wait complete. 2024-04-25T00:24:06.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.926+0000 7f395cb96740 20 client.17310 trim_cache size 0 max 16384 2024-04-25T00:24:06.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.943+0000 7f395cb96740 1 Processor -- start 2024-04-25T00:24:06.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.943+0000 7f395cb96740 1 -- start start 2024-04-25T00:24:06.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.943+0000 7f395cb96740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f4a630 0x55b0d2f54c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:06.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.943+0000 7f395cb96740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0d2f4af50 0x55b0d2f52d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:06.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.943+0000 7f395cb96740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0d2f55620 0x55b0d2f4cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:06.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.943+0000 7f395cb96740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f39541b3630 con 0x55b0d2f4a630 2024-04-25T00:24:06.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.943+0000 7f395cb96740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f39541b0f00 con 0x55b0d2f55620 2024-04-25T00:24:06.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.943+0000 7f395cb96740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f39541aed70 con 0x55b0d2f4af50 2024-04-25T00:24:06.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.943+0000 7f39537fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f4a630 0x55b0d2f54c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:06.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.943+0000 7f39537fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f4a630 0x55b0d2f54c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42586/0 (socket says 172.21.15.107:42586) 2024-04-25T00:24:06.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.943+0000 7f39537fe640 1 -- 172.21.15.107:0/2665339563 learned_addr learned my addr 172.21.15.107:0/2665339563 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:06.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.943+0000 7f394bfff640 1 --2- 172.21.15.107:0/2665339563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0d2f4af50 0x55b0d2f52d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:06.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.943+0000 7f3953fff640 1 --2- 172.21.15.107:0/2665339563 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0d2f55620 0x55b0d2f4cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:06.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.944+0000 7f39537fe640 1 -- 172.21.15.107:0/2665339563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0d2f4af50 msgr2=0x55b0d2f52d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:06.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.944+0000 7f39537fe640 1 --2- 172.21.15.107:0/2665339563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0d2f4af50 0x55b0d2f52d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.944+0000 7f39537fe640 1 -- 172.21.15.107:0/2665339563 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0d2f55620 msgr2=0x55b0d2f4cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:06.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.944+0000 7f39537fe640 1 --2- 172.21.15.107:0/2665339563 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0d2f55620 0x55b0d2f4cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.944+0000 7f39537fe640 1 -- 172.21.15.107:0/2665339563 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0d2f50a30 con 0x55b0d2f4a630 2024-04-25T00:24:06.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.944+0000 7f39537fe640 1 --2- 172.21.15.107:0/2665339563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f4a630 0x55b0d2f54c10 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f3944000c00 tx=0x7f39440138f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1617598a559219d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:06.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.944+0000 7f39497fa640 1 -- 172.21.15.107:0/2665339563 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39440040f0 con 0x55b0d2f4a630 2024-04-25T00:24:06.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.944+0000 7f39497fa640 1 -- 172.21.15.107:0/2665339563 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3944004290 con 0x55b0d2f4a630 2024-04-25T00:24:06.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.944+0000 7f39497fa640 1 -- 172.21.15.107:0/2665339563 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3944003cc0 con 0x55b0d2f4a630 2024-04-25T00:24:06.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.944+0000 7f395cb96740 1 -- 172.21.15.107:0/2665339563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f4a630 msgr2=0x55b0d2f54c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:06.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.944+0000 7f395cb96740 1 --2- 172.21.15.107:0/2665339563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f4a630 0x55b0d2f54c10 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f3944000c00 tx=0x7f39440138f0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.944+0000 7f395cb96740 1 -- 172.21.15.107:0/2665339563 shutdown_connections 2024-04-25T00:24:06.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.944+0000 7f395cb96740 1 --2- 172.21.15.107:0/2665339563 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0d2f55620 0x55b0d2f4cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.944+0000 7f395cb96740 1 --2- 172.21.15.107:0/2665339563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0d2f4af50 0x55b0d2f52d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.944+0000 7f395cb96740 1 --2- 172.21.15.107:0/2665339563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f4a630 0x55b0d2f54c10 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.944+0000 7f395cb96740 1 -- 172.21.15.107:0/2665339563 >> 172.21.15.107:0/2665339563 conn(0x55b0d2f46950 msgr2=0x55b0d2f45260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:06.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.944+0000 7f395cb96740 1 -- 172.21.15.107:0/2665339563 shutdown_connections 2024-04-25T00:24:06.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.945+0000 7f395cb96740 1 -- 172.21.15.107:0/2665339563 wait complete. 2024-04-25T00:24:06.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.945+0000 7f395cb96740 1 -- start start 2024-04-25T00:24:06.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.945+0000 7f395cb96740 1 Processor -- start 2024-04-25T00:24:06.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.945+0000 7f395cb96740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f4a630 0x55b0d2f590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:06.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.945+0000 7f395cb96740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0d2f4af50 0x55b0d2f59630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:06.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.945+0000 7f395cb96740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0d2f55620 0x55b0d2f59b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:06.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.945+0000 7f395cb96740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f39541af3c0 con 0x55b0d2f4a630 2024-04-25T00:24:06.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.945+0000 7f395cb96740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f39541b0830 con 0x55b0d2f4af50 2024-04-25T00:24:06.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.945+0000 7f395cb96740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f39541afa90 con 0x55b0d2f55620 2024-04-25T00:24:06.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.945+0000 7f395cb96740 10 client.0 _ll_register_callbacks cb 0x7f394c0c5ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:06.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.945+0000 7f39537fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f4a630 0x55b0d2f590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:06.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.945+0000 7f39537fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f4a630 0x55b0d2f590f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42588/0 (socket says 172.21.15.107:42588) 2024-04-25T00:24:06.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.945+0000 7f39537fe640 1 -- 172.21.15.107:0/3755032837 learned_addr learned my addr 172.21.15.107:0/3755032837 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:06.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.946+0000 7f394bfff640 1 --2- 172.21.15.107:0/3755032837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0d2f4af50 0x55b0d2f59630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:06.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.946+0000 7f39537fe640 1 -- 172.21.15.107:0/3755032837 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0d2f55620 msgr2=0x55b0d2f59b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:06.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.946+0000 7f39537fe640 1 --2- 172.21.15.107:0/3755032837 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0d2f55620 0x55b0d2f59b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.946+0000 7f39537fe640 1 -- 172.21.15.107:0/3755032837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0d2f4af50 msgr2=0x55b0d2f59630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:06.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.946+0000 7f39537fe640 1 --2- 172.21.15.107:0/3755032837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0d2f4af50 0x55b0d2f59630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.946+0000 7f39537fe640 1 -- 172.21.15.107:0/3755032837 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0d2fcab70 con 0x55b0d2f4a630 2024-04-25T00:24:06.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.946+0000 7f39537fe640 1 --2- 172.21.15.107:0/3755032837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f4a630 0x55b0d2f590f0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f394400bfd0 tx=0x7f394400b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36969d0759e4f5f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:06.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.946+0000 7f3932ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:24:06.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.946+0000 7f3932ffd640 1 -- 172.21.15.107:0/3755032837 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394401d470 con 0x55b0d2f4a630 2024-04-25T00:24:06.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.946+0000 7f3932ffd640 1 -- 172.21.15.107:0/3755032837 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f394401d610 con 0x55b0d2f4a630 2024-04-25T00:24:06.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.946+0000 7f395cb96740 1 -- 172.21.15.107:0/3755032837 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3944004290 con 0x55b0d2f4a630 2024-04-25T00:24:06.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.946+0000 7f3932ffd640 1 -- 172.21.15.107:0/3755032837 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3944038050 con 0x55b0d2f4a630 2024-04-25T00:24:06.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.946+0000 7f395cb96740 10 client.17316 fetch_fsmap learned FSMap version 34 2024-04-25T00:24:06.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.946+0000 7f395cb96740 1 -- 172.21.15.107:0/3755032837 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b0d2fcbe40 con 0x55b0d2f4a630 2024-04-25T00:24:06.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.946+0000 7f3932ffd640 1 -- 172.21.15.107:0/3755032837 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3944004290 con 0x55b0d2f4a630 2024-04-25T00:24:06.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.946+0000 7f3932ffd640 1 -- 172.21.15.107:0/3755032837 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f39440106c0 con 0x55b0d2f4a630 2024-04-25T00:24:06.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.947+0000 7f395cb96740 10 client.17316 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:24:06.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.947+0000 7f395cb96740 10 client.17316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:06.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.947+0000 7f395cb96740 20 client.17316 populate_metadata read hostname 'smithi107' 2024-04-25T00:24:06.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.947+0000 7f395cb96740 1 --2- 172.21.15.107:0/3755032837 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b0d2fcc530 0x55b0d2fce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:06.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.947+0000 7f395cb96740 4 client.17316 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:24:06.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.947+0000 7f395cb96740 1 -- 172.21.15.107:0/3755032837 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b0d2eb6f40 con 0x55b0d2fcc530 2024-04-25T00:24:06.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.947+0000 7f394bfff640 1 --2- 172.21.15.107:0/3755032837 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b0d2fcc530 0x55b0d2fce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:06.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.947+0000 7f394bfff640 1 --2- 172.21.15.107:0/3755032837 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b0d2fcc530 0x55b0d2fce970 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9109415e29225f7c server_cookie=41d784fed15bf733 in_seq=0 out_seq=0 2024-04-25T00:24:06.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.947+0000 7f3932ffd640 10 client.17316 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:06.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.947+0000 7f3932ffd640 1 -- 172.21.15.107:0/3755032837 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b0d2eb6f40 con 0x55b0d2fcc530 2024-04-25T00:24:06.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.947+0000 7f3932ffd640 10 client.17316 handle_command_reply: tid=0 2024-04-25T00:24:06.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.947+0000 7f395cb96740 1 client.17316 shutdown 2024-04-25T00:24:06.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.947+0000 7f395cb96740 1 -- 172.21.15.107:0/3755032837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f4a630 msgr2=0x55b0d2f590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:06.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.947+0000 7f395cb96740 1 --2- 172.21.15.107:0/3755032837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f4a630 0x55b0d2f590f0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f394400bfd0 tx=0x7f394400b7b0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.948+0000 7f395cb96740 1 -- 172.21.15.107:0/3755032837 shutdown_connections 2024-04-25T00:24:06.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.948+0000 7f395cb96740 1 --2- 172.21.15.107:0/3755032837 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b0d2fcc530 0x55b0d2fce970 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.948+0000 7f395cb96740 1 --2- 172.21.15.107:0/3755032837 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b0d2f55620 0x55b0d2f59b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.948+0000 7f395cb96740 1 --2- 172.21.15.107:0/3755032837 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0d2f4af50 0x55b0d2f59630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.948+0000 7f3932ffd640 0 client.17316 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:06.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.948+0000 7f395cb96740 1 --2- 172.21.15.107:0/3755032837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0d2f4a630 0x55b0d2f590f0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.948+0000 7f395cb96740 1 -- 172.21.15.107:0/3755032837 >> 172.21.15.107:0/3755032837 conn(0x55b0d2f46950 msgr2=0x55b0d2f52b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:06.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.948+0000 7f395cb96740 1 -- 172.21.15.107:0/3755032837 shutdown_connections 2024-04-25T00:24:06.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.948+0000 7f395cb96740 1 -- 172.21.15.107:0/3755032837 wait complete. 2024-04-25T00:24:06.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.948+0000 7f395cb96740 20 client.17316 trim_cache size 0 max 16384 2024-04-25T00:24:06.956 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:24:06.956 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-04-25T00:24:06.956 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-04-25T00:24:06.956 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:24:06.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.948+0000 7f3959e8c640 1 -- 172.21.15.107:0/1667104587 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f39400a2d00 msgr2=0x7f39400a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:06.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.948+0000 7f3959e8c640 1 --2- 172.21.15.107:0/1667104587 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f39400a2d00 0x7f39400a51c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f39540febf0 tx=0x7f393c0138e0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.948+0000 7f3959e8c640 1 -- 172.21.15.107:0/1667104587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3954107720 msgr2=0x7f395419eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:06.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.948+0000 7f3959e8c640 1 --2- 172.21.15.107:0/1667104587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3954107720 0x7f395419eab0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f394c005cc0 tx=0x7f394c003970 comp rx=0 tx=0).stop 2024-04-25T00:24:06.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.948+0000 7f3959e8c640 1 -- 172.21.15.107:0/1667104587 shutdown_connections 2024-04-25T00:24:06.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.948+0000 7f3959e8c640 1 --2- 172.21.15.107:0/1667104587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3954107720 0x7f395419eab0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.948+0000 7f3959e8c640 1 --2- 172.21.15.107:0/1667104587 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f39400a2d00 0x7f39400a51c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.948+0000 7f3959e8c640 1 --2- 172.21.15.107:0/1667104587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f39541020d0 0x7f395419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.948+0000 7f3959e8c640 1 --2- 172.21.15.107:0/1667104587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3954101710 0x7f395419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:06.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.948+0000 7f3959e8c640 1 -- 172.21.15.107:0/1667104587 >> 172.21.15.107:0/1667104587 conn(0x7f3954078b90 msgr2=0x7f395410ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:06.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.948+0000 7f3959e8c640 1 -- 172.21.15.107:0/1667104587 shutdown_connections 2024-04-25T00:24:06.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:06.948+0000 7f3959e8c640 1 -- 172.21.15.107:0/1667104587 wait complete. 2024-04-25T00:24:06.959 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:24:06.960 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:24:07.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.057+0000 7f36bc7f9640 1 -- 172.21.15.107:0/4051428356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36b4106630 msgr2=0x7f36b4106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.058+0000 7f36bc7f9640 1 --2- 172.21.15.107:0/4051428356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36b4106630 0x7f36b4106a10 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009930 tx=0x7f36a402a420 comp rx=0 tx=0).stop 2024-04-25T00:24:07.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.058+0000 7f36bc7f9640 1 -- 172.21.15.107:0/4051428356 shutdown_connections 2024-04-25T00:24:07.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.058+0000 7f36bc7f9640 1 --2- 172.21.15.107:0/4051428356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36b4100fe0 0x7f36b410d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.058+0000 7f36bc7f9640 1 --2- 172.21.15.107:0/4051428356 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36b4100620 0x7f36b4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.058+0000 7f36bc7f9640 1 --2- 172.21.15.107:0/4051428356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36b4106630 0x7f36b4106a10 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.058+0000 7f36bc7f9640 1 -- 172.21.15.107:0/4051428356 >> 172.21.15.107:0/4051428356 conn(0x7f36b4077740 msgr2=0x7f36b4077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.058+0000 7f36bc7f9640 1 -- 172.21.15.107:0/4051428356 shutdown_connections 2024-04-25T00:24:07.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.058+0000 7f36bc7f9640 1 -- 172.21.15.107:0/4051428356 wait complete. 2024-04-25T00:24:07.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.058+0000 7f36bc7f9640 1 Processor -- start 2024-04-25T00:24:07.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.058+0000 7f36bc7f9640 1 -- start start 2024-04-25T00:24:07.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.058+0000 7f36bc7f9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36b4100620 0x7f36b4198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.058+0000 7f36bc7f9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36b4100fe0 0x7f36b41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.058+0000 7f36bc7f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36b4106630 0x7f36b419c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.058+0000 7f36bc7f9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f36b4110a40 con 0x7f36b4106630 2024-04-25T00:24:07.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.058+0000 7f36bc7f9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f36b41108c0 con 0x7f36b4100620 2024-04-25T00:24:07.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.058+0000 7f36bc7f9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f36b4110bc0 con 0x7f36b4100fe0 2024-04-25T00:24:07.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.059+0000 7f36bad96640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36b4106630 0x7f36b419c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.059+0000 7f36bad96640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36b4106630 0x7f36b419c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42614/0 (socket says 172.21.15.107:42614) 2024-04-25T00:24:07.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.059+0000 7f36bad96640 1 -- 172.21.15.107:0/2297970245 learned_addr learned my addr 172.21.15.107:0/2297970245 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:07.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.059+0000 7f36ba595640 1 --2- 172.21.15.107:0/2297970245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36b4100620 0x7f36b4198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.059+0000 7f36b9d94640 1 --2- 172.21.15.107:0/2297970245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36b4100fe0 0x7f36b41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.059+0000 7f36bad96640 1 -- 172.21.15.107:0/2297970245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36b4100fe0 msgr2=0x7f36b41985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.059+0000 7f36bad96640 1 --2- 172.21.15.107:0/2297970245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36b4100fe0 0x7f36b41985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.059+0000 7f36bad96640 1 -- 172.21.15.107:0/2297970245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36b4100620 msgr2=0x7f36b4198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.059+0000 7f36bad96640 1 --2- 172.21.15.107:0/2297970245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36b4100620 0x7f36b4198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.059+0000 7f36bad96640 1 -- 172.21.15.107:0/2297970245 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b419d060 con 0x7f36b4106630 2024-04-25T00:24:07.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.059+0000 7f36bad96640 1 --2- 172.21.15.107:0/2297970245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36b4106630 0x7f36b419c930 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009fd0 tx=0x7f36ac00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.059+0000 7f36ab7fe640 1 -- 172.21.15.107:0/2297970245 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac017070 con 0x7f36b4106630 2024-04-25T00:24:07.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.059+0000 7f36ab7fe640 1 -- 172.21.15.107:0/2297970245 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36ac003e50 con 0x7f36b4106630 2024-04-25T00:24:07.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.059+0000 7f36bc7f9640 1 -- 172.21.15.107:0/2297970245 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b419d350 con 0x7f36b4106630 2024-04-25T00:24:07.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.059+0000 7f36ab7fe640 1 -- 172.21.15.107:0/2297970245 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac0106c0 con 0x7f36b4106630 2024-04-25T00:24:07.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.059+0000 7f36bc7f9640 1 -- 172.21.15.107:0/2297970245 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b41a4be0 con 0x7f36b4106630 2024-04-25T00:24:07.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.060+0000 7f36bc7f9640 1 -- 172.21.15.107:0/2297970245 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36b4068560 con 0x7f36b4106630 2024-04-25T00:24:07.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.061+0000 7f36ab7fe640 1 -- 172.21.15.107:0/2297970245 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f36ac003970 con 0x7f36b4106630 2024-04-25T00:24:07.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.062+0000 7f36ab7fe640 1 --2- 172.21.15.107:0/2297970245 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f36a00a2d00 0x7f36a00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.062+0000 7f36ba595640 1 --2- 172.21.15.107:0/2297970245 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f36a00a2d00 0x7f36a00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.062+0000 7f36ab7fe640 1 -- 172.21.15.107:0/2297970245 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7f36ac0cf4a0 con 0x7f36b4106630 2024-04-25T00:24:07.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.062+0000 7f36ba595640 1 --2- 172.21.15.107:0/2297970245 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f36a00a2d00 0x7f36a00a51c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009900 tx=0x7f36a40058d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.066+0000 7f36ab7fe640 1 -- 172.21.15.107:0/2297970245 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f36ac09bda0 con 0x7f36b4106630 2024-04-25T00:24:07.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.278+0000 7f36bc7f9640 1 -- 172.21.15.107:0/2297970245 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36b4109610 con 0x7f36b4106630 2024-04-25T00:24:07.279 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.279+0000 7f36ab7fe640 1 -- 172.21.15.107:0/2297970245 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11897 (secure 0 0 0) 0x7f36ac09b6a0 con 0x7f36b4106630 2024-04-25T00:24:07.280 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:24:07.280 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:22:27.402680+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24493,24521]},"id":1}]} 2024-04-25T00:24:07.280 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 34 2024-04-25T00:24:07.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.281+0000 7f36bc7f9640 1 -- 172.21.15.107:0/2297970245 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f36a00a2d00 msgr2=0x7f36a00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.281+0000 7f36bc7f9640 1 --2- 172.21.15.107:0/2297970245 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f36a00a2d00 0x7f36a00a51c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009900 tx=0x7f36a40058d0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.281+0000 7f36bc7f9640 1 -- 172.21.15.107:0/2297970245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36b4106630 msgr2=0x7f36b419c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.281+0000 7f36bc7f9640 1 --2- 172.21.15.107:0/2297970245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36b4106630 0x7f36b419c930 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009fd0 tx=0x7f36ac00b260 comp rx=0 tx=0).stop 2024-04-25T00:24:07.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.281+0000 7f36bc7f9640 1 -- 172.21.15.107:0/2297970245 shutdown_connections 2024-04-25T00:24:07.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.281+0000 7f36bc7f9640 1 --2- 172.21.15.107:0/2297970245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36b4106630 0x7f36b419c930 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.281+0000 7f36bc7f9640 1 --2- 172.21.15.107:0/2297970245 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f36b4100fe0 0x7f36b41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.281+0000 7f36bc7f9640 1 --2- 172.21.15.107:0/2297970245 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f36a00a2d00 0x7f36a00a51c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.281+0000 7f36bc7f9640 1 --2- 172.21.15.107:0/2297970245 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f36b4100620 0x7f36b4198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.281+0000 7f36bc7f9640 1 -- 172.21.15.107:0/2297970245 >> 172.21.15.107:0/2297970245 conn(0x7f36b4077740 msgr2=0x7f36b4068c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.281+0000 7f36bc7f9640 1 -- 172.21.15.107:0/2297970245 shutdown_connections 2024-04-25T00:24:07.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.281+0000 7f36bc7f9640 1 -- 172.21.15.107:0/2297970245 wait complete. 2024-04-25T00:24:07.293 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:24:07.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.392+0000 7fe2b71ee640 1 -- 172.21.15.107:0/3238039018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2b0107730 msgr2=0x7fe2b0107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.392+0000 7fe2b71ee640 1 --2- 172.21.15.107:0/3238039018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2b0107730 0x7fe2b0107b10 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009930 tx=0x7fe2a402a420 comp rx=0 tx=0).stop 2024-04-25T00:24:07.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.392+0000 7fe2b71ee640 1 -- 172.21.15.107:0/3238039018 shutdown_connections 2024-04-25T00:24:07.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.392+0000 7fe2b71ee640 1 --2- 172.21.15.107:0/3238039018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2b006a780 0x7fe2b010e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.392+0000 7fe2b71ee640 1 --2- 172.21.15.107:0/3238039018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b0069dc0 0x7fe2b006a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.392+0000 7fe2b71ee640 1 --2- 172.21.15.107:0/3238039018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2b0107730 0x7fe2b0107b10 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.392+0000 7fe2b71ee640 1 -- 172.21.15.107:0/3238039018 >> 172.21.15.107:0/3238039018 conn(0x7fe2b00776a0 msgr2=0x7fe2b0077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.392+0000 7fe2b71ee640 1 -- 172.21.15.107:0/3238039018 shutdown_connections 2024-04-25T00:24:07.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.392+0000 7fe2b71ee640 1 -- 172.21.15.107:0/3238039018 wait complete. 2024-04-25T00:24:07.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.393+0000 7fe2b71ee640 1 Processor -- start 2024-04-25T00:24:07.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.393+0000 7fe2b71ee640 1 -- start start 2024-04-25T00:24:07.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.393+0000 7fe2b71ee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2b0069dc0 0x7fe2b0198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.393+0000 7fe2b71ee640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2b006a780 0x7fe2b01985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.393+0000 7fe2b71ee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b0107730 0x7fe2b019c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.393+0000 7fe2b71ee640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b0111b40 con 0x7fe2b0069dc0 2024-04-25T00:24:07.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.393+0000 7fe2b71ee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b01119c0 con 0x7fe2b0107730 2024-04-25T00:24:07.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.393+0000 7fe2b71ee640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b0111cc0 con 0x7fe2b006a780 2024-04-25T00:24:07.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.393+0000 7fe2b4f8a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2b0069dc0 0x7fe2b0198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.393+0000 7fe2b4f8a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2b0069dc0 0x7fe2b0198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42638/0 (socket says 172.21.15.107:42638) 2024-04-25T00:24:07.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.393+0000 7fe2b4f8a640 1 -- 172.21.15.107:0/4123009205 learned_addr learned my addr 172.21.15.107:0/4123009205 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:07.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.393+0000 7fe2b578b640 1 --2- 172.21.15.107:0/4123009205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b0107730 0x7fe2b019c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.393+0000 7fe2affff640 1 --2- 172.21.15.107:0/4123009205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2b006a780 0x7fe2b01985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.393+0000 7fe2b4f8a640 1 -- 172.21.15.107:0/4123009205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2b006a780 msgr2=0x7fe2b01985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.393+0000 7fe2b4f8a640 1 --2- 172.21.15.107:0/4123009205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2b006a780 0x7fe2b01985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.393+0000 7fe2b4f8a640 1 -- 172.21.15.107:0/4123009205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b0107730 msgr2=0x7fe2b019c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.393+0000 7fe2b4f8a640 1 --2- 172.21.15.107:0/4123009205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b0107730 0x7fe2b019c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.393+0000 7fe2b4f8a640 1 -- 172.21.15.107:0/4123009205 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b019d000 con 0x7fe2b0069dc0 2024-04-25T00:24:07.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.393+0000 7fe2b578b640 1 --2- 172.21.15.107:0/4123009205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b0107730 0x7fe2b019c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:07.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.393+0000 7fe2b4f8a640 1 --2- 172.21.15.107:0/4123009205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2b0069dc0 0x7fe2b0198060 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009a50 tx=0x7fe2a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.393+0000 7fe2adffb640 1 -- 172.21.15.107:0/4123009205 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a403f030 con 0x7fe2b0069dc0 2024-04-25T00:24:07.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.394+0000 7fe2adffb640 1 -- 172.21.15.107:0/4123009205 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2a4003ce0 con 0x7fe2b0069dc0 2024-04-25T00:24:07.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.394+0000 7fe2b71ee640 1 -- 172.21.15.107:0/4123009205 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b019d290 con 0x7fe2b0069dc0 2024-04-25T00:24:07.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.394+0000 7fe2adffb640 1 -- 172.21.15.107:0/4123009205 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4035070 con 0x7fe2b0069dc0 2024-04-25T00:24:07.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.394+0000 7fe2b71ee640 1 -- 172.21.15.107:0/4123009205 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b01a4b80 con 0x7fe2b0069dc0 2024-04-25T00:24:07.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.396+0000 7fe2adffb640 1 -- 172.21.15.107:0/4123009205 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe2a4042030 con 0x7fe2b0069dc0 2024-04-25T00:24:07.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.396+0000 7fe2b9fdd740 1 Processor -- start 2024-04-25T00:24:07.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.396+0000 7fe2b9fdd740 1 -- start start 2024-04-25T00:24:07.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.396+0000 7fe2adffb640 1 --2- 172.21.15.107:0/4123009205 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe2a00a2d00 0x7fe2a00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.396+0000 7fe2b9fdd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e26f6ac2b0 0x55e26f6ac690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.396+0000 7fe2b9fdd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6acc60 0x55e26f6bb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.396+0000 7fe2b9fdd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e26f6bb980 0x55e26f6bdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.397+0000 7fe2b9fdd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e26f611670 con 0x55e26f6acc60 2024-04-25T00:24:07.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.397+0000 7fe2b9fdd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e26f5b9970 con 0x55e26f6ac2b0 2024-04-25T00:24:07.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.397+0000 7fe2b9fdd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e26f60d630 con 0x55e26f6bb980 2024-04-25T00:24:07.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.397+0000 7fe2b578b640 1 --2- 172.21.15.107:0/4123009205 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe2a00a2d00 0x7fe2a00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.397+0000 7fe2adffb640 1 -- 172.21.15.107:0/4123009205 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fe2a40efe40 con 0x7fe2b0069dc0 2024-04-25T00:24:07.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.397+0000 7fe2affff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6acc60 0x55e26f6bb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.397+0000 7fe2affff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6acc60 0x55e26f6bb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42654/0 (socket says 172.21.15.107:42654) 2024-04-25T00:24:07.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.397+0000 7fe2affff640 1 -- 172.21.15.107:0/2579219431 learned_addr learned my addr 172.21.15.107:0/2579219431 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:07.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.397+0000 7fe2b578b640 1 --2- 172.21.15.107:0/2579219431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e26f6bb980 0x55e26f6bdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.397+0000 7fe2b4f8a640 1 --2- 172.21.15.107:0/2579219431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e26f6ac2b0 0x55e26f6ac690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.397+0000 7fe2affff640 1 -- 172.21.15.107:0/2579219431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e26f6bb980 msgr2=0x55e26f6bdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.397+0000 7fe2affff640 1 --2- 172.21.15.107:0/2579219431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e26f6bb980 0x55e26f6bdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.397+0000 7fe2affff640 1 -- 172.21.15.107:0/2579219431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e26f6ac2b0 msgr2=0x55e26f6ac690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.397+0000 7fe2affff640 1 --2- 172.21.15.107:0/2579219431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e26f6ac2b0 0x55e26f6ac690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.397+0000 7fe2affff640 1 -- 172.21.15.107:0/2579219431 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e26f5dd400 con 0x55e26f6acc60 2024-04-25T00:24:07.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.397+0000 7fe2b578b640 1 --2- 172.21.15.107:0/2579219431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e26f6bb980 0x55e26f6bdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:07.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.397+0000 7fe2b578b640 1 --2- 172.21.15.107:0/4123009205 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe2a00a2d00 0x7fe2a00a51c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800c810 tx=0x7fe2a800cc10 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.397+0000 7fe2affff640 1 --2- 172.21.15.107:0/2579219431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6acc60 0x55e26f6bb440 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00cbc0 tx=0x7fe29c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98af226cc14022ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.397+0000 7fe29affd640 1 -- 172.21.15.107:0/2579219431 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c010630 con 0x55e26f6acc60 2024-04-25T00:24:07.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.397+0000 7fe29affd640 1 -- 172.21.15.107:0/2579219431 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe29c010cb0 con 0x55e26f6acc60 2024-04-25T00:24:07.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.397+0000 7fe29affd640 1 -- 172.21.15.107:0/2579219431 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c01abc0 con 0x55e26f6acc60 2024-04-25T00:24:07.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.398+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/2579219431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6acc60 msgr2=0x55e26f6bb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.398+0000 7fe2b9fdd740 1 --2- 172.21.15.107:0/2579219431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6acc60 0x55e26f6bb440 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00cbc0 tx=0x7fe29c00e720 comp rx=0 tx=0).stop 2024-04-25T00:24:07.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.398+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/2579219431 shutdown_connections 2024-04-25T00:24:07.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.398+0000 7fe2b9fdd740 1 --2- 172.21.15.107:0/2579219431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e26f6bb980 0x55e26f6bdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.398+0000 7fe2b9fdd740 1 --2- 172.21.15.107:0/2579219431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6acc60 0x55e26f6bb440 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.398+0000 7fe2b9fdd740 1 --2- 172.21.15.107:0/2579219431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e26f6ac2b0 0x55e26f6ac690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.398+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/2579219431 >> 172.21.15.107:0/2579219431 conn(0x55e26f6a88a0 msgr2=0x55e26f6a8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.398+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/2579219431 shutdown_connections 2024-04-25T00:24:07.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.398+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/2579219431 wait complete. 2024-04-25T00:24:07.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.398+0000 7fe2b9fdd740 1 -- start start 2024-04-25T00:24:07.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.398+0000 7fe2b9fdd740 1 Processor -- start 2024-04-25T00:24:07.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.399+0000 7fe2b9fdd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e26f6ac2b0 0x55e26f6bb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.399+0000 7fe2b9fdd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e26f6acc60 0x55e26f6b85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.399+0000 7fe2b9fdd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6bb980 0x55e26f6b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.399+0000 7fe2b9fdd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e26f6be740 con 0x55e26f6bb980 2024-04-25T00:24:07.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.399+0000 7fe2b9fdd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e26f5cd050 con 0x55e26f6acc60 2024-04-25T00:24:07.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.399+0000 7fe2b9fdd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55e26f6be8c0 con 0x55e26f6ac2b0 2024-04-25T00:24:07.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.399+0000 7fe2b9fdd740 10 client.0 _ll_register_callbacks cb 0x55e26f324be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:07.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.399+0000 7fe2b578b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6bb980 0x55e26f6b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.399+0000 7fe2b578b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6bb980 0x55e26f6b8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42668/0 (socket says 172.21.15.107:42668) 2024-04-25T00:24:07.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.399+0000 7fe2b578b640 1 -- 172.21.15.107:0/1323691847 learned_addr learned my addr 172.21.15.107:0/1323691847 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:07.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.399+0000 7fe2affff640 1 --2- 172.21.15.107:0/1323691847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e26f6acc60 0x55e26f6b85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.399+0000 7fe2b4f8a640 1 --2- 172.21.15.107:0/1323691847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e26f6ac2b0 0x55e26f6bb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.399+0000 7fe2b578b640 1 -- 172.21.15.107:0/1323691847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e26f6ac2b0 msgr2=0x55e26f6bb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.399+0000 7fe2b578b640 1 --2- 172.21.15.107:0/1323691847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e26f6ac2b0 0x55e26f6bb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.399+0000 7fe2b578b640 1 -- 172.21.15.107:0/1323691847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e26f6acc60 msgr2=0x55e26f6b85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.399+0000 7fe2b578b640 1 --2- 172.21.15.107:0/1323691847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e26f6acc60 0x55e26f6b85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.399+0000 7fe2b578b640 1 -- 172.21.15.107:0/1323691847 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e26f59edc0 con 0x55e26f6bb980 2024-04-25T00:24:07.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.399+0000 7fe2b4f8a640 1 --2- 172.21.15.107:0/1323691847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e26f6ac2b0 0x55e26f6bb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:07.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.399+0000 7fe2b578b640 1 --2- 172.21.15.107:0/1323691847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6bb980 0x55e26f6b8b40 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fe2a801a690 tx=0x7fe2a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bffd7ee6374caff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.399+0000 7fe28ffff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:24:07.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.399+0000 7fe28ffff640 1 -- 172.21.15.107:0/1323691847 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8029060 con 0x55e26f6bb980 2024-04-25T00:24:07.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.399+0000 7fe28ffff640 1 -- 172.21.15.107:0/1323691847 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fe2a8024440 con 0x55e26f6bb980 2024-04-25T00:24:07.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.399+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/1323691847 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe29c010cb0 con 0x55e26f6bb980 2024-04-25T00:24:07.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.400+0000 7fe28ffff640 1 -- 172.21.15.107:0/1323691847 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe2a8030050 con 0x55e26f6bb980 2024-04-25T00:24:07.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.400+0000 7fe2b9fdd740 10 client.17334 fetch_fsmap learned FSMap version 34 2024-04-25T00:24:07.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.400+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/1323691847 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e26f3253f0 con 0x55e26f6bb980 2024-04-25T00:24:07.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.400+0000 7fe28ffff640 1 -- 172.21.15.107:0/1323691847 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe29c010cb0 con 0x55e26f6bb980 2024-04-25T00:24:07.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.400+0000 7fe28ffff640 1 -- 172.21.15.107:0/1323691847 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fe2a801c690 con 0x55e26f6bb980 2024-04-25T00:24:07.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.400+0000 7fe2b9fdd740 10 client.17334 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:24:07.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.400+0000 7fe2b9fdd740 10 client.17334 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:24:07.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.400+0000 7fe2b9fdd740 20 client.17334 populate_metadata read hostname 'smithi107' 2024-04-25T00:24:07.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.400+0000 7fe2b9fdd740 1 --2- 172.21.15.107:0/1323691847 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55e26f6cba10 0x55e26f6cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.400+0000 7fe2b9fdd740 4 client.17334 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:07.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.400+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/1323691847 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e26f61ebb0 con 0x55e26f6cba10 2024-04-25T00:24:07.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.400+0000 7fe2b4f8a640 1 --2- 172.21.15.107:0/1323691847 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55e26f6cba10 0x55e26f6cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.401+0000 7fe2b4f8a640 1 --2- 172.21.15.107:0/1323691847 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55e26f6cba10 0x55e26f6cde70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8ef749e15e4c1de0 server_cookie=8894df43294a2a02 in_seq=0 out_seq=0 2024-04-25T00:24:07.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.401+0000 7fe28ffff640 10 client.17334 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:24:07.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.401+0000 7fe28ffff640 1 -- 172.21.15.107:0/1323691847 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e26f61ebb0 con 0x55e26f6cba10 2024-04-25T00:24:07.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.402+0000 7fe28ffff640 10 client.17334 handle_command_reply: tid=0 2024-04-25T00:24:07.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.402+0000 7fe2b9fdd740 1 client.17334 shutdown 2024-04-25T00:24:07.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.402+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/1323691847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6bb980 msgr2=0x55e26f6b8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.402+0000 7fe2b9fdd740 1 --2- 172.21.15.107:0/1323691847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6bb980 0x55e26f6b8b40 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fe2a801a690 tx=0x7fe2a8004a40 comp rx=0 tx=0).stop 2024-04-25T00:24:07.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.402+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/1323691847 shutdown_connections 2024-04-25T00:24:07.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.402+0000 7fe2b9fdd740 1 --2- 172.21.15.107:0/1323691847 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55e26f6cba10 0x55e26f6cde70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.402+0000 7fe2b9fdd740 1 --2- 172.21.15.107:0/1323691847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6bb980 0x55e26f6b8b40 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.402+0000 7fe2b9fdd740 1 --2- 172.21.15.107:0/1323691847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e26f6acc60 0x55e26f6b85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.402+0000 7fe28ffff640 0 client.17334 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:24:07.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.402+0000 7fe2b9fdd740 1 --2- 172.21.15.107:0/1323691847 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e26f6ac2b0 0x55e26f6bb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.402+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/1323691847 >> 172.21.15.107:0/1323691847 conn(0x55e26f6a88a0 msgr2=0x55e26f6ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.402+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/1323691847 shutdown_connections 2024-04-25T00:24:07.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.402+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/1323691847 wait complete. 2024-04-25T00:24:07.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.402+0000 7fe2b9fdd740 20 client.17334 trim_cache size 0 max 16384 2024-04-25T00:24:07.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.415+0000 7fe2b9fdd740 1 Processor -- start 2024-04-25T00:24:07.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.415+0000 7fe2b9fdd740 1 -- start start 2024-04-25T00:24:07.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.415+0000 7fe2b9fdd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e26f6ac2b0 0x55e26f6c7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.415+0000 7fe2b9fdd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6acc60 0x55e26f6d3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.415+0000 7fe2b9fdd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e26f6bb980 0x55e26f6d3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.415+0000 7fe2b9fdd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b01af2f0 con 0x55e26f6acc60 2024-04-25T00:24:07.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.415+0000 7fe2b9fdd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b01acbc0 con 0x55e26f6ac2b0 2024-04-25T00:24:07.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.415+0000 7fe2b9fdd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b01aaa30 con 0x55e26f6bb980 2024-04-25T00:24:07.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.415+0000 7fe2affff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6acc60 0x55e26f6d3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.415+0000 7fe2affff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6acc60 0x55e26f6d3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42680/0 (socket says 172.21.15.107:42680) 2024-04-25T00:24:07.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.415+0000 7fe2affff640 1 -- 172.21.15.107:0/1683129627 learned_addr learned my addr 172.21.15.107:0/1683129627 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:07.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.415+0000 7fe2b4f8a640 1 --2- 172.21.15.107:0/1683129627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e26f6ac2b0 0x55e26f6c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.415+0000 7fe2affff640 1 -- 172.21.15.107:0/1683129627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e26f6bb980 msgr2=0x55e26f6d3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.415+0000 7fe2affff640 1 --2- 172.21.15.107:0/1683129627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e26f6bb980 0x55e26f6d3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.415+0000 7fe2affff640 1 -- 172.21.15.107:0/1683129627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e26f6ac2b0 msgr2=0x55e26f6c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.416+0000 7fe2affff640 1 --2- 172.21.15.107:0/1683129627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e26f6ac2b0 0x55e26f6c7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.416+0000 7fe2affff640 1 -- 172.21.15.107:0/1683129627 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e26f6b8eb0 con 0x55e26f6acc60 2024-04-25T00:24:07.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.416+0000 7fe2b4f8a640 1 --2- 172.21.15.107:0/1683129627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e26f6ac2b0 0x55e26f6c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:07.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.416+0000 7fe2affff640 1 --2- 172.21.15.107:0/1683129627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6acc60 0x55e26f6d3740 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00cbc0 tx=0x7fe29c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a383199f5a0d31f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.416+0000 7fe29a7fc640 1 -- 172.21.15.107:0/1683129627 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c004220 con 0x55e26f6acc60 2024-04-25T00:24:07.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.416+0000 7fe29a7fc640 1 -- 172.21.15.107:0/1683129627 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe29c0043c0 con 0x55e26f6acc60 2024-04-25T00:24:07.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.416+0000 7fe29a7fc640 1 -- 172.21.15.107:0/1683129627 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c004e00 con 0x55e26f6acc60 2024-04-25T00:24:07.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.416+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/1683129627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6acc60 msgr2=0x55e26f6d3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.416+0000 7fe2b9fdd740 1 --2- 172.21.15.107:0/1683129627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6acc60 0x55e26f6d3740 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00cbc0 tx=0x7fe29c00c860 comp rx=0 tx=0).stop 2024-04-25T00:24:07.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.416+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/1683129627 shutdown_connections 2024-04-25T00:24:07.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.416+0000 7fe2b9fdd740 1 --2- 172.21.15.107:0/1683129627 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e26f6bb980 0x55e26f6d3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.416+0000 7fe2b9fdd740 1 --2- 172.21.15.107:0/1683129627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6acc60 0x55e26f6d3740 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.416+0000 7fe2b9fdd740 1 --2- 172.21.15.107:0/1683129627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e26f6ac2b0 0x55e26f6c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.416+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/1683129627 >> 172.21.15.107:0/1683129627 conn(0x55e26f6a88a0 msgr2=0x55e26f6b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.416+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/1683129627 shutdown_connections 2024-04-25T00:24:07.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.416+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/1683129627 wait complete. 2024-04-25T00:24:07.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.417+0000 7fe2b9fdd740 1 -- start start 2024-04-25T00:24:07.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.417+0000 7fe2b9fdd740 1 Processor -- start 2024-04-25T00:24:07.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.417+0000 7fe2b9fdd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e26f6ac2b0 0x55e26f6bf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.417+0000 7fe2b9fdd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6acc60 0x55e26f6bfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.417+0000 7fe2b9fdd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e26f6bb980 0x55e26f730fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.417+0000 7fe2b9fdd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b01ab080 con 0x55e26f6acc60 2024-04-25T00:24:07.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.417+0000 7fe2b9fdd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b01ac4f0 con 0x55e26f6ac2b0 2024-04-25T00:24:07.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.417+0000 7fe2b9fdd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b01ab750 con 0x55e26f6bb980 2024-04-25T00:24:07.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.417+0000 7fe2b9fdd740 10 client.0 _ll_register_callbacks cb 0x7fe2a8007c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:07.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.417+0000 7fe2affff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6acc60 0x55e26f6bfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.417+0000 7fe2affff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6acc60 0x55e26f6bfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42694/0 (socket says 172.21.15.107:42694) 2024-04-25T00:24:07.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.417+0000 7fe2affff640 1 -- 172.21.15.107:0/1179586735 learned_addr learned my addr 172.21.15.107:0/1179586735 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:07.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.417+0000 7fe2b4f8a640 1 --2- 172.21.15.107:0/1179586735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e26f6ac2b0 0x55e26f6bf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.417+0000 7fe2b578b640 1 --2- 172.21.15.107:0/1179586735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e26f6bb980 0x55e26f730fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.418+0000 7fe2affff640 1 -- 172.21.15.107:0/1179586735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e26f6bb980 msgr2=0x55e26f730fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.418+0000 7fe2affff640 1 --2- 172.21.15.107:0/1179586735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e26f6bb980 0x55e26f730fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.418+0000 7fe2affff640 1 -- 172.21.15.107:0/1179586735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e26f6ac2b0 msgr2=0x55e26f6bf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.418+0000 7fe2affff640 1 --2- 172.21.15.107:0/1179586735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e26f6ac2b0 0x55e26f6bf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.418+0000 7fe2affff640 1 -- 172.21.15.107:0/1179586735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e26f3ef7d0 con 0x55e26f6acc60 2024-04-25T00:24:07.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.418+0000 7fe2affff640 1 --2- 172.21.15.107:0/1179586735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6acc60 0x55e26f6bfe50 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00f010 tx=0x7fe29c0042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd054cb859cd65e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.418+0000 7fe28ffff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:24:07.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.418+0000 7fe28ffff640 1 -- 172.21.15.107:0/1179586735 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c031020 con 0x55e26f6acc60 2024-04-25T00:24:07.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.418+0000 7fe28ffff640 1 -- 172.21.15.107:0/1179586735 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fe29c005450 con 0x55e26f6acc60 2024-04-25T00:24:07.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.418+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/1179586735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe29c0043c0 con 0x55e26f6acc60 2024-04-25T00:24:07.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.418+0000 7fe28ffff640 1 -- 172.21.15.107:0/1179586735 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe29c03c050 con 0x55e26f6acc60 2024-04-25T00:24:07.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.418+0000 7fe2b9fdd740 10 client.17340 fetch_fsmap learned FSMap version 34 2024-04-25T00:24:07.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.418+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/1179586735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e26f7323d0 con 0x55e26f6acc60 2024-04-25T00:24:07.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.418+0000 7fe28ffff640 1 -- 172.21.15.107:0/1179586735 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe29c0043c0 con 0x55e26f6acc60 2024-04-25T00:24:07.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.418+0000 7fe28ffff640 1 -- 172.21.15.107:0/1179586735 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fe29c003850 con 0x55e26f6acc60 2024-04-25T00:24:07.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.418+0000 7fe2b9fdd740 10 client.17340 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:24:07.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.418+0000 7fe2b9fdd740 10 client.17340 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:24:07.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.418+0000 7fe2b9fdd740 20 client.17340 populate_metadata read hostname 'smithi107' 2024-04-25T00:24:07.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.418+0000 7fe2b9fdd740 1 --2- 172.21.15.107:0/1179586735 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55e26f7328c0 0x55e26f734ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.418+0000 7fe2b9fdd740 4 client.17340 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:24:07.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.418+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/1179586735 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e26f618f00 con 0x55e26f7328c0 2024-04-25T00:24:07.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.419+0000 7fe2b4f8a640 1 --2- 172.21.15.107:0/1179586735 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55e26f7328c0 0x55e26f734ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.419+0000 7fe2b4f8a640 1 --2- 172.21.15.107:0/1179586735 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55e26f7328c0 0x55e26f734ce0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=acca7d01108263b8 server_cookie=fecdc577738b128b in_seq=0 out_seq=0 2024-04-25T00:24:07.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.419+0000 7fe28ffff640 10 client.17340 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:24:07.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.419+0000 7fe28ffff640 1 -- 172.21.15.107:0/1179586735 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e26f618f00 con 0x55e26f7328c0 2024-04-25T00:24:07.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.419+0000 7fe28ffff640 10 client.17340 handle_command_reply: tid=0 2024-04-25T00:24:07.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.419+0000 7fe2b9fdd740 1 client.17340 shutdown 2024-04-25T00:24:07.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.419+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/1179586735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6acc60 msgr2=0x55e26f6bfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.419+0000 7fe2b9fdd740 1 --2- 172.21.15.107:0/1179586735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6acc60 0x55e26f6bfe50 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fe29c00f010 tx=0x7fe29c0042d0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.419+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/1179586735 shutdown_connections 2024-04-25T00:24:07.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.419+0000 7fe2b9fdd740 1 --2- 172.21.15.107:0/1179586735 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55e26f7328c0 0x55e26f734ce0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.419+0000 7fe2b9fdd740 1 --2- 172.21.15.107:0/1179586735 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55e26f6bb980 0x55e26f730fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.419+0000 7fe28ffff640 0 client.17340 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:24:07.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.419+0000 7fe2b9fdd740 1 --2- 172.21.15.107:0/1179586735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e26f6acc60 0x55e26f6bfe50 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.419+0000 7fe2b9fdd740 1 --2- 172.21.15.107:0/1179586735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e26f6ac2b0 0x55e26f6bf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.419+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/1179586735 >> 172.21.15.107:0/1179586735 conn(0x55e26f6a88a0 msgr2=0x55e26f6b9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.420+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/1179586735 shutdown_connections 2024-04-25T00:24:07.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.420+0000 7fe2b9fdd740 1 -- 172.21.15.107:0/1179586735 wait complete. 2024-04-25T00:24:07.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.420+0000 7fe2b9fdd740 20 client.17340 trim_cache size 0 max 16384 2024-04-25T00:24:07.428 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:24:07.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.420+0000 7fe2b71ee640 1 -- 172.21.15.107:0/4123009205 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe2a00a2d00 msgr2=0x7fe2a00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.420+0000 7fe2b71ee640 1 --2- 172.21.15.107:0/4123009205 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe2a00a2d00 0x7fe2a00a51c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe2a800c810 tx=0x7fe2a800cc10 comp rx=0 tx=0).stop 2024-04-25T00:24:07.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.420+0000 7fe2b71ee640 1 -- 172.21.15.107:0/4123009205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2b0069dc0 msgr2=0x7fe2b0198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.420+0000 7fe2b71ee640 1 --2- 172.21.15.107:0/4123009205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2b0069dc0 0x7fe2b0198060 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009a50 tx=0x7fe2a4004270 comp rx=0 tx=0).stop 2024-04-25T00:24:07.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.420+0000 7fe2b71ee640 1 -- 172.21.15.107:0/4123009205 shutdown_connections 2024-04-25T00:24:07.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.420+0000 7fe2b71ee640 1 --2- 172.21.15.107:0/4123009205 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe2a00a2d00 0x7fe2a00a51c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.420+0000 7fe2b71ee640 1 --2- 172.21.15.107:0/4123009205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe2b0107730 0x7fe2b019c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.420+0000 7fe2b71ee640 1 --2- 172.21.15.107:0/4123009205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe2b006a780 0x7fe2b01985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.420+0000 7fe2b71ee640 1 --2- 172.21.15.107:0/4123009205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2b0069dc0 0x7fe2b0198060 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.420+0000 7fe2b71ee640 1 -- 172.21.15.107:0/4123009205 >> 172.21.15.107:0/4123009205 conn(0x7fe2b00776a0 msgr2=0x7fe2b00fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.420+0000 7fe2b71ee640 1 -- 172.21.15.107:0/4123009205 shutdown_connections 2024-04-25T00:24:07.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.420+0000 7fe2b71ee640 1 -- 172.21.15.107:0/4123009205 wait complete. 2024-04-25T00:24:07.431 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:24:07.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.530+0000 7faba9d77640 1 -- 172.21.15.107:0/2135224449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faba4107720 msgr2=0x7faba4107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.530+0000 7faba9d77640 1 --2- 172.21.15.107:0/2135224449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faba4107720 0x7faba4107b00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fab94009930 tx=0x7fab9402a420 comp rx=0 tx=0).stop 2024-04-25T00:24:07.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.530+0000 7faba9d77640 1 -- 172.21.15.107:0/2135224449 shutdown_connections 2024-04-25T00:24:07.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.530+0000 7faba9d77640 1 --2- 172.21.15.107:0/2135224449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faba41020d0 0x7faba410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.530+0000 7faba9d77640 1 --2- 172.21.15.107:0/2135224449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faba4101710 0x7faba4101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.530+0000 7faba9d77640 1 --2- 172.21.15.107:0/2135224449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faba4107720 0x7faba4107b00 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.530+0000 7faba9d77640 1 -- 172.21.15.107:0/2135224449 >> 172.21.15.107:0/2135224449 conn(0x7faba4078b90 msgr2=0x7faba4078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.530+0000 7faba9d77640 1 -- 172.21.15.107:0/2135224449 shutdown_connections 2024-04-25T00:24:07.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.530+0000 7faba9d77640 1 -- 172.21.15.107:0/2135224449 wait complete. 2024-04-25T00:24:07.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.531+0000 7faba9d77640 1 Processor -- start 2024-04-25T00:24:07.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.531+0000 7faba9d77640 1 -- start start 2024-04-25T00:24:07.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.531+0000 7faba9d77640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faba4101710 0x7faba4198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.531+0000 7faba9d77640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faba41020d0 0x7faba4198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.531+0000 7faba9d77640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faba4107720 0x7faba419c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.531+0000 7faba9d77640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faba4111b30 con 0x7faba4101710 2024-04-25T00:24:07.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.531+0000 7faba9d77640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faba41119b0 con 0x7faba41020d0 2024-04-25T00:24:07.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.531+0000 7faba9d77640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faba4111cb0 con 0x7faba4107720 2024-04-25T00:24:07.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.531+0000 7faba37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faba4101710 0x7faba4198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.531+0000 7faba37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faba4101710 0x7faba4198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42708/0 (socket says 172.21.15.107:42708) 2024-04-25T00:24:07.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.531+0000 7faba37fe640 1 -- 172.21.15.107:0/502826112 learned_addr learned my addr 172.21.15.107:0/502826112 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:07.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.531+0000 7faba2ffd640 1 --2- 172.21.15.107:0/502826112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faba41020d0 0x7faba4198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.531+0000 7faba37fe640 1 -- 172.21.15.107:0/502826112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faba4107720 msgr2=0x7faba419c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.531+0000 7faba37fe640 1 --2- 172.21.15.107:0/502826112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faba4107720 0x7faba419c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.531+0000 7faba37fe640 1 -- 172.21.15.107:0/502826112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faba41020d0 msgr2=0x7faba4198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.531+0000 7faba37fe640 1 --2- 172.21.15.107:0/502826112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faba41020d0 0x7faba4198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.531+0000 7faba37fe640 1 -- 172.21.15.107:0/502826112 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba419d020 con 0x7faba4101710 2024-04-25T00:24:07.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.532+0000 7faba37fe640 1 --2- 172.21.15.107:0/502826112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faba4101710 0x7faba4198030 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fab94009230 tx=0x7fab94004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.532+0000 7faba0ff9640 1 -- 172.21.15.107:0/502826112 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9403f030 con 0x7faba4101710 2024-04-25T00:24:07.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.532+0000 7faba9d77640 1 -- 172.21.15.107:0/502826112 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba419d2b0 con 0x7faba4101710 2024-04-25T00:24:07.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.532+0000 7faba0ff9640 1 -- 172.21.15.107:0/502826112 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab94003ad0 con 0x7faba4101710 2024-04-25T00:24:07.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.532+0000 7faba0ff9640 1 -- 172.21.15.107:0/502826112 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab94035070 con 0x7faba4101710 2024-04-25T00:24:07.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.532+0000 7faba9d77640 1 -- 172.21.15.107:0/502826112 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba41a4ba0 con 0x7faba4101710 2024-04-25T00:24:07.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.534+0000 7faba0ff9640 1 -- 172.21.15.107:0/502826112 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fab94042030 con 0x7faba4101710 2024-04-25T00:24:07.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.534+0000 7fabacb65740 1 Processor -- start 2024-04-25T00:24:07.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.534+0000 7fabacb65740 1 -- start start 2024-04-25T00:24:07.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.534+0000 7fabacb65740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a76bea580 0x561a76bea960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.534+0000 7fabacb65740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76beaea0 0x561a76bf5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.535+0000 7fabacb65740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a76bf5570 0x561a76bf7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.535+0000 7fabacb65740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561a76b4f670 con 0x561a76beaea0 2024-04-25T00:24:07.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.535+0000 7fabacb65740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561a76af7970 con 0x561a76bf5570 2024-04-25T00:24:07.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.535+0000 7fabacb65740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561a76b4b630 con 0x561a76bea580 2024-04-25T00:24:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.535+0000 7faba0ff9640 1 --2- 172.21.15.107:0/502826112 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fab8c0a2d00 0x7fab8c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.535+0000 7faba37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76beaea0 0x561a76bf5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.535+0000 7faba37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76beaea0 0x561a76bf5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42712/0 (socket says 172.21.15.107:42712) 2024-04-25T00:24:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.535+0000 7faba37fe640 1 -- 172.21.15.107:0/2445764236 learned_addr learned my addr 172.21.15.107:0/2445764236 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.535+0000 7faba3fff640 1 --2- 172.21.15.107:0/502826112 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fab8c0a2d00 0x7fab8c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.535+0000 7faba3fff640 1 --2- 172.21.15.107:0/2445764236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a76bea580 0x561a76bea960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.535+0000 7faba2ffd640 1 --2- 172.21.15.107:0/2445764236 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a76bf5570 0x561a76bf7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.535+0000 7faba0ff9640 1 -- 172.21.15.107:0/502826112 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fab940efe70 con 0x7faba4101710 2024-04-25T00:24:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.535+0000 7faba37fe640 1 -- 172.21.15.107:0/2445764236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a76bea580 msgr2=0x561a76bea960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.535+0000 7faba37fe640 1 --2- 172.21.15.107:0/2445764236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a76bea580 0x561a76bea960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.535+0000 7faba37fe640 1 -- 172.21.15.107:0/2445764236 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a76bf5570 msgr2=0x561a76bf7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.535+0000 7faba37fe640 1 --2- 172.21.15.107:0/2445764236 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a76bf5570 0x561a76bf7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.535+0000 7faba37fe640 1 -- 172.21.15.107:0/2445764236 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a76b1b400 con 0x561a76beaea0 2024-04-25T00:24:07.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.535+0000 7faba3fff640 1 --2- 172.21.15.107:0/502826112 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fab8c0a2d00 0x7fab8c0a51c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0092f0 tx=0x7fab9c009bf0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.535+0000 7faba37fe640 1 --2- 172.21.15.107:0/2445764236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76beaea0 0x561a76bf5030 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fab940c46b0 tx=0x7fab940bbbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d07dac7695044b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.535+0000 7fab89ffb640 1 -- 172.21.15.107:0/2445764236 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab94049070 con 0x561a76beaea0 2024-04-25T00:24:07.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.535+0000 7fab89ffb640 1 -- 172.21.15.107:0/2445764236 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab940bbf70 con 0x561a76beaea0 2024-04-25T00:24:07.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.535+0000 7fab89ffb640 1 -- 172.21.15.107:0/2445764236 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab94003e10 con 0x561a76beaea0 2024-04-25T00:24:07.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.536+0000 7fabacb65740 1 -- 172.21.15.107:0/2445764236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76beaea0 msgr2=0x561a76bf5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.536+0000 7fabacb65740 1 --2- 172.21.15.107:0/2445764236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76beaea0 0x561a76bf5030 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fab940c46b0 tx=0x7fab940bbbe0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.536+0000 7fabacb65740 1 -- 172.21.15.107:0/2445764236 shutdown_connections 2024-04-25T00:24:07.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.536+0000 7fabacb65740 1 --2- 172.21.15.107:0/2445764236 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a76bf5570 0x561a76bf7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.536+0000 7fabacb65740 1 --2- 172.21.15.107:0/2445764236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76beaea0 0x561a76bf5030 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.536+0000 7fabacb65740 1 --2- 172.21.15.107:0/2445764236 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a76bea580 0x561a76bea960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.536+0000 7fabacb65740 1 -- 172.21.15.107:0/2445764236 >> 172.21.15.107:0/2445764236 conn(0x561a76be68a0 msgr2=0x561a76be6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.536+0000 7fabacb65740 1 -- 172.21.15.107:0/2445764236 shutdown_connections 2024-04-25T00:24:07.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.536+0000 7fabacb65740 1 -- 172.21.15.107:0/2445764236 wait complete. 2024-04-25T00:24:07.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.536+0000 7fabacb65740 1 -- start start 2024-04-25T00:24:07.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.537+0000 7fabacb65740 1 Processor -- start 2024-04-25T00:24:07.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.537+0000 7fabacb65740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76bea580 0x561a76bf4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.537+0000 7fabacb65740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a76beaea0 0x561a76bf2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.537+0000 7fabacb65740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a76bf5570 0x561a76bf2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.537+0000 7fabacb65740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561a76b4f670 con 0x561a76bea580 2024-04-25T00:24:07.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.537+0000 7fabacb65740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561a76b0b050 con 0x561a76bf5570 2024-04-25T00:24:07.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.537+0000 7fabacb65740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561a76bf8120 con 0x561a76beaea0 2024-04-25T00:24:07.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.537+0000 7fabacb65740 10 client.0 _ll_register_callbacks cb 0x561a76862be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:07.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.537+0000 7faba37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76bea580 0x561a76bf4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.537+0000 7faba37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76bea580 0x561a76bf4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42716/0 (socket says 172.21.15.107:42716) 2024-04-25T00:24:07.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.537+0000 7faba37fe640 1 -- 172.21.15.107:0/275021001 learned_addr learned my addr 172.21.15.107:0/275021001 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:07.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.537+0000 7faba3fff640 1 --2- 172.21.15.107:0/275021001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a76bf5570 0x561a76bf2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.537+0000 7faba2ffd640 1 --2- 172.21.15.107:0/275021001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a76beaea0 0x561a76bf2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.537+0000 7faba37fe640 1 -- 172.21.15.107:0/275021001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a76beaea0 msgr2=0x561a76bf2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.537+0000 7faba37fe640 1 --2- 172.21.15.107:0/275021001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a76beaea0 0x561a76bf2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.537+0000 7faba37fe640 1 -- 172.21.15.107:0/275021001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a76bf5570 msgr2=0x561a76bf2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.537+0000 7faba37fe640 1 --2- 172.21.15.107:0/275021001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a76bf5570 0x561a76bf2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.537+0000 7faba37fe640 1 -- 172.21.15.107:0/275021001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a76adcdc0 con 0x561a76bea580 2024-04-25T00:24:07.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.537+0000 7faba3fff640 1 --2- 172.21.15.107:0/275021001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a76bf5570 0x561a76bf2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:07.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.537+0000 7faba37fe640 1 --2- 172.21.15.107:0/275021001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76bea580 0x561a76bf4f60 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fab940bbf00 tx=0x7fab94005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536aa1c5ad9c2fc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.537+0000 7fab82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:24:07.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.537+0000 7fab82ffd640 1 -- 172.21.15.107:0/275021001 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab940d6030 con 0x561a76bea580 2024-04-25T00:24:07.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.538+0000 7fab82ffd640 1 -- 172.21.15.107:0/275021001 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fab940ba270 con 0x561a76bea580 2024-04-25T00:24:07.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.538+0000 7fabacb65740 1 -- 172.21.15.107:0/275021001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab940bbf70 con 0x561a76bea580 2024-04-25T00:24:07.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.538+0000 7fab82ffd640 1 -- 172.21.15.107:0/275021001 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fab94049050 con 0x561a76bea580 2024-04-25T00:24:07.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.538+0000 7fabacb65740 10 client.17352 fetch_fsmap learned FSMap version 34 2024-04-25T00:24:07.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.538+0000 7fabacb65740 1 -- 172.21.15.107:0/275021001 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561a768633f0 con 0x561a76bea580 2024-04-25T00:24:07.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.538+0000 7fab82ffd640 1 -- 172.21.15.107:0/275021001 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab940bbf70 con 0x561a76bea580 2024-04-25T00:24:07.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.538+0000 7fab82ffd640 1 -- 172.21.15.107:0/275021001 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fab940c4750 con 0x561a76bea580 2024-04-25T00:24:07.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.538+0000 7fabacb65740 10 client.17352 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:24:07.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.538+0000 7fabacb65740 10 client.17352 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:24:07.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.538+0000 7fabacb65740 20 client.17352 populate_metadata read hostname 'smithi107' 2024-04-25T00:24:07.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.538+0000 7fabacb65740 1 --2- 172.21.15.107:0/275021001 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x561a76c05110 0x561a76c07500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.538+0000 7fabacb65740 4 client.17352 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:07.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.538+0000 7fabacb65740 1 -- 172.21.15.107:0/275021001 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a76b5cbb0 con 0x561a76c05110 2024-04-25T00:24:07.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.538+0000 7faba2ffd640 1 --2- 172.21.15.107:0/275021001 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x561a76c05110 0x561a76c07500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.539+0000 7faba2ffd640 1 --2- 172.21.15.107:0/275021001 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x561a76c05110 0x561a76c07500 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3611e79ccab30525 server_cookie=5f2ca9f65de3acfa in_seq=0 out_seq=0 2024-04-25T00:24:07.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.539+0000 7fab82ffd640 10 client.17352 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:24:07.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.539+0000 7fab82ffd640 1 -- 172.21.15.107:0/275021001 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a76b5cbb0 con 0x561a76c05110 2024-04-25T00:24:07.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.539+0000 7fab82ffd640 10 client.17352 handle_command_reply: tid=0 2024-04-25T00:24:07.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.539+0000 7fabacb65740 1 client.17352 shutdown 2024-04-25T00:24:07.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.540+0000 7fabacb65740 1 -- 172.21.15.107:0/275021001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76bea580 msgr2=0x561a76bf4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.540+0000 7fabacb65740 1 --2- 172.21.15.107:0/275021001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76bea580 0x561a76bf4f60 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fab940bbf00 tx=0x7fab94005c00 comp rx=0 tx=0).stop 2024-04-25T00:24:07.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.540+0000 7fabacb65740 1 -- 172.21.15.107:0/275021001 shutdown_connections 2024-04-25T00:24:07.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.540+0000 7fabacb65740 1 --2- 172.21.15.107:0/275021001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a76bf5570 0x561a76bf2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.540+0000 7fabacb65740 1 --2- 172.21.15.107:0/275021001 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a76beaea0 0x561a76bf2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.540+0000 7fabacb65740 1 --2- 172.21.15.107:0/275021001 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x561a76c05110 0x561a76c07500 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.540+0000 7fabacb65740 1 --2- 172.21.15.107:0/275021001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76bea580 0x561a76bf4f60 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.540+0000 7fab82ffd640 0 client.17352 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:24:07.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.540+0000 7fabacb65740 1 -- 172.21.15.107:0/275021001 >> 172.21.15.107:0/275021001 conn(0x561a76be68a0 msgr2=0x561a76bec6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.540+0000 7fabacb65740 1 -- 172.21.15.107:0/275021001 shutdown_connections 2024-04-25T00:24:07.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.540+0000 7fabacb65740 1 -- 172.21.15.107:0/275021001 wait complete. 2024-04-25T00:24:07.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.540+0000 7fabacb65740 20 client.17352 trim_cache size 0 max 16384 2024-04-25T00:24:07.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.553+0000 7fabacb65740 1 Processor -- start 2024-04-25T00:24:07.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.553+0000 7fabacb65740 1 -- start start 2024-04-25T00:24:07.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.553+0000 7fabacb65740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76bea580 0x561a76c00e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.553+0000 7fabacb65740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a76beaea0 0x561a76be5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.553+0000 7fabacb65740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a76bf5570 0x561a76c0cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.553+0000 7fabacb65740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faba41b15f0 con 0x561a76bea580 2024-04-25T00:24:07.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.553+0000 7fabacb65740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faba41aeec0 con 0x561a76beaea0 2024-04-25T00:24:07.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.553+0000 7fabacb65740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faba41acd30 con 0x561a76bf5570 2024-04-25T00:24:07.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.553+0000 7faba37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76bea580 0x561a76c00e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.553+0000 7faba37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76bea580 0x561a76c00e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42732/0 (socket says 172.21.15.107:42732) 2024-04-25T00:24:07.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.553+0000 7faba37fe640 1 -- 172.21.15.107:0/3253335364 learned_addr learned my addr 172.21.15.107:0/3253335364 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:07.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.553+0000 7faba37fe640 1 -- 172.21.15.107:0/3253335364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a76bf5570 msgr2=0x561a76c0cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.553+0000 7faba37fe640 1 --2- 172.21.15.107:0/3253335364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a76bf5570 0x561a76c0cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.553+0000 7faba37fe640 1 -- 172.21.15.107:0/3253335364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a76beaea0 msgr2=0x561a76be5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.553+0000 7faba37fe640 1 --2- 172.21.15.107:0/3253335364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a76beaea0 0x561a76be5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.553+0000 7faba37fe640 1 -- 172.21.15.107:0/3253335364 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a76c07870 con 0x561a76bea580 2024-04-25T00:24:07.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.553+0000 7faba37fe640 1 --2- 172.21.15.107:0/3253335364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76bea580 0x561a76c00e60 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fab940b83d0 tx=0x7fab940bbbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b892ef668afd71a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.553+0000 7fab897fa640 1 -- 172.21.15.107:0/3253335364 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9409e070 con 0x561a76bea580 2024-04-25T00:24:07.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.554+0000 7fab897fa640 1 -- 172.21.15.107:0/3253335364 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab940bbf70 con 0x561a76bea580 2024-04-25T00:24:07.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.554+0000 7fab897fa640 1 -- 172.21.15.107:0/3253335364 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab940d6020 con 0x561a76bea580 2024-04-25T00:24:07.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.554+0000 7fabacb65740 1 -- 172.21.15.107:0/3253335364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76bea580 msgr2=0x561a76c00e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.554+0000 7fabacb65740 1 --2- 172.21.15.107:0/3253335364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76bea580 0x561a76c00e60 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fab940b83d0 tx=0x7fab940bbbe0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.554+0000 7fabacb65740 1 -- 172.21.15.107:0/3253335364 shutdown_connections 2024-04-25T00:24:07.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.554+0000 7fabacb65740 1 --2- 172.21.15.107:0/3253335364 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a76bf5570 0x561a76c0cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.554+0000 7fabacb65740 1 --2- 172.21.15.107:0/3253335364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a76beaea0 0x561a76be5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.554+0000 7fabacb65740 1 --2- 172.21.15.107:0/3253335364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76bea580 0x561a76c00e60 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.554+0000 7fabacb65740 1 -- 172.21.15.107:0/3253335364 >> 172.21.15.107:0/3253335364 conn(0x561a76be68a0 msgr2=0x561a76bf12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.554+0000 7fabacb65740 1 -- 172.21.15.107:0/3253335364 shutdown_connections 2024-04-25T00:24:07.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.554+0000 7fabacb65740 1 -- 172.21.15.107:0/3253335364 wait complete. 2024-04-25T00:24:07.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.554+0000 7fabacb65740 1 -- start start 2024-04-25T00:24:07.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.555+0000 7fabacb65740 1 Processor -- start 2024-04-25T00:24:07.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.555+0000 7fabacb65740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76bea580 0x561a76bfd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.555+0000 7fabacb65740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a76beaea0 0x561a76bfaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.555+0000 7fabacb65740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a76bf5570 0x561a76bfafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.555+0000 7fabacb65740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faba41ad380 con 0x561a76bea580 2024-04-25T00:24:07.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.555+0000 7fabacb65740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faba41ae7f0 con 0x561a76beaea0 2024-04-25T00:24:07.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.555+0000 7fabacb65740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faba41ada50 con 0x561a76bf5570 2024-04-25T00:24:07.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.555+0000 7fabacb65740 10 client.0 _ll_register_callbacks cb 0x7fab9c0124d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:07.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.555+0000 7faba37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76bea580 0x561a76bfd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.555+0000 7faba37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76bea580 0x561a76bfd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42738/0 (socket says 172.21.15.107:42738) 2024-04-25T00:24:07.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.555+0000 7faba37fe640 1 -- 172.21.15.107:0/885847874 learned_addr learned my addr 172.21.15.107:0/885847874 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:07.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.555+0000 7faba3fff640 1 --2- 172.21.15.107:0/885847874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a76bf5570 0x561a76bfafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.555+0000 7faba2ffd640 1 --2- 172.21.15.107:0/885847874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a76beaea0 0x561a76bfaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.555+0000 7faba37fe640 1 -- 172.21.15.107:0/885847874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a76bf5570 msgr2=0x561a76bfafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.555+0000 7faba37fe640 1 --2- 172.21.15.107:0/885847874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a76bf5570 0x561a76bfafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.555+0000 7faba37fe640 1 -- 172.21.15.107:0/885847874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a76beaea0 msgr2=0x561a76bfaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.555+0000 7faba37fe640 1 --2- 172.21.15.107:0/885847874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a76beaea0 0x561a76bfaa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.555+0000 7faba37fe640 1 -- 172.21.15.107:0/885847874 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a7692d7d0 con 0x561a76bea580 2024-04-25T00:24:07.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.555+0000 7faba37fe640 1 --2- 172.21.15.107:0/885847874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76bea580 0x561a76bfd700 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fab940871b0 tx=0x7fab94005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d94c2efecc1ebd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.555+0000 7fab82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:24:07.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.555+0000 7fab82ffd640 1 -- 172.21.15.107:0/885847874 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab940d6070 con 0x561a76bea580 2024-04-25T00:24:07.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.555+0000 7fab82ffd640 1 -- 172.21.15.107:0/885847874 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fab940c4750 con 0x561a76bea580 2024-04-25T00:24:07.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.555+0000 7fabacb65740 1 -- 172.21.15.107:0/885847874 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab940bbf70 con 0x561a76bea580 2024-04-25T00:24:07.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.556+0000 7fab82ffd640 1 -- 172.21.15.107:0/885847874 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fab9409e070 con 0x561a76bea580 2024-04-25T00:24:07.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.556+0000 7fabacb65740 10 client.17358 fetch_fsmap learned FSMap version 34 2024-04-25T00:24:07.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.556+0000 7fabacb65740 1 -- 172.21.15.107:0/885847874 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561a7692d7d0 con 0x561a76bea580 2024-04-25T00:24:07.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.556+0000 7fab82ffd640 1 -- 172.21.15.107:0/885847874 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab940bbf70 con 0x561a76bea580 2024-04-25T00:24:07.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.556+0000 7fab82ffd640 1 -- 172.21.15.107:0/885847874 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fab940a5110 con 0x561a76bea580 2024-04-25T00:24:07.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.556+0000 7fabacb65740 10 client.17358 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:24:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.556+0000 7fabacb65740 10 client.17358 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:24:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.556+0000 7fabacb65740 20 client.17358 populate_metadata read hostname 'smithi107' 2024-04-25T00:24:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.556+0000 7fabacb65740 1 --2- 172.21.15.107:0/885847874 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x561a76c6b6c0 0x561a76c6dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.556+0000 7fabacb65740 4 client.17358 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:24:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.556+0000 7fabacb65740 1 -- 172.21.15.107:0/885847874 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a76b56f00 con 0x561a76c6b6c0 2024-04-25T00:24:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.556+0000 7faba2ffd640 1 --2- 172.21.15.107:0/885847874 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x561a76c6b6c0 0x561a76c6dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.557+0000 7fab82ffd640 10 client.17358 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:24:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.557+0000 7faba2ffd640 1 --2- 172.21.15.107:0/885847874 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x561a76c6b6c0 0x561a76c6dab0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=abb613a24eb05026 server_cookie=a79c9b7a96d75a07 in_seq=0 out_seq=0 2024-04-25T00:24:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.557+0000 7fab82ffd640 1 -- 172.21.15.107:0/885847874 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a76b56f00 con 0x561a76c6b6c0 2024-04-25T00:24:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.557+0000 7fab82ffd640 10 client.17358 handle_command_reply: tid=0 2024-04-25T00:24:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.557+0000 7fabacb65740 1 client.17358 shutdown 2024-04-25T00:24:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.557+0000 7fabacb65740 1 -- 172.21.15.107:0/885847874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76bea580 msgr2=0x561a76bfd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.557+0000 7fabacb65740 1 --2- 172.21.15.107:0/885847874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76bea580 0x561a76bfd700 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fab940871b0 tx=0x7fab94005e50 comp rx=0 tx=0).stop 2024-04-25T00:24:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.557+0000 7fabacb65740 1 -- 172.21.15.107:0/885847874 shutdown_connections 2024-04-25T00:24:07.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.557+0000 7fabacb65740 1 --2- 172.21.15.107:0/885847874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561a76bf5570 0x561a76bfafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.557+0000 7fabacb65740 1 --2- 172.21.15.107:0/885847874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561a76beaea0 0x561a76bfaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.557+0000 7fabacb65740 1 --2- 172.21.15.107:0/885847874 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x561a76c6b6c0 0x561a76c6dab0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.557+0000 7fabacb65740 1 --2- 172.21.15.107:0/885847874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561a76bea580 0x561a76bfd700 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.557+0000 7fab82ffd640 0 client.17358 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:24:07.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.557+0000 7fabacb65740 1 -- 172.21.15.107:0/885847874 >> 172.21.15.107:0/885847874 conn(0x561a76be68a0 msgr2=0x561a76be5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.557+0000 7fabacb65740 1 -- 172.21.15.107:0/885847874 shutdown_connections 2024-04-25T00:24:07.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.557+0000 7fabacb65740 1 -- 172.21.15.107:0/885847874 wait complete. 2024-04-25T00:24:07.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.557+0000 7fabacb65740 20 client.17358 trim_cache size 0 max 16384 2024-04-25T00:24:07.564 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:24:07.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.558+0000 7faba9d77640 1 -- 172.21.15.107:0/502826112 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fab8c0a2d00 msgr2=0x7fab8c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.558+0000 7faba9d77640 1 --2- 172.21.15.107:0/502826112 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fab8c0a2d00 0x7fab8c0a51c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fab9c0092f0 tx=0x7fab9c009bf0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.558+0000 7faba9d77640 1 -- 172.21.15.107:0/502826112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faba4101710 msgr2=0x7faba4198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.558+0000 7faba9d77640 1 --2- 172.21.15.107:0/502826112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faba4101710 0x7faba4198030 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fab94009230 tx=0x7fab94004270 comp rx=0 tx=0).stop 2024-04-25T00:24:07.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.558+0000 7faba9d77640 1 -- 172.21.15.107:0/502826112 shutdown_connections 2024-04-25T00:24:07.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.558+0000 7faba9d77640 1 --2- 172.21.15.107:0/502826112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faba4107720 0x7faba419c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.558+0000 7faba9d77640 1 --2- 172.21.15.107:0/502826112 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fab8c0a2d00 0x7fab8c0a51c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.558+0000 7faba9d77640 1 --2- 172.21.15.107:0/502826112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faba41020d0 0x7faba4198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.558+0000 7faba9d77640 1 --2- 172.21.15.107:0/502826112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faba4101710 0x7faba4198030 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.558+0000 7faba9d77640 1 -- 172.21.15.107:0/502826112 >> 172.21.15.107:0/502826112 conn(0x7faba4078b90 msgr2=0x7faba40ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.558+0000 7faba9d77640 1 -- 172.21.15.107:0/502826112 shutdown_connections 2024-04-25T00:24:07.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.558+0000 7faba9d77640 1 -- 172.21.15.107:0/502826112 wait complete. 2024-04-25T00:24:07.569 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:24:07.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.668+0000 7fc8cc24d640 1 -- 172.21.15.107:0/452717926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8c41020d0 msgr2=0x7fc8c410e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.668+0000 7fc8cc24d640 1 --2- 172.21.15.107:0/452717926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8c41020d0 0x7fc8c410e5e0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009930 tx=0x7fc8bc02a420 comp rx=0 tx=0).stop 2024-04-25T00:24:07.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.668+0000 7fc8cc24d640 1 -- 172.21.15.107:0/452717926 shutdown_connections 2024-04-25T00:24:07.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.668+0000 7fc8cc24d640 1 --2- 172.21.15.107:0/452717926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8c41020d0 0x7fc8c410e5e0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.668+0000 7fc8cc24d640 1 --2- 172.21.15.107:0/452717926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8c4101710 0x7fc8c4101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.668+0000 7fc8cc24d640 1 --2- 172.21.15.107:0/452717926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8c4107720 0x7fc8c4107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.668+0000 7fc8cc24d640 1 -- 172.21.15.107:0/452717926 >> 172.21.15.107:0/452717926 conn(0x7fc8c4078b90 msgr2=0x7fc8c4078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.669+0000 7fc8cc24d640 1 -- 172.21.15.107:0/452717926 shutdown_connections 2024-04-25T00:24:07.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.669+0000 7fc8cc24d640 1 -- 172.21.15.107:0/452717926 wait complete. 2024-04-25T00:24:07.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.669+0000 7fc8cc24d640 1 Processor -- start 2024-04-25T00:24:07.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.669+0000 7fc8cc24d640 1 -- start start 2024-04-25T00:24:07.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.669+0000 7fc8cc24d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8c4101710 0x7fc8c4198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.669+0000 7fc8cc24d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8c41020d0 0x7fc8c41985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.669+0000 7fc8cc24d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8c4107720 0x7fc8c419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.669+0000 7fc8cc24d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c4111b30 con 0x7fc8c4101710 2024-04-25T00:24:07.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.669+0000 7fc8cc24d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c41119b0 con 0x7fc8c4107720 2024-04-25T00:24:07.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.669+0000 7fc8cc24d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c4111cb0 con 0x7fc8c41020d0 2024-04-25T00:24:07.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.669+0000 7fc8c9fe9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8c4101710 0x7fc8c4198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.669+0000 7fc8c9fe9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8c4101710 0x7fc8c4198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42758/0 (socket says 172.21.15.107:42758) 2024-04-25T00:24:07.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.669+0000 7fc8c9fe9640 1 -- 172.21.15.107:0/3876348176 learned_addr learned my addr 172.21.15.107:0/3876348176 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:07.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.669+0000 7fc8ca7ea640 1 --2- 172.21.15.107:0/3876348176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8c4107720 0x7fc8c419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.669+0000 7fc8c97e8640 1 --2- 172.21.15.107:0/3876348176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8c41020d0 0x7fc8c41985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.670+0000 7fc8c9fe9640 1 -- 172.21.15.107:0/3876348176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8c41020d0 msgr2=0x7fc8c41985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.670+0000 7fc8c9fe9640 1 --2- 172.21.15.107:0/3876348176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8c41020d0 0x7fc8c41985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.670+0000 7fc8c9fe9640 1 -- 172.21.15.107:0/3876348176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8c4107720 msgr2=0x7fc8c419c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.670+0000 7fc8c9fe9640 1 --2- 172.21.15.107:0/3876348176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8c4107720 0x7fc8c419c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.670+0000 7fc8c9fe9640 1 -- 172.21.15.107:0/3876348176 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8c419d030 con 0x7fc8c4101710 2024-04-25T00:24:07.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.670+0000 7fc8c9fe9640 1 --2- 172.21.15.107:0/3876348176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8c4101710 0x7fc8c4198090 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fc8c000ebc0 tx=0x7fc8c000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.670+0000 7fc8baffd640 1 -- 172.21.15.107:0/3876348176 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0010630 con 0x7fc8c4101710 2024-04-25T00:24:07.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.670+0000 7fc8cc24d640 1 -- 172.21.15.107:0/3876348176 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8c419d320 con 0x7fc8c4101710 2024-04-25T00:24:07.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.670+0000 7fc8baffd640 1 -- 172.21.15.107:0/3876348176 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8c0010cb0 con 0x7fc8c4101710 2024-04-25T00:24:07.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.670+0000 7fc8baffd640 1 -- 172.21.15.107:0/3876348176 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0018bd0 con 0x7fc8c4101710 2024-04-25T00:24:07.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.670+0000 7fc8cc24d640 1 -- 172.21.15.107:0/3876348176 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8c41a4bb0 con 0x7fc8c4101710 2024-04-25T00:24:07.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.672+0000 7fc8baffd640 1 -- 172.21.15.107:0/3876348176 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc8c00107d0 con 0x7fc8c4101710 2024-04-25T00:24:07.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.672+0000 7fc8cebfd740 1 Processor -- start 2024-04-25T00:24:07.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.673+0000 7fc8cebfd740 1 -- start start 2024-04-25T00:24:07.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.673+0000 7fc8baffd640 1 --2- 172.21.15.107:0/3876348176 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc8b00a2d00 0x7fc8b00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.673+0000 7fc8cebfd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafc0580 0x5623bafc0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.673+0000 7fc8cebfd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623bafc0ea0 0x5623bafcb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.673+0000 7fc8cebfd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623bafcb570 0x5623bafcd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.673+0000 7fc8cebfd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5623baf25670 con 0x5623bafc0580 2024-04-25T00:24:07.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.673+0000 7fc8cebfd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5623baecd970 con 0x5623bafcb570 2024-04-25T00:24:07.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.673+0000 7fc8cebfd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5623baf21630 con 0x5623bafc0ea0 2024-04-25T00:24:07.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.673+0000 7fc8ca7ea640 1 --2- 172.21.15.107:0/3876348176 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc8b00a2d00 0x7fc8b00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.673+0000 7fc8baffd640 1 -- 172.21.15.107:0/3876348176 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fc8c00cfed0 con 0x7fc8c4101710 2024-04-25T00:24:07.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.673+0000 7fc8c9fe9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafc0580 0x5623bafc0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.673+0000 7fc8c9fe9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafc0580 0x5623bafc0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42774/0 (socket says 172.21.15.107:42774) 2024-04-25T00:24:07.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.673+0000 7fc8c9fe9640 1 -- 172.21.15.107:0/125830581 learned_addr learned my addr 172.21.15.107:0/125830581 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:07.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.673+0000 7fc8c9fe9640 1 -- 172.21.15.107:0/125830581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623bafc0ea0 msgr2=0x5623bafcb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.673+0000 7fc8c97e8640 1 --2- 172.21.15.107:0/125830581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623bafc0ea0 0x5623bafcb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.673+0000 7fc8c9fe9640 1 --2- 172.21.15.107:0/125830581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623bafc0ea0 0x5623bafcb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.673+0000 7fc8c9fe9640 1 -- 172.21.15.107:0/125830581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623bafcb570 msgr2=0x5623bafcd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.673+0000 7fc8c9fe9640 1 --2- 172.21.15.107:0/125830581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623bafcb570 0x5623bafcd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.673+0000 7fc8c9fe9640 1 -- 172.21.15.107:0/125830581 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623baef1400 con 0x5623bafc0580 2024-04-25T00:24:07.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.673+0000 7fc8c97e8640 1 --2- 172.21.15.107:0/125830581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623bafc0ea0 0x5623bafcb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:07.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.673+0000 7fc8ca7ea640 1 --2- 172.21.15.107:0/3876348176 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc8b00a2d00 0x7fc8b00a51c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc005ab0 tx=0x7fc8bc005e10 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.673+0000 7fc8c9fe9640 1 --2- 172.21.15.107:0/125830581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafc0580 0x5623bafc0960 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0098a00 tx=0x7fc8c009bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90cdf81341afd01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.674+0000 7fc8affff640 1 -- 172.21.15.107:0/125830581 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0010e60 con 0x5623bafc0580 2024-04-25T00:24:07.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.674+0000 7fc8affff640 1 -- 172.21.15.107:0/125830581 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8c009c730 con 0x5623bafc0580 2024-04-25T00:24:07.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.674+0000 7fc8affff640 1 -- 172.21.15.107:0/125830581 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c009a330 con 0x5623bafc0580 2024-04-25T00:24:07.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.674+0000 7fc8cebfd740 1 -- 172.21.15.107:0/125830581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafc0580 msgr2=0x5623bafc0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.674+0000 7fc8cebfd740 1 --2- 172.21.15.107:0/125830581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafc0580 0x5623bafc0960 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0098a00 tx=0x7fc8c009bb70 comp rx=0 tx=0).stop 2024-04-25T00:24:07.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.674+0000 7fc8cebfd740 1 -- 172.21.15.107:0/125830581 shutdown_connections 2024-04-25T00:24:07.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.674+0000 7fc8cebfd740 1 --2- 172.21.15.107:0/125830581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623bafcb570 0x5623bafcd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.674+0000 7fc8cebfd740 1 --2- 172.21.15.107:0/125830581 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623bafc0ea0 0x5623bafcb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.674+0000 7fc8cebfd740 1 --2- 172.21.15.107:0/125830581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafc0580 0x5623bafc0960 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.674+0000 7fc8cebfd740 1 -- 172.21.15.107:0/125830581 >> 172.21.15.107:0/125830581 conn(0x5623bafbc8a0 msgr2=0x5623bafbcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.674+0000 7fc8cebfd740 1 -- 172.21.15.107:0/125830581 shutdown_connections 2024-04-25T00:24:07.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.674+0000 7fc8cebfd740 1 -- 172.21.15.107:0/125830581 wait complete. 2024-04-25T00:24:07.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.675+0000 7fc8cebfd740 1 -- start start 2024-04-25T00:24:07.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.675+0000 7fc8cebfd740 1 Processor -- start 2024-04-25T00:24:07.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.675+0000 7fc8cebfd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafc0580 0x5623bafcae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.675+0000 7fc8cebfd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623bafc0ea0 0x5623bafc8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.675+0000 7fc8cebfd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623bafcb570 0x5623bafc8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.675+0000 7fc8cebfd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5623bafce300 con 0x5623bafc0580 2024-04-25T00:24:07.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.675+0000 7fc8cebfd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5623baee1050 con 0x5623bafcb570 2024-04-25T00:24:07.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.675+0000 7fc8cebfd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5623bafce480 con 0x5623bafc0ea0 2024-04-25T00:24:07.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.675+0000 7fc8cebfd740 10 client.0 _ll_register_callbacks cb 0x5623bac38be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:07.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.675+0000 7fc8c9fe9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafc0580 0x5623bafcae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.675+0000 7fc8c9fe9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafc0580 0x5623bafcae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42778/0 (socket says 172.21.15.107:42778) 2024-04-25T00:24:07.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.675+0000 7fc8c9fe9640 1 -- 172.21.15.107:0/2052572853 learned_addr learned my addr 172.21.15.107:0/2052572853 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:07.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.675+0000 7fc8c9fe9640 1 -- 172.21.15.107:0/2052572853 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623bafc0ea0 msgr2=0x5623bafc8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.675+0000 7fc8c97e8640 1 --2- 172.21.15.107:0/2052572853 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623bafc0ea0 0x5623bafc8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.675+0000 7fc8c9fe9640 1 --2- 172.21.15.107:0/2052572853 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623bafc0ea0 0x5623bafc8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.675+0000 7fc8c9fe9640 1 -- 172.21.15.107:0/2052572853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623bafcb570 msgr2=0x5623bafc8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.675+0000 7fc8c9fe9640 1 --2- 172.21.15.107:0/2052572853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623bafcb570 0x5623bafc8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.675+0000 7fc8c9fe9640 1 -- 172.21.15.107:0/2052572853 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623baeb2dc0 con 0x5623bafc0580 2024-04-25T00:24:07.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.676+0000 7fc8c97e8640 1 --2- 172.21.15.107:0/2052572853 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623bafc0ea0 0x5623bafc8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:07.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.676+0000 7fc8c9fe9640 1 --2- 172.21.15.107:0/2052572853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafc0580 0x5623bafcae20 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc8c009bfc0 tx=0x7fc8c0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599051d13d973d87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.676+0000 7fc8ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:24:07.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.676+0000 7fc8ad7fa640 1 -- 172.21.15.107:0/2052572853 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0010e60 con 0x5623bafc0580 2024-04-25T00:24:07.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.676+0000 7fc8ad7fa640 1 -- 172.21.15.107:0/2052572853 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fc8c009a330 con 0x5623bafc0580 2024-04-25T00:24:07.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.676+0000 7fc8cebfd740 1 -- 172.21.15.107:0/2052572853 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8c009c730 con 0x5623bafc0580 2024-04-25T00:24:07.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.676+0000 7fc8ad7fa640 1 -- 172.21.15.107:0/2052572853 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc8c00a4ca0 con 0x5623bafc0580 2024-04-25T00:24:07.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.676+0000 7fc8cebfd740 10 client.17370 fetch_fsmap learned FSMap version 34 2024-04-25T00:24:07.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.676+0000 7fc8cebfd740 1 -- 172.21.15.107:0/2052572853 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5623bac393f0 con 0x5623bafc0580 2024-04-25T00:24:07.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.676+0000 7fc8ad7fa640 1 -- 172.21.15.107:0/2052572853 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8c009c730 con 0x5623bafc0580 2024-04-25T00:24:07.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.676+0000 7fc8ad7fa640 1 -- 172.21.15.107:0/2052572853 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fc8c004d440 con 0x5623bafc0580 2024-04-25T00:24:07.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.676+0000 7fc8cebfd740 10 client.17370 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:24:07.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.676+0000 7fc8cebfd740 10 client.17370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:07.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.676+0000 7fc8cebfd740 20 client.17370 populate_metadata read hostname 'smithi107' 2024-04-25T00:24:07.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.676+0000 7fc8cebfd740 1 --2- 172.21.15.107:0/2052572853 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5623bafdb2a0 0x5623bafdd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.676+0000 7fc8cebfd740 4 client.17370 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:07.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.676+0000 7fc8cebfd740 1 -- 172.21.15.107:0/2052572853 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623baf32bb0 con 0x5623bafdb2a0 2024-04-25T00:24:07.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.677+0000 7fc8c97e8640 1 --2- 172.21.15.107:0/2052572853 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5623bafdb2a0 0x5623bafdd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.677+0000 7fc8c97e8640 1 --2- 172.21.15.107:0/2052572853 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5623bafdb2a0 0x5623bafdd690 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbe497d849ca7c01 server_cookie=5227b2340fab79e6 in_seq=0 out_seq=0 2024-04-25T00:24:07.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.677+0000 7fc8ad7fa640 10 client.17370 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:07.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.678+0000 7fc8ad7fa640 1 -- 172.21.15.107:0/2052572853 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5623baf32bb0 con 0x5623bafdb2a0 2024-04-25T00:24:07.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.678+0000 7fc8ad7fa640 10 client.17370 handle_command_reply: tid=0 2024-04-25T00:24:07.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.678+0000 7fc8cebfd740 1 client.17370 shutdown 2024-04-25T00:24:07.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.678+0000 7fc8cebfd740 1 -- 172.21.15.107:0/2052572853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafc0580 msgr2=0x5623bafcae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.678+0000 7fc8cebfd740 1 --2- 172.21.15.107:0/2052572853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafc0580 0x5623bafcae20 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc8c009bfc0 tx=0x7fc8c0000f40 comp rx=0 tx=0).stop 2024-04-25T00:24:07.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.678+0000 7fc8cebfd740 1 -- 172.21.15.107:0/2052572853 shutdown_connections 2024-04-25T00:24:07.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.678+0000 7fc8cebfd740 1 --2- 172.21.15.107:0/2052572853 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5623bafdb2a0 0x5623bafdd690 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.678+0000 7fc8cebfd740 1 --2- 172.21.15.107:0/2052572853 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623bafcb570 0x5623bafc8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.678+0000 7fc8cebfd740 1 --2- 172.21.15.107:0/2052572853 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623bafc0ea0 0x5623bafc8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.678+0000 7fc8cebfd740 1 --2- 172.21.15.107:0/2052572853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafc0580 0x5623bafcae20 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.678+0000 7fc8ad7fa640 0 client.17370 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:07.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.678+0000 7fc8cebfd740 1 -- 172.21.15.107:0/2052572853 >> 172.21.15.107:0/2052572853 conn(0x5623bafbc8a0 msgr2=0x5623bafc26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.678+0000 7fc8cebfd740 1 -- 172.21.15.107:0/2052572853 shutdown_connections 2024-04-25T00:24:07.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.678+0000 7fc8cebfd740 1 -- 172.21.15.107:0/2052572853 wait complete. 2024-04-25T00:24:07.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.678+0000 7fc8cebfd740 20 client.17370 trim_cache size 0 max 16384 2024-04-25T00:24:07.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.691+0000 7fc8cebfd740 1 Processor -- start 2024-04-25T00:24:07.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.691+0000 7fc8cebfd740 1 -- start start 2024-04-25T00:24:07.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.691+0000 7fc8cebfd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafc0580 0x5623bafd71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.691+0000 7fc8cebfd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623bafc0ea0 0x5623bafca950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.691+0000 7fc8cebfd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623bafcb570 0x5623bafe2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.691+0000 7fc8cebfd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c41b1550 con 0x5623bafc0580 2024-04-25T00:24:07.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.691+0000 7fc8cebfd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c41aee20 con 0x5623bafcb570 2024-04-25T00:24:07.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.691+0000 7fc8cebfd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c41acc90 con 0x5623bafc0ea0 2024-04-25T00:24:07.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.691+0000 7fc8c9fe9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafc0580 0x5623bafd71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.691+0000 7fc8c9fe9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafc0580 0x5623bafd71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42780/0 (socket says 172.21.15.107:42780) 2024-04-25T00:24:07.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.691+0000 7fc8c9fe9640 1 -- 172.21.15.107:0/829383192 learned_addr learned my addr 172.21.15.107:0/829383192 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:07.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.691+0000 7fc8c9fe9640 1 -- 172.21.15.107:0/829383192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623bafc0ea0 msgr2=0x5623bafca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.691+0000 7fc8c9fe9640 1 --2- 172.21.15.107:0/829383192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623bafc0ea0 0x5623bafca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.691+0000 7fc8c9fe9640 1 -- 172.21.15.107:0/829383192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623bafcb570 msgr2=0x5623bafe2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.691+0000 7fc8c9fe9640 1 --2- 172.21.15.107:0/829383192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623bafcb570 0x5623bafe2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.692+0000 7fc8c9fe9640 1 -- 172.21.15.107:0/829383192 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623bad037d0 con 0x5623bafc0580 2024-04-25T00:24:07.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.692+0000 7fc8c9fe9640 1 --2- 172.21.15.107:0/829383192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafc0580 0x5623bafd71c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0011fd0 tx=0x7fc8c0018670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feff65f467036de0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.692+0000 7fc8af7fe640 1 -- 172.21.15.107:0/829383192 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c0010e60 con 0x5623bafc0580 2024-04-25T00:24:07.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.692+0000 7fc8af7fe640 1 -- 172.21.15.107:0/829383192 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8c009c730 con 0x5623bafc0580 2024-04-25T00:24:07.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.692+0000 7fc8af7fe640 1 -- 172.21.15.107:0/829383192 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c004d3f0 con 0x5623bafc0580 2024-04-25T00:24:07.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.692+0000 7fc8cebfd740 1 -- 172.21.15.107:0/829383192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafc0580 msgr2=0x5623bafd71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.692+0000 7fc8cebfd740 1 --2- 172.21.15.107:0/829383192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafc0580 0x5623bafd71c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0011fd0 tx=0x7fc8c0018670 comp rx=0 tx=0).stop 2024-04-25T00:24:07.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.692+0000 7fc8cebfd740 1 -- 172.21.15.107:0/829383192 shutdown_connections 2024-04-25T00:24:07.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.692+0000 7fc8cebfd740 1 --2- 172.21.15.107:0/829383192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623bafcb570 0x5623bafe2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.692+0000 7fc8cebfd740 1 --2- 172.21.15.107:0/829383192 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623bafc0ea0 0x5623bafca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.692+0000 7fc8cebfd740 1 --2- 172.21.15.107:0/829383192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafc0580 0x5623bafd71c0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.692+0000 7fc8cebfd740 1 -- 172.21.15.107:0/829383192 >> 172.21.15.107:0/829383192 conn(0x5623bafbc8a0 msgr2=0x5623bafc3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.692+0000 7fc8cebfd740 1 -- 172.21.15.107:0/829383192 shutdown_connections 2024-04-25T00:24:07.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.692+0000 7fc8cebfd740 1 -- 172.21.15.107:0/829383192 wait complete. 2024-04-25T00:24:07.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.693+0000 7fc8cebfd740 1 -- start start 2024-04-25T00:24:07.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.693+0000 7fc8cebfd740 1 Processor -- start 2024-04-25T00:24:07.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.693+0000 7fc8cebfd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623bafc0580 0x5623bafd36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.693+0000 7fc8cebfd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623bafc0ea0 0x5623bafd3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.693+0000 7fc8cebfd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafcb570 0x5623bafc48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.693+0000 7fc8cebfd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c41ad2e0 con 0x5623bafcb570 2024-04-25T00:24:07.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.693+0000 7fc8cebfd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c41ae750 con 0x5623bafc0ea0 2024-04-25T00:24:07.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.693+0000 7fc8cebfd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc8c41ad9b0 con 0x5623bafc0580 2024-04-25T00:24:07.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.693+0000 7fc8cebfd740 10 client.0 _ll_register_callbacks cb 0x7fc8c0010ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:07.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.693+0000 7fc8ca7ea640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafcb570 0x5623bafc48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.693+0000 7fc8ca7ea640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafcb570 0x5623bafc48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42796/0 (socket says 172.21.15.107:42796) 2024-04-25T00:24:07.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.693+0000 7fc8ca7ea640 1 -- 172.21.15.107:0/3131572074 learned_addr learned my addr 172.21.15.107:0/3131572074 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:07.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.693+0000 7fc8c97e8640 1 --2- 172.21.15.107:0/3131572074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623bafc0ea0 0x5623bafd3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.693+0000 7fc8c9fe9640 1 --2- 172.21.15.107:0/3131572074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623bafc0580 0x5623bafd36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.693+0000 7fc8ca7ea640 1 -- 172.21.15.107:0/3131572074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623bafc0580 msgr2=0x5623bafd36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.693+0000 7fc8ca7ea640 1 --2- 172.21.15.107:0/3131572074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623bafc0580 0x5623bafd36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.693+0000 7fc8ca7ea640 1 -- 172.21.15.107:0/3131572074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623bafc0ea0 msgr2=0x5623bafd3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.693+0000 7fc8ca7ea640 1 --2- 172.21.15.107:0/3131572074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623bafc0ea0 0x5623bafd3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.693+0000 7fc8ca7ea640 1 -- 172.21.15.107:0/3131572074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623bafc50f0 con 0x5623bafcb570 2024-04-25T00:24:07.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.693+0000 7fc8c9fe9640 1 --2- 172.21.15.107:0/3131572074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623bafc0580 0x5623bafd36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:07.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.694+0000 7fc8ca7ea640 1 --2- 172.21.15.107:0/3131572074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafcb570 0x5623bafc48c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009930 tx=0x7fc8bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b77569a81e4b84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.694+0000 7fc8ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:24:07.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.694+0000 7fc8ad7fa640 1 -- 172.21.15.107:0/3131572074 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc035050 con 0x5623bafcb570 2024-04-25T00:24:07.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.694+0000 7fc8ad7fa640 1 -- 172.21.15.107:0/3131572074 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9483+0+0 (secure 0 0 0) 0x7fc8bc0046c0 con 0x5623bafcb570 2024-04-25T00:24:07.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.694+0000 7fc8cebfd740 1 -- 172.21.15.107:0/3131572074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8c009c730 con 0x5623bafcb570 2024-04-25T00:24:07.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.694+0000 7fc8ad7fa640 1 -- 172.21.15.107:0/3131572074 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc8bc056050 con 0x5623bafcb570 2024-04-25T00:24:07.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.694+0000 7fc8cebfd740 10 client.17376 fetch_fsmap learned FSMap version 34 2024-04-25T00:24:07.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.694+0000 7fc8cebfd740 1 -- 172.21.15.107:0/3131572074 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5623bb041250 con 0x5623bafcb570 2024-04-25T00:24:07.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.694+0000 7fc8ad7fa640 1 -- 172.21.15.107:0/3131572074 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8c009c730 con 0x5623bafcb570 2024-04-25T00:24:07.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.694+0000 7fc8ad7fa640 1 -- 172.21.15.107:0/3131572074 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fc8bc0037a0 con 0x5623bafcb570 2024-04-25T00:24:07.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.694+0000 7fc8cebfd740 10 client.17376 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:24:07.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.694+0000 7fc8cebfd740 10 client.17376 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:07.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.694+0000 7fc8cebfd740 20 client.17376 populate_metadata read hostname 'smithi107' 2024-04-25T00:24:07.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.694+0000 7fc8cebfd740 1 --2- 172.21.15.107:0/3131572074 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5623bb041a50 0x5623bb043e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.694+0000 7fc8cebfd740 4 client.17376 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:24:07.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.694+0000 7fc8cebfd740 1 -- 172.21.15.107:0/3131572074 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5623baf2cf00 con 0x5623bb041a50 2024-04-25T00:24:07.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.695+0000 7fc8c9fe9640 1 --2- 172.21.15.107:0/3131572074 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5623bb041a50 0x5623bb043e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.695+0000 7fc8c9fe9640 1 --2- 172.21.15.107:0/3131572074 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5623bb041a50 0x5623bb043e70 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e564dedaed3639be server_cookie=9943bf217865ccf0 in_seq=0 out_seq=0 2024-04-25T00:24:07.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.695+0000 7fc8ad7fa640 10 client.17376 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:07.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.695+0000 7fc8ad7fa640 1 -- 172.21.15.107:0/3131572074 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5623baf2cf00 con 0x5623bb041a50 2024-04-25T00:24:07.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.695+0000 7fc8ad7fa640 10 client.17376 handle_command_reply: tid=0 2024-04-25T00:24:07.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.695+0000 7fc8cebfd740 1 client.17376 shutdown 2024-04-25T00:24:07.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.695+0000 7fc8cebfd740 1 -- 172.21.15.107:0/3131572074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafcb570 msgr2=0x5623bafc48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.695+0000 7fc8cebfd740 1 --2- 172.21.15.107:0/3131572074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafcb570 0x5623bafc48c0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009930 tx=0x7fc8bc004270 comp rx=0 tx=0).stop 2024-04-25T00:24:07.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.695+0000 7fc8cebfd740 1 -- 172.21.15.107:0/3131572074 shutdown_connections 2024-04-25T00:24:07.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.695+0000 7fc8cebfd740 1 --2- 172.21.15.107:0/3131572074 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5623bb041a50 0x5623bb043e70 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.695+0000 7fc8cebfd740 1 --2- 172.21.15.107:0/3131572074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623bafcb570 0x5623bafc48c0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.695+0000 7fc8cebfd740 1 --2- 172.21.15.107:0/3131572074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623bafc0ea0 0x5623bafd3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.695+0000 7fc8cebfd740 1 --2- 172.21.15.107:0/3131572074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5623bafc0580 0x5623bafd36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.695+0000 7fc8ad7fa640 0 client.17376 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:07.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.695+0000 7fc8cebfd740 1 -- 172.21.15.107:0/3131572074 >> 172.21.15.107:0/3131572074 conn(0x5623bafbc8a0 msgr2=0x5623bafc2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.695+0000 7fc8cebfd740 1 -- 172.21.15.107:0/3131572074 shutdown_connections 2024-04-25T00:24:07.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.695+0000 7fc8cebfd740 1 -- 172.21.15.107:0/3131572074 wait complete. 2024-04-25T00:24:07.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.695+0000 7fc8cebfd740 20 client.17376 trim_cache size 0 max 16384 2024-04-25T00:24:07.703 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:24:07.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.696+0000 7fc8cc24d640 1 -- 172.21.15.107:0/3876348176 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc8b00a2d00 msgr2=0x7fc8b00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.696+0000 7fc8cc24d640 1 --2- 172.21.15.107:0/3876348176 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc8b00a2d00 0x7fc8b00a51c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc005ab0 tx=0x7fc8bc005e10 comp rx=0 tx=0).stop 2024-04-25T00:24:07.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.696+0000 7fc8cc24d640 1 -- 172.21.15.107:0/3876348176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8c4101710 msgr2=0x7fc8c4198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.696+0000 7fc8cc24d640 1 --2- 172.21.15.107:0/3876348176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8c4101710 0x7fc8c4198090 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fc8c000ebc0 tx=0x7fc8c000c6a0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.696+0000 7fc8cc24d640 1 -- 172.21.15.107:0/3876348176 shutdown_connections 2024-04-25T00:24:07.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.696+0000 7fc8cc24d640 1 --2- 172.21.15.107:0/3876348176 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc8b00a2d00 0x7fc8b00a51c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.696+0000 7fc8cc24d640 1 --2- 172.21.15.107:0/3876348176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8c4107720 0x7fc8c419c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.696+0000 7fc8cc24d640 1 --2- 172.21.15.107:0/3876348176 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc8c41020d0 0x7fc8c41985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.696+0000 7fc8cc24d640 1 --2- 172.21.15.107:0/3876348176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8c4101710 0x7fc8c4198090 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.696+0000 7fc8cc24d640 1 -- 172.21.15.107:0/3876348176 >> 172.21.15.107:0/3876348176 conn(0x7fc8c4078b90 msgr2=0x7fc8c40ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.696+0000 7fc8cc24d640 1 -- 172.21.15.107:0/3876348176 shutdown_connections 2024-04-25T00:24:07.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:07.696+0000 7fc8cc24d640 1 -- 172.21.15.107:0/3876348176 wait complete. 2024-04-25T00:24:08.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:08 smithi195 ceph-mon[32833]: pgmap v235: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:24:08.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:08 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2297970245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:24:08.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:08 smithi170 ceph-mon[32807]: pgmap v235: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:24:08.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:08 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2297970245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:24:08.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:08 smithi107 ceph-mon[29153]: pgmap v235: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:24:08.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:08 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2297970245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:24:08.707 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:24:08.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:24:08.708 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:24:08.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.882+0000 7fcc44bb0640 1 -- 172.21.15.107:0/2151449470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc380986e0 msgr2=0x7fcc38098b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:08.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.882+0000 7fcc44bb0640 1 --2- 172.21.15.107:0/2151449470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc380986e0 0x7fcc38098b60 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fcc40066ea0 tx=0x7fcc400a70a0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.883+0000 7fcc44bb0640 1 -- 172.21.15.107:0/2151449470 shutdown_connections 2024-04-25T00:24:08.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.883+0000 7fcc44bb0640 1 --2- 172.21.15.107:0/2151449470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc380986e0 0x7fcc38098b60 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.883+0000 7fcc44bb0640 1 --2- 172.21.15.107:0/2151449470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc38004c80 0x7fcc38005100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.883+0000 7fcc44bb0640 1 --2- 172.21.15.107:0/2151449470 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc3809acf0 0x7fcc38004740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.883+0000 7fcc44bb0640 1 -- 172.21.15.107:0/2151449470 >> 172.21.15.107:0/2151449470 conn(0x7fcc3800a740 msgr2=0x7fcc3800ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:08.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.884+0000 7fcc44bb0640 1 -- 172.21.15.107:0/2151449470 shutdown_connections 2024-04-25T00:24:08.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.885+0000 7fcc44bb0640 1 -- 172.21.15.107:0/2151449470 wait complete. 2024-04-25T00:24:08.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.886+0000 7fcc44bb0640 1 Processor -- start 2024-04-25T00:24:08.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.887+0000 7fcc44bb0640 1 -- start start 2024-04-25T00:24:08.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.888+0000 7fcc44bb0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc38004c80 0x7fcc380a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:08.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.888+0000 7fcc44bb0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc380986e0 0x7fcc380a59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:08.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.888+0000 7fcc44bb0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc3809acf0 0x7fcc3812eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:08.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.888+0000 7fcc44bb0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3809dbe0 con 0x7fcc3809acf0 2024-04-25T00:24:08.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.888+0000 7fcc44bb0640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3809da60 con 0x7fcc38004c80 2024-04-25T00:24:08.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.888+0000 7fcc44bb0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3809dd60 con 0x7fcc380986e0 2024-04-25T00:24:08.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.889+0000 7fcc3ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc3809acf0 0x7fcc3812eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:08.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.889+0000 7fcc3ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc3809acf0 0x7fcc3812eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42808/0 (socket says 172.21.15.107:42808) 2024-04-25T00:24:08.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.889+0000 7fcc3ffff640 1 -- 172.21.15.107:0/2979008569 learned_addr learned my addr 172.21.15.107:0/2979008569 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:08.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.890+0000 7fcc3effd640 1 --2- 172.21.15.107:0/2979008569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc380986e0 0x7fcc380a59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:08.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.890+0000 7fcc3f7fe640 1 --2- 172.21.15.107:0/2979008569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc38004c80 0x7fcc380a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:08.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.891+0000 7fcc3f7fe640 1 -- 172.21.15.107:0/2979008569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc380986e0 msgr2=0x7fcc380a59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:08.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.891+0000 7fcc3f7fe640 1 --2- 172.21.15.107:0/2979008569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc380986e0 0x7fcc380a59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.891+0000 7fcc3f7fe640 1 -- 172.21.15.107:0/2979008569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc3809acf0 msgr2=0x7fcc3812eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:08.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.891+0000 7fcc3f7fe640 1 --2- 172.21.15.107:0/2979008569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc3809acf0 0x7fcc3812eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.891+0000 7fcc3f7fe640 1 -- 172.21.15.107:0/2979008569 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3812f2c0 con 0x7fcc38004c80 2024-04-25T00:24:08.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.892+0000 7fcc3f7fe640 1 --2- 172.21.15.107:0/2979008569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc38004c80 0x7fcc380a5490 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009fd0 tx=0x7fcc2c00b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:08.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.892+0000 7fcc3cff9640 1 -- 172.21.15.107:0/2979008569 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c015020 con 0x7fcc38004c80 2024-04-25T00:24:08.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.893+0000 7fcc3cff9640 1 -- 172.21.15.107:0/2979008569 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc2c011d10 con 0x7fcc38004c80 2024-04-25T00:24:08.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.893+0000 7fcc3cff9640 1 -- 172.21.15.107:0/2979008569 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c00f2e0 con 0x7fcc38004c80 2024-04-25T00:24:08.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.894+0000 7fcc44bb0640 1 -- 172.21.15.107:0/2979008569 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc3812f5b0 con 0x7fcc38004c80 2024-04-25T00:24:08.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.894+0000 7fcc44bb0640 1 -- 172.21.15.107:0/2979008569 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc38147d60 con 0x7fcc38004c80 2024-04-25T00:24:08.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.901+0000 7fcc48bf7740 1 Processor -- start 2024-04-25T00:24:08.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.902+0000 7fcc48bf7740 1 -- start start 2024-04-25T00:24:08.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.902+0000 7fcc48bf7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff6eb70 0x55acaff6ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:08.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.902+0000 7fcc48bf7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55acaff6d480 0x55acaff6d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:08.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.902+0000 7fcc48bf7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acaff78200 0x55acaff7a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:08.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.902+0000 7fcc48bf7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55acafed12c0 con 0x55acaff6eb70 2024-04-25T00:24:08.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.902+0000 7fcc48bf7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55acafe79910 con 0x55acaff6d480 2024-04-25T00:24:08.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.902+0000 7fcc48bf7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55acafecd280 con 0x55acaff78200 2024-04-25T00:24:08.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.903+0000 7fcc3ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff6eb70 0x55acaff6ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:08.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.903+0000 7fcc3ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff6eb70 0x55acaff6ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42820/0 (socket says 172.21.15.107:42820) 2024-04-25T00:24:08.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.903+0000 7fcc3ffff640 1 -- 172.21.15.107:0/2087960554 learned_addr learned my addr 172.21.15.107:0/2087960554 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:08.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.904+0000 7fcc3ffff640 1 -- 172.21.15.107:0/2087960554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acaff78200 msgr2=0x55acaff7a5f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:24:08.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.904+0000 7fcc3ffff640 1 --2- 172.21.15.107:0/2087960554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acaff78200 0x55acaff7a5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.904+0000 7fcc3ffff640 1 -- 172.21.15.107:0/2087960554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55acaff6d480 msgr2=0x55acaff6d900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:24:08.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.904+0000 7fcc3ffff640 1 --2- 172.21.15.107:0/2087960554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55acaff6d480 0x55acaff6d900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.904+0000 7fcc3ffff640 1 -- 172.21.15.107:0/2087960554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acafe8afb0 con 0x55acaff6eb70 2024-04-25T00:24:08.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.905+0000 7fcc3ffff640 1 --2- 172.21.15.107:0/2087960554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff6eb70 0x55acaff6ceb0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fcc40067650 tx=0x7fcc400690d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f4bb5d14a0a20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:08.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.905+0000 7fcc29ffb640 1 -- 172.21.15.107:0/2087960554 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc40073630 con 0x55acaff6eb70 2024-04-25T00:24:08.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.905+0000 7fcc29ffb640 1 -- 172.21.15.107:0/2087960554 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc40069720 con 0x55acaff6eb70 2024-04-25T00:24:08.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.905+0000 7fcc29ffb640 1 -- 172.21.15.107:0/2087960554 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc40071b90 con 0x55acaff6eb70 2024-04-25T00:24:08.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.911+0000 7fcc48bf7740 1 -- 172.21.15.107:0/2087960554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff6eb70 msgr2=0x55acaff6ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:08.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.911+0000 7fcc48bf7740 1 --2- 172.21.15.107:0/2087960554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff6eb70 0x55acaff6ceb0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fcc40067650 tx=0x7fcc400690d0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.912+0000 7fcc48bf7740 1 -- 172.21.15.107:0/2087960554 shutdown_connections 2024-04-25T00:24:08.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.912+0000 7fcc48bf7740 1 --2- 172.21.15.107:0/2087960554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acaff78200 0x55acaff7a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.912+0000 7fcc48bf7740 1 --2- 172.21.15.107:0/2087960554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55acaff6d480 0x55acaff6d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.912+0000 7fcc48bf7740 1 --2- 172.21.15.107:0/2087960554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff6eb70 0x55acaff6ceb0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.912+0000 7fcc48bf7740 1 -- 172.21.15.107:0/2087960554 >> 172.21.15.107:0/2087960554 conn(0x55acaff35950 msgr2=0x55acaff699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:08.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.912+0000 7fcc3cff9640 1 -- 172.21.15.107:0/2979008569 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fcc2c018020 con 0x7fcc38004c80 2024-04-25T00:24:08.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.913+0000 7fcc3cff9640 1 --2- 172.21.15.107:0/2979008569 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fcc300a2e00 0x7fcc300a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:08.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.913+0000 7fcc3cff9640 1 -- 172.21.15.107:0/2979008569 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9571+0+0 (secure 0 0 0) 0x7fcc2c0cefd0 con 0x7fcc38004c80 2024-04-25T00:24:08.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.914+0000 7fcc48bf7740 1 -- 172.21.15.107:0/2087960554 shutdown_connections 2024-04-25T00:24:08.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.914+0000 7fcc3effd640 1 --2- 172.21.15.107:0/2979008569 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fcc300a2e00 0x7fcc300a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:08.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.914+0000 7fcc48bf7740 1 -- 172.21.15.107:0/2087960554 wait complete. 2024-04-25T00:24:08.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.914+0000 7fcc48bf7740 1 -- start start 2024-04-25T00:24:08.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.914+0000 7fcc48bf7740 1 Processor -- start 2024-04-25T00:24:08.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.915+0000 7fcc48bf7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55acaff6eb70 0x55acaff75250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:08.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.915+0000 7fcc48bf7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff78200 0x55acaff75790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:08.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.915+0000 7fcc48bf7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acaff75cd0 0x55acaff845e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:08.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.915+0000 7fcc48bf7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55acaff7b0d0 con 0x55acaff78200 2024-04-25T00:24:08.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.915+0000 7fcc48bf7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55acafe8ce90 con 0x55acaff6eb70 2024-04-25T00:24:08.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.915+0000 7fcc48bf7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55acaff7b250 con 0x55acaff75cd0 2024-04-25T00:24:08.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.915+0000 7fcc48bf7740 10 client.0 _ll_register_callbacks cb 0x55acafec8970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:08.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.916+0000 7fcc3effd640 1 --2- 172.21.15.107:0/2979008569 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fcc300a2e00 0x7fcc300a52c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x55acaff6ab70 tx=0x7fcc34005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:08.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.916+0000 7fcc3effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff78200 0x55acaff75790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:08.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.916+0000 7fcc3effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff78200 0x55acaff75790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42830/0 (socket says 172.21.15.107:42830) 2024-04-25T00:24:08.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.916+0000 7fcc3effd640 1 -- 172.21.15.107:0/3392244318 learned_addr learned my addr 172.21.15.107:0/3392244318 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:08.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.916+0000 7fcc3effd640 1 -- 172.21.15.107:0/3392244318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acaff75cd0 msgr2=0x55acaff845e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:24:08.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.916+0000 7fcc3effd640 1 --2- 172.21.15.107:0/3392244318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acaff75cd0 0x55acaff845e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.916+0000 7fcc3effd640 1 -- 172.21.15.107:0/3392244318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55acaff6eb70 msgr2=0x55acaff75250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:08.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.916+0000 7fcc3effd640 1 --2- 172.21.15.107:0/3392244318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55acaff6eb70 0x55acaff75250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.916+0000 7fcc3effd640 1 -- 172.21.15.107:0/3392244318 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acafcad7e0 con 0x55acaff78200 2024-04-25T00:24:08.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.917+0000 7fcc3effd640 1 --2- 172.21.15.107:0/3392244318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff78200 0x55acaff75790 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fcc34012700 tx=0x7fcc34004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb2933202f1d293 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:08.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.917+0000 7fcc1effd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:24:08.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.917+0000 7fcc1effd640 1 -- 172.21.15.107:0/3392244318 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc3400fa60 con 0x55acaff78200 2024-04-25T00:24:08.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.917+0000 7fcc1effd640 1 -- 172.21.15.107:0/3392244318 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9571+0+0 (secure 0 0 0) 0x7fcc3400fc00 con 0x55acaff78200 2024-04-25T00:24:08.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.918+0000 7fcc48bf7740 1 -- 172.21.15.107:0/3392244318 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc40069720 con 0x55acaff78200 2024-04-25T00:24:08.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.919+0000 7fcc1effd640 1 -- 172.21.15.107:0/3392244318 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcc3400fe00 con 0x55acaff78200 2024-04-25T00:24:08.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.920+0000 7fcc48bf7740 10 client.17388 fetch_fsmap learned FSMap version 34 2024-04-25T00:24:08.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.920+0000 7fcc48bf7740 1 -- 172.21.15.107:0/3392244318 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55acafd0df70 con 0x55acaff78200 2024-04-25T00:24:08.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.920+0000 7fcc1effd640 1 -- 172.21.15.107:0/3392244318 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc40069720 con 0x55acaff78200 2024-04-25T00:24:08.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.920+0000 7fcc1effd640 1 -- 172.21.15.107:0/3392244318 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fcc3401f420 con 0x55acaff78200 2024-04-25T00:24:08.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.921+0000 7fcc48bf7740 10 client.17388 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:24:08.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.921+0000 7fcc48bf7740 10 client.17388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:08.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.921+0000 7fcc48bf7740 20 client.17388 populate_metadata read hostname 'smithi107' 2024-04-25T00:24:08.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.921+0000 7fcc48bf7740 1 --2- 172.21.15.107:0/3392244318 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55acaff889c0 0x55acaff8ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:08.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.921+0000 7fcc48bf7740 4 client.17388 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:08.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.921+0000 7fcc48bf7740 1 -- 172.21.15.107:0/3392244318 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acafea55a0 con 0x55acaff889c0 2024-04-25T00:24:08.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.922+0000 7fcc3f7fe640 1 --2- 172.21.15.107:0/3392244318 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55acaff889c0 0x55acaff8ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:08.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.923+0000 7fcc3f7fe640 1 --2- 172.21.15.107:0/3392244318 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55acaff889c0 0x55acaff8ae40 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d8ce58c4431b07d server_cookie=661b9f88a6a0a7f0 in_seq=0 out_seq=0 2024-04-25T00:24:08.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.923+0000 7fcc1effd640 10 client.17388 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:08.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.926+0000 7fcc1effd640 1 -- 172.21.15.107:0/3392244318 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55acafea55a0 con 0x55acaff889c0 2024-04-25T00:24:08.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.926+0000 7fcc1effd640 10 client.17388 handle_command_reply: tid=0 2024-04-25T00:24:08.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.926+0000 7fcc48bf7740 1 client.17388 shutdown 2024-04-25T00:24:08.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.926+0000 7fcc48bf7740 1 -- 172.21.15.107:0/3392244318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff78200 msgr2=0x55acaff75790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:08.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.926+0000 7fcc48bf7740 1 --2- 172.21.15.107:0/3392244318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff78200 0x55acaff75790 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fcc34012700 tx=0x7fcc34004730 comp rx=0 tx=0).stop 2024-04-25T00:24:08.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.928+0000 7fcc48bf7740 1 -- 172.21.15.107:0/3392244318 shutdown_connections 2024-04-25T00:24:08.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.928+0000 7fcc48bf7740 1 --2- 172.21.15.107:0/3392244318 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55acaff889c0 0x55acaff8ae40 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.928+0000 7fcc48bf7740 1 --2- 172.21.15.107:0/3392244318 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acaff75cd0 0x55acaff845e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.928+0000 7fcc48bf7740 1 --2- 172.21.15.107:0/3392244318 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff78200 0x55acaff75790 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.928+0000 7fcc48bf7740 1 --2- 172.21.15.107:0/3392244318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55acaff6eb70 0x55acaff75250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.928+0000 7fcc48bf7740 1 -- 172.21.15.107:0/3392244318 >> 172.21.15.107:0/3392244318 conn(0x55acaff35950 msgr2=0x55acaff6f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:08.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.928+0000 7fcc1effd640 0 client.17388 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:08.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.928+0000 7fcc48bf7740 1 -- 172.21.15.107:0/3392244318 shutdown_connections 2024-04-25T00:24:08.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.928+0000 7fcc48bf7740 1 -- 172.21.15.107:0/3392244318 wait complete. 2024-04-25T00:24:08.930 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.928+0000 7fcc48bf7740 20 client.17388 trim_cache size 0 max 16384 2024-04-25T00:24:08.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.967+0000 7fcc48bf7740 1 Processor -- start 2024-04-25T00:24:08.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.967+0000 7fcc48bf7740 1 -- start start 2024-04-25T00:24:08.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.968+0000 7fcc48bf7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acaff6eb70 0x55acaff75780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:08.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.968+0000 7fcc48bf7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55acaff75cd0 0x55acaff689c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:08.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.968+0000 7fcc48bf7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff78200 0x55acaff90750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:08.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.968+0000 7fcc48bf7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcc381548b0 con 0x55acaff78200 2024-04-25T00:24:08.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.968+0000 7fcc48bf7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcc38152180 con 0x55acaff75cd0 2024-04-25T00:24:08.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.968+0000 7fcc48bf7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3814fff0 con 0x55acaff6eb70 2024-04-25T00:24:08.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.969+0000 7fcc3effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55acaff75cd0 0x55acaff689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:08.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.969+0000 7fcc3effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55acaff75cd0 0x55acaff689c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.107:50742/0 (socket says 172.21.15.107:50742) 2024-04-25T00:24:08.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.969+0000 7fcc3effd640 1 -- 172.21.15.107:0/1716691752 learned_addr learned my addr 172.21.15.107:0/1716691752 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:08.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.969+0000 7fcc3f7fe640 1 --2- 172.21.15.107:0/1716691752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acaff6eb70 0x55acaff75780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:08.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.970+0000 7fcc3ffff640 1 --2- 172.21.15.107:0/1716691752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff78200 0x55acaff90750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:08.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.970+0000 7fcc3ffff640 1 -- 172.21.15.107:0/1716691752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acaff6eb70 msgr2=0x55acaff75780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:08.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.970+0000 7fcc3ffff640 1 --2- 172.21.15.107:0/1716691752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acaff6eb70 0x55acaff75780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.970+0000 7fcc3ffff640 1 -- 172.21.15.107:0/1716691752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55acaff75cd0 msgr2=0x55acaff689c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:08.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.970+0000 7fcc3ffff640 1 --2- 172.21.15.107:0/1716691752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55acaff75cd0 0x55acaff689c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.971 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.970+0000 7fcc3ffff640 1 -- 172.21.15.107:0/1716691752 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acafba5170 con 0x55acaff78200 2024-04-25T00:24:08.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.971+0000 7fcc3ffff640 1 --2- 172.21.15.107:0/1716691752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff78200 0x55acaff90750 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fcc400672b0 tx=0x7fcc40050470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85d445fa4f6001e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:08.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.971+0000 7fcc297fa640 1 -- 172.21.15.107:0/1716691752 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc40073770 con 0x55acaff78200 2024-04-25T00:24:08.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.971+0000 7fcc297fa640 1 -- 172.21.15.107:0/1716691752 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc40069720 con 0x55acaff78200 2024-04-25T00:24:08.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.971+0000 7fcc297fa640 1 -- 172.21.15.107:0/1716691752 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc400c1d00 con 0x55acaff78200 2024-04-25T00:24:08.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.972+0000 7fcc48bf7740 1 -- 172.21.15.107:0/1716691752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff78200 msgr2=0x55acaff90750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:08.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.972+0000 7fcc48bf7740 1 --2- 172.21.15.107:0/1716691752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff78200 0x55acaff90750 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fcc400672b0 tx=0x7fcc40050470 comp rx=0 tx=0).stop 2024-04-25T00:24:08.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.972+0000 7fcc48bf7740 1 -- 172.21.15.107:0/1716691752 shutdown_connections 2024-04-25T00:24:08.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.972+0000 7fcc48bf7740 1 --2- 172.21.15.107:0/1716691752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff78200 0x55acaff90750 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.972+0000 7fcc48bf7740 1 --2- 172.21.15.107:0/1716691752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55acaff75cd0 0x55acaff689c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.972+0000 7fcc48bf7740 1 --2- 172.21.15.107:0/1716691752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acaff6eb70 0x55acaff75780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.972+0000 7fcc48bf7740 1 -- 172.21.15.107:0/1716691752 >> 172.21.15.107:0/1716691752 conn(0x55acaff35950 msgr2=0x55acaff682b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:08.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.974+0000 7fcc48bf7740 1 -- 172.21.15.107:0/1716691752 shutdown_connections 2024-04-25T00:24:08.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.974+0000 7fcc48bf7740 1 -- 172.21.15.107:0/1716691752 wait complete. 2024-04-25T00:24:08.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.975+0000 7fcc48bf7740 1 -- start start 2024-04-25T00:24:08.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.975+0000 7fcc48bf7740 1 Processor -- start 2024-04-25T00:24:08.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.976+0000 7fcc48bf7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff6eb70 0x55acaff78cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:08.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.976+0000 7fcc48bf7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acaff75cd0 0x55acaff79200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:08.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.976+0000 7fcc48bf7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55acaff78200 0x55acaff79740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:08.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.976+0000 7fcc48bf7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcc38150640 con 0x55acaff6eb70 2024-04-25T00:24:08.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.976+0000 7fcc48bf7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcc38151ab0 con 0x55acaff78200 2024-04-25T00:24:08.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.976+0000 7fcc48bf7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fcc38150d10 con 0x55acaff75cd0 2024-04-25T00:24:08.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.976+0000 7fcc48bf7740 10 client.0 _ll_register_callbacks cb 0x7fcc340097d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:08.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.976+0000 7fcc3f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff6eb70 0x55acaff78cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:08.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.976+0000 7fcc3f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff6eb70 0x55acaff78cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42852/0 (socket says 172.21.15.107:42852) 2024-04-25T00:24:08.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.976+0000 7fcc3f7fe640 1 -- 172.21.15.107:0/2293206591 learned_addr learned my addr 172.21.15.107:0/2293206591 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:08.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.977+0000 7fcc3f7fe640 1 -- 172.21.15.107:0/2293206591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acaff75cd0 msgr2=0x55acaff79200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:24:08.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.977+0000 7fcc3f7fe640 1 --2- 172.21.15.107:0/2293206591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acaff75cd0 0x55acaff79200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.977+0000 7fcc3f7fe640 1 -- 172.21.15.107:0/2293206591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55acaff78200 msgr2=0x55acaff79740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:08.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.977+0000 7fcc3f7fe640 1 --2- 172.21.15.107:0/2293206591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55acaff78200 0x55acaff79740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.977+0000 7fcc3f7fe640 1 -- 172.21.15.107:0/2293206591 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acaff7d600 con 0x55acaff6eb70 2024-04-25T00:24:08.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.977+0000 7fcc3f7fe640 1 --2- 172.21.15.107:0/2293206591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff6eb70 0x55acaff78cc0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c09ae80 tx=0x7fcc2c010820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc89fe151cec47e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:08.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.979+0000 7fcc1effd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:24:08.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.979+0000 7fcc1effd640 1 -- 172.21.15.107:0/2293206591 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c09b880 con 0x55acaff6eb70 2024-04-25T00:24:08.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.979+0000 7fcc1effd640 1 -- 172.21.15.107:0/2293206591 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9571+0+0 (secure 0 0 0) 0x7fcc2c0bb0b0 con 0x55acaff6eb70 2024-04-25T00:24:08.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.979+0000 7fcc48bf7740 1 -- 172.21.15.107:0/2293206591 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc40069720 con 0x55acaff6eb70 2024-04-25T00:24:08.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.982+0000 7fcc1effd640 1 -- 172.21.15.107:0/2293206591 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcc2c09b180 con 0x55acaff6eb70 2024-04-25T00:24:08.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.983+0000 7fcc48bf7740 10 client.17394 fetch_fsmap learned FSMap version 34 2024-04-25T00:24:08.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.983+0000 7fcc48bf7740 1 -- 172.21.15.107:0/2293206591 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55acaffef2e0 con 0x55acaff6eb70 2024-04-25T00:24:08.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.985+0000 7fcc1effd640 1 -- 172.21.15.107:0/2293206591 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc40069720 con 0x55acaff6eb70 2024-04-25T00:24:08.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.985+0000 7fcc1effd640 1 -- 172.21.15.107:0/2293206591 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fcc2c09f0b0 con 0x55acaff6eb70 2024-04-25T00:24:08.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.987+0000 7fcc48bf7740 10 client.17394 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:24:08.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.987+0000 7fcc48bf7740 10 client.17394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:08.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.987+0000 7fcc48bf7740 20 client.17394 populate_metadata read hostname 'smithi107' 2024-04-25T00:24:08.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.987+0000 7fcc48bf7740 1 --2- 172.21.15.107:0/2293206591 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55acaffef940 0x55acafff1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:08.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.987+0000 7fcc48bf7740 4 client.17394 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:24:08.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.987+0000 7fcc48bf7740 1 -- 172.21.15.107:0/2293206591 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55acafede800 con 0x55acaffef940 2024-04-25T00:24:08.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.988+0000 7fcc3effd640 1 --2- 172.21.15.107:0/2293206591 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55acaffef940 0x55acafff1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:08.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.991+0000 7fcc3effd640 1 --2- 172.21.15.107:0/2293206591 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55acaffef940 0x55acafff1d80 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c59fd513ea402efc server_cookie=514b74e11ca2d69 in_seq=0 out_seq=0 2024-04-25T00:24:08.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.992+0000 7fcc1effd640 10 client.17394 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:08.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.995+0000 7fcc1effd640 1 -- 172.21.15.107:0/2293206591 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55acafede800 con 0x55acaffef940 2024-04-25T00:24:08.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.995+0000 7fcc1effd640 10 client.17394 handle_command_reply: tid=0 2024-04-25T00:24:08.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.995+0000 7fcc48bf7740 1 client.17394 shutdown 2024-04-25T00:24:08.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.997+0000 7fcc48bf7740 1 -- 172.21.15.107:0/2293206591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff6eb70 msgr2=0x55acaff78cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:08.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.997+0000 7fcc48bf7740 1 --2- 172.21.15.107:0/2293206591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff6eb70 0x55acaff78cc0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c09ae80 tx=0x7fcc2c010820 comp rx=0 tx=0).stop 2024-04-25T00:24:08.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.997+0000 7fcc48bf7740 1 -- 172.21.15.107:0/2293206591 shutdown_connections 2024-04-25T00:24:08.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.997+0000 7fcc48bf7740 1 --2- 172.21.15.107:0/2293206591 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55acaffef940 0x55acafff1d80 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.997+0000 7fcc48bf7740 1 --2- 172.21.15.107:0/2293206591 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55acaff78200 0x55acaff79740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.997+0000 7fcc48bf7740 1 --2- 172.21.15.107:0/2293206591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55acaff75cd0 0x55acaff79200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:08.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.997+0000 7fcc48bf7740 1 --2- 172.21.15.107:0/2293206591 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55acaff6eb70 0x55acaff78cc0 secure :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c09ae80 tx=0x7fcc2c010820 comp rx=0 tx=0).stop 2024-04-25T00:24:08.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.997+0000 7fcc48bf7740 1 -- 172.21.15.107:0/2293206591 >> 172.21.15.107:0/2293206591 conn(0x55acaff35950 msgr2=0x55acaff68b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:08.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.997+0000 7fcc1effd640 0 client.17394 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:09.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:08.999+0000 7fcc48bf7740 1 -- 172.21.15.107:0/2293206591 shutdown_connections 2024-04-25T00:24:09.000 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.000+0000 7fcc48bf7740 1 -- 172.21.15.107:0/2293206591 wait complete. 2024-04-25T00:24:09.001 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.000+0000 7fcc48bf7740 20 client.17394 trim_cache size 0 max 16384 2024-04-25T00:24:09.003 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:24:09.004 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-04-25T00:24:09.004 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "0e5e33e5-fe1d-4ad3-bf57-bd10e2d29ad2", 2024-04-25T00:24:09.004 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-04-25T00:24:09.004 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:24:09.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.005+0000 7fcc44bb0640 1 -- 172.21.15.107:0/2979008569 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fcc300a2e00 msgr2=0x7fcc300a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:09.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.005+0000 7fcc44bb0640 1 --2- 172.21.15.107:0/2979008569 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fcc300a2e00 0x7fcc300a52c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x55acaff6ab70 tx=0x7fcc34005f90 comp rx=0 tx=0).stop 2024-04-25T00:24:09.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.006+0000 7fcc44bb0640 1 -- 172.21.15.107:0/2979008569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc38004c80 msgr2=0x7fcc380a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:09.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.006+0000 7fcc44bb0640 1 --2- 172.21.15.107:0/2979008569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc38004c80 0x7fcc380a5490 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009fd0 tx=0x7fcc2c00b870 comp rx=0 tx=0).stop 2024-04-25T00:24:09.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.006+0000 7fcc44bb0640 1 -- 172.21.15.107:0/2979008569 shutdown_connections 2024-04-25T00:24:09.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.006+0000 7fcc44bb0640 1 --2- 172.21.15.107:0/2979008569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc3809acf0 0x7fcc3812eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.006+0000 7fcc44bb0640 1 --2- 172.21.15.107:0/2979008569 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcc380986e0 0x7fcc380a59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.006+0000 7fcc44bb0640 1 --2- 172.21.15.107:0/2979008569 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fcc300a2e00 0x7fcc300a52c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.006+0000 7fcc44bb0640 1 --2- 172.21.15.107:0/2979008569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcc38004c80 0x7fcc380a5490 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.006+0000 7fcc44bb0640 1 -- 172.21.15.107:0/2979008569 >> 172.21.15.107:0/2979008569 conn(0x7fcc3800a740 msgr2=0x7fcc38097480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:09.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.007+0000 7fcc44bb0640 1 -- 172.21.15.107:0/2979008569 shutdown_connections 2024-04-25T00:24:09.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.007+0000 7fcc44bb0640 1 -- 172.21.15.107:0/2979008569 wait complete. 2024-04-25T00:24:09.025 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:24:09.026 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:24:09.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.205+0000 7f887791d640 1 -- 172.21.15.107:0/730156992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8870100b20 msgr2=0x7f8870100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:09.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.205+0000 7f887791d640 1 --2- 172.21.15.107:0/730156992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8870100b20 0x7f8870100fa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f886000d910 tx=0x7f886002c390 comp rx=0 tx=0).stop 2024-04-25T00:24:09.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.206+0000 7f887791d640 1 -- 172.21.15.107:0/730156992 shutdown_connections 2024-04-25T00:24:09.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.206+0000 7f887791d640 1 --2- 172.21.15.107:0/730156992 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88701014e0 0x7f887010d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.206+0000 7f887791d640 1 --2- 172.21.15.107:0/730156992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8870100b20 0x7f8870100fa0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.206+0000 7f887791d640 1 --2- 172.21.15.107:0/730156992 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88701066a0 0x7f8870106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.206+0000 7f887791d640 1 -- 172.21.15.107:0/730156992 >> 172.21.15.107:0/730156992 conn(0x7f88700761c0 msgr2=0x7f88700765d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:09.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.208+0000 7f887791d640 1 -- 172.21.15.107:0/730156992 shutdown_connections 2024-04-25T00:24:09.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.209+0000 7f887791d640 1 -- 172.21.15.107:0/730156992 wait complete. 2024-04-25T00:24:09.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.210+0000 7f887791d640 1 Processor -- start 2024-04-25T00:24:09.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.210+0000 7f887791d640 1 -- start start 2024-04-25T00:24:09.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.210+0000 7f887791d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8870100b20 0x7f8870115fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:09.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.211+0000 7f887791d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88701014e0 0x7f8870110020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:09.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.211+0000 7f887791d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88701066a0 0x7f8870110560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:09.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.211+0000 7f887791d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f887010ead0 con 0x7f88701014e0 2024-04-25T00:24:09.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.211+0000 7f887791d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f887010e950 con 0x7f88701066a0 2024-04-25T00:24:09.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.211+0000 7f887791d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f887010ec50 con 0x7f8870100b20 2024-04-25T00:24:09.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.211+0000 7f88756b9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8870100b20 0x7f8870115fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:09.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.211+0000 7f88756b9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8870100b20 0x7f8870115fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:46340/0 (socket says 172.21.15.107:46340) 2024-04-25T00:24:09.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.211+0000 7f88756b9640 1 -- 172.21.15.107:0/1368082554 learned_addr learned my addr 172.21.15.107:0/1368082554 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:09.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.211+0000 7f8874eb8640 1 --2- 172.21.15.107:0/1368082554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88701014e0 0x7f8870110020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:09.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.212+0000 7f8874eb8640 1 -- 172.21.15.107:0/1368082554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8870100b20 msgr2=0x7f8870115fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:09.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.212+0000 7f8874eb8640 1 --2- 172.21.15.107:0/1368082554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8870100b20 0x7f8870115fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.212+0000 7f8874eb8640 1 -- 172.21.15.107:0/1368082554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88701066a0 msgr2=0x7f8870110560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:09.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.212+0000 7f8874eb8640 1 --2- 172.21.15.107:0/1368082554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88701066a0 0x7f8870110560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.212+0000 7f8874eb8640 1 -- 172.21.15.107:0/1368082554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8870110c90 con 0x7f88701014e0 2024-04-25T00:24:09.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.212+0000 7f8874eb8640 1 --2- 172.21.15.107:0/1368082554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88701014e0 0x7f8870110020 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f886000d860 tx=0x7f8860034790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:09.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.214+0000 7f88667fc640 1 -- 172.21.15.107:0/1368082554 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8860034b50 con 0x7f88701014e0 2024-04-25T00:24:09.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.214+0000 7f88667fc640 1 -- 172.21.15.107:0/1368082554 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8860034cf0 con 0x7f88701014e0 2024-04-25T00:24:09.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.214+0000 7f88667fc640 1 -- 172.21.15.107:0/1368082554 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8860033520 con 0x7f88701014e0 2024-04-25T00:24:09.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.215+0000 7f887791d640 1 -- 172.21.15.107:0/1368082554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88701b3430 con 0x7f88701014e0 2024-04-25T00:24:09.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.215+0000 7f887791d640 1 -- 172.21.15.107:0/1368082554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88701b37e0 con 0x7f88701014e0 2024-04-25T00:24:09.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.219+0000 7f88667fc640 1 -- 172.21.15.107:0/1368082554 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f8860043020 con 0x7f88701014e0 2024-04-25T00:24:09.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.220+0000 7f88667fc640 1 --2- 172.21.15.107:0/1368082554 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f885c0a2f10 0x7f885c0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:09.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.220+0000 7f88667fc640 1 -- 172.21.15.107:0/1368082554 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9571+0+0 (secure 0 0 0) 0x7f88600eefb0 con 0x7f88701014e0 2024-04-25T00:24:09.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.221+0000 7f88756b9640 1 --2- 172.21.15.107:0/1368082554 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f885c0a2f10 0x7f885c0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:09.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.221+0000 7f887a622740 1 Processor -- start 2024-04-25T00:24:09.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.221+0000 7f887a622740 1 -- start start 2024-04-25T00:24:09.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.222+0000 7f88756b9640 1 --2- 172.21.15.107:0/1368082554 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f885c0a2f10 0x7f885c0a53d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8868005fd0 tx=0x7f8868007770 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:09.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.222+0000 7f887a622740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5db630 0x56544b5dba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:09.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.222+0000 7f887a622740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56544b5dbf50 0x56544b5e60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:09.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.222+0000 7f887a622740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56544b5e6620 0x56544b5e8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:09.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.222+0000 7f887a622740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56544b5406b0 con 0x56544b5e6620 2024-04-25T00:24:09.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.222+0000 7f887a622740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56544b4e8880 con 0x56544b5db630 2024-04-25T00:24:09.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.222+0000 7f887a622740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56544b53c670 con 0x56544b5dbf50 2024-04-25T00:24:09.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.222+0000 7f88756b9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5db630 0x56544b5dba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:09.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.222+0000 7f88756b9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5db630 0x56544b5dba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.107:50778/0 (socket says 172.21.15.107:50778) 2024-04-25T00:24:09.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.222+0000 7f88756b9640 1 -- 172.21.15.107:0/7648417 learned_addr learned my addr 172.21.15.107:0/7648417 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:09.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.222+0000 7f8874eb8640 1 --2- 172.21.15.107:0/7648417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56544b5dbf50 0x56544b5e60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:09.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.222+0000 7f8875eba640 1 --2- 172.21.15.107:0/7648417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56544b5e6620 0x56544b5e8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:09.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.223+0000 7f88756b9640 1 -- 172.21.15.107:0/7648417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56544b5dbf50 msgr2=0x56544b5e60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:09.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.223+0000 7f88756b9640 1 --2- 172.21.15.107:0/7648417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56544b5dbf50 0x56544b5e60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.223+0000 7f88756b9640 1 -- 172.21.15.107:0/7648417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56544b5e6620 msgr2=0x56544b5e8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:09.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.223+0000 7f88756b9640 1 --2- 172.21.15.107:0/7648417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56544b5e6620 0x56544b5e8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.223+0000 7f88756b9640 1 -- 172.21.15.107:0/7648417 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56544b31e7d0 con 0x56544b5db630 2024-04-25T00:24:09.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.223+0000 7f88756b9640 1 --2- 172.21.15.107:0/7648417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5db630 0x56544b5dba10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8868007fd0 tx=0x7f8868011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfc521710f1c3555 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:09.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.224+0000 7f885b7fe640 1 -- 172.21.15.107:0/7648417 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8868021070 con 0x56544b5db630 2024-04-25T00:24:09.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.224+0000 7f885b7fe640 1 -- 172.21.15.107:0/7648417 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f886801dd40 con 0x56544b5db630 2024-04-25T00:24:09.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.224+0000 7f885b7fe640 1 -- 172.21.15.107:0/7648417 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886800fe30 con 0x56544b5db630 2024-04-25T00:24:09.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.225+0000 7f887a622740 1 -- 172.21.15.107:0/7648417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5db630 msgr2=0x56544b5dba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:09.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.225+0000 7f887a622740 1 --2- 172.21.15.107:0/7648417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5db630 0x56544b5dba10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8868007fd0 tx=0x7f8868011000 comp rx=0 tx=0).stop 2024-04-25T00:24:09.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.226+0000 7f887a622740 1 -- 172.21.15.107:0/7648417 shutdown_connections 2024-04-25T00:24:09.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.226+0000 7f887a622740 1 --2- 172.21.15.107:0/7648417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56544b5e6620 0x56544b5e8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.226+0000 7f887a622740 1 --2- 172.21.15.107:0/7648417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56544b5dbf50 0x56544b5e60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.226+0000 7f887a622740 1 --2- 172.21.15.107:0/7648417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5db630 0x56544b5dba10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.226+0000 7f887a622740 1 -- 172.21.15.107:0/7648417 >> 172.21.15.107:0/7648417 conn(0x56544b5d7950 msgr2=0x56544b5d7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:09.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.231+0000 7f887a622740 1 -- 172.21.15.107:0/7648417 shutdown_connections 2024-04-25T00:24:09.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.231+0000 7f887a622740 1 -- 172.21.15.107:0/7648417 wait complete. 2024-04-25T00:24:09.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.232+0000 7f887a622740 1 -- start start 2024-04-25T00:24:09.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.232+0000 7f887a622740 1 Processor -- start 2024-04-25T00:24:09.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.233+0000 7f887a622740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56544b5db630 0x56544b5e3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:09.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.233+0000 7f887a622740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5dbf50 0x56544b5e3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:09.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.233+0000 7f887a622740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56544b5e6620 0x56544b5e4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:09.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.233+0000 7f887a622740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56544b5e9410 con 0x56544b5e6620 2024-04-25T00:24:09.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.233+0000 7f887a622740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56544b4fbf50 con 0x56544b5dbf50 2024-04-25T00:24:09.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.234+0000 7f887a622740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56544b5e9590 con 0x56544b5db630 2024-04-25T00:24:09.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.234+0000 7f887a622740 10 client.0 _ll_register_callbacks cb 0x56544b537000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:09.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.234+0000 7f8874eb8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5dbf50 0x56544b5e3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:09.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.234+0000 7f8874eb8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5dbf50 0x56544b5e3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.107:50786/0 (socket says 172.21.15.107:50786) 2024-04-25T00:24:09.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.234+0000 7f8874eb8640 1 -- 172.21.15.107:0/1650765916 learned_addr learned my addr 172.21.15.107:0/1650765916 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:09.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.235+0000 7f8874eb8640 1 -- 172.21.15.107:0/1650765916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56544b5db630 msgr2=0x56544b5e3580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:24:09.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.235+0000 7f8874eb8640 1 --2- 172.21.15.107:0/1650765916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56544b5db630 0x56544b5e3580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.235+0000 7f8874eb8640 1 -- 172.21.15.107:0/1650765916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56544b5e6620 msgr2=0x56544b5e4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:09.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.235+0000 7f8874eb8640 1 --2- 172.21.15.107:0/1650765916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56544b5e6620 0x56544b5e4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.235+0000 7f8874eb8640 1 -- 172.21.15.107:0/1650765916 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56544b2543f0 con 0x56544b5dbf50 2024-04-25T00:24:09.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.238+0000 7f8874eb8640 1 --2- 172.21.15.107:0/1650765916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5dbf50 0x56544b5e3ac0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8860006010 tx=0x7f88600bb160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69c653470ff671c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:09.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.238+0000 7f8858ff9640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-04-25T00:24:09.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.239+0000 7f8858ff9640 1 -- 172.21.15.107:0/1650765916 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88600db0c0 con 0x56544b5dbf50 2024-04-25T00:24:09.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.239+0000 7f8858ff9640 1 -- 172.21.15.107:0/1650765916 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9571+0+0 (secure 0 0 0) 0x7f88600bf150 con 0x56544b5dbf50 2024-04-25T00:24:09.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.239+0000 7f887a622740 1 -- 172.21.15.107:0/1650765916 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f886801dd40 con 0x56544b5dbf50 2024-04-25T00:24:09.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.239+0000 7f8858ff9640 1 -- 172.21.15.107:0/1650765916 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f88600db260 con 0x56544b5dbf50 2024-04-25T00:24:09.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.239+0000 7f887a622740 10 client.26453 fetch_fsmap learned FSMap version 34 2024-04-25T00:24:09.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.239+0000 7f887a622740 1 -- 172.21.15.107:0/1650765916 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56544b2140f0 con 0x56544b5dbf50 2024-04-25T00:24:09.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.240+0000 7f8858ff9640 1 -- 172.21.15.107:0/1650765916 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f886801dd40 con 0x56544b5dbf50 2024-04-25T00:24:09.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.240+0000 7f8858ff9640 1 -- 172.21.15.107:0/1650765916 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f88600c3940 con 0x56544b5dbf50 2024-04-25T00:24:09.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.242+0000 7f887a622740 10 client.26453 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:24:09.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.242+0000 7f887a622740 10 client.26453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:09.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.242+0000 7f887a622740 20 client.26453 populate_metadata read hostname 'smithi107' 2024-04-25T00:24:09.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.242+0000 7f887a622740 1 --2- 172.21.15.107:0/1650765916 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56544b5f6610 0x56544b5f8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:09.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.242+0000 7f887a622740 4 client.26453 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:09.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.242+0000 7f887a622740 1 -- 172.21.15.107:0/1650765916 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56544b54dbf0 con 0x56544b5f6610 2024-04-25T00:24:09.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.245+0000 7f88756b9640 1 --2- 172.21.15.107:0/1650765916 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56544b5f6610 0x56544b5f8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:09.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.248+0000 7f88756b9640 1 --2- 172.21.15.107:0/1650765916 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56544b5f6610 0x56544b5f8a80 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=655f6e8ba329070d server_cookie=b6ac737a8cf3d97a in_seq=0 out_seq=0 2024-04-25T00:24:09.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.248+0000 7f8858ff9640 10 client.26453 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:09.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.251+0000 7f8858ff9640 1 -- 172.21.15.107:0/1650765916 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56544b54dbf0 con 0x56544b5f6610 2024-04-25T00:24:09.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.251+0000 7f8858ff9640 10 client.26453 handle_command_reply: tid=0 2024-04-25T00:24:09.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.251+0000 7f887a622740 1 client.26453 shutdown 2024-04-25T00:24:09.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.252+0000 7f887a622740 1 -- 172.21.15.107:0/1650765916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5dbf50 msgr2=0x56544b5e3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:09.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.252+0000 7f887a622740 1 --2- 172.21.15.107:0/1650765916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5dbf50 0x56544b5e3ac0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8860006010 tx=0x7f88600bb160 comp rx=0 tx=0).stop 2024-04-25T00:24:09.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.252+0000 7f887a622740 1 -- 172.21.15.107:0/1650765916 shutdown_connections 2024-04-25T00:24:09.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.252+0000 7f887a622740 1 --2- 172.21.15.107:0/1650765916 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56544b5f6610 0x56544b5f8a80 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.252+0000 7f887a622740 1 --2- 172.21.15.107:0/1650765916 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56544b5e6620 0x56544b5e4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.252+0000 7f887a622740 1 --2- 172.21.15.107:0/1650765916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5dbf50 0x56544b5e3ac0 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8860006010 tx=0x7f88600bb160 comp rx=0 tx=0).stop 2024-04-25T00:24:09.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.252+0000 7f887a622740 1 --2- 172.21.15.107:0/1650765916 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56544b5db630 0x56544b5e3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.252+0000 7f887a622740 1 -- 172.21.15.107:0/1650765916 >> 172.21.15.107:0/1650765916 conn(0x56544b5d7950 msgr2=0x56544b5dd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:09.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.253+0000 7f8858ff9640 0 client.26453 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.257+0000 7f887a622740 1 -- 172.21.15.107:0/1650765916 shutdown_connections 2024-04-25T00:24:09.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.260+0000 7f887a622740 1 -- 172.21.15.107:0/1650765916 wait complete. 2024-04-25T00:24:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.261+0000 7f887a622740 20 client.26453 trim_cache size 0 max 16384 2024-04-25T00:24:09.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.298+0000 7f887a622740 1 Processor -- start 2024-04-25T00:24:09.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.301+0000 7f887a622740 1 -- start start 2024-04-25T00:24:09.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.301+0000 7f887a622740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56544b5db630 0x56544b5f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:09.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.301+0000 7f887a622740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5e6620 0x56544b5d6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:09.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.301+0000 7f887a622740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56544b5e2500 0x56544b5e3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:09.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.301+0000 7f887a622740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f88701c0430 con 0x56544b5db630 2024-04-25T00:24:09.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.301+0000 7f887a622740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f88701bdd00 con 0x56544b5e6620 2024-04-25T00:24:09.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.302+0000 7f887a622740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f88701bbb70 con 0x56544b5e2500 2024-04-25T00:24:09.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.302+0000 7f8874eb8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5e6620 0x56544b5d6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:09.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.302+0000 7f8874eb8640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5e6620 0x56544b5d6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.107:50790/0 (socket says 172.21.15.107:50790) 2024-04-25T00:24:09.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.302+0000 7f8874eb8640 1 -- 172.21.15.107:0/1322752257 learned_addr learned my addr 172.21.15.107:0/1322752257 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:09.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.304+0000 7f88756b9640 1 --2- 172.21.15.107:0/1322752257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56544b5db630 0x56544b5f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:09.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.304+0000 7f8874eb8640 1 -- 172.21.15.107:0/1322752257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56544b5e2500 msgr2=0x56544b5e3e20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:24:09.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.304+0000 7f8874eb8640 1 --2- 172.21.15.107:0/1322752257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56544b5e2500 0x56544b5e3e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.304+0000 7f8874eb8640 1 -- 172.21.15.107:0/1322752257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56544b5db630 msgr2=0x56544b5f22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:09.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.304+0000 7f8874eb8640 1 --2- 172.21.15.107:0/1322752257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56544b5db630 0x56544b5f22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.304+0000 7f8874eb8640 1 -- 172.21.15.107:0/1322752257 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56544b5dbd80 con 0x56544b5e6620 2024-04-25T00:24:09.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.304+0000 7f8874eb8640 1 --2- 172.21.15.107:0/1322752257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5e6620 0x56544b5d6af0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f88600b9410 tx=0x7f886003ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=850ec42884a7c13c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:09.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.305+0000 7f885affd640 1 -- 172.21.15.107:0/1322752257 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88600c8080 con 0x56544b5e6620 2024-04-25T00:24:09.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.305+0000 7f885affd640 1 -- 172.21.15.107:0/1322752257 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8860030020 con 0x56544b5e6620 2024-04-25T00:24:09.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.306+0000 7f887a622740 1 -- 172.21.15.107:0/1322752257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5e6620 msgr2=0x56544b5d6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:09.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.306+0000 7f887a622740 1 --2- 172.21.15.107:0/1322752257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5e6620 0x56544b5d6af0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f88600b9410 tx=0x7f886003ce30 comp rx=0 tx=0).stop 2024-04-25T00:24:09.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.306+0000 7f887a622740 1 -- 172.21.15.107:0/1322752257 shutdown_connections 2024-04-25T00:24:09.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.306+0000 7f887a622740 1 --2- 172.21.15.107:0/1322752257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56544b5e2500 0x56544b5e3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.307+0000 7f887a622740 1 --2- 172.21.15.107:0/1322752257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5e6620 0x56544b5d6af0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.307+0000 7f887a622740 1 --2- 172.21.15.107:0/1322752257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56544b5db630 0x56544b5f22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.307+0000 7f887a622740 1 -- 172.21.15.107:0/1322752257 >> 172.21.15.107:0/1322752257 conn(0x56544b5d7950 msgr2=0x56544b5e4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:09.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.307+0000 7f887a622740 1 -- 172.21.15.107:0/1322752257 shutdown_connections 2024-04-25T00:24:09.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.309+0000 7f887a622740 1 -- 172.21.15.107:0/1322752257 wait complete. 2024-04-25T00:24:09.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.309+0000 7f887a622740 1 -- start start 2024-04-25T00:24:09.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.310+0000 7f887a622740 1 Processor -- start 2024-04-25T00:24:09.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.311+0000 7f887a622740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56544b5db630 0x56544b5e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:09.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.311+0000 7f887a622740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56544b5e2500 0x56544b5e0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:09.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.311+0000 7f887a622740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5e6620 0x56544b5e0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:09.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.311+0000 7f887a622740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f88701bc1c0 con 0x56544b5e2500 2024-04-25T00:24:09.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.311+0000 7f887a622740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f88701bd630 con 0x56544b5e6620 2024-04-25T00:24:09.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.311+0000 7f887a622740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f88701bc890 con 0x56544b5db630 2024-04-25T00:24:09.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.311+0000 7f887a622740 10 client.0 _ll_register_callbacks cb 0x7f88600bab90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:09.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.312+0000 7f8874eb8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56544b5e2500 0x56544b5e0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:09.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.312+0000 7f8874eb8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56544b5e2500 0x56544b5e0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42932/0 (socket says 172.21.15.107:42932) 2024-04-25T00:24:09.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.312+0000 7f8874eb8640 1 -- 172.21.15.107:0/3915897244 learned_addr learned my addr 172.21.15.107:0/3915897244 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:09.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.312+0000 7f88756b9640 1 --2- 172.21.15.107:0/3915897244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56544b5db630 0x56544b5e0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:09.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.312+0000 7f8875eba640 1 --2- 172.21.15.107:0/3915897244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5e6620 0x56544b5e0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:09.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.312+0000 7f8874eb8640 1 -- 172.21.15.107:0/3915897244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56544b5db630 msgr2=0x56544b5e0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:09.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.312+0000 7f8874eb8640 1 --2- 172.21.15.107:0/3915897244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56544b5db630 0x56544b5e0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.312+0000 7f8874eb8640 1 -- 172.21.15.107:0/3915897244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5e6620 msgr2=0x56544b5e0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:09.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.312+0000 7f8874eb8640 1 --2- 172.21.15.107:0/3915897244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5e6620 0x56544b5e0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.312+0000 7f8874eb8640 1 -- 172.21.15.107:0/3915897244 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56544b66cdd0 con 0x56544b5e2500 2024-04-25T00:24:09.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.313+0000 7f8874eb8640 1 --2- 172.21.15.107:0/3915897244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56544b5e2500 0x56544b5e0970 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f88600b7b30 tx=0x7f88600bb860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc115637dc4752f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:09.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.314+0000 7f8858ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:24:09.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.315+0000 7f8858ff9640 1 -- 172.21.15.107:0/3915897244 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88600baf30 con 0x56544b5e2500 2024-04-25T00:24:09.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.315+0000 7f8858ff9640 1 -- 172.21.15.107:0/3915897244 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9571+0+0 (secure 0 0 0) 0x7f88600b6eb0 con 0x56544b5e2500 2024-04-25T00:24:09.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.317+0000 7f887a622740 1 -- 172.21.15.107:0/3915897244 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56544b547f40 con 0x56544b5e2500 2024-04-25T00:24:09.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.318+0000 7f8858ff9640 1 -- 172.21.15.107:0/3915897244 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f88600db0b0 con 0x56544b5e2500 2024-04-25T00:24:09.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.318+0000 7f887a622740 10 client.17406 fetch_fsmap learned FSMap version 34 2024-04-25T00:24:09.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.318+0000 7f887a622740 1 -- 172.21.15.107:0/3915897244 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56544b66cdd0 con 0x56544b5e2500 2024-04-25T00:24:09.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.319+0000 7f8858ff9640 1 -- 172.21.15.107:0/3915897244 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56544b547f40 con 0x56544b5e2500 2024-04-25T00:24:09.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.319+0000 7f8858ff9640 1 -- 172.21.15.107:0/3915897244 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f88600c06a0 con 0x56544b5e2500 2024-04-25T00:24:09.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.319+0000 7f887a622740 10 client.17406 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:24:09.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.319+0000 7f887a622740 10 client.17406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:09.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.319+0000 7f887a622740 20 client.17406 populate_metadata read hostname 'smithi107' 2024-04-25T00:24:09.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.319+0000 7f887a622740 1 --2- 172.21.15.107:0/3915897244 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56544b66e2c0 0x56544b6706b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:09.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.319+0000 7f887a622740 4 client.17406 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:24:09.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.319+0000 7f887a622740 1 -- 172.21.15.107:0/3915897244 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56544b4c56f0 con 0x56544b66e2c0 2024-04-25T00:24:09.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.324+0000 7f88756b9640 1 --2- 172.21.15.107:0/3915897244 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56544b66e2c0 0x56544b6706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:09.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.331+0000 7f88756b9640 1 --2- 172.21.15.107:0/3915897244 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56544b66e2c0 0x56544b6706b0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c3998dc41df7441 server_cookie=7f3bdc1168ec5c50 in_seq=0 out_seq=0 2024-04-25T00:24:09.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.331+0000 7f8858ff9640 10 client.17406 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:09.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.333+0000 7f8858ff9640 1 -- 172.21.15.107:0/3915897244 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56544b4c56f0 con 0x56544b66e2c0 2024-04-25T00:24:09.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.333+0000 7f8858ff9640 10 client.17406 handle_command_reply: tid=0 2024-04-25T00:24:09.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.334+0000 7f887a622740 1 client.17406 shutdown 2024-04-25T00:24:09.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.336+0000 7f887a622740 1 -- 172.21.15.107:0/3915897244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56544b5e2500 msgr2=0x56544b5e0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:09.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.336+0000 7f887a622740 1 --2- 172.21.15.107:0/3915897244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56544b5e2500 0x56544b5e0970 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f88600b7b30 tx=0x7f88600bb860 comp rx=0 tx=0).stop 2024-04-25T00:24:09.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:09 smithi195 ceph-mon[32833]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T00:24:09.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.340+0000 7f887a622740 1 -- 172.21.15.107:0/3915897244 shutdown_connections 2024-04-25T00:24:09.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.340+0000 7f887a622740 1 --2- 172.21.15.107:0/3915897244 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56544b66e2c0 0x56544b6706b0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.340+0000 7f887a622740 1 --2- 172.21.15.107:0/3915897244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56544b5e6620 0x56544b5e0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.340+0000 7f887a622740 1 --2- 172.21.15.107:0/3915897244 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56544b5e2500 0x56544b5e0970 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.340+0000 7f887a622740 1 --2- 172.21.15.107:0/3915897244 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56544b5db630 0x56544b5e0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.340+0000 7f887a622740 1 -- 172.21.15.107:0/3915897244 >> 172.21.15.107:0/3915897244 conn(0x56544b5d7950 msgr2=0x56544b5d6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:09.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.340+0000 7f8858ff9640 0 client.17406 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:09.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.341+0000 7f887a622740 1 -- 172.21.15.107:0/3915897244 shutdown_connections 2024-04-25T00:24:09.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.341+0000 7f887a622740 1 -- 172.21.15.107:0/3915897244 wait complete. 2024-04-25T00:24:09.343 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:24:09.344 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:24:09.344 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-04-25T00:24:09.344 INFO:teuthology.orchestra.run.smithi107.stdout: "0e5e33e5-fe1d-4ad3-bf57-bd10e2d29ad2": { 2024-04-25T00:24:09.344 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-04-25T00:24:09.344 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "0e5e33e5-fe1d-4ad3-bf57-bd10e2d29ad2", 2024-04-25T00:24:09.344 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-04-25T00:24:09.344 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:24:09.344 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:24:09.344 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:24:09.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.343+0000 7f887a622740 20 client.17406 trim_cache size 0 max 16384 2024-04-25T00:24:09.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.345+0000 7f887791d640 1 -- 172.21.15.107:0/1368082554 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f885c0a2f10 msgr2=0x7f885c0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:09.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.345+0000 7f887791d640 1 --2- 172.21.15.107:0/1368082554 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f885c0a2f10 0x7f885c0a53d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8868005fd0 tx=0x7f8868007770 comp rx=0 tx=0).stop 2024-04-25T00:24:09.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.345+0000 7f887791d640 1 -- 172.21.15.107:0/1368082554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88701014e0 msgr2=0x7f8870110020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:09.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.345+0000 7f887791d640 1 --2- 172.21.15.107:0/1368082554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88701014e0 0x7f8870110020 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f886000d860 tx=0x7f8860034790 comp rx=0 tx=0).stop 2024-04-25T00:24:09.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.346+0000 7f887791d640 1 -- 172.21.15.107:0/1368082554 shutdown_connections 2024-04-25T00:24:09.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.346+0000 7f887791d640 1 --2- 172.21.15.107:0/1368082554 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f885c0a2f10 0x7f885c0a53d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.346+0000 7f887791d640 1 --2- 172.21.15.107:0/1368082554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f88701066a0 0x7f8870110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.346+0000 7f887791d640 1 --2- 172.21.15.107:0/1368082554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88701014e0 0x7f8870110020 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.346+0000 7f887791d640 1 --2- 172.21.15.107:0/1368082554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8870100b20 0x7f8870115fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:09.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.346+0000 7f887791d640 1 -- 172.21.15.107:0/1368082554 >> 172.21.15.107:0/1368082554 conn(0x7f88700761c0 msgr2=0x7f88700ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:09.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.346+0000 7f887791d640 1 -- 172.21.15.107:0/1368082554 shutdown_connections 2024-04-25T00:24:09.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:09.347+0000 7f887791d640 1 -- 172.21.15.107:0/1368082554 wait complete. 2024-04-25T00:24:09.364 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:09 smithi107 ceph-mon[29153]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T00:24:09.366 INFO:tasks.cephfs.filesystem:scrub status for tag:0e5e33e5-fe1d-4ad3-bf57-bd10e2d29ad2 - {'path': '/', 'tag': '0e5e33e5-fe1d-4ad3-bf57-bd10e2d29ad2', 'options': 'recursive,force'} 2024-04-25T00:24:09.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:09 smithi170 ceph-mon[32807]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T00:24:10.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:10 smithi195 ceph-mon[32833]: pgmap v237: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T00:24:10.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:10 smithi195 ceph-mon[32833]: scrub queued for path: / 2024-04-25T00:24:10.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:10 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:24:10.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:10 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:24:10.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:10 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:24:10.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:10 smithi170 ceph-mon[32807]: pgmap v237: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T00:24:10.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:10 smithi170 ceph-mon[32807]: scrub queued for path: / 2024-04-25T00:24:10.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:10 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:24:10.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:10 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:24:10.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:10 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:24:10.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:10 smithi107 ceph-mon[29153]: pgmap v237: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T00:24:10.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:10 smithi107 ceph-mon[29153]: scrub queued for path: / 2024-04-25T00:24:10.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:10 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:24:10.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:10 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:24:10.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:10 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:24:12.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:12 smithi195 ceph-mon[32833]: pgmap v238: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T00:24:12.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:12 smithi170 ceph-mon[32807]: pgmap v238: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T00:24:12.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:12 smithi107 ceph-mon[29153]: pgmap v238: 129 pgs: 129 active+clean; 254 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T00:24:13.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:13 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:24:13.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:13 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:24:13.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:13 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:24:14.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:14 smithi195 ceph-mon[32833]: pgmap v239: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:24:14.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:14 smithi170 ceph-mon[32807]: pgmap v239: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:24:14.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:14 smithi107 ceph-mon[29153]: pgmap v239: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:24:16.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:16 smithi195 ceph-mon[32833]: pgmap v240: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:24:16.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:16 smithi170 ceph-mon[32807]: pgmap v240: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:24:16.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:16 smithi107 ceph-mon[29153]: pgmap v240: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:24:17.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:17 smithi170 ceph-mon[32807]: pgmap v241: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:24:17.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:17 smithi170 ceph-mon[32807]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T00:24:17.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:17 smithi107 ceph-mon[29153]: pgmap v241: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:24:17.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:17 smithi107 ceph-mon[29153]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T00:24:18.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:17 smithi195 ceph-mon[32833]: pgmap v241: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:24:18.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:17 smithi195 ceph-mon[32833]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T00:24:19.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:19 smithi170 ceph-mon[32807]: pgmap v243: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:24:19.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:19 smithi107 ceph-mon[29153]: pgmap v243: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:24:20.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:19 smithi195 ceph-mon[32833]: pgmap v243: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:24:21.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:21 smithi170 ceph-mon[32807]: pgmap v244: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:24:21.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:21 smithi107 ceph-mon[29153]: pgmap v244: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:24:22.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:21 smithi195 ceph-mon[32833]: pgmap v244: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:24:23.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:23 smithi170 ceph-mon[32807]: pgmap v245: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:24:23.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:23 smithi107 ceph-mon[29153]: pgmap v245: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:24:24.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:23 smithi195 ceph-mon[32833]: pgmap v245: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:24:25.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:25 smithi170 ceph-mon[32807]: pgmap v246: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:24:25.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:25 smithi107 ceph-mon[29153]: pgmap v246: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:24:26.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:25 smithi195 ceph-mon[32833]: pgmap v246: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:24:27.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:27 smithi170 ceph-mon[32807]: pgmap v247: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:24:27.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:27 smithi107 ceph-mon[29153]: pgmap v247: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:24:28.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:27 smithi195 ceph-mon[32833]: pgmap v247: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:24:29.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:29 smithi170 ceph-mon[32807]: pgmap v248: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2024-04-25T00:24:29.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:29 smithi107 ceph-mon[29153]: pgmap v248: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2024-04-25T00:24:30.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:29 smithi195 ceph-mon[32833]: pgmap v248: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2024-04-25T00:24:31.934 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:31 smithi170 ceph-mon[32807]: pgmap v249: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:24:31.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:31 smithi107 ceph-mon[29153]: pgmap v249: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:24:32.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:31 smithi195 ceph-mon[32833]: pgmap v249: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:24:33.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:33 smithi170 ceph-mon[32807]: pgmap v250: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:24:33.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:33 smithi107 ceph-mon[29153]: pgmap v250: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:24:34.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:33 smithi195 ceph-mon[32833]: pgmap v250: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:24:35.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:35 smithi170 ceph-mon[32807]: pgmap v251: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:24:35.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:35 smithi107 ceph-mon[29153]: pgmap v251: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:24:36.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:35 smithi195 ceph-mon[32833]: pgmap v251: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:24:37.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:37 smithi170 ceph-mon[32807]: pgmap v252: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:24:37.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:37 smithi107 ceph-mon[29153]: pgmap v252: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:24:38.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:37 smithi195 ceph-mon[32833]: pgmap v252: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:24:38.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:38 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:38.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:38 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:39.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:38 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:39.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:24:39.368 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:24:39.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.470+0000 7fd4e836a640 1 -- 172.21.15.107:0/1676791431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4e0106810 msgr2=0x7fd4e0106bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:39.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.470+0000 7fd4e836a640 1 --2- 172.21.15.107:0/1676791431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4e0106810 0x7fd4e0106bf0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009930 tx=0x7fd4dc02a420 comp rx=0 tx=0).stop 2024-04-25T00:24:39.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.470+0000 7fd4e836a640 1 -- 172.21.15.107:0/1676791431 shutdown_connections 2024-04-25T00:24:39.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.470+0000 7fd4e836a640 1 --2- 172.21.15.107:0/1676791431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4e01012e0 0x7fd4e010d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.470+0000 7fd4e836a640 1 --2- 172.21.15.107:0/1676791431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4e0100920 0x7fd4e0100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.470+0000 7fd4e836a640 1 --2- 172.21.15.107:0/1676791431 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4e0106810 0x7fd4e0106bf0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.470+0000 7fd4e836a640 1 -- 172.21.15.107:0/1676791431 >> 172.21.15.107:0/1676791431 conn(0x7fd4e0077620 msgr2=0x7fd4e0077a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:39.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.470+0000 7fd4e836a640 1 -- 172.21.15.107:0/1676791431 shutdown_connections 2024-04-25T00:24:39.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.470+0000 7fd4e836a640 1 -- 172.21.15.107:0/1676791431 wait complete. 2024-04-25T00:24:39.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.471+0000 7fd4e836a640 1 Processor -- start 2024-04-25T00:24:39.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.471+0000 7fd4e836a640 1 -- start start 2024-04-25T00:24:39.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.471+0000 7fd4e836a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4e0100920 0x7fd4e019a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.471+0000 7fd4e836a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4e01012e0 0x7fd4e019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.471+0000 7fd4e836a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4e0106810 0x7fd4e019ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.471+0000 7fd4e836a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e0110ae0 con 0x7fd4e0106810 2024-04-25T00:24:39.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.471+0000 7fd4e836a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e0110960 con 0x7fd4e01012e0 2024-04-25T00:24:39.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.471+0000 7fd4e836a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e0110c60 con 0x7fd4e0100920 2024-04-25T00:24:39.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.471+0000 7fd4e6907640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4e0106810 0x7fd4e019ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.471+0000 7fd4e6907640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4e0106810 0x7fd4e019ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40456/0 (socket says 172.21.15.107:40456) 2024-04-25T00:24:39.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.471+0000 7fd4e6907640 1 -- 172.21.15.107:0/1924470305 learned_addr learned my addr 172.21.15.107:0/1924470305 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:39.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.471+0000 7fd4e5905640 1 --2- 172.21.15.107:0/1924470305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4e01012e0 0x7fd4e019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.471+0000 7fd4e6907640 1 -- 172.21.15.107:0/1924470305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4e0100920 msgr2=0x7fd4e019a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:39.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.471+0000 7fd4e6907640 1 --2- 172.21.15.107:0/1924470305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4e0100920 0x7fd4e019a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.471+0000 7fd4e6907640 1 -- 172.21.15.107:0/1924470305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4e01012e0 msgr2=0x7fd4e019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:39.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.471+0000 7fd4e6907640 1 --2- 172.21.15.107:0/1924470305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4e01012e0 0x7fd4e019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.471+0000 7fd4e6907640 1 -- 172.21.15.107:0/1924470305 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e019f1c0 con 0x7fd4e0106810 2024-04-25T00:24:39.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.471+0000 7fd4e5905640 1 --2- 172.21.15.107:0/1924470305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4e01012e0 0x7fd4e019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:39.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.472+0000 7fd4e6907640 1 --2- 172.21.15.107:0/1924470305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4e0106810 0x7fd4e019ea90 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fd4d800eaa0 tx=0x7fd4d800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:39.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.472+0000 7fd4d77fe640 1 -- 172.21.15.107:0/1924470305 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d800cdb0 con 0x7fd4e0106810 2024-04-25T00:24:39.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.472+0000 7fd4d77fe640 1 -- 172.21.15.107:0/1924470305 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4d8005020 con 0x7fd4e0106810 2024-04-25T00:24:39.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.472+0000 7fd4e836a640 1 -- 172.21.15.107:0/1924470305 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4e019f4b0 con 0x7fd4e0106810 2024-04-25T00:24:39.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.472+0000 7fd4d77fe640 1 -- 172.21.15.107:0/1924470305 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d80116b0 con 0x7fd4e0106810 2024-04-25T00:24:39.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.472+0000 7fd4e836a640 1 -- 172.21.15.107:0/1924470305 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4e01a6d40 con 0x7fd4e0106810 2024-04-25T00:24:39.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.474+0000 7fd4d77fe640 1 -- 172.21.15.107:0/1924470305 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fd4d8005560 con 0x7fd4e0106810 2024-04-25T00:24:39.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.474+0000 7fd4eb071740 1 Processor -- start 2024-04-25T00:24:39.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.474+0000 7fd4eb071740 1 -- start start 2024-04-25T00:24:39.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.475+0000 7fd4eb071740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c849a4c630 0x55c849a4ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.475+0000 7fd4eb071740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a4cf50 0x55c849a570e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.475+0000 7fd4eb071740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c849a57620 0x55c849a59a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.475+0000 7fd4eb071740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c8499b16b0 con 0x55c849a4cf50 2024-04-25T00:24:39.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.475+0000 7fd4eb071740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c849959880 con 0x55c849a4c630 2024-04-25T00:24:39.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.475+0000 7fd4eb071740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c8499ad670 con 0x55c849a57620 2024-04-25T00:24:39.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.475+0000 7fd4d77fe640 1 --2- 172.21.15.107:0/1924470305 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd4cc0a2d80 0x7fd4cc0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.475+0000 7fd4e6106640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a4cf50 0x55c849a570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.475+0000 7fd4e6106640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a4cf50 0x55c849a570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40472/0 (socket says 172.21.15.107:40472) 2024-04-25T00:24:39.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.475+0000 7fd4e6106640 1 -- 172.21.15.107:0/3553829357 learned_addr learned my addr 172.21.15.107:0/3553829357 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:39.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.475+0000 7fd4e6907640 1 --2- 172.21.15.107:0/1924470305 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd4cc0a2d80 0x7fd4cc0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.475+0000 7fd4e5905640 1 --2- 172.21.15.107:0/3553829357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c849a4c630 0x55c849a4ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.475+0000 7fd4d77fe640 1 -- 172.21.15.107:0/1924470305 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7fd4d80cfa70 con 0x7fd4e0106810 2024-04-25T00:24:39.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.475+0000 7fd4e5905640 1 --2- 172.21.15.107:0/3553829357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c849a57620 0x55c849a59a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.475+0000 7fd4e6106640 1 -- 172.21.15.107:0/3553829357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c849a57620 msgr2=0x55c849a59a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:39.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.475+0000 7fd4e6106640 1 --2- 172.21.15.107:0/3553829357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c849a57620 0x55c849a59a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.475+0000 7fd4e6106640 1 -- 172.21.15.107:0/3553829357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c849a4c630 msgr2=0x55c849a4ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:39.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.475+0000 7fd4e6106640 1 --2- 172.21.15.107:0/3553829357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c849a4c630 0x55c849a4ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.475+0000 7fd4e6106640 1 -- 172.21.15.107:0/3553829357 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c84978f7d0 con 0x55c849a4cf50 2024-04-25T00:24:39.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.475+0000 7fd4e5905640 1 --2- 172.21.15.107:0/3553829357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c849a57620 0x55c849a59a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:39.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.475+0000 7fd4e6907640 1 --2- 172.21.15.107:0/1924470305 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd4cc0a2d80 0x7fd4cc0a5240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80cceb0 tx=0x7fd4d8000f40 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:39.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.475+0000 7fd4e6106640 1 --2- 172.21.15.107:0/3553829357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a4cf50 0x55c849a570e0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc031540 tx=0x7fd4dc005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1646c7c2a9e3e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:39.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.475+0000 7fd4d4bfe640 1 -- 172.21.15.107:0/3553829357 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc03e030 con 0x55c849a4cf50 2024-04-25T00:24:39.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.476+0000 7fd4d4bfe640 1 -- 172.21.15.107:0/3553829357 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4dc032cb0 con 0x55c849a4cf50 2024-04-25T00:24:39.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.476+0000 7fd4d4bfe640 1 -- 172.21.15.107:0/3553829357 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc035040 con 0x55c849a4cf50 2024-04-25T00:24:39.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.476+0000 7fd4eb071740 1 -- 172.21.15.107:0/3553829357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a4cf50 msgr2=0x55c849a570e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:39.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.476+0000 7fd4eb071740 1 --2- 172.21.15.107:0/3553829357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a4cf50 0x55c849a570e0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc031540 tx=0x7fd4dc005d50 comp rx=0 tx=0).stop 2024-04-25T00:24:39.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.476+0000 7fd4eb071740 1 -- 172.21.15.107:0/3553829357 shutdown_connections 2024-04-25T00:24:39.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.476+0000 7fd4eb071740 1 --2- 172.21.15.107:0/3553829357 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c849a57620 0x55c849a59a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.476+0000 7fd4eb071740 1 --2- 172.21.15.107:0/3553829357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a4cf50 0x55c849a570e0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.476+0000 7fd4eb071740 1 --2- 172.21.15.107:0/3553829357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c849a4c630 0x55c849a4ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.476+0000 7fd4eb071740 1 -- 172.21.15.107:0/3553829357 >> 172.21.15.107:0/3553829357 conn(0x55c849a48950 msgr2=0x55c849a48d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:39.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.476+0000 7fd4eb071740 1 -- 172.21.15.107:0/3553829357 shutdown_connections 2024-04-25T00:24:39.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.476+0000 7fd4eb071740 1 -- 172.21.15.107:0/3553829357 wait complete. 2024-04-25T00:24:39.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.477+0000 7fd4eb071740 1 -- start start 2024-04-25T00:24:39.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.477+0000 7fd4eb071740 1 Processor -- start 2024-04-25T00:24:39.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.477+0000 7fd4eb071740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a4c630 0x55c849a544c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.477+0000 7fd4eb071740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c849a4cf50 0x55c849a54a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.477+0000 7fd4eb071740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c849a57620 0x55c849a54f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.477+0000 7fd4eb071740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c849a5a3b0 con 0x55c849a4c630 2024-04-25T00:24:39.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.477+0000 7fd4eb071740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c84996cf50 con 0x55c849a4cf50 2024-04-25T00:24:39.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.477+0000 7fd4eb071740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c849a5a530 con 0x55c849a57620 2024-04-25T00:24:39.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.477+0000 7fd4eb071740 10 client.0 _ll_register_callbacks cb 0x55c8499a8000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:39.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.477+0000 7fd4e5905640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a4c630 0x55c849a544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.477+0000 7fd4e5905640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a4c630 0x55c849a544c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40482/0 (socket says 172.21.15.107:40482) 2024-04-25T00:24:39.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.477+0000 7fd4e5905640 1 -- 172.21.15.107:0/595519101 learned_addr learned my addr 172.21.15.107:0/595519101 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:39.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.477+0000 7fd4e6106640 1 --2- 172.21.15.107:0/595519101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c849a4cf50 0x55c849a54a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.477+0000 7fd4e5905640 1 -- 172.21.15.107:0/595519101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c849a57620 msgr2=0x55c849a54f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:39.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.477+0000 7fd4e5905640 1 --2- 172.21.15.107:0/595519101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c849a57620 0x55c849a54f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.477+0000 7fd4e5905640 1 -- 172.21.15.107:0/595519101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c849a4cf50 msgr2=0x55c849a54a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:39.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.477+0000 7fd4e5905640 1 --2- 172.21.15.107:0/595519101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c849a4cf50 0x55c849a54a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.477+0000 7fd4e5905640 1 -- 172.21.15.107:0/595519101 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8496c53f0 con 0x55c849a4c630 2024-04-25T00:24:39.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.477+0000 7fd4e5905640 1 --2- 172.21.15.107:0/595519101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a4c630 0x55c849a544c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fd4d000ded0 tx=0x7fd4d0012e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc52a673bb1447e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:39.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.477+0000 7fd4c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:24:39.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.478+0000 7fd4c9ffb640 1 -- 172.21.15.107:0/595519101 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0010d20 con 0x55c849a4c630 2024-04-25T00:24:39.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.478+0000 7fd4c9ffb640 1 -- 172.21.15.107:0/595519101 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7fd4d00047a0 con 0x55c849a4c630 2024-04-25T00:24:39.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.478+0000 7fd4eb071740 1 -- 172.21.15.107:0/595519101 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4dc032cb0 con 0x55c849a4c630 2024-04-25T00:24:39.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.478+0000 7fd4c9ffb640 1 -- 172.21.15.107:0/595519101 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd4d0025050 con 0x55c849a4c630 2024-04-25T00:24:39.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.478+0000 7fd4eb071740 10 client.17418 fetch_fsmap learned FSMap version 34 2024-04-25T00:24:39.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.478+0000 7fd4eb071740 1 -- 172.21.15.107:0/595519101 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c8496850f0 con 0x55c849a4c630 2024-04-25T00:24:39.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.478+0000 7fd4c9ffb640 1 -- 172.21.15.107:0/595519101 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4dc032cb0 con 0x55c849a4c630 2024-04-25T00:24:39.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.478+0000 7fd4c9ffb640 1 -- 172.21.15.107:0/595519101 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fd4d000f6c0 con 0x55c849a4c630 2024-04-25T00:24:39.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.478+0000 7fd4eb071740 10 client.17418 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:24:39.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.478+0000 7fd4eb071740 10 client.17418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:39.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.478+0000 7fd4eb071740 20 client.17418 populate_metadata read hostname 'smithi107' 2024-04-25T00:24:39.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.478+0000 7fd4eb071740 1 --2- 172.21.15.107:0/595519101 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c849a67840 0x55c849a69cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.478+0000 7fd4eb071740 4 client.17418 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:39.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.478+0000 7fd4eb071740 1 -- 172.21.15.107:0/595519101 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8499bebf0 con 0x55c849a67840 2024-04-25T00:24:39.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.478+0000 7fd4e6106640 1 --2- 172.21.15.107:0/595519101 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c849a67840 0x55c849a69cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.479+0000 7fd4c9ffb640 10 client.17418 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:39.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.479+0000 7fd4e6106640 1 --2- 172.21.15.107:0/595519101 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c849a67840 0x55c849a69cc0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45b9001ed28af26e server_cookie=840a7564051a6de4 in_seq=0 out_seq=0 2024-04-25T00:24:39.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.479+0000 7fd4c9ffb640 1 -- 172.21.15.107:0/595519101 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c8499bebf0 con 0x55c849a67840 2024-04-25T00:24:39.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.479+0000 7fd4c9ffb640 10 client.17418 handle_command_reply: tid=0 2024-04-25T00:24:39.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.479+0000 7fd4eb071740 1 client.17418 shutdown 2024-04-25T00:24:39.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.480+0000 7fd4eb071740 1 -- 172.21.15.107:0/595519101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a4c630 msgr2=0x55c849a544c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:39.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.480+0000 7fd4eb071740 1 --2- 172.21.15.107:0/595519101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a4c630 0x55c849a544c0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fd4d000ded0 tx=0x7fd4d0012e10 comp rx=0 tx=0).stop 2024-04-25T00:24:39.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.480+0000 7fd4eb071740 1 -- 172.21.15.107:0/595519101 shutdown_connections 2024-04-25T00:24:39.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.480+0000 7fd4eb071740 1 --2- 172.21.15.107:0/595519101 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c849a67840 0x55c849a69cc0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.480+0000 7fd4eb071740 1 --2- 172.21.15.107:0/595519101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c849a57620 0x55c849a54f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.480+0000 7fd4eb071740 1 --2- 172.21.15.107:0/595519101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c849a4cf50 0x55c849a54a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.480+0000 7fd4eb071740 1 --2- 172.21.15.107:0/595519101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a4c630 0x55c849a544c0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.480+0000 7fd4c9ffb640 0 client.17418 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:39.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.480+0000 7fd4eb071740 1 -- 172.21.15.107:0/595519101 >> 172.21.15.107:0/595519101 conn(0x55c849a48950 msgr2=0x55c849a4e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:39.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.480+0000 7fd4eb071740 1 -- 172.21.15.107:0/595519101 shutdown_connections 2024-04-25T00:24:39.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.480+0000 7fd4eb071740 1 -- 172.21.15.107:0/595519101 wait complete. 2024-04-25T00:24:39.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.480+0000 7fd4eb071740 20 client.17418 trim_cache size 0 max 16384 2024-04-25T00:24:39.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.497+0000 7fd4eb071740 1 Processor -- start 2024-04-25T00:24:39.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.497+0000 7fd4eb071740 1 -- start start 2024-04-25T00:24:39.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.497+0000 7fd4eb071740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c849a4c630 0x55c849a632a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.497+0000 7fd4eb071740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c849a4cf50 0x55c849a53cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.497+0000 7fd4eb071740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a57620 0x55c849a54b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.497+0000 7fd4eb071740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e01b35e0 con 0x55c849a57620 2024-04-25T00:24:39.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.497+0000 7fd4eb071740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e01b0eb0 con 0x55c849a4c630 2024-04-25T00:24:39.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.497+0000 7fd4eb071740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e01aed20 con 0x55c849a4cf50 2024-04-25T00:24:39.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.497+0000 7fd4e5905640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a57620 0x55c849a54b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.497+0000 7fd4e5905640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a57620 0x55c849a54b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40494/0 (socket says 172.21.15.107:40494) 2024-04-25T00:24:39.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.497+0000 7fd4e5905640 1 -- 172.21.15.107:0/3880948043 learned_addr learned my addr 172.21.15.107:0/3880948043 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:39.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.497+0000 7fd4e5905640 1 --2- 172.21.15.107:0/3880948043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c849a4c630 0x55c849a632a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.497+0000 7fd4e5905640 1 -- 172.21.15.107:0/3880948043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c849a4cf50 msgr2=0x55c849a53cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:39.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.497+0000 7fd4e5905640 1 --2- 172.21.15.107:0/3880948043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c849a4cf50 0x55c849a53cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.497+0000 7fd4e5905640 1 -- 172.21.15.107:0/3880948043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c849a4c630 msgr2=0x55c849a632a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:39.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.497+0000 7fd4e5905640 1 --2- 172.21.15.107:0/3880948043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c849a4c630 0x55c849a632a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.497+0000 7fd4e5905640 1 -- 172.21.15.107:0/3880948043 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c849a3a090 con 0x55c849a57620 2024-04-25T00:24:39.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.498+0000 7fd4e5905640 1 --2- 172.21.15.107:0/3880948043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a57620 0x55c849a54b90 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0005ee0 tx=0x7fd4d0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726067e9e50c8535 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:39.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.498+0000 7fd4cbfff640 1 -- 172.21.15.107:0/3880948043 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0010b20 con 0x55c849a57620 2024-04-25T00:24:39.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.498+0000 7fd4cbfff640 1 -- 172.21.15.107:0/3880948043 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4d0010cc0 con 0x55c849a57620 2024-04-25T00:24:39.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.498+0000 7fd4cbfff640 1 -- 172.21.15.107:0/3880948043 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d001a8a0 con 0x55c849a57620 2024-04-25T00:24:39.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.498+0000 7fd4eb071740 1 -- 172.21.15.107:0/3880948043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a57620 msgr2=0x55c849a54b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:39.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.498+0000 7fd4eb071740 1 --2- 172.21.15.107:0/3880948043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a57620 0x55c849a54b90 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0005ee0 tx=0x7fd4d0001e70 comp rx=0 tx=0).stop 2024-04-25T00:24:39.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.498+0000 7fd4eb071740 1 -- 172.21.15.107:0/3880948043 shutdown_connections 2024-04-25T00:24:39.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.498+0000 7fd4eb071740 1 --2- 172.21.15.107:0/3880948043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a57620 0x55c849a54b90 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.498+0000 7fd4eb071740 1 --2- 172.21.15.107:0/3880948043 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c849a4cf50 0x55c849a53cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.498+0000 7fd4eb071740 1 --2- 172.21.15.107:0/3880948043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c849a4c630 0x55c849a632a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.498+0000 7fd4eb071740 1 -- 172.21.15.107:0/3880948043 >> 172.21.15.107:0/3880948043 conn(0x55c849a48950 msgr2=0x55c849a54480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:39.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.498+0000 7fd4eb071740 1 -- 172.21.15.107:0/3880948043 shutdown_connections 2024-04-25T00:24:39.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.498+0000 7fd4eb071740 1 -- 172.21.15.107:0/3880948043 wait complete. 2024-04-25T00:24:39.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.499+0000 7fd4eb071740 1 -- start start 2024-04-25T00:24:39.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.499+0000 7fd4eb071740 1 Processor -- start 2024-04-25T00:24:39.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.499+0000 7fd4eb071740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a4c630 0x55c849a5f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.499+0000 7fd4eb071740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c849a4cf50 0x55c849a5fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.499+0000 7fd4eb071740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c849a57620 0x55c849a50950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.499+0000 7fd4eb071740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e01af370 con 0x55c849a4c630 2024-04-25T00:24:39.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.499+0000 7fd4eb071740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e01b07e0 con 0x55c849a4cf50 2024-04-25T00:24:39.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.499+0000 7fd4eb071740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e01afa40 con 0x55c849a57620 2024-04-25T00:24:39.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.499+0000 7fd4eb071740 10 client.0 _ll_register_callbacks cb 0x7fd4d000dce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:39.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.499+0000 7fd4e5905640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a4c630 0x55c849a5f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.499+0000 7fd4e5905640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a4c630 0x55c849a5f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40506/0 (socket says 172.21.15.107:40506) 2024-04-25T00:24:39.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.499+0000 7fd4e5905640 1 -- 172.21.15.107:0/3579341148 learned_addr learned my addr 172.21.15.107:0/3579341148 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:39.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.499+0000 7fd4e6106640 1 --2- 172.21.15.107:0/3579341148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c849a4cf50 0x55c849a5fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.499+0000 7fd4e5905640 1 -- 172.21.15.107:0/3579341148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c849a57620 msgr2=0x55c849a50950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:39.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.499+0000 7fd4e5905640 1 --2- 172.21.15.107:0/3579341148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c849a57620 0x55c849a50950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.499+0000 7fd4e5905640 1 -- 172.21.15.107:0/3579341148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c849a4cf50 msgr2=0x55c849a5fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:39.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.499+0000 7fd4e5905640 1 --2- 172.21.15.107:0/3579341148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c849a4cf50 0x55c849a5fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.499+0000 7fd4e5905640 1 -- 172.21.15.107:0/3579341148 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c849a51150 con 0x55c849a4c630 2024-04-25T00:24:39.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.499+0000 7fd4e5905640 1 --2- 172.21.15.107:0/3579341148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a4c630 0x55c849a5f7b0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fd4d000fc50 tx=0x7fd4d0012660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56825c5f36f4f171 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:39.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.499+0000 7fd4c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:24:39.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.500+0000 7fd4c9ffb640 1 -- 172.21.15.107:0/3579341148 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0039060 con 0x55c849a4c630 2024-04-25T00:24:39.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.500+0000 7fd4c9ffb640 1 -- 172.21.15.107:0/3579341148 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7fd4d000f5c0 con 0x55c849a4c630 2024-04-25T00:24:39.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.500+0000 7fd4eb071740 1 -- 172.21.15.107:0/3579341148 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4d0010cc0 con 0x55c849a4c630 2024-04-25T00:24:39.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.500+0000 7fd4c9ffb640 1 -- 172.21.15.107:0/3579341148 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd4d0010b20 con 0x55c849a4c630 2024-04-25T00:24:39.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.500+0000 7fd4eb071740 10 client.17424 fetch_fsmap learned FSMap version 34 2024-04-25T00:24:39.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.500+0000 7fd4eb071740 1 -- 172.21.15.107:0/3579341148 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c849acd830 con 0x55c849a4c630 2024-04-25T00:24:39.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.500+0000 7fd4c9ffb640 1 -- 172.21.15.107:0/3579341148 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4d0010cc0 con 0x55c849a4c630 2024-04-25T00:24:39.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.500+0000 7fd4c9ffb640 1 -- 172.21.15.107:0/3579341148 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fd4d0005200 con 0x55c849a4c630 2024-04-25T00:24:39.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.500+0000 7fd4eb071740 10 client.17424 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:24:39.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.500+0000 7fd4eb071740 10 client.17424 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:39.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.500+0000 7fd4eb071740 20 client.17424 populate_metadata read hostname 'smithi107' 2024-04-25T00:24:39.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.500+0000 7fd4eb071740 1 --2- 172.21.15.107:0/3579341148 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c849acde40 0x55c849ad0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.500+0000 7fd4eb071740 4 client.17424 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:24:39.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.500+0000 7fd4eb071740 1 -- 172.21.15.107:0/3579341148 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c8499b8f40 con 0x55c849acde40 2024-04-25T00:24:39.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.500+0000 7fd4e6106640 1 --2- 172.21.15.107:0/3579341148 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c849acde40 0x55c849ad0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.501+0000 7fd4c9ffb640 10 client.17424 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:39.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.501+0000 7fd4e6106640 1 --2- 172.21.15.107:0/3579341148 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c849acde40 0x55c849ad0260 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce22e13e6f5b7b80 server_cookie=9480e006dfe6bae5 in_seq=0 out_seq=0 2024-04-25T00:24:39.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.501+0000 7fd4c9ffb640 1 -- 172.21.15.107:0/3579341148 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c8499b8f40 con 0x55c849acde40 2024-04-25T00:24:39.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.501+0000 7fd4c9ffb640 10 client.17424 handle_command_reply: tid=0 2024-04-25T00:24:39.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.501+0000 7fd4eb071740 1 client.17424 shutdown 2024-04-25T00:24:39.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.501+0000 7fd4eb071740 1 -- 172.21.15.107:0/3579341148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a4c630 msgr2=0x55c849a5f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:39.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.501+0000 7fd4eb071740 1 --2- 172.21.15.107:0/3579341148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a4c630 0x55c849a5f7b0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fd4d000fc50 tx=0x7fd4d0012660 comp rx=0 tx=0).stop 2024-04-25T00:24:39.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.501+0000 7fd4eb071740 1 -- 172.21.15.107:0/3579341148 shutdown_connections 2024-04-25T00:24:39.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.501+0000 7fd4eb071740 1 --2- 172.21.15.107:0/3579341148 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c849acde40 0x55c849ad0260 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.501+0000 7fd4eb071740 1 --2- 172.21.15.107:0/3579341148 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c849a57620 0x55c849a50950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.501+0000 7fd4c9ffb640 0 client.17424 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:39.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.501+0000 7fd4eb071740 1 --2- 172.21.15.107:0/3579341148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c849a4cf50 0x55c849a5fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.501+0000 7fd4eb071740 1 --2- 172.21.15.107:0/3579341148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c849a4c630 0x55c849a5f7b0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.501+0000 7fd4eb071740 1 -- 172.21.15.107:0/3579341148 >> 172.21.15.107:0/3579341148 conn(0x55c849a48950 msgr2=0x55c849a4e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:39.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.501+0000 7fd4eb071740 1 -- 172.21.15.107:0/3579341148 shutdown_connections 2024-04-25T00:24:39.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.501+0000 7fd4eb071740 1 -- 172.21.15.107:0/3579341148 wait complete. 2024-04-25T00:24:39.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.501+0000 7fd4eb071740 20 client.17424 trim_cache size 0 max 16384 2024-04-25T00:24:39.510 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:24:39.510 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-04-25T00:24:39.510 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-04-25T00:24:39.510 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:24:39.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.501+0000 7fd4e836a640 1 -- 172.21.15.107:0/1924470305 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd4cc0a2d80 msgr2=0x7fd4cc0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:39.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.501+0000 7fd4e836a640 1 --2- 172.21.15.107:0/1924470305 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd4cc0a2d80 0x7fd4cc0a5240 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd4d80cceb0 tx=0x7fd4d8000f40 comp rx=0 tx=0).stop 2024-04-25T00:24:39.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.501+0000 7fd4e836a640 1 -- 172.21.15.107:0/1924470305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4e0106810 msgr2=0x7fd4e019ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:39.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.501+0000 7fd4e836a640 1 --2- 172.21.15.107:0/1924470305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4e0106810 0x7fd4e019ea90 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fd4d800eaa0 tx=0x7fd4d800ef60 comp rx=0 tx=0).stop 2024-04-25T00:24:39.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.502+0000 7fd4e836a640 1 -- 172.21.15.107:0/1924470305 shutdown_connections 2024-04-25T00:24:39.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.502+0000 7fd4e836a640 1 --2- 172.21.15.107:0/1924470305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4e0106810 0x7fd4e019ea90 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.502+0000 7fd4e836a640 1 --2- 172.21.15.107:0/1924470305 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd4cc0a2d80 0x7fd4cc0a5240 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.502+0000 7fd4e836a640 1 --2- 172.21.15.107:0/1924470305 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd4e01012e0 0x7fd4e019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.502+0000 7fd4e836a640 1 --2- 172.21.15.107:0/1924470305 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4e0100920 0x7fd4e019a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.502+0000 7fd4e836a640 1 -- 172.21.15.107:0/1924470305 >> 172.21.15.107:0/1924470305 conn(0x7fd4e0077620 msgr2=0x7fd4e00ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:39.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.502+0000 7fd4e836a640 1 -- 172.21.15.107:0/1924470305 shutdown_connections 2024-04-25T00:24:39.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.502+0000 7fd4e836a640 1 -- 172.21.15.107:0/1924470305 wait complete. 2024-04-25T00:24:39.513 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:24:39.513 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:24:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.610+0000 7f19ef65d640 1 -- 172.21.15.107:0/37348758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19e81020d0 msgr2=0x7f19e810e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.610+0000 7f19ef65d640 1 --2- 172.21.15.107:0/37348758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19e81020d0 0x7f19e810e5e0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009cd0 tx=0x7f19e002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.610+0000 7f19ef65d640 1 -- 172.21.15.107:0/37348758 shutdown_connections 2024-04-25T00:24:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.610+0000 7f19ef65d640 1 --2- 172.21.15.107:0/37348758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19e81020d0 0x7f19e810e5e0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.610+0000 7f19ef65d640 1 --2- 172.21.15.107:0/37348758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19e8101710 0x7f19e8101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.610+0000 7f19ef65d640 1 --2- 172.21.15.107:0/37348758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19e8107720 0x7f19e8107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.610+0000 7f19ef65d640 1 -- 172.21.15.107:0/37348758 >> 172.21.15.107:0/37348758 conn(0x7f19e8078b00 msgr2=0x7f19e8078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.610+0000 7f19ef65d640 1 -- 172.21.15.107:0/37348758 shutdown_connections 2024-04-25T00:24:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.610+0000 7f19ef65d640 1 -- 172.21.15.107:0/37348758 wait complete. 2024-04-25T00:24:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.611+0000 7f19ef65d640 1 Processor -- start 2024-04-25T00:24:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.611+0000 7f19ef65d640 1 -- start start 2024-04-25T00:24:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.611+0000 7f19ef65d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19e8101710 0x7f19e8073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.611+0000 7f19ef65d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19e81020d0 0x7f19e8073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.611+0000 7f19ef65d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19e8107720 0x7f19e80744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.611+0000 7f19ef65d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f19e8111b30 con 0x7f19e8107720 2024-04-25T00:24:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.611+0000 7f19ef65d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f19e81119b0 con 0x7f19e81020d0 2024-04-25T00:24:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.611+0000 7f19ef65d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f19e8111cb0 con 0x7f19e8101710 2024-04-25T00:24:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.611+0000 7f19edbfa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19e8107720 0x7f19e80744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.611+0000 7f19edbfa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19e8107720 0x7f19e80744c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40518/0 (socket says 172.21.15.107:40518) 2024-04-25T00:24:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.611+0000 7f19edbfa640 1 -- 172.21.15.107:0/1597083199 learned_addr learned my addr 172.21.15.107:0/1597083199 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.611+0000 7f19ed3f9640 1 --2- 172.21.15.107:0/1597083199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19e8101710 0x7f19e8073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.611+0000 7f19ecbf8640 1 --2- 172.21.15.107:0/1597083199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19e81020d0 0x7f19e8073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.612+0000 7f19edbfa640 1 -- 172.21.15.107:0/1597083199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19e8101710 msgr2=0x7f19e8073a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.612+0000 7f19edbfa640 1 --2- 172.21.15.107:0/1597083199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19e8101710 0x7f19e8073a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.612+0000 7f19edbfa640 1 -- 172.21.15.107:0/1597083199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19e81020d0 msgr2=0x7f19e8073f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.612+0000 7f19edbfa640 1 --2- 172.21.15.107:0/1597083199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19e81020d0 0x7f19e8073f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.612+0000 7f19edbfa640 1 -- 172.21.15.107:0/1597083199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e81a64b0 con 0x7f19e8107720 2024-04-25T00:24:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.612+0000 7f19ecbf8640 1 --2- 172.21.15.107:0/1597083199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19e81020d0 0x7f19e8073f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.612+0000 7f19edbfa640 1 --2- 172.21.15.107:0/1597083199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19e8107720 0x7f19e80744c0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f19e0005cc0 tx=0x7f19e0032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.612+0000 7f19de7fc640 1 -- 172.21.15.107:0/1597083199 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0004040 con 0x7f19e8107720 2024-04-25T00:24:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.612+0000 7f19de7fc640 1 -- 172.21.15.107:0/1597083199 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e00046c0 con 0x7f19e8107720 2024-04-25T00:24:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.612+0000 7f19ef65d640 1 -- 172.21.15.107:0/1597083199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e81a6740 con 0x7f19e8107720 2024-04-25T00:24:39.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.612+0000 7f19de7fc640 1 -- 172.21.15.107:0/1597083199 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0030e00 con 0x7f19e8107720 2024-04-25T00:24:39.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.612+0000 7f19ef65d640 1 -- 172.21.15.107:0/1597083199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e81a6c90 con 0x7f19e8107720 2024-04-25T00:24:39.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.613+0000 7f19ef65d640 1 -- 172.21.15.107:0/1597083199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e80689b0 con 0x7f19e8107720 2024-04-25T00:24:39.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.614+0000 7f19de7fc640 1 -- 172.21.15.107:0/1597083199 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f19e00305a0 con 0x7f19e8107720 2024-04-25T00:24:39.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.615+0000 7f19de7fc640 1 --2- 172.21.15.107:0/1597083199 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f19d40a2d30 0x7f19d40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.615+0000 7f19ed3f9640 1 --2- 172.21.15.107:0/1597083199 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f19d40a2d30 0x7f19d40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.615+0000 7f19de7fc640 1 -- 172.21.15.107:0/1597083199 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f19e002e030 con 0x7f19e8107720 2024-04-25T00:24:39.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.615+0000 7f19ed3f9640 1 --2- 172.21.15.107:0/1597083199 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f19d40a2d30 0x7f19d40a51f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009ae0 tx=0x7f19e4005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:39.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.618+0000 7f19de7fc640 1 -- 172.21.15.107:0/1597083199 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f19e004b060 con 0x7f19e8107720 2024-04-25T00:24:39.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.830+0000 7f19ef65d640 1 -- 172.21.15.107:0/1597083199 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19e8109ca0 con 0x7f19e8107720 2024-04-25T00:24:39.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.831+0000 7f19de7fc640 1 -- 172.21.15.107:0/1597083199 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11897 (secure 0 0 0) 0x7f19e00bdaa0 con 0x7f19e8107720 2024-04-25T00:24:39.832 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:24:39.832 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:22:27.402680+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24493,24521]},"id":1}]} 2024-04-25T00:24:39.832 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 34 2024-04-25T00:24:39.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.833+0000 7f19ef65d640 1 -- 172.21.15.107:0/1597083199 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f19d40a2d30 msgr2=0x7f19d40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:39.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.833+0000 7f19ef65d640 1 --2- 172.21.15.107:0/1597083199 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f19d40a2d30 0x7f19d40a51f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009ae0 tx=0x7f19e4005f90 comp rx=0 tx=0).stop 2024-04-25T00:24:39.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.833+0000 7f19ef65d640 1 -- 172.21.15.107:0/1597083199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19e8107720 msgr2=0x7f19e80744c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:39.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.833+0000 7f19ef65d640 1 --2- 172.21.15.107:0/1597083199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19e8107720 0x7f19e80744c0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f19e0005cc0 tx=0x7f19e0032f40 comp rx=0 tx=0).stop 2024-04-25T00:24:39.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.833+0000 7f19ef65d640 1 -- 172.21.15.107:0/1597083199 shutdown_connections 2024-04-25T00:24:39.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.833+0000 7f19ef65d640 1 --2- 172.21.15.107:0/1597083199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19e8107720 0x7f19e80744c0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.833+0000 7f19ef65d640 1 --2- 172.21.15.107:0/1597083199 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f19d40a2d30 0x7f19d40a51f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.833+0000 7f19ef65d640 1 --2- 172.21.15.107:0/1597083199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f19e81020d0 0x7f19e8073f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.833+0000 7f19ef65d640 1 --2- 172.21.15.107:0/1597083199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19e8101710 0x7f19e8073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.833+0000 7f19ef65d640 1 -- 172.21.15.107:0/1597083199 >> 172.21.15.107:0/1597083199 conn(0x7f19e8078b00 msgr2=0x7f19e81040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:39.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.833+0000 7f19ef65d640 1 -- 172.21.15.107:0/1597083199 shutdown_connections 2024-04-25T00:24:39.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.833+0000 7f19ef65d640 1 -- 172.21.15.107:0/1597083199 wait complete. 2024-04-25T00:24:39.844 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:24:39.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:39 smithi170 ceph-mon[32807]: pgmap v253: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:24:39.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:39 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:39.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:39 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:39.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:39 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:24:39.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.943+0000 7fe5bf0b3640 1 -- 172.21.15.107:0/1444691164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5b8100610 msgr2=0x7fe5b8100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:39.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.943+0000 7fe5bf0b3640 1 --2- 172.21.15.107:0/1444691164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5b8100610 0x7fe5b8100a90 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0009930 tx=0x7fe5a002a420 comp rx=0 tx=0).stop 2024-04-25T00:24:39.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.944+0000 7fe5bf0b3640 1 -- 172.21.15.107:0/1444691164 shutdown_connections 2024-04-25T00:24:39.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.944+0000 7fe5bf0b3640 1 --2- 172.21.15.107:0/1444691164 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5b8100fd0 0x7fe5b810d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.944+0000 7fe5bf0b3640 1 --2- 172.21.15.107:0/1444691164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5b8100610 0x7fe5b8100a90 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.944+0000 7fe5bf0b3640 1 --2- 172.21.15.107:0/1444691164 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5b81066b0 0x7fe5b8106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.944+0000 7fe5bf0b3640 1 -- 172.21.15.107:0/1444691164 >> 172.21.15.107:0/1444691164 conn(0x7fe5b8077740 msgr2=0x7fe5b8077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:39.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.944+0000 7fe5bf0b3640 1 -- 172.21.15.107:0/1444691164 shutdown_connections 2024-04-25T00:24:39.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.944+0000 7fe5bf0b3640 1 -- 172.21.15.107:0/1444691164 wait complete. 2024-04-25T00:24:39.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.944+0000 7fe5bf0b3640 1 Processor -- start 2024-04-25T00:24:39.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.944+0000 7fe5bf0b3640 1 -- start start 2024-04-25T00:24:39.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.944+0000 7fe5bf0b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5b8100610 0x7fe5b8198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.944+0000 7fe5bf0b3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5b8100fd0 0x7fe5b8199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.944+0000 7fe5bf0b3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5b81066b0 0x7fe5b81a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.944+0000 7fe5bf0b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b8110a30 con 0x7fe5b8100610 2024-04-25T00:24:39.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.944+0000 7fe5bf0b3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b81108b0 con 0x7fe5b81066b0 2024-04-25T00:24:39.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.944+0000 7fe5bf0b3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b8110bb0 con 0x7fe5b8100fd0 2024-04-25T00:24:39.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.944+0000 7fe5bce4f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5b8100610 0x7fe5b8198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.944+0000 7fe5bce4f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5b8100610 0x7fe5b8198e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40530/0 (socket says 172.21.15.107:40530) 2024-04-25T00:24:39.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.944+0000 7fe5bce4f640 1 -- 172.21.15.107:0/2934627768 learned_addr learned my addr 172.21.15.107:0/2934627768 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:39.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.945+0000 7fe5b7fff640 1 --2- 172.21.15.107:0/2934627768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5b8100fd0 0x7fe5b8199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.945+0000 7fe5bd650640 1 --2- 172.21.15.107:0/2934627768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5b81066b0 0x7fe5b81a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.945+0000 7fe5bce4f640 1 -- 172.21.15.107:0/2934627768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5b8100fd0 msgr2=0x7fe5b8199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:39.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.945+0000 7fe5bce4f640 1 --2- 172.21.15.107:0/2934627768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5b8100fd0 0x7fe5b8199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.945+0000 7fe5bce4f640 1 -- 172.21.15.107:0/2934627768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5b81066b0 msgr2=0x7fe5b81a3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:39.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.945+0000 7fe5bce4f640 1 --2- 172.21.15.107:0/2934627768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5b81066b0 0x7fe5b81a3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.945+0000 7fe5bce4f640 1 -- 172.21.15.107:0/2934627768 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b81a43f0 con 0x7fe5b8100610 2024-04-25T00:24:39.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.945+0000 7fe5bd650640 1 --2- 172.21.15.107:0/2934627768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5b81066b0 0x7fe5b81a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:39.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.945+0000 7fe5b7fff640 1 --2- 172.21.15.107:0/2934627768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5b8100fd0 0x7fe5b8199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:39.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.945+0000 7fe5bce4f640 1 --2- 172.21.15.107:0/2934627768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5b8100610 0x7fe5b8198e30 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800d950 tx=0x7fe5a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:39.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.945+0000 7fe5b5ffb640 1 -- 172.21.15.107:0/2934627768 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8017070 con 0x7fe5b8100610 2024-04-25T00:24:39.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.945+0000 7fe5b5ffb640 1 -- 172.21.15.107:0/2934627768 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5a8009c30 con 0x7fe5b8100610 2024-04-25T00:24:39.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.945+0000 7fe5bf0b3640 1 -- 172.21.15.107:0/2934627768 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b81a45c0 con 0x7fe5b8100610 2024-04-25T00:24:39.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.945+0000 7fe5b5ffb640 1 -- 172.21.15.107:0/2934627768 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a800f600 con 0x7fe5b8100610 2024-04-25T00:24:39.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.945+0000 7fe5bf0b3640 1 -- 172.21.15.107:0/2934627768 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b81a4a90 con 0x7fe5b8100610 2024-04-25T00:24:39.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.947+0000 7fe5b5ffb640 1 -- 172.21.15.107:0/2934627768 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe5a801a070 con 0x7fe5b8100610 2024-04-25T00:24:39.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.948+0000 7fe5c1dbe740 1 Processor -- start 2024-04-25T00:24:39.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.948+0000 7fe5c1dbe740 1 -- start start 2024-04-25T00:24:39.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.948+0000 7fe5b5ffb640 1 --2- 172.21.15.107:0/2934627768 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe5a40a2d00 0x7fe5a40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.948+0000 7fe5c1dbe740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d9effd480 0x564d9effd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.948+0000 7fe5c1dbe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9effdda0 0x564d9f00c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.948+0000 7fe5c1dbe740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d9f00c980 0x564d9f00ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.948+0000 7fe5c1dbe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564d9ef62670 con 0x564d9effdda0 2024-04-25T00:24:39.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.948+0000 7fe5c1dbe740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564d9ef0a970 con 0x564d9f00c980 2024-04-25T00:24:39.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.948+0000 7fe5c1dbe740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564d9ef5e630 con 0x564d9effd480 2024-04-25T00:24:39.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.948+0000 7fe5b7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9effdda0 0x564d9f00c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.948+0000 7fe5bd650640 1 --2- 172.21.15.107:0/2934627768 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe5a40a2d00 0x7fe5a40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.948+0000 7fe5b5ffb640 1 -- 172.21.15.107:0/2934627768 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7fe5a80cf2d0 con 0x7fe5b8100610 2024-04-25T00:24:39.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.948+0000 7fe5b7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9effdda0 0x564d9f00c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40546/0 (socket says 172.21.15.107:40546) 2024-04-25T00:24:39.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.948+0000 7fe5b7fff640 1 -- 172.21.15.107:0/3412517570 learned_addr learned my addr 172.21.15.107:0/3412517570 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:39.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.948+0000 7fe5bce4f640 1 --2- 172.21.15.107:0/3412517570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d9effd480 0x564d9effd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.948+0000 7fe5b7fff640 1 -- 172.21.15.107:0/3412517570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d9effd480 msgr2=0x564d9effd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:39.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.948+0000 7fe5b7fff640 1 --2- 172.21.15.107:0/3412517570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d9effd480 0x564d9effd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.948+0000 7fe5b7fff640 1 -- 172.21.15.107:0/3412517570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d9f00c980 msgr2=0x564d9f00ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.948+0000 7fe5b7fff640 1 --2- 172.21.15.107:0/3412517570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d9f00c980 0x564d9f00ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.948+0000 7fe5b7fff640 1 -- 172.21.15.107:0/3412517570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d9ef2e400 con 0x564d9effdda0 2024-04-25T00:24:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.948+0000 7fe5bd650640 1 --2- 172.21.15.107:0/2934627768 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe5a40a2d00 0x7fe5a40a51c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009270 tx=0x7fe5b0009880 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.948+0000 7fe5b7fff640 1 --2- 172.21.15.107:0/3412517570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9effdda0 0x564d9f00c440 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0009930 tx=0x7fe5a0032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655ea11139c7f223 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.949+0000 7fe5aeffd640 1 -- 172.21.15.107:0/3412517570 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0035030 con 0x564d9effdda0 2024-04-25T00:24:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.949+0000 7fe5aeffd640 1 -- 172.21.15.107:0/3412517570 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5a00317b0 con 0x564d9effdda0 2024-04-25T00:24:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.949+0000 7fe5aeffd640 1 -- 172.21.15.107:0/3412517570 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a00046c0 con 0x564d9effdda0 2024-04-25T00:24:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.949+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/3412517570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9effdda0 msgr2=0x564d9f00c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.949+0000 7fe5c1dbe740 1 --2- 172.21.15.107:0/3412517570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9effdda0 0x564d9f00c440 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0009930 tx=0x7fe5a0032cb0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.949+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/3412517570 shutdown_connections 2024-04-25T00:24:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.949+0000 7fe5c1dbe740 1 --2- 172.21.15.107:0/3412517570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d9f00c980 0x564d9f00ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.949+0000 7fe5c1dbe740 1 --2- 172.21.15.107:0/3412517570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9effdda0 0x564d9f00c440 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.949+0000 7fe5c1dbe740 1 --2- 172.21.15.107:0/3412517570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d9effd480 0x564d9effd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.949+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/3412517570 >> 172.21.15.107:0/3412517570 conn(0x564d9eff98a0 msgr2=0x564d9eff9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.949+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/3412517570 shutdown_connections 2024-04-25T00:24:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.949+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/3412517570 wait complete. 2024-04-25T00:24:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.950+0000 7fe5c1dbe740 1 -- start start 2024-04-25T00:24:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.950+0000 7fe5c1dbe740 1 Processor -- start 2024-04-25T00:24:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.950+0000 7fe5c1dbe740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d9effd480 0x564d9f00c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.950+0000 7fe5c1dbe740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d9effdda0 0x564d9f0094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.950+0000 7fe5c1dbe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9f00c980 0x564d9f009a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.950+0000 7fe5c1dbe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564d9f00f6e0 con 0x564d9f00c980 2024-04-25T00:24:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.950+0000 7fe5c1dbe740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564d9ef1e050 con 0x564d9effdda0 2024-04-25T00:24:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.950+0000 7fe5c1dbe740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564d9f00f860 con 0x564d9effd480 2024-04-25T00:24:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.950+0000 7fe5c1dbe740 10 client.0 _ll_register_callbacks cb 0x564d9ec75be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.950+0000 7fe5bd650640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9f00c980 0x564d9f009a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.950+0000 7fe5bd650640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9f00c980 0x564d9f009a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40552/0 (socket says 172.21.15.107:40552) 2024-04-25T00:24:39.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.950+0000 7fe5bd650640 1 -- 172.21.15.107:0/1378645902 learned_addr learned my addr 172.21.15.107:0/1378645902 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:39.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.950+0000 7fe5bd650640 1 -- 172.21.15.107:0/1378645902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d9effd480 msgr2=0x564d9f00c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:39.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.950+0000 7fe5bd650640 1 --2- 172.21.15.107:0/1378645902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d9effd480 0x564d9f00c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.950+0000 7fe5bd650640 1 -- 172.21.15.107:0/1378645902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d9effdda0 msgr2=0x564d9f0094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:39.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.950+0000 7fe5bd650640 1 --2- 172.21.15.107:0/1378645902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d9effdda0 0x564d9f0094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.951+0000 7fe5bd650640 1 -- 172.21.15.107:0/1378645902 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d9eeefdc0 con 0x564d9f00c980 2024-04-25T00:24:39.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.951+0000 7fe5bd650640 1 --2- 172.21.15.107:0/1378645902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9f00c980 0x564d9f009a20 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00127f0 tx=0x7fe5b0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9943c003f8d9f5e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:39.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.951+0000 7fe597fff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:24:39.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.951+0000 7fe597fff640 1 -- 172.21.15.107:0/1378645902 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0023070 con 0x564d9f00c980 2024-04-25T00:24:39.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.951+0000 7fe597fff640 1 -- 172.21.15.107:0/1378645902 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7fe5b000ece0 con 0x564d9f00c980 2024-04-25T00:24:39.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.951+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/1378645902 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5a00317b0 con 0x564d9f00c980 2024-04-25T00:24:39.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.951+0000 7fe597fff640 1 -- 172.21.15.107:0/1378645902 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe5b002f050 con 0x564d9f00c980 2024-04-25T00:24:39.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.951+0000 7fe5c1dbe740 10 client.17442 fetch_fsmap learned FSMap version 34 2024-04-25T00:24:39.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.951+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/1378645902 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564d9ec763f0 con 0x564d9f00c980 2024-04-25T00:24:39.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.951+0000 7fe597fff640 1 -- 172.21.15.107:0/1378645902 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5a00317b0 con 0x564d9f00c980 2024-04-25T00:24:39.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.951+0000 7fe597fff640 1 -- 172.21.15.107:0/1378645902 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fe5b00273b0 con 0x564d9f00c980 2024-04-25T00:24:39.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.951+0000 7fe5c1dbe740 10 client.17442 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:24:39.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.951+0000 7fe5c1dbe740 10 client.17442 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:24:39.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.951+0000 7fe5c1dbe740 20 client.17442 populate_metadata read hostname 'smithi107' 2024-04-25T00:24:39.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.951+0000 7fe5c1dbe740 1 --2- 172.21.15.107:0/1378645902 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x564d9f01c9f0 0x564d9f01ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.951+0000 7fe5c1dbe740 4 client.17442 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:39.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.951+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/1378645902 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d9ef6fbb0 con 0x564d9f01c9f0 2024-04-25T00:24:39.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.952+0000 7fe5bce4f640 1 --2- 172.21.15.107:0/1378645902 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x564d9f01c9f0 0x564d9f01ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.952+0000 7fe597fff640 10 client.17442 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:24:39.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.952+0000 7fe5bce4f640 1 --2- 172.21.15.107:0/1378645902 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x564d9f01c9f0 0x564d9f01ee80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=abddf519c297a35a server_cookie=9ff59588e8868fd in_seq=0 out_seq=0 2024-04-25T00:24:39.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.953+0000 7fe597fff640 1 -- 172.21.15.107:0/1378645902 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d9ef6fbb0 con 0x564d9f01c9f0 2024-04-25T00:24:39.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.953+0000 7fe597fff640 10 client.17442 handle_command_reply: tid=0 2024-04-25T00:24:39.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.953+0000 7fe5c1dbe740 1 client.17442 shutdown 2024-04-25T00:24:39.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.953+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/1378645902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9f00c980 msgr2=0x564d9f009a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:39.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.953+0000 7fe5c1dbe740 1 --2- 172.21.15.107:0/1378645902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9f00c980 0x564d9f009a20 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe5b00127f0 tx=0x7fe5b0004ab0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.953+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/1378645902 shutdown_connections 2024-04-25T00:24:39.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.953+0000 7fe5c1dbe740 1 --2- 172.21.15.107:0/1378645902 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x564d9f01c9f0 0x564d9f01ee80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.953+0000 7fe5c1dbe740 1 --2- 172.21.15.107:0/1378645902 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9f00c980 0x564d9f009a20 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.953+0000 7fe5c1dbe740 1 --2- 172.21.15.107:0/1378645902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d9effdda0 0x564d9f0094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.953+0000 7fe597fff640 0 client.17442 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:24:39.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.953+0000 7fe5c1dbe740 1 --2- 172.21.15.107:0/1378645902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d9effd480 0x564d9f00c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.953+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/1378645902 >> 172.21.15.107:0/1378645902 conn(0x564d9eff98a0 msgr2=0x564d9eff8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:39.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.953+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/1378645902 shutdown_connections 2024-04-25T00:24:39.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.953+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/1378645902 wait complete. 2024-04-25T00:24:39.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.953+0000 7fe5c1dbe740 20 client.17442 trim_cache size 0 max 16384 2024-04-25T00:24:39.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.966+0000 7fe5c1dbe740 1 Processor -- start 2024-04-25T00:24:39.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.966+0000 7fe5c1dbe740 1 -- start start 2024-04-25T00:24:39.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.966+0000 7fe5c1dbe740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d9effd480 0x564d9f004c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.967+0000 7fe5c1dbe740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d9effdda0 0x564d9f00bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.967+0000 7fe5c1dbe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9f00c980 0x564d9f024750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.967+0000 7fe5c1dbe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b81af210 con 0x564d9f00c980 2024-04-25T00:24:39.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.967+0000 7fe5c1dbe740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b81acae0 con 0x564d9effdda0 2024-04-25T00:24:39.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.967+0000 7fe5c1dbe740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b81aa950 con 0x564d9effd480 2024-04-25T00:24:39.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.967+0000 7fe5bd650640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9f00c980 0x564d9f024750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.967+0000 7fe5bd650640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9f00c980 0x564d9f024750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40560/0 (socket says 172.21.15.107:40560) 2024-04-25T00:24:39.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.967+0000 7fe5bd650640 1 -- 172.21.15.107:0/371781616 learned_addr learned my addr 172.21.15.107:0/371781616 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:39.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.967+0000 7fe5bd650640 1 -- 172.21.15.107:0/371781616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d9effd480 msgr2=0x564d9f004c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:39.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.967+0000 7fe5bd650640 1 --2- 172.21.15.107:0/371781616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d9effd480 0x564d9f004c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.967+0000 7fe5bd650640 1 -- 172.21.15.107:0/371781616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d9effdda0 msgr2=0x564d9f00bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:39.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.967+0000 7fe5bd650640 1 --2- 172.21.15.107:0/371781616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d9effdda0 0x564d9f00bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.967+0000 7fe5bd650640 1 -- 172.21.15.107:0/371781616 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d9ed407d0 con 0x564d9f00c980 2024-04-25T00:24:39.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.967+0000 7fe5bd650640 1 --2- 172.21.15.107:0/371781616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9f00c980 0x564d9f024750 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0006010 tx=0x7fe5b0011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ba0f1de78770e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:39.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.967+0000 7fe5ae7fc640 1 -- 172.21.15.107:0/371781616 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0023070 con 0x564d9f00c980 2024-04-25T00:24:39.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.967+0000 7fe5ae7fc640 1 -- 172.21.15.107:0/371781616 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b0016070 con 0x564d9f00c980 2024-04-25T00:24:39.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.967+0000 7fe5ae7fc640 1 -- 172.21.15.107:0/371781616 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0028440 con 0x564d9f00c980 2024-04-25T00:24:39.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.968+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/371781616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9f00c980 msgr2=0x564d9f024750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:39.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.968+0000 7fe5c1dbe740 1 --2- 172.21.15.107:0/371781616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9f00c980 0x564d9f024750 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0006010 tx=0x7fe5b0011d80 comp rx=0 tx=0).stop 2024-04-25T00:24:39.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.968+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/371781616 shutdown_connections 2024-04-25T00:24:39.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.968+0000 7fe5c1dbe740 1 --2- 172.21.15.107:0/371781616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9f00c980 0x564d9f024750 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.968+0000 7fe5c1dbe740 1 --2- 172.21.15.107:0/371781616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d9effdda0 0x564d9f00bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.968+0000 7fe5c1dbe740 1 --2- 172.21.15.107:0/371781616 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d9effd480 0x564d9f004c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.968+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/371781616 >> 172.21.15.107:0/371781616 conn(0x564d9eff98a0 msgr2=0x564d9effed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:39.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.968+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/371781616 shutdown_connections 2024-04-25T00:24:39.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.968+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/371781616 wait complete. 2024-04-25T00:24:39.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.968+0000 7fe5c1dbe740 1 -- start start 2024-04-25T00:24:39.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.968+0000 7fe5c1dbe740 1 Processor -- start 2024-04-25T00:24:39.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe5c1dbe740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d9effd480 0x564d9f006140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe5c1dbe740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d9effdda0 0x564d9f006680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe5c1dbe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9f00c980 0x564d9f006bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe5c1dbe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b81aafa0 con 0x564d9f00c980 2024-04-25T00:24:39.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe5c1dbe740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b81ac410 con 0x564d9effdda0 2024-04-25T00:24:39.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe5c1dbe740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b81ab670 con 0x564d9effd480 2024-04-25T00:24:39.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe5c1dbe740 10 client.0 _ll_register_callbacks cb 0x7fe5b00124c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:39.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe5bd650640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9f00c980 0x564d9f006bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe5bd650640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9f00c980 0x564d9f006bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40562/0 (socket says 172.21.15.107:40562) 2024-04-25T00:24:39.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe5bd650640 1 -- 172.21.15.107:0/1746152777 learned_addr learned my addr 172.21.15.107:0/1746152777 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:39.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe5bce4f640 1 --2- 172.21.15.107:0/1746152777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d9effd480 0x564d9f006140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe5b7fff640 1 --2- 172.21.15.107:0/1746152777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d9effdda0 0x564d9f006680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe5bd650640 1 -- 172.21.15.107:0/1746152777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d9effd480 msgr2=0x564d9f006140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:39.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe5bd650640 1 --2- 172.21.15.107:0/1746152777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d9effd480 0x564d9f006140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe5bd650640 1 -- 172.21.15.107:0/1746152777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d9effdda0 msgr2=0x564d9f006680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:39.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe5bd650640 1 --2- 172.21.15.107:0/1746152777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d9effdda0 0x564d9f006680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe5bd650640 1 -- 172.21.15.107:0/1746152777 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d9f081fc0 con 0x564d9f00c980 2024-04-25T00:24:39.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe5bd650640 1 --2- 172.21.15.107:0/1746152777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9f00c980 0x564d9f006bc0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0012490 tx=0x7fe5b0027a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbc8dc9950fba8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:39.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe597fff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:24:39.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe597fff640 1 -- 172.21.15.107:0/1746152777 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5b0023040 con 0x564d9f00c980 2024-04-25T00:24:39.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe597fff640 1 -- 172.21.15.107:0/1746152777 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7fe5b000ece0 con 0x564d9f00c980 2024-04-25T00:24:39.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/1746152777 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5b0016070 con 0x564d9f00c980 2024-04-25T00:24:39.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe597fff640 1 -- 172.21.15.107:0/1746152777 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe5b0049050 con 0x564d9f00c980 2024-04-25T00:24:39.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe5c1dbe740 10 client.17448 fetch_fsmap learned FSMap version 34 2024-04-25T00:24:39.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/1746152777 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564d9f0831c0 con 0x564d9f00c980 2024-04-25T00:24:39.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.969+0000 7fe597fff640 1 -- 172.21.15.107:0/1746152777 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5b0016070 con 0x564d9f00c980 2024-04-25T00:24:39.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.970+0000 7fe597fff640 1 -- 172.21.15.107:0/1746152777 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fe5b0028aa0 con 0x564d9f00c980 2024-04-25T00:24:39.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.970+0000 7fe5c1dbe740 10 client.17448 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:24:39.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.970+0000 7fe5c1dbe740 10 client.17448 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:24:39.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.970+0000 7fe5c1dbe740 20 client.17448 populate_metadata read hostname 'smithi107' 2024-04-25T00:24:39.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.970+0000 7fe5c1dbe740 1 --2- 172.21.15.107:0/1746152777 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x564d9f01c9f0 0x564d9f085800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:39.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.970+0000 7fe5c1dbe740 4 client.17448 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:24:39.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.970+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/1746152777 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d9ef69f00 con 0x564d9f01c9f0 2024-04-25T00:24:39.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.970+0000 7fe5bce4f640 1 --2- 172.21.15.107:0/1746152777 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x564d9f01c9f0 0x564d9f085800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:39.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.970+0000 7fe5bce4f640 1 --2- 172.21.15.107:0/1746152777 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x564d9f01c9f0 0x564d9f085800 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9c82f3be40ccb39 server_cookie=fa764052e91ca27 in_seq=0 out_seq=0 2024-04-25T00:24:39.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.970+0000 7fe597fff640 10 client.17448 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:24:39.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.970+0000 7fe597fff640 1 -- 172.21.15.107:0/1746152777 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d9ef69f00 con 0x564d9f01c9f0 2024-04-25T00:24:39.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.970+0000 7fe597fff640 10 client.17448 handle_command_reply: tid=0 2024-04-25T00:24:39.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.970+0000 7fe5c1dbe740 1 client.17448 shutdown 2024-04-25T00:24:39.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.971+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/1746152777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9f00c980 msgr2=0x564d9f006bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:39.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.971+0000 7fe5c1dbe740 1 --2- 172.21.15.107:0/1746152777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9f00c980 0x564d9f006bc0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0012490 tx=0x7fe5b0027a60 comp rx=0 tx=0).stop 2024-04-25T00:24:39.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.971+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/1746152777 shutdown_connections 2024-04-25T00:24:39.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.971+0000 7fe5c1dbe740 1 --2- 172.21.15.107:0/1746152777 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x564d9f01c9f0 0x564d9f085800 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.971+0000 7fe5c1dbe740 1 --2- 172.21.15.107:0/1746152777 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d9f00c980 0x564d9f006bc0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.971+0000 7fe5c1dbe740 1 --2- 172.21.15.107:0/1746152777 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564d9effdda0 0x564d9f006680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.971+0000 7fe597fff640 0 client.17448 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:24:39.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.971+0000 7fe5c1dbe740 1 --2- 172.21.15.107:0/1746152777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564d9effd480 0x564d9f006140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.971+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/1746152777 >> 172.21.15.107:0/1746152777 conn(0x564d9eff98a0 msgr2=0x564d9efff600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:39.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.971+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/1746152777 shutdown_connections 2024-04-25T00:24:39.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.971+0000 7fe5c1dbe740 1 -- 172.21.15.107:0/1746152777 wait complete. 2024-04-25T00:24:39.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.971+0000 7fe5c1dbe740 20 client.17448 trim_cache size 0 max 16384 2024-04-25T00:24:39.978 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:24:39.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.971+0000 7fe5bf0b3640 1 -- 172.21.15.107:0/2934627768 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe5a40a2d00 msgr2=0x7fe5a40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:39.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.971+0000 7fe5bf0b3640 1 --2- 172.21.15.107:0/2934627768 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe5a40a2d00 0x7fe5a40a51c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009270 tx=0x7fe5b0009880 comp rx=0 tx=0).stop 2024-04-25T00:24:39.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.971+0000 7fe5bf0b3640 1 -- 172.21.15.107:0/2934627768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5b8100610 msgr2=0x7fe5b8198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:39.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.971+0000 7fe5bf0b3640 1 --2- 172.21.15.107:0/2934627768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5b8100610 0x7fe5b8198e30 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800d950 tx=0x7fe5a800de10 comp rx=0 tx=0).stop 2024-04-25T00:24:39.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.971+0000 7fe5bf0b3640 1 -- 172.21.15.107:0/2934627768 shutdown_connections 2024-04-25T00:24:39.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.971+0000 7fe5bf0b3640 1 --2- 172.21.15.107:0/2934627768 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe5a40a2d00 0x7fe5a40a51c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.971+0000 7fe5bf0b3640 1 --2- 172.21.15.107:0/2934627768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5b81066b0 0x7fe5b81a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.971+0000 7fe5bf0b3640 1 --2- 172.21.15.107:0/2934627768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5b8100fd0 0x7fe5b8199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.971+0000 7fe5bf0b3640 1 --2- 172.21.15.107:0/2934627768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5b8100610 0x7fe5b8198e30 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:39.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.971+0000 7fe5bf0b3640 1 -- 172.21.15.107:0/2934627768 >> 172.21.15.107:0/2934627768 conn(0x7fe5b8077740 msgr2=0x7fe5b80ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:39.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.971+0000 7fe5bf0b3640 1 -- 172.21.15.107:0/2934627768 shutdown_connections 2024-04-25T00:24:39.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:39.971+0000 7fe5bf0b3640 1 -- 172.21.15.107:0/2934627768 wait complete. 2024-04-25T00:24:39.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:39 smithi107 ceph-mon[29153]: pgmap v253: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:24:39.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:39 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:39.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:39 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:39.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:39 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:24:39.982 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:24:40.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.081+0000 7f8486c45640 1 -- 172.21.15.107:0/847045035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f848006a7c0 msgr2=0x7f848010e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:40.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.081+0000 7f8486c45640 1 --2- 172.21.15.107:0/847045035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f848006a7c0 0x7f848010e5f0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f8478009cd0 tx=0x7f847802a8e0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.081+0000 7f8486c45640 1 -- 172.21.15.107:0/847045035 shutdown_connections 2024-04-25T00:24:40.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.081+0000 7f8486c45640 1 --2- 172.21.15.107:0/847045035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f848006a7c0 0x7f848010e5f0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.081+0000 7f8486c45640 1 --2- 172.21.15.107:0/847045035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8480069e00 0x7f848006a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.081+0000 7f8486c45640 1 --2- 172.21.15.107:0/847045035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8480107730 0x7f8480107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.081+0000 7f8486c45640 1 -- 172.21.15.107:0/847045035 >> 172.21.15.107:0/847045035 conn(0x7f84800776a0 msgr2=0x7f8480077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:40.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.081+0000 7f8486c45640 1 -- 172.21.15.107:0/847045035 shutdown_connections 2024-04-25T00:24:40.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.081+0000 7f8486c45640 1 -- 172.21.15.107:0/847045035 wait complete. 2024-04-25T00:24:40.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.082+0000 7f8486c45640 1 Processor -- start 2024-04-25T00:24:40.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.082+0000 7f8486c45640 1 -- start start 2024-04-25T00:24:40.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.082+0000 7f8486c45640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8480069e00 0x7f848019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.082+0000 7f8486c45640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f848006a7c0 0x7f848019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.082+0000 7f8486c45640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8480107730 0x7f848019eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.082+0000 7f8486c45640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8480102140 con 0x7f848006a7c0 2024-04-25T00:24:40.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.082+0000 7f8486c45640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8480101fc0 con 0x7f8480069e00 2024-04-25T00:24:40.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.082+0000 7f8486c45640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f84801022c0 con 0x7f8480107730 2024-04-25T00:24:40.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.082+0000 7f847ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f848006a7c0 0x7f848019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.082+0000 7f847ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f848006a7c0 0x7f848019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40582/0 (socket says 172.21.15.107:40582) 2024-04-25T00:24:40.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.082+0000 7f847ffff640 1 -- 172.21.15.107:0/2734218394 learned_addr learned my addr 172.21.15.107:0/2734218394 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:40.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.082+0000 7f84849e1640 1 --2- 172.21.15.107:0/2734218394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8480069e00 0x7f848019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.082+0000 7f84851e2640 1 --2- 172.21.15.107:0/2734218394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8480107730 0x7f848019eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.083+0000 7f847ffff640 1 -- 172.21.15.107:0/2734218394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8480107730 msgr2=0x7f848019eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:40.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.083+0000 7f847ffff640 1 --2- 172.21.15.107:0/2734218394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8480107730 0x7f848019eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.083+0000 7f847ffff640 1 -- 172.21.15.107:0/2734218394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8480069e00 msgr2=0x7f848019a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:40.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.083+0000 7f847ffff640 1 --2- 172.21.15.107:0/2734218394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8480069e00 0x7f848019a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.083+0000 7f847ffff640 1 -- 172.21.15.107:0/2734218394 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848019f1f0 con 0x7f848006a7c0 2024-04-25T00:24:40.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.083+0000 7f84849e1640 1 --2- 172.21.15.107:0/2734218394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8480069e00 0x7f848019a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:40.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.083+0000 7f847ffff640 1 --2- 172.21.15.107:0/2734218394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f848006a7c0 0x7f848019a740 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f846c00d770 tx=0x7f846c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:40.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.083+0000 7f847dffb640 1 -- 172.21.15.107:0/2734218394 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c017070 con 0x7f848006a7c0 2024-04-25T00:24:40.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.083+0000 7f847dffb640 1 -- 172.21.15.107:0/2734218394 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f846c005500 con 0x7f848006a7c0 2024-04-25T00:24:40.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.083+0000 7f8486c45640 1 -- 172.21.15.107:0/2734218394 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f848019f4e0 con 0x7f848006a7c0 2024-04-25T00:24:40.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.083+0000 7f847dffb640 1 -- 172.21.15.107:0/2734218394 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f846c004e00 con 0x7f848006a7c0 2024-04-25T00:24:40.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.083+0000 7f8486c45640 1 -- 172.21.15.107:0/2734218394 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84801a6d70 con 0x7f848006a7c0 2024-04-25T00:24:40.093 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:39 smithi195 ceph-mon[32833]: pgmap v253: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:24:40.093 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:39 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:40.093 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:39 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:40.093 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:39 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:24:40.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.085+0000 7f847dffb640 1 -- 172.21.15.107:0/2734218394 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f846c004fa0 con 0x7f848006a7c0 2024-04-25T00:24:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.085+0000 7f8489a2a740 1 Processor -- start 2024-04-25T00:24:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.085+0000 7f8489a2a740 1 -- start start 2024-04-25T00:24:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.086+0000 7f847dffb640 1 --2- 172.21.15.107:0/2734218394 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f84700a2d00 0x7f84700a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.086+0000 7f8489a2a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ece580 0x55f034ece960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.086+0000 7f8489a2a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f034eceea0 0x55f034ed9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.086+0000 7f8489a2a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f034ed9570 0x55f034edb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.086+0000 7f8489a2a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f034e33670 con 0x55f034ece580 2024-04-25T00:24:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.086+0000 7f8489a2a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f034ddb970 con 0x55f034eceea0 2024-04-25T00:24:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.086+0000 7f8489a2a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f034e2f630 con 0x55f034ed9570 2024-04-25T00:24:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.086+0000 7f84851e2640 1 --2- 172.21.15.107:0/2734218394 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f84700a2d00 0x7f84700a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.086+0000 7f84849e1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ece580 0x55f034ece960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.086+0000 7f847dffb640 1 -- 172.21.15.107:0/2734218394 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f846c0cf340 con 0x7f848006a7c0 2024-04-25T00:24:40.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.086+0000 7f84849e1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ece580 0x55f034ece960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40588/0 (socket says 172.21.15.107:40588) 2024-04-25T00:24:40.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.086+0000 7f84849e1640 1 -- 172.21.15.107:0/1757909995 learned_addr learned my addr 172.21.15.107:0/1757909995 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:40.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.086+0000 7f847ffff640 1 --2- 172.21.15.107:0/1757909995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f034eceea0 0x55f034ed9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.086+0000 7f84851e2640 1 --2- 172.21.15.107:0/1757909995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f034ed9570 0x55f034edb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.086+0000 7f84849e1640 1 -- 172.21.15.107:0/1757909995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f034ed9570 msgr2=0x55f034edb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:40.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.086+0000 7f84849e1640 1 --2- 172.21.15.107:0/1757909995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f034ed9570 0x55f034edb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.086+0000 7f84849e1640 1 -- 172.21.15.107:0/1757909995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f034eceea0 msgr2=0x55f034ed9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:40.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.086+0000 7f84849e1640 1 --2- 172.21.15.107:0/1757909995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f034eceea0 0x55f034ed9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.086+0000 7f84849e1640 1 -- 172.21.15.107:0/1757909995 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f034dff400 con 0x55f034ece580 2024-04-25T00:24:40.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.086+0000 7f84851e2640 1 --2- 172.21.15.107:0/2734218394 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f84700a2d00 0x7f84700a51c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8478005cc0 tx=0x7f8478031630 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:40.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.086+0000 7f84849e1640 1 --2- 172.21.15.107:0/1757909995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ece580 0x55f034ece960 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f847400dc40 tx=0x7f847400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8db0b66c65db97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:40.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.086+0000 7f846affd640 1 -- 172.21.15.107:0/1757909995 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8474011440 con 0x55f034ece580 2024-04-25T00:24:40.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.086+0000 7f846affd640 1 -- 172.21.15.107:0/1757909995 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8474011ac0 con 0x55f034ece580 2024-04-25T00:24:40.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.086+0000 7f846affd640 1 -- 172.21.15.107:0/1757909995 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847401ba70 con 0x55f034ece580 2024-04-25T00:24:40.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.087+0000 7f8489a2a740 1 -- 172.21.15.107:0/1757909995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ece580 msgr2=0x55f034ece960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:40.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.087+0000 7f8489a2a740 1 --2- 172.21.15.107:0/1757909995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ece580 0x55f034ece960 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f847400dc40 tx=0x7f847400f730 comp rx=0 tx=0).stop 2024-04-25T00:24:40.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.087+0000 7f8489a2a740 1 -- 172.21.15.107:0/1757909995 shutdown_connections 2024-04-25T00:24:40.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.087+0000 7f8489a2a740 1 --2- 172.21.15.107:0/1757909995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f034ed9570 0x55f034edb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.087+0000 7f8489a2a740 1 --2- 172.21.15.107:0/1757909995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f034eceea0 0x55f034ed9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.087+0000 7f8489a2a740 1 --2- 172.21.15.107:0/1757909995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ece580 0x55f034ece960 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.087+0000 7f8489a2a740 1 -- 172.21.15.107:0/1757909995 >> 172.21.15.107:0/1757909995 conn(0x55f034eca8a0 msgr2=0x55f034ecac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:40.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.087+0000 7f8489a2a740 1 -- 172.21.15.107:0/1757909995 shutdown_connections 2024-04-25T00:24:40.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.087+0000 7f8489a2a740 1 -- 172.21.15.107:0/1757909995 wait complete. 2024-04-25T00:24:40.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.087+0000 7f8489a2a740 1 -- start start 2024-04-25T00:24:40.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.088+0000 7f8489a2a740 1 Processor -- start 2024-04-25T00:24:40.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.088+0000 7f8489a2a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ece580 0x55f034ed8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.088+0000 7f8489a2a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f034eceea0 0x55f034ed61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.088+0000 7f8489a2a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f034ed9570 0x55f034ed6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.088+0000 7f8489a2a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f034edc360 con 0x55f034ece580 2024-04-25T00:24:40.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.088+0000 7f8489a2a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f034def050 con 0x55f034eceea0 2024-04-25T00:24:40.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.088+0000 7f8489a2a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f034edc4e0 con 0x55f034ed9570 2024-04-25T00:24:40.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.088+0000 7f8489a2a740 10 client.0 _ll_register_callbacks cb 0x55f034b46be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:40.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.088+0000 7f84849e1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ece580 0x55f034ed8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.088+0000 7f84849e1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ece580 0x55f034ed8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40604/0 (socket says 172.21.15.107:40604) 2024-04-25T00:24:40.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.088+0000 7f84849e1640 1 -- 172.21.15.107:0/3925094206 learned_addr learned my addr 172.21.15.107:0/3925094206 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:40.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.088+0000 7f84851e2640 1 --2- 172.21.15.107:0/3925094206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f034ed9570 0x55f034ed6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.088+0000 7f847ffff640 1 --2- 172.21.15.107:0/3925094206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f034eceea0 0x55f034ed61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.088+0000 7f84849e1640 1 -- 172.21.15.107:0/3925094206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f034ed9570 msgr2=0x55f034ed6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:40.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.088+0000 7f84849e1640 1 --2- 172.21.15.107:0/3925094206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f034ed9570 0x55f034ed6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.088+0000 7f84849e1640 1 -- 172.21.15.107:0/3925094206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f034eceea0 msgr2=0x55f034ed61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:40.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.088+0000 7f84849e1640 1 --2- 172.21.15.107:0/3925094206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f034eceea0 0x55f034ed61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.088+0000 7f84849e1640 1 -- 172.21.15.107:0/3925094206 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f034dc0dc0 con 0x55f034ece580 2024-04-25T00:24:40.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.088+0000 7f84849e1640 1 --2- 172.21.15.107:0/3925094206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ece580 0x55f034ed8ee0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f847401b690 tx=0x7f847400df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce1073cb7c05ff2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:40.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.088+0000 7f845ffff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:24:40.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.089+0000 7f845ffff640 1 -- 172.21.15.107:0/3925094206 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8474029060 con 0x55f034ece580 2024-04-25T00:24:40.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.089+0000 7f845ffff640 1 -- 172.21.15.107:0/3925094206 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f8474011650 con 0x55f034ece580 2024-04-25T00:24:40.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.089+0000 7f8489a2a740 1 -- 172.21.15.107:0/3925094206 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8474011ac0 con 0x55f034ece580 2024-04-25T00:24:40.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.089+0000 7f845ffff640 1 -- 172.21.15.107:0/3925094206 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8474030050 con 0x55f034ece580 2024-04-25T00:24:40.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.089+0000 7f8489a2a740 10 client.17460 fetch_fsmap learned FSMap version 34 2024-04-25T00:24:40.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.089+0000 7f8489a2a740 1 -- 172.21.15.107:0/3925094206 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f034b473f0 con 0x55f034ece580 2024-04-25T00:24:40.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.089+0000 7f845ffff640 1 -- 172.21.15.107:0/3925094206 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8474011ac0 con 0x55f034ece580 2024-04-25T00:24:40.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.089+0000 7f845ffff640 1 -- 172.21.15.107:0/3925094206 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f847401bbb0 con 0x55f034ece580 2024-04-25T00:24:40.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.089+0000 7f8489a2a740 10 client.17460 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:24:40.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.089+0000 7f8489a2a740 10 client.17460 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:24:40.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.089+0000 7f8489a2a740 20 client.17460 populate_metadata read hostname 'smithi107' 2024-04-25T00:24:40.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.089+0000 7f8489a2a740 1 --2- 172.21.15.107:0/3925094206 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55f034ee9560 0x55f034eeb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.089+0000 7f8489a2a740 4 client.17460 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:40.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.089+0000 7f8489a2a740 1 -- 172.21.15.107:0/3925094206 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f034e40bb0 con 0x55f034ee9560 2024-04-25T00:24:40.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.089+0000 7f847ffff640 1 --2- 172.21.15.107:0/3925094206 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55f034ee9560 0x55f034eeb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.090+0000 7f847ffff640 1 --2- 172.21.15.107:0/3925094206 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55f034ee9560 0x55f034eeb950 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b8ad1577f42b4e2 server_cookie=d2be7c29e49ccacd in_seq=0 out_seq=0 2024-04-25T00:24:40.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.090+0000 7f845ffff640 10 client.17460 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:24:40.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.091+0000 7f845ffff640 1 -- 172.21.15.107:0/3925094206 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f034e40bb0 con 0x55f034ee9560 2024-04-25T00:24:40.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.091+0000 7f845ffff640 10 client.17460 handle_command_reply: tid=0 2024-04-25T00:24:40.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.091+0000 7f8489a2a740 1 client.17460 shutdown 2024-04-25T00:24:40.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.091+0000 7f8489a2a740 1 -- 172.21.15.107:0/3925094206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ece580 msgr2=0x55f034ed8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:40.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.091+0000 7f8489a2a740 1 --2- 172.21.15.107:0/3925094206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ece580 0x55f034ed8ee0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f847401b690 tx=0x7f847400df60 comp rx=0 tx=0).stop 2024-04-25T00:24:40.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.091+0000 7f8489a2a740 1 -- 172.21.15.107:0/3925094206 shutdown_connections 2024-04-25T00:24:40.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.091+0000 7f8489a2a740 1 --2- 172.21.15.107:0/3925094206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f034ed9570 0x55f034ed6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.091+0000 7f8489a2a740 1 --2- 172.21.15.107:0/3925094206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f034eceea0 0x55f034ed61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.091+0000 7f8489a2a740 1 --2- 172.21.15.107:0/3925094206 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55f034ee9560 0x55f034eeb950 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.091+0000 7f8489a2a740 1 --2- 172.21.15.107:0/3925094206 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ece580 0x55f034ed8ee0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.091+0000 7f845ffff640 0 client.17460 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:24:40.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.091+0000 7f8489a2a740 1 -- 172.21.15.107:0/3925094206 >> 172.21.15.107:0/3925094206 conn(0x55f034eca8a0 msgr2=0x55f034ec9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:40.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.091+0000 7f8489a2a740 1 -- 172.21.15.107:0/3925094206 shutdown_connections 2024-04-25T00:24:40.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.091+0000 7f8489a2a740 1 -- 172.21.15.107:0/3925094206 wait complete. 2024-04-25T00:24:40.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.091+0000 7f8489a2a740 20 client.17460 trim_cache size 0 max 16384 2024-04-25T00:24:40.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.104+0000 7f8489a2a740 1 Processor -- start 2024-04-25T00:24:40.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.104+0000 7f8489a2a740 1 -- start start 2024-04-25T00:24:40.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.104+0000 7f8489a2a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f034ece580 0x55f034ee5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.104+0000 7f8489a2a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f034eceea0 0x55f034ed8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.104+0000 7f8489a2a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ed9570 0x55f034ef0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.104+0000 7f8489a2a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f84801b3690 con 0x55f034ed9570 2024-04-25T00:24:40.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.104+0000 7f8489a2a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f84801b0f60 con 0x55f034ece580 2024-04-25T00:24:40.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.104+0000 7f8489a2a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f84801aedd0 con 0x55f034eceea0 2024-04-25T00:24:40.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.104+0000 7f84851e2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ed9570 0x55f034ef0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.104+0000 7f84851e2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ed9570 0x55f034ef0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40616/0 (socket says 172.21.15.107:40616) 2024-04-25T00:24:40.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.104+0000 7f84851e2640 1 -- 172.21.15.107:0/1844322480 learned_addr learned my addr 172.21.15.107:0/1844322480 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:40.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.105+0000 7f847ffff640 1 --2- 172.21.15.107:0/1844322480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f034eceea0 0x55f034ed8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.105+0000 7f84849e1640 1 --2- 172.21.15.107:0/1844322480 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f034ece580 0x55f034ee5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.105+0000 7f84851e2640 1 -- 172.21.15.107:0/1844322480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f034eceea0 msgr2=0x55f034ed8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:40.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.105+0000 7f84851e2640 1 --2- 172.21.15.107:0/1844322480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f034eceea0 0x55f034ed8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.105+0000 7f84851e2640 1 -- 172.21.15.107:0/1844322480 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f034ece580 msgr2=0x55f034ee5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:40.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.105+0000 7f84851e2640 1 --2- 172.21.15.107:0/1844322480 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f034ece580 0x55f034ee5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.105+0000 7f84851e2640 1 -- 172.21.15.107:0/1844322480 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f034c117d0 con 0x55f034ed9570 2024-04-25T00:24:40.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.105+0000 7f84849e1640 1 --2- 172.21.15.107:0/1844322480 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f034ece580 0x55f034ee5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:40.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.105+0000 7f84851e2640 1 --2- 172.21.15.107:0/1844322480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ed9570 0x55f034ef0f20 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f8478009cd0 tx=0x7f8478005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=243e19e8c1361f2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:40.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.105+0000 7f846a7fc640 1 -- 172.21.15.107:0/1844322480 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847804d020 con 0x55f034ed9570 2024-04-25T00:24:40.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.105+0000 7f846a7fc640 1 -- 172.21.15.107:0/1844322480 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8478004590 con 0x55f034ed9570 2024-04-25T00:24:40.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.105+0000 7f846a7fc640 1 -- 172.21.15.107:0/1844322480 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478049420 con 0x55f034ed9570 2024-04-25T00:24:40.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.105+0000 7f8489a2a740 1 -- 172.21.15.107:0/1844322480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ed9570 msgr2=0x55f034ef0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:40.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.105+0000 7f8489a2a740 1 --2- 172.21.15.107:0/1844322480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ed9570 0x55f034ef0f20 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f8478009cd0 tx=0x7f8478005c50 comp rx=0 tx=0).stop 2024-04-25T00:24:40.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.106+0000 7f8489a2a740 1 -- 172.21.15.107:0/1844322480 shutdown_connections 2024-04-25T00:24:40.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.106+0000 7f8489a2a740 1 --2- 172.21.15.107:0/1844322480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ed9570 0x55f034ef0f20 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.106+0000 7f8489a2a740 1 --2- 172.21.15.107:0/1844322480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f034eceea0 0x55f034ed8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.106+0000 7f8489a2a740 1 --2- 172.21.15.107:0/1844322480 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f034ece580 0x55f034ee5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.106+0000 7f8489a2a740 1 -- 172.21.15.107:0/1844322480 >> 172.21.15.107:0/1844322480 conn(0x55f034eca8a0 msgr2=0x55f034ed1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:40.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.106+0000 7f8489a2a740 1 -- 172.21.15.107:0/1844322480 shutdown_connections 2024-04-25T00:24:40.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.106+0000 7f8489a2a740 1 -- 172.21.15.107:0/1844322480 wait complete. 2024-04-25T00:24:40.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.106+0000 7f8489a2a740 1 -- start start 2024-04-25T00:24:40.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.106+0000 7f8489a2a740 1 Processor -- start 2024-04-25T00:24:40.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.106+0000 7f8489a2a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ece580 0x55f034ee1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.106+0000 7f8489a2a740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f034eceea0 0x55f034edce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.106+0000 7f8489a2a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f034ed9570 0x55f034edd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.106+0000 7f8489a2a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f84801af420 con 0x55f034ece580 2024-04-25T00:24:40.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.106+0000 7f8489a2a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f84801b0890 con 0x55f034ed9570 2024-04-25T00:24:40.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.106+0000 7f8489a2a740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f84801afaf0 con 0x55f034eceea0 2024-04-25T00:24:40.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.106+0000 7f8489a2a740 10 client.0 _ll_register_callbacks cb 0x7f8474015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:40.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.106+0000 7f84849e1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ece580 0x55f034ee1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.106+0000 7f84849e1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ece580 0x55f034ee1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40624/0 (socket says 172.21.15.107:40624) 2024-04-25T00:24:40.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.106+0000 7f84849e1640 1 -- 172.21.15.107:0/2573352841 learned_addr learned my addr 172.21.15.107:0/2573352841 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:40.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.107+0000 7f847ffff640 1 --2- 172.21.15.107:0/2573352841 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f034eceea0 0x55f034edce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.107+0000 7f84849e1640 1 -- 172.21.15.107:0/2573352841 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f034eceea0 msgr2=0x55f034edce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:40.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.107+0000 7f84849e1640 1 --2- 172.21.15.107:0/2573352841 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f034eceea0 0x55f034edce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.107+0000 7f84849e1640 1 -- 172.21.15.107:0/2573352841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f034ed9570 msgr2=0x55f034edd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:40.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.107+0000 7f84849e1640 1 --2- 172.21.15.107:0/2573352841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f034ed9570 0x55f034edd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.107+0000 7f84849e1640 1 -- 172.21.15.107:0/2573352841 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f034eddab0 con 0x55f034ece580 2024-04-25T00:24:40.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.107+0000 7f847ffff640 1 --2- 172.21.15.107:0/2573352841 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f034eceea0 0x55f034edce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:40.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.107+0000 7f84849e1640 1 --2- 172.21.15.107:0/2573352841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ece580 0x55f034ee1a20 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f8474006010 tx=0x7f847400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17baa3015d8851e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:40.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.107+0000 7f845ffff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:24:40.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.107+0000 7f845ffff640 1 -- 172.21.15.107:0/2573352841 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8474029030 con 0x55f034ece580 2024-04-25T00:24:40.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.107+0000 7f845ffff640 1 -- 172.21.15.107:0/2573352841 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f8474011650 con 0x55f034ece580 2024-04-25T00:24:40.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.107+0000 7f8489a2a740 1 -- 172.21.15.107:0/2573352841 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8478004590 con 0x55f034ece580 2024-04-25T00:24:40.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.107+0000 7f845ffff640 1 -- 172.21.15.107:0/2573352841 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f847403d050 con 0x55f034ece580 2024-04-25T00:24:40.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.107+0000 7f8489a2a740 10 client.17466 fetch_fsmap learned FSMap version 34 2024-04-25T00:24:40.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.107+0000 7f8489a2a740 1 -- 172.21.15.107:0/2573352841 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f034f4f630 con 0x55f034ece580 2024-04-25T00:24:40.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.107+0000 7f845ffff640 1 -- 172.21.15.107:0/2573352841 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8478004590 con 0x55f034ece580 2024-04-25T00:24:40.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.107+0000 7f845ffff640 1 -- 172.21.15.107:0/2573352841 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f847401b6c0 con 0x55f034ece580 2024-04-25T00:24:40.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.108+0000 7f8489a2a740 10 client.17466 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:24:40.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.108+0000 7f8489a2a740 10 client.17466 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:24:40.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.108+0000 7f8489a2a740 20 client.17466 populate_metadata read hostname 'smithi107' 2024-04-25T00:24:40.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.108+0000 7f8489a2a740 1 --2- 172.21.15.107:0/2573352841 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55f034f4fdb0 0x55f034f52210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.108+0000 7f8489a2a740 4 client.17466 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:24:40.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.108+0000 7f8489a2a740 1 -- 172.21.15.107:0/2573352841 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f034e3af00 con 0x55f034f4fdb0 2024-04-25T00:24:40.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.108+0000 7f847ffff640 1 --2- 172.21.15.107:0/2573352841 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55f034f4fdb0 0x55f034f52210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.108+0000 7f847ffff640 1 --2- 172.21.15.107:0/2573352841 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55f034f4fdb0 0x55f034f52210 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dae8e9a2f6e56327 server_cookie=e3bd1ddf1af77b33 in_seq=0 out_seq=0 2024-04-25T00:24:40.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.108+0000 7f845ffff640 10 client.17466 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:24:40.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.108+0000 7f845ffff640 1 -- 172.21.15.107:0/2573352841 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f034e3af00 con 0x55f034f4fdb0 2024-04-25T00:24:40.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.108+0000 7f845ffff640 10 client.17466 handle_command_reply: tid=0 2024-04-25T00:24:40.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.109+0000 7f8489a2a740 1 client.17466 shutdown 2024-04-25T00:24:40.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.109+0000 7f8489a2a740 1 -- 172.21.15.107:0/2573352841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ece580 msgr2=0x55f034ee1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:40.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.109+0000 7f8489a2a740 1 --2- 172.21.15.107:0/2573352841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ece580 0x55f034ee1a20 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f8474006010 tx=0x7f847400dac0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.109+0000 7f8489a2a740 1 -- 172.21.15.107:0/2573352841 shutdown_connections 2024-04-25T00:24:40.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.109+0000 7f8489a2a740 1 --2- 172.21.15.107:0/2573352841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f034ed9570 0x55f034edd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.109+0000 7f8489a2a740 1 --2- 172.21.15.107:0/2573352841 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f034eceea0 0x55f034edce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.109+0000 7f8489a2a740 1 --2- 172.21.15.107:0/2573352841 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55f034f4fdb0 0x55f034f52210 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.109+0000 7f8489a2a740 1 --2- 172.21.15.107:0/2573352841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f034ece580 0x55f034ee1a20 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.109+0000 7f8489a2a740 1 -- 172.21.15.107:0/2573352841 >> 172.21.15.107:0/2573352841 conn(0x55f034eca8a0 msgr2=0x55f034ee4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:40.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.109+0000 7f845ffff640 0 client.17466 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:24:40.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.109+0000 7f8489a2a740 1 -- 172.21.15.107:0/2573352841 shutdown_connections 2024-04-25T00:24:40.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.109+0000 7f8489a2a740 1 -- 172.21.15.107:0/2573352841 wait complete. 2024-04-25T00:24:40.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.109+0000 7f8489a2a740 20 client.17466 trim_cache size 0 max 16384 2024-04-25T00:24:40.117 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:24:40.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.109+0000 7f8486c45640 1 -- 172.21.15.107:0/2734218394 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f84700a2d00 msgr2=0x7f84700a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:40.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.109+0000 7f8486c45640 1 --2- 172.21.15.107:0/2734218394 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f84700a2d00 0x7f84700a51c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8478005cc0 tx=0x7f8478031630 comp rx=0 tx=0).stop 2024-04-25T00:24:40.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.109+0000 7f8486c45640 1 -- 172.21.15.107:0/2734218394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f848006a7c0 msgr2=0x7f848019a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:40.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.109+0000 7f8486c45640 1 --2- 172.21.15.107:0/2734218394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f848006a7c0 0x7f848019a740 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f846c00d770 tx=0x7f846c00dc30 comp rx=0 tx=0).stop 2024-04-25T00:24:40.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.109+0000 7f8486c45640 1 -- 172.21.15.107:0/2734218394 shutdown_connections 2024-04-25T00:24:40.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.109+0000 7f8486c45640 1 --2- 172.21.15.107:0/2734218394 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8480107730 0x7f848019eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.109+0000 7f8486c45640 1 --2- 172.21.15.107:0/2734218394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f848006a7c0 0x7f848019a740 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.109+0000 7f8486c45640 1 --2- 172.21.15.107:0/2734218394 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f84700a2d00 0x7f84700a51c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.109+0000 7f8486c45640 1 --2- 172.21.15.107:0/2734218394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8480069e00 0x7f848019a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.109+0000 7f8486c45640 1 -- 172.21.15.107:0/2734218394 >> 172.21.15.107:0/2734218394 conn(0x7f84800776a0 msgr2=0x7f84800fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:40.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.110+0000 7f8486c45640 1 -- 172.21.15.107:0/2734218394 shutdown_connections 2024-04-25T00:24:40.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.110+0000 7f8486c45640 1 -- 172.21.15.107:0/2734218394 wait complete. 2024-04-25T00:24:40.120 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:24:40.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.221+0000 7f90ec815640 1 -- 172.21.15.107:0/914333092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90e8101710 msgr2=0x7f90e8101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:40.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.221+0000 7f90ec815640 1 --2- 172.21.15.107:0/914333092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90e8101710 0x7f90e8101b90 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009930 tx=0x7f90dc02a420 comp rx=0 tx=0).stop 2024-04-25T00:24:40.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.221+0000 7f90ec815640 1 -- 172.21.15.107:0/914333092 shutdown_connections 2024-04-25T00:24:40.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.221+0000 7f90ec815640 1 --2- 172.21.15.107:0/914333092 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90e81020d0 0x7f90e810e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.221+0000 7f90ec815640 1 --2- 172.21.15.107:0/914333092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90e8101710 0x7f90e8101b90 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.221+0000 7f90ec815640 1 --2- 172.21.15.107:0/914333092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90e8107720 0x7f90e8107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.221+0000 7f90ec815640 1 -- 172.21.15.107:0/914333092 >> 172.21.15.107:0/914333092 conn(0x7f90e80fd180 msgr2=0x7f90e80ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:40.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.221+0000 7f90ec815640 1 -- 172.21.15.107:0/914333092 shutdown_connections 2024-04-25T00:24:40.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.221+0000 7f90ec815640 1 -- 172.21.15.107:0/914333092 wait complete. 2024-04-25T00:24:40.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.222+0000 7f90ec815640 1 Processor -- start 2024-04-25T00:24:40.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.222+0000 7f90ec815640 1 -- start start 2024-04-25T00:24:40.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.222+0000 7f90ec815640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90e8101710 0x7f90e81980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.222+0000 7f90ec815640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90e81020d0 0x7f90e8198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.222+0000 7f90ec815640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90e8107720 0x7f90e819c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.222+0000 7f90ec815640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f90e8111b30 con 0x7f90e8107720 2024-04-25T00:24:40.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.222+0000 7f90ec815640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f90e81119b0 con 0x7f90e8101710 2024-04-25T00:24:40.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.222+0000 7f90ec815640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f90e8111cb0 con 0x7f90e81020d0 2024-04-25T00:24:40.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.222+0000 7f90e6d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90e8107720 0x7f90e819c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.222+0000 7f90e6d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90e8107720 0x7f90e819c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40644/0 (socket says 172.21.15.107:40644) 2024-04-25T00:24:40.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.222+0000 7f90e6d9d640 1 -- 172.21.15.107:0/135356576 learned_addr learned my addr 172.21.15.107:0/135356576 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:40.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.222+0000 7f90e659c640 1 --2- 172.21.15.107:0/135356576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90e8101710 0x7f90e81980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.222+0000 7f90e5d9b640 1 --2- 172.21.15.107:0/135356576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90e81020d0 0x7f90e8198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.222+0000 7f90e6d9d640 1 -- 172.21.15.107:0/135356576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90e81020d0 msgr2=0x7f90e8198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:40.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.222+0000 7f90e6d9d640 1 --2- 172.21.15.107:0/135356576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90e81020d0 0x7f90e8198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.222+0000 7f90e6d9d640 1 -- 172.21.15.107:0/135356576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90e8101710 msgr2=0x7f90e81980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:40.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.222+0000 7f90e6d9d640 1 --2- 172.21.15.107:0/135356576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90e8101710 0x7f90e81980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.222+0000 7f90e6d9d640 1 -- 172.21.15.107:0/135356576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e819d0c0 con 0x7f90e8107720 2024-04-25T00:24:40.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.222+0000 7f90e6d9d640 1 --2- 172.21.15.107:0/135356576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90e8107720 0x7f90e819c990 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f90e000eaa0 tx=0x7f90e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:40.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.223+0000 7f90d77fe640 1 -- 172.21.15.107:0/135356576 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e000cdb0 con 0x7f90e8107720 2024-04-25T00:24:40.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.223+0000 7f90ec815640 1 -- 172.21.15.107:0/135356576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e819d3b0 con 0x7f90e8107720 2024-04-25T00:24:40.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.223+0000 7f90d77fe640 1 -- 172.21.15.107:0/135356576 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90e0005020 con 0x7f90e8107720 2024-04-25T00:24:40.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.223+0000 7f90d77fe640 1 -- 172.21.15.107:0/135356576 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e00116d0 con 0x7f90e8107720 2024-04-25T00:24:40.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.223+0000 7f90ec815640 1 -- 172.21.15.107:0/135356576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e81a4c40 con 0x7f90e8107720 2024-04-25T00:24:40.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.225+0000 7f90d77fe640 1 -- 172.21.15.107:0/135356576 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f90e00054d0 con 0x7f90e8107720 2024-04-25T00:24:40.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.226+0000 7f90ef600740 1 Processor -- start 2024-04-25T00:24:40.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.226+0000 7f90ef600740 1 -- start start 2024-04-25T00:24:40.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.226+0000 7f90ef600740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc942580 0x5644fc942960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.226+0000 7f90ef600740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644fc942ea0 0x5644fc94d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.226+0000 7f90ef600740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644fc94d570 0x5644fc94f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.226+0000 7f90ef600740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5644fc8a7670 con 0x5644fc942580 2024-04-25T00:24:40.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.226+0000 7f90ef600740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5644fc84f970 con 0x5644fc94d570 2024-04-25T00:24:40.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.226+0000 7f90ef600740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5644fc8a3630 con 0x5644fc942ea0 2024-04-25T00:24:40.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.226+0000 7f90e5d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc942580 0x5644fc942960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.226+0000 7f90e5d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc942580 0x5644fc942960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40654/0 (socket says 172.21.15.107:40654) 2024-04-25T00:24:40.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.226+0000 7f90e5d9b640 1 -- 172.21.15.107:0/777510758 learned_addr learned my addr 172.21.15.107:0/777510758 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:40.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.226+0000 7f90d77fe640 1 --2- 172.21.15.107:0/135356576 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f90d00a2d00 0x7f90d00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.226+0000 7f90e6d9d640 1 --2- 172.21.15.107:0/135356576 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f90d00a2d00 0x7f90d00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.226+0000 7f90e659c640 1 --2- 172.21.15.107:0/777510758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644fc942ea0 0x5644fc94d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.226+0000 7f90e5d9b640 1 --2- 172.21.15.107:0/777510758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644fc94d570 0x5644fc94f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.226+0000 7f90e5d9b640 1 -- 172.21.15.107:0/777510758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644fc942ea0 msgr2=0x5644fc94d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:40.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.226+0000 7f90e5d9b640 1 --2- 172.21.15.107:0/777510758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644fc942ea0 0x5644fc94d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.226+0000 7f90e5d9b640 1 -- 172.21.15.107:0/777510758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644fc94d570 msgr2=0x5644fc94f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:40.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.226+0000 7f90e5d9b640 1 --2- 172.21.15.107:0/777510758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644fc94d570 0x5644fc94f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.226+0000 7f90e5d9b640 1 -- 172.21.15.107:0/777510758 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644fc873400 con 0x5644fc942580 2024-04-25T00:24:40.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.226+0000 7f90d77fe640 1 -- 172.21.15.107:0/135356576 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f90e00cf980 con 0x7f90e8107720 2024-04-25T00:24:40.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.227+0000 7f90e5d9b640 1 --2- 172.21.15.107:0/777510758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc942580 0x5644fc942960 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009930 tx=0x7f90dc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ef44fc9f788121 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:40.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.227+0000 7f90e6d9d640 1 --2- 172.21.15.107:0/135356576 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f90d00a2d00 0x7f90d00a51c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f90e00ccdc0 tx=0x7f90e006db50 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:40.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.227+0000 7f90d4ff9640 1 -- 172.21.15.107:0/777510758 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc035030 con 0x5644fc942580 2024-04-25T00:24:40.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.227+0000 7f90d4ff9640 1 -- 172.21.15.107:0/777510758 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90dc042070 con 0x5644fc942580 2024-04-25T00:24:40.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.227+0000 7f90d4ff9640 1 -- 172.21.15.107:0/777510758 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc0047a0 con 0x5644fc942580 2024-04-25T00:24:40.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.227+0000 7f90ef600740 1 -- 172.21.15.107:0/777510758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc942580 msgr2=0x5644fc942960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:40.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.227+0000 7f90ef600740 1 --2- 172.21.15.107:0/777510758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc942580 0x5644fc942960 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009930 tx=0x7f90dc005fd0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.228+0000 7f90ef600740 1 -- 172.21.15.107:0/777510758 shutdown_connections 2024-04-25T00:24:40.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.228+0000 7f90ef600740 1 --2- 172.21.15.107:0/777510758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644fc94d570 0x5644fc94f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.228+0000 7f90ef600740 1 --2- 172.21.15.107:0/777510758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644fc942ea0 0x5644fc94d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.228+0000 7f90ef600740 1 --2- 172.21.15.107:0/777510758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc942580 0x5644fc942960 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.228+0000 7f90ef600740 1 -- 172.21.15.107:0/777510758 >> 172.21.15.107:0/777510758 conn(0x5644fc93e8a0 msgr2=0x5644fc93ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:40.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.228+0000 7f90ef600740 1 -- 172.21.15.107:0/777510758 shutdown_connections 2024-04-25T00:24:40.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.228+0000 7f90ef600740 1 -- 172.21.15.107:0/777510758 wait complete. 2024-04-25T00:24:40.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.228+0000 7f90ef600740 1 -- start start 2024-04-25T00:24:40.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.228+0000 7f90ef600740 1 Processor -- start 2024-04-25T00:24:40.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.228+0000 7f90ef600740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644fc942580 0x5644fc94cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.228+0000 7f90ef600740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644fc942ea0 0x5644fc94a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.228+0000 7f90ef600740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc94d570 0x5644fc94a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.228+0000 7f90ef600740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5644fc950300 con 0x5644fc94d570 2024-04-25T00:24:40.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.228+0000 7f90ef600740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5644fc863050 con 0x5644fc942ea0 2024-04-25T00:24:40.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.228+0000 7f90ef600740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5644fc950480 con 0x5644fc942580 2024-04-25T00:24:40.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.228+0000 7f90ef600740 10 client.0 _ll_register_callbacks cb 0x5644fc5babe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:40.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.228+0000 7f90e5d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc94d570 0x5644fc94a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.228+0000 7f90e5d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc94d570 0x5644fc94a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40656/0 (socket says 172.21.15.107:40656) 2024-04-25T00:24:40.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.228+0000 7f90e5d9b640 1 -- 172.21.15.107:0/2213342461 learned_addr learned my addr 172.21.15.107:0/2213342461 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:40.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.229+0000 7f90e5d9b640 1 -- 172.21.15.107:0/2213342461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644fc942580 msgr2=0x5644fc94cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:40.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.229+0000 7f90e5d9b640 1 --2- 172.21.15.107:0/2213342461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644fc942580 0x5644fc94cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.229+0000 7f90e5d9b640 1 -- 172.21.15.107:0/2213342461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644fc942ea0 msgr2=0x5644fc94a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:40.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.229+0000 7f90e5d9b640 1 --2- 172.21.15.107:0/2213342461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644fc942ea0 0x5644fc94a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.229+0000 7f90e5d9b640 1 -- 172.21.15.107:0/2213342461 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644fc834dc0 con 0x5644fc94d570 2024-04-25T00:24:40.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.229+0000 7f90e5d9b640 1 --2- 172.21.15.107:0/2213342461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc94d570 0x5644fc94a730 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f90dc02a940 tx=0x7f90dc005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2b29ffb8f99ca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:40.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.229+0000 7f90cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:24:40.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.229+0000 7f90cdffb640 1 -- 172.21.15.107:0/2213342461 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc035050 con 0x5644fc94d570 2024-04-25T00:24:40.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.229+0000 7f90cdffb640 1 -- 172.21.15.107:0/2213342461 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f90dc004020 con 0x5644fc94d570 2024-04-25T00:24:40.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.229+0000 7f90ef600740 1 -- 172.21.15.107:0/2213342461 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90dc042070 con 0x5644fc94d570 2024-04-25T00:24:40.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.229+0000 7f90cdffb640 1 -- 172.21.15.107:0/2213342461 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f90dc059050 con 0x5644fc94d570 2024-04-25T00:24:40.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.229+0000 7f90ef600740 10 client.17478 fetch_fsmap learned FSMap version 34 2024-04-25T00:24:40.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.229+0000 7f90ef600740 1 -- 172.21.15.107:0/2213342461 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5644fc5bb3f0 con 0x5644fc94d570 2024-04-25T00:24:40.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.229+0000 7f90cdffb640 1 -- 172.21.15.107:0/2213342461 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90dc042070 con 0x5644fc94d570 2024-04-25T00:24:40.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.229+0000 7f90cdffb640 1 -- 172.21.15.107:0/2213342461 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f90dc003710 con 0x5644fc94d570 2024-04-25T00:24:40.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.229+0000 7f90ef600740 10 client.17478 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:24:40.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.229+0000 7f90ef600740 10 client.17478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:40.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.229+0000 7f90ef600740 20 client.17478 populate_metadata read hostname 'smithi107' 2024-04-25T00:24:40.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.229+0000 7f90ef600740 1 --2- 172.21.15.107:0/2213342461 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5644fc95d420 0x5644fc95f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.229+0000 7f90ef600740 4 client.17478 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:40.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.229+0000 7f90ef600740 1 -- 172.21.15.107:0/2213342461 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644fc8b4bb0 con 0x5644fc95d420 2024-04-25T00:24:40.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.230+0000 7f90e659c640 1 --2- 172.21.15.107:0/2213342461 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5644fc95d420 0x5644fc95f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.230+0000 7f90e659c640 1 --2- 172.21.15.107:0/2213342461 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5644fc95d420 0x5644fc95f810 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b21933e7e0db7cf0 server_cookie=3f7c37b6ea6b8451 in_seq=0 out_seq=0 2024-04-25T00:24:40.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.230+0000 7f90cdffb640 10 client.17478 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:40.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.231+0000 7f90cdffb640 1 -- 172.21.15.107:0/2213342461 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5644fc8b4bb0 con 0x5644fc95d420 2024-04-25T00:24:40.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.231+0000 7f90cdffb640 10 client.17478 handle_command_reply: tid=0 2024-04-25T00:24:40.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.231+0000 7f90ef600740 1 client.17478 shutdown 2024-04-25T00:24:40.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.231+0000 7f90ef600740 1 -- 172.21.15.107:0/2213342461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc94d570 msgr2=0x5644fc94a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:40.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.231+0000 7f90ef600740 1 --2- 172.21.15.107:0/2213342461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc94d570 0x5644fc94a730 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f90dc02a940 tx=0x7f90dc005760 comp rx=0 tx=0).stop 2024-04-25T00:24:40.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.231+0000 7f90ef600740 1 -- 172.21.15.107:0/2213342461 shutdown_connections 2024-04-25T00:24:40.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.231+0000 7f90ef600740 1 --2- 172.21.15.107:0/2213342461 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5644fc95d420 0x5644fc95f810 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.231+0000 7f90ef600740 1 --2- 172.21.15.107:0/2213342461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc94d570 0x5644fc94a730 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.231+0000 7f90cdffb640 0 client.17478 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:40.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.231+0000 7f90ef600740 1 --2- 172.21.15.107:0/2213342461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644fc942ea0 0x5644fc94a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.231+0000 7f90ef600740 1 --2- 172.21.15.107:0/2213342461 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644fc942580 0x5644fc94cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.231+0000 7f90ef600740 1 -- 172.21.15.107:0/2213342461 >> 172.21.15.107:0/2213342461 conn(0x5644fc93e8a0 msgr2=0x5644fc93db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:40.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.231+0000 7f90ef600740 1 -- 172.21.15.107:0/2213342461 shutdown_connections 2024-04-25T00:24:40.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.231+0000 7f90ef600740 1 -- 172.21.15.107:0/2213342461 wait complete. 2024-04-25T00:24:40.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.232+0000 7f90ef600740 20 client.17478 trim_cache size 0 max 16384 2024-04-25T00:24:40.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.245+0000 7f90ef600740 1 Processor -- start 2024-04-25T00:24:40.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.245+0000 7f90ef600740 1 -- start start 2024-04-25T00:24:40.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.245+0000 7f90ef600740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644fc942580 0x5644fc958e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.245+0000 7f90ef600740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644fc942ea0 0x5644fc94ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.245+0000 7f90ef600740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc94d570 0x5644fc962830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.245+0000 7f90ef600740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f90e81b1560 con 0x5644fc94d570 2024-04-25T00:24:40.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.245+0000 7f90ef600740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f90e81aee30 con 0x5644fc942580 2024-04-25T00:24:40.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.245+0000 7f90ef600740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f90e81acca0 con 0x5644fc942ea0 2024-04-25T00:24:40.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.245+0000 7f90e5d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc94d570 0x5644fc962830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.245+0000 7f90e5d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc94d570 0x5644fc962830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40670/0 (socket says 172.21.15.107:40670) 2024-04-25T00:24:40.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.245+0000 7f90e5d9b640 1 -- 172.21.15.107:0/3947549191 learned_addr learned my addr 172.21.15.107:0/3947549191 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:40.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.245+0000 7f90e659c640 1 --2- 172.21.15.107:0/3947549191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644fc942ea0 0x5644fc94ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.245+0000 7f90e5d9b640 1 --2- 172.21.15.107:0/3947549191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644fc942580 0x5644fc958e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.245+0000 7f90e5d9b640 1 -- 172.21.15.107:0/3947549191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644fc942ea0 msgr2=0x5644fc94ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:40.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.245+0000 7f90e5d9b640 1 --2- 172.21.15.107:0/3947549191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644fc942ea0 0x5644fc94ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.245+0000 7f90e5d9b640 1 -- 172.21.15.107:0/3947549191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644fc942580 msgr2=0x5644fc958e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:40.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.245+0000 7f90e5d9b640 1 --2- 172.21.15.107:0/3947549191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644fc942580 0x5644fc958e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.245+0000 7f90e5d9b640 1 -- 172.21.15.107:0/3947549191 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644fc94aaa0 con 0x5644fc94d570 2024-04-25T00:24:40.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.245+0000 7f90e5d9b640 1 --2- 172.21.15.107:0/3947549191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc94d570 0x5644fc962830 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f90dc00c000 tx=0x7f90dc040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c8fac17c69a7313 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:40.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.245+0000 7f90cffff640 1 -- 172.21.15.107:0/3947549191 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc035070 con 0x5644fc94d570 2024-04-25T00:24:40.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.245+0000 7f90cffff640 1 -- 172.21.15.107:0/3947549191 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90dc042040 con 0x5644fc94d570 2024-04-25T00:24:40.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.245+0000 7f90cffff640 1 -- 172.21.15.107:0/3947549191 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc031a50 con 0x5644fc94d570 2024-04-25T00:24:40.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.246+0000 7f90ef600740 1 -- 172.21.15.107:0/3947549191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc94d570 msgr2=0x5644fc962830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:40.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.246+0000 7f90ef600740 1 --2- 172.21.15.107:0/3947549191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc94d570 0x5644fc962830 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f90dc00c000 tx=0x7f90dc040040 comp rx=0 tx=0).stop 2024-04-25T00:24:40.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.246+0000 7f90ef600740 1 -- 172.21.15.107:0/3947549191 shutdown_connections 2024-04-25T00:24:40.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.246+0000 7f90ef600740 1 --2- 172.21.15.107:0/3947549191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc94d570 0x5644fc962830 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.246+0000 7f90ef600740 1 --2- 172.21.15.107:0/3947549191 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644fc942ea0 0x5644fc94ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.246+0000 7f90ef600740 1 --2- 172.21.15.107:0/3947549191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644fc942580 0x5644fc958e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.246+0000 7f90ef600740 1 -- 172.21.15.107:0/3947549191 >> 172.21.15.107:0/3947549191 conn(0x5644fc93e8a0 msgr2=0x5644fc958700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:40.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.246+0000 7f90ef600740 1 -- 172.21.15.107:0/3947549191 shutdown_connections 2024-04-25T00:24:40.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.246+0000 7f90ef600740 1 -- 172.21.15.107:0/3947549191 wait complete. 2024-04-25T00:24:40.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.246+0000 7f90ef600740 1 -- start start 2024-04-25T00:24:40.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.246+0000 7f90ef600740 1 Processor -- start 2024-04-25T00:24:40.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.247+0000 7f90ef600740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc942580 0x5644fc9bba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.247+0000 7f90ef600740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644fc942ea0 0x5644fc9bbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.247+0000 7f90ef600740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644fc94d570 0x5644fc95fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.247+0000 7f90ef600740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f90e81ad2f0 con 0x5644fc942580 2024-04-25T00:24:40.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.247+0000 7f90ef600740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f90e81ae760 con 0x5644fc94d570 2024-04-25T00:24:40.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.247+0000 7f90ef600740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f90e81ad9c0 con 0x5644fc942ea0 2024-04-25T00:24:40.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.247+0000 7f90ef600740 10 client.0 _ll_register_callbacks cb 0x7f90dc005d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:40.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.247+0000 7f90e5d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc942580 0x5644fc9bba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.247+0000 7f90e5d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc942580 0x5644fc9bba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40680/0 (socket says 172.21.15.107:40680) 2024-04-25T00:24:40.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.247+0000 7f90e5d9b640 1 -- 172.21.15.107:0/26237813 learned_addr learned my addr 172.21.15.107:0/26237813 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:40.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.247+0000 7f90e5d9b640 1 --2- 172.21.15.107:0/26237813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644fc94d570 0x5644fc95fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.247+0000 7f90e659c640 1 --2- 172.21.15.107:0/26237813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644fc942ea0 0x5644fc9bbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.247+0000 7f90e5d9b640 1 -- 172.21.15.107:0/26237813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644fc942ea0 msgr2=0x5644fc9bbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:40.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.247+0000 7f90e5d9b640 1 --2- 172.21.15.107:0/26237813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644fc942ea0 0x5644fc9bbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.247+0000 7f90e5d9b640 1 -- 172.21.15.107:0/26237813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644fc94d570 msgr2=0x5644fc95fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:40.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.247+0000 7f90e5d9b640 1 --2- 172.21.15.107:0/26237813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644fc94d570 0x5644fc95fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.247+0000 7f90e5d9b640 1 -- 172.21.15.107:0/26237813 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644fc6857d0 con 0x5644fc942580 2024-04-25T00:24:40.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.247+0000 7f90e659c640 1 --2- 172.21.15.107:0/26237813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644fc942ea0 0x5644fc9bbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:40.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.247+0000 7f90e5d9b640 1 --2- 172.21.15.107:0/26237813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc942580 0x5644fc9bba00 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f90dc02a940 tx=0x7f90dc0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15bab076d0443f57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:40.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.247+0000 7f90cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:24:40.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.247+0000 7f90cdffb640 1 -- 172.21.15.107:0/26237813 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc035040 con 0x5644fc942580 2024-04-25T00:24:40.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.247+0000 7f90cdffb640 1 -- 172.21.15.107:0/26237813 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f90dc003cd0 con 0x5644fc942580 2024-04-25T00:24:40.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.247+0000 7f90ef600740 1 -- 172.21.15.107:0/26237813 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90dc042040 con 0x5644fc942580 2024-04-25T00:24:40.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.247+0000 7f90cdffb640 1 -- 172.21.15.107:0/26237813 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f90dc003f60 con 0x5644fc942580 2024-04-25T00:24:40.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.248+0000 7f90ef600740 10 client.17484 fetch_fsmap learned FSMap version 34 2024-04-25T00:24:40.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.248+0000 7f90ef600740 1 -- 172.21.15.107:0/26237813 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5644fc9c2fe0 con 0x5644fc942580 2024-04-25T00:24:40.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.248+0000 7f90cdffb640 1 -- 172.21.15.107:0/26237813 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90dc042040 con 0x5644fc942580 2024-04-25T00:24:40.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.248+0000 7f90cdffb640 1 -- 172.21.15.107:0/26237813 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f90dc066070 con 0x5644fc942580 2024-04-25T00:24:40.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.248+0000 7f90ef600740 10 client.17484 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:24:40.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.248+0000 7f90ef600740 10 client.17484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:40.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.248+0000 7f90ef600740 20 client.17484 populate_metadata read hostname 'smithi107' 2024-04-25T00:24:40.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.248+0000 7f90ef600740 1 --2- 172.21.15.107:0/26237813 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5644fc9c3460 0x5644fc9c5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:40.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.248+0000 7f90ef600740 4 client.17484 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:24:40.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.248+0000 7f90ef600740 1 -- 172.21.15.107:0/26237813 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5644fc8aef00 con 0x5644fc9c3460 2024-04-25T00:24:40.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.248+0000 7f90e659c640 1 --2- 172.21.15.107:0/26237813 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5644fc9c3460 0x5644fc9c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:40.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.248+0000 7f90e659c640 1 --2- 172.21.15.107:0/26237813 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5644fc9c3460 0x5644fc9c5850 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ad86e06e2deb665 server_cookie=a3b1e14357b39ed in_seq=0 out_seq=0 2024-04-25T00:24:40.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.248+0000 7f90cdffb640 10 client.17484 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:40.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.249+0000 7f90cdffb640 1 -- 172.21.15.107:0/26237813 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5644fc8aef00 con 0x5644fc9c3460 2024-04-25T00:24:40.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.249+0000 7f90cdffb640 10 client.17484 handle_command_reply: tid=0 2024-04-25T00:24:40.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.249+0000 7f90ef600740 1 client.17484 shutdown 2024-04-25T00:24:40.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.249+0000 7f90ef600740 1 -- 172.21.15.107:0/26237813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc942580 msgr2=0x5644fc9bba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:40.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.249+0000 7f90ef600740 1 --2- 172.21.15.107:0/26237813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc942580 0x5644fc9bba00 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f90dc02a940 tx=0x7f90dc0058d0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.249+0000 7f90ef600740 1 -- 172.21.15.107:0/26237813 shutdown_connections 2024-04-25T00:24:40.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.249+0000 7f90ef600740 1 --2- 172.21.15.107:0/26237813 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5644fc9c3460 0x5644fc9c5850 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.249+0000 7f90ef600740 1 --2- 172.21.15.107:0/26237813 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5644fc94d570 0x5644fc95fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.249+0000 7f90ef600740 1 --2- 172.21.15.107:0/26237813 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5644fc942ea0 0x5644fc9bbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.249+0000 7f90cdffb640 0 client.17484 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:40.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.249+0000 7f90ef600740 1 --2- 172.21.15.107:0/26237813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5644fc942580 0x5644fc9bba00 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.249+0000 7f90ef600740 1 -- 172.21.15.107:0/26237813 >> 172.21.15.107:0/26237813 conn(0x5644fc93e8a0 msgr2=0x5644fc958f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:40.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.249+0000 7f90ef600740 1 -- 172.21.15.107:0/26237813 shutdown_connections 2024-04-25T00:24:40.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.249+0000 7f90ef600740 1 -- 172.21.15.107:0/26237813 wait complete. 2024-04-25T00:24:40.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.249+0000 7f90ef600740 20 client.17484 trim_cache size 0 max 16384 2024-04-25T00:24:40.257 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:24:40.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.249+0000 7f90ec815640 1 -- 172.21.15.107:0/135356576 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f90d00a2d00 msgr2=0x7f90d00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:40.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.249+0000 7f90ec815640 1 --2- 172.21.15.107:0/135356576 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f90d00a2d00 0x7f90d00a51c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f90e00ccdc0 tx=0x7f90e006db50 comp rx=0 tx=0).stop 2024-04-25T00:24:40.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.249+0000 7f90ec815640 1 -- 172.21.15.107:0/135356576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90e8107720 msgr2=0x7f90e819c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:40.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.249+0000 7f90ec815640 1 --2- 172.21.15.107:0/135356576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90e8107720 0x7f90e819c990 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f90e000eaa0 tx=0x7f90e000ef60 comp rx=0 tx=0).stop 2024-04-25T00:24:40.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.250+0000 7f90ec815640 1 -- 172.21.15.107:0/135356576 shutdown_connections 2024-04-25T00:24:40.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.250+0000 7f90ec815640 1 --2- 172.21.15.107:0/135356576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90e8107720 0x7f90e819c990 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.250+0000 7f90ec815640 1 --2- 172.21.15.107:0/135356576 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90e81020d0 0x7f90e8198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.250+0000 7f90ec815640 1 --2- 172.21.15.107:0/135356576 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f90d00a2d00 0x7f90d00a51c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.250+0000 7f90ec815640 1 --2- 172.21.15.107:0/135356576 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90e8101710 0x7f90e81980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:40.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.250+0000 7f90ec815640 1 -- 172.21.15.107:0/135356576 >> 172.21.15.107:0/135356576 conn(0x7f90e80fd180 msgr2=0x7f90e80fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:40.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.250+0000 7f90ec815640 1 -- 172.21.15.107:0/135356576 shutdown_connections 2024-04-25T00:24:40.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:40.250+0000 7f90ec815640 1 -- 172.21.15.107:0/135356576 wait complete. 2024-04-25T00:24:40.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:40 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1597083199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:24:40.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:40 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1597083199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:24:41.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:40 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1597083199' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:24:41.261 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:24:41.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:24:41.261 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:24:41.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.374+0000 7fc283c15640 1 -- 172.21.15.107:0/1721648955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc27c101710 msgr2=0x7fc27c101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:41.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.374+0000 7fc283c15640 1 --2- 172.21.15.107:0/1721648955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc27c101710 0x7fc27c101b90 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009930 tx=0x7fc26c02a420 comp rx=0 tx=0).stop 2024-04-25T00:24:41.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.374+0000 7fc283c15640 1 -- 172.21.15.107:0/1721648955 shutdown_connections 2024-04-25T00:24:41.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.374+0000 7fc283c15640 1 --2- 172.21.15.107:0/1721648955 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc27c1020d0 0x7fc27c10e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.374+0000 7fc283c15640 1 --2- 172.21.15.107:0/1721648955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc27c101710 0x7fc27c101b90 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.374+0000 7fc283c15640 1 --2- 172.21.15.107:0/1721648955 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc27c107720 0x7fc27c107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.374+0000 7fc283c15640 1 -- 172.21.15.107:0/1721648955 >> 172.21.15.107:0/1721648955 conn(0x7fc27c078b90 msgr2=0x7fc27c078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:41.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.374+0000 7fc283c15640 1 -- 172.21.15.107:0/1721648955 shutdown_connections 2024-04-25T00:24:41.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.374+0000 7fc283c15640 1 -- 172.21.15.107:0/1721648955 wait complete. 2024-04-25T00:24:41.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.375+0000 7fc283c15640 1 Processor -- start 2024-04-25T00:24:41.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.375+0000 7fc283c15640 1 -- start start 2024-04-25T00:24:41.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.375+0000 7fc283c15640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc27c101710 0x7fc27c1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.375+0000 7fc283c15640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc27c1020d0 0x7fc27c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.375+0000 7fc283c15640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc27c107720 0x7fc27c19c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.375+0000 7fc283c15640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c111b30 con 0x7fc27c1020d0 2024-04-25T00:24:41.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.375+0000 7fc283c15640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c1119b0 con 0x7fc27c101710 2024-04-25T00:24:41.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.375+0000 7fc283c15640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c111cb0 con 0x7fc27c107720 2024-04-25T00:24:41.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.375+0000 7fc2811b0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc27c1020d0 0x7fc27c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.375+0000 7fc2811b0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc27c1020d0 0x7fc27c1985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40694/0 (socket says 172.21.15.107:40694) 2024-04-25T00:24:41.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.375+0000 7fc2811b0640 1 -- 172.21.15.107:0/1130968128 learned_addr learned my addr 172.21.15.107:0/1130968128 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:41.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.376+0000 7fc2819b1640 1 --2- 172.21.15.107:0/1130968128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc27c101710 0x7fc27c1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.376+0000 7fc2821b2640 1 --2- 172.21.15.107:0/1130968128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc27c107720 0x7fc27c19c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.376+0000 7fc2811b0640 1 -- 172.21.15.107:0/1130968128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc27c107720 msgr2=0x7fc27c19c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:41.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.376+0000 7fc2811b0640 1 --2- 172.21.15.107:0/1130968128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc27c107720 0x7fc27c19c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.376+0000 7fc2811b0640 1 -- 172.21.15.107:0/1130968128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc27c101710 msgr2=0x7fc27c1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:41.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.376+0000 7fc2811b0640 1 --2- 172.21.15.107:0/1130968128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc27c101710 0x7fc27c1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.376+0000 7fc2811b0640 1 -- 172.21.15.107:0/1130968128 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27c19d050 con 0x7fc27c1020d0 2024-04-25T00:24:41.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.376+0000 7fc2821b2640 1 --2- 172.21.15.107:0/1130968128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc27c107720 0x7fc27c19c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:41.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.376+0000 7fc2811b0640 1 --2- 172.21.15.107:0/1130968128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc27c1020d0 0x7fc27c1985f0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009230 tx=0x7fc26c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:41.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.376+0000 7fc272ffd640 1 -- 172.21.15.107:0/1130968128 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c03f030 con 0x7fc27c1020d0 2024-04-25T00:24:41.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.376+0000 7fc272ffd640 1 -- 172.21.15.107:0/1130968128 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc26c003ad0 con 0x7fc27c1020d0 2024-04-25T00:24:41.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.376+0000 7fc283c15640 1 -- 172.21.15.107:0/1130968128 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc27c19d2e0 con 0x7fc27c1020d0 2024-04-25T00:24:41.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.376+0000 7fc272ffd640 1 -- 172.21.15.107:0/1130968128 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26c035070 con 0x7fc27c1020d0 2024-04-25T00:24:41.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.376+0000 7fc283c15640 1 -- 172.21.15.107:0/1130968128 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc27c1a4bd0 con 0x7fc27c1020d0 2024-04-25T00:24:41.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.378+0000 7fc272ffd640 1 -- 172.21.15.107:0/1130968128 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc26c042030 con 0x7fc27c1020d0 2024-04-25T00:24:41.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.379+0000 7fc286a00740 1 Processor -- start 2024-04-25T00:24:41.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.379+0000 7fc286a00740 1 -- start start 2024-04-25T00:24:41.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.379+0000 7fc286a00740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd8c6a980 0x561cd8c68cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.379+0000 7fc286a00740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c69290 0x561cd8c69710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.379+0000 7fc286a00740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cd8c78420 0x561cd8c7a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.379+0000 7fc286a00740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561cd8bcd2c0 con 0x561cd8c69290 2024-04-25T00:24:41.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.379+0000 7fc272ffd640 1 --2- 172.21.15.107:0/1130968128 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc2680a2d00 0x7fc2680a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.379+0000 7fc286a00740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561cd8b75910 con 0x561cd8c78420 2024-04-25T00:24:41.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.379+0000 7fc286a00740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561cd8bc9280 con 0x561cd8c6a980 2024-04-25T00:24:41.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.379+0000 7fc2819b1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c69290 0x561cd8c69710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.379+0000 7fc2821b2640 1 --2- 172.21.15.107:0/1130968128 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc2680a2d00 0x7fc2680a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.380+0000 7fc2819b1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c69290 0x561cd8c69710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40700/0 (socket says 172.21.15.107:40700) 2024-04-25T00:24:41.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.380+0000 7fc2819b1640 1 -- 172.21.15.107:0/1987950257 learned_addr learned my addr 172.21.15.107:0/1987950257 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:41.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.380+0000 7fc2821b2640 1 --2- 172.21.15.107:0/1987950257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd8c6a980 0x561cd8c68cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.380+0000 7fc272ffd640 1 -- 172.21.15.107:0/1130968128 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7fc26c0efd10 con 0x7fc27c1020d0 2024-04-25T00:24:41.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.380+0000 7fc2811b0640 1 --2- 172.21.15.107:0/1987950257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cd8c78420 0x561cd8c7a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.380+0000 7fc2819b1640 1 -- 172.21.15.107:0/1987950257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd8c6a980 msgr2=0x561cd8c68cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:41.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.380+0000 7fc2819b1640 1 --2- 172.21.15.107:0/1987950257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd8c6a980 0x561cd8c68cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.380+0000 7fc2819b1640 1 -- 172.21.15.107:0/1987950257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cd8c78420 msgr2=0x561cd8c7a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:41.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.380+0000 7fc2819b1640 1 --2- 172.21.15.107:0/1987950257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cd8c78420 0x561cd8c7a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.380+0000 7fc2819b1640 1 -- 172.21.15.107:0/1987950257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cd8b86fb0 con 0x561cd8c69290 2024-04-25T00:24:41.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.380+0000 7fc2821b2640 1 --2- 172.21.15.107:0/1130968128 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc2680a2d00 0x7fc2680a51c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc274005c40 tx=0x7fc274009dd0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:41.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.380+0000 7fc2819b1640 1 --2- 172.21.15.107:0/1987950257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c69290 0x561cd8c69710 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fc27800bc90 tx=0x7fc2780107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7725b4b6bc61a45c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:41.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.380+0000 7fc267fff640 1 -- 172.21.15.107:0/1987950257 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27801c480 con 0x561cd8c69290 2024-04-25T00:24:41.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.380+0000 7fc267fff640 1 -- 172.21.15.107:0/1987950257 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc27801cb00 con 0x561cd8c69290 2024-04-25T00:24:41.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.380+0000 7fc267fff640 1 -- 172.21.15.107:0/1987950257 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278011b40 con 0x561cd8c69290 2024-04-25T00:24:41.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.381+0000 7fc286a00740 1 -- 172.21.15.107:0/1987950257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c69290 msgr2=0x561cd8c69710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:41.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.381+0000 7fc286a00740 1 --2- 172.21.15.107:0/1987950257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c69290 0x561cd8c69710 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fc27800bc90 tx=0x7fc2780107a0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.381+0000 7fc286a00740 1 -- 172.21.15.107:0/1987950257 shutdown_connections 2024-04-25T00:24:41.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.381+0000 7fc286a00740 1 --2- 172.21.15.107:0/1987950257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cd8c78420 0x561cd8c7a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.381+0000 7fc286a00740 1 --2- 172.21.15.107:0/1987950257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c69290 0x561cd8c69710 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.381+0000 7fc286a00740 1 --2- 172.21.15.107:0/1987950257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd8c6a980 0x561cd8c68cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.381+0000 7fc286a00740 1 -- 172.21.15.107:0/1987950257 >> 172.21.15.107:0/1987950257 conn(0x561cd8c31950 msgr2=0x561cd8c657f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:41.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.381+0000 7fc286a00740 1 -- 172.21.15.107:0/1987950257 shutdown_connections 2024-04-25T00:24:41.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.381+0000 7fc286a00740 1 -- 172.21.15.107:0/1987950257 wait complete. 2024-04-25T00:24:41.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.381+0000 7fc286a00740 1 -- start start 2024-04-25T00:24:41.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.381+0000 7fc286a00740 1 Processor -- start 2024-04-25T00:24:41.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.382+0000 7fc286a00740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cd8c69290 0x561cd8c75300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.382+0000 7fc286a00740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd8c6a980 0x561cd8c75840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.382+0000 7fc286a00740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c78420 0x561cd8c75d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.382+0000 7fc286a00740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561cd8c7b2b0 con 0x561cd8c78420 2024-04-25T00:24:41.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.382+0000 7fc286a00740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561cd8b88e90 con 0x561cd8c69290 2024-04-25T00:24:41.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.382+0000 7fc286a00740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x561cd8c7b430 con 0x561cd8c6a980 2024-04-25T00:24:41.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.382+0000 7fc286a00740 10 client.0 _ll_register_callbacks cb 0x561cd8bc4970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:41.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.382+0000 7fc2821b2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c78420 0x561cd8c75d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.382+0000 7fc2819b1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cd8c69290 0x561cd8c75300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.382+0000 7fc2821b2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c78420 0x561cd8c75d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40706/0 (socket says 172.21.15.107:40706) 2024-04-25T00:24:41.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.382+0000 7fc2821b2640 1 -- 172.21.15.107:0/1963985846 learned_addr learned my addr 172.21.15.107:0/1963985846 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:41.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.382+0000 7fc2811b0640 1 --2- 172.21.15.107:0/1963985846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd8c6a980 0x561cd8c75840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.382+0000 7fc2821b2640 1 -- 172.21.15.107:0/1963985846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd8c6a980 msgr2=0x561cd8c75840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:41.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.382+0000 7fc2821b2640 1 --2- 172.21.15.107:0/1963985846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd8c6a980 0x561cd8c75840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.382+0000 7fc2821b2640 1 -- 172.21.15.107:0/1963985846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cd8c69290 msgr2=0x561cd8c75300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:41.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.382+0000 7fc2821b2640 1 --2- 172.21.15.107:0/1963985846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cd8c69290 0x561cd8c75300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.382+0000 7fc2821b2640 1 -- 172.21.15.107:0/1963985846 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cd89a97e0 con 0x561cd8c78420 2024-04-25T00:24:41.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.382+0000 7fc2819b1640 1 --2- 172.21.15.107:0/1963985846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cd8c69290 0x561cd8c75300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:41.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.383+0000 7fc2657fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:24:41.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.382+0000 7fc2821b2640 1 --2- 172.21.15.107:0/1963985846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c78420 0x561cd8c75d80 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fc274009d20 tx=0x7fc274005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da60f5b4bdc615d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:41.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.383+0000 7fc2657fa640 1 -- 172.21.15.107:0/1963985846 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274024070 con 0x561cd8c78420 2024-04-25T00:24:41.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.383+0000 7fc2657fa640 1 -- 172.21.15.107:0/1963985846 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7fc274029460 con 0x561cd8c78420 2024-04-25T00:24:41.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.383+0000 7fc286a00740 1 -- 172.21.15.107:0/1963985846 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc27801cb00 con 0x561cd8c78420 2024-04-25T00:24:41.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.383+0000 7fc2657fa640 1 -- 172.21.15.107:0/1963985846 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc274011e30 con 0x561cd8c78420 2024-04-25T00:24:41.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.383+0000 7fc286a00740 10 client.17496 fetch_fsmap learned FSMap version 34 2024-04-25T00:24:41.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.383+0000 7fc286a00740 1 -- 172.21.15.107:0/1963985846 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561cd8a09f70 con 0x561cd8c78420 2024-04-25T00:24:41.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.383+0000 7fc2657fa640 1 -- 172.21.15.107:0/1963985846 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc27801cb00 con 0x561cd8c78420 2024-04-25T00:24:41.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.383+0000 7fc2657fa640 1 -- 172.21.15.107:0/1963985846 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fc27401f070 con 0x561cd8c78420 2024-04-25T00:24:41.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.383+0000 7fc286a00740 10 client.17496 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:24:41.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.383+0000 7fc286a00740 10 client.17496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:41.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.383+0000 7fc286a00740 20 client.17496 populate_metadata read hostname 'smithi107' 2024-04-25T00:24:41.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.383+0000 7fc286a00740 1 --2- 172.21.15.107:0/1963985846 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x561cd8c884d0 0x561cd8c8a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.383+0000 7fc286a00740 4 client.17496 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:41.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.383+0000 7fc286a00740 1 -- 172.21.15.107:0/1963985846 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cd8ba15a0 con 0x561cd8c884d0 2024-04-25T00:24:41.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.384+0000 7fc2819b1640 1 --2- 172.21.15.107:0/1963985846 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x561cd8c884d0 0x561cd8c8a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.384+0000 7fc2819b1640 1 --2- 172.21.15.107:0/1963985846 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x561cd8c884d0 0x561cd8c8a920 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51cac8dce9688161 server_cookie=ade5cf2add73a2ff in_seq=0 out_seq=0 2024-04-25T00:24:41.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.384+0000 7fc2657fa640 10 client.17496 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:41.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.385+0000 7fc2657fa640 1 -- 172.21.15.107:0/1963985846 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561cd8ba15a0 con 0x561cd8c884d0 2024-04-25T00:24:41.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.385+0000 7fc2657fa640 10 client.17496 handle_command_reply: tid=0 2024-04-25T00:24:41.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.385+0000 7fc286a00740 1 client.17496 shutdown 2024-04-25T00:24:41.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.385+0000 7fc286a00740 1 -- 172.21.15.107:0/1963985846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c78420 msgr2=0x561cd8c75d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:41.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.385+0000 7fc286a00740 1 --2- 172.21.15.107:0/1963985846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c78420 0x561cd8c75d80 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fc274009d20 tx=0x7fc274005bd0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.385+0000 7fc286a00740 1 -- 172.21.15.107:0/1963985846 shutdown_connections 2024-04-25T00:24:41.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.385+0000 7fc286a00740 1 --2- 172.21.15.107:0/1963985846 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x561cd8c884d0 0x561cd8c8a920 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.385+0000 7fc286a00740 1 --2- 172.21.15.107:0/1963985846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c78420 0x561cd8c75d80 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.385+0000 7fc286a00740 1 --2- 172.21.15.107:0/1963985846 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd8c6a980 0x561cd8c75840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.385+0000 7fc2657fa640 0 client.17496 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:41.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.385+0000 7fc286a00740 1 --2- 172.21.15.107:0/1963985846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cd8c69290 0x561cd8c75300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.385+0000 7fc286a00740 1 -- 172.21.15.107:0/1963985846 >> 172.21.15.107:0/1963985846 conn(0x561cd8c31950 msgr2=0x561cd8c6b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:41.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.385+0000 7fc286a00740 1 -- 172.21.15.107:0/1963985846 shutdown_connections 2024-04-25T00:24:41.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.385+0000 7fc286a00740 1 -- 172.21.15.107:0/1963985846 wait complete. 2024-04-25T00:24:41.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.385+0000 7fc286a00740 20 client.17496 trim_cache size 0 max 16384 2024-04-25T00:24:41.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.402+0000 7fc286a00740 1 Processor -- start 2024-04-25T00:24:41.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.403+0000 7fc286a00740 1 -- start start 2024-04-25T00:24:41.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.403+0000 7fc286a00740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd8c69290 0x561cd8c841b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.403+0000 7fc286a00740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cd8c6a980 0x561cd8c647d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.403+0000 7fc286a00740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c78420 0x561cd8c901f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.403+0000 7fc286a00740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c1af2c0 con 0x561cd8c78420 2024-04-25T00:24:41.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.403+0000 7fc286a00740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c1acb90 con 0x561cd8c6a980 2024-04-25T00:24:41.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.403+0000 7fc286a00740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c1aaa00 con 0x561cd8c69290 2024-04-25T00:24:41.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.403+0000 7fc2821b2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c78420 0x561cd8c901f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.403+0000 7fc2821b2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c78420 0x561cd8c901f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40716/0 (socket says 172.21.15.107:40716) 2024-04-25T00:24:41.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.403+0000 7fc2821b2640 1 -- 172.21.15.107:0/3048745982 learned_addr learned my addr 172.21.15.107:0/3048745982 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:41.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.403+0000 7fc2821b2640 1 -- 172.21.15.107:0/3048745982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd8c69290 msgr2=0x561cd8c841b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:41.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.403+0000 7fc2821b2640 1 --2- 172.21.15.107:0/3048745982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd8c69290 0x561cd8c841b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.403+0000 7fc2821b2640 1 -- 172.21.15.107:0/3048745982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cd8c6a980 msgr2=0x561cd8c647d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:41.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.403+0000 7fc2821b2640 1 --2- 172.21.15.107:0/3048745982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cd8c6a980 0x561cd8c647d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.403+0000 7fc2821b2640 1 -- 172.21.15.107:0/3048745982 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cd88a1170 con 0x561cd8c78420 2024-04-25T00:24:41.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.403+0000 7fc2821b2640 1 --2- 172.21.15.107:0/3048745982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c78420 0x561cd8c901f0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fc27401e840 tx=0x7fc2740095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72d44d99d53fc74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:41.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.403+0000 7fc2677fe640 1 -- 172.21.15.107:0/3048745982 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274024040 con 0x561cd8c78420 2024-04-25T00:24:41.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.403+0000 7fc2677fe640 1 -- 172.21.15.107:0/3048745982 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc274009b70 con 0x561cd8c78420 2024-04-25T00:24:41.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.403+0000 7fc2677fe640 1 -- 172.21.15.107:0/3048745982 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27401e400 con 0x561cd8c78420 2024-04-25T00:24:41.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.404+0000 7fc286a00740 1 -- 172.21.15.107:0/3048745982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c78420 msgr2=0x561cd8c901f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:41.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.404+0000 7fc286a00740 1 --2- 172.21.15.107:0/3048745982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c78420 0x561cd8c901f0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fc27401e840 tx=0x7fc2740095b0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.404+0000 7fc286a00740 1 -- 172.21.15.107:0/3048745982 shutdown_connections 2024-04-25T00:24:41.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.404+0000 7fc286a00740 1 --2- 172.21.15.107:0/3048745982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c78420 0x561cd8c901f0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.404+0000 7fc286a00740 1 --2- 172.21.15.107:0/3048745982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cd8c6a980 0x561cd8c647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.404+0000 7fc286a00740 1 --2- 172.21.15.107:0/3048745982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd8c69290 0x561cd8c841b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.404+0000 7fc286a00740 1 -- 172.21.15.107:0/3048745982 >> 172.21.15.107:0/3048745982 conn(0x561cd8c31950 msgr2=0x561cd8c640c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:41.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.404+0000 7fc286a00740 1 -- 172.21.15.107:0/3048745982 shutdown_connections 2024-04-25T00:24:41.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.404+0000 7fc286a00740 1 -- 172.21.15.107:0/3048745982 wait complete. 2024-04-25T00:24:41.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.404+0000 7fc286a00740 1 -- start start 2024-04-25T00:24:41.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.404+0000 7fc286a00740 1 Processor -- start 2024-04-25T00:24:41.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.405+0000 7fc286a00740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd8c69290 0x561cd8c806c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.405+0000 7fc286a00740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cd8c6a980 0x561cd8c80c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.405+0000 7fc286a00740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c78420 0x561cd8c71570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.405+0000 7fc286a00740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c1ab050 con 0x561cd8c78420 2024-04-25T00:24:41.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.405+0000 7fc286a00740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c1ac4c0 con 0x561cd8c6a980 2024-04-25T00:24:41.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.405+0000 7fc286a00740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc27c1ab720 con 0x561cd8c69290 2024-04-25T00:24:41.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.405+0000 7fc286a00740 10 client.0 _ll_register_callbacks cb 0x7fc274009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:41.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.405+0000 7fc2821b2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c78420 0x561cd8c71570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.405+0000 7fc2821b2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c78420 0x561cd8c71570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40718/0 (socket says 172.21.15.107:40718) 2024-04-25T00:24:41.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.405+0000 7fc2821b2640 1 -- 172.21.15.107:0/3191785715 learned_addr learned my addr 172.21.15.107:0/3191785715 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:41.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.405+0000 7fc2811b0640 1 --2- 172.21.15.107:0/3191785715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cd8c6a980 0x561cd8c80c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.405+0000 7fc2819b1640 1 --2- 172.21.15.107:0/3191785715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd8c69290 0x561cd8c806c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.405+0000 7fc2821b2640 1 -- 172.21.15.107:0/3191785715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd8c69290 msgr2=0x561cd8c806c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:41.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.405+0000 7fc2821b2640 1 --2- 172.21.15.107:0/3191785715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd8c69290 0x561cd8c806c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.405+0000 7fc2821b2640 1 -- 172.21.15.107:0/3191785715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cd8c6a980 msgr2=0x561cd8c80c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:41.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.405+0000 7fc2821b2640 1 --2- 172.21.15.107:0/3191785715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cd8c6a980 0x561cd8c80c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.405+0000 7fc2821b2640 1 -- 172.21.15.107:0/3191785715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cd8c71d70 con 0x561cd8c78420 2024-04-25T00:24:41.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.405+0000 7fc2821b2640 1 --2- 172.21.15.107:0/3191785715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c78420 0x561cd8c71570 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fc274010820 tx=0x7fc274012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4dface3fe238d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:41.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.405+0000 7fc2657fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:24:41.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.406+0000 7fc2657fa640 1 -- 172.21.15.107:0/3191785715 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274024040 con 0x561cd8c78420 2024-04-25T00:24:41.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.406+0000 7fc2657fa640 1 -- 172.21.15.107:0/3191785715 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7fc27401ea30 con 0x561cd8c78420 2024-04-25T00:24:41.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.406+0000 7fc286a00740 1 -- 172.21.15.107:0/3191785715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc274009b70 con 0x561cd8c78420 2024-04-25T00:24:41.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.406+0000 7fc2657fa640 1 -- 172.21.15.107:0/3191785715 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc274047050 con 0x561cd8c78420 2024-04-25T00:24:41.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.406+0000 7fc286a00740 10 client.17502 fetch_fsmap learned FSMap version 34 2024-04-25T00:24:41.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.406+0000 7fc286a00740 1 -- 172.21.15.107:0/3191785715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561cd8c730b0 con 0x561cd8c78420 2024-04-25T00:24:41.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.406+0000 7fc2657fa640 1 -- 172.21.15.107:0/3191785715 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc274009b70 con 0x561cd8c78420 2024-04-25T00:24:41.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.406+0000 7fc2657fa640 1 -- 172.21.15.107:0/3191785715 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fc274029830 con 0x561cd8c78420 2024-04-25T00:24:41.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.406+0000 7fc286a00740 10 client.17502 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:24:41.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.406+0000 7fc286a00740 10 client.17502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:41.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.406+0000 7fc286a00740 20 client.17502 populate_metadata read hostname 'smithi107' 2024-04-25T00:24:41.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.406+0000 7fc286a00740 1 --2- 172.21.15.107:0/3191785715 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x561cd8cefca0 0x561cd8cf20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.406+0000 7fc286a00740 4 client.17502 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:24:41.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.406+0000 7fc286a00740 1 -- 172.21.15.107:0/3191785715 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561cd8bda800 con 0x561cd8cefca0 2024-04-25T00:24:41.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.406+0000 7fc2819b1640 1 --2- 172.21.15.107:0/3191785715 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x561cd8cefca0 0x561cd8cf20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.407+0000 7fc2819b1640 1 --2- 172.21.15.107:0/3191785715 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x561cd8cefca0 0x561cd8cf20c0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49108e0a9779c014 server_cookie=e0ec3461e318c188 in_seq=0 out_seq=0 2024-04-25T00:24:41.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.407+0000 7fc2657fa640 10 client.17502 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:41.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.408+0000 7fc2657fa640 1 -- 172.21.15.107:0/3191785715 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561cd8bda800 con 0x561cd8cefca0 2024-04-25T00:24:41.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.408+0000 7fc2657fa640 10 client.17502 handle_command_reply: tid=0 2024-04-25T00:24:41.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.408+0000 7fc286a00740 1 client.17502 shutdown 2024-04-25T00:24:41.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.408+0000 7fc286a00740 1 -- 172.21.15.107:0/3191785715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c78420 msgr2=0x561cd8c71570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:41.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.408+0000 7fc286a00740 1 --2- 172.21.15.107:0/3191785715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c78420 0x561cd8c71570 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fc274010820 tx=0x7fc274012450 comp rx=0 tx=0).stop 2024-04-25T00:24:41.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.408+0000 7fc286a00740 1 -- 172.21.15.107:0/3191785715 shutdown_connections 2024-04-25T00:24:41.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.408+0000 7fc286a00740 1 --2- 172.21.15.107:0/3191785715 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x561cd8cefca0 0x561cd8cf20c0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.408+0000 7fc286a00740 1 --2- 172.21.15.107:0/3191785715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cd8c78420 0x561cd8c71570 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.408+0000 7fc286a00740 1 --2- 172.21.15.107:0/3191785715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561cd8c6a980 0x561cd8c80c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.408+0000 7fc2657fa640 0 client.17502 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:41.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.408+0000 7fc286a00740 1 --2- 172.21.15.107:0/3191785715 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x561cd8c69290 0x561cd8c806c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.408+0000 7fc286a00740 1 -- 172.21.15.107:0/3191785715 >> 172.21.15.107:0/3191785715 conn(0x561cd8c31950 msgr2=0x561cd8c64950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:41.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.408+0000 7fc286a00740 1 -- 172.21.15.107:0/3191785715 shutdown_connections 2024-04-25T00:24:41.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.408+0000 7fc286a00740 1 -- 172.21.15.107:0/3191785715 wait complete. 2024-04-25T00:24:41.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.408+0000 7fc286a00740 20 client.17502 trim_cache size 0 max 16384 2024-04-25T00:24:41.415 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:24:41.415 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-04-25T00:24:41.415 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "acdb60e6-62ed-44d5-8c0d-d26810113bc6", 2024-04-25T00:24:41.415 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-04-25T00:24:41.415 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:24:41.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.409+0000 7fc283c15640 1 -- 172.21.15.107:0/1130968128 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc2680a2d00 msgr2=0x7fc2680a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:41.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.409+0000 7fc283c15640 1 --2- 172.21.15.107:0/1130968128 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc2680a2d00 0x7fc2680a51c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc274005c40 tx=0x7fc274009dd0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.409+0000 7fc283c15640 1 -- 172.21.15.107:0/1130968128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc27c1020d0 msgr2=0x7fc27c1985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:41.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.409+0000 7fc283c15640 1 --2- 172.21.15.107:0/1130968128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc27c1020d0 0x7fc27c1985f0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009230 tx=0x7fc26c004270 comp rx=0 tx=0).stop 2024-04-25T00:24:41.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.409+0000 7fc283c15640 1 -- 172.21.15.107:0/1130968128 shutdown_connections 2024-04-25T00:24:41.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.409+0000 7fc283c15640 1 --2- 172.21.15.107:0/1130968128 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc27c107720 0x7fc27c19c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.409+0000 7fc283c15640 1 --2- 172.21.15.107:0/1130968128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc27c1020d0 0x7fc27c1985f0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.409+0000 7fc283c15640 1 --2- 172.21.15.107:0/1130968128 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc2680a2d00 0x7fc2680a51c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.409+0000 7fc283c15640 1 --2- 172.21.15.107:0/1130968128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc27c101710 0x7fc27c1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.409+0000 7fc283c15640 1 -- 172.21.15.107:0/1130968128 >> 172.21.15.107:0/1130968128 conn(0x7fc27c078b90 msgr2=0x7fc27c0ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:41.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.409+0000 7fc283c15640 1 -- 172.21.15.107:0/1130968128 shutdown_connections 2024-04-25T00:24:41.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.409+0000 7fc283c15640 1 -- 172.21.15.107:0/1130968128 wait complete. 2024-04-25T00:24:41.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:24:41.420 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:24:41.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.521+0000 7f8452ebd640 1 -- 172.21.15.107:0/2898460495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f844c100620 msgr2=0x7f844c100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:41.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.521+0000 7f8452ebd640 1 --2- 172.21.15.107:0/2898460495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f844c100620 0x7f844c100aa0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f8438009930 tx=0x7f843802a420 comp rx=0 tx=0).stop 2024-04-25T00:24:41.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.521+0000 7f8452ebd640 1 -- 172.21.15.107:0/2898460495 shutdown_connections 2024-04-25T00:24:41.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.521+0000 7f8452ebd640 1 --2- 172.21.15.107:0/2898460495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f844c100fe0 0x7f844c10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.521+0000 7f8452ebd640 1 --2- 172.21.15.107:0/2898460495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f844c100620 0x7f844c100aa0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.521+0000 7f8452ebd640 1 --2- 172.21.15.107:0/2898460495 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f844c106630 0x7f844c106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.521+0000 7f8452ebd640 1 -- 172.21.15.107:0/2898460495 >> 172.21.15.107:0/2898460495 conn(0x7f844c0776a0 msgr2=0x7f844c077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:41.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.521+0000 7f8452ebd640 1 -- 172.21.15.107:0/2898460495 shutdown_connections 2024-04-25T00:24:41.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.521+0000 7f8452ebd640 1 -- 172.21.15.107:0/2898460495 wait complete. 2024-04-25T00:24:41.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.522+0000 7f8452ebd640 1 Processor -- start 2024-04-25T00:24:41.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.522+0000 7f8452ebd640 1 -- start start 2024-04-25T00:24:41.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.522+0000 7f8452ebd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f844c100620 0x7f844c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.522+0000 7f8452ebd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f844c100fe0 0x7f844c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.522+0000 7f8452ebd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f844c106630 0x7f844c19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.522+0000 7f8452ebd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f844c110a60 con 0x7f844c106630 2024-04-25T00:24:41.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.522+0000 7f8452ebd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f844c1108e0 con 0x7f844c100fe0 2024-04-25T00:24:41.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.522+0000 7f8452ebd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f844c110be0 con 0x7f844c100620 2024-04-25T00:24:41.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.522+0000 7f845145a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f844c106630 0x7f844c19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.522+0000 7f845145a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f844c106630 0x7f844c19ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40728/0 (socket says 172.21.15.107:40728) 2024-04-25T00:24:41.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.522+0000 7f845145a640 1 -- 172.21.15.107:0/1943071637 learned_addr learned my addr 172.21.15.107:0/1943071637 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:41.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.522+0000 7f844bfff640 1 --2- 172.21.15.107:0/1943071637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f844c100fe0 0x7f844c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.522+0000 7f8450c59640 1 --2- 172.21.15.107:0/1943071637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f844c100620 0x7f844c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.522+0000 7f845145a640 1 -- 172.21.15.107:0/1943071637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f844c100620 msgr2=0x7f844c198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:41.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.522+0000 7f845145a640 1 --2- 172.21.15.107:0/1943071637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f844c100620 0x7f844c198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.522+0000 7f845145a640 1 -- 172.21.15.107:0/1943071637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f844c100fe0 msgr2=0x7f844c1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:41.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.522+0000 7f845145a640 1 --2- 172.21.15.107:0/1943071637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f844c100fe0 0x7f844c1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.522+0000 7f845145a640 1 -- 172.21.15.107:0/1943071637 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844c19d180 con 0x7f844c106630 2024-04-25T00:24:41.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.522+0000 7f8450c59640 1 --2- 172.21.15.107:0/1943071637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f844c100620 0x7f844c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:41.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.522+0000 7f845145a640 1 --2- 172.21.15.107:0/1943071637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f844c106630 0x7f844c19ca50 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f844400b970 tx=0x7f844400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:41.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.522+0000 7f8449ffb640 1 -- 172.21.15.107:0/1943071637 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444019070 con 0x7f844c106630 2024-04-25T00:24:41.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.523+0000 7f8449ffb640 1 -- 172.21.15.107:0/1943071637 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f844400cb40 con 0x7f844c106630 2024-04-25T00:24:41.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.523+0000 7f8452ebd640 1 -- 172.21.15.107:0/1943071637 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f844c19d470 con 0x7f844c106630 2024-04-25T00:24:41.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.523+0000 7f8449ffb640 1 -- 172.21.15.107:0/1943071637 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8444007650 con 0x7f844c106630 2024-04-25T00:24:41.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.523+0000 7f8452ebd640 1 -- 172.21.15.107:0/1943071637 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f844c1a4d00 con 0x7f844c106630 2024-04-25T00:24:41.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.525+0000 7f8449ffb640 1 -- 172.21.15.107:0/1943071637 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f844401c070 con 0x7f844c106630 2024-04-25T00:24:41.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.525+0000 7f8455bce740 1 Processor -- start 2024-04-25T00:24:41.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.525+0000 7f8449ffb640 1 --2- 172.21.15.107:0/1943071637 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f843c0a2d00 0x7f843c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.525+0000 7f8455bce740 1 -- start start 2024-04-25T00:24:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.525+0000 7f8450c59640 1 --2- 172.21.15.107:0/1943071637 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f843c0a2d00 0x7f843c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.525+0000 7f8449ffb640 1 -- 172.21.15.107:0/1943071637 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f84440cece0 con 0x7f844c106630 2024-04-25T00:24:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.525+0000 7f8455bce740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c11d357630 0x55c11d357a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.525+0000 7f8455bce740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c11d357f50 0x55c11d3620e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.525+0000 7f8455bce740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d362620 0x55c11d364a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.525+0000 7f8455bce740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c11d2bc6b0 con 0x55c11d362620 2024-04-25T00:24:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.525+0000 7f8455bce740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c11d264880 con 0x55c11d357f50 2024-04-25T00:24:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.525+0000 7f8455bce740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c11d2b8670 con 0x55c11d357630 2024-04-25T00:24:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.526+0000 7f845145a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d362620 0x55c11d364a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.526+0000 7f845145a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d362620 0x55c11d364a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40740/0 (socket says 172.21.15.107:40740) 2024-04-25T00:24:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.526+0000 7f845145a640 1 -- 172.21.15.107:0/1987459124 learned_addr learned my addr 172.21.15.107:0/1987459124 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.526+0000 7f8450c59640 1 --2- 172.21.15.107:0/1987459124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c11d357630 0x55c11d357a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.526+0000 7f844bfff640 1 --2- 172.21.15.107:0/1987459124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c11d357f50 0x55c11d3620e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.526+0000 7f8450c59640 1 --2- 172.21.15.107:0/1943071637 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f843c0a2d00 0x7f843c0a51c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f84400058c0 tx=0x7f844000f730 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.526+0000 7f845145a640 1 -- 172.21.15.107:0/1987459124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c11d357630 msgr2=0x55c11d357a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.526+0000 7f845145a640 1 --2- 172.21.15.107:0/1987459124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c11d357630 0x55c11d357a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.526+0000 7f845145a640 1 -- 172.21.15.107:0/1987459124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c11d357f50 msgr2=0x55c11d3620e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.526+0000 7f845145a640 1 --2- 172.21.15.107:0/1987459124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c11d357f50 0x55c11d3620e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.526+0000 7f845145a640 1 -- 172.21.15.107:0/1987459124 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c11d09a7d0 con 0x55c11d362620 2024-04-25T00:24:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.526+0000 7f8450c59640 1 --2- 172.21.15.107:0/1987459124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c11d357630 0x55c11d357a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.526+0000 7f844bfff640 1 --2- 172.21.15.107:0/1987459124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c11d357f50 0x55c11d3620e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.526+0000 7f845145a640 1 --2- 172.21.15.107:0/1987459124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d362620 0x55c11d364a10 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f8444097460 tx=0x7f844409b0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab44fe5e501e5e0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.526+0000 7f8436ffd640 1 -- 172.21.15.107:0/1987459124 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844400ccf0 con 0x55c11d362620 2024-04-25T00:24:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.526+0000 7f8436ffd640 1 -- 172.21.15.107:0/1987459124 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84440cc2f0 con 0x55c11d362620 2024-04-25T00:24:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.526+0000 7f8436ffd640 1 -- 172.21.15.107:0/1987459124 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84440a3990 con 0x55c11d362620 2024-04-25T00:24:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.527+0000 7f8455bce740 1 -- 172.21.15.107:0/1987459124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d362620 msgr2=0x55c11d364a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.527+0000 7f8455bce740 1 --2- 172.21.15.107:0/1987459124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d362620 0x55c11d364a10 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f8444097460 tx=0x7f844409b0c0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.527+0000 7f8455bce740 1 -- 172.21.15.107:0/1987459124 shutdown_connections 2024-04-25T00:24:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.527+0000 7f8455bce740 1 --2- 172.21.15.107:0/1987459124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d362620 0x55c11d364a10 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.527+0000 7f8455bce740 1 --2- 172.21.15.107:0/1987459124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c11d357f50 0x55c11d3620e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.527+0000 7f8455bce740 1 --2- 172.21.15.107:0/1987459124 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c11d357630 0x55c11d357a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.527+0000 7f8455bce740 1 -- 172.21.15.107:0/1987459124 >> 172.21.15.107:0/1987459124 conn(0x55c11d353950 msgr2=0x55c11d353d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:41.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.527+0000 7f8455bce740 1 -- 172.21.15.107:0/1987459124 shutdown_connections 2024-04-25T00:24:41.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.527+0000 7f8455bce740 1 -- 172.21.15.107:0/1987459124 wait complete. 2024-04-25T00:24:41.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.527+0000 7f8455bce740 1 -- start start 2024-04-25T00:24:41.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.527+0000 7f8455bce740 1 Processor -- start 2024-04-25T00:24:41.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f8455bce740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c11d357630 0x55c11d35f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f8455bce740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c11d357f50 0x55c11d35fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f8455bce740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d362620 0x55c11d35ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f8455bce740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c11d365410 con 0x55c11d362620 2024-04-25T00:24:41.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f8455bce740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c11d277f50 con 0x55c11d357f50 2024-04-25T00:24:41.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f8455bce740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c11d365590 con 0x55c11d357630 2024-04-25T00:24:41.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f8455bce740 10 client.0 _ll_register_callbacks cb 0x55c11d2b3000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:41.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f845145a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d362620 0x55c11d35ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f845145a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d362620 0x55c11d35ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40756/0 (socket says 172.21.15.107:40756) 2024-04-25T00:24:41.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f845145a640 1 -- 172.21.15.107:0/3114369356 learned_addr learned my addr 172.21.15.107:0/3114369356 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:41.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f8450c59640 1 --2- 172.21.15.107:0/3114369356 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c11d357630 0x55c11d35f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f844bfff640 1 --2- 172.21.15.107:0/3114369356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c11d357f50 0x55c11d35fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f845145a640 1 -- 172.21.15.107:0/3114369356 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c11d357630 msgr2=0x55c11d35f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:41.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f845145a640 1 --2- 172.21.15.107:0/3114369356 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c11d357630 0x55c11d35f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f845145a640 1 -- 172.21.15.107:0/3114369356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c11d357f50 msgr2=0x55c11d35fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:41.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f845145a640 1 --2- 172.21.15.107:0/3114369356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c11d357f50 0x55c11d35fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f845145a640 1 -- 172.21.15.107:0/3114369356 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c11cfd03f0 con 0x55c11d362620 2024-04-25T00:24:41.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f845145a640 1 --2- 172.21.15.107:0/3114369356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d362620 0x55c11d35ff60 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f844400b940 tx=0x7f84440bb130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4954a8f204edfdc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:41.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f842bfff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:24:41.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f842bfff640 1 -- 172.21.15.107:0/3114369356 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84440a3730 con 0x55c11d362620 2024-04-25T00:24:41.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f842bfff640 1 -- 172.21.15.107:0/3114369356 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f84440a38d0 con 0x55c11d362620 2024-04-25T00:24:41.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f8455bce740 1 -- 172.21.15.107:0/3114369356 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84440cc2f0 con 0x55c11d362620 2024-04-25T00:24:41.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f842bfff640 1 -- 172.21.15.107:0/3114369356 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f84440b9b90 con 0x55c11d362620 2024-04-25T00:24:41.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f8455bce740 10 client.17514 fetch_fsmap learned FSMap version 34 2024-04-25T00:24:41.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f8455bce740 1 -- 172.21.15.107:0/3114369356 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c11cf900f0 con 0x55c11d362620 2024-04-25T00:24:41.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.528+0000 7f842bfff640 1 -- 172.21.15.107:0/3114369356 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84440cc2f0 con 0x55c11d362620 2024-04-25T00:24:41.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.529+0000 7f842bfff640 1 -- 172.21.15.107:0/3114369356 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f84440b8070 con 0x55c11d362620 2024-04-25T00:24:41.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.529+0000 7f8455bce740 10 client.17514 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:24:41.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.529+0000 7f8455bce740 10 client.17514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:41.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.529+0000 7f8455bce740 20 client.17514 populate_metadata read hostname 'smithi107' 2024-04-25T00:24:41.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.529+0000 7f8455bce740 1 --2- 172.21.15.107:0/3114369356 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c11d372630 0x55c11d374a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.529+0000 7f8455bce740 4 client.17514 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:41.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.529+0000 7f8455bce740 1 -- 172.21.15.107:0/3114369356 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c11d2c9bf0 con 0x55c11d372630 2024-04-25T00:24:41.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.529+0000 7f8450c59640 1 --2- 172.21.15.107:0/3114369356 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c11d372630 0x55c11d374a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.529+0000 7f842bfff640 10 client.17514 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:41.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.529+0000 7f8450c59640 1 --2- 172.21.15.107:0/3114369356 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c11d372630 0x55c11d374a20 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50f89a089eb0399c server_cookie=9b0288061a590fed in_seq=0 out_seq=0 2024-04-25T00:24:41.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.530+0000 7f842bfff640 1 -- 172.21.15.107:0/3114369356 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c11d2c9bf0 con 0x55c11d372630 2024-04-25T00:24:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.530+0000 7f842bfff640 10 client.17514 handle_command_reply: tid=0 2024-04-25T00:24:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.530+0000 7f8455bce740 1 client.17514 shutdown 2024-04-25T00:24:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.530+0000 7f8455bce740 1 -- 172.21.15.107:0/3114369356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d362620 msgr2=0x55c11d35ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.530+0000 7f8455bce740 1 --2- 172.21.15.107:0/3114369356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d362620 0x55c11d35ff60 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f844400b940 tx=0x7f84440bb130 comp rx=0 tx=0).stop 2024-04-25T00:24:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.530+0000 7f8455bce740 1 -- 172.21.15.107:0/3114369356 shutdown_connections 2024-04-25T00:24:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.530+0000 7f8455bce740 1 --2- 172.21.15.107:0/3114369356 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c11d372630 0x55c11d374a20 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.530+0000 7f8455bce740 1 --2- 172.21.15.107:0/3114369356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d362620 0x55c11d35ff60 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.530+0000 7f8455bce740 1 --2- 172.21.15.107:0/3114369356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c11d357f50 0x55c11d35fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.530+0000 7f8455bce740 1 --2- 172.21.15.107:0/3114369356 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c11d357630 0x55c11d35f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.530+0000 7f842bfff640 0 client.17514 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.530+0000 7f8455bce740 1 -- 172.21.15.107:0/3114369356 >> 172.21.15.107:0/3114369356 conn(0x55c11d353950 msgr2=0x55c11d359750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.530+0000 7f8455bce740 1 -- 172.21.15.107:0/3114369356 shutdown_connections 2024-04-25T00:24:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.530+0000 7f8455bce740 1 -- 172.21.15.107:0/3114369356 wait complete. 2024-04-25T00:24:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.530+0000 7f8455bce740 20 client.17514 trim_cache size 0 max 16384 2024-04-25T00:24:41.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.548+0000 7f8455bce740 1 Processor -- start 2024-04-25T00:24:41.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.548+0000 7f8455bce740 1 -- start start 2024-04-25T00:24:41.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.548+0000 7f8455bce740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d357630 0x55c11d352af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.548+0000 7f8455bce740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c11d357f50 0x55c11d35fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.548+0000 7f8455bce740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c11d362620 0x55c11d3595a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.548+0000 7f8455bce740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f844c1b15b0 con 0x55c11d357630 2024-04-25T00:24:41.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.548+0000 7f8455bce740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f844c1aee80 con 0x55c11d362620 2024-04-25T00:24:41.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.548+0000 7f8455bce740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f844c1accf0 con 0x55c11d357f50 2024-04-25T00:24:41.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.548+0000 7f8450c59640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d357630 0x55c11d352af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.548+0000 7f8450c59640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d357630 0x55c11d352af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40766/0 (socket says 172.21.15.107:40766) 2024-04-25T00:24:41.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.548+0000 7f8450c59640 1 -- 172.21.15.107:0/2402314624 learned_addr learned my addr 172.21.15.107:0/2402314624 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:41.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.548+0000 7f8450c59640 1 -- 172.21.15.107:0/2402314624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c11d357f50 msgr2=0x55c11d35fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:41.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.548+0000 7f8450c59640 1 --2- 172.21.15.107:0/2402314624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c11d357f50 0x55c11d35fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.548+0000 7f845145a640 1 --2- 172.21.15.107:0/2402314624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c11d362620 0x55c11d3595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.548+0000 7f8450c59640 1 -- 172.21.15.107:0/2402314624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c11d362620 msgr2=0x55c11d3595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:41.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.548+0000 7f8450c59640 1 --2- 172.21.15.107:0/2402314624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c11d362620 0x55c11d3595a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.548+0000 7f8450c59640 1 -- 172.21.15.107:0/2402314624 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c11d357d80 con 0x55c11d357630 2024-04-25T00:24:41.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.548+0000 7f8450c59640 1 --2- 172.21.15.107:0/2402314624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d357630 0x55c11d352af0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f8440005f00 tx=0x7f8440014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f00f352f2ea81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:41.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.548+0000 7f84367fc640 1 -- 172.21.15.107:0/2402314624 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8440013e10 con 0x55c11d357630 2024-04-25T00:24:41.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.548+0000 7f84367fc640 1 -- 172.21.15.107:0/2402314624 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8440027b50 con 0x55c11d357630 2024-04-25T00:24:41.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.549+0000 7f84367fc640 1 -- 172.21.15.107:0/2402314624 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8440011670 con 0x55c11d357630 2024-04-25T00:24:41.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.549+0000 7f8455bce740 1 -- 172.21.15.107:0/2402314624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d357630 msgr2=0x55c11d352af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:41.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.549+0000 7f8455bce740 1 --2- 172.21.15.107:0/2402314624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d357630 0x55c11d352af0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f8440005f00 tx=0x7f8440014000 comp rx=0 tx=0).stop 2024-04-25T00:24:41.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.549+0000 7f8455bce740 1 -- 172.21.15.107:0/2402314624 shutdown_connections 2024-04-25T00:24:41.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.549+0000 7f8455bce740 1 --2- 172.21.15.107:0/2402314624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c11d362620 0x55c11d3595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.549+0000 7f8455bce740 1 --2- 172.21.15.107:0/2402314624 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c11d357f50 0x55c11d35fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.549+0000 7f8455bce740 1 --2- 172.21.15.107:0/2402314624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d357630 0x55c11d352af0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.549+0000 7f8455bce740 1 -- 172.21.15.107:0/2402314624 >> 172.21.15.107:0/2402314624 conn(0x55c11d353950 msgr2=0x55c11d358e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:41.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.549+0000 7f8455bce740 1 -- 172.21.15.107:0/2402314624 shutdown_connections 2024-04-25T00:24:41.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.549+0000 7f8455bce740 1 -- 172.21.15.107:0/2402314624 wait complete. 2024-04-25T00:24:41.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.549+0000 7f8455bce740 1 -- start start 2024-04-25T00:24:41.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.549+0000 7f8455bce740 1 Processor -- start 2024-04-25T00:24:41.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.550+0000 7f8455bce740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c11d357630 0x55c11d377e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.550+0000 7f8455bce740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d357f50 0x55c11d3752a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.550+0000 7f8455bce740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c11d362620 0x55c11d375800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.550+0000 7f8455bce740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f844c1ad340 con 0x55c11d357f50 2024-04-25T00:24:41.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.550+0000 7f8455bce740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f844c1ae7b0 con 0x55c11d362620 2024-04-25T00:24:41.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.550+0000 7f8455bce740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f844c1ada10 con 0x55c11d357630 2024-04-25T00:24:41.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.550+0000 7f8455bce740 10 client.0 _ll_register_callbacks cb 0x7f84440b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:41.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.550+0000 7f844bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d357f50 0x55c11d3752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.550+0000 7f844bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d357f50 0x55c11d3752a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40774/0 (socket says 172.21.15.107:40774) 2024-04-25T00:24:41.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.550+0000 7f844bfff640 1 -- 172.21.15.107:0/1987081407 learned_addr learned my addr 172.21.15.107:0/1987081407 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:24:41.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.550+0000 7f844bfff640 1 -- 172.21.15.107:0/1987081407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c11d357630 msgr2=0x55c11d377e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:41.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.550+0000 7f844bfff640 1 --2- 172.21.15.107:0/1987081407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c11d357630 0x55c11d377e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.550+0000 7f845145a640 1 --2- 172.21.15.107:0/1987081407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c11d362620 0x55c11d375800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.550+0000 7f844bfff640 1 -- 172.21.15.107:0/1987081407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c11d362620 msgr2=0x55c11d375800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:41.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.550+0000 7f844bfff640 1 --2- 172.21.15.107:0/1987081407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c11d362620 0x55c11d375800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.550+0000 7f844bfff640 1 -- 172.21.15.107:0/1987081407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c11d376080 con 0x55c11d357f50 2024-04-25T00:24:41.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.550+0000 7f845145a640 1 --2- 172.21.15.107:0/1987081407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c11d362620 0x55c11d375800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.550+0000 7f844bfff640 1 --2- 172.21.15.107:0/1987081407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d357f50 0x55c11d3752a0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f8438009930 tx=0x7f8438031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f131462df2cf42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.550+0000 7f842bfff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:24:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.551+0000 7f842bfff640 1 -- 172.21.15.107:0/1987081407 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8438035040 con 0x55c11d357f50 2024-04-25T00:24:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.551+0000 7f842bfff640 1 -- 172.21.15.107:0/1987081407 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9483+0+0 (secure 0 0 0) 0x7f8438004240 con 0x55c11d357f50 2024-04-25T00:24:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.551+0000 7f8455bce740 1 -- 172.21.15.107:0/1987081407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8440027b50 con 0x55c11d357f50 2024-04-25T00:24:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.551+0000 7f842bfff640 1 -- 172.21.15.107:0/1987081407 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f843804f050 con 0x55c11d357f50 2024-04-25T00:24:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.551+0000 7f8455bce740 10 client.17520 fetch_fsmap learned FSMap version 34 2024-04-25T00:24:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.551+0000 7f8455bce740 1 -- 172.21.15.107:0/1987081407 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c11d3dcca0 con 0x55c11d357f50 2024-04-25T00:24:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.551+0000 7f842bfff640 1 -- 172.21.15.107:0/1987081407 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8440027b50 con 0x55c11d357f50 2024-04-25T00:24:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.551+0000 7f842bfff640 1 -- 172.21.15.107:0/1987081407 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f84380466c0 con 0x55c11d357f50 2024-04-25T00:24:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.551+0000 7f8455bce740 10 client.17520 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:24:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.551+0000 7f8455bce740 10 client.17520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.551+0000 7f8455bce740 20 client.17520 populate_metadata read hostname 'smithi107' 2024-04-25T00:24:41.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.551+0000 7f8455bce740 1 --2- 172.21.15.107:0/1987081407 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c11d3dd480 0x55c11d3df8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:41.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.551+0000 7f8455bce740 4 client.17520 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:24:41.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.551+0000 7f8455bce740 1 -- 172.21.15.107:0/1987081407 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c11d2c3f40 con 0x55c11d3dd480 2024-04-25T00:24:41.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.551+0000 7f8450c59640 1 --2- 172.21.15.107:0/1987081407 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c11d3dd480 0x55c11d3df8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:41.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.552+0000 7f842bfff640 10 client.17520 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:41.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.552+0000 7f8450c59640 1 --2- 172.21.15.107:0/1987081407 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c11d3dd480 0x55c11d3df8a0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b509b9484ec50df server_cookie=ccd25d6c5aa0e973 in_seq=0 out_seq=0 2024-04-25T00:24:41.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.552+0000 7f842bfff640 1 -- 172.21.15.107:0/1987081407 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c11d2c3f40 con 0x55c11d3dd480 2024-04-25T00:24:41.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.552+0000 7f842bfff640 10 client.17520 handle_command_reply: tid=0 2024-04-25T00:24:41.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.552+0000 7f8455bce740 1 client.17520 shutdown 2024-04-25T00:24:41.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.552+0000 7f8455bce740 1 -- 172.21.15.107:0/1987081407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d357f50 msgr2=0x55c11d3752a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:41.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.552+0000 7f8455bce740 1 --2- 172.21.15.107:0/1987081407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d357f50 0x55c11d3752a0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f8438009930 tx=0x7f8438031920 comp rx=0 tx=0).stop 2024-04-25T00:24:41.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.552+0000 7f8455bce740 1 -- 172.21.15.107:0/1987081407 shutdown_connections 2024-04-25T00:24:41.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.552+0000 7f8455bce740 1 --2- 172.21.15.107:0/1987081407 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c11d3dd480 0x55c11d3df8a0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.552+0000 7f8455bce740 1 --2- 172.21.15.107:0/1987081407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c11d362620 0x55c11d375800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.552+0000 7f842bfff640 0 client.17520 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:24:41.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.552+0000 7f8455bce740 1 --2- 172.21.15.107:0/1987081407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c11d357f50 0x55c11d3752a0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.552+0000 7f8455bce740 1 --2- 172.21.15.107:0/1987081407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c11d357630 0x55c11d377e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.552+0000 7f8455bce740 1 -- 172.21.15.107:0/1987081407 >> 172.21.15.107:0/1987081407 conn(0x55c11d353950 msgr2=0x55c11d36e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:41.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.552+0000 7f8455bce740 1 -- 172.21.15.107:0/1987081407 shutdown_connections 2024-04-25T00:24:41.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.552+0000 7f8455bce740 1 -- 172.21.15.107:0/1987081407 wait complete. 2024-04-25T00:24:41.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.553+0000 7f8455bce740 20 client.17520 trim_cache size 0 max 16384 2024-04-25T00:24:41.560 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:24:41.560 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:24:41.560 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-04-25T00:24:41.560 INFO:teuthology.orchestra.run.smithi107.stdout: "acdb60e6-62ed-44d5-8c0d-d26810113bc6": { 2024-04-25T00:24:41.560 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-04-25T00:24:41.561 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "acdb60e6-62ed-44d5-8c0d-d26810113bc6", 2024-04-25T00:24:41.561 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-04-25T00:24:41.561 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:24:41.561 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:24:41.561 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:24:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.553+0000 7f8452ebd640 1 -- 172.21.15.107:0/1943071637 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f843c0a2d00 msgr2=0x7f843c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.553+0000 7f8452ebd640 1 --2- 172.21.15.107:0/1943071637 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f843c0a2d00 0x7f843c0a51c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f84400058c0 tx=0x7f844000f730 comp rx=0 tx=0).stop 2024-04-25T00:24:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.553+0000 7f8452ebd640 1 -- 172.21.15.107:0/1943071637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f844c106630 msgr2=0x7f844c19ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.553+0000 7f8452ebd640 1 --2- 172.21.15.107:0/1943071637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f844c106630 0x7f844c19ca50 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f844400b970 tx=0x7f844400be30 comp rx=0 tx=0).stop 2024-04-25T00:24:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.553+0000 7f8452ebd640 1 -- 172.21.15.107:0/1943071637 shutdown_connections 2024-04-25T00:24:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.553+0000 7f8452ebd640 1 --2- 172.21.15.107:0/1943071637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f844c106630 0x7f844c19ca50 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.553+0000 7f8452ebd640 1 --2- 172.21.15.107:0/1943071637 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f843c0a2d00 0x7f843c0a51c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.553+0000 7f8452ebd640 1 --2- 172.21.15.107:0/1943071637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f844c100fe0 0x7f844c1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.553+0000 7f8452ebd640 1 --2- 172.21.15.107:0/1943071637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f844c100620 0x7f844c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.553+0000 7f8452ebd640 1 -- 172.21.15.107:0/1943071637 >> 172.21.15.107:0/1943071637 conn(0x7f844c0776a0 msgr2=0x7f844c06a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.553+0000 7f8452ebd640 1 -- 172.21.15.107:0/1943071637 shutdown_connections 2024-04-25T00:24:41.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:24:41.553+0000 7f8452ebd640 1 -- 172.21.15.107:0/1943071637 wait complete. 2024-04-25T00:24:41.564 INFO:tasks.cephfs.filesystem:scrub status for tag:acdb60e6-62ed-44d5-8c0d-d26810113bc6 - {'path': '/', 'tag': 'acdb60e6-62ed-44d5-8c0d-d26810113bc6', 'options': 'recursive,force'} 2024-04-25T00:24:41.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:41 smithi170 ceph-mon[32807]: pgmap v254: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:24:41.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:41 smithi170 ceph-mon[32807]: scrub queued for path: / 2024-04-25T00:24:41.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:41 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:24:41.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:41 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:24:41.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:41 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:24:41.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:41 smithi107 ceph-mon[29153]: pgmap v254: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:24:41.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:41 smithi107 ceph-mon[29153]: scrub queued for path: / 2024-04-25T00:24:41.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:41 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:24:41.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:41 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:24:41.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:41 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:24:42.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:41 smithi195 ceph-mon[32833]: pgmap v254: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:24:42.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:41 smithi195 ceph-mon[32833]: scrub queued for path: / 2024-04-25T00:24:42.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:41 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:24:42.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:41 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:24:42.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:41 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:24:42.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:42 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:24:43.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:42 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:24:43.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:42 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:24:43.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:43 smithi107 ceph-mon[29153]: pgmap v255: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:24:44.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:43 smithi195 ceph-mon[32833]: pgmap v255: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:24:44.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:43 smithi170 ceph-mon[32807]: pgmap v255: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:24:45.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:45 smithi107 ceph-mon[29153]: pgmap v256: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:24:46.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:45 smithi195 ceph-mon[32833]: pgmap v256: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:24:46.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:45 smithi170 ceph-mon[32807]: pgmap v256: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:24:47.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:47 smithi107 ceph-mon[29153]: pgmap v257: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:24:47.987 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:47 smithi170 ceph-mon[32807]: pgmap v257: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:24:48.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:47 smithi195 ceph-mon[32833]: pgmap v257: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:24:49.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:49 smithi107 ceph-mon[29153]: pgmap v258: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:24:50.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:49 smithi195 ceph-mon[32833]: pgmap v258: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:24:50.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:49 smithi170 ceph-mon[32807]: pgmap v258: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:24:51.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:51 smithi107 ceph-mon[29153]: pgmap v259: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:24:52.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:51 smithi195 ceph-mon[32833]: pgmap v259: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:24:52.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:51 smithi170 ceph-mon[32807]: pgmap v259: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:24:53.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:53 smithi107 ceph-mon[29153]: pgmap v260: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:24:54.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:53 smithi195 ceph-mon[32833]: pgmap v260: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:24:54.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:53 smithi170 ceph-mon[32807]: pgmap v260: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:24:55.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:55 smithi107 ceph-mon[29153]: pgmap v261: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:24:56.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:55 smithi195 ceph-mon[32833]: pgmap v261: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:24:56.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:55 smithi170 ceph-mon[32807]: pgmap v261: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:24:57.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:57 smithi107 ceph-mon[29153]: pgmap v262: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:24:58.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:57 smithi195 ceph-mon[32833]: pgmap v262: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:24:58.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:57 smithi170 ceph-mon[32807]: pgmap v262: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:24:59.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:24:59 smithi107 ceph-mon[29153]: pgmap v263: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:25:00.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:24:59 smithi195 ceph-mon[32833]: pgmap v263: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:25:00.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:24:59 smithi170 ceph-mon[32807]: pgmap v263: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:25:01.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:01 smithi107 ceph-mon[29153]: pgmap v264: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:25:02.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:01 smithi195 ceph-mon[32833]: pgmap v264: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:25:02.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:01 smithi170 ceph-mon[32807]: pgmap v264: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:25:04.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:03 smithi195 ceph-mon[32833]: pgmap v265: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T00:25:04.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:03 smithi170 ceph-mon[32807]: pgmap v265: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T00:25:04.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:03 smithi107 ceph-mon[29153]: pgmap v265: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T00:25:06.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:05 smithi195 ceph-mon[32833]: pgmap v266: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:25:06.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:05 smithi170 ceph-mon[32807]: pgmap v266: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:25:06.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:05 smithi107 ceph-mon[29153]: pgmap v266: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:25:08.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:07 smithi195 ceph-mon[32833]: pgmap v267: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:25:08.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:07 smithi170 ceph-mon[32807]: pgmap v267: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:25:08.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:07 smithi107 ceph-mon[29153]: pgmap v267: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:25:09.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:08 smithi195 ceph-mon[32833]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T00:25:09.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:08 smithi170 ceph-mon[32807]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T00:25:09.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:08 smithi107 ceph-mon[29153]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T00:25:10.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:09 smithi195 ceph-mon[32833]: pgmap v269: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T00:25:10.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:09 smithi170 ceph-mon[32807]: pgmap v269: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T00:25:10.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:09 smithi107 ceph-mon[29153]: pgmap v269: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T00:25:11.566 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:25:11.566 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:25:11.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.735+0000 7faed9f66640 1 -- 172.21.15.107:0/341829883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faecc0a56f0 msgr2=0x7faecc0b7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:11.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.735+0000 7faed9f66640 1 --2- 172.21.15.107:0/341829883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faecc0a56f0 0x7faecc0b7520 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7faec801a910 tx=0x7faec803b4e0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.735+0000 7faed9f66640 1 -- 172.21.15.107:0/341829883 shutdown_connections 2024-04-25T00:25:11.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.735+0000 7faed9f66640 1 --2- 172.21.15.107:0/341829883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faecc0b7a60 0x7faecc0b9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.735+0000 7faed9f66640 1 --2- 172.21.15.107:0/341829883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faecc0a56f0 0x7faecc0b7520 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.735+0000 7faed9f66640 1 --2- 172.21.15.107:0/341829883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faecc0a4dd0 0x7faecc0a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.735+0000 7faed9f66640 1 -- 172.21.15.107:0/341829883 >> 172.21.15.107:0/341829883 conn(0x7faecc01a630 msgr2=0x7faecc01aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:11.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.735+0000 7faed9f66640 1 -- 172.21.15.107:0/341829883 shutdown_connections 2024-04-25T00:25:11.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.735+0000 7faed9f66640 1 -- 172.21.15.107:0/341829883 wait complete. 2024-04-25T00:25:11.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.736+0000 7faed9f66640 1 Processor -- start 2024-04-25T00:25:11.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.739+0000 7faed9f66640 1 -- start start 2024-04-25T00:25:11.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.739+0000 7faed9f66640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faecc0a4dd0 0x7faecc0b7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:11.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.739+0000 7faed9f66640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faecc0a56f0 0x7faecc0b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:11.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.739+0000 7faed9f66640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faecc0b7a60 0x7faecc0b3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:11.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.739+0000 7faed9765640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faecc0b7a60 0x7faecc0b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:11.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.739+0000 7faed9765640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faecc0b7a60 0x7faecc0b3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60794/0 (socket says 172.21.15.107:60794) 2024-04-25T00:25:11.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.739+0000 7faed9765640 1 -- 172.21.15.107:0/2843380650 learned_addr learned my addr 172.21.15.107:0/2843380650 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:11.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.739+0000 7faed3fff640 1 --2- 172.21.15.107:0/2843380650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faecc0a56f0 0x7faecc0b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:11.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.739+0000 7faed8f64640 1 --2- 172.21.15.107:0/2843380650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faecc0a4dd0 0x7faecc0b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:11.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.739+0000 7faed9f66640 1 -- 172.21.15.107:0/2843380650 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faecc0af7a0 con 0x7faecc0b7a60 2024-04-25T00:25:11.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.739+0000 7faed9f66640 1 -- 172.21.15.107:0/2843380650 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faecc0af620 con 0x7faecc0a4dd0 2024-04-25T00:25:11.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.739+0000 7faed9f66640 1 -- 172.21.15.107:0/2843380650 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faecc0af920 con 0x7faecc0a56f0 2024-04-25T00:25:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.741+0000 7faed3fff640 1 -- 172.21.15.107:0/2843380650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faecc0a4dd0 msgr2=0x7faecc0b7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.741+0000 7faed3fff640 1 --2- 172.21.15.107:0/2843380650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faecc0a4dd0 0x7faecc0b7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.741+0000 7faed3fff640 1 -- 172.21.15.107:0/2843380650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faecc0b7a60 msgr2=0x7faecc0b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.741+0000 7faed3fff640 1 --2- 172.21.15.107:0/2843380650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faecc0b7a60 0x7faecc0b3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.741+0000 7faed3fff640 1 -- 172.21.15.107:0/2843380650 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faecc0b3be0 con 0x7faecc0a56f0 2024-04-25T00:25:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.741+0000 7faed3fff640 1 --2- 172.21.15.107:0/2843380650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faecc0a56f0 0x7faecc0b30e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faec801a860 tx=0x7faec8004cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.741+0000 7faed1ffb640 1 -- 172.21.15.107:0/2843380650 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec804f070 con 0x7faecc0a56f0 2024-04-25T00:25:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.741+0000 7faed1ffb640 1 -- 172.21.15.107:0/2843380650 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faec8040c00 con 0x7faecc0a56f0 2024-04-25T00:25:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.741+0000 7faed8f64640 1 --2- 172.21.15.107:0/2843380650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faecc0a4dd0 0x7faecc0b7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:11.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.741+0000 7faed1ffb640 1 -- 172.21.15.107:0/2843380650 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec8046030 con 0x7faecc0a56f0 2024-04-25T00:25:11.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.743+0000 7faed9f66640 1 -- 172.21.15.107:0/2843380650 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faecc0b3e70 con 0x7faecc0a56f0 2024-04-25T00:25:11.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.743+0000 7faed9f66640 1 -- 172.21.15.107:0/2843380650 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faecc147da0 con 0x7faecc0a56f0 2024-04-25T00:25:11.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.747+0000 7faeddfb3740 1 Processor -- start 2024-04-25T00:25:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.747+0000 7faeddfb3740 1 -- start start 2024-04-25T00:25:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.747+0000 7faeddfb3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562271b31800 0x562271b31be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.747+0000 7faeddfb3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562271b32120 0x562271b3c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.748+0000 7faed9765640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562271b31800 0x562271b31be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.748+0000 7faed9765640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562271b31800 0x562271b31be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.107:59406/0 (socket says 172.21.15.107:59406) 2024-04-25T00:25:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.748+0000 7faeddfb3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562271b3c7f0 0x562271b3ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.748+0000 7faeddfb3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562271a966b0 con 0x562271b3c7f0 2024-04-25T00:25:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.748+0000 7faeddfb3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562271a3e880 con 0x562271b31800 2024-04-25T00:25:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.748+0000 7faeddfb3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562271a92670 con 0x562271b32120 2024-04-25T00:25:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.748+0000 7faed9765640 1 -- 172.21.15.107:0/81004193 learned_addr learned my addr 172.21.15.107:0/81004193 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.748+0000 7faed8f64640 1 --2- 172.21.15.107:0/81004193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562271b32120 0x562271b3c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.748+0000 7faed9765640 1 -- 172.21.15.107:0/81004193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562271b32120 msgr2=0x562271b3c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:11.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.748+0000 7faed9765640 1 --2- 172.21.15.107:0/81004193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562271b32120 0x562271b3c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.748+0000 7faed9765640 1 -- 172.21.15.107:0/81004193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562271b3c7f0 msgr2=0x562271b3ebe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:25:11.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.748+0000 7faed9765640 1 --2- 172.21.15.107:0/81004193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562271b3c7f0 0x562271b3ebe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.748+0000 7faed9765640 1 -- 172.21.15.107:0/81004193 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622718747d0 con 0x562271b31800 2024-04-25T00:25:11.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.749+0000 7faed9765640 1 --2- 172.21.15.107:0/81004193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562271b31800 0x562271b31be0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faed4050770 tx=0x7faed4072900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3ef4b9f76e1405d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:11.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.749+0000 7faebeffd640 1 -- 172.21.15.107:0/81004193 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed407b030 con 0x562271b31800 2024-04-25T00:25:11.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.749+0000 7faebeffd640 1 -- 172.21.15.107:0/81004193 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faed4061b80 con 0x562271b31800 2024-04-25T00:25:11.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.749+0000 7faebeffd640 1 -- 172.21.15.107:0/81004193 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed40622f0 con 0x562271b31800 2024-04-25T00:25:11.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.750+0000 7faeddfb3740 1 -- 172.21.15.107:0/81004193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562271b31800 msgr2=0x562271b31be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:11.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.750+0000 7faeddfb3740 1 --2- 172.21.15.107:0/81004193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562271b31800 0x562271b31be0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7faed4050770 tx=0x7faed4072900 comp rx=0 tx=0).stop 2024-04-25T00:25:11.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.750+0000 7faeddfb3740 1 -- 172.21.15.107:0/81004193 shutdown_connections 2024-04-25T00:25:11.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.750+0000 7faeddfb3740 1 --2- 172.21.15.107:0/81004193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562271b3c7f0 0x562271b3ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.750+0000 7faeddfb3740 1 --2- 172.21.15.107:0/81004193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562271b32120 0x562271b3c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.750+0000 7faeddfb3740 1 --2- 172.21.15.107:0/81004193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562271b31800 0x562271b31be0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.750+0000 7faeddfb3740 1 -- 172.21.15.107:0/81004193 >> 172.21.15.107:0/81004193 conn(0x562271b2db20 msgr2=0x562271b2df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:11.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.753+0000 7faeddfb3740 1 -- 172.21.15.107:0/81004193 shutdown_connections 2024-04-25T00:25:11.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.754+0000 7faeddfb3740 1 -- 172.21.15.107:0/81004193 wait complete. 2024-04-25T00:25:11.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.754+0000 7faed1ffb640 1 -- 172.21.15.107:0/2843380650 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7faec8052070 con 0x7faecc0a56f0 2024-04-25T00:25:11.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.754+0000 7faeddfb3740 1 -- start start 2024-04-25T00:25:11.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.754+0000 7faeddfb3740 1 Processor -- start 2024-04-25T00:25:11.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.756+0000 7faeddfb3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562271b31800 0x562271b39780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:11.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.756+0000 7faeddfb3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562271b32120 0x562271b39cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:11.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.757+0000 7faeddfb3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562271b3a200 0x562271b48ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:11.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.757+0000 7faeddfb3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562271b3f5e0 con 0x562271b31800 2024-04-25T00:25:11.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.757+0000 7faeddfb3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562271a51f50 con 0x562271b32120 2024-04-25T00:25:11.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.757+0000 7faeddfb3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562271b3f760 con 0x562271b3a200 2024-04-25T00:25:11.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.757+0000 7faed3fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562271b3a200 0x562271b48ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:11.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.757+0000 7faed3fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562271b3a200 0x562271b48ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:40254/0 (socket says 172.21.15.107:40254) 2024-04-25T00:25:11.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.757+0000 7faed3fff640 1 -- 172.21.15.107:0/3134415584 learned_addr learned my addr 172.21.15.107:0/3134415584 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:11.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.757+0000 7faeddfb3740 10 client.0 _ll_register_callbacks cb 0x562271a8d000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:11.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.757+0000 7faed8f64640 1 --2- 172.21.15.107:0/3134415584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562271b32120 0x562271b39cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:11.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.757+0000 7faed3fff640 1 -- 172.21.15.107:0/3134415584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562271b32120 msgr2=0x562271b39cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:11.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.757+0000 7faed3fff640 1 --2- 172.21.15.107:0/3134415584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562271b32120 0x562271b39cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.757+0000 7faed3fff640 1 -- 172.21.15.107:0/3134415584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562271b31800 msgr2=0x562271b39780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:11.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.757+0000 7faed3fff640 1 --2- 172.21.15.107:0/3134415584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562271b31800 0x562271b39780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.757+0000 7faed3fff640 1 -- 172.21.15.107:0/3134415584 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622717aa3f0 con 0x562271b3a200 2024-04-25T00:25:11.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.758+0000 7faed3fff640 1 --2- 172.21.15.107:0/3134415584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562271b3a200 0x562271b48ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faec80fdd70 tx=0x7faec8106e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cbaebbcb4aa9d3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:11.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.758+0000 7faed1ffb640 1 --2- 172.21.15.107:0/2843380650 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7faec40a2d50 0x7faec40a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:11.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.758+0000 7faed1ffb640 1 -- 172.21.15.107:0/2843380650 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9571+0+0 (secure 0 0 0) 0x7faec8100b30 con 0x7faecc0a56f0 2024-04-25T00:25:11.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.759+0000 7faeb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-04-25T00:25:11.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.759+0000 7faeb7fff640 1 -- 172.21.15.107:0/3134415584 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec8110020 con 0x562271b3a200 2024-04-25T00:25:11.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.759+0000 7faeddfb3740 1 -- 172.21.15.107:0/3134415584 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faed4061b80 con 0x562271b3a200 2024-04-25T00:25:11.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.759+0000 7faeb7fff640 1 -- 172.21.15.107:0/3134415584 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9571+0+0 (secure 0 0 0) 0x7faec8108ce0 con 0x562271b3a200 2024-04-25T00:25:11.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.759+0000 7faed9765640 1 --2- 172.21.15.107:0/2843380650 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7faec40a2d50 0x7faec40a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:11.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.759+0000 7faeb7fff640 1 -- 172.21.15.107:0/3134415584 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7faec80c8e70 con 0x562271b3a200 2024-04-25T00:25:11.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.761+0000 7faeddfb3740 10 client.26422 fetch_fsmap learned FSMap version 34 2024-04-25T00:25:11.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.761+0000 7faeddfb3740 1 -- 172.21.15.107:0/3134415584 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56227176a0f0 con 0x562271b3a200 2024-04-25T00:25:11.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.762+0000 7faed9765640 1 --2- 172.21.15.107:0/2843380650 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7faec40a2d50 0x7faec40a5210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faed406d040 tx=0x7faed4067610 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:11.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.762+0000 7faeb7fff640 1 -- 172.21.15.107:0/3134415584 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faed4061b80 con 0x562271b3a200 2024-04-25T00:25:11.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.762+0000 7faeb7fff640 1 -- 172.21.15.107:0/3134415584 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7faec80fe080 con 0x562271b3a200 2024-04-25T00:25:11.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.762+0000 7faeddfb3740 10 client.26422 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:25:11.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.763+0000 7faeddfb3740 10 client.26422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:11.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.763+0000 7faeddfb3740 20 client.26422 populate_metadata read hostname 'smithi107' 2024-04-25T00:25:11.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.763+0000 7faeddfb3740 1 --2- 172.21.15.107:0/3134415584 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562271b4ceb0 0x562271b4f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:11.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.763+0000 7faeddfb3740 4 client.26422 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:11.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.763+0000 7faeddfb3740 1 -- 172.21.15.107:0/3134415584 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562271aa3bf0 con 0x562271b4ceb0 2024-04-25T00:25:11.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.764+0000 7faed8f64640 1 --2- 172.21.15.107:0/3134415584 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562271b4ceb0 0x562271b4f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:11.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.768+0000 7faed8f64640 1 --2- 172.21.15.107:0/3134415584 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562271b4ceb0 0x562271b4f2d0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=540a56ec15794105 server_cookie=2bffa6a20a2bc142 in_seq=0 out_seq=0 2024-04-25T00:25:11.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.768+0000 7faeb7fff640 10 client.26422 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:11.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.770+0000 7faeb7fff640 1 -- 172.21.15.107:0/3134415584 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562271aa3bf0 con 0x562271b4ceb0 2024-04-25T00:25:11.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.770+0000 7faeb7fff640 10 client.26422 handle_command_reply: tid=0 2024-04-25T00:25:11.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.771+0000 7faeddfb3740 1 client.26422 shutdown 2024-04-25T00:25:11.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.774+0000 7faeddfb3740 1 -- 172.21.15.107:0/3134415584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562271b3a200 msgr2=0x562271b48ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:11.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.774+0000 7faeddfb3740 1 --2- 172.21.15.107:0/3134415584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562271b3a200 0x562271b48ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faec80fdd70 tx=0x7faec8106e10 comp rx=0 tx=0).stop 2024-04-25T00:25:11.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.777+0000 7faeddfb3740 1 -- 172.21.15.107:0/3134415584 shutdown_connections 2024-04-25T00:25:11.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.777+0000 7faeddfb3740 1 --2- 172.21.15.107:0/3134415584 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562271b4ceb0 0x562271b4f2d0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.777+0000 7faeddfb3740 1 --2- 172.21.15.107:0/3134415584 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562271b3a200 0x562271b48ae0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.777+0000 7faeddfb3740 1 --2- 172.21.15.107:0/3134415584 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562271b32120 0x562271b39cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.777+0000 7faeddfb3740 1 --2- 172.21.15.107:0/3134415584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562271b31800 0x562271b39780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.777+0000 7faeddfb3740 1 -- 172.21.15.107:0/3134415584 >> 172.21.15.107:0/3134415584 conn(0x562271b2db20 msgr2=0x562271b338a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:11.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.777+0000 7faeb7fff640 0 client.26422 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:11.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.778+0000 7faeddfb3740 1 -- 172.21.15.107:0/3134415584 shutdown_connections 2024-04-25T00:25:11.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.779+0000 7faeddfb3740 1 -- 172.21.15.107:0/3134415584 wait complete. 2024-04-25T00:25:11.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.779+0000 7faeddfb3740 20 client.26422 trim_cache size 0 max 16384 2024-04-25T00:25:11.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.813+0000 7faeddfb3740 1 Processor -- start 2024-04-25T00:25:11.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.813+0000 7faeddfb3740 1 -- start start 2024-04-25T00:25:11.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.814+0000 7faeddfb3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562271b31800 0x562271b2df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:11.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.814+0000 7faeddfb3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562271b32120 0x562271b54ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:11.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.814+0000 7faeddfb3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562271b3a200 0x562271b550e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:11.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.814+0000 7faeddfb3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faecc154970 con 0x562271b32120 2024-04-25T00:25:11.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.814+0000 7faeddfb3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faecc152240 con 0x562271b3a200 2024-04-25T00:25:11.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.814+0000 7faeddfb3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faecc1500b0 con 0x562271b31800 2024-04-25T00:25:11.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.814+0000 7faed9765640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562271b3a200 0x562271b550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:11.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.814+0000 7faed9765640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562271b3a200 0x562271b550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.107:59432/0 (socket says 172.21.15.107:59432) 2024-04-25T00:25:11.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.814+0000 7faed9765640 1 -- 172.21.15.107:0/139705065 learned_addr learned my addr 172.21.15.107:0/139705065 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:11.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.815+0000 7faed3fff640 1 --2- 172.21.15.107:0/139705065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562271b32120 0x562271b54ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:11.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.815+0000 7faed8f64640 1 --2- 172.21.15.107:0/139705065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562271b31800 0x562271b2df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:11.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.815+0000 7faed3fff640 1 -- 172.21.15.107:0/139705065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562271b31800 msgr2=0x562271b2df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:11.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.815+0000 7faed3fff640 1 --2- 172.21.15.107:0/139705065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562271b31800 0x562271b2df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.815+0000 7faed3fff640 1 -- 172.21.15.107:0/139705065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562271b3a200 msgr2=0x562271b550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:11.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.815+0000 7faed3fff640 1 --2- 172.21.15.107:0/139705065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562271b3a200 0x562271b550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.815+0000 7faed3fff640 1 -- 172.21.15.107:0/139705065 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562271b29470 con 0x562271b32120 2024-04-25T00:25:11.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.816+0000 7faed3fff640 1 --2- 172.21.15.107:0/139705065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562271b32120 0x562271b54ba0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7faec80c95a0 tx=0x7faec8005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e421582701f21b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:11.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.816+0000 7faebe7fc640 1 -- 172.21.15.107:0/139705065 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec8110050 con 0x562271b32120 2024-04-25T00:25:11.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.816+0000 7faebe7fc640 1 -- 172.21.15.107:0/139705065 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faec80d0f80 con 0x562271b32120 2024-04-25T00:25:11.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.816+0000 7faebe7fc640 1 -- 172.21.15.107:0/139705065 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec80cc530 con 0x562271b32120 2024-04-25T00:25:11.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.817+0000 7faeddfb3740 1 -- 172.21.15.107:0/139705065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562271b32120 msgr2=0x562271b54ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:11.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.817+0000 7faeddfb3740 1 --2- 172.21.15.107:0/139705065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562271b32120 0x562271b54ba0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7faec80c95a0 tx=0x7faec8005d50 comp rx=0 tx=0).stop 2024-04-25T00:25:11.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.817+0000 7faeddfb3740 1 -- 172.21.15.107:0/139705065 shutdown_connections 2024-04-25T00:25:11.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.817+0000 7faeddfb3740 1 --2- 172.21.15.107:0/139705065 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562271b3a200 0x562271b550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.817+0000 7faeddfb3740 1 --2- 172.21.15.107:0/139705065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562271b32120 0x562271b54ba0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.817+0000 7faeddfb3740 1 --2- 172.21.15.107:0/139705065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562271b31800 0x562271b2df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.817+0000 7faeddfb3740 1 -- 172.21.15.107:0/139705065 >> 172.21.15.107:0/139705065 conn(0x562271b2db20 msgr2=0x562271b34a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:11.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.819+0000 7faeddfb3740 1 -- 172.21.15.107:0/139705065 shutdown_connections 2024-04-25T00:25:11.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.820+0000 7faeddfb3740 1 -- 172.21.15.107:0/139705065 wait complete. 2024-04-25T00:25:11.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.820+0000 7faeddfb3740 1 -- start start 2024-04-25T00:25:11.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.820+0000 7faeddfb3740 1 Processor -- start 2024-04-25T00:25:11.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.821+0000 7faeddfb3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562271b31800 0x562271bab770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:11.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.821+0000 7faeddfb3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562271b32120 0x562271babcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:11.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.821+0000 7faeddfb3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562271b3a200 0x562271b3fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:11.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.821+0000 7faeddfb3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faecc150700 con 0x562271b32120 2024-04-25T00:25:11.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.822+0000 7faeddfb3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faecc151b70 con 0x562271b31800 2024-04-25T00:25:11.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.822+0000 7faeddfb3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7faecc150dd0 con 0x562271b3a200 2024-04-25T00:25:11.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.822+0000 7faeddfb3740 10 client.0 _ll_register_callbacks cb 0x7faec8106340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:11.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.822+0000 7faed9765640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562271b3a200 0x562271b3fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:11.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.822+0000 7faed9765640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562271b3a200 0x562271b3fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:40278/0 (socket says 172.21.15.107:40278) 2024-04-25T00:25:11.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.822+0000 7faed9765640 1 -- 172.21.15.107:0/253341079 learned_addr learned my addr 172.21.15.107:0/253341079 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:11.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.822+0000 7faed8f64640 1 --2- 172.21.15.107:0/253341079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562271b31800 0x562271bab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:11.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.823+0000 7faed9765640 1 -- 172.21.15.107:0/253341079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562271b31800 msgr2=0x562271bab770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:11.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.823+0000 7faed9765640 1 --2- 172.21.15.107:0/253341079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562271b31800 0x562271bab770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.823+0000 7faed9765640 1 -- 172.21.15.107:0/253341079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562271b32120 msgr2=0x562271babcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:11.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.823+0000 7faed9765640 1 --2- 172.21.15.107:0/253341079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562271b32120 0x562271babcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.823+0000 7faed9765640 1 -- 172.21.15.107:0/253341079 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562271b40500 con 0x562271b3a200 2024-04-25T00:25:11.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.825+0000 7faed9765640 1 --2- 172.21.15.107:0/253341079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562271b3a200 0x562271b3fe10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7faed404f7d0 tx=0x7faed4067370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fe467ce19250f8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:11.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.825+0000 7faeb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-04-25T00:25:11.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.825+0000 7faeb7fff640 1 -- 172.21.15.107:0/253341079 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4086070 con 0x562271b3a200 2024-04-25T00:25:11.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.825+0000 7faeb7fff640 1 -- 172.21.15.107:0/253341079 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9571+0+0 (secure 0 0 0) 0x7faed4080440 con 0x562271b3a200 2024-04-25T00:25:11.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.825+0000 7faeddfb3740 1 -- 172.21.15.107:0/253341079 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faec80d0f80 con 0x562271b3a200 2024-04-25T00:25:11.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.825+0000 7faeb7fff640 1 -- 172.21.15.107:0/253341079 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7faed4080640 con 0x562271b3a200 2024-04-25T00:25:11.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.825+0000 7faeddfb3740 10 client.26428 fetch_fsmap learned FSMap version 34 2024-04-25T00:25:11.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.825+0000 7faeddfb3740 1 -- 172.21.15.107:0/253341079 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562271bb2f20 con 0x562271b3a200 2024-04-25T00:25:11.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.827+0000 7faeb7fff640 1 -- 172.21.15.107:0/253341079 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faec80d0f80 con 0x562271b3a200 2024-04-25T00:25:11.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.827+0000 7faeb7fff640 1 -- 172.21.15.107:0/253341079 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7faed4093070 con 0x562271b3a200 2024-04-25T00:25:11.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.827+0000 7faeddfb3740 10 client.26428 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:25:11.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.827+0000 7faeddfb3740 10 client.26428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:11.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.827+0000 7faeddfb3740 20 client.26428 populate_metadata read hostname 'smithi107' 2024-04-25T00:25:11.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.827+0000 7faeddfb3740 1 --2- 172.21.15.107:0/253341079 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562271bb33c0 0x562271bb57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:11.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.827+0000 7faeddfb3740 4 client.26428 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:25:11.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.827+0000 7faeddfb3740 1 -- 172.21.15.107:0/253341079 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562271a9df40 con 0x562271bb33c0 2024-04-25T00:25:11.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.829+0000 7faed8f64640 1 --2- 172.21.15.107:0/253341079 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562271bb33c0 0x562271bb57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:11.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.830+0000 7faed8f64640 1 --2- 172.21.15.107:0/253341079 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562271bb33c0 0x562271bb57b0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a15591076c199dd server_cookie=6f6da40c50a58ba8 in_seq=0 out_seq=0 2024-04-25T00:25:11.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.830+0000 7faeb7fff640 10 client.26428 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:11.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.832+0000 7faeb7fff640 1 -- 172.21.15.107:0/253341079 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562271a9df40 con 0x562271bb33c0 2024-04-25T00:25:11.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.832+0000 7faeb7fff640 10 client.26428 handle_command_reply: tid=0 2024-04-25T00:25:11.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.833+0000 7faeddfb3740 1 client.26428 shutdown 2024-04-25T00:25:11.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.834+0000 7faeddfb3740 1 -- 172.21.15.107:0/253341079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562271b3a200 msgr2=0x562271b3fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:11.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.834+0000 7faeddfb3740 1 --2- 172.21.15.107:0/253341079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562271b3a200 0x562271b3fe10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7faed404f7d0 tx=0x7faed4067370 comp rx=0 tx=0).stop 2024-04-25T00:25:11.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.835+0000 7faeddfb3740 1 -- 172.21.15.107:0/253341079 shutdown_connections 2024-04-25T00:25:11.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.835+0000 7faeddfb3740 1 --2- 172.21.15.107:0/253341079 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562271bb33c0 0x562271bb57b0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.835+0000 7faeb7fff640 0 client.26428 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:11.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.835+0000 7faeddfb3740 1 --2- 172.21.15.107:0/253341079 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x562271b3a200 0x562271b3fe10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.835+0000 7faeddfb3740 1 --2- 172.21.15.107:0/253341079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562271b32120 0x562271babcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.835+0000 7faeddfb3740 1 --2- 172.21.15.107:0/253341079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562271b31800 0x562271bab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.835+0000 7faeddfb3740 1 -- 172.21.15.107:0/253341079 >> 172.21.15.107:0/253341079 conn(0x562271b2db20 msgr2=0x562271b37f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:11.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.836+0000 7faeddfb3740 1 -- 172.21.15.107:0/253341079 shutdown_connections 2024-04-25T00:25:11.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.837+0000 7faeddfb3740 1 -- 172.21.15.107:0/253341079 wait complete. 2024-04-25T00:25:11.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.839+0000 7faeddfb3740 20 client.26428 trim_cache size 0 max 16384 2024-04-25T00:25:11.840 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:25:11.840 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-04-25T00:25:11.840 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-04-25T00:25:11.840 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:25:11.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.840+0000 7faed9f66640 1 -- 172.21.15.107:0/2843380650 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7faec40a2d50 msgr2=0x7faec40a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:11.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.840+0000 7faed9f66640 1 --2- 172.21.15.107:0/2843380650 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7faec40a2d50 0x7faec40a5210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faed406d040 tx=0x7faed4067610 comp rx=0 tx=0).stop 2024-04-25T00:25:11.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.840+0000 7faed9f66640 1 -- 172.21.15.107:0/2843380650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faecc0a56f0 msgr2=0x7faecc0b30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:11.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.840+0000 7faed9f66640 1 --2- 172.21.15.107:0/2843380650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faecc0a56f0 0x7faecc0b30e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7faec801a860 tx=0x7faec8004cf0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.841+0000 7faed9f66640 1 -- 172.21.15.107:0/2843380650 shutdown_connections 2024-04-25T00:25:11.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.841+0000 7faed9f66640 1 --2- 172.21.15.107:0/2843380650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faecc0b7a60 0x7faecc0b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.841+0000 7faed9f66640 1 --2- 172.21.15.107:0/2843380650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faecc0a56f0 0x7faecc0b30e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.841+0000 7faed9f66640 1 --2- 172.21.15.107:0/2843380650 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7faec40a2d50 0x7faec40a5210 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.841+0000 7faed9f66640 1 --2- 172.21.15.107:0/2843380650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faecc0a4dd0 0x7faecc0b7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:11.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.841+0000 7faed9f66640 1 -- 172.21.15.107:0/2843380650 >> 172.21.15.107:0/2843380650 conn(0x7faecc01a630 msgr2=0x7faecc0a3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:11.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.843+0000 7faed9f66640 1 -- 172.21.15.107:0/2843380650 shutdown_connections 2024-04-25T00:25:11.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:11.843+0000 7faed9f66640 1 -- 172.21.15.107:0/2843380650 wait complete. 2024-04-25T00:25:11.860 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:11 smithi107 ceph-mon[29153]: pgmap v270: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T00:25:11.861 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:25:11.861 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:25:12.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.028+0000 7f4d2dbfb640 1 -- 172.21.15.107:0/2157338785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d28100610 msgr2=0x7f4d28104180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:12.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.028+0000 7f4d2dbfb640 1 --2- 172.21.15.107:0/2157338785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d28100610 0x7f4d28104180 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009900 tx=0x7f4d1802a400 comp rx=0 tx=0).stop 2024-04-25T00:25:12.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.029+0000 7f4d2dbfb640 1 -- 172.21.15.107:0/2157338785 shutdown_connections 2024-04-25T00:25:12.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.029+0000 7f4d2dbfb640 1 --2- 172.21.15.107:0/2157338785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d28100610 0x7f4d28104180 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.029+0000 7f4d2dbfb640 1 --2- 172.21.15.107:0/2157338785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d280ffc50 0x7f4d281000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.029+0000 7f4d2dbfb640 1 --2- 172.21.15.107:0/2157338785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d28106720 0x7f4d28106b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.029+0000 7f4d2dbfb640 1 -- 172.21.15.107:0/2157338785 >> 172.21.15.107:0/2157338785 conn(0x7f4d28076be0 msgr2=0x7f4d280fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:12.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.029+0000 7f4d2dbfb640 1 -- 172.21.15.107:0/2157338785 shutdown_connections 2024-04-25T00:25:12.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.029+0000 7f4d2dbfb640 1 -- 172.21.15.107:0/2157338785 wait complete. 2024-04-25T00:25:12.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.030+0000 7f4d2dbfb640 1 Processor -- start 2024-04-25T00:25:12.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.030+0000 7f4d2dbfb640 1 -- start start 2024-04-25T00:25:12.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.031+0000 7f4d2dbfb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d280ffc50 0x7f4d2819e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.031+0000 7f4d2dbfb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d28100610 0x7f4d2819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.031+0000 7f4d2dbfb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d28106720 0x7f4d281a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.031+0000 7f4d2dbfb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4d281180b0 con 0x7f4d280ffc50 2024-04-25T00:25:12.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.031+0000 7f4d2dbfb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28117f30 con 0x7f4d28100610 2024-04-25T00:25:12.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.031+0000 7f4d2dbfb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28118230 con 0x7f4d28106720 2024-04-25T00:25:12.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.031+0000 7f4d27fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d28106720 0x7f4d281a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.031+0000 7f4d27fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d28106720 0x7f4d281a2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:40300/0 (socket says 172.21.15.107:40300) 2024-04-25T00:25:12.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.031+0000 7f4d27fff640 1 -- 172.21.15.107:0/408310908 learned_addr learned my addr 172.21.15.107:0/408310908 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:12.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.031+0000 7f4d26ffd640 1 --2- 172.21.15.107:0/408310908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d28100610 0x7f4d2819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.031+0000 7f4d27fff640 1 -- 172.21.15.107:0/408310908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d28100610 msgr2=0x7f4d2819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:12.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.031+0000 7f4d27fff640 1 --2- 172.21.15.107:0/408310908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d28100610 0x7f4d2819ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.031+0000 7f4d27fff640 1 -- 172.21.15.107:0/408310908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d280ffc50 msgr2=0x7f4d2819e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:12.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.031+0000 7f4d27fff640 1 --2- 172.21.15.107:0/408310908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d280ffc50 0x7f4d2819e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.031+0000 7f4d27fff640 1 -- 172.21.15.107:0/408310908 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d281a34d0 con 0x7f4d28106720 2024-04-25T00:25:12.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.032+0000 7f4d27fff640 1 --2- 172.21.15.107:0/408310908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d28106720 0x7f4d281a2dd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009850 tx=0x7f4d18003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:12.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.033+0000 7f4d24ff9640 1 -- 172.21.15.107:0/408310908 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1803f070 con 0x7f4d28106720 2024-04-25T00:25:12.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.034+0000 7f4d24ff9640 1 -- 172.21.15.107:0/408310908 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d18031c90 con 0x7f4d28106720 2024-04-25T00:25:12.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.035+0000 7f4d24ff9640 1 -- 172.21.15.107:0/408310908 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18035030 con 0x7f4d28106720 2024-04-25T00:25:12.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.035+0000 7f4d2dbfb640 1 -- 172.21.15.107:0/408310908 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d281a3760 con 0x7f4d28106720 2024-04-25T00:25:12.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.035+0000 7f4d2dbfb640 1 -- 172.21.15.107:0/408310908 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d281a3bd0 con 0x7f4d28106720 2024-04-25T00:25:12.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.037+0000 7f4d2dbfb640 1 -- 172.21.15.107:0/408310908 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d28100c60 con 0x7f4d28106720 2024-04-25T00:25:12.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.040+0000 7f4d24ff9640 1 -- 172.21.15.107:0/408310908 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4d18042070 con 0x7f4d28106720 2024-04-25T00:25:12.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.041+0000 7f4d24ff9640 1 --2- 172.21.15.107:0/408310908 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4d100a2ce0 0x7f4d100a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.041+0000 7f4d24ff9640 1 -- 172.21.15.107:0/408310908 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9571+0+0 (secure 0 0 0) 0x7f4d180f07c0 con 0x7f4d28106720 2024-04-25T00:25:12.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.042+0000 7f4d277fe640 1 --2- 172.21.15.107:0/408310908 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4d100a2ce0 0x7f4d100a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.042+0000 7f4d277fe640 1 --2- 172.21.15.107:0/408310908 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4d100a2ce0 0x7f4d100a51a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4d28078160 tx=0x7f4d200093c0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:12.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.047+0000 7f4d24ff9640 1 -- 172.21.15.107:0/408310908 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4d18049070 con 0x7f4d28106720 2024-04-25T00:25:12.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:11 smithi195 ceph-mon[32833]: pgmap v270: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T00:25:12.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:11 smithi170 ceph-mon[32807]: pgmap v270: 129 pgs: 129 active+clean; 243 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T00:25:12.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.452+0000 7f4d2dbfb640 1 -- 172.21.15.107:0/408310908 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d2819f860 con 0x7f4d28106720 2024-04-25T00:25:12.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.453+0000 7f4d24ff9640 1 -- 172.21.15.107:0/408310908 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11897 (secure 0 0 0) 0x7f4d18030e50 con 0x7f4d28106720 2024-04-25T00:25:12.453 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:25:12.454 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:22:27.402680+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24493,24521]},"id":1}]} 2024-04-25T00:25:12.454 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 34 2024-04-25T00:25:12.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.456+0000 7f4d2dbfb640 1 -- 172.21.15.107:0/408310908 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4d100a2ce0 msgr2=0x7f4d100a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:12.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.456+0000 7f4d2dbfb640 1 --2- 172.21.15.107:0/408310908 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4d100a2ce0 0x7f4d100a51a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4d28078160 tx=0x7f4d200093c0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.456+0000 7f4d2dbfb640 1 -- 172.21.15.107:0/408310908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d28106720 msgr2=0x7f4d281a2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:12.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.456+0000 7f4d2dbfb640 1 --2- 172.21.15.107:0/408310908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d28106720 0x7f4d281a2dd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009850 tx=0x7f4d18003b90 comp rx=0 tx=0).stop 2024-04-25T00:25:12.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.456+0000 7f4d2dbfb640 1 -- 172.21.15.107:0/408310908 shutdown_connections 2024-04-25T00:25:12.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.456+0000 7f4d2dbfb640 1 --2- 172.21.15.107:0/408310908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4d28106720 0x7f4d281a2dd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.456+0000 7f4d2dbfb640 1 --2- 172.21.15.107:0/408310908 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4d100a2ce0 0x7f4d100a51a0 secure :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4d28078160 tx=0x7f4d200093c0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.456+0000 7f4d2dbfb640 1 --2- 172.21.15.107:0/408310908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4d28100610 0x7f4d2819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.456+0000 7f4d2dbfb640 1 --2- 172.21.15.107:0/408310908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d280ffc50 0x7f4d2819e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.456+0000 7f4d2dbfb640 1 -- 172.21.15.107:0/408310908 >> 172.21.15.107:0/408310908 conn(0x7f4d28076be0 msgr2=0x7f4d28101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:12.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.456+0000 7f4d2dbfb640 1 -- 172.21.15.107:0/408310908 shutdown_connections 2024-04-25T00:25:12.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.456+0000 7f4d2dbfb640 1 -- 172.21.15.107:0/408310908 wait complete. 2024-04-25T00:25:12.474 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:25:12.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.626+0000 7fc786c1b640 1 -- 172.21.15.107:0/1759904351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc780100fe0 msgr2=0x7fc78010d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:12.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.626+0000 7fc786c1b640 1 --2- 172.21.15.107:0/1759904351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc780100fe0 0x7fc78010d4f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc778009be0 tx=0x7fc77802a6c0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.626+0000 7fc786c1b640 1 -- 172.21.15.107:0/1759904351 shutdown_connections 2024-04-25T00:25:12.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.626+0000 7fc786c1b640 1 --2- 172.21.15.107:0/1759904351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc780100fe0 0x7fc78010d4f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.626+0000 7fc786c1b640 1 --2- 172.21.15.107:0/1759904351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc780100620 0x7fc780100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.626+0000 7fc786c1b640 1 --2- 172.21.15.107:0/1759904351 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc780106630 0x7fc780106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.626+0000 7fc786c1b640 1 -- 172.21.15.107:0/1759904351 >> 172.21.15.107:0/1759904351 conn(0x7fc7800769c0 msgr2=0x7fc780076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:12.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.626+0000 7fc786c1b640 1 -- 172.21.15.107:0/1759904351 shutdown_connections 2024-04-25T00:25:12.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.627+0000 7fc786c1b640 1 -- 172.21.15.107:0/1759904351 wait complete. 2024-04-25T00:25:12.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.628+0000 7fc786c1b640 1 Processor -- start 2024-04-25T00:25:12.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.628+0000 7fc786c1b640 1 -- start start 2024-04-25T00:25:12.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.628+0000 7fc786c1b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc780100620 0x7fc78010c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.628+0000 7fc786c1b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc780100fe0 0x7fc78010cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.628+0000 7fc786c1b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc780106630 0x7fc7801a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.628+0000 7fc786c1b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc780111b40 con 0x7fc780100620 2024-04-25T00:25:12.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.628+0000 7fc786c1b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc7801119c0 con 0x7fc780100fe0 2024-04-25T00:25:12.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.628+0000 7fc786c1b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc780111cc0 con 0x7fc780106630 2024-04-25T00:25:12.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.628+0000 7fc7849b7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc780100620 0x7fc78010c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.628+0000 7fc7851b8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc780106630 0x7fc7801a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.628+0000 7fc77ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc780100fe0 0x7fc78010cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.628+0000 7fc7851b8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc780106630 0x7fc7801a2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:40322/0 (socket says 172.21.15.107:40322) 2024-04-25T00:25:12.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.628+0000 7fc7851b8640 1 -- 172.21.15.107:0/2380216453 learned_addr learned my addr 172.21.15.107:0/2380216453 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:12.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.629+0000 7fc77ffff640 1 -- 172.21.15.107:0/2380216453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc780106630 msgr2=0x7fc7801a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:12.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.629+0000 7fc77ffff640 1 --2- 172.21.15.107:0/2380216453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc780106630 0x7fc7801a2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.629+0000 7fc77ffff640 1 -- 172.21.15.107:0/2380216453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc780100620 msgr2=0x7fc78010c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:12.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.629+0000 7fc77ffff640 1 --2- 172.21.15.107:0/2380216453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc780100620 0x7fc78010c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.629+0000 7fc77ffff640 1 -- 172.21.15.107:0/2380216453 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7801a3590 con 0x7fc780100fe0 2024-04-25T00:25:12.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.629+0000 7fc77ffff640 1 --2- 172.21.15.107:0/2380216453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc780100fe0 0x7fc78010cd30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc77400e9f0 tx=0x7fc77400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:12.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.630+0000 7fc77dffb640 1 -- 172.21.15.107:0/2380216453 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77400ccc0 con 0x7fc780100fe0 2024-04-25T00:25:12.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.630+0000 7fc77dffb640 1 -- 172.21.15.107:0/2380216453 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc774010910 con 0x7fc780100fe0 2024-04-25T00:25:12.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.630+0000 7fc786c1b640 1 -- 172.21.15.107:0/2380216453 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7801a3880 con 0x7fc780100fe0 2024-04-25T00:25:12.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.630+0000 7fc77dffb640 1 -- 172.21.15.107:0/2380216453 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7740189c0 con 0x7fc780100fe0 2024-04-25T00:25:12.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.630+0000 7fc786c1b640 1 -- 172.21.15.107:0/2380216453 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7801ab110 con 0x7fc780100fe0 2024-04-25T00:25:12.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.633+0000 7fc7899fd740 1 Processor -- start 2024-04-25T00:25:12.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.635+0000 7fc77dffb640 1 -- 172.21.15.107:0/2380216453 <== mon.2 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fc774010ac0 con 0x7fc780100fe0 2024-04-25T00:25:12.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.636+0000 7fc7899fd740 1 -- start start 2024-04-25T00:25:12.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.636+0000 7fc7899fd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f898f6580 0x555f898f6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.636+0000 7fc7899fd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f898f6ea0 0x555f89901030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.636+0000 7fc7899fd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f89901570 0x555f89903960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.636+0000 7fc7899fd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555f8985b670 con 0x555f898f6580 2024-04-25T00:25:12.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.636+0000 7fc7899fd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555f89803970 con 0x555f89901570 2024-04-25T00:25:12.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.636+0000 7fc7899fd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555f89857630 con 0x555f898f6ea0 2024-04-25T00:25:12.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.637+0000 7fc77ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f89901570 0x555f89903960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.637+0000 7fc77ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f89901570 0x555f89903960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.107:59488/0 (socket says 172.21.15.107:59488) 2024-04-25T00:25:12.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.637+0000 7fc77ffff640 1 -- 172.21.15.107:0/4035790936 learned_addr learned my addr 172.21.15.107:0/4035790936 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:12.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.637+0000 7fc7851b8640 1 --2- 172.21.15.107:0/4035790936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f898f6580 0x555f898f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.637+0000 7fc77ffff640 1 -- 172.21.15.107:0/4035790936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f898f6ea0 msgr2=0x555f89901030 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:25:12.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.637+0000 7fc77ffff640 1 --2- 172.21.15.107:0/4035790936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f898f6ea0 0x555f89901030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.637+0000 7fc77ffff640 1 -- 172.21.15.107:0/4035790936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f898f6580 msgr2=0x555f898f6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:12.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.637+0000 7fc77ffff640 1 --2- 172.21.15.107:0/4035790936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f898f6580 0x555f898f6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.637+0000 7fc77ffff640 1 -- 172.21.15.107:0/4035790936 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f89827400 con 0x555f89901570 2024-04-25T00:25:12.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.637+0000 7fc7851b8640 1 --2- 172.21.15.107:0/4035790936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f898f6580 0x555f898f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:12.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.638+0000 7fc77dffb640 1 --2- 172.21.15.107:0/2380216453 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc7700a2ed0 0x7fc7700a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.638+0000 7fc7851b8640 1 --2- 172.21.15.107:0/2380216453 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc7700a2ed0 0x7fc7700a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.638+0000 7fc77dffb640 1 -- 172.21.15.107:0/2380216453 <== mon.2 v2:172.21.15.170:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9571+0+0 (secure 0 0 0) 0x7fc7740cfc00 con 0x7fc780100fe0 2024-04-25T00:25:12.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.638+0000 7fc7851b8640 1 --2- 172.21.15.107:0/2380216453 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc7700a2ed0 0x7fc7700a5390 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc7780099a0 tx=0x7fc778030c00 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:12.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.639+0000 7fc77ffff640 1 --2- 172.21.15.107:0/4035790936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f89901570 0x555f89903960 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc7740cce10 tx=0x7fc7740d8270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d243b1ff6bce3fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:12.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.639+0000 7fc76affd640 1 -- 172.21.15.107:0/4035790936 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc774097f40 con 0x555f89901570 2024-04-25T00:25:12.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.639+0000 7fc76affd640 1 -- 172.21.15.107:0/4035790936 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc77409c3a0 con 0x555f89901570 2024-04-25T00:25:12.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.639+0000 7fc76affd640 1 -- 172.21.15.107:0/4035790936 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77409b600 con 0x555f89901570 2024-04-25T00:25:12.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.640+0000 7fc7899fd740 1 -- 172.21.15.107:0/4035790936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f89901570 msgr2=0x555f89903960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:12.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.640+0000 7fc7899fd740 1 --2- 172.21.15.107:0/4035790936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f89901570 0x555f89903960 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc7740cce10 tx=0x7fc7740d8270 comp rx=0 tx=0).stop 2024-04-25T00:25:12.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.641+0000 7fc7899fd740 1 -- 172.21.15.107:0/4035790936 shutdown_connections 2024-04-25T00:25:12.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.641+0000 7fc7899fd740 1 --2- 172.21.15.107:0/4035790936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f89901570 0x555f89903960 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.641+0000 7fc7899fd740 1 --2- 172.21.15.107:0/4035790936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f898f6ea0 0x555f89901030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.641+0000 7fc7899fd740 1 --2- 172.21.15.107:0/4035790936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f898f6580 0x555f898f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.641+0000 7fc7899fd740 1 -- 172.21.15.107:0/4035790936 >> 172.21.15.107:0/4035790936 conn(0x555f898f28a0 msgr2=0x555f898f2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:12.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.641+0000 7fc7899fd740 1 -- 172.21.15.107:0/4035790936 shutdown_connections 2024-04-25T00:25:12.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.641+0000 7fc7899fd740 1 -- 172.21.15.107:0/4035790936 wait complete. 2024-04-25T00:25:12.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.641+0000 7fc7899fd740 1 -- start start 2024-04-25T00:25:12.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.642+0000 7fc7899fd740 1 Processor -- start 2024-04-25T00:25:12.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.642+0000 7fc7899fd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f898f6580 0x555f89900e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.642+0000 7fc7899fd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f898f6ea0 0x555f898fe1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.642+0000 7fc7899fd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f89901570 0x555f898fe730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.642+0000 7fc7899fd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555f89904300 con 0x555f89901570 2024-04-25T00:25:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.642+0000 7fc7899fd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555f89817050 con 0x555f898f6ea0 2024-04-25T00:25:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.642+0000 7fc7899fd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x555f89904480 con 0x555f898f6580 2024-04-25T00:25:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.642+0000 7fc7899fd740 10 client.0 _ll_register_callbacks cb 0x555f8956ebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.643+0000 7fc77ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f898f6ea0 0x555f898fe1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.643+0000 7fc77ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f898f6ea0 0x555f898fe1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.107:59498/0 (socket says 172.21.15.107:59498) 2024-04-25T00:25:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.643+0000 7fc77ffff640 1 -- 172.21.15.107:0/898484768 learned_addr learned my addr 172.21.15.107:0/898484768 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.643+0000 7fc7851b8640 1 --2- 172.21.15.107:0/898484768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f89901570 0x555f898fe730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.643+0000 7fc77ffff640 1 -- 172.21.15.107:0/898484768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f898f6580 msgr2=0x555f89900e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.643+0000 7fc77ffff640 1 --2- 172.21.15.107:0/898484768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f898f6580 0x555f89900e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.643+0000 7fc77ffff640 1 -- 172.21.15.107:0/898484768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f89901570 msgr2=0x555f898fe730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.643+0000 7fc77ffff640 1 --2- 172.21.15.107:0/898484768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f89901570 0x555f898fe730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.643+0000 7fc77ffff640 1 -- 172.21.15.107:0/898484768 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f897e8dc0 con 0x555f898f6ea0 2024-04-25T00:25:12.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.644+0000 7fc77ffff640 1 --2- 172.21.15.107:0/898484768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f898f6ea0 0x555f898fe1d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc7740d8770 tx=0x7fc7740cfe50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e7c5f5cf4c212c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:12.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.644+0000 7fc75ffff640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-04-25T00:25:12.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.644+0000 7fc75ffff640 1 -- 172.21.15.107:0/898484768 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77409b7e0 con 0x555f898f6ea0 2024-04-25T00:25:12.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.644+0000 7fc7899fd740 1 -- 172.21.15.107:0/898484768 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc77409c3a0 con 0x555f898f6ea0 2024-04-25T00:25:12.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.644+0000 7fc75ffff640 1 -- 172.21.15.107:0/898484768 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9571+0+0 (secure 0 0 0) 0x7fc7740a4480 con 0x555f898f6ea0 2024-04-25T00:25:12.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.644+0000 7fc75ffff640 1 -- 172.21.15.107:0/898484768 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc77409ba70 con 0x555f898f6ea0 2024-04-25T00:25:12.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.644+0000 7fc7899fd740 10 client.26546 fetch_fsmap learned FSMap version 34 2024-04-25T00:25:12.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.644+0000 7fc7899fd740 1 -- 172.21.15.107:0/898484768 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555f8956f3f0 con 0x555f898f6ea0 2024-04-25T00:25:12.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.644+0000 7fc75ffff640 1 -- 172.21.15.107:0/898484768 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc77409c3a0 con 0x555f898f6ea0 2024-04-25T00:25:12.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.645+0000 7fc75ffff640 1 -- 172.21.15.107:0/898484768 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fc7740a4b60 con 0x555f898f6ea0 2024-04-25T00:25:12.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.645+0000 7fc7899fd740 10 client.26546 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:25:12.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.645+0000 7fc7899fd740 10 client.26546 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:25:12.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.645+0000 7fc7899fd740 20 client.26546 populate_metadata read hostname 'smithi107' 2024-04-25T00:25:12.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.645+0000 7fc7899fd740 1 --2- 172.21.15.107:0/898484768 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x555f899113f0 0x555f899137e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.645+0000 7fc7899fd740 4 client.26546 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:12.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.645+0000 7fc7899fd740 1 -- 172.21.15.107:0/898484768 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f89868bb0 con 0x555f899113f0 2024-04-25T00:25:12.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.645+0000 7fc7849b7640 1 --2- 172.21.15.107:0/898484768 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x555f899113f0 0x555f899137e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.646+0000 7fc75ffff640 10 client.26546 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:25:12.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.646+0000 7fc7849b7640 1 --2- 172.21.15.107:0/898484768 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x555f899113f0 0x555f899137e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7eed392d14d69a18 server_cookie=97d9fc1d71086055 in_seq=0 out_seq=0 2024-04-25T00:25:12.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.647+0000 7fc75ffff640 1 -- 172.21.15.107:0/898484768 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f89868bb0 con 0x555f899113f0 2024-04-25T00:25:12.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.648+0000 7fc75ffff640 10 client.26546 handle_command_reply: tid=0 2024-04-25T00:25:12.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.648+0000 7fc7899fd740 1 client.26546 shutdown 2024-04-25T00:25:12.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.648+0000 7fc7899fd740 1 -- 172.21.15.107:0/898484768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f898f6ea0 msgr2=0x555f898fe1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:12.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.648+0000 7fc7899fd740 1 --2- 172.21.15.107:0/898484768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f898f6ea0 0x555f898fe1d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc7740d8770 tx=0x7fc7740cfe50 comp rx=0 tx=0).stop 2024-04-25T00:25:12.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.649+0000 7fc7899fd740 1 -- 172.21.15.107:0/898484768 shutdown_connections 2024-04-25T00:25:12.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.649+0000 7fc7899fd740 1 --2- 172.21.15.107:0/898484768 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x555f899113f0 0x555f899137e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.649+0000 7fc7899fd740 1 --2- 172.21.15.107:0/898484768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f89901570 0x555f898fe730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.649+0000 7fc7899fd740 1 --2- 172.21.15.107:0/898484768 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f898f6ea0 0x555f898fe1d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.649+0000 7fc7899fd740 1 --2- 172.21.15.107:0/898484768 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f898f6580 0x555f89900e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.649+0000 7fc7899fd740 1 -- 172.21.15.107:0/898484768 >> 172.21.15.107:0/898484768 conn(0x555f898f28a0 msgr2=0x555f898f1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:12.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.650+0000 7fc75ffff640 0 client.26546 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:25:12.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.650+0000 7fc7899fd740 1 -- 172.21.15.107:0/898484768 shutdown_connections 2024-04-25T00:25:12.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.650+0000 7fc7899fd740 1 -- 172.21.15.107:0/898484768 wait complete. 2024-04-25T00:25:12.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.650+0000 7fc7899fd740 20 client.26546 trim_cache size 0 max 16384 2024-04-25T00:25:12.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.675+0000 7fc7899fd740 1 Processor -- start 2024-04-25T00:25:12.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.675+0000 7fc7899fd740 1 -- start start 2024-04-25T00:25:12.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.675+0000 7fc7899fd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f898f6580 0x555f899004d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.675+0000 7fc7899fd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f898f6ea0 0x555f898f22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.676+0000 7fc7899fd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f89901570 0x555f89918d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.676+0000 7fc7899fd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc7801b7b70 con 0x555f898f6ea0 2024-04-25T00:25:12.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.676+0000 7fc7899fd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc7801b5440 con 0x555f898f6580 2024-04-25T00:25:12.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.676+0000 7fc7899fd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc7801b32b0 con 0x555f89901570 2024-04-25T00:25:12.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.676+0000 7fc77ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f898f6ea0 0x555f898f22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.676+0000 7fc77ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f898f6ea0 0x555f898f22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60898/0 (socket says 172.21.15.107:60898) 2024-04-25T00:25:12.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.676+0000 7fc77ffff640 1 -- 172.21.15.107:0/2219413989 learned_addr learned my addr 172.21.15.107:0/2219413989 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:12.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.676+0000 7fc7851b8640 1 --2- 172.21.15.107:0/2219413989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f89901570 0x555f89918d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.676+0000 7fc7849b7640 1 --2- 172.21.15.107:0/2219413989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f898f6580 0x555f899004d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.676+0000 7fc77ffff640 1 -- 172.21.15.107:0/2219413989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f89901570 msgr2=0x555f89918d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:12.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.676+0000 7fc77ffff640 1 --2- 172.21.15.107:0/2219413989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f89901570 0x555f89918d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.676+0000 7fc77ffff640 1 -- 172.21.15.107:0/2219413989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f898f6580 msgr2=0x555f899004d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:12.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.676+0000 7fc77ffff640 1 --2- 172.21.15.107:0/2219413989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f898f6580 0x555f899004d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.676+0000 7fc77ffff640 1 -- 172.21.15.107:0/2219413989 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f896397d0 con 0x555f898f6ea0 2024-04-25T00:25:12.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.676+0000 7fc7849b7640 1 --2- 172.21.15.107:0/2219413989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f898f6580 0x555f899004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:12.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.677+0000 7fc77ffff640 1 --2- 172.21.15.107:0/2219413989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f898f6ea0 0x555f898f22c0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fc7740d3ab0 tx=0x7fc77400c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b5286cb810933a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:12.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.677+0000 7fc76a7fc640 1 -- 172.21.15.107:0/2219413989 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7740a4960 con 0x555f898f6ea0 2024-04-25T00:25:12.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.677+0000 7fc76a7fc640 1 -- 172.21.15.107:0/2219413989 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc77409c3a0 con 0x555f898f6ea0 2024-04-25T00:25:12.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.677+0000 7fc76a7fc640 1 -- 172.21.15.107:0/2219413989 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc774097f40 con 0x555f898f6ea0 2024-04-25T00:25:12.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.678+0000 7fc7899fd740 1 -- 172.21.15.107:0/2219413989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f898f6ea0 msgr2=0x555f898f22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:12.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.678+0000 7fc7899fd740 1 --2- 172.21.15.107:0/2219413989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f898f6ea0 0x555f898f22c0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fc7740d3ab0 tx=0x7fc77400c7d0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.678+0000 7fc7899fd740 1 -- 172.21.15.107:0/2219413989 shutdown_connections 2024-04-25T00:25:12.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.678+0000 7fc7899fd740 1 --2- 172.21.15.107:0/2219413989 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f89901570 0x555f89918d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.678+0000 7fc7899fd740 1 --2- 172.21.15.107:0/2219413989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f898f6ea0 0x555f898f22c0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.678+0000 7fc7899fd740 1 --2- 172.21.15.107:0/2219413989 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f898f6580 0x555f899004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.678+0000 7fc7899fd740 1 -- 172.21.15.107:0/2219413989 >> 172.21.15.107:0/2219413989 conn(0x555f898f28a0 msgr2=0x555f898f85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:12.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.679+0000 7fc7899fd740 1 -- 172.21.15.107:0/2219413989 shutdown_connections 2024-04-25T00:25:12.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.679+0000 7fc7899fd740 1 -- 172.21.15.107:0/2219413989 wait complete. 2024-04-25T00:25:12.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.679+0000 7fc7899fd740 1 -- start start 2024-04-25T00:25:12.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.679+0000 7fc7899fd740 1 Processor -- start 2024-04-25T00:25:12.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.680+0000 7fc7899fd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f898f6580 0x555f89909a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.680+0000 7fc7899fd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f898f6ea0 0x555f898fab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.680+0000 7fc7899fd740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f89901570 0x555f898fb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.680+0000 7fc7899fd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc7801b3900 con 0x555f898f6580 2024-04-25T00:25:12.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.680+0000 7fc7899fd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc7801b4d70 con 0x555f898f6ea0 2024-04-25T00:25:12.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.680+0000 7fc7899fd740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fc7801b3fd0 con 0x555f89901570 2024-04-25T00:25:12.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.680+0000 7fc7899fd740 10 client.0 _ll_register_callbacks cb 0x7fc7740d39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:12.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.680+0000 7fc7849b7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f898f6580 0x555f89909a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.680+0000 7fc7849b7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f898f6580 0x555f89909a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60914/0 (socket says 172.21.15.107:60914) 2024-04-25T00:25:12.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.680+0000 7fc7849b7640 1 -- 172.21.15.107:0/1426003073 learned_addr learned my addr 172.21.15.107:0/1426003073 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:12.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.680+0000 7fc77ffff640 1 --2- 172.21.15.107:0/1426003073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f898f6ea0 0x555f898fab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.680+0000 7fc7851b8640 1 --2- 172.21.15.107:0/1426003073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f89901570 0x555f898fb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.681+0000 7fc77ffff640 1 -- 172.21.15.107:0/1426003073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f89901570 msgr2=0x555f898fb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:12.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.681+0000 7fc77ffff640 1 --2- 172.21.15.107:0/1426003073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f89901570 0x555f898fb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.681+0000 7fc77ffff640 1 -- 172.21.15.107:0/1426003073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f898f6580 msgr2=0x555f89909a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:12.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.681+0000 7fc77ffff640 1 --2- 172.21.15.107:0/1426003073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f898f6580 0x555f89909a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.681+0000 7fc77ffff640 1 -- 172.21.15.107:0/1426003073 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f898fb780 con 0x555f898f6ea0 2024-04-25T00:25:12.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.681+0000 7fc77ffff640 1 --2- 172.21.15.107:0/1426003073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f898f6ea0 0x555f898fab00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x555f89656df0 tx=0x7fc7740a4840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58f2790cca0a0698 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:12.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.681+0000 7fc75ffff640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-04-25T00:25:12.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.681+0000 7fc75ffff640 1 -- 172.21.15.107:0/1426003073 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7740a9060 con 0x555f898f6ea0 2024-04-25T00:25:12.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.682+0000 7fc7899fd740 1 -- 172.21.15.107:0/1426003073 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc77409c3a0 con 0x555f898f6ea0 2024-04-25T00:25:12.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.682+0000 7fc75ffff640 1 -- 172.21.15.107:0/1426003073 <== mon.2 v2:172.21.15.170:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9571+0+0 (secure 0 0 0) 0x7fc774018d60 con 0x555f898f6ea0 2024-04-25T00:25:12.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.682+0000 7fc75ffff640 1 -- 172.21.15.107:0/1426003073 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc7740b9070 con 0x555f898f6ea0 2024-04-25T00:25:12.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.682+0000 7fc7899fd740 10 client.26552 fetch_fsmap learned FSMap version 34 2024-04-25T00:25:12.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.682+0000 7fc7899fd740 1 -- 172.21.15.107:0/1426003073 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555f898fb780 con 0x555f898f6ea0 2024-04-25T00:25:12.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.682+0000 7fc75ffff640 1 -- 172.21.15.107:0/1426003073 <== mon.2 v2:172.21.15.170:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc77409c3a0 con 0x555f898f6ea0 2024-04-25T00:25:12.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.683+0000 7fc75ffff640 1 -- 172.21.15.107:0/1426003073 <== mon.2 v2:172.21.15.170:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fc77409b0f0 con 0x555f898f6ea0 2024-04-25T00:25:12.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.683+0000 7fc7899fd740 10 client.26552 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:25:12.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.683+0000 7fc7899fd740 10 client.26552 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:25:12.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.683+0000 7fc7899fd740 20 client.26552 populate_metadata read hostname 'smithi107' 2024-04-25T00:25:12.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.683+0000 7fc7899fd740 1 --2- 172.21.15.107:0/1426003073 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x555f89977810 0x555f89979c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.683+0000 7fc7899fd740 4 client.26552 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:25:12.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.683+0000 7fc7899fd740 1 -- 172.21.15.107:0/1426003073 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f89862f00 con 0x555f89977810 2024-04-25T00:25:12.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.683+0000 7fc7849b7640 1 --2- 172.21.15.107:0/1426003073 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x555f89977810 0x555f89979c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.684+0000 7fc7849b7640 1 --2- 172.21.15.107:0/1426003073 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x555f89977810 0x555f89979c00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e47bee5676cce502 server_cookie=872568d635782607 in_seq=0 out_seq=0 2024-04-25T00:25:12.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.684+0000 7fc75ffff640 10 client.26552 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:25:12.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.684+0000 7fc75ffff640 1 -- 172.21.15.107:0/1426003073 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f89862f00 con 0x555f89977810 2024-04-25T00:25:12.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.684+0000 7fc75ffff640 10 client.26552 handle_command_reply: tid=0 2024-04-25T00:25:12.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.684+0000 7fc7899fd740 1 client.26552 shutdown 2024-04-25T00:25:12.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.685+0000 7fc7899fd740 1 -- 172.21.15.107:0/1426003073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f898f6ea0 msgr2=0x555f898fab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:12.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.685+0000 7fc7899fd740 1 --2- 172.21.15.107:0/1426003073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f898f6ea0 0x555f898fab00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x555f89656df0 tx=0x7fc7740a4840 comp rx=0 tx=0).stop 2024-04-25T00:25:12.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.685+0000 7fc7899fd740 1 -- 172.21.15.107:0/1426003073 shutdown_connections 2024-04-25T00:25:12.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.685+0000 7fc7899fd740 1 --2- 172.21.15.107:0/1426003073 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x555f89977810 0x555f89979c00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.685+0000 7fc7899fd740 1 --2- 172.21.15.107:0/1426003073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x555f89901570 0x555f898fb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.685+0000 7fc7899fd740 1 --2- 172.21.15.107:0/1426003073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555f898f6ea0 0x555f898fab00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.685+0000 7fc7899fd740 1 --2- 172.21.15.107:0/1426003073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f898f6580 0x555f89909a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.685+0000 7fc7899fd740 1 -- 172.21.15.107:0/1426003073 >> 172.21.15.107:0/1426003073 conn(0x555f898f28a0 msgr2=0x555f898f8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:12.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.685+0000 7fc75ffff640 0 client.26552 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:25:12.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.685+0000 7fc7899fd740 1 -- 172.21.15.107:0/1426003073 shutdown_connections 2024-04-25T00:25:12.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.685+0000 7fc7899fd740 1 -- 172.21.15.107:0/1426003073 wait complete. 2024-04-25T00:25:12.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.686+0000 7fc7899fd740 20 client.26552 trim_cache size 0 max 16384 2024-04-25T00:25:12.688 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:25:12.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.687+0000 7fc786c1b640 1 -- 172.21.15.107:0/2380216453 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc7700a2ed0 msgr2=0x7fc7700a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:12.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.687+0000 7fc786c1b640 1 --2- 172.21.15.107:0/2380216453 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc7700a2ed0 0x7fc7700a5390 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc7780099a0 tx=0x7fc778030c00 comp rx=0 tx=0).stop 2024-04-25T00:25:12.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.687+0000 7fc786c1b640 1 -- 172.21.15.107:0/2380216453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc780100fe0 msgr2=0x7fc78010cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:12.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.687+0000 7fc786c1b640 1 --2- 172.21.15.107:0/2380216453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc780100fe0 0x7fc78010cd30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc77400e9f0 tx=0x7fc77400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.687+0000 7fc786c1b640 1 -- 172.21.15.107:0/2380216453 shutdown_connections 2024-04-25T00:25:12.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.687+0000 7fc786c1b640 1 --2- 172.21.15.107:0/2380216453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc780106630 0x7fc7801a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.687+0000 7fc786c1b640 1 --2- 172.21.15.107:0/2380216453 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fc7700a2ed0 0x7fc7700a5390 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.687+0000 7fc786c1b640 1 --2- 172.21.15.107:0/2380216453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc780100fe0 0x7fc78010cd30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.687+0000 7fc786c1b640 1 --2- 172.21.15.107:0/2380216453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc780100620 0x7fc78010c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.687+0000 7fc786c1b640 1 -- 172.21.15.107:0/2380216453 >> 172.21.15.107:0/2380216453 conn(0x7fc7800769c0 msgr2=0x7fc7800fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:12.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.688+0000 7fc786c1b640 1 -- 172.21.15.107:0/2380216453 shutdown_connections 2024-04-25T00:25:12.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.688+0000 7fc786c1b640 1 -- 172.21.15.107:0/2380216453 wait complete. 2024-04-25T00:25:12.704 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:25:12.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.863+0000 7fa553bb7640 1 -- 172.21.15.107:0/63921407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa54c1077a0 msgr2=0x7fa54c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:12.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.863+0000 7fa553bb7640 1 --2- 172.21.15.107:0/63921407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa54c1077a0 0x7fa54c107b80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa548009900 tx=0x7fa54802e820 comp rx=0 tx=0).stop 2024-04-25T00:25:12.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.863+0000 7fa553bb7640 1 -- 172.21.15.107:0/63921407 shutdown_connections 2024-04-25T00:25:12.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.863+0000 7fa553bb7640 1 --2- 172.21.15.107:0/63921407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa54c102150 0x7fa54c10e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.863+0000 7fa553bb7640 1 --2- 172.21.15.107:0/63921407 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa54c101790 0x7fa54c101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.863+0000 7fa553bb7640 1 --2- 172.21.15.107:0/63921407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa54c1077a0 0x7fa54c107b80 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa548009900 tx=0x7fa54802e820 comp rx=0 tx=0).stop 2024-04-25T00:25:12.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.863+0000 7fa553bb7640 1 -- 172.21.15.107:0/63921407 >> 172.21.15.107:0/63921407 conn(0x7fa54c0fd2a0 msgr2=0x7fa54c0ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:12.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.864+0000 7fa553bb7640 1 -- 172.21.15.107:0/63921407 shutdown_connections 2024-04-25T00:25:12.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.864+0000 7fa553bb7640 1 -- 172.21.15.107:0/63921407 wait complete. 2024-04-25T00:25:12.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.865+0000 7fa553bb7640 1 Processor -- start 2024-04-25T00:25:12.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.865+0000 7fa553bb7640 1 -- start start 2024-04-25T00:25:12.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.865+0000 7fa553bb7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa54c101790 0x7fa54c073030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.866+0000 7fa553bb7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa54c102150 0x7fa54c06d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.866+0000 7fa551953640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa54c101790 0x7fa54c073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.866+0000 7fa551953640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa54c101790 0x7fa54c073030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:40370/0 (socket says 172.21.15.107:40370) 2024-04-25T00:25:12.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.866+0000 7fa551953640 1 -- 172.21.15.107:0/3287683555 learned_addr learned my addr 172.21.15.107:0/3287683555 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:12.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.866+0000 7fa553bb7640 1 --2- 172.21.15.107:0/3287683555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa54c06d5e0 0x7fa54c06da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.866+0000 7fa551152640 1 --2- 172.21.15.107:0/3287683555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa54c102150 0x7fa54c06d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.866+0000 7fa553bb7640 1 -- 172.21.15.107:0/3287683555 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c06b8b0 con 0x7fa54c06d5e0 2024-04-25T00:25:12.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.866+0000 7fa553bb7640 1 -- 172.21.15.107:0/3287683555 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c06b730 con 0x7fa54c102150 2024-04-25T00:25:12.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.866+0000 7fa553bb7640 1 -- 172.21.15.107:0/3287683555 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c06ba30 con 0x7fa54c101790 2024-04-25T00:25:12.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.867+0000 7fa551953640 1 -- 172.21.15.107:0/3287683555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa54c102150 msgr2=0x7fa54c06d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:12.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.867+0000 7fa551953640 1 --2- 172.21.15.107:0/3287683555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa54c102150 0x7fa54c06d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.867+0000 7fa551953640 1 -- 172.21.15.107:0/3287683555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa54c06d5e0 msgr2=0x7fa54c06da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:12.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.867+0000 7fa551953640 1 --2- 172.21.15.107:0/3287683555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa54c06d5e0 0x7fa54c06da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.867+0000 7fa551953640 1 -- 172.21.15.107:0/3287683555 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa54c1aafa0 con 0x7fa54c101790 2024-04-25T00:25:12.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.867+0000 7fa551953640 1 --2- 172.21.15.107:0/3287683555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa54c101790 0x7fa54c073030 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa548036450 tx=0x7fa548004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:12.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.868+0000 7fa542ffd640 1 -- 172.21.15.107:0/3287683555 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548045020 con 0x7fa54c101790 2024-04-25T00:25:12.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.868+0000 7fa542ffd640 1 -- 172.21.15.107:0/3287683555 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa548003ad0 con 0x7fa54c101790 2024-04-25T00:25:12.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.868+0000 7fa542ffd640 1 -- 172.21.15.107:0/3287683555 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa548040990 con 0x7fa54c101790 2024-04-25T00:25:12.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.868+0000 7fa553bb7640 1 -- 172.21.15.107:0/3287683555 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa54c1ab230 con 0x7fa54c101790 2024-04-25T00:25:12.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.868+0000 7fa553bb7640 1 -- 172.21.15.107:0/3287683555 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa54c1ab700 con 0x7fa54c101790 2024-04-25T00:25:12.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.873+0000 7fa5569a7740 1 Processor -- start 2024-04-25T00:25:12.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.873+0000 7fa5569a7740 1 -- start start 2024-04-25T00:25:12.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.873+0000 7fa542ffd640 1 -- 172.21.15.107:0/3287683555 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa548048020 con 0x7fa54c101790 2024-04-25T00:25:12.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.873+0000 7fa5569a7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ae20052b0 0x564ae2005690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.873+0000 7fa5569a7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ae2005c60 0x564ae2018840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.873+0000 7fa5569a7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ae2018d80 0x564ae201b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.873+0000 7fa5569a7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564ae1f6a670 con 0x564ae2018d80 2024-04-25T00:25:12.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.873+0000 7fa5569a7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564ae1f12970 con 0x564ae2005c60 2024-04-25T00:25:12.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.873+0000 7fa5569a7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564ae1f66630 con 0x564ae20052b0 2024-04-25T00:25:12.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.874+0000 7fa552154640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ae20052b0 0x564ae2005690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.874+0000 7fa552154640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ae20052b0 0x564ae2005690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:40376/0 (socket says 172.21.15.107:40376) 2024-04-25T00:25:12.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.874+0000 7fa552154640 1 -- 172.21.15.107:0/2704166144 learned_addr learned my addr 172.21.15.107:0/2704166144 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:12.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.874+0000 7fa551953640 1 --2- 172.21.15.107:0/2704166144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ae2005c60 0x564ae2018840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.874+0000 7fa551152640 1 --2- 172.21.15.107:0/2704166144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ae2018d80 0x564ae201b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.874+0000 7fa552154640 1 -- 172.21.15.107:0/2704166144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ae2005c60 msgr2=0x564ae2018840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:12.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.874+0000 7fa552154640 1 --2- 172.21.15.107:0/2704166144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ae2005c60 0x564ae2018840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.874+0000 7fa552154640 1 -- 172.21.15.107:0/2704166144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ae2018d80 msgr2=0x564ae201b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:12.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.874+0000 7fa552154640 1 --2- 172.21.15.107:0/2704166144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ae2018d80 0x564ae201b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.874+0000 7fa552154640 1 -- 172.21.15.107:0/2704166144 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ae1f36400 con 0x564ae20052b0 2024-04-25T00:25:12.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.874+0000 7fa542ffd640 1 --2- 172.21.15.107:0/3287683555 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa5380a2ed0 0x7fa5380a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.874+0000 7fa552154640 1 --2- 172.21.15.107:0/2704166144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ae20052b0 0x564ae2005690 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa544009e50 tx=0x7fa54400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a76732d132d85c51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:12.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.874+0000 7fa542ffd640 1 -- 172.21.15.107:0/3287683555 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9571+0+0 (secure 0 0 0) 0x7fa54800e910 con 0x7fa54c101790 2024-04-25T00:25:12.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.875+0000 7fa551953640 1 --2- 172.21.15.107:0/3287683555 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa5380a2ed0 0x7fa5380a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.875+0000 7fa537fff640 1 -- 172.21.15.107:0/2704166144 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544004b20 con 0x564ae20052b0 2024-04-25T00:25:12.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.875+0000 7fa537fff640 1 -- 172.21.15.107:0/2704166144 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa544004cc0 con 0x564ae20052b0 2024-04-25T00:25:12.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.875+0000 7fa537fff640 1 -- 172.21.15.107:0/2704166144 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544010990 con 0x564ae20052b0 2024-04-25T00:25:12.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.876+0000 7fa5569a7740 1 -- 172.21.15.107:0/2704166144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ae20052b0 msgr2=0x564ae2005690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:12.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.876+0000 7fa5569a7740 1 --2- 172.21.15.107:0/2704166144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ae20052b0 0x564ae2005690 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa544009e50 tx=0x7fa54400e720 comp rx=0 tx=0).stop 2024-04-25T00:25:12.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.876+0000 7fa5569a7740 1 -- 172.21.15.107:0/2704166144 shutdown_connections 2024-04-25T00:25:12.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.876+0000 7fa5569a7740 1 --2- 172.21.15.107:0/2704166144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ae2018d80 0x564ae201b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.876+0000 7fa5569a7740 1 --2- 172.21.15.107:0/2704166144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ae2005c60 0x564ae2018840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.876+0000 7fa5569a7740 1 --2- 172.21.15.107:0/2704166144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ae20052b0 0x564ae2005690 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.876+0000 7fa5569a7740 1 -- 172.21.15.107:0/2704166144 >> 172.21.15.107:0/2704166144 conn(0x564ae20018a0 msgr2=0x564ae2001c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:12.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.876+0000 7fa551953640 1 --2- 172.21.15.107:0/3287683555 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa5380a2ed0 0x7fa5380a5390 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x564ae2007eb0 tx=0x7fa548030040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:12.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.877+0000 7fa5569a7740 1 -- 172.21.15.107:0/2704166144 shutdown_connections 2024-04-25T00:25:12.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.877+0000 7fa5569a7740 1 -- 172.21.15.107:0/2704166144 wait complete. 2024-04-25T00:25:12.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.877+0000 7fa5569a7740 1 -- start start 2024-04-25T00:25:12.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.877+0000 7fa5569a7740 1 Processor -- start 2024-04-25T00:25:12.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.879+0000 7fa5569a7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ae20052b0 0x564ae2018780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.879+0000 7fa5569a7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ae2005c60 0x564ae2015a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.879+0000 7fa5569a7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ae2018d80 0x564ae2016080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.879+0000 7fa5569a7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564ae201bb40 con 0x564ae2018d80 2024-04-25T00:25:12.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.879+0000 7fa5569a7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564ae1f26050 con 0x564ae2005c60 2024-04-25T00:25:12.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.879+0000 7fa5569a7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x564ae201bcc0 con 0x564ae20052b0 2024-04-25T00:25:12.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.879+0000 7fa5569a7740 10 client.0 _ll_register_callbacks cb 0x564ae1c7dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:12.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.879+0000 7fa552154640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ae2018d80 0x564ae2016080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.879+0000 7fa552154640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ae2018d80 0x564ae2016080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60960/0 (socket says 172.21.15.107:60960) 2024-04-25T00:25:12.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.879+0000 7fa552154640 1 -- 172.21.15.107:0/1948188675 learned_addr learned my addr 172.21.15.107:0/1948188675 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:12.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.879+0000 7fa552154640 1 --2- 172.21.15.107:0/1948188675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ae20052b0 0x564ae2018780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.879+0000 7fa551152640 1 --2- 172.21.15.107:0/1948188675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ae2005c60 0x564ae2015a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.880+0000 7fa552154640 1 -- 172.21.15.107:0/1948188675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ae20052b0 msgr2=0x564ae2018780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:12.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.880+0000 7fa552154640 1 --2- 172.21.15.107:0/1948188675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ae20052b0 0x564ae2018780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.880+0000 7fa552154640 1 -- 172.21.15.107:0/1948188675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ae2005c60 msgr2=0x564ae2015a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:12.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.880+0000 7fa552154640 1 --2- 172.21.15.107:0/1948188675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ae2005c60 0x564ae2015a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.880+0000 7fa552154640 1 -- 172.21.15.107:0/1948188675 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ae1ef7dc0 con 0x564ae2018d80 2024-04-25T00:25:12.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.880+0000 7fa552154640 1 --2- 172.21.15.107:0/1948188675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ae2018d80 0x564ae2016080 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fa544004f90 tx=0x7fa544009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23dca9ac48cf7e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:12.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.881+0000 7fa5357fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:25:12.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.881+0000 7fa5357fa640 1 -- 172.21.15.107:0/1948188675 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54402c060 con 0x564ae2018d80 2024-04-25T00:25:12.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.882+0000 7fa5569a7740 1 -- 172.21.15.107:0/1948188675 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa544004cc0 con 0x564ae2018d80 2024-04-25T00:25:12.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.882+0000 7fa5357fa640 1 -- 172.21.15.107:0/1948188675 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9571+0+0 (secure 0 0 0) 0x7fa544005000 con 0x564ae2018d80 2024-04-25T00:25:12.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.882+0000 7fa5357fa640 1 -- 172.21.15.107:0/1948188675 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa544033050 con 0x564ae2018d80 2024-04-25T00:25:12.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.882+0000 7fa5569a7740 10 client.17556 fetch_fsmap learned FSMap version 34 2024-04-25T00:25:12.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.882+0000 7fa5569a7740 1 -- 172.21.15.107:0/1948188675 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564ae1c7e3f0 con 0x564ae2018d80 2024-04-25T00:25:12.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.882+0000 7fa5357fa640 1 -- 172.21.15.107:0/1948188675 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa544004cc0 con 0x564ae2018d80 2024-04-25T00:25:12.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.882+0000 7fa5357fa640 1 -- 172.21.15.107:0/1948188675 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fa544018a00 con 0x564ae2018d80 2024-04-25T00:25:12.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.882+0000 7fa5569a7740 10 client.17556 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:25:12.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.882+0000 7fa5569a7740 10 client.17556 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:25:12.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.883+0000 7fa5569a7740 20 client.17556 populate_metadata read hostname 'smithi107' 2024-04-25T00:25:12.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.883+0000 7fa5569a7740 1 --2- 172.21.15.107:0/1948188675 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x564ae2028e60 0x564ae202b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.883+0000 7fa5569a7740 4 client.17556 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:12.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.883+0000 7fa5569a7740 1 -- 172.21.15.107:0/1948188675 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ae1f77bb0 con 0x564ae2028e60 2024-04-25T00:25:12.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.883+0000 7fa551152640 1 --2- 172.21.15.107:0/1948188675 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x564ae2028e60 0x564ae202b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.883+0000 7fa551152640 1 --2- 172.21.15.107:0/1948188675 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x564ae2028e60 0x564ae202b250 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fd94d10573fe1256 server_cookie=95cfff013d9fe026 in_seq=0 out_seq=0 2024-04-25T00:25:12.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.883+0000 7fa5357fa640 10 client.17556 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:25:12.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.884+0000 7fa5357fa640 1 -- 172.21.15.107:0/1948188675 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564ae1f77bb0 con 0x564ae2028e60 2024-04-25T00:25:12.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.884+0000 7fa5357fa640 10 client.17556 handle_command_reply: tid=0 2024-04-25T00:25:12.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.884+0000 7fa5569a7740 1 client.17556 shutdown 2024-04-25T00:25:12.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.885+0000 7fa5569a7740 1 -- 172.21.15.107:0/1948188675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ae2018d80 msgr2=0x564ae2016080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:12.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.885+0000 7fa5569a7740 1 --2- 172.21.15.107:0/1948188675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ae2018d80 0x564ae2016080 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fa544004f90 tx=0x7fa544009230 comp rx=0 tx=0).stop 2024-04-25T00:25:12.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.885+0000 7fa5569a7740 1 -- 172.21.15.107:0/1948188675 shutdown_connections 2024-04-25T00:25:12.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.885+0000 7fa5569a7740 1 --2- 172.21.15.107:0/1948188675 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x564ae2028e60 0x564ae202b250 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.885+0000 7fa5569a7740 1 --2- 172.21.15.107:0/1948188675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ae2018d80 0x564ae2016080 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.885+0000 7fa5569a7740 1 --2- 172.21.15.107:0/1948188675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ae2005c60 0x564ae2015a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.885+0000 7fa5569a7740 1 --2- 172.21.15.107:0/1948188675 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ae20052b0 0x564ae2018780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.885+0000 7fa5569a7740 1 -- 172.21.15.107:0/1948188675 >> 172.21.15.107:0/1948188675 conn(0x564ae20018a0 msgr2=0x564ae2007710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:12.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.886+0000 7fa5357fa640 0 client.17556 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:25:12.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.886+0000 7fa5569a7740 1 -- 172.21.15.107:0/1948188675 shutdown_connections 2024-04-25T00:25:12.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.886+0000 7fa5569a7740 1 -- 172.21.15.107:0/1948188675 wait complete. 2024-04-25T00:25:12.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.887+0000 7fa5569a7740 20 client.17556 trim_cache size 0 max 16384 2024-04-25T00:25:12.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.912+0000 7fa5569a7740 1 Processor -- start 2024-04-25T00:25:12.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.912+0000 7fa5569a7740 1 -- start start 2024-04-25T00:25:12.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.912+0000 7fa5569a7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ae20052b0 0x564ae2007950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.912+0000 7fa5569a7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ae2005c60 0x564ae20182b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.912+0000 7fa5569a7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ae2018d80 0x564ae202de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.912+0000 7fa5569a7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c1b82d0 con 0x564ae2018d80 2024-04-25T00:25:12.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.912+0000 7fa5569a7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c1b5ba0 con 0x564ae2005c60 2024-04-25T00:25:12.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.912+0000 7fa5569a7740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c1b3a10 con 0x564ae20052b0 2024-04-25T00:25:12.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.913+0000 7fa552154640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ae20052b0 0x564ae2007950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.913+0000 7fa552154640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ae20052b0 0x564ae2007950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:40402/0 (socket says 172.21.15.107:40402) 2024-04-25T00:25:12.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.913+0000 7fa552154640 1 -- 172.21.15.107:0/2731362780 learned_addr learned my addr 172.21.15.107:0/2731362780 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:12.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.913+0000 7fa552154640 1 --2- 172.21.15.107:0/2731362780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ae2018d80 0x564ae202de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.913+0000 7fa551152640 1 --2- 172.21.15.107:0/2731362780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ae2005c60 0x564ae20182b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.913+0000 7fa552154640 1 -- 172.21.15.107:0/2731362780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ae2005c60 msgr2=0x564ae20182b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:12.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.913+0000 7fa552154640 1 --2- 172.21.15.107:0/2731362780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ae2005c60 0x564ae20182b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.913+0000 7fa552154640 1 -- 172.21.15.107:0/2731362780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ae2018d80 msgr2=0x564ae202de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:12.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.913+0000 7fa552154640 1 --2- 172.21.15.107:0/2731362780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ae2018d80 0x564ae202de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.913+0000 7fa552154640 1 -- 172.21.15.107:0/2731362780 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ae1d487d0 con 0x564ae20052b0 2024-04-25T00:25:12.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.913+0000 7fa552154640 1 --2- 172.21.15.107:0/2731362780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ae20052b0 0x564ae2007950 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa544004a80 tx=0x7fa54400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1f474438847e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:12.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.913+0000 7fa5377fe640 1 -- 172.21.15.107:0/2731362780 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54402c070 con 0x564ae20052b0 2024-04-25T00:25:12.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.913+0000 7fa5377fe640 1 -- 172.21.15.107:0/2731362780 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa54401e040 con 0x564ae20052b0 2024-04-25T00:25:12.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.913+0000 7fa5377fe640 1 -- 172.21.15.107:0/2731362780 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544027770 con 0x564ae20052b0 2024-04-25T00:25:12.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.914+0000 7fa5569a7740 1 -- 172.21.15.107:0/2731362780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ae20052b0 msgr2=0x564ae2007950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:12.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.914+0000 7fa5569a7740 1 --2- 172.21.15.107:0/2731362780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ae20052b0 0x564ae2007950 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa544004a80 tx=0x7fa54400e720 comp rx=0 tx=0).stop 2024-04-25T00:25:12.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.915+0000 7fa5569a7740 1 -- 172.21.15.107:0/2731362780 shutdown_connections 2024-04-25T00:25:12.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.915+0000 7fa5569a7740 1 --2- 172.21.15.107:0/2731362780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ae2018d80 0x564ae202de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.915+0000 7fa5569a7740 1 --2- 172.21.15.107:0/2731362780 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ae2005c60 0x564ae20182b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.915+0000 7fa5569a7740 1 --2- 172.21.15.107:0/2731362780 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ae20052b0 0x564ae2007950 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.915+0000 7fa5569a7740 1 -- 172.21.15.107:0/2731362780 >> 172.21.15.107:0/2731362780 conn(0x564ae20018a0 msgr2=0x564ae2011070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:12.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.916+0000 7fa5569a7740 1 -- 172.21.15.107:0/2731362780 shutdown_connections 2024-04-25T00:25:12.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.916+0000 7fa5569a7740 1 -- 172.21.15.107:0/2731362780 wait complete. 2024-04-25T00:25:12.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.917+0000 7fa5569a7740 1 -- start start 2024-04-25T00:25:12.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.917+0000 7fa5569a7740 1 Processor -- start 2024-04-25T00:25:12.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.917+0000 7fa5569a7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ae20052b0 0x564ae202afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.918+0000 7fa5569a7740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ae2005c60 0x564ae202b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.918+0000 7fa552154640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ae20052b0 0x564ae202afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.918+0000 7fa552154640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ae20052b0 0x564ae202afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60980/0 (socket says 172.21.15.107:60980) 2024-04-25T00:25:12.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.918+0000 7fa552154640 1 -- 172.21.15.107:0/4084967058 learned_addr learned my addr 172.21.15.107:0/4084967058 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:12.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.918+0000 7fa5569a7740 1 --2- 172.21.15.107:0/4084967058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ae2018d80 0x564ae208d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.918+0000 7fa5569a7740 1 -- 172.21.15.107:0/4084967058 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c1b4060 con 0x564ae20052b0 2024-04-25T00:25:12.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.918+0000 7fa5569a7740 1 -- 172.21.15.107:0/4084967058 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c1b54d0 con 0x564ae2018d80 2024-04-25T00:25:12.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.918+0000 7fa5569a7740 1 -- 172.21.15.107:0/4084967058 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa54c1b4730 con 0x564ae2005c60 2024-04-25T00:25:12.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.918+0000 7fa5569a7740 10 client.0 _ll_register_callbacks cb 0x7fa544009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:12.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.918+0000 7fa551152640 1 --2- 172.21.15.107:0/4084967058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ae2005c60 0x564ae202b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.918+0000 7fa552154640 1 --2- 172.21.15.107:0/4084967058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ae2018d80 0x564ae208d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.919+0000 7fa552154640 1 -- 172.21.15.107:0/4084967058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ae2005c60 msgr2=0x564ae202b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:12.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.919+0000 7fa552154640 1 --2- 172.21.15.107:0/4084967058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ae2005c60 0x564ae202b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.919+0000 7fa552154640 1 -- 172.21.15.107:0/4084967058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ae2018d80 msgr2=0x564ae208d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:12.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.919+0000 7fa552154640 1 --2- 172.21.15.107:0/4084967058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ae2018d80 0x564ae208d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.919+0000 7fa552154640 1 -- 172.21.15.107:0/4084967058 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ae208d570 con 0x564ae20052b0 2024-04-25T00:25:12.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.919+0000 7fa552154640 1 --2- 172.21.15.107:0/4084967058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ae20052b0 0x564ae202afa0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fa544009660 tx=0x7fa544005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14270461cae4e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:12.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.919+0000 7fa5357fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:25:12.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.919+0000 7fa5357fa640 1 -- 172.21.15.107:0/4084967058 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54402c070 con 0x564ae20052b0 2024-04-25T00:25:12.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.919+0000 7fa5357fa640 1 -- 172.21.15.107:0/4084967058 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9571+0+0 (secure 0 0 0) 0x7fa544003b70 con 0x564ae20052b0 2024-04-25T00:25:12.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.920+0000 7fa5569a7740 1 -- 172.21.15.107:0/4084967058 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa54401e040 con 0x564ae20052b0 2024-04-25T00:25:12.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.920+0000 7fa5357fa640 1 -- 172.21.15.107:0/4084967058 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa544011a30 con 0x564ae20052b0 2024-04-25T00:25:12.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.920+0000 7fa5569a7740 10 client.17562 fetch_fsmap learned FSMap version 34 2024-04-25T00:25:12.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.920+0000 7fa5569a7740 1 -- 172.21.15.107:0/4084967058 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564ae208e810 con 0x564ae20052b0 2024-04-25T00:25:12.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.921+0000 7fa5357fa640 1 -- 172.21.15.107:0/4084967058 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa544011540 con 0x564ae20052b0 2024-04-25T00:25:12.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.921+0000 7fa5357fa640 1 -- 172.21.15.107:0/4084967058 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fa544036040 con 0x564ae20052b0 2024-04-25T00:25:12.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.921+0000 7fa5569a7740 10 client.17562 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:25:12.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.921+0000 7fa5569a7740 10 client.17562 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:25:12.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.921+0000 7fa5569a7740 20 client.17562 populate_metadata read hostname 'smithi107' 2024-04-25T00:25:12.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.921+0000 7fa5569a7740 1 --2- 172.21.15.107:0/4084967058 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x564ae208ee80 0x564ae2091270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:12.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.921+0000 7fa5569a7740 4 client.17562 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:25:12.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.921+0000 7fa5569a7740 1 -- 172.21.15.107:0/4084967058 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa54401e040 con 0x564ae208ee80 2024-04-25T00:25:12.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.922+0000 7fa551152640 1 --2- 172.21.15.107:0/4084967058 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x564ae208ee80 0x564ae2091270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:12.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.923+0000 7fa551152640 1 --2- 172.21.15.107:0/4084967058 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x564ae208ee80 0x564ae2091270 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc067d742c16b5d4 server_cookie=36b1c36ac662f955 in_seq=0 out_seq=0 2024-04-25T00:25:12.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.923+0000 7fa5357fa640 10 client.17562 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:25:12.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.923+0000 7fa5357fa640 1 -- 172.21.15.107:0/4084967058 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa54401e040 con 0x564ae208ee80 2024-04-25T00:25:12.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.923+0000 7fa5357fa640 10 client.17562 handle_command_reply: tid=0 2024-04-25T00:25:12.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.923+0000 7fa5569a7740 1 client.17562 shutdown 2024-04-25T00:25:12.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.924+0000 7fa5569a7740 1 -- 172.21.15.107:0/4084967058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ae20052b0 msgr2=0x564ae202afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:12.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.924+0000 7fa5569a7740 1 --2- 172.21.15.107:0/4084967058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ae20052b0 0x564ae202afa0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fa544009660 tx=0x7fa544005cf0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.924+0000 7fa5569a7740 1 -- 172.21.15.107:0/4084967058 shutdown_connections 2024-04-25T00:25:12.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.924+0000 7fa5569a7740 1 --2- 172.21.15.107:0/4084967058 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564ae2018d80 0x564ae208d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.924+0000 7fa5569a7740 1 --2- 172.21.15.107:0/4084967058 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x564ae2005c60 0x564ae202b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.924+0000 7fa5569a7740 1 --2- 172.21.15.107:0/4084967058 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x564ae208ee80 0x564ae2091270 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.924+0000 7fa5569a7740 1 --2- 172.21.15.107:0/4084967058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564ae20052b0 0x564ae202afa0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.924+0000 7fa5569a7740 1 -- 172.21.15.107:0/4084967058 >> 172.21.15.107:0/4084967058 conn(0x564ae20018a0 msgr2=0x564ae2023f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:12.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.924+0000 7fa5357fa640 0 client.17562 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:25:12.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.924+0000 7fa5569a7740 1 -- 172.21.15.107:0/4084967058 shutdown_connections 2024-04-25T00:25:12.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.924+0000 7fa5569a7740 1 -- 172.21.15.107:0/4084967058 wait complete. 2024-04-25T00:25:12.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.924+0000 7fa5569a7740 20 client.17562 trim_cache size 0 max 16384 2024-04-25T00:25:12.926 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:25:12.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.925+0000 7fa553bb7640 1 -- 172.21.15.107:0/3287683555 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa5380a2ed0 msgr2=0x7fa5380a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:12.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.925+0000 7fa553bb7640 1 --2- 172.21.15.107:0/3287683555 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa5380a2ed0 0x7fa5380a5390 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x564ae2007eb0 tx=0x7fa548030040 comp rx=0 tx=0).stop 2024-04-25T00:25:12.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.925+0000 7fa553bb7640 1 -- 172.21.15.107:0/3287683555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa54c101790 msgr2=0x7fa54c073030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:12.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.925+0000 7fa553bb7640 1 --2- 172.21.15.107:0/3287683555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa54c101790 0x7fa54c073030 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa548036450 tx=0x7fa548004270 comp rx=0 tx=0).stop 2024-04-25T00:25:12.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.925+0000 7fa553bb7640 1 -- 172.21.15.107:0/3287683555 shutdown_connections 2024-04-25T00:25:12.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.925+0000 7fa553bb7640 1 --2- 172.21.15.107:0/3287683555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa54c06d5e0 0x7fa54c06da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.925+0000 7fa553bb7640 1 --2- 172.21.15.107:0/3287683555 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa5380a2ed0 0x7fa5380a5390 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.925+0000 7fa553bb7640 1 --2- 172.21.15.107:0/3287683555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa54c102150 0x7fa54c06d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.925+0000 7fa553bb7640 1 --2- 172.21.15.107:0/3287683555 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa54c101790 0x7fa54c073030 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:12.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.925+0000 7fa553bb7640 1 -- 172.21.15.107:0/3287683555 >> 172.21.15.107:0/3287683555 conn(0x7fa54c0fd2a0 msgr2=0x7fa54c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:12.927 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.926+0000 7fa553bb7640 1 -- 172.21.15.107:0/3287683555 shutdown_connections 2024-04-25T00:25:12.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:12.926+0000 7fa553bb7640 1 -- 172.21.15.107:0/3287683555 wait complete. 2024-04-25T00:25:12.943 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:25:13.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:12 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/408310908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:25:13.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.097+0000 7f67936b3640 1 -- 172.21.15.107:0/2047724493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f678c073470 msgr2=0x7f678c06b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:13.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.097+0000 7f67936b3640 1 --2- 172.21.15.107:0/2047724493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f678c073470 0x7f678c06b250 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6788009900 tx=0x7f678802a480 comp rx=0 tx=0).stop 2024-04-25T00:25:13.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.097+0000 7f67936b3640 1 -- 172.21.15.107:0/2047724493 shutdown_connections 2024-04-25T00:25:13.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.097+0000 7f67936b3640 1 --2- 172.21.15.107:0/2047724493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c112cb0 0x7f678c1150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.097+0000 7f67936b3640 1 --2- 172.21.15.107:0/2047724493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f678c06b790 0x7f678c06bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.097+0000 7f67936b3640 1 --2- 172.21.15.107:0/2047724493 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f678c073470 0x7f678c06b250 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.097+0000 7f67936b3640 1 -- 172.21.15.107:0/2047724493 >> 172.21.15.107:0/2047724493 conn(0x7f678c076be0 msgr2=0x7f678c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:13.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.097+0000 7f67936b3640 1 -- 172.21.15.107:0/2047724493 shutdown_connections 2024-04-25T00:25:13.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.097+0000 7f67936b3640 1 -- 172.21.15.107:0/2047724493 wait complete. 2024-04-25T00:25:13.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.098+0000 7f67936b3640 1 Processor -- start 2024-04-25T00:25:13.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.098+0000 7f67936b3640 1 -- start start 2024-04-25T00:25:13.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.098+0000 7f67936b3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f678c06b790 0x7f678c105460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.098+0000 7f67936b3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f678c073470 0x7f678c1059a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.098+0000 7f67936b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c112cb0 0x7f678c1a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.098+0000 7f67936b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f678c101da0 con 0x7f678c112cb0 2024-04-25T00:25:13.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.098+0000 7f67936b3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f678c101c20 con 0x7f678c06b790 2024-04-25T00:25:13.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.098+0000 7f67936b3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f678c101f20 con 0x7f678c073470 2024-04-25T00:25:13.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.098+0000 7f6790c4e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f678c073470 0x7f678c1059a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.098+0000 7f6790c4e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f678c073470 0x7f678c1059a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:40420/0 (socket says 172.21.15.107:40420) 2024-04-25T00:25:13.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.099+0000 7f6790c4e640 1 -- 172.21.15.107:0/2122288348 learned_addr learned my addr 172.21.15.107:0/2122288348 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:13.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.099+0000 7f6791c50640 1 --2- 172.21.15.107:0/2122288348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c112cb0 0x7f678c1a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.099+0000 7f679144f640 1 --2- 172.21.15.107:0/2122288348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f678c06b790 0x7f678c105460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.099+0000 7f6790c4e640 1 -- 172.21.15.107:0/2122288348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f678c06b790 msgr2=0x7f678c105460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:13.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.099+0000 7f6790c4e640 1 --2- 172.21.15.107:0/2122288348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f678c06b790 0x7f678c105460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.099+0000 7f6790c4e640 1 -- 172.21.15.107:0/2122288348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c112cb0 msgr2=0x7f678c1a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:13.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.099+0000 7f6790c4e640 1 --2- 172.21.15.107:0/2122288348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c112cb0 0x7f678c1a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.099+0000 7f6790c4e640 1 -- 172.21.15.107:0/2122288348 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f678c1a2280 con 0x7f678c073470 2024-04-25T00:25:13.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.099+0000 7f6790c4e640 1 --2- 172.21.15.107:0/2122288348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f678c073470 0x7f678c1059a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f677c02fa00 tx=0x7f677c02fec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:13.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.099+0000 7f67827fc640 1 -- 172.21.15.107:0/2122288348 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c039070 con 0x7f678c073470 2024-04-25T00:25:13.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.099+0000 7f67827fc640 1 -- 172.21.15.107:0/2122288348 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f677c005020 con 0x7f678c073470 2024-04-25T00:25:13.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.099+0000 7f67827fc640 1 -- 172.21.15.107:0/2122288348 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c004de0 con 0x7f678c073470 2024-04-25T00:25:13.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.099+0000 7f67936b3640 1 -- 172.21.15.107:0/2122288348 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f678c1a2450 con 0x7f678c073470 2024-04-25T00:25:13.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.099+0000 7f67936b3640 1 -- 172.21.15.107:0/2122288348 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f678c1a28f0 con 0x7f678c073470 2024-04-25T00:25:13.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.103+0000 7f67963bf740 1 Processor -- start 2024-04-25T00:25:13.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.103+0000 7f67827fc640 1 -- 172.21.15.107:0/2122288348 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f677c0051d0 con 0x7f678c073470 2024-04-25T00:25:13.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.103+0000 7f67963bf740 1 -- start start 2024-04-25T00:25:13.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.104+0000 7f67963bf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bb580 0x558df87bb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.104+0000 7f67963bf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87bbea0 0x558df87c6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.104+0000 7f67963bf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558df87c6570 0x558df87c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.104+0000 7f67963bf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558df8720670 con 0x558df87bbea0 2024-04-25T00:25:13.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.104+0000 7f67963bf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558df86c8970 con 0x558df87c6570 2024-04-25T00:25:13.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.104+0000 7f67963bf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558df871c630 con 0x558df87bb580 2024-04-25T00:25:13.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.104+0000 7f679144f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87bbea0 0x558df87c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.104+0000 7f6790c4e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558df87c6570 0x558df87c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.104+0000 7f6791c50640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bb580 0x558df87bb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.104+0000 7f6791c50640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bb580 0x558df87bb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:40436/0 (socket says 172.21.15.107:40436) 2024-04-25T00:25:13.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.104+0000 7f6791c50640 1 -- 172.21.15.107:0/398739069 learned_addr learned my addr 172.21.15.107:0/398739069 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:13.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.104+0000 7f679144f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87bbea0 0x558df87c6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32770/0 (socket says 172.21.15.107:32770) 2024-04-25T00:25:13.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.104+0000 7f6790c4e640 1 -- 172.21.15.107:0/398739069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bb580 msgr2=0x558df87bb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:13.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.104+0000 7f6790c4e640 1 --2- 172.21.15.107:0/398739069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bb580 0x558df87bb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.104+0000 7f6790c4e640 1 -- 172.21.15.107:0/398739069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87bbea0 msgr2=0x558df87c6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:13.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.104+0000 7f6790c4e640 1 --2- 172.21.15.107:0/398739069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87bbea0 0x558df87c6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.104+0000 7f6790c4e640 1 -- 172.21.15.107:0/398739069 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558df86ec400 con 0x558df87c6570 2024-04-25T00:25:13.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.104+0000 7f67827fc640 1 --2- 172.21.15.107:0/2122288348 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f67780a2de0 0x7f67780a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.104+0000 7f6790c4e640 1 --2- 172.21.15.107:0/398739069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558df87c6570 0x558df87c8960 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f677c0ee4f0 tx=0x7f677c0f9910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53f06e1b4bd94326 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:13.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.105+0000 7f67827fc640 1 -- 172.21.15.107:0/2122288348 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9571+0+0 (secure 0 0 0) 0x7f677c0f1220 con 0x7f678c073470 2024-04-25T00:25:13.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.105+0000 7f6791c50640 1 --2- 172.21.15.107:0/2122288348 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f67780a2de0 0x7f67780a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.105+0000 7f67777fe640 1 -- 172.21.15.107:0/398739069 <== mon.2 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c042070 con 0x558df87c6570 2024-04-25T00:25:13.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.105+0000 7f67777fe640 1 -- 172.21.15.107:0/398739069 <== mon.2 v2:172.21.15.170:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f677c0bd370 con 0x558df87c6570 2024-04-25T00:25:13.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.105+0000 7f67777fe640 1 -- 172.21.15.107:0/398739069 <== mon.2 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c0c1670 con 0x558df87c6570 2024-04-25T00:25:13.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.106+0000 7f67963bf740 1 -- 172.21.15.107:0/398739069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558df87c6570 msgr2=0x558df87c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:13.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.106+0000 7f6791c50640 1 --2- 172.21.15.107:0/2122288348 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f67780a2de0 0x7f67780a52a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6784011630 tx=0x7f6784009500 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:13.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.106+0000 7f67963bf740 1 --2- 172.21.15.107:0/398739069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558df87c6570 0x558df87c8960 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f677c0ee4f0 tx=0x7f677c0f9910 comp rx=0 tx=0).stop 2024-04-25T00:25:13.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.106+0000 7f67963bf740 1 -- 172.21.15.107:0/398739069 shutdown_connections 2024-04-25T00:25:13.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.106+0000 7f67963bf740 1 --2- 172.21.15.107:0/398739069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558df87c6570 0x558df87c8960 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.106+0000 7f67963bf740 1 --2- 172.21.15.107:0/398739069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87bbea0 0x558df87c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.106+0000 7f67963bf740 1 --2- 172.21.15.107:0/398739069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bb580 0x558df87bb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.106+0000 7f67963bf740 1 -- 172.21.15.107:0/398739069 >> 172.21.15.107:0/398739069 conn(0x558df87b78a0 msgr2=0x558df87b7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:13.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.106+0000 7f67963bf740 1 -- 172.21.15.107:0/398739069 shutdown_connections 2024-04-25T00:25:13.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.106+0000 7f67963bf740 1 -- 172.21.15.107:0/398739069 wait complete. 2024-04-25T00:25:13.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.106+0000 7f67963bf740 1 -- start start 2024-04-25T00:25:13.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.106+0000 7f67963bf740 1 Processor -- start 2024-04-25T00:25:13.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.107+0000 7f67963bf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87bb580 0x558df87c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.107+0000 7f67963bf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bbea0 0x558df87c3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.107+0000 7f67963bf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558df87c6570 0x558df87c3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.107+0000 7f67963bf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558df87da370 con 0x558df87bb580 2024-04-25T00:25:13.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.107+0000 7f67963bf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558df86dc050 con 0x558df87c6570 2024-04-25T00:25:13.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.107+0000 7f67963bf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x558df87da4f0 con 0x558df87bbea0 2024-04-25T00:25:13.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.107+0000 7f67963bf740 10 client.0 _ll_register_callbacks cb 0x558df8433be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:13.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.107+0000 7f679144f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87bb580 0x558df87c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.107+0000 7f679144f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87bb580 0x558df87c5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32782/0 (socket says 172.21.15.107:32782) 2024-04-25T00:25:13.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.107+0000 7f679144f640 1 -- 172.21.15.107:0/1891124350 learned_addr learned my addr 172.21.15.107:0/1891124350 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:13.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.107+0000 7f6790c4e640 1 --2- 172.21.15.107:0/1891124350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bbea0 0x558df87c3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.107+0000 7f6791c50640 1 --2- 172.21.15.107:0/1891124350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558df87c6570 0x558df87c3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.107+0000 7f6790c4e640 1 -- 172.21.15.107:0/1891124350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558df87c6570 msgr2=0x558df87c3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:13.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.107+0000 7f6790c4e640 1 --2- 172.21.15.107:0/1891124350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558df87c6570 0x558df87c3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.107+0000 7f6790c4e640 1 -- 172.21.15.107:0/1891124350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87bb580 msgr2=0x558df87c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:13.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.107+0000 7f6790c4e640 1 --2- 172.21.15.107:0/1891124350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87bb580 0x558df87c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.107+0000 7f6790c4e640 1 -- 172.21.15.107:0/1891124350 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558df86addc0 con 0x558df87bbea0 2024-04-25T00:25:13.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.107+0000 7f679144f640 1 --2- 172.21.15.107:0/1891124350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87bb580 0x558df87c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:25:13.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.107+0000 7f6790c4e640 1 --2- 172.21.15.107:0/1891124350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bbea0 0x558df87c3220 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f677c0f4e90 tx=0x7f677c0bccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72ad3cdccc3a6cc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:13.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.107+0000 7f6774ff9640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-04-25T00:25:13.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.108+0000 7f6774ff9640 1 -- 172.21.15.107:0/1891124350 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c042070 con 0x558df87bbea0 2024-04-25T00:25:13.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.108+0000 7f6774ff9640 1 -- 172.21.15.107:0/1891124350 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9571+0+0 (secure 0 0 0) 0x7f677c0ee770 con 0x558df87bbea0 2024-04-25T00:25:13.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.108+0000 7f67963bf740 1 -- 172.21.15.107:0/1891124350 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f677c0bd370 con 0x558df87bbea0 2024-04-25T00:25:13.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.108+0000 7f6774ff9640 1 -- 172.21.15.107:0/1891124350 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f677c0b9610 con 0x558df87bbea0 2024-04-25T00:25:13.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.108+0000 7f67963bf740 10 client.26476 fetch_fsmap learned FSMap version 34 2024-04-25T00:25:13.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.108+0000 7f67963bf740 1 -- 172.21.15.107:0/1891124350 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558df84343f0 con 0x558df87bbea0 2024-04-25T00:25:13.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.108+0000 7f6774ff9640 1 -- 172.21.15.107:0/1891124350 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f677c0bd370 con 0x558df87bbea0 2024-04-25T00:25:13.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.108+0000 7f6774ff9640 1 -- 172.21.15.107:0/1891124350 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f677c030040 con 0x558df87bbea0 2024-04-25T00:25:13.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.108+0000 7f67963bf740 10 client.26476 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:25:13.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.108+0000 7f67963bf740 10 client.26476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:13.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.108+0000 7f67963bf740 20 client.26476 populate_metadata read hostname 'smithi107' 2024-04-25T00:25:13.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.109+0000 7f67963bf740 1 --2- 172.21.15.107:0/1891124350 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x558df87e74c0 0x558df87e98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.109+0000 7f67963bf740 4 client.26476 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:13.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.109+0000 7f67963bf740 1 -- 172.21.15.107:0/1891124350 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558df872dbb0 con 0x558df87e74c0 2024-04-25T00:25:13.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.109+0000 7f679144f640 1 --2- 172.21.15.107:0/1891124350 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x558df87e74c0 0x558df87e98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.109+0000 7f679144f640 1 --2- 172.21.15.107:0/1891124350 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x558df87e74c0 0x558df87e98b0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35988d0e0e47aa52 server_cookie=991544f6a272bcf8 in_seq=0 out_seq=0 2024-04-25T00:25:13.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.109+0000 7f6774ff9640 10 client.26476 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:13.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.110+0000 7f6774ff9640 1 -- 172.21.15.107:0/1891124350 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558df872dbb0 con 0x558df87e74c0 2024-04-25T00:25:13.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.110+0000 7f6774ff9640 10 client.26476 handle_command_reply: tid=0 2024-04-25T00:25:13.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.110+0000 7f67963bf740 1 client.26476 shutdown 2024-04-25T00:25:13.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.110+0000 7f67963bf740 1 -- 172.21.15.107:0/1891124350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bbea0 msgr2=0x558df87c3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:13.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.110+0000 7f67963bf740 1 --2- 172.21.15.107:0/1891124350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bbea0 0x558df87c3220 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f677c0f4e90 tx=0x7f677c0bccd0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.110+0000 7f67963bf740 1 -- 172.21.15.107:0/1891124350 shutdown_connections 2024-04-25T00:25:13.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.110+0000 7f67963bf740 1 --2- 172.21.15.107:0/1891124350 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x558df87e74c0 0x558df87e98b0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.111+0000 7f67963bf740 1 --2- 172.21.15.107:0/1891124350 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558df87c6570 0x558df87c3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.111+0000 7f67963bf740 1 --2- 172.21.15.107:0/1891124350 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bbea0 0x558df87c3220 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.111+0000 7f67963bf740 1 --2- 172.21.15.107:0/1891124350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87bb580 0x558df87c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.111+0000 7f67963bf740 1 -- 172.21.15.107:0/1891124350 >> 172.21.15.107:0/1891124350 conn(0x558df87b78a0 msgr2=0x558df87b6b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:13.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.111+0000 7f6774ff9640 0 client.26476 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:13.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.111+0000 7f67963bf740 1 -- 172.21.15.107:0/1891124350 shutdown_connections 2024-04-25T00:25:13.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.111+0000 7f67963bf740 1 -- 172.21.15.107:0/1891124350 wait complete. 2024-04-25T00:25:13.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.111+0000 7f67963bf740 20 client.26476 trim_cache size 0 max 16384 2024-04-25T00:25:13.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.130+0000 7f67963bf740 1 Processor -- start 2024-04-25T00:25:13.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.130+0000 7f67963bf740 1 -- start start 2024-04-25T00:25:13.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.131+0000 7f67963bf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bb580 0x558df87c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.131+0000 7f67963bf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558df87bbea0 0x558df87c04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.131+0000 7f67963bf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87c6570 0x558df87c09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.131+0000 7f679144f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bb580 0x558df87c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.131+0000 7f679144f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bb580 0x558df87c5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:40450/0 (socket says 172.21.15.107:40450) 2024-04-25T00:25:13.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.131+0000 7f679144f640 1 -- 172.21.15.107:0/2703662276 learned_addr learned my addr 172.21.15.107:0/2703662276 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:13.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.131+0000 7f67963bf740 1 -- 172.21.15.107:0/2703662276 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f678c1a6c10 con 0x558df87c6570 2024-04-25T00:25:13.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.131+0000 7f67963bf740 1 -- 172.21.15.107:0/2703662276 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f678c1a44e0 con 0x558df87bbea0 2024-04-25T00:25:13.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.131+0000 7f67963bf740 1 -- 172.21.15.107:0/2703662276 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f678c067cd0 con 0x558df87bb580 2024-04-25T00:25:13.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.132+0000 7f6790c4e640 1 --2- 172.21.15.107:0/2703662276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558df87bbea0 0x558df87c04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.132+0000 7f6791c50640 1 --2- 172.21.15.107:0/2703662276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87c6570 0x558df87c09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.132+0000 7f679144f640 1 -- 172.21.15.107:0/2703662276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558df87bbea0 msgr2=0x558df87c04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:13.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.132+0000 7f679144f640 1 --2- 172.21.15.107:0/2703662276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558df87bbea0 0x558df87c04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.132+0000 7f679144f640 1 -- 172.21.15.107:0/2703662276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87c6570 msgr2=0x558df87c09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:13.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.132+0000 7f679144f640 1 --2- 172.21.15.107:0/2703662276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87c6570 0x558df87c09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.132+0000 7f679144f640 1 -- 172.21.15.107:0/2703662276 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558df87c3ba0 con 0x558df87bb580 2024-04-25T00:25:13.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.132+0000 7f6791c50640 1 --2- 172.21.15.107:0/2703662276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87c6570 0x558df87c09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:13.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.132+0000 7f679144f640 1 --2- 172.21.15.107:0/2703662276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bb580 0x558df87c5a60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6788005940 tx=0x7f67880316d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8563375153fec4c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:13.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.132+0000 7f6776ffd640 1 -- 172.21.15.107:0/2703662276 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678804a020 con 0x558df87bb580 2024-04-25T00:25:13.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.132+0000 7f6776ffd640 1 -- 172.21.15.107:0/2703662276 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6788046bc0 con 0x558df87bb580 2024-04-25T00:25:13.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.132+0000 7f6776ffd640 1 -- 172.21.15.107:0/2703662276 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788035040 con 0x558df87bb580 2024-04-25T00:25:13.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.133+0000 7f67963bf740 1 -- 172.21.15.107:0/2703662276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bb580 msgr2=0x558df87c5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:13.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.133+0000 7f67963bf740 1 --2- 172.21.15.107:0/2703662276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bb580 0x558df87c5a60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6788005940 tx=0x7f67880316d0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.133+0000 7f67963bf740 1 -- 172.21.15.107:0/2703662276 shutdown_connections 2024-04-25T00:25:13.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.133+0000 7f67963bf740 1 --2- 172.21.15.107:0/2703662276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87c6570 0x558df87c09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.133+0000 7f67963bf740 1 --2- 172.21.15.107:0/2703662276 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558df87bbea0 0x558df87c04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.133+0000 7f67963bf740 1 --2- 172.21.15.107:0/2703662276 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bb580 0x558df87c5a60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.133+0000 7f67963bf740 1 -- 172.21.15.107:0/2703662276 >> 172.21.15.107:0/2703662276 conn(0x558df87b78a0 msgr2=0x558df87be860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:13.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.133+0000 7f67963bf740 1 -- 172.21.15.107:0/2703662276 shutdown_connections 2024-04-25T00:25:13.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.133+0000 7f67963bf740 1 -- 172.21.15.107:0/2703662276 wait complete. 2024-04-25T00:25:13.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.134+0000 7f67963bf740 1 -- start start 2024-04-25T00:25:13.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.134+0000 7f67963bf740 1 Processor -- start 2024-04-25T00:25:13.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.135+0000 7f67963bf740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bb580 0x558df87e94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.135+0000 7f67963bf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558df87bbea0 0x558df87e9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.135+0000 7f67963bf740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87c6570 0x558df87e9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.135+0000 7f67963bf740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f678c111dd0 con 0x558df87c6570 2024-04-25T00:25:13.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.135+0000 7f67963bf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f678c1a3e10 con 0x558df87bbea0 2024-04-25T00:25:13.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.135+0000 7f67963bf740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f678c1a3070 con 0x558df87bb580 2024-04-25T00:25:13.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.135+0000 7f67963bf740 10 client.0 _ll_register_callbacks cb 0x7f6788009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:13.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.135+0000 7f679144f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bb580 0x558df87e94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.135+0000 7f679144f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bb580 0x558df87e94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.107:40466/0 (socket says 172.21.15.107:40466) 2024-04-25T00:25:13.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.135+0000 7f679144f640 1 -- 172.21.15.107:0/1304561161 learned_addr learned my addr 172.21.15.107:0/1304561161 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:13.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.135+0000 7f6790c4e640 1 --2- 172.21.15.107:0/1304561161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558df87bbea0 0x558df87e9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.135+0000 7f6791c50640 1 --2- 172.21.15.107:0/1304561161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87c6570 0x558df87e9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.135+0000 7f679144f640 1 -- 172.21.15.107:0/1304561161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558df87bbea0 msgr2=0x558df87e9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:13.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.135+0000 7f679144f640 1 --2- 172.21.15.107:0/1304561161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558df87bbea0 0x558df87e9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.135+0000 7f679144f640 1 -- 172.21.15.107:0/1304561161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87c6570 msgr2=0x558df87e9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:13.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.135+0000 7f679144f640 1 --2- 172.21.15.107:0/1304561161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87c6570 0x558df87e9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.135+0000 7f679144f640 1 -- 172.21.15.107:0/1304561161 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558df84fe7d0 con 0x558df87bb580 2024-04-25T00:25:13.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.135+0000 7f6791c50640 1 --2- 172.21.15.107:0/1304561161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87c6570 0x558df87e9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:13.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.135+0000 7f679144f640 1 --2- 172.21.15.107:0/1304561161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bb580 0x558df87e94d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f678802a9a0 tx=0x7f6788009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d186b5687fc8511e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:13.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.135+0000 7f6774ff9640 10 client.0 ms_handle_connect on v2:172.21.15.195:3300/0 2024-04-25T00:25:13.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.136+0000 7f6774ff9640 1 -- 172.21.15.107:0/1304561161 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678804a030 con 0x558df87bb580 2024-04-25T00:25:13.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.136+0000 7f67963bf740 1 -- 172.21.15.107:0/1304561161 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6788046bc0 con 0x558df87bb580 2024-04-25T00:25:13.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.136+0000 7f6774ff9640 1 -- 172.21.15.107:0/1304561161 <== mon.1 v2:172.21.15.195:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9571+0+0 (secure 0 0 0) 0x7f6788046880 con 0x558df87bb580 2024-04-25T00:25:13.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.136+0000 7f6774ff9640 1 -- 172.21.15.107:0/1304561161 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6788035050 con 0x558df87bb580 2024-04-25T00:25:13.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.136+0000 7f67963bf740 10 client.26482 fetch_fsmap learned FSMap version 34 2024-04-25T00:25:13.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.136+0000 7f67963bf740 1 -- 172.21.15.107:0/1304561161 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558df884c920 con 0x558df87bb580 2024-04-25T00:25:13.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.136+0000 7f6774ff9640 1 -- 172.21.15.107:0/1304561161 <== mon.1 v2:172.21.15.195:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6788046bc0 con 0x558df87bb580 2024-04-25T00:25:13.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.136+0000 7f6774ff9640 1 -- 172.21.15.107:0/1304561161 <== mon.1 v2:172.21.15.195:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f6788003ce0 con 0x558df87bb580 2024-04-25T00:25:13.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.136+0000 7f67963bf740 10 client.26482 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:25:13.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.136+0000 7f67963bf740 10 client.26482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:13.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.136+0000 7f67963bf740 20 client.26482 populate_metadata read hostname 'smithi107' 2024-04-25T00:25:13.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.136+0000 7f67963bf740 1 --2- 172.21.15.107:0/1304561161 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x558df884cf80 0x558df884f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.136+0000 7f67963bf740 4 client.26482 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:25:13.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.136+0000 7f67963bf740 1 -- 172.21.15.107:0/1304561161 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558df8727f00 con 0x558df884cf80 2024-04-25T00:25:13.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.137+0000 7f6790c4e640 1 --2- 172.21.15.107:0/1304561161 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x558df884cf80 0x558df884f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.137+0000 7f6774ff9640 10 client.26482 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:13.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.137+0000 7f6790c4e640 1 --2- 172.21.15.107:0/1304561161 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x558df884cf80 0x558df884f3a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21b0687d0276d0fd server_cookie=401ce531d1a3dd2b in_seq=0 out_seq=0 2024-04-25T00:25:13.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.137+0000 7f6774ff9640 1 -- 172.21.15.107:0/1304561161 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558df8727f00 con 0x558df884cf80 2024-04-25T00:25:13.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.137+0000 7f6774ff9640 10 client.26482 handle_command_reply: tid=0 2024-04-25T00:25:13.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.137+0000 7f67963bf740 1 client.26482 shutdown 2024-04-25T00:25:13.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.137+0000 7f67963bf740 1 -- 172.21.15.107:0/1304561161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bb580 msgr2=0x558df87e94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:13.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.137+0000 7f67963bf740 1 --2- 172.21.15.107:0/1304561161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bb580 0x558df87e94d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f678802a9a0 tx=0x7f6788009610 comp rx=0 tx=0).stop 2024-04-25T00:25:13.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.138+0000 7f67963bf740 1 -- 172.21.15.107:0/1304561161 shutdown_connections 2024-04-25T00:25:13.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.138+0000 7f67963bf740 1 --2- 172.21.15.107:0/1304561161 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x558df884cf80 0x558df884f3a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.138+0000 7f67963bf740 1 --2- 172.21.15.107:0/1304561161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558df87c6570 0x558df87e9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.138+0000 7f6774ff9640 0 client.26482 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:13.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.138+0000 7f67963bf740 1 --2- 172.21.15.107:0/1304561161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558df87bbea0 0x558df87e9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.138+0000 7f67963bf740 1 --2- 172.21.15.107:0/1304561161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x558df87bb580 0x558df87e94d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.138+0000 7f67963bf740 1 -- 172.21.15.107:0/1304561161 >> 172.21.15.107:0/1304561161 conn(0x558df87b78a0 msgr2=0x558df87e2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:13.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.139+0000 7f67963bf740 1 -- 172.21.15.107:0/1304561161 shutdown_connections 2024-04-25T00:25:13.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.139+0000 7f67963bf740 1 -- 172.21.15.107:0/1304561161 wait complete. 2024-04-25T00:25:13.143 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:25:13.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.139+0000 7f67963bf740 20 client.26482 trim_cache size 0 max 16384 2024-04-25T00:25:13.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.140+0000 7f67936b3640 1 -- 172.21.15.107:0/2122288348 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f67780a2de0 msgr2=0x7f67780a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:13.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.140+0000 7f67936b3640 1 --2- 172.21.15.107:0/2122288348 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f67780a2de0 0x7f67780a52a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6784011630 tx=0x7f6784009500 comp rx=0 tx=0).stop 2024-04-25T00:25:13.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.140+0000 7f67936b3640 1 -- 172.21.15.107:0/2122288348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f678c073470 msgr2=0x7f678c1059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:13.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.140+0000 7f67936b3640 1 --2- 172.21.15.107:0/2122288348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f678c073470 0x7f678c1059a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f677c02fa00 tx=0x7f677c02fec0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.141+0000 7f67936b3640 1 -- 172.21.15.107:0/2122288348 shutdown_connections 2024-04-25T00:25:13.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.141+0000 7f67936b3640 1 --2- 172.21.15.107:0/2122288348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f678c112cb0 0x7f678c1a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.141+0000 7f67936b3640 1 --2- 172.21.15.107:0/2122288348 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f678c073470 0x7f678c1059a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.141+0000 7f67936b3640 1 --2- 172.21.15.107:0/2122288348 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f67780a2de0 0x7f67780a52a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.141+0000 7f67936b3640 1 --2- 172.21.15.107:0/2122288348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f678c06b790 0x7f678c105460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.141+0000 7f67936b3640 1 -- 172.21.15.107:0/2122288348 >> 172.21.15.107:0/2122288348 conn(0x7f678c076be0 msgr2=0x7f678c113160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:13.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.141+0000 7f67936b3640 1 -- 172.21.15.107:0/2122288348 shutdown_connections 2024-04-25T00:25:13.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:13.141+0000 7f67936b3640 1 -- 172.21.15.107:0/2122288348 wait complete. 2024-04-25T00:25:13.155 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:12 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/408310908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:25:13.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:12 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/408310908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:25:14.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:13 smithi195 ceph-mon[32833]: pgmap v271: 129 pgs: 6 active+clean+snaptrim, 109 active+clean, 14 active+clean+snaptrim_wait; 195 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:25:14.157 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:25:14.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:25:14.157 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:25:14.181 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:13 smithi107 ceph-mon[29153]: pgmap v271: 129 pgs: 6 active+clean+snaptrim, 109 active+clean, 14 active+clean+snaptrim_wait; 195 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:25:14.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:13 smithi170 ceph-mon[32807]: pgmap v271: 129 pgs: 6 active+clean+snaptrim, 109 active+clean, 14 active+clean+snaptrim_wait; 195 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:25:14.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.268+0000 7f089aeb3640 1 -- 172.21.15.107:0/1776860519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08941020d0 msgr2=0x7f089410e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:14.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.268+0000 7f089aeb3640 1 --2- 172.21.15.107:0/1776860519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08941020d0 0x7f089410e5e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f088c009cd0 tx=0x7f088c02a8e0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.268+0000 7f089aeb3640 1 -- 172.21.15.107:0/1776860519 shutdown_connections 2024-04-25T00:25:14.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.268+0000 7f089aeb3640 1 --2- 172.21.15.107:0/1776860519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08941020d0 0x7f089410e5e0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.268+0000 7f089aeb3640 1 --2- 172.21.15.107:0/1776860519 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0894101710 0x7f0894101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.268+0000 7f089aeb3640 1 --2- 172.21.15.107:0/1776860519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0894107720 0x7f0894107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.268+0000 7f089aeb3640 1 -- 172.21.15.107:0/1776860519 >> 172.21.15.107:0/1776860519 conn(0x7f0894078b90 msgr2=0x7f0894078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:14.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.268+0000 7f089aeb3640 1 -- 172.21.15.107:0/1776860519 shutdown_connections 2024-04-25T00:25:14.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.268+0000 7f089aeb3640 1 -- 172.21.15.107:0/1776860519 wait complete. 2024-04-25T00:25:14.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.269+0000 7f089aeb3640 1 Processor -- start 2024-04-25T00:25:14.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.269+0000 7f089aeb3640 1 -- start start 2024-04-25T00:25:14.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.269+0000 7f089aeb3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0894101710 0x7f0894197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.269+0000 7f089aeb3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08941020d0 0x7f0894198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.269+0000 7f089aeb3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0894107720 0x7f089419c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.269+0000 7f089aeb3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0894111b30 con 0x7f0894101710 2024-04-25T00:25:14.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.269+0000 7f089aeb3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f08941119b0 con 0x7f08941020d0 2024-04-25T00:25:14.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.269+0000 7f089aeb3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0894111cb0 con 0x7f0894107720 2024-04-25T00:25:14.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.269+0000 7f0898c4f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0894101710 0x7f0894197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.269+0000 7f0898c4f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0894101710 0x7f0894197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32820/0 (socket says 172.21.15.107:32820) 2024-04-25T00:25:14.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.269+0000 7f0898c4f640 1 -- 172.21.15.107:0/1380768466 learned_addr learned my addr 172.21.15.107:0/1380768466 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:14.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.269+0000 7f0898c4f640 1 -- 172.21.15.107:0/1380768466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0894107720 msgr2=0x7f089419c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:14.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.269+0000 7f0898c4f640 1 --2- 172.21.15.107:0/1380768466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0894107720 0x7f089419c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.269+0000 7f0898c4f640 1 -- 172.21.15.107:0/1380768466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08941020d0 msgr2=0x7f0894198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:14.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.269+0000 7f0898c4f640 1 --2- 172.21.15.107:0/1380768466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08941020d0 0x7f0894198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.269+0000 7f0898c4f640 1 -- 172.21.15.107:0/1380768466 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089419cf40 con 0x7f0894101710 2024-04-25T00:25:14.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.270+0000 7f0898c4f640 1 --2- 172.21.15.107:0/1380768466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0894101710 0x7f0894197f50 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f088800d950 tx=0x7f088800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:14.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.270+0000 7f0891ffb640 1 -- 172.21.15.107:0/1380768466 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0888017070 con 0x7f0894101710 2024-04-25T00:25:14.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.270+0000 7f0891ffb640 1 -- 172.21.15.107:0/1380768466 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0888009c30 con 0x7f0894101710 2024-04-25T00:25:14.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.270+0000 7f0891ffb640 1 -- 172.21.15.107:0/1380768466 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088800f6b0 con 0x7f0894101710 2024-04-25T00:25:14.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.270+0000 7f089aeb3640 1 -- 172.21.15.107:0/1380768466 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f089419d230 con 0x7f0894101710 2024-04-25T00:25:14.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.270+0000 7f089aeb3640 1 -- 172.21.15.107:0/1380768466 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08941a4ac0 con 0x7f0894101710 2024-04-25T00:25:14.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.273+0000 7f0891ffb640 1 -- 172.21.15.107:0/1380768466 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f08880109d0 con 0x7f0894101710 2024-04-25T00:25:14.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.273+0000 7f089dbbe740 1 Processor -- start 2024-04-25T00:25:14.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.273+0000 7f089dbbe740 1 -- start start 2024-04-25T00:25:14.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.273+0000 7f089dbbe740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624157a4980 0x5624157a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.273+0000 7f089dbbe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a3290 0x5624157a3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.273+0000 7f089dbbe740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624157ae010 0x5624157b0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.273+0000 7f089dbbe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5624157072c0 con 0x5624157a3290 2024-04-25T00:25:14.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.273+0000 7f089dbbe740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5624156af910 con 0x5624157ae010 2024-04-25T00:25:14.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.273+0000 7f089dbbe740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x562415703280 con 0x5624157a4980 2024-04-25T00:25:14.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.273+0000 7f0898c4f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a3290 0x5624157a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.273+0000 7f0891ffb640 1 --2- 172.21.15.107:0/1380768466 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f08840a2d80 0x7f08840a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.273+0000 7f0898c4f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a3290 0x5624157a3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32836/0 (socket says 172.21.15.107:32836) 2024-04-25T00:25:14.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.273+0000 7f0898c4f640 1 -- 172.21.15.107:0/2671652649 learned_addr learned my addr 172.21.15.107:0/2671652649 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:14.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.273+0000 7f0893fff640 1 --2- 172.21.15.107:0/2671652649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624157ae010 0x5624157b0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.273+0000 7f0899450640 1 --2- 172.21.15.107:0/1380768466 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f08840a2d80 0x7f08840a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.274+0000 7f0899450640 1 --2- 172.21.15.107:0/2671652649 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624157a4980 0x5624157a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.274+0000 7f0891ffb640 1 -- 172.21.15.107:0/1380768466 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9571+0+0 (secure 0 0 0) 0x7f08880cf440 con 0x7f0894101710 2024-04-25T00:25:14.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.274+0000 7f0898c4f640 1 -- 172.21.15.107:0/2671652649 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624157a4980 msgr2=0x5624157a2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:14.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.274+0000 7f0898c4f640 1 --2- 172.21.15.107:0/2671652649 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624157a4980 0x5624157a2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.274+0000 7f0898c4f640 1 -- 172.21.15.107:0/2671652649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624157ae010 msgr2=0x5624157b0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:14.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.274+0000 7f0898c4f640 1 --2- 172.21.15.107:0/2671652649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624157ae010 0x5624157b0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.274+0000 7f0898c4f640 1 -- 172.21.15.107:0/2671652649 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624156c0fb0 con 0x5624157a3290 2024-04-25T00:25:14.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.274+0000 7f0899450640 1 --2- 172.21.15.107:0/2671652649 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624157a4980 0x5624157a2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:14.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.274+0000 7f0893fff640 1 --2- 172.21.15.107:0/2671652649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624157ae010 0x5624157b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:25:14.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.274+0000 7f0898c4f640 1 --2- 172.21.15.107:0/2671652649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a3290 0x5624157a3710 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f0888097f10 tx=0x7f088809b3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b26e1e79ed91db2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:14.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.274+0000 7f0899450640 1 --2- 172.21.15.107:0/1380768466 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f08840a2d80 0x7f08840a5240 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f088c009a60 tx=0x7f088c0093a0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:14.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.274+0000 7f087effd640 1 -- 172.21.15.107:0/2671652649 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0888020070 con 0x5624157a3290 2024-04-25T00:25:14.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.274+0000 7f087effd640 1 -- 172.21.15.107:0/2671652649 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08880bb070 con 0x5624157a3290 2024-04-25T00:25:14.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.274+0000 7f087effd640 1 -- 172.21.15.107:0/2671652649 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0888097bb0 con 0x5624157a3290 2024-04-25T00:25:14.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.275+0000 7f089dbbe740 1 -- 172.21.15.107:0/2671652649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a3290 msgr2=0x5624157a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:14.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.275+0000 7f089dbbe740 1 --2- 172.21.15.107:0/2671652649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a3290 0x5624157a3710 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f0888097f10 tx=0x7f088809b3c0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.275+0000 7f089dbbe740 1 -- 172.21.15.107:0/2671652649 shutdown_connections 2024-04-25T00:25:14.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.275+0000 7f089dbbe740 1 --2- 172.21.15.107:0/2671652649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624157ae010 0x5624157b0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.275+0000 7f089dbbe740 1 --2- 172.21.15.107:0/2671652649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a3290 0x5624157a3710 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.275+0000 7f089dbbe740 1 --2- 172.21.15.107:0/2671652649 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624157a4980 0x5624157a2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.275+0000 7f089dbbe740 1 -- 172.21.15.107:0/2671652649 >> 172.21.15.107:0/2671652649 conn(0x56241576b950 msgr2=0x56241579f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:14.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.275+0000 7f089dbbe740 1 -- 172.21.15.107:0/2671652649 shutdown_connections 2024-04-25T00:25:14.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.275+0000 7f089dbbe740 1 -- 172.21.15.107:0/2671652649 wait complete. 2024-04-25T00:25:14.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.275+0000 7f089dbbe740 1 -- start start 2024-04-25T00:25:14.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.275+0000 7f089dbbe740 1 Processor -- start 2024-04-25T00:25:14.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f089dbbe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a3290 0x5624157aaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f089dbbe740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624157a4980 0x5624157ab4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f089dbbe740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624157ae010 0x5624157aba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f089dbbe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5624157b0de0 con 0x5624157a3290 2024-04-25T00:25:14.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f089dbbe740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5624156c2e90 con 0x5624157ae010 2024-04-25T00:25:14.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f089dbbe740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5624157b0f60 con 0x5624157a4980 2024-04-25T00:25:14.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f089dbbe740 10 client.0 _ll_register_callbacks cb 0x5624156fe970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:14.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f0898c4f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a3290 0x5624157aaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f0898c4f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a3290 0x5624157aaf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32846/0 (socket says 172.21.15.107:32846) 2024-04-25T00:25:14.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f0898c4f640 1 -- 172.21.15.107:0/755101995 learned_addr learned my addr 172.21.15.107:0/755101995 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:14.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f0899450640 1 --2- 172.21.15.107:0/755101995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624157ae010 0x5624157aba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f0893fff640 1 --2- 172.21.15.107:0/755101995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624157a4980 0x5624157ab4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f0898c4f640 1 -- 172.21.15.107:0/755101995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624157a4980 msgr2=0x5624157ab4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:14.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f0898c4f640 1 --2- 172.21.15.107:0/755101995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624157a4980 0x5624157ab4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f0898c4f640 1 -- 172.21.15.107:0/755101995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624157ae010 msgr2=0x5624157aba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:14.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f0898c4f640 1 --2- 172.21.15.107:0/755101995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624157ae010 0x5624157aba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f0898c4f640 1 -- 172.21.15.107:0/755101995 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624154e37e0 con 0x5624157a3290 2024-04-25T00:25:14.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f0898c4f640 1 --2- 172.21.15.107:0/755101995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a3290 0x5624157aaf90 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f08880a4370 tx=0x7f08880ccc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0bc92a6959160b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:14.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f0873fff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:25:14.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f0873fff640 1 -- 172.21.15.107:0/755101995 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0888020070 con 0x5624157a3290 2024-04-25T00:25:14.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f0873fff640 1 -- 172.21.15.107:0/755101995 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9571+0+0 (secure 0 0 0) 0x7f0888097bb0 con 0x5624157a3290 2024-04-25T00:25:14.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f089dbbe740 1 -- 172.21.15.107:0/755101995 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08880bb070 con 0x5624157a3290 2024-04-25T00:25:14.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f0873fff640 1 -- 172.21.15.107:0/755101995 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f088809f8f0 con 0x5624157a3290 2024-04-25T00:25:14.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f089dbbe740 10 client.17592 fetch_fsmap learned FSMap version 34 2024-04-25T00:25:14.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.276+0000 7f089dbbe740 1 -- 172.21.15.107:0/755101995 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562415543f70 con 0x5624157a3290 2024-04-25T00:25:14.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.277+0000 7f0873fff640 1 -- 172.21.15.107:0/755101995 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08880bb070 con 0x5624157a3290 2024-04-25T00:25:14.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.277+0000 7f0873fff640 1 -- 172.21.15.107:0/755101995 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f0888010460 con 0x5624157a3290 2024-04-25T00:25:14.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.277+0000 7f089dbbe740 10 client.17592 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:25:14.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.277+0000 7f089dbbe740 10 client.17592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:14.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.277+0000 7f089dbbe740 20 client.17592 populate_metadata read hostname 'smithi107' 2024-04-25T00:25:14.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.277+0000 7f089dbbe740 1 --2- 172.21.15.107:0/755101995 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5624157bdf70 0x5624157c0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.277+0000 7f089dbbe740 4 client.17592 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:14.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.277+0000 7f089dbbe740 1 -- 172.21.15.107:0/755101995 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624156db5a0 con 0x5624157bdf70 2024-04-25T00:25:14.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.277+0000 7f0893fff640 1 --2- 172.21.15.107:0/755101995 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5624157bdf70 0x5624157c0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.277+0000 7f0873fff640 10 client.17592 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:14.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.277+0000 7f0893fff640 1 --2- 172.21.15.107:0/755101995 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5624157bdf70 0x5624157c0360 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff8f0b8b1245911a server_cookie=1abf0e70ff9e5cc0 in_seq=0 out_seq=0 2024-04-25T00:25:14.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.278+0000 7f0873fff640 1 -- 172.21.15.107:0/755101995 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5624156db5a0 con 0x5624157bdf70 2024-04-25T00:25:14.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.278+0000 7f0873fff640 10 client.17592 handle_command_reply: tid=0 2024-04-25T00:25:14.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.278+0000 7f089dbbe740 1 client.17592 shutdown 2024-04-25T00:25:14.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.278+0000 7f089dbbe740 1 -- 172.21.15.107:0/755101995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a3290 msgr2=0x5624157aaf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:14.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.278+0000 7f089dbbe740 1 --2- 172.21.15.107:0/755101995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a3290 0x5624157aaf90 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f08880a4370 tx=0x7f08880ccc80 comp rx=0 tx=0).stop 2024-04-25T00:25:14.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.278+0000 7f089dbbe740 1 -- 172.21.15.107:0/755101995 shutdown_connections 2024-04-25T00:25:14.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.278+0000 7f089dbbe740 1 --2- 172.21.15.107:0/755101995 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5624157bdf70 0x5624157c0360 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.278+0000 7f089dbbe740 1 --2- 172.21.15.107:0/755101995 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624157ae010 0x5624157aba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.278+0000 7f089dbbe740 1 --2- 172.21.15.107:0/755101995 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624157a4980 0x5624157ab4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.278+0000 7f0873fff640 0 client.17592 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:14.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.278+0000 7f089dbbe740 1 --2- 172.21.15.107:0/755101995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a3290 0x5624157aaf90 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.278+0000 7f089dbbe740 1 -- 172.21.15.107:0/755101995 >> 172.21.15.107:0/755101995 conn(0x56241576b950 msgr2=0x5624157a5660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:14.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.278+0000 7f089dbbe740 1 -- 172.21.15.107:0/755101995 shutdown_connections 2024-04-25T00:25:14.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.278+0000 7f089dbbe740 1 -- 172.21.15.107:0/755101995 wait complete. 2024-04-25T00:25:14.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.279+0000 7f089dbbe740 20 client.17592 trim_cache size 0 max 16384 2024-04-25T00:25:14.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.296+0000 7f089dbbe740 1 Processor -- start 2024-04-25T00:25:14.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.296+0000 7f089dbbe740 1 -- start start 2024-04-25T00:25:14.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.296+0000 7f089dbbe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a3290 0x5624157b9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.296+0000 7f089dbbe740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624157a4980 0x5624157c3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.296+0000 7f089dbbe740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624157ae010 0x5624157c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.296+0000 7f089dbbe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f08941b1660 con 0x5624157a3290 2024-04-25T00:25:14.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.296+0000 7f089dbbe740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f08941aef30 con 0x5624157a4980 2024-04-25T00:25:14.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.296+0000 7f089dbbe740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f08941acda0 con 0x5624157ae010 2024-04-25T00:25:14.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.296+0000 7f0898c4f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a3290 0x5624157b9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.296+0000 7f0898c4f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a3290 0x5624157b9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32860/0 (socket says 172.21.15.107:32860) 2024-04-25T00:25:14.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.296+0000 7f0898c4f640 1 -- 172.21.15.107:0/1382148098 learned_addr learned my addr 172.21.15.107:0/1382148098 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:14.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.296+0000 7f0898c4f640 1 -- 172.21.15.107:0/1382148098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624157ae010 msgr2=0x5624157c38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:14.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.296+0000 7f0898c4f640 1 --2- 172.21.15.107:0/1382148098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624157ae010 0x5624157c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.296+0000 7f0898c4f640 1 -- 172.21.15.107:0/1382148098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624157a4980 msgr2=0x5624157c3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:14.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.296+0000 7f0893fff640 1 --2- 172.21.15.107:0/1382148098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624157a4980 0x5624157c3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.296+0000 7f0898c4f640 1 --2- 172.21.15.107:0/1382148098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624157a4980 0x5624157c3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.296+0000 7f0898c4f640 1 -- 172.21.15.107:0/1382148098 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56241579fa00 con 0x5624157a3290 2024-04-25T00:25:14.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.296+0000 7f0893fff640 1 --2- 172.21.15.107:0/1382148098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624157a4980 0x5624157c3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:14.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.297+0000 7f0898c4f640 1 --2- 172.21.15.107:0/1382148098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a3290 0x5624157b9930 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f08880ccd70 tx=0x7f088809b1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cf732c35e701cf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:14.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.297+0000 7f087e7fc640 1 -- 172.21.15.107:0/1382148098 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0888020070 con 0x5624157a3290 2024-04-25T00:25:14.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.297+0000 7f087e7fc640 1 -- 172.21.15.107:0/1382148098 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08880bb040 con 0x5624157a3290 2024-04-25T00:25:14.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.297+0000 7f087e7fc640 1 -- 172.21.15.107:0/1382148098 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0888010460 con 0x5624157a3290 2024-04-25T00:25:14.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.297+0000 7f089dbbe740 1 -- 172.21.15.107:0/1382148098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a3290 msgr2=0x5624157b9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:14.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.297+0000 7f089dbbe740 1 --2- 172.21.15.107:0/1382148098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a3290 0x5624157b9930 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f08880ccd70 tx=0x7f088809b1b0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.297+0000 7f089dbbe740 1 -- 172.21.15.107:0/1382148098 shutdown_connections 2024-04-25T00:25:14.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.297+0000 7f089dbbe740 1 --2- 172.21.15.107:0/1382148098 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624157ae010 0x5624157c38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.297+0000 7f089dbbe740 1 --2- 172.21.15.107:0/1382148098 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624157a4980 0x5624157c3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.297+0000 7f089dbbe740 1 --2- 172.21.15.107:0/1382148098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a3290 0x5624157b9930 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.297+0000 7f089dbbe740 1 -- 172.21.15.107:0/1382148098 >> 172.21.15.107:0/1382148098 conn(0x56241576b950 msgr2=0x5624157b9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:14.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.297+0000 7f089dbbe740 1 -- 172.21.15.107:0/1382148098 shutdown_connections 2024-04-25T00:25:14.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.297+0000 7f089dbbe740 1 -- 172.21.15.107:0/1382148098 wait complete. 2024-04-25T00:25:14.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.298+0000 7f089dbbe740 1 -- start start 2024-04-25T00:25:14.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.298+0000 7f089dbbe740 1 Processor -- start 2024-04-25T00:25:14.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.298+0000 7f089dbbe740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624157a3290 0x5624157c10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.298+0000 7f089dbbe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a4980 0x5624157c15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.298+0000 7f089dbbe740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624157ae010 0x562415823150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.298+0000 7f089dbbe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f08941ad3f0 con 0x5624157a4980 2024-04-25T00:25:14.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.298+0000 7f089dbbe740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f08941ae860 con 0x5624157a3290 2024-04-25T00:25:14.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.298+0000 7f089dbbe740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f08941adac0 con 0x5624157ae010 2024-04-25T00:25:14.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.298+0000 7f089dbbe740 10 client.0 _ll_register_callbacks cb 0x7f0888086030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:14.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.298+0000 7f0893fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a4980 0x5624157c15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.298+0000 7f0893fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a4980 0x5624157c15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32866/0 (socket says 172.21.15.107:32866) 2024-04-25T00:25:14.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.298+0000 7f0893fff640 1 -- 172.21.15.107:0/761259629 learned_addr learned my addr 172.21.15.107:0/761259629 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:14.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.298+0000 7f0893fff640 1 -- 172.21.15.107:0/761259629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624157ae010 msgr2=0x562415823150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:14.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.298+0000 7f0898c4f640 1 --2- 172.21.15.107:0/761259629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624157a3290 0x5624157c10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.298+0000 7f0893fff640 1 --2- 172.21.15.107:0/761259629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624157ae010 0x562415823150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.298+0000 7f0893fff640 1 -- 172.21.15.107:0/761259629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624157a3290 msgr2=0x5624157c10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:14.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.298+0000 7f0893fff640 1 --2- 172.21.15.107:0/761259629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624157a3290 0x5624157c10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.298+0000 7f0893fff640 1 -- 172.21.15.107:0/761259629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624153db170 con 0x5624157a4980 2024-04-25T00:25:14.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.299+0000 7f0893fff640 1 --2- 172.21.15.107:0/761259629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a4980 0x5624157c15f0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f0880000c00 tx=0x7f088001af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f57d530618060a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:14.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.299+0000 7f0873fff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:25:14.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.299+0000 7f0873fff640 1 -- 172.21.15.107:0/761259629 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088001a740 con 0x5624157a4980 2024-04-25T00:25:14.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.299+0000 7f0873fff640 1 -- 172.21.15.107:0/761259629 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9571+0+0 (secure 0 0 0) 0x7f088001a8e0 con 0x5624157a4980 2024-04-25T00:25:14.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.299+0000 7f089dbbe740 1 -- 172.21.15.107:0/761259629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08880bb040 con 0x5624157a4980 2024-04-25T00:25:14.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.299+0000 7f0873fff640 1 -- 172.21.15.107:0/761259629 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f088002a050 con 0x5624157a4980 2024-04-25T00:25:14.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.299+0000 7f089dbbe740 10 client.17598 fetch_fsmap learned FSMap version 34 2024-04-25T00:25:14.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.299+0000 7f089dbbe740 1 -- 172.21.15.107:0/761259629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5624158245d0 con 0x5624157a4980 2024-04-25T00:25:14.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.299+0000 7f0873fff640 1 -- 172.21.15.107:0/761259629 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08880bb040 con 0x5624157a4980 2024-04-25T00:25:14.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.299+0000 7f0873fff640 1 -- 172.21.15.107:0/761259629 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f08800118c0 con 0x5624157a4980 2024-04-25T00:25:14.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.299+0000 7f089dbbe740 10 client.17598 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:25:14.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.299+0000 7f089dbbe740 10 client.17598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:14.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.299+0000 7f089dbbe740 20 client.17598 populate_metadata read hostname 'smithi107' 2024-04-25T00:25:14.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.299+0000 7f089dbbe740 1 --2- 172.21.15.107:0/761259629 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562415824cd0 0x5624158270f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.299+0000 7f089dbbe740 4 client.17598 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:25:14.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.299+0000 7f089dbbe740 1 -- 172.21.15.107:0/761259629 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562415714800 con 0x562415824cd0 2024-04-25T00:25:14.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.300+0000 7f0898c4f640 1 --2- 172.21.15.107:0/761259629 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562415824cd0 0x5624158270f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.300+0000 7f0873fff640 10 client.17598 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:14.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.300+0000 7f0898c4f640 1 --2- 172.21.15.107:0/761259629 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562415824cd0 0x5624158270f0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8514086e381d0dd8 server_cookie=80ffac4b1f147e8b in_seq=0 out_seq=0 2024-04-25T00:25:14.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.301+0000 7f0873fff640 1 -- 172.21.15.107:0/761259629 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562415714800 con 0x562415824cd0 2024-04-25T00:25:14.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.301+0000 7f0873fff640 10 client.17598 handle_command_reply: tid=0 2024-04-25T00:25:14.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.301+0000 7f089dbbe740 1 client.17598 shutdown 2024-04-25T00:25:14.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.302+0000 7f089dbbe740 1 -- 172.21.15.107:0/761259629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a4980 msgr2=0x5624157c15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:14.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.302+0000 7f089dbbe740 1 --2- 172.21.15.107:0/761259629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a4980 0x5624157c15f0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f0880000c00 tx=0x7f088001af90 comp rx=0 tx=0).stop 2024-04-25T00:25:14.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.302+0000 7f089dbbe740 1 -- 172.21.15.107:0/761259629 shutdown_connections 2024-04-25T00:25:14.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.302+0000 7f089dbbe740 1 --2- 172.21.15.107:0/761259629 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x562415824cd0 0x5624158270f0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.302+0000 7f089dbbe740 1 --2- 172.21.15.107:0/761259629 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5624157ae010 0x562415823150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.302+0000 7f089dbbe740 1 --2- 172.21.15.107:0/761259629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5624157a4980 0x5624157c15f0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.302+0000 7f089dbbe740 1 --2- 172.21.15.107:0/761259629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5624157a3290 0x5624157c10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.302+0000 7f089dbbe740 1 -- 172.21.15.107:0/761259629 >> 172.21.15.107:0/761259629 conn(0x56241576b950 msgr2=0x5624157b9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:14.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.302+0000 7f0873fff640 0 client.17598 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:14.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.302+0000 7f089dbbe740 1 -- 172.21.15.107:0/761259629 shutdown_connections 2024-04-25T00:25:14.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.302+0000 7f089dbbe740 1 -- 172.21.15.107:0/761259629 wait complete. 2024-04-25T00:25:14.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.302+0000 7f089dbbe740 20 client.17598 trim_cache size 0 max 16384 2024-04-25T00:25:14.308 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:25:14.308 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-04-25T00:25:14.308 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "5f428943-d7ed-4b8c-918c-8183e0d9620f", 2024-04-25T00:25:14.308 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-04-25T00:25:14.308 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:25:14.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.302+0000 7f089aeb3640 1 -- 172.21.15.107:0/1380768466 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f08840a2d80 msgr2=0x7f08840a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:14.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.302+0000 7f089aeb3640 1 --2- 172.21.15.107:0/1380768466 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f08840a2d80 0x7f08840a5240 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f088c009a60 tx=0x7f088c0093a0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.302+0000 7f089aeb3640 1 -- 172.21.15.107:0/1380768466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0894101710 msgr2=0x7f0894197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:14.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.302+0000 7f089aeb3640 1 --2- 172.21.15.107:0/1380768466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0894101710 0x7f0894197f50 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f088800d950 tx=0x7f088800de10 comp rx=0 tx=0).stop 2024-04-25T00:25:14.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.302+0000 7f089aeb3640 1 -- 172.21.15.107:0/1380768466 shutdown_connections 2024-04-25T00:25:14.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.302+0000 7f089aeb3640 1 --2- 172.21.15.107:0/1380768466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0894107720 0x7f089419c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.302+0000 7f089aeb3640 1 --2- 172.21.15.107:0/1380768466 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f08840a2d80 0x7f08840a5240 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.302+0000 7f089aeb3640 1 --2- 172.21.15.107:0/1380768466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f08941020d0 0x7f0894198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.302+0000 7f089aeb3640 1 --2- 172.21.15.107:0/1380768466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0894101710 0x7f0894197f50 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.302+0000 7f089aeb3640 1 -- 172.21.15.107:0/1380768466 >> 172.21.15.107:0/1380768466 conn(0x7f0894078b90 msgr2=0x7f08940fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:14.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.303+0000 7f089aeb3640 1 -- 172.21.15.107:0/1380768466 shutdown_connections 2024-04-25T00:25:14.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.303+0000 7f089aeb3640 1 -- 172.21.15.107:0/1380768466 wait complete. 2024-04-25T00:25:14.313 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:25:14.314 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:25:14.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.455+0000 7fb41455a640 1 -- 172.21.15.107:0/3110818714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb40c10bd00 msgr2=0x7fb40c10e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:14.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.455+0000 7fb41455a640 1 --2- 172.21.15.107:0/3110818714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb40c10bd00 0x7fb40c10e1a0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fb404009900 tx=0x7fb40402a510 comp rx=0 tx=0).stop 2024-04-25T00:25:14.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.456+0000 7fb41455a640 1 -- 172.21.15.107:0/3110818714 shutdown_connections 2024-04-25T00:25:14.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.456+0000 7fb41455a640 1 --2- 172.21.15.107:0/3110818714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb40c10bd00 0x7fb40c10e1a0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.456+0000 7fb41455a640 1 --2- 172.21.15.107:0/3110818714 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb40c1032e0 0x7fb40c10b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.456+0000 7fb41455a640 1 --2- 172.21.15.107:0/3110818714 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb40c1029c0 0x7fb40c102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.456+0000 7fb41455a640 1 -- 172.21.15.107:0/3110818714 >> 172.21.15.107:0/3110818714 conn(0x7fb40c077740 msgr2=0x7fb40c077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:14.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.456+0000 7fb41455a640 1 -- 172.21.15.107:0/3110818714 shutdown_connections 2024-04-25T00:25:14.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.456+0000 7fb41455a640 1 -- 172.21.15.107:0/3110818714 wait complete. 2024-04-25T00:25:14.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.456+0000 7fb41455a640 1 Processor -- start 2024-04-25T00:25:14.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.456+0000 7fb41455a640 1 -- start start 2024-04-25T00:25:14.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.456+0000 7fb41455a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb40c1029c0 0x7fb40c100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.456+0000 7fb41455a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb40c1032e0 0x7fb40c100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.456+0000 7fb41455a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb40c10bd00 0x7fb40c101090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.456+0000 7fb41455a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c111b00 con 0x7fb40c10bd00 2024-04-25T00:25:14.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.456+0000 7fb41455a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c111980 con 0x7fb40c1032e0 2024-04-25T00:25:14.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.456+0000 7fb41455a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c111c80 con 0x7fb40c1029c0 2024-04-25T00:25:14.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.456+0000 7fb412af7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb40c10bd00 0x7fb40c101090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.457+0000 7fb412af7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb40c10bd00 0x7fb40c101090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32894/0 (socket says 172.21.15.107:32894) 2024-04-25T00:25:14.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.457+0000 7fb412af7640 1 -- 172.21.15.107:0/844114975 learned_addr learned my addr 172.21.15.107:0/844114975 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:14.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.457+0000 7fb4122f6640 1 --2- 172.21.15.107:0/844114975 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb40c1029c0 0x7fb40c100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.457+0000 7fb412af7640 1 -- 172.21.15.107:0/844114975 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb40c1029c0 msgr2=0x7fb40c100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:14.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.457+0000 7fb412af7640 1 --2- 172.21.15.107:0/844114975 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb40c1029c0 0x7fb40c100610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.457+0000 7fb412af7640 1 -- 172.21.15.107:0/844114975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb40c1032e0 msgr2=0x7fb40c100b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:14.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.457+0000 7fb412af7640 1 --2- 172.21.15.107:0/844114975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb40c1032e0 0x7fb40c100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.457+0000 7fb412af7640 1 -- 172.21.15.107:0/844114975 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40c1a64a0 con 0x7fb40c10bd00 2024-04-25T00:25:14.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.457+0000 7fb412af7640 1 --2- 172.21.15.107:0/844114975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb40c10bd00 0x7fb40c101090 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fb404005ab0 tx=0x7fb404004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:14.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.457+0000 7fb4037fe640 1 -- 172.21.15.107:0/844114975 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40403f030 con 0x7fb40c10bd00 2024-04-25T00:25:14.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.457+0000 7fb4037fe640 1 -- 172.21.15.107:0/844114975 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb404003ad0 con 0x7fb40c10bd00 2024-04-25T00:25:14.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.457+0000 7fb4037fe640 1 -- 172.21.15.107:0/844114975 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404035070 con 0x7fb40c10bd00 2024-04-25T00:25:14.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.457+0000 7fb41455a640 1 -- 172.21.15.107:0/844114975 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40c1a6730 con 0x7fb40c10bd00 2024-04-25T00:25:14.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.457+0000 7fb41455a640 1 -- 172.21.15.107:0/844114975 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb40c1a6c20 con 0x7fb40c10bd00 2024-04-25T00:25:14.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.459+0000 7fb4037fe640 1 -- 172.21.15.107:0/844114975 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb404042030 con 0x7fb40c10bd00 2024-04-25T00:25:14.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.460+0000 7fb417265740 1 Processor -- start 2024-04-25T00:25:14.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.460+0000 7fb417265740 1 -- start start 2024-04-25T00:25:14.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.460+0000 7fb417265740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635622c8630 0x5635622c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.460+0000 7fb417265740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622c8f50 0x5635622d30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.460+0000 7fb417265740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635622d3620 0x5635622d5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.460+0000 7fb417265740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56356222d6b0 con 0x5635622c8f50 2024-04-25T00:25:14.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.460+0000 7fb417265740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5635621d5880 con 0x5635622d3620 2024-04-25T00:25:14.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.460+0000 7fb417265740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563562229670 con 0x5635622c8630 2024-04-25T00:25:14.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.460+0000 7fb4037fe640 1 --2- 172.21.15.107:0/844114975 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb3f80a2d30 0x7fb3f80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.460+0000 7fb4122f6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622c8f50 0x5635622d30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.460+0000 7fb4122f6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622c8f50 0x5635622d30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32906/0 (socket says 172.21.15.107:32906) 2024-04-25T00:25:14.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.460+0000 7fb4122f6640 1 -- 172.21.15.107:0/2810264005 learned_addr learned my addr 172.21.15.107:0/2810264005 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:14.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.461+0000 7fb412af7640 1 --2- 172.21.15.107:0/844114975 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb3f80a2d30 0x7fb3f80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.461+0000 7fb411af5640 1 --2- 172.21.15.107:0/2810264005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635622c8630 0x5635622c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.461+0000 7fb411af5640 1 --2- 172.21.15.107:0/2810264005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635622d3620 0x5635622d5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.461+0000 7fb4037fe640 1 -- 172.21.15.107:0/844114975 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9571+0+0 (secure 0 0 0) 0x7fb4040efc20 con 0x7fb40c10bd00 2024-04-25T00:25:14.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.461+0000 7fb4122f6640 1 -- 172.21.15.107:0/2810264005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635622c8630 msgr2=0x5635622c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:14.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.461+0000 7fb4122f6640 1 --2- 172.21.15.107:0/2810264005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635622c8630 0x5635622c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.461+0000 7fb4122f6640 1 -- 172.21.15.107:0/2810264005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635622d3620 msgr2=0x5635622d5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:14.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.461+0000 7fb4122f6640 1 --2- 172.21.15.107:0/2810264005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635622d3620 0x5635622d5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.461+0000 7fb4122f6640 1 -- 172.21.15.107:0/2810264005 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56356200b7d0 con 0x5635622c8f50 2024-04-25T00:25:14.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.461+0000 7fb4122f6640 1 --2- 172.21.15.107:0/2810264005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622c8f50 0x5635622d30e0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fb4080103f0 tx=0x7fb4080108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694dbd4eb8599873 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:14.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.461+0000 7fb412af7640 1 --2- 172.21.15.107:0/844114975 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb3f80a2d30 0x7fb3f80a51f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb4040c4400 tx=0x7fb4040b7f60 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:14.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.461+0000 7fb400bfe640 1 -- 172.21.15.107:0/2810264005 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408016070 con 0x5635622c8f50 2024-04-25T00:25:14.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.461+0000 7fb400bfe640 1 -- 172.21.15.107:0/2810264005 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb408004d20 con 0x5635622c8f50 2024-04-25T00:25:14.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.461+0000 7fb400bfe640 1 -- 172.21.15.107:0/2810264005 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408005420 con 0x5635622c8f50 2024-04-25T00:25:14.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.462+0000 7fb417265740 1 -- 172.21.15.107:0/2810264005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622c8f50 msgr2=0x5635622d30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:14.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.462+0000 7fb417265740 1 --2- 172.21.15.107:0/2810264005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622c8f50 0x5635622d30e0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fb4080103f0 tx=0x7fb4080108b0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.462+0000 7fb417265740 1 -- 172.21.15.107:0/2810264005 shutdown_connections 2024-04-25T00:25:14.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.462+0000 7fb417265740 1 --2- 172.21.15.107:0/2810264005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635622d3620 0x5635622d5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.462+0000 7fb417265740 1 --2- 172.21.15.107:0/2810264005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622c8f50 0x5635622d30e0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.462+0000 7fb417265740 1 --2- 172.21.15.107:0/2810264005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635622c8630 0x5635622c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.462+0000 7fb417265740 1 -- 172.21.15.107:0/2810264005 >> 172.21.15.107:0/2810264005 conn(0x5635622c4950 msgr2=0x5635622c4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:14.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.462+0000 7fb417265740 1 -- 172.21.15.107:0/2810264005 shutdown_connections 2024-04-25T00:25:14.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.462+0000 7fb417265740 1 -- 172.21.15.107:0/2810264005 wait complete. 2024-04-25T00:25:14.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.462+0000 7fb417265740 1 -- start start 2024-04-25T00:25:14.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.462+0000 7fb417265740 1 Processor -- start 2024-04-25T00:25:14.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.463+0000 7fb417265740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622c8630 0x5635622d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.463+0000 7fb417265740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635622c8f50 0x5635622d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.463+0000 7fb417265740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635622d3620 0x5635622d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.463+0000 7fb417265740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5635622d6410 con 0x5635622c8630 2024-04-25T00:25:14.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.463+0000 7fb417265740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5635621e8f50 con 0x5635622d3620 2024-04-25T00:25:14.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.463+0000 7fb417265740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5635622d6590 con 0x5635622c8f50 2024-04-25T00:25:14.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.463+0000 7fb417265740 10 client.0 _ll_register_callbacks cb 0x563562224000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:14.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.463+0000 7fb411af5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622c8630 0x5635622d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.463+0000 7fb411af5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622c8630 0x5635622d05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32910/0 (socket says 172.21.15.107:32910) 2024-04-25T00:25:14.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.463+0000 7fb411af5640 1 -- 172.21.15.107:0/754196149 learned_addr learned my addr 172.21.15.107:0/754196149 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:14.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.463+0000 7fb4122f6640 1 --2- 172.21.15.107:0/754196149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635622c8f50 0x5635622d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.463+0000 7fb411af5640 1 --2- 172.21.15.107:0/754196149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635622d3620 0x5635622d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.463+0000 7fb411af5640 1 -- 172.21.15.107:0/754196149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635622c8f50 msgr2=0x5635622d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:14.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.463+0000 7fb411af5640 1 --2- 172.21.15.107:0/754196149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635622c8f50 0x5635622d0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.463+0000 7fb411af5640 1 -- 172.21.15.107:0/754196149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635622d3620 msgr2=0x5635622d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:14.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.463+0000 7fb411af5640 1 --2- 172.21.15.107:0/754196149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635622d3620 0x5635622d1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.463+0000 7fb411af5640 1 -- 172.21.15.107:0/754196149 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563561f413f0 con 0x5635622c8630 2024-04-25T00:25:14.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.463+0000 7fb411af5640 1 --2- 172.21.15.107:0/754196149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622c8630 0x5635622d05c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc009ef0 tx=0x7fb3fc009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70946b38fdc6e533 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:14.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.463+0000 7fb3f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:25:14.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.463+0000 7fb3f5ffb640 1 -- 172.21.15.107:0/754196149 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc004250 con 0x5635622c8630 2024-04-25T00:25:14.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.463+0000 7fb3f5ffb640 1 -- 172.21.15.107:0/754196149 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9571+0+0 (secure 0 0 0) 0x7fb3fc0043f0 con 0x5635622c8630 2024-04-25T00:25:14.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.463+0000 7fb417265740 1 -- 172.21.15.107:0/754196149 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb408004d20 con 0x5635622c8630 2024-04-25T00:25:14.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.463+0000 7fb3f5ffb640 1 -- 172.21.15.107:0/754196149 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb3fc026050 con 0x5635622c8630 2024-04-25T00:25:14.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.464+0000 7fb417265740 10 client.17610 fetch_fsmap learned FSMap version 34 2024-04-25T00:25:14.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.464+0000 7fb417265740 1 -- 172.21.15.107:0/754196149 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563561f010f0 con 0x5635622c8630 2024-04-25T00:25:14.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.464+0000 7fb3f5ffb640 1 -- 172.21.15.107:0/754196149 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb408004d20 con 0x5635622c8630 2024-04-25T00:25:14.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.464+0000 7fb3f5ffb640 1 -- 172.21.15.107:0/754196149 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fb3fc004cc0 con 0x5635622c8630 2024-04-25T00:25:14.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.464+0000 7fb417265740 10 client.17610 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:25:14.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.464+0000 7fb417265740 10 client.17610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:14.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.464+0000 7fb417265740 20 client.17610 populate_metadata read hostname 'smithi107' 2024-04-25T00:25:14.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.464+0000 7fb417265740 1 --2- 172.21.15.107:0/754196149 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5635622e38a0 0x5635622e5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.464+0000 7fb417265740 4 client.17610 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:14.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.464+0000 7fb417265740 1 -- 172.21.15.107:0/754196149 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56356223abf0 con 0x5635622e38a0 2024-04-25T00:25:14.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.464+0000 7fb4122f6640 1 --2- 172.21.15.107:0/754196149 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5635622e38a0 0x5635622e5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.465+0000 7fb4122f6640 1 --2- 172.21.15.107:0/754196149 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5635622e38a0 0x5635622e5cf0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ae78652c0a9aa3e server_cookie=9e14297c0310df89 in_seq=0 out_seq=0 2024-04-25T00:25:14.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.465+0000 7fb3f5ffb640 10 client.17610 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:14.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.465+0000 7fb3f5ffb640 1 -- 172.21.15.107:0/754196149 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56356223abf0 con 0x5635622e38a0 2024-04-25T00:25:14.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.465+0000 7fb3f5ffb640 10 client.17610 handle_command_reply: tid=0 2024-04-25T00:25:14.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.465+0000 7fb417265740 1 client.17610 shutdown 2024-04-25T00:25:14.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.465+0000 7fb417265740 1 -- 172.21.15.107:0/754196149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622c8630 msgr2=0x5635622d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:14.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.465+0000 7fb417265740 1 --2- 172.21.15.107:0/754196149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622c8630 0x5635622d05c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc009ef0 tx=0x7fb3fc009950 comp rx=0 tx=0).stop 2024-04-25T00:25:14.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.465+0000 7fb417265740 1 -- 172.21.15.107:0/754196149 shutdown_connections 2024-04-25T00:25:14.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.465+0000 7fb417265740 1 --2- 172.21.15.107:0/754196149 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5635622e38a0 0x5635622e5cf0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.465+0000 7fb417265740 1 --2- 172.21.15.107:0/754196149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635622d3620 0x5635622d1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.465+0000 7fb417265740 1 --2- 172.21.15.107:0/754196149 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635622c8f50 0x5635622d0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.465+0000 7fb417265740 1 --2- 172.21.15.107:0/754196149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622c8630 0x5635622d05c0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.465+0000 7fb3f5ffb640 0 client.17610 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:14.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.465+0000 7fb417265740 1 -- 172.21.15.107:0/754196149 >> 172.21.15.107:0/754196149 conn(0x5635622c4950 msgr2=0x5635622ca750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:14.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.465+0000 7fb417265740 1 -- 172.21.15.107:0/754196149 shutdown_connections 2024-04-25T00:25:14.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.465+0000 7fb417265740 1 -- 172.21.15.107:0/754196149 wait complete. 2024-04-25T00:25:14.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.466+0000 7fb417265740 20 client.17610 trim_cache size 0 max 16384 2024-04-25T00:25:14.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.483+0000 7fb417265740 1 Processor -- start 2024-04-25T00:25:14.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.483+0000 7fb417265740 1 -- start start 2024-04-25T00:25:14.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.483+0000 7fb417265740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635622c8630 0x5635622df2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.483+0000 7fb417265740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622c8f50 0x5635622c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.483+0000 7fb417265740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635622d3620 0x5635622caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.483+0000 7fb417265740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c1b3850 con 0x5635622c8f50 2024-04-25T00:25:14.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.483+0000 7fb417265740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c1b1120 con 0x5635622d3620 2024-04-25T00:25:14.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.483+0000 7fb417265740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c1aef90 con 0x5635622c8630 2024-04-25T00:25:14.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.483+0000 7fb4122f6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622c8f50 0x5635622c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.483+0000 7fb4122f6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622c8f50 0x5635622c4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32914/0 (socket says 172.21.15.107:32914) 2024-04-25T00:25:14.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.483+0000 7fb4122f6640 1 -- 172.21.15.107:0/4072879521 learned_addr learned my addr 172.21.15.107:0/4072879521 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:14.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.483+0000 7fb411af5640 1 --2- 172.21.15.107:0/4072879521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635622c8630 0x5635622df2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.483+0000 7fb4122f6640 1 -- 172.21.15.107:0/4072879521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635622c8630 msgr2=0x5635622df2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:14.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.483+0000 7fb4122f6640 1 --2- 172.21.15.107:0/4072879521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635622c8630 0x5635622df2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.483+0000 7fb411af5640 1 --2- 172.21.15.107:0/4072879521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635622d3620 0x5635622caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.483+0000 7fb4122f6640 1 -- 172.21.15.107:0/4072879521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635622d3620 msgr2=0x5635622caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:14.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.483+0000 7fb4122f6640 1 --2- 172.21.15.107:0/4072879521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635622d3620 0x5635622caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.483+0000 7fb4122f6640 1 -- 172.21.15.107:0/4072879521 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635622b6090 con 0x5635622c8f50 2024-04-25T00:25:14.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.483+0000 7fb4122f6640 1 --2- 172.21.15.107:0/4072879521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622c8f50 0x5635622c4d30 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb408019000 tx=0x7fb408009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1d226f7f0dcfda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:14.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.484+0000 7fb3f7fff640 1 -- 172.21.15.107:0/4072879521 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408016040 con 0x5635622c8f50 2024-04-25T00:25:14.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.484+0000 7fb3f7fff640 1 -- 172.21.15.107:0/4072879521 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb40802d070 con 0x5635622c8f50 2024-04-25T00:25:14.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.484+0000 7fb3f7fff640 1 -- 172.21.15.107:0/4072879521 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb408028940 con 0x5635622c8f50 2024-04-25T00:25:14.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.484+0000 7fb417265740 1 -- 172.21.15.107:0/4072879521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622c8f50 msgr2=0x5635622c4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:14.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.484+0000 7fb417265740 1 --2- 172.21.15.107:0/4072879521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622c8f50 0x5635622c4d30 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb408019000 tx=0x7fb408009c30 comp rx=0 tx=0).stop 2024-04-25T00:25:14.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.484+0000 7fb417265740 1 -- 172.21.15.107:0/4072879521 shutdown_connections 2024-04-25T00:25:14.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.484+0000 7fb417265740 1 --2- 172.21.15.107:0/4072879521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635622d3620 0x5635622caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.484+0000 7fb417265740 1 --2- 172.21.15.107:0/4072879521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622c8f50 0x5635622c4d30 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.484+0000 7fb417265740 1 --2- 172.21.15.107:0/4072879521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635622c8630 0x5635622df2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.484+0000 7fb417265740 1 -- 172.21.15.107:0/4072879521 >> 172.21.15.107:0/4072879521 conn(0x5635622c4950 msgr2=0x5635622d0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:14.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.484+0000 7fb417265740 1 -- 172.21.15.107:0/4072879521 shutdown_connections 2024-04-25T00:25:14.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.484+0000 7fb417265740 1 -- 172.21.15.107:0/4072879521 wait complete. 2024-04-25T00:25:14.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.485+0000 7fb417265740 1 -- start start 2024-04-25T00:25:14.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.485+0000 7fb417265740 1 Processor -- start 2024-04-25T00:25:14.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.485+0000 7fb417265740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635622c8630 0x5635622db860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.485+0000 7fb417265740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635622c8f50 0x5635622d6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.485+0000 7fb417265740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622d3620 0x5635622d7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.485+0000 7fb417265740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c1af5e0 con 0x5635622d3620 2024-04-25T00:25:14.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.485+0000 7fb417265740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c1b0a50 con 0x5635622c8f50 2024-04-25T00:25:14.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.485+0000 7fb417265740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c1afcb0 con 0x5635622c8630 2024-04-25T00:25:14.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.485+0000 7fb417265740 10 client.0 _ll_register_callbacks cb 0x7fb3fc009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:14.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.485+0000 7fb411af5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622d3620 0x5635622d7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.485+0000 7fb411af5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622d3620 0x5635622d7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32930/0 (socket says 172.21.15.107:32930) 2024-04-25T00:25:14.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.485+0000 7fb411af5640 1 -- 172.21.15.107:0/987717135 learned_addr learned my addr 172.21.15.107:0/987717135 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:14.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.485+0000 7fb4122f6640 1 --2- 172.21.15.107:0/987717135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635622c8f50 0x5635622d6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.485+0000 7fb411af5640 1 --2- 172.21.15.107:0/987717135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635622c8630 0x5635622db860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.485+0000 7fb411af5640 1 -- 172.21.15.107:0/987717135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635622c8630 msgr2=0x5635622db860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:14.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.485+0000 7fb411af5640 1 --2- 172.21.15.107:0/987717135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635622c8630 0x5635622db860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.485+0000 7fb411af5640 1 -- 172.21.15.107:0/987717135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635622c8f50 msgr2=0x5635622d6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:14.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.485+0000 7fb411af5640 1 --2- 172.21.15.107:0/987717135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635622c8f50 0x5635622d6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.485+0000 7fb411af5640 1 -- 172.21.15.107:0/987717135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635622d7840 con 0x5635622d3620 2024-04-25T00:25:14.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.485+0000 7fb411af5640 1 --2- 172.21.15.107:0/987717135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622d3620 0x5635622d7300 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0059f0 tx=0x7fb3fc021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897c186fbe62774c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:14.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.485+0000 7fb3f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:25:14.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.486+0000 7fb3f5ffb640 1 -- 172.21.15.107:0/987717135 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc003c60 con 0x5635622d3620 2024-04-25T00:25:14.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.486+0000 7fb3f5ffb640 1 -- 172.21.15.107:0/987717135 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9571+0+0 (secure 0 0 0) 0x7fb3fc003e00 con 0x5635622d3620 2024-04-25T00:25:14.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.486+0000 7fb417265740 1 -- 172.21.15.107:0/987717135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb40802d070 con 0x5635622d3620 2024-04-25T00:25:14.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.486+0000 7fb3f5ffb640 1 -- 172.21.15.107:0/987717135 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb3fc004000 con 0x5635622d3620 2024-04-25T00:25:14.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.486+0000 7fb417265740 10 client.17616 fetch_fsmap learned FSMap version 34 2024-04-25T00:25:14.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.486+0000 7fb417265740 1 -- 172.21.15.107:0/987717135 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563562349b30 con 0x5635622d3620 2024-04-25T00:25:14.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.486+0000 7fb3f5ffb640 1 -- 172.21.15.107:0/987717135 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb40802d070 con 0x5635622d3620 2024-04-25T00:25:14.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.486+0000 7fb3f5ffb640 1 -- 172.21.15.107:0/987717135 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fb3fc01e770 con 0x5635622d3620 2024-04-25T00:25:14.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.486+0000 7fb417265740 10 client.17616 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:25:14.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.486+0000 7fb417265740 10 client.17616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:14.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.486+0000 7fb417265740 20 client.17616 populate_metadata read hostname 'smithi107' 2024-04-25T00:25:14.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.486+0000 7fb417265740 1 --2- 172.21.15.107:0/987717135 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56356234a130 0x56356234c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:14.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.486+0000 7fb417265740 4 client.17616 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:25:14.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.486+0000 7fb417265740 1 -- 172.21.15.107:0/987717135 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563562234f40 con 0x56356234a130 2024-04-25T00:25:14.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.486+0000 7fb4122f6640 1 --2- 172.21.15.107:0/987717135 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56356234a130 0x56356234c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:14.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.487+0000 7fb4122f6640 1 --2- 172.21.15.107:0/987717135 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56356234a130 0x56356234c520 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17c6a9f488659837 server_cookie=bc8f4c78d590d495 in_seq=0 out_seq=0 2024-04-25T00:25:14.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.487+0000 7fb3f5ffb640 10 client.17616 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:14.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.487+0000 7fb3f5ffb640 1 -- 172.21.15.107:0/987717135 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563562234f40 con 0x56356234a130 2024-04-25T00:25:14.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.487+0000 7fb3f5ffb640 10 client.17616 handle_command_reply: tid=0 2024-04-25T00:25:14.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.487+0000 7fb417265740 1 client.17616 shutdown 2024-04-25T00:25:14.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.487+0000 7fb417265740 1 -- 172.21.15.107:0/987717135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622d3620 msgr2=0x5635622d7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:14.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.487+0000 7fb417265740 1 --2- 172.21.15.107:0/987717135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622d3620 0x5635622d7300 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0059f0 tx=0x7fb3fc021040 comp rx=0 tx=0).stop 2024-04-25T00:25:14.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.487+0000 7fb417265740 1 -- 172.21.15.107:0/987717135 shutdown_connections 2024-04-25T00:25:14.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.487+0000 7fb417265740 1 --2- 172.21.15.107:0/987717135 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56356234a130 0x56356234c520 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.487+0000 7fb417265740 1 --2- 172.21.15.107:0/987717135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5635622d3620 0x5635622d7300 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.487+0000 7fb417265740 1 --2- 172.21.15.107:0/987717135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5635622c8f50 0x5635622d6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.487+0000 7fb417265740 1 --2- 172.21.15.107:0/987717135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5635622c8630 0x5635622db860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.487+0000 7fb3f5ffb640 0 client.17616 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:14.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.487+0000 7fb417265740 1 -- 172.21.15.107:0/987717135 >> 172.21.15.107:0/987717135 conn(0x5635622c4950 msgr2=0x5635622d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:14.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.487+0000 7fb417265740 1 -- 172.21.15.107:0/987717135 shutdown_connections 2024-04-25T00:25:14.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.487+0000 7fb417265740 1 -- 172.21.15.107:0/987717135 wait complete. 2024-04-25T00:25:14.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.487+0000 7fb417265740 20 client.17616 trim_cache size 0 max 16384 2024-04-25T00:25:14.495 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:25:14.495 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:25:14.496 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-04-25T00:25:14.496 INFO:teuthology.orchestra.run.smithi107.stdout: "5f428943-d7ed-4b8c-918c-8183e0d9620f": { 2024-04-25T00:25:14.496 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-04-25T00:25:14.496 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "5f428943-d7ed-4b8c-918c-8183e0d9620f", 2024-04-25T00:25:14.496 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-04-25T00:25:14.496 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:25:14.496 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:25:14.496 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:25:14.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.488+0000 7fb41455a640 1 -- 172.21.15.107:0/844114975 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb3f80a2d30 msgr2=0x7fb3f80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:14.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.488+0000 7fb41455a640 1 --2- 172.21.15.107:0/844114975 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb3f80a2d30 0x7fb3f80a51f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb4040c4400 tx=0x7fb4040b7f60 comp rx=0 tx=0).stop 2024-04-25T00:25:14.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.488+0000 7fb41455a640 1 -- 172.21.15.107:0/844114975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb40c10bd00 msgr2=0x7fb40c101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:14.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.488+0000 7fb41455a640 1 --2- 172.21.15.107:0/844114975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb40c10bd00 0x7fb40c101090 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fb404005ab0 tx=0x7fb404004270 comp rx=0 tx=0).stop 2024-04-25T00:25:14.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.488+0000 7fb41455a640 1 -- 172.21.15.107:0/844114975 shutdown_connections 2024-04-25T00:25:14.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.488+0000 7fb41455a640 1 --2- 172.21.15.107:0/844114975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb40c10bd00 0x7fb40c101090 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.488+0000 7fb41455a640 1 --2- 172.21.15.107:0/844114975 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb3f80a2d30 0x7fb3f80a51f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.488+0000 7fb41455a640 1 --2- 172.21.15.107:0/844114975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb40c1032e0 0x7fb40c100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.488+0000 7fb41455a640 1 --2- 172.21.15.107:0/844114975 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb40c1029c0 0x7fb40c100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:14.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.488+0000 7fb41455a640 1 -- 172.21.15.107:0/844114975 >> 172.21.15.107:0/844114975 conn(0x7fb40c077740 msgr2=0x7fb40c0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:14.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.488+0000 7fb41455a640 1 -- 172.21.15.107:0/844114975 shutdown_connections 2024-04-25T00:25:14.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:14.488+0000 7fb41455a640 1 -- 172.21.15.107:0/844114975 wait complete. 2024-04-25T00:25:14.499 INFO:tasks.cephfs.filesystem:scrub status for tag:5f428943-d7ed-4b8c-918c-8183e0d9620f - {'path': '/', 'tag': '5f428943-d7ed-4b8c-918c-8183e0d9620f', 'options': 'recursive,force'} 2024-04-25T00:25:15.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:14 smithi195 ceph-mon[32833]: scrub queued for path: / 2024-04-25T00:25:15.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:14 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:25:15.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:14 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:25:15.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:14 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:25:15.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:14 smithi170 ceph-mon[32807]: scrub queued for path: / 2024-04-25T00:25:15.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:14 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:25:15.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:14 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:25:15.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:14 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:25:15.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:14 smithi107 ceph-mon[29153]: scrub queued for path: / 2024-04-25T00:25:15.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:14 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:25:15.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:14 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:25:15.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:14 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:25:16.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:15 smithi195 ceph-mon[32833]: pgmap v272: 129 pgs: 6 active+clean+snaptrim, 109 active+clean, 14 active+clean+snaptrim_wait; 195 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:25:16.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:15 smithi170 ceph-mon[32807]: pgmap v272: 129 pgs: 6 active+clean+snaptrim, 109 active+clean, 14 active+clean+snaptrim_wait; 195 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:25:16.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:15 smithi107 ceph-mon[29153]: pgmap v272: 129 pgs: 6 active+clean+snaptrim, 109 active+clean, 14 active+clean+snaptrim_wait; 195 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:25:18.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:17 smithi107 ceph-mon[29153]: pgmap v273: 129 pgs: 6 active+clean+snaptrim, 109 active+clean, 14 active+clean+snaptrim_wait; 195 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:25:18.036 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:17 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:25:18.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:17 smithi195 ceph-mon[32833]: pgmap v273: 129 pgs: 6 active+clean+snaptrim, 109 active+clean, 14 active+clean+snaptrim_wait; 195 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:25:18.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:17 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:25:18.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:17 smithi170 ceph-mon[32807]: pgmap v273: 129 pgs: 6 active+clean+snaptrim, 109 active+clean, 14 active+clean+snaptrim_wait; 195 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:25:18.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:17 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:25:20.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:19 smithi195 ceph-mon[32833]: pgmap v274: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 192 B/s wr, 6 op/s 2024-04-25T00:25:20.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:19 smithi170 ceph-mon[32807]: pgmap v274: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 192 B/s wr, 6 op/s 2024-04-25T00:25:20.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:19 smithi107 ceph-mon[29153]: pgmap v274: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 192 B/s wr, 6 op/s 2024-04-25T00:25:22.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:21 smithi195 ceph-mon[32833]: pgmap v275: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:25:22.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:21 smithi195 ceph-mon[32833]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T00:25:22.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:21 smithi170 ceph-mon[32807]: pgmap v275: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:25:22.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:21 smithi170 ceph-mon[32807]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T00:25:22.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:21 smithi107 ceph-mon[29153]: pgmap v275: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:25:22.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:21 smithi107 ceph-mon[29153]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T00:25:24.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:23 smithi195 ceph-mon[32833]: pgmap v277: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:25:24.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:23 smithi170 ceph-mon[32807]: pgmap v277: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:25:24.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:23 smithi107 ceph-mon[29153]: pgmap v277: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:25:26.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:25 smithi195 ceph-mon[32833]: pgmap v278: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:25:26.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:25 smithi170 ceph-mon[32807]: pgmap v278: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:25:26.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:25 smithi107 ceph-mon[29153]: pgmap v278: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:25:28.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:27 smithi195 ceph-mon[32833]: pgmap v279: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:25:28.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:27 smithi170 ceph-mon[32807]: pgmap v279: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:25:28.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:27 smithi107 ceph-mon[29153]: pgmap v279: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:25:30.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:29 smithi195 ceph-mon[32833]: pgmap v280: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T00:25:30.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:29 smithi170 ceph-mon[32807]: pgmap v280: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T00:25:30.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:29 smithi107 ceph-mon[29153]: pgmap v280: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T00:25:32.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:31 smithi195 ceph-mon[32833]: pgmap v281: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T00:25:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:31 smithi170 ceph-mon[32807]: pgmap v281: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T00:25:32.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:31 smithi107 ceph-mon[29153]: pgmap v281: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T00:25:34.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:33 smithi195 ceph-mon[32833]: pgmap v282: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:25:34.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:33 smithi170 ceph-mon[32807]: pgmap v282: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:25:34.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:33 smithi107 ceph-mon[29153]: pgmap v282: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-25T00:25:36.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:35 smithi195 ceph-mon[32833]: pgmap v283: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:25:36.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:35 smithi170 ceph-mon[32807]: pgmap v283: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:25:36.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:35 smithi107 ceph-mon[29153]: pgmap v283: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:25:38.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:37 smithi195 ceph-mon[32833]: pgmap v284: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:25:38.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:37 smithi170 ceph-mon[32807]: pgmap v284: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:25:38.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:37 smithi107 ceph-mon[29153]: pgmap v284: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:25:40.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:39 smithi195 ceph-mon[32833]: pgmap v285: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:25:40.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:39 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:25:40.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:39 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:40.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:39 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:25:40.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:39 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:25:40.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:39 smithi170 ceph-mon[32807]: pgmap v285: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:25:40.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:39 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:25:40.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:39 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:40.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:39 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:25:40.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:39 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:25:40.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:39 smithi107 ceph-mon[29153]: pgmap v285: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:25:40.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:39 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:25:40.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:39 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:40.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:39 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:25:40.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:39 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:25:42.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:41 smithi195 ceph-mon[32833]: pgmap v286: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:25:42.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:41 smithi170 ceph-mon[32807]: pgmap v286: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:25:42.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:41 smithi107 ceph-mon[29153]: pgmap v286: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:25:44.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:43 smithi195 ceph-mon[32833]: pgmap v287: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:25:44.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:43 smithi170 ceph-mon[32807]: pgmap v287: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:25:44.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:43 smithi107 ceph-mon[29153]: pgmap v287: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:25:44.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:25:44.500 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:25:44.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.611+0000 7f216fc9a640 1 -- 172.21.15.107:0/1489577779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2168069dc0 msgr2=0x7f216806a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:44.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.611+0000 7f216fc9a640 1 --2- 172.21.15.107:0/1489577779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2168069dc0 0x7f216806a1a0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f2164009900 tx=0x7f216402a410 comp rx=0 tx=0).stop 2024-04-25T00:25:44.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.611+0000 7f216fc9a640 1 -- 172.21.15.107:0/1489577779 shutdown_connections 2024-04-25T00:25:44.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.611+0000 7f216fc9a640 1 --2- 172.21.15.107:0/1489577779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f216810e150 0x7f2168110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.611+0000 7f216fc9a640 1 --2- 172.21.15.107:0/1489577779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f216806a6e0 0x7f216810dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.611+0000 7f216fc9a640 1 --2- 172.21.15.107:0/1489577779 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2168069dc0 0x7f216806a1a0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.611+0000 7f216fc9a640 1 -- 172.21.15.107:0/1489577779 >> 172.21.15.107:0/1489577779 conn(0x7f21680776a0 msgr2=0x7f2168077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.611+0000 7f216fc9a640 1 -- 172.21.15.107:0/1489577779 shutdown_connections 2024-04-25T00:25:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.611+0000 7f216fc9a640 1 -- 172.21.15.107:0/1489577779 wait complete. 2024-04-25T00:25:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.612+0000 7f216fc9a640 1 Processor -- start 2024-04-25T00:25:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.612+0000 7f216fc9a640 1 -- start start 2024-04-25T00:25:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.612+0000 7f216fc9a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2168069dc0 0x7f216819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.612+0000 7f216fc9a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f216806a6e0 0x7f216819a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.612+0000 7f216fc9a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f216810e150 0x7f216819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.612+0000 7f216fc9a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2168113d70 con 0x7f216810e150 2024-04-25T00:25:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.612+0000 7f216fc9a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2168113bf0 con 0x7f216806a6e0 2024-04-25T00:25:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.612+0000 7f216fc9a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2168113ef0 con 0x7f2168069dc0 2024-04-25T00:25:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.612+0000 7f216e237640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f216810e150 0x7f216819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.612+0000 7f216e237640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f216810e150 0x7f216819eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55772/0 (socket says 172.21.15.107:55772) 2024-04-25T00:25:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.612+0000 7f216e237640 1 -- 172.21.15.107:0/2415159670 learned_addr learned my addr 172.21.15.107:0/2415159670 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:44.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.612+0000 7f216da36640 1 --2- 172.21.15.107:0/2415159670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2168069dc0 0x7f216819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:44.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.612+0000 7f216e237640 1 -- 172.21.15.107:0/2415159670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2168069dc0 msgr2=0x7f216819a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:44.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.612+0000 7f216e237640 1 --2- 172.21.15.107:0/2415159670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2168069dc0 0x7f216819a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.612+0000 7f216d235640 1 --2- 172.21.15.107:0/2415159670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f216806a6e0 0x7f216819a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:44.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.612+0000 7f216e237640 1 -- 172.21.15.107:0/2415159670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f216806a6e0 msgr2=0x7f216819a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:44.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.612+0000 7f216e237640 1 --2- 172.21.15.107:0/2415159670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f216806a6e0 0x7f216819a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.612+0000 7f216e237640 1 -- 172.21.15.107:0/2415159670 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216819f280 con 0x7f216810e150 2024-04-25T00:25:44.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.612+0000 7f216da36640 1 --2- 172.21.15.107:0/2415159670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2168069dc0 0x7f216819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:44.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.613+0000 7f216e237640 1 --2- 172.21.15.107:0/2415159670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f216810e150 0x7f216819eb50 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f216000cab0 tx=0x7f216000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:44.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.613+0000 7f215effd640 1 -- 172.21.15.107:0/2415159670 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216000ae20 con 0x7f216810e150 2024-04-25T00:25:44.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.613+0000 7f215effd640 1 -- 172.21.15.107:0/2415159670 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f216000f930 con 0x7f216810e150 2024-04-25T00:25:44.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.613+0000 7f216fc9a640 1 -- 172.21.15.107:0/2415159670 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f216819f570 con 0x7f216810e150 2024-04-25T00:25:44.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.613+0000 7f215effd640 1 -- 172.21.15.107:0/2415159670 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2160018b80 con 0x7f216810e150 2024-04-25T00:25:44.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.613+0000 7f216fc9a640 1 -- 172.21.15.107:0/2415159670 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21681a6e00 con 0x7f216810e150 2024-04-25T00:25:44.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.615+0000 7f215effd640 1 -- 172.21.15.107:0/2415159670 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f216000fae0 con 0x7f216810e150 2024-04-25T00:25:44.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.615+0000 7f21729a6740 1 Processor -- start 2024-04-25T00:25:44.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.616+0000 7f21729a6740 1 -- start start 2024-04-25T00:25:44.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.616+0000 7f21729a6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5af630 0x56523c5afa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:44.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.616+0000 7f21729a6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56523c5aff50 0x56523c5ba0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:44.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.616+0000 7f21729a6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56523c5ba620 0x56523c5bca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:44.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.616+0000 7f21729a6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56523c5146b0 con 0x56523c5af630 2024-04-25T00:25:44.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.616+0000 7f21729a6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56523c4bc880 con 0x56523c5ba620 2024-04-25T00:25:44.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.616+0000 7f215effd640 1 --2- 172.21.15.107:0/2415159670 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f21540a2d00 0x7f21540a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:44.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.616+0000 7f21729a6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56523c510670 con 0x56523c5aff50 2024-04-25T00:25:44.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.616+0000 7f216d235640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5af630 0x56523c5afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:44.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.616+0000 7f216d235640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5af630 0x56523c5afa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55782/0 (socket says 172.21.15.107:55782) 2024-04-25T00:25:44.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.616+0000 7f216e237640 1 --2- 172.21.15.107:0/2415159670 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f21540a2d00 0x7f21540a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:44.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.616+0000 7f216d235640 1 -- 172.21.15.107:0/1703345651 learned_addr learned my addr 172.21.15.107:0/1703345651 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:44.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.616+0000 7f215effd640 1 -- 172.21.15.107:0/2415159670 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7f21600cffd0 con 0x7f216810e150 2024-04-25T00:25:44.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.616+0000 7f216da36640 1 --2- 172.21.15.107:0/1703345651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56523c5aff50 0x56523c5ba0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:44.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.616+0000 7f216d235640 1 --2- 172.21.15.107:0/1703345651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56523c5ba620 0x56523c5bca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:44.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.616+0000 7f216d235640 1 -- 172.21.15.107:0/1703345651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56523c5aff50 msgr2=0x56523c5ba0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:44.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.616+0000 7f216d235640 1 --2- 172.21.15.107:0/1703345651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56523c5aff50 0x56523c5ba0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.616+0000 7f216d235640 1 -- 172.21.15.107:0/1703345651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56523c5ba620 msgr2=0x56523c5bca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:44.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.616+0000 7f216d235640 1 --2- 172.21.15.107:0/1703345651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56523c5ba620 0x56523c5bca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.616+0000 7f216d235640 1 -- 172.21.15.107:0/1703345651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56523c2f27d0 con 0x56523c5af630 2024-04-25T00:25:44.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.617+0000 7f216d235640 1 --2- 172.21.15.107:0/1703345651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5af630 0x56523c5afa10 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f215800aed0 tx=0x7f215800a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b4770f5d196f1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:44.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.617+0000 7f216e237640 1 --2- 172.21.15.107:0/2415159670 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f21540a2d00 0x7f21540a51c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f21600cd410 tx=0x7f21600915c0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:44.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.617+0000 7f2153fff640 1 -- 172.21.15.107:0/1703345651 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215801b070 con 0x56523c5af630 2024-04-25T00:25:44.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.617+0000 7f2153fff640 1 -- 172.21.15.107:0/1703345651 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f215800edc0 con 0x56523c5af630 2024-04-25T00:25:44.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.617+0000 7f2153fff640 1 -- 172.21.15.107:0/1703345651 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215800f040 con 0x56523c5af630 2024-04-25T00:25:44.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.618+0000 7f21729a6740 1 -- 172.21.15.107:0/1703345651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5af630 msgr2=0x56523c5afa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:44.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.618+0000 7f21729a6740 1 --2- 172.21.15.107:0/1703345651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5af630 0x56523c5afa10 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f215800aed0 tx=0x7f215800a320 comp rx=0 tx=0).stop 2024-04-25T00:25:44.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.618+0000 7f21729a6740 1 -- 172.21.15.107:0/1703345651 shutdown_connections 2024-04-25T00:25:44.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.618+0000 7f21729a6740 1 --2- 172.21.15.107:0/1703345651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56523c5ba620 0x56523c5bca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.618+0000 7f21729a6740 1 --2- 172.21.15.107:0/1703345651 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56523c5aff50 0x56523c5ba0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.618+0000 7f21729a6740 1 --2- 172.21.15.107:0/1703345651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5af630 0x56523c5afa10 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.618+0000 7f21729a6740 1 -- 172.21.15.107:0/1703345651 >> 172.21.15.107:0/1703345651 conn(0x56523c5ab950 msgr2=0x56523c5abd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:44.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.618+0000 7f21729a6740 1 -- 172.21.15.107:0/1703345651 shutdown_connections 2024-04-25T00:25:44.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.618+0000 7f21729a6740 1 -- 172.21.15.107:0/1703345651 wait complete. 2024-04-25T00:25:44.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.618+0000 7f21729a6740 1 -- start start 2024-04-25T00:25:44.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.618+0000 7f21729a6740 1 Processor -- start 2024-04-25T00:25:44.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.619+0000 7f21729a6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56523c5af630 0x56523c5b75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:44.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.619+0000 7f21729a6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56523c5aff50 0x56523c5b7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:44.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.619+0000 7f21729a6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5ba620 0x56523c5b8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:44.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.619+0000 7f21729a6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56523c5bd410 con 0x56523c5ba620 2024-04-25T00:25:44.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.619+0000 7f21729a6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56523c4cff50 con 0x56523c5aff50 2024-04-25T00:25:44.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.619+0000 7f21729a6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56523c5bd590 con 0x56523c5af630 2024-04-25T00:25:44.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.619+0000 7f21729a6740 10 client.0 _ll_register_callbacks cb 0x56523c50b000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:44.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.619+0000 7f216d235640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5ba620 0x56523c5b8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:44.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.619+0000 7f216d235640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5ba620 0x56523c5b8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55794/0 (socket says 172.21.15.107:55794) 2024-04-25T00:25:44.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.619+0000 7f216d235640 1 -- 172.21.15.107:0/240372738 learned_addr learned my addr 172.21.15.107:0/240372738 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:44.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.619+0000 7f216d235640 1 --2- 172.21.15.107:0/240372738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56523c5af630 0x56523c5b75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:44.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.619+0000 7f216da36640 1 --2- 172.21.15.107:0/240372738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56523c5aff50 0x56523c5b7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:44.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.619+0000 7f216d235640 1 -- 172.21.15.107:0/240372738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56523c5af630 msgr2=0x56523c5b75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:44.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.619+0000 7f216d235640 1 --2- 172.21.15.107:0/240372738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56523c5af630 0x56523c5b75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.619+0000 7f216d235640 1 -- 172.21.15.107:0/240372738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56523c5aff50 msgr2=0x56523c5b7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:44.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.619+0000 7f216d235640 1 --2- 172.21.15.107:0/240372738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56523c5aff50 0x56523c5b7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.619+0000 7f216d235640 1 -- 172.21.15.107:0/240372738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56523c2283f0 con 0x56523c5ba620 2024-04-25T00:25:44.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.619+0000 7f216da36640 1 --2- 172.21.15.107:0/240372738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56523c5aff50 0x56523c5b7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:44.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.619+0000 7f216d235640 1 --2- 172.21.15.107:0/240372738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5ba620 0x56523c5b8060 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f2158004700 tx=0x7f2158016ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4384086a1479f87e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:44.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.619+0000 7f21517fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:25:44.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.619+0000 7f21517fa640 1 -- 172.21.15.107:0/240372738 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215801b070 con 0x56523c5ba620 2024-04-25T00:25:44.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.619+0000 7f21517fa640 1 -- 172.21.15.107:0/240372738 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7f215800ebc0 con 0x56523c5ba620 2024-04-25T00:25:44.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.619+0000 7f21729a6740 1 -- 172.21.15.107:0/240372738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f215800edc0 con 0x56523c5ba620 2024-04-25T00:25:44.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.620+0000 7f21517fa640 1 -- 172.21.15.107:0/240372738 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f21580059c0 con 0x56523c5ba620 2024-04-25T00:25:44.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.620+0000 7f21729a6740 10 client.17628 fetch_fsmap learned FSMap version 34 2024-04-25T00:25:44.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.620+0000 7f21729a6740 1 -- 172.21.15.107:0/240372738 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56523c1e80f0 con 0x56523c5ba620 2024-04-25T00:25:44.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.620+0000 7f21517fa640 1 -- 172.21.15.107:0/240372738 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f215800edc0 con 0x56523c5ba620 2024-04-25T00:25:44.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.620+0000 7f21517fa640 1 -- 172.21.15.107:0/240372738 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f2158015070 con 0x56523c5ba620 2024-04-25T00:25:44.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.620+0000 7f21729a6740 10 client.17628 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:25:44.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.620+0000 7f21729a6740 10 client.17628 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:44.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.620+0000 7f21729a6740 20 client.17628 populate_metadata read hostname 'smithi107' 2024-04-25T00:25:44.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.620+0000 7f21729a6740 1 --2- 172.21.15.107:0/240372738 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56523c5ca880 0x56523c5ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:44.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.620+0000 7f21729a6740 4 client.17628 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:44.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.620+0000 7f21729a6740 1 -- 172.21.15.107:0/240372738 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56523c521bf0 con 0x56523c5ca880 2024-04-25T00:25:44.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.620+0000 7f216da36640 1 --2- 172.21.15.107:0/240372738 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56523c5ca880 0x56523c5ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:44.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.621+0000 7f216da36640 1 --2- 172.21.15.107:0/240372738 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56523c5ca880 0x56523c5ccc70 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f22a52ffa8c13ef server_cookie=78b0decfe5269c2b in_seq=0 out_seq=0 2024-04-25T00:25:44.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.621+0000 7f21517fa640 10 client.17628 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:44.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.622+0000 7f21517fa640 1 -- 172.21.15.107:0/240372738 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56523c521bf0 con 0x56523c5ca880 2024-04-25T00:25:44.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.622+0000 7f21517fa640 10 client.17628 handle_command_reply: tid=0 2024-04-25T00:25:44.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.622+0000 7f21729a6740 1 client.17628 shutdown 2024-04-25T00:25:44.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.622+0000 7f21729a6740 1 -- 172.21.15.107:0/240372738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5ba620 msgr2=0x56523c5b8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:44.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.622+0000 7f21729a6740 1 --2- 172.21.15.107:0/240372738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5ba620 0x56523c5b8060 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f2158004700 tx=0x7f2158016ee0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.622+0000 7f21729a6740 1 -- 172.21.15.107:0/240372738 shutdown_connections 2024-04-25T00:25:44.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.622+0000 7f21729a6740 1 --2- 172.21.15.107:0/240372738 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56523c5ca880 0x56523c5ccc70 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.622+0000 7f21729a6740 1 --2- 172.21.15.107:0/240372738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5ba620 0x56523c5b8060 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.622+0000 7f21729a6740 1 --2- 172.21.15.107:0/240372738 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56523c5aff50 0x56523c5b7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.622+0000 7f21517fa640 0 client.17628 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:44.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.622+0000 7f21729a6740 1 --2- 172.21.15.107:0/240372738 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56523c5af630 0x56523c5b75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.622+0000 7f21729a6740 1 -- 172.21.15.107:0/240372738 >> 172.21.15.107:0/240372738 conn(0x56523c5ab950 msgr2=0x56523c5b1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:44.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.622+0000 7f21729a6740 1 -- 172.21.15.107:0/240372738 shutdown_connections 2024-04-25T00:25:44.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.622+0000 7f21729a6740 1 -- 172.21.15.107:0/240372738 wait complete. 2024-04-25T00:25:44.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.622+0000 7f21729a6740 20 client.17628 trim_cache size 0 max 16384 2024-04-25T00:25:44.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.639+0000 7f21729a6740 1 Processor -- start 2024-04-25T00:25:44.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.639+0000 7f21729a6740 1 -- start start 2024-04-25T00:25:44.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.639+0000 7f21729a6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5af630 0x56523c5b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:44.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.639+0000 7f21729a6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56523c5aff50 0x56523c5b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:44.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.639+0000 7f21729a6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56523c5ba620 0x56523c5cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:44.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.639+0000 7f21729a6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f21681b3790 con 0x56523c5af630 2024-04-25T00:25:44.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.639+0000 7f21729a6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f21681b1060 con 0x56523c5aff50 2024-04-25T00:25:44.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.639+0000 7f21729a6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f21681aeed0 con 0x56523c5ba620 2024-04-25T00:25:44.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.639+0000 7f216d235640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5af630 0x56523c5b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:44.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.639+0000 7f216d235640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5af630 0x56523c5b1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55808/0 (socket says 172.21.15.107:55808) 2024-04-25T00:25:44.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.639+0000 7f216d235640 1 -- 172.21.15.107:0/736467515 learned_addr learned my addr 172.21.15.107:0/736467515 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:44.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.640+0000 7f216d235640 1 -- 172.21.15.107:0/736467515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56523c5ba620 msgr2=0x56523c5cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:44.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.640+0000 7f216d235640 1 --2- 172.21.15.107:0/736467515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56523c5ba620 0x56523c5cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.640+0000 7f216d235640 1 -- 172.21.15.107:0/736467515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56523c5aff50 msgr2=0x56523c5b6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:44.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.640+0000 7f216d235640 1 --2- 172.21.15.107:0/736467515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56523c5aff50 0x56523c5b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.640+0000 7f216d235640 1 -- 172.21.15.107:0/736467515 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56523c59d090 con 0x56523c5af630 2024-04-25T00:25:44.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.640+0000 7f216d235640 1 --2- 172.21.15.107:0/736467515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5af630 0x56523c5b1d90 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f2158015570 tx=0x7f21580155a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa30bc8d42c14dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:44.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.640+0000 7f21537fe640 1 -- 172.21.15.107:0/736467515 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215801b040 con 0x56523c5af630 2024-04-25T00:25:44.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.640+0000 7f21537fe640 1 -- 172.21.15.107:0/736467515 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2158016bf0 con 0x56523c5af630 2024-04-25T00:25:44.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.640+0000 7f21537fe640 1 -- 172.21.15.107:0/736467515 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215800ebc0 con 0x56523c5af630 2024-04-25T00:25:44.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.640+0000 7f21729a6740 1 -- 172.21.15.107:0/736467515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5af630 msgr2=0x56523c5b1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:44.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.640+0000 7f21729a6740 1 --2- 172.21.15.107:0/736467515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5af630 0x56523c5b1d90 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f2158015570 tx=0x7f21580155a0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.640+0000 7f21729a6740 1 -- 172.21.15.107:0/736467515 shutdown_connections 2024-04-25T00:25:44.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.640+0000 7f21729a6740 1 --2- 172.21.15.107:0/736467515 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56523c5ba620 0x56523c5cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.640+0000 7f21729a6740 1 --2- 172.21.15.107:0/736467515 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56523c5aff50 0x56523c5b6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.640+0000 7f21729a6740 1 --2- 172.21.15.107:0/736467515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5af630 0x56523c5b1d90 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.640+0000 7f21729a6740 1 -- 172.21.15.107:0/736467515 >> 172.21.15.107:0/736467515 conn(0x56523c5ab950 msgr2=0x56523c5b7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:44.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.641+0000 7f21729a6740 1 -- 172.21.15.107:0/736467515 shutdown_connections 2024-04-25T00:25:44.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.641+0000 7f21729a6740 1 -- 172.21.15.107:0/736467515 wait complete. 2024-04-25T00:25:44.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.641+0000 7f21729a6740 1 -- start start 2024-04-25T00:25:44.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.641+0000 7f21729a6740 1 Processor -- start 2024-04-25T00:25:44.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.641+0000 7f21729a6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5af630 0x56523c5cc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:44.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.641+0000 7f21729a6740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56523c5aff50 0x56523c5cce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:44.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.641+0000 7f21729a6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56523c5ba620 0x56523c5cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:44.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.641+0000 7f21729a6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f21681af520 con 0x56523c5af630 2024-04-25T00:25:44.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.641+0000 7f21729a6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f21681b0990 con 0x56523c5ba620 2024-04-25T00:25:44.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.641+0000 7f21729a6740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f21681afbf0 con 0x56523c5aff50 2024-04-25T00:25:44.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.641+0000 7f21729a6740 10 client.0 _ll_register_callbacks cb 0x7f2158019070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:44.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.641+0000 7f216d235640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5af630 0x56523c5cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:44.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.641+0000 7f216d235640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5af630 0x56523c5cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55822/0 (socket says 172.21.15.107:55822) 2024-04-25T00:25:44.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.641+0000 7f216d235640 1 -- 172.21.15.107:0/337804690 learned_addr learned my addr 172.21.15.107:0/337804690 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:44.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.641+0000 7f216da36640 1 --2- 172.21.15.107:0/337804690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56523c5aff50 0x56523c5cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:44.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.642+0000 7f216d235640 1 -- 172.21.15.107:0/337804690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56523c5aff50 msgr2=0x56523c5cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:44.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.642+0000 7f216d235640 1 --2- 172.21.15.107:0/337804690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56523c5aff50 0x56523c5cce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.642+0000 7f216d235640 1 -- 172.21.15.107:0/337804690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56523c5ba620 msgr2=0x56523c5cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:44.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.642+0000 7f216d235640 1 --2- 172.21.15.107:0/337804690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56523c5ba620 0x56523c5cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.642+0000 7f216d235640 1 -- 172.21.15.107:0/337804690 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56523c62ecf0 con 0x56523c5af630 2024-04-25T00:25:44.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.642+0000 7f216d235640 1 --2- 172.21.15.107:0/337804690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5af630 0x56523c5cc950 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f215800a440 tx=0x7f2158015400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdace4f2cd265d96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:44.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.642+0000 7f21517fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:25:44.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.642+0000 7f21517fa640 1 -- 172.21.15.107:0/337804690 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215801b040 con 0x56523c5af630 2024-04-25T00:25:44.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.642+0000 7f21517fa640 1 -- 172.21.15.107:0/337804690 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7f2158015970 con 0x56523c5af630 2024-04-25T00:25:44.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.642+0000 7f21729a6740 1 -- 172.21.15.107:0/337804690 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2158016bf0 con 0x56523c5af630 2024-04-25T00:25:44.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.642+0000 7f21517fa640 1 -- 172.21.15.107:0/337804690 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2158048050 con 0x56523c5af630 2024-04-25T00:25:44.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.642+0000 7f21729a6740 10 client.17634 fetch_fsmap learned FSMap version 34 2024-04-25T00:25:44.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.642+0000 7f21729a6740 1 -- 172.21.15.107:0/337804690 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56523c62ffe0 con 0x56523c5af630 2024-04-25T00:25:44.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.642+0000 7f21517fa640 1 -- 172.21.15.107:0/337804690 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2158016bf0 con 0x56523c5af630 2024-04-25T00:25:44.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.642+0000 7f21517fa640 1 -- 172.21.15.107:0/337804690 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f215803d020 con 0x56523c5af630 2024-04-25T00:25:44.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.643+0000 7f21729a6740 10 client.17634 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:25:44.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.643+0000 7f21729a6740 10 client.17634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:44.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.643+0000 7f21729a6740 20 client.17634 populate_metadata read hostname 'smithi107' 2024-04-25T00:25:44.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.643+0000 7f21729a6740 1 --2- 172.21.15.107:0/337804690 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56523c6305e0 0x56523c6329d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:44.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.643+0000 7f21729a6740 4 client.17634 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:25:44.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.643+0000 7f21729a6740 1 -- 172.21.15.107:0/337804690 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56523c51bf40 con 0x56523c6305e0 2024-04-25T00:25:44.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.643+0000 7f216da36640 1 --2- 172.21.15.107:0/337804690 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56523c6305e0 0x56523c6329d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:44.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.643+0000 7f216da36640 1 --2- 172.21.15.107:0/337804690 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56523c6305e0 0x56523c6329d0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d678fd0fe88ebd2 server_cookie=f74da55e3814f8a0 in_seq=0 out_seq=0 2024-04-25T00:25:44.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.643+0000 7f21517fa640 10 client.17634 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:44.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.643+0000 7f21517fa640 1 -- 172.21.15.107:0/337804690 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56523c51bf40 con 0x56523c6305e0 2024-04-25T00:25:44.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.643+0000 7f21517fa640 10 client.17634 handle_command_reply: tid=0 2024-04-25T00:25:44.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.643+0000 7f21729a6740 1 client.17634 shutdown 2024-04-25T00:25:44.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.643+0000 7f21729a6740 1 -- 172.21.15.107:0/337804690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5af630 msgr2=0x56523c5cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:44.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.643+0000 7f21729a6740 1 --2- 172.21.15.107:0/337804690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5af630 0x56523c5cc950 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f215800a440 tx=0x7f2158015400 comp rx=0 tx=0).stop 2024-04-25T00:25:44.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.644+0000 7f21729a6740 1 -- 172.21.15.107:0/337804690 shutdown_connections 2024-04-25T00:25:44.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.644+0000 7f21729a6740 1 --2- 172.21.15.107:0/337804690 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56523c6305e0 0x56523c6329d0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.644+0000 7f21729a6740 1 --2- 172.21.15.107:0/337804690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56523c5ba620 0x56523c5cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.644+0000 7f21729a6740 1 --2- 172.21.15.107:0/337804690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56523c5aff50 0x56523c5cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.644+0000 7f21517fa640 0 client.17634 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:44.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.644+0000 7f21729a6740 1 --2- 172.21.15.107:0/337804690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56523c5af630 0x56523c5cc950 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.644+0000 7f21729a6740 1 -- 172.21.15.107:0/337804690 >> 172.21.15.107:0/337804690 conn(0x56523c5ab950 msgr2=0x56523c5aa260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:44.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.644+0000 7f21729a6740 1 -- 172.21.15.107:0/337804690 shutdown_connections 2024-04-25T00:25:44.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.644+0000 7f21729a6740 1 -- 172.21.15.107:0/337804690 wait complete. 2024-04-25T00:25:44.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.644+0000 7f21729a6740 20 client.17634 trim_cache size 0 max 16384 2024-04-25T00:25:44.651 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:25:44.651 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-04-25T00:25:44.651 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-04-25T00:25:44.651 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:25:44.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.644+0000 7f216fc9a640 1 -- 172.21.15.107:0/2415159670 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f21540a2d00 msgr2=0x7f21540a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:44.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.644+0000 7f216fc9a640 1 --2- 172.21.15.107:0/2415159670 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f21540a2d00 0x7f21540a51c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f21600cd410 tx=0x7f21600915c0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.644+0000 7f216fc9a640 1 -- 172.21.15.107:0/2415159670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f216810e150 msgr2=0x7f216819eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:44.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.644+0000 7f216fc9a640 1 --2- 172.21.15.107:0/2415159670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f216810e150 0x7f216819eb50 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f216000cab0 tx=0x7f216000cf70 comp rx=0 tx=0).stop 2024-04-25T00:25:44.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.644+0000 7f216fc9a640 1 -- 172.21.15.107:0/2415159670 shutdown_connections 2024-04-25T00:25:44.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.644+0000 7f216fc9a640 1 --2- 172.21.15.107:0/2415159670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f216810e150 0x7f216819eb50 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.644+0000 7f216fc9a640 1 --2- 172.21.15.107:0/2415159670 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f21540a2d00 0x7f21540a51c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.644+0000 7f216fc9a640 1 --2- 172.21.15.107:0/2415159670 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f216806a6e0 0x7f216819a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.644+0000 7f216fc9a640 1 --2- 172.21.15.107:0/2415159670 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2168069dc0 0x7f216819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.644+0000 7f216fc9a640 1 -- 172.21.15.107:0/2415159670 >> 172.21.15.107:0/2415159670 conn(0x7f21680776a0 msgr2=0x7f216810f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:44.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.644+0000 7f216fc9a640 1 -- 172.21.15.107:0/2415159670 shutdown_connections 2024-04-25T00:25:44.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.644+0000 7f216fc9a640 1 -- 172.21.15.107:0/2415159670 wait complete. 2024-04-25T00:25:44.655 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:25:44.656 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:25:44.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.753+0000 7f82cc763640 1 -- 172.21.15.107:0/1270797166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c4107720 msgr2=0x7f82c4107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:44.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.753+0000 7f82cc763640 1 --2- 172.21.15.107:0/1270797166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c4107720 0x7f82c4107b00 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009cd0 tx=0x7f82bc02a8e0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.753+0000 7f82cc763640 1 -- 172.21.15.107:0/1270797166 shutdown_connections 2024-04-25T00:25:44.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.753+0000 7f82cc763640 1 --2- 172.21.15.107:0/1270797166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82c41020d0 0x7f82c410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.753+0000 7f82cc763640 1 --2- 172.21.15.107:0/1270797166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82c4101710 0x7f82c4101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.753+0000 7f82cc763640 1 --2- 172.21.15.107:0/1270797166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c4107720 0x7f82c4107b00 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.753+0000 7f82cc763640 1 -- 172.21.15.107:0/1270797166 >> 172.21.15.107:0/1270797166 conn(0x7f82c4078b00 msgr2=0x7f82c4078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:44.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.753+0000 7f82cc763640 1 -- 172.21.15.107:0/1270797166 shutdown_connections 2024-04-25T00:25:44.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.753+0000 7f82cc763640 1 -- 172.21.15.107:0/1270797166 wait complete. 2024-04-25T00:25:44.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.753+0000 7f82cc763640 1 Processor -- start 2024-04-25T00:25:44.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.753+0000 7f82cc763640 1 -- start start 2024-04-25T00:25:44.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.754+0000 7f82cc763640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c4101710 0x7f82c4198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:44.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.754+0000 7f82cc763640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82c41020d0 0x7f82c41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:44.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.754+0000 7f82cc763640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82c4107720 0x7f82c419c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:44.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.754+0000 7f82cc763640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f82c4111b30 con 0x7f82c4101710 2024-04-25T00:25:44.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.754+0000 7f82cc763640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f82c41119b0 con 0x7f82c41020d0 2024-04-25T00:25:44.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.754+0000 7f82cc763640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f82c4111cb0 con 0x7f82c4107720 2024-04-25T00:25:44.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.754+0000 7f82ca4ff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c4101710 0x7f82c4198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:44.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.754+0000 7f82ca4ff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c4101710 0x7f82c4198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55830/0 (socket says 172.21.15.107:55830) 2024-04-25T00:25:44.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.754+0000 7f82ca4ff640 1 -- 172.21.15.107:0/1130291924 learned_addr learned my addr 172.21.15.107:0/1130291924 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:44.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.754+0000 7f82c9cfe640 1 --2- 172.21.15.107:0/1130291924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82c41020d0 0x7f82c41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:44.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.754+0000 7f82ca4ff640 1 -- 172.21.15.107:0/1130291924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82c4107720 msgr2=0x7f82c419c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:44.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.754+0000 7f82cad00640 1 --2- 172.21.15.107:0/1130291924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82c4107720 0x7f82c419c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:44.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.754+0000 7f82ca4ff640 1 --2- 172.21.15.107:0/1130291924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82c4107720 0x7f82c419c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.754+0000 7f82ca4ff640 1 -- 172.21.15.107:0/1130291924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82c41020d0 msgr2=0x7f82c41985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:44.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.754+0000 7f82ca4ff640 1 --2- 172.21.15.107:0/1130291924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82c41020d0 0x7f82c41985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.754+0000 7f82ca4ff640 1 -- 172.21.15.107:0/1130291924 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82c419d060 con 0x7f82c4101710 2024-04-25T00:25:44.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.754+0000 7f82c9cfe640 1 --2- 172.21.15.107:0/1130291924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82c41020d0 0x7f82c41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:44.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.754+0000 7f82ca4ff640 1 --2- 172.21.15.107:0/1130291924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c4101710 0x7f82c4198070 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f82bc02ae00 tx=0x7f82bc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:44.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.754+0000 7f82bb7fe640 1 -- 172.21.15.107:0/1130291924 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc004240 con 0x7f82c4101710 2024-04-25T00:25:44.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.754+0000 7f82bb7fe640 1 -- 172.21.15.107:0/1130291924 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82bc0305a0 con 0x7f82c4101710 2024-04-25T00:25:44.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.754+0000 7f82cc763640 1 -- 172.21.15.107:0/1130291924 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82c419d2f0 con 0x7f82c4101710 2024-04-25T00:25:44.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.754+0000 7f82bb7fe640 1 -- 172.21.15.107:0/1130291924 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc03a660 con 0x7f82c4101710 2024-04-25T00:25:44.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.754+0000 7f82cc763640 1 -- 172.21.15.107:0/1130291924 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82c41a4be0 con 0x7f82c4101710 2024-04-25T00:25:44.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.755+0000 7f82cc763640 1 -- 172.21.15.107:0/1130291924 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82c40689b0 con 0x7f82c4101710 2024-04-25T00:25:44.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.756+0000 7f82bb7fe640 1 -- 172.21.15.107:0/1130291924 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f82bc030ac0 con 0x7f82c4101710 2024-04-25T00:25:44.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.757+0000 7f82bb7fe640 1 --2- 172.21.15.107:0/1130291924 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f82b00a2d00 0x7f82b00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:44.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.757+0000 7f82c9cfe640 1 --2- 172.21.15.107:0/1130291924 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f82b00a2d00 0x7f82b00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:44.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.757+0000 7f82bb7fe640 1 -- 172.21.15.107:0/1130291924 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7f82bc02e030 con 0x7f82c4101710 2024-04-25T00:25:44.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.757+0000 7f82c9cfe640 1 --2- 172.21.15.107:0/1130291924 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f82b00a2d00 0x7f82b00a51c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f82b4005fd0 tx=0x7f82b4005ea0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:44.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.761+0000 7f82bb7fe640 1 -- 172.21.15.107:0/1130291924 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f82bc0098b0 con 0x7f82c4101710 2024-04-25T00:25:44.973 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.972+0000 7f82cc763640 1 -- 172.21.15.107:0/1130291924 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82c4109ce0 con 0x7f82c4101710 2024-04-25T00:25:44.974 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.973+0000 7f82bb7fe640 1 -- 172.21.15.107:0/1130291924 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11897 (secure 0 0 0) 0x7f82bc0bd3c0 con 0x7f82c4101710 2024-04-25T00:25:44.975 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:25:44.975 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:22:27.402680+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24493,24521]},"id":1}]} 2024-04-25T00:25:44.977 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 34 2024-04-25T00:25:44.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.975+0000 7f82cc763640 1 -- 172.21.15.107:0/1130291924 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f82b00a2d00 msgr2=0x7f82b00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:44.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.975+0000 7f82cc763640 1 --2- 172.21.15.107:0/1130291924 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f82b00a2d00 0x7f82b00a51c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f82b4005fd0 tx=0x7f82b4005ea0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.975+0000 7f82cc763640 1 -- 172.21.15.107:0/1130291924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c4101710 msgr2=0x7f82c4198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:44.977 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.975+0000 7f82cc763640 1 --2- 172.21.15.107:0/1130291924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c4101710 0x7f82c4198070 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f82bc02ae00 tx=0x7f82bc003ba0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.975+0000 7f82cc763640 1 -- 172.21.15.107:0/1130291924 shutdown_connections 2024-04-25T00:25:44.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.975+0000 7f82cc763640 1 --2- 172.21.15.107:0/1130291924 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f82c4107720 0x7f82c419c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.975+0000 7f82cc763640 1 --2- 172.21.15.107:0/1130291924 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f82b00a2d00 0x7f82b00a51c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.978 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.975+0000 7f82cc763640 1 --2- 172.21.15.107:0/1130291924 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f82c41020d0 0x7f82c41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.975+0000 7f82cc763640 1 --2- 172.21.15.107:0/1130291924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82c4101710 0x7f82c4198070 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:44.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.975+0000 7f82cc763640 1 -- 172.21.15.107:0/1130291924 >> 172.21.15.107:0/1130291924 conn(0x7f82c4078b00 msgr2=0x7f82c41041a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:44.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.975+0000 7f82cc763640 1 -- 172.21.15.107:0/1130291924 shutdown_connections 2024-04-25T00:25:44.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:44.975+0000 7f82cc763640 1 -- 172.21.15.107:0/1130291924 wait complete. 2024-04-25T00:25:44.987 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:25:45.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.086+0000 7feb7b59e640 1 -- 172.21.15.107:0/1016278362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb74100570 msgr2=0x7feb7410e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.086+0000 7feb7b59e640 1 --2- 172.21.15.107:0/1016278362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb74100570 0x7feb7410e550 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7feb70009cd0 tx=0x7feb7002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.086+0000 7feb7b59e640 1 -- 172.21.15.107:0/1016278362 shutdown_connections 2024-04-25T00:25:45.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.086+0000 7feb7b59e640 1 --2- 172.21.15.107:0/1016278362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb74100570 0x7feb7410e550 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.086+0000 7feb7b59e640 1 --2- 172.21.15.107:0/1016278362 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb740ffbb0 0x7feb74100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.086+0000 7feb7b59e640 1 --2- 172.21.15.107:0/1016278362 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb74107690 0x7feb74107a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.086+0000 7feb7b59e640 1 -- 172.21.15.107:0/1016278362 >> 172.21.15.107:0/1016278362 conn(0x7feb74076b70 msgr2=0x7feb740fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:45.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.086+0000 7feb7b59e640 1 -- 172.21.15.107:0/1016278362 shutdown_connections 2024-04-25T00:25:45.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.086+0000 7feb7b59e640 1 -- 172.21.15.107:0/1016278362 wait complete. 2024-04-25T00:25:45.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.087+0000 7feb7b59e640 1 Processor -- start 2024-04-25T00:25:45.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.087+0000 7feb7b59e640 1 -- start start 2024-04-25T00:25:45.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.087+0000 7feb7b59e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb740ffbb0 0x7feb74100d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.087+0000 7feb7b59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb74100570 0x7feb741012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.087+0000 7feb7b59e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb74107690 0x7feb74101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.087+0000 7feb7b59e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feb74111aa0 con 0x7feb74100570 2024-04-25T00:25:45.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.087+0000 7feb7b59e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7feb74111920 con 0x7feb740ffbb0 2024-04-25T00:25:45.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.087+0000 7feb7b59e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feb74111c20 con 0x7feb74107690 2024-04-25T00:25:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.087+0000 7feb79d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb74100570 0x7feb741012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.087+0000 7feb79d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb74100570 0x7feb741012d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55842/0 (socket says 172.21.15.107:55842) 2024-04-25T00:25:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.087+0000 7feb79d9b640 1 -- 172.21.15.107:0/1342646904 learned_addr learned my addr 172.21.15.107:0/1342646904 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.088+0000 7feb7a59c640 1 --2- 172.21.15.107:0/1342646904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb740ffbb0 0x7feb74100d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.088+0000 7feb79d9b640 1 -- 172.21.15.107:0/1342646904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb74107690 msgr2=0x7feb74101810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.088+0000 7feb79d9b640 1 --2- 172.21.15.107:0/1342646904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb74107690 0x7feb74101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.088+0000 7feb79d9b640 1 -- 172.21.15.107:0/1342646904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb740ffbb0 msgr2=0x7feb74100d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.088+0000 7feb79d9b640 1 --2- 172.21.15.107:0/1342646904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb740ffbb0 0x7feb74100d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.088+0000 7feb79d9b640 1 -- 172.21.15.107:0/1342646904 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb741a6360 con 0x7feb74100570 2024-04-25T00:25:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.088+0000 7feb79d9b640 1 --2- 172.21.15.107:0/1342646904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb74100570 0x7feb741012d0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7feb6800d950 tx=0x7feb6800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.088+0000 7feb677fe640 1 -- 172.21.15.107:0/1342646904 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb68017070 con 0x7feb74100570 2024-04-25T00:25:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.088+0000 7feb677fe640 1 -- 172.21.15.107:0/1342646904 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb68009c30 con 0x7feb74100570 2024-04-25T00:25:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.088+0000 7feb7b59e640 1 -- 172.21.15.107:0/1342646904 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb741a6650 con 0x7feb74100570 2024-04-25T00:25:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.088+0000 7feb677fe640 1 -- 172.21.15.107:0/1342646904 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6800f6b0 con 0x7feb74100570 2024-04-25T00:25:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.088+0000 7feb7b59e640 1 -- 172.21.15.107:0/1342646904 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb741a6ba0 con 0x7feb74100570 2024-04-25T00:25:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.090+0000 7feb677fe640 1 -- 172.21.15.107:0/1342646904 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7feb680109d0 con 0x7feb74100570 2024-04-25T00:25:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.090+0000 7feb7f5eb740 1 Processor -- start 2024-04-25T00:25:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.091+0000 7feb7f5eb740 1 -- start start 2024-04-25T00:25:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.091+0000 7feb7f5eb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4305710 0x560be4305af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.091+0000 7feb7f5eb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560be4306030 0x560be43101c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.091+0000 7feb7f5eb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560be4310700 0x560be4312af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.091+0000 7feb7f5eb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560be426a670 con 0x560be4305710 2024-04-25T00:25:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.091+0000 7feb7f5eb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560be4212970 con 0x560be4306030 2024-04-25T00:25:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.091+0000 7feb7f5eb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560be4266630 con 0x560be4310700 2024-04-25T00:25:45.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.091+0000 7feb677fe640 1 --2- 172.21.15.107:0/1342646904 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7feb600a2d30 0x7feb600a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.091+0000 7feb7ad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4305710 0x560be4305af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.091+0000 7feb7ad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4305710 0x560be4305af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55856/0 (socket says 172.21.15.107:55856) 2024-04-25T00:25:45.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.091+0000 7feb7ad9d640 1 -- 172.21.15.107:0/2428756849 learned_addr learned my addr 172.21.15.107:0/2428756849 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:45.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.091+0000 7feb7a59c640 1 --2- 172.21.15.107:0/2428756849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560be4306030 0x560be43101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.091+0000 7feb7ad9d640 1 --2- 172.21.15.107:0/1342646904 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7feb600a2d30 0x7feb600a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.091+0000 7feb677fe640 1 -- 172.21.15.107:0/1342646904 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7feb680cf4e0 con 0x7feb74100570 2024-04-25T00:25:45.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.091+0000 7feb79d9b640 1 --2- 172.21.15.107:0/2428756849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560be4310700 0x560be4312af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.091+0000 7feb7ad9d640 1 -- 172.21.15.107:0/2428756849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560be4310700 msgr2=0x560be4312af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:45.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.091+0000 7feb7ad9d640 1 --2- 172.21.15.107:0/2428756849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560be4310700 0x560be4312af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.091+0000 7feb7ad9d640 1 -- 172.21.15.107:0/2428756849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560be4306030 msgr2=0x560be43101c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:45.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.091+0000 7feb7ad9d640 1 --2- 172.21.15.107:0/2428756849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560be4306030 0x560be43101c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.091+0000 7feb7ad9d640 1 -- 172.21.15.107:0/2428756849 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560be4236400 con 0x560be4305710 2024-04-25T00:25:45.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.091+0000 7feb79d9b640 1 --2- 172.21.15.107:0/2428756849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560be4310700 0x560be4312af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:45.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.092+0000 7feb7ad9d640 1 --2- 172.21.15.107:0/2428756849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4305710 0x560be4305af0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7feb700097b0 tx=0x7feb70031820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42c6b100fa38cc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:45.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.092+0000 7feb7ad9d640 1 --2- 172.21.15.107:0/1342646904 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7feb600a2d30 0x7feb600a51f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feb70009ca0 tx=0x7feb70036000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:45.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.092+0000 7feb64ff9640 1 -- 172.21.15.107:0/2428756849 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7004c070 con 0x560be4305710 2024-04-25T00:25:45.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.092+0000 7feb64ff9640 1 -- 172.21.15.107:0/2428756849 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb70036070 con 0x560be4305710 2024-04-25T00:25:45.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.092+0000 7feb64ff9640 1 -- 172.21.15.107:0/2428756849 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70030ae0 con 0x560be4305710 2024-04-25T00:25:45.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.092+0000 7feb7f5eb740 1 -- 172.21.15.107:0/2428756849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4305710 msgr2=0x560be4305af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.092+0000 7feb7f5eb740 1 --2- 172.21.15.107:0/2428756849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4305710 0x560be4305af0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7feb700097b0 tx=0x7feb70031820 comp rx=0 tx=0).stop 2024-04-25T00:25:45.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.092+0000 7feb7f5eb740 1 -- 172.21.15.107:0/2428756849 shutdown_connections 2024-04-25T00:25:45.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.092+0000 7feb7f5eb740 1 --2- 172.21.15.107:0/2428756849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560be4310700 0x560be4312af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.092+0000 7feb7f5eb740 1 --2- 172.21.15.107:0/2428756849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560be4306030 0x560be43101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.092+0000 7feb7f5eb740 1 --2- 172.21.15.107:0/2428756849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4305710 0x560be4305af0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.092+0000 7feb7f5eb740 1 -- 172.21.15.107:0/2428756849 >> 172.21.15.107:0/2428756849 conn(0x560be4301a30 msgr2=0x560be4301e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:45.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.093+0000 7feb7f5eb740 1 -- 172.21.15.107:0/2428756849 shutdown_connections 2024-04-25T00:25:45.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.093+0000 7feb7f5eb740 1 -- 172.21.15.107:0/2428756849 wait complete. 2024-04-25T00:25:45.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.093+0000 7feb7f5eb740 1 -- start start 2024-04-25T00:25:45.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.093+0000 7feb7f5eb740 1 Processor -- start 2024-04-25T00:25:45.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.093+0000 7feb7f5eb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560be4305710 0x560be4310120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.093+0000 7feb7f5eb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4306030 0x560be430d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.093+0000 7feb7f5eb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560be4310700 0x560be430da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.093+0000 7feb7f5eb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560be4313490 con 0x560be4306030 2024-04-25T00:25:45.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.093+0000 7feb7f5eb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560be4226050 con 0x560be4310700 2024-04-25T00:25:45.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.093+0000 7feb7f5eb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560be4313610 con 0x560be4305710 2024-04-25T00:25:45.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.093+0000 7feb7f5eb740 10 client.0 _ll_register_callbacks cb 0x560be3f7dbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:45.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.093+0000 7feb79d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4306030 0x560be430d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.093+0000 7feb79d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4306030 0x560be430d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55866/0 (socket says 172.21.15.107:55866) 2024-04-25T00:25:45.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.093+0000 7feb79d9b640 1 -- 172.21.15.107:0/188890161 learned_addr learned my addr 172.21.15.107:0/188890161 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:45.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.093+0000 7feb7ad9d640 1 --2- 172.21.15.107:0/188890161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560be4310700 0x560be430da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.094+0000 7feb79d9b640 1 -- 172.21.15.107:0/188890161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560be4305710 msgr2=0x560be4310120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:45.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.094+0000 7feb79d9b640 1 --2- 172.21.15.107:0/188890161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560be4305710 0x560be4310120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.094+0000 7feb79d9b640 1 -- 172.21.15.107:0/188890161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560be4310700 msgr2=0x560be430da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:45.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.094+0000 7feb79d9b640 1 --2- 172.21.15.107:0/188890161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560be4310700 0x560be430da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.094+0000 7feb79d9b640 1 -- 172.21.15.107:0/188890161 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560be41f7dc0 con 0x560be4306030 2024-04-25T00:25:45.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.094+0000 7feb7ad9d640 1 --2- 172.21.15.107:0/188890161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560be4310700 0x560be430da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:45.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.094+0000 7feb79d9b640 1 --2- 172.21.15.107:0/188890161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4306030 0x560be430d410 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7feb68098010 tx=0x7feb6809b290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216547357ab79b85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:45.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.094+0000 7feb5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:25:45.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.094+0000 7feb5dffb640 1 -- 172.21.15.107:0/188890161 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb68020040 con 0x560be4306030 2024-04-25T00:25:45.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.094+0000 7feb5dffb640 1 -- 172.21.15.107:0/188890161 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7feb68097cb0 con 0x560be4306030 2024-04-25T00:25:45.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.094+0000 7feb7f5eb740 1 -- 172.21.15.107:0/188890161 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb70036070 con 0x560be4306030 2024-04-25T00:25:45.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.094+0000 7feb5dffb640 1 -- 172.21.15.107:0/188890161 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7feb6809f9f0 con 0x560be4306030 2024-04-25T00:25:45.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.094+0000 7feb7f5eb740 10 client.17652 fetch_fsmap learned FSMap version 34 2024-04-25T00:25:45.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.094+0000 7feb7f5eb740 1 -- 172.21.15.107:0/188890161 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560be3f7e3f0 con 0x560be4306030 2024-04-25T00:25:45.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.094+0000 7feb5dffb640 1 -- 172.21.15.107:0/188890161 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb70036070 con 0x560be4306030 2024-04-25T00:25:45.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.094+0000 7feb5dffb640 1 -- 172.21.15.107:0/188890161 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7feb6809aa90 con 0x560be4306030 2024-04-25T00:25:45.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.094+0000 7feb7f5eb740 10 client.17652 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:25:45.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.094+0000 7feb7f5eb740 10 client.17652 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:25:45.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.094+0000 7feb7f5eb740 20 client.17652 populate_metadata read hostname 'smithi107' 2024-04-25T00:25:45.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.094+0000 7feb7f5eb740 1 --2- 172.21.15.107:0/188890161 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x560be4320780 0x560be4322bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.094+0000 7feb7f5eb740 4 client.17652 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:45.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.094+0000 7feb7f5eb740 1 -- 172.21.15.107:0/188890161 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560be4277bb0 con 0x560be4320780 2024-04-25T00:25:45.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.095+0000 7feb7a59c640 1 --2- 172.21.15.107:0/188890161 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x560be4320780 0x560be4322bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.095+0000 7feb5dffb640 10 client.17652 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:25:45.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.095+0000 7feb7a59c640 1 --2- 172.21.15.107:0/188890161 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x560be4320780 0x560be4322bd0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7bfd8273110a9751 server_cookie=1ac07efbbca005f0 in_seq=0 out_seq=0 2024-04-25T00:25:45.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.096+0000 7feb5dffb640 1 -- 172.21.15.107:0/188890161 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560be4277bb0 con 0x560be4320780 2024-04-25T00:25:45.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.096+0000 7feb5dffb640 10 client.17652 handle_command_reply: tid=0 2024-04-25T00:25:45.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.096+0000 7feb7f5eb740 1 client.17652 shutdown 2024-04-25T00:25:45.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.096+0000 7feb7f5eb740 1 -- 172.21.15.107:0/188890161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4306030 msgr2=0x560be430d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.096+0000 7feb7f5eb740 1 --2- 172.21.15.107:0/188890161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4306030 0x560be430d410 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7feb68098010 tx=0x7feb6809b290 comp rx=0 tx=0).stop 2024-04-25T00:25:45.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.096+0000 7feb7f5eb740 1 -- 172.21.15.107:0/188890161 shutdown_connections 2024-04-25T00:25:45.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.096+0000 7feb7f5eb740 1 --2- 172.21.15.107:0/188890161 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x560be4320780 0x560be4322bd0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.096+0000 7feb7f5eb740 1 --2- 172.21.15.107:0/188890161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560be4310700 0x560be430da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.096+0000 7feb7f5eb740 1 --2- 172.21.15.107:0/188890161 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4306030 0x560be430d410 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.096+0000 7feb5dffb640 0 client.17652 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:25:45.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.096+0000 7feb7f5eb740 1 --2- 172.21.15.107:0/188890161 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560be4305710 0x560be4310120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.096+0000 7feb7f5eb740 1 -- 172.21.15.107:0/188890161 >> 172.21.15.107:0/188890161 conn(0x560be4301a30 msgr2=0x560be4307830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:45.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.096+0000 7feb7f5eb740 1 -- 172.21.15.107:0/188890161 shutdown_connections 2024-04-25T00:25:45.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.096+0000 7feb7f5eb740 1 -- 172.21.15.107:0/188890161 wait complete. 2024-04-25T00:25:45.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.096+0000 7feb7f5eb740 20 client.17652 trim_cache size 0 max 16384 2024-04-25T00:25:45.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.109+0000 7feb7f5eb740 1 Processor -- start 2024-04-25T00:25:45.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.109+0000 7feb7f5eb740 1 -- start start 2024-04-25T00:25:45.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.109+0000 7feb7f5eb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4305710 0x560be431c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.109+0000 7feb7f5eb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560be4306030 0x560be430fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.109+0000 7feb7f5eb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560be4310700 0x560be43009c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.109+0000 7feb7f5eb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feb741b3440 con 0x560be4305710 2024-04-25T00:25:45.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.110+0000 7feb7f5eb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7feb741b0d10 con 0x560be4306030 2024-04-25T00:25:45.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.110+0000 7feb7f5eb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feb741aeb80 con 0x560be4310700 2024-04-25T00:25:45.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.110+0000 7feb7a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4305710 0x560be431c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.110+0000 7feb7a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4305710 0x560be431c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55874/0 (socket says 172.21.15.107:55874) 2024-04-25T00:25:45.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.110+0000 7feb7a59c640 1 -- 172.21.15.107:0/2494149366 learned_addr learned my addr 172.21.15.107:0/2494149366 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:45.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.110+0000 7feb7a59c640 1 -- 172.21.15.107:0/2494149366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560be4310700 msgr2=0x560be43009c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:45.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.110+0000 7feb7a59c640 1 --2- 172.21.15.107:0/2494149366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560be4310700 0x560be43009c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.110+0000 7feb7a59c640 1 -- 172.21.15.107:0/2494149366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560be4306030 msgr2=0x560be430fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:45.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.110+0000 7feb7a59c640 1 --2- 172.21.15.107:0/2494149366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560be4306030 0x560be430fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.110+0000 7feb7a59c640 1 -- 172.21.15.107:0/2494149366 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560be40487d0 con 0x560be4305710 2024-04-25T00:25:45.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.110+0000 7feb7a59c640 1 --2- 172.21.15.107:0/2494149366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4305710 0x560be431c350 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x560be4323410 tx=0x7feb6c009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11108088e08b95c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:45.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.110+0000 7feb5ffff640 1 -- 172.21.15.107:0/2494149366 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c01f030 con 0x560be4305710 2024-04-25T00:25:45.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.110+0000 7feb5ffff640 1 -- 172.21.15.107:0/2494149366 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb6c01b880 con 0x560be4305710 2024-04-25T00:25:45.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.110+0000 7feb5ffff640 1 -- 172.21.15.107:0/2494149366 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c004a00 con 0x560be4305710 2024-04-25T00:25:45.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.111+0000 7feb7f5eb740 1 -- 172.21.15.107:0/2494149366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4305710 msgr2=0x560be431c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.111+0000 7feb7f5eb740 1 --2- 172.21.15.107:0/2494149366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4305710 0x560be431c350 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x560be4323410 tx=0x7feb6c009670 comp rx=0 tx=0).stop 2024-04-25T00:25:45.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.111+0000 7feb7f5eb740 1 -- 172.21.15.107:0/2494149366 shutdown_connections 2024-04-25T00:25:45.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.111+0000 7feb7f5eb740 1 --2- 172.21.15.107:0/2494149366 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560be4310700 0x560be43009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.111+0000 7feb7f5eb740 1 --2- 172.21.15.107:0/2494149366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560be4306030 0x560be430fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.111+0000 7feb7f5eb740 1 --2- 172.21.15.107:0/2494149366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4305710 0x560be431c350 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.111+0000 7feb7f5eb740 1 -- 172.21.15.107:0/2494149366 >> 172.21.15.107:0/2494149366 conn(0x560be4301a30 msgr2=0x560be43002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:45.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.111+0000 7feb7f5eb740 1 -- 172.21.15.107:0/2494149366 shutdown_connections 2024-04-25T00:25:45.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.111+0000 7feb7f5eb740 1 -- 172.21.15.107:0/2494149366 wait complete. 2024-04-25T00:25:45.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.111+0000 7feb7f5eb740 1 -- start start 2024-04-25T00:25:45.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.111+0000 7feb7f5eb740 1 Processor -- start 2024-04-25T00:25:45.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.111+0000 7feb7f5eb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560be4305710 0x560be4318cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.111+0000 7feb7f5eb740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560be4306030 0x560be4309ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.111+0000 7feb7f5eb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4310700 0x560be430a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.111+0000 7feb7f5eb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feb741af1d0 con 0x560be4310700 2024-04-25T00:25:45.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.111+0000 7feb7f5eb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7feb741b0640 con 0x560be4305710 2024-04-25T00:25:45.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.111+0000 7feb7f5eb740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7feb741af8a0 con 0x560be4306030 2024-04-25T00:25:45.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.111+0000 7feb7f5eb740 10 client.0 _ll_register_callbacks cb 0x7feb6800df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:45.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.111+0000 7feb7ad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4310700 0x560be430a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.111+0000 7feb7ad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4310700 0x560be430a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55878/0 (socket says 172.21.15.107:55878) 2024-04-25T00:25:45.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.111+0000 7feb7ad9d640 1 -- 172.21.15.107:0/2120230608 learned_addr learned my addr 172.21.15.107:0/2120230608 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:45.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb79d9b640 1 --2- 172.21.15.107:0/2120230608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560be4306030 0x560be4309ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb7a59c640 1 --2- 172.21.15.107:0/2120230608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560be4305710 0x560be4318cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb7ad9d640 1 -- 172.21.15.107:0/2120230608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560be4306030 msgr2=0x560be4309ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:45.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb7ad9d640 1 --2- 172.21.15.107:0/2120230608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560be4306030 0x560be4309ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb7ad9d640 1 -- 172.21.15.107:0/2120230608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560be4305710 msgr2=0x560be4318cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:45.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb7ad9d640 1 --2- 172.21.15.107:0/2120230608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560be4305710 0x560be4318cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb7ad9d640 1 -- 172.21.15.107:0/2120230608 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560be430ad30 con 0x560be4310700 2024-04-25T00:25:45.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb79d9b640 1 --2- 172.21.15.107:0/2120230608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560be4306030 0x560be4309ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:45.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb7a59c640 1 --2- 172.21.15.107:0/2120230608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560be4305710 0x560be4318cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:45.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb7ad9d640 1 --2- 172.21.15.107:0/2120230608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4310700 0x560be430a500 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7feb70006010 tx=0x7feb700307d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cdb4e3310a3079 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:45.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:25:45.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb5dffb640 1 -- 172.21.15.107:0/2120230608 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb70055070 con 0x560be4310700 2024-04-25T00:25:45.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb5dffb640 1 -- 172.21.15.107:0/2120230608 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7feb70031620 con 0x560be4310700 2024-04-25T00:25:45.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb7f5eb740 1 -- 172.21.15.107:0/2120230608 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb6c01b880 con 0x560be4310700 2024-04-25T00:25:45.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb5dffb640 1 -- 172.21.15.107:0/2120230608 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7feb70061050 con 0x560be4310700 2024-04-25T00:25:45.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb7f5eb740 10 client.17658 fetch_fsmap learned FSMap version 34 2024-04-25T00:25:45.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb7f5eb740 1 -- 172.21.15.107:0/2120230608 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560be4386e10 con 0x560be4310700 2024-04-25T00:25:45.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb5dffb640 1 -- 172.21.15.107:0/2120230608 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb6c01b880 con 0x560be4310700 2024-04-25T00:25:45.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb5dffb640 1 -- 172.21.15.107:0/2120230608 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7feb70047070 con 0x560be4310700 2024-04-25T00:25:45.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb7f5eb740 10 client.17658 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:25:45.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb7f5eb740 10 client.17658 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:25:45.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb7f5eb740 20 client.17658 populate_metadata read hostname 'smithi107' 2024-04-25T00:25:45.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb7f5eb740 1 --2- 172.21.15.107:0/2120230608 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x560be4387210 0x560be4389600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb7f5eb740 4 client.17658 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:25:45.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.112+0000 7feb7f5eb740 1 -- 172.21.15.107:0/2120230608 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560be4271f00 con 0x560be4387210 2024-04-25T00:25:45.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.113+0000 7feb7a59c640 1 --2- 172.21.15.107:0/2120230608 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x560be4387210 0x560be4389600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.113+0000 7feb5dffb640 10 client.17658 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:25:45.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.113+0000 7feb7a59c640 1 --2- 172.21.15.107:0/2120230608 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x560be4387210 0x560be4389600 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c88b393136a7d30 server_cookie=c00be35918c99835 in_seq=0 out_seq=0 2024-04-25T00:25:45.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.113+0000 7feb5dffb640 1 -- 172.21.15.107:0/2120230608 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560be4271f00 con 0x560be4387210 2024-04-25T00:25:45.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.113+0000 7feb5dffb640 10 client.17658 handle_command_reply: tid=0 2024-04-25T00:25:45.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.113+0000 7feb7f5eb740 1 client.17658 shutdown 2024-04-25T00:25:45.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.113+0000 7feb7f5eb740 1 -- 172.21.15.107:0/2120230608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4310700 msgr2=0x560be430a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.113+0000 7feb7f5eb740 1 --2- 172.21.15.107:0/2120230608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4310700 0x560be430a500 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7feb70006010 tx=0x7feb700307d0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.113+0000 7feb7f5eb740 1 -- 172.21.15.107:0/2120230608 shutdown_connections 2024-04-25T00:25:45.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.113+0000 7feb7f5eb740 1 --2- 172.21.15.107:0/2120230608 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x560be4387210 0x560be4389600 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.113+0000 7feb7f5eb740 1 --2- 172.21.15.107:0/2120230608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560be4310700 0x560be430a500 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.113+0000 7feb7f5eb740 1 --2- 172.21.15.107:0/2120230608 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x560be4306030 0x560be4309ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.113+0000 7feb5dffb640 0 client.17658 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:25:45.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.113+0000 7feb7f5eb740 1 --2- 172.21.15.107:0/2120230608 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560be4305710 0x560be4318cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.113+0000 7feb7f5eb740 1 -- 172.21.15.107:0/2120230608 >> 172.21.15.107:0/2120230608 conn(0x560be4301a30 msgr2=0x560be4307730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:45.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.114+0000 7feb7f5eb740 1 -- 172.21.15.107:0/2120230608 shutdown_connections 2024-04-25T00:25:45.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.114+0000 7feb7f5eb740 1 -- 172.21.15.107:0/2120230608 wait complete. 2024-04-25T00:25:45.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.114+0000 7feb7f5eb740 20 client.17658 trim_cache size 0 max 16384 2024-04-25T00:25:45.121 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:25:45.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.114+0000 7feb7b59e640 1 -- 172.21.15.107:0/1342646904 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7feb600a2d30 msgr2=0x7feb600a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.114+0000 7feb7b59e640 1 --2- 172.21.15.107:0/1342646904 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7feb600a2d30 0x7feb600a51f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feb70009ca0 tx=0x7feb70036000 comp rx=0 tx=0).stop 2024-04-25T00:25:45.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.114+0000 7feb7b59e640 1 -- 172.21.15.107:0/1342646904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb74100570 msgr2=0x7feb741012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.114+0000 7feb7b59e640 1 --2- 172.21.15.107:0/1342646904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb74100570 0x7feb741012d0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7feb6800d950 tx=0x7feb6800de10 comp rx=0 tx=0).stop 2024-04-25T00:25:45.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.114+0000 7feb7b59e640 1 -- 172.21.15.107:0/1342646904 shutdown_connections 2024-04-25T00:25:45.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.114+0000 7feb7b59e640 1 --2- 172.21.15.107:0/1342646904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7feb74107690 0x7feb74101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.114+0000 7feb7b59e640 1 --2- 172.21.15.107:0/1342646904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb74100570 0x7feb741012d0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.114+0000 7feb7b59e640 1 --2- 172.21.15.107:0/1342646904 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7feb600a2d30 0x7feb600a51f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.114+0000 7feb7b59e640 1 --2- 172.21.15.107:0/1342646904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feb740ffbb0 0x7feb74100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.114+0000 7feb7b59e640 1 -- 172.21.15.107:0/1342646904 >> 172.21.15.107:0/1342646904 conn(0x7feb74076b70 msgr2=0x7feb740fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:45.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.114+0000 7feb7b59e640 1 -- 172.21.15.107:0/1342646904 shutdown_connections 2024-04-25T00:25:45.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.114+0000 7feb7b59e640 1 -- 172.21.15.107:0/1342646904 wait complete. 2024-04-25T00:25:45.125 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:25:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.224+0000 7f24279f9640 1 -- 172.21.15.107:0/1524384077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24201008d0 msgr2=0x7f2420106310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.224+0000 7f24279f9640 1 --2- 172.21.15.107:0/1524384077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24201008d0 0x7f2420106310 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f241c009930 tx=0x7f241c02a420 comp rx=0 tx=0).stop 2024-04-25T00:25:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.224+0000 7f24279f9640 1 -- 172.21.15.107:0/1524384077 shutdown_connections 2024-04-25T00:25:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.224+0000 7f24279f9640 1 --2- 172.21.15.107:0/1524384077 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2420106a40 0x7f242010e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.224+0000 7f24279f9640 1 --2- 172.21.15.107:0/1524384077 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24201008d0 0x7f2420106310 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.224+0000 7f24279f9640 1 --2- 172.21.15.107:0/1524384077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24200fff20 0x7f2420100300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.224+0000 7f24279f9640 1 -- 172.21.15.107:0/1524384077 >> 172.21.15.107:0/1524384077 conn(0x7f2420076c40 msgr2=0x7f24200fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.224+0000 7f24279f9640 1 -- 172.21.15.107:0/1524384077 shutdown_connections 2024-04-25T00:25:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.224+0000 7f24279f9640 1 -- 172.21.15.107:0/1524384077 wait complete. 2024-04-25T00:25:45.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.224+0000 7f24279f9640 1 Processor -- start 2024-04-25T00:25:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.224+0000 7f24279f9640 1 -- start start 2024-04-25T00:25:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.225+0000 7f24279f9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24200fff20 0x7f2420197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.225+0000 7f24279f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2420106a40 0x7f2420198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.225+0000 7f24279f9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f242019c8a0 0x7f242019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.225+0000 7f24279f9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2420111b00 con 0x7f2420106a40 2024-04-25T00:25:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.225+0000 7f24279f9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2420111980 con 0x7f242019c8a0 2024-04-25T00:25:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.225+0000 7f24279f9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f2420111c80 con 0x7f24200fff20 2024-04-25T00:25:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.225+0000 7f2424f94640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2420106a40 0x7f2420198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.225+0000 7f2424f94640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2420106a40 0x7f2420198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55892/0 (socket says 172.21.15.107:55892) 2024-04-25T00:25:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.225+0000 7f2424f94640 1 -- 172.21.15.107:0/2446741293 learned_addr learned my addr 172.21.15.107:0/2446741293 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.225+0000 7f2425795640 1 --2- 172.21.15.107:0/2446741293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24200fff20 0x7f2420197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.225+0000 7f2425f96640 1 --2- 172.21.15.107:0/2446741293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f242019c8a0 0x7f242019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.225+0000 7f2424f94640 1 -- 172.21.15.107:0/2446741293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24200fff20 msgr2=0x7f2420197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.225+0000 7f2424f94640 1 --2- 172.21.15.107:0/2446741293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24200fff20 0x7f2420197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.225+0000 7f2424f94640 1 -- 172.21.15.107:0/2446741293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f242019c8a0 msgr2=0x7f242019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.225+0000 7f2424f94640 1 --2- 172.21.15.107:0/2446741293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f242019c8a0 0x7f242019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.225+0000 7f2424f94640 1 -- 172.21.15.107:0/2446741293 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242019d390 con 0x7f2420106a40 2024-04-25T00:25:45.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.225+0000 7f2424f94640 1 --2- 172.21.15.107:0/2446741293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2420106a40 0x7f2420198520 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f241c005940 tx=0x7f241c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:45.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.225+0000 7f24167fc640 1 -- 172.21.15.107:0/2446741293 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c03f030 con 0x7f2420106a40 2024-04-25T00:25:45.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.225+0000 7f24167fc640 1 -- 172.21.15.107:0/2446741293 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f241c003ce0 con 0x7f2420106a40 2024-04-25T00:25:45.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.225+0000 7f24279f9640 1 -- 172.21.15.107:0/2446741293 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f242019d620 con 0x7f2420106a40 2024-04-25T00:25:45.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.225+0000 7f24167fc640 1 -- 172.21.15.107:0/2446741293 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c035070 con 0x7f2420106a40 2024-04-25T00:25:45.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.225+0000 7f24279f9640 1 -- 172.21.15.107:0/2446741293 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24201a4de0 con 0x7f2420106a40 2024-04-25T00:25:45.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.228+0000 7f24167fc640 1 -- 172.21.15.107:0/2446741293 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f241c042030 con 0x7f2420106a40 2024-04-25T00:25:45.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.228+0000 7f242a7e2740 1 Processor -- start 2024-04-25T00:25:45.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.228+0000 7f242a7e2740 1 -- start start 2024-04-25T00:25:45.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.228+0000 7f242a7e2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cd8c4ca580 0x55cd8c4ca960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.228+0000 7f242a7e2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8c4caea0 0x55cd8c4d5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.228+0000 7f242a7e2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4d5570 0x55cd8c4d7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.228+0000 7f242a7e2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cd8c42f670 con 0x55cd8c4d5570 2024-04-25T00:25:45.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.228+0000 7f242a7e2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cd8c3d7970 con 0x55cd8c4caea0 2024-04-25T00:25:45.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.228+0000 7f242a7e2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cd8c42b630 con 0x55cd8c4ca580 2024-04-25T00:25:45.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.228+0000 7f24167fc640 1 --2- 172.21.15.107:0/2446741293 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f240c0a2d00 0x7f240c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.228+0000 7f2424f94640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4d5570 0x55cd8c4d7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.229+0000 7f2424f94640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4d5570 0x55cd8c4d7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55904/0 (socket says 172.21.15.107:55904) 2024-04-25T00:25:45.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.229+0000 7f2424f94640 1 -- 172.21.15.107:0/3488973288 learned_addr learned my addr 172.21.15.107:0/3488973288 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:45.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.229+0000 7f2425f96640 1 --2- 172.21.15.107:0/2446741293 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f240c0a2d00 0x7f240c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.229+0000 7f2425f96640 1 --2- 172.21.15.107:0/3488973288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cd8c4ca580 0x55cd8c4ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.229+0000 7f2425795640 1 --2- 172.21.15.107:0/3488973288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8c4caea0 0x55cd8c4d5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.229+0000 7f24167fc640 1 -- 172.21.15.107:0/2446741293 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7f241c0efd30 con 0x7f2420106a40 2024-04-25T00:25:45.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.229+0000 7f2424f94640 1 -- 172.21.15.107:0/3488973288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cd8c4ca580 msgr2=0x55cd8c4ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:45.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.229+0000 7f2424f94640 1 --2- 172.21.15.107:0/3488973288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cd8c4ca580 0x55cd8c4ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.229+0000 7f2424f94640 1 -- 172.21.15.107:0/3488973288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8c4caea0 msgr2=0x55cd8c4d5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:45.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.229+0000 7f2424f94640 1 --2- 172.21.15.107:0/3488973288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8c4caea0 0x55cd8c4d5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.229+0000 7f2424f94640 1 -- 172.21.15.107:0/3488973288 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd8c3fb400 con 0x55cd8c4d5570 2024-04-25T00:25:45.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.229+0000 7f2425f96640 1 --2- 172.21.15.107:0/3488973288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cd8c4ca580 0x55cd8c4ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:45.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.229+0000 7f2425f96640 1 --2- 172.21.15.107:0/2446741293 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f240c0a2d00 0x7f240c0a51c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2418005fd0 tx=0x7f2418009a30 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:45.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.229+0000 7f2424f94640 1 --2- 172.21.15.107:0/3488973288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4d5570 0x55cd8c4d7960 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f241c009900 tx=0x7f241c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98aec98613a83ae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:45.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.229+0000 7f240b7fe640 1 -- 172.21.15.107:0/3488973288 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c049070 con 0x55cd8c4d5570 2024-04-25T00:25:45.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.229+0000 7f240b7fe640 1 -- 172.21.15.107:0/3488973288 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f241c0bc590 con 0x55cd8c4d5570 2024-04-25T00:25:45.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.229+0000 7f240b7fe640 1 -- 172.21.15.107:0/3488973288 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c0ed340 con 0x55cd8c4d5570 2024-04-25T00:25:45.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.230+0000 7f242a7e2740 1 -- 172.21.15.107:0/3488973288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4d5570 msgr2=0x55cd8c4d7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.230+0000 7f242a7e2740 1 --2- 172.21.15.107:0/3488973288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4d5570 0x55cd8c4d7960 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f241c009900 tx=0x7f241c02a420 comp rx=0 tx=0).stop 2024-04-25T00:25:45.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.230+0000 7f242a7e2740 1 -- 172.21.15.107:0/3488973288 shutdown_connections 2024-04-25T00:25:45.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.230+0000 7f242a7e2740 1 --2- 172.21.15.107:0/3488973288 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4d5570 0x55cd8c4d7960 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.230+0000 7f242a7e2740 1 --2- 172.21.15.107:0/3488973288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8c4caea0 0x55cd8c4d5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.230+0000 7f242a7e2740 1 --2- 172.21.15.107:0/3488973288 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cd8c4ca580 0x55cd8c4ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.230+0000 7f242a7e2740 1 -- 172.21.15.107:0/3488973288 >> 172.21.15.107:0/3488973288 conn(0x55cd8c4c68a0 msgr2=0x55cd8c4c6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:45.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.230+0000 7f242a7e2740 1 -- 172.21.15.107:0/3488973288 shutdown_connections 2024-04-25T00:25:45.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.230+0000 7f242a7e2740 1 -- 172.21.15.107:0/3488973288 wait complete. 2024-04-25T00:25:45.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.230+0000 7f242a7e2740 1 -- start start 2024-04-25T00:25:45.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.230+0000 7f242a7e2740 1 Processor -- start 2024-04-25T00:25:45.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.231+0000 7f242a7e2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cd8c4ca580 0x55cd8c4d4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.231+0000 7f242a7e2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8c4caea0 0x55cd8c4d21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.231+0000 7f242a7e2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4d5570 0x55cd8c4d2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.231+0000 7f242a7e2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55cd8c4d8120 con 0x55cd8c4d5570 2024-04-25T00:25:45.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.231+0000 7f242a7e2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cd8c3eb050 con 0x55cd8c4caea0 2024-04-25T00:25:45.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.231+0000 7f242a7e2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55cd8c4d82a0 con 0x55cd8c4ca580 2024-04-25T00:25:45.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.231+0000 7f242a7e2740 10 client.0 _ll_register_callbacks cb 0x55cd8c142be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:45.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.231+0000 7f2425f96640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4d5570 0x55cd8c4d2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.231+0000 7f2425f96640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4d5570 0x55cd8c4d2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55914/0 (socket says 172.21.15.107:55914) 2024-04-25T00:25:45.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.231+0000 7f2425f96640 1 -- 172.21.15.107:0/4166541002 learned_addr learned my addr 172.21.15.107:0/4166541002 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:45.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.231+0000 7f2424f94640 1 --2- 172.21.15.107:0/4166541002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8c4caea0 0x55cd8c4d21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.231+0000 7f2425795640 1 --2- 172.21.15.107:0/4166541002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cd8c4ca580 0x55cd8c4d4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.231+0000 7f2425f96640 1 -- 172.21.15.107:0/4166541002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cd8c4ca580 msgr2=0x55cd8c4d4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:45.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.231+0000 7f2425f96640 1 --2- 172.21.15.107:0/4166541002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cd8c4ca580 0x55cd8c4d4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.231+0000 7f2425f96640 1 -- 172.21.15.107:0/4166541002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8c4caea0 msgr2=0x55cd8c4d21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:45.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.231+0000 7f2425f96640 1 --2- 172.21.15.107:0/4166541002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8c4caea0 0x55cd8c4d21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.231+0000 7f2425f96640 1 -- 172.21.15.107:0/4166541002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd8c3bcdc0 con 0x55cd8c4d5570 2024-04-25T00:25:45.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.231+0000 7f2425f96640 1 --2- 172.21.15.107:0/4166541002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4d5570 0x55cd8c4d2710 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f2418010a70 tx=0x7f2418005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a105d3e48982f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:45.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.231+0000 7f2408ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:25:45.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.231+0000 7f2408ff9640 1 -- 172.21.15.107:0/4166541002 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418029410 con 0x55cd8c4d5570 2024-04-25T00:25:45.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.232+0000 7f2408ff9640 1 -- 172.21.15.107:0/4166541002 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7f24180295b0 con 0x55cd8c4d5570 2024-04-25T00:25:45.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.232+0000 7f242a7e2740 1 -- 172.21.15.107:0/4166541002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f241c0bc590 con 0x55cd8c4d5570 2024-04-25T00:25:45.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.232+0000 7f2408ff9640 1 -- 172.21.15.107:0/4166541002 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f24180297b0 con 0x55cd8c4d5570 2024-04-25T00:25:45.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.232+0000 7f242a7e2740 10 client.17670 fetch_fsmap learned FSMap version 34 2024-04-25T00:25:45.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.232+0000 7f242a7e2740 1 -- 172.21.15.107:0/4166541002 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55cd8c1433f0 con 0x55cd8c4d5570 2024-04-25T00:25:45.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.232+0000 7f2408ff9640 1 -- 172.21.15.107:0/4166541002 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f241c0bc590 con 0x55cd8c4d5570 2024-04-25T00:25:45.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.232+0000 7f2408ff9640 1 -- 172.21.15.107:0/4166541002 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f241801fa30 con 0x55cd8c4d5570 2024-04-25T00:25:45.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.232+0000 7f242a7e2740 10 client.17670 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:25:45.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.232+0000 7f242a7e2740 10 client.17670 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:25:45.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.232+0000 7f242a7e2740 20 client.17670 populate_metadata read hostname 'smithi107' 2024-04-25T00:25:45.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.232+0000 7f242a7e2740 1 --2- 172.21.15.107:0/4166541002 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55cd8c4e52c0 0x55cd8c4e76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.232+0000 7f242a7e2740 4 client.17670 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:45.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.232+0000 7f242a7e2740 1 -- 172.21.15.107:0/4166541002 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd8c43cbb0 con 0x55cd8c4e52c0 2024-04-25T00:25:45.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.232+0000 7f2425795640 1 --2- 172.21.15.107:0/4166541002 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55cd8c4e52c0 0x55cd8c4e76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.233+0000 7f2408ff9640 10 client.17670 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:25:45.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.233+0000 7f2425795640 1 --2- 172.21.15.107:0/4166541002 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55cd8c4e52c0 0x55cd8c4e76f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b9bca94abb2d1c64 server_cookie=e7830516f77dfd42 in_seq=0 out_seq=0 2024-04-25T00:25:45.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.233+0000 7f2408ff9640 1 -- 172.21.15.107:0/4166541002 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cd8c43cbb0 con 0x55cd8c4e52c0 2024-04-25T00:25:45.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.233+0000 7f2408ff9640 10 client.17670 handle_command_reply: tid=0 2024-04-25T00:25:45.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.233+0000 7f242a7e2740 1 client.17670 shutdown 2024-04-25T00:25:45.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.233+0000 7f242a7e2740 1 -- 172.21.15.107:0/4166541002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4d5570 msgr2=0x55cd8c4d2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.233+0000 7f242a7e2740 1 --2- 172.21.15.107:0/4166541002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4d5570 0x55cd8c4d2710 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f2418010a70 tx=0x7f2418005bd0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.234+0000 7f242a7e2740 1 -- 172.21.15.107:0/4166541002 shutdown_connections 2024-04-25T00:25:45.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.234+0000 7f242a7e2740 1 --2- 172.21.15.107:0/4166541002 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55cd8c4e52c0 0x55cd8c4e76f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.234+0000 7f242a7e2740 1 --2- 172.21.15.107:0/4166541002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4d5570 0x55cd8c4d2710 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.234+0000 7f242a7e2740 1 --2- 172.21.15.107:0/4166541002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8c4caea0 0x55cd8c4d21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.234+0000 7f242a7e2740 1 --2- 172.21.15.107:0/4166541002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cd8c4ca580 0x55cd8c4d4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.234+0000 7f2408ff9640 0 client.17670 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:25:45.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.234+0000 7f242a7e2740 1 -- 172.21.15.107:0/4166541002 >> 172.21.15.107:0/4166541002 conn(0x55cd8c4c68a0 msgr2=0x55cd8c4cc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:45.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.234+0000 7f242a7e2740 1 -- 172.21.15.107:0/4166541002 shutdown_connections 2024-04-25T00:25:45.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.234+0000 7f242a7e2740 1 -- 172.21.15.107:0/4166541002 wait complete. 2024-04-25T00:25:45.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.234+0000 7f242a7e2740 20 client.17670 trim_cache size 0 max 16384 2024-04-25T00:25:45.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.247+0000 7f242a7e2740 1 Processor -- start 2024-04-25T00:25:45.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.247+0000 7f242a7e2740 1 -- start start 2024-04-25T00:25:45.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.247+0000 7f242a7e2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4ca580 0x55cd8c4ecfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.247+0000 7f242a7e2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8c4caea0 0x55cd8c4ed500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.247+0000 7f242a7e2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cd8c4d5570 0x55cd8c4cf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.247+0000 7f242a7e2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f24201b13c0 con 0x55cd8c4ca580 2024-04-25T00:25:45.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.247+0000 7f242a7e2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f24201aec90 con 0x55cd8c4caea0 2024-04-25T00:25:45.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.247+0000 7f242a7e2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f24201acb00 con 0x55cd8c4d5570 2024-04-25T00:25:45.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.247+0000 7f2425795640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4ca580 0x55cd8c4ecfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.247+0000 7f2425795640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4ca580 0x55cd8c4ecfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55918/0 (socket says 172.21.15.107:55918) 2024-04-25T00:25:45.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.247+0000 7f2425795640 1 -- 172.21.15.107:0/729353793 learned_addr learned my addr 172.21.15.107:0/729353793 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:45.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.248+0000 7f2425795640 1 -- 172.21.15.107:0/729353793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cd8c4d5570 msgr2=0x55cd8c4cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:45.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.248+0000 7f2425795640 1 --2- 172.21.15.107:0/729353793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cd8c4d5570 0x55cd8c4cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.248+0000 7f2425795640 1 -- 172.21.15.107:0/729353793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8c4caea0 msgr2=0x55cd8c4ed500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:45.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.248+0000 7f2425795640 1 --2- 172.21.15.107:0/729353793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8c4caea0 0x55cd8c4ed500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.248+0000 7f2425795640 1 -- 172.21.15.107:0/729353793 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd8c20d7d0 con 0x55cd8c4ca580 2024-04-25T00:25:45.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.248+0000 7f2425795640 1 --2- 172.21.15.107:0/729353793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4ca580 0x55cd8c4ecfc0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x55cd8c4e7f60 tx=0x7f241000a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e5df27dfa569b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:45.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.248+0000 7f240affd640 1 -- 172.21.15.107:0/729353793 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410005220 con 0x55cd8c4ca580 2024-04-25T00:25:45.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.248+0000 7f240affd640 1 -- 172.21.15.107:0/729353793 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2410024030 con 0x55cd8c4ca580 2024-04-25T00:25:45.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.248+0000 7f240affd640 1 -- 172.21.15.107:0/729353793 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410020540 con 0x55cd8c4ca580 2024-04-25T00:25:45.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.248+0000 7f242a7e2740 1 -- 172.21.15.107:0/729353793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4ca580 msgr2=0x55cd8c4ecfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.248+0000 7f242a7e2740 1 --2- 172.21.15.107:0/729353793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4ca580 0x55cd8c4ecfc0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x55cd8c4e7f60 tx=0x7f241000a250 comp rx=0 tx=0).stop 2024-04-25T00:25:45.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.248+0000 7f242a7e2740 1 -- 172.21.15.107:0/729353793 shutdown_connections 2024-04-25T00:25:45.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.248+0000 7f242a7e2740 1 --2- 172.21.15.107:0/729353793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cd8c4d5570 0x55cd8c4cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.248+0000 7f242a7e2740 1 --2- 172.21.15.107:0/729353793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8c4caea0 0x55cd8c4ed500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.248+0000 7f242a7e2740 1 --2- 172.21.15.107:0/729353793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4ca580 0x55cd8c4ecfc0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.248+0000 7f242a7e2740 1 -- 172.21.15.107:0/729353793 >> 172.21.15.107:0/729353793 conn(0x55cd8c4c68a0 msgr2=0x55cd8c4c5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:45.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.248+0000 7f242a7e2740 1 -- 172.21.15.107:0/729353793 shutdown_connections 2024-04-25T00:25:45.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.249+0000 7f242a7e2740 1 -- 172.21.15.107:0/729353793 wait complete. 2024-04-25T00:25:45.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.249+0000 7f242a7e2740 1 -- start start 2024-04-25T00:25:45.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.249+0000 7f242a7e2740 1 Processor -- start 2024-04-25T00:25:45.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.249+0000 7f242a7e2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4ca580 0x55cd8c4dd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.249+0000 7f242a7e2740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cd8c4caea0 0x55cd8c4daae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.249+0000 7f242a7e2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8c4d5570 0x55cd8c4db020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.249+0000 7f242a7e2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f24201ad150 con 0x55cd8c4ca580 2024-04-25T00:25:45.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.249+0000 7f242a7e2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f24201ae5c0 con 0x55cd8c4d5570 2024-04-25T00:25:45.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.249+0000 7f242a7e2740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f24201ad820 con 0x55cd8c4caea0 2024-04-25T00:25:45.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.249+0000 7f242a7e2740 10 client.0 _ll_register_callbacks cb 0x7f2418010770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:45.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.249+0000 7f2425795640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4ca580 0x55cd8c4dd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.249+0000 7f2425795640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4ca580 0x55cd8c4dd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55930/0 (socket says 172.21.15.107:55930) 2024-04-25T00:25:45.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.249+0000 7f2425795640 1 -- 172.21.15.107:0/2524281267 learned_addr learned my addr 172.21.15.107:0/2524281267 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:45.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.250+0000 7f2425f96640 1 --2- 172.21.15.107:0/2524281267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8c4d5570 0x55cd8c4db020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.250+0000 7f2425795640 1 -- 172.21.15.107:0/2524281267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cd8c4caea0 msgr2=0x55cd8c4daae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:45.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.250+0000 7f2425795640 1 --2- 172.21.15.107:0/2524281267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cd8c4caea0 0x55cd8c4daae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.250+0000 7f2425795640 1 -- 172.21.15.107:0/2524281267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8c4d5570 msgr2=0x55cd8c4db020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:45.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.250+0000 7f2425795640 1 --2- 172.21.15.107:0/2524281267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8c4d5570 0x55cd8c4db020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.250+0000 7f2425795640 1 -- 172.21.15.107:0/2524281267 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd8c4db760 con 0x55cd8c4ca580 2024-04-25T00:25:45.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.250+0000 7f2425f96640 1 --2- 172.21.15.107:0/2524281267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8c4d5570 0x55cd8c4db020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:45.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.250+0000 7f2425795640 1 --2- 172.21.15.107:0/2524281267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4ca580 0x55cd8c4dd750 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f2410005af0 tx=0x7f2410005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd23e841ac3e0a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:45.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.250+0000 7f2408ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:25:45.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.250+0000 7f2408ff9640 1 -- 172.21.15.107:0/2524281267 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410004220 con 0x55cd8c4ca580 2024-04-25T00:25:45.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.250+0000 7f2408ff9640 1 -- 172.21.15.107:0/2524281267 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7f24100043c0 con 0x55cd8c4ca580 2024-04-25T00:25:45.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.250+0000 7f242a7e2740 1 -- 172.21.15.107:0/2524281267 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2410024030 con 0x55cd8c4ca580 2024-04-25T00:25:45.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.250+0000 7f2408ff9640 1 -- 172.21.15.107:0/2524281267 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f24100056b0 con 0x55cd8c4ca580 2024-04-25T00:25:45.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.250+0000 7f242a7e2740 10 client.17676 fetch_fsmap learned FSMap version 34 2024-04-25T00:25:45.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.250+0000 7f242a7e2740 1 -- 172.21.15.107:0/2524281267 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55cd8c54b770 con 0x55cd8c4ca580 2024-04-25T00:25:45.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.250+0000 7f2408ff9640 1 -- 172.21.15.107:0/2524281267 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2410024030 con 0x55cd8c4ca580 2024-04-25T00:25:45.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.250+0000 7f2408ff9640 1 -- 172.21.15.107:0/2524281267 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f2410020700 con 0x55cd8c4ca580 2024-04-25T00:25:45.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.250+0000 7f242a7e2740 10 client.17676 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:25:45.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.250+0000 7f242a7e2740 10 client.17676 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:25:45.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.250+0000 7f242a7e2740 20 client.17676 populate_metadata read hostname 'smithi107' 2024-04-25T00:25:45.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.250+0000 7f242a7e2740 1 --2- 172.21.15.107:0/2524281267 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55cd8c54be30 0x55cd8c54e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.250+0000 7f242a7e2740 4 client.17676 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:25:45.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.250+0000 7f242a7e2740 1 -- 172.21.15.107:0/2524281267 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cd8c436f00 con 0x55cd8c54be30 2024-04-25T00:25:45.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.251+0000 7f2424f94640 1 --2- 172.21.15.107:0/2524281267 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55cd8c54be30 0x55cd8c54e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.251+0000 7f2424f94640 1 --2- 172.21.15.107:0/2524281267 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55cd8c54be30 0x55cd8c54e250 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fd04e8c6268cb28d server_cookie=4bcc74ee88f12cca in_seq=0 out_seq=0 2024-04-25T00:25:45.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.251+0000 7f2408ff9640 10 client.17676 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:25:45.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.251+0000 7f2408ff9640 1 -- 172.21.15.107:0/2524281267 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cd8c436f00 con 0x55cd8c54be30 2024-04-25T00:25:45.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.251+0000 7f2408ff9640 10 client.17676 handle_command_reply: tid=0 2024-04-25T00:25:45.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.251+0000 7f242a7e2740 1 client.17676 shutdown 2024-04-25T00:25:45.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.251+0000 7f242a7e2740 1 -- 172.21.15.107:0/2524281267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4ca580 msgr2=0x55cd8c4dd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.251+0000 7f242a7e2740 1 --2- 172.21.15.107:0/2524281267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4ca580 0x55cd8c4dd750 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f2410005af0 tx=0x7f2410005280 comp rx=0 tx=0).stop 2024-04-25T00:25:45.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.251+0000 7f242a7e2740 1 -- 172.21.15.107:0/2524281267 shutdown_connections 2024-04-25T00:25:45.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.251+0000 7f242a7e2740 1 --2- 172.21.15.107:0/2524281267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cd8c4d5570 0x55cd8c4db020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.251+0000 7f242a7e2740 1 --2- 172.21.15.107:0/2524281267 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55cd8c4caea0 0x55cd8c4daae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.251+0000 7f242a7e2740 1 --2- 172.21.15.107:0/2524281267 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55cd8c54be30 0x55cd8c54e250 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.251+0000 7f242a7e2740 1 --2- 172.21.15.107:0/2524281267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd8c4ca580 0x55cd8c4dd750 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.251+0000 7f242a7e2740 1 -- 172.21.15.107:0/2524281267 >> 172.21.15.107:0/2524281267 conn(0x55cd8c4c68a0 msgr2=0x55cd8c4dff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:45.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.251+0000 7f2408ff9640 0 client.17676 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:25:45.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.251+0000 7f242a7e2740 1 -- 172.21.15.107:0/2524281267 shutdown_connections 2024-04-25T00:25:45.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.251+0000 7f242a7e2740 1 -- 172.21.15.107:0/2524281267 wait complete. 2024-04-25T00:25:45.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.252+0000 7f242a7e2740 20 client.17676 trim_cache size 0 max 16384 2024-04-25T00:25:45.261 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:25:45.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.252+0000 7f24279f9640 1 -- 172.21.15.107:0/2446741293 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f240c0a2d00 msgr2=0x7f240c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.252+0000 7f24279f9640 1 --2- 172.21.15.107:0/2446741293 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f240c0a2d00 0x7f240c0a51c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2418005fd0 tx=0x7f2418009a30 comp rx=0 tx=0).stop 2024-04-25T00:25:45.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.252+0000 7f24279f9640 1 -- 172.21.15.107:0/2446741293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2420106a40 msgr2=0x7f2420198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.252+0000 7f24279f9640 1 --2- 172.21.15.107:0/2446741293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2420106a40 0x7f2420198520 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f241c005940 tx=0x7f241c0042f0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.252+0000 7f24279f9640 1 -- 172.21.15.107:0/2446741293 shutdown_connections 2024-04-25T00:25:45.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.252+0000 7f24279f9640 1 --2- 172.21.15.107:0/2446741293 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f240c0a2d00 0x7f240c0a51c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.252+0000 7f24279f9640 1 --2- 172.21.15.107:0/2446741293 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f242019c8a0 0x7f242019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.252+0000 7f24279f9640 1 --2- 172.21.15.107:0/2446741293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2420106a40 0x7f2420198520 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.252+0000 7f24279f9640 1 --2- 172.21.15.107:0/2446741293 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24200fff20 0x7f2420197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.252+0000 7f24279f9640 1 -- 172.21.15.107:0/2446741293 >> 172.21.15.107:0/2446741293 conn(0x7f2420076c40 msgr2=0x7f24200fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:45.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.252+0000 7f24279f9640 1 -- 172.21.15.107:0/2446741293 shutdown_connections 2024-04-25T00:25:45.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.252+0000 7f24279f9640 1 -- 172.21.15.107:0/2446741293 wait complete. 2024-04-25T00:25:45.264 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:25:45.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.362+0000 7f005ed10640 1 -- 172.21.15.107:0/2160760883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0058107720 msgr2=0x7f0058107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.362+0000 7f005ed10640 1 --2- 172.21.15.107:0/2160760883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0058107720 0x7f0058107b00 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f0048009930 tx=0x7f004802a420 comp rx=0 tx=0).stop 2024-04-25T00:25:45.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.362+0000 7f005ed10640 1 -- 172.21.15.107:0/2160760883 shutdown_connections 2024-04-25T00:25:45.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.362+0000 7f005ed10640 1 --2- 172.21.15.107:0/2160760883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f005806a700 0x7f005810e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.362+0000 7f005ed10640 1 --2- 172.21.15.107:0/2160760883 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0058069d40 0x7f005806a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.362+0000 7f005ed10640 1 --2- 172.21.15.107:0/2160760883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0058107720 0x7f0058107b00 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.362+0000 7f005ed10640 1 -- 172.21.15.107:0/2160760883 >> 172.21.15.107:0/2160760883 conn(0x7f0058076440 msgr2=0x7f0058076850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:45.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.362+0000 7f005ed10640 1 -- 172.21.15.107:0/2160760883 shutdown_connections 2024-04-25T00:25:45.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.362+0000 7f005ed10640 1 -- 172.21.15.107:0/2160760883 wait complete. 2024-04-25T00:25:45.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.363+0000 7f005ed10640 1 Processor -- start 2024-04-25T00:25:45.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.363+0000 7f005ed10640 1 -- start start 2024-04-25T00:25:45.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.363+0000 7f005ed10640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0058069d40 0x7f0058198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.363+0000 7f005ed10640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f005806a700 0x7f0058198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.363+0000 7f005ed10640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0058107720 0x7f005819c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.363+0000 7f005ed10640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0058111b30 con 0x7f0058107720 2024-04-25T00:25:45.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.363+0000 7f005ed10640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f00581119b0 con 0x7f005806a700 2024-04-25T00:25:45.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.363+0000 7f005ed10640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0058111cb0 con 0x7f0058069d40 2024-04-25T00:25:45.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.363+0000 7f005d2ad640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0058107720 0x7f005819c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.363+0000 7f005d2ad640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0058107720 0x7f005819c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55940/0 (socket says 172.21.15.107:55940) 2024-04-25T00:25:45.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.363+0000 7f005d2ad640 1 -- 172.21.15.107:0/3547659225 learned_addr learned my addr 172.21.15.107:0/3547659225 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:45.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.363+0000 7f0057fff640 1 --2- 172.21.15.107:0/3547659225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f005806a700 0x7f0058198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.363+0000 7f005caac640 1 --2- 172.21.15.107:0/3547659225 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0058069d40 0x7f0058198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.363+0000 7f005d2ad640 1 -- 172.21.15.107:0/3547659225 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0058069d40 msgr2=0x7f0058198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.363+0000 7f005d2ad640 1 --2- 172.21.15.107:0/3547659225 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0058069d40 0x7f0058198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.363+0000 7f005d2ad640 1 -- 172.21.15.107:0/3547659225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f005806a700 msgr2=0x7f0058198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.364+0000 7f005d2ad640 1 --2- 172.21.15.107:0/3547659225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f005806a700 0x7f0058198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.364+0000 7f005d2ad640 1 -- 172.21.15.107:0/3547659225 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005819d040 con 0x7f0058107720 2024-04-25T00:25:45.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.364+0000 7f005caac640 1 --2- 172.21.15.107:0/3547659225 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0058069d40 0x7f0058198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:45.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.364+0000 7f005d2ad640 1 --2- 172.21.15.107:0/3547659225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0058107720 0x7f005819c910 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f005000cad0 tx=0x7f005000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:45.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.364+0000 7f0055ffb640 1 -- 172.21.15.107:0/3547659225 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005000ae20 con 0x7f0058107720 2024-04-25T00:25:45.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.364+0000 7f0055ffb640 1 -- 172.21.15.107:0/3547659225 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0050005020 con 0x7f0058107720 2024-04-25T00:25:45.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.364+0000 7f0055ffb640 1 -- 172.21.15.107:0/3547659225 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005000f6f0 con 0x7f0058107720 2024-04-25T00:25:45.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.364+0000 7f005ed10640 1 -- 172.21.15.107:0/3547659225 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005819d330 con 0x7f0058107720 2024-04-25T00:25:45.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.364+0000 7f005ed10640 1 -- 172.21.15.107:0/3547659225 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00581a4bc0 con 0x7f0058107720 2024-04-25T00:25:45.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.366+0000 7f0055ffb640 1 -- 172.21.15.107:0/3547659225 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f00500051d0 con 0x7f0058107720 2024-04-25T00:25:45.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.366+0000 7f0061a18740 1 Processor -- start 2024-04-25T00:25:45.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.366+0000 7f0061a18740 1 -- start start 2024-04-25T00:25:45.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.367+0000 7f0055ffb640 1 --2- 172.21.15.107:0/3547659225 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f00400a2d00 0x7f00400a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.367+0000 7f0061a18740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56161a0b2580 0x56161a0b2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.367+0000 7f0061a18740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2ea0 0x56161a0bd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.367+0000 7f0061a18740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56161a0bd570 0x56161a0bf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.367+0000 7f0061a18740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56161a017670 con 0x56161a0b2ea0 2024-04-25T00:25:45.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.367+0000 7f0061a18740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561619fbf970 con 0x56161a0b2580 2024-04-25T00:25:45.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.367+0000 7f0061a18740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56161a013630 con 0x56161a0bd570 2024-04-25T00:25:45.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.367+0000 7f0057fff640 1 --2- 172.21.15.107:0/3547659225 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f00400a2d00 0x7f00400a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.367+0000 7f0055ffb640 1 -- 172.21.15.107:0/3547659225 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7f00500cfaf0 con 0x7f0058107720 2024-04-25T00:25:45.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.367+0000 7f0057fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2ea0 0x56161a0bd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.367+0000 7f0057fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2ea0 0x56161a0bd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55946/0 (socket says 172.21.15.107:55946) 2024-04-25T00:25:45.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.367+0000 7f0057fff640 1 -- 172.21.15.107:0/146188793 learned_addr learned my addr 172.21.15.107:0/146188793 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:45.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.367+0000 7f005caac640 1 --2- 172.21.15.107:0/146188793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56161a0b2580 0x56161a0b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.367+0000 7f005d2ad640 1 --2- 172.21.15.107:0/146188793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56161a0bd570 0x56161a0bf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.367+0000 7f0057fff640 1 -- 172.21.15.107:0/146188793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56161a0bd570 msgr2=0x56161a0bf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:45.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.367+0000 7f0057fff640 1 --2- 172.21.15.107:0/146188793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56161a0bd570 0x56161a0bf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.367+0000 7f0057fff640 1 -- 172.21.15.107:0/146188793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56161a0b2580 msgr2=0x56161a0b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:45.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.367+0000 7f0057fff640 1 --2- 172.21.15.107:0/146188793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56161a0b2580 0x56161a0b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.367+0000 7f0057fff640 1 -- 172.21.15.107:0/146188793 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561619fe3400 con 0x56161a0b2ea0 2024-04-25T00:25:45.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.367+0000 7f0057fff640 1 --2- 172.21.15.107:0/3547659225 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f00400a2d00 0x7f00400a51c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f004c012960 tx=0x7f004c00bb70 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:45.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.367+0000 7f0057fff640 1 --2- 172.21.15.107:0/146188793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2ea0 0x56161a0bd030 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f004c010740 tx=0x7f004c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60825ac91a24c457 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:45.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.367+0000 7f003effd640 1 -- 172.21.15.107:0/146188793 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c024070 con 0x56161a0b2ea0 2024-04-25T00:25:45.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.367+0000 7f003effd640 1 -- 172.21.15.107:0/146188793 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f004c010e00 con 0x56161a0b2ea0 2024-04-25T00:25:45.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.367+0000 7f003effd640 1 -- 172.21.15.107:0/146188793 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c020970 con 0x56161a0b2ea0 2024-04-25T00:25:45.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.368+0000 7f0061a18740 1 -- 172.21.15.107:0/146188793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2ea0 msgr2=0x56161a0bd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.368+0000 7f0061a18740 1 --2- 172.21.15.107:0/146188793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2ea0 0x56161a0bd030 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f004c010740 tx=0x7f004c01c000 comp rx=0 tx=0).stop 2024-04-25T00:25:45.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.368+0000 7f0061a18740 1 -- 172.21.15.107:0/146188793 shutdown_connections 2024-04-25T00:25:45.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.368+0000 7f0061a18740 1 --2- 172.21.15.107:0/146188793 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56161a0bd570 0x56161a0bf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.368+0000 7f0061a18740 1 --2- 172.21.15.107:0/146188793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2ea0 0x56161a0bd030 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.368+0000 7f0061a18740 1 --2- 172.21.15.107:0/146188793 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56161a0b2580 0x56161a0b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.368+0000 7f0061a18740 1 -- 172.21.15.107:0/146188793 >> 172.21.15.107:0/146188793 conn(0x56161a0ae8a0 msgr2=0x56161a0aec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:45.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.368+0000 7f0061a18740 1 -- 172.21.15.107:0/146188793 shutdown_connections 2024-04-25T00:25:45.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.368+0000 7f0061a18740 1 -- 172.21.15.107:0/146188793 wait complete. 2024-04-25T00:25:45.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.368+0000 7f0061a18740 1 -- start start 2024-04-25T00:25:45.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.368+0000 7f0061a18740 1 Processor -- start 2024-04-25T00:25:45.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.369+0000 7f0061a18740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56161a0b2580 0x56161a0bcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.369+0000 7f0061a18740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2ea0 0x56161a0ba1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.369+0000 7f0061a18740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56161a0bd570 0x56161a0ba730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.369+0000 7f0061a18740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56161a0c0300 con 0x56161a0b2ea0 2024-04-25T00:25:45.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.369+0000 7f0061a18740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561619fd3050 con 0x56161a0b2580 2024-04-25T00:25:45.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.369+0000 7f0061a18740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56161a0c0480 con 0x56161a0bd570 2024-04-25T00:25:45.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.369+0000 7f0061a18740 10 client.0 _ll_register_callbacks cb 0x561619d2abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:45.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.369+0000 7f0057fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2ea0 0x56161a0ba1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.369+0000 7f0057fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2ea0 0x56161a0ba1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55960/0 (socket says 172.21.15.107:55960) 2024-04-25T00:25:45.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.369+0000 7f0057fff640 1 -- 172.21.15.107:0/1127122634 learned_addr learned my addr 172.21.15.107:0/1127122634 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:45.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.369+0000 7f005caac640 1 --2- 172.21.15.107:0/1127122634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56161a0b2580 0x56161a0bcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.369+0000 7f0057fff640 1 -- 172.21.15.107:0/1127122634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56161a0bd570 msgr2=0x56161a0ba730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:45.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.369+0000 7f0057fff640 1 --2- 172.21.15.107:0/1127122634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56161a0bd570 0x56161a0ba730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.369+0000 7f0057fff640 1 -- 172.21.15.107:0/1127122634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56161a0b2580 msgr2=0x56161a0bcee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:45.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.369+0000 7f0057fff640 1 --2- 172.21.15.107:0/1127122634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56161a0b2580 0x56161a0bcee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.369+0000 7f0057fff640 1 -- 172.21.15.107:0/1127122634 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561619fa4dc0 con 0x56161a0b2ea0 2024-04-25T00:25:45.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.369+0000 7f005caac640 1 --2- 172.21.15.107:0/1127122634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56161a0b2580 0x56161a0bcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:45.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.369+0000 7f0057fff640 1 --2- 172.21.15.107:0/1127122634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2ea0 0x56161a0ba1d0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f004c0058a0 tx=0x7f004c0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0ee8e69b7eefe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:45.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.369+0000 7f0037fff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:25:45.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.370+0000 7f0037fff640 1 -- 172.21.15.107:0/1127122634 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c024050 con 0x56161a0b2ea0 2024-04-25T00:25:45.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.370+0000 7f0037fff640 1 -- 172.21.15.107:0/1127122634 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7f004c00d720 con 0x56161a0b2ea0 2024-04-25T00:25:45.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.370+0000 7f0061a18740 1 -- 172.21.15.107:0/1127122634 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f004c010e00 con 0x56161a0b2ea0 2024-04-25T00:25:45.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.370+0000 7f0037fff640 1 -- 172.21.15.107:0/1127122634 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f004c00d920 con 0x56161a0b2ea0 2024-04-25T00:25:45.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.370+0000 7f0061a18740 10 client.17688 fetch_fsmap learned FSMap version 34 2024-04-25T00:25:45.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.370+0000 7f0061a18740 1 -- 172.21.15.107:0/1127122634 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561619d2b3f0 con 0x56161a0b2ea0 2024-04-25T00:25:45.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.370+0000 7f0037fff640 1 -- 172.21.15.107:0/1127122634 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f004c010e00 con 0x56161a0b2ea0 2024-04-25T00:25:45.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.370+0000 7f0037fff640 1 -- 172.21.15.107:0/1127122634 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f004c01f070 con 0x56161a0b2ea0 2024-04-25T00:25:45.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.370+0000 7f0061a18740 10 client.17688 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:25:45.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.370+0000 7f0061a18740 10 client.17688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:45.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.370+0000 7f0061a18740 20 client.17688 populate_metadata read hostname 'smithi107' 2024-04-25T00:25:45.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.370+0000 7f0061a18740 1 --2- 172.21.15.107:0/1127122634 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56161a0cd440 0x56161a0cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.370+0000 7f0061a18740 4 client.17688 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:45.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.370+0000 7f0061a18740 1 -- 172.21.15.107:0/1127122634 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56161a024bb0 con 0x56161a0cd440 2024-04-25T00:25:45.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.371+0000 7f005caac640 1 --2- 172.21.15.107:0/1127122634 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56161a0cd440 0x56161a0cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.371+0000 7f0037fff640 10 client.17688 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:45.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.371+0000 7f005caac640 1 --2- 172.21.15.107:0/1127122634 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56161a0cd440 0x56161a0cf830 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea0cce70b55b003b server_cookie=1497cc67a17a634 in_seq=0 out_seq=0 2024-04-25T00:25:45.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.372+0000 7f0037fff640 1 -- 172.21.15.107:0/1127122634 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56161a024bb0 con 0x56161a0cd440 2024-04-25T00:25:45.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.372+0000 7f0037fff640 10 client.17688 handle_command_reply: tid=0 2024-04-25T00:25:45.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.372+0000 7f0061a18740 1 client.17688 shutdown 2024-04-25T00:25:45.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.372+0000 7f0061a18740 1 -- 172.21.15.107:0/1127122634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2ea0 msgr2=0x56161a0ba1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.372+0000 7f0061a18740 1 --2- 172.21.15.107:0/1127122634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2ea0 0x56161a0ba1d0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f004c0058a0 tx=0x7f004c0052f0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.372+0000 7f0061a18740 1 -- 172.21.15.107:0/1127122634 shutdown_connections 2024-04-25T00:25:45.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.372+0000 7f0061a18740 1 --2- 172.21.15.107:0/1127122634 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56161a0cd440 0x56161a0cf830 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.372+0000 7f0061a18740 1 --2- 172.21.15.107:0/1127122634 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56161a0bd570 0x56161a0ba730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.372+0000 7f0061a18740 1 --2- 172.21.15.107:0/1127122634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2ea0 0x56161a0ba1d0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.372+0000 7f0061a18740 1 --2- 172.21.15.107:0/1127122634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56161a0b2580 0x56161a0bcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.372+0000 7f0037fff640 0 client.17688 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:45.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.372+0000 7f0061a18740 1 -- 172.21.15.107:0/1127122634 >> 172.21.15.107:0/1127122634 conn(0x56161a0ae8a0 msgr2=0x56161a0adb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:45.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.372+0000 7f0061a18740 1 -- 172.21.15.107:0/1127122634 shutdown_connections 2024-04-25T00:25:45.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.372+0000 7f0061a18740 1 -- 172.21.15.107:0/1127122634 wait complete. 2024-04-25T00:25:45.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.372+0000 7f0061a18740 20 client.17688 trim_cache size 0 max 16384 2024-04-25T00:25:45.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.385+0000 7f0061a18740 1 Processor -- start 2024-04-25T00:25:45.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.385+0000 7f0061a18740 1 -- start start 2024-04-25T00:25:45.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.385+0000 7f0061a18740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2580 0x56161a0ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.385+0000 7f0061a18740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56161a0b2ea0 0x56161a0c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.385+0000 7f0061a18740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56161a0bd570 0x56161a0d4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.385+0000 7f0061a18740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f00581b15e0 con 0x56161a0b2580 2024-04-25T00:25:45.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.385+0000 7f0061a18740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f00581aeeb0 con 0x56161a0b2ea0 2024-04-25T00:25:45.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.385+0000 7f0061a18740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f00581acd20 con 0x56161a0bd570 2024-04-25T00:25:45.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.385+0000 7f005caac640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2580 0x56161a0ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.385+0000 7f005caac640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2580 0x56161a0ae2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55964/0 (socket says 172.21.15.107:55964) 2024-04-25T00:25:45.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.385+0000 7f005caac640 1 -- 172.21.15.107:0/3710094241 learned_addr learned my addr 172.21.15.107:0/3710094241 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:45.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.385+0000 7f0057fff640 1 --2- 172.21.15.107:0/3710094241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56161a0b2ea0 0x56161a0c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.385+0000 7f005caac640 1 -- 172.21.15.107:0/3710094241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56161a0bd570 msgr2=0x56161a0d4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:45.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.385+0000 7f005d2ad640 1 --2- 172.21.15.107:0/3710094241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56161a0bd570 0x56161a0d4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.386+0000 7f005caac640 1 --2- 172.21.15.107:0/3710094241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56161a0bd570 0x56161a0d4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.386+0000 7f005caac640 1 -- 172.21.15.107:0/3710094241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56161a0b2ea0 msgr2=0x56161a0c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:45.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.386+0000 7f005caac640 1 --2- 172.21.15.107:0/3710094241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56161a0b2ea0 0x56161a0c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.386+0000 7f005caac640 1 -- 172.21.15.107:0/3710094241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561619df57d0 con 0x56161a0b2580 2024-04-25T00:25:45.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.386+0000 7f0057fff640 1 --2- 172.21.15.107:0/3710094241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56161a0b2ea0 0x56161a0c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:45.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.386+0000 7f005d2ad640 1 --2- 172.21.15.107:0/3710094241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56161a0bd570 0x56161a0d4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:45.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.386+0000 7f005caac640 1 --2- 172.21.15.107:0/3710094241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2580 0x56161a0ae2c0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f0048005ab0 tx=0x7f0048031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d51dfa09f45deb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:45.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.386+0000 7f003e7fc640 1 -- 172.21.15.107:0/3710094241 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048046020 con 0x56161a0b2580 2024-04-25T00:25:45.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.386+0000 7f003e7fc640 1 -- 172.21.15.107:0/3710094241 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0048032990 con 0x56161a0b2580 2024-04-25T00:25:45.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.386+0000 7f003e7fc640 1 -- 172.21.15.107:0/3710094241 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048035040 con 0x56161a0b2580 2024-04-25T00:25:45.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.386+0000 7f0061a18740 1 -- 172.21.15.107:0/3710094241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2580 msgr2=0x56161a0ae2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.386+0000 7f0061a18740 1 --2- 172.21.15.107:0/3710094241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2580 0x56161a0ae2c0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f0048005ab0 tx=0x7f0048031400 comp rx=0 tx=0).stop 2024-04-25T00:25:45.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.386+0000 7f0061a18740 1 -- 172.21.15.107:0/3710094241 shutdown_connections 2024-04-25T00:25:45.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.386+0000 7f0061a18740 1 --2- 172.21.15.107:0/3710094241 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56161a0bd570 0x56161a0d4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.386+0000 7f0061a18740 1 --2- 172.21.15.107:0/3710094241 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56161a0b2ea0 0x56161a0c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.386+0000 7f0061a18740 1 --2- 172.21.15.107:0/3710094241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2580 0x56161a0ae2c0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.386+0000 7f0061a18740 1 -- 172.21.15.107:0/3710094241 >> 172.21.15.107:0/3710094241 conn(0x56161a0ae8a0 msgr2=0x56161a0c8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:45.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.386+0000 7f0061a18740 1 -- 172.21.15.107:0/3710094241 shutdown_connections 2024-04-25T00:25:45.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.386+0000 7f0061a18740 1 -- 172.21.15.107:0/3710094241 wait complete. 2024-04-25T00:25:45.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.387+0000 7f0061a18740 1 -- start start 2024-04-25T00:25:45.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.387+0000 7f0061a18740 1 Processor -- start 2024-04-25T00:25:45.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.387+0000 7f0061a18740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56161a0b2580 0x56161a0c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.387+0000 7f0061a18740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2ea0 0x56161a0c3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.387+0000 7f0061a18740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56161a0bd570 0x56161a0c3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.387+0000 7f0061a18740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f00581ad370 con 0x56161a0b2ea0 2024-04-25T00:25:45.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.387+0000 7f0061a18740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f00581ae7e0 con 0x56161a0b2580 2024-04-25T00:25:45.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.387+0000 7f0061a18740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f00581ada40 con 0x56161a0bd570 2024-04-25T00:25:45.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.387+0000 7f0061a18740 10 client.0 _ll_register_callbacks cb 0x7f004c00b390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:45.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.387+0000 7f0057fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2ea0 0x56161a0c3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.387+0000 7f0057fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2ea0 0x56161a0c3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55980/0 (socket says 172.21.15.107:55980) 2024-04-25T00:25:45.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.387+0000 7f0057fff640 1 -- 172.21.15.107:0/2791513315 learned_addr learned my addr 172.21.15.107:0/2791513315 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:45.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.387+0000 7f005d2ad640 1 --2- 172.21.15.107:0/2791513315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56161a0bd570 0x56161a0c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.387+0000 7f005caac640 1 --2- 172.21.15.107:0/2791513315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56161a0b2580 0x56161a0c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.387+0000 7f0057fff640 1 -- 172.21.15.107:0/2791513315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56161a0bd570 msgr2=0x56161a0c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:45.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.387+0000 7f0057fff640 1 --2- 172.21.15.107:0/2791513315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56161a0bd570 0x56161a0c3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.395 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.387+0000 7f0057fff640 1 -- 172.21.15.107:0/2791513315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56161a0b2580 msgr2=0x56161a0c5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:45.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.387+0000 7f0057fff640 1 --2- 172.21.15.107:0/2791513315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56161a0b2580 0x56161a0c5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.388+0000 7f0057fff640 1 -- 172.21.15.107:0/2791513315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56161a132840 con 0x56161a0b2ea0 2024-04-25T00:25:45.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.388+0000 7f0057fff640 1 --2- 172.21.15.107:0/2791513315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2ea0 0x56161a0c3020 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f004c000c00 tx=0x7f004c0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c42ba24108c43c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:45.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.388+0000 7f0037fff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:25:45.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.388+0000 7f0037fff640 1 -- 172.21.15.107:0/2791513315 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c042060 con 0x56161a0b2ea0 2024-04-25T00:25:45.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.388+0000 7f0037fff640 1 -- 172.21.15.107:0/2791513315 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7f004c010e00 con 0x56161a0b2ea0 2024-04-25T00:25:45.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.388+0000 7f0061a18740 1 -- 172.21.15.107:0/2791513315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0048032990 con 0x56161a0b2ea0 2024-04-25T00:25:45.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.388+0000 7f0037fff640 1 -- 172.21.15.107:0/2791513315 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f004c049050 con 0x56161a0b2ea0 2024-04-25T00:25:45.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.388+0000 7f0061a18740 10 client.17694 fetch_fsmap learned FSMap version 34 2024-04-25T00:25:45.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.388+0000 7f0061a18740 1 -- 172.21.15.107:0/2791513315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56161a133950 con 0x56161a0b2ea0 2024-04-25T00:25:45.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.388+0000 7f0037fff640 1 -- 172.21.15.107:0/2791513315 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0048032990 con 0x56161a0b2ea0 2024-04-25T00:25:45.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.388+0000 7f0037fff640 1 -- 172.21.15.107:0/2791513315 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f004c0207a0 con 0x56161a0b2ea0 2024-04-25T00:25:45.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.388+0000 7f0061a18740 10 client.17694 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:25:45.396 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.388+0000 7f0061a18740 10 client.17694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:45.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.388+0000 7f0061a18740 20 client.17694 populate_metadata read hostname 'smithi107' 2024-04-25T00:25:45.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.388+0000 7f0061a18740 1 --2- 172.21.15.107:0/2791513315 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56161a134040 0x56161a136460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:45.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.388+0000 7f0061a18740 4 client.17694 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:25:45.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.388+0000 7f0061a18740 1 -- 172.21.15.107:0/2791513315 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56161a01ef00 con 0x56161a134040 2024-04-25T00:25:45.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.389+0000 7f005caac640 1 --2- 172.21.15.107:0/2791513315 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56161a134040 0x56161a136460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:45.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.389+0000 7f005caac640 1 --2- 172.21.15.107:0/2791513315 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56161a134040 0x56161a136460 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9840f850c1dba222 server_cookie=59fe98a1cbb467bd in_seq=0 out_seq=0 2024-04-25T00:25:45.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.389+0000 7f0037fff640 10 client.17694 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:45.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.389+0000 7f0037fff640 1 -- 172.21.15.107:0/2791513315 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56161a01ef00 con 0x56161a134040 2024-04-25T00:25:45.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.389+0000 7f0037fff640 10 client.17694 handle_command_reply: tid=0 2024-04-25T00:25:45.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.389+0000 7f0061a18740 1 client.17694 shutdown 2024-04-25T00:25:45.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.389+0000 7f0061a18740 1 -- 172.21.15.107:0/2791513315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2ea0 msgr2=0x56161a0c3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.389+0000 7f0061a18740 1 --2- 172.21.15.107:0/2791513315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2ea0 0x56161a0c3020 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f004c000c00 tx=0x7f004c0058d0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.389+0000 7f0061a18740 1 -- 172.21.15.107:0/2791513315 shutdown_connections 2024-04-25T00:25:45.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.389+0000 7f0061a18740 1 --2- 172.21.15.107:0/2791513315 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56161a134040 0x56161a136460 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.389+0000 7f0061a18740 1 --2- 172.21.15.107:0/2791513315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56161a0bd570 0x56161a0c3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.389+0000 7f0061a18740 1 --2- 172.21.15.107:0/2791513315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56161a0b2ea0 0x56161a0c3020 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.389+0000 7f0037fff640 0 client.17694 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:45.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.389+0000 7f0061a18740 1 --2- 172.21.15.107:0/2791513315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56161a0b2580 0x56161a0c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.389+0000 7f0061a18740 1 -- 172.21.15.107:0/2791513315 >> 172.21.15.107:0/2791513315 conn(0x56161a0ae8a0 msgr2=0x56161a0ad2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:45.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.390+0000 7f0061a18740 1 -- 172.21.15.107:0/2791513315 shutdown_connections 2024-04-25T00:25:45.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.390+0000 7f0061a18740 1 -- 172.21.15.107:0/2791513315 wait complete. 2024-04-25T00:25:45.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.390+0000 7f0061a18740 20 client.17694 trim_cache size 0 max 16384 2024-04-25T00:25:45.398 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:25:45.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.390+0000 7f005ed10640 1 -- 172.21.15.107:0/3547659225 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f00400a2d00 msgr2=0x7f00400a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.390+0000 7f005ed10640 1 --2- 172.21.15.107:0/3547659225 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f00400a2d00 0x7f00400a51c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f004c012960 tx=0x7f004c00bb70 comp rx=0 tx=0).stop 2024-04-25T00:25:45.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.390+0000 7f005ed10640 1 -- 172.21.15.107:0/3547659225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0058107720 msgr2=0x7f005819c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:45.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.390+0000 7f005ed10640 1 --2- 172.21.15.107:0/3547659225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0058107720 0x7f005819c910 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f005000cad0 tx=0x7f005000cf90 comp rx=0 tx=0).stop 2024-04-25T00:25:45.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.390+0000 7f005ed10640 1 -- 172.21.15.107:0/3547659225 shutdown_connections 2024-04-25T00:25:45.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.390+0000 7f005ed10640 1 --2- 172.21.15.107:0/3547659225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0058107720 0x7f005819c910 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.390+0000 7f005ed10640 1 --2- 172.21.15.107:0/3547659225 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f00400a2d00 0x7f00400a51c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.390+0000 7f005ed10640 1 --2- 172.21.15.107:0/3547659225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f005806a700 0x7f0058198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.390+0000 7f005ed10640 1 --2- 172.21.15.107:0/3547659225 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0058069d40 0x7f0058198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:45.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.390+0000 7f005ed10640 1 -- 172.21.15.107:0/3547659225 >> 172.21.15.107:0/3547659225 conn(0x7f0058076440 msgr2=0x7f00580fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:45.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.390+0000 7f005ed10640 1 -- 172.21.15.107:0/3547659225 shutdown_connections 2024-04-25T00:25:45.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:45.390+0000 7f005ed10640 1 -- 172.21.15.107:0/3547659225 wait complete. 2024-04-25T00:25:46.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:45 smithi195 ceph-mon[32833]: pgmap v288: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:25:46.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:45 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1130291924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:25:46.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:45 smithi170 ceph-mon[32807]: pgmap v288: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:25:46.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:45 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1130291924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:25:46.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:45 smithi107 ceph-mon[29153]: pgmap v288: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:25:46.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:45 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1130291924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:25:46.402 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:25:46.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:25:46.403 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:25:46.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.515+0000 7fd63a276640 1 -- 172.21.15.107:0/962983731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd634107720 msgr2=0x7fd634107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:46.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.515+0000 7fd63a276640 1 --2- 172.21.15.107:0/962983731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd634107720 0x7fd634107b00 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fd624009930 tx=0x7fd62402a420 comp rx=0 tx=0).stop 2024-04-25T00:25:46.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.515+0000 7fd63a276640 1 -- 172.21.15.107:0/962983731 shutdown_connections 2024-04-25T00:25:46.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.515+0000 7fd63a276640 1 --2- 172.21.15.107:0/962983731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6341020d0 0x7fd63410e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.515+0000 7fd63a276640 1 --2- 172.21.15.107:0/962983731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd634101710 0x7fd634101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.515+0000 7fd63a276640 1 --2- 172.21.15.107:0/962983731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd634107720 0x7fd634107b00 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.515+0000 7fd63a276640 1 -- 172.21.15.107:0/962983731 >> 172.21.15.107:0/962983731 conn(0x7fd634078b90 msgr2=0x7fd634078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:46.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.515+0000 7fd63a276640 1 -- 172.21.15.107:0/962983731 shutdown_connections 2024-04-25T00:25:46.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.515+0000 7fd63a276640 1 -- 172.21.15.107:0/962983731 wait complete. 2024-04-25T00:25:46.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.516+0000 7fd63a276640 1 Processor -- start 2024-04-25T00:25:46.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.516+0000 7fd63a276640 1 -- start start 2024-04-25T00:25:46.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.516+0000 7fd63a276640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd634101710 0x7fd634198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.516+0000 7fd63a276640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6341020d0 0x7fd6341985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.516+0000 7fd63a276640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd634107720 0x7fd63419c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.516+0000 7fd63a276640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd634111b30 con 0x7fd634107720 2024-04-25T00:25:46.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.516+0000 7fd63a276640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd6341119b0 con 0x7fd634101710 2024-04-25T00:25:46.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.516+0000 7fd63a276640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd634111cb0 con 0x7fd6341020d0 2024-04-25T00:25:46.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.516+0000 7fd638813640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd634107720 0x7fd63419c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.516+0000 7fd638813640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd634107720 0x7fd63419c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56004/0 (socket says 172.21.15.107:56004) 2024-04-25T00:25:46.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.516+0000 7fd638813640 1 -- 172.21.15.107:0/1615671561 learned_addr learned my addr 172.21.15.107:0/1615671561 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:46.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.517+0000 7fd6337fe640 1 --2- 172.21.15.107:0/1615671561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6341020d0 0x7fd6341985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.517+0000 7fd633fff640 1 --2- 172.21.15.107:0/1615671561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd634101710 0x7fd634198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.517+0000 7fd638813640 1 -- 172.21.15.107:0/1615671561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6341020d0 msgr2=0x7fd6341985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:46.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.517+0000 7fd638813640 1 --2- 172.21.15.107:0/1615671561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6341020d0 0x7fd6341985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.517+0000 7fd638813640 1 -- 172.21.15.107:0/1615671561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd634101710 msgr2=0x7fd634198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:46.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.517+0000 7fd638813640 1 --2- 172.21.15.107:0/1615671561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd634101710 0x7fd634198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.517+0000 7fd638813640 1 -- 172.21.15.107:0/1615671561 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63419d060 con 0x7fd634107720 2024-04-25T00:25:46.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.517+0000 7fd638813640 1 --2- 172.21.15.107:0/1615671561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd634107720 0x7fd63419c930 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00cad0 tx=0x7fd62c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:46.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.517+0000 7fd6317fa640 1 -- 172.21.15.107:0/1615671561 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c00ae20 con 0x7fd634107720 2024-04-25T00:25:46.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.517+0000 7fd63a276640 1 -- 172.21.15.107:0/1615671561 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63419d350 con 0x7fd634107720 2024-04-25T00:25:46.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.517+0000 7fd6317fa640 1 -- 172.21.15.107:0/1615671561 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd62c005020 con 0x7fd634107720 2024-04-25T00:25:46.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.517+0000 7fd6317fa640 1 -- 172.21.15.107:0/1615671561 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c00f6b0 con 0x7fd634107720 2024-04-25T00:25:46.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.517+0000 7fd63a276640 1 -- 172.21.15.107:0/1615671561 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6341a4be0 con 0x7fd634107720 2024-04-25T00:25:46.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.520+0000 7fd6317fa640 1 -- 172.21.15.107:0/1615671561 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fd62c0051d0 con 0x7fd634107720 2024-04-25T00:25:46.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.520+0000 7fd63cf89740 1 Processor -- start 2024-04-25T00:25:46.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.520+0000 7fd63cf89740 1 -- start start 2024-04-25T00:25:46.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.520+0000 7fd6317fa640 1 --2- 172.21.15.107:0/1615671561 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd61c0a2d00 0x7fd61c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.520+0000 7fd63cf89740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56283d2c0980 0x56283d2becc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.520+0000 7fd63cf89740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56283d2bf290 0x56283d2bf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.520+0000 7fd63cf89740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2ca010 0x56283d2cc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.520+0000 7fd63cf89740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56283d2232c0 con 0x56283d2ca010 2024-04-25T00:25:46.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.520+0000 7fd63cf89740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56283d1cb910 con 0x56283d2c0980 2024-04-25T00:25:46.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.520+0000 7fd63cf89740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56283d21f280 con 0x56283d2bf290 2024-04-25T00:25:46.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.520+0000 7fd6337fe640 1 --2- 172.21.15.107:0/1615671561 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd61c0a2d00 0x7fd61c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.520+0000 7fd6317fa640 1 -- 172.21.15.107:0/1615671561 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7fd62c0cfa40 con 0x7fd634107720 2024-04-25T00:25:46.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.520+0000 7fd638813640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2ca010 0x56283d2cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.520+0000 7fd638813640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2ca010 0x56283d2cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56014/0 (socket says 172.21.15.107:56014) 2024-04-25T00:25:46.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.520+0000 7fd638813640 1 -- 172.21.15.107:0/2571794587 learned_addr learned my addr 172.21.15.107:0/2571794587 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:46.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.520+0000 7fd633fff640 1 --2- 172.21.15.107:0/2571794587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56283d2c0980 0x56283d2becc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.520+0000 7fd6337fe640 1 --2- 172.21.15.107:0/2571794587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56283d2bf290 0x56283d2bf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.521+0000 7fd638813640 1 -- 172.21.15.107:0/2571794587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56283d2bf290 msgr2=0x56283d2bf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:46.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.521+0000 7fd638813640 1 --2- 172.21.15.107:0/2571794587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56283d2bf290 0x56283d2bf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.521+0000 7fd6337fe640 1 --2- 172.21.15.107:0/1615671561 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd61c0a2d00 0x7fd61c0a51c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd634101060 tx=0x7fd628013880 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:46.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.521+0000 7fd638813640 1 -- 172.21.15.107:0/2571794587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56283d2c0980 msgr2=0x56283d2becc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:46.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.521+0000 7fd638813640 1 --2- 172.21.15.107:0/2571794587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56283d2c0980 0x56283d2becc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.521+0000 7fd638813640 1 -- 172.21.15.107:0/2571794587 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56283d1dcfb0 con 0x56283d2ca010 2024-04-25T00:25:46.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.521+0000 7fd633fff640 1 --2- 172.21.15.107:0/2571794587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56283d2c0980 0x56283d2becc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:25:46.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.521+0000 7fd638813640 1 --2- 172.21.15.107:0/2571794587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2ca010 0x56283d2cc400 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fd62c0980b0 tx=0x7fd62c09bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2050e30abb92f3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:46.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.521+0000 7fd61a7fc640 1 -- 172.21.15.107:0/2571794587 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c01d070 con 0x56283d2ca010 2024-04-25T00:25:46.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.521+0000 7fd61a7fc640 1 -- 172.21.15.107:0/2571794587 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd62c005560 con 0x56283d2ca010 2024-04-25T00:25:46.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.521+0000 7fd61a7fc640 1 -- 172.21.15.107:0/2571794587 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c0a4380 con 0x56283d2ca010 2024-04-25T00:25:46.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.522+0000 7fd63cf89740 1 -- 172.21.15.107:0/2571794587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2ca010 msgr2=0x56283d2cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:46.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.522+0000 7fd63cf89740 1 --2- 172.21.15.107:0/2571794587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2ca010 0x56283d2cc400 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fd62c0980b0 tx=0x7fd62c09bd60 comp rx=0 tx=0).stop 2024-04-25T00:25:46.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.522+0000 7fd63cf89740 1 -- 172.21.15.107:0/2571794587 shutdown_connections 2024-04-25T00:25:46.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.522+0000 7fd63cf89740 1 --2- 172.21.15.107:0/2571794587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2ca010 0x56283d2cc400 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.522+0000 7fd63cf89740 1 --2- 172.21.15.107:0/2571794587 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56283d2bf290 0x56283d2bf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.522+0000 7fd63cf89740 1 --2- 172.21.15.107:0/2571794587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56283d2c0980 0x56283d2becc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.522+0000 7fd63cf89740 1 -- 172.21.15.107:0/2571794587 >> 172.21.15.107:0/2571794587 conn(0x56283d287950 msgr2=0x56283d2bb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:46.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.522+0000 7fd63cf89740 1 -- 172.21.15.107:0/2571794587 shutdown_connections 2024-04-25T00:25:46.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.522+0000 7fd63cf89740 1 -- 172.21.15.107:0/2571794587 wait complete. 2024-04-25T00:25:46.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.522+0000 7fd63cf89740 1 -- start start 2024-04-25T00:25:46.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.522+0000 7fd63cf89740 1 Processor -- start 2024-04-25T00:25:46.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.523+0000 7fd63cf89740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56283d2bf290 0x56283d2c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.523+0000 7fd63cf89740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2c0980 0x56283d2c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.523+0000 7fd63cf89740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56283d2ca010 0x56283d2c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.523+0000 7fd63cf89740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56283d2ccea0 con 0x56283d2c0980 2024-04-25T00:25:46.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.523+0000 7fd63cf89740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56283d1dee90 con 0x56283d2ca010 2024-04-25T00:25:46.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.523+0000 7fd63cf89740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x56283d2cd020 con 0x56283d2bf290 2024-04-25T00:25:46.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.523+0000 7fd63cf89740 10 client.0 _ll_register_callbacks cb 0x56283d21a970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:46.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.523+0000 7fd6337fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2c0980 0x56283d2c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.523+0000 7fd633fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56283d2bf290 0x56283d2c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.523+0000 7fd6337fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2c0980 0x56283d2c74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56016/0 (socket says 172.21.15.107:56016) 2024-04-25T00:25:46.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.523+0000 7fd6337fe640 1 -- 172.21.15.107:0/2090627285 learned_addr learned my addr 172.21.15.107:0/2090627285 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:46.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.523+0000 7fd638813640 1 --2- 172.21.15.107:0/2090627285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56283d2ca010 0x56283d2c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.523+0000 7fd6337fe640 1 -- 172.21.15.107:0/2090627285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56283d2bf290 msgr2=0x56283d2c6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:46.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.523+0000 7fd6337fe640 1 --2- 172.21.15.107:0/2090627285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56283d2bf290 0x56283d2c6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.523+0000 7fd6337fe640 1 -- 172.21.15.107:0/2090627285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56283d2ca010 msgr2=0x56283d2c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:46.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.523+0000 7fd6337fe640 1 --2- 172.21.15.107:0/2090627285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56283d2ca010 0x56283d2c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.523+0000 7fd6337fe640 1 -- 172.21.15.107:0/2090627285 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56283cfff7e0 con 0x56283d2c0980 2024-04-25T00:25:46.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.523+0000 7fd6337fe640 1 --2- 172.21.15.107:0/2090627285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2c0980 0x56283d2c74e0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fd628011d00 tx=0x7fd628019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c0abe971aa2fcb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:46.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.523+0000 7fd6137fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:25:46.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.523+0000 7fd6137fe640 1 -- 172.21.15.107:0/2090627285 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628005220 con 0x56283d2c0980 2024-04-25T00:25:46.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.523+0000 7fd6137fe640 1 -- 172.21.15.107:0/2090627285 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7fd6280053c0 con 0x56283d2c0980 2024-04-25T00:25:46.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.523+0000 7fd63cf89740 1 -- 172.21.15.107:0/2090627285 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd62c005560 con 0x56283d2c0980 2024-04-25T00:25:46.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.524+0000 7fd6137fe640 1 -- 172.21.15.107:0/2090627285 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd628032050 con 0x56283d2c0980 2024-04-25T00:25:46.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.524+0000 7fd63cf89740 10 client.17706 fetch_fsmap learned FSMap version 34 2024-04-25T00:25:46.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.524+0000 7fd63cf89740 1 -- 172.21.15.107:0/2090627285 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56283d05ff70 con 0x56283d2c0980 2024-04-25T00:25:46.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.524+0000 7fd6137fe640 1 -- 172.21.15.107:0/2090627285 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd62c005560 con 0x56283d2c0980 2024-04-25T00:25:46.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.524+0000 7fd6137fe640 1 -- 172.21.15.107:0/2090627285 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fd628026030 con 0x56283d2c0980 2024-04-25T00:25:46.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.524+0000 7fd63cf89740 10 client.17706 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:25:46.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.524+0000 7fd63cf89740 10 client.17706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:46.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.524+0000 7fd63cf89740 20 client.17706 populate_metadata read hostname 'smithi107' 2024-04-25T00:25:46.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.524+0000 7fd63cf89740 1 --2- 172.21.15.107:0/2090627285 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56283d2da1f0 0x56283d2dc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.524+0000 7fd63cf89740 4 client.17706 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:46.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.524+0000 7fd63cf89740 1 -- 172.21.15.107:0/2090627285 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56283d1f75a0 con 0x56283d2da1f0 2024-04-25T00:25:46.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.524+0000 7fd633fff640 1 --2- 172.21.15.107:0/2090627285 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56283d2da1f0 0x56283d2dc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.525+0000 7fd6137fe640 10 client.17706 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:46.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.525+0000 7fd633fff640 1 --2- 172.21.15.107:0/2090627285 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56283d2da1f0 0x56283d2dc600 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aff4154cc376cdb server_cookie=3899a902fde82139 in_seq=0 out_seq=0 2024-04-25T00:25:46.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.525+0000 7fd6137fe640 1 -- 172.21.15.107:0/2090627285 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56283d1f75a0 con 0x56283d2da1f0 2024-04-25T00:25:46.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.525+0000 7fd6137fe640 10 client.17706 handle_command_reply: tid=0 2024-04-25T00:25:46.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.525+0000 7fd63cf89740 1 client.17706 shutdown 2024-04-25T00:25:46.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.526+0000 7fd63cf89740 1 -- 172.21.15.107:0/2090627285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2c0980 msgr2=0x56283d2c74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:46.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.526+0000 7fd63cf89740 1 --2- 172.21.15.107:0/2090627285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2c0980 0x56283d2c74e0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fd628011d00 tx=0x7fd628019000 comp rx=0 tx=0).stop 2024-04-25T00:25:46.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.526+0000 7fd63cf89740 1 -- 172.21.15.107:0/2090627285 shutdown_connections 2024-04-25T00:25:46.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.526+0000 7fd63cf89740 1 --2- 172.21.15.107:0/2090627285 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56283d2da1f0 0x56283d2dc600 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.526+0000 7fd63cf89740 1 --2- 172.21.15.107:0/2090627285 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56283d2ca010 0x56283d2c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.526+0000 7fd63cf89740 1 --2- 172.21.15.107:0/2090627285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2c0980 0x56283d2c74e0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.526+0000 7fd63cf89740 1 --2- 172.21.15.107:0/2090627285 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56283d2bf290 0x56283d2c6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.526+0000 7fd6137fe640 0 client.17706 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:46.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.526+0000 7fd63cf89740 1 -- 172.21.15.107:0/2090627285 >> 172.21.15.107:0/2090627285 conn(0x56283d287950 msgr2=0x56283d2c1660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:46.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.526+0000 7fd63cf89740 1 -- 172.21.15.107:0/2090627285 shutdown_connections 2024-04-25T00:25:46.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.526+0000 7fd63cf89740 1 -- 172.21.15.107:0/2090627285 wait complete. 2024-04-25T00:25:46.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.526+0000 7fd63cf89740 20 client.17706 trim_cache size 0 max 16384 2024-04-25T00:25:46.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.543+0000 7fd63cf89740 1 Processor -- start 2024-04-25T00:25:46.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.543+0000 7fd63cf89740 1 -- start start 2024-04-25T00:25:46.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.543+0000 7fd63cf89740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56283d2bf290 0x56283d2d5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.543+0000 7fd63cf89740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2c0980 0x56283d2ba7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.543+0000 7fd63cf89740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56283d2ca010 0x56283d2e1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.543+0000 7fd63cf89740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd6341b1550 con 0x56283d2c0980 2024-04-25T00:25:46.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.543+0000 7fd63cf89740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd6341aee20 con 0x56283d2bf290 2024-04-25T00:25:46.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.543+0000 7fd63cf89740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd6341acc90 con 0x56283d2ca010 2024-04-25T00:25:46.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.543+0000 7fd6337fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2c0980 0x56283d2ba7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.543+0000 7fd6337fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2c0980 0x56283d2ba7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56018/0 (socket says 172.21.15.107:56018) 2024-04-25T00:25:46.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.543+0000 7fd6337fe640 1 -- 172.21.15.107:0/4258278193 learned_addr learned my addr 172.21.15.107:0/4258278193 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:46.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.543+0000 7fd6337fe640 1 -- 172.21.15.107:0/4258278193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56283d2ca010 msgr2=0x56283d2e1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:46.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.543+0000 7fd6337fe640 1 --2- 172.21.15.107:0/4258278193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56283d2ca010 0x56283d2e1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.543+0000 7fd6337fe640 1 -- 172.21.15.107:0/4258278193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56283d2bf290 msgr2=0x56283d2d5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:46.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.543+0000 7fd6337fe640 1 --2- 172.21.15.107:0/4258278193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56283d2bf290 0x56283d2d5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.543+0000 7fd6337fe640 1 -- 172.21.15.107:0/4258278193 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56283cef7170 con 0x56283d2c0980 2024-04-25T00:25:46.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.543+0000 7fd6337fe640 1 --2- 172.21.15.107:0/4258278193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2c0980 0x56283d2ba7d0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fd62800b010 tx=0x7fd62800a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6630dd208cee51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:46.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.544+0000 7fd619ffb640 1 -- 172.21.15.107:0/4258278193 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62800ac70 con 0x56283d2c0980 2024-04-25T00:25:46.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.544+0000 7fd619ffb640 1 -- 172.21.15.107:0/4258278193 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd62803d070 con 0x56283d2c0980 2024-04-25T00:25:46.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.544+0000 7fd619ffb640 1 -- 172.21.15.107:0/4258278193 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd628004950 con 0x56283d2c0980 2024-04-25T00:25:46.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.544+0000 7fd63cf89740 1 -- 172.21.15.107:0/4258278193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2c0980 msgr2=0x56283d2ba7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:46.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.544+0000 7fd63cf89740 1 --2- 172.21.15.107:0/4258278193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2c0980 0x56283d2ba7d0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fd62800b010 tx=0x7fd62800a2d0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.544+0000 7fd63cf89740 1 -- 172.21.15.107:0/4258278193 shutdown_connections 2024-04-25T00:25:46.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.544+0000 7fd63cf89740 1 --2- 172.21.15.107:0/4258278193 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56283d2ca010 0x56283d2e1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.544+0000 7fd63cf89740 1 --2- 172.21.15.107:0/4258278193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2c0980 0x56283d2ba7d0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.544+0000 7fd63cf89740 1 --2- 172.21.15.107:0/4258278193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56283d2bf290 0x56283d2d5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.544+0000 7fd63cf89740 1 -- 172.21.15.107:0/4258278193 >> 172.21.15.107:0/4258278193 conn(0x56283d287950 msgr2=0x56283d2ba0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:46.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.544+0000 7fd63cf89740 1 -- 172.21.15.107:0/4258278193 shutdown_connections 2024-04-25T00:25:46.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.544+0000 7fd63cf89740 1 -- 172.21.15.107:0/4258278193 wait complete. 2024-04-25T00:25:46.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.544+0000 7fd63cf89740 1 -- start start 2024-04-25T00:25:46.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.545+0000 7fd63cf89740 1 Processor -- start 2024-04-25T00:25:46.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.545+0000 7fd63cf89740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56283d2bf290 0x56283d2ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.545+0000 7fd63cf89740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56283d2c0980 0x56283d2cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.545+0000 7fd63cf89740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2ca010 0x56283d339710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.545+0000 7fd63cf89740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd6341ad2e0 con 0x56283d2ca010 2024-04-25T00:25:46.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.545+0000 7fd63cf89740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd6341ae750 con 0x56283d2bf290 2024-04-25T00:25:46.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.545+0000 7fd63cf89740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd6341ad9b0 con 0x56283d2c0980 2024-04-25T00:25:46.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.545+0000 7fd63cf89740 10 client.0 _ll_register_callbacks cb 0x7fd62800a690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:46.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.545+0000 7fd638813640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2ca010 0x56283d339710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.545+0000 7fd638813640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2ca010 0x56283d339710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56024/0 (socket says 172.21.15.107:56024) 2024-04-25T00:25:46.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.545+0000 7fd638813640 1 -- 172.21.15.107:0/2483896268 learned_addr learned my addr 172.21.15.107:0/2483896268 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:46.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.545+0000 7fd633fff640 1 --2- 172.21.15.107:0/2483896268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56283d2bf290 0x56283d2ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.545+0000 7fd638813640 1 -- 172.21.15.107:0/2483896268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56283d2c0980 msgr2=0x56283d2cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:46.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.545+0000 7fd638813640 1 --2- 172.21.15.107:0/2483896268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56283d2c0980 0x56283d2cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.545+0000 7fd638813640 1 -- 172.21.15.107:0/2483896268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56283d2bf290 msgr2=0x56283d2ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:46.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.545+0000 7fd638813640 1 --2- 172.21.15.107:0/2483896268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56283d2bf290 0x56283d2ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.545+0000 7fd638813640 1 -- 172.21.15.107:0/2483896268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56283d339d00 con 0x56283d2ca010 2024-04-25T00:25:46.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.545+0000 7fd638813640 1 --2- 172.21.15.107:0/2483896268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2ca010 0x56283d339710 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fd62c0a0110 tx=0x7fd62c0075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a93ddc7e93b932db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:46.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.545+0000 7fd6137fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:25:46.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.546+0000 7fd6137fe640 1 -- 172.21.15.107:0/2483896268 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd62c0b9060 con 0x56283d2ca010 2024-04-25T00:25:46.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.546+0000 7fd6137fe640 1 -- 172.21.15.107:0/2483896268 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7fd62c09c0b0 con 0x56283d2ca010 2024-04-25T00:25:46.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.546+0000 7fd63cf89740 1 -- 172.21.15.107:0/2483896268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd62803d070 con 0x56283d2ca010 2024-04-25T00:25:46.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.546+0000 7fd6137fe640 1 -- 172.21.15.107:0/2483896268 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd62c01d040 con 0x56283d2ca010 2024-04-25T00:25:46.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.546+0000 7fd63cf89740 10 client.17712 fetch_fsmap learned FSMap version 34 2024-04-25T00:25:46.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.546+0000 7fd63cf89740 1 -- 172.21.15.107:0/2483896268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56283d3410c0 con 0x56283d2ca010 2024-04-25T00:25:46.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.546+0000 7fd6137fe640 1 -- 172.21.15.107:0/2483896268 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd62803d070 con 0x56283d2ca010 2024-04-25T00:25:46.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.546+0000 7fd6137fe640 1 -- 172.21.15.107:0/2483896268 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fd62c0979a0 con 0x56283d2ca010 2024-04-25T00:25:46.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.546+0000 7fd63cf89740 10 client.17712 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:25:46.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.546+0000 7fd63cf89740 10 client.17712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:46.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.546+0000 7fd63cf89740 20 client.17712 populate_metadata read hostname 'smithi107' 2024-04-25T00:25:46.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.546+0000 7fd63cf89740 1 --2- 172.21.15.107:0/2483896268 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56283d341490 0x56283d343880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.546+0000 7fd63cf89740 4 client.17712 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:25:46.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.546+0000 7fd63cf89740 1 -- 172.21.15.107:0/2483896268 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56283d230800 con 0x56283d341490 2024-04-25T00:25:46.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.547+0000 7fd633fff640 1 --2- 172.21.15.107:0/2483896268 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56283d341490 0x56283d343880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.547+0000 7fd633fff640 1 --2- 172.21.15.107:0/2483896268 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56283d341490 0x56283d343880 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae55b5d0c0d2ffc8 server_cookie=f46f6b3e2c1986c9 in_seq=0 out_seq=0 2024-04-25T00:25:46.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.547+0000 7fd6137fe640 10 client.17712 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:46.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.548+0000 7fd6137fe640 1 -- 172.21.15.107:0/2483896268 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56283d230800 con 0x56283d341490 2024-04-25T00:25:46.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.548+0000 7fd6137fe640 10 client.17712 handle_command_reply: tid=0 2024-04-25T00:25:46.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.548+0000 7fd63cf89740 1 client.17712 shutdown 2024-04-25T00:25:46.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.548+0000 7fd63cf89740 1 -- 172.21.15.107:0/2483896268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2ca010 msgr2=0x56283d339710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:46.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.548+0000 7fd63cf89740 1 --2- 172.21.15.107:0/2483896268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2ca010 0x56283d339710 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fd62c0a0110 tx=0x7fd62c0075f0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.548+0000 7fd63cf89740 1 -- 172.21.15.107:0/2483896268 shutdown_connections 2024-04-25T00:25:46.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.548+0000 7fd63cf89740 1 --2- 172.21.15.107:0/2483896268 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x56283d341490 0x56283d343880 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.548+0000 7fd63cf89740 1 --2- 172.21.15.107:0/2483896268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56283d2ca010 0x56283d339710 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.548+0000 7fd63cf89740 1 --2- 172.21.15.107:0/2483896268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x56283d2c0980 0x56283d2cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.548+0000 7fd6137fe640 0 client.17712 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:46.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.548+0000 7fd63cf89740 1 --2- 172.21.15.107:0/2483896268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56283d2bf290 0x56283d2ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.548+0000 7fd63cf89740 1 -- 172.21.15.107:0/2483896268 >> 172.21.15.107:0/2483896268 conn(0x56283d287950 msgr2=0x56283d2ba950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:46.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.548+0000 7fd63cf89740 1 -- 172.21.15.107:0/2483896268 shutdown_connections 2024-04-25T00:25:46.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.549+0000 7fd63cf89740 1 -- 172.21.15.107:0/2483896268 wait complete. 2024-04-25T00:25:46.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.549+0000 7fd63cf89740 20 client.17712 trim_cache size 0 max 16384 2024-04-25T00:25:46.556 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:25:46.556 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-04-25T00:25:46.557 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "eb7b8d40-6919-461b-bf96-e86571ceb5ba", 2024-04-25T00:25:46.557 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-04-25T00:25:46.557 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:25:46.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.549+0000 7fd63a276640 1 -- 172.21.15.107:0/1615671561 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd61c0a2d00 msgr2=0x7fd61c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:46.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.549+0000 7fd63a276640 1 --2- 172.21.15.107:0/1615671561 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd61c0a2d00 0x7fd61c0a51c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd634101060 tx=0x7fd628013880 comp rx=0 tx=0).stop 2024-04-25T00:25:46.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.549+0000 7fd63a276640 1 -- 172.21.15.107:0/1615671561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd634107720 msgr2=0x7fd63419c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:46.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.549+0000 7fd63a276640 1 --2- 172.21.15.107:0/1615671561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd634107720 0x7fd63419c930 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00cad0 tx=0x7fd62c00cf90 comp rx=0 tx=0).stop 2024-04-25T00:25:46.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.549+0000 7fd63a276640 1 -- 172.21.15.107:0/1615671561 shutdown_connections 2024-04-25T00:25:46.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.549+0000 7fd63a276640 1 --2- 172.21.15.107:0/1615671561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd634107720 0x7fd63419c930 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.549+0000 7fd63a276640 1 --2- 172.21.15.107:0/1615671561 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6341020d0 0x7fd6341985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.549+0000 7fd63a276640 1 --2- 172.21.15.107:0/1615671561 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd61c0a2d00 0x7fd61c0a51c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.549+0000 7fd63a276640 1 --2- 172.21.15.107:0/1615671561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd634101710 0x7fd634198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.549+0000 7fd63a276640 1 -- 172.21.15.107:0/1615671561 >> 172.21.15.107:0/1615671561 conn(0x7fd634078b90 msgr2=0x7fd6340ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:46.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.549+0000 7fd63a276640 1 -- 172.21.15.107:0/1615671561 shutdown_connections 2024-04-25T00:25:46.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.549+0000 7fd63a276640 1 -- 172.21.15.107:0/1615671561 wait complete. 2024-04-25T00:25:46.560 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:25:46.561 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:25:46.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.660+0000 7fbc841ed640 1 -- 172.21.15.107:0/4089330892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc7c1066b0 msgr2=0x7fbc7c106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:46.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.660+0000 7fbc841ed640 1 --2- 172.21.15.107:0/4089330892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc7c1066b0 0x7fbc7c106a90 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fbc70009930 tx=0x7fbc7002a420 comp rx=0 tx=0).stop 2024-04-25T00:25:46.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.661+0000 7fbc841ed640 1 -- 172.21.15.107:0/4089330892 shutdown_connections 2024-04-25T00:25:46.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.661+0000 7fbc841ed640 1 --2- 172.21.15.107:0/4089330892 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc7c100fd0 0x7fbc7c10d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.661+0000 7fbc841ed640 1 --2- 172.21.15.107:0/4089330892 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc7c100610 0x7fbc7c100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.661+0000 7fbc841ed640 1 --2- 172.21.15.107:0/4089330892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc7c1066b0 0x7fbc7c106a90 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.661+0000 7fbc841ed640 1 -- 172.21.15.107:0/4089330892 >> 172.21.15.107:0/4089330892 conn(0x7fbc7c077740 msgr2=0x7fbc7c077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:46.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.661+0000 7fbc841ed640 1 -- 172.21.15.107:0/4089330892 shutdown_connections 2024-04-25T00:25:46.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.661+0000 7fbc841ed640 1 -- 172.21.15.107:0/4089330892 wait complete. 2024-04-25T00:25:46.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.661+0000 7fbc841ed640 1 Processor -- start 2024-04-25T00:25:46.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.661+0000 7fbc841ed640 1 -- start start 2024-04-25T00:25:46.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.661+0000 7fbc841ed640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc7c100610 0x7fbc7c198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.661+0000 7fbc841ed640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc7c100fd0 0x7fbc7c199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.661+0000 7fbc841ed640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc7c1066b0 0x7fbc7c1a3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.661+0000 7fbc841ed640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c110a30 con 0x7fbc7c1066b0 2024-04-25T00:25:46.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.661+0000 7fbc841ed640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c1108b0 con 0x7fbc7c100fd0 2024-04-25T00:25:46.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.661+0000 7fbc841ed640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c110bb0 con 0x7fbc7c100610 2024-04-25T00:25:46.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.662+0000 7fbc8278a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc7c1066b0 0x7fbc7c1a3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.662+0000 7fbc8278a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc7c1066b0 0x7fbc7c1a3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56042/0 (socket says 172.21.15.107:56042) 2024-04-25T00:25:46.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.662+0000 7fbc8278a640 1 -- 172.21.15.107:0/4080442449 learned_addr learned my addr 172.21.15.107:0/4080442449 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:46.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.662+0000 7fbc81788640 1 --2- 172.21.15.107:0/4080442449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc7c100fd0 0x7fbc7c199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.662+0000 7fbc81f89640 1 --2- 172.21.15.107:0/4080442449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc7c100610 0x7fbc7c198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.662+0000 7fbc8278a640 1 -- 172.21.15.107:0/4080442449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc7c100610 msgr2=0x7fbc7c198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:46.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.662+0000 7fbc8278a640 1 --2- 172.21.15.107:0/4080442449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc7c100610 0x7fbc7c198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.662+0000 7fbc8278a640 1 -- 172.21.15.107:0/4080442449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc7c100fd0 msgr2=0x7fbc7c199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:46.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.662+0000 7fbc8278a640 1 --2- 172.21.15.107:0/4080442449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc7c100fd0 0x7fbc7c199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.662+0000 7fbc8278a640 1 -- 172.21.15.107:0/4080442449 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7c1a4490 con 0x7fbc7c1066b0 2024-04-25T00:25:46.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.662+0000 7fbc8278a640 1 --2- 172.21.15.107:0/4080442449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc7c1066b0 0x7fbc7c1a3f50 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400eaa0 tx=0x7fbc7400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:46.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.663+0000 7fbc6effd640 1 -- 172.21.15.107:0/4080442449 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7400cdb0 con 0x7fbc7c1066b0 2024-04-25T00:25:46.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.663+0000 7fbc6effd640 1 -- 172.21.15.107:0/4080442449 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc74005020 con 0x7fbc7c1066b0 2024-04-25T00:25:46.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.663+0000 7fbc6effd640 1 -- 172.21.15.107:0/4080442449 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc740116b0 con 0x7fbc7c1066b0 2024-04-25T00:25:46.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.663+0000 7fbc841ed640 1 -- 172.21.15.107:0/4080442449 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc7c1a4660 con 0x7fbc7c1066b0 2024-04-25T00:25:46.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.663+0000 7fbc841ed640 1 -- 172.21.15.107:0/4080442449 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc7c1a4b80 con 0x7fbc7c1066b0 2024-04-25T00:25:46.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.665+0000 7fbc6effd640 1 -- 172.21.15.107:0/4080442449 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fbc74005560 con 0x7fbc7c1066b0 2024-04-25T00:25:46.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.665+0000 7fbc86fd0740 1 Processor -- start 2024-04-25T00:25:46.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.665+0000 7fbc86fd0740 1 -- start start 2024-04-25T00:25:46.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.665+0000 7fbc86fd0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b105161630 0x55b105161a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.665+0000 7fbc86fd0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b105161f50 0x55b10516c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.665+0000 7fbc86fd0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b10516c620 0x55b10516ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.665+0000 7fbc86fd0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b1050c66b0 con 0x55b10516c620 2024-04-25T00:25:46.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.665+0000 7fbc86fd0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b10506e880 con 0x55b105161f50 2024-04-25T00:25:46.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.665+0000 7fbc86fd0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b1050c2670 con 0x55b105161630 2024-04-25T00:25:46.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.666+0000 7fbc6effd640 1 --2- 172.21.15.107:0/4080442449 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbc680a2d80 0x7fbc680a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.666+0000 7fbc81788640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b10516c620 0x55b10516ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.666+0000 7fbc8278a640 1 --2- 172.21.15.107:0/4080442449 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbc680a2d80 0x7fbc680a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.666+0000 7fbc81788640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b10516c620 0x55b10516ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56044/0 (socket says 172.21.15.107:56044) 2024-04-25T00:25:46.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.666+0000 7fbc81788640 1 -- 172.21.15.107:0/378717490 learned_addr learned my addr 172.21.15.107:0/378717490 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:46.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.666+0000 7fbc81f89640 1 --2- 172.21.15.107:0/378717490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b105161f50 0x55b10516c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.666+0000 7fbc81788640 1 --2- 172.21.15.107:0/378717490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b105161630 0x55b105161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.666+0000 7fbc6effd640 1 -- 172.21.15.107:0/4080442449 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7fbc740cf9a0 con 0x7fbc7c1066b0 2024-04-25T00:25:46.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.666+0000 7fbc81788640 1 -- 172.21.15.107:0/378717490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b105161630 msgr2=0x55b105161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:46.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.666+0000 7fbc81788640 1 --2- 172.21.15.107:0/378717490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b105161630 0x55b105161a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.666+0000 7fbc81788640 1 -- 172.21.15.107:0/378717490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b105161f50 msgr2=0x55b10516c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:46.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.666+0000 7fbc81788640 1 --2- 172.21.15.107:0/378717490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b105161f50 0x55b10516c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.666+0000 7fbc81788640 1 -- 172.21.15.107:0/378717490 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b104ea47d0 con 0x55b10516c620 2024-04-25T00:25:46.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.666+0000 7fbc8278a640 1 --2- 172.21.15.107:0/4080442449 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbc680a2d80 0x7fbc680a5240 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbc740984d0 tx=0x7fbc740054d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:46.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.666+0000 7fbc81788640 1 --2- 172.21.15.107:0/378717490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b10516c620 0x55b10516ea10 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fbc780127a0 tx=0x7fbc7800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb964dfea35fdcea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:46.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.666+0000 7fbc67fff640 1 -- 172.21.15.107:0/378717490 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc78012d20 con 0x55b10516c620 2024-04-25T00:25:46.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.666+0000 7fbc67fff640 1 -- 172.21.15.107:0/378717490 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc78005020 con 0x55b10516c620 2024-04-25T00:25:46.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.666+0000 7fbc67fff640 1 -- 172.21.15.107:0/378717490 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc78004ac0 con 0x55b10516c620 2024-04-25T00:25:46.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.667+0000 7fbc86fd0740 1 -- 172.21.15.107:0/378717490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b10516c620 msgr2=0x55b10516ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:46.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.667+0000 7fbc86fd0740 1 --2- 172.21.15.107:0/378717490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b10516c620 0x55b10516ea10 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fbc780127a0 tx=0x7fbc7800b7c0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.667+0000 7fbc86fd0740 1 -- 172.21.15.107:0/378717490 shutdown_connections 2024-04-25T00:25:46.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.667+0000 7fbc86fd0740 1 --2- 172.21.15.107:0/378717490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b10516c620 0x55b10516ea10 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.667+0000 7fbc86fd0740 1 --2- 172.21.15.107:0/378717490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b105161f50 0x55b10516c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.667+0000 7fbc86fd0740 1 --2- 172.21.15.107:0/378717490 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b105161630 0x55b105161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.667+0000 7fbc86fd0740 1 -- 172.21.15.107:0/378717490 >> 172.21.15.107:0/378717490 conn(0x55b10515d950 msgr2=0x55b10515dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:46.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.667+0000 7fbc86fd0740 1 -- 172.21.15.107:0/378717490 shutdown_connections 2024-04-25T00:25:46.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.667+0000 7fbc86fd0740 1 -- 172.21.15.107:0/378717490 wait complete. 2024-04-25T00:25:46.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.668+0000 7fbc86fd0740 1 -- start start 2024-04-25T00:25:46.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.668+0000 7fbc86fd0740 1 Processor -- start 2024-04-25T00:25:46.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.668+0000 7fbc86fd0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b105161630 0x55b1051695c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.668+0000 7fbc86fd0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b105161f50 0x55b105169b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.668+0000 7fbc86fd0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b10516c620 0x55b10516a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.668+0000 7fbc86fd0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b10516f410 con 0x55b105161630 2024-04-25T00:25:46.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.668+0000 7fbc86fd0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b105081f50 con 0x55b10516c620 2024-04-25T00:25:46.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.668+0000 7fbc86fd0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b10516f590 con 0x55b105161f50 2024-04-25T00:25:46.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.668+0000 7fbc86fd0740 10 client.0 _ll_register_callbacks cb 0x55b1050bd000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:46.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.668+0000 7fbc81788640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b105161630 0x55b1051695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.668+0000 7fbc81788640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b105161630 0x55b1051695c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56052/0 (socket says 172.21.15.107:56052) 2024-04-25T00:25:46.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.668+0000 7fbc81788640 1 -- 172.21.15.107:0/1686909198 learned_addr learned my addr 172.21.15.107:0/1686909198 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:46.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.668+0000 7fbc81788640 1 --2- 172.21.15.107:0/1686909198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b10516c620 0x55b10516a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.668+0000 7fbc81f89640 1 --2- 172.21.15.107:0/1686909198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b105161f50 0x55b105169b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.668+0000 7fbc81788640 1 -- 172.21.15.107:0/1686909198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b105161f50 msgr2=0x55b105169b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:46.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.668+0000 7fbc81788640 1 --2- 172.21.15.107:0/1686909198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b105161f50 0x55b105169b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.668+0000 7fbc81788640 1 -- 172.21.15.107:0/1686909198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b10516c620 msgr2=0x55b10516a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:46.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.668+0000 7fbc81788640 1 --2- 172.21.15.107:0/1686909198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b10516c620 0x55b10516a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.668+0000 7fbc81788640 1 -- 172.21.15.107:0/1686909198 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b104dda3f0 con 0x55b105161630 2024-04-25T00:25:46.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.669+0000 7fbc81788640 1 --2- 172.21.15.107:0/1686909198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b105161630 0x55b1051695c0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fbc780047b0 tx=0x7fbc78005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950e22a151d10b9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:46.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.669+0000 7fbc657fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:25:46.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.669+0000 7fbc657fa640 1 -- 172.21.15.107:0/1686909198 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc780048a0 con 0x55b105161630 2024-04-25T00:25:46.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.669+0000 7fbc657fa640 1 -- 172.21.15.107:0/1686909198 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7fbc78004a40 con 0x55b105161630 2024-04-25T00:25:46.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.669+0000 7fbc86fd0740 1 -- 172.21.15.107:0/1686909198 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc78005020 con 0x55b105161630 2024-04-25T00:25:46.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.669+0000 7fbc657fa640 1 -- 172.21.15.107:0/1686909198 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbc78004c40 con 0x55b105161630 2024-04-25T00:25:46.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.669+0000 7fbc86fd0740 10 client.17724 fetch_fsmap learned FSMap version 34 2024-04-25T00:25:46.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.669+0000 7fbc86fd0740 1 -- 172.21.15.107:0/1686909198 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b104d9a0f0 con 0x55b105161630 2024-04-25T00:25:46.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.669+0000 7fbc657fa640 1 -- 172.21.15.107:0/1686909198 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc78005020 con 0x55b105161630 2024-04-25T00:25:46.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.669+0000 7fbc657fa640 1 -- 172.21.15.107:0/1686909198 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fbc7802a020 con 0x55b105161630 2024-04-25T00:25:46.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.669+0000 7fbc86fd0740 10 client.17724 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:25:46.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.669+0000 7fbc86fd0740 10 client.17724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:46.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.669+0000 7fbc86fd0740 20 client.17724 populate_metadata read hostname 'smithi107' 2024-04-25T00:25:46.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.669+0000 7fbc86fd0740 1 --2- 172.21.15.107:0/1686909198 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b10517c790 0x55b10517eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.669+0000 7fbc86fd0740 4 client.17724 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:46.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.669+0000 7fbc86fd0740 1 -- 172.21.15.107:0/1686909198 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1050d3bf0 con 0x55b10517c790 2024-04-25T00:25:46.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.670+0000 7fbc81f89640 1 --2- 172.21.15.107:0/1686909198 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b10517c790 0x55b10517eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.670+0000 7fbc657fa640 10 client.17724 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:46.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.670+0000 7fbc81f89640 1 --2- 172.21.15.107:0/1686909198 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b10517c790 0x55b10517eb80 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fbcb9055f3d06f3 server_cookie=84761806b87098cb in_seq=0 out_seq=0 2024-04-25T00:25:46.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.671+0000 7fbc657fa640 1 -- 172.21.15.107:0/1686909198 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b1050d3bf0 con 0x55b10517c790 2024-04-25T00:25:46.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.671+0000 7fbc657fa640 10 client.17724 handle_command_reply: tid=0 2024-04-25T00:25:46.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.671+0000 7fbc86fd0740 1 client.17724 shutdown 2024-04-25T00:25:46.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.671+0000 7fbc86fd0740 1 -- 172.21.15.107:0/1686909198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b105161630 msgr2=0x55b1051695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:46.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.671+0000 7fbc86fd0740 1 --2- 172.21.15.107:0/1686909198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b105161630 0x55b1051695c0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fbc780047b0 tx=0x7fbc78005f70 comp rx=0 tx=0).stop 2024-04-25T00:25:46.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.671+0000 7fbc86fd0740 1 -- 172.21.15.107:0/1686909198 shutdown_connections 2024-04-25T00:25:46.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.671+0000 7fbc86fd0740 1 --2- 172.21.15.107:0/1686909198 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b10517c790 0x55b10517eb80 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.671+0000 7fbc86fd0740 1 --2- 172.21.15.107:0/1686909198 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b10516c620 0x55b10516a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.671+0000 7fbc657fa640 0 client.17724 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:46.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.671+0000 7fbc86fd0740 1 --2- 172.21.15.107:0/1686909198 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b105161f50 0x55b105169b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.671+0000 7fbc86fd0740 1 --2- 172.21.15.107:0/1686909198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b105161630 0x55b1051695c0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.671+0000 7fbc86fd0740 1 -- 172.21.15.107:0/1686909198 >> 172.21.15.107:0/1686909198 conn(0x55b10515d950 msgr2=0x55b105163750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:46.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.671+0000 7fbc86fd0740 1 -- 172.21.15.107:0/1686909198 shutdown_connections 2024-04-25T00:25:46.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.671+0000 7fbc86fd0740 1 -- 172.21.15.107:0/1686909198 wait complete. 2024-04-25T00:25:46.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.671+0000 7fbc86fd0740 20 client.17724 trim_cache size 0 max 16384 2024-04-25T00:25:46.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.688+0000 7fbc86fd0740 1 Processor -- start 2024-04-25T00:25:46.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.688+0000 7fbc86fd0740 1 -- start start 2024-04-25T00:25:46.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.688+0000 7fbc86fd0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b105161630 0x55b1051782d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.688+0000 7fbc86fd0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b105161f50 0x55b105169e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.688+0000 7fbc86fd0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b10516c620 0x55b105163ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.688+0000 7fbc86fd0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c1b15c0 con 0x55b105161f50 2024-04-25T00:25:46.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.688+0000 7fbc86fd0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c1aee90 con 0x55b105161630 2024-04-25T00:25:46.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.688+0000 7fbc86fd0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c1acd00 con 0x55b10516c620 2024-04-25T00:25:46.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.688+0000 7fbc81f89640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b105161f50 0x55b105169e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.688+0000 7fbc81f89640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b105161f50 0x55b105169e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56068/0 (socket says 172.21.15.107:56068) 2024-04-25T00:25:46.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.688+0000 7fbc81f89640 1 -- 172.21.15.107:0/1721721690 learned_addr learned my addr 172.21.15.107:0/1721721690 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:46.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.688+0000 7fbc81788640 1 --2- 172.21.15.107:0/1721721690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b105161630 0x55b1051782d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.688+0000 7fbc81f89640 1 -- 172.21.15.107:0/1721721690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b10516c620 msgr2=0x55b105163ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:46.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.688+0000 7fbc81f89640 1 --2- 172.21.15.107:0/1721721690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b10516c620 0x55b105163ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.688+0000 7fbc81f89640 1 -- 172.21.15.107:0/1721721690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b105161630 msgr2=0x55b1051782d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:46.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.688+0000 7fbc81f89640 1 --2- 172.21.15.107:0/1721721690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b105161630 0x55b1051782d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.688+0000 7fbc81f89640 1 -- 172.21.15.107:0/1721721690 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b10514f090 con 0x55b105161f50 2024-04-25T00:25:46.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.689+0000 7fbc81f89640 1 --2- 172.21.15.107:0/1721721690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b105161f50 0x55b105169e10 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fbc70009340 tx=0x7fbc700058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa8a287d1edf8cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:46.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.689+0000 7fbc677fe640 1 -- 172.21.15.107:0/1721721690 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70035040 con 0x55b105161f50 2024-04-25T00:25:46.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.689+0000 7fbc677fe640 1 -- 172.21.15.107:0/1721721690 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc70043b50 con 0x55b105161f50 2024-04-25T00:25:46.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.689+0000 7fbc677fe640 1 -- 172.21.15.107:0/1721721690 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70043530 con 0x55b105161f50 2024-04-25T00:25:46.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.689+0000 7fbc86fd0740 1 -- 172.21.15.107:0/1721721690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b105161f50 msgr2=0x55b105169e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:46.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.689+0000 7fbc86fd0740 1 --2- 172.21.15.107:0/1721721690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b105161f50 0x55b105169e10 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fbc70009340 tx=0x7fbc700058b0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.689+0000 7fbc86fd0740 1 -- 172.21.15.107:0/1721721690 shutdown_connections 2024-04-25T00:25:46.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.689+0000 7fbc86fd0740 1 --2- 172.21.15.107:0/1721721690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b10516c620 0x55b105163ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.689+0000 7fbc86fd0740 1 --2- 172.21.15.107:0/1721721690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b105161f50 0x55b105169e10 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.689+0000 7fbc86fd0740 1 --2- 172.21.15.107:0/1721721690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b105161630 0x55b1051782d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.689+0000 7fbc86fd0740 1 -- 172.21.15.107:0/1721721690 >> 172.21.15.107:0/1721721690 conn(0x55b10515d950 msgr2=0x55b10516ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:46.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.689+0000 7fbc86fd0740 1 -- 172.21.15.107:0/1721721690 shutdown_connections 2024-04-25T00:25:46.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.689+0000 7fbc86fd0740 1 -- 172.21.15.107:0/1721721690 wait complete. 2024-04-25T00:25:46.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.690+0000 7fbc86fd0740 1 -- start start 2024-04-25T00:25:46.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.690+0000 7fbc86fd0740 1 Processor -- start 2024-04-25T00:25:46.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.690+0000 7fbc86fd0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b105161630 0x55b105174cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.690+0000 7fbc86fd0740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b105161f50 0x55b105170020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.690+0000 7fbc86fd0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b10516c620 0x55b105170630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.690+0000 7fbc86fd0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c1ad350 con 0x55b105161630 2024-04-25T00:25:46.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.690+0000 7fbc86fd0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c1ae7c0 con 0x55b10516c620 2024-04-25T00:25:46.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.690+0000 7fbc86fd0740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c1ada20 con 0x55b105161f50 2024-04-25T00:25:46.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.690+0000 7fbc86fd0740 10 client.0 _ll_register_callbacks cb 0x7fbc70009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:46.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.690+0000 7fbc81788640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b105161630 0x55b105174cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.690+0000 7fbc81788640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b105161630 0x55b105174cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56076/0 (socket says 172.21.15.107:56076) 2024-04-25T00:25:46.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.690+0000 7fbc81788640 1 -- 172.21.15.107:0/3750738885 learned_addr learned my addr 172.21.15.107:0/3750738885 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:25:46.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.690+0000 7fbc81788640 1 -- 172.21.15.107:0/3750738885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b105161f50 msgr2=0x55b105170020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:46.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.690+0000 7fbc81788640 1 --2- 172.21.15.107:0/3750738885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b105161f50 0x55b105170020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.690+0000 7fbc81788640 1 -- 172.21.15.107:0/3750738885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b10516c620 msgr2=0x55b105170630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:46.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.690+0000 7fbc81788640 1 --2- 172.21.15.107:0/3750738885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b10516c620 0x55b105170630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.690+0000 7fbc81788640 1 -- 172.21.15.107:0/3750738885 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b105170e30 con 0x55b105161630 2024-04-25T00:25:46.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.691+0000 7fbc81788640 1 --2- 172.21.15.107:0/3750738885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b105161630 0x55b105174cb0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fbc7802cad0 tx=0x7fbc78005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790ca5e7768f2fd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:46.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.691+0000 7fbc657fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:25:46.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.691+0000 7fbc657fa640 1 -- 172.21.15.107:0/3750738885 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7802c440 con 0x55b105161630 2024-04-25T00:25:46.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.691+0000 7fbc657fa640 1 -- 172.21.15.107:0/3750738885 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9483+0+0 (secure 0 0 0) 0x7fbc7800ba90 con 0x55b105161630 2024-04-25T00:25:46.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.691+0000 7fbc86fd0740 1 -- 172.21.15.107:0/3750738885 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc70043b50 con 0x55b105161630 2024-04-25T00:25:46.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.691+0000 7fbc657fa640 1 -- 172.21.15.107:0/3750738885 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbc7803e050 con 0x55b105161630 2024-04-25T00:25:46.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.691+0000 7fbc86fd0740 10 client.17730 fetch_fsmap learned FSMap version 34 2024-04-25T00:25:46.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.691+0000 7fbc86fd0740 1 -- 172.21.15.107:0/3750738885 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b1051e2d30 con 0x55b105161630 2024-04-25T00:25:46.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.691+0000 7fbc657fa640 1 -- 172.21.15.107:0/3750738885 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc70043b50 con 0x55b105161630 2024-04-25T00:25:46.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.691+0000 7fbc657fa640 1 -- 172.21.15.107:0/3750738885 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fbc7802a040 con 0x55b105161630 2024-04-25T00:25:46.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.691+0000 7fbc86fd0740 10 client.17730 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:25:46.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.691+0000 7fbc86fd0740 10 client.17730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:46.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.691+0000 7fbc86fd0740 20 client.17730 populate_metadata read hostname 'smithi107' 2024-04-25T00:25:46.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.691+0000 7fbc86fd0740 1 --2- 172.21.15.107:0/3750738885 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b1051e33a0 0x55b1051e57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:46.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.691+0000 7fbc86fd0740 4 client.17730 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:25:46.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.691+0000 7fbc86fd0740 1 -- 172.21.15.107:0/3750738885 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b1050cdf40 con 0x55b1051e33a0 2024-04-25T00:25:46.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.692+0000 7fbc81f89640 1 --2- 172.21.15.107:0/3750738885 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b1051e33a0 0x55b1051e57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:46.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.692+0000 7fbc657fa640 10 client.17730 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:46.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.692+0000 7fbc81f89640 1 --2- 172.21.15.107:0/3750738885 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b1051e33a0 0x55b1051e57c0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e141ef77dae56a server_cookie=37a38af4c5624631 in_seq=0 out_seq=0 2024-04-25T00:25:46.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.692+0000 7fbc657fa640 1 -- 172.21.15.107:0/3750738885 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b1050cdf40 con 0x55b1051e33a0 2024-04-25T00:25:46.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.692+0000 7fbc657fa640 10 client.17730 handle_command_reply: tid=0 2024-04-25T00:25:46.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.692+0000 7fbc86fd0740 1 client.17730 shutdown 2024-04-25T00:25:46.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.692+0000 7fbc86fd0740 1 -- 172.21.15.107:0/3750738885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b105161630 msgr2=0x55b105174cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:46.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.692+0000 7fbc86fd0740 1 --2- 172.21.15.107:0/3750738885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b105161630 0x55b105174cb0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fbc7802cad0 tx=0x7fbc78005670 comp rx=0 tx=0).stop 2024-04-25T00:25:46.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.692+0000 7fbc86fd0740 1 -- 172.21.15.107:0/3750738885 shutdown_connections 2024-04-25T00:25:46.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.692+0000 7fbc86fd0740 1 --2- 172.21.15.107:0/3750738885 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55b1051e33a0 0x55b1051e57c0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.692+0000 7fbc86fd0740 1 --2- 172.21.15.107:0/3750738885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b10516c620 0x55b105170630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.692+0000 7fbc86fd0740 1 --2- 172.21.15.107:0/3750738885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b105161f50 0x55b105170020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.692+0000 7fbc657fa640 0 client.17730 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:25:46.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.692+0000 7fbc86fd0740 1 --2- 172.21.15.107:0/3750738885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b105161630 0x55b105174cb0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.692+0000 7fbc86fd0740 1 -- 172.21.15.107:0/3750738885 >> 172.21.15.107:0/3750738885 conn(0x55b10515d950 msgr2=0x55b105169c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:46.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.692+0000 7fbc86fd0740 1 -- 172.21.15.107:0/3750738885 shutdown_connections 2024-04-25T00:25:46.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.692+0000 7fbc86fd0740 1 -- 172.21.15.107:0/3750738885 wait complete. 2024-04-25T00:25:46.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.693+0000 7fbc86fd0740 20 client.17730 trim_cache size 0 max 16384 2024-04-25T00:25:46.700 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:25:46.700 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:25:46.700 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-04-25T00:25:46.700 INFO:teuthology.orchestra.run.smithi107.stdout: "eb7b8d40-6919-461b-bf96-e86571ceb5ba": { 2024-04-25T00:25:46.700 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-04-25T00:25:46.700 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "eb7b8d40-6919-461b-bf96-e86571ceb5ba", 2024-04-25T00:25:46.700 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-04-25T00:25:46.700 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:25:46.701 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:25:46.701 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:25:46.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.693+0000 7fbc841ed640 1 -- 172.21.15.107:0/4080442449 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbc680a2d80 msgr2=0x7fbc680a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:46.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.693+0000 7fbc841ed640 1 --2- 172.21.15.107:0/4080442449 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbc680a2d80 0x7fbc680a5240 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbc740984d0 tx=0x7fbc740054d0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.693+0000 7fbc841ed640 1 -- 172.21.15.107:0/4080442449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc7c1066b0 msgr2=0x7fbc7c1a3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:46.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.693+0000 7fbc841ed640 1 --2- 172.21.15.107:0/4080442449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc7c1066b0 0x7fbc7c1a3f50 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fbc7400eaa0 tx=0x7fbc7400ef60 comp rx=0 tx=0).stop 2024-04-25T00:25:46.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.693+0000 7fbc841ed640 1 -- 172.21.15.107:0/4080442449 shutdown_connections 2024-04-25T00:25:46.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.693+0000 7fbc841ed640 1 --2- 172.21.15.107:0/4080442449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc7c1066b0 0x7fbc7c1a3f50 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.693+0000 7fbc841ed640 1 --2- 172.21.15.107:0/4080442449 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbc680a2d80 0x7fbc680a5240 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.693+0000 7fbc841ed640 1 --2- 172.21.15.107:0/4080442449 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbc7c100fd0 0x7fbc7c199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.693+0000 7fbc841ed640 1 --2- 172.21.15.107:0/4080442449 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc7c100610 0x7fbc7c198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:46.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.693+0000 7fbc841ed640 1 -- 172.21.15.107:0/4080442449 >> 172.21.15.107:0/4080442449 conn(0x7fbc7c077740 msgr2=0x7fbc7c0ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:46.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.693+0000 7fbc841ed640 1 -- 172.21.15.107:0/4080442449 shutdown_connections 2024-04-25T00:25:46.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:25:46.693+0000 7fbc841ed640 1 -- 172.21.15.107:0/4080442449 wait complete. 2024-04-25T00:25:46.704 INFO:tasks.cephfs.filesystem:scrub status for tag:eb7b8d40-6919-461b-bf96-e86571ceb5ba - {'path': '/', 'tag': 'eb7b8d40-6919-461b-bf96-e86571ceb5ba', 'options': 'recursive,force'} 2024-04-25T00:25:48.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:47 smithi195 ceph-mon[32833]: pgmap v289: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:25:48.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:47 smithi195 ceph-mon[32833]: scrub queued for path: / 2024-04-25T00:25:48.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:47 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:25:48.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:47 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:25:48.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:47 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:25:48.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:47 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:25:48.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:47 smithi170 ceph-mon[32807]: pgmap v289: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:25:48.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:47 smithi170 ceph-mon[32807]: scrub queued for path: / 2024-04-25T00:25:48.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:47 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:25:48.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:47 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:25:48.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:47 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:25:48.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:47 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:25:48.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:47 smithi107 ceph-mon[29153]: pgmap v289: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:25:48.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:47 smithi107 ceph-mon[29153]: scrub queued for path: / 2024-04-25T00:25:48.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:47 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:25:48.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:47 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:25:48.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:47 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:25:48.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:47 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:25:50.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:49 smithi195 ceph-mon[32833]: pgmap v290: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:25:50.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:49 smithi170 ceph-mon[32807]: pgmap v290: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:25:50.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:49 smithi107 ceph-mon[29153]: pgmap v290: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:25:52.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:51 smithi195 ceph-mon[32833]: pgmap v291: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:25:52.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:51 smithi170 ceph-mon[32807]: pgmap v291: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:25:52.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:51 smithi107 ceph-mon[29153]: pgmap v291: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:25:54.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:53 smithi195 ceph-mon[32833]: pgmap v292: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:25:54.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:53 smithi170 ceph-mon[32807]: pgmap v292: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:25:54.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:53 smithi107 ceph-mon[29153]: pgmap v292: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:25:56.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:55 smithi195 ceph-mon[32833]: pgmap v293: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:25:56.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:55 smithi170 ceph-mon[32807]: pgmap v293: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:25:56.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:55 smithi107 ceph-mon[29153]: pgmap v293: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:25:58.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:57 smithi170 ceph-mon[32807]: pgmap v294: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:25:58.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:57 smithi107 ceph-mon[29153]: pgmap v294: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:25:58.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:57 smithi195 ceph-mon[32833]: pgmap v294: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:26:00.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:25:59 smithi170 ceph-mon[32807]: pgmap v295: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:26:00.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:25:59 smithi107 ceph-mon[29153]: pgmap v295: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:26:00.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:25:59 smithi195 ceph-mon[32833]: pgmap v295: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:26:02.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:01 smithi170 ceph-mon[32807]: pgmap v296: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:26:02.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:01 smithi107 ceph-mon[29153]: pgmap v296: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:26:02.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:01 smithi195 ceph-mon[32833]: pgmap v296: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:26:04.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:03 smithi170 ceph-mon[32807]: pgmap v297: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T00:26:04.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:03 smithi107 ceph-mon[29153]: pgmap v297: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T00:26:04.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:03 smithi195 ceph-mon[32833]: pgmap v297: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T00:26:06.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:05 smithi170 ceph-mon[32807]: pgmap v298: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:26:06.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:05 smithi107 ceph-mon[29153]: pgmap v298: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:26:06.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:05 smithi195 ceph-mon[32833]: pgmap v298: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:26:08.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:07 smithi170 ceph-mon[32807]: pgmap v299: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:26:08.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:07 smithi107 ceph-mon[29153]: pgmap v299: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:26:08.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:07 smithi195 ceph-mon[32833]: pgmap v299: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:26:09.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:08 smithi170 ceph-mon[32807]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T00:26:09.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:08 smithi107 ceph-mon[29153]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T00:26:09.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:08 smithi195 ceph-mon[32833]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T00:26:10.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:09 smithi170 ceph-mon[32807]: pgmap v301: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T00:26:10.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:09 smithi107 ceph-mon[29153]: pgmap v301: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T00:26:10.340 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:09 smithi195 ceph-mon[32833]: pgmap v301: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T00:26:12.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:11 smithi170 ceph-mon[32807]: pgmap v302: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T00:26:12.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:11 smithi107 ceph-mon[29153]: pgmap v302: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T00:26:12.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:11 smithi195 ceph-mon[32833]: pgmap v302: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T00:26:14.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:13 smithi170 ceph-mon[32807]: pgmap v303: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:26:14.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:13 smithi107 ceph-mon[29153]: pgmap v303: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:26:14.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:13 smithi195 ceph-mon[32833]: pgmap v303: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:26:16.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:15 smithi170 ceph-mon[32807]: pgmap v304: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:26:16.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:15 smithi107 ceph-mon[29153]: pgmap v304: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:26:16.339 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:15 smithi195 ceph-mon[32833]: pgmap v304: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:26:16.705 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:26:16.706 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:26:16.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.817+0000 7f0d51441640 1 -- 172.21.15.107:0/778701979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d4c101330 msgr2=0x7f0d4c10d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:16.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.817+0000 7f0d51441640 1 --2- 172.21.15.107:0/778701979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d4c101330 0x7f0d4c10d460 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f0d44009900 tx=0x7f0d4402a410 comp rx=0 tx=0).stop 2024-04-25T00:26:16.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.817+0000 7f0d51441640 1 -- 172.21.15.107:0/778701979 shutdown_connections 2024-04-25T00:26:16.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.817+0000 7f0d51441640 1 --2- 172.21.15.107:0/778701979 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d4c101330 0x7f0d4c10d460 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.817+0000 7f0d51441640 1 --2- 172.21.15.107:0/778701979 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d4c100990 0x7f0d4c100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.817+0000 7f0d51441640 1 --2- 172.21.15.107:0/778701979 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d4c106690 0x7f0d4c106a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.817+0000 7f0d51441640 1 -- 172.21.15.107:0/778701979 >> 172.21.15.107:0/778701979 conn(0x7f0d4c076440 msgr2=0x7f0d4c076850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:16.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.817+0000 7f0d51441640 1 -- 172.21.15.107:0/778701979 shutdown_connections 2024-04-25T00:26:16.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.817+0000 7f0d51441640 1 -- 172.21.15.107:0/778701979 wait complete. 2024-04-25T00:26:16.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.818+0000 7f0d51441640 1 Processor -- start 2024-04-25T00:26:16.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.818+0000 7f0d51441640 1 -- start start 2024-04-25T00:26:16.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.818+0000 7f0d51441640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d4c100990 0x7f0d4c1020c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:16.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.818+0000 7f0d51441640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d4c101330 0x7f0d4c102600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:16.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.818+0000 7f0d51441640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d4c106690 0x7f0d4c102b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:16.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.818+0000 7f0d51441640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c1109e0 con 0x7f0d4c106690 2024-04-25T00:26:16.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.818+0000 7f0d51441640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c110860 con 0x7f0d4c100990 2024-04-25T00:26:16.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.818+0000 7f0d51441640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c110b60 con 0x7f0d4c101330 2024-04-25T00:26:16.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.818+0000 7f0d4b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d4c106690 0x7f0d4c102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:16.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.818+0000 7f0d4b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d4c106690 0x7f0d4c102b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43938/0 (socket says 172.21.15.107:43938) 2024-04-25T00:26:16.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.818+0000 7f0d4b7fe640 1 -- 172.21.15.107:0/2582506006 learned_addr learned my addr 172.21.15.107:0/2582506006 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:16.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.818+0000 7f0d4affd640 1 --2- 172.21.15.107:0/2582506006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d4c100990 0x7f0d4c1020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:16.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.818+0000 7f0d4a7fc640 1 --2- 172.21.15.107:0/2582506006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d4c101330 0x7f0d4c102600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:16.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.818+0000 7f0d4b7fe640 1 -- 172.21.15.107:0/2582506006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d4c101330 msgr2=0x7f0d4c102600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:16.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.818+0000 7f0d4b7fe640 1 --2- 172.21.15.107:0/2582506006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d4c101330 0x7f0d4c102600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.818+0000 7f0d4b7fe640 1 -- 172.21.15.107:0/2582506006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d4c100990 msgr2=0x7f0d4c1020c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:16.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.818+0000 7f0d4b7fe640 1 --2- 172.21.15.107:0/2582506006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d4c100990 0x7f0d4c1020c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.818+0000 7f0d4b7fe640 1 -- 172.21.15.107:0/2582506006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4c1a63b0 con 0x7f0d4c106690 2024-04-25T00:26:16.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.818+0000 7f0d4a7fc640 1 --2- 172.21.15.107:0/2582506006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d4c101330 0x7f0d4c102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:16.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.819+0000 7f0d4b7fe640 1 --2- 172.21.15.107:0/2582506006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d4c106690 0x7f0d4c102b40 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f0d44005ab0 tx=0x7f0d44004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:16.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.819+0000 7f0d33fff640 1 -- 172.21.15.107:0/2582506006 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4403f030 con 0x7f0d4c106690 2024-04-25T00:26:16.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.819+0000 7f0d51441640 1 -- 172.21.15.107:0/2582506006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4c1a6640 con 0x7f0d4c106690 2024-04-25T00:26:16.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.819+0000 7f0d33fff640 1 -- 172.21.15.107:0/2582506006 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d44003ad0 con 0x7f0d4c106690 2024-04-25T00:26:16.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.819+0000 7f0d33fff640 1 -- 172.21.15.107:0/2582506006 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44035070 con 0x7f0d4c106690 2024-04-25T00:26:16.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.819+0000 7f0d51441640 1 -- 172.21.15.107:0/2582506006 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d4c1a6b30 con 0x7f0d4c106690 2024-04-25T00:26:16.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.821+0000 7f0d33fff640 1 -- 172.21.15.107:0/2582506006 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f0d44042030 con 0x7f0d4c106690 2024-04-25T00:26:16.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.821+0000 7f0d5421f740 1 Processor -- start 2024-04-25T00:26:16.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.822+0000 7f0d5421f740 1 -- start start 2024-04-25T00:26:16.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.822+0000 7f0d33fff640 1 --2- 172.21.15.107:0/2582506006 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f0d340a2d00 0x7f0d340a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:16.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.822+0000 7f0d5421f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1e8e630 0x55c7f1e8ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:16.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.822+0000 7f0d5421f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1e8ef50 0x55c7f1e990e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:16.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.822+0000 7f0d5421f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e99620 0x55c7f1e9ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:16.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.822+0000 7f0d5421f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c7f1df36b0 con 0x55c7f1e99620 2024-04-25T00:26:16.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.822+0000 7f0d5421f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c7f1d9b880 con 0x55c7f1e8e630 2024-04-25T00:26:16.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.822+0000 7f0d5421f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c7f1def670 con 0x55c7f1e8ef50 2024-04-25T00:26:16.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.822+0000 7f0d4a7fc640 1 --2- 172.21.15.107:0/2582506006 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f0d340a2d00 0x7f0d340a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:16.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.822+0000 7f0d33fff640 1 -- 172.21.15.107:0/2582506006 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f0d440f0250 con 0x7f0d4c106690 2024-04-25T00:26:16.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.822+0000 7f0d4b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e99620 0x55c7f1e9ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:16.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.822+0000 7f0d4b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e99620 0x55c7f1e9ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43950/0 (socket says 172.21.15.107:43950) 2024-04-25T00:26:16.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.822+0000 7f0d4b7fe640 1 -- 172.21.15.107:0/3621921698 learned_addr learned my addr 172.21.15.107:0/3621921698 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:16.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.822+0000 7f0d4a7fc640 1 --2- 172.21.15.107:0/3621921698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1e8ef50 0x55c7f1e990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:16.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.822+0000 7f0d4affd640 1 --2- 172.21.15.107:0/3621921698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1e8e630 0x55c7f1e8ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:16.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.822+0000 7f0d4b7fe640 1 -- 172.21.15.107:0/3621921698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1e8ef50 msgr2=0x55c7f1e990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:16.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.822+0000 7f0d4b7fe640 1 --2- 172.21.15.107:0/3621921698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1e8ef50 0x55c7f1e990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.822+0000 7f0d4b7fe640 1 -- 172.21.15.107:0/3621921698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1e8e630 msgr2=0x55c7f1e8ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:16.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.822+0000 7f0d4b7fe640 1 --2- 172.21.15.107:0/3621921698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1e8e630 0x55c7f1e8ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.822+0000 7f0d4b7fe640 1 -- 172.21.15.107:0/3621921698 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7f1bd17d0 con 0x55c7f1e99620 2024-04-25T00:26:16.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.822+0000 7f0d4affd640 1 --2- 172.21.15.107:0/3621921698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1e8e630 0x55c7f1e8ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:16.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.822+0000 7f0d4a7fc640 1 --2- 172.21.15.107:0/2582506006 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f0d340a2d00 0x7f0d340a51c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c100320 tx=0x7f0d400138e0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:16.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.822+0000 7f0d4b7fe640 1 --2- 172.21.15.107:0/3621921698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e99620 0x55c7f1e9ba10 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f0d440b8810 tx=0x7f0d440bbfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef41102f741ac3c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:16.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.823+0000 7f0d317fa640 1 -- 172.21.15.107:0/3621921698 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44049070 con 0x55c7f1e99620 2024-04-25T00:26:16.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.823+0000 7f0d317fa640 1 -- 172.21.15.107:0/3621921698 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d440bc3b0 con 0x55c7f1e99620 2024-04-25T00:26:16.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.823+0000 7f0d317fa640 1 -- 172.21.15.107:0/3621921698 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d440035f0 con 0x55c7f1e99620 2024-04-25T00:26:16.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.823+0000 7f0d5421f740 1 -- 172.21.15.107:0/3621921698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e99620 msgr2=0x55c7f1e9ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:16.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.823+0000 7f0d5421f740 1 --2- 172.21.15.107:0/3621921698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e99620 0x55c7f1e9ba10 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f0d440b8810 tx=0x7f0d440bbfb0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.823+0000 7f0d5421f740 1 -- 172.21.15.107:0/3621921698 shutdown_connections 2024-04-25T00:26:16.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.823+0000 7f0d5421f740 1 --2- 172.21.15.107:0/3621921698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e99620 0x55c7f1e9ba10 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.823+0000 7f0d5421f740 1 --2- 172.21.15.107:0/3621921698 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1e8ef50 0x55c7f1e990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.823+0000 7f0d5421f740 1 --2- 172.21.15.107:0/3621921698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1e8e630 0x55c7f1e8ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.823+0000 7f0d5421f740 1 -- 172.21.15.107:0/3621921698 >> 172.21.15.107:0/3621921698 conn(0x55c7f1e8a950 msgr2=0x55c7f1e8ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:16.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.823+0000 7f0d5421f740 1 -- 172.21.15.107:0/3621921698 shutdown_connections 2024-04-25T00:26:16.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.823+0000 7f0d5421f740 1 -- 172.21.15.107:0/3621921698 wait complete. 2024-04-25T00:26:16.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.824+0000 7f0d5421f740 1 -- start start 2024-04-25T00:26:16.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.824+0000 7f0d5421f740 1 Processor -- start 2024-04-25T00:26:16.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.824+0000 7f0d5421f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1e8e630 0x55c7f1e96550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:16.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.824+0000 7f0d5421f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1e8ef50 0x55c7f1e96a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:16.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.824+0000 7f0d5421f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e99620 0x55c7f1e96fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:16.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.824+0000 7f0d5421f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c7f1e9c1d0 con 0x55c7f1e99620 2024-04-25T00:26:16.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.824+0000 7f0d5421f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c7f1daef50 con 0x55c7f1e8e630 2024-04-25T00:26:16.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.824+0000 7f0d5421f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55c7f1e9c350 con 0x55c7f1e8ef50 2024-04-25T00:26:16.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.824+0000 7f0d5421f740 10 client.0 _ll_register_callbacks cb 0x55c7f1dea000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:16.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.824+0000 7f0d4b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e99620 0x55c7f1e96fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:16.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.824+0000 7f0d4b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e99620 0x55c7f1e96fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43956/0 (socket says 172.21.15.107:43956) 2024-04-25T00:26:16.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.824+0000 7f0d4b7fe640 1 -- 172.21.15.107:0/2777200118 learned_addr learned my addr 172.21.15.107:0/2777200118 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:16.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.824+0000 7f0d4affd640 1 --2- 172.21.15.107:0/2777200118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1e8e630 0x55c7f1e96550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:16.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.824+0000 7f0d4a7fc640 1 --2- 172.21.15.107:0/2777200118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1e8ef50 0x55c7f1e96a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:16.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.824+0000 7f0d4b7fe640 1 -- 172.21.15.107:0/2777200118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1e8ef50 msgr2=0x55c7f1e96a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:16.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.824+0000 7f0d4b7fe640 1 --2- 172.21.15.107:0/2777200118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1e8ef50 0x55c7f1e96a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.824+0000 7f0d4b7fe640 1 -- 172.21.15.107:0/2777200118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1e8e630 msgr2=0x55c7f1e96550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:16.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.824+0000 7f0d4b7fe640 1 --2- 172.21.15.107:0/2777200118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1e8e630 0x55c7f1e96550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.824+0000 7f0d4b7fe640 1 -- 172.21.15.107:0/2777200118 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7f1b073f0 con 0x55c7f1e99620 2024-04-25T00:26:16.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.824+0000 7f0d4b7fe640 1 --2- 172.21.15.107:0/2777200118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e99620 0x55c7f1e96fd0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f0d440edbe0 tx=0x7f0d440058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf4c58c55d96509 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:16.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.824+0000 7f0d2eff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:26:16.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.825+0000 7f0d2eff5640 1 -- 172.21.15.107:0/2777200118 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44049070 con 0x55c7f1e99620 2024-04-25T00:26:16.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.825+0000 7f0d2eff5640 1 -- 172.21.15.107:0/2777200118 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f0d440d5250 con 0x55c7f1e99620 2024-04-25T00:26:16.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.825+0000 7f0d5421f740 1 -- 172.21.15.107:0/2777200118 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d440bc3b0 con 0x55c7f1e99620 2024-04-25T00:26:16.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.825+0000 7f0d2eff5640 1 -- 172.21.15.107:0/2777200118 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0d440c0760 con 0x55c7f1e99620 2024-04-25T00:26:16.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.825+0000 7f0d5421f740 10 client.17742 fetch_fsmap learned FSMap version 34 2024-04-25T00:26:16.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.825+0000 7f0d5421f740 1 -- 172.21.15.107:0/2777200118 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c7f1ac70f0 con 0x55c7f1e99620 2024-04-25T00:26:16.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.825+0000 7f0d2eff5640 1 -- 172.21.15.107:0/2777200118 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d440bc3b0 con 0x55c7f1e99620 2024-04-25T00:26:16.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.825+0000 7f0d2eff5640 1 -- 172.21.15.107:0/2777200118 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f0d440bc610 con 0x55c7f1e99620 2024-04-25T00:26:16.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.825+0000 7f0d5421f740 10 client.17742 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:26:16.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.825+0000 7f0d5421f740 10 client.17742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:16.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.825+0000 7f0d5421f740 20 client.17742 populate_metadata read hostname 'smithi107' 2024-04-25T00:26:16.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.825+0000 7f0d5421f740 1 --2- 172.21.15.107:0/2777200118 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c7f1ea93e0 0x55c7f1eab7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:16.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.825+0000 7f0d5421f740 4 client.17742 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:16.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.825+0000 7f0d5421f740 1 -- 172.21.15.107:0/2777200118 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7f1e00bf0 con 0x55c7f1ea93e0 2024-04-25T00:26:16.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.825+0000 7f0d4affd640 1 --2- 172.21.15.107:0/2777200118 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c7f1ea93e0 0x55c7f1eab7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:16.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.826+0000 7f0d2eff5640 10 client.17742 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:26:16.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.826+0000 7f0d4affd640 1 --2- 172.21.15.107:0/2777200118 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c7f1ea93e0 0x55c7f1eab7d0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f83ceeb0efc562ee server_cookie=b81ce91903a5f645 in_seq=0 out_seq=0 2024-04-25T00:26:16.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.826+0000 7f0d2eff5640 1 -- 172.21.15.107:0/2777200118 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c7f1e00bf0 con 0x55c7f1ea93e0 2024-04-25T00:26:16.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.826+0000 7f0d2eff5640 10 client.17742 handle_command_reply: tid=0 2024-04-25T00:26:16.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.827+0000 7f0d5421f740 1 client.17742 shutdown 2024-04-25T00:26:16.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.827+0000 7f0d5421f740 1 -- 172.21.15.107:0/2777200118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e99620 msgr2=0x55c7f1e96fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:16.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.827+0000 7f0d5421f740 1 --2- 172.21.15.107:0/2777200118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e99620 0x55c7f1e96fd0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f0d440edbe0 tx=0x7f0d440058b0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.827+0000 7f0d5421f740 1 -- 172.21.15.107:0/2777200118 shutdown_connections 2024-04-25T00:26:16.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.827+0000 7f0d5421f740 1 --2- 172.21.15.107:0/2777200118 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c7f1ea93e0 0x55c7f1eab7d0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.827+0000 7f0d5421f740 1 --2- 172.21.15.107:0/2777200118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e99620 0x55c7f1e96fd0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.827+0000 7f0d5421f740 1 --2- 172.21.15.107:0/2777200118 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1e8ef50 0x55c7f1e96a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.827+0000 7f0d2eff5640 0 client.17742 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:26:16.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.827+0000 7f0d5421f740 1 --2- 172.21.15.107:0/2777200118 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1e8e630 0x55c7f1e96550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.827+0000 7f0d5421f740 1 -- 172.21.15.107:0/2777200118 >> 172.21.15.107:0/2777200118 conn(0x55c7f1e8a950 msgr2=0x55c7f1e90750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:16.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.827+0000 7f0d5421f740 1 -- 172.21.15.107:0/2777200118 shutdown_connections 2024-04-25T00:26:16.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.827+0000 7f0d5421f740 1 -- 172.21.15.107:0/2777200118 wait complete. 2024-04-25T00:26:16.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.827+0000 7f0d5421f740 20 client.17742 trim_cache size 0 max 16384 2024-04-25T00:26:16.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.844+0000 7f0d5421f740 1 Processor -- start 2024-04-25T00:26:16.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.844+0000 7f0d5421f740 1 -- start start 2024-04-25T00:26:16.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.844+0000 7f0d5421f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1e8e630 0x55c7f1ea4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:16.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.844+0000 7f0d5421f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e8ef50 0x55c7f1e96df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:16.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.844+0000 7f0d5421f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1e99620 0x55c7f1e95400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:16.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.844+0000 7f0d5421f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c1b35e0 con 0x55c7f1e8ef50 2024-04-25T00:26:16.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.844+0000 7f0d5421f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c1b0eb0 con 0x55c7f1e99620 2024-04-25T00:26:16.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.844+0000 7f0d5421f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c1aed20 con 0x55c7f1e8e630 2024-04-25T00:26:16.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.844+0000 7f0d4a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e8ef50 0x55c7f1e96df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:16.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.844+0000 7f0d4a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e8ef50 0x55c7f1e96df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43966/0 (socket says 172.21.15.107:43966) 2024-04-25T00:26:16.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.844+0000 7f0d4a7fc640 1 -- 172.21.15.107:0/3018118783 learned_addr learned my addr 172.21.15.107:0/3018118783 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:16.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.845+0000 7f0d4affd640 1 --2- 172.21.15.107:0/3018118783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1e8e630 0x55c7f1ea4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:16.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.845+0000 7f0d4a7fc640 1 -- 172.21.15.107:0/3018118783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1e8e630 msgr2=0x55c7f1ea4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:16.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.845+0000 7f0d4a7fc640 1 --2- 172.21.15.107:0/3018118783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1e8e630 0x55c7f1ea4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.845+0000 7f0d4b7fe640 1 --2- 172.21.15.107:0/3018118783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1e99620 0x55c7f1e95400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:16.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.845+0000 7f0d4a7fc640 1 -- 172.21.15.107:0/3018118783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1e99620 msgr2=0x55c7f1e95400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:16.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.845+0000 7f0d4a7fc640 1 --2- 172.21.15.107:0/3018118783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1e99620 0x55c7f1e95400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.845+0000 7f0d4a7fc640 1 -- 172.21.15.107:0/3018118783 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7f1e7c090 con 0x55c7f1e8ef50 2024-04-25T00:26:16.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.845+0000 7f0d4a7fc640 1 --2- 172.21.15.107:0/3018118783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e8ef50 0x55c7f1e96df0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f0d40013de0 tx=0x7f0d4001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1a98e567ee61bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:16.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.845+0000 7f0d30ff9640 1 -- 172.21.15.107:0/3018118783 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4002e070 con 0x55c7f1e8ef50 2024-04-25T00:26:16.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.845+0000 7f0d30ff9640 1 -- 172.21.15.107:0/3018118783 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d40021da0 con 0x55c7f1e8ef50 2024-04-25T00:26:16.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.845+0000 7f0d30ff9640 1 -- 172.21.15.107:0/3018118783 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d400103d0 con 0x55c7f1e8ef50 2024-04-25T00:26:16.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.845+0000 7f0d5421f740 1 -- 172.21.15.107:0/3018118783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e8ef50 msgr2=0x55c7f1e96df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:16.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.845+0000 7f0d5421f740 1 --2- 172.21.15.107:0/3018118783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e8ef50 0x55c7f1e96df0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f0d40013de0 tx=0x7f0d4001f000 comp rx=0 tx=0).stop 2024-04-25T00:26:16.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.846+0000 7f0d5421f740 1 -- 172.21.15.107:0/3018118783 shutdown_connections 2024-04-25T00:26:16.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.846+0000 7f0d5421f740 1 --2- 172.21.15.107:0/3018118783 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1e99620 0x55c7f1e95400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.846+0000 7f0d5421f740 1 --2- 172.21.15.107:0/3018118783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e8ef50 0x55c7f1e96df0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.846+0000 7f0d5421f740 1 --2- 172.21.15.107:0/3018118783 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1e8e630 0x55c7f1ea4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.846+0000 7f0d5421f740 1 -- 172.21.15.107:0/3018118783 >> 172.21.15.107:0/3018118783 conn(0x55c7f1e8a950 msgr2=0x55c7f1e94cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:16.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.846+0000 7f0d5421f740 1 -- 172.21.15.107:0/3018118783 shutdown_connections 2024-04-25T00:26:16.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.846+0000 7f0d5421f740 1 -- 172.21.15.107:0/3018118783 wait complete. 2024-04-25T00:26:16.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.846+0000 7f0d5421f740 1 -- start start 2024-04-25T00:26:16.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.846+0000 7f0d5421f740 1 Processor -- start 2024-04-25T00:26:16.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.846+0000 7f0d5421f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1e8e630 0x55c7f1ea1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:16.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.846+0000 7f0d5421f740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1e8ef50 0x55c7f1e9cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:16.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.846+0000 7f0d5421f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e99620 0x55c7f1e9d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:16.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.846+0000 7f0d5421f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c1af370 con 0x55c7f1e99620 2024-04-25T00:26:16.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.846+0000 7f0d5421f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c1b07e0 con 0x55c7f1e8e630 2024-04-25T00:26:16.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.846+0000 7f0d5421f740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c1afa40 con 0x55c7f1e8ef50 2024-04-25T00:26:16.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.846+0000 7f0d5421f740 10 client.0 _ll_register_callbacks cb 0x7f0d440bc180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:16.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.846+0000 7f0d4b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e99620 0x55c7f1e9d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:16.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.846+0000 7f0d4b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e99620 0x55c7f1e9d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43980/0 (socket says 172.21.15.107:43980) 2024-04-25T00:26:16.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.846+0000 7f0d4b7fe640 1 -- 172.21.15.107:0/3444760482 learned_addr learned my addr 172.21.15.107:0/3444760482 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:16.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.846+0000 7f0d4affd640 1 --2- 172.21.15.107:0/3444760482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1e8e630 0x55c7f1ea1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:16.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.846+0000 7f0d4a7fc640 1 --2- 172.21.15.107:0/3444760482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1e8ef50 0x55c7f1e9cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:16.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.847+0000 7f0d4b7fe640 1 -- 172.21.15.107:0/3444760482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1e8ef50 msgr2=0x55c7f1e9cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:16.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.847+0000 7f0d4b7fe640 1 --2- 172.21.15.107:0/3444760482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1e8ef50 0x55c7f1e9cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.847+0000 7f0d4b7fe640 1 -- 172.21.15.107:0/3444760482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1e8e630 msgr2=0x55c7f1ea1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:16.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.847+0000 7f0d4b7fe640 1 --2- 172.21.15.107:0/3444760482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1e8e630 0x55c7f1ea1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.847+0000 7f0d4b7fe640 1 -- 172.21.15.107:0/3444760482 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7f1f0e8a0 con 0x55c7f1e99620 2024-04-25T00:26:16.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.847+0000 7f0d4a7fc640 1 --2- 172.21.15.107:0/3444760482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1e8ef50 0x55c7f1e9cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:16.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.847+0000 7f0d4b7fe640 1 --2- 172.21.15.107:0/3444760482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e99620 0x55c7f1e9d520 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f0d440ed690 tx=0x7f0d440c4f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ef28305ab2c498 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:16.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.847+0000 7f0d2eff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:26:16.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.847+0000 7f0d2eff5640 1 -- 172.21.15.107:0/3444760482 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44049070 con 0x55c7f1e99620 2024-04-25T00:26:16.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.847+0000 7f0d2eff5640 1 -- 172.21.15.107:0/3444760482 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f0d44083dd0 con 0x55c7f1e99620 2024-04-25T00:26:16.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.847+0000 7f0d5421f740 1 -- 172.21.15.107:0/3444760482 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d40021da0 con 0x55c7f1e99620 2024-04-25T00:26:16.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.847+0000 7f0d2eff5640 1 -- 172.21.15.107:0/3444760482 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0d440bc3b0 con 0x55c7f1e99620 2024-04-25T00:26:16.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.847+0000 7f0d5421f740 10 client.17748 fetch_fsmap learned FSMap version 34 2024-04-25T00:26:16.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.847+0000 7f0d5421f740 1 -- 172.21.15.107:0/3444760482 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c7f1f0e8a0 con 0x55c7f1e99620 2024-04-25T00:26:16.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.847+0000 7f0d2eff5640 1 -- 172.21.15.107:0/3444760482 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d40021da0 con 0x55c7f1e99620 2024-04-25T00:26:16.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.847+0000 7f0d2eff5640 1 -- 172.21.15.107:0/3444760482 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f0d440bc610 con 0x55c7f1e99620 2024-04-25T00:26:16.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.847+0000 7f0d5421f740 10 client.17748 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:26:16.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.847+0000 7f0d5421f740 10 client.17748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:16.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.847+0000 7f0d5421f740 20 client.17748 populate_metadata read hostname 'smithi107' 2024-04-25T00:26:16.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.847+0000 7f0d5421f740 1 --2- 172.21.15.107:0/3444760482 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c7f1f0fdd0 0x55c7f1f121c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:16.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.847+0000 7f0d5421f740 4 client.17748 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:26:16.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.847+0000 7f0d5421f740 1 -- 172.21.15.107:0/3444760482 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c7f1dfaf40 con 0x55c7f1f0fdd0 2024-04-25T00:26:16.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.848+0000 7f0d4affd640 1 --2- 172.21.15.107:0/3444760482 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c7f1f0fdd0 0x55c7f1f121c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:16.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.848+0000 7f0d2eff5640 10 client.17748 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:26:16.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.848+0000 7f0d4affd640 1 --2- 172.21.15.107:0/3444760482 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c7f1f0fdd0 0x55c7f1f121c0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=178386f29a13735e server_cookie=cbf9f6595f546417 in_seq=0 out_seq=0 2024-04-25T00:26:16.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.848+0000 7f0d2eff5640 1 -- 172.21.15.107:0/3444760482 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c7f1dfaf40 con 0x55c7f1f0fdd0 2024-04-25T00:26:16.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.848+0000 7f0d2eff5640 10 client.17748 handle_command_reply: tid=0 2024-04-25T00:26:16.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.848+0000 7f0d5421f740 1 client.17748 shutdown 2024-04-25T00:26:16.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.848+0000 7f0d5421f740 1 -- 172.21.15.107:0/3444760482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e99620 msgr2=0x55c7f1e9d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:16.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.848+0000 7f0d5421f740 1 --2- 172.21.15.107:0/3444760482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e99620 0x55c7f1e9d520 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f0d440ed690 tx=0x7f0d440c4f50 comp rx=0 tx=0).stop 2024-04-25T00:26:16.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.848+0000 7f0d5421f740 1 -- 172.21.15.107:0/3444760482 shutdown_connections 2024-04-25T00:26:16.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.848+0000 7f0d5421f740 1 --2- 172.21.15.107:0/3444760482 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55c7f1f0fdd0 0x55c7f1f121c0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.848+0000 7f0d5421f740 1 --2- 172.21.15.107:0/3444760482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c7f1e99620 0x55c7f1e9d520 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.848+0000 7f0d5421f740 1 --2- 172.21.15.107:0/3444760482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55c7f1e8ef50 0x55c7f1e9cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.848+0000 7f0d2eff5640 0 client.17748 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:26:16.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.848+0000 7f0d5421f740 1 --2- 172.21.15.107:0/3444760482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c7f1e8e630 0x55c7f1ea1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.848+0000 7f0d5421f740 1 -- 172.21.15.107:0/3444760482 >> 172.21.15.107:0/3444760482 conn(0x55c7f1e8a950 msgr2=0x55c7f1e95580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:16.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.848+0000 7f0d5421f740 1 -- 172.21.15.107:0/3444760482 shutdown_connections 2024-04-25T00:26:16.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.848+0000 7f0d5421f740 1 -- 172.21.15.107:0/3444760482 wait complete. 2024-04-25T00:26:16.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.848+0000 7f0d5421f740 20 client.17748 trim_cache size 0 max 16384 2024-04-25T00:26:16.858 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:26:16.858 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-04-25T00:26:16.858 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-04-25T00:26:16.858 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:26:16.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.849+0000 7f0d51441640 1 -- 172.21.15.107:0/2582506006 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f0d340a2d00 msgr2=0x7f0d340a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:16.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.849+0000 7f0d51441640 1 --2- 172.21.15.107:0/2582506006 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f0d340a2d00 0x7f0d340a51c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c100320 tx=0x7f0d400138e0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.849+0000 7f0d51441640 1 -- 172.21.15.107:0/2582506006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d4c106690 msgr2=0x7f0d4c102b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:16.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.849+0000 7f0d51441640 1 --2- 172.21.15.107:0/2582506006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d4c106690 0x7f0d4c102b40 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f0d44005ab0 tx=0x7f0d44004270 comp rx=0 tx=0).stop 2024-04-25T00:26:16.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.849+0000 7f0d51441640 1 -- 172.21.15.107:0/2582506006 shutdown_connections 2024-04-25T00:26:16.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.849+0000 7f0d51441640 1 --2- 172.21.15.107:0/2582506006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d4c106690 0x7f0d4c102b40 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.849+0000 7f0d51441640 1 --2- 172.21.15.107:0/2582506006 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0d4c101330 0x7f0d4c102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.849+0000 7f0d51441640 1 --2- 172.21.15.107:0/2582506006 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f0d340a2d00 0x7f0d340a51c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.849+0000 7f0d51441640 1 --2- 172.21.15.107:0/2582506006 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0d4c100990 0x7f0d4c1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.849+0000 7f0d51441640 1 -- 172.21.15.107:0/2582506006 >> 172.21.15.107:0/2582506006 conn(0x7f0d4c076440 msgr2=0x7f0d4c069f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:16.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.849+0000 7f0d51441640 1 -- 172.21.15.107:0/2582506006 shutdown_connections 2024-04-25T00:26:16.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.849+0000 7f0d51441640 1 -- 172.21.15.107:0/2582506006 wait complete. 2024-04-25T00:26:16.861 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:26:16.861 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:26:16.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.959+0000 7fbcaf7fb640 1 -- 172.21.15.107:0/1488664209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca8100090 msgr2=0x7fbca8100510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:16.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.959+0000 7fbcaf7fb640 1 --2- 172.21.15.107:0/1488664209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca8100090 0x7fbca8100510 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009900 tx=0x7fbc9802a480 comp rx=0 tx=0).stop 2024-04-25T00:26:16.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.959+0000 7fbcaf7fb640 1 -- 172.21.15.107:0/1488664209 shutdown_connections 2024-04-25T00:26:16.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.959+0000 7fbcaf7fb640 1 --2- 172.21.15.107:0/1488664209 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbca8100a50 0x7fbca810cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.959+0000 7fbcaf7fb640 1 --2- 172.21.15.107:0/1488664209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca8100090 0x7fbca8100510 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.962 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.959+0000 7fbcaf7fb640 1 --2- 172.21.15.107:0/1488664209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca8104dd0 0x7fbca81051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.959+0000 7fbcaf7fb640 1 -- 172.21.15.107:0/1488664209 >> 172.21.15.107:0/1488664209 conn(0x7fbca8075920 msgr2=0x7fbca8075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:16.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.959+0000 7fbcaf7fb640 1 -- 172.21.15.107:0/1488664209 shutdown_connections 2024-04-25T00:26:16.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.959+0000 7fbcaf7fb640 1 -- 172.21.15.107:0/1488664209 wait complete. 2024-04-25T00:26:16.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.960+0000 7fbcaf7fb640 1 Processor -- start 2024-04-25T00:26:16.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.960+0000 7fbcaf7fb640 1 -- start start 2024-04-25T00:26:16.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.960+0000 7fbcaf7fb640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca8100090 0x7fbca8195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:16.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.960+0000 7fbcaf7fb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca8100a50 0x7fbca8196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:16.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.960+0000 7fbcaf7fb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbca8104dd0 0x7fbca819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:16.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.960+0000 7fbcaf7fb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbca810f960 con 0x7fbca8100a50 2024-04-25T00:26:16.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.960+0000 7fbcaf7fb640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbca810f7e0 con 0x7fbca8100090 2024-04-25T00:26:16.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.960+0000 7fbcaf7fb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbca810fae0 con 0x7fbca8104dd0 2024-04-25T00:26:16.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.960+0000 7fbcacd96640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca8100a50 0x7fbca8196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:16.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.960+0000 7fbcacd96640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca8100a50 0x7fbca8196320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44002/0 (socket says 172.21.15.107:44002) 2024-04-25T00:26:16.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.960+0000 7fbcacd96640 1 -- 172.21.15.107:0/623967126 learned_addr learned my addr 172.21.15.107:0/623967126 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:16.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.960+0000 7fbcad597640 1 --2- 172.21.15.107:0/623967126 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca8100090 0x7fbca8195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:16.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.960+0000 7fbcadd98640 1 --2- 172.21.15.107:0/623967126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbca8104dd0 0x7fbca819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:16.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.960+0000 7fbcacd96640 1 -- 172.21.15.107:0/623967126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbca8104dd0 msgr2=0x7fbca819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:16.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.960+0000 7fbcacd96640 1 --2- 172.21.15.107:0/623967126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbca8104dd0 0x7fbca819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.960+0000 7fbcacd96640 1 -- 172.21.15.107:0/623967126 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca8100090 msgr2=0x7fbca8195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:16.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.960+0000 7fbcacd96640 1 --2- 172.21.15.107:0/623967126 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca8100090 0x7fbca8195de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:16.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.960+0000 7fbcacd96640 1 -- 172.21.15.107:0/623967126 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca819ad80 con 0x7fbca8100a50 2024-04-25T00:26:16.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.960+0000 7fbcacd96640 1 --2- 172.21.15.107:0/623967126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca8100a50 0x7fbca8196320 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fbc98006010 tx=0x7fbc98003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:16.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.961+0000 7fbc9e7fc640 1 -- 172.21.15.107:0/623967126 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98035070 con 0x7fbca8100a50 2024-04-25T00:26:16.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.961+0000 7fbc9e7fc640 1 -- 172.21.15.107:0/623967126 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc98031a20 con 0x7fbca8100a50 2024-04-25T00:26:16.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.961+0000 7fbcaf7fb640 1 -- 172.21.15.107:0/623967126 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca819b010 con 0x7fbca8100a50 2024-04-25T00:26:16.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.961+0000 7fbc9e7fc640 1 -- 172.21.15.107:0/623967126 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc98038bf0 con 0x7fbca8100a50 2024-04-25T00:26:16.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.961+0000 7fbcaf7fb640 1 -- 172.21.15.107:0/623967126 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca81a2900 con 0x7fbca8100a50 2024-04-25T00:26:16.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.962+0000 7fbcaf7fb640 1 -- 172.21.15.107:0/623967126 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbca8101500 con 0x7fbca8100a50 2024-04-25T00:26:16.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.963+0000 7fbc9e7fc640 1 -- 172.21.15.107:0/623967126 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fbc98031540 con 0x7fbca8100a50 2024-04-25T00:26:16.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.963+0000 7fbc9e7fc640 1 --2- 172.21.15.107:0/623967126 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbc940a2cf0 0x7fbc940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:16.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.963+0000 7fbcad597640 1 --2- 172.21.15.107:0/623967126 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbc940a2cf0 0x7fbc940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:16.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.963+0000 7fbc9e7fc640 1 -- 172.21.15.107:0/623967126 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7fbc980f1910 con 0x7fbca8100a50 2024-04-25T00:26:16.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.964+0000 7fbcad597640 1 --2- 172.21.15.107:0/623967126 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbc940a2cf0 0x7fbc940a51b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbca8078150 tx=0x7fbca400a2d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:16.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:16.968+0000 7fbc9e7fc640 1 -- 172.21.15.107:0/623967126 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbc980be170 con 0x7fbca8100a50 2024-04-25T00:26:17.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.183+0000 7fbcaf7fb640 1 -- 172.21.15.107:0/623967126 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbca8108350 con 0x7fbca8100a50 2024-04-25T00:26:17.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.184+0000 7fbc9e7fc640 1 -- 172.21.15.107:0/623967126 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11897 (secure 0 0 0) 0x7fbc980bda70 con 0x7fbca8100a50 2024-04-25T00:26:17.184 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:26:17.185 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:22:27.402680+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24493,24521]},"id":1}]} 2024-04-25T00:26:17.185 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 34 2024-04-25T00:26:17.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.186+0000 7fbcaf7fb640 1 -- 172.21.15.107:0/623967126 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbc940a2cf0 msgr2=0x7fbc940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.186+0000 7fbcaf7fb640 1 --2- 172.21.15.107:0/623967126 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbc940a2cf0 0x7fbc940a51b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbca8078150 tx=0x7fbca400a2d0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.186+0000 7fbcaf7fb640 1 -- 172.21.15.107:0/623967126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca8100a50 msgr2=0x7fbca8196320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.186+0000 7fbcaf7fb640 1 --2- 172.21.15.107:0/623967126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca8100a50 0x7fbca8196320 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fbc98006010 tx=0x7fbc98003b90 comp rx=0 tx=0).stop 2024-04-25T00:26:17.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.186+0000 7fbcaf7fb640 1 -- 172.21.15.107:0/623967126 shutdown_connections 2024-04-25T00:26:17.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.186+0000 7fbcaf7fb640 1 --2- 172.21.15.107:0/623967126 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbca8104dd0 0x7fbca819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.186+0000 7fbcaf7fb640 1 --2- 172.21.15.107:0/623967126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbca8100a50 0x7fbca8196320 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.186+0000 7fbcaf7fb640 1 --2- 172.21.15.107:0/623967126 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbc940a2cf0 0x7fbc940a51b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.186+0000 7fbcaf7fb640 1 --2- 172.21.15.107:0/623967126 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca8100090 0x7fbca8195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.186+0000 7fbcaf7fb640 1 -- 172.21.15.107:0/623967126 >> 172.21.15.107:0/623967126 conn(0x7fbca8075920 msgr2=0x7fbca80fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:17.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.186+0000 7fbcaf7fb640 1 -- 172.21.15.107:0/623967126 shutdown_connections 2024-04-25T00:26:17.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.186+0000 7fbcaf7fb640 1 -- 172.21.15.107:0/623967126 wait complete. 2024-04-25T00:26:17.197 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:26:17.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.300+0000 7fef39914640 1 -- 172.21.15.107:0/3458919556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef34100db0 msgr2=0x7fef34101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.300+0000 7fef39914640 1 --2- 172.21.15.107:0/3458919556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef34100db0 0x7fef34101230 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fef28009900 tx=0x7fef2802a410 comp rx=0 tx=0).stop 2024-04-25T00:26:17.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.300+0000 7fef39914640 1 -- 172.21.15.107:0/3458919556 shutdown_connections 2024-04-25T00:26:17.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.300+0000 7fef39914640 1 --2- 172.21.15.107:0/3458919556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef34101770 0x7fef3410d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.300+0000 7fef39914640 1 --2- 172.21.15.107:0/3458919556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef34100db0 0x7fef34101230 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.300+0000 7fef39914640 1 --2- 172.21.15.107:0/3458919556 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef341055f0 0x7fef341059d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.300+0000 7fef39914640 1 -- 172.21.15.107:0/3458919556 >> 172.21.15.107:0/3458919556 conn(0x7fef34076c00 msgr2=0x7fef340fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:17.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.300+0000 7fef39914640 1 -- 172.21.15.107:0/3458919556 shutdown_connections 2024-04-25T00:26:17.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.300+0000 7fef39914640 1 -- 172.21.15.107:0/3458919556 wait complete. 2024-04-25T00:26:17.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.300+0000 7fef39914640 1 Processor -- start 2024-04-25T00:26:17.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.301+0000 7fef39914640 1 -- start start 2024-04-25T00:26:17.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.301+0000 7fef39914640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef34100db0 0x7fef34198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.301+0000 7fef39914640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef34101770 0x7fef34198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.301+0000 7fef39914640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef341055f0 0x7fef3419c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.301+0000 7fef39914640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fef34111bb0 con 0x7fef341055f0 2024-04-25T00:26:17.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.301+0000 7fef39914640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fef34111a30 con 0x7fef34100db0 2024-04-25T00:26:17.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.301+0000 7fef39914640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fef34111d30 con 0x7fef34101770 2024-04-25T00:26:17.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.301+0000 7fef337fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef341055f0 0x7fef3419c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.301+0000 7fef337fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef341055f0 0x7fef3419c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44032/0 (socket says 172.21.15.107:44032) 2024-04-25T00:26:17.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.301+0000 7fef337fe640 1 -- 172.21.15.107:0/1906622104 learned_addr learned my addr 172.21.15.107:0/1906622104 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:17.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.301+0000 7fef327fc640 1 --2- 172.21.15.107:0/1906622104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef34101770 0x7fef34198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.301+0000 7fef32ffd640 1 --2- 172.21.15.107:0/1906622104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef34100db0 0x7fef34198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.301+0000 7fef337fe640 1 -- 172.21.15.107:0/1906622104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef34101770 msgr2=0x7fef34198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.301+0000 7fef337fe640 1 --2- 172.21.15.107:0/1906622104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef34101770 0x7fef34198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.301+0000 7fef337fe640 1 -- 172.21.15.107:0/1906622104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef34100db0 msgr2=0x7fef34198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.301+0000 7fef337fe640 1 --2- 172.21.15.107:0/1906622104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef34100db0 0x7fef34198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.301+0000 7fef337fe640 1 -- 172.21.15.107:0/1906622104 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef3419d120 con 0x7fef341055f0 2024-04-25T00:26:17.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.301+0000 7fef32ffd640 1 --2- 172.21.15.107:0/1906622104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef34100db0 0x7fef34198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:17.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.302+0000 7fef337fe640 1 --2- 172.21.15.107:0/1906622104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef341055f0 0x7fef3419c9f0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fef24009fd0 tx=0x7fef2400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:17.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.302+0000 7fef38912640 1 -- 172.21.15.107:0/1906622104 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef24011830 con 0x7fef341055f0 2024-04-25T00:26:17.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.302+0000 7fef38912640 1 -- 172.21.15.107:0/1906622104 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef24016070 con 0x7fef341055f0 2024-04-25T00:26:17.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.302+0000 7fef39914640 1 -- 172.21.15.107:0/1906622104 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef3419d410 con 0x7fef341055f0 2024-04-25T00:26:17.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.302+0000 7fef38912640 1 -- 172.21.15.107:0/1906622104 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2400f2e0 con 0x7fef341055f0 2024-04-25T00:26:17.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.302+0000 7fef39914640 1 -- 172.21.15.107:0/1906622104 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef341a4ca0 con 0x7fef341055f0 2024-04-25T00:26:17.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.304+0000 7fef38912640 1 -- 172.21.15.107:0/1906622104 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fef240119d0 con 0x7fef341055f0 2024-04-25T00:26:17.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.304+0000 7fef3c624740 1 Processor -- start 2024-04-25T00:26:17.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.304+0000 7fef3c624740 1 -- start start 2024-04-25T00:26:17.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.304+0000 7fef3c624740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563cfac362b0 0x563cfac36690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.304+0000 7fef3c624740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac36c60 0x563cfac45440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.305+0000 7fef3c624740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfac45980 0x563cfac47d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.305+0000 7fef3c624740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563cfab9b670 con 0x563cfac36c60 2024-04-25T00:26:17.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.305+0000 7fef3c624740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563cfab43970 con 0x563cfac362b0 2024-04-25T00:26:17.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.305+0000 7fef3c624740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563cfab97630 con 0x563cfac45980 2024-04-25T00:26:17.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.305+0000 7fef38912640 1 --2- 172.21.15.107:0/1906622104 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fef1c0a2d00 0x7fef1c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.305+0000 7fef32ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac36c60 0x563cfac45440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.305+0000 7fef32ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac36c60 0x563cfac45440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44048/0 (socket says 172.21.15.107:44048) 2024-04-25T00:26:17.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.305+0000 7fef32ffd640 1 -- 172.21.15.107:0/3941416440 learned_addr learned my addr 172.21.15.107:0/3941416440 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:17.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.305+0000 7fef337fe640 1 --2- 172.21.15.107:0/1906622104 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fef1c0a2d00 0x7fef1c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.305+0000 7fef38912640 1 -- 172.21.15.107:0/1906622104 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7fef240ced20 con 0x7fef341055f0 2024-04-25T00:26:17.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.305+0000 7fef327fc640 1 --2- 172.21.15.107:0/3941416440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfac45980 0x563cfac47d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.305+0000 7fef32ffd640 1 -- 172.21.15.107:0/3941416440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfac45980 msgr2=0x563cfac47d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:17.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.305+0000 7fef327fc640 1 --2- 172.21.15.107:0/3941416440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563cfac362b0 0x563cfac36690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.305+0000 7fef32ffd640 1 --2- 172.21.15.107:0/3941416440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfac45980 0x563cfac47d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.305+0000 7fef32ffd640 1 -- 172.21.15.107:0/3941416440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563cfac362b0 msgr2=0x563cfac36690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:17.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.305+0000 7fef32ffd640 1 --2- 172.21.15.107:0/3941416440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563cfac362b0 0x563cfac36690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.305+0000 7fef32ffd640 1 -- 172.21.15.107:0/3941416440 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cfab67400 con 0x563cfac36c60 2024-04-25T00:26:17.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.305+0000 7fef327fc640 1 --2- 172.21.15.107:0/3941416440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563cfac362b0 0x563cfac36690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:17.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.305+0000 7fef32ffd640 1 --2- 172.21.15.107:0/3941416440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac36c60 0x563cfac45440 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fef2c007fb0 tx=0x7fef2c00c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12808e4021f43321 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:17.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.305+0000 7fef337fe640 1 --2- 172.21.15.107:0/1906622104 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fef1c0a2d00 0x7fef1c0a51c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fef2409ac90 tx=0x7fef2406cf50 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:17.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.305+0000 7fef19ffb640 1 -- 172.21.15.107:0/3941416440 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c014070 con 0x563cfac36c60 2024-04-25T00:26:17.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.305+0000 7fef19ffb640 1 -- 172.21.15.107:0/3941416440 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef2c00fd30 con 0x563cfac36c60 2024-04-25T00:26:17.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.305+0000 7fef19ffb640 1 -- 172.21.15.107:0/3941416440 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c00eda0 con 0x563cfac36c60 2024-04-25T00:26:17.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.306+0000 7fef3c624740 1 -- 172.21.15.107:0/3941416440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac36c60 msgr2=0x563cfac45440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.306+0000 7fef3c624740 1 --2- 172.21.15.107:0/3941416440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac36c60 0x563cfac45440 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fef2c007fb0 tx=0x7fef2c00c750 comp rx=0 tx=0).stop 2024-04-25T00:26:17.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.306+0000 7fef3c624740 1 -- 172.21.15.107:0/3941416440 shutdown_connections 2024-04-25T00:26:17.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.306+0000 7fef3c624740 1 --2- 172.21.15.107:0/3941416440 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfac45980 0x563cfac47d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.306+0000 7fef3c624740 1 --2- 172.21.15.107:0/3941416440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac36c60 0x563cfac45440 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.306+0000 7fef3c624740 1 --2- 172.21.15.107:0/3941416440 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563cfac362b0 0x563cfac36690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.306+0000 7fef3c624740 1 -- 172.21.15.107:0/3941416440 >> 172.21.15.107:0/3941416440 conn(0x563cfac328a0 msgr2=0x563cfac32c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:17.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.306+0000 7fef3c624740 1 -- 172.21.15.107:0/3941416440 shutdown_connections 2024-04-25T00:26:17.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.306+0000 7fef3c624740 1 -- 172.21.15.107:0/3941416440 wait complete. 2024-04-25T00:26:17.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.306+0000 7fef3c624740 1 -- start start 2024-04-25T00:26:17.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.307+0000 7fef3c624740 1 Processor -- start 2024-04-25T00:26:17.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.307+0000 7fef3c624740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfac362b0 0x563cfac452a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.307+0000 7fef3c624740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac36c60 0x563cfac42590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.307+0000 7fef3c624740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563cfac45980 0x563cfac42ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.307+0000 7fef3c624740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563cfac48740 con 0x563cfac36c60 2024-04-25T00:26:17.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.307+0000 7fef3c624740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563cfab57050 con 0x563cfac45980 2024-04-25T00:26:17.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.307+0000 7fef3c624740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x563cfac488c0 con 0x563cfac362b0 2024-04-25T00:26:17.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.307+0000 7fef3c624740 10 client.0 _ll_register_callbacks cb 0x563cfa8aebe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:17.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.307+0000 7fef32ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac36c60 0x563cfac42590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.307+0000 7fef32ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac36c60 0x563cfac42590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44060/0 (socket says 172.21.15.107:44060) 2024-04-25T00:26:17.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.307+0000 7fef32ffd640 1 -- 172.21.15.107:0/1635874869 learned_addr learned my addr 172.21.15.107:0/1635874869 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:17.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.307+0000 7fef327fc640 1 --2- 172.21.15.107:0/1635874869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563cfac45980 0x563cfac42ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.307+0000 7fef327fc640 1 --2- 172.21.15.107:0/1635874869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfac362b0 0x563cfac452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.307+0000 7fef32ffd640 1 -- 172.21.15.107:0/1635874869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfac362b0 msgr2=0x563cfac452a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:17.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.307+0000 7fef32ffd640 1 --2- 172.21.15.107:0/1635874869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfac362b0 0x563cfac452a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.307+0000 7fef32ffd640 1 -- 172.21.15.107:0/1635874869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563cfac45980 msgr2=0x563cfac42ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:17.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.307+0000 7fef32ffd640 1 --2- 172.21.15.107:0/1635874869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563cfac45980 0x563cfac42ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.307+0000 7fef32ffd640 1 -- 172.21.15.107:0/1635874869 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cfab28dc0 con 0x563cfac36c60 2024-04-25T00:26:17.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.307+0000 7fef327fc640 1 --2- 172.21.15.107:0/1635874869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfac362b0 0x563cfac452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:17.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.307+0000 7fef32ffd640 1 --2- 172.21.15.107:0/1635874869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac36c60 0x563cfac42590 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fef2c004a80 tx=0x7fef2c007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10104f5cfd865ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:17.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.307+0000 7fef12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:26:17.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.307+0000 7fef12ffd640 1 -- 172.21.15.107:0/1635874869 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c014040 con 0x563cfac36c60 2024-04-25T00:26:17.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.307+0000 7fef12ffd640 1 -- 172.21.15.107:0/1635874869 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7fef2c00f850 con 0x563cfac36c60 2024-04-25T00:26:17.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.307+0000 7fef3c624740 1 -- 172.21.15.107:0/1635874869 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef2c00fd30 con 0x563cfac36c60 2024-04-25T00:26:17.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.308+0000 7fef12ffd640 1 -- 172.21.15.107:0/1635874869 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fef2c02f050 con 0x563cfac36c60 2024-04-25T00:26:17.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.308+0000 7fef3c624740 10 client.17766 fetch_fsmap learned FSMap version 34 2024-04-25T00:26:17.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.308+0000 7fef3c624740 1 -- 172.21.15.107:0/1635874869 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563cfa8af3f0 con 0x563cfac36c60 2024-04-25T00:26:17.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.308+0000 7fef12ffd640 1 -- 172.21.15.107:0/1635874869 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef2c00fd30 con 0x563cfac36c60 2024-04-25T00:26:17.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.308+0000 7fef12ffd640 1 -- 172.21.15.107:0/1635874869 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fef2c021420 con 0x563cfac36c60 2024-04-25T00:26:17.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.308+0000 7fef3c624740 10 client.17766 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:26:17.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.308+0000 7fef3c624740 10 client.17766 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:26:17.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.308+0000 7fef3c624740 20 client.17766 populate_metadata read hostname 'smithi107' 2024-04-25T00:26:17.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.308+0000 7fef3c624740 1 --2- 172.21.15.107:0/1635874869 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x563cfac55840 0x563cfac57c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.308+0000 7fef3c624740 4 client.17766 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:17.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.308+0000 7fef3c624740 1 -- 172.21.15.107:0/1635874869 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cfaba8bb0 con 0x563cfac55840 2024-04-25T00:26:17.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.308+0000 7fef327fc640 1 --2- 172.21.15.107:0/1635874869 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x563cfac55840 0x563cfac57c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.309+0000 7fef12ffd640 10 client.17766 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:26:17.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.309+0000 7fef327fc640 1 --2- 172.21.15.107:0/1635874869 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x563cfac55840 0x563cfac57c30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4f1d04c1b7d3bc2e server_cookie=202561896bb7f0df in_seq=0 out_seq=0 2024-04-25T00:26:17.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.309+0000 7fef12ffd640 1 -- 172.21.15.107:0/1635874869 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563cfaba8bb0 con 0x563cfac55840 2024-04-25T00:26:17.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.309+0000 7fef12ffd640 10 client.17766 handle_command_reply: tid=0 2024-04-25T00:26:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.310+0000 7fef3c624740 1 client.17766 shutdown 2024-04-25T00:26:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.310+0000 7fef3c624740 1 -- 172.21.15.107:0/1635874869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac36c60 msgr2=0x563cfac42590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.310+0000 7fef3c624740 1 --2- 172.21.15.107:0/1635874869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac36c60 0x563cfac42590 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fef2c004a80 tx=0x7fef2c007620 comp rx=0 tx=0).stop 2024-04-25T00:26:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.310+0000 7fef3c624740 1 -- 172.21.15.107:0/1635874869 shutdown_connections 2024-04-25T00:26:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.310+0000 7fef3c624740 1 --2- 172.21.15.107:0/1635874869 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x563cfac55840 0x563cfac57c30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.310+0000 7fef3c624740 1 --2- 172.21.15.107:0/1635874869 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563cfac45980 0x563cfac42ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.310+0000 7fef3c624740 1 --2- 172.21.15.107:0/1635874869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac36c60 0x563cfac42590 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.310+0000 7fef3c624740 1 --2- 172.21.15.107:0/1635874869 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfac362b0 0x563cfac452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.310+0000 7fef12ffd640 0 client.17766 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:26:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.310+0000 7fef3c624740 1 -- 172.21.15.107:0/1635874869 >> 172.21.15.107:0/1635874869 conn(0x563cfac328a0 msgr2=0x563cfac38710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.310+0000 7fef3c624740 1 -- 172.21.15.107:0/1635874869 shutdown_connections 2024-04-25T00:26:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.310+0000 7fef3c624740 1 -- 172.21.15.107:0/1635874869 wait complete. 2024-04-25T00:26:17.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.310+0000 7fef3c624740 20 client.17766 trim_cache size 0 max 16384 2024-04-25T00:26:17.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.323+0000 7fef3c624740 1 Processor -- start 2024-04-25T00:26:17.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.324+0000 7fef3c624740 1 -- start start 2024-04-25T00:26:17.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.324+0000 7fef3c624740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfac362b0 0x563cfac51640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.324+0000 7fef3c624740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac36c60 0x563cfac41710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.324+0000 7fef3c624740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563cfac45980 0x563cfac3daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.324+0000 7fef3c624740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fef341af4b0 con 0x563cfac36c60 2024-04-25T00:26:17.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.324+0000 7fef3c624740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fef341acd80 con 0x563cfac45980 2024-04-25T00:26:17.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.324+0000 7fef3c624740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fef341aabf0 con 0x563cfac362b0 2024-04-25T00:26:17.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.324+0000 7fef32ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac36c60 0x563cfac41710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.324+0000 7fef32ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac36c60 0x563cfac41710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44072/0 (socket says 172.21.15.107:44072) 2024-04-25T00:26:17.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.324+0000 7fef32ffd640 1 -- 172.21.15.107:0/321788033 learned_addr learned my addr 172.21.15.107:0/321788033 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:17.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.324+0000 7fef327fc640 1 --2- 172.21.15.107:0/321788033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfac362b0 0x563cfac51640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.324+0000 7fef327fc640 1 --2- 172.21.15.107:0/321788033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563cfac45980 0x563cfac3daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.324+0000 7fef32ffd640 1 -- 172.21.15.107:0/321788033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfac362b0 msgr2=0x563cfac51640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:17.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.324+0000 7fef32ffd640 1 --2- 172.21.15.107:0/321788033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfac362b0 0x563cfac51640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.324+0000 7fef32ffd640 1 -- 172.21.15.107:0/321788033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563cfac45980 msgr2=0x563cfac3daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:17.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.324+0000 7fef32ffd640 1 --2- 172.21.15.107:0/321788033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563cfac45980 0x563cfac3daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.324+0000 7fef32ffd640 1 -- 172.21.15.107:0/321788033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cfa9797d0 con 0x563cfac36c60 2024-04-25T00:26:17.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.324+0000 7fef327fc640 1 --2- 172.21.15.107:0/321788033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563cfac45980 0x563cfac3daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:17.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.324+0000 7fef32ffd640 1 --2- 172.21.15.107:0/321788033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac36c60 0x563cfac41710 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fef2c007d40 tx=0x7fef2c007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=952ac3b600851929 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:17.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.324+0000 7fef197fa640 1 -- 172.21.15.107:0/321788033 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c014070 con 0x563cfac36c60 2024-04-25T00:26:17.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.324+0000 7fef197fa640 1 -- 172.21.15.107:0/321788033 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef2c038070 con 0x563cfac36c60 2024-04-25T00:26:17.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.324+0000 7fef197fa640 1 -- 172.21.15.107:0/321788033 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2c021e60 con 0x563cfac36c60 2024-04-25T00:26:17.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.325+0000 7fef3c624740 1 -- 172.21.15.107:0/321788033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac36c60 msgr2=0x563cfac41710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.325+0000 7fef3c624740 1 --2- 172.21.15.107:0/321788033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac36c60 0x563cfac41710 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fef2c007d40 tx=0x7fef2c007690 comp rx=0 tx=0).stop 2024-04-25T00:26:17.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.325+0000 7fef3c624740 1 -- 172.21.15.107:0/321788033 shutdown_connections 2024-04-25T00:26:17.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.325+0000 7fef3c624740 1 --2- 172.21.15.107:0/321788033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563cfac45980 0x563cfac3daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.325+0000 7fef3c624740 1 --2- 172.21.15.107:0/321788033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac36c60 0x563cfac41710 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.325+0000 7fef3c624740 1 --2- 172.21.15.107:0/321788033 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfac362b0 0x563cfac51640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.325+0000 7fef3c624740 1 -- 172.21.15.107:0/321788033 >> 172.21.15.107:0/321788033 conn(0x563cfac328a0 msgr2=0x563cfac3d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:17.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.325+0000 7fef3c624740 1 -- 172.21.15.107:0/321788033 shutdown_connections 2024-04-25T00:26:17.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.325+0000 7fef3c624740 1 -- 172.21.15.107:0/321788033 wait complete. 2024-04-25T00:26:17.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.325+0000 7fef3c624740 1 -- start start 2024-04-25T00:26:17.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.326+0000 7fef3c624740 1 Processor -- start 2024-04-25T00:26:17.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.326+0000 7fef3c624740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac362b0 0x563cfac40290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.326+0000 7fef3c624740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563cfac36c60 0x563cfac4db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.326+0000 7fef3c624740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfac45980 0x563cfac3e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.326+0000 7fef3c624740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fef341ab240 con 0x563cfac362b0 2024-04-25T00:26:17.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.326+0000 7fef3c624740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fef341ac6b0 con 0x563cfac36c60 2024-04-25T00:26:17.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.326+0000 7fef3c624740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fef341ab910 con 0x563cfac45980 2024-04-25T00:26:17.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.326+0000 7fef3c624740 10 client.0 _ll_register_callbacks cb 0x7fef2c007e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:17.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.326+0000 7fef327fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac362b0 0x563cfac40290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.326+0000 7fef327fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac362b0 0x563cfac40290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44080/0 (socket says 172.21.15.107:44080) 2024-04-25T00:26:17.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.326+0000 7fef327fc640 1 -- 172.21.15.107:0/3956223961 learned_addr learned my addr 172.21.15.107:0/3956223961 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:17.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.326+0000 7fef32ffd640 1 --2- 172.21.15.107:0/3956223961 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563cfac36c60 0x563cfac4db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.326+0000 7fef327fc640 1 -- 172.21.15.107:0/3956223961 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfac45980 msgr2=0x563cfac3e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:17.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.326+0000 7fef327fc640 1 --2- 172.21.15.107:0/3956223961 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfac45980 0x563cfac3e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.326+0000 7fef327fc640 1 -- 172.21.15.107:0/3956223961 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563cfac36c60 msgr2=0x563cfac4db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:17.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.326+0000 7fef327fc640 1 --2- 172.21.15.107:0/3956223961 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563cfac36c60 0x563cfac4db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.326+0000 7fef327fc640 1 -- 172.21.15.107:0/3956223961 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cfac3ed60 con 0x563cfac362b0 2024-04-25T00:26:17.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.326+0000 7fef327fc640 1 --2- 172.21.15.107:0/3956223961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac362b0 0x563cfac40290 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fef28009900 tx=0x7fef2802a9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c871242bd9e986 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:17.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.326+0000 7fef12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:26:17.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.326+0000 7fef12ffd640 1 -- 172.21.15.107:0/3956223961 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef2804e070 con 0x563cfac362b0 2024-04-25T00:26:17.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.327+0000 7fef12ffd640 1 -- 172.21.15.107:0/3956223961 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7fef28048860 con 0x563cfac362b0 2024-04-25T00:26:17.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.327+0000 7fef3c624740 1 -- 172.21.15.107:0/3956223961 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef2c038070 con 0x563cfac362b0 2024-04-25T00:26:17.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.327+0000 7fef12ffd640 1 -- 172.21.15.107:0/3956223961 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fef2805a050 con 0x563cfac362b0 2024-04-25T00:26:17.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.327+0000 7fef3c624740 10 client.17772 fetch_fsmap learned FSMap version 34 2024-04-25T00:26:17.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.327+0000 7fef3c624740 1 -- 172.21.15.107:0/3956223961 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563cfacbb1e0 con 0x563cfac362b0 2024-04-25T00:26:17.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.327+0000 7fef12ffd640 1 -- 172.21.15.107:0/3956223961 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef2c038070 con 0x563cfac362b0 2024-04-25T00:26:17.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.327+0000 7fef12ffd640 1 -- 172.21.15.107:0/3956223961 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fef28031840 con 0x563cfac362b0 2024-04-25T00:26:17.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.327+0000 7fef3c624740 10 client.17772 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:26:17.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.327+0000 7fef3c624740 10 client.17772 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:26:17.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.327+0000 7fef3c624740 20 client.17772 populate_metadata read hostname 'smithi107' 2024-04-25T00:26:17.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.327+0000 7fef3c624740 1 --2- 172.21.15.107:0/3956223961 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x563cfacbb910 0x563cfacbdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.327+0000 7fef3c624740 4 client.17772 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:26:17.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.327+0000 7fef3c624740 1 -- 172.21.15.107:0/3956223961 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563cfaba2f00 con 0x563cfacbb910 2024-04-25T00:26:17.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.327+0000 7fef32ffd640 1 --2- 172.21.15.107:0/3956223961 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x563cfacbb910 0x563cfacbdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.328+0000 7fef12ffd640 10 client.17772 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:26:17.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.328+0000 7fef32ffd640 1 --2- 172.21.15.107:0/3956223961 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x563cfacbb910 0x563cfacbdd00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=97112996f340b126 server_cookie=ef288ff4636781a1 in_seq=0 out_seq=0 2024-04-25T00:26:17.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.328+0000 7fef12ffd640 1 -- 172.21.15.107:0/3956223961 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563cfaba2f00 con 0x563cfacbb910 2024-04-25T00:26:17.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.328+0000 7fef12ffd640 10 client.17772 handle_command_reply: tid=0 2024-04-25T00:26:17.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.328+0000 7fef3c624740 1 client.17772 shutdown 2024-04-25T00:26:17.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.328+0000 7fef3c624740 1 -- 172.21.15.107:0/3956223961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac362b0 msgr2=0x563cfac40290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.328+0000 7fef3c624740 1 --2- 172.21.15.107:0/3956223961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac362b0 0x563cfac40290 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fef28009900 tx=0x7fef2802a9b0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.328+0000 7fef3c624740 1 -- 172.21.15.107:0/3956223961 shutdown_connections 2024-04-25T00:26:17.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.328+0000 7fef3c624740 1 --2- 172.21.15.107:0/3956223961 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x563cfacbb910 0x563cfacbdd00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.328+0000 7fef3c624740 1 --2- 172.21.15.107:0/3956223961 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x563cfac45980 0x563cfac3e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.328+0000 7fef3c624740 1 --2- 172.21.15.107:0/3956223961 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563cfac36c60 0x563cfac4db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.328+0000 7fef12ffd640 0 client.17772 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:26:17.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.328+0000 7fef3c624740 1 --2- 172.21.15.107:0/3956223961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563cfac362b0 0x563cfac40290 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.328+0000 7fef3c624740 1 -- 172.21.15.107:0/3956223961 >> 172.21.15.107:0/3956223961 conn(0x563cfac328a0 msgr2=0x563cfac3d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:17.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.328+0000 7fef3c624740 1 -- 172.21.15.107:0/3956223961 shutdown_connections 2024-04-25T00:26:17.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.328+0000 7fef3c624740 1 -- 172.21.15.107:0/3956223961 wait complete. 2024-04-25T00:26:17.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.328+0000 7fef3c624740 20 client.17772 trim_cache size 0 max 16384 2024-04-25T00:26:17.336 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:26:17.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.329+0000 7fef39914640 1 -- 172.21.15.107:0/1906622104 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fef1c0a2d00 msgr2=0x7fef1c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.329+0000 7fef39914640 1 --2- 172.21.15.107:0/1906622104 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fef1c0a2d00 0x7fef1c0a51c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fef2409ac90 tx=0x7fef2406cf50 comp rx=0 tx=0).stop 2024-04-25T00:26:17.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.329+0000 7fef39914640 1 -- 172.21.15.107:0/1906622104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef341055f0 msgr2=0x7fef3419c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.329+0000 7fef39914640 1 --2- 172.21.15.107:0/1906622104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef341055f0 0x7fef3419c9f0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fef24009fd0 tx=0x7fef2400b8e0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.329+0000 7fef39914640 1 -- 172.21.15.107:0/1906622104 shutdown_connections 2024-04-25T00:26:17.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.329+0000 7fef39914640 1 --2- 172.21.15.107:0/1906622104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef341055f0 0x7fef3419c9f0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.329+0000 7fef39914640 1 --2- 172.21.15.107:0/1906622104 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef34101770 0x7fef34198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.329+0000 7fef39914640 1 --2- 172.21.15.107:0/1906622104 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fef1c0a2d00 0x7fef1c0a51c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.329+0000 7fef39914640 1 --2- 172.21.15.107:0/1906622104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef34100db0 0x7fef34198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.329+0000 7fef39914640 1 -- 172.21.15.107:0/1906622104 >> 172.21.15.107:0/1906622104 conn(0x7fef34076c00 msgr2=0x7fef34103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:17.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.329+0000 7fef39914640 1 -- 172.21.15.107:0/1906622104 shutdown_connections 2024-04-25T00:26:17.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.329+0000 7fef39914640 1 -- 172.21.15.107:0/1906622104 wait complete. 2024-04-25T00:26:17.340 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:26:17.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.440+0000 7f6345893640 1 -- 172.21.15.107:0/2490991734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63401020d0 msgr2=0x7f634010e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.440+0000 7f6345893640 1 --2- 172.21.15.107:0/2490991734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63401020d0 0x7f634010e5e0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f6338009cd0 tx=0x7f633802a8e0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.440+0000 7f6345893640 1 -- 172.21.15.107:0/2490991734 shutdown_connections 2024-04-25T00:26:17.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.440+0000 7f6345893640 1 --2- 172.21.15.107:0/2490991734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63401020d0 0x7f634010e5e0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.440+0000 7f6345893640 1 --2- 172.21.15.107:0/2490991734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6340101710 0x7f6340101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.440+0000 7f6345893640 1 --2- 172.21.15.107:0/2490991734 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6340107720 0x7f6340107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.440+0000 7f6345893640 1 -- 172.21.15.107:0/2490991734 >> 172.21.15.107:0/2490991734 conn(0x7f6340078b90 msgr2=0x7f6340078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:17.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.440+0000 7f6345893640 1 -- 172.21.15.107:0/2490991734 shutdown_connections 2024-04-25T00:26:17.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.440+0000 7f6345893640 1 -- 172.21.15.107:0/2490991734 wait complete. 2024-04-25T00:26:17.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.441+0000 7f6345893640 1 Processor -- start 2024-04-25T00:26:17.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.441+0000 7f6345893640 1 -- start start 2024-04-25T00:26:17.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.441+0000 7f6345893640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6340101710 0x7f6340073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.441+0000 7f6345893640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63401020d0 0x7f6340073f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.441+0000 7f6345893640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6340107720 0x7f63400744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.441+0000 7f6345893640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6340111b30 con 0x7f63401020d0 2024-04-25T00:26:17.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.441+0000 7f6345893640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f63401119b0 con 0x7f6340101710 2024-04-25T00:26:17.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.441+0000 7f6345893640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f6340111cb0 con 0x7f6340107720 2024-04-25T00:26:17.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.441+0000 7f633e7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63401020d0 0x7f6340073f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.441+0000 7f633e7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63401020d0 0x7f6340073f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44098/0 (socket says 172.21.15.107:44098) 2024-04-25T00:26:17.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.441+0000 7f633e7fc640 1 -- 172.21.15.107:0/902275296 learned_addr learned my addr 172.21.15.107:0/902275296 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:17.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.441+0000 7f633effd640 1 --2- 172.21.15.107:0/902275296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6340101710 0x7f6340073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.441+0000 7f633f7fe640 1 --2- 172.21.15.107:0/902275296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6340107720 0x7f63400744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.442+0000 7f633e7fc640 1 -- 172.21.15.107:0/902275296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6340107720 msgr2=0x7f63400744c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.442+0000 7f633e7fc640 1 --2- 172.21.15.107:0/902275296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6340107720 0x7f63400744c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.442+0000 7f633e7fc640 1 -- 172.21.15.107:0/902275296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6340101710 msgr2=0x7f6340073a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.442+0000 7f633e7fc640 1 --2- 172.21.15.107:0/902275296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6340101710 0x7f6340073a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.442+0000 7f633e7fc640 1 -- 172.21.15.107:0/902275296 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63401a6530 con 0x7f63401020d0 2024-04-25T00:26:17.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.442+0000 7f633e7fc640 1 --2- 172.21.15.107:0/902275296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63401020d0 0x7f6340073f80 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f633400d950 tx=0x7f633400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:17.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.442+0000 7f6344891640 1 -- 172.21.15.107:0/902275296 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334017070 con 0x7f63401020d0 2024-04-25T00:26:17.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.442+0000 7f6344891640 1 -- 172.21.15.107:0/902275296 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6334009c30 con 0x7f63401020d0 2024-04-25T00:26:17.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.442+0000 7f6345893640 1 -- 172.21.15.107:0/902275296 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63401a6820 con 0x7f63401020d0 2024-04-25T00:26:17.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.442+0000 7f6344891640 1 -- 172.21.15.107:0/902275296 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633400f6b0 con 0x7f63401020d0 2024-04-25T00:26:17.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.442+0000 7f6345893640 1 -- 172.21.15.107:0/902275296 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63401a6d70 con 0x7f63401020d0 2024-04-25T00:26:17.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.444+0000 7f6344891640 1 -- 172.21.15.107:0/902275296 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f63340109d0 con 0x7f63401020d0 2024-04-25T00:26:17.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.444+0000 7f634859e740 1 Processor -- start 2024-04-25T00:26:17.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.444+0000 7f634859e740 1 -- start start 2024-04-25T00:26:17.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f634859e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5591305c3580 0x5591305c3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f634859e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305c3ea0 0x5591305ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f634859e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5591305ce570 0x5591305d0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f6344891640 1 --2- 172.21.15.107:0/902275296 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f63280a2d00 0x7f63280a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f634859e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559130528670 con 0x5591305c3ea0 2024-04-25T00:26:17.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f634859e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5591304d0970 con 0x5591305c3580 2024-04-25T00:26:17.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f634859e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x559130524630 con 0x5591305ce570 2024-04-25T00:26:17.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f633effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305c3ea0 0x5591305ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f633f7fe640 1 --2- 172.21.15.107:0/902275296 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f63280a2d00 0x7f63280a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f633effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305c3ea0 0x5591305ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44114/0 (socket says 172.21.15.107:44114) 2024-04-25T00:26:17.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f633effd640 1 -- 172.21.15.107:0/2919992815 learned_addr learned my addr 172.21.15.107:0/2919992815 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:17.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f633e7fc640 1 --2- 172.21.15.107:0/2919992815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5591305ce570 0x5591305d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f633f7fe640 1 --2- 172.21.15.107:0/2919992815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5591305c3580 0x5591305c3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f6344891640 1 -- 172.21.15.107:0/902275296 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f63340cf3f0 con 0x7f63401020d0 2024-04-25T00:26:17.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f633effd640 1 -- 172.21.15.107:0/2919992815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5591305ce570 msgr2=0x5591305d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:17.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f633effd640 1 --2- 172.21.15.107:0/2919992815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5591305ce570 0x5591305d0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f633effd640 1 -- 172.21.15.107:0/2919992815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5591305c3580 msgr2=0x5591305c3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:17.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f633effd640 1 --2- 172.21.15.107:0/2919992815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5591305c3580 0x5591305c3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f633effd640 1 -- 172.21.15.107:0/2919992815 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591304f4400 con 0x5591305c3ea0 2024-04-25T00:26:17.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f633e7fc640 1 --2- 172.21.15.107:0/2919992815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5591305ce570 0x5591305d0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:26:17.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f633f7fe640 1 --2- 172.21.15.107:0/902275296 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f63280a2d00 0x7f63280a51c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6338009430 tx=0x7f63380098f0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:17.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f633effd640 1 --2- 172.21.15.107:0/2919992815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305c3ea0 0x5591305ce030 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f6330009ca0 tx=0x7f6330011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11edb48252154944 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:17.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f6325ffb640 1 -- 172.21.15.107:0/2919992815 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633001e070 con 0x5591305c3ea0 2024-04-25T00:26:17.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f6325ffb640 1 -- 172.21.15.107:0/2919992815 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6330019b10 con 0x5591305c3ea0 2024-04-25T00:26:17.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.445+0000 7f6325ffb640 1 -- 172.21.15.107:0/2919992815 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633000e9f0 con 0x5591305c3ea0 2024-04-25T00:26:17.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.446+0000 7f634859e740 1 -- 172.21.15.107:0/2919992815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305c3ea0 msgr2=0x5591305ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.446+0000 7f634859e740 1 --2- 172.21.15.107:0/2919992815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305c3ea0 0x5591305ce030 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f6330009ca0 tx=0x7f6330011810 comp rx=0 tx=0).stop 2024-04-25T00:26:17.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.446+0000 7f634859e740 1 -- 172.21.15.107:0/2919992815 shutdown_connections 2024-04-25T00:26:17.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.446+0000 7f634859e740 1 --2- 172.21.15.107:0/2919992815 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5591305ce570 0x5591305d0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.446+0000 7f634859e740 1 --2- 172.21.15.107:0/2919992815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305c3ea0 0x5591305ce030 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.446+0000 7f634859e740 1 --2- 172.21.15.107:0/2919992815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5591305c3580 0x5591305c3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.446+0000 7f634859e740 1 -- 172.21.15.107:0/2919992815 >> 172.21.15.107:0/2919992815 conn(0x5591305bf8a0 msgr2=0x5591305bfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:17.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.446+0000 7f634859e740 1 -- 172.21.15.107:0/2919992815 shutdown_connections 2024-04-25T00:26:17.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.446+0000 7f634859e740 1 -- 172.21.15.107:0/2919992815 wait complete. 2024-04-25T00:26:17.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.446+0000 7f634859e740 1 -- start start 2024-04-25T00:26:17.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.447+0000 7f634859e740 1 Processor -- start 2024-04-25T00:26:17.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.447+0000 7f634859e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5591305c3580 0x5591305cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.447+0000 7f634859e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305c3ea0 0x5591305cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.447+0000 7f634859e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5591305ce570 0x5591305cb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.447+0000 7f634859e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5591305d1360 con 0x5591305c3ea0 2024-04-25T00:26:17.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.447+0000 7f634859e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5591304e4050 con 0x5591305ce570 2024-04-25T00:26:17.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.447+0000 7f634859e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5591305d14e0 con 0x5591305c3580 2024-04-25T00:26:17.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.447+0000 7f633e7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305c3ea0 0x5591305cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.447+0000 7f634859e740 10 client.0 _ll_register_callbacks cb 0x55913023bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:17.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.447+0000 7f633e7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305c3ea0 0x5591305cb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44130/0 (socket says 172.21.15.107:44130) 2024-04-25T00:26:17.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.447+0000 7f633e7fc640 1 -- 172.21.15.107:0/685959442 learned_addr learned my addr 172.21.15.107:0/685959442 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:17.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.447+0000 7f633effd640 1 --2- 172.21.15.107:0/685959442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5591305c3580 0x5591305cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.447+0000 7f633f7fe640 1 --2- 172.21.15.107:0/685959442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5591305ce570 0x5591305cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.447+0000 7f633e7fc640 1 -- 172.21.15.107:0/685959442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5591305c3580 msgr2=0x5591305cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:17.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.447+0000 7f633e7fc640 1 --2- 172.21.15.107:0/685959442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5591305c3580 0x5591305cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.447+0000 7f633e7fc640 1 -- 172.21.15.107:0/685959442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5591305ce570 msgr2=0x5591305cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:17.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.447+0000 7f633e7fc640 1 --2- 172.21.15.107:0/685959442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5591305ce570 0x5591305cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.447+0000 7f633e7fc640 1 -- 172.21.15.107:0/685959442 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591304b5dc0 con 0x5591305c3ea0 2024-04-25T00:26:17.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.447+0000 7f633e7fc640 1 --2- 172.21.15.107:0/685959442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305c3ea0 0x5591305cb220 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f6334097fd0 tx=0x7f6334010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ce4f3aa22e7d69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:17.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.447+0000 7f631effd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:26:17.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.448+0000 7f631effd640 1 -- 172.21.15.107:0/685959442 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6334020040 con 0x5591305c3ea0 2024-04-25T00:26:17.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.448+0000 7f631effd640 1 -- 172.21.15.107:0/685959442 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f633409a130 con 0x5591305c3ea0 2024-04-25T00:26:17.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.448+0000 7f634859e740 1 -- 172.21.15.107:0/685959442 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6330019b10 con 0x5591305c3ea0 2024-04-25T00:26:17.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.448+0000 7f631effd640 1 -- 172.21.15.107:0/685959442 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6334099740 con 0x5591305c3ea0 2024-04-25T00:26:17.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.448+0000 7f634859e740 10 client.17784 fetch_fsmap learned FSMap version 34 2024-04-25T00:26:17.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.448+0000 7f634859e740 1 -- 172.21.15.107:0/685959442 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55913023c3f0 con 0x5591305c3ea0 2024-04-25T00:26:17.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.448+0000 7f631effd640 1 -- 172.21.15.107:0/685959442 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6330019b10 con 0x5591305c3ea0 2024-04-25T00:26:17.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.448+0000 7f631effd640 1 -- 172.21.15.107:0/685959442 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f6334099bc0 con 0x5591305c3ea0 2024-04-25T00:26:17.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.448+0000 7f634859e740 10 client.17784 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:26:17.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.448+0000 7f634859e740 10 client.17784 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:26:17.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.448+0000 7f634859e740 20 client.17784 populate_metadata read hostname 'smithi107' 2024-04-25T00:26:17.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.448+0000 7f634859e740 1 --2- 172.21.15.107:0/685959442 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5591305de570 0x5591305e09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.448+0000 7f634859e740 4 client.17784 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:17.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.448+0000 7f634859e740 1 -- 172.21.15.107:0/685959442 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559130535bb0 con 0x5591305de570 2024-04-25T00:26:17.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.448+0000 7f633effd640 1 --2- 172.21.15.107:0/685959442 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5591305de570 0x5591305e09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.449+0000 7f633effd640 1 --2- 172.21.15.107:0/685959442 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5591305de570 0x5591305e09b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ef708785cbf8457 server_cookie=2012523aa560589b in_seq=0 out_seq=0 2024-04-25T00:26:17.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.449+0000 7f631effd640 10 client.17784 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:26:17.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.450+0000 7f631effd640 1 -- 172.21.15.107:0/685959442 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559130535bb0 con 0x5591305de570 2024-04-25T00:26:17.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.450+0000 7f631effd640 10 client.17784 handle_command_reply: tid=0 2024-04-25T00:26:17.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.450+0000 7f634859e740 1 client.17784 shutdown 2024-04-25T00:26:17.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.450+0000 7f634859e740 1 -- 172.21.15.107:0/685959442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305c3ea0 msgr2=0x5591305cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.450+0000 7f634859e740 1 --2- 172.21.15.107:0/685959442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305c3ea0 0x5591305cb220 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f6334097fd0 tx=0x7f6334010d70 comp rx=0 tx=0).stop 2024-04-25T00:26:17.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.450+0000 7f634859e740 1 -- 172.21.15.107:0/685959442 shutdown_connections 2024-04-25T00:26:17.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.450+0000 7f634859e740 1 --2- 172.21.15.107:0/685959442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5591305ce570 0x5591305cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.450+0000 7f634859e740 1 --2- 172.21.15.107:0/685959442 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5591305de570 0x5591305e09b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.450+0000 7f634859e740 1 --2- 172.21.15.107:0/685959442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305c3ea0 0x5591305cb220 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.450+0000 7f634859e740 1 --2- 172.21.15.107:0/685959442 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5591305c3580 0x5591305cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.450+0000 7f631effd640 0 client.17784 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:26:17.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.450+0000 7f634859e740 1 -- 172.21.15.107:0/685959442 >> 172.21.15.107:0/685959442 conn(0x5591305bf8a0 msgr2=0x5591305beb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:17.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.450+0000 7f634859e740 1 -- 172.21.15.107:0/685959442 shutdown_connections 2024-04-25T00:26:17.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.450+0000 7f634859e740 1 -- 172.21.15.107:0/685959442 wait complete. 2024-04-25T00:26:17.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.450+0000 7f634859e740 20 client.17784 trim_cache size 0 max 16384 2024-04-25T00:26:17.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.463+0000 7f634859e740 1 Processor -- start 2024-04-25T00:26:17.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.463+0000 7f634859e740 1 -- start start 2024-04-25T00:26:17.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.463+0000 7f634859e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305c3580 0x5591305ca380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.463+0000 7f634859e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5591305c3ea0 0x5591305cda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.463+0000 7f634859e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5591305ce570 0x5591305d6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.463+0000 7f634859e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f63401b3700 con 0x5591305c3580 2024-04-25T00:26:17.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.463+0000 7f634859e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f63401b0fd0 con 0x5591305ce570 2024-04-25T00:26:17.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.463+0000 7f634859e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f63401aee40 con 0x5591305c3ea0 2024-04-25T00:26:17.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.463+0000 7f633effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305c3580 0x5591305ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.463+0000 7f633effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305c3580 0x5591305ca380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44132/0 (socket says 172.21.15.107:44132) 2024-04-25T00:26:17.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.463+0000 7f633effd640 1 -- 172.21.15.107:0/3069892059 learned_addr learned my addr 172.21.15.107:0/3069892059 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:17.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.464+0000 7f633e7fc640 1 --2- 172.21.15.107:0/3069892059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5591305c3ea0 0x5591305cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.464+0000 7f633f7fe640 1 --2- 172.21.15.107:0/3069892059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5591305ce570 0x5591305d6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.464+0000 7f633effd640 1 -- 172.21.15.107:0/3069892059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5591305c3ea0 msgr2=0x5591305cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:17.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.464+0000 7f633effd640 1 --2- 172.21.15.107:0/3069892059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5591305c3ea0 0x5591305cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.464+0000 7f633effd640 1 -- 172.21.15.107:0/3069892059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5591305ce570 msgr2=0x5591305d6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:17.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.464+0000 7f633effd640 1 --2- 172.21.15.107:0/3069892059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5591305ce570 0x5591305d6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.464+0000 7f633effd640 1 -- 172.21.15.107:0/3069892059 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591305c3cd0 con 0x5591305c3580 2024-04-25T00:26:17.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.464+0000 7f633f7fe640 1 --2- 172.21.15.107:0/3069892059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5591305ce570 0x5591305d6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:17.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.464+0000 7f633effd640 1 --2- 172.21.15.107:0/3069892059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305c3580 0x5591305ca380 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f6330009250 tx=0x7f6330005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe1106cfa7a5b02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:17.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.464+0000 7f63257fa640 1 -- 172.21.15.107:0/3069892059 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6330029e60 con 0x5591305c3580 2024-04-25T00:26:17.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.464+0000 7f63257fa640 1 -- 172.21.15.107:0/3069892059 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f633002e070 con 0x5591305c3580 2024-04-25T00:26:17.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.464+0000 7f63257fa640 1 -- 172.21.15.107:0/3069892059 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633001e040 con 0x5591305c3580 2024-04-25T00:26:17.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.464+0000 7f634859e740 1 -- 172.21.15.107:0/3069892059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305c3580 msgr2=0x5591305ca380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.464+0000 7f634859e740 1 --2- 172.21.15.107:0/3069892059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305c3580 0x5591305ca380 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f6330009250 tx=0x7f6330005850 comp rx=0 tx=0).stop 2024-04-25T00:26:17.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.465+0000 7f634859e740 1 -- 172.21.15.107:0/3069892059 shutdown_connections 2024-04-25T00:26:17.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.465+0000 7f634859e740 1 --2- 172.21.15.107:0/3069892059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5591305ce570 0x5591305d6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.465+0000 7f634859e740 1 --2- 172.21.15.107:0/3069892059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5591305c3ea0 0x5591305cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.465+0000 7f634859e740 1 --2- 172.21.15.107:0/3069892059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305c3580 0x5591305ca380 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.465+0000 7f634859e740 1 -- 172.21.15.107:0/3069892059 >> 172.21.15.107:0/3069892059 conn(0x5591305bf8a0 msgr2=0x5591305d9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:17.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.465+0000 7f634859e740 1 -- 172.21.15.107:0/3069892059 shutdown_connections 2024-04-25T00:26:17.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.465+0000 7f634859e740 1 -- 172.21.15.107:0/3069892059 wait complete. 2024-04-25T00:26:17.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.465+0000 7f634859e740 1 -- start start 2024-04-25T00:26:17.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.465+0000 7f634859e740 1 Processor -- start 2024-04-25T00:26:17.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.465+0000 7f634859e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5591305c3580 0x5591305d48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.465+0000 7f634859e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5591305c3ea0 0x5591305d4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.465+0000 7f634859e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305ce570 0x559130643b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.465+0000 7f634859e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f63401af490 con 0x5591305ce570 2024-04-25T00:26:17.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.465+0000 7f634859e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f63401b0900 con 0x5591305c3ea0 2024-04-25T00:26:17.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.465+0000 7f634859e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f63401afb60 con 0x5591305c3580 2024-04-25T00:26:17.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.465+0000 7f634859e740 10 client.0 _ll_register_callbacks cb 0x7f63340860b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:17.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.465+0000 7f633f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305ce570 0x559130643b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.465+0000 7f633f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305ce570 0x559130643b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44148/0 (socket says 172.21.15.107:44148) 2024-04-25T00:26:17.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.465+0000 7f633f7fe640 1 -- 172.21.15.107:0/2266522323 learned_addr learned my addr 172.21.15.107:0/2266522323 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:17.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.465+0000 7f633e7fc640 1 --2- 172.21.15.107:0/2266522323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5591305c3ea0 0x5591305d4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.465+0000 7f633effd640 1 --2- 172.21.15.107:0/2266522323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5591305c3580 0x5591305d48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.466+0000 7f633f7fe640 1 -- 172.21.15.107:0/2266522323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5591305c3580 msgr2=0x5591305d48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:17.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.466+0000 7f633f7fe640 1 --2- 172.21.15.107:0/2266522323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5591305c3580 0x5591305d48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.466+0000 7f633f7fe640 1 -- 172.21.15.107:0/2266522323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5591305c3ea0 msgr2=0x5591305d4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:17.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.466+0000 7f633f7fe640 1 --2- 172.21.15.107:0/2266522323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5591305c3ea0 0x5591305d4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.466+0000 7f633f7fe640 1 -- 172.21.15.107:0/2266522323 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591303067d0 con 0x5591305ce570 2024-04-25T00:26:17.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.466+0000 7f633e7fc640 1 --2- 172.21.15.107:0/2266522323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5591305c3ea0 0x5591305d4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:26:17.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.466+0000 7f633f7fe640 1 --2- 172.21.15.107:0/2266522323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305ce570 0x559130643b00 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f6338031b80 tx=0x7f6338004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c94c7dac547e9cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:17.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.466+0000 7f631effd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:26:17.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.466+0000 7f631effd640 1 -- 172.21.15.107:0/2266522323 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63380045c0 con 0x5591305ce570 2024-04-25T00:26:17.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.466+0000 7f631effd640 1 -- 172.21.15.107:0/2266522323 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f6338004760 con 0x5591305ce570 2024-04-25T00:26:17.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.466+0000 7f634859e740 1 -- 172.21.15.107:0/2266522323 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f633002e070 con 0x5591305ce570 2024-04-25T00:26:17.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.466+0000 7f631effd640 1 -- 172.21.15.107:0/2266522323 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f63380305a0 con 0x5591305ce570 2024-04-25T00:26:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.466+0000 7f634859e740 10 client.17790 fetch_fsmap learned FSMap version 34 2024-04-25T00:26:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.466+0000 7f634859e740 1 -- 172.21.15.107:0/2266522323 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559130645130 con 0x5591305ce570 2024-04-25T00:26:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.466+0000 7f631effd640 1 -- 172.21.15.107:0/2266522323 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f633002e070 con 0x5591305ce570 2024-04-25T00:26:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.467+0000 7f631effd640 1 -- 172.21.15.107:0/2266522323 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f6338052070 con 0x5591305ce570 2024-04-25T00:26:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.467+0000 7f634859e740 10 client.17790 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:26:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.467+0000 7f634859e740 10 client.17790 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:26:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.467+0000 7f634859e740 20 client.17790 populate_metadata read hostname 'smithi107' 2024-04-25T00:26:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.467+0000 7f634859e740 1 --2- 172.21.15.107:0/2266522323 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5591306458e0 0x559130647d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.467+0000 7f634859e740 4 client.17790 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:26:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.467+0000 7f634859e740 1 -- 172.21.15.107:0/2266522323 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55913052ff00 con 0x5591306458e0 2024-04-25T00:26:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.467+0000 7f633effd640 1 --2- 172.21.15.107:0/2266522323 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5591306458e0 0x559130647d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.467+0000 7f631effd640 10 client.17790 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:26:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.467+0000 7f633effd640 1 --2- 172.21.15.107:0/2266522323 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5591306458e0 0x559130647d00 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8fac131a95867413 server_cookie=6f8bf1995b171d89 in_seq=0 out_seq=0 2024-04-25T00:26:17.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.467+0000 7f631effd640 1 -- 172.21.15.107:0/2266522323 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55913052ff00 con 0x5591306458e0 2024-04-25T00:26:17.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.467+0000 7f631effd640 10 client.17790 handle_command_reply: tid=0 2024-04-25T00:26:17.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.467+0000 7f634859e740 1 client.17790 shutdown 2024-04-25T00:26:17.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.468+0000 7f634859e740 1 -- 172.21.15.107:0/2266522323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305ce570 msgr2=0x559130643b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.468+0000 7f634859e740 1 --2- 172.21.15.107:0/2266522323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305ce570 0x559130643b00 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f6338031b80 tx=0x7f6338004a40 comp rx=0 tx=0).stop 2024-04-25T00:26:17.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.468+0000 7f634859e740 1 -- 172.21.15.107:0/2266522323 shutdown_connections 2024-04-25T00:26:17.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.468+0000 7f634859e740 1 --2- 172.21.15.107:0/2266522323 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x5591306458e0 0x559130647d00 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.468+0000 7f634859e740 1 --2- 172.21.15.107:0/2266522323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5591305ce570 0x559130643b00 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.468+0000 7f631effd640 0 client.17790 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:26:17.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.468+0000 7f634859e740 1 --2- 172.21.15.107:0/2266522323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5591305c3ea0 0x5591305d4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.468+0000 7f634859e740 1 --2- 172.21.15.107:0/2266522323 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5591305c3580 0x5591305d48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.468+0000 7f634859e740 1 -- 172.21.15.107:0/2266522323 >> 172.21.15.107:0/2266522323 conn(0x5591305bf8a0 msgr2=0x5591305ca200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:17.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.468+0000 7f634859e740 1 -- 172.21.15.107:0/2266522323 shutdown_connections 2024-04-25T00:26:17.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.468+0000 7f634859e740 1 -- 172.21.15.107:0/2266522323 wait complete. 2024-04-25T00:26:17.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.468+0000 7f634859e740 20 client.17790 trim_cache size 0 max 16384 2024-04-25T00:26:17.477 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:26:17.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.468+0000 7f6345893640 1 -- 172.21.15.107:0/902275296 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f63280a2d00 msgr2=0x7f63280a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.468+0000 7f6345893640 1 --2- 172.21.15.107:0/902275296 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f63280a2d00 0x7f63280a51c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6338009430 tx=0x7f63380098f0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.468+0000 7f6345893640 1 -- 172.21.15.107:0/902275296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63401020d0 msgr2=0x7f6340073f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.468+0000 7f6345893640 1 --2- 172.21.15.107:0/902275296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63401020d0 0x7f6340073f80 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f633400d950 tx=0x7f633400de10 comp rx=0 tx=0).stop 2024-04-25T00:26:17.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.468+0000 7f6345893640 1 -- 172.21.15.107:0/902275296 shutdown_connections 2024-04-25T00:26:17.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.468+0000 7f6345893640 1 --2- 172.21.15.107:0/902275296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6340107720 0x7f63400744c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.468+0000 7f6345893640 1 --2- 172.21.15.107:0/902275296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63401020d0 0x7f6340073f80 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.468+0000 7f6345893640 1 --2- 172.21.15.107:0/902275296 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f63280a2d00 0x7f63280a51c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.468+0000 7f6345893640 1 --2- 172.21.15.107:0/902275296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6340101710 0x7f6340073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.468+0000 7f6345893640 1 -- 172.21.15.107:0/902275296 >> 172.21.15.107:0/902275296 conn(0x7f6340078b90 msgr2=0x7f63400ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:17.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.468+0000 7f6345893640 1 -- 172.21.15.107:0/902275296 shutdown_connections 2024-04-25T00:26:17.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.468+0000 7f6345893640 1 -- 172.21.15.107:0/902275296 wait complete. 2024-04-25T00:26:17.480 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:26:17.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.580+0000 7f93dbfff640 1 -- 172.21.15.107:0/1958176458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93dc100ea0 msgr2=0x7f93dc10d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.580+0000 7f93dbfff640 1 --2- 172.21.15.107:0/1958176458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93dc100ea0 0x7f93dc10d360 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009ca0 tx=0x7f93d402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.580+0000 7f93dbfff640 1 -- 172.21.15.107:0/1958176458 shutdown_connections 2024-04-25T00:26:17.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.580+0000 7f93dbfff640 1 --2- 172.21.15.107:0/1958176458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93dc100ea0 0x7f93dc10d360 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.580+0000 7f93dbfff640 1 --2- 172.21.15.107:0/1958176458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93dc1004e0 0x7f93dc100960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.580+0000 7f93dbfff640 1 --2- 172.21.15.107:0/1958176458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93dc1065a0 0x7f93dc106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.580+0000 7f93dbfff640 1 -- 172.21.15.107:0/1958176458 >> 172.21.15.107:0/1958176458 conn(0x7f93dc0fbf90 msgr2=0x7f93dc0fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:17.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.580+0000 7f93dbfff640 1 -- 172.21.15.107:0/1958176458 shutdown_connections 2024-04-25T00:26:17.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.580+0000 7f93dbfff640 1 -- 172.21.15.107:0/1958176458 wait complete. 2024-04-25T00:26:17.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.580+0000 7f93dbfff640 1 Processor -- start 2024-04-25T00:26:17.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.581+0000 7f93dbfff640 1 -- start start 2024-04-25T00:26:17.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.581+0000 7f93dbfff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93dc1004e0 0x7f93dc197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.581+0000 7f93dbfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93dc100ea0 0x7f93dc198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.581+0000 7f93dbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93dc1065a0 0x7f93dc19c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.581+0000 7f93dbfff640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc111a00 con 0x7f93dc1065a0 2024-04-25T00:26:17.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.581+0000 7f93dbfff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc111880 con 0x7f93dc100ea0 2024-04-25T00:26:17.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.581+0000 7f93dbfff640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc111b80 con 0x7f93dc1004e0 2024-04-25T00:26:17.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.581+0000 7f93db7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93dc1065a0 0x7f93dc19c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.581+0000 7f93db7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93dc1065a0 0x7f93dc19c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44176/0 (socket says 172.21.15.107:44176) 2024-04-25T00:26:17.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.581+0000 7f93db7fe640 1 -- 172.21.15.107:0/935154597 learned_addr learned my addr 172.21.15.107:0/935154597 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:17.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.581+0000 7f93daffd640 1 --2- 172.21.15.107:0/935154597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93dc1004e0 0x7f93dc197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.581+0000 7f93d3fff640 1 --2- 172.21.15.107:0/935154597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93dc100ea0 0x7f93dc198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.581+0000 7f93db7fe640 1 -- 172.21.15.107:0/935154597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93dc1004e0 msgr2=0x7f93dc197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.581+0000 7f93db7fe640 1 --2- 172.21.15.107:0/935154597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93dc1004e0 0x7f93dc197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.581+0000 7f93db7fe640 1 -- 172.21.15.107:0/935154597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93dc100ea0 msgr2=0x7f93dc198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.581+0000 7f93db7fe640 1 --2- 172.21.15.107:0/935154597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93dc100ea0 0x7f93dc198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.581+0000 7f93db7fe640 1 -- 172.21.15.107:0/935154597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93dc19cd30 con 0x7f93dc1065a0 2024-04-25T00:26:17.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.581+0000 7f93db7fe640 1 --2- 172.21.15.107:0/935154597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93dc1065a0 0x7f93dc19c600 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f93d4006010 tx=0x7f93d4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:17.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.582+0000 7f93e0a00640 1 -- 172.21.15.107:0/935154597 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4031650 con 0x7f93dc1065a0 2024-04-25T00:26:17.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.582+0000 7f93dbfff640 1 -- 172.21.15.107:0/935154597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93dc19cfc0 con 0x7f93dc1065a0 2024-04-25T00:26:17.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.582+0000 7f93e0a00640 1 -- 172.21.15.107:0/935154597 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93d40317f0 con 0x7f93dc1065a0 2024-04-25T00:26:17.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.582+0000 7f93e0a00640 1 -- 172.21.15.107:0/935154597 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d40309c0 con 0x7f93dc1065a0 2024-04-25T00:26:17.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.582+0000 7f93dbfff640 1 -- 172.21.15.107:0/935154597 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93dc1a48b0 con 0x7f93dc1065a0 2024-04-25T00:26:17.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.584+0000 7f93e0a00640 1 -- 172.21.15.107:0/935154597 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f93d4030b60 con 0x7f93dc1065a0 2024-04-25T00:26:17.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.584+0000 7f93e47ea740 1 Processor -- start 2024-04-25T00:26:17.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.584+0000 7f93e47ea740 1 -- start start 2024-04-25T00:26:17.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.585+0000 7f93e47ea740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5607b8aec710 0x5607b8aecaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.585+0000 7f93e47ea740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8aed030 0x5607b8af71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.585+0000 7f93e47ea740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607b8af7700 0x5607b8af9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.585+0000 7f93e47ea740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5607b8a51670 con 0x5607b8aed030 2024-04-25T00:26:17.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.585+0000 7f93e47ea740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5607b89f9970 con 0x5607b8af7700 2024-04-25T00:26:17.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.585+0000 7f93e47ea740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5607b8a4d630 con 0x5607b8aec710 2024-04-25T00:26:17.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.585+0000 7f93e0a00640 1 --2- 172.21.15.107:0/935154597 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f93c80a2d00 0x7f93c80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.585+0000 7f93daffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8aed030 0x5607b8af71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.585+0000 7f93daffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8aed030 0x5607b8af71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44192/0 (socket says 172.21.15.107:44192) 2024-04-25T00:26:17.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.585+0000 7f93daffd640 1 -- 172.21.15.107:0/1636504762 learned_addr learned my addr 172.21.15.107:0/1636504762 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:17.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.585+0000 7f93db7fe640 1 --2- 172.21.15.107:0/935154597 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f93c80a2d00 0x7f93c80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.585+0000 7f93d3fff640 1 --2- 172.21.15.107:0/1636504762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5607b8aec710 0x5607b8aecaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.585+0000 7f93d3fff640 1 --2- 172.21.15.107:0/1636504762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607b8af7700 0x5607b8af9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.585+0000 7f93e0a00640 1 -- 172.21.15.107:0/935154597 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f93d40f0b20 con 0x7f93dc1065a0 2024-04-25T00:26:17.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.585+0000 7f93daffd640 1 -- 172.21.15.107:0/1636504762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5607b8aec710 msgr2=0x5607b8aecaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:17.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.585+0000 7f93daffd640 1 --2- 172.21.15.107:0/1636504762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5607b8aec710 0x5607b8aecaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.585+0000 7f93daffd640 1 -- 172.21.15.107:0/1636504762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607b8af7700 msgr2=0x5607b8af9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:17.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.585+0000 7f93daffd640 1 --2- 172.21.15.107:0/1636504762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607b8af7700 0x5607b8af9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.585+0000 7f93daffd640 1 -- 172.21.15.107:0/1636504762 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607b8a1d400 con 0x5607b8aed030 2024-04-25T00:26:17.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.585+0000 7f93db7fe640 1 --2- 172.21.15.107:0/935154597 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f93c80a2d00 0x7f93c80a51c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f93d40b91d0 tx=0x7f93d402a8c0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:17.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.585+0000 7f93daffd640 1 --2- 172.21.15.107:0/1636504762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8aed030 0x5607b8af71c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f93cc00b760 tx=0x7f93cc00bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a126d30b75fb8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:17.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.585+0000 7f93d1ffb640 1 -- 172.21.15.107:0/1636504762 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc003dd0 con 0x5607b8aed030 2024-04-25T00:26:17.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.586+0000 7f93d1ffb640 1 -- 172.21.15.107:0/1636504762 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93cc015070 con 0x5607b8aed030 2024-04-25T00:26:17.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.586+0000 7f93d1ffb640 1 -- 172.21.15.107:0/1636504762 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93cc004e10 con 0x5607b8aed030 2024-04-25T00:26:17.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.586+0000 7f93e47ea740 1 -- 172.21.15.107:0/1636504762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8aed030 msgr2=0x5607b8af71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.586+0000 7f93e47ea740 1 --2- 172.21.15.107:0/1636504762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8aed030 0x5607b8af71c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f93cc00b760 tx=0x7f93cc00bc20 comp rx=0 tx=0).stop 2024-04-25T00:26:17.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.586+0000 7f93e47ea740 1 -- 172.21.15.107:0/1636504762 shutdown_connections 2024-04-25T00:26:17.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.586+0000 7f93e47ea740 1 --2- 172.21.15.107:0/1636504762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607b8af7700 0x5607b8af9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.586+0000 7f93e47ea740 1 --2- 172.21.15.107:0/1636504762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8aed030 0x5607b8af71c0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.586+0000 7f93e47ea740 1 --2- 172.21.15.107:0/1636504762 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5607b8aec710 0x5607b8aecaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.586+0000 7f93e47ea740 1 -- 172.21.15.107:0/1636504762 >> 172.21.15.107:0/1636504762 conn(0x5607b8ae8a30 msgr2=0x5607b8ae8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:17.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.586+0000 7f93e47ea740 1 -- 172.21.15.107:0/1636504762 shutdown_connections 2024-04-25T00:26:17.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.586+0000 7f93e47ea740 1 -- 172.21.15.107:0/1636504762 wait complete. 2024-04-25T00:26:17.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.586+0000 7f93e47ea740 1 -- start start 2024-04-25T00:26:17.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.587+0000 7f93e47ea740 1 Processor -- start 2024-04-25T00:26:17.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.587+0000 7f93e47ea740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8aec710 0x5607b8af7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.587+0000 7f93e47ea740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607b8aed030 0x5607b8af4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.587+0000 7f93e47ea740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5607b8af7700 0x5607b8af48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.587+0000 7f93e47ea740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5607b8afa4f0 con 0x5607b8aec710 2024-04-25T00:26:17.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.587+0000 7f93e47ea740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5607b8a0d050 con 0x5607b8aed030 2024-04-25T00:26:17.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.587+0000 7f93e47ea740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5607b8afa670 con 0x5607b8af7700 2024-04-25T00:26:17.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.587+0000 7f93e47ea740 10 client.0 _ll_register_callbacks cb 0x5607b8764be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:17.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.587+0000 7f93d3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8aec710 0x5607b8af7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.587+0000 7f93d3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8aec710 0x5607b8af7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44200/0 (socket says 172.21.15.107:44200) 2024-04-25T00:26:17.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.587+0000 7f93d3fff640 1 -- 172.21.15.107:0/3646212922 learned_addr learned my addr 172.21.15.107:0/3646212922 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:17.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.587+0000 7f93d3fff640 1 -- 172.21.15.107:0/3646212922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5607b8af7700 msgr2=0x5607b8af48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:17.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.587+0000 7f93d3fff640 1 --2- 172.21.15.107:0/3646212922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5607b8af7700 0x5607b8af48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.587+0000 7f93d3fff640 1 -- 172.21.15.107:0/3646212922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607b8aed030 msgr2=0x5607b8af4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:17.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.587+0000 7f93daffd640 1 --2- 172.21.15.107:0/3646212922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607b8aed030 0x5607b8af4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.587+0000 7f93d3fff640 1 --2- 172.21.15.107:0/3646212922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607b8aed030 0x5607b8af4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.587+0000 7f93d3fff640 1 -- 172.21.15.107:0/3646212922 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607b89dedc0 con 0x5607b8aec710 2024-04-25T00:26:17.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.588+0000 7f93d3fff640 1 --2- 172.21.15.107:0/3646212922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8aec710 0x5607b8af7050 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f93c400a5c0 tx=0x7f93c400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27dc566945e96189 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:17.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.588+0000 7f93baffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:26:17.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.588+0000 7f93baffd640 1 -- 172.21.15.107:0/3646212922 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4005220 con 0x5607b8aec710 2024-04-25T00:26:17.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.588+0000 7f93baffd640 1 -- 172.21.15.107:0/3646212922 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f93c40053c0 con 0x5607b8aec710 2024-04-25T00:26:17.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.588+0000 7f93e47ea740 1 -- 172.21.15.107:0/3646212922 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93cc015070 con 0x5607b8aec710 2024-04-25T00:26:17.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.588+0000 7f93baffd640 1 -- 172.21.15.107:0/3646212922 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f93c40055c0 con 0x5607b8aec710 2024-04-25T00:26:17.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.588+0000 7f93e47ea740 10 client.17802 fetch_fsmap learned FSMap version 34 2024-04-25T00:26:17.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.588+0000 7f93e47ea740 1 -- 172.21.15.107:0/3646212922 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5607b87653f0 con 0x5607b8aec710 2024-04-25T00:26:17.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.588+0000 7f93baffd640 1 -- 172.21.15.107:0/3646212922 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93cc015070 con 0x5607b8aec710 2024-04-25T00:26:17.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.588+0000 7f93baffd640 1 -- 172.21.15.107:0/3646212922 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f93c401b070 con 0x5607b8aec710 2024-04-25T00:26:17.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.588+0000 7f93e47ea740 10 client.17802 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:26:17.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.588+0000 7f93e47ea740 10 client.17802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:17.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.588+0000 7f93e47ea740 20 client.17802 populate_metadata read hostname 'smithi107' 2024-04-25T00:26:17.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.588+0000 7f93e47ea740 1 --2- 172.21.15.107:0/3646212922 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5607b8b07810 0x5607b8b09cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.588+0000 7f93e47ea740 4 client.17802 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:17.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.588+0000 7f93e47ea740 1 -- 172.21.15.107:0/3646212922 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607b8a5ebb0 con 0x5607b8b07810 2024-04-25T00:26:17.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.589+0000 7f93daffd640 1 --2- 172.21.15.107:0/3646212922 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5607b8b07810 0x5607b8b09cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.589+0000 7f93baffd640 10 client.17802 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:26:17.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.589+0000 7f93daffd640 1 --2- 172.21.15.107:0/3646212922 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5607b8b07810 0x5607b8b09cb0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c23bdb9ec6d7178 server_cookie=8016ffae4b7e413a in_seq=0 out_seq=0 2024-04-25T00:26:17.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.590+0000 7f93baffd640 1 -- 172.21.15.107:0/3646212922 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5607b8a5ebb0 con 0x5607b8b07810 2024-04-25T00:26:17.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.590+0000 7f93baffd640 10 client.17802 handle_command_reply: tid=0 2024-04-25T00:26:17.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.590+0000 7f93e47ea740 1 client.17802 shutdown 2024-04-25T00:26:17.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.590+0000 7f93e47ea740 1 -- 172.21.15.107:0/3646212922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8aec710 msgr2=0x5607b8af7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.590+0000 7f93e47ea740 1 --2- 172.21.15.107:0/3646212922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8aec710 0x5607b8af7050 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f93c400a5c0 tx=0x7f93c400f7f0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.590+0000 7f93e47ea740 1 -- 172.21.15.107:0/3646212922 shutdown_connections 2024-04-25T00:26:17.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.590+0000 7f93e47ea740 1 --2- 172.21.15.107:0/3646212922 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5607b8b07810 0x5607b8b09cb0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.590+0000 7f93e47ea740 1 --2- 172.21.15.107:0/3646212922 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5607b8af7700 0x5607b8af48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.590+0000 7f93e47ea740 1 --2- 172.21.15.107:0/3646212922 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607b8aed030 0x5607b8af4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.590+0000 7f93baffd640 0 client.17802 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:26:17.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.590+0000 7f93e47ea740 1 --2- 172.21.15.107:0/3646212922 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8aec710 0x5607b8af7050 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.590+0000 7f93e47ea740 1 -- 172.21.15.107:0/3646212922 >> 172.21.15.107:0/3646212922 conn(0x5607b8ae8a30 msgr2=0x5607b8aee730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:17.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.590+0000 7f93e47ea740 1 -- 172.21.15.107:0/3646212922 shutdown_connections 2024-04-25T00:26:17.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.590+0000 7f93e47ea740 1 -- 172.21.15.107:0/3646212922 wait complete. 2024-04-25T00:26:17.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.590+0000 7f93e47ea740 20 client.17802 trim_cache size 0 max 16384 2024-04-25T00:26:17.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.603+0000 7f93e47ea740 1 Processor -- start 2024-04-25T00:26:17.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.603+0000 7f93e47ea740 1 -- start start 2024-04-25T00:26:17.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.603+0000 7f93e47ea740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5607b8aec710 0x5607b8b03410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.603+0000 7f93e47ea740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607b8aed030 0x5607b8ae8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.603+0000 7f93e47ea740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8af7700 0x5607b8af6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.603+0000 7f93e47ea740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1b1140 con 0x5607b8af7700 2024-04-25T00:26:17.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.603+0000 7f93e47ea740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1aea10 con 0x5607b8aed030 2024-04-25T00:26:17.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.603+0000 7f93e47ea740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1ac880 con 0x5607b8aec710 2024-04-25T00:26:17.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.603+0000 7f93d3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8af7700 0x5607b8af6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.603+0000 7f93d3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8af7700 0x5607b8af6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44206/0 (socket says 172.21.15.107:44206) 2024-04-25T00:26:17.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.603+0000 7f93d3fff640 1 -- 172.21.15.107:0/972139205 learned_addr learned my addr 172.21.15.107:0/972139205 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:17.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.604+0000 7f93d3fff640 1 --2- 172.21.15.107:0/972139205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5607b8aec710 0x5607b8b03410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.604+0000 7f93d3fff640 1 -- 172.21.15.107:0/972139205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5607b8aec710 msgr2=0x5607b8b03410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:17.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.604+0000 7f93d3fff640 1 --2- 172.21.15.107:0/972139205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5607b8aec710 0x5607b8b03410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.604+0000 7f93d3fff640 1 -- 172.21.15.107:0/972139205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607b8aed030 msgr2=0x5607b8ae8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:17.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.604+0000 7f93d3fff640 1 --2- 172.21.15.107:0/972139205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607b8aed030 0x5607b8ae8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.604+0000 7f93d3fff640 1 -- 172.21.15.107:0/972139205 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607b8af4c10 con 0x5607b8af7700 2024-04-25T00:26:17.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.604+0000 7f93d3fff640 1 --2- 172.21.15.107:0/972139205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8af7700 0x5607b8af6b80 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f93c400a3f0 tx=0x7f93c400a380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf01c2b5bd31878 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:17.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.604+0000 7f93d17fa640 1 -- 172.21.15.107:0/972139205 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4030070 con 0x5607b8af7700 2024-04-25T00:26:17.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.604+0000 7f93d17fa640 1 -- 172.21.15.107:0/972139205 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93c401c5f0 con 0x5607b8af7700 2024-04-25T00:26:17.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.604+0000 7f93d17fa640 1 -- 172.21.15.107:0/972139205 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c402a390 con 0x5607b8af7700 2024-04-25T00:26:17.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.605+0000 7f93e47ea740 1 -- 172.21.15.107:0/972139205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8af7700 msgr2=0x5607b8af6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.605+0000 7f93e47ea740 1 --2- 172.21.15.107:0/972139205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8af7700 0x5607b8af6b80 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f93c400a3f0 tx=0x7f93c400a380 comp rx=0 tx=0).stop 2024-04-25T00:26:17.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.605+0000 7f93e47ea740 1 -- 172.21.15.107:0/972139205 shutdown_connections 2024-04-25T00:26:17.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.605+0000 7f93e47ea740 1 --2- 172.21.15.107:0/972139205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8af7700 0x5607b8af6b80 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.605+0000 7f93e47ea740 1 --2- 172.21.15.107:0/972139205 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607b8aed030 0x5607b8ae8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.605+0000 7f93e47ea740 1 --2- 172.21.15.107:0/972139205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5607b8aec710 0x5607b8b03410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.605+0000 7f93e47ea740 1 -- 172.21.15.107:0/972139205 >> 172.21.15.107:0/972139205 conn(0x5607b8ae8a30 msgr2=0x5607b8aee6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:17.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.605+0000 7f93e47ea740 1 -- 172.21.15.107:0/972139205 shutdown_connections 2024-04-25T00:26:17.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.605+0000 7f93e47ea740 1 -- 172.21.15.107:0/972139205 wait complete. 2024-04-25T00:26:17.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.605+0000 7f93e47ea740 1 -- start start 2024-04-25T00:26:17.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.605+0000 7f93e47ea740 1 Processor -- start 2024-04-25T00:26:17.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.605+0000 7f93e47ea740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607b8aec710 0x5607b8affe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.605+0000 7f93e47ea740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5607b8aed030 0x5607b8afb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.605+0000 7f93e47ea740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8af7700 0x5607b8afb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.605+0000 7f93e47ea740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1aced0 con 0x5607b8af7700 2024-04-25T00:26:17.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.605+0000 7f93e47ea740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1ae340 con 0x5607b8aec710 2024-04-25T00:26:17.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.605+0000 7f93e47ea740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f93dc1ad5a0 con 0x5607b8aed030 2024-04-25T00:26:17.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.605+0000 7f93e47ea740 10 client.0 _ll_register_callbacks cb 0x7f93c400acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:17.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.606+0000 7f93d3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8af7700 0x5607b8afb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.606+0000 7f93d3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8af7700 0x5607b8afb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44210/0 (socket says 172.21.15.107:44210) 2024-04-25T00:26:17.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.606+0000 7f93d3fff640 1 -- 172.21.15.107:0/1929603785 learned_addr learned my addr 172.21.15.107:0/1929603785 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:17.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.606+0000 7f93d3fff640 1 --2- 172.21.15.107:0/1929603785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607b8aec710 0x5607b8affe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.606+0000 7f93d3fff640 1 -- 172.21.15.107:0/1929603785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5607b8aed030 msgr2=0x5607b8afb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:17.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.606+0000 7f93daffd640 1 --2- 172.21.15.107:0/1929603785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5607b8aed030 0x5607b8afb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.606+0000 7f93d3fff640 1 --2- 172.21.15.107:0/1929603785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5607b8aed030 0x5607b8afb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.606+0000 7f93d3fff640 1 -- 172.21.15.107:0/1929603785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607b8aec710 msgr2=0x5607b8affe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:17.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.606+0000 7f93d3fff640 1 --2- 172.21.15.107:0/1929603785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607b8aec710 0x5607b8affe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.606+0000 7f93d3fff640 1 -- 172.21.15.107:0/1929603785 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607b882f7d0 con 0x5607b8af7700 2024-04-25T00:26:17.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.606+0000 7f93d3fff640 1 --2- 172.21.15.107:0/1929603785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8af7700 0x5607b8afb780 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f93c400fd70 tx=0x7f93c402baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2475d5fd5690fad4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:17.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.606+0000 7f93baffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:26:17.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.606+0000 7f93baffd640 1 -- 172.21.15.107:0/1929603785 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c403d060 con 0x5607b8af7700 2024-04-25T00:26:17.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.606+0000 7f93e47ea740 1 -- 172.21.15.107:0/1929603785 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93c401c5f0 con 0x5607b8af7700 2024-04-25T00:26:17.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.606+0000 7f93baffd640 1 -- 172.21.15.107:0/1929603785 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f93c400aac0 con 0x5607b8af7700 2024-04-25T00:26:17.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.606+0000 7f93baffd640 1 -- 172.21.15.107:0/1929603785 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f93c4044050 con 0x5607b8af7700 2024-04-25T00:26:17.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.606+0000 7f93e47ea740 10 client.17808 fetch_fsmap learned FSMap version 34 2024-04-25T00:26:17.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.606+0000 7f93e47ea740 1 -- 172.21.15.107:0/1929603785 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5607b8b6dd60 con 0x5607b8af7700 2024-04-25T00:26:17.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.606+0000 7f93baffd640 1 -- 172.21.15.107:0/1929603785 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93c401c5f0 con 0x5607b8af7700 2024-04-25T00:26:17.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.607+0000 7f93baffd640 1 -- 172.21.15.107:0/1929603785 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f93c401b070 con 0x5607b8af7700 2024-04-25T00:26:17.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.607+0000 7f93e47ea740 10 client.17808 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:26:17.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.607+0000 7f93e47ea740 10 client.17808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:17.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.607+0000 7f93e47ea740 20 client.17808 populate_metadata read hostname 'smithi107' 2024-04-25T00:26:17.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.607+0000 7f93e47ea740 1 --2- 172.21.15.107:0/1929603785 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5607b8b6e360 0x5607b8b70750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:17.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.607+0000 7f93e47ea740 4 client.17808 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:26:17.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.607+0000 7f93e47ea740 1 -- 172.21.15.107:0/1929603785 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5607b8a58f00 con 0x5607b8b6e360 2024-04-25T00:26:17.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.607+0000 7f93daffd640 1 --2- 172.21.15.107:0/1929603785 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5607b8b6e360 0x5607b8b70750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:17.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.607+0000 7f93baffd640 10 client.17808 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:26:17.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.607+0000 7f93daffd640 1 --2- 172.21.15.107:0/1929603785 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5607b8b6e360 0x5607b8b70750 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=863883de37d30d64 server_cookie=fff58d0d242c8717 in_seq=0 out_seq=0 2024-04-25T00:26:17.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.607+0000 7f93baffd640 1 -- 172.21.15.107:0/1929603785 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5607b8a58f00 con 0x5607b8b6e360 2024-04-25T00:26:17.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.607+0000 7f93baffd640 10 client.17808 handle_command_reply: tid=0 2024-04-25T00:26:17.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.607+0000 7f93e47ea740 1 client.17808 shutdown 2024-04-25T00:26:17.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.608+0000 7f93e47ea740 1 -- 172.21.15.107:0/1929603785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8af7700 msgr2=0x5607b8afb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.608+0000 7f93e47ea740 1 --2- 172.21.15.107:0/1929603785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8af7700 0x5607b8afb780 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f93c400fd70 tx=0x7f93c402baf0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.608+0000 7f93e47ea740 1 -- 172.21.15.107:0/1929603785 shutdown_connections 2024-04-25T00:26:17.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.608+0000 7f93e47ea740 1 --2- 172.21.15.107:0/1929603785 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5607b8b6e360 0x5607b8b70750 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.608+0000 7f93e47ea740 1 --2- 172.21.15.107:0/1929603785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5607b8af7700 0x5607b8afb780 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.608+0000 7f93e47ea740 1 --2- 172.21.15.107:0/1929603785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5607b8aed030 0x5607b8afb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.608+0000 7f93e47ea740 1 --2- 172.21.15.107:0/1929603785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5607b8aec710 0x5607b8affe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.608+0000 7f93baffd640 0 client.17808 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:26:17.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.608+0000 7f93e47ea740 1 -- 172.21.15.107:0/1929603785 >> 172.21.15.107:0/1929603785 conn(0x5607b8ae8a30 msgr2=0x5607b8aee6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:17.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.608+0000 7f93e47ea740 1 -- 172.21.15.107:0/1929603785 shutdown_connections 2024-04-25T00:26:17.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.608+0000 7f93e47ea740 1 -- 172.21.15.107:0/1929603785 wait complete. 2024-04-25T00:26:17.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.608+0000 7f93e47ea740 20 client.17808 trim_cache size 0 max 16384 2024-04-25T00:26:17.616 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:26:17.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.608+0000 7f93dbfff640 1 -- 172.21.15.107:0/935154597 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f93c80a2d00 msgr2=0x7f93c80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.608+0000 7f93dbfff640 1 --2- 172.21.15.107:0/935154597 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f93c80a2d00 0x7f93c80a51c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f93d40b91d0 tx=0x7f93d402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.608+0000 7f93dbfff640 1 -- 172.21.15.107:0/935154597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93dc1065a0 msgr2=0x7f93dc19c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:17.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.608+0000 7f93dbfff640 1 --2- 172.21.15.107:0/935154597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93dc1065a0 0x7f93dc19c600 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f93d4006010 tx=0x7f93d4003f10 comp rx=0 tx=0).stop 2024-04-25T00:26:17.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.608+0000 7f93dbfff640 1 -- 172.21.15.107:0/935154597 shutdown_connections 2024-04-25T00:26:17.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.608+0000 7f93dbfff640 1 --2- 172.21.15.107:0/935154597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93dc1065a0 0x7f93dc19c600 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.608+0000 7f93dbfff640 1 --2- 172.21.15.107:0/935154597 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f93c80a2d00 0x7f93c80a51c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.608+0000 7f93dbfff640 1 --2- 172.21.15.107:0/935154597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93dc100ea0 0x7f93dc198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.608+0000 7f93dbfff640 1 --2- 172.21.15.107:0/935154597 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f93dc1004e0 0x7f93dc197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:17.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.608+0000 7f93dbfff640 1 -- 172.21.15.107:0/935154597 >> 172.21.15.107:0/935154597 conn(0x7f93dc0fbf90 msgr2=0x7f93dc0fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:17.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.608+0000 7f93dbfff640 1 -- 172.21.15.107:0/935154597 shutdown_connections 2024-04-25T00:26:17.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:17.608+0000 7f93dbfff640 1 -- 172.21.15.107:0/935154597 wait complete. 2024-04-25T00:26:17.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:17 smithi107 ceph-mon[29153]: pgmap v305: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:26:17.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:17 smithi107 ceph-mon[29153]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T00:26:17.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:17 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/623967126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:26:17.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:17 smithi170 ceph-mon[32807]: pgmap v305: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:26:17.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:17 smithi170 ceph-mon[32807]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T00:26:17.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:17 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/623967126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:26:18.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:17 smithi195 ceph-mon[32833]: pgmap v305: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:26:18.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:17 smithi195 ceph-mon[32833]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T00:26:18.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:17 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/623967126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:26:18.620 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-25T00:26:18.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:26:18.620 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-25T00:26:18.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.733+0000 7fced5d12640 1 -- 172.21.15.107:0/3872829668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fced01029c0 msgr2=0x7fced0102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:18.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.733+0000 7fced5d12640 1 --2- 172.21.15.107:0/3872829668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fced01029c0 0x7fced0102da0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009900 tx=0x7fcec002a510 comp rx=0 tx=0).stop 2024-04-25T00:26:18.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.733+0000 7fced5d12640 1 -- 172.21.15.107:0/3872829668 shutdown_connections 2024-04-25T00:26:18.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.733+0000 7fced5d12640 1 --2- 172.21.15.107:0/3872829668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fced010bd00 0x7fced010e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.733+0000 7fced5d12640 1 --2- 172.21.15.107:0/3872829668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fced01032e0 0x7fced010b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.733+0000 7fced5d12640 1 --2- 172.21.15.107:0/3872829668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fced01029c0 0x7fced0102da0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.733+0000 7fced5d12640 1 -- 172.21.15.107:0/3872829668 >> 172.21.15.107:0/3872829668 conn(0x7fced0077740 msgr2=0x7fced0077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:18.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.733+0000 7fced5d12640 1 -- 172.21.15.107:0/3872829668 shutdown_connections 2024-04-25T00:26:18.736 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.733+0000 7fced5d12640 1 -- 172.21.15.107:0/3872829668 wait complete. 2024-04-25T00:26:18.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.734+0000 7fced5d12640 1 Processor -- start 2024-04-25T00:26:18.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.734+0000 7fced5d12640 1 -- start start 2024-04-25T00:26:18.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.734+0000 7fced5d12640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fced01029c0 0x7fced0198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.734+0000 7fced5d12640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fced01032e0 0x7fced01986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.734+0000 7fced5d12640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fced010bd00 0x7fced019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.734+0000 7fced5d12640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fced0111bc0 con 0x7fced01032e0 2024-04-25T00:26:18.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.734+0000 7fced5d12640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fced0111a40 con 0x7fced01029c0 2024-04-25T00:26:18.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.734+0000 7fced5d12640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fced0111d40 con 0x7fced010bd00 2024-04-25T00:26:18.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.734+0000 7fceceffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fced01032e0 0x7fced01986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.734+0000 7fceceffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fced01032e0 0x7fced01986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44234/0 (socket says 172.21.15.107:44234) 2024-04-25T00:26:18.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.734+0000 7fceceffd640 1 -- 172.21.15.107:0/3927334413 learned_addr learned my addr 172.21.15.107:0/3927334413 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:18.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.734+0000 7fcecf7fe640 1 --2- 172.21.15.107:0/3927334413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fced01029c0 0x7fced0198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.734+0000 7fcecffff640 1 --2- 172.21.15.107:0/3927334413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fced010bd00 0x7fced019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.735+0000 7fceceffd640 1 -- 172.21.15.107:0/3927334413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fced010bd00 msgr2=0x7fced019ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:18.737 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.735+0000 7fceceffd640 1 --2- 172.21.15.107:0/3927334413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fced010bd00 0x7fced019ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.735+0000 7fceceffd640 1 -- 172.21.15.107:0/3927334413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fced01029c0 msgr2=0x7fced0198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:18.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.735+0000 7fceceffd640 1 --2- 172.21.15.107:0/3927334413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fced01029c0 0x7fced0198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.735+0000 7fceceffd640 1 -- 172.21.15.107:0/3927334413 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced019d170 con 0x7fced01032e0 2024-04-25T00:26:18.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.735+0000 7fceceffd640 1 --2- 172.21.15.107:0/3927334413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fced01032e0 0x7fced01986c0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fcec400b4a0 tx=0x7fcec400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:18.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.735+0000 7fceccff9640 1 -- 172.21.15.107:0/3927334413 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4014070 con 0x7fced01032e0 2024-04-25T00:26:18.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.735+0000 7fceccff9640 1 -- 172.21.15.107:0/3927334413 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcec4010af0 con 0x7fced01032e0 2024-04-25T00:26:18.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.735+0000 7fced5d12640 1 -- 172.21.15.107:0/3927334413 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fced019d460 con 0x7fced01032e0 2024-04-25T00:26:18.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.735+0000 7fceccff9640 1 -- 172.21.15.107:0/3927334413 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4017d10 con 0x7fced01032e0 2024-04-25T00:26:18.738 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.735+0000 7fced5d12640 1 -- 172.21.15.107:0/3927334413 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fced01a4cf0 con 0x7fced01032e0 2024-04-25T00:26:18.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.738+0000 7fceccff9640 1 -- 172.21.15.107:0/3927334413 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fcec4017070 con 0x7fced01032e0 2024-04-25T00:26:18.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.738+0000 7fced8a17740 1 Processor -- start 2024-04-25T00:26:18.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.738+0000 7fced8a17740 1 -- start start 2024-04-25T00:26:18.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.738+0000 7fced8a17740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f57362e980 0x55f57362ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.738+0000 7fced8a17740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57362d290 0x55f57362d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.738+0000 7fced8a17740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f573638010 0x55f57363a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.738+0000 7fced8a17740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f5735912c0 con 0x55f573638010 2024-04-25T00:26:18.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.738+0000 7fced8a17740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f573539910 con 0x55f57362d290 2024-04-25T00:26:18.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.738+0000 7fced8a17740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f57358d280 con 0x55f57362e980 2024-04-25T00:26:18.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.738+0000 7fceceffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f573638010 0x55f57363a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.738+0000 7fceceffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f573638010 0x55f57363a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44250/0 (socket says 172.21.15.107:44250) 2024-04-25T00:26:18.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.738+0000 7fceceffd640 1 -- 172.21.15.107:0/3954236333 learned_addr learned my addr 172.21.15.107:0/3954236333 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:18.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.738+0000 7fceccff9640 1 --2- 172.21.15.107:0/3927334413 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fceb80a2d00 0x7fceb80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.738+0000 7fcecffff640 1 --2- 172.21.15.107:0/3954236333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f57362e980 0x55f57362ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.739+0000 7fcecf7fe640 1 --2- 172.21.15.107:0/3954236333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57362d290 0x55f57362d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.739+0000 7fcecffff640 1 --2- 172.21.15.107:0/3927334413 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fceb80a2d00 0x7fceb80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.739+0000 7fceceffd640 1 -- 172.21.15.107:0/3954236333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f57362e980 msgr2=0x55f57362ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:18.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.739+0000 7fceceffd640 1 --2- 172.21.15.107:0/3954236333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f57362e980 0x55f57362ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.739+0000 7fceceffd640 1 -- 172.21.15.107:0/3954236333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57362d290 msgr2=0x55f57362d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:18.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.739+0000 7fceceffd640 1 --2- 172.21.15.107:0/3954236333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57362d290 0x55f57362d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.739+0000 7fceceffd640 1 -- 172.21.15.107:0/3954236333 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f57354afb0 con 0x55f573638010 2024-04-25T00:26:18.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.739+0000 7fceccff9640 1 -- 172.21.15.107:0/3927334413 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7fcec40cea50 con 0x7fced01032e0 2024-04-25T00:26:18.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.739+0000 7fceceffd640 1 --2- 172.21.15.107:0/3954236333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f573638010 0x55f57363a400 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fcec409ad00 tx=0x7fcec40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee80ed1a6a9cdff1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:18.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.739+0000 7fcecffff640 1 --2- 172.21.15.107:0/3927334413 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fceb80a2d00 0x7fceb80a51c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcec8009fd0 tx=0x7fcec800d040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:18.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.739+0000 7fceb5ffb640 1 -- 172.21.15.107:0/3954236333 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec401e070 con 0x55f573638010 2024-04-25T00:26:18.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.739+0000 7fceb5ffb640 1 -- 172.21.15.107:0/3954236333 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcec409ad70 con 0x55f573638010 2024-04-25T00:26:18.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.739+0000 7fceb5ffb640 1 -- 172.21.15.107:0/3954236333 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec40cc060 con 0x55f573638010 2024-04-25T00:26:18.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.740+0000 7fced8a17740 1 -- 172.21.15.107:0/3954236333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f573638010 msgr2=0x55f57363a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:18.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.740+0000 7fced8a17740 1 --2- 172.21.15.107:0/3954236333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f573638010 0x55f57363a400 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fcec409ad00 tx=0x7fcec40046c0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.740+0000 7fced8a17740 1 -- 172.21.15.107:0/3954236333 shutdown_connections 2024-04-25T00:26:18.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.740+0000 7fced8a17740 1 --2- 172.21.15.107:0/3954236333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f573638010 0x55f57363a400 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.740+0000 7fced8a17740 1 --2- 172.21.15.107:0/3954236333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57362d290 0x55f57362d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.740+0000 7fced8a17740 1 --2- 172.21.15.107:0/3954236333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f57362e980 0x55f57362ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.740+0000 7fced8a17740 1 -- 172.21.15.107:0/3954236333 >> 172.21.15.107:0/3954236333 conn(0x55f5735f5950 msgr2=0x55f5736297f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:18.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.740+0000 7fced8a17740 1 -- 172.21.15.107:0/3954236333 shutdown_connections 2024-04-25T00:26:18.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.740+0000 7fced8a17740 1 -- 172.21.15.107:0/3954236333 wait complete. 2024-04-25T00:26:18.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.740+0000 7fced8a17740 1 -- start start 2024-04-25T00:26:18.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.740+0000 7fced8a17740 1 Processor -- start 2024-04-25T00:26:18.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.741+0000 7fced8a17740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57362d290 0x55f573634f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.741+0000 7fced8a17740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f57362e980 0x55f573635450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.741+0000 7fced8a17740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f573638010 0x55f573635990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.741+0000 7fced8a17740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f57363ae40 con 0x55f573638010 2024-04-25T00:26:18.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.741+0000 7fced8a17740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f57354ce90 con 0x55f57362d290 2024-04-25T00:26:18.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.741+0000 7fced8a17740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55f57363afc0 con 0x55f57362e980 2024-04-25T00:26:18.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.741+0000 7fced8a17740 10 client.0 _ll_register_callbacks cb 0x55f573588970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:18.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.741+0000 7fcecffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f573638010 0x55f573635990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.741+0000 7fcecffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f573638010 0x55f573635990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44266/0 (socket says 172.21.15.107:44266) 2024-04-25T00:26:18.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.741+0000 7fcecffff640 1 -- 172.21.15.107:0/2930258268 learned_addr learned my addr 172.21.15.107:0/2930258268 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:18.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.741+0000 7fceceffd640 1 --2- 172.21.15.107:0/2930258268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f57362e980 0x55f573635450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.741+0000 7fcecffff640 1 -- 172.21.15.107:0/2930258268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f57362e980 msgr2=0x55f573635450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:18.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.741+0000 7fcecffff640 1 --2- 172.21.15.107:0/2930258268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f57362e980 0x55f573635450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.741+0000 7fcecffff640 1 -- 172.21.15.107:0/2930258268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57362d290 msgr2=0x55f573634f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:18.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.741+0000 7fcecffff640 1 --2- 172.21.15.107:0/2930258268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57362d290 0x55f573634f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.741+0000 7fcecffff640 1 -- 172.21.15.107:0/2930258268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f57336d7e0 con 0x55f573638010 2024-04-25T00:26:18.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.741+0000 7fcecffff640 1 --2- 172.21.15.107:0/2930258268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f573638010 0x55f573635990 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fcec8018c60 tx=0x7fcec8004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7832cab6bc6c0170 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:18.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.741+0000 7fceaeffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:26:18.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.741+0000 7fceaeffd640 1 -- 172.21.15.107:0/2930258268 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec8014040 con 0x55f573638010 2024-04-25T00:26:18.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.741+0000 7fceaeffd640 1 -- 172.21.15.107:0/2930258268 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7fcec80225f0 con 0x55f573638010 2024-04-25T00:26:18.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.741+0000 7fced8a17740 1 -- 172.21.15.107:0/2930258268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcec409ad70 con 0x55f573638010 2024-04-25T00:26:18.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.742+0000 7fceaeffd640 1 -- 172.21.15.107:0/2930258268 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcec8030050 con 0x55f573638010 2024-04-25T00:26:18.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.742+0000 7fced8a17740 10 client.17820 fetch_fsmap learned FSMap version 34 2024-04-25T00:26:18.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.742+0000 7fced8a17740 1 -- 172.21.15.107:0/2930258268 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f5733cdf70 con 0x55f573638010 2024-04-25T00:26:18.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.742+0000 7fceaeffd640 1 -- 172.21.15.107:0/2930258268 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcec409ad70 con 0x55f573638010 2024-04-25T00:26:18.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.742+0000 7fceaeffd640 1 -- 172.21.15.107:0/2930258268 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fcec802b6b0 con 0x55f573638010 2024-04-25T00:26:18.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.742+0000 7fced8a17740 10 client.17820 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:26:18.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.742+0000 7fced8a17740 10 client.17820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:18.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.742+0000 7fced8a17740 20 client.17820 populate_metadata read hostname 'smithi107' 2024-04-25T00:26:18.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.742+0000 7fced8a17740 1 --2- 172.21.15.107:0/2930258268 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55f5736480e0 0x55f57364a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.742+0000 7fced8a17740 4 client.17820 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:18.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.742+0000 7fced8a17740 1 -- 172.21.15.107:0/2930258268 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5735655a0 con 0x55f5736480e0 2024-04-25T00:26:18.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.742+0000 7fcecf7fe640 1 --2- 172.21.15.107:0/2930258268 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55f5736480e0 0x55f57364a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.743+0000 7fceaeffd640 10 client.17820 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:26:18.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.743+0000 7fcecf7fe640 1 --2- 172.21.15.107:0/2930258268 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55f5736480e0 0x55f57364a5a0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14658ff05e040398 server_cookie=6be9cd9ad5cd76be in_seq=0 out_seq=0 2024-04-25T00:26:18.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.743+0000 7fceaeffd640 1 -- 172.21.15.107:0/2930258268 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f5735655a0 con 0x55f5736480e0 2024-04-25T00:26:18.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.743+0000 7fceaeffd640 10 client.17820 handle_command_reply: tid=0 2024-04-25T00:26:18.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.743+0000 7fced8a17740 1 client.17820 shutdown 2024-04-25T00:26:18.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.744+0000 7fced8a17740 1 -- 172.21.15.107:0/2930258268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f573638010 msgr2=0x55f573635990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:18.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.744+0000 7fced8a17740 1 --2- 172.21.15.107:0/2930258268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f573638010 0x55f573635990 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fcec8018c60 tx=0x7fcec8004b50 comp rx=0 tx=0).stop 2024-04-25T00:26:18.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.744+0000 7fced8a17740 1 -- 172.21.15.107:0/2930258268 shutdown_connections 2024-04-25T00:26:18.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.744+0000 7fced8a17740 1 --2- 172.21.15.107:0/2930258268 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55f5736480e0 0x55f57364a5a0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.744+0000 7fced8a17740 1 --2- 172.21.15.107:0/2930258268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f573638010 0x55f573635990 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.744+0000 7fced8a17740 1 --2- 172.21.15.107:0/2930258268 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f57362e980 0x55f573635450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.744+0000 7fceaeffd640 0 client.17820 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:26:18.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.744+0000 7fced8a17740 1 --2- 172.21.15.107:0/2930258268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57362d290 0x55f573634f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.744+0000 7fced8a17740 1 -- 172.21.15.107:0/2930258268 >> 172.21.15.107:0/2930258268 conn(0x55f5735f5950 msgr2=0x55f57362f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:18.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.744+0000 7fced8a17740 1 -- 172.21.15.107:0/2930258268 shutdown_connections 2024-04-25T00:26:18.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.744+0000 7fced8a17740 1 -- 172.21.15.107:0/2930258268 wait complete. 2024-04-25T00:26:18.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.744+0000 7fced8a17740 20 client.17820 trim_cache size 0 max 16384 2024-04-25T00:26:18.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.761+0000 7fced8a17740 1 Processor -- start 2024-04-25T00:26:18.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.761+0000 7fced8a17740 1 -- start start 2024-04-25T00:26:18.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.761+0000 7fced8a17740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57362d290 0x55f573643d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.761+0000 7fced8a17740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f57362e980 0x55f5736287d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.761+0000 7fced8a17740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f573638010 0x55f57364fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.761+0000 7fced8a17740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fced01b16f0 con 0x55f573638010 2024-04-25T00:26:18.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.761+0000 7fced8a17740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fced01aefc0 con 0x55f57362d290 2024-04-25T00:26:18.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.761+0000 7fced8a17740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fced01ace30 con 0x55f57362e980 2024-04-25T00:26:18.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.761+0000 7fcecffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f573638010 0x55f57364fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.762+0000 7fcecffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f573638010 0x55f57364fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44278/0 (socket says 172.21.15.107:44278) 2024-04-25T00:26:18.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.762+0000 7fcecffff640 1 -- 172.21.15.107:0/848559874 learned_addr learned my addr 172.21.15.107:0/848559874 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:18.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.762+0000 7fcecf7fe640 1 --2- 172.21.15.107:0/848559874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57362d290 0x55f573643d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.762+0000 7fcecffff640 1 -- 172.21.15.107:0/848559874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f57362e980 msgr2=0x55f5736287d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:18.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.762+0000 7fcecffff640 1 --2- 172.21.15.107:0/848559874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f57362e980 0x55f5736287d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.762+0000 7fcecffff640 1 -- 172.21.15.107:0/848559874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57362d290 msgr2=0x55f573643d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:18.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.762+0000 7fcecffff640 1 --2- 172.21.15.107:0/848559874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57362d290 0x55f573643d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.762+0000 7fcecffff640 1 -- 172.21.15.107:0/848559874 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f573633ec0 con 0x55f573638010 2024-04-25T00:26:18.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.762+0000 7fcecffff640 1 --2- 172.21.15.107:0/848559874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f573638010 0x55f57364fec0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fcec800af80 tx=0x7fcec8009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccfae412233e85f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:18.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.762+0000 7fceb57fa640 1 -- 172.21.15.107:0/848559874 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec8018810 con 0x55f573638010 2024-04-25T00:26:18.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.762+0000 7fceb57fa640 1 -- 172.21.15.107:0/848559874 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcec8027070 con 0x55f573638010 2024-04-25T00:26:18.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.762+0000 7fceb57fa640 1 -- 172.21.15.107:0/848559874 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec8034c70 con 0x55f573638010 2024-04-25T00:26:18.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.763+0000 7fced8a17740 1 -- 172.21.15.107:0/848559874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f573638010 msgr2=0x55f57364fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:18.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.763+0000 7fced8a17740 1 --2- 172.21.15.107:0/848559874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f573638010 0x55f57364fec0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fcec800af80 tx=0x7fcec8009d60 comp rx=0 tx=0).stop 2024-04-25T00:26:18.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.763+0000 7fced8a17740 1 -- 172.21.15.107:0/848559874 shutdown_connections 2024-04-25T00:26:18.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.763+0000 7fced8a17740 1 --2- 172.21.15.107:0/848559874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f573638010 0x55f57364fec0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.763+0000 7fced8a17740 1 --2- 172.21.15.107:0/848559874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f57362e980 0x55f5736287d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.763+0000 7fced8a17740 1 --2- 172.21.15.107:0/848559874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f57362d290 0x55f573643d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.763+0000 7fced8a17740 1 -- 172.21.15.107:0/848559874 >> 172.21.15.107:0/848559874 conn(0x55f5735f5950 msgr2=0x55f5736280c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:18.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.763+0000 7fced8a17740 1 -- 172.21.15.107:0/848559874 shutdown_connections 2024-04-25T00:26:18.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.763+0000 7fced8a17740 1 -- 172.21.15.107:0/848559874 wait complete. 2024-04-25T00:26:18.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.763+0000 7fced8a17740 1 -- start start 2024-04-25T00:26:18.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.763+0000 7fced8a17740 1 Processor -- start 2024-04-25T00:26:18.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.763+0000 7fced8a17740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f57362d290 0x55f573640450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.763+0000 7fced8a17740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f57362e980 0x55f57363b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.770 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.763+0000 7fced8a17740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f573638010 0x55f57363b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.763+0000 7fced8a17740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fced01ad480 con 0x55f57362d290 2024-04-25T00:26:18.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.763+0000 7fced8a17740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fced01ae8f0 con 0x55f573638010 2024-04-25T00:26:18.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.763+0000 7fced8a17740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fced01adb50 con 0x55f57362e980 2024-04-25T00:26:18.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.763+0000 7fced8a17740 10 client.0 _ll_register_callbacks cb 0x7fcec8009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:18.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.763+0000 7fcecf7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f57362d290 0x55f573640450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.763+0000 7fcecf7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f57362d290 0x55f573640450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44284/0 (socket says 172.21.15.107:44284) 2024-04-25T00:26:18.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.763+0000 7fcecf7fe640 1 -- 172.21.15.107:0/4272115469 learned_addr learned my addr 172.21.15.107:0/4272115469 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:18.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.764+0000 7fcecffff640 1 --2- 172.21.15.107:0/4272115469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f573638010 0x55f57363b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.764+0000 7fcecf7fe640 1 -- 172.21.15.107:0/4272115469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f57362e980 msgr2=0x55f57363b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:18.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.764+0000 7fcecf7fe640 1 --2- 172.21.15.107:0/4272115469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f57362e980 0x55f57363b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.764+0000 7fcecf7fe640 1 -- 172.21.15.107:0/4272115469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f573638010 msgr2=0x55f57363b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:18.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.764+0000 7fcecf7fe640 1 --2- 172.21.15.107:0/4272115469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f573638010 0x55f57363b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.764+0000 7fcecf7fe640 1 -- 172.21.15.107:0/4272115469 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f573265170 con 0x55f57362d290 2024-04-25T00:26:18.771 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.764+0000 7fcecffff640 1 --2- 172.21.15.107:0/4272115469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f573638010 0x55f57363b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:18.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.764+0000 7fcecf7fe640 1 --2- 172.21.15.107:0/4272115469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f57362d290 0x55f573640450 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009900 tx=0x7fcec002a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c08fd60d443981a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:18.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.764+0000 7fceaeffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:26:18.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.764+0000 7fceaeffd640 1 -- 172.21.15.107:0/4272115469 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec0047070 con 0x55f57362d290 2024-04-25T00:26:18.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.764+0000 7fceaeffd640 1 -- 172.21.15.107:0/4272115469 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7fcec0043d50 con 0x55f57362d290 2024-04-25T00:26:18.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.764+0000 7fced8a17740 1 -- 172.21.15.107:0/4272115469 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcec8027070 con 0x55f57362d290 2024-04-25T00:26:18.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.764+0000 7fceaeffd640 1 -- 172.21.15.107:0/4272115469 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcec0052050 con 0x55f57362d290 2024-04-25T00:26:18.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.764+0000 7fced8a17740 10 client.17826 fetch_fsmap learned FSMap version 34 2024-04-25T00:26:18.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.764+0000 7fced8a17740 1 -- 172.21.15.107:0/4272115469 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f5736af390 con 0x55f57362d290 2024-04-25T00:26:18.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.764+0000 7fceaeffd640 1 -- 172.21.15.107:0/4272115469 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcec8027070 con 0x55f57362d290 2024-04-25T00:26:18.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.764+0000 7fceaeffd640 1 -- 172.21.15.107:0/4272115469 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fcec00035f0 con 0x55f57362d290 2024-04-25T00:26:18.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.765+0000 7fced8a17740 10 client.17826 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:26:18.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.765+0000 7fced8a17740 10 client.17826 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:18.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.765+0000 7fced8a17740 20 client.17826 populate_metadata read hostname 'smithi107' 2024-04-25T00:26:18.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.765+0000 7fced8a17740 1 --2- 172.21.15.107:0/4272115469 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55f5736afaf0 0x55f5736b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.772 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.765+0000 7fced8a17740 4 client.17826 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-25T00:26:18.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.765+0000 7fced8a17740 1 -- 172.21.15.107:0/4272115469 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f57359e800 con 0x55f5736afaf0 2024-04-25T00:26:18.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.765+0000 7fceceffd640 1 --2- 172.21.15.107:0/4272115469 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55f5736afaf0 0x55f5736b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.765+0000 7fceceffd640 1 --2- 172.21.15.107:0/4272115469 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55f5736afaf0 0x55f5736b1f10 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40b1705b7fca95e8 server_cookie=6c75d0b223db20f3 in_seq=0 out_seq=0 2024-04-25T00:26:18.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.765+0000 7fceaeffd640 10 client.17826 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:26:18.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.766+0000 7fceaeffd640 1 -- 172.21.15.107:0/4272115469 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f57359e800 con 0x55f5736afaf0 2024-04-25T00:26:18.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.766+0000 7fceaeffd640 10 client.17826 handle_command_reply: tid=0 2024-04-25T00:26:18.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.766+0000 7fced8a17740 1 client.17826 shutdown 2024-04-25T00:26:18.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.767+0000 7fced8a17740 1 -- 172.21.15.107:0/4272115469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f57362d290 msgr2=0x55f573640450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:18.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.767+0000 7fced8a17740 1 --2- 172.21.15.107:0/4272115469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f57362d290 0x55f573640450 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009900 tx=0x7fcec002a510 comp rx=0 tx=0).stop 2024-04-25T00:26:18.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.767+0000 7fced8a17740 1 -- 172.21.15.107:0/4272115469 shutdown_connections 2024-04-25T00:26:18.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.767+0000 7fced8a17740 1 --2- 172.21.15.107:0/4272115469 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55f5736afaf0 0x55f5736b1f10 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.767+0000 7fced8a17740 1 --2- 172.21.15.107:0/4272115469 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f573638010 0x55f57363b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.767+0000 7fced8a17740 1 --2- 172.21.15.107:0/4272115469 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55f57362e980 0x55f57363b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.767+0000 7fced8a17740 1 --2- 172.21.15.107:0/4272115469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f57362d290 0x55f573640450 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.767+0000 7fced8a17740 1 -- 172.21.15.107:0/4272115469 >> 172.21.15.107:0/4272115469 conn(0x55f5735f5950 msgr2=0x55f573628950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:18.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.767+0000 7fceaeffd640 0 client.17826 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:26:18.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.767+0000 7fced8a17740 1 -- 172.21.15.107:0/4272115469 shutdown_connections 2024-04-25T00:26:18.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.767+0000 7fced8a17740 1 -- 172.21.15.107:0/4272115469 wait complete. 2024-04-25T00:26:18.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.767+0000 7fced8a17740 20 client.17826 trim_cache size 0 max 16384 2024-04-25T00:26:18.774 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:26:18.774 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-04-25T00:26:18.774 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "0a6868ef-4d56-4d25-882c-e115a13cab96", 2024-04-25T00:26:18.774 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-04-25T00:26:18.774 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:26:18.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.767+0000 7fced5d12640 1 -- 172.21.15.107:0/3927334413 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fceb80a2d00 msgr2=0x7fceb80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:18.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.767+0000 7fced5d12640 1 --2- 172.21.15.107:0/3927334413 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fceb80a2d00 0x7fceb80a51c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcec8009fd0 tx=0x7fcec800d040 comp rx=0 tx=0).stop 2024-04-25T00:26:18.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.767+0000 7fced5d12640 1 -- 172.21.15.107:0/3927334413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fced01032e0 msgr2=0x7fced01986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:18.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.767+0000 7fced5d12640 1 --2- 172.21.15.107:0/3927334413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fced01032e0 0x7fced01986c0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fcec400b4a0 tx=0x7fcec400b960 comp rx=0 tx=0).stop 2024-04-25T00:26:18.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.767+0000 7fced5d12640 1 -- 172.21.15.107:0/3927334413 shutdown_connections 2024-04-25T00:26:18.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.767+0000 7fced5d12640 1 --2- 172.21.15.107:0/3927334413 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fced010bd00 0x7fced019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.767+0000 7fced5d12640 1 --2- 172.21.15.107:0/3927334413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fced01032e0 0x7fced01986c0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.767+0000 7fced5d12640 1 --2- 172.21.15.107:0/3927334413 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fceb80a2d00 0x7fceb80a51c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.767+0000 7fced5d12640 1 --2- 172.21.15.107:0/3927334413 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fced01029c0 0x7fced0198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.767+0000 7fced5d12640 1 -- 172.21.15.107:0/3927334413 >> 172.21.15.107:0/3927334413 conn(0x7fced0077740 msgr2=0x7fced00fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:18.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.768+0000 7fced5d12640 1 -- 172.21.15.107:0/3927334413 shutdown_connections 2024-04-25T00:26:18.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.768+0000 7fced5d12640 1 -- 172.21.15.107:0/3927334413 wait complete. 2024-04-25T00:26:18.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:26:18.779 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:26:18.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.879+0000 7f9c44ba5640 1 -- 172.21.15.107:0/4095318787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c401066b0 msgr2=0x7f9c40106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:18.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.879+0000 7f9c44ba5640 1 --2- 172.21.15.107:0/4095318787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c401066b0 0x7f9c40106a90 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f9c30009930 tx=0x7f9c3002a420 comp rx=0 tx=0).stop 2024-04-25T00:26:18.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.879+0000 7f9c44ba5640 1 -- 172.21.15.107:0/4095318787 shutdown_connections 2024-04-25T00:26:18.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.879+0000 7f9c44ba5640 1 --2- 172.21.15.107:0/4095318787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c40100fd0 0x7f9c4010d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.879+0000 7f9c44ba5640 1 --2- 172.21.15.107:0/4095318787 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c40100610 0x7f9c40100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.879+0000 7f9c44ba5640 1 --2- 172.21.15.107:0/4095318787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c401066b0 0x7f9c40106a90 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.879+0000 7f9c44ba5640 1 -- 172.21.15.107:0/4095318787 >> 172.21.15.107:0/4095318787 conn(0x7f9c40077740 msgr2=0x7f9c40077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:18.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.879+0000 7f9c44ba5640 1 -- 172.21.15.107:0/4095318787 shutdown_connections 2024-04-25T00:26:18.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.879+0000 7f9c44ba5640 1 -- 172.21.15.107:0/4095318787 wait complete. 2024-04-25T00:26:18.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.879+0000 7f9c44ba5640 1 Processor -- start 2024-04-25T00:26:18.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.879+0000 7f9c44ba5640 1 -- start start 2024-04-25T00:26:18.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.880+0000 7f9c44ba5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c40100610 0x7f9c40197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.880+0000 7f9c44ba5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c40100fd0 0x7f9c401984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.880+0000 7f9c44ba5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c401066b0 0x7f9c4019c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.880+0000 7f9c44ba5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9c40110a30 con 0x7f9c40100fd0 2024-04-25T00:26:18.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.880+0000 7f9c44ba5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9c401108b0 con 0x7f9c401066b0 2024-04-25T00:26:18.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.880+0000 7f9c44ba5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9c40110bb0 con 0x7f9c40100610 2024-04-25T00:26:18.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.880+0000 7f9c3dd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c40100fd0 0x7f9c401984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.880+0000 7f9c3dd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c40100fd0 0x7f9c401984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44316/0 (socket says 172.21.15.107:44316) 2024-04-25T00:26:18.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.880+0000 7f9c3dd9b640 1 -- 172.21.15.107:0/2739527040 learned_addr learned my addr 172.21.15.107:0/2739527040 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:18.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.880+0000 7f9c3e59c640 1 --2- 172.21.15.107:0/2739527040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c40100610 0x7f9c40197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.880+0000 7f9c3ed9d640 1 --2- 172.21.15.107:0/2739527040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c401066b0 0x7f9c4019c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.880+0000 7f9c3dd9b640 1 -- 172.21.15.107:0/2739527040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c40100610 msgr2=0x7f9c40197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:18.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.880+0000 7f9c3dd9b640 1 --2- 172.21.15.107:0/2739527040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c40100610 0x7f9c40197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.880+0000 7f9c3dd9b640 1 -- 172.21.15.107:0/2739527040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c401066b0 msgr2=0x7f9c4019c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:18.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.880+0000 7f9c3dd9b640 1 --2- 172.21.15.107:0/2739527040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c401066b0 0x7f9c4019c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.880+0000 7f9c3dd9b640 1 -- 172.21.15.107:0/2739527040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c4019cfa0 con 0x7f9c40100fd0 2024-04-25T00:26:18.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.880+0000 7f9c3dd9b640 1 --2- 172.21.15.107:0/2739527040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c40100fd0 0x7f9c401984f0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400e9f0 tx=0x7f9c3400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:18.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.880+0000 7f9c2f7fe640 1 -- 172.21.15.107:0/2739527040 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3400cd80 con 0x7f9c40100fd0 2024-04-25T00:26:18.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.880+0000 7f9c2f7fe640 1 -- 172.21.15.107:0/2739527040 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c34005500 con 0x7f9c40100fd0 2024-04-25T00:26:18.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.880+0000 7f9c44ba5640 1 -- 172.21.15.107:0/2739527040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c4019d290 con 0x7f9c40100fd0 2024-04-25T00:26:18.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.880+0000 7f9c2f7fe640 1 -- 172.21.15.107:0/2739527040 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c34010690 con 0x7f9c40100fd0 2024-04-25T00:26:18.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.880+0000 7f9c44ba5640 1 -- 172.21.15.107:0/2739527040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c401a4b20 con 0x7f9c40100fd0 2024-04-25T00:26:18.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.882+0000 7f9c2f7fe640 1 -- 172.21.15.107:0/2739527040 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f9c340046c0 con 0x7f9c40100fd0 2024-04-25T00:26:18.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.883+0000 7f9c4798e740 1 Processor -- start 2024-04-25T00:26:18.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.883+0000 7f9c4798e740 1 -- start start 2024-04-25T00:26:18.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.883+0000 7f9c4798e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155b4630 0x5603155b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.883+0000 7f9c4798e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603155b4f50 0x5603155bf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.883+0000 7f9c4798e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603155bf620 0x5603155c1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.883+0000 7f9c4798e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5603155196b0 con 0x5603155b4630 2024-04-25T00:26:18.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.883+0000 7f9c4798e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5603154c1880 con 0x5603155b4f50 2024-04-25T00:26:18.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.883+0000 7f9c4798e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x560315515670 con 0x5603155bf620 2024-04-25T00:26:18.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.883+0000 7f9c3ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155b4630 0x5603155b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.883+0000 7f9c3ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155b4630 0x5603155b4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44332/0 (socket says 172.21.15.107:44332) 2024-04-25T00:26:18.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.883+0000 7f9c3ed9d640 1 -- 172.21.15.107:0/1879946257 learned_addr learned my addr 172.21.15.107:0/1879946257 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:18.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.883+0000 7f9c2f7fe640 1 --2- 172.21.15.107:0/2739527040 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f9c280a2d00 0x7f9c280a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.883+0000 7f9c3ed9d640 1 --2- 172.21.15.107:0/2739527040 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f9c280a2d00 0x7f9c280a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.883+0000 7f9c3e59c640 1 --2- 172.21.15.107:0/1879946257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603155b4f50 0x5603155bf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.884+0000 7f9c2f7fe640 1 -- 172.21.15.107:0/2739527040 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f9c340cfd40 con 0x7f9c40100fd0 2024-04-25T00:26:18.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.884+0000 7f9c3dd9b640 1 --2- 172.21.15.107:0/1879946257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603155bf620 0x5603155c1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.884+0000 7f9c3ed9d640 1 -- 172.21.15.107:0/1879946257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603155bf620 msgr2=0x5603155c1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:18.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.884+0000 7f9c3ed9d640 1 --2- 172.21.15.107:0/1879946257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603155bf620 0x5603155c1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.884+0000 7f9c3ed9d640 1 -- 172.21.15.107:0/1879946257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603155b4f50 msgr2=0x5603155bf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:18.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.884+0000 7f9c3ed9d640 1 --2- 172.21.15.107:0/1879946257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603155b4f50 0x5603155bf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.884+0000 7f9c3ed9d640 1 -- 172.21.15.107:0/1879946257 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603152f77d0 con 0x5603155b4630 2024-04-25T00:26:18.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.884+0000 7f9c3dd9b640 1 --2- 172.21.15.107:0/1879946257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603155bf620 0x5603155c1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:18.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.884+0000 7f9c3ed9d640 1 --2- 172.21.15.107:0/1879946257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155b4630 0x5603155b4a10 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f9c38012a70 tx=0x7f9c38012f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db397d9e49e9dcad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:18.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.884+0000 7f9c2cff9640 1 -- 172.21.15.107:0/1879946257 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c38004230 con 0x5603155b4630 2024-04-25T00:26:18.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.884+0000 7f9c3ed9d640 1 --2- 172.21.15.107:0/2739527040 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f9c280a2d00 0x7f9c280a51c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9c38026000 tx=0x7f9c38023000 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:18.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.884+0000 7f9c2cff9640 1 -- 172.21.15.107:0/1879946257 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c38012490 con 0x5603155b4630 2024-04-25T00:26:18.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.884+0000 7f9c2cff9640 1 -- 172.21.15.107:0/1879946257 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c38026b90 con 0x5603155b4630 2024-04-25T00:26:18.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.885+0000 7f9c4798e740 1 -- 172.21.15.107:0/1879946257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155b4630 msgr2=0x5603155b4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:18.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.885+0000 7f9c4798e740 1 --2- 172.21.15.107:0/1879946257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155b4630 0x5603155b4a10 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f9c38012a70 tx=0x7f9c38012f30 comp rx=0 tx=0).stop 2024-04-25T00:26:18.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.885+0000 7f9c4798e740 1 -- 172.21.15.107:0/1879946257 shutdown_connections 2024-04-25T00:26:18.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.885+0000 7f9c4798e740 1 --2- 172.21.15.107:0/1879946257 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603155bf620 0x5603155c1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.885+0000 7f9c4798e740 1 --2- 172.21.15.107:0/1879946257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603155b4f50 0x5603155bf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.885+0000 7f9c4798e740 1 --2- 172.21.15.107:0/1879946257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155b4630 0x5603155b4a10 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.885+0000 7f9c4798e740 1 -- 172.21.15.107:0/1879946257 >> 172.21.15.107:0/1879946257 conn(0x5603155b0950 msgr2=0x5603155b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:18.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.885+0000 7f9c4798e740 1 -- 172.21.15.107:0/1879946257 shutdown_connections 2024-04-25T00:26:18.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.885+0000 7f9c4798e740 1 -- 172.21.15.107:0/1879946257 wait complete. 2024-04-25T00:26:18.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.885+0000 7f9c4798e740 1 -- start start 2024-04-25T00:26:18.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.885+0000 7f9c4798e740 1 Processor -- start 2024-04-25T00:26:18.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.885+0000 7f9c4798e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603155b4630 0x5603155bc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.885+0000 7f9c4798e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155b4f50 0x5603155bcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.885+0000 7f9c4798e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603155bf620 0x5603155bd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.885+0000 7f9c4798e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5603155c2410 con 0x5603155b4f50 2024-04-25T00:26:18.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.885+0000 7f9c4798e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5603154d4f50 con 0x5603155bf620 2024-04-25T00:26:18.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.885+0000 7f9c4798e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x5603155c2590 con 0x5603155b4630 2024-04-25T00:26:18.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.886+0000 7f9c4798e740 10 client.0 _ll_register_callbacks cb 0x560315510000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:18.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.886+0000 7f9c3dd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155b4f50 0x5603155bcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.886+0000 7f9c3dd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155b4f50 0x5603155bcb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44334/0 (socket says 172.21.15.107:44334) 2024-04-25T00:26:18.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.886+0000 7f9c3dd9b640 1 -- 172.21.15.107:0/4167394936 learned_addr learned my addr 172.21.15.107:0/4167394936 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:18.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.886+0000 7f9c3dd9b640 1 -- 172.21.15.107:0/4167394936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603155b4630 msgr2=0x5603155bc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:18.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.886+0000 7f9c3dd9b640 1 --2- 172.21.15.107:0/4167394936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603155b4630 0x5603155bc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.886+0000 7f9c3dd9b640 1 -- 172.21.15.107:0/4167394936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603155bf620 msgr2=0x5603155bd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:18.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.886+0000 7f9c3dd9b640 1 --2- 172.21.15.107:0/4167394936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603155bf620 0x5603155bd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.886+0000 7f9c3dd9b640 1 -- 172.21.15.107:0/4167394936 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56031522d3f0 con 0x5603155b4f50 2024-04-25T00:26:18.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.886+0000 7f9c3dd9b640 1 --2- 172.21.15.107:0/4167394936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155b4f50 0x5603155bcb50 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f9c34098810 tx=0x7f9c3409b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3265c524a7f9d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:18.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.886+0000 7f9c25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:26:18.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.886+0000 7f9c25ffb640 1 -- 172.21.15.107:0/4167394936 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c3409e030 con 0x5603155b4f50 2024-04-25T00:26:18.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.886+0000 7f9c25ffb640 1 -- 172.21.15.107:0/4167394936 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f9c340ba670 con 0x5603155b4f50 2024-04-25T00:26:18.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.886+0000 7f9c4798e740 1 -- 172.21.15.107:0/4167394936 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560315526bf0 con 0x5603155b4f50 2024-04-25T00:26:18.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.886+0000 7f9c25ffb640 1 -- 172.21.15.107:0/4167394936 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9c340cd500 con 0x5603155b4f50 2024-04-25T00:26:18.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.886+0000 7f9c4798e740 10 client.17838 fetch_fsmap learned FSMap version 34 2024-04-25T00:26:18.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.886+0000 7f9c4798e740 1 -- 172.21.15.107:0/4167394936 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5603151ed0f0 con 0x5603155b4f50 2024-04-25T00:26:18.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.886+0000 7f9c25ffb640 1 -- 172.21.15.107:0/4167394936 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560315526bf0 con 0x5603155b4f50 2024-04-25T00:26:18.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.886+0000 7f9c25ffb640 1 -- 172.21.15.107:0/4167394936 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f9c340a4620 con 0x5603155b4f50 2024-04-25T00:26:18.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.887+0000 7f9c4798e740 10 client.17838 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:26:18.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.887+0000 7f9c4798e740 10 client.17838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:18.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.887+0000 7f9c4798e740 20 client.17838 populate_metadata read hostname 'smithi107' 2024-04-25T00:26:18.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.887+0000 7f9c4798e740 1 --2- 172.21.15.107:0/4167394936 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5603155cf8c0 0x5603155d1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.887+0000 7f9c4798e740 4 client.17838 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:18.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.887+0000 7f9c4798e740 1 -- 172.21.15.107:0/4167394936 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560315520f40 con 0x5603155cf8c0 2024-04-25T00:26:18.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.887+0000 7f9c3e59c640 1 --2- 172.21.15.107:0/4167394936 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5603155cf8c0 0x5603155d1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.887+0000 7f9c3e59c640 1 --2- 172.21.15.107:0/4167394936 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5603155cf8c0 0x5603155d1d50 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e58e816f4770f4b0 server_cookie=4b12ae6cc2a94477 in_seq=0 out_seq=0 2024-04-25T00:26:18.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.887+0000 7f9c25ffb640 10 client.17838 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:26:18.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.888+0000 7f9c25ffb640 1 -- 172.21.15.107:0/4167394936 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560315520f40 con 0x5603155cf8c0 2024-04-25T00:26:18.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.888+0000 7f9c25ffb640 10 client.17838 handle_command_reply: tid=0 2024-04-25T00:26:18.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.888+0000 7f9c4798e740 1 client.17838 shutdown 2024-04-25T00:26:18.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.888+0000 7f9c4798e740 1 -- 172.21.15.107:0/4167394936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155b4f50 msgr2=0x5603155bcb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:18.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.888+0000 7f9c4798e740 1 --2- 172.21.15.107:0/4167394936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155b4f50 0x5603155bcb50 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f9c34098810 tx=0x7f9c3409b860 comp rx=0 tx=0).stop 2024-04-25T00:26:18.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.888+0000 7f9c4798e740 1 -- 172.21.15.107:0/4167394936 shutdown_connections 2024-04-25T00:26:18.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.888+0000 7f9c4798e740 1 --2- 172.21.15.107:0/4167394936 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5603155cf8c0 0x5603155d1d50 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.888+0000 7f9c4798e740 1 --2- 172.21.15.107:0/4167394936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603155bf620 0x5603155bd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.888+0000 7f9c4798e740 1 --2- 172.21.15.107:0/4167394936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155b4f50 0x5603155bcb50 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.888+0000 7f9c25ffb640 0 client.17838 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:26:18.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.888+0000 7f9c4798e740 1 --2- 172.21.15.107:0/4167394936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603155b4630 0x5603155bc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.888+0000 7f9c4798e740 1 -- 172.21.15.107:0/4167394936 >> 172.21.15.107:0/4167394936 conn(0x5603155b0950 msgr2=0x5603155b6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:18.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.888+0000 7f9c4798e740 1 -- 172.21.15.107:0/4167394936 shutdown_connections 2024-04-25T00:26:18.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.888+0000 7f9c4798e740 1 -- 172.21.15.107:0/4167394936 wait complete. 2024-04-25T00:26:18.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.888+0000 7f9c4798e740 20 client.17838 trim_cache size 0 max 16384 2024-04-25T00:26:18.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.905+0000 7f9c4798e740 1 Processor -- start 2024-04-25T00:26:18.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.905+0000 7f9c4798e740 1 -- start start 2024-04-25T00:26:18.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.906+0000 7f9c4798e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155b4630 0x5603155cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.906+0000 7f9c4798e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603155b4f50 0x5603155b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.906+0000 7f9c4798e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603155bf620 0x5603155b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.906+0000 7f9c4798e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9c401af2c0 con 0x5603155b4630 2024-04-25T00:26:18.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.906+0000 7f9c4798e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9c401acb90 con 0x5603155bf620 2024-04-25T00:26:18.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.906+0000 7f9c4798e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9c401aaa00 con 0x5603155b4f50 2024-04-25T00:26:18.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.906+0000 7f9c3e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155b4630 0x5603155cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.906+0000 7f9c3e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155b4630 0x5603155cb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44340/0 (socket says 172.21.15.107:44340) 2024-04-25T00:26:18.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.906+0000 7f9c3e59c640 1 -- 172.21.15.107:0/516565859 learned_addr learned my addr 172.21.15.107:0/516565859 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:18.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.906+0000 7f9c3dd9b640 1 --2- 172.21.15.107:0/516565859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603155b4f50 0x5603155b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.906+0000 7f9c3ed9d640 1 --2- 172.21.15.107:0/516565859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603155bf620 0x5603155b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.906+0000 7f9c3e59c640 1 -- 172.21.15.107:0/516565859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603155b4f50 msgr2=0x5603155b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:18.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.906+0000 7f9c3e59c640 1 --2- 172.21.15.107:0/516565859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603155b4f50 0x5603155b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.906+0000 7f9c3e59c640 1 -- 172.21.15.107:0/516565859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603155bf620 msgr2=0x5603155b6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:18.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.906+0000 7f9c3e59c640 1 --2- 172.21.15.107:0/516565859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603155bf620 0x5603155b6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.906+0000 7f9c3e59c640 1 -- 172.21.15.107:0/516565859 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603155a2090 con 0x5603155b4630 2024-04-25T00:26:18.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.906+0000 7f9c3dd9b640 1 --2- 172.21.15.107:0/516565859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603155b4f50 0x5603155b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:18.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.906+0000 7f9c3ed9d640 1 --2- 172.21.15.107:0/516565859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603155bf620 0x5603155b6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:18.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.906+0000 7f9c3e59c640 1 --2- 172.21.15.107:0/516565859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155b4630 0x5603155cb2d0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f9c30005780 tx=0x7f9c30000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa90cf5be7fac0be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:18.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.906+0000 7f9c27fff640 1 -- 172.21.15.107:0/516565859 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30035040 con 0x5603155b4630 2024-04-25T00:26:18.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.906+0000 7f9c27fff640 1 -- 172.21.15.107:0/516565859 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c30043d20 con 0x5603155b4630 2024-04-25T00:26:18.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.906+0000 7f9c27fff640 1 -- 172.21.15.107:0/516565859 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c300435c0 con 0x5603155b4630 2024-04-25T00:26:18.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.907+0000 7f9c4798e740 1 -- 172.21.15.107:0/516565859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155b4630 msgr2=0x5603155cb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:18.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.907+0000 7f9c4798e740 1 --2- 172.21.15.107:0/516565859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155b4630 0x5603155cb2d0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f9c30005780 tx=0x7f9c30000bc0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.907+0000 7f9c4798e740 1 -- 172.21.15.107:0/516565859 shutdown_connections 2024-04-25T00:26:18.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.907+0000 7f9c4798e740 1 --2- 172.21.15.107:0/516565859 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603155bf620 0x5603155b6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.907+0000 7f9c4798e740 1 --2- 172.21.15.107:0/516565859 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603155b4f50 0x5603155b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.907+0000 7f9c4798e740 1 --2- 172.21.15.107:0/516565859 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155b4630 0x5603155cb2d0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.907+0000 7f9c4798e740 1 -- 172.21.15.107:0/516565859 >> 172.21.15.107:0/516565859 conn(0x5603155b0950 msgr2=0x5603155bc5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:18.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.907+0000 7f9c4798e740 1 -- 172.21.15.107:0/516565859 shutdown_connections 2024-04-25T00:26:18.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.907+0000 7f9c4798e740 1 -- 172.21.15.107:0/516565859 wait complete. 2024-04-25T00:26:18.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.907+0000 7f9c4798e740 1 -- start start 2024-04-25T00:26:18.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.907+0000 7f9c4798e740 1 Processor -- start 2024-04-25T00:26:18.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.908+0000 7f9c4798e740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603155b4630 0x5603155c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.908+0000 7f9c4798e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603155b4f50 0x5603155c2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.908+0000 7f9c4798e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155bf620 0x5603155c3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.908+0000 7f9c4798e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9c401ab050 con 0x5603155bf620 2024-04-25T00:26:18.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.908+0000 7f9c4798e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9c401ac4c0 con 0x5603155b4f50 2024-04-25T00:26:18.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.908+0000 7f9c4798e740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9c401ab720 con 0x5603155b4630 2024-04-25T00:26:18.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.908+0000 7f9c4798e740 10 client.0 _ll_register_callbacks cb 0x7f9c3400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:18.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.908+0000 7f9c3ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155bf620 0x5603155c3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.908+0000 7f9c3ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155bf620 0x5603155c3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44356/0 (socket says 172.21.15.107:44356) 2024-04-25T00:26:18.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.908+0000 7f9c3ed9d640 1 -- 172.21.15.107:0/1698696451 learned_addr learned my addr 172.21.15.107:0/1698696451 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:18.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.908+0000 7f9c3dd9b640 1 --2- 172.21.15.107:0/1698696451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603155b4f50 0x5603155c2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.908+0000 7f9c3e59c640 1 --2- 172.21.15.107:0/1698696451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603155b4630 0x5603155c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.908+0000 7f9c3ed9d640 1 -- 172.21.15.107:0/1698696451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603155b4630 msgr2=0x5603155c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:18.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.908+0000 7f9c3ed9d640 1 --2- 172.21.15.107:0/1698696451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603155b4630 0x5603155c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.908+0000 7f9c3ed9d640 1 -- 172.21.15.107:0/1698696451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603155b4f50 msgr2=0x5603155c2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:18.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.908+0000 7f9c3ed9d640 1 --2- 172.21.15.107:0/1698696451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603155b4f50 0x5603155c2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.908+0000 7f9c3ed9d640 1 -- 172.21.15.107:0/1698696451 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603155c3840 con 0x5603155bf620 2024-04-25T00:26:18.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.908+0000 7f9c3ed9d640 1 --2- 172.21.15.107:0/1698696451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155bf620 0x5603155c3300 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009fd0 tx=0x7f9c38026c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbcb398e876050a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:18.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.908+0000 7f9c25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:26:18.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.908+0000 7f9c25ffb640 1 -- 172.21.15.107:0/1698696451 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c38009ab0 con 0x5603155bf620 2024-04-25T00:26:18.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.908+0000 7f9c25ffb640 1 -- 172.21.15.107:0/1698696451 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f9c380052f0 con 0x5603155bf620 2024-04-25T00:26:18.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.908+0000 7f9c4798e740 1 -- 172.21.15.107:0/1698696451 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c30043d20 con 0x5603155bf620 2024-04-25T00:26:18.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.909+0000 7f9c25ffb640 1 -- 172.21.15.107:0/1698696451 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9c38039050 con 0x5603155bf620 2024-04-25T00:26:18.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.909+0000 7f9c4798e740 10 client.17844 fetch_fsmap learned FSMap version 34 2024-04-25T00:26:18.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.909+0000 7f9c4798e740 1 -- 172.21.15.107:0/1698696451 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560315635ac0 con 0x5603155bf620 2024-04-25T00:26:18.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.909+0000 7f9c25ffb640 1 -- 172.21.15.107:0/1698696451 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c30043d20 con 0x5603155bf620 2024-04-25T00:26:18.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.909+0000 7f9c25ffb640 1 -- 172.21.15.107:0/1698696451 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f9c3801c950 con 0x5603155bf620 2024-04-25T00:26:18.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.909+0000 7f9c4798e740 10 client.17844 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:26:18.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.909+0000 7f9c4798e740 10 client.17844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:18.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.909+0000 7f9c4798e740 20 client.17844 populate_metadata read hostname 'smithi107' 2024-04-25T00:26:18.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.909+0000 7f9c4798e740 1 --2- 172.21.15.107:0/1698696451 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5603156361b0 0x5603156385d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:18.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.909+0000 7f9c4798e740 4 client.17844 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:26:18.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.909+0000 7f9c4798e740 1 -- 172.21.15.107:0/1698696451 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56031549e6f0 con 0x5603156361b0 2024-04-25T00:26:18.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.909+0000 7f9c3e59c640 1 --2- 172.21.15.107:0/1698696451 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5603156361b0 0x5603156385d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:18.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.909+0000 7f9c25ffb640 10 client.17844 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:26:18.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.909+0000 7f9c3e59c640 1 --2- 172.21.15.107:0/1698696451 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5603156361b0 0x5603156385d0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d35292a956ee8ac0 server_cookie=c9bf568d3a9c8b30 in_seq=0 out_seq=0 2024-04-25T00:26:18.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c25ffb640 1 -- 172.21.15.107:0/1698696451 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56031549e6f0 con 0x5603156361b0 2024-04-25T00:26:18.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c25ffb640 10 client.17844 handle_command_reply: tid=0 2024-04-25T00:26:18.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c4798e740 1 client.17844 shutdown 2024-04-25T00:26:18.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c4798e740 1 -- 172.21.15.107:0/1698696451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155bf620 msgr2=0x5603155c3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:18.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c4798e740 1 --2- 172.21.15.107:0/1698696451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155bf620 0x5603155c3300 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009fd0 tx=0x7f9c38026c60 comp rx=0 tx=0).stop 2024-04-25T00:26:18.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c4798e740 1 -- 172.21.15.107:0/1698696451 shutdown_connections 2024-04-25T00:26:18.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c4798e740 1 --2- 172.21.15.107:0/1698696451 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x5603156361b0 0x5603156385d0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c4798e740 1 --2- 172.21.15.107:0/1698696451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5603155bf620 0x5603155c3300 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c25ffb640 0 client.17844 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:26:18.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c4798e740 1 --2- 172.21.15.107:0/1698696451 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603155b4f50 0x5603155c2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c4798e740 1 --2- 172.21.15.107:0/1698696451 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x5603155b4630 0x5603155c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c4798e740 1 -- 172.21.15.107:0/1698696451 >> 172.21.15.107:0/1698696451 conn(0x5603155b0950 msgr2=0x5603155bce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:18.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c4798e740 1 -- 172.21.15.107:0/1698696451 shutdown_connections 2024-04-25T00:26:18.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c4798e740 1 -- 172.21.15.107:0/1698696451 wait complete. 2024-04-25T00:26:18.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c4798e740 20 client.17844 trim_cache size 0 max 16384 2024-04-25T00:26:18.920 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:26:18.920 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T00:26:18.920 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-04-25T00:26:18.920 INFO:teuthology.orchestra.run.smithi107.stdout: "0a6868ef-4d56-4d25-882c-e115a13cab96": { 2024-04-25T00:26:18.920 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-04-25T00:26:18.920 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "0a6868ef-4d56-4d25-882c-e115a13cab96", 2024-04-25T00:26:18.921 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-04-25T00:26:18.921 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:18.921 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:18.921 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:26:18.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c44ba5640 1 -- 172.21.15.107:0/2739527040 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f9c280a2d00 msgr2=0x7f9c280a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:18.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c44ba5640 1 --2- 172.21.15.107:0/2739527040 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f9c280a2d00 0x7f9c280a51c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9c38026000 tx=0x7f9c38023000 comp rx=0 tx=0).stop 2024-04-25T00:26:18.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c44ba5640 1 -- 172.21.15.107:0/2739527040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c40100fd0 msgr2=0x7f9c401984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:18.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c44ba5640 1 --2- 172.21.15.107:0/2739527040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c40100fd0 0x7f9c401984f0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f9c3400e9f0 tx=0x7f9c3400edf0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c44ba5640 1 -- 172.21.15.107:0/2739527040 shutdown_connections 2024-04-25T00:26:18.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c44ba5640 1 --2- 172.21.15.107:0/2739527040 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f9c280a2d00 0x7f9c280a51c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c44ba5640 1 --2- 172.21.15.107:0/2739527040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9c401066b0 0x7f9c4019c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c44ba5640 1 --2- 172.21.15.107:0/2739527040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c40100fd0 0x7f9c401984f0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c44ba5640 1 --2- 172.21.15.107:0/2739527040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c40100610 0x7f9c40197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:18.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.910+0000 7f9c44ba5640 1 -- 172.21.15.107:0/2739527040 >> 172.21.15.107:0/2739527040 conn(0x7f9c40077740 msgr2=0x7f9c400ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:18.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.911+0000 7f9c44ba5640 1 -- 172.21.15.107:0/2739527040 shutdown_connections 2024-04-25T00:26:18.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:18.911+0000 7f9c44ba5640 1 -- 172.21.15.107:0/2739527040 wait complete. 2024-04-25T00:26:18.924 INFO:tasks.cephfs.filesystem:scrub status for tag:0a6868ef-4d56-4d25-882c-e115a13cab96 - {'path': '/', 'tag': '0a6868ef-4d56-4d25-882c-e115a13cab96', 'options': 'recursive,force'} 2024-04-25T00:26:19.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:19 smithi170 ceph-mon[32807]: pgmap v307: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:26:19.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:19 smithi170 ceph-mon[32807]: scrub queued for path: / 2024-04-25T00:26:19.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:26:19.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:19 smithi170 ceph-mon[32807]: scrub summary: active paths [/] 2024-04-25T00:26:19.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:19 smithi170 ceph-mon[32807]: scrub summary: idle+waiting paths [/] 2024-04-25T00:26:19.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:19 smithi107 ceph-mon[29153]: pgmap v307: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:26:19.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:19 smithi107 ceph-mon[29153]: scrub queued for path: / 2024-04-25T00:26:19.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:26:19.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:19 smithi107 ceph-mon[29153]: scrub summary: active paths [/] 2024-04-25T00:26:19.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:19 smithi107 ceph-mon[29153]: scrub summary: idle+waiting paths [/] 2024-04-25T00:26:20.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:19 smithi195 ceph-mon[32833]: pgmap v307: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:26:20.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:19 smithi195 ceph-mon[32833]: scrub queued for path: / 2024-04-25T00:26:20.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:26:20.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:19 smithi195 ceph-mon[32833]: scrub summary: active paths [/] 2024-04-25T00:26:20.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:19 smithi195 ceph-mon[32833]: scrub summary: idle+waiting paths [/] 2024-04-25T00:26:21.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:21 smithi170 ceph-mon[32807]: pgmap v308: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:26:21.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:21 smithi107 ceph-mon[29153]: pgmap v308: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:26:22.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:21 smithi195 ceph-mon[32833]: pgmap v308: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:26:22.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:22 smithi107 ceph-mon[29153]: scrub summary: idle 2024-04-25T00:26:23.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:22 smithi195 ceph-mon[32833]: scrub summary: idle 2024-04-25T00:26:23.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:22 smithi170 ceph-mon[32807]: scrub summary: idle 2024-04-25T00:26:23.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:23 smithi107 ceph-mon[29153]: pgmap v309: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:26:24.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:23 smithi195 ceph-mon[32833]: pgmap v309: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:26:24.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:23 smithi170 ceph-mon[32807]: pgmap v309: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:26:25.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:25 smithi107 ceph-mon[29153]: pgmap v310: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:26:26.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:25 smithi195 ceph-mon[32833]: pgmap v310: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:26:26.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:25 smithi170 ceph-mon[32807]: pgmap v310: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:26:27.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:27 smithi107 ceph-mon[29153]: pgmap v311: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:26:28.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:27 smithi195 ceph-mon[32833]: pgmap v311: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:26:28.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:27 smithi170 ceph-mon[32807]: pgmap v311: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:26:29.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:29 smithi107 ceph-mon[29153]: pgmap v312: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 87 B/s wr, 8 op/s 2024-04-25T00:26:30.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:29 smithi195 ceph-mon[32833]: pgmap v312: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 87 B/s wr, 8 op/s 2024-04-25T00:26:30.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:29 smithi170 ceph-mon[32807]: pgmap v312: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 87 B/s wr, 8 op/s 2024-04-25T00:26:31.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:31 smithi107 ceph-mon[29153]: pgmap v313: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:26:32.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:31 smithi195 ceph-mon[32833]: pgmap v313: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:26:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:31 smithi170 ceph-mon[32807]: pgmap v313: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:26:33.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:33 smithi107 ceph-mon[29153]: pgmap v314: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:26:34.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:33 smithi195 ceph-mon[32833]: pgmap v314: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:26:34.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:33 smithi170 ceph-mon[32807]: pgmap v314: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:26:35.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:35 smithi107 ceph-mon[29153]: pgmap v315: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:26:36.065 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T00:26:36.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:35 smithi195 ceph-mon[32833]: pgmap v315: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:26:36.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.175+0000 7fd95c7d5640 1 -- 172.21.15.107:0/2955980154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd95410bd10 msgr2=0x7fd95410e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:36.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.175+0000 7fd95c7d5640 1 --2- 172.21.15.107:0/2955980154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd95410bd10 0x7fd95410e1b0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009900 tx=0x7fd94c02a510 comp rx=0 tx=0).stop 2024-04-25T00:26:36.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.175+0000 7fd95c7d5640 1 -- 172.21.15.107:0/2955980154 shutdown_connections 2024-04-25T00:26:36.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.175+0000 7fd95c7d5640 1 --2- 172.21.15.107:0/2955980154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd95410bd10 0x7fd95410e1b0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:36.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.175+0000 7fd95c7d5640 1 --2- 172.21.15.107:0/2955980154 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9541032f0 0x7fd95410b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:36.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.175+0000 7fd95c7d5640 1 --2- 172.21.15.107:0/2955980154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd9541029d0 0x7fd954102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:36.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.175+0000 7fd95c7d5640 1 -- 172.21.15.107:0/2955980154 >> 172.21.15.107:0/2955980154 conn(0x7fd9540777c0 msgr2=0x7fd954077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:36.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.175+0000 7fd95c7d5640 1 -- 172.21.15.107:0/2955980154 shutdown_connections 2024-04-25T00:26:36.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.175+0000 7fd95c7d5640 1 -- 172.21.15.107:0/2955980154 wait complete. 2024-04-25T00:26:36.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.175+0000 7fd95c7d5640 1 Processor -- start 2024-04-25T00:26:36.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.175+0000 7fd95c7d5640 1 -- start start 2024-04-25T00:26:36.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.176+0000 7fd95c7d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9541029d0 0x7fd954197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:36.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.176+0000 7fd95c7d5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9541032f0 0x7fd954198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:36.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.176+0000 7fd95c7d5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd95410bd10 0x7fd95419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:36.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.176+0000 7fd95c7d5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd954111b10 con 0x7fd9541029d0 2024-04-25T00:26:36.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.176+0000 7fd95c7d5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd954111990 con 0x7fd9541032f0 2024-04-25T00:26:36.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.176+0000 7fd95c7d5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fd954111c90 con 0x7fd95410bd10 2024-04-25T00:26:36.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.176+0000 7fd95a571640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9541029d0 0x7fd954197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:36.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.176+0000 7fd95a571640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9541029d0 0x7fd954197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57022/0 (socket says 172.21.15.107:57022) 2024-04-25T00:26:36.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.176+0000 7fd95a571640 1 -- 172.21.15.107:0/2906909874 learned_addr learned my addr 172.21.15.107:0/2906909874 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:36.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.176+0000 7fd95a571640 1 -- 172.21.15.107:0/2906909874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd95410bd10 msgr2=0x7fd95419c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:36.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.176+0000 7fd959d70640 1 --2- 172.21.15.107:0/2906909874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9541032f0 0x7fd954198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:36.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.176+0000 7fd95ad72640 1 --2- 172.21.15.107:0/2906909874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd95410bd10 0x7fd95419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:36.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.176+0000 7fd95a571640 1 --2- 172.21.15.107:0/2906909874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd95410bd10 0x7fd95419c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:36.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.176+0000 7fd95a571640 1 -- 172.21.15.107:0/2906909874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9541032f0 msgr2=0x7fd954198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:36.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.176+0000 7fd95a571640 1 --2- 172.21.15.107:0/2906909874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9541032f0 0x7fd954198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:36.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.176+0000 7fd95a571640 1 -- 172.21.15.107:0/2906909874 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95419cf00 con 0x7fd9541029d0 2024-04-25T00:26:36.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.176+0000 7fd95a571640 1 --2- 172.21.15.107:0/2906909874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9541029d0 0x7fd954197f10 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fd95000eab0 tx=0x7fd95000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:36.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.176+0000 7fd94b7fe640 1 -- 172.21.15.107:0/2906909874 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95000ce40 con 0x7fd9541029d0 2024-04-25T00:26:36.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.176+0000 7fd95c7d5640 1 -- 172.21.15.107:0/2906909874 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd95419d1f0 con 0x7fd9541029d0 2024-04-25T00:26:36.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.176+0000 7fd94b7fe640 1 -- 172.21.15.107:0/2906909874 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd950011950 con 0x7fd9541029d0 2024-04-25T00:26:36.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.176+0000 7fd94b7fe640 1 -- 172.21.15.107:0/2906909874 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd950010ae0 con 0x7fd9541029d0 2024-04-25T00:26:36.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.176+0000 7fd95c7d5640 1 -- 172.21.15.107:0/2906909874 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9541a4a80 con 0x7fd9541029d0 2024-04-25T00:26:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.177+0000 7fd95c7d5640 1 -- 172.21.15.107:0/2906909874 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd95404f500 con 0x7fd9541029d0 2024-04-25T00:26:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.178+0000 7fd94b7fe640 1 -- 172.21.15.107:0/2906909874 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fd950011b00 con 0x7fd9541029d0 2024-04-25T00:26:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.179+0000 7fd94b7fe640 1 --2- 172.21.15.107:0/2906909874 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd9400a2da0 0x7fd9400a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.179+0000 7fd959d70640 1 --2- 172.21.15.107:0/2906909874 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd9400a2da0 0x7fd9400a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.179+0000 7fd94b7fe640 1 -- 172.21.15.107:0/2906909874 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7fd9500cfff0 con 0x7fd9541029d0 2024-04-25T00:26:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.179+0000 7fd959d70640 1 --2- 172.21.15.107:0/2906909874 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd9400a2da0 0x7fd9400a5260 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd95410da10 tx=0x7fd944005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:36.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.184+0000 7fd94b7fe640 1 -- 172.21.15.107:0/2906909874 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd950016030 con 0x7fd9541029d0 2024-04-25T00:26:36.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:35 smithi170 ceph-mon[32807]: pgmap v315: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:26:36.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.382+0000 7fd95c7d5640 1 -- 172.21.15.107:0/2906909874 --> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fd9540fdde0 con 0x7fd9400a2da0 2024-04-25T00:26:36.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.390+0000 7fd94b7fe640 1 -- 172.21.15.107:0/2906909874 <== mgr.15456 v2:172.21.15.107:6800/1021104172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fd9540fdde0 con 0x7fd9400a2da0 2024-04-25T00:26:36.391 INFO:teuthology.orchestra.run.smithi107.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T00:16:34", "first": "2024-04-25T00:17:00", "last": "2024-04-25T00:26:00", "last_pruned": "2024-04-25T00:26:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-25T00:26:36.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.392+0000 7fd95c7d5640 1 -- 172.21.15.107:0/2906909874 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd9400a2da0 msgr2=0x7fd9400a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:36.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.392+0000 7fd95c7d5640 1 --2- 172.21.15.107:0/2906909874 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd9400a2da0 0x7fd9400a5260 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd95410da10 tx=0x7fd944005f90 comp rx=0 tx=0).stop 2024-04-25T00:26:36.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.392+0000 7fd95c7d5640 1 -- 172.21.15.107:0/2906909874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9541029d0 msgr2=0x7fd954197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:36.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.392+0000 7fd95c7d5640 1 --2- 172.21.15.107:0/2906909874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9541029d0 0x7fd954197f10 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fd95000eab0 tx=0x7fd95000ef70 comp rx=0 tx=0).stop 2024-04-25T00:26:36.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.392+0000 7fd95c7d5640 1 -- 172.21.15.107:0/2906909874 shutdown_connections 2024-04-25T00:26:36.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.392+0000 7fd95c7d5640 1 --2- 172.21.15.107:0/2906909874 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd95410bd10 0x7fd95419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:36.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.392+0000 7fd95c7d5640 1 --2- 172.21.15.107:0/2906909874 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fd9400a2da0 0x7fd9400a5260 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:36.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.392+0000 7fd95c7d5640 1 --2- 172.21.15.107:0/2906909874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9541032f0 0x7fd954198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:36.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.392+0000 7fd95c7d5640 1 --2- 172.21.15.107:0/2906909874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9541029d0 0x7fd954197f10 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:36.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.392+0000 7fd95c7d5640 1 -- 172.21.15.107:0/2906909874 >> 172.21.15.107:0/2906909874 conn(0x7fd9540777c0 msgr2=0x7fd9540fd6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:36.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.392+0000 7fd95c7d5640 1 -- 172.21.15.107:0/2906909874 shutdown_connections 2024-04-25T00:26:36.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.392+0000 7fd95c7d5640 1 -- 172.21.15.107:0/2906909874 wait complete. 2024-04-25T00:26:36.403 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T00:26:36.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.503+0000 7f8695b34640 1 -- 172.21.15.107:0/816592222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8690106630 msgr2=0x7f8690106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:36.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.503+0000 7f8695b34640 1 --2- 172.21.15.107:0/816592222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8690106630 0x7f8690106a10 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f8680009930 tx=0x7f868002a420 comp rx=0 tx=0).stop 2024-04-25T00:26:36.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.503+0000 7f8695b34640 1 -- 172.21.15.107:0/816592222 shutdown_connections 2024-04-25T00:26:36.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.503+0000 7f8695b34640 1 --2- 172.21.15.107:0/816592222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8690101030 0x7f869010d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:36.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.503+0000 7f8695b34640 1 --2- 172.21.15.107:0/816592222 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8690100670 0x7f8690100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:36.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.503+0000 7f8695b34640 1 --2- 172.21.15.107:0/816592222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8690106630 0x7f8690106a10 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:36.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.503+0000 7f8695b34640 1 -- 172.21.15.107:0/816592222 >> 172.21.15.107:0/816592222 conn(0x7f86900777c0 msgr2=0x7f8690077bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:36.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.503+0000 7f8695b34640 1 -- 172.21.15.107:0/816592222 shutdown_connections 2024-04-25T00:26:36.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.503+0000 7f8695b34640 1 -- 172.21.15.107:0/816592222 wait complete. 2024-04-25T00:26:36.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.503+0000 7f8695b34640 1 Processor -- start 2024-04-25T00:26:36.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.504+0000 7f8695b34640 1 -- start start 2024-04-25T00:26:36.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.504+0000 7f8695b34640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8690100670 0x7f8690198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:36.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.504+0000 7f8695b34640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8690101030 0x7f8690199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:36.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.504+0000 7f8695b34640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8690106630 0x7f86901a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:36.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.504+0000 7f8695b34640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8690110a40 con 0x7f8690106630 2024-04-25T00:26:36.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.504+0000 7f8695b34640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f86901108c0 con 0x7f8690101030 2024-04-25T00:26:36.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.504+0000 7f8695b34640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f8690110bc0 con 0x7f8690100670 2024-04-25T00:26:36.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.504+0000 7f868ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8690106630 0x7f86901a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:36.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.504+0000 7f868ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8690106630 0x7f86901a3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57036/0 (socket says 172.21.15.107:57036) 2024-04-25T00:26:36.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.504+0000 7f868ffff640 1 -- 172.21.15.107:0/1443513315 learned_addr learned my addr 172.21.15.107:0/1443513315 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:36.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.504+0000 7f868effd640 1 --2- 172.21.15.107:0/1443513315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8690101030 0x7f8690199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:36.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.504+0000 7f868f7fe640 1 --2- 172.21.15.107:0/1443513315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8690100670 0x7f8690198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:36.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.504+0000 7f868ffff640 1 -- 172.21.15.107:0/1443513315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8690100670 msgr2=0x7f8690198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:36.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.504+0000 7f868ffff640 1 --2- 172.21.15.107:0/1443513315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8690100670 0x7f8690198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:36.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.504+0000 7f868ffff640 1 -- 172.21.15.107:0/1443513315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8690101030 msgr2=0x7f8690199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:36.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.504+0000 7f868ffff640 1 --2- 172.21.15.107:0/1443513315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8690101030 0x7f8690199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:36.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.504+0000 7f868ffff640 1 -- 172.21.15.107:0/1443513315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86901a44a0 con 0x7f8690106630 2024-04-25T00:26:36.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.504+0000 7f868effd640 1 --2- 172.21.15.107:0/1443513315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8690101030 0x7f8690199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:36.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.504+0000 7f868f7fe640 1 --2- 172.21.15.107:0/1443513315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8690100670 0x7f8690198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:36.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.504+0000 7f868ffff640 1 --2- 172.21.15.107:0/1443513315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8690106630 0x7f86901a3f60 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f868800b970 tx=0x7f868800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:36.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.504+0000 7f868cff9640 1 -- 172.21.15.107:0/1443513315 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8688019070 con 0x7f8690106630 2024-04-25T00:26:36.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.505+0000 7f868cff9640 1 -- 172.21.15.107:0/1443513315 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f868800cb40 con 0x7f8690106630 2024-04-25T00:26:36.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.505+0000 7f8695b34640 1 -- 172.21.15.107:0/1443513315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86901a4670 con 0x7f8690106630 2024-04-25T00:26:36.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.505+0000 7f868cff9640 1 -- 172.21.15.107:0/1443513315 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8688007650 con 0x7f8690106630 2024-04-25T00:26:36.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.505+0000 7f8695b34640 1 -- 172.21.15.107:0/1443513315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86901a4b40 con 0x7f8690106630 2024-04-25T00:26:36.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.505+0000 7f8695b34640 1 -- 172.21.15.107:0/1443513315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f869004f500 con 0x7f8690106630 2024-04-25T00:26:36.509 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.507+0000 7f868cff9640 1 -- 172.21.15.107:0/1443513315 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f868801c070 con 0x7f8690106630 2024-04-25T00:26:36.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.507+0000 7f868cff9640 1 --2- 172.21.15.107:0/1443513315 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f86780a2da0 0x7f86780a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:36.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.507+0000 7f868f7fe640 1 --2- 172.21.15.107:0/1443513315 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f86780a2da0 0x7f86780a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:36.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.507+0000 7f868cff9640 1 -- 172.21.15.107:0/1443513315 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f86880cf210 con 0x7f8690106630 2024-04-25T00:26:36.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.508+0000 7f868f7fe640 1 --2- 172.21.15.107:0/1443513315 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f86780a2da0 0x7f86780a5260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8680000c00 tx=0x7f8680032c70 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:36.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.511+0000 7f868cff9640 1 -- 172.21.15.107:0/1443513315 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f868809bb80 con 0x7f8690106630 2024-04-25T00:26:36.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.708+0000 7f8695b34640 1 -- 172.21.15.107:0/1443513315 --> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f8690061b80 con 0x7f86780a2da0 2024-04-25T00:26:36.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.715+0000 7f868cff9640 1 -- 172.21.15.107:0/1443513315 <== mgr.15456 v2:172.21.15.107:6800/1021104172 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f8690061b80 con 0x7f86780a2da0 2024-04-25T00:26:36.716 INFO:teuthology.orchestra.run.smithi107.stdout:/ 1m 6m3h 2024-04-25T00:26:36.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.717+0000 7f8695b34640 1 -- 172.21.15.107:0/1443513315 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f86780a2da0 msgr2=0x7f86780a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:36.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.717+0000 7f8695b34640 1 --2- 172.21.15.107:0/1443513315 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f86780a2da0 0x7f86780a5260 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8680000c00 tx=0x7f8680032c70 comp rx=0 tx=0).stop 2024-04-25T00:26:36.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.717+0000 7f8695b34640 1 -- 172.21.15.107:0/1443513315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8690106630 msgr2=0x7f86901a3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:36.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.717+0000 7f8695b34640 1 --2- 172.21.15.107:0/1443513315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8690106630 0x7f86901a3f60 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f868800b970 tx=0x7f868800be30 comp rx=0 tx=0).stop 2024-04-25T00:26:36.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.717+0000 7f8695b34640 1 -- 172.21.15.107:0/1443513315 shutdown_connections 2024-04-25T00:26:36.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.717+0000 7f8695b34640 1 --2- 172.21.15.107:0/1443513315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8690106630 0x7f86901a3f60 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:36.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.717+0000 7f8695b34640 1 --2- 172.21.15.107:0/1443513315 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f86780a2da0 0x7f86780a5260 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:36.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.717+0000 7f8695b34640 1 --2- 172.21.15.107:0/1443513315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8690101030 0x7f8690199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:36.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.717+0000 7f8695b34640 1 --2- 172.21.15.107:0/1443513315 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8690100670 0x7f8690198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:36.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.718+0000 7f8695b34640 1 -- 172.21.15.107:0/1443513315 >> 172.21.15.107:0/1443513315 conn(0x7f86900777c0 msgr2=0x7f86901031f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:36.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.718+0000 7f8695b34640 1 -- 172.21.15.107:0/1443513315 shutdown_connections 2024-04-25T00:26:36.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:36.718+0000 7f8695b34640 1 -- 172.21.15.107:0/1443513315 wait complete. 2024-04-25T00:26:36.729 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-25T00:26:36.740 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-25T00:26:37.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:37 smithi107 ceph-mon[29153]: from='client.17850 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:26:37.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:37 smithi107 ceph-mon[29153]: pgmap v316: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:26:37.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:37 smithi107 ceph-mon[29153]: from='client.17856 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:26:38.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:37 smithi195 ceph-mon[32833]: from='client.17850 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:26:38.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:37 smithi195 ceph-mon[32833]: pgmap v316: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:26:38.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:37 smithi195 ceph-mon[32833]: from='client.17856 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:26:38.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:37 smithi170 ceph-mon[32807]: from='client.17850 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:26:38.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:37 smithi170 ceph-mon[32807]: pgmap v316: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:26:38.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:37 smithi170 ceph-mon[32807]: from='client.17856 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:26:39.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:39 smithi107 ceph-mon[29153]: pgmap v317: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T00:26:39.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:39 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:26:40.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:39 smithi195 ceph-mon[32833]: pgmap v317: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T00:26:40.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:39 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:26:40.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:39 smithi170 ceph-mon[32807]: pgmap v317: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T00:26:40.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:39 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:26:41.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:41 smithi107 ceph-mon[29153]: pgmap v318: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T00:26:41.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:41 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:26:41.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:41 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:26:41.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:41 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:26:41.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:41 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:26:41.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:41 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:26:41.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:41 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:26:42.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:41 smithi195 ceph-mon[32833]: pgmap v318: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T00:26:42.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:41 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:26:42.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:41 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:26:42.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:41 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:26:42.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:41 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:26:42.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:41 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:26:42.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:41 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:26:42.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:41 smithi170 ceph-mon[32807]: pgmap v318: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T00:26:42.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:41 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:26:42.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:41 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:26:42.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:41 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:26:42.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:41 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:26:42.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:41 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:26:42.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:41 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:26:42.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:42 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:42.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:42 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:42.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:42 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:42.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:42 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:42.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:42 smithi107 ceph-mon[29153]: Adjusting osd_memory_target on smithi170 to 176.2M 2024-04-25T00:26:42.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:42 smithi107 ceph-mon[29153]: Unable to set osd_memory_target on smithi170 to 184790323: error parsing value: Value '184790323' is below minimum 939524096 2024-04-25T00:26:42.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:42 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:42.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:42 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:42.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:42 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:42.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:42 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:42.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:42 smithi107 ceph-mon[29153]: Adjusting osd_memory_target on smithi195 to 176.2M 2024-04-25T00:26:42.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:42 smithi107 ceph-mon[29153]: Unable to set osd_memory_target on smithi195 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T00:26:42.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:42 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:42.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:42 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:42.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:42 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:42.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:42 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:42.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:42 smithi107 ceph-mon[29153]: Adjusting osd_memory_target on smithi107 to 176.1M 2024-04-25T00:26:42.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:42 smithi107 ceph-mon[29153]: Unable to set osd_memory_target on smithi107 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-04-25T00:26:42.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:42 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:42.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:42 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:26:42.986 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:42 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:26:43.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:42 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:43.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:42 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:43.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:42 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:43.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:42 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:43.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:42 smithi195 ceph-mon[32833]: Adjusting osd_memory_target on smithi170 to 176.2M 2024-04-25T00:26:43.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:42 smithi195 ceph-mon[32833]: Unable to set osd_memory_target on smithi170 to 184790323: error parsing value: Value '184790323' is below minimum 939524096 2024-04-25T00:26:43.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:42 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:43.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:42 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:43.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:42 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:43.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:42 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:43.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:42 smithi195 ceph-mon[32833]: Adjusting osd_memory_target on smithi195 to 176.2M 2024-04-25T00:26:43.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:42 smithi195 ceph-mon[32833]: Unable to set osd_memory_target on smithi195 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T00:26:43.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:42 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:43.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:42 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:43.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:42 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:43.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:42 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:43.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:42 smithi195 ceph-mon[32833]: Adjusting osd_memory_target on smithi107 to 176.1M 2024-04-25T00:26:43.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:42 smithi195 ceph-mon[32833]: Unable to set osd_memory_target on smithi107 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-04-25T00:26:43.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:42 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:43.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:42 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:26:43.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:42 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:26:43.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:42 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:43.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:42 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:43.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:42 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:43.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:42 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:43.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:42 smithi170 ceph-mon[32807]: Adjusting osd_memory_target on smithi170 to 176.2M 2024-04-25T00:26:43.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:42 smithi170 ceph-mon[32807]: Unable to set osd_memory_target on smithi170 to 184790323: error parsing value: Value '184790323' is below minimum 939524096 2024-04-25T00:26:43.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:42 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:43.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:42 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:43.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:42 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:43.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:42 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:43.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:42 smithi170 ceph-mon[32807]: Adjusting osd_memory_target on smithi195 to 176.2M 2024-04-25T00:26:43.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:42 smithi170 ceph-mon[32807]: Unable to set osd_memory_target on smithi195 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-25T00:26:43.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:42 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:43.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:42 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:43.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:42 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:43.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:42 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:43.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:42 smithi170 ceph-mon[32807]: Adjusting osd_memory_target on smithi107 to 176.1M 2024-04-25T00:26:43.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:42 smithi170 ceph-mon[32807]: Unable to set osd_memory_target on smithi107 to 184726528: error parsing value: Value '184726528' is below minimum 939524096 2024-04-25T00:26:43.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:42 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:43.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:42 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:26:43.187 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:42 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' 2024-04-25T00:26:44.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:44 smithi170 ceph-mon[32807]: pgmap v319: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T00:26:44.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:44 smithi107 ceph-mon[29153]: pgmap v319: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T00:26:44.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:44 smithi195 ceph-mon[32833]: pgmap v319: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T00:26:46.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:46 smithi170 ceph-mon[32807]: pgmap v320: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T00:26:46.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:46 smithi107 ceph-mon[29153]: pgmap v320: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T00:26:46.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:46 smithi195 ceph-mon[32833]: pgmap v320: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T00:26:48.391 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:48 smithi195 ceph-mon[32833]: pgmap v321: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T00:26:48.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:48 smithi170 ceph-mon[32807]: pgmap v321: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T00:26:48.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:48 smithi107 ceph-mon[29153]: pgmap v321: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T00:26:48.924 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-25T00:26:48.924 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-25T00:26:49.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.038+0000 7f72e5e9e640 1 -- 172.21.15.107:0/4123756278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72e01066b0 msgr2=0x7f72e0106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.038+0000 7f72e5e9e640 1 --2- 172.21.15.107:0/4123756278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72e01066b0 0x7f72e0106a90 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f72d4009930 tx=0x7f72d402a420 comp rx=0 tx=0).stop 2024-04-25T00:26:49.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.038+0000 7f72e5e9e640 1 -- 172.21.15.107:0/4123756278 shutdown_connections 2024-04-25T00:26:49.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.038+0000 7f72e5e9e640 1 --2- 172.21.15.107:0/4123756278 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72e0100fd0 0x7f72e010d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.038+0000 7f72e5e9e640 1 --2- 172.21.15.107:0/4123756278 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72e0100610 0x7f72e0100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.038+0000 7f72e5e9e640 1 --2- 172.21.15.107:0/4123756278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72e01066b0 0x7f72e0106a90 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.038+0000 7f72e5e9e640 1 -- 172.21.15.107:0/4123756278 >> 172.21.15.107:0/4123756278 conn(0x7f72e0077740 msgr2=0x7f72e0077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.038+0000 7f72e5e9e640 1 -- 172.21.15.107:0/4123756278 shutdown_connections 2024-04-25T00:26:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.038+0000 7f72e5e9e640 1 -- 172.21.15.107:0/4123756278 wait complete. 2024-04-25T00:26:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.038+0000 7f72e5e9e640 1 Processor -- start 2024-04-25T00:26:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.039+0000 7f72e5e9e640 1 -- start start 2024-04-25T00:26:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.039+0000 7f72e5e9e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72e0100610 0x7f72e0198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.039+0000 7f72e5e9e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72e0100fd0 0x7f72e01993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.039+0000 7f72e5e9e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72e01066b0 0x7f72e01a3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.039+0000 7f72e5e9e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f72e0110a30 con 0x7f72e0100fd0 2024-04-25T00:26:49.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.039+0000 7f72e5e9e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f72e01108b0 con 0x7f72e0100610 2024-04-25T00:26:49.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.039+0000 7f72e5e9e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f72e0110bb0 con 0x7f72e01066b0 2024-04-25T00:26:49.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.039+0000 7f72deffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72e0100fd0 0x7f72e01993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.039+0000 7f72deffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72e0100fd0 0x7f72e01993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53734/0 (socket says 172.21.15.107:53734) 2024-04-25T00:26:49.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.039+0000 7f72deffd640 1 -- 172.21.15.107:0/1627580397 learned_addr learned my addr 172.21.15.107:0/1627580397 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:49.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.039+0000 7f72dffff640 1 --2- 172.21.15.107:0/1627580397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72e01066b0 0x7f72e01a3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.039+0000 7f72df7fe640 1 --2- 172.21.15.107:0/1627580397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72e0100610 0x7f72e0198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.039+0000 7f72deffd640 1 -- 172.21.15.107:0/1627580397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72e01066b0 msgr2=0x7f72e01a3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.039+0000 7f72deffd640 1 --2- 172.21.15.107:0/1627580397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72e01066b0 0x7f72e01a3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.039+0000 7f72deffd640 1 -- 172.21.15.107:0/1627580397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72e0100610 msgr2=0x7f72e0198e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.039+0000 7f72deffd640 1 --2- 172.21.15.107:0/1627580397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72e0100610 0x7f72e0198e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.039+0000 7f72deffd640 1 -- 172.21.15.107:0/1627580397 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e01a4430 con 0x7f72e0100fd0 2024-04-25T00:26:49.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.039+0000 7f72dffff640 1 --2- 172.21.15.107:0/1627580397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72e01066b0 0x7f72e01a3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:49.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.040+0000 7f72deffd640 1 --2- 172.21.15.107:0/1627580397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72e0100fd0 0x7f72e01993b0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f72d000e9f0 tx=0x7f72d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.040+0000 7f72dcff9640 1 -- 172.21.15.107:0/1627580397 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d000cd80 con 0x7f72e0100fd0 2024-04-25T00:26:49.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.040+0000 7f72dcff9640 1 -- 172.21.15.107:0/1627580397 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72d0005500 con 0x7f72e0100fd0 2024-04-25T00:26:49.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.040+0000 7f72e5e9e640 1 -- 172.21.15.107:0/1627580397 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72e01a4600 con 0x7f72e0100fd0 2024-04-25T00:26:49.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.040+0000 7f72dcff9640 1 -- 172.21.15.107:0/1627580397 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d0010690 con 0x7f72e0100fd0 2024-04-25T00:26:49.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.040+0000 7f72e5e9e640 1 -- 172.21.15.107:0/1627580397 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72e01a4a70 con 0x7f72e0100fd0 2024-04-25T00:26:49.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.042+0000 7f72dcff9640 1 -- 172.21.15.107:0/1627580397 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f72d00046c0 con 0x7f72e0100fd0 2024-04-25T00:26:49.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.042+0000 7f72e8bab740 1 Processor -- start 2024-04-25T00:26:49.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.042+0000 7f72dcff9640 1 --2- 172.21.15.107:0/1627580397 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f72c80a2d00 0x7f72c80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.042+0000 7f72e8bab740 1 -- start start 2024-04-25T00:26:49.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.043+0000 7f72dffff640 1 --2- 172.21.15.107:0/1627580397 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f72c80a2d00 0x7f72c80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.043+0000 7f72dcff9640 1 -- 172.21.15.107:0/1627580397 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f72d00cfd70 con 0x7f72e0100fd0 2024-04-25T00:26:49.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.043+0000 7f72e8bab740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55595a16f630 0x55595a16fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.043+0000 7f72e8bab740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a16ff50 0x55595a17a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.043+0000 7f72e8bab740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55595a17a620 0x55595a17ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.043+0000 7f72e8bab740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55595a0d46b0 con 0x55595a16ff50 2024-04-25T00:26:49.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.043+0000 7f72e8bab740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55595a07c880 con 0x55595a17a620 2024-04-25T00:26:49.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.043+0000 7f72e8bab740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55595a0d0670 con 0x55595a16f630 2024-04-25T00:26:49.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.043+0000 7f72deffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a16ff50 0x55595a17a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.043+0000 7f72deffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a16ff50 0x55595a17a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53746/0 (socket says 172.21.15.107:53746) 2024-04-25T00:26:49.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.043+0000 7f72deffd640 1 -- 172.21.15.107:0/161297678 learned_addr learned my addr 172.21.15.107:0/161297678 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:49.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.043+0000 7f72df7fe640 1 --2- 172.21.15.107:0/161297678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55595a16f630 0x55595a16fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.043+0000 7f72dffff640 1 --2- 172.21.15.107:0/1627580397 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f72c80a2d00 0x7f72c80a51c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f72d8005fd0 tx=0x7f72d8009500 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.043+0000 7f72dffff640 1 --2- 172.21.15.107:0/161297678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55595a17a620 0x55595a17ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.043+0000 7f72deffd640 1 -- 172.21.15.107:0/161297678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55595a16f630 msgr2=0x55595a16fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.043+0000 7f72deffd640 1 --2- 172.21.15.107:0/161297678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55595a16f630 0x55595a16fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.043+0000 7f72deffd640 1 -- 172.21.15.107:0/161297678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55595a17a620 msgr2=0x55595a17ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.043+0000 7f72deffd640 1 --2- 172.21.15.107:0/161297678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55595a17a620 0x55595a17ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.043+0000 7f72deffd640 1 -- 172.21.15.107:0/161297678 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555959eb27d0 con 0x55595a16ff50 2024-04-25T00:26:49.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.043+0000 7f72dffff640 1 --2- 172.21.15.107:0/161297678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55595a17a620 0x55595a17ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:49.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.043+0000 7f72deffd640 1 --2- 172.21.15.107:0/161297678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a16ff50 0x55595a17a0e0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f72d0009380 tx=0x7f72d009b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d452429f791df9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.043+0000 7f72c5ffb640 1 -- 172.21.15.107:0/161297678 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d00a0280 con 0x55595a16ff50 2024-04-25T00:26:49.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.043+0000 7f72c5ffb640 1 -- 172.21.15.107:0/161297678 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72d00a4b40 con 0x55595a16ff50 2024-04-25T00:26:49.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.044+0000 7f72c5ffb640 1 -- 172.21.15.107:0/161297678 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d00a48b0 con 0x55595a16ff50 2024-04-25T00:26:49.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.044+0000 7f72e8bab740 1 -- 172.21.15.107:0/161297678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a16ff50 msgr2=0x55595a17a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.044+0000 7f72e8bab740 1 --2- 172.21.15.107:0/161297678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a16ff50 0x55595a17a0e0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f72d0009380 tx=0x7f72d009b830 comp rx=0 tx=0).stop 2024-04-25T00:26:49.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.044+0000 7f72e8bab740 1 -- 172.21.15.107:0/161297678 shutdown_connections 2024-04-25T00:26:49.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.044+0000 7f72e8bab740 1 --2- 172.21.15.107:0/161297678 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55595a17a620 0x55595a17ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.044+0000 7f72e8bab740 1 --2- 172.21.15.107:0/161297678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a16ff50 0x55595a17a0e0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.044+0000 7f72e8bab740 1 --2- 172.21.15.107:0/161297678 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55595a16f630 0x55595a16fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.044+0000 7f72e8bab740 1 -- 172.21.15.107:0/161297678 >> 172.21.15.107:0/161297678 conn(0x55595a16b950 msgr2=0x55595a16bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.044+0000 7f72e8bab740 1 -- 172.21.15.107:0/161297678 shutdown_connections 2024-04-25T00:26:49.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.044+0000 7f72e8bab740 1 -- 172.21.15.107:0/161297678 wait complete. 2024-04-25T00:26:49.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.044+0000 7f72e8bab740 1 -- start start 2024-04-25T00:26:49.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.045+0000 7f72e8bab740 1 Processor -- start 2024-04-25T00:26:49.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.045+0000 7f72e8bab740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55595a16f630 0x55595a177620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.045+0000 7f72e8bab740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55595a16ff50 0x55595a177b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.045+0000 7f72e8bab740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a17a620 0x55595a1780a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.045+0000 7f72e8bab740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55595a17d410 con 0x55595a17a620 2024-04-25T00:26:49.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.045+0000 7f72e8bab740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55595a08ff50 con 0x55595a16ff50 2024-04-25T00:26:49.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.045+0000 7f72e8bab740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55595a17d590 con 0x55595a16f630 2024-04-25T00:26:49.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.045+0000 7f72e8bab740 10 client.0 _ll_register_callbacks cb 0x55595a0cb000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:49.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.045+0000 7f72dffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a17a620 0x55595a1780a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.045+0000 7f72dffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a17a620 0x55595a1780a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53760/0 (socket says 172.21.15.107:53760) 2024-04-25T00:26:49.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.045+0000 7f72dffff640 1 -- 172.21.15.107:0/2478309568 learned_addr learned my addr 172.21.15.107:0/2478309568 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:49.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.045+0000 7f72df7fe640 1 --2- 172.21.15.107:0/2478309568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55595a16f630 0x55595a177620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.045+0000 7f72deffd640 1 --2- 172.21.15.107:0/2478309568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55595a16ff50 0x55595a177b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.045+0000 7f72dffff640 1 -- 172.21.15.107:0/2478309568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55595a16f630 msgr2=0x55595a177620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.045+0000 7f72dffff640 1 --2- 172.21.15.107:0/2478309568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55595a16f630 0x55595a177620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.045+0000 7f72dffff640 1 -- 172.21.15.107:0/2478309568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55595a16ff50 msgr2=0x55595a177b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.045+0000 7f72dffff640 1 --2- 172.21.15.107:0/2478309568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55595a16ff50 0x55595a177b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.045+0000 7f72dffff640 1 -- 172.21.15.107:0/2478309568 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555959de83f0 con 0x55595a17a620 2024-04-25T00:26:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.046+0000 7f72dffff640 1 --2- 172.21.15.107:0/2478309568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a17a620 0x55595a1780a0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f72d8011d30 tx=0x7f72d8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a098c5919945427b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.046+0000 7f72beffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:26:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.046+0000 7f72beffd640 1 -- 172.21.15.107:0/2478309568 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8003970 con 0x55595a17a620 2024-04-25T00:26:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.046+0000 7f72beffd640 1 -- 172.21.15.107:0/2478309568 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f72d8003b10 con 0x55595a17a620 2024-04-25T00:26:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.046+0000 7f72e8bab740 1 -- 172.21.15.107:0/2478309568 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72d00a4b40 con 0x55595a17a620 2024-04-25T00:26:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.046+0000 7f72beffd640 1 -- 172.21.15.107:0/2478309568 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f72d8031050 con 0x55595a17a620 2024-04-25T00:26:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.046+0000 7f72e8bab740 10 client.17868 fetch_fsmap learned FSMap version 34 2024-04-25T00:26:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.046+0000 7f72e8bab740 1 -- 172.21.15.107:0/2478309568 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555959da80f0 con 0x55595a17a620 2024-04-25T00:26:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.046+0000 7f72beffd640 1 -- 172.21.15.107:0/2478309568 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72d00a4b40 con 0x55595a17a620 2024-04-25T00:26:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.046+0000 7f72beffd640 1 -- 172.21.15.107:0/2478309568 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f72d801f4c0 con 0x55595a17a620 2024-04-25T00:26:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.046+0000 7f72e8bab740 10 client.17868 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:26:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.046+0000 7f72e8bab740 10 client.17868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.046+0000 7f72e8bab740 20 client.17868 populate_metadata read hostname 'smithi107' 2024-04-25T00:26:49.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.046+0000 7f72e8bab740 1 --2- 172.21.15.107:0/2478309568 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55595a18a9c0 0x55595a18cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.046+0000 7f72e8bab740 4 client.17868 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.046+0000 7f72e8bab740 1 -- 172.21.15.107:0/2478309568 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55595a0e1bf0 con 0x55595a18a9c0 2024-04-25T00:26:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.047+0000 7f72df7fe640 1 --2- 172.21.15.107:0/2478309568 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55595a18a9c0 0x55595a18cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.047+0000 7f72df7fe640 1 --2- 172.21.15.107:0/2478309568 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55595a18a9c0 0x55595a18cdb0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ce7e192837691be server_cookie=8c297aa93dc44b2a in_seq=0 out_seq=0 2024-04-25T00:26:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.047+0000 7f72beffd640 10 client.17868 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:26:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.048+0000 7f72beffd640 1 -- 172.21.15.107:0/2478309568 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55595a0e1bf0 con 0x55595a18a9c0 2024-04-25T00:26:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.048+0000 7f72beffd640 10 client.17868 handle_command_reply: tid=0 2024-04-25T00:26:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.048+0000 7f72e8bab740 1 client.17868 shutdown 2024-04-25T00:26:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.048+0000 7f72e8bab740 1 -- 172.21.15.107:0/2478309568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a17a620 msgr2=0x55595a1780a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.048+0000 7f72e8bab740 1 --2- 172.21.15.107:0/2478309568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a17a620 0x55595a1780a0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f72d8011d30 tx=0x7f72d8004ab0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.048+0000 7f72e8bab740 1 -- 172.21.15.107:0/2478309568 shutdown_connections 2024-04-25T00:26:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.048+0000 7f72e8bab740 1 --2- 172.21.15.107:0/2478309568 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55595a18a9c0 0x55595a18cdb0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.048+0000 7f72e8bab740 1 --2- 172.21.15.107:0/2478309568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a17a620 0x55595a1780a0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.048+0000 7f72e8bab740 1 --2- 172.21.15.107:0/2478309568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55595a16ff50 0x55595a177b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.048+0000 7f72beffd640 0 client.17868 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:26:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.048+0000 7f72e8bab740 1 --2- 172.21.15.107:0/2478309568 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55595a16f630 0x55595a177620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.048+0000 7f72e8bab740 1 -- 172.21.15.107:0/2478309568 >> 172.21.15.107:0/2478309568 conn(0x55595a16b950 msgr2=0x55595a171750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.048+0000 7f72e8bab740 1 -- 172.21.15.107:0/2478309568 shutdown_connections 2024-04-25T00:26:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.048+0000 7f72e8bab740 1 -- 172.21.15.107:0/2478309568 wait complete. 2024-04-25T00:26:49.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.048+0000 7f72e8bab740 20 client.17868 trim_cache size 0 max 16384 2024-04-25T00:26:49.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.066+0000 7f72e8bab740 1 Processor -- start 2024-04-25T00:26:49.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.066+0000 7f72e8bab740 1 -- start start 2024-04-25T00:26:49.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.066+0000 7f72e8bab740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55595a16f630 0x55595a179d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.066+0000 7f72e8bab740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55595a16ff50 0x55595a1862d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.066+0000 7f72e8bab740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a17a620 0x55595a16a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.066+0000 7f72e8bab740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f72e01b1540 con 0x55595a17a620 2024-04-25T00:26:49.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.066+0000 7f72e8bab740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f72e01aee10 con 0x55595a16f630 2024-04-25T00:26:49.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.066+0000 7f72e8bab740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f72e01acc80 con 0x55595a16ff50 2024-04-25T00:26:49.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.066+0000 7f72dffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a17a620 0x55595a16a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.066+0000 7f72dffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a17a620 0x55595a16a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53766/0 (socket says 172.21.15.107:53766) 2024-04-25T00:26:49.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.066+0000 7f72dffff640 1 -- 172.21.15.107:0/3000474524 learned_addr learned my addr 172.21.15.107:0/3000474524 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:49.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.066+0000 7f72dffff640 1 -- 172.21.15.107:0/3000474524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55595a16ff50 msgr2=0x55595a1862d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.066+0000 7f72df7fe640 1 --2- 172.21.15.107:0/3000474524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55595a16f630 0x55595a179d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.066+0000 7f72dffff640 1 --2- 172.21.15.107:0/3000474524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55595a16ff50 0x55595a1862d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.066+0000 7f72dffff640 1 -- 172.21.15.107:0/3000474524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55595a16f630 msgr2=0x55595a179d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.066+0000 7f72dffff640 1 --2- 172.21.15.107:0/3000474524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55595a16f630 0x55595a179d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.066+0000 7f72dffff640 1 -- 172.21.15.107:0/3000474524 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55595a17cd80 con 0x55595a17a620 2024-04-25T00:26:49.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.066+0000 7f72df7fe640 1 --2- 172.21.15.107:0/3000474524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55595a16f630 0x55595a179d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:49.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.066+0000 7f72dffff640 1 --2- 172.21.15.107:0/3000474524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a17a620 0x55595a16a970 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009a90 tx=0x7f72d8009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae46b0a80cd7c884 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.067+0000 7f72c57fa640 1 -- 172.21.15.107:0/3000474524 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8039020 con 0x55595a17a620 2024-04-25T00:26:49.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.067+0000 7f72c57fa640 1 -- 172.21.15.107:0/3000474524 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72d801ec20 con 0x55595a17a620 2024-04-25T00:26:49.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.067+0000 7f72c57fa640 1 -- 172.21.15.107:0/3000474524 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8004720 con 0x55595a17a620 2024-04-25T00:26:49.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.067+0000 7f72e8bab740 1 -- 172.21.15.107:0/3000474524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a17a620 msgr2=0x55595a16a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.067+0000 7f72e8bab740 1 --2- 172.21.15.107:0/3000474524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a17a620 0x55595a16a970 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009a90 tx=0x7f72d8009690 comp rx=0 tx=0).stop 2024-04-25T00:26:49.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.067+0000 7f72e8bab740 1 -- 172.21.15.107:0/3000474524 shutdown_connections 2024-04-25T00:26:49.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.067+0000 7f72e8bab740 1 --2- 172.21.15.107:0/3000474524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a17a620 0x55595a16a970 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.067+0000 7f72e8bab740 1 --2- 172.21.15.107:0/3000474524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55595a16ff50 0x55595a1862d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.067+0000 7f72e8bab740 1 --2- 172.21.15.107:0/3000474524 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55595a16f630 0x55595a179d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.067+0000 7f72e8bab740 1 -- 172.21.15.107:0/3000474524 >> 172.21.15.107:0/3000474524 conn(0x55595a16b950 msgr2=0x55595a16a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.067+0000 7f72e8bab740 1 -- 172.21.15.107:0/3000474524 shutdown_connections 2024-04-25T00:26:49.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.067+0000 7f72e8bab740 1 -- 172.21.15.107:0/3000474524 wait complete. 2024-04-25T00:26:49.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.067+0000 7f72e8bab740 1 -- start start 2024-04-25T00:26:49.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.068+0000 7f72e8bab740 1 Processor -- start 2024-04-25T00:26:49.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.068+0000 7f72e8bab740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55595a16f630 0x55595a17e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.068+0000 7f72e8bab740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a16ff50 0x55595a17e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.068+0000 7f72e8bab740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55595a17a620 0x55595a17ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.068+0000 7f72e8bab740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f72e01ad2d0 con 0x55595a16ff50 2024-04-25T00:26:49.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.068+0000 7f72e8bab740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f72e01ae740 con 0x55595a16f630 2024-04-25T00:26:49.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.068+0000 7f72e8bab740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f72e01ad9a0 con 0x55595a17a620 2024-04-25T00:26:49.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.068+0000 7f72e8bab740 10 client.0 _ll_register_callbacks cb 0x7f72d8009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:49.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.068+0000 7f72deffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a16ff50 0x55595a17e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.068+0000 7f72deffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a16ff50 0x55595a17e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53780/0 (socket says 172.21.15.107:53780) 2024-04-25T00:26:49.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.068+0000 7f72deffd640 1 -- 172.21.15.107:0/334517185 learned_addr learned my addr 172.21.15.107:0/334517185 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:49.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.068+0000 7f72df7fe640 1 --2- 172.21.15.107:0/334517185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55595a16f630 0x55595a17e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.068+0000 7f72dffff640 1 --2- 172.21.15.107:0/334517185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55595a17a620 0x55595a17ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.068+0000 7f72deffd640 1 -- 172.21.15.107:0/334517185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55595a17a620 msgr2=0x55595a17ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.068+0000 7f72deffd640 1 --2- 172.21.15.107:0/334517185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55595a17a620 0x55595a17ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.068+0000 7f72deffd640 1 -- 172.21.15.107:0/334517185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55595a16f630 msgr2=0x55595a17e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.068+0000 7f72deffd640 1 --2- 172.21.15.107:0/334517185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55595a16f630 0x55595a17e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.068+0000 7f72deffd640 1 -- 172.21.15.107:0/334517185 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55595a1f4300 con 0x55595a16ff50 2024-04-25T00:26:49.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.068+0000 7f72dffff640 1 --2- 172.21.15.107:0/334517185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55595a17a620 0x55595a17ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:49.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.069+0000 7f72deffd640 1 --2- 172.21.15.107:0/334517185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a16ff50 0x55595a17e6a0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f72d00988a0 tx=0x7f72d000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eced3ee9e3c9f48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.069+0000 7f72beffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:26:49.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.069+0000 7f72beffd640 1 -- 172.21.15.107:0/334517185 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d0066060 con 0x55595a16ff50 2024-04-25T00:26:49.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.069+0000 7f72beffd640 1 -- 172.21.15.107:0/334517185 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f72d00a46b0 con 0x55595a16ff50 2024-04-25T00:26:49.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.069+0000 7f72e8bab740 1 -- 172.21.15.107:0/334517185 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72d801ec20 con 0x55595a16ff50 2024-04-25T00:26:49.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.069+0000 7f72beffd640 1 -- 172.21.15.107:0/334517185 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f72d00a4b40 con 0x55595a16ff50 2024-04-25T00:26:49.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.069+0000 7f72e8bab740 10 client.17874 fetch_fsmap learned FSMap version 34 2024-04-25T00:26:49.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.069+0000 7f72e8bab740 1 -- 172.21.15.107:0/334517185 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55595a1f54c0 con 0x55595a16ff50 2024-04-25T00:26:49.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.069+0000 7f72beffd640 1 -- 172.21.15.107:0/334517185 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72d801ec20 con 0x55595a16ff50 2024-04-25T00:26:49.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.069+0000 7f72beffd640 1 -- 172.21.15.107:0/334517185 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f72d0097c20 con 0x55595a16ff50 2024-04-25T00:26:49.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.069+0000 7f72e8bab740 10 client.17874 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:26:49.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.069+0000 7f72e8bab740 10 client.17874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:49.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.069+0000 7f72e8bab740 20 client.17874 populate_metadata read hostname 'smithi107' 2024-04-25T00:26:49.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.069+0000 7f72e8bab740 1 --2- 172.21.15.107:0/334517185 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55595a1f57a0 0x55595a1f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.069+0000 7f72e8bab740 4 client.17874 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T00:26:49.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.069+0000 7f72e8bab740 1 -- 172.21.15.107:0/334517185 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55595a0dbf40 con 0x55595a1f57a0 2024-04-25T00:26:49.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.070+0000 7f72df7fe640 1 --2- 172.21.15.107:0/334517185 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55595a1f57a0 0x55595a1f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.070+0000 7f72df7fe640 1 --2- 172.21.15.107:0/334517185 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55595a1f57a0 0x55595a1f7b90 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2713640637caa71d server_cookie=2b729516b16a4bde in_seq=0 out_seq=0 2024-04-25T00:26:49.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.070+0000 7f72beffd640 10 client.17874 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:26:49.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.070+0000 7f72beffd640 1 -- 172.21.15.107:0/334517185 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55595a0dbf40 con 0x55595a1f57a0 2024-04-25T00:26:49.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.070+0000 7f72beffd640 10 client.17874 handle_command_reply: tid=0 2024-04-25T00:26:49.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.070+0000 7f72e8bab740 1 client.17874 shutdown 2024-04-25T00:26:49.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.070+0000 7f72e8bab740 1 -- 172.21.15.107:0/334517185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a16ff50 msgr2=0x55595a17e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.070+0000 7f72e8bab740 1 --2- 172.21.15.107:0/334517185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a16ff50 0x55595a17e6a0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f72d00988a0 tx=0x7f72d000cf20 comp rx=0 tx=0).stop 2024-04-25T00:26:49.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.070+0000 7f72e8bab740 1 -- 172.21.15.107:0/334517185 shutdown_connections 2024-04-25T00:26:49.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.070+0000 7f72e8bab740 1 --2- 172.21.15.107:0/334517185 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55595a1f57a0 0x55595a1f7b90 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.070+0000 7f72e8bab740 1 --2- 172.21.15.107:0/334517185 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55595a17a620 0x55595a17ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.070+0000 7f72e8bab740 1 --2- 172.21.15.107:0/334517185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55595a16ff50 0x55595a17e6a0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.070+0000 7f72e8bab740 1 --2- 172.21.15.107:0/334517185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55595a16f630 0x55595a17e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.070+0000 7f72beffd640 0 client.17874 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:26:49.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.070+0000 7f72e8bab740 1 -- 172.21.15.107:0/334517185 >> 172.21.15.107:0/334517185 conn(0x55595a16b950 msgr2=0x55595a16aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.070+0000 7f72e8bab740 1 -- 172.21.15.107:0/334517185 shutdown_connections 2024-04-25T00:26:49.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.070+0000 7f72e8bab740 1 -- 172.21.15.107:0/334517185 wait complete. 2024-04-25T00:26:49.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.071+0000 7f72e8bab740 20 client.17874 trim_cache size 0 max 16384 2024-04-25T00:26:49.079 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:26:49.079 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-04-25T00:26:49.079 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-04-25T00:26:49.080 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:26:49.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.071+0000 7f72e5e9e640 1 -- 172.21.15.107:0/1627580397 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f72c80a2d00 msgr2=0x7f72c80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.071+0000 7f72e5e9e640 1 --2- 172.21.15.107:0/1627580397 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f72c80a2d00 0x7f72c80a51c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f72d8005fd0 tx=0x7f72d8009500 comp rx=0 tx=0).stop 2024-04-25T00:26:49.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.071+0000 7f72e5e9e640 1 -- 172.21.15.107:0/1627580397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72e0100fd0 msgr2=0x7f72e01993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.071+0000 7f72e5e9e640 1 --2- 172.21.15.107:0/1627580397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72e0100fd0 0x7f72e01993b0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f72d000e9f0 tx=0x7f72d000edf0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.071+0000 7f72e5e9e640 1 -- 172.21.15.107:0/1627580397 shutdown_connections 2024-04-25T00:26:49.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.071+0000 7f72e5e9e640 1 --2- 172.21.15.107:0/1627580397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f72e01066b0 0x7f72e01a3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.071+0000 7f72e5e9e640 1 --2- 172.21.15.107:0/1627580397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f72e0100fd0 0x7f72e01993b0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.071+0000 7f72e5e9e640 1 --2- 172.21.15.107:0/1627580397 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f72c80a2d00 0x7f72c80a51c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.071+0000 7f72e5e9e640 1 --2- 172.21.15.107:0/1627580397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72e0100610 0x7f72e0198e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.071+0000 7f72e5e9e640 1 -- 172.21.15.107:0/1627580397 >> 172.21.15.107:0/1627580397 conn(0x7f72e0077740 msgr2=0x7f72e00ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.071+0000 7f72e5e9e640 1 -- 172.21.15.107:0/1627580397 shutdown_connections 2024-04-25T00:26:49.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.071+0000 7f72e5e9e640 1 -- 172.21.15.107:0/1627580397 wait complete. 2024-04-25T00:26:49.084 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T00:26:49.084 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:26:49.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.182+0000 7f5442c5d640 1 -- 172.21.15.107:0/3316456238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f543c101710 msgr2=0x7f543c101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.182+0000 7f5442c5d640 1 --2- 172.21.15.107:0/3316456238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f543c101710 0x7f543c101b90 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f542c009930 tx=0x7f542c02a420 comp rx=0 tx=0).stop 2024-04-25T00:26:49.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.182+0000 7f5442c5d640 1 -- 172.21.15.107:0/3316456238 shutdown_connections 2024-04-25T00:26:49.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.182+0000 7f5442c5d640 1 --2- 172.21.15.107:0/3316456238 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f543c1020d0 0x7f543c10e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.182+0000 7f5442c5d640 1 --2- 172.21.15.107:0/3316456238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f543c101710 0x7f543c101b90 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.182+0000 7f5442c5d640 1 --2- 172.21.15.107:0/3316456238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f543c107720 0x7f543c107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.182+0000 7f5442c5d640 1 -- 172.21.15.107:0/3316456238 >> 172.21.15.107:0/3316456238 conn(0x7f543c078b00 msgr2=0x7f543c078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.182+0000 7f5442c5d640 1 -- 172.21.15.107:0/3316456238 shutdown_connections 2024-04-25T00:26:49.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.182+0000 7f5442c5d640 1 -- 172.21.15.107:0/3316456238 wait complete. 2024-04-25T00:26:49.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.183+0000 7f5442c5d640 1 Processor -- start 2024-04-25T00:26:49.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.183+0000 7f5442c5d640 1 -- start start 2024-04-25T00:26:49.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.183+0000 7f5442c5d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f543c101710 0x7f543c198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.183+0000 7f5442c5d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f543c1020d0 0x7f543c198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.183+0000 7f5442c5d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f543c107720 0x7f543c19c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.183+0000 7f5442c5d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f543c111b30 con 0x7f543c101710 2024-04-25T00:26:49.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.183+0000 7f5442c5d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f543c1119b0 con 0x7f543c107720 2024-04-25T00:26:49.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.183+0000 7f5442c5d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f543c111cb0 con 0x7f543c1020d0 2024-04-25T00:26:49.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.183+0000 7f54409f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f543c101710 0x7f543c198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.183+0000 7f54409f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f543c101710 0x7f543c198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53792/0 (socket says 172.21.15.107:53792) 2024-04-25T00:26:49.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.183+0000 7f54409f9640 1 -- 172.21.15.107:0/2132240462 learned_addr learned my addr 172.21.15.107:0/2132240462 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:49.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.183+0000 7f54411fa640 1 --2- 172.21.15.107:0/2132240462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f543c107720 0x7f543c19c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.183+0000 7f54409f9640 1 -- 172.21.15.107:0/2132240462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f543c1020d0 msgr2=0x7f543c198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.183+0000 7f54409f9640 1 --2- 172.21.15.107:0/2132240462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f543c1020d0 0x7f543c198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.183+0000 7f54409f9640 1 -- 172.21.15.107:0/2132240462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f543c107720 msgr2=0x7f543c19c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.183+0000 7f54409f9640 1 --2- 172.21.15.107:0/2132240462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f543c107720 0x7f543c19c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.183+0000 7f54409f9640 1 -- 172.21.15.107:0/2132240462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f543c19d000 con 0x7f543c101710 2024-04-25T00:26:49.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.183+0000 7f54411fa640 1 --2- 172.21.15.107:0/2132240462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f543c107720 0x7f543c19c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:49.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.183+0000 7f54409f9640 1 --2- 172.21.15.107:0/2132240462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f543c101710 0x7f543c198010 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f543800d950 tx=0x7f543800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.184+0000 7f5431ffb640 1 -- 172.21.15.107:0/2132240462 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438017070 con 0x7f543c101710 2024-04-25T00:26:49.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.184+0000 7f5431ffb640 1 -- 172.21.15.107:0/2132240462 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5438009c30 con 0x7f543c101710 2024-04-25T00:26:49.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.184+0000 7f5442c5d640 1 -- 172.21.15.107:0/2132240462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f543c19d2f0 con 0x7f543c101710 2024-04-25T00:26:49.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.184+0000 7f5431ffb640 1 -- 172.21.15.107:0/2132240462 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543800f6b0 con 0x7f543c101710 2024-04-25T00:26:49.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.184+0000 7f5442c5d640 1 -- 172.21.15.107:0/2132240462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f543c1a4b80 con 0x7f543c101710 2024-04-25T00:26:49.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.185+0000 7f5442c5d640 1 -- 172.21.15.107:0/2132240462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f543c0689b0 con 0x7f543c101710 2024-04-25T00:26:49.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.186+0000 7f5431ffb640 1 -- 172.21.15.107:0/2132240462 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f54380109d0 con 0x7f543c101710 2024-04-25T00:26:49.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.186+0000 7f5431ffb640 1 --2- 172.21.15.107:0/2132240462 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f54240a2d00 0x7f54240a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.186+0000 7f5433fff640 1 --2- 172.21.15.107:0/2132240462 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f54240a2d00 0x7f54240a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.186+0000 7f5431ffb640 1 -- 172.21.15.107:0/2132240462 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f54380cf460 con 0x7f543c101710 2024-04-25T00:26:49.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.187+0000 7f5433fff640 1 --2- 172.21.15.107:0/2132240462 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f54240a2d00 0x7f54240a51c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f542c02a940 tx=0x7f542c000bc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.191+0000 7f5431ffb640 1 -- 172.21.15.107:0/2132240462 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5438009de0 con 0x7f543c101710 2024-04-25T00:26:49.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.404+0000 7f5442c5d640 1 -- 172.21.15.107:0/2132240462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f543c199360 con 0x7f543c101710 2024-04-25T00:26:49.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.404+0000 7f5431ffb640 1 -- 172.21.15.107:0/2132240462 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11897 (secure 0 0 0) 0x7f543809bcc0 con 0x7f543c101710 2024-04-25T00:26:49.405 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:26:49.405 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:22:27.402680+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24493,24521]},"id":1}]} 2024-04-25T00:26:49.405 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 34 2024-04-25T00:26:49.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.406+0000 7f5442c5d640 1 -- 172.21.15.107:0/2132240462 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f54240a2d00 msgr2=0x7f54240a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.406+0000 7f5442c5d640 1 --2- 172.21.15.107:0/2132240462 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f54240a2d00 0x7f54240a51c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f542c02a940 tx=0x7f542c000bc0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.406+0000 7f5442c5d640 1 -- 172.21.15.107:0/2132240462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f543c101710 msgr2=0x7f543c198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.406+0000 7f5442c5d640 1 --2- 172.21.15.107:0/2132240462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f543c101710 0x7f543c198010 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f543800d950 tx=0x7f543800de10 comp rx=0 tx=0).stop 2024-04-25T00:26:49.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.406+0000 7f5442c5d640 1 -- 172.21.15.107:0/2132240462 shutdown_connections 2024-04-25T00:26:49.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.406+0000 7f5442c5d640 1 --2- 172.21.15.107:0/2132240462 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f54240a2d00 0x7f54240a51c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.406+0000 7f5442c5d640 1 --2- 172.21.15.107:0/2132240462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f543c107720 0x7f543c19c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.406+0000 7f5442c5d640 1 --2- 172.21.15.107:0/2132240462 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f543c1020d0 0x7f543c198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.406+0000 7f5442c5d640 1 --2- 172.21.15.107:0/2132240462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f543c101710 0x7f543c198010 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.406+0000 7f5442c5d640 1 -- 172.21.15.107:0/2132240462 >> 172.21.15.107:0/2132240462 conn(0x7f543c078b00 msgr2=0x7f543c104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.407+0000 7f5442c5d640 1 -- 172.21.15.107:0/2132240462 shutdown_connections 2024-04-25T00:26:49.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.407+0000 7f5442c5d640 1 -- 172.21.15.107:0/2132240462 wait complete. 2024-04-25T00:26:49.418 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-25T00:26:49.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.518+0000 7fdb46ce1640 1 -- 172.21.15.107:0/2390462402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb401020d0 msgr2=0x7fdb4010e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.518+0000 7fdb46ce1640 1 --2- 172.21.15.107:0/2390462402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb401020d0 0x7fdb4010e5e0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009cd0 tx=0x7fdb3802a8e0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.518+0000 7fdb46ce1640 1 -- 172.21.15.107:0/2390462402 shutdown_connections 2024-04-25T00:26:49.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.518+0000 7fdb46ce1640 1 --2- 172.21.15.107:0/2390462402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb401020d0 0x7fdb4010e5e0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.518+0000 7fdb46ce1640 1 --2- 172.21.15.107:0/2390462402 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb40101710 0x7fdb40101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.518+0000 7fdb46ce1640 1 --2- 172.21.15.107:0/2390462402 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb40107720 0x7fdb40107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.518+0000 7fdb46ce1640 1 -- 172.21.15.107:0/2390462402 >> 172.21.15.107:0/2390462402 conn(0x7fdb400fd180 msgr2=0x7fdb400ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.518+0000 7fdb46ce1640 1 -- 172.21.15.107:0/2390462402 shutdown_connections 2024-04-25T00:26:49.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.518+0000 7fdb46ce1640 1 -- 172.21.15.107:0/2390462402 wait complete. 2024-04-25T00:26:49.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.518+0000 7fdb46ce1640 1 Processor -- start 2024-04-25T00:26:49.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.518+0000 7fdb46ce1640 1 -- start start 2024-04-25T00:26:49.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.518+0000 7fdb46ce1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb40101710 0x7fdb40198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.518+0000 7fdb46ce1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb401020d0 0x7fdb401985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.519+0000 7fdb46ce1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb40107720 0x7fdb4019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.519+0000 7fdb46ce1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb40111b30 con 0x7fdb401020d0 2024-04-25T00:26:49.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.519+0000 7fdb46ce1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb401119b0 con 0x7fdb40101710 2024-04-25T00:26:49.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.519+0000 7fdb46ce1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdb40111cb0 con 0x7fdb40107720 2024-04-25T00:26:49.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.519+0000 7fdb3ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb401020d0 0x7fdb401985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.519+0000 7fdb3ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb401020d0 0x7fdb401985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56854/0 (socket says 172.21.15.107:56854) 2024-04-25T00:26:49.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.519+0000 7fdb3ffff640 1 -- 172.21.15.107:0/266810263 learned_addr learned my addr 172.21.15.107:0/266810263 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:49.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.519+0000 7fdb4527e640 1 --2- 172.21.15.107:0/266810263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb40107720 0x7fdb4019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.519+0000 7fdb44a7d640 1 --2- 172.21.15.107:0/266810263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb40101710 0x7fdb40198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.519+0000 7fdb3ffff640 1 -- 172.21.15.107:0/266810263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb40107720 msgr2=0x7fdb4019c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.519+0000 7fdb3ffff640 1 --2- 172.21.15.107:0/266810263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb40107720 0x7fdb4019c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.519+0000 7fdb3ffff640 1 -- 172.21.15.107:0/266810263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb40101710 msgr2=0x7fdb40198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.519+0000 7fdb3ffff640 1 --2- 172.21.15.107:0/266810263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb40101710 0x7fdb40198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.519+0000 7fdb3ffff640 1 -- 172.21.15.107:0/266810263 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4019d060 con 0x7fdb401020d0 2024-04-25T00:26:49.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.519+0000 7fdb44a7d640 1 --2- 172.21.15.107:0/266810263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb40101710 0x7fdb40198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:49.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.519+0000 7fdb3ffff640 1 --2- 172.21.15.107:0/266810263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb401020d0 0x7fdb401985b0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c00e9f0 tx=0x7fdb2c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.519+0000 7fdb3dffb640 1 -- 172.21.15.107:0/266810263 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c00cd80 con 0x7fdb401020d0 2024-04-25T00:26:49.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.519+0000 7fdb3dffb640 1 -- 172.21.15.107:0/266810263 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb2c005500 con 0x7fdb401020d0 2024-04-25T00:26:49.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.519+0000 7fdb46ce1640 1 -- 172.21.15.107:0/266810263 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb4019d350 con 0x7fdb401020d0 2024-04-25T00:26:49.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.519+0000 7fdb3dffb640 1 -- 172.21.15.107:0/266810263 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c010690 con 0x7fdb401020d0 2024-04-25T00:26:49.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.519+0000 7fdb46ce1640 1 -- 172.21.15.107:0/266810263 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb401a4be0 con 0x7fdb401020d0 2024-04-25T00:26:49.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.521+0000 7fdb3dffb640 1 -- 172.21.15.107:0/266810263 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fdb2c0046c0 con 0x7fdb401020d0 2024-04-25T00:26:49.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.522+0000 7fdb499f3740 1 Processor -- start 2024-04-25T00:26:49.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.522+0000 7fdb499f3740 1 -- start start 2024-04-25T00:26:49.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.522+0000 7fdb3dffb640 1 --2- 172.21.15.107:0/266810263 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdb300a2d00 0x7fdb300a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.522+0000 7fdb499f3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b65dfe1580 0x55b65dfe1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.522+0000 7fdb499f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfe1ea0 0x55b65dfec030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.522+0000 7fdb499f3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b65dfec570 0x55b65dfee960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.522+0000 7fdb499f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b65df46670 con 0x55b65dfe1ea0 2024-04-25T00:26:49.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.522+0000 7fdb499f3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b65deee970 con 0x55b65dfe1580 2024-04-25T00:26:49.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.522+0000 7fdb499f3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b65df42630 con 0x55b65dfec570 2024-04-25T00:26:49.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.522+0000 7fdb4527e640 1 --2- 172.21.15.107:0/266810263 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdb300a2d00 0x7fdb300a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.522+0000 7fdb3dffb640 1 -- 172.21.15.107:0/266810263 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7fdb2c0cfd50 con 0x7fdb401020d0 2024-04-25T00:26:49.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.522+0000 7fdb3ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfe1ea0 0x55b65dfec030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.522+0000 7fdb3ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfe1ea0 0x55b65dfec030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56870/0 (socket says 172.21.15.107:56870) 2024-04-25T00:26:49.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.522+0000 7fdb3ffff640 1 -- 172.21.15.107:0/2590067475 learned_addr learned my addr 172.21.15.107:0/2590067475 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:49.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.522+0000 7fdb44a7d640 1 --2- 172.21.15.107:0/2590067475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b65dfe1580 0x55b65dfe1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.522+0000 7fdb4527e640 1 --2- 172.21.15.107:0/2590067475 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b65dfec570 0x55b65dfee960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.523+0000 7fdb3ffff640 1 -- 172.21.15.107:0/2590067475 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b65dfec570 msgr2=0x55b65dfee960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.523+0000 7fdb3ffff640 1 --2- 172.21.15.107:0/2590067475 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b65dfec570 0x55b65dfee960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.523+0000 7fdb3ffff640 1 -- 172.21.15.107:0/2590067475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b65dfe1580 msgr2=0x55b65dfe1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.523+0000 7fdb3ffff640 1 --2- 172.21.15.107:0/2590067475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b65dfe1580 0x55b65dfe1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.523+0000 7fdb3ffff640 1 -- 172.21.15.107:0/2590067475 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b65df12400 con 0x55b65dfe1ea0 2024-04-25T00:26:49.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.523+0000 7fdb4527e640 1 --2- 172.21.15.107:0/266810263 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdb300a2d00 0x7fdb300a51c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009a60 tx=0x7fdb38005ef0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.523+0000 7fdb3ffff640 1 --2- 172.21.15.107:0/2590067475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfe1ea0 0x55b65dfec030 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0a45f0 tx=0x7fdb2c09b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4169d5d2112d9cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.523+0000 7fdb2affd640 1 -- 172.21.15.107:0/2590067475 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c09beb0 con 0x55b65dfe1ea0 2024-04-25T00:26:49.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.523+0000 7fdb2affd640 1 -- 172.21.15.107:0/2590067475 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb2c0a0260 con 0x55b65dfe1ea0 2024-04-25T00:26:49.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.523+0000 7fdb2affd640 1 -- 172.21.15.107:0/2590067475 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c05d520 con 0x55b65dfe1ea0 2024-04-25T00:26:49.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.523+0000 7fdb499f3740 1 -- 172.21.15.107:0/2590067475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfe1ea0 msgr2=0x55b65dfec030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.523+0000 7fdb499f3740 1 --2- 172.21.15.107:0/2590067475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfe1ea0 0x55b65dfec030 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0a45f0 tx=0x7fdb2c09b9f0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.524+0000 7fdb499f3740 1 -- 172.21.15.107:0/2590067475 shutdown_connections 2024-04-25T00:26:49.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.524+0000 7fdb499f3740 1 --2- 172.21.15.107:0/2590067475 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b65dfec570 0x55b65dfee960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.524+0000 7fdb499f3740 1 --2- 172.21.15.107:0/2590067475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfe1ea0 0x55b65dfec030 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.524+0000 7fdb499f3740 1 --2- 172.21.15.107:0/2590067475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b65dfe1580 0x55b65dfe1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.524+0000 7fdb499f3740 1 -- 172.21.15.107:0/2590067475 >> 172.21.15.107:0/2590067475 conn(0x55b65dfdd8a0 msgr2=0x55b65dfddc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.524+0000 7fdb499f3740 1 -- 172.21.15.107:0/2590067475 shutdown_connections 2024-04-25T00:26:49.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.524+0000 7fdb499f3740 1 -- 172.21.15.107:0/2590067475 wait complete. 2024-04-25T00:26:49.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.524+0000 7fdb499f3740 1 -- start start 2024-04-25T00:26:49.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.524+0000 7fdb499f3740 1 Processor -- start 2024-04-25T00:26:49.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.524+0000 7fdb499f3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b65dfe1580 0x55b65dfebf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.524+0000 7fdb499f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfe1ea0 0x55b65dfe91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.524+0000 7fdb499f3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b65dfec570 0x55b65dfe9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.524+0000 7fdb499f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b65dfef360 con 0x55b65dfe1ea0 2024-04-25T00:26:49.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.524+0000 7fdb499f3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b65df02050 con 0x55b65dfe1580 2024-04-25T00:26:49.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.524+0000 7fdb499f3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55b65dfef4e0 con 0x55b65dfec570 2024-04-25T00:26:49.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.524+0000 7fdb499f3740 10 client.0 _ll_register_callbacks cb 0x55b65dc59be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:49.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.524+0000 7fdb3ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfe1ea0 0x55b65dfe91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.524+0000 7fdb3ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfe1ea0 0x55b65dfe91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56874/0 (socket says 172.21.15.107:56874) 2024-04-25T00:26:49.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.524+0000 7fdb3ffff640 1 -- 172.21.15.107:0/1856641377 learned_addr learned my addr 172.21.15.107:0/1856641377 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:49.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.525+0000 7fdb3ffff640 1 -- 172.21.15.107:0/1856641377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b65dfec570 msgr2=0x55b65dfe9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.525+0000 7fdb3ffff640 1 --2- 172.21.15.107:0/1856641377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b65dfec570 0x55b65dfe9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.525+0000 7fdb3ffff640 1 -- 172.21.15.107:0/1856641377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b65dfe1580 msgr2=0x55b65dfebf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.525+0000 7fdb3ffff640 1 --2- 172.21.15.107:0/1856641377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b65dfe1580 0x55b65dfebf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.525+0000 7fdb3ffff640 1 -- 172.21.15.107:0/1856641377 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b65ded3dc0 con 0x55b65dfe1ea0 2024-04-25T00:26:49.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.525+0000 7fdb3ffff640 1 --2- 172.21.15.107:0/1856641377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfe1ea0 0x55b65dfe91f0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c098310 tx=0x7fdb2c0cd520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a764250fa1ec5ddf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.525+0000 7fdb1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:26:49.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.525+0000 7fdb1ffff640 1 -- 172.21.15.107:0/1856641377 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c09beb0 con 0x55b65dfe1ea0 2024-04-25T00:26:49.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.525+0000 7fdb1ffff640 1 -- 172.21.15.107:0/1856641377 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7fdb2c0610d0 con 0x55b65dfe1ea0 2024-04-25T00:26:49.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.525+0000 7fdb499f3740 1 -- 172.21.15.107:0/1856641377 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb2c0a0260 con 0x55b65dfe1ea0 2024-04-25T00:26:49.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.525+0000 7fdb1ffff640 1 -- 172.21.15.107:0/1856641377 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdb2c0a4690 con 0x55b65dfe1ea0 2024-04-25T00:26:49.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.525+0000 7fdb499f3740 10 client.17892 fetch_fsmap learned FSMap version 34 2024-04-25T00:26:49.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.525+0000 7fdb499f3740 1 -- 172.21.15.107:0/1856641377 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b65dc5a3f0 con 0x55b65dfe1ea0 2024-04-25T00:26:49.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.525+0000 7fdb1ffff640 1 -- 172.21.15.107:0/1856641377 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb2c0a0260 con 0x55b65dfe1ea0 2024-04-25T00:26:49.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.525+0000 7fdb1ffff640 1 -- 172.21.15.107:0/1856641377 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fdb2c09b300 con 0x55b65dfe1ea0 2024-04-25T00:26:49.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.525+0000 7fdb499f3740 10 client.17892 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:26:49.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.525+0000 7fdb499f3740 10 client.17892 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:26:49.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.525+0000 7fdb499f3740 20 client.17892 populate_metadata read hostname 'smithi107' 2024-04-25T00:26:49.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.525+0000 7fdb499f3740 1 --2- 172.21.15.107:0/1856641377 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55b65dffc470 0x55b65dffe860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.525+0000 7fdb499f3740 4 client.17892 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:49.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.525+0000 7fdb499f3740 1 -- 172.21.15.107:0/1856641377 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b65df53bb0 con 0x55b65dffc470 2024-04-25T00:26:49.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.526+0000 7fdb44a7d640 1 --2- 172.21.15.107:0/1856641377 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55b65dffc470 0x55b65dffe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.526+0000 7fdb1ffff640 10 client.17892 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:26:49.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.526+0000 7fdb44a7d640 1 --2- 172.21.15.107:0/1856641377 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55b65dffc470 0x55b65dffe860 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=23bb37e3a1b83a36 server_cookie=da899dd1978d855c in_seq=0 out_seq=0 2024-04-25T00:26:49.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.527+0000 7fdb1ffff640 1 -- 172.21.15.107:0/1856641377 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b65df53bb0 con 0x55b65dffc470 2024-04-25T00:26:49.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.527+0000 7fdb1ffff640 10 client.17892 handle_command_reply: tid=0 2024-04-25T00:26:49.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.527+0000 7fdb499f3740 1 client.17892 shutdown 2024-04-25T00:26:49.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.527+0000 7fdb499f3740 1 -- 172.21.15.107:0/1856641377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfe1ea0 msgr2=0x55b65dfe91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.527+0000 7fdb499f3740 1 --2- 172.21.15.107:0/1856641377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfe1ea0 0x55b65dfe91f0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c098310 tx=0x7fdb2c0cd520 comp rx=0 tx=0).stop 2024-04-25T00:26:49.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.527+0000 7fdb499f3740 1 -- 172.21.15.107:0/1856641377 shutdown_connections 2024-04-25T00:26:49.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.527+0000 7fdb499f3740 1 --2- 172.21.15.107:0/1856641377 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55b65dffc470 0x55b65dffe860 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.527+0000 7fdb499f3740 1 --2- 172.21.15.107:0/1856641377 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b65dfec570 0x55b65dfe9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.527+0000 7fdb499f3740 1 --2- 172.21.15.107:0/1856641377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfe1ea0 0x55b65dfe91f0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.527+0000 7fdb1ffff640 0 client.17892 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:26:49.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.527+0000 7fdb499f3740 1 --2- 172.21.15.107:0/1856641377 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b65dfe1580 0x55b65dfebf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.527+0000 7fdb499f3740 1 -- 172.21.15.107:0/1856641377 >> 172.21.15.107:0/1856641377 conn(0x55b65dfdd8a0 msgr2=0x55b65dfdcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.527+0000 7fdb499f3740 1 -- 172.21.15.107:0/1856641377 shutdown_connections 2024-04-25T00:26:49.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.527+0000 7fdb499f3740 1 -- 172.21.15.107:0/1856641377 wait complete. 2024-04-25T00:26:49.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.528+0000 7fdb499f3740 20 client.17892 trim_cache size 0 max 16384 2024-04-25T00:26:49.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.541+0000 7fdb499f3740 1 Processor -- start 2024-04-25T00:26:49.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.541+0000 7fdb499f3740 1 -- start start 2024-04-25T00:26:49.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.541+0000 7fdb499f3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b65dfe1580 0x55b65dfdc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.541+0000 7fdb499f3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b65dfe1ea0 0x55b65dfeba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.541+0000 7fdb499f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfec570 0x55b65e001890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.541+0000 7fdb499f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb401b1500 con 0x55b65dfec570 2024-04-25T00:26:49.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.541+0000 7fdb499f3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb401aedd0 con 0x55b65dfe1580 2024-04-25T00:26:49.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.541+0000 7fdb499f3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdb401acc40 con 0x55b65dfe1ea0 2024-04-25T00:26:49.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.541+0000 7fdb4527e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfec570 0x55b65e001890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.541+0000 7fdb4527e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfec570 0x55b65e001890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56882/0 (socket says 172.21.15.107:56882) 2024-04-25T00:26:49.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.541+0000 7fdb4527e640 1 -- 172.21.15.107:0/767735101 learned_addr learned my addr 172.21.15.107:0/767735101 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:49.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.541+0000 7fdb44a7d640 1 --2- 172.21.15.107:0/767735101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b65dfe1580 0x55b65dfdc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.541+0000 7fdb4527e640 1 -- 172.21.15.107:0/767735101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b65dfe1ea0 msgr2=0x55b65dfeba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.541+0000 7fdb4527e640 1 --2- 172.21.15.107:0/767735101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b65dfe1ea0 0x55b65dfeba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.541+0000 7fdb4527e640 1 -- 172.21.15.107:0/767735101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b65dfe1580 msgr2=0x55b65dfdc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.541+0000 7fdb4527e640 1 --2- 172.21.15.107:0/767735101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b65dfe1580 0x55b65dfdc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.541+0000 7fdb4527e640 1 -- 172.21.15.107:0/767735101 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b65dfe9ae0 con 0x55b65dfec570 2024-04-25T00:26:49.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.541+0000 7fdb4527e640 1 --2- 172.21.15.107:0/767735101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfec570 0x55b65e001890 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fdb3802ae00 tx=0x7fdb3803f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2e94da31a33432 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.541+0000 7fdb2a7fc640 1 -- 172.21.15.107:0/767735101 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38043070 con 0x55b65dfec570 2024-04-25T00:26:49.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.541+0000 7fdb2a7fc640 1 -- 172.21.15.107:0/767735101 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb38036040 con 0x55b65dfec570 2024-04-25T00:26:49.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.541+0000 7fdb2a7fc640 1 -- 172.21.15.107:0/767735101 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb380309c0 con 0x55b65dfec570 2024-04-25T00:26:49.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.542+0000 7fdb499f3740 1 -- 172.21.15.107:0/767735101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfec570 msgr2=0x55b65e001890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.542+0000 7fdb499f3740 1 --2- 172.21.15.107:0/767735101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfec570 0x55b65e001890 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fdb3802ae00 tx=0x7fdb3803f070 comp rx=0 tx=0).stop 2024-04-25T00:26:49.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.542+0000 7fdb499f3740 1 -- 172.21.15.107:0/767735101 shutdown_connections 2024-04-25T00:26:49.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.542+0000 7fdb499f3740 1 --2- 172.21.15.107:0/767735101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfec570 0x55b65e001890 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.542+0000 7fdb499f3740 1 --2- 172.21.15.107:0/767735101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b65dfe1ea0 0x55b65dfeba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.542+0000 7fdb499f3740 1 --2- 172.21.15.107:0/767735101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b65dfe1580 0x55b65dfdc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.542+0000 7fdb499f3740 1 -- 172.21.15.107:0/767735101 >> 172.21.15.107:0/767735101 conn(0x55b65dfdd8a0 msgr2=0x55b65dfdc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.542+0000 7fdb499f3740 1 -- 172.21.15.107:0/767735101 shutdown_connections 2024-04-25T00:26:49.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.542+0000 7fdb499f3740 1 -- 172.21.15.107:0/767735101 wait complete. 2024-04-25T00:26:49.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.542+0000 7fdb499f3740 1 -- start start 2024-04-25T00:26:49.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.542+0000 7fdb499f3740 1 Processor -- start 2024-04-25T00:26:49.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.543+0000 7fdb499f3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b65dfe1580 0x55b65e0567d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.543+0000 7fdb499f3740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b65dfe1ea0 0x55b65e056d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.543+0000 7fdb499f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfec570 0x55b65e05aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.543+0000 7fdb499f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb401ad290 con 0x55b65dfec570 2024-04-25T00:26:49.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.543+0000 7fdb499f3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb401ae700 con 0x55b65dfe1580 2024-04-25T00:26:49.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.543+0000 7fdb499f3740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fdb401ad960 con 0x55b65dfe1ea0 2024-04-25T00:26:49.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.543+0000 7fdb499f3740 10 client.0 _ll_register_callbacks cb 0x7fdb340092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:49.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.543+0000 7fdb4527e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfec570 0x55b65e05aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.543+0000 7fdb4527e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfec570 0x55b65e05aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56898/0 (socket says 172.21.15.107:56898) 2024-04-25T00:26:49.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.543+0000 7fdb4527e640 1 -- 172.21.15.107:0/1660148454 learned_addr learned my addr 172.21.15.107:0/1660148454 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:49.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.543+0000 7fdb44a7d640 1 --2- 172.21.15.107:0/1660148454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b65dfe1580 0x55b65e0567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.543+0000 7fdb4527e640 1 -- 172.21.15.107:0/1660148454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b65dfe1ea0 msgr2=0x55b65e056d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.543+0000 7fdb3ffff640 1 --2- 172.21.15.107:0/1660148454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b65dfe1ea0 0x55b65e056d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.543+0000 7fdb4527e640 1 --2- 172.21.15.107:0/1660148454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b65dfe1ea0 0x55b65e056d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.543+0000 7fdb4527e640 1 -- 172.21.15.107:0/1660148454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b65dfe1580 msgr2=0x55b65e0567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.543+0000 7fdb4527e640 1 --2- 172.21.15.107:0/1660148454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b65dfe1580 0x55b65e0567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.543+0000 7fdb4527e640 1 -- 172.21.15.107:0/1660148454 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b65dd247d0 con 0x55b65dfec570 2024-04-25T00:26:49.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.543+0000 7fdb4527e640 1 --2- 172.21.15.107:0/1660148454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfec570 0x55b65e05aba0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fdb380097b0 tx=0x7fdb38004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3c6e49a9e926a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.543+0000 7fdb1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:26:49.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.543+0000 7fdb1ffff640 1 -- 172.21.15.107:0/1660148454 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38043040 con 0x55b65dfec570 2024-04-25T00:26:49.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.543+0000 7fdb1ffff640 1 -- 172.21.15.107:0/1660148454 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7fdb3804c040 con 0x55b65dfec570 2024-04-25T00:26:49.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.543+0000 7fdb499f3740 1 -- 172.21.15.107:0/1660148454 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb38036040 con 0x55b65dfec570 2024-04-25T00:26:49.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.544+0000 7fdb1ffff640 1 -- 172.21.15.107:0/1660148454 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdb38066050 con 0x55b65dfec570 2024-04-25T00:26:49.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.544+0000 7fdb499f3740 10 client.17898 fetch_fsmap learned FSMap version 34 2024-04-25T00:26:49.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.544+0000 7fdb499f3740 1 -- 172.21.15.107:0/1660148454 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b65e0635f0 con 0x55b65dfec570 2024-04-25T00:26:49.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.544+0000 7fdb1ffff640 1 -- 172.21.15.107:0/1660148454 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb38036040 con 0x55b65dfec570 2024-04-25T00:26:49.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.544+0000 7fdb1ffff640 1 -- 172.21.15.107:0/1660148454 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fdb38030070 con 0x55b65dfec570 2024-04-25T00:26:49.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.544+0000 7fdb499f3740 10 client.17898 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:26:49.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.544+0000 7fdb499f3740 10 client.17898 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T00:26:49.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.544+0000 7fdb499f3740 20 client.17898 populate_metadata read hostname 'smithi107' 2024-04-25T00:26:49.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.544+0000 7fdb499f3740 1 --2- 172.21.15.107:0/1660148454 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55b65e063d90 0x55b65e0661b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.544+0000 7fdb499f3740 4 client.17898 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:26:49.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.544+0000 7fdb499f3740 1 -- 172.21.15.107:0/1660148454 --> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b65df4df00 con 0x55b65e063d90 2024-04-25T00:26:49.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.544+0000 7fdb44a7d640 1 --2- 172.21.15.107:0/1660148454 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55b65e063d90 0x55b65e0661b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.545+0000 7fdb44a7d640 1 --2- 172.21.15.107:0/1660148454 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55b65e063d90 0x55b65e0661b0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed0a1d90321100e7 server_cookie=ce850d75c85caa70 in_seq=0 out_seq=0 2024-04-25T00:26:49.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.545+0000 7fdb1ffff640 10 client.17898 ms_handle_connect on v2:172.21.15.170:6836/3616315471 2024-04-25T00:26:49.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.545+0000 7fdb1ffff640 1 -- 172.21.15.107:0/1660148454 <== mds.2 v2:172.21.15.170:6836/3616315471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b65df4df00 con 0x55b65e063d90 2024-04-25T00:26:49.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.545+0000 7fdb1ffff640 10 client.17898 handle_command_reply: tid=0 2024-04-25T00:26:49.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.545+0000 7fdb499f3740 1 client.17898 shutdown 2024-04-25T00:26:49.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.545+0000 7fdb499f3740 1 -- 172.21.15.107:0/1660148454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfec570 msgr2=0x55b65e05aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.545+0000 7fdb499f3740 1 --2- 172.21.15.107:0/1660148454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfec570 0x55b65e05aba0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fdb380097b0 tx=0x7fdb38004380 comp rx=0 tx=0).stop 2024-04-25T00:26:49.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.545+0000 7fdb499f3740 1 -- 172.21.15.107:0/1660148454 shutdown_connections 2024-04-25T00:26:49.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.545+0000 7fdb499f3740 1 --2- 172.21.15.107:0/1660148454 >> [v2:172.21.15.170:6836/3616315471,v1:172.21.15.170:6837/3616315471] conn(0x55b65e063d90 0x55b65e0661b0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.545+0000 7fdb499f3740 1 --2- 172.21.15.107:0/1660148454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b65dfec570 0x55b65e05aba0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.545+0000 7fdb499f3740 1 --2- 172.21.15.107:0/1660148454 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55b65dfe1ea0 0x55b65e056d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.545+0000 7fdb1ffff640 0 client.17898 ms_handle_reset on v2:172.21.15.170:6836/3616315471 2024-04-25T00:26:49.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.545+0000 7fdb499f3740 1 --2- 172.21.15.107:0/1660148454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b65dfe1580 0x55b65e0567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.545+0000 7fdb499f3740 1 -- 172.21.15.107:0/1660148454 >> 172.21.15.107:0/1660148454 conn(0x55b65dfdd8a0 msgr2=0x55b65dfdcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.545+0000 7fdb499f3740 1 -- 172.21.15.107:0/1660148454 shutdown_connections 2024-04-25T00:26:49.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.545+0000 7fdb499f3740 1 -- 172.21.15.107:0/1660148454 wait complete. 2024-04-25T00:26:49.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.545+0000 7fdb499f3740 20 client.17898 trim_cache size 0 max 16384 2024-04-25T00:26:49.553 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:26:49.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.546+0000 7fdb46ce1640 1 -- 172.21.15.107:0/266810263 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdb300a2d00 msgr2=0x7fdb300a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.546+0000 7fdb46ce1640 1 --2- 172.21.15.107:0/266810263 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdb300a2d00 0x7fdb300a51c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009a60 tx=0x7fdb38005ef0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.546+0000 7fdb46ce1640 1 -- 172.21.15.107:0/266810263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb401020d0 msgr2=0x7fdb401985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.546+0000 7fdb46ce1640 1 --2- 172.21.15.107:0/266810263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb401020d0 0x7fdb401985b0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c00e9f0 tx=0x7fdb2c00edf0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.546+0000 7fdb46ce1640 1 -- 172.21.15.107:0/266810263 shutdown_connections 2024-04-25T00:26:49.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.546+0000 7fdb46ce1640 1 --2- 172.21.15.107:0/266810263 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fdb40107720 0x7fdb4019c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.546+0000 7fdb46ce1640 1 --2- 172.21.15.107:0/266810263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb401020d0 0x7fdb401985b0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.546+0000 7fdb46ce1640 1 --2- 172.21.15.107:0/266810263 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fdb300a2d00 0x7fdb300a51c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.546+0000 7fdb46ce1640 1 --2- 172.21.15.107:0/266810263 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb40101710 0x7fdb40198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.546+0000 7fdb46ce1640 1 -- 172.21.15.107:0/266810263 >> 172.21.15.107:0/266810263 conn(0x7fdb400fd180 msgr2=0x7fdb400fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.546+0000 7fdb46ce1640 1 -- 172.21.15.107:0/266810263 shutdown_connections 2024-04-25T00:26:49.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.546+0000 7fdb46ce1640 1 -- 172.21.15.107:0/266810263 wait complete. 2024-04-25T00:26:49.557 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-25T00:26:49.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.656+0000 7f9f71702640 1 -- 172.21.15.107:0/4131016002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f6c107720 msgr2=0x7f9f6c107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.656+0000 7f9f71702640 1 --2- 172.21.15.107:0/4131016002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f6c107720 0x7f9f6c107b00 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c009930 tx=0x7f9f5c02a420 comp rx=0 tx=0).stop 2024-04-25T00:26:49.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.656+0000 7f9f71702640 1 -- 172.21.15.107:0/4131016002 shutdown_connections 2024-04-25T00:26:49.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.656+0000 7f9f71702640 1 --2- 172.21.15.107:0/4131016002 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f6c100600 0x7f9f6c10e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.656+0000 7f9f71702640 1 --2- 172.21.15.107:0/4131016002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f6c0ffc60 0x7f9f6c1000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.656+0000 7f9f71702640 1 --2- 172.21.15.107:0/4131016002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f6c107720 0x7f9f6c107b00 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.656+0000 7f9f71702640 1 -- 172.21.15.107:0/4131016002 >> 172.21.15.107:0/4131016002 conn(0x7f9f6c076c00 msgr2=0x7f9f6c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.656+0000 7f9f71702640 1 -- 172.21.15.107:0/4131016002 shutdown_connections 2024-04-25T00:26:49.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.656+0000 7f9f71702640 1 -- 172.21.15.107:0/4131016002 wait complete. 2024-04-25T00:26:49.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.657+0000 7f9f71702640 1 Processor -- start 2024-04-25T00:26:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.657+0000 7f9f71702640 1 -- start start 2024-04-25T00:26:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.657+0000 7f9f71702640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f6c0ffc60 0x7f9f6c198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.657+0000 7f9f71702640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f6c100600 0x7f9f6c198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.657+0000 7f9f71702640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f6c107720 0x7f9f6c19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.657+0000 7f9f71702640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c111b60 con 0x7f9f6c107720 2024-04-25T00:26:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.657+0000 7f9f71702640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c1119e0 con 0x7f9f6c0ffc60 2024-04-25T00:26:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.657+0000 7f9f71702640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c111ce0 con 0x7f9f6c100600 2024-04-25T00:26:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.657+0000 7f9f6b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f6c107720 0x7f9f6c19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.657+0000 7f9f6b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f6c107720 0x7f9f6c19c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56920/0 (socket says 172.21.15.107:56920) 2024-04-25T00:26:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.657+0000 7f9f6b7fe640 1 -- 172.21.15.107:0/250792080 learned_addr learned my addr 172.21.15.107:0/250792080 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.657+0000 7f9f63fff640 1 --2- 172.21.15.107:0/250792080 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f6c100600 0x7f9f6c198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.657+0000 7f9f6affd640 1 --2- 172.21.15.107:0/250792080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f6c0ffc60 0x7f9f6c198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.657+0000 7f9f6b7fe640 1 -- 172.21.15.107:0/250792080 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f6c100600 msgr2=0x7f9f6c198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.657+0000 7f9f6b7fe640 1 --2- 172.21.15.107:0/250792080 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f6c100600 0x7f9f6c198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.657+0000 7f9f6b7fe640 1 -- 172.21.15.107:0/250792080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f6c0ffc60 msgr2=0x7f9f6c198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.657+0000 7f9f6b7fe640 1 --2- 172.21.15.107:0/250792080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f6c0ffc60 0x7f9f6c198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.657+0000 7f9f6b7fe640 1 -- 172.21.15.107:0/250792080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f6c19d030 con 0x7f9f6c107720 2024-04-25T00:26:49.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.657+0000 7f9f63fff640 1 --2- 172.21.15.107:0/250792080 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f6c100600 0x7f9f6c198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:49.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.658+0000 7f9f6b7fe640 1 --2- 172.21.15.107:0/250792080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f6c107720 0x7f9f6c19c900 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400eaa0 tx=0x7f9f6400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.658+0000 7f9f68ff9640 1 -- 172.21.15.107:0/250792080 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6400cdb0 con 0x7f9f6c107720 2024-04-25T00:26:49.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.658+0000 7f9f68ff9640 1 -- 172.21.15.107:0/250792080 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f64005020 con 0x7f9f6c107720 2024-04-25T00:26:49.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.658+0000 7f9f71702640 1 -- 172.21.15.107:0/250792080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f6c19d320 con 0x7f9f6c107720 2024-04-25T00:26:49.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.658+0000 7f9f68ff9640 1 -- 172.21.15.107:0/250792080 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f640116b0 con 0x7f9f6c107720 2024-04-25T00:26:49.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.658+0000 7f9f71702640 1 -- 172.21.15.107:0/250792080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f6c1a4bb0 con 0x7f9f6c107720 2024-04-25T00:26:49.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.660+0000 7f9f68ff9640 1 -- 172.21.15.107:0/250792080 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f9f64005560 con 0x7f9f6c107720 2024-04-25T00:26:49.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.660+0000 7f9f74405740 1 Processor -- start 2024-04-25T00:26:49.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.660+0000 7f9f74405740 1 -- start start 2024-04-25T00:26:49.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.660+0000 7f9f74405740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a85d891990 0x55a85d891d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.660+0000 7f9f74405740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a85d8922b0 0x55a85d89c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.660+0000 7f9f74405740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d89c980 0x55a85d89ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.660+0000 7f9f74405740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a85d7f2670 con 0x55a85d89c980 2024-04-25T00:26:49.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.660+0000 7f9f74405740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a85d79a970 con 0x55a85d891990 2024-04-25T00:26:49.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.660+0000 7f9f74405740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a85d7ee630 con 0x55a85d8922b0 2024-04-25T00:26:49.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.661+0000 7f9f68ff9640 1 --2- 172.21.15.107:0/250792080 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f9f580a2d80 0x7f9f580a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.670 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.661+0000 7f9f63fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d89c980 0x55a85d89ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.661+0000 7f9f63fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d89c980 0x55a85d89ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56922/0 (socket says 172.21.15.107:56922) 2024-04-25T00:26:49.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.661+0000 7f9f63fff640 1 -- 172.21.15.107:0/1930559370 learned_addr learned my addr 172.21.15.107:0/1930559370 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:49.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.661+0000 7f9f6b7fe640 1 --2- 172.21.15.107:0/250792080 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f9f580a2d80 0x7f9f580a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.661+0000 7f9f63fff640 1 --2- 172.21.15.107:0/1930559370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a85d891990 0x55a85d891d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.661+0000 7f9f6affd640 1 --2- 172.21.15.107:0/1930559370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a85d8922b0 0x55a85d89c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.661+0000 7f9f63fff640 1 -- 172.21.15.107:0/1930559370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a85d8922b0 msgr2=0x55a85d89c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.661+0000 7f9f63fff640 1 --2- 172.21.15.107:0/1930559370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a85d8922b0 0x55a85d89c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.661+0000 7f9f63fff640 1 -- 172.21.15.107:0/1930559370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a85d891990 msgr2=0x55a85d891d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.661+0000 7f9f63fff640 1 --2- 172.21.15.107:0/1930559370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a85d891990 0x55a85d891d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.661+0000 7f9f63fff640 1 -- 172.21.15.107:0/1930559370 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a85d7be400 con 0x55a85d89c980 2024-04-25T00:26:49.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.661+0000 7f9f6affd640 1 --2- 172.21.15.107:0/1930559370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a85d8922b0 0x55a85d89c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:49.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.661+0000 7f9f68ff9640 1 -- 172.21.15.107:0/250792080 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f9f640cfa50 con 0x7f9f6c107720 2024-04-25T00:26:49.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.661+0000 7f9f63fff640 1 --2- 172.21.15.107:0/1930559370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d89c980 0x55a85d89ed70 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f9f540124f0 tx=0x7f9f5400b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc295b218db01d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.671 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.661+0000 7f9f6b7fe640 1 --2- 172.21.15.107:0/250792080 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f9f580a2d80 0x7f9f580a5240 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9f64098010 tx=0x7f9f64097ed0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.661+0000 7f9f617fa640 1 -- 172.21.15.107:0/1930559370 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f54018070 con 0x55a85d89c980 2024-04-25T00:26:49.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.661+0000 7f9f617fa640 1 -- 172.21.15.107:0/1930559370 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f54010910 con 0x55a85d89c980 2024-04-25T00:26:49.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.661+0000 7f9f617fa640 1 -- 172.21.15.107:0/1930559370 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5400e900 con 0x55a85d89c980 2024-04-25T00:26:49.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.662+0000 7f9f74405740 1 -- 172.21.15.107:0/1930559370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d89c980 msgr2=0x55a85d89ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.662+0000 7f9f74405740 1 --2- 172.21.15.107:0/1930559370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d89c980 0x55a85d89ed70 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f9f540124f0 tx=0x7f9f5400b600 comp rx=0 tx=0).stop 2024-04-25T00:26:49.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.662+0000 7f9f74405740 1 -- 172.21.15.107:0/1930559370 shutdown_connections 2024-04-25T00:26:49.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.662+0000 7f9f74405740 1 --2- 172.21.15.107:0/1930559370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d89c980 0x55a85d89ed70 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.662+0000 7f9f74405740 1 --2- 172.21.15.107:0/1930559370 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a85d8922b0 0x55a85d89c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.662+0000 7f9f74405740 1 --2- 172.21.15.107:0/1930559370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a85d891990 0x55a85d891d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.662+0000 7f9f74405740 1 -- 172.21.15.107:0/1930559370 >> 172.21.15.107:0/1930559370 conn(0x55a85d88dcb0 msgr2=0x55a85d88e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.662+0000 7f9f74405740 1 -- 172.21.15.107:0/1930559370 shutdown_connections 2024-04-25T00:26:49.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.662+0000 7f9f74405740 1 -- 172.21.15.107:0/1930559370 wait complete. 2024-04-25T00:26:49.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.662+0000 7f9f74405740 1 -- start start 2024-04-25T00:26:49.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.662+0000 7f9f74405740 1 Processor -- start 2024-04-25T00:26:49.672 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.663+0000 7f9f74405740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d891990 0x55a85d89c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.663+0000 7f9f74405740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a85d8922b0 0x55a85d899620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.663+0000 7f9f74405740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a85d89c980 0x55a85d899bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.663+0000 7f9f74405740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a85d89f770 con 0x55a85d891990 2024-04-25T00:26:49.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.663+0000 7f9f74405740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a85d7ae050 con 0x55a85d89c980 2024-04-25T00:26:49.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.663+0000 7f9f74405740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55a85d89f8f0 con 0x55a85d8922b0 2024-04-25T00:26:49.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.663+0000 7f9f74405740 10 client.0 _ll_register_callbacks cb 0x55a85d505be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:49.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.663+0000 7f9f63fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d891990 0x55a85d89c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.663+0000 7f9f63fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d891990 0x55a85d89c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56930/0 (socket says 172.21.15.107:56930) 2024-04-25T00:26:49.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.663+0000 7f9f63fff640 1 -- 172.21.15.107:0/492712513 learned_addr learned my addr 172.21.15.107:0/492712513 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:49.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.663+0000 7f9f6affd640 1 --2- 172.21.15.107:0/492712513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a85d8922b0 0x55a85d899620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.663+0000 7f9f63fff640 1 -- 172.21.15.107:0/492712513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a85d8922b0 msgr2=0x55a85d899620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.663+0000 7f9f63fff640 1 --2- 172.21.15.107:0/492712513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a85d8922b0 0x55a85d899620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.663+0000 7f9f63fff640 1 -- 172.21.15.107:0/492712513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a85d89c980 msgr2=0x55a85d899bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.673 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.663+0000 7f9f63fff640 1 --2- 172.21.15.107:0/492712513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a85d89c980 0x55a85d899bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.663+0000 7f9f63fff640 1 -- 172.21.15.107:0/492712513 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a85d77fdc0 con 0x55a85d891990 2024-04-25T00:26:49.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.663+0000 7f9f63fff640 1 --2- 172.21.15.107:0/492712513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d891990 0x55a85d89c330 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400e480 tx=0x7f9f5400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e154f7539d28963 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.663+0000 7f9f527fc640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:26:49.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.663+0000 7f9f527fc640 1 -- 172.21.15.107:0/492712513 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f54024070 con 0x55a85d891990 2024-04-25T00:26:49.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.663+0000 7f9f527fc640 1 -- 172.21.15.107:0/492712513 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f9f5400e880 con 0x55a85d891990 2024-04-25T00:26:49.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.663+0000 7f9f74405740 1 -- 172.21.15.107:0/492712513 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f54010910 con 0x55a85d891990 2024-04-25T00:26:49.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.663+0000 7f9f527fc640 1 -- 172.21.15.107:0/492712513 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9f54018040 con 0x55a85d891990 2024-04-25T00:26:49.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.663+0000 7f9f74405740 10 client.17910 fetch_fsmap learned FSMap version 34 2024-04-25T00:26:49.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.663+0000 7f9f74405740 1 -- 172.21.15.107:0/492712513 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a85d5063f0 con 0x55a85d891990 2024-04-25T00:26:49.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.664+0000 7f9f527fc640 1 -- 172.21.15.107:0/492712513 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f54010910 con 0x55a85d891990 2024-04-25T00:26:49.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.664+0000 7f9f527fc640 1 -- 172.21.15.107:0/492712513 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f9f5400f040 con 0x55a85d891990 2024-04-25T00:26:49.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.664+0000 7f9f74405740 10 client.17910 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:26:49.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.664+0000 7f9f74405740 10 client.17910 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:26:49.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.664+0000 7f9f74405740 20 client.17910 populate_metadata read hostname 'smithi107' 2024-04-25T00:26:49.674 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.664+0000 7f9f74405740 1 --2- 172.21.15.107:0/492712513 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55a85d8ac870 0x55a85d8aec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.664+0000 7f9f74405740 4 client.17910 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:49.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.664+0000 7f9f74405740 1 -- 172.21.15.107:0/492712513 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a85d7ffbb0 con 0x55a85d8ac870 2024-04-25T00:26:49.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.664+0000 7f9f6affd640 1 --2- 172.21.15.107:0/492712513 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55a85d8ac870 0x55a85d8aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.664+0000 7f9f527fc640 10 client.17910 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:26:49.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.664+0000 7f9f6affd640 1 --2- 172.21.15.107:0/492712513 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55a85d8ac870 0x55a85d8aec60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5446da319e23c3da server_cookie=a363cae14bac7cd3 in_seq=0 out_seq=0 2024-04-25T00:26:49.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.665+0000 7f9f527fc640 1 -- 172.21.15.107:0/492712513 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a85d7ffbb0 con 0x55a85d8ac870 2024-04-25T00:26:49.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.665+0000 7f9f527fc640 10 client.17910 handle_command_reply: tid=0 2024-04-25T00:26:49.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.665+0000 7f9f74405740 1 client.17910 shutdown 2024-04-25T00:26:49.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.665+0000 7f9f74405740 1 -- 172.21.15.107:0/492712513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d891990 msgr2=0x55a85d89c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.665+0000 7f9f74405740 1 --2- 172.21.15.107:0/492712513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d891990 0x55a85d89c330 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400e480 tx=0x7f9f5400b3b0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.665+0000 7f9f74405740 1 -- 172.21.15.107:0/492712513 shutdown_connections 2024-04-25T00:26:49.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.665+0000 7f9f74405740 1 --2- 172.21.15.107:0/492712513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a85d89c980 0x55a85d899bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.665+0000 7f9f74405740 1 --2- 172.21.15.107:0/492712513 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a85d8922b0 0x55a85d899620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.665+0000 7f9f74405740 1 --2- 172.21.15.107:0/492712513 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55a85d8ac870 0x55a85d8aec60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.665+0000 7f9f74405740 1 --2- 172.21.15.107:0/492712513 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d891990 0x55a85d89c330 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.665+0000 7f9f527fc640 0 client.17910 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:26:49.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.665+0000 7f9f74405740 1 -- 172.21.15.107:0/492712513 >> 172.21.15.107:0/492712513 conn(0x55a85d88dcb0 msgr2=0x55a85d888b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.665+0000 7f9f74405740 1 -- 172.21.15.107:0/492712513 shutdown_connections 2024-04-25T00:26:49.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.665+0000 7f9f74405740 1 -- 172.21.15.107:0/492712513 wait complete. 2024-04-25T00:26:49.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.666+0000 7f9f74405740 20 client.17910 trim_cache size 0 max 16384 2024-04-25T00:26:49.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.678+0000 7f9f74405740 1 Processor -- start 2024-04-25T00:26:49.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.678+0000 7f9f74405740 1 -- start start 2024-04-25T00:26:49.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.679+0000 7f9f74405740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a85d891990 0x55a85d8a8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.679+0000 7f9f74405740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a85d8922b0 0x55a85d898f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.679+0000 7f9f74405740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d89c980 0x55a85d8b4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.679+0000 7f9f74405740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c1af420 con 0x55a85d89c980 2024-04-25T00:26:49.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.679+0000 7f9f74405740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c1accf0 con 0x55a85d8922b0 2024-04-25T00:26:49.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.679+0000 7f9f74405740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c1aab60 con 0x55a85d891990 2024-04-25T00:26:49.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.679+0000 7f9f63fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d89c980 0x55a85d8b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.679+0000 7f9f63fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d89c980 0x55a85d8b4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56932/0 (socket says 172.21.15.107:56932) 2024-04-25T00:26:49.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.679+0000 7f9f63fff640 1 -- 172.21.15.107:0/3486823792 learned_addr learned my addr 172.21.15.107:0/3486823792 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:49.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.679+0000 7f9f6affd640 1 --2- 172.21.15.107:0/3486823792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a85d8922b0 0x55a85d898f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.679+0000 7f9f63fff640 1 --2- 172.21.15.107:0/3486823792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a85d891990 0x55a85d8a8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.679+0000 7f9f63fff640 1 -- 172.21.15.107:0/3486823792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a85d891990 msgr2=0x55a85d8a8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.679+0000 7f9f63fff640 1 --2- 172.21.15.107:0/3486823792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a85d891990 0x55a85d8a8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.679+0000 7f9f63fff640 1 -- 172.21.15.107:0/3486823792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a85d8922b0 msgr2=0x55a85d898f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.679+0000 7f9f63fff640 1 --2- 172.21.15.107:0/3486823792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a85d8922b0 0x55a85d898f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.679+0000 7f9f63fff640 1 -- 172.21.15.107:0/3486823792 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a85d8920e0 con 0x55a85d89c980 2024-04-25T00:26:49.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.679+0000 7f9f6affd640 1 --2- 172.21.15.107:0/3486823792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a85d8922b0 0x55a85d898f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:26:49.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.679+0000 7f9f63fff640 1 --2- 172.21.15.107:0/3486823792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d89c980 0x55a85d8b4230 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400b6e0 tx=0x7f9f5400b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697a54bb16b9e37e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.680+0000 7f9f60ff9640 1 -- 172.21.15.107:0/3486823792 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f54024070 con 0x55a85d89c980 2024-04-25T00:26:49.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.680+0000 7f9f60ff9640 1 -- 172.21.15.107:0/3486823792 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f540120e0 con 0x55a85d89c980 2024-04-25T00:26:49.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.680+0000 7f9f60ff9640 1 -- 172.21.15.107:0/3486823792 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5400eaf0 con 0x55a85d89c980 2024-04-25T00:26:49.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.680+0000 7f9f74405740 1 -- 172.21.15.107:0/3486823792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d89c980 msgr2=0x55a85d8b4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.680+0000 7f9f74405740 1 --2- 172.21.15.107:0/3486823792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d89c980 0x55a85d8b4230 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400b6e0 tx=0x7f9f5400b340 comp rx=0 tx=0).stop 2024-04-25T00:26:49.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.680+0000 7f9f74405740 1 -- 172.21.15.107:0/3486823792 shutdown_connections 2024-04-25T00:26:49.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.680+0000 7f9f74405740 1 --2- 172.21.15.107:0/3486823792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d89c980 0x55a85d8b4230 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.680+0000 7f9f74405740 1 --2- 172.21.15.107:0/3486823792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a85d8922b0 0x55a85d898f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.680+0000 7f9f74405740 1 --2- 172.21.15.107:0/3486823792 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a85d891990 0x55a85d8a8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.680+0000 7f9f74405740 1 -- 172.21.15.107:0/3486823792 >> 172.21.15.107:0/3486823792 conn(0x55a85d88dcb0 msgr2=0x55a85d8a7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.680+0000 7f9f74405740 1 -- 172.21.15.107:0/3486823792 shutdown_connections 2024-04-25T00:26:49.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.680+0000 7f9f74405740 1 -- 172.21.15.107:0/3486823792 wait complete. 2024-04-25T00:26:49.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.681+0000 7f9f74405740 1 -- start start 2024-04-25T00:26:49.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.681+0000 7f9f74405740 1 Processor -- start 2024-04-25T00:26:49.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.681+0000 7f9f74405740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a85d891990 0x55a85d8a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.681+0000 7f9f74405740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d8922b0 0x55a85d8a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.681+0000 7f9f74405740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a85d89c980 0x55a85d8a3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.681+0000 7f9f74405740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c1ab1b0 con 0x55a85d8922b0 2024-04-25T00:26:49.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.681+0000 7f9f74405740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c1ac620 con 0x55a85d89c980 2024-04-25T00:26:49.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.681+0000 7f9f74405740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6c1ab880 con 0x55a85d891990 2024-04-25T00:26:49.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.681+0000 7f9f74405740 10 client.0 _ll_register_callbacks cb 0x7f9f5400bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:49.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.681+0000 7f9f6affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d8922b0 0x55a85d8a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.681+0000 7f9f6affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d8922b0 0x55a85d8a2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56948/0 (socket says 172.21.15.107:56948) 2024-04-25T00:26:49.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.681+0000 7f9f6affd640 1 -- 172.21.15.107:0/3226014945 learned_addr learned my addr 172.21.15.107:0/3226014945 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:49.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.681+0000 7f9f63fff640 1 --2- 172.21.15.107:0/3226014945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a85d89c980 0x55a85d8a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.681+0000 7f9f63fff640 1 --2- 172.21.15.107:0/3226014945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a85d891990 0x55a85d8a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.681+0000 7f9f6affd640 1 -- 172.21.15.107:0/3226014945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a85d891990 msgr2=0x55a85d8a25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.681+0000 7f9f6affd640 1 --2- 172.21.15.107:0/3226014945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a85d891990 0x55a85d8a25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.681+0000 7f9f6affd640 1 -- 172.21.15.107:0/3226014945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a85d89c980 msgr2=0x55a85d8a3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.681+0000 7f9f6affd640 1 --2- 172.21.15.107:0/3226014945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a85d89c980 0x55a85d8a3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.681+0000 7f9f6affd640 1 -- 172.21.15.107:0/3226014945 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a85d5d07d0 con 0x55a85d8922b0 2024-04-25T00:26:49.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.681+0000 7f9f63fff640 1 --2- 172.21.15.107:0/3226014945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a85d891990 0x55a85d8a25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:49.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.681+0000 7f9f6affd640 1 --2- 172.21.15.107:0/3226014945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d8922b0 0x55a85d8a2af0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c005c40 tx=0x7f9f5c032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75bcc832a305771c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.681+0000 7f9f527fc640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:26:49.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.681+0000 7f9f527fc640 1 -- 172.21.15.107:0/3226014945 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c035040 con 0x55a85d8922b0 2024-04-25T00:26:49.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.682+0000 7f9f527fc640 1 -- 172.21.15.107:0/3226014945 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f9f5c005040 con 0x55a85d8922b0 2024-04-25T00:26:49.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.682+0000 7f9f74405740 1 -- 172.21.15.107:0/3226014945 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f540120e0 con 0x55a85d8922b0 2024-04-25T00:26:49.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.682+0000 7f9f527fc640 1 -- 172.21.15.107:0/3226014945 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9f5c052050 con 0x55a85d8922b0 2024-04-25T00:26:49.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.682+0000 7f9f74405740 10 client.17916 fetch_fsmap learned FSMap version 34 2024-04-25T00:26:49.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.682+0000 7f9f74405740 1 -- 172.21.15.107:0/3226014945 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a85d912bd0 con 0x55a85d8922b0 2024-04-25T00:26:49.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.682+0000 7f9f527fc640 1 -- 172.21.15.107:0/3226014945 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f540120e0 con 0x55a85d8922b0 2024-04-25T00:26:49.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.682+0000 7f9f527fc640 1 -- 172.21.15.107:0/3226014945 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7f9f5c040130 con 0x55a85d8922b0 2024-04-25T00:26:49.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.682+0000 7f9f74405740 10 client.17916 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:26:49.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.682+0000 7f9f74405740 10 client.17916 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T00:26:49.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.682+0000 7f9f74405740 20 client.17916 populate_metadata read hostname 'smithi107' 2024-04-25T00:26:49.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.682+0000 7f9f74405740 1 --2- 172.21.15.107:0/3226014945 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55a85d913300 0x55a85d915720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.682+0000 7f9f74405740 4 client.17916 mds_command: new command op to 24493 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:26:49.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.682+0000 7f9f74405740 1 -- 172.21.15.107:0/3226014945 --> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a85d7f9f00 con 0x55a85d913300 2024-04-25T00:26:49.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.682+0000 7f9f63fff640 1 --2- 172.21.15.107:0/3226014945 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55a85d913300 0x55a85d915720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.683+0000 7f9f63fff640 1 --2- 172.21.15.107:0/3226014945 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55a85d913300 0x55a85d915720 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e59b56095bf13741 server_cookie=f1112591753219f8 in_seq=0 out_seq=0 2024-04-25T00:26:49.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.683+0000 7f9f527fc640 10 client.17916 ms_handle_connect on v2:172.21.15.195:6838/4133864926 2024-04-25T00:26:49.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.683+0000 7f9f527fc640 1 -- 172.21.15.107:0/3226014945 <== mds.1 v2:172.21.15.195:6838/4133864926 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a85d7f9f00 con 0x55a85d913300 2024-04-25T00:26:49.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.683+0000 7f9f527fc640 10 client.17916 handle_command_reply: tid=0 2024-04-25T00:26:49.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.683+0000 7f9f74405740 1 client.17916 shutdown 2024-04-25T00:26:49.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.683+0000 7f9f74405740 1 -- 172.21.15.107:0/3226014945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d8922b0 msgr2=0x55a85d8a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.683+0000 7f9f74405740 1 --2- 172.21.15.107:0/3226014945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d8922b0 0x55a85d8a2af0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c005c40 tx=0x7f9f5c032f90 comp rx=0 tx=0).stop 2024-04-25T00:26:49.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.683+0000 7f9f74405740 1 -- 172.21.15.107:0/3226014945 shutdown_connections 2024-04-25T00:26:49.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.683+0000 7f9f74405740 1 --2- 172.21.15.107:0/3226014945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a85d89c980 0x55a85d8a3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.683+0000 7f9f74405740 1 --2- 172.21.15.107:0/3226014945 >> [v2:172.21.15.195:6838/4133864926,v1:172.21.15.195:6839/4133864926] conn(0x55a85d913300 0x55a85d915720 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.683+0000 7f9f74405740 1 --2- 172.21.15.107:0/3226014945 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a85d8922b0 0x55a85d8a2af0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.683+0000 7f9f74405740 1 --2- 172.21.15.107:0/3226014945 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a85d891990 0x55a85d8a25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.683+0000 7f9f74405740 1 -- 172.21.15.107:0/3226014945 >> 172.21.15.107:0/3226014945 conn(0x55a85d88dcb0 msgr2=0x55a85d897ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.683+0000 7f9f527fc640 0 client.17916 ms_handle_reset on v2:172.21.15.195:6838/4133864926 2024-04-25T00:26:49.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.683+0000 7f9f74405740 1 -- 172.21.15.107:0/3226014945 shutdown_connections 2024-04-25T00:26:49.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.683+0000 7f9f74405740 1 -- 172.21.15.107:0/3226014945 wait complete. 2024-04-25T00:26:49.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.683+0000 7f9f74405740 20 client.17916 trim_cache size 0 max 16384 2024-04-25T00:26:49.693 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:26:49.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.684+0000 7f9f71702640 1 -- 172.21.15.107:0/250792080 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f9f580a2d80 msgr2=0x7f9f580a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.684+0000 7f9f71702640 1 --2- 172.21.15.107:0/250792080 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f9f580a2d80 0x7f9f580a5240 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9f64098010 tx=0x7f9f64097ed0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.684+0000 7f9f71702640 1 -- 172.21.15.107:0/250792080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f6c107720 msgr2=0x7f9f6c19c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.684+0000 7f9f71702640 1 --2- 172.21.15.107:0/250792080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f6c107720 0x7f9f6c19c900 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f9f6400eaa0 tx=0x7f9f6400ef60 comp rx=0 tx=0).stop 2024-04-25T00:26:49.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.684+0000 7f9f71702640 1 -- 172.21.15.107:0/250792080 shutdown_connections 2024-04-25T00:26:49.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.684+0000 7f9f71702640 1 --2- 172.21.15.107:0/250792080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f6c107720 0x7f9f6c19c900 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.684+0000 7f9f71702640 1 --2- 172.21.15.107:0/250792080 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9f6c100600 0x7f9f6c198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.684+0000 7f9f71702640 1 --2- 172.21.15.107:0/250792080 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f9f580a2d80 0x7f9f580a5240 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.684+0000 7f9f71702640 1 --2- 172.21.15.107:0/250792080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9f6c0ffc60 0x7f9f6c198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.684+0000 7f9f71702640 1 -- 172.21.15.107:0/250792080 >> 172.21.15.107:0/250792080 conn(0x7f9f6c076c00 msgr2=0x7f9f6c0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.684+0000 7f9f71702640 1 -- 172.21.15.107:0/250792080 shutdown_connections 2024-04-25T00:26:49.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.684+0000 7f9f71702640 1 -- 172.21.15.107:0/250792080 wait complete. 2024-04-25T00:26:49.696 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-25T00:26:49.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.795+0000 7ff88f3a9640 1 -- 172.21.15.107:0/1636730199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8881010e0 msgr2=0x7ff8881055a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.795+0000 7ff88f3a9640 1 --2- 172.21.15.107:0/1636730199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8881010e0 0x7ff8881055a0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7ff878009900 tx=0x7ff87802a510 comp rx=0 tx=0).stop 2024-04-25T00:26:49.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.795+0000 7ff88f3a9640 1 -- 172.21.15.107:0/1636730199 shutdown_connections 2024-04-25T00:26:49.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.795+0000 7ff88f3a9640 1 --2- 172.21.15.107:0/1636730199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff888105ae0 0x7ff888105f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.795+0000 7ff88f3a9640 1 --2- 172.21.15.107:0/1636730199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8881010e0 0x7ff8881055a0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.795+0000 7ff88f3a9640 1 --2- 172.21.15.107:0/1636730199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8881007c0 0x7ff888100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.795+0000 7ff88f3a9640 1 -- 172.21.15.107:0/1636730199 >> 172.21.15.107:0/1636730199 conn(0x7ff8880769c0 msgr2=0x7ff888076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.795+0000 7ff88f3a9640 1 -- 172.21.15.107:0/1636730199 shutdown_connections 2024-04-25T00:26:49.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.795+0000 7ff88f3a9640 1 -- 172.21.15.107:0/1636730199 wait complete. 2024-04-25T00:26:49.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.796+0000 7ff88f3a9640 1 Processor -- start 2024-04-25T00:26:49.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.796+0000 7ff88f3a9640 1 -- start start 2024-04-25T00:26:49.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.796+0000 7ff88f3a9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8881007c0 0x7ff88819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.796+0000 7ff88f3a9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8881010e0 0x7ff88819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.796+0000 7ff88f3a9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff888105ae0 0x7ff8881a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.796+0000 7ff88f3a9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881180e0 con 0x7ff8881007c0 2024-04-25T00:26:49.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.796+0000 7ff88f3a9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff888117f60 con 0x7ff8881010e0 2024-04-25T00:26:49.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.796+0000 7ff88f3a9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff888118260 con 0x7ff888105ae0 2024-04-25T00:26:49.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.796+0000 7ff88d145640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8881007c0 0x7ff88819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.796+0000 7ff88d145640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8881007c0 0x7ff88819e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56964/0 (socket says 172.21.15.107:56964) 2024-04-25T00:26:49.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.796+0000 7ff88d145640 1 -- 172.21.15.107:0/3973972626 learned_addr learned my addr 172.21.15.107:0/3973972626 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:49.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.796+0000 7ff88d946640 1 --2- 172.21.15.107:0/3973972626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff888105ae0 0x7ff8881a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.796+0000 7ff88c944640 1 --2- 172.21.15.107:0/3973972626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8881010e0 0x7ff88819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.796+0000 7ff88d145640 1 -- 172.21.15.107:0/3973972626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff888105ae0 msgr2=0x7ff8881a2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.796+0000 7ff88d145640 1 --2- 172.21.15.107:0/3973972626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff888105ae0 0x7ff8881a2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.796+0000 7ff88d145640 1 -- 172.21.15.107:0/3973972626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8881010e0 msgr2=0x7ff88819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.796+0000 7ff88d145640 1 --2- 172.21.15.107:0/3973972626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8881010e0 0x7ff88819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.796+0000 7ff88d145640 1 -- 172.21.15.107:0/3973972626 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8881a35b0 con 0x7ff8881007c0 2024-04-25T00:26:49.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.796+0000 7ff88c944640 1 --2- 172.21.15.107:0/3973972626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8881010e0 0x7ff88819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:49.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.796+0000 7ff88d145640 1 --2- 172.21.15.107:0/3973972626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8881007c0 0x7ff88819e5c0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7ff88400c920 tx=0x7ff88400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.797+0000 7ff87e7fc640 1 -- 172.21.15.107:0/3973972626 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff884007c70 con 0x7ff8881007c0 2024-04-25T00:26:49.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.797+0000 7ff87e7fc640 1 -- 172.21.15.107:0/3973972626 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff884007e10 con 0x7ff8881007c0 2024-04-25T00:26:49.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.797+0000 7ff88f3a9640 1 -- 172.21.15.107:0/3973972626 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8881a38a0 con 0x7ff8881007c0 2024-04-25T00:26:49.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.797+0000 7ff87e7fc640 1 -- 172.21.15.107:0/3973972626 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff884018890 con 0x7ff8881007c0 2024-04-25T00:26:49.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.797+0000 7ff88f3a9640 1 -- 172.21.15.107:0/3973972626 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8881ab130 con 0x7ff8881007c0 2024-04-25T00:26:49.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.799+0000 7ff87e7fc640 1 -- 172.21.15.107:0/3973972626 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff884018a70 con 0x7ff8881007c0 2024-04-25T00:26:49.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.799+0000 7ff892198740 1 Processor -- start 2024-04-25T00:26:49.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.799+0000 7ff892198740 1 -- start start 2024-04-25T00:26:49.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.800+0000 7ff892198740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4a580 0x55fbc1a4a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.800+0000 7ff892198740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbc1a4aea0 0x55fbc1a55030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.800+0000 7ff87e7fc640 1 --2- 172.21.15.107:0/3973972626 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff8740a2d80 0x7ff8740a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.800+0000 7ff892198740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fbc1a55570 0x55fbc1a57960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.800+0000 7ff892198740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fbc19af670 con 0x55fbc1a4a580 2024-04-25T00:26:49.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.800+0000 7ff892198740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55fbc1957970 con 0x55fbc1a55570 2024-04-25T00:26:49.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.800+0000 7ff892198740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fbc19ab630 con 0x55fbc1a4aea0 2024-04-25T00:26:49.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.800+0000 7ff88d946640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4a580 0x55fbc1a4a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.800+0000 7ff88c944640 1 --2- 172.21.15.107:0/3973972626 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff8740a2d80 0x7ff8740a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.800+0000 7ff88d946640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4a580 0x55fbc1a4a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56970/0 (socket says 172.21.15.107:56970) 2024-04-25T00:26:49.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.800+0000 7ff88d946640 1 -- 172.21.15.107:0/3719508978 learned_addr learned my addr 172.21.15.107:0/3719508978 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:49.808 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.800+0000 7ff88d145640 1 --2- 172.21.15.107:0/3719508978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbc1a4aea0 0x55fbc1a55030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.800+0000 7ff87e7fc640 1 -- 172.21.15.107:0/3973972626 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7ff8840cf970 con 0x7ff8881007c0 2024-04-25T00:26:49.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.800+0000 7ff88d946640 1 -- 172.21.15.107:0/3719508978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbc1a4aea0 msgr2=0x55fbc1a55030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.800+0000 7ff88d946640 1 --2- 172.21.15.107:0/3719508978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbc1a4aea0 0x55fbc1a55030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.800+0000 7ff88d946640 1 -- 172.21.15.107:0/3719508978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fbc1a55570 msgr2=0x55fbc1a57960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.800+0000 7ff88d946640 1 --2- 172.21.15.107:0/3719508978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fbc1a55570 0x55fbc1a57960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.800+0000 7ff88d946640 1 -- 172.21.15.107:0/3719508978 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbc197b400 con 0x55fbc1a4a580 2024-04-25T00:26:49.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.800+0000 7ff88c944640 1 --2- 172.21.15.107:0/3973972626 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff8740a2d80 0x7ff8740a5240 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff878000c00 tx=0x7ff878032d50 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.800+0000 7ff88d946640 1 --2- 172.21.15.107:0/3719508978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4a580 0x55fbc1a4a960 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ff880008de0 tx=0x7ff880010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772a6889a5c2985f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.800+0000 7ff8737fe640 1 -- 172.21.15.107:0/3719508978 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880016070 con 0x55fbc1a4a580 2024-04-25T00:26:49.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.800+0000 7ff8737fe640 1 -- 172.21.15.107:0/3719508978 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff880010e50 con 0x55fbc1a4a580 2024-04-25T00:26:49.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.800+0000 7ff8737fe640 1 -- 172.21.15.107:0/3719508978 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff880011920 con 0x55fbc1a4a580 2024-04-25T00:26:49.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.801+0000 7ff892198740 1 -- 172.21.15.107:0/3719508978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4a580 msgr2=0x55fbc1a4a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.809 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.801+0000 7ff892198740 1 --2- 172.21.15.107:0/3719508978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4a580 0x55fbc1a4a960 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ff880008de0 tx=0x7ff880010740 comp rx=0 tx=0).stop 2024-04-25T00:26:49.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.801+0000 7ff892198740 1 -- 172.21.15.107:0/3719508978 shutdown_connections 2024-04-25T00:26:49.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.801+0000 7ff892198740 1 --2- 172.21.15.107:0/3719508978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fbc1a55570 0x55fbc1a57960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.801+0000 7ff892198740 1 --2- 172.21.15.107:0/3719508978 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbc1a4aea0 0x55fbc1a55030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.801+0000 7ff892198740 1 --2- 172.21.15.107:0/3719508978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4a580 0x55fbc1a4a960 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.801+0000 7ff892198740 1 -- 172.21.15.107:0/3719508978 >> 172.21.15.107:0/3719508978 conn(0x55fbc1a468a0 msgr2=0x55fbc1a46c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.801+0000 7ff892198740 1 -- 172.21.15.107:0/3719508978 shutdown_connections 2024-04-25T00:26:49.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.801+0000 7ff892198740 1 -- 172.21.15.107:0/3719508978 wait complete. 2024-04-25T00:26:49.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.801+0000 7ff892198740 1 -- start start 2024-04-25T00:26:49.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.801+0000 7ff892198740 1 Processor -- start 2024-04-25T00:26:49.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.802+0000 7ff892198740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbc1a4a580 0x55fbc1a54ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.802+0000 7ff892198740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4aea0 0x55fbc1a521c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.802+0000 7ff892198740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fbc1a55570 0x55fbc1a52720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.802+0000 7ff892198740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fbc1a58360 con 0x55fbc1a4aea0 2024-04-25T00:26:49.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.802+0000 7ff892198740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55fbc196b050 con 0x55fbc1a55570 2024-04-25T00:26:49.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.802+0000 7ff892198740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x55fbc1a584e0 con 0x55fbc1a4a580 2024-04-25T00:26:49.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.802+0000 7ff892198740 10 client.0 _ll_register_callbacks cb 0x55fbc16c2be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:49.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.802+0000 7ff88c944640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4aea0 0x55fbc1a521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.802+0000 7ff88c944640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4aea0 0x55fbc1a521c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56986/0 (socket says 172.21.15.107:56986) 2024-04-25T00:26:49.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.802+0000 7ff88c944640 1 -- 172.21.15.107:0/848103417 learned_addr learned my addr 172.21.15.107:0/848103417 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:49.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.802+0000 7ff88d145640 1 --2- 172.21.15.107:0/848103417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbc1a4a580 0x55fbc1a54ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.802+0000 7ff88d946640 1 --2- 172.21.15.107:0/848103417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fbc1a55570 0x55fbc1a52720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.802+0000 7ff88c944640 1 -- 172.21.15.107:0/848103417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbc1a4a580 msgr2=0x55fbc1a54ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.802+0000 7ff88c944640 1 --2- 172.21.15.107:0/848103417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbc1a4a580 0x55fbc1a54ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.802+0000 7ff88c944640 1 -- 172.21.15.107:0/848103417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fbc1a55570 msgr2=0x55fbc1a52720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.802+0000 7ff88c944640 1 --2- 172.21.15.107:0/848103417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fbc1a55570 0x55fbc1a52720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.802+0000 7ff88c944640 1 -- 172.21.15.107:0/848103417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbc193cdc0 con 0x55fbc1a4aea0 2024-04-25T00:26:49.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.802+0000 7ff88d946640 1 --2- 172.21.15.107:0/848103417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fbc1a55570 0x55fbc1a52720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:49.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.802+0000 7ff88c944640 1 --2- 172.21.15.107:0/848103417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4aea0 0x55fbc1a521c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7ff8780314d0 tx=0x7ff878009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc52c918ccf0d59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.802+0000 7ff870ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:26:49.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.803+0000 7ff870ff9640 1 -- 172.21.15.107:0/848103417 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87804b020 con 0x55fbc1a4aea0 2024-04-25T00:26:49.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.803+0000 7ff870ff9640 1 -- 172.21.15.107:0/848103417 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7ff8780037f0 con 0x55fbc1a4aea0 2024-04-25T00:26:49.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.803+0000 7ff892198740 1 -- 172.21.15.107:0/848103417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff880010e50 con 0x55fbc1a4aea0 2024-04-25T00:26:49.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.803+0000 7ff870ff9640 1 -- 172.21.15.107:0/848103417 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff878056050 con 0x55fbc1a4aea0 2024-04-25T00:26:49.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.803+0000 7ff892198740 10 client.17928 fetch_fsmap learned FSMap version 34 2024-04-25T00:26:49.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.803+0000 7ff892198740 1 -- 172.21.15.107:0/848103417 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fbc16c33f0 con 0x55fbc1a4aea0 2024-04-25T00:26:49.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.803+0000 7ff870ff9640 1 -- 172.21.15.107:0/848103417 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff880010e50 con 0x55fbc1a4aea0 2024-04-25T00:26:49.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.803+0000 7ff870ff9640 1 -- 172.21.15.107:0/848103417 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7ff87803bc60 con 0x55fbc1a4aea0 2024-04-25T00:26:49.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.803+0000 7ff892198740 10 client.17928 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:26:49.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.803+0000 7ff892198740 10 client.17928 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:49.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.803+0000 7ff892198740 20 client.17928 populate_metadata read hostname 'smithi107' 2024-04-25T00:26:49.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.803+0000 7ff892198740 1 --2- 172.21.15.107:0/848103417 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55fbc1a65670 0x55fbc1a67aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.803+0000 7ff892198740 4 client.17928 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:49.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.803+0000 7ff892198740 1 -- 172.21.15.107:0/848103417 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbc19bcbb0 con 0x55fbc1a65670 2024-04-25T00:26:49.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.803+0000 7ff88d145640 1 --2- 172.21.15.107:0/848103417 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55fbc1a65670 0x55fbc1a67aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.804+0000 7ff870ff9640 10 client.17928 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:26:49.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.804+0000 7ff88d145640 1 --2- 172.21.15.107:0/848103417 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55fbc1a65670 0x55fbc1a67aa0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=535706ed9fb4e53c server_cookie=5768c1c7df72bc12 in_seq=0 out_seq=0 2024-04-25T00:26:49.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.804+0000 7ff870ff9640 1 -- 172.21.15.107:0/848103417 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fbc19bcbb0 con 0x55fbc1a65670 2024-04-25T00:26:49.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.804+0000 7ff870ff9640 10 client.17928 handle_command_reply: tid=0 2024-04-25T00:26:49.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.804+0000 7ff892198740 1 client.17928 shutdown 2024-04-25T00:26:49.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.805+0000 7ff892198740 1 -- 172.21.15.107:0/848103417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4aea0 msgr2=0x55fbc1a521c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.805+0000 7ff892198740 1 --2- 172.21.15.107:0/848103417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4aea0 0x55fbc1a521c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7ff8780314d0 tx=0x7ff878009260 comp rx=0 tx=0).stop 2024-04-25T00:26:49.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.805+0000 7ff892198740 1 -- 172.21.15.107:0/848103417 shutdown_connections 2024-04-25T00:26:49.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.805+0000 7ff892198740 1 --2- 172.21.15.107:0/848103417 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55fbc1a65670 0x55fbc1a67aa0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.805+0000 7ff892198740 1 --2- 172.21.15.107:0/848103417 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fbc1a55570 0x55fbc1a52720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.805+0000 7ff892198740 1 --2- 172.21.15.107:0/848103417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4aea0 0x55fbc1a521c0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.805+0000 7ff870ff9640 0 client.17928 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:26:49.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.805+0000 7ff892198740 1 --2- 172.21.15.107:0/848103417 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbc1a4a580 0x55fbc1a54ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.805+0000 7ff892198740 1 -- 172.21.15.107:0/848103417 >> 172.21.15.107:0/848103417 conn(0x55fbc1a468a0 msgr2=0x55fbc1a45b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.805+0000 7ff892198740 1 -- 172.21.15.107:0/848103417 shutdown_connections 2024-04-25T00:26:49.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.805+0000 7ff892198740 1 -- 172.21.15.107:0/848103417 wait complete. 2024-04-25T00:26:49.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.805+0000 7ff892198740 20 client.17928 trim_cache size 0 max 16384 2024-04-25T00:26:49.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.818+0000 7ff892198740 1 Processor -- start 2024-04-25T00:26:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.818+0000 7ff892198740 1 -- start start 2024-04-25T00:26:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.818+0000 7ff892198740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fbc1a4a580 0x55fbc1a61280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.818+0000 7ff892198740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4aea0 0x55fbc1a54a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.818+0000 7ff892198740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbc1a55570 0x55fbc1a6d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.818+0000 7ff892198740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881b7af0 con 0x55fbc1a4aea0 2024-04-25T00:26:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.818+0000 7ff892198740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881b53c0 con 0x55fbc1a4a580 2024-04-25T00:26:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.818+0000 7ff892198740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881b3230 con 0x55fbc1a55570 2024-04-25T00:26:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.818+0000 7ff88c944640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4aea0 0x55fbc1a54a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.818+0000 7ff88c944640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4aea0 0x55fbc1a54a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57002/0 (socket says 172.21.15.107:57002) 2024-04-25T00:26:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.818+0000 7ff88c944640 1 -- 172.21.15.107:0/2316102165 learned_addr learned my addr 172.21.15.107:0/2316102165 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.818+0000 7ff88d145640 1 --2- 172.21.15.107:0/2316102165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fbc1a4a580 0x55fbc1a61280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.818+0000 7ff88c944640 1 -- 172.21.15.107:0/2316102165 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbc1a55570 msgr2=0x55fbc1a6d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.818+0000 7ff88c944640 1 --2- 172.21.15.107:0/2316102165 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbc1a55570 0x55fbc1a6d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.818+0000 7ff88c944640 1 -- 172.21.15.107:0/2316102165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fbc1a4a580 msgr2=0x55fbc1a61280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.818+0000 7ff88c944640 1 --2- 172.21.15.107:0/2316102165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fbc1a4a580 0x55fbc1a61280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.818+0000 7ff88c944640 1 -- 172.21.15.107:0/2316102165 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbc178d7d0 con 0x55fbc1a4aea0 2024-04-25T00:26:49.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.818+0000 7ff88c944640 1 --2- 172.21.15.107:0/2316102165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4aea0 0x55fbc1a54a00 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7ff878009230 tx=0x7ff87803b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc0bfeffbf321cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.818+0000 7ff872ffd640 1 -- 172.21.15.107:0/2316102165 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87805f070 con 0x55fbc1a4aea0 2024-04-25T00:26:49.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.818+0000 7ff872ffd640 1 -- 172.21.15.107:0/2316102165 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff878005510 con 0x55fbc1a4aea0 2024-04-25T00:26:49.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.818+0000 7ff872ffd640 1 -- 172.21.15.107:0/2316102165 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87804b070 con 0x55fbc1a4aea0 2024-04-25T00:26:49.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.819+0000 7ff892198740 1 -- 172.21.15.107:0/2316102165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4aea0 msgr2=0x55fbc1a54a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.819+0000 7ff892198740 1 --2- 172.21.15.107:0/2316102165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4aea0 0x55fbc1a54a00 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7ff878009230 tx=0x7ff87803b000 comp rx=0 tx=0).stop 2024-04-25T00:26:49.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.819+0000 7ff892198740 1 -- 172.21.15.107:0/2316102165 shutdown_connections 2024-04-25T00:26:49.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.819+0000 7ff892198740 1 --2- 172.21.15.107:0/2316102165 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbc1a55570 0x55fbc1a6d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.819+0000 7ff892198740 1 --2- 172.21.15.107:0/2316102165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4aea0 0x55fbc1a54a00 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.819+0000 7ff892198740 1 --2- 172.21.15.107:0/2316102165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fbc1a4a580 0x55fbc1a61280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.819+0000 7ff892198740 1 -- 172.21.15.107:0/2316102165 >> 172.21.15.107:0/2316102165 conn(0x55fbc1a468a0 msgr2=0x55fbc1a4d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.819+0000 7ff892198740 1 -- 172.21.15.107:0/2316102165 shutdown_connections 2024-04-25T00:26:49.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.819+0000 7ff892198740 1 -- 172.21.15.107:0/2316102165 wait complete. 2024-04-25T00:26:49.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.819+0000 7ff892198740 1 -- start start 2024-04-25T00:26:49.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.819+0000 7ff892198740 1 Processor -- start 2024-04-25T00:26:49.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.820+0000 7ff892198740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fbc1a4a580 0x55fbc1a5dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.820+0000 7ff892198740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4aea0 0x55fbc1a4ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.820+0000 7ff892198740 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbc1a55570 0x55fbc1a4f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.820+0000 7ff892198740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881b3880 con 0x55fbc1a4aea0 2024-04-25T00:26:49.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.820+0000 7ff892198740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881b4cf0 con 0x55fbc1a4a580 2024-04-25T00:26:49.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.820+0000 7ff892198740 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff8881b3f50 con 0x55fbc1a55570 2024-04-25T00:26:49.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.820+0000 7ff892198740 10 client.0 _ll_register_callbacks cb 0x7ff878009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:49.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.820+0000 7ff88c944640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4aea0 0x55fbc1a4ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.820+0000 7ff88c944640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4aea0 0x55fbc1a4ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57010/0 (socket says 172.21.15.107:57010) 2024-04-25T00:26:49.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.820+0000 7ff88c944640 1 -- 172.21.15.107:0/3881234806 learned_addr learned my addr 172.21.15.107:0/3881234806 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.820+0000 7ff88d145640 1 --2- 172.21.15.107:0/3881234806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fbc1a4a580 0x55fbc1a5dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.820+0000 7ff88c944640 1 -- 172.21.15.107:0/3881234806 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbc1a55570 msgr2=0x55fbc1a4f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.820+0000 7ff88d946640 1 --2- 172.21.15.107:0/3881234806 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbc1a55570 0x55fbc1a4f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.820+0000 7ff88c944640 1 --2- 172.21.15.107:0/3881234806 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbc1a55570 0x55fbc1a4f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.820+0000 7ff88c944640 1 -- 172.21.15.107:0/3881234806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fbc1a4a580 msgr2=0x55fbc1a5dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.820+0000 7ff88c944640 1 --2- 172.21.15.107:0/3881234806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fbc1a4a580 0x55fbc1a5dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.820+0000 7ff88c944640 1 -- 172.21.15.107:0/3881234806 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbc1a4fb50 con 0x55fbc1a4aea0 2024-04-25T00:26:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.820+0000 7ff88c944640 1 --2- 172.21.15.107:0/3881234806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4aea0 0x55fbc1a4ed10 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7ff87805d040 tx=0x7ff87802ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b84fc41a189f19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.820+0000 7ff870ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T00:26:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.821+0000 7ff870ff9640 1 -- 172.21.15.107:0/3881234806 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87804b070 con 0x55fbc1a4aea0 2024-04-25T00:26:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.821+0000 7ff870ff9640 1 -- 172.21.15.107:0/3881234806 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7ff8780035f0 con 0x55fbc1a4aea0 2024-04-25T00:26:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.821+0000 7ff892198740 1 -- 172.21.15.107:0/3881234806 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff878005510 con 0x55fbc1a4aea0 2024-04-25T00:26:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.821+0000 7ff870ff9640 1 -- 172.21.15.107:0/3881234806 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff87805f070 con 0x55fbc1a4aea0 2024-04-25T00:26:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.821+0000 7ff892198740 10 client.17934 fetch_fsmap learned FSMap version 34 2024-04-25T00:26:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.821+0000 7ff892198740 1 -- 172.21.15.107:0/3881234806 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fbc1acbce0 con 0x55fbc1a4aea0 2024-04-25T00:26:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.821+0000 7ff870ff9640 1 -- 172.21.15.107:0/3881234806 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff878005510 con 0x55fbc1a4aea0 2024-04-25T00:26:49.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.821+0000 7ff870ff9640 1 -- 172.21.15.107:0/3881234806 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7ff87803ba70 con 0x55fbc1a4aea0 2024-04-25T00:26:49.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.821+0000 7ff892198740 10 client.17934 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T00:26:49.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.821+0000 7ff892198740 10 client.17934 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:49.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.821+0000 7ff892198740 20 client.17934 populate_metadata read hostname 'smithi107' 2024-04-25T00:26:49.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.821+0000 7ff892198740 1 --2- 172.21.15.107:0/3881234806 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55fbc1acc1b0 0x55fbc1ace5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:49.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.821+0000 7ff892198740 4 client.17934 mds_command: new command op to 24521 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T00:26:49.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.821+0000 7ff892198740 1 -- 172.21.15.107:0/3881234806 --> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fbc19b6f00 con 0x55fbc1acc1b0 2024-04-25T00:26:49.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.821+0000 7ff88d145640 1 --2- 172.21.15.107:0/3881234806 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55fbc1acc1b0 0x55fbc1ace5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:49.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.822+0000 7ff870ff9640 10 client.17934 ms_handle_connect on v2:172.21.15.170:6838/3558784302 2024-04-25T00:26:49.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.822+0000 7ff88d145640 1 --2- 172.21.15.107:0/3881234806 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55fbc1acc1b0 0x55fbc1ace5a0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a16865f9145e97bb server_cookie=c371fb59d294cd56 in_seq=0 out_seq=0 2024-04-25T00:26:49.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.822+0000 7ff870ff9640 1 -- 172.21.15.107:0/3881234806 <== mds.0 v2:172.21.15.170:6838/3558784302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fbc19b6f00 con 0x55fbc1acc1b0 2024-04-25T00:26:49.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.822+0000 7ff870ff9640 10 client.17934 handle_command_reply: tid=0 2024-04-25T00:26:49.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.822+0000 7ff892198740 1 client.17934 shutdown 2024-04-25T00:26:49.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.822+0000 7ff892198740 1 -- 172.21.15.107:0/3881234806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4aea0 msgr2=0x55fbc1a4ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.822+0000 7ff892198740 1 --2- 172.21.15.107:0/3881234806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4aea0 0x55fbc1a4ed10 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7ff87805d040 tx=0x7ff87802ac20 comp rx=0 tx=0).stop 2024-04-25T00:26:49.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.822+0000 7ff892198740 1 -- 172.21.15.107:0/3881234806 shutdown_connections 2024-04-25T00:26:49.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.822+0000 7ff892198740 1 --2- 172.21.15.107:0/3881234806 >> [v2:172.21.15.170:6838/3558784302,v1:172.21.15.170:6839/3558784302] conn(0x55fbc1acc1b0 0x55fbc1ace5a0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.822+0000 7ff892198740 1 --2- 172.21.15.107:0/3881234806 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55fbc1a55570 0x55fbc1a4f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.822+0000 7ff892198740 1 --2- 172.21.15.107:0/3881234806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fbc1a4aea0 0x55fbc1a4ed10 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.822+0000 7ff892198740 1 --2- 172.21.15.107:0/3881234806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fbc1a4a580 0x55fbc1a5dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.822+0000 7ff870ff9640 0 client.17934 ms_handle_reset on v2:172.21.15.170:6838/3558784302 2024-04-25T00:26:49.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.822+0000 7ff892198740 1 -- 172.21.15.107:0/3881234806 >> 172.21.15.107:0/3881234806 conn(0x55fbc1a468a0 msgr2=0x55fbc1a4bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.822+0000 7ff892198740 1 -- 172.21.15.107:0/3881234806 shutdown_connections 2024-04-25T00:26:49.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.822+0000 7ff892198740 1 -- 172.21.15.107:0/3881234806 wait complete. 2024-04-25T00:26:49.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.822+0000 7ff892198740 20 client.17934 trim_cache size 0 max 16384 2024-04-25T00:26:49.832 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-04-25T00:26:49.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.823+0000 7ff88f3a9640 1 -- 172.21.15.107:0/3973972626 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff8740a2d80 msgr2=0x7ff8740a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.823+0000 7ff88f3a9640 1 --2- 172.21.15.107:0/3973972626 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff8740a2d80 0x7ff8740a5240 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff878000c00 tx=0x7ff878032d50 comp rx=0 tx=0).stop 2024-04-25T00:26:49.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.823+0000 7ff88f3a9640 1 -- 172.21.15.107:0/3973972626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8881007c0 msgr2=0x7ff88819e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:49.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.823+0000 7ff88f3a9640 1 --2- 172.21.15.107:0/3973972626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8881007c0 0x7ff88819e5c0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7ff88400c920 tx=0x7ff88400cde0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.823+0000 7ff88f3a9640 1 -- 172.21.15.107:0/3973972626 shutdown_connections 2024-04-25T00:26:49.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.823+0000 7ff88f3a9640 1 --2- 172.21.15.107:0/3973972626 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff888105ae0 0x7ff8881a2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.823+0000 7ff88f3a9640 1 --2- 172.21.15.107:0/3973972626 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff8740a2d80 0x7ff8740a5240 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.823+0000 7ff88f3a9640 1 --2- 172.21.15.107:0/3973972626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff8881010e0 0x7ff88819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.823+0000 7ff88f3a9640 1 --2- 172.21.15.107:0/3973972626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8881007c0 0x7ff88819e5c0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:49.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.823+0000 7ff88f3a9640 1 -- 172.21.15.107:0/3973972626 >> 172.21.15.107:0/3973972626 conn(0x7ff8880769c0 msgr2=0x7ff888103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:49.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.823+0000 7ff88f3a9640 1 -- 172.21.15.107:0/3973972626 shutdown_connections 2024-04-25T00:26:49.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:49.823+0000 7ff88f3a9640 1 -- 172.21.15.107:0/3973972626 wait complete. 2024-04-25T00:26:50.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:50 smithi170 ceph-mon[32807]: pgmap v322: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T00:26:50.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:50 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2132240462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:26:50.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:50 smithi107 ceph-mon[29153]: pgmap v322: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T00:26:50.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:50 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2132240462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:26:50.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:50 smithi195 ceph-mon[32833]: pgmap v322: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T00:26:50.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:50 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2132240462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:26:50.835 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-04-25T00:26:50.835 INFO:tasks.fwd_scrub:done joining 2024-04-25T00:26:50.836 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T00:26:50.846 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:26:50.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.952+0000 7f01ac8bd640 1 -- 172.21.15.107:0/2813896156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01a81020d0 msgr2=0x7f01a810e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:50.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.952+0000 7f01ac8bd640 1 --2- 172.21.15.107:0/2813896156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01a81020d0 0x7f01a810e5e0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f01a0009cd0 tx=0x7f01a002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:26:50.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.952+0000 7f01ac8bd640 1 -- 172.21.15.107:0/2813896156 shutdown_connections 2024-04-25T00:26:50.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.952+0000 7f01ac8bd640 1 --2- 172.21.15.107:0/2813896156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01a81020d0 0x7f01a810e5e0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:50.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.952+0000 7f01ac8bd640 1 --2- 172.21.15.107:0/2813896156 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01a8101710 0x7f01a8101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:50.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.952+0000 7f01ac8bd640 1 --2- 172.21.15.107:0/2813896156 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01a8107720 0x7f01a8107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:50.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.952+0000 7f01ac8bd640 1 -- 172.21.15.107:0/2813896156 >> 172.21.15.107:0/2813896156 conn(0x7f01a8078b00 msgr2=0x7f01a8078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:50.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.952+0000 7f01ac8bd640 1 -- 172.21.15.107:0/2813896156 shutdown_connections 2024-04-25T00:26:50.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.952+0000 7f01ac8bd640 1 -- 172.21.15.107:0/2813896156 wait complete. 2024-04-25T00:26:50.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.953+0000 7f01ac8bd640 1 Processor -- start 2024-04-25T00:26:50.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.953+0000 7f01ac8bd640 1 -- start start 2024-04-25T00:26:50.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.953+0000 7f01ac8bd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01a8101710 0x7f01a8198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:50.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.953+0000 7f01ac8bd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01a81020d0 0x7f01a8198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:50.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.953+0000 7f01ac8bd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01a8107720 0x7f01a819c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:50.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.953+0000 7f01ac8bd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f01a8111b30 con 0x7f01a8101710 2024-04-25T00:26:50.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.953+0000 7f01ac8bd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f01a81119b0 con 0x7f01a8107720 2024-04-25T00:26:50.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.953+0000 7f01ac8bd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f01a8111cb0 con 0x7f01a81020d0 2024-04-25T00:26:50.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.953+0000 7f01a659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01a8101710 0x7f01a8198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:50.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.953+0000 7f01a659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01a8101710 0x7f01a8198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57032/0 (socket says 172.21.15.107:57032) 2024-04-25T00:26:50.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.953+0000 7f01a659c640 1 -- 172.21.15.107:0/4293401724 learned_addr learned my addr 172.21.15.107:0/4293401724 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:50.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.953+0000 7f01a6d9d640 1 --2- 172.21.15.107:0/4293401724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01a8107720 0x7f01a819c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:50.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.953+0000 7f01a659c640 1 -- 172.21.15.107:0/4293401724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01a81020d0 msgr2=0x7f01a8198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:50.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.953+0000 7f01a659c640 1 --2- 172.21.15.107:0/4293401724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01a81020d0 0x7f01a8198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:50.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.953+0000 7f01a659c640 1 -- 172.21.15.107:0/4293401724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01a8107720 msgr2=0x7f01a819c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:50.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.953+0000 7f01a659c640 1 --2- 172.21.15.107:0/4293401724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01a8107720 0x7f01a819c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:50.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.953+0000 7f01a659c640 1 -- 172.21.15.107:0/4293401724 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a819cff0 con 0x7f01a8101710 2024-04-25T00:26:50.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.953+0000 7f01a6d9d640 1 --2- 172.21.15.107:0/4293401724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01a8107720 0x7f01a819c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:50.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.954+0000 7f01a659c640 1 --2- 172.21.15.107:0/4293401724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01a8101710 0x7f01a8198050 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f019c00d770 tx=0x7f019c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:50.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.954+0000 7f01977fe640 1 -- 172.21.15.107:0/4293401724 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c017070 con 0x7f01a8101710 2024-04-25T00:26:50.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.954+0000 7f01977fe640 1 -- 172.21.15.107:0/4293401724 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f019c005500 con 0x7f01a8101710 2024-04-25T00:26:50.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.954+0000 7f01ac8bd640 1 -- 172.21.15.107:0/4293401724 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a819d2e0 con 0x7f01a8101710 2024-04-25T00:26:50.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.954+0000 7f01977fe640 1 -- 172.21.15.107:0/4293401724 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c004e00 con 0x7f01a8101710 2024-04-25T00:26:50.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.954+0000 7f01ac8bd640 1 -- 172.21.15.107:0/4293401724 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a81a4b70 con 0x7f01a8101710 2024-04-25T00:26:50.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.955+0000 7f01ac8bd640 1 -- 172.21.15.107:0/4293401724 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01a80689b0 con 0x7f01a8101710 2024-04-25T00:26:50.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.956+0000 7f01977fe640 1 -- 172.21.15.107:0/4293401724 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f019c004fa0 con 0x7f01a8101710 2024-04-25T00:26:50.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.956+0000 7f01977fe640 1 --2- 172.21.15.107:0/4293401724 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f01900a2d00 0x7f01900a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:50.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.956+0000 7f01a5d9b640 1 --2- 172.21.15.107:0/4293401724 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f01900a2d00 0x7f01900a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:50.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.956+0000 7f01977fe640 1 -- 172.21.15.107:0/4293401724 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f019c0cf2b0 con 0x7f01a8101710 2024-04-25T00:26:50.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.957+0000 7f01a5d9b640 1 --2- 172.21.15.107:0/4293401724 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f01900a2d00 0x7f01900a51c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0198005fd0 tx=0x7f0198005850 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:50.961 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:50.961+0000 7f01977fe640 1 -- 172.21.15.107:0/4293401724 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f019c09bb10 con 0x7f01a8101710 2024-04-25T00:26:51.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.138+0000 7f01ac8bd640 1 -- 172.21.15.107:0/4293401724 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01a8199350 con 0x7f01a8101710 2024-04-25T00:26:51.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.140+0000 7f01977fe640 1 -- 172.21.15.107:0/4293401724 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+25256 (secure 0 0 0) 0x7f019c09b410 con 0x7f01a8101710 2024-04-25T00:26:51.140 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:26:51.141 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":105,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:26:16.670245+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:59.672801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:00.957782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:14:05.065846+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8},{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/151603643":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/1884923543":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/111369358":"2024-04-26T00:16:26.229442+0000","172.21.15.107:6800/1644942185":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3936375305":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/1644942185":"2024-04-26T00:16:26.229442+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:26:51.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.142+0000 7f01ac8bd640 1 -- 172.21.15.107:0/4293401724 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f01900a2d00 msgr2=0x7f01900a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:51.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.142+0000 7f01ac8bd640 1 --2- 172.21.15.107:0/4293401724 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f01900a2d00 0x7f01900a51c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0198005fd0 tx=0x7f0198005850 comp rx=0 tx=0).stop 2024-04-25T00:26:51.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.142+0000 7f01ac8bd640 1 -- 172.21.15.107:0/4293401724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01a8101710 msgr2=0x7f01a8198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:51.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.142+0000 7f01ac8bd640 1 --2- 172.21.15.107:0/4293401724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01a8101710 0x7f01a8198050 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f019c00d770 tx=0x7f019c00dc30 comp rx=0 tx=0).stop 2024-04-25T00:26:51.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.142+0000 7f01ac8bd640 1 -- 172.21.15.107:0/4293401724 shutdown_connections 2024-04-25T00:26:51.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.142+0000 7f01ac8bd640 1 --2- 172.21.15.107:0/4293401724 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f01900a2d00 0x7f01900a51c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.142+0000 7f01ac8bd640 1 --2- 172.21.15.107:0/4293401724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01a8107720 0x7f01a819c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.142+0000 7f01ac8bd640 1 --2- 172.21.15.107:0/4293401724 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f01a81020d0 0x7f01a8198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.142+0000 7f01ac8bd640 1 --2- 172.21.15.107:0/4293401724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01a8101710 0x7f01a8198050 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.142+0000 7f01ac8bd640 1 -- 172.21.15.107:0/4293401724 >> 172.21.15.107:0/4293401724 conn(0x7f01a8078b00 msgr2=0x7f01a8104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:51.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.142+0000 7f01ac8bd640 1 -- 172.21.15.107:0/4293401724 shutdown_connections 2024-04-25T00:26:51.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.142+0000 7f01ac8bd640 1 -- 172.21.15.107:0/4293401724 wait complete. 2024-04-25T00:26:51.154 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:09:25.160329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:13:59.672801+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:14:00.957782+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:14:05.065846+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 104, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:26:51.154 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:26:51.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.257+0000 7ff599eae640 1 -- 172.21.15.107:0/1637791035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff594100610 msgr2=0x7ff594100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:51.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.257+0000 7ff599eae640 1 --2- 172.21.15.107:0/1637791035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff594100610 0x7ff594100a90 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009930 tx=0x7ff57c02a420 comp rx=0 tx=0).stop 2024-04-25T00:26:51.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.257+0000 7ff599eae640 1 -- 172.21.15.107:0/1637791035 shutdown_connections 2024-04-25T00:26:51.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.257+0000 7ff599eae640 1 --2- 172.21.15.107:0/1637791035 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff594100fd0 0x7ff59410d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.257+0000 7ff599eae640 1 --2- 172.21.15.107:0/1637791035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff594100610 0x7ff594100a90 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.257+0000 7ff599eae640 1 --2- 172.21.15.107:0/1637791035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff5941066b0 0x7ff594106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.257+0000 7ff599eae640 1 -- 172.21.15.107:0/1637791035 >> 172.21.15.107:0/1637791035 conn(0x7ff594077740 msgr2=0x7ff594077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:51.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.257+0000 7ff599eae640 1 -- 172.21.15.107:0/1637791035 shutdown_connections 2024-04-25T00:26:51.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.257+0000 7ff599eae640 1 -- 172.21.15.107:0/1637791035 wait complete. 2024-04-25T00:26:51.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.257+0000 7ff599eae640 1 Processor -- start 2024-04-25T00:26:51.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.258+0000 7ff599eae640 1 -- start start 2024-04-25T00:26:51.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.258+0000 7ff599eae640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff594100610 0x7ff59419a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:51.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.258+0000 7ff599eae640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff594100fd0 0x7ff59419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:51.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.258+0000 7ff599eae640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5941066b0 0x7ff59419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:51.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.258+0000 7ff599eae640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff5941109f0 con 0x7ff594100fd0 2024-04-25T00:26:51.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.258+0000 7ff599eae640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff594110870 con 0x7ff594100610 2024-04-25T00:26:51.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.258+0000 7ff599eae640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ff594110b70 con 0x7ff5941066b0 2024-04-25T00:26:51.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.258+0000 7ff58bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff594100fd0 0x7ff59419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:51.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.258+0000 7ff58bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff594100fd0 0x7ff59419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57052/0 (socket says 172.21.15.107:57052) 2024-04-25T00:26:51.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.258+0000 7ff58bfff640 1 -- 172.21.15.107:0/3686426330 learned_addr learned my addr 172.21.15.107:0/3686426330 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:51.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.258+0000 7ff5937fe640 1 --2- 172.21.15.107:0/3686426330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff594100610 0x7ff59419a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:51.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.258+0000 7ff593fff640 1 --2- 172.21.15.107:0/3686426330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5941066b0 0x7ff59419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:51.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.258+0000 7ff58bfff640 1 -- 172.21.15.107:0/3686426330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5941066b0 msgr2=0x7ff59419e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:51.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.258+0000 7ff58bfff640 1 --2- 172.21.15.107:0/3686426330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5941066b0 0x7ff59419e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.258+0000 7ff58bfff640 1 -- 172.21.15.107:0/3686426330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff594100610 msgr2=0x7ff59419a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:51.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.258+0000 7ff58bfff640 1 --2- 172.21.15.107:0/3686426330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff594100610 0x7ff59419a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.258+0000 7ff58bfff640 1 -- 172.21.15.107:0/3686426330 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff59419f0a0 con 0x7ff594100fd0 2024-04-25T00:26:51.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.258+0000 7ff593fff640 1 --2- 172.21.15.107:0/3686426330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5941066b0 0x7ff59419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:51.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.258+0000 7ff58bfff640 1 --2- 172.21.15.107:0/3686426330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff594100fd0 0x7ff59419a5f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009a50 tx=0x7ff57c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:51.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.258+0000 7ff5917fa640 1 -- 172.21.15.107:0/3686426330 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c03f030 con 0x7ff594100fd0 2024-04-25T00:26:51.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.258+0000 7ff599eae640 1 -- 172.21.15.107:0/3686426330 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff59419f330 con 0x7ff594100fd0 2024-04-25T00:26:51.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.259+0000 7ff5917fa640 1 -- 172.21.15.107:0/3686426330 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff57c003ce0 con 0x7ff594100fd0 2024-04-25T00:26:51.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.259+0000 7ff5917fa640 1 -- 172.21.15.107:0/3686426330 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c035070 con 0x7ff594100fd0 2024-04-25T00:26:51.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.259+0000 7ff599eae640 1 -- 172.21.15.107:0/3686426330 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5941a6c20 con 0x7ff594100fd0 2024-04-25T00:26:51.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.260+0000 7ff599eae640 1 -- 172.21.15.107:0/3686426330 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff594068560 con 0x7ff594100fd0 2024-04-25T00:26:51.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.261+0000 7ff5917fa640 1 -- 172.21.15.107:0/3686426330 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ff57c042030 con 0x7ff594100fd0 2024-04-25T00:26:51.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.261+0000 7ff5917fa640 1 --2- 172.21.15.107:0/3686426330 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff5800a2d00 0x7ff5800a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:51.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.261+0000 7ff5937fe640 1 --2- 172.21.15.107:0/3686426330 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff5800a2d00 0x7ff5800a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:51.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.261+0000 7ff5917fa640 1 -- 172.21.15.107:0/3686426330 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7ff57c0efd90 con 0x7ff594100fd0 2024-04-25T00:26:51.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.262+0000 7ff5937fe640 1 --2- 172.21.15.107:0/3686426330 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff5800a2d00 0x7ff5800a51c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff594101750 tx=0x7ff584005f90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:51.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.266+0000 7ff5917fa640 1 -- 172.21.15.107:0/3686426330 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff57c0bc5f0 con 0x7ff594100fd0 2024-04-25T00:26:51.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.450+0000 7ff599eae640 1 -- 172.21.15.107:0/3686426330 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff5941a6e20 con 0x7ff594100fd0 2024-04-25T00:26:51.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.451+0000 7ff5917fa640 1 -- 172.21.15.107:0/3686426330 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7ff57c0bbef0 con 0x7ff594100fd0 2024-04-25T00:26:51.452 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-04-25T00:26:51.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.453+0000 7ff599eae640 1 -- 172.21.15.107:0/3686426330 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff5800a2d00 msgr2=0x7ff5800a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:51.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.453+0000 7ff599eae640 1 --2- 172.21.15.107:0/3686426330 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff5800a2d00 0x7ff5800a51c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff594101750 tx=0x7ff584005f90 comp rx=0 tx=0).stop 2024-04-25T00:26:51.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.453+0000 7ff599eae640 1 -- 172.21.15.107:0/3686426330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff594100fd0 msgr2=0x7ff59419a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:51.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.453+0000 7ff599eae640 1 --2- 172.21.15.107:0/3686426330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff594100fd0 0x7ff59419a5f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7ff57c009a50 tx=0x7ff57c004270 comp rx=0 tx=0).stop 2024-04-25T00:26:51.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.453+0000 7ff599eae640 1 -- 172.21.15.107:0/3686426330 shutdown_connections 2024-04-25T00:26:51.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.453+0000 7ff599eae640 1 --2- 172.21.15.107:0/3686426330 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5941066b0 0x7ff59419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.453+0000 7ff599eae640 1 --2- 172.21.15.107:0/3686426330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff594100fd0 0x7ff59419a5f0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.453+0000 7ff599eae640 1 --2- 172.21.15.107:0/3686426330 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ff5800a2d00 0x7ff5800a51c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.453+0000 7ff599eae640 1 --2- 172.21.15.107:0/3686426330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff594100610 0x7ff59419a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.453+0000 7ff599eae640 1 -- 172.21.15.107:0/3686426330 >> 172.21.15.107:0/3686426330 conn(0x7ff594077740 msgr2=0x7ff5940ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:51.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.453+0000 7ff599eae640 1 -- 172.21.15.107:0/3686426330 shutdown_connections 2024-04-25T00:26:51.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.453+0000 7ff599eae640 1 -- 172.21.15.107:0/3686426330 wait complete. 2024-04-25T00:26:51.464 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:26:51.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.567+0000 7fe9339dc640 1 -- 172.21.15.107:0/3576245545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe92c106630 msgr2=0x7fe92c106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:51.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.567+0000 7fe9339dc640 1 --2- 172.21.15.107:0/3576245545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe92c106630 0x7fe92c106a10 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fe928009930 tx=0x7fe92802a420 comp rx=0 tx=0).stop 2024-04-25T00:26:51.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.567+0000 7fe9339dc640 1 -- 172.21.15.107:0/3576245545 shutdown_connections 2024-04-25T00:26:51.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.567+0000 7fe9339dc640 1 --2- 172.21.15.107:0/3576245545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe92c100fe0 0x7fe92c10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.567+0000 7fe9339dc640 1 --2- 172.21.15.107:0/3576245545 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe92c100620 0x7fe92c100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.567+0000 7fe9339dc640 1 --2- 172.21.15.107:0/3576245545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe92c106630 0x7fe92c106a10 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.567+0000 7fe9339dc640 1 -- 172.21.15.107:0/3576245545 >> 172.21.15.107:0/3576245545 conn(0x7fe92c0769c0 msgr2=0x7fe92c076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:51.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.568+0000 7fe9339dc640 1 -- 172.21.15.107:0/3576245545 shutdown_connections 2024-04-25T00:26:51.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.568+0000 7fe9339dc640 1 -- 172.21.15.107:0/3576245545 wait complete. 2024-04-25T00:26:51.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.568+0000 7fe9339dc640 1 Processor -- start 2024-04-25T00:26:51.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.568+0000 7fe9339dc640 1 -- start start 2024-04-25T00:26:51.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.569+0000 7fe9339dc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe92c100620 0x7fe92c198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:51.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.569+0000 7fe9339dc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe92c100fe0 0x7fe92c199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:51.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.569+0000 7fe9339dc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe92c106630 0x7fe92c1a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:51.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.569+0000 7fe9339dc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c110a00 con 0x7fe92c106630 2024-04-25T00:26:51.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.569+0000 7fe9339dc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c110880 con 0x7fe92c100fe0 2024-04-25T00:26:51.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.569+0000 7fe9339dc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c110b80 con 0x7fe92c100620 2024-04-25T00:26:51.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.569+0000 7fe931f79640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe92c106630 0x7fe92c1a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:51.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.569+0000 7fe931f79640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe92c106630 0x7fe92c1a3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57076/0 (socket says 172.21.15.107:57076) 2024-04-25T00:26:51.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.569+0000 7fe931f79640 1 -- 172.21.15.107:0/4175132059 learned_addr learned my addr 172.21.15.107:0/4175132059 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:51.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.569+0000 7fe931778640 1 --2- 172.21.15.107:0/4175132059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe92c100620 0x7fe92c198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:51.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.569+0000 7fe930f77640 1 --2- 172.21.15.107:0/4175132059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe92c100fe0 0x7fe92c199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:51.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.569+0000 7fe931f79640 1 -- 172.21.15.107:0/4175132059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe92c100620 msgr2=0x7fe92c198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:51.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.569+0000 7fe931f79640 1 --2- 172.21.15.107:0/4175132059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe92c100620 0x7fe92c198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.569+0000 7fe931f79640 1 -- 172.21.15.107:0/4175132059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe92c100fe0 msgr2=0x7fe92c199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:51.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.569+0000 7fe931f79640 1 --2- 172.21.15.107:0/4175132059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe92c100fe0 0x7fe92c199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.569+0000 7fe931f79640 1 -- 172.21.15.107:0/4175132059 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92c1a4480 con 0x7fe92c106630 2024-04-25T00:26:51.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.569+0000 7fe931f79640 1 --2- 172.21.15.107:0/4175132059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe92c106630 0x7fe92c1a3f40 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fe92400eaa0 tx=0x7fe92400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:51.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.569+0000 7fe9227fc640 1 -- 172.21.15.107:0/4175132059 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92400cdb0 con 0x7fe92c106630 2024-04-25T00:26:51.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.569+0000 7fe9227fc640 1 -- 172.21.15.107:0/4175132059 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe924005020 con 0x7fe92c106630 2024-04-25T00:26:51.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.570+0000 7fe9339dc640 1 -- 172.21.15.107:0/4175132059 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe92c1a4650 con 0x7fe92c106630 2024-04-25T00:26:51.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.570+0000 7fe9227fc640 1 -- 172.21.15.107:0/4175132059 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9240116b0 con 0x7fe92c106630 2024-04-25T00:26:51.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.570+0000 7fe9339dc640 1 -- 172.21.15.107:0/4175132059 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe92c1a4b20 con 0x7fe92c106630 2024-04-25T00:26:51.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.570+0000 7fe9339dc640 1 -- 172.21.15.107:0/4175132059 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe92c101d60 con 0x7fe92c106630 2024-04-25T00:26:51.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.571+0000 7fe9227fc640 1 -- 172.21.15.107:0/4175132059 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe924005560 con 0x7fe92c106630 2024-04-25T00:26:51.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.572+0000 7fe9227fc640 1 --2- 172.21.15.107:0/4175132059 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe9180a2d00 0x7fe9180a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:51.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.572+0000 7fe931778640 1 --2- 172.21.15.107:0/4175132059 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe9180a2d00 0x7fe9180a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:51.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.572+0000 7fe9227fc640 1 -- 172.21.15.107:0/4175132059 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7fe9240cf9d0 con 0x7fe92c106630 2024-04-25T00:26:51.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.573+0000 7fe931778640 1 --2- 172.21.15.107:0/4175132059 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe9180a2d00 0x7fe9180a51c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe928009900 tx=0x7fe9280058d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:51.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.577+0000 7fe9227fc640 1 -- 172.21.15.107:0/4175132059 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe924016030 con 0x7fe92c106630 2024-04-25T00:26:51.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.758+0000 7fe9339dc640 1 -- 172.21.15.107:0/4175132059 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe92c02d170 con 0x7fe92c106630 2024-04-25T00:26:51.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.759+0000 7fe9227fc640 1 -- 172.21.15.107:0/4175132059 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) ==== 104+0+11 (secure 0 0 0) 0x7fe92409bb80 con 0x7fe92c106630 2024-04-25T00:26:51.759 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-25T00:26:51.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.761+0000 7fe9339dc640 1 -- 172.21.15.107:0/4175132059 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe9180a2d00 msgr2=0x7fe9180a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:51.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.761+0000 7fe9339dc640 1 --2- 172.21.15.107:0/4175132059 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe9180a2d00 0x7fe9180a51c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe928009900 tx=0x7fe9280058d0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.761+0000 7fe9339dc640 1 -- 172.21.15.107:0/4175132059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe92c106630 msgr2=0x7fe92c1a3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:51.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.761+0000 7fe9339dc640 1 --2- 172.21.15.107:0/4175132059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe92c106630 0x7fe92c1a3f40 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fe92400eaa0 tx=0x7fe92400ef60 comp rx=0 tx=0).stop 2024-04-25T00:26:51.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.761+0000 7fe9339dc640 1 -- 172.21.15.107:0/4175132059 shutdown_connections 2024-04-25T00:26:51.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.761+0000 7fe9339dc640 1 --2- 172.21.15.107:0/4175132059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe92c106630 0x7fe92c1a3f40 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.761+0000 7fe9339dc640 1 --2- 172.21.15.107:0/4175132059 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe9180a2d00 0x7fe9180a51c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.761+0000 7fe9339dc640 1 --2- 172.21.15.107:0/4175132059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe92c100fe0 0x7fe92c199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.761+0000 7fe9339dc640 1 --2- 172.21.15.107:0/4175132059 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe92c100620 0x7fe92c198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.761+0000 7fe9339dc640 1 -- 172.21.15.107:0/4175132059 >> 172.21.15.107:0/4175132059 conn(0x7fe92c0769c0 msgr2=0x7fe92c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:51.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.761+0000 7fe9339dc640 1 -- 172.21.15.107:0/4175132059 shutdown_connections 2024-04-25T00:26:51.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.761+0000 7fe9339dc640 1 -- 172.21.15.107:0/4175132059 wait complete. 2024-04-25T00:26:51.773 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:26:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.876+0000 7f03f881c640 1 -- 172.21.15.107:0/465211508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03f4100fd0 msgr2=0x7f03f410d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.876+0000 7f03f881c640 1 --2- 172.21.15.107:0/465211508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03f4100fd0 0x7f03f410d4e0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f03ec009cd0 tx=0x7f03ec02a8e0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.876+0000 7f03f881c640 1 -- 172.21.15.107:0/465211508 shutdown_connections 2024-04-25T00:26:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.876+0000 7f03f881c640 1 --2- 172.21.15.107:0/465211508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03f4100fd0 0x7f03f410d4e0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.876+0000 7f03f881c640 1 --2- 172.21.15.107:0/465211508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03f4100610 0x7f03f4100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.876+0000 7f03f881c640 1 --2- 172.21.15.107:0/465211508 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03f41066b0 0x7f03f4106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.876+0000 7f03f881c640 1 -- 172.21.15.107:0/465211508 >> 172.21.15.107:0/465211508 conn(0x7f03f4077740 msgr2=0x7f03f4077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:51.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.876+0000 7f03f881c640 1 -- 172.21.15.107:0/465211508 shutdown_connections 2024-04-25T00:26:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.876+0000 7f03f881c640 1 -- 172.21.15.107:0/465211508 wait complete. 2024-04-25T00:26:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.877+0000 7f03f881c640 1 Processor -- start 2024-04-25T00:26:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.877+0000 7f03f881c640 1 -- start start 2024-04-25T00:26:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.877+0000 7f03f881c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03f4100610 0x7f03f4196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.877+0000 7f03f881c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03f4100fd0 0x7f03f41973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.877+0000 7f03f881c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03f41066b0 0x7f03f419b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.877+0000 7f03f881c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f03f41109f0 con 0x7f03f4100fd0 2024-04-25T00:26:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.877+0000 7f03f881c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f03f4110870 con 0x7f03f41066b0 2024-04-25T00:26:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.877+0000 7f03f881c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f03f4110b70 con 0x7f03f4100610 2024-04-25T00:26:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.877+0000 7f03ebfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03f4100fd0 0x7f03f41973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.877+0000 7f03ebfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03f4100fd0 0x7f03f41973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57086/0 (socket says 172.21.15.107:57086) 2024-04-25T00:26:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.877+0000 7f03ebfff640 1 -- 172.21.15.107:0/1316731041 learned_addr learned my addr 172.21.15.107:0/1316731041 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:51.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.878+0000 7f03f259c640 1 --2- 172.21.15.107:0/1316731041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03f4100610 0x7f03f4196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.878+0000 7f03f2d9d640 1 --2- 172.21.15.107:0/1316731041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03f41066b0 0x7f03f419b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.878+0000 7f03ebfff640 1 -- 172.21.15.107:0/1316731041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03f4100610 msgr2=0x7f03f4196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.878+0000 7f03ebfff640 1 --2- 172.21.15.107:0/1316731041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03f4100610 0x7f03f4196e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.878+0000 7f03ebfff640 1 -- 172.21.15.107:0/1316731041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03f41066b0 msgr2=0x7f03f419b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.878+0000 7f03ebfff640 1 --2- 172.21.15.107:0/1316731041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03f41066b0 0x7f03f419b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.878+0000 7f03ebfff640 1 -- 172.21.15.107:0/1316731041 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f419be30 con 0x7f03f4100fd0 2024-04-25T00:26:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.878+0000 7f03f2d9d640 1 --2- 172.21.15.107:0/1316731041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03f41066b0 0x7f03f419b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.878+0000 7f03ebfff640 1 --2- 172.21.15.107:0/1316731041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03f4100fd0 0x7f03f41973d0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009970 tx=0x7f03dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.878+0000 7f03eb7fe640 1 -- 172.21.15.107:0/1316731041 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc00d020 con 0x7f03f4100fd0 2024-04-25T00:26:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.878+0000 7f03f881c640 1 -- 172.21.15.107:0/1316731041 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f419c120 con 0x7f03f4100fd0 2024-04-25T00:26:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.878+0000 7f03eb7fe640 1 -- 172.21.15.107:0/1316731041 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03dc007d40 con 0x7f03f4100fd0 2024-04-25T00:26:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.878+0000 7f03eb7fe640 1 -- 172.21.15.107:0/1316731041 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03dc00ccc0 con 0x7f03f4100fd0 2024-04-25T00:26:51.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.878+0000 7f03f881c640 1 -- 172.21.15.107:0/1316731041 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f40688d0 con 0x7f03f4100fd0 2024-04-25T00:26:51.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.879+0000 7f03f881c640 1 -- 172.21.15.107:0/1316731041 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f4068560 con 0x7f03f4100fd0 2024-04-25T00:26:51.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.881+0000 7f03eb7fe640 1 -- 172.21.15.107:0/1316731041 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f03dc00c460 con 0x7f03f4100fd0 2024-04-25T00:26:51.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.882+0000 7f03eb7fe640 1 --2- 172.21.15.107:0/1316731041 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f03e00a2d00 0x7f03e00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:51.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.882+0000 7f03f259c640 1 --2- 172.21.15.107:0/1316731041 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f03e00a2d00 0x7f03e00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:51.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.882+0000 7f03eb7fe640 1 -- 172.21.15.107:0/1316731041 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f03dc0d11b0 con 0x7f03f4100fd0 2024-04-25T00:26:51.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.882+0000 7f03f259c640 1 --2- 172.21.15.107:0/1316731041 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f03e00a2d00 0x7f03e00a51c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f03e4005fd0 tx=0x7f03e4005850 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:51.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:51.885+0000 7f03eb7fe640 1 -- 172.21.15.107:0/1316731041 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f03dc09da10 con 0x7f03f4100fd0 2024-04-25T00:26:52.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.063+0000 7f03f881c640 1 -- 172.21.15.107:0/1316731041 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f03f40693c0 con 0x7f03f4100fd0 2024-04-25T00:26:52.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.064+0000 7f03eb7fe640 1 -- 172.21.15.107:0/1316731041 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) ==== 100+0+11 (secure 0 0 0) 0x7f03dc09d310 con 0x7f03f4100fd0 2024-04-25T00:26:52.065 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-04-25T00:26:52.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.066+0000 7f03f881c640 1 -- 172.21.15.107:0/1316731041 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f03e00a2d00 msgr2=0x7f03e00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:52.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.066+0000 7f03f881c640 1 --2- 172.21.15.107:0/1316731041 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f03e00a2d00 0x7f03e00a51c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f03e4005fd0 tx=0x7f03e4005850 comp rx=0 tx=0).stop 2024-04-25T00:26:52.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.066+0000 7f03f881c640 1 -- 172.21.15.107:0/1316731041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03f4100fd0 msgr2=0x7f03f41973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:52.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.066+0000 7f03f881c640 1 --2- 172.21.15.107:0/1316731041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03f4100fd0 0x7f03f41973d0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009970 tx=0x7f03dc009e30 comp rx=0 tx=0).stop 2024-04-25T00:26:52.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.066+0000 7f03f881c640 1 -- 172.21.15.107:0/1316731041 shutdown_connections 2024-04-25T00:26:52.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.066+0000 7f03f881c640 1 --2- 172.21.15.107:0/1316731041 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f03e00a2d00 0x7f03e00a51c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:52.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.066+0000 7f03f881c640 1 --2- 172.21.15.107:0/1316731041 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03f41066b0 0x7f03f419b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:52.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.066+0000 7f03f881c640 1 --2- 172.21.15.107:0/1316731041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03f4100fd0 0x7f03f41973d0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:52.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.066+0000 7f03f881c640 1 --2- 172.21.15.107:0/1316731041 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03f4100610 0x7f03f4196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:52.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.066+0000 7f03f881c640 1 -- 172.21.15.107:0/1316731041 >> 172.21.15.107:0/1316731041 conn(0x7f03f4077740 msgr2=0x7f03f40ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:52.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.066+0000 7f03f881c640 1 -- 172.21.15.107:0/1316731041 shutdown_connections 2024-04-25T00:26:52.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.066+0000 7f03f881c640 1 -- 172.21.15.107:0/1316731041 wait complete. 2024-04-25T00:26:52.077 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:26:52.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.179+0000 7f7016493640 1 -- 172.21.15.107:0/3291094425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f701010bd00 msgr2=0x7f701010e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:52.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.179+0000 7f7016493640 1 --2- 172.21.15.107:0/3291094425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f701010bd00 0x7f701010e1a0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f7008009ca0 tx=0x7f700802a9e0 comp rx=0 tx=0).stop 2024-04-25T00:26:52.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.179+0000 7f7016493640 1 -- 172.21.15.107:0/3291094425 shutdown_connections 2024-04-25T00:26:52.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.179+0000 7f7016493640 1 --2- 172.21.15.107:0/3291094425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f701010bd00 0x7f701010e1a0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:52.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.179+0000 7f7016493640 1 --2- 172.21.15.107:0/3291094425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70101032e0 0x7f701010b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:52.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.179+0000 7f7016493640 1 --2- 172.21.15.107:0/3291094425 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70101029c0 0x7f7010102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:52.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.179+0000 7f7016493640 1 -- 172.21.15.107:0/3291094425 >> 172.21.15.107:0/3291094425 conn(0x7f7010077740 msgr2=0x7f7010077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:52.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.180+0000 7f7016493640 1 -- 172.21.15.107:0/3291094425 shutdown_connections 2024-04-25T00:26:52.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.180+0000 7f7016493640 1 -- 172.21.15.107:0/3291094425 wait complete. 2024-04-25T00:26:52.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.180+0000 7f7016493640 1 Processor -- start 2024-04-25T00:26:52.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.180+0000 7f7016493640 1 -- start start 2024-04-25T00:26:52.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.180+0000 7f7016493640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70101029c0 0x7f7010197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:52.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.180+0000 7f7016493640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70101032e0 0x7f7010198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:52.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.180+0000 7f7016493640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f701010bd00 0x7f701019c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:52.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.180+0000 7f7016493640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7010111b00 con 0x7f70101032e0 2024-04-25T00:26:52.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.180+0000 7f7016493640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7010111980 con 0x7f701010bd00 2024-04-25T00:26:52.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.180+0000 7f7016493640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f7010111c80 con 0x7f70101029c0 2024-04-25T00:26:52.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.181+0000 7f700f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70101032e0 0x7f7010198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:52.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.181+0000 7f700f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70101032e0 0x7f7010198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57110/0 (socket says 172.21.15.107:57110) 2024-04-25T00:26:52.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.181+0000 7f700f7fe640 1 -- 172.21.15.107:0/3733985727 learned_addr learned my addr 172.21.15.107:0/3733985727 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:52.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.181+0000 7f700ffff640 1 --2- 172.21.15.107:0/3733985727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70101029c0 0x7f7010197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:52.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.181+0000 7f7014a30640 1 --2- 172.21.15.107:0/3733985727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f701010bd00 0x7f701019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:52.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.181+0000 7f700f7fe640 1 -- 172.21.15.107:0/3733985727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70101029c0 msgr2=0x7f7010197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:52.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.181+0000 7f700f7fe640 1 --2- 172.21.15.107:0/3733985727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70101029c0 0x7f7010197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:52.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.181+0000 7f700f7fe640 1 -- 172.21.15.107:0/3733985727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f701010bd00 msgr2=0x7f701019c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:52.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.181+0000 7f700f7fe640 1 --2- 172.21.15.107:0/3733985727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f701010bd00 0x7f701019c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:52.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.181+0000 7f700f7fe640 1 -- 172.21.15.107:0/3733985727 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701019cf30 con 0x7f70101032e0 2024-04-25T00:26:52.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.181+0000 7f700f7fe640 1 --2- 172.21.15.107:0/3733985727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70101032e0 0x7f7010198480 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f700000c920 tx=0x7f700000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:52.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.181+0000 7f700d7fa640 1 -- 172.21.15.107:0/3733985727 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000007c70 con 0x7f70101032e0 2024-04-25T00:26:52.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.181+0000 7f7016493640 1 -- 172.21.15.107:0/3733985727 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701019d220 con 0x7f70101032e0 2024-04-25T00:26:52.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.181+0000 7f700d7fa640 1 -- 172.21.15.107:0/3733985727 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7000007e10 con 0x7f70101032e0 2024-04-25T00:26:52.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.181+0000 7f700d7fa640 1 -- 172.21.15.107:0/3733985727 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000018890 con 0x7f70101032e0 2024-04-25T00:26:52.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.181+0000 7f7016493640 1 -- 172.21.15.107:0/3733985727 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70101a4ab0 con 0x7f70101032e0 2024-04-25T00:26:52.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.182+0000 7f7016493640 1 -- 172.21.15.107:0/3733985727 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7010068560 con 0x7f70101032e0 2024-04-25T00:26:52.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.184+0000 7f700d7fa640 1 -- 172.21.15.107:0/3733985727 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f7000018a50 con 0x7f70101032e0 2024-04-25T00:26:52.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.184+0000 7f700d7fa640 1 --2- 172.21.15.107:0/3733985727 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f6ff80a2d00 0x7f6ff80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:52.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.184+0000 7f700ffff640 1 --2- 172.21.15.107:0/3733985727 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f6ff80a2d00 0x7f6ff80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:52.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.184+0000 7f700d7fa640 1 -- 172.21.15.107:0/3733985727 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f70000cfab0 con 0x7f70101032e0 2024-04-25T00:26:52.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.185+0000 7f700ffff640 1 --2- 172.21.15.107:0/3733985727 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f6ff80a2d00 0x7f6ff80a51c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7004005fd0 tx=0x7f700400b390 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:52.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.191+0000 7f700d7fa640 1 -- 172.21.15.107:0/3733985727 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7000014030 con 0x7f70101032e0 2024-04-25T00:26:52.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.374+0000 7f7016493640 1 -- 172.21.15.107:0/3733985727 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f70101a4db0 con 0x7f70101032e0 2024-04-25T00:26:52.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.375+0000 7f700d7fa640 1 -- 172.21.15.107:0/3733985727 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v105) ==== 103+0+11 (secure 0 0 0) 0x7f700009c3a0 con 0x7f70101032e0 2024-04-25T00:26:52.376 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-25T00:26:52.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.377+0000 7f7016493640 1 -- 172.21.15.107:0/3733985727 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f6ff80a2d00 msgr2=0x7f6ff80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:52.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.377+0000 7f7016493640 1 --2- 172.21.15.107:0/3733985727 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f6ff80a2d00 0x7f6ff80a51c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7004005fd0 tx=0x7f700400b390 comp rx=0 tx=0).stop 2024-04-25T00:26:52.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.377+0000 7f7016493640 1 -- 172.21.15.107:0/3733985727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70101032e0 msgr2=0x7f7010198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:52.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.377+0000 7f7016493640 1 --2- 172.21.15.107:0/3733985727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70101032e0 0x7f7010198480 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f700000c920 tx=0x7f700000cde0 comp rx=0 tx=0).stop 2024-04-25T00:26:52.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.377+0000 7f7016493640 1 -- 172.21.15.107:0/3733985727 shutdown_connections 2024-04-25T00:26:52.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.377+0000 7f7016493640 1 --2- 172.21.15.107:0/3733985727 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f6ff80a2d00 0x7f6ff80a51c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:52.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.377+0000 7f7016493640 1 --2- 172.21.15.107:0/3733985727 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f701010bd00 0x7f701019c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:52.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.377+0000 7f7016493640 1 --2- 172.21.15.107:0/3733985727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70101032e0 0x7f7010198480 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:52.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.377+0000 7f7016493640 1 --2- 172.21.15.107:0/3733985727 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70101029c0 0x7f7010197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:52.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.377+0000 7f7016493640 1 -- 172.21.15.107:0/3733985727 >> 172.21.15.107:0/3733985727 conn(0x7f7010077740 msgr2=0x7f70100fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:52.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.377+0000 7f7016493640 1 -- 172.21.15.107:0/3733985727 shutdown_connections 2024-04-25T00:26:52.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.377+0000 7f7016493640 1 -- 172.21.15.107:0/3733985727 wait complete. 2024-04-25T00:26:52.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:52 smithi107 ceph-mon[29153]: pgmap v323: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:26:52.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:52 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/4293401724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:26:52.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:52 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3686426330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:26:52.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:52 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/4175132059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:26:52.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:52 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1316731041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:26:52.388 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T00:26:52.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:52 smithi170 ceph-mon[32807]: pgmap v323: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:26:52.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:52 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/4293401724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:26:52.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:52 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3686426330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:26:52.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:52 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/4175132059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:26:52.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:52 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1316731041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:26:52.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.487+0000 7f68f3d1d640 1 -- 172.21.15.107:0/2672753237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68ec100620 msgr2=0x7f68ec100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:52.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.487+0000 7f68f3d1d640 1 --2- 172.21.15.107:0/2672753237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68ec100620 0x7f68ec100aa0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009930 tx=0x7f68dc02a420 comp rx=0 tx=0).stop 2024-04-25T00:26:52.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.487+0000 7f68f3d1d640 1 -- 172.21.15.107:0/2672753237 shutdown_connections 2024-04-25T00:26:52.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.487+0000 7f68f3d1d640 1 --2- 172.21.15.107:0/2672753237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68ec100fe0 0x7f68ec10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:52.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.487+0000 7f68f3d1d640 1 --2- 172.21.15.107:0/2672753237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68ec100620 0x7f68ec100aa0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:52.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.487+0000 7f68f3d1d640 1 --2- 172.21.15.107:0/2672753237 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68ec106630 0x7f68ec106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:52.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.487+0000 7f68f3d1d640 1 -- 172.21.15.107:0/2672753237 >> 172.21.15.107:0/2672753237 conn(0x7f68ec077740 msgr2=0x7f68ec077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:52.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.487+0000 7f68f3d1d640 1 -- 172.21.15.107:0/2672753237 shutdown_connections 2024-04-25T00:26:52.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.487+0000 7f68f3d1d640 1 -- 172.21.15.107:0/2672753237 wait complete. 2024-04-25T00:26:52.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.487+0000 7f68f3d1d640 1 Processor -- start 2024-04-25T00:26:52.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.487+0000 7f68f3d1d640 1 -- start start 2024-04-25T00:26:52.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.488+0000 7f68f3d1d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68ec100620 0x7f68ec1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:52.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.488+0000 7f68f3d1d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68ec100fe0 0x7f68ec198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:52.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.488+0000 7f68f3d1d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68ec106630 0x7f68ec19c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:52.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.488+0000 7f68f3d1d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f68ec111b40 con 0x7f68ec106630 2024-04-25T00:26:52.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.488+0000 7f68f3d1d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f68ec1119c0 con 0x7f68ec100fe0 2024-04-25T00:26:52.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.488+0000 7f68f3d1d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f68ec111cc0 con 0x7f68ec100620 2024-04-25T00:26:52.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.488+0000 7f68f22ba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68ec106630 0x7f68ec19c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:52.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.488+0000 7f68f22ba640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68ec106630 0x7f68ec19c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57118/0 (socket says 172.21.15.107:57118) 2024-04-25T00:26:52.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.488+0000 7f68f22ba640 1 -- 172.21.15.107:0/3225446890 learned_addr learned my addr 172.21.15.107:0/3225446890 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:26:52.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.488+0000 7f68f1ab9640 1 --2- 172.21.15.107:0/3225446890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68ec100620 0x7f68ec1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:52.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.488+0000 7f68f12b8640 1 --2- 172.21.15.107:0/3225446890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68ec100fe0 0x7f68ec198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:52.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.488+0000 7f68f22ba640 1 -- 172.21.15.107:0/3225446890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68ec100620 msgr2=0x7f68ec1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:52.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.488+0000 7f68f22ba640 1 --2- 172.21.15.107:0/3225446890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68ec100620 0x7f68ec1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:52.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.488+0000 7f68f22ba640 1 -- 172.21.15.107:0/3225446890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68ec100fe0 msgr2=0x7f68ec198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:52.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.488+0000 7f68f22ba640 1 --2- 172.21.15.107:0/3225446890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68ec100fe0 0x7f68ec198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:52.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.488+0000 7f68f22ba640 1 -- 172.21.15.107:0/3225446890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68ec19d0b0 con 0x7f68ec106630 2024-04-25T00:26:52.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.488+0000 7f68f22ba640 1 --2- 172.21.15.107:0/3225446890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68ec106630 0x7f68ec19c980 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f68e800c970 tx=0x7f68e800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:52.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.488+0000 7f68e2ffd640 1 -- 172.21.15.107:0/3225446890 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e8007bb0 con 0x7f68ec106630 2024-04-25T00:26:52.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.488+0000 7f68e2ffd640 1 -- 172.21.15.107:0/3225446890 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68e8007d50 con 0x7f68ec106630 2024-04-25T00:26:52.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.488+0000 7f68f3d1d640 1 -- 172.21.15.107:0/3225446890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68ec19d3a0 con 0x7f68ec106630 2024-04-25T00:26:52.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.488+0000 7f68e2ffd640 1 -- 172.21.15.107:0/3225446890 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e800f450 con 0x7f68ec106630 2024-04-25T00:26:52.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.488+0000 7f68f3d1d640 1 -- 172.21.15.107:0/3225446890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68ec1a4c30 con 0x7f68ec106630 2024-04-25T00:26:52.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.489+0000 7f68f3d1d640 1 -- 172.21.15.107:0/3225446890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68ec068560 con 0x7f68ec106630 2024-04-25T00:26:52.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.491+0000 7f68e2ffd640 1 -- 172.21.15.107:0/3225446890 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f68e8004f00 con 0x7f68ec106630 2024-04-25T00:26:52.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.492+0000 7f68e2ffd640 1 --2- 172.21.15.107:0/3225446890 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f68d80a2d00 0x7f68d80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:52.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.492+0000 7f68f1ab9640 1 --2- 172.21.15.107:0/3225446890 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f68d80a2d00 0x7f68d80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:52.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.492+0000 7f68e2ffd640 1 -- 172.21.15.107:0/3225446890 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9483+0+0 (secure 0 0 0) 0x7f68e80cf930 con 0x7f68ec106630 2024-04-25T00:26:52.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.492+0000 7f68f1ab9640 1 --2- 172.21.15.107:0/3225446890 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f68d80a2d00 0x7f68d80a51c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f68e4009f20 tx=0x7f68e40094f0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:52.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.495+0000 7f68e2ffd640 1 -- 172.21.15.107:0/3225446890 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f68e8014030 con 0x7f68ec106630 2024-04-25T00:26:52.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:52 smithi195 ceph-mon[32833]: pgmap v323: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:26:52.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:52 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/4293401724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:26:52.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:52 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3686426330' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:26:52.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:52 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/4175132059' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:26:52.590 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:52 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1316731041' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:26:52.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.718+0000 7f68f3d1d640 1 -- 172.21.15.107:0/3225446890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f68ec108620 con 0x7f68ec106630 2024-04-25T00:26:52.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.726+0000 7f68e2ffd640 1 -- 172.21.15.107:0/3225446890 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+740912 (secure 0 0 0) 0x7f68e809c190 con 0x7f68ec106630 2024-04-25T00:26:52.737 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:26:52.737 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:26:52.737 INFO:teuthology.orchestra.run.smithi107.stdout: "epoch": 25, 2024-04-25T00:26:52.737 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.737 INFO:teuthology.orchestra.run.smithi107.stdout: "active_gid": 15456, 2024-04-25T00:26:52.737 INFO:teuthology.orchestra.run.smithi107.stdout: "active_name": "x", 2024-04-25T00:26:52.737 INFO:teuthology.orchestra.run.smithi107.stdout: "active_addrs": { 2024-04-25T00:26:52.737 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-04-25T00:26:52.737 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:52.738 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-04-25T00:26:52.738 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6800", 2024-04-25T00:26:52.738 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1021104172 2024-04-25T00:26:52.738 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.738 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:52.738 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-04-25T00:26:52.738 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6801", 2024-04-25T00:26:52.738 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1021104172 2024-04-25T00:26:52.738 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.738 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-04-25T00:26:52.738 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.738 INFO:teuthology.orchestra.run.smithi107.stdout: "active_addr": "172.21.15.107:6801/1021104172", 2024-04-25T00:26:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: "active_change": "2024-04-25T00:16:26.229707+0000", 2024-04-25T00:26:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T00:26:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: "available": true, 2024-04-25T00:26:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: "standbys": [ 2024-04-25T00:26:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: "gid": 25009, 2024-04-25T00:26:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "z", 2024-04-25T00:26:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_features": 4540701547738038271, 2024-04-25T00:26:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: "available_modules": [ 2024-04-25T00:26:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "alerts", 2024-04-25T00:26:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-04-25T00:26:52.740 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-04-25T00:26:52.740 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:52.740 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.740 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.740 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-04-25T00:26:52.740 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.740 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.740 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.740 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T00:26:52.740 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.740 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.740 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.740 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.740 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:52.740 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.742 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.742 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.742 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.742 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.742 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.742 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:52.742 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:52.742 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.742 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.742 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.742 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.742 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.742 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_destination": { 2024-04-25T00:26:52.746 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_destination", 2024-04-25T00:26:52.746 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.746 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.746 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.746 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.746 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.746 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.746 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.746 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email address to send alerts to", 2024-04-25T00:26:52.746 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.746 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.746 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.746 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.747 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_from_name": { 2024-04-25T00:26:52.747 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_from_name", 2024-04-25T00:26:52.747 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.747 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.747 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.747 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "Ceph", 2024-04-25T00:26:52.747 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.747 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.747 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.747 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email From: name", 2024-04-25T00:26:52.747 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.747 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.747 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.747 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_host": { 2024-04-25T00:26:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_host", 2024-04-25T00:26:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP server", 2024-04-25T00:26:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_password": { 2024-04-25T00:26:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_password", 2024-04-25T00:26:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Password to authenticate with", 2024-04-25T00:26:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.750 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_port": { 2024-04-25T00:26:52.750 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_port", 2024-04-25T00:26:52.750 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.750 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.750 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.750 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "465", 2024-04-25T00:26:52.750 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.750 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.750 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.750 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP port", 2024-04-25T00:26:52.750 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.750 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.750 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.750 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.751 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_sender": { 2024-04-25T00:26:52.751 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_sender", 2024-04-25T00:26:52.751 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.751 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.751 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.751 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.751 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.751 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.751 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.751 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP envelope sender", 2024-04-25T00:26:52.751 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.751 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.751 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_ssl": { 2024-04-25T00:26:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_ssl", 2024-04-25T00:26:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T00:26:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_user": { 2024-04-25T00:26:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_user", 2024-04-25T00:26:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "User to authenticate as", 2024-04-25T00:26:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "balancer", 2024-04-25T00:26:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: "active": { 2024-04-25T00:26:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "active", 2024-04-25T00:26:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.755 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.755 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.755 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.755 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T00:26:52.755 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.755 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.755 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.755 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.755 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_time": { 2024-04-25T00:26:52.755 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_time", 2024-04-25T00:26:52.755 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.755 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.756 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.756 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0000", 2024-04-25T00:26:52.756 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.756 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.756 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.756 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T00:26:52.756 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:26:52.756 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.756 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.756 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.756 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_weekday": { 2024-04-25T00:26:52.756 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_weekday", 2024-04-25T00:26:52.756 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-04-25T00:26:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-04-25T00:26:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-04-25T00:26:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-04-25T00:26:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T00:26:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:26:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_max_iterations": { 2024-04-25T00:26:52.758 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_max_iterations", 2024-04-25T00:26:52.758 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-04-25T00:26:52.758 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.758 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.758 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "25", 2024-04-25T00:26:52.758 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-04-25T00:26:52.758 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "250", 2024-04-25T00:26:52.758 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.758 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T00:26:52.758 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.758 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.758 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_metrics": { 2024-04-25T00:26:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_metrics", 2024-04-25T00:26:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T00:26:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T00:26:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T00:26:52.760 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.760 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.760 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.760 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_step": { 2024-04-25T00:26:52.760 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_step", 2024-04-25T00:26:52.760 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-04-25T00:26:52.760 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.760 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.760 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.5", 2024-04-25T00:26:52.760 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0.001", 2024-04-25T00:26:52.760 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "0.999", 2024-04-25T00:26:52.760 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.761 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "aggressiveness of optimization", 2024-04-25T00:26:52.761 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T00:26:52.761 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.761 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.761 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.761 INFO:teuthology.orchestra.run.smithi107.stdout: "end_time": { 2024-04-25T00:26:52.761 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_time", 2024-04-25T00:26:52.761 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.761 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.761 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.761 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2359", 2024-04-25T00:26:52.761 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.761 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.762 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.762 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T00:26:52.762 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:26:52.762 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.762 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.762 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.762 INFO:teuthology.orchestra.run.smithi107.stdout: "end_weekday": { 2024-04-25T00:26:52.762 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_weekday", 2024-04-25T00:26:52.762 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-04-25T00:26:52.762 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.762 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.762 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-04-25T00:26:52.763 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-04-25T00:26:52.763 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-04-25T00:26:52.763 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.763 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T00:26:52.763 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:26:52.763 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.763 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.763 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.763 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:52.763 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:52.763 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.763 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.763 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.763 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.765 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.765 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:52.765 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:52.765 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.765 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.765 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.765 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.765 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.765 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.765 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.765 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.765 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.765 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.766 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.766 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.766 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:52.766 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:52.766 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.766 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.766 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.766 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:52.766 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.766 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.766 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.766 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.766 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.766 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.767 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.767 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.767 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.767 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.767 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.767 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.767 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.767 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.767 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.767 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:52.767 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:52.767 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.767 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: "min_score": { 2024-04-25T00:26:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_score", 2024-04-25T00:26:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-04-25T00:26:52.769 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.769 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.769 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-04-25T00:26:52.769 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.769 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.769 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.769 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T00:26:52.769 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.769 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.769 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.769 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.769 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-04-25T00:26:52.769 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-04-25T00:26:52.769 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.769 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.770 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.770 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "upmap", 2024-04-25T00:26:52.770 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.770 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.770 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.770 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-compat", 2024-04-25T00:26:52.770 INFO:teuthology.orchestra.run.smithi107.stdout: "none", 2024-04-25T00:26:52.770 INFO:teuthology.orchestra.run.smithi107.stdout: "read", 2024-04-25T00:26:52.770 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap", 2024-04-25T00:26:52.770 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap-read" 2024-04-25T00:26:52.770 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.770 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Balancer mode", 2024-04-25T00:26:52.770 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.771 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.771 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.771 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.771 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_ids": { 2024-04-25T00:26:52.771 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_ids", 2024-04-25T00:26:52.771 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.771 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.771 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.771 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.771 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.771 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.771 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.771 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T00:26:52.771 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.772 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.772 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.772 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.772 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-04-25T00:26:52.772 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-04-25T00:26:52.772 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:52.772 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.772 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.772 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-04-25T00:26:52.772 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.772 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.772 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.772 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T00:26:52.772 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.772 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_deviation": { 2024-04-25T00:26:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_deviation", 2024-04-25T00:26:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-04-25T00:26:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-04-25T00:26:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T00:26:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T00:26:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_optimizations": { 2024-04-25T00:26:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_optimizations", 2024-04-25T00:26:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-04-25T00:26:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-04-25T00:26:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T00:26:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.775 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.775 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.775 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.775 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.775 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.775 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:52.775 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm", 2024-04-25T00:26:52.775 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:52.775 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:52.775 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:52.775 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_down_multiplier": { 2024-04-25T00:26:52.775 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_down_multiplier", 2024-04-25T00:26:52.775 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-04-25T00:26:52.775 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.776 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.776 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-04-25T00:26:52.776 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.776 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.776 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.776 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T00:26:52.776 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.776 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.776 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.776 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.777 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_refresh_rate": { 2024-04-25T00:26:52.777 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_refresh_rate", 2024-04-25T00:26:52.777 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:52.777 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.777 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.777 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "20", 2024-04-25T00:26:52.777 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.777 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.777 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.777 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T00:26:52.777 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.777 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.777 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.778 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.778 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_starting_port": { 2024-04-25T00:26:52.778 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_starting_port", 2024-04-25T00:26:52.778 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.778 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.778 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.778 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4721", 2024-04-25T00:26:52.778 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.778 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.778 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.778 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T00:26:52.778 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.778 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.779 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.779 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.779 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_ptrace": { 2024-04-25T00:26:52.779 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_ptrace", 2024-04-25T00:26:52.779 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.779 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.779 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.779 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.779 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.779 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.779 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.779 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T00:26:52.779 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T00:26:52.779 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_interval": { 2024-04-25T00:26:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_interval", 2024-04-25T00:26:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-04-25T00:26:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T00:26:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.781 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.781 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_memory_target_ratio": { 2024-04-25T00:26:52.781 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T00:26:52.781 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-04-25T00:26:52.781 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.781 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.781 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.7", 2024-04-25T00:26:52.781 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.781 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.781 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.781 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T00:26:52.781 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.781 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.781 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm_log_destination": { 2024-04-25T00:26:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm_log_destination", 2024-04-25T00:26:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: "file", 2024-04-25T00:26:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: "file,syslog", 2024-04-25T00:26:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: "syslog" 2024-04-25T00:26:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T00:26:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: "cgroups_split": { 2024-04-25T00:26:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cgroups_split", 2024-04-25T00:26:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T00:26:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: "config_checks_enabled": { 2024-04-25T00:26:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_checks_enabled", 2024-04-25T00:26:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T00:26:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.785 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.785 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.785 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.785 INFO:teuthology.orchestra.run.smithi107.stdout: "config_dashboard": { 2024-04-25T00:26:52.785 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_dashboard", 2024-04-25T00:26:52.785 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.785 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.785 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.785 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.785 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.785 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.785 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.785 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T00:26:52.785 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.786 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.786 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.786 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.786 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_alertmanager": { 2024-04-25T00:26:52.786 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_alertmanager", 2024-04-25T00:26:52.786 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.786 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.786 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.786 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T00:26:52.786 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.786 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.786 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.786 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-04-25T00:26:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_base": { 2024-04-25T00:26:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_base", 2024-04-25T00:26:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T00:26:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Container image name, without the tag", 2024-04-25T00:26:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_elasticsearch": { 2024-04-25T00:26:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_elasticsearch", 2024-04-25T00:26:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T00:26:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "elasticsearch container image", 2024-04-25T00:26:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_grafana": { 2024-04-25T00:26:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_grafana", 2024-04-25T00:26:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T00:26:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-04-25T00:26:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.790 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_haproxy": { 2024-04-25T00:26:52.790 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_haproxy", 2024-04-25T00:26:52.790 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.790 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.790 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.790 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T00:26:52.790 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.790 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.790 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.790 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "HAproxy container image", 2024-04-25T00:26:52.790 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.790 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.790 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_agent": { 2024-04-25T00:26:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_agent", 2024-04-25T00:26:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T00:26:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger agent container image", 2024-04-25T00:26:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.792 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_collector": { 2024-04-25T00:26:52.792 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_collector", 2024-04-25T00:26:52.792 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.792 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.792 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.792 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T00:26:52.792 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.792 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.792 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.792 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger collector container image", 2024-04-25T00:26:52.792 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_query": { 2024-04-25T00:26:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_query", 2024-04-25T00:26:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T00:26:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger query container image", 2024-04-25T00:26:52.794 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.794 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.794 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.794 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.794 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_keepalived": { 2024-04-25T00:26:52.794 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_keepalived", 2024-04-25T00:26:52.794 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.794 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.794 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.794 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T00:26:52.794 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.794 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.794 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.795 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Keepalived container image", 2024-04-25T00:26:52.795 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.795 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.795 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.795 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.795 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_loki": { 2024-04-25T00:26:52.795 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_loki", 2024-04-25T00:26:52.795 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.795 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.795 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.795 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T00:26:52.795 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.795 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.795 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Loki container image", 2024-04-25T00:26:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_node_exporter": { 2024-04-25T00:26:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_node_exporter", 2024-04-25T00:26:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T00:26:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.797 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-04-25T00:26:52.797 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.797 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.797 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.797 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_nvmeof": { 2024-04-25T00:26:52.797 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_nvmeof", 2024-04-25T00:26:52.797 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.797 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.797 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.797 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T00:26:52.797 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.797 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.797 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.797 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Nvme-of container image", 2024-04-25T00:26:52.797 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_prometheus": { 2024-04-25T00:26:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_prometheus", 2024-04-25T00:26:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T00:26:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-04-25T00:26:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_promtail": { 2024-04-25T00:26:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_promtail", 2024-04-25T00:26:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T00:26:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Promtail container image", 2024-04-25T00:26:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.800 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_samba": { 2024-04-25T00:26:52.800 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_samba", 2024-04-25T00:26:52.800 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.800 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.800 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.800 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T00:26:52.800 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.800 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.800 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.800 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Samba/SMB container image", 2024-04-25T00:26:52.800 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.800 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.800 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.800 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_snmp_gateway": { 2024-04-25T00:26:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_snmp_gateway", 2024-04-25T00:26:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T00:26:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SNMP Gateway container image", 2024-04-25T00:26:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: "container_init": { 2024-04-25T00:26:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_init", 2024-04-25T00:26:52.802 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.802 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.802 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.802 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.802 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.802 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.802 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.802 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T00:26:52.802 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.802 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.802 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.803 INFO:teuthology.orchestra.run.smithi107.stdout: "daemon_cache_timeout": { 2024-04-25T00:26:52.803 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "daemon_cache_timeout", 2024-04-25T00:26:52.803 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:52.803 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.803 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.803 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-04-25T00:26:52.803 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.803 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.803 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.803 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T00:26:52.803 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.803 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.803 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.804 INFO:teuthology.orchestra.run.smithi107.stdout: "default_cephadm_command_timeout": { 2024-04-25T00:26:52.804 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T00:26:52.804 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.804 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.804 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.804 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "900", 2024-04-25T00:26:52.804 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.804 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.804 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.804 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T00:26:52.804 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.804 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.804 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: "default_registry": { 2024-04-25T00:26:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_registry", 2024-04-25T00:26:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io", 2024-04-25T00:26:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T00:26:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: "device_cache_timeout": { 2024-04-25T00:26:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_cache_timeout", 2024-04-25T00:26:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1800", 2024-04-25T00:26:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache device inventory", 2024-04-25T00:26:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: "device_enhanced_scan": { 2024-04-25T00:26:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_enhanced_scan", 2024-04-25T00:26:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T00:26:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: "facts_cache_timeout": { 2024-04-25T00:26:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "facts_cache_timeout", 2024-04-25T00:26:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-04-25T00:26:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache host facts data", 2024-04-25T00:26:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: "grafana_dashboards_path": { 2024-04-25T00:26:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "grafana_dashboards_path", 2024-04-25T00:26:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T00:26:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T00:26:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "host_check_interval": { 2024-04-25T00:26:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "host_check_interval", 2024-04-25T00:26:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:52.810 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.810 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.810 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-04-25T00:26:52.810 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.810 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.810 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.810 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to perform a host check", 2024-04-25T00:26:52.810 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.810 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.810 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.810 INFO:teuthology.orchestra.run.smithi107.stdout: "hw_monitoring": { 2024-04-25T00:26:52.810 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hw_monitoring", 2024-04-25T00:26:52.810 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.810 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.811 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.811 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.811 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.811 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.811 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.811 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T00:26:52.811 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.811 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.811 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.811 INFO:teuthology.orchestra.run.smithi107.stdout: "inventory_list_all": { 2024-04-25T00:26:52.811 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "inventory_list_all", 2024-04-25T00:26:52.811 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.811 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.811 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T00:26:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.813 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.813 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.813 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.813 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.813 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.813 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.813 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.813 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.813 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.813 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.813 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.813 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.813 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.813 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.814 INFO:teuthology.orchestra.run.smithi107.stdout: "log_refresh_metadata": { 2024-04-25T00:26:52.814 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_refresh_metadata", 2024-04-25T00:26:52.814 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.814 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.814 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.814 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.814 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.814 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.814 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.814 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T00:26:52.814 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.814 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.815 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.815 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:52.815 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:52.815 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.815 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.815 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.815 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.815 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.815 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.815 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.815 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T00:26:52.815 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.815 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.815 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.818 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.818 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.818 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.818 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.818 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.818 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.818 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.818 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T00:26:52.818 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T00:26:52.818 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.818 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.818 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.818 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T00:26:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T00:26:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T00:26:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "*", 2024-04-25T00:26:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T00:26:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: "max_count_per_host": { 2024-04-25T00:26:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_count_per_host", 2024-04-25T00:26:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-04-25T00:26:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of daemons per service per host", 2024-04-25T00:26:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd_draining_count": { 2024-04-25T00:26:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_osd_draining_count", 2024-04-25T00:26:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-04-25T00:26:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T00:26:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: "migration_current": { 2024-04-25T00:26:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "migration_current", 2024-04-25T00:26:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "internal - do not modify", 2024-04-25T00:26:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-04-25T00:26:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-04-25T00:26:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "root", 2024-04-25T00:26:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm-package", 2024-04-25T00:26:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: "root" 2024-04-25T00:26:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T00:26:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: "oob_default_addr": { 2024-04-25T00:26:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "oob_default_addr", 2024-04-25T00:26:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "169.254.1.1", 2024-04-25T00:26:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T00:26:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "prometheus_alerts_path": { 2024-04-25T00:26:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus_alerts_path", 2024-04-25T00:26:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.826 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T00:26:52.826 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.826 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.826 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.826 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T00:26:52.826 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.826 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.826 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.826 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.826 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_insecure": { 2024-04-25T00:26:52.826 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_insecure", 2024-04-25T00:26:52.826 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.826 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T00:26:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_password": { 2024-04-25T00:26:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_password", 2024-04-25T00:26:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T00:26:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_url": { 2024-04-25T00:26:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_url", 2024-04-25T00:26:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T00:26:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_username": { 2024-04-25T00:26:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_username", 2024-04-25T00:26:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T00:26:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: "secure_monitoring_stack": { 2024-04-25T00:26:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "secure_monitoring_stack", 2024-04-25T00:26:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T00:26:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: "service_discovery_port": { 2024-04-25T00:26:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "service_discovery_port", 2024-04-25T00:26:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8765", 2024-04-25T00:26:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "cephadm service discovery port", 2024-04-25T00:26:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: "ssh_config_file": { 2024-04-25T00:26:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssh_config_file", 2024-04-25T00:26:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T00:26:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "use_agent": { 2024-04-25T00:26:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_agent", 2024-04-25T00:26:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.834 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.834 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.834 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T00:26:52.834 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.834 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.834 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.834 INFO:teuthology.orchestra.run.smithi107.stdout: "use_repo_digest": { 2024-04-25T00:26:52.834 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_repo_digest", 2024-04-25T00:26:52.834 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.834 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.834 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.834 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T00:26:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_failed_host_check": { 2024-04-25T00:26:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_failed_host_check", 2024-04-25T00:26:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.836 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.836 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.836 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.836 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T00:26:52.836 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.836 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.836 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.836 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_daemons": { 2024-04-25T00:26:52.836 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_daemons", 2024-04-25T00:26:52.836 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.836 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.836 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.836 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.836 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.837 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.837 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.837 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T00:26:52.837 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.837 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.837 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.837 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_hosts": { 2024-04-25T00:26:52.837 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_hosts", 2024-04-25T00:26:52.837 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.837 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.837 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.837 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.837 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.838 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.838 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.838 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T00:26:52.838 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.838 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.838 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.838 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.838 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.838 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:52.838 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crash", 2024-04-25T00:26:52.838 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:52.838 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.840 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.840 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.840 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.840 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.840 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.840 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.840 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.840 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:52.840 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:52.840 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.840 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.840 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.840 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.840 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.842 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.842 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.842 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.842 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.842 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.842 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.842 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.842 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.842 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.842 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.842 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.842 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.842 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:52.843 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:52.843 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.843 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.843 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.843 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.843 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.843 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.843 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.843 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.843 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.843 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.843 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.843 INFO:teuthology.orchestra.run.smithi107.stdout: "retain_interval": { 2024-04-25T00:26:52.844 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retain_interval", 2024-04-25T00:26:52.844 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:52.844 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.844 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.844 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "31536000", 2024-04-25T00:26:52.844 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.844 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.844 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.844 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T00:26:52.844 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.844 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.844 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.844 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_recent_interval": { 2024-04-25T00:26:52.844 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_recent_interval", 2024-04-25T00:26:52.845 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:52.845 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.845 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.845 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1209600", 2024-04-25T00:26:52.845 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.845 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.845 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.845 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T00:26:52.845 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.845 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.845 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.845 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.845 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dashboard", 2024-04-25T00:26:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T00:26:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T00:26:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-04-25T00:26:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T00:26:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T00:26:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T00:26:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T00:26:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_ENABLED": { 2024-04-25T00:26:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T00:26:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.850 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T00:26:52.850 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T00:26:52.850 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.850 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.850 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.850 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.850 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.850 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.850 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.850 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.850 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.850 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.850 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T00:26:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T00:26:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T00:26:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T00:26:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T00:26:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T00:26:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.854 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.854 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T00:26:52.854 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T00:26:52.854 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.854 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.854 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.854 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.854 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.854 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.854 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.854 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.854 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.854 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.855 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.855 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T00:26:52.855 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T00:26:52.855 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.855 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.855 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.855 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.855 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.855 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.855 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.855 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.855 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.855 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.856 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.856 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T00:26:52.856 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T00:26:52.856 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.856 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.856 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.856 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.856 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.856 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.856 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.856 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.856 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.856 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.857 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.857 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T00:26:52.857 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T00:26:52.857 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.857 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.857 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.857 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.857 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.857 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.857 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.857 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.857 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.857 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T00:26:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T00:26:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T00:26:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T00:26:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T00:26:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T00:26:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-04-25T00:26:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T00:26:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T00:26:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_URL": { 2024-04-25T00:26:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_URL", 2024-04-25T00:26:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T00:26:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T00:26:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-04-25T00:26:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T00:26:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T00:26:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T00:26:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T00:26:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T00:26:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T00:26:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.867 INFO:teuthology.orchestra.run.smithi107.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T00:26:52.867 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T00:26:52.867 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.867 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.867 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.867 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.867 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.867 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.867 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.867 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.867 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.867 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.867 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.867 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T00:26:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T00:26:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T00:26:52.869 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T00:26:52.869 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.869 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.869 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.869 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.869 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.869 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.869 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.869 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.869 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.869 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.869 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.869 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T00:26:52.869 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T00:26:52.870 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.870 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.870 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.870 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.870 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.870 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.870 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.870 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.870 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.870 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.870 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.870 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T00:26:52.871 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T00:26:52.871 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.871 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.871 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.871 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.871 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.871 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.871 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.871 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.871 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.871 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.871 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.871 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T00:26:52.872 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T00:26:52.872 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.872 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.872 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.872 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.872 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.872 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.872 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.872 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.872 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.872 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.872 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.872 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T00:26:52.873 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T00:26:52.873 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.873 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.873 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.873 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.873 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.873 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.873 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.873 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.873 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.873 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.873 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.873 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T00:26:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T00:26:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T00:26:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T00:26:52.875 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.875 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.875 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.875 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.875 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.875 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.875 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.875 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.875 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.875 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.875 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.875 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T00:26:52.875 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T00:26:52.876 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.876 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.876 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.876 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.876 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.876 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.876 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.876 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.876 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.876 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.876 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.876 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T00:26:52.876 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T00:26:52.876 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.877 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.877 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.877 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.877 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.877 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.877 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.877 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.877 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.877 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.877 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.877 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T00:26:52.878 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T00:26:52.878 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.878 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.878 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.878 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T00:26:52.878 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.878 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.878 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.878 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.878 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.878 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.878 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.878 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.878 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T00:26:52.878 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T00:26:52.879 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.879 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.879 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.879 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-04-25T00:26:52.879 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.879 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.879 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.879 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.879 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.879 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.879 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.879 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T00:26:52.879 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T00:26:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8", 2024-04-25T00:26:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T00:26:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T00:26:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "45", 2024-04-25T00:26:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T00:26:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T00:26:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T00:26:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T00:26:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.883 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-04-25T00:26:52.883 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.883 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.883 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.883 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.883 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.883 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.883 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.883 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T00:26:52.883 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T00:26:52.883 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.883 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.883 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T00:26:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T00:26:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T00:26:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T00:26:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-04-25T00:26:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T00:26:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T00:26:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-04-25T00:26:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T00:26:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T00:26:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-04-25T00:26:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "cross_origin_url": { 2024-04-25T00:26:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cross_origin_url", 2024-04-25T00:26:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: "crt_file": { 2024-04-25T00:26:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crt_file", 2024-04-25T00:26:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.890 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.890 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.890 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.890 INFO:teuthology.orchestra.run.smithi107.stdout: "debug": { 2024-04-25T00:26:52.890 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "debug", 2024-04-25T00:26:52.890 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.890 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.890 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.890 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.890 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.890 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.890 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.891 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable/disable debug options", 2024-04-25T00:26:52.891 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.891 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.891 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.891 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.891 INFO:teuthology.orchestra.run.smithi107.stdout: "jwt_token_ttl": { 2024-04-25T00:26:52.891 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "jwt_token_ttl", 2024-04-25T00:26:52.891 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.891 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.891 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.891 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "28800", 2024-04-25T00:26:52.891 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.891 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.891 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.892 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.892 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.892 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.892 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.892 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-04-25T00:26:52.892 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-04-25T00:26:52.892 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.892 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.892 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.892 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.892 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.892 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.892 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: "motd": { 2024-04-25T00:26:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "motd", 2024-04-25T00:26:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "The message of the day", 2024-04-25T00:26:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect_resolve_ip_addr": { 2024-04-25T00:26:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T00:26:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-04-25T00:26:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-04-25T00:26:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-04-25T00:26:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-04-25T00:26:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-04-25T00:26:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8080", 2024-04-25T00:26:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-04-25T00:26:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-04-25T00:26:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl_server_port": { 2024-04-25T00:26:52.903 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl_server_port", 2024-04-25T00:26:52.903 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.903 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.903 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.903 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8443", 2024-04-25T00:26:52.903 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.903 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.903 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.903 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.903 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.903 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.903 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-04-25T00:26:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-04-25T00:26:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "redirect", 2024-04-25T00:26:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect" 2024-04-25T00:26:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.905 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.905 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.905 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.905 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-04-25T00:26:52.905 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-04-25T00:26:52.905 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.905 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.905 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.905 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-04-25T00:26:52.905 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-04-25T00:26:52.905 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-04-25T00:26:52.905 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.905 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.905 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: "url_prefix": { 2024-04-25T00:26:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url_prefix", 2024-04-25T00:26:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.907 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.907 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.907 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.907 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.907 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:52.907 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "devicehealth", 2024-04-25T00:26:52.907 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:52.907 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:52.907 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:52.907 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_monitoring": { 2024-04-25T00:26:52.907 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_monitoring", 2024-04-25T00:26:52.907 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.907 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.907 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "monitor device health metrics", 2024-04-25T00:26:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.909 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.909 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.909 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.909 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.909 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.909 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.909 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.909 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.909 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.909 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.909 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.909 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.909 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.909 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.909 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.910 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.910 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.910 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:52.910 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:52.910 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.910 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.910 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.910 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.910 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.910 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.910 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.910 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.910 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.910 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.911 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.911 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:52.911 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:52.911 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.911 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.911 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.911 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:52.911 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.911 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.911 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.911 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.911 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.911 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.911 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.912 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.912 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.912 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.912 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.912 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.912 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.912 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.912 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:52.912 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:52.912 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.912 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.912 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.912 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: "mark_out_threshold": { 2024-04-25T00:26:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mark_out_threshold", 2024-04-25T00:26:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2419200", 2024-04-25T00:26:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.914 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.914 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.914 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T00:26:52.914 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.914 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.914 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.914 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": { 2024-04-25T00:26:52.914 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_name", 2024-04-25T00:26:52.914 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.914 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.914 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.914 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "device_health_metrics", 2024-04-25T00:26:52.914 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.915 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.915 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.915 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T00:26:52.915 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.915 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.915 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.915 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.915 INFO:teuthology.orchestra.run.smithi107.stdout: "retention_period": { 2024-04-25T00:26:52.915 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retention_period", 2024-04-25T00:26:52.915 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:52.915 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.915 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.915 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15552000", 2024-04-25T00:26:52.915 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.915 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.916 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.916 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain device health metrics", 2024-04-25T00:26:52.916 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.916 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.916 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.916 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_frequency": { 2024-04-25T00:26:52.916 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_frequency", 2024-04-25T00:26:52.916 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:52.916 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.916 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.916 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-04-25T00:26:52.916 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.917 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.917 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.917 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T00:26:52.917 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.917 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.917 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.917 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.917 INFO:teuthology.orchestra.run.smithi107.stdout: "self_heal": { 2024-04-25T00:26:52.917 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "self_heal", 2024-04-25T00:26:52.917 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.917 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.917 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.917 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:52.917 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.918 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.918 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.918 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T00:26:52.918 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.918 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.918 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.918 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.918 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-04-25T00:26:52.918 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-04-25T00:26:52.918 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:52.918 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.918 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.918 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-04-25T00:26:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T00:26:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_threshold": { 2024-04-25T00:26:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_threshold", 2024-04-25T00:26:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7257600", 2024-04-25T00:26:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T00:26:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "diskprediction_local", 2024-04-25T00:26:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.922 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.922 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.922 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.922 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.922 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.922 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.922 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:52.922 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:52.922 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.922 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.922 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.922 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.922 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: "predict_interval": { 2024-04-25T00:26:52.926 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predict_interval", 2024-04-25T00:26:52.926 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.926 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.926 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.926 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-04-25T00:26:52.926 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.926 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.926 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.926 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.926 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.926 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.926 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.926 INFO:teuthology.orchestra.run.smithi107.stdout: "predictor_model": { 2024-04-25T00:26:52.927 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predictor_model", 2024-04-25T00:26:52.927 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.927 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.927 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.927 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "prophetstor", 2024-04-25T00:26:52.927 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.927 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.927 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.927 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.927 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.927 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.927 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-04-25T00:26:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-04-25T00:26:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-04-25T00:26:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "influx", 2024-04-25T00:26:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": false, 2024-04-25T00:26:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "influxdb python module not found", 2024-04-25T00:26:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: "batch_size": { 2024-04-25T00:26:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "batch_size", 2024-04-25T00:26:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5000", 2024-04-25T00:26:52.930 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.930 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.930 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.930 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T00:26:52.930 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.930 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.930 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.930 INFO:teuthology.orchestra.run.smithi107.stdout: "database": { 2024-04-25T00:26:52.930 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "database", 2024-04-25T00:26:52.930 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.930 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.930 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "ceph", 2024-04-25T00:26:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T00:26:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": { 2024-04-25T00:26:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hostname", 2024-04-25T00:26:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.932 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.932 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.932 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.932 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server hostname", 2024-04-25T00:26:52.932 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.932 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.932 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.932 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-04-25T00:26:52.932 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-04-25T00:26:52.932 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:52.932 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.932 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.932 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "30", 2024-04-25T00:26:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "5", 2024-04-25T00:26:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T00:26:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.934 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.934 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.934 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.934 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.934 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.934 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.934 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.934 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.934 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.934 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.934 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.934 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.934 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.934 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.935 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:52.935 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:52.935 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.935 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.935 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.935 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.935 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.935 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.935 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.935 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.935 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.935 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.935 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.937 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.937 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.937 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.937 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.937 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:52.937 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:52.937 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.937 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.937 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.937 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.937 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.937 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.937 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.937 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.938 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.938 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.938 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.938 INFO:teuthology.orchestra.run.smithi107.stdout: "password": { 2024-04-25T00:26:52.938 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "password", 2024-04-25T00:26:52.938 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.938 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.938 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.938 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.938 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.938 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.938 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.938 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "password of InfluxDB server user", 2024-04-25T00:26:52.939 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.939 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.939 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.939 INFO:teuthology.orchestra.run.smithi107.stdout: "port": { 2024-04-25T00:26:52.939 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "port", 2024-04-25T00:26:52.939 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.939 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.939 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.939 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8086", 2024-04-25T00:26:52.939 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.939 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.939 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.939 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server port", 2024-04-25T00:26:52.940 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.940 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.940 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.940 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-04-25T00:26:52.940 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-04-25T00:26:52.940 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.940 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.940 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.940 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "false", 2024-04-25T00:26:52.940 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.940 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.940 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.940 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:26:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: "threads": { 2024-04-25T00:26:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threads", 2024-04-25T00:26:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-04-25T00:26:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-04-25T00:26:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "32", 2024-04-25T00:26:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T00:26:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.942 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.942 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.942 INFO:teuthology.orchestra.run.smithi107.stdout: "username": { 2024-04-25T00:26:52.942 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "username", 2024-04-25T00:26:52.942 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.942 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.942 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.942 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.942 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.942 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.942 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.942 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "username of InfluxDB server user", 2024-04-25T00:26:52.942 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.943 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.943 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.943 INFO:teuthology.orchestra.run.smithi107.stdout: "verify_ssl": { 2024-04-25T00:26:52.943 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "verify_ssl", 2024-04-25T00:26:52.943 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.943 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.943 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.943 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "true", 2024-04-25T00:26:52.943 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.943 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.943 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.943 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:26:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "insights", 2024-04-25T00:26:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.945 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.945 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.945 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.945 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.945 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.945 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.945 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.945 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.945 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.945 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.945 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.945 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.945 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.945 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.945 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.946 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.946 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:52.946 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:52.946 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.946 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.946 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.946 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.946 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.946 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.946 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.946 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.946 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.946 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.947 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.947 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:52.947 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:52.947 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.947 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.947 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.947 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:52.947 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.947 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.947 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.947 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.947 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.947 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.947 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.947 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.948 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.948 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.948 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.948 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.948 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.948 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.948 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:52.948 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:52.948 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.948 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.948 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.948 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "iostat", 2024-04-25T00:26:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "k8sevents", 2024-04-25T00:26:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_event_retention_days": { 2024-04-25T00:26:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ceph_event_retention_days", 2024-04-25T00:26:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7", 2024-04-25T00:26:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T00:26:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: "config_check_secs": { 2024-04-25T00:26:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_check_secs", 2024-04-25T00:26:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-04-25T00:26:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "10", 2024-04-25T00:26:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T00:26:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.962 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.962 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.962 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.962 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:52.962 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "localpool", 2024-04-25T00:26:52.962 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:52.962 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:52.962 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:52.962 INFO:teuthology.orchestra.run.smithi107.stdout: "failure_domain": { 2024-04-25T00:26:52.962 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "failure_domain", 2024-04-25T00:26:52.962 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.965 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.965 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "host", 2024-04-25T00:26:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "failure domain for any created local pool", 2024-04-25T00:26:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T00:26:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": { 2024-04-25T00:26:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_size", 2024-04-25T00:26:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default min_size for any created local pool", 2024-04-25T00:26:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T00:26:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "num_rep": { 2024-04-25T00:26:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "num_rep", 2024-04-25T00:26:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3", 2024-04-25T00:26:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.973 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.973 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.973 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default replica count for any created local pool", 2024-04-25T00:26:52.973 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.973 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.973 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.973 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": { 2024-04-25T00:26:52.973 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_num", 2024-04-25T00:26:52.973 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:52.973 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.973 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "128", 2024-04-25T00:26:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T00:26:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: "prefix": { 2024-04-25T00:26:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prefix", 2024-04-25T00:26:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.975 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.975 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.975 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.975 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.975 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name prefix for any created local pool", 2024-04-25T00:26:52.975 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.975 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.975 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.975 INFO:teuthology.orchestra.run.smithi107.stdout: "subtree": { 2024-04-25T00:26:52.975 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "subtree", 2024-04-25T00:26:52.975 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.975 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.975 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "rack", 2024-04-25T00:26:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T00:26:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T00:26:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mds_autoscaler", 2024-04-25T00:26:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:52.977 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:52.977 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:52.977 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:52.977 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.977 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.977 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.977 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.977 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.977 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.977 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.977 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.977 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.977 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.977 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.979 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.979 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.979 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.979 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.979 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.979 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.979 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.979 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:52.979 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:52.979 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.979 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.979 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.979 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:52.980 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.980 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.980 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.980 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.980 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.980 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.980 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.980 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.980 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.980 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.980 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.980 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.980 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.980 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.981 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:52.981 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:52.981 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.981 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.981 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.981 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.981 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.981 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.981 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.981 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.981 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.981 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.981 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.981 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.982 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.982 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:52.982 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirroring", 2024-04-25T00:26:52.982 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:52.982 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:52.982 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:52.982 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:52.982 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:52.982 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.982 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.982 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.983 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.983 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.983 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.983 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.983 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.983 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.983 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.983 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.983 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.983 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.983 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.983 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.983 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.983 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.984 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.984 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:52.984 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:52.984 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.984 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.984 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.984 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.984 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.984 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.984 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.984 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.984 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.984 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.984 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.986 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.986 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.986 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.986 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.986 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.986 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:52.986 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:52.986 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.986 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.986 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.986 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "nfs", 2024-04-25T00:26:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:52.988 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:52.988 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:52.988 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.988 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.988 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.988 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.988 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.988 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.988 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.988 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.988 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.988 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.988 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-04-25T00:26:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: "fail_fs": { 2024-04-25T00:26:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "fail_fs", 2024-04-25T00:26:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.994 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.994 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.994 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T00:26:52.994 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.994 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.994 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.994 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:52.994 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:52.994 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.994 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.994 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.994 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.994 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.994 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.995 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.995 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.995 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.995 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.995 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.995 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.995 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.995 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.995 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.995 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.995 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.995 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.995 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator": { 2024-04-25T00:26:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-04-25T00:26:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm", 2024-04-25T00:26:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: "rook", 2024-04-25T00:26:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: "test_orchestrator" 2024-04-25T00:26:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Orchestrator backend", 2024-04-25T00:26:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_perf_query", 2024-04-25T00:26:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.006 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.006 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.006 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.006 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.006 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_support", 2024-04-25T00:26:53.006 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.006 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.006 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.006 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.006 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.006 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.006 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.007 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.007 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.007 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.007 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.007 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.007 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.007 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.007 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.007 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.007 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.007 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.007 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.007 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.007 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.009 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.009 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.009 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.009 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.009 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.009 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.009 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.009 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.009 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.009 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.009 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.009 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.010 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.010 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.010 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.010 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.010 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.010 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.010 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.010 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.010 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.010 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.010 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.010 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.010 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.010 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.010 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.011 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.011 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.011 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.011 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.011 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.011 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.011 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.011 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.011 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.011 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.011 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.011 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.011 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_autoscaler", 2024-04-25T00:26:53.011 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.012 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.012 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.012 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.012 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.012 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.012 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.012 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.012 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.012 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.012 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.012 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.012 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.012 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.012 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.013 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.013 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.013 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.013 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.013 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.013 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.013 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.013 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.013 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.013 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.013 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.013 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.013 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.013 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.013 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.014 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.014 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.014 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.014 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.014 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.014 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.014 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.014 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.014 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.014 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.014 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.014 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.015 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.015 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.015 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.015 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.015 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.015 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.015 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.015 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.015 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.015 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.015 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.015 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.015 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.015 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.016 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.016 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.016 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.016 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.016 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.016 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.016 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.016 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.016 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.016 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.016 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.016 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.016 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.016 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.017 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.017 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-04-25T00:26:53.017 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-04-25T00:26:53.017 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.017 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.017 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.017 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-04-25T00:26:53.017 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.017 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.017 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.017 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.017 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.017 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.017 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.017 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.018 INFO:teuthology.orchestra.run.smithi107.stdout: "threshold": { 2024-04-25T00:26:53.018 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threshold", 2024-04-25T00:26:53.018 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-04-25T00:26:53.018 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.018 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.018 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-04-25T00:26:53.018 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1.0", 2024-04-25T00:26:53.018 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.018 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.018 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "scaling threshold", 2024-04-25T00:26:53.018 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T00:26:53.018 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.018 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.019 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.019 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.019 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.019 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.019 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "progress", 2024-04-25T00:26:53.019 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.019 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.019 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.019 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_pg_recovery_event": { 2024-04-25T00:26:53.019 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_pg_recovery_event", 2024-04-25T00:26:53.019 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.019 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.020 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.020 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.020 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.020 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.020 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.020 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T00:26:53.020 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.020 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.020 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.020 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-04-25T00:26:53.020 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-04-25T00:26:53.020 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.020 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.020 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: "max_completed_events": { 2024-04-25T00:26:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_completed_events", 2024-04-25T00:26:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "50", 2024-04-25T00:26:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "number of past completed events to remember", 2024-04-25T00:26:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-04-25T00:26:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-04-25T00:26:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.028 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-04-25T00:26:53.028 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.028 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.028 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.028 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long the module is going to sleep", 2024-04-25T00:26:53.028 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.028 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.028 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.028 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.028 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.028 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.028 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus", 2024-04-25T00:26:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "cache": { 2024-04-25T00:26:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cache", 2024-04-25T00:26:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.030 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.030 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.030 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.030 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.030 INFO:teuthology.orchestra.run.smithi107.stdout: "exclude_perf_counters": { 2024-04-25T00:26:53.030 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "exclude_perf_counters", 2024-04-25T00:26:53.030 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.030 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.030 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.030 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.030 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.030 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.030 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T00:26:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T00:26:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.034 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.034 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.034 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.034 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.034 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.034 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.034 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.034 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.034 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.034 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.034 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.034 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.034 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.034 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools": { 2024-04-25T00:26:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools", 2024-04-25T00:26:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T00:26:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T00:26:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "300", 2024-04-25T00:26:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_interval": { 2024-04-25T00:26:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_interval", 2024-04-25T00:26:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-04-25T00:26:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15.0", 2024-04-25T00:26:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-04-25T00:26:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-04-25T00:26:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-04-25T00:26:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T00:26:53.040 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.040 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.040 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.040 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.040 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-04-25T00:26:53.040 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-04-25T00:26:53.040 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.040 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.040 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.040 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "9283", 2024-04-25T00:26:53.040 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.040 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.040 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.041 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T00:26:53.041 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.041 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.041 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.041 INFO:teuthology.orchestra.run.smithi107.stdout: "stale_cache_strategy": { 2024-04-25T00:26:53.041 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stale_cache_strategy", 2024-04-25T00:26:53.041 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.041 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.041 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.041 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "log", 2024-04-25T00:26:53.041 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.042 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.042 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.042 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.042 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.042 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.042 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.042 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-04-25T00:26:53.042 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-04-25T00:26:53.042 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.042 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.042 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.042 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "default", 2024-04-25T00:26:53.042 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.042 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.043 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.043 INFO:teuthology.orchestra.run.smithi107.stdout: "default", 2024-04-25T00:26:53.043 INFO:teuthology.orchestra.run.smithi107.stdout: "error" 2024-04-25T00:26:53.043 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.043 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.043 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.043 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.043 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.043 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-04-25T00:26:53.043 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-04-25T00:26:53.043 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.043 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.043 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-04-25T00:26:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-04-25T00:26:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-04-25T00:26:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_support", 2024-04-25T00:26:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.047 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.047 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.047 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.047 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.047 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.047 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.047 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.047 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.047 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.047 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.047 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.047 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.049 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.049 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.049 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.049 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.049 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.049 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.049 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.049 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.049 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.049 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.049 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.049 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.049 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.050 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.050 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.050 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_snap_create": { 2024-04-25T00:26:53.050 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_snap_create", 2024-04-25T00:26:53.050 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.050 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.050 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.050 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-04-25T00:26:53.050 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.050 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.050 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.050 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.050 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.051 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.051 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.051 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.051 INFO:teuthology.orchestra.run.smithi107.stdout: "mirror_snapshot_schedule": { 2024-04-25T00:26:53.051 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T00:26:53.051 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.051 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.051 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.051 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.051 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.051 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.051 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.051 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.051 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.052 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.052 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.052 INFO:teuthology.orchestra.run.smithi107.stdout: "trash_purge_schedule": { 2024-04-25T00:26:53.052 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "trash_purge_schedule", 2024-04-25T00:26:53.052 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.052 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.052 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.052 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.052 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.052 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.052 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.052 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.052 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.053 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.053 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.053 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.053 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.053 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.053 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "restful", 2024-04-25T00:26:53.053 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.053 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.053 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.053 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_auth": { 2024-04-25T00:26:53.053 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_auth", 2024-04-25T00:26:53.053 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.053 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.053 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.054 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.054 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.054 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.054 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.054 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.054 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.054 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.054 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.054 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-04-25T00:26:53.054 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-04-25T00:26:53.054 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.054 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.054 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.054 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.055 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.055 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.055 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.055 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.055 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.055 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.055 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.055 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.055 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-04-25T00:26:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-04-25T00:26:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-04-25T00:26:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-04-25T00:26:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.062 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.062 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.062 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.062 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.062 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.062 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.062 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.062 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rgw", 2024-04-25T00:26:53.062 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.062 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.062 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.062 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.062 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.063 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.063 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.063 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.063 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.063 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.063 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.063 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.063 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.063 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.063 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.063 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.063 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.063 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.063 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.063 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.064 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.064 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.064 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.064 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.064 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.064 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.064 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.064 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.064 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.064 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.064 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.064 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.064 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.064 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.064 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.065 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.065 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.065 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.065 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.065 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.065 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.065 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.065 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.065 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.065 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.065 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.065 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.065 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.065 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.066 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.066 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.066 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.066 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.066 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.066 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.066 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.066 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.066 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.066 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.066 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.066 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.066 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rook", 2024-04-25T00:26:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.069 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.069 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.069 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.069 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.069 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.069 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.069 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.069 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.069 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.069 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.069 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.069 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.070 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.070 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.070 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.070 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.070 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.070 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.070 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.070 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.070 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.070 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.070 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.070 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.070 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: "storage_class": { 2024-04-25T00:26:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "storage_class", 2024-04-25T00:26:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "local", 2024-04-25T00:26:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.074 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.074 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.074 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T00:26:53.074 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.074 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.074 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.074 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.074 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.074 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.074 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "selftest", 2024-04-25T00:26:53.074 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.074 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.077 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.077 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.077 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.077 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.077 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.077 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.077 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.077 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.077 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.077 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.077 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.077 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.077 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.077 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.078 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.078 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.078 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.078 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.078 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.078 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.078 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.078 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.078 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.078 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.078 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.078 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.078 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.078 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: "roption1": { 2024-04-25T00:26:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption1", 2024-04-25T00:26:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.080 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.080 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.080 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.080 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.080 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.080 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.080 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.080 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.080 INFO:teuthology.orchestra.run.smithi107.stdout: "roption2": { 2024-04-25T00:26:53.080 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption2", 2024-04-25T00:26:53.080 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.080 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.080 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.080 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "xyz", 2024-04-25T00:26:53.081 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.081 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.081 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.081 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.081 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.081 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.081 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.081 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption1": { 2024-04-25T00:26:53.081 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption1", 2024-04-25T00:26:53.081 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.081 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.081 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.082 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.082 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.082 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.082 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.082 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.082 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.082 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.082 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.082 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption2": { 2024-04-25T00:26:53.082 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption2", 2024-04-25T00:26:53.082 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.082 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.082 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.083 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.083 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.083 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.083 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.083 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.083 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.083 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.083 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.083 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption3": { 2024-04-25T00:26:53.083 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption3", 2024-04-25T00:26:53.083 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-04-25T00:26:53.083 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.083 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.084 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.084 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.084 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.084 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.084 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.084 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.084 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.084 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.084 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption4": { 2024-04-25T00:26:53.084 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption4", 2024-04-25T00:26:53.084 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.084 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.084 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.084 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption5": { 2024-04-25T00:26:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption5", 2024-04-25T00:26:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.086 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.086 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.086 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.086 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.086 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.086 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.086 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption6": { 2024-04-25T00:26:53.086 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption6", 2024-04-25T00:26:53.086 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.086 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.086 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.086 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.086 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.087 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.087 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.087 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.087 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.087 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.087 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.087 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption7": { 2024-04-25T00:26:53.087 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption7", 2024-04-25T00:26:53.087 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.087 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.087 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.087 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.087 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-04-25T00:26:53.088 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "42", 2024-04-25T00:26:53.088 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.088 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.088 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.088 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.088 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.088 INFO:teuthology.orchestra.run.smithi107.stdout: "testkey": { 2024-04-25T00:26:53.088 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testkey", 2024-04-25T00:26:53.088 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.088 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.088 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.088 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.088 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.088 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.089 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.089 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.089 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.089 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.089 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.089 INFO:teuthology.orchestra.run.smithi107.stdout: "testlkey": { 2024-04-25T00:26:53.089 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testlkey", 2024-04-25T00:26:53.089 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.089 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.089 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.089 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.089 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.089 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.089 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.090 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.090 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.090 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.090 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.090 INFO:teuthology.orchestra.run.smithi107.stdout: "testnewline": { 2024-04-25T00:26:53.090 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testnewline", 2024-04-25T00:26:53.090 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.090 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.090 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.090 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.090 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.090 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.090 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2024-04-25T00:26:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_m_granularity": { 2024-04-25T00:26:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_m_granularity", 2024-04-25T00:26:53.092 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.092 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.092 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.092 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.092 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.092 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.092 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.092 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T00:26:53.092 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.092 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.092 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.092 INFO:teuthology.orchestra.run.smithi107.stdout: "dump_on_update": { 2024-04-25T00:26:53.092 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dump_on_update", 2024-04-25T00:26:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "dump database to debug log on update", 2024-04-25T00:26:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stats", 2024-04-25T00:26:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "status", 2024-04-25T00:26:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.106 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.106 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.106 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.106 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.106 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.106 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.106 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.106 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.106 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.106 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.106 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.106 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.106 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telegraf", 2024-04-25T00:26:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: "address": { 2024-04-25T00:26:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "address", 2024-04-25T00:26:53.110 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.110 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.110 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.110 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T00:26:53.110 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.110 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.110 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.110 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.110 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.110 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.110 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-04-25T00:26:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-04-25T00:26:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15", 2024-04-25T00:26:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.112 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.112 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.112 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.112 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.112 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.112 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.112 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.112 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.112 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.112 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.112 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.112 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.112 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.112 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.114 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.114 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.114 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.114 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.114 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.114 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.114 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.114 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.114 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.114 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.114 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.114 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.114 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.116 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.116 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.116 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.116 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.116 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.116 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.116 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.116 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.116 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.116 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.116 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.116 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.116 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telemetry", 2024-04-25T00:26:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_basic": { 2024-04-25T00:26:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_basic", 2024-04-25T00:26:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T00:26:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_crash": { 2024-04-25T00:26:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_crash", 2024-04-25T00:26:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T00:26:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_device": { 2024-04-25T00:26:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_device", 2024-04-25T00:26:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T00:26:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_ident": { 2024-04-25T00:26:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_ident", 2024-04-25T00:26:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.121 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T00:26:53.121 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.121 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.121 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.121 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_perf": { 2024-04-25T00:26:53.121 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_perf", 2024-04-25T00:26:53.121 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.121 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.121 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.121 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.121 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.121 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.121 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.121 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T00:26:53.122 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.122 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.122 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.122 INFO:teuthology.orchestra.run.smithi107.stdout: "contact": { 2024-04-25T00:26:53.122 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "contact", 2024-04-25T00:26:53.122 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.122 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.122 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.122 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.122 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.122 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.122 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.123 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.123 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.123 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.123 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.123 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.123 INFO:teuthology.orchestra.run.smithi107.stdout: "description": { 2024-04-25T00:26:53.123 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "description", 2024-04-25T00:26:53.123 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.125 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.125 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.125 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.125 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.125 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.125 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.125 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.125 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.125 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.125 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.126 INFO:teuthology.orchestra.run.smithi107.stdout: "device_url": { 2024-04-25T00:26:53.126 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_url", 2024-04-25T00:26:53.126 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.126 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.126 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.126 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T00:26:53.126 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.126 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.126 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.126 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.126 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.126 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.126 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-04-25T00:26:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-04-25T00:26:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.128 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.128 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-04-25T00:26:53.128 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-04-25T00:26:53.128 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.128 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.128 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.128 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "24", 2024-04-25T00:26:53.128 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "8", 2024-04-25T00:26:53.128 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.128 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.128 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.128 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.129 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.129 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.129 INFO:teuthology.orchestra.run.smithi107.stdout: "last_opt_revision": { 2024-04-25T00:26:53.129 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "last_opt_revision", 2024-04-25T00:26:53.129 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.129 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.129 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.129 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1", 2024-04-25T00:26:53.129 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.129 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.129 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.129 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.130 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.130 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.130 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.130 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard": { 2024-04-25T00:26:53.130 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard", 2024-04-25T00:26:53.130 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.130 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.130 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.130 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.130 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.130 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.130 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.130 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard_description": { 2024-04-25T00:26:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard_description", 2024-04-25T00:26:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.134 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.134 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.134 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.134 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.134 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.134 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.134 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.134 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.134 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.134 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.134 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.134 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.137 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.137 INFO:teuthology.orchestra.run.smithi107.stdout: "organization": { 2024-04-25T00:26:53.137 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "organization", 2024-04-25T00:26:53.137 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.137 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.137 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.137 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.137 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.137 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.137 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.137 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.137 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.137 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.138 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.138 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.138 INFO:teuthology.orchestra.run.smithi107.stdout: "proxy": { 2024-04-25T00:26:53.138 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "proxy", 2024-04-25T00:26:53.138 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.138 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.138 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.138 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.138 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.138 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.138 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.138 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.138 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.139 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.139 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.139 INFO:teuthology.orchestra.run.smithi107.stdout: "url": { 2024-04-25T00:26:53.139 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url", 2024-04-25T00:26:53.139 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.139 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.139 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.139 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T00:26:53.139 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.139 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.139 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.139 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "test_orchestrator", 2024-04-25T00:26:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "volumes", 2024-04-25T00:26:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.148 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.148 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.148 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.148 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.148 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.148 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.148 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.148 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.148 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.148 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.148 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.148 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.149 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.149 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.149 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.149 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.149 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.149 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.149 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.149 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.149 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.149 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.149 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.149 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.149 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.149 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.149 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.151 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.151 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.151 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.151 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_clones": { 2024-04-25T00:26:53.151 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_clones", 2024-04-25T00:26:53.151 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.151 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.151 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.151 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4", 2024-04-25T00:26:53.151 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.151 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.151 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.151 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T00:26:53.151 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.152 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.152 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.152 INFO:teuthology.orchestra.run.smithi107.stdout: "periodic_async_work": { 2024-04-25T00:26:53.152 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "periodic_async_work", 2024-04-25T00:26:53.152 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.152 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.152 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.152 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.152 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.152 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.152 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.152 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Periodically check for async work", 2024-04-25T00:26:53.152 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.153 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.153 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.153 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_delay": { 2024-04-25T00:26:53.153 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_delay", 2024-04-25T00:26:53.153 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.153 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.153 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.153 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-04-25T00:26:53.153 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.153 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.153 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.153 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T00:26:53.153 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.153 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_no_wait": { 2024-04-25T00:26:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T00:26:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T00:26:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix", 2024-04-25T00:26:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: "discovery_interval": { 2024-04-25T00:26:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "discovery_interval", 2024-04-25T00:26:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-04-25T00:26:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "100", 2024-04-25T00:26:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.156 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.156 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.156 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.156 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.156 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.156 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.156 INFO:teuthology.orchestra.run.smithi107.stdout: "identifier": { 2024-04-25T00:26:53.156 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "identifier", 2024-04-25T00:26:53.156 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.156 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.156 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.156 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.156 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.157 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.157 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.157 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.157 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.157 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.157 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.157 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.157 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-04-25T00:26:53.157 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-04-25T00:26:53.157 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.157 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.157 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.157 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-04-25T00:26:53.157 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.158 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.158 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.158 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.158 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.158 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.158 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.158 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.158 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.158 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.158 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.158 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.158 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.158 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.158 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.159 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.159 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.159 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.159 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.159 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.159 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.159 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.159 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.159 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.159 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.159 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.159 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.160 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.160 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.160 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.160 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.160 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.160 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.160 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.160 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.160 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.160 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.160 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.160 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.160 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.161 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.161 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.161 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.161 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.161 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.161 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.161 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.161 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.161 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.161 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.161 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.161 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.161 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.161 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.161 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_host": { 2024-04-25T00:26:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_host", 2024-04-25T00:26:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_port": { 2024-04-25T00:26:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_port", 2024-04-25T00:26:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10051", 2024-04-25T00:26:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_sender": { 2024-04-25T00:26:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_sender", 2024-04-25T00:26:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T00:26:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-04-25T00:26:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: "gid": 25082, 2024-04-25T00:26:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "y", 2024-04-25T00:26:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_features": 4540701547738038271, 2024-04-25T00:26:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: "available_modules": [ 2024-04-25T00:26:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "alerts", 2024-04-25T00:26:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-04-25T00:26:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-04-25T00:26:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-04-25T00:26:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T00:26:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_destination": { 2024-04-25T00:26:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_destination", 2024-04-25T00:26:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email address to send alerts to", 2024-04-25T00:26:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_from_name": { 2024-04-25T00:26:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_from_name", 2024-04-25T00:26:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "Ceph", 2024-04-25T00:26:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email From: name", 2024-04-25T00:26:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_host": { 2024-04-25T00:26:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_host", 2024-04-25T00:26:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP server", 2024-04-25T00:26:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_password": { 2024-04-25T00:26:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_password", 2024-04-25T00:26:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.176 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.176 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.176 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.176 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.176 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.176 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Password to authenticate with", 2024-04-25T00:26:53.176 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.176 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.176 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.176 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.176 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_port": { 2024-04-25T00:26:53.176 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_port", 2024-04-25T00:26:53.176 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.176 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "465", 2024-04-25T00:26:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP port", 2024-04-25T00:26:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_sender": { 2024-04-25T00:26:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_sender", 2024-04-25T00:26:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP envelope sender", 2024-04-25T00:26:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_ssl": { 2024-04-25T00:26:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_ssl", 2024-04-25T00:26:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T00:26:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_user": { 2024-04-25T00:26:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_user", 2024-04-25T00:26:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "User to authenticate as", 2024-04-25T00:26:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "balancer", 2024-04-25T00:26:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.181 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.181 INFO:teuthology.orchestra.run.smithi107.stdout: "active": { 2024-04-25T00:26:53.181 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "active", 2024-04-25T00:26:53.181 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.181 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.181 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.181 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.181 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.181 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.181 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.181 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T00:26:53.181 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.181 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.181 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.182 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.182 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_time": { 2024-04-25T00:26:53.182 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_time", 2024-04-25T00:26:53.182 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.182 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.182 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.182 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0000", 2024-04-25T00:26:53.182 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.182 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.182 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.182 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T00:26:53.182 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:26:53.182 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.182 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_weekday": { 2024-04-25T00:26:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_weekday", 2024-04-25T00:26:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-04-25T00:26:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-04-25T00:26:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-04-25T00:26:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-04-25T00:26:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T00:26:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:26:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.184 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_max_iterations": { 2024-04-25T00:26:53.184 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_max_iterations", 2024-04-25T00:26:53.184 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-04-25T00:26:53.184 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.184 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.184 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "25", 2024-04-25T00:26:53.184 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-04-25T00:26:53.184 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "250", 2024-04-25T00:26:53.184 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.184 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T00:26:53.184 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.184 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.184 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.184 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.184 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_metrics": { 2024-04-25T00:26:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_metrics", 2024-04-25T00:26:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T00:26:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T00:26:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T00:26:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_step": { 2024-04-25T00:26:53.186 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_step", 2024-04-25T00:26:53.186 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-04-25T00:26:53.186 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.186 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.186 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.5", 2024-04-25T00:26:53.186 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0.001", 2024-04-25T00:26:53.186 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "0.999", 2024-04-25T00:26:53.186 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.186 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "aggressiveness of optimization", 2024-04-25T00:26:53.186 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T00:26:53.186 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.186 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.186 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.186 INFO:teuthology.orchestra.run.smithi107.stdout: "end_time": { 2024-04-25T00:26:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_time", 2024-04-25T00:26:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2359", 2024-04-25T00:26:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T00:26:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:26:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "end_weekday": { 2024-04-25T00:26:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_weekday", 2024-04-25T00:26:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-04-25T00:26:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.188 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.188 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-04-25T00:26:53.188 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-04-25T00:26:53.188 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-04-25T00:26:53.188 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.188 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T00:26:53.188 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:26:53.188 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.188 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.188 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.188 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.188 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.188 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.188 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.188 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.189 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.189 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.189 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.189 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.189 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.189 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.189 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.189 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.189 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.189 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.189 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.189 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.189 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.189 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.189 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.189 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.190 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.190 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.190 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.190 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.190 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.190 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.190 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.190 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.190 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.190 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.190 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.190 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.190 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.190 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.193 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.193 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.193 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.193 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.193 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.193 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.193 INFO:teuthology.orchestra.run.smithi107.stdout: "min_score": { 2024-04-25T00:26:53.193 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_score", 2024-04-25T00:26:53.193 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-04-25T00:26:53.193 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.193 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.193 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-04-25T00:26:53.193 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.193 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.194 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.194 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T00:26:53.194 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.194 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.194 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.194 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.194 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-04-25T00:26:53.194 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-04-25T00:26:53.194 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.194 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.194 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.194 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "upmap", 2024-04-25T00:26:53.194 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.194 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.195 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.195 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-compat", 2024-04-25T00:26:53.195 INFO:teuthology.orchestra.run.smithi107.stdout: "none", 2024-04-25T00:26:53.195 INFO:teuthology.orchestra.run.smithi107.stdout: "read", 2024-04-25T00:26:53.195 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap", 2024-04-25T00:26:53.195 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap-read" 2024-04-25T00:26:53.195 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.195 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Balancer mode", 2024-04-25T00:26:53.195 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.195 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.195 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.195 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.195 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_ids": { 2024-04-25T00:26:53.195 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_ids", 2024-04-25T00:26:53.195 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T00:26:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-04-25T00:26:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-04-25T00:26:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-04-25T00:26:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T00:26:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_deviation": { 2024-04-25T00:26:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_deviation", 2024-04-25T00:26:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-04-25T00:26:53.198 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-04-25T00:26:53.198 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.198 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.198 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T00:26:53.198 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T00:26:53.198 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.198 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.198 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.198 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_optimizations": { 2024-04-25T00:26:53.198 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_optimizations", 2024-04-25T00:26:53.198 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-04-25T00:26:53.198 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.198 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.198 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-04-25T00:26:53.199 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.199 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.199 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.199 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T00:26:53.199 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.199 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.199 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.199 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.199 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.199 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.199 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.199 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm", 2024-04-25T00:26:53.199 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.200 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.200 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.200 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_down_multiplier": { 2024-04-25T00:26:53.200 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_down_multiplier", 2024-04-25T00:26:53.200 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-04-25T00:26:53.200 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.200 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.200 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-04-25T00:26:53.200 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.200 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.200 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.200 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T00:26:53.200 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.200 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.200 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.201 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.201 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_refresh_rate": { 2024-04-25T00:26:53.201 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_refresh_rate", 2024-04-25T00:26:53.201 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.201 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.201 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.201 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "20", 2024-04-25T00:26:53.201 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.201 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.201 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.201 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T00:26:53.201 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.201 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.201 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.201 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.202 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_starting_port": { 2024-04-25T00:26:53.202 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_starting_port", 2024-04-25T00:26:53.202 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.202 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.202 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.202 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4721", 2024-04-25T00:26:53.202 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.202 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.202 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.202 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T00:26:53.202 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.202 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.202 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.202 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.202 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_ptrace": { 2024-04-25T00:26:53.203 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_ptrace", 2024-04-25T00:26:53.203 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.203 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.203 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.203 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.203 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.203 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.203 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.203 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T00:26:53.203 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T00:26:53.203 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.203 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.203 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.203 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_interval": { 2024-04-25T00:26:53.204 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_interval", 2024-04-25T00:26:53.204 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.204 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.204 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.204 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-04-25T00:26:53.204 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.204 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.204 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.204 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T00:26:53.204 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.204 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.204 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.204 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.204 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_memory_target_ratio": { 2024-04-25T00:26:53.205 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T00:26:53.205 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-04-25T00:26:53.205 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.205 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.205 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.7", 2024-04-25T00:26:53.205 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.205 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.205 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.205 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T00:26:53.205 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.205 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.205 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.205 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.205 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm_log_destination": { 2024-04-25T00:26:53.205 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm_log_destination", 2024-04-25T00:26:53.206 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.206 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.206 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.206 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.206 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.206 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.206 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.206 INFO:teuthology.orchestra.run.smithi107.stdout: "file", 2024-04-25T00:26:53.206 INFO:teuthology.orchestra.run.smithi107.stdout: "file,syslog", 2024-04-25T00:26:53.206 INFO:teuthology.orchestra.run.smithi107.stdout: "syslog" 2024-04-25T00:26:53.206 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.206 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T00:26:53.206 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.206 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.207 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.207 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.207 INFO:teuthology.orchestra.run.smithi107.stdout: "cgroups_split": { 2024-04-25T00:26:53.207 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cgroups_split", 2024-04-25T00:26:53.207 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.207 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.207 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.207 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.207 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.207 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.207 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.207 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T00:26:53.207 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.207 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.208 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.208 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.208 INFO:teuthology.orchestra.run.smithi107.stdout: "config_checks_enabled": { 2024-04-25T00:26:53.208 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_checks_enabled", 2024-04-25T00:26:53.208 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.208 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.208 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.208 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.208 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.208 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.208 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.208 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T00:26:53.208 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.208 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.208 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: "config_dashboard": { 2024-04-25T00:26:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_dashboard", 2024-04-25T00:26:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T00:26:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_alertmanager": { 2024-04-25T00:26:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_alertmanager", 2024-04-25T00:26:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T00:26:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-04-25T00:26:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_base": { 2024-04-25T00:26:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_base", 2024-04-25T00:26:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T00:26:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Container image name, without the tag", 2024-04-25T00:26:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_elasticsearch": { 2024-04-25T00:26:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_elasticsearch", 2024-04-25T00:26:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T00:26:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "elasticsearch container image", 2024-04-25T00:26:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_grafana": { 2024-04-25T00:26:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_grafana", 2024-04-25T00:26:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T00:26:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-04-25T00:26:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_haproxy": { 2024-04-25T00:26:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_haproxy", 2024-04-25T00:26:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T00:26:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "HAproxy container image", 2024-04-25T00:26:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_agent": { 2024-04-25T00:26:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_agent", 2024-04-25T00:26:53.215 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.215 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.215 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.215 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T00:26:53.215 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.215 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.215 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.215 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger agent container image", 2024-04-25T00:26:53.215 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.215 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.215 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.215 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_collector": { 2024-04-25T00:26:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_collector", 2024-04-25T00:26:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T00:26:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger collector container image", 2024-04-25T00:26:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_query": { 2024-04-25T00:26:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_query", 2024-04-25T00:26:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T00:26:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger query container image", 2024-04-25T00:26:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_keepalived": { 2024-04-25T00:26:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_keepalived", 2024-04-25T00:26:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.218 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.218 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T00:26:53.218 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.218 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.218 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.218 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Keepalived container image", 2024-04-25T00:26:53.218 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.218 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.218 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.218 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_loki": { 2024-04-25T00:26:53.218 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_loki", 2024-04-25T00:26:53.218 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T00:26:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Loki container image", 2024-04-25T00:26:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_node_exporter": { 2024-04-25T00:26:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_node_exporter", 2024-04-25T00:26:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.220 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.220 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T00:26:53.220 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.220 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.220 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.220 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-04-25T00:26:53.220 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.220 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.220 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.220 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_nvmeof": { 2024-04-25T00:26:53.220 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_nvmeof", 2024-04-25T00:26:53.220 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.220 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.220 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.221 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T00:26:53.221 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.221 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.221 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.221 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Nvme-of container image", 2024-04-25T00:26:53.221 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.221 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.221 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.221 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_prometheus": { 2024-04-25T00:26:53.221 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_prometheus", 2024-04-25T00:26:53.221 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.221 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.221 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T00:26:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-04-25T00:26:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_promtail": { 2024-04-25T00:26:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_promtail", 2024-04-25T00:26:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T00:26:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Promtail container image", 2024-04-25T00:26:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_samba": { 2024-04-25T00:26:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_samba", 2024-04-25T00:26:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.224 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T00:26:53.224 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.224 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.224 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.224 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Samba/SMB container image", 2024-04-25T00:26:53.224 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.224 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.224 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.224 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_snmp_gateway": { 2024-04-25T00:26:53.224 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_snmp_gateway", 2024-04-25T00:26:53.224 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.224 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.224 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.225 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T00:26:53.225 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.225 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.225 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.225 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SNMP Gateway container image", 2024-04-25T00:26:53.225 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.225 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.225 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.225 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.225 INFO:teuthology.orchestra.run.smithi107.stdout: "container_init": { 2024-04-25T00:26:53.225 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_init", 2024-04-25T00:26:53.225 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.225 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.225 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.225 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.226 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.226 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.226 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.226 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T00:26:53.226 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.226 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.226 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.226 INFO:teuthology.orchestra.run.smithi107.stdout: "daemon_cache_timeout": { 2024-04-25T00:26:53.226 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "daemon_cache_timeout", 2024-04-25T00:26:53.226 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.226 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.226 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-04-25T00:26:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T00:26:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "default_cephadm_command_timeout": { 2024-04-25T00:26:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T00:26:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "900", 2024-04-25T00:26:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T00:26:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: "default_registry": { 2024-04-25T00:26:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_registry", 2024-04-25T00:26:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io", 2024-04-25T00:26:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T00:26:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "device_cache_timeout": { 2024-04-25T00:26:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_cache_timeout", 2024-04-25T00:26:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1800", 2024-04-25T00:26:53.230 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.230 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.230 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.230 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache device inventory", 2024-04-25T00:26:53.230 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.230 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.230 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.230 INFO:teuthology.orchestra.run.smithi107.stdout: "device_enhanced_scan": { 2024-04-25T00:26:53.230 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_enhanced_scan", 2024-04-25T00:26:53.230 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.230 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.230 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.230 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.231 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.231 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.231 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.231 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T00:26:53.231 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.231 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.231 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.231 INFO:teuthology.orchestra.run.smithi107.stdout: "facts_cache_timeout": { 2024-04-25T00:26:53.231 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "facts_cache_timeout", 2024-04-25T00:26:53.231 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.231 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.231 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.232 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-04-25T00:26:53.232 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.232 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.232 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.232 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache host facts data", 2024-04-25T00:26:53.232 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.232 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.232 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.232 INFO:teuthology.orchestra.run.smithi107.stdout: "grafana_dashboards_path": { 2024-04-25T00:26:53.232 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "grafana_dashboards_path", 2024-04-25T00:26:53.232 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.232 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T00:26:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T00:26:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "host_check_interval": { 2024-04-25T00:26:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "host_check_interval", 2024-04-25T00:26:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-04-25T00:26:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to perform a host check", 2024-04-25T00:26:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: "hw_monitoring": { 2024-04-25T00:26:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hw_monitoring", 2024-04-25T00:26:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.235 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.235 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.235 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T00:26:53.235 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.235 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.235 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.235 INFO:teuthology.orchestra.run.smithi107.stdout: "inventory_list_all": { 2024-04-25T00:26:53.235 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "inventory_list_all", 2024-04-25T00:26:53.235 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.235 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.235 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.235 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.235 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.235 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.235 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.236 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T00:26:53.236 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.236 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.236 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.236 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.236 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.236 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.236 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.236 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.236 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.236 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.236 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.236 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.236 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.236 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.236 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.237 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.237 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.237 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.237 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.237 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.237 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.237 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.237 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.237 INFO:teuthology.orchestra.run.smithi107.stdout: "log_refresh_metadata": { 2024-04-25T00:26:53.237 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_refresh_metadata", 2024-04-25T00:26:53.237 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.237 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.237 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.237 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T00:26:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.239 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.239 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.239 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.239 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T00:26:53.239 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.239 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.239 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.239 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.239 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.239 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.239 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.239 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.239 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.240 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.240 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.240 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.240 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.240 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.240 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.240 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.240 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.240 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.240 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.240 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.240 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.240 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.240 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T00:26:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T00:26:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T00:26:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T00:26:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T00:26:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "*", 2024-04-25T00:26:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T00:26:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "max_count_per_host": { 2024-04-25T00:26:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_count_per_host", 2024-04-25T00:26:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-04-25T00:26:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of daemons per service per host", 2024-04-25T00:26:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.245 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.245 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd_draining_count": { 2024-04-25T00:26:53.245 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_osd_draining_count", 2024-04-25T00:26:53.245 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.245 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.245 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.245 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-04-25T00:26:53.245 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.245 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.245 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.245 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T00:26:53.245 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.245 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.245 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: "migration_current": { 2024-04-25T00:26:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "migration_current", 2024-04-25T00:26:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "internal - do not modify", 2024-04-25T00:26:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-04-25T00:26:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-04-25T00:26:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "root", 2024-04-25T00:26:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm-package", 2024-04-25T00:26:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: "root" 2024-04-25T00:26:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T00:26:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: "oob_default_addr": { 2024-04-25T00:26:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "oob_default_addr", 2024-04-25T00:26:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "169.254.1.1", 2024-04-25T00:26:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T00:26:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.249 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.249 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.249 INFO:teuthology.orchestra.run.smithi107.stdout: "prometheus_alerts_path": { 2024-04-25T00:26:53.249 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus_alerts_path", 2024-04-25T00:26:53.249 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.249 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.249 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.249 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T00:26:53.249 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.249 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.249 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.249 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T00:26:53.249 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.249 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.250 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.250 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_insecure": { 2024-04-25T00:26:53.250 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_insecure", 2024-04-25T00:26:53.250 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.250 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.250 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.250 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.250 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.250 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.250 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.250 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T00:26:53.250 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_password": { 2024-04-25T00:26:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_password", 2024-04-25T00:26:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T00:26:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_url": { 2024-04-25T00:26:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_url", 2024-04-25T00:26:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T00:26:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_username": { 2024-04-25T00:26:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_username", 2024-04-25T00:26:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T00:26:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.254 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.254 INFO:teuthology.orchestra.run.smithi107.stdout: "secure_monitoring_stack": { 2024-04-25T00:26:53.254 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "secure_monitoring_stack", 2024-04-25T00:26:53.254 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.254 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.254 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.254 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.254 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.254 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.254 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.254 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T00:26:53.254 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.254 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.255 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.255 INFO:teuthology.orchestra.run.smithi107.stdout: "service_discovery_port": { 2024-04-25T00:26:53.255 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "service_discovery_port", 2024-04-25T00:26:53.255 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.255 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.255 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.255 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8765", 2024-04-25T00:26:53.255 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.255 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.255 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.255 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "cephadm service discovery port", 2024-04-25T00:26:53.255 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.255 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.256 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.256 INFO:teuthology.orchestra.run.smithi107.stdout: "ssh_config_file": { 2024-04-25T00:26:53.256 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssh_config_file", 2024-04-25T00:26:53.256 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.256 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.256 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.256 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.256 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.256 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.256 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.256 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T00:26:53.256 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.256 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.257 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.257 INFO:teuthology.orchestra.run.smithi107.stdout: "use_agent": { 2024-04-25T00:26:53.257 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_agent", 2024-04-25T00:26:53.257 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.257 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.257 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.257 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.257 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.257 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.257 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.257 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T00:26:53.257 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.257 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.258 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.258 INFO:teuthology.orchestra.run.smithi107.stdout: "use_repo_digest": { 2024-04-25T00:26:53.258 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_repo_digest", 2024-04-25T00:26:53.258 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.258 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.258 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.258 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.258 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.258 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.258 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.258 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T00:26:53.258 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.258 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.258 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.259 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_failed_host_check": { 2024-04-25T00:26:53.259 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_failed_host_check", 2024-04-25T00:26:53.259 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.259 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.259 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.259 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.259 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.259 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.259 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.259 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T00:26:53.259 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.259 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.259 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.259 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.259 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_daemons": { 2024-04-25T00:26:53.259 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_daemons", 2024-04-25T00:26:53.260 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.260 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.260 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.260 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.260 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.260 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.260 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.260 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T00:26:53.260 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.260 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.260 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.260 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_hosts": { 2024-04-25T00:26:53.260 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_hosts", 2024-04-25T00:26:53.260 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.261 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.261 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.261 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.261 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.261 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.261 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.261 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T00:26:53.261 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.261 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.261 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.261 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.261 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.261 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.261 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crash", 2024-04-25T00:26:53.262 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.262 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.262 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.262 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.262 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.262 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.262 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.262 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.262 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.262 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.262 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.262 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.262 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.262 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.263 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.263 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.263 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.263 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.263 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.263 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.263 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.263 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.263 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.263 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.263 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.263 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.263 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.266 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.266 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.266 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.266 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.266 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.266 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.266 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.266 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.266 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.266 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.266 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.266 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.266 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.267 INFO:teuthology.orchestra.run.smithi107.stdout: "retain_interval": { 2024-04-25T00:26:53.267 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retain_interval", 2024-04-25T00:26:53.267 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.267 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.267 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.267 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "31536000", 2024-04-25T00:26:53.267 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.267 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.267 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.267 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T00:26:53.267 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.267 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.267 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.267 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_recent_interval": { 2024-04-25T00:26:53.268 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_recent_interval", 2024-04-25T00:26:53.268 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.268 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.268 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.268 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1209600", 2024-04-25T00:26:53.268 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.268 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.268 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.268 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T00:26:53.268 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.268 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.268 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.268 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.268 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.269 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.269 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dashboard", 2024-04-25T00:26:53.269 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.269 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.269 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.269 INFO:teuthology.orchestra.run.smithi107.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T00:26:53.269 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T00:26:53.269 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.269 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.269 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.269 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-04-25T00:26:53.269 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.269 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.269 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.269 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.270 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.270 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.270 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.270 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T00:26:53.270 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T00:26:53.270 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.270 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.270 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.270 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.270 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.270 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.270 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.270 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.270 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T00:26:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T00:26:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_ENABLED": { 2024-04-25T00:26:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T00:26:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T00:26:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T00:26:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T00:26:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T00:26:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.274 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.274 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.274 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.274 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.274 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.274 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.274 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.274 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.274 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.274 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.274 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T00:26:53.274 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T00:26:53.274 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T00:26:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T00:26:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T00:26:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T00:26:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T00:26:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T00:26:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.278 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.278 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.278 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.278 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.278 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.278 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T00:26:53.278 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T00:26:53.278 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.278 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.278 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.278 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.278 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.278 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.279 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.279 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.279 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.279 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.279 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.279 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T00:26:53.279 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T00:26:53.279 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.279 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.279 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.279 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.279 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.279 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.279 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T00:26:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T00:26:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.281 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.281 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.281 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.281 INFO:teuthology.orchestra.run.smithi107.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T00:26:53.281 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T00:26:53.281 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.281 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.281 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.281 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.281 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.281 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.281 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T00:26:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T00:26:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-04-25T00:26:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T00:26:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T00:26:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_URL": { 2024-04-25T00:26:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_URL", 2024-04-25T00:26:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.285 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.285 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.285 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.285 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.285 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.285 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T00:26:53.285 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T00:26:53.285 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.285 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.285 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.285 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-04-25T00:26:53.285 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.285 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.285 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T00:26:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T00:26:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.287 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.287 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.287 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.287 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.287 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.287 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T00:26:53.287 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T00:26:53.287 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.287 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.287 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.287 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.287 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.287 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.287 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.287 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.287 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T00:26:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T00:26:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T00:26:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T00:26:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T00:26:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T00:26:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T00:26:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T00:26:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T00:26:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T00:26:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T00:26:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T00:26:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.294 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T00:26:53.294 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T00:26:53.294 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.294 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.294 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.294 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.294 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.294 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.294 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.294 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.294 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.294 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.294 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T00:26:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T00:26:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T00:26:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T00:26:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T00:26:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T00:26:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T00:26:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T00:26:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T00:26:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T00:26:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T00:26:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T00:26:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T00:26:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T00:26:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T00:26:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-04-25T00:26:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T00:26:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T00:26:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8", 2024-04-25T00:26:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T00:26:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T00:26:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "45", 2024-04-25T00:26:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T00:26:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T00:26:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T00:26:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T00:26:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-04-25T00:26:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T00:26:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T00:26:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T00:26:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T00:26:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T00:26:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T00:26:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-04-25T00:26:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T00:26:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T00:26:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-04-25T00:26:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T00:26:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T00:26:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-04-25T00:26:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "cross_origin_url": { 2024-04-25T00:26:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cross_origin_url", 2024-04-25T00:26:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "crt_file": { 2024-04-25T00:26:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crt_file", 2024-04-25T00:26:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "debug": { 2024-04-25T00:26:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "debug", 2024-04-25T00:26:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable/disable debug options", 2024-04-25T00:26:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "jwt_token_ttl": { 2024-04-25T00:26:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "jwt_token_ttl", 2024-04-25T00:26:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "28800", 2024-04-25T00:26:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-04-25T00:26:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-04-25T00:26:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "motd": { 2024-04-25T00:26:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "motd", 2024-04-25T00:26:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "The message of the day", 2024-04-25T00:26:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect_resolve_ip_addr": { 2024-04-25T00:26:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T00:26:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-04-25T00:26:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-04-25T00:26:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-04-25T00:26:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-04-25T00:26:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-04-25T00:26:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8080", 2024-04-25T00:26:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-04-25T00:26:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-04-25T00:26:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl_server_port": { 2024-04-25T00:26:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl_server_port", 2024-04-25T00:26:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8443", 2024-04-25T00:26:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-04-25T00:26:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-04-25T00:26:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "redirect", 2024-04-25T00:26:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect" 2024-04-25T00:26:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-04-25T00:26:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-04-25T00:26:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-04-25T00:26:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-04-25T00:26:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-04-25T00:26:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "url_prefix": { 2024-04-25T00:26:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url_prefix", 2024-04-25T00:26:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "devicehealth", 2024-04-25T00:26:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_monitoring": { 2024-04-25T00:26:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_monitoring", 2024-04-25T00:26:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "monitor device health metrics", 2024-04-25T00:26:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.329 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.329 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.329 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.329 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.329 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.329 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.329 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.329 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.329 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.329 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.329 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.329 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.329 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.329 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "mark_out_threshold": { 2024-04-25T00:26:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mark_out_threshold", 2024-04-25T00:26:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2419200", 2024-04-25T00:26:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T00:26:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": { 2024-04-25T00:26:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_name", 2024-04-25T00:26:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "device_health_metrics", 2024-04-25T00:26:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T00:26:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: "retention_period": { 2024-04-25T00:26:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retention_period", 2024-04-25T00:26:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15552000", 2024-04-25T00:26:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain device health metrics", 2024-04-25T00:26:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_frequency": { 2024-04-25T00:26:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_frequency", 2024-04-25T00:26:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-04-25T00:26:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.337 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.337 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.337 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T00:26:53.337 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.337 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.337 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.337 INFO:teuthology.orchestra.run.smithi107.stdout: "self_heal": { 2024-04-25T00:26:53.337 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "self_heal", 2024-04-25T00:26:53.337 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.337 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.337 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.337 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T00:26:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-04-25T00:26:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-04-25T00:26:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-04-25T00:26:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T00:26:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_threshold": { 2024-04-25T00:26:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_threshold", 2024-04-25T00:26:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7257600", 2024-04-25T00:26:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.340 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.340 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.340 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T00:26:53.340 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.340 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.340 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.340 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.340 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.340 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.340 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "diskprediction_local", 2024-04-25T00:26:53.340 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.340 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: "predict_interval": { 2024-04-25T00:26:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predict_interval", 2024-04-25T00:26:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-04-25T00:26:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "predictor_model": { 2024-04-25T00:26:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predictor_model", 2024-04-25T00:26:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "prophetstor", 2024-04-25T00:26:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.348 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-04-25T00:26:53.348 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-04-25T00:26:53.348 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.348 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.348 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.348 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-04-25T00:26:53.348 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.348 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.348 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.348 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.348 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.348 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.348 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.348 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "influx", 2024-04-25T00:26:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": false, 2024-04-25T00:26:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "influxdb python module not found", 2024-04-25T00:26:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: "batch_size": { 2024-04-25T00:26:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "batch_size", 2024-04-25T00:26:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5000", 2024-04-25T00:26:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.350 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.350 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.350 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T00:26:53.350 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.350 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.350 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.350 INFO:teuthology.orchestra.run.smithi107.stdout: "database": { 2024-04-25T00:26:53.350 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "database", 2024-04-25T00:26:53.350 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.350 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.350 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.350 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "ceph", 2024-04-25T00:26:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T00:26:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": { 2024-04-25T00:26:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hostname", 2024-04-25T00:26:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server hostname", 2024-04-25T00:26:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-04-25T00:26:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-04-25T00:26:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "30", 2024-04-25T00:26:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "5", 2024-04-25T00:26:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T00:26:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: "password": { 2024-04-25T00:26:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "password", 2024-04-25T00:26:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "password of InfluxDB server user", 2024-04-25T00:26:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "port": { 2024-04-25T00:26:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "port", 2024-04-25T00:26:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8086", 2024-04-25T00:26:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server port", 2024-04-25T00:26:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-04-25T00:26:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-04-25T00:26:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "false", 2024-04-25T00:26:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:26:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "threads": { 2024-04-25T00:26:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threads", 2024-04-25T00:26:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-04-25T00:26:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-04-25T00:26:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "32", 2024-04-25T00:26:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T00:26:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "username": { 2024-04-25T00:26:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "username", 2024-04-25T00:26:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "username of InfluxDB server user", 2024-04-25T00:26:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: "verify_ssl": { 2024-04-25T00:26:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "verify_ssl", 2024-04-25T00:26:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "true", 2024-04-25T00:26:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:26:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "insights", 2024-04-25T00:26:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "iostat", 2024-04-25T00:26:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.371 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.371 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.371 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.371 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.371 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.371 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.371 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.371 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.373 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.373 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.373 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.373 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.373 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.373 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.373 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.373 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "k8sevents", 2024-04-25T00:26:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_event_retention_days": { 2024-04-25T00:26:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ceph_event_retention_days", 2024-04-25T00:26:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7", 2024-04-25T00:26:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T00:26:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "config_check_secs": { 2024-04-25T00:26:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_check_secs", 2024-04-25T00:26:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-04-25T00:26:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "10", 2024-04-25T00:26:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T00:26:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "localpool", 2024-04-25T00:26:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: "failure_domain": { 2024-04-25T00:26:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "failure_domain", 2024-04-25T00:26:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "host", 2024-04-25T00:26:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "failure domain for any created local pool", 2024-04-25T00:26:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T00:26:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.386 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.386 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.386 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.386 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.386 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.386 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.386 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.386 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.386 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.386 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.386 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.386 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": { 2024-04-25T00:26:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_size", 2024-04-25T00:26:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default min_size for any created local pool", 2024-04-25T00:26:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T00:26:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "num_rep": { 2024-04-25T00:26:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "num_rep", 2024-04-25T00:26:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3", 2024-04-25T00:26:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default replica count for any created local pool", 2024-04-25T00:26:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": { 2024-04-25T00:26:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_num", 2024-04-25T00:26:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "128", 2024-04-25T00:26:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T00:26:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "prefix": { 2024-04-25T00:26:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prefix", 2024-04-25T00:26:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name prefix for any created local pool", 2024-04-25T00:26:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "subtree": { 2024-04-25T00:26:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "subtree", 2024-04-25T00:26:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "rack", 2024-04-25T00:26:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T00:26:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T00:26:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mds_autoscaler", 2024-04-25T00:26:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirroring", 2024-04-25T00:26:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "nfs", 2024-04-25T00:26:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-04-25T00:26:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "fail_fs": { 2024-04-25T00:26:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "fail_fs", 2024-04-25T00:26:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T00:26:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator": { 2024-04-25T00:26:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-04-25T00:26:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm", 2024-04-25T00:26:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "rook", 2024-04-25T00:26:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "test_orchestrator" 2024-04-25T00:26:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Orchestrator backend", 2024-04-25T00:26:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_perf_query", 2024-04-25T00:26:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_support", 2024-04-25T00:26:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_autoscaler", 2024-04-25T00:26:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-04-25T00:26:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-04-25T00:26:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-04-25T00:26:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: "threshold": { 2024-04-25T00:26:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threshold", 2024-04-25T00:26:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-04-25T00:26:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-04-25T00:26:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1.0", 2024-04-25T00:26:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "scaling threshold", 2024-04-25T00:26:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T00:26:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "progress", 2024-04-25T00:26:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_pg_recovery_event": { 2024-04-25T00:26:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_pg_recovery_event", 2024-04-25T00:26:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T00:26:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-04-25T00:26:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-04-25T00:26:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: "max_completed_events": { 2024-04-25T00:26:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_completed_events", 2024-04-25T00:26:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "50", 2024-04-25T00:26:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "number of past completed events to remember", 2024-04-25T00:26:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-04-25T00:26:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-04-25T00:26:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.446 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-04-25T00:26:53.446 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.446 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.446 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.446 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long the module is going to sleep", 2024-04-25T00:26:53.446 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.446 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.446 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.446 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.446 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.446 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.446 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus", 2024-04-25T00:26:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "cache": { 2024-04-25T00:26:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cache", 2024-04-25T00:26:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: "exclude_perf_counters": { 2024-04-25T00:26:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "exclude_perf_counters", 2024-04-25T00:26:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T00:26:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T00:26:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools": { 2024-04-25T00:26:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools", 2024-04-25T00:26:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T00:26:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T00:26:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "300", 2024-04-25T00:26:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_interval": { 2024-04-25T00:26:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_interval", 2024-04-25T00:26:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-04-25T00:26:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15.0", 2024-04-25T00:26:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-04-25T00:26:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-04-25T00:26:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-04-25T00:26:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T00:26:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-04-25T00:26:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-04-25T00:26:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "9283", 2024-04-25T00:26:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T00:26:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "stale_cache_strategy": { 2024-04-25T00:26:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stale_cache_strategy", 2024-04-25T00:26:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "log", 2024-04-25T00:26:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-04-25T00:26:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-04-25T00:26:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "default", 2024-04-25T00:26:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "default", 2024-04-25T00:26:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "error" 2024-04-25T00:26:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-04-25T00:26:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-04-25T00:26:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-04-25T00:26:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-04-25T00:26:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-04-25T00:26:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_support", 2024-04-25T00:26:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_snap_create": { 2024-04-25T00:26:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_snap_create", 2024-04-25T00:26:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-04-25T00:26:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "mirror_snapshot_schedule": { 2024-04-25T00:26:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T00:26:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "trash_purge_schedule": { 2024-04-25T00:26:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "trash_purge_schedule", 2024-04-25T00:26:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "restful", 2024-04-25T00:26:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_auth": { 2024-04-25T00:26:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_auth", 2024-04-25T00:26:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-04-25T00:26:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-04-25T00:26:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.473 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-04-25T00:26:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-04-25T00:26:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-04-25T00:26:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-04-25T00:26:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rgw", 2024-04-25T00:26:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rook", 2024-04-25T00:26:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "storage_class": { 2024-04-25T00:26:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "storage_class", 2024-04-25T00:26:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "local", 2024-04-25T00:26:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T00:26:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "selftest", 2024-04-25T00:26:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "roption1": { 2024-04-25T00:26:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption1", 2024-04-25T00:26:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "roption2": { 2024-04-25T00:26:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption2", 2024-04-25T00:26:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "xyz", 2024-04-25T00:26:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption1": { 2024-04-25T00:26:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption1", 2024-04-25T00:26:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption2": { 2024-04-25T00:26:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption2", 2024-04-25T00:26:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption3": { 2024-04-25T00:26:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption3", 2024-04-25T00:26:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-04-25T00:26:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption4": { 2024-04-25T00:26:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption4", 2024-04-25T00:26:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption5": { 2024-04-25T00:26:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption5", 2024-04-25T00:26:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption6": { 2024-04-25T00:26:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption6", 2024-04-25T00:26:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption7": { 2024-04-25T00:26:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption7", 2024-04-25T00:26:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-04-25T00:26:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "42", 2024-04-25T00:26:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "testkey": { 2024-04-25T00:26:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testkey", 2024-04-25T00:26:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "testlkey": { 2024-04-25T00:26:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testlkey", 2024-04-25T00:26:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "testnewline": { 2024-04-25T00:26:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testnewline", 2024-04-25T00:26:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2024-04-25T00:26:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_m_granularity": { 2024-04-25T00:26:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_m_granularity", 2024-04-25T00:26:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T00:26:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "dump_on_update": { 2024-04-25T00:26:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dump_on_update", 2024-04-25T00:26:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "dump database to debug log on update", 2024-04-25T00:26:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.512 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.513 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.514 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.515 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stats", 2024-04-25T00:26:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.516 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.517 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.518 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.519 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.520 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.521 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.522 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.522 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.522 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.522 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "status", 2024-04-25T00:26:53.522 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.522 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.522 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.522 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.522 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.522 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.522 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.522 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.523 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.524 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.525 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.526 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telegraf", 2024-04-25T00:26:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.527 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.528 INFO:teuthology.orchestra.run.smithi107.stdout: "address": { 2024-04-25T00:26:53.528 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "address", 2024-04-25T00:26:53.528 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.528 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.528 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.528 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T00:26:53.528 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.528 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.528 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.528 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.528 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.528 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.528 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.528 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-04-25T00:26:53.528 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-04-25T00:26:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15", 2024-04-25T00:26:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.529 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.530 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.531 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.532 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.533 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telemetry", 2024-04-25T00:26:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_basic": { 2024-04-25T00:26:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_basic", 2024-04-25T00:26:53.534 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T00:26:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_crash": { 2024-04-25T00:26:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_crash", 2024-04-25T00:26:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.535 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T00:26:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_device": { 2024-04-25T00:26:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_device", 2024-04-25T00:26:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.536 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T00:26:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_ident": { 2024-04-25T00:26:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_ident", 2024-04-25T00:26:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.537 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T00:26:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_perf": { 2024-04-25T00:26:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_perf", 2024-04-25T00:26:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.538 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T00:26:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: "contact": { 2024-04-25T00:26:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "contact", 2024-04-25T00:26:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.539 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: "description": { 2024-04-25T00:26:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "description", 2024-04-25T00:26:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.540 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: "device_url": { 2024-04-25T00:26:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_url", 2024-04-25T00:26:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T00:26:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.541 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-04-25T00:26:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-04-25T00:26:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.542 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-04-25T00:26:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-04-25T00:26:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "24", 2024-04-25T00:26:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "8", 2024-04-25T00:26:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.543 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: "last_opt_revision": { 2024-04-25T00:26:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "last_opt_revision", 2024-04-25T00:26:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1", 2024-04-25T00:26:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.544 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard": { 2024-04-25T00:26:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard", 2024-04-25T00:26:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.545 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.546 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.546 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.546 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.546 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard_description": { 2024-04-25T00:26:53.546 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard_description", 2024-04-25T00:26:53.546 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.546 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.546 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.546 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.546 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.546 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.546 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.546 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.546 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.547 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.548 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.548 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.548 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.548 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.548 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.548 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.548 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.548 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.548 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.548 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.548 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.548 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.548 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.549 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.550 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.550 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.550 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.550 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.550 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.550 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.550 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.550 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.550 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.550 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.550 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.550 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.550 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.550 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.552 INFO:teuthology.orchestra.run.smithi107.stdout: "organization": { 2024-04-25T00:26:53.552 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "organization", 2024-04-25T00:26:53.552 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.552 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.552 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.552 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.552 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.552 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.552 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.552 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.552 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.552 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.552 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.552 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.552 INFO:teuthology.orchestra.run.smithi107.stdout: "proxy": { 2024-04-25T00:26:53.553 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "proxy", 2024-04-25T00:26:53.553 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.553 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.553 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.553 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.553 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.553 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.553 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.553 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.553 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.553 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.553 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.553 INFO:teuthology.orchestra.run.smithi107.stdout: "url": { 2024-04-25T00:26:53.553 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url", 2024-04-25T00:26:53.554 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.554 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.554 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.554 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T00:26:53.554 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.554 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.554 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.554 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.554 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.554 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.554 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.554 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.554 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.554 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.555 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.555 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "test_orchestrator", 2024-04-25T00:26:53.555 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.555 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.555 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.555 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.555 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.555 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.555 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.555 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.555 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.555 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.555 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.556 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.556 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.556 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.556 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.556 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.556 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.556 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.556 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.556 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.556 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.556 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.556 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.556 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.556 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.557 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.558 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.558 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.558 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.558 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.558 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.558 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.558 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.558 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.558 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.558 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.558 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.558 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.558 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.559 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.559 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.559 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.559 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.559 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.559 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.559 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.559 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.559 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.559 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.559 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.559 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "volumes", 2024-04-25T00:26:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.560 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.561 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.562 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.563 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.564 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_clones": { 2024-04-25T00:26:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_clones", 2024-04-25T00:26:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4", 2024-04-25T00:26:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T00:26:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.566 INFO:teuthology.orchestra.run.smithi107.stdout: "periodic_async_work": { 2024-04-25T00:26:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "periodic_async_work", 2024-04-25T00:26:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Periodically check for async work", 2024-04-25T00:26:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.567 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_delay": { 2024-04-25T00:26:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_delay", 2024-04-25T00:26:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-04-25T00:26:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T00:26:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.568 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_no_wait": { 2024-04-25T00:26:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T00:26:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T00:26:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.569 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix", 2024-04-25T00:26:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: "discovery_interval": { 2024-04-25T00:26:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "discovery_interval", 2024-04-25T00:26:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-04-25T00:26:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "100", 2024-04-25T00:26:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.570 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: "identifier": { 2024-04-25T00:26:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "identifier", 2024-04-25T00:26:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.571 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-04-25T00:26:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-04-25T00:26:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-04-25T00:26:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.572 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.573 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.574 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.575 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.576 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_host": { 2024-04-25T00:26:53.577 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_host", 2024-04-25T00:26:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_port": { 2024-04-25T00:26:53.578 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_port", 2024-04-25T00:26:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10051", 2024-04-25T00:26:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.579 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_sender": { 2024-04-25T00:26:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_sender", 2024-04-25T00:26:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T00:26:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.580 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-04-25T00:26:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: "modules": [ 2024-04-25T00:26:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm", 2024-04-25T00:26:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: "dashboard", 2024-04-25T00:26:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: "iostat", 2024-04-25T00:26:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: "nfs", 2024-04-25T00:26:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: "restful", 2024-04-25T00:26:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: "snap_schedule" 2024-04-25T00:26:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: "available_modules": [ 2024-04-25T00:26:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "alerts", 2024-04-25T00:26:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.581 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-04-25T00:26:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-04-25T00:26:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-04-25T00:26:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T00:26:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.582 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.583 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.584 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.585 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.586 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_destination": { 2024-04-25T00:26:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_destination", 2024-04-25T00:26:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email address to send alerts to", 2024-04-25T00:26:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.588 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_from_name": { 2024-04-25T00:26:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_from_name", 2024-04-25T00:26:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "Ceph", 2024-04-25T00:26:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email From: name", 2024-04-25T00:26:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.589 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_host": { 2024-04-25T00:26:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_host", 2024-04-25T00:26:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP server", 2024-04-25T00:26:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_password": { 2024-04-25T00:26:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_password", 2024-04-25T00:26:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Password to authenticate with", 2024-04-25T00:26:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.591 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_port": { 2024-04-25T00:26:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_port", 2024-04-25T00:26:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "465", 2024-04-25T00:26:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP port", 2024-04-25T00:26:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.592 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_sender": { 2024-04-25T00:26:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_sender", 2024-04-25T00:26:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP envelope sender", 2024-04-25T00:26:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.593 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_ssl": { 2024-04-25T00:26:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_ssl", 2024-04-25T00:26:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T00:26:53.594 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_user": { 2024-04-25T00:26:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_user", 2024-04-25T00:26:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "User to authenticate as", 2024-04-25T00:26:53.595 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "balancer", 2024-04-25T00:26:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: "active": { 2024-04-25T00:26:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "active", 2024-04-25T00:26:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.596 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T00:26:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_time": { 2024-04-25T00:26:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_time", 2024-04-25T00:26:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.597 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0000", 2024-04-25T00:26:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T00:26:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:26:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_weekday": { 2024-04-25T00:26:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_weekday", 2024-04-25T00:26:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-04-25T00:26:53.598 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-04-25T00:26:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-04-25T00:26:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-04-25T00:26:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T00:26:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:26:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_max_iterations": { 2024-04-25T00:26:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_max_iterations", 2024-04-25T00:26:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-04-25T00:26:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.599 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "25", 2024-04-25T00:26:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-04-25T00:26:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "250", 2024-04-25T00:26:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T00:26:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_metrics": { 2024-04-25T00:26:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_metrics", 2024-04-25T00:26:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.600 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T00:26:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T00:26:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T00:26:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_step": { 2024-04-25T00:26:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_step", 2024-04-25T00:26:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-04-25T00:26:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.601 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.5", 2024-04-25T00:26:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0.001", 2024-04-25T00:26:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "0.999", 2024-04-25T00:26:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "aggressiveness of optimization", 2024-04-25T00:26:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T00:26:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: "end_time": { 2024-04-25T00:26:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_time", 2024-04-25T00:26:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.602 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2359", 2024-04-25T00:26:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T00:26:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:26:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: "end_weekday": { 2024-04-25T00:26:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_weekday", 2024-04-25T00:26:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-04-25T00:26:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.603 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-04-25T00:26:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-04-25T00:26:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-04-25T00:26:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T00:26:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:26:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.604 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.605 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.607 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.608 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: "min_score": { 2024-04-25T00:26:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_score", 2024-04-25T00:26:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-04-25T00:26:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.609 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-04-25T00:26:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T00:26:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-04-25T00:26:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-04-25T00:26:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.610 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "upmap", 2024-04-25T00:26:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-compat", 2024-04-25T00:26:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: "none", 2024-04-25T00:26:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: "read", 2024-04-25T00:26:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap", 2024-04-25T00:26:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap-read" 2024-04-25T00:26:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Balancer mode", 2024-04-25T00:26:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_ids": { 2024-04-25T00:26:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_ids", 2024-04-25T00:26:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T00:26:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.612 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-04-25T00:26:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-04-25T00:26:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-04-25T00:26:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T00:26:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_deviation": { 2024-04-25T00:26:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_deviation", 2024-04-25T00:26:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-04-25T00:26:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-04-25T00:26:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T00:26:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T00:26:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.614 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_optimizations": { 2024-04-25T00:26:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_optimizations", 2024-04-25T00:26:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-04-25T00:26:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-04-25T00:26:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T00:26:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.615 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm", 2024-04-25T00:26:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_down_multiplier": { 2024-04-25T00:26:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_down_multiplier", 2024-04-25T00:26:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-04-25T00:26:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-04-25T00:26:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.616 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T00:26:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_refresh_rate": { 2024-04-25T00:26:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_refresh_rate", 2024-04-25T00:26:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "20", 2024-04-25T00:26:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.617 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T00:26:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_starting_port": { 2024-04-25T00:26:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_starting_port", 2024-04-25T00:26:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4721", 2024-04-25T00:26:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.618 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T00:26:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_ptrace": { 2024-04-25T00:26:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_ptrace", 2024-04-25T00:26:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T00:26:53.619 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T00:26:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_interval": { 2024-04-25T00:26:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_interval", 2024-04-25T00:26:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-04-25T00:26:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T00:26:53.620 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_memory_target_ratio": { 2024-04-25T00:26:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T00:26:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-04-25T00:26:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.7", 2024-04-25T00:26:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T00:26:53.621 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm_log_destination": { 2024-04-25T00:26:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm_log_destination", 2024-04-25T00:26:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: "file", 2024-04-25T00:26:53.622 INFO:teuthology.orchestra.run.smithi107.stdout: "file,syslog", 2024-04-25T00:26:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: "syslog" 2024-04-25T00:26:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T00:26:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: "cgroups_split": { 2024-04-25T00:26:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cgroups_split", 2024-04-25T00:26:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.623 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T00:26:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "config_checks_enabled": { 2024-04-25T00:26:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_checks_enabled", 2024-04-25T00:26:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.624 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T00:26:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: "config_dashboard": { 2024-04-25T00:26:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_dashboard", 2024-04-25T00:26:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.625 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T00:26:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_alertmanager": { 2024-04-25T00:26:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_alertmanager", 2024-04-25T00:26:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T00:26:53.626 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-04-25T00:26:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_base": { 2024-04-25T00:26:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_base", 2024-04-25T00:26:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T00:26:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.627 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Container image name, without the tag", 2024-04-25T00:26:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_elasticsearch": { 2024-04-25T00:26:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_elasticsearch", 2024-04-25T00:26:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T00:26:53.628 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "elasticsearch container image", 2024-04-25T00:26:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_grafana": { 2024-04-25T00:26:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_grafana", 2024-04-25T00:26:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T00:26:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.629 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-04-25T00:26:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_haproxy": { 2024-04-25T00:26:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_haproxy", 2024-04-25T00:26:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T00:26:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.630 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "HAproxy container image", 2024-04-25T00:26:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_agent": { 2024-04-25T00:26:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_agent", 2024-04-25T00:26:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T00:26:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.631 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger agent container image", 2024-04-25T00:26:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_collector": { 2024-04-25T00:26:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_collector", 2024-04-25T00:26:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T00:26:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.632 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger collector container image", 2024-04-25T00:26:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_query": { 2024-04-25T00:26:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_query", 2024-04-25T00:26:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T00:26:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.633 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger query container image", 2024-04-25T00:26:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_keepalived": { 2024-04-25T00:26:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_keepalived", 2024-04-25T00:26:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T00:26:53.634 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Keepalived container image", 2024-04-25T00:26:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_loki": { 2024-04-25T00:26:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_loki", 2024-04-25T00:26:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T00:26:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.635 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Loki container image", 2024-04-25T00:26:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_node_exporter": { 2024-04-25T00:26:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_node_exporter", 2024-04-25T00:26:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T00:26:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.636 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-04-25T00:26:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_nvmeof": { 2024-04-25T00:26:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_nvmeof", 2024-04-25T00:26:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T00:26:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.637 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Nvme-of container image", 2024-04-25T00:26:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_prometheus": { 2024-04-25T00:26:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_prometheus", 2024-04-25T00:26:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T00:26:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.638 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-04-25T00:26:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_promtail": { 2024-04-25T00:26:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_promtail", 2024-04-25T00:26:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T00:26:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.639 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Promtail container image", 2024-04-25T00:26:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_samba": { 2024-04-25T00:26:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_samba", 2024-04-25T00:26:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T00:26:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.640 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Samba/SMB container image", 2024-04-25T00:26:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_snmp_gateway": { 2024-04-25T00:26:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_snmp_gateway", 2024-04-25T00:26:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T00:26:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SNMP Gateway container image", 2024-04-25T00:26:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.641 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "container_init": { 2024-04-25T00:26:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_init", 2024-04-25T00:26:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T00:26:53.642 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: "daemon_cache_timeout": { 2024-04-25T00:26:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "daemon_cache_timeout", 2024-04-25T00:26:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-04-25T00:26:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.643 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T00:26:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "default_cephadm_command_timeout": { 2024-04-25T00:26:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T00:26:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "900", 2024-04-25T00:26:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.644 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T00:26:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: "default_registry": { 2024-04-25T00:26:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_registry", 2024-04-25T00:26:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io", 2024-04-25T00:26:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.645 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T00:26:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "device_cache_timeout": { 2024-04-25T00:26:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_cache_timeout", 2024-04-25T00:26:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1800", 2024-04-25T00:26:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.646 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache device inventory", 2024-04-25T00:26:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: "device_enhanced_scan": { 2024-04-25T00:26:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_enhanced_scan", 2024-04-25T00:26:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.647 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.648 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T00:26:53.648 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.648 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.648 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.648 INFO:teuthology.orchestra.run.smithi107.stdout: "facts_cache_timeout": { 2024-04-25T00:26:53.648 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "facts_cache_timeout", 2024-04-25T00:26:53.648 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.648 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.648 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.648 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-04-25T00:26:53.648 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.649 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.649 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.649 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache host facts data", 2024-04-25T00:26:53.649 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.649 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.649 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.649 INFO:teuthology.orchestra.run.smithi107.stdout: "grafana_dashboards_path": { 2024-04-25T00:26:53.649 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "grafana_dashboards_path", 2024-04-25T00:26:53.649 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.649 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.649 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.649 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T00:26:53.650 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.650 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.650 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.650 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T00:26:53.650 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.650 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.650 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.650 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.650 INFO:teuthology.orchestra.run.smithi107.stdout: "host_check_interval": { 2024-04-25T00:26:53.650 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "host_check_interval", 2024-04-25T00:26:53.650 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.650 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.650 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.651 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-04-25T00:26:53.651 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.651 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.651 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.651 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to perform a host check", 2024-04-25T00:26:53.651 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.651 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.651 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.651 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.651 INFO:teuthology.orchestra.run.smithi107.stdout: "hw_monitoring": { 2024-04-25T00:26:53.651 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hw_monitoring", 2024-04-25T00:26:53.651 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.651 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.651 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.652 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.652 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.652 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.652 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.652 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T00:26:53.652 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.652 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.652 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.652 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.652 INFO:teuthology.orchestra.run.smithi107.stdout: "inventory_list_all": { 2024-04-25T00:26:53.652 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "inventory_list_all", 2024-04-25T00:26:53.652 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.652 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.653 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.653 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.653 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.653 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.653 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.653 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T00:26:53.653 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.653 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.653 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.653 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.653 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.653 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.653 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.654 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.654 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.654 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.654 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.654 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.654 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.654 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.654 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.654 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.654 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.654 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.654 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.654 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.654 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.655 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.655 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.655 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.655 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.655 INFO:teuthology.orchestra.run.smithi107.stdout: "log_refresh_metadata": { 2024-04-25T00:26:53.655 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_refresh_metadata", 2024-04-25T00:26:53.655 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.655 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.655 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.655 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.655 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.655 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.655 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.656 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T00:26:53.656 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.656 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.656 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.656 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.656 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.656 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.656 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.656 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.656 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.656 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.656 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.657 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.657 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T00:26:53.657 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.657 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.657 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.657 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.657 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.657 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.657 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.657 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.657 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.657 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.657 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.657 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.658 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.658 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.658 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.658 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.658 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.658 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.658 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.658 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.658 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.658 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.658 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.658 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.658 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.659 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.659 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.659 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.659 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.659 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.659 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.659 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.659 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.659 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.659 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.659 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.659 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.659 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T00:26:53.659 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T00:26:53.660 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.660 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.660 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.660 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.660 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.660 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.660 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.660 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T00:26:53.660 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.660 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.660 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.660 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.660 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T00:26:53.660 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T00:26:53.661 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.661 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.661 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.661 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "*", 2024-04-25T00:26:53.661 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.661 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.661 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.661 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T00:26:53.661 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.661 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.661 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.661 INFO:teuthology.orchestra.run.smithi107.stdout: "max_count_per_host": { 2024-04-25T00:26:53.661 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_count_per_host", 2024-04-25T00:26:53.662 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.662 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.662 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.662 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-04-25T00:26:53.662 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.662 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.662 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.662 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of daemons per service per host", 2024-04-25T00:26:53.662 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.662 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.662 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.662 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd_draining_count": { 2024-04-25T00:26:53.663 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_osd_draining_count", 2024-04-25T00:26:53.663 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.663 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.663 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.663 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-04-25T00:26:53.663 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.663 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.663 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.663 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T00:26:53.663 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.663 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.663 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.663 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.663 INFO:teuthology.orchestra.run.smithi107.stdout: "migration_current": { 2024-04-25T00:26:53.663 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "migration_current", 2024-04-25T00:26:53.664 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.664 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.664 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.664 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.664 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.664 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.664 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.664 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "internal - do not modify", 2024-04-25T00:26:53.664 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.664 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.664 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.664 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.664 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-04-25T00:26:53.664 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-04-25T00:26:53.665 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.665 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.665 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.665 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "root", 2024-04-25T00:26:53.665 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.665 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.665 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.665 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm-package", 2024-04-25T00:26:53.665 INFO:teuthology.orchestra.run.smithi107.stdout: "root" 2024-04-25T00:26:53.665 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.665 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T00:26:53.665 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.665 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.666 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.666 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.666 INFO:teuthology.orchestra.run.smithi107.stdout: "oob_default_addr": { 2024-04-25T00:26:53.666 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "oob_default_addr", 2024-04-25T00:26:53.666 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.666 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.666 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.666 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "169.254.1.1", 2024-04-25T00:26:53.666 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.666 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.666 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.666 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T00:26:53.666 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.666 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.666 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.667 INFO:teuthology.orchestra.run.smithi107.stdout: "prometheus_alerts_path": { 2024-04-25T00:26:53.667 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus_alerts_path", 2024-04-25T00:26:53.667 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.667 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.667 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.667 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T00:26:53.667 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.667 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.667 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.667 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T00:26:53.667 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.667 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.667 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.668 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_insecure": { 2024-04-25T00:26:53.668 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_insecure", 2024-04-25T00:26:53.668 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.668 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.668 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.668 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.668 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.668 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.668 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.668 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T00:26:53.668 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.668 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.668 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.669 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_password": { 2024-04-25T00:26:53.669 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_password", 2024-04-25T00:26:53.669 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.669 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.669 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.669 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.669 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.669 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.669 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.669 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T00:26:53.669 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.669 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.669 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.670 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.670 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_url": { 2024-04-25T00:26:53.670 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_url", 2024-04-25T00:26:53.670 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.670 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.670 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.670 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.670 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.670 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.670 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.670 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T00:26:53.670 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.670 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.671 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.671 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_username": { 2024-04-25T00:26:53.671 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_username", 2024-04-25T00:26:53.671 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.671 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.671 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.671 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.671 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.671 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.671 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.671 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T00:26:53.671 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.671 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.672 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.672 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.672 INFO:teuthology.orchestra.run.smithi107.stdout: "secure_monitoring_stack": { 2024-04-25T00:26:53.672 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "secure_monitoring_stack", 2024-04-25T00:26:53.672 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.672 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.672 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.672 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.672 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.672 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.672 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.672 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T00:26:53.672 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.673 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.673 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.673 INFO:teuthology.orchestra.run.smithi107.stdout: "service_discovery_port": { 2024-04-25T00:26:53.673 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "service_discovery_port", 2024-04-25T00:26:53.673 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.673 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.673 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.673 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8765", 2024-04-25T00:26:53.673 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.673 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.673 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.673 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "cephadm service discovery port", 2024-04-25T00:26:53.673 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.674 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.674 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.674 INFO:teuthology.orchestra.run.smithi107.stdout: "ssh_config_file": { 2024-04-25T00:26:53.674 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssh_config_file", 2024-04-25T00:26:53.674 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.674 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.674 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.674 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.674 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.674 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.674 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.674 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T00:26:53.675 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.675 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.675 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.675 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.675 INFO:teuthology.orchestra.run.smithi107.stdout: "use_agent": { 2024-04-25T00:26:53.675 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_agent", 2024-04-25T00:26:53.675 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.675 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.675 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.675 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.675 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.675 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.675 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.675 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T00:26:53.676 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.676 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.676 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.676 INFO:teuthology.orchestra.run.smithi107.stdout: "use_repo_digest": { 2024-04-25T00:26:53.676 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_repo_digest", 2024-04-25T00:26:53.676 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.676 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.676 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.676 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.676 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.677 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.677 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.677 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T00:26:53.677 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.677 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.677 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.677 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_failed_host_check": { 2024-04-25T00:26:53.677 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_failed_host_check", 2024-04-25T00:26:53.677 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.677 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.677 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.678 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.678 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.678 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.678 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.678 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T00:26:53.678 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.678 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.678 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.678 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.678 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_daemons": { 2024-04-25T00:26:53.678 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_daemons", 2024-04-25T00:26:53.678 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.678 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.678 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.679 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.679 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.679 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.679 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.679 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T00:26:53.679 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.679 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.679 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.679 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_hosts": { 2024-04-25T00:26:53.679 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_hosts", 2024-04-25T00:26:53.679 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.679 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.679 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.680 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.680 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.680 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.680 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.680 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T00:26:53.680 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.680 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.680 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.680 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.680 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.680 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.680 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crash", 2024-04-25T00:26:53.680 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.681 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.681 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.681 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.681 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.681 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.681 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.681 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.681 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.681 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.681 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.681 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.681 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.681 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.682 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.682 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.682 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.682 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.682 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.682 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.682 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.682 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.682 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.682 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.682 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.682 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.682 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.683 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.683 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.683 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.683 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.683 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.683 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.683 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.683 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.683 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.683 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.683 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.683 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.683 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.683 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.684 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.684 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.684 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.684 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.684 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.684 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.684 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.684 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.684 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.684 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.684 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.684 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.684 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.684 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.684 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.685 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.685 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.685 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.685 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.685 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.685 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.685 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.685 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.685 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.685 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.685 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.685 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.685 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.686 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.686 INFO:teuthology.orchestra.run.smithi107.stdout: "retain_interval": { 2024-04-25T00:26:53.686 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retain_interval", 2024-04-25T00:26:53.686 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.686 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.686 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.686 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "31536000", 2024-04-25T00:26:53.686 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.686 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.686 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.686 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T00:26:53.686 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.686 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.687 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.687 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_recent_interval": { 2024-04-25T00:26:53.687 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_recent_interval", 2024-04-25T00:26:53.687 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.687 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.687 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.687 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1209600", 2024-04-25T00:26:53.687 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.687 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.687 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.687 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T00:26:53.687 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.687 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.687 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.688 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.688 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.688 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.688 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dashboard", 2024-04-25T00:26:53.688 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.688 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.688 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.688 INFO:teuthology.orchestra.run.smithi107.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T00:26:53.688 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T00:26:53.688 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.688 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.688 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.688 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-04-25T00:26:53.689 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.689 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.689 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.689 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.689 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.689 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.689 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.689 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.689 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T00:26:53.689 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T00:26:53.689 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.689 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.689 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.689 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.689 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.689 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.689 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.690 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.690 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.690 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.690 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.690 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T00:26:53.690 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T00:26:53.690 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.690 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.690 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.690 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.690 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.690 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.690 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.691 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.691 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.691 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.691 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.691 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_ENABLED": { 2024-04-25T00:26:53.691 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T00:26:53.691 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.691 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.691 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.691 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.691 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.691 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.691 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.691 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.691 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.692 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.692 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.692 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.692 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T00:26:53.692 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T00:26:53.692 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.692 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.692 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.692 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.692 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.692 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.692 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.692 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.692 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.692 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.693 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.693 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.693 INFO:teuthology.orchestra.run.smithi107.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T00:26:53.693 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T00:26:53.693 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.693 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.693 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.693 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.693 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.693 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.693 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.693 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.693 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.693 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.694 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.694 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.694 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T00:26:53.694 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T00:26:53.694 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.694 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.694 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.694 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.694 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.694 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.694 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.694 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.694 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.694 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.694 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.695 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T00:26:53.695 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T00:26:53.695 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.695 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.695 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.695 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.695 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.695 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.695 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.695 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.695 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.695 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.695 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.696 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T00:26:53.696 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T00:26:53.696 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.696 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.696 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.696 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.696 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.696 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.696 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.696 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.696 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.696 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.696 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.696 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.696 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T00:26:53.697 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T00:26:53.697 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.697 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.697 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.697 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.697 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.697 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.697 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.697 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.697 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.697 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.697 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.697 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.697 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T00:26:53.697 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T00:26:53.698 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.698 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.698 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.698 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.698 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.698 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.698 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.698 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.698 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.698 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.698 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.698 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.698 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T00:26:53.698 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T00:26:53.698 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.699 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.699 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.699 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.699 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.699 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.699 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.699 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.699 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.699 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.699 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.699 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.699 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T00:26:53.699 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T00:26:53.699 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.699 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.699 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.699 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.700 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.700 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.700 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.700 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.700 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.700 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.700 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.700 INFO:teuthology.orchestra.run.smithi107.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T00:26:53.700 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T00:26:53.700 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.700 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.700 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.700 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.700 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.700 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.700 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.701 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.701 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.701 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.701 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.701 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.701 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T00:26:53.701 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T00:26:53.701 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.701 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.701 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.701 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-04-25T00:26:53.701 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.701 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.701 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.701 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.702 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.702 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.702 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.702 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T00:26:53.702 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T00:26:53.702 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.702 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.702 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.702 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.702 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.702 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.702 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.702 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.703 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.703 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.703 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.703 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_URL": { 2024-04-25T00:26:53.703 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_URL", 2024-04-25T00:26:53.703 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.703 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.703 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.703 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.703 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.703 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.703 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.703 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.703 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.703 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.704 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.704 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T00:26:53.704 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T00:26:53.704 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.704 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.704 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.704 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-04-25T00:26:53.704 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.704 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.704 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.704 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.704 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.704 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.704 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.705 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T00:26:53.705 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T00:26:53.705 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.705 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.705 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.705 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.705 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.705 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.705 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.705 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.705 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.705 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.705 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.705 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.705 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T00:26:53.705 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T00:26:53.706 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.706 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.706 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.706 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.706 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.706 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.706 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.706 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.706 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.706 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.706 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.706 INFO:teuthology.orchestra.run.smithi107.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T00:26:53.707 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T00:26:53.707 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.707 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.707 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.707 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.707 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.707 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.707 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.707 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.707 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.707 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.707 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.707 INFO:teuthology.orchestra.run.smithi107.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T00:26:53.707 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T00:26:53.707 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.708 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.708 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.708 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.708 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.708 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.708 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.708 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.708 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.708 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.708 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.708 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.708 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T00:26:53.708 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T00:26:53.709 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.709 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.709 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.709 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.709 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.709 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.709 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.709 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.709 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.709 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.709 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.710 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T00:26:53.710 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T00:26:53.710 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.710 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.710 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.710 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.710 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.710 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.710 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.710 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.710 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.710 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.710 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.711 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T00:26:53.711 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T00:26:53.711 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.711 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.711 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.711 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.711 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.711 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.711 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.711 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.711 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.711 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.711 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.712 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T00:26:53.712 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T00:26:53.712 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.712 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.712 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.712 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.712 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.712 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.712 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.712 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.712 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.712 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.712 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.713 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T00:26:53.713 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T00:26:53.713 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.713 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.713 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.713 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.713 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.713 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.713 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.713 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.713 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.713 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.713 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.714 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T00:26:53.714 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T00:26:53.714 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.714 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.714 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.714 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.714 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.714 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.714 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.714 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.714 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.714 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.714 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.715 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T00:26:53.715 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T00:26:53.715 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.715 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.715 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.715 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.715 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.715 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.715 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.715 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.715 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.715 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.715 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.716 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T00:26:53.716 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T00:26:53.716 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.716 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.716 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.716 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.716 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.716 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.716 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.716 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.716 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.716 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.716 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.717 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T00:26:53.717 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T00:26:53.717 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.717 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.717 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.717 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.717 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.717 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.717 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.717 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.717 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.717 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.717 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.718 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T00:26:53.718 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T00:26:53.718 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.718 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.718 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.718 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.718 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.718 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.718 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.718 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.718 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.718 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.718 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.719 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T00:26:53.719 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T00:26:53.719 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.719 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.719 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.719 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T00:26:53.719 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.719 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.719 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.719 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.719 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.719 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.720 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.720 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T00:26:53.720 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T00:26:53.720 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.720 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.720 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.720 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-04-25T00:26:53.720 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.720 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.720 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.720 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.720 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.721 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.721 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.721 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T00:26:53.721 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T00:26:53.721 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.721 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.721 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.721 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8", 2024-04-25T00:26:53.721 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.721 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.721 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.721 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.721 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.721 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.721 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.722 INFO:teuthology.orchestra.run.smithi107.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T00:26:53.722 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T00:26:53.722 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.722 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.722 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.722 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "45", 2024-04-25T00:26:53.722 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.722 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.722 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.722 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.722 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.723 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.723 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.723 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.723 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T00:26:53.723 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T00:26:53.723 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.723 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.723 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.723 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.723 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.723 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.723 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.723 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.723 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.723 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.724 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.724 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T00:26:53.724 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T00:26:53.724 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.724 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.724 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.724 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-04-25T00:26:53.724 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.724 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.724 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.724 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.724 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.724 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.724 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.724 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.725 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T00:26:53.725 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T00:26:53.725 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.725 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.725 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.725 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.725 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.725 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.725 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.725 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.725 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.725 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.725 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.726 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T00:26:53.726 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T00:26:53.726 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.726 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.726 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.726 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.726 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.726 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.726 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.726 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.726 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.726 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.726 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.727 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T00:26:53.727 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T00:26:53.727 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.727 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.727 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.727 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-04-25T00:26:53.727 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.727 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.727 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.727 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.727 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.727 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.727 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.727 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.728 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T00:26:53.728 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T00:26:53.728 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.728 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.728 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.728 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-04-25T00:26:53.728 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.728 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.728 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.728 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.728 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.728 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.728 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.728 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.728 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T00:26:53.729 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T00:26:53.729 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.729 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.729 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.729 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-04-25T00:26:53.729 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.729 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.729 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.729 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.729 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.729 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.729 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.729 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.729 INFO:teuthology.orchestra.run.smithi107.stdout: "cross_origin_url": { 2024-04-25T00:26:53.730 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cross_origin_url", 2024-04-25T00:26:53.730 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.730 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.730 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.730 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.730 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.730 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.730 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.730 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.730 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.730 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.730 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.730 INFO:teuthology.orchestra.run.smithi107.stdout: "crt_file": { 2024-04-25T00:26:53.731 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crt_file", 2024-04-25T00:26:53.731 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.731 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.731 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.731 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.731 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.731 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.731 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.731 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.731 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.731 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.731 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.731 INFO:teuthology.orchestra.run.smithi107.stdout: "debug": { 2024-04-25T00:26:53.731 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "debug", 2024-04-25T00:26:53.731 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.732 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.732 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.732 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.732 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.732 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.732 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.732 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable/disable debug options", 2024-04-25T00:26:53.732 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.732 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.732 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.732 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.732 INFO:teuthology.orchestra.run.smithi107.stdout: "jwt_token_ttl": { 2024-04-25T00:26:53.732 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "jwt_token_ttl", 2024-04-25T00:26:53.732 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.733 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.733 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.733 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "28800", 2024-04-25T00:26:53.733 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.733 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.733 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.733 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.733 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.733 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.733 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.733 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-04-25T00:26:53.733 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-04-25T00:26:53.733 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.733 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.733 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.734 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.734 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.734 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.734 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.734 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.734 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.734 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.734 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.734 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.734 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.734 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.734 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.734 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.734 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.735 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.735 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.735 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.735 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.735 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.735 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.735 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.735 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.735 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.735 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.735 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.735 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.735 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.735 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.736 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.736 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.736 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.736 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.736 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.736 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.736 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.736 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.736 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.736 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.736 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.736 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.736 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.736 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.737 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.737 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.737 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.737 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.737 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.737 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.737 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.737 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.737 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.737 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.737 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.737 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.737 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.737 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.738 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.738 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.738 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.738 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.738 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.738 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.738 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.738 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.738 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.738 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.738 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.738 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.738 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.738 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.738 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "motd": { 2024-04-25T00:26:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "motd", 2024-04-25T00:26:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.739 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "The message of the day", 2024-04-25T00:26:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect_resolve_ip_addr": { 2024-04-25T00:26:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T00:26:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.740 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-04-25T00:26:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-04-25T00:26:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-04-25T00:26:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.741 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-04-25T00:26:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-04-25T00:26:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8080", 2024-04-25T00:26:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.742 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-04-25T00:26:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-04-25T00:26:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.743 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl_server_port": { 2024-04-25T00:26:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl_server_port", 2024-04-25T00:26:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8443", 2024-04-25T00:26:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.744 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-04-25T00:26:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-04-25T00:26:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "redirect", 2024-04-25T00:26:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect" 2024-04-25T00:26:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.745 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-04-25T00:26:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-04-25T00:26:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-04-25T00:26:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-04-25T00:26:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-04-25T00:26:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.746 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "url_prefix": { 2024-04-25T00:26:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url_prefix", 2024-04-25T00:26:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.747 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "devicehealth", 2024-04-25T00:26:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_monitoring": { 2024-04-25T00:26:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_monitoring", 2024-04-25T00:26:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.748 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "monitor device health metrics", 2024-04-25T00:26:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.749 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.750 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.751 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.752 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.753 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "mark_out_threshold": { 2024-04-25T00:26:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mark_out_threshold", 2024-04-25T00:26:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2419200", 2024-04-25T00:26:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.754 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T00:26:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": { 2024-04-25T00:26:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_name", 2024-04-25T00:26:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "device_health_metrics", 2024-04-25T00:26:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.755 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T00:26:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "retention_period": { 2024-04-25T00:26:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retention_period", 2024-04-25T00:26:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15552000", 2024-04-25T00:26:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.756 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain device health metrics", 2024-04-25T00:26:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_frequency": { 2024-04-25T00:26:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_frequency", 2024-04-25T00:26:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-04-25T00:26:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.757 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T00:26:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "self_heal": { 2024-04-25T00:26:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "self_heal", 2024-04-25T00:26:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T00:26:53.758 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-04-25T00:26:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-04-25T00:26:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-04-25T00:26:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T00:26:53.759 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_threshold": { 2024-04-25T00:26:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_threshold", 2024-04-25T00:26:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7257600", 2024-04-25T00:26:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T00:26:53.760 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "diskprediction_local", 2024-04-25T00:26:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.761 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.762 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.763 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.764 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.765 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: "predict_interval": { 2024-04-25T00:26:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predict_interval", 2024-04-25T00:26:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.766 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-04-25T00:26:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: "predictor_model": { 2024-04-25T00:26:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predictor_model", 2024-04-25T00:26:53.767 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "prophetstor", 2024-04-25T00:26:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-04-25T00:26:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-04-25T00:26:53.768 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-04-25T00:26:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.769 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "influx", 2024-04-25T00:26:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": false, 2024-04-25T00:26:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "influxdb python module not found", 2024-04-25T00:26:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: "batch_size": { 2024-04-25T00:26:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "batch_size", 2024-04-25T00:26:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5000", 2024-04-25T00:26:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.770 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T00:26:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "database": { 2024-04-25T00:26:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "database", 2024-04-25T00:26:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "ceph", 2024-04-25T00:26:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T00:26:53.771 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": { 2024-04-25T00:26:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hostname", 2024-04-25T00:26:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server hostname", 2024-04-25T00:26:53.772 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-04-25T00:26:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-04-25T00:26:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "30", 2024-04-25T00:26:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "5", 2024-04-25T00:26:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T00:26:53.773 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.774 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.775 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.776 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.777 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.778 INFO:teuthology.orchestra.run.smithi107.stdout: "password": { 2024-04-25T00:26:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "password", 2024-04-25T00:26:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "password of InfluxDB server user", 2024-04-25T00:26:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: "port": { 2024-04-25T00:26:53.779 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "port", 2024-04-25T00:26:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8086", 2024-04-25T00:26:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server port", 2024-04-25T00:26:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-04-25T00:26:53.780 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-04-25T00:26:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "false", 2024-04-25T00:26:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:26:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: "threads": { 2024-04-25T00:26:53.781 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threads", 2024-04-25T00:26:53.782 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.782 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.782 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.782 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-04-25T00:26:53.782 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-04-25T00:26:53.782 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "32", 2024-04-25T00:26:53.782 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.782 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T00:26:53.783 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.783 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.783 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.783 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.783 INFO:teuthology.orchestra.run.smithi107.stdout: "username": { 2024-04-25T00:26:53.783 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "username", 2024-04-25T00:26:53.785 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.785 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.785 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.785 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.786 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.786 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.786 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.786 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "username of InfluxDB server user", 2024-04-25T00:26:53.786 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.786 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.786 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.786 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.786 INFO:teuthology.orchestra.run.smithi107.stdout: "verify_ssl": { 2024-04-25T00:26:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "verify_ssl", 2024-04-25T00:26:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "true", 2024-04-25T00:26:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:26:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.787 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "insights", 2024-04-25T00:26:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.788 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.789 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.790 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.792 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.792 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.792 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.792 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.792 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.792 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.792 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.792 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.792 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.792 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.792 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.792 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.792 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.792 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.792 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.793 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.793 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "iostat", 2024-04-25T00:26:53.793 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.793 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.793 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.793 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.793 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.793 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.793 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.793 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.793 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.793 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.793 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.793 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.794 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.795 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.796 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.797 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "k8sevents", 2024-04-25T00:26:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_event_retention_days": { 2024-04-25T00:26:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ceph_event_retention_days", 2024-04-25T00:26:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7", 2024-04-25T00:26:53.798 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T00:26:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: "config_check_secs": { 2024-04-25T00:26:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_check_secs", 2024-04-25T00:26:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-04-25T00:26:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "10", 2024-04-25T00:26:53.799 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T00:26:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.800 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.801 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.802 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.803 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.804 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "localpool", 2024-04-25T00:26:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: "failure_domain": { 2024-04-25T00:26:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "failure_domain", 2024-04-25T00:26:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.805 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "host", 2024-04-25T00:26:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "failure domain for any created local pool", 2024-04-25T00:26:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T00:26:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.806 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.807 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.808 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.809 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.810 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": { 2024-04-25T00:26:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_size", 2024-04-25T00:26:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.811 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default min_size for any created local pool", 2024-04-25T00:26:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T00:26:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "num_rep": { 2024-04-25T00:26:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "num_rep", 2024-04-25T00:26:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3", 2024-04-25T00:26:53.812 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default replica count for any created local pool", 2024-04-25T00:26:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": { 2024-04-25T00:26:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_num", 2024-04-25T00:26:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "128", 2024-04-25T00:26:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.813 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.814 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.814 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T00:26:53.814 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.814 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.814 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.814 INFO:teuthology.orchestra.run.smithi107.stdout: "prefix": { 2024-04-25T00:26:53.814 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prefix", 2024-04-25T00:26:53.814 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.814 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.814 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.814 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.814 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.814 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.814 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.815 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name prefix for any created local pool", 2024-04-25T00:26:53.815 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.815 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.815 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.815 INFO:teuthology.orchestra.run.smithi107.stdout: "subtree": { 2024-04-25T00:26:53.815 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "subtree", 2024-04-25T00:26:53.815 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.815 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.815 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.815 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "rack", 2024-04-25T00:26:53.815 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.815 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.815 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.816 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T00:26:53.816 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T00:26:53.816 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.816 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.816 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.816 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.816 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.816 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mds_autoscaler", 2024-04-25T00:26:53.816 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.816 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.816 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.816 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.816 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.816 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.817 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.817 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.817 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.817 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.817 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.817 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.817 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.817 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.817 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.817 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.817 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.817 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.817 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.817 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.817 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.818 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.818 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.818 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.818 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.818 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.818 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.818 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.818 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.818 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.818 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.818 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.818 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.818 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.819 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.819 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.819 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.819 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.819 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.819 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.819 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.819 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.819 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.819 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.819 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.819 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.819 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.819 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.820 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.820 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.820 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.820 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.820 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.820 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.820 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.820 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.820 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.820 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.820 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.820 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.820 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.820 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.821 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.821 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.821 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.821 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.821 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.821 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.821 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.821 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.821 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.821 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.821 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirroring", 2024-04-25T00:26:53.821 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.821 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.821 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.822 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.822 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.822 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.822 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.822 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.822 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.822 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.822 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.822 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.822 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.822 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.822 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.822 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.822 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.823 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.823 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.823 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.823 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.823 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.823 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.823 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.823 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.823 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.823 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.823 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.823 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.823 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.824 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.824 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.824 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.824 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.824 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.824 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.824 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.824 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.824 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.824 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.824 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.824 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.824 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.824 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.825 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.825 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.825 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.825 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.825 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.825 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.825 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.825 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.825 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.825 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.825 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.825 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.825 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.825 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.826 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.826 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.826 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.826 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.826 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.826 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.826 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.826 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.826 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.826 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.826 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.826 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.827 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.827 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.827 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "nfs", 2024-04-25T00:26:53.827 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.827 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.827 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.827 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.827 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.827 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.827 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.827 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.827 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.827 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.827 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.828 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.828 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.828 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.828 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.828 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.828 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.828 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.828 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.828 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.828 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.828 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.828 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.828 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.829 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.829 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.829 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.829 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.829 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.829 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.829 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.829 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.829 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.829 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.829 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.829 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.829 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.829 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.830 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.830 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.830 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.830 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.830 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.830 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.830 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.830 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.830 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.830 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.830 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.830 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.830 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.830 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.831 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.831 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.831 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.831 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.831 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.831 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.831 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.831 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.831 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.831 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.831 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.831 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.831 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.831 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.832 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.832 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.832 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.832 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.832 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.832 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.832 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-04-25T00:26:53.832 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.832 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.832 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.832 INFO:teuthology.orchestra.run.smithi107.stdout: "fail_fs": { 2024-04-25T00:26:53.832 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "fail_fs", 2024-04-25T00:26:53.832 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.832 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.833 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.833 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.833 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.833 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.833 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.833 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T00:26:53.833 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.833 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.833 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.833 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.833 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.833 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.833 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.833 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.834 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.834 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.834 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.834 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.834 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.834 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.834 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.834 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.834 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.834 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.834 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.834 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.834 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.834 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.835 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.835 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.835 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.835 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.835 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.835 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.835 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.835 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.835 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.835 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.835 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.835 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.835 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.835 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.835 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.836 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.836 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.836 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.836 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.836 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.836 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.836 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.836 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.836 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.836 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.836 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.836 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.836 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.836 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.836 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.837 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.837 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.837 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.837 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.837 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.837 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.837 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.837 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.837 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.837 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.837 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.837 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.837 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.837 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.838 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.838 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.838 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.838 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.838 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator": { 2024-04-25T00:26:53.838 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-04-25T00:26:53.838 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.838 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.838 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.838 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.838 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.838 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.838 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.839 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm", 2024-04-25T00:26:53.839 INFO:teuthology.orchestra.run.smithi107.stdout: "rook", 2024-04-25T00:26:53.839 INFO:teuthology.orchestra.run.smithi107.stdout: "test_orchestrator" 2024-04-25T00:26:53.839 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.839 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Orchestrator backend", 2024-04-25T00:26:53.839 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.839 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.839 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.839 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.839 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.839 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.839 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_perf_query", 2024-04-25T00:26:53.839 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.840 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.840 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.840 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.840 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.840 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.840 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.840 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.840 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.840 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.840 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.840 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.840 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.840 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.840 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.841 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.841 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.841 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.841 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.841 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.841 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.841 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.841 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.841 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.841 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.841 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.841 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.841 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.841 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.842 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.842 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.842 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.842 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.842 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.842 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.842 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.842 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.842 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.842 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.842 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.842 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.842 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.843 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.843 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.843 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.843 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.843 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.843 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.843 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.843 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.843 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.843 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.843 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.843 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.843 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.843 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.844 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.844 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.844 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.844 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.844 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.844 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.844 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.844 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.844 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.844 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.844 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.844 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.844 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.845 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.845 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.845 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.845 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_support", 2024-04-25T00:26:53.845 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.845 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.845 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.845 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.845 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.845 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.845 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.845 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.845 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.845 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.846 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.846 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.846 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.846 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.846 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.846 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.846 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.846 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.846 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.846 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.846 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.846 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.846 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.847 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.847 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.847 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.847 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.847 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.847 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.847 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.847 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.847 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.847 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.847 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.847 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.847 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.847 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.848 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.848 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.848 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.848 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.848 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.848 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.848 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.848 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.848 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.848 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.848 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.848 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.848 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.848 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.848 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.849 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.849 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.849 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.849 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.849 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.849 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.849 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.849 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.849 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.849 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.849 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.849 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.849 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.849 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.849 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.850 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.850 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.850 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.850 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.850 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.850 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_autoscaler", 2024-04-25T00:26:53.850 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.850 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.850 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.850 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.850 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.850 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.851 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.851 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.851 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.851 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.851 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.851 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.851 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.851 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.851 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.851 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.851 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.851 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.851 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.851 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.851 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.852 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.852 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.852 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.852 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.852 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.852 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.852 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.852 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.852 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.852 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.852 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.852 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.852 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.852 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.853 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.853 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.853 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.853 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.853 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.853 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.853 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.853 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.853 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.853 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.853 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.853 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.853 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.854 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.854 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.854 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.854 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.854 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.854 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.854 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.854 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.854 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.854 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.854 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.854 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.854 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.855 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.855 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.855 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.855 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.855 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.855 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.855 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.855 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.855 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.855 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.855 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-04-25T00:26:53.855 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-04-25T00:26:53.855 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.855 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.855 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.856 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-04-25T00:26:53.856 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.856 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.856 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.856 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.856 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.856 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.856 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.856 INFO:teuthology.orchestra.run.smithi107.stdout: "threshold": { 2024-04-25T00:26:53.856 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threshold", 2024-04-25T00:26:53.856 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-04-25T00:26:53.856 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.856 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.857 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-04-25T00:26:53.857 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1.0", 2024-04-25T00:26:53.857 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.857 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.857 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "scaling threshold", 2024-04-25T00:26:53.857 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T00:26:53.857 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.857 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.857 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.857 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.857 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.857 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "progress", 2024-04-25T00:26:53.857 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.857 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.858 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.858 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_pg_recovery_event": { 2024-04-25T00:26:53.858 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_pg_recovery_event", 2024-04-25T00:26:53.858 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.858 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.858 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.858 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.858 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.858 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.858 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.858 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T00:26:53.858 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.858 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.858 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.859 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-04-25T00:26:53.859 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-04-25T00:26:53.859 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.859 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.859 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.859 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.859 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.859 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.859 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.859 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.859 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.859 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.859 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.860 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.860 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.860 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.860 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.860 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.860 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.860 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.860 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.860 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.860 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.860 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.860 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.860 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.860 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.861 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.861 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.861 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.861 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.861 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.861 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.861 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.861 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.861 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.861 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.861 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.861 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.861 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.862 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.862 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.862 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.862 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.862 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.862 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.862 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.862 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.862 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.862 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.862 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.862 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.862 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.863 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.863 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.863 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.863 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.863 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.863 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.863 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.863 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.863 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.863 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.863 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.863 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.863 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.863 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.863 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.864 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.864 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.864 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.864 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.864 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.864 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.864 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.864 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.864 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.864 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.864 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.864 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.864 INFO:teuthology.orchestra.run.smithi107.stdout: "max_completed_events": { 2024-04-25T00:26:53.865 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_completed_events", 2024-04-25T00:26:53.865 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.865 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.865 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.865 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "50", 2024-04-25T00:26:53.865 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.865 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.865 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.865 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "number of past completed events to remember", 2024-04-25T00:26:53.865 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.865 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.865 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.865 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-04-25T00:26:53.866 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-04-25T00:26:53.866 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.866 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.866 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.866 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-04-25T00:26:53.866 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.866 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.866 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.866 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long the module is going to sleep", 2024-04-25T00:26:53.866 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.866 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.866 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.866 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.866 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.867 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.867 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.867 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus", 2024-04-25T00:26:53.867 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.867 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.867 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.867 INFO:teuthology.orchestra.run.smithi107.stdout: "cache": { 2024-04-25T00:26:53.867 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cache", 2024-04-25T00:26:53.867 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.867 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.867 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.867 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.867 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.867 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.867 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.868 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.868 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.868 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.868 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.868 INFO:teuthology.orchestra.run.smithi107.stdout: "exclude_perf_counters": { 2024-04-25T00:26:53.868 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "exclude_perf_counters", 2024-04-25T00:26:53.868 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.868 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.868 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.868 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.868 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.868 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.868 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.868 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T00:26:53.869 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T00:26:53.869 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.869 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.869 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.869 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.869 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.869 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.869 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.869 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.869 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.869 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.869 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.869 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.869 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.870 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.870 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.870 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.870 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.870 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.870 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.870 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.870 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.870 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.870 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.870 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.870 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.871 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.871 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.871 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.871 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.871 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.871 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.871 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.871 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.871 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.871 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.871 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.871 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.871 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.871 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.872 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.872 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.872 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.872 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.872 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.872 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.872 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.872 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.872 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.872 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.872 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.872 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.872 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.872 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.872 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.873 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.873 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.873 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.873 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.873 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.873 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.873 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.873 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.873 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.873 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.873 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.873 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.873 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.873 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.874 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools": { 2024-04-25T00:26:53.874 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools", 2024-04-25T00:26:53.874 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.874 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.874 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.874 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.874 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.874 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.874 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.874 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.874 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.874 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.874 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.875 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T00:26:53.875 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T00:26:53.875 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.875 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.875 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.875 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "300", 2024-04-25T00:26:53.875 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.875 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.875 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.875 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.875 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.875 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.875 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.875 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_interval": { 2024-04-25T00:26:53.876 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_interval", 2024-04-25T00:26:53.876 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-04-25T00:26:53.876 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.876 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.876 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15.0", 2024-04-25T00:26:53.876 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.876 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.876 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.876 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.876 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.876 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.876 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.876 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-04-25T00:26:53.876 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-04-25T00:26:53.876 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.877 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.877 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.877 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-04-25T00:26:53.877 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.877 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.877 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.877 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T00:26:53.877 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.877 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.877 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.877 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-04-25T00:26:53.877 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-04-25T00:26:53.877 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.878 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.878 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.878 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "9283", 2024-04-25T00:26:53.878 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.878 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.878 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.878 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T00:26:53.878 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.878 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.878 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.878 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.878 INFO:teuthology.orchestra.run.smithi107.stdout: "stale_cache_strategy": { 2024-04-25T00:26:53.878 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stale_cache_strategy", 2024-04-25T00:26:53.878 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.879 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.879 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.879 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "log", 2024-04-25T00:26:53.879 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.879 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.879 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.879 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.879 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.879 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.879 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.879 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-04-25T00:26:53.879 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-04-25T00:26:53.879 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.879 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.880 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.880 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "default", 2024-04-25T00:26:53.880 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.880 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.880 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.880 INFO:teuthology.orchestra.run.smithi107.stdout: "default", 2024-04-25T00:26:53.880 INFO:teuthology.orchestra.run.smithi107.stdout: "error" 2024-04-25T00:26:53.880 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.880 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.880 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.880 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.880 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.880 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-04-25T00:26:53.881 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-04-25T00:26:53.881 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.881 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.881 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.881 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-04-25T00:26:53.881 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-04-25T00:26:53.881 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-04-25T00:26:53.881 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.881 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.881 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.881 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.881 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.881 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.881 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.882 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.882 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_support", 2024-04-25T00:26:53.882 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.882 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.882 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.882 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.882 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.882 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.882 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.882 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.882 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.882 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.882 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.882 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.883 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.883 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.883 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.883 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.883 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.883 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.883 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.883 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.883 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.883 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.883 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.883 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.883 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.884 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.884 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.884 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.884 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.884 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.884 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.884 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.884 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.884 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.884 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.884 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.884 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.884 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.884 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.884 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.885 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.885 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.885 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.885 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.885 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.885 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.885 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.885 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.885 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.885 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.885 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.885 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.885 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.885 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.885 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.885 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.886 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.886 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.886 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.886 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.886 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.886 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.886 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.886 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.886 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.886 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.886 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.886 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.887 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.887 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_snap_create": { 2024-04-25T00:26:53.887 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_snap_create", 2024-04-25T00:26:53.887 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.887 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.887 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.887 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-04-25T00:26:53.887 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.887 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.887 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.887 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.887 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.887 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.887 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.888 INFO:teuthology.orchestra.run.smithi107.stdout: "mirror_snapshot_schedule": { 2024-04-25T00:26:53.888 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T00:26:53.888 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.888 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.888 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.888 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.888 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.888 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.888 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.888 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.888 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.888 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.888 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.888 INFO:teuthology.orchestra.run.smithi107.stdout: "trash_purge_schedule": { 2024-04-25T00:26:53.888 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "trash_purge_schedule", 2024-04-25T00:26:53.889 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.889 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.889 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.889 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.889 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.889 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.889 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.889 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.889 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.889 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.889 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.889 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.889 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.890 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.890 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "restful", 2024-04-25T00:26:53.890 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.890 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.890 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.890 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_auth": { 2024-04-25T00:26:53.890 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_auth", 2024-04-25T00:26:53.890 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.890 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.890 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.890 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.890 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.890 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.890 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.891 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.891 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.891 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.891 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.891 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.891 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-04-25T00:26:53.891 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-04-25T00:26:53.891 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.891 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.891 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.891 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.891 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.891 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.891 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.891 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.891 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.892 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.892 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.892 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.892 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.892 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.892 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.892 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.892 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.892 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.892 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.892 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.892 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.892 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.893 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.893 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.893 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.893 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.893 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.893 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.893 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.893 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.893 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.893 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.893 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.893 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.893 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.893 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.893 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.893 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.894 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.894 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.894 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.894 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.894 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.894 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.894 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.894 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.894 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.894 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.894 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.894 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.894 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.895 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.895 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.895 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.895 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.895 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.895 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.895 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.895 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.895 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.895 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.895 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.895 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.895 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.895 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.895 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.896 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.896 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.896 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.896 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.896 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.896 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.896 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.896 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.896 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.896 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.896 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.896 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.896 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.896 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-04-25T00:26:53.896 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-04-25T00:26:53.896 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.896 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.897 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.897 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.897 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.897 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.897 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.897 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.897 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.897 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.897 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.897 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-04-25T00:26:53.897 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-04-25T00:26:53.897 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.897 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.897 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.897 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.897 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.898 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.898 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.898 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.898 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.898 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.898 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.898 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.898 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.898 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.898 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rgw", 2024-04-25T00:26:53.898 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.898 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.898 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.899 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.899 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.899 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.899 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.899 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.899 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.899 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.899 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.899 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.899 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.899 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.899 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.899 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.899 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.899 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.899 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.900 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.900 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.900 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.900 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.900 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.900 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.900 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.900 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.900 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.900 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.900 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.900 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.900 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.901 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.901 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.901 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.901 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.901 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.901 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.901 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.901 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.901 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.901 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.901 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.901 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.901 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.901 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.902 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.902 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.902 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.902 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.902 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.902 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.902 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.902 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.902 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.902 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.902 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.902 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.902 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.903 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.903 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.903 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.903 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.903 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.903 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.903 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.903 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.903 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.903 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.903 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.903 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.903 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.904 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rook", 2024-04-25T00:26:53.904 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.904 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.904 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.904 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.904 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.904 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.904 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.904 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.904 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.904 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.904 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.904 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.904 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.904 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.904 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.905 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.905 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.905 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.905 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.905 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.905 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.905 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.905 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.905 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.905 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.905 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.905 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.905 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.906 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.906 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.906 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.906 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.906 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.906 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.906 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.906 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.906 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.906 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.906 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.906 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.906 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.906 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.907 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.907 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.907 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.907 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.907 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.907 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.907 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.907 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.907 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.907 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.907 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.907 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.907 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.907 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.908 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.908 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.908 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.908 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.908 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.908 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.908 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.908 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.908 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.908 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.908 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.908 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.908 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.908 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.909 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.909 INFO:teuthology.orchestra.run.smithi107.stdout: "storage_class": { 2024-04-25T00:26:53.909 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "storage_class", 2024-04-25T00:26:53.909 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.909 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.909 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.909 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "local", 2024-04-25T00:26:53.909 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.909 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.909 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.909 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T00:26:53.909 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.909 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.909 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.909 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.910 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.910 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.910 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.910 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "selftest", 2024-04-25T00:26:53.910 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.910 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.910 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.910 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.910 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.910 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.910 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.910 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.910 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.910 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.910 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.910 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.911 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.911 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.911 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.911 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.911 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.911 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.911 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.911 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.911 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.911 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.911 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.911 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.911 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.911 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.911 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.912 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.912 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.912 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.912 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.912 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.912 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.912 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.912 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.912 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.912 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.912 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.912 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.912 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.912 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.912 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.913 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.913 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.913 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.913 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.913 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.913 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.913 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.913 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.913 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.913 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.913 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.913 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.913 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.913 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.914 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.914 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.914 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.914 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.914 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.914 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.914 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.914 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.914 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.914 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.914 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.914 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.914 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: "roption1": { 2024-04-25T00:26:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption1", 2024-04-25T00:26:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: "roption2": { 2024-04-25T00:26:53.916 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption2", 2024-04-25T00:26:53.916 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.916 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.916 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.916 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "xyz", 2024-04-25T00:26:53.916 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.916 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.916 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.916 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.916 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.916 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.916 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.916 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption1": { 2024-04-25T00:26:53.916 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption1", 2024-04-25T00:26:53.916 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.917 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.917 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.917 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.917 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.917 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.917 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.917 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.917 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.917 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.917 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.917 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.917 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption2": { 2024-04-25T00:26:53.917 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption2", 2024-04-25T00:26:53.917 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.917 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.917 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.918 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.918 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.918 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.918 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.918 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.918 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.918 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.918 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.918 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.918 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption3": { 2024-04-25T00:26:53.918 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption3", 2024-04-25T00:26:53.918 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-04-25T00:26:53.918 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.918 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.919 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.919 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.919 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.919 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.919 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.919 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.919 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.919 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.919 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption4": { 2024-04-25T00:26:53.919 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption4", 2024-04-25T00:26:53.919 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.919 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.919 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.919 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.920 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.920 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.920 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.920 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.920 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.920 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.920 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.920 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.920 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption5": { 2024-04-25T00:26:53.920 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption5", 2024-04-25T00:26:53.920 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.920 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.920 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.920 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.920 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.921 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.921 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.921 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.921 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.921 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.921 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.921 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.921 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption6": { 2024-04-25T00:26:53.921 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption6", 2024-04-25T00:26:53.921 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.921 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.921 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.921 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.921 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.922 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.922 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.922 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.922 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.922 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.922 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.922 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.922 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption7": { 2024-04-25T00:26:53.922 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption7", 2024-04-25T00:26:53.922 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.922 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.922 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.922 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.922 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-04-25T00:26:53.923 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "42", 2024-04-25T00:26:53.923 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.923 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.923 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.923 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.923 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.923 INFO:teuthology.orchestra.run.smithi107.stdout: "testkey": { 2024-04-25T00:26:53.923 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testkey", 2024-04-25T00:26:53.923 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.923 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.923 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.923 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.924 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.924 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.924 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.924 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.924 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.924 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.924 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.924 INFO:teuthology.orchestra.run.smithi107.stdout: "testlkey": { 2024-04-25T00:26:53.924 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testlkey", 2024-04-25T00:26:53.924 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.924 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.924 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.924 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.924 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.925 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.925 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.925 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.925 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.925 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.925 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.925 INFO:teuthology.orchestra.run.smithi107.stdout: "testnewline": { 2024-04-25T00:26:53.925 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testnewline", 2024-04-25T00:26:53.925 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.925 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.925 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.925 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.925 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.925 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.926 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.926 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.926 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.926 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.926 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.926 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.926 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.926 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.926 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2024-04-25T00:26:53.926 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.926 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.926 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.926 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_m_granularity": { 2024-04-25T00:26:53.926 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_m_granularity", 2024-04-25T00:26:53.927 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.927 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.927 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.927 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.927 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.927 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.927 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.927 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T00:26:53.927 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.927 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.927 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.927 INFO:teuthology.orchestra.run.smithi107.stdout: "dump_on_update": { 2024-04-25T00:26:53.927 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dump_on_update", 2024-04-25T00:26:53.927 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.928 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.928 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.928 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.928 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.928 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.928 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.928 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "dump database to debug log on update", 2024-04-25T00:26:53.928 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.928 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.928 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.928 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.928 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.928 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.928 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.929 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.930 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.930 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.930 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.930 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.930 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.930 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.930 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.930 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.930 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.930 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.930 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.930 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.930 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.930 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.931 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.932 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stats", 2024-04-25T00:26:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.933 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.934 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.935 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.936 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.937 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "status", 2024-04-25T00:26:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.938 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.939 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.940 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.940 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.940 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.940 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.940 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.940 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.940 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.940 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.940 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.940 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.940 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.940 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.940 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.941 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.941 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.941 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.941 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.941 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.941 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.941 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.941 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.941 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.941 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.941 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.941 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.941 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.942 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.942 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.942 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.942 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.942 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.942 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.942 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.942 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.942 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.942 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.942 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.942 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.942 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.942 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.943 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.943 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.943 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.943 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.943 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.943 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.943 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.943 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.943 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.943 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.943 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.943 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.943 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.943 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.944 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.944 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.944 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telegraf", 2024-04-25T00:26:53.944 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.944 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.944 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.944 INFO:teuthology.orchestra.run.smithi107.stdout: "address": { 2024-04-25T00:26:53.944 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "address", 2024-04-25T00:26:53.944 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.944 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.944 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.944 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T00:26:53.944 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.944 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.944 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.945 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.945 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.945 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.945 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.945 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-04-25T00:26:53.945 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-04-25T00:26:53.945 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.945 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.945 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.945 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15", 2024-04-25T00:26:53.945 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.945 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.945 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.946 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.946 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.946 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.946 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.946 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.946 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.946 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.946 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.946 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.946 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.946 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.946 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.946 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.947 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.947 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.947 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.947 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.947 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.947 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.947 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.947 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.947 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.947 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.947 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.947 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.947 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.947 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.948 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.948 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.948 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.948 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.948 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.948 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.948 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.948 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.948 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.948 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.948 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.948 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.948 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.948 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.949 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.949 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.949 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.949 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.949 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.949 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.949 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.949 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.949 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.949 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.949 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.949 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.949 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.949 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.949 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.949 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.950 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.950 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.950 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.950 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.950 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.950 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.950 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.950 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.950 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.950 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.950 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.950 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.950 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.951 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.951 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.951 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.951 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telemetry", 2024-04-25T00:26:53.951 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.951 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.951 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.951 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_basic": { 2024-04-25T00:26:53.951 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_basic", 2024-04-25T00:26:53.951 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.951 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.951 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.951 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.951 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.952 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.952 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.952 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T00:26:53.952 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.952 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.952 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.952 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_crash": { 2024-04-25T00:26:53.952 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_crash", 2024-04-25T00:26:53.952 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.952 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.952 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.952 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.952 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.952 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.953 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.953 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T00:26:53.953 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.953 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.953 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.953 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_device": { 2024-04-25T00:26:53.953 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_device", 2024-04-25T00:26:53.953 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.953 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.953 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.953 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.953 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.953 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.954 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.954 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T00:26:53.954 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.954 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.954 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.954 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_ident": { 2024-04-25T00:26:53.954 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_ident", 2024-04-25T00:26:53.954 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.954 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.954 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.954 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.954 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.954 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.955 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.955 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T00:26:53.955 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.955 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.955 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.955 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_perf": { 2024-04-25T00:26:53.955 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_perf", 2024-04-25T00:26:53.955 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.955 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.955 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.955 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.955 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.955 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.956 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.956 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T00:26:53.956 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.956 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.956 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.956 INFO:teuthology.orchestra.run.smithi107.stdout: "contact": { 2024-04-25T00:26:53.956 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "contact", 2024-04-25T00:26:53.956 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.956 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.956 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.956 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.956 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.956 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.956 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.957 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.957 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.957 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.957 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.957 INFO:teuthology.orchestra.run.smithi107.stdout: "description": { 2024-04-25T00:26:53.957 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "description", 2024-04-25T00:26:53.957 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.957 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.957 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.957 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.957 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.957 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.957 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.957 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.958 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.958 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.958 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.958 INFO:teuthology.orchestra.run.smithi107.stdout: "device_url": { 2024-04-25T00:26:53.958 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_url", 2024-04-25T00:26:53.958 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.958 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.958 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.958 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T00:26:53.958 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.958 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.958 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.958 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.959 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.959 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.959 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.959 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-04-25T00:26:53.959 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-04-25T00:26:53.959 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.959 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.959 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.959 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.959 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.959 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.959 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.959 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.959 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.960 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.960 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.960 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-04-25T00:26:53.960 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-04-25T00:26:53.960 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.960 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.960 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.960 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "24", 2024-04-25T00:26:53.960 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "8", 2024-04-25T00:26:53.960 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.960 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.960 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.960 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.960 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.961 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.961 INFO:teuthology.orchestra.run.smithi107.stdout: "last_opt_revision": { 2024-04-25T00:26:53.961 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "last_opt_revision", 2024-04-25T00:26:53.961 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.961 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.961 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.961 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1", 2024-04-25T00:26:53.961 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.961 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.961 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.961 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.961 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.961 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.962 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.962 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard": { 2024-04-25T00:26:53.962 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard", 2024-04-25T00:26:53.962 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.962 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.962 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.962 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.962 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.962 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.962 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.962 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.962 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.962 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.963 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.963 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard_description": { 2024-04-25T00:26:53.963 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard_description", 2024-04-25T00:26:53.963 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.963 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.963 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.963 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.963 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.963 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.963 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.963 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.963 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.963 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.963 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.964 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.964 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.964 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.964 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.964 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.964 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.964 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.964 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.964 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.964 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.964 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.964 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.964 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.964 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.965 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.965 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.965 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.965 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.965 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.965 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.965 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.965 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.965 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.965 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.965 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.965 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.965 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.966 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.966 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.966 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.966 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.966 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.966 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.966 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.966 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.966 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.966 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.966 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.966 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.966 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.967 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.967 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.967 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.967 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.967 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.967 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.967 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.967 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.967 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.967 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.967 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.967 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.967 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.967 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.968 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.968 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.968 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.968 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.968 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.968 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.968 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.968 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.968 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.968 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.968 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.968 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.968 INFO:teuthology.orchestra.run.smithi107.stdout: "organization": { 2024-04-25T00:26:53.968 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "organization", 2024-04-25T00:26:53.969 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.969 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.969 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.969 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.969 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.969 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.969 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.969 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.969 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.969 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.969 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.969 INFO:teuthology.orchestra.run.smithi107.stdout: "proxy": { 2024-04-25T00:26:53.969 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "proxy", 2024-04-25T00:26:53.970 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.970 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.970 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.970 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.970 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.970 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.970 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.970 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.970 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.970 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.970 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.970 INFO:teuthology.orchestra.run.smithi107.stdout: "url": { 2024-04-25T00:26:53.970 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url", 2024-04-25T00:26:53.971 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.971 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.971 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.971 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T00:26:53.971 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.971 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.971 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.971 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.971 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.971 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.971 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.971 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.971 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.971 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.972 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "test_orchestrator", 2024-04-25T00:26:53.972 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.972 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.972 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.972 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.972 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.972 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.972 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.972 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.972 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.972 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.972 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.972 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.972 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.973 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.973 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.973 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.973 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.973 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.973 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.973 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.973 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.973 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.973 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.973 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.973 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.973 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.973 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.974 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.974 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.974 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.974 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.974 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.974 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.974 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.974 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.974 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.974 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.974 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.974 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.974 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.975 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.975 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.975 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.975 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.975 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.975 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.975 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.975 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.975 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.975 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.975 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.975 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.975 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.975 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.975 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.976 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.976 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.976 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.976 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.976 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.976 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.976 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.976 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.976 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.976 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.976 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.976 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.976 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.977 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.977 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.977 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.977 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.977 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "volumes", 2024-04-25T00:26:53.977 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.977 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.977 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.977 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.977 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.977 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.977 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.977 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.977 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.978 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.978 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.978 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.978 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.978 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.978 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.978 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.978 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.978 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.978 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.978 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.978 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.978 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.978 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.979 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.979 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.979 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.979 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.979 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.979 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.979 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.979 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.979 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.979 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.979 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.979 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.979 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.980 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.980 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.980 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.980 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.980 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.980 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.980 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.980 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.980 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.980 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.980 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.980 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.980 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.980 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.981 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.981 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.981 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.981 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.981 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.981 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.981 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.981 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.981 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.981 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.981 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.981 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.981 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.981 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.981 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.981 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.981 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.982 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.982 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.982 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.982 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_clones": { 2024-04-25T00:26:53.982 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_clones", 2024-04-25T00:26:53.982 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.982 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.982 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.982 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4", 2024-04-25T00:26:53.982 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.982 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.982 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.983 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T00:26:53.983 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.983 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.983 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.983 INFO:teuthology.orchestra.run.smithi107.stdout: "periodic_async_work": { 2024-04-25T00:26:53.983 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "periodic_async_work", 2024-04-25T00:26:53.983 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.983 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.983 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.983 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.983 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.984 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.984 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.984 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Periodically check for async work", 2024-04-25T00:26:53.984 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.984 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.984 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.984 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.984 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_delay": { 2024-04-25T00:26:53.985 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_delay", 2024-04-25T00:26:53.985 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.985 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.985 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.985 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-04-25T00:26:53.985 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.985 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.985 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.985 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T00:26:53.985 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.986 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.986 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.986 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_no_wait": { 2024-04-25T00:26:53.986 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T00:26:53.986 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.986 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.986 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.986 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-04-25T00:26:53.986 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.986 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.986 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.986 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T00:26:53.986 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.987 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.987 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.987 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.987 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.987 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:53.987 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix", 2024-04-25T00:26:53.987 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-04-25T00:26:53.987 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-04-25T00:26:53.987 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-04-25T00:26:53.987 INFO:teuthology.orchestra.run.smithi107.stdout: "discovery_interval": { 2024-04-25T00:26:53.987 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "discovery_interval", 2024-04-25T00:26:53.987 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-04-25T00:26:53.987 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.988 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.988 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "100", 2024-04-25T00:26:53.988 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.988 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.988 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.988 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.988 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.988 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.988 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.988 INFO:teuthology.orchestra.run.smithi107.stdout: "identifier": { 2024-04-25T00:26:53.988 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "identifier", 2024-04-25T00:26:53.988 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.988 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.988 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.989 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.989 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.989 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.989 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.989 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.989 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.989 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.989 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.989 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-04-25T00:26:53.989 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-04-25T00:26:53.989 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-04-25T00:26:53.989 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.989 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.989 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-04-25T00:26:53.990 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.990 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.990 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.990 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.990 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.990 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.990 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.990 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-04-25T00:26:53.990 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-04-25T00:26:53.990 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.990 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.990 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.990 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.991 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.991 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.991 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.991 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.991 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.991 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.991 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.991 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.991 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.991 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.991 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.991 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.991 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.991 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.991 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-04-25T00:26:53.992 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-04-25T00:26:53.992 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.992 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.992 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.992 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.992 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.992 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.992 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.992 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.992 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.992 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.992 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.992 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-04-25T00:26:53.992 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-04-25T00:26:53.993 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.993 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.993 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.993 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-04-25T00:26:53.993 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.993 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.993 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-04-25T00:26:53.993 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-04-25T00:26:53.993 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-04-25T00:26:53.993 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-04-25T00:26:53.993 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-04-25T00:26:53.993 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-04-25T00:26:53.993 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-04-25T00:26:53.993 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.993 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.994 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.994 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.994 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.994 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-04-25T00:26:53.994 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-04-25T00:26:53.994 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-04-25T00:26:53.994 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.994 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-04-25T00:26:53.994 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-04-25T00:26:53.994 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.994 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.994 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.994 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.994 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.995 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.995 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.995 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_host": { 2024-04-25T00:26:53.995 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_host", 2024-04-25T00:26:53.995 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.995 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.995 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.995 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-04-25T00:26:53.995 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.995 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.995 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.995 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.995 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.995 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.996 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.996 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_port": { 2024-04-25T00:26:53.996 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_port", 2024-04-25T00:26:53.996 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-04-25T00:26:53.996 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.996 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.996 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10051", 2024-04-25T00:26:53.996 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.996 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.996 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.996 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.996 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.996 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.997 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.997 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_sender": { 2024-04-25T00:26:53.997 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_sender", 2024-04-25T00:26:53.997 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-04-25T00:26:53.997 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-04-25T00:26:53.997 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-04-25T00:26:53.997 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T00:26:53.997 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-04-25T00:26:53.997 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-04-25T00:26:53.997 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-04-25T00:26:53.997 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-04-25T00:26:53.998 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-04-25T00:26:53.998 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-04-25T00:26:53.998 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-04-25T00:26:53.998 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.998 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.998 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:53.998 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.998 INFO:teuthology.orchestra.run.smithi107.stdout: "services": { 2024-04-25T00:26:53.998 INFO:teuthology.orchestra.run.smithi107.stdout: "dashboard": "https://172.21.15.107:8443/" 2024-04-25T00:26:53.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:53.998 INFO:teuthology.orchestra.run.smithi107.stdout: "always_on_modules": { 2024-04-25T00:26:53.998 INFO:teuthology.orchestra.run.smithi107.stdout: "octopus": [ 2024-04-25T00:26:53.998 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-04-25T00:26:53.998 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-04-25T00:26:53.999 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-04-25T00:26:53.999 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-04-25T00:26:53.999 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-04-25T00:26:53.999 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-04-25T00:26:53.999 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-04-25T00:26:53.999 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-04-25T00:26:53.999 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-04-25T00:26:53.999 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-04-25T00:26:53.999 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:53.999 INFO:teuthology.orchestra.run.smithi107.stdout: "pacific": [ 2024-04-25T00:26:53.999 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-04-25T00:26:53.999 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-04-25T00:26:53.999 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-04-25T00:26:53.999 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-04-25T00:26:53.999 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-04-25T00:26:54.000 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-04-25T00:26:54.000 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-04-25T00:26:54.000 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-04-25T00:26:54.000 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-04-25T00:26:54.000 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-04-25T00:26:54.000 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:54.000 INFO:teuthology.orchestra.run.smithi107.stdout: "quincy": [ 2024-04-25T00:26:54.000 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-04-25T00:26:54.000 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-04-25T00:26:54.000 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-04-25T00:26:54.000 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-04-25T00:26:54.000 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-04-25T00:26:54.000 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-04-25T00:26:54.000 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-04-25T00:26:54.000 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-04-25T00:26:54.001 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-04-25T00:26:54.001 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-04-25T00:26:54.001 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:54.001 INFO:teuthology.orchestra.run.smithi107.stdout: "reef": [ 2024-04-25T00:26:54.001 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-04-25T00:26:54.001 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-04-25T00:26:54.001 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-04-25T00:26:54.001 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-04-25T00:26:54.001 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-04-25T00:26:54.001 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-04-25T00:26:54.001 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-04-25T00:26:54.001 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-04-25T00:26:54.001 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-04-25T00:26:54.001 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-04-25T00:26:54.002 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-04-25T00:26:54.002 INFO:teuthology.orchestra.run.smithi107.stdout: "squid": [ 2024-04-25T00:26:54.002 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-04-25T00:26:54.002 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-04-25T00:26:54.002 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-04-25T00:26:54.002 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-04-25T00:26:54.002 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-04-25T00:26:54.002 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-04-25T00:26:54.002 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-04-25T00:26:54.002 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-04-25T00:26:54.002 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-04-25T00:26:54.002 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-04-25T00:26:54.002 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-04-25T00:26:54.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:54.003 INFO:teuthology.orchestra.run.smithi107.stdout: "last_failure_osd_epoch": 97, 2024-04-25T00:26:54.003 INFO:teuthology.orchestra.run.smithi107.stdout: "active_clients": [ 2024-04-25T00:26:54.003 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:54.003 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "devicehealth", 2024-04-25T00:26:54.003 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-04-25T00:26:54.003 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:54.003 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-04-25T00:26:54.003 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-04-25T00:26:54.003 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 2959134888 2024-04-25T00:26:54.003 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:54.003 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-04-25T00:26:54.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:54.003 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:54.003 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "libcephsqlite", 2024-04-25T00:26:54.003 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-04-25T00:26:54.004 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:54.004 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-04-25T00:26:54.004 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-04-25T00:26:54.004 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 2073274253 2024-04-25T00:26:54.004 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:54.004 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-04-25T00:26:54.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:54.004 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:54.004 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_support", 2024-04-25T00:26:54.004 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-04-25T00:26:54.004 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:54.004 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-04-25T00:26:54.004 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-04-25T00:26:54.004 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 2220371308 2024-04-25T00:26:54.004 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:54.005 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-04-25T00:26:54.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:54.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:54.005 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2024-04-25T00:26:54.005 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-04-25T00:26:54.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:54.005 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-04-25T00:26:54.005 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-04-25T00:26:54.005 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1201113043 2024-04-25T00:26:54.005 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:54.005 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-04-25T00:26:54.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:54.005 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:54.005 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2024-04-25T00:26:54.006 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-04-25T00:26:54.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:54.006 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-04-25T00:26:54.006 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-04-25T00:26:54.006 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 312767301 2024-04-25T00:26:54.006 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:54.006 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-04-25T00:26:54.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:54.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:54.006 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "volumes", 2024-04-25T00:26:54.006 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-04-25T00:26:54.006 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-04-25T00:26:54.006 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-04-25T00:26:54.006 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-04-25T00:26:54.006 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 4137039291 2024-04-25T00:26:54.007 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:54.007 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-04-25T00:26:54.007 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:54.007 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-04-25T00:26:54.007 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:26:54.008 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:53 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3733985727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:26:54.008 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:53 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3225446890' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T00:26:54.008 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:53 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3733985727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:26:54.009 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:53 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3225446890' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T00:26:54.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.731+0000 7f68f3d1d640 1 -- 172.21.15.107:0/3225446890 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f68d80a2d00 msgr2=0x7f68d80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:54.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.731+0000 7f68f3d1d640 1 --2- 172.21.15.107:0/3225446890 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f68d80a2d00 0x7f68d80a51c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f68e4009f20 tx=0x7f68e40094f0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.731+0000 7f68f3d1d640 1 -- 172.21.15.107:0/3225446890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68ec106630 msgr2=0x7f68ec19c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:54.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.731+0000 7f68f3d1d640 1 --2- 172.21.15.107:0/3225446890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68ec106630 0x7f68ec19c980 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f68e800c970 tx=0x7f68e800ce30 comp rx=0 tx=0).stop 2024-04-25T00:26:54.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.731+0000 7f68f3d1d640 1 -- 172.21.15.107:0/3225446890 shutdown_connections 2024-04-25T00:26:54.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.731+0000 7f68f3d1d640 1 --2- 172.21.15.107:0/3225446890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68ec106630 0x7f68ec19c980 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.731+0000 7f68f3d1d640 1 --2- 172.21.15.107:0/3225446890 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f68d80a2d00 0x7f68d80a51c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.731+0000 7f68f3d1d640 1 --2- 172.21.15.107:0/3225446890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f68ec100fe0 0x7f68ec198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.731+0000 7f68f3d1d640 1 --2- 172.21.15.107:0/3225446890 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68ec100620 0x7f68ec1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:54.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.731+0000 7f68f3d1d640 1 -- 172.21.15.107:0/3225446890 >> 172.21.15.107:0/3225446890 conn(0x7f68ec077740 msgr2=0x7f68ec102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:54.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.731+0000 7f68f3d1d640 1 -- 172.21.15.107:0/3225446890 shutdown_connections 2024-04-25T00:26:54.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:26:52.731+0000 7f68f3d1d640 1 -- 172.21.15.107:0/3225446890 wait complete. 2024-04-25T00:26:54.016 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T00:26:54.016 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T00:26:54.017 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:53 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3733985727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:26:54.017 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:53 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3225446890' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T00:26:54.267 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:26:54.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:54 smithi107 ceph-mon[29153]: pgmap v324: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:26:54.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:54 smithi170 ceph-mon[32807]: pgmap v324: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:26:54.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:54 smithi195 ceph-mon[32833]: pgmap v324: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:26:56.035 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:26:56.035 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:26:56.035 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 0, 2024-04-25T00:26:56.035 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 2, 2024-04-25T00:26:56.035 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 0, 2024-04-25T00:26:56.035 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 0, 2024-04-25T00:26:56.036 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-04-25T00:26:56.036 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 0, 2024-04-25T00:26:56.036 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.001237671, 2024-04-25T00:26:56.036 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T00:26:56.036 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.000279217, 2024-04-25T00:26:56.036 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:26:56.036 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:26:56.036 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-25T00:26:56.036 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-25T00:26:56.036 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-25T00:26:56.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:56.036 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:26:56.036 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-25T00:26:56.036 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-25T00:26:56.037 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-25T00:26:56.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:56.037 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:26:56.037 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:26:56.037 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:56.037 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:26:56.037 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 221, 2024-04-25T00:26:56.037 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 211, 2024-04-25T00:26:56.037 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 1443008, 2024-04-25T00:26:56.037 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 129344, 2024-04-25T00:26:56.037 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 2, 2024-04-25T00:26:56.037 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 1, 2024-04-25T00:26:56.037 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.041669173, 2024-04-25T00:26:56.037 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.014159436, 2024-04-25T00:26:56.038 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.031027044, 2024-04-25T00:26:56.038 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.004306111, 2024-04-25T00:26:56.038 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:26:56.038 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 210, 2024-04-25T00:26:56.038 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.009357017, 2024-04-25T00:26:56.038 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000044557 2024-04-25T00:26:56.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:56.038 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:26:56.038 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-25T00:26:56.038 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-25T00:26:56.038 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-25T00:26:56.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:56.039 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 1443008, 2024-04-25T00:26:56.039 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 129344 2024-04-25T00:26:56.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:56.039 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:26:56.039 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 6, 2024-04-25T00:26:56.039 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 162, 2024-04-25T00:26:56.039 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 3344, 2024-04-25T00:26:56.039 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 184016, 2024-04-25T00:26:56.039 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 5, 2024-04-25T00:26:56.039 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 3, 2024-04-25T00:26:56.039 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.013306942, 2024-04-25T00:26:56.039 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.008961046, 2024-04-25T00:26:56.039 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.002231152, 2024-04-25T00:26:56.039 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:26:56.039 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:26:56.040 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 160, 2024-04-25T00:26:56.040 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.014097001, 2024-04-25T00:26:56.040 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000088106 2024-04-25T00:26:56.040 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:56.040 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:26:56.040 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-25T00:26:56.040 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-25T00:26:56.040 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-25T00:26:56.040 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:56.040 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-25T00:26:56.040 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 184016 2024-04-25T00:26:56.040 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:56.040 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-04-25T00:26:56.040 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 0, 2024-04-25T00:26:56.040 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-04-25T00:26:56.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:56.041 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-04-25T00:26:56.041 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:26:56.041 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-04-25T00:26:56.041 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:26:56.041 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:26:56.041 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:26:56.041 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:26:56.041 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:26:56.041 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:26:56.041 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:26:56.041 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:26:56.041 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:26:56.041 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:26:56.042 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:26:56.042 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:26:56.042 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:26:56.042 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-04-25T00:26:56.042 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:26:56.042 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-04-25T00:26:56.042 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:26:56.042 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:26:56.042 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:26:56.042 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:26:56.042 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:26:56.042 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:26:56.042 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:26:56.042 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-04-25T00:26:56.042 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:26:56.043 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:26:56.043 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:26:56.043 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-04-25T00:26:56.043 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-04-25T00:26:56.043 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-04-25T00:26:56.043 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:26:56.043 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:26:56.043 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:26:56.043 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:26:56.043 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 149057, 2024-04-25T00:26:56.043 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 32, 2024-04-25T00:26:56.043 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:26:56.044 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-04-25T00:26:56.044 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-04-25T00:26:56.044 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-04-25T00:26:56.044 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:26:56.044 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-04-25T00:26:56.044 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:26:56.044 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-04-25T00:26:56.044 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 0, 2024-04-25T00:26:56.044 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 0, 2024-04-25T00:26:56.044 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:26:56.044 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:26:56.044 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-04-25T00:26:56.044 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-04-25T00:26:56.045 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 928, 2024-04-25T00:26:56.045 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 2, 2024-04-25T00:26:56.045 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-04-25T00:26:56.045 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-04-25T00:26:56.045 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-04-25T00:26:56.045 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-04-25T00:26:56.045 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:56.045 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:26:56.045 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-04-25T00:26:56.045 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-04-25T00:26:56.045 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-04-25T00:26:56.045 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 223, 2024-04-25T00:26:56.045 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 1411878, 2024-04-25T00:26:56.045 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-04-25T00:26:56.046 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 223, 2024-04-25T00:26:56.046 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-04-25T00:26:56.046 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-04-25T00:26:56.046 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 223, 2024-04-25T00:26:56.046 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1411878, 2024-04-25T00:26:56.046 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-04-25T00:26:56.046 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-25T00:26:56.046 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-25T00:26:56.046 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-25T00:26:56.046 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:56.046 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:56.046 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:26:56.433 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:56 smithi107 ceph-mon[29153]: pgmap v325: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:26:56.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:56 smithi170 ceph-mon[32807]: pgmap v325: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:26:56.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:56 smithi195 ceph-mon[32833]: pgmap v325: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:26:56.814 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-25T00:26:56.815 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-25T00:26:56.815 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-25T00:26:56.815 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-25T00:26:56.815 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-25T00:26:56.815 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T00:26:56.815 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T00:26:57.053 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:26:58.156 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:26:58 smithi107 ceph-mon[29153]: pgmap v326: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:26:58.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:26:58 smithi170 ceph-mon[32807]: pgmap v326: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:26:58.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:26:58 smithi195 ceph-mon[32833]: pgmap v326: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:26:58.710 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:26:58.710 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:26:58.710 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 0, 2024-04-25T00:26:58.710 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 2, 2024-04-25T00:26:58.711 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 0, 2024-04-25T00:26:58.711 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 0, 2024-04-25T00:26:58.711 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-04-25T00:26:58.711 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 0, 2024-04-25T00:26:58.711 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.001213277, 2024-04-25T00:26:58.711 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T00:26:58.711 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.000263027, 2024-04-25T00:26:58.711 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:26:58.711 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:26:58.711 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-25T00:26:58.711 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-25T00:26:58.711 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-25T00:26:58.711 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:26:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-25T00:26:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-25T00:26:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-25T00:26:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:26:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:26:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:26:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 6, 2024-04-25T00:26:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 167, 2024-04-25T00:26:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 3344, 2024-04-25T00:26:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 184896, 2024-04-25T00:26:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 5, 2024-04-25T00:26:58.712 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 3, 2024-04-25T00:26:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.014906557, 2024-04-25T00:26:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.010226903, 2024-04-25T00:26:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.001986751, 2024-04-25T00:26:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:26:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:26:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 165, 2024-04-25T00:26:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.017520496, 2024-04-25T00:26:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000106184 2024-04-25T00:26:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:26:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-25T00:26:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-25T00:26:58.713 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-25T00:26:58.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:58.714 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-25T00:26:58.714 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 184896 2024-04-25T00:26:58.714 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:58.714 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:26:58.714 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 226, 2024-04-25T00:26:58.714 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 216, 2024-04-25T00:26:58.714 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 1444368, 2024-04-25T00:26:58.714 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 132544, 2024-04-25T00:26:58.714 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 2, 2024-04-25T00:26:58.714 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 1, 2024-04-25T00:26:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.040410670, 2024-04-25T00:26:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.013732208, 2024-04-25T00:26:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.030495396, 2024-04-25T00:26:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.004215804, 2024-04-25T00:26:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:26:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 215, 2024-04-25T00:26:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.011240070, 2024-04-25T00:26:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000052279 2024-04-25T00:26:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:26:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-25T00:26:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-25T00:26:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-25T00:26:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:58.715 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 1444368, 2024-04-25T00:26:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 132544 2024-04-25T00:26:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-04-25T00:26:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 0, 2024-04-25T00:26:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-04-25T00:26:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-04-25T00:26:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:26:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-04-25T00:26:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:26:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:26:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:26:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:26:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:26:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:26:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:26:58.716 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:26:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:26:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:26:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:26:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:26:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:26:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-04-25T00:26:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:26:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-04-25T00:26:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:26:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:26:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:26:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:26:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:26:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:26:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:26:58.717 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-04-25T00:26:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:26:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:26:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:26:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-04-25T00:26:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-04-25T00:26:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-04-25T00:26:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:26:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:26:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:26:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:26:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 149057, 2024-04-25T00:26:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 32, 2024-04-25T00:26:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:26:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-04-25T00:26:58.718 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-04-25T00:26:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-04-25T00:26:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:26:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-04-25T00:26:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:26:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-04-25T00:26:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 0, 2024-04-25T00:26:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 0, 2024-04-25T00:26:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:26:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:26:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-04-25T00:26:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-04-25T00:26:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 928, 2024-04-25T00:26:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 2, 2024-04-25T00:26:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-04-25T00:26:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-04-25T00:26:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-04-25T00:26:58.719 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-04-25T00:26:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:26:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:26:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-04-25T00:26:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-04-25T00:26:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-04-25T00:26:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 228, 2024-04-25T00:26:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 1412528, 2024-04-25T00:26:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-04-25T00:26:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 228, 2024-04-25T00:26:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-04-25T00:26:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-04-25T00:26:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 228, 2024-04-25T00:26:58.720 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1412528, 2024-04-25T00:26:58.721 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-04-25T00:26:58.721 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-25T00:26:58.721 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-25T00:26:58.721 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-25T00:26:58.721 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:58.721 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:26:58.721 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:26:59.410 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-25T00:26:59.410 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-25T00:26:59.410 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-25T00:26:59.410 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-25T00:26:59.410 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-25T00:26:59.410 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T00:26:59.411 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T00:26:59.649 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:27:00.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:00 smithi170 ceph-mon[32807]: pgmap v327: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:27:00.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:00 smithi107 ceph-mon[29153]: pgmap v327: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:27:00.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:00 smithi195 ceph-mon[32833]: pgmap v327: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:27:01.395 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:27:01.395 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:27:01.395 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 227, 2024-04-25T00:27:01.396 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 215, 2024-04-25T00:27:01.396 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 1446576, 2024-04-25T00:27:01.396 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 131072, 2024-04-25T00:27:01.396 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-04-25T00:27:01.396 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 0, 2024-04-25T00:27:01.396 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.043265058, 2024-04-25T00:27:01.396 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.014408211, 2024-04-25T00:27:01.396 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.030855459, 2024-04-25T00:27:01.396 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.004331384, 2024-04-25T00:27:01.396 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:01.396 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 215, 2024-04-25T00:27:01.396 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.010983858, 2024-04-25T00:27:01.396 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000051087 2024-04-25T00:27:01.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:27:01.397 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:01.397 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-25T00:27:01.397 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-25T00:27:01.397 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-25T00:27:01.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:27:01.397 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 1446576, 2024-04-25T00:27:01.397 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 131072 2024-04-25T00:27:01.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:27:01.397 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:27:01.397 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 2, 2024-04-25T00:27:01.397 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 162, 2024-04-25T00:27:01.397 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 320, 2024-04-25T00:27:01.397 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 183920, 2024-04-25T00:27:01.397 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 5, 2024-04-25T00:27:01.398 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 3, 2024-04-25T00:27:01.398 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.014047180, 2024-04-25T00:27:01.398 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.009473400, 2024-04-25T00:27:01.398 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.002418610, 2024-04-25T00:27:01.398 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:27:01.398 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:01.398 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 159, 2024-04-25T00:27:01.398 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.015147899, 2024-04-25T00:27:01.398 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000095269 2024-04-25T00:27:01.398 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:27:01.398 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:01.398 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-25T00:27:01.398 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-25T00:27:01.398 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-25T00:27:01.398 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:27:01.399 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T00:27:01.399 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 183920 2024-04-25T00:27:01.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:27:01.399 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:27:01.399 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 0, 2024-04-25T00:27:01.399 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 2, 2024-04-25T00:27:01.399 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 0, 2024-04-25T00:27:01.399 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 0, 2024-04-25T00:27:01.399 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 2, 2024-04-25T00:27:01.399 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 1, 2024-04-25T00:27:01.399 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.000262155, 2024-04-25T00:27:01.399 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T00:27:01.399 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.000211256, 2024-04-25T00:27:01.400 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:27:01.400 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:01.400 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-25T00:27:01.400 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-25T00:27:01.400 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-25T00:27:01.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:27:01.400 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:01.400 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-25T00:27:01.400 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-25T00:27:01.400 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-25T00:27:01.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:27:01.400 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:27:01.400 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:27:01.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:27:01.400 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-04-25T00:27:01.400 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 0, 2024-04-25T00:27:01.401 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-04-25T00:27:01.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:27:01.401 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-04-25T00:27:01.401 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:27:01.401 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-04-25T00:27:01.401 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:27:01.401 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:27:01.401 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:27:01.401 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:27:01.401 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:27:01.401 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:27:01.401 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:27:01.401 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:27:01.401 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:27:01.401 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:27:01.401 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:27:01.402 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:27:01.402 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:27:01.402 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-04-25T00:27:01.402 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:27:01.402 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-04-25T00:27:01.402 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:27:01.402 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:27:01.402 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:27:01.402 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:27:01.402 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:27:01.402 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:27:01.402 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:27:01.402 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-04-25T00:27:01.402 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:27:01.403 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:27:01.403 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:27:01.403 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-04-25T00:27:01.403 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-04-25T00:27:01.403 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-04-25T00:27:01.403 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:27:01.403 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:27:01.403 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:27:01.403 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:27:01.403 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 148977, 2024-04-25T00:27:01.403 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 30, 2024-04-25T00:27:01.403 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:27:01.403 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-04-25T00:27:01.403 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-04-25T00:27:01.403 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-04-25T00:27:01.404 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:27:01.404 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-04-25T00:27:01.404 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:27:01.404 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-04-25T00:27:01.404 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 0, 2024-04-25T00:27:01.404 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 0, 2024-04-25T00:27:01.404 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:27:01.404 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:27:01.404 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-04-25T00:27:01.404 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-04-25T00:27:01.404 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 928, 2024-04-25T00:27:01.404 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 2, 2024-04-25T00:27:01.404 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-04-25T00:27:01.404 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-04-25T00:27:01.404 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-04-25T00:27:01.405 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-04-25T00:27:01.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:27:01.405 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:27:01.405 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-04-25T00:27:01.405 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-04-25T00:27:01.405 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-04-25T00:27:01.405 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 225, 2024-04-25T00:27:01.405 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 1412138, 2024-04-25T00:27:01.405 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:01.405 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 225, 2024-04-25T00:27:01.405 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-04-25T00:27:01.405 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-04-25T00:27:01.405 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 225, 2024-04-25T00:27:01.405 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1412138, 2024-04-25T00:27:01.406 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-04-25T00:27:01.406 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-25T00:27:01.406 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-25T00:27:01.406 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-25T00:27:01.406 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:27:01.406 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:27:01.406 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:27:02.106 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-25T00:27:02.106 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-25T00:27:02.107 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-25T00:27:02.107 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-25T00:27:02.107 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-25T00:27:02.107 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T00:27:02.107 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T00:27:02.349 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:27:02.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:02 smithi170 ceph-mon[32807]: pgmap v328: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:27:02.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:02 smithi107 ceph-mon[29153]: pgmap v328: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:27:02.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:02 smithi195 ceph-mon[32833]: pgmap v328: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:27:04.053 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T00:27:04.054 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:27:04.054 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 228, 2024-04-25T00:27:04.054 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 218, 2024-04-25T00:27:04.054 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 1444912, 2024-04-25T00:27:04.054 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 133824, 2024-04-25T00:27:04.054 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-04-25T00:27:04.054 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 0, 2024-04-25T00:27:04.054 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.045610560, 2024-04-25T00:27:04.054 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.014488842, 2024-04-25T00:27:04.054 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.033752131, 2024-04-25T00:27:04.055 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.004453433, 2024-04-25T00:27:04.055 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:04.055 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 217, 2024-04-25T00:27:04.055 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.011123062, 2024-04-25T00:27:04.055 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000051258 2024-04-25T00:27:04.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:27:04.055 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:04.055 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-25T00:27:04.055 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-25T00:27:04.055 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-25T00:27:04.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:27:04.055 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 1444912, 2024-04-25T00:27:04.055 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 133824 2024-04-25T00:27:04.056 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:27:04.056 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:27:04.056 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 2, 2024-04-25T00:27:04.056 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 167, 2024-04-25T00:27:04.056 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 320, 2024-04-25T00:27:04.056 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 184800, 2024-04-25T00:27:04.056 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 5, 2024-04-25T00:27:04.056 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 3, 2024-04-25T00:27:04.056 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.013630318, 2024-04-25T00:27:04.056 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.009039958, 2024-04-25T00:27:04.056 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.002325663, 2024-04-25T00:27:04.056 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:27:04.056 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:04.057 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 164, 2024-04-25T00:27:04.057 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.018341067, 2024-04-25T00:27:04.057 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000111835 2024-04-25T00:27:04.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:27:04.057 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:04.057 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-25T00:27:04.057 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-25T00:27:04.057 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-25T00:27:04.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:27:04.057 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T00:27:04.057 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 184800 2024-04-25T00:27:04.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:27:04.057 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:27:04.058 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 4, 2024-04-25T00:27:04.058 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 4, 2024-04-25T00:27:04.058 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 3024, 2024-04-25T00:27:04.058 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 448, 2024-04-25T00:27:04.058 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 2, 2024-04-25T00:27:04.058 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 1, 2024-04-25T00:27:04.058 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.000688561, 2024-04-25T00:27:04.058 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.000034453, 2024-04-25T00:27:04.058 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.000632047, 2024-04-25T00:27:04.058 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:27:04.058 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:04.058 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3, 2024-04-25T00:27:04.058 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000639101, 2024-04-25T00:27:04.058 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000213033 2024-04-25T00:27:04.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:27:04.059 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:04.059 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-25T00:27:04.059 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-25T00:27:04.059 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-25T00:27:04.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:27:04.059 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-25T00:27:04.059 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:27:04.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:27:04.059 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-04-25T00:27:04.059 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 0, 2024-04-25T00:27:04.059 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-04-25T00:27:04.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:27:04.059 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-04-25T00:27:04.059 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:27:04.060 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-04-25T00:27:04.060 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:27:04.060 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:27:04.060 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:27:04.060 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:27:04.060 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:27:04.060 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:27:04.060 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:27:04.060 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:27:04.060 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:27:04.060 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:27:04.060 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:27:04.060 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:27:04.061 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:27:04.061 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-04-25T00:27:04.061 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:27:04.061 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-04-25T00:27:04.061 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:27:04.061 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:27:04.061 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:27:04.061 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:27:04.061 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:27:04.061 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:27:04.061 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:27:04.061 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-04-25T00:27:04.061 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:27:04.061 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:27:04.062 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:27:04.062 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-04-25T00:27:04.062 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-04-25T00:27:04.062 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-04-25T00:27:04.062 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:27:04.062 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:27:04.062 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:27:04.062 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:27:04.062 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 149057, 2024-04-25T00:27:04.062 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 32, 2024-04-25T00:27:04.062 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:27:04.062 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-04-25T00:27:04.062 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-04-25T00:27:04.062 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-04-25T00:27:04.063 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:27:04.063 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-04-25T00:27:04.063 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:27:04.063 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-04-25T00:27:04.063 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 0, 2024-04-25T00:27:04.063 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 0, 2024-04-25T00:27:04.063 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:27:04.063 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:27:04.063 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-04-25T00:27:04.063 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-04-25T00:27:04.063 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 928, 2024-04-25T00:27:04.063 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 2, 2024-04-25T00:27:04.063 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-04-25T00:27:04.063 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-04-25T00:27:04.064 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-04-25T00:27:04.064 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-04-25T00:27:04.064 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T00:27:04.064 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:27:04.064 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-04-25T00:27:04.064 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-04-25T00:27:04.064 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-04-25T00:27:04.064 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 230, 2024-04-25T00:27:04.064 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 1412788, 2024-04-25T00:27:04.064 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:04.064 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 230, 2024-04-25T00:27:04.064 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-04-25T00:27:04.064 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-04-25T00:27:04.064 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 230, 2024-04-25T00:27:04.064 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 1412788, 2024-04-25T00:27:04.065 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-04-25T00:27:04.065 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-04-25T00:27:04.065 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-04-25T00:27:04.065 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-04-25T00:27:04.065 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:27:04.065 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T00:27:04.065 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T00:27:04.425 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:04 smithi107 ceph-mon[29153]: pgmap v329: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T00:27:04.434 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:04 smithi170 ceph-mon[32807]: pgmap v329: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T00:27:04.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:04 smithi195 ceph-mon[32833]: pgmap v329: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T00:27:04.762 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-25T00:27:04.762 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-25T00:27:04.763 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-25T00:27:04.763 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-25T00:27:04.763 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-25T00:27:04.763 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T00:27:04.763 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T00:27:05.013 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:27:06.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:06 smithi170 ceph-mon[32807]: pgmap v330: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:27:06.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:06 smithi107 ceph-mon[29153]: pgmap v330: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:27:06.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:06 smithi195 ceph-mon[32833]: pgmap v330: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:27:06.885 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-04-25T00:27:06.886 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:27:06.886 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 18174, 2024-04-25T00:27:06.886 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 15467, 2024-04-25T00:27:06.886 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 8149013, 2024-04-25T00:27:06.886 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 10616515, 2024-04-25T00:27:06.886 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 15, 2024-04-25T00:27:06.886 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 4, 2024-04-25T00:27:06.886 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 1.185712744, 2024-04-25T00:27:06.886 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.477679327, 2024-04-25T00:27:06.887 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.674536791, 2024-04-25T00:27:06.887 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.145499806, 2024-04-25T00:27:06.887 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:06.887 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 15467, 2024-04-25T00:27:06.887 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.755008125, 2024-04-25T00:27:06.887 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000048814 2024-04-25T00:27:06.887 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.887 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:06.887 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 26765, 2024-04-25T00:27:06.887 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.004796771, 2024-04-25T00:27:06.887 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000179 2024-04-25T00:27:06.887 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.887 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1915776, 2024-04-25T00:27:06.887 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 296640 2024-04-25T00:27:06.888 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.888 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:27:06.888 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 18860, 2024-04-25T00:27:06.888 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 31343, 2024-04-25T00:27:06.888 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 6349944, 2024-04-25T00:27:06.888 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 35095074, 2024-04-25T00:27:06.888 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 171, 2024-04-25T00:27:06.888 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 9, 2024-04-25T00:27:06.888 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 1.258529081, 2024-04-25T00:27:06.888 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.490369741, 2024-04-25T00:27:06.888 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.730289299, 2024-04-25T00:27:06.888 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.165138802, 2024-04-25T00:27:06.888 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:06.888 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 31340, 2024-04-25T00:27:06.888 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.293692572, 2024-04-25T00:27:06.889 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000041279 2024-04-25T00:27:06.889 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.889 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:06.889 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 37557, 2024-04-25T00:27:06.889 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.005801489, 2024-04-25T00:27:06.889 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000154 2024-04-25T00:27:06.889 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.889 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 88624, 2024-04-25T00:27:06.889 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 489456 2024-04-25T00:27:06.889 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.889 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:27:06.889 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 7170, 2024-04-25T00:27:06.889 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 7263, 2024-04-25T00:27:06.889 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 2182057, 2024-04-25T00:27:06.890 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 15561000, 2024-04-25T00:27:06.890 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 12, 2024-04-25T00:27:06.890 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 6, 2024-04-25T00:27:06.890 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.547488636, 2024-04-25T00:27:06.890 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.199606295, 2024-04-25T00:27:06.890 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.173021863, 2024-04-25T00:27:06.890 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.153257771, 2024-04-25T00:27:06.890 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:06.890 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 7261, 2024-04-25T00:27:06.890 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.545267150, 2024-04-25T00:27:06.890 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000075095 2024-04-25T00:27:06.890 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.890 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:06.890 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4348, 2024-04-25T00:27:06.891 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000778758, 2024-04-25T00:27:06.891 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000179 2024-04-25T00:27:06.891 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.891 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 42560, 2024-04-25T00:27:06.891 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 1936 2024-04-25T00:27:06.891 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.891 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-04-25T00:27:06.891 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-04-25T00:27:06.891 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-04-25T00:27:06.891 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.891 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-04-25T00:27:06.891 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-04-25T00:27:06.891 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-04-25T00:27:06.891 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 5869, 2024-04-25T00:27:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 8.705937127, 2024-04-25T00:27:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001483376 2024-04-25T00:27:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-04-25T00:27:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-04-25T00:27:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-04-25T00:27:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4385, 2024-04-25T00:27:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.701866225, 2024-04-25T00:27:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000160060 2024-04-25T00:27:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-04-25T00:27:06.892 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 8696, 2024-04-25T00:27:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 8646, 2024-04-25T00:27:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-04-25T00:27:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 8646, 2024-04-25T00:27:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 18.617262200, 2024-04-25T00:27:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002153280 2024-04-25T00:27:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-04-25T00:27:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 50, 2024-04-25T00:27:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 0, 2024-04-25T00:27:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2024-04-25T00:27:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 21, 2024-04-25T00:27:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 1, 2024-04-25T00:27:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2024-04-25T00:27:06.893 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 3060, 2024-04-25T00:27:06.894 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 6, 2024-04-25T00:27:06.894 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 8, 2024-04-25T00:27:06.894 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 3046, 2024-04-25T00:27:06.894 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 3051, 2024-04-25T00:27:06.894 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 2914, 2024-04-25T00:27:06.894 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 3, 2024-04-25T00:27:06.894 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 3, 2024-04-25T00:27:06.894 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 6, 2024-04-25T00:27:06.894 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 12660, 2024-04-25T00:27:06.894 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 11302, 2024-04-25T00:27:06.894 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 26, 2024-04-25T00:27:06.894 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 164, 2024-04-25T00:27:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:27:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 5, 2024-04-25T00:27:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 9, 2024-04-25T00:27:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2024-04-25T00:27:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-04-25T00:27:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 223, 2024-04-25T00:27:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 2406, 2024-04-25T00:27:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 83, 2024-04-25T00:27:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 1021, 2024-04-25T00:27:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-04-25T00:27:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 4226, 2024-04-25T00:27:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 17, 2024-04-25T00:27:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 4, 2024-04-25T00:27:06.895 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 234, 2024-04-25T00:27:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 6, 2024-04-25T00:27:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 4226, 2024-04-25T00:27:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-04-25T00:27:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:27:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:27:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 438, 2024-04-25T00:27:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 20, 2024-04-25T00:27:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 418, 2024-04-25T00:27:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 3788, 2024-04-25T00:27:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 48, 2024-04-25T00:27:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 1142, 2024-04-25T00:27:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 6126, 2024-04-25T00:27:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 67, 2024-04-25T00:27:06.896 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:27:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:27:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 5277, 2024-04-25T00:27:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 4130, 2024-04-25T00:27:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 100, 2024-04-25T00:27:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 4354 2024-04-25T00:27:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-04-25T00:27:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 0, 2024-04-25T00:27:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 0, 2024-04-25T00:27:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 0, 2024-04-25T00:27:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 196, 2024-04-25T00:27:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 279, 2024-04-25T00:27:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 99, 2024-04-25T00:27:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-04-25T00:27:06.897 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:27:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 2411, 2024-04-25T00:27:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 2282, 2024-04-25T00:27:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 31, 2024-04-25T00:27:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 4, 2024-04-25T00:27:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-04-25T00:27:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:27:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:27:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-04-25T00:27:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-04-25T00:27:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:27:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:27:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 20, 2024-04-25T00:27:06.898 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 223, 2024-04-25T00:27:06.899 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-04-25T00:27:06.899 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 1, 2024-04-25T00:27:06.899 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-04-25T00:27:06.899 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-04-25T00:27:06.899 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.899 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-04-25T00:27:06.899 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2024-04-25T00:27:06.899 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 12973, 2024-04-25T00:27:06.899 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2024-04-25T00:27:06.899 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2024-04-25T00:27:06.899 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 12973, 2024-04-25T00:27:06.899 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-04-25T00:27:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2024-04-25T00:27:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 19, 2024-04-25T00:27:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2024-04-25T00:27:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2024-04-25T00:27:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 19, 2024-04-25T00:27:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 3, 2024-04-25T00:27:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-04-25T00:27:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2024-04-25T00:27:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2024-04-25T00:27:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 38288465, 2024-04-25T00:27:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 0, 2024-04-25T00:27:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-04-25T00:27:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2060, 2024-04-25T00:27:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 31.096652968, 2024-04-25T00:27:06.900 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.015095462 2024-04-25T00:27:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 0 2024-04-25T00:27:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-04-25T00:27:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 1631, 2024-04-25T00:27:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 6095, 2024-04-25T00:27:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 4464, 2024-04-25T00:27:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 169, 2024-04-25T00:27:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 792, 2024-04-25T00:27:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 623, 2024-04-25T00:27:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 3060, 2024-04-25T00:27:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 9278, 2024-04-25T00:27:06.901 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 6218, 2024-04-25T00:27:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 3, 2024-04-25T00:27:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 4307, 2024-04-25T00:27:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 4304, 2024-04-25T00:27:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 111684, 2024-04-25T00:27:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 231804 2024-04-25T00:27:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-04-25T00:27:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 11585, 2024-04-25T00:27:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 837, 2024-04-25T00:27:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 8696, 2024-04-25T00:27:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 372, 2024-04-25T00:27:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 2256, 2024-04-25T00:27:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-04-25T00:27:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2681, 2024-04-25T00:27:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 5.190758457, 2024-04-25T00:27:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001936127 2024-04-25T00:27:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-04-25T00:27:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 154, 2024-04-25T00:27:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.121551866, 2024-04-25T00:27:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000789297 2024-04-25T00:27:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-04-25T00:27:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:06.903 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-04-25T00:27:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 82, 2024-04-25T00:27:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.575348339, 2024-04-25T00:27:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.007016443 2024-04-25T00:27:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-04-25T00:27:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1729, 2024-04-25T00:27:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.119280992, 2024-04-25T00:27:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000647357 2024-04-25T00:27:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-04-25T00:27:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:06.904 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.905 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-04-25T00:27:06.905 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1, 2024-04-25T00:27:06.905 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000152762, 2024-04-25T00:27:06.905 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000152762 2024-04-25T00:27:06.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.905 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-04-25T00:27:06.905 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:06.905 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:06.905 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:06.905 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.905 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-04-25T00:27:06.905 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:06.905 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:06.906 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:06.906 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.906 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-04-25T00:27:06.906 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:06.906 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:06.906 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:06.906 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.906 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-04-25T00:27:06.906 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 11, 2024-04-25T00:27:06.906 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.001570997, 2024-04-25T00:27:06.906 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000142817 2024-04-25T00:27:06.906 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.906 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-04-25T00:27:06.907 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 515, 2024-04-25T00:27:06.907 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.723958700, 2024-04-25T00:27:06.907 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001405745 2024-04-25T00:27:06.907 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.907 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-04-25T00:27:06.907 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 163, 2024-04-25T00:27:06.907 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.263026922, 2024-04-25T00:27:06.907 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001613662 2024-04-25T00:27:06.907 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.907 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-04-25T00:27:06.907 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 11, 2024-04-25T00:27:06.907 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.086702791, 2024-04-25T00:27:06.907 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.007882071 2024-04-25T00:27:06.907 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.908 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-04-25T00:27:06.908 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2, 2024-04-25T00:27:06.908 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000569468, 2024-04-25T00:27:06.908 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000284734 2024-04-25T00:27:06.908 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.908 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-04-25T00:27:06.908 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 341, 2024-04-25T00:27:06.908 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.318082766, 2024-04-25T00:27:06.908 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000932794 2024-04-25T00:27:06.908 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.908 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-04-25T00:27:06.908 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 193, 2024-04-25T00:27:06.908 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 2.052875071, 2024-04-25T00:27:06.908 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.010636658 2024-04-25T00:27:06.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.909 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-04-25T00:27:06.909 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:06.909 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:06.909 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:06.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.909 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-04-25T00:27:06.909 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:06.909 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:06.909 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:06.909 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.909 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-04-25T00:27:06.909 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 316, 2024-04-25T00:27:06.910 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 3.535389569, 2024-04-25T00:27:06.910 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.011187941 2024-04-25T00:27:06.910 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.910 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-04-25T00:27:06.910 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 5, 2024-04-25T00:27:06.910 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.021094975, 2024-04-25T00:27:06.910 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.004218995 2024-04-25T00:27:06.910 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.910 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-04-25T00:27:06.910 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:06.910 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:06.910 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:06.910 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.911 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-04-25T00:27:06.911 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 70, 2024-04-25T00:27:06.911 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.145968224, 2024-04-25T00:27:06.911 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002085260 2024-04-25T00:27:06.911 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.911 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-04-25T00:27:06.911 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:06.911 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:06.911 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:06.911 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.911 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-04-25T00:27:06.911 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:06.911 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:06.911 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:06.911 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.912 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-04-25T00:27:06.912 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:06.912 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:06.912 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:06.912 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.912 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-04-25T00:27:06.912 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 9, 2024-04-25T00:27:06.912 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.019791896, 2024-04-25T00:27:06.912 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002199099 2024-04-25T00:27:06.912 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.912 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-04-25T00:27:06.912 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 156, 2024-04-25T00:27:06.912 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.349920072, 2024-04-25T00:27:06.912 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002243077 2024-04-25T00:27:06.913 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.913 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-04-25T00:27:06.913 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2207, 2024-04-25T00:27:06.913 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 4.091218333, 2024-04-25T00:27:06.913 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001853746 2024-04-25T00:27:06.913 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.913 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:27:06.913 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:27:06.913 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-04-25T00:27:06.913 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:06.913 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:06.913 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:06.913 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:06.913 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.913 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-04-25T00:27:06.914 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 2, 2024-04-25T00:27:06.914 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 7, 2024-04-25T00:27:06.914 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 5, 2024-04-25T00:27:06.914 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 2, 2024-04-25T00:27:06.914 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-04-25T00:27:06.914 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 54, 2024-04-25T00:27:06.914 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 27, 2024-04-25T00:27:06.914 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 619, 2024-04-25T00:27:06.914 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-04-25T00:27:06.914 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.914 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-04-25T00:27:06.914 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:27:06.914 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-04-25T00:27:06.914 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:27:06.915 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:27:06.915 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:27:06.915 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:27:06.915 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:27:06.915 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:27:06.915 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:27:06.915 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:27:06.915 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:27:06.915 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:27:06.915 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:27:06.915 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:27:06.915 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:27:06.915 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-04-25T00:27:06.915 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:27:06.916 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-04-25T00:27:06.916 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:27:06.916 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:27:06.916 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:27:06.916 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:27:06.916 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:27:06.916 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:27:06.916 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:27:06.916 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-04-25T00:27:06.916 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:27:06.916 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:27:06.916 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:27:06.916 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-04-25T00:27:06.917 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-04-25T00:27:06.917 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-04-25T00:27:06.917 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:27:06.917 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:27:06.917 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:27:06.917 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:27:06.917 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 1201080, 2024-04-25T00:27:06.917 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 1903, 2024-04-25T00:27:06.917 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 704, 2024-04-25T00:27:06.917 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 8, 2024-04-25T00:27:06.917 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-04-25T00:27:06.917 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-04-25T00:27:06.917 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:27:06.917 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-04-25T00:27:06.918 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:27:06.918 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-04-25T00:27:06.918 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 9432, 2024-04-25T00:27:06.918 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 182, 2024-04-25T00:27:06.918 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:27:06.918 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:27:06.918 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-04-25T00:27:06.918 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-04-25T00:27:06.918 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 6556810, 2024-04-25T00:27:06.918 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 126204, 2024-04-25T00:27:06.918 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-04-25T00:27:06.918 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-04-25T00:27:06.918 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-04-25T00:27:06.918 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-04-25T00:27:06.919 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.919 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-04-25T00:27:06.919 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-04-25T00:27:06.919 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-04-25T00:27:06.919 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 11891, 2024-04-25T00:27:06.919 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 35062142, 2024-04-25T00:27:06.919 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 34, 2024-04-25T00:27:06.919 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 11891, 2024-04-25T00:27:06.919 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-04-25T00:27:06.919 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 11891, 2024-04-25T00:27:06.919 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 67.434451879, 2024-04-25T00:27:06.919 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.005671049 2024-04-25T00:27:06.919 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.919 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-04-25T00:27:06.920 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-04-25T00:27:06.920 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 11891, 2024-04-25T00:27:06.920 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 12070 2024-04-25T00:27:06.920 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.920 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 11891, 2024-04-25T00:27:06.920 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 4420, 2024-04-25T00:27:06.920 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 7471, 2024-04-25T00:27:06.920 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-04-25T00:27:06.920 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-04-25T00:27:06.920 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 0, 2024-04-25T00:27:06.920 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 11, 2024-04-25T00:27:06.920 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 174, 2024-04-25T00:27:06.920 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 2214, 2024-04-25T00:27:06.920 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 103, 2024-04-25T00:27:06.920 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-04-25T00:27:06.921 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-04-25T00:27:06.921 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-04-25T00:27:06.921 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 1, 2024-04-25T00:27:06.921 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 4946, 2024-04-25T00:27:06.921 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-04-25T00:27:06.921 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-04-25T00:27:06.921 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-04-25T00:27:06.921 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 4246, 2024-04-25T00:27:06.921 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 22, 2024-04-25T00:27:06.921 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:27:06.921 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-04-25T00:27:06.921 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:27:06.921 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-04-25T00:27:06.921 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-04-25T00:27:06.922 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-04-25T00:27:06.922 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:27:06.922 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-04-25T00:27:06.922 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:27:06.922 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 76, 2024-04-25T00:27:06.922 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-04-25T00:27:06.922 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-04-25T00:27:06.922 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-04-25T00:27:06.922 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-04-25T00:27:06.922 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-04-25T00:27:06.922 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-04-25T00:27:06.922 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-04-25T00:27:06.922 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-04-25T00:27:06.923 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-04-25T00:27:06.923 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-04-25T00:27:06.923 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-04-25T00:27:06.923 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-04-25T00:27:06.923 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-04-25T00:27:06.923 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-04-25T00:27:06.923 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-04-25T00:27:06.923 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-04-25T00:27:06.923 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 105, 2024-04-25T00:27:06.923 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-04-25T00:27:06.923 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 20, 2024-04-25T00:27:06.923 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 12, 2024-04-25T00:27:06.923 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 12, 2024-04-25T00:27:06.923 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-04-25T00:27:06.924 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-04-25T00:27:06.924 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 208, 2024-04-25T00:27:06.924 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 0, 2024-04-25T00:27:06.924 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 69 2024-04-25T00:27:06.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.924 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-04-25T00:27:06.924 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 1, 2024-04-25T00:27:06.924 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:27:06.924 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-04-25T00:27:06.924 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-04-25T00:27:06.924 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.924 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-04-25T00:27:06.924 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 0 2024-04-25T00:27:06.925 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.925 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-04-25T00:27:06.925 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-04-25T00:27:06.925 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 21, 2024-04-25T00:27:06.925 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-04-25T00:27:06.925 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 14, 2024-04-25T00:27:06.925 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed_ops": 4426, 2024-04-25T00:27:06.925 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 2282, 2024-04-25T00:27:06.925 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-04-25T00:27:06.925 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.925 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:27:06.925 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-25T00:27:06.925 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-04-25T00:27:06.925 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-25T00:27:06.926 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 44200, 2024-04-25T00:27:06.926 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 12631112, 2024-04-25T00:27:06.926 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:06.926 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 44200, 2024-04-25T00:27:06.926 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-04-25T00:27:06.926 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-04-25T00:27:06.926 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 44200, 2024-04-25T00:27:06.926 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 12631112, 2024-04-25T00:27:06.926 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-25T00:27:06.926 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:06.926 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:06.926 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:06.926 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:06.926 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.926 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-04-25T00:27:06.927 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-25T00:27:06.927 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-04-25T00:27:06.927 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-25T00:27:06.927 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-04-25T00:27:06.927 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-04-25T00:27:06.927 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:06.927 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-04-25T00:27:06.927 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 11891, 2024-04-25T00:27:06.927 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 35328176, 2024-04-25T00:27:06.927 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 11891, 2024-04-25T00:27:06.927 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 35328176, 2024-04-25T00:27:06.927 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-25T00:27:06.927 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:06.927 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:06.927 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:06.927 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:06.928 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.928 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-04-25T00:27:06.928 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-25T00:27:06.928 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-04-25T00:27:06.928 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-25T00:27:06.928 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-04-25T00:27:06.928 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-04-25T00:27:06.928 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:06.928 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-04-25T00:27:06.928 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 11891, 2024-04-25T00:27:06.928 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 11891, 2024-04-25T00:27:06.928 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 11891, 2024-04-25T00:27:06.928 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 11891, 2024-04-25T00:27:06.928 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-25T00:27:06.928 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:06.929 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:06.929 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:06.929 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:06.929 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.929 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:27:06.929 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-25T00:27:06.929 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-04-25T00:27:06.929 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-25T00:27:06.929 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 2282, 2024-04-25T00:27:06.929 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 266034, 2024-04-25T00:27:06.929 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:06.929 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 2282, 2024-04-25T00:27:06.929 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-04-25T00:27:06.929 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-04-25T00:27:06.929 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 154, 2024-04-25T00:27:06.929 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 266034, 2024-04-25T00:27:06.930 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-25T00:27:06.930 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:06.930 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:06.930 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:06.930 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:06.930 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:06.930 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x55852f824500": { 2024-04-25T00:27:06.930 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 60, 2024-04-25T00:27:06.930 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-04-25T00:27:06.930 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-25T00:27:06.930 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 12973, 2024-04-25T00:27:06.930 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 34094161, 2024-04-25T00:27:06.930 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:06.930 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 12973, 2024-04-25T00:27:06.930 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-04-25T00:27:06.931 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-04-25T00:27:06.931 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 2060, 2024-04-25T00:27:06.931 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 34094101, 2024-04-25T00:27:06.931 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-25T00:27:06.931 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:06.931 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:06.931 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:06.931 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:06.931 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:06.931 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-04-25T00:27:07.617 INFO:tasks.check_counter:Daemon mds.b mds.exported=223 2024-04-25T00:27:07.617 INFO:tasks.check_counter:Daemon mds.b mds.imported=83 2024-04-25T00:27:07.617 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-25T00:27:07.618 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-04-25T00:27:07.618 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-04-25T00:27:07.618 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T00:27:07.618 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T00:27:07.864 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:27:08.156 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:08 smithi170 ceph-mon[32807]: pgmap v331: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:27:08.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:08 smithi107 ceph-mon[29153]: pgmap v331: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:27:08.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:08 smithi195 ceph-mon[32833]: pgmap v331: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T00:27:09.322 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:09 smithi170 ceph-mon[32807]: osdmap e106: 12 total, 12 up, 12 in 2024-04-25T00:27:09.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:09 smithi107 ceph-mon[29153]: osdmap e106: 12 total, 12 up, 12 in 2024-04-25T00:27:09.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:09 smithi195 ceph-mon[32833]: osdmap e106: 12 total, 12 up, 12 in 2024-04-25T00:27:09.590 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-04-25T00:27:09.590 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:27:09.590 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 345, 2024-04-25T00:27:09.590 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 347, 2024-04-25T00:27:09.590 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 42043810, 2024-04-25T00:27:09.590 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 109774, 2024-04-25T00:27:09.590 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 7, 2024-04-25T00:27:09.590 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 3, 2024-04-25T00:27:09.590 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.050525492, 2024-04-25T00:27:09.590 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.009858500, 2024-04-25T00:27:09.590 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.029245095, 2024-04-25T00:27:09.591 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.007848923, 2024-04-25T00:27:09.591 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:09.591 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 345, 2024-04-25T00:27:09.591 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.013385884, 2024-04-25T00:27:09.591 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000038799 2024-04-25T00:27:09.591 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.591 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:09.591 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.591 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.591 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.591 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.591 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:27:09.591 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:27:09.591 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.592 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:27:09.592 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 1264, 2024-04-25T00:27:09.592 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 1429, 2024-04-25T00:27:09.592 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 77263686, 2024-04-25T00:27:09.592 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 850313, 2024-04-25T00:27:09.592 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 8, 2024-04-25T00:27:09.592 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 6, 2024-04-25T00:27:09.592 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.149028089, 2024-04-25T00:27:09.592 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.029969302, 2024-04-25T00:27:09.592 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.070628528, 2024-04-25T00:27:09.592 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.039812846, 2024-04-25T00:27:09.592 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:09.592 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1426, 2024-04-25T00:27:09.592 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.062969367, 2024-04-25T00:27:09.592 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000044158 2024-04-25T00:27:09.593 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.593 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:09.593 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.593 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.593 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.593 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.593 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T00:27:09.593 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 448992 2024-04-25T00:27:09.593 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.593 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:27:09.593 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 1144, 2024-04-25T00:27:09.593 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 1121, 2024-04-25T00:27:09.593 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 39557974, 2024-04-25T00:27:09.593 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 418044, 2024-04-25T00:27:09.593 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 5, 2024-04-25T00:27:09.594 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 4, 2024-04-25T00:27:09.594 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.164460622, 2024-04-25T00:27:09.594 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.043635113, 2024-04-25T00:27:09.594 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.080525946, 2024-04-25T00:27:09.594 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.039358111, 2024-04-25T00:27:09.594 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:09.594 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1121, 2024-04-25T00:27:09.594 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.059406029, 2024-04-25T00:27:09.594 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000052993 2024-04-25T00:27:09.594 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.594 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:09.594 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.595 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.595 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.595 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1521136, 2024-04-25T00:27:09.595 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 137568 2024-04-25T00:27:09.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.595 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-04-25T00:27:09.595 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-04-25T00:27:09.595 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-04-25T00:27:09.595 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.595 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-04-25T00:27:09.595 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-04-25T00:27:09.595 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-04-25T00:27:09.596 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 3930, 2024-04-25T00:27:09.596 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.115687076, 2024-04-25T00:27:09.596 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000029436 2024-04-25T00:27:09.596 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:09.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.596 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-04-25T00:27:09.596 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-04-25T00:27:09.596 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-04-25T00:27:09.596 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.596 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.596 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.596 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:09.596 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.596 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-04-25T00:27:09.597 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 0, 2024-04-25T00:27:09.597 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 0, 2024-04-25T00:27:09.597 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-04-25T00:27:09.597 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.597 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.597 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.597 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.597 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-04-25T00:27:09.597 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 0, 2024-04-25T00:27:09.597 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 0, 2024-04-25T00:27:09.597 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2024-04-25T00:27:09.597 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 0, 2024-04-25T00:27:09.597 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 0, 2024-04-25T00:27:09.598 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2024-04-25T00:27:09.598 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 34633, 2024-04-25T00:27:09.598 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 0, 2024-04-25T00:27:09.598 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 34633, 2024-04-25T00:27:09.598 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 0, 2024-04-25T00:27:09.598 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 34633, 2024-04-25T00:27:09.598 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 0, 2024-04-25T00:27:09.598 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 0, 2024-04-25T00:27:09.598 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 0, 2024-04-25T00:27:09.598 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 1, 2024-04-25T00:27:09.598 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 0, 2024-04-25T00:27:09.598 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 0, 2024-04-25T00:27:09.598 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 0, 2024-04-25T00:27:09.598 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 0, 2024-04-25T00:27:09.598 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:27:09.599 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2024-04-25T00:27:09.599 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 0, 2024-04-25T00:27:09.599 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2024-04-25T00:27:09.599 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-04-25T00:27:09.599 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 0, 2024-04-25T00:27:09.599 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 0, 2024-04-25T00:27:09.599 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 0, 2024-04-25T00:27:09.599 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 0, 2024-04-25T00:27:09.599 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-04-25T00:27:09.599 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:27:09.599 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2024-04-25T00:27:09.599 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 0, 2024-04-25T00:27:09.599 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 0, 2024-04-25T00:27:09.599 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 0, 2024-04-25T00:27:09.600 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:27:09.600 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-04-25T00:27:09.600 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:27:09.600 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:27:09.600 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:27:09.600 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:27:09.600 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:27:09.600 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 0, 2024-04-25T00:27:09.600 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:27:09.600 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:27:09.600 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:27:09.600 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:27:09.600 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:27:09.600 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:27:09.600 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 0, 2024-04-25T00:27:09.601 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:27:09.601 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 0, 2024-04-25T00:27:09.601 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 0 2024-04-25T00:27:09.601 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.601 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-04-25T00:27:09.601 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 0, 2024-04-25T00:27:09.601 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 0, 2024-04-25T00:27:09.601 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 0, 2024-04-25T00:27:09.601 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 0, 2024-04-25T00:27:09.601 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 0, 2024-04-25T00:27:09.601 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 0, 2024-04-25T00:27:09.601 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-04-25T00:27:09.601 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:27:09.601 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 0, 2024-04-25T00:27:09.601 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 0, 2024-04-25T00:27:09.602 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2024-04-25T00:27:09.602 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2024-04-25T00:27:09.602 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-04-25T00:27:09.602 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:27:09.602 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:27:09.602 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-04-25T00:27:09.602 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-04-25T00:27:09.602 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:27:09.602 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:27:09.602 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:27:09.602 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 0, 2024-04-25T00:27:09.602 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-04-25T00:27:09.602 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:27:09.602 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-04-25T00:27:09.602 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-04-25T00:27:09.603 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.603 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-04-25T00:27:09.603 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2024-04-25T00:27:09.603 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 0, 2024-04-25T00:27:09.603 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2024-04-25T00:27:09.603 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2024-04-25T00:27:09.603 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 60597, 2024-04-25T00:27:09.603 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-04-25T00:27:09.603 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2024-04-25T00:27:09.603 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 0, 2024-04-25T00:27:09.603 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2024-04-25T00:27:09.603 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2024-04-25T00:27:09.603 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 78, 2024-04-25T00:27:09.603 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 6, 2024-04-25T00:27:09.603 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-04-25T00:27:09.603 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2024-04-25T00:27:09.604 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2024-04-25T00:27:09.604 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 160827163, 2024-04-25T00:27:09.604 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 160822149, 2024-04-25T00:27:09.604 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-04-25T00:27:09.604 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.604 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.604 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.604 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.604 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 60597 2024-04-25T00:27:09.604 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.604 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-04-25T00:27:09.604 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 12790, 2024-04-25T00:27:09.604 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 24360, 2024-04-25T00:27:09.604 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 11570, 2024-04-25T00:27:09.605 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 1353, 2024-04-25T00:27:09.605 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 1905, 2024-04-25T00:27:09.605 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 552, 2024-04-25T00:27:09.605 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 34633, 2024-04-25T00:27:09.605 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 38199, 2024-04-25T00:27:09.605 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 3566, 2024-04-25T00:27:09.605 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 0, 2024-04-25T00:27:09.605 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 0, 2024-04-25T00:27:09.605 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 0, 2024-04-25T00:27:09.605 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 129348, 2024-04-25T00:27:09.605 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 215420 2024-04-25T00:27:09.605 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.605 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-04-25T00:27:09.605 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 0, 2024-04-25T00:27:09.605 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 0, 2024-04-25T00:27:09.605 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 0, 2024-04-25T00:27:09.606 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 0, 2024-04-25T00:27:09.606 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 0, 2024-04-25T00:27:09.606 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-04-25T00:27:09.606 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.606 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.606 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.606 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-04-25T00:27:09.606 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.606 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.606 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.606 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.606 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-04-25T00:27:09.606 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.607 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.607 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.607 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.607 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-04-25T00:27:09.607 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.607 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.607 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.607 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.607 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-04-25T00:27:09.607 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.607 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.607 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.607 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.607 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-04-25T00:27:09.607 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.608 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.608 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.608 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.608 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-04-25T00:27:09.608 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.608 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.608 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.608 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.608 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-04-25T00:27:09.608 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.608 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.608 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.608 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.608 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-04-25T00:27:09.608 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.609 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.609 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.609 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.609 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-04-25T00:27:09.609 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.609 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.609 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.609 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.609 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-04-25T00:27:09.609 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.609 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.609 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.609 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.610 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-04-25T00:27:09.610 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.610 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.610 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.610 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.610 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-04-25T00:27:09.610 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.611 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.611 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.611 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.611 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-04-25T00:27:09.611 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.611 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.611 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.611 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.611 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-04-25T00:27:09.611 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.611 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.611 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.611 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.611 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-04-25T00:27:09.612 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.612 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.612 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.612 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.612 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-04-25T00:27:09.612 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.612 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.612 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.612 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.612 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-04-25T00:27:09.612 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.612 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.612 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.612 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.612 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-04-25T00:27:09.612 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.613 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.613 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.613 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.613 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-04-25T00:27:09.613 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.613 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.613 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.613 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.613 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-04-25T00:27:09.613 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.613 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.613 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.613 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.613 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-04-25T00:27:09.613 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.614 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.614 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.614 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.614 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-04-25T00:27:09.614 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.614 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.614 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.614 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.614 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-04-25T00:27:09.614 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.614 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.614 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.614 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.614 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-04-25T00:27:09.615 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.615 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.615 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.615 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.615 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-04-25T00:27:09.615 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.615 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.615 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.615 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.615 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-04-25T00:27:09.615 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.615 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.615 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.615 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.615 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-04-25T00:27:09.615 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.616 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.616 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.616 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.616 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-04-25T00:27:09.616 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.616 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.616 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.616 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.616 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:27:09.616 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:27:09.616 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-04-25T00:27:09.616 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.616 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.616 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.616 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:09.617 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.617 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-04-25T00:27:09.617 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 2, 2024-04-25T00:27:09.617 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 5, 2024-04-25T00:27:09.617 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 3, 2024-04-25T00:27:09.617 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 5, 2024-04-25T00:27:09.617 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-04-25T00:27:09.617 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 0, 2024-04-25T00:27:09.617 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 0, 2024-04-25T00:27:09.617 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 3494, 2024-04-25T00:27:09.617 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-04-25T00:27:09.617 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.617 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-04-25T00:27:09.617 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:27:09.617 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-04-25T00:27:09.618 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:27:09.618 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:27:09.618 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:27:09.618 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:27:09.618 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:27:09.618 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:27:09.618 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:27:09.618 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:27:09.618 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:27:09.618 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:27:09.618 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:27:09.618 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:27:09.618 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:27:09.618 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-04-25T00:27:09.618 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:27:09.619 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-04-25T00:27:09.619 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:27:09.619 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:27:09.619 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:27:09.619 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:27:09.619 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:27:09.619 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:27:09.619 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:27:09.619 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-04-25T00:27:09.619 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:27:09.619 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:27:09.619 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:27:09.619 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-04-25T00:27:09.619 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-04-25T00:27:09.620 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-04-25T00:27:09.620 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:27:09.620 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:27:09.620 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:27:09.620 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:27:09.620 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 747755, 2024-04-25T00:27:09.620 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 14265, 2024-04-25T00:27:09.620 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:27:09.620 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2024-04-25T00:27:09.620 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-04-25T00:27:09.620 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-04-25T00:27:09.620 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:27:09.620 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-04-25T00:27:09.620 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:27:09.620 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-04-25T00:27:09.621 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 9432, 2024-04-25T00:27:09.621 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 182, 2024-04-25T00:27:09.621 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:27:09.621 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:27:09.621 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-04-25T00:27:09.621 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-04-25T00:27:09.621 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 57303907, 2024-04-25T00:27:09.621 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 738452, 2024-04-25T00:27:09.621 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-04-25T00:27:09.621 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-04-25T00:27:09.621 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-04-25T00:27:09.621 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-04-25T00:27:09.621 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.621 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-04-25T00:27:09.621 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-04-25T00:27:09.622 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-04-25T00:27:09.622 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 2489, 2024-04-25T00:27:09.622 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 21, 2024-04-25T00:27:09.622 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 3, 2024-04-25T00:27:09.622 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 2489, 2024-04-25T00:27:09.622 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-04-25T00:27:09.622 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2489, 2024-04-25T00:27:09.622 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 2.069952482, 2024-04-25T00:27:09.622 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000831640 2024-04-25T00:27:09.622 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.622 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-04-25T00:27:09.622 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-04-25T00:27:09.622 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2489, 2024-04-25T00:27:09.622 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 2490 2024-04-25T00:27:09.623 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.623 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 2489, 2024-04-25T00:27:09.623 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 2489, 2024-04-25T00:27:09.623 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 0, 2024-04-25T00:27:09.623 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-04-25T00:27:09.623 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-04-25T00:27:09.623 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 1592, 2024-04-25T00:27:09.623 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 0, 2024-04-25T00:27:09.623 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 895, 2024-04-25T00:27:09.623 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 0, 2024-04-25T00:27:09.623 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 0, 2024-04-25T00:27:09.623 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-04-25T00:27:09.623 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-04-25T00:27:09.623 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-04-25T00:27:09.623 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 0, 2024-04-25T00:27:09.624 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 0, 2024-04-25T00:27:09.624 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-04-25T00:27:09.624 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-04-25T00:27:09.624 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-04-25T00:27:09.624 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 1, 2024-04-25T00:27:09.624 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 0, 2024-04-25T00:27:09.624 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:27:09.624 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-04-25T00:27:09.624 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:27:09.624 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-04-25T00:27:09.624 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-04-25T00:27:09.624 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-04-25T00:27:09.624 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:27:09.624 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-04-25T00:27:09.625 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:27:09.625 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2024-04-25T00:27:09.625 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-04-25T00:27:09.625 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-04-25T00:27:09.625 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-04-25T00:27:09.625 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-04-25T00:27:09.625 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-04-25T00:27:09.625 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-04-25T00:27:09.625 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-04-25T00:27:09.625 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-04-25T00:27:09.625 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-04-25T00:27:09.625 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-04-25T00:27:09.625 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-04-25T00:27:09.625 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-04-25T00:27:09.625 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-04-25T00:27:09.626 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-04-25T00:27:09.626 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-04-25T00:27:09.626 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-04-25T00:27:09.626 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 106, 2024-04-25T00:27:09.626 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-04-25T00:27:09.626 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 14, 2024-04-25T00:27:09.626 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 9, 2024-04-25T00:27:09.626 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 9, 2024-04-25T00:27:09.626 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-04-25T00:27:09.626 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-04-25T00:27:09.626 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 0, 2024-04-25T00:27:09.626 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 2, 2024-04-25T00:27:09.626 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 0 2024-04-25T00:27:09.626 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.627 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-04-25T00:27:09.627 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 0, 2024-04-25T00:27:09.627 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:27:09.627 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-04-25T00:27:09.627 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-04-25T00:27:09.627 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.627 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-04-25T00:27:09.627 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 0 2024-04-25T00:27:09.627 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.627 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-04-25T00:27:09.627 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-04-25T00:27:09.627 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:27:09.627 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-04-25T00:27:09.627 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 0, 2024-04-25T00:27:09.628 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed_ops": 0, 2024-04-25T00:27:09.628 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 0, 2024-04-25T00:27:09.628 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-04-25T00:27:09.628 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.628 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:27:09.628 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-25T00:27:09.628 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-04-25T00:27:09.628 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-25T00:27:09.628 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 2749, 2024-04-25T00:27:09.628 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 158601896, 2024-04-25T00:27:09.628 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:09.628 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 2749, 2024-04-25T00:27:09.628 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-04-25T00:27:09.628 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-04-25T00:27:09.629 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 2749, 2024-04-25T00:27:09.629 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 158601896, 2024-04-25T00:27:09.629 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-25T00:27:09.629 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.629 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.629 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.629 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:09.629 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.629 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-04-25T00:27:09.629 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-25T00:27:09.629 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-04-25T00:27:09.629 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-25T00:27:09.629 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-04-25T00:27:09.629 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-04-25T00:27:09.629 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:09.630 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-04-25T00:27:09.630 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 2489, 2024-04-25T00:27:09.630 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 156646664, 2024-04-25T00:27:09.630 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 2489, 2024-04-25T00:27:09.630 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 156646664, 2024-04-25T00:27:09.630 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-25T00:27:09.630 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.630 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.630 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.630 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:09.630 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.630 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-04-25T00:27:09.630 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-25T00:27:09.630 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-04-25T00:27:09.631 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-25T00:27:09.631 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-04-25T00:27:09.631 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-04-25T00:27:09.631 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:09.631 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-04-25T00:27:09.631 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 2489, 2024-04-25T00:27:09.631 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 2489, 2024-04-25T00:27:09.631 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 2489, 2024-04-25T00:27:09.631 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 2489, 2024-04-25T00:27:09.631 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-25T00:27:09.631 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.631 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.631 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.631 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:09.632 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.632 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:27:09.632 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-25T00:27:09.632 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-04-25T00:27:09.632 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-25T00:27:09.632 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-04-25T00:27:09.632 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-04-25T00:27:09.632 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:09.632 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-04-25T00:27:09.632 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-04-25T00:27:09.632 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-04-25T00:27:09.632 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2024-04-25T00:27:09.632 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2024-04-25T00:27:09.632 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-25T00:27:09.632 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.633 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.633 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.633 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:09.633 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:09.633 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x55b69b0a66e0": { 2024-04-25T00:27:09.633 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-25T00:27:09.633 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-04-25T00:27:09.633 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-25T00:27:09.633 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-04-25T00:27:09.633 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-04-25T00:27:09.633 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:09.633 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-04-25T00:27:09.633 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-04-25T00:27:09.633 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-04-25T00:27:09.633 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 0, 2024-04-25T00:27:09.634 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 0, 2024-04-25T00:27:09.634 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-25T00:27:09.634 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:09.634 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:09.634 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:09.634 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:09.634 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:09.634 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-04-25T00:27:10.411 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:10 smithi170 ceph-mon[32807]: pgmap v333: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T00:27:10.440 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-25T00:27:10.440 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-25T00:27:10.440 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-04-25T00:27:10.441 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-25T00:27:10.441 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T00:27:10.441 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T00:27:10.441 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T00:27:10.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:10 smithi107 ceph-mon[29153]: pgmap v333: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T00:27:10.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:10 smithi195 ceph-mon[32833]: pgmap v333: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T00:27:10.681 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:27:12.398 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:12 smithi170 ceph-mon[32807]: pgmap v334: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T00:27:12.424 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-04-25T00:27:12.424 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:27:12.424 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 87902, 2024-04-25T00:27:12.424 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 139413, 2024-04-25T00:27:12.424 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 28986058, 2024-04-25T00:27:12.424 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 138723264, 2024-04-25T00:27:12.424 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 10, 2024-04-25T00:27:12.425 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 4, 2024-04-25T00:27:12.425 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 5.302812461, 2024-04-25T00:27:12.425 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 2.341588362, 2024-04-25T00:27:12.425 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 4.572347612, 2024-04-25T00:27:12.425 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.639411337, 2024-04-25T00:27:12.425 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:12.425 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 139415, 2024-04-25T00:27:12.425 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 4.943411540, 2024-04-25T00:27:12.425 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000035458 2024-04-25T00:27:12.425 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.425 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:12.425 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 160960, 2024-04-25T00:27:12.426 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.023784312, 2024-04-25T00:27:12.426 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000147 2024-04-25T00:27:12.426 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.426 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:27:12.426 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:27:12.426 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.426 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:27:12.426 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 15241, 2024-04-25T00:27:12.426 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 15438, 2024-04-25T00:27:12.426 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 4110947, 2024-04-25T00:27:12.426 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 58502036, 2024-04-25T00:27:12.426 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 45, 2024-04-25T00:27:12.426 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 7, 2024-04-25T00:27:12.426 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 1.196021545, 2024-04-25T00:27:12.427 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.455594994, 2024-04-25T00:27:12.427 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.251158416, 2024-04-25T00:27:12.427 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.386810227, 2024-04-25T00:27:12.427 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:12.427 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 15435, 2024-04-25T00:27:12.427 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.169259186, 2024-04-25T00:27:12.427 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000075753 2024-04-25T00:27:12.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.427 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:12.427 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 98, 2024-04-25T00:27:12.427 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000016177, 2024-04-25T00:27:12.427 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000165 2024-04-25T00:27:12.427 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.427 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 4896, 2024-04-25T00:27:12.428 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 462480 2024-04-25T00:27:12.428 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.428 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:27:12.428 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 19779, 2024-04-25T00:27:12.428 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 22451, 2024-04-25T00:27:12.428 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 8140655, 2024-04-25T00:27:12.428 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 31563679, 2024-04-25T00:27:12.428 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 20, 2024-04-25T00:27:12.428 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 7, 2024-04-25T00:27:12.428 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 1.486824954, 2024-04-25T00:27:12.428 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.577628721, 2024-04-25T00:27:12.428 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.527633950, 2024-04-25T00:27:12.428 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.347799048, 2024-04-25T00:27:12.428 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:12.429 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 22447, 2024-04-25T00:27:12.429 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.357426471, 2024-04-25T00:27:12.429 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000060472 2024-04-25T00:27:12.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.429 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:12.429 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 15117, 2024-04-25T00:27:12.429 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.002325563, 2024-04-25T00:27:12.429 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000153 2024-04-25T00:27:12.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.429 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1535696, 2024-04-25T00:27:12.429 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 150640 2024-04-25T00:27:12.429 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.429 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-04-25T00:27:12.429 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-04-25T00:27:12.430 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-04-25T00:27:12.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.430 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-04-25T00:27:12.430 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-04-25T00:27:12.430 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-04-25T00:27:12.430 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 20662, 2024-04-25T00:27:12.430 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 40.263854108, 2024-04-25T00:27:12.430 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001948691 2024-04-25T00:27:12.430 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:12.430 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.430 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-04-25T00:27:12.430 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-04-25T00:27:12.430 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-04-25T00:27:12.430 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 17873, 2024-04-25T00:27:12.430 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 3.999576895, 2024-04-25T00:27:12.431 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000223777 2024-04-25T00:27:12.431 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:12.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.431 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-04-25T00:27:12.431 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 31830, 2024-04-25T00:27:12.431 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 31657, 2024-04-25T00:27:12.431 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-04-25T00:27:12.431 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 31657, 2024-04-25T00:27:12.431 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 74.827121691, 2024-04-25T00:27:12.431 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002363683 2024-04-25T00:27:12.431 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.431 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-04-25T00:27:12.431 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 173, 2024-04-25T00:27:12.431 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 71, 2024-04-25T00:27:12.432 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2024-04-25T00:27:12.432 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 297, 2024-04-25T00:27:12.432 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 34, 2024-04-25T00:27:12.432 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2024-04-25T00:27:12.432 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 23694, 2024-04-25T00:27:12.432 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 1242, 2024-04-25T00:27:12.432 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 21277, 2024-04-25T00:27:12.432 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 1175, 2024-04-25T00:27:12.432 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 21916, 2024-04-25T00:27:12.432 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 1614, 2024-04-25T00:27:12.432 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 1, 2024-04-25T00:27:12.432 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 2, 2024-04-25T00:27:12.432 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 60, 2024-04-25T00:27:12.432 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 40546, 2024-04-25T00:27:12.432 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 39210, 2024-04-25T00:27:12.433 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 78, 2024-04-25T00:27:12.433 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 277, 2024-04-25T00:27:12.433 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:27:12.433 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 2, 2024-04-25T00:27:12.433 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 15, 2024-04-25T00:27:12.433 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2024-04-25T00:27:12.433 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-04-25T00:27:12.433 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 127, 2024-04-25T00:27:12.433 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 1184, 2024-04-25T00:27:12.433 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 261, 2024-04-25T00:27:12.433 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 2938, 2024-04-25T00:27:12.433 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-04-25T00:27:12.433 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 25643, 2024-04-25T00:27:12.433 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 4, 2024-04-25T00:27:12.433 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 4, 2024-04-25T00:27:12.434 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 234, 2024-04-25T00:27:12.434 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 6, 2024-04-25T00:27:12.434 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 25643, 2024-04-25T00:27:12.434 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-04-25T00:27:12.434 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:27:12.434 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:27:12.434 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 3437, 2024-04-25T00:27:12.434 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:27:12.434 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 3437, 2024-04-25T00:27:12.434 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 22206, 2024-04-25T00:27:12.434 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 309, 2024-04-25T00:27:12.434 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 3751, 2024-04-25T00:27:12.434 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 24754, 2024-04-25T00:27:12.434 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 156, 2024-04-25T00:27:12.435 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:27:12.435 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:27:12.435 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 19231, 2024-04-25T00:27:12.435 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 15012, 2024-04-25T00:27:12.435 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 201, 2024-04-25T00:27:12.435 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 17181 2024-04-25T00:27:12.435 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.435 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-04-25T00:27:12.436 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 0, 2024-04-25T00:27:12.436 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 0, 2024-04-25T00:27:12.436 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 0, 2024-04-25T00:27:12.436 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 293, 2024-04-25T00:27:12.436 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 297, 2024-04-25T00:27:12.436 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 1111, 2024-04-25T00:27:12.436 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-04-25T00:27:12.436 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:27:12.436 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 11631, 2024-04-25T00:27:12.436 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 10459, 2024-04-25T00:27:12.436 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 66, 2024-04-25T00:27:12.436 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 5, 2024-04-25T00:27:12.436 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-04-25T00:27:12.436 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:27:12.436 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:27:12.437 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-04-25T00:27:12.437 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-04-25T00:27:12.437 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:27:12.437 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:27:12.437 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:27:12.437 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 127, 2024-04-25T00:27:12.437 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-04-25T00:27:12.437 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 34, 2024-04-25T00:27:12.437 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-04-25T00:27:12.437 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-04-25T00:27:12.437 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.437 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-04-25T00:27:12.437 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2024-04-25T00:27:12.437 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 60597, 2024-04-25T00:27:12.438 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2024-04-25T00:27:12.438 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2024-04-25T00:27:12.438 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 60597, 2024-04-25T00:27:12.438 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-04-25T00:27:12.438 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2024-04-25T00:27:12.438 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 78, 2024-04-25T00:27:12.438 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2024-04-25T00:27:12.438 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2024-04-25T00:27:12.438 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 78, 2024-04-25T00:27:12.438 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 6, 2024-04-25T00:27:12.438 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-04-25T00:27:12.438 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2024-04-25T00:27:12.438 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2024-04-25T00:27:12.438 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 160827163, 2024-04-25T00:27:12.439 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 0, 2024-04-25T00:27:12.439 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-04-25T00:27:12.439 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4973, 2024-04-25T00:27:12.439 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 184.988745345, 2024-04-25T00:27:12.439 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.037198621 2024-04-25T00:27:12.439 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.439 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 0 2024-04-25T00:27:12.441 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.441 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-04-25T00:27:12.441 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 12803, 2024-04-25T00:27:12.441 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 24542, 2024-04-25T00:27:12.442 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 11739, 2024-04-25T00:27:12.442 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 1371, 2024-04-25T00:27:12.442 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 1953, 2024-04-25T00:27:12.442 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 582, 2024-04-25T00:27:12.442 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 23694, 2024-04-25T00:27:12.442 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 38638, 2024-04-25T00:27:12.442 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 14944, 2024-04-25T00:27:12.442 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 2, 2024-04-25T00:27:12.442 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 13276, 2024-04-25T00:27:12.442 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 13274, 2024-04-25T00:27:12.445 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 187460, 2024-04-25T00:27:12.446 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 223612 2024-04-25T00:27:12.446 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.446 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-04-25T00:27:12.446 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 39805, 2024-04-25T00:27:12.446 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 2093, 2024-04-25T00:27:12.446 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 31830, 2024-04-25T00:27:12.446 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 383, 2024-04-25T00:27:12.446 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 3757, 2024-04-25T00:27:12.446 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-04-25T00:27:12.446 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 8974, 2024-04-25T00:27:12.446 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 11.991339709, 2024-04-25T00:27:12.446 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001336231 2024-04-25T00:27:12.446 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.446 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-04-25T00:27:12.447 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 110, 2024-04-25T00:27:12.447 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.627805136, 2024-04-25T00:27:12.447 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.005707319 2024-04-25T00:27:12.447 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.447 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-04-25T00:27:12.447 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:12.447 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:12.447 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:12.447 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.447 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-04-25T00:27:12.447 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 165, 2024-04-25T00:27:12.447 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.674943753, 2024-04-25T00:27:12.447 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.004090568 2024-04-25T00:27:12.447 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.448 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-04-25T00:27:12.448 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 6293, 2024-04-25T00:27:12.448 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 3.378711791, 2024-04-25T00:27:12.448 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000536900 2024-04-25T00:27:12.448 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.448 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-04-25T00:27:12.448 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:12.448 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:12.448 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:12.448 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.448 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-04-25T00:27:12.448 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1, 2024-04-25T00:27:12.448 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000104078, 2024-04-25T00:27:12.448 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000104078 2024-04-25T00:27:12.448 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.449 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-04-25T00:27:12.449 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:12.449 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:12.449 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:12.449 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.449 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-04-25T00:27:12.449 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:12.449 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:12.449 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:12.449 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.449 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-04-25T00:27:12.449 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:12.449 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:12.450 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:12.450 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.450 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-04-25T00:27:12.450 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:12.450 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:12.450 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:12.451 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.451 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-04-25T00:27:12.451 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1303, 2024-04-25T00:27:12.451 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 2.618184768, 2024-04-25T00:27:12.451 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002009351 2024-04-25T00:27:12.451 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.451 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-04-25T00:27:12.451 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 345, 2024-04-25T00:27:12.451 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.991455567, 2024-04-25T00:27:12.451 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002873784 2024-04-25T00:27:12.451 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.451 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-04-25T00:27:12.451 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:12.451 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:12.452 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:12.452 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.452 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-04-25T00:27:12.452 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 45, 2024-04-25T00:27:12.452 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 8.409347956, 2024-04-25T00:27:12.452 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.186874399 2024-04-25T00:27:12.452 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.452 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-04-25T00:27:12.452 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1701, 2024-04-25T00:27:12.452 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 2.298185174, 2024-04-25T00:27:12.452 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001351078 2024-04-25T00:27:12.452 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.452 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-04-25T00:27:12.452 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 389, 2024-04-25T00:27:12.453 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 2.960340112, 2024-04-25T00:27:12.453 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.007610128 2024-04-25T00:27:12.453 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.453 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-04-25T00:27:12.453 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:12.453 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:12.453 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:12.453 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.453 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-04-25T00:27:12.453 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:12.453 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:12.453 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:12.453 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.453 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-04-25T00:27:12.454 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1667, 2024-04-25T00:27:12.454 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 21.366714196, 2024-04-25T00:27:12.454 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.012817465 2024-04-25T00:27:12.454 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.454 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-04-25T00:27:12.454 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:12.454 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:12.454 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:12.454 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.454 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-04-25T00:27:12.454 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:12.455 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:12.455 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:12.455 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.455 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-04-25T00:27:12.455 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 158, 2024-04-25T00:27:12.455 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.736107060, 2024-04-25T00:27:12.455 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.010988019 2024-04-25T00:27:12.455 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.455 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-04-25T00:27:12.455 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:12.455 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:12.455 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:12.455 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.456 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-04-25T00:27:12.456 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:12.456 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:12.456 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:12.456 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.456 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-04-25T00:27:12.456 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:12.456 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:12.456 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:12.456 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.456 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-04-25T00:27:12.456 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:12.456 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:12.456 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:12.456 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.457 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-04-25T00:27:12.457 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 302, 2024-04-25T00:27:12.457 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.953655447, 2024-04-25T00:27:12.457 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.003157799 2024-04-25T00:27:12.457 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.457 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-04-25T00:27:12.457 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 10204, 2024-04-25T00:27:12.457 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 16.820226944, 2024-04-25T00:27:12.457 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001648395 2024-04-25T00:27:12.457 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.457 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:27:12.457 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:27:12.457 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-04-25T00:27:12.457 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:12.458 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:12.458 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:12.458 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:12.458 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.458 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-04-25T00:27:12.458 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 2, 2024-04-25T00:27:12.458 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 5, 2024-04-25T00:27:12.458 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 3, 2024-04-25T00:27:12.458 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 2, 2024-04-25T00:27:12.458 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-04-25T00:27:12.458 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 18733, 2024-04-25T00:27:12.459 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 9366, 2024-04-25T00:27:12.459 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 622, 2024-04-25T00:27:12.459 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-04-25T00:27:12.459 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.459 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-04-25T00:27:12.459 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:27:12.459 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-04-25T00:27:12.459 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:27:12.460 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:27:12.460 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:27:12.460 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:27:12.460 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:27:12.460 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:27:12.460 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:27:12.460 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:27:12.460 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:27:12.460 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:27:12.460 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:27:12.460 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:27:12.460 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:27:12.460 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-04-25T00:27:12.460 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:27:12.461 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-04-25T00:27:12.461 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:27:12.461 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:27:12.461 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:27:12.461 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:27:12.461 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:27:12.461 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:27:12.461 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:27:12.461 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-04-25T00:27:12.461 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:27:12.461 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:27:12.461 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:27:12.461 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-04-25T00:27:12.461 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-04-25T00:27:12.462 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-04-25T00:27:12.462 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:27:12.462 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:27:12.462 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:27:12.462 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:27:12.462 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 9206422, 2024-04-25T00:27:12.462 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 12125, 2024-04-25T00:27:12.462 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:27:12.462 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2024-04-25T00:27:12.462 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-04-25T00:27:12.462 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-04-25T00:27:12.462 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:27:12.462 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-04-25T00:27:12.462 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:27:12.462 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-04-25T00:27:12.463 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 9672, 2024-04-25T00:27:12.463 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 188, 2024-04-25T00:27:12.463 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:27:12.463 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:27:12.463 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-04-25T00:27:12.463 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-04-25T00:27:12.463 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 49757593, 2024-04-25T00:27:12.463 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 724941, 2024-04-25T00:27:12.463 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-04-25T00:27:12.463 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-04-25T00:27:12.463 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-04-25T00:27:12.463 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-04-25T00:27:12.463 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.463 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-04-25T00:27:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-04-25T00:27:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-04-25T00:27:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 52628, 2024-04-25T00:27:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 163878010, 2024-04-25T00:27:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 33, 2024-04-25T00:27:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 52628, 2024-04-25T00:27:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-04-25T00:27:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 52628, 2024-04-25T00:27:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 140.770764202, 2024-04-25T00:27:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002674826 2024-04-25T00:27:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-04-25T00:27:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-04-25T00:27:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 52628, 2024-04-25T00:27:12.464 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 53216 2024-04-25T00:27:12.465 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.465 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 52628, 2024-04-25T00:27:12.465 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 26068, 2024-04-25T00:27:12.465 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 26560, 2024-04-25T00:27:12.465 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-04-25T00:27:12.465 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-04-25T00:27:12.465 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 0, 2024-04-25T00:27:12.465 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 10, 2024-04-25T00:27:12.465 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 248, 2024-04-25T00:27:12.465 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 5221, 2024-04-25T00:27:12.465 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 119, 2024-04-25T00:27:12.465 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-04-25T00:27:12.465 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-04-25T00:27:12.465 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-04-25T00:27:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 18, 2024-04-25T00:27:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 20602, 2024-04-25T00:27:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-04-25T00:27:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-04-25T00:27:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-04-25T00:27:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 25643, 2024-04-25T00:27:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 20, 2024-04-25T00:27:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:27:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-04-25T00:27:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:27:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-04-25T00:27:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-04-25T00:27:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-04-25T00:27:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:27:12.466 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-04-25T00:27:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:27:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 184, 2024-04-25T00:27:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-04-25T00:27:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-04-25T00:27:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-04-25T00:27:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-04-25T00:27:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-04-25T00:27:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-04-25T00:27:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-04-25T00:27:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-04-25T00:27:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-04-25T00:27:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-04-25T00:27:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-04-25T00:27:12.467 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-04-25T00:27:12.469 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-04-25T00:27:12.469 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-04-25T00:27:12.469 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-04-25T00:27:12.469 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-04-25T00:27:12.469 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 106, 2024-04-25T00:27:12.469 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-04-25T00:27:12.469 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 14, 2024-04-25T00:27:12.469 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 12, 2024-04-25T00:27:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 12, 2024-04-25T00:27:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-04-25T00:27:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-04-25T00:27:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 793, 2024-04-25T00:27:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 248, 2024-04-25T00:27:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 110 2024-04-25T00:27:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-04-25T00:27:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 1, 2024-04-25T00:27:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:27:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-04-25T00:27:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-04-25T00:27:12.470 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.471 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-04-25T00:27:12.471 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 0 2024-04-25T00:27:12.471 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.471 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-04-25T00:27:12.471 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-04-25T00:27:12.471 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 23, 2024-04-25T00:27:12.471 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-04-25T00:27:12.471 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 16, 2024-04-25T00:27:12.471 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed_ops": 20537, 2024-04-25T00:27:12.471 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 10459, 2024-04-25T00:27:12.472 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-04-25T00:27:12.472 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.472 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:27:12.472 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-25T00:27:12.472 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-04-25T00:27:12.472 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-25T00:27:12.472 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 122918, 2024-04-25T00:27:12.472 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 30155655, 2024-04-25T00:27:12.472 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:12.472 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 122918, 2024-04-25T00:27:12.472 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-04-25T00:27:12.472 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-04-25T00:27:12.472 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 122918, 2024-04-25T00:27:12.473 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 30155655, 2024-04-25T00:27:12.473 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-25T00:27:12.473 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:12.473 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:12.473 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:12.473 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:12.473 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.473 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-04-25T00:27:12.473 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-25T00:27:12.473 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-04-25T00:27:12.474 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-25T00:27:12.474 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-04-25T00:27:12.474 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-04-25T00:27:12.474 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:12.474 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-04-25T00:27:12.474 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 52628, 2024-04-25T00:27:12.474 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 165097255, 2024-04-25T00:27:12.474 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 52628, 2024-04-25T00:27:12.474 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 165097255, 2024-04-25T00:27:12.474 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-25T00:27:12.474 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:12.474 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:12.475 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:12.475 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:12.475 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.475 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-04-25T00:27:12.475 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-25T00:27:12.475 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-04-25T00:27:12.475 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-25T00:27:12.475 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-04-25T00:27:12.475 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-04-25T00:27:12.475 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:12.476 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-04-25T00:27:12.476 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 52628, 2024-04-25T00:27:12.476 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 52628, 2024-04-25T00:27:12.476 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 52628, 2024-04-25T00:27:12.476 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 52628, 2024-04-25T00:27:12.476 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-25T00:27:12.476 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:12.476 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:12.476 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:12.476 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:27:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-25T00:27:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-04-25T00:27:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-25T00:27:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 10459, 2024-04-25T00:27:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 1224471, 2024-04-25T00:27:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 10459, 2024-04-25T00:27:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-04-25T00:27:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-04-25T00:27:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 248, 2024-04-25T00:27:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 1224471, 2024-04-25T00:27:12.477 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-25T00:27:12.478 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:12.478 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:12.478 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:12.478 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:12.478 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:12.478 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x5641d35b8500": { 2024-04-25T00:27:12.478 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-25T00:27:12.478 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-04-25T00:27:12.478 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-25T00:27:12.478 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 60597, 2024-04-25T00:27:12.478 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 156632859, 2024-04-25T00:27:12.478 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:12.478 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 60597, 2024-04-25T00:27:12.479 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-04-25T00:27:12.479 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-04-25T00:27:12.479 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 4973, 2024-04-25T00:27:12.479 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 156632859, 2024-04-25T00:27:12.479 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-25T00:27:12.479 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:12.479 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:12.479 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:12.479 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:12.479 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:12.479 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-04-25T00:27:12.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:12 smithi107 ceph-mon[29153]: pgmap v334: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T00:27:12.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:12 smithi195 ceph-mon[32833]: pgmap v334: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T00:27:13.231 INFO:tasks.check_counter:Daemon mds.h mds.exported=127 2024-04-25T00:27:13.231 INFO:tasks.check_counter:Daemon mds.h mds.imported=261 2024-04-25T00:27:13.232 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-25T00:27:13.232 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-25T00:27:13.232 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T00:27:13.232 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T00:27:13.232 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T00:27:13.470 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:27:14.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:14 smithi170 ceph-mon[32807]: pgmap v335: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:27:14.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:14 smithi107 ceph-mon[29153]: pgmap v335: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:27:14.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:14 smithi195 ceph-mon[32833]: pgmap v335: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:27:15.182 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-04-25T00:27:15.182 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:27:15.183 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 4, 2024-04-25T00:27:15.183 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 4, 2024-04-25T00:27:15.183 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 3024, 2024-04-25T00:27:15.183 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 448, 2024-04-25T00:27:15.183 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 4, 2024-04-25T00:27:15.183 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 0, 2024-04-25T00:27:15.183 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.001256153, 2024-04-25T00:27:15.183 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.000034192, 2024-04-25T00:27:15.183 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.000506646, 2024-04-25T00:27:15.183 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:27:15.183 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:15.183 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 3, 2024-04-25T00:27:15.183 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000715602, 2024-04-25T00:27:15.184 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000238534 2024-04-25T00:27:15.184 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:15.184 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:15.184 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:15.184 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:15.184 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:15.184 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:15.184 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-25T00:27:15.184 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:27:15.184 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:15.184 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:27:15.184 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 2, 2024-04-25T00:27:15.184 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 169, 2024-04-25T00:27:15.184 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 320, 2024-04-25T00:27:15.185 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 185152, 2024-04-25T00:27:15.185 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 5, 2024-04-25T00:27:15.185 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 3, 2024-04-25T00:27:15.185 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.011465803, 2024-04-25T00:27:15.185 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.007181458, 2024-04-25T00:27:15.185 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.001915136, 2024-04-25T00:27:15.185 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:27:15.185 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:15.185 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 166, 2024-04-25T00:27:15.185 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.020221556, 2024-04-25T00:27:15.185 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000121816 2024-04-25T00:27:15.185 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:15.185 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:15.186 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:15.186 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:15.186 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:15.186 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:15.186 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T00:27:15.186 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 185152 2024-04-25T00:27:15.186 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:15.186 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:27:15.186 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 232, 2024-04-25T00:27:15.186 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 222, 2024-04-25T00:27:15.186 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 1446000, 2024-04-25T00:27:15.186 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 136384, 2024-04-25T00:27:15.186 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 2, 2024-04-25T00:27:15.187 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 1, 2024-04-25T00:27:15.187 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.045272624, 2024-04-25T00:27:15.187 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.013302916, 2024-04-25T00:27:15.187 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.034886689, 2024-04-25T00:27:15.187 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.005329252, 2024-04-25T00:27:15.187 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:15.187 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 221, 2024-04-25T00:27:15.187 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.012468018, 2024-04-25T00:27:15.187 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000056416 2024-04-25T00:27:15.187 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:15.187 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:15.187 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:15.187 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:15.187 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:15.188 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:15.188 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1446000, 2024-04-25T00:27:15.188 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 136384 2024-04-25T00:27:15.188 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:15.188 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-04-25T00:27:15.188 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 0, 2024-04-25T00:27:15.188 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-04-25T00:27:15.188 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:15.188 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-04-25T00:27:15.188 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:27:15.188 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-04-25T00:27:15.188 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:27:15.188 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:27:15.188 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:27:15.189 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:27:15.189 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:27:15.189 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:27:15.189 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:27:15.189 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:27:15.189 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:27:15.189 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:27:15.189 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:27:15.189 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:27:15.189 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:27:15.189 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-04-25T00:27:15.189 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:27:15.189 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-04-25T00:27:15.189 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:27:15.190 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:27:15.190 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:27:15.190 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:27:15.190 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:27:15.190 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:27:15.190 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:27:15.190 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-04-25T00:27:15.190 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:27:15.190 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:27:15.190 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:27:15.190 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-04-25T00:27:15.190 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-04-25T00:27:15.190 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-04-25T00:27:15.190 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:27:15.190 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:27:15.191 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:27:15.191 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:27:15.191 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 148972, 2024-04-25T00:27:15.191 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 31, 2024-04-25T00:27:15.191 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:27:15.191 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2024-04-25T00:27:15.191 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-04-25T00:27:15.191 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-04-25T00:27:15.191 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:27:15.191 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-04-25T00:27:15.191 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:27:15.191 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-04-25T00:27:15.191 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 0, 2024-04-25T00:27:15.192 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 0, 2024-04-25T00:27:15.192 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:27:15.192 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:27:15.192 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-04-25T00:27:15.192 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-04-25T00:27:15.192 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 928, 2024-04-25T00:27:15.192 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 2, 2024-04-25T00:27:15.192 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-04-25T00:27:15.192 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-04-25T00:27:15.192 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-04-25T00:27:15.192 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-04-25T00:27:15.192 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-04-25T00:27:15.192 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:27:15.192 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-04-25T00:27:15.193 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-04-25T00:27:15.193 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-04-25T00:27:15.193 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 234, 2024-04-25T00:27:15.193 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 1413308, 2024-04-25T00:27:15.193 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:15.193 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 234, 2024-04-25T00:27:15.193 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-04-25T00:27:15.193 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-04-25T00:27:15.193 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 234, 2024-04-25T00:27:15.193 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 1413308, 2024-04-25T00:27:15.193 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-04-25T00:27:15.193 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-04-25T00:27:15.193 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-04-25T00:27:15.194 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-04-25T00:27:15.194 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:15.194 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-04-25T00:27:15.194 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-04-25T00:27:15.941 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-25T00:27:15.942 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-25T00:27:15.942 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-25T00:27:15.942 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-25T00:27:15.942 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-25T00:27:15.942 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T00:27:15.942 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T00:27:16.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:16 smithi170 ceph-mon[32807]: pgmap v336: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:27:16.194 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:27:16.314 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:16 smithi195 ceph-mon[32833]: pgmap v336: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:27:16.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:16 smithi107 ceph-mon[29153]: pgmap v336: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:27:17.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:17 smithi107 ceph-mon[29153]: pgmap v337: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:27:17.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:17 smithi107 ceph-mon[29153]: osdmap e107: 12 total, 12 up, 12 in 2024-04-25T00:27:18.005 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:17 smithi195 ceph-mon[32833]: pgmap v337: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:27:18.005 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:17 smithi195 ceph-mon[32833]: osdmap e107: 12 total, 12 up, 12 in 2024-04-25T00:27:18.024 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-04-25T00:27:18.024 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:27:18.024 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 2016, 2024-04-25T00:27:18.024 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 2018, 2024-04-25T00:27:18.024 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 9505683, 2024-04-25T00:27:18.024 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 641098, 2024-04-25T00:27:18.024 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 7, 2024-04-25T00:27:18.024 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 3, 2024-04-25T00:27:18.024 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.184720801, 2024-04-25T00:27:18.025 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.057058147, 2024-04-25T00:27:18.025 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.050279495, 2024-04-25T00:27:18.025 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.061111275, 2024-04-25T00:27:18.025 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:18.025 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2016, 2024-04-25T00:27:18.025 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.077532265, 2024-04-25T00:27:18.025 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000038458 2024-04-25T00:27:18.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.025 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:18.025 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.025 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.025 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.025 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:27:18.025 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:27:18.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.026 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:27:18.026 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 423, 2024-04-25T00:27:18.026 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 402, 2024-04-25T00:27:18.026 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 18338108, 2024-04-25T00:27:18.026 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 189678, 2024-04-25T00:27:18.026 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 4, 2024-04-25T00:27:18.026 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 3, 2024-04-25T00:27:18.026 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.066457394, 2024-04-25T00:27:18.026 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.018532832, 2024-04-25T00:27:18.026 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.046147365, 2024-04-25T00:27:18.026 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.009407047, 2024-04-25T00:27:18.026 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:18.026 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 401, 2024-04-25T00:27:18.027 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.020578641, 2024-04-25T00:27:18.027 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000051318 2024-04-25T00:27:18.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.027 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:18.027 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.027 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.027 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.027 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 1519056, 2024-04-25T00:27:18.027 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 138096 2024-04-25T00:27:18.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.027 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:27:18.027 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 291, 2024-04-25T00:27:18.027 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 455, 2024-04-25T00:27:18.028 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 8464226, 2024-04-25T00:27:18.028 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 543479, 2024-04-25T00:27:18.028 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 7, 2024-04-25T00:27:18.028 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 5, 2024-04-25T00:27:18.028 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.040648868, 2024-04-25T00:27:18.028 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.015423966, 2024-04-25T00:27:18.028 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.012777450, 2024-04-25T00:27:18.028 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.008411298, 2024-04-25T00:27:18.028 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:18.028 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 453, 2024-04-25T00:27:18.028 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.030137282, 2024-04-25T00:27:18.028 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000066528 2024-04-25T00:27:18.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.028 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:18.028 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.029 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.029 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.029 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-25T00:27:18.029 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 452848 2024-04-25T00:27:18.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.029 INFO:teuthology.orchestra.run.smithi195.stdout: "cct": { 2024-04-25T00:27:18.029 INFO:teuthology.orchestra.run.smithi195.stdout: "total_workers": 1, 2024-04-25T00:27:18.029 INFO:teuthology.orchestra.run.smithi195.stdout: "unhealthy_workers": 0 2024-04-25T00:27:18.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.029 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-MDSRank": { 2024-04-25T00:27:18.029 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-04-25T00:27:18.029 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-04-25T00:27:18.029 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 4105, 2024-04-25T00:27:18.029 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.115417646, 2024-04-25T00:27:18.029 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000028116 2024-04-25T00:27:18.030 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:18.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.030 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-PurgeQueue": { 2024-04-25T00:27:18.030 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-04-25T00:27:18.030 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-04-25T00:27:18.030 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.030 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.030 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.030 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:18.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.030 INFO:teuthology.orchestra.run.smithi195.stdout: "mds": { 2024-04-25T00:27:18.030 INFO:teuthology.orchestra.run.smithi195.stdout: "request": 0, 2024-04-25T00:27:18.030 INFO:teuthology.orchestra.run.smithi195.stdout: "reply": 0, 2024-04-25T00:27:18.030 INFO:teuthology.orchestra.run.smithi195.stdout: "reply_latency": { 2024-04-25T00:27:18.031 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.031 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.031 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.031 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_reply": 0, 2024-04-25T00:27:18.031 INFO:teuthology.orchestra.run.smithi195.stdout: "forward": 0, 2024-04-25T00:27:18.031 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_complete": 0, 2024-04-25T00:27:18.031 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_keys": 0, 2024-04-25T00:27:18.031 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_commit": 0, 2024-04-25T00:27:18.031 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_split": 0, 2024-04-25T00:27:18.031 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_merge": 0, 2024-04-25T00:27:18.031 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes": 5332, 2024-04-25T00:27:18.031 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_top": 0, 2024-04-25T00:27:18.031 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_bottom": 5332, 2024-04-25T00:27:18.031 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pin_tail": 0, 2024-04-25T00:27:18.031 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pinned": 5332, 2024-04-25T00:27:18.032 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_expired": 1, 2024-04-25T00:27:18.032 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_with_caps": 0, 2024-04-25T00:27:18.032 INFO:teuthology.orchestra.run.smithi195.stdout: "caps": 0, 2024-04-25T00:27:18.032 INFO:teuthology.orchestra.run.smithi195.stdout: "subtrees": 2, 2024-04-25T00:27:18.032 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse": 0, 2024-04-25T00:27:18.032 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_hit": 0, 2024-04-25T00:27:18.032 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_forward": 0, 2024-04-25T00:27:18.032 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_discover": 0, 2024-04-25T00:27:18.032 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:27:18.032 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_remote_ino": 0, 2024-04-25T00:27:18.032 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_lock": 0, 2024-04-25T00:27:18.032 INFO:teuthology.orchestra.run.smithi195.stdout: "load_cent": 0, 2024-04-25T00:27:18.032 INFO:teuthology.orchestra.run.smithi195.stdout: "q": 0, 2024-04-25T00:27:18.032 INFO:teuthology.orchestra.run.smithi195.stdout: "exported": 0, 2024-04-25T00:27:18.032 INFO:teuthology.orchestra.run.smithi195.stdout: "exported_inodes": 0, 2024-04-25T00:27:18.032 INFO:teuthology.orchestra.run.smithi195.stdout: "imported": 0, 2024-04-25T00:27:18.033 INFO:teuthology.orchestra.run.smithi195.stdout: "imported_inodes": 0, 2024-04-25T00:27:18.033 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_dir_fetch": 0, 2024-04-25T00:27:18.033 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:27:18.033 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_peer_discover": 0, 2024-04-25T00:27:18.033 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rfiles": 4, 2024-04-25T00:27:18.033 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rbytes": 234, 2024-04-25T00:27:18.033 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rsnaps": 6, 2024-04-25T00:27:18.033 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:27:18.033 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_set_tag": 0, 2024-04-25T00:27:18.033 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:27:18.033 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:27:18.033 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:27:18.033 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:27:18.033 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:27:18.033 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_file_inodes": 0, 2024-04-25T00:27:18.033 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:27:18.034 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:27:18.034 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:27:18.034 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:27:18.034 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:27:18.034 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:27:18.034 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps": 0, 2024-04-25T00:27:18.034 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:27:18.034 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_cap_release": 0, 2024-04-25T00:27:18.034 INFO:teuthology.orchestra.run.smithi195.stdout: "process_request_cap_release": 0 2024-04-25T00:27:18.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.034 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_cache": { 2024-04-25T00:27:18.034 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update": 0, 2024-04-25T00:27:18.034 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update_receipt": 0, 2024-04-25T00:27:18.034 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_try_discover": 0, 2024-04-25T00:27:18.035 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_send_discover": 0, 2024-04-25T00:27:18.035 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_handle_discover": 0, 2024-04-25T00:27:18.035 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays": 0, 2024-04-25T00:27:18.035 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_delayed": 0, 2024-04-25T00:27:18.035 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:27:18.035 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_created": 0, 2024-04-25T00:27:18.035 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_enqueued": 0, 2024-04-25T00:27:18.035 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_reintegrated": 0, 2024-04-25T00:27:18.035 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_migrated": 0, 2024-04-25T00:27:18.035 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_processing": 0, 2024-04-25T00:27:18.035 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:27:18.035 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:27:18.035 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_started": 0, 2024-04-25T00:27:18.035 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_completed": 0, 2024-04-25T00:27:18.035 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:27:18.035 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:27:18.036 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:27:18.036 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_exportdir": 0, 2024-04-25T00:27:18.036 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_flush": 0, 2024-04-25T00:27:18.036 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:27:18.036 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragstats": 0, 2024-04-25T00:27:18.036 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_inodestats": 0 2024-04-25T00:27:18.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.036 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_log": { 2024-04-25T00:27:18.036 INFO:teuthology.orchestra.run.smithi195.stdout: "evlrg": 0, 2024-04-25T00:27:18.036 INFO:teuthology.orchestra.run.smithi195.stdout: "evadd": 0, 2024-04-25T00:27:18.036 INFO:teuthology.orchestra.run.smithi195.stdout: "evex": 0, 2024-04-25T00:27:18.036 INFO:teuthology.orchestra.run.smithi195.stdout: "evtrm": 0, 2024-04-25T00:27:18.036 INFO:teuthology.orchestra.run.smithi195.stdout: "ev": 12974, 2024-04-25T00:27:18.036 INFO:teuthology.orchestra.run.smithi195.stdout: "evexg": 0, 2024-04-25T00:27:18.036 INFO:teuthology.orchestra.run.smithi195.stdout: "evexd": 0, 2024-04-25T00:27:18.037 INFO:teuthology.orchestra.run.smithi195.stdout: "segadd": 0, 2024-04-25T00:27:18.037 INFO:teuthology.orchestra.run.smithi195.stdout: "segex": 0, 2024-04-25T00:27:18.037 INFO:teuthology.orchestra.run.smithi195.stdout: "segtrm": 0, 2024-04-25T00:27:18.037 INFO:teuthology.orchestra.run.smithi195.stdout: "seg": 19, 2024-04-25T00:27:18.037 INFO:teuthology.orchestra.run.smithi195.stdout: "segmjr": 3, 2024-04-25T00:27:18.037 INFO:teuthology.orchestra.run.smithi195.stdout: "segexg": 0, 2024-04-25T00:27:18.037 INFO:teuthology.orchestra.run.smithi195.stdout: "segexd": 0, 2024-04-25T00:27:18.037 INFO:teuthology.orchestra.run.smithi195.stdout: "expos": 4194304, 2024-04-25T00:27:18.037 INFO:teuthology.orchestra.run.smithi195.stdout: "wrpos": 38288601, 2024-04-25T00:27:18.037 INFO:teuthology.orchestra.run.smithi195.stdout: "rdpos": 38288465, 2024-04-25T00:27:18.037 INFO:teuthology.orchestra.run.smithi195.stdout: "jlat": { 2024-04-25T00:27:18.037 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.037 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.037 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.037 INFO:teuthology.orchestra.run.smithi195.stdout: "replayed": 12974 2024-04-25T00:27:18.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.038 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_mem": { 2024-04-25T00:27:18.038 INFO:teuthology.orchestra.run.smithi195.stdout: "ino": 1620, 2024-04-25T00:27:18.038 INFO:teuthology.orchestra.run.smithi195.stdout: "ino+": 6085, 2024-04-25T00:27:18.038 INFO:teuthology.orchestra.run.smithi195.stdout: "ino-": 4465, 2024-04-25T00:27:18.038 INFO:teuthology.orchestra.run.smithi195.stdout: "dir": 131, 2024-04-25T00:27:18.038 INFO:teuthology.orchestra.run.smithi195.stdout: "dir+": 783, 2024-04-25T00:27:18.038 INFO:teuthology.orchestra.run.smithi195.stdout: "dir-": 652, 2024-04-25T00:27:18.038 INFO:teuthology.orchestra.run.smithi195.stdout: "dn": 5332, 2024-04-25T00:27:18.038 INFO:teuthology.orchestra.run.smithi195.stdout: "dn+": 9040, 2024-04-25T00:27:18.038 INFO:teuthology.orchestra.run.smithi195.stdout: "dn-": 3708, 2024-04-25T00:27:18.038 INFO:teuthology.orchestra.run.smithi195.stdout: "cap": 0, 2024-04-25T00:27:18.038 INFO:teuthology.orchestra.run.smithi195.stdout: "cap+": 0, 2024-04-25T00:27:18.038 INFO:teuthology.orchestra.run.smithi195.stdout: "cap-": 0, 2024-04-25T00:27:18.039 INFO:teuthology.orchestra.run.smithi195.stdout: "rss": 69192, 2024-04-25T00:27:18.039 INFO:teuthology.orchestra.run.smithi195.stdout: "heap": 215420 2024-04-25T00:27:18.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.039 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_server": { 2024-04-25T00:27:18.039 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_client_request": 0, 2024-04-25T00:27:18.039 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_server_request": 0, 2024-04-25T00:27:18.039 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_request": 0, 2024-04-25T00:27:18.039 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_session": 0, 2024-04-25T00:27:18.039 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_peer_request": 0, 2024-04-25T00:27:18.039 INFO:teuthology.orchestra.run.smithi195.stdout: "req_create_latency": { 2024-04-25T00:27:18.039 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.039 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.039 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.039 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getattr_latency": { 2024-04-25T00:27:18.040 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.040 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.040 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.040 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getfilelock_latency": { 2024-04-25T00:27:18.040 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.040 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.040 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.040 INFO:teuthology.orchestra.run.smithi195.stdout: "req_link_latency": { 2024-04-25T00:27:18.040 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.040 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.040 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.040 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookup_latency": { 2024-04-25T00:27:18.041 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.041 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.041 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.041 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookuphash_latency": { 2024-04-25T00:27:18.041 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.041 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.041 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.041 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupino_latency": { 2024-04-25T00:27:18.041 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.041 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.041 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.042 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupname_latency": { 2024-04-25T00:27:18.042 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.042 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.042 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.042 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupparent_latency": { 2024-04-25T00:27:18.042 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.042 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.042 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.042 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupsnap_latency": { 2024-04-25T00:27:18.042 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.042 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.042 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.043 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lssnap_latency": { 2024-04-25T00:27:18.043 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.043 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.043 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.043 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mkdir_latency": { 2024-04-25T00:27:18.043 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.043 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.044 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.044 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mknod_latency": { 2024-04-25T00:27:18.044 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.044 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.044 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.044 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mksnap_latency": { 2024-04-25T00:27:18.044 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.044 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.044 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.044 INFO:teuthology.orchestra.run.smithi195.stdout: "req_open_latency": { 2024-04-25T00:27:18.044 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.045 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.045 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.045 INFO:teuthology.orchestra.run.smithi195.stdout: "req_readdir_latency": { 2024-04-25T00:27:18.045 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.045 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.045 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.045 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rename_latency": { 2024-04-25T00:27:18.045 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.045 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.045 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.045 INFO:teuthology.orchestra.run.smithi195.stdout: "req_renamesnap_latency": { 2024-04-25T00:27:18.045 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.046 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.046 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.046 INFO:teuthology.orchestra.run.smithi195.stdout: "req_snapdiff_latency": { 2024-04-25T00:27:18.046 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.046 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.046 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.046 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmdir_latency": { 2024-04-25T00:27:18.046 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.046 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.046 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.046 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmsnap_latency": { 2024-04-25T00:27:18.047 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.047 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.047 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.047 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmxattr_latency": { 2024-04-25T00:27:18.047 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.047 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.047 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.047 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setattr_latency": { 2024-04-25T00:27:18.047 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.047 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.047 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.047 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setdirlayout_latency": { 2024-04-25T00:27:18.048 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.048 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.048 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.048 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setfilelock_latency": { 2024-04-25T00:27:18.048 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.048 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.048 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.048 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setlayout_latency": { 2024-04-25T00:27:18.048 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.048 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.048 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.049 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setxattr_latency": { 2024-04-25T00:27:18.049 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.049 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.049 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.049 INFO:teuthology.orchestra.run.smithi195.stdout: "req_symlink_latency": { 2024-04-25T00:27:18.049 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.049 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.049 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.049 INFO:teuthology.orchestra.run.smithi195.stdout: "req_unlink_latency": { 2024-04-25T00:27:18.049 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.049 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.049 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.050 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:27:18.050 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:27:18.050 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getvxattr_latency": { 2024-04-25T00:27:18.050 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.050 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.050 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.050 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:18.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.050 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_sessions": { 2024-04-25T00:27:18.050 INFO:teuthology.orchestra.run.smithi195.stdout: "session_count": 2, 2024-04-25T00:27:18.050 INFO:teuthology.orchestra.run.smithi195.stdout: "session_add": 7, 2024-04-25T00:27:18.050 INFO:teuthology.orchestra.run.smithi195.stdout: "session_remove": 5, 2024-04-25T00:27:18.050 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_open": 5, 2024-04-25T00:27:18.050 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_stale": 0, 2024-04-25T00:27:18.051 INFO:teuthology.orchestra.run.smithi195.stdout: "total_load": 0, 2024-04-25T00:27:18.051 INFO:teuthology.orchestra.run.smithi195.stdout: "average_load": 0, 2024-04-25T00:27:18.051 INFO:teuthology.orchestra.run.smithi195.stdout: "avg_session_uptime": 3516, 2024-04-25T00:27:18.051 INFO:teuthology.orchestra.run.smithi195.stdout: "mdthresh_evicted": 0 2024-04-25T00:27:18.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.051 INFO:teuthology.orchestra.run.smithi195.stdout: "mempool": { 2024-04-25T00:27:18.051 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:27:18.051 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_items": 0, 2024-04-25T00:27:18.051 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:27:18.051 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:27:18.051 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:27:18.051 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:27:18.051 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:27:18.051 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:27:18.051 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:27:18.052 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:27:18.052 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:27:18.052 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:27:18.052 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:27:18.052 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:27:18.052 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:27:18.052 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_items": 0, 2024-04-25T00:27:18.052 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:27:18.052 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_items": 0, 2024-04-25T00:27:18.052 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:27:18.052 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:27:18.052 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:27:18.052 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:27:18.052 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:27:18.052 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:27:18.052 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:27:18.053 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_items": 0, 2024-04-25T00:27:18.053 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:27:18.053 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:27:18.053 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:27:18.053 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_items": 0, 2024-04-25T00:27:18.053 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_bytes": 0, 2024-04-25T00:27:18.053 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_items": 0, 2024-04-25T00:27:18.053 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:27:18.053 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:27:18.053 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:27:18.053 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:27:18.053 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_bytes": 272397, 2024-04-25T00:27:18.053 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_items": 1847, 2024-04-25T00:27:18.053 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:27:18.053 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_items": 0, 2024-04-25T00:27:18.053 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_bytes": 0, 2024-04-25T00:27:18.054 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_items": 0, 2024-04-25T00:27:18.054 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:27:18.054 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_items": 0, 2024-04-25T00:27:18.054 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:27:18.054 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_items": 0, 2024-04-25T00:27:18.054 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_bytes": 9672, 2024-04-25T00:27:18.054 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_items": 188, 2024-04-25T00:27:18.054 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:27:18.054 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:27:18.054 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_bytes": 0, 2024-04-25T00:27:18.054 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_items": 0, 2024-04-25T00:27:18.054 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_bytes": 7984570, 2024-04-25T00:27:18.054 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_items": 125424, 2024-04-25T00:27:18.055 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_bytes": 0, 2024-04-25T00:27:18.055 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_items": 0, 2024-04-25T00:27:18.055 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_bytes": 0, 2024-04-25T00:27:18.055 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_items": 0 2024-04-25T00:27:18.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.055 INFO:teuthology.orchestra.run.smithi195.stdout: "objecter": { 2024-04-25T00:27:18.055 INFO:teuthology.orchestra.run.smithi195.stdout: "op_active": 0, 2024-04-25T00:27:18.055 INFO:teuthology.orchestra.run.smithi195.stdout: "op_laggy": 0, 2024-04-25T00:27:18.055 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send": 2463, 2024-04-25T00:27:18.055 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send_bytes": 26, 2024-04-25T00:27:18.055 INFO:teuthology.orchestra.run.smithi195.stdout: "op_resend": 4, 2024-04-25T00:27:18.055 INFO:teuthology.orchestra.run.smithi195.stdout: "op_reply": 2463, 2024-04-25T00:27:18.055 INFO:teuthology.orchestra.run.smithi195.stdout: "op_latency": { 2024-04-25T00:27:18.055 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2463, 2024-04-25T00:27:18.055 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 2.694948858, 2024-04-25T00:27:18.056 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.001094173 2024-04-25T00:27:18.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.056 INFO:teuthology.orchestra.run.smithi195.stdout: "op_inflight": 0, 2024-04-25T00:27:18.056 INFO:teuthology.orchestra.run.smithi195.stdout: "oplen_avg": { 2024-04-25T00:27:18.056 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2463, 2024-04-25T00:27:18.056 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 2464 2024-04-25T00:27:18.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.056 INFO:teuthology.orchestra.run.smithi195.stdout: "op": 2463, 2024-04-25T00:27:18.056 INFO:teuthology.orchestra.run.smithi195.stdout: "op_r": 2463, 2024-04-25T00:27:18.056 INFO:teuthology.orchestra.run.smithi195.stdout: "op_w": 0, 2024-04-25T00:27:18.056 INFO:teuthology.orchestra.run.smithi195.stdout: "op_rmw": 0, 2024-04-25T00:27:18.056 INFO:teuthology.orchestra.run.smithi195.stdout: "op_pg": 0, 2024-04-25T00:27:18.056 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_stat": 1547, 2024-04-25T00:27:18.056 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_create": 0, 2024-04-25T00:27:18.056 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_read": 913, 2024-04-25T00:27:18.057 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_write": 0, 2024-04-25T00:27:18.057 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writefull": 0, 2024-04-25T00:27:18.057 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writesame": 0, 2024-04-25T00:27:18.057 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_append": 0, 2024-04-25T00:27:18.057 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_zero": 0, 2024-04-25T00:27:18.057 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_truncate": 0, 2024-04-25T00:27:18.057 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_delete": 0, 2024-04-25T00:27:18.057 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_mapext": 0, 2024-04-25T00:27:18.057 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_sparse_read": 0, 2024-04-25T00:27:18.057 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_clonerange": 0, 2024-04-25T00:27:18.057 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_getxattr": 2, 2024-04-25T00:27:18.057 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_setxattr": 0, 2024-04-25T00:27:18.057 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:27:18.057 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_rmxattr": 0, 2024-04-25T00:27:18.057 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:27:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_call": 0, 2024-04-25T00:27:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_watch": 0, 2024-04-25T00:27:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_notify": 0, 2024-04-25T00:27:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:27:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls": 0, 2024-04-25T00:27:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:27:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_other": 0, 2024-04-25T00:27:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_active": 0, 2024-04-25T00:27:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_send": 0, 2024-04-25T00:27:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_resend": 0, 2024-04-25T00:27:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_ping": 0, 2024-04-25T00:27:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_active": 0, 2024-04-25T00:27:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_send": 0, 2024-04-25T00:27:18.058 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_resend": 0, 2024-04-25T00:27:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_active": 0, 2024-04-25T00:27:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_send": 0, 2024-04-25T00:27:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_resend": 0, 2024-04-25T00:27:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_active": 0, 2024-04-25T00:27:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_send": 0, 2024-04-25T00:27:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_resend": 0, 2024-04-25T00:27:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: "command_active": 0, 2024-04-25T00:27:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: "command_send": 0, 2024-04-25T00:27:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: "command_resend": 0, 2024-04-25T00:27:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: "map_epoch": 107, 2024-04-25T00:27:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: "map_full": 0, 2024-04-25T00:27:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: "map_inc": 15, 2024-04-25T00:27:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_sessions": 7, 2024-04-25T00:27:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_open": 7, 2024-04-25T00:27:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_close": 0, 2024-04-25T00:27:18.059 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_laggy": 0, 2024-04-25T00:27:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_wr": 0, 2024-04-25T00:27:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_rd": 2, 2024-04-25T00:27:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_del": 0 2024-04-25T00:27:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: "oft": { 2024-04-25T00:27:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_objs": 0, 2024-04-25T00:27:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:27:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_updates": 0, 2024-04-25T00:27:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_removes": 0 2024-04-25T00:27:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: "osd-slow-ops": { 2024-04-25T00:27:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_ops_count": 0 2024-04-25T00:27:18.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: "purge_queue": { 2024-04-25T00:27:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops": 0, 2024-04-25T00:27:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:27:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing": 0, 2024-04-25T00:27:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_high_water": 0, 2024-04-25T00:27:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed_ops": 0, 2024-04-25T00:27:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed": 0, 2024-04-25T00:27:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_item_in_journal": 0 2024-04-25T00:27:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:27:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-04-25T00:27:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-04-25T00:27:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-04-25T00:27:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 2726, 2024-04-25T00:27:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 36046369, 2024-04-25T00:27:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:18.061 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 2726, 2024-04-25T00:27:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-04-25T00:27:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-04-25T00:27:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 2726, 2024-04-25T00:27:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 36046369, 2024-04-25T00:27:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-04-25T00:27:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_bytes": { 2024-04-25T00:27:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-04-25T00:27:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-04-25T00:27:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-04-25T00:27:18.062 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-04-25T00:27:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-04-25T00:27:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-04-25T00:27:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 2463, 2024-04-25T00:27:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 34096283, 2024-04-25T00:27:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 2463, 2024-04-25T00:27:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 34096283, 2024-04-25T00:27:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-04-25T00:27:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_ops": { 2024-04-25T00:27:18.063 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-04-25T00:27:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 1024, 2024-04-25T00:27:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-04-25T00:27:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-04-25T00:27:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-04-25T00:27:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-04-25T00:27:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 2463, 2024-04-25T00:27:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 2463, 2024-04-25T00:27:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 2463, 2024-04-25T00:27:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 2463, 2024-04-25T00:27:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-04-25T00:27:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:18.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:27:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-04-25T00:27:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-04-25T00:27:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-04-25T00:27:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-04-25T00:27:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-04-25T00:27:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-04-25T00:27:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-04-25T00:27:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-04-25T00:27:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 0, 2024-04-25T00:27:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 0, 2024-04-25T00:27:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-04-25T00:27:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.065 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle-0x55af6227e6e0": { 2024-04-25T00:27:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-04-25T00:27:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-04-25T00:27:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-04-25T00:27:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-04-25T00:27:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-04-25T00:27:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-04-25T00:27:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-04-25T00:27:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-04-25T00:27:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 0, 2024-04-25T00:27:18.066 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 0, 2024-04-25T00:27:18.067 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-04-25T00:27:18.067 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:18.067 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:18.067 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:18.067 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:18.067 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:18.067 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-04-25T00:27:18.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:17 smithi170 ceph-mon[32807]: pgmap v337: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:27:18.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:17 smithi170 ceph-mon[32807]: osdmap e107: 12 total, 12 up, 12 in 2024-04-25T00:27:18.831 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-25T00:27:18.831 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-25T00:27:18.831 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-25T00:27:18.831 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-25T00:27:18.831 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T00:27:18.832 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T00:27:18.832 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T00:27:19.079 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:27:19.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:19 smithi195 ceph-mon[32833]: pgmap v339: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:27:19.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:19 smithi170 ceph-mon[32807]: pgmap v339: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:27:19.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:19 smithi107 ceph-mon[29153]: pgmap v339: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:27:20.876 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-04-25T00:27:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:27:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 1033, 2024-04-25T00:27:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 1012, 2024-04-25T00:27:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 1778496, 2024-04-25T00:27:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 384394, 2024-04-25T00:27:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 5, 2024-04-25T00:27:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 1, 2024-04-25T00:27:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.141929624, 2024-04-25T00:27:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.042930144, 2024-04-25T00:27:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.060755650, 2024-04-25T00:27:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.037176753, 2024-04-25T00:27:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:20.877 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1011, 2024-04-25T00:27:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.055122568, 2024-04-25T00:27:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000054522 2024-04-25T00:27:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 1519440, 2024-04-25T00:27:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 139216 2024-04-25T00:27:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:27:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 4, 2024-04-25T00:27:20.878 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 170, 2024-04-25T00:27:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 876, 2024-04-25T00:27:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 451482, 2024-04-25T00:27:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 6, 2024-04-25T00:27:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 4, 2024-04-25T00:27:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.010008880, 2024-04-25T00:27:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.006544023, 2024-04-25T00:27:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.001538038, 2024-04-25T00:27:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.000075382, 2024-04-25T00:27:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 167, 2024-04-25T00:27:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.013795714, 2024-04-25T00:27:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000082609 2024-04-25T00:27:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.879 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T00:27:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 450784 2024-04-25T00:27:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:27:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 1567, 2024-04-25T00:27:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 1567, 2024-04-25T00:27:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 6654515, 2024-04-25T00:27:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 497489, 2024-04-25T00:27:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 4, 2024-04-25T00:27:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 3, 2024-04-25T00:27:20.880 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.107886495, 2024-04-25T00:27:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.035545886, 2024-04-25T00:27:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.028352334, 2024-04-25T00:27:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.036106019, 2024-04-25T00:27:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1566, 2024-04-25T00:27:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.061860147, 2024-04-25T00:27:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000039502 2024-04-25T00:27:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.881 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-25T00:27:20.882 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:27:20.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.882 INFO:teuthology.orchestra.run.smithi195.stdout: "cct": { 2024-04-25T00:27:20.882 INFO:teuthology.orchestra.run.smithi195.stdout: "total_workers": 1, 2024-04-25T00:27:20.882 INFO:teuthology.orchestra.run.smithi195.stdout: "unhealthy_workers": 0 2024-04-25T00:27:20.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.882 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-MDSRank": { 2024-04-25T00:27:20.882 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-04-25T00:27:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-04-25T00:27:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 3949, 2024-04-25T00:27:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.103198823, 2024-04-25T00:27:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000026132 2024-04-25T00:27:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-PurgeQueue": { 2024-04-25T00:27:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-04-25T00:27:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-04-25T00:27:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:20.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: "mds": { 2024-04-25T00:27:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: "request": 0, 2024-04-25T00:27:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: "reply": 0, 2024-04-25T00:27:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: "reply_latency": { 2024-04-25T00:27:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_reply": 0, 2024-04-25T00:27:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: "forward": 0, 2024-04-25T00:27:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_complete": 0, 2024-04-25T00:27:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_keys": 0, 2024-04-25T00:27:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_commit": 0, 2024-04-25T00:27:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_split": 0, 2024-04-25T00:27:20.884 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_merge": 0, 2024-04-25T00:27:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes": 605, 2024-04-25T00:27:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_top": 0, 2024-04-25T00:27:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_bottom": 605, 2024-04-25T00:27:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pin_tail": 0, 2024-04-25T00:27:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pinned": 605, 2024-04-25T00:27:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_expired": 0, 2024-04-25T00:27:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_with_caps": 0, 2024-04-25T00:27:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: "caps": 0, 2024-04-25T00:27:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: "subtrees": 1, 2024-04-25T00:27:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse": 0, 2024-04-25T00:27:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_hit": 0, 2024-04-25T00:27:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_forward": 0, 2024-04-25T00:27:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_discover": 0, 2024-04-25T00:27:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:27:20.885 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_remote_ino": 0, 2024-04-25T00:27:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_lock": 0, 2024-04-25T00:27:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: "load_cent": 0, 2024-04-25T00:27:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: "q": 0, 2024-04-25T00:27:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: "exported": 0, 2024-04-25T00:27:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: "exported_inodes": 0, 2024-04-25T00:27:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: "imported": 0, 2024-04-25T00:27:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: "imported_inodes": 0, 2024-04-25T00:27:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_dir_fetch": 0, 2024-04-25T00:27:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:27:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_peer_discover": 0, 2024-04-25T00:27:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rfiles": 0, 2024-04-25T00:27:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rbytes": 0, 2024-04-25T00:27:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rsnaps": 0, 2024-04-25T00:27:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:27:20.886 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_set_tag": 0, 2024-04-25T00:27:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:27:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:27:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:27:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:27:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:27:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_file_inodes": 0, 2024-04-25T00:27:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:27:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:27:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:27:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:27:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:27:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:27:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps": 0, 2024-04-25T00:27:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:27:20.887 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_cap_release": 0, 2024-04-25T00:27:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: "process_request_cap_release": 0 2024-04-25T00:27:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_cache": { 2024-04-25T00:27:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update": 0, 2024-04-25T00:27:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update_receipt": 0, 2024-04-25T00:27:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_try_discover": 0, 2024-04-25T00:27:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_send_discover": 0, 2024-04-25T00:27:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_handle_discover": 0, 2024-04-25T00:27:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays": 0, 2024-04-25T00:27:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_delayed": 0, 2024-04-25T00:27:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:27:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_created": 0, 2024-04-25T00:27:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_enqueued": 0, 2024-04-25T00:27:20.888 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_reintegrated": 0, 2024-04-25T00:27:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_migrated": 0, 2024-04-25T00:27:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_processing": 0, 2024-04-25T00:27:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:27:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:27:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_started": 0, 2024-04-25T00:27:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_completed": 0, 2024-04-25T00:27:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:27:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:27:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:27:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_exportdir": 0, 2024-04-25T00:27:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_flush": 0, 2024-04-25T00:27:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:27:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragstats": 0, 2024-04-25T00:27:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_inodestats": 0 2024-04-25T00:27:20.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_log": { 2024-04-25T00:27:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: "evlrg": 0, 2024-04-25T00:27:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: "evadd": 0, 2024-04-25T00:27:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: "evex": 0, 2024-04-25T00:27:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: "evtrm": 0, 2024-04-25T00:27:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: "ev": 2752, 2024-04-25T00:27:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: "evexg": 0, 2024-04-25T00:27:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: "evexd": 0, 2024-04-25T00:27:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: "segadd": 0, 2024-04-25T00:27:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: "segex": 0, 2024-04-25T00:27:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: "segtrm": 0, 2024-04-25T00:27:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: "seg": 4, 2024-04-25T00:27:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: "segmjr": 2, 2024-04-25T00:27:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: "segexg": 0, 2024-04-25T00:27:20.890 INFO:teuthology.orchestra.run.smithi195.stdout: "segexd": 0, 2024-04-25T00:27:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: "expos": 4194304, 2024-04-25T00:27:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: "wrpos": 10448076, 2024-04-25T00:27:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: "rdpos": 10445988, 2024-04-25T00:27:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: "jlat": { 2024-04-25T00:27:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: "replayed": 2752 2024-04-25T00:27:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_mem": { 2024-04-25T00:27:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: "ino": 18, 2024-04-25T00:27:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: "ino+": 1330, 2024-04-25T00:27:20.891 INFO:teuthology.orchestra.run.smithi195.stdout: "ino-": 1312, 2024-04-25T00:27:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: "dir": 7, 2024-04-25T00:27:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: "dir+": 295, 2024-04-25T00:27:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: "dir-": 288, 2024-04-25T00:27:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: "dn": 605, 2024-04-25T00:27:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: "dn+": 1958, 2024-04-25T00:27:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: "dn-": 1353, 2024-04-25T00:27:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: "cap": 0, 2024-04-25T00:27:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: "cap+": 0, 2024-04-25T00:27:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: "cap-": 0, 2024-04-25T00:27:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: "rss": 53340, 2024-04-25T00:27:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: "heap": 215420 2024-04-25T00:27:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_server": { 2024-04-25T00:27:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_client_request": 0, 2024-04-25T00:27:20.892 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_server_request": 0, 2024-04-25T00:27:20.893 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_request": 0, 2024-04-25T00:27:20.893 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_session": 0, 2024-04-25T00:27:20.893 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_peer_request": 0, 2024-04-25T00:27:20.893 INFO:teuthology.orchestra.run.smithi195.stdout: "req_create_latency": { 2024-04-25T00:27:20.893 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.893 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.893 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.893 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.893 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getattr_latency": { 2024-04-25T00:27:20.893 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.893 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.893 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.893 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.893 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getfilelock_latency": { 2024-04-25T00:27:20.894 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.894 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.894 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.894 INFO:teuthology.orchestra.run.smithi195.stdout: "req_link_latency": { 2024-04-25T00:27:20.894 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.894 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.894 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.894 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookup_latency": { 2024-04-25T00:27:20.894 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.894 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.894 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookuphash_latency": { 2024-04-25T00:27:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupino_latency": { 2024-04-25T00:27:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupname_latency": { 2024-04-25T00:27:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupparent_latency": { 2024-04-25T00:27:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupsnap_latency": { 2024-04-25T00:27:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lssnap_latency": { 2024-04-25T00:27:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.897 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mkdir_latency": { 2024-04-25T00:27:20.897 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.897 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mknod_latency": { 2024-04-25T00:27:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mksnap_latency": { 2024-04-25T00:27:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: "req_open_latency": { 2024-04-25T00:27:20.898 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: "req_readdir_latency": { 2024-04-25T00:27:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rename_latency": { 2024-04-25T00:27:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: "req_renamesnap_latency": { 2024-04-25T00:27:20.899 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: "req_snapdiff_latency": { 2024-04-25T00:27:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmdir_latency": { 2024-04-25T00:27:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.900 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmsnap_latency": { 2024-04-25T00:27:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmxattr_latency": { 2024-04-25T00:27:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setattr_latency": { 2024-04-25T00:27:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.901 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setdirlayout_latency": { 2024-04-25T00:27:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setfilelock_latency": { 2024-04-25T00:27:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setlayout_latency": { 2024-04-25T00:27:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.902 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setxattr_latency": { 2024-04-25T00:27:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: "req_symlink_latency": { 2024-04-25T00:27:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: "req_unlink_latency": { 2024-04-25T00:27:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:27:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:27:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getvxattr_latency": { 2024-04-25T00:27:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_sessions": { 2024-04-25T00:27:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: "session_count": 2, 2024-04-25T00:27:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: "session_add": 3, 2024-04-25T00:27:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: "session_remove": 1, 2024-04-25T00:27:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_open": 2, 2024-04-25T00:27:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_stale": 0, 2024-04-25T00:27:20.904 INFO:teuthology.orchestra.run.smithi195.stdout: "total_load": 0, 2024-04-25T00:27:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: "average_load": 0, 2024-04-25T00:27:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: "avg_session_uptime": 1801, 2024-04-25T00:27:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: "mdthresh_evicted": 0 2024-04-25T00:27:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: "mempool": { 2024-04-25T00:27:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:27:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_items": 0, 2024-04-25T00:27:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:27:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:27:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:27:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:27:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:27:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:27:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:27:20.905 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:27:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:27:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:27:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:27:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:27:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:27:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_items": 0, 2024-04-25T00:27:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:27:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_items": 0, 2024-04-25T00:27:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:27:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:27:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:27:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:27:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:27:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:27:20.906 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:27:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_items": 0, 2024-04-25T00:27:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:27:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:27:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:27:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_items": 0, 2024-04-25T00:27:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_bytes": 0, 2024-04-25T00:27:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_items": 0, 2024-04-25T00:27:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:27:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:27:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:27:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:27:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_bytes": 181868, 2024-04-25T00:27:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_items": 62, 2024-04-25T00:27:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:27:20.907 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_items": 0, 2024-04-25T00:27:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_bytes": 0, 2024-04-25T00:27:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_items": 0, 2024-04-25T00:27:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:27:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_items": 0, 2024-04-25T00:27:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:27:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_items": 0, 2024-04-25T00:27:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_bytes": 9432, 2024-04-25T00:27:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_items": 182, 2024-04-25T00:27:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:27:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:27:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_bytes": 0, 2024-04-25T00:27:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_items": 0, 2024-04-25T00:27:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_bytes": 486811, 2024-04-25T00:27:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_items": 3472, 2024-04-25T00:27:20.908 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_bytes": 0, 2024-04-25T00:27:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_items": 0, 2024-04-25T00:27:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_bytes": 0, 2024-04-25T00:27:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_items": 0 2024-04-25T00:27:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: "objecter": { 2024-04-25T00:27:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: "op_active": 0, 2024-04-25T00:27:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: "op_laggy": 0, 2024-04-25T00:27:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send": 2336, 2024-04-25T00:27:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send_bytes": 21, 2024-04-25T00:27:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: "op_resend": 3, 2024-04-25T00:27:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: "op_reply": 2336, 2024-04-25T00:27:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: "op_latency": { 2024-04-25T00:27:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2336, 2024-04-25T00:27:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 1.129978467, 2024-04-25T00:27:20.909 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000483723 2024-04-25T00:27:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: "op_inflight": 0, 2024-04-25T00:27:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: "oplen_avg": { 2024-04-25T00:27:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2336, 2024-04-25T00:27:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 2337 2024-04-25T00:27:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: "op": 2336, 2024-04-25T00:27:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: "op_r": 2336, 2024-04-25T00:27:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: "op_w": 0, 2024-04-25T00:27:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: "op_rmw": 0, 2024-04-25T00:27:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: "op_pg": 0, 2024-04-25T00:27:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_stat": 1479, 2024-04-25T00:27:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_create": 0, 2024-04-25T00:27:20.910 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_read": 855, 2024-04-25T00:27:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_write": 0, 2024-04-25T00:27:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writefull": 0, 2024-04-25T00:27:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writesame": 0, 2024-04-25T00:27:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_append": 0, 2024-04-25T00:27:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_zero": 0, 2024-04-25T00:27:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_truncate": 0, 2024-04-25T00:27:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_delete": 0, 2024-04-25T00:27:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_mapext": 0, 2024-04-25T00:27:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_sparse_read": 0, 2024-04-25T00:27:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_clonerange": 0, 2024-04-25T00:27:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_getxattr": 1, 2024-04-25T00:27:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_setxattr": 0, 2024-04-25T00:27:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:27:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_rmxattr": 0, 2024-04-25T00:27:20.911 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:27:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_call": 0, 2024-04-25T00:27:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_watch": 0, 2024-04-25T00:27:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_notify": 0, 2024-04-25T00:27:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:27:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls": 0, 2024-04-25T00:27:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:27:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_other": 0, 2024-04-25T00:27:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_active": 0, 2024-04-25T00:27:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_send": 0, 2024-04-25T00:27:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_resend": 0, 2024-04-25T00:27:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_ping": 0, 2024-04-25T00:27:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_active": 0, 2024-04-25T00:27:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_send": 0, 2024-04-25T00:27:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_resend": 0, 2024-04-25T00:27:20.912 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_active": 0, 2024-04-25T00:27:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_send": 0, 2024-04-25T00:27:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_resend": 0, 2024-04-25T00:27:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_active": 0, 2024-04-25T00:27:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_send": 0, 2024-04-25T00:27:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_resend": 0, 2024-04-25T00:27:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: "command_active": 0, 2024-04-25T00:27:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: "command_send": 0, 2024-04-25T00:27:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: "command_resend": 0, 2024-04-25T00:27:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: "map_epoch": 107, 2024-04-25T00:27:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: "map_full": 0, 2024-04-25T00:27:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: "map_inc": 15, 2024-04-25T00:27:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_sessions": 4, 2024-04-25T00:27:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_open": 4, 2024-04-25T00:27:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_close": 0, 2024-04-25T00:27:20.913 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_laggy": 0, 2024-04-25T00:27:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_wr": 0, 2024-04-25T00:27:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_rd": 2, 2024-04-25T00:27:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_del": 0 2024-04-25T00:27:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: "oft": { 2024-04-25T00:27:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_objs": 0, 2024-04-25T00:27:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:27:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_updates": 0, 2024-04-25T00:27:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_removes": 0 2024-04-25T00:27:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: "osd-slow-ops": { 2024-04-25T00:27:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_ops_count": 0 2024-04-25T00:27:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: "purge_queue": { 2024-04-25T00:27:20.914 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops": 0, 2024-04-25T00:27:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:27:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing": 0, 2024-04-25T00:27:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_high_water": 0, 2024-04-25T00:27:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed_ops": 0, 2024-04-25T00:27:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed": 0, 2024-04-25T00:27:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_item_in_journal": 0 2024-04-25T00:27:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:27:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-04-25T00:27:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-04-25T00:27:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-04-25T00:27:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 2600, 2024-04-25T00:27:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 8183521, 2024-04-25T00:27:20.915 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 2600, 2024-04-25T00:27:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-04-25T00:27:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-04-25T00:27:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 2600, 2024-04-25T00:27:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 8183521, 2024-04-25T00:27:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-04-25T00:27:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_bytes": { 2024-04-25T00:27:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-04-25T00:27:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-04-25T00:27:20.916 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-04-25T00:27:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-04-25T00:27:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-04-25T00:27:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-04-25T00:27:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 2336, 2024-04-25T00:27:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 6253777, 2024-04-25T00:27:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 2336, 2024-04-25T00:27:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 6253777, 2024-04-25T00:27:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-04-25T00:27:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.917 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_ops": { 2024-04-25T00:27:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-04-25T00:27:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 1024, 2024-04-25T00:27:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-04-25T00:27:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-04-25T00:27:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-04-25T00:27:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-04-25T00:27:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 2336, 2024-04-25T00:27:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 2336, 2024-04-25T00:27:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 2336, 2024-04-25T00:27:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 2336, 2024-04-25T00:27:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-04-25T00:27:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.918 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:27:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-04-25T00:27:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-04-25T00:27:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-04-25T00:27:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-04-25T00:27:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-04-25T00:27:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-04-25T00:27:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-04-25T00:27:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-04-25T00:27:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 0, 2024-04-25T00:27:20.919 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 0, 2024-04-25T00:27:20.920 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-04-25T00:27:20.920 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.920 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.920 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.920 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:20.920 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:20.920 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle-0x561f1e69c6e0": { 2024-04-25T00:27:20.920 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-04-25T00:27:20.920 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-04-25T00:27:20.920 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-04-25T00:27:20.920 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-04-25T00:27:20.920 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-04-25T00:27:20.920 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:20.920 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-04-25T00:27:20.921 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-04-25T00:27:20.921 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-04-25T00:27:20.921 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 0, 2024-04-25T00:27:20.921 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 0, 2024-04-25T00:27:20.921 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-04-25T00:27:20.921 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:20.921 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:20.921 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:20.921 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:20.921 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:20.921 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-04-25T00:27:21.776 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-25T00:27:21.776 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-25T00:27:21.776 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-25T00:27:21.776 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-25T00:27:21.776 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T00:27:21.776 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T00:27:21.777 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T00:27:21.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:21 smithi107 ceph-mon[29153]: pgmap v340: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:27:22.026 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:27:22.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:21 smithi195 ceph-mon[32833]: pgmap v340: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:27:22.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:21 smithi170 ceph-mon[32807]: pgmap v340: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T00:27:23.793 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:23 smithi195 ceph-mon[32833]: pgmap v341: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:27:23.794 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-04-25T00:27:23.794 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:27:23.794 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 3666, 2024-04-25T00:27:23.794 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 5705, 2024-04-25T00:27:23.794 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 1090333, 2024-04-25T00:27:23.794 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 9213565, 2024-04-25T00:27:23.794 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 16, 2024-04-25T00:27:23.794 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 6, 2024-04-25T00:27:23.794 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.273486984, 2024-04-25T00:27:23.794 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.107637128, 2024-04-25T00:27:23.794 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.137092503, 2024-04-25T00:27:23.795 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.035184817, 2024-04-25T00:27:23.795 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:23.795 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 5700, 2024-04-25T00:27:23.795 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.179034726, 2024-04-25T00:27:23.795 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000031409 2024-04-25T00:27:23.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.795 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:23.795 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 5723, 2024-04-25T00:27:23.795 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.001006479, 2024-04-25T00:27:23.795 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000175 2024-04-25T00:27:23.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.795 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 6688, 2024-04-25T00:27:23.795 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 7904 2024-04-25T00:27:23.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.796 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:27:23.796 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 12038, 2024-04-25T00:27:23.796 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 9152, 2024-04-25T00:27:23.796 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 6145875, 2024-04-25T00:27:23.796 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 3438455, 2024-04-25T00:27:23.796 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 41, 2024-04-25T00:27:23.796 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 9, 2024-04-25T00:27:23.796 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.653353260, 2024-04-25T00:27:23.796 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.246594265, 2024-04-25T00:27:23.796 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.512010470, 2024-04-25T00:27:23.796 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.020717157, 2024-04-25T00:27:23.796 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:23.796 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 9155, 2024-04-25T00:27:23.796 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.258470467, 2024-04-25T00:27:23.797 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000028232 2024-04-25T00:27:23.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.797 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:23.797 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 19389, 2024-04-25T00:27:23.797 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.002862652, 2024-04-25T00:27:23.797 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000147 2024-04-25T00:27:23.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.797 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 1534288, 2024-04-25T00:27:23.797 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 184640 2024-04-25T00:27:23.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.797 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:27:23.797 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 1661, 2024-04-25T00:27:23.797 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 1826, 2024-04-25T00:27:23.797 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 490617, 2024-04-25T00:27:23.797 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 1219608, 2024-04-25T00:27:23.798 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 16, 2024-04-25T00:27:23.798 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 6, 2024-04-25T00:27:23.798 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.112459997, 2024-04-25T00:27:23.798 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.041170497, 2024-04-25T00:27:23.798 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.027696476, 2024-04-25T00:27:23.798 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.034083527, 2024-04-25T00:27:23.798 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:23.798 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1823, 2024-04-25T00:27:23.798 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.088439280, 2024-04-25T00:27:23.798 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000048513 2024-04-25T00:27:23.798 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.798 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:23.798 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 6, 2024-04-25T00:27:23.798 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000599, 2024-04-25T00:27:23.799 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000099 2024-04-25T00:27:23.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.799 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T00:27:23.799 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 452544 2024-04-25T00:27:23.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.799 INFO:teuthology.orchestra.run.smithi195.stdout: "cct": { 2024-04-25T00:27:23.799 INFO:teuthology.orchestra.run.smithi195.stdout: "total_workers": 1, 2024-04-25T00:27:23.799 INFO:teuthology.orchestra.run.smithi195.stdout: "unhealthy_workers": 0 2024-04-25T00:27:23.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.799 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-MDSRank": { 2024-04-25T00:27:23.799 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-04-25T00:27:23.799 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-04-25T00:27:23.799 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 2253, 2024-04-25T00:27:23.799 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 1.155643599, 2024-04-25T00:27:23.800 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000512935 2024-04-25T00:27:23.800 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:23.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.800 INFO:teuthology.orchestra.run.smithi195.stdout: "finisher-PurgeQueue": { 2024-04-25T00:27:23.800 INFO:teuthology.orchestra.run.smithi195.stdout: "queue_len": 0, 2024-04-25T00:27:23.800 INFO:teuthology.orchestra.run.smithi195.stdout: "complete_latency": { 2024-04-25T00:27:23.800 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1301, 2024-04-25T00:27:23.800 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.100680167, 2024-04-25T00:27:23.800 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000077386 2024-04-25T00:27:23.800 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:23.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.800 INFO:teuthology.orchestra.run.smithi195.stdout: "mds": { 2024-04-25T00:27:23.800 INFO:teuthology.orchestra.run.smithi195.stdout: "request": 1368, 2024-04-25T00:27:23.800 INFO:teuthology.orchestra.run.smithi195.stdout: "reply": 1356, 2024-04-25T00:27:23.800 INFO:teuthology.orchestra.run.smithi195.stdout: "reply_latency": { 2024-04-25T00:27:23.801 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1356, 2024-04-25T00:27:23.801 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 3.115634390, 2024-04-25T00:27:23.801 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.002297665 2024-04-25T00:27:23.801 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.801 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_reply": 0, 2024-04-25T00:27:23.801 INFO:teuthology.orchestra.run.smithi195.stdout: "forward": 12, 2024-04-25T00:27:23.801 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_complete": 0, 2024-04-25T00:27:23.801 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_fetch_keys": 0, 2024-04-25T00:27:23.801 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_commit": 11, 2024-04-25T00:27:23.801 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_split": 0, 2024-04-25T00:27:23.801 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_merge": 0, 2024-04-25T00:27:23.801 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes": 47, 2024-04-25T00:27:23.801 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_top": 8, 2024-04-25T00:27:23.801 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_bottom": 4, 2024-04-25T00:27:23.801 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pin_tail": 35, 2024-04-25T00:27:23.802 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_pinned": 35, 2024-04-25T00:27:23.802 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_expired": 1319, 2024-04-25T00:27:23.802 INFO:teuthology.orchestra.run.smithi195.stdout: "inodes_with_caps": 0, 2024-04-25T00:27:23.802 INFO:teuthology.orchestra.run.smithi195.stdout: "caps": 0, 2024-04-25T00:27:23.802 INFO:teuthology.orchestra.run.smithi195.stdout: "subtrees": 4, 2024-04-25T00:27:23.802 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse": 3050, 2024-04-25T00:27:23.802 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_hit": 2221, 2024-04-25T00:27:23.802 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_forward": 5, 2024-04-25T00:27:23.802 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_discover": 128, 2024-04-25T00:27:23.802 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:27:23.802 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_remote_ino": 3, 2024-04-25T00:27:23.802 INFO:teuthology.orchestra.run.smithi195.stdout: "traverse_lock": 4, 2024-04-25T00:27:23.802 INFO:teuthology.orchestra.run.smithi195.stdout: "load_cent": 0, 2024-04-25T00:27:23.802 INFO:teuthology.orchestra.run.smithi195.stdout: "q": 0, 2024-04-25T00:27:23.803 INFO:teuthology.orchestra.run.smithi195.stdout: "exported": 63, 2024-04-25T00:27:23.803 INFO:teuthology.orchestra.run.smithi195.stdout: "exported_inodes": 860, 2024-04-25T00:27:23.803 INFO:teuthology.orchestra.run.smithi195.stdout: "imported": 69, 2024-04-25T00:27:23.803 INFO:teuthology.orchestra.run.smithi195.stdout: "imported_inodes": 491, 2024-04-25T00:27:23.803 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_dir_fetch": 0, 2024-04-25T00:27:23.803 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_backtrace_fetch": 1233, 2024-04-25T00:27:23.803 INFO:teuthology.orchestra.run.smithi195.stdout: "openino_peer_discover": 11, 2024-04-25T00:27:23.803 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rfiles": 4, 2024-04-25T00:27:23.803 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rbytes": 234, 2024-04-25T00:27:23.803 INFO:teuthology.orchestra.run.smithi195.stdout: "root_rsnaps": 6, 2024-04-25T00:27:23.803 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_fetch": 1233, 2024-04-25T00:27:23.803 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_set_tag": 0, 2024-04-25T00:27:23.803 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:27:23.803 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:27:23.803 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_inodes": 56, 2024-04-25T00:27:23.804 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:27:23.804 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_dirfrag_rstats": 56, 2024-04-25T00:27:23.804 INFO:teuthology.orchestra.run.smithi195.stdout: "scrub_file_inodes": 1177, 2024-04-25T00:27:23.804 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_inode_file_caps": 18, 2024-04-25T00:27:23.804 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_revoke": 305, 2024-04-25T00:27:23.804 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_grant": 859, 2024-04-25T00:27:23.804 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_trunc": 8, 2024-04-25T00:27:23.804 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:27:23.804 INFO:teuthology.orchestra.run.smithi195.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:27:23.804 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps": 841, 2024-04-25T00:27:23.804 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_caps_dirty": 549, 2024-04-25T00:27:23.804 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_cap_release": 62, 2024-04-25T00:27:23.804 INFO:teuthology.orchestra.run.smithi195.stdout: "process_request_cap_release": 1165 2024-04-25T00:27:23.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.804 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_cache": { 2024-04-25T00:27:23.805 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update": 0, 2024-04-25T00:27:23.805 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_update_receipt": 0, 2024-04-25T00:27:23.805 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_try_discover": 0, 2024-04-25T00:27:23.805 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_send_discover": 150, 2024-04-25T00:27:23.805 INFO:teuthology.orchestra.run.smithi195.stdout: "dir_handle_discover": 63, 2024-04-25T00:27:23.805 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays": 2, 2024-04-25T00:27:23.805 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_delayed": 0, 2024-04-25T00:27:23.805 INFO:teuthology.orchestra.run.smithi195.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:27:23.805 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_created": 585, 2024-04-25T00:27:23.805 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_enqueued": 581, 2024-04-25T00:27:23.805 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_reintegrated": 5, 2024-04-25T00:27:23.805 INFO:teuthology.orchestra.run.smithi195.stdout: "strays_migrated": 0, 2024-04-25T00:27:23.805 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_processing": 0, 2024-04-25T00:27:23.805 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:27:23.806 INFO:teuthology.orchestra.run.smithi195.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:27:23.806 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_started": 0, 2024-04-25T00:27:23.806 INFO:teuthology.orchestra.run.smithi195.stdout: "recovery_completed": 0, 2024-04-25T00:27:23.806 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:27:23.806 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:27:23.806 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:27:23.806 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_exportdir": 63, 2024-04-25T00:27:23.806 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_flush": 0, 2024-04-25T00:27:23.806 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:27:23.806 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_fragstats": 0, 2024-04-25T00:27:23.806 INFO:teuthology.orchestra.run.smithi195.stdout: "ireq_inodestats": 0 2024-04-25T00:27:23.806 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.806 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_log": { 2024-04-25T00:27:23.806 INFO:teuthology.orchestra.run.smithi195.stdout: "evlrg": 0, 2024-04-25T00:27:23.807 INFO:teuthology.orchestra.run.smithi195.stdout: "evadd": 2752, 2024-04-25T00:27:23.807 INFO:teuthology.orchestra.run.smithi195.stdout: "evex": 0, 2024-04-25T00:27:23.807 INFO:teuthology.orchestra.run.smithi195.stdout: "evtrm": 0, 2024-04-25T00:27:23.807 INFO:teuthology.orchestra.run.smithi195.stdout: "ev": 2752, 2024-04-25T00:27:23.807 INFO:teuthology.orchestra.run.smithi195.stdout: "evexg": 0, 2024-04-25T00:27:23.807 INFO:teuthology.orchestra.run.smithi195.stdout: "evexd": 0, 2024-04-25T00:27:23.807 INFO:teuthology.orchestra.run.smithi195.stdout: "segadd": 4, 2024-04-25T00:27:23.807 INFO:teuthology.orchestra.run.smithi195.stdout: "segex": 0, 2024-04-25T00:27:23.807 INFO:teuthology.orchestra.run.smithi195.stdout: "segtrm": 0, 2024-04-25T00:27:23.807 INFO:teuthology.orchestra.run.smithi195.stdout: "seg": 4, 2024-04-25T00:27:23.807 INFO:teuthology.orchestra.run.smithi195.stdout: "segmjr": 2, 2024-04-25T00:27:23.807 INFO:teuthology.orchestra.run.smithi195.stdout: "segexg": 0, 2024-04-25T00:27:23.807 INFO:teuthology.orchestra.run.smithi195.stdout: "segexd": 0, 2024-04-25T00:27:23.807 INFO:teuthology.orchestra.run.smithi195.stdout: "expos": 4194304, 2024-04-25T00:27:23.807 INFO:teuthology.orchestra.run.smithi195.stdout: "wrpos": 10448076, 2024-04-25T00:27:23.808 INFO:teuthology.orchestra.run.smithi195.stdout: "rdpos": 0, 2024-04-25T00:27:23.808 INFO:teuthology.orchestra.run.smithi195.stdout: "jlat": { 2024-04-25T00:27:23.808 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 623, 2024-04-25T00:27:23.808 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 2.286054941, 2024-04-25T00:27:23.808 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.003669430 2024-04-25T00:27:23.808 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.808 INFO:teuthology.orchestra.run.smithi195.stdout: "replayed": 0 2024-04-25T00:27:23.808 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.808 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_mem": { 2024-04-25T00:27:23.808 INFO:teuthology.orchestra.run.smithi195.stdout: "ino": 41, 2024-04-25T00:27:23.809 INFO:teuthology.orchestra.run.smithi195.stdout: "ino+": 1335, 2024-04-25T00:27:23.809 INFO:teuthology.orchestra.run.smithi195.stdout: "ino-": 1294, 2024-04-25T00:27:23.809 INFO:teuthology.orchestra.run.smithi195.stdout: "dir": 59, 2024-04-25T00:27:23.809 INFO:teuthology.orchestra.run.smithi195.stdout: "dir+": 322, 2024-04-25T00:27:23.809 INFO:teuthology.orchestra.run.smithi195.stdout: "dir-": 263, 2024-04-25T00:27:23.809 INFO:teuthology.orchestra.run.smithi195.stdout: "dn": 47, 2024-04-25T00:27:23.809 INFO:teuthology.orchestra.run.smithi195.stdout: "dn+": 2167, 2024-04-25T00:27:23.809 INFO:teuthology.orchestra.run.smithi195.stdout: "dn-": 2120, 2024-04-25T00:27:23.809 INFO:teuthology.orchestra.run.smithi195.stdout: "cap": 0, 2024-04-25T00:27:23.809 INFO:teuthology.orchestra.run.smithi195.stdout: "cap+": 985, 2024-04-25T00:27:23.809 INFO:teuthology.orchestra.run.smithi195.stdout: "cap-": 985, 2024-04-25T00:27:23.809 INFO:teuthology.orchestra.run.smithi195.stdout: "rss": 75712, 2024-04-25T00:27:23.809 INFO:teuthology.orchestra.run.smithi195.stdout: "heap": 223612 2024-04-25T00:27:23.809 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.809 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_server": { 2024-04-25T00:27:23.809 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_client_request": 2534, 2024-04-25T00:27:23.810 INFO:teuthology.orchestra.run.smithi195.stdout: "dispatch_server_request": 326, 2024-04-25T00:27:23.810 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_request": 1368, 2024-04-25T00:27:23.810 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_client_session": 80, 2024-04-25T00:27:23.810 INFO:teuthology.orchestra.run.smithi195.stdout: "handle_peer_request": 1122, 2024-04-25T00:27:23.810 INFO:teuthology.orchestra.run.smithi195.stdout: "req_create_latency": { 2024-04-25T00:27:23.810 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 44, 2024-04-25T00:27:23.810 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.072050841, 2024-04-25T00:27:23.810 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.001637519 2024-04-25T00:27:23.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.810 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getattr_latency": { 2024-04-25T00:27:23.810 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 3, 2024-04-25T00:27:23.810 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.001401734, 2024-04-25T00:27:23.810 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000467244 2024-04-25T00:27:23.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.811 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getfilelock_latency": { 2024-04-25T00:27:23.811 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:23.811 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:23.811 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:23.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.811 INFO:teuthology.orchestra.run.smithi195.stdout: "req_link_latency": { 2024-04-25T00:27:23.811 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 9, 2024-04-25T00:27:23.811 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.239493211, 2024-04-25T00:27:23.811 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.026610356 2024-04-25T00:27:23.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.811 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookup_latency": { 2024-04-25T00:27:23.811 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 497, 2024-04-25T00:27:23.811 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.186763939, 2024-04-25T00:27:23.811 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000375782 2024-04-25T00:27:23.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.812 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookuphash_latency": { 2024-04-25T00:27:23.812 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:23.812 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:23.812 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:23.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.812 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupino_latency": { 2024-04-25T00:27:23.812 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:23.812 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:23.812 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:23.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.812 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupname_latency": { 2024-04-25T00:27:23.812 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:23.812 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:23.812 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:23.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.813 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupparent_latency": { 2024-04-25T00:27:23.813 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:23.813 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:23.813 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:23.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.813 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lookupsnap_latency": { 2024-04-25T00:27:23.813 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:23.813 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:23.813 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:23.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.813 INFO:teuthology.orchestra.run.smithi195.stdout: "req_lssnap_latency": { 2024-04-25T00:27:23.813 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:23.814 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:23.814 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:23.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.814 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mkdir_latency": { 2024-04-25T00:27:23.814 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 20, 2024-04-25T00:27:23.815 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.041067342, 2024-04-25T00:27:23.815 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.002053367 2024-04-25T00:27:23.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.815 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mknod_latency": { 2024-04-25T00:27:23.815 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 26, 2024-04-25T00:27:23.815 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.046395333, 2024-04-25T00:27:23.815 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.001784435 2024-04-25T00:27:23.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.815 INFO:teuthology.orchestra.run.smithi195.stdout: "req_mksnap_latency": { 2024-04-25T00:27:23.815 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:23.816 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:23.816 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:23.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.816 INFO:teuthology.orchestra.run.smithi195.stdout: "req_open_latency": { 2024-04-25T00:27:23.816 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 1, 2024-04-25T00:27:23.816 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000357278, 2024-04-25T00:27:23.816 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000357278 2024-04-25T00:27:23.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.816 INFO:teuthology.orchestra.run.smithi195.stdout: "req_readdir_latency": { 2024-04-25T00:27:23.816 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 107, 2024-04-25T00:27:23.816 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.101153256, 2024-04-25T00:27:23.816 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000945357 2024-04-25T00:27:23.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.816 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rename_latency": { 2024-04-25T00:27:23.817 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 27, 2024-04-25T00:27:23.817 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.580248511, 2024-04-25T00:27:23.817 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.021490685 2024-04-25T00:27:23.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.817 INFO:teuthology.orchestra.run.smithi195.stdout: "req_renamesnap_latency": { 2024-04-25T00:27:23.817 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:23.817 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:23.817 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:23.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.817 INFO:teuthology.orchestra.run.smithi195.stdout: "req_snapdiff_latency": { 2024-04-25T00:27:23.817 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:23.817 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:23.817 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:23.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.818 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmdir_latency": { 2024-04-25T00:27:23.818 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 52, 2024-04-25T00:27:23.818 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.544356742, 2024-04-25T00:27:23.818 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.010468398 2024-04-25T00:27:23.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.818 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmsnap_latency": { 2024-04-25T00:27:23.818 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:23.818 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:23.818 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:23.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.818 INFO:teuthology.orchestra.run.smithi195.stdout: "req_rmxattr_latency": { 2024-04-25T00:27:23.818 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:23.818 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:23.818 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:23.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.819 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setattr_latency": { 2024-04-25T00:27:23.819 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 8, 2024-04-25T00:27:23.819 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.008350523, 2024-04-25T00:27:23.819 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.001043815 2024-04-25T00:27:23.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.819 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setdirlayout_latency": { 2024-04-25T00:27:23.819 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:23.819 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:23.819 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:23.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.819 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setfilelock_latency": { 2024-04-25T00:27:23.819 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:23.819 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:23.819 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:23.820 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.820 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setlayout_latency": { 2024-04-25T00:27:23.820 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:23.820 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:23.820 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:23.820 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.820 INFO:teuthology.orchestra.run.smithi195.stdout: "req_setxattr_latency": { 2024-04-25T00:27:23.820 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:23.820 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:23.820 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:23.820 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.820 INFO:teuthology.orchestra.run.smithi195.stdout: "req_symlink_latency": { 2024-04-25T00:27:23.820 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 15, 2024-04-25T00:27:23.820 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.022730402, 2024-04-25T00:27:23.820 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.001515360 2024-04-25T00:27:23.821 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.821 INFO:teuthology.orchestra.run.smithi195.stdout: "req_unlink_latency": { 2024-04-25T00:27:23.821 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 547, 2024-04-25T00:27:23.821 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 1.271265278, 2024-04-25T00:27:23.821 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.002324068 2024-04-25T00:27:23.821 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.821 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:27:23.821 INFO:teuthology.orchestra.run.smithi195.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:27:23.821 INFO:teuthology.orchestra.run.smithi195.stdout: "req_getvxattr_latency": { 2024-04-25T00:27:23.821 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:23.821 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:23.822 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:23.822 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:23.822 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.822 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_sessions": { 2024-04-25T00:27:23.822 INFO:teuthology.orchestra.run.smithi195.stdout: "session_count": 2, 2024-04-25T00:27:23.822 INFO:teuthology.orchestra.run.smithi195.stdout: "session_add": 3, 2024-04-25T00:27:23.822 INFO:teuthology.orchestra.run.smithi195.stdout: "session_remove": 1, 2024-04-25T00:27:23.822 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_open": 2, 2024-04-25T00:27:23.822 INFO:teuthology.orchestra.run.smithi195.stdout: "sessions_stale": 0, 2024-04-25T00:27:23.822 INFO:teuthology.orchestra.run.smithi195.stdout: "total_load": 923, 2024-04-25T00:27:23.822 INFO:teuthology.orchestra.run.smithi195.stdout: "average_load": 461, 2024-04-25T00:27:23.822 INFO:teuthology.orchestra.run.smithi195.stdout: "avg_session_uptime": 617, 2024-04-25T00:27:23.822 INFO:teuthology.orchestra.run.smithi195.stdout: "mdthresh_evicted": 0 2024-04-25T00:27:23.823 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.823 INFO:teuthology.orchestra.run.smithi195.stdout: "mempool": { 2024-04-25T00:27:23.823 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:27:23.823 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_items": 0, 2024-04-25T00:27:23.823 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:27:23.823 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:27:23.823 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:27:23.823 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:27:23.823 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:27:23.823 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:27:23.823 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:27:23.823 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:27:23.823 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:27:23.824 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:27:23.824 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:27:23.824 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:27:23.824 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:27:23.824 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_items": 0, 2024-04-25T00:27:23.824 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:27:23.824 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_items": 0, 2024-04-25T00:27:23.824 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:27:23.824 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:27:23.824 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:27:23.824 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:27:23.824 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:27:23.824 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:27:23.824 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:27:23.825 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_items": 0, 2024-04-25T00:27:23.825 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:27:23.825 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:27:23.825 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:27:23.825 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_items": 0, 2024-04-25T00:27:23.825 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_bytes": 0, 2024-04-25T00:27:23.825 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_items": 0, 2024-04-25T00:27:23.825 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:27:23.825 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:27:23.825 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:27:23.825 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:27:23.825 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_bytes": 327345, 2024-04-25T00:27:23.825 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_items": 153, 2024-04-25T00:27:23.825 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_bytes": 1144, 2024-04-25T00:27:23.825 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_items": 13, 2024-04-25T00:27:23.826 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_bytes": 0, 2024-04-25T00:27:23.826 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_items": 0, 2024-04-25T00:27:23.826 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:27:23.826 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_items": 0, 2024-04-25T00:27:23.826 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:27:23.826 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_items": 0, 2024-04-25T00:27:23.826 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_bytes": 9432, 2024-04-25T00:27:23.826 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_items": 182, 2024-04-25T00:27:23.826 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:27:23.826 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:27:23.826 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_bytes": 0, 2024-04-25T00:27:23.826 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_items": 0, 2024-04-25T00:27:23.826 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_bytes": 237879, 2024-04-25T00:27:23.826 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_items": 1788, 2024-04-25T00:27:23.827 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_bytes": 0, 2024-04-25T00:27:23.827 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_items": 0, 2024-04-25T00:27:23.827 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_bytes": 0, 2024-04-25T00:27:23.827 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_items": 0 2024-04-25T00:27:23.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.827 INFO:teuthology.orchestra.run.smithi195.stdout: "objecter": { 2024-04-25T00:27:23.827 INFO:teuthology.orchestra.run.smithi195.stdout: "op_active": 0, 2024-04-25T00:27:23.827 INFO:teuthology.orchestra.run.smithi195.stdout: "op_laggy": 0, 2024-04-25T00:27:23.827 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send": 3367, 2024-04-25T00:27:23.827 INFO:teuthology.orchestra.run.smithi195.stdout: "op_send_bytes": 6417290, 2024-04-25T00:27:23.827 INFO:teuthology.orchestra.run.smithi195.stdout: "op_resend": 29, 2024-04-25T00:27:23.827 INFO:teuthology.orchestra.run.smithi195.stdout: "op_reply": 3367, 2024-04-25T00:27:23.827 INFO:teuthology.orchestra.run.smithi195.stdout: "op_latency": { 2024-04-25T00:27:23.827 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 3367, 2024-04-25T00:27:23.828 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 8.253843330, 2024-04-25T00:27:23.828 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.002451393 2024-04-25T00:27:23.828 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.828 INFO:teuthology.orchestra.run.smithi195.stdout: "op_inflight": 0, 2024-04-25T00:27:23.828 INFO:teuthology.orchestra.run.smithi195.stdout: "oplen_avg": { 2024-04-25T00:27:23.828 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 3367, 2024-04-25T00:27:23.828 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 3505 2024-04-25T00:27:23.828 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.828 INFO:teuthology.orchestra.run.smithi195.stdout: "op": 3367, 2024-04-25T00:27:23.828 INFO:teuthology.orchestra.run.smithi195.stdout: "op_r": 1337, 2024-04-25T00:27:23.828 INFO:teuthology.orchestra.run.smithi195.stdout: "op_w": 2030, 2024-04-25T00:27:23.828 INFO:teuthology.orchestra.run.smithi195.stdout: "op_rmw": 0, 2024-04-25T00:27:23.828 INFO:teuthology.orchestra.run.smithi195.stdout: "op_pg": 0, 2024-04-25T00:27:23.828 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_stat": 0, 2024-04-25T00:27:23.828 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_create": 10, 2024-04-25T00:27:23.829 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_read": 104, 2024-04-25T00:27:23.829 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_write": 727, 2024-04-25T00:27:23.829 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writefull": 71, 2024-04-25T00:27:23.829 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_writesame": 0, 2024-04-25T00:27:23.829 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_append": 0, 2024-04-25T00:27:23.829 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_zero": 0, 2024-04-25T00:27:23.829 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_truncate": 0, 2024-04-25T00:27:23.829 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_delete": 1121, 2024-04-25T00:27:23.829 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_mapext": 0, 2024-04-25T00:27:23.829 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_sparse_read": 0, 2024-04-25T00:27:23.829 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_clonerange": 0, 2024-04-25T00:27:23.829 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_getxattr": 1233, 2024-04-25T00:27:23.829 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_setxattr": 20, 2024-04-25T00:27:23.829 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:27:23.829 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_rmxattr": 0, 2024-04-25T00:27:23.830 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:27:23.830 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_call": 0, 2024-04-25T00:27:23.830 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_watch": 0, 2024-04-25T00:27:23.830 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_notify": 0, 2024-04-25T00:27:23.830 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:27:23.830 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls": 0, 2024-04-25T00:27:23.830 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:27:23.830 INFO:teuthology.orchestra.run.smithi195.stdout: "osdop_other": 9, 2024-04-25T00:27:23.830 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_active": 0, 2024-04-25T00:27:23.830 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_send": 0, 2024-04-25T00:27:23.830 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_resend": 0, 2024-04-25T00:27:23.830 INFO:teuthology.orchestra.run.smithi195.stdout: "linger_ping": 0, 2024-04-25T00:27:23.830 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_active": 0, 2024-04-25T00:27:23.830 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_send": 0, 2024-04-25T00:27:23.831 INFO:teuthology.orchestra.run.smithi195.stdout: "poolop_resend": 0, 2024-04-25T00:27:23.831 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_active": 0, 2024-04-25T00:27:23.831 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_send": 0, 2024-04-25T00:27:23.831 INFO:teuthology.orchestra.run.smithi195.stdout: "poolstat_resend": 0, 2024-04-25T00:27:23.831 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_active": 0, 2024-04-25T00:27:23.831 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_send": 0, 2024-04-25T00:27:23.831 INFO:teuthology.orchestra.run.smithi195.stdout: "statfs_resend": 0, 2024-04-25T00:27:23.831 INFO:teuthology.orchestra.run.smithi195.stdout: "command_active": 0, 2024-04-25T00:27:23.831 INFO:teuthology.orchestra.run.smithi195.stdout: "command_send": 0, 2024-04-25T00:27:23.831 INFO:teuthology.orchestra.run.smithi195.stdout: "command_resend": 0, 2024-04-25T00:27:23.831 INFO:teuthology.orchestra.run.smithi195.stdout: "map_epoch": 107, 2024-04-25T00:27:23.831 INFO:teuthology.orchestra.run.smithi195.stdout: "map_full": 0, 2024-04-25T00:27:23.831 INFO:teuthology.orchestra.run.smithi195.stdout: "map_inc": 15, 2024-04-25T00:27:23.831 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_sessions": 12, 2024-04-25T00:27:23.831 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_open": 12, 2024-04-25T00:27:23.832 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_session_close": 0, 2024-04-25T00:27:23.832 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_laggy": 0, 2024-04-25T00:27:23.832 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_wr": 155, 2024-04-25T00:27:23.832 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_rd": 0, 2024-04-25T00:27:23.832 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_del": 55 2024-04-25T00:27:23.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.832 INFO:teuthology.orchestra.run.smithi195.stdout: "oft": { 2024-04-25T00:27:23.832 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_objs": 1, 2024-04-25T00:27:23.832 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:27:23.832 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_updates": 0, 2024-04-25T00:27:23.832 INFO:teuthology.orchestra.run.smithi195.stdout: "omap_total_removes": 0 2024-04-25T00:27:23.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.832 INFO:teuthology.orchestra.run.smithi195.stdout: "osd-slow-ops": { 2024-04-25T00:27:23.832 INFO:teuthology.orchestra.run.smithi195.stdout: "slow_ops_count": 0 2024-04-25T00:27:23.833 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.833 INFO:teuthology.orchestra.run.smithi195.stdout: "purge_queue": { 2024-04-25T00:27:23.833 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops": 0, 2024-04-25T00:27:23.833 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_ops_high_water": 21, 2024-04-25T00:27:23.833 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing": 0, 2024-04-25T00:27:23.833 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executing_high_water": 12, 2024-04-25T00:27:23.833 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed_ops": 1108, 2024-04-25T00:27:23.833 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_executed": 581, 2024-04-25T00:27:23.833 INFO:teuthology.orchestra.run.smithi195.stdout: "pq_item_in_journal": 0 2024-04-25T00:27:23.833 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.833 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:27:23.833 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-04-25T00:27:23.833 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-04-25T00:27:23.833 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-04-25T00:27:23.833 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 17361, 2024-04-25T00:27:23.834 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 6145491, 2024-04-25T00:27:23.834 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:23.834 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 17361, 2024-04-25T00:27:23.834 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-04-25T00:27:23.834 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-04-25T00:27:23.834 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 17361, 2024-04-25T00:27:23.834 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 6145491, 2024-04-25T00:27:23.834 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-04-25T00:27:23.834 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:23.834 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:23.834 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:23.834 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:23.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.834 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_bytes": { 2024-04-25T00:27:23.835 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-04-25T00:27:23.835 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-04-25T00:27:23.835 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-04-25T00:27:23.835 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-04-25T00:27:23.835 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-04-25T00:27:23.835 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:23.835 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-04-25T00:27:23.835 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 3367, 2024-04-25T00:27:23.835 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 6483803, 2024-04-25T00:27:23.835 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 3367, 2024-04-25T00:27:23.835 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 6483803, 2024-04-25T00:27:23.835 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-04-25T00:27:23.835 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:23.835 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:23.836 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:23.836 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:23.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.836 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-objecter_ops": { 2024-04-25T00:27:23.836 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-04-25T00:27:23.836 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 1024, 2024-04-25T00:27:23.836 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-04-25T00:27:23.836 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 0, 2024-04-25T00:27:23.836 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 0, 2024-04-25T00:27:23.836 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:23.836 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 0, 2024-04-25T00:27:23.836 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 3367, 2024-04-25T00:27:23.836 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 3367, 2024-04-25T00:27:23.836 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 3367, 2024-04-25T00:27:23.836 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 3367, 2024-04-25T00:27:23.837 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-04-25T00:27:23.837 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:23.837 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:23.837 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:23.837 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:23.837 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.837 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:27:23.837 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-04-25T00:27:23.837 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-04-25T00:27:23.837 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-04-25T00:27:23.837 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 581, 2024-04-25T00:27:23.837 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 66513, 2024-04-25T00:27:23.837 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:23.837 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 581, 2024-04-25T00:27:23.838 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-04-25T00:27:23.838 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-04-25T00:27:23.838 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 104, 2024-04-25T00:27:23.838 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 66513, 2024-04-25T00:27:23.838 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-04-25T00:27:23.838 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:23.838 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:23.838 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:23.838 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:23.838 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:23.838 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-write_buf_throttle-0x5570a4314500": { 2024-04-25T00:27:23.838 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-04-25T00:27:23.838 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 3758096384, 2024-04-25T00:27:23.838 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-04-25T00:27:23.838 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 2752, 2024-04-25T00:27:23.839 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 6253772, 2024-04-25T00:27:23.839 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:23.839 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 2752, 2024-04-25T00:27:23.839 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-04-25T00:27:23.839 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-04-25T00:27:23.839 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 623, 2024-04-25T00:27:23.839 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 6253772, 2024-04-25T00:27:23.839 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-04-25T00:27:23.839 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:23.839 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:23.839 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:23.839 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:23.839 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:23.839 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-04-25T00:27:23.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:23 smithi107 ceph-mon[29153]: pgmap v341: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:27:24.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:23 smithi170 ceph-mon[32807]: pgmap v341: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:27:24.605 INFO:tasks.check_counter:Daemon mds.i mds.exported=63 2024-04-25T00:27:24.605 INFO:tasks.check_counter:Daemon mds.i mds.imported=69 2024-04-25T00:27:24.605 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-25T00:27:24.605 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-25T00:27:24.605 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T00:27:24.605 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T00:27:24.606 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 79602f58-0297-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T00:27:24.842 INFO:teuthology.orchestra.run.smithi195.stderr:Inferring config /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:27:25.971 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:25 smithi195 ceph-mon[32833]: pgmap v342: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:27:25.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:25 smithi107 ceph-mon[29153]: pgmap v342: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:27:26.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:25 smithi170 ceph-mon[32807]: pgmap v342: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:27:26.568 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-04-25T00:27:26.568 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:27:26.568 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 0, 2024-04-25T00:27:26.568 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 2, 2024-04-25T00:27:26.568 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 0, 2024-04-25T00:27:26.568 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 0, 2024-04-25T00:27:26.569 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 4, 2024-04-25T00:27:26.569 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 0, 2024-04-25T00:27:26.569 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.001058454, 2024-04-25T00:27:26.569 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T00:27:26.569 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.000289177, 2024-04-25T00:27:26.569 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:27:26.569 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:26.569 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:26.569 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:26.569 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:26.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:26.569 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:26.569 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:26.569 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:26.569 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:26.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:26.570 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:27:26.570 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:27:26.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:26.570 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:27:26.570 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 2, 2024-04-25T00:27:26.570 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 172, 2024-04-25T00:27:26.570 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 320, 2024-04-25T00:27:26.570 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 185680, 2024-04-25T00:27:26.570 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 5, 2024-04-25T00:27:26.570 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 3, 2024-04-25T00:27:26.570 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.012127482, 2024-04-25T00:27:26.570 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.008191056, 2024-04-25T00:27:26.570 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.001518302, 2024-04-25T00:27:26.571 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:27:26.571 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:26.571 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 169, 2024-04-25T00:27:26.571 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.015623644, 2024-04-25T00:27:26.571 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000092447 2024-04-25T00:27:26.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:26.571 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:26.571 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:26.571 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:26.571 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:26.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:26.571 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T00:27:26.571 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 185680 2024-04-25T00:27:26.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:26.571 INFO:teuthology.orchestra.run.smithi195.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:27:26.571 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_messages": 240, 2024-04-25T00:27:26.571 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages": 228, 2024-04-25T00:27:26.572 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_bytes": 1450112, 2024-04-25T00:27:26.572 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_bytes": 139392, 2024-04-25T00:27:26.572 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_created_connections": 2, 2024-04-25T00:27:26.572 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_active_connections": 1, 2024-04-25T00:27:26.572 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_total_time": 0.047889748, 2024-04-25T00:27:26.572 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_send_time": 0.014912215, 2024-04-25T00:27:26.572 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_recv_time": 0.034942635, 2024-04-25T00:27:26.572 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_running_fast_dispatch_time": 0.005766903, 2024-04-25T00:27:26.572 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:27:26.572 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 228, 2024-04-25T00:27:26.572 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.012458872, 2024-04-25T00:27:26.572 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000054644 2024-04-25T00:27:26.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:26.572 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:27:26.572 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:26.573 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:26.573 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:26.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:26.573 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_recv_encrypted_bytes": 1450112, 2024-04-25T00:27:26.573 INFO:teuthology.orchestra.run.smithi195.stdout: "msgr_send_encrypted_bytes": 139392 2024-04-25T00:27:26.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:26.573 INFO:teuthology.orchestra.run.smithi195.stdout: "cct": { 2024-04-25T00:27:26.573 INFO:teuthology.orchestra.run.smithi195.stdout: "total_workers": 0, 2024-04-25T00:27:26.573 INFO:teuthology.orchestra.run.smithi195.stdout: "unhealthy_workers": 0 2024-04-25T00:27:26.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:26.573 INFO:teuthology.orchestra.run.smithi195.stdout: "mempool": { 2024-04-25T00:27:26.573 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:27:26.573 INFO:teuthology.orchestra.run.smithi195.stdout: "bloom_filter_items": 0, 2024-04-25T00:27:26.573 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:27:26.573 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:27:26.573 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:27:26.574 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:27:26.574 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:27:26.574 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:27:26.574 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:27:26.574 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:27:26.574 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:27:26.574 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:27:26.574 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:27:26.574 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:27:26.574 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:27:26.574 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_extent_items": 0, 2024-04-25T00:27:26.574 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:27:26.574 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_blob_items": 0, 2024-04-25T00:27:26.574 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:27:26.574 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:27:26.575 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:27:26.575 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:27:26.575 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:27:26.575 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:27:26.575 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:27:26.575 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_txc_items": 0, 2024-04-25T00:27:26.575 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:27:26.575 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:27:26.575 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:27:26.575 INFO:teuthology.orchestra.run.smithi195.stdout: "bluestore_writing_items": 0, 2024-04-25T00:27:26.575 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_bytes": 0, 2024-04-25T00:27:26.575 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_items": 0, 2024-04-25T00:27:26.575 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:27:26.575 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:27:26.575 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:27:26.575 INFO:teuthology.orchestra.run.smithi195.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:27:26.576 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_bytes": 148794, 2024-04-25T00:27:26.576 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_anon_items": 32, 2024-04-25T00:27:26.576 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:27:26.576 INFO:teuthology.orchestra.run.smithi195.stdout: "buffer_meta_items": 0, 2024-04-25T00:27:26.576 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_bytes": 0, 2024-04-25T00:27:26.576 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_items": 0, 2024-04-25T00:27:26.576 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:27:26.576 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_mapbl_items": 0, 2024-04-25T00:27:26.576 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:27:26.576 INFO:teuthology.orchestra.run.smithi195.stdout: "osd_pglog_items": 0, 2024-04-25T00:27:26.576 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_bytes": 0, 2024-04-25T00:27:26.576 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_items": 0, 2024-04-25T00:27:26.576 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:27:26.576 INFO:teuthology.orchestra.run.smithi195.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:27:26.576 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_bytes": 0, 2024-04-25T00:27:26.576 INFO:teuthology.orchestra.run.smithi195.stdout: "pgmap_items": 0, 2024-04-25T00:27:26.576 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_bytes": 928, 2024-04-25T00:27:26.577 INFO:teuthology.orchestra.run.smithi195.stdout: "mds_co_items": 2, 2024-04-25T00:27:26.577 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_bytes": 0, 2024-04-25T00:27:26.577 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_1_items": 0, 2024-04-25T00:27:26.577 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_bytes": 0, 2024-04-25T00:27:26.577 INFO:teuthology.orchestra.run.smithi195.stdout: "unittest_2_items": 0 2024-04-25T00:27:26.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-04-25T00:27:26.577 INFO:teuthology.orchestra.run.smithi195.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:27:26.577 INFO:teuthology.orchestra.run.smithi195.stdout: "val": 0, 2024-04-25T00:27:26.577 INFO:teuthology.orchestra.run.smithi195.stdout: "max": 104857600, 2024-04-25T00:27:26.577 INFO:teuthology.orchestra.run.smithi195.stdout: "get_started": 0, 2024-04-25T00:27:26.577 INFO:teuthology.orchestra.run.smithi195.stdout: "get": 238, 2024-04-25T00:27:26.577 INFO:teuthology.orchestra.run.smithi195.stdout: "get_sum": 1413828, 2024-04-25T00:27:26.577 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_fail": 0, 2024-04-25T00:27:26.577 INFO:teuthology.orchestra.run.smithi195.stdout: "get_or_fail_success": 238, 2024-04-25T00:27:26.577 INFO:teuthology.orchestra.run.smithi195.stdout: "take": 0, 2024-04-25T00:27:26.577 INFO:teuthology.orchestra.run.smithi195.stdout: "take_sum": 0, 2024-04-25T00:27:26.578 INFO:teuthology.orchestra.run.smithi195.stdout: "put": 238, 2024-04-25T00:27:26.578 INFO:teuthology.orchestra.run.smithi195.stdout: "put_sum": 1413828, 2024-04-25T00:27:26.578 INFO:teuthology.orchestra.run.smithi195.stdout: "wait": { 2024-04-25T00:27:26.578 INFO:teuthology.orchestra.run.smithi195.stdout: "avgcount": 0, 2024-04-25T00:27:26.578 INFO:teuthology.orchestra.run.smithi195.stdout: "sum": 0.000000000, 2024-04-25T00:27:26.578 INFO:teuthology.orchestra.run.smithi195.stdout: "avgtime": 0.000000000 2024-04-25T00:27:26.578 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:26.578 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-04-25T00:27:26.578 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-04-25T00:27:27.511 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-25T00:27:27.511 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-25T00:27:27.511 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-25T00:27:27.511 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-25T00:27:27.511 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-25T00:27:27.512 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T00:27:27.523 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T00:27:27.523 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T00:27:27.523 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:27:27.545 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T00:27:27.545 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:27:27.605 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:27:27.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.756+0000 7f2581f3b640 1 -- 172.21.15.107:0/3237395863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f257c103340 msgr2=0x7f257c10b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.756+0000 7f2581f3b640 1 --2- 172.21.15.107:0/3237395863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f257c103340 0x7f257c10b820 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f2570009900 tx=0x7f257002a480 comp rx=0 tx=0).stop 2024-04-25T00:27:27.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.756+0000 7f2581f3b640 1 -- 172.21.15.107:0/3237395863 shutdown_connections 2024-04-25T00:27:27.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.756+0000 7f2581f3b640 1 --2- 172.21.15.107:0/3237395863 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f257c10bd60 0x7f257c10e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.756+0000 7f2581f3b640 1 --2- 172.21.15.107:0/3237395863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f257c103340 0x7f257c10b820 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.756+0000 7f2581f3b640 1 --2- 172.21.15.107:0/3237395863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f257c102a20 0x7f257c102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.756+0000 7f2581f3b640 1 -- 172.21.15.107:0/3237395863 >> 172.21.15.107:0/3237395863 conn(0x7f257c0777a0 msgr2=0x7f257c077bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:27.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.756+0000 7f2581f3b640 1 -- 172.21.15.107:0/3237395863 shutdown_connections 2024-04-25T00:27:27.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.756+0000 7f2581f3b640 1 -- 172.21.15.107:0/3237395863 wait complete. 2024-04-25T00:27:27.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.756+0000 7f2581f3b640 1 Processor -- start 2024-04-25T00:27:27.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.756+0000 7f2581f3b640 1 -- start start 2024-04-25T00:27:27.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.757+0000 7f2581f3b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f257c102a20 0x7f257c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.757+0000 7f2581f3b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f257c103340 0x7f257c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.757+0000 7f2581f3b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f257c10bd60 0x7f257c19c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.757+0000 7f2581f3b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f257c111b30 con 0x7f257c10bd60 2024-04-25T00:27:27.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.757+0000 7f2581f3b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f257c1119b0 con 0x7f257c103340 2024-04-25T00:27:27.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.757+0000 7f2581f3b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f257c111cb0 con 0x7f257c102a20 2024-04-25T00:27:27.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.757+0000 7f257bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f257c10bd60 0x7f257c19c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.757+0000 7f257bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f257c10bd60 0x7f257c19c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58742/0 (socket says 172.21.15.107:58742) 2024-04-25T00:27:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.757+0000 7f257bfff640 1 -- 172.21.15.107:0/2389207333 learned_addr learned my addr 172.21.15.107:0/2389207333 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:27:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.757+0000 7f257affd640 1 --2- 172.21.15.107:0/2389207333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f257c103340 0x7f257c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.757+0000 7f257b7fe640 1 --2- 172.21.15.107:0/2389207333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f257c102a20 0x7f257c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.757+0000 7f257bfff640 1 -- 172.21.15.107:0/2389207333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f257c102a20 msgr2=0x7f257c1980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.757+0000 7f257bfff640 1 --2- 172.21.15.107:0/2389207333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f257c102a20 0x7f257c1980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.757+0000 7f257bfff640 1 -- 172.21.15.107:0/2389207333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f257c103340 msgr2=0x7f257c198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.757+0000 7f257bfff640 1 --2- 172.21.15.107:0/2389207333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f257c103340 0x7f257c198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.757+0000 7f257bfff640 1 -- 172.21.15.107:0/2389207333 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f257c19d070 con 0x7f257c10bd60 2024-04-25T00:27:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.757+0000 7f257bfff640 1 --2- 172.21.15.107:0/2389207333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f257c10bd60 0x7f257c19c940 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f257400ea60 tx=0x7f257400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.757+0000 7f2578ff9640 1 -- 172.21.15.107:0/2389207333 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257400ce40 con 0x7f257c10bd60 2024-04-25T00:27:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.757+0000 7f2578ff9640 1 -- 172.21.15.107:0/2389207333 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2574005020 con 0x7f257c10bd60 2024-04-25T00:27:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.757+0000 7f2581f3b640 1 -- 172.21.15.107:0/2389207333 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f257c19d360 con 0x7f257c10bd60 2024-04-25T00:27:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.757+0000 7f2578ff9640 1 -- 172.21.15.107:0/2389207333 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2574010690 con 0x7f257c10bd60 2024-04-25T00:27:27.761 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.758+0000 7f2581f3b640 1 -- 172.21.15.107:0/2389207333 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f257c1a4bf0 con 0x7f257c10bd60 2024-04-25T00:27:27.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.759+0000 7f2581f3b640 1 -- 172.21.15.107:0/2389207333 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f257c0689b0 con 0x7f257c10bd60 2024-04-25T00:27:27.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.760+0000 7f2578ff9640 1 -- 172.21.15.107:0/2389207333 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f2574005520 con 0x7f257c10bd60 2024-04-25T00:27:27.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.760+0000 7f2578ff9640 1 --2- 172.21.15.107:0/2389207333 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f25640a2d00 0x7f25640a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:27.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.761+0000 7f257b7fe640 1 --2- 172.21.15.107:0/2389207333 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f25640a2d00 0x7f25640a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:27.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.761+0000 7f2578ff9640 1 -- 172.21.15.107:0/2389207333 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f25740cfc20 con 0x7f257c10bd60 2024-04-25T00:27:27.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.761+0000 7f257b7fe640 1 --2- 172.21.15.107:0/2389207333 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f25640a2d00 0x7f25640a51c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f256c00bbe0 tx=0x7f256c00b390 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:27.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.765+0000 7f2578ff9640 1 -- 172.21.15.107:0/2389207333 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f257409c480 con 0x7f257c10bd60 2024-04-25T00:27:27.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:27 smithi195 ceph-mon[32833]: pgmap v343: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:27:27.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.944+0000 7f2581f3b640 1 -- 172.21.15.107:0/2389207333 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f257c106950 con 0x7f257c10bd60 2024-04-25T00:27:27.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.945+0000 7f2578ff9640 1 -- 172.21.15.107:0/2389207333 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+25256 (secure 0 0 0) 0x7f257409bd80 con 0x7f257c10bd60 2024-04-25T00:27:27.947 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:27:27.947 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":107,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:27:16.683152+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:59.672801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:00.957782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:14:05.065846+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8},{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/151603643":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/1884923543":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/111369358":"2024-04-26T00:16:26.229442+0000","172.21.15.107:6800/1644942185":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3936375305":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/1644942185":"2024-04-26T00:16:26.229442+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:27:27.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.947+0000 7f2581f3b640 1 -- 172.21.15.107:0/2389207333 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f25640a2d00 msgr2=0x7f25640a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.947+0000 7f2581f3b640 1 --2- 172.21.15.107:0/2389207333 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f25640a2d00 0x7f25640a51c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f256c00bbe0 tx=0x7f256c00b390 comp rx=0 tx=0).stop 2024-04-25T00:27:27.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.947+0000 7f2581f3b640 1 -- 172.21.15.107:0/2389207333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f257c10bd60 msgr2=0x7f257c19c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:27.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.947+0000 7f2581f3b640 1 --2- 172.21.15.107:0/2389207333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f257c10bd60 0x7f257c19c940 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f257400ea60 tx=0x7f257400ef20 comp rx=0 tx=0).stop 2024-04-25T00:27:27.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.947+0000 7f2581f3b640 1 -- 172.21.15.107:0/2389207333 shutdown_connections 2024-04-25T00:27:27.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.947+0000 7f2581f3b640 1 --2- 172.21.15.107:0/2389207333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f257c10bd60 0x7f257c19c940 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.947+0000 7f2581f3b640 1 --2- 172.21.15.107:0/2389207333 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f25640a2d00 0x7f25640a51c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.947+0000 7f2581f3b640 1 --2- 172.21.15.107:0/2389207333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f257c103340 0x7f257c198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.947+0000 7f2581f3b640 1 --2- 172.21.15.107:0/2389207333 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f257c102a20 0x7f257c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:27.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.947+0000 7f2581f3b640 1 -- 172.21.15.107:0/2389207333 >> 172.21.15.107:0/2389207333 conn(0x7f257c0777a0 msgr2=0x7f257c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:27.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.947+0000 7f2581f3b640 1 -- 172.21.15.107:0/2389207333 shutdown_connections 2024-04-25T00:27:27.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:27.947+0000 7f2581f3b640 1 -- 172.21.15.107:0/2389207333 wait complete. 2024-04-25T00:27:27.958 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:27 smithi107 ceph-mon[29153]: pgmap v343: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:27:27.960 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:09:25.160329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:13:59.672801+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:14:00.957782+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 106, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T00:14:05.065846+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 106, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T00:27:27.960 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:27:28.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.059+0000 7fa134415640 1 -- 172.21.15.107:0/1223206668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa12c107730 msgr2=0x7fa12c107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.059+0000 7fa134415640 1 --2- 172.21.15.107:0/1223206668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa12c107730 0x7fa12c107b10 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fa128009930 tx=0x7fa12802a420 comp rx=0 tx=0).stop 2024-04-25T00:27:28.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.060+0000 7fa134415640 1 -- 172.21.15.107:0/1223206668 shutdown_connections 2024-04-25T00:27:28.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.060+0000 7fa134415640 1 --2- 172.21.15.107:0/1223206668 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa12c1020e0 0x7fa12c10e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.060+0000 7fa134415640 1 --2- 172.21.15.107:0/1223206668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa12c101720 0x7fa12c101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.060+0000 7fa134415640 1 --2- 172.21.15.107:0/1223206668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa12c107730 0x7fa12c107b10 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.060+0000 7fa134415640 1 -- 172.21.15.107:0/1223206668 >> 172.21.15.107:0/1223206668 conn(0x7fa12c078b40 msgr2=0x7fa12c078f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.060+0000 7fa134415640 1 -- 172.21.15.107:0/1223206668 shutdown_connections 2024-04-25T00:27:28.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.060+0000 7fa134415640 1 -- 172.21.15.107:0/1223206668 wait complete. 2024-04-25T00:27:28.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.060+0000 7fa134415640 1 Processor -- start 2024-04-25T00:27:28.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.060+0000 7fa134415640 1 -- start start 2024-04-25T00:27:28.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.060+0000 7fa134415640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa12c101720 0x7fa12c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.060+0000 7fa134415640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa12c1020e0 0x7fa12c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.060+0000 7fa134415640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa12c107730 0x7fa12c19c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.060+0000 7fa134415640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c111b00 con 0x7fa12c101720 2024-04-25T00:27:28.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.060+0000 7fa134415640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c111980 con 0x7fa12c1020e0 2024-04-25T00:27:28.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.060+0000 7fa134415640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c111c80 con 0x7fa12c107730 2024-04-25T00:27:28.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.061+0000 7fa1321b1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa12c101720 0x7fa12c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.061+0000 7fa1321b1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa12c101720 0x7fa12c198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58758/0 (socket says 172.21.15.107:58758) 2024-04-25T00:27:28.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.061+0000 7fa1321b1640 1 -- 172.21.15.107:0/4142925069 learned_addr learned my addr 172.21.15.107:0/4142925069 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:27:28.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.061+0000 7fa1319b0640 1 --2- 172.21.15.107:0/4142925069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa12c1020e0 0x7fa12c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.061+0000 7fa1329b2640 1 --2- 172.21.15.107:0/4142925069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa12c107730 0x7fa12c19c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.061+0000 7fa1321b1640 1 -- 172.21.15.107:0/4142925069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa12c107730 msgr2=0x7fa12c19c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.061+0000 7fa1321b1640 1 --2- 172.21.15.107:0/4142925069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa12c107730 0x7fa12c19c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.061+0000 7fa1321b1640 1 -- 172.21.15.107:0/4142925069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa12c1020e0 msgr2=0x7fa12c1985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.061+0000 7fa1321b1640 1 --2- 172.21.15.107:0/4142925069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa12c1020e0 0x7fa12c1985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.061+0000 7fa1321b1640 1 -- 172.21.15.107:0/4142925069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12c19d050 con 0x7fa12c101720 2024-04-25T00:27:28.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.061+0000 7fa1319b0640 1 --2- 172.21.15.107:0/4142925069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa12c1020e0 0x7fa12c1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:27:28.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.061+0000 7fa1329b2640 1 --2- 172.21.15.107:0/4142925069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa12c107730 0x7fa12c19c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:27:28.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.061+0000 7fa1321b1640 1 --2- 172.21.15.107:0/4142925069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa12c101720 0x7fa12c198060 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fa128009a50 tx=0x7fa128004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.061+0000 7fa1237fe640 1 -- 172.21.15.107:0/4142925069 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12803f030 con 0x7fa12c101720 2024-04-25T00:27:28.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.061+0000 7fa1237fe640 1 -- 172.21.15.107:0/4142925069 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa128003ce0 con 0x7fa12c101720 2024-04-25T00:27:28.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.061+0000 7fa134415640 1 -- 172.21.15.107:0/4142925069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa12c19d2e0 con 0x7fa12c101720 2024-04-25T00:27:28.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.061+0000 7fa1237fe640 1 -- 172.21.15.107:0/4142925069 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa128035070 con 0x7fa12c101720 2024-04-25T00:27:28.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.061+0000 7fa134415640 1 -- 172.21.15.107:0/4142925069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa12c1a4bd0 con 0x7fa12c101720 2024-04-25T00:27:28.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.062+0000 7fa134415640 1 -- 172.21.15.107:0/4142925069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa12c068560 con 0x7fa12c101720 2024-04-25T00:27:28.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.063+0000 7fa1237fe640 1 -- 172.21.15.107:0/4142925069 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa128042030 con 0x7fa12c101720 2024-04-25T00:27:28.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.064+0000 7fa1237fe640 1 --2- 172.21.15.107:0/4142925069 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa1180a2d00 0x7fa1180a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.064+0000 7fa1319b0640 1 --2- 172.21.15.107:0/4142925069 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa1180a2d00 0x7fa1180a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.064+0000 7fa1237fe640 1 -- 172.21.15.107:0/4142925069 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7fa1280efe50 con 0x7fa12c101720 2024-04-25T00:27:28.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.064+0000 7fa1319b0640 1 --2- 172.21.15.107:0/4142925069 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa1180a2d00 0x7fa1180a51c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa12c103960 tx=0x7fa11c005d30 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.068+0000 7fa1237fe640 1 -- 172.21.15.107:0/4142925069 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa1280bc6b0 con 0x7fa12c101720 2024-04-25T00:27:28.184 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:27 smithi170 ceph-mon[32807]: pgmap v343: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T00:27:28.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.243+0000 7fa134415640 1 -- 172.21.15.107:0/4142925069 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa12c109c90 con 0x7fa12c101720 2024-04-25T00:27:28.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.243+0000 7fa1237fe640 1 -- 172.21.15.107:0/4142925069 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) ==== 93+0+10 (secure 0 0 0) 0x7fa1280bbfb0 con 0x7fa12c101720 2024-04-25T00:27:28.245 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-04-25T00:27:28.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.245+0000 7fa134415640 1 -- 172.21.15.107:0/4142925069 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa1180a2d00 msgr2=0x7fa1180a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.245+0000 7fa134415640 1 --2- 172.21.15.107:0/4142925069 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa1180a2d00 0x7fa1180a51c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa12c103960 tx=0x7fa11c005d30 comp rx=0 tx=0).stop 2024-04-25T00:27:28.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.245+0000 7fa134415640 1 -- 172.21.15.107:0/4142925069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa12c101720 msgr2=0x7fa12c198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.245+0000 7fa134415640 1 --2- 172.21.15.107:0/4142925069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa12c101720 0x7fa12c198060 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fa128009a50 tx=0x7fa128004270 comp rx=0 tx=0).stop 2024-04-25T00:27:28.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.245+0000 7fa134415640 1 -- 172.21.15.107:0/4142925069 shutdown_connections 2024-04-25T00:27:28.247 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.245+0000 7fa134415640 1 --2- 172.21.15.107:0/4142925069 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa12c107730 0x7fa12c19c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.245+0000 7fa134415640 1 --2- 172.21.15.107:0/4142925069 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa1180a2d00 0x7fa1180a51c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.245+0000 7fa134415640 1 --2- 172.21.15.107:0/4142925069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa12c1020e0 0x7fa12c1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.245+0000 7fa134415640 1 --2- 172.21.15.107:0/4142925069 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa12c101720 0x7fa12c198060 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.245+0000 7fa134415640 1 -- 172.21.15.107:0/4142925069 >> 172.21.15.107:0/4142925069 conn(0x7fa12c078b40 msgr2=0x7fa12c104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.245+0000 7fa134415640 1 -- 172.21.15.107:0/4142925069 shutdown_connections 2024-04-25T00:27:28.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.245+0000 7fa134415640 1 -- 172.21.15.107:0/4142925069 wait complete. 2024-04-25T00:27:28.257 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:27:28.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.356+0000 7f1ace21b640 1 -- 172.21.15.107:0/2184075306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ac81066b0 msgr2=0x7f1ac8106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.356+0000 7f1ace21b640 1 --2- 172.21.15.107:0/2184075306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ac81066b0 0x7f1ac8106a90 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009930 tx=0x7f1ab802a420 comp rx=0 tx=0).stop 2024-04-25T00:27:28.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.356+0000 7f1ace21b640 1 -- 172.21.15.107:0/2184075306 shutdown_connections 2024-04-25T00:27:28.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.356+0000 7f1ace21b640 1 --2- 172.21.15.107:0/2184075306 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ac8100fd0 0x7f1ac810d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.356+0000 7f1ace21b640 1 --2- 172.21.15.107:0/2184075306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ac8100610 0x7f1ac8100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.356+0000 7f1ace21b640 1 --2- 172.21.15.107:0/2184075306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ac81066b0 0x7f1ac8106a90 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.356+0000 7f1ace21b640 1 -- 172.21.15.107:0/2184075306 >> 172.21.15.107:0/2184075306 conn(0x7f1ac8077740 msgr2=0x7f1ac8077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.356+0000 7f1ace21b640 1 -- 172.21.15.107:0/2184075306 shutdown_connections 2024-04-25T00:27:28.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.356+0000 7f1ace21b640 1 -- 172.21.15.107:0/2184075306 wait complete. 2024-04-25T00:27:28.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.357+0000 7f1ace21b640 1 Processor -- start 2024-04-25T00:27:28.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.357+0000 7f1ace21b640 1 -- start start 2024-04-25T00:27:28.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.357+0000 7f1ace21b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ac8100610 0x7f1ac8196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.357+0000 7f1ace21b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ac8100fd0 0x7f1ac8197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.357+0000 7f1ace21b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ac81066b0 0x7f1ac819b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.357+0000 7f1ace21b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac8110a30 con 0x7f1ac8100610 2024-04-25T00:27:28.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.357+0000 7f1ace21b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac81108b0 con 0x7f1ac81066b0 2024-04-25T00:27:28.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.357+0000 7f1ace21b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac8110bb0 con 0x7f1ac8100fd0 2024-04-25T00:27:28.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.357+0000 7f1ac77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ac8100610 0x7f1ac8196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.357+0000 7f1ac77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ac8100610 0x7f1ac8196ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58782/0 (socket says 172.21.15.107:58782) 2024-04-25T00:27:28.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.357+0000 7f1ac77fe640 1 -- 172.21.15.107:0/2193718588 learned_addr learned my addr 172.21.15.107:0/2193718588 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:27:28.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.358+0000 7f1ac7fff640 1 --2- 172.21.15.107:0/2193718588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ac81066b0 0x7f1ac819b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.358+0000 7f1ac77fe640 1 -- 172.21.15.107:0/2193718588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ac8100fd0 msgr2=0x7f1ac8197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.358+0000 7f1ac77fe640 1 --2- 172.21.15.107:0/2193718588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ac8100fd0 0x7f1ac8197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.358+0000 7f1ac77fe640 1 -- 172.21.15.107:0/2193718588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ac81066b0 msgr2=0x7f1ac819b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.358+0000 7f1ac77fe640 1 --2- 172.21.15.107:0/2193718588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ac81066b0 0x7f1ac819b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.358+0000 7f1ac77fe640 1 -- 172.21.15.107:0/2193718588 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac819bee0 con 0x7f1ac8100610 2024-04-25T00:27:28.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.358+0000 7f1ac7fff640 1 --2- 172.21.15.107:0/2193718588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ac81066b0 0x7f1ac819b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:28.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.358+0000 7f1ac77fe640 1 --2- 172.21.15.107:0/2193718588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ac8100610 0x7f1ac8196ef0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8005940 tx=0x7f1ab80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.358+0000 7f1ac57fa640 1 -- 172.21.15.107:0/2193718588 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab803f030 con 0x7f1ac8100610 2024-04-25T00:27:28.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.358+0000 7f1ace21b640 1 -- 172.21.15.107:0/2193718588 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac819c170 con 0x7f1ac8100610 2024-04-25T00:27:28.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.358+0000 7f1ac57fa640 1 -- 172.21.15.107:0/2193718588 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ab8003ce0 con 0x7f1ac8100610 2024-04-25T00:27:28.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.358+0000 7f1ac57fa640 1 -- 172.21.15.107:0/2193718588 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8035070 con 0x7f1ac8100610 2024-04-25T00:27:28.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.358+0000 7f1ace21b640 1 -- 172.21.15.107:0/2193718588 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac81a3a60 con 0x7f1ac8100610 2024-04-25T00:27:28.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.359+0000 7f1ace21b640 1 -- 172.21.15.107:0/2193718588 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac8068560 con 0x7f1ac8100610 2024-04-25T00:27:28.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.360+0000 7f1ac57fa640 1 -- 172.21.15.107:0/2193718588 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f1ab8042030 con 0x7f1ac8100610 2024-04-25T00:27:28.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.361+0000 7f1ac57fa640 1 --2- 172.21.15.107:0/2193718588 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f1ab40a2d00 0x7f1ab40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.361+0000 7f1abeffd640 1 --2- 172.21.15.107:0/2193718588 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f1ab40a2d00 0x7f1ab40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.361+0000 7f1ac57fa640 1 -- 172.21.15.107:0/2193718588 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f1ab80efde0 con 0x7f1ac8100610 2024-04-25T00:27:28.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.361+0000 7f1abeffd640 1 --2- 172.21.15.107:0/2193718588 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f1ab40a2d00 0x7f1ab40a51c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8105d50 tx=0x7f1ab0006c90 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.365+0000 7f1ac57fa640 1 -- 172.21.15.107:0/2193718588 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1ab80bc640 con 0x7f1ac8100610 2024-04-25T00:27:28.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.539+0000 7f1ace21b640 1 -- 172.21.15.107:0/2193718588 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1ac8108bb0 con 0x7f1ac8100610 2024-04-25T00:27:28.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.539+0000 7f1ac57fa640 1 -- 172.21.15.107:0/2193718588 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) ==== 104+0+11 (secure 0 0 0) 0x7f1ab80bbf40 con 0x7f1ac8100610 2024-04-25T00:27:28.541 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-25T00:27:28.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.541+0000 7f1ace21b640 1 -- 172.21.15.107:0/2193718588 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f1ab40a2d00 msgr2=0x7f1ab40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.541+0000 7f1ace21b640 1 --2- 172.21.15.107:0/2193718588 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f1ab40a2d00 0x7f1ab40a51c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8105d50 tx=0x7f1ab0006c90 comp rx=0 tx=0).stop 2024-04-25T00:27:28.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.541+0000 7f1ace21b640 1 -- 172.21.15.107:0/2193718588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ac8100610 msgr2=0x7f1ac8196ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.541+0000 7f1ace21b640 1 --2- 172.21.15.107:0/2193718588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ac8100610 0x7f1ac8196ef0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8005940 tx=0x7f1ab80042f0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.541+0000 7f1ace21b640 1 -- 172.21.15.107:0/2193718588 shutdown_connections 2024-04-25T00:27:28.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.541+0000 7f1ace21b640 1 --2- 172.21.15.107:0/2193718588 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f1ab40a2d00 0x7f1ab40a51c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.541+0000 7f1ace21b640 1 --2- 172.21.15.107:0/2193718588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ac81066b0 0x7f1ac819b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.541+0000 7f1ace21b640 1 --2- 172.21.15.107:0/2193718588 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1ac8100fd0 0x7f1ac8197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.541+0000 7f1ace21b640 1 --2- 172.21.15.107:0/2193718588 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ac8100610 0x7f1ac8196ef0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.541+0000 7f1ace21b640 1 -- 172.21.15.107:0/2193718588 >> 172.21.15.107:0/2193718588 conn(0x7f1ac8077740 msgr2=0x7f1ac80ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.541+0000 7f1ace21b640 1 -- 172.21.15.107:0/2193718588 shutdown_connections 2024-04-25T00:27:28.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.541+0000 7f1ace21b640 1 -- 172.21.15.107:0/2193718588 wait complete. 2024-04-25T00:27:28.553 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:27:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.653+0000 7f3a19cee640 1 -- 172.21.15.107:0/775961878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a14100610 msgr2=0x7f3a14100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.653+0000 7f3a19cee640 1 --2- 172.21.15.107:0/775961878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a14100610 0x7f3a14100a90 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f39fc009930 tx=0x7f39fc02a420 comp rx=0 tx=0).stop 2024-04-25T00:27:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.653+0000 7f3a19cee640 1 -- 172.21.15.107:0/775961878 shutdown_connections 2024-04-25T00:27:28.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.653+0000 7f3a19cee640 1 --2- 172.21.15.107:0/775961878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a14100fd0 0x7f3a1410d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.653+0000 7f3a19cee640 1 --2- 172.21.15.107:0/775961878 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a14100610 0x7f3a14100a90 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.653+0000 7f3a19cee640 1 --2- 172.21.15.107:0/775961878 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a141066b0 0x7f3a14106a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.653+0000 7f3a19cee640 1 -- 172.21.15.107:0/775961878 >> 172.21.15.107:0/775961878 conn(0x7f3a14077740 msgr2=0x7f3a14077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.653+0000 7f3a19cee640 1 -- 172.21.15.107:0/775961878 shutdown_connections 2024-04-25T00:27:28.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.653+0000 7f3a19cee640 1 -- 172.21.15.107:0/775961878 wait complete. 2024-04-25T00:27:28.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.653+0000 7f3a19cee640 1 Processor -- start 2024-04-25T00:27:28.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.654+0000 7f3a19cee640 1 -- start start 2024-04-25T00:27:28.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.654+0000 7f3a19cee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a14100610 0x7f3a14198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.654+0000 7f3a19cee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a14100fd0 0x7f3a14199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.654+0000 7f3a19cee640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a141066b0 0x7f3a141a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.654+0000 7f3a19cee640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3a141109f0 con 0x7f3a14100fd0 2024-04-25T00:27:28.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.654+0000 7f3a19cee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3a14110870 con 0x7f3a14100610 2024-04-25T00:27:28.658 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.654+0000 7f3a19cee640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f3a14110b70 con 0x7f3a141066b0 2024-04-25T00:27:28.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.654+0000 7f3a0bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a14100fd0 0x7f3a14199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.654+0000 7f3a0bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a14100fd0 0x7f3a14199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58802/0 (socket says 172.21.15.107:58802) 2024-04-25T00:27:28.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.654+0000 7f3a0bfff640 1 -- 172.21.15.107:0/2751355718 learned_addr learned my addr 172.21.15.107:0/2751355718 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:27:28.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.654+0000 7f3a137fe640 1 --2- 172.21.15.107:0/2751355718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a14100610 0x7f3a14198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.654+0000 7f3a13fff640 1 --2- 172.21.15.107:0/2751355718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a141066b0 0x7f3a141a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.654+0000 7f3a0bfff640 1 -- 172.21.15.107:0/2751355718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a141066b0 msgr2=0x7f3a141a3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.654+0000 7f3a0bfff640 1 --2- 172.21.15.107:0/2751355718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a141066b0 0x7f3a141a3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.654+0000 7f3a0bfff640 1 -- 172.21.15.107:0/2751355718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a14100610 msgr2=0x7f3a14198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.654+0000 7f3a0bfff640 1 --2- 172.21.15.107:0/2751355718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a14100610 0x7f3a14198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.654+0000 7f3a0bfff640 1 -- 172.21.15.107:0/2751355718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a141a4410 con 0x7f3a14100fd0 2024-04-25T00:27:28.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.654+0000 7f3a0bfff640 1 --2- 172.21.15.107:0/2751355718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a14100fd0 0x7f3a14199390 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f39fc009230 tx=0x7f39fc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.654+0000 7f3a117fa640 1 -- 172.21.15.107:0/2751355718 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc03f030 con 0x7f3a14100fd0 2024-04-25T00:27:28.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.654+0000 7f3a117fa640 1 -- 172.21.15.107:0/2751355718 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39fc003ad0 con 0x7f3a14100fd0 2024-04-25T00:27:28.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.654+0000 7f3a19cee640 1 -- 172.21.15.107:0/2751355718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a141a45e0 con 0x7f3a14100fd0 2024-04-25T00:27:28.659 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.655+0000 7f3a117fa640 1 -- 172.21.15.107:0/2751355718 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc035070 con 0x7f3a14100fd0 2024-04-25T00:27:28.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.655+0000 7f3a19cee640 1 -- 172.21.15.107:0/2751355718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a141a4aa0 con 0x7f3a14100fd0 2024-04-25T00:27:28.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.656+0000 7f3a19cee640 1 -- 172.21.15.107:0/2751355718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a14068560 con 0x7f3a14100fd0 2024-04-25T00:27:28.660 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.657+0000 7f3a117fa640 1 -- 172.21.15.107:0/2751355718 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f39fc042030 con 0x7f3a14100fd0 2024-04-25T00:27:28.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.658+0000 7f3a117fa640 1 --2- 172.21.15.107:0/2751355718 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f3a000a2d80 0x7f3a000a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.658+0000 7f3a137fe640 1 --2- 172.21.15.107:0/2751355718 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f3a000a2d80 0x7f3a000a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.658+0000 7f3a117fa640 1 -- 172.21.15.107:0/2751355718 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f39fc0f00a0 con 0x7f3a14100fd0 2024-04-25T00:27:28.661 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.658+0000 7f3a137fe640 1 --2- 172.21.15.107:0/2751355718 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f3a000a2d80 0x7f3a000a5240 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3a04005fd0 tx=0x7f3a0400b390 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.662 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.661+0000 7f3a117fa640 1 -- 172.21.15.107:0/2751355718 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f39fc0bc900 con 0x7f3a14100fd0 2024-04-25T00:27:28.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.837+0000 7f3a19cee640 1 -- 172.21.15.107:0/2751355718 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3a14108b80 con 0x7f3a14100fd0 2024-04-25T00:27:28.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.837+0000 7f3a117fa640 1 -- 172.21.15.107:0/2751355718 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) ==== 100+0+11 (secure 0 0 0) 0x7f39fc0bc200 con 0x7f3a14100fd0 2024-04-25T00:27:28.839 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-04-25T00:27:28.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.839+0000 7f3a19cee640 1 -- 172.21.15.107:0/2751355718 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f3a000a2d80 msgr2=0x7f3a000a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.839+0000 7f3a19cee640 1 --2- 172.21.15.107:0/2751355718 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f3a000a2d80 0x7f3a000a5240 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3a04005fd0 tx=0x7f3a0400b390 comp rx=0 tx=0).stop 2024-04-25T00:27:28.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.839+0000 7f3a19cee640 1 -- 172.21.15.107:0/2751355718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a14100fd0 msgr2=0x7f3a14199390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.839+0000 7f3a19cee640 1 --2- 172.21.15.107:0/2751355718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a14100fd0 0x7f3a14199390 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f39fc009230 tx=0x7f39fc004270 comp rx=0 tx=0).stop 2024-04-25T00:27:28.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.839+0000 7f3a19cee640 1 -- 172.21.15.107:0/2751355718 shutdown_connections 2024-04-25T00:27:28.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.839+0000 7f3a19cee640 1 --2- 172.21.15.107:0/2751355718 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a141066b0 0x7f3a141a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.839+0000 7f3a19cee640 1 --2- 172.21.15.107:0/2751355718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a14100fd0 0x7f3a14199390 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.839+0000 7f3a19cee640 1 --2- 172.21.15.107:0/2751355718 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f3a000a2d80 0x7f3a000a5240 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.839+0000 7f3a19cee640 1 --2- 172.21.15.107:0/2751355718 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a14100610 0x7f3a14198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.839+0000 7f3a19cee640 1 -- 172.21.15.107:0/2751355718 >> 172.21.15.107:0/2751355718 conn(0x7f3a14077740 msgr2=0x7f3a140ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.839+0000 7f3a19cee640 1 -- 172.21.15.107:0/2751355718 shutdown_connections 2024-04-25T00:27:28.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.840+0000 7f3a19cee640 1 -- 172.21.15.107:0/2751355718 wait complete. 2024-04-25T00:27:28.850 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:28 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2389207333' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:27:28.850 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:28 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/4142925069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:27:28.850 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:28 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2193718588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:27:28.851 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T00:27:28.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.950+0000 7ffa44734640 1 -- 172.21.15.107:0/4246889302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa3c0ffc40 msgr2=0x7ffa3c1000c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.950+0000 7ffa44734640 1 --2- 172.21.15.107:0/4246889302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa3c0ffc40 0x7ffa3c1000c0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009930 tx=0x7ffa2c02a420 comp rx=0 tx=0).stop 2024-04-25T00:27:28.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.950+0000 7ffa44734640 1 -- 172.21.15.107:0/4246889302 shutdown_connections 2024-04-25T00:27:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.950+0000 7ffa44734640 1 --2- 172.21.15.107:0/4246889302 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa3c100600 0x7ffa3c10e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.950+0000 7ffa44734640 1 --2- 172.21.15.107:0/4246889302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa3c0ffc40 0x7ffa3c1000c0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.950+0000 7ffa44734640 1 --2- 172.21.15.107:0/4246889302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa3c107720 0x7ffa3c107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.950+0000 7ffa44734640 1 -- 172.21.15.107:0/4246889302 >> 172.21.15.107:0/4246889302 conn(0x7ffa3c076bd0 msgr2=0x7ffa3c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.951+0000 7ffa44734640 1 -- 172.21.15.107:0/4246889302 shutdown_connections 2024-04-25T00:27:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.951+0000 7ffa44734640 1 -- 172.21.15.107:0/4246889302 wait complete. 2024-04-25T00:27:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.951+0000 7ffa44734640 1 Processor -- start 2024-04-25T00:27:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.951+0000 7ffa44734640 1 -- start start 2024-04-25T00:27:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.951+0000 7ffa44734640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa3c0ffc40 0x7ffa3c198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.951+0000 7ffa44734640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa3c100600 0x7ffa3c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.951+0000 7ffa44734640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa3c107720 0x7ffa3c19c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.951+0000 7ffa44734640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c111b30 con 0x7ffa3c0ffc40 2024-04-25T00:27:28.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.951+0000 7ffa44734640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c1119b0 con 0x7ffa3c107720 2024-04-25T00:27:28.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.951+0000 7ffa44734640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c111cb0 con 0x7ffa3c100600 2024-04-25T00:27:28.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.951+0000 7ffa424d0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa3c0ffc40 0x7ffa3c198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.951+0000 7ffa424d0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa3c0ffc40 0x7ffa3c198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58818/0 (socket says 172.21.15.107:58818) 2024-04-25T00:27:28.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.951+0000 7ffa424d0640 1 -- 172.21.15.107:0/2308786550 learned_addr learned my addr 172.21.15.107:0/2308786550 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:27:28.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.952+0000 7ffa424d0640 1 -- 172.21.15.107:0/2308786550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa3c100600 msgr2=0x7ffa3c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.952+0000 7ffa424d0640 1 --2- 172.21.15.107:0/2308786550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa3c100600 0x7ffa3c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.952+0000 7ffa42cd1640 1 --2- 172.21.15.107:0/2308786550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa3c107720 0x7ffa3c19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.952+0000 7ffa424d0640 1 -- 172.21.15.107:0/2308786550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa3c107720 msgr2=0x7ffa3c19c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:28.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.952+0000 7ffa424d0640 1 --2- 172.21.15.107:0/2308786550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa3c107720 0x7ffa3c19c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:28.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.952+0000 7ffa424d0640 1 -- 172.21.15.107:0/2308786550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3c19d040 con 0x7ffa3c0ffc40 2024-04-25T00:27:28.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.952+0000 7ffa42cd1640 1 --2- 172.21.15.107:0/2308786550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa3c107720 0x7ffa3c19c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:27:28.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.952+0000 7ffa424d0640 1 --2- 172.21.15.107:0/2308786550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa3c0ffc40 0x7ffa3c198050 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800d950 tx=0x7ffa3800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.952+0000 7ffa337fe640 1 -- 172.21.15.107:0/2308786550 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38017070 con 0x7ffa3c0ffc40 2024-04-25T00:27:28.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.952+0000 7ffa44734640 1 -- 172.21.15.107:0/2308786550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa3c19d330 con 0x7ffa3c0ffc40 2024-04-25T00:27:28.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.952+0000 7ffa337fe640 1 -- 172.21.15.107:0/2308786550 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa38009c30 con 0x7ffa3c0ffc40 2024-04-25T00:27:28.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.952+0000 7ffa337fe640 1 -- 172.21.15.107:0/2308786550 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3800f6b0 con 0x7ffa3c0ffc40 2024-04-25T00:27:28.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.952+0000 7ffa44734640 1 -- 172.21.15.107:0/2308786550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa3c1a4bc0 con 0x7ffa3c0ffc40 2024-04-25T00:27:28.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.953+0000 7ffa44734640 1 -- 172.21.15.107:0/2308786550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa3c0689b0 con 0x7ffa3c0ffc40 2024-04-25T00:27:28.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.955+0000 7ffa337fe640 1 -- 172.21.15.107:0/2308786550 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7ffa380109d0 con 0x7ffa3c0ffc40 2024-04-25T00:27:28.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.955+0000 7ffa337fe640 1 --2- 172.21.15.107:0/2308786550 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ffa280a2d00 0x7ffa280a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:28.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.955+0000 7ffa41ccf640 1 --2- 172.21.15.107:0/2308786550 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ffa280a2d00 0x7ffa280a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:28.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.955+0000 7ffa337fe640 1 -- 172.21.15.107:0/2308786550 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7ffa380cf470 con 0x7ffa3c0ffc40 2024-04-25T00:27:28.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.956+0000 7ffa41ccf640 1 --2- 172.21.15.107:0/2308786550 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ffa280a2d00 0x7ffa280a51c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c02a940 tx=0x7ffa2c000bc0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:28.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:28.959+0000 7ffa337fe640 1 -- 172.21.15.107:0/2308786550 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffa38009de0 con 0x7ffa3c0ffc40 2024-04-25T00:27:28.984 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:28 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2389207333' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:27:28.984 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:28 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/4142925069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:27:28.984 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:28 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2193718588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:27:29.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:28 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2389207333' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:27:29.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:28 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/4142925069' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:27:29.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:28 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2193718588' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:27:29.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.134+0000 7ffa44734640 1 -- 172.21.15.107:0/2308786550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ffa3c109ce0 con 0x7ffa3c0ffc40 2024-04-25T00:27:29.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.134+0000 7ffa337fe640 1 -- 172.21.15.107:0/2308786550 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v107) ==== 103+0+11 (secure 0 0 0) 0x7ffa3809bcd0 con 0x7ffa3c0ffc40 2024-04-25T00:27:29.136 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-04-25T00:27:29.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.136+0000 7ffa44734640 1 -- 172.21.15.107:0/2308786550 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ffa280a2d00 msgr2=0x7ffa280a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.136+0000 7ffa44734640 1 --2- 172.21.15.107:0/2308786550 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ffa280a2d00 0x7ffa280a51c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c02a940 tx=0x7ffa2c000bc0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.136+0000 7ffa44734640 1 -- 172.21.15.107:0/2308786550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa3c0ffc40 msgr2=0x7ffa3c198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.136+0000 7ffa44734640 1 --2- 172.21.15.107:0/2308786550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa3c0ffc40 0x7ffa3c198050 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800d950 tx=0x7ffa3800de10 comp rx=0 tx=0).stop 2024-04-25T00:27:29.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.136+0000 7ffa44734640 1 -- 172.21.15.107:0/2308786550 shutdown_connections 2024-04-25T00:27:29.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.136+0000 7ffa44734640 1 --2- 172.21.15.107:0/2308786550 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7ffa280a2d00 0x7ffa280a51c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.136+0000 7ffa44734640 1 --2- 172.21.15.107:0/2308786550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa3c107720 0x7ffa3c19c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.136+0000 7ffa44734640 1 --2- 172.21.15.107:0/2308786550 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa3c100600 0x7ffa3c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.136+0000 7ffa44734640 1 --2- 172.21.15.107:0/2308786550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa3c0ffc40 0x7ffa3c198050 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.136+0000 7ffa44734640 1 -- 172.21.15.107:0/2308786550 >> 172.21.15.107:0/2308786550 conn(0x7ffa3c076bd0 msgr2=0x7ffa3c0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:29.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.136+0000 7ffa44734640 1 -- 172.21.15.107:0/2308786550 shutdown_connections 2024-04-25T00:27:29.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.136+0000 7ffa44734640 1 -- 172.21.15.107:0/2308786550 wait complete. 2024-04-25T00:27:29.148 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:27:29.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.244+0000 7f9648a29640 1 -- 172.21.15.107:0/1503165753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9644106630 msgr2=0x7f9644106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.244+0000 7f9648a29640 1 --2- 172.21.15.107:0/1503165753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9644106630 0x7f9644106a10 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f9634009930 tx=0x7f963402a420 comp rx=0 tx=0).stop 2024-04-25T00:27:29.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.245+0000 7f9648a29640 1 -- 172.21.15.107:0/1503165753 shutdown_connections 2024-04-25T00:27:29.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.245+0000 7f9648a29640 1 --2- 172.21.15.107:0/1503165753 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9644100fe0 0x7f964410d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.245+0000 7f9648a29640 1 --2- 172.21.15.107:0/1503165753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9644100620 0x7f9644100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.245+0000 7f9648a29640 1 --2- 172.21.15.107:0/1503165753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9644106630 0x7f9644106a10 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.245+0000 7f9648a29640 1 -- 172.21.15.107:0/1503165753 >> 172.21.15.107:0/1503165753 conn(0x7f96440776a0 msgr2=0x7f9644077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:29.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.245+0000 7f9648a29640 1 -- 172.21.15.107:0/1503165753 shutdown_connections 2024-04-25T00:27:29.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.245+0000 7f9648a29640 1 -- 172.21.15.107:0/1503165753 wait complete. 2024-04-25T00:27:29.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.245+0000 7f9648a29640 1 Processor -- start 2024-04-25T00:27:29.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.245+0000 7f9648a29640 1 -- start start 2024-04-25T00:27:29.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.246+0000 7f9648a29640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9644100620 0x7f9644196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.246+0000 7f9648a29640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9644100fe0 0x7f9644197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.246+0000 7f9648a29640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9644106630 0x7f9644197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.246+0000 7f9648a29640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9644110a40 con 0x7f9644100fe0 2024-04-25T00:27:29.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.246+0000 7f9648a29640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f96441108c0 con 0x7f9644100620 2024-04-25T00:27:29.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.246+0000 7f9648a29640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f9644110bc0 con 0x7f9644106630 2024-04-25T00:27:29.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.246+0000 7f9641d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9644100fe0 0x7f9644197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.246+0000 7f9641d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9644100fe0 0x7f9644197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58828/0 (socket says 172.21.15.107:58828) 2024-04-25T00:27:29.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.246+0000 7f9641d9b640 1 -- 172.21.15.107:0/1144305231 learned_addr learned my addr 172.21.15.107:0/1144305231 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:27:29.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.246+0000 7f9642d9d640 1 --2- 172.21.15.107:0/1144305231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9644106630 0x7f9644197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.246+0000 7f964259c640 1 --2- 172.21.15.107:0/1144305231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9644100620 0x7f9644196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.246+0000 7f9641d9b640 1 -- 172.21.15.107:0/1144305231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9644106630 msgr2=0x7f9644197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.246+0000 7f9641d9b640 1 --2- 172.21.15.107:0/1144305231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9644106630 0x7f9644197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.246+0000 7f9641d9b640 1 -- 172.21.15.107:0/1144305231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9644100620 msgr2=0x7f9644196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.246+0000 7f9641d9b640 1 --2- 172.21.15.107:0/1144305231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9644100620 0x7f9644196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.246+0000 7f9641d9b640 1 -- 172.21.15.107:0/1144305231 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96441a6440 con 0x7f9644100fe0 2024-04-25T00:27:29.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.246+0000 7f964259c640 1 --2- 172.21.15.107:0/1144305231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9644100620 0x7f9644196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:27:29.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.246+0000 7f9641d9b640 1 --2- 172.21.15.107:0/1144305231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9644100fe0 0x7f9644197110 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f963800c920 tx=0x7f963800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:29.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.246+0000 7f96337fe640 1 -- 172.21.15.107:0/1144305231 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9638007c70 con 0x7f9644100fe0 2024-04-25T00:27:29.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.246+0000 7f96337fe640 1 -- 172.21.15.107:0/1144305231 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9638007e10 con 0x7f9644100fe0 2024-04-25T00:27:29.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.246+0000 7f9648a29640 1 -- 172.21.15.107:0/1144305231 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96441a6730 con 0x7f9644100fe0 2024-04-25T00:27:29.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.246+0000 7f96337fe640 1 -- 172.21.15.107:0/1144305231 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9638018890 con 0x7f9644100fe0 2024-04-25T00:27:29.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.246+0000 7f9648a29640 1 -- 172.21.15.107:0/1144305231 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96441a6c80 con 0x7f9644100fe0 2024-04-25T00:27:29.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.247+0000 7f9648a29640 1 -- 172.21.15.107:0/1144305231 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96440689b0 con 0x7f9644100fe0 2024-04-25T00:27:29.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.248+0000 7f96337fe640 1 -- 172.21.15.107:0/1144305231 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f9638018a50 con 0x7f9644100fe0 2024-04-25T00:27:29.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.249+0000 7f96337fe640 1 --2- 172.21.15.107:0/1144305231 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f962c0a2d00 0x7f962c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.249+0000 7f964259c640 1 --2- 172.21.15.107:0/1144305231 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f962c0a2d00 0x7f962c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.249+0000 7f96337fe640 1 -- 172.21.15.107:0/1144305231 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f96380cfbf0 con 0x7f9644100fe0 2024-04-25T00:27:29.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.249+0000 7f964259c640 1 --2- 172.21.15.107:0/1144305231 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f962c0a2d00 0x7f962c0a51c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9634000c00 tx=0x7f9634032d50 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:29.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.253+0000 7f96337fe640 1 -- 172.21.15.107:0/1144305231 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9638014030 con 0x7f9644100fe0 2024-04-25T00:27:29.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.439+0000 7f9648a29640 1 -- 172.21.15.107:0/1144305231 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f964406a330 con 0x7f9644100fe0 2024-04-25T00:27:29.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.439+0000 7f96337fe640 1 -- 172.21.15.107:0/1144305231 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v107) ==== 81+0+954 (secure 0 0 0) 0x7f964406a330 con 0x7f9644100fe0 2024-04-25T00:27:29.441 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/151603643 2024-04-26T00:16:26.229442+0000 2024-04-25T00:27:29.441 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1884923543 2024-04-26T00:16:26.229442+0000 2024-04-25T00:27:29.441 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/111369358 2024-04-26T00:16:26.229442+0000 2024-04-25T00:27:29.441 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/1644942185 2024-04-26T00:16:26.229442+0000 2024-04-25T00:27:29.441 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1174178948 2024-04-26T00:07:00.838246+0000 2024-04-25T00:27:29.442 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/153169302 2024-04-26T00:07:00.838246+0000 2024-04-25T00:27:29.442 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/120616774 2024-04-26T00:06:34.124795+0000 2024-04-25T00:27:29.442 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/120616774 2024-04-26T00:06:34.124795+0000 2024-04-25T00:27:29.442 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/971124970 2024-04-26T00:06:34.124795+0000 2024-04-25T00:27:29.442 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3936375305 2024-04-26T00:16:26.229442+0000 2024-04-25T00:27:29.442 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3022844652 2024-04-26T00:06:34.124795+0000 2024-04-25T00:27:29.442 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/1644942185 2024-04-26T00:16:26.229442+0000 2024-04-25T00:27:29.442 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/153169302 2024-04-26T00:07:00.838246+0000 2024-04-25T00:27:29.442 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/4215203700 2024-04-26T00:07:00.838246+0000 2024-04-25T00:27:29.442 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/750288374 2024-04-26T00:06:34.124795+0000 2024-04-25T00:27:29.442 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3718680642 2024-04-26T00:07:00.838246+0000 2024-04-25T00:27:29.443 INFO:teuthology.orchestra.run.smithi107.stderr:listed 16 entries 2024-04-25T00:27:29.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.441+0000 7f9648a29640 1 -- 172.21.15.107:0/1144305231 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f962c0a2d00 msgr2=0x7f962c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.441+0000 7f9648a29640 1 --2- 172.21.15.107:0/1144305231 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f962c0a2d00 0x7f962c0a51c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9634000c00 tx=0x7f9634032d50 comp rx=0 tx=0).stop 2024-04-25T00:27:29.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.441+0000 7f9648a29640 1 -- 172.21.15.107:0/1144305231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9644100fe0 msgr2=0x7f9644197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.441+0000 7f9648a29640 1 --2- 172.21.15.107:0/1144305231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9644100fe0 0x7f9644197110 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f963800c920 tx=0x7f963800cde0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.442+0000 7f9648a29640 1 -- 172.21.15.107:0/1144305231 shutdown_connections 2024-04-25T00:27:29.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.442+0000 7f9648a29640 1 --2- 172.21.15.107:0/1144305231 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9644106630 0x7f9644197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.442+0000 7f9648a29640 1 --2- 172.21.15.107:0/1144305231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9644100fe0 0x7f9644197110 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.442+0000 7f9648a29640 1 --2- 172.21.15.107:0/1144305231 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f962c0a2d00 0x7f962c0a51c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.442+0000 7f9648a29640 1 --2- 172.21.15.107:0/1144305231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9644100620 0x7f9644196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.442+0000 7f9648a29640 1 -- 172.21.15.107:0/1144305231 >> 172.21.15.107:0/1144305231 conn(0x7f96440776a0 msgr2=0x7f9644103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:29.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.442+0000 7f9648a29640 1 -- 172.21.15.107:0/1144305231 shutdown_connections 2024-04-25T00:27:29.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.442+0000 7f9648a29640 1 -- 172.21.15.107:0/1144305231 wait complete. 2024-04-25T00:27:29.454 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:27:29.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:27:29.455 DEBUG:teuthology.orchestra.run.smithi107:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T00:27:29.496 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T00:27:29.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:27:29.496 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:27:29.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:27:29.561 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-04-25T00:27:29.619 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:27:29.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:27:29.619 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:27:29.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:27:29.683 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2024-04-25T00:27:29.787 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T00:27:29.798 INFO:tasks.cephadm:Teardown begin 2024-04-25T00:27:29.798 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:27:29.798 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:27:29.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.897+0000 7fe45d6b3640 1 -- 172.21.15.107:0/863429448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4581066b0 msgr2=0x7fe458106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.897+0000 7fe45d6b3640 1 --2- 172.21.15.107:0/863429448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4581066b0 0x7fe458106a90 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fe450009930 tx=0x7fe45002a420 comp rx=0 tx=0).stop 2024-04-25T00:27:29.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.897+0000 7fe45d6b3640 1 -- 172.21.15.107:0/863429448 shutdown_connections 2024-04-25T00:27:29.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.897+0000 7fe45d6b3640 1 --2- 172.21.15.107:0/863429448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe458100fd0 0x7fe45810d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.897+0000 7fe45d6b3640 1 --2- 172.21.15.107:0/863429448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe458100610 0x7fe458100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.897+0000 7fe45d6b3640 1 --2- 172.21.15.107:0/863429448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4581066b0 0x7fe458106a90 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.897+0000 7fe45d6b3640 1 -- 172.21.15.107:0/863429448 >> 172.21.15.107:0/863429448 conn(0x7fe4580769c0 msgr2=0x7fe458076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:29.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.897+0000 7fe45d6b3640 1 -- 172.21.15.107:0/863429448 shutdown_connections 2024-04-25T00:27:29.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.897+0000 7fe45d6b3640 1 -- 172.21.15.107:0/863429448 wait complete. 2024-04-25T00:27:29.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.898+0000 7fe45d6b3640 1 Processor -- start 2024-04-25T00:27:29.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.898+0000 7fe45d6b3640 1 -- start start 2024-04-25T00:27:29.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.898+0000 7fe45d6b3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe458100610 0x7fe458198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.898+0000 7fe45d6b3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe458100fd0 0x7fe4581985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.898+0000 7fe45d6b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4581066b0 0x7fe45819c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.898+0000 7fe45d6b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe458110a30 con 0x7fe4581066b0 2024-04-25T00:27:29.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.898+0000 7fe45d6b3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe4581108b0 con 0x7fe458100610 2024-04-25T00:27:29.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.898+0000 7fe45d6b3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fe458110bb0 con 0x7fe458100fd0 2024-04-25T00:27:29.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.898+0000 7fe4577fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4581066b0 0x7fe45819c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.898+0000 7fe4577fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4581066b0 0x7fe45819c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57910/0 (socket says 172.21.15.107:57910) 2024-04-25T00:27:29.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.898+0000 7fe4577fe640 1 -- 172.21.15.107:0/2108621845 learned_addr learned my addr 172.21.15.107:0/2108621845 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:27:29.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.898+0000 7fe456ffd640 1 --2- 172.21.15.107:0/2108621845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe458100610 0x7fe458198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.899+0000 7fe4567fc640 1 --2- 172.21.15.107:0/2108621845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe458100fd0 0x7fe4581985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.899+0000 7fe4577fe640 1 -- 172.21.15.107:0/2108621845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe458100fd0 msgr2=0x7fe4581985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.899+0000 7fe4577fe640 1 --2- 172.21.15.107:0/2108621845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe458100fd0 0x7fe4581985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.899+0000 7fe4577fe640 1 -- 172.21.15.107:0/2108621845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe458100610 msgr2=0x7fe458198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.899+0000 7fe4577fe640 1 --2- 172.21.15.107:0/2108621845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe458100610 0x7fe458198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.899+0000 7fe4577fe640 1 -- 172.21.15.107:0/2108621845 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe45819d070 con 0x7fe4581066b0 2024-04-25T00:27:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.899+0000 7fe4577fe640 1 --2- 172.21.15.107:0/2108621845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4581066b0 0x7fe45819c940 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fe44800c970 tx=0x7fe44800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.899+0000 7fe43ffff640 1 -- 172.21.15.107:0/2108621845 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe448007bb0 con 0x7fe4581066b0 2024-04-25T00:27:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.899+0000 7fe43ffff640 1 -- 172.21.15.107:0/2108621845 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe448007d50 con 0x7fe4581066b0 2024-04-25T00:27:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.899+0000 7fe45d6b3640 1 -- 172.21.15.107:0/2108621845 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe45819d360 con 0x7fe4581066b0 2024-04-25T00:27:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.899+0000 7fe43ffff640 1 -- 172.21.15.107:0/2108621845 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44800f450 con 0x7fe4581066b0 2024-04-25T00:27:29.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.899+0000 7fe45d6b3640 1 -- 172.21.15.107:0/2108621845 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4581a4bf0 con 0x7fe4581066b0 2024-04-25T00:27:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.900+0000 7fe45d6b3640 1 -- 172.21.15.107:0/2108621845 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe458101d50 con 0x7fe4581066b0 2024-04-25T00:27:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.901+0000 7fe43ffff640 1 -- 172.21.15.107:0/2108621845 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fe448004f00 con 0x7fe4581066b0 2024-04-25T00:27:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.902+0000 7fe43ffff640 1 --2- 172.21.15.107:0/2108621845 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe4400a2d00 0x7fe4400a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.902+0000 7fe456ffd640 1 --2- 172.21.15.107:0/2108621845 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe4400a2d00 0x7fe4400a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.902+0000 7fe43ffff640 1 -- 172.21.15.107:0/2108621845 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7fe4480cf7f0 con 0x7fe4581066b0 2024-04-25T00:27:29.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.902+0000 7fe456ffd640 1 --2- 172.21.15.107:0/2108621845 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe4400a2d00 0x7fe4400a51c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe450004770 tx=0x7fe4500058d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:29.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:29.905+0000 7fe43ffff640 1 -- 172.21.15.107:0/2108621845 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe448014030 con 0x7fe4581066b0 2024-04-25T00:27:29.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:29 smithi170 ceph-mon[32807]: pgmap v344: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2024-04-25T00:27:29.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:29 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2751355718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:27:29.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:29 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2308786550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:27:29.935 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:29 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1144305231' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:27:29.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:29 smithi107 ceph-mon[29153]: pgmap v344: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2024-04-25T00:27:29.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:29 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2751355718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:27:29.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:29 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2308786550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:27:29.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:29 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1144305231' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:27:30.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:29 smithi195 ceph-mon[32833]: pgmap v344: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2024-04-25T00:27:30.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:29 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2751355718' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:27:30.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:29 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2308786550' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-25T00:27:30.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:29 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1144305231' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:27:30.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.119+0000 7fe45d6b3640 1 -- 172.21.15.107:0/2108621845 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4581993d0 con 0x7fe4581066b0 2024-04-25T00:27:30.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.119+0000 7fe43ffff640 1 -- 172.21.15.107:0/2108621845 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11897 (secure 0 0 0) 0x7fe44809c050 con 0x7fe4581066b0 2024-04-25T00:27:30.121 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:27:30.121 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:22:27.402680+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24493,24521]},"id":1}]} 2024-04-25T00:27:30.121 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 34 2024-04-25T00:27:30.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.121+0000 7fe45d6b3640 1 -- 172.21.15.107:0/2108621845 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe4400a2d00 msgr2=0x7fe4400a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:30.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.121+0000 7fe45d6b3640 1 --2- 172.21.15.107:0/2108621845 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe4400a2d00 0x7fe4400a51c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe450004770 tx=0x7fe4500058d0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.121+0000 7fe45d6b3640 1 -- 172.21.15.107:0/2108621845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4581066b0 msgr2=0x7fe45819c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:30.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.121+0000 7fe45d6b3640 1 --2- 172.21.15.107:0/2108621845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4581066b0 0x7fe45819c940 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fe44800c970 tx=0x7fe44800ce30 comp rx=0 tx=0).stop 2024-04-25T00:27:30.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.121+0000 7fe45d6b3640 1 -- 172.21.15.107:0/2108621845 shutdown_connections 2024-04-25T00:27:30.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.121+0000 7fe45d6b3640 1 --2- 172.21.15.107:0/2108621845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4581066b0 0x7fe45819c940 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.121+0000 7fe45d6b3640 1 --2- 172.21.15.107:0/2108621845 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe458100fd0 0x7fe4581985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.121+0000 7fe45d6b3640 1 --2- 172.21.15.107:0/2108621845 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fe4400a2d00 0x7fe4400a51c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.121+0000 7fe45d6b3640 1 --2- 172.21.15.107:0/2108621845 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe458100610 0x7fe458198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.121+0000 7fe45d6b3640 1 -- 172.21.15.107:0/2108621845 >> 172.21.15.107:0/2108621845 conn(0x7fe4580769c0 msgr2=0x7fe4580ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:30.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.122+0000 7fe45d6b3640 1 -- 172.21.15.107:0/2108621845 shutdown_connections 2024-04-25T00:27:30.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.122+0000 7fe45d6b3640 1 -- 172.21.15.107:0/2108621845 wait complete. 2024-04-25T00:27:30.134 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:27:30.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.230+0000 7fefb0a72640 1 -- 172.21.15.107:0/1772897466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefac1066b0 msgr2=0x7fefac106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:30.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.230+0000 7fefb0a72640 1 --2- 172.21.15.107:0/1772897466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefac1066b0 0x7fefac106a90 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009930 tx=0x7fefa402a420 comp rx=0 tx=0).stop 2024-04-25T00:27:30.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.230+0000 7fefb0a72640 1 -- 172.21.15.107:0/1772897466 shutdown_connections 2024-04-25T00:27:30.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.230+0000 7fefb0a72640 1 --2- 172.21.15.107:0/1772897466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefac100fd0 0x7fefac10d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.230+0000 7fefb0a72640 1 --2- 172.21.15.107:0/1772897466 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefac100610 0x7fefac100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.230+0000 7fefb0a72640 1 --2- 172.21.15.107:0/1772897466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefac1066b0 0x7fefac106a90 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.230+0000 7fefb0a72640 1 -- 172.21.15.107:0/1772897466 >> 172.21.15.107:0/1772897466 conn(0x7fefac0769c0 msgr2=0x7fefac076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:30.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.230+0000 7fefb0a72640 1 -- 172.21.15.107:0/1772897466 shutdown_connections 2024-04-25T00:27:30.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.230+0000 7fefb0a72640 1 -- 172.21.15.107:0/1772897466 wait complete. 2024-04-25T00:27:30.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.231+0000 7fefb0a72640 1 Processor -- start 2024-04-25T00:27:30.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.231+0000 7fefb0a72640 1 -- start start 2024-04-25T00:27:30.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.231+0000 7fefb0a72640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefac100610 0x7fefac197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:30.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.231+0000 7fefb0a72640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefac100fd0 0x7fefac198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:30.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.231+0000 7fefb0a72640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefac1066b0 0x7fefac19c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:30.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.231+0000 7fefb0a72640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fefac110a30 con 0x7fefac100610 2024-04-25T00:27:30.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.231+0000 7fefb0a72640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fefac1108b0 con 0x7fefac1066b0 2024-04-25T00:27:30.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.231+0000 7fefb0a72640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fefac110bb0 con 0x7fefac100fd0 2024-04-25T00:27:30.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.231+0000 7fefaa59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefac100610 0x7fefac197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:30.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.231+0000 7fefaa59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefac100610 0x7fefac197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57932/0 (socket says 172.21.15.107:57932) 2024-04-25T00:27:30.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.231+0000 7fefaa59c640 1 -- 172.21.15.107:0/938908760 learned_addr learned my addr 172.21.15.107:0/938908760 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:27:30.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.231+0000 7fefa9d9b640 1 --2- 172.21.15.107:0/938908760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefac100fd0 0x7fefac198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:30.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.231+0000 7fefaad9d640 1 --2- 172.21.15.107:0/938908760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefac1066b0 0x7fefac19c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:30.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.231+0000 7fefaa59c640 1 -- 172.21.15.107:0/938908760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefac100fd0 msgr2=0x7fefac198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:30.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.231+0000 7fefaa59c640 1 --2- 172.21.15.107:0/938908760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefac100fd0 0x7fefac198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.231+0000 7fefaa59c640 1 -- 172.21.15.107:0/938908760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefac1066b0 msgr2=0x7fefac19c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:30.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.231+0000 7fefaa59c640 1 --2- 172.21.15.107:0/938908760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefac1066b0 0x7fefac19c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.231+0000 7fefaa59c640 1 -- 172.21.15.107:0/938908760 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefac19cfd0 con 0x7fefac100610 2024-04-25T00:27:30.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.231+0000 7fefaad9d640 1 --2- 172.21.15.107:0/938908760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefac1066b0 0x7fefac19c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:27:30.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.231+0000 7fefaa59c640 1 --2- 172.21.15.107:0/938908760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefac100610 0x7fefac197fe0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fefa4005940 tx=0x7fefa4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:30.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.232+0000 7fef9b7fe640 1 -- 172.21.15.107:0/938908760 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa403f030 con 0x7fefac100610 2024-04-25T00:27:30.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.232+0000 7fefb0a72640 1 -- 172.21.15.107:0/938908760 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefac19d260 con 0x7fefac100610 2024-04-25T00:27:30.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.232+0000 7fef9b7fe640 1 -- 172.21.15.107:0/938908760 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefa4003ad0 con 0x7fefac100610 2024-04-25T00:27:30.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.232+0000 7fef9b7fe640 1 -- 172.21.15.107:0/938908760 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4035070 con 0x7fefac100610 2024-04-25T00:27:30.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.232+0000 7fefb0a72640 1 -- 172.21.15.107:0/938908760 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefac1a4b50 con 0x7fefac100610 2024-04-25T00:27:30.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.233+0000 7fefb0a72640 1 -- 172.21.15.107:0/938908760 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefac0689b0 con 0x7fefac100610 2024-04-25T00:27:30.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.234+0000 7fef9b7fe640 1 -- 172.21.15.107:0/938908760 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fefa4042030 con 0x7fefac100610 2024-04-25T00:27:30.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.234+0000 7fef9b7fe640 1 --2- 172.21.15.107:0/938908760 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fef940a2d00 0x7fef940a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:30.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.234+0000 7fefa9d9b640 1 --2- 172.21.15.107:0/938908760 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fef940a2d00 0x7fef940a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:30.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.234+0000 7fef9b7fe640 1 -- 172.21.15.107:0/938908760 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7fefa40eff70 con 0x7fefac100610 2024-04-25T00:27:30.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.235+0000 7fefa9d9b640 1 --2- 172.21.15.107:0/938908760 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fef940a2d00 0x7fef940a51c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fefac06aec0 tx=0x7fefa0009500 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:30.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.239+0000 7fef9b7fe640 1 -- 172.21.15.107:0/938908760 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fefa40bc7d0 con 0x7fefac100610 2024-04-25T00:27:30.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.451+0000 7fefb0a72640 1 -- 172.21.15.107:0/938908760 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fefac1a4d20 con 0x7fefac100610 2024-04-25T00:27:30.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.452+0000 7fef9b7fe640 1 -- 172.21.15.107:0/938908760 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11897 (secure 0 0 0) 0x7fefa40bc0d0 con 0x7fefac100610 2024-04-25T00:27:30.452 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:27:30.452 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14544,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2628342623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2628342623},{"type":"v1","addr":"172.21.15.107:6835","nonce":2628342623}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14556,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2695944193","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2695944193},{"type":"v1","addr":"172.21.15.107:6837","nonce":2695944193}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14619,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/2028240372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":2028240372},{"type":"v1","addr":"172.21.15.107:6839","nonce":2028240372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14631,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/3282278035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":3282278035},{"type":"v1","addr":"172.21.15.107:6841","nonce":3282278035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24413,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/674283763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":674283763},{"type":"v1","addr":"172.21.15.170:6833","nonce":674283763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6833/2162039457","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":2162039457},{"type":"v1","addr":"172.21.15.195:6833","nonce":2162039457}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:14:01.788955+0000","modified":"2024-04-25T00:22:27.402680+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24521,"mds_1":24493,"mds_2":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24443":{"gid":24443,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.170:6835/2926641872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":2926641872},{"type":"v1","addr":"172.21.15.170:6835","nonce":2926641872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6835/2828335061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":2828335061},{"type":"v1","addr":"172.21.15.195:6835","nonce":2828335061}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.195:6837/2245799531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6836","nonce":2245799531},{"type":"v1","addr":"172.21.15.195:6837","nonce":2245799531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.170:6837/3616315471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":3616315471},{"type":"v1","addr":"172.21.15.170:6837","nonce":3616315471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.195:6839/4133864926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6838","nonce":4133864926},{"type":"v1","addr":"172.21.15.195:6839","nonce":4133864926}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.170:6839/3558784302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3558784302},{"type":"v1","addr":"172.21.15.170:6839","nonce":3558784302}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24521,"qdb_cluster":[24485,24493,24521]},"id":1}]} 2024-04-25T00:27:30.453 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 34 2024-04-25T00:27:30.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.454+0000 7fefb0a72640 1 -- 172.21.15.107:0/938908760 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fef940a2d00 msgr2=0x7fef940a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:30.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.454+0000 7fefb0a72640 1 --2- 172.21.15.107:0/938908760 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fef940a2d00 0x7fef940a51c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fefac06aec0 tx=0x7fefa0009500 comp rx=0 tx=0).stop 2024-04-25T00:27:30.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.454+0000 7fefb0a72640 1 -- 172.21.15.107:0/938908760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefac100610 msgr2=0x7fefac197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:30.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.454+0000 7fefb0a72640 1 --2- 172.21.15.107:0/938908760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefac100610 0x7fefac197fe0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fefa4005940 tx=0x7fefa4004270 comp rx=0 tx=0).stop 2024-04-25T00:27:30.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.454+0000 7fefb0a72640 1 -- 172.21.15.107:0/938908760 shutdown_connections 2024-04-25T00:27:30.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.454+0000 7fefb0a72640 1 --2- 172.21.15.107:0/938908760 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fef940a2d00 0x7fef940a51c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.454+0000 7fefb0a72640 1 --2- 172.21.15.107:0/938908760 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fefac1066b0 0x7fefac19c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.454+0000 7fefb0a72640 1 --2- 172.21.15.107:0/938908760 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fefac100fd0 0x7fefac198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.454+0000 7fefb0a72640 1 --2- 172.21.15.107:0/938908760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefac100610 0x7fefac197fe0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.454+0000 7fefb0a72640 1 -- 172.21.15.107:0/938908760 >> 172.21.15.107:0/938908760 conn(0x7fefac0769c0 msgr2=0x7fefac104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:30.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.454+0000 7fefb0a72640 1 -- 172.21.15.107:0/938908760 shutdown_connections 2024-04-25T00:27:30.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.454+0000 7fefb0a72640 1 -- 172.21.15.107:0/938908760 wait complete. 2024-04-25T00:27:30.465 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:27:30.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.562+0000 7fb8000a5640 1 -- 172.21.15.107:0/4087009802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f8100fe0 msgr2=0x7fb7f810d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:30.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.562+0000 7fb8000a5640 1 --2- 172.21.15.107:0/4087009802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f8100fe0 0x7fb7f810d4f0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0009cd0 tx=0x7fb7f002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.562+0000 7fb8000a5640 1 -- 172.21.15.107:0/4087009802 shutdown_connections 2024-04-25T00:27:30.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.562+0000 7fb8000a5640 1 --2- 172.21.15.107:0/4087009802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f8100fe0 0x7fb7f810d4f0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.562+0000 7fb8000a5640 1 --2- 172.21.15.107:0/4087009802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7f8100620 0x7fb7f8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.562+0000 7fb8000a5640 1 --2- 172.21.15.107:0/4087009802 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7f8106630 0x7fb7f8106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.562+0000 7fb8000a5640 1 -- 172.21.15.107:0/4087009802 >> 172.21.15.107:0/4087009802 conn(0x7fb7f8077740 msgr2=0x7fb7f8077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:30.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.562+0000 7fb8000a5640 1 -- 172.21.15.107:0/4087009802 shutdown_connections 2024-04-25T00:27:30.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.562+0000 7fb8000a5640 1 -- 172.21.15.107:0/4087009802 wait complete. 2024-04-25T00:27:30.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.562+0000 7fb8000a5640 1 Processor -- start 2024-04-25T00:27:30.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.562+0000 7fb8000a5640 1 -- start start 2024-04-25T00:27:30.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.562+0000 7fb8000a5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7f8100620 0x7fb7f8198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:30.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.562+0000 7fb8000a5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7f8100fe0 0x7fb7f8198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:30.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.562+0000 7fb8000a5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f8106630 0x7fb7f819c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:30.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.562+0000 7fb8000a5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f8111b40 con 0x7fb7f8106630 2024-04-25T00:27:30.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.562+0000 7fb8000a5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f81119c0 con 0x7fb7f8100620 2024-04-25T00:27:30.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.562+0000 7fb8000a5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f8111cc0 con 0x7fb7f8100fe0 2024-04-25T00:27:30.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.563+0000 7fb7fe642640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f8106630 0x7fb7f819c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:30.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.563+0000 7fb7fe642640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f8106630 0x7fb7f819c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57940/0 (socket says 172.21.15.107:57940) 2024-04-25T00:27:30.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.563+0000 7fb7fe642640 1 -- 172.21.15.107:0/2501354166 learned_addr learned my addr 172.21.15.107:0/2501354166 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:27:30.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.563+0000 7fb7fde41640 1 --2- 172.21.15.107:0/2501354166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7f8100620 0x7fb7f8198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:30.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.563+0000 7fb7fd640640 1 --2- 172.21.15.107:0/2501354166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7f8100fe0 0x7fb7f8198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:30.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.563+0000 7fb7fe642640 1 -- 172.21.15.107:0/2501354166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7f8100fe0 msgr2=0x7fb7f8198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:30.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.563+0000 7fb7fe642640 1 --2- 172.21.15.107:0/2501354166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7f8100fe0 0x7fb7f8198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.563+0000 7fb7fe642640 1 -- 172.21.15.107:0/2501354166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7f8100620 msgr2=0x7fb7f8198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:30.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.563+0000 7fb7fe642640 1 --2- 172.21.15.107:0/2501354166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7f8100620 0x7fb7f8198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.563+0000 7fb7fe642640 1 -- 172.21.15.107:0/2501354166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f819cfc0 con 0x7fb7f8106630 2024-04-25T00:27:30.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.563+0000 7fb7fe642640 1 --2- 172.21.15.107:0/2501354166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f8106630 0x7fb7f819c890 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0005cc0 tx=0x7fb7f0032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:30.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.563+0000 7fb7eeffd640 1 -- 172.21.15.107:0/2501354166 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0004040 con 0x7fb7f8106630 2024-04-25T00:27:30.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.563+0000 7fb7eeffd640 1 -- 172.21.15.107:0/2501354166 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7f00046c0 con 0x7fb7f8106630 2024-04-25T00:27:30.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.563+0000 7fb8000a5640 1 -- 172.21.15.107:0/2501354166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f819d250 con 0x7fb7f8106630 2024-04-25T00:27:30.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.563+0000 7fb7eeffd640 1 -- 172.21.15.107:0/2501354166 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0030e00 con 0x7fb7f8106630 2024-04-25T00:27:30.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.563+0000 7fb8000a5640 1 -- 172.21.15.107:0/2501354166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f81a4b40 con 0x7fb7f8106630 2024-04-25T00:27:30.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.564+0000 7fb8000a5640 1 -- 172.21.15.107:0/2501354166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f8068560 con 0x7fb7f8106630 2024-04-25T00:27:30.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.565+0000 7fb7eeffd640 1 -- 172.21.15.107:0/2501354166 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb7f00305a0 con 0x7fb7f8106630 2024-04-25T00:27:30.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.565+0000 7fb7eeffd640 1 --2- 172.21.15.107:0/2501354166 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb7e40a2d00 0x7fb7e40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:30.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.566+0000 7fb7fde41640 1 --2- 172.21.15.107:0/2501354166 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb7e40a2d00 0x7fb7e40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:30.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.566+0000 7fb7eeffd640 1 -- 172.21.15.107:0/2501354166 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7fb7f002e030 con 0x7fb7f8106630 2024-04-25T00:27:30.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.566+0000 7fb7fde41640 1 --2- 172.21.15.107:0/2501354166 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb7e40a2d00 0x7fb7e40a51c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800bb60 tx=0x7fb7e800b390 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:30.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.570+0000 7fb7eeffd640 1 -- 172.21.15.107:0/2501354166 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb7f004b060 con 0x7fb7f8106630 2024-04-25T00:27:30.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:30 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2108621845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:30.734 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:30 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/938908760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:30.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.753+0000 7fb8000a5640 1 -- 172.21.15.107:0/2501354166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb7f8108c10 con 0x7fb7f8106630 2024-04-25T00:27:30.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.755+0000 7fb7eeffd640 1 -- 172.21.15.107:0/2501354166 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+25256 (secure 0 0 0) 0x7fb7f00bd7e0 con 0x7fb7f8106630 2024-04-25T00:27:30.755 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T00:27:30.755 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":107,"fsid":"79602f58-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:05:56.053373+0000","modified":"2024-04-25T00:27:16.683152+0000","last_up_change":"2024-04-25T00:13:10.505047+0000","last_in_change":"2024-04-25T00:12:50.765717+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:09:25.160329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:13:59.672801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:14:00.957782+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T00:14:05.065846+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5a74e9b9-fc11-4472-913b-ea0084ac1492","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6803","nonce":331467636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6805","nonce":331467636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6809","nonce":331467636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":331467636},{"type":"v1","addr":"172.21.15.107:6807","nonce":331467636}]},"public_addr":"172.21.15.107:6803/331467636","cluster_addr":"172.21.15.107:6805/331467636","heartbeat_back_addr":"172.21.15.107:6809/331467636","heartbeat_front_addr":"172.21.15.107:6807/331467636","state":["exists","up"]},{"osd":1,"uuid":"d009a8aa-9bf8-44c8-b873-ca2ab7c0587c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6801","nonce":453760841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6803","nonce":453760841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6807","nonce":453760841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":453760841},{"type":"v1","addr":"172.21.15.170:6805","nonce":453760841}]},"public_addr":"172.21.15.170:6801/453760841","cluster_addr":"172.21.15.170:6803/453760841","heartbeat_back_addr":"172.21.15.170:6807/453760841","heartbeat_front_addr":"172.21.15.170:6805/453760841","state":["exists","up"]},{"osd":2,"uuid":"4df1456c-7657-4efd-8295-bf8248fcbcaf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6801","nonce":90978480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6803","nonce":90978480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6807","nonce":90978480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":90978480},{"type":"v1","addr":"172.21.15.195:6805","nonce":90978480}]},"public_addr":"172.21.15.195:6801/90978480","cluster_addr":"172.21.15.195:6803/90978480","heartbeat_back_addr":"172.21.15.195:6807/90978480","heartbeat_front_addr":"172.21.15.195:6805/90978480","state":["exists","up"]},{"osd":3,"uuid":"2c75ecce-0fee-4e65-8442-71125e8e5ad3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6811","nonce":2902386444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6813","nonce":2902386444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6817","nonce":2902386444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":2902386444},{"type":"v1","addr":"172.21.15.107:6815","nonce":2902386444}]},"public_addr":"172.21.15.107:6811/2902386444","cluster_addr":"172.21.15.107:6813/2902386444","heartbeat_back_addr":"172.21.15.107:6817/2902386444","heartbeat_front_addr":"172.21.15.107:6815/2902386444","state":["exists","up"]},{"osd":4,"uuid":"6f61a3aa-62cc-43aa-b4b0-8931c9837b57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6809","nonce":3780266808}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6811","nonce":3780266808}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6815","nonce":3780266808}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":3780266808},{"type":"v1","addr":"172.21.15.170:6813","nonce":3780266808}]},"public_addr":"172.21.15.170:6809/3780266808","cluster_addr":"172.21.15.170:6811/3780266808","heartbeat_back_addr":"172.21.15.170:6815/3780266808","heartbeat_front_addr":"172.21.15.170:6813/3780266808","state":["exists","up"]},{"osd":5,"uuid":"104da40f-019b-4dc3-8c62-090920ebe24f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6809","nonce":2322632437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6811","nonce":2322632437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6815","nonce":2322632437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":2322632437},{"type":"v1","addr":"172.21.15.195:6813","nonce":2322632437}]},"public_addr":"172.21.15.195:6809/2322632437","cluster_addr":"172.21.15.195:6811/2322632437","heartbeat_back_addr":"172.21.15.195:6815/2322632437","heartbeat_front_addr":"172.21.15.195:6813/2322632437","state":["exists","up"]},{"osd":6,"uuid":"543e0a0a-23fc-402f-8a71-3bcd276999db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6819","nonce":251620415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6821","nonce":251620415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6825","nonce":251620415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":251620415},{"type":"v1","addr":"172.21.15.107:6823","nonce":251620415}]},"public_addr":"172.21.15.107:6819/251620415","cluster_addr":"172.21.15.107:6821/251620415","heartbeat_back_addr":"172.21.15.107:6825/251620415","heartbeat_front_addr":"172.21.15.107:6823/251620415","state":["exists","up"]},{"osd":7,"uuid":"aa97be8a-73cd-4452-b143-825370290232","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6817","nonce":3231152590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6819","nonce":3231152590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6823","nonce":3231152590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":3231152590},{"type":"v1","addr":"172.21.15.170:6821","nonce":3231152590}]},"public_addr":"172.21.15.170:6817/3231152590","cluster_addr":"172.21.15.170:6819/3231152590","heartbeat_back_addr":"172.21.15.170:6823/3231152590","heartbeat_front_addr":"172.21.15.170:6821/3231152590","state":["exists","up"]},{"osd":8,"uuid":"ebddecde-4dac-4b5e-af58-f01f285aa7c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6817","nonce":2391866428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6819","nonce":2391866428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6823","nonce":2391866428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6820","nonce":2391866428},{"type":"v1","addr":"172.21.15.195:6821","nonce":2391866428}]},"public_addr":"172.21.15.195:6817/2391866428","cluster_addr":"172.21.15.195:6819/2391866428","heartbeat_back_addr":"172.21.15.195:6823/2391866428","heartbeat_front_addr":"172.21.15.195:6821/2391866428","state":["exists","up"]},{"osd":9,"uuid":"f85eab7a-7430-4520-9b2d-932398199b4c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6827","nonce":174023257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6829","nonce":174023257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6833","nonce":174023257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":174023257},{"type":"v1","addr":"172.21.15.107:6831","nonce":174023257}]},"public_addr":"172.21.15.107:6827/174023257","cluster_addr":"172.21.15.107:6829/174023257","heartbeat_back_addr":"172.21.15.107:6833/174023257","heartbeat_front_addr":"172.21.15.107:6831/174023257","state":["exists","up"]},{"osd":10,"uuid":"4acd7b2f-39b0-44d8-8d3a-69d58cbd2226","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6825","nonce":1847892477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6827","nonce":1847892477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6831","nonce":1847892477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":1847892477},{"type":"v1","addr":"172.21.15.170:6829","nonce":1847892477}]},"public_addr":"172.21.15.170:6825/1847892477","cluster_addr":"172.21.15.170:6827/1847892477","heartbeat_back_addr":"172.21.15.170:6831/1847892477","heartbeat_front_addr":"172.21.15.170:6829/1847892477","state":["exists","up"]},{"osd":11,"uuid":"61c096c8-f55a-442c-9fa8-00231fdbd258","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6825","nonce":645474884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6827","nonce":645474884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6831","nonce":645474884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":645474884},{"type":"v1","addr":"172.21.15.195:6829","nonce":645474884}]},"public_addr":"172.21.15.195:6825/645474884","cluster_addr":"172.21.15.195:6827/645474884","heartbeat_back_addr":"172.21.15.195:6831/645474884","heartbeat_front_addr":"172.21.15.195:6829/645474884","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:34.358178+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:08:58.250077+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:21.489371+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:09:46.793224+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:11.477227+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:35.947277+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:01.487687+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:26.608380+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:51.474516+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:17.749468+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:43.538930+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.518606+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":1}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.6","mappings":[{"from":5,"to":8},{"from":9,"to":11}]},{"pgid":"4.13","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/151603643":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/1884923543":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/111369358":"2024-04-26T00:16:26.229442+0000","172.21.15.107:6800/1644942185":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/1174178948":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:6800/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/120616774":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/971124970":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3936375305":"2024-04-26T00:16:26.229442+0000","172.21.15.107:0/3022844652":"2024-04-26T00:06:34.124795+0000","172.21.15.107:6801/1644942185":"2024-04-26T00:16:26.229442+0000","172.21.15.107:6801/153169302":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/4215203700":"2024-04-26T00:07:00.838246+0000","172.21.15.107:0/750288374":"2024-04-26T00:06:34.124795+0000","172.21.15.107:0/3718680642":"2024-04-26T00:07:00.838246+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:27:30.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.757+0000 7fb8000a5640 1 -- 172.21.15.107:0/2501354166 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb7e40a2d00 msgr2=0x7fb7e40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:30.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.757+0000 7fb8000a5640 1 --2- 172.21.15.107:0/2501354166 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb7e40a2d00 0x7fb7e40a51c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800bb60 tx=0x7fb7e800b390 comp rx=0 tx=0).stop 2024-04-25T00:27:30.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.757+0000 7fb8000a5640 1 -- 172.21.15.107:0/2501354166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f8106630 msgr2=0x7fb7f819c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:30.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.757+0000 7fb8000a5640 1 --2- 172.21.15.107:0/2501354166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f8106630 0x7fb7f819c890 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0005cc0 tx=0x7fb7f0032f40 comp rx=0 tx=0).stop 2024-04-25T00:27:30.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.757+0000 7fb8000a5640 1 -- 172.21.15.107:0/2501354166 shutdown_connections 2024-04-25T00:27:30.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.757+0000 7fb8000a5640 1 --2- 172.21.15.107:0/2501354166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7f8106630 0x7fb7f819c890 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.757+0000 7fb8000a5640 1 --2- 172.21.15.107:0/2501354166 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7f8100fe0 0x7fb7f8198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.757+0000 7fb8000a5640 1 --2- 172.21.15.107:0/2501354166 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb7e40a2d00 0x7fb7e40a51c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.757+0000 7fb8000a5640 1 --2- 172.21.15.107:0/2501354166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb7f8100620 0x7fb7f8198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.757+0000 7fb8000a5640 1 -- 172.21.15.107:0/2501354166 >> 172.21.15.107:0/2501354166 conn(0x7fb7f8077740 msgr2=0x7fb7f8102d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:30.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.757+0000 7fb8000a5640 1 -- 172.21.15.107:0/2501354166 shutdown_connections 2024-04-25T00:27:30.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.757+0000 7fb8000a5640 1 -- 172.21.15.107:0/2501354166 wait complete. 2024-04-25T00:27:30.768 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:27:30.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.865+0000 7f90bb709640 1 -- 172.21.15.107:0/969989962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90b4100fe0 msgr2=0x7f90b410d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:30.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.865+0000 7f90bb709640 1 --2- 172.21.15.107:0/969989962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90b4100fe0 0x7f90b410d4f0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009cd0 tx=0x7f90ac02a8e0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.865+0000 7f90bb709640 1 -- 172.21.15.107:0/969989962 shutdown_connections 2024-04-25T00:27:30.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.865+0000 7f90bb709640 1 --2- 172.21.15.107:0/969989962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90b4100fe0 0x7f90b410d4f0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.865+0000 7f90bb709640 1 --2- 172.21.15.107:0/969989962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90b4100620 0x7f90b4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.865+0000 7f90bb709640 1 --2- 172.21.15.107:0/969989962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90b4106630 0x7f90b4106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.865+0000 7f90bb709640 1 -- 172.21.15.107:0/969989962 >> 172.21.15.107:0/969989962 conn(0x7f90b40769c0 msgr2=0x7f90b4076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:30.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.866+0000 7f90bb709640 1 -- 172.21.15.107:0/969989962 shutdown_connections 2024-04-25T00:27:30.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.866+0000 7f90bb709640 1 -- 172.21.15.107:0/969989962 wait complete. 2024-04-25T00:27:30.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.866+0000 7f90bb709640 1 Processor -- start 2024-04-25T00:27:30.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.866+0000 7f90bb709640 1 -- start start 2024-04-25T00:27:30.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.866+0000 7f90bb709640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90b4100620 0x7f90b4198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:30.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.866+0000 7f90bb709640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90b4100fe0 0x7f90b4198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:30.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.866+0000 7f90bb709640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90b4106630 0x7f90b419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:30.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.866+0000 7f90bb709640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f90b4111b40 con 0x7f90b4100fe0 2024-04-25T00:27:30.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.866+0000 7f90bb709640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f90b41119c0 con 0x7f90b4100620 2024-04-25T00:27:30.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.866+0000 7f90bb709640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f90b4111cc0 con 0x7f90b4106630 2024-04-25T00:27:30.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.866+0000 7f90b8ca4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90b4100fe0 0x7f90b4198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:30.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.866+0000 7f90b8ca4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90b4100fe0 0x7f90b4198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57948/0 (socket says 172.21.15.107:57948) 2024-04-25T00:27:30.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.866+0000 7f90b8ca4640 1 -- 172.21.15.107:0/1017307637 learned_addr learned my addr 172.21.15.107:0/1017307637 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:27:30.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.867+0000 7f90b94a5640 1 --2- 172.21.15.107:0/1017307637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90b4100620 0x7f90b4198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:30.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.867+0000 7f90b9ca6640 1 --2- 172.21.15.107:0/1017307637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90b4106630 0x7f90b419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:30.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.867+0000 7f90b8ca4640 1 -- 172.21.15.107:0/1017307637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90b4106630 msgr2=0x7f90b419c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:30.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.867+0000 7f90b8ca4640 1 --2- 172.21.15.107:0/1017307637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90b4106630 0x7f90b419c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.867+0000 7f90b8ca4640 1 -- 172.21.15.107:0/1017307637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90b4100620 msgr2=0x7f90b4198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:30.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.867+0000 7f90b8ca4640 1 --2- 172.21.15.107:0/1017307637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90b4100620 0x7f90b4198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.867+0000 7f90b8ca4640 1 -- 172.21.15.107:0/1017307637 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b419d030 con 0x7f90b4100fe0 2024-04-25T00:27:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.867+0000 7f90b94a5640 1 --2- 172.21.15.107:0/1017307637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90b4100620 0x7f90b4198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.867+0000 7f90b8ca4640 1 --2- 172.21.15.107:0/1017307637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90b4100fe0 0x7f90b4198580 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f90a400b4a0 tx=0x7f90a400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.867+0000 7f90aa7fc640 1 -- 172.21.15.107:0/1017307637 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4014070 con 0x7f90b4100fe0 2024-04-25T00:27:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.867+0000 7f90bb709640 1 -- 172.21.15.107:0/1017307637 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b419d320 con 0x7f90b4100fe0 2024-04-25T00:27:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.867+0000 7f90aa7fc640 1 -- 172.21.15.107:0/1017307637 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90a4010af0 con 0x7f90b4100fe0 2024-04-25T00:27:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.867+0000 7f90aa7fc640 1 -- 172.21.15.107:0/1017307637 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4017d10 con 0x7f90b4100fe0 2024-04-25T00:27:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.867+0000 7f90bb709640 1 -- 172.21.15.107:0/1017307637 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b41a4bb0 con 0x7f90b4100fe0 2024-04-25T00:27:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.868+0000 7f90bb709640 1 -- 172.21.15.107:0/1017307637 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90b40689b0 con 0x7f90b4100fe0 2024-04-25T00:27:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.869+0000 7f90aa7fc640 1 -- 172.21.15.107:0/1017307637 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f90a4017070 con 0x7f90b4100fe0 2024-04-25T00:27:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.870+0000 7f90aa7fc640 1 --2- 172.21.15.107:0/1017307637 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f90a00a2d00 0x7f90a00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.870+0000 7f90b94a5640 1 --2- 172.21.15.107:0/1017307637 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f90a00a2d00 0x7f90a00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.870+0000 7f90aa7fc640 1 -- 172.21.15.107:0/1017307637 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9483+0+0 (secure 0 0 0) 0x7f90a40ceb80 con 0x7f90b4100fe0 2024-04-25T00:27:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.870+0000 7f90b94a5640 1 --2- 172.21.15.107:0/1017307637 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f90a00a2d00 0x7f90a00a51c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f90b00058c0 tx=0x7f90b0005ec0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:30.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:30.874+0000 7f90aa7fc640 1 -- 172.21.15.107:0/1017307637 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f90a409b4f0 con 0x7f90b4100fe0 2024-04-25T00:27:31.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.084+0000 7f90bb709640 1 -- 172.21.15.107:0/1017307637 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f90b4199390 con 0x7f90b4100fe0 2024-04-25T00:27:31.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.086+0000 7f90aa7fc640 1 -- 172.21.15.107:0/1017307637 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) ==== 161+0+0 (secure 0 0 0) 0x7f90b4199390 con 0x7f90b4100fe0 2024-04-25T00:27:31.087 INFO:teuthology.orchestra.run.smithi107.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:27:31.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.089+0000 7f90bb709640 1 -- 172.21.15.107:0/1017307637 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f90a00a2d00 msgr2=0x7f90a00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:31.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.089+0000 7f90bb709640 1 --2- 172.21.15.107:0/1017307637 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f90a00a2d00 0x7f90a00a51c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f90b00058c0 tx=0x7f90b0005ec0 comp rx=0 tx=0).stop 2024-04-25T00:27:31.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.089+0000 7f90bb709640 1 -- 172.21.15.107:0/1017307637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90b4100fe0 msgr2=0x7f90b4198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:31.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.089+0000 7f90bb709640 1 --2- 172.21.15.107:0/1017307637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90b4100fe0 0x7f90b4198580 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f90a400b4a0 tx=0x7f90a400b960 comp rx=0 tx=0).stop 2024-04-25T00:27:31.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.089+0000 7f90bb709640 1 -- 172.21.15.107:0/1017307637 shutdown_connections 2024-04-25T00:27:31.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.089+0000 7f90bb709640 1 --2- 172.21.15.107:0/1017307637 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f90b4106630 0x7f90b419c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:31.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.089+0000 7f90bb709640 1 --2- 172.21.15.107:0/1017307637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90b4100fe0 0x7f90b4198580 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:31.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.089+0000 7f90bb709640 1 --2- 172.21.15.107:0/1017307637 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f90a00a2d00 0x7f90a00a51c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:31.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.089+0000 7f90bb709640 1 --2- 172.21.15.107:0/1017307637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90b4100620 0x7f90b4198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:31.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.089+0000 7f90bb709640 1 -- 172.21.15.107:0/1017307637 >> 172.21.15.107:0/1017307637 conn(0x7f90b40769c0 msgr2=0x7f90b40fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:31.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.089+0000 7f90bb709640 1 -- 172.21.15.107:0/1017307637 shutdown_connections 2024-04-25T00:27:31.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.089+0000 7f90bb709640 1 -- 172.21.15.107:0/1017307637 wait complete. 2024-04-25T00:27:31.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:30 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2108621845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:31.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:30 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/938908760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:31.101 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:27:31.135 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:30 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2108621845' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:31.135 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:30 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/938908760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:27:31.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.205+0000 7fb5520ee640 1 -- 172.21.15.107:0/329384553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb54c101180 msgr2=0x7fb54c101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:31.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.205+0000 7fb5520ee640 1 --2- 172.21.15.107:0/329384553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb54c101180 0x7fb54c101600 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fb540009930 tx=0x7fb54002a420 comp rx=0 tx=0).stop 2024-04-25T00:27:31.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.205+0000 7fb5520ee640 1 -- 172.21.15.107:0/329384553 shutdown_connections 2024-04-25T00:27:31.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.205+0000 7fb5520ee640 1 --2- 172.21.15.107:0/329384553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb54c101b40 0x7fb54c10d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:31.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.205+0000 7fb5520ee640 1 --2- 172.21.15.107:0/329384553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb54c101180 0x7fb54c101600 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:31.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.205+0000 7fb5520ee640 1 --2- 172.21.15.107:0/329384553 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb54c105610 0x7fb54c1059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:31.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.205+0000 7fb5520ee640 1 -- 172.21.15.107:0/329384553 >> 172.21.15.107:0/329384553 conn(0x7fb54c0758d0 msgr2=0x7fb54c075ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:31.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.205+0000 7fb5520ee640 1 -- 172.21.15.107:0/329384553 shutdown_connections 2024-04-25T00:27:31.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.205+0000 7fb5520ee640 1 -- 172.21.15.107:0/329384553 wait complete. 2024-04-25T00:27:31.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.206+0000 7fb5520ee640 1 Processor -- start 2024-04-25T00:27:31.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.206+0000 7fb5520ee640 1 -- start start 2024-04-25T00:27:31.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.206+0000 7fb5520ee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb54c101180 0x7fb54c198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:31.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.206+0000 7fb5520ee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb54c101b40 0x7fb54c198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:31.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.206+0000 7fb5520ee640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb54c105610 0x7fb54c19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:31.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.206+0000 7fb5520ee640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c111b40 con 0x7fb54c101180 2024-04-25T00:27:31.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.206+0000 7fb5520ee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c1119c0 con 0x7fb54c101b40 2024-04-25T00:27:31.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.206+0000 7fb5520ee640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c111cc0 con 0x7fb54c105610 2024-04-25T00:27:31.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.206+0000 7fb54b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb54c101180 0x7fb54c198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:31.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.206+0000 7fb54b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb54c101180 0x7fb54c198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57966/0 (socket says 172.21.15.107:57966) 2024-04-25T00:27:31.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.206+0000 7fb54b7fe640 1 -- 172.21.15.107:0/3783866341 learned_addr learned my addr 172.21.15.107:0/3783866341 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:27:31.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.207+0000 7fb54affd640 1 --2- 172.21.15.107:0/3783866341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb54c101b40 0x7fb54c198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:31.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.207+0000 7fb54bfff640 1 --2- 172.21.15.107:0/3783866341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb54c105610 0x7fb54c19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:31.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.207+0000 7fb54b7fe640 1 -- 172.21.15.107:0/3783866341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb54c105610 msgr2=0x7fb54c19c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:31.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.207+0000 7fb54b7fe640 1 --2- 172.21.15.107:0/3783866341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb54c105610 0x7fb54c19c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:31.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.207+0000 7fb54b7fe640 1 -- 172.21.15.107:0/3783866341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb54c101b40 msgr2=0x7fb54c198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:31.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.207+0000 7fb54b7fe640 1 --2- 172.21.15.107:0/3783866341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb54c101b40 0x7fb54c198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:31.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.207+0000 7fb54b7fe640 1 -- 172.21.15.107:0/3783866341 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54c19d030 con 0x7fb54c101180 2024-04-25T00:27:31.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.207+0000 7fb54bfff640 1 --2- 172.21.15.107:0/3783866341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb54c105610 0x7fb54c19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:31.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.207+0000 7fb54b7fe640 1 --2- 172.21.15.107:0/3783866341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb54c101180 0x7fb54c198040 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00d950 tx=0x7fb53c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:31.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.207+0000 7fb548ff9640 1 -- 172.21.15.107:0/3783866341 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c017070 con 0x7fb54c101180 2024-04-25T00:27:31.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.207+0000 7fb5520ee640 1 -- 172.21.15.107:0/3783866341 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb54c19d320 con 0x7fb54c101180 2024-04-25T00:27:31.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.207+0000 7fb548ff9640 1 -- 172.21.15.107:0/3783866341 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb53c009c30 con 0x7fb54c101180 2024-04-25T00:27:31.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.207+0000 7fb548ff9640 1 -- 172.21.15.107:0/3783866341 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c00f600 con 0x7fb54c101180 2024-04-25T00:27:31.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.207+0000 7fb5520ee640 1 -- 172.21.15.107:0/3783866341 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb54c1a4bb0 con 0x7fb54c101180 2024-04-25T00:27:31.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.208+0000 7fb5520ee640 1 -- 172.21.15.107:0/3783866341 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb54c0689b0 con 0x7fb54c101180 2024-04-25T00:27:31.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.209+0000 7fb548ff9640 1 -- 172.21.15.107:0/3783866341 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fb53c01a070 con 0x7fb54c101180 2024-04-25T00:27:31.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.210+0000 7fb548ff9640 1 --2- 172.21.15.107:0/3783866341 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb5340a2d00 0x7fb5340a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:31.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.210+0000 7fb54affd640 1 --2- 172.21.15.107:0/3783866341 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb5340a2d00 0x7fb5340a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:31.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.210+0000 7fb548ff9640 1 -- 172.21.15.107:0/3783866341 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9653+0+0 (secure 0 0 0) 0x7fb53c0cf2d0 con 0x7fb54c101180 2024-04-25T00:27:31.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.210+0000 7fb54affd640 1 --2- 172.21.15.107:0/3783866341 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb5340a2d00 0x7fb5340a51c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb540009900 tx=0x7fb5400058d0 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:31.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.215+0000 7fb548ff9640 1 -- 172.21.15.107:0/3783866341 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb53c009de0 con 0x7fb54c101180 2024-04-25T00:27:31.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:31.427+0000 7fb5520ee640 1 -- 172.21.15.107:0/3783866341 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb54c108b40 con 0x7fb54c101180 2024-04-25T00:27:31.435 INFO:journalctl@ceph.mds.b.smithi170.stdout:Apr 25 00:27:31 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-b[62656]: ignoring --setuser ceph since I am not root 2024-04-25T00:27:31.435 INFO:journalctl@ceph.mds.b.smithi170.stdout:Apr 25 00:27:31 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-b[62656]: ignoring --setgroup ceph since I am not root 2024-04-25T00:27:31.435 INFO:journalctl@ceph.mds.b.smithi170.stdout:Apr 25 00:27:31 smithi170 ceph-mds[62660]: -- 172.21.15.170:0/1400200158 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a23fa29a40 con 0x55a23f9fd000 2024-04-25T00:27:31.435 INFO:journalctl@ceph.mds.b.smithi170.stdout:Apr 25 00:27:31 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-b[62656]: starting mds.b at 2024-04-25T00:27:31.435 INFO:journalctl@ceph.mds.h.smithi170.stdout:Apr 25 00:27:31 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-h[62346]: ignoring --setuser ceph since I am not root 2024-04-25T00:27:31.435 INFO:journalctl@ceph.mds.h.smithi170.stdout:Apr 25 00:27:31 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-h[62346]: ignoring --setgroup ceph since I am not root 2024-04-25T00:27:31.435 INFO:journalctl@ceph.mds.h.smithi170.stdout:Apr 25 00:27:31 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-h[62346]: starting mds.h at 2024-04-25T00:27:31.436 INFO:journalctl@ceph.mds.e.smithi170.stdout:Apr 25 00:27:31 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-e[62024]: ignoring --setuser ceph since I am not root 2024-04-25T00:27:31.436 INFO:journalctl@ceph.mds.e.smithi170.stdout:Apr 25 00:27:31 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-e[62024]: ignoring --setgroup ceph since I am not root 2024-04-25T00:27:31.436 INFO:journalctl@ceph.mds.e.smithi170.stdout:Apr 25 00:27:31 smithi170 ceph-mds[62028]: -- 172.21.15.170:0/1753465138 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556a25f95a40 con 0x556a25f68800 2024-04-25T00:27:31.436 INFO:journalctl@ceph.mds.e.smithi170.stdout:Apr 25 00:27:31 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-e[62024]: starting mds.e at 2024-04-25T00:27:31.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:31 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-a[29149]: 2024-04-25T00:27:31.085+0000 7fd7892b9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:27:31.589 INFO:journalctl@ceph.mds.i.smithi195.stdout:Apr 25 00:27:31 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-i[64104]: ignoring --setuser ceph since I am not root 2024-04-25T00:27:31.589 INFO:journalctl@ceph.mds.i.smithi195.stdout:Apr 25 00:27:31 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-i[64104]: ignoring --setgroup ceph since I am not root 2024-04-25T00:27:31.589 INFO:journalctl@ceph.mds.i.smithi195.stdout:Apr 25 00:27:31 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-i[64104]: starting mds.i at 2024-04-25T00:27:31.590 INFO:journalctl@ceph.mds.c.smithi195.stdout:Apr 25 00:27:31 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-c[63784]: ignoring --setuser ceph since I am not root 2024-04-25T00:27:31.590 INFO:journalctl@ceph.mds.c.smithi195.stdout:Apr 25 00:27:31 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-c[63784]: ignoring --setgroup ceph since I am not root 2024-04-25T00:27:31.590 INFO:journalctl@ceph.mds.c.smithi195.stdout:Apr 25 00:27:31 smithi195 ceph-mds[63788]: -- 172.21.15.195:0/3047886929 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559dec457a40 con 0x559dec42b000 2024-04-25T00:27:31.590 INFO:journalctl@ceph.mds.c.smithi195.stdout:Apr 25 00:27:31 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-c[63784]: starting mds.c at 2024-04-25T00:27:31.590 INFO:journalctl@ceph.mds.f.smithi195.stdout:Apr 25 00:27:31 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-f[63483]: ignoring --setuser ceph since I am not root 2024-04-25T00:27:31.590 INFO:journalctl@ceph.mds.f.smithi195.stdout:Apr 25 00:27:31 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-f[63483]: ignoring --setgroup ceph since I am not root 2024-04-25T00:27:31.590 INFO:journalctl@ceph.mds.f.smithi195.stdout:Apr 25 00:27:31 smithi195 ceph-mds[63487]: -- 172.21.15.195:0/209480456 <== mon.1 v2:172.21.15.195:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555ec98d5a40 con 0x555ec98a8800 2024-04-25T00:27:31.591 INFO:journalctl@ceph.mds.f.smithi195.stdout:Apr 25 00:27:31 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-f[63483]: starting mds.f at 2024-04-25T00:27:31.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:31 smithi107 ceph-mon[29153]: pgmap v345: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:27:31.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:31 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/2501354166' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:27:31.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:31 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1017307637' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T00:27:31.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:31 smithi107 ceph-mon[29153]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T00:27:31.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:31 smithi107 ceph-mon[29153]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:27:31.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:31 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1017307637' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T00:27:31.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:31 smithi107 ceph-mon[29153]: osdmap e108: 12 total, 12 up, 12 in 2024-04-25T00:27:31.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:31 smithi107 ceph-mon[29153]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T00:27:31.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:31 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3783866341' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:27:32.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.089+0000 7fb548ff9640 1 -- 172.21.15.107:0/3783866341 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) ==== 104+0+0 (secure 0 0 0) 0x7fb53c09ba80 con 0x7fb54c101180 2024-04-25T00:27:32.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:31 smithi195 ceph-mon[32833]: pgmap v345: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:27:32.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:31 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/2501354166' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:27:32.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:31 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1017307637' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T00:27:32.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:31 smithi195 ceph-mon[32833]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T00:27:32.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:31 smithi195 ceph-mon[32833]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:27:32.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:31 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1017307637' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T00:27:32.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:31 smithi195 ceph-mon[32833]: osdmap e108: 12 total, 12 up, 12 in 2024-04-25T00:27:32.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:31 smithi195 ceph-mon[32833]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T00:27:32.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:31 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3783866341' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:27:32.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.091+0000 7fb5520ee640 1 -- 172.21.15.107:0/3783866341 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb5340a2d00 msgr2=0x7fb5340a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:32.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.091+0000 7fb5520ee640 1 --2- 172.21.15.107:0/3783866341 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb5340a2d00 0x7fb5340a51c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb540009900 tx=0x7fb5400058d0 comp rx=0 tx=0).stop 2024-04-25T00:27:32.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.091+0000 7fb5520ee640 1 -- 172.21.15.107:0/3783866341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb54c101180 msgr2=0x7fb54c198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:32.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.091+0000 7fb5520ee640 1 --2- 172.21.15.107:0/3783866341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb54c101180 0x7fb54c198040 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fb53c00d950 tx=0x7fb53c00de10 comp rx=0 tx=0).stop 2024-04-25T00:27:32.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.091+0000 7fb5520ee640 1 -- 172.21.15.107:0/3783866341 shutdown_connections 2024-04-25T00:27:32.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.091+0000 7fb5520ee640 1 --2- 172.21.15.107:0/3783866341 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb54c105610 0x7fb54c19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:32.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.091+0000 7fb5520ee640 1 --2- 172.21.15.107:0/3783866341 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fb5340a2d00 0x7fb5340a51c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:32.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.091+0000 7fb5520ee640 1 --2- 172.21.15.107:0/3783866341 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb54c101b40 0x7fb54c198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:32.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.091+0000 7fb5520ee640 1 --2- 172.21.15.107:0/3783866341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb54c101180 0x7fb54c198040 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:32.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.091+0000 7fb5520ee640 1 -- 172.21.15.107:0/3783866341 >> 172.21.15.107:0/3783866341 conn(0x7fb54c0758d0 msgr2=0x7fb54c0feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:32.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.091+0000 7fb5520ee640 1 -- 172.21.15.107:0/3783866341 shutdown_connections 2024-04-25T00:27:32.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.091+0000 7fb5520ee640 1 -- 172.21.15.107:0/3783866341 wait complete. 2024-04-25T00:27:32.106 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:27:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:31 smithi170 ceph-mon[32807]: pgmap v345: 129 pgs: 129 active+clean; 191 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:27:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:31 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/2501354166' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:27:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:31 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1017307637' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T00:27:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:31 smithi170 ceph-mon[32807]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T00:27:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:31 smithi170 ceph-mon[32807]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:27:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:31 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1017307637' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T00:27:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:31 smithi170 ceph-mon[32807]: osdmap e108: 12 total, 12 up, 12 in 2024-04-25T00:27:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:31 smithi170 ceph-mon[32807]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T00:27:32.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:31 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3783866341' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:27:32.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.211+0000 7f4e199f4640 1 -- 172.21.15.107:0/2676396498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e141012e0 msgr2=0x7f4e1410d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:32.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.211+0000 7f4e199f4640 1 --2- 172.21.15.107:0/2676396498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e141012e0 0x7f4e1410d4b0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c009cd0 tx=0x7f4e0c02a8e0 comp rx=0 tx=0).stop 2024-04-25T00:27:32.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.211+0000 7f4e199f4640 1 -- 172.21.15.107:0/2676396498 shutdown_connections 2024-04-25T00:27:32.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.211+0000 7f4e199f4640 1 --2- 172.21.15.107:0/2676396498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e141012e0 0x7f4e1410d4b0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:32.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.211+0000 7f4e199f4640 1 --2- 172.21.15.107:0/2676396498 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e14100920 0x7f4e14100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:32.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.211+0000 7f4e199f4640 1 --2- 172.21.15.107:0/2676396498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e14106810 0x7f4e14106bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:32.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.211+0000 7f4e199f4640 1 -- 172.21.15.107:0/2676396498 >> 172.21.15.107:0/2676396498 conn(0x7f4e14077620 msgr2=0x7f4e14077a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:32.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.211+0000 7f4e199f4640 1 -- 172.21.15.107:0/2676396498 shutdown_connections 2024-04-25T00:27:32.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.211+0000 7f4e199f4640 1 -- 172.21.15.107:0/2676396498 wait complete. 2024-04-25T00:27:32.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.212+0000 7f4e199f4640 1 Processor -- start 2024-04-25T00:27:32.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.212+0000 7f4e199f4640 1 -- start start 2024-04-25T00:27:32.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.212+0000 7f4e199f4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e14100920 0x7f4e14196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:32.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.212+0000 7f4e199f4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e141012e0 0x7f4e14197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:32.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.212+0000 7f4e199f4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e14106810 0x7f4e1419b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:32.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.212+0000 7f4e199f4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4e14110b20 con 0x7f4e141012e0 2024-04-25T00:27:32.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.212+0000 7f4e199f4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4e141109a0 con 0x7f4e14106810 2024-04-25T00:27:32.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.212+0000 7f4e199f4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7f4e14110ca0 con 0x7f4e14100920 2024-04-25T00:27:32.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.212+0000 7f4e0bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e141012e0 0x7f4e14197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:32.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.212+0000 7f4e0bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e141012e0 0x7f4e14197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57998/0 (socket says 172.21.15.107:57998) 2024-04-25T00:27:32.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.212+0000 7f4e0bfff640 1 -- 172.21.15.107:0/3467056716 learned_addr learned my addr 172.21.15.107:0/3467056716 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:27:32.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.212+0000 7f4e12ffd640 1 --2- 172.21.15.107:0/3467056716 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e14100920 0x7f4e14196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:32.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.212+0000 7f4e0bfff640 1 -- 172.21.15.107:0/3467056716 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e14100920 msgr2=0x7f4e14196fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:32.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.212+0000 7f4e0bfff640 1 --2- 172.21.15.107:0/3467056716 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e14100920 0x7f4e14196fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:32.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.212+0000 7f4e137fe640 1 --2- 172.21.15.107:0/3467056716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e14106810 0x7f4e1419b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:32.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.212+0000 7f4e0bfff640 1 -- 172.21.15.107:0/3467056716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e14106810 msgr2=0x7f4e1419b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:32.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.212+0000 7f4e0bfff640 1 --2- 172.21.15.107:0/3467056716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e14106810 0x7f4e1419b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:32.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.212+0000 7f4e0bfff640 1 -- 172.21.15.107:0/3467056716 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e1419bf80 con 0x7f4e141012e0 2024-04-25T00:27:32.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.213+0000 7f4e0bfff640 1 --2- 172.21.15.107:0/3467056716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e141012e0 0x7f4e14197520 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00d770 tx=0x7f4dfc00db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:32.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.213+0000 7f4e189f2640 1 -- 172.21.15.107:0/3467056716 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc017070 con 0x7f4e141012e0 2024-04-25T00:27:32.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.213+0000 7f4e189f2640 1 -- 172.21.15.107:0/3467056716 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4dfc009820 con 0x7f4e141012e0 2024-04-25T00:27:32.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.213+0000 7f4e199f4640 1 -- 172.21.15.107:0/3467056716 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e1419c270 con 0x7f4e141012e0 2024-04-25T00:27:32.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.213+0000 7f4e189f2640 1 -- 172.21.15.107:0/3467056716 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc004cd0 con 0x7f4e141012e0 2024-04-25T00:27:32.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.213+0000 7f4e199f4640 1 -- 172.21.15.107:0/3467056716 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e141a3b00 con 0x7f4e141012e0 2024-04-25T00:27:32.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.214+0000 7f4e199f4640 1 -- 172.21.15.107:0/3467056716 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e140689b0 con 0x7f4e141012e0 2024-04-25T00:27:32.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.215+0000 7f4e189f2640 1 -- 172.21.15.107:0/3467056716 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7f4dfc0099d0 con 0x7f4e141012e0 2024-04-25T00:27:32.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.216+0000 7f4e189f2640 1 --2- 172.21.15.107:0/3467056716 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4e000a2d00 0x7f4e000a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:32.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.216+0000 7f4e189f2640 1 -- 172.21.15.107:0/3467056716 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9653+0+0 (secure 0 0 0) 0x7f4dfc0cf300 con 0x7f4e141012e0 2024-04-25T00:27:32.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.216+0000 7f4e12ffd640 1 --2- 172.21.15.107:0/3467056716 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4e000a2d00 0x7f4e000a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:32.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.216+0000 7f4e12ffd640 1 --2- 172.21.15.107:0/3467056716 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4e000a2d00 0x7f4e000a51c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009990 tx=0x7f4e04009110 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:32.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.220+0000 7f4e189f2640 1 -- 172.21.15.107:0/3467056716 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4dfc09bab0 con 0x7f4e141012e0 2024-04-25T00:27:32.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:32.398+0000 7f4e199f4640 1 -- 172.21.15.107:0/3467056716 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4e14108b30 con 0x7f4e141012e0 2024-04-25T00:27:32.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:32 smithi107 ceph-mon[29153]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T00:27:32.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:32 smithi107 ceph-mon[29153]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T00:27:32.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:32 smithi107 ceph-mon[29153]: Cluster is now healthy 2024-04-25T00:27:32.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:32 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3783866341' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:27:32.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:32 smithi107 ceph-mon[29153]: mds.? [v2:172.21.15.195:6834/335157084,v1:172.21.15.195:6837/335157084] up:boot 2024-04-25T00:27:32.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:32 smithi107 ceph-mon[29153]: mds.? [v2:172.21.15.195:6835/3151310782,v1:172.21.15.195:6838/3151310782] up:boot 2024-04-25T00:27:32.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:32 smithi107 ceph-mon[29153]: mds.? [v2:172.21.15.195:6836/3541188910,v1:172.21.15.195:6839/3541188910] up:boot 2024-04-25T00:27:32.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:32 smithi107 ceph-mon[29153]: mds.? [v2:172.21.15.170:6834/96717858,v1:172.21.15.170:6836/96717858] up:boot 2024-04-25T00:27:32.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:32 smithi107 ceph-mon[29153]: mds.? [v2:172.21.15.170:6835/2124923261,v1:172.21.15.170:6837/2124923261] up:boot 2024-04-25T00:27:32.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:32 smithi107 ceph-mon[29153]: mds.? [v2:172.21.15.170:6838/566202820,v1:172.21.15.170:6839/566202820] up:boot 2024-04-25T00:27:32.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:32 smithi107 ceph-mon[29153]: fsmap 12 up:standby 2024-04-25T00:27:32.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:32 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:27:32.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:32 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:27:32.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:32 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:27:32.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:32 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:27:32.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:32 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:27:32.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:32 smithi107 ceph-mon[29153]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:27:32.985 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:32 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3467056716' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:27:33.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:32 smithi195 ceph-mon[32833]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T00:27:33.089 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:32 smithi195 ceph-mon[32833]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T00:27:33.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:32 smithi195 ceph-mon[32833]: Cluster is now healthy 2024-04-25T00:27:33.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:32 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3783866341' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:27:33.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:32 smithi195 ceph-mon[32833]: mds.? [v2:172.21.15.195:6834/335157084,v1:172.21.15.195:6837/335157084] up:boot 2024-04-25T00:27:33.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:32 smithi195 ceph-mon[32833]: mds.? [v2:172.21.15.195:6835/3151310782,v1:172.21.15.195:6838/3151310782] up:boot 2024-04-25T00:27:33.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:32 smithi195 ceph-mon[32833]: mds.? [v2:172.21.15.195:6836/3541188910,v1:172.21.15.195:6839/3541188910] up:boot 2024-04-25T00:27:33.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:32 smithi195 ceph-mon[32833]: mds.? [v2:172.21.15.170:6834/96717858,v1:172.21.15.170:6836/96717858] up:boot 2024-04-25T00:27:33.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:32 smithi195 ceph-mon[32833]: mds.? [v2:172.21.15.170:6835/2124923261,v1:172.21.15.170:6837/2124923261] up:boot 2024-04-25T00:27:33.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:32 smithi195 ceph-mon[32833]: mds.? [v2:172.21.15.170:6838/566202820,v1:172.21.15.170:6839/566202820] up:boot 2024-04-25T00:27:33.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:32 smithi195 ceph-mon[32833]: fsmap 12 up:standby 2024-04-25T00:27:33.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:32 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:27:33.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:32 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:27:33.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:32 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:27:33.090 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:32 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:27:33.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:32 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:27:33.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:32 smithi195 ceph-mon[32833]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:27:33.091 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:32 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3467056716' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:27:33.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.090+0000 7f4e189f2640 1 -- 172.21.15.107:0/3467056716 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v109) ==== 181+0+0 (secure 0 0 0) 0x7f4dfc09b3b0 con 0x7f4e141012e0 2024-04-25T00:27:33.091 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:27:33.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.094+0000 7f4e199f4640 1 -- 172.21.15.107:0/3467056716 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4e000a2d00 msgr2=0x7f4e000a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:33.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.094+0000 7f4e199f4640 1 --2- 172.21.15.107:0/3467056716 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4e000a2d00 0x7f4e000a51c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009990 tx=0x7f4e04009110 comp rx=0 tx=0).stop 2024-04-25T00:27:33.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.094+0000 7f4e199f4640 1 -- 172.21.15.107:0/3467056716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e141012e0 msgr2=0x7f4e14197520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:33.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.094+0000 7f4e199f4640 1 --2- 172.21.15.107:0/3467056716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e141012e0 0x7f4e14197520 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00d770 tx=0x7f4dfc00db70 comp rx=0 tx=0).stop 2024-04-25T00:27:33.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.094+0000 7f4e199f4640 1 -- 172.21.15.107:0/3467056716 shutdown_connections 2024-04-25T00:27:33.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.094+0000 7f4e199f4640 1 --2- 172.21.15.107:0/3467056716 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7f4e000a2d00 0x7f4e000a51c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:33.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.094+0000 7f4e199f4640 1 --2- 172.21.15.107:0/3467056716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4e14106810 0x7f4e1419b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:33.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.094+0000 7f4e199f4640 1 --2- 172.21.15.107:0/3467056716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e141012e0 0x7f4e14197520 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:33.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.094+0000 7f4e199f4640 1 --2- 172.21.15.107:0/3467056716 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e14100920 0x7f4e14196fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:33.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.094+0000 7f4e199f4640 1 -- 172.21.15.107:0/3467056716 >> 172.21.15.107:0/3467056716 conn(0x7f4e14077620 msgr2=0x7f4e140ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:33.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.094+0000 7f4e199f4640 1 -- 172.21.15.107:0/3467056716 shutdown_connections 2024-04-25T00:27:33.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.094+0000 7f4e199f4640 1 -- 172.21.15.107:0/3467056716 wait complete. 2024-04-25T00:27:33.111 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:27:33.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:32 smithi170 ceph-mon[32807]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T00:27:33.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:32 smithi170 ceph-mon[32807]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T00:27:33.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:32 smithi170 ceph-mon[32807]: Cluster is now healthy 2024-04-25T00:27:33.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:32 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3783866341' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:27:33.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:32 smithi170 ceph-mon[32807]: mds.? [v2:172.21.15.195:6834/335157084,v1:172.21.15.195:6837/335157084] up:boot 2024-04-25T00:27:33.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:32 smithi170 ceph-mon[32807]: mds.? [v2:172.21.15.195:6835/3151310782,v1:172.21.15.195:6838/3151310782] up:boot 2024-04-25T00:27:33.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:32 smithi170 ceph-mon[32807]: mds.? [v2:172.21.15.195:6836/3541188910,v1:172.21.15.195:6839/3541188910] up:boot 2024-04-25T00:27:33.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:32 smithi170 ceph-mon[32807]: mds.? [v2:172.21.15.170:6834/96717858,v1:172.21.15.170:6836/96717858] up:boot 2024-04-25T00:27:33.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:32 smithi170 ceph-mon[32807]: mds.? [v2:172.21.15.170:6835/2124923261,v1:172.21.15.170:6837/2124923261] up:boot 2024-04-25T00:27:33.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:32 smithi170 ceph-mon[32807]: mds.? [v2:172.21.15.170:6838/566202820,v1:172.21.15.170:6839/566202820] up:boot 2024-04-25T00:27:33.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:32 smithi170 ceph-mon[32807]: fsmap 12 up:standby 2024-04-25T00:27:33.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:32 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:27:33.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:32 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:27:33.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:32 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:27:33.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:32 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:27:33.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:32 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:27:33.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:32 smithi170 ceph-mon[32807]: from='mgr.15456 172.21.15.107:0/2396438813' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:27:33.186 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:32 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3467056716' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:27:33.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.216+0000 7fa9393a2640 1 -- 172.21.15.107:0/3659784756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9341020d0 msgr2=0x7fa93410e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:33.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.216+0000 7fa9393a2640 1 --2- 172.21.15.107:0/3659784756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9341020d0 0x7fa93410e5e0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fa92c009cd0 tx=0x7fa92c02a8e0 comp rx=0 tx=0).stop 2024-04-25T00:27:33.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.216+0000 7fa9393a2640 1 -- 172.21.15.107:0/3659784756 shutdown_connections 2024-04-25T00:27:33.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.216+0000 7fa9393a2640 1 --2- 172.21.15.107:0/3659784756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9341020d0 0x7fa93410e5e0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:33.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.216+0000 7fa9393a2640 1 --2- 172.21.15.107:0/3659784756 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa934101710 0x7fa934101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:33.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.216+0000 7fa9393a2640 1 --2- 172.21.15.107:0/3659784756 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa934107720 0x7fa934107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:33.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.216+0000 7fa9393a2640 1 -- 172.21.15.107:0/3659784756 >> 172.21.15.107:0/3659784756 conn(0x7fa934078b90 msgr2=0x7fa934078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:33.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.216+0000 7fa9393a2640 1 -- 172.21.15.107:0/3659784756 shutdown_connections 2024-04-25T00:27:33.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.216+0000 7fa9393a2640 1 -- 172.21.15.107:0/3659784756 wait complete. 2024-04-25T00:27:33.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.216+0000 7fa9393a2640 1 Processor -- start 2024-04-25T00:27:33.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.217+0000 7fa9393a2640 1 -- start start 2024-04-25T00:27:33.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.217+0000 7fa9393a2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa934101710 0x7fa934198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:33.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.217+0000 7fa9393a2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9341020d0 0x7fa934198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:33.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.217+0000 7fa9393a2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa934107720 0x7fa93419c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:33.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.217+0000 7fa9393a2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa934111bb0 con 0x7fa934107720 2024-04-25T00:27:33.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.217+0000 7fa9393a2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa934111a30 con 0x7fa934101710 2024-04-25T00:27:33.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.217+0000 7fa9393a2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fa934111d30 con 0x7fa9341020d0 2024-04-25T00:27:33.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.217+0000 7fa9337fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa934107720 0x7fa93419c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:33.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.217+0000 7fa9337fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa934107720 0x7fa93419c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58014/0 (socket says 172.21.15.107:58014) 2024-04-25T00:27:33.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.217+0000 7fa9337fe640 1 -- 172.21.15.107:0/3553198957 learned_addr learned my addr 172.21.15.107:0/3553198957 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:27:33.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.217+0000 7fa9327fc640 1 --2- 172.21.15.107:0/3553198957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9341020d0 0x7fa934198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:33.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.217+0000 7fa932ffd640 1 --2- 172.21.15.107:0/3553198957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa934101710 0x7fa934198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:33.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.217+0000 7fa9337fe640 1 -- 172.21.15.107:0/3553198957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9341020d0 msgr2=0x7fa934198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:33.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.217+0000 7fa9337fe640 1 --2- 172.21.15.107:0/3553198957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9341020d0 0x7fa934198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:33.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.217+0000 7fa9337fe640 1 -- 172.21.15.107:0/3553198957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa934101710 msgr2=0x7fa934198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:33.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.217+0000 7fa9337fe640 1 --2- 172.21.15.107:0/3553198957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa934101710 0x7fa934198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:33.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.217+0000 7fa9337fe640 1 -- 172.21.15.107:0/3553198957 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93419d120 con 0x7fa934107720 2024-04-25T00:27:33.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.217+0000 7fa932ffd640 1 --2- 172.21.15.107:0/3553198957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa934101710 0x7fa934198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:33.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.217+0000 7fa9327fc640 1 --2- 172.21.15.107:0/3553198957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9341020d0 0x7fa934198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:33.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.217+0000 7fa9337fe640 1 --2- 172.21.15.107:0/3553198957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa934107720 0x7fa93419c9f0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fa92c006010 tx=0x7fa92c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:33.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.217+0000 7fa91bfff640 1 -- 172.21.15.107:0/3553198957 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c004450 con 0x7fa934107720 2024-04-25T00:27:33.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.218+0000 7fa9393a2640 1 -- 172.21.15.107:0/3553198957 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa93419d3b0 con 0x7fa934107720 2024-04-25T00:27:33.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.218+0000 7fa91bfff640 1 -- 172.21.15.107:0/3553198957 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa92c003e10 con 0x7fa934107720 2024-04-25T00:27:33.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.218+0000 7fa91bfff640 1 -- 172.21.15.107:0/3553198957 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92c0466e0 con 0x7fa934107720 2024-04-25T00:27:33.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.218+0000 7fa9393a2640 1 -- 172.21.15.107:0/3553198957 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9341a4ca0 con 0x7fa934107720 2024-04-25T00:27:33.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.218+0000 7fa9393a2640 1 -- 172.21.15.107:0/3553198957 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9340689b0 con 0x7fa934107720 2024-04-25T00:27:33.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.219+0000 7fa91bfff640 1 -- 172.21.15.107:0/3553198957 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fa92c0305a0 con 0x7fa934107720 2024-04-25T00:27:33.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.220+0000 7fa91bfff640 1 --2- 172.21.15.107:0/3553198957 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa91c0a2d00 0x7fa91c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:33.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.220+0000 7fa932ffd640 1 --2- 172.21.15.107:0/3553198957 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa91c0a2d00 0x7fa91c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:33.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.220+0000 7fa91bfff640 1 -- 172.21.15.107:0/3553198957 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9120+0+0 (secure 0 0 0) 0x7fa92c02e030 con 0x7fa934107720 2024-04-25T00:27:33.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.221+0000 7fa932ffd640 1 --2- 172.21.15.107:0/3553198957 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa91c0a2d00 0x7fa91c0a51c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa928009930 tx=0x7fa928008040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:33.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.225+0000 7fa91bfff640 1 -- 172.21.15.107:0/3553198957 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa92c04b060 con 0x7fa934107720 2024-04-25T00:27:33.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:33.399+0000 7fa9393a2640 1 -- 172.21.15.107:0/3553198957 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa934109d00 con 0x7fa934107720 2024-04-25T00:27:34.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.098+0000 7fa91bfff640 1 -- 172.21.15.107:0/3553198957 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v110) ==== 169+0+0 (secure 0 0 0) 0x7fa92c0bdac0 con 0x7fa934107720 2024-04-25T00:27:34.098 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_data' removed 2024-04-25T00:27:34.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.101+0000 7fa9393a2640 1 -- 172.21.15.107:0/3553198957 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa91c0a2d00 msgr2=0x7fa91c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:34.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.101+0000 7fa9393a2640 1 --2- 172.21.15.107:0/3553198957 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa91c0a2d00 0x7fa91c0a51c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa928009930 tx=0x7fa928008040 comp rx=0 tx=0).stop 2024-04-25T00:27:34.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.101+0000 7fa9393a2640 1 -- 172.21.15.107:0/3553198957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa934107720 msgr2=0x7fa93419c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:34.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.101+0000 7fa9393a2640 1 --2- 172.21.15.107:0/3553198957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa934107720 0x7fa93419c9f0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fa92c006010 tx=0x7fa92c003c20 comp rx=0 tx=0).stop 2024-04-25T00:27:34.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.101+0000 7fa9393a2640 1 -- 172.21.15.107:0/3553198957 shutdown_connections 2024-04-25T00:27:34.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.101+0000 7fa9393a2640 1 --2- 172.21.15.107:0/3553198957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa934107720 0x7fa93419c9f0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:34.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.101+0000 7fa9393a2640 1 --2- 172.21.15.107:0/3553198957 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa9341020d0 0x7fa934198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:34.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.101+0000 7fa9393a2640 1 --2- 172.21.15.107:0/3553198957 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fa91c0a2d00 0x7fa91c0a51c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:34.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.101+0000 7fa9393a2640 1 --2- 172.21.15.107:0/3553198957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa934101710 0x7fa934198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:34.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.101+0000 7fa9393a2640 1 -- 172.21.15.107:0/3553198957 >> 172.21.15.107:0/3553198957 conn(0x7fa934078b90 msgr2=0x7fa9340ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:34.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.101+0000 7fa9393a2640 1 -- 172.21.15.107:0/3553198957 shutdown_connections 2024-04-25T00:27:34.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.102+0000 7fa9393a2640 1 -- 172.21.15.107:0/3553198957 wait complete. 2024-04-25T00:27:34.117 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T00:27:34.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.220+0000 7fbd3dd3b640 1 -- 172.21.15.107:0/2465288551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd381029c0 msgr2=0x7fbd38102da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:34.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.220+0000 7fbd3dd3b640 1 --2- 172.21.15.107:0/2465288551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd381029c0 0x7fbd38102da0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fbd28009ca0 tx=0x7fbd2802a9e0 comp rx=0 tx=0).stop 2024-04-25T00:27:34.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.220+0000 7fbd3dd3b640 1 -- 172.21.15.107:0/2465288551 shutdown_connections 2024-04-25T00:27:34.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.220+0000 7fbd3dd3b640 1 --2- 172.21.15.107:0/2465288551 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd3810bd00 0x7fbd3810e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:34.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.220+0000 7fbd3dd3b640 1 --2- 172.21.15.107:0/2465288551 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd381032e0 0x7fbd3810b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:34.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.220+0000 7fbd3dd3b640 1 --2- 172.21.15.107:0/2465288551 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd381029c0 0x7fbd38102da0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:34.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.220+0000 7fbd3dd3b640 1 -- 172.21.15.107:0/2465288551 >> 172.21.15.107:0/2465288551 conn(0x7fbd38077740 msgr2=0x7fbd38077b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:34.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.220+0000 7fbd3dd3b640 1 -- 172.21.15.107:0/2465288551 shutdown_connections 2024-04-25T00:27:34.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.220+0000 7fbd3dd3b640 1 -- 172.21.15.107:0/2465288551 wait complete. 2024-04-25T00:27:34.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.220+0000 7fbd3dd3b640 1 Processor -- start 2024-04-25T00:27:34.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.221+0000 7fbd3dd3b640 1 -- start start 2024-04-25T00:27:34.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.221+0000 7fbd3dd3b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd381029c0 0x7fbd38100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:34.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.221+0000 7fbd3dd3b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd381032e0 0x7fbd38100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:34.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.221+0000 7fbd3dd3b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd3810bd00 0x7fbd38101090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.221+0000 7fbd3dd3b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbd38111bc0 con 0x7fbd381032e0 2024-04-25T00:27:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.221+0000 7fbd3dd3b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbd38111a40 con 0x7fbd381029c0 2024-04-25T00:27:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.221+0000 7fbd3dd3b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 -- 0x7fbd38111d40 con 0x7fbd3810bd00 2024-04-25T00:27:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.221+0000 7fbd36ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd381032e0 0x7fbd38100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.221+0000 7fbd36ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd381032e0 0x7fbd38100b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58030/0 (socket says 172.21.15.107:58030) 2024-04-25T00:27:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.221+0000 7fbd36ffd640 1 -- 172.21.15.107:0/1224974196 learned_addr learned my addr 172.21.15.107:0/1224974196 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T00:27:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.221+0000 7fbd377fe640 1 --2- 172.21.15.107:0/1224974196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd381029c0 0x7fbd38100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.221+0000 7fbd37fff640 1 --2- 172.21.15.107:0/1224974196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd3810bd00 0x7fbd38101090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.221+0000 7fbd36ffd640 1 -- 172.21.15.107:0/1224974196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd3810bd00 msgr2=0x7fbd38101090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.221+0000 7fbd36ffd640 1 --2- 172.21.15.107:0/1224974196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd3810bd00 0x7fbd38101090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:34.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.221+0000 7fbd36ffd640 1 -- 172.21.15.107:0/1224974196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd381029c0 msgr2=0x7fbd38100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:34.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.221+0000 7fbd36ffd640 1 --2- 172.21.15.107:0/1224974196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd381029c0 0x7fbd38100610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:34.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.221+0000 7fbd36ffd640 1 -- 172.21.15.107:0/1224974196 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd381a6680 con 0x7fbd381032e0 2024-04-25T00:27:34.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.221+0000 7fbd36ffd640 1 --2- 172.21.15.107:0/1224974196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd381032e0 0x7fbd38100b50 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000b4a0 tx=0x7fbd2000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:34.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.222+0000 7fbd34ff9640 1 -- 172.21.15.107:0/1224974196 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd20014070 con 0x7fbd381032e0 2024-04-25T00:27:34.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.222+0000 7fbd3dd3b640 1 -- 172.21.15.107:0/1224974196 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd381a6970 con 0x7fbd381032e0 2024-04-25T00:27:34.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.222+0000 7fbd34ff9640 1 -- 172.21.15.107:0/1224974196 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd20010af0 con 0x7fbd381032e0 2024-04-25T00:27:34.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.222+0000 7fbd34ff9640 1 -- 172.21.15.107:0/1224974196 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd20017d10 con 0x7fbd381032e0 2024-04-25T00:27:34.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.222+0000 7fbd3dd3b640 1 -- 172.21.15.107:0/1224974196 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd381a6ec0 con 0x7fbd381032e0 2024-04-25T00:27:34.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.223+0000 7fbd3dd3b640 1 -- 172.21.15.107:0/1224974196 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd38068560 con 0x7fbd381032e0 2024-04-25T00:27:34.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.224+0000 7fbd34ff9640 1 -- 172.21.15.107:0/1224974196 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 25) ==== 137965+0+0 (secure 0 0 0) 0x7fbd20017070 con 0x7fbd381032e0 2024-04-25T00:27:34.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.224+0000 7fbd34ff9640 1 --2- 172.21.15.107:0/1224974196 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbd240a2d00 0x7fbd240a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:34.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.224+0000 7fbd377fe640 1 --2- 172.21.15.107:0/1224974196 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbd240a2d00 0x7fbd240a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:34.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.224+0000 7fbd34ff9640 1 -- 172.21.15.107:0/1224974196 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8561+0+0 (secure 0 0 0) 0x7fbd200cebb0 con 0x7fbd381032e0 2024-04-25T00:27:34.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.225+0000 7fbd377fe640 1 --2- 172.21.15.107:0/1224974196 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbd240a2d00 0x7fbd240a51c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbd28000f80 tx=0x7fbd2803a040 comp rx=0 tx=0).ready entity=mgr.15456 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:34.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.228+0000 7fbd34ff9640 1 -- 172.21.15.107:0/1224974196 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fbd2009b850 con 0x7fbd381032e0 2024-04-25T00:27:34.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:34.404+0000 7fbd3dd3b640 1 -- 172.21.15.107:0/1224974196 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbd38101b40 con 0x7fbd381032e0 2024-04-25T00:27:34.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:34 smithi170 ceph-mon[32807]: pgmap v347: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 921 B/s wr, 6 op/s 2024-04-25T00:27:34.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:34 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3467056716' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T00:27:34.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:34 smithi170 ceph-mon[32807]: osdmap e109: 12 total, 12 up, 12 in 2024-04-25T00:27:34.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:34 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3553198957' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:27:34.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:34 smithi107 ceph-mon[29153]: pgmap v347: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 921 B/s wr, 6 op/s 2024-04-25T00:27:34.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:34 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3467056716' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T00:27:34.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:34 smithi107 ceph-mon[29153]: osdmap e109: 12 total, 12 up, 12 in 2024-04-25T00:27:34.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:34 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3553198957' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:27:34.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:34 smithi195 ceph-mon[32833]: pgmap v347: 129 pgs: 129 active+clean; 191 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 921 B/s wr, 6 op/s 2024-04-25T00:27:34.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:34 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3467056716' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T00:27:34.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:34 smithi195 ceph-mon[32833]: osdmap e109: 12 total, 12 up, 12 in 2024-04-25T00:27:34.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:34 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3553198957' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:27:35.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:35.103+0000 7fbd34ff9640 1 -- 172.21.15.107:0/1224974196 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v111) ==== 178+0+0 (secure 0 0 0) 0x7fbd2009b150 con 0x7fbd381032e0 2024-04-25T00:27:35.104 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_data_ec' removed 2024-04-25T00:27:35.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:35.105+0000 7fbd3dd3b640 1 -- 172.21.15.107:0/1224974196 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbd240a2d00 msgr2=0x7fbd240a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:35.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:35.105+0000 7fbd3dd3b640 1 --2- 172.21.15.107:0/1224974196 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbd240a2d00 0x7fbd240a51c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbd28000f80 tx=0x7fbd2803a040 comp rx=0 tx=0).stop 2024-04-25T00:27:35.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:35.105+0000 7fbd3dd3b640 1 -- 172.21.15.107:0/1224974196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd381032e0 msgr2=0x7fbd38100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:35.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:35.105+0000 7fbd3dd3b640 1 --2- 172.21.15.107:0/1224974196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd381032e0 0x7fbd38100b50 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000b4a0 tx=0x7fbd2000b960 comp rx=0 tx=0).stop 2024-04-25T00:27:35.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:35.105+0000 7fbd3dd3b640 1 -- 172.21.15.107:0/1224974196 shutdown_connections 2024-04-25T00:27:35.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:35.105+0000 7fbd3dd3b640 1 --2- 172.21.15.107:0/1224974196 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbd3810bd00 0x7fbd38101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:35.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:35.105+0000 7fbd3dd3b640 1 --2- 172.21.15.107:0/1224974196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd381032e0 0x7fbd38100b50 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:35.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:35.105+0000 7fbd3dd3b640 1 --2- 172.21.15.107:0/1224974196 >> [v2:172.21.15.107:6800/1021104172,v1:172.21.15.107:6801/1021104172] conn(0x7fbd240a2d00 0x7fbd240a51c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:35.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:35.105+0000 7fbd3dd3b640 1 --2- 172.21.15.107:0/1224974196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd381029c0 0x7fbd38100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:35.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:35.105+0000 7fbd3dd3b640 1 -- 172.21.15.107:0/1224974196 >> 172.21.15.107:0/1224974196 conn(0x7fbd38077740 msgr2=0x7fbd380fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:35.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:35.105+0000 7fbd3dd3b640 1 -- 172.21.15.107:0/1224974196 shutdown_connections 2024-04-25T00:27:35.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T00:27:35.105+0000 7fbd3dd3b640 1 -- 172.21.15.107:0/1224974196 wait complete. 2024-04-25T00:27:35.121 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:27:35.149 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:27:35.181 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:27:35.210 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T00:27:35.210 DEBUG:teuthology.orchestra.run.smithi107:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T00:27:35.226 DEBUG:teuthology.orchestra.run.smithi170:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T00:27:35.241 DEBUG:teuthology.orchestra.run.smithi195:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T00:27:35.266 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T00:27:35.266 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T00:27:35.266 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@mon.a 2024-04-25T00:27:35.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:35 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/3553198957' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T00:27:35.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:35 smithi170 ceph-mon[32807]: osdmap e110: 12 total, 12 up, 12 in 2024-04-25T00:27:35.435 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:35 smithi170 ceph-mon[32807]: from='client.? 172.21.15.107:0/1224974196' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:27:35.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:35 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/3553198957' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T00:27:35.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:35 smithi107 ceph-mon[29153]: osdmap e110: 12 total, 12 up, 12 in 2024-04-25T00:27:35.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:35 smithi107 ceph-mon[29153]: from='client.? 172.21.15.107:0/1224974196' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:27:35.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:35 smithi107 systemd[1]: Stopping Ceph mon.a for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:27:35.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:35 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/3553198957' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T00:27:35.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:35 smithi195 ceph-mon[32833]: osdmap e110: 12 total, 12 up, 12 in 2024-04-25T00:27:35.589 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:35 smithi195 ceph-mon[32833]: from='client.? 172.21.15.107:0/1224974196' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:27:35.929 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:35 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-a[29149]: 2024-04-25T00:27:35.562+0000 7fd78eac4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:27:35.929 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:35 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-a[29149]: 2024-04-25T00:27:35.562+0000 7fd78eac4700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T00:27:35.929 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:35 smithi107 podman[95275]: 2024-04-25 00:27:35.614047788 +0000 UTC m=+0.162685089 container died 8662034e7721df30bfaf633987d37d05c01485dfd9cce32ba3dbfc5ca1c54ed2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T00:27:36.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:36 smithi107 podman[95275]: 2024-04-25 00:27:36.063096376 +0000 UTC m=+0.611733668 container cleanup 8662034e7721df30bfaf633987d37d05c01485dfd9cce32ba3dbfc5ca1c54ed2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:27:36.234 INFO:journalctl@ceph.mon.a.smithi107.stdout:Apr 25 00:27:36 smithi107 bash[95275]: ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-a 2024-04-25T00:27:36.519 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T00:27:36.575 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:27:36.575 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T00:27:36.575 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T00:27:36.575 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@mon.b 2024-04-25T00:27:36.900 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:36 smithi170 systemd[1]: Stopping Ceph mon.b for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:27:37.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:36 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-b[32803]: 2024-04-25T00:27:36.899+0000 7f76a9bce700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:27:37.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:36 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-b[32803]: 2024-04-25T00:27:36.899+0000 7f76a9bce700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T00:27:37.185 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:37 smithi170 podman[68178]: 2024-04-25 00:27:37.134273445 +0000 UTC m=+0.361748895 container died 21037d33514913d2b774353f386998aa34b2df3702c6492f51e3d224e71a9ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T00:27:37.684 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:37 smithi170 podman[68178]: 2024-04-25 00:27:37.508943379 +0000 UTC m=+0.736418818 container cleanup 21037d33514913d2b774353f386998aa34b2df3702c6492f51e3d224e71a9ca1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-25T00:27:37.685 INFO:journalctl@ceph.mon.b.smithi170.stdout:Apr 25 00:27:37 smithi170 bash[68178]: ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-b 2024-04-25T00:27:37.894 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T00:27:37.939 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:27:37.940 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T00:27:37.940 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T00:27:37.940 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@mon.c 2024-04-25T00:27:38.253 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:37 smithi195 systemd[1]: Stopping Ceph mon.c for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:27:38.579 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:38 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-c[32829]: 2024-04-25T00:27:38.251+0000 7f2944b5e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:27:38.579 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:38 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-c[32829]: 2024-04-25T00:27:38.251+0000 7f2944b5e700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T00:27:38.579 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:38 smithi195 podman[69758]: 2024-04-25 00:27:38.324957761 +0000 UTC m=+0.192958430 container died a6bea1777566720a83b270ead10994456067e5c708e376c45c3def7866f39070 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T00:27:38.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:38 smithi195 podman[69758]: 2024-04-25 00:27:38.654334887 +0000 UTC m=+0.522335599 container cleanup a6bea1777566720a83b270ead10994456067e5c708e376c45c3def7866f39070 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-c, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_CLEAN=True) 2024-04-25T00:27:38.839 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:38 smithi195 bash[69758]: ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-c 2024-04-25T00:27:39.124 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T00:27:39.145 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:38 smithi195 podman[69770]: 2024-04-25 00:27:38.873562805 +0000 UTC m=+0.551766568 container remove a6bea1777566720a83b270ead10994456067e5c708e376c45c3def7866f39070 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mon-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T00:27:39.146 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:39 smithi195 systemd[1]: ceph-79602f58-0297-11ef-bc93-c7b262605968@mon.c.service: Deactivated successfully. 2024-04-25T00:27:39.146 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:39 smithi195 systemd[1]: Stopped Ceph mon.c for 79602f58-0297-11ef-bc93-c7b262605968. 2024-04-25T00:27:39.146 INFO:journalctl@ceph.mon.c.smithi195.stdout:Apr 25 00:27:39 smithi195 systemd[1]: ceph-79602f58-0297-11ef-bc93-c7b262605968@mon.c.service: Consumed 13.218s CPU time. 2024-04-25T00:27:39.173 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:27:39.173 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T00:27:39.173 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T00:27:39.173 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@mgr.x 2024-04-25T00:27:39.484 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:27:39 smithi107 systemd[1]: Stopping Ceph mgr.x for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:27:39.928 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:27:39 smithi107 podman[95485]: 2024-04-25 00:27:39.64036883 +0000 UTC m=+0.274364061 container died 081648f9ff72297e7f440f7ab3000caaefe90cb202b9a4d6ae28841084bc278c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True) 2024-04-25T00:27:40.234 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:27:40 smithi107 podman[95485]: 2024-04-25 00:27:40.152269722 +0000 UTC m=+0.786264955 container cleanup 081648f9ff72297e7f440f7ab3000caaefe90cb202b9a4d6ae28841084bc278c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T00:27:40.234 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Apr 25 00:27:40 smithi107 bash[95485]: ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-x 2024-04-25T00:27:40.589 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T00:27:40.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:27:40.623 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T00:27:40.623 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T00:27:40.623 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@mgr.y 2024-04-25T00:27:40.935 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:27:40 smithi170 systemd[1]: Stopping Ceph mgr.y for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:27:41.256 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:27:40 smithi170 podman[68301]: 2024-04-25 00:27:40.976498061 +0000 UTC m=+0.161790954 container died 655ed70eabc80bf95f6219a8040b8130c30956292630472dd34950ae477b220c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T00:27:41.608 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:27:41 smithi170 podman[68301]: 2024-04-25 00:27:41.373006773 +0000 UTC m=+0.558299673 container cleanup 655ed70eabc80bf95f6219a8040b8130c30956292630472dd34950ae477b220c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T00:27:41.608 INFO:journalctl@ceph.mgr.y.smithi170.stdout:Apr 25 00:27:41 smithi170 bash[68301]: ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-y 2024-04-25T00:27:41.755 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T00:27:41.789 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:27:41.789 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T00:27:41.789 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T00:27:41.789 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@mgr.z 2024-04-25T00:27:42.089 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:27:41 smithi195 systemd[1]: Stopping Ceph mgr.z for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:27:42.418 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:27:42 smithi195 podman[69879]: 2024-04-25 00:27:42.112278926 +0000 UTC m=+0.135247735 container died 52ec1f7c266ae7c2707d8caafa86c2e8efd88dd45998dda67259f6c5ac40c96e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD) 2024-04-25T00:27:42.711 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:27:42 smithi195 podman[69879]: 2024-04-25 00:27:42.535084922 +0000 UTC m=+0.558053738 container cleanup 52ec1f7c266ae7c2707d8caafa86c2e8efd88dd45998dda67259f6c5ac40c96e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9) 2024-04-25T00:27:42.712 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:27:42 smithi195 bash[69879]: ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z 2024-04-25T00:27:42.943 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T00:27:42.963 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:27:42 smithi195 podman[69893]: 2024-04-25 00:27:42.711795939 +0000 UTC m=+0.594874978 container remove 52ec1f7c266ae7c2707d8caafa86c2e8efd88dd45998dda67259f6c5ac40c96e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T00:27:42.963 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:27:42 smithi195 systemd[1]: ceph-79602f58-0297-11ef-bc93-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-04-25T00:27:42.963 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:27:42 smithi195 systemd[1]: ceph-79602f58-0297-11ef-bc93-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-04-25T00:27:42.963 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:27:42 smithi195 systemd[1]: Stopped Ceph mgr.z for 79602f58-0297-11ef-bc93-c7b262605968. 2024-04-25T00:27:42.963 INFO:journalctl@ceph.mgr.z.smithi195.stdout:Apr 25 00:27:42 smithi195 systemd[1]: ceph-79602f58-0297-11ef-bc93-c7b262605968@mgr.z.service: Consumed 20.437s CPU time. 2024-04-25T00:27:42.975 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:27:42.976 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T00:27:42.976 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T00:27:42.976 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.0 2024-04-25T00:27:43.357 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 25 00:27:43 smithi107 systemd[1]: Stopping Ceph osd.0 for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:27:43.734 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 25 00:27:43 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-0[41253]: 2024-04-25T00:27:43.356+0000 7f4ce578c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:27:43.734 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 25 00:27:43 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-0[41253]: 2024-04-25T00:27:43.356+0000 7f4ce578c700 -1 osd.0 111 *** Got signal Terminated *** 2024-04-25T00:27:43.734 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 25 00:27:43 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-0[41253]: 2024-04-25T00:27:43.356+0000 7f4ce578c700 -1 osd.0 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:27:49.207 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 25 00:27:48 smithi107 podman[95608]: 2024-04-25 00:27:48.917074114 +0000 UTC m=+5.748340660 container died 0196ae328dc93ed1fcf490b8963b37c2520af75c422358383e8e083d2c89e74d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:27:49.484 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 25 00:27:49 smithi107 podman[95608]: 2024-04-25 00:27:49.298742908 +0000 UTC m=+6.130009455 container cleanup 0196ae328dc93ed1fcf490b8963b37c2520af75c422358383e8e083d2c89e74d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T00:27:49.484 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 25 00:27:49 smithi107 bash[95608]: ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-0 2024-04-25T00:27:49.859 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 25 00:27:49 smithi107 podman[95625]: 2024-04-25 00:27:49.550796184 +0000 UTC m=+0.628133142 container remove 0196ae328dc93ed1fcf490b8963b37c2520af75c422358383e8e083d2c89e74d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T00:27:50.179 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 25 00:27:49 smithi107 podman[95714]: 2024-04-25 00:27:49.860104297 +0000 UTC m=+0.103017088 container create bd4c9a09eb4809f7fbc73416619b6269df3636467984c53eb00ce9bbddf3fdfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424) 2024-04-25T00:27:50.179 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 25 00:27:49 smithi107 podman[95714]: 2024-04-25 00:27:49.779138242 +0000 UTC m=+0.022051019 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:27:50.179 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 25 00:27:50 smithi107 podman[95714]: 2024-04-25 00:27:50.096475548 +0000 UTC m=+0.339388308 container init bd4c9a09eb4809f7fbc73416619b6269df3636467984c53eb00ce9bbddf3fdfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:27:50.179 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 25 00:27:50 smithi107 podman[95714]: 2024-04-25 00:27:50.099797725 +0000 UTC m=+0.342710475 container start bd4c9a09eb4809f7fbc73416619b6269df3636467984c53eb00ce9bbddf3fdfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T00:27:50.179 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 25 00:27:50 smithi107 podman[95714]: 2024-04-25 00:27:50.179352618 +0000 UTC m=+0.422265369 container attach bd4c9a09eb4809f7fbc73416619b6269df3636467984c53eb00ce9bbddf3fdfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T00:27:50.484 INFO:journalctl@ceph.osd.0.smithi107.stdout:Apr 25 00:27:50 smithi107 podman[95714]: 2024-04-25 00:27:50.388002251 +0000 UTC m=+0.630915010 container died bd4c9a09eb4809f7fbc73416619b6269df3636467984c53eb00ce9bbddf3fdfb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:27:51.053 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T00:27:51.095 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:27:51.095 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T00:27:51.095 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T00:27:51.095 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.1 2024-04-25T00:27:51.334 INFO:journalctl@ceph.osd.1.smithi170.stdout:Apr 25 00:27:51 smithi170 systemd[1]: Stopping Ceph osd.1 for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:27:51.685 INFO:journalctl@ceph.osd.1.smithi170.stdout:Apr 25 00:27:51 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-1[37418]: 2024-04-25T00:27:51.418+0000 7f09e7a1f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:27:51.685 INFO:journalctl@ceph.osd.1.smithi170.stdout:Apr 25 00:27:51 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-1[37418]: 2024-04-25T00:27:51.418+0000 7f09e7a1f700 -1 osd.1 111 *** Got signal Terminated *** 2024-04-25T00:27:51.685 INFO:journalctl@ceph.osd.1.smithi170.stdout:Apr 25 00:27:51 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-1[37418]: 2024-04-25T00:27:51.418+0000 7f09e7a1f700 -1 osd.1 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:27:57.045 INFO:journalctl@ceph.osd.1.smithi170.stdout:Apr 25 00:27:56 smithi170 podman[68429]: 2024-04-25 00:27:56.769318913 +0000 UTC m=+5.479264228 container died 49eca87544e9b2d495390961be899884b25a76a04bf9260663ba60306bdf420d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-1, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T00:27:57.429 INFO:journalctl@ceph.osd.1.smithi170.stdout:Apr 25 00:27:57 smithi170 podman[68429]: 2024-04-25 00:27:57.144334209 +0000 UTC m=+5.854279554 container cleanup 49eca87544e9b2d495390961be899884b25a76a04bf9260663ba60306bdf420d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-1, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T00:27:57.429 INFO:journalctl@ceph.osd.1.smithi170.stdout:Apr 25 00:27:57 smithi170 bash[68429]: ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-1 2024-04-25T00:27:57.684 INFO:journalctl@ceph.osd.1.smithi170.stdout:Apr 25 00:27:57 smithi170 podman[68457]: 2024-04-25 00:27:57.429280296 +0000 UTC m=+0.658285956 container remove 49eca87544e9b2d495390961be899884b25a76a04bf9260663ba60306bdf420d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-1, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:27:58.058 INFO:journalctl@ceph.osd.1.smithi170.stdout:Apr 25 00:27:57 smithi170 podman[68549]: 2024-04-25 00:27:57.623259315 +0000 UTC m=+0.017724326 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:27:58.058 INFO:journalctl@ceph.osd.1.smithi170.stdout:Apr 25 00:27:57 smithi170 podman[68549]: 2024-04-25 00:27:57.747155125 +0000 UTC m=+0.141620134 container create b97d5cc6b884e89cb8816b0d907a77da57e7776b79430ab69bcb26094283d975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:27:58.058 INFO:journalctl@ceph.osd.1.smithi170.stdout:Apr 25 00:27:57 smithi170 podman[68549]: 2024-04-25 00:27:57.958213413 +0000 UTC m=+0.352678423 container init b97d5cc6b884e89cb8816b0d907a77da57e7776b79430ab69bcb26094283d975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T00:27:58.058 INFO:journalctl@ceph.osd.1.smithi170.stdout:Apr 25 00:27:57 smithi170 podman[68549]: 2024-04-25 00:27:57.961202107 +0000 UTC m=+0.355667117 container start b97d5cc6b884e89cb8816b0d907a77da57e7776b79430ab69bcb26094283d975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-25T00:27:58.058 INFO:journalctl@ceph.osd.1.smithi170.stdout:Apr 25 00:27:58 smithi170 podman[68549]: 2024-04-25 00:27:58.058144866 +0000 UTC m=+0.452609902 container attach b97d5cc6b884e89cb8816b0d907a77da57e7776b79430ab69bcb26094283d975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_CLEAN=True) 2024-04-25T00:27:58.435 INFO:journalctl@ceph.osd.1.smithi170.stdout:Apr 25 00:27:58 smithi170 podman[68549]: 2024-04-25 00:27:58.254067963 +0000 UTC m=+0.648532980 container died b97d5cc6b884e89cb8816b0d907a77da57e7776b79430ab69bcb26094283d975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T00:27:58.884 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T00:27:58.922 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:27:58.922 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T00:27:58.922 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T00:27:58.922 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.2 2024-04-25T00:27:59.248 INFO:journalctl@ceph.osd.2.smithi195.stdout:Apr 25 00:27:58 smithi195 systemd[1]: Stopping Ceph osd.2 for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:27:59.589 INFO:journalctl@ceph.osd.2.smithi195.stdout:Apr 25 00:27:59 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-2[38010]: 2024-04-25T00:27:59.247+0000 7f462b19c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:27:59.589 INFO:journalctl@ceph.osd.2.smithi195.stdout:Apr 25 00:27:59 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-2[38010]: 2024-04-25T00:27:59.247+0000 7f462b19c700 -1 osd.2 111 *** Got signal Terminated *** 2024-04-25T00:27:59.589 INFO:journalctl@ceph.osd.2.smithi195.stdout:Apr 25 00:27:59 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-2[38010]: 2024-04-25T00:27:59.247+0000 7f462b19c700 -1 osd.2 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:28:04.982 INFO:journalctl@ceph.osd.2.smithi195.stdout:Apr 25 00:28:04 smithi195 podman[70016]: 2024-04-25 00:28:04.692808201 +0000 UTC m=+5.565401756 container died 4b76fac40e5e5fd842c66ff968bd4ca951c5f6a43267fbfa0c5fc9c3f32ae173 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-25T00:28:05.339 INFO:journalctl@ceph.osd.2.smithi195.stdout:Apr 25 00:28:05 smithi195 podman[70016]: 2024-04-25 00:28:05.082054648 +0000 UTC m=+5.954648204 container cleanup 4b76fac40e5e5fd842c66ff968bd4ca951c5f6a43267fbfa0c5fc9c3f32ae173 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T00:28:05.339 INFO:journalctl@ceph.osd.2.smithi195.stdout:Apr 25 00:28:05 smithi195 bash[70016]: ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-2 2024-04-25T00:28:05.694 INFO:journalctl@ceph.osd.2.smithi195.stdout:Apr 25 00:28:05 smithi195 podman[70036]: 2024-04-25 00:28:05.341762515 +0000 UTC m=+0.647352219 container remove 4b76fac40e5e5fd842c66ff968bd4ca951c5f6a43267fbfa0c5fc9c3f32ae173 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-2, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-25T00:28:05.945 INFO:journalctl@ceph.osd.2.smithi195.stdout:Apr 25 00:28:05 smithi195 podman[70127]: 2024-04-25 00:28:05.597215294 +0000 UTC m=+0.018071292 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:28:05.945 INFO:journalctl@ceph.osd.2.smithi195.stdout:Apr 25 00:28:05 smithi195 podman[70127]: 2024-04-25 00:28:05.701066719 +0000 UTC m=+0.121922759 container create ce516aa8528d782e29cc3b2c5b1848547f02b0e2872b48eca1317e0e52529a1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424) 2024-04-25T00:28:06.237 INFO:journalctl@ceph.osd.2.smithi195.stdout:Apr 25 00:28:05 smithi195 podman[70127]: 2024-04-25 00:28:05.945484316 +0000 UTC m=+0.366340314 container init ce516aa8528d782e29cc3b2c5b1848547f02b0e2872b48eca1317e0e52529a1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:28:06.237 INFO:journalctl@ceph.osd.2.smithi195.stdout:Apr 25 00:28:05 smithi195 podman[70127]: 2024-04-25 00:28:05.94869487 +0000 UTC m=+0.369550878 container start ce516aa8528d782e29cc3b2c5b1848547f02b0e2872b48eca1317e0e52529a1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:28:06.237 INFO:journalctl@ceph.osd.2.smithi195.stdout:Apr 25 00:28:06 smithi195 podman[70127]: 2024-04-25 00:28:06.053897257 +0000 UTC m=+0.474753274 container attach ce516aa8528d782e29cc3b2c5b1848547f02b0e2872b48eca1317e0e52529a1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:28:06.507 INFO:journalctl@ceph.osd.2.smithi195.stdout:Apr 25 00:28:06 smithi195 podman[70127]: 2024-04-25 00:28:06.237264081 +0000 UTC m=+0.658120090 container died ce516aa8528d782e29cc3b2c5b1848547f02b0e2872b48eca1317e0e52529a1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424) 2024-04-25T00:28:06.922 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T00:28:06.959 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:28:06.959 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T00:28:06.959 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T00:28:06.959 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.3 2024-04-25T00:28:07.268 INFO:journalctl@ceph.osd.3.smithi107.stdout:Apr 25 00:28:07 smithi107 systemd[1]: Stopping Ceph osd.3 for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:28:07.734 INFO:journalctl@ceph.osd.3.smithi107.stdout:Apr 25 00:28:07 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-3[47784]: 2024-04-25T00:28:07.267+0000 7fccaac23700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:28:07.734 INFO:journalctl@ceph.osd.3.smithi107.stdout:Apr 25 00:28:07 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-3[47784]: 2024-04-25T00:28:07.267+0000 7fccaac23700 -1 osd.3 111 *** Got signal Terminated *** 2024-04-25T00:28:07.734 INFO:journalctl@ceph.osd.3.smithi107.stdout:Apr 25 00:28:07 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-3[47784]: 2024-04-25T00:28:07.267+0000 7fccaac23700 -1 osd.3 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:28:10.484 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:10 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:10.169+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:10.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:10 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:10.288+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:11.435 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:10 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:10.992+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:11.484 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:11 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:11.153+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:11.484 INFO:journalctl@ceph.osd.3.smithi107.stdout:Apr 25 00:28:11 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-3[47784]: 2024-04-25T00:28:11.304+0000 7fcca320f700 -1 osd.3 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.203372+0000 front 2024-04-25T00:27:48.203486+0000 (oldest deadline 2024-04-25T00:28:11.103487+0000) 2024-04-25T00:28:11.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:11 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:11.331+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:12.435 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:12 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:12.007+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:12.484 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:12 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:12.148+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:12.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:12 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:12.355+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:12.984 INFO:journalctl@ceph.osd.3.smithi107.stdout:Apr 25 00:28:12 smithi107 podman[95831]: 2024-04-25 00:28:12.650812378 +0000 UTC m=+5.495175439 container died 0029e29e7d9fe596343186d205c9660e31db0ea327d9f12e0a138e8e7d318402 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-04-25T00:28:13.311 INFO:journalctl@ceph.osd.3.smithi107.stdout:Apr 25 00:28:13 smithi107 podman[95831]: 2024-04-25 00:28:13.101046807 +0000 UTC m=+5.945409868 container cleanup 0029e29e7d9fe596343186d205c9660e31db0ea327d9f12e0a138e8e7d318402 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-3, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T00:28:13.311 INFO:journalctl@ceph.osd.3.smithi107.stdout:Apr 25 00:28:13 smithi107 bash[95831]: ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-3 2024-04-25T00:28:13.312 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:13 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:13.111+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:13.434 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:13 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:13.043+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:13.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:13 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:13.316+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:13.589 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:13 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:13.294+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.625483+0000 front 2024-04-25T00:27:47.625589+0000 (oldest deadline 2024-04-25T00:28:12.925478+0000) 2024-04-25T00:28:13.711 INFO:journalctl@ceph.osd.3.smithi107.stdout:Apr 25 00:28:13 smithi107 podman[95849]: 2024-04-25 00:28:13.311165078 +0000 UTC m=+0.653228781 container remove 0029e29e7d9fe596343186d205c9660e31db0ea327d9f12e0a138e8e7d318402 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:28:13.984 INFO:journalctl@ceph.osd.3.smithi107.stdout:Apr 25 00:28:13 smithi107 podman[95938]: 2024-04-25 00:28:13.71190817 +0000 UTC m=+0.114593641 container create 148b80f19184b831d044a6258e9d99ebda987419acc1b861ab9e6a7f65ddff61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T00:28:13.984 INFO:journalctl@ceph.osd.3.smithi107.stdout:Apr 25 00:28:13 smithi107 podman[95938]: 2024-04-25 00:28:13.615423626 +0000 UTC m=+0.018109094 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:28:13.984 INFO:journalctl@ceph.osd.3.smithi107.stdout:Apr 25 00:28:13 smithi107 podman[95938]: 2024-04-25 00:28:13.956242678 +0000 UTC m=+0.358928146 container init 148b80f19184b831d044a6258e9d99ebda987419acc1b861ab9e6a7f65ddff61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T00:28:13.985 INFO:journalctl@ceph.osd.3.smithi107.stdout:Apr 25 00:28:13 smithi107 podman[95938]: 2024-04-25 00:28:13.959276955 +0000 UTC m=+0.361962432 container start 148b80f19184b831d044a6258e9d99ebda987419acc1b861ab9e6a7f65ddff61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T00:28:14.251 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:14 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:14.099+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:14.251 INFO:journalctl@ceph.osd.3.smithi107.stdout:Apr 25 00:28:14 smithi107 podman[95938]: 2024-04-25 00:28:14.056291442 +0000 UTC m=+0.458976929 container attach 148b80f19184b831d044a6258e9d99ebda987419acc1b861ab9e6a7f65ddff61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-25T00:28:14.435 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:14 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:14.055+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:14.524 INFO:journalctl@ceph.osd.3.smithi107.stdout:Apr 25 00:28:14 smithi107 podman[95938]: 2024-04-25 00:28:14.251320982 +0000 UTC m=+0.654006457 container died 148b80f19184b831d044a6258e9d99ebda987419acc1b861ab9e6a7f65ddff61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T00:28:14.589 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:14 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:14.248+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.625483+0000 front 2024-04-25T00:27:47.625589+0000 (oldest deadline 2024-04-25T00:28:12.925478+0000) 2024-04-25T00:28:14.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:14 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:14.302+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:14.870 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T00:28:14.908 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:28:14.908 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T00:28:14.909 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T00:28:14.909 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.4 2024-04-25T00:28:15.234 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:15 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:15.059+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:15.423 INFO:journalctl@ceph.osd.4.smithi170.stdout:Apr 25 00:28:14 smithi170 systemd[1]: Stopping Ceph osd.4 for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:28:15.423 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:15 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:15.015+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:15.589 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:15 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:15.266+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.625483+0000 front 2024-04-25T00:27:47.625589+0000 (oldest deadline 2024-04-25T00:28:12.925478+0000) 2024-04-25T00:28:15.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:15 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:15.278+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:15.685 INFO:journalctl@ceph.osd.4.smithi170.stdout:Apr 25 00:28:15 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-4[43915]: 2024-04-25T00:28:15.422+0000 7fbaf8c91700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:28:15.685 INFO:journalctl@ceph.osd.4.smithi170.stdout:Apr 25 00:28:15 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-4[43915]: 2024-04-25T00:28:15.422+0000 7fbaf8c91700 -1 osd.4 111 *** Got signal Terminated *** 2024-04-25T00:28:15.685 INFO:journalctl@ceph.osd.4.smithi170.stdout:Apr 25 00:28:15 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-4[43915]: 2024-04-25T00:28:15.422+0000 7fbaf8c91700 -1 osd.4 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:28:16.435 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:15 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:15.986+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:16.484 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:16 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:16.089+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:16.589 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:16 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:16.294+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.625483+0000 front 2024-04-25T00:27:47.625589+0000 (oldest deadline 2024-04-25T00:28:12.925478+0000) 2024-04-25T00:28:16.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:16 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:16.247+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:17.434 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:16 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:16.970+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:17.484 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:17 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:17.119+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:17.513 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:17 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:17.216+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:17.513 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:17 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:17.272+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.625483+0000 front 2024-04-25T00:27:47.625589+0000 (oldest deadline 2024-04-25T00:28:12.925478+0000) 2024-04-25T00:28:17.513 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:17 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:17.272+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:52.926015+0000 front 2024-04-25T00:27:52.926045+0000 (oldest deadline 2024-04-25T00:28:17.026186+0000) 2024-04-25T00:28:18.231 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:17 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:17.902+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:18.435 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:17 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:17.945+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:18.484 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:18 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:18.160+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:18.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:18 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:18.231+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:18.589 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:18 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:18.238+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.625483+0000 front 2024-04-25T00:27:47.625589+0000 (oldest deadline 2024-04-25T00:28:12.925478+0000) 2024-04-25T00:28:18.589 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:18 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:18.238+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:52.926015+0000 front 2024-04-25T00:27:52.926045+0000 (oldest deadline 2024-04-25T00:28:17.026186+0000) 2024-04-25T00:28:18.914 INFO:journalctl@ceph.osd.4.smithi170.stdout:Apr 25 00:28:18 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-4[43915]: 2024-04-25T00:28:18.532+0000 7fbaf127d700 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.821506+0000 front 2024-04-25T00:27:55.821528+0000 (oldest deadline 2024-04-25T00:28:18.121635+0000) 2024-04-25T00:28:19.184 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:18 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:18.913+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:19.200 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:18 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:18.916+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:19.484 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:19 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:19.178+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:19.484 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:19 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:19.178+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:19.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:19 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:19.200+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:19.589 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:19 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:19.216+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.625483+0000 front 2024-04-25T00:27:47.625589+0000 (oldest deadline 2024-04-25T00:28:12.925478+0000) 2024-04-25T00:28:19.590 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:19 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:19.216+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:52.926015+0000 front 2024-04-25T00:27:52.926045+0000 (oldest deadline 2024-04-25T00:28:17.026186+0000) 2024-04-25T00:28:19.886 INFO:journalctl@ceph.osd.4.smithi170.stdout:Apr 25 00:28:19 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-4[43915]: 2024-04-25T00:28:19.540+0000 7fbaf127d700 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.821506+0000 front 2024-04-25T00:27:55.821528+0000 (oldest deadline 2024-04-25T00:28:18.121635+0000) 2024-04-25T00:28:19.886 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:19 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:19.650+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:20.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:19 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:19.885+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:20.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:19 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:19.885+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:20.194 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:19 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:19.901+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.235181+0000 front 2024-04-25T00:27:55.235226+0000 (oldest deadline 2024-04-25T00:28:19.335101+0000) 2024-04-25T00:28:20.339 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:19 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:19.945+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:20.339 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:20 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:20.180+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:20.340 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:20 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:20.180+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.625483+0000 front 2024-04-25T00:27:47.625589+0000 (oldest deadline 2024-04-25T00:28:12.925478+0000) 2024-04-25T00:28:20.340 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:20 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:20.180+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:52.926015+0000 front 2024-04-25T00:27:52.926045+0000 (oldest deadline 2024-04-25T00:28:17.026186+0000) 2024-04-25T00:28:20.484 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:20 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:20.193+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:20.484 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:20 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:20.193+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:20.935 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:20 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:20.651+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:20.935 INFO:journalctl@ceph.osd.4.smithi170.stdout:Apr 25 00:28:20 smithi170 podman[68656]: 2024-04-25 00:28:20.874019396 +0000 UTC m=+5.770718172 container died 89fdbcd6a62885c9b3e2017825e2d19c1f34b032b7228b8dbd0da5086f312c27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-4, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T00:28:20.935 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:20 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:20.873+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:20.935 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:20 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:20.873+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:21.204 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:20 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:20.885+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.235181+0000 front 2024-04-25T00:27:55.235226+0000 (oldest deadline 2024-04-25T00:28:19.335101+0000) 2024-04-25T00:28:21.212 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:20 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:20.912+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:21.434 INFO:journalctl@ceph.osd.4.smithi170.stdout:Apr 25 00:28:21 smithi170 podman[68656]: 2024-04-25 00:28:21.223168945 +0000 UTC m=+6.119867725 container cleanup 89fdbcd6a62885c9b3e2017825e2d19c1f34b032b7228b8dbd0da5086f312c27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-4, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD) 2024-04-25T00:28:21.435 INFO:journalctl@ceph.osd.4.smithi170.stdout:Apr 25 00:28:21 smithi170 bash[68656]: ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-4 2024-04-25T00:28:21.484 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:21 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:21.203+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:21.484 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:21 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:21.203+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:21.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:21 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:21.229+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:21.589 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:21 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:21.212+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.625483+0000 front 2024-04-25T00:27:47.625589+0000 (oldest deadline 2024-04-25T00:28:12.925478+0000) 2024-04-25T00:28:21.590 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:21 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:21.212+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:52.926015+0000 front 2024-04-25T00:27:52.926045+0000 (oldest deadline 2024-04-25T00:28:17.026186+0000) 2024-04-25T00:28:21.835 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:21 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:21.647+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:21.836 INFO:journalctl@ceph.osd.4.smithi170.stdout:Apr 25 00:28:21 smithi170 podman[68673]: 2024-04-25 00:28:21.549460823 +0000 UTC m=+0.673571183 container remove 89fdbcd6a62885c9b3e2017825e2d19c1f34b032b7228b8dbd0da5086f312c27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-4, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:28:22.119 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:21 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:21.835+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:22.119 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:21 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:21.835+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:22.120 INFO:journalctl@ceph.osd.4.smithi170.stdout:Apr 25 00:28:21 smithi170 podman[68764]: 2024-04-25 00:28:21.755871471 +0000 UTC m=+0.017922800 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:28:22.120 INFO:journalctl@ceph.osd.4.smithi170.stdout:Apr 25 00:28:21 smithi170 podman[68764]: 2024-04-25 00:28:21.877410901 +0000 UTC m=+0.139462220 container create 47649e905a63e42bdb9ae49c5123ce26bf67656cc96ed9ace6b4ec78c6eb610e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240424, ceph=True, RELEASE=squid-d2c45d9, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:28:22.221 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:21 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:21.919+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.235181+0000 front 2024-04-25T00:27:55.235226+0000 (oldest deadline 2024-04-25T00:28:19.335101+0000) 2024-04-25T00:28:22.234 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:21 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:21.942+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:22.420 INFO:journalctl@ceph.osd.4.smithi170.stdout:Apr 25 00:28:22 smithi170 podman[68764]: 2024-04-25 00:28:22.119866927 +0000 UTC m=+0.381918246 container init 47649e905a63e42bdb9ae49c5123ce26bf67656cc96ed9ace6b4ec78c6eb610e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T00:28:22.420 INFO:journalctl@ceph.osd.4.smithi170.stdout:Apr 25 00:28:22 smithi170 podman[68764]: 2024-04-25 00:28:22.122983704 +0000 UTC m=+0.385035039 container start 47649e905a63e42bdb9ae49c5123ce26bf67656cc96ed9ace6b4ec78c6eb610e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, ceph=True) 2024-04-25T00:28:22.420 INFO:journalctl@ceph.osd.4.smithi170.stdout:Apr 25 00:28:22 smithi170 podman[68764]: 2024-04-25 00:28:22.212122648 +0000 UTC m=+0.474173972 container attach 47649e905a63e42bdb9ae49c5123ce26bf67656cc96ed9ace6b4ec78c6eb610e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-d2c45d9) 2024-04-25T00:28:22.484 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:22 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:22.220+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:22.484 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:22 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:22.220+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:22.589 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:22 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:22.239+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.625483+0000 front 2024-04-25T00:27:47.625589+0000 (oldest deadline 2024-04-25T00:28:12.925478+0000) 2024-04-25T00:28:22.589 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:22 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:22.239+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:52.926015+0000 front 2024-04-25T00:27:52.926045+0000 (oldest deadline 2024-04-25T00:28:17.026186+0000) 2024-04-25T00:28:22.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:22 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:22.234+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:22.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:22 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:22.598+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:22.685 INFO:journalctl@ceph.osd.4.smithi170.stdout:Apr 25 00:28:22 smithi170 podman[68764]: 2024-04-25 00:28:22.419945433 +0000 UTC m=+0.681996756 container died 47649e905a63e42bdb9ae49c5123ce26bf67656cc96ed9ace6b4ec78c6eb610e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:28:23.033 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:22 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:22.846+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:23.034 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:22 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:22.846+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:23.058 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T00:28:23.096 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:28:23.096 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T00:28:23.096 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T00:28:23.097 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.5 2024-04-25T00:28:23.213 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:22 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:22.930+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.235181+0000 front 2024-04-25T00:27:55.235226+0000 (oldest deadline 2024-04-25T00:28:19.335101+0000) 2024-04-25T00:28:23.220 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:23 smithi195 systemd[1]: Stopping Ceph osd.5 for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:28:23.220 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:22 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:22.906+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:23.484 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:23 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:23.213+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:23.484 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:23 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:23.213+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:23.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:23 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:23.219+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:23.590 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:23 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:23.286+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.625483+0000 front 2024-04-25T00:27:47.625589+0000 (oldest deadline 2024-04-25T00:28:12.925478+0000) 2024-04-25T00:28:23.590 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:23 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:23.286+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:52.926015+0000 front 2024-04-25T00:27:52.926045+0000 (oldest deadline 2024-04-25T00:28:17.026186+0000) 2024-04-25T00:28:23.590 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:23 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:23.402+0000 7f7e01029700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:28:23.590 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:23 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:23.403+0000 7f7e01029700 -1 osd.5 111 *** Got signal Terminated *** 2024-04-25T00:28:23.591 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:23 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:23.403+0000 7f7e01029700 -1 osd.5 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:28:23.935 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:23 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:23.579+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:23.935 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:23 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:23.823+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:23.935 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:23 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:23.823+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:24.210 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:23 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:23.882+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.235181+0000 front 2024-04-25T00:27:55.235226+0000 (oldest deadline 2024-04-25T00:28:19.335101+0000) 2024-04-25T00:28:24.228 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:23 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:23.948+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:24.484 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:24 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:24.209+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:24.484 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:24 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:24.209+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:24.589 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:24 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:24.270+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.625483+0000 front 2024-04-25T00:27:47.625589+0000 (oldest deadline 2024-04-25T00:28:12.925478+0000) 2024-04-25T00:28:24.590 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:24 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:24.270+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:52.926015+0000 front 2024-04-25T00:27:52.926045+0000 (oldest deadline 2024-04-25T00:28:17.026186+0000) 2024-04-25T00:28:24.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:24 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:24.227+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:24.935 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:24 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:24.621+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:24.935 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:24 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:24.846+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:24.935 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:24 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:24.846+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:25.228 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:24 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:24.893+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.235181+0000 front 2024-04-25T00:27:55.235226+0000 (oldest deadline 2024-04-25T00:28:19.335101+0000) 2024-04-25T00:28:25.296 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:25 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:25.181+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:25.296 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:24 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:24.944+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:25.484 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:25 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:25.227+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:25.484 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:25 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:25.227+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:25.589 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:25 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:25.295+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.625483+0000 front 2024-04-25T00:27:47.625589+0000 (oldest deadline 2024-04-25T00:28:12.925478+0000) 2024-04-25T00:28:25.589 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:25 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:25.295+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:52.926015+0000 front 2024-04-25T00:27:52.926045+0000 (oldest deadline 2024-04-25T00:28:17.026186+0000) 2024-04-25T00:28:25.589 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:25 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:25.295+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.927167+0000 front 2024-04-25T00:28:02.927192+0000 (oldest deadline 2024-04-25T00:28:25.227198+0000) 2024-04-25T00:28:25.849 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:25 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:25.572+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:26.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:25 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:25.849+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:26.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:25 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:25.849+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:26.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:25 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:25.849+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:26.234 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:25 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:25.884+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.235181+0000 front 2024-04-25T00:27:55.235226+0000 (oldest deadline 2024-04-25T00:28:19.335101+0000) 2024-04-25T00:28:26.234 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:25 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:25.884+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:27:59.335636+0000 front 2024-04-25T00:27:59.335677+0000 (oldest deadline 2024-04-25T00:28:25.136168+0000) 2024-04-25T00:28:26.274 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:26 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:26.146+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:26.274 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:26 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:26.146+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:26.274 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:25 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:25.949+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:26.589 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:26 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:26.273+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.625483+0000 front 2024-04-25T00:27:47.625589+0000 (oldest deadline 2024-04-25T00:28:12.925478+0000) 2024-04-25T00:28:26.589 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:26 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:26.273+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:52.926015+0000 front 2024-04-25T00:27:52.926045+0000 (oldest deadline 2024-04-25T00:28:17.026186+0000) 2024-04-25T00:28:26.589 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:26 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:26.273+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.927167+0000 front 2024-04-25T00:28:02.927192+0000 (oldest deadline 2024-04-25T00:28:25.227198+0000) 2024-04-25T00:28:26.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:26 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:26.236+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:26.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:26 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:26.236+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:26.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:26 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:26.236+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:26.872 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:26 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:26.552+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:27.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:26 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:26.871+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:27.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:26 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:26.871+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:27.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:26 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:26.871+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:27.234 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:26 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:26.851+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.235181+0000 front 2024-04-25T00:27:55.235226+0000 (oldest deadline 2024-04-25T00:28:19.335101+0000) 2024-04-25T00:28:27.234 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:26 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:26.851+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:27:59.335636+0000 front 2024-04-25T00:27:59.335677+0000 (oldest deadline 2024-04-25T00:28:25.136168+0000) 2024-04-25T00:28:27.242 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:26 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:26.917+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:27.242 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:26 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:26.917+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:27.243 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:27 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:27.106+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:27.243 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:27 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:27.106+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:27.589 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:27 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:27.242+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.625483+0000 front 2024-04-25T00:27:47.625589+0000 (oldest deadline 2024-04-25T00:28:12.925478+0000) 2024-04-25T00:28:27.589 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:27 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:27.242+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:52.926015+0000 front 2024-04-25T00:27:52.926045+0000 (oldest deadline 2024-04-25T00:28:17.026186+0000) 2024-04-25T00:28:27.589 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:27 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:27.242+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.927167+0000 front 2024-04-25T00:28:02.927192+0000 (oldest deadline 2024-04-25T00:28:25.227198+0000) 2024-04-25T00:28:27.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:27 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:27.257+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:27.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:27 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:27.257+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:27.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:27 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:27.257+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:27.829 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:27 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:27.561+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:27.829 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:27 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:27.561+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:01.034281+0000 front 2024-04-25T00:28:01.034251+0000 (oldest deadline 2024-04-25T00:28:26.934402+0000) 2024-04-25T00:28:28.186 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:27 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:27.829+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:28.187 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:27 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:27.829+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:28.187 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:27 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:27.829+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:28.234 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:27 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:27.857+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.235181+0000 front 2024-04-25T00:27:55.235226+0000 (oldest deadline 2024-04-25T00:28:19.335101+0000) 2024-04-25T00:28:28.234 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:27 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:27.857+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:27:59.335636+0000 front 2024-04-25T00:27:59.335677+0000 (oldest deadline 2024-04-25T00:28:25.136168+0000) 2024-04-25T00:28:28.252 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:27 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:27.929+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:28.252 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:27 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:27.929+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:28.252 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:28 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:28.111+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:28.252 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:28 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:28.111+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:28.595 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:28 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:28.251+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.625483+0000 front 2024-04-25T00:27:47.625589+0000 (oldest deadline 2024-04-25T00:28:12.925478+0000) 2024-04-25T00:28:28.596 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:28 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:28.251+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:52.926015+0000 front 2024-04-25T00:27:52.926045+0000 (oldest deadline 2024-04-25T00:28:17.026186+0000) 2024-04-25T00:28:28.597 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:28 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5[44822]: 2024-04-25T00:28:28.251+0000 7f7dfd61d700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.927167+0000 front 2024-04-25T00:28:02.927192+0000 (oldest deadline 2024-04-25T00:28:25.227198+0000) 2024-04-25T00:28:28.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:28 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:28.291+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:28.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:28 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:28.291+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:28.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:28 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:28.291+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:28.791 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:28 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:28.522+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:28.791 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:28 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:28.522+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:01.034281+0000 front 2024-04-25T00:28:01.034251+0000 (oldest deadline 2024-04-25T00:28:26.934402+0000) 2024-04-25T00:28:29.072 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:28 smithi195 podman[70243]: 2024-04-25 00:28:28.755005879 +0000 UTC m=+5.468021427 container died b66de97a4bceab847e175d43d7e4137fb1cc8965706a8d01ce5a60154b5170a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:28:29.072 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:28 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:28.968+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:29.072 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:28 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:28.968+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:29.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:28 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:28.791+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:29.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:28 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:28.791+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:29.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:28 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:28.791+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:29.234 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:28 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:28.834+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.235181+0000 front 2024-04-25T00:27:55.235226+0000 (oldest deadline 2024-04-25T00:28:19.335101+0000) 2024-04-25T00:28:29.234 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:28 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:28.834+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:27:59.335636+0000 front 2024-04-25T00:27:59.335677+0000 (oldest deadline 2024-04-25T00:28:25.136168+0000) 2024-04-25T00:28:29.339 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:29 smithi195 podman[70243]: 2024-04-25 00:28:29.170897307 +0000 UTC m=+5.883912851 container cleanup b66de97a4bceab847e175d43d7e4137fb1cc8965706a8d01ce5a60154b5170a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:28:29.340 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:29 smithi195 bash[70243]: ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5 2024-04-25T00:28:29.340 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:29 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:29.146+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:29.340 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:29 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:29.146+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:29.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:29 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:29.294+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:29.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:29 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:29.294+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:29.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:29 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:29.294+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:29.781 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:29 smithi195 podman[70260]: 2024-04-25 00:28:29.380561996 +0000 UTC m=+0.625976915 container remove b66de97a4bceab847e175d43d7e4137fb1cc8965706a8d01ce5a60154b5170a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-25T00:28:29.819 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:29 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:29.517+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:29.820 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:29 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:29.517+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:01.034281+0000 front 2024-04-25T00:28:01.034251+0000 (oldest deadline 2024-04-25T00:28:26.934402+0000) 2024-04-25T00:28:30.084 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:29 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:29.962+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:30.084 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:29 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:29.962+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:30.084 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:29 smithi195 podman[70352]: 2024-04-25 00:28:29.684356874 +0000 UTC m=+0.018034409 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:28:30.084 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:29 smithi195 podman[70352]: 2024-04-25 00:28:29.781378059 +0000 UTC m=+0.115055582 container create 6e3e9a894116dca938ba829298a877550186eb083cff0545e875dad9f839ec2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T00:28:30.084 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:29 smithi195 podman[70352]: 2024-04-25 00:28:29.983857907 +0000 UTC m=+0.317535430 container init 6e3e9a894116dca938ba829298a877550186eb083cff0545e875dad9f839ec2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:28:30.085 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:29 smithi195 podman[70352]: 2024-04-25 00:28:29.986837792 +0000 UTC m=+0.320515315 container start 6e3e9a894116dca938ba829298a877550186eb083cff0545e875dad9f839ec2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS) 2024-04-25T00:28:30.085 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:30 smithi195 podman[70352]: 2024-04-25 00:28:30.083831832 +0000 UTC m=+0.417509369 container attach 6e3e9a894116dca938ba829298a877550186eb083cff0545e875dad9f839ec2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T00:28:30.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:29 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:29.819+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:30.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:29 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:29.819+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:30.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:29 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:29.819+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:30.234 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:29 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:29.810+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.235181+0000 front 2024-04-25T00:27:55.235226+0000 (oldest deadline 2024-04-25T00:28:19.335101+0000) 2024-04-25T00:28:30.234 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:29 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:29.810+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:27:59.335636+0000 front 2024-04-25T00:27:59.335677+0000 (oldest deadline 2024-04-25T00:28:25.136168+0000) 2024-04-25T00:28:30.339 INFO:journalctl@ceph.osd.5.smithi195.stdout:Apr 25 00:28:30 smithi195 podman[70352]: 2024-04-25 00:28:30.279792533 +0000 UTC m=+0.613470064 container died 6e3e9a894116dca938ba829298a877550186eb083cff0545e875dad9f839ec2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T00:28:30.340 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:30 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:30.180+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:30.340 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:30 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:30.181+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:30.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:30 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:30.326+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:30.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:30 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:30.326+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:30.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:30 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:30.326+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:30.779 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:30 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:30.502+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:30.779 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:30 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:30.502+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:01.034281+0000 front 2024-04-25T00:28:01.034251+0000 (oldest deadline 2024-04-25T00:28:26.934402+0000) 2024-04-25T00:28:30.915 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T00:28:30.954 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:28:30.954 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T00:28:30.955 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T00:28:30.955 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.6 2024-04-25T00:28:31.170 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:31 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:31.011+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:31.171 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:31 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:31.011+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:31.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:30 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:30.779+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:31.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:30 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:30.779+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:31.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:30 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:30.779+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:31.234 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:30 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:30.813+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.235181+0000 front 2024-04-25T00:27:55.235226+0000 (oldest deadline 2024-04-25T00:28:19.335101+0000) 2024-04-25T00:28:31.234 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:30 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:30.813+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:27:59.335636+0000 front 2024-04-25T00:27:59.335677+0000 (oldest deadline 2024-04-25T00:28:25.136168+0000) 2024-04-25T00:28:31.235 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:31 smithi107 systemd[1]: Stopping Ceph osd.6 for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:28:31.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:31 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:31.170+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:31.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:31 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:31.170+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:31.734 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:31 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:31.246+0000 7f8cec48e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:28:31.734 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:31 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:31.246+0000 7f8cec48e700 -1 osd.6 111 *** Got signal Terminated *** 2024-04-25T00:28:31.734 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:31 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:31.246+0000 7f8cec48e700 -1 osd.6 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:28:31.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:31 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:31.300+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:31.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:31 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:31.300+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:31.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:31 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:31.300+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:31.740 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:31 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:31.461+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:31.740 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:31 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:31.461+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:01.034281+0000 front 2024-04-25T00:28:01.034251+0000 (oldest deadline 2024-04-25T00:28:26.934402+0000) 2024-04-25T00:28:32.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:31 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:31.740+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:32.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:31 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:31.740+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:32.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:31 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:31.740+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:32.234 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:31 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:31.853+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.235181+0000 front 2024-04-25T00:27:55.235226+0000 (oldest deadline 2024-04-25T00:28:19.335101+0000) 2024-04-25T00:28:32.234 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:31 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:31.853+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:27:59.335636+0000 front 2024-04-25T00:27:59.335677+0000 (oldest deadline 2024-04-25T00:28:25.136168+0000) 2024-04-25T00:28:32.339 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:32 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:32.215+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:32.339 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:32 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:32.215+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:32.340 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:31 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:31.970+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:32.340 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:31 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:31.970+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:32.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:32 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:32.431+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:32.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:32 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:32.431+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:01.034281+0000 front 2024-04-25T00:28:01.034251+0000 (oldest deadline 2024-04-25T00:28:26.934402+0000) 2024-04-25T00:28:32.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:32 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:32.290+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:32.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:32 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:32.290+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:32.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:32 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:32.290+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:33.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:32 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:32.776+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:33.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:32 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:32.776+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:33.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:32 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:32.776+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:33.233 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:32 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:32.929+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:33.234 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:32 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:32.929+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:33.234 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:32 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:32.836+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.235181+0000 front 2024-04-25T00:27:55.235226+0000 (oldest deadline 2024-04-25T00:28:19.335101+0000) 2024-04-25T00:28:33.234 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:32 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:32.836+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:27:59.335636+0000 front 2024-04-25T00:27:59.335677+0000 (oldest deadline 2024-04-25T00:28:25.136168+0000) 2024-04-25T00:28:33.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:33 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:33.232+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:33.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:33 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:33.232+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:33.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:33 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:33.431+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:33.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:33 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:33.431+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:01.034281+0000 front 2024-04-25T00:28:01.034251+0000 (oldest deadline 2024-04-25T00:28:26.934402+0000) 2024-04-25T00:28:33.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:33 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:33.431+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:12.235476+0000 front 2024-04-25T00:28:12.235318+0000 (oldest deadline 2024-04-25T00:28:32.735346+0000) 2024-04-25T00:28:33.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:33 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:33.314+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:33.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:33 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:33.314+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:33.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:33 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:33.314+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:34.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:33 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:33.749+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:34.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:33 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:33.749+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:34.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:33 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:33.749+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:34.234 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:33 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:33.824+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.235181+0000 front 2024-04-25T00:27:55.235226+0000 (oldest deadline 2024-04-25T00:28:19.335101+0000) 2024-04-25T00:28:34.235 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:33 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:33.824+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:27:59.335636+0000 front 2024-04-25T00:27:59.335677+0000 (oldest deadline 2024-04-25T00:28:25.136168+0000) 2024-04-25T00:28:34.252 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:33 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:33.899+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:34.252 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:33 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:33.899+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:34.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:34 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:34.251+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:34.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:34 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:34.251+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:34.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:34 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:34.420+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:34.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:34 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:34.420+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:01.034281+0000 front 2024-04-25T00:28:01.034251+0000 (oldest deadline 2024-04-25T00:28:26.934402+0000) 2024-04-25T00:28:34.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:34 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:34.420+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:12.235476+0000 front 2024-04-25T00:28:12.235318+0000 (oldest deadline 2024-04-25T00:28:32.735346+0000) 2024-04-25T00:28:34.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:34 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:34.344+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:34.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:34 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:34.344+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:34.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:34 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:34.344+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:35.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:34 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:34.789+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:35.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:34 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:34.789+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:35.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:34 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:34.789+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:35.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:34 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:34.789+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:35.234 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:34 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:34.794+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.235181+0000 front 2024-04-25T00:27:55.235226+0000 (oldest deadline 2024-04-25T00:28:19.335101+0000) 2024-04-25T00:28:35.234 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:34 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:34.794+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:27:59.335636+0000 front 2024-04-25T00:27:59.335677+0000 (oldest deadline 2024-04-25T00:28:25.136168+0000) 2024-04-25T00:28:35.234 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:34 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:34.794+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.837056+0000 front 2024-04-25T00:28:09.837081+0000 (oldest deadline 2024-04-25T00:28:34.537238+0000) 2024-04-25T00:28:35.292 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:34 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:34.891+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:35.292 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:34 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:34.891+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:35.292 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:34 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:34.891+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:10.665802+0000 front 2024-04-25T00:28:10.665887+0000 (oldest deadline 2024-04-25T00:28:34.066228+0000) 2024-04-25T00:28:35.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:35 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:35.291+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:35.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:35 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:35.291+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:35.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:35 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:35.291+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:35.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:35 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:35.425+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:35.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:35 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:35.425+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:01.034281+0000 front 2024-04-25T00:28:01.034251+0000 (oldest deadline 2024-04-25T00:28:26.934402+0000) 2024-04-25T00:28:35.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:35 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:35.425+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:12.235476+0000 front 2024-04-25T00:28:12.235318+0000 (oldest deadline 2024-04-25T00:28:32.735346+0000) 2024-04-25T00:28:35.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:35 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:35.353+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:35.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:35 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:35.353+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:35.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:35 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:35.353+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:35.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:35 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:35.353+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:36.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:35 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:35.806+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:36.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:35 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:35.806+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:36.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:35 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:35.806+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:36.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:35 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:35.806+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:36.234 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:35 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:35.787+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.235181+0000 front 2024-04-25T00:27:55.235226+0000 (oldest deadline 2024-04-25T00:28:19.335101+0000) 2024-04-25T00:28:36.234 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:35 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:35.787+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:27:59.335636+0000 front 2024-04-25T00:27:59.335677+0000 (oldest deadline 2024-04-25T00:28:25.136168+0000) 2024-04-25T00:28:36.235 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:35 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6[54802]: 2024-04-25T00:28:35.787+0000 7f8ce4a7a700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.837056+0000 front 2024-04-25T00:28:09.837081+0000 (oldest deadline 2024-04-25T00:28:34.537238+0000) 2024-04-25T00:28:36.268 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:35 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:35.900+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:36.268 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:35 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:35.900+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:36.268 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:35 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:35.900+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:10.665802+0000 front 2024-04-25T00:28:10.665887+0000 (oldest deadline 2024-04-25T00:28:34.066228+0000) 2024-04-25T00:28:36.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:36 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:36.267+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:36.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:36 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:36.267+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:36.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:36 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:36.267+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:36.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:36 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:36.376+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:36.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:36 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:36.376+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:01.034281+0000 front 2024-04-25T00:28:01.034251+0000 (oldest deadline 2024-04-25T00:28:26.934402+0000) 2024-04-25T00:28:36.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:36 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:36.376+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:12.235476+0000 front 2024-04-25T00:28:12.235318+0000 (oldest deadline 2024-04-25T00:28:32.735346+0000) 2024-04-25T00:28:36.719 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:36 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:36.359+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:36.719 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:36 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:36.359+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:36.719 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:36 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:36.359+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:36.719 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:36 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:36.359+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:36.984 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:36 smithi107 podman[96056]: 2024-04-25 00:28:36.719412901 +0000 UTC m=+5.569825017 container died d0939b4021da38b69d3f0b53aeead3a032abdc93e6298c12391c59edd96158da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T00:28:37.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:36 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:36.783+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:37.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:36 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:36.783+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:37.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:36 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:36.783+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:37.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:36 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:36.783+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:37.229 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:36 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:36.927+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:37.229 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:36 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:36.927+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:37.229 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:36 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:36.927+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:10.665802+0000 front 2024-04-25T00:28:10.665887+0000 (oldest deadline 2024-04-25T00:28:34.066228+0000) 2024-04-25T00:28:37.378 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:37 smithi107 podman[96056]: 2024-04-25 00:28:37.096080597 +0000 UTC m=+5.946492704 container cleanup d0939b4021da38b69d3f0b53aeead3a032abdc93e6298c12391c59edd96158da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T00:28:37.378 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:37 smithi107 bash[96056]: ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6 2024-04-25T00:28:37.378 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:37 smithi107 podman[96075]: 2024-04-25 00:28:37.272366005 +0000 UTC m=+0.545367883 container remove d0939b4021da38b69d3f0b53aeead3a032abdc93e6298c12391c59edd96158da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-25T00:28:37.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:37 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:37.228+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:37.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:37 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:37.228+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:37.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:37 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:37.228+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:37.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:37 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:37.345+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:37.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:37 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:37.345+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:01.034281+0000 front 2024-04-25T00:28:01.034251+0000 (oldest deadline 2024-04-25T00:28:26.934402+0000) 2024-04-25T00:28:37.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:37 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:37.345+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:12.235476+0000 front 2024-04-25T00:28:12.235318+0000 (oldest deadline 2024-04-25T00:28:32.735346+0000) 2024-04-25T00:28:37.706 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:37 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:37.377+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:37.706 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:37 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:37.377+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:37.707 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:37 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:37.377+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:37.707 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:37 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:37.377+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:37.967 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:37 smithi107 podman[96165]: 2024-04-25 00:28:37.610058068 +0000 UTC m=+0.017831173 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:28:37.967 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:37 smithi107 podman[96165]: 2024-04-25 00:28:37.723016911 +0000 UTC m=+0.130790006 container create 0472c4bd2e1dc9ea02627a9776398486230f878a26f1cc9b5cf64d43b43c72c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:28:38.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:37 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:37.796+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:38.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:37 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:37.796+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:38.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:37 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:37.796+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:38.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:37 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:37.796+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:38.218 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:37 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:37.941+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:38.218 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:37 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:37.941+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:38.218 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:37 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:37.941+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:10.665802+0000 front 2024-04-25T00:28:10.665887+0000 (oldest deadline 2024-04-25T00:28:34.066228+0000) 2024-04-25T00:28:38.234 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:37 smithi107 podman[96165]: 2024-04-25 00:28:37.967250165 +0000 UTC m=+0.375023260 container init 0472c4bd2e1dc9ea02627a9776398486230f878a26f1cc9b5cf64d43b43c72c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T00:28:38.234 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:37 smithi107 podman[96165]: 2024-04-25 00:28:37.970301873 +0000 UTC m=+0.378074980 container start 0472c4bd2e1dc9ea02627a9776398486230f878a26f1cc9b5cf64d43b43c72c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:28:38.235 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:38 smithi107 podman[96165]: 2024-04-25 00:28:38.067337649 +0000 UTC m=+0.475110760 container attach 0472c4bd2e1dc9ea02627a9776398486230f878a26f1cc9b5cf64d43b43c72c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:28:38.569 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:38 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:38.332+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:38.570 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:38 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:38.332+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:38.570 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:38 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:38.332+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:38.570 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:38 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:38.332+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:38.570 INFO:journalctl@ceph.osd.6.smithi107.stdout:Apr 25 00:28:38 smithi107 podman[96165]: 2024-04-25 00:28:38.258931301 +0000 UTC m=+0.666704409 container died 0472c4bd2e1dc9ea02627a9776398486230f878a26f1cc9b5cf64d43b43c72c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T00:28:38.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:38 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:38.217+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:38.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:38 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:38.217+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:38.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:38 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:38.217+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:38.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:38 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:38.307+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:38.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:38 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:38.307+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:01.034281+0000 front 2024-04-25T00:28:01.034251+0000 (oldest deadline 2024-04-25T00:28:26.934402+0000) 2024-04-25T00:28:38.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:38 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:38.307+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:12.235476+0000 front 2024-04-25T00:28:12.235318+0000 (oldest deadline 2024-04-25T00:28:32.735346+0000) 2024-04-25T00:28:38.844 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T00:28:38.883 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:28:38.883 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T00:28:38.883 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T00:28:38.883 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.7 2024-04-25T00:28:39.184 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:38 smithi170 systemd[1]: Stopping Ceph osd.7 for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:28:39.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:38 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:38.778+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:39.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:38 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:38.778+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:39.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:38 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:38.778+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:39.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:38 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:38.778+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:39.244 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:38 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:38.931+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:39.245 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:38 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:38.931+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:39.245 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:38 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:38.931+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:10.665802+0000 front 2024-04-25T00:28:10.665887+0000 (oldest deadline 2024-04-25T00:28:34.066228+0000) 2024-04-25T00:28:39.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:39 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:39.244+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:39.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:39 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:39.244+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:39.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:39 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:39.244+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:39.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:39 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:39.185+0000 7f7a71fe6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:28:39.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:39 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:39.185+0000 7f7a71fe6700 -1 osd.7 111 *** Got signal Terminated *** 2024-04-25T00:28:39.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:39 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:39.185+0000 7f7a71fe6700 -1 osd.7 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:28:39.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:39 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:39.303+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:39.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:39 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:39.303+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:01.034281+0000 front 2024-04-25T00:28:01.034251+0000 (oldest deadline 2024-04-25T00:28:26.934402+0000) 2024-04-25T00:28:39.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:39 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:39.303+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:12.235476+0000 front 2024-04-25T00:28:12.235318+0000 (oldest deadline 2024-04-25T00:28:32.735346+0000) 2024-04-25T00:28:39.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:39 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:39.381+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:39.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:39 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:39.381+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:39.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:39 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:39.381+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:39.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:39 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:39.381+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:40.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:39 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:39.804+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:40.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:39 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:39.804+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:40.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:39 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:39.804+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:40.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:39 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:39.804+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:40.268 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:39 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:39.929+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:40.269 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:39 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:39.929+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:40.269 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:39 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:39.929+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:10.665802+0000 front 2024-04-25T00:28:10.665887+0000 (oldest deadline 2024-04-25T00:28:34.066228+0000) 2024-04-25T00:28:40.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:40 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:40.267+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:40.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:40 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:40.267+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:40.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:40 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:40.267+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:40.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:40 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:40.332+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:40.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:40 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:40.332+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:01.034281+0000 front 2024-04-25T00:28:01.034251+0000 (oldest deadline 2024-04-25T00:28:26.934402+0000) 2024-04-25T00:28:40.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:40 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:40.332+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:12.235476+0000 front 2024-04-25T00:28:12.235318+0000 (oldest deadline 2024-04-25T00:28:32.735346+0000) 2024-04-25T00:28:40.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:40 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:40.400+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:40.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:40 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:40.400+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:40.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:40 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:40.400+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:40.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:40 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:40.400+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:41.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:40 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:40.797+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:41.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:40 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:40.797+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:41.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:40 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:40.797+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:41.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:40 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:40.797+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:41.237 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:40 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:40.934+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:41.237 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:40 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:40.934+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:41.237 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:40 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:40.934+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:10.665802+0000 front 2024-04-25T00:28:10.665887+0000 (oldest deadline 2024-04-25T00:28:34.066228+0000) 2024-04-25T00:28:41.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:41 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:41.236+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:41.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:41 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:41.236+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:41.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:41 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:41.236+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:41.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:41 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:41.294+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:41.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:41 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:41.294+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:01.034281+0000 front 2024-04-25T00:28:01.034251+0000 (oldest deadline 2024-04-25T00:28:26.934402+0000) 2024-04-25T00:28:41.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:41 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:41.294+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:12.235476+0000 front 2024-04-25T00:28:12.235318+0000 (oldest deadline 2024-04-25T00:28:32.735346+0000) 2024-04-25T00:28:41.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:41 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:41.352+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:41.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:41 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:41.352+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:41.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:41 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:41.352+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:41.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:41 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:41.352+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:42.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:41 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:41.781+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:42.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:41 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:41.781+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:42.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:41 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:41.781+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:42.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:41 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:41.781+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:42.339 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:42 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:42.187+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:42.339 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:42 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:42.187+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:42.339 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:42 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:42.187+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:42.339 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:41 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:41.977+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:42.340 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:41 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:41.977+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:42.340 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:41 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:41.977+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:10.665802+0000 front 2024-04-25T00:28:10.665887+0000 (oldest deadline 2024-04-25T00:28:34.066228+0000) 2024-04-25T00:28:42.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:42 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:42.330+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:42.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:42 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:42.330+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:01.034281+0000 front 2024-04-25T00:28:01.034251+0000 (oldest deadline 2024-04-25T00:28:26.934402+0000) 2024-04-25T00:28:42.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:42 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:42.330+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:12.235476+0000 front 2024-04-25T00:28:12.235318+0000 (oldest deadline 2024-04-25T00:28:32.735346+0000) 2024-04-25T00:28:42.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:42 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:42.370+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:42.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:42 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:42.370+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:42.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:42 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:42.370+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:42.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:42 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:42.370+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:43.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:42 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:42.791+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:43.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:42 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:42.791+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:43.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:42 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:42.791+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:43.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:42 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:42.791+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:43.339 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:42 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:42.935+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:43.339 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:42 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:42.935+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:43.339 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:42 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:42.935+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:10.665802+0000 front 2024-04-25T00:28:10.665887+0000 (oldest deadline 2024-04-25T00:28:34.066228+0000) 2024-04-25T00:28:43.340 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:43 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:43.166+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:43.340 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:43 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:43.166+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:43.340 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:43 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:43.166+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:43.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:43 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:43.290+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.433186+0000 front 2024-04-25T00:27:53.433232+0000 (oldest deadline 2024-04-25T00:28:18.733323+0000) 2024-04-25T00:28:43.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:43 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:43.290+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:01.034281+0000 front 2024-04-25T00:28:01.034251+0000 (oldest deadline 2024-04-25T00:28:26.934402+0000) 2024-04-25T00:28:43.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:43 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:43.290+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:12.235476+0000 front 2024-04-25T00:28:12.235318+0000 (oldest deadline 2024-04-25T00:28:32.735346+0000) 2024-04-25T00:28:43.685 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:43 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7[50776]: 2024-04-25T00:28:43.290+0000 7f7a6e5da700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:18.636226+0000 front 2024-04-25T00:28:18.636166+0000 (oldest deadline 2024-04-25T00:28:42.736371+0000) 2024-04-25T00:28:43.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:43 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:43.402+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:43.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:43 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:43.402+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:43.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:43 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:43.402+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:43.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:43 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:43.402+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:44.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:43 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:43.837+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:44.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:43 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:43.837+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:44.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:43 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:43.837+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:44.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:43 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:43.837+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:44.216 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:43 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:43.938+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:44.216 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:43 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:43.938+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:44.216 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:43 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:43.938+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:10.665802+0000 front 2024-04-25T00:28:10.665887+0000 (oldest deadline 2024-04-25T00:28:34.066228+0000) 2024-04-25T00:28:44.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:44 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:44.215+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:44.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:44 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:44.215+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:44.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:44 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:44.215+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:44.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:44 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:44.403+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:44.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:44 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:44.403+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:44.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:44 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:44.403+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:44.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:44 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:44.403+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:44.828 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:44 smithi170 podman[68871]: 2024-04-25 00:28:44.532242809 +0000 UTC m=+5.456256597 container died ddb46930f9624f9ad11efc596df46f445e95c4cd21436b9cccaba3f76c8a0a35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:28:44.829 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:44 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:44.827+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:44.829 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:44 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:44.827+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:45.087 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:44 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:44.827+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:45.087 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:44 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:44.827+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:45.087 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:44 smithi170 podman[68871]: 2024-04-25 00:28:44.869155226 +0000 UTC m=+5.793169015 container cleanup ddb46930f9624f9ad11efc596df46f445e95c4cd21436b9cccaba3f76c8a0a35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T00:28:45.088 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:44 smithi170 bash[68871]: ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7 2024-04-25T00:28:45.339 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:45 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:45.204+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:45.339 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:45 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:45.204+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:45.340 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:45 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:45.204+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:45.340 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:45 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:45.204+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:28:45.340 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:44 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:44.971+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:45.340 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:44 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:44.971+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:45.340 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:44 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:44.971+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:10.665802+0000 front 2024-04-25T00:28:10.665887+0000 (oldest deadline 2024-04-25T00:28:34.066228+0000) 2024-04-25T00:28:45.435 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:45 smithi170 podman[68890]: 2024-04-25 00:28:45.087307747 +0000 UTC m=+0.549534527 container remove ddb46930f9624f9ad11efc596df46f445e95c4cd21436b9cccaba3f76c8a0a35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-25T00:28:45.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:45 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:45.413+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:45.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:45 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:45.413+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:45.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:45 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:45.413+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:45.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:45 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:45.413+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:45.810 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:45 smithi170 podman[68980]: 2024-04-25 00:28:45.392594293 +0000 UTC m=+0.017998548 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:28:45.810 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:45 smithi170 podman[68980]: 2024-04-25 00:28:45.513527944 +0000 UTC m=+0.138932185 container create 585a8962f4d4187d194ef734260418793dbab3485615f6fae46ae44d0e278d0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T00:28:45.810 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:45 smithi170 podman[68980]: 2024-04-25 00:28:45.732539716 +0000 UTC m=+0.357943957 container init 585a8962f4d4187d194ef734260418793dbab3485615f6fae46ae44d0e278d0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T00:28:45.810 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:45 smithi170 podman[68980]: 2024-04-25 00:28:45.735597844 +0000 UTC m=+0.361002089 container start 585a8962f4d4187d194ef734260418793dbab3485615f6fae46ae44d0e278d0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T00:28:46.185 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:45 smithi170 podman[68980]: 2024-04-25 00:28:45.849374208 +0000 UTC m=+0.474778457 container attach 585a8962f4d4187d194ef734260418793dbab3485615f6fae46ae44d0e278d0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:28:46.185 INFO:journalctl@ceph.osd.7.smithi170.stdout:Apr 25 00:28:46 smithi170 podman[68980]: 2024-04-25 00:28:46.027813509 +0000 UTC m=+0.653217752 container died 585a8962f4d4187d194ef734260418793dbab3485615f6fae46ae44d0e278d0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T00:28:46.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:45 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:45.808+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:46.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:45 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:45.808+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:46.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:45 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:45.808+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:46.186 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:45 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:45.808+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:46.186 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:45 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:45.808+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.152669+0000 front 2024-04-25T00:28:20.152625+0000 (oldest deadline 2024-04-25T00:28:45.452796+0000) 2024-04-25T00:28:46.227 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:45 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:45.942+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:46.227 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:45 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:45.942+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:46.227 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:45 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:45.942+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:10.665802+0000 front 2024-04-25T00:28:10.665887+0000 (oldest deadline 2024-04-25T00:28:34.066228+0000) 2024-04-25T00:28:46.227 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:45 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:45.942+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:19.768428+0000 front 2024-04-25T00:28:19.768563+0000 (oldest deadline 2024-04-25T00:28:45.668370+0000) 2024-04-25T00:28:46.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:46 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:46.226+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:46.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:46 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:46.226+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:46.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:46 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:46.226+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:46.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:46 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:46.226+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:28:46.702 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T00:28:46.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:46 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:46.411+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:46.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:46 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:46.411+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:46.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:46 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:46.411+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:46.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:46 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:46.411+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:46.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:46 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:46.411+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:15.172569+0000 front 2024-04-25T00:28:15.172543+0000 (oldest deadline 2024-04-25T00:28:46.372898+0000) 2024-04-25T00:28:46.739 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:28:46.739 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T00:28:46.740 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T00:28:46.740 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.8 2024-04-25T00:28:47.041 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:46 smithi195 systemd[1]: Stopping Ceph osd.8 for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:28:47.041 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:46 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:46.936+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:47.041 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:46 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:46.936+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:47.041 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:46 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:46.936+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:10.665802+0000 front 2024-04-25T00:28:10.665887+0000 (oldest deadline 2024-04-25T00:28:34.066228+0000) 2024-04-25T00:28:47.041 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:46 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:46.936+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:19.768428+0000 front 2024-04-25T00:28:19.768563+0000 (oldest deadline 2024-04-25T00:28:45.668370+0000) 2024-04-25T00:28:47.041 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:47 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:47.040+0000 7fbd4b68a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:28:47.041 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:47 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:47.040+0000 7fbd4b68a700 -1 osd.8 111 *** Got signal Terminated *** 2024-04-25T00:28:47.041 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:47 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:47.040+0000 7fbd4b68a700 -1 osd.8 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:28:47.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:46 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:46.775+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:47.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:46 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:46.775+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:47.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:46 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:46.775+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:47.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:46 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:46.775+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:47.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:46 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:46.775+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.152669+0000 front 2024-04-25T00:28:20.152625+0000 (oldest deadline 2024-04-25T00:28:45.452796+0000) 2024-04-25T00:28:47.339 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:47 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:47.262+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:47.339 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:47 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:47.262+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:47.339 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:47 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:47.262+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:47.339 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:47 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:47.262+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:28:47.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:47 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:47.420+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:47.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:47 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:47.420+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:47.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:47 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:47.420+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:47.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:47 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:47.420+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:47.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:47 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:47.420+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:15.172569+0000 front 2024-04-25T00:28:15.172543+0000 (oldest deadline 2024-04-25T00:28:46.372898+0000) 2024-04-25T00:28:48.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:47 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:47.748+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:48.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:47 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:47.748+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:48.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:47 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:47.748+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:48.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:47 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:47.748+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:48.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:47 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:47.748+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.152669+0000 front 2024-04-25T00:28:20.152625+0000 (oldest deadline 2024-04-25T00:28:45.452796+0000) 2024-04-25T00:28:48.221 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:47 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:47.924+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:48.221 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:47 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:47.924+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:48.221 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:47 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:47.924+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:10.665802+0000 front 2024-04-25T00:28:10.665887+0000 (oldest deadline 2024-04-25T00:28:34.066228+0000) 2024-04-25T00:28:48.221 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:47 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:47.924+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:19.768428+0000 front 2024-04-25T00:28:19.768563+0000 (oldest deadline 2024-04-25T00:28:45.668370+0000) 2024-04-25T00:28:48.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:48 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:48.220+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:48.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:48 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:48.220+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:48.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:48 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:48.220+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:48.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:48 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:48.220+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:28:48.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:48 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:48.460+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:48.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:48 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:48.460+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:48.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:48 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:48.460+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:48.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:48 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:48.460+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:48.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:48 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:48.460+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:15.172569+0000 front 2024-04-25T00:28:15.172543+0000 (oldest deadline 2024-04-25T00:28:46.372898+0000) 2024-04-25T00:28:49.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:48 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:48.701+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:49.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:48 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:48.701+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:49.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:48 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:48.701+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:49.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:48 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:48.701+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:49.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:48 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:48.701+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.152669+0000 front 2024-04-25T00:28:20.152625+0000 (oldest deadline 2024-04-25T00:28:45.452796+0000) 2024-04-25T00:28:49.201 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:48 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:48.924+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:49.201 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:48 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:48.924+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:49.201 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:48 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:48.924+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:10.665802+0000 front 2024-04-25T00:28:10.665887+0000 (oldest deadline 2024-04-25T00:28:34.066228+0000) 2024-04-25T00:28:49.201 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:48 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:48.924+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:19.768428+0000 front 2024-04-25T00:28:19.768563+0000 (oldest deadline 2024-04-25T00:28:45.668370+0000) 2024-04-25T00:28:49.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:49 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:49.201+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:49.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:49 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:49.201+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:49.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:49 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:49.201+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:49.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:49 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:49.201+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:28:49.984 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:49 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:49.495+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:49.984 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:49 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:49.495+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:49.984 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:49 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:49.495+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:49.984 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:49 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:49.495+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:49.985 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:49 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:49.495+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:15.172569+0000 front 2024-04-25T00:28:15.172543+0000 (oldest deadline 2024-04-25T00:28:46.372898+0000) 2024-04-25T00:28:49.985 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:49 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:49.495+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:26.373555+0000 front 2024-04-25T00:28:26.373539+0000 (oldest deadline 2024-04-25T00:28:49.273305+0000) 2024-04-25T00:28:50.086 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:49 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:49.731+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:50.086 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:49 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:49.731+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:50.086 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:49 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:49.731+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:50.086 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:49 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:49.731+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:50.086 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:49 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:49.731+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.152669+0000 front 2024-04-25T00:28:20.152625+0000 (oldest deadline 2024-04-25T00:28:45.452796+0000) 2024-04-25T00:28:50.251 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:49 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:49.883+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:50.251 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:49 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:49.883+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:50.252 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:49 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:49.883+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:10.665802+0000 front 2024-04-25T00:28:10.665887+0000 (oldest deadline 2024-04-25T00:28:34.066228+0000) 2024-04-25T00:28:50.252 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:49 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:49.883+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:19.768428+0000 front 2024-04-25T00:28:19.768563+0000 (oldest deadline 2024-04-25T00:28:45.668370+0000) 2024-04-25T00:28:50.527 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:50 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:50.250+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:50.528 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:50 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:50.251+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:50.528 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:50 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:50.251+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:50.528 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:50 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:50.251+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:28:50.984 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:50 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:50.520+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:50.984 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:50 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:50.520+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:50.984 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:50 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:50.520+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:50.984 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:50 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:50.520+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:50.985 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:50 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:50.520+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:15.172569+0000 front 2024-04-25T00:28:15.172543+0000 (oldest deadline 2024-04-25T00:28:46.372898+0000) 2024-04-25T00:28:50.985 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:50 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:50.520+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:26.373555+0000 front 2024-04-25T00:28:26.373539+0000 (oldest deadline 2024-04-25T00:28:49.273305+0000) 2024-04-25T00:28:51.097 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:50 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:50.749+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:51.098 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:50 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:50.749+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:51.098 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:50 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:50.749+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:51.098 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:50 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:50.749+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:51.098 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:50 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:50.749+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.152669+0000 front 2024-04-25T00:28:20.152625+0000 (oldest deadline 2024-04-25T00:28:45.452796+0000) 2024-04-25T00:28:51.098 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:50 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:50.749+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:27.753658+0000 front 2024-04-25T00:28:27.753711+0000 (oldest deadline 2024-04-25T00:28:50.053645+0000) 2024-04-25T00:28:51.281 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:50 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:50.886+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:51.281 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:50 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:50.886+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:51.281 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:50 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:50.886+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:10.665802+0000 front 2024-04-25T00:28:10.665887+0000 (oldest deadline 2024-04-25T00:28:34.066228+0000) 2024-04-25T00:28:51.281 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:50 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:50.886+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:19.768428+0000 front 2024-04-25T00:28:19.768563+0000 (oldest deadline 2024-04-25T00:28:45.668370+0000) 2024-04-25T00:28:51.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:51 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:51.280+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:51.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:51 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:51.280+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:51.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:51 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:51.280+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:51.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:51 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:51.280+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:28:51.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:51 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:51.483+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:51.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:51 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:51.483+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:51.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:51 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:51.483+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:51.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:51 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:51.483+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:51.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:51 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:51.483+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:15.172569+0000 front 2024-04-25T00:28:15.172543+0000 (oldest deadline 2024-04-25T00:28:46.372898+0000) 2024-04-25T00:28:51.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:51 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:51.483+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:26.373555+0000 front 2024-04-25T00:28:26.373539+0000 (oldest deadline 2024-04-25T00:28:49.273305+0000) 2024-04-25T00:28:52.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:51 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:51.779+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:52.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:51 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:51.779+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:52.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:51 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:51.779+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:52.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:51 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:51.779+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:52.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:51 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:51.779+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.152669+0000 front 2024-04-25T00:28:20.152625+0000 (oldest deadline 2024-04-25T00:28:45.452796+0000) 2024-04-25T00:28:52.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:51 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:51.779+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:27.753658+0000 front 2024-04-25T00:28:27.753711+0000 (oldest deadline 2024-04-25T00:28:50.053645+0000) 2024-04-25T00:28:52.316 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:51 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:51.879+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:54.263431+0000 front 2024-04-25T00:27:54.263367+0000 (oldest deadline 2024-04-25T00:28:17.163467+0000) 2024-04-25T00:28:52.316 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:51 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:51.879+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.064742+0000 front 2024-04-25T00:28:03.064539+0000 (oldest deadline 2024-04-25T00:28:25.964601+0000) 2024-04-25T00:28:52.316 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:51 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:51.879+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:10.665802+0000 front 2024-04-25T00:28:10.665887+0000 (oldest deadline 2024-04-25T00:28:34.066228+0000) 2024-04-25T00:28:52.316 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:51 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:51.879+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:19.768428+0000 front 2024-04-25T00:28:19.768563+0000 (oldest deadline 2024-04-25T00:28:45.668370+0000) 2024-04-25T00:28:52.316 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:51 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8[51844]: 2024-04-25T00:28:51.879+0000 7fbd47c7e700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:25.668785+0000 front 2024-04-25T00:28:25.668729+0000 (oldest deadline 2024-04-25T00:28:50.968905+0000) 2024-04-25T00:28:52.589 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:52 smithi195 podman[70458]: 2024-04-25 00:28:52.462380807 +0000 UTC m=+5.534641710 container died 65e510050ab01891c246a5296b06cc6cb8eb0a4bda661dd7e5a1c0e46d9e9298 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T00:28:52.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:52 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:52.315+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:52.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:52 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:52.315+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:52.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:52 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:52.315+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:52.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:52 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:52.315+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:28:52.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:52 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:52.460+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:52.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:52 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:52.460+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:52.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:52 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:52.460+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:52.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:52 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:52.461+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:52.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:52 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:52.461+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:15.172569+0000 front 2024-04-25T00:28:15.172543+0000 (oldest deadline 2024-04-25T00:28:46.372898+0000) 2024-04-25T00:28:52.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:52 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:52.461+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:26.373555+0000 front 2024-04-25T00:28:26.373539+0000 (oldest deadline 2024-04-25T00:28:49.273305+0000) 2024-04-25T00:28:53.089 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:52 smithi195 podman[70458]: 2024-04-25 00:28:52.866151964 +0000 UTC m=+5.938412873 container cleanup 65e510050ab01891c246a5296b06cc6cb8eb0a4bda661dd7e5a1c0e46d9e9298 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True) 2024-04-25T00:28:53.089 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:52 smithi195 bash[70458]: ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8 2024-04-25T00:28:53.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:52 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:52.746+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:53.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:52 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:52.746+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:53.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:52 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:52.746+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:53.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:52 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:52.746+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:53.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:52 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:52.746+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.152669+0000 front 2024-04-25T00:28:20.152625+0000 (oldest deadline 2024-04-25T00:28:45.452796+0000) 2024-04-25T00:28:53.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:52 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:52.746+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:27.753658+0000 front 2024-04-25T00:28:27.753711+0000 (oldest deadline 2024-04-25T00:28:50.053645+0000) 2024-04-25T00:28:53.468 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:53 smithi195 podman[70485]: 2024-04-25 00:28:53.125819449 +0000 UTC m=+0.662164047 container remove 65e510050ab01891c246a5296b06cc6cb8eb0a4bda661dd7e5a1c0e46d9e9298 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T00:28:53.468 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:53 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:53.348+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:53.468 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:53 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:53.348+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:53.468 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:53 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:53.348+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:53.468 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:53 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:53.348+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:28:53.468 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:53 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:53.348+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:28.320719+0000 front 2024-04-25T00:28:28.320789+0000 (oldest deadline 2024-04-25T00:28:53.020814+0000) 2024-04-25T00:28:53.804 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:53 smithi195 podman[70583]: 2024-04-25 00:28:53.468187446 +0000 UTC m=+0.104355076 container create f26d956cdb8ab40acc3c322ff298cfe040a1050dbff9c69b4048b1a48bc3bd54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T00:28:53.804 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:53 smithi195 podman[70583]: 2024-04-25 00:28:53.382310859 +0000 UTC m=+0.018478519 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:28:53.804 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:53 smithi195 podman[70583]: 2024-04-25 00:28:53.712621251 +0000 UTC m=+0.348788882 container init f26d956cdb8ab40acc3c322ff298cfe040a1050dbff9c69b4048b1a48bc3bd54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T00:28:53.804 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:53 smithi195 podman[70583]: 2024-04-25 00:28:53.715755622 +0000 UTC m=+0.351923258 container start f26d956cdb8ab40acc3c322ff298cfe040a1050dbff9c69b4048b1a48bc3bd54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:28:53.804 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:53 smithi195 podman[70583]: 2024-04-25 00:28:53.804309257 +0000 UTC m=+0.440476889 container attach f26d956cdb8ab40acc3c322ff298cfe040a1050dbff9c69b4048b1a48bc3bd54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T00:28:53.984 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:53 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:53.496+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:53.984 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:53 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:53.496+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:53.984 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:53 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:53.496+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:53.985 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:53 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:53.496+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:53.985 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:53 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:53.496+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:15.172569+0000 front 2024-04-25T00:28:15.172543+0000 (oldest deadline 2024-04-25T00:28:46.372898+0000) 2024-04-25T00:28:53.985 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:53 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:53.496+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:26.373555+0000 front 2024-04-25T00:28:26.373539+0000 (oldest deadline 2024-04-25T00:28:49.273305+0000) 2024-04-25T00:28:54.089 INFO:journalctl@ceph.osd.8.smithi195.stdout:Apr 25 00:28:54 smithi195 podman[70583]: 2024-04-25 00:28:54.00910957 +0000 UTC m=+0.645277213 container died f26d956cdb8ab40acc3c322ff298cfe040a1050dbff9c69b4048b1a48bc3bd54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T00:28:54.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:53 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:53.737+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:54.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:53 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:53.738+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:54.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:53 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:53.738+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:54.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:53 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:53.738+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:54.186 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:53 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:53.738+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.152669+0000 front 2024-04-25T00:28:20.152625+0000 (oldest deadline 2024-04-25T00:28:45.452796+0000) 2024-04-25T00:28:54.186 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:53 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:53.738+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:27.753658+0000 front 2024-04-25T00:28:27.753711+0000 (oldest deadline 2024-04-25T00:28:50.053645+0000) 2024-04-25T00:28:54.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:54 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:54.327+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:54.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:54 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:54.327+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:54.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:54 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:54.327+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:54.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:54 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:54.327+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:28:54.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:54 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:54.327+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:28.320719+0000 front 2024-04-25T00:28:28.320789+0000 (oldest deadline 2024-04-25T00:28:53.020814+0000) 2024-04-25T00:28:54.645 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T00:28:54.684 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:28:54.684 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T00:28:54.685 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T00:28:54.685 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.9 2024-04-25T00:28:54.980 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:54 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:54.498+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:54.980 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:54 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:54.498+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:54.980 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:54 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:54.498+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:54.980 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:54 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:54.498+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:54.980 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:54 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:54.498+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:15.172569+0000 front 2024-04-25T00:28:15.172543+0000 (oldest deadline 2024-04-25T00:28:46.372898+0000) 2024-04-25T00:28:54.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:54 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:54.498+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:26.373555+0000 front 2024-04-25T00:28:26.373539+0000 (oldest deadline 2024-04-25T00:28:49.273305+0000) 2024-04-25T00:28:54.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:54 smithi107 systemd[1]: Stopping Ceph osd.9 for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:28:54.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:54 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:54.979+0000 7fc8a3d49700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:28:54.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:54 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:54.979+0000 7fc8a3d49700 -1 osd.9 111 *** Got signal Terminated *** 2024-04-25T00:28:54.981 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:54 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:54.979+0000 7fc8a3d49700 -1 osd.9 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:28:55.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:54 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:54.746+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:55.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:54 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:54.746+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:55.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:54 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:54.746+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:55.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:54 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:54.746+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:55.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:54 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:54.746+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.152669+0000 front 2024-04-25T00:28:20.152625+0000 (oldest deadline 2024-04-25T00:28:45.452796+0000) 2024-04-25T00:28:55.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:54 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:54.746+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:27.753658+0000 front 2024-04-25T00:28:27.753711+0000 (oldest deadline 2024-04-25T00:28:50.053645+0000) 2024-04-25T00:28:55.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:55 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:55.335+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:55.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:55 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:55.335+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:55.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:55 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:55.335+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:55.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:55 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:55.335+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:28:55.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:55 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:55.335+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:28.320719+0000 front 2024-04-25T00:28:28.320789+0000 (oldest deadline 2024-04-25T00:28:53.020814+0000) 2024-04-25T00:28:55.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:55 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:55.448+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:55.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:55 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:55.448+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:55.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:55 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:55.448+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:55.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:55 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:55.448+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:55.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:55 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:55.448+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:15.172569+0000 front 2024-04-25T00:28:15.172543+0000 (oldest deadline 2024-04-25T00:28:46.372898+0000) 2024-04-25T00:28:55.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:55 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:55.448+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:26.373555+0000 front 2024-04-25T00:28:26.373539+0000 (oldest deadline 2024-04-25T00:28:49.273305+0000) 2024-04-25T00:28:56.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:55 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:55.785+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:56.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:55 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:55.785+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:56.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:55 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:55.785+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:56.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:55 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:55.785+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:56.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:55 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:55.785+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.152669+0000 front 2024-04-25T00:28:20.152625+0000 (oldest deadline 2024-04-25T00:28:45.452796+0000) 2024-04-25T00:28:56.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:55 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:55.785+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:27.753658+0000 front 2024-04-25T00:28:27.753711+0000 (oldest deadline 2024-04-25T00:28:50.053645+0000) 2024-04-25T00:28:56.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:56 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:56.404+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:56.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:56 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:56.404+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:56.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:56 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:56.404+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:56.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:56 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:56.404+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:56.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:56 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:56.404+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:15.172569+0000 front 2024-04-25T00:28:15.172543+0000 (oldest deadline 2024-04-25T00:28:46.372898+0000) 2024-04-25T00:28:56.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:56 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:56.404+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:26.373555+0000 front 2024-04-25T00:28:26.373539+0000 (oldest deadline 2024-04-25T00:28:49.273305+0000) 2024-04-25T00:28:56.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:56 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:56.375+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:56.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:56 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:56.375+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:56.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:56 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:56.375+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:56.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:56 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:56.375+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:28:56.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:56 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:56.375+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:28.320719+0000 front 2024-04-25T00:28:28.320789+0000 (oldest deadline 2024-04-25T00:28:53.020814+0000) 2024-04-25T00:28:57.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:56 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:56.792+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:57.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:56 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:56.792+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:57.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:56 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:56.792+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:57.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:56 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:56.792+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:57.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:56 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:56.792+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.152669+0000 front 2024-04-25T00:28:20.152625+0000 (oldest deadline 2024-04-25T00:28:45.452796+0000) 2024-04-25T00:28:57.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:56 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:56.792+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:27.753658+0000 front 2024-04-25T00:28:27.753711+0000 (oldest deadline 2024-04-25T00:28:50.053645+0000) 2024-04-25T00:28:57.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:57 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:57.398+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:57.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:57 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:57.398+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:57.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:57 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:57.398+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:57.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:57 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:57.398+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:57.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:57 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:57.398+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:15.172569+0000 front 2024-04-25T00:28:15.172543+0000 (oldest deadline 2024-04-25T00:28:46.372898+0000) 2024-04-25T00:28:57.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:57 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:57.398+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:26.373555+0000 front 2024-04-25T00:28:26.373539+0000 (oldest deadline 2024-04-25T00:28:49.273305+0000) 2024-04-25T00:28:57.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:57 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:57.369+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:57.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:57 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:57.369+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:57.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:57 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:57.369+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:57.840 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:57 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:57.369+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:28:57.840 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:57 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:57.369+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:28.320719+0000 front 2024-04-25T00:28:28.320789+0000 (oldest deadline 2024-04-25T00:28:53.020814+0000) 2024-04-25T00:28:58.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:57 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:57.793+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:58.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:57 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:57.793+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:58.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:57 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:57.793+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:58.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:57 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:57.793+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:58.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:57 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:57.793+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.152669+0000 front 2024-04-25T00:28:20.152625+0000 (oldest deadline 2024-04-25T00:28:45.452796+0000) 2024-04-25T00:28:58.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:57 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:57.793+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:27.753658+0000 front 2024-04-25T00:28:27.753711+0000 (oldest deadline 2024-04-25T00:28:50.053645+0000) 2024-04-25T00:28:58.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:58 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:58.333+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:58.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:58 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:58.333+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:58.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:58 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:58.333+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:58.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:58 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:58.333+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:28:58.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:58 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:58.333+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:28.320719+0000 front 2024-04-25T00:28:28.320789+0000 (oldest deadline 2024-04-25T00:28:53.020814+0000) 2024-04-25T00:28:58.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:58 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:58.333+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:33.021381+0000 front 2024-04-25T00:28:33.021452+0000 (oldest deadline 2024-04-25T00:28:58.321264+0000) 2024-04-25T00:28:58.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:58 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:58.406+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:58.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:58 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:58.406+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:58.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:58 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:58.406+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:58.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:58 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:58.406+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:58.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:58 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:58.406+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:15.172569+0000 front 2024-04-25T00:28:15.172543+0000 (oldest deadline 2024-04-25T00:28:46.372898+0000) 2024-04-25T00:28:58.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:58 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:58.406+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:26.373555+0000 front 2024-04-25T00:28:26.373539+0000 (oldest deadline 2024-04-25T00:28:49.273305+0000) 2024-04-25T00:28:59.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:58 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:58.772+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:28:59.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:58 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:58.772+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:28:59.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:58 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:58.772+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:28:59.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:58 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:58.772+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:28:59.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:58 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:58.772+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.152669+0000 front 2024-04-25T00:28:20.152625+0000 (oldest deadline 2024-04-25T00:28:45.452796+0000) 2024-04-25T00:28:59.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:58 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:58.772+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:27.753658+0000 front 2024-04-25T00:28:27.753711+0000 (oldest deadline 2024-04-25T00:28:50.053645+0000) 2024-04-25T00:28:59.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:59 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:59.313+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:28:59.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:59 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:59.313+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:28:59.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:59 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:59.313+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:28:59.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:59 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:59.313+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:28:59.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:59 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:59.313+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:28.320719+0000 front 2024-04-25T00:28:28.320789+0000 (oldest deadline 2024-04-25T00:28:53.020814+0000) 2024-04-25T00:28:59.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:28:59 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:28:59.313+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:33.021381+0000 front 2024-04-25T00:28:33.021452+0000 (oldest deadline 2024-04-25T00:28:58.321264+0000) 2024-04-25T00:28:59.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:59 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:59.416+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:46.067479+0000 front 2024-04-25T00:27:46.067456+0000 (oldest deadline 2024-04-25T00:28:10.167605+0000) 2024-04-25T00:28:59.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:59 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:59.416+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:53.469530+0000 front 2024-04-25T00:27:53.469644+0000 (oldest deadline 2024-04-25T00:28:18.169399+0000) 2024-04-25T00:28:59.734 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:59 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:59.416+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:02.870928+0000 front 2024-04-25T00:28:02.871084+0000 (oldest deadline 2024-04-25T00:28:25.770471+0000) 2024-04-25T00:28:59.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:59 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:59.416+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:11.671609+0000 front 2024-04-25T00:28:11.671624+0000 (oldest deadline 2024-04-25T00:28:35.171770+0000) 2024-04-25T00:28:59.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:59 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:59.416+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:15.172569+0000 front 2024-04-25T00:28:15.172543+0000 (oldest deadline 2024-04-25T00:28:46.372898+0000) 2024-04-25T00:28:59.735 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:28:59 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9[61407]: 2024-04-25T00:28:59.416+0000 7fc89c335700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:26.373555+0000 front 2024-04-25T00:28:26.373539+0000 (oldest deadline 2024-04-25T00:28:49.273305+0000) 2024-04-25T00:29:00.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:59 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:59.807+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:29:00.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:59 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:59.807+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:29:00.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:59 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:59.807+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:29:00.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:59 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:59.807+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:29:00.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:59 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:59.807+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.152669+0000 front 2024-04-25T00:28:20.152625+0000 (oldest deadline 2024-04-25T00:28:45.452796+0000) 2024-04-25T00:29:00.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:59 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:59.807+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:27.753658+0000 front 2024-04-25T00:28:27.753711+0000 (oldest deadline 2024-04-25T00:28:50.053645+0000) 2024-04-25T00:29:00.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:28:59 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:28:59.807+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:35.354905+0000 front 2024-04-25T00:28:35.354785+0000 (oldest deadline 2024-04-25T00:28:58.854593+0000) 2024-04-25T00:29:00.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:00 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:00.297+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:29:00.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:00 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:00.297+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:29:00.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:00 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:00.297+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:29:00.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:00 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:00.297+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:29:00.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:00 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:00.297+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:28.320719+0000 front 2024-04-25T00:28:28.320789+0000 (oldest deadline 2024-04-25T00:28:53.020814+0000) 2024-04-25T00:29:00.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:00 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:00.297+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:33.021381+0000 front 2024-04-25T00:28:33.021452+0000 (oldest deadline 2024-04-25T00:28:58.321264+0000) 2024-04-25T00:29:00.838 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:29:00 smithi107 podman[96284]: 2024-04-25 00:29:00.512331894 +0000 UTC m=+5.631761708 container died bf0a854faba3268c20a132b490a823481b80825d387496e25fdc47570ffa544f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T00:29:01.130 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:29:00 smithi107 podman[96284]: 2024-04-25 00:29:00.929094023 +0000 UTC m=+6.048523845 container cleanup bf0a854faba3268c20a132b490a823481b80825d387496e25fdc47570ffa544f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-25T00:29:01.130 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:29:00 smithi107 bash[96284]: ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9 2024-04-25T00:29:01.150 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:00 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:00.802+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:29:01.150 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:00 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:00.802+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:29:01.150 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:00 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:00.802+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:29:01.150 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:00 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:00.802+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:29:01.150 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:00 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:00.802+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.152669+0000 front 2024-04-25T00:28:20.152625+0000 (oldest deadline 2024-04-25T00:28:45.452796+0000) 2024-04-25T00:29:01.151 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:00 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:00.802+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:27.753658+0000 front 2024-04-25T00:28:27.753711+0000 (oldest deadline 2024-04-25T00:28:50.053645+0000) 2024-04-25T00:29:01.151 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:00 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:00.802+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:35.354905+0000 front 2024-04-25T00:28:35.354785+0000 (oldest deadline 2024-04-25T00:28:58.854593+0000) 2024-04-25T00:29:01.484 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:29:01 smithi107 podman[96302]: 2024-04-25 00:29:01.130152989 +0000 UTC m=+0.609398050 container remove bf0a854faba3268c20a132b490a823481b80825d387496e25fdc47570ffa544f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T00:29:01.833 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:29:01 smithi107 podman[96390]: 2024-04-25 00:29:01.539379693 +0000 UTC m=+0.109775176 container create a1df00b0656ce0d28b27fbc26bd7f7dda1b9dd8cf8d57167d50d0cd4d16686c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True) 2024-04-25T00:29:01.833 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:29:01 smithi107 podman[96390]: 2024-04-25 00:29:01.447151872 +0000 UTC m=+0.017547296 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:29:01.833 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:29:01 smithi107 podman[96390]: 2024-04-25 00:29:01.758129071 +0000 UTC m=+0.328524481 container init a1df00b0656ce0d28b27fbc26bd7f7dda1b9dd8cf8d57167d50d0cd4d16686c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:29:01.833 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:29:01 smithi107 podman[96390]: 2024-04-25 00:29:01.761221752 +0000 UTC m=+0.331617168 container start a1df00b0656ce0d28b27fbc26bd7f7dda1b9dd8cf8d57167d50d0cd4d16686c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T00:29:01.833 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:29:01 smithi107 podman[96390]: 2024-04-25 00:29:01.833053403 +0000 UTC m=+0.403448817 container attach a1df00b0656ce0d28b27fbc26bd7f7dda1b9dd8cf8d57167d50d0cd4d16686c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, ceph=True) 2024-04-25T00:29:01.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:01 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:01.338+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:29:01.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:01 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:01.338+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:29:01.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:01 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:01.338+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:29:01.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:01 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:01.338+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:29:01.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:01 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:01.338+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:28.320719+0000 front 2024-04-25T00:28:28.320789+0000 (oldest deadline 2024-04-25T00:28:53.020814+0000) 2024-04-25T00:29:01.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:01 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:01.338+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:33.021381+0000 front 2024-04-25T00:28:33.021452+0000 (oldest deadline 2024-04-25T00:28:58.321264+0000) 2024-04-25T00:29:02.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:01 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:01.826+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:29:02.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:01 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:01.826+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:29:02.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:01 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:01.826+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:29:02.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:01 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:01.826+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:29:02.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:01 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:01.826+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.152669+0000 front 2024-04-25T00:28:20.152625+0000 (oldest deadline 2024-04-25T00:28:45.452796+0000) 2024-04-25T00:29:02.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:01 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:01.826+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:27.753658+0000 front 2024-04-25T00:28:27.753711+0000 (oldest deadline 2024-04-25T00:28:50.053645+0000) 2024-04-25T00:29:02.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:01 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:01.826+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:35.354905+0000 front 2024-04-25T00:28:35.354785+0000 (oldest deadline 2024-04-25T00:28:58.854593+0000) 2024-04-25T00:29:02.234 INFO:journalctl@ceph.osd.9.smithi107.stdout:Apr 25 00:29:02 smithi107 podman[96390]: 2024-04-25 00:29:02.047233988 +0000 UTC m=+0.617629399 container died a1df00b0656ce0d28b27fbc26bd7f7dda1b9dd8cf8d57167d50d0cd4d16686c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=) 2024-04-25T00:29:02.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:02 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:02.330+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:29:02.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:02 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:02.330+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:29:02.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:02 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:02.330+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:29:02.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:02 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:02.330+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:29:02.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:02 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:02.330+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:28.320719+0000 front 2024-04-25T00:28:28.320789+0000 (oldest deadline 2024-04-25T00:28:53.020814+0000) 2024-04-25T00:29:02.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:02 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:02.330+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:33.021381+0000 front 2024-04-25T00:28:33.021452+0000 (oldest deadline 2024-04-25T00:28:58.321264+0000) 2024-04-25T00:29:02.592 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T00:29:02.628 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:29:02.628 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T00:29:02.629 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T00:29:02.629 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.10 2024-04-25T00:29:02.935 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:02 smithi170 systemd[1]: Stopping Ceph osd.10 for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:29:02.935 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:02 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:02.875+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:29:02.935 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:02 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:02.875+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:29:02.935 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:02 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:02.875+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:29:02.935 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:02 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:02.875+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:29:02.935 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:02 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:02.875+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.152669+0000 front 2024-04-25T00:28:20.152625+0000 (oldest deadline 2024-04-25T00:28:45.452796+0000) 2024-04-25T00:29:02.935 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:02 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:02.875+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:27.753658+0000 front 2024-04-25T00:28:27.753711+0000 (oldest deadline 2024-04-25T00:28:50.053645+0000) 2024-04-25T00:29:02.935 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:02 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:02.875+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:35.354905+0000 front 2024-04-25T00:28:35.354785+0000 (oldest deadline 2024-04-25T00:28:58.854593+0000) 2024-04-25T00:29:03.435 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:02 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:02.965+0000 7fd9f4094700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:29:03.435 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:02 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:02.965+0000 7fd9f4094700 -1 osd.10 111 *** Got signal Terminated *** 2024-04-25T00:29:03.435 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:02 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:02.966+0000 7fd9f4094700 -1 osd.10 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:29:03.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:03 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:03.347+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:29:03.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:03 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:03.347+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:29:03.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:03 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:03.347+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:29:03.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:03 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:03.347+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:29:03.840 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:03 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:03.347+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:28.320719+0000 front 2024-04-25T00:28:28.320789+0000 (oldest deadline 2024-04-25T00:28:53.020814+0000) 2024-04-25T00:29:03.840 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:03 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:03.347+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:33.021381+0000 front 2024-04-25T00:28:33.021452+0000 (oldest deadline 2024-04-25T00:28:58.321264+0000) 2024-04-25T00:29:04.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:03 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:03.885+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:29:04.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:03 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:03.885+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:29:04.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:03 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:03.885+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:29:04.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:03 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:03.885+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:29:04.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:03 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:03.885+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.152669+0000 front 2024-04-25T00:28:20.152625+0000 (oldest deadline 2024-04-25T00:28:45.452796+0000) 2024-04-25T00:29:04.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:03 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:03.885+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:27.753658+0000 front 2024-04-25T00:28:27.753711+0000 (oldest deadline 2024-04-25T00:28:50.053645+0000) 2024-04-25T00:29:04.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:03 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:03.885+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:35.354905+0000 front 2024-04-25T00:28:35.354785+0000 (oldest deadline 2024-04-25T00:28:58.854593+0000) 2024-04-25T00:29:04.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:04 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:04.343+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:29:04.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:04 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:04.343+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:29:04.840 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:04 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:04.343+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:29:04.840 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:04 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:04.343+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:29:04.840 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:04 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:04.343+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:28.320719+0000 front 2024-04-25T00:28:28.320789+0000 (oldest deadline 2024-04-25T00:28:53.020814+0000) 2024-04-25T00:29:04.840 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:04 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:04.343+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:33.021381+0000 front 2024-04-25T00:28:33.021452+0000 (oldest deadline 2024-04-25T00:28:58.321264+0000) 2024-04-25T00:29:05.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:04 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:04.859+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:29:05.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:04 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:04.859+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:29:05.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:04 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:04.859+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:29:05.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:04 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:04.859+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:29:05.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:04 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:04.859+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.152669+0000 front 2024-04-25T00:28:20.152625+0000 (oldest deadline 2024-04-25T00:28:45.452796+0000) 2024-04-25T00:29:05.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:04 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:04.859+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:27.753658+0000 front 2024-04-25T00:28:27.753711+0000 (oldest deadline 2024-04-25T00:28:50.053645+0000) 2024-04-25T00:29:05.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:04 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:04.859+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:35.354905+0000 front 2024-04-25T00:28:35.354785+0000 (oldest deadline 2024-04-25T00:28:58.854593+0000) 2024-04-25T00:29:05.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:05 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:05.368+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:29:05.840 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:05 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:05.368+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:29:05.840 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:05 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:05.368+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:29:05.840 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:05 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:05.368+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:29:05.840 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:05 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:05.368+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:28.320719+0000 front 2024-04-25T00:28:28.320789+0000 (oldest deadline 2024-04-25T00:28:53.020814+0000) 2024-04-25T00:29:05.840 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:05 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:05.368+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:33.021381+0000 front 2024-04-25T00:28:33.021452+0000 (oldest deadline 2024-04-25T00:28:58.321264+0000) 2024-04-25T00:29:06.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:05 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:05.822+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:29:06.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:05 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:05.822+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:29:06.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:05 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:05.822+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:29:06.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:05 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:05.823+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:29:06.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:05 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:05.823+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.152669+0000 front 2024-04-25T00:28:20.152625+0000 (oldest deadline 2024-04-25T00:28:45.452796+0000) 2024-04-25T00:29:06.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:05 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:05.823+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:27.753658+0000 front 2024-04-25T00:28:27.753711+0000 (oldest deadline 2024-04-25T00:28:50.053645+0000) 2024-04-25T00:29:06.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:05 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:05.823+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:35.354905+0000 front 2024-04-25T00:28:35.354785+0000 (oldest deadline 2024-04-25T00:28:58.854593+0000) 2024-04-25T00:29:06.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:06 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:06.377+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:29:06.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:06 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:06.377+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:29:06.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:06 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:06.377+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:29:06.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:06 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:06.377+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:29:06.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:06 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:06.377+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:28.320719+0000 front 2024-04-25T00:28:28.320789+0000 (oldest deadline 2024-04-25T00:28:53.020814+0000) 2024-04-25T00:29:06.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:06 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:06.377+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:33.021381+0000 front 2024-04-25T00:28:33.021452+0000 (oldest deadline 2024-04-25T00:28:58.321264+0000) 2024-04-25T00:29:07.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:06 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:06.824+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:29:07.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:06 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:06.824+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:29:07.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:06 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:06.824+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:29:07.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:06 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:06.824+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:29:07.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:06 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:06.824+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.152669+0000 front 2024-04-25T00:28:20.152625+0000 (oldest deadline 2024-04-25T00:28:45.452796+0000) 2024-04-25T00:29:07.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:06 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:06.824+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:27.753658+0000 front 2024-04-25T00:28:27.753711+0000 (oldest deadline 2024-04-25T00:28:50.053645+0000) 2024-04-25T00:29:07.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:06 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:06.824+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:35.354905+0000 front 2024-04-25T00:28:35.354785+0000 (oldest deadline 2024-04-25T00:28:58.854593+0000) 2024-04-25T00:29:07.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:07 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:07.351+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:29:07.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:07 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:07.351+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:29:07.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:07 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:07.351+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:29:07.840 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:07 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:07.351+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:29:07.840 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:07 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:07.351+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:28.320719+0000 front 2024-04-25T00:28:28.320789+0000 (oldest deadline 2024-04-25T00:28:53.020814+0000) 2024-04-25T00:29:07.840 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:07 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:07.351+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:33.021381+0000 front 2024-04-25T00:28:33.021452+0000 (oldest deadline 2024-04-25T00:28:58.321264+0000) 2024-04-25T00:29:08.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:07 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:07.795+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:48.047543+0000 front 2024-04-25T00:27:48.047460+0000 (oldest deadline 2024-04-25T00:28:10.347598+0000) 2024-04-25T00:29:08.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:07 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:07.795+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6804 osd.1 since back 2024-04-25T00:27:55.048632+0000 front 2024-04-25T00:27:55.048714+0000 (oldest deadline 2024-04-25T00:28:19.748779+0000) 2024-04-25T00:29:08.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:07 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:07.795+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:03.249751+0000 front 2024-04-25T00:28:03.249886+0000 (oldest deadline 2024-04-25T00:28:24.949822+0000) 2024-04-25T00:29:08.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:07 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:07.795+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:08.451171+0000 front 2024-04-25T00:28:08.451061+0000 (oldest deadline 2024-04-25T00:28:34.350982+0000) 2024-04-25T00:29:08.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:07 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:07.795+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.152669+0000 front 2024-04-25T00:28:20.152625+0000 (oldest deadline 2024-04-25T00:28:45.452796+0000) 2024-04-25T00:29:08.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:07 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:07.795+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:27.753658+0000 front 2024-04-25T00:28:27.753711+0000 (oldest deadline 2024-04-25T00:28:50.053645+0000) 2024-04-25T00:29:08.185 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:07 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10[57393]: 2024-04-25T00:29:07.795+0000 7fd9ec680700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:35.354905+0000 front 2024-04-25T00:28:35.354785+0000 (oldest deadline 2024-04-25T00:28:58.854593+0000) 2024-04-25T00:29:08.685 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:08 smithi170 podman[69097]: 2024-04-25 00:29:08.420655098 +0000 UTC m=+5.593849422 container died 41f0ab43c0d19db5975902eccc3016002bd9fb4afa91315c86222920fefb621a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:29:08.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:08 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:08.350+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:29:08.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:08 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:08.350+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:29:08.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:08 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:08.350+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:29:08.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:08 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:08.350+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:29:08.840 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:08 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:08.350+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:28.320719+0000 front 2024-04-25T00:28:28.320789+0000 (oldest deadline 2024-04-25T00:28:53.020814+0000) 2024-04-25T00:29:08.840 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:08 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:08.350+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:33.021381+0000 front 2024-04-25T00:28:33.021452+0000 (oldest deadline 2024-04-25T00:28:58.321264+0000) 2024-04-25T00:29:08.840 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:08 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:08.350+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6820 osd.7 since back 2024-04-25T00:28:42.322905+0000 front 2024-04-25T00:28:42.322875+0000 (oldest deadline 2024-04-25T00:29:08.222699+0000) 2024-04-25T00:29:08.957 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:08 smithi170 podman[69097]: 2024-04-25 00:29:08.764775998 +0000 UTC m=+5.937970320 container cleanup 41f0ab43c0d19db5975902eccc3016002bd9fb4afa91315c86222920fefb621a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:29:08.957 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:08 smithi170 bash[69097]: ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10 2024-04-25T00:29:09.366 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:08 smithi170 podman[69114]: 2024-04-25 00:29:08.957646583 +0000 UTC m=+0.537607518 container remove 41f0ab43c0d19db5975902eccc3016002bd9fb4afa91315c86222920fefb621a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T00:29:09.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:09 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:09.301+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:29:09.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:09 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:09.301+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:29:09.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:09 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:09.301+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:29:09.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:09 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:09.301+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:29:09.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:09 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:09.301+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:28.320719+0000 front 2024-04-25T00:28:28.320789+0000 (oldest deadline 2024-04-25T00:28:53.020814+0000) 2024-04-25T00:29:09.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:09 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:09.301+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:33.021381+0000 front 2024-04-25T00:28:33.021452+0000 (oldest deadline 2024-04-25T00:28:58.321264+0000) 2024-04-25T00:29:09.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:09 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:09.301+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6820 osd.7 since back 2024-04-25T00:28:42.322905+0000 front 2024-04-25T00:28:42.322875+0000 (oldest deadline 2024-04-25T00:29:08.222699+0000) 2024-04-25T00:29:09.636 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:09 smithi170 podman[69203]: 2024-04-25 00:29:09.366782924 +0000 UTC m=+0.104584546 container create 6dbc0b2b59ca4b8ffb15aa4b35d0017263ed8175c00a4853eefd6723bbca79a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9) 2024-04-25T00:29:09.636 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:09 smithi170 podman[69203]: 2024-04-25 00:29:09.279859089 +0000 UTC m=+0.017660729 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:29:09.636 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:09 smithi170 podman[69203]: 2024-04-25 00:29:09.56102813 +0000 UTC m=+0.298829753 container init 6dbc0b2b59ca4b8ffb15aa4b35d0017263ed8175c00a4853eefd6723bbca79a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T00:29:09.636 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:09 smithi170 podman[69203]: 2024-04-25 00:29:09.564424578 +0000 UTC m=+0.302226201 container start 6dbc0b2b59ca4b8ffb15aa4b35d0017263ed8175c00a4853eefd6723bbca79a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-25T00:29:09.636 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:09 smithi170 podman[69203]: 2024-04-25 00:29:09.635978157 +0000 UTC m=+0.373779789 container attach 6dbc0b2b59ca4b8ffb15aa4b35d0017263ed8175c00a4853eefd6723bbca79a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T00:29:09.934 INFO:journalctl@ceph.osd.10.smithi170.stdout:Apr 25 00:29:09 smithi170 podman[69203]: 2024-04-25 00:29:09.860586387 +0000 UTC m=+0.598388018 container died 6dbc0b2b59ca4b8ffb15aa4b35d0017263ed8175c00a4853eefd6723bbca79a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T00:29:10.441 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T00:29:10.479 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:29:10.480 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T00:29:10.480 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T00:29:10.480 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.11 2024-04-25T00:29:10.524 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:10 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:10.267+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:29:10.524 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:10 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:10.267+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:29:10.524 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:10 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:10.267+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:29:10.524 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:10 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:10.267+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:29:10.524 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:10 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:10.267+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:28.320719+0000 front 2024-04-25T00:28:28.320789+0000 (oldest deadline 2024-04-25T00:28:53.020814+0000) 2024-04-25T00:29:10.524 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:10 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:10.267+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:33.021381+0000 front 2024-04-25T00:28:33.021452+0000 (oldest deadline 2024-04-25T00:28:58.321264+0000) 2024-04-25T00:29:10.524 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:10 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:10.267+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6820 osd.7 since back 2024-04-25T00:28:42.322905+0000 front 2024-04-25T00:28:42.322875+0000 (oldest deadline 2024-04-25T00:29:08.222699+0000) 2024-04-25T00:29:10.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:10 smithi195 systemd[1]: Stopping Ceph osd.11 for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:29:10.839 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:10 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:10.760+0000 7fc6a0661700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:29:10.840 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:10 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:10.760+0000 7fc6a0661700 -1 osd.11 111 *** Got signal Terminated *** 2024-04-25T00:29:10.840 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:10 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:10.760+0000 7fc6a0661700 -1 osd.11 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:29:11.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:11 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:11.273+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:29:11.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:11 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:11.273+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:29:11.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:11 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:11.273+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:29:11.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:11 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:11.273+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:29:11.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:11 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:11.273+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:28.320719+0000 front 2024-04-25T00:28:28.320789+0000 (oldest deadline 2024-04-25T00:28:53.020814+0000) 2024-04-25T00:29:11.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:11 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:11.273+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:33.021381+0000 front 2024-04-25T00:28:33.021452+0000 (oldest deadline 2024-04-25T00:28:58.321264+0000) 2024-04-25T00:29:11.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:11 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:11.273+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6820 osd.7 since back 2024-04-25T00:28:42.322905+0000 front 2024-04-25T00:28:42.322875+0000 (oldest deadline 2024-04-25T00:29:08.222699+0000) 2024-04-25T00:29:12.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:12 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:12.297+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:29:12.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:12 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:12.297+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:29:12.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:12 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:12.297+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:29:12.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:12 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:12.297+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:29:12.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:12 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:12.297+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:28.320719+0000 front 2024-04-25T00:28:28.320789+0000 (oldest deadline 2024-04-25T00:28:53.020814+0000) 2024-04-25T00:29:12.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:12 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:12.297+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:33.021381+0000 front 2024-04-25T00:28:33.021452+0000 (oldest deadline 2024-04-25T00:28:58.321264+0000) 2024-04-25T00:29:12.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:12 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:12.297+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6820 osd.7 since back 2024-04-25T00:28:42.322905+0000 front 2024-04-25T00:28:42.322875+0000 (oldest deadline 2024-04-25T00:29:08.222699+0000) 2024-04-25T00:29:13.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:13 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:13.312+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:29:13.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:13 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:13.312+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:29:13.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:13 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:13.312+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:29:13.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:13 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:13.312+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:29:13.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:13 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:13.312+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:28.320719+0000 front 2024-04-25T00:28:28.320789+0000 (oldest deadline 2024-04-25T00:28:53.020814+0000) 2024-04-25T00:29:13.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:13 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:13.312+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:33.021381+0000 front 2024-04-25T00:28:33.021452+0000 (oldest deadline 2024-04-25T00:28:58.321264+0000) 2024-04-25T00:29:13.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:13 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:13.312+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6820 osd.7 since back 2024-04-25T00:28:42.322905+0000 front 2024-04-25T00:28:42.322875+0000 (oldest deadline 2024-04-25T00:29:08.222699+0000) 2024-04-25T00:29:13.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:13 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:13.312+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6820 osd.8 since back 2024-04-25T00:28:48.223029+0000 front 2024-04-25T00:28:48.223091+0000 (oldest deadline 2024-04-25T00:29:12.923094+0000) 2024-04-25T00:29:14.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:14 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:14.287+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:29:14.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:14 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:14.287+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:29:14.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:14 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:14.287+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:29:14.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:14 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:14.287+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:29:14.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:14 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:14.287+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:28.320719+0000 front 2024-04-25T00:28:28.320789+0000 (oldest deadline 2024-04-25T00:28:53.020814+0000) 2024-04-25T00:29:14.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:14 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:14.287+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:33.021381+0000 front 2024-04-25T00:28:33.021452+0000 (oldest deadline 2024-04-25T00:28:58.321264+0000) 2024-04-25T00:29:14.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:14 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:14.287+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6820 osd.7 since back 2024-04-25T00:28:42.322905+0000 front 2024-04-25T00:28:42.322875+0000 (oldest deadline 2024-04-25T00:29:08.222699+0000) 2024-04-25T00:29:14.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:14 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:14.287+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6820 osd.8 since back 2024-04-25T00:28:48.223029+0000 front 2024-04-25T00:28:48.223091+0000 (oldest deadline 2024-04-25T00:29:12.923094+0000) 2024-04-25T00:29:15.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:15 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:15.248+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-04-25T00:27:47.713362+0000 front 2024-04-25T00:27:47.713339+0000 (oldest deadline 2024-04-25T00:28:10.013268+0000) 2024-04-25T00:29:15.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:15 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:15.248+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6804 osd.2 since back 2024-04-25T00:28:04.115299+0000 front 2024-04-25T00:28:04.115319+0000 (oldest deadline 2024-04-25T00:28:25.715954+0000) 2024-04-25T00:29:15.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:15 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:15.248+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-04-25T00:28:09.717506+0000 front 2024-04-25T00:28:09.717629+0000 (oldest deadline 2024-04-25T00:28:34.417434+0000) 2024-04-25T00:29:15.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:15 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:15.248+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6812 osd.4 since back 2024-04-25T00:28:20.219039+0000 front 2024-04-25T00:28:20.219150+0000 (oldest deadline 2024-04-25T00:28:44.819721+0000) 2024-04-25T00:29:15.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:15 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:15.248+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6812 osd.5 since back 2024-04-25T00:28:28.320719+0000 front 2024-04-25T00:28:28.320789+0000 (oldest deadline 2024-04-25T00:28:53.020814+0000) 2024-04-25T00:29:15.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:15 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:15.248+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-04-25T00:28:33.021381+0000 front 2024-04-25T00:28:33.021452+0000 (oldest deadline 2024-04-25T00:28:58.321264+0000) 2024-04-25T00:29:15.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:15 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:15.248+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.170:6820 osd.7 since back 2024-04-25T00:28:42.322905+0000 front 2024-04-25T00:28:42.322875+0000 (oldest deadline 2024-04-25T00:29:08.222699+0000) 2024-04-25T00:29:15.590 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:15 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11[58758]: 2024-04-25T00:29:15.248+0000 7fc69cc55700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.195:6820 osd.8 since back 2024-04-25T00:28:48.223029+0000 front 2024-04-25T00:28:48.223091+0000 (oldest deadline 2024-04-25T00:29:12.923094+0000) 2024-04-25T00:29:16.427 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:16 smithi195 podman[70692]: 2024-04-25 00:29:16.135174242 +0000 UTC m=+5.464297162 container died 72d775667ced2951c355adf248e9da2c2342af171561da2bc3a8e8238620c43d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T00:29:16.427 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:16 smithi195 podman[70692]: 2024-04-25 00:29:16.427065803 +0000 UTC m=+5.756188707 container cleanup 72d775667ced2951c355adf248e9da2c2342af171561da2bc3a8e8238620c43d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T00:29:16.678 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:16 smithi195 bash[70692]: ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11 2024-04-25T00:29:16.991 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:16 smithi195 podman[70709]: 2024-04-25 00:29:16.678227425 +0000 UTC m=+0.539837914 container remove 72d775667ced2951c355adf248e9da2c2342af171561da2bc3a8e8238620c43d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-25T00:29:17.265 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:16 smithi195 podman[70795]: 2024-04-25 00:29:16.894042433 +0000 UTC m=+0.018187166 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:29:17.265 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:17 smithi195 podman[70795]: 2024-04-25 00:29:17.012799236 +0000 UTC m=+0.136943938 container create 985c5fbb9a1f0db4604864183d71d627688976fef868894e43359858fce7f4fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:29:17.265 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:17 smithi195 podman[70795]: 2024-04-25 00:29:17.206977305 +0000 UTC m=+0.331122018 container init 985c5fbb9a1f0db4604864183d71d627688976fef868894e43359858fce7f4fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T00:29:17.265 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:17 smithi195 podman[70795]: 2024-04-25 00:29:17.210103368 +0000 UTC m=+0.334248084 container start 985c5fbb9a1f0db4604864183d71d627688976fef868894e43359858fce7f4fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T00:29:17.265 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:17 smithi195 podman[70795]: 2024-04-25 00:29:17.265081528 +0000 UTC m=+0.389226240 container attach 985c5fbb9a1f0db4604864183d71d627688976fef868894e43359858fce7f4fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:29:17.589 INFO:journalctl@ceph.osd.11.smithi195.stdout:Apr 25 00:29:17 smithi195 podman[70795]: 2024-04-25 00:29:17.508280764 +0000 UTC m=+0.632425481 container died 985c5fbb9a1f0db4604864183d71d627688976fef868894e43359858fce7f4fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T00:29:18.076 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T00:29:18.154 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:29:18.154 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T00:29:18.154 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T00:29:18.154 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.a 2024-04-25T00:29:18.445 INFO:journalctl@ceph.mds.a.smithi107.stdout:Apr 25 00:29:18 smithi107 systemd[1]: Stopping Ceph mds.a for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:29:18.734 INFO:journalctl@ceph.mds.a.smithi107.stdout:Apr 25 00:29:18 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-a[67963]: 2024-04-25T00:29:18.443+0000 7f5bea7d6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:29:18.734 INFO:journalctl@ceph.mds.a.smithi107.stdout:Apr 25 00:29:18 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-a[67963]: 2024-04-25T00:29:18.443+0000 7f5bea7d6700 -1 mds.a *** got signal Terminated *** 2024-04-25T00:29:22.728 INFO:journalctl@ceph.mds.a.smithi107.stdout:Apr 25 00:29:22 smithi107 podman[96494]: 2024-04-25 00:29:22.451496472 +0000 UTC m=+4.108881530 container died 956c23a13e3b12b48ec09edff62cdd9e21e53f008c0d337d324df9fe733f5df3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:29:22.984 INFO:journalctl@ceph.mds.a.smithi107.stdout:Apr 25 00:29:22 smithi107 podman[96494]: 2024-04-25 00:29:22.852465794 +0000 UTC m=+4.509850857 container cleanup 956c23a13e3b12b48ec09edff62cdd9e21e53f008c0d337d324df9fe733f5df3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-a, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T00:29:22.984 INFO:journalctl@ceph.mds.a.smithi107.stdout:Apr 25 00:29:22 smithi107 bash[96494]: ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-a 2024-04-25T00:29:23.217 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T00:29:23.250 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:29:23.250 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T00:29:23.251 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T00:29:23.251 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.d 2024-04-25T00:29:23.484 INFO:journalctl@ceph.mds.d.smithi107.stdout:Apr 25 00:29:23 smithi107 systemd[1]: Stopping Ceph mds.d for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:29:23.984 INFO:journalctl@ceph.mds.d.smithi107.stdout:Apr 25 00:29:23 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-d[67002]: 2024-04-25T00:29:23.571+0000 7f4d7d40a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:29:23.984 INFO:journalctl@ceph.mds.d.smithi107.stdout:Apr 25 00:29:23 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-d[67002]: 2024-04-25T00:29:23.571+0000 7f4d7d40a700 -1 mds.d *** got signal Terminated *** 2024-04-25T00:29:27.298 INFO:journalctl@ceph.mds.d.smithi107.stdout:Apr 25 00:29:27 smithi107 podman[96617]: 2024-04-25 00:29:27.012726315 +0000 UTC m=+3.552171272 container died df3dd6c9d0de6bed60777bd070d7f1053733ca867888c8c46818917ace4f1425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T00:29:27.673 INFO:journalctl@ceph.mds.d.smithi107.stdout:Apr 25 00:29:27 smithi107 podman[96617]: 2024-04-25 00:29:27.397104852 +0000 UTC m=+3.936549811 container cleanup df3dd6c9d0de6bed60777bd070d7f1053733ca867888c8c46818917ace4f1425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T00:29:27.674 INFO:journalctl@ceph.mds.d.smithi107.stdout:Apr 25 00:29:27 smithi107 bash[96617]: ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-d 2024-04-25T00:29:27.822 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T00:29:27.855 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:29:27.855 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T00:29:27.855 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T00:29:27.855 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.g 2024-04-25T00:29:28.234 INFO:journalctl@ceph.mds.g.smithi107.stdout:Apr 25 00:29:27 smithi107 systemd[1]: Stopping Ceph mds.g for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:29:28.234 INFO:journalctl@ceph.mds.g.smithi107.stdout:Apr 25 00:29:28 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-g[67607]: 2024-04-25T00:29:28.174+0000 7fc28aa47700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:29:28.234 INFO:journalctl@ceph.mds.g.smithi107.stdout:Apr 25 00:29:28 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-g[67607]: 2024-04-25T00:29:28.174+0000 7fc28aa47700 -1 mds.g *** got signal Terminated *** 2024-04-25T00:29:32.185 INFO:journalctl@ceph.mds.g.smithi107.stdout:Apr 25 00:29:31 smithi107 podman[96750]: 2024-04-25 00:29:31.783099687 +0000 UTC m=+3.711655332 container died 631b408df0e5cb9cc4ef3baa7c6bd4def536a5325bd09299d2210b52f25ebe1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T00:29:32.484 INFO:journalctl@ceph.mds.g.smithi107.stdout:Apr 25 00:29:32 smithi107 podman[96750]: 2024-04-25 00:29:32.258860716 +0000 UTC m=+4.187416366 container cleanup 631b408df0e5cb9cc4ef3baa7c6bd4def536a5325bd09299d2210b52f25ebe1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T00:29:32.484 INFO:journalctl@ceph.mds.g.smithi107.stdout:Apr 25 00:29:32 smithi107 bash[96750]: ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-g 2024-04-25T00:29:32.648 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T00:29:32.683 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:29:32.683 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T00:29:32.683 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T00:29:32.684 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.j 2024-04-25T00:29:33.234 INFO:journalctl@ceph.mds.j.smithi107.stdout:Apr 25 00:29:32 smithi107 systemd[1]: Stopping Ceph mds.j for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:29:33.234 INFO:journalctl@ceph.mds.j.smithi107.stdout:Apr 25 00:29:32 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-j[66653]: 2024-04-25T00:29:32.986+0000 7f9da8126700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:29:33.234 INFO:journalctl@ceph.mds.j.smithi107.stdout:Apr 25 00:29:32 smithi107 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-j[66653]: 2024-04-25T00:29:32.986+0000 7f9da8126700 -1 mds.j *** got signal Terminated *** 2024-04-25T00:29:36.484 INFO:journalctl@ceph.mds.j.smithi107.stdout:Apr 25 00:29:36 smithi107 podman[96870]: 2024-04-25 00:29:36.22098181 +0000 UTC m=+3.328881564 container died 260019258da807624c706215e501e7c6c75f508e9836cc3ce41b229ab983b647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:29:36.830 INFO:journalctl@ceph.mds.j.smithi107.stdout:Apr 25 00:29:36 smithi107 podman[96870]: 2024-04-25 00:29:36.528220024 +0000 UTC m=+3.636119767 container cleanup 260019258da807624c706215e501e7c6c75f508e9836cc3ce41b229ab983b647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T00:29:36.830 INFO:journalctl@ceph.mds.j.smithi107.stdout:Apr 25 00:29:36 smithi107 bash[96870]: ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-j 2024-04-25T00:29:36.830 INFO:journalctl@ceph.mds.j.smithi107.stdout:Apr 25 00:29:36 smithi107 podman[96884]: 2024-04-25 00:29:36.721145113 +0000 UTC m=+0.490956807 container remove 260019258da807624c706215e501e7c6c75f508e9836cc3ce41b229ab983b647 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:29:36.841 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T00:29:36.872 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:29:36.873 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T00:29:36.873 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T00:29:36.873 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.b 2024-04-25T00:29:37.181 INFO:journalctl@ceph.mds.b.smithi170.stdout:Apr 25 00:29:36 smithi170 systemd[1]: Stopping Ceph mds.b for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:29:37.434 INFO:journalctl@ceph.mds.b.smithi170.stdout:Apr 25 00:29:37 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-b[62656]: 2024-04-25T00:29:37.180+0000 7f27037dc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:29:37.435 INFO:journalctl@ceph.mds.b.smithi170.stdout:Apr 25 00:29:37 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-b[62656]: 2024-04-25T00:29:37.180+0000 7f27037dc700 -1 mds.b *** got signal Terminated *** 2024-04-25T00:29:39.415 INFO:journalctl@ceph.mds.b.smithi170.stdout:Apr 25 00:29:39 smithi170 podman[69309]: 2024-04-25 00:29:39.159716094 +0000 UTC m=+2.091283629 container died d651c23f6a0c494ae26f34eaae19664c3242e2a6f296953ee11b75324e176a78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:29:39.684 INFO:journalctl@ceph.mds.b.smithi170.stdout:Apr 25 00:29:39 smithi170 podman[69309]: 2024-04-25 00:29:39.514851419 +0000 UTC m=+2.446418955 container cleanup d651c23f6a0c494ae26f34eaae19664c3242e2a6f296953ee11b75324e176a78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T00:29:39.685 INFO:journalctl@ceph.mds.b.smithi170.stdout:Apr 25 00:29:39 smithi170 bash[69309]: ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-b 2024-04-25T00:29:39.914 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T00:29:39.949 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:29:39.949 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T00:29:39.949 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T00:29:39.949 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.e 2024-04-25T00:29:40.184 INFO:journalctl@ceph.mds.e.smithi170.stdout:Apr 25 00:29:40 smithi170 systemd[1]: Stopping Ceph mds.e for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:29:40.685 INFO:journalctl@ceph.mds.e.smithi170.stdout:Apr 25 00:29:40 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-e[62024]: 2024-04-25T00:29:40.284+0000 7fef090c9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:29:40.685 INFO:journalctl@ceph.mds.e.smithi170.stdout:Apr 25 00:29:40 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-e[62024]: 2024-04-25T00:29:40.284+0000 7fef090c9700 -1 mds.e *** got signal Terminated *** 2024-04-25T00:29:43.428 INFO:journalctl@ceph.mds.e.smithi170.stdout:Apr 25 00:29:43 smithi170 podman[69442]: 2024-04-25 00:29:43.155175391 +0000 UTC m=+2.988903426 container died 2bd248e19770260215a5b5acb5f4e893f71dd94577dadb0b0b585059d5f9b57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-e, RELEASE=squid-d2c45d9, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:29:43.685 INFO:journalctl@ceph.mds.e.smithi170.stdout:Apr 25 00:29:43 smithi170 podman[69442]: 2024-04-25 00:29:43.501921031 +0000 UTC m=+3.335649052 container cleanup 2bd248e19770260215a5b5acb5f4e893f71dd94577dadb0b0b585059d5f9b57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-e, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T00:29:43.685 INFO:journalctl@ceph.mds.e.smithi170.stdout:Apr 25 00:29:43 smithi170 bash[69442]: ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-e 2024-04-25T00:29:43.910 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T00:29:43.942 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:29:43.943 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T00:29:43.945 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T00:29:43.945 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.h 2024-04-25T00:29:44.435 INFO:journalctl@ceph.mds.h.smithi170.stdout:Apr 25 00:29:44 smithi170 systemd[1]: Stopping Ceph mds.h for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:29:44.435 INFO:journalctl@ceph.mds.h.smithi170.stdout:Apr 25 00:29:44 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-h[62346]: 2024-04-25T00:29:44.247+0000 7f424ff3e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:29:44.435 INFO:journalctl@ceph.mds.h.smithi170.stdout:Apr 25 00:29:44 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-h[62346]: 2024-04-25T00:29:44.247+0000 7f424ff3e700 -1 mds.h *** got signal Terminated *** 2024-04-25T00:29:47.415 INFO:journalctl@ceph.mds.h.smithi170.stdout:Apr 25 00:29:47 smithi170 podman[69563]: 2024-04-25 00:29:47.157995177 +0000 UTC m=+3.002288987 container died bf9945b981a9839a10bd91042134e6734c0dff8fb3a097bb68b35a0099f75b7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:29:47.685 INFO:journalctl@ceph.mds.h.smithi170.stdout:Apr 25 00:29:47 smithi170 podman[69563]: 2024-04-25 00:29:47.514273499 +0000 UTC m=+3.358567298 container cleanup bf9945b981a9839a10bd91042134e6734c0dff8fb3a097bb68b35a0099f75b7d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T00:29:47.685 INFO:journalctl@ceph.mds.h.smithi170.stdout:Apr 25 00:29:47 smithi170 bash[69563]: ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-h 2024-04-25T00:29:47.935 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T00:29:47.965 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:29:47.966 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T00:29:47.966 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T00:29:47.966 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.k 2024-04-25T00:29:48.430 INFO:journalctl@ceph.mds.k.smithi170.stdout:Apr 25 00:29:48 smithi170 systemd[1]: Stopping Ceph mds.k for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:29:48.430 INFO:journalctl@ceph.mds.k.smithi170.stdout:Apr 25 00:29:48 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-k[61707]: 2024-04-25T00:29:48.284+0000 7f22d0a30700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:29:48.430 INFO:journalctl@ceph.mds.k.smithi170.stdout:Apr 25 00:29:48 smithi170 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-k[61707]: 2024-04-25T00:29:48.284+0000 7f22d0a30700 -1 mds.k *** got signal Terminated *** 2024-04-25T00:29:49.685 INFO:journalctl@ceph.mds.k.smithi170.stdout:Apr 25 00:29:49 smithi170 podman[69683]: 2024-04-25 00:29:49.430814557 +0000 UTC m=+1.250905677 container died 94fc407f53443b7ff85e6b4f7b2f32b4af8ac5b32352477f13977d6165646f79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-k, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-25T00:29:49.944 INFO:journalctl@ceph.mds.k.smithi170.stdout:Apr 25 00:29:49 smithi170 podman[69683]: 2024-04-25 00:29:49.759419561 +0000 UTC m=+1.579510685 container cleanup 94fc407f53443b7ff85e6b4f7b2f32b4af8ac5b32352477f13977d6165646f79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-k, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:29:49.944 INFO:journalctl@ceph.mds.k.smithi170.stdout:Apr 25 00:29:49 smithi170 bash[69683]: ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-k 2024-04-25T00:29:50.190 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T00:29:50.210 INFO:journalctl@ceph.mds.k.smithi170.stdout:Apr 25 00:29:49 smithi170 podman[69697]: 2024-04-25 00:29:49.943890406 +0000 UTC m=+0.506495399 container remove 94fc407f53443b7ff85e6b4f7b2f32b4af8ac5b32352477f13977d6165646f79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-k, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:29:50.210 INFO:journalctl@ceph.mds.k.smithi170.stdout:Apr 25 00:29:50 smithi170 systemd[1]: ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.k.service: Deactivated successfully. 2024-04-25T00:29:50.210 INFO:journalctl@ceph.mds.k.smithi170.stdout:Apr 25 00:29:50 smithi170 systemd[1]: Stopped Ceph mds.k for 79602f58-0297-11ef-bc93-c7b262605968. 2024-04-25T00:29:50.210 INFO:journalctl@ceph.mds.k.smithi170.stdout:Apr 25 00:29:50 smithi170 systemd[1]: ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.k.service: Consumed 1.063s CPU time. 2024-04-25T00:29:50.221 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:29:50.221 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T00:29:50.221 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T00:29:50.221 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.c 2024-04-25T00:29:50.502 INFO:journalctl@ceph.mds.c.smithi195.stdout:Apr 25 00:29:50 smithi195 systemd[1]: Stopping Ceph mds.c for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:29:50.764 INFO:journalctl@ceph.mds.c.smithi195.stdout:Apr 25 00:29:50 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-c[63784]: 2024-04-25T00:29:50.501+0000 7fd2c252c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:29:50.765 INFO:journalctl@ceph.mds.c.smithi195.stdout:Apr 25 00:29:50 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-c[63784]: 2024-04-25T00:29:50.501+0000 7fd2c252c700 -1 mds.c *** got signal Terminated *** 2024-04-25T00:29:54.802 INFO:journalctl@ceph.mds.c.smithi195.stdout:Apr 25 00:29:54 smithi195 podman[70902]: 2024-04-25 00:29:54.509879015 +0000 UTC m=+4.093193303 container died 2eabc1c069458c00056194044b6b66ade50f14a35d6db4cca1a2eeec7867387e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T00:29:55.089 INFO:journalctl@ceph.mds.c.smithi195.stdout:Apr 25 00:29:54 smithi195 podman[70902]: 2024-04-25 00:29:54.876322488 +0000 UTC m=+4.459636826 container cleanup 2eabc1c069458c00056194044b6b66ade50f14a35d6db4cca1a2eeec7867387e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T00:29:55.090 INFO:journalctl@ceph.mds.c.smithi195.stdout:Apr 25 00:29:54 smithi195 bash[70902]: ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-c 2024-04-25T00:29:55.260 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T00:29:55.296 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:29:55.296 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T00:29:55.296 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T00:29:55.296 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.f 2024-04-25T00:29:55.589 INFO:journalctl@ceph.mds.f.smithi195.stdout:Apr 25 00:29:55 smithi195 systemd[1]: Stopping Ceph mds.f for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:29:56.089 INFO:journalctl@ceph.mds.f.smithi195.stdout:Apr 25 00:29:55 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-f[63483]: 2024-04-25T00:29:55.612+0000 7f6f70791700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:29:56.089 INFO:journalctl@ceph.mds.f.smithi195.stdout:Apr 25 00:29:55 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-f[63483]: 2024-04-25T00:29:55.612+0000 7f6f70791700 -1 mds.f *** got signal Terminated *** 2024-04-25T00:29:59.496 INFO:journalctl@ceph.mds.f.smithi195.stdout:Apr 25 00:29:59 smithi195 podman[71036]: 2024-04-25 00:29:59.158073915 +0000 UTC m=+3.648100378 container died ad2dd935d23d8f8ad1d28e84ab2886a99dd941d73148e548ebe233ed3dc35893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-f, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-25T00:29:59.774 INFO:journalctl@ceph.mds.f.smithi195.stdout:Apr 25 00:29:59 smithi195 podman[71036]: 2024-04-25 00:29:59.59616005 +0000 UTC m=+4.086186511 container cleanup ad2dd935d23d8f8ad1d28e84ab2886a99dd941d73148e548ebe233ed3dc35893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T00:29:59.774 INFO:journalctl@ceph.mds.f.smithi195.stdout:Apr 25 00:29:59 smithi195 bash[71036]: ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-f 2024-04-25T00:30:00.017 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T00:30:00.038 INFO:journalctl@ceph.mds.f.smithi195.stdout:Apr 25 00:29:59 smithi195 podman[71051]: 2024-04-25 00:29:59.774512988 +0000 UTC m=+0.616685585 container remove ad2dd935d23d8f8ad1d28e84ab2886a99dd941d73148e548ebe233ed3dc35893 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9) 2024-04-25T00:30:00.038 INFO:journalctl@ceph.mds.f.smithi195.stdout:Apr 25 00:30:00 smithi195 systemd[1]: ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.f.service: Deactivated successfully. 2024-04-25T00:30:00.038 INFO:journalctl@ceph.mds.f.smithi195.stdout:Apr 25 00:30:00 smithi195 systemd[1]: Stopped Ceph mds.f for 79602f58-0297-11ef-bc93-c7b262605968. 2024-04-25T00:30:00.038 INFO:journalctl@ceph.mds.f.smithi195.stdout:Apr 25 00:30:00 smithi195 systemd[1]: ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.f.service: Consumed 2.775s CPU time. 2024-04-25T00:30:00.049 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:30:00.049 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T00:30:00.050 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T00:30:00.050 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.i 2024-04-25T00:30:00.342 INFO:journalctl@ceph.mds.i.smithi195.stdout:Apr 25 00:30:00 smithi195 systemd[1]: Stopping Ceph mds.i for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:30:00.839 INFO:journalctl@ceph.mds.i.smithi195.stdout:Apr 25 00:30:00 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-i[64104]: 2024-04-25T00:30:00.351+0000 7f18f1b66700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:30:00.839 INFO:journalctl@ceph.mds.i.smithi195.stdout:Apr 25 00:30:00 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-i[64104]: 2024-04-25T00:30:00.351+0000 7f18f1b66700 -1 mds.i *** got signal Terminated *** 2024-04-25T00:30:03.485 INFO:journalctl@ceph.mds.i.smithi195.stdout:Apr 25 00:30:03 smithi195 podman[71158]: 2024-04-25 00:30:03.152057849 +0000 UTC m=+2.888669921 container died d7c9c88192ac0cac3d749e83156dc942abd0eb1b47c6c5428d063ea06dd6904c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-i, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:30:03.770 INFO:journalctl@ceph.mds.i.smithi195.stdout:Apr 25 00:30:03 smithi195 podman[71158]: 2024-04-25 00:30:03.569101175 +0000 UTC m=+3.305713244 container cleanup d7c9c88192ac0cac3d749e83156dc942abd0eb1b47c6c5428d063ea06dd6904c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:30:03.770 INFO:journalctl@ceph.mds.i.smithi195.stdout:Apr 25 00:30:03 smithi195 bash[71158]: ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-i 2024-04-25T00:30:03.988 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T00:30:04.019 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:30:04.020 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T00:30:04.020 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T00:30:04.020 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.l 2024-04-25T00:30:04.331 INFO:journalctl@ceph.mds.l.smithi195.stdout:Apr 25 00:30:04 smithi195 systemd[1]: Stopping Ceph mds.l for 79602f58-0297-11ef-bc93-c7b262605968... 2024-04-25T00:30:04.589 INFO:journalctl@ceph.mds.l.smithi195.stdout:Apr 25 00:30:04 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-l[63171]: 2024-04-25T00:30:04.330+0000 7f2f1d398700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:30:04.589 INFO:journalctl@ceph.mds.l.smithi195.stdout:Apr 25 00:30:04 smithi195 ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-l[63171]: 2024-04-25T00:30:04.330+0000 7f2f1d398700 -1 mds.l *** got signal Terminated *** 2024-04-25T00:30:06.765 INFO:journalctl@ceph.mds.l.smithi195.stdout:Apr 25 00:30:06 smithi195 podman[71278]: 2024-04-25 00:30:06.425289206 +0000 UTC m=+2.192693097 container died 261749b799aff3759687abf7b03c21d8cd033e51f57b03adb5afa28b9bb9a6df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-25T00:30:07.058 INFO:journalctl@ceph.mds.l.smithi195.stdout:Apr 25 00:30:06 smithi195 podman[71278]: 2024-04-25 00:30:06.86482628 +0000 UTC m=+2.632230180 container cleanup 261749b799aff3759687abf7b03c21d8cd033e51f57b03adb5afa28b9bb9a6df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9) 2024-04-25T00:30:07.058 INFO:journalctl@ceph.mds.l.smithi195.stdout:Apr 25 00:30:06 smithi195 bash[71278]: ceph-79602f58-0297-11ef-bc93-c7b262605968-mds-l 2024-04-25T00:30:07.285 DEBUG:teuthology.orchestra.run.smithi195:> sudo pkill -f 'journalctl -f -n 0 -u ceph-79602f58-0297-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T00:30:07.316 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:30:07.316 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T00:30:07.316 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 79602f58-0297-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T00:30:07.508 INFO:teuthology.orchestra.run.smithi107.stdout:Deleting cluster with fsid: 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:30:09.899 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 79602f58-0297-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T00:30:10.088 INFO:teuthology.orchestra.run.smithi170.stdout:Deleting cluster with fsid: 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:30:12.514 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 79602f58-0297-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T00:30:12.703 INFO:teuthology.orchestra.run.smithi195.stdout:Deleting cluster with fsid: 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:30:15.006 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:30:15.038 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:30:15.069 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:30:15.095 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T00:30:15.097 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672239/remote/smithi107/crash 2024-04-25T00:30:15.098 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/crash -- . 2024-04-25T00:30:15.124 INFO:teuthology.orchestra.run.smithi107.stderr:tar: /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T00:30:15.125 INFO:teuthology.orchestra.run.smithi107.stderr:tar: Error is not recoverable: exiting now 2024-04-25T00:30:15.126 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672239/remote/smithi170/crash 2024-04-25T00:30:15.127 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/crash -- . 2024-04-25T00:30:15.165 INFO:teuthology.orchestra.run.smithi170.stderr:tar: /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T00:30:15.166 INFO:teuthology.orchestra.run.smithi170.stderr:tar: Error is not recoverable: exiting now 2024-04-25T00:30:15.168 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672239/remote/smithi195/crash 2024-04-25T00:30:15.168 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/crash -- . 2024-04-25T00:30:15.199 INFO:teuthology.orchestra.run.smithi195.stderr:tar: /var/lib/ceph/79602f58-0297-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T00:30:15.199 INFO:teuthology.orchestra.run.smithi195.stderr:tar: Error is not recoverable: exiting now 2024-04-25T00:30:15.200 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T00:30:15.200 DEBUG:teuthology.orchestra.run.smithi107:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-25T00:30:15.237 INFO:tasks.cephadm:Compressing logs... 2024-04-25T00:30:15.238 DEBUG:teuthology.orchestra.run.smithi107:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:30:15.280 DEBUG:teuthology.orchestra.run.smithi170:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:30:15.282 DEBUG:teuthology.orchestra.run.smithi195:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:30:15.304 INFO:teuthology.orchestra.run.smithi107.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.log 2024-04-25T00:30:15.304 INFO:teuthology.orchestra.run.smithi107.stderr:: No such file or directory 2024-04-25T00:30:15.304 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T00:30:15.304 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.g.log/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.log: 2024-04-25T00:30:15.305 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T00:30:15.305 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T00:30:15.305 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T00:30:15.323 INFO:teuthology.orchestra.run.smithi195.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.log 2024-04-25T00:30:15.323 INFO:teuthology.orchestra.run.smithi195.stderr:: No such file or directory 2024-04-25T00:30:15.323 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T00:30:15.323 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.log: --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T00:30:15.323 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T00:30:15.323 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T00:30:15.323 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T00:30:15.324 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T00:30:15.324 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T00:30:15.324 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T00:30:15.324 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mgr.z.log: 88.0% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T00:30:15.324 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T00:30:15.324 INFO:teuthology.orchestra.run.smithi195.stderr: 90.2%/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.i.log: -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T00:30:15.324 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T00:30:15.324 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.0% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T00:30:15.324 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T00:30:15.324 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.11.log: 92.0% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T00:30:15.324 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T00:30:15.326 INFO:teuthology.orchestra.run.smithi170.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.log 2024-04-25T00:30:15.326 INFO:teuthology.orchestra.run.smithi170.stderr:: No such file or directory 2024-04-25T00:30:15.326 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T00:30:15.326 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T00:30:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T00:30:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T00:30:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T00:30:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.b.log: /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T00:30:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T00:30:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T00:30:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.h.log: 90.1% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T00:30:15.327 INFO:teuthology.orchestra.run.smithi170.stderr: 88.0% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T00:30:15.327 INFO:teuthology.orchestra.run.smithi170.stderr: 91.9% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T00:30:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T00:30:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T00:30:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T00:30:15.327 INFO:teuthology.orchestra.run.smithi170.stderr: 84.1% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T00:30:15.327 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T00:30:15.328 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T00:30:15.328 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T00:30:15.328 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T00:30:15.328 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mon.a.log: 88.0% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T00:30:15.328 INFO:teuthology.orchestra.run.smithi107.stderr: 90.0% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T00:30:15.328 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T00:30:15.328 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T00:30:15.328 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.cephadm.log: 94.6% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T00:30:15.328 INFO:teuthology.orchestra.run.smithi107.stderr: 84.2% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T00:30:15.328 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T00:30:15.328 INFO:teuthology.orchestra.run.smithi107.stderr: 94.7% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T00:30:15.328 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T00:30:15.328 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T00:30:15.329 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T00:30:15.329 INFO:teuthology.orchestra.run.smithi107.stderr: 94.6% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T00:30:15.329 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T00:30:15.329 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-25T00:30:15.329 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T00:30:15.329 INFO:teuthology.orchestra.run.smithi107.stderr: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T00:30:15.330 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.log: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-25T00:30:15.330 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T00:30:15.335 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T00:30:15.336 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T00:30:15.337 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T00:30:15.347 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T00:30:15.354 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 94.6% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T00:30:15.359 INFO:teuthology.orchestra.run.smithi195.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T00:30:15.361 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 94.6% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T00:30:15.362 INFO:teuthology.orchestra.run.smithi170.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T00:30:15.366 INFO:teuthology.orchestra.run.smithi195.stderr: 91.0% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T00:30:15.560 INFO:teuthology.orchestra.run.smithi107.stderr: 90.3% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T00:30:15.713 INFO:teuthology.orchestra.run.smithi195.stderr: 93.7% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T00:30:15.777 INFO:teuthology.orchestra.run.smithi195.stderr: 91.9% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T00:30:15.804 INFO:teuthology.orchestra.run.smithi170.stderr: 92.0% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T00:30:16.588 INFO:teuthology.orchestra.run.smithi107.stderr: 90.7% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T00:30:16.703 INFO:teuthology.orchestra.run.smithi195.stderr: 94.8% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T00:30:17.857 INFO:teuthology.orchestra.run.smithi195.stderr: 93.8% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T00:30:20.537 INFO:teuthology.orchestra.run.smithi170.stderr: 95.3% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T00:30:22.754 INFO:teuthology.orchestra.run.smithi195.stderr: 91.7% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T00:30:23.160 INFO:teuthology.orchestra.run.smithi195.stderr: 91.6% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T00:30:23.171 INFO:teuthology.orchestra.run.smithi170.stderr: 92.3% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T00:30:23.645 INFO:teuthology.orchestra.run.smithi107.stderr: 91.8% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T00:30:23.785 INFO:teuthology.orchestra.run.smithi170.stderr: 91.8% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T00:30:24.606 INFO:teuthology.orchestra.run.smithi107.stderr: 91.8% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T00:30:25.158 INFO:teuthology.orchestra.run.smithi195.stderr: 91.7% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T00:30:25.418 INFO:teuthology.orchestra.run.smithi107.stderr: 91.7% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T00:30:26.503 INFO:teuthology.orchestra.run.smithi170.stderr: 91.5% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T00:30:27.071 INFO:teuthology.orchestra.run.smithi170.stderr: 93.3% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T00:30:27.552 INFO:teuthology.orchestra.run.smithi107.stderr: 91.6% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T00:30:27.554 INFO:teuthology.orchestra.run.smithi107.stderr: 2024-04-25T00:30:27.554 INFO:teuthology.orchestra.run.smithi107.stderr:real 0m12.260s 2024-04-25T00:30:27.554 INFO:teuthology.orchestra.run.smithi107.stderr:user 0m40.223s 2024-04-25T00:30:27.554 INFO:teuthology.orchestra.run.smithi107.stderr:sys 0m1.301s 2024-04-25T00:30:27.721 INFO:teuthology.orchestra.run.smithi195.stderr: 91.5% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T00:30:27.723 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-04-25T00:30:27.723 INFO:teuthology.orchestra.run.smithi195.stderr:real 0m12.427s 2024-04-25T00:30:27.723 INFO:teuthology.orchestra.run.smithi195.stderr:user 0m41.004s 2024-04-25T00:30:27.723 INFO:teuthology.orchestra.run.smithi195.stderr:sys 0m1.300s 2024-04-25T00:30:29.980 INFO:teuthology.orchestra.run.smithi170.stderr: 91.9% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T00:31:03.573 INFO:teuthology.orchestra.run.smithi170.stderr: 93.1% -- replaced with /var/log/ceph/79602f58-0297-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T00:31:03.575 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-04-25T00:31:03.575 INFO:teuthology.orchestra.run.smithi170.stderr:real 0m48.278s 2024-04-25T00:31:03.575 INFO:teuthology.orchestra.run.smithi170.stderr:user 1m44.498s 2024-04-25T00:31:03.575 INFO:teuthology.orchestra.run.smithi170.stderr:sys 0m3.127s 2024-04-25T00:31:03.575 INFO:tasks.cephadm:Archiving logs... 2024-04-25T00:31:03.576 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672239/remote/smithi107/log 2024-04-25T00:31:03.577 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:31:10.547 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672239/remote/smithi170/log 2024-04-25T00:31:10.548 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:31:25.089 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672239/remote/smithi195/log 2024-04-25T00:31:25.090 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T00:31:31.848 INFO:tasks.cephadm:Removing cluster... 2024-04-25T00:31:31.848 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 79602f58-0297-11ef-bc93-c7b262605968 --force 2024-04-25T00:31:32.040 INFO:teuthology.orchestra.run.smithi107.stdout:Deleting cluster with fsid: 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:31:32.409 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 79602f58-0297-11ef-bc93-c7b262605968 --force 2024-04-25T00:31:32.600 INFO:teuthology.orchestra.run.smithi170.stdout:Deleting cluster with fsid: 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:31:33.050 DEBUG:teuthology.orchestra.run.smithi195:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 79602f58-0297-11ef-bc93-c7b262605968 --force 2024-04-25T00:31:33.241 INFO:teuthology.orchestra.run.smithi195.stdout:Deleting cluster with fsid: 79602f58-0297-11ef-bc93-c7b262605968 2024-04-25T00:31:33.645 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T00:31:33.646 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T00:31:33.665 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T00:31:33.716 DEBUG:teuthology.orchestra.run.smithi195:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T00:31:33.732 INFO:tasks.cephadm:Teardown complete 2024-04-25T00:31:33.732 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T00:31:33.743 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T00:31:33.743 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:31:33.746 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:31:33.759 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T00:31:33.870 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T00:31:33.881 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T00:31:33.881 DEBUG:teuthology.orchestra.run.smithi107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:31:33.884 DEBUG:teuthology.orchestra.run.smithi170:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:31:33.886 DEBUG:teuthology.orchestra.run.smithi195:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:31:33.898 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:31:33.898 INFO:teuthology.orchestra.run.smithi170.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:31:33.900 INFO:teuthology.orchestra.run.smithi195.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:31:33.903 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:31:33.903 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-04-25T00:31:33.903 INFO:teuthology.orchestra.run.smithi107.stdout:^- hv01.front.sepia.ceph.com 2 6 377 10 +927us[ +933us] +/- 581ms 2024-04-25T00:31:33.903 INFO:teuthology.orchestra.run.smithi107.stdout:^* hv02.front.sepia.ceph.com 3 6 377 8 -15us[-9205ns] +/- 79ms 2024-04-25T00:31:33.903 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 76 +54us[ +73us] +/- 69ms 2024-04-25T00:31:33.903 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:31:33.904 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:31:33.904 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2024-04-25T00:31:33.905 INFO:teuthology.orchestra.run.smithi170.stdout:^- hv01.front.sepia.ceph.com 2 6 377 11 +940us[ +941us] +/- 581ms 2024-04-25T00:31:33.905 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 9 -45us[ -44us] +/- 79ms 2024-04-25T00:31:33.905 INFO:teuthology.orchestra.run.smithi170.stdout:^* hv03.front.sepia.ceph.com 3 6 377 10 +34us[ +34us] +/- 70ms 2024-04-25T00:31:33.905 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:31:33.906 INFO:teuthology.orchestra.run.smithi195.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:31:33.906 INFO:teuthology.orchestra.run.smithi195.stdout:=============================================================================== 2024-04-25T00:31:33.906 INFO:teuthology.orchestra.run.smithi195.stdout:^- hv01.front.sepia.ceph.com 2 6 377 10 +935us[ +935us] +/- 581ms 2024-04-25T00:31:33.906 INFO:teuthology.orchestra.run.smithi195.stdout:^* hv02.front.sepia.ceph.com 3 6 377 11 -39us[ -39us] +/- 79ms 2024-04-25T00:31:33.906 INFO:teuthology.orchestra.run.smithi195.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 +36us[ +36us] +/- 70ms 2024-04-25T00:31:33.906 INFO:teuthology.orchestra.run.smithi195.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:31:33.906 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T00:31:33.919 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T00:31:33.920 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T00:31:33.929 DEBUG:teuthology.orchestra.run.smithi107:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T00:31:34.025 DEBUG:teuthology.orchestra.run.smithi170:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T00:31:34.092 DEBUG:teuthology.orchestra.run.smithi195:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T00:31:34.156 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:31:34.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:31:34.186 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:31:34.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:31:34.213 DEBUG:teuthology.orchestra.run.smithi195:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T00:31:34.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:31:34.239 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T00:31:34.277 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T00:31:34.290 INFO:teuthology.task.internal:Duration was 1954.809973 seconds 2024-04-25T00:31:34.291 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T00:31:34.304 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T00:31:34.304 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:31:34.306 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:31:34.308 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T00:31:34.341 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:31:34.345 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:31:34.345 INFO:teuthology.orchestra.run.smithi195.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T00:31:34.720 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T00:31:34.720 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T00:31:34.720 DEBUG:teuthology.orchestra.run.smithi107:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:31:34.740 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi170.front.sepia.ceph.com 2024-04-25T00:31:34.741 DEBUG:teuthology.orchestra.run.smithi170:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:31:34.781 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi195.front.sepia.ceph.com 2024-04-25T00:31:34.781 DEBUG:teuthology.orchestra.run.smithi195:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T00:31:34.800 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T00:31:34.800 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:31:34.803 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:31:34.823 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T00:31:34.865 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T00:31:34.865 DEBUG:teuthology.orchestra.run.smithi107:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:31:34.868 DEBUG:teuthology.orchestra.run.smithi170:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:31:34.890 DEBUG:teuthology.orchestra.run.smithi195:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T00:31:35.266 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T00:31:35.372 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T00:31:35.372 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:31:35.396 DEBUG:teuthology.orchestra.run.smithi170:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:31:35.421 DEBUG:teuthology.orchestra.run.smithi195:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T00:31:35.445 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T00:31:35.470 DEBUG:teuthology.orchestra.run.smithi107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:31:35.473 DEBUG:teuthology.orchestra.run.smithi170:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:31:35.475 DEBUG:teuthology.orchestra.run.smithi195:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:31:35.492 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = core 2024-04-25T00:31:35.495 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = core 2024-04-25T00:31:35.507 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = core 2024-04-25T00:31:35.531 DEBUG:teuthology.orchestra.run.smithi107:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:31:35.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:31:35.559 DEBUG:teuthology.orchestra.run.smithi170:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:31:35.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:31:35.574 DEBUG:teuthology.orchestra.run.smithi195:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T00:31:35.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:31:35.588 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T00:31:35.604 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T00:31:35.605 DEBUG:teuthology.misc:Transferring archived files from smithi107:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672239/remote/smithi107 2024-04-25T00:31:35.605 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:31:35.645 DEBUG:teuthology.misc:Transferring archived files from smithi170:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672239/remote/smithi170 2024-04-25T00:31:35.645 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:31:35.679 DEBUG:teuthology.misc:Transferring archived files from smithi195:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672239/remote/smithi195 2024-04-25T00:31:35.680 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T00:31:35.720 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T00:31:35.721 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:31:35.723 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:31:35.725 DEBUG:teuthology.orchestra.run.smithi195:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T00:31:35.763 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T00:31:35.778 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T00:31:35.779 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T00:31:35.791 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T00:31:35.791 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:31:35.794 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:31:35.796 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T00:31:35.806 INFO:teuthology.orchestra.run.smithi107.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:31 /home/ubuntu/cephtest 2024-04-25T00:31:35.808 INFO:teuthology.orchestra.run.smithi170.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:31 /home/ubuntu/cephtest 2024-04-25T00:31:35.817 INFO:teuthology.orchestra.run.smithi195.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 00:31 /home/ubuntu/cephtest 2024-04-25T00:31:35.818 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T00:31:35.831 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T00:31:35.865 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/fsstress}} duration: 1954.809972524643 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-25T00:31:35.865 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T00:31:35.951 INFO:teuthology.run:pass